library: scraper options — per-source + per-field auto-apply, review-queue order (R1) (#726)

* library: scraper options — per-source + per-field auto-apply toggles, review-queue order (R1)

Grows the Settings→Library "Metadata matching" card into the full
scraper-options panel — not everybody needs the same things out of a
scraper:

- Sources: enrich_src_musicbrainz gates the background matcher (phase 2;
  identity hashes still stamp, and manual Fix-match/search stays
  available — same contract as the master toggle). enrich_src_caa gates
  the Cover Art Archive fetch (phase 3). Rows skipped by an off toggle
  stay unevaluated, so re-enabling picks them up on the next pass —
  nothing is permanently forfeited.
- Auto-apply fields: enrich_apply_names/year/genres filter what an
  AUTOMATIC match may canonicalize (_enrich_field_filter, applied on all
  three automatic paths: cache copy, mbid/isrc exact keys, text auto).
  MusicBrainz ids always stamp — they're identity, not display; the art
  fetch and future re-matching need them. A match the user confirms in
  the review modal applies in full. enrich_apply_art gates the art fetch
  alongside the CAA source toggle (two axes, one behaviour today —
  future art sources slot in without re-teaching the panel).
- Review queue order: enrich_review_order = missing_first (default,
  today's behaviour) | artist | recent, read by GET /api/enrichment/review;
  unknown stored values degrade to the default.
- Settings card: Sources / Auto-apply / Review-queue-order groups wired
  in match-review.js; the master toggle is relabelled "Match songs
  automatically" so it doesn't read the same as the new MusicBrainz
  source toggle. No tailwind rebuild needed — every class was already
  scanned from core source.

Tests: tests/test_scraper_options.py (9) — settings validation,
MB-source-off stamps-without-matching + re-enable, per-field stripping
on auto matches with ids preserved, review-accept full-apply despite
toggles, CAA gating on both axes, review-order modes incl. the
unknown-value fallback. Full-suite failure set A/B-identical to the
base (39 env/pre-existing).

Stacked on feat/enrichment-art (#715) — merge that first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nm7tHs1Yvjjtnnu4nzJgdN

* library: per-field auto-apply honours "nothing forfeited" (backfill + no partial seeding)

The R1 per-FIELD auto-apply toggles settled a `matched` row with the
disabled fields stripped, but enrichment_pending() never revisits an
unchanged-hash matched row — so re-enabling a field never backfilled it,
and enrichment_cache_lookup() (which gates only on mb_recording_id) could
seed sibling charts with the stripped blanks. This broke the same
"nothing is permanently forfeited" contract the source (mb_on) and art
(art_on) toggles already keep.

Fix: persist an `apply_mask` marker (sorted blocked apply-keys) on every
AUTOMATIC match:
- migration: additive `apply_mask TEXT` column (idempotent ALTER).
- enrichment_pending(allowed_keys=...): re-queues a `matched` row whose
  apply_mask names a field that is now re-enabled → backfill on re-enable,
  converges (a fully-applied row is never re-queued).
- enrichment_cache_lookup: only fully-applied donors (apply_mask empty/NULL)
  may seed siblings; a partial row is skipped and the sibling falls through
  to its own re-filtered match.
- _enrich_apply_mask()/_enrich_blocked_apply_keys() helpers; threaded through
  _enrich_one → apply_enrichment_match. Review/manual writers leave it NULL
  (a confirmed pick applies in full).

Tests: re-enable-backfills-and-converges; partial row is not a cache donor
(fully-applied one is). 13 scraper-options tests pass; 170 enrichment/
settings tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ChrisBeWithYou <christian.a.cowan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Byron Gamatos
2026-07-02 20:57:42 +02:00
committed by GitHub
co-authored by Claude Opus 4.8 ChrisBeWithYou
parent 7ca736d525
commit f5c9c34291
4 changed files with 579 additions and 49 deletions
+29 -1
View File
@@ -742,7 +742,7 @@
<div class="fb-srow-desc">Matches your charts against MusicBrainz in the background to tidy names, years and genres — display only, your files are never modified. Matches at or above the confidence level apply automatically; the rest wait in the library's review queue. Turning this off never disables manual match fixes.</div>
</div>
<div class="grid grid-cols-2 gap-2 mb-1 text-xs text-gray-400 fb-srow-wide">
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-enabled" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Match songs against MusicBrainz</label>
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-enabled" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Match songs automatically</label>
<label class="flex items-center gap-2">Auto-apply confidence
<select id="enrich-threshold" class="bg-dark-700 border border-gray-800 rounded-xl px-2 py-1.5 text-xs text-gray-300 outline-none">
<option value="0.85">85% — more auto-matches</option>
@@ -752,6 +752,34 @@
</select>
</label>
</div>
<!-- R1 scraper options: sources (who may be contacted) ×
auto-apply fields (what a confident match fills in). -->
<div class="fb-srow-wide mb-1">
<div class="text-[10px] uppercase tracking-wide text-gray-500 mb-1">Sources</div>
<div class="grid grid-cols-2 gap-2 text-xs text-gray-400">
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-src-musicbrainz" checked class="rounded border-gray-600 bg-dark-700 text-accent"> MusicBrainz — names, years, genres</label>
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-src-caa" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Cover Art Archive — album covers</label>
</div>
</div>
<div class="fb-srow-wide mb-1">
<div class="text-[10px] uppercase tracking-wide text-gray-500 mb-1">Auto-apply</div>
<div class="grid grid-cols-4 gap-2 text-xs text-gray-400">
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-apply-names" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Names</label>
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-apply-year" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Year</label>
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-apply-genres" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Genres</label>
<label class="flex items-center gap-2"><input type="checkbox" id="enrich-apply-art" checked class="rounded border-gray-600 bg-dark-700 text-accent"> Cover art</label>
</div>
<div class="text-[11px] text-gray-600 mt-1">What a confident match may fill in on its own — matches you confirm in the review queue always apply in full.</div>
</div>
<div class="grid grid-cols-2 gap-2 mb-1 text-xs text-gray-400 fb-srow-wide">
<label class="flex items-center gap-2">Review queue order
<select id="enrich-review-order" class="bg-dark-700 border border-gray-800 rounded-xl px-2 py-1.5 text-xs text-gray-300 outline-none">
<option value="missing_first" selected>Missing info first</option>
<option value="artist">Artist AZ</option>
<option value="recent">Recently added</option>
</select>
</label>
</div>
<div class="fb-srow-control">
<button id="enrich-match-now" class="bg-dark-600 hover:bg-dark-500 px-5 py-2.5 rounded-xl text-sm text-gray-300 transition">Match Now</button>
<span id="enrich-status" class="text-xs text-gray-500"></span>
+23 -4
View File
@@ -401,16 +401,28 @@
// Markup lives statically in index.html (the v3 settings pattern); this
// wires it. All null-guarded so v2 (which lacks the elements) no-ops.
function wireSettingsCard() {
const toggle = document.getElementById('enrich-enabled');
const sel = document.getElementById('enrich-threshold');
const order = document.getElementById('enrich-review-order');
const btn = document.getElementById('enrich-match-now');
if (!toggle && !sel && !btn) return;
// Boolean toggles, element id → settings key. enrich-enabled is the
// master background switch; the rest are the R1 scraper options
// (per-source + per-field auto-apply).
const toggles = [
['enrich-enabled', 'enrich_enabled'],
['enrich-src-musicbrainz', 'enrich_src_musicbrainz'],
['enrich-src-caa', 'enrich_src_caa'],
['enrich-apply-names', 'enrich_apply_names'],
['enrich-apply-year', 'enrich_apply_year'],
['enrich-apply-genres', 'enrich_apply_genres'],
['enrich-apply-art', 'enrich_apply_art'],
].map(([id, key]) => [document.getElementById(id), key]).filter(([el]) => el);
if (!toggles.length && !sel && !btn) return;
(async () => {
try {
const r = await fetch('/api/settings');
if (r.ok) {
const cfg = await r.json();
if (toggle) toggle.checked = cfg.enrich_enabled !== false;
for (const [el, key] of toggles) el.checked = cfg[key] !== false;
if (sel) {
const t = Number(cfg.enrich_auto_threshold);
const want = Number.isFinite(t) ? t : 0.9;
@@ -421,13 +433,20 @@
}
if (best) sel.value = best.value;
}
if (order) {
const v = String(cfg.enrich_review_order || 'missing_first');
order.value = ['missing_first', 'artist', 'recent'].includes(v) ? v : 'missing_first';
}
}
} catch (_) { /* leave markup defaults */ }
refreshChip(); // also fills #enrich-status
})();
const save = (key, value) => post('/api/settings', { [key]: value });
toggle?.addEventListener('change', () => save('enrich_enabled', !!toggle.checked));
for (const [el, key] of toggles) {
el.addEventListener('change', () => save(key, !!el.checked));
}
sel?.addEventListener('change', () => save('enrich_auto_threshold', Number(sel.value)));
order?.addEventListener('change', () => save('enrich_review_order', order.value));
btn?.addEventListener('click', async () => {
await post('/api/enrichment/kick');
const line = document.getElementById('enrich-status');