* refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3)
MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and
the background enrichment worker (~930 lines, 61 defs) leave server.py as one
cohesive unit. Bodies are verbatim; the only changes are seam reads:
meta_db / config_dir / sloppak_cache_dir / art_cache_dir -> appstate.<slot>
_song_pack_art_exists / _art_override_paths (stay in server.py for the art +
delete routes) -> appstate.<callable> (new seam slots, injected by reference)
_env_flag -> env_compat.env_flag_compat (the existing identical helper)
_artist_title_from_filename -> imported from metadata_db (its home)
the User-Agent VERSION lookup: Path(__file__).parent ->
Path(__file__).resolve().parents[1] (lib/enrichment.py -> app root)
server.py drives the worker through the module (import enrichment; the routes +
scan lifecycle call enrichment.X). Tests that faked the network on `server`
(_mb_http_get, _enrich_network_enabled, _caa_http_get, ...) now patch the same
names on `enrichment` — the module attribute is resolved at call time, so one
setattr reaches both the routes and the worker's internal callers. Acyclic:
enrichment imports appstate/appconfig/dlc_paths/metadata_db/mb_match/
acoustid_match/sloppak/loosefolder, never server.
server.py: 6,917 -> 5,988 (-929).
Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2400 passed (140 enrichment/art cases
incl the offline-safety + transport-error-pauses-pass contracts that fake the
network); test_packaging 44 passed (enrichment.py resolves under lib/); eslint 0.
Boot smoke: /api/enrichment/status + POST /kick serve from the new module.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: reset enrichment worker state between tests (CodeRabbit)
lib/enrichment.py now owns the worker, and it stays imported for the whole
session while the `server` fixtures pop-and-reimport `server` — so the cancel
Event / status dict / caches would leak across tests, and a stale `_enrich_cancel`
could short-circuit a later direct `_background_enrich()`. An autouse conftest
fixture clears that process-global state before each test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: tighten the enrichment-reset fixture (CodeRabbit)
Narrow the import guard to ImportError (not blind Exception, BLE001), and stop
clearing _caa_index_locks — it's guarded by _caa_index_locks_guard, so an
unlocked clear() would race a still-alive worker, and its per-release mutexes
carry no test state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a media-server-style "Refresh Metadata" control to the Songs toolbar
(beside "⟳ Refresh") — the metadata counterpart to a file scan.
- Re-matches the songs currently SHOWN (the visible grid window) against
MusicBrainz: a per-view refresh that's visible even on an already-matched
library. The button doubles as Stop while a pass runs; a batch progress bar
+ per-tile queued→working→done badges show what's happening. User-pinned
`manual` matches are never re-matched.
- Backend: POST /api/enrichment/{cancel,states,rematch}; /status gains
total/matched/current/cancelling; a cooperative cancel Event is checked
between songs in the match + art phases so Stop halts without waiting for
the whole queue. `states` is read-only (open); `cancel`/`rematch` are
demo-blocked.
- Matcher: when a pack's `artist` field is blank (common in community
charts), derive artist/title from the CDLC `Artist_Song-Title` filename
convention as a SEARCH SEED so text matching can identify it — the displayed
values still come only from the confirmed MusicBrainz match, nothing
estimated is shown as author-set. Rescues blank-artist packs that otherwise
always failed.
Tests: enrichment_states_for, the three new routes, cancel-halts-a-pass,
kick-clears-stale-cancel, filename parse, blank-artist seeding, and
present-artist-not-overridden.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the flaky pytest segfault (exit 139): the background scan and enrichment daemon threads (_scan_runner/_enrich_runner) use the shared MetadataDB connection, but test fixtures closed that connection in teardown without stopping them. A daemon thread mid-query on a freed SQLite conn is a native use-after-free → SIGSEGV. The app's startup kicks a scan, so almost any app-booting fixture was vulnerable. It only surfaced now because got-feedback/feedBack#728 added a push trigger, so ci/test runs on every push to main.
Fix: server.py retains the scan/enrich thread handles and adds _join_background_db_threads(); every test fixture now joins the workers before conn.close(). Verified: the full suite runs to completion (no segfault) where it previously crashed at ~25%.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* library: metadata-enrichment plumbing (cache table + worker lifecycle) — P7
The pipeline around a deliberately NO-OP matcher, so the real
MusicBrainz text matcher (next slice) replaces exactly one function and
inherits the queue, throttle, lifecycle, and safety contracts:
- song_enrichment cache table: one row per song holding the match
lifecycle (unscanned -> matched(source,score) | manual | failed) plus
the canonical values a confident match supplies. A DISPLAY cache -
canonical values are never auto-written into pack files. Never purged
on rescan (only by the explicit per-song delete); dead rows filtered
at read time; re-derivable, so a lost row just re-enriches.
- Identity hashing: sha1 of normalized artist|title|album|duration.
Filename-free, so a renamed pack keeps its enrichment; unchanged hash
makes re-enrichment a no-op (idempotent).
- Queue rules (test-pinned): no row / unscanned / identity-changed ->
re-match; matched + current hash = settled; a MANUAL row is the
user's pinned pick and is never auto-reset (state and hash both
survive metadata edits); failed waits for the matcher's backoff
policy (attempts column ready).
- Worker: _kick_enrich/_enrich_runner mirror the scan's single-flight +
coalescing pattern, kicked when a scan pass fully completes (the scan
pool is a no-network process pool by design; the 5-minute periodic
rescan is the natural retry hook). One bounded pass per kick - no
drain-loop, since the no-op matcher legitimately leaves rows
unscanned. _enrich_throttle() is the <=1 req/s seam every matcher
must call before a network request, and the never-hold-meta_db._lock-
across-a-fetch rule is documented at the seam.
- CONFIG_DIR/art_cache dir helper (the cover-art slice adds the LRU
cap) + GET /api/enrichment/status (worker flags + counts by state).
8 new tests; full-suite failure set identical to unmodified main.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nm7tHs1Yvjjtnnu4nzJgdN
* fix(library): lock enrichment reads on shared conn + skip redundant stub writes (PR #707 review)
Issue 1: wrap the SELECT+fetch in enrichment_pending, get_enrichment and
enrichment_state_counts in self._lock so request-thread reads no longer
interleave with the worker's execute+commit on the shared connection.
Issue 2: guard upsert_enrichment_stub so an already-settled row (manual
pick, or a non-manual row whose content_hash already matches) skips the
UPDATE/commit — stops the no-op matcher re-writing every song each pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: byrongamatos <xasiklas@gmail.com>
