refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3) (#861)

* refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3)

MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and
the background enrichment worker (~930 lines, 61 defs) leave server.py as one
cohesive unit. Bodies are verbatim; the only changes are seam reads:

  meta_db / config_dir / sloppak_cache_dir / art_cache_dir -> appstate.<slot>
  _song_pack_art_exists / _art_override_paths (stay in server.py for the art +
    delete routes) -> appstate.<callable> (new seam slots, injected by reference)
  _env_flag -> env_compat.env_flag_compat (the existing identical helper)
  _artist_title_from_filename -> imported from metadata_db (its home)
  the User-Agent VERSION lookup: Path(__file__).parent ->
    Path(__file__).resolve().parents[1] (lib/enrichment.py -> app root)

server.py drives the worker through the module (import enrichment; the routes +
scan lifecycle call enrichment.X). Tests that faked the network on `server`
(_mb_http_get, _enrich_network_enabled, _caa_http_get, ...) now patch the same
names on `enrichment` — the module attribute is resolved at call time, so one
setattr reaches both the routes and the worker's internal callers. Acyclic:
enrichment imports appstate/appconfig/dlc_paths/metadata_db/mb_match/
acoustid_match/sloppak/loosefolder, never server.

server.py: 6,917 -> 5,988 (-929).

Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2400 passed (140 enrichment/art cases
incl the offline-safety + transport-error-pauses-pass contracts that fake the
network); test_packaging 44 passed (enrichment.py resolves under lib/); eslint 0.
Boot smoke: /api/enrichment/status + POST /kick serve from the new module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: reset enrichment worker state between tests (CodeRabbit)

lib/enrichment.py now owns the worker, and it stays imported for the whole
session while the `server` fixtures pop-and-reimport `server` — so the cancel
Event / status dict / caches would leak across tests, and a stale `_enrich_cancel`
could short-circuit a later direct `_background_enrich()`. An autouse conftest
fixture clears that process-global state before each test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: tighten the enrichment-reset fixture (CodeRabbit)

Narrow the import guard to ImportError (not blind Exception, BLE001), and stop
clearing _caa_index_locks — it's guarded by _caa_index_locks_guard, so an
unlocked clear() would race a still-alive worker, and its per-release mutexes
carry no test state.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Byron Gamatos
2026-07-11 11:32:21 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 508829c012
commit 165475d115
14 changed files with 1346 additions and 1126 deletions
+10 -9
View File
@@ -12,6 +12,7 @@ tests/test_art_layer.py.
"""
import importlib
import enrichment
import io as _io
import sys
@@ -119,8 +120,8 @@ def caa_index(server, monkeypatch):
calls.append(release_id)
return indexes.get(release_id) # unknown release → None (a CAA 404)
fake.calls, fake.indexes = calls, indexes
monkeypatch.setattr(server, "_caa_release_index", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_caa_release_index", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
@@ -274,10 +275,10 @@ def test_malicious_release_id_rejected_no_fetch_no_write(server, caa_index):
"""A crafted release id (path traversal) never matches _CAA_ID_RE, so it
yields no images, opens no socket, and writes no cache file — inside the
art dir or anywhere else."""
art_dir = server._enrichment_art_dir()
art_dir = enrichment._enrichment_art_dir()
before = set(art_dir.glob("*"))
assert not server._CAA_ID_RE.match("../../etc/x")
assert server._caa_index_cached("../../etc/x") == []
assert not enrichment._CAA_ID_RE.match("../../etc/x")
assert enrichment._caa_index_cached("../../etc/x") == []
assert caa_index.calls == [] # the seam was never asked
assert set(art_dir.glob("*")) == before # nothing written
# And nothing landed at the traversal target beside the cache dir either.
@@ -340,7 +341,7 @@ def test_fetch_art_url_follows_redirects_validating_each_hop(server, monkeypatch
return _FakeResp(200, chunks=[b"IMGDATA"])
monkeypatch.setattr(requests, "get", fake_get)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_url_host_is_internal",
lambda u: (checked.append(u), False)[1])
data = server._fetch_art_url("https://coverartarchive.example/release/x/front-500")
@@ -354,7 +355,7 @@ def test_fetch_art_url_blocks_redirect_to_internal(server, monkeypatch):
import requests
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
302, {"Location": "http://internal.example/x.png"}))
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_url_host_is_internal",
lambda u: "internal" in u)
with pytest.raises(ValueError):
@@ -365,7 +366,7 @@ def test_fetch_art_url_redirect_budget(server, monkeypatch):
import requests
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
307, {"Location": "https://public.example/next.png"}))
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_url_host_is_internal", lambda u: False)
with pytest.raises(server.EnrichTransportError):
with pytest.raises(enrichment.EnrichTransportError):
server._fetch_art_url("https://public.example/x.png")