mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-08-11 11:19:24 +00:00
refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3) (#861)
* refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3)
MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and
the background enrichment worker (~930 lines, 61 defs) leave server.py as one
cohesive unit. Bodies are verbatim; the only changes are seam reads:
meta_db / config_dir / sloppak_cache_dir / art_cache_dir -> appstate.<slot>
_song_pack_art_exists / _art_override_paths (stay in server.py for the art +
delete routes) -> appstate.<callable> (new seam slots, injected by reference)
_env_flag -> env_compat.env_flag_compat (the existing identical helper)
_artist_title_from_filename -> imported from metadata_db (its home)
the User-Agent VERSION lookup: Path(__file__).parent ->
Path(__file__).resolve().parents[1] (lib/enrichment.py -> app root)
server.py drives the worker through the module (import enrichment; the routes +
scan lifecycle call enrichment.X). Tests that faked the network on `server`
(_mb_http_get, _enrich_network_enabled, _caa_http_get, ...) now patch the same
names on `enrichment` — the module attribute is resolved at call time, so one
setattr reaches both the routes and the worker's internal callers. Acyclic:
enrichment imports appstate/appconfig/dlc_paths/metadata_db/mb_match/
acoustid_match/sloppak/loosefolder, never server.
server.py: 6,917 -> 5,988 (-929).
Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2400 passed (140 enrichment/art cases
incl the offline-safety + transport-error-pauses-pass contracts that fake the
network); test_packaging 44 passed (enrichment.py resolves under lib/); eslint 0.
Boot smoke: /api/enrichment/status + POST /kick serve from the new module.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: reset enrichment worker state between tests (CodeRabbit)
lib/enrichment.py now owns the worker, and it stays imported for the whole
session while the `server` fixtures pop-and-reimport `server` — so the cancel
Event / status dict / caches would leak across tests, and a stale `_enrich_cancel`
could short-circuit a later direct `_background_enrich()`. An autouse conftest
fixture clears that process-global state before each test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: tighten the enrichment-reset fixture (CodeRabbit)
Narrow the import guard to ImportError (not blind Exception, BLE001), and stop
clearing _caa_index_locks — it's guarded by _caa_index_locks_guard, so an
unlocked clear() would race a still-alive worker, and its per-release mutexes
carry no test state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
508829c012
commit
165475d115
@@ -40,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
the second slot. The `_demo_mode_guard` middleware still blocks all four moved write
|
the second slot. The `_demo_mode_guard` middleware still blocks all four moved write
|
||||||
routes with 403, and `Query(...)` validation still 422s — both checked against a running
|
routes with 403, and `Query(...)` validation still 422s — both checked against a running
|
||||||
server. `server.py`: **9,445 → 9,386 lines**.
|
server. `server.py`: **9,445 → 9,386 lines**.
|
||||||
|
- **`lib/enrichment.py` — the metadata-enrichment subsystem leaves `server.py` (R3, move-only).** MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and the background enrichment worker (~930 lines, 61 defs) move out as one cohesive unit. Bodies are verbatim; the only changes are seam reads — `meta_db`/`config_dir`/`sloppak_cache_dir`/`art_cache_dir` and the two shared art helpers (`song_pack_art_exists`, `art_override_paths`, which stay in `server.py` for the art/delete routes) are reached through `appstate` at call time, and the User-Agent VERSION lookup is corrected for the module's new depth. `server.py` drives the worker through the module (`import enrichment`; the routes + scan lifecycle call `enrichment.X`); tests that faked the network on `server` now patch the same names on `enrichment` (module attribute resolved at call time, so one `setattr` reaches both the routes and the worker's internal callers). Acyclic — `enrichment` imports no `server`. Route table byte-identical; full suite green. `server.py`: 6,917 → 5,988.
|
||||||
- **`appstate.py` — the router seam (R3).** Route modules moving out of `server.py`
|
- **`appstate.py` — the router seam (R3).** Route modules moving out of `server.py`
|
||||||
need `meta_db` and friends but must not `import server`, or the import graph goes
|
need `meta_db` and friends but must not `import server`, or the import graph goes
|
||||||
circular the moment `server` imports them back. So `server.py` keeps *constructing*
|
circular the moment `server` imports them back. So `server.py` keeps *constructing*
|
||||||
|
|||||||
@@ -55,8 +55,8 @@ without a *signed* exemption" is unenforceable.
|
|||||||
## Planned, NOT exempt (owned by split plans — listed so nothing falls between states)
|
## Planned, NOT exempt (owned by split plans — listed so nothing falls between states)
|
||||||
|
|
||||||
core `static/app.js` (11,852) · `static/highway.js` (4,168, whole file) · `server.py`
|
core `static/app.js` (11,852) · `static/highway.js` (4,168, whole file) · `server.py`
|
||||||
(6,917 — was 14,037; ratcheted by the R3 `MetadataDB` + `AudioEffectsMappingDB`
|
(5,988 — was 14,037; ratcheted by the R3 `MetadataDB` + `AudioEffectsMappingDB`
|
||||||
extractions and fifteen `routers/` modules) ·
|
extractions and fifteen `routers/` modules; the ~930-line metadata-enrichment subsystem — MB/CAA/AcoustID transport, matcher, background worker — now lives in `lib/enrichment.py`) ·
|
||||||
`lib/metadata_db.py` (4,373 — new in R3; the `MetadataDB` class alone is 4,018 lines
|
`lib/metadata_db.py` (4,373 — new in R3; the `MetadataDB` class alone is 4,018 lines
|
||||||
and is a monolith in its own right, to be split per-table once the router train
|
and is a monolith in its own right, to be split per-table once the router train
|
||||||
lands) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
|
lands) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
|
||||||
|
|||||||
@@ -92,6 +92,12 @@ audio_cache_dir = None
|
|||||||
get_progression_content = None
|
get_progression_content = None
|
||||||
builtin_diagnostic_filename = None
|
builtin_diagnostic_filename = None
|
||||||
running_version = None
|
running_version = None
|
||||||
|
# Art helpers that stay in server.py (shared with the art/delete routes) but are
|
||||||
|
# also called by the enrichment worker in lib/enrichment.py — injected as
|
||||||
|
# callables to keep enrichment acyclic. art_cache_dir is server's ART_CACHE_DIR.
|
||||||
|
art_cache_dir = None
|
||||||
|
song_pack_art_exists = None
|
||||||
|
art_override_paths = None
|
||||||
|
|
||||||
_SLOTS = frozenset({
|
_SLOTS = frozenset({
|
||||||
"meta_db", "audio_effect_mappings", "tuning_providers",
|
"meta_db", "audio_effect_mappings", "tuning_providers",
|
||||||
@@ -99,6 +105,7 @@ _SLOTS = frozenset({
|
|||||||
"static_dir", "sloppak_cache_dir", "audio_cache_dir",
|
"static_dir", "sloppak_cache_dir", "audio_cache_dir",
|
||||||
"get_progression_content", "builtin_diagnostic_filename",
|
"get_progression_content", "builtin_diagnostic_filename",
|
||||||
"running_version",
|
"running_version",
|
||||||
|
"art_cache_dir", "song_pack_art_exists", "art_override_paths",
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+1103
File diff suppressed because it is too large
Load Diff
@@ -9,6 +9,36 @@ import structlog
|
|||||||
_LOGGING_NAMES = ("feedBack", "uvicorn", "uvicorn.error", "uvicorn.access")
|
_LOGGING_NAMES = ("feedBack", "uvicorn", "uvicorn.error", "uvicorn.access")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_enrichment_state():
|
||||||
|
"""Reset the enrichment worker's process-global state between tests.
|
||||||
|
|
||||||
|
The `server` fixtures pop-and-reimport `server`, but `lib/enrichment.py`
|
||||||
|
(which now owns the worker) stays imported for the whole session, so its
|
||||||
|
module globals — the cancel Event, the status dict, the caches — would
|
||||||
|
otherwise leak across tests. A test that set `_enrich_cancel` (or a stale
|
||||||
|
`running` status) could silently short-circuit a later direct
|
||||||
|
`_background_enrich()` call. Clear it up front so each test starts clean.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import enrichment
|
||||||
|
except ImportError:
|
||||||
|
yield
|
||||||
|
return
|
||||||
|
enrichment._enrich_cancel.clear()
|
||||||
|
enrichment._enrich_pending_pass = False
|
||||||
|
enrichment._enrich_status.update(
|
||||||
|
{"running": False, "processed": 0, "last_pass_at": None,
|
||||||
|
"total": 0, "matched": 0, "current": None})
|
||||||
|
enrichment._enrich_last_fetch = 0.0
|
||||||
|
enrichment._artist_alias_cache.clear()
|
||||||
|
# _caa_index_locks is deliberately left alone: it's guarded by
|
||||||
|
# _caa_index_locks_guard, so clearing it here (unlocked) would race a
|
||||||
|
# still-alive worker thread, and its entries are stateless per-release
|
||||||
|
# mutexes that don't leak test state anyway.
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def isolate_logging():
|
def isolate_logging():
|
||||||
"""Restore feedBack / uvicorn logger state after each test.
|
"""Restore feedBack / uvicorn logger state after each test.
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ tests/test_art_layer.py.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import io as _io
|
import io as _io
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
@@ -119,8 +120,8 @@ def caa_index(server, monkeypatch):
|
|||||||
calls.append(release_id)
|
calls.append(release_id)
|
||||||
return indexes.get(release_id) # unknown release → None (a CAA 404)
|
return indexes.get(release_id) # unknown release → None (a CAA 404)
|
||||||
fake.calls, fake.indexes = calls, indexes
|
fake.calls, fake.indexes = calls, indexes
|
||||||
monkeypatch.setattr(server, "_caa_release_index", fake)
|
monkeypatch.setattr(enrichment, "_caa_release_index", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
@@ -274,10 +275,10 @@ def test_malicious_release_id_rejected_no_fetch_no_write(server, caa_index):
|
|||||||
"""A crafted release id (path traversal) never matches _CAA_ID_RE, so it
|
"""A crafted release id (path traversal) never matches _CAA_ID_RE, so it
|
||||||
yields no images, opens no socket, and writes no cache file — inside the
|
yields no images, opens no socket, and writes no cache file — inside the
|
||||||
art dir or anywhere else."""
|
art dir or anywhere else."""
|
||||||
art_dir = server._enrichment_art_dir()
|
art_dir = enrichment._enrichment_art_dir()
|
||||||
before = set(art_dir.glob("*"))
|
before = set(art_dir.glob("*"))
|
||||||
assert not server._CAA_ID_RE.match("../../etc/x")
|
assert not enrichment._CAA_ID_RE.match("../../etc/x")
|
||||||
assert server._caa_index_cached("../../etc/x") == []
|
assert enrichment._caa_index_cached("../../etc/x") == []
|
||||||
assert caa_index.calls == [] # the seam was never asked
|
assert caa_index.calls == [] # the seam was never asked
|
||||||
assert set(art_dir.glob("*")) == before # nothing written
|
assert set(art_dir.glob("*")) == before # nothing written
|
||||||
# And nothing landed at the traversal target beside the cache dir either.
|
# And nothing landed at the traversal target beside the cache dir either.
|
||||||
@@ -340,7 +341,7 @@ def test_fetch_art_url_follows_redirects_validating_each_hop(server, monkeypatch
|
|||||||
return _FakeResp(200, chunks=[b"IMGDATA"])
|
return _FakeResp(200, chunks=[b"IMGDATA"])
|
||||||
|
|
||||||
monkeypatch.setattr(requests, "get", fake_get)
|
monkeypatch.setattr(requests, "get", fake_get)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
monkeypatch.setattr(server, "_url_host_is_internal",
|
monkeypatch.setattr(server, "_url_host_is_internal",
|
||||||
lambda u: (checked.append(u), False)[1])
|
lambda u: (checked.append(u), False)[1])
|
||||||
data = server._fetch_art_url("https://coverartarchive.example/release/x/front-500")
|
data = server._fetch_art_url("https://coverartarchive.example/release/x/front-500")
|
||||||
@@ -354,7 +355,7 @@ def test_fetch_art_url_blocks_redirect_to_internal(server, monkeypatch):
|
|||||||
import requests
|
import requests
|
||||||
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
|
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
|
||||||
302, {"Location": "http://internal.example/x.png"}))
|
302, {"Location": "http://internal.example/x.png"}))
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
monkeypatch.setattr(server, "_url_host_is_internal",
|
monkeypatch.setattr(server, "_url_host_is_internal",
|
||||||
lambda u: "internal" in u)
|
lambda u: "internal" in u)
|
||||||
with pytest.raises(ValueError):
|
with pytest.raises(ValueError):
|
||||||
@@ -365,7 +366,7 @@ def test_fetch_art_url_redirect_budget(server, monkeypatch):
|
|||||||
import requests
|
import requests
|
||||||
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
|
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
|
||||||
307, {"Location": "https://public.example/next.png"}))
|
307, {"Location": "https://public.example/next.png"}))
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
monkeypatch.setattr(server, "_url_host_is_internal", lambda u: False)
|
monkeypatch.setattr(server, "_url_host_is_internal", lambda u: False)
|
||||||
with pytest.raises(server.EnrichTransportError):
|
with pytest.raises(enrichment.EnrichTransportError):
|
||||||
server._fetch_art_url("https://public.example/x.png")
|
server._fetch_art_url("https://public.example/x.png")
|
||||||
|
|||||||
+19
-18
@@ -7,6 +7,7 @@ here opens a socket, and the offline default is itself asserted.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import io as _io
|
import io as _io
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
@@ -176,15 +177,15 @@ def caa(server, monkeypatch):
|
|||||||
calls.append(release_id)
|
calls.append(release_id)
|
||||||
return art.get(release_id)
|
return art.get(release_id)
|
||||||
fake.calls, fake.art = calls, art
|
fake.calls, fake.art = calls, art
|
||||||
monkeypatch.setattr(server, "_caa_http_get", fake)
|
monkeypatch.setattr(enrichment, "_caa_http_get", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
def test_caa_fetch_fills_missing_art(server, client, caa):
|
def test_caa_fetch_fills_missing_art(server, client, caa):
|
||||||
make_sloppak(server, "a.sloppak") # no pack art
|
make_sloppak(server, "a.sloppak") # no pack art
|
||||||
_match_row(server, "a.sloppak")
|
_match_row(server, "a.sloppak")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["art_state"] == "caa"
|
assert row["art_state"] == "caa"
|
||||||
assert row["art_cache_path"] and row["art_cache_path"].endswith("caa_rel-1.jpg")
|
assert row["art_cache_path"] and row["art_cache_path"].endswith("caa_rel-1.jpg")
|
||||||
@@ -193,7 +194,7 @@ def test_caa_fetch_fills_missing_art(server, client, caa):
|
|||||||
assert r.headers["content-type"] == "image/jpeg"
|
assert r.headers["content-type"] == "image/jpeg"
|
||||||
# Settled: the next pass never re-fetches.
|
# Settled: the next pass never re-fetches.
|
||||||
n = len(caa.calls)
|
n = len(caa.calls)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(caa.calls) == n
|
assert len(caa.calls) == n
|
||||||
|
|
||||||
|
|
||||||
@@ -204,7 +205,7 @@ def test_caa_skips_pack_art_and_dedupes_by_release(server, caa):
|
|||||||
_match_row(server, "haspack.sloppak")
|
_match_row(server, "haspack.sloppak")
|
||||||
_match_row(server, "b.sloppak") # same release as c
|
_match_row(server, "b.sloppak") # same release as c
|
||||||
_match_row(server, "c.sloppak")
|
_match_row(server, "c.sloppak")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("haspack.sloppak")["art_state"] == "pack"
|
assert server.meta_db.get_enrichment("haspack.sloppak")["art_state"] == "pack"
|
||||||
assert server.meta_db.get_enrichment("b.sloppak")["art_state"] == "caa"
|
assert server.meta_db.get_enrichment("b.sloppak")["art_state"] == "caa"
|
||||||
assert server.meta_db.get_enrichment("c.sloppak")["art_state"] == "caa"
|
assert server.meta_db.get_enrichment("c.sloppak")["art_state"] == "caa"
|
||||||
@@ -214,10 +215,10 @@ def test_caa_skips_pack_art_and_dedupes_by_release(server, caa):
|
|||||||
def test_caa_404_marks_none(server, caa):
|
def test_caa_404_marks_none(server, caa):
|
||||||
make_sloppak(server, "a.sloppak")
|
make_sloppak(server, "a.sloppak")
|
||||||
_match_row(server, "a.sloppak", release_id="rel-missing")
|
_match_row(server, "a.sloppak", release_id="rel-missing")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "none"
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "none"
|
||||||
n = len(caa.calls)
|
n = len(caa.calls)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(caa.calls) == n # never re-hammered
|
assert len(caa.calls) == n # never re-hammered
|
||||||
|
|
||||||
|
|
||||||
@@ -226,13 +227,13 @@ def test_caa_transport_error_leaves_row_unevaluated(server, caa, monkeypatch):
|
|||||||
_match_row(server, "a.sloppak")
|
_match_row(server, "a.sloppak")
|
||||||
|
|
||||||
def _down(release_id):
|
def _down(release_id):
|
||||||
raise server.EnrichTransportError("down")
|
raise enrichment.EnrichTransportError("down")
|
||||||
monkeypatch.setattr(server, "_caa_http_get", _down)
|
monkeypatch.setattr(enrichment, "_caa_http_get", _down)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
||||||
# Network back → next pass completes it.
|
# Network back → next pass completes it.
|
||||||
monkeypatch.setattr(server, "_caa_http_get", caa)
|
monkeypatch.setattr(enrichment, "_caa_http_get", caa)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
|
||||||
|
|
||||||
|
|
||||||
@@ -240,22 +241,22 @@ def test_offline_default_skips_art_worker(server, monkeypatch):
|
|||||||
"""Under the plain test env the whole art phase is skipped with the rest
|
"""Under the plain test env the whole art phase is skipped with the rest
|
||||||
of the network work."""
|
of the network work."""
|
||||||
calls = []
|
calls = []
|
||||||
monkeypatch.setattr(server, "_caa_http_get", lambda rid: calls.append(rid))
|
monkeypatch.setattr(enrichment, "_caa_http_get", lambda rid: calls.append(rid))
|
||||||
make_sloppak(server, "a.sloppak")
|
make_sloppak(server, "a.sloppak")
|
||||||
_match_row(server, "a.sloppak")
|
_match_row(server, "a.sloppak")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert calls == []
|
assert calls == []
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
||||||
|
|
||||||
|
|
||||||
def test_lru_prune_evicts_oldest_and_resets_rows(server, caa, monkeypatch):
|
def test_lru_prune_evicts_oldest_and_resets_rows(server, caa, monkeypatch):
|
||||||
monkeypatch.setattr(server, "_CAA_CACHE_CAP_BYTES", 1) # everything over cap
|
monkeypatch.setattr(enrichment, "_CAA_CACHE_CAP_BYTES", 1) # everything over cap
|
||||||
make_sloppak(server, "a.sloppak", title="One")
|
make_sloppak(server, "a.sloppak", title="One")
|
||||||
make_sloppak(server, "b.sloppak", title="Two")
|
make_sloppak(server, "b.sloppak", title="Two")
|
||||||
caa.art["rel-2"] = png_bytes((1, 1, 1))
|
caa.art["rel-2"] = png_bytes((1, 1, 1))
|
||||||
_match_row(server, "a.sloppak", release_id="rel-1")
|
_match_row(server, "a.sloppak", release_id="rel-1")
|
||||||
_match_row(server, "b.sloppak", release_id="rel-2")
|
_match_row(server, "b.sloppak", release_id="rel-2")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
# With a 1-byte cap every fetch immediately evicts — the rows that pointed
|
# With a 1-byte cap every fetch immediately evicts — the rows that pointed
|
||||||
# at evicted files were reset to unevaluated.
|
# at evicted files were reset to unevaluated.
|
||||||
caa_files = list(server.ART_CACHE_DIR.glob("caa_*.jpg"))
|
caa_files = list(server.ART_CACHE_DIR.glob("caa_*.jpg"))
|
||||||
@@ -282,13 +283,13 @@ def test_delete_override_restores_caa_fallback(server, client, caa):
|
|||||||
_match_row(server, "a.sloppak")
|
_match_row(server, "a.sloppak")
|
||||||
# Pin an override BEFORE the art worker runs → the pass stamps art_state='user'.
|
# Pin an override BEFORE the art worker runs → the pass stamps art_state='user'.
|
||||||
client.post("/api/song/a.sloppak/art/upload", json={"image": b64(png_bytes())})
|
client.post("/api/song/a.sloppak/art/upload", json={"image": b64(png_bytes())})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "user"
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "user"
|
||||||
# Remove it → the row resets to unevaluated…
|
# Remove it → the row resets to unevaluated…
|
||||||
assert client.delete("/api/art/a.sloppak/override").json()["removed"]
|
assert client.delete("/api/art/a.sloppak/override").json()["removed"]
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
||||||
# …and the next pass fetches + serves the release's front cover.
|
# …and the next pass fetches + serves the release's front cover.
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
|
||||||
r = client.get("/api/song/a.sloppak/art")
|
r = client.get("/api/song/a.sloppak/art")
|
||||||
assert r.status_code == 200
|
assert r.status_code == 200
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ Two halves, mirroring the design's split:
|
|||||||
|
|
||||||
* GET /api/artist/{name}/links + POST .../links/refresh — the lazy, cached,
|
* GET /api/artist/{name}/links + POST .../links/refresh — the lazy, cached,
|
||||||
opt-in external-links layer. The HTTP transport is a fake over
|
opt-in external-links layer. The HTTP transport is a fake over
|
||||||
`server._mb_http_get` (the ONE network seam — same pattern as
|
`enrichment._mb_http_get` (the ONE network seam — same pattern as
|
||||||
tests/test_mb_enrichment.py), so nothing here opens a socket. Covers the
|
tests/test_mb_enrichment.py), so nothing here opens a socket. Covers the
|
||||||
url-rel whitelist mapping, the http(s) scheme gate (a hostile javascript:
|
url-rel whitelist mapping, the http(s) scheme gate (a hostile javascript:
|
||||||
resource never reaches a link slot), cache-hit second calls making no
|
resource never reaches a link slot), cache-hit second calls making no
|
||||||
@@ -19,6 +19,7 @@ Two halves, mirroring the design's split:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import json
|
import json
|
||||||
import sys
|
import sys
|
||||||
from urllib.parse import quote
|
from urllib.parse import quote
|
||||||
@@ -88,7 +89,7 @@ class FakeMBArtist:
|
|||||||
|
|
||||||
def __call__(self, path, params):
|
def __call__(self, path, params):
|
||||||
if self.raise_transport:
|
if self.raise_transport:
|
||||||
raise self._srv.EnrichTransportError("fake network down")
|
raise enrichment.EnrichTransportError("fake network down")
|
||||||
self.calls.append((path, dict(params)))
|
self.calls.append((path, dict(params)))
|
||||||
if path == f"artist/{MBID}":
|
if path == f"artist/{MBID}":
|
||||||
return self.doc
|
return self.doc
|
||||||
@@ -100,8 +101,8 @@ def mb_artist(server, monkeypatch):
|
|||||||
"""Install the fake transport AND enable the network flag (the test env
|
"""Install the fake transport AND enable the network flag (the test env
|
||||||
disables it by default — see test_links_offline_returns_empty)."""
|
disables it by default — see test_links_offline_returns_empty)."""
|
||||||
fake = FakeMBArtist(server)
|
fake = FakeMBArtist(server)
|
||||||
monkeypatch.setattr(server, "_mb_http_get", fake)
|
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ contents). The refresh flow reuses the P8 fake-transport pattern — nothing
|
|||||||
here opens a socket."""
|
here opens a socket."""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -85,9 +86,9 @@ def test_refresh_resets_even_a_manual_pin_and_rematches(server, client, monkeypa
|
|||||||
"status": "Official", "date": "1990-09-24",
|
"status": "Official", "date": "1990-09-24",
|
||||||
"release-group": {"primary-type": "Album"}}],
|
"release-group": {"primary-type": "Album"}}],
|
||||||
}]}
|
}]}
|
||||||
monkeypatch.setattr(server, "_mb_http_get", fake)
|
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["mb_recording_id"] == "rec-new"
|
assert server.meta_db.get_enrichment("a.sloppak")["mb_recording_id"] == "rec-new"
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ contracts it will inherit: rename-survivable idempotent hashing, manual rows
|
|||||||
never auto-reset, never purged on rescan, purged on explicit delete."""
|
never auto-reset, never purged on rescan, purged on explicit delete."""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -58,7 +59,7 @@ def test_pending_covers_new_unscanned_and_changed(server):
|
|||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
assert [r["filename"] for r in server.meta_db.enrichment_pending()] == ["a.archive"]
|
assert [r["filename"] for r in server.meta_db.enrichment_pending()] == ["a.archive"]
|
||||||
# stubbed → still unscanned → still pending (the matcher hasn't run)
|
# stubbed → still unscanned → still pending (the matcher hasn't run)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert [r["filename"] for r in server.meta_db.enrichment_pending()] == ["a.archive"]
|
assert [r["filename"] for r in server.meta_db.enrichment_pending()] == ["a.archive"]
|
||||||
# a matched row with the CURRENT hash is settled…
|
# a matched row with the CURRENT hash is settled…
|
||||||
h = server.meta_db.enrichment_content_hash("Artist", "Song", "", 100)
|
h = server.meta_db.enrichment_content_hash("Artist", "Song", "", 100)
|
||||||
@@ -76,7 +77,7 @@ def test_pending_covers_new_unscanned_and_changed(server):
|
|||||||
def test_hash_change_resets_matched_but_never_manual(server):
|
def test_hash_change_resets_matched_but_never_manual(server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
_put(server, "b.archive", title="Other")
|
_put(server, "b.archive", title="Other")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
with server.meta_db._lock:
|
with server.meta_db._lock:
|
||||||
server.meta_db.conn.execute(
|
server.meta_db.conn.execute(
|
||||||
"UPDATE song_enrichment SET match_state = 'matched' WHERE filename = 'a.archive'")
|
"UPDATE song_enrichment SET match_state = 'matched' WHERE filename = 'a.archive'")
|
||||||
@@ -86,7 +87,7 @@ def test_hash_change_resets_matched_but_never_manual(server):
|
|||||||
# identity edits…
|
# identity edits…
|
||||||
_put(server, "a.archive", title="Song v2")
|
_put(server, "a.archive", title="Song v2")
|
||||||
_put(server, "b.archive", title="Other v2")
|
_put(server, "b.archive", title="Other v2")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
a = server.meta_db.get_enrichment("a.archive")
|
a = server.meta_db.get_enrichment("a.archive")
|
||||||
b = server.meta_db.get_enrichment("b.archive")
|
b = server.meta_db.get_enrichment("b.archive")
|
||||||
# …drop a stale MATCH back to unscanned with the fresh hash
|
# …drop a stale MATCH back to unscanned with the fresh hash
|
||||||
@@ -99,7 +100,7 @@ def test_hash_change_resets_matched_but_never_manual(server):
|
|||||||
|
|
||||||
def test_failed_rows_not_requeued_by_pending(server):
|
def test_failed_rows_not_requeued_by_pending(server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
with server.meta_db._lock:
|
with server.meta_db._lock:
|
||||||
server.meta_db.conn.execute(
|
server.meta_db.conn.execute(
|
||||||
"UPDATE song_enrichment SET match_state = 'failed' WHERE filename = 'a.archive'")
|
"UPDATE song_enrichment SET match_state = 'failed' WHERE filename = 'a.archive'")
|
||||||
@@ -113,7 +114,7 @@ def test_failed_rows_not_requeued_by_pending(server):
|
|||||||
def test_enrich_pass_stamps_every_song(server):
|
def test_enrich_pass_stamps_every_song(server):
|
||||||
for i in range(5):
|
for i in range(5):
|
||||||
_put(server, f"s{i}.archive", title=f"Song {i}")
|
_put(server, f"s{i}.archive", title=f"Song {i}")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
for i in range(5):
|
for i in range(5):
|
||||||
row = server.meta_db.get_enrichment(f"s{i}.archive")
|
row = server.meta_db.get_enrichment(f"s{i}.archive")
|
||||||
assert row is not None
|
assert row is not None
|
||||||
@@ -126,7 +127,7 @@ def test_enrich_pass_stamps_every_song(server):
|
|||||||
|
|
||||||
def test_rescan_never_purges_enrichment(server):
|
def test_rescan_never_purges_enrichment(server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
server.meta_db.delete_missing(set()) # file vanished from a scan snapshot
|
server.meta_db.delete_missing(set()) # file vanished from a scan snapshot
|
||||||
assert server.meta_db.get_enrichment("a.archive") is not None # row survives
|
assert server.meta_db.get_enrichment("a.archive") is not None # row survives
|
||||||
# …and is invisible in the read-time-filtered counts
|
# …and is invisible in the read-time-filtered counts
|
||||||
@@ -138,7 +139,7 @@ def test_rescan_never_purges_enrichment(server):
|
|||||||
def test_status_endpoint_counts(client, server):
|
def test_status_endpoint_counts(client, server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
_put(server, "b.archive", title="Other")
|
_put(server, "b.archive", title="Other")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
body = client.get("/api/enrichment/status").json()
|
body = client.get("/api/enrichment/status").json()
|
||||||
assert body["states"] == {"unscanned": 2}
|
assert body["states"] == {"unscanned": 2}
|
||||||
assert body["total_songs"] == 2
|
assert body["total_songs"] == 2
|
||||||
@@ -147,7 +148,7 @@ def test_status_endpoint_counts(client, server):
|
|||||||
|
|
||||||
|
|
||||||
def test_art_cache_dir_created(server):
|
def test_art_cache_dir_created(server):
|
||||||
d = server._enrichment_art_dir()
|
d = enrichment._enrichment_art_dir()
|
||||||
assert d.is_dir()
|
assert d.is_dir()
|
||||||
assert d.name == "art_cache"
|
assert d.name == "art_cache"
|
||||||
|
|
||||||
@@ -156,7 +157,7 @@ def test_art_cache_dir_created(server):
|
|||||||
|
|
||||||
def test_states_for_returns_only_known_filenames(server):
|
def test_states_for_returns_only_known_filenames(server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
got = server.meta_db.enrichment_states_for(["a.archive", "nope.archive"])
|
got = server.meta_db.enrichment_states_for(["a.archive", "nope.archive"])
|
||||||
assert got == {"a.archive": "unscanned"} # unknown filename absent
|
assert got == {"a.archive": "unscanned"} # unknown filename absent
|
||||||
assert server.meta_db.enrichment_states_for([]) == {}
|
assert server.meta_db.enrichment_states_for([]) == {}
|
||||||
@@ -165,7 +166,7 @@ def test_states_for_returns_only_known_filenames(server):
|
|||||||
def test_states_endpoint(client, server):
|
def test_states_endpoint(client, server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
_put(server, "b.archive", title="Other")
|
_put(server, "b.archive", title="Other")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
body = client.post("/api/enrichment/states",
|
body = client.post("/api/enrichment/states",
|
||||||
json={"filenames": ["a.archive", "zzz.missing"]}).json()
|
json={"filenames": ["a.archive", "zzz.missing"]}).json()
|
||||||
assert body["states"] == {"a.archive": "unscanned"}
|
assert body["states"] == {"a.archive": "unscanned"}
|
||||||
@@ -175,7 +176,7 @@ def test_states_endpoint(client, server):
|
|||||||
|
|
||||||
def test_status_exposes_progress_fields(client, server):
|
def test_status_exposes_progress_fields(client, server):
|
||||||
_put(server, "a.archive")
|
_put(server, "a.archive")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
body = client.get("/api/enrichment/status").json()
|
body = client.get("/api/enrichment/status").json()
|
||||||
for k in ("total", "matched", "current", "cancelling"):
|
for k in ("total", "matched", "current", "cancelling"):
|
||||||
assert k in body
|
assert k in body
|
||||||
@@ -186,7 +187,7 @@ def test_cancel_is_noop_when_idle(client, server):
|
|||||||
body = client.post("/api/enrichment/cancel").json()
|
body = client.post("/api/enrichment/cancel").json()
|
||||||
assert body == {"ok": True, "was_running": False}
|
assert body == {"ok": True, "was_running": False}
|
||||||
# A no-op must not arm the flag (which would then poison the next pass).
|
# A no-op must not arm the flag (which would then poison the next pass).
|
||||||
assert server._enrich_cancel.is_set() is False
|
assert enrichment._enrich_cancel.is_set() is False
|
||||||
|
|
||||||
|
|
||||||
def test_cancel_flag_halts_matching_loop_between_songs(server, monkeypatch):
|
def test_cancel_flag_halts_matching_loop_between_songs(server, monkeypatch):
|
||||||
@@ -195,28 +196,28 @@ def test_cancel_flag_halts_matching_loop_between_songs(server, monkeypatch):
|
|||||||
# Force the matcher path on (the test env is offline by default) and stub the
|
# Force the matcher path on (the test env is offline by default) and stub the
|
||||||
# per-song matcher so nothing touches the network — it just trips Stop after
|
# per-song matcher so nothing touches the network — it just trips Stop after
|
||||||
# the first song, exactly as the /cancel route would mid-pass.
|
# the first song, exactly as the /cancel route would mid-pass.
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
calls = []
|
calls = []
|
||||||
|
|
||||||
def fake_enrich_one(row, **_kw):
|
def fake_enrich_one(row, **_kw):
|
||||||
calls.append(row["filename"])
|
calls.append(row["filename"])
|
||||||
server._enrich_cancel.set()
|
enrichment._enrich_cancel.set()
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_enrich_one", fake_enrich_one)
|
monkeypatch.setattr(enrichment, "_enrich_one", fake_enrich_one)
|
||||||
server._enrich_cancel.clear()
|
enrichment._enrich_cancel.clear()
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
# The loop checks cancel BEFORE each song, so exactly one is processed before
|
# The loop checks cancel BEFORE each song, so exactly one is processed before
|
||||||
# it breaks — not the whole 4-row queue.
|
# it breaks — not the whole 4-row queue.
|
||||||
assert calls == ["s0.archive"]
|
assert calls == ["s0.archive"]
|
||||||
assert server._enrich_status["total"] == 4
|
assert enrichment._enrich_status["total"] == 4
|
||||||
assert server._enrich_status["matched"] == 1
|
assert enrichment._enrich_status["matched"] == 1
|
||||||
|
|
||||||
|
|
||||||
def test_rematch_requeues_visible_but_skips_manual(server, client):
|
def test_rematch_requeues_visible_but_skips_manual(server, client):
|
||||||
_put(server, "a.archive") # will be 'matched'
|
_put(server, "a.archive") # will be 'matched'
|
||||||
_put(server, "b.archive", title="Other") # will be 'failed'
|
_put(server, "b.archive", title="Other") # will be 'failed'
|
||||||
_put(server, "c.archive", title="Pinned") # will be 'manual' — untouchable
|
_put(server, "c.archive", title="Pinned") # will be 'manual' — untouchable
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
with server.meta_db._lock:
|
with server.meta_db._lock:
|
||||||
server.meta_db.conn.execute(
|
server.meta_db.conn.execute(
|
||||||
"UPDATE song_enrichment SET match_state='matched' WHERE filename='a.archive'")
|
"UPDATE song_enrichment SET match_state='matched' WHERE filename='a.archive'")
|
||||||
@@ -240,7 +241,7 @@ def test_rematch_requeues_visible_but_skips_manual(server, client):
|
|||||||
# ── filename-derived artist/title fallback (blank-artist packs) ───────────────
|
# ── filename-derived artist/title fallback (blank-artist packs) ───────────────
|
||||||
|
|
||||||
def test_filename_artist_title_parse(server):
|
def test_filename_artist_title_parse(server):
|
||||||
f = server._artist_title_from_filename
|
f = enrichment._artist_title_from_filename
|
||||||
assert f("CDLC/0 - City Pop/Tatsuro-Yamashita_Ride-On-Time_v1_p.feedpak") == \
|
assert f("CDLC/0 - City Pop/Tatsuro-Yamashita_Ride-On-Time_v1_p.feedpak") == \
|
||||||
{"artist": "Tatsuro Yamashita", "title": "Ride On Time"}
|
{"artist": "Tatsuro Yamashita", "title": "Ride On Time"}
|
||||||
assert f("Anri_Windy-Summer_v1_p.feedpak") == {"artist": "Anri", "title": "Windy Summer"}
|
assert f("Anri_Windy-Summer_v1_p.feedpak") == {"artist": "Anri", "title": "Windy Summer"}
|
||||||
@@ -255,18 +256,18 @@ def test_blank_artist_seeds_match_from_filename(server, monkeypatch):
|
|||||||
server.meta_db.put("Tatsuro-Yamashita_Ride-On-Time_v1_p.feedpak", 0, 0, {
|
server.meta_db.put("Tatsuro-Yamashita_Ride-On-Time_v1_p.feedpak", 0, 0, {
|
||||||
"title": "Tatsuro-Yamashita_Ride-On-Time_v1_p", "artist": "", "album": "",
|
"title": "Tatsuro-Yamashita_Ride-On-Time_v1_p", "artist": "", "album": "",
|
||||||
"duration": 240, "arrangements": [{"name": "Bass", "index": 0}]})
|
"duration": 240, "arrangements": [{"name": "Bass", "index": 0}]})
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
monkeypatch.setattr(server, "_manifest_exact_ids", lambda fn: {})
|
monkeypatch.setattr(enrichment, "_manifest_exact_ids", lambda fn: {})
|
||||||
seen = {}
|
seen = {}
|
||||||
|
|
||||||
def fake_search(artist, title, limit=8):
|
def fake_search(artist, title, limit=8):
|
||||||
seen["artist"], seen["title"] = artist, title
|
seen["artist"], seen["title"] = artist, title
|
||||||
return []
|
return []
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_search_recordings", fake_search)
|
monkeypatch.setattr(enrichment, "_mb_search_recordings", fake_search)
|
||||||
row = next(r for r in server.meta_db.enrichment_pending()
|
row = next(r for r in server.meta_db.enrichment_pending()
|
||||||
if r["filename"].startswith("Tatsuro"))
|
if r["filename"].startswith("Tatsuro"))
|
||||||
server._enrich_one(row)
|
enrichment._enrich_one(row)
|
||||||
# the blank pack artist was replaced by the filename-derived identity for
|
# the blank pack artist was replaced by the filename-derived identity for
|
||||||
# the search (this is exactly what rescues the 'failed' pile)
|
# the search (this is exactly what rescues the 'failed' pile)
|
||||||
assert seen == {"artist": "Tatsuro Yamashita", "title": "Ride On Time"}
|
assert seen == {"artist": "Tatsuro Yamashita", "title": "Ride On Time"}
|
||||||
@@ -276,18 +277,18 @@ def test_present_artist_is_not_overridden_by_filename(server, monkeypatch):
|
|||||||
server.meta_db.put("Weird-Filename_x_y.feedpak", 0, 0, {
|
server.meta_db.put("Weird-Filename_x_y.feedpak", 0, 0, {
|
||||||
"title": "Real Title", "artist": "Real Artist", "album": "", "duration": 100,
|
"title": "Real Title", "artist": "Real Artist", "album": "", "duration": 100,
|
||||||
"arrangements": [{"name": "Lead", "index": 0}]})
|
"arrangements": [{"name": "Lead", "index": 0}]})
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
monkeypatch.setattr(server, "_manifest_exact_ids", lambda fn: {})
|
monkeypatch.setattr(enrichment, "_manifest_exact_ids", lambda fn: {})
|
||||||
seen = {}
|
seen = {}
|
||||||
|
|
||||||
def fake_search(artist, title, limit=8):
|
def fake_search(artist, title, limit=8):
|
||||||
seen["artist"], seen["title"] = artist, title
|
seen["artist"], seen["title"] = artist, title
|
||||||
return []
|
return []
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_search_recordings", fake_search)
|
monkeypatch.setattr(enrichment, "_mb_search_recordings", fake_search)
|
||||||
row = next(r for r in server.meta_db.enrichment_pending()
|
row = next(r for r in server.meta_db.enrichment_pending()
|
||||||
if r["filename"].startswith("Weird"))
|
if r["filename"].startswith("Weird"))
|
||||||
server._enrich_one(row)
|
enrichment._enrich_one(row)
|
||||||
# a pack that DOES carry an artist keeps it — the filename is never consulted
|
# a pack that DOES carry an artist keeps it — the filename is never consulted
|
||||||
assert seen == {"artist": "Real Artist", "title": "Real Title"}
|
assert seen == {"artist": "Real Artist", "title": "Real Title"}
|
||||||
|
|
||||||
@@ -295,7 +296,7 @@ def test_present_artist_is_not_overridden_by_filename(server, monkeypatch):
|
|||||||
def test_kick_clears_a_stale_cancel(server):
|
def test_kick_clears_a_stale_cancel(server):
|
||||||
# A cancelled-then-rekicked pass must start clean: _kick_enrich clears the
|
# A cancelled-then-rekicked pass must start clean: _kick_enrich clears the
|
||||||
# flag so the fresh pass isn't aborted the instant it checks.
|
# flag so the fresh pass isn't aborted the instant it checks.
|
||||||
server._enrich_cancel.set()
|
enrichment._enrich_cancel.set()
|
||||||
server._kick_enrich()
|
enrichment._kick_enrich()
|
||||||
server._join_background_db_threads()
|
server._join_background_db_threads()
|
||||||
assert server._enrich_cancel.is_set() is False
|
assert enrichment._enrich_cancel.is_set() is False
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ song (delete_song). Locks pin a field against a later auto-match.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -149,7 +150,7 @@ def test_locked_fields_reader(server):
|
|||||||
|
|
||||||
|
|
||||||
def test_compose_lock_filter_strips_locked_cand_keys(server):
|
def test_compose_lock_filter_strips_locked_cand_keys(server):
|
||||||
f = server._compose_lock_filter(None, {"artist", "year"})
|
f = enrichment._compose_lock_filter(None, {"artist", "year"})
|
||||||
cand = {"recording_id": "r", "artist": "X", "artist_sort": "X", "title": "T",
|
cand = {"recording_id": "r", "artist": "X", "artist_sort": "X", "title": "T",
|
||||||
"year": "1990", "album": "A", "genres": ["rock"]}
|
"year": "1990", "album": "A", "genres": ["rock"]}
|
||||||
out = f(cand)
|
out = f(cand)
|
||||||
@@ -158,7 +159,7 @@ def test_compose_lock_filter_strips_locked_cand_keys(server):
|
|||||||
# …identity + unlocked display fields survive
|
# …identity + unlocked display fields survive
|
||||||
assert out["recording_id"] == "r" and out["title"] == "T" and out["album"] == "A"
|
assert out["recording_id"] == "r" and out["title"] == "T" and out["album"] == "A"
|
||||||
# no locks → base filter returned unchanged (zero-copy common path)
|
# no locks → base filter returned unchanged (zero-copy common path)
|
||||||
assert server._compose_lock_filter(None, set()) is None
|
assert enrichment._compose_lock_filter(None, set()) is None
|
||||||
|
|
||||||
|
|
||||||
# ── display overlay in the grid (slice 3) ─────────────────────────────────────
|
# ── display overlay in the grid (slice 3) ─────────────────────────────────────
|
||||||
|
|||||||
+53
-52
@@ -1,6 +1,6 @@
|
|||||||
"""Server-level tests for the P8 MusicBrainz matcher + Match-Review flow.
|
"""Server-level tests for the P8 MusicBrainz matcher + Match-Review flow.
|
||||||
|
|
||||||
The HTTP transport is a fake installed over `server._mb_http_get` — the ONE
|
The HTTP transport is a fake installed over `enrichment._mb_http_get` — the ONE
|
||||||
seam enrichment uses to reach the network — so nothing here ever opens a
|
seam enrichment uses to reach the network — so nothing here ever opens a
|
||||||
socket. The offline default is itself under test: without explicitly
|
socket. The offline default is itself under test: without explicitly
|
||||||
enabling the network flag, a pass must skip matching entirely (pytest can
|
enabling the network flag, a pass must skip matching entirely (pytest can
|
||||||
@@ -8,6 +8,7 @@ never hit MusicBrainz, whatever a test triggers).
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -50,7 +51,7 @@ class FakeMB:
|
|||||||
|
|
||||||
def __call__(self, path, params):
|
def __call__(self, path, params):
|
||||||
if self.raise_transport:
|
if self.raise_transport:
|
||||||
raise self._srv.EnrichTransportError("fake network down")
|
raise enrichment.EnrichTransportError("fake network down")
|
||||||
self.calls.append((path, dict(params)))
|
self.calls.append((path, dict(params)))
|
||||||
if path == "recording":
|
if path == "recording":
|
||||||
return self.search_response
|
return self.search_response
|
||||||
@@ -71,8 +72,8 @@ def mb(server, monkeypatch):
|
|||||||
disables it by default — see test_offline_default_skips_matching)."""
|
disables it by default — see test_offline_default_skips_matching)."""
|
||||||
fake = FakeMB()
|
fake = FakeMB()
|
||||||
fake._srv = server
|
fake._srv = server
|
||||||
monkeypatch.setattr(server, "_mb_http_get", fake)
|
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
@@ -111,8 +112,8 @@ def test_search_falls_back_to_loose_when_strict_is_empty(server, monkeypatch):
|
|||||||
return {"recordings": []}
|
return {"recordings": []}
|
||||||
return {"recordings": [mb_doc(rid="rec-x", title="Telephone Number")]}
|
return {"recordings": [mb_doc(rid="rec-x", title="Telephone Number")]}
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_http_get", _routed)
|
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
|
||||||
cands = server._mb_search_recordings("Junko Ohashi", "Telephone Number")
|
cands = enrichment._mb_search_recordings("Junko Ohashi", "Telephone Number")
|
||||||
assert len(cands) == 1
|
assert len(cands) == 1
|
||||||
assert len(calls) == 2 # strict first, then the loose retry
|
assert len(calls) == 2 # strict first, then the loose retry
|
||||||
assert calls[0].startswith("recording:") # strict is the field-phrase form
|
assert calls[0].startswith("recording:") # strict is the field-phrase form
|
||||||
@@ -128,8 +129,8 @@ def test_search_does_not_retry_when_strict_hits(server, monkeypatch):
|
|||||||
calls.append(params.get("query", ""))
|
calls.append(params.get("query", ""))
|
||||||
return {"recordings": [mb_doc()]}
|
return {"recordings": [mb_doc()]}
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_http_get", _routed)
|
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
|
||||||
cands = server._mb_search_recordings("AC/DC", "Thunderstruck")
|
cands = enrichment._mb_search_recordings("AC/DC", "Thunderstruck")
|
||||||
assert len(cands) == 1
|
assert len(cands) == 1
|
||||||
assert len(calls) == 1
|
assert len(calls) == 1
|
||||||
|
|
||||||
@@ -147,10 +148,10 @@ def test_artist_aliases_fetched_and_cached(server, monkeypatch):
|
|||||||
return {"sort-name": "Ohashi, Junko",
|
return {"sort-name": "Ohashi, Junko",
|
||||||
"aliases": [{"name": "Junko Ohashi"}, {"name": "大橋 純子"}]}
|
"aliases": [{"name": "Junko Ohashi"}, {"name": "大橋 純子"}]}
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_http_get", fake)
|
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
|
||||||
names = server._mb_artist_aliases(_AID)
|
names = enrichment._mb_artist_aliases(_AID)
|
||||||
assert "Junko Ohashi" in names and "Ohashi, Junko" in names
|
assert "Junko Ohashi" in names and "Ohashi, Junko" in names
|
||||||
server._mb_artist_aliases(_AID) # cached → no second request
|
enrichment._mb_artist_aliases(_AID) # cached → no second request
|
||||||
assert len(calls) == 1
|
assert len(calls) == 1
|
||||||
|
|
||||||
|
|
||||||
@@ -158,8 +159,8 @@ def test_artist_aliases_rejects_bad_id(server, monkeypatch):
|
|||||||
def boom(path, params):
|
def boom(path, params):
|
||||||
raise AssertionError("must not fetch for a non-UUID id")
|
raise AssertionError("must not fetch for a non-UUID id")
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_http_get", boom)
|
monkeypatch.setattr(enrichment, "_mb_http_get", boom)
|
||||||
assert server._mb_artist_aliases("not-a-uuid") == []
|
assert enrichment._mb_artist_aliases("not-a-uuid") == []
|
||||||
|
|
||||||
|
|
||||||
def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
|
def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
|
||||||
@@ -176,9 +177,9 @@ def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
|
|||||||
return {"recordings": [mb_doc(rid="rec-jp", title="Telephone Number",
|
return {"recordings": [mb_doc(rid="rec-jp", title="Telephone Number",
|
||||||
artist="大橋純子", artist_id=_AID)]} # loose hit
|
artist="大橋純子", artist_id=_AID)]} # loose hit
|
||||||
|
|
||||||
monkeypatch.setattr(server, "_mb_http_get", _routed)
|
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("x.sloppak")
|
row = server.meta_db.get_enrichment("x.sloppak")
|
||||||
# The romanized alias lifts the artist over the auto floor → auto-confirmed.
|
# The romanized alias lifts the artist over the auto floor → auto-confirmed.
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
@@ -190,10 +191,10 @@ def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
|
|||||||
def test_locked_field_not_canonicalized_by_auto_match(server, monkeypatch):
|
def test_locked_field_not_canonicalized_by_auto_match(server, monkeypatch):
|
||||||
_put(server, "x.sloppak") # title "Thunderstruck (v2)", artist "ACDC"
|
_put(server, "x.sloppak") # title "Thunderstruck (v2)", artist "ACDC"
|
||||||
server.meta_db.set_song_override("x.sloppak", "artist", locked=True)
|
server.meta_db.set_song_override("x.sloppak", "artist", locked=True)
|
||||||
monkeypatch.setattr(server, "_mb_http_get",
|
monkeypatch.setattr(enrichment, "_mb_http_get",
|
||||||
lambda path, params: {"recordings": [mb_doc()]})
|
lambda path, params: {"recordings": [mb_doc()]})
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("x.sloppak")
|
row = server.meta_db.get_enrichment("x.sloppak")
|
||||||
assert row["match_state"] == "matched" # still matches (identity applies)…
|
assert row["match_state"] == "matched" # still matches (identity applies)…
|
||||||
assert row["canon_artist"] is None # …but the LOCKED artist isn't canonicalized
|
assert row["canon_artist"] is None # …but the LOCKED artist isn't canonicalized
|
||||||
@@ -208,9 +209,9 @@ def test_offline_default_skips_matching(server, monkeypatch):
|
|||||||
but never matches — even with a transport installed."""
|
but never matches — even with a transport installed."""
|
||||||
fake = FakeMB()
|
fake = FakeMB()
|
||||||
fake._srv = server
|
fake._srv = server
|
||||||
monkeypatch.setattr(server, "_mb_http_get", fake)
|
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert fake.calls == []
|
assert fake.calls == []
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "unscanned"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "unscanned"
|
||||||
|
|
||||||
@@ -218,15 +219,15 @@ def test_offline_default_skips_matching(server, monkeypatch):
|
|||||||
def test_real_transport_refuses_when_offline(server):
|
def test_real_transport_refuses_when_offline(server):
|
||||||
"""_mb_http_get itself raises (before any socket) when the network is
|
"""_mb_http_get itself raises (before any socket) when the network is
|
||||||
disabled — defence in depth under pytest."""
|
disabled — defence in depth under pytest."""
|
||||||
with pytest.raises(server.EnrichTransportError):
|
with pytest.raises(enrichment.EnrichTransportError):
|
||||||
server._mb_http_get("recording", {"query": "x"})
|
enrichment._mb_http_get("recording", {"query": "x"})
|
||||||
|
|
||||||
|
|
||||||
def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
|
def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
_put(server, "b.sloppak", title="Other Song")
|
_put(server, "b.sloppak", title="Other Song")
|
||||||
mb.raise_transport = True
|
mb.raise_transport = True
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
for fn in ("a.sloppak", "b.sloppak"):
|
for fn in ("a.sloppak", "b.sloppak"):
|
||||||
row = server.meta_db.get_enrichment(fn)
|
row = server.meta_db.get_enrichment(fn)
|
||||||
assert row["match_state"] == "unscanned"
|
assert row["match_state"] == "unscanned"
|
||||||
@@ -234,7 +235,7 @@ def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
|
|||||||
# Network comes back → the next kick matches both.
|
# Network comes back → the next kick matches both.
|
||||||
mb.raise_transport = False
|
mb.raise_transport = False
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
||||||
|
|
||||||
|
|
||||||
@@ -243,7 +244,7 @@ def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
|
|||||||
def test_high_confidence_auto_matches_and_settles(server, mb):
|
def test_high_confidence_auto_matches_and_settles(server, mb):
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["match_source"] == "text"
|
assert row["match_source"] == "text"
|
||||||
@@ -256,13 +257,13 @@ def test_high_confidence_auto_matches_and_settles(server, mb):
|
|||||||
assert row["genres"] == ["hard rock"]
|
assert row["genres"] == ["hard rock"]
|
||||||
# Settled: another pass makes NO further network calls…
|
# Settled: another pass makes NO further network calls…
|
||||||
n = len(mb.calls)
|
n = len(mb.calls)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(mb.calls) == n
|
assert len(mb.calls) == n
|
||||||
# …until the identity changes, which re-matches.
|
# …until the identity changes, which re-matches.
|
||||||
_put(server, "a.sloppak", title="Back in Black")
|
_put(server, "a.sloppak", title="Back in Black")
|
||||||
mb.search_response = {"recordings": [mb_doc(rid="rec-2", title="Back in Black",
|
mb.search_response = {"recordings": [mb_doc(rid="rec-2", title="Back in Black",
|
||||||
album="Back in Black", date="1980-07-25")]}
|
album="Back in Black", date="1980-07-25")]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["mb_recording_id"] == "rec-2"
|
assert row["mb_recording_id"] == "rec-2"
|
||||||
|
|
||||||
@@ -271,7 +272,7 @@ def test_medium_confidence_goes_to_review_not_canonical(server, mb):
|
|||||||
# Partial artist agreement → medium confidence.
|
# Partial artist agreement → medium confidence.
|
||||||
_put(server, "a.sloppak", artist="AC/DC ft Nobody")
|
_put(server, "a.sloppak", artist="AC/DC ft Nobody")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "review"
|
assert row["match_state"] == "review"
|
||||||
assert row["match_source"] == "text"
|
assert row["match_source"] == "text"
|
||||||
@@ -281,7 +282,7 @@ def test_medium_confidence_goes_to_review_not_canonical(server, mb):
|
|||||||
assert row["candidates"] and row["candidates"][0]["recording_id"] == "rec-1"
|
assert row["candidates"] and row["candidates"][0]["recording_id"] == "rec-1"
|
||||||
# A review row is settled while its identity is unchanged — no re-query.
|
# A review row is settled while its identity is unchanged — no re-query.
|
||||||
n = len(mb.calls)
|
n = len(mb.calls)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(mb.calls) == n
|
assert len(mb.calls) == n
|
||||||
|
|
||||||
|
|
||||||
@@ -289,14 +290,14 @@ def test_low_confidence_fails_with_backoff(server, mb):
|
|||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc(rid="rec-x", title="Sunrise",
|
mb.search_response = {"recordings": [mb_doc(rid="rec-x", title="Sunrise",
|
||||||
artist="Norah Jones")]}
|
artist="Norah Jones")]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "failed"
|
assert row["match_state"] == "failed"
|
||||||
assert row["attempts"] == 1
|
assert row["attempts"] == 1
|
||||||
assert row["last_attempt_at"] is not None
|
assert row["last_attempt_at"] is not None
|
||||||
# Immediately after, the backoff hasn't elapsed → no retry, no network.
|
# Immediately after, the backoff hasn't elapsed → no retry, no network.
|
||||||
n = len(mb.calls)
|
n = len(mb.calls)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(mb.calls) == n
|
assert len(mb.calls) == n
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["attempts"] == 1
|
assert server.meta_db.get_enrichment("a.sloppak")["attempts"] == 1
|
||||||
# Rewind the clock two hours → eligible again, attempts increments.
|
# Rewind the clock two hours → eligible again, attempts increments.
|
||||||
@@ -304,7 +305,7 @@ def test_low_confidence_fails_with_backoff(server, mb):
|
|||||||
server.meta_db.conn.execute(
|
server.meta_db.conn.execute(
|
||||||
"UPDATE song_enrichment SET last_attempt_at = last_attempt_at - 7200")
|
"UPDATE song_enrichment SET last_attempt_at = last_attempt_at - 7200")
|
||||||
server.meta_db.conn.commit()
|
server.meta_db.conn.commit()
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(mb.calls) == n + 1
|
assert len(mb.calls) == n + 1
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["attempts"] == 2
|
assert server.meta_db.get_enrichment("a.sloppak")["attempts"] == 2
|
||||||
|
|
||||||
@@ -312,7 +313,7 @@ def test_low_confidence_fails_with_backoff(server, mb):
|
|||||||
def test_no_results_fails(server, mb):
|
def test_no_results_fails(server, mb):
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": []}
|
mb.search_response = {"recordings": []}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "failed"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "failed"
|
||||||
|
|
||||||
|
|
||||||
@@ -322,7 +323,7 @@ def test_cache_hit_copies_match_without_network(server, mb):
|
|||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
_put(server, "b.sloppak") # identical identity → same content_hash
|
_put(server, "b.sloppak") # identical identity → same content_hash
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(mb.search_calls) == 1 # ONE search covered both charts
|
assert len(mb.search_calls) == 1 # ONE search covered both charts
|
||||||
a = server.meta_db.get_enrichment("a.sloppak")
|
a = server.meta_db.get_enrichment("a.sloppak")
|
||||||
b = server.meta_db.get_enrichment("b.sloppak")
|
b = server.meta_db.get_enrichment("b.sloppak")
|
||||||
@@ -347,7 +348,7 @@ def test_manifest_mbid_tier0(server, mb):
|
|||||||
_write_sloppak_manifest(server, "a.sloppak", f"mbid: {mbid}\n")
|
_write_sloppak_manifest(server, "a.sloppak", f"mbid: {mbid}\n")
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.recording_lookups[mbid] = mb_doc(rid=mbid)
|
mb.recording_lookups[mbid] = mb_doc(rid=mbid)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["match_source"] == "mbid"
|
assert row["match_source"] == "mbid"
|
||||||
@@ -360,7 +361,7 @@ def test_manifest_isrc_tier1(server, mb):
|
|||||||
_write_sloppak_manifest(server, "a.sloppak", "isrc: AUAP09000045\n")
|
_write_sloppak_manifest(server, "a.sloppak", "isrc: AUAP09000045\n")
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.isrc_lookups["AUAP09000045"] = {"recordings": [mb_doc()]}
|
mb.isrc_lookups["AUAP09000045"] = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["match_source"] == "isrc"
|
assert row["match_source"] == "isrc"
|
||||||
@@ -374,7 +375,7 @@ def test_manifest_isrc_display_hyphens_stripped(server, mb):
|
|||||||
_write_sloppak_manifest(server, "a.sloppak", "isrc: AU-AP0-90-00045\n")
|
_write_sloppak_manifest(server, "a.sloppak", "isrc: AU-AP0-90-00045\n")
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.isrc_lookups["AUAP09000045"] = {"recordings": [mb_doc()]}
|
mb.isrc_lookups["AUAP09000045"] = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["match_source"] == "isrc"
|
assert row["match_source"] == "isrc"
|
||||||
@@ -387,7 +388,7 @@ def test_bad_manifest_mbid_falls_through_to_text(server, mb):
|
|||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.recording_lookups.clear() # lookup 404s (typo'd manifest)
|
mb.recording_lookups.clear() # lookup 404s (typo'd manifest)
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["match_source"] == "text"
|
assert row["match_source"] == "text"
|
||||||
@@ -401,7 +402,7 @@ def test_manual_never_overwritten_by_matcher(server, mb):
|
|||||||
"a.sloppak", {"recording_id": "user-pick", "title": "Thunderstruck",
|
"a.sloppak", {"recording_id": "user-pick", "title": "Thunderstruck",
|
||||||
"artist": "AC/DC"}, source="search")
|
"artist": "AC/DC"}, source="search")
|
||||||
mb.search_response = {"recordings": [mb_doc(rid="machine-pick")]}
|
mb.search_response = {"recordings": [mb_doc(rid="machine-pick")]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "manual"
|
assert row["match_state"] == "manual"
|
||||||
assert row["mb_recording_id"] == "user-pick"
|
assert row["mb_recording_id"] == "user-pick"
|
||||||
@@ -419,7 +420,7 @@ def _seed_review(server, mb, fn="a.sloppak", title="Thunderstruck (v2)"):
|
|||||||
# legitimately copies an earlier row instead of running the text tiers).
|
# legitimately copies an earlier row instead of running the text tiers).
|
||||||
_put(server, fn, title=title, artist="AC/DC ft Nobody")
|
_put(server, fn, title=title, artist="AC/DC ft Nobody")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment(fn)["match_state"] == "review"
|
assert server.meta_db.get_enrichment(fn)["match_state"] == "review"
|
||||||
|
|
||||||
|
|
||||||
@@ -457,11 +458,11 @@ def test_review_reject_route_never_retries(server, mb, client):
|
|||||||
assert row["match_source"] == "rejected"
|
assert row["match_source"] == "rejected"
|
||||||
# Rejected rows are excluded from the retry backoff forever…
|
# Rejected rows are excluded from the retry backoff forever…
|
||||||
n = len(mb.calls)
|
n = len(mb.calls)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert len(mb.calls) == n
|
assert len(mb.calls) == n
|
||||||
# …but an identity edit re-queues (the user fixed the metadata).
|
# …but an identity edit re-queues (the user fixed the metadata).
|
||||||
_put(server, "a.sloppak", artist="AC/DC")
|
_put(server, "a.sloppak", artist="AC/DC")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
||||||
# Rejecting a manual row is refused.
|
# Rejecting a manual row is refused.
|
||||||
r = client.post("/api/enrichment/review/a.sloppak/reject")
|
r = client.post("/api/enrichment/review/a.sloppak/reject")
|
||||||
@@ -502,8 +503,8 @@ def test_search_proxy(server, mb, client, monkeypatch):
|
|||||||
assert body["candidates"][0]["score"] > 0.9
|
assert body["candidates"][0]["score"] > 0.9
|
||||||
# Transport failure surfaces as 503, not a 500.
|
# Transport failure surfaces as 503, not a 500.
|
||||||
def _down(path, params):
|
def _down(path, params):
|
||||||
raise server.EnrichTransportError("down")
|
raise enrichment.EnrichTransportError("down")
|
||||||
monkeypatch.setattr(server, "_mb_http_get", _down)
|
monkeypatch.setattr(enrichment, "_mb_http_get", _down)
|
||||||
r = client.get("/api/enrichment/search", params={"title": "x"})
|
r = client.get("/api/enrichment/search", params={"title": "x"})
|
||||||
assert r.status_code == 503
|
assert r.status_code == 503
|
||||||
|
|
||||||
@@ -523,8 +524,8 @@ def test_match_facet_filters_grid_and_stats(server, mb, client, monkeypatch):
|
|||||||
if "revsong" in q:
|
if "revsong" in q:
|
||||||
return {"recordings": [mb_doc(rid="rec-r", title="Revsong")]}
|
return {"recordings": [mb_doc(rid="rec-r", title="Revsong")]}
|
||||||
return {"recordings": []}
|
return {"recordings": []}
|
||||||
monkeypatch.setattr(server, "_mb_http_get", _routed)
|
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
# Pendsong got failed by the pass (no results); reset it to unscanned to
|
# Pendsong got failed by the pass (no results); reset it to unscanned to
|
||||||
# represent the not-yet-scanned band.
|
# represent the not-yet-scanned band.
|
||||||
with server.meta_db._lock:
|
with server.meta_db._lock:
|
||||||
@@ -566,14 +567,14 @@ def test_auto_threshold_setting_moves_the_auto_review_boundary(server, mb, clien
|
|||||||
client.post("/api/settings", json={"enrich_auto_threshold": 0.95})
|
client.post("/api/settings", json={"enrich_auto_threshold": 0.95})
|
||||||
_put(server, "a.sloppak", title="Highway to Hell", artist="AC/DC",
|
_put(server, "a.sloppak", title="Highway to Hell", artist="AC/DC",
|
||||||
year="", duration=0)
|
year="", duration=0)
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "review"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "review"
|
||||||
# Lower the bar to the default 0.90 → an identity edit re-queues, and the
|
# Lower the bar to the default 0.90 → an identity edit re-queues, and the
|
||||||
# same 0.90-scored candidate now auto-applies.
|
# same 0.90-scored candidate now auto-applies.
|
||||||
client.post("/api/settings", json={"enrich_auto_threshold": 0.9})
|
client.post("/api/settings", json={"enrich_auto_threshold": 0.9})
|
||||||
_put(server, "a.sloppak", title="Highway to Hell", artist="AC/DC",
|
_put(server, "a.sloppak", title="Highway to Hell", artist="AC/DC",
|
||||||
year="", duration=0, album="Different Album")
|
year="", duration=0, album="Different Album")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert abs(row["match_score"] - 0.9) < 1e-6
|
assert abs(row["match_score"] - 0.9) < 1e-6
|
||||||
@@ -583,7 +584,7 @@ def test_enrich_enabled_setting_gates_background_matching(server, mb, client):
|
|||||||
client.post("/api/settings", json={"enrich_enabled": False})
|
client.post("/api/settings", json={"enrich_enabled": False})
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert mb.calls == []
|
assert mb.calls == []
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "unscanned"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "unscanned"
|
||||||
# Manual search/fix stays available while the background matcher is off.
|
# Manual search/fix stays available while the background matcher is off.
|
||||||
@@ -591,7 +592,7 @@ def test_enrich_enabled_setting_gates_background_matching(server, mb, client):
|
|||||||
assert r.status_code == 200
|
assert r.status_code == 200
|
||||||
# Re-enable → the next pass matches.
|
# Re-enable → the next pass matches.
|
||||||
client.post("/api/settings", json={"enrich_enabled": True})
|
client.post("/api/settings", json={"enrich_enabled": True})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
||||||
|
|
||||||
|
|
||||||
@@ -625,7 +626,7 @@ def test_review_queue_orders_missing_data_first(server, mb, client):
|
|||||||
_seed_review(server, mb, fn="aa.sloppak", title="Thunderstruck (v2)")
|
_seed_review(server, mb, fn="aa.sloppak", title="Thunderstruck (v2)")
|
||||||
_put(server, "zz.sloppak", title="Thunderstruck (Live)",
|
_put(server, "zz.sloppak", title="Thunderstruck (Live)",
|
||||||
artist="AC/DC ft Nobody", album="", year="")
|
artist="AC/DC ft Nobody", album="", year="")
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("zz.sloppak")["match_state"] == "review"
|
assert server.meta_db.get_enrichment("zz.sloppak")["match_state"] == "review"
|
||||||
songs = client.get("/api/enrichment/review").json()["songs"]
|
songs = client.get("/api/enrichment/review").json()["songs"]
|
||||||
assert [s["filename"] for s in songs] == ["zz.sloppak", "aa.sloppak"]
|
assert [s["filename"] for s in songs] == ["zz.sloppak", "aa.sloppak"]
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ flag is only force-enabled where a test needs the pipeline to run.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib
|
import importlib
|
||||||
|
import enrichment
|
||||||
import io as _io
|
import io as _io
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
@@ -57,8 +58,8 @@ class FakeMB:
|
|||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def mb(server, monkeypatch):
|
def mb(server, monkeypatch):
|
||||||
fake = FakeMB()
|
fake = FakeMB()
|
||||||
monkeypatch.setattr(server, "_mb_http_get", fake)
|
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
@@ -116,13 +117,13 @@ def test_musicbrainz_source_off_stamps_without_matching(server, mb, client):
|
|||||||
client.post("/api/settings", json={"enrich_src_musicbrainz": False})
|
client.post("/api/settings", json={"enrich_src_musicbrainz": False})
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert mb.calls == []
|
assert mb.calls == []
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "unscanned" # hash stamped, no match
|
assert row["match_state"] == "unscanned" # hash stamped, no match
|
||||||
# Re-enabling picks the same row up on the next pass.
|
# Re-enabling picks the same row up on the next pass.
|
||||||
client.post("/api/settings", json={"enrich_src_musicbrainz": True})
|
client.post("/api/settings", json={"enrich_src_musicbrainz": True})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
|
||||||
|
|
||||||
|
|
||||||
@@ -133,7 +134,7 @@ def test_field_toggles_strip_auto_applied_fields(server, mb, client):
|
|||||||
"enrich_apply_genres": False})
|
"enrich_apply_genres": False})
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["canon_artist"] == "AC/DC"
|
assert row["canon_artist"] == "AC/DC"
|
||||||
@@ -150,7 +151,7 @@ def test_names_toggle_keeps_ids_and_other_fields(server, mb, client):
|
|||||||
client.post("/api/settings", json={"enrich_apply_names": False})
|
client.post("/api/settings", json={"enrich_apply_names": False})
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["canon_artist"] is None
|
assert row["canon_artist"] is None
|
||||||
@@ -170,7 +171,7 @@ def test_review_accept_applies_all_fields_despite_toggles(server, mb, client):
|
|||||||
# Partial artist agreement → review tier (candidates stored unfiltered).
|
# Partial artist agreement → review tier (candidates stored unfiltered).
|
||||||
_put(server, "a.sloppak", artist="AC/DC ft Nobody")
|
_put(server, "a.sloppak", artist="AC/DC ft Nobody")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "review"
|
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "review"
|
||||||
r = client.post("/api/enrichment/review/a.sloppak/accept",
|
r = client.post("/api/enrichment/review/a.sloppak/accept",
|
||||||
json={"recording_id": "rec-1"})
|
json={"recording_id": "rec-1"})
|
||||||
@@ -192,21 +193,21 @@ def test_reenabling_field_backfills_matched_row(server, mb, client):
|
|||||||
client.post("/api/settings", json={"enrich_apply_year": False})
|
client.post("/api/settings", json={"enrich_apply_year": False})
|
||||||
_put(server, "a.sloppak")
|
_put(server, "a.sloppak")
|
||||||
mb.search_response = {"recordings": [mb_doc()]}
|
mb.search_response = {"recordings": [mb_doc()]}
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["canon_year"] is None # suppressed
|
assert row["canon_year"] is None # suppressed
|
||||||
assert row["apply_mask"] == "enrich_apply_year" # …and remembered
|
assert row["apply_mask"] == "enrich_apply_year" # …and remembered
|
||||||
# Re-enable → next pass re-queues and backfills the year (hash unchanged).
|
# Re-enable → next pass re-queues and backfills the year (hash unchanged).
|
||||||
client.post("/api/settings", json={"enrich_apply_year": True})
|
client.post("/api/settings", json={"enrich_apply_year": True})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["match_state"] == "matched"
|
assert row["match_state"] == "matched"
|
||||||
assert row["canon_year"] == "1990" # backfilled
|
assert row["canon_year"] == "1990" # backfilled
|
||||||
assert row["apply_mask"] in (None, "") # fully applied now
|
assert row["apply_mask"] in (None, "") # fully applied now
|
||||||
# Converged: a fully-applied row is not re-queued again.
|
# Converged: a fully-applied row is not re-queued again.
|
||||||
assert server.meta_db.enrichment_pending(
|
assert server.meta_db.enrichment_pending(
|
||||||
allowed_keys=frozenset(server._ENRICH_APPLY_FIELDS)) == []
|
allowed_keys=frozenset(enrichment._ENRICH_APPLY_FIELDS)) == []
|
||||||
|
|
||||||
|
|
||||||
def test_partial_match_is_not_a_cache_donor(server):
|
def test_partial_match_is_not_a_cache_donor(server):
|
||||||
@@ -271,8 +272,8 @@ def caa(server, monkeypatch):
|
|||||||
calls.append(release_id)
|
calls.append(release_id)
|
||||||
return art.get(release_id)
|
return art.get(release_id)
|
||||||
fake.calls = calls
|
fake.calls = calls
|
||||||
monkeypatch.setattr(server, "_caa_http_get", fake)
|
monkeypatch.setattr(enrichment, "_caa_http_get", fake)
|
||||||
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
|
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
@@ -280,13 +281,13 @@ def test_caa_source_toggle_gates_art_fetch(server, client, caa):
|
|||||||
make_sloppak(server, "a.sloppak")
|
make_sloppak(server, "a.sloppak")
|
||||||
_match_row(server, "a.sloppak")
|
_match_row(server, "a.sloppak")
|
||||||
client.post("/api/settings", json={"enrich_src_caa": False})
|
client.post("/api/settings", json={"enrich_src_caa": False})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert caa.calls == []
|
assert caa.calls == []
|
||||||
row = server.meta_db.get_enrichment("a.sloppak")
|
row = server.meta_db.get_enrichment("a.sloppak")
|
||||||
assert row["art_state"] is None # not forfeited, just skipped
|
assert row["art_state"] is None # not forfeited, just skipped
|
||||||
# Re-enable → the same row is picked up.
|
# Re-enable → the same row is picked up.
|
||||||
client.post("/api/settings", json={"enrich_src_caa": True})
|
client.post("/api/settings", json={"enrich_src_caa": True})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
|
||||||
|
|
||||||
|
|
||||||
@@ -294,7 +295,7 @@ def test_apply_art_toggle_gates_art_fetch(server, client, caa):
|
|||||||
make_sloppak(server, "a.sloppak")
|
make_sloppak(server, "a.sloppak")
|
||||||
_match_row(server, "a.sloppak")
|
_match_row(server, "a.sloppak")
|
||||||
client.post("/api/settings", json={"enrich_apply_art": False})
|
client.post("/api/settings", json={"enrich_apply_art": False})
|
||||||
server._background_enrich()
|
enrichment._background_enrich()
|
||||||
assert caa.calls == []
|
assert caa.calls == []
|
||||||
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user