Compare commits

...
Author SHA1 Message Date
byrongamatosandClaude Opus 4.8 7105511a28 refactor(server): extract the artist routes into routers/artist.py (R3)
The artist page + external-links payload (/api/artist/{name}/page, /links,
/links/refresh) plus their exclusive helpers (_artist_links_payload,
_artist_links_from_mb, the URL-slot table) move to lib/routers/artist.py. Bodies
verbatim except @app->@router and the seam reads (meta_db->appstate.meta_db,
CONFIG_DIR->appstate.config_dir, _default_settings->appstate.default_settings).
MB link enrichment is reached as enrichment.X; the URL-safety validator is
imported from lib/library_registry.py. No new seams.

server.py: 2,507 -> 2,413 (-94).

Verified: pyflakes clean; route set unchanged (143); full pytest 2395 passed.
eslint 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:43:07 +02:00
0547f55844 refactor(server): extract the media/file-serving routes into routers/media.py (R3) (#869)
Song audio (/audio/{f}), the local-audio-path resolver (/api/audio-local-path),
and raw sloppak-member serving (/api/sloppak/{f}/file/{rel}) — plus the shared
_resolve_sloppak_local_file helper — move to lib/routers/media.py. Bodies verbatim
except @app->@router and the cache/static path seams (AUDIO_CACHE_DIR->
appstate.audio_cache_dir, STATIC_DIR->appstate.static_dir, SLOPPAK_CACHE_DIR->
appstate.sloppak_cache_dir — all already in the seam). No new slots.

The two test fixtures that redirect STATIC_DIR to a temp dir now also patch
appstate.static_dir (the moved routes read the seam, not server's global).

server.py: 2,638 -> 2,507 (-131).

Verified: pyflakes clean; route set identical (143), all unique-path (no
catch-all, no shadowing); full pytest 2395 passed (the audio-local-path +
sloppak-file-traversal cases). eslint 0. Boot smoke: /audio, /api/sloppak/{}/file
serve 404 for unknown, 0 tracebacks.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:36:22 +02:00
b7624b7e65 refactor(server): extract the enrichment route handlers into routers/enrichment.py (R3) (#868)
The 14 /api/enrichment/* routes — status, kick/cancel, per-song state, the
Match-Review queue (accept/reject/pick/search/rematch/refresh/states), and the
AcoustID fingerprint identify endpoints — plus the route-exclusive candidate
sanitizer move to lib/routers/enrichment.py. Bodies verbatim except @app->@router
and the seam reads (meta_db->appstate.meta_db, CONFIG_DIR->appstate.config_dir).
The enrichment engine (transport, matcher, worker, upload caps) already lives in
lib/enrichment.py from the earlier subsystem move and is reached as enrichment.X.
No new seams.

server.py: 2,925 -> 2,638 (-287).

Verified: pyflakes clean; route set identical (143); full pytest 2396 passed (the
enrichment route + Match-Review + identify cases, which fake the network on the
enrichment module). eslint 0.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:19:34 +02:00
Byron GamatosandGitHub f00ba2217d Revert "feat(audio): loopback feeder mode — all app audio under exclusive/ASIO (#865)" (#867)
This reverts commit 9a58a55fe8.
2026-07-11 14:56:35 +02:00
f09c4a217f refactor(server): extract library + collections routes + the provider registry (R3) (#866)
The library query surface (songs/albums/artists/stats/genres/tuning-names/
practice-suggestions), the provider list/art/sync endpoints, and collection CRUD
move to lib/routers/library.py. The registry itself — LibraryProviderRegistry,
LocalLibraryProvider, SmartCollectionProvider, the collection-provider lifecycle
(_sync/_unregister_collection_provider), and the shared query/collection helpers
(_library_filter_args, _split_csv, _sanitize_collection_rules, _safe_art_redirect_url,
the filter-key sets) — moves to lib/library_registry.py.

The PLUGIN CONTRACT is untouched: server.py still constructs the singleton
(LocalLibraryProvider needs meta_db), still exposes register_library_provider /
unregister_library_provider to plugins via plugin_context (with the per-plugin
ownership scoping in plugins/__init__.py), and injects the registry + local
provider into appstate. The router reads appstate.library_providers /
appstate.local_library_provider at call time; the provider classes are duck-typed
so no plugin imports a base class. Acyclic: library_registry imports
routers.art (for LocalLibraryProvider.get_art) + appstate, never server.

server.py: 3,692 -> 2,925 (-767).

Verified: pyflakes clean; ORDERED route table identical set (library block mounts
at one site — all exact/specific-path, no catch-all, no shadowing); full pytest
2397 passed (incl the plugin register/unregister + collection-as-provider tests).
eslint 0. Boot smoke: /api/library + providers list "local"; a created collection
surfaces as a `collection:N` provider through the seam; collection CRUD clean.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:56:12 +02:00
9a58a55fe8 feat(audio): loopback feeder mode — all app audio under exclusive/ASIO (#865)
* feat(audio): loopback feeder mode — all app audio under exclusive/ASIO

Tester-confirmed (2026-07-11 log): song previews and other
plugin-private audio bypass the per-surface feeder taps and leak to the
default WASAPI device under ASIO output. Also confirmed: the element
capture path poisons itself when highway_3d already owns #audio's
one-shot MediaElementSource (InvalidStateError with _elCtx assigned
pre-throw → TypeError every later tick).

- New preferred mode 'loopback': one getDisplayMedia frame-audio capture
  (desktop main answers with the app's own frame) covers song, previews,
  and UI sounds for the whole exclusive session — engages even with no
  song loaded. Local playback silenced via suppressLocalAudioPlayback,
  page-mute IPC fallback otherwise.
- Sticky fallback to the existing stems/element surface modes when
  capture is unavailable (old desktop main, denied, Docker sphere).
- Element capture: assign module state only after the whole chain
  succeeds; close the context on failure — collision now retries clean.
- Failed engage now disables the bus and tears down loopback (no more
  bus-enabled-with-no-producer stranding).
- Tests: 12 (5 new — loopback engage/preference/mute-fallback/sticky
  fallback, collision retry).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(audio): close loopback capture context on teardown (release tap worklet)

The loopback context was reused across engages (_lbCtx || new), but teardown
only stopped the stream + deactivated the tap — never closing the context or
detaching the worklet node. Each exclusive<->shared switch orphaned a live
tap worklet on the long-lived context. Use a fresh context per session and
close it on disengage. Adds a test asserting the context is closed on teardown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Byron Gamatos <xasiklas@gmail.com>
2026-07-11 14:54:40 +02:00
bbdff4e10f refactor(server): extract the song routes into routers/song.py (R3) (#864)
Upload/delete, the catalog-metadata write-back, user-meta, overrides, gap-fill,
and the per-song info payload (11 routes) move to lib/routers/song.py with their
exclusive helpers (the atomic upload commit + the song-IO lock, the upload caps,
the gap-fill proposal builders). Bodies verbatim except @app->@router and the
seam reads: meta_db->appstate.meta_db, art_override_paths->appstate.art_override_paths,
and the scan/ingest helpers that stay in server.py (the scan lifecycle owns them)
-> new appstate seam callables: kick_scan, invalidate_song_caches, stat_for_cache,
and scan_status() (a getter — the underlying dict is reassigned). The gap-fill
MBID/ISRC regexes are reached as enrichment.X; _MULTIPART_OVERHEAD_SLACK (shared
with the staying AcoustID-identify route) moves to lib/enrichment.py beside
_ACOUSTID_MAX_UPLOAD_BYTES.

ROUTE ORDER: song_router mounts AFTER art_router — get_song_info's catch-all
`/api/song/{filename:path}` would otherwise shadow `/api/song/{path}/art*`
(Starlette matches first-registered; the :path converter is greedy).

server.py: 4,478 -> 3,692 (-786).

Verified: pyflakes clean; ORDERED route table preserves the specific-before-catch-all
invariant; full pytest 2397 passed (incl the art/cover 304 + CAA-fetch tests that
caught the shadowing before it was fixed). eslint 0.

BEHAVIORAL — needs an on-device pass (upload a sloppak, edit metadata write-back,
delete a song) before merge.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 13:14:52 +02:00
7258e1066a refactor(server): extract the settings routes into routers/settings.py (R3) (#863)
GET/POST /api/settings, /api/settings/reset, and the two-phase atomic
export/import bundle (/api/settings/export|import) move to lib/routers/settings.py
with their exclusive helpers (the relpath allowlist validator, the atomic writer,
the library-DB snapshot + sqlite integrity gate, the config-type validator, the
bundle schema). Bodies verbatim except @app->@router and the seam reads:
meta_db->appstate.meta_db, CONFIG_DIR->appstate.config_dir,
_running_version->appstate.running_version(), and _default_settings->
appstate.default_settings (the canonical defaults builder stays in server.py —
the scan + artist-links code share it — and is injected as a new seam callable).

server.py: 5,539 -> 4,478 (-1,061).

Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2397 passed (154 settings cases incl the
export→import round-trip + library-DB snapshot/restore + relpath-allowlist SSRF/
traversal guards, retargeted onto the settings module). eslint 0.

BEHAVIORAL — needs an on-device settings export→import round-trip sign-off before
merge (do not merge on green CI alone).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 12:35:05 +02:00
73127d5416 refactor(server): extract the album-art routes into routers/art.py (R3) (#862)
The six song-art routes — GET /api/song/{f}/art, .../art/cover-search,
.../art/candidates, POST .../art/upload, .../art/url, DELETE /api/art/{f}/override
— plus their exclusive helpers (the ETag/304 response machinery, _save_art_override,
_url_host_is_internal, _fetch_art_url + the art size/redirect caps) move to
lib/routers/art.py. Bodies verbatim except @app->@router and the seam reads:
meta_db->appstate.meta_db, ART_CACHE_DIR->appstate.art_cache_dir, and the three
shared art helpers that stay in server.py (used by the song/delete routes too)
-> appstate.<callable> (_song_pack_art_exists, _art_override_paths — already
seam-injected for the enrichment worker — plus a new art_safe_name slot). The
CAA / release-search transport lives in lib/enrichment.py and is reached as
enrichment.X. LocalLibraryProvider.get_art now calls art_router.get_song_art.

server.py: 5,988 -> 5,540 (-448).

Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2399 passed (33 art serve/candidates/
override/url cases incl the SSRF-guard _url_host_is_internal + _fetch_art_url
size-cap tests, retargeted onto the art module); test_packaging 43; eslint 0.
Boot smoke: /art 404, /art/candidates 404, DELETE /override 200 from the router.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:56:53 +02:00
165475d115 refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3) (#861)
* refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3)

MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and
the background enrichment worker (~930 lines, 61 defs) leave server.py as one
cohesive unit. Bodies are verbatim; the only changes are seam reads:

  meta_db / config_dir / sloppak_cache_dir / art_cache_dir -> appstate.<slot>
  _song_pack_art_exists / _art_override_paths (stay in server.py for the art +
    delete routes) -> appstate.<callable> (new seam slots, injected by reference)
  _env_flag -> env_compat.env_flag_compat (the existing identical helper)
  _artist_title_from_filename -> imported from metadata_db (its home)
  the User-Agent VERSION lookup: Path(__file__).parent ->
    Path(__file__).resolve().parents[1] (lib/enrichment.py -> app root)

server.py drives the worker through the module (import enrichment; the routes +
scan lifecycle call enrichment.X). Tests that faked the network on `server`
(_mb_http_get, _enrich_network_enabled, _caa_http_get, ...) now patch the same
names on `enrichment` — the module attribute is resolved at call time, so one
setattr reaches both the routes and the worker's internal callers. Acyclic:
enrichment imports appstate/appconfig/dlc_paths/metadata_db/mb_match/
acoustid_match/sloppak/loosefolder, never server.

server.py: 6,917 -> 5,988 (-929).

Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2400 passed (140 enrichment/art cases
incl the offline-safety + transport-error-pauses-pass contracts that fake the
network); test_packaging 44 passed (enrichment.py resolves under lib/); eslint 0.
Boot smoke: /api/enrichment/status + POST /kick serve from the new module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: reset enrichment worker state between tests (CodeRabbit)

lib/enrichment.py now owns the worker, and it stays imported for the whole
session while the `server` fixtures pop-and-reimport `server` — so the cancel
Event / status dict / caches would leak across tests, and a stale `_enrich_cancel`
could short-circuit a later direct `_background_enrich()`. An autouse conftest
fixture clears that process-global state before each test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: tighten the enrichment-reset fixture (CodeRabbit)

Narrow the import guard to ImportError (not blind Exception, BLE001), and stop
clearing _caa_index_locks — it's guarded by _caa_index_locks_guard, so an
unlocked clear() would race a still-alive worker, and its per-release mutexes
carry no test state.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:32:21 +02:00
508829c012 refactor(server): extract /api/tunings into routers/tunings.py + lib/appconfig.py (R3) (#858)
ship-ci / ci (push) Waiting to run
The merged-tuning-catalog route moves to lib/routers/tunings.py, verbatim except
@app->@router, CONFIG_DIR->appstate.config_dir, and the two seam substrates it
needed:

  - lib/appconfig.py — the pure config.json reader `_load_config` (used by ~11
    server sites + future config-reading routers). server.py re-imports it, so
    those call sites and any `server._load_config` test reference are unchanged.
  - appstate.tuning_providers — the TuningProviderRegistry instance injected by
    reference (a stable object mutated in place via register()/unregister()), so
    the router reads the same registry plugins populate through plugin_context.
    The instance stays defined in server.py, so `server.tuning_providers` still
    resolves — zero test retargets.

The tuning constants (DEFAULT_REFERENCE_PITCH, TUNING_PRESET_MIDIS,
freqs_to_midis) already live in lib/tunings.py and are imported directly.

server.py: 6,960 -> 6,917.

Verified: pyflakes clean (bar the pre-existing unused `tuning_name` import);
route table IDENTICAL (143); full pytest 2400 passed (110 tuning/config cases);
eslint 0. Boot smoke: /api/tunings serves referencePitch + tunings + tuningMidis.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 01:43:19 +02:00
31 changed files with 5609 additions and 4838 deletions
+2 -1
View File
@@ -27,7 +27,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **Perf harness now measures 2D-highway frame time (R3c gate).** `scripts/perf-baseline.mjs` gains a `--song` mode that reports per-frame draw-cost p50/p95/p99 (draw-tagged via `highway.addDrawHook`), the metric that gates the `highway.js` split. Maintainer/CI-only; baseline recorded in `docs/perf-baseline.md`.
- **`routers/` — extracting `server.py`'s route layer, cheapest-first (R3).** Each PR moves a cohesive route group into a `fastapi.APIRouter` under `lib/routers/`, mounted with `app.include_router(...)` at its original site (FastAPI matches in registration order; the full route table stays byte-identical). Bodies are verbatim — only the decorator receiver (`@app` → `@router`) and singleton reads (`meta_db` → `appstate.meta_db`, resolved at call time) change. So far: `audio_effects` (5), `artist_aliases` (5), `loops` (3), `playlists` (12 + covers), `ws_highway` (the 902-line highway chart WebSocket), `chart` (split/unsplit/work/fileinfo — unblocked by the DLC-path substrate), `library_extras`, `wanted`, `shop`, `progression`, `profile`, `stats` (the `/api/stats/{path}` catch-all stays registered last so it can't shadow `/recent` `/best` `/top`), `version` (`/api/version`; VERSION-file lookup adjusted for the router subdir depth), and `diagnostics` (`/api/diagnostics/export|preview|hardware`; the plugins-root lookup adjusted for the router subdir depth, `_running_version` reached through the `appstate` seam, pure payload-cap helpers re-exported for the `server._diag_*` tests). The DLC library-path resolution (`_get_dlc_dir`, pure `_resolve_dlc_path`) moved to `lib/dlc_paths.py`, reading paths through the seam; `config_dir`/`dlc_dir`/`dlc_dir_env` now ride the `appstate` seam (env-derived, so the pop-and-reimport fixtures reconfigure it for free), and the shared request-field sanitizer `_clean_str` moved to `lib/reqfields.py`. The next cut is picked by a dependency-closure scan that ranks groups by how many `monkeypatch.setattr(server, …)` targets they'd drag along.
- **`routers/` — extracting `server.py`'s route layer, cheapest-first (R3).** Each PR moves a cohesive route group into a `fastapi.APIRouter` under `lib/routers/`, mounted with `app.include_router(...)` at its original site (FastAPI matches in registration order; the full route table stays byte-identical). Bodies are verbatim — only the decorator receiver (`@app` → `@router`) and singleton reads (`meta_db` → `appstate.meta_db`, resolved at call time) change. So far: `audio_effects` (5), `artist_aliases` (5), `loops` (3), `playlists` (12 + covers), `ws_highway` (the 902-line highway chart WebSocket), `chart` (split/unsplit/work/fileinfo — unblocked by the DLC-path substrate), `library_extras`, `wanted`, `shop`, `progression`, `profile`, `stats` (the `/api/stats/{path}` catch-all stays registered last so it can't shadow `/recent` `/best` `/top`), `version` (`/api/version`; VERSION-file lookup adjusted for the router subdir depth), `art` (the `/api/song/{f}/art*` serve/cover-search/candidates/upload/url + `/api/art/{f}/override` routes; the shared `_song_pack_art_exists`/`_art_override_paths`/`_art_safe_name` helpers stay in `server.py` for the song/delete routes and are reached through the `appstate` seam, the CAA/release transport as `enrichment.X`), and `settings` (`GET`/`POST /api/settings`, `/reset`, and the two-phase atomic export/import bundle `/api/settings/export|import`; the shared `_default_settings` builder stays in `server.py` and is reached through the `appstate` seam), and `song` (upload/delete + the metadata write-back, user-meta, overrides, gap-fill, and per-song info routes; the scan/ingest helpers stay in `server.py` and are reached through new `appstate` seams — `kick_scan`, `invalidate_song_caches`, `stat_for_cache`, and a `scan_status()` getter — the `get_song_info` catch-all mounts after the art routes so it can't shadow them), and `library` + collections (the provider list/art/sync endpoints, the library query surface, and collection CRUD → `lib/routers/library.py`; the `LibraryProviderRegistry`/`LocalLibraryProvider`/`SmartCollectionProvider` classes + shared query/collection helpers move to `lib/library_registry.py`, and the registry instance + local provider ride the `appstate` seam — server.py still constructs the singleton and exposes `register_library_provider`/`unregister_library_provider` to plugins via `plugin_context` unchanged), and the `enrichment` route handlers (`/api/enrichment/*`: status, kick/cancel, per-song state, the Match-Review queue, and AcoustID identify → `lib/routers/enrichment.py`; the engine already lives in `lib/enrichment.py` and is reached as `enrichment.X`), and `media` (the file-serving routes — song audio `/audio/{f}`, the local-audio-path resolver `/api/audio-local-path`, and raw sloppak-member serving `/api/sloppak/{f}/file/{rel}` → `lib/routers/media.py`; the cache/static path seams were already in `appstate`), and `artist` (the artist page + external-links payload `/api/artist/{name}/page|links|links/refresh` → `lib/routers/artist.py`; MB link enrichment reached as `enrichment.X`), and `diagnostics` (`/api/diagnostics/export|preview|hardware`; the plugins-root lookup adjusted for the router subdir depth, `_running_version` reached through the `appstate` seam, pure payload-cap helpers re-exported for the `server._diag_*` tests), and `tunings` (`/api/tunings`; the pure `config.json` reader moved to `lib/appconfig.py`, the tuning-provider registry read through the `appstate` seam so plugin-contributed tunings still merge). The DLC library-path resolution (`_get_dlc_dir`, pure `_resolve_dlc_path`) moved to `lib/dlc_paths.py`, reading paths through the seam; `config_dir`/`dlc_dir`/`dlc_dir_env` now ride the `appstate` seam (env-derived, so the pop-and-reimport fixtures reconfigure it for free), and the shared request-field sanitizer `_clean_str` moved to `lib/reqfields.py`. The next cut is picked by a dependency-closure scan that ranks groups by how many `monkeypatch.setattr(server, …)` targets they'd drag along.
- **`routers/` — the first extracted route module (R3).** The five audio-effects mapping
endpoints move out of `server.py` into `lib/routers/audio_effects.py` as a
`fastapi.APIRouter`, mounted with `app.include_router(...)` **at the point in the file
@@ -40,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
the second slot. The `_demo_mode_guard` middleware still blocks all four moved write
routes with 403, and `Query(...)` validation still 422s — both checked against a running
server. `server.py`: **9,445 → 9,386 lines**.
- **`lib/enrichment.py` — the metadata-enrichment subsystem leaves `server.py` (R3, move-only).** MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and the background enrichment worker (~930 lines, 61 defs) move out as one cohesive unit. Bodies are verbatim; the only changes are seam reads — `meta_db`/`config_dir`/`sloppak_cache_dir`/`art_cache_dir` and the two shared art helpers (`song_pack_art_exists`, `art_override_paths`, which stay in `server.py` for the art/delete routes) are reached through `appstate` at call time, and the User-Agent VERSION lookup is corrected for the module's new depth. `server.py` drives the worker through the module (`import enrichment`; the routes + scan lifecycle call `enrichment.X`); tests that faked the network on `server` now patch the same names on `enrichment` (module attribute resolved at call time, so one `setattr` reaches both the routes and the worker's internal callers). Acyclic — `enrichment` imports no `server`. Route table byte-identical; full suite green. `server.py`: 6,917 → 5,988.
- **`appstate.py` — the router seam (R3).** Route modules moving out of `server.py`
need `meta_db` and friends but must not `import server`, or the import graph goes
circular the moment `server` imports them back. So `server.py` keeps *constructing*
+2 -2
View File
@@ -55,8 +55,8 @@ without a *signed* exemption" is unenforceable.
## Planned, NOT exempt (owned by split plans — listed so nothing falls between states)
core `static/app.js` (11,852) · `static/highway.js` (4,168, whole file) · `server.py`
(6,960 — was 14,037; ratcheted by the R3 `MetadataDB` + `AudioEffectsMappingDB`
extractions and fourteen `routers/` modules) ·
(2,413 — was 14,037; ratcheted by the R3 `MetadataDB` + `AudioEffectsMappingDB`
extractions and twenty-two `routers/` modules, plus lib/library_registry.py for the provider-registry classes (album-art in `lib/routers/art.py`, the settings + export/import bundle in `lib/routers/settings.py`); the ~930-line metadata-enrichment subsystem — MB/CAA/AcoustID transport, matcher, background worker — now lives in `lib/enrichment.py`) ·
`lib/metadata_db.py` (4,373 — new in R3; the `MetadataDB` class alone is 4,018 lines
and is a monolith in its own right, to be split per-table once the router train
lands) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
+28
View File
@@ -0,0 +1,28 @@
"""Reading the app's config.json — the one shared, pure helper (R3).
Extracted verbatim from server.py so route modules that need a config value
(reference pitch, server_config, …) can read it without reaching back into the
host file. server.py re-imports it, so its ~11 call sites and any
`server._load_config` test reference keep resolving unchanged.
"""
import json
def _load_config(config_file):
"""Read and parse config.json. Returns the parsed dict, or None if
the file is missing, unreadable, invalid JSON, or parses to a
non-dict (e.g. the file contains `[]` or `42`). Callers treat None
as "fall back to defaults". Shared between GET and POST so both
handle bad files the same way."""
if not config_file.exists():
return None
try:
# Explicit UTF-8: save_settings()/import write config.json as
# UTF-8 bytes, so the read must not depend on the platform's
# default text encoding (cp1252 on Windows would mojibake or
# UnicodeDecodeError on a non-ASCII DLC path).
parsed = json.loads(config_file.read_text(encoding="utf-8"))
except Exception:
return None
return parsed if isinstance(parsed, dict) else None
+33 -1
View File
@@ -61,6 +61,16 @@ copies a hardcoded file list — that regression is what moved this file here.
# The singletons routers may read. Every name here must also be a `_SLOTS` key.
meta_db = None
audio_effect_mappings = None
# The tuning-provider registry instance (built-ins + plugin-contributed). A
# stable object mutated in place via register()/unregister() — injected here by
# reference so routers read the same registry plugins populate.
tuning_providers = None
# The library-provider registry instance + the local provider, constructed in
# server.py (LocalLibraryProvider needs meta_db) and injected by reference. The
# classes live in lib/library_registry.py; plugins register their own providers
# through the registry via plugin_context.
library_providers = None
local_library_provider = None
# Config paths. server.py derives these from the environment (fresh on every
# import, so the ~49 pop-and-reimport fixtures keep working) and injects them
@@ -88,13 +98,35 @@ audio_cache_dir = None
get_progression_content = None
builtin_diagnostic_filename = None
running_version = None
# Art helpers that stay in server.py (shared with the art/delete routes) but are
# also called by the enrichment worker in lib/enrichment.py — injected as
# callables to keep enrichment acyclic. art_cache_dir is server's ART_CACHE_DIR.
art_cache_dir = None
song_pack_art_exists = None
art_override_paths = None
art_safe_name = None
# The canonical settings-defaults builder — stays in server.py (shared with the
# scan/artist-links code) but the settings router calls it through the seam.
default_settings = None
# Scan/ingest seam for the song routes (routers/song.py). kick_scan/
# invalidate_song_caches/stat_for_cache stay in server.py (scan lifecycle owns
# them); scan_status is a GETTER (the underlying dict is reassigned, so a value
# would go stale) — call appstate.scan_status() to read the live status.
kick_scan = None
invalidate_song_caches = None
stat_for_cache = None
scan_status = None
_SLOTS = frozenset({
"meta_db", "audio_effect_mappings",
"meta_db", "audio_effect_mappings", "tuning_providers",
"library_providers", "local_library_provider",
"config_dir", "dlc_dir", "dlc_dir_env",
"static_dir", "sloppak_cache_dir", "audio_cache_dir",
"get_progression_content", "builtin_diagnostic_filename",
"running_version",
"art_cache_dir", "song_pack_art_exists", "art_override_paths", "art_safe_name",
"default_settings",
"kick_scan", "invalidate_song_caches", "stat_for_cache", "scan_status",
})
+1107
View File
File diff suppressed because it is too large Load Diff
+417
View File
@@ -0,0 +1,417 @@
"""The library-provider registry — the plugin extension point for song sources.
`LocalLibraryProvider` wraps the local `MetadataDB`; third-party plugins register
their own providers (duck-typed: any object with the advertised methods) through
`LibraryProviderRegistry`, and smart collections are surfaced as
`SmartCollectionProvider`s over the local one. server.py constructs the singleton
(`library_providers`), injects it + the local provider into appstate, and exposes
`register_library_provider`/`unregister_library_provider` to plugins via
plugin_context (with per-plugin ownership scoping in plugins/__init__.py).
Moved verbatim out of server.py (R3). The shared query/collection helpers live
here too so routers/library.py can import them without reaching into server.
"""
import re
import threading
from typing import ClassVar
import appstate
from metadata_db import MetadataDB, _tuning_group_key_sql
from routers import art as art_router
import logging
log = logging.getLogger("feedBack.server")
def _safe_art_redirect_url(url: str) -> str | None:
"""Return the URL if it is safe to redirect to (http/https only), else None."""
from urllib.parse import urlparse
if not url or not isinstance(url, str):
return None
try:
parsed = urlparse(url)
if parsed.scheme.lower() not in ("http", "https"):
return None
if not parsed.hostname:
return None
return url
except Exception:
return None
_TUNING_GROUP_KEY_SQL = _tuning_group_key_sql("songs")
class LocalLibraryProvider:
id = "local"
label = "My Library"
kind = "local"
capabilities = (
"library.read",
"art.read",
"song.play",
"favorite.write",
"metadata.write",
)
def __init__(self, db: MetadataDB):
self._db = db
def query_page(self, **kwargs) -> tuple[list[dict], int]:
return self._db.query_page(**kwargs)
def query_artists(self, **kwargs) -> tuple[list[dict], int]:
return self._db.query_artists(**kwargs)
def query_albums(self, **kwargs) -> tuple[list[dict], int]:
return self._db.query_albums(**kwargs)
def query_stats(self, **kwargs) -> dict:
return self._db.query_stats(**kwargs)
def tuning_names(self) -> dict:
# Group custom tunings on their raw offsets so distinct ones stay
# distinct (tuning_name collapses them all to "Custom Tuning"); named
# tunings keep grouping by name (stable across the rescan boundary, no
# offsets/name split). `key` is the value the client sends back as the
# filter selector — equal to the name for named tunings, the offsets
# string for customs; offsets also feed the client's custom-pill label.
with self._db._lock:
rows = self._db.conn.execute(
f"SELECT tuning_name, {_TUNING_GROUP_KEY_SQL} AS gkey, "
"MIN(tuning_sort_key), COUNT(*), MIN(tuning_offsets) "
"FROM songs WHERE title != '' AND COALESCE(tuning_name, '') != '' "
"GROUP BY gkey COLLATE NOCASE "
"ORDER BY ABS(COALESCE(MIN(tuning_sort_key), 0)), "
"COALESCE(MIN(tuning_sort_key), 0) ASC, "
"tuning_name COLLATE NOCASE"
).fetchall()
return {
"tunings": [
{"name": name, "key": gkey, "offsets": offs or "",
"sort_key": int(sk or 0), "count": count}
for name, gkey, sk, count, offs in rows
],
}
async def get_art(self, song_id: str):
return await art_router.get_song_art(song_id)
class LibraryProviderRegistry:
# Methods required per declared capability — only validated when the
# provider advertises the corresponding capability so action-only providers
# (e.g. art.read + song.sync without library.read) don't need to implement
# unused stubs.
_CAPABILITY_METHODS: ClassVar[dict[str, tuple[str, ...]]] = {
"library.read": ("query_page", "query_artists", "query_stats", "tuning_names"),
"art.read": ("get_art",),
"song.sync": ("sync_song",),
}
_ID_RE: ClassVar[re.Pattern[str]] = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}$")
def __init__(self):
self._providers: dict[str, object] = {}
# Capabilities inferred at registration for legacy providers that omit
# the `capabilities` field. Merged with provider_capabilities() so that
# runtime capability checks see the complete effective capability set.
self._inferred_caps: dict[str, set[str]] = {}
self._owner_plugin_ids: dict[str, str] = {}
self._lock = threading.RLock()
def register(self, provider: object, *, replace: bool = False, owner_plugin_id: str | None = None) -> object:
provider_id = self.provider_id(provider)
if not self._ID_RE.match(provider_id):
raise ValueError(
"library provider id must start with an alphanumeric character "
"and contain only letters, digits, _, ., :, or -"
)
if not self.provider_label(provider):
raise ValueError("library provider label must be a non-empty string")
# Use declared-only caps during validation — never include stale inferred
# caps from a previous provider registered under the same id (replace=True).
caps = self._declared_capabilities(provider)
# Backward compatibility: providers that predate explicit capability
# declarations may omit `capabilities` entirely. If the browse methods
# are all present, infer `library.read` so they still work unchanged.
# If capabilities are absent but the browse surface is also absent,
# raise a clear error rather than letting the provider register and
# then fail on every API call with a late 501.
inferred: set[str] = set()
if not caps:
browse_methods = self._CAPABILITY_METHODS["library.read"]
if all(callable(self.provider_method(provider, m)) for m in browse_methods):
# Legacy provider without explicit capabilities — infer library.read
# from the presence of all browse methods. Store in _inferred_caps
# so that runtime capability checks see the full effective set.
inferred = {"library.read"}
caps = inferred
else:
raise TypeError(
f"library provider {provider_id!r} must declare at least one capability "
f"(or implement the {browse_methods!r} browse methods for backward compatibility)"
)
for cap, methods in self._CAPABILITY_METHODS.items():
if cap not in caps:
continue
for method_name in methods:
if not callable(self.provider_method(provider, method_name)):
raise TypeError(f"library provider {provider_id!r} declares {cap!r} but is missing callable {method_name}()")
with self._lock:
if provider_id == "local" and provider_id in self._providers and self._providers[provider_id] is not provider:
raise ValueError("the local library provider cannot be replaced")
if provider_id in self._providers and not replace:
raise ValueError(f"library provider {provider_id!r} is already registered")
self._providers[provider_id] = provider
# owner_plugin_id is attribution that flows into the browser
# capability participant id. The scoped register_library_provider
# wrappers force it to the trusted loading plugin id, so the spoof
# vector is closed there. Here we only normalize: trim and require a
# non-empty string. We deliberately do NOT apply the provider-id
# grammar (_ID_RE) — plugin ids aren't constrained to it at load
# time, so that would silently drop attribution for valid plugins.
owner = owner_plugin_id.strip() if isinstance(owner_plugin_id, str) else ""
owner = owner or None
if owner:
self._owner_plugin_ids[provider_id] = owner
else:
self._owner_plugin_ids.pop(provider_id, None)
if inferred:
self._inferred_caps[provider_id] = inferred
else:
self._inferred_caps.pop(provider_id, None)
return provider
def unregister(self, provider_id: str) -> bool:
if provider_id == "local":
raise ValueError("the local library provider cannot be unregistered")
with self._lock:
self._inferred_caps.pop(provider_id, None)
self._owner_plugin_ids.pop(provider_id, None)
return self._providers.pop(provider_id, None) is not None
def get(self, provider_id: str = "local") -> object | None:
with self._lock:
return self._providers.get(provider_id or "local")
def list(self) -> list[dict]:
with self._lock:
providers = list(self._providers.values())
return [self.describe(provider) for provider in providers]
def describe(self, provider: object) -> dict:
provider_id = self.provider_id(provider)
with self._lock:
owner_plugin_id = self._owner_plugin_ids.get(provider_id)
return {
"id": provider_id,
"label": self.provider_label(provider),
"kind": self.provider_field(provider, "kind", "local" if provider_id == "local" else "remote"),
"capabilities": sorted(self.provider_capabilities(provider)),
"owner_plugin_id": owner_plugin_id,
"default": provider_id == "local",
}
def provider_field(self, provider: object, name: str, default=None):
if isinstance(provider, dict):
return provider.get(name, default)
return getattr(provider, name, default)
def provider_id(self, provider: object) -> str:
provider_id = self.provider_field(provider, "id", "")
if not isinstance(provider_id, str) or not provider_id:
raise ValueError("library provider id must be a non-empty string")
return provider_id
def provider_label(self, provider: object) -> str:
label = self.provider_field(provider, "label", self.provider_field(provider, "name", ""))
if not isinstance(label, str):
return ""
return label.strip()
def _declared_capabilities(self, provider: object) -> set[str]:
"""Return only the capabilities explicitly declared on the provider object."""
raw = self.provider_field(provider, "capabilities", ())
if raw is None:
raw = ()
if isinstance(raw, str):
raw = (raw,) if raw else ()
return {str(cap) for cap in raw if cap}
def provider_capabilities(self, provider: object) -> set[str]:
# Guard against a common plugin authoring mistake: passing a single string
# instead of a list/tuple. Iterating a string produces individual characters,
# none of which would match a valid capability name.
declared = self._declared_capabilities(provider)
# Merge with any capabilities inferred at registration time for legacy
# providers that omit the `capabilities` field but implement browse methods.
provider_id = self.provider_id(provider)
with self._lock:
inferred = self._inferred_caps.get(provider_id, set())
return declared | inferred
def provider_method(self, provider: object, name: str):
if isinstance(provider, dict):
return provider.get(name)
return getattr(provider, name, None)
# Keys `_library_filter_args` (and a smart collection's stored `rules`) accept.
_LIBRARY_FILTER_PARAM_KEYS = frozenset((
"q", "favorites", "format", "artist", "album",
"arrangements_has", "arrangements_lacks", "stems_has", "stems_lacks",
"has_lyrics", "tunings",
))
# Rules mirror the raw /api/library query params (so the provider can feed them
# straight through `_library_filter_args`, and the frontend can build a rule from
# the same query string it already constructs). Multi-value filters are CSV
# strings; `favorites` is 0/1; the rest are plain strings.
_RULE_CSV_KEYS = frozenset((
"tunings", "arrangements_has", "arrangements_lacks", "stems_has", "stems_lacks",
))
_RULE_STR_KEYS = frozenset(("q", "format", "artist", "album", "has_lyrics", "sort"))
def _sanitize_collection_rules(raw) -> dict:
"""Normalize rules to the raw query-param format, keeping only known keys. A
list for a multi-value filter is joined to CSV; `favorites` becomes 0/1.
Unknown keys are dropped so a rule survives a filter-vocab change rather than
500-ing. Applied at API ingress AND when a provider loads a persisted row, so
a hand-edited / imported bad value (e.g. an int where a string is expected,
or a list for `sort`) can never crash a query."""
if not isinstance(raw, dict):
return {}
out: dict = {}
for k, v in raw.items():
if k in _RULE_CSV_KEYS:
if isinstance(v, list):
vals = [str(x) for x in v if isinstance(x, (str, int)) and not isinstance(x, bool)]
elif isinstance(v, str):
vals = [s for s in (p.strip() for p in v.split(",")) if s]
else:
continue
if vals:
out[k] = ",".join(vals)
elif k == "favorites":
if v:
out[k] = 1
elif k in _RULE_STR_KEYS:
if isinstance(v, (str, int)) and not isinstance(v, bool):
s = str(v).strip()
if s:
out[k] = s
return out
class SmartCollectionProvider:
"""A saved library filter, surfaced as a source (#636 item 2). Browse/stats
delegate to the local DB with the collection's stored `rules` applied — so
selecting it in the v3 source picker shows exactly that filtered slice with
the whole Songs UI (paging, stats, AZ rail, art) for free. P1: the rules
ARE the query (live in-collection search is a P2 nicety). The matched songs
are local rows, so `kind="local"` keeps the client's play/art paths on the
local (not remote-sync) branch and art delegates straight through."""
kind = "local"
capabilities = ("library.read", "art.read")
def __init__(self, collection: dict, local: "LocalLibraryProvider"):
self._local = local
self.update(collection)
def update(self, collection: dict) -> None:
self.id = f"collection:{collection['id']}"
self.collection_id = collection["id"]
self.label = collection.get("name") or "Collection"
# Re-sanitize on load: persisted JSON may predate the current vocab or
# have been hand-edited; never let a bad value reach a query.
self._rules = _sanitize_collection_rules(collection.get("rules") or {})
def _filter_kwargs(self) -> dict:
return _library_filter_args(**{k: v for k, v in self._rules.items()
if k in _LIBRARY_FILTER_PARAM_KEYS})
def _sort(self, fallback: str) -> str:
# A collection may pin its own sort (e.g. "recently added"); query_page
# falls back safely for an unknown value, so no validation needed here.
return self._rules.get("sort") or fallback
def query_page(self, *, page=0, size=24, sort="artist", direction="asc",
naming_mode="legacy", **_ignore):
return self._local._db.query_page(
page=page, size=size, sort=self._sort(sort), direction=direction,
naming_mode=naming_mode, **self._filter_kwargs())
def query_artists(self, *, letter="", page=0, size=50, naming_mode="legacy", **_ignore):
return self._local._db.query_artists(
letter=letter, page=page, size=size, naming_mode=naming_mode,
**self._filter_kwargs())
def query_albums(self, *, page=0, size=120, naming_mode="legacy", **_ignore):
return self._local._db.query_albums(
page=page, size=size, naming_mode=naming_mode, **self._filter_kwargs())
def query_stats(self, *, sort="artist", want_sort_letters=False,
naming_mode="legacy", **_ignore):
return self._local._db.query_stats(
sort=self._sort(sort), want_sort_letters=want_sort_letters,
naming_mode=naming_mode, **self._filter_kwargs())
def tuning_names(self):
return self._local.tuning_names()
async def get_art(self, song_id: str):
return await self._local.get_art(song_id)
def _split_csv(raw: str) -> list[str]:
"""Parse a comma-separated query-string list. Empty / whitespace-only
entries are dropped so `arrangements_has=` (no value) and
`arrangements_has=,` both mean 'no filter'."""
if not raw:
return []
return [s.strip() for s in raw.split(",") if s.strip()]
def _parse_has_lyrics(raw: str) -> int | None:
"""Tri-state parse for has_lyrics. `1` → require, `0` → exclude,
anything else (including empty) → no filter."""
if raw == "1":
return 1
if raw == "0":
return 0
return None
def _library_filter_args(q: str = "", favorites: int = 0, format: str = "",
artist: str = "", album: str = "",
arrangements_has: str = "", arrangements_lacks: str = "",
stems_has: str = "", stems_lacks: str = "",
has_lyrics: str = "", tunings: str = "") -> dict:
fmt = format if format in ("archive", "sloppak", "loose") else ""
return {
"q": q,
"favorites_only": bool(favorites),
"format_filter": fmt,
"artist_filter": (artist or "").strip(),
"album_filter": (album or "").strip(),
"arrangements_has": _split_csv(arrangements_has),
"arrangements_lacks": _split_csv(arrangements_lacks),
"stems_has": _split_csv(stems_has),
"stems_lacks": _split_csv(stems_lacks),
"has_lyrics": _parse_has_lyrics(has_lyrics),
"tunings": _split_csv(tunings),
}
def _sync_collection_provider(collection: dict) -> None:
"""Register (or replace) the provider for one collection."""
appstate.library_providers.register(
SmartCollectionProvider(collection, appstate.local_library_provider), replace=True)
def _unregister_collection_provider(pid: int) -> None:
appstate.library_providers.unregister(f"collection:{pid}")
+513
View File
@@ -0,0 +1,513 @@
"""Album-art routes: serve / cover-search / candidates / upload / url / remove
(/api/song/{filename}/art*, /api/art/{filename}/override).
Extracted verbatim from server.py (R3). Only the decorators (@app -> @router) and
the seam reads change: meta_db -> appstate.meta_db, ART_CACHE_DIR ->
appstate.art_cache_dir, and the three shared art helpers that stay in server.py
(they are also used by the song/delete routes) -> appstate.<callable>
(_song_pack_art_exists, _art_override_paths, _art_safe_name). The CAA / release
search transport lives in lib/enrichment.py and is reached as enrichment.X.
"""
import asyncio
import hashlib
import ipaddress
from pathlib import Path
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, JSONResponse, Response
import appstate
import enrichment
import loosefolder as loosefolder_mod
import sloppak as sloppak_mod
from dlc_paths import _get_dlc_dir, _resolve_dlc_path
import logging
log = logging.getLogger("feedBack.server")
router = APIRouter()
def _if_none_match_hits(header: str | None, etag: str) -> bool:
"""True if an If-None-Match header matches `etag` (weak comparison).
Handles the `*` wildcard and comma-separated lists, and ignores a weak
`W/` prefix on either side — the standard semantics for a conditional GET.
"""
if not header:
return False
bare = etag.removeprefix("W/")
for tok in header.split(","):
t = tok.strip()
if t == "*" or t.removeprefix("W/") == bare:
return True
return False
# Album art is served with a strong validator (an ETag on the sloppak byte
# path; FileResponse's own ETag/Last-Modified on the file paths) and revalidated
# with `no-cache`. That keeps re-scroll cheap — a conditional GET returns a
# bodyless 304 — without ever serving a stale cover. A long `immutable` max-age
# was rejected: the frontend's `?v=<mtime>` buster is only second-resolution, so
# a same-second cover rewrite would keep the URL and pin the old bytes for the
# cache lifetime. Validation cost is negligible for a localhost backend.
_ART_CACHE_HEADERS = {"Cache-Control": "no-cache"}
def _art_etag(path: Path) -> str | None:
"""Strong validator for an art file: nanosecond mtime + size (so a
same-second rewrite still changes it). None if the file can't be stat'd."""
try:
st = path.stat()
return f'"{st.st_mtime_ns}-{st.st_size}"'
except OSError:
return None
def _art_conditional(etag: str | None, request: Request | None):
"""Return (headers, not_modified) for an art response. `not_modified` is
True when the client's If-None-Match already matches `etag` → caller should
return a bodyless 304. Starlette's FileResponse emits an ETag but does NOT
itself evaluate If-None-Match, so every art path routes through here to get
real conditional handling."""
headers = dict(_ART_CACHE_HEADERS)
if etag:
headers["ETag"] = etag
inm = request.headers.get("if-none-match") if request is not None else None
return headers, bool(etag) and _if_none_match_hits(inm, etag)
def _file_art_response(path: Path, media_type: str, request: Request | None):
"""FileResponse for an on-disk art file, with no-cache + ETag and a bodyless
304 when the client's validator still matches."""
headers, not_modified = _art_conditional(_art_etag(path), request)
if not_modified:
return Response(status_code=304, headers=headers)
return FileResponse(str(path), media_type=media_type, headers=headers)
@router.get("/api/song/{filename:path}/art")
async def get_song_art(filename: str, request: Request = None, source: str = ""):
"""Serve album art for a song, walking the R3 override chain:
1. USER OVERRIDE (upload / URL-fetch, {safe_name}.gif|.png in the art
cache) — art the user explicitly pinned outranks everything, pack
art included. GIF is allowed HERE only: an animated cover is a
local-only bonus; packs stay jpg/png/webp and nothing ever writes
art into a pack file.
2. PACK ART — sloppak cover (single member read, no full unpack) or
the loose folder's discovered image.
3. COVER ART ARCHIVE cache — fetched by the enrichment art worker for
matched songs that lack pack art, keyed by release MBID.
`?source=pack` narrows the chain to step 2 only (no override, no CAA):
the cover picker's "Pack original" tile must show the pack's own art
even while a user override is what the plain route serves. 404 when the
song ships no art of its own.
"""
dlc = _get_dlc_dir()
if not dlc:
return JSONResponse({"error": "not configured"}, 404)
song_path = _resolve_dlc_path(dlc, filename)
if song_path is None:
return JSONResponse({"error": "forbidden"}, 403)
if not song_path.exists():
return JSONResponse({"error": "not found"}, 404)
pack_only = source == "pack"
# 1. User override — GIF first (it wins over a stale PNG override).
if not pack_only:
for cached in appstate.art_override_paths(filename):
mt = "image/gif" if cached.suffix == ".gif" else "image/png"
return _file_art_response(cached, mt, request)
# 2a. Sloppak: read the cover (manifest-declared or default) straight from
# the package. For a zip-form sloppak this opens just the cover member —
# NOT the whole archive — so the library grid never triggers a full unpack
# of stems just to paint a thumbnail.
if sloppak_mod.is_sloppak(song_path):
# Read the cover (cheap — single member, no full unpack) and validate by
# its CONTENT. A stat-based ETag would be wrong for directory-form
# sloppaks: editing cover.jpg in place changes the file's mtime, not the
# directory's, so a dir-stat ETag could emit a stale 304. Content hashing
# is correct for both dir- and zip-form. Raw byte Response lacks
# FileResponse's validators, so we attach the ETag + honor If-None-Match.
try:
art = await asyncio.to_thread(sloppak_mod.read_cover_bytes, song_path)
except Exception:
art = None
if art is not None:
data, mt = art
etag = f'"{hashlib.sha1(data).hexdigest()}"'
headers, not_modified = _art_conditional(etag, request)
if not_modified:
return Response(status_code=304, headers=headers)
return Response(content=data, media_type=mt, headers=headers)
# 2b. Loose folder: serve the discovered art file directly.
# song_path is already validated against DLC_DIR by _resolve_dlc_path.
elif loosefolder_mod.is_loose_song(song_path):
art_path = loosefolder_mod.find_art(song_path)
if art_path:
# Re-resolve in case the matched file is a symlink — a crafted
# custom song could put `album_art.jpg` as a symlink to anywhere on
# disk. Insist the final target stays inside the song folder.
art_resolved = art_path.resolve()
try:
art_resolved.relative_to(song_path)
except ValueError:
return JSONResponse({"error": "forbidden"}, 403)
if art_resolved.is_file():
mt = {
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp",
}.get(art_resolved.suffix.lower(), "image/jpeg")
return _file_art_response(art_resolved, mt, request)
# 3. Cover Art Archive cache (the enrichment art worker's fetch).
if not pack_only:
row = appstate.meta_db.get_enrichment(filename)
if row and row.get("art_state") == "caa" and row.get("art_cache_path"):
caa = Path(row["art_cache_path"])
if caa.is_file():
return _file_art_response(caa, "image/jpeg", request)
return JSONResponse({"error": "no art"}, 404)
# ── Cover picker (PR-C): candidate assembly ───────────────────────────────────
# Enumerated ON OPEN, never at scan time (charrette §8), and NO image bytes
# are fetched here — Cover Art Archive release INDEX jsons only (1-3 throttled
# calls on a cache miss); the tiles' thumbnails load straight from the archive
# in the client. Applying a pick never grows a new write path: the client
# POSTs the chosen thumb URL to the EXISTING …/art/url route (the override
# lane — never evicted, survives a re-match), "Pack original" DELETEs the
# override, uploads keep the existing upload route.
_ART_PICKER_MAX_CAA = 12
@router.get("/api/song/{filename:path}/art/cover-search")
def api_art_cover_search(filename: str, q: str = ""):
"""Search Cover Art Archive (via MusicBrainz release-groups) for album covers
— powers the Change-cover picker's search box, so a cover can be found even
for a song with no metadata match (the unmatched city-pop pile, where
/art/candidates is empty). `q` defaults to the song's own artist + album/
title (romaji fallback applied). Read-only; the picker renders the thumbs and
applies a pick through the existing /art/url route."""
query = (q or "").strip()
if not query:
pack = appstate.meta_db.pack_fields(appstate.meta_db._canonical_song_filename(filename))
query = " ".join(x for x in (pack.get("artist"), pack.get("album") or pack.get("title")) if x).strip()
if not query:
return {"query": "", "covers": []}
try:
return {"query": query, "covers": enrichment._mb_search_release_groups(query, limit=8)}
except enrichment.EnrichTransportError:
return {"query": query, "covers": [], "error": "unavailable"}
@router.get("/api/song/{filename:path}/art/candidates")
def get_song_art_candidates(filename: str):
"""Everything the cover picker can offer for one song, without fetching a
single image: the current cover (with its provenance), the pack original
when the song ships art, and CAA candidates for the matched/manual
release plus any distinct releases among the stored review candidates.
Sync route on purpose (the CAA index fetch sleeps in the shared
throttle — FastAPI runs `def` routes in the threadpool). One response,
`pending` always False — the client shows a spinner for the request's own
latency; offline / CAA-down just means an empty caa tail (the instant
tiles keep working), never an error."""
from urllib.parse import quote
dlc = _get_dlc_dir()
song_path = _resolve_dlc_path(dlc, filename) if dlc else None
if song_path is None or not song_path.exists():
raise HTTPException(status_code=404, detail="unknown song")
row = appstate.meta_db.get_enrichment(filename) or {}
has_pack = appstate.song_pack_art_exists(filename)
art_url = f"/api/song/{quote(filename)}/art"
# What the plain art route would serve right now — the serve chain's
# order (override > pack > CAA cache) restated as provenance.
if appstate.art_override_paths(filename):
provenance = "yours"
elif has_pack:
provenance = "pack"
elif row.get("art_state") == "caa" and row.get("art_cache_path"):
provenance = "matched"
else:
provenance = "none"
candidates: list[dict] = [{
"id": "current", "kind": "current", "label": "Current",
"thumb_url": art_url, "provenance": provenance,
}]
if has_pack:
candidates.append({
"id": "pack", "kind": "pack", "label": "Pack original",
"thumb_url": art_url + "?source=pack", "provenance": "pack",
})
# Releases worth asking the archive about: the matched/manual release
# first (it seeds the best candidates), then any distinct release among
# the stored review candidates (a review row has no mb_release_id of its
# own — its releases live in the candidates JSON).
# Only spend the shared CAA rate budget on rows whose match warrants it:
# a matched/manual release seeds the best candidates, and a review row's
# stored candidates are still live proposals. A failed/rejected (or
# unscanned) row has no accepted match — asking would burn the budget and
# surface releases already rejected as non-matches. The Current + Pack
# tiles above serve regardless, so those songs still get a picker.
rids: list[str] = []
if row.get("match_state") in ("matched", "manual", "review"):
if row.get("match_state") in ("matched", "manual") and row.get("mb_release_id"):
rids.append(str(row["mb_release_id"]))
for cand in (row.get("candidates") or []):
rid = str(cand.get("release_id") or "") if isinstance(cand, dict) else ""
if rid and rid not in rids:
rids.append(rid)
caa_entries: list[dict] = []
for rid in rids:
if len(caa_entries) >= _ART_PICKER_MAX_CAA:
break
try:
imgs = enrichment._caa_index_cached(rid)
except enrichment.EnrichTransportError:
# Offline / archive down — stop asking (each further miss would
# only burn a timeout). The instant tiles still serve; a later
# picker-open retries naturally (failures are never cached).
break
# Front covers first, approved before pending, otherwise index order
# (the picker grammar is a RANKED list — §7/§9).
def _rank(img):
types = img.get("types") or []
is_front = bool(img.get("front")) or "Front" in types
return (not is_front, not bool(img.get("approved")))
for img in sorted((i for i in imgs if isinstance(i, dict)), key=_rank):
if len(caa_entries) >= _ART_PICKER_MAX_CAA:
break
thumbs = img.get("thumbnails") or {}
if not isinstance(thumbs, dict):
continue
thumb = (thumbs.get("500") or thumbs.get("large")
or thumbs.get("250") or thumbs.get("small"))
if not thumb:
continue
types = [str(t) for t in (img.get("types") or []) if isinstance(t, str)]
caa_entries.append({
"id": f"caa-{rid}-{img.get('id', '')}",
"kind": "caa",
"label": ", ".join(types) or "Cover",
"thumb_url": str(thumb),
"provenance": "matched",
"types": types,
"approved": bool(img.get("approved")),
"release_id": rid,
})
return {"candidates": candidates + caa_entries, "pending": False}
def _save_art_override(filename: str, img_data: bytes) -> dict:
"""Persist a user art override into the art cache (R3). One override per
song: GIF input is validated and kept VERBATIM as .gif (animation intact —
the local-only bonus; it is never written into the pack file), everything
else is normalized to RGB PNG via PIL. Saving either kind removes the
other so the serve chain has exactly one user file to find."""
appstate.art_cache_dir.mkdir(parents=True, exist_ok=True)
stem = appstate.art_safe_name(filename)
png_path = appstate.art_cache_dir / f"{stem}.png"
gif_path = appstate.art_cache_dir / f"{stem}.gif"
from PIL import Image
import io as _io
if img_data[:6] in (b"GIF87a", b"GIF89a"):
try:
probe = Image.open(_io.BytesIO(img_data))
probe.verify() # decodes headers/frames without keeping the image
if probe.format != "GIF":
raise ValueError("not a GIF")
except Exception as e:
return {"error": f"Invalid image: {e}"}
gif_path.write_bytes(img_data)
png_path.unlink(missing_ok=True)
return {"ok": True, "kind": "gif"}
try:
img = Image.open(_io.BytesIO(img_data)).convert("RGB")
img.save(str(png_path), "PNG")
except Exception as e:
return {"error": f"Invalid image: {e}"}
gif_path.unlink(missing_ok=True)
return {"ok": True, "kind": "png"}
@router.post("/api/song/{filename:path}/art/upload")
async def upload_song_art_b64(filename: str, data: dict):
"""Upload a custom cover as base64 (PNG/JPG/WebP → normalized PNG;
GIF → kept animated, local-only). The override outranks pack art in the
serve chain; remove it via DELETE …/art/override."""
import base64
# Reject art for a filename that doesn't resolve to a real song (mirrors the
# url route's guard) — no writing stray override files for unknown keys.
dlc = _get_dlc_dir()
song_path = _resolve_dlc_path(dlc, filename) if dlc else None
if song_path is None or not song_path.exists():
raise HTTPException(status_code=404, detail="unknown song")
b64 = data.get("image", "")
if not b64:
return {"error": "No image data"}
# Strip data URL prefix if present
if "," in b64:
b64 = b64.split(",", 1)[1]
try:
img_data = base64.b64decode(b64)
except Exception:
return {"error": "Invalid base64"}
if len(img_data) > _ART_URL_MAX_BYTES:
raise HTTPException(status_code=400, detail="image larger than 10 MB")
return _save_art_override(filename, img_data)
# Art-by-URL fetch cap — a cover, not a wallpaper pack.
_ART_URL_MAX_BYTES = 10 * 1024 * 1024
def _url_host_is_internal(url: str) -> bool:
"""True when a user-supplied URL's host resolves to a loopback, private,
link-local, reserved, multicast or unspecified address — an SSRF target we
refuse to fetch on the user's behalf (e.g. 169.254.169.254 metadata, LAN
services). Fails CLOSED: an unresolvable or unparseable host is treated as
internal. Every resolved address must be public for the URL to pass."""
from urllib.parse import urlparse
import socket
host = urlparse(url).hostname
if not host:
return True
try:
infos = socket.getaddrinfo(host, None)
except OSError:
return True
if not infos:
return True
for info in infos:
raw = info[4][0].split("%", 1)[0] # strip any zone id
try:
ip = ipaddress.ip_address(raw)
except ValueError:
return True
if (ip.is_private or ip.is_loopback or ip.is_link_local
or ip.is_reserved or ip.is_multicast or ip.is_unspecified):
return True
return False
# Art-by-URL redirect budget. Cover hosts commonly answer with a redirect —
# the Cover Art Archive (whose thumbs the cover picker applies through this
# very route) 307s every image to archive.org — so redirects must work; 5
# hops is generous for any real CDN chain while still bounding the walk.
_ART_URL_MAX_REDIRECTS = 5
def _fetch_art_url(url: str) -> bytes:
"""The one place art-by-URL touches the network (tests fake this seam).
User-initiated, so not throttled like the background workers — but the
same offline guard applies (pytest can never fetch), the host is checked
against internal/reserved ranges (SSRF), redirects are followed MANUALLY
with the scheme + internal-host guard re-applied to every hop (so a
redirect can't smuggle the request to an internal target — a blanket
no-redirect rule would break every Cover Art Archive pick, which always
redirects to archive.org), and the size cap is enforced while streaming
so a huge response never fully downloads.
Residual, accepted: each hop's host is resolved here and again by
requests, so a rebinding DNS name is a theoretical TOCTOU. Not closed
with an IP-pinned connection because (a) this is a single-user, no-auth
app (constitution §I) and the route is demo-blocked, so there is no
untrusted submission path, and (b) no other in-tree client (MusicBrainz,
CAA) pins either — a bespoke pinned+SNI adapter here would be
inconsistent and disproportionate. The cheap guards above still stop the
realistic vectors (direct internal URL, redirect-to-internal)."""
if not enrichment._enrich_network_enabled():
raise enrichment.EnrichTransportError("art fetch disabled (offline)")
import requests
from urllib.parse import urljoin, urlparse
for _hop in range(_ART_URL_MAX_REDIRECTS + 1):
# Re-validate EVERY hop, not just the user's original URL: the whole
# point of handling redirects ourselves is that each target gets the
# same scheme + SSRF gate before any request is made.
if urlparse(url).scheme not in ("http", "https"):
raise ValueError("url must be http(s)")
if _url_host_is_internal(url):
raise ValueError("url host is not allowed")
try:
with requests.get(url, timeout=15, stream=True, allow_redirects=False,
headers={"User-Agent": enrichment._enrich_user_agent()}) as resp:
if resp.status_code in (301, 302, 303, 307, 308):
loc = resp.headers.get("Location") or ""
if not loc:
raise enrichment.EnrichTransportError(
f"HTTP {resp.status_code} without a Location")
url = urljoin(url, loc)
continue
if resp.status_code != 200:
raise enrichment.EnrichTransportError(f"HTTP {resp.status_code}")
data = b""
for chunk in resp.iter_content(65536):
data += chunk
if len(data) > _ART_URL_MAX_BYTES:
raise ValueError("image larger than 10 MB")
return data
except requests.RequestException as e:
raise enrichment.EnrichTransportError(str(e)) from e
raise enrichment.EnrichTransportError("too many redirects")
@router.post("/api/song/{filename:path}/art/url")
def set_song_art_from_url(filename: str, data: dict):
"""Paste-a-link cover art (the media-server idiom): the server fetches the
image and stores it as this song's local override — identical result to an
upload, including the GIF-stays-local rule. http(s) only."""
url = str((data or {}).get("url") or "").strip()
from urllib.parse import urlparse
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise HTTPException(status_code=400, detail="url must be http(s)")
dlc = _get_dlc_dir()
song_path = _resolve_dlc_path(dlc, filename) if dlc else None
if song_path is None or not song_path.exists():
raise HTTPException(status_code=404, detail="unknown song")
try:
img_data = _fetch_art_url(url)
except enrichment.EnrichTransportError as e:
return JSONResponse({"error": "could not fetch image", "detail": str(e)},
status_code=502)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return _save_art_override(filename, img_data)
@router.delete("/api/art/{filename:path}/override")
def remove_song_art_override(filename: str):
"""Drop the user art override — the serve chain falls back to pack art,
then the Cover Art Archive cache. Lives under /api/art (NOT /api/song) so
the greedy DELETE /api/song/{path} catch-all can't shadow it — the same
dodge the chart split/unsplit routes use."""
removed = False
for p in appstate.art_override_paths(filename):
try:
p.unlink()
removed = True
except OSError:
pass
if removed:
# The art worker may have settled this row as 'user' (override present,
# no pack art). Reset it so the next enrichment pass re-evaluates and the
# CAA fallback resumes — otherwise a removed override strands the row
# (enrichment_art_pending only re-queues art_state IS NULL) and the song
# is left with no art at all.
try:
appstate.meta_db.set_enrichment_art(filename, None, None)
except Exception:
log.exception("art override delete: failed to reset enrichment state")
return {"ok": True, "removed": removed}
+126
View File
@@ -0,0 +1,126 @@
"""Artist routes: the artist page + external-links payload
(/api/artist/{name}/page, /links, /links/refresh).
Extracted verbatim from server.py (R3) except @app->@router and the seam reads
(meta_db->appstate.meta_db, CONFIG_DIR->appstate.config_dir, _default_settings->
appstate.default_settings). MusicBrainz link enrichment is reached as
enrichment.X; the shared URL-safety validator lives in lib/library_registry.py.
"""
from fastapi import APIRouter
import appstate
import enrichment
from appconfig import _load_config
from library_registry import _safe_art_redirect_url
import logging
log = logging.getLogger("feedBack.server")
router = APIRouter()
# MB artist url-relation types → the page's link slots (locked position 4:
# whitelist only, links-only forever). Everything not listed is dropped.
_ARTIST_URL_REL_SLOTS = {
"official homepage": "official",
"setlistfm": "tour",
"concerts": "tour",
"youtube": "video",
"video channel": "video",
"social network": "social",
"bandcamp": "social",
"soundcloud": "social",
"wikipedia": "wikipedia",
"wikidata": "wikipedia",
}
def _artist_links_from_mb(body: dict) -> tuple[dict, list]:
"""Whitelist an MB artist doc's url-relations into the page's link slots:
{official, tour, video, social: [...], wikipedia}. Every URL passes the
same http(s)-scheme gate as art redirects (_safe_art_redirect_url) so a
hostile javascript:/data:/file: resource can never reach an href. First
URL wins per single slot; social collects up to 5; wikipedia is preferred
over wikidata when both exist. Also returns MB's genre names (capped)."""
links: dict = {}
social: list = []
wikidata_url = None
for rel in (body or {}).get("relations") or []:
if not isinstance(rel, dict):
continue
rtype = str(rel.get("type") or "").strip().lower()
slot = _ARTIST_URL_REL_SLOTS.get(rtype)
if not slot:
continue
url = rel.get("url")
url = url.get("resource") if isinstance(url, dict) else url
if _safe_art_redirect_url(url) is None:
continue
if slot == "social":
if url not in social and len(social) < 5:
social.append(url)
elif rtype == "wikidata":
wikidata_url = wikidata_url or url
elif slot not in links:
links[slot] = url
if social:
links["social"] = social
if "wikipedia" not in links and wikidata_url:
links["wikipedia"] = wikidata_url
genres = [str(g.get("name")) for g in (body or {}).get("genres") or []
if isinstance(g, dict) and g.get("name")]
return links, genres[:8]
def _artist_links_payload(name: str, force: bool = False) -> dict:
"""Shared by GET links + POST refresh. Order of gates: the user's opt-in
setting (external links are OFF by default — the dev-chat thread's call),
then a known mb_artist_id (no id → nothing to look up), then the cache
(unless force), then the offline guard, then ONE throttled fetch."""
cfg = _load_config(appstate.config_dir / "config.json") or appstate.default_settings()
if cfg.get("artist_external_links") is not True:
return {"links": {}, "matched": False, "disabled": True}
canonical = appstate.meta_db._terminal_canonical((name or "").strip())
mbid = appstate.meta_db.artist_known_mb_id(appstate.meta_db._raw_variants_for(canonical))
mbid = (mbid or "").strip().lower()
# The id is interpolated into the MB request path — same strict-shape rule
# as the manifest identity keys (_MBID_RE), so a junk/hostile value stored
# via a hand-rolled /pick body can never reach the request line.
if not mbid or not enrichment._MBID_RE.match(mbid):
return {"links": {}, "matched": False}
if not force:
cached = appstate.meta_db.get_artist_enrichment(mbid)
if cached:
return {"links": cached["url_rels"], "genres": cached["genres"],
"matched": True, "cached": True, "mb_artist_id": mbid}
if not enrichment._enrich_network_enabled():
return {"links": {}, "matched": True, "offline": True, "mb_artist_id": mbid}
try:
body = enrichment._mb_http_get(f"artist/{mbid}", {"inc": "url-rels+genres+tags"})
except enrichment.EnrichTransportError:
return {"links": {}, "matched": True, "offline": True, "mb_artist_id": mbid}
links, genres = _artist_links_from_mb(body or {})
appstate.meta_db.put_artist_enrichment(mbid, links, genres)
return {"links": links, "genres": genres, "matched": True, "cached": False,
"mb_artist_id": mbid}
@router.get("/api/artist/{name:path}/page")
def api_artist_page(name: str):
"""The artist page's all-LOCAL payload — counts, albums, aliases, similar-
in-library, mosaic art, play-all seed. Never touches the network; an
unmatched or even unknown artist still returns a functional page."""
return appstate.meta_db.artist_page(name)
@router.get("/api/artist/{name:path}/links")
def api_artist_links(name: str):
"""External links for a matched artist — cached after the first call.
Sync route on purpose (like /api/enrichment/search): FastAPI runs it in
the threadpool so the MB throttle's sleep never blocks the event loop."""
return _artist_links_payload(name)
@router.post("/api/artist/{name:path}/links/refresh")
def api_artist_links_refresh(name: str):
"""Explicit re-fetch of the cached links (the page's manual Refresh)."""
return _artist_links_payload(name, force=True)
+346
View File
@@ -0,0 +1,346 @@
"""Metadata-enrichment route handlers (/api/enrichment/*): status, kick/cancel,
per-song state, the Match-Review queue (accept/reject/pick/search), and AcoustID
fingerprint identify.
Extracted verbatim from server.py (R3) except @app->@router and the seam reads
(meta_db->appstate.meta_db, CONFIG_DIR->appstate.config_dir). The enrichment
engine itself — transport, matcher, the background worker, and the upload caps —
lives in lib/enrichment.py and is reached here as enrichment.X.
"""
import asyncio
import os
import shutil
from pathlib import Path
from fastapi import APIRouter, Body, HTTPException, Request, UploadFile
from fastapi.responses import JSONResponse
import appstate
import enrichment
import mb_match
from appconfig import _load_config
import logging
log = logging.getLogger("feedBack.server")
router = APIRouter()
@router.get("/api/enrichment/status")
def enrichment_status():
"""Enrichment pipeline state: worker flags + row counts by match_state.
Ambient tool-state for the match-review UI (never a home-screen score —
design §11); also what tests poke."""
return {
"running": enrichment._enrich_status["running"],
"processed": enrichment._enrich_status["processed"],
"last_pass_at": enrichment._enrich_status["last_pass_at"],
"states": appstate.meta_db.enrichment_state_counts(),
"total_songs": appstate.meta_db.count(),
# Per-pass matching progress for the "Refresh Metadata" batch bar +
# per-tile badges (total = songs queued to match this pass, matched =
# done so far, current = the one being matched now).
"total": enrichment._enrich_status.get("total", 0),
"matched": enrichment._enrich_status.get("matched", 0),
"current": enrichment._enrich_status.get("current"),
"cancelling": enrichment._enrich_cancel.is_set(),
}
@router.get("/api/enrichment/song/{filename:path}")
def api_enrichment_song(filename: str):
"""Read-only per-song match provenance for the Details drawer (launch
polish): which canonical identity this chart matched and how. A tiny
projection of the cache row — no candidates, no cache paths."""
row = appstate.meta_db.get_enrichment(filename)
if not row:
raise HTTPException(status_code=404, detail="no enrichment row")
return {k: row.get(k) for k in
("match_state", "canon_artist", "canon_title",
"match_source", "match_score")}
@router.post("/api/enrichment/kick")
def api_enrichment_kick():
"""The Settings "Match now" button AND the library's "Refresh Metadata"
button: request an enrichment pass without waiting for a scan to complete.
Processes the songs that still need it (unscanned/changed + retriable
failures) — already-matched songs are left alone, so on a fully-matched
library this is a fast no-op. Single-flight + coalescing like every other
kick — spamming it queues at most one follow-up pass."""
return {"started": enrichment._kick_enrich()}
@router.post("/api/enrichment/cancel")
def api_enrichment_cancel():
"""Stop button on the "Refresh Metadata" batch: signal the running pass to
halt after the current song (an in-flight ≤1/s lookup can't be interrupted,
but no new one is started) and drop any coalesced follow-up. A no-op when
nothing is running."""
was_running = enrichment._enrich_status["running"]
if was_running:
enrichment._enrich_cancel.set()
return {"ok": True, "was_running": was_running}
@router.post("/api/enrichment/rematch")
def api_enrichment_rematch(data: dict = Body(...)):
"""The library "Refresh Metadata" button: force a fresh re-match of the
songs the grid is SHOWING (its visible/filtered window). Resets each to
`unscanned` so the next pass re-fetches it from scratch — EXCEPT user-pinned
`manual` rows, which are never auto-overwritten (apply_enrichment_match
guards that) — then kicks one pass. Scoped to the visible set on purpose:
fast (dozens of songs), visible (tiles animate), and it can't blow the whole
≤1/s rate budget on a 1000-song library the way a full re-sweep would.
Returns the filenames actually queued so the UI badges exactly those."""
raw = (data or {}).get("filenames") or []
fns = [str(f) for f in raw if isinstance(f, str)][:500]
queued: list[str] = []
for fn in fns:
song = appstate.meta_db.enrichment_song_row(fn)
if not song:
continue
h = appstate.meta_db.enrichment_content_hash(
song["artist"], song["title"], song["album"], song["duration"])
# allow_manual_overwrite=False → a manual pin is left as-is (returns
# False), everything else resets to unscanned (returns True).
if appstate.meta_db.apply_enrichment_match(fn, h, "unscanned",
allow_manual_overwrite=False):
queued.append(fn)
started = enrichment._kick_enrich() if queued else False
return {"queued": queued, "count": len(queued), "started": started}
@router.post("/api/enrichment/states")
def api_enrichment_states(data: dict = Body(...)):
"""Per-tile match states for the grid's VISIBLE window during a metadata
refresh: the client posts the filenames it is showing and gets back each
one's match_state (+ the song being matched right now, + whether a pass is
running), so a card can animate queued→working→result without a per-song
round-trip. Read-only — safe for demo visitors (no network, no mutation)."""
raw = (data or {}).get("filenames") or []
# Bound the batch: a visible grid window is dozens of cards; cap defensively.
fns = [str(f) for f in raw if isinstance(f, str)][:500]
return {
"states": appstate.meta_db.enrichment_states_for(fns),
"current": enrichment._enrich_status.get("current"),
"running": enrichment._enrich_status["running"],
}
@router.post("/api/enrichment/refresh/{filename:path}")
def api_enrichment_refresh(filename: str):
"""The context menu's "Refresh metadata": reset THIS song's match to
unscanned (canonical values + candidates cleared, backoff zeroed) and
kick a pass so it re-matches immediately. An EXPLICIT user action, so it
may discard a manual pin — the automation never does, but the user
asking for a re-match is the one party who owns that pin."""
song = appstate.meta_db.enrichment_song_row(filename)
if not song:
raise HTTPException(status_code=404, detail="unknown song")
h = appstate.meta_db.enrichment_content_hash(
song["artist"], song["title"], song["album"], song["duration"])
appstate.meta_db.apply_enrichment_match(filename, h, "unscanned",
allow_manual_overwrite=True)
return {"ok": True, "started": enrichment._kick_enrich()}
@router.get("/api/enrichment/review")
def api_enrichment_review(limit: int = 200):
"""The Match-Review queue: songs whose text match landed in the medium-
confidence review tier, each with its stored candidate list — the drawer
renders straight from this, no MusicBrainz round-trip. Ordered by the
user's enrich_review_order setting."""
limit = max(1, min(int(limit), 500))
cfg = _load_config(appstate.config_dir / "config.json") or {}
order = cfg.get("enrich_review_order", "missing_first")
return {
"songs": appstate.meta_db.enrichment_review_queue(limit=limit, order=order),
"total_review": appstate.meta_db.enrichment_state_counts().get("review", 0),
}
@router.post("/api/enrichment/review/{filename:path}/accept")
def api_enrichment_accept(filename: str, data: dict = Body(...)):
"""Accept one of the stored review candidates: the row becomes a
user-pinned `manual` match (never auto-reset). Display-only, like every
enrichment write — nothing touches the pack file."""
recording_id = str((data or {}).get("recording_id") or "")
row = appstate.meta_db.get_enrichment(filename)
if not row or row["match_state"] != "review":
raise HTTPException(status_code=404, detail="no review row for this song")
cand = next((c for c in (row.get("candidates") or [])
if c.get("recording_id") == recording_id), None)
if not cand:
raise HTTPException(status_code=404, detail="candidate not in the stored list")
if not appstate.meta_db.set_enrichment_manual(filename, cand, source="review"):
raise HTTPException(status_code=404, detail="unknown song")
return {"ok": True, "enrichment": appstate.meta_db.get_enrichment(filename)}
@router.post("/api/enrichment/review/{filename:path}/reject")
def api_enrichment_reject(filename: str):
""""None of these" — clears any canonical values and parks the row as
failed/rejected (never auto-retried; editing the song's metadata
re-queues it). Valid from `review` or `matched`, never from `manual`."""
if not appstate.meta_db.set_enrichment_rejected(filename):
raise HTTPException(status_code=404, detail="no rejectable match for this song")
return {"ok": True, "enrichment": appstate.meta_db.get_enrichment(filename)}
# The candidate fields a manual pick is allowed to carry — the payload comes
# from our own /api/enrichment/search proxy, but the route re-sanitizes so a
# hand-rolled client can't stuff arbitrary keys/types into the cache row.
_CAND_STR_FIELDS = ("recording_id", "title", "artist", "artist_id",
"artist_sort", "release_id", "album", "year", "isrc")
def _sanitize_candidate(raw: dict) -> dict | None:
if not isinstance(raw, dict):
return None
out = {k: str(raw.get(k) or "") for k in _CAND_STR_FIELDS}
if not out["recording_id"] or not out["title"]:
return None
genres = raw.get("genres") or []
out["genres"] = [str(g) for g in genres if isinstance(g, str)][:5] \
if isinstance(genres, list) else []
return out
@router.post("/api/enrichment/review/{filename:path}/pick")
def api_enrichment_pick(filename: str, data: dict = Body(...)):
"""Fix-match / manual search-and-pick: pin a candidate the user found via
/api/enrichment/search (not limited to the stored review list — this is
the escape hatch for a wrong auto-match too). Sets `manual`, the
highest-authority state."""
cand = _sanitize_candidate((data or {}).get("candidate"))
if not cand:
raise HTTPException(status_code=400, detail="candidate needs recording_id + title")
if not appstate.meta_db.set_enrichment_manual(filename, cand, source="search"):
raise HTTPException(status_code=404, detail="unknown song")
return {"ok": True, "enrichment": appstate.meta_db.get_enrichment(filename)}
@router.get("/api/enrichment/search")
def api_enrichment_search(artist: str = "", title: str = "", limit: int = 8,
filename: str = "", duration: float = 0.0):
"""Manual-search proxy to MusicBrainz (throttled + identified like the
background matcher — a user typing in the drawer must not sidestep the
rate limit). `filename` optionally scores results against that song's
stored identity (year/duration corroboration) instead of just the typed
text. `duration` (seconds) lets a caller that HAS the audio but no library
row — e.g. the editor's create modal, which holds the master track — pass
its length so the studio take ranks above live/extended cuts. Sync route on
purpose: FastAPI runs it in the threadpool, so the throttle's sleep never
blocks the event loop."""
if not (artist.strip() or title.strip()):
raise HTTPException(status_code=400, detail="artist or title required")
limit = max(1, min(int(limit), 25))
try:
cands = enrichment._mb_search_recordings(artist, title, limit=limit)
except enrichment.EnrichTransportError as e:
return JSONResponse({"error": "musicbrainz unavailable", "detail": str(e)},
status_code=503)
ref = None
if filename:
ref = appstate.meta_db.enrichment_song_row(filename)
if ref is None:
ref = {"artist": artist, "title": title}
# A caller-supplied duration corroborates the take even without a library row.
if duration and duration > 0 and not ref.get("duration"):
ref = dict(ref)
ref["duration"] = duration
# Alias-enrich so a non-Latin-primary artist (大橋純子) ranks by its
# romanized alias against the typed query ("Junko Ohashi") instead of
# sinking to the bottom with a 0 artist score.
try:
enrichment._alias_enrich(ref, cands)
except enrichment.EnrichTransportError:
pass # aliases are a ranking nicety here; fall back to primary-name scoring
return {"candidates": mb_match.rank_candidates(ref, cands)}
@router.post("/api/enrichment/identify")
async def api_enrichment_identify(request: Request):
"""Identify a song by AUDIO FINGERPRINT (AcoustID) rather than text — the
reliable way to get the EXACT recording/version (the studio take, not a live
bootleg or an extended cut). Upload the master audio; returns candidates in
the same shape as /search, so the review UI and the editor's Match popup can
render fingerprint hits identically. 412 `needs_setup` when the user hasn't
opted in / has no key (the UI nudges them to Settings); 503 when it's set up
but the fpcalc Chromaprint binary is missing or the network is off. Async so
the multipart is size-capped BEFORE spooling; the blocking fpcalc subprocess
+ AcoustID HTTP run in the threadpool via run_in_executor."""
gate = enrichment._acoustid_gate()
if gate is not None:
return gate
# Pre-parse Content-Length guard — reject an oversized body before Starlette
# spools the multipart to temp disk (mirrors the song-upload endpoint). The
# per-part cap below is the authoritative limit; this is the fast up-front no.
cl = request.headers.get("content-length")
if cl is not None:
try:
cl_int = int(cl)
except ValueError:
return JSONResponse({"error": "Invalid Content-Length header"}, status_code=400)
if cl_int > enrichment._ACOUSTID_MAX_UPLOAD_BYTES + enrichment._MULTIPART_OVERHEAD_SLACK:
return JSONResponse({"error": "audio upload too large (256 MB max)"}, status_code=413)
try:
form = await request.form(max_part_size=enrichment._ACOUSTID_MAX_UPLOAD_BYTES)
except Exception:
return JSONResponse({"error": "audio upload too large (256 MB max)"}, status_code=413)
file = form.get("file")
if not isinstance(file, UploadFile):
raise HTTPException(status_code=400, detail="missing file upload")
import tempfile
ext = (Path(file.filename or "").suffix or ".bin").lower()
tmpdir = tempfile.mkdtemp(prefix="feedback_acoustid_")
tmp = os.path.join(tmpdir, "audio" + ext)
try:
total = 0
with open(tmp, "wb") as fh:
while True:
chunk = await file.read(1024 * 1024)
if not chunk:
break
total += len(chunk)
if total > enrichment._ACOUSTID_MAX_UPLOAD_BYTES:
return JSONResponse(
{"error": "audio upload too large (256 MB max)"}, status_code=413)
fh.write(chunk)
if total == 0:
raise HTTPException(status_code=400, detail="empty upload")
# fpcalc subprocess + AcoustID HTTP are blocking — off the event loop.
cands = await asyncio.get_event_loop().run_in_executor(
None, enrichment._identify_by_fingerprint, tmp)
except enrichment.EnrichTransportError as e:
return JSONResponse({"error": "acoustid unavailable", "detail": str(e)},
status_code=503)
finally:
shutil.rmtree(tmpdir, ignore_errors=True)
return {"candidates": cands}
@router.post("/api/enrichment/identify/{filename:path}")
def api_enrichment_identify_song(filename: str):
"""Identify an EXISTING library song by AUDIO FINGERPRINT — the library-side
counterpart to /api/enrichment/identify (which takes an upload). Fingerprints
the song's own master audio on disk (the manual "Identify by audio" action in
the Fix-metadata / match-review flow). Same candidate shape as /search, so the
review UI renders fingerprint hits like text hits. Same 412/503 gating; 404
when the song has no full-mix audio to fingerprint."""
gate = enrichment._acoustid_gate()
if gate is not None:
return gate
audio = enrichment._song_audio_file(filename)
if not audio:
return JSONResponse(
{"error": "no audio",
"detail": "couldn't find this song's master audio to fingerprint "
"(a stems-only pack has no full mix to identify)."},
status_code=404)
try:
cands = enrichment._identify_by_fingerprint(audio)
except enrichment.EnrichTransportError as e:
return JSONResponse({"error": "acoustid unavailable", "detail": str(e)},
status_code=503)
return {"candidates": cands}
+485
View File
@@ -0,0 +1,485 @@
"""Library + smart-collection routes: the provider list/art/sync endpoints, the
library query surface (songs, albums, artists, stats, genres, tuning-names,
practice-suggestions), and collection CRUD.
Extracted verbatim from server.py (R3) except @app->@router and the seam reads:
meta_db->appstate.meta_db, and the registry singletons ->
appstate.library_providers / appstate.local_library_provider (constructed +
owned by server.py; plugins register providers through plugin_context). The
provider classes + shared query/collection helpers live in lib/library_registry.py.
"""
import inspect
from pathlib import Path
from typing import Any
from fastapi import APIRouter, HTTPException
from fastapi.responses import FileResponse, JSONResponse, RedirectResponse, Response
from starlette.concurrency import run_in_threadpool
import appstate
from library_registry import (
_library_filter_args, _sanitize_collection_rules,
_safe_art_redirect_url, _split_csv, _sync_collection_provider,
_unregister_collection_provider,
)
from metadata_db import _effective_keyset_sort, next_library_cursor
from reqfields import _clean_str
import logging
log = logging.getLogger("feedBack.server")
router = APIRouter()
def _get_library_provider(provider: str = "local") -> object:
library_provider = appstate.library_providers.get(provider or "local")
if library_provider is None:
raise HTTPException(status_code=404, detail=f"Unknown library provider: {provider}")
return library_provider
def _require_library_provider_capability(provider: object, capability: str) -> None:
if capability in appstate.library_providers.provider_capabilities(provider):
return
provider_id = appstate.library_providers.provider_id(provider)
raise HTTPException(
status_code=501,
detail=f"Library provider {provider_id!r} does not declare capability {capability!r}",
)
_OPTIONAL_NEW_PROVIDER_KWARGS = ("naming_mode", "sort", "want_sort_letters", "after",
"mastery", "match_states")
def _filter_provider_kwargs(method: object, kwargs: dict) -> dict:
"""Drop kwargs that the method's signature does not declare.
Provides backward-compat for third-party library providers whose
query_page/query_artists/query_stats methods were written before
naming_mode was added — calling them with the extra kwarg would
raise TypeError and return a 500 to the client.
When ``inspect.signature`` cannot introspect the method (rare: C
extensions / built-ins / exotic callables), fall back to stripping
only the kwargs we know were added later — older providers won't
accept them, anything else stays so the call still works.
"""
try:
sig = inspect.signature(method) # type: ignore[arg-type]
for p in sig.parameters.values():
if p.kind == inspect.Parameter.VAR_KEYWORD:
return kwargs # method accepts **kwargs, pass everything
return {k: v for k, v in kwargs.items() if k in sig.parameters}
except (ValueError, TypeError):
return {k: v for k, v in kwargs.items() if k not in _OPTIONAL_NEW_PROVIDER_KWARGS}
def _call_library_provider(provider: object, method_name: str, **kwargs) -> Any:
method = appstate.library_providers.provider_method(provider, method_name)
if not callable(method):
provider_id = appstate.library_providers.provider_id(provider)
raise HTTPException(
status_code=501,
detail=f"Library provider {provider_id!r} does not support {method_name}",
)
try:
return method(**_filter_provider_kwargs(method, kwargs))
except HTTPException:
raise
except Exception as exc:
provider_id = appstate.library_providers.provider_id(provider)
# A provider with an explicit kind="local" is treated as local even if
# its id is not "local" (e.g. a kind="local" plugin variant). Otherwise
# fall back to provider_id comparison so providers that omit `kind` are
# still wrapped correctly — the safe default for unknown providers is to
# surface an offline message rather than leaking raw exceptions.
provider_kind = str(appstate.library_providers.provider_field(provider, "kind", "") or "")
if provider_kind:
is_remote = provider_kind not in ("", "local")
else:
is_remote = provider_id != "local"
if is_remote:
detail = f"This source appears to be offline ({provider_id})."
message = str(exc).strip()
if message:
detail = f"{detail} {message}"
raise HTTPException(status_code=503, detail=detail) from exc
raise
def _is_async_callable(obj: object) -> bool:
"""Return True if obj is an async function or a callable object with an async __call__.
``inspect.iscoroutinefunction`` only recognises bare coroutine functions; it returns
False for class instances whose ``__call__`` method is defined as ``async def``.
Checking both handles the common plugin pattern of wrapping an async method in a
callable object.
"""
if inspect.iscoroutinefunction(obj):
return True
_call = getattr(obj, "__call__", None)
return _call is not None and inspect.iscoroutinefunction(_call)
async def _call_library_provider_async(provider: object, method_name: str, **kwargs) -> Any:
method = appstate.library_providers.provider_method(provider, method_name)
if _is_async_callable(method):
# Async provider method — call directly on the event loop.
try:
return await method(**_filter_provider_kwargs(method, kwargs))
except HTTPException:
raise
except Exception as exc:
provider_id = appstate.library_providers.provider_id(provider)
provider_kind = str(appstate.library_providers.provider_field(provider, "kind", "") or "")
if provider_kind:
is_remote = provider_kind not in ("", "local")
else:
is_remote = provider_id != "local"
if is_remote:
detail = f"This source appears to be offline ({provider_id})."
message = str(exc).strip()
if message:
detail = f"{detail} {message}"
raise HTTPException(status_code=503, detail=detail) from exc
raise
# Synchronous provider method — run in a threadpool so the event loop stays free.
return await run_in_threadpool(_call_library_provider, provider, method_name, **kwargs)
def _library_art_response(result: Any) -> Response:
if result is None:
raise HTTPException(status_code=404, detail="Library provider returned no art")
if isinstance(result, Response):
return result
if isinstance(result, (bytes, bytearray, memoryview)):
return Response(content=bytes(result), media_type="image/png")
if isinstance(result, str):
safe_url = _safe_art_redirect_url(result)
if safe_url is not None:
return RedirectResponse(safe_url)
# If the string looks like a URL (contains a scheme separator) but
# didn't pass the http/https check, refuse it rather than treating
# it as a filesystem path — a provider returning ftp:// or file://
# should get a 400, not a 500 from FileResponse failing on a URL.
if "://" in result:
raise HTTPException(
status_code=400,
detail="Library provider returned an unsupported URL scheme for art",
)
if not Path(result).is_file():
raise HTTPException(status_code=404, detail="Library provider returned an unreadable art path")
return FileResponse(result)
if isinstance(result, Path):
if not result.is_file():
raise HTTPException(status_code=404, detail="Library provider returned an unreadable art path")
return FileResponse(str(result))
if isinstance(result, dict):
url = result.get("url") or result.get("art_url") or result.get("artUrl")
if isinstance(url, str) and url:
safe_url = _safe_art_redirect_url(url)
if safe_url is None:
raise HTTPException(status_code=400, detail="Library provider returned an unsafe art URL")
return RedirectResponse(safe_url)
path = result.get("path") or result.get("file")
if isinstance(path, (str, Path)):
media_type = result.get("media_type") or result.get("content_type")
if not Path(path).is_file():
raise HTTPException(status_code=404, detail="Library provider returned an unreadable art path")
return FileResponse(str(path), media_type=media_type)
content = result.get("content") or result.get("bytes")
if isinstance(content, (bytes, bytearray, memoryview)):
media_type = result.get("media_type") or result.get("content_type") or "image/png"
return Response(content=bytes(content), media_type=media_type)
raise HTTPException(status_code=500, detail="Library provider returned unsupported art data")
@router.get("/api/library/providers")
def list_library_providers():
"""List registered library providers."""
return {"providers": appstate.library_providers.list()}
@router.get("/api/library/providers/{provider_id}/songs/{song_id:path}/art")
async def get_library_provider_song_art(provider_id: str, song_id: str):
"""Return album art for a song owned by a library provider."""
library_provider = _get_library_provider(provider_id)
_require_library_provider_capability(library_provider, "art.read")
result = await _call_library_provider_async(library_provider, "get_art", song_id=song_id)
return _library_art_response(result)
@router.post("/api/library/providers/{provider_id}/songs/{song_id:path}/sync")
async def sync_library_provider_song(provider_id: str, song_id: str):
"""Ask a provider to sync a remote song into the local library/cache."""
library_provider = _get_library_provider(provider_id)
_require_library_provider_capability(library_provider, "song.sync")
result = await _call_library_provider_async(library_provider, "sync_song", song_id=song_id)
if result is None:
return {"ok": True}
if isinstance(result, dict):
return result
return {"ok": True, "result": result}
@router.get("/api/library")
async def list_library(q: str = "", page: int = 0, size: int = 24, sort: str = "artist",
dir: str = "asc", favorites: int = 0, format: str = "",
artist: str = "", album: str = "",
arrangements_has: str = "", arrangements_lacks: str = "",
stems_has: str = "", stems_lacks: str = "",
has_lyrics: str = "", tunings: str = "", provider: str = "local",
mastery: str = "", tags: str = "", user_difficulty: str = "",
match: str = "", genre: str = "", after: str = "", group: int = 0,
naming_mode: str = "legacy"):
"""Paginated library search through the selected library provider.
`after` is an opaque keyset cursor (feedBack#636 item 3): pass back the
`next_cursor` from the previous response to fetch the next page with a
WHERE-seek instead of OFFSET. Providers that don't support it ignore it and
page by OFFSET, so the client can always fall back."""
size = min(size, 100)
library_provider = _get_library_provider(provider)
_require_library_provider_capability(library_provider, "library.read")
# Only the true local provider keysets: it's the one whose effective sort is
# exactly the request `sort`. A smart collection may pin its own sort and
# remote providers don't keyset — both must page by OFFSET, so never hand
# them a cursor (a mismatched one would mis-seek).
is_local = getattr(library_provider, "id", "") == "local"
songs, total = await _call_library_provider_async(
library_provider,
"query_page",
page=page,
size=size,
sort=sort,
direction=dir,
after=((after or None) if is_local else None),
group=bool(group),
naming_mode=naming_mode,
mastery=_split_csv(mastery),
tags_has=_split_csv(tags),
user_difficulty_in=_split_csv(user_difficulty),
match_states=_split_csv(match),
genre=_split_csv(genre),
**_library_filter_args(
q=q, favorites=favorites, format=format,
artist=artist, album=album,
arrangements_has=arrangements_has, arrangements_lacks=arrangements_lacks,
stems_has=stems_has, stems_lacks=stems_lacks,
has_lyrics=has_lyrics, tunings=tunings,
),
)
# The cursor to resume after this page (effective sort folds in dir=desc).
next_cursor = (next_library_cursor(_effective_keyset_sort(sort, dir), songs[-1])
if (is_local and songs) else None)
# Drop the private raw-title stash query_page attached for the cursor — it's
# an internal keyset detail, not part of the card payload.
for s in songs:
s.pop("_sort_title", None)
return {"songs": songs, "total": total, "page": page, "size": size,
"next_cursor": next_cursor}
@router.get("/api/library/albums")
async def list_library_albums(q: str = "", page: int = 0, size: int = 120,
favorites: int = 0, format: str = "",
artist: str = "", album: str = "",
arrangements_has: str = "", arrangements_lacks: str = "",
stems_has: str = "", stems_lacks: str = "",
has_lyrics: str = "", tunings: str = "", mastery: str = "",
match: str = "", genre: str = "",
provider: str = "local"):
"""Album-condensed browse: distinct (artist, album) groups with a track count
and a representative cover song. Paged by album. Same filters as /api/library."""
size = min(size, 500)
library_provider = _get_library_provider(provider)
_require_library_provider_capability(library_provider, "library.read")
albums, total = await _call_library_provider_async(
library_provider, "query_albums",
page=page, size=size, mastery=_split_csv(mastery),
match_states=_split_csv(match), genre=_split_csv(genre),
**_library_filter_args(
q=q, favorites=favorites, format=format, artist=artist, album=album,
arrangements_has=arrangements_has, arrangements_lacks=arrangements_lacks,
stems_has=stems_has, stems_lacks=stems_lacks,
has_lyrics=has_lyrics, tunings=tunings,
),
)
return {"albums": albums, "total": total, "page": page, "size": size}
@router.get("/api/library/artists")
async def list_artists(letter: str = "", q: str = "", favorites: int = 0, page: int = 0,
size: int = 50, format: str = "",
artist: str = "", album: str = "",
arrangements_has: str = "", arrangements_lacks: str = "",
stems_has: str = "", stems_lacks: str = "",
has_lyrics: str = "", tunings: str = "", provider: str = "local",
naming_mode: str = "legacy"):
"""Get artists grouped by letter with albums and songs (for tree view)."""
size = min(size, 100)
library_provider = _get_library_provider(provider)
_require_library_provider_capability(library_provider, "library.read")
artists, total = await _call_library_provider_async(
library_provider,
"query_artists",
letter=letter,
page=page,
size=size,
naming_mode=naming_mode,
**_library_filter_args(
q=q, favorites=favorites, format=format,
artist=artist, album=album,
arrangements_has=arrangements_has, arrangements_lacks=arrangements_lacks,
stems_has=stems_has, stems_lacks=stems_lacks,
has_lyrics=has_lyrics, tunings=tunings,
),
)
return {"artists": artists, "total_artists": total, "page": page, "size": size}
@router.get("/api/library/stats")
async def library_stats(favorites: int = 0, q: str = "", format: str = "",
artist: str = "", album: str = "",
arrangements_has: str = "", arrangements_lacks: str = "",
stems_has: str = "", stems_lacks: str = "",
has_lyrics: str = "", tunings: str = "", provider: str = "local",
match: str = "",
sort: str = "artist", sort_letters: int = 0,
group: int = 0, naming_mode: str = "legacy"):
"""Aggregate stats for the UI. Accepts the same filter params as
/api/library so the letter bar mirrors the active grid filter set.
`sort` selects the column the jump rail's `sort_letters` keys on;
`sort_letters=1` opts into that breakdown (the rail), so non-rail
callers skip the extra per-letter aggregate. `group=1` counts works not
charts (mirrors the grouped grid)."""
library_provider = _get_library_provider(provider)
_require_library_provider_capability(library_provider, "library.read")
return await _call_library_provider_async(
library_provider,
"query_stats",
naming_mode=naming_mode,
sort=sort,
want_sort_letters=bool(sort_letters),
group=bool(group),
# The match facet rides the stats call too — the AZ rail's letter
# counts must agree with the grid under the facet or its cumulative
# seek + sizer geometry break.
match_states=_split_csv(match),
**_library_filter_args(
q=q, favorites=favorites, format=format,
artist=artist, album=album,
arrangements_has=arrangements_has, arrangements_lacks=arrangements_lacks,
stems_has=stems_has, stems_lacks=stems_lacks,
has_lyrics=has_lyrics, tunings=tunings,
),
)
@router.get("/api/library/genres")
def library_genres(provider: str = "local"):
"""Distinct non-empty genres for the filter facet.
Genres are a local-library facet: they're populated from the feedpak
`genres` field at scan time and live in the local meta DB. Local-backed
providers (the local library and its smart collections, kind="local")
share that DB, so they surface the same set. Remote providers don't
expose genres here, so return an empty facet for them — the client then
hides the filter rather than offering local genres that don't apply to
the remote grid. Mirrors the local/remote gating used elsewhere for
provider calls (see `_call_library_provider`)."""
library_provider = _get_library_provider(provider)
kind = str(appstate.library_providers.provider_field(library_provider, "kind", "") or "")
is_remote = kind not in ("", "local") if kind else provider != "local"
if is_remote:
return {"genres": []}
with appstate.meta_db._lock:
g = appstate.meta_db._effective_genre_expr()
rows = appstate.meta_db.conn.execute(
f"SELECT g FROM (SELECT DISTINCT ({g}) AS g FROM songs) "
"WHERE g IS NOT NULL AND g != '' ORDER BY g COLLATE NOCASE"
).fetchall()
return {"genres": [r[0] for r in rows]}
@router.get("/api/library/tuning-names")
async def list_tuning_names(provider: str = "local"):
"""Distinct tuning names present in the library, with per-tuning
counts. Powers the tuning multi-select. Sorted by `tuning_sort_key`
so names appear in the same musical order the sort uses
(feedBack#22) — E Standard first, then nearest neighbors."""
library_provider = _get_library_provider(provider)
_require_library_provider_capability(library_provider, "library.read")
return await _call_library_provider_async(library_provider, "tuning_names")
@router.get("/api/library/practice-suggestions")
def api_practice_suggestions(limit: int = 8):
"""Growth-edge 'practice next' shelf (P3): attempted-but-not-mastered songs
ranked by difficulty-appropriateness × mastery-proximity, joined to song
metadata. Replaces the recency-only 'Keep practicing' shelf ordering. Local
library only — reads local practice stats."""
from urllib.parse import quote
out = []
for r in appstate.meta_db.growth_edge_suggestions(limit):
meta = appstate.meta_db.conn.execute(
"SELECT title, artist, tuning_name FROM songs WHERE filename = ?",
(r["filename"],),
).fetchone()
title, artist, tuning_name = meta if meta else (None, None, None)
out.append({
**r,
"title": title or r["filename"],
"artist": artist or "",
"tuning_name": tuning_name or "",
"art_url": f"/api/song/{quote(r['filename'])}/art",
})
return out
@router.get("/api/collections")
def api_list_collections():
"""Smart/dynamic collections (saved live library filters)."""
return {"collections": appstate.meta_db.list_collections()}
@router.post("/api/collections")
def api_create_collection(data: dict):
"""Create a collection from a name + a set of library filter rules. It
immediately appears as a source in the library provider picker."""
if not isinstance(data, dict):
return JSONResponse({"error": "body must be an object"}, status_code=400)
name = _clean_str(data.get("name"))
if not name:
return JSONResponse({"error": "name required"}, status_code=400)
col = appstate.meta_db.create_collection(name, _sanitize_collection_rules(data.get("rules")))
_sync_collection_provider(col)
return {"ok": True, "collection": col}
@router.put("/api/collections/{pid}")
def api_update_collection(pid: int, data: dict):
"""Rename a collection and/or replace its rules."""
if not isinstance(data, dict):
return JSONResponse({"error": "body must be an object"}, status_code=400)
name = _clean_str(data.get("name")) or None
rules = _sanitize_collection_rules(data["rules"]) if "rules" in data else None
col = appstate.meta_db.update_collection(pid, name=name, rules=rules)
if col is None:
return JSONResponse({"error": "collection not found"}, status_code=404)
_sync_collection_provider(col)
return {"ok": True, "collection": col}
@router.delete("/api/collections/{pid}")
def api_delete_collection(pid: int):
"""Delete a collection and unregister its provider."""
if not appstate.meta_db.is_collection(pid):
return JSONResponse({"error": "collection not found"}, status_code=404)
appstate.meta_db.delete_playlist(pid)
_unregister_collection_provider(pid)
return {"ok": True}
+162
View File
@@ -0,0 +1,162 @@
"""Media/file-serving routes: song audio (/audio/{f}), the local-audio-path
resolver (/api/audio-local-path), and raw sloppak member serving
(/api/sloppak/{f}/file/{rel}).
Extracted verbatim from server.py (R3) except @app->@router and the cache/static
path seams (AUDIO_CACHE_DIR->appstate.audio_cache_dir, STATIC_DIR->
appstate.static_dir, SLOPPAK_CACHE_DIR->appstate.sloppak_cache_dir).
"""
import ipaddress
import re
from fastapi import APIRouter, Request
from fastapi.responses import FileResponse, JSONResponse
import appstate
import sloppak as sloppak_mod
from dlc_paths import _get_dlc_dir, _resolve_dlc_path
import logging
log = logging.getLogger("feedBack.server")
router = APIRouter()
def _resolve_sloppak_local_file(filename: str, rel_path: str):
"""Resolve a file inside a sloppak to its on-disk path.
Applies the same containment guards as ``serve_sloppak_file``. Returns the
resolved ``Path`` on success, or an ``(error, status)`` tuple on failure so
callers can produce their endpoint-appropriate response.
"""
dlc = _get_dlc_dir()
if not dlc:
return ("not configured", 404)
# `filename` is caller-controlled. Contain it under DLC_DIR before it
# reaches the resolver (see serve_sloppak_file for the traversal rationale).
resolved = _resolve_dlc_path(dlc, filename)
if resolved is None:
return ("forbidden", 403)
# Confine to actual sloppak bundles — otherwise any plain subdirectory
# would become a read-any-file-under-DLC_DIR source.
if not sloppak_mod.is_sloppak(resolved):
return ("not found", 404)
# Canonicalise the cache key against the resolved path so equivalent URL
# forms of the same sloppak converge on one _source_cache entry.
try:
filename = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
# safe_join already proved containment; fail closed regardless.
return ("forbidden", 403)
src = sloppak_mod.get_cached_source_dir(filename)
if src is None:
try:
src = sloppak_mod.resolve_source_dir(filename, dlc, appstate.sloppak_cache_dir)
except Exception:
return ("not found", 404)
# Prevent path traversal within the sloppak.
target = (src / rel_path).resolve()
try:
target.relative_to(src.resolve())
except ValueError:
return ("forbidden", 403)
if not target.exists() or not target.is_file():
return ("not found", 404)
return target
@router.get("/api/sloppak/{filename:path}/file/{rel_path:path}")
def serve_sloppak_file(filename: str, rel_path: str):
"""Serve a file from inside a sloppak (stems, cover, etc.)."""
result = _resolve_sloppak_local_file(filename, rel_path)
if isinstance(result, tuple):
error, status = result
return JSONResponse({"error": error}, status)
target = result
ext = target.suffix.lower()
mt = {
".ogg": "audio/ogg", ".opus": "audio/ogg", ".oga": "audio/ogg",
".mp3": "audio/mpeg", ".wav": "audio/wav", ".flac": "audio/flac",
".m4a": "audio/mp4",
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp",
".json": "application/json",
}.get(ext)
return FileResponse(str(target), media_type=mt) if mt else FileResponse(str(target))
@router.get("/api/audio-local-path")
def audio_local_path(url: str, request: Request):
"""Return absolute local filesystem path for a song URL (Electron desktop only).
Accepts ``/audio/<path>`` where ``<path>`` may include subdirectory segments —
no scheme, no host, no query string, no fragment. The resolved path must stay
inside appstate.audio_cache_dir or appstate.static_dir; ``..`` traversal, backslashes, and
absolute ``filename`` values are rejected.
Also accepts ``/api/sloppak/<filename>/file/<rel>`` (percent-encoded, as
emitted by the highway song payload) and resolves it to the unpacked
sloppak cache file via the same containment guards as
``serve_sloppak_file`` — this lets the desktop engine play a feedpak
full-mix natively under WASAPI-exclusive output.
This endpoint returns a raw filesystem path and is intended exclusively for
the Electron desktop process (which runs on loopback). Requests from non-
loopback clients are rejected with 403.
"""
# Loopback-only — only the local Electron process should call this
client_host = request.client.host if request.client else None
try:
is_loopback = bool(client_host and ipaddress.ip_address(client_host).is_loopback)
except ValueError:
is_loopback = client_host == "localhost"
if not is_loopback:
return JSONResponse({"error": "forbidden"}, status_code=403)
# Sloppak in-pack file (feedpak full-mix): /api/sloppak/<fn>/file/<rel>.
# Both segments arrive percent-encoded (built with urllib quote() in the
# highway payload); decode before handing to the shared resolver, which
# re-applies all containment guards on the decoded values.
slop_match = re.fullmatch(r"/api/sloppak/([^?#]+)/file/([^?#]+)", url)
if slop_match:
from urllib.parse import unquote
result = _resolve_sloppak_local_file(
unquote(slop_match.group(1)), unquote(slop_match.group(2))
)
if isinstance(result, tuple):
error, status = result
return JSONResponse({"error": error}, status_code=status)
return JSONResponse({"path": str(result)})
# Accept only simple /audio/<filename> — no scheme, no host, no query/fragment
if not re.fullmatch(r"/audio/[^?#]+", url):
return JSONResponse({"error": "invalid url"}, status_code=400)
filename = url[len("/audio/"):]
# Reject traversal, absolute paths, and backslash separators
if ".." in filename.split("/") or filename.startswith("/") or "\\" in filename:
return JSONResponse({"error": "invalid url"}, status_code=400)
for d in [appstate.audio_cache_dir, appstate.static_dir]:
candidate = (d / filename).resolve()
# Ensure resolved path is inside the allowed directory
try:
candidate.relative_to(d.resolve())
except ValueError:
continue
if candidate.is_file():
return JSONResponse({"path": str(candidate)})
return JSONResponse({"error": "not found"}, status_code=404)
@router.get("/audio/{filename:path}")
def serve_audio(filename: str):
"""Serve audio files from the writable audio cache directory."""
# Reject traversal attempts and absolute-path components
if ".." in filename.split("/") or filename.startswith("/") or "\\" in filename:
return JSONResponse({"error": "not found"}, status_code=404)
for d in [appstate.audio_cache_dir, appstate.static_dir]:
candidate = (d / filename).resolve()
try:
candidate.relative_to(d.resolve())
except ValueError:
continue
if candidate.is_file():
return FileResponse(str(candidate))
return JSONResponse({"error": "not found"}, status_code=404)
File diff suppressed because it is too large Load Diff
+867
View File
@@ -0,0 +1,867 @@
"""Song routes: upload / delete / metadata (user-meta, overrides, catalog meta
write-back), gap-fill proposals, and the per-song info payload.
Extracted verbatim from server.py (R3) except @app->@router and the seam reads:
meta_db->appstate.meta_db, and the scan/ingest helpers that stay in server.py
(the scan lifecycle owns them) -> appstate.<callable>: kick_scan,
invalidate_song_caches, stat_for_cache, scan_status() (a getter — the underlying
dict is reassigned), plus art_override_paths. The gap-fill MBID/ISRC regexes live
in lib/enrichment.py and are reached as enrichment.X.
"""
import os
import shutil
import tempfile
import threading
from pathlib import Path
from fastapi import APIRouter, Request, UploadFile
from fastapi.responses import JSONResponse
from starlette.concurrency import run_in_threadpool
import appstate
import enrichment
import loosefolder as loosefolder_mod
import sloppak as sloppak_mod
from dlc_paths import _get_dlc_dir, _resolve_dlc_path
from scan_worker import _extract_meta_for_file
import logging
log = logging.getLogger("feedBack.server")
router = APIRouter()
_ALLOWED_SONG_EXTS = set(sloppak_mod.SONG_EXTS)
_MAX_UPLOAD_BYTES = 1024 * 1024 * 1024 # 1 GB — covers sloppaks bundled with stems
# Per-request batch cap. Lets a user drop a whole album of sloppaks at once
# without giving a hostile client a 1000-file DoS surface via Starlette's
# default max_files=1000. The pre-parse Content-Length guard is sized as
# _MAX_UPLOAD_FILES * _MAX_UPLOAD_BYTES + slack.
_MAX_UPLOAD_FILES = 50
# Serializes the mutating step of upload (os.replace into DLC_DIR) with
# delete_song so the two endpoints can't interleave on the same path —
# e.g. an upload finishing right after a concurrent delete shouldn't
# resurrect a song the user just removed, and a delete arriving mid-
# overwrite shouldn't strand a half-written file. threading.Lock (not
# asyncio.Lock) because delete_song is sync (runs in the threadpool);
# upload acquires it inside ``run_in_threadpool`` for the same reason.
_song_io_lock = threading.Lock()
def _commit_uploaded_song(tmp_path: Path, dest: Path, overwrite: bool, base: str):
"""Atomically move a validated temp upload into ``dest`` under ``_song_io_lock``.
Returns ``None`` on success or an error result dict matching the upload
endpoint's contract. Holds the lock across the directory re-check and
the final ``os.replace`` so a concurrent delete or upload can't slip
between them. Always cleans up the temp file on the error paths.
"""
with _song_io_lock:
if dest.exists():
if not overwrite:
# Lost the race against a concurrent upload of the same name.
try:
tmp_path.unlink()
except OSError:
pass
return {"status": "exists", "filename": base,
"error": "A file with this name already exists"}
# Re-check directory state under the lock — the pre-check
# may have raced an unrelated mkdir, and a sloppak directory
# has to be removed before os.replace() can write over it.
if dest.is_dir():
if not sloppak_mod.is_sloppak(dest):
try:
tmp_path.unlink()
except OSError:
pass
return {"status": "exists", "filename": base,
"error": "A directory with this name exists and is not "
"a sloppak — refusing to overwrite"}
shutil.rmtree(str(dest))
os.replace(str(tmp_path), str(dest))
return None
@router.post("/api/songs/upload")
async def upload_song(request: Request):
"""Upload one or more .sloppak files into the configured DLC folder.
Multipart body with one or more ``file`` fields (up to ``_MAX_UPLOAD_FILES``
per request). Query string:
``overwrite=1`` — replace existing files with the same name.
Response shape (always HTTP 200 once we've gotten past request-level guards
like DLC-not-configured / payload-too-large):
``{"results": [{"filename": "...", "status": "ok" | "exists" | "error",
"error"?: "...", "size"?: N, "format"?: "sloppak"}, ...]}``
Per-file conflicts surface as ``status: "exists"`` so a batch upload can
surface ALL conflicts at once instead of bailing on the first one. The
client re-POSTs just the conflicting files with ``overwrite=1`` if the
user opts in.
The DLC directory is resolved via ``_get_dlc_dir()`` which honours the
``DLC_DIR`` env var first and falls back to ``dlc_dir`` in
``config.json`` — so uploads land in whichever folder the rest of the
app already considers the library root, regardless of which mechanism
configured it.
"""
dlc = _get_dlc_dir()
if dlc is None:
return JSONResponse(
{"error": "DLC folder is not configured. Set DLC_DIR or configure it in Settings."},
status_code=503,
)
if not os.access(str(dlc), os.W_OK):
return JSONResponse(
{"error": f"DLC folder {dlc} is not writable by the server process."},
status_code=500,
)
# Pre-parse Content-Length guard — fail fast before reading any body.
# Multipart Content-Length is file bytes + boundary + per-part headers, so
# we can't use _MAX_UPLOAD_BYTES as an exact cap here (a file right at the
# advertised max would be rejected before _save_uploaded_song() can apply
# the real per-file byte cap). For batch uploads we allow up to
# _MAX_UPLOAD_FILES files at _MAX_UPLOAD_BYTES each; the parser still
# enforces per-part size via max_part_size and per-batch count via
# max_files. The streaming check inside _save_uploaded_song() is the
# authoritative per-file size cap.
max_total = _MAX_UPLOAD_FILES * _MAX_UPLOAD_BYTES + enrichment._MULTIPART_OVERHEAD_SLACK
cl = request.headers.get("content-length")
if cl is not None:
try:
cl_int = int(cl)
except ValueError:
return JSONResponse({"error": "Invalid Content-Length header"}, status_code=400)
if cl_int < 0:
return JSONResponse({"error": "Invalid Content-Length header"}, status_code=400)
if cl_int > max_total:
return JSONResponse(
{"error": f"Batch upload exceeds {_MAX_UPLOAD_FILES} files × "
f"{_MAX_UPLOAD_BYTES // (1024 * 1024)} MB limit"},
status_code=413,
)
overwrite = request.query_params.get("overwrite") == "1"
# Tighten the parser to the handler's contract: up to _MAX_UPLOAD_FILES
# file parts, no text parts (overwrite comes from query params).
# Starlette's defaults of max_files=1000 / max_fields=1000 would
# otherwise let a client force the parser to spool far more parts than
# the endpoint is willing to process.
form = await request.form(
max_files=_MAX_UPLOAD_FILES,
max_fields=0,
max_part_size=_MAX_UPLOAD_BYTES,
)
try:
from starlette.datastructures import UploadFile as _StarletteUploadFile
# form.getlist("file") returns all parts named "file" in submission
# order. Filter to file parts only — Starlette would yield strings
# for text parts, but we've capped max_fields=0 so any non-file part
# is already a parser error before reaching here.
uploads = [u for u in form.getlist("file") if isinstance(u, _StarletteUploadFile)]
if not uploads:
return JSONResponse(
{"error": "Expected one or more files in multipart field 'file'"},
status_code=400,
)
results = []
any_saved = False
for upload in uploads:
try:
result = await _save_uploaded_song(upload, dlc, overwrite)
results.append(result)
if result.get("status") == "ok":
any_saved = True
except Exception as e:
# Per-file failure must not abort the batch — record and
# continue so the client gets a complete report.
log.exception("upload failed for %r", getattr(upload, "filename", "?"))
results.append({
"filename": Path(getattr(upload, "filename", "") or "").name or "?",
"status": "error",
"error": f"Upload failed: {e}",
})
finally:
try:
await upload.close()
except Exception:
log.debug("failed to close upload file handle", exc_info=True)
if any_saved:
appstate.kick_scan()
return {"results": results}
finally:
try:
await form.close()
except Exception:
log.debug("failed to close form", exc_info=True)
async def _save_uploaded_song(upload: UploadFile, dlc: Path, overwrite: bool) -> dict:
"""Save one upload into ``dlc``. Returns a per-file result dict (never
a JSONResponse) so batch uploads can aggregate.
Shape:
ok: ``{"status": "ok", "filename": base, "size": N, "format": "sloppak"}``
exists: ``{"status": "exists", "filename": base, "error": "..."}``
error: ``{"status": "error", "filename": base, "error": "..."}``
"""
# Strip any path components a client may have included in the filename —
# only the basename lands in the DLC root. Path traversal would otherwise
# let a crafted upload escape the library directory.
raw_name = upload.filename or ""
base = Path(raw_name).name
if not base or base in (".", "..") or "/" in base or "\\" in base:
return {"status": "error", "filename": raw_name or "?", "error": "Invalid filename"}
suffix = Path(base).suffix.lower()
if suffix not in _ALLOWED_SONG_EXTS:
return {"status": "error", "filename": base,
"error": "Only .feedpak files are accepted"}
dest = dlc / base
if dest.exists():
if not overwrite:
return {"status": "exists", "filename": base,
"error": "A file with this name already exists"}
# overwrite=1 must handle directory-form sloppaks (the scanner and
# delete path both treat them as song entries). os.replace() can't
# clobber a non-empty directory, so without the rmtree below the
# whole upload would write to a temp file and then surface a late
# 500 at the os.replace() call. Refuse other directories so an
# unrelated folder isn't blown away by a same-named upload.
if dest.is_dir() and not sloppak_mod.is_sloppak(dest):
return {"status": "exists", "filename": base,
"error": "A directory with this name exists and is not a sloppak — "
"refusing to overwrite"}
# Temp file in the DLC dir itself so os.replace is atomic (same filesystem).
# Dot-prefix keeps it out of the rglob("*.sloppak") scan glob.
fd, tmp_name = await run_in_threadpool(
tempfile.mkstemp, dir=str(dlc), prefix=".upload-", suffix=".part"
)
tmp_path = Path(tmp_name)
bytes_read = 0
head = b""
error_result: dict | None = None
try:
try:
tmpf = await run_in_threadpool(os.fdopen, fd, "wb")
except BaseException:
try:
await run_in_threadpool(os.close, fd)
except OSError:
pass
raise
try:
while True:
chunk = await upload.read(1024 * 1024)
if not chunk:
break
bytes_read += len(chunk)
if bytes_read > _MAX_UPLOAD_BYTES:
error_result = {
"status": "error", "filename": base,
"error": f"Upload exceeds {_MAX_UPLOAD_BYTES // (1024 * 1024)} MB cap",
}
break
if len(head) < 4:
head += chunk[: 4 - len(head)]
await run_in_threadpool(tmpf.write, chunk)
finally:
await run_in_threadpool(tmpf.close)
if error_result is None:
if bytes_read == 0:
error_result = {"status": "error", "filename": base,
"error": "Empty upload — file is 0 bytes"}
elif suffix in _ALLOWED_SONG_EXTS:
if head[:2] != b"PK":
error_result = {"status": "error", "filename": base,
"error": "Not a valid feedpak file (expected zip archive)"}
else:
# ZIP magic alone admits any renamed zip — verify the sloppak
# loader can actually parse a manifest.yaml inside. Without
# this, /api/songs/upload returns "ok" for files the rest of
# the backend would refuse to scan or load.
try:
await run_in_threadpool(sloppak_mod.load_manifest, tmp_path)
except Exception as e:
error_result = {"status": "error", "filename": base,
"error": f"Not a valid sloppak file: {e}"}
if error_result is not None:
try:
await run_in_threadpool(tmp_path.unlink)
except OSError:
pass
return error_result
# Single sync helper so the lock is held for the whole commit —
# ``async with _upload_lock`` would have released between every
# ``run_in_threadpool`` and let a concurrent delete or upload slip
# in between the dir check and the final ``os.replace``.
commit_result = await run_in_threadpool(
_commit_uploaded_song, tmp_path, dest, overwrite, base
)
if commit_result is not None:
return commit_result
except BaseException:
try:
await run_in_threadpool(tmp_path.unlink)
except OSError:
pass
raise
# Even on a fresh (non-overwrite) upload, evict any stale entries left
# over from a previous delete+re-upload of the same name.
await run_in_threadpool(appstate.invalidate_song_caches, base)
log.info("Uploaded %s (%d bytes) to %s", base, bytes_read, dlc)
return {"status": "ok", "filename": base, "size": bytes_read,
"format": suffix.lstrip(".")}
@router.delete("/api/song/{filename:path}")
def delete_song(filename: str):
"""Remove a song from the DLC folder and clear its cache entries.
Works for both formats: ``.sloppak`` files OR directories, and
loose-folder songs (the directory containing the chart). The path is
resolved through ``_resolve_dlc_path`` so URL-encoded ``..`` segments
cannot escape the library root.
"""
dlc = _get_dlc_dir()
if dlc is None:
return JSONResponse({"error": "DLC folder not configured"}, status_code=503)
resolved = _resolve_dlc_path(dlc, filename)
if resolved is None:
return JSONResponse({"error": "forbidden"}, status_code=403)
if not resolved.exists():
return JSONResponse({"error": "File not found"}, status_code=404)
if resolved == dlc.resolve():
return JSONResponse({"error": "Refusing to delete the DLC root"}, status_code=400)
# Only delete actual song entries. Without this, DELETE /api/song/ArtistName
# would recursively wipe a whole artist subfolder — far broader than the
# UI's per-song contract. Sloppak detection wins over loose because a
# sloppak dir can also contain WEM/XML (matches the scanner's precedence).
is_sloppak = sloppak_mod.is_sloppak(resolved)
is_loose = (
resolved.is_dir()
and not is_sloppak
and loosefolder_mod.is_loose_song(resolved)
)
if not (is_sloppak or is_loose):
return JSONResponse(
{"error": "Not a song entry — only sloppaks "
"or loose-folder songs can be deleted"},
status_code=400,
)
# Hold ``_song_io_lock`` across the filesystem removal AND the DB/cache
# eviction. Without it, an upload of the same filename could ``os.replace``
# a new file into place between our removal and DB delete, leaving the
# new generation stranded with no library row; or the reverse, where
# delete runs between an upload's directory check and its replace and
# the upload then resurrects the song we just removed.
with _song_io_lock:
try:
if resolved.is_dir():
shutil.rmtree(resolved)
else:
resolved.unlink()
except OSError as e:
log.error("Failed to delete %s: %s", resolved, e)
return JSONResponse({"error": f"Delete failed: {e}"}, status_code=500)
# Canonicalise the cache key the same way update_song_meta does so we
# hit the row the scanner indexed under.
try:
cache_key = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
cache_key = filename
with appstate.meta_db._lock:
appstate.meta_db.conn.execute("DELETE FROM songs WHERE filename = ?", (cache_key,))
appstate.meta_db.conn.execute("DELETE FROM favorites WHERE filename = ?", (cache_key,))
appstate.meta_db.conn.execute("DELETE FROM loops WHERE filename = ?", (cache_key,))
# Purge the v3 filename-keyed state too, so the deleted song stops
# surfacing in stats / recent / continue / playlists immediately.
appstate.meta_db.conn.execute("DELETE FROM song_stats WHERE filename = ?", (cache_key,))
appstate.meta_db.conn.execute("DELETE FROM playlist_songs WHERE filename = ?", (cache_key,))
# Personal difficulty / notes / tags for this song (we hold the
# lock, so purge is lock-free).
appstate.meta_db.purge_song_user_data(cache_key)
# Multi-chart grouping (P5a): drop this chart's split + read-model rows,
# and any preferred-chart pointer that named it (the work re-auto-picks).
# work_key-keyed prefs for OTHER charts survive. Mark the read-model
# dirty so the affected work regroups on the next grouped query.
appstate.meta_db.conn.execute("DELETE FROM chart_group_split WHERE filename = ?", (cache_key,))
appstate.meta_db.conn.execute("DELETE FROM work_display WHERE filename = ?", (cache_key,))
appstate.meta_db.conn.execute("DELETE FROM chart_group_pref WHERE preferred_filename = ?", (cache_key,))
appstate.meta_db._work_display_dirty = True
# Enrichment is never purged on rescan (delete_missing), only here
# on the explicit per-song delete — the never-clobber contract.
appstate.meta_db.conn.execute("DELETE FROM song_enrichment WHERE filename = ?", (cache_key,))
appstate.meta_db.conn.commit()
# User art overrides go with the song (CAA cache files are keyed by
# RELEASE and may be shared with other charts — the LRU owns those).
for _p in appstate.art_override_paths(cache_key):
try:
_p.unlink()
except OSError:
pass
appstate.invalidate_song_caches(cache_key)
log.info("Deleted song %s", cache_key)
# If a scan was mid-flight when we removed the row, it may already have
# listed (and not yet processed) the file and will call ``appstate.meta_db.put()``
# for it after our DB delete — reinserting a ghost row. Coalesce a
# follow-up pass via ``appstate.kick_scan`` so the next scan's ``delete_missing()``
# purges that entry. Cheap no-op when no scan is running.
if appstate.scan_status()["running"]:
appstate.kick_scan()
return {"ok": True, "filename": cache_key}
@router.get("/api/song/{filename:path}/user-meta")
def get_song_user_meta(filename: str):
"""Read {user_difficulty, notes, tags} for one song."""
return appstate.meta_db.get_song_user_meta(appstate.meta_db._canonical_song_filename(filename))
@router.put("/api/song/{filename:path}/user-meta")
def put_song_user_meta(filename: str, data: dict):
"""Partial update. Send any of: `user_difficulty` (int 15, or null/"" to
clear), `notes` (string, or null to clear), `tags` (a full-replace array of
strings). Omitted keys are preserved. Returns the merged meta.
Tag removal is a full-replace `tags` array (send the new set) rather than a
granular DELETE sub-route, because `DELETE /api/song/{filename:path}` already
owns every DELETE under /api/song and would shadow it."""
key = appstate.meta_db._canonical_song_filename(filename)
kwargs: dict = {}
if "user_difficulty" in data:
v = data["user_difficulty"]
if v is None or v == "":
kwargs["user_difficulty"] = None
else:
# Reject bools (int subclass) and non-integral floats so 2.5 / true
# can't silently truncate into a valid band.
if isinstance(v, bool) or (isinstance(v, float) and not v.is_integer()):
return JSONResponse({"error": "user_difficulty must be an integer 15 or null"}, 400)
try:
iv = int(v)
except (TypeError, ValueError):
return JSONResponse({"error": "user_difficulty must be an integer 15 or null"}, 400)
if not (1 <= iv <= 5):
return JSONResponse({"error": "user_difficulty must be 15 or null"}, 400)
kwargs["user_difficulty"] = iv
if "notes" in data:
n = data["notes"]
if n is None:
kwargs["notes"] = None
elif isinstance(n, str):
kwargs["notes"] = n.strip()[:4000]
else:
return JSONResponse({"error": "notes must be a string or null"}, 400)
tags = data.get("tags", "__absent__")
if tags != "__absent__" and not isinstance(tags, list):
return JSONResponse({"error": "tags must be an array of strings"}, 400)
if not kwargs and tags == "__absent__":
return JSONResponse({"error": "No fields to update"}, 400)
if kwargs:
appstate.meta_db.set_song_user_meta(key, **kwargs)
if tags != "__absent__":
appstate.meta_db.set_song_tags(key, tags)
return appstate.meta_db.get_song_user_meta(key)
# Catalog fields the Fix-metadata popup may override/lock — the intersection of
# "displayable identity" and "safe to correct locally". Guitar/practice facts
# and personal fields are never overrides.
_OVERRIDE_FIELDS = frozenset({"title", "artist", "album", "year", "genre"})
@router.get("/api/song/{filename:path}/overrides")
def get_song_overrides(filename: str):
"""Per-field metadata overrides + locks for one song (Fix-metadata popup):
{"overrides": {field: {"value": str|null, "locked": bool}},
"pack": {field: str}}. `pack` is the stored value each override sits on top
of — the popup's Details tab renders it as the revert-to-pack reference and
the Yours/Pack provenance."""
key = appstate.meta_db._canonical_song_filename(filename)
return {"overrides": appstate.meta_db.get_song_overrides(key),
"pack": appstate.meta_db.pack_fields(key)}
@router.put("/api/song/{filename:path}/overrides")
def put_song_overrides(filename: str, data: dict):
"""Set/clear per-field overrides + locks. Body:
`{"overrides": {field: {"value": str|null, "locked": bool}}}`. Only catalog
fields (title/artist/album/year/genre) are accepted. A field left with no
value and unlocked is removed. Returns the merged override map.
Clearing rides this PUT (send value:null, locked:false) rather than a DELETE
sub-route, because `DELETE /api/song/{filename:path}` already owns every
DELETE under /api/song and would shadow it (same reason as tags)."""
ov = (data or {}).get("overrides")
if not isinstance(ov, dict) or not ov:
return JSONResponse({"error": "overrides must be a non-empty object"}, 400)
bad = sorted(f for f in ov if f not in _OVERRIDE_FIELDS)
if bad:
return JSONResponse({"error": "unknown field(s): " + ", ".join(bad)}, 400)
key = appstate.meta_db._canonical_song_filename(filename)
for field, spec in ov.items():
if not isinstance(spec, dict):
return JSONResponse({"error": f"'{field}' must be an object with value/locked"}, 400)
kwargs: dict = {}
if "value" in spec:
v = spec["value"]
if v is None:
kwargs["value"] = None
elif isinstance(v, (str, int, float)) and not isinstance(v, bool):
kwargs["value"] = str(v).strip()[:500]
else:
return JSONResponse({"error": f"'{field}' value must be a string or null"}, 400)
if "locked" in spec:
kwargs["locked"] = bool(spec["locked"])
if kwargs:
appstate.meta_db.set_song_override(key, field, **kwargs)
return {"overrides": appstate.meta_db.get_song_overrides(key)}
@router.post("/api/songs/user-meta/batch")
def batch_song_user_meta(data: dict):
"""Bulk personal-meta edit over a selection — one request instead of N×2
per-song round-trips (the batch bar's apply-to-all). DB-only; never touches
files. Body:
{"filenames": [...], # required, non-empty
"set_difficulty": 1-5 | null, # optional: set on all / clear on all
"add_tags": [...], # optional: add to all (never full-replace)
"remove_tags": [...]} # optional: remove from all
Omit `set_difficulty` entirely to leave each song's difficulty as-is
(mixed-state "leave unchanged"). Returns {"updated": N, "tags": [...]} so the
caller can refresh the tag-filter list without a second call."""
fns = data.get("filenames")
if not isinstance(fns, list) or not fns:
return JSONResponse({"error": "filenames must be a non-empty array"}, 400)
if not all(isinstance(f, str) and f for f in fns):
return JSONResponse({"error": "filenames must be non-empty strings"}, 400)
kwargs: dict = {}
if "set_difficulty" in data:
v = data["set_difficulty"]
if v is None or v == "":
kwargs["set_difficulty"] = None
else:
if isinstance(v, bool) or (isinstance(v, float) and not v.is_integer()):
return JSONResponse({"error": "set_difficulty must be an integer 15 or null"}, 400)
try:
iv = int(v)
except (TypeError, ValueError):
return JSONResponse({"error": "set_difficulty must be an integer 15 or null"}, 400)
if not (1 <= iv <= 5):
return JSONResponse({"error": "set_difficulty must be 15 or null"}, 400)
kwargs["set_difficulty"] = iv
add_tags = data.get("add_tags")
remove_tags = data.get("remove_tags")
for name, val in (("add_tags", add_tags), ("remove_tags", remove_tags)):
if val is not None and not isinstance(val, list):
return JSONResponse({"error": f"{name} must be an array of strings"}, 400)
if "set_difficulty" not in data and not add_tags and not remove_tags:
return JSONResponse({"error": "Nothing to apply"}, 400)
keys = [appstate.meta_db._canonical_song_filename(f) for f in fns]
n = appstate.meta_db.batch_user_meta(keys, add_tags=add_tags, remove_tags=remove_tags, **kwargs)
return {"updated": n, "tags": appstate.meta_db.all_tags()}
@router.post("/api/song/{filename:path}/meta")
def update_song_meta(filename: str, data: dict):
"""Update song metadata, persisting it back into the underlying file.
The library scanner re-derives title/artist/album/year from the file
(archive manifest Attributes / sloppak manifest.yaml) on every full rescan,
so a DB-only edit reverts. We write the edit into the file first, then
refresh the cache row (including mtime/size) to match. Loose-folder and
unwritable songs fall back to a DB-only update (which still survives an
incremental rescan via the mtime/size cache hit).
"""
# Canonicalise to the same key get_song_info uses so an update via
# one URL form (e.g. with `..` segments) lands on the row that
# later reads will see.
dlc = _get_dlc_dir()
cache_key = filename
resolved = None
if dlc:
resolved = _resolve_dlc_path(dlc, filename)
if resolved is None:
return JSONResponse({"error": "forbidden"}, 403)
try:
cache_key = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
pass
fields = {k: data[k] for k in ("title", "artist", "album", "year") if k in data}
if not fields:
return {"error": "No fields to update"}
# Normalise the year value so the DB and file stay in sync. The file
# writer (songmeta) coerces empty/non-numeric years to 0, which the
# scanner reads back as "". Store "" in the DB instead of a raw
# non-numeric string so that if the mtime/size are updated (making the
# row cache-fresh) the DB still matches what the scanner would derive.
if "year" in fields:
try:
_yr_int = int(fields["year"])
except (TypeError, ValueError):
_yr_int = 0
fields = {**fields, "year": str(_yr_int) if _yr_int else ""}
# Persist into the file so the edit survives a full rescan.
# Hold _song_io_lock across the existence check and file write so a
# concurrent delete cannot remove the file between our check and the
# repack's atomic replace, and so a concurrent upload cannot be clobbered
# by our atomic rename. archive repack is slow — the lock is held longer
# than a simple upload/delete, but correctness requires serialisation.
persisted = False
with _song_io_lock:
if resolved is not None and resolved.exists():
try:
import songmeta
persisted = songmeta.write_song_metadata(resolved, fields)
except Exception:
log.warning("metadata file write failed for %s", cache_key, exc_info=True)
with appstate.meta_db._lock:
updates = [f"{field} = ?" for field in fields]
params = list(fields.values())
if persisted:
# The file changed — re-stat so an incremental rescan sees a
# consistent cache row instead of re-reading the (now matching)
# file.
try:
mtime, size = appstate.stat_for_cache(resolved)
updates += ["mtime = ?", "size = ?"]
params += [mtime, size]
except OSError:
pass
params.append(cache_key)
appstate.meta_db.conn.execute(
f"UPDATE songs SET {', '.join(updates)} WHERE filename = ?", params
)
appstate.meta_db.conn.commit()
if persisted:
appstate.invalidate_song_caches(cache_key)
# Coalesce a follow-up scan so a mid-flight scan's stale appstate.meta_db.put()
# for this file can't win: if a scan is running appstate.kick_scan() queues a
# pending pass; if not it starts a fresh one. Unconditional to avoid a
# race where the scan finishes between our DB commit and a guarded check.
appstate.kick_scan()
return {"ok": True, "persisted": persisted}
# ── Gap-fill: write CONFIRMED missing metadata into the pack (R4a) ────────────
# The agreed write-back contract (spec-alignment §7): opt-in + user-initiated
# (nothing here runs in the background), adds ABSENT keys only (never replaces
# an author-set value — the writer refuses, and existing manifest bytes are
# preserved verbatim by appending), spec'd-keys allowlist, values only from a
# CONFIRMED identity (an auto/exact match or a user pin — review-tier rows are
# not eligible until a human confirms), atomic write + .bak. Single-song only;
# batch write-back stays an open question with the spec chair.
_GAP_FILL_KEYS = ("album", "year", "genres", "mbid", "isrc")
def _gap_fill_manifest_absent(manifest: dict, key: str) -> bool:
"""A key is a GAP only when it's genuinely MISSING from the manifest.
Gap-fill is append-only: the writer's never-clobber guard raises on ANY
key already present, and appending a second `album:` line to a manifest
that already carries `album: ''` would just create a duplicate YAML key.
So a present-but-empty value (None / '' / [] / year 0) is NOT a gap the
append-only writer can fill — offering it in the preview would only lead
to a POST the writer refuses. Present-but-empty keys are therefore left
to the metadata editor (which re-serializes and can replace in place)."""
return key not in manifest
def _gap_fill_proposals(cache_key: str, resolved) -> tuple[dict, str]:
"""What gap-fill could add for this song: (proposals, reason). Empty
proposals explain themselves via reason — 'not-sloppak', 'no-match'
(nothing confirmed yet), 'review' (a human hasn't confirmed the match),
or 'nothing-missing'."""
if resolved is None or not resolved.exists() or not sloppak_mod.is_sloppak(resolved):
return {}, "not-sloppak"
row = appstate.meta_db.get_enrichment(cache_key)
if not row or row.get("match_state") not in ("matched", "manual"):
state = (row or {}).get("match_state")
return {}, ("review" if state == "review" else "no-match")
try:
manifest = sloppak_mod.load_manifest(resolved) or {}
except Exception:
return {}, "not-sloppak"
# A LOCKED field (Fix-metadata popup) is never gap-filled — the user pinned
# it away from the matched value, so writing that value to the file would
# be exactly the clobber the lock exists to prevent. (The lock field name is
# `genre`; the manifest/gap-fill key is `genres`.)
locked = appstate.meta_db.locked_fields(cache_key)
out = {}
album = (row.get("canon_album") or "").strip()
if album and "album" not in locked and _gap_fill_manifest_absent(manifest, "album"):
out["album"] = album
year = (row.get("canon_year") or "").strip()
if (year.isdigit() and int(year) and "year" not in locked
and _gap_fill_manifest_absent(manifest, "year")):
out["year"] = int(year)
genres = [str(g) for g in (row.get("genres") or []) if isinstance(g, str) and g.strip()]
if genres and "genre" not in locked and _gap_fill_manifest_absent(manifest, "genres"):
out["genres"] = genres
# Identity keys (feedpak spec 1.14.0) — written in canonical form only.
mbid = (row.get("mb_recording_id") or "").strip().lower()
if enrichment._MBID_RE.match(mbid) and _gap_fill_manifest_absent(manifest, "mbid"):
out["mbid"] = mbid
isrc = (row.get("isrc") or "").strip().upper().replace("-", "").replace(" ", "")
if enrichment._ISRC_RE.match(isrc) and _gap_fill_manifest_absent(manifest, "isrc"):
out["isrc"] = isrc
return out, ("" if out else "nothing-missing")
@router.get("/api/song/{filename:path}/gap-fill")
def get_song_gap_fill(filename: str):
"""Preview what "Write missing info to file" would add — the Details
drawer renders its confirm list straight from this. Read-only."""
dlc = _get_dlc_dir()
cache_key, resolved = filename, None
if dlc:
resolved = _resolve_dlc_path(dlc, filename)
if resolved is None:
return JSONResponse({"error": "forbidden"}, 403)
try:
cache_key = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
pass
proposals, reason = _gap_fill_proposals(cache_key, resolved)
row = appstate.meta_db.get_enrichment(cache_key) or {}
return {
"eligible": bool(proposals),
"reason": reason,
"match_state": row.get("match_state"),
"missing": [{"key": k, "value": v} for k, v in proposals.items()],
}
@router.post("/api/song/{filename:path}/gap-fill")
def post_song_gap_fill(filename: str, data: dict):
"""Write the user-confirmed subset of the preview into the pack file.
Proposals are recomputed under the io lock, so a key that gained an
author value between preview and confirm is skipped, never replaced."""
keys = (data or {}).get("keys")
if not isinstance(keys, list) or not keys:
return JSONResponse({"error": "keys must be a non-empty list"}, 400)
bad = [k for k in keys if k not in _GAP_FILL_KEYS]
if bad:
return JSONResponse(
{"error": "unknown key(s): " + ", ".join(sorted(set(map(str, bad))))}, 400)
dlc = _get_dlc_dir()
cache_key, resolved = filename, None
if dlc:
resolved = _resolve_dlc_path(dlc, filename)
if resolved is None:
return JSONResponse({"error": "forbidden"}, 403)
try:
cache_key = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
pass
with _song_io_lock:
proposals, reason = _gap_fill_proposals(cache_key, resolved)
additions = {k: proposals[k] for k in _GAP_FILL_KEYS if k in keys and k in proposals}
skipped = sorted(set(keys) - set(additions))
if not additions:
return JSONResponse({"error": "nothing to write", "reason": reason,
"skipped": skipped}, 409)
try:
import songmeta
songmeta.gap_fill_sloppak(resolved, additions)
except Exception:
log.warning("gap-fill write failed for %s", cache_key, exc_info=True)
return JSONResponse({"error": "write failed"}, 500)
# Keep the cache row consistent with what the scanner would now derive
# (same contract as the metadata editor above): sync the columns the
# scan reads from the keys we appended, then re-stat so the row stays
# cache-fresh.
fields = {}
if "album" in additions:
fields["album"] = additions["album"]
if "year" in additions:
fields["year"] = str(additions["year"])
if "genres" in additions:
fields["genre"] = additions["genres"][0]
with appstate.meta_db._lock:
updates = [f"{field} = ?" for field in fields]
params = list(fields.values())
try:
mtime, size = appstate.stat_for_cache(resolved)
updates += ["mtime = ?", "size = ?"]
params += [mtime, size]
except OSError:
pass
if updates:
params.append(cache_key)
appstate.meta_db.conn.execute(
f"UPDATE songs SET {', '.join(updates)} WHERE filename = ?", params)
appstate.meta_db.conn.commit()
appstate.invalidate_song_caches(cache_key)
appstate.kick_scan()
return {"ok": True, "written": additions, "skipped": skipped}
@router.get("/api/song/{filename:path}")
async def get_song_info(filename: str):
"""Return song metadata, from cache or by extracting it from the song source."""
import asyncio
dlc = _get_dlc_dir()
if not dlc:
return JSONResponse({"error": "DLC folder not configured"}, 404)
song_path = _resolve_dlc_path(dlc, filename)
if song_path is None:
return JSONResponse({"error": "forbidden"}, 403)
if not song_path.exists():
return JSONResponse({"error": "File not found"}, 404)
# Canonicalise the cache key against the resolved path so two URL
# forms of the same physical file (e.g. `Artist/song.sloppak` vs
# `Artist/../Artist/song.sloppak`) converge on a single row instead
# of fragmenting / shadowing each other in appstate.meta_db.
try:
cache_key = song_path.relative_to(dlc.resolve()).as_posix()
except ValueError:
cache_key = filename
mtime, size = appstate.stat_for_cache(song_path)
cached = appstate.meta_db.get(cache_key, mtime, size)
if cached:
return cached
# Extract in thread pool
def _extract():
meta = _extract_meta_for_file(song_path, dlc)
appstate.meta_db.put(cache_key, mtime, size, meta)
return meta
meta = await asyncio.get_event_loop().run_in_executor(None, _extract)
return meta
+46
View File
@@ -0,0 +1,46 @@
"""The merged tuning catalog (/api/tunings).
Extracted verbatim from server.py (R3) except @app->@router, CONFIG_DIR->
appstate.config_dir, _load_config imported from lib/appconfig, and the tuning
registry read through the appstate seam (appstate.tuning_providers — the same
instance plugins register into via the plugin_context in server.py).
"""
from fastapi import APIRouter
import appstate
from appconfig import _load_config
from tunings import DEFAULT_REFERENCE_PITCH, TUNING_PRESET_MIDIS, freqs_to_midis
router = APIRouter()
@router.get("/api/tunings")
def get_tunings():
cfg = _load_config(appstate.config_dir / "config.json") or {}
ref = cfg.get("reference_pitch", DEFAULT_REFERENCE_PITCH)
try:
ref = float(ref)
if not (430.0 <= ref <= 450.0):
ref = DEFAULT_REFERENCE_PITCH
except (TypeError, ValueError):
ref = DEFAULT_REFERENCE_PITCH
merged = appstate.tuning_providers.get_merged(ref)
# tuningMidis: the same catalog as exact integer MIDI notes (low → high).
# Built-ins come straight from TUNING_PRESET_MIDIS (no float round-trip);
# provider-contributed entries are recovered from their frequencies at the
# served reference pitch. Every consumer today (the v3 badges, plugins)
# reconstructs midis client-side via log2 — a rounding footgun at non-440
# references — so serve the integers once, host-side. Additive: the
# existing referencePitch/tunings shape is unchanged.
tuning_midis: dict[str, dict[str, list[int]]] = {}
for key, names in merged.items():
builtin = TUNING_PRESET_MIDIS.get(key, {})
resolved: dict[str, list[int]] = {}
for name, freqs in names.items():
midis = builtin.get(name) or freqs_to_midis(freqs, ref)
if midis:
resolved[name] = list(midis)
if resolved:
tuning_midis[key] = resolved
return {"referencePitch": ref, "tunings": merged, "tuningMidis": tuning_midis}
+116 -4662
View File
File diff suppressed because it is too large Load Diff
+30
View File
@@ -9,6 +9,36 @@ import structlog
_LOGGING_NAMES = ("feedBack", "uvicorn", "uvicorn.error", "uvicorn.access")
@pytest.fixture(autouse=True)
def _reset_enrichment_state():
"""Reset the enrichment worker's process-global state between tests.
The `server` fixtures pop-and-reimport `server`, but `lib/enrichment.py`
(which now owns the worker) stays imported for the whole session, so its
module globals the cancel Event, the status dict, the caches would
otherwise leak across tests. A test that set `_enrich_cancel` (or a stale
`running` status) could silently short-circuit a later direct
`_background_enrich()` call. Clear it up front so each test starts clean.
"""
try:
import enrichment
except ImportError:
yield
return
enrichment._enrich_cancel.clear()
enrichment._enrich_pending_pass = False
enrichment._enrich_status.update(
{"running": False, "processed": 0, "last_pass_at": None,
"total": 0, "matched": 0, "current": None})
enrichment._enrich_last_fetch = 0.0
enrichment._artist_alias_cache.clear()
# _caa_index_locks is deliberately left alone: it's guarded by
# _caa_index_locks_guard, so clearing it here (unlocked) would race a
# still-alive worker thread, and its entries are stateless per-release
# mutexes that don't leak test state anyway.
yield
@pytest.fixture()
def isolate_logging():
"""Restore feedBack / uvicorn logger state after each test.
+18 -16
View File
@@ -12,6 +12,8 @@ tests/test_art_layer.py.
"""
import importlib
import enrichment
from routers import art
import io as _io
import sys
@@ -119,8 +121,8 @@ def caa_index(server, monkeypatch):
calls.append(release_id)
return indexes.get(release_id) # unknown release → None (a CAA 404)
fake.calls, fake.indexes = calls, indexes
monkeypatch.setattr(server, "_caa_release_index", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_caa_release_index", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
@@ -251,7 +253,7 @@ def test_caa_candidates_capped_at_12(server, client, caa_index):
caa_index.indexes["rel-big"] = {
"images": [_img(300 + i, front=(i == 0)) for i in range(20)]}
_match_row(server, "a.sloppak", release_id="rel-big")
assert len(_caa(_get(client))) == server._ART_PICKER_MAX_CAA == 12
assert len(_caa(_get(client))) == art._ART_PICKER_MAX_CAA == 12
def test_demo_mode_blocks_candidates(server, client, monkeypatch):
@@ -274,10 +276,10 @@ def test_malicious_release_id_rejected_no_fetch_no_write(server, caa_index):
"""A crafted release id (path traversal) never matches _CAA_ID_RE, so it
yields no images, opens no socket, and writes no cache file inside the
art dir or anywhere else."""
art_dir = server._enrichment_art_dir()
art_dir = enrichment._enrichment_art_dir()
before = set(art_dir.glob("*"))
assert not server._CAA_ID_RE.match("../../etc/x")
assert server._caa_index_cached("../../etc/x") == []
assert not enrichment._CAA_ID_RE.match("../../etc/x")
assert enrichment._caa_index_cached("../../etc/x") == []
assert caa_index.calls == [] # the seam was never asked
assert set(art_dir.glob("*")) == before # nothing written
# And nothing landed at the traversal target beside the cache dir either.
@@ -340,10 +342,10 @@ def test_fetch_art_url_follows_redirects_validating_each_hop(server, monkeypatch
return _FakeResp(200, chunks=[b"IMGDATA"])
monkeypatch.setattr(requests, "get", fake_get)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_url_host_is_internal",
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(art, "_url_host_is_internal",
lambda u: (checked.append(u), False)[1])
data = server._fetch_art_url("https://coverartarchive.example/release/x/front-500")
data = art._fetch_art_url("https://coverartarchive.example/release/x/front-500")
assert data == b"IMGDATA"
assert fetched == ["https://coverartarchive.example/release/x/front-500",
"https://archive.example/img.png"]
@@ -354,18 +356,18 @@ def test_fetch_art_url_blocks_redirect_to_internal(server, monkeypatch):
import requests
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
302, {"Location": "http://internal.example/x.png"}))
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_url_host_is_internal",
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(art, "_url_host_is_internal",
lambda u: "internal" in u)
with pytest.raises(ValueError):
server._fetch_art_url("https://public.example/x.png")
art._fetch_art_url("https://public.example/x.png")
def test_fetch_art_url_redirect_budget(server, monkeypatch):
import requests
monkeypatch.setattr(requests, "get", lambda url, **kw: _FakeResp(
307, {"Location": "https://public.example/next.png"}))
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_url_host_is_internal", lambda u: False)
with pytest.raises(server.EnrichTransportError):
server._fetch_art_url("https://public.example/x.png")
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(art, "_url_host_is_internal", lambda u: False)
with pytest.raises(enrichment.EnrichTransportError):
art._fetch_art_url("https://public.example/x.png")
+30 -28
View File
@@ -7,6 +7,8 @@ here opens a socket, and the offline default is itself asserted.
"""
import importlib
import enrichment
from routers import art
import io as _io
import sys
@@ -121,7 +123,7 @@ def test_bad_upload_rejected(server, client):
def test_art_url_fetches_and_overrides(server, client, monkeypatch):
make_sloppak(server, "a.sloppak", with_cover=True)
monkeypatch.setattr(server, "_fetch_art_url", lambda url: png_bytes((9, 9, 9)))
monkeypatch.setattr(art, "_fetch_art_url", lambda url: png_bytes((9, 9, 9)))
body = client.post("/api/song/a.sloppak/art/url",
json={"url": "https://example.com/cover.png"}).json()
assert body == {"ok": True, "kind": "png"}
@@ -138,7 +140,7 @@ def test_art_url_validation(server, client, monkeypatch):
# Oversize → 400 (the seam raises ValueError at the cap).
def _huge(url):
raise ValueError("image larger than 10 MB")
monkeypatch.setattr(server, "_fetch_art_url", _huge)
monkeypatch.setattr(art, "_fetch_art_url", _huge)
assert client.post("/api/song/a.sloppak/art/url",
json={"url": "https://example.com/x.png"}).status_code == 400
@@ -176,15 +178,15 @@ def caa(server, monkeypatch):
calls.append(release_id)
return art.get(release_id)
fake.calls, fake.art = calls, art
monkeypatch.setattr(server, "_caa_http_get", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_caa_http_get", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
def test_caa_fetch_fills_missing_art(server, client, caa):
make_sloppak(server, "a.sloppak") # no pack art
_match_row(server, "a.sloppak")
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["art_state"] == "caa"
assert row["art_cache_path"] and row["art_cache_path"].endswith("caa_rel-1.jpg")
@@ -193,7 +195,7 @@ def test_caa_fetch_fills_missing_art(server, client, caa):
assert r.headers["content-type"] == "image/jpeg"
# Settled: the next pass never re-fetches.
n = len(caa.calls)
server._background_enrich()
enrichment._background_enrich()
assert len(caa.calls) == n
@@ -204,7 +206,7 @@ def test_caa_skips_pack_art_and_dedupes_by_release(server, caa):
_match_row(server, "haspack.sloppak")
_match_row(server, "b.sloppak") # same release as c
_match_row(server, "c.sloppak")
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("haspack.sloppak")["art_state"] == "pack"
assert server.meta_db.get_enrichment("b.sloppak")["art_state"] == "caa"
assert server.meta_db.get_enrichment("c.sloppak")["art_state"] == "caa"
@@ -214,10 +216,10 @@ def test_caa_skips_pack_art_and_dedupes_by_release(server, caa):
def test_caa_404_marks_none(server, caa):
make_sloppak(server, "a.sloppak")
_match_row(server, "a.sloppak", release_id="rel-missing")
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "none"
n = len(caa.calls)
server._background_enrich()
enrichment._background_enrich()
assert len(caa.calls) == n # never re-hammered
@@ -226,13 +228,13 @@ def test_caa_transport_error_leaves_row_unevaluated(server, caa, monkeypatch):
_match_row(server, "a.sloppak")
def _down(release_id):
raise server.EnrichTransportError("down")
monkeypatch.setattr(server, "_caa_http_get", _down)
server._background_enrich()
raise enrichment.EnrichTransportError("down")
monkeypatch.setattr(enrichment, "_caa_http_get", _down)
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
# Network back → next pass completes it.
monkeypatch.setattr(server, "_caa_http_get", caa)
server._background_enrich()
monkeypatch.setattr(enrichment, "_caa_http_get", caa)
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
@@ -240,22 +242,22 @@ def test_offline_default_skips_art_worker(server, monkeypatch):
"""Under the plain test env the whole art phase is skipped with the rest
of the network work."""
calls = []
monkeypatch.setattr(server, "_caa_http_get", lambda rid: calls.append(rid))
monkeypatch.setattr(enrichment, "_caa_http_get", lambda rid: calls.append(rid))
make_sloppak(server, "a.sloppak")
_match_row(server, "a.sloppak")
server._background_enrich()
enrichment._background_enrich()
assert calls == []
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
def test_lru_prune_evicts_oldest_and_resets_rows(server, caa, monkeypatch):
monkeypatch.setattr(server, "_CAA_CACHE_CAP_BYTES", 1) # everything over cap
monkeypatch.setattr(enrichment, "_CAA_CACHE_CAP_BYTES", 1) # everything over cap
make_sloppak(server, "a.sloppak", title="One")
make_sloppak(server, "b.sloppak", title="Two")
caa.art["rel-2"] = png_bytes((1, 1, 1))
_match_row(server, "a.sloppak", release_id="rel-1")
_match_row(server, "b.sloppak", release_id="rel-2")
server._background_enrich()
enrichment._background_enrich()
# With a 1-byte cap every fetch immediately evicts — the rows that pointed
# at evicted files were reset to unevaluated.
caa_files = list(server.ART_CACHE_DIR.glob("caa_*.jpg"))
@@ -282,13 +284,13 @@ def test_delete_override_restores_caa_fallback(server, client, caa):
_match_row(server, "a.sloppak")
# Pin an override BEFORE the art worker runs → the pass stamps art_state='user'.
client.post("/api/song/a.sloppak/art/upload", json={"image": b64(png_bytes())})
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "user"
# Remove it → the row resets to unevaluated…
assert client.delete("/api/art/a.sloppak/override").json()["removed"]
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
# …and the next pass fetches + serves the release's front cover.
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
r = client.get("/api/song/a.sloppak/art")
assert r.status_code == 200
@@ -302,7 +304,7 @@ def test_upload_rejects_unknown_song_and_oversize(server, client):
assert server._art_override_paths("ghost.sloppak") == []
# Oversize decoded payload → 400 (bounds the base64 upload path).
make_sloppak(server, "a.sloppak")
huge = b64(b"\x00" * (server._ART_URL_MAX_BYTES + 1))
huge = b64(b"\x00" * (art._ART_URL_MAX_BYTES + 1))
assert client.post("/api/song/a.sloppak/art/upload",
json={"image": huge}).status_code == 400
@@ -310,10 +312,10 @@ def test_upload_rejects_unknown_song_and_oversize(server, client):
def test_fetch_art_url_blocks_internal_hosts(server):
"""The SSRF guard refuses loopback / link-local / private targets before
any request is made (the real seam, not the faked one)."""
assert server._url_host_is_internal("http://127.0.0.1/x.png")
assert server._url_host_is_internal("http://localhost/x.png")
assert server._url_host_is_internal("http://169.254.169.254/latest/meta-data")
assert server._url_host_is_internal("http://10.0.0.5/x.png")
assert server._url_host_is_internal("http://[::1]/x.png")
assert server._url_host_is_internal("http://nonexistent.invalid/x.png") # unresolvable → closed
assert not server._url_host_is_internal("http://93.184.216.34/x.png") # public literal
assert art._url_host_is_internal("http://127.0.0.1/x.png")
assert art._url_host_is_internal("http://localhost/x.png")
assert art._url_host_is_internal("http://169.254.169.254/latest/meta-data")
assert art._url_host_is_internal("http://10.0.0.5/x.png")
assert art._url_host_is_internal("http://[::1]/x.png")
assert art._url_host_is_internal("http://nonexistent.invalid/x.png") # unresolvable → closed
assert not art._url_host_is_internal("http://93.184.216.34/x.png") # public literal
+5 -4
View File
@@ -10,7 +10,7 @@ Two halves, mirroring the design's split:
* GET /api/artist/{name}/links + POST .../links/refresh the lazy, cached,
opt-in external-links layer. The HTTP transport is a fake over
`server._mb_http_get` (the ONE network seam same pattern as
`enrichment._mb_http_get` (the ONE network seam same pattern as
tests/test_mb_enrichment.py), so nothing here opens a socket. Covers the
url-rel whitelist mapping, the http(s) scheme gate (a hostile javascript:
resource never reaches a link slot), cache-hit second calls making no
@@ -19,6 +19,7 @@ Two halves, mirroring the design's split:
"""
import importlib
import enrichment
import json
import sys
from urllib.parse import quote
@@ -88,7 +89,7 @@ class FakeMBArtist:
def __call__(self, path, params):
if self.raise_transport:
raise self._srv.EnrichTransportError("fake network down")
raise enrichment.EnrichTransportError("fake network down")
self.calls.append((path, dict(params)))
if path == f"artist/{MBID}":
return self.doc
@@ -100,8 +101,8 @@ def mb_artist(server, monkeypatch):
"""Install the fake transport AND enable the network flag (the test env
disables it by default see test_links_offline_returns_empty)."""
fake = FakeMBArtist(server)
monkeypatch.setattr(server, "_mb_http_get", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
+2
View File
@@ -25,6 +25,7 @@ def client_and_server(tmp_path, monkeypatch):
static_tmp = tmp_path / "static"
static_tmp.mkdir()
monkeypatch.setattr(server, "STATIC_DIR", static_tmp)
monkeypatch.setattr(server.appstate, "static_dir", static_tmp)
# Pass client=("127.0.0.1", 50000) so request.client.host is a loopback address
test_client = TestClient(server.app, client=("127.0.0.1", 50000))
try:
@@ -123,6 +124,7 @@ def dlc_client(tmp_path, monkeypatch):
static_tmp = tmp_path / "static"
static_tmp.mkdir()
monkeypatch.setattr(server, "STATIC_DIR", static_tmp)
monkeypatch.setattr(server.appstate, "static_dir", static_tmp)
tc = TestClient(server.app, client=("127.0.0.1", 50000))
try:
yield tc, server, dlc
+2 -1
View File
@@ -7,6 +7,7 @@ result, not stored songs.
"""
import importlib
import library_registry
import sys
import pytest
@@ -133,7 +134,7 @@ def test_collection_tolerates_corrupt_persisted_rules(client, server_mod):
('{"artist": [], "sort": [], "tunings": ["Drop D"]}', cid),
)
server_mod.meta_db.conn.commit()
server_mod._sync_collection_provider(server_mod.meta_db.get_collection(cid))
library_registry._sync_collection_provider(server_mod.meta_db.get_collection(cid))
r = client.get("/api/library", params={"provider": f"collection:{cid}"})
assert r.status_code == 200 # no 500/503 from bad rules
+4 -3
View File
@@ -4,6 +4,7 @@ contents). The refresh flow reuses the P8 fake-transport pattern — nothing
here opens a socket."""
import importlib
import enrichment
import sys
import pytest
@@ -85,9 +86,9 @@ def test_refresh_resets_even_a_manual_pin_and_rematches(server, client, monkeypa
"status": "Official", "date": "1990-09-24",
"release-group": {"primary-type": "Album"}}],
}]}
monkeypatch.setattr(server, "_mb_http_get", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
server._background_enrich()
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["mb_recording_id"] == "rec-new"
+33 -32
View File
@@ -5,6 +5,7 @@ contracts it will inherit: rename-survivable idempotent hashing, manual rows
never auto-reset, never purged on rescan, purged on explicit delete."""
import importlib
import enrichment
import sys
import pytest
@@ -58,7 +59,7 @@ def test_pending_covers_new_unscanned_and_changed(server):
_put(server, "a.archive")
assert [r["filename"] for r in server.meta_db.enrichment_pending()] == ["a.archive"]
# stubbed → still unscanned → still pending (the matcher hasn't run)
server._background_enrich()
enrichment._background_enrich()
assert [r["filename"] for r in server.meta_db.enrichment_pending()] == ["a.archive"]
# a matched row with the CURRENT hash is settled…
h = server.meta_db.enrichment_content_hash("Artist", "Song", "", 100)
@@ -76,7 +77,7 @@ def test_pending_covers_new_unscanned_and_changed(server):
def test_hash_change_resets_matched_but_never_manual(server):
_put(server, "a.archive")
_put(server, "b.archive", title="Other")
server._background_enrich()
enrichment._background_enrich()
with server.meta_db._lock:
server.meta_db.conn.execute(
"UPDATE song_enrichment SET match_state = 'matched' WHERE filename = 'a.archive'")
@@ -86,7 +87,7 @@ def test_hash_change_resets_matched_but_never_manual(server):
# identity edits…
_put(server, "a.archive", title="Song v2")
_put(server, "b.archive", title="Other v2")
server._background_enrich()
enrichment._background_enrich()
a = server.meta_db.get_enrichment("a.archive")
b = server.meta_db.get_enrichment("b.archive")
# …drop a stale MATCH back to unscanned with the fresh hash
@@ -99,7 +100,7 @@ def test_hash_change_resets_matched_but_never_manual(server):
def test_failed_rows_not_requeued_by_pending(server):
_put(server, "a.archive")
server._background_enrich()
enrichment._background_enrich()
with server.meta_db._lock:
server.meta_db.conn.execute(
"UPDATE song_enrichment SET match_state = 'failed' WHERE filename = 'a.archive'")
@@ -113,7 +114,7 @@ def test_failed_rows_not_requeued_by_pending(server):
def test_enrich_pass_stamps_every_song(server):
for i in range(5):
_put(server, f"s{i}.archive", title=f"Song {i}")
server._background_enrich()
enrichment._background_enrich()
for i in range(5):
row = server.meta_db.get_enrichment(f"s{i}.archive")
assert row is not None
@@ -126,7 +127,7 @@ def test_enrich_pass_stamps_every_song(server):
def test_rescan_never_purges_enrichment(server):
_put(server, "a.archive")
server._background_enrich()
enrichment._background_enrich()
server.meta_db.delete_missing(set()) # file vanished from a scan snapshot
assert server.meta_db.get_enrichment("a.archive") is not None # row survives
# …and is invisible in the read-time-filtered counts
@@ -138,7 +139,7 @@ def test_rescan_never_purges_enrichment(server):
def test_status_endpoint_counts(client, server):
_put(server, "a.archive")
_put(server, "b.archive", title="Other")
server._background_enrich()
enrichment._background_enrich()
body = client.get("/api/enrichment/status").json()
assert body["states"] == {"unscanned": 2}
assert body["total_songs"] == 2
@@ -147,7 +148,7 @@ def test_status_endpoint_counts(client, server):
def test_art_cache_dir_created(server):
d = server._enrichment_art_dir()
d = enrichment._enrichment_art_dir()
assert d.is_dir()
assert d.name == "art_cache"
@@ -156,7 +157,7 @@ def test_art_cache_dir_created(server):
def test_states_for_returns_only_known_filenames(server):
_put(server, "a.archive")
server._background_enrich()
enrichment._background_enrich()
got = server.meta_db.enrichment_states_for(["a.archive", "nope.archive"])
assert got == {"a.archive": "unscanned"} # unknown filename absent
assert server.meta_db.enrichment_states_for([]) == {}
@@ -165,7 +166,7 @@ def test_states_for_returns_only_known_filenames(server):
def test_states_endpoint(client, server):
_put(server, "a.archive")
_put(server, "b.archive", title="Other")
server._background_enrich()
enrichment._background_enrich()
body = client.post("/api/enrichment/states",
json={"filenames": ["a.archive", "zzz.missing"]}).json()
assert body["states"] == {"a.archive": "unscanned"}
@@ -175,7 +176,7 @@ def test_states_endpoint(client, server):
def test_status_exposes_progress_fields(client, server):
_put(server, "a.archive")
server._background_enrich()
enrichment._background_enrich()
body = client.get("/api/enrichment/status").json()
for k in ("total", "matched", "current", "cancelling"):
assert k in body
@@ -186,7 +187,7 @@ def test_cancel_is_noop_when_idle(client, server):
body = client.post("/api/enrichment/cancel").json()
assert body == {"ok": True, "was_running": False}
# A no-op must not arm the flag (which would then poison the next pass).
assert server._enrich_cancel.is_set() is False
assert enrichment._enrich_cancel.is_set() is False
def test_cancel_flag_halts_matching_loop_between_songs(server, monkeypatch):
@@ -195,28 +196,28 @@ def test_cancel_flag_halts_matching_loop_between_songs(server, monkeypatch):
# Force the matcher path on (the test env is offline by default) and stub the
# per-song matcher so nothing touches the network — it just trips Stop after
# the first song, exactly as the /cancel route would mid-pass.
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
calls = []
def fake_enrich_one(row, **_kw):
calls.append(row["filename"])
server._enrich_cancel.set()
enrichment._enrich_cancel.set()
monkeypatch.setattr(server, "_enrich_one", fake_enrich_one)
server._enrich_cancel.clear()
server._background_enrich()
monkeypatch.setattr(enrichment, "_enrich_one", fake_enrich_one)
enrichment._enrich_cancel.clear()
enrichment._background_enrich()
# The loop checks cancel BEFORE each song, so exactly one is processed before
# it breaks — not the whole 4-row queue.
assert calls == ["s0.archive"]
assert server._enrich_status["total"] == 4
assert server._enrich_status["matched"] == 1
assert enrichment._enrich_status["total"] == 4
assert enrichment._enrich_status["matched"] == 1
def test_rematch_requeues_visible_but_skips_manual(server, client):
_put(server, "a.archive") # will be 'matched'
_put(server, "b.archive", title="Other") # will be 'failed'
_put(server, "c.archive", title="Pinned") # will be 'manual' — untouchable
server._background_enrich()
enrichment._background_enrich()
with server.meta_db._lock:
server.meta_db.conn.execute(
"UPDATE song_enrichment SET match_state='matched' WHERE filename='a.archive'")
@@ -240,7 +241,7 @@ def test_rematch_requeues_visible_but_skips_manual(server, client):
# ── filename-derived artist/title fallback (blank-artist packs) ───────────────
def test_filename_artist_title_parse(server):
f = server._artist_title_from_filename
f = enrichment._artist_title_from_filename
assert f("CDLC/0 - City Pop/Tatsuro-Yamashita_Ride-On-Time_v1_p.feedpak") == \
{"artist": "Tatsuro Yamashita", "title": "Ride On Time"}
assert f("Anri_Windy-Summer_v1_p.feedpak") == {"artist": "Anri", "title": "Windy Summer"}
@@ -255,18 +256,18 @@ def test_blank_artist_seeds_match_from_filename(server, monkeypatch):
server.meta_db.put("Tatsuro-Yamashita_Ride-On-Time_v1_p.feedpak", 0, 0, {
"title": "Tatsuro-Yamashita_Ride-On-Time_v1_p", "artist": "", "album": "",
"duration": 240, "arrangements": [{"name": "Bass", "index": 0}]})
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_manifest_exact_ids", lambda fn: {})
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_manifest_exact_ids", lambda fn: {})
seen = {}
def fake_search(artist, title, limit=8):
seen["artist"], seen["title"] = artist, title
return []
monkeypatch.setattr(server, "_mb_search_recordings", fake_search)
monkeypatch.setattr(enrichment, "_mb_search_recordings", fake_search)
row = next(r for r in server.meta_db.enrichment_pending()
if r["filename"].startswith("Tatsuro"))
server._enrich_one(row)
enrichment._enrich_one(row)
# the blank pack artist was replaced by the filename-derived identity for
# the search (this is exactly what rescues the 'failed' pile)
assert seen == {"artist": "Tatsuro Yamashita", "title": "Ride On Time"}
@@ -276,18 +277,18 @@ def test_present_artist_is_not_overridden_by_filename(server, monkeypatch):
server.meta_db.put("Weird-Filename_x_y.feedpak", 0, 0, {
"title": "Real Title", "artist": "Real Artist", "album": "", "duration": 100,
"arrangements": [{"name": "Lead", "index": 0}]})
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(server, "_manifest_exact_ids", lambda fn: {})
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_manifest_exact_ids", lambda fn: {})
seen = {}
def fake_search(artist, title, limit=8):
seen["artist"], seen["title"] = artist, title
return []
monkeypatch.setattr(server, "_mb_search_recordings", fake_search)
monkeypatch.setattr(enrichment, "_mb_search_recordings", fake_search)
row = next(r for r in server.meta_db.enrichment_pending()
if r["filename"].startswith("Weird"))
server._enrich_one(row)
enrichment._enrich_one(row)
# a pack that DOES carry an artist keeps it — the filename is never consulted
assert seen == {"artist": "Real Artist", "title": "Real Title"}
@@ -295,7 +296,7 @@ def test_present_artist_is_not_overridden_by_filename(server, monkeypatch):
def test_kick_clears_a_stale_cancel(server):
# A cancelled-then-rekicked pass must start clean: _kick_enrich clears the
# flag so the fresh pass isn't aborted the instant it checks.
server._enrich_cancel.set()
server._kick_enrich()
enrichment._enrich_cancel.set()
enrichment._kick_enrich()
server._join_background_db_threads()
assert server._enrich_cancel.is_set() is False
assert enrichment._enrich_cancel.is_set() is False
+2 -1
View File
@@ -14,6 +14,7 @@ back-compat for `.sloppak` libraries or stop accepting the new `.feedpak`:
from __future__ import annotations
import importlib
from routers import settings as settings_router
import io
import sys
import zipfile
@@ -242,7 +243,7 @@ def test_settings_dlc_count_includes_both_suffixes(tmp_path, settings_server):
(dlc / "c.FEEDPAK").write_bytes(b"") # case-insensitive (suffix.lower())
(dlc / "notes.txt").write_bytes(b"") # ignored
result = settings_server.save_settings({"dlc_dir": str(dlc)})
result = settings_router.save_settings({"dlc_dir": str(dlc)})
assert "error" not in result, result
# save_settings joins its notices into a single ``message`` string.
+3 -2
View File
@@ -6,6 +6,7 @@ song (delete_song). Locks pin a field against a later auto-match.
"""
import importlib
import enrichment
import sys
import pytest
@@ -149,7 +150,7 @@ def test_locked_fields_reader(server):
def test_compose_lock_filter_strips_locked_cand_keys(server):
f = server._compose_lock_filter(None, {"artist", "year"})
f = enrichment._compose_lock_filter(None, {"artist", "year"})
cand = {"recording_id": "r", "artist": "X", "artist_sort": "X", "title": "T",
"year": "1990", "album": "A", "genres": ["rock"]}
out = f(cand)
@@ -158,7 +159,7 @@ def test_compose_lock_filter_strips_locked_cand_keys(server):
# …identity + unlocked display fields survive
assert out["recording_id"] == "r" and out["title"] == "T" and out["album"] == "A"
# no locks → base filter returned unchanged (zero-copy common path)
assert server._compose_lock_filter(None, set()) is None
assert enrichment._compose_lock_filter(None, set()) is None
# ── display overlay in the grid (slice 3) ─────────────────────────────────────
+53 -52
View File
@@ -1,6 +1,6 @@
"""Server-level tests for the P8 MusicBrainz matcher + Match-Review flow.
The HTTP transport is a fake installed over `server._mb_http_get` the ONE
The HTTP transport is a fake installed over `enrichment._mb_http_get` the ONE
seam enrichment uses to reach the network so nothing here ever opens a
socket. The offline default is itself under test: without explicitly
enabling the network flag, a pass must skip matching entirely (pytest can
@@ -8,6 +8,7 @@ never hit MusicBrainz, whatever a test triggers).
"""
import importlib
import enrichment
import sys
import pytest
@@ -50,7 +51,7 @@ class FakeMB:
def __call__(self, path, params):
if self.raise_transport:
raise self._srv.EnrichTransportError("fake network down")
raise enrichment.EnrichTransportError("fake network down")
self.calls.append((path, dict(params)))
if path == "recording":
return self.search_response
@@ -71,8 +72,8 @@ def mb(server, monkeypatch):
disables it by default see test_offline_default_skips_matching)."""
fake = FakeMB()
fake._srv = server
monkeypatch.setattr(server, "_mb_http_get", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
@@ -111,8 +112,8 @@ def test_search_falls_back_to_loose_when_strict_is_empty(server, monkeypatch):
return {"recordings": []}
return {"recordings": [mb_doc(rid="rec-x", title="Telephone Number")]}
monkeypatch.setattr(server, "_mb_http_get", _routed)
cands = server._mb_search_recordings("Junko Ohashi", "Telephone Number")
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
cands = enrichment._mb_search_recordings("Junko Ohashi", "Telephone Number")
assert len(cands) == 1
assert len(calls) == 2 # strict first, then the loose retry
assert calls[0].startswith("recording:") # strict is the field-phrase form
@@ -128,8 +129,8 @@ def test_search_does_not_retry_when_strict_hits(server, monkeypatch):
calls.append(params.get("query", ""))
return {"recordings": [mb_doc()]}
monkeypatch.setattr(server, "_mb_http_get", _routed)
cands = server._mb_search_recordings("AC/DC", "Thunderstruck")
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
cands = enrichment._mb_search_recordings("AC/DC", "Thunderstruck")
assert len(cands) == 1
assert len(calls) == 1
@@ -147,10 +148,10 @@ def test_artist_aliases_fetched_and_cached(server, monkeypatch):
return {"sort-name": "Ohashi, Junko",
"aliases": [{"name": "Junko Ohashi"}, {"name": "大橋 純子"}]}
monkeypatch.setattr(server, "_mb_http_get", fake)
names = server._mb_artist_aliases(_AID)
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
names = enrichment._mb_artist_aliases(_AID)
assert "Junko Ohashi" in names and "Ohashi, Junko" in names
server._mb_artist_aliases(_AID) # cached → no second request
enrichment._mb_artist_aliases(_AID) # cached → no second request
assert len(calls) == 1
@@ -158,8 +159,8 @@ def test_artist_aliases_rejects_bad_id(server, monkeypatch):
def boom(path, params):
raise AssertionError("must not fetch for a non-UUID id")
monkeypatch.setattr(server, "_mb_http_get", boom)
assert server._mb_artist_aliases("not-a-uuid") == []
monkeypatch.setattr(enrichment, "_mb_http_get", boom)
assert enrichment._mb_artist_aliases("not-a-uuid") == []
def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
@@ -176,9 +177,9 @@ def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
return {"recordings": [mb_doc(rid="rec-jp", title="Telephone Number",
artist="大橋純子", artist_id=_AID)]} # loose hit
monkeypatch.setattr(server, "_mb_http_get", _routed)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
server._background_enrich()
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
enrichment._background_enrich()
row = server.meta_db.get_enrichment("x.sloppak")
# The romanized alias lifts the artist over the auto floor → auto-confirmed.
assert row["match_state"] == "matched"
@@ -190,10 +191,10 @@ def test_enrich_auto_matches_japanese_primary_via_alias(server, monkeypatch):
def test_locked_field_not_canonicalized_by_auto_match(server, monkeypatch):
_put(server, "x.sloppak") # title "Thunderstruck (v2)", artist "ACDC"
server.meta_db.set_song_override("x.sloppak", "artist", locked=True)
monkeypatch.setattr(server, "_mb_http_get",
monkeypatch.setattr(enrichment, "_mb_http_get",
lambda path, params: {"recordings": [mb_doc()]})
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
server._background_enrich()
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
enrichment._background_enrich()
row = server.meta_db.get_enrichment("x.sloppak")
assert row["match_state"] == "matched" # still matches (identity applies)…
assert row["canon_artist"] is None # …but the LOCKED artist isn't canonicalized
@@ -208,9 +209,9 @@ def test_offline_default_skips_matching(server, monkeypatch):
but never matches even with a transport installed."""
fake = FakeMB()
fake._srv = server
monkeypatch.setattr(server, "_mb_http_get", fake)
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
_put(server, "a.sloppak")
server._background_enrich()
enrichment._background_enrich()
assert fake.calls == []
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "unscanned"
@@ -218,15 +219,15 @@ def test_offline_default_skips_matching(server, monkeypatch):
def test_real_transport_refuses_when_offline(server):
"""_mb_http_get itself raises (before any socket) when the network is
disabled defence in depth under pytest."""
with pytest.raises(server.EnrichTransportError):
server._mb_http_get("recording", {"query": "x"})
with pytest.raises(enrichment.EnrichTransportError):
enrichment._mb_http_get("recording", {"query": "x"})
def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
_put(server, "a.sloppak")
_put(server, "b.sloppak", title="Other Song")
mb.raise_transport = True
server._background_enrich()
enrichment._background_enrich()
for fn in ("a.sloppak", "b.sloppak"):
row = server.meta_db.get_enrichment(fn)
assert row["match_state"] == "unscanned"
@@ -234,7 +235,7 @@ def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
# Network comes back → the next kick matches both.
mb.raise_transport = False
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
@@ -243,7 +244,7 @@ def test_transport_error_pauses_pass_without_burning_attempts(server, mb):
def test_high_confidence_auto_matches_and_settles(server, mb):
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["match_source"] == "text"
@@ -256,13 +257,13 @@ def test_high_confidence_auto_matches_and_settles(server, mb):
assert row["genres"] == ["hard rock"]
# Settled: another pass makes NO further network calls…
n = len(mb.calls)
server._background_enrich()
enrichment._background_enrich()
assert len(mb.calls) == n
# …until the identity changes, which re-matches.
_put(server, "a.sloppak", title="Back in Black")
mb.search_response = {"recordings": [mb_doc(rid="rec-2", title="Back in Black",
album="Back in Black", date="1980-07-25")]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["mb_recording_id"] == "rec-2"
@@ -271,7 +272,7 @@ def test_medium_confidence_goes_to_review_not_canonical(server, mb):
# Partial artist agreement → medium confidence.
_put(server, "a.sloppak", artist="AC/DC ft Nobody")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "review"
assert row["match_source"] == "text"
@@ -281,7 +282,7 @@ def test_medium_confidence_goes_to_review_not_canonical(server, mb):
assert row["candidates"] and row["candidates"][0]["recording_id"] == "rec-1"
# A review row is settled while its identity is unchanged — no re-query.
n = len(mb.calls)
server._background_enrich()
enrichment._background_enrich()
assert len(mb.calls) == n
@@ -289,14 +290,14 @@ def test_low_confidence_fails_with_backoff(server, mb):
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc(rid="rec-x", title="Sunrise",
artist="Norah Jones")]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "failed"
assert row["attempts"] == 1
assert row["last_attempt_at"] is not None
# Immediately after, the backoff hasn't elapsed → no retry, no network.
n = len(mb.calls)
server._background_enrich()
enrichment._background_enrich()
assert len(mb.calls) == n
assert server.meta_db.get_enrichment("a.sloppak")["attempts"] == 1
# Rewind the clock two hours → eligible again, attempts increments.
@@ -304,7 +305,7 @@ def test_low_confidence_fails_with_backoff(server, mb):
server.meta_db.conn.execute(
"UPDATE song_enrichment SET last_attempt_at = last_attempt_at - 7200")
server.meta_db.conn.commit()
server._background_enrich()
enrichment._background_enrich()
assert len(mb.calls) == n + 1
assert server.meta_db.get_enrichment("a.sloppak")["attempts"] == 2
@@ -312,7 +313,7 @@ def test_low_confidence_fails_with_backoff(server, mb):
def test_no_results_fails(server, mb):
_put(server, "a.sloppak")
mb.search_response = {"recordings": []}
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "failed"
@@ -322,7 +323,7 @@ def test_cache_hit_copies_match_without_network(server, mb):
_put(server, "a.sloppak")
_put(server, "b.sloppak") # identical identity → same content_hash
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
assert len(mb.search_calls) == 1 # ONE search covered both charts
a = server.meta_db.get_enrichment("a.sloppak")
b = server.meta_db.get_enrichment("b.sloppak")
@@ -347,7 +348,7 @@ def test_manifest_mbid_tier0(server, mb):
_write_sloppak_manifest(server, "a.sloppak", f"mbid: {mbid}\n")
_put(server, "a.sloppak")
mb.recording_lookups[mbid] = mb_doc(rid=mbid)
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["match_source"] == "mbid"
@@ -360,7 +361,7 @@ def test_manifest_isrc_tier1(server, mb):
_write_sloppak_manifest(server, "a.sloppak", "isrc: AUAP09000045\n")
_put(server, "a.sloppak")
mb.isrc_lookups["AUAP09000045"] = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["match_source"] == "isrc"
@@ -374,7 +375,7 @@ def test_manifest_isrc_display_hyphens_stripped(server, mb):
_write_sloppak_manifest(server, "a.sloppak", "isrc: AU-AP0-90-00045\n")
_put(server, "a.sloppak")
mb.isrc_lookups["AUAP09000045"] = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["match_source"] == "isrc"
@@ -387,7 +388,7 @@ def test_bad_manifest_mbid_falls_through_to_text(server, mb):
_put(server, "a.sloppak")
mb.recording_lookups.clear() # lookup 404s (typo'd manifest)
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["match_source"] == "text"
@@ -401,7 +402,7 @@ def test_manual_never_overwritten_by_matcher(server, mb):
"a.sloppak", {"recording_id": "user-pick", "title": "Thunderstruck",
"artist": "AC/DC"}, source="search")
mb.search_response = {"recordings": [mb_doc(rid="machine-pick")]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "manual"
assert row["mb_recording_id"] == "user-pick"
@@ -419,7 +420,7 @@ def _seed_review(server, mb, fn="a.sloppak", title="Thunderstruck (v2)"):
# legitimately copies an earlier row instead of running the text tiers).
_put(server, fn, title=title, artist="AC/DC ft Nobody")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment(fn)["match_state"] == "review"
@@ -457,11 +458,11 @@ def test_review_reject_route_never_retries(server, mb, client):
assert row["match_source"] == "rejected"
# Rejected rows are excluded from the retry backoff forever…
n = len(mb.calls)
server._background_enrich()
enrichment._background_enrich()
assert len(mb.calls) == n
# …but an identity edit re-queues (the user fixed the metadata).
_put(server, "a.sloppak", artist="AC/DC")
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
# Rejecting a manual row is refused.
r = client.post("/api/enrichment/review/a.sloppak/reject")
@@ -502,8 +503,8 @@ def test_search_proxy(server, mb, client, monkeypatch):
assert body["candidates"][0]["score"] > 0.9
# Transport failure surfaces as 503, not a 500.
def _down(path, params):
raise server.EnrichTransportError("down")
monkeypatch.setattr(server, "_mb_http_get", _down)
raise enrichment.EnrichTransportError("down")
monkeypatch.setattr(enrichment, "_mb_http_get", _down)
r = client.get("/api/enrichment/search", params={"title": "x"})
assert r.status_code == 503
@@ -523,8 +524,8 @@ def test_match_facet_filters_grid_and_stats(server, mb, client, monkeypatch):
if "revsong" in q:
return {"recordings": [mb_doc(rid="rec-r", title="Revsong")]}
return {"recordings": []}
monkeypatch.setattr(server, "_mb_http_get", _routed)
server._background_enrich()
monkeypatch.setattr(enrichment, "_mb_http_get", _routed)
enrichment._background_enrich()
# Pendsong got failed by the pass (no results); reset it to unscanned to
# represent the not-yet-scanned band.
with server.meta_db._lock:
@@ -566,14 +567,14 @@ def test_auto_threshold_setting_moves_the_auto_review_boundary(server, mb, clien
client.post("/api/settings", json={"enrich_auto_threshold": 0.95})
_put(server, "a.sloppak", title="Highway to Hell", artist="AC/DC",
year="", duration=0)
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "review"
# Lower the bar to the default 0.90 → an identity edit re-queues, and the
# same 0.90-scored candidate now auto-applies.
client.post("/api/settings", json={"enrich_auto_threshold": 0.9})
_put(server, "a.sloppak", title="Highway to Hell", artist="AC/DC",
year="", duration=0, album="Different Album")
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert abs(row["match_score"] - 0.9) < 1e-6
@@ -583,7 +584,7 @@ def test_enrich_enabled_setting_gates_background_matching(server, mb, client):
client.post("/api/settings", json={"enrich_enabled": False})
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
assert mb.calls == []
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "unscanned"
# Manual search/fix stays available while the background matcher is off.
@@ -591,7 +592,7 @@ def test_enrich_enabled_setting_gates_background_matching(server, mb, client):
assert r.status_code == 200
# Re-enable → the next pass matches.
client.post("/api/settings", json={"enrich_enabled": True})
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
@@ -625,7 +626,7 @@ def test_review_queue_orders_missing_data_first(server, mb, client):
_seed_review(server, mb, fn="aa.sloppak", title="Thunderstruck (v2)")
_put(server, "zz.sloppak", title="Thunderstruck (Live)",
artist="AC/DC ft Nobody", album="", year="")
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("zz.sloppak")["match_state"] == "review"
songs = client.get("/api/enrichment/review").json()["songs"]
assert [s["filename"] for s in songs] == ["zz.sloppak", "aa.sloppak"]
+16 -15
View File
@@ -8,6 +8,7 @@ flag is only force-enabled where a test needs the pipeline to run.
"""
import importlib
import enrichment
import io as _io
import sys
@@ -57,8 +58,8 @@ class FakeMB:
@pytest.fixture()
def mb(server, monkeypatch):
fake = FakeMB()
monkeypatch.setattr(server, "_mb_http_get", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_mb_http_get", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
@@ -116,13 +117,13 @@ def test_musicbrainz_source_off_stamps_without_matching(server, mb, client):
client.post("/api/settings", json={"enrich_src_musicbrainz": False})
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
assert mb.calls == []
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "unscanned" # hash stamped, no match
# Re-enabling picks the same row up on the next pass.
client.post("/api/settings", json={"enrich_src_musicbrainz": True})
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "matched"
@@ -133,7 +134,7 @@ def test_field_toggles_strip_auto_applied_fields(server, mb, client):
"enrich_apply_genres": False})
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["canon_artist"] == "AC/DC"
@@ -150,7 +151,7 @@ def test_names_toggle_keeps_ids_and_other_fields(server, mb, client):
client.post("/api/settings", json={"enrich_apply_names": False})
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["canon_artist"] is None
@@ -170,7 +171,7 @@ def test_review_accept_applies_all_fields_despite_toggles(server, mb, client):
# Partial artist agreement → review tier (candidates stored unfiltered).
_put(server, "a.sloppak", artist="AC/DC ft Nobody")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["match_state"] == "review"
r = client.post("/api/enrichment/review/a.sloppak/accept",
json={"recording_id": "rec-1"})
@@ -192,21 +193,21 @@ def test_reenabling_field_backfills_matched_row(server, mb, client):
client.post("/api/settings", json={"enrich_apply_year": False})
_put(server, "a.sloppak")
mb.search_response = {"recordings": [mb_doc()]}
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["canon_year"] is None # suppressed
assert row["apply_mask"] == "enrich_apply_year" # …and remembered
# Re-enable → next pass re-queues and backfills the year (hash unchanged).
client.post("/api/settings", json={"enrich_apply_year": True})
server._background_enrich()
enrichment._background_enrich()
row = server.meta_db.get_enrichment("a.sloppak")
assert row["match_state"] == "matched"
assert row["canon_year"] == "1990" # backfilled
assert row["apply_mask"] in (None, "") # fully applied now
# Converged: a fully-applied row is not re-queued again.
assert server.meta_db.enrichment_pending(
allowed_keys=frozenset(server._ENRICH_APPLY_FIELDS)) == []
allowed_keys=frozenset(enrichment._ENRICH_APPLY_FIELDS)) == []
def test_partial_match_is_not_a_cache_donor(server):
@@ -271,8 +272,8 @@ def caa(server, monkeypatch):
calls.append(release_id)
return art.get(release_id)
fake.calls = calls
monkeypatch.setattr(server, "_caa_http_get", fake)
monkeypatch.setattr(server, "_enrich_network_enabled", lambda: True)
monkeypatch.setattr(enrichment, "_caa_http_get", fake)
monkeypatch.setattr(enrichment, "_enrich_network_enabled", lambda: True)
return fake
@@ -280,13 +281,13 @@ def test_caa_source_toggle_gates_art_fetch(server, client, caa):
make_sloppak(server, "a.sloppak")
_match_row(server, "a.sloppak")
client.post("/api/settings", json={"enrich_src_caa": False})
server._background_enrich()
enrichment._background_enrich()
assert caa.calls == []
row = server.meta_db.get_enrichment("a.sloppak")
assert row["art_state"] is None # not forfeited, just skipped
# Re-enable → the same row is picked up.
client.post("/api/settings", json={"enrich_src_caa": True})
server._background_enrich()
enrichment._background_enrich()
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] == "caa"
@@ -294,7 +295,7 @@ def test_apply_art_toggle_gates_art_fetch(server, client, caa):
make_sloppak(server, "a.sloppak")
_match_row(server, "a.sloppak")
client.post("/api/settings", json={"enrich_apply_art": False})
server._background_enrich()
enrichment._background_enrich()
assert caa.calls == []
assert server.meta_db.get_enrichment("a.sloppak")["art_state"] is None
+5 -4
View File
@@ -10,6 +10,7 @@ shadow the config.json dlc_dir fallback.
"""
import importlib
from routers import settings as settings_router
import json
import sys
@@ -32,13 +33,13 @@ class _DirectSettingsClient:
def get(self, path):
if path != "/api/settings":
raise ValueError(f"unsupported path: {path}")
return _DirectResponse(self._server.get_settings())
return _DirectResponse(settings_router.get_settings())
def post(self, path, json):
if path == "/api/settings":
return _DirectResponse(self._server.save_settings(json))
return _DirectResponse(settings_router.save_settings(json))
if path == "/api/settings/reset":
return _DirectResponse(self._server.reset_settings(json))
return _DirectResponse(settings_router.reset_settings(json))
raise ValueError(f"unsupported path: {path}")
def close(self):
@@ -643,7 +644,7 @@ def test_achievements_enabled_persists_and_validates(api_client, tmp_path):
def test_achievements_enabled_is_resettable(server_module):
"""The flag is in the resettable allow-list so a Reset clears it to default."""
assert "achievements_enabled" in server_module._RESETTABLE_SETTINGS_KEYS
assert "achievements_enabled" in settings_router._RESETTABLE_SETTINGS_KEYS
def test_skip_startup_tasks_drives_startup_to_complete(api_client):
+5 -4
View File
@@ -11,6 +11,7 @@ is exercised separately in `test_plugins.py`.
import base64
import importlib
from routers import settings as settings_router
import json
import os
import sys
@@ -478,7 +479,7 @@ def test_normalize_export_paths_consistency(server_mod, tmp_path):
# Wraps `_validate_relpath` to assert it doesn't raise the
# hard-failure ValueErrors. _UndeclaredFile would mean the
# allowlist is wrong, not that the relpath shape is bad.
server_mod._validate_relpath(rel, cleaned, tmp_path)
settings_router._validate_relpath(rel, cleaned, tmp_path)
# ── Atomic write: unique tmp + cleanup on failure ───────────────────────────
@@ -502,7 +503,7 @@ def test_atomic_write_cleans_up_tmp_on_failure(server_mod, tmp_path, monkeypatch
for _ in range(2):
with pytest.raises(OSError):
server_mod._atomic_write_file(target, b"payload")
settings_router._atomic_write_file(target, b"payload")
# Both attempts cleaned up. No .tmp.import residue means the
# mkstemp + finally-unlink pattern held even across failures.
@@ -512,7 +513,7 @@ def test_atomic_write_cleans_up_tmp_on_failure(server_mod, tmp_path, monkeypatch
# Restoring real replace, the function should still work end-to-end.
monkeypatch.setattr(server_mod.os, "replace", real_replace)
server_mod._atomic_write_file(target, b"payload")
settings_router._atomic_write_file(target, b"payload")
assert target.read_bytes() == b"payload"
assert list(tmp_path.glob("*.tmp.import")) == []
@@ -764,7 +765,7 @@ def test_atomic_write_closes_fd_when_fdopen_fails(server_mod, tmp_path, monkeypa
monkeypatch.setattr(server_mod.os, "fdopen", boom_fdopen)
with pytest.raises(OSError, match="simulated EMFILE"):
server_mod._atomic_write_file(target, b"payload")
settings_router._atomic_write_file(target, b"payload")
# fd was closed (so it didn't leak), and the temp file mkstemp
# created was removed (so it doesn't litter / lock on Windows).
+11 -10
View File
@@ -15,6 +15,7 @@ this file pins the additive `core_server_files` section:
import base64
import importlib
from routers import settings as settings_router
import sqlite3
import sys
from pathlib import Path
@@ -118,7 +119,7 @@ def test_import_stages_db_restore_without_touching_live_db(client, server_mod, t
payload = _valid_db_bytes(tmp_path, name="incoming.db", marker="restored")
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"web_library.db": {"encoding": "base64",
@@ -143,7 +144,7 @@ def test_import_rejects_corrupt_db_with_valid_magic_header(client, server_mod, t
# fail to open the bad restore.
corrupt = b"SQLite format 3\x00" + b"\xff" * 200
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"web_library.db": {"encoding": "base64",
@@ -158,7 +159,7 @@ def test_import_rejects_non_sqlite_db_payload(client, server_mod, tmp_path):
# A truncated / wrong file staged as the restore would brick startup —
# reject anything lacking the SQLite magic header, before touching disk.
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"web_library.db": {"encoding": "base64",
@@ -171,7 +172,7 @@ def test_import_rejects_non_sqlite_db_payload(client, server_mod, tmp_path):
def test_import_writes_custom_art_immediately(client, server_mod, tmp_path):
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"playlist_covers/7.png": {"encoding": "base64",
@@ -186,7 +187,7 @@ def test_import_writes_custom_art_immediately(client, server_mod, tmp_path):
def test_import_core_path_traversal_rejected(client, server_mod, tmp_path):
secret = tmp_path.parent / "escape.txt"
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"../escape.txt": {"encoding": "base64",
@@ -201,7 +202,7 @@ def test_import_core_undeclared_path_skipped_not_fatal(client, server_mod, tmp_p
# A relpath outside the core allowlist is a warn-and-skip, not a refusal —
# the rest of the bundle still applies.
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"audio_cache/x.ogg": {"encoding": "base64",
@@ -289,7 +290,7 @@ def test_full_db_backup_restore_round_trip(client, server_mod, tmp_path):
def test_export_fails_hard_when_db_snapshot_unavailable(client, server_mod, monkeypatch):
# A backup that silently omits the library DB is a data-loss trap — the
# export must error rather than hand back an incomplete-looking bundle.
monkeypatch.setattr(server_mod, "_snapshot_library_db", lambda: None)
monkeypatch.setattr(settings_router, "_snapshot_library_db", lambda: None)
r = client.get("/api/settings/export")
assert r.status_code == 500
assert "library database" in r.json()["error"].lower()
@@ -299,16 +300,16 @@ def test_failed_import_disarms_staged_db_restore(client, server_mod, tmp_path, m
# If a later write in phase 2 fails, the request 500s — but a staged DB
# restore must NOT survive to swap in on the next restart.
payload = _valid_db_bytes(tmp_path, name="incoming.db")
real_write = server_mod._atomic_write_file
real_write = settings_router._atomic_write_file
def boom(target, data):
if target.name == "config.json": # last write of the commit
raise OSError("disk full")
return real_write(target, data)
monkeypatch.setattr(server_mod, "_atomic_write_file", boom)
monkeypatch.setattr(settings_router, "_atomic_write_file", boom)
r = client.post("/api/settings/import", json={
"schema": server_mod.SETTINGS_BUNDLE_SCHEMA,
"schema": settings_router.SETTINGS_BUNDLE_SCHEMA,
"server_config": {},
"core_server_files": {
"web_library.db": {"encoding": "base64",
+1
View File
@@ -37,6 +37,7 @@ def dlc_client(tmp_path, monkeypatch):
static_tmp = tmp_path / "static"
static_tmp.mkdir()
monkeypatch.setattr(server, "STATIC_DIR", static_tmp)
monkeypatch.setattr(server.appstate, "static_dir", static_tmp)
tc = TestClient(server.app, client=("127.0.0.1", 50000))
try:
yield tc, server, dlc