mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-07-22 12:52:29 +00:00
main
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Byron Gamatos
|
73127d5416
|
refactor(server): extract the album-art routes into routers/art.py (R3) (#862)
The six song-art routes — GET /api/song/{f}/art, .../art/cover-search,
.../art/candidates, POST .../art/upload, .../art/url, DELETE /api/art/{f}/override
— plus their exclusive helpers (the ETag/304 response machinery, _save_art_override,
_url_host_is_internal, _fetch_art_url + the art size/redirect caps) move to
lib/routers/art.py. Bodies verbatim except @app->@router and the seam reads:
meta_db->appstate.meta_db, ART_CACHE_DIR->appstate.art_cache_dir, and the three
shared art helpers that stay in server.py (used by the song/delete routes too)
-> appstate.<callable> (_song_pack_art_exists, _art_override_paths — already
seam-injected for the enrichment worker — plus a new art_safe_name slot). The
CAA / release-search transport lives in lib/enrichment.py and is reached as
enrichment.X. LocalLibraryProvider.get_art now calls art_router.get_song_art.
server.py: 5,988 -> 5,540 (-448).
Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2399 passed (33 art serve/candidates/
override/url cases incl the SSRF-guard _url_host_is_internal + _fetch_art_url
size-cap tests, retargeted onto the art module); test_packaging 43; eslint 0.
Boot smoke: /art 404, /art/candidates 404, DELETE /override 200 from the router.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Byron Gamatos
|
165475d115
|
refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3) (#861)
* refactor(server): move the metadata-enrichment subsystem into lib/enrichment.py (R3)
MusicBrainz / Cover-Art-Archive / AcoustID transport, the match-scorer glue, and
the background enrichment worker (~930 lines, 61 defs) leave server.py as one
cohesive unit. Bodies are verbatim; the only changes are seam reads:
meta_db / config_dir / sloppak_cache_dir / art_cache_dir -> appstate.<slot>
_song_pack_art_exists / _art_override_paths (stay in server.py for the art +
delete routes) -> appstate.<callable> (new seam slots, injected by reference)
_env_flag -> env_compat.env_flag_compat (the existing identical helper)
_artist_title_from_filename -> imported from metadata_db (its home)
the User-Agent VERSION lookup: Path(__file__).parent ->
Path(__file__).resolve().parents[1] (lib/enrichment.py -> app root)
server.py drives the worker through the module (import enrichment; the routes +
scan lifecycle call enrichment.X). Tests that faked the network on `server`
(_mb_http_get, _enrich_network_enabled, _caa_http_get, ...) now patch the same
names on `enrichment` — the module attribute is resolved at call time, so one
setattr reaches both the routes and the worker's internal callers. Acyclic:
enrichment imports appstate/appconfig/dlc_paths/metadata_db/mb_match/
acoustid_match/sloppak/loosefolder, never server.
server.py: 6,917 -> 5,988 (-929).
Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2400 passed (140 enrichment/art cases
incl the offline-safety + transport-error-pauses-pass contracts that fake the
network); test_packaging 44 passed (enrichment.py resolves under lib/); eslint 0.
Boot smoke: /api/enrichment/status + POST /kick serve from the new module.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: reset enrichment worker state between tests (CodeRabbit)
lib/enrichment.py now owns the worker, and it stays imported for the whole
session while the `server` fixtures pop-and-reimport `server` — so the cancel
Event / status dict / caches would leak across tests, and a stale `_enrich_cancel`
could short-circuit a later direct `_background_enrich()`. An autouse conftest
fixture clears that process-global state before each test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: tighten the enrichment-reset fixture (CodeRabbit)
Narrow the import guard to ImportError (not blind Exception, BLE001), and stop
clearing _caa_index_locks — it's guarded by _caa_index_locks_guard, so an
unlocked clear() would race a still-alive worker, and its per-release mutexes
carry no test state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Byron Gamatos
|
97a941c45d
|
fix(tests): join background scan/enrich workers before closing the DB (#735)
Root cause of the flaky pytest segfault (exit 139): the background scan and enrichment daemon threads (_scan_runner/_enrich_runner) use the shared MetadataDB connection, but test fixtures closed that connection in teardown without stopping them. A daemon thread mid-query on a freed SQLite conn is a native use-after-free → SIGSEGV. The app's startup kicks a scan, so almost any app-booting fixture was vulnerable. It only surfaced now because got-feedback/feedBack#728 added a push trigger, so ci/test runs on every push to main. Fix: server.py retains the scan/enrich thread handles and adds _join_background_db_threads(); every test fixture now joins the workers before conn.close(). Verified: the full suite runs to completion (no segfault) where it previously crashed at ~25%. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
ChrisBeWithYou
|
7ca736d525
|
library: cover-art layer — CAA auto-fetch + user overrides, GIF local-only (P9) (#715)
* library: cover-art layer — CAA auto-fetch + user overrides, GIF local-only (R3/P9)
Third slice of the enrichment series (stacked on the matcher): covers.
- Serve chain for GET /api/song/{fn}/art: USER OVERRIDE -> PACK ART ->
COVER ART ARCHIVE cache -> 404. Behaviour change, deliberate: a
user-uploaded cover now OVERRIDES pack art (previously the upload only
filled the no-art gap, which made custom art look broken on any song
that already shipped a cover).
- GIF is allowed as an override and kept VERBATIM (animation intact) —
a local-only bonus. Everything else normalizes to RGB PNG as before.
One override per song (saving either kind removes the other), and
nothing ever writes art INTO a pack file — test-pinned: the pack's
cover.jpg is byte-identical after a GIF upload.
- Art by URL: POST /api/song/{fn}/art/url fetches server-side (http(s)
only, 10 MB cap enforced while streaming) into the same override slot.
DELETE /api/art/{fn}/override drops it — under /api/art because the
greedy DELETE /api/song/{path} catch-all shadows anything beneath it
(the same dodge the chart split/unsplit routes use).
- Cover Art Archive fetch as phase 3 of the enrichment pass: matched
songs that LACK pack art get their release's front cover, throttled +
identified + offline-guarded exactly like the MusicBrainz client
(pytest can never reach the network; a transport error pauses the
pass without burning the row). The cache is keyed by RELEASE MBID —
ten charts of one album cost one fetch — and every outcome writes an
art_state (pack/user/caa/none/error) so a row is evaluated once.
- LRU cap (200 MB) on the CAA side of the cache only; user overrides
are never evicted, and evicted rows reset so a later pass may
re-fetch. Deleting a song removes its override files (CAA files stay
— they may be shared by other charts of the release).
No frontend changes: the grid, the review modal, and the player pick
the new art up through the same route they already use. The
upload/paste-a-link surfaces in the Details drawer land with the
context-menu slice once the drawer PR merges.
13 new tests (tests/test_art_layer.py) + demo-mode routes; full-suite
failure set byte-identical with the change stashed vs applied.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nm7tHs1Yvjjtnnu4nzJgdN
* library: harden cover-art layer — SSRF guard, demo/size caps, override-delete state reset
Follow-up hardening on the R3 cover-art layer:
- remove_song_art_override: reset the enrichment row (set_enrichment_art(fn,
None, None)) when an override is deleted, so a row previously settled as
'user' re-queues and the CAA fallback resumes. Previously a removed override
stranded the row (enrichment_art_pending only re-queues art_state IS NULL),
leaving the song with no art at all.
- Base64 art upload: block it in demo mode (was open — a write/disk-fill vector,
worse now that GIFs are stored verbatim), validate the filename resolves to a
real song (mirrors the url route), and cap the decoded payload at 10 MB.
- Art-by-URL: reject hosts that resolve to loopback/private/link-local/reserved/
multicast/unspecified addresses (SSRF, e.g. cloud metadata) and stop following
redirects (allow_redirects=False) so a redirect can't smuggle the request to an
internal target. Fails closed on unresolvable/unparseable hosts.
- _caa_http_get: stream with a per-file 10 MB cap (bounds any one response
independently of the aggregate LRU); guard release_id against a conservative
token before interpolating it into a cache-file path (no separators/dots).
Tests: delete-override→CAA-fallback, upload unknown-song/oversize rejection,
SSRF internal-host guard, and a demo-mode block assertion for art/upload.
Note: art_state='error' rows are intentionally not auto-retried — there is no
per-row attempt counter on the art side, so an unbounded retry could storm CAA
for permanently-bad rows; a bounded retry would need extra state, left out here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: byrongamatos <xasiklas@gmail.com>
|