Fix slow library cover loading: serve sloppak art without unpacking + revalidated caching (#534)

* sloppak: read cover without unpacking + serialize/cap zip unpacks

Album art for a zip-form sloppak was served by resolve_source_dir(), which
unpacks the ENTIRE archive (stems included, ~30 MB) to disk just to read
cover.jpg. On the library grid that meant a full extraction per card on scroll.

- read_cover_bytes(): opens only the cover member from the zip (or reads the
  file for dir-form), with zip-slip guarding. ~4 ms vs a full unpack.
- resolve_source_dir(): per-file lock + bounded global semaphore so concurrent
  callers don't rmtree + re-extract the same dest at once (a race), and a burst
  can't saturate disk/CPU. 8 concurrent calls now dedupe to 1 unpack.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* server: serve sloppak art via read_cover_bytes + cache album-art responses

- get_song_art() sloppak branch now reads the cover directly (no full unpack),
  off-thread via asyncio.to_thread.
- All art responses carry Cache-Control: public, max-age=86400. URLs are already
  cache-busted with ?v=<mtime>, so the browser stops re-fetching every cover on
  scroll-back; day bound self-heals any URL missing ?v.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* v3 library: lazy-load + async-decode card cover images

The grid (24 cards/page) and artist-row thumbnails emitted plain <img> with no
loading hint, so a whole page of covers fetched + decoded at once on each
scroll batch. Add loading="lazy" decoding="async" to defer off-screen fetches
and keep image decode off the main thread.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* address Codex review: zip cover normalization + correct art revalidation

Findings from the preflight Codex passes:

- sloppak.read_cover_bytes (zip form) read the raw manifest cover string via
  zf.read(), so a non-canonical name like './cover.jpg' or 'art/../cover.jpg'
  404'd. Normalize via safe_join → relative member; reject escape and the
  degenerate root-collapse case ('.', 'subdir/..') like _unpack_zip does.

- Album-art caching is correctness-first: Cache-Control: no-cache plus a strong
  validator, with real conditional handling (Starlette FileResponse emits an
  ETag but doesn't evaluate If-None-Match). All three art paths route through
  _art_conditional/_file_art_response → bodyless 304 on a matching validator.
  A long immutable max-age was rejected because the frontend ?v=<mtime> buster
  is only second-resolution and would pin a same-second rewrite.

- The sloppak cover is validated by CONTENT (sha1 of the bytes), not a stat:
  a dir-form sloppak edited in place changes the cover file's mtime but not the
  directory's, so a dir-stat ETag could emit a stale 304. Content hashing is
  correct for both dir- and zip-form. get_song_art gained an optional request
  (internal get_art caller passes none — safe).

Adds tests/test_sloppak_cover_art.py pinning read_cover_bytes (canonical,
non-canonical, degenerate/escape, dir/zip, webp) and the endpoint's 304 contract
incl. the dir-form in-place-edit no-stale-304 regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Byron Gamatos
2026-06-20 00:22:43 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent a7a93e9bef
commit 21997f4b5c
4 changed files with 341 additions and 28 deletions
+84 -23
View File
@@ -6298,13 +6298,72 @@ def diagnostics_hardware():
def _if_none_match_hits(header: str | None, etag: str) -> bool:
"""True if an If-None-Match header matches `etag` (weak comparison).
Handles the `*` wildcard and comma-separated lists, and ignores a weak
`W/` prefix on either side the standard semantics for a conditional GET.
"""
if not header:
return False
bare = etag.removeprefix("W/")
for tok in header.split(","):
t = tok.strip()
if t == "*" or t.removeprefix("W/") == bare:
return True
return False
# Album art is served with a strong validator (an ETag on the sloppak byte
# path; FileResponse's own ETag/Last-Modified on the file paths) and revalidated
# with `no-cache`. That keeps re-scroll cheap — a conditional GET returns a
# bodyless 304 — without ever serving a stale cover. A long `immutable` max-age
# was rejected: the frontend's `?v=<mtime>` buster is only second-resolution, so
# a same-second cover rewrite would keep the URL and pin the old bytes for the
# cache lifetime. Validation cost is negligible for a localhost backend.
_ART_CACHE_HEADERS = {"Cache-Control": "no-cache"}
def _art_etag(path: Path) -> str | None:
"""Strong validator for an art file: nanosecond mtime + size (so a
same-second rewrite still changes it). None if the file can't be stat'd."""
try:
st = path.stat()
return f'"{st.st_mtime_ns}-{st.st_size}"'
except OSError:
return None
def _art_conditional(etag: str | None, request: Request | None):
"""Return (headers, not_modified) for an art response. `not_modified` is
True when the client's If-None-Match already matches `etag` → caller should
return a bodyless 304. Starlette's FileResponse emits an ETag but does NOT
itself evaluate If-None-Match, so every art path routes through here to get
real conditional handling."""
headers = dict(_ART_CACHE_HEADERS)
if etag:
headers["ETag"] = etag
inm = request.headers.get("if-none-match") if request is not None else None
return headers, bool(etag) and _if_none_match_hits(inm, etag)
def _file_art_response(path: Path, media_type: str, request: Request | None):
"""FileResponse for an on-disk art file, with no-cache + ETag and a bodyless
304 when the client's validator still matches."""
headers, not_modified = _art_conditional(_art_etag(path), request)
if not_modified:
return Response(status_code=304, headers=headers)
return FileResponse(str(path), media_type=media_type, headers=headers)
@app.get("/api/song/{filename:path}/art")
async def get_song_art(filename: str):
async def get_song_art(filename: str, request: Request = None):
"""Serve album art for a song.
Dispatches by format and returns the appropriate media type:
- Sloppak: serves `cover.jpg` (or manifest-declared cover) from
the source dir as JPEG/PNG/WebP.
- Sloppak: serves `cover.jpg` (or manifest-declared cover) read directly
from the package (the single cover member for zip-form sloppaks no
full unpack) as JPEG/PNG/WebP.
- Loose folder: serves the discovered art file directly as
JPEG/PNG/WebP.
"""
@@ -6318,27 +6377,29 @@ async def get_song_art(filename: str):
if not song_path.exists():
return JSONResponse({"error": "not found"}, 404)
# Sloppak path: pull cover.jpg from the source dir (manifest-declared or default).
# Sloppak path: read the cover (manifest-declared or default) straight from
# the package. For a zip-form sloppak this opens just the cover member —
# NOT the whole archive — so the library grid never triggers a full unpack
# of stems just to paint a thumbnail.
if sloppak_mod.is_sloppak(song_path):
# Read the cover (cheap — single member, no full unpack) and validate by
# its CONTENT. A stat-based ETag would be wrong for directory-form
# sloppaks: editing cover.jpg in place changes the file's mtime, not the
# directory's, so a dir-stat ETag could emit a stale 304. Content hashing
# is correct for both dir- and zip-form. Raw byte Response lacks
# FileResponse's validators, so we attach the ETag + honor If-None-Match.
try:
src = sloppak_mod.resolve_source_dir(filename, dlc, SLOPPAK_CACHE_DIR)
manifest = sloppak_mod.load_manifest(song_path)
cover_rel = str(manifest.get("cover") or "cover.jpg")
cover_path = (src / cover_rel).resolve()
# Prevent escape and fall back to default name if missing.
try:
cover_path.relative_to(src.resolve())
except ValueError:
return JSONResponse({"error": "forbidden"}, 403)
if cover_path.exists() and cover_path.is_file():
mt = {
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp",
}.get(cover_path.suffix.lower(), "image/jpeg")
return FileResponse(str(cover_path), media_type=mt)
art = await asyncio.to_thread(sloppak_mod.read_cover_bytes, song_path)
except Exception:
pass
return JSONResponse({"error": "no art"}, 404)
art = None
if art is None:
return JSONResponse({"error": "no art"}, 404)
data, mt = art
etag = f'"{hashlib.sha1(data).hexdigest()}"'
headers, not_modified = _art_conditional(etag, request)
if not_modified:
return Response(status_code=304, headers=headers)
return Response(content=data, media_type=mt, headers=headers)
# Loose folder path: serve art file directly.
# song_path is already validated against DLC_DIR by _resolve_dlc_path.
@@ -6358,7 +6419,7 @@ async def get_song_art(filename: str):
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp",
}.get(art_resolved.suffix.lower(), "image/jpeg")
return FileResponse(str(art_resolved), media_type=mt)
return _file_art_response(art_resolved, mt, request)
return JSONResponse({"error": "no art"}, 404)
# Custom art uploaded via /art/upload is cached as PNG under ART_CACHE_DIR;
@@ -6367,7 +6428,7 @@ async def get_song_art(filename: str):
safe_name = filename.replace("/", "_").replace(" ", "_")
cached = art_cache / f"{safe_name}.png"
if cached.exists():
return FileResponse(str(cached), media_type="image/png")
return _file_art_response(cached, "image/png", request)
return JSONResponse({"error": "no art"}, 404)