feat(song-info): publish the playable stem list so stems can preload (fixes the 698ms freeze) (#972)

* feat(song-info): publish the playable stem list, so stems can preload

The stems plugin could only learn its stem list from the highway's WS `ready`,
which arrives once the highway is already up. So it fetched, decoded, and then
handed every stem's PCM to its audio worklet — copying the WHOLE SONG — with the
player already on screen.

For a 4-minute 6-stem pack that is over half a GIGABYTE of memcpy, in one frame,
on the main thread. Measured on a real load: a 698 ms frame, right as the
song-credits card appeared, with the venue video visibly stopping. That is the
"the video pauses when the author appears" report.

GET /api/song/{f}?stems=1 now returns the same list — [{id, url, default}] plus
full_mix_url — so the plugin can start the whole load at `song:loading`, before
the highway (and the venue) is drawn, where a stalled frame costs nothing.
Nothing about the work changes; only WHEN.

Opt-in via the query param so the library's own metadata calls — the hot path —
pay nothing. Deliberately NOT stored in the metadata cache: that is a
fixed-column table, and widening it would mean a schema migration plus a stale
row for every song already scanned, to cache something that is a plain manifest
read on an already-unpacked pack.

The safety property: REST and the WS must publish the SAME list. If they
disagreed the plugin would preload a graph and then throw it away and rebuild —
strictly worse than not preloading. So both now resolve `default` through one
shared helper (stem_default_on, extracted from load_song), and a test rebuilds
the WS's payload from load_song and requires the REST helper to produce the
identical list, rather than pinning either against a snapshot.

Also pinned: the mixdown is lifted OUT of the stem list (spec 5.3 — `full` is
not a layer; listing it beside the instruments would play the whole song on top
of the stems) while staying reachable as full_mix_url, a single-`full` pack keeps
it as its only playable stem, and an unreadable pack yields an empty list rather
than failing the request. Full suite 2608 passed.

Consumed by feedBack-plugin-stems (preloadSong).

* fix(song-info): call load_song for the stem payload — do not reimplement it

CodeRabbit caught a real bug, and it would have hit most real libraries.

load_song() falls back to the DEPRECATED `original_audio:` key when a pack has no
reserved `full` stem — which is every pack written before feedpak 1.15.0. My
payload rebuilt the full-mix rule from extract_meta and returned None for those:
REST would say "no full mix" while the WS said there was one.

Worse than a wrong field: the plugin would preload a graph WITHOUT the pristine
mix and — because the stem signature still matched — never rebuild. Unity
playback would silently downgrade to the lossy stem recombination.

That is exactly the drift this PR claims to prevent, and my test had a hole: I
only covered packs that carry a `full` stem.

So stop reimplementing. The payload now calls load_song, whose LoadedSloppak
already carries the partitioned stems and the resolved full mix, and builds the
URLs exactly as ws_highway does. Drift is now impossible by construction rather
than by agreement. extract_meta is reverted to its original shape (it never
needed to change), and the shared stem_default_on helper stays as the one place
`default: off` is resolved.

Tests rewritten to compare against load_song — the WS's own function — for a
reserved-`full` pack, a LEGACY original_audio pack (the case that was broken), and
a single-`full` pack. Also documents the `?stems=1` contract in CHANGELOG.md.
Full suite green.

lib/routers/song.py

@@ -829,9 +829,60 @@ def post_song_gap_fill(filename: str, data: dict):
     return {"ok": True, "written": additions, "skipped": skipped}
 
 
+def _playable_stems_payload(filename: str, dlc) -> dict:
+    """The playable stems (id/url/default) + full-mix URL for a sloppak.
+
+    Why it exists: the stems plugin could only learn its stem list from the
+    highway's WS `ready`, which arrives once the highway is already up. So it
+    decoded, and then copied the whole song's PCM to its worklet, with the player
+    on screen — half a gigabyte of memcpy in one frame, ~700 ms, freezing the
+    venue video. Given the list at `song:loading` it can do all of that BEFORE the
+    highway appears, behind the loading overlay where a stall costs nothing.
+
+    The list MUST be the same one the WS sends a moment later. If it is not, the
+    plugin preloads a graph and then throws it away and rebuilds — strictly worse
+    than not preloading. So this does not reimplement the WS's construction, it
+    calls THE SAME FUNCTION: load_song, whose LoadedSloppak already carries the
+    partitioned stems and the resolved full mix, and then builds the URLs exactly
+    as ws_highway does. Drift is impossible by construction rather than by
+    agreement — which matters, because `full_mix` in particular is not simply the
+    `full` stem: load_song falls back to the deprecated `original_audio:` key for
+    every pack written before feedpak 1.15.0, and reimplementing that (I did, at
+    first) silently dropped the pristine full mix for most real libraries.
+
+    Opt-in (`?stems=1`) so the library's own metadata calls — the hot path — pay
+    nothing for it. Non-sloppak sources (archives, loose folders) have no stems
+    to preload: load_song raises and we return the empty list.
+    """
+    from urllib.parse import quote
+
+    try:
+        loaded = sloppak_mod.load_song(filename, dlc, appstate.sloppak_cache_dir)
+    except Exception:
+        return {"stems": [], "full_mix_url": None}
+
+    q_fn = quote(filename, safe="")
+
+    def _url(rel: str) -> str:
+        return f"/api/sloppak/{q_fn}/file/{quote(rel)}"
+
+    return {
+        "stems": [
+            {"id": s["id"], "url": _url(s["file"]), "default": s["default"]}
+            for s in loaded.stems
+        ],
+        "full_mix_url": _url(loaded.full_mix) if loaded.full_mix else None,
+    }
+
+
 @router.get("/api/song/{filename:path}")
-async def get_song_info(filename: str):
-    """Return song metadata, from cache or by extracting it from the song source."""
+async def get_song_info(filename: str, stems: int = 0):
+    """Return song metadata, from cache or by extracting it from the song source.
+
+    `?stems=1` additionally returns the playable stem list with URLs, so the
+    stems plugin can start fetching/decoding on `song:loading` instead of waiting
+    for the highway's WS `ready` (see _playable_stems_payload).
+    """
     import asyncio
     dlc = _get_dlc_dir()
     if not dlc:
@@ -854,8 +905,21 @@ async def get_song_info(filename: str):
 
     mtime, size = appstate.stat_for_cache(song_path)
     cached = appstate.meta_db.get(cache_key, mtime, size)
+    loop = asyncio.get_event_loop()
+
+    # The stem list is NOT stored in the metadata cache: that is a fixed-column
+    # table, and widening it would mean a migration plus a stale row for every
+    # song already scanned. It is cheap to read on demand (the pack is unpacked
+    # by then, so this is a plain manifest read), and only the opt-in caller pays.
+    async def _with_stems(meta: dict) -> dict:
+        if not stems:
+            return meta
+        extra = await loop.run_in_executor(
+            None, _playable_stems_payload, filename, dlc)
+        return {**meta, **extra}
+
     if cached:
-        return cached
+        return await _with_stems(cached)
 
     # Extract in thread pool
     def _extract():
@@ -863,5 +927,5 @@ async def get_song_info(filename: str):
         appstate.meta_db.put(cache_key, mtime, size, meta)
         return meta
 
-    meta = await asyncio.get_event_loop().run_in_executor(None, _extract)
-    return meta
+    meta = await loop.run_in_executor(None, _extract)
+    return await _with_stems(meta)

lib/sloppak.py

@@ -80,6 +80,20 @@ def find_full_mix(stems: list[dict]) -> dict | None:
     )
 
 
+def stem_default_on(raw) -> bool:
+    """Whether a manifest stem entry plays by default.
+
+    Absent means on. A string is honoured so a hand-written manifest can say
+    `default: off`. Extracted so the WS `ready` payload and the REST song-info
+    payload cannot drift: the stems plugin now preloads from REST and then has
+    to agree with what the WS says a moment later, or it would rebuild the whole
+    graph for nothing.
+    """
+    if isinstance(raw, str):
+        return raw.lower() not in ("off", "false", "0", "no")
+    return bool(raw)
+
+
 def partition_stems(stems: list[dict]) -> tuple[dict | None, list[dict]]:
     """Split stem descriptors into (mixdown, instrument_stems) for PLAYBACK.
 
@@ -1100,12 +1114,11 @@ def load_song(
         sfile = str(s.get("file", ""))
         if not sid or not sfile:
             continue
-        default_val = s.get("default", True)
-        if isinstance(default_val, str):
-            default_on = default_val.lower() not in ("off", "false", "0", "no")
-        else:
-            default_on = bool(default_val)
-        stems.append({"id": sid, "file": sfile, "default": default_on})
+        stems.append({
+            "id": sid,
+            "file": sfile,
+            "default": stem_default_on(s.get("default", True)),
+        })
 
     # The complete mixdown is a stem (spec §5.3), but it is not a *layer*: lift
     # it out so that no consumer of `stems` — the mixer, the library's stem