fix(sloppak): the full mix is a stem — drop the invented original_audio key (#946)
Nightly / build-docker (push) Has been cancelled
ship-ci / ci (push) Waiting to run

* fix(sloppak): the full mix is a stem — drop the invented `original_audio` key (#933)

Core read, served, and depended on `original_audio:` — a top-level manifest key
this repo invented in #583 that the feedpak spec never defined. The format
already had a home for the pre-separation mixdown: it is a stem. feedpak 1.15.0
(feedpak-spec#53) RESERVES the id `full` for it, so read it from there.

The key existed to work around a bug in our own reader. The packer's comment
said so plainly: "we must NOT list the full mix as a playable stem — the player
sums every entry in `stems` and does not gate playback on `default`, so a listed
full mix plays on top of the stems". Faced with a reader that would double the
song, the packer put the mixdown outside `stems` and invented a key to point at
it. The fix belongs in the reader, and that is what this is.

load_song() now partitions the stem list: `full` comes out as
LoadedSloppak.full_mix, the instruments stay in .stems. Nothing that sums stems
or draws one fader per stem can see the mixdown, so retaining it is safe — which
is what lets the packer put it where the format says it goes.

- ws_highway: `song_info` gains full_mix_url / has_full_mix. The old
  original_audio_url / has_original_audio remain as deprecated aliases for one
  release so an older stems plugin keeps working (#945).
- `stems` on the wire, and stem_ids / stem_count in the library index, are now
  INSTRUMENT stems only — a separated pack that retains its mixdown no longer
  advertises a bogus "full" chip or an inflated stem count.
- enrichment: fingerprint against the mixdown wherever it lives. This widens
  coverage — _song_audio_file() previously returned None for any pack without
  the invented key, so fingerprinting silently did nothing for nearly every pack.
- sloppak: `original_audio:` is still READ as a deprecated fallback, because
  every pack in the wild carries it and would otherwise lose its pristine mix.
  tools/migrate_full_mix_stem.py rewrites those packs into the spec shape
  (original/full.ogg -> stems/full.ogg, add the `full` stem at default:off, drop
  the key); the fallback and the aliases die with #945.

The spec gate keeps the debt honest: the grandfather entry now tracks #945, and
the gate fails if it goes stale.

Verified: spec gate OK (4/4, incl. ingesting the spec's new example pack that
retains `full`); 2493 python tests, 995 js tests; migrator round-tripped over
real packs from the library and the results pass the spec's reference validator.

* fix(migrate): discover directory-form packs instead of silently skipping them

iter_packs() searched only files, so a directory-form pack (`song.sloppak/`, the
authoring shape) was walked INTO and never yielded — silently missed by a run
that's meant to be exhaustive. Discover suffix-named directories too (yielded
whole, not descended into), and route packs through migrate_pack/verify_pack.

Directory packs are REPORTED as `dir-form-unsupported`, not rewritten in place:
a single-file pack is replaced atomically (a fully-built temp archive swapped in
with one os.replace), but a populated directory can't be swapped that way, so an
interrupted in-place rewrite could leave an authoring pack half-migrated. The
status is a problem status, so it counts against the run's exit code and shows
in the summary — the operator re-packs or migrates it as a `.feedpak` instead of
it vanishing from the report. Addresses a CodeRabbit review finding.

Signed-off-by: Kris Anderson <topkoa@gmail.com>

* fix(migrate): verify requires an explicit `off` on a retained full mix

verify_zip accepted any non-truthy `default` on a multi-stem `full` (missing,
empty, boolean, `false`/`no`/`0`, malformed) as "ok". But core defaults an ABSENT
`default` to True — ON (lib/sloppak.py: `s.get("default", True)`) — and treats an
empty/unrecognized string as ON too, so a migrated-shape pack whose `full` stem
has a missing or blank default beside instrument stems would actually play the
mixdown on open and double the song. verify was certifying that as safe.

Require an explicit normalized `off` beside instrument stems: `on`-ish values are
reported `full-stem-default-on` (actively plays), everything that is not a
normalized `off` is reported `full-stem-default-not-off`. The migrator already
writes the literal `off`, so its own output is unaffected; this also certifies
the pack is in the tool's canonical, most-portable shape. The len>1 gate is kept,
so a sole `full` stem (which IS the audio) is not policed.

Adds parametrized coverage for missing / empty / boolean / off-ish / malformed
defaults, and a sole-full-stem case. Addresses a CodeRabbit review finding.

Signed-off-by: Kris Anderson <topkoa@gmail.com>

---------

Signed-off-by: Kris Anderson <topkoa@gmail.com>
Co-authored-by: Kris Anderson <topkoa@gmail.com>
This commit is contained in:
Byron Gamatos
2026-07-13 12:22:42 -04:00
committed by GitHub
co-authored by Kris Anderson
parent d876ded00f
commit 329cc86315
10 changed files with 1430 additions and 195 deletions
+43 -19
View File
@@ -321,11 +321,16 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
audio_url = None
audio_error: str | None = None # Surfaced in song_info when audio_url is None
stems_payload: list[dict] = []
# URL of the single full-mix audio (sloppak `original_audio:`), when the
# pack ships one. The stems plugin uses this to play the untouched mix
# while every stem slider is at unity; None otherwise (separate stems
# only, loose folder, or archive).
original_audio_url: str | None = None
# URL of the pack's complete mixdown — the RESERVED `full` stem (spec
# §5.3), which sloppak.load_song() lifts out of `stems` because it is a
# mixdown, not a layer. The stems plugin plays it while every stem slider
# is at unity (separation is lossy, so it beats re-summing the stems) and
# crosses to the separated stems as soon as one is attenuated.
#
# None when the pack has no mixdown to offer separately from its stems:
# a single-mix pack (its one stem IS the mixdown), a loose folder, or an
# archive.
full_mix_url: str | None = None
if is_loose:
# Loose folder filenames are relative paths (artist/album/song).
# Hash the *canonical* dlc-relative path (so two URL spellings
@@ -365,21 +370,25 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
url = f"/api/sloppak/{q_fn}/file/{quote(s['file'])}"
stems_payload.append({"id": s["id"], "url": url, "default": s["default"]})
# Full-mix URL (served by the same /api/sloppak/.../file/ endpoint).
if loaded_slop is not None and loaded_slop.original_audio:
original_audio_url = (
f"/api/sloppak/{q_fn}/file/{quote(loaded_slop.original_audio)}"
if loaded_slop is not None and loaded_slop.full_mix:
full_mix_url = (
f"/api/sloppak/{q_fn}/file/{quote(loaded_slop.full_mix)}"
)
if stems_payload:
# Stems present: keep the core <audio> pointed at stem[0]. This
# URL is only ever heard in the degraded path (stems plugin
# refuses takeover / decode fails); the full-mix↔stems switch is
# driven client-side by `original_audio_url`, not `audio_url`.
# driven client-side by `full_mix_url`, not `audio_url`.
audio_url = stems_payload[0]["url"]
elif original_audio_url:
elif full_mix_url:
# Stem-less full-mix pack: nothing to separate, so play the full
# mix natively through the core <audio>. The stems plugin's
# onSongReady returns early on an empty stems list (no graph).
audio_url = original_audio_url
# Reachable only via the deprecated `original_audio:` key, whose
# packs put the mixdown outside `stems` — a pack that carries its
# mixdown as the `full` stem has it IN `stems`, so it lands in the
# branch above with stems_payload == [full].
audio_url = full_mix_url
else:
audio_error = "This sloppak has no playable stems."
else:
@@ -521,16 +530,31 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
# for the credits overlay, so minigames / synthetic highway uses
# (no manifest) never trigger it.
"authors": _sanitize_authors(loaded_slop.manifest) if (is_slop and loaded_slop is not None) else [],
# Instrument stems ONLY. The pack's complete mixdown (the RESERVED
# `full` stem, spec §5.3) is deliberately NOT in this list: consumers
# sum `stems` into one mix and render one fader per entry, and the
# mixdown is neither a layer nor an instrument — summing it would
# double the whole song. It is surfaced separately, below.
"stems": stems_payload,
# Full-mix audio (sloppak `original_audio:`) served alongside the
# separate `stems`. The stems plugin plays this single file while
# every stem slider is at unity and switches to the separate stems
# the moment one drops below 100%. None when the pack ships stems
# only. `has_*` flags mirror the has_drum_tab/has_keys convention so
# a client can branch without re-deriving from the URLs.
"original_audio_url": original_audio_url,
"has_original_audio": bool(original_audio_url),
# The complete mixdown, served by the same /api/sloppak/.../file/
# endpoint as the stems. The stems plugin plays this single file
# while every stem slider is at unity and crosses to the separated
# stems the moment one drops below 100% — separation is lossy, so the
# mixdown is strictly better audio when nothing is muted. None when
# the pack has no mixdown apart from its stems. The `has_*` flags
# mirror the has_drum_tab/has_keys convention so a client can branch
# without re-deriving from the URLs.
"full_mix_url": full_mix_url,
"has_full_mix": bool(full_mix_url),
"has_stems": bool(stems_payload),
# DEPRECATED aliases of the two keys above, kept so a client built
# against the old frame keeps working across one release. They were
# named after `original_audio:` — a manifest key this repo invented
# and the feedpak spec never had (#933). The key is gone; the mixdown
# is a stem. Remove these once the shipped stems plugin reads
# `full_mix_url` (#945).
"original_audio_url": full_mix_url,
"has_original_audio": bool(full_mix_url),
# Surface a drum_tab presence flag so the visualization picker
# can auto-activate the drums plugin even when the chosen
# arrangement isn't named "Drums" (drum_tab.json lives next