mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-08-11 03:09:57 +00:00
fix(sloppak): the full mix is a stem — drop the invented original_audio key (#946)
* fix(sloppak): the full mix is a stem — drop the invented `original_audio` key (#933) Core read, served, and depended on `original_audio:` — a top-level manifest key this repo invented in #583 that the feedpak spec never defined. The format already had a home for the pre-separation mixdown: it is a stem. feedpak 1.15.0 (feedpak-spec#53) RESERVES the id `full` for it, so read it from there. The key existed to work around a bug in our own reader. The packer's comment said so plainly: "we must NOT list the full mix as a playable stem — the player sums every entry in `stems` and does not gate playback on `default`, so a listed full mix plays on top of the stems". Faced with a reader that would double the song, the packer put the mixdown outside `stems` and invented a key to point at it. The fix belongs in the reader, and that is what this is. load_song() now partitions the stem list: `full` comes out as LoadedSloppak.full_mix, the instruments stay in .stems. Nothing that sums stems or draws one fader per stem can see the mixdown, so retaining it is safe — which is what lets the packer put it where the format says it goes. - ws_highway: `song_info` gains full_mix_url / has_full_mix. The old original_audio_url / has_original_audio remain as deprecated aliases for one release so an older stems plugin keeps working (#945). - `stems` on the wire, and stem_ids / stem_count in the library index, are now INSTRUMENT stems only — a separated pack that retains its mixdown no longer advertises a bogus "full" chip or an inflated stem count. - enrichment: fingerprint against the mixdown wherever it lives. This widens coverage — _song_audio_file() previously returned None for any pack without the invented key, so fingerprinting silently did nothing for nearly every pack. - sloppak: `original_audio:` is still READ as a deprecated fallback, because every pack in the wild carries it and would otherwise lose its pristine mix. tools/migrate_full_mix_stem.py rewrites those packs into the spec shape (original/full.ogg -> stems/full.ogg, add the `full` stem at default:off, drop the key); the fallback and the aliases die with #945. The spec gate keeps the debt honest: the grandfather entry now tracks #945, and the gate fails if it goes stale. Verified: spec gate OK (4/4, incl. ingesting the spec's new example pack that retains `full`); 2493 python tests, 995 js tests; migrator round-tripped over real packs from the library and the results pass the spec's reference validator. * fix(migrate): discover directory-form packs instead of silently skipping them iter_packs() searched only files, so a directory-form pack (`song.sloppak/`, the authoring shape) was walked INTO and never yielded — silently missed by a run that's meant to be exhaustive. Discover suffix-named directories too (yielded whole, not descended into), and route packs through migrate_pack/verify_pack. Directory packs are REPORTED as `dir-form-unsupported`, not rewritten in place: a single-file pack is replaced atomically (a fully-built temp archive swapped in with one os.replace), but a populated directory can't be swapped that way, so an interrupted in-place rewrite could leave an authoring pack half-migrated. The status is a problem status, so it counts against the run's exit code and shows in the summary — the operator re-packs or migrates it as a `.feedpak` instead of it vanishing from the report. Addresses a CodeRabbit review finding. Signed-off-by: Kris Anderson <topkoa@gmail.com> * fix(migrate): verify requires an explicit `off` on a retained full mix verify_zip accepted any non-truthy `default` on a multi-stem `full` (missing, empty, boolean, `false`/`no`/`0`, malformed) as "ok". But core defaults an ABSENT `default` to True — ON (lib/sloppak.py: `s.get("default", True)`) — and treats an empty/unrecognized string as ON too, so a migrated-shape pack whose `full` stem has a missing or blank default beside instrument stems would actually play the mixdown on open and double the song. verify was certifying that as safe. Require an explicit normalized `off` beside instrument stems: `on`-ish values are reported `full-stem-default-on` (actively plays), everything that is not a normalized `off` is reported `full-stem-default-not-off`. The migrator already writes the literal `off`, so its own output is unaffected; this also certifies the pack is in the tool's canonical, most-portable shape. The len>1 gate is kept, so a sole `full` stem (which IS the audio) is not policed. Adds parametrized coverage for missing / empty / boolean / off-ish / malformed defaults, and a sole-full-stem case. Addresses a CodeRabbit review finding. Signed-off-by: Kris Anderson <topkoa@gmail.com> --------- Signed-off-by: Kris Anderson <topkoa@gmail.com> Co-authored-by: Kris Anderson <topkoa@gmail.com>
This commit is contained in:
co-authored by
Kris Anderson
parent
d876ded00f
commit
329cc86315
@@ -0,0 +1,429 @@
|
||||
"""tools/migrate_full_mix_stem.py — packs off the deprecated `original_audio:` key.
|
||||
|
||||
The migration moves real audio inside tens of thousands of archives, so the
|
||||
interesting cases are the ones where it must NOT act: a pack it would corrupt, a
|
||||
pack it has already done, a pack whose mixdown isn't where the key claims.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
_SPEC = importlib.util.spec_from_file_location(
|
||||
"migrate_full_mix_stem",
|
||||
Path(__file__).resolve().parent.parent / "tools" / "migrate_full_mix_stem.py",
|
||||
)
|
||||
mig = importlib.util.module_from_spec(_SPEC)
|
||||
_SPEC.loader.exec_module(mig)
|
||||
|
||||
|
||||
def _manifest(**extra) -> dict:
|
||||
m = {
|
||||
"feedpak_version": "1.13.0",
|
||||
"title": "T",
|
||||
"artist": "A",
|
||||
"duration": 1.0,
|
||||
"arrangements": [{"id": "lead", "file": "arrangements/lead.json"}],
|
||||
"stems": [
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
|
||||
{"id": "drums", "file": "stems/drums.ogg", "default": "on"},
|
||||
],
|
||||
"original_audio": "original/full.ogg",
|
||||
}
|
||||
m.update(extra)
|
||||
return m
|
||||
|
||||
|
||||
def _write_pack(path: Path, manifest: dict, files: dict[str, bytes] | None = None) -> Path:
|
||||
files = files or {
|
||||
"original/full.ogg": b"MIXDOWN",
|
||||
"stems/guitar.ogg": b"g",
|
||||
"stems/drums.ogg": b"d",
|
||||
"arrangements/lead.json": b"{}",
|
||||
}
|
||||
with zipfile.ZipFile(path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.writestr("manifest.yaml", yaml.safe_dump(manifest, sort_keys=False))
|
||||
for name, data in files.items():
|
||||
zf.writestr(name, data)
|
||||
return path
|
||||
|
||||
|
||||
def _read(path: Path) -> tuple[dict, set[str]]:
|
||||
with zipfile.ZipFile(path) as zf:
|
||||
return yaml.safe_load(zf.read("manifest.yaml")), set(zf.namelist())
|
||||
|
||||
|
||||
# ── plan_manifest: the decisions, without the archives ──────────────────────
|
||||
|
||||
def test_plan_adds_the_full_stem_and_drops_the_key():
|
||||
new, move = mig.plan_manifest(_manifest())
|
||||
assert move == "original/full.ogg"
|
||||
assert "original_audio" not in new
|
||||
assert new["stems"][0] == {
|
||||
"id": "full",
|
||||
"file": "stems/full.ogg",
|
||||
"default": "off",
|
||||
}
|
||||
# The separated stems survive, in order, untouched.
|
||||
assert [s["id"] for s in new["stems"]] == ["full", "guitar", "drums"]
|
||||
assert new["feedpak_version"] == "1.15.0"
|
||||
|
||||
|
||||
def test_plan_marks_the_retained_mixdown_default_off():
|
||||
"""The one line that keeps a pre-1.15.0 reader from doubling the song: a
|
||||
reader that sums every stem still won't play `full` on open if it honours
|
||||
`default`, which has been normative since 1.0.0."""
|
||||
new, _ = mig.plan_manifest(_manifest())
|
||||
assert new["stems"][0]["default"] == "off"
|
||||
|
||||
|
||||
def test_plan_marks_a_sole_mixdown_default_on():
|
||||
"""With no separated stems the mixdown IS the audio — off would mute the pack."""
|
||||
new, _ = mig.plan_manifest(_manifest(stems=[]))
|
||||
assert new["stems"] == [{"id": "full", "file": "stems/full.ogg", "default": "on"}]
|
||||
|
||||
|
||||
def test_plan_preserves_unknown_keys_verbatim():
|
||||
"""Spec §3: a writer that re-emits a pack SHOULD preserve unknown keys."""
|
||||
new, _ = mig.plan_manifest(_manifest(source_tool="ExampleTool v1.2.3", rigs="rigs.json"))
|
||||
assert new["source_tool"] == "ExampleTool v1.2.3"
|
||||
assert new["rigs"] == "rigs.json"
|
||||
|
||||
|
||||
def test_plan_skips_an_already_migrated_pack():
|
||||
m = _manifest(
|
||||
stems=[{"id": "full", "file": "stems/full.ogg", "default": "off"}],
|
||||
)
|
||||
del m["original_audio"]
|
||||
with pytest.raises(mig.Skip):
|
||||
mig.plan_manifest(m)
|
||||
|
||||
|
||||
def test_plan_skips_a_pack_that_never_had_the_key():
|
||||
m = _manifest()
|
||||
del m["original_audio"]
|
||||
with pytest.raises(mig.Skip):
|
||||
mig.plan_manifest(m)
|
||||
|
||||
|
||||
def test_plan_drops_a_stale_key_without_moving_anything():
|
||||
"""Mixdown already a stem, dead key lingering beside it."""
|
||||
new, move = mig.plan_manifest(
|
||||
_manifest(stems=[{"id": "full", "file": "stems/full.ogg", "default": "off"}])
|
||||
)
|
||||
assert move == ""
|
||||
assert "original_audio" not in new
|
||||
assert [s["id"] for s in new["stems"]] == ["full"]
|
||||
|
||||
|
||||
def test_plan_forces_an_existing_full_stem_off_beside_instrument_stems():
|
||||
"""Dropping the stale key is not enough if the mixdown it duplicated is left
|
||||
ENABLED: a reader that honours `default` would then play the whole song on top
|
||||
of the stems on open. The migration must not hand back a pack in the exact
|
||||
state it exists to remove."""
|
||||
new, move = mig.plan_manifest(
|
||||
_manifest(
|
||||
stems=[
|
||||
{"id": "full", "file": "stems/full.ogg", "default": "on"},
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
|
||||
]
|
||||
)
|
||||
)
|
||||
assert move == ""
|
||||
assert new["stems"][0] == {"id": "full", "file": "stems/full.ogg", "default": "off"}
|
||||
assert new["stems"][1]["default"] == "on" # instruments untouched
|
||||
|
||||
|
||||
def test_plan_leaves_a_sole_full_stem_enabled_when_dropping_a_stale_key():
|
||||
"""No instruments beside it — the mixdown IS the audio. Forcing it off here
|
||||
would mute the pack."""
|
||||
new, _ = mig.plan_manifest(
|
||||
_manifest(stems=[{"id": "full", "file": "stems/full.ogg", "default": "on"}])
|
||||
)
|
||||
assert new["stems"] == [{"id": "full", "file": "stems/full.ogg", "default": "on"}]
|
||||
|
||||
|
||||
def test_plan_needs_no_move_when_the_key_already_points_at_the_canonical_path():
|
||||
new, move = mig.plan_manifest(_manifest(original_audio="stems/full.ogg"))
|
||||
assert move == ""
|
||||
assert new["stems"][0]["file"] == "stems/full.ogg"
|
||||
|
||||
|
||||
# ── migrate_zip: the archive rewrite ────────────────────────────────────────
|
||||
|
||||
def test_migrate_moves_the_audio_and_rewrites_the_manifest(tmp_path: Path):
|
||||
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
|
||||
|
||||
manifest, names = _read(pak)
|
||||
assert "original/full.ogg" not in names # the invented directory is gone
|
||||
assert "stems/full.ogg" in names # audio lives where the format says
|
||||
assert "original_audio" not in manifest
|
||||
assert manifest["stems"][0]["id"] == "full"
|
||||
assert mig.verify_zip(pak) == "ok"
|
||||
|
||||
|
||||
def test_migrate_preserves_the_mixdown_bytes(tmp_path: Path):
|
||||
"""It is a rename, not a re-encode. Losing a byte here loses the master audio."""
|
||||
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
|
||||
mig.migrate_zip(pak, dry_run=False)
|
||||
with zipfile.ZipFile(pak) as zf:
|
||||
assert zf.read("stems/full.ogg") == b"MIXDOWN"
|
||||
assert zf.read("stems/guitar.ogg") == b"g"
|
||||
|
||||
|
||||
def test_migrate_is_idempotent(tmp_path: Path):
|
||||
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
|
||||
before = pak.read_bytes()
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "skip"
|
||||
assert pak.read_bytes() == before # a re-run touches nothing
|
||||
|
||||
|
||||
def test_dry_run_changes_nothing(tmp_path: Path):
|
||||
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
|
||||
before = pak.read_bytes()
|
||||
assert mig.migrate_zip(pak, dry_run=True) == "would-migrate"
|
||||
assert pak.read_bytes() == before
|
||||
|
||||
|
||||
def test_migrate_refuses_when_the_mixdown_is_absent(tmp_path: Path):
|
||||
"""The key points at audio the archive doesn't contain. Fabricating a stem
|
||||
entry for a missing file would break every reader — refuse, don't guess."""
|
||||
pak = _write_pack(
|
||||
tmp_path / "song.feedpak",
|
||||
_manifest(),
|
||||
files={"stems/guitar.ogg": b"g", "arrangements/lead.json": b"{}"},
|
||||
)
|
||||
before = pak.read_bytes()
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "missing-audio"
|
||||
assert pak.read_bytes() == before
|
||||
|
||||
|
||||
def test_migrate_refuses_when_the_target_path_is_taken(tmp_path: Path):
|
||||
"""A `stems/full.ogg` that is NOT the mixdown already occupies the target.
|
||||
Overwriting it would destroy a stem."""
|
||||
pak = _write_pack(
|
||||
tmp_path / "song.feedpak",
|
||||
_manifest(),
|
||||
files={
|
||||
"original/full.ogg": b"MIXDOWN",
|
||||
"stems/full.ogg": b"SOMETHING-ELSE",
|
||||
"arrangements/lead.json": b"{}",
|
||||
},
|
||||
)
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "target-occupied"
|
||||
with zipfile.ZipFile(pak) as zf:
|
||||
assert zf.read("stems/full.ogg") == b"SOMETHING-ELSE"
|
||||
|
||||
|
||||
def test_migrate_drops_a_stale_key_beside_a_non_canonical_full_stem(tmp_path: Path):
|
||||
"""The mixdown is already a stem, but at a path of the pack's own choosing —
|
||||
which is legal (§2.2: readers resolve through the manifest, never by
|
||||
filename). Only the dead key needs removing. Demanding `stems/full.ogg` here
|
||||
would reject a perfectly valid pack as `missing-audio`."""
|
||||
m = _manifest(stems=[{"id": "full", "file": "audio/mixdown.ogg", "default": "off"}])
|
||||
pak = _write_pack(
|
||||
tmp_path / "song.feedpak",
|
||||
m,
|
||||
files={"audio/mixdown.ogg": b"MIXDOWN", "arrangements/lead.json": b"{}"},
|
||||
)
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
|
||||
|
||||
manifest, names = _read(pak)
|
||||
assert "original_audio" not in manifest
|
||||
assert manifest["stems"] == [
|
||||
{"id": "full", "file": "audio/mixdown.ogg", "default": "off"}
|
||||
]
|
||||
assert "audio/mixdown.ogg" in names # the audio never moved
|
||||
assert mig.verify_zip(pak) == "ok"
|
||||
|
||||
|
||||
# ── verify_zip ──────────────────────────────────────────────────────────────
|
||||
|
||||
def test_verify_rejects_a_retained_mixdown_that_plays_on_open(tmp_path: Path):
|
||||
"""The hazard the migration must never create: `full` alongside instrument
|
||||
stems AND default-on means a summing reader plays the whole song twice."""
|
||||
m = _manifest(
|
||||
stems=[
|
||||
{"id": "full", "file": "stems/full.ogg", "default": "on"},
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
|
||||
]
|
||||
)
|
||||
del m["original_audio"]
|
||||
pak = _write_pack(
|
||||
tmp_path / "song.feedpak",
|
||||
m,
|
||||
files={"stems/full.ogg": b"M", "stems/guitar.ogg": b"g"},
|
||||
)
|
||||
assert mig.verify_zip(pak) == "full-stem-default-on"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"default, expected",
|
||||
[
|
||||
({"default": "off"}, "ok"), # the one safe, canonical shape
|
||||
({"default": "OFF"}, "ok"), # case-insensitive
|
||||
({"default": " off "}, "ok"), # surrounding whitespace tolerated
|
||||
({}, "full-stem-default-not-off"), # MISSING — core defaults to True (ON)
|
||||
({"default": ""}, "full-stem-default-not-off"), # empty → ON in core
|
||||
({"default": False}, "full-stem-default-not-off"), # boolean, not the string
|
||||
({"default": True}, "full-stem-default-on"), # boolean truthy → plays
|
||||
({"default": "false"}, "full-stem-default-not-off"), # off-ish but non-canonical
|
||||
({"default": "0"}, "full-stem-default-not-off"),
|
||||
({"default": "no"}, "full-stem-default-not-off"),
|
||||
({"default": "maybe"}, "full-stem-default-not-off"), # malformed
|
||||
({"default": "on"}, "full-stem-default-on"),
|
||||
({"default": "yes"}, "full-stem-default-on"),
|
||||
({"default": "1"}, "full-stem-default-on"),
|
||||
],
|
||||
)
|
||||
def test_verify_requires_an_explicit_off_on_a_retained_mixdown(tmp_path, default, expected):
|
||||
"""Beside instrument stems, `full` is safe only with an explicit normalized
|
||||
`off`. Core defaults an ABSENT `default` to ON and treats empty/unknown as
|
||||
ON, so a missing or blank default is the double-audio hazard itself, not a
|
||||
lesser one — `verify` must not certify it."""
|
||||
m = _manifest(
|
||||
stems=[
|
||||
{"id": "full", "file": "stems/full.ogg", **default},
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
|
||||
]
|
||||
)
|
||||
del m["original_audio"]
|
||||
pak = _write_pack(
|
||||
tmp_path / f"{tmp_path.name}.feedpak",
|
||||
m,
|
||||
files={"stems/full.ogg": b"M", "stems/guitar.ogg": b"g"},
|
||||
)
|
||||
assert mig.verify_zip(pak) == expected
|
||||
|
||||
|
||||
def test_verify_ignores_default_on_a_sole_full_stem(tmp_path: Path):
|
||||
"""A single `full` stem IS the audio — the len>1 gate means its default is
|
||||
not policed, so an on/absent default is fine (off would mute the pack)."""
|
||||
for default in ({"default": "on"}, {}, {"default": ""}):
|
||||
m = _manifest(stems=[{"id": "full", "file": "stems/full.ogg", **default}])
|
||||
del m["original_audio"]
|
||||
pak = _write_pack(
|
||||
tmp_path / f"{tmp_path.name}-{len(default)}.feedpak",
|
||||
m,
|
||||
files={"stems/full.ogg": b"M"},
|
||||
)
|
||||
assert mig.verify_zip(pak) == "ok"
|
||||
|
||||
|
||||
def test_verify_rejects_an_unmigrated_pack(tmp_path: Path):
|
||||
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
|
||||
assert mig.verify_zip(pak) == "still-has-key"
|
||||
|
||||
|
||||
# ── Unsafe manifest paths must not be laundered into playable audio ─────────
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"rel", ["../outside.ogg", "/etc/passwd", "a/../../x.ogg", "C:/x.ogg", "a\\b.ogg"]
|
||||
)
|
||||
def test_migrate_refuses_an_unsafe_full_mix_path(tmp_path: Path, rel: str):
|
||||
"""Core's loader REFUSES a full-mix path that escapes the pack — such a pack
|
||||
simply has no full mix, and the audio is inert. Migrating it into
|
||||
`stems/full.ogg` would take content the reader deliberately rejected and hand
|
||||
it back as a valid, playable stem. Report it; never promote it."""
|
||||
pak = _write_pack(
|
||||
tmp_path / "song.feedpak",
|
||||
_manifest(original_audio=rel),
|
||||
files={rel: b"EVIL", "stems/guitar.ogg": b"g"},
|
||||
)
|
||||
before = pak.read_bytes()
|
||||
assert mig.migrate_zip(pak, dry_run=False) == "unsafe-path"
|
||||
assert pak.read_bytes() == before
|
||||
|
||||
|
||||
def test_safe_relpath_accepts_ordinary_pack_paths():
|
||||
assert mig.is_safe_relpath("stems/full.ogg")
|
||||
assert mig.is_safe_relpath("original/full.ogg")
|
||||
assert not mig.is_safe_relpath("")
|
||||
assert not mig.is_safe_relpath("a//b.ogg")
|
||||
|
||||
|
||||
# ── Damaged packs must not abort the run ────────────────────────────────────
|
||||
|
||||
def test_a_corrupt_archive_is_reported_not_fatal(tmp_path: Path, capsys):
|
||||
"""A real library has damage in it — a truncated download, an archive left
|
||||
half-written by an interrupted converter. One of those must not kill a
|
||||
50,000-pack run and throw away the summary: the pack is reported, skipped,
|
||||
and everything else still migrates."""
|
||||
good = _write_pack(tmp_path / "good.feedpak", _manifest())
|
||||
bad = tmp_path / "bad.feedpak"
|
||||
bad.write_bytes(b"this is not a zip file at all")
|
||||
|
||||
rc = mig.main([str(tmp_path)])
|
||||
out = capsys.readouterr().out
|
||||
|
||||
assert rc == 1 # a problem pack fails the run's exit code
|
||||
assert "corrupt-zip" in out
|
||||
assert "migrated" in out
|
||||
assert mig.verify_zip(good) == "ok" # the healthy pack still got migrated
|
||||
assert bad.read_bytes() == b"this is not a zip file at all" # untouched
|
||||
|
||||
|
||||
# ── Directory-form (authoring) packs are discovered, not silently skipped ────
|
||||
|
||||
def _write_dir_pack(path: Path, manifest: dict, files: dict[str, bytes] | None = None) -> Path:
|
||||
"""Build a directory-form pack (`song.sloppak/`), the authoring shape."""
|
||||
files = files or {
|
||||
"original/full.ogg": b"MIXDOWN",
|
||||
"stems/guitar.ogg": b"g",
|
||||
"stems/drums.ogg": b"d",
|
||||
"arrangements/lead.json": b"{}",
|
||||
}
|
||||
path.mkdir()
|
||||
(path / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
|
||||
for name, data in files.items():
|
||||
p = path / name
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_bytes(data)
|
||||
return path
|
||||
|
||||
|
||||
def test_iter_packs_discovers_directory_form_packs(tmp_path: Path):
|
||||
"""A `song.sloppak/` directory is a pack; os.walk must yield it whole and
|
||||
NOT descend into it (its stems/ are contents, not packs)."""
|
||||
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
|
||||
z = _write_pack(tmp_path / "other.feedpak", _manifest())
|
||||
found = set(mig.iter_packs(tmp_path))
|
||||
assert d in found and z in found
|
||||
# Nothing inside the directory pack was yielded as its own pack.
|
||||
assert not any(d in p.parents for p in found)
|
||||
|
||||
|
||||
def test_iter_packs_yields_a_directly_passed_dir_pack(tmp_path: Path):
|
||||
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
|
||||
assert list(mig.iter_packs(d)) == [d]
|
||||
|
||||
|
||||
def test_directory_form_pack_is_reported_not_silently_skipped(tmp_path: Path, capsys):
|
||||
"""The migrator rewrites single-file packs atomically; a directory can't be
|
||||
swapped that way, so it is surfaced as a problem rather than vanishing from
|
||||
the run (the silent-skip this guards against) or being rewritten unsafely."""
|
||||
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
|
||||
good = _write_pack(tmp_path / "good.feedpak", _manifest())
|
||||
|
||||
rc = mig.main([str(tmp_path)])
|
||||
out = capsys.readouterr().out
|
||||
|
||||
assert rc == 1 # a reported problem fails the exit code
|
||||
assert "dir-form-unsupported" in out
|
||||
assert mig.verify_zip(good) == "ok" # the zip pack still migrated
|
||||
# The directory pack is untouched: legacy key intact, mixdown not moved.
|
||||
manifest = yaml.safe_load((d / "manifest.yaml").read_text())
|
||||
assert manifest.get("original_audio") == "original/full.ogg"
|
||||
assert (d / "original" / "full.ogg").read_bytes() == b"MIXDOWN"
|
||||
|
||||
|
||||
def test_verify_reports_directory_form_packs(tmp_path: Path):
|
||||
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
|
||||
assert mig.verify_pack(d) == "dir-form-unsupported"
|
||||
@@ -0,0 +1,277 @@
|
||||
"""The sloppak loader's handling of a pack's complete mixdown (#933).
|
||||
|
||||
The mixdown is a stem: feedpak spec §5.3 RESERVES the id `full` for it. It is a
|
||||
mixdown, not a layer — it already contains every instrument, so a reader that
|
||||
sums `stems` must never include it in that sum, and `load_song()` therefore
|
||||
lifts it OUT of `LoadedSloppak.stems` and onto `LoadedSloppak.full_mix`.
|
||||
|
||||
Also covers the DEPRECATED `original_audio:` manifest key — a key this repo
|
||||
invented (#583) before the spec reserved `full`, which every pack in the wild
|
||||
still carries. We read it as a fallback so those packs keep their full mix; we
|
||||
never write it. Those tests are the deprecation contract: they go when the key
|
||||
does (#945).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
import sloppak as sloppak_mod
|
||||
|
||||
|
||||
def _write_dir_sloppak(
|
||||
root: Path,
|
||||
manifest_extras: dict,
|
||||
*,
|
||||
write_legacy_full_mix: bool = False,
|
||||
stems: list[dict] | None = None,
|
||||
) -> Path:
|
||||
"""Build a minimal directory-form sloppak that load_song will accept.
|
||||
|
||||
Uses the tmp_path leaf name to make the sloppak filename unique per test,
|
||||
avoiding the module-level ``resolve_source_dir`` cache being poisoned by a
|
||||
previous test that happened to share the same "song.sloppak" filename.
|
||||
"""
|
||||
pak = root / f"{root.name}.sloppak"
|
||||
pak.mkdir()
|
||||
arr_dir = pak / "arrangements"
|
||||
arr_dir.mkdir()
|
||||
|
||||
arr = {
|
||||
"name": "Lead",
|
||||
"tuning": [0, 0, 0, 0, 0, 0],
|
||||
"capo": 0,
|
||||
"notes": [],
|
||||
"chords": [],
|
||||
"anchors": [],
|
||||
"handshapes": [],
|
||||
"templates": [],
|
||||
"beats": [],
|
||||
"sections": [],
|
||||
}
|
||||
(arr_dir / "lead.json").write_text(json.dumps(arr))
|
||||
|
||||
manifest = {
|
||||
"title": "Test",
|
||||
"artist": "Tester",
|
||||
"album": "",
|
||||
"year": 2026,
|
||||
"duration": 10.0,
|
||||
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
|
||||
"stems": (
|
||||
stems
|
||||
if stems is not None
|
||||
else [{"id": "guitar", "file": "stems/guitar.ogg", "default": True}]
|
||||
),
|
||||
}
|
||||
manifest.update(manifest_extras)
|
||||
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
|
||||
|
||||
if write_legacy_full_mix:
|
||||
orig_dir = pak / "original"
|
||||
orig_dir.mkdir()
|
||||
# The loader only checks presence (is_file); contents are irrelevant.
|
||||
(orig_dir / "full.ogg").write_bytes(b"OggS-not-real")
|
||||
|
||||
return pak
|
||||
|
||||
|
||||
def _load(pak_path: Path, tmp_path: Path):
|
||||
dlc_root = pak_path.parent
|
||||
cache = tmp_path / "cache"
|
||||
cache.mkdir()
|
||||
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
|
||||
|
||||
|
||||
def _separated(**extra) -> list[dict]:
|
||||
"""A separated pack that RETAINS its mixdown, as spec §5.3 asks writers to."""
|
||||
return [
|
||||
{"id": "full", "file": "stems/full.ogg", "default": False, **extra},
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
|
||||
{"id": "drums", "file": "stems/drums.ogg", "default": True},
|
||||
]
|
||||
|
||||
|
||||
# ── The `full` stem is the mixdown (spec §5.3) ───────────────────────────────
|
||||
|
||||
def test_full_stem_is_surfaced_as_the_mixdown(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
|
||||
loaded = _load(pak, tmp_path)
|
||||
# Manifest-relative, so the WS builds its URL exactly as it builds a stem's.
|
||||
assert loaded.full_mix == "stems/full.ogg"
|
||||
|
||||
|
||||
def test_full_stem_is_removed_from_the_stem_list(tmp_path: Path):
|
||||
"""The regression this whole change exists to prevent.
|
||||
|
||||
Every consumer sums `stems` into one mix and renders one fader per entry. The
|
||||
mixdown already contains every instrument, so leaving it in the list doubles
|
||||
the entire song — and muting `guitar` would still leave guitar audible inside
|
||||
it. That exact trap is why the packer invented `original_audio` rather than
|
||||
putting the mixdown where the format says it goes.
|
||||
"""
|
||||
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert [s["id"] for s in loaded.stems] == ["guitar", "drums"]
|
||||
|
||||
|
||||
def test_single_mix_pack_keeps_full_as_its_only_stem(tmp_path: Path):
|
||||
"""A pack whose ONLY stem is `full` is a single-mix pack, not a separated one.
|
||||
|
||||
There are no instruments to be pristine against, so the mixdown stays the sole
|
||||
playable stem and nothing is surfaced separately. Anything else would strip the
|
||||
stem list of the most common pack shape in the library and leave it silent.
|
||||
"""
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {}, stems=[{"id": "full", "file": "stems/full.ogg", "default": True}]
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
assert [s["id"] for s in loaded.stems] == ["full"]
|
||||
|
||||
|
||||
def test_every_full_entry_is_removed_not_just_the_first(tmp_path: Path):
|
||||
"""A malformed pack listing `full` twice must not leave one behind.
|
||||
|
||||
Removing the mixdown by object identity would drop only the entry we surface
|
||||
and leave its duplicate in the stem list — a whole copy of the song, summed
|
||||
with the instruments. That is the exact bug this partition prevents, so a
|
||||
duplicate must not smuggle it back in.
|
||||
"""
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path,
|
||||
{},
|
||||
stems=[
|
||||
{"id": "full", "file": "stems/full.ogg", "default": False},
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
|
||||
{"id": "full", "file": "original/full.ogg", "default": True},
|
||||
],
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix == "stems/full.ogg"
|
||||
assert [s["id"] for s in loaded.stems] == ["guitar"]
|
||||
|
||||
|
||||
def test_separated_pack_without_a_full_stem_has_no_mixdown(tmp_path: Path):
|
||||
"""Stems only, mixdown discarded — the pre-1.15.0 shape. Nothing to surface."""
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path,
|
||||
{},
|
||||
stems=[
|
||||
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
|
||||
{"id": "drums", "file": "stems/drums.ogg", "default": True},
|
||||
],
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
assert [s["id"] for s in loaded.stems] == ["guitar", "drums"]
|
||||
|
||||
|
||||
def test_single_mix_pack_ignores_a_lingering_deprecated_key(tmp_path: Path):
|
||||
"""`full` is the pack's only stem AND the old key is still there.
|
||||
|
||||
The stem wins, and it stays the sole playable stem — falling back to the key
|
||||
would surface the mixdown twice: once as the stem the player is already
|
||||
playing, and once as a "pristine" track for it to cross over to.
|
||||
"""
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path,
|
||||
{"original_audio": "original/full.ogg"},
|
||||
stems=[{"id": "full", "file": "stems/full.ogg", "default": True}],
|
||||
write_legacy_full_mix=True,
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
assert [s["id"] for s in loaded.stems] == ["full"]
|
||||
|
||||
|
||||
def test_full_stem_wins_over_the_deprecated_key(tmp_path: Path):
|
||||
"""A migrated pack that still carries the old key must use the stem."""
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path,
|
||||
{"original_audio": "original/full.ogg"},
|
||||
stems=_separated(),
|
||||
write_legacy_full_mix=True,
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix == "stems/full.ogg"
|
||||
|
||||
|
||||
# ── The library index must not advertise the mixdown as an instrument ────────
|
||||
|
||||
def test_extract_meta_excludes_the_mixdown_from_stem_ids(tmp_path: Path):
|
||||
"""The library's stem chips / stem_count come from here, and must agree with
|
||||
load_song() — otherwise the filter offers a "full" chip beside guitar+drums
|
||||
and counts a third stem that no mixer will ever show."""
|
||||
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
|
||||
meta = sloppak_mod.extract_meta(pak)
|
||||
assert meta["stem_ids"] == ["guitar", "drums"]
|
||||
assert meta["stem_count"] == 2
|
||||
|
||||
|
||||
def test_extract_meta_keeps_full_for_a_single_mix_pack(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {}, stems=[{"id": "full", "file": "stems/full.ogg", "default": True}]
|
||||
)
|
||||
meta = sloppak_mod.extract_meta(pak)
|
||||
assert meta["stem_ids"] == ["full"]
|
||||
assert meta["stem_count"] == 1
|
||||
|
||||
|
||||
# ── DEPRECATED `original_audio:` fallback — delete with the key (#945) ───────
|
||||
|
||||
def test_legacy_key_still_provides_the_full_mix(tmp_path: Path):
|
||||
"""Every pack written before the spec reserved `full` looks like this. Dropping
|
||||
the read would silently take the pristine mix away from all of them."""
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "original/full.ogg"}, write_legacy_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix == "original/full.ogg"
|
||||
# The legacy mixdown lives OUTSIDE `stems`, so the stem list is untouched.
|
||||
assert [s["id"] for s in loaded.stems] == ["guitar"]
|
||||
|
||||
|
||||
def test_legacy_key_absent_means_no_full_mix(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(tmp_path, {}, write_legacy_full_mix=True)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
|
||||
|
||||
def test_legacy_key_none_when_file_missing(tmp_path: Path):
|
||||
# Manifest points at a full mix that isn't on disk — disabled silently.
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "original/full.ogg"}, write_legacy_full_mix=False
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
|
||||
|
||||
def test_legacy_key_none_when_value_blank(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": " "}, write_legacy_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
|
||||
|
||||
# ── Security / path-traversal branches (legacy key only — a stem `file` is
|
||||
# resolved through the same /api/sloppak/.../file/ guard as every other stem)
|
||||
|
||||
def test_legacy_key_none_when_path_escapes_sloppak(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "../outside.ogg"}, write_legacy_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
|
||||
|
||||
def test_legacy_key_none_when_path_is_absolute(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "/etc/passwd"}, write_legacy_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.full_mix is None
|
||||
@@ -1,119 +0,0 @@
|
||||
"""End-to-end test for the sloppak loader recognising an `original_audio:`
|
||||
manifest key (the single full-mix file shipped alongside the separate stems)
|
||||
and surfacing the manifest-relative path on the LoadedSloppak."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
import sloppak as sloppak_mod
|
||||
|
||||
|
||||
def _write_dir_sloppak(root: Path, manifest_extras: dict, *, write_full_mix: bool) -> Path:
|
||||
"""Build a minimal directory-form sloppak that load_song will accept.
|
||||
|
||||
Uses the tmp_path leaf name to make the sloppak filename unique per test,
|
||||
avoiding the module-level ``resolve_source_dir`` cache being poisoned by a
|
||||
previous test that happened to share the same "song.sloppak" filename.
|
||||
"""
|
||||
pak = root / f"{root.name}.sloppak"
|
||||
pak.mkdir()
|
||||
arr_dir = pak / "arrangements"
|
||||
arr_dir.mkdir()
|
||||
|
||||
arr = {
|
||||
"name": "Lead",
|
||||
"tuning": [0, 0, 0, 0, 0, 0],
|
||||
"capo": 0,
|
||||
"notes": [],
|
||||
"chords": [],
|
||||
"anchors": [],
|
||||
"handshapes": [],
|
||||
"templates": [],
|
||||
"beats": [],
|
||||
"sections": [],
|
||||
}
|
||||
(arr_dir / "lead.json").write_text(json.dumps(arr))
|
||||
|
||||
manifest = {
|
||||
"title": "Test",
|
||||
"artist": "Tester",
|
||||
"album": "",
|
||||
"year": 2026,
|
||||
"duration": 10.0,
|
||||
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
|
||||
"stems": [{"id": "guitar", "file": "stems/guitar.ogg", "default": True}],
|
||||
}
|
||||
manifest.update(manifest_extras)
|
||||
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
|
||||
|
||||
if write_full_mix:
|
||||
orig_dir = pak / "original"
|
||||
orig_dir.mkdir()
|
||||
# The loader only checks presence (is_file); contents are irrelevant.
|
||||
(orig_dir / "full.ogg").write_bytes(b"OggS-not-real")
|
||||
|
||||
return pak
|
||||
|
||||
|
||||
def _load(pak_path: Path, tmp_path: Path):
|
||||
dlc_root = pak_path.parent
|
||||
cache = tmp_path / "cache"
|
||||
cache.mkdir()
|
||||
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
|
||||
|
||||
|
||||
# ── Happy path ───────────────────────────────────────────────────────────────
|
||||
|
||||
def test_load_song_attaches_original_audio_when_manifest_opts_in(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "original/full.ogg"}, write_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
# Stored as the manifest-relative string so server.py can build the URL the
|
||||
# same way it builds stem URLs.
|
||||
assert loaded.original_audio == "original/full.ogg"
|
||||
|
||||
|
||||
# ── Absent / degraded branches ───────────────────────────────────────────────
|
||||
|
||||
def test_load_song_original_audio_none_when_manifest_silent(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(tmp_path, {}, write_full_mix=True)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.original_audio is None
|
||||
|
||||
|
||||
def test_load_song_original_audio_none_when_file_missing(tmp_path: Path):
|
||||
# Manifest points at a full mix that isn't on disk — disabled silently.
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "original/full.ogg"}, write_full_mix=False
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.original_audio is None
|
||||
|
||||
|
||||
def test_load_song_original_audio_none_when_value_blank(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(tmp_path, {"original_audio": " "}, write_full_mix=True)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.original_audio is None
|
||||
|
||||
|
||||
# ── Security / path-traversal branches ──────────────────────────────────────
|
||||
|
||||
def test_load_song_original_audio_none_when_path_escapes_sloppak(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "../outside.ogg"}, write_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.original_audio is None
|
||||
|
||||
|
||||
def test_load_song_original_audio_none_when_path_is_absolute(tmp_path: Path):
|
||||
pak = _write_dir_sloppak(
|
||||
tmp_path, {"original_audio": "/etc/passwd"}, write_full_mix=True
|
||||
)
|
||||
loaded = _load(pak, tmp_path)
|
||||
assert loaded.original_audio is None
|
||||
Reference in New Issue
Block a user