Compare commits

...
Author SHA1 Message Date
be473dc7af Career v3: Gold tier — verified improv upgrades an earned badge (#960)
* feat(career): Gold tier — a family-style goldImprov artifact upgrades an earned badge

The drill-state relay's goldImprov map (virtuoso gold_improv mints,
gained-only merged like drill nodes) turns an earned badge gold when the
passport's genre — or its genre family — has a verified improv artifact.
Gold never substitutes for the badge bar: gold-without-bronze stays
in_progress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(career): Gold tier frontend — relay, ceremony, slam, gold ink everywhere

The drill-state relay now carries virtuoso's goldImprov map; a badge that
comes back gold gets its own ceremony + notification (tier-suffixed seen
ids — the bronze moment stays seen under its legacy id, a gold slam marks
both), a gold stamp slam in the book, gold ink on the shelf-cover mini
stamp, and the real gold foil chip. The bronze page's dashed 'Gold rung
coming' preview becomes a live invitation to jam the style in Virtuoso.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(career): gold review fixes — family-space style matching, intake guards, rail counter

The review's showstopper: virtuoso mints goldImprov under raw
STYLE_PALETTES ids ('punk', 'djent', 'disco'), which are mostly NOT
family keys — the tier check now matches in family space (artifact style
and passport genre bucket through the same _genre_family keyword match),
so a 'punk' gold reaches a 'punk rock' passport. Also: non-dict
goldImprov 400s loudly instead of silently dropping; evidence-free
artifacts (no verifier) never mint; goldImprov gets the same pre-merge
size bound byNode has (junk under the cap could otherwise persist
forever and wedge every later relay at the post-merge check); the
instrument-rail badge counter counts gold (earning gold no longer made a
badge vanish from the rail); first-artifact-wins is now asserted against
the persisted snapshot instead of vacuously.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:26:36 +02:00
K. O. A.andGitHub 0d35228d56 fix: remote transcription posts to /transcribe, not /align (stem-splitter#17) (#959)
ship-ci / ci (push) Waiting to run
* fix: remote transcription posts to /transcribe, not /align (stem-splitter#17)

transcribe_vocals_remote() POSTed the vocal stem to /align. That endpoint is FORCED ALIGNMENT —
"here are the lyrics, tell me when each word is sung" — and its `text` field is required. We have
no lyrics; transcribing them is the entire point. So the server rejected every request with a 422
from FastAPI's validation layer, before its handler ever ran, and remote transcription has never
worked for anyone.

It now posts to /transcribe (added in feedBack-demucs-server#14), which takes only the audio.

`language` moves from the query string to the FORM BODY, where the server actually reads it
(Form("")). As a query param it was silently ignored, so an explicit hint did nothing and
Whisper's auto-detection quietly decided instead — loading the wrong wav2vec2 aligner. It
"worked", it was just wrong, which is the failure mode that hides for months.

Error bodies are no longer cut at 300 chars. The body IS the diagnosis: a 422's JSON names the
field it rejected, a 500's traceback answers on its LAST line. Both got decapitated — which is
part of why this stayed invisible for so long. The message explaining the bug was inside the part
that got cut.

Nothing caught any of this because every test of this module tested the MAPPER, fed a hand-written
dict. The mapper was always fine. The request was never exercised, and the request was the bug.
tests/test_lyrics_transcribe_remote.py now pins it: the endpoint, the form field, the multipart
upload, the bearer token, an instrumental returning no lyrics rather than an error, and a 404
saying the server is too old. Verified they FAIL against /align + params.

Signed-off-by: topkoa <topkoa@gmail.com>

* fix: make the error-body cap an actual bound; correct the docstring's endpoint

- _err_body() appended the truncation marker AFTER slicing to _MAX_ERR_BODY, so the result could
  exceed the cap it exists to enforce (4014 chars for a 4000 bound). A cap that is only a
  suggestion surprises exactly the callers who trust it — a log line, a job record persisted to
  disk and re-read on every load. The marker now fits inside the bound.

  It also stripped after measuring, so a short JSON body followed by kilobytes of trailing
  whitespace got truncated: real content cut to make room for blanks. Strip first, then measure.

- The public docstring still advertised /align — the exact contract this PR exists to change, in
  the one place a reader would look for it. It now says what the function does and why, and that
  an older server answers 404.

Found by Copilot and CodeRabbit on #959.

Signed-off-by: topkoa <topkoa@gmail.com>

* fix: keep the exception line when truncating — the tail is the answer

_err_body() kept only the HEAD of an over-long body. On a traceback the last line is the
diagnosis, and the docstring said exactly that while the code threw it away: a 4000-char window
holding "Traceback (most recent call last)" and none of the exception is a window onto nothing.
Same mistake as the 300-char cap it replaced, one level up — cutting off precisely the part the
function exists to preserve.

Head AND tail now, both inside the bound: two thirds head (what was being attempted), one third
tail (what actually went wrong), with the marker between them. Verified the test FAILS against
head-only truncation.

Found by Copilot on #959.

Signed-off-by: topkoa <topkoa@gmail.com>

* fix: every failure out of transcribe_vocals_remote() is a RuntimeError; 404 says why

The docstring promised one failure mode — RuntimeError — and the caller (_maybe_transcribe_lyrics)
catches exactly that so one song's failed lyrics don't take down the batch around it. But a DNS
failure, a timeout, a reset connection or an unreadable stem escaped as requests.RequestException
or OSError, walked straight past that handler, and turned "this song's lyrics failed" into "the
whole batch died".

A 404 now explains itself. Bare "404" sends someone hunting for a typo in their server URL; the
real answer is that their server predates /transcribe, and we are the only ones in a position to
know that.

Found by Copilot on #959.

Signed-off-by: topkoa <topkoa@gmail.com>

---------

Signed-off-by: topkoa <topkoa@gmail.com>
2026-07-14 01:58:24 -04:00
dd1927e27b feat(career): gigs frontend — poster, runner strip, summary, encore (#956)
ship-ci / ci (push) Waiting to run
* feat(career): gigs frontend — poster, runner strip, summary, encore

Career v3, WS3 (frontend half), rebuilt cleanly on merged main (v3-a/b/c
in) after git interleaved the structurally-similar canvas functions:

- Book a gig from any opened passport: /gigs/propose renders as a GIG
  POSTER (venue presents GENRE NIGHT, numbered bill) with re-roll and
  Save/Copy poster (natively-drawn canvas via blob-io, audible failure
  paths, slash-safe filenames).
- Play the gig: venue override + Venue viz handoff, then
  playQueue.start(..., {source:'gig'}) — the queue's auto-advance runs
  the set; zero new playback machinery.
- Floating gig strip (body-level, pointer-events none, z 35 per the
  chrome invariant) tracks set position and names what's next.
- Completion = song:ended with an empty queue → POST /gigs → summary
  poster overlay with per-song accuracies; encore fires the crowd
  celebrate + confetti (reduced-motion: neither). song:stop with a dead
  queue = abandoned (no log); end-of-song teardown (queue still active)
  must NOT abandon.
- Gigs played render as dated rows in the passport book.

vm tests: runner advance/abandon semantics via the queue-state seam.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(career): stage restore survives a setViz throw; size-register row

CodeRabbit on #956: a setViz failure nulled the restore snapshot AFTER
the overrides were written, permanently borrowing the stage — snapshot
now captured before any write, write failures keep it intact. Also
registers career screen.js in docs/size-exemptions.md 'Planned, not
exempt' (1,516 lines; max-lines WARNS non-blocking — the split plan
needs Byron's sign-off).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: register career screen.js in the size register (planned, not exempt)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 01:54:07 +02:00
7c897e9f2b feat(career): gigs backend — propose a setlist, log the completed set (#954)
* feat(career): gigs backend — propose a setlist, log the completed set

Career v3, WS3 (backend half). A gig is career's verb:

- POST /gigs/propose {instrument, genre, size}: setlist from the
  passport's own stubs — qualifying songs (per the genre's badge bar,
  family-aware) shuffled for a free re-roll, topped with the
  highest-accuracy near-bar songs as stakes, and filled from UNPLAYED
  genre songs when the passport is young (the first gig is how stubs
  start). Names the highest venue the current stars can book.
- POST /gigs: logs a COMPLETED set only (abandoned sets never log — no
  fail state). Per-song accuracy = MAX(last_accuracy) from song_stats,
  freshly written by the set's own plays; encore = avg ≥ the data-driven
  bar (passports.json gig.encore_accuracy, 0.75). Appends to the career
  state file (same atomic _save_json pattern).
- Passports view: per-passport gigs (newest first, capped 20) and
  per-instrument gig_count — the profile wall's gig line lights up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(career): gig backfill offsets by qualifying taken, not picks length

CodeRabbit on #954: after the stakes loop appends near-bar songs,
qualifying[len(picks):] overshoots and skips eligible qualifying songs
— a stocked passport could still get a short set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 01:26:23 +02:00
6272af8d33 feat(career): profile passport wall, home career card, shareable PNG card (#955)
* feat(career): profile passport wall, home career card, shareable PNG card

Career v3, WS2. The identity artifact leaves the plugin tab:

- Profile: #v3-profile-passports-mount (core, one div) filled by career
  on v3:profile-rendered — per-instrument shelves of earned covers,
  hours, gig count, open-career link. Absent-not-empty.
- Home: the plugin-count stat tile becomes #v3-dash-career-slot with the
  old stat as fallback content; career replaces it with a trading-card
  tile (leather + foil shine, badge count, hours, closest-stamp ask) on
  the existing v3:dashboard-rendered event.
- Shareable card: static/js/blob-io.js (downloadBlob lifts the idiom
  duplicated verbatim in settings-io/diagnostics-export — both
  refactored; copyImageBlob wraps ClipboardItem, returns false to signal
  the download fallback). Earned passports get Save/Copy card: a
  natively-drawn 480×640 canvas (leather, stamp ring, stubs+hours line);
  copy falls back to download with a notice when the clipboard refuses.
- Mount-point convention documented in docs/plugin-v3-ui.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(career): external surfaces stay absent until a passport exists

CodeRabbit on #955: a bare commitment produced a zero-passport wall and
replaced the dashboard fallback. Docs also now say mounts may hold
fallback content and plugins REPLACE, never append.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 01:25:39 +02:00
831117fb96 feat(career): practice invitations — closest stamps + bring-these-up (#953)
* feat(career): practice invitations — closest stamps + bring-these-up

Career v3, WS1. The passport now points at the practice that pays:

- Stubs carry next_star_at (the same primitive _stars() uses) and each
  passport exposes `nearest`: the top 3 non-qualifying songs by distance
  to their next star, in the worklist order.
- "Closest stamps" strip above the shelf: the in-progress graded
  passports nearest to minting, each row naming the ask — N more songs
  (with the nearest title + best %) or the blocking Virtuoso drill.
  Rows open the passport.
- "Bring these up" list on the stubs page of in-progress passports;
  earned pages stay memorabilia (no homework on a won badge).
- Invitation-voiced throughout: no meters, no completion pressure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(test): comment says qualifying-bar ranking, matching the assertion

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 01:10:55 +02:00
6cc0312661 feat(career): genre families — sub-genres inherit the family drill (#951)
The enrichment fallback made the passport rack real (hundreds of MB
sub-genres) but only the five exact umbrella keys carried Virtuoso
drills. Genres now resolve to a family by keyword substring (MB's
vocabulary is open — 'metalcore' must hit metal without an alias),
first-match-wins in list order ('blues rock' → blues), and inherit the
family's requirement from the same genres map. Exact entries still win;
per-instrument scoping unchanged; unmatched genres stay songs-only.

Data: families for metal (incl. djent/grindcore/thrash/doom), blues,
jazz (bebop/swing/bossa), funk (disco), rock (punk/grunge/shoegaze).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 20:05:05 +02:00
329cc86315 fix(sloppak): the full mix is a stem — drop the invented original_audio key (#946)
Nightly / build-docker (push) Has been cancelled
ship-ci / ci (push) Waiting to run
* fix(sloppak): the full mix is a stem — drop the invented `original_audio` key (#933)

Core read, served, and depended on `original_audio:` — a top-level manifest key
this repo invented in #583 that the feedpak spec never defined. The format
already had a home for the pre-separation mixdown: it is a stem. feedpak 1.15.0
(feedpak-spec#53) RESERVES the id `full` for it, so read it from there.

The key existed to work around a bug in our own reader. The packer's comment
said so plainly: "we must NOT list the full mix as a playable stem — the player
sums every entry in `stems` and does not gate playback on `default`, so a listed
full mix plays on top of the stems". Faced with a reader that would double the
song, the packer put the mixdown outside `stems` and invented a key to point at
it. The fix belongs in the reader, and that is what this is.

load_song() now partitions the stem list: `full` comes out as
LoadedSloppak.full_mix, the instruments stay in .stems. Nothing that sums stems
or draws one fader per stem can see the mixdown, so retaining it is safe — which
is what lets the packer put it where the format says it goes.

- ws_highway: `song_info` gains full_mix_url / has_full_mix. The old
  original_audio_url / has_original_audio remain as deprecated aliases for one
  release so an older stems plugin keeps working (#945).
- `stems` on the wire, and stem_ids / stem_count in the library index, are now
  INSTRUMENT stems only — a separated pack that retains its mixdown no longer
  advertises a bogus "full" chip or an inflated stem count.
- enrichment: fingerprint against the mixdown wherever it lives. This widens
  coverage — _song_audio_file() previously returned None for any pack without
  the invented key, so fingerprinting silently did nothing for nearly every pack.
- sloppak: `original_audio:` is still READ as a deprecated fallback, because
  every pack in the wild carries it and would otherwise lose its pristine mix.
  tools/migrate_full_mix_stem.py rewrites those packs into the spec shape
  (original/full.ogg -> stems/full.ogg, add the `full` stem at default:off, drop
  the key); the fallback and the aliases die with #945.

The spec gate keeps the debt honest: the grandfather entry now tracks #945, and
the gate fails if it goes stale.

Verified: spec gate OK (4/4, incl. ingesting the spec's new example pack that
retains `full`); 2493 python tests, 995 js tests; migrator round-tripped over
real packs from the library and the results pass the spec's reference validator.

* fix(migrate): discover directory-form packs instead of silently skipping them

iter_packs() searched only files, so a directory-form pack (`song.sloppak/`, the
authoring shape) was walked INTO and never yielded — silently missed by a run
that's meant to be exhaustive. Discover suffix-named directories too (yielded
whole, not descended into), and route packs through migrate_pack/verify_pack.

Directory packs are REPORTED as `dir-form-unsupported`, not rewritten in place:
a single-file pack is replaced atomically (a fully-built temp archive swapped in
with one os.replace), but a populated directory can't be swapped that way, so an
interrupted in-place rewrite could leave an authoring pack half-migrated. The
status is a problem status, so it counts against the run's exit code and shows
in the summary — the operator re-packs or migrates it as a `.feedpak` instead of
it vanishing from the report. Addresses a CodeRabbit review finding.

Signed-off-by: Kris Anderson <topkoa@gmail.com>

* fix(migrate): verify requires an explicit `off` on a retained full mix

verify_zip accepted any non-truthy `default` on a multi-stem `full` (missing,
empty, boolean, `false`/`no`/`0`, malformed) as "ok". But core defaults an ABSENT
`default` to True — ON (lib/sloppak.py: `s.get("default", True)`) — and treats an
empty/unrecognized string as ON too, so a migrated-shape pack whose `full` stem
has a missing or blank default beside instrument stems would actually play the
mixdown on open and double the song. verify was certifying that as safe.

Require an explicit normalized `off` beside instrument stems: `on`-ish values are
reported `full-stem-default-on` (actively plays), everything that is not a
normalized `off` is reported `full-stem-default-not-off`. The migrator already
writes the literal `off`, so its own output is unaffected; this also certifies
the pack is in the tool's canonical, most-portable shape. The len>1 gate is kept,
so a sole `full` stem (which IS the audio) is not policed.

Adds parametrized coverage for missing / empty / boolean / off-ish / malformed
defaults, and a sole-full-stem case. Addresses a CodeRabbit review finding.

Signed-off-by: Kris Anderson <topkoa@gmail.com>

---------

Signed-off-by: Kris Anderson <topkoa@gmail.com>
Co-authored-by: Kris Anderson <topkoa@gmail.com>
2026-07-13 12:22:42 -04:00
Byron GamatosandGitHub d876ded00f fix(sloppak): bound the unpack cache; add read_member_bytes() so callers stop unpacking whole songs (#950)
* fix(sloppak): bound the unpack cache, and add a way to read a song without unpacking it

A tester's sloppak_cache reached 60 GB from an 1800-song library — his entire
library, unpacked, none of it played. Stems are already-compressed audio, so an
unpacked pack is ~1.1x its zip: the cache is a second, DECOMPRESSED copy of every
song it touches. It had no size cap, no LRU, and no cleanup of any kind — not even
when the song itself was deleted.

Two halves:

1. resolve_source_dir() now evicts least-recently-used songs to stay under a cap
   (FEEDBACK_SLOPPAK_CACHE_MAX_MB, default 4 GB ≈ 130 songs of recency; 0 disables).
   The sweep runs on unpack — the only moment the cache grows — so it can't drift.
   An evicted song is dropped from _source_cache too: get_cached_source_dir() is
   the only thing media.py consults before falling back, so a stale path there
   would 404 every stem for the rest of the process instead of re-unpacking.
   get_cached_source_dir() now also verifies the dir still exists, which makes
   "just delete sloppak_cache/ to reclaim disk" safe advice.

2. read_member_bytes() reads ONE file out of a pack without unpacking it — the
   same trick read_cover_bytes() uses so the library grid doesn't explode every
   pack to show a cover. Unpacking a whole song to read a few KB of JSON is ~45x
   write amplification; doing it in a loop over the library is what produced the
   60 GB. rig_builder's library-wide tone batch is the caller that did exactly
   that (fixed separately); this gives it, and everyone else, the right primitive.

Eviction is concurrency-safe: unpacks run 2-at-a-time, so a dir being written is
marked in-flight and the sweep skips it — checked and rmtree'd under one hold of
the guard, and the marker is released even if the unpack raises (a leaked marker
would make that dir permanently un-evictable).

read_member_bytes normalizes both the requested path AND the archive's stored
member names through safe_join, taking the last match — so './arrangements/x.json',
backslash members from Windows tooling, and duplicate members that normalize to
the same path all read back exactly as unpack-then-read did. Zip-slip is rejected
before anything is opened.

Tests: tests/test_sloppak_unpack_cache.py. All bite-tested (reverted each fix,
watched it fail) — including one that was passing vacuously: a freshly-unpacked
dir is the most-recently-used, so the LRU never reaches it and the in-flight race
test proved nothing until the packs were sized to force the sweep that far.

* test: split semicolon-joined statements (E702)

CodeRabbit on #950. Style only; no behaviour change.
2026-07-13 17:13:08 +02:00
18d77d2d41 feat(library): effective genre falls back to MusicBrainz enrichment (#949)
* feat(library): effective genre falls back to MusicBrainz enrichment

Converted packs rarely carry a genres manifest key — on Byron's real
library 1188/1190 songs had no genre, starving the genre facet and the
career passport rack (2 usable genres) while song_enrichment already
held MB genres for 636 matched songs.

The effective-genre expression now resolves: per-song override → pack
genre → json_extract(enrichment.genres, '$[0]') for MATCHED rows only
(review/failed candidates could carry the wrong recording's genres).
Same fast-path gating as before: the plain indexed column is used
unless overrides or enrichment genres actually exist; stand-in DBs
without the table degrade via the OperationalError guard. Career
passports pick this up automatically through _effective_genre_expr().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: changelog names manual rows in the enrichment fallback

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:53:06 +02:00
5921157f35 test(stats): prove seconds-only recency on a fresh row (#948)
CodeRabbit on #947: the prior lastPlayPosition POST already stamped
last_played_at, so the assertion passed even if the seconds-only path
left it unchanged — a guard that cannot fail. Assert on a fresh row.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 15:36:05 +02:00
31 changed files with 4030 additions and 326 deletions
+78
View File
@@ -8,6 +8,84 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **Gold tier (career passports)** — an earned badge turns **gold** when
Virtuoso verifies an improvised jam in the passport's style (the
`gold_improv` artifact relays with the drill snapshot; a genre inherits its
family's style, gained-only, and gold never substitutes for the badge bar
itself). Gold gets its own ceremony, stamp slam, foil chip, and gold ink on
the shelf cover, profile wall, and passport card; the bronze page's "Gold
rung coming" preview becomes a live invitation to jam it.
- **Gigs (the career verb, frontend)** — book a gig from any opened passport:
a gig poster proposes the setlist (re-roll for a different bill; save or
copy the poster as a PNG), "Play the gig" hands the set to the play queue
with the venue on stage, a floating strip tracks the set, and finishing it
logs dated entries with per-song accuracies in the passport book — with an
encore celebration (crowd eruption + confetti) when the whole set clears
the bar, and a summary poster to share. Quitting mid-set simply abandons
it: no log, no fail state.
- **Career on the Profile and Home pages** — the Profile gains a passport
wall (earned-badge covers per instrument, hours, gig count; absent until a
passport exists), injected through the same mount-point + rendered-event
seam the achievements plugin uses (now documented in docs/plugin-v3-ui.md).
The home page's plugin-count stat tile becomes a career trading card
(badges, hours, the closest stamp ask, foil shine) with the old stat as the
built-in fallback when career has no state. Earned passports gain **Save
card / Copy card** — a natively-drawn PNG passport card, downloadable or
copied straight to the clipboard for pasting outside the app (shared
`blob-io` helpers replace the download idiom previously duplicated in
settings-io and diagnostics-export).
- **Gigs (backend)** — career mode gains its verb: `POST
/api/plugins/career/gigs/propose` builds a playable setlist for an
instrument+genre (your qualifying songs plus a couple of stakes songs near
the bar; a young passport fills from unplayed genre songs — the first gig
is how stubs start; re-roll by calling again), naming the room your stars
can book. `POST /gigs` logs a **completed** set — per-song accuracies read
from the set's own freshly-recorded stats, an encore flag at the
data-driven bar (avg ≥ 75%) — into the career state; abandoned sets never
log (no fail state: the gig you finished is the gig you played). Passports
carry their gig log; instruments their gig count.
### Changed
- **The full mix is a stem** (#933) — core no longer depends on `original_audio:`, a
top-level manifest key this repo invented (#583) that the feedpak spec never had.
The format already carried the pre-separation mixdown as a stem; feedpak 1.15.0
(feedpak-spec#53) reserves the id **`full`** for it, so that is where core reads it
from now.
`full` is a mixdown, not a layer — it already contains every instrument — so
`load_song()` lifts it OUT of `LoadedSloppak.stems` onto `LoadedSloppak.full_mix`.
Nothing that sums stems or renders one fader per stem can see it, which is what
makes retaining it safe; leaving it in the list would double the whole song and
leave "guitar" audible with the guitar fader muted. That trap is exactly why the
packer invented the key instead of putting the mixdown where the format says it
goes — the bug was in the reader, and this fixes the reader.
Consequences worth knowing:
- The highway WS `song_info` frame gains `full_mix_url` / `has_full_mix`.
`original_audio_url` / `has_original_audio` remain as **deprecated aliases**
(same values) for one release so a client built against the old frame keeps
working; they go with the fallback below (#945).
- `stems` on `song_info`, and `stem_ids` / `stem_count` in the library index, now
describe *instrument* stems only — a separated pack that retains its mixdown no
longer advertises a bogus "full" stem chip or an inflated stem count.
- Audio fingerprinting (`lib/enrichment.py`) now resolves the mixdown the same
way, which **widens** its coverage: it previously returned `None` for any pack
without the invented key, so fingerprinting silently did nothing for the
overwhelming majority of packs.
- Core still **reads** `original_audio:` as a deprecated fallback, because every
pack written before the spec caught up carries it and would otherwise lose its
pristine mix. `tools/migrate_full_mix_stem.py` rewrites those packs into the
spec shape (moves `original/full.ogg` → `stems/full.ogg`, adds the `full` stem
at `default: off`, drops the key); the fallback and the aliases are removed once
they are migrated (#945).
### Added
- **Genres fall back to MusicBrainz enrichment** — the effective genre now
resolves override → pack genre → the enrichment match's primary genre
(matched or user-pinned manual rows only). Converted packs rarely carry a `genres` manifest key,
which starved the library genre facet and career passports on real
libraries; with the fallback, every enriched song's genre is browsable and
passport-able immediately, and coverage grows as enrichment runs.
- **Badge ceremony in the venue** — earning a genre badge now stages a moment:
the crowd layer erupts (new public `v3VenueCrowd.celebrate()` — instant
ecstatic loop bypassing the stability/dwell hysteresis, plus a cheer stinger;
+20
View File
@@ -189,3 +189,23 @@ out of the capability graph.
- [ ] `#player` overlays keep `z-index` ≤ the chrome layers (transport/HUD 20,
rail 30, popovers 40).
- [ ] Verify at `/` — it and `/v3` serve the same (and only) v3 shell.
## Injecting into core shells (profile, dashboard)
Core screens that accept plugin sections render **mount points** — usually
empty, sometimes holding core's own **fallback content** (the Dashboard's
career slot ships the plugin-count stat) — and announce each (re)build with a
DOM event, because their `innerHTML` swap wipes anything previously injected.
A plugin listens for the event and **replaces the mount's content** (never
append — a fallback may be present) by id — the same seam every time:
| Shell | Event | Mounts |
| --- | --- | --- |
| Profile | `v3:profile-rendered` | `#v3-profile-passports-mount` (career wall), `#v3-profile-feats-slot`, `#v3-profile-achievements-mount` |
| Dashboard | `v3:dashboard-rendered` | `#v3-dash-career-slot` (career card; core's plugin-count stat is the fallback content a plugin may replace) |
| Settings | `v3:settings-rendered` | per-plugin `settings.html` panels |
Rules: inject on every event (the mount is fresh), keep the section
**absent-not-empty** (no state → leave the mount alone / empty), and guard
re-wired listeners with a `dataset` flag when your own refresh path can run
against an unwiped mount.
+3 -1
View File
@@ -61,6 +61,8 @@ extractions and twenty-two `routers/` modules, plus lib/library_registry.py for
and is a monolith in its own right, to be split per-table once the router train
lands) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
(2,974) · `plugins/highway_3d/screen.js` (15,656) · `plugins/keys_highway_3d/screen.js`
(3,780) · `plugins/drum_highway_3d/screen.js` (3,597) — and every monolith with a PR
(3,780) · `plugins/drum_highway_3d/screen.js` (3,597) · `plugins/career/screen.js`
(1,530 — career v3 gigs + gold pushed it over; split plan: carve the gig block into a
`scriptType: module` file when career work next touches it) — and every monolith with a PR
train in the refactor plan. Test files (e.g. `tests/test_plugins.py`) are out of scope
by policy — the norm governs source files.
+15 -9
View File
@@ -34,17 +34,23 @@
exceptions:
- key: original_audio
issue: https://github.com/got-feedback/feedback/issues/933
issue: https://github.com/got-feedback/feedback/issues/945
reason: >-
Added by #583 (the full mix played while every stem fader sits at unity,
since demucs recombination is lossy). Core, lib/enrichment.py, and the
stems plugin all depend on it, but it never went through a FEP and the
since demucs recombination is lossy). It never went through a FEP and the
spec does not define it — the drift this gate exists to prevent.
The resolution is REMOVAL, not a FEP: the spec already carries the mixdown
as a stem ({id: full, file: stems/full.ogg}), so this key added a second,
redundant location for audio to a format that already had one. See #933.
#933 fixed the drift: feedpak 1.15.0 RESERVES the stem id `full` for the
complete mixdown (feedpak-spec#53), and core now reads the full mix from
that stem. Nothing depends on this key any more — not the loader, not
lib/enrichment.py, not the stems plugin, and the packer no longer writes it.
Grandfathered so the gate can land green and start blocking the *next*
instance immediately, rather than blocking on #933. This entry goes away
when core no longer reads or writes the key.
What remains is a READ-ONLY deprecated fallback in lib/sloppak.py
(_legacy_full_mix), kept for one release because every pack produced before
the spec caught up carries `original_audio: original/full.ogg` and would
otherwise silently lose its pristine mix. tools/migrate_full_mix_stem.py
rewrites those packs into the spec shape.
This entry disappears with that fallback — tracked by #945, which cannot be
forgotten: the gate fails if the entry goes stale, and deleting the read is
what makes it stale.
+20 -5
View File
@@ -368,10 +368,12 @@ def _acoustid_gate() -> "JSONResponse | None":
def _song_audio_file(filename: str) -> "str | None":
"""Resolve a LIBRARY song (by filename/id) to a local master-audio file for
fingerprinting: the full-mix `original_audio` extracted from a sloppak, or a
loose folder's audio. None when the song can't be found or ships no full-mix
audio (some packs carry only stems). Mirrors serve_sloppak_file's containment
guards so a crafted filename can't read outside DLC_DIR / the pack."""
fingerprinting: a sloppak's complete mixdown, or a loose folder's audio. None
when the song can't be found or carries no mixdown (a pack that kept only its
separated stems — an acoustic fingerprint of one re-summed from them would not
match the recording, so we decline rather than submit a lossy reconstruction).
Mirrors serve_sloppak_file's containment guards so a crafted filename can't
read outside DLC_DIR / the pack."""
dlc = _get_dlc_dir()
if not dlc:
return None
@@ -383,7 +385,20 @@ def _song_audio_file(filename: str) -> "str | None":
canon = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
return None
rel = (sloppak_mod.load_manifest(resolved) or {}).get("original_audio")
manifest = sloppak_mod.load_manifest(resolved) or {}
# The mixdown is the RESERVED `full` stem (spec §5.3). Unlike playback,
# fingerprinting wants it even when it is the pack's ONLY stem — a
# single-mix pack is exactly the master audio we want to fingerprint —
# so this asks find_full_mix() rather than partition_stems().
stems = manifest.get("stems") or []
full = sloppak_mod.find_full_mix(
[s for s in stems if isinstance(s, dict)]
)
rel = full.get("file") if full else None
# DEPRECATED fallback: packs written before the spec reserved `full` put
# the mixdown behind a top-level `original_audio:` key instead (#933).
if not isinstance(rel, str) or not rel.strip():
rel = manifest.get("original_audio")
if not isinstance(rel, str) or not rel.strip():
return None
src = sloppak_mod.get_cached_source_dir(canon)
+98 -15
View File
@@ -23,9 +23,18 @@ Engine selection
Two transcription paths share a common output:
* `transcribe_vocals_remote(path, server_url, ...)` — POST the vocal
stem to the `/align` endpoint on a feedBack-demucs-server (got-feedBack's
reference server already hosts WhisperX alongside Demucs at the same
URL).
stem to the `/transcribe` endpoint on a feedBack-demucs-server
(got-feedBack's reference server already hosts WhisperX alongside
Demucs at the same URL).
It used to POST to `/align`, which is *forced alignment* — "here are
the lyrics, tell me when each word is sung". Its `text` field is
required and we have no lyrics (transcribing them is the point), so
the server answered 422 from FastAPI's validation layer before its
handler ran, and remote transcription never worked for anyone
(feedBack-plugin-stem-splitter#17). `/transcribe` takes only audio.
Requires feedBack-demucs-server ≥ the revision adding that endpoint;
an older server answers 404 and the error says so.
* `transcribe_vocals_local(path, ...)` — load WhisperX in-process. Heavy
(~3 GB of model weights for `large-v2` + the wav2vec2 aligner) and
@@ -416,6 +425,38 @@ def transcribe_vocals_local(
# ── Remote transcription ────────────────────────────────────────────────────
_MAX_ERR_BODY = 4000
def _err_body(resp) -> str:
"""The server's error body, whole if it plausibly is one, and marked when it isn't.
This was capped at 300 chars, which is enough for "Internal Server Error" and not much else.
The bodies carrying the most diagnosis are the long ones — a FastAPI validation body naming
the field it rejected, a 500 whose traceback answers on its LAST line — and those are exactly
the ones a 300-char cap decapitates. The cap survives so a server answering with a 2 MB HTML
error page can't dump a novel into a log line.
"""
# Strip FIRST, then measure: a body that is 300 chars of JSON and 3900 of trailing whitespace
# is not a long body, and truncating it would cut real content to make room for blanks.
text = (getattr(resp, "text", "") or "").strip()
if len(text) <= _MAX_ERR_BODY:
return text
# Keep the HEAD **and the TAIL**. Head-only truncation throws away the exception line — and
# on a traceback the exception line is the answer. This docstring said as much while the code
# did the opposite: it cut off precisely the part it exists to preserve, which is the same
# mistake, one level up, as the 300-char cap it replaced.
#
# The marker sits inside the bound, not past it: otherwise _MAX_ERR_BODY is a suggestion, and
# the callers who trust it (a log line, a job record persisted to disk) are the ones surprised.
marker = f"\n… [truncated, {len(text)} chars total] …\n"
budget = max(0, _MAX_ERR_BODY - len(marker))
head = budget * 2 // 3 # context: what was being attempted
tail = budget - head # verdict: what actually went wrong
return text[:head].rstrip() + marker + text[len(text) - tail:].lstrip()
def transcribe_vocals_remote(
vocals_path: Path,
server_url: str,
@@ -426,7 +467,17 @@ def transcribe_vocals_remote(
min_word_score: float = 0.35,
progress_cb: ProgressCB = None,
) -> list[dict]:
"""POST the vocal stem to `{server_url}/align` and parse the response.
"""POST the vocal stem to `{server_url}/transcribe` and parse the response.
NOT `/align` — that endpoint is forced alignment ("here are the lyrics,
tell me when each word is sung") and its `text` field is required. We
have no lyrics; producing them is the point. Posting there returned a
422 from FastAPI's validation layer before the server's handler ran, so
remote transcription never worked at all
(feedBack-plugin-stem-splitter#17).
Requires a feedBack-demucs-server carrying `/transcribe`; an older one
answers 404 and the raised error says so.
Expects the server to respond with a JSON object carrying a `words` (or
`segments`) field in WhisperX's native shape; `_whisperx_to_sloppak`
@@ -454,21 +505,53 @@ def transcribe_vocals_remote(
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
params: dict[str, str] = {}
# POST to /transcribe, not /align.
#
# /align is FORCED ALIGNMENT: "here are the lyrics, tell me when each word is sung". Its
# `text` field is required, and we have no lyrics — transcription is the whole point. So the
# server rejected every request with a 422 in FastAPI's validation layer, before its handler
# ever ran, and remote transcription has never worked for anyone. /transcribe answers the
# question we are actually asking and takes only the audio.
# (feedBack-plugin-stem-splitter#17; endpoint added in feedBack-demucs-server#14.)
#
# `language` goes in the FORM BODY, not the query string: the server reads it with
# Form(""), and a query param would be silently ignored — so an explicit language hint would
# do nothing and Whisper's auto-detection would quietly decide instead, which is exactly the
# kind of "it works but it's wrong" that hides for months.
form: dict[str, str] = {}
if language:
params["language"] = language
form["language"] = language
with open(vocals_path, "rb") as f:
resp = requests.post(
f"{server_url}/align",
files={"file": (vocals_path.name, f, "audio/ogg")},
params=params,
headers=headers or None,
timeout=timeout,
# Everything that can go wrong out here comes back as RuntimeError, which is what the
# docstring promises and what the caller catches. A DNS failure, a timeout, a reset
# connection or an unreadable stem file would otherwise surface as requests.RequestException
# or OSError and escape the one handler written to log-and-continue — turning "this song's
# lyrics failed" into "the whole batch died".
try:
with open(vocals_path, "rb") as f:
resp = requests.post(
f"{server_url}/transcribe",
files={"file": (vocals_path.name, f, "audio/ogg")},
data=form or None,
headers=headers or None,
timeout=timeout,
)
except requests.RequestException as e:
raise RuntimeError(f"could not reach the WhisperX server at {server_url}: {e}") from e
except OSError as e:
raise RuntimeError(f"could not read the vocal stem {vocals_path.name}: {e}") from e
if resp.status_code == 404:
# The endpoint isn't there. Say what that means, because "404" on its own sends someone
# hunting for a typo in their URL when the real answer is that their server predates the
# feature. (feedBack-demucs-server#14 added /transcribe.)
raise RuntimeError(
f"the WhisperX server at {server_url} has no /transcribe endpoint (404) — it "
f"predates remote transcription support. Update the server, or use 'Check for "
f"update' if it is the plugin-managed one."
)
if resp.status_code != 200:
raise RuntimeError(f"WhisperX server error ({resp.status_code}): {resp.text[:300]}")
raise RuntimeError(f"WhisperX server error ({resp.status_code}): {_err_body(resp)}")
data = resp.json()
+40 -9
View File
@@ -1085,26 +1085,57 @@ class MetadataDB:
vals["artist"], vals["title"] = self._romaji_display(filename, vals["artist"], vals["title"])
return vals
# Effective genre = a per-song genre OVERRIDE (Fix-metadata popup) else the
# scanned pack genre. Applied at FILTER/FACET time (like the P4 artist alias)
# so a corrected genre is browsable — the correlated subquery is used ONLY
# when genre overrides actually exist; the common case stays on the plain
# indexed `genre` column. Genre stays a library-only overlay (it isn't a
# write-to-file field), so it never touches the pack.
_EFFECTIVE_GENRE_SQL = (
# Effective genre precedence: per-song OVERRIDE (Fix-metadata popup)
# scanned pack genre → MusicBrainz enrichment primary genre (matched/manual rows
# only — a 'review'/'failed' candidate's genres could belong to the wrong
# recording). Applied at FILTER/FACET time (like the P4 artist alias) so a
# corrected or enriched genre is browsable. The vast majority of converted
# packs carry no `genres` manifest key, so without the enrichment leg the
# genre facet (and career passports) starve on real libraries. The
# correlated subqueries are used ONLY when overrides/enrichment genres
# actually exist; the common case stays on the plain indexed `genre`
# column. Genre stays a library-only overlay (it isn't a write-to-file
# field), so it never touches the pack.
_EFFECTIVE_GENRE_OVERRIDE_SQL = (
"COALESCE((SELECT o.value FROM song_field_override o "
"WHERE o.filename = songs.filename AND o.field = 'genre' "
"AND o.value IS NOT NULL AND o.value != ''), genre)"
)
_EFFECTIVE_GENRE_SQL = (
"COALESCE((SELECT o.value FROM song_field_override o "
"WHERE o.filename = songs.filename AND o.field = 'genre' "
"AND o.value IS NOT NULL AND o.value != ''), "
"NULLIF(genre, ''), "
"(SELECT json_extract(e.genres, '$[0]') FROM song_enrichment e "
"WHERE e.filename = songs.filename AND e.match_state IN ('matched', 'manual') "
"AND e.genres IS NOT NULL AND e.genres NOT IN ('', '[]')), "
"'')"
)
def _has_genre_overrides(self) -> bool:
return self.conn.execute(
"SELECT 1 FROM song_field_override WHERE field = 'genre' "
"AND value IS NOT NULL AND value != '' LIMIT 1").fetchone() is not None
def _has_enrichment_genres(self) -> bool:
try:
return self.conn.execute(
"SELECT 1 FROM song_enrichment WHERE match_state IN ('matched', 'manual') "
"AND genres IS NOT NULL AND genres NOT IN ('', '[]') "
"LIMIT 1").fetchone() is not None
except sqlite3.OperationalError:
return False # stand-ins / DBs without the enrichment table
def _effective_genre_expr(self) -> str:
"""`genre` normally; the override-aware COALESCE only when overrides exist."""
return self._EFFECTIVE_GENRE_SQL if self._has_genre_overrides() else "genre"
"""`genre` normally; the enrichment-aware COALESCE only when trusted
enrichment genres exist (which also proves the table exists — a
stand-in DB without song_enrichment must never receive SQL that
references it); the override-only form when just overrides exist."""
if self._has_enrichment_genres():
return self._EFFECTIVE_GENRE_SQL
if self._has_genre_overrides():
return self._EFFECTIVE_GENRE_OVERRIDE_SQL
return "genre"
def set_song_tags(self, filename: str, tags) -> list:
"""Replace ALL of a song's tags with the given set (each normalized;
+43 -19
View File
@@ -321,11 +321,16 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
audio_url = None
audio_error: str | None = None # Surfaced in song_info when audio_url is None
stems_payload: list[dict] = []
# URL of the single full-mix audio (sloppak `original_audio:`), when the
# pack ships one. The stems plugin uses this to play the untouched mix
# while every stem slider is at unity; None otherwise (separate stems
# only, loose folder, or archive).
original_audio_url: str | None = None
# URL of the pack's complete mixdown — the RESERVED `full` stem (spec
# §5.3), which sloppak.load_song() lifts out of `stems` because it is a
# mixdown, not a layer. The stems plugin plays it while every stem slider
# is at unity (separation is lossy, so it beats re-summing the stems) and
# crosses to the separated stems as soon as one is attenuated.
#
# None when the pack has no mixdown to offer separately from its stems:
# a single-mix pack (its one stem IS the mixdown), a loose folder, or an
# archive.
full_mix_url: str | None = None
if is_loose:
# Loose folder filenames are relative paths (artist/album/song).
# Hash the *canonical* dlc-relative path (so two URL spellings
@@ -365,21 +370,25 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
url = f"/api/sloppak/{q_fn}/file/{quote(s['file'])}"
stems_payload.append({"id": s["id"], "url": url, "default": s["default"]})
# Full-mix URL (served by the same /api/sloppak/.../file/ endpoint).
if loaded_slop is not None and loaded_slop.original_audio:
original_audio_url = (
f"/api/sloppak/{q_fn}/file/{quote(loaded_slop.original_audio)}"
if loaded_slop is not None and loaded_slop.full_mix:
full_mix_url = (
f"/api/sloppak/{q_fn}/file/{quote(loaded_slop.full_mix)}"
)
if stems_payload:
# Stems present: keep the core <audio> pointed at stem[0]. This
# URL is only ever heard in the degraded path (stems plugin
# refuses takeover / decode fails); the full-mix↔stems switch is
# driven client-side by `original_audio_url`, not `audio_url`.
# driven client-side by `full_mix_url`, not `audio_url`.
audio_url = stems_payload[0]["url"]
elif original_audio_url:
elif full_mix_url:
# Stem-less full-mix pack: nothing to separate, so play the full
# mix natively through the core <audio>. The stems plugin's
# onSongReady returns early on an empty stems list (no graph).
audio_url = original_audio_url
# Reachable only via the deprecated `original_audio:` key, whose
# packs put the mixdown outside `stems` — a pack that carries its
# mixdown as the `full` stem has it IN `stems`, so it lands in the
# branch above with stems_payload == [full].
audio_url = full_mix_url
else:
audio_error = "This sloppak has no playable stems."
else:
@@ -521,16 +530,31 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
# for the credits overlay, so minigames / synthetic highway uses
# (no manifest) never trigger it.
"authors": _sanitize_authors(loaded_slop.manifest) if (is_slop and loaded_slop is not None) else [],
# Instrument stems ONLY. The pack's complete mixdown (the RESERVED
# `full` stem, spec §5.3) is deliberately NOT in this list: consumers
# sum `stems` into one mix and render one fader per entry, and the
# mixdown is neither a layer nor an instrument — summing it would
# double the whole song. It is surfaced separately, below.
"stems": stems_payload,
# Full-mix audio (sloppak `original_audio:`) served alongside the
# separate `stems`. The stems plugin plays this single file while
# every stem slider is at unity and switches to the separate stems
# the moment one drops below 100%. None when the pack ships stems
# only. `has_*` flags mirror the has_drum_tab/has_keys convention so
# a client can branch without re-deriving from the URLs.
"original_audio_url": original_audio_url,
"has_original_audio": bool(original_audio_url),
# The complete mixdown, served by the same /api/sloppak/.../file/
# endpoint as the stems. The stems plugin plays this single file
# while every stem slider is at unity and crosses to the separated
# stems the moment one drops below 100% — separation is lossy, so the
# mixdown is strictly better audio when nothing is muted. None when
# the pack has no mixdown apart from its stems. The `has_*` flags
# mirror the has_drum_tab/has_keys convention so a client can branch
# without re-deriving from the URLs.
"full_mix_url": full_mix_url,
"has_full_mix": bool(full_mix_url),
"has_stems": bool(stems_payload),
# DEPRECATED aliases of the two keys above, kept so a client built
# against the old frame keeps working across one release. They were
# named after `original_audio:` — a manifest key this repo invented
# and the feedpak spec never had (#933). The key is gone; the mixdown
# is a stem. Remove these once the shipped stems plugin reads
# `full_mix_url` (#945).
"original_audio_url": full_mix_url,
"has_original_audio": bool(full_mix_url),
# Surface a drum_tab presence flag so the visualization picker
# can auto-activate the drums plugin even when the chosen
# arrangement isn't named "Drums" (drum_tab.json lives next
+409 -61
View File
@@ -15,6 +15,7 @@ from __future__ import annotations
import logging
import math
import os
import shutil
import threading
import zipfile
@@ -34,6 +35,21 @@ FEEDPAK_EXT = ".feedpak"
SLOPPAK_EXT = ".sloppak"
SONG_EXTS = (FEEDPAK_EXT, SLOPPAK_EXT) # accepted on read/discovery
# ── The full mix ──────────────────────────────────────────────────────────────
#
# Spec §5.3 RESERVES the stem id `full` for the song's complete mixdown: the
# whole song in one file, as heard before source separation. It is a stem — it
# lives in `stems` like every other audio file in a pack — but it is a *mixdown,
# not a layer*. A reader that sums stems must never include it in the sum: it
# already contains every instrument, so summing it doubles the whole song and
# muting `guitar` still leaves guitar audible inside it.
#
# Keeping it matters because separation is lossy: re-summing guitar+bass+drums+
# vocals does NOT reproduce the file they came from. The mixdown is the only
# faithful rendering of the song a pack can carry, so we play it whenever every
# stem sits at unity and nothing is muted.
FULL_MIX_STEM_ID = "full"
import yaml
from jsonc import load_json
@@ -51,6 +67,97 @@ import drums as drums_mod
import notation as notation_mod
def find_full_mix(stems: list[dict]) -> dict | None:
"""The RESERVED `full` stem (spec §5.3) — the pack's complete mixdown — or None.
Answers "what is this pack's master audio", which is what fingerprinting
wants. For playback use partition_stems() instead: a pack whose *only* stem
is `full` has no mixdown to play *separately from* its stems, and this
function still returns it.
"""
return next(
(s for s in stems if str(s.get("id", "")) == FULL_MIX_STEM_ID), None
)
def partition_stems(stems: list[dict]) -> tuple[dict | None, list[dict]]:
"""Split stem descriptors into (mixdown, instrument_stems) for PLAYBACK.
The mixdown is lifted OUT of the stem list because every consumer of `stems`
treats that list as layers to sum or to show as mixer channels, and `full` is
neither (spec §5.3). Leaving it in is precisely the bug that made the packer
invent `original_audio` in the first place: a listed full mix plays on top of
the stems.
A pack whose only stem is `full` is a single-mix pack, not a separated one:
there are no instruments to be pristine *against*, so `full` stays the sole
playable stem and no mixdown is surfaced. That keeps the freshly-converted
single-stem pack — much the most common shape — behaving exactly as before.
EVERY entry with the reserved id is removed, not just the one we surface. A
malformed pack that lists `full` twice would otherwise leave a copy of the
whole song behind in the stem list, to be summed with the instruments — the
precise failure this function exists to prevent, reintroduced by a duplicate.
"""
if len(stems) < 2:
return None, stems
full = find_full_mix(stems)
if full is None:
return None, stems
return full, [s for s in stems if str(s.get("id", "")) != FULL_MIX_STEM_ID]
def _legacy_full_mix(manifest: dict, source_dir: Path) -> str | None:
"""Full mix from the DEPRECATED `original_audio:` manifest key, or None.
Before feedpak 1.15.0 reserved `full`, §5.3 said the mixdown was "commonly
replaced" by the per-instrument stems on splitting — so it had nowhere to
live, and this repo invented a top-level key pointing at a parallel
`original/` directory (#583) to hold it. That key was never in the spec, and
#933 removed our dependence on it: the mixdown is a stem.
We still READ it, because every pack written before the spec caught up
carries `original_audio: original/full.ogg` and would otherwise lose its full
mix. We never write it. Delete this once those packs are migrated (#945);
`tools/migrate_full_mix_stem.py` is the migration.
NOTE the string literal below. tools/check_spec_conformance.py AST-scans for
`manifest.get("<literal>")` to prove every manifest key core reads is one the
spec declares. Hoisting "original_audio" into a named constant would hide
this read from that scan — the gate would conclude core no longer touches the
key, and the grandfather entry that documents this debt would go stale. The
literal is what keeps the deprecation honest and visible to CI. Leave it.
Same permissive, path-traversal-guarded posture as the optional side-files: a
missing / escaping / unreadable file leaves the pack without a full mix (the
player falls back to the separated stems) rather than aborting the load.
Returns the manifest-relative string, so callers build its URL exactly as
they build a stem's.
"""
rel_raw = manifest.get("original_audio")
if not isinstance(rel_raw, str) or not rel_raw.strip():
return None
rel = rel_raw.strip()
try:
target = (source_dir / rel).resolve()
target.relative_to(source_dir.resolve())
except ValueError:
log.warning("sloppak: original_audio path %r escapes source_dir — skipped", rel)
return None
except OSError as e:
log.warning("sloppak: original_audio path resolution failed (%s) — skipped", e)
return None
if not target.is_file():
return None
log.info(
"sloppak: pack uses the deprecated `original_audio:` key (%r) — the full mix "
"is a stem (id `full`, feedpak spec §5.3). Re-pack with "
"tools/migrate_full_mix_stem.py; support for this key will be removed.",
rel,
)
return rel
# ── Format detection ──────────────────────────────────────────────────────────
def is_sloppak(path: Path) -> bool:
@@ -81,6 +188,116 @@ _unpack_semaphore = threading.BoundedSemaphore(_UNPACK_MAX_CONCURRENCY)
_unpack_locks: dict[str, threading.Lock] = {}
_unpack_locks_guard = threading.Lock()
# Destinations with an unpack in flight right now. Eviction MUST skip these: two
# unpacks run concurrently, so one finishing could otherwise rmtree the other's
# half-written directory and leave that resolver caching an incomplete song.
_unpacking: set[Path] = set()
_unpacking_guard = threading.Lock()
# Cap the unpack cache. Stems are already-compressed audio, so an unpacked song
# is ~1.1x its zip — the cache is effectively a second, DECOMPRESSED copy of
# every song it holds, and it used to grow without any bound at all. A tester
# reached 60 GB from a 1800-song library: their whole library, unpacked, because
# one caller looped the library calling load_song(). Nothing ever deleted any of
# it — not even when the song itself was deleted.
#
# Default 4 GB ≈ 130 average songs of recency, which is far more than the "the
# song I'm playing, and the last few I played" that this cache actually exists
# to serve. Override with FEEDBACK_SLOPPAK_CACHE_MAX_MB (0 disables eviction).
def _unpack_cache_cap_bytes() -> int:
raw = os.environ.get("FEEDBACK_SLOPPAK_CACHE_MAX_MB", "").strip()
try:
mb = int(raw) if raw else 4096
except ValueError:
mb = 4096
return max(0, mb) * 1024 * 1024
def _dir_size(path: Path) -> int:
total = 0
for f in path.rglob("*"):
try:
if f.is_file():
total += f.stat().st_size
except OSError:
continue
return total
def _touch(path: Path) -> None:
"""Bump mtime so the LRU sweep below treats this song as recently used.
Reading files out of an unpacked dir doesn't change the DIRECTORY's mtime,
so without this the song you are actively playing looks as stale as one you
unpacked days ago — and a burst of unpacks could evict it mid-song.
"""
try:
os.utime(path, None)
except OSError:
pass
def _evict_unpack_cache(root: Path, keep: Path | None = None) -> None:
"""Bound the unpack cache: drop least-recently-used songs until under the cap.
`keep` is never evicted — it's the song the caller just resolved, i.e. almost
certainly the one about to be played.
Evicting a directory MUST also drop its `_source_cache` entry. Otherwise
get_cached_source_dir() keeps handing out a path that no longer exists and
the media route 404s on every stem instead of re-unpacking (it only falls
back to resolve_source_dir when the cache returns None).
"""
cap = _unpack_cache_cap_bytes()
if cap <= 0:
return
try:
entries = []
total = 0
for d in root.iterdir():
if not d.is_dir():
continue
try:
size = _dir_size(d)
mtime = d.stat().st_mtime
except OSError:
continue
entries.append((mtime, size, d))
total += size
if total <= cap:
return
keep_resolved = keep.resolve() if keep else None
entries.sort(key=lambda e: e[0]) # oldest first
for _mtime, size, d in entries:
if total <= cap:
break
try:
if keep_resolved and d.resolve() == keep_resolved:
continue
except OSError:
continue
# Check-and-delete under ONE hold of the guard. Releasing between the
# two would let a resolver mark this dest in-flight and start writing
# into it in the gap, and we'd rmtree a song mid-unpack. A resolver
# that blocks here simply proceeds afterwards — _unpack_zip recreates
# the directory anyway.
with _unpacking_guard:
if d in _unpacking:
continue # another thread is writing this
shutil.rmtree(d, ignore_errors=True)
if d.exists():
continue # couldn't remove — don't claim the bytes back
total -= size
with _source_lock:
for fn, (cached_dir, _m, _s) in list(_source_cache.items()):
if cached_dir == d:
_source_cache.pop(fn, None)
log.info("sloppak: evicted %s from the unpack cache (%.0f MB)",
d.name, size / 1e6)
except OSError:
log.warning("sloppak: unpack-cache eviction failed", exc_info=True)
def _unpack_lock_for(filename: str) -> threading.Lock:
"""Return a stable per-file lock so concurrent unpacks of the same sloppak
@@ -145,10 +362,17 @@ def resolve_source_dir(
re-unpacks if mtime/size changed, then returns that dir.
Caches the resolution so subsequent calls are ~free.
NOTE: this writes the WHOLE pack — every stem — to disk. Only call it for a
song you are about to play. To read a *part* of a song (an arrangement, the
lyrics, a tone blob), use read_member_bytes(): unpacking a pack to read a few
KB of JSON is ~45x write amplification, and doing it in a loop over the
library fills the disk with a decompressed copy of every song.
"""
path = dlc_root / filename
stat = path.stat()
mtime, size = stat.st_mtime, stat.st_size
guarded: Path | None = None # a dir WE unpacked, shielded from eviction
with _source_lock:
cached = _source_cache.get(filename)
@@ -159,42 +383,76 @@ def resolve_source_dir(
and cached_size == size
and cached_dir.exists()
):
# Mark it recently-used before returning — see _touch().
if cached_dir != path:
_touch(cached_dir)
return cached_dir
if path.is_dir():
resolved = path
else:
# Zip form — unpack to the cache. Serialize per-file (so concurrent
# callers don't rmtree + re-extract the same dest at once) and cap
# global unpack concurrency (so a burst can't saturate disk/CPU).
dest = unpack_cache_root / _safe_id(filename)
with _unpack_lock_for(filename):
# Re-check the cache inside the per-file lock — a prior holder may
# have just finished unpacking this exact (mtime, size).
with _source_lock:
cached = _source_cache.get(filename)
if (
cached
and cached[1] == mtime
and cached[2] == size
and cached[0].exists()
):
resolved = cached[0]
else:
with _unpack_semaphore:
_unpack_zip(path, dest)
resolved = dest
try:
if path.is_dir():
resolved = path
else:
# Zip form — unpack to the cache. Serialize per-file (so concurrent
# callers don't rmtree + re-extract the same dest at once) and cap
# global unpack concurrency (so a burst can't saturate disk/CPU).
dest = unpack_cache_root / _safe_id(filename)
with _unpack_lock_for(filename):
# Re-check the cache inside the per-file lock — a prior holder may
# have just finished unpacking this exact (mtime, size).
with _source_lock:
cached = _source_cache.get(filename)
if (
cached
and cached[1] == mtime
and cached[2] == size
and cached[0].exists()
):
resolved = cached[0]
else:
# Shield `dest` from eviction from the moment we start writing
# until it is safely in _source_cache. `keep` only shields it
# from OUR OWN sweep — a concurrent resolver sweeping with a
# different `keep` would delete it, and we would then cache and
# return a path that no longer exists. The `finally` below
# releases it on EVERY exit, including a failed unpack: leaving
# a dest marked in-flight would make it un-evictable forever.
with _unpacking_guard:
_unpacking.add(dest)
guarded = dest
with _unpack_semaphore:
_unpack_zip(path, dest)
resolved = dest
# The only moment this cache grows. Sweep here rather than on a
# timer so it can never drift far past the cap.
_evict_unpack_cache(unpack_cache_root, keep=dest)
with _source_lock:
_source_cache[filename] = (resolved, mtime, size)
return resolved
with _source_lock:
_source_cache[filename] = (resolved, mtime, size)
return resolved
finally:
if guarded is not None:
with _unpacking_guard:
_unpacking.discard(guarded)
def get_cached_source_dir(filename: str) -> Path | None:
"""Return the cached source dir for a sloppak if one is known."""
"""Return the cached source dir for a sloppak if one is known AND still there.
The existence check is load-bearing: callers (media.py) only fall back to
resolve_source_dir() when this returns None, so handing back a path that has
been evicted — or that the user deleted by hand to reclaim disk — would 404
every stem for the rest of the process instead of re-unpacking.
"""
with _source_lock:
cached = _source_cache.get(filename)
return cached[0] if cached else None
if not cached:
return None
src = cached[0]
if not src.is_dir():
_source_cache.pop(filename, None)
return None
_touch(src)
return src
# ── Manifest + song loading ───────────────────────────────────────────────────
@@ -233,6 +491,82 @@ def load_manifest(path: Path) -> dict:
return _read_manifest_from_zip(path)
_ZIP_ROOT = Path("/_root").resolve()
def _zip_member_key(name: str) -> str | None:
"""Canonical lookup key for a zip member name, or None if it escapes the root.
Collapses './', 'a/../b' and backslash separators — the same normalization
_unpack_zip()/safe_join() apply when extracting. Both the name the caller asks
for AND the names the archive actually stores must go through this, or a pack
that stores './arrangements/lead.json' unpacks fine but reads back as missing.
"""
safe = safe_join(_ZIP_ROOT, name or "")
# None → escapes the root; == root → a degenerate name like "." or "a/..".
if safe is None or safe == _ZIP_ROOT:
return None
return safe.relative_to(_ZIP_ROOT).as_posix()
def read_member_bytes(path: Path, rel: str) -> bytes | None:
"""Return the bytes of ONE file inside a sloppak, or None if it isn't there.
For a zipped sloppak this opens that single member instead of unpacking the
archive — the same trick read_cover_bytes() uses to keep the library grid
from exploding every pack just to show a cover.
Reach for this whenever you want a *part* of a song (an arrangement's JSON,
the lyrics, a tone blob) rather than a song you're about to play. The
alternative, load_song(), calls resolve_source_dir() and writes the WHOLE
pack — every stem — into the unpack cache. That is a ~45x write amplification
when all you wanted was a few KB of JSON, and looping the library on it
unpacks the entire library (got-feedBack/feedBack: a tester hit 60 GB that
way). Stems are already-compressed audio, so an unpacked song is ~1.1x its
zip: the cache becomes a second, decompressed copy of everything it touches.
"""
rel = (rel or "").strip()
if not rel:
return None
if path.is_dir():
target = safe_join(path.resolve(), rel)
if target is None or not target.is_file():
return None
try:
return target.read_bytes()
except OSError:
return None
# Zip form — read just that member, no unpack. Zip-slip is rejected before we
# open anything, and both sides of the comparison are normalized, so a
# non-canonical-but-valid name ('./arrangements/lead.json') resolves the same
# way it did when we unpacked first.
member = _zip_member_key(rel)
if member is None:
log.warning("sloppak: rejected unsafe member name %r in %r", rel, path)
return None
try:
with zipfile.ZipFile(str(path), "r") as zf:
# Match on the NORMALIZED stored name, and take the LAST match — the
# archive may store './x' or a backslash path (Windows tooling), and
# if it stores two names that normalize to the same file, _unpack_zip
# writes them in order so the last one wins. Reading the raw member by
# exact name would miss the first case and return the wrong bytes in
# the second. A pack has a handful of members; the scan is free.
info = None
for cand in zf.infolist():
if _zip_member_key(cand.filename) == member:
info = cand
if info is None or info.is_dir():
return None
with zf.open(info) as f:
return f.read()
except (zipfile.BadZipFile, OSError, RuntimeError) as e:
log.warning("sloppak: failed to read %r from %s: %s", rel, path.name, e)
return None
_COVER_MEDIA_TYPES = {
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp",
@@ -367,14 +701,21 @@ class LoadedSloppak:
# song.arrangements (not to manifest["arrangements"]) — skipped entries are
# absent so indexing by song.arrangements index is safe.
arrangement_ids: list[str | None] = field(default_factory=list)
# Manifest-relative path to the single full-mix audio file, taken from the
# manifest `original_audio:` key (e.g. "original/full.ogg"). This is the
# pre-separation mixdown that exists alongside the per-instrument `stems`.
# None when the key is absent, points outside source_dir, or the file is
# missing on disk. Served to the front-end via the highway WS as
# `original_audio_url`; the stems plugin uses it to play the untouched mix
# when every stem slider is at unity (and the separate stems otherwise).
original_audio: str | None = None
# Manifest-relative path to the pack's complete mixdown — the whole song in
# one file, as heard before source separation. This is the RESERVED `full`
# stem (spec §5.3), lifted out of `stems` above precisely because it is NOT
# an instrument layer: summing it with the per-instrument stems it was split
# into would double the entire song. See partition_stems().
#
# None when the pack has no mixdown to offer *separately* from its stems —
# which includes the common single-mix pack, whose only stem IS the mixdown
# (there is nothing to be pristine against, so it stays in `stems`).
#
# Served to the front-end via the highway WS as `full_mix_url`; the stems
# plugin plays it while every stem slider sits at unity and crosses to the
# separated stems the moment one drops below 100% — demucs recombination is
# lossy, so the mixdown is strictly the better audio when nothing is muted.
full_mix: str | None = None
def load_song(
@@ -766,6 +1107,13 @@ def load_song(
default_on = bool(default_val)
stems.append({"id": sid, "file": sfile, "default": default_on})
# The complete mixdown is a stem (spec §5.3), but it is not a *layer*: lift
# it out so that no consumer of `stems` — the mixer, the library's stem
# chips, the WS payload — sums it with, or lists it beside, the instruments
# it was separated into. `full_mix_stem` is None for a single-mix pack,
# whose only stem IS the mixdown and stays in the list.
full_mix_stem, stems = partition_stems(stems)
# Optional keys.json — song-level, instrument-independent key/scale track
# (manifest `keys:` key, spec §7.7). Permissive like the other side-files:
# missing / unreadable / malformed -> None, never fatal. Stored as a
@@ -828,28 +1176,22 @@ def load_song(
}
_fpv = manifest.get("feedpak_version")
# Optional full-mix audio — manifest `original_audio:` key. The single
# pre-separation mixdown that ships alongside the per-instrument stems.
# Same permissive, path-traversal-guarded posture as drum_tab above: a
# missing/escaping/absent file simply leaves the full mix unavailable (the
# player falls back to the separate stems) rather than aborting the load.
# We store the manifest-relative string so server.py can build its URL the
# same way it builds stem URLs (via the /api/sloppak/.../file/ endpoint).
original_audio_data: str | None = None
original_audio_rel = manifest.get("original_audio")
if isinstance(original_audio_rel, str) and original_audio_rel.strip():
rel = original_audio_rel.strip()
try:
oa_path = (source_dir / rel).resolve()
oa_path.relative_to(source_dir.resolve())
except ValueError:
log.warning("sloppak: original_audio path %r escapes source_dir — skipped", rel)
oa_path = None
except OSError as e:
log.warning("sloppak: original_audio path resolution failed (%s) — skipped", e)
oa_path = None
if oa_path is not None and oa_path.is_file():
original_audio_data = rel
# The pack's full mix. Normally the RESERVED `full` stem partitioned out
# above (spec §5.3) — no path work needed, it was validated with the other
# stems and its URL is built the same way. Only when the pack has no `full`
# stem do we fall back to the DEPRECATED `original_audio:` key, which is the
# shape every pack written before feedpak 1.15.0 uses.
if full_mix_stem is not None:
full_mix_data: str | None = full_mix_stem["file"]
elif find_full_mix(stems) is not None:
# Single-mix pack: its ONE stem is the mixdown, so there is no mixdown to
# offer *apart from* the stems. Never fall through to the legacy key here
# — a pack that both carries a `full` stem and names the old key would
# otherwise surface the mixdown twice (once as the stem the player is
# already playing, once as a "pristine" track to cross to).
full_mix_data = None
else:
full_mix_data = _legacy_full_mix(manifest, source_dir)
return LoadedSloppak(
song=song,
@@ -864,7 +1206,7 @@ def load_song(
keys=keys_data,
notation_by_id=notation_by_id_data,
arrangement_ids=arrangement_ids_acc,
original_audio=original_audio_data,
full_mix=full_mix_data,
)
@@ -909,7 +1251,7 @@ def extract_meta(path: Path) -> dict:
tuning_offsets = _tuning_for_meta(arr_list)
stems_list = manifest.get("stems", []) or []
stem_ids: list[str] = []
valid_stems: list[dict] = []
for s in stems_list:
if not isinstance(s, dict):
continue
@@ -923,7 +1265,13 @@ def extract_meta(path: Path) -> dict:
isinstance(sid, str) and sid
and isinstance(sfile, str) and sfile
):
stem_ids.append(sid)
valid_stems.append({"id": sid, "file": sfile})
# Partition exactly as load_song() does, for the same reason the library
# filter must not lie: `full` is the mixdown, not an instrument (spec §5.3).
# A separated pack that retains it would otherwise offer the user a "full"
# stem chip alongside guitar/bass/drums and count it as a seventh stem.
_full, instrument_stems = partition_stems(valid_stems)
stem_ids = [s["id"] for s in instrument_stems]
stem_count = len(stem_ids)
return {
+234 -4
View File
@@ -99,7 +99,8 @@
.pp-inst-plus { color: #6b7280; }
/* Leather covers — per-instrument hue, embossed with layered shadows and a
subtle grain gradient (no image assets). */
subtle grain gradient (no image assets). Keep the hex pairs in sync with
PP_LEATHER_HEX in screen.js (the canvas card draws the same leather). */
.pp-leather-guitar { background: linear-gradient(160deg, #5c2321, #401412); }
.pp-leather-bass { background: linear-gradient(160deg, #1f3252, #131f36); }
.pp-leather-keys { background: linear-gradient(160deg, #1e4034, #122a21); }
@@ -524,7 +525,18 @@
pointer-events: none;
}
/* Gold foil preview — honest "coming", never earnable-looking. */
/* Gold ink — a REAL gold badge (comb-verified improv). */
.pp-stamp-gold {
border-color: #b8860b;
color: #a97b1b;
box-shadow: inset 0 0 0 3px #f3e8c8, inset 0 0 0 4px #b8860b;
}
.pp-stamp-mini.pp-stamp-gold {
color: #f0c75e;
border-color: #f0c75e;
box-shadow: none;
}
/* Gold foil chip — rendered only alongside an earned gold stamp. */
.pp-gold-foil {
position: relative;
overflow: hidden;
@@ -534,8 +546,8 @@
margin-top: 0.9rem;
padding: 0.28rem 0.85rem;
border-radius: 999px;
border: 2px dashed #c8b273;
color: #a8946d;
border: 2px solid #d9a253;
color: #c89040;
font-size: 0.58rem;
font-weight: 700;
letter-spacing: 0.32em;
@@ -559,3 +571,221 @@
/* The hover glint is motion theatrics too — not just the JS tilt. */
.pp-tilt::after { display: none; }
}
/* Practice invitations — closest stamps + bring-these-up */
.pp-closest {
border: 1px solid rgba(75, 85, 99, 0.45);
border-radius: 0.6rem;
background: linear-gradient(165deg, rgba(45, 55, 72, 0.4), rgba(31, 41, 55, 0.4));
padding: 0.6rem 0.75rem;
display: flex;
flex-direction: column;
gap: 0.3rem;
}
.pp-closest-head {
font-size: 0.62rem;
letter-spacing: 0.22em;
text-transform: uppercase;
color: #9ca3af;
}
.pp-closest-row {
display: flex;
align-items: baseline;
gap: 0.75rem;
text-align: left;
font-size: 0.8rem;
padding: 0.15rem 0.25rem;
border-radius: 0.35rem;
}
.pp-closest-row:hover { background: rgba(55, 65, 81, 0.5); }
.pp-closest-genre { color: #e5e7eb; font-weight: 600; white-space: nowrap; }
.pp-closest-ask { color: #9ca3af; font-size: 0.72rem; }
.pp-closest-ask em { color: #cbd5e1; font-style: italic; }
.pp-nearest { margin-top: 0.6rem; border-top: 1px dashed rgba(138, 122, 94, 0.4); padding-top: 0.5rem; }
.pp-nearest-head {
font-size: 0.58rem;
letter-spacing: 0.22em;
text-transform: uppercase;
color: #8a7a5e;
margin-bottom: 0.25rem;
}
.pp-nearest-row { font-size: 0.7rem; color: #6d5d40; padding: 0.1rem 0; }
.pp-nearest-row em { color: #3f3428; }
/* ── Career surfaces outside the plugin: profile wall + home card ───────── */
.pp-wall { display: flex; flex-direction: column; gap: 0.6rem; }
.pp-wall-head {
display: flex;
align-items: baseline;
justify-content: space-between;
font-weight: 600;
color: #e5e7eb;
font-size: 0.9rem;
}
.pp-wall-meta { color: #9ca3af; font-size: 0.7rem; font-weight: 400; }
.pp-wall-shelf {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.5rem;
padding: 0.35rem 0;
border-bottom: 1px solid rgba(75, 85, 99, 0.25);
}
.pp-wall-inst {
font-size: 0.62rem;
letter-spacing: 0.18em;
text-transform: uppercase;
color: #6b7280;
min-width: 3.6rem;
}
.pp-wall-cover {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 0.1rem;
width: 4.2rem;
height: 5.6rem;
border-radius: 0.3rem 0.45rem 0.45rem 0.3rem;
box-shadow: inset 0 0 0 1px rgba(255, 255, 255, 0.07),
inset 0.25rem 0 0.4rem -0.25rem rgba(0, 0, 0, 0.8),
0 3px 8px rgba(0, 0, 0, 0.4);
padding: 0.3rem;
transition: transform 0.15s ease;
}
.pp-wall-cover:hover { transform: translateY(-3px); }
.pp-wall-cover span {
font-size: 0.5rem;
font-weight: 700;
letter-spacing: 0.1em;
color: rgba(240, 226, 195, 0.9);
overflow-wrap: anywhere;
text-align: center;
}
.pp-wall-cover em {
font-size: 0.42rem;
letter-spacing: 0.22em;
font-style: normal;
color: #d9a253;
}
.pp-wall-none { font-size: 0.7rem; color: #6b7280; font-style: italic; }
.pp-wall-link {
align-self: flex-end;
font-size: 0.72rem;
color: #22d3ee;
padding: 0.15rem 0.3rem;
}
.pp-wall-link:hover { text-decoration: underline; }
/* The home-page career card — a trading card among stat tiles. */
.pp-dash-card {
position: relative;
overflow: hidden;
width: 100%;
height: 100%;
display: flex;
flex-direction: column;
justify-content: center;
gap: 0.2rem;
text-align: left;
padding: 1rem;
border-radius: 0.5rem;
border: 1px solid rgba(217, 162, 83, 0.35);
background:
linear-gradient(135deg, rgba(92, 35, 33, 0.85), rgba(30, 27, 34, 0.92)),
linear-gradient(160deg, #2b1414, #17111c);
box-shadow: inset 0 0 0 1px rgba(255, 255, 255, 0.05), 0 4px 14px rgba(0, 0, 0, 0.35);
transition: transform 0.15s ease, box-shadow 0.15s ease;
}
.pp-dash-card:hover { transform: translateY(-2px); box-shadow: 0 8px 20px rgba(0, 0, 0, 0.5); }
.pp-dash-shine {
position: absolute;
inset: 0;
background: linear-gradient(105deg, transparent 42%, rgba(255, 223, 128, 0.18) 50%, transparent 58%);
transform: translateX(-130%);
pointer-events: none;
}
.pp-dash-card:hover .pp-dash-shine { animation: pp-foil 1.4s ease-out; }
.pp-dash-head {
font-size: 0.58rem;
letter-spacing: 0.3em;
text-transform: uppercase;
color: #d9a253;
}
.pp-dash-badges { color: #f3ead2; font-size: 1.05rem; }
.pp-dash-badges b { font-weight: 700; margin: 0 0.25rem 0 0.35rem; }
.pp-dash-meta { color: #b5a488; font-size: 0.72rem; }
.pp-dash-ask { color: #8d9aa8; font-size: 0.66rem; }
.pp-dash-ask em { color: #cbd5e1; }
.pp-card-actions { display: flex; gap: 0.5rem; margin-top: 0.9rem; }
@media (prefers-reduced-motion: reduce) {
.pp-dash-card:hover .pp-dash-shine { animation: none; }
.pp-wall-cover, .pp-dash-card { transition: none; }
}
/* ── Gigs: poster, runner strip, summary, log ───────────────────────────── */
.pp-poster {
position: relative;
width: min(92vw, 420px);
padding: 2rem 1.6rem 1.4rem;
border-radius: 0.5rem;
background: linear-gradient(180deg, #141019, #241318);
border: 2px solid rgba(217, 162, 83, 0.45);
box-shadow: 0 10px 32px rgba(0, 0, 0, 0.6);
display: flex;
flex-direction: column;
align-items: center;
gap: 0.35rem;
text-align: center;
}
.pp-poster-venue { color: rgba(240, 226, 195, 0.7); font-size: 0.95rem; letter-spacing: 0.08em; }
.pp-poster-presents { color: rgba(240, 226, 195, 0.4); font-size: 0.58rem; letter-spacing: 0.4em; text-transform: uppercase; }
.pp-poster-title {
color: #d9a253;
font-size: 1.7rem;
font-weight: 800;
letter-spacing: 0.1em;
line-height: 1.15;
overflow-wrap: anywhere;
}
.pp-poster-inst { color: rgba(240, 226, 195, 0.5); font-size: 0.68rem; letter-spacing: 0.2em; text-transform: uppercase; }
.pp-poster-bill { margin: 0.9rem 0 0.5rem; display: flex; flex-direction: column; gap: 0.35rem; width: 100%; }
.pp-poster-line { color: rgba(240, 226, 195, 0.85); font-size: 0.85rem; }
.pp-poster-line span { color: rgba(217, 162, 83, 0.7); margin-right: 0.35rem; }
.pp-poster-line em { color: rgba(240, 226, 195, 0.5); font-style: italic; font-size: 0.72rem; }
.pp-poster-line b { color: #f3d179; margin-left: 0.3rem; }
.pp-poster-actions { display: flex; flex-wrap: wrap; gap: 0.5rem; justify-content: center; margin-top: 0.6rem; }
.pp-poster-summary { cursor: default; }
.pp-gig-strip {
position: fixed;
top: 0.5rem;
left: 50%;
transform: translateX(-50%);
z-index: 35; /* above the rail (30), under popovers (40) — the chrome invariant */
background: rgba(10, 8, 14, 0.85);
border: 1px solid rgba(217, 162, 83, 0.4);
border-radius: 999px;
color: rgba(240, 226, 195, 0.85);
font-size: 0.72rem;
padding: 0.3rem 0.9rem;
pointer-events: none;
backdrop-filter: blur(2px);
}
.pp-gig-strip b { color: #d9a253; letter-spacing: 0.2em; }
.pp-gig-strip em { color: #f3ead2; font-style: italic; }
.pp-giglog { margin-top: 0.6rem; border-top: 1px dashed rgba(138, 122, 94, 0.4); padding-top: 0.5rem; }
.pp-giglog-head {
font-size: 0.58rem;
letter-spacing: 0.22em;
text-transform: uppercase;
color: #8a7a5e;
margin-bottom: 0.25rem;
}
.pp-giglog-row { font-size: 0.7rem; color: #6d5d40; padding: 0.1rem 0; }
.pp-giglog-row b { color: #9a5b16; letter-spacing: 0.06em; }
+13
View File
@@ -3,6 +3,19 @@
"songs": 5,
"min_stars": 2
},
"gig": {
"min_songs": 3,
"max_songs": 5,
"stakes_songs": 2,
"encore_accuracy": 0.75
},
"families": [
{ "key": "metal", "match": ["metal", "djent", "grindcore", "thrash", "doom"] },
{ "key": "blues", "match": ["blues"] },
{ "key": "jazz", "match": ["jazz", "bebop", "swing", "bossa"] },
{ "key": "funk", "match": ["funk", "disco"] },
{ "key": "rock", "match": ["rock", "punk", "grunge", "shoegaze"] }
],
"genres": {
"blues": { "virtuoso_nodes": { "guitar": ["blues_shuffle"] } },
"rock": { "virtuoso_nodes": { "guitar": ["rock_power_backbeat"] } },
+271 -11
View File
@@ -26,11 +26,14 @@ Endpoints (all under /api/plugins/career/):
POST /passports/commit commit to an instrument (the wax seal, Stage 0)
POST /passports/open open a genre passport for an instrument
POST /drill-state relayed virtuoso.progress snapshot (drill intake)
POST /gigs/propose build a playable setlist for a genre gig
POST /gigs log a COMPLETED gig (abandoned sets never log)
"""
import hashlib
import json
import logging
import random
import re
import shutil
import tempfile
@@ -114,10 +117,9 @@ def _stars():
detail = []
for filename, acc, title, artist in rows:
acc = acc or 0.0
stars = sum(1 for t in thresholds if acc >= t)
stars, next_at = _star_progress(acc, thresholds)
if stars:
per_song[filename] = stars
next_at = next((t for t in thresholds if acc < t), None)
detail.append({
"filename": filename,
"title": title or filename,
@@ -249,10 +251,18 @@ def _played_by_instrument_genre():
for stub in stubs.values():
acc = stub["best_accuracy"]
stub["best_accuracy"] = round(acc, 4)
stub["stars"] = sum(1 for t in thresholds if acc >= t)
stub["stars"], stub["next_star_at"] = _star_progress(acc, thresholds)
return out, seconds
def _star_progress(acc, thresholds):
"""(stars, next_star_at) — the one place the ascending-thresholds
assumption lives; _stars() and the passport stubs both use it."""
stars = sum(1 for t in thresholds if acc >= t)
next_at = next((t for t in thresholds if acc < t), None)
return stars, next_at
def _library_genres():
"""Distinct effective genres across the live library (the brochure rack)."""
db = _state["meta_db"]
@@ -276,12 +286,33 @@ def _library_genres():
key=lambda r: (-r["songs_in_library"], r["genre_key"]))
def _genre_family(gkey):
"""First family whose keyword appears in the genre key (substring — MB's
vocabulary is open: 'metalcore' must hit the 'metal' family without an
exact alias). List order decides ambiguity: families are checked top to
bottom, so 'blues rock' lands on whichever of blues/rock is listed first."""
for fam in _state["passports_content"].get("families") or []:
if not isinstance(fam, dict):
continue
for kw in fam.get("match") or []:
if isinstance(kw, str) and kw and kw in gkey:
return fam.get("key")
return None
def _badge_requirement(gkey, instrument="guitar"):
cfg = _state["passports_content"]
req = dict(cfg.get("badge_requirement") or {})
req.setdefault("songs", 5)
req.setdefault("min_stars", 2)
override = (cfg.get("genres") or {}).get(gkey)
# Exact per-genre override wins; otherwise the genre inherits its FAMILY's
# requirement — so 'death metal' / 'metalcore' passports carry the metal
# drill without curating every MB sub-genre by hand.
genres_cfg = cfg.get("genres") or {}
override = genres_cfg.get(gkey)
if not isinstance(override, dict):
family = _genre_family(gkey)
override = genres_cfg.get(family) if family else None
if isinstance(override, dict):
req.update(override)
# virtuoso_nodes: {instrument: [node_ids]} — a passport only carries its
@@ -299,10 +330,11 @@ def _badge_requirement(gkey, instrument="guitar"):
def _drill_by_node():
doc = _load_json(_drill_file(), {})
if not isinstance(doc, dict):
return None, {}
return None, {}, {}
snapshot = doc.get("snapshot") if isinstance(doc.get("snapshot"), dict) else {}
by_node = snapshot.get("byNode") if isinstance(snapshot.get("byNode"), dict) else {}
return doc.get("received_at"), by_node
gold = snapshot.get("goldImprov") if isinstance(snapshot.get("goldImprov"), dict) else {}
return doc.get("received_at"), by_node, gold
def _merge_drill_nodes(old, new):
@@ -336,6 +368,16 @@ def _merge_drill_nodes(old, new):
return out
def _merge_gold(old, new):
"""Gained-only merge of goldImprov artifacts: a minted style never
un-mints via a stale relay; the FIRST artifact per style is kept."""
out = dict(old)
for style_id, art in (new or {}).items():
if isinstance(art, dict) and style_id not in out:
out[style_id] = art
return out
def _node_cleared(by_node, node_id):
"""A drill counts as cleared on real completion evidence: mastered, any
depth rung flipped true, or a key cleared (a top-tier clean pass in one
@@ -355,8 +397,9 @@ def _passports_view():
cfg = _state["passports_content"]
graded = set(cfg.get("graded_instruments") or [])
st = _career_state()
all_gigs = st.get("gigs") if isinstance(st.get("gigs"), list) else []
played, played_seconds = _played_by_instrument_genre()
received_at, by_node = _drill_by_node()
received_at, by_node, gold_improv = _drill_by_node()
instruments = {}
for inst in cfg.get("instruments") or []:
committed_at = (st["instruments"].get(inst) or {}).get("committed_at")
@@ -382,9 +425,33 @@ def _passports_view():
# false badge denial — the doc's shown-not-judged rule.
badge = "shown_not_judged"
elif qualifying >= req["songs"] and len(cleared) == len(required):
badge = "earned"
# Bronze is earned; GOLD upgrades it when a verified improv
# artifact exists for this genre's jam style. Virtuoso mints
# under raw STYLE_PALETTES ids ('punk', 'djent', 'disco', ...),
# which are mostly NOT family keys — so match in family space:
# the same keyword bucketing genres get ('punk' and 'punk
# rock' both bucket to 'rock'), with the exact key as a direct
# hit. Bronze remains a standalone win; gold never becomes an
# obligation.
fam = _genre_family(gkey)
gold = any(
s == gkey or (fam is not None and _genre_family(s) == fam)
for s in gold_improv
)
badge = "gold" if gold else "earned"
else:
badge = "in_progress"
# Practice invitation: the non-qualifying songs closest to the
# QUALIFYING bar (the badge ask), nearest first — invitation
# data, the UI voices it without meters.
thresholds = _state["content"]["star_accuracy_thresholds"]
bar = (thresholds[req["min_stars"] - 1]
if 0 < req["min_stars"] <= len(thresholds) else None)
nearest = [] if bar is None else sorted(
(s for s in songs if not s["qualifies"]),
key=lambda s: bar - s["best_accuracy"])[:3]
for s in nearest:
s["bar_at"] = bar
passports.append({
"genre_key": gkey,
"genre": meta.get("genre") or gkey,
@@ -393,13 +460,18 @@ def _passports_view():
"graded": is_graded,
"songs": songs,
"qualifying_count": qualifying,
"nearest": nearest,
# Honest hours odometer (Stage 5 post-cap): a true fact that
# only grows — never a target, never a meter.
"seconds_total": round(played_seconds.get((inst, gkey), 0.0), 1),
"drills": {"required": required, "cleared": cleared},
"badge": badge,
})
instruments[inst] = {"committed_at": committed_at, "passports": passports}
inst_gigs = [g for g in all_gigs if g.get("instrument") == inst]
for p in passports:
p["gigs"] = [g for g in inst_gigs if g.get("genre_key") == p["genre_key"]][-20:][::-1]
instruments[inst] = {"committed_at": committed_at, "passports": passports,
"gig_count": len(inst_gigs)}
return {
"config": {
"badge_requirement": cfg.get("badge_requirement") or {},
@@ -414,6 +486,60 @@ def _passports_view():
}
def _gig_config():
cfg = _state["passports_content"].get("gig")
cfg = cfg if isinstance(cfg, dict) else {}
def _num(key, default, cast):
# Tuning data, not code: junk falls back instead of 500ing both gig
# endpoints, and a legitimate 0 (stakes_songs: 0) is respected.
val = cfg.get(key)
if isinstance(val, bool) or not isinstance(val, (int, float)):
return default
return cast(val)
return {
"min_songs": max(1, _num("min_songs", 3, int)),
"max_songs": max(1, _num("max_songs", 5, int)),
"stakes_songs": max(0, _num("stakes_songs", 2, int)),
"encore_accuracy": _num("encore_accuracy", 0.75, float),
}
def _current_venue():
"""Highest unlocked venue (the room you can book today)."""
stars_total, _, _ = _stars()
best = None
for v in _state["content"]["venues"]:
if stars_total >= v["star_threshold"]:
if best is None or v["star_threshold"] >= best["star_threshold"]:
best = v
return best
def _unplayed_genre_songs(gkey, exclude, limit):
"""Library songs of a genre with no stats yet — a young passport's gig
still gets a full set (playing them is how stubs start).
ponytail: full stat-less scan + python-side genre match (a few ms at 7k
songs, single-user); push the match into SQL if propose ever feels slow."""
db = _state["meta_db"]
if db is None:
return []
rows = db.conn.execute(
f"SELECT filename, title, artist, {_genre_expr(db)} AS g FROM songs "
"WHERE filename NOT IN (SELECT filename FROM song_stats)"
).fetchall()
out = []
for filename, title, artist, genre in rows:
if _genre_key(genre) != gkey or filename in exclude:
continue
out.append({"filename": filename, "title": title or filename,
"artist": artist or ""})
if len(out) >= limit:
break
return out
def _validate_pack_dir(pack_dir: Path):
"""Raise ValueError unless pack_dir holds a complete venue pack."""
manifest_path = pack_dir / "manifest.json"
@@ -585,16 +711,150 @@ def setup(app, context):
# drops junk entries, which must not become a size-guard bypass.
if len(json.dumps(body["byNode"])) > DRILL_SNAPSHOT_MAX_BYTES:
raise HTTPException(413, "Snapshot too large.")
gold_in = body.get("goldImprov", {})
if not isinstance(gold_in, dict):
# A relay bug must be LOUD, not a silent 200 that drops gold.
raise HTTPException(400, "goldImprov must be an object keyed by style id.")
# Keep only plausible artifacts: a dict that names its verifier —
# an empty {} must not mint an evidence-free gold.
gold_in = {k: v for k, v in gold_in.items()
if isinstance(v, dict) and v.get("verifier")}
# Same pre-merge bound byNode gets: the gained-only merge dropping
# junk must not become a size-guard bypass (nor lock-held CPU burn).
if len(json.dumps(gold_in)) > DRILL_SNAPSHOT_MAX_BYTES:
raise HTTPException(413, "Snapshot too large.")
with _lock:
_, existing = _drill_by_node()
_, existing, existing_gold = _drill_by_node()
snapshot = {"mode": body.get("mode"), "xp": body.get("xp"),
"byNode": _merge_drill_nodes(existing, body["byNode"])}
"byNode": _merge_drill_nodes(existing, body["byNode"]),
"goldImprov": _merge_gold(existing_gold, gold_in)}
if len(json.dumps(snapshot)) > DRILL_SNAPSHOT_MAX_BYTES:
raise HTTPException(413, "Snapshot too large.")
_save_json(_drill_file(), {"received_at": _now_iso(),
"snapshot": snapshot})
return {"ok": True}
@app.post(f"/api/plugins/{PLUGIN_ID}/gigs/propose")
def propose_gig(body: dict = Body(...)):
inst = str((body or {}).get("instrument") or "")
genre = _genre_display((body or {}).get("genre"))
gkey = genre.lower()
if inst not in (_state["passports_content"].get("instruments") or []):
raise HTTPException(400, "Unknown instrument.")
if not gkey or len(genre) > GENRE_MAX_LEN:
raise HTTPException(400, "Provide a genre.")
cfg = _gig_config()
try:
size = int((body or {}).get("size") or 4)
except (TypeError, ValueError):
raise HTTPException(400, "size must be a number.")
size = max(cfg["min_songs"], min(cfg["max_songs"], size))
played, _seconds = _played_by_instrument_genre()
stubs = list(played.get((inst, gkey), {}).values())
req = _badge_requirement(gkey, inst)
qualifying = [s for s in stubs if s["stars"] >= req["min_stars"]]
rest = [s for s in stubs if s["stars"] < req["min_stars"]]
# The set: mostly songs you own, plus a couple of stakes songs near
# the bar; a young passport fills from unplayed genre songs so the
# first gig is how stubs start. random per call = free re-roll.
random.shuffle(qualifying)
rest.sort(key=lambda s: -s["best_accuracy"])
qtaken = max(1, size - cfg["stakes_songs"])
picks = qualifying[:qtaken]
for s in rest:
if len(picks) >= size:
break
picks.append(s)
# Surplus qualifying songs backfill a short set — a mature passport
# with no near-bar songs left must still fill the bill. Offset by how
# many QUALIFYING songs were taken, not len(picks): rest's stakes
# additions would otherwise skip eligible qualifying songs entirely.
for s in qualifying[qtaken:]:
if len(picks) >= size:
break
picks.append(s)
if len(picks) < size:
exclude = {s["filename"] for s in picks}
picks.extend(_unplayed_genre_songs(gkey, exclude, size - len(picks)))
if not picks:
raise HTTPException(404, "No songs of this genre in the library.")
venue = _current_venue()
return {
"instrument": inst,
"genre": genre,
"genre_key": gkey,
"venue_id": venue["id"] if venue else None,
"venue_name": venue["name"] if venue else "",
"songs": [{"filename": s["filename"], "title": s.get("title") or s["filename"],
"artist": s.get("artist") or ""} for s in picks[:size]],
}
@app.post(f"/api/plugins/{PLUGIN_ID}/gigs")
def log_gig(body: dict = Body(...)):
# Called by the runner ONLY when the set completed — an abandoned set
# never logs (no fail state; the gig you finished is the gig you
# played). Accuracies come from song_stats, freshly written by the
# set's own plays.
inst = str((body or {}).get("instrument") or "")
genre = _genre_display((body or {}).get("genre"))
gkey = genre.lower()
venue_id = str((body or {}).get("venue_id") or "")
songs = (body or {}).get("songs")
if inst not in (_state["passports_content"].get("instruments") or []):
raise HTTPException(400, "Unknown instrument.")
if not gkey or len(genre) > GENRE_MAX_LEN:
raise HTTPException(400, "Provide a genre.")
if venue_id and (not VENUE_ID_RE.fullmatch(venue_id) or _venue(venue_id) is None):
raise HTTPException(400, "Unknown venue.")
if (not isinstance(songs, list) or not songs or len(songs) > 8
or not all(isinstance(f, str) and f.strip() for f in songs)):
raise HTTPException(400, "songs must be 1-8 filenames.")
db = _state["meta_db"]
entries = []
accuracies = []
for filename in songs:
title = filename
accuracy = None
if db is not None:
# The NEWEST row is the set's own just-recorded play — a
# MAX(last_accuracy) across arrangements would happily log a
# stale higher score from another instrument's old session.
row = db.conn.execute(
"SELECT last_accuracy FROM song_stats WHERE filename = ? "
"ORDER BY last_played_at DESC LIMIT 1",
(filename,)).fetchone()
if row and row[0] is not None:
accuracy = round(float(row[0]), 4)
accuracies.append(accuracy)
trow = db.conn.execute(
"SELECT title FROM songs WHERE filename = ?", (filename,)).fetchone()
if trow and trow[0]:
title = trow[0]
entries.append({"filename": filename, "title": title, "accuracy": accuracy})
# Encore needs the WHOLE set scored at the bar — one scored song must
# not earn an encore for a set that was 4/5 unheard.
encore = (len(accuracies) == len(songs) and
sum(accuracies) / len(accuracies) >= _gig_config()["encore_accuracy"])
gig = {
"at": _now_iso(),
"venue_id": venue_id or None,
"instrument": inst,
"genre": genre,
"genre_key": gkey,
"songs": entries,
"encore": encore,
}
with _lock:
st = _career_state()
if not isinstance(st.get("gigs"), list):
st["gigs"] = []
st["gigs"].append(gig)
# ponytail: hard cap — nothing reads past the last 20 per
# passport; the state file must not grow (and export) forever.
st["gigs"] = st["gigs"][-500:]
_save_json(_state_file(), st)
return {"ok": True, "gig": gig}
@app.post(f"/api/plugins/{PLUGIN_ID}/packs/{{venue_id}}/download")
def start_download(venue_id: str):
venue = _venue(venue_id) if VENUE_ID_RE.fullmatch(venue_id) else None
+1
View File
@@ -29,6 +29,7 @@
<div id="career-tab-passports" class="hidden" role="tabpanel" aria-labelledby="career-tab-btn-passports">
<p class="text-sm text-gray-400 mb-4">Commit to an instrument, pick a genre, and stamp your way to its badge — five ★★ songs mint a Bronze. Your passport wall is who you are as a musician.</p>
<div id="pp-instruments" class="pp-instruments"></div>
<div id="pp-closest" class="mt-4"></div>
<div id="pp-shelf-wrap" class="mt-5">
<div id="pp-shelf" class="pp-shelf"></div>
</div>
+671 -38
View File
@@ -38,6 +38,8 @@
let _ppCeremonyActive = false;
let _ppBootstrapped = false;
let _ppNotified = {}; // badges chimed this session (slam still pending)
let _ppGigProposal = null; // the booking poster's proposal, while open
let _ppGigRun = null; // {songs, venue_id, genre, genre_key, instrument, idx} mid-set
function $(id) { return document.getElementById(id); }
@@ -302,11 +304,15 @@
} catch (_) { return {}; }
}
function badgeId(inst, gkey) { return inst + '/' + gkey; }
// Bronze keeps the legacy un-suffixed id, so badges seen before the Gold
// tier existed stay seen; gold is a distinct moment with its own id.
function badgeId(inst, gkey, tier) { return inst + '/' + gkey + (tier === 'gold' ? '@gold' : ''); }
function markBadgeSeen(inst, gkey) {
function markBadgeSeen(inst, gkey, tier) {
const seen = seenBadges();
seen[badgeId(inst, gkey)] = 1;
seen[badgeId(inst, gkey, tier)] = 1;
// A gold slam covers the bronze moment too — never queue both.
if (tier === 'gold') seen[badgeId(inst, gkey)] = 1;
lsSet(PP_SEEN_KEY, JSON.stringify(seen));
}
@@ -318,15 +324,19 @@
const seen = seenBadges();
for (const inst of Object.keys(view.instruments || {})) {
for (const p of (view.instruments[inst].passports || [])) {
const id = badgeId(inst, p.genre_key);
if (p.badge !== 'earned' || seen[id] || _ppNotified[id]) continue;
if (p.badge !== 'earned' && p.badge !== 'gold') continue;
const gold = p.badge === 'gold';
const id = badgeId(inst, p.genre_key, p.badge);
if (seen[id] || _ppNotified[id]) continue;
_ppNotified[id] = true;
sfx('chime');
if (window.fbNotify && typeof window.fbNotify.show === 'function') {
window.fbNotify.show({
big: true, icon: '🛂', accent: '#b45309',
title: 'Badge earned!',
message: `${p.genre} — Bronze, ready to stamp into your ${ppLabel(inst)} passport.`,
big: true, icon: gold ? '🏅' : '🛂', accent: gold ? '#d9a253' : '#b45309',
title: gold ? 'Gold — a verified improv!' : 'Badge earned!',
message: gold
? `${p.genre} — your ${ppLabel(inst)} badge turns gold.`
: `${p.genre} — Bronze, ready to stamp into your ${ppLabel(inst)} passport.`,
});
}
badgeCeremony(inst, p);
@@ -375,11 +385,11 @@
el.innerHTML = `
<canvas class="pp-confetti"></canvas>
<div class="pp-ceremony-card">
<div class="pp-stamp pp-stamp-page pp-ceremony-stamp" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg">
<div class="pp-stamp pp-stamp-page pp-ceremony-stamp${p.badge === 'gold' ? ' pp-stamp-gold' : ''}" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg">
<span class="pp-stamp-genre">${esc(p.genre.toUpperCase())}</span>
<span class="pp-stamp-tier">BRONZE</span>
<span class="pp-stamp-tier">${p.badge === 'gold' ? 'GOLD' : 'BRONZE'}</span>
</div>
<div class="pp-ceremony-title">Badge earned</div>
<div class="pp-ceremony-title">${p.badge === 'gold' ? 'Gold — a verified improv' : 'Badge earned'}</div>
<div class="pp-ceremony-sub">${esc(p.genre)}${esc(ppLabel(inst))} passport</div>
</div>`;
let timer = 0;
@@ -443,7 +453,11 @@
fetch(`${API}/drill-state`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ mode: snap.mode, xp: snap.xp, byNode: snap.byNode }),
body: JSON.stringify({
mode: snap.mode, xp: snap.xp, byNode: snap.byNode,
...(snap.goldImprov && typeof snap.goldImprov === 'object' && !Array.isArray(snap.goldImprov)
? { goldImprov: snap.goldImprov } : {}),
}),
}).then(() => refreshPassports()).catch(() => { /* next event retries */ });
}, 1500);
}
@@ -458,6 +472,8 @@
_pp = view;
detectNewBadges(view);
renderPassports();
renderProfileWall();
renderDashCard();
if (!_ppBootstrapped) {
_ppBootstrapped = true;
// Sync the local drill snapshot once per session — drill progress
@@ -479,9 +495,9 @@
function ppCoverHTML(inst, p) {
const rot = ppJitter(inst + p.genre_key, 1.6).toFixed(2);
const earned = p.badge === 'earned';
const earned = p.badge === 'earned' || p.badge === 'gold';
const stamp = earned
? `<span class="pp-stamp pp-stamp-mini" style="--pp-rot:${ppJitter(p.genre_key, 8).toFixed(1)}deg">BRONZE</span>`
? `<span class="pp-stamp pp-stamp-mini${p.badge === 'gold' ? ' pp-stamp-gold' : ''}" style="--pp-rot:${ppJitter(p.genre_key, 8).toFixed(1)}deg">${p.badge === 'gold' ? 'GOLD' : 'BRONZE'}</span>`
: '';
const stubs = p.qualifying_count === 1 ? '1 stub' : `${p.qualifying_count} stubs`;
const hours = fmtHours(p.seconds_total);
@@ -497,6 +513,56 @@
</button>`;
}
// Practice invitations: which stamps are closest, and what would bring
// them home. Invitations only — no meters, no obligations.
// Floor, never round: 74.9% must not display as the already-met "75%".
function pct(frac) { return Math.floor((Number(frac) || 0) * 100); }
function ppNeed(p) {
return Math.max(0, ((p.requirement || {}).songs || 0) - (p.qualifying_count || 0));
}
// The one blocker phrase — shared by the Closest-stamps strip and the
// passport book's invite line so they can never contradict each other.
function ppAskHTML(p, withHint) {
const req = p.requirement || {};
const need = ppNeed(p);
const starGl = '★'.repeat(req.min_stars || 0);
if (need > 0) {
const near = withHint ? (p.nearest || [])[0] : null;
const hint = near
? ` · nearest: <em>${esc(near.title)}</em> at ${pct(near.best_accuracy)}%`
: '';
return `${need === 1 ? `one more ${starGl} song` : `${need} more ${starGl} songs`}${hint}`;
}
const labels = ((_pp && _pp.config) || {}).drill_labels || {};
const drills = p.drills || {};
const pending = (drills.required || []).filter((n) => !(drills.cleared || []).includes(n));
return `clear ${pending.map((n) => esc(labels[n] || n)).join(', ') || 'the genre drill'} in Virtuoso`;
}
function closestLineHTML(p) {
return `<button class="pp-closest-row" data-pp-open="${esc(p.genre_key)}">
<span class="pp-closest-genre">${esc(p.genre)}</span>
<span class="pp-closest-ask">${ppAskHTML(p, true)}</span>
</button>`;
}
function renderClosest(inst, data) {
const host = $('pp-closest');
if (!host) return;
const candidates = (data.passports || [])
.filter((p) => p.badge === 'in_progress')
.sort((a, b) => ppNeed(a) - ppNeed(b))
.slice(0, 3);
if (!candidates.length) { host.innerHTML = ''; return; }
host.innerHTML = `<div class="pp-closest">
<div class="pp-closest-head">Closest stamps</div>
${candidates.map(closestLineHTML).join('')}
</div>`;
}
function renderShelf(inst, data) {
const shelf = $('pp-shelf');
if (!shelf) return;
@@ -545,12 +611,13 @@
const data = (_pp.instruments || {})[inst] || { passports: [] };
host.innerHTML = ((_pp.config || {}).instruments || []).map((i) => {
const d = (_pp.instruments || {})[i] || {};
const earned = (d.passports || []).filter((p) => p.badge === 'earned').length;
const earned = (d.passports || []).filter((p) => p.badge === 'earned' || p.badge === 'gold').length;
const committed = !!d.committed_at;
return `<button class="pp-inst${i === inst ? ' active' : ''}${committed ? '' : ' uncommitted'}" data-pp-inst="${esc(i)}">
${esc(ppLabel(i))}${earned ? ` <span class="pp-inst-badges">⚡${earned}</span>` : ''}${committed ? '' : ' <span class="pp-inst-plus">+</span>'}
</button>`;
}).join('');
renderClosest(inst, data);
renderShelf(inst, data);
renderRack(inst, data);
}
@@ -576,39 +643,36 @@
function ppBookHTML(inst, p, pendingSlam) {
const req = p.requirement || {};
const need = Math.max(0, (req.songs || 0) - p.qualifying_count);
const starGl = '★'.repeat(req.min_stars || 0);
const reqNodes = (p.drills || {}).required || [];
const clearedNodes = new Set((p.drills || {}).cleared || []);
const labels = ((_pp && _pp.config) || {}).drill_labels || {};
const pendingDrills = reqNodes.filter((n) => !clearedNodes.has(n));
// The invite names what actually blocks the stamp: songs first, then
// the genre drill once the song bar is met.
let invite;
if (need > 0) {
invite = need === 1 ? `One more ${starGl} song mints this stamp.`
: `${need} more ${starGl} songs mint this stamp.`;
} else {
const names = pendingDrills.map((n) => labels[n] || n).join(', ');
invite = `Clear ${names || 'the genre drill'} in Virtuoso to mint this stamp.`;
}
// The invite names what actually blocks the stamp — same shared
// phrase as the Closest-stamps strip, so they can't contradict.
const invite = `${ppAskHTML(p, false)} mints this stamp.`;
let badgeArea = '';
if (p.badge === 'shown_not_judged') {
badgeArea = `<div class="pp-snj">Shown, not judged — your ${esc(ppLabel(inst).toLowerCase())} repertoire speaks for itself.</div>`;
} else if (p.badge === 'earned') {
badgeArea = `<div class="pp-stamp pp-stamp-page${pendingSlam ? ' pp-stamp-hidden' : ' pp-tilt'}" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg">
} else if (p.badge === 'earned' || p.badge === 'gold') {
const gold = p.badge === 'gold';
badgeArea = `<div class="pp-stamp pp-stamp-page${pendingSlam ? ' pp-stamp-hidden' : ' pp-tilt'}${gold ? ' pp-stamp-gold' : ''}" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg">
<span class="pp-stamp-genre">${esc(p.genre.toUpperCase())}</span>
<span class="pp-stamp-tier">BRONZE</span>
<span class="pp-stamp-tier">${gold ? 'GOLD' : 'BRONZE'}</span>
</div>
<div class="pp-gold-foil" aria-hidden="true">GOLD</div>
<div class="pp-gold-note">Gold rung coming — improvise it, verified.</div>`;
${gold
? '<div class="pp-gold-foil" aria-hidden="true">GOLD</div><div class="pp-gold-note">A verified improv — the comb heard it live.</div>'
: '<div class="pp-gold-note">Gold rung: improvise over this style in a Virtuoso jam — verified, not self-reported.</div>'}
<div class="pp-card-actions">
<button class="career-btn career-btn-ghost" data-pp-card="save">Save card</button>
<button class="career-btn career-btn-ghost" data-pp-card="copy">Copy card</button>
</div>`;
} else {
const fill = (ppFillFraction(p) * 100).toFixed(0);
badgeArea = `<div class="pp-stamp pp-stamp-page pp-stamp-ghost" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg; --pp-fill:${fill}%">
<span class="pp-stamp-genre">${esc(p.genre.toUpperCase())}</span>
<span class="pp-stamp-tier">BRONZE</span>
</div>
<div class="pp-invite">${esc(invite)}</div>`;
<div class="pp-invite">${invite.charAt(0).toUpperCase()}${invite.slice(1)}</div>`;
}
const hours = fmtHours(p.seconds_total);
const odometer = hours
@@ -628,15 +692,34 @@
: `Play ${esc(p.genre)} songs at ${starGl} to collect ticket stubs.`;
const stubsHTML = stubs.length ? stubs.map(ppStubHTML).join('')
: `<div class="pp-stub-empty">${emptyLine}</div>`;
// Bring-these-up: nearest-to-the-bar songs (graded, unearned only —
// an earned page is memorabilia, not homework).
let nearest = '';
if (p.badge === 'in_progress' && (p.nearest || []).length) {
nearest = `<div class="pp-nearest">
<div class="pp-nearest-head">Bring these up</div>
${p.nearest.map((s) =>
`<div class="pp-nearest-row"><em>${esc(s.title)}</em> — best ${pct(s.best_accuracy)}%, ${starGl} at ${pct(s.bar_at)}%</div>`).join('')}
</div>`;
}
let gigLog = '';
if ((p.gigs || []).length) {
gigLog = `<div class="pp-giglog">
<div class="pp-giglog-head">Gigs played</div>
${p.gigs.slice(0, 6).map((g) =>
`<div class="pp-giglog-row">${esc((g.at || '').slice(0, 10))} · ${esc(_venueName(g.venue_id))}${g.encore ? ' · <b>encore</b>' : ''}</div>`).join('')}
</div>`;
}
return `<div class="pp-book-wrap" data-pp-close-bg="1" role="dialog" aria-modal="true" aria-label="${esc(p.genre)} ${esc(ppLabel(inst))} passport">
<div class="pp-book">
<div class="pp-page pp-page-left">
<div class="pp-page-head">${esc(p.genre)}${esc(ppLabel(inst))}</div>
${badgeArea}${odometer}${drills}
<button class="career-btn career-btn-primary pp-gig-book" data-pp-gig="${esc(p.genre_key)}">Book a gig</button>
</div>
<div class="pp-page pp-page-right">
<div class="pp-page-head">Ticket stubs</div>
<div class="pp-stubs">${stubsHTML}</div>
<div class="pp-stubs">${stubsHTML}${nearest}${gigLog}</div>
</div>
<div class="pp-book-cover pp-leather-${esc(inst)}">
<span class="pp-cover-title">${esc(p.genre.toUpperCase())}</span>
@@ -655,7 +738,8 @@
if (!p || !overlay) return;
_ppBook = { inst, gkey };
_ppReturnFocus = document.activeElement;
const pending = p.badge === 'earned' && !seenBadges()[badgeId(inst, gkey)];
const pending = (p.badge === 'earned' || p.badge === 'gold')
&& !seenBadges()[badgeId(inst, gkey, p.badge)];
overlay.innerHTML = ppBookHTML(inst, p, pending);
overlay.classList.remove('hidden');
const close = overlay.querySelector('.pp-book-close');
@@ -677,7 +761,7 @@
stamp.classList.add('pp-tilt'); // freshly slammed = trading card too
if (book) book.classList.add('pp-shake');
sfx('stamp');
markBadgeSeen(inst, gkey);
markBadgeSeen(inst, gkey, p.badge);
renderPassports(); // the shelf cover gains its mini-stamp
}, 950);
}
@@ -685,6 +769,7 @@
function closeBook() {
_ppBook = null;
_ppGigProposal = null; // a dismissed poster is a dismissed booking
const overlay = $('pp-overlay');
if (overlay) { overlay.classList.add('hidden'); overlay.innerHTML = ''; }
if (_ppReturnFocus && typeof _ppReturnFocus.focus === 'function' &&
@@ -771,6 +856,523 @@
if (_tiltEl) { resetTilt(_tiltEl); _tiltEl = null; }
}
// ── Shareable passport card (canvas → PNG, save or clipboard) ─────────
// Keep in sync with the .pp-leather-* gradients in assets/career.css —
// canvas can't consume a CSS class, so the pairs live twice on purpose.
const PP_LEATHER_HEX = {
guitar: ['#5c2321', '#401412'],
bass: ['#1f3252', '#131f36'],
keys: ['#1e4034', '#122a21'],
drums: ['#3f3f46', '#26262b'],
};
function drawPassportCard(inst, p) {
const W = 480;
const H = 640;
const canvas = document.createElement('canvas');
canvas.width = W;
canvas.height = H;
const ctx = canvas.getContext('2d');
const [c1, c2] = PP_LEATHER_HEX[inst] || PP_LEATHER_HEX.guitar;
const bg = ctx.createLinearGradient(0, 0, W, H);
bg.addColorStop(0, c1);
bg.addColorStop(1, c2);
ctx.fillStyle = bg;
ctx.fillRect(0, 0, W, H);
// Emboss frame
ctx.strokeStyle = 'rgba(240,226,195,0.35)';
ctx.lineWidth = 3;
ctx.strokeRect(18, 18, W - 36, H - 36);
// Genre title
ctx.fillStyle = 'rgba(240,226,195,0.95)';
ctx.textAlign = 'center';
ctx.font = '700 34px Georgia, serif';
ctx.fillText(p.genre.toUpperCase(), W / 2, 92, W - 80);
ctx.font = '400 15px Georgia, serif';
ctx.fillStyle = 'rgba(240,226,195,0.55)';
ctx.fillText(`${ppLabel(inst).toUpperCase()} PASSPORT`, W / 2, 122);
// Stamp ring
const gold = p.badge === 'gold';
const ink = gold ? '#d9a253' : '#b06a2a';
const cy = 330;
ctx.strokeStyle = ink;
ctx.lineWidth = 6;
ctx.beginPath();
ctx.arc(W / 2, cy, 118, 0, Math.PI * 2);
ctx.stroke();
ctx.lineWidth = 2;
ctx.beginPath();
ctx.arc(W / 2, cy, 106, 0, Math.PI * 2);
ctx.stroke();
ctx.fillStyle = ink;
ctx.font = '800 26px Georgia, serif';
ctx.fillText(p.genre.toUpperCase(), W / 2, cy - 6, 190);
ctx.font = '600 16px Georgia, serif';
ctx.fillText(gold ? 'G O L D' : 'B R O N Z E', W / 2, cy + 28);
// Facts
const stubCount = (p.songs || []).filter((sng) => sng.qualifies).length;
const hours = fmtHours(p.seconds_total);
ctx.fillStyle = 'rgba(240,226,195,0.75)';
ctx.font = '400 17px Georgia, serif';
ctx.fillText(`${stubCount} ticket stub${stubCount === 1 ? '' : 's'}${hours ? ` · ${hours} played` : ''}`, W / 2, 512);
ctx.fillStyle = 'rgba(240,226,195,0.4)';
ctx.font = '400 13px Georgia, serif';
ctx.fillText('fee[dB]ack · career passport', W / 2, H - 44);
return canvas;
}
// One export path for every canvas artifact: copy (with download
// fallback + notice) or save, failures audible.
function exportCanvasPng(canvas, filename, mode, noun) {
canvas.toBlob(async (blob) => {
const fail = (why) => {
if (window.fbNotify) window.fbNotify.show({ icon: '⚠️', title: `${noun} export failed`, message: why });
};
if (!blob) { fail('The canvas produced no image.'); return; }
try {
const io = await import('/static/js/blob-io.js');
if (mode === 'copy') {
const ok = await io.copyImageBlob(blob);
if (ok) {
if (window.fbNotify) window.fbNotify.show({ icon: '📋', title: `${noun} copied`, message: 'Paste it anywhere.' });
return;
}
if (window.fbNotify) window.fbNotify.show({ icon: '💾', title: 'Clipboard unavailable', message: `Saved the ${noun.toLowerCase()} instead.` });
}
io.downloadBlob(blob, filename);
} catch (e) { fail('Export helper unavailable.'); }
}, 'image/png');
}
function exportPassportCard(mode) {
if (!_ppBook || !_pp) return;
const { inst, gkey } = _ppBook;
const p = (((_pp.instruments || {})[inst] || {}).passports || [])
.find((x) => x.genre_key === gkey);
if (!p) return;
const canvas = drawPassportCard(inst, p);
exportCanvasPng(canvas, `passport-${inst}-${gkey.replace(/[^a-z0-9-]+/g, '-')}.png`, mode, 'Card');
}
// ── Career surfaces outside the plugin screen ─────────────────────────
// Profile passport wall + the home-page career card. Both inject into
// core-owned mounts announced by v3:profile-rendered /
// v3:dashboard-rendered (the achievements seam). Absent-not-empty: with
// no committed instrument they render nothing and the dashboard keeps
// its built-in fallback stat.
function careerTotals() {
if (!_pp) return null;
let badges = 0;
let seconds = 0;
let gigs = 0;
const walls = [];
for (const inst of (_pp.config || {}).instruments || []) {
const d = (_pp.instruments || {})[inst];
// A commitment with no opened passport isn't a wall yet — the
// external surfaces (profile, home card) stay ABSENT until a
// passport exists (absent-not-empty).
if (!d || !d.committed_at || !(d.passports || []).length) continue;
const earned = (d.passports || []).filter((p) => p.badge === 'earned' || p.badge === 'gold');
badges += earned.length;
seconds += (d.passports || []).reduce((t, p) => t + (p.seconds_total || 0), 0);
gigs += d.gig_count || 0;
walls.push({ inst, earned, opened: d.passports.length });
}
if (!walls.length) return null;
return { badges, seconds, gigs, walls };
}
function closestAskHTML() {
if (!_pp) return '';
let best = null;
for (const inst of (_pp.config || {}).instruments || []) {
for (const p of (((_pp.instruments || {})[inst] || {}).passports || [])) {
if (p.badge !== 'in_progress') continue;
const need = Math.max(0, ((p.requirement || {}).songs || 0) - p.qualifying_count);
if (!best || need < best.need) best = { p, need };
}
}
if (!best) return '';
const starGl = '★'.repeat((best.p.requirement || {}).min_stars || 0);
if (best.need > 0) {
return `${esc(best.p.genre)}${best.need === 1 ? `one more ${starGl} song` : `${best.need} more ${starGl} songs`}`;
}
return `${esc(best.p.genre)} — one drill away`;
}
function renderProfileWall() {
const mount = document.getElementById('v3-profile-passports-mount');
if (!mount) return;
const totals = careerTotals();
if (!totals) { mount.innerHTML = ''; return; }
const shelves = totals.walls.map(({ inst, earned, opened }) => {
const covers = earned.map((p) =>
`<button class="pp-wall-cover pp-leather-${esc(inst)}" data-pp-wall-inst="${esc(inst)}" data-pp-wall-gkey="${esc(p.genre_key)}" title="${esc(p.genre)}">
<span>${esc(p.genre.toUpperCase())}</span>
<em>${p.badge === 'gold' ? 'GOLD' : 'BRONZE'}</em>
</button>`).join('');
const line = earned.length
? covers
: `<span class="pp-wall-none">${opened} passport${opened === 1 ? '' : 's'} open — first stamp pending</span>`;
return `<div class="pp-wall-shelf"><span class="pp-wall-inst">${esc(ppLabel(inst))}</span>${line}</div>`;
}).join('');
const hours = fmtHours(totals.seconds);
mount.innerHTML = `<div class="bg-fb-card/80 backdrop-blur rounded-lg p-4 border border-fb-border/50 pp-wall">
<div class="pp-wall-head">
<span>Passport wall</span>
<span class="pp-wall-meta">${totals.badges} badge${totals.badges === 1 ? '' : 's'}${hours ? ` · ${hours} played` : ''}${totals.gigs ? ` · ${totals.gigs} gig${totals.gigs === 1 ? '' : 's'}` : ''}</span>
</div>
${shelves}
<button class="pp-wall-link" data-pp-wall-career="1">Open career →</button>
</div>`;
if (!mount.dataset.ppWired) {
mount.dataset.ppWired = '1';
mount.addEventListener('click', onWallClick);
}
}
function onWallClick(e) {
const open = e.target.closest('[data-pp-wall-inst]');
if (open) {
// Two attributes, not a '/'-joined pair: a genre key may itself
// contain '/' ("drum/bass") and must round-trip intact.
const inst = open.dataset.ppWallInst;
const gkey = open.dataset.ppWallGkey;
lsSet(PP_INST_KEY, inst);
if (window.showScreen) window.showScreen('plugin-career');
showCareerTab('passports');
renderPassports();
openBook(inst, gkey);
return;
}
if (e.target.closest('[data-pp-wall-career]')) {
if (window.showScreen) window.showScreen('plugin-career');
showCareerTab('passports');
}
}
function renderDashCard() {
const slot = document.getElementById('v3-dash-career-slot');
if (!slot) return;
const totals = careerTotals();
if (!totals) return; // keep core's fallback stat card
const hours = fmtHours(totals.seconds);
const ask = closestAskHTML();
slot.innerHTML = `<button class="pp-dash-card" data-pp-wall-career="1">
<span class="pp-dash-shine" aria-hidden="true"></span>
<span class="pp-dash-head">Career</span>
<span class="pp-dash-badges">${'⚡'.repeat(Math.min(totals.badges, 5))}<b>${totals.badges}</b> badge${totals.badges === 1 ? '' : 's'}</span>
<span class="pp-dash-meta">${hours ? `${hours} played` : 'the stage is set'}${totals.gigs ? ` · ${totals.gigs} gig${totals.gigs === 1 ? '' : 's'}` : ''}</span>
${ask ? `<span class="pp-dash-ask">closest: ${ask}</span>` : ''}
</button>`;
if (!slot.dataset.ppWired) {
slot.dataset.ppWired = '1';
slot.addEventListener('click', onWallClick);
}
}
// ── Gigs: booking poster → set runner → summary ──────────────────────
function _venueName(venueId) {
const v = (_state && _state.venues || []).find((x) => x.id === venueId);
return v ? v.name : (venueId || 'the stage');
}
function gigPosterHTML(prop) {
const bill = prop.songs.map((s, i) =>
`<div class="pp-poster-line"><span>${i + 1}.</span> ${esc(s.title)}${s.artist ? ` <em>${esc(s.artist)}</em>` : ''}</div>`).join('');
return `<div class="pp-book-wrap" data-pp-close-bg="1" role="dialog" aria-modal="true" aria-label="Gig poster">
<div class="pp-poster">
<div class="pp-poster-venue">${esc(prop.venue_name || 'The stage')}</div>
<div class="pp-poster-presents">presents</div>
<div class="pp-poster-title">${esc(prop.genre.toUpperCase())} NIGHT</div>
<div class="pp-poster-inst">${esc(ppLabel(prop.instrument))} · tonight</div>
<div class="pp-poster-bill">${bill}</div>
<div class="pp-poster-actions">
<button class="career-btn career-btn-primary" data-pp-gig-play="1">Play the gig</button>
<button class="career-btn career-btn-ghost" data-pp-gig-reroll="1">Re-roll</button>
<button class="career-btn career-btn-ghost" data-pp-poster="save">Save</button>
<button class="career-btn career-btn-ghost" data-pp-poster="copy">Copy</button>
</div>
<button class="pp-book-close" data-pp-close="1" aria-label="Close">✕</button>
</div>
</div>`;
}
async function bookGig(gkey) {
if (!_pp) return;
const inst = activeInstrument();
const p = (((_pp.instruments || {})[inst] || {}).passports || [])
.find((x) => x.genre_key === gkey);
if (!p) return;
try {
const res = await fetch(`${API}/gigs/propose`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ instrument: inst, genre: p.genre }),
});
if (!res.ok) return;
_ppGigProposal = await res.json();
} catch (_) { return; }
const overlay = $('pp-overlay');
if (!overlay) return;
_ppBook = null; // the poster replaces the book in the overlay
overlay.innerHTML = gigPosterHTML(_ppGigProposal);
overlay.classList.remove('hidden');
sfx('page');
}
function startGig() {
const prop = _ppGigProposal;
const q = window.feedBack && window.feedBack.playQueue;
if (!prop || !q || typeof q.start !== 'function' || typeof window.playSong !== 'function') return;
// The gig BORROWS the stage: stash whatever venue/viz the user had so
// the set ending gives it back (unlike "Play here", which is an
// explicit persistent choice on the venue card).
let restore = null;
if (prop.venue_id) {
// Capture the restore snapshot BEFORE any write: if a later write
// (or setViz) throws, the stage must still be returnable.
try {
restore = {
venue: localStorage.getItem(VENUE_OVERRIDE_KEY),
viz: localStorage.getItem('vizSelection'),
};
} catch (_) { restore = null; }
try {
localStorage.setItem(VENUE_OVERRIDE_KEY, prop.venue_id);
localStorage.setItem('vizSelection', 'venue');
if (typeof window.setViz === 'function') window.setViz('venue');
} catch (_) { /* viz optional — restore stays intact */ }
}
_appliedManifestVenue = null;
_ppGigRun = {
songs: prop.songs,
venue_id: prop.venue_id,
genre: prop.genre,
genre_key: prop.genre_key,
instrument: prop.instrument,
idx: 0,
restore,
};
closeBook();
_ppGigProposal = null;
// RAW filenames: the queue itself encodes for playSong — pre-encoding
// double-encodes and breaks loading + the stats/gig filename join.
if (!q.start(prop.songs.map((s) => s.filename), { source: 'gig' })) {
_ppGigRun = null;
return;
}
renderGigStrip();
}
function restoreGigStage(run) {
const r = run && run.restore;
if (!r) return;
try {
if (r.venue == null) localStorage.removeItem(VENUE_OVERRIDE_KEY);
else localStorage.setItem(VENUE_OVERRIDE_KEY, r.venue);
if (r.viz && r.viz !== 'venue') {
localStorage.setItem('vizSelection', r.viz);
if (typeof window.setViz === 'function') window.setViz(r.viz);
}
} catch (_) { /* best effort */ }
_appliedManifestVenue = null;
}
function renderGigStrip() {
if (!_ppGigRun || !document.body || typeof document.createElement !== 'function') return;
let strip = document.getElementById('pp-gig-strip');
if (!strip) {
strip = document.createElement('div');
strip.id = 'pp-gig-strip';
strip.className = 'pp-gig-strip';
document.body.appendChild(strip);
}
const run = _ppGigRun;
const next = run.songs[run.idx + 1];
strip.innerHTML = `<b>GIG</b> · ${esc(run.genre)} at ${esc(_venueName(run.venue_id))} · set ${Math.min(run.idx + 1, run.songs.length)}/${run.songs.length}${next ? ` — next: <em>${esc(next.title)}</em>` : ' — closer!'}`;
}
function removeGigStrip() {
const strip = document.getElementById('pp-gig-strip');
if (strip) strip.remove();
}
function abandonGig() {
// No fail state: an abandoned set logs nothing and says nothing.
const run = _ppGigRun;
_ppGigRun = null;
removeGigStrip();
restoreGigStage(run);
}
function completeGig() {
const run = _ppGigRun;
_ppGigRun = null;
removeGigStrip();
restoreGigStage(run);
// The final song's own stats POST races this moment (both ride
// song:ended): wait for its stats:recorded — or a short timeout, since
// an UNSCORED play never emits one — so /gigs reads the set's real
// accuracies, not last week's.
const lastFile = run.songs[run.songs.length - 1].filename;
const sm = window.feedBack;
let done = false;
const proceed = () => {
if (done) return;
done = true;
if (sm && typeof sm.off === 'function') { try { sm.off('stats:recorded', onRec); } catch (_) { /* ok */ } }
postGig(run);
};
const onRec = (e) => {
const d = (e && e.detail) || {};
if (d.filename === lastFile) proceed();
};
if (sm && typeof sm.on === 'function') sm.on('stats:recorded', onRec);
setTimeout(proceed, 3500);
}
async function postGig(run) {
let gig = null;
try {
const res = await fetch(`${API}/gigs`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
instrument: run.instrument,
genre: run.genre,
venue_id: run.venue_id,
songs: run.songs.map((s) => s.filename),
}),
});
if (res.ok) gig = (await res.json()).gig;
} catch (_) { /* summary still shows, unlogged */ }
showGigSummary(run, gig);
refreshPassports();
}
function showGigSummary(run, gig) {
if (!document.body || typeof document.createElement !== 'function') return;
const entries = (gig && gig.songs) || run.songs.map((s) => ({ filename: s.filename, title: s.title, accuracy: null }));
const encore = !!(gig && gig.encore);
if (encore && !reducedMotion()) {
const crowd = window.v3VenueCrowd;
if (crowd && typeof crowd.celebrate === 'function') {
try { crowd.celebrate(); } catch (_) { /* optional */ }
}
}
const el = document.createElement('div');
el.id = 'pp-gig-summary';
el.className = 'pp-ceremony-overlay';
el.innerHTML = `<canvas class="pp-confetti"></canvas>
<div class="pp-poster pp-poster-summary">
<div class="pp-poster-venue">${esc(_venueName(run.venue_id))}</div>
<div class="pp-poster-title">${esc(run.genre.toUpperCase())} NIGHT</div>
<div class="pp-poster-inst">${encore ? 'ENCORE! ' : ''}the set, as played</div>
<div class="pp-poster-bill">${entries.map((s, i) =>
`<div class="pp-poster-line"><span>${i + 1}.</span> ${esc(s.title)}${s.accuracy != null ? ` <b>${Math.floor(s.accuracy * 100)}%</b>` : ''}</div>`).join('')}</div>
<div class="pp-poster-actions">
<button class="career-btn career-btn-ghost" data-pp-poster="save">Save poster</button>
<button class="career-btn career-btn-ghost" data-pp-poster="copy">Copy poster</button>
<button class="career-btn career-btn-primary" data-pp-gig-done="1">Done</button>
</div>
</div>`;
el.addEventListener('click', (e) => {
if (e.target === el || e.target.closest('[data-pp-gig-done]')) {
el.remove();
} else if (e.target.closest('[data-pp-poster]')) {
exportGigPoster(e.target.closest('[data-pp-poster]').dataset.ppPoster,
{ ...run, encore, entries });
}
});
document.body.appendChild(el);
if (encore && !reducedMotion()) confettiBurst(el.querySelector('.pp-confetti'));
}
function drawGigPoster(data) {
const W = 480;
const H = 640;
const canvas = document.createElement('canvas');
canvas.width = W;
canvas.height = H;
const ctx = canvas.getContext('2d');
const bg = ctx.createLinearGradient(0, 0, 0, H);
bg.addColorStop(0, '#141019');
bg.addColorStop(1, '#241318');
ctx.fillStyle = bg;
ctx.fillRect(0, 0, W, H);
ctx.strokeStyle = 'rgba(217,162,83,0.5)';
ctx.lineWidth = 3;
ctx.strokeRect(16, 16, W - 32, H - 32);
ctx.textAlign = 'center';
ctx.fillStyle = 'rgba(240,226,195,0.65)';
ctx.font = '400 18px Georgia, serif';
ctx.fillText(_venueName(data.venue_id), W / 2, 76, W - 80);
ctx.font = '400 12px Georgia, serif';
ctx.fillText('P R E S E N T S', W / 2, 102);
ctx.fillStyle = '#d9a253';
ctx.font = '800 40px Georgia, serif';
ctx.fillText(`${data.genre.toUpperCase()}`, W / 2, 160, W - 60);
ctx.font = '800 26px Georgia, serif';
ctx.fillText('NIGHT', W / 2, 194);
if (data.encore) {
ctx.fillStyle = '#f3d179';
ctx.font = '700 16px Georgia, serif';
ctx.fillText('— E N C O R E —', W / 2, 226);
}
ctx.fillStyle = 'rgba(240,226,195,0.85)';
ctx.font = '400 18px Georgia, serif';
const entries = data.entries || data.songs || [];
entries.slice(0, 6).forEach((sng, i) => {
const acc = sng.accuracy != null ? ` · ${Math.floor(sng.accuracy * 100)}%` : '';
ctx.fillText(`${sng.title}${acc}`, W / 2, 290 + i * 44, W - 80);
});
ctx.fillStyle = 'rgba(240,226,195,0.4)';
ctx.font = '400 13px Georgia, serif';
ctx.fillText(`${ppLabel(data.instrument || 'guitar')} · fee[dB]ack career`, W / 2, H - 42);
return canvas;
}
function exportGigPoster(mode, data) {
exportCanvasPng(drawGigPoster(data),
`gig-${(data.genre_key || 'set').replace(/[^a-z0-9-]+/g, '-')}.png`, mode, 'Poster');
}
// Queue lifecycle: advance the strip per song; complete or abandon.
function onGigSongLoading() {
if (!_ppGigRun) return;
renderGigStrip();
}
function onGigSongEnded() {
if (!_ppGigRun) return;
const q = window.feedBack && window.feedBack.playQueue;
if (!q || typeof q.remaining !== 'function') return;
// Only OUR live queue counts: remaining()===0 is also true for a
// cleared/foreign queue (a manual play silently clears the gig queue,
// and that unrelated song's end must not log a gig).
if (q.source && q.source() !== 'gig') { abandonGig(); return; }
if (!q.remaining()) {
if (q.active && q.active()) completeGig();
else abandonGig();
return;
}
_ppGigRun.idx = Math.min(_ppGigRun.idx + 1, _ppGigRun.songs.length - 1);
renderGigStrip();
}
function onGigSongStop() {
// A deliberate quit mid-set (Escape clears the queue) abandons the
// gig — but the LAST song's teardown also fires song:stop after
// song:ended, so only abandon while songs genuinely remain.
if (!_ppGigRun) return;
const q = window.feedBack && window.feedBack.playQueue;
const active = q && typeof q.active === 'function' ? q.active() : false;
if (!active) abandonGig();
}
function openGenre(inst, genre) {
fetch(`${API}/passports/open`, {
method: 'POST',
@@ -820,6 +1422,23 @@
closeBook();
return;
}
const gigBtn = e.target.closest('[data-pp-gig]');
if (gigBtn) { bookGig(gigBtn.dataset.ppGig); return; }
if (e.target.closest('[data-pp-gig-play]')) { startGig(); return; }
if (e.target.closest('[data-pp-gig-reroll]')) {
if (_ppGigProposal) bookGig(_ppGigProposal.genre_key);
return;
}
const posterBtn = e.target.closest('[data-pp-poster]');
if (posterBtn && _ppGigProposal) {
exportGigPoster(posterBtn.dataset.ppPoster, _ppGigProposal);
return;
}
const cardBtn = e.target.closest('[data-pp-card]');
if (cardBtn) {
exportPassportCard(cardBtn.dataset.ppCard);
return;
}
const dlBtn = e.target.closest('[data-career-download]');
const delBtn = e.target.closest('[data-career-delete]');
const playBtn = e.target.closest('[data-career-play]');
@@ -870,6 +1489,10 @@
if (sm && typeof sm.on === 'function') {
// New song stats can add stars → thresholds may cross mid-session.
sm.on('stats:recorded', () => refresh());
// Gig runner lifecycle (no-ops when no gig is live).
sm.on('song:loading', onGigSongLoading);
sm.on('song:ended', onGigSongEnded);
sm.on('song:stop', onGigSongStop);
// Virtuoso's progress emits are the drill-state relay trigger; the
// payload is a thin delta, so the relay reads the full localStorage
// snapshot instead (see relayDrillState).
@@ -877,8 +1500,14 @@
}
showCareerTab(lsGet(PP_TAB_KEY) === 'passports' ? 'passports' : 'venues');
document.addEventListener('keydown', (e) => {
if (e.key === 'Escape' && _ppBook) closeBook();
if (e.key !== 'Escape') return;
const overlay = $('pp-overlay');
if (_ppBook || (overlay && !overlay.classList.contains('hidden'))) closeBook();
});
// Core re-renders profile/dashboard shells (innerHTML wipe) and
// announces the fresh mount points — same seam achievements uses.
document.addEventListener('v3:profile-rendered', renderProfileWall);
document.addEventListener('v3:dashboard-rendered', renderDashCard);
refresh();
}
@@ -886,7 +1515,11 @@
// the badge-diff logic; nothing here touches the DOM.
window.__careerPassportTest = {
ppKey, ppJitter, ppLabel, detectNewBadges, seenBadges, markBadgeSeen,
fmtHours, ppFillFraction,
fmtHours, ppFillFraction, careerTotals, closestAskHTML,
onGigSongEnded, onGigSongStop,
setGigRun(r) { _ppGigRun = r; },
getGigRun() { return _ppGigRun; },
setView(v) { _pp = v; },
};
if (document.readyState === 'loading') {
+97
View File
@@ -150,3 +150,100 @@ test('ppFillFraction: song progress toward the bar, in-progress only', () => {
assert.equal(ppFillFraction(p('in_progress', 3, 0)), 0); // no bar → no fill
assert.equal(ppFillFraction(null), 0);
});
test('careerTotals / wall + dash card stay absent without commitment', () => {
const w = load();
const t = w.__careerPassportTest;
// No _pp at all → null; committed-less view → null (absent-not-empty).
assert.equal(t.careerTotals(), null);
t.setView({ config: { instruments: ['guitar'] },
instruments: { guitar: { committed_at: null, passports: [] } } });
assert.equal(t.careerTotals(), null);
// Committed but zero passports opened: still absent (no zero-wall).
t.setView({ config: { instruments: ['guitar'] },
instruments: { guitar: { committed_at: 'x', passports: [] } } });
assert.equal(t.careerTotals(), null);
// Committed with an earned badge + hours → totals aggregate.
t.setView({ config: { instruments: ['guitar', 'bass'] },
instruments: {
guitar: { committed_at: 'x', passports: [
{ badge: 'earned', seconds_total: 3600, genre: 'Blues', genre_key: 'blues' },
{ badge: 'in_progress', seconds_total: 120, genre: 'Funk', genre_key: 'funk',
qualifying_count: 4, requirement: { songs: 5, min_stars: 2 } }] },
bass: { committed_at: null, passports: [] },
} });
const totals = t.careerTotals();
assert.equal(totals.badges, 1);
assert.equal(totals.seconds, 3720);
assert.equal(totals.walls.length, 1);
});
test('gig runner lifecycle: advance on ended, abandon on dead-queue stop', () => {
const w = load();
const t = w.__careerPassportTest;
let remaining = 1;
w.feedBack = { playQueue: { remaining: () => remaining, active: () => remaining > 0 } };
t.setGigRun({
songs: [{ filename: 'a', title: 'A' }, { filename: 'b', title: 'B' }],
venue_id: null, genre: 'Soul', genre_key: 'soul', instrument: 'guitar', idx: 0,
});
// First song ends, one remains → the strip advances, no completion.
t.onGigSongEnded();
assert.equal(t.getGigRun().idx, 1);
// Stop while the queue is still active (end-of-song teardown) → run survives.
t.onGigSongStop();
assert.notEqual(t.getGigRun(), null);
// User quits: queue cleared → stop with a dead queue abandons (no log).
remaining = 0;
t.onGigSongStop();
assert.equal(t.getGigRun(), null);
});
test('a gold upgrade notifies even when the bronze moment was already seen', () => {
// Bronze seen under the legacy un-suffixed id; the badge then turns gold.
const w = load({ 'feedBack-career-badges-seen': '{"guitar/blues":1}' });
const t = w.__careerPassportTest;
const view = { instruments: { guitar: { passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'gold' }] } } };
t.detectNewBadges(view);
assert.equal(w.notifications.length, 1);
assert.match(w.notifications[0].title, /Gold/);
// Same session: no duplicate.
t.detectNewBadges(view);
assert.equal(w.notifications.length, 1);
// Gold slam seen → fresh session stays silent.
t.markBadgeSeen('guitar', 'blues', 'gold');
const w2 = load({ 'feedBack-career-badges-seen': JSON.stringify(t.seenBadges()) });
w2.__careerPassportTest.detectNewBadges(view);
assert.equal(w2.notifications.length, 0);
});
test('a gold slam marks the bronze moment seen too — never both ceremonies', () => {
const w = load();
const t = w.__careerPassportTest;
t.markBadgeSeen('guitar', 'blues', 'gold');
const seen = JSON.parse(JSON.stringify(t.seenBadges()));
assert.equal(seen['guitar/blues@gold'], 1);
assert.equal(seen['guitar/blues'], 1);
// A later view where the badge reads 'earned' (e.g. gold state lost
// server-side) must not replay the bronze ceremony.
const view = { instruments: { guitar: { passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'earned' }] } } };
const w2 = load({ 'feedBack-career-badges-seen': JSON.stringify(seen) });
w2.__careerPassportTest.detectNewBadges(view);
assert.equal(w2.notifications.length, 0);
});
test('careerTotals counts gold badges on the wall', () => {
const t = load().__careerPassportTest;
t.setView({
config: { instruments: ['guitar'] },
instruments: { guitar: { committed_at: 1, gig_count: 0, passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'gold', seconds_total: 60 },
{ genre_key: 'funk', genre: 'Funk', badge: 'in_progress', seconds_total: 0 },
] } },
});
const totals = t.careerTotals();
assert.equal(totals.badges, 1);
assert.equal(totals.walls[0].earned[0].badge, 'gold');
});
+14 -10
View File
@@ -1908,17 +1908,21 @@ function createHighway() {
// never routable.
const isAudioUrl = msg.audio_url.startsWith('/audio/');
// "Full mix" covers BOTH single-mix pack shapes:
// - stem-less packs (original_audio: in the manifest,
// audio_url == original_audio_url), and
// - single-stem packs (stems: [full.ogg] only) — the server
// puts the full mix in the stems list, has_original_audio
// is false, and audio_url points at the one stem. With one
// stem there is no per-stem mix to preserve, so routing it
// natively loses nothing. Real multi-stem (>1) stays out
// until Phase 2.
// - single-stem packs (stems: [full.ogg] only) — the pack's
// one stem IS its mixdown, so the server leaves it in the
// stems list, has_full_mix is false, and audio_url points
// at that one stem; and
// - legacy stem-less packs, whose mixdown sits outside stems
// behind the deprecated original_audio: key, so has_stems
// is false and audio_url == full_mix_url.
// Either way there is one audible source and no per-stem mix
// to preserve, so routing it natively loses nothing. A pack
// that retains its `full` stem ALONGSIDE separated stems is
// multi-stem (has_full_mix && has_stems) and stays out until
// Phase 2 — routing it natively would drop the mixer.
const isFeedpakFullMix = !isAudioUrl
&& msg.audio_url.startsWith('/api/sloppak/')
&& ((!!msg.has_original_audio && !msg.has_stems)
&& ((!!msg.has_full_mix && !msg.has_stems)
|| (msg.stems || []).length === 1);
// Record the loaded song's audio so app.js can re-route it
// between the HTML5 and JUCE paths if the audio engine is
@@ -1943,7 +1947,7 @@ function createHighway() {
'isFeedpakFullMix=', isFeedpakFullMix,
'has_stems=', !!msg.has_stems,
'stems=', (msg.stems || []).length,
'has_original_audio=', !!msg.has_original_audio,
'has_full_mix=', !!msg.has_full_mix,
'format=', msg.format,
'alreadyLoaded=', alreadyLoaded,
'juceApi=', !!window.feedBackDesktop?.audio);
+28
View File
@@ -0,0 +1,28 @@
// Blob export helpers — the download idiom that used to be duplicated in
// settings-io.js and diagnostics-export.js, plus image-to-clipboard for
// shareable cards/posters. A LEAF module: imports nothing. Classic-script
// plugins reach it via dynamic import('/static/js/blob-io.js').
export function downloadBlob(blob, filename) {
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
}
// Copy an image blob to the system clipboard. Returns true on success, false
// when the Clipboard API is unavailable or refuses (insecure context, no user
// gesture, permission denied) — callers fall back to downloadBlob and say so.
export async function copyImageBlob(blob) {
try {
if (!navigator.clipboard || typeof ClipboardItem === 'undefined') return false;
await navigator.clipboard.write([new ClipboardItem({ [blob.type || 'image/png']: blob })]);
return true;
} catch (_) {
return false;
}
}
+3 -8
View File
@@ -21,6 +21,8 @@
// redact toggles.
// 3. Stream the returned zip to disk.
import { downloadBlob } from './blob-io.js';
function _diagIncludeFromUI() {
const v = (id) => document.getElementById(id)?.checked !== false;
return {
@@ -265,14 +267,7 @@ export async function exportDiagnostics() {
}
try {
const blob = await resp.blob();
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
downloadBlob(blob, filename);
status.textContent = `Exported ${filename}`;
} catch (e) {
status.textContent = `Export failed during download: ${e.message}`;
+4 -9
View File
@@ -1,6 +1,6 @@
// Settings backup — the export / import bundle.
//
// Carved verbatim out of static/app.js (R3a). A LEAF module: imports nothing.
// Carved verbatim out of static/app.js (R3a). Imports only the blob-io leaf.
//
// Two entry points, both inline handlers on the Settings screen, so app.js keeps
// re-exposing them on window. The import is two-phase (server first, atomic; then
@@ -29,6 +29,8 @@
// phase 2; the localStorage side is best-effort merge after server
// success. Failures are reported, never silenced.
import { downloadBlob } from './blob-io.js';
export async function exportSettings() {
const status = document.getElementById('backup-status');
status.textContent = 'Exporting...';
@@ -66,14 +68,7 @@ export async function exportSettings() {
if (match) filename = match[1];
}
const blob = new Blob([JSON.stringify(bundle, null, 2)], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
downloadBlob(blob, filename);
status.textContent = `Exported ${filename}`;
} catch (e) {
status.textContent = `Export failed: ${e.message}`;
+6 -1
View File
@@ -208,12 +208,17 @@
'</div></div></div>' +
continueCard +
'</div>' +
// Stats row
// Stats row. The third slot belongs to the career plugin (it
// replaces the slot's content on v3:dashboard-rendered); the
// plugin-count stat is the built-in fallback when career is
// absent or has no state yet.
'<div class="grid md:grid-cols-3 gap-6 mt-6">' +
audioRoutingCard() +
statCard(String(songCount), 'songs', 'text-fb-gold') +
'<div id="v3-dash-career-slot" class="grid">' +
statCard(String(pluginCount), 'active', 'text-fb-good') +
'</div>' +
'</div>' +
recentSection +
'</div>';
+4
View File
@@ -191,6 +191,10 @@
'<div class="space-y-6">' +
headerCard +
bestsCard +
// Passport wall — rendered by the career plugin on
// v3:profile-rendered (absent-not-empty: nothing shows until a
// passport exists).
'<div id="v3-profile-passports-mount"></div>' +
// Feats of Power trophy shelf — rendered by the achievements plugin
// (earned Feats only; hidden-until-earned, so empty when none).
'<div id="v3-profile-feats-slot"></div>' +
+7 -5
View File
@@ -26,7 +26,7 @@ class FakeMetaDb:
self.conn.execute(
"""CREATE TABLE song_stats (
filename TEXT, arrangement TEXT, best_accuracy REAL,
last_played_at TEXT,
last_accuracy REAL, last_played_at TEXT,
seconds_total REAL NOT NULL DEFAULT 0
)"""
)
@@ -38,10 +38,12 @@ class FakeMetaDb:
)
def add(self, filename, arrangement, best_accuracy, in_library=True,
genre="", arrangements=None, last_played_at=None, seconds_total=0):
self.conn.execute("INSERT INTO song_stats VALUES (?, ?, ?, ?, ?)",
(filename, arrangement, best_accuracy, last_played_at,
seconds_total))
genre="", arrangements=None, last_played_at=None, seconds_total=0,
last_accuracy=None):
self.conn.execute("INSERT INTO song_stats VALUES (?, ?, ?, ?, ?, ?)",
(filename, arrangement, best_accuracy,
last_accuracy if last_accuracy is not None else best_accuracy,
last_played_at, seconds_total))
if in_library:
self.conn.execute(
"INSERT INTO songs SELECT ?, ?, ?, ?, ? WHERE NOT EXISTS "
+237
View File
@@ -208,3 +208,240 @@ def test_drill_state_merge_is_gained_only(client, meta_db):
p = _passport(client)
assert p["drills"]["cleared"] == ["blues_shuffle"]
assert p["badge"] == "earned"
def test_genre_families_inherit_drills(client, meta_db):
# 'death metal' has no exact entry — it inherits the metal family's drill.
for i in range(5):
meta_db.add(f"dm{i}.feedpak", 0, 0.9, genre="Death Metal", arrangements=LEAD)
_open(client, "guitar", "Death Metal")
p = _passport(client, "guitar", "death metal")
assert p["drills"]["required"] == ["melodic_metal_gallop"]
assert p["badge"] == "in_progress"
# 'metalcore' (single word) matches by substring, no alias needed.
_open(client, "guitar", "Metalcore")
assert _passport(client, "guitar", "metalcore")["drills"]["required"] == \
["melodic_metal_gallop"]
# 'blues rock' resolves by family LIST ORDER: blues comes before rock.
_open(client, "guitar", "Blues Rock")
assert _passport(client, "guitar", "blues rock")["drills"]["required"] == \
["blues_shuffle"]
# A genre outside every family stays songs-only.
_open(client, "guitar", "Reggae")
assert _passport(client, "guitar", "reggae")["drills"]["required"] == []
# Exact per-genre entries still beat the family (the shipped 'metal' entry
# IS the exact entry for genre key 'metal').
_open(client, "guitar", "Metal")
assert _passport(client, "guitar", "metal")["drills"]["required"] == \
["melodic_metal_gallop"]
def test_family_drills_stay_per_instrument(client, meta_db):
# Family inheritance must not leak guitar drills onto other instruments.
keys_arr = [{"type": "lead", "name": "Keys"}]
for i in range(5):
meta_db.add(f"kdm{i}.feedpak", 0, 0.9, genre="Death Metal", arrangements=keys_arr)
_open(client, "keys", "Death Metal")
p = _passport(client, "keys", "death metal")
assert p["drills"]["required"] == []
assert p["badge"] == "earned"
def test_nearest_invitations_order_and_exclusions(client, meta_db):
# Non-qualifying songs sorted by distance to the QUALIFYING bar;
# qualifying songs never appear; capped at 3.
meta_db.add("q.feedpak", 0, 0.80, genre="Soul", arrangements=LEAD) # qualifies
meta_db.add("close.feedpak", 0, 0.74, genre="Soul", arrangements=LEAD) # 1% to 2★
meta_db.add("mid.feedpak", 0, 0.70, genre="Soul", arrangements=LEAD) # 5% to 2★
meta_db.add("far.feedpak", 0, 0.30, genre="Soul", arrangements=LEAD) # 30% to 1★
meta_db.add("far2.feedpak", 0, 0.25, genre="Soul", arrangements=LEAD)
_open(client, "guitar", "Soul")
p = _passport(client, "guitar", "soul")
names = [s["filename"] for s in p["nearest"]]
assert names == ["close.feedpak", "mid.feedpak", "far.feedpak"]
assert all(s["next_star_at"] is not None for s in p["nearest"])
assert "q.feedpak" not in names
def test_nearest_targets_the_qualifying_bar_not_next_star(client, meta_db):
# A 0★ song 1% from its NEXT star is farther from the ★★ badge bar than
# a 1★ song 5% from it — nearest must rank by the badge bar.
meta_db.add("one_star.feedpak", 0, 0.70, genre="Soul", arrangements=LEAD) # 5% to bar
meta_db.add("zero_star.feedpak", 0, 0.59, genre="Soul", arrangements=LEAD) # 1% to next ★, 16% to bar
_open(client, "guitar", "Soul")
p = _passport(client, "guitar", "soul")
assert [s["filename"] for s in p["nearest"]] == ["one_star.feedpak", "zero_star.feedpak"]
assert all(s["bar_at"] == 0.75 for s in p["nearest"])
# ── Gigs ──────────────────────────────────────────────────────────────────────
def test_gig_propose_mixes_owned_and_stakes(client, meta_db):
for i in range(4):
meta_db.add(f"own{i}.feedpak", 0, 0.85, genre="Soul", arrangements=LEAD)
meta_db.add("stake.feedpak", 0, 0.70, genre="Soul", arrangements=LEAD)
meta_db.add_song_only("fresh.feedpak", genre="Soul")
res = client.post("/api/plugins/career/gigs/propose",
json={"instrument": "guitar", "genre": "Soul", "size": 4})
assert res.status_code == 200
gig = res.json()
files = [s["filename"] for s in gig["songs"]]
assert len(files) == 4
assert "stake.feedpak" in files # a near-bar song gives the set stakes
assert gig["venue_id"] == "bar" # 9 stars < 50: the dive bar
# A young passport (nothing played) still gets a playable set from the
# library's unplayed genre songs.
res2 = client.post("/api/plugins/career/gigs/propose",
json={"instrument": "guitar", "genre": "Ska"})
assert res2.status_code == 404 # no ska in the library at all
meta_db.add_song_only("ska1.feedpak", genre="Ska")
res3 = client.post("/api/plugins/career/gigs/propose",
json={"instrument": "guitar", "genre": "Ska"})
assert [s["filename"] for s in res3.json()["songs"]] == ["ska1.feedpak"]
def test_gig_log_computes_encore_and_surfaces_in_passports(client, meta_db):
for i in range(2):
meta_db.add(f"s{i}.feedpak", 0, 0.9, genre="Soul", arrangements=LEAD,
last_accuracy=0.9)
_open(client, "guitar", "Soul")
res = client.post("/api/plugins/career/gigs", json={
"instrument": "guitar", "genre": "Soul", "venue_id": "bar",
"songs": ["s0.feedpak", "s1.feedpak"]})
assert res.status_code == 200
gig = res.json()["gig"]
assert gig["encore"] is True # avg 0.9 ≥ 0.75
assert gig["songs"][0]["accuracy"] == 0.9
view = client.get("/api/plugins/career/passports").json()
assert view["instruments"]["guitar"]["gig_count"] == 1
p = _passport(client, "guitar", "soul")
assert len(p["gigs"]) == 1 and p["gigs"][0]["encore"] is True
def test_gig_log_validation_and_no_fail_state(client):
# Unknown venue / bad songs shapes are rejected; nothing is ever logged
# as a failed gig — the endpoint only appends completed sets.
assert client.post("/api/plugins/career/gigs", json={
"instrument": "guitar", "genre": "Soul", "venue_id": "nope",
"songs": ["x"]}).status_code == 400
assert client.post("/api/plugins/career/gigs", json={
"instrument": "guitar", "genre": "Soul", "songs": []}).status_code == 400
assert client.post("/api/plugins/career/gigs", json={
"instrument": "guitar", "genre": "Soul",
"songs": ["f"] * 9}).status_code == 400
def test_gig_accuracy_reads_newest_row_and_encore_needs_full_set(client, meta_db):
# Newest row wins: a stale higher accuracy on another arrangement must
# not inflate the gig log.
meta_db.add("dual.feedpak", 1, 0.95, genre="Soul", arrangements=BASS,
last_accuracy=0.95, last_played_at="2026-06-01T00:00:00")
meta_db.add("dual.feedpak", 0, 0.60, genre="Soul", arrangements=LEAD,
last_accuracy=0.60, last_played_at="2026-07-14T00:00:00")
res = client.post("/api/plugins/career/gigs", json={
"instrument": "guitar", "genre": "Soul", "songs": ["dual.feedpak"]})
assert res.json()["gig"]["songs"][0]["accuracy"] == 0.6
# A set with an unscored song never earns the encore off one good song.
meta_db.add("scored.feedpak", 0, 0.9, genre="Soul", arrangements=LEAD,
last_accuracy=0.9, last_played_at="2026-07-14T00:01:00")
res2 = client.post("/api/plugins/career/gigs", json={
"instrument": "guitar", "genre": "Soul",
"songs": ["scored.feedpak", "ghost.feedpak"]})
assert res2.json()["gig"]["encore"] is False
def test_gig_propose_backfills_from_surplus_qualifying(client, meta_db):
# Mature passport: plenty of qualifying songs, nothing near the bar,
# nothing unplayed — the set still fills to size.
for i in range(8):
meta_db.add(f"own{i}.feedpak", 0, 0.9, genre="Ska", arrangements=LEAD)
res = client.post("/api/plugins/career/gigs/propose",
json={"instrument": "guitar", "genre": "Ska", "size": 5})
assert len(res.json()["songs"]) == 5
def test_gig_propose_backfill_offset_survives_stakes(client, meta_db):
# 4 qualifying + 1 near-bar stake, size 5: the stake must not shift the
# qualifying backfill window past eligible songs.
for i in range(4):
meta_db.add(f"q{i}.feedpak", 0, 0.9, genre="Reggae", arrangements=LEAD)
meta_db.add("near.feedpak", 0, 0.7, genre="Reggae", arrangements=LEAD)
res = client.post("/api/plugins/career/gigs/propose",
json={"instrument": "guitar", "genre": "Reggae", "size": 5})
files = [s["filename"] for s in res.json()["songs"]]
assert len(files) == 5 and len(set(files)) == 5
assert "near.feedpak" in files
# ── Gold rung ─────────────────────────────────────────────────────────────────
def test_gold_upgrades_bronze_via_family_style_artifact(client, meta_db):
# Bronze earned on 'death metal' (family: metal); a metal gold artifact
# from the jam verifier upgrades it — bronze-only stays 'earned' elsewhere.
for i in range(5):
meta_db.add(f"dm{i}.feedpak", 0, 0.9, genre="Death Metal", arrangements=LEAD)
career_routes._state["passports_content"]["genres"]["metal"] = {} # no drill gate for this test
_open(client, "guitar", "Death Metal")
client.post("/api/plugins/career/drill-state", json={"byNode": {}})
assert _passport(client, "guitar", "death metal")["badge"] == "earned"
client.post("/api/plugins/career/drill-state", json={
"byNode": {}, "goldImprov": {"metal": {"at": 1, "verifier": "comb", "inKeyPct": 0.9}}})
assert _passport(client, "guitar", "death metal")["badge"] == "gold"
def test_gold_without_bronze_stays_in_progress(client, meta_db):
meta_db.add("one.feedpak", 0, 0.9, genre="Soul", arrangements=LEAD)
_open(client, "guitar", "Soul")
client.post("/api/plugins/career/drill-state", json={
"byNode": {}, "goldImprov": {"soul": {"at": 1, "verifier": "comb"}}})
assert _passport(client, "guitar", "soul")["badge"] == "in_progress"
def test_gold_merge_is_gained_only(client, meta_db):
for i in range(5):
meta_db.add(f"s{i}.feedpak", 0, 0.9, genre="Soul", arrangements=LEAD)
_open(client, "guitar", "Soul")
client.post("/api/plugins/career/drill-state", json={
"byNode": {}, "goldImprov": {"soul": {"at": 1, "verifier": "comb"}}})
assert _passport(client, "guitar", "soul")["badge"] == "gold"
# A stale relay without the artifact never un-mints.
client.post("/api/plugins/career/drill-state", json={"byNode": {}})
assert _passport(client, "guitar", "soul")["badge"] == "gold"
# And a different artifact for the same style never overwrites the first —
# asserted against the PERSISTED snapshot (the view doesn't expose
# artifact contents), so a last-write-wins regression can't stay green.
client.post("/api/plugins/career/drill-state", json={
"byNode": {}, "goldImprov": {"soul": {"at": 999, "verifier": "yin"}}})
_, _, gold = career_routes._drill_by_node()
assert gold["soul"] == {"at": 1, "verifier": "comb"}
def test_gold_matches_raw_style_id_through_family(client, meta_db):
# Virtuoso mints under raw STYLE_PALETTES ids ('punk', not 'rock'): a
# 'punk rock' passport (family rock) must go gold from a 'punk' artifact.
for i in range(5):
meta_db.add(f"pk{i}.feedpak", 0, 0.9, genre="Punk Rock", arrangements=LEAD)
career_routes._state["passports_content"]["genres"]["rock"] = {} # no drill gate
_open(client, "guitar", "Punk Rock")
client.post("/api/plugins/career/drill-state", json={
"byNode": {}, "goldImprov": {"punk": {"at": 1, "verifier": "comb"}}})
assert _passport(client, "guitar", "punk rock")["badge"] == "gold"
def test_gold_intake_rejects_junk(client, meta_db):
# A non-dict goldImprov is a relay bug: loud 400, never a silent drop.
res = client.post("/api/plugins/career/drill-state",
json={"byNode": {}, "goldImprov": ["metal"]})
assert res.status_code == 400
# Evidence-free artifacts (no verifier) never mint.
for i in range(5):
meta_db.add(f"j{i}.feedpak", 0, 0.9, genre="Soul", arrangements=LEAD)
_open(client, "guitar", "Soul")
client.post("/api/plugins/career/drill-state", json={
"byNode": {}, "goldImprov": {"soul": {}}})
assert _passport(client, "guitar", "soul")["badge"] == "earned"
# An oversized goldImprov is bounded BEFORE the merge, like byNode.
blob = {f"s{i}": {"verifier": "comb", "pad": "x" * 4096} for i in range(200)}
res = client.post("/api/plugins/career/drill-state",
json={"byNode": {}, "goldImprov": blob})
assert res.status_code == 413
+48
View File
@@ -273,3 +273,51 @@ def test_title_keyset_paging_is_complete_with_overrides(client, server):
if not cursor:
break
assert sorted(seen) == ["a.archive", "b.archive", "c.archive"] # each exactly once
def test_enrichment_genre_fallback_precedence(client, server):
# Precedence: override → pack genre → MusicBrainz enrichment (matched only).
_put(server, "a.archive", title="A", genre="Rock") # pack wins over enrichment
_put(server, "b.archive", title="B", genre="") # falls back to enrichment
_put(server, "c.archive", title="C", genre="") # override beats enrichment
_put(server, "d.archive", title="D", genre="") # unmatched candidate: ignored
ins = "INSERT INTO song_enrichment (filename, match_state, genres) VALUES (?, ?, ?)"
server.meta_db.conn.execute(ins, ("a.archive", "matched", '["metal"]'))
server.meta_db.conn.execute(ins, ("b.archive", "matched", '["progressive rock", "rock"]'))
server.meta_db.conn.execute(ins, ("c.archive", "matched", '["jazz"]'))
server.meta_db.conn.execute(ins, ("d.archive", "review", '["country"]'))
_put(server, "e.archive", title="E", genre="") # manual pin is trusted too
server.meta_db.conn.execute(ins, ("e.archive", "manual", '["ska"]'))
server.meta_db.conn.commit()
server.meta_db.set_song_override("c.archive", "genre", value="City Pop")
genres = client.get("/api/library/genres").json()["genres"]
assert "Rock" in genres # pack value kept for a
assert "metal" not in genres # enrichment never overrides a pack genre
assert "progressive rock" in genres # b: enrichment primary ([0]) surfaces
assert "City Pop" in genres and "jazz" not in genres # override beats enrichment
assert "country" not in genres # review/failed candidates never leak
assert "ska" in genres # user-pinned (manual) matches count
# Filtering by the enriched genre finds the song.
r = client.get("/api/library", params={"genre": "progressive rock"}).json()
assert [s["filename"] for s in r["songs"]] == ["b.archive"]
def test_no_enrichment_and_no_overrides_uses_plain_column(server):
_put(server, "a.archive", title="A", genre="Rock")
assert server.meta_db._effective_genre_expr() == "genre"
def test_overrides_without_enrichment_table_stay_safe(server):
# A stand-in scenario: overrides exist but song_enrichment is gone — the
# expression must not reference the missing table.
server.meta_db.conn.execute("DROP TABLE song_enrichment")
_put(server, "a.archive", title="A", genre="")
server.meta_db.set_song_override("a.archive", "genre", value="City Pop")
expr = server.meta_db._effective_genre_expr()
assert "song_enrichment" not in expr
# And it still evaluates: the override surfaces through the facet query.
row = server.meta_db.conn.execute(
f"SELECT {expr} FROM songs WHERE filename = 'a.archive'").fetchone()
assert row[0] == "City Pop"
+201
View File
@@ -0,0 +1,201 @@
"""The remote transcription REQUEST — the thing that was never tested and never worked.
`transcribe_vocals_remote()` POSTed the vocal stem to `/align`. That endpoint is *forced
alignment*: "here are the lyrics, tell me when each word is sung". Its `text` field is required,
and we have no lyrics transcribing them is the entire point. So the server rejected every
request with a 422 from FastAPI's validation layer, before its handler ever ran, and remote
transcription had never worked for anybody (feedBack-plugin-stem-splitter#17).
Nothing caught it because every test of this module tested the *mapper* `_whisperx_to_sloppak`,
fed a hand-written dict. The mapper was always fine. The request was never exercised, and the
request was the bug.
So these tests assert the request: which endpoint, and how `language` is carried. Both are
invisible to a mapper test, and both are wrong in ways that fail quietly rather than loudly.
"""
from pathlib import Path
from unittest import mock
import pytest
from lyrics_transcribe import transcribe_vocals_remote
_ALIGNED = {
"segments": [{
"start": 1.0, "end": 2.0, "text": "hello world",
"words": [
{"word": "hello", "start": 1.0, "end": 1.4, "score": 0.9},
{"word": "world", "start": 1.5, "end": 2.0, "score": 0.9},
],
}]
}
class _Resp:
def __init__(self, status=200, payload=None, text=""):
self.status_code = status
self._payload = payload if payload is not None else _ALIGNED
self.text = text
def json(self):
return self._payload
@pytest.fixture
def vocals(tmp_path: Path) -> Path:
p = tmp_path / "vocals.ogg"
p.write_bytes(b"not really ogg, we never decode it here")
return p
def _post_call(vocals: Path, resp: _Resp, **kw):
with mock.patch("requests.post", return_value=resp) as post:
out = transcribe_vocals_remote(vocals, "http://server:7865", **kw)
return post.call_args, out
def test_it_posts_to_transcribe_not_align(vocals):
"""THE regression. /align requires `text`; we have none, so it 422s every time."""
call, out = _post_call(vocals, _Resp())
url = call.args[0]
assert url.endswith("/transcribe"), (
f"posted to {url!r} — /align is forced alignment and its `text` field is required, so "
f"this request is rejected with a 422 before the server's handler ever runs"
)
assert "/align" not in url
assert out, "a successful transcription must return syllables"
def test_the_language_hint_is_a_form_field_not_a_query_param(vocals):
"""The server reads `language` with Form(""). Sent as a query param it is silently ignored —
so an explicit hint does nothing, Whisper's auto-detection quietly decides instead, and the
wrong wav2vec2 aligner gets loaded. It "works", it's just wrong: the failure mode that hides
for months."""
call, _ = _post_call(vocals, _Resp(), language="es")
assert (call.kwargs.get("data") or {}).get("language") == "es", (
"the language hint must ride in the form body — the server reads Form('language'), and "
"a query param is dropped without a word"
)
assert "language" not in (call.kwargs.get("params") or {})
def test_no_language_sends_no_hint(vocals):
# Absent is not the empty string: "" would pin detection to a language named "".
call, _ = _post_call(vocals, _Resp())
assert not (call.kwargs.get("data") or {})
def test_the_file_is_sent_as_a_multipart_upload(vocals):
call, _ = _post_call(vocals, _Resp())
files = call.kwargs.get("files") or {}
assert "file" in files, "the server reads File('file')"
assert files["file"][0] == "vocals.ogg"
def test_an_api_key_is_sent_as_a_bearer_token(vocals):
call, _ = _post_call(vocals, _Resp(), api_key="secret")
assert (call.kwargs.get("headers") or {})["Authorization"] == "Bearer secret"
def test_an_instrumental_is_an_answer_not_a_crash(vocals):
# The server returns 200 + no segments for a stem with no singing in it. That is a valid
# answer ("this song has no vocals"), and it must not read as a failure.
_, out = _post_call(vocals, _Resp(payload={"segments": [], "language": "en"}))
assert out == []
def test_a_server_error_surfaces_the_whole_body(vocals):
"""The error body IS the diagnosis. A 422's JSON names the field it rejected; a 500's
traceback answers on its last line. The old 300-char cap decapitated both which is how
this bug stayed invisible: the message explaining it was inside the part that got cut."""
tb = "Traceback (most recent call last):\n" + (" File x, line 1\n" * 40) + \
"RuntimeError: CUDA out of memory"
assert len(tb) > 300 and "CUDA out of memory" not in tb[:300]
with pytest.raises(RuntimeError) as exc:
_post_call(vocals, _Resp(status=500, text=tb))
assert "CUDA out of memory" in str(exc.value)
def test_truncation_keeps_the_exception_line_not_just_the_header():
"""A traceback's ANSWER is its last line. Head-only truncation throws it away.
This is the same mistake as the 300-char cap, one level up: cutting off precisely the part
the function exists to preserve. A 4000-char window that contains "Traceback (most recent
call last)" and none of the exception is a window onto nothing."""
from lyrics_transcribe import _MAX_ERR_BODY, _err_body
frames = "".join(f' File "/app/server.py", line {i}, in run\n step()\n'
for i in range(2000)) # far over the cap on its own
tb = "Traceback (most recent call last):\n" + frames + \
"RuntimeError: CUDA out of memory. Tried to allocate 2.20 GiB"
body = _err_body(_Resp(text=tb))
assert len(body) <= _MAX_ERR_BODY
assert "CUDA out of memory" in body, (
"the exception line is the diagnosis — a truncation that drops it keeps the part that "
"says work was happening and discards the part that says what went wrong"
)
assert "Traceback (most recent call last)" in body, "the head is context worth keeping too"
assert "truncated" in body
def test_the_cap_is_a_bound_not_a_suggestion():
"""The truncation marker must fit INSIDE _MAX_ERR_BODY, not be appended past it.
Otherwise the cap is advisory, and the callers who trust it a log line, a job record
persisted to disk and re-read on every load are the ones that get surprised."""
from lyrics_transcribe import _MAX_ERR_BODY, _err_body
body = _err_body(_Resp(text="x" * 500_000))
assert len(body) <= _MAX_ERR_BODY, (
f"body is {len(body)} chars, over the {_MAX_ERR_BODY} cap it claims to enforce"
)
assert "truncated" in body and "500000" in body
def test_trailing_whitespace_is_not_content():
# A 300-char JSON body followed by 3900 blanks is not a long body, and cutting real content
# to make room for whitespace would be a silly way to lose the diagnosis.
from lyrics_transcribe import _err_body
payload = '{"detail":"nope"}'
assert _err_body(_Resp(text=payload + " " * 8000)) == payload
def test_a_404_explains_that_the_server_is_too_old(vocals):
"""A bare "404" sends someone hunting for a typo in their URL. The real answer is that their
server predates the endpoint, and only we can know that."""
with pytest.raises(RuntimeError) as exc:
_post_call(vocals, _Resp(status=404, text='{"detail":"Not Found"}'))
msg = str(exc.value)
assert "404" in msg
assert "/transcribe" in msg
assert "predates" in msg or "Update the server" in msg
class TestEverythingFailsAsRuntimeError:
"""The docstring promises one failure mode: RuntimeError. The caller
(`_maybe_transcribe_lyrics`) catches exactly that so one song's failed lyrics don't take down
the batch around it. A transport error escaping as requests.RequestException walks straight
past that handler turning "this song's lyrics failed" into "the whole batch died"."""
def test_a_connection_failure(self, vocals):
import requests
with mock.patch("requests.post",
side_effect=requests.ConnectionError("name resolution failed")):
with pytest.raises(RuntimeError, match="could not reach"):
transcribe_vocals_remote(vocals, "http://nope:7865")
def test_a_timeout(self, vocals):
import requests
with mock.patch("requests.post", side_effect=requests.Timeout("timed out")):
with pytest.raises(RuntimeError, match="could not reach"):
transcribe_vocals_remote(vocals, "http://server:7865")
def test_an_unreadable_stem(self, tmp_path):
missing = tmp_path / "gone.ogg" # never created
with pytest.raises(RuntimeError, match="could not read"):
transcribe_vocals_remote(missing, "http://server:7865")
+429
View File
@@ -0,0 +1,429 @@
"""tools/migrate_full_mix_stem.py — packs off the deprecated `original_audio:` key.
The migration moves real audio inside tens of thousands of archives, so the
interesting cases are the ones where it must NOT act: a pack it would corrupt, a
pack it has already done, a pack whose mixdown isn't where the key claims.
"""
from __future__ import annotations
import importlib.util
import zipfile
from pathlib import Path
import pytest
import yaml
_SPEC = importlib.util.spec_from_file_location(
"migrate_full_mix_stem",
Path(__file__).resolve().parent.parent / "tools" / "migrate_full_mix_stem.py",
)
mig = importlib.util.module_from_spec(_SPEC)
_SPEC.loader.exec_module(mig)
def _manifest(**extra) -> dict:
m = {
"feedpak_version": "1.13.0",
"title": "T",
"artist": "A",
"duration": 1.0,
"arrangements": [{"id": "lead", "file": "arrangements/lead.json"}],
"stems": [
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
{"id": "drums", "file": "stems/drums.ogg", "default": "on"},
],
"original_audio": "original/full.ogg",
}
m.update(extra)
return m
def _write_pack(path: Path, manifest: dict, files: dict[str, bytes] | None = None) -> Path:
files = files or {
"original/full.ogg": b"MIXDOWN",
"stems/guitar.ogg": b"g",
"stems/drums.ogg": b"d",
"arrangements/lead.json": b"{}",
}
with zipfile.ZipFile(path, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("manifest.yaml", yaml.safe_dump(manifest, sort_keys=False))
for name, data in files.items():
zf.writestr(name, data)
return path
def _read(path: Path) -> tuple[dict, set[str]]:
with zipfile.ZipFile(path) as zf:
return yaml.safe_load(zf.read("manifest.yaml")), set(zf.namelist())
# ── plan_manifest: the decisions, without the archives ──────────────────────
def test_plan_adds_the_full_stem_and_drops_the_key():
new, move = mig.plan_manifest(_manifest())
assert move == "original/full.ogg"
assert "original_audio" not in new
assert new["stems"][0] == {
"id": "full",
"file": "stems/full.ogg",
"default": "off",
}
# The separated stems survive, in order, untouched.
assert [s["id"] for s in new["stems"]] == ["full", "guitar", "drums"]
assert new["feedpak_version"] == "1.15.0"
def test_plan_marks_the_retained_mixdown_default_off():
"""The one line that keeps a pre-1.15.0 reader from doubling the song: a
reader that sums every stem still won't play `full` on open if it honours
`default`, which has been normative since 1.0.0."""
new, _ = mig.plan_manifest(_manifest())
assert new["stems"][0]["default"] == "off"
def test_plan_marks_a_sole_mixdown_default_on():
"""With no separated stems the mixdown IS the audio — off would mute the pack."""
new, _ = mig.plan_manifest(_manifest(stems=[]))
assert new["stems"] == [{"id": "full", "file": "stems/full.ogg", "default": "on"}]
def test_plan_preserves_unknown_keys_verbatim():
"""Spec §3: a writer that re-emits a pack SHOULD preserve unknown keys."""
new, _ = mig.plan_manifest(_manifest(source_tool="ExampleTool v1.2.3", rigs="rigs.json"))
assert new["source_tool"] == "ExampleTool v1.2.3"
assert new["rigs"] == "rigs.json"
def test_plan_skips_an_already_migrated_pack():
m = _manifest(
stems=[{"id": "full", "file": "stems/full.ogg", "default": "off"}],
)
del m["original_audio"]
with pytest.raises(mig.Skip):
mig.plan_manifest(m)
def test_plan_skips_a_pack_that_never_had_the_key():
m = _manifest()
del m["original_audio"]
with pytest.raises(mig.Skip):
mig.plan_manifest(m)
def test_plan_drops_a_stale_key_without_moving_anything():
"""Mixdown already a stem, dead key lingering beside it."""
new, move = mig.plan_manifest(
_manifest(stems=[{"id": "full", "file": "stems/full.ogg", "default": "off"}])
)
assert move == ""
assert "original_audio" not in new
assert [s["id"] for s in new["stems"]] == ["full"]
def test_plan_forces_an_existing_full_stem_off_beside_instrument_stems():
"""Dropping the stale key is not enough if the mixdown it duplicated is left
ENABLED: a reader that honours `default` would then play the whole song on top
of the stems on open. The migration must not hand back a pack in the exact
state it exists to remove."""
new, move = mig.plan_manifest(
_manifest(
stems=[
{"id": "full", "file": "stems/full.ogg", "default": "on"},
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
]
)
)
assert move == ""
assert new["stems"][0] == {"id": "full", "file": "stems/full.ogg", "default": "off"}
assert new["stems"][1]["default"] == "on" # instruments untouched
def test_plan_leaves_a_sole_full_stem_enabled_when_dropping_a_stale_key():
"""No instruments beside it — the mixdown IS the audio. Forcing it off here
would mute the pack."""
new, _ = mig.plan_manifest(
_manifest(stems=[{"id": "full", "file": "stems/full.ogg", "default": "on"}])
)
assert new["stems"] == [{"id": "full", "file": "stems/full.ogg", "default": "on"}]
def test_plan_needs_no_move_when_the_key_already_points_at_the_canonical_path():
new, move = mig.plan_manifest(_manifest(original_audio="stems/full.ogg"))
assert move == ""
assert new["stems"][0]["file"] == "stems/full.ogg"
# ── migrate_zip: the archive rewrite ────────────────────────────────────────
def test_migrate_moves_the_audio_and_rewrites_the_manifest(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
manifest, names = _read(pak)
assert "original/full.ogg" not in names # the invented directory is gone
assert "stems/full.ogg" in names # audio lives where the format says
assert "original_audio" not in manifest
assert manifest["stems"][0]["id"] == "full"
assert mig.verify_zip(pak) == "ok"
def test_migrate_preserves_the_mixdown_bytes(tmp_path: Path):
"""It is a rename, not a re-encode. Losing a byte here loses the master audio."""
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
mig.migrate_zip(pak, dry_run=False)
with zipfile.ZipFile(pak) as zf:
assert zf.read("stems/full.ogg") == b"MIXDOWN"
assert zf.read("stems/guitar.ogg") == b"g"
def test_migrate_is_idempotent(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=False) == "skip"
assert pak.read_bytes() == before # a re-run touches nothing
def test_dry_run_changes_nothing(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=True) == "would-migrate"
assert pak.read_bytes() == before
def test_migrate_refuses_when_the_mixdown_is_absent(tmp_path: Path):
"""The key points at audio the archive doesn't contain. Fabricating a stem
entry for a missing file would break every reader refuse, don't guess."""
pak = _write_pack(
tmp_path / "song.feedpak",
_manifest(),
files={"stems/guitar.ogg": b"g", "arrangements/lead.json": b"{}"},
)
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=False) == "missing-audio"
assert pak.read_bytes() == before
def test_migrate_refuses_when_the_target_path_is_taken(tmp_path: Path):
"""A `stems/full.ogg` that is NOT the mixdown already occupies the target.
Overwriting it would destroy a stem."""
pak = _write_pack(
tmp_path / "song.feedpak",
_manifest(),
files={
"original/full.ogg": b"MIXDOWN",
"stems/full.ogg": b"SOMETHING-ELSE",
"arrangements/lead.json": b"{}",
},
)
assert mig.migrate_zip(pak, dry_run=False) == "target-occupied"
with zipfile.ZipFile(pak) as zf:
assert zf.read("stems/full.ogg") == b"SOMETHING-ELSE"
def test_migrate_drops_a_stale_key_beside_a_non_canonical_full_stem(tmp_path: Path):
"""The mixdown is already a stem, but at a path of the pack's own choosing —
which is legal (§2.2: readers resolve through the manifest, never by
filename). Only the dead key needs removing. Demanding `stems/full.ogg` here
would reject a perfectly valid pack as `missing-audio`."""
m = _manifest(stems=[{"id": "full", "file": "audio/mixdown.ogg", "default": "off"}])
pak = _write_pack(
tmp_path / "song.feedpak",
m,
files={"audio/mixdown.ogg": b"MIXDOWN", "arrangements/lead.json": b"{}"},
)
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
manifest, names = _read(pak)
assert "original_audio" not in manifest
assert manifest["stems"] == [
{"id": "full", "file": "audio/mixdown.ogg", "default": "off"}
]
assert "audio/mixdown.ogg" in names # the audio never moved
assert mig.verify_zip(pak) == "ok"
# ── verify_zip ──────────────────────────────────────────────────────────────
def test_verify_rejects_a_retained_mixdown_that_plays_on_open(tmp_path: Path):
"""The hazard the migration must never create: `full` alongside instrument
stems AND default-on means a summing reader plays the whole song twice."""
m = _manifest(
stems=[
{"id": "full", "file": "stems/full.ogg", "default": "on"},
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
]
)
del m["original_audio"]
pak = _write_pack(
tmp_path / "song.feedpak",
m,
files={"stems/full.ogg": b"M", "stems/guitar.ogg": b"g"},
)
assert mig.verify_zip(pak) == "full-stem-default-on"
@pytest.mark.parametrize(
"default, expected",
[
({"default": "off"}, "ok"), # the one safe, canonical shape
({"default": "OFF"}, "ok"), # case-insensitive
({"default": " off "}, "ok"), # surrounding whitespace tolerated
({}, "full-stem-default-not-off"), # MISSING — core defaults to True (ON)
({"default": ""}, "full-stem-default-not-off"), # empty → ON in core
({"default": False}, "full-stem-default-not-off"), # boolean, not the string
({"default": True}, "full-stem-default-on"), # boolean truthy → plays
({"default": "false"}, "full-stem-default-not-off"), # off-ish but non-canonical
({"default": "0"}, "full-stem-default-not-off"),
({"default": "no"}, "full-stem-default-not-off"),
({"default": "maybe"}, "full-stem-default-not-off"), # malformed
({"default": "on"}, "full-stem-default-on"),
({"default": "yes"}, "full-stem-default-on"),
({"default": "1"}, "full-stem-default-on"),
],
)
def test_verify_requires_an_explicit_off_on_a_retained_mixdown(tmp_path, default, expected):
"""Beside instrument stems, `full` is safe only with an explicit normalized
`off`. Core defaults an ABSENT `default` to ON and treats empty/unknown as
ON, so a missing or blank default is the double-audio hazard itself, not a
lesser one `verify` must not certify it."""
m = _manifest(
stems=[
{"id": "full", "file": "stems/full.ogg", **default},
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
]
)
del m["original_audio"]
pak = _write_pack(
tmp_path / f"{tmp_path.name}.feedpak",
m,
files={"stems/full.ogg": b"M", "stems/guitar.ogg": b"g"},
)
assert mig.verify_zip(pak) == expected
def test_verify_ignores_default_on_a_sole_full_stem(tmp_path: Path):
"""A single `full` stem IS the audio — the len>1 gate means its default is
not policed, so an on/absent default is fine (off would mute the pack)."""
for default in ({"default": "on"}, {}, {"default": ""}):
m = _manifest(stems=[{"id": "full", "file": "stems/full.ogg", **default}])
del m["original_audio"]
pak = _write_pack(
tmp_path / f"{tmp_path.name}-{len(default)}.feedpak",
m,
files={"stems/full.ogg": b"M"},
)
assert mig.verify_zip(pak) == "ok"
def test_verify_rejects_an_unmigrated_pack(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
assert mig.verify_zip(pak) == "still-has-key"
# ── Unsafe manifest paths must not be laundered into playable audio ─────────
@pytest.mark.parametrize(
"rel", ["../outside.ogg", "/etc/passwd", "a/../../x.ogg", "C:/x.ogg", "a\\b.ogg"]
)
def test_migrate_refuses_an_unsafe_full_mix_path(tmp_path: Path, rel: str):
"""Core's loader REFUSES a full-mix path that escapes the pack — such a pack
simply has no full mix, and the audio is inert. Migrating it into
`stems/full.ogg` would take content the reader deliberately rejected and hand
it back as a valid, playable stem. Report it; never promote it."""
pak = _write_pack(
tmp_path / "song.feedpak",
_manifest(original_audio=rel),
files={rel: b"EVIL", "stems/guitar.ogg": b"g"},
)
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=False) == "unsafe-path"
assert pak.read_bytes() == before
def test_safe_relpath_accepts_ordinary_pack_paths():
assert mig.is_safe_relpath("stems/full.ogg")
assert mig.is_safe_relpath("original/full.ogg")
assert not mig.is_safe_relpath("")
assert not mig.is_safe_relpath("a//b.ogg")
# ── Damaged packs must not abort the run ────────────────────────────────────
def test_a_corrupt_archive_is_reported_not_fatal(tmp_path: Path, capsys):
"""A real library has damage in it — a truncated download, an archive left
half-written by an interrupted converter. One of those must not kill a
50,000-pack run and throw away the summary: the pack is reported, skipped,
and everything else still migrates."""
good = _write_pack(tmp_path / "good.feedpak", _manifest())
bad = tmp_path / "bad.feedpak"
bad.write_bytes(b"this is not a zip file at all")
rc = mig.main([str(tmp_path)])
out = capsys.readouterr().out
assert rc == 1 # a problem pack fails the run's exit code
assert "corrupt-zip" in out
assert "migrated" in out
assert mig.verify_zip(good) == "ok" # the healthy pack still got migrated
assert bad.read_bytes() == b"this is not a zip file at all" # untouched
# ── Directory-form (authoring) packs are discovered, not silently skipped ────
def _write_dir_pack(path: Path, manifest: dict, files: dict[str, bytes] | None = None) -> Path:
"""Build a directory-form pack (`song.sloppak/`), the authoring shape."""
files = files or {
"original/full.ogg": b"MIXDOWN",
"stems/guitar.ogg": b"g",
"stems/drums.ogg": b"d",
"arrangements/lead.json": b"{}",
}
path.mkdir()
(path / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
for name, data in files.items():
p = path / name
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(data)
return path
def test_iter_packs_discovers_directory_form_packs(tmp_path: Path):
"""A `song.sloppak/` directory is a pack; os.walk must yield it whole and
NOT descend into it (its stems/ are contents, not packs)."""
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
z = _write_pack(tmp_path / "other.feedpak", _manifest())
found = set(mig.iter_packs(tmp_path))
assert d in found and z in found
# Nothing inside the directory pack was yielded as its own pack.
assert not any(d in p.parents for p in found)
def test_iter_packs_yields_a_directly_passed_dir_pack(tmp_path: Path):
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
assert list(mig.iter_packs(d)) == [d]
def test_directory_form_pack_is_reported_not_silently_skipped(tmp_path: Path, capsys):
"""The migrator rewrites single-file packs atomically; a directory can't be
swapped that way, so it is surfaced as a problem rather than vanishing from
the run (the silent-skip this guards against) or being rewritten unsafely."""
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
good = _write_pack(tmp_path / "good.feedpak", _manifest())
rc = mig.main([str(tmp_path)])
out = capsys.readouterr().out
assert rc == 1 # a reported problem fails the exit code
assert "dir-form-unsupported" in out
assert mig.verify_zip(good) == "ok" # the zip pack still migrated
# The directory pack is untouched: legacy key intact, mixdown not moved.
manifest = yaml.safe_load((d / "manifest.yaml").read_text())
assert manifest.get("original_audio") == "original/full.ogg"
assert (d / "original" / "full.ogg").read_bytes() == b"MIXDOWN"
def test_verify_reports_directory_form_packs(tmp_path: Path):
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
assert mig.verify_pack(d) == "dir-form-unsupported"
+277
View File
@@ -0,0 +1,277 @@
"""The sloppak loader's handling of a pack's complete mixdown (#933).
The mixdown is a stem: feedpak spec §5.3 RESERVES the id `full` for it. It is a
mixdown, not a layer it already contains every instrument, so a reader that
sums `stems` must never include it in that sum, and `load_song()` therefore
lifts it OUT of `LoadedSloppak.stems` and onto `LoadedSloppak.full_mix`.
Also covers the DEPRECATED `original_audio:` manifest key a key this repo
invented (#583) before the spec reserved `full`, which every pack in the wild
still carries. We read it as a fallback so those packs keep their full mix; we
never write it. Those tests are the deprecation contract: they go when the key
does (#945).
"""
from __future__ import annotations
import json
from pathlib import Path
import yaml
import sloppak as sloppak_mod
def _write_dir_sloppak(
root: Path,
manifest_extras: dict,
*,
write_legacy_full_mix: bool = False,
stems: list[dict] | None = None,
) -> Path:
"""Build a minimal directory-form sloppak that load_song will accept.
Uses the tmp_path leaf name to make the sloppak filename unique per test,
avoiding the module-level ``resolve_source_dir`` cache being poisoned by a
previous test that happened to share the same "song.sloppak" filename.
"""
pak = root / f"{root.name}.sloppak"
pak.mkdir()
arr_dir = pak / "arrangements"
arr_dir.mkdir()
arr = {
"name": "Lead",
"tuning": [0, 0, 0, 0, 0, 0],
"capo": 0,
"notes": [],
"chords": [],
"anchors": [],
"handshapes": [],
"templates": [],
"beats": [],
"sections": [],
}
(arr_dir / "lead.json").write_text(json.dumps(arr))
manifest = {
"title": "Test",
"artist": "Tester",
"album": "",
"year": 2026,
"duration": 10.0,
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
"stems": (
stems
if stems is not None
else [{"id": "guitar", "file": "stems/guitar.ogg", "default": True}]
),
}
manifest.update(manifest_extras)
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
if write_legacy_full_mix:
orig_dir = pak / "original"
orig_dir.mkdir()
# The loader only checks presence (is_file); contents are irrelevant.
(orig_dir / "full.ogg").write_bytes(b"OggS-not-real")
return pak
def _load(pak_path: Path, tmp_path: Path):
dlc_root = pak_path.parent
cache = tmp_path / "cache"
cache.mkdir()
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
def _separated(**extra) -> list[dict]:
"""A separated pack that RETAINS its mixdown, as spec §5.3 asks writers to."""
return [
{"id": "full", "file": "stems/full.ogg", "default": False, **extra},
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
{"id": "drums", "file": "stems/drums.ogg", "default": True},
]
# ── The `full` stem is the mixdown (spec §5.3) ───────────────────────────────
def test_full_stem_is_surfaced_as_the_mixdown(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
loaded = _load(pak, tmp_path)
# Manifest-relative, so the WS builds its URL exactly as it builds a stem's.
assert loaded.full_mix == "stems/full.ogg"
def test_full_stem_is_removed_from_the_stem_list(tmp_path: Path):
"""The regression this whole change exists to prevent.
Every consumer sums `stems` into one mix and renders one fader per entry. The
mixdown already contains every instrument, so leaving it in the list doubles
the entire song and muting `guitar` would still leave guitar audible inside
it. That exact trap is why the packer invented `original_audio` rather than
putting the mixdown where the format says it goes.
"""
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
loaded = _load(pak, tmp_path)
assert [s["id"] for s in loaded.stems] == ["guitar", "drums"]
def test_single_mix_pack_keeps_full_as_its_only_stem(tmp_path: Path):
"""A pack whose ONLY stem is `full` is a single-mix pack, not a separated one.
There are no instruments to be pristine against, so the mixdown stays the sole
playable stem and nothing is surfaced separately. Anything else would strip the
stem list of the most common pack shape in the library and leave it silent.
"""
pak = _write_dir_sloppak(
tmp_path, {}, stems=[{"id": "full", "file": "stems/full.ogg", "default": True}]
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
assert [s["id"] for s in loaded.stems] == ["full"]
def test_every_full_entry_is_removed_not_just_the_first(tmp_path: Path):
"""A malformed pack listing `full` twice must not leave one behind.
Removing the mixdown by object identity would drop only the entry we surface
and leave its duplicate in the stem list a whole copy of the song, summed
with the instruments. That is the exact bug this partition prevents, so a
duplicate must not smuggle it back in.
"""
pak = _write_dir_sloppak(
tmp_path,
{},
stems=[
{"id": "full", "file": "stems/full.ogg", "default": False},
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
{"id": "full", "file": "original/full.ogg", "default": True},
],
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix == "stems/full.ogg"
assert [s["id"] for s in loaded.stems] == ["guitar"]
def test_separated_pack_without_a_full_stem_has_no_mixdown(tmp_path: Path):
"""Stems only, mixdown discarded — the pre-1.15.0 shape. Nothing to surface."""
pak = _write_dir_sloppak(
tmp_path,
{},
stems=[
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
{"id": "drums", "file": "stems/drums.ogg", "default": True},
],
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
assert [s["id"] for s in loaded.stems] == ["guitar", "drums"]
def test_single_mix_pack_ignores_a_lingering_deprecated_key(tmp_path: Path):
"""`full` is the pack's only stem AND the old key is still there.
The stem wins, and it stays the sole playable stem falling back to the key
would surface the mixdown twice: once as the stem the player is already
playing, and once as a "pristine" track for it to cross over to.
"""
pak = _write_dir_sloppak(
tmp_path,
{"original_audio": "original/full.ogg"},
stems=[{"id": "full", "file": "stems/full.ogg", "default": True}],
write_legacy_full_mix=True,
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
assert [s["id"] for s in loaded.stems] == ["full"]
def test_full_stem_wins_over_the_deprecated_key(tmp_path: Path):
"""A migrated pack that still carries the old key must use the stem."""
pak = _write_dir_sloppak(
tmp_path,
{"original_audio": "original/full.ogg"},
stems=_separated(),
write_legacy_full_mix=True,
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix == "stems/full.ogg"
# ── The library index must not advertise the mixdown as an instrument ────────
def test_extract_meta_excludes_the_mixdown_from_stem_ids(tmp_path: Path):
"""The library's stem chips / stem_count come from here, and must agree with
load_song() otherwise the filter offers a "full" chip beside guitar+drums
and counts a third stem that no mixer will ever show."""
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
meta = sloppak_mod.extract_meta(pak)
assert meta["stem_ids"] == ["guitar", "drums"]
assert meta["stem_count"] == 2
def test_extract_meta_keeps_full_for_a_single_mix_pack(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {}, stems=[{"id": "full", "file": "stems/full.ogg", "default": True}]
)
meta = sloppak_mod.extract_meta(pak)
assert meta["stem_ids"] == ["full"]
assert meta["stem_count"] == 1
# ── DEPRECATED `original_audio:` fallback — delete with the key (#945) ───────
def test_legacy_key_still_provides_the_full_mix(tmp_path: Path):
"""Every pack written before the spec reserved `full` looks like this. Dropping
the read would silently take the pristine mix away from all of them."""
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix == "original/full.ogg"
# The legacy mixdown lives OUTSIDE `stems`, so the stem list is untouched.
assert [s["id"] for s in loaded.stems] == ["guitar"]
def test_legacy_key_absent_means_no_full_mix(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {}, write_legacy_full_mix=True)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
def test_legacy_key_none_when_file_missing(tmp_path: Path):
# Manifest points at a full mix that isn't on disk — disabled silently.
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_legacy_full_mix=False
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
def test_legacy_key_none_when_value_blank(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": " "}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
# ── Security / path-traversal branches (legacy key only — a stem `file` is
# resolved through the same /api/sloppak/.../file/ guard as every other stem)
def test_legacy_key_none_when_path_escapes_sloppak(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "../outside.ogg"}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
def test_legacy_key_none_when_path_is_absolute(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "/etc/passwd"}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
-119
View File
@@ -1,119 +0,0 @@
"""End-to-end test for the sloppak loader recognising an `original_audio:`
manifest key (the single full-mix file shipped alongside the separate stems)
and surfacing the manifest-relative path on the LoadedSloppak."""
from __future__ import annotations
import json
from pathlib import Path
import yaml
import sloppak as sloppak_mod
def _write_dir_sloppak(root: Path, manifest_extras: dict, *, write_full_mix: bool) -> Path:
"""Build a minimal directory-form sloppak that load_song will accept.
Uses the tmp_path leaf name to make the sloppak filename unique per test,
avoiding the module-level ``resolve_source_dir`` cache being poisoned by a
previous test that happened to share the same "song.sloppak" filename.
"""
pak = root / f"{root.name}.sloppak"
pak.mkdir()
arr_dir = pak / "arrangements"
arr_dir.mkdir()
arr = {
"name": "Lead",
"tuning": [0, 0, 0, 0, 0, 0],
"capo": 0,
"notes": [],
"chords": [],
"anchors": [],
"handshapes": [],
"templates": [],
"beats": [],
"sections": [],
}
(arr_dir / "lead.json").write_text(json.dumps(arr))
manifest = {
"title": "Test",
"artist": "Tester",
"album": "",
"year": 2026,
"duration": 10.0,
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
"stems": [{"id": "guitar", "file": "stems/guitar.ogg", "default": True}],
}
manifest.update(manifest_extras)
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
if write_full_mix:
orig_dir = pak / "original"
orig_dir.mkdir()
# The loader only checks presence (is_file); contents are irrelevant.
(orig_dir / "full.ogg").write_bytes(b"OggS-not-real")
return pak
def _load(pak_path: Path, tmp_path: Path):
dlc_root = pak_path.parent
cache = tmp_path / "cache"
cache.mkdir()
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
# ── Happy path ───────────────────────────────────────────────────────────────
def test_load_song_attaches_original_audio_when_manifest_opts_in(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_full_mix=True
)
loaded = _load(pak, tmp_path)
# Stored as the manifest-relative string so server.py can build the URL the
# same way it builds stem URLs.
assert loaded.original_audio == "original/full.ogg"
# ── Absent / degraded branches ───────────────────────────────────────────────
def test_load_song_original_audio_none_when_manifest_silent(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {}, write_full_mix=True)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
def test_load_song_original_audio_none_when_file_missing(tmp_path: Path):
# Manifest points at a full mix that isn't on disk — disabled silently.
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_full_mix=False
)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
def test_load_song_original_audio_none_when_value_blank(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {"original_audio": " "}, write_full_mix=True)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
# ── Security / path-traversal branches ──────────────────────────────────────
def test_load_song_original_audio_none_when_path_escapes_sloppak(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "../outside.ogg"}, write_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
def test_load_song_original_audio_none_when_path_is_absolute(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "/etc/passwd"}, write_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
+309
View File
@@ -0,0 +1,309 @@
"""The unpack cache is bounded, and reading part of a song doesn't explode it.
`sloppak_cache/` holds every song ever unpacked, fully decompressed. Stems are
already-compressed audio, so an unpacked song is ~1.1x its zip the cache is a
second copy of the library. It used to have no cap, no LRU, and no cleanup at
all: a tester reached 60 GB from an 1800-song library because one caller looped
the library calling load_song() (rig_builder's library-wide tone batch), which
unpacks the WHOLE pack stems included to read a few KB of tone JSON.
Pins, so neither half can silently come back:
- resolve_source_dir() evicts LRU songs to stay under the cap,
- it never evicts the song the caller just asked for,
- an evicted song is dropped from _source_cache too (otherwise the media route
keeps serving a path that no longer exists and 404s every stem instead of
re-unpacking),
- get_cached_source_dir() self-heals if the cache dir is deleted by hand,
- read_member_bytes() reads one file WITHOUT unpacking anything.
"""
import importlib
import zipfile
import pytest
import yaml
import sloppak as sloppak_mod
STEM = b"\x00" * (400 * 1024) # 400 KB of "audio" — the bulk of a real pack
ARR = b'{"tones": {"definitions": [{"Key": "clean"}]}}'
def _zip_pack(path, stem_bytes=STEM):
with zipfile.ZipFile(path, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({
"title": path.stem,
"arrangements": [{"file": "arrangements/lead.json", "name": "Lead"}],
"stems": [{"id": "full", "file": "stems/audio.ogg"}],
}))
zf.writestr("arrangements/lead.json", ARR)
zf.writestr("stems/audio.ogg", stem_bytes)
return path
@pytest.fixture(autouse=True)
def _fresh_module_state():
# _source_cache is module state and would leak across tests.
importlib.reload(sloppak_mod)
yield
importlib.reload(sloppak_mod)
def _cap_mb(monkeypatch, mb):
monkeypatch.setenv("FEEDBACK_SLOPPAK_CACHE_MAX_MB", str(mb))
def test_read_member_bytes_does_not_unpack(tmp_path, monkeypatch):
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
data = sloppak_mod.read_member_bytes(pack, "arrangements/lead.json")
assert data == ARR
assert list(cache.iterdir()) == [], (
"reading one member must not unpack the pack — this is the whole point: "
"load_song() would have written the 400 KB stem to disk to get 45 bytes of JSON"
)
def test_read_member_bytes_missing_member_is_none(tmp_path):
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
assert sloppak_mod.read_member_bytes(pack, "arrangements/nope.json") is None
assert sloppak_mod.read_member_bytes(pack, "") is None
def test_unpack_cache_evicts_lru_to_stay_under_cap(tmp_path, monkeypatch):
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1) # 1 MB — holds ~2 of our 400 KB packs
for i in range(6):
_zip_pack(dlc / f"song{i}.feedpak")
for i in range(6):
sloppak_mod.resolve_source_dir(f"song{i}.feedpak", dlc, cache)
total = sum(f.stat().st_size for f in cache.rglob("*") if f.is_file())
assert total <= 1 * 1024 * 1024, (
f"unpack cache ran to {total/1e6:.1f} MB against a 1 MB cap — this is the "
"unbounded growth that reached 60 GB in the field"
)
# The most recent song must survive; the oldest must not.
names = {d.name for d in cache.iterdir()}
assert "song5.feedpak" in names, "the song just resolved must never be evicted"
assert "song0.feedpak" not in names, "the least-recently-used song should go first"
def test_eviction_drops_the_source_cache_entry(tmp_path, monkeypatch):
"""An evicted song must not keep being handed out by get_cached_source_dir().
media.py only falls back to resolve_source_dir() when this returns None. If a
stale path survives, every stem 404s for the rest of the process instead of
re-unpacking a silently broken song, not a slow one.
"""
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1)
for i in range(6):
_zip_pack(dlc / f"song{i}.feedpak")
for i in range(6):
sloppak_mod.resolve_source_dir(f"song{i}.feedpak", dlc, cache)
evicted = sloppak_mod.get_cached_source_dir("song0.feedpak")
assert evicted is None, "an evicted song must be dropped from _source_cache"
# ...and asking for it again just re-unpacks it. Self-healing, not broken.
again = sloppak_mod.resolve_source_dir("song0.feedpak", dlc, cache)
assert (again / "stems" / "audio.ogg").is_file()
def test_get_cached_source_dir_self_heals_after_manual_delete(tmp_path, monkeypatch):
"""Telling a user to delete sloppak_cache/ to reclaim disk must be safe."""
import shutil
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 0) # eviction off — isolate the delete
_zip_pack(dlc / "song.feedpak")
src = sloppak_mod.resolve_source_dir("song.feedpak", dlc, cache)
assert sloppak_mod.get_cached_source_dir("song.feedpak") == src
shutil.rmtree(src) # the user clears the folder
assert sloppak_mod.get_cached_source_dir("song.feedpak") is None, (
"a path that no longer exists must not be served — the caller would 404 "
"every stem instead of re-unpacking"
)
assert (sloppak_mod.resolve_source_dir("song.feedpak", dlc, cache)
/ "stems" / "audio.ogg").is_file()
def test_cap_of_zero_disables_eviction(tmp_path, monkeypatch):
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 0)
for i in range(4):
_zip_pack(dlc / f"song{i}.feedpak")
for i in range(4):
sloppak_mod.resolve_source_dir(f"song{i}.feedpak", dlc, cache)
assert len(list(cache.iterdir())) == 4, "cap 0 must mean 'never evict'"
def test_read_member_bytes_normalizes_non_canonical_names(tmp_path):
"""A manifest may name a member './arrangements/lead.json' — valid, and it
resolved fine once unpacked. Reading the zip member by the raw string would
KeyError and silently report no tones. Same trap read_cover_bytes already hit."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
assert sloppak_mod.read_member_bytes(pack, "./arrangements/lead.json") == ARR
assert sloppak_mod.read_member_bytes(pack, "stems/../arrangements/lead.json") == ARR
assert sloppak_mod.read_member_bytes(pack, "arrangements\\lead.json") == ARR
def test_read_member_bytes_rejects_zip_slip(tmp_path):
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
assert sloppak_mod.read_member_bytes(pack, "../../etc/passwd") is None
assert sloppak_mod.read_member_bytes(pack, "/etc/passwd") is None
assert sloppak_mod.read_member_bytes(pack, ".") is None
def test_eviction_never_deletes_an_in_flight_unpack(tmp_path, monkeypatch):
"""Two unpacks run concurrently (_UNPACK_MAX_CONCURRENCY = 2). One finishing
must not rmtree the other's half-written dir — that resolver would then cache
an incomplete song and serve a broken pack.
Sized so the sweep genuinely has to reach the in-flight directory: each pack
is ~700 KB against a 1 MB cap, so once `keep` is protected the sweep must
delete EVERY other dir to get under the cap including the one being written.
(A naive version of this test passes even without the guard, because a
freshly-created dir is the most-recently-used and the sweep never gets to it.)
"""
import threading
big = b"\x00" * (700 * 1024)
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1)
for i in range(3):
_zip_pack(dlc / f"song{i}.feedpak", stem_bytes=big)
victim = cache / "song2.feedpak"
started = threading.Event()
release = threading.Event()
real_unpack = sloppak_mod._unpack_zip
def slow_unpack(zip_path, dest):
real_unpack(zip_path, dest) # dir now exists — "half written"
if dest == victim:
started.set()
release.wait(5) # hold it open while the other sweeps
monkeypatch.setattr(sloppak_mod, "_unpack_zip", slow_unpack)
t = threading.Thread(target=sloppak_mod.resolve_source_dir,
args=("song2.feedpak", dlc, cache))
t.start()
assert started.wait(5), "victim unpack did not start"
# song0 lands and sweeps: keep=song0, cache holds song0+song2 = 1.4 MB > 1 MB,
# so the sweep MUST try to delete song2 — which is still being written.
sloppak_mod.resolve_source_dir("song0.feedpak", dlc, cache)
in_flight_survived = victim.is_dir()
release.set()
t.join(5)
assert in_flight_survived, (
"eviction deleted a directory another thread was still unpacking into — "
"that resolver caches an incomplete song and serves a broken pack"
)
def test_read_member_bytes_finds_backslash_members(tmp_path):
"""Windows-authored packs store members as 'arrangements\\lead.json'.
_unpack_zip() normalizes those on extract, so unpack-then-read found them.
An exact getinfo() would not and we'd silently report the song has no tones."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = dlc / "win.feedpak"
with zipfile.ZipFile(pack, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({"title": "w"}))
zf.writestr("arrangements\\lead.json", ARR) # backslash member name
assert sloppak_mod.read_member_bytes(pack, "arrangements/lead.json") == ARR
def test_read_member_bytes_finds_non_canonical_STORED_names(tmp_path):
"""The archive itself may store './arrangements/lead.json'. _unpack_zip()
normalizes stored names on extract, so unpack-then-read resolved it. Both the
requested path and the stored name must be normalized, or the tones vanish."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = dlc / "odd.feedpak"
with zipfile.ZipFile(pack, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({"title": "o"}))
zf.writestr("./arrangements/lead.json", ARR) # stored non-canonically
assert sloppak_mod.read_member_bytes(pack, "arrangements/lead.json") == ARR
def test_read_member_bytes_matches_unpack_last_write_wins(tmp_path):
"""If a pack stores two names that normalize to the same file, _unpack_zip
writes them in order and the LAST one is what ends up on disk. Reading the
raw member by exact name would hand back the first stale arrangement data
that no unpacked read would ever have produced."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = dlc / "dupe.feedpak"
with zipfile.ZipFile(pack, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({"title": "d"}))
zf.writestr("arrangements/lead.json", b'{"tones": {"definitions": [{"Key": "STALE"}]}}')
zf.writestr("./arrangements/lead.json", ARR) # normalizes to the same path
assert sloppak_mod.read_member_bytes(pack, "arrangements/lead.json") == ARR
def test_failed_unpack_does_not_leave_the_dir_un_evictable(tmp_path, monkeypatch):
"""A dir marked in-flight is skipped by eviction. If a failed unpack leaves the
marker behind, that dir becomes permanently un-evictable a slow leak of
exactly the thing this cap exists to prevent."""
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1)
_zip_pack(dlc / "boom.feedpak")
def blow_up(zip_path, dest):
dest.mkdir(parents=True, exist_ok=True)
raise OSError("disk full")
monkeypatch.setattr(sloppak_mod, "_unpack_zip", blow_up)
with pytest.raises(OSError):
sloppak_mod.resolve_source_dir("boom.feedpak", dlc, cache)
assert not sloppak_mod._unpacking, (
"a failed unpack left its destination marked in-flight — eviction will "
"skip it forever"
)
+5 -2
View File
@@ -480,8 +480,11 @@ def test_seconds_only_post_accrues_without_touching_position(client):
# overwrite Continue with the end-of-song offset).
assert row["plays"] == 0
assert row["last_position"] == pytest.approx(42.0)
# But the song WAS played — recency ordering must see it.
assert row["last_played_at"]
# Recency must come from the seconds-only POST itself — prove it on a
# FRESH row (the position touch above already stamps last_played_at,
# which would make an assertion here vacuous).
r2 = client.post("/api/stats", json={"filename": "fresh.archive", "seconds": 30})
assert r2.json()["stats"]["last_played_at"]
# Still counts as playing today for the streak.
assert r.json()["progress"]["current_streak"] == 1
+445
View File
@@ -0,0 +1,445 @@
#!/usr/bin/env python3
"""Migrate packs off the deprecated `original_audio:` key — the full mix is a stem.
Before feedpak 1.15.0 reserved the stem id `full`, spec §5.3 said the mixdown was
"commonly replaced" by the per-instrument stems when a pack was split so after
separation it had nowhere to live. This repo worked around that by inventing a
top-level `original_audio:` manifest key pointing at a parallel `original/`
directory (#583). The key was never in the spec, and #933 removed core's
dependence on it: the mixdown is a stem, and its id is `full`.
This rewrites a pack into the shape the spec now defines:
original/full.ogg -> stems/full.ogg (entry moved)
original_audio: original/full.ogg -> stems: [{id: full, file: stems/full.ogg,
default: 'off'}, ...]
`default: 'off'` is what makes the retained mixdown safe: a reader that honours
`default` (normative since feedpak 1.0.0) will not play it on open, so it never
doubles the mix even in a reader that predates the reserved id.
Nothing else in the pack is touched every other key, file and stem is preserved
verbatim, and `feedpak_version` is stamped to the version the result conforms to.
The rewrite is atomic per pack: a new archive is built beside the original and
renamed over it only on success, so an interrupted run leaves every pack either
fully migrated or untouched never truncated.
Idempotent: a pack that already carries a `full` stem and no `original_audio:` is
reported as `skip` and left alone, so a partial run can simply be re-run.
Usage:
python tools/migrate_full_mix_stem.py --dry-run /path/to/packs # report only
python tools/migrate_full_mix_stem.py /path/to/packs # migrate
python tools/migrate_full_mix_stem.py --verify /path/to/packs # check results
Exit status is 0 only when every pack ended up in the migrated shape (or was
already there).
"""
from __future__ import annotations
import argparse
import os
import shutil
import sys
import tempfile
import threading
import time
import zipfile
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
import yaml
# The version this migration brings a pack up to: the one that reserved `full`.
TARGET_FEEDPAK_VERSION = "1.15.0"
FULL_MIX_STEM_ID = "full"
LEGACY_KEY = "original_audio"
# Where the mixdown lands. §2.1's conventional layout — readers resolve through
# the manifest and never care about the path, but a pack that says `stems/` and
# means it is the one a human can read.
CANONICAL_FULL_MIX_PATH = "stems/full.ogg"
PACK_EXTS = (".feedpak", ".sloppak")
class Skip(Exception):
"""Pack needs no migration."""
def is_safe_relpath(rel: str) -> bool:
"""True when `rel` is a manifest path the spec allows (§2.2 rule 2).
POSIX-style relative: forward slashes, no leading `/`, no `..` segments, no
empty segments, no colon (which excludes drive letters and NTFS alternate
data streams), no backslashes.
This is a TRUST BOUNDARY, not a tidiness check. Core's loader refuses a
full-mix path that escapes the pack and reports the pack as having no full
mix the audio is inert. A migration that moved such an entry into
`stems/full.ogg` would take content the reader deliberately rejected and
hand it back as a valid, playable stem. So a pack like this is reported, not
migrated.
"""
if not rel or rel.startswith("/") or "\\" in rel or ":" in rel:
return False
parts = rel.split("/")
return all(p and p != ".." for p in parts)
def plan_manifest(manifest: dict) -> tuple[dict, str]:
"""Return (new_manifest, relpath_of_audio_to_move); "" = no file needs moving.
Raises Skip when the pack needs no migration. Pure no I/O so the part
with the decisions in it is testable without building archives.
"""
raw_stems = manifest.get("stems")
stems: list = raw_stems if isinstance(raw_stems, list) else []
has_full_stem = any(
isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID for s in stems
)
legacy_rel = manifest.get(LEGACY_KEY)
legacy_rel = legacy_rel.strip() if isinstance(legacy_rel, str) else ""
if not legacy_rel:
# Nothing invented to undo: either the pack already keeps its mixdown as
# the `full` stem, or it never carried one.
raise Skip("already migrated" if has_full_stem else "no original_audio key")
if has_full_stem:
# The mixdown is already a stem and the dead key merely lingers beside it.
# Drop the key; move nothing. But do NOT trust its `default`: a mixdown
# left enabled beside instrument stems is the double-audio hazard this
# migration exists to remove, and a reader that honours `default` would
# play the whole song on top of the stems on open. Force it off — unless
# `full` is the only stem, in which case it IS the audio.
others = [
s
for s in stems
if isinstance(s, dict) and str(s.get("id", "")) != FULL_MIX_STEM_ID
]
new_stems = []
for s in stems:
if isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID and others:
s = {**s, "default": "off"}
new_stems.append(s)
to_move = ""
else:
# `default` decides whether a reader plays this on open, and that is the
# whole safety margin: alongside per-instrument stems the mixdown must be
# OFF (a reader that sums the list would otherwise double the song), but
# when it is the pack's only stem it IS the audio and must be ON.
entry = {
"id": FULL_MIX_STEM_ID,
"file": CANONICAL_FULL_MIX_PATH,
"default": "off" if stems else "on",
}
# First in the list, matching the spec's §5.3 example.
new_stems = [entry, *stems]
# If the key already pointed at the canonical path, only the manifest is wrong.
to_move = "" if legacy_rel == CANONICAL_FULL_MIX_PATH else legacy_rel
out: dict = {}
for k, v in manifest.items():
if k == LEGACY_KEY:
continue # the invented key disappears
out[k] = new_stems if k == "stems" else v
out.setdefault("stems", new_stems) # a pack that had no stems list gets one
out["feedpak_version"] = TARGET_FEEDPAK_VERSION
return out, to_move
def migrate_zip(path: Path, dry_run: bool) -> str:
"""Rewrite one zipped pack in place. Returns a one-word status."""
with zipfile.ZipFile(path) as zf:
try:
raw = zf.read("manifest.yaml")
except KeyError:
return "no-manifest"
manifest = yaml.safe_load(raw) or {}
try:
new_manifest, old_rel = plan_manifest(manifest)
except Skip:
return "skip"
names = set(zf.namelist())
if old_rel:
if not is_safe_relpath(old_rel):
# Core refuses this path and plays no full mix for the pack. Do
# not launder it into a valid stem — see is_safe_relpath().
return "unsafe-path"
if old_rel not in names:
# The key points at audio that isn't in the archive. Core already
# treats that as "no full mix"; migrating would fabricate a stem
# entry for a file that does not exist and break every reader.
return "missing-audio"
if CANONICAL_FULL_MIX_PATH in names:
return "target-occupied"
else:
# Manifest-only rewrite (a stale key beside a mixdown that is already
# a stem, or a key that already pointed at the canonical path). Check
# the file the resulting `full` stem will actually NAME — not the
# canonical path, which an already-migrated pack is free not to use:
# §2.2 says readers resolve through the manifest, so a valid pack may
# keep its mixdown anywhere.
full_file = next(
(
s.get("file")
for s in new_manifest.get("stems", [])
if isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID
),
None,
)
if full_file not in names:
return "missing-audio"
if dry_run:
return "would-migrate"
# Build the replacement beside the original, on the same filesystem, so
# the final rename is atomic and an interrupted run can't truncate a pack.
tmp_fd, tmp_name = tempfile.mkstemp(
dir=str(path.parent), prefix=f".{path.name}.", suffix=".tmp"
)
os.close(tmp_fd)
tmp_path = Path(tmp_name)
try:
with zipfile.ZipFile(tmp_path, "w", zipfile.ZIP_DEFLATED) as out:
for item in zf.infolist():
if item.filename == "manifest.yaml":
out.writestr(
item,
yaml.safe_dump(
new_manifest, sort_keys=False, allow_unicode=True
),
)
continue
data = zf.read(item.filename)
if old_rel and item.filename == old_rel:
# Same bytes, same compression, new name: the mixdown moves
# from original/ into stems/ where the format says audio goes.
moved = zipfile.ZipInfo(
CANONICAL_FULL_MIX_PATH, date_time=item.date_time
)
moved.compress_type = item.compress_type
moved.external_attr = item.external_attr
out.writestr(moved, data)
continue
out.writestr(item, data)
except BaseException:
tmp_path.unlink(missing_ok=True)
raise
shutil.copystat(path, tmp_path)
os.replace(tmp_path, path) # atomic
return "migrated"
def verify_zip(path: Path) -> str:
"""Confirm a pack is in the migrated shape and its mixdown is really there."""
with zipfile.ZipFile(path) as zf:
try:
manifest = yaml.safe_load(zf.read("manifest.yaml")) or {}
except KeyError:
return "no-manifest"
if LEGACY_KEY in manifest:
return "still-has-key"
stems = manifest.get("stems") or []
full = next(
(
s
for s in stems
if isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID
),
None,
)
if full is None:
return "no-full-stem"
if full.get("file") not in set(zf.namelist()):
return "full-stem-missing-file"
# A retained mixdown that plays on open would double the mix in any reader
# that sums the stem list — the whole hazard this migration must not create.
# Beside instrument stems, `full` MUST carry an explicit, normalized "off":
# - core (lib/sloppak.py) defaults an ABSENT `default` to True (ON) and
# treats an empty / unrecognized string as ON, so a missing or blank
# default is not merely non-canonical — core would play the mixdown on
# open, doubling the song. It is the exact hazard, not a lesser one.
# - the migrator always writes the literal "off", so requiring it also
# certifies the pack is in the shape this tool produces — the most
# portable spelling, understood even by a reader that only knows
# "on"/"off" and would choke on a boolean or `false`/`0`/`no`.
# So: `on`-ish values are reported as actively-playing; everything that is
# not a normalized "off" (missing, empty, boolean, `false`/`no`/`0`,
# malformed) is reported as an unsafe/non-canonical default.
if len(stems) > 1:
default = str(full.get("default", "")).strip().lower()
if default in ("true", "on", "yes", "1"):
return "full-stem-default-on"
if default != "off":
return "full-stem-default-not-off"
return "ok"
def migrate_pack(path: Path, dry_run: bool) -> str:
"""Dispatch by pack form. ZIP-file packs are rewritten in place; directory
(authoring) packs are REPORTED, not rewritten.
A single-file pack is replaced atomically a fully-built temp archive
swapped in with one os.replace(), so an interrupted run leaves it either
fully migrated or untouched. A directory can't be swapped that way (no
atomic replace of a populated directory), so an in-place rewrite could leave
an authoring pack half-migrated. Rather than risk that, directory packs are
surfaced as `dir-form-unsupported` (a problem status, so the run's exit code
and summary flag them) for the operator to re-pack or migrate as a `.feedpak`.
"""
if path.is_dir():
return "dir-form-unsupported"
return migrate_zip(path, dry_run)
def verify_pack(path: Path) -> str:
"""Verify a pack; directory (authoring) packs are reported, see migrate_pack."""
if path.is_dir():
return "dir-form-unsupported"
return verify_zip(path)
def iter_packs(root: Path):
"""Yield every pack under `root`. A pack is a suffix-named ZIP FILE or a
suffix-named DIRECTORY (the authoring form) both are discovered so a
directory-form pack is never silently walked past. A directory pack is
yielded whole, not descended into: its `stems/` and `arrangements/` are pack
contents, not packs. (migrate/verify then report directory packs rather than
rewriting them in place see migrate_pack.)"""
if root.is_file():
yield root
return
# A directory whose OWN name is a pack suffix is a single directory-form
# pack passed directly, not a tree of packs to search.
if root.name.endswith(PACK_EXTS):
yield root
return
for dirpath, dirnames, filenames in os.walk(root):
for fn in sorted(filenames):
if fn.endswith(PACK_EXTS):
yield Path(dirpath) / fn
for dn in sorted(dn for dn in dirnames if dn.endswith(PACK_EXTS)):
yield Path(dirpath) / dn
# Don't descend INTO a pack directory — its contents aren't packs.
dirnames[:] = [dn for dn in dirnames if not dn.endswith(PACK_EXTS)]
def main(argv: list[str] | None = None) -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("root", type=Path, help="pack, or directory of packs")
ap.add_argument("--dry-run", action="store_true", help="report, change nothing")
ap.add_argument("--verify", action="store_true", help="check the migrated shape")
ap.add_argument("--jobs", type=int, default=8, help="parallel packs (default 8)")
args = ap.parse_args(argv)
if not args.root.exists():
print(f"error: {args.root} does not exist", file=sys.stderr)
return 2
packs = list(iter_packs(args.root))
if not packs:
print(f"no packs found under {args.root}", file=sys.stderr)
return 2
action = verify_pack if args.verify else (lambda p: migrate_pack(p, args.dry_run))
def work(pack: Path) -> str:
"""Never raise. One unreadable pack must not kill a 50,000-pack run.
A library this size has damage in it a truncated download, an archive
left half-written by an interrupted converter. Letting that propagate
aborts the whole job partway through and throws away the summary, which
is exactly when you most need to know what happened. Report it as a
problem status instead: the pack is untouched, the run continues, and the
final report names it.
"""
try:
return action(pack)
except zipfile.BadZipFile:
return "corrupt-zip"
except OSError as e:
return f"io-error ({e.__class__.__name__})"
except Exception as e: # malformed YAML, unexpected manifest shape, …
return f"error ({e.__class__.__name__})"
counts: dict[str, int] = {}
problems: list[tuple[str, Path]] = []
# A real run rewrites every archive under `root` — tens of thousands of packs
# and hundreds of gigabytes. Printing only a final summary means hours of
# silence, in which a stall and steady progress look identical. Emit a
# heartbeat instead: rate and ETA come from the packs actually finished, so
# it stays honest when the disk slows down. stderr, so `> report.txt` keeps
# the summary clean.
total = len(packs)
started = time.monotonic()
# The heartbeat runs on its OWN CLOCK, in its own thread.
#
# Two weaker designs were tried and both go quiet exactly when you need them
# to speak. Ticking every N packs ties the cadence to how slow a pack is: 500
# packs is a blink in a --dry-run and many minutes in a real migration, so the
# run that most needs watching says the least. Ticking on time but only when a
# pack *finishes* is no better: if every worker is grinding on a huge archive,
# nothing completes, so nothing prints — and a stall becomes indistinguishable
# from progress, which is the one thing a progress meter must never allow.
#
# A daemon thread on a fixed interval reports regardless. If the count stops
# advancing between beats, you are looking at a stall, and you can see it.
HEARTBEAT_SECONDS = 10.0
done = 0 # only the main loop writes it; the beat thread only reads
stop_beat = threading.Event()
def heartbeat() -> None:
while not stop_beat.wait(HEARTBEAT_SECONDS):
elapsed = time.monotonic() - started
rate = done / elapsed if elapsed > 0 else 0.0
eta = (total - done) / rate if rate > 0 else 0.0
print(
f" {done}/{total} ({100 * done / total:.1f}%) "
f"{rate:.1f} packs/s eta {eta / 60:.0f}m "
f"[{len(problems)} problem(s)]",
file=sys.stderr,
flush=True,
)
print(
f"{total} pack(s) under {args.root}"
f"{'verifying' if args.verify else 'dry run' if args.dry_run else 'migrating'} "
f"with {max(1, args.jobs)} job(s)",
file=sys.stderr,
flush=True,
)
beat = threading.Thread(target=heartbeat, daemon=True)
beat.start()
# as_completed, not pool.map: map yields in SUBMISSION order, so the counter
# would stall behind one slow pack while later ones were already done — a
# progress meter that lies about progress. Count each pack as it finishes.
try:
with ThreadPoolExecutor(max_workers=max(1, args.jobs)) as pool:
futures = {pool.submit(work, p): p for p in packs}
for fut in as_completed(futures):
pack = futures[fut]
status = fut.result()
counts[status] = counts.get(status, 0) + 1
if status not in ("migrated", "skip", "would-migrate", "ok"):
problems.append((status, pack))
done += 1
finally:
stop_beat.set()
beat.join(timeout=1)
print(f"\n{len(packs)} pack(s) under {args.root}")
for status, n in sorted(counts.items(), key=lambda kv: -kv[1]):
print(f" {n:>7} {status}")
if problems:
print(f"\n{len(problems)} pack(s) need a look:", file=sys.stderr)
for status, pack in problems[:20]:
print(f" {status:<22} {pack}", file=sys.stderr)
if len(problems) > 20:
print(f" … and {len(problems) - 20} more", file=sys.stderr)
return 1 if problems else 0
if __name__ == "__main__":
raise SystemExit(main())