Compare commits

..
Author SHA1 Message Date
byrongamatosandClaude Fable 5 e1a534ab63 feat(career): passport backend — genre badges computed from stars
The badge-journey layer on top of career stars (Christian's career-mode
v2 design, composed with the shipped venue system). Badges are computed
on read from song_stats × the library's effective genre — never stored:
Bronze = N genre songs at min_stars (data-driven in passports.json,
default 5 songs at 2★) plus any configured virtuoso drill nodes.

New endpoints under /api/plugins/career/:
- GET  /passports        passport walls per instrument: badges, ticket
                         stubs (qualifying songs), library genres, drills
- POST /passports/commit instrument commitment (idempotent wax seal)
- POST /passports/open   open a genre passport (implies commitment)
- POST /drill-state      intake for the relayed virtuoso.progress
                         snapshot (career's frontend listens on the bus)

Instrument attribution reuses progression.instrument_for_arrangement via
the song_stats arrangement index; the genre column goes through the
host's override-aware effective-genre SQL. Non-graded instruments (bass,
drums) render shown-not-judged — repertoire, never a false badge denial.

Persisted state (commitments, opened passports, drill snapshot) lives
under CONFIG_DIR/career/ and rides the settings export bundle via
settings.server_files; a minimal settings.html documents it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 11:13:31 +02:00
53 changed files with 286 additions and 5137 deletions
-17
View File
@@ -1,17 +0,0 @@
## What
<!-- What does this PR do, and why? Link the issue it addresses. -->
## feedpak surface
<!-- The feedpak spec is sacrosanct: the spec defines the format, this app implements it.
Delete this section ONLY if your change doesn't touch how the app reads or writes packs. -->
- [ ] This PR does **not** change how the app reads/writes feedpaks (manifest keys, pack files, folder layout)
- [ ] …or it does, and the spec change landed first via the [FEP process](https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md) — FEP / spec PR: `got-feedback/feedpak-spec#___` (once it merges, re-run this PR's checks and the gate goes green)
## Checklist
- [ ] `CHANGELOG.md` `[Unreleased]` updated (user-visible changes)
- [ ] Tests added/updated for new behaviour
- [ ] Commits are DCO signed off (`git commit -s`)
-88
View File
@@ -124,94 +124,6 @@ jobs:
print(f"Validated {len(manifests)} manifest(s) — OK")
EOF
feedpak-spec:
# Guard that core stays faithful to the feedpak format spec, which lives in
# its own repo (got-feedback/feedpak-spec) and is the contract third-party
# packers and players build against. Four surface checks: core reads/writes
# only manifest keys the spec declares (and the scanned-module list can't
# fall behind); the exception allowlist never grows, so the FEP process is
# the only way a new key lands; core ingests the spec's example packs; packs
# committed here pass the spec's reference validator. Motivated by
# #933, where a manifest key (`original_audio`) shipped in core without ever
# reaching the spec.
#
# The gate checks against the spec repo's HEAD, deliberately: the app must
# conform to the LIVING spec, always. The dev flow is self-serve — a gated
# PR opens a FEP, the spec PR merges, re-running this job goes green; no
# pin file to bump, nothing to maintain. Accepted trade-off: a BREAKING
# spec change (rare, deliberate, MAJOR per the spec's compatibility policy)
# reddens every PR here until core conforms — which is the correct
# org-wide signal that the app is out of conformance. The normal FEP is
# additive and can never redden this job.
name: feedpak-spec
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# This job runs repository code (tools/check_spec_conformance.py) and
# never pushes; don't leave the token in git config for it.
# fetch-depth: 0 so the base branch is available — the gate must prove
# the exception allowlist didn't grow in this PR.
with:
persist-credentials: false
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Check out feedpak-spec at HEAD
uses: actions/checkout@v4
with:
repository: got-feedback/feedpak-spec
ref: main
path: .feedpak-spec
persist-credentials: false
- name: Record the spec commit this run verified against
# HEAD-tracking means CI results can differ across time on the same
# commit. Log the exact spec SHA so a red run is reproducible.
run: git -C .feedpak-spec rev-parse HEAD
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
# CI-only: the spec's reference validator needs jsonschema. Not a
# runtime dependency — this gate never runs on the serve/Docker path
# (constitution Principle I). Pinned for the same reason the spec SHA
# is: an upstream release must not turn this job red on a PR that
# changed neither this repo nor the spec.
pip install 'jsonschema==4.26.0'
- name: Fetch the base branch's exception allowlist
id: baseline
run: |
# The allowlist is closed: it grandfathers keys that predate this gate
# and may only shrink. Prove that by diffing against the base branch —
# without this, anyone could append an entry and route around the FEP
# process from inside this repo.
#
# Resolve the base rather than hardcoding `main`: ship-ci.yml also runs
# this workflow for PRs into release/** and for pushes to release/**,
# where a main baseline would diff against the wrong branch.
# PR -> the branch it merges into
# push -> the branch itself (its tip already contains the change, so
# this is a no-op; enforcement happens at PR time)
BASE="${{ github.event.pull_request.base.ref || github.ref_name }}"
echo "diffing the allowlist against origin/$BASE"
git fetch --no-tags --depth=1 origin "$BASE"
if git cat-file -e FETCH_HEAD:feedpak-spec-exceptions.yml 2>/dev/null; then
git show FETCH_HEAD:feedpak-spec-exceptions.yml > "$RUNNER_TEMP/baseline-exceptions.yml"
echo "args=--baseline-exceptions $RUNNER_TEMP/baseline-exceptions.yml" >> "$GITHUB_OUTPUT"
else
# Only true until the PR that introduces this gate lands.
echo "args=--bootstrap-allowlist" >> "$GITHUB_OUTPUT"
fi
- name: Check feedpak spec conformance
run: python tools/check_spec_conformance.py --spec .feedpak-spec ${{ steps.baseline.outputs.args }}
lint:
# Maintainer/CI-only size + module-hygiene gate (constitution Principle I:
# dev tooling, never on the serve/Docker path — same category as
+2 -119
View File
@@ -7,90 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- **The full mix is a stem** (#933) — core no longer depends on `original_audio:`, a
top-level manifest key this repo invented (#583) that the feedpak spec never had.
The format already carried the pre-separation mixdown as a stem; feedpak 1.15.0
(feedpak-spec#53) reserves the id **`full`** for it, so that is where core reads it
from now.
`full` is a mixdown, not a layer — it already contains every instrument — so
`load_song()` lifts it OUT of `LoadedSloppak.stems` onto `LoadedSloppak.full_mix`.
Nothing that sums stems or renders one fader per stem can see it, which is what
makes retaining it safe; leaving it in the list would double the whole song and
leave "guitar" audible with the guitar fader muted. That trap is exactly why the
packer invented the key instead of putting the mixdown where the format says it
goes — the bug was in the reader, and this fixes the reader.
Consequences worth knowing:
- The highway WS `song_info` frame gains `full_mix_url` / `has_full_mix`.
`original_audio_url` / `has_original_audio` remain as **deprecated aliases**
(same values) for one release so a client built against the old frame keeps
working; they go with the fallback below (#945).
- `stems` on `song_info`, and `stem_ids` / `stem_count` in the library index, now
describe *instrument* stems only — a separated pack that retains its mixdown no
longer advertises a bogus "full" stem chip or an inflated stem count.
- Audio fingerprinting (`lib/enrichment.py`) now resolves the mixdown the same
way, which **widens** its coverage: it previously returned `None` for any pack
without the invented key, so fingerprinting silently did nothing for the
overwhelming majority of packs.
- Core still **reads** `original_audio:` as a deprecated fallback, because every
pack written before the spec caught up carries it and would otherwise lose its
pristine mix. `tools/migrate_full_mix_stem.py` rewrites those packs into the
spec shape (moves `original/full.ogg``stems/full.ogg`, adds the `full` stem
at `default: off`, drops the key); the fallback and the aliases are removed once
they are migrated (#945).
### Added
- **Genres fall back to MusicBrainz enrichment** — the effective genre now
resolves override → pack genre → the enrichment match's primary genre
(matched or user-pinned manual rows only). Converted packs rarely carry a `genres` manifest key,
which starved the library genre facet and career passports on real
libraries; with the fallback, every enriched song's genre is browsable and
passport-able immediately, and coverage grows as enrichment runs.
- **Badge ceremony in the venue** — earning a genre badge now stages a moment:
the crowd layer erupts (new public `v3VenueCrowd.celebrate()` — instant
ecstatic loop bypassing the stability/dwell hysteresis, plus a cheer stinger;
a no-op without a venue pack) and a full-screen overlay drops the bronze
stamp with a shine sweep and a confetti burst over whatever screen is active
(badges land right after `stats:recorded`, while the player is still up).
Click or wait ~4s to dismiss; `prefers-reduced-motion` gets the existing
chime + notification only. The stamp still slams into the passport book on
next open, unchanged.
- **Hours-per-genre odometer (career passports)** — the app now measures real
play time: the stats recorder accrues **wall-clock** seconds across
play/resume ↔ pause/stop/end spans (wall time, not song position — position
deltas double-count A-B loops and mis-read seeks; single spans clamp at 2h
against suspend/sleep inflation) and piggybacks them as `seconds` on the
`POST /api/stats` calls it already makes. New additive
`song_stats.seconds_total` column; a seconds-only POST banks time for
unscored plays that run to the song's natural end without touching the
resume position (and still counts as playing today for the streak).
Passports surface it honestly: "14.2 h in Blues" under the badge and on the
shelf cover — a true fact that only grows, never a target or a meter.
- **Career passport drills, curated** — Bronze in blues/rock/metal/funk/jazz
now also asks for that genre's signature Virtuoso drill (Blues Shuffle,
Power Chords & Backbeat, Gallop Picking, 16th Pocket, Shell Voicings — one
per genre, data-driven in `passports.json` with display labels). Drill
lists are per-instrument (`virtuoso_nodes: {instrument: [nodes]}`; a flat
list still means guitar), so a keys passport never demands a guitar drill.
A drill counts as cleared on the first real completion artifact — a
top-tier clean pass in one key (`keysCleared`), any depth rung, or
mastery — rather than only the maxed-speed depth flips. Genres without a
curated drill stay songs-only.
- **Career passport visuals pack** — earned covers and badge stamps become
trading cards (pointer-tracked tilt + light glint, hover-capable devices
only); the ghost stamp visibly "carves in" as qualifying songs land (a
conic ink fill, no numbers added); the Gold rung preview is a small foil
chip with a shimmer sweep, still honestly labeled coming. All theatrics
disabled under `prefers-reduced-motion`.
- **Career passports (backend)** — the badge-journey layer on top of career stars.
New career-plugin endpoints: `GET /api/plugins/career/passports` (per-instrument
passport walls: genre badges computed on read from `song_stats` × the library's
effective genre — Bronze = N genre songs at K★, data-driven in
`plugins/career/passports.json`, default 5 songs at 2★ — plus qualifying-song
"ticket stubs", the library genre list, and drill status), `POST /passports/commit`
(instrument commitment), `POST /passports/open` (open a genre
"ticket stubs", the library genre list, and drill status), `POST
/passports/commit` (instrument commitment), `POST /passports/open` (open a genre
passport), and `POST /drill-state` (intake for the relayed Virtuoso
`virtuoso.progress` snapshot, so drill requirements can gate badges
server-side). Badges are never stored; the only persisted state (commitments,
@@ -99,42 +23,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
attributed via the existing progression arrangement→instrument mapping;
non-graded instruments (bass, drums) render shown-not-judged — repertoire
without a pass bar, never a false badge denial.
- **Career passports (UI)** — the Career screen gains a Passports tab beside
Venues: a physical per-instrument passport book (embossed leather cover, 3D
page-turn) with a wax-seal commitment ceremony (Stage 0), rubber-stamp badge
slam with ink bleed and deterministic per-genre jitter, qualifying songs as
collected ticket stubs, and unopened genres as an "Explore next"
travel-brochure rack (invitations, never greyed-out slots or completion
meters). Badge earns chime + notify immediately; the stamp slam plays when
the passport is next opened. Four small synthesized sound effects ship as
plugin assets. The career screen also relays the Virtuoso `virtuoso.progress`
localStorage snapshot to the drill-state intake on `virtuoso:progress` bus
events (debounced, plus a one-time bootstrap), closing the
fires-into-a-void seam without touching the virtuoso plugin.
- **CI gate: core must stay faithful to the feedpak spec (`feedpak-spec` job).** feedpak is published as
an open format with its own repo, normative spec, JSON Schemas, and reference validator — but nothing
stopped core from reading a manifest key the spec never defined, which is exactly what happened with
`original_audio` (#583#933). `tools/check_spec_conformance.py` now enforces four surface properties
in CI: (1) **key-coverage** — every manifest key core reads *or writes* is declared in the spec's
`manifest.schema.json`, found by walking the AST of `lib/sloppak.py`, `lib/enrichment.py`, and
`lib/songmeta.py`, `lib/gp2notation.py`, and `lib/routers/ws_highway.py` (writes are gated too — including
`setdefault()` — and reported separately: a key core writes lands in every pack we emit, so an undeclared
one seeds the ecosystem with non-spec data; a **readers-complete** guard fails the build if that module
list falls behind the codebase); (2) **allowlist-closed**`feedpak-spec-exceptions.yml` never grows;
(3) **forward** — core's `load_song()` ingests every example pack the spec ships;
(4) **reverse** — every pack committed here passes the spec's own `tools/validate.py` (7/7 pass today).
The gate verifies against the spec repo's **HEAD** — the app must conform to the living spec, and the
flow is self-serve: a gated PR opens a FEP, the spec PR merges, re-running checks goes green. Nothing to
pin, nothing to bump. Each run logs the spec SHA it verified against so results are reproducible.
**There is no in-repo escape hatch, by design.** A blocked PR has exactly one route: land the key in the
spec via the [FEP process](https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md), then
re-run the PR's checks — the gate verifies against the spec's HEAD, so it goes green once the key is real. `feedpak-spec-exceptions.yml` is a **closed
grandfather list** for keys that predate the gate, not a bypass: a fourth check (**allowlist-closed**)
diffs it against the base branch and fails any PR that *adds* an entry, so it may only shrink.
`original_audio` is grandfathered there against #933 so the gate lands green and starts blocking the
*next* instance immediately; the gate takes no position on how #933 resolves (the expected outcome is
removing the key, since the spec already carries the mixdown as a stem — not adopting it). Docs:
[docs/feedpak-spec-gate.md](docs/feedpak-spec-gate.md).
### Removed
- **The classic v2 UI shell is gone — v3 is the only UI (R3a).** `static/index.html`, the
@@ -156,11 +44,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
engine (`app.js`, `highway.js`, `playSong`, `showScreen`, the capability registry).
### Fixed
- **Career passports review polish** — the passport tabs and book overlay carry
proper ARIA semantics (`aria-selected`/`aria-controls`/`tabpanel`;
`role="dialog"` + `aria-modal` with focus moved to the close button on open
and restored on close), and a corrupt stored seen-badges value (e.g. a stray
`"null"`) can no longer throw on every passport refresh.
- **The packaged desktop app could not start (`ModuleNotFoundError: No module named
'appstate'`).** feedback-desktop's `scripts/bundle-slopsmith.sh` copies a *hardcoded
list* of core files into the app bundle — `server.py`, `VERSION`, `lib/`, `data/`,
-15
View File
@@ -588,21 +588,6 @@ tab, key/scale annotations, etc.). Published as **feedpak**; this codebase still
**sloppak** name internally — same on-disk format. [docs/sloppak-spec.md](docs/sloppak-spec.md) is
a local pointer + code map.
**The spec is sacrosanct — read it BEFORE changing how this app reads or writes packs.** The
spec repo defines the format; this app merely implements it ("a change is not part of the format
until it lands here" — feedpak-spec/GOVERNANCE.md). Any new manifest key, file, or directory the
app touches must land in the spec **first**, via the
[FEP process](https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md) (proposal
issue → one spec PR updating spec + schemas + example + changelog → then re-run your PR's checks
here; the gate verifies against the spec's HEAD, so it goes green the moment your key is real).
CI enforces this: the `feedpak-spec` job
([docs/feedpak-spec-gate.md](docs/feedpak-spec-gate.md)) fails any PR whose code touches a
manifest key the spec doesn't declare, and there is **no in-repo bypass** — the exceptions
file is a closed grandfather list that only shrinks. If the format seems to be missing something
you need, that's a FEP conversation, not a workaround. (Cautionary tale: `original_audio`, #933 —
shipped without a spec entry, and third-party packers reverse-engineered a folder convention out
of a code comment.)
**Key code:**
- `lib/sloppak.py` — format detection, zip/directory resolution, metadata extraction, song loading
- `lib/sloppak_convert.py` — sloppak assembly pipeline, Demucs stem splitting
-132
View File
@@ -1,132 +0,0 @@
# The feedpak spec-conformance gate
`tools/check_spec_conformance.py`, run in CI as the `feedpak-spec` job.
## Why
feedpak is published as an **open format**: its own repo
([got-feedback/feedpak-spec](https://github.com/got-feedback/feedpak-spec)), a normative spec, JSON
Schemas, and a reference validator. That is a promise to everyone outside this codebase — third-party
packers, converters, and players build against the spec, and the spec is meant to be the complete and
authoritative description of a pack.
The moment core reads a manifest key the spec doesn't define, that promise breaks silently:
- A spec-compliant pack is no longer guaranteed to be a fully-working pack.
- The reference validator can't warn authors about a key it has never heard of — it will happily green-light
the key, and every misspelling of it.
- The format's real definition drifts into our source tree. In the case that motivated this gate
([#933](https://github.com/got-feedback/feedback/issues/933)), third-party tooling started emitting an
`original/` directory that no code anywhere requires — the convention was reverse-engineered from an
example in a *code comment*.
The rule this gate enforces: **any manifest key core reads _or writes_ must be in the spec before core
ships code that depends on it.** Spec first, implementation second. Writes are not exempt — a key core
writes lands in every pack we emit, so an undeclared one seeds the ecosystem with non-spec data.
Note that "get it into the spec" is not automatically the right fix for an existing violation — for
`original_audio` it isn't. The spec already carries the pre-separation mixdown as a stem
(`{id: full, file: stems/full.ogg}`), so that key added a *second, redundant* location for audio to a format
that already had one, and the resolution is to remove it rather than bless it. The gate takes no position on
which way a violation resolves; it only insists that one of the two happens deliberately, in the open,
before the code merges.
## What it checks
We can't mechanically prove core *interprets* a key the way the spec means. We can prove four surface
properties, and they cover the drift that actually occurs.
| Layer | Check | Catches |
|---|---|---|
| 1. key-coverage | Every manifest key core reads **or writes** is declared in the spec's `manifest.schema.json`. | Core growing a key the spec never defined — the #933 class. |
| 2. allowlist-closed | `feedpak-spec-exceptions.yml` has not **grown** relative to the base branch. | Someone routing around the FEP process by allowlisting their own new key. |
| 3. forward | Core's `load_song()` ingests every example pack the spec ships. | The spec adding or tightening something core ignores or breaks on. |
| 4. reverse | Every pack committed to this repo passes the spec's `tools/validate.py`. | Core (or a contributor) committing a pack the spec would reject. |
Layer 1 works by walking the AST of the modules listed in `READERS` and collecting every literal key touched
on a manifest dict (`manifest.get("x")`, `manifest["x"]`, and the wrapped
`(load_manifest(p) or {}).get("x")` form used in `lib/enrichment.py`).
**Reads and writes are both checked, and reported differently.** A key core *writes*
(`manifest["x"] = v`, as `lib/songmeta.py` does) is spec surface pointed outward: it puts a key into every
pack we emit, so an undeclared one seeds the ecosystem with non-spec data. Subscripts are classified by AST
context — `Store` is a write, `Load` is a read — so `manifest["year"] = ...` is not miscounted as a read.
## When it fails
You added a manifest key the spec doesn't define. **There is exactly one way forward, and it is not in this
repo.**
Land the key in the spec through the **feedpak Enhancement Proposal (FEP)** process
([feedpak-spec/CONTRIBUTING.md](https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md)):
1. **Open a FEP issue** on `got-feedback/feedpak-spec` — the problem, the proposed on-disk shape (manifest
key and/or side-file), backward compatibility, and the version bump it implies.
2. **Discuss**, until it has a clear shape and rough consensus.
3. **Land one PR there** that updates the normative spec (`spec/feedpak-v1.md`), the relevant JSON
Schema(s), an example in `examples/` that exercises it, and the changelog — *together*. A PR touching
only one of those is incomplete.
4. **Back here**, just re-run your PR's checks. The gate verifies against the spec's HEAD, so the moment
your key is genuinely part of the format, your PR goes green — nothing to bump, nothing to maintain.
That's deliberately the only route — no experimental prefix, no self-serve allowlist — and it's usually a
quick one for additive keys. The reason it's worth the round-trip: the gate checks the whole repo against
the living spec, so if non-conformance ever lands, it shows up as red CI on *every* teammate's open PR, and
only the person who introduced it can clear it. Going through the FEP keeps your change clean and keeps
everyone else unblocked.
The spec's own governance says the same thing:
> This repository defines the format only. Applications that read or write feedpak ... track this spec as a
> dependency; they do not drive it. **A change is not part of the format until it lands here.**
> — [feedpak-spec/GOVERNANCE.md](https://github.com/got-feedback/feedpak-spec/blob/main/GOVERNANCE.md)
### `feedpak-spec-exceptions.yml` is a closed grandfather list, not a hatch
It exists solely because `original_audio` predates the gate. **CI fails any PR that adds an entry** (layer 2
diffs it against the base branch), so the list can only ever shrink. Entries are debt, each carries a
tracking issue, and each disappears when the underlying key is removed from core. The gate also fails on a
*stale* entry — the spec caught up, or core stopped touching the key — so the file cannot quietly become
somewhere drift accumulates.
Deleting an entry does not, by itself, get you past the gate: layer 1 still fails while core reads the key.
The entry goes when the **code** goes.
## Tracking the spec's HEAD
The gate checks out `feedpak-spec` at **HEAD**, on purpose: the app must conform to the *living* spec, and
nobody should have to maintain a pin. The dev flow is fully self-serve — gated PR → FEP → spec merge →
re-run checks → green.
Two properties to know about:
- **The normal FEP is additive** (a new optional key), which only ever makes the gate *looser* — it cannot
redden anyone's PR. Only a **breaking** spec change (removing/renaming a key the app uses, tightening the
validator against committed packs) turns PRs red repo-wide — and per the spec's compatibility policy that
is a rare, deliberate MAJOR event, exactly when an org-wide "the app is out of conformance" signal is the
right outcome. The CI job logs the exact spec SHA each run verified against, so a red run is reproducible.
- **CI results can change over time on the same commit** — that is inherent to tracking a living contract,
and it is the point: green means "conformant *now*", not "conformant when written".
## Limitations
Known, and worth fixing in follow-ups rather than blocking on:
- **Layer 1's receiver detection is heuristic.** Locals *assigned from* `load_manifest(...)` are discovered
flow-aware whatever they're called (chart.py's `m` taught us that), and the inline
`(load_manifest(p) or {}).get(...)` form is recognised — but a manifest that arrives as a **function
parameter** is only recognised by name (`MANIFEST_VARS`: `manifest`, `mf`). A parameter called something
else would slip. The hardening step is to route all manifest access through a single declared
`KNOWN_MANIFEST_KEYS` registry in `lib/sloppak.py`; the gate then compares registry against schema exactly
instead of inferring.
- **Layer 1 covers top-level keys only.** Nested structure (`arrangements[].file`, `.id`, `.notation`) isn't
checked. Extending to it means walking the schema's `$ref` subschemas.
- **Layer 1 recognises `get`, `setdefault`, subscripts, and the known gap-fill helper** as key access.
`update()` and `pop()` aren't used against a feedpak manifest anywhere in the tree, so they're deliberately
not special-cased rather than speculatively handled. `readers-complete` reuses the same scanner
(`keys_touched()`), so this blind spot is shared, not doubled: a module using only unrecognised access forms
would evade both.
- **Layer 4 can't catch unknown keys**, because `manifest.schema.json` sets `additionalProperties: true` and
the reference validator deliberately "treats unknown keys/files as forward-compatible". Fixing this
properly belongs in the spec (tighten the schema, or give the validator a `--strict` mode). Until then,
layer 1 is the only thing standing between us and the next `original_audio`.
-56
View File
@@ -1,56 +0,0 @@
# CLOSED grandfather list — manifest keys core reads or writes that predate the
# spec-conformance gate and that the feedpak spec does not define.
#
# Please don't add entries here — CI will flag any PR that grows this list, so
# it can only shrink over time. That's by design, not distrust: the moment the
# app touches a key the spec doesn't define, every teammate's PR starts failing
# the conformance gate too, and whoever added the key is the only person who
# can fix it. The FEP process below avoids putting anyone in that spot. The
# feedpak spec's own governance is explicit:
#
# "This repository defines the format only. Applications that read or write
# feedpak ... track this spec as a dependency; they do not drive it.
# A change is not part of the format until it lands here."
# — got-feedback/feedpak-spec, GOVERNANCE.md
#
# So a new manifest key goes through the feedpak Enhancement Proposal (FEP)
# process — see feedpak-spec/CONTRIBUTING.md:
#
# 1. Open a FEP issue on got-feedback/feedpak-spec describing the problem, the
# on-disk shape, backward compatibility, and the version bump implied.
# 2. Land one PR there updating the normative spec, the JSON Schemas, an
# example that exercises it, and the changelog — together.
# 3. Back here, re-run this PR's checks. The gate verifies against the spec's
# HEAD, so once your key is in the spec, the gate goes green.
#
# That's the supported route — and usually a quick one for additive keys. If
# your PR is blocked by this gate, a FEP will get you unblocked properly; an
# entry here won't (CI rejects it).
#
# Entries below exist ONLY because they predate the gate. Each is debt with a
# tracking issue, and each disappears when its issue is fixed. The gate also
# fails if an entry goes stale — the spec caught up, or core no longer reads or
# writes the key — so this file cannot quietly become a place drift hides.
exceptions:
- key: original_audio
issue: https://github.com/got-feedback/feedback/issues/945
reason: >-
Added by #583 (the full mix played while every stem fader sits at unity,
since demucs recombination is lossy). It never went through a FEP and the
spec does not define it — the drift this gate exists to prevent.
#933 fixed the drift: feedpak 1.15.0 RESERVES the stem id `full` for the
complete mixdown (feedpak-spec#53), and core now reads the full mix from
that stem. Nothing depends on this key any more — not the loader, not
lib/enrichment.py, not the stems plugin, and the packer no longer writes it.
What remains is a READ-ONLY deprecated fallback in lib/sloppak.py
(_legacy_full_mix), kept for one release because every pack produced before
the spec caught up carries `original_audio: original/full.ogg` and would
otherwise silently lose its pristine mix. tools/migrate_full_mix_stem.py
rewrites those packs into the spec shape.
This entry disappears with that fallback — tracked by #945, which cannot be
forgotten: the gate fails if the entry goes stale, and deleting the read is
what makes it stale.
+5 -20
View File
@@ -368,12 +368,10 @@ def _acoustid_gate() -> "JSONResponse | None":
def _song_audio_file(filename: str) -> "str | None":
"""Resolve a LIBRARY song (by filename/id) to a local master-audio file for
fingerprinting: a sloppak's complete mixdown, or a loose folder's audio. None
when the song can't be found or carries no mixdown (a pack that kept only its
separated stems — an acoustic fingerprint of one re-summed from them would not
match the recording, so we decline rather than submit a lossy reconstruction).
Mirrors serve_sloppak_file's containment guards so a crafted filename can't
read outside DLC_DIR / the pack."""
fingerprinting: the full-mix `original_audio` extracted from a sloppak, or a
loose folder's audio. None when the song can't be found or ships no full-mix
audio (some packs carry only stems). Mirrors serve_sloppak_file's containment
guards so a crafted filename can't read outside DLC_DIR / the pack."""
dlc = _get_dlc_dir()
if not dlc:
return None
@@ -385,20 +383,7 @@ def _song_audio_file(filename: str) -> "str | None":
canon = resolved.relative_to(dlc.resolve()).as_posix()
except ValueError:
return None
manifest = sloppak_mod.load_manifest(resolved) or {}
# The mixdown is the RESERVED `full` stem (spec §5.3). Unlike playback,
# fingerprinting wants it even when it is the pack's ONLY stem — a
# single-mix pack is exactly the master audio we want to fingerprint —
# so this asks find_full_mix() rather than partition_stems().
stems = manifest.get("stems") or []
full = sloppak_mod.find_full_mix(
[s for s in stems if isinstance(s, dict)]
)
rel = full.get("file") if full else None
# DEPRECATED fallback: packs written before the spec reserved `full` put
# the mixdown behind a top-level `original_audio:` key instead (#933).
if not isinstance(rel, str) or not rel.strip():
rel = manifest.get("original_audio")
rel = (sloppak_mod.load_manifest(resolved) or {}).get("original_audio")
if not isinstance(rel, str) or not rel.strip():
return None
src = sloppak_mod.get_cached_source_dir(canon)
+19 -94
View File
@@ -614,14 +614,6 @@ class MetadataDB:
)
""")
self.conn.execute("CREATE INDEX IF NOT EXISTS idx_song_stats_recent ON song_stats(last_played_at DESC)")
# Cumulative wall-clock play time (career "hours in genre" odometer).
# Fed by the same POST /api/stats the recorder already sends; additive
# + idempotent like every other song_stats change.
try:
self.conn.execute(
"ALTER TABLE song_stats ADD COLUMN seconds_total REAL NOT NULL DEFAULT 0")
except sqlite3.OperationalError:
pass
# Playlists + the reserved "Saved for Later" system playlist. Additive.
self.conn.execute("""
CREATE TABLE IF NOT EXISTS playlists (
@@ -909,9 +901,6 @@ class MetadataDB:
"best_accuracy": max(cur["best_accuracy"] or 0.0, r["best_accuracy"] or 0.0),
"last_score": newer["last_score"], "last_accuracy": newer["last_accuracy"],
"last_position": newer["last_position"],
# Play time is additive: both encodings' hours belong to
# the one canonical song.
"seconds_total": (cur.get("seconds_total") or 0.0) + (r.get("seconds_total") or 0.0),
"last_played_at": newer["last_played_at"], "updated_at": newer["updated_at"],
}
# Atomic swap: clear and reinsert the canonicalized set in one txn.
@@ -1085,31 +1074,16 @@ class MetadataDB:
vals["artist"], vals["title"] = self._romaji_display(filename, vals["artist"], vals["title"])
return vals
# Effective genre precedence: per-song OVERRIDE (Fix-metadata popup)
# scanned pack genre → MusicBrainz enrichment primary genre (matched/manual rows
# only — a 'review'/'failed' candidate's genres could belong to the wrong
# recording). Applied at FILTER/FACET time (like the P4 artist alias) so a
# corrected or enriched genre is browsable. The vast majority of converted
# packs carry no `genres` manifest key, so without the enrichment leg the
# genre facet (and career passports) starve on real libraries. The
# correlated subqueries are used ONLY when overrides/enrichment genres
# actually exist; the common case stays on the plain indexed `genre`
# column. Genre stays a library-only overlay (it isn't a write-to-file
# field), so it never touches the pack.
_EFFECTIVE_GENRE_OVERRIDE_SQL = (
"COALESCE((SELECT o.value FROM song_field_override o "
"WHERE o.filename = songs.filename AND o.field = 'genre' "
"AND o.value IS NOT NULL AND o.value != ''), genre)"
)
# Effective genre = a per-song genre OVERRIDE (Fix-metadata popup) else the
# scanned pack genre. Applied at FILTER/FACET time (like the P4 artist alias)
# so a corrected genre is browsable — the correlated subquery is used ONLY
# when genre overrides actually exist; the common case stays on the plain
# indexed `genre` column. Genre stays a library-only overlay (it isn't a
# write-to-file field), so it never touches the pack.
_EFFECTIVE_GENRE_SQL = (
"COALESCE((SELECT o.value FROM song_field_override o "
"WHERE o.filename = songs.filename AND o.field = 'genre' "
"AND o.value IS NOT NULL AND o.value != ''), "
"NULLIF(genre, ''), "
"(SELECT json_extract(e.genres, '$[0]') FROM song_enrichment e "
"WHERE e.filename = songs.filename AND e.match_state IN ('matched', 'manual') "
"AND e.genres IS NOT NULL AND e.genres NOT IN ('', '[]')), "
"'')"
"AND o.value IS NOT NULL AND o.value != ''), genre)"
)
def _has_genre_overrides(self) -> bool:
@@ -1117,25 +1091,9 @@ class MetadataDB:
"SELECT 1 FROM song_field_override WHERE field = 'genre' "
"AND value IS NOT NULL AND value != '' LIMIT 1").fetchone() is not None
def _has_enrichment_genres(self) -> bool:
try:
return self.conn.execute(
"SELECT 1 FROM song_enrichment WHERE match_state IN ('matched', 'manual') "
"AND genres IS NOT NULL AND genres NOT IN ('', '[]') "
"LIMIT 1").fetchone() is not None
except sqlite3.OperationalError:
return False # stand-ins / DBs without the enrichment table
def _effective_genre_expr(self) -> str:
"""`genre` normally; the enrichment-aware COALESCE only when trusted
enrichment genres exist (which also proves the table exists — a
stand-in DB without song_enrichment must never receive SQL that
references it); the override-only form when just overrides exist."""
if self._has_enrichment_genres():
return self._EFFECTIVE_GENRE_SQL
if self._has_genre_overrides():
return self._EFFECTIVE_GENRE_OVERRIDE_SQL
return "genre"
"""`genre` normally; the override-aware COALESCE only when overrides exist."""
return self._EFFECTIVE_GENRE_SQL if self._has_genre_overrides() else "genre"
def set_song_tags(self, filename: str, tags) -> list:
"""Replace ALL of a song's tags with the given set (each normalized;
@@ -1735,8 +1693,7 @@ class MetadataDB:
# ── Per-song practice stats ───────────────────────────────────────────---
_STATS_COLS = (
"filename", "arrangement", "plays", "best_score", "best_accuracy",
"last_score", "last_accuracy", "last_position", "seconds_total",
"last_played_at", "updated_at",
"last_score", "last_accuracy", "last_position", "last_played_at", "updated_at",
)
def _stats_row(self, filename: str, arrangement: int) -> dict | None:
@@ -2103,9 +2060,8 @@ class MetadataDB:
self.conn.commit()
def record_session(self, filename: str, arrangement: int, *, score: int,
accuracy: float, last_position=None, seconds: float = 0) -> dict:
"""Record a scored play: plays += 1, best_* = max, last_* = new.
`seconds` (wall-clock play time from the recorder) accrues."""
accuracy: float, last_position=None) -> dict:
"""Record a scored play: plays += 1, best_* = max, last_* = new."""
from song_score import merge_stats
with self._lock:
existing = self._stats_row(filename, int(arrangement))
@@ -2115,9 +2071,8 @@ class MetadataDB:
self.conn.execute(
"""INSERT INTO song_stats
(filename, arrangement, plays, best_score, best_accuracy,
last_score, last_accuracy, last_position, seconds_total,
last_played_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?,
last_score, last_accuracy, last_position, last_played_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?,
strftime('%Y-%m-%d %H:%M:%f','now'), strftime('%Y-%m-%d %H:%M:%f','now'))
ON CONFLICT(filename, arrangement) DO UPDATE SET
plays = excluded.plays,
@@ -2126,62 +2081,32 @@ class MetadataDB:
last_score = excluded.last_score,
last_accuracy = excluded.last_accuracy,
last_position = excluded.last_position,
seconds_total = song_stats.seconds_total + excluded.seconds_total,
last_played_at = excluded.last_played_at,
updated_at = excluded.updated_at""",
(filename, int(arrangement), merged["plays"], merged["best_score"],
merged["best_accuracy"], merged["last_score"], merged["last_accuracy"],
merged["last_position"], float(seconds or 0)),
merged["last_position"]),
)
self.conn.commit()
return self._stats_row(filename, int(arrangement))
def touch_position(self, filename: str, arrangement: int, last_position: float,
seconds: float = 0) -> dict:
def touch_position(self, filename: str, arrangement: int, last_position: float) -> dict:
"""Persist just the resume position (no plays/score change), so
Continue-Playing works for non-scored plays. Also stamps
last_played_at — both /api/stats/recent and /api/session/continue
filter/order on it, so a position-only touch must set it or the song
never surfaces as 'recent' / 'continue playing'. `seconds` accrues
wall-clock play time (career hours odometer)."""
never surfaces as 'recent' / 'continue playing'."""
with self._lock:
self.conn.execute(
"""INSERT INTO song_stats (filename, arrangement, last_position,
seconds_total, last_played_at, updated_at)
VALUES (?, ?, ?, ?, strftime('%Y-%m-%d %H:%M:%f','now'),
strftime('%Y-%m-%d %H:%M:%f','now'))
ON CONFLICT(filename, arrangement) DO UPDATE SET
last_position = excluded.last_position,
seconds_total = song_stats.seconds_total + excluded.seconds_total,
last_played_at = excluded.last_played_at,
updated_at = excluded.updated_at""",
(filename, int(arrangement), float(last_position), float(seconds or 0)),
)
self.conn.commit()
return self._stats_row(filename, int(arrangement))
def add_play_seconds(self, filename: str, arrangement: int, seconds: float) -> dict:
"""Accrue wall-clock play time (no plays/score/position change) —
the recorder's seconds-only flush for unscored plays that ran to the
song's natural end (no resume position to touch there: `song:ended`
must not overwrite Continue with the end-of-song offset). Stamps
last_played_at like touch_position does: the song WAS played, so
/api/stats/recent and Continue ordering must see it. Accepted skew:
the recorder retries FAILED flushes later, which stamps recency at
retry time — rare (offline corner), self-healing on the next play,
and preferable to the alternative (keep-existing would leave repeat
plays looking stale, the common case)."""
with self._lock:
self.conn.execute(
"""INSERT INTO song_stats (filename, arrangement, seconds_total,
last_played_at, updated_at)
VALUES (?, ?, ?, strftime('%Y-%m-%d %H:%M:%f','now'),
strftime('%Y-%m-%d %H:%M:%f','now'))
ON CONFLICT(filename, arrangement) DO UPDATE SET
seconds_total = song_stats.seconds_total + excluded.seconds_total,
last_position = excluded.last_position,
last_played_at = excluded.last_played_at,
updated_at = excluded.updated_at""",
(filename, int(arrangement), float(seconds)),
(filename, int(arrangement), float(last_position)),
)
self.conn.commit()
return self._stats_row(filename, int(arrangement))
+2 -34
View File
@@ -76,22 +76,6 @@ def api_record_stats(data: dict):
last_pos = data.get("lastPlayPosition", data.get("last_position"))
if isinstance(last_pos, bool): # float(False)=0.0 would otherwise store a bogus position
return JSONResponse({"error": "lastPlayPosition must be a finite number"}, status_code=400)
# Optional wall-clock play time (career hours odometer). Bounded per POST:
# the recorder flushes on pause/stop/end, so a single delta beyond 6h is a
# clock artifact (suspend/sleep), not practice.
seconds = data.get("seconds")
if seconds is not None:
if isinstance(seconds, bool):
return JSONResponse({"error": "seconds must be a positive number"}, status_code=400)
try:
seconds = float(seconds)
if not math.isfinite(seconds):
raise ValueError("non-finite")
except (TypeError, ValueError, OverflowError):
return JSONResponse({"error": "seconds must be a positive number"}, status_code=400)
if not (0 < seconds <= 6 * 3600):
return JSONResponse({"error": "seconds must be between 0 and 21600"}, status_code=400)
seconds = seconds or 0.0
# A scored session needs BOTH score and accuracy. Exactly one provided is
# ambiguous — don't silently fall through to the position-only branch.
@@ -131,8 +115,7 @@ def api_record_stats(data: dict):
except (TypeError, ValueError, OverflowError):
return JSONResponse({"error": "lastPlayPosition must be a finite number"}, status_code=400)
row = appstate.meta_db.record_session(filename, arrangement, score=score,
accuracy=accuracy, last_position=last_pos,
seconds=seconds)
accuracy=accuracy, last_position=last_pos)
# Unified XP + streak side-effects — never let these drop the stat write.
progress = None
try:
@@ -169,21 +152,6 @@ def api_record_stats(data: dict):
log.warning("stats side-effects (progression) failed", exc_info=True)
return {"stats": row, "progress": progress, "progression": progression_summary}
# Seconds-only accrual: an unscored play that ran to the song's natural
# end has play time to bank but no resume position to touch (song:ended
# must not overwrite Continue with the end-of-song offset). Still counts
# as playing today for the streak below.
if last_pos is None and seconds:
row = appstate.meta_db.add_play_seconds(filename, arrangement, seconds)
progress = None
try:
from datetime import date
appstate.meta_db.record_active_day(date.today().isoformat())
progress = appstate.meta_db.get_progress()
except Exception:
log.warning("stats side-effects (streak) failed", exc_info=True)
return {"stats": row, "progress": progress}
# Position-only touch.
if last_pos is None:
return JSONResponse(
@@ -194,7 +162,7 @@ def api_record_stats(data: dict):
pos = float(last_pos)
if not math.isfinite(pos):
raise ValueError("non-finite")
row = appstate.meta_db.touch_position(filename, arrangement, pos, seconds=seconds)
row = appstate.meta_db.touch_position(filename, arrangement, pos)
except (TypeError, ValueError, OverflowError):
return JSONResponse({"error": "lastPlayPosition must be a finite number"}, status_code=400)
# A resume session still counts as playing today: advance the streak (no XP —
+19 -43
View File
@@ -321,16 +321,11 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
audio_url = None
audio_error: str | None = None # Surfaced in song_info when audio_url is None
stems_payload: list[dict] = []
# URL of the pack's complete mixdown — the RESERVED `full` stem (spec
# §5.3), which sloppak.load_song() lifts out of `stems` because it is a
# mixdown, not a layer. The stems plugin plays it while every stem slider
# is at unity (separation is lossy, so it beats re-summing the stems) and
# crosses to the separated stems as soon as one is attenuated.
#
# None when the pack has no mixdown to offer separately from its stems:
# a single-mix pack (its one stem IS the mixdown), a loose folder, or an
# archive.
full_mix_url: str | None = None
# URL of the single full-mix audio (sloppak `original_audio:`), when the
# pack ships one. The stems plugin uses this to play the untouched mix
# while every stem slider is at unity; None otherwise (separate stems
# only, loose folder, or archive).
original_audio_url: str | None = None
if is_loose:
# Loose folder filenames are relative paths (artist/album/song).
# Hash the *canonical* dlc-relative path (so two URL spellings
@@ -370,25 +365,21 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
url = f"/api/sloppak/{q_fn}/file/{quote(s['file'])}"
stems_payload.append({"id": s["id"], "url": url, "default": s["default"]})
# Full-mix URL (served by the same /api/sloppak/.../file/ endpoint).
if loaded_slop is not None and loaded_slop.full_mix:
full_mix_url = (
f"/api/sloppak/{q_fn}/file/{quote(loaded_slop.full_mix)}"
if loaded_slop is not None and loaded_slop.original_audio:
original_audio_url = (
f"/api/sloppak/{q_fn}/file/{quote(loaded_slop.original_audio)}"
)
if stems_payload:
# Stems present: keep the core <audio> pointed at stem[0]. This
# URL is only ever heard in the degraded path (stems plugin
# refuses takeover / decode fails); the full-mix↔stems switch is
# driven client-side by `full_mix_url`, not `audio_url`.
# driven client-side by `original_audio_url`, not `audio_url`.
audio_url = stems_payload[0]["url"]
elif full_mix_url:
elif original_audio_url:
# Stem-less full-mix pack: nothing to separate, so play the full
# mix natively through the core <audio>. The stems plugin's
# onSongReady returns early on an empty stems list (no graph).
# Reachable only via the deprecated `original_audio:` key, whose
# packs put the mixdown outside `stems` — a pack that carries its
# mixdown as the `full` stem has it IN `stems`, so it lands in the
# branch above with stems_payload == [full].
audio_url = full_mix_url
audio_url = original_audio_url
else:
audio_error = "This sloppak has no playable stems."
else:
@@ -530,31 +521,16 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
# for the credits overlay, so minigames / synthetic highway uses
# (no manifest) never trigger it.
"authors": _sanitize_authors(loaded_slop.manifest) if (is_slop and loaded_slop is not None) else [],
# Instrument stems ONLY. The pack's complete mixdown (the RESERVED
# `full` stem, spec §5.3) is deliberately NOT in this list: consumers
# sum `stems` into one mix and render one fader per entry, and the
# mixdown is neither a layer nor an instrument — summing it would
# double the whole song. It is surfaced separately, below.
"stems": stems_payload,
# The complete mixdown, served by the same /api/sloppak/.../file/
# endpoint as the stems. The stems plugin plays this single file
# while every stem slider is at unity and crosses to the separated
# stems the moment one drops below 100% — separation is lossy, so the
# mixdown is strictly better audio when nothing is muted. None when
# the pack has no mixdown apart from its stems. The `has_*` flags
# mirror the has_drum_tab/has_keys convention so a client can branch
# without re-deriving from the URLs.
"full_mix_url": full_mix_url,
"has_full_mix": bool(full_mix_url),
# Full-mix audio (sloppak `original_audio:`) served alongside the
# separate `stems`. The stems plugin plays this single file while
# every stem slider is at unity and switches to the separate stems
# the moment one drops below 100%. None when the pack ships stems
# only. `has_*` flags mirror the has_drum_tab/has_keys convention so
# a client can branch without re-deriving from the URLs.
"original_audio_url": original_audio_url,
"has_original_audio": bool(original_audio_url),
"has_stems": bool(stems_payload),
# DEPRECATED aliases of the two keys above, kept so a client built
# against the old frame keeps working across one release. They were
# named after `original_audio:` — a manifest key this repo invented
# and the feedpak spec never had (#933). The key is gone; the mixdown
# is a stem. Remove these once the shipped stems plugin reads
# `full_mix_url` (#945).
"original_audio_url": full_mix_url,
"has_original_audio": bool(full_mix_url),
# Surface a drum_tab presence flag so the visualization picker
# can auto-activate the drums plugin even when the chosen
# arrangement isn't named "Drums" (drum_tab.json lives next
+61 -409
View File
@@ -15,7 +15,6 @@ from __future__ import annotations
import logging
import math
import os
import shutil
import threading
import zipfile
@@ -35,21 +34,6 @@ FEEDPAK_EXT = ".feedpak"
SLOPPAK_EXT = ".sloppak"
SONG_EXTS = (FEEDPAK_EXT, SLOPPAK_EXT) # accepted on read/discovery
# ── The full mix ──────────────────────────────────────────────────────────────
#
# Spec §5.3 RESERVES the stem id `full` for the song's complete mixdown: the
# whole song in one file, as heard before source separation. It is a stem — it
# lives in `stems` like every other audio file in a pack — but it is a *mixdown,
# not a layer*. A reader that sums stems must never include it in the sum: it
# already contains every instrument, so summing it doubles the whole song and
# muting `guitar` still leaves guitar audible inside it.
#
# Keeping it matters because separation is lossy: re-summing guitar+bass+drums+
# vocals does NOT reproduce the file they came from. The mixdown is the only
# faithful rendering of the song a pack can carry, so we play it whenever every
# stem sits at unity and nothing is muted.
FULL_MIX_STEM_ID = "full"
import yaml
from jsonc import load_json
@@ -67,97 +51,6 @@ import drums as drums_mod
import notation as notation_mod
def find_full_mix(stems: list[dict]) -> dict | None:
"""The RESERVED `full` stem (spec §5.3) — the pack's complete mixdown — or None.
Answers "what is this pack's master audio", which is what fingerprinting
wants. For playback use partition_stems() instead: a pack whose *only* stem
is `full` has no mixdown to play *separately from* its stems, and this
function still returns it.
"""
return next(
(s for s in stems if str(s.get("id", "")) == FULL_MIX_STEM_ID), None
)
def partition_stems(stems: list[dict]) -> tuple[dict | None, list[dict]]:
"""Split stem descriptors into (mixdown, instrument_stems) for PLAYBACK.
The mixdown is lifted OUT of the stem list because every consumer of `stems`
treats that list as layers to sum or to show as mixer channels, and `full` is
neither (spec §5.3). Leaving it in is precisely the bug that made the packer
invent `original_audio` in the first place: a listed full mix plays on top of
the stems.
A pack whose only stem is `full` is a single-mix pack, not a separated one:
there are no instruments to be pristine *against*, so `full` stays the sole
playable stem and no mixdown is surfaced. That keeps the freshly-converted
single-stem pack much the most common shape behaving exactly as before.
EVERY entry with the reserved id is removed, not just the one we surface. A
malformed pack that lists `full` twice would otherwise leave a copy of the
whole song behind in the stem list, to be summed with the instruments the
precise failure this function exists to prevent, reintroduced by a duplicate.
"""
if len(stems) < 2:
return None, stems
full = find_full_mix(stems)
if full is None:
return None, stems
return full, [s for s in stems if str(s.get("id", "")) != FULL_MIX_STEM_ID]
def _legacy_full_mix(manifest: dict, source_dir: Path) -> str | None:
"""Full mix from the DEPRECATED `original_audio:` manifest key, or None.
Before feedpak 1.15.0 reserved `full`, §5.3 said the mixdown was "commonly
replaced" by the per-instrument stems on splitting — so it had nowhere to
live, and this repo invented a top-level key pointing at a parallel
`original/` directory (#583) to hold it. That key was never in the spec, and
#933 removed our dependence on it: the mixdown is a stem.
We still READ it, because every pack written before the spec caught up
carries `original_audio: original/full.ogg` and would otherwise lose its full
mix. We never write it. Delete this once those packs are migrated (#945);
`tools/migrate_full_mix_stem.py` is the migration.
NOTE the string literal below. tools/check_spec_conformance.py AST-scans for
`manifest.get("<literal>")` to prove every manifest key core reads is one the
spec declares. Hoisting "original_audio" into a named constant would hide
this read from that scan the gate would conclude core no longer touches the
key, and the grandfather entry that documents this debt would go stale. The
literal is what keeps the deprecation honest and visible to CI. Leave it.
Same permissive, path-traversal-guarded posture as the optional side-files: a
missing / escaping / unreadable file leaves the pack without a full mix (the
player falls back to the separated stems) rather than aborting the load.
Returns the manifest-relative string, so callers build its URL exactly as
they build a stem's.
"""
rel_raw = manifest.get("original_audio")
if not isinstance(rel_raw, str) or not rel_raw.strip():
return None
rel = rel_raw.strip()
try:
target = (source_dir / rel).resolve()
target.relative_to(source_dir.resolve())
except ValueError:
log.warning("sloppak: original_audio path %r escapes source_dir — skipped", rel)
return None
except OSError as e:
log.warning("sloppak: original_audio path resolution failed (%s) — skipped", e)
return None
if not target.is_file():
return None
log.info(
"sloppak: pack uses the deprecated `original_audio:` key (%r) — the full mix "
"is a stem (id `full`, feedpak spec §5.3). Re-pack with "
"tools/migrate_full_mix_stem.py; support for this key will be removed.",
rel,
)
return rel
# ── Format detection ──────────────────────────────────────────────────────────
def is_sloppak(path: Path) -> bool:
@@ -188,116 +81,6 @@ _unpack_semaphore = threading.BoundedSemaphore(_UNPACK_MAX_CONCURRENCY)
_unpack_locks: dict[str, threading.Lock] = {}
_unpack_locks_guard = threading.Lock()
# Destinations with an unpack in flight right now. Eviction MUST skip these: two
# unpacks run concurrently, so one finishing could otherwise rmtree the other's
# half-written directory and leave that resolver caching an incomplete song.
_unpacking: set[Path] = set()
_unpacking_guard = threading.Lock()
# Cap the unpack cache. Stems are already-compressed audio, so an unpacked song
# is ~1.1x its zip — the cache is effectively a second, DECOMPRESSED copy of
# every song it holds, and it used to grow without any bound at all. A tester
# reached 60 GB from a 1800-song library: their whole library, unpacked, because
# one caller looped the library calling load_song(). Nothing ever deleted any of
# it — not even when the song itself was deleted.
#
# Default 4 GB ≈ 130 average songs of recency, which is far more than the "the
# song I'm playing, and the last few I played" that this cache actually exists
# to serve. Override with FEEDBACK_SLOPPAK_CACHE_MAX_MB (0 disables eviction).
def _unpack_cache_cap_bytes() -> int:
raw = os.environ.get("FEEDBACK_SLOPPAK_CACHE_MAX_MB", "").strip()
try:
mb = int(raw) if raw else 4096
except ValueError:
mb = 4096
return max(0, mb) * 1024 * 1024
def _dir_size(path: Path) -> int:
total = 0
for f in path.rglob("*"):
try:
if f.is_file():
total += f.stat().st_size
except OSError:
continue
return total
def _touch(path: Path) -> None:
"""Bump mtime so the LRU sweep below treats this song as recently used.
Reading files out of an unpacked dir doesn't change the DIRECTORY's mtime,
so without this the song you are actively playing looks as stale as one you
unpacked days ago and a burst of unpacks could evict it mid-song.
"""
try:
os.utime(path, None)
except OSError:
pass
def _evict_unpack_cache(root: Path, keep: Path | None = None) -> None:
"""Bound the unpack cache: drop least-recently-used songs until under the cap.
`keep` is never evicted it's the song the caller just resolved, i.e. almost
certainly the one about to be played.
Evicting a directory MUST also drop its `_source_cache` entry. Otherwise
get_cached_source_dir() keeps handing out a path that no longer exists and
the media route 404s on every stem instead of re-unpacking (it only falls
back to resolve_source_dir when the cache returns None).
"""
cap = _unpack_cache_cap_bytes()
if cap <= 0:
return
try:
entries = []
total = 0
for d in root.iterdir():
if not d.is_dir():
continue
try:
size = _dir_size(d)
mtime = d.stat().st_mtime
except OSError:
continue
entries.append((mtime, size, d))
total += size
if total <= cap:
return
keep_resolved = keep.resolve() if keep else None
entries.sort(key=lambda e: e[0]) # oldest first
for _mtime, size, d in entries:
if total <= cap:
break
try:
if keep_resolved and d.resolve() == keep_resolved:
continue
except OSError:
continue
# Check-and-delete under ONE hold of the guard. Releasing between the
# two would let a resolver mark this dest in-flight and start writing
# into it in the gap, and we'd rmtree a song mid-unpack. A resolver
# that blocks here simply proceeds afterwards — _unpack_zip recreates
# the directory anyway.
with _unpacking_guard:
if d in _unpacking:
continue # another thread is writing this
shutil.rmtree(d, ignore_errors=True)
if d.exists():
continue # couldn't remove — don't claim the bytes back
total -= size
with _source_lock:
for fn, (cached_dir, _m, _s) in list(_source_cache.items()):
if cached_dir == d:
_source_cache.pop(fn, None)
log.info("sloppak: evicted %s from the unpack cache (%.0f MB)",
d.name, size / 1e6)
except OSError:
log.warning("sloppak: unpack-cache eviction failed", exc_info=True)
def _unpack_lock_for(filename: str) -> threading.Lock:
"""Return a stable per-file lock so concurrent unpacks of the same sloppak
@@ -362,17 +145,10 @@ def resolve_source_dir(
re-unpacks if mtime/size changed, then returns that dir.
Caches the resolution so subsequent calls are ~free.
NOTE: this writes the WHOLE pack every stem to disk. Only call it for a
song you are about to play. To read a *part* of a song (an arrangement, the
lyrics, a tone blob), use read_member_bytes(): unpacking a pack to read a few
KB of JSON is ~45x write amplification, and doing it in a loop over the
library fills the disk with a decompressed copy of every song.
"""
path = dlc_root / filename
stat = path.stat()
mtime, size = stat.st_mtime, stat.st_size
guarded: Path | None = None # a dir WE unpacked, shielded from eviction
with _source_lock:
cached = _source_cache.get(filename)
@@ -383,76 +159,42 @@ def resolve_source_dir(
and cached_size == size
and cached_dir.exists()
):
# Mark it recently-used before returning — see _touch().
if cached_dir != path:
_touch(cached_dir)
return cached_dir
try:
if path.is_dir():
resolved = path
else:
# Zip form — unpack to the cache. Serialize per-file (so concurrent
# callers don't rmtree + re-extract the same dest at once) and cap
# global unpack concurrency (so a burst can't saturate disk/CPU).
dest = unpack_cache_root / _safe_id(filename)
with _unpack_lock_for(filename):
# Re-check the cache inside the per-file lock — a prior holder may
# have just finished unpacking this exact (mtime, size).
with _source_lock:
cached = _source_cache.get(filename)
if (
cached
and cached[1] == mtime
and cached[2] == size
and cached[0].exists()
):
resolved = cached[0]
else:
# Shield `dest` from eviction from the moment we start writing
# until it is safely in _source_cache. `keep` only shields it
# from OUR OWN sweep — a concurrent resolver sweeping with a
# different `keep` would delete it, and we would then cache and
# return a path that no longer exists. The `finally` below
# releases it on EVERY exit, including a failed unpack: leaving
# a dest marked in-flight would make it un-evictable forever.
with _unpacking_guard:
_unpacking.add(dest)
guarded = dest
with _unpack_semaphore:
_unpack_zip(path, dest)
resolved = dest
# The only moment this cache grows. Sweep here rather than on a
# timer so it can never drift far past the cap.
_evict_unpack_cache(unpack_cache_root, keep=dest)
if path.is_dir():
resolved = path
else:
# Zip form — unpack to the cache. Serialize per-file (so concurrent
# callers don't rmtree + re-extract the same dest at once) and cap
# global unpack concurrency (so a burst can't saturate disk/CPU).
dest = unpack_cache_root / _safe_id(filename)
with _unpack_lock_for(filename):
# Re-check the cache inside the per-file lock — a prior holder may
# have just finished unpacking this exact (mtime, size).
with _source_lock:
cached = _source_cache.get(filename)
if (
cached
and cached[1] == mtime
and cached[2] == size
and cached[0].exists()
):
resolved = cached[0]
else:
with _unpack_semaphore:
_unpack_zip(path, dest)
resolved = dest
with _source_lock:
_source_cache[filename] = (resolved, mtime, size)
return resolved
finally:
if guarded is not None:
with _unpacking_guard:
_unpacking.discard(guarded)
with _source_lock:
_source_cache[filename] = (resolved, mtime, size)
return resolved
def get_cached_source_dir(filename: str) -> Path | None:
"""Return the cached source dir for a sloppak if one is known AND still there.
The existence check is load-bearing: callers (media.py) only fall back to
resolve_source_dir() when this returns None, so handing back a path that has
been evicted or that the user deleted by hand to reclaim disk would 404
every stem for the rest of the process instead of re-unpacking.
"""
"""Return the cached source dir for a sloppak if one is known."""
with _source_lock:
cached = _source_cache.get(filename)
if not cached:
return None
src = cached[0]
if not src.is_dir():
_source_cache.pop(filename, None)
return None
_touch(src)
return src
return cached[0] if cached else None
# ── Manifest + song loading ───────────────────────────────────────────────────
@@ -491,82 +233,6 @@ def load_manifest(path: Path) -> dict:
return _read_manifest_from_zip(path)
_ZIP_ROOT = Path("/_root").resolve()
def _zip_member_key(name: str) -> str | None:
"""Canonical lookup key for a zip member name, or None if it escapes the root.
Collapses './', 'a/../b' and backslash separators the same normalization
_unpack_zip()/safe_join() apply when extracting. Both the name the caller asks
for AND the names the archive actually stores must go through this, or a pack
that stores './arrangements/lead.json' unpacks fine but reads back as missing.
"""
safe = safe_join(_ZIP_ROOT, name or "")
# None → escapes the root; == root → a degenerate name like "." or "a/..".
if safe is None or safe == _ZIP_ROOT:
return None
return safe.relative_to(_ZIP_ROOT).as_posix()
def read_member_bytes(path: Path, rel: str) -> bytes | None:
"""Return the bytes of ONE file inside a sloppak, or None if it isn't there.
For a zipped sloppak this opens that single member instead of unpacking the
archive the same trick read_cover_bytes() uses to keep the library grid
from exploding every pack just to show a cover.
Reach for this whenever you want a *part* of a song (an arrangement's JSON,
the lyrics, a tone blob) rather than a song you're about to play. The
alternative, load_song(), calls resolve_source_dir() and writes the WHOLE
pack every stem into the unpack cache. That is a ~45x write amplification
when all you wanted was a few KB of JSON, and looping the library on it
unpacks the entire library (got-feedBack/feedBack: a tester hit 60 GB that
way). Stems are already-compressed audio, so an unpacked song is ~1.1x its
zip: the cache becomes a second, decompressed copy of everything it touches.
"""
rel = (rel or "").strip()
if not rel:
return None
if path.is_dir():
target = safe_join(path.resolve(), rel)
if target is None or not target.is_file():
return None
try:
return target.read_bytes()
except OSError:
return None
# Zip form — read just that member, no unpack. Zip-slip is rejected before we
# open anything, and both sides of the comparison are normalized, so a
# non-canonical-but-valid name ('./arrangements/lead.json') resolves the same
# way it did when we unpacked first.
member = _zip_member_key(rel)
if member is None:
log.warning("sloppak: rejected unsafe member name %r in %r", rel, path)
return None
try:
with zipfile.ZipFile(str(path), "r") as zf:
# Match on the NORMALIZED stored name, and take the LAST match — the
# archive may store './x' or a backslash path (Windows tooling), and
# if it stores two names that normalize to the same file, _unpack_zip
# writes them in order so the last one wins. Reading the raw member by
# exact name would miss the first case and return the wrong bytes in
# the second. A pack has a handful of members; the scan is free.
info = None
for cand in zf.infolist():
if _zip_member_key(cand.filename) == member:
info = cand
if info is None or info.is_dir():
return None
with zf.open(info) as f:
return f.read()
except (zipfile.BadZipFile, OSError, RuntimeError) as e:
log.warning("sloppak: failed to read %r from %s: %s", rel, path.name, e)
return None
_COVER_MEDIA_TYPES = {
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".webp": "image/webp",
@@ -701,21 +367,14 @@ class LoadedSloppak:
# song.arrangements (not to manifest["arrangements"]) — skipped entries are
# absent so indexing by song.arrangements index is safe.
arrangement_ids: list[str | None] = field(default_factory=list)
# Manifest-relative path to the pack's complete mixdown — the whole song in
# one file, as heard before source separation. This is the RESERVED `full`
# stem (spec §5.3), lifted out of `stems` above precisely because it is NOT
# an instrument layer: summing it with the per-instrument stems it was split
# into would double the entire song. See partition_stems().
#
# None when the pack has no mixdown to offer *separately* from its stems —
# which includes the common single-mix pack, whose only stem IS the mixdown
# (there is nothing to be pristine against, so it stays in `stems`).
#
# Served to the front-end via the highway WS as `full_mix_url`; the stems
# plugin plays it while every stem slider sits at unity and crosses to the
# separated stems the moment one drops below 100% — demucs recombination is
# lossy, so the mixdown is strictly the better audio when nothing is muted.
full_mix: str | None = None
# Manifest-relative path to the single full-mix audio file, taken from the
# manifest `original_audio:` key (e.g. "original/full.ogg"). This is the
# pre-separation mixdown that exists alongside the per-instrument `stems`.
# None when the key is absent, points outside source_dir, or the file is
# missing on disk. Served to the front-end via the highway WS as
# `original_audio_url`; the stems plugin uses it to play the untouched mix
# when every stem slider is at unity (and the separate stems otherwise).
original_audio: str | None = None
def load_song(
@@ -1107,13 +766,6 @@ def load_song(
default_on = bool(default_val)
stems.append({"id": sid, "file": sfile, "default": default_on})
# The complete mixdown is a stem (spec §5.3), but it is not a *layer*: lift
# it out so that no consumer of `stems` — the mixer, the library's stem
# chips, the WS payload — sums it with, or lists it beside, the instruments
# it was separated into. `full_mix_stem` is None for a single-mix pack,
# whose only stem IS the mixdown and stays in the list.
full_mix_stem, stems = partition_stems(stems)
# Optional keys.json — song-level, instrument-independent key/scale track
# (manifest `keys:` key, spec §7.7). Permissive like the other side-files:
# missing / unreadable / malformed -> None, never fatal. Stored as a
@@ -1176,22 +828,28 @@ def load_song(
}
_fpv = manifest.get("feedpak_version")
# The pack's full mix. Normally the RESERVED `full` stem partitioned out
# above (spec §5.3) — no path work needed, it was validated with the other
# stems and its URL is built the same way. Only when the pack has no `full`
# stem do we fall back to the DEPRECATED `original_audio:` key, which is the
# shape every pack written before feedpak 1.15.0 uses.
if full_mix_stem is not None:
full_mix_data: str | None = full_mix_stem["file"]
elif find_full_mix(stems) is not None:
# Single-mix pack: its ONE stem is the mixdown, so there is no mixdown to
# offer *apart from* the stems. Never fall through to the legacy key here
# — a pack that both carries a `full` stem and names the old key would
# otherwise surface the mixdown twice (once as the stem the player is
# already playing, once as a "pristine" track to cross to).
full_mix_data = None
else:
full_mix_data = _legacy_full_mix(manifest, source_dir)
# Optional full-mix audio — manifest `original_audio:` key. The single
# pre-separation mixdown that ships alongside the per-instrument stems.
# Same permissive, path-traversal-guarded posture as drum_tab above: a
# missing/escaping/absent file simply leaves the full mix unavailable (the
# player falls back to the separate stems) rather than aborting the load.
# We store the manifest-relative string so server.py can build its URL the
# same way it builds stem URLs (via the /api/sloppak/.../file/ endpoint).
original_audio_data: str | None = None
original_audio_rel = manifest.get("original_audio")
if isinstance(original_audio_rel, str) and original_audio_rel.strip():
rel = original_audio_rel.strip()
try:
oa_path = (source_dir / rel).resolve()
oa_path.relative_to(source_dir.resolve())
except ValueError:
log.warning("sloppak: original_audio path %r escapes source_dir — skipped", rel)
oa_path = None
except OSError as e:
log.warning("sloppak: original_audio path resolution failed (%s) — skipped", e)
oa_path = None
if oa_path is not None and oa_path.is_file():
original_audio_data = rel
return LoadedSloppak(
song=song,
@@ -1206,7 +864,7 @@ def load_song(
keys=keys_data,
notation_by_id=notation_by_id_data,
arrangement_ids=arrangement_ids_acc,
full_mix=full_mix_data,
original_audio=original_audio_data,
)
@@ -1251,7 +909,7 @@ def extract_meta(path: Path) -> dict:
tuning_offsets = _tuning_for_meta(arr_list)
stems_list = manifest.get("stems", []) or []
valid_stems: list[dict] = []
stem_ids: list[str] = []
for s in stems_list:
if not isinstance(s, dict):
continue
@@ -1265,13 +923,7 @@ def extract_meta(path: Path) -> dict:
isinstance(sid, str) and sid
and isinstance(sfile, str) and sfile
):
valid_stems.append({"id": sid, "file": sfile})
# Partition exactly as load_song() does, for the same reason the library
# filter must not lie: `full` is the mixdown, not an instrument (spec §5.3).
# A separated pack that retains it would otherwise offer the user a "full"
# stem chip alongside guitar/bass/drums and count it as a seventh stem.
_full, instrument_stems = partition_stems(valid_stems)
stem_ids = [s["id"] for s in instrument_stems]
stem_ids.append(sid)
stem_count = len(stem_ids)
return {
+2 -4
View File
@@ -101,16 +101,14 @@ def open_midis_to_freqs(midis: list[int], reference_pitch: float = DEFAULT_REFER
def freqs_to_midis(freqs: list[float], reference_pitch: float = DEFAULT_REFERENCE_PITCH) -> list[int] | None:
"""Return absolute open-string MIDI notes for frequencies at the supplied
A4 reference the inverse of open_midis_to_freqs. None if any entry is
non-numeric, non-finite, or non-positive (a provider could hand us
anything; NaN/Infinity would otherwise raise inside int(round(...)) and
500 the /api/tunings endpoint)."""
non-numeric or non-positive (a provider could hand us anything)."""
out: list[int] = []
for f in freqs:
try:
f = float(f)
except (TypeError, ValueError):
return None
if not math.isfinite(f) or f <= 0:
if f <= 0:
return None
out.append(int(round(69 + 12 * math.log2(f / reference_pitch))))
return out
-495
View File
@@ -64,498 +64,3 @@
.career-star-row .song .artist { color: #9ca3af; }
.career-star-row .hint { color: #6b7280; white-space: nowrap; }
.career-star-row .hint.close { color: #22d3ee; }
/* ── Passports (badge journey) ─────────────────────────────────────────── */
.career-tabs {
display: flex;
gap: 0.25rem;
margin-bottom: 1rem;
border-bottom: 1px solid rgba(55, 65, 81, 0.6);
}
.career-tab {
padding: 0.375rem 0.875rem;
font-size: 0.85rem;
color: #9ca3af;
border-bottom: 2px solid transparent;
margin-bottom: -1px;
}
.career-tab:hover { color: #e5e7eb; }
.career-tab.active { color: #fff; border-bottom-color: #06b6d4; }
.pp-instruments { display: flex; flex-wrap: wrap; gap: 0.5rem; }
.pp-inst {
padding: 0.3rem 0.8rem;
border-radius: 999px;
font-size: 0.8rem;
color: #d1d5db;
background-color: rgba(31, 41, 55, 0.7);
border: 1px solid transparent;
}
.pp-inst:hover { background-color: rgba(55, 65, 81, 0.9); }
.pp-inst.active { border-color: #06b6d4; color: #fff; }
.pp-inst.uncommitted { color: #6b7280; border-style: dashed; border-color: rgba(107, 114, 128, 0.5); }
.pp-inst-badges { color: #fbbf24; font-size: 0.7rem; }
.pp-inst-plus { color: #6b7280; }
/* Leather covers per-instrument hue, embossed with layered shadows and a
subtle grain gradient (no image assets). */
.pp-leather-guitar { background: linear-gradient(160deg, #5c2321, #401412); }
.pp-leather-bass { background: linear-gradient(160deg, #1f3252, #131f36); }
.pp-leather-keys { background: linear-gradient(160deg, #1e4034, #122a21); }
.pp-leather-drums { background: linear-gradient(160deg, #3f3f46, #26262b); }
.pp-shelf { display: flex; flex-wrap: wrap; gap: 1rem; align-items: flex-end; }
.pp-cover, .pp-commit-cover {
position: relative;
width: 9.5rem;
height: 13rem;
border-radius: 0.5rem 0.75rem 0.75rem 0.5rem;
box-shadow:
inset 0 0 0 1px rgba(255, 255, 255, 0.06),
inset 0.5rem 0 0.75rem -0.5rem rgba(0, 0, 0, 0.8),
0 6px 16px rgba(0, 0, 0, 0.45);
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 0.35rem;
padding: 0.75rem;
text-align: center;
}
.pp-cover { transition: transform 0.15s ease, box-shadow 0.15s ease; }
.pp-cover:not(.pp-tilt):hover { transform: translateY(-4px) !important; box-shadow: 0 10px 22px rgba(0, 0, 0, 0.55); }
.pp-cover-title {
font-weight: 700;
font-size: 0.85rem;
letter-spacing: 0.14em;
color: rgba(240, 226, 195, 0.92);
text-shadow: 0 1px 0 rgba(0, 0, 0, 0.7), 0 -1px 0 rgba(255, 255, 255, 0.12);
overflow-wrap: anywhere;
}
.pp-cover-inst {
font-size: 0.6rem;
letter-spacing: 0.2em;
text-transform: uppercase;
color: rgba(240, 226, 195, 0.55);
}
.pp-cover-sub {
position: absolute;
bottom: 0.6rem;
font-size: 0.6rem;
color: rgba(240, 226, 195, 0.5);
}
.pp-commit-card {
display: flex;
gap: 1.25rem;
align-items: center;
padding: 1rem;
border-radius: 0.75rem;
border: 1px solid rgba(55, 65, 81, 0.6);
background-color: rgba(31, 41, 55, 0.35);
}
.pp-commit-card .pp-commit-cover { width: 7rem; height: 9.5rem; flex: none; }
.pp-rack { display: grid; gap: 0.75rem; grid-template-columns: repeat(auto-fill, minmax(10.5rem, 1fr)); }
.pp-brochure {
display: flex;
flex-direction: column;
align-items: flex-start;
gap: 0.15rem;
padding: 0.75rem 0.875rem;
border-radius: 0.5rem;
text-align: left;
background: linear-gradient(165deg, rgba(45, 55, 72, 0.55), rgba(31, 41, 55, 0.55));
border: 1px solid rgba(75, 85, 99, 0.5);
transition: transform 0.15s ease, border-color 0.15s ease;
}
.pp-brochure:hover { transform: translateY(-2px); border-color: #06b6d4; }
.pp-brochure-art { font-size: 1.4rem; }
.pp-brochure-name { color: #e5e7eb; font-size: 0.85rem; font-weight: 600; }
.pp-brochure-sub { color: #6b7280; font-size: 0.65rem; }
/* The open book */
.pp-overlay { position: fixed; inset: 0; z-index: 60; }
.pp-book-wrap {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
background: rgba(3, 7, 18, 0.72);
backdrop-filter: blur(2px);
}
.pp-book {
position: relative;
width: min(92vw, 720px);
height: min(72vh, 470px);
perspective: 1800px;
}
.pp-page {
position: absolute;
top: 0;
bottom: 0;
width: 50%;
background:
linear-gradient(105deg, rgba(0, 0, 0, 0.08), transparent 12%),
#efe6d0;
color: #3f3428;
padding: 1.1rem 1.2rem;
overflow: hidden;
display: flex;
flex-direction: column;
gap: 0.75rem;
}
.pp-page-left { left: 0; border-radius: 0.6rem 0 0 0.6rem; opacity: 0; transition: opacity 0.35s ease 0.3s; align-items: center; }
.pp-page-right { right: 0; border-radius: 0 0.6rem 0.6rem 0; box-shadow: inset 0.4rem 0 0.6rem -0.4rem rgba(0, 0, 0, 0.35); }
.pp-book.open .pp-page-left { opacity: 1; }
.pp-book-cover {
position: absolute;
top: 0;
bottom: 0;
left: 50%;
width: 50%;
border-radius: 0 0.6rem 0.6rem 0;
transform-origin: left center;
transform: rotateY(0deg);
backface-visibility: hidden;
transition: transform 0.8s cubic-bezier(0.4, 0.1, 0.2, 1);
z-index: 5;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 0.35rem;
box-shadow: inset 0 0 0 1px rgba(255, 255, 255, 0.06), 0 6px 20px rgba(0, 0, 0, 0.5);
}
.pp-book.open .pp-book-cover { transform: rotateY(-180deg); }
.pp-book-close {
position: absolute;
top: -0.75rem;
right: -0.75rem;
z-index: 8;
width: 2rem;
height: 2rem;
border-radius: 999px;
background: rgba(17, 24, 39, 0.95);
color: #d1d5db;
border: 1px solid rgba(107, 114, 128, 0.5);
}
.pp-book-close:hover { color: #fff; border-color: #06b6d4; }
.pp-page-head {
font-size: 0.7rem;
letter-spacing: 0.18em;
text-transform: uppercase;
color: #8a7a5e;
border-bottom: 1px solid rgba(138, 122, 94, 0.35);
padding-bottom: 0.4rem;
width: 100%;
text-align: center;
}
/* The rubber stamp */
.pp-stamp {
--pp-rot: 0deg;
position: relative;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 0.1rem;
width: 9rem;
height: 9rem;
border-radius: 999px;
border: 3px solid #9a5b16;
box-shadow: inset 0 0 0 3px #efe6d0, inset 0 0 0 4px #9a5b16;
color: #9a5b16;
transform: rotate(var(--pp-rot));
margin-top: 1.25rem;
text-align: center;
padding: 0.75rem;
opacity: 0.92;
}
.pp-stamp-genre { font-size: 0.72rem; font-weight: 800; letter-spacing: 0.16em; overflow-wrap: anywhere; }
.pp-stamp-tier { font-size: 0.58rem; letter-spacing: 0.3em; }
.pp-stamp-ghost {
border-style: dashed;
box-shadow: none;
border-color: #b3a68b;
color: #b3a68b;
opacity: 0.8;
}
.pp-stamp-hidden { opacity: 0; }
.pp-stamp-mini {
position: absolute;
top: 0.5rem;
right: 0.5rem;
width: auto;
height: auto;
border-width: 2px;
box-shadow: none;
border-radius: 999px;
font-size: 0.5rem;
font-weight: 800;
letter-spacing: 0.2em;
color: #d9a253;
border-color: #d9a253;
padding: 0.2rem 0.4rem;
margin: 0;
display: inline-block;
transform: rotate(var(--pp-rot));
opacity: 0.95;
}
.pp-stamp-page::after {
content: '';
position: absolute;
inset: -10%;
border-radius: 999px;
background: radial-gradient(closest-side, rgba(154, 91, 22, 0.25), transparent 72%);
filter: blur(5px);
opacity: 0;
pointer-events: none;
}
.pp-slam { animation: pp-slam 0.5s cubic-bezier(0.2, 0.8, 0.3, 1) forwards; }
.pp-slam::after { animation: pp-ink 0.45s ease-out 0.12s forwards; }
@keyframes pp-slam {
0% { transform: rotate(calc(var(--pp-rot) - 15deg)) scale(2.5); opacity: 0; }
55% { transform: rotate(var(--pp-rot)) scale(0.92); opacity: 1; }
75% { transform: rotate(var(--pp-rot)) scale(1.05); }
100% { transform: rotate(var(--pp-rot)) scale(1); opacity: 0.92; }
}
@keyframes pp-ink {
from { opacity: 0; transform: scale(0.6); }
to { opacity: 1; transform: scale(1); }
}
.pp-shake { animation: pp-shake 0.4s ease-out 0.28s; }
@keyframes pp-shake {
0%, 100% { transform: translate(0, 0) rotate(0); }
25% { transform: translate(2px, 1px) rotate(0.3deg); }
50% { transform: translate(-2px, 2px) rotate(-0.25deg); }
75% { transform: translate(1px, -1px) rotate(0.15deg); }
}
.pp-invite, .pp-snj, .pp-gold-note { font-size: 0.75rem; text-align: center; }
.pp-invite { color: #6d5d40; }
.pp-snj { color: #6d5d40; margin-top: 2rem; font-style: italic; max-width: 15rem; }
.pp-gold-note { color: #a8946d; font-size: 0.62rem; margin-top: 0.5rem; }
.pp-drills { display: flex; flex-direction: column; gap: 0.25rem; font-size: 0.7rem; color: #6d5d40; }
.pp-drill.cleared { color: #4d7c0f; }
/* Ticket stubs */
.pp-stubs { flex: 1; overflow-y: auto; display: flex; flex-direction: column; gap: 0.5rem; padding-right: 0.25rem; }
.pp-stub {
background: #f7f1e3;
border: 1px solid #d8cbaa;
border-left: 2px dashed #b6a98c;
border-radius: 0.25rem 0.4rem 0.4rem 0.25rem;
padding: 0.4rem 0.6rem 0.4rem 0.75rem;
display: grid;
grid-template-columns: auto 1fr;
column-gap: 0.6rem;
align-items: baseline;
box-shadow: 0 1px 2px rgba(63, 52, 40, 0.15);
}
.pp-stub-stars { color: #b8860b; font-size: 0.7rem; letter-spacing: 0.08em; grid-row: span 2; }
.pp-stub-title { font-size: 0.78rem; font-weight: 600; color: #3f3428; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.pp-stub-artist { grid-column: 2; font-size: 0.65rem; color: #6d5d40; }
.pp-stub-meta { grid-column: 2; font-size: 0.6rem; color: #8a7a5e; }
.pp-stub-empty { font-size: 0.72rem; color: #8a7a5e; font-style: italic; padding: 0.75rem 0.25rem; }
/* Wax-seal commitment ceremony */
.pp-ceremony { width: 11rem; height: 15rem; }
.pp-wax {
position: absolute;
bottom: 1.4rem;
display: flex;
align-items: center;
justify-content: center;
width: 3.4rem;
height: 3.4rem;
border-radius: 999px;
background:
radial-gradient(circle at 32% 30%, #d24545 0%, #a41f1f 42%, #7c1414 100%);
box-shadow:
inset 0 0 0 4px rgba(124, 20, 20, 0.9),
inset 0 2px 4px rgba(255, 255, 255, 0.25),
0 3px 8px rgba(0, 0, 0, 0.55);
color: rgba(255, 235, 235, 0.9);
font-weight: 700;
font-size: 1.15rem;
animation: pp-seal-drop 0.9s cubic-bezier(0.25, 0.9, 0.3, 1.15) 0.35s backwards;
}
@keyframes pp-seal-drop {
0% { transform: translateY(-120px) scale(2.1); opacity: 0; }
60% { transform: translateY(0) scale(0.9); opacity: 1; }
80% { transform: translateY(0) scale(1.05); }
100% { transform: translateY(0) scale(1); }
}
/* Small screens: the spread stacks; the flip cover would straddle both
pages, so the book simply opens. */
@media (max-width: 640px) {
.pp-book { height: min(80vh, 620px); }
.pp-page { position: static; width: 100%; height: 50%; border-radius: 0; }
.pp-page-left { border-radius: 0.6rem 0.6rem 0 0; opacity: 1; }
.pp-page-right { border-radius: 0 0 0.6rem 0.6rem; }
.pp-book-cover { display: none; }
.pp-book { display: flex; flex-direction: column; }
}
@media (prefers-reduced-motion: reduce) {
.pp-book-cover, .pp-page-left, .pp-cover { transition: none; }
.pp-slam, .pp-slam::after, .pp-shake, .pp-wax { animation: none; }
.pp-slam, .pp-stamp-page::after { opacity: 1; }
.pp-stamp-hidden { opacity: 0.92; }
}
/* Badge ceremony (body-level overlay — shows over the player) */
.pp-ceremony-overlay {
position: fixed;
inset: 0;
z-index: 220;
display: flex;
align-items: center;
justify-content: center;
background: rgba(3, 7, 18, 0.55);
backdrop-filter: blur(1.5px);
animation: pp-ceremony-in 0.3s ease-out;
cursor: pointer;
}
.pp-ceremony-out { opacity: 0; transition: opacity 0.3s ease-out; }
.pp-confetti { position: absolute; inset: 0; width: 100%; height: 100%; pointer-events: none; }
.pp-ceremony-card {
display: flex;
flex-direction: column;
align-items: center;
gap: 0.5rem;
text-align: center;
}
.pp-ceremony-stamp {
position: relative;
overflow: hidden;
background: rgba(239, 230, 208, 0.97);
transform: rotate(var(--pp-rot)) scale(1.25);
animation: pp-slam 0.55s cubic-bezier(0.2, 0.8, 0.3, 1) 0.15s backwards;
margin-top: 0;
}
.pp-ceremony-stamp::before {
content: '';
position: absolute;
inset: -40%;
background: linear-gradient(115deg, transparent 42%, rgba(255, 255, 255, 0.55) 50%, transparent 58%);
transform: translateX(-120%);
animation: pp-shine 1.1s ease-out 0.75s forwards;
pointer-events: none;
}
@keyframes pp-shine {
to { transform: translateX(120%); }
}
@keyframes pp-ceremony-in {
from { opacity: 0; }
to { opacity: 1; }
}
.pp-ceremony-title {
margin-top: 1rem;
font-size: 1.15rem;
font-weight: 700;
letter-spacing: 0.18em;
text-transform: uppercase;
color: #f0e2c3;
text-shadow: 0 2px 8px rgba(0, 0, 0, 0.8);
}
.pp-ceremony-sub { font-size: 0.8rem; color: #d1d5db; text-shadow: 0 1px 4px rgba(0, 0, 0, 0.8); }
/* Hours odometer (Stage 5 post-cap — a true fact, never a meter) */
.pp-hours {
font-size: 0.72rem;
letter-spacing: 0.08em;
color: #8a7a5e;
margin-top: 0.75rem;
font-variant-numeric: tabular-nums;
}
/* ── Visuals pack: trading-card tilt, emerging ink, gold foil ──────────── */
/* Trading-card tilt (earned artifacts; JS feeds --pp-tilt-* on hover-capable
pointers only). */
.pp-tilt {
position: relative;
overflow: hidden;
will-change: transform;
}
.pp-cover.pp-tilt {
transform: perspective(700px)
rotateX(var(--pp-tilt-x, 0deg)) rotateY(var(--pp-tilt-y, 0deg))
rotate(var(--pp-cover-rot, 0deg));
transition: transform 0.12s ease, box-shadow 0.15s ease;
}
.pp-cover.pp-tilt:hover { box-shadow: 0 12px 26px rgba(0, 0, 0, 0.6); }
.pp-stamp-page.pp-tilt {
overflow: visible;
transform: perspective(600px)
rotateX(var(--pp-tilt-x, 0deg)) rotateY(var(--pp-tilt-y, 0deg))
rotate(var(--pp-rot));
transition: transform 0.12s ease;
}
.pp-tilt::after {
content: '';
position: absolute;
inset: 0;
border-radius: inherit;
background: linear-gradient(105deg,
transparent calc(var(--pp-glint-x, 50%) - 14%),
rgba(255, 255, 255, 0.16) var(--pp-glint-x, 50%),
transparent calc(var(--pp-glint-x, 50%) + 14%));
opacity: 0;
transition: opacity 0.2s ease;
pointer-events: none;
}
.pp-tilt:hover::after { opacity: 1; }
/* Emerging-stamp ink: the ghost fills as qualifying songs land. */
.pp-stamp-ghost::before {
content: '';
position: absolute;
inset: 7%;
border-radius: 999px;
background: conic-gradient(rgba(154, 91, 22, 0.16) var(--pp-fill, 0%), transparent 0);
pointer-events: none;
}
/* Gold foil preview — honest "coming", never earnable-looking. */
.pp-gold-foil {
position: relative;
overflow: hidden;
display: inline-flex;
align-items: center;
justify-content: center;
margin-top: 0.9rem;
padding: 0.28rem 0.85rem;
border-radius: 999px;
border: 2px dashed #c8b273;
color: #a8946d;
font-size: 0.58rem;
font-weight: 700;
letter-spacing: 0.32em;
}
.pp-gold-foil::after {
content: '';
position: absolute;
inset: 0;
background: linear-gradient(100deg, transparent 40%, rgba(255, 223, 128, 0.35) 50%, transparent 60%);
transform: translateX(-120%);
animation: pp-foil 3.4s ease-in-out infinite;
}
@keyframes pp-foil {
0%, 55% { transform: translateX(-120%); }
100% { transform: translateX(120%); }
}
@media (prefers-reduced-motion: reduce) {
.pp-gold-foil::after { animation: none; }
.pp-cover.pp-tilt, .pp-stamp-page.pp-tilt { transition: none; }
/* The hover glint is motion theatrics too — not just the JS tilt. */
.pp-tilt::after { display: none; }
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+1 -21
View File
@@ -3,27 +3,7 @@
"songs": 5,
"min_stars": 2
},
"families": [
{ "key": "metal", "match": ["metal", "djent", "grindcore", "thrash", "doom"] },
{ "key": "blues", "match": ["blues"] },
{ "key": "jazz", "match": ["jazz", "bebop", "swing", "bossa"] },
{ "key": "funk", "match": ["funk", "disco"] },
{ "key": "rock", "match": ["rock", "punk", "grunge", "shoegaze"] }
],
"genres": {
"blues": { "virtuoso_nodes": { "guitar": ["blues_shuffle"] } },
"rock": { "virtuoso_nodes": { "guitar": ["rock_power_backbeat"] } },
"metal": { "virtuoso_nodes": { "guitar": ["melodic_metal_gallop"] } },
"funk": { "virtuoso_nodes": { "guitar": ["sixteenth_pocket"] } },
"jazz": { "virtuoso_nodes": { "guitar": ["vl_shells"] } }
},
"drill_labels": {
"blues_shuffle": "Blues Shuffle",
"rock_power_backbeat": "Power Chords & Backbeat",
"melodic_metal_gallop": "Gallop Picking",
"sixteenth_pocket": "16th Pocket",
"vl_shells": "Shell Voicings"
},
"genres": {},
"graded_instruments": [
"guitar",
"keys"
+20 -99
View File
@@ -203,24 +203,21 @@ def _instrument_of(arrangements, arrangement):
def _played_by_instrument_genre():
"""((instrument, genre_key) → {filename: stub dict},
(instrument, genre_key) total played seconds).
Best accuracy per (instrument, song); seconds sum across every
arrangement row; the JOIN keeps the same dead-song filter as _stars()."""
"""(instrument, genre_key) → {filename: stub dict}. Best accuracy per
(instrument, song); the JOIN keeps the same dead-song filter as _stars()."""
db = _state["meta_db"]
if db is None:
return {}, {}
return {}
thresholds = _state["content"]["star_accuracy_thresholds"]
rows = db.conn.execute(
"SELECT s.filename, s.arrangement, s.best_accuracy, s.last_played_at, "
" s.seconds_total, songs.title, songs.artist, songs.arrangements, "
" songs.title, songs.artist, songs.arrangements, "
f" {_genre_expr(db)} "
"FROM song_stats s JOIN songs ON songs.filename = s.filename"
).fetchall()
arrs_cache = {}
out = {}
seconds = {}
for filename, arrangement, acc, played_at, secs, title, artist, arrs_json, genre in rows:
for filename, arrangement, acc, played_at, title, artist, arrs_json, genre in rows:
gkey = _genre_key(genre)
if not gkey:
continue
@@ -230,12 +227,10 @@ def _played_by_instrument_genre():
except (TypeError, ValueError):
arrs_cache[filename] = None
instrument = _instrument_of(arrs_cache[filename], arrangement)
key = (instrument, gkey)
seconds[key] = seconds.get(key, 0.0) + (secs or 0.0)
acc = acc or 0.0
stub = out.setdefault(key, {}).get(filename)
stub = out.setdefault((instrument, gkey), {}).get(filename)
if stub is None:
out[key][filename] = {
out[(instrument, gkey)][filename] = {
"filename": filename,
"title": title or filename,
"artist": artist or "",
@@ -250,7 +245,7 @@ def _played_by_instrument_genre():
acc = stub["best_accuracy"]
stub["best_accuracy"] = round(acc, 4)
stub["stars"] = sum(1 for t in thresholds if acc >= t)
return out, seconds
return out
def _library_genres():
@@ -276,44 +271,16 @@ def _library_genres():
key=lambda r: (-r["songs_in_library"], r["genre_key"]))
def _genre_family(gkey):
"""First family whose keyword appears in the genre key (substring — MB's
vocabulary is open: 'metalcore' must hit the 'metal' family without an
exact alias). List order decides ambiguity: families are checked top to
bottom, so 'blues rock' lands on whichever of blues/rock is listed first."""
for fam in _state["passports_content"].get("families") or []:
if not isinstance(fam, dict):
continue
for kw in fam.get("match") or []:
if isinstance(kw, str) and kw and kw in gkey:
return fam.get("key")
return None
def _badge_requirement(gkey, instrument="guitar"):
def _badge_requirement(gkey):
cfg = _state["passports_content"]
req = dict(cfg.get("badge_requirement") or {})
req.setdefault("songs", 5)
req.setdefault("min_stars", 2)
# Exact per-genre override wins; otherwise the genre inherits its FAMILY's
# requirement — so 'death metal' / 'metalcore' passports carry the metal
# drill without curating every MB sub-genre by hand.
genres_cfg = cfg.get("genres") or {}
override = genres_cfg.get(gkey)
if not isinstance(override, dict):
family = _genre_family(gkey)
override = genres_cfg.get(family) if family else None
override = (cfg.get("genres") or {}).get(gkey)
if isinstance(override, dict):
req.update(override)
# virtuoso_nodes: {instrument: [node_ids]} — a passport only carries its
# own instrument's drills. A flat list keeps meaning guitar (back-compat;
# virtuoso's drill content is guitar-first).
nodes = req.get("virtuoso_nodes") or []
if isinstance(nodes, dict):
nodes = nodes.get(instrument) or []
elif instrument != "guitar":
nodes = []
req["virtuoso_nodes"] = [n for n in nodes if isinstance(n, str)]
req["virtuoso_nodes"] = [n for n in (req.get("virtuoso_nodes") or [])
if isinstance(n, str)]
return req
@@ -326,57 +293,21 @@ def _drill_by_node():
return doc.get("received_at"), by_node
def _merge_drill_nodes(old, new):
"""Gained-only merge of virtuoso byNode snapshots: a completion artifact
once relayed never un-earns via a stale snapshot (multi-browser races,
settings import, the once-per-session boot relay). Incoming wins the
descriptive fields; masteredAt / depth flips / keysCleared only grow."""
out = dict(old)
for node_id, incoming in new.items():
if not isinstance(incoming, dict):
continue
cur = out.get(node_id)
if not isinstance(cur, dict):
out[node_id] = incoming
continue
merged = dict(cur)
merged.update(incoming)
merged["masteredAt"] = cur.get("masteredAt") or incoming.get("masteredAt")
d_old = cur.get("depth") if isinstance(cur.get("depth"), dict) else {}
d_new = incoming.get("depth") if isinstance(incoming.get("depth"), dict) else {}
depth = dict(d_new)
for axis, val in d_old.items():
if val and not depth.get(axis):
depth[axis] = val
if depth:
merged["depth"] = depth
keys_old = cur.get("keysCleared") if isinstance(cur.get("keysCleared"), list) else []
keys_new = incoming.get("keysCleared") if isinstance(incoming.get("keysCleared"), list) else []
merged["keysCleared"] = keys_old + [k for k in keys_new if k not in keys_old]
out[node_id] = merged
return out
def _node_cleared(by_node, node_id):
"""A drill counts as cleared on real completion evidence: mastered, any
depth rung flipped true, or a key cleared (a top-tier clean pass in one
key virtuoso's first gained-only artifact, and an achievable Bronze
bar; the depth rungs additionally require a maxed speed tier)."""
"""A drill counts as cleared on real completion evidence: mastered, or any
depth rung flipped true (virtuoso's gained-only false→true artifacts)."""
entry = by_node.get(node_id)
if not isinstance(entry, dict):
return False
depth = entry.get("depth") if isinstance(entry.get("depth"), dict) else {}
keys = entry.get("keysCleared")
return (bool(entry.get("masteredAt"))
or any(bool(v) for v in depth.values())
or bool(isinstance(keys, list) and keys))
return bool(entry.get("masteredAt")) or any(bool(v) for v in depth.values())
def _passports_view():
cfg = _state["passports_content"]
graded = set(cfg.get("graded_instruments") or [])
st = _career_state()
played, played_seconds = _played_by_instrument_genre()
played = _played_by_instrument_genre()
received_at, by_node = _drill_by_node()
instruments = {}
for inst in cfg.get("instruments") or []:
@@ -387,7 +318,7 @@ def _passports_view():
for gkey, meta in sorted(opened.items(),
key=lambda kv: ((kv[1] or {}).get("opened_at") or "", kv[0])):
meta = meta if isinstance(meta, dict) else {}
req = _badge_requirement(gkey, inst)
req = _badge_requirement(gkey)
songs = list(played.get((inst, gkey), {}).values())
for s in songs:
s["qualifies"] = s["stars"] >= req["min_stars"]
@@ -414,9 +345,6 @@ def _passports_view():
"graded": is_graded,
"songs": songs,
"qualifying_count": qualifying,
# Honest hours odometer (Stage 5 post-cap): a true fact that
# only grows — never a target, never a meter.
"seconds_total": round(played_seconds.get((inst, gkey), 0.0), 1),
"drills": {"required": required, "cleared": cleared},
"badge": badge,
})
@@ -426,8 +354,6 @@ def _passports_view():
"badge_requirement": cfg.get("badge_requirement") or {},
"graded_instruments": sorted(graded),
"instruments": list(cfg.get("instruments") or []),
# Career-side display names for virtuoso drill node ids.
"drill_labels": dict(cfg.get("drill_labels") or {}),
},
"instruments": instruments,
"genres": _library_genres(),
@@ -602,16 +528,11 @@ def setup(app, context):
# Only the fields the badge check reads are kept.
if not isinstance(body, dict) or not isinstance(body.get("byNode"), dict):
raise HTTPException(400, "Expected a progress snapshot with byNode.")
# Bound the INCOMING snapshot before the merge — the gained-only merge
# drops junk entries, which must not become a size-guard bypass.
if len(json.dumps(body["byNode"])) > DRILL_SNAPSHOT_MAX_BYTES:
snapshot = {"mode": body.get("mode"), "xp": body.get("xp"),
"byNode": body["byNode"]}
if len(json.dumps(snapshot)) > DRILL_SNAPSHOT_MAX_BYTES:
raise HTTPException(413, "Snapshot too large.")
with _lock:
_, existing = _drill_by_node()
snapshot = {"mode": body.get("mode"), "xp": body.get("xp"),
"byNode": _merge_drill_nodes(existing, body["byNode"])}
if len(json.dumps(snapshot)) > DRILL_SNAPSHOT_MAX_BYTES:
raise HTTPException(413, "Snapshot too large.")
_save_json(_drill_file(), {"received_at": _now_iso(),
"snapshot": snapshot})
return {"ok": True}
+12 -33
View File
@@ -3,40 +3,19 @@
<h1 class="text-2xl font-bold text-white">Career</h1>
<div id="career-stars-summary" class="text-sm text-gray-400"></div>
</div>
<div class="career-tabs" role="tablist">
<button class="career-tab" data-career-tab="venues" role="tab" id="career-tab-btn-venues" aria-controls="career-tab-venues">Venues</button>
<button class="career-tab" data-career-tab="passports" role="tab" id="career-tab-btn-passports" aria-controls="career-tab-passports">Passports</button>
<p class="text-sm text-gray-400 mb-4">Earn stars by playing songs well — 60% accuracy is a star, 75% two, 85% three. Stars unlock bigger stages, and the crowd plays along with you.</p>
<div id="career-progress-wrap" class="mb-6">
<div class="career-bar-track">
<div id="career-progress-bar" class="career-bar-fill" style="width:0%"></div>
</div>
<div id="career-progress-label" class="text-xs text-gray-500 mt-1"></div>
</div>
<div id="career-tab-venues" role="tabpanel" aria-labelledby="career-tab-btn-venues">
<p class="text-sm text-gray-400 mb-4">Earn stars by playing songs well — 60% accuracy is a star, 75% two, 85% three. Stars unlock bigger stages, and the crowd plays along with you.</p>
<div id="career-progress-wrap" class="mb-6">
<div class="career-bar-track">
<div id="career-progress-bar" class="career-bar-fill" style="width:0%"></div>
</div>
<div id="career-progress-label" class="text-xs text-gray-500 mt-1"></div>
</div>
<div id="career-venues" class="career-venues"></div>
<div class="mt-8">
<div class="flex items-end justify-between flex-wrap gap-2 mb-2">
<h2 class="text-lg font-semibold text-white">Your star collection</h2>
<div id="career-star-summary" class="text-xs text-gray-400"></div>
</div>
<div id="career-star-list" class="career-star-list"></div>
</div>
</div>
<div id="career-tab-passports" class="hidden" role="tabpanel" aria-labelledby="career-tab-btn-passports">
<p class="text-sm text-gray-400 mb-4">Commit to an instrument, pick a genre, and stamp your way to its badge — five ★★ songs mint a Bronze. Your passport wall is who you are as a musician.</p>
<div id="pp-instruments" class="pp-instruments"></div>
<div id="pp-shelf-wrap" class="mt-5">
<div id="pp-shelf" class="pp-shelf"></div>
</div>
<div id="pp-rack-wrap" class="mt-8">
<h2 class="text-lg font-semibold text-white mb-1">Explore next</h2>
<p class="text-xs text-gray-500 mb-3">More genres, whenever you want them — your wall is complete as it is.</p>
<div id="pp-rack" class="pp-rack"></div>
<div id="career-venues" class="career-venues"></div>
<div class="mt-8">
<div class="flex items-end justify-between flex-wrap gap-2 mb-2">
<h2 class="text-lg font-semibold text-white">Your star collection</h2>
<div id="career-star-summary" class="text-xs text-gray-400"></div>
</div>
<div id="career-star-list" class="career-star-list"></div>
</div>
</div>
<div id="pp-overlay" class="pp-overlay hidden"></div>
+1 -618
View File
@@ -17,27 +17,11 @@
const PREV_VIZ_KEY = 'feedBack-career-prev-viz';
const POLL_MS = 2000;
// Passports (the badge-journey layer; see routes.py — badges are computed
// server-side, this file only renders and relays).
const PP_SEEN_KEY = 'feedBack-career-badges-seen';
const PP_INST_KEY = 'feedBack-career-instrument';
const PP_TAB_KEY = 'feedBack-career-tab';
const PP_LABELS = { guitar: 'Guitar', bass: 'Bass', keys: 'Keys', drums: 'Drums' };
const PP_BROCHURE_ART = ['🎸', '🎷', '🎹', '🥁', '🎺', '🎻', '🎤', '🪕'];
let _state = null;
let _pollTimer = 0;
let _appliedManifestVenue = null;
let _manifestReqGen = 0; // invalidates in-flight manifest fetches
let _prevUnlockedIds = null;
let _pp = null; // last /passports view
let _ppRelayTimer = 0;
let _ppBook = null; // {inst, gkey} of the open spread
let _ppReturnFocus = null; // element to refocus when the book closes
let _ppCeremonyQueue = []; // badges awaiting their ceremony overlay
let _ppCeremonyActive = false;
let _ppBootstrapped = false;
let _ppNotified = {}; // badges chimed this session (slam still pending)
function $(id) { return document.getElementById(id); }
@@ -235,591 +219,9 @@
render(state);
schedulePoll(state);
pushCrowdManifest(state);
refreshPassports(); // independent fetch; failures don't touch venues
}
// ── Passports ─────────────────────────────────────────────────────────
function lsGet(k) { try { return localStorage.getItem(k); } catch (_) { return null; } }
function lsSet(k, v) { try { localStorage.setItem(k, v); } catch (_) { /* ok */ } }
function ppLabel(inst) {
return PP_LABELS[inst] || (inst.charAt(0).toUpperCase() + inst.slice(1));
}
function ppKey(genre) {
return String(genre || '').trim().replace(/\s+/g, ' ').toLowerCase();
}
function ppHash(seed) {
let h = 0;
for (let i = 0; i < seed.length; i++) h = (h * 31 + seed.charCodeAt(i)) | 0;
return h;
}
// Deterministic per-key jitter (sin-hash): stamps and stubs land slightly
// askew, the same way on every visit.
function ppJitter(seed, range) {
return (Math.abs(Math.sin(ppHash(seed))) * 2 - 1) * range;
}
function sfx(name) {
try {
const a = new Audio(`${API}/assets/sfx/${name}.mp3`);
a.volume = 0.45;
a.play().catch(() => { /* autoplay policy — silent is fine */ });
} catch (_) { /* no Audio — fine */ }
}
function showCareerTab(tab) {
lsSet(PP_TAB_KEY, tab);
const venues = $('career-tab-venues');
const pp = $('career-tab-passports');
if (!venues || !pp) return;
venues.classList.toggle('hidden', tab !== 'venues');
pp.classList.toggle('hidden', tab !== 'passports');
document.querySelectorAll('#plugin-career .career-tab').forEach((b) => {
const active = b.dataset.careerTab === tab;
b.classList.toggle('active', active);
b.setAttribute('aria-selected', active ? 'true' : 'false');
});
}
function activeInstrument() {
const list = (_pp && _pp.config && _pp.config.instruments) || [];
const saved = lsGet(PP_INST_KEY);
if (saved && list.includes(saved)) return saved;
const committed = list.find((i) => ((_pp.instruments || {})[i] || {}).committed_at);
return committed || list[0] || 'guitar';
}
function seenBadges() {
try {
const seen = JSON.parse(lsGet(PP_SEEN_KEY) || '{}');
// Guard non-object JSON (a stray "null" or array) — a broken
// stored value must not throw on every passport refresh.
return seen && typeof seen === 'object' && !Array.isArray(seen) ? seen : {};
} catch (_) { return {}; }
}
function badgeId(inst, gkey) { return inst + '/' + gkey; }
function markBadgeSeen(inst, gkey) {
const seen = seenBadges();
seen[badgeId(inst, gkey)] = 1;
lsSet(PP_SEEN_KEY, JSON.stringify(seen));
}
// New badge → chime + notification + the venue ceremony, once per
// session; the stamp SLAM plays when the passport is next opened (and
// only then is the badge marked seen, so a pending slam survives a
// reload).
function detectNewBadges(view) {
const seen = seenBadges();
for (const inst of Object.keys(view.instruments || {})) {
for (const p of (view.instruments[inst].passports || [])) {
const id = badgeId(inst, p.genre_key);
if (p.badge !== 'earned' || seen[id] || _ppNotified[id]) continue;
_ppNotified[id] = true;
sfx('chime');
if (window.fbNotify && typeof window.fbNotify.show === 'function') {
window.fbNotify.show({
big: true, icon: '🛂', accent: '#b45309',
title: 'Badge earned!',
message: `${p.genre} — Bronze, ready to stamp into your ${ppLabel(inst)} passport.`,
});
}
badgeCeremony(inst, p);
}
}
}
function reducedMotion() {
try { return window.matchMedia('(prefers-reduced-motion: reduce)').matches; } catch (_) { return false; }
}
// The badge moment: the crowd erupts first (if a venue pack is live —
// badges land post-stats:recorded while the player is still on screen),
// then a body-level overlay. It CANNOT live in #pp-overlay: #plugin-career
// is display:none during playback.
function badgeCeremony(inst, p) {
// Reduced motion: the chime + fbNotify already delivered the news —
// no overlay, and no app-initiated crowd eruption either.
if (reducedMotion()) return;
const crowd = window.v3VenueCrowd;
if (crowd && typeof crowd.celebrate === 'function') {
try { crowd.celebrate(); } catch (_) { /* crowd layer optional */ }
}
if (!document.body || typeof document.createElement !== 'function') return;
// Several badges can land in one refresh (first load, drill-snapshot
// bootstrap): queue the ceremonies and play them back to back.
_ppCeremonyQueue.push({ inst, p });
if (!_ppCeremonyActive) setTimeout(drainCeremonies, 300);
}
function drainCeremonies() {
if (_ppCeremonyActive) return;
const queued = _ppCeremonyQueue.shift();
if (!queued) return;
_ppCeremonyActive = true;
showCeremonyOverlay(queued.inst, queued.p, () => {
_ppCeremonyActive = false;
setTimeout(drainCeremonies, 250);
});
}
function showCeremonyOverlay(inst, p, done) {
const el = document.createElement('div');
el.id = 'pp-ceremony';
el.className = 'pp-ceremony-overlay';
el.innerHTML = `
<canvas class="pp-confetti"></canvas>
<div class="pp-ceremony-card">
<div class="pp-stamp pp-stamp-page pp-ceremony-stamp" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg">
<span class="pp-stamp-genre">${esc(p.genre.toUpperCase())}</span>
<span class="pp-stamp-tier">BRONZE</span>
</div>
<div class="pp-ceremony-title">Badge earned</div>
<div class="pp-ceremony-sub">${esc(p.genre)} ${esc(ppLabel(inst))} passport</div>
</div>`;
let timer = 0;
let closed = false;
const dismiss = () => {
if (closed) return;
closed = true;
clearTimeout(timer);
el.classList.add('pp-ceremony-out');
setTimeout(() => { el.remove(); done(); }, 350);
};
el.addEventListener('click', dismiss);
document.body.appendChild(el);
timer = setTimeout(dismiss, 4200);
confettiBurst(el.querySelector('.pp-confetti'));
}
function confettiBurst(canvas) {
if (!canvas || typeof canvas.getContext !== 'function') return;
const ctx = canvas.getContext('2d');
if (!ctx) return;
canvas.width = canvas.clientWidth;
canvas.height = canvas.clientHeight;
const colors = ['#d9a253', '#b45309', '#facc15', '#06b6d4', '#e5e7eb'];
const parts = Array.from({ length: 42 }, () => ({
x: canvas.width / 2 + (Math.random() - 0.5) * 90,
y: canvas.height * 0.42,
vx: (Math.random() - 0.5) * 9,
vy: -(4 + Math.random() * 7),
rot: Math.random() * Math.PI,
vr: (Math.random() - 0.5) * 0.3,
w: 5 + Math.random() * 5,
h: 3 + Math.random() * 4,
c: colors[(Math.random() * colors.length) | 0],
}));
let frames = 0;
(function tick() {
if (!canvas.isConnected || frames++ > 240) return;
ctx.clearRect(0, 0, canvas.width, canvas.height);
for (const q of parts) {
q.x += q.vx; q.y += q.vy; q.vy += 0.18; q.rot += q.vr;
ctx.save();
ctx.translate(q.x, q.y);
ctx.rotate(q.rot);
ctx.fillStyle = q.c;
ctx.fillRect(-q.w / 2, -q.h / 2, q.w, q.h);
ctx.restore();
}
requestAnimationFrame(tick);
}());
}
// Relay the Virtuoso drill snapshot (localStorage doc, not the thin bus
// payload) to the server intake, debounced across event bursts.
function relayDrillState() {
clearTimeout(_ppRelayTimer);
_ppRelayTimer = setTimeout(() => {
let snap = null;
try { snap = JSON.parse(lsGet('virtuoso.progress') || 'null'); } catch (_) { /* corrupt */ }
if (!snap || typeof snap !== 'object' || !snap.byNode || typeof snap.byNode !== 'object') return;
fetch(`${API}/drill-state`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ mode: snap.mode, xp: snap.xp, byNode: snap.byNode }),
}).then(() => refreshPassports()).catch(() => { /* next event retries */ });
}, 1500);
}
async function refreshPassports() {
let view;
try {
const res = await fetch(`${API}/passports`);
if (!res.ok) return;
view = await res.json();
} catch (_) { return; }
_pp = view;
detectNewBadges(view);
renderPassports();
if (!_ppBootstrapped) {
_ppBootstrapped = true;
// Sync the local drill snapshot once per session — drill progress
// made before the career plugin existed (or a relay POST that
// failed) must not deny a gated badge until the next virtuoso
// event happens to fire. Tiny payload, single-user app.
relayDrillState();
}
}
// Honest hours odometer (Stage 5 post-cap). Below a minute of history
// there is nothing meaningful to show.
function fmtHours(seconds) {
const s = Number(seconds) || 0;
if (s < 60) return '';
if (s < 3600) return `${Math.round(s / 60)} min`;
return `${(s / 3600).toFixed(1).replace(/\.0$/, '')} h`;
}
function ppCoverHTML(inst, p) {
const rot = ppJitter(inst + p.genre_key, 1.6).toFixed(2);
const earned = p.badge === 'earned';
const stamp = earned
? `<span class="pp-stamp pp-stamp-mini" style="--pp-rot:${ppJitter(p.genre_key, 8).toFixed(1)}deg">BRONZE</span>`
: '';
const stubs = p.qualifying_count === 1 ? '1 stub' : `${p.qualifying_count} stubs`;
const hours = fmtHours(p.seconds_total);
// Earned covers are trading cards: rotation moves into a CSS var so
// the pointer-tracked tilt transform can compose with it.
const style = earned
? `--pp-cover-rot:${rot}deg` : `transform:rotate(${rot}deg)`;
return `<button class="pp-cover${earned ? ' pp-tilt' : ''} pp-leather-${esc(inst)}" data-pp-open="${esc(p.genre_key)}" style="${style}">
<span class="pp-cover-title">${esc(p.genre.toUpperCase())}</span>
<span class="pp-cover-inst">${esc(ppLabel(inst))} passport</span>
${stamp}
<span class="pp-cover-sub">${stubs}${hours ? ` · ${hours}` : ''}</span>
</button>`;
}
function renderShelf(inst, data) {
const shelf = $('pp-shelf');
if (!shelf) return;
if (!data.committed_at) {
shelf.innerHTML = `<div class="pp-commit-card">
<div class="pp-commit-cover pp-leather-${esc(inst)}">
<span class="pp-cover-title">${esc(ppLabel(inst).toUpperCase())}</span>
<span class="pp-cover-inst">passport</span>
</div>
<div>
<div class="text-sm text-gray-200 font-medium mb-1">Pick up the ${esc(ppLabel(inst).toLowerCase())}.</div>
<div class="text-xs text-gray-400 mb-2">Press your seal to commit then choose a genre below and go deep.</div>
<button class="career-btn career-btn-primary" data-pp-commit="${esc(inst)}">Press the seal</button>
</div>
</div>`;
return;
}
const books = (data.passports || []).map((p) => ppCoverHTML(inst, p)).join('');
shelf.innerHTML = books ||
'<div class="text-xs text-gray-500">Your shelf is ready — open your first genre passport below.</div>';
}
function renderRack(inst, data) {
const rack = $('pp-rack');
if (!rack || !_pp) return;
const openedKeys = new Set((data.passports || []).map((p) => p.genre_key));
const genres = (_pp.genres || []).filter((g) => !openedKeys.has(g.genre_key));
if (!genres.length) {
rack.innerHTML = '<div class="text-xs text-gray-500">No further genres in your library yet — new songs bring new brochures.</div>';
return;
}
rack.innerHTML = genres.map((g) => {
const art = PP_BROCHURE_ART[Math.abs(ppHash(g.genre_key)) % PP_BROCHURE_ART.length];
return `<button class="pp-brochure" data-pp-genre="${esc(g.genre)}">
<span class="pp-brochure-art" aria-hidden="true">${art}</span>
<span class="pp-brochure-name">${esc(g.genre)}</span>
<span class="pp-brochure-sub">${g.songs_in_library === 1 ? '1 song' : `${g.songs_in_library} songs`} in your library</span>
</button>`;
}).join('');
}
function renderPassports() {
const host = $('pp-instruments');
if (!host || !_pp) return;
const inst = activeInstrument();
const data = (_pp.instruments || {})[inst] || { passports: [] };
host.innerHTML = ((_pp.config || {}).instruments || []).map((i) => {
const d = (_pp.instruments || {})[i] || {};
const earned = (d.passports || []).filter((p) => p.badge === 'earned').length;
const committed = !!d.committed_at;
return `<button class="pp-inst${i === inst ? ' active' : ''}${committed ? '' : ' uncommitted'}" data-pp-inst="${esc(i)}">
${esc(ppLabel(i))}${earned ? ` <span class="pp-inst-badges">⚡${earned}</span>` : ''}${committed ? '' : ' <span class="pp-inst-plus">+</span>'}
</button>`;
}).join('');
renderShelf(inst, data);
renderRack(inst, data);
}
function ppStubHTML(s) {
const date = (s.last_played_at || '').slice(0, 10);
return `<div class="pp-stub" style="transform:rotate(${ppJitter(s.filename, 1.2).toFixed(2)}deg)">
<span class="pp-stub-stars">${'★'.repeat(s.stars)}</span>
<span class="pp-stub-title">${esc(s.title)}</span>
${s.artist ? `<span class="pp-stub-artist">${esc(s.artist)}</span>` : ''}
<span class="pp-stub-meta">${date ? `${esc(date)} · ` : ''}best ${(s.best_accuracy * 100).toFixed(0)}%</span>
</div>`;
}
// Emerging-stamp ink: how much of the ghost stamp has "carved in".
// Song progress toward the bar only — the invite line stays the words.
function ppFillFraction(p) {
if (!p || p.badge !== 'in_progress') return 0;
const need = Number((p.requirement || {}).songs) || 0;
if (need <= 0) return 0;
return Math.max(0, Math.min(1, (p.qualifying_count || 0) / need));
}
function ppBookHTML(inst, p, pendingSlam) {
const req = p.requirement || {};
const need = Math.max(0, (req.songs || 0) - p.qualifying_count);
const starGl = '★'.repeat(req.min_stars || 0);
const reqNodes = (p.drills || {}).required || [];
const clearedNodes = new Set((p.drills || {}).cleared || []);
const labels = ((_pp && _pp.config) || {}).drill_labels || {};
const pendingDrills = reqNodes.filter((n) => !clearedNodes.has(n));
// The invite names what actually blocks the stamp: songs first, then
// the genre drill once the song bar is met.
let invite;
if (need > 0) {
invite = need === 1 ? `One more ${starGl} song mints this stamp.`
: `${need} more ${starGl} songs mint this stamp.`;
} else {
const names = pendingDrills.map((n) => labels[n] || n).join(', ');
invite = `Clear ${names || 'the genre drill'} in Virtuoso to mint this stamp.`;
}
let badgeArea = '';
if (p.badge === 'shown_not_judged') {
badgeArea = `<div class="pp-snj">Shown, not judged — your ${esc(ppLabel(inst).toLowerCase())} repertoire speaks for itself.</div>`;
} else if (p.badge === 'earned') {
badgeArea = `<div class="pp-stamp pp-stamp-page${pendingSlam ? ' pp-stamp-hidden' : ' pp-tilt'}" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg">
<span class="pp-stamp-genre">${esc(p.genre.toUpperCase())}</span>
<span class="pp-stamp-tier">BRONZE</span>
</div>
<div class="pp-gold-foil" aria-hidden="true">GOLD</div>
<div class="pp-gold-note">Gold rung coming improvise it, verified.</div>`;
} else {
const fill = (ppFillFraction(p) * 100).toFixed(0);
badgeArea = `<div class="pp-stamp pp-stamp-page pp-stamp-ghost" style="--pp-rot:${ppJitter(p.genre_key, 7).toFixed(1)}deg; --pp-fill:${fill}%">
<span class="pp-stamp-genre">${esc(p.genre.toUpperCase())}</span>
<span class="pp-stamp-tier">BRONZE</span>
</div>
<div class="pp-invite">${esc(invite)}</div>`;
}
const hours = fmtHours(p.seconds_total);
const odometer = hours
? `<div class="pp-hours">${hours} in ${esc(p.genre)}</div>` : '';
let drills = '';
if (reqNodes.length) {
drills = `<div class="pp-drills">${reqNodes.map((n) =>
`<div class="pp-drill${clearedNodes.has(n) ? ' cleared' : ''}">${clearedNodes.has(n) ? '✓' : '○'} ${esc(labels[n] || n)}</div>`).join('')}</div>`;
}
// Graded instruments collect stubs at the badge bar; shown-not-judged
// instruments have no bar — every played genre song is repertoire.
const stubs = p.badge === 'shown_not_judged'
? (p.songs || [])
: (p.songs || []).filter((s) => s.qualifies);
const emptyLine = p.badge === 'shown_not_judged'
? `Play ${esc(p.genre)} songs to fill this page.`
: `Play ${esc(p.genre)} songs at ${starGl} to collect ticket stubs.`;
const stubsHTML = stubs.length ? stubs.map(ppStubHTML).join('')
: `<div class="pp-stub-empty">${emptyLine}</div>`;
return `<div class="pp-book-wrap" data-pp-close-bg="1" role="dialog" aria-modal="true" aria-label="${esc(p.genre)} ${esc(ppLabel(inst))} passport">
<div class="pp-book">
<div class="pp-page pp-page-left">
<div class="pp-page-head">${esc(p.genre)} ${esc(ppLabel(inst))}</div>
${badgeArea}${odometer}${drills}
</div>
<div class="pp-page pp-page-right">
<div class="pp-page-head">Ticket stubs</div>
<div class="pp-stubs">${stubsHTML}</div>
</div>
<div class="pp-book-cover pp-leather-${esc(inst)}">
<span class="pp-cover-title">${esc(p.genre.toUpperCase())}</span>
<span class="pp-cover-inst">${esc(ppLabel(inst))} passport</span>
</div>
<button class="pp-book-close" data-pp-close="1" aria-label="Close"></button>
</div>
</div>`;
}
function openBook(inst, gkey) {
if (!_pp) return;
const p = (((_pp.instruments || {})[inst] || {}).passports || [])
.find((x) => x.genre_key === gkey);
const overlay = $('pp-overlay');
if (!p || !overlay) return;
_ppBook = { inst, gkey };
_ppReturnFocus = document.activeElement;
const pending = p.badge === 'earned' && !seenBadges()[badgeId(inst, gkey)];
overlay.innerHTML = ppBookHTML(inst, p, pending);
overlay.classList.remove('hidden');
const close = overlay.querySelector('.pp-book-close');
if (close) close.focus();
sfx('page');
// Double rAF so the cover's closed state paints before the transition.
requestAnimationFrame(() => requestAnimationFrame(() => {
const book = overlay.querySelector('.pp-book');
if (book) book.classList.add('open');
}));
if (pending) {
setTimeout(() => {
if (!_ppBook || _ppBook.gkey !== gkey || _ppBook.inst !== inst) return;
const stamp = overlay.querySelector('.pp-stamp-page');
const book = overlay.querySelector('.pp-book');
if (!stamp) return;
stamp.classList.remove('pp-stamp-hidden');
stamp.classList.add('pp-slam');
stamp.classList.add('pp-tilt'); // freshly slammed = trading card too
if (book) book.classList.add('pp-shake');
sfx('stamp');
markBadgeSeen(inst, gkey);
renderPassports(); // the shelf cover gains its mini-stamp
}, 950);
}
}
function closeBook() {
_ppBook = null;
const overlay = $('pp-overlay');
if (overlay) { overlay.classList.add('hidden'); overlay.innerHTML = ''; }
if (_ppReturnFocus && typeof _ppReturnFocus.focus === 'function' &&
document.contains(_ppReturnFocus)) {
_ppReturnFocus.focus();
}
_ppReturnFocus = null;
}
function commitInstrument(inst, after) {
fetch(`${API}/passports/commit`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ instrument: inst }),
}).then(() => refreshPassports())
.then(() => { if (after) after(); })
.catch(() => { /* server restarting; user retries */ });
}
// Stage 0 — the wax seal. Purely theatrical: the overlay plays the press,
// the POST commits, the shelf re-renders committed.
function sealCeremony(inst, after) {
const overlay = $('pp-overlay');
if (!overlay) { commitInstrument(inst, after); return; }
overlay.innerHTML = `<div class="pp-book-wrap">
<div class="pp-commit-cover pp-ceremony pp-leather-${esc(inst)}">
<span class="pp-cover-title">${esc(ppLabel(inst).toUpperCase())}</span>
<span class="pp-cover-inst">passport</span>
<span class="pp-wax"><span>${esc(ppLabel(inst).charAt(0))}</span></span>
</div>
</div>`;
overlay.classList.remove('hidden');
setTimeout(() => sfx('seal'), 450);
setTimeout(() => {
overlay.classList.add('hidden');
overlay.innerHTML = '';
commitInstrument(inst, after);
}, 1500);
}
// ── Trading-card tilt (earned artifacts only) ─────────────────────────
let _tiltRaf = 0;
let _tiltEl = null;
function tiltAllowed() {
try {
return window.matchMedia('(hover: hover)').matches &&
!window.matchMedia('(prefers-reduced-motion: reduce)').matches;
} catch (_) { return false; }
}
function resetTilt(el) {
if (!el) return;
el.style.removeProperty('--pp-tilt-x');
el.style.removeProperty('--pp-tilt-y');
el.style.removeProperty('--pp-glint-x');
}
function onTiltMove(e) {
if (!tiltAllowed()) return;
const card = e.target && e.target.closest ? e.target.closest('.pp-tilt') : null;
if (_tiltEl && _tiltEl !== card) { resetTilt(_tiltEl); _tiltEl = null; }
if (!card) return;
_tiltEl = card;
if (_tiltRaf) return;
const x = e.clientX;
const y = e.clientY;
_tiltRaf = requestAnimationFrame(() => {
_tiltRaf = 0;
const r = card.getBoundingClientRect();
if (!r.width || !r.height) return;
const px = (x - r.left) / r.width;
const py = (y - r.top) / r.height;
card.style.setProperty('--pp-tilt-x', `${((0.5 - py) * 10).toFixed(2)}deg`);
card.style.setProperty('--pp-tilt-y', `${((px - 0.5) * 12).toFixed(2)}deg`);
card.style.setProperty('--pp-glint-x', `${(px * 100).toFixed(1)}%`);
});
}
function onTiltLeave() {
// Cancel any queued frame: it closes over the departed card and would
// re-apply tilt vars after the pointer has left.
if (_tiltRaf) { cancelAnimationFrame(_tiltRaf); _tiltRaf = 0; }
if (_tiltEl) { resetTilt(_tiltEl); _tiltEl = null; }
}
function openGenre(inst, genre) {
fetch(`${API}/passports/open`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ instrument: inst, genre }),
}).then((res) => { if (!res.ok) throw new Error('open ' + res.status); })
.then(() => refreshPassports())
.then(() => openBook(inst, ppKey(genre)))
.catch(() => { /* validation/restart; rack stays */ });
}
function onClick(e) {
const tabBtn = e.target.closest('[data-career-tab]');
const instBtn = e.target.closest('[data-pp-inst]');
const commitBtn = e.target.closest('[data-pp-commit]');
const coverBtn = e.target.closest('[data-pp-open]');
const brochureBtn = e.target.closest('[data-pp-genre]');
if (tabBtn) {
showCareerTab(tabBtn.dataset.careerTab);
return;
}
if (instBtn) {
lsSet(PP_INST_KEY, instBtn.dataset.ppInst);
renderPassports();
return;
}
if (commitBtn) {
sealCeremony(commitBtn.dataset.ppCommit);
return;
}
if (coverBtn) {
openBook(activeInstrument(), coverBtn.dataset.ppOpen);
return;
}
if (brochureBtn) {
const inst = activeInstrument();
const genre = brochureBtn.dataset.ppGenre;
const committed = _pp && ((_pp.instruments || {})[inst] || {}).committed_at;
// Opening your first passport on an instrument IS the commitment —
// the seal ceremony runs first, then the passport opens.
if (committed) openGenre(inst, genre);
else sealCeremony(inst, () => openGenre(inst, genre));
return;
}
if (e.target.closest('[data-pp-close]') ||
(e.target.dataset && e.target.dataset.ppCloseBg)) {
closeBook();
return;
}
const dlBtn = e.target.closest('[data-career-download]');
const delBtn = e.target.closest('[data-career-delete]');
const playBtn = e.target.closest('[data-career-play]');
@@ -861,34 +263,15 @@
function boot() {
const screen = document.getElementById('plugin-career');
if (screen) {
screen.addEventListener('click', onClick);
screen.addEventListener('pointermove', onTiltMove);
screen.addEventListener('pointerleave', onTiltLeave);
}
if (screen) screen.addEventListener('click', onClick);
const sm = window.feedBack;
if (sm && typeof sm.on === 'function') {
// New song stats can add stars → thresholds may cross mid-session.
sm.on('stats:recorded', () => refresh());
// Virtuoso's progress emits are the drill-state relay trigger; the
// payload is a thin delta, so the relay reads the full localStorage
// snapshot instead (see relayDrillState).
sm.on('virtuoso:progress', relayDrillState);
}
showCareerTab(lsGet(PP_TAB_KEY) === 'passports' ? 'passports' : 'venues');
document.addEventListener('keydown', (e) => {
if (e.key === 'Escape' && _ppBook) closeBook();
});
refresh();
}
// Test seam (bare-vm harness, see plugins/career/tests/): pure helpers +
// the badge-diff logic; nothing here touches the DOM.
window.__careerPassportTest = {
ppKey, ppJitter, ppLabel, detectNewBadges, seenBadges, markBadgeSeen,
fmtHours, ppFillFraction,
};
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', boot);
} else {
-22
View File
@@ -8,25 +8,3 @@
Virtuoso plugin reports. These ride along in
<em>Settings → Export</em> automatically.</p>
</div>
<hr class="border-gray-800 my-3">
<div class="space-y-3 text-sm">
<label class="flex items-center justify-between gap-4">
<span>
<span class="text-gray-200 font-medium">Crowd sound reactions</span>
<span class="block text-xs text-gray-500">Cheers when the crowd's mood rises, boos when it drops. Uses each venue's own recordings.</span>
</span>
<input type="checkbox" id="career-sfx-toggle" class="accent-cyan-500 w-4 h-4">
</label>
</div>
<script>
(function () {
'use strict';
var KEY = 'feedBack-venue-crowd-sfx';
var box = document.getElementById('career-sfx-toggle');
if (!box) return;
try { box.checked = localStorage.getItem(KEY) === 'on'; } catch (e) { /* ok */ }
box.addEventListener('change', function () {
try { localStorage.setItem(KEY, box.checked ? 'on' : 'off'); } catch (e) { /* ok */ }
});
}());
</script>
-152
View File
@@ -1,152 +0,0 @@
// Passport UI pure-logic tests: load screen.js in a bare vm window and
// exercise the __careerPassportTest seam (no DOM beyond stubs, no network).
const { test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const vm = require('node:vm');
function load(seed) {
const store = Object.assign({}, seed);
const window = {
console,
localStorage: {
getItem: (k) => (k in store ? store[k] : null),
setItem: (k, v) => { store[k] = String(v); },
},
document: {
readyState: 'complete',
getElementById: () => null,
querySelectorAll: () => [],
addEventListener: () => {},
},
notifications: [],
};
window.window = window;
window.globalThis = window;
window.fbNotify = { show: (n) => window.notifications.push(n) };
const context = vm.createContext(window);
// `document` and `localStorage` resolve as bare names inside the IIFE.
context.document = window.document;
context.localStorage = window.localStorage;
const src = fs.readFileSync(path.join(__dirname, '..', 'screen.js'), 'utf8');
vm.runInContext(src, context, { filename: 'career/screen.js' });
return window;
}
test('module loads (and boots) in a bare vm window', () => {
const w = load();
assert.equal(typeof w.__careerPassportTest.ppKey, 'function');
});
test('ppKey normalizes case and whitespace', () => {
const { ppKey } = load().__careerPassportTest;
assert.equal(ppKey(' Blues Rock '), 'blues rock');
assert.equal(ppKey('FUNK'), 'funk');
assert.equal(ppKey(''), '');
assert.equal(ppKey(null), '');
});
test('ppJitter is deterministic and bounded', () => {
const { ppJitter } = load().__careerPassportTest;
assert.equal(ppJitter('blues', 8), ppJitter('blues', 8));
for (const seed of ['blues', 'funk', 'jazz', 'metal']) {
const j = ppJitter(seed, 8);
assert.ok(j >= -8 && j <= 8, `${seed}${j}`);
}
assert.notEqual(ppJitter('blues', 8), ppJitter('funk', 8));
});
test('detectNewBadges notifies once per badge, never after it is seen', () => {
const w = load();
const t = w.__careerPassportTest;
const view = {
instruments: {
guitar: {
passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'earned' },
{ genre_key: 'funk', genre: 'Funk', badge: 'in_progress' },
],
},
},
};
t.detectNewBadges(view);
assert.equal(w.notifications.length, 1);
assert.match(w.notifications[0].message, /Blues/);
// Same view again in the same session: no duplicate notification.
t.detectNewBadges(view);
assert.equal(w.notifications.length, 1);
// Seen (slam played) → a fresh session stays quiet too.
t.markBadgeSeen('guitar', 'blues');
// JSON-compare: vm objects carry a foreign Object prototype.
assert.equal(JSON.stringify(t.seenBadges()), '{"guitar/blues":1}');
// Fresh session (new vm, empty notify cache) with the badge already seen:
// detection must stay silent.
const w2 = load({ 'feedBack-career-badges-seen': '{"guitar/blues":1}' });
w2.__careerPassportTest.detectNewBadges(view);
assert.equal(w2.notifications.length, 0);
});
test('a new badge triggers the crowd celebrate() exactly once', () => {
const w = load();
let calls = 0;
w.v3VenueCrowd = { celebrate: () => { calls += 1; } };
const view = { instruments: { guitar: { passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'earned' }] } } };
w.__careerPassportTest.detectNewBadges(view);
assert.equal(calls, 1);
// Same session, same view: no re-celebration.
w.__careerPassportTest.detectNewBadges(view);
assert.equal(calls, 1);
});
test('ceremony degrades when the crowd layer is absent or throws', () => {
const w = load();
const view = { instruments: { guitar: { passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'earned' }] } } };
// No v3VenueCrowd at all (already exercised elsewhere, explicit here).
w.__careerPassportTest.detectNewBadges(view);
assert.equal(w.notifications.length, 1);
// celebrate() throwing must not break detection.
const w2 = load();
w2.v3VenueCrowd = { celebrate: () => { throw new Error('no pack'); } };
w2.__careerPassportTest.detectNewBadges(view);
assert.equal(w2.notifications.length, 1);
});
test('seenBadges tolerates corrupt stored values', () => {
for (const bad of ['null', '[1,2]', '"x"', '{{{']) {
const w = load({ 'feedBack-career-badges-seen': bad });
const t = w.__careerPassportTest;
assert.equal(JSON.stringify(t.seenBadges()), '{}', `stored ${bad}`);
// And detection still works on top of the recovered empty state.
t.detectNewBadges({ instruments: { guitar: { passports: [
{ genre_key: 'blues', genre: 'Blues', badge: 'earned' }] } } });
assert.equal(w.notifications.length, 1, `stored ${bad}`);
}
});
test('fmtHours: silent under a minute, minutes under an hour, tenths after', () => {
const { fmtHours } = load().__careerPassportTest;
assert.equal(fmtHours(0), '');
assert.equal(fmtHours(59), '');
assert.equal(fmtHours(60), '1 min');
assert.equal(fmtHours(1800), '30 min');
assert.equal(fmtHours(3600), '1 h');
assert.equal(fmtHours(51120), '14.2 h');
assert.equal(fmtHours(null), '');
assert.equal(fmtHours('junk'), '');
});
test('ppFillFraction: song progress toward the bar, in-progress only', () => {
const { ppFillFraction } = load().__careerPassportTest;
const p = (badge, q, songs) => ({ badge, qualifying_count: q, requirement: { songs } });
assert.equal(ppFillFraction(p('in_progress', 3, 5)), 0.6);
assert.equal(ppFillFraction(p('in_progress', 0, 5)), 0);
assert.equal(ppFillFraction(p('in_progress', 7, 5)), 1); // clamped
assert.equal(ppFillFraction(p('earned', 5, 5)), 0); // no fill once earned
assert.equal(ppFillFraction(p('shown_not_judged', 3, 5)), 0);
assert.equal(ppFillFraction(p('in_progress', 3, 0)), 0); // no bar → no fill
assert.equal(ppFillFraction(null), 0);
});
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -14,9 +14,5 @@
"intro": {
"video": "intro.mp4",
"audio": "bar-ambience.mp3"
},
"sfx": {
"up": "sfx-up.mp3",
"down": "sfx-down.mp3"
}
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
-1
View File
@@ -218,7 +218,6 @@ import {
setAvOffsetMs,
setInstrumentPathway,
setupAppUpdates,
setupWindowOptions,
syncDefaultArrangementPin,
} from './js/settings.js';
import {
+10 -14
View File
@@ -1908,21 +1908,17 @@ function createHighway() {
// never routable.
const isAudioUrl = msg.audio_url.startsWith('/audio/');
// "Full mix" covers BOTH single-mix pack shapes:
// - single-stem packs (stems: [full.ogg] only) — the pack's
// one stem IS its mixdown, so the server leaves it in the
// stems list, has_full_mix is false, and audio_url points
// at that one stem; and
// - legacy stem-less packs, whose mixdown sits outside stems
// behind the deprecated original_audio: key, so has_stems
// is false and audio_url == full_mix_url.
// Either way there is one audible source and no per-stem mix
// to preserve, so routing it natively loses nothing. A pack
// that retains its `full` stem ALONGSIDE separated stems is
// multi-stem (has_full_mix && has_stems) and stays out until
// Phase 2 — routing it natively would drop the mixer.
// - stem-less packs (original_audio: in the manifest,
// audio_url == original_audio_url), and
// - single-stem packs (stems: [full.ogg] only) — the server
// puts the full mix in the stems list, has_original_audio
// is false, and audio_url points at the one stem. With one
// stem there is no per-stem mix to preserve, so routing it
// natively loses nothing. Real multi-stem (>1) stays out
// until Phase 2.
const isFeedpakFullMix = !isAudioUrl
&& msg.audio_url.startsWith('/api/sloppak/')
&& ((!!msg.has_full_mix && !msg.has_stems)
&& ((!!msg.has_original_audio && !msg.has_stems)
|| (msg.stems || []).length === 1);
// Record the loaded song's audio so app.js can re-route it
// between the HTML5 and JUCE paths if the audio engine is
@@ -1947,7 +1943,7 @@ function createHighway() {
'isFeedpakFullMix=', isFeedpakFullMix,
'has_stems=', !!msg.has_stems,
'stems=', (msg.stems || []).length,
'has_full_mix=', !!msg.has_full_mix,
'has_original_audio=', !!msg.has_original_audio,
'format=', msg.format,
'alreadyLoaded=', alreadyLoaded,
'juceApi=', !!window.feedBackDesktop?.audio);
-42
View File
@@ -100,7 +100,6 @@ export async function loadSettings() {
// failed fetch below still leaves the desktop updater wired up.
// setupAppUpdates() is idempotent via _appUpdatesWired.
setupAppUpdates();
setupWindowOptions();
const resp = await fetch('/api/settings');
const data = await resp.json();
// Null-guard the form fields: on the v3 tabbed settings page the markup is
@@ -168,47 +167,6 @@ export async function loadSettings() {
hwcInitSettingsUI();
}
// ── Window options (desktop-only) ────────────────────────────────────────
// Desktop-only window preferences (start-in-fullscreen, …). The whole block
// stays hidden in the plain web / Docker app; unhide + wire only when the
// feedBack-desktop bridge (window.feedBackDesktop.window) exposes the getter
// and setter. Persistence lives desktop-side because only the Electron main
// process can read the pref at window-creation time — core just proxies.
export let _windowOptionsWired = false;
export function setupWindowOptions() {
const block = document.getElementById('window-options-block');
if (!block) return;
const winApi = window.feedBackDesktop?.window;
// Per-method capability check: a partial/older bridge may expose `window`
// without this shape. Leave the block hidden rather than half-wiring it.
if (!winApi
|| typeof winApi.getStartFullscreen !== 'function'
|| typeof winApi.setStartFullscreen !== 'function') {
return;
}
block.classList.remove('hidden');
const cb = document.getElementById('setting-start-fullscreen');
if (!cb) return;
// Hydrate from the desktop-persisted value. The getter may be sync or
// async (IPC round-trip); Promise.resolve normalises both.
Promise.resolve(winApi.getStartFullscreen()).then(function (on) {
cb.checked = !!on;
}).catch(function () { /* leave unchecked on error */ });
// Guard only the listener against double-binding; unhide + re-hydrate
// stay idempotent so re-entering Settings refreshes the checkbox.
if (!_windowOptionsWired) {
_windowOptionsWired = true;
cb.addEventListener('change', function () {
try { winApi.setStartFullscreen(cb.checked); } catch (_) { /* best-effort */ }
});
}
}
export const APP_UPDATE_CHANNELS = ['stable', 'rc', 'beta', 'alpha'];
export let _appUpdatesWired = false;
+1 -1
View File
File diff suppressed because one or more lines are too long
+2 -11
View File
@@ -120,20 +120,11 @@
if (!r.ok) return;
const data = await r.json();
_tuningsByKey = data.tunings || {};
// Build TUNING_NOTE from the lowest string of each tuning. Prefer the
// exact integer midis the server now sends (tuningMidis, #829) — the
// frequency path reconstructs the note via log2 against a hardcoded
// 440 and can land a semitone off at non-440 reference pitches.
// Frequencies remain the fallback for older cached responses.
const midisByKey = data.tuningMidis || {};
// Build TUNING_NOTE from the first (lowest) string frequency of each tuning.
TUNING_NOTE = {};
for (const key of Object.keys(_tuningsByKey)) {
for (const [name, freqs] of Object.entries(_tuningsByKey[key])) {
if (name in TUNING_NOTE) continue;
const midis = midisByKey[key] && midisByKey[key][name];
if (Array.isArray(midis) && midis.length > 0 && Number.isFinite(midis[0])) {
TUNING_NOTE[name] = NOTE_NAMES[((midis[0] % 12) + 12) % 12];
} else if (Array.isArray(freqs) && freqs.length > 0) {
if (!(name in TUNING_NOTE) && Array.isArray(freqs) && freqs.length > 0) {
TUNING_NOTE[name] = _freqToNote(freqs[0]);
}
}
-15
View File
@@ -753,21 +753,6 @@
<a href="https://github.com/got-feedback/feedback-desktop/releases" target="_blank" rel="noopener" class="text-accent hover:text-accent-light underline">download new versions from GitHub Releases</a>.
</p>
</div>
<!-- Window options — desktop-only; setupWindowOptions() unhides. -->
<div id="window-options-block" class="hidden">
<div class="fb-srow">
<div class="fb-srow-main">
<div class="fb-srow-title">Fullscreen</div>
<div class="fb-srow-desc">Run fee[dB]ack in fullscreen mode. On macOS, changes take effect on the next launch.</div>
</div>
<div class="fb-srow-control">
<label class="fb-switch">
<input type="checkbox" id="setting-start-fullscreen">
<span class="fb-switch-track"></span>
</label>
</div>
</div>
</div>
<!-- Library folder path -->
<div class="fb-srow fb-srow-stack">
<div class="fb-srow-main">
+3 -81
View File
@@ -27,60 +27,7 @@
let cur = null; // active session
let recordedThisSession = false;
// Wall-clock play time (career hours odometer). Accrued across
// play/resume ↔ pause/stop/ended spans — wall time, NOT song position:
// position deltas double-count A-B loops and mis-read seeks.
let playingSince = 0; // performance.now() at span start, 0 while not playing
let accruedSeconds = 0; // played time not yet sent
// Failed seconds keep their song identity — restoring them into the
// global accumulator would let the NEXT song claim them after a session
// switch. Bounded; oldest dropped beyond the cap (honest loss beats
// misattribution).
let pendingSeconds = []; // [{filename, arrangement, seconds}] awaiting retry
function queuePendingSeconds(filename, arrangement, seconds) {
pendingSeconds.push({ filename, arrangement, seconds });
if (pendingSeconds.length > 20) pendingSeconds.shift();
}
function retryPendingSeconds() {
if (!pendingSeconds.length) return;
const batch = pendingSeconds;
pendingSeconds = [];
for (const body of batch) {
post(body).then((r) => { if (r == null) queuePendingSeconds(body.filename, body.arrangement, body.seconds); });
}
}
function clockStart() { if (!playingSince) playingSince = performance.now(); }
function clockStop() {
if (!playingSince) return;
const delta = (performance.now() - playingSince) / 1000;
playingSince = 0;
// A single unbroken span beyond 2h of wall clock is a suspend/sleep
// artifact, not practice — clamp it.
if (Number.isFinite(delta) && delta > 0) accruedSeconds += Math.min(delta, 7200);
}
// Take whatever has accrued (closing any open span) for sending; the
// caller restores it if the POST fails so the time isn't lost.
function takeSeconds() {
clockStop();
const s = Math.round(accruedSeconds);
accruedSeconds = 0;
return s > 0 ? s : 0;
}
// Unsent seconds belong to the outgoing song/arrangement — flush before
// a session reset would re-attribute them.
function flushSeconds() {
const s = takeSeconds();
if (!s) return;
if (!cur || !cur.filename) return; // no session to attribute to — drop
const body = { filename: cur.filename, arrangement: cur.arrangement, seconds: s };
post(body).then((r) => { if (r == null) queuePendingSeconds(body.filename, body.arrangement, s); });
}
function reset(filename, arrangement) {
flushSeconds();
cur = {
filename: filename || null,
arrangement: Number.isFinite(arrangement) ? arrangement : 0,
@@ -101,10 +48,6 @@
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
// A 4xx/5xx JSON error body must read as FAILURE — callers
// re-queue accrued seconds on null, and a parsed error object
// would silently drop them.
if (!r.ok) return null;
try { return await r.json(); } catch (e) { return null; }
} catch (e) { return null; /* offline / endpoint absent — non-fatal */ }
}
@@ -141,7 +84,6 @@
if (!cur || !cur.filename || recordedThisSession) return;
if (!cur.scored || (cur.hits + cur.misses) <= 0) return; // no real scoring this session
recordedThisSession = true;
const seconds = takeSeconds();
const body = {
filename: cur.filename,
arrangement: cur.arrangement,
@@ -152,9 +94,7 @@
bestStreak: cur.bestStreak,
lastPlayPosition: Number.isFinite(position) ? position : cur.lastTime,
};
if (seconds) body.seconds = seconds;
post(body).then(async (response) => {
if (response == null && seconds) queuePendingSeconds(body.filename, body.arrangement, seconds);
await notifyProgression(response, body, !!natural);
// Refresh the profile badge AFTER the progression state moved so
// the rank/dB it renders are post-award values.
@@ -172,10 +112,7 @@
// Allow 0: restarting a song and stopping at the very beginning must be
// able to clear a stale Continue offset. Only negatives are invalid.
if (!Number.isFinite(position) || position < 0) return;
const seconds = takeSeconds();
const body = { filename: cur.filename, arrangement: cur.arrangement, lastPlayPosition: position };
if (seconds) body.seconds = seconds;
post(body).then((r) => { if (r == null && seconds) queuePendingSeconds(body.filename, body.arrangement, seconds); });
post({ filename: cur.filename, arrangement: cur.arrangement, lastPlayPosition: position });
}
// ── Session lifecycle ─────────────────────────────────────────────────--
@@ -227,28 +164,13 @@
});
});
// ── Play-time clock ───────────────────────────────────────────────────--
sm.on('song:play', () => { clockStart(); retryPendingSeconds(); });
sm.on('song:resume', clockStart);
// ── Finalize / resume-position ────────────────────────────────────────--
sm.on('song:ended', (e) => {
clockStop();
finalizeScored(e && e.detail && e.detail.time, true);
// Unscored natural end: no finalize POST and no position touch
// (Continue must not point at the end of the song) — bank the play
// time on its own.
flushSeconds();
});
sm.on('song:pause', (e) => {
clockStop();
touchPosition(e && e.detail && e.detail.time);
});
sm.on('song:ended', (e) => finalizeScored(e && e.detail && e.detail.time, true));
sm.on('song:pause', (e) => touchPosition(e && e.detail && e.detail.time));
sm.on('song:stop', (e) => {
// Record the scored session if it wasn't already (e.g. user closed the
// player before the track ended), then persist the resume position.
// Not a natural end — no calibration-retry prompt for deliberate quits.
clockStop();
const t = e && e.detail && e.detail.time;
finalizeScored(t, false);
touchPosition(t);
+1 -63
View File
@@ -34,7 +34,6 @@
const STREAK_MILESTONES = [25, 50, 100];
const CANPLAY_TIMEOUT_MS = 4000;
const DEV_FLAG_KEY = 'feedBack-venue-crowd-dev';
const SFX_KEY = 'feedBack-venue-crowd-sfx'; // 'on' | 'off' (default off)
// ---------------------------------------------------------------------
// Pure, clock-injected decision logic (unit-tested in
@@ -59,15 +58,6 @@
candidate = null;
lastSwitchAt = -Infinity;
},
// Commit a state NOW, bypassing stability/dwell (badge ceremony).
// Stamping lastSwitchAt makes the dwell window hold the forced
// state before the real perf machine can reassert.
force(state, nowMs) {
if (!CROWD_STATES.includes(state)) return;
current = state;
candidate = null;
lastSwitchAt = nowMs;
},
// Feed the latest perf state; returns the new crowd state when a
// transition commits, else null.
update(perfState, nowMs) {
@@ -154,11 +144,7 @@
video: abs(m.intro && m.intro.video),
audio: abs(m.intro && m.intro.audio),
};
const sfx = {
up: abs(m.sfx && m.sfx.up),
down: abs(m.sfx && m.sfx.down),
};
return { loops, stingers, intro, sfx };
return { loops, stingers, intro };
}
function ensureVideos() {
@@ -424,30 +410,6 @@
return true;
}
let _sfxEl = null;
function sfxEnabled() {
try { return localStorage.getItem(SFX_KEY) === 'on'; } catch (_) { return false; }
}
// One-shot crowd reaction on committed mood transitions (toggleable):
// up the ladder → cheer, down → boos. Committed transitions are already
// hysteresis-limited, so this can't spam.
function playMoodSfx(direction) {
if (!sfxEnabled() || !_manifest || !_manifest.sfx || _introActive) return;
const url = direction > 0 ? _manifest.sfx.up : _manifest.sfx.down;
if (!url || typeof document === 'undefined') return;
if (!_sfxEl) {
_sfxEl = document.createElement('audio');
_sfxEl.preload = 'auto';
_sfxEl.style.display = 'none';
document.body.appendChild(_sfxEl);
}
_sfxEl.src = url;
_sfxEl.volume = 0.6;
_sfxEl.play().catch(() => { /* pre-gesture; skip silently */ });
}
function onSongPlay() {
// Song audio starting is the hard cue: the ambience must yield.
fadeAudioOut(1000);
@@ -468,10 +430,8 @@
if (sting && !_introActive && CROWD_RANK[machine.current] >= CROWD_RANK.neutral) {
playStinger(sting);
}
const prevRank = CROWD_RANK[machine.current];
const next = machine.update(d.state, now());
if (next) {
playMoodSfx(CROWD_RANK[next] - prevRank);
// A stinger or the intro owns the idle layer; defer the switch.
if (_stingerUntilEnded || _introActive) _pendingLoop = next;
else showLoop(next, FADE_MS);
@@ -529,7 +489,6 @@
_introGen++;
_introActive = false;
stopAudio();
if (_sfxEl && !_sfxEl.paused) _sfxEl.pause();
_stingerUntilEnded = false;
_pendingLoop = null;
_loadingLoop = null;
@@ -605,26 +564,6 @@
if (dev && !_manifest) setManifest(dev);
}
// Badge-ceremony hook (career passports): the crowd erupts NOW — ecstatic
// loop bypassing stability/dwell (the dwell window then holds it while
// the real perf state waits its turn) plus a cheer. Degrades to a no-op
// without a pack / outside the player, like every other entry point.
function celebrate() {
if (!_venueActive || !_manifest || !_videos[0]) return false;
machine.force('ecstatic', now());
if (_stingerUntilEnded || _introActive) {
// A stinger/intro owns the idle layer (likely the end-of-song
// accuracy cheer — the crowd is already reacting); queue the
// ecstatic loop for when it ends, same as onPerformanceState.
_pendingLoop = 'ecstatic';
} else {
showLoop('ecstatic', FADE_MS);
_lastStingerAt = -Infinity; // a badge earn always gets its cheer
playStinger('cheer');
}
return true;
}
function getState() {
return {
venueActive: _venueActive,
@@ -650,7 +589,6 @@
setVenueActive,
bindRuntime,
getState,
celebrate,
};
if (root) root.v3VenueCrowd = api;
-19
View File
@@ -128,22 +128,3 @@ test('venue-scene-3d activates/deactivates the crowd layer', () => {
assert.match(src, /syncCrowd\(false\)/);
assert.match(src, /v3VenueCrowd/);
});
test('machine.force commits instantly and dwell holds the forced state', () => {
const m = crowd.createCrowdMachine();
m.force('ecstatic', 100000);
assert.equal(m.current, 'ecstatic');
// The real perf state cannot reassert until the dwell window passes.
m.update('smoke', 100000 + crowd.STABLE_MS);
assert.equal(m.update('smoke', 100000 + crowd.DWELL_MS - 1), null);
assert.equal(m.current, 'ecstatic');
assert.equal(m.update('smoke', 100000 + crowd.DWELL_MS), 'bored');
// Bogus states are ignored.
m.force('confused', 200000);
assert.equal(m.current, 'bored');
});
test('celebrate() is exported and no-ops without a manifest/active venue', () => {
assert.equal(typeof crowd.celebrate, 'function');
assert.equal(crowd.celebrate(), false);
});
+4 -6
View File
@@ -26,8 +26,7 @@ class FakeMetaDb:
self.conn.execute(
"""CREATE TABLE song_stats (
filename TEXT, arrangement TEXT, best_accuracy REAL,
last_played_at TEXT,
seconds_total REAL NOT NULL DEFAULT 0
last_played_at TEXT
)"""
)
self.conn.execute(
@@ -38,10 +37,9 @@ class FakeMetaDb:
)
def add(self, filename, arrangement, best_accuracy, in_library=True,
genre="", arrangements=None, last_played_at=None, seconds_total=0):
self.conn.execute("INSERT INTO song_stats VALUES (?, ?, ?, ?, ?)",
(filename, arrangement, best_accuracy, last_played_at,
seconds_total))
genre="", arrangements=None, last_played_at=None):
self.conn.execute("INSERT INTO song_stats VALUES (?, ?, ?, ?)",
(filename, arrangement, best_accuracy, last_played_at))
if in_library:
self.conn.execute(
"INSERT INTO songs SELECT ?, ?, ?, ?, ? WHERE NOT EXISTS "
+2 -104
View File
@@ -27,47 +27,13 @@ def _passport(client, instrument="guitar", genre_key="blues"):
def test_badge_earned_at_five_genre_songs_two_stars(client, meta_db):
# Soul has no curated drill requirement — songs alone mint the badge.
for i in range(5):
meta_db.add(f"soul{i}.feedpak", 0, 0.8, genre="Soul", arrangements=LEAD)
_open(client, "guitar", "Soul")
p = _passport(client, "guitar", "soul")
assert p["badge"] == "earned"
assert p["qualifying_count"] == 5
assert all(s["qualifies"] and s["stars"] == 2 for s in p["songs"])
def test_shipped_blues_drill_gates_and_keys_cleared_clears_it(client, meta_db):
# Blues ships a guitar drill (blues_shuffle): songs alone are not enough.
for i in range(5):
meta_db.add(f"blues{i}.feedpak", 0, 0.8, genre="Blues", arrangements=LEAD)
_open(client)
p = _passport(client)
assert p["badge"] == "in_progress"
assert p["drills"]["required"] == ["blues_shuffle"]
# One key cleared (a top-tier clean pass) counts as cleared — the depth
# rungs are a higher bar than Bronze needs.
res = client.post("/api/plugins/career/drill-state", json={
"mode": "casual", "xp": 10,
"byNode": {"blues_shuffle": {"reps": 12, "keysCleared": ["E"],
"depth": {"travel": None, "clean": None},
"masteredAt": None}}})
assert res.status_code == 200
p = _passport(client)
assert p["drills"]["cleared"] == ["blues_shuffle"]
assert p["badge"] == "earned"
def test_drill_lists_are_per_instrument(client, meta_db):
# Keys is graded but Blues curates only a GUITAR drill — a keys passport
# earns on songs alone.
keys_arr = [{"type": "lead", "name": "Keys"}]
for i in range(5):
meta_db.add(f"kb{i}.feedpak", 0, 0.9, genre="Blues", arrangements=keys_arr)
_open(client, "keys")
p = _passport(client, "keys")
assert p["drills"]["required"] == []
assert p["badge"] == "earned"
assert p["qualifying_count"] == 5
assert all(s["qualifies"] and s["stars"] == 2 for s in p["songs"])
def test_badge_in_progress_below_the_bar(client, meta_db):
@@ -177,71 +143,3 @@ def test_drill_state_validation(client):
huge = {"byNode": {"pad": "x" * (300 * 1024)}}
assert client.post("/api/plugins/career/drill-state",
json=huge).status_code == 413
def test_hours_odometer_sums_seconds_per_instrument_and_genre(client, meta_db):
both = [{"type": "lead", "name": "Lead"}, {"type": "bass", "name": "Bass"}]
# Two lead arrangements' time sums; the bass row stays on the bass passport.
meta_db.add("a.feedpak", 0, 0.8, genre="Blues", arrangements=both, seconds_total=600)
meta_db.add("b.feedpak", 0, 0.8, genre="Blues", arrangements=both, seconds_total=300)
meta_db.add("b.feedpak", 1, 0.9, genre="Blues", arrangements=both, seconds_total=1200)
_open(client, "guitar")
_open(client, "bass")
assert _passport(client, "guitar")["seconds_total"] == 900
assert _passport(client, "bass")["seconds_total"] == 1200
def test_drill_state_merge_is_gained_only(client, meta_db):
# A cleared drill survives a later STALE snapshot that lacks it
# (multi-browser race / settings import / the boot relay).
for i in range(5):
meta_db.add(f"blues{i}.feedpak", 0, 0.8, genre="Blues", arrangements=LEAD)
_open(client)
client.post("/api/plugins/career/drill-state", json={
"byNode": {"blues_shuffle": {"keysCleared": ["E"]}}})
assert _passport(client)["badge"] == "earned"
# Stale relay: empty byNode, then one with the node but nothing earned.
client.post("/api/plugins/career/drill-state", json={"byNode": {}})
client.post("/api/plugins/career/drill-state", json={
"byNode": {"blues_shuffle": {"reps": 2, "keysCleared": [],
"depth": {"travel": None}, "masteredAt": None}}})
p = _passport(client)
assert p["drills"]["cleared"] == ["blues_shuffle"]
assert p["badge"] == "earned"
def test_genre_families_inherit_drills(client, meta_db):
# 'death metal' has no exact entry — it inherits the metal family's drill.
for i in range(5):
meta_db.add(f"dm{i}.feedpak", 0, 0.9, genre="Death Metal", arrangements=LEAD)
_open(client, "guitar", "Death Metal")
p = _passport(client, "guitar", "death metal")
assert p["drills"]["required"] == ["melodic_metal_gallop"]
assert p["badge"] == "in_progress"
# 'metalcore' (single word) matches by substring, no alias needed.
_open(client, "guitar", "Metalcore")
assert _passport(client, "guitar", "metalcore")["drills"]["required"] == \
["melodic_metal_gallop"]
# 'blues rock' resolves by family LIST ORDER: blues comes before rock.
_open(client, "guitar", "Blues Rock")
assert _passport(client, "guitar", "blues rock")["drills"]["required"] == \
["blues_shuffle"]
# A genre outside every family stays songs-only.
_open(client, "guitar", "Reggae")
assert _passport(client, "guitar", "reggae")["drills"]["required"] == []
# Exact per-genre entries still beat the family (the shipped 'metal' entry
# IS the exact entry for genre key 'metal').
_open(client, "guitar", "Metal")
assert _passport(client, "guitar", "metal")["drills"]["required"] == \
["melodic_metal_gallop"]
def test_family_drills_stay_per_instrument(client, meta_db):
# Family inheritance must not leak guitar drills onto other instruments.
keys_arr = [{"type": "lead", "name": "Keys"}]
for i in range(5):
meta_db.add(f"kdm{i}.feedpak", 0, 0.9, genre="Death Metal", arrangements=keys_arr)
_open(client, "keys", "Death Metal")
p = _passport(client, "keys", "death metal")
assert p["drills"]["required"] == []
assert p["badge"] == "earned"
-48
View File
@@ -273,51 +273,3 @@ def test_title_keyset_paging_is_complete_with_overrides(client, server):
if not cursor:
break
assert sorted(seen) == ["a.archive", "b.archive", "c.archive"] # each exactly once
def test_enrichment_genre_fallback_precedence(client, server):
# Precedence: override → pack genre → MusicBrainz enrichment (matched only).
_put(server, "a.archive", title="A", genre="Rock") # pack wins over enrichment
_put(server, "b.archive", title="B", genre="") # falls back to enrichment
_put(server, "c.archive", title="C", genre="") # override beats enrichment
_put(server, "d.archive", title="D", genre="") # unmatched candidate: ignored
ins = "INSERT INTO song_enrichment (filename, match_state, genres) VALUES (?, ?, ?)"
server.meta_db.conn.execute(ins, ("a.archive", "matched", '["metal"]'))
server.meta_db.conn.execute(ins, ("b.archive", "matched", '["progressive rock", "rock"]'))
server.meta_db.conn.execute(ins, ("c.archive", "matched", '["jazz"]'))
server.meta_db.conn.execute(ins, ("d.archive", "review", '["country"]'))
_put(server, "e.archive", title="E", genre="") # manual pin is trusted too
server.meta_db.conn.execute(ins, ("e.archive", "manual", '["ska"]'))
server.meta_db.conn.commit()
server.meta_db.set_song_override("c.archive", "genre", value="City Pop")
genres = client.get("/api/library/genres").json()["genres"]
assert "Rock" in genres # pack value kept for a
assert "metal" not in genres # enrichment never overrides a pack genre
assert "progressive rock" in genres # b: enrichment primary ([0]) surfaces
assert "City Pop" in genres and "jazz" not in genres # override beats enrichment
assert "country" not in genres # review/failed candidates never leak
assert "ska" in genres # user-pinned (manual) matches count
# Filtering by the enriched genre finds the song.
r = client.get("/api/library", params={"genre": "progressive rock"}).json()
assert [s["filename"] for s in r["songs"]] == ["b.archive"]
def test_no_enrichment_and_no_overrides_uses_plain_column(server):
_put(server, "a.archive", title="A", genre="Rock")
assert server.meta_db._effective_genre_expr() == "genre"
def test_overrides_without_enrichment_table_stay_safe(server):
# A stand-in scenario: overrides exist but song_enrichment is gone — the
# expression must not reference the missing table.
server.meta_db.conn.execute("DROP TABLE song_enrichment")
_put(server, "a.archive", title="A", genre="")
server.meta_db.set_song_override("a.archive", "genre", value="City Pop")
expr = server.meta_db._effective_genre_expr()
assert "song_enrichment" not in expr
# And it still evaluates: the override surfaces through the facet query.
row = server.meta_db.conn.execute(
f"SELECT {expr} FROM songs WHERE filename = 'a.archive'").fetchone()
assert row[0] == "City Pop"
-429
View File
@@ -1,429 +0,0 @@
"""tools/migrate_full_mix_stem.py — packs off the deprecated `original_audio:` key.
The migration moves real audio inside tens of thousands of archives, so the
interesting cases are the ones where it must NOT act: a pack it would corrupt, a
pack it has already done, a pack whose mixdown isn't where the key claims.
"""
from __future__ import annotations
import importlib.util
import zipfile
from pathlib import Path
import pytest
import yaml
_SPEC = importlib.util.spec_from_file_location(
"migrate_full_mix_stem",
Path(__file__).resolve().parent.parent / "tools" / "migrate_full_mix_stem.py",
)
mig = importlib.util.module_from_spec(_SPEC)
_SPEC.loader.exec_module(mig)
def _manifest(**extra) -> dict:
m = {
"feedpak_version": "1.13.0",
"title": "T",
"artist": "A",
"duration": 1.0,
"arrangements": [{"id": "lead", "file": "arrangements/lead.json"}],
"stems": [
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
{"id": "drums", "file": "stems/drums.ogg", "default": "on"},
],
"original_audio": "original/full.ogg",
}
m.update(extra)
return m
def _write_pack(path: Path, manifest: dict, files: dict[str, bytes] | None = None) -> Path:
files = files or {
"original/full.ogg": b"MIXDOWN",
"stems/guitar.ogg": b"g",
"stems/drums.ogg": b"d",
"arrangements/lead.json": b"{}",
}
with zipfile.ZipFile(path, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("manifest.yaml", yaml.safe_dump(manifest, sort_keys=False))
for name, data in files.items():
zf.writestr(name, data)
return path
def _read(path: Path) -> tuple[dict, set[str]]:
with zipfile.ZipFile(path) as zf:
return yaml.safe_load(zf.read("manifest.yaml")), set(zf.namelist())
# ── plan_manifest: the decisions, without the archives ──────────────────────
def test_plan_adds_the_full_stem_and_drops_the_key():
new, move = mig.plan_manifest(_manifest())
assert move == "original/full.ogg"
assert "original_audio" not in new
assert new["stems"][0] == {
"id": "full",
"file": "stems/full.ogg",
"default": "off",
}
# The separated stems survive, in order, untouched.
assert [s["id"] for s in new["stems"]] == ["full", "guitar", "drums"]
assert new["feedpak_version"] == "1.15.0"
def test_plan_marks_the_retained_mixdown_default_off():
"""The one line that keeps a pre-1.15.0 reader from doubling the song: a
reader that sums every stem still won't play `full` on open if it honours
`default`, which has been normative since 1.0.0."""
new, _ = mig.plan_manifest(_manifest())
assert new["stems"][0]["default"] == "off"
def test_plan_marks_a_sole_mixdown_default_on():
"""With no separated stems the mixdown IS the audio — off would mute the pack."""
new, _ = mig.plan_manifest(_manifest(stems=[]))
assert new["stems"] == [{"id": "full", "file": "stems/full.ogg", "default": "on"}]
def test_plan_preserves_unknown_keys_verbatim():
"""Spec §3: a writer that re-emits a pack SHOULD preserve unknown keys."""
new, _ = mig.plan_manifest(_manifest(source_tool="ExampleTool v1.2.3", rigs="rigs.json"))
assert new["source_tool"] == "ExampleTool v1.2.3"
assert new["rigs"] == "rigs.json"
def test_plan_skips_an_already_migrated_pack():
m = _manifest(
stems=[{"id": "full", "file": "stems/full.ogg", "default": "off"}],
)
del m["original_audio"]
with pytest.raises(mig.Skip):
mig.plan_manifest(m)
def test_plan_skips_a_pack_that_never_had_the_key():
m = _manifest()
del m["original_audio"]
with pytest.raises(mig.Skip):
mig.plan_manifest(m)
def test_plan_drops_a_stale_key_without_moving_anything():
"""Mixdown already a stem, dead key lingering beside it."""
new, move = mig.plan_manifest(
_manifest(stems=[{"id": "full", "file": "stems/full.ogg", "default": "off"}])
)
assert move == ""
assert "original_audio" not in new
assert [s["id"] for s in new["stems"]] == ["full"]
def test_plan_forces_an_existing_full_stem_off_beside_instrument_stems():
"""Dropping the stale key is not enough if the mixdown it duplicated is left
ENABLED: a reader that honours `default` would then play the whole song on top
of the stems on open. The migration must not hand back a pack in the exact
state it exists to remove."""
new, move = mig.plan_manifest(
_manifest(
stems=[
{"id": "full", "file": "stems/full.ogg", "default": "on"},
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
]
)
)
assert move == ""
assert new["stems"][0] == {"id": "full", "file": "stems/full.ogg", "default": "off"}
assert new["stems"][1]["default"] == "on" # instruments untouched
def test_plan_leaves_a_sole_full_stem_enabled_when_dropping_a_stale_key():
"""No instruments beside it — the mixdown IS the audio. Forcing it off here
would mute the pack."""
new, _ = mig.plan_manifest(
_manifest(stems=[{"id": "full", "file": "stems/full.ogg", "default": "on"}])
)
assert new["stems"] == [{"id": "full", "file": "stems/full.ogg", "default": "on"}]
def test_plan_needs_no_move_when_the_key_already_points_at_the_canonical_path():
new, move = mig.plan_manifest(_manifest(original_audio="stems/full.ogg"))
assert move == ""
assert new["stems"][0]["file"] == "stems/full.ogg"
# ── migrate_zip: the archive rewrite ────────────────────────────────────────
def test_migrate_moves_the_audio_and_rewrites_the_manifest(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
manifest, names = _read(pak)
assert "original/full.ogg" not in names # the invented directory is gone
assert "stems/full.ogg" in names # audio lives where the format says
assert "original_audio" not in manifest
assert manifest["stems"][0]["id"] == "full"
assert mig.verify_zip(pak) == "ok"
def test_migrate_preserves_the_mixdown_bytes(tmp_path: Path):
"""It is a rename, not a re-encode. Losing a byte here loses the master audio."""
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
mig.migrate_zip(pak, dry_run=False)
with zipfile.ZipFile(pak) as zf:
assert zf.read("stems/full.ogg") == b"MIXDOWN"
assert zf.read("stems/guitar.ogg") == b"g"
def test_migrate_is_idempotent(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=False) == "skip"
assert pak.read_bytes() == before # a re-run touches nothing
def test_dry_run_changes_nothing(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=True) == "would-migrate"
assert pak.read_bytes() == before
def test_migrate_refuses_when_the_mixdown_is_absent(tmp_path: Path):
"""The key points at audio the archive doesn't contain. Fabricating a stem
entry for a missing file would break every reader refuse, don't guess."""
pak = _write_pack(
tmp_path / "song.feedpak",
_manifest(),
files={"stems/guitar.ogg": b"g", "arrangements/lead.json": b"{}"},
)
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=False) == "missing-audio"
assert pak.read_bytes() == before
def test_migrate_refuses_when_the_target_path_is_taken(tmp_path: Path):
"""A `stems/full.ogg` that is NOT the mixdown already occupies the target.
Overwriting it would destroy a stem."""
pak = _write_pack(
tmp_path / "song.feedpak",
_manifest(),
files={
"original/full.ogg": b"MIXDOWN",
"stems/full.ogg": b"SOMETHING-ELSE",
"arrangements/lead.json": b"{}",
},
)
assert mig.migrate_zip(pak, dry_run=False) == "target-occupied"
with zipfile.ZipFile(pak) as zf:
assert zf.read("stems/full.ogg") == b"SOMETHING-ELSE"
def test_migrate_drops_a_stale_key_beside_a_non_canonical_full_stem(tmp_path: Path):
"""The mixdown is already a stem, but at a path of the pack's own choosing —
which is legal (§2.2: readers resolve through the manifest, never by
filename). Only the dead key needs removing. Demanding `stems/full.ogg` here
would reject a perfectly valid pack as `missing-audio`."""
m = _manifest(stems=[{"id": "full", "file": "audio/mixdown.ogg", "default": "off"}])
pak = _write_pack(
tmp_path / "song.feedpak",
m,
files={"audio/mixdown.ogg": b"MIXDOWN", "arrangements/lead.json": b"{}"},
)
assert mig.migrate_zip(pak, dry_run=False) == "migrated"
manifest, names = _read(pak)
assert "original_audio" not in manifest
assert manifest["stems"] == [
{"id": "full", "file": "audio/mixdown.ogg", "default": "off"}
]
assert "audio/mixdown.ogg" in names # the audio never moved
assert mig.verify_zip(pak) == "ok"
# ── verify_zip ──────────────────────────────────────────────────────────────
def test_verify_rejects_a_retained_mixdown_that_plays_on_open(tmp_path: Path):
"""The hazard the migration must never create: `full` alongside instrument
stems AND default-on means a summing reader plays the whole song twice."""
m = _manifest(
stems=[
{"id": "full", "file": "stems/full.ogg", "default": "on"},
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
]
)
del m["original_audio"]
pak = _write_pack(
tmp_path / "song.feedpak",
m,
files={"stems/full.ogg": b"M", "stems/guitar.ogg": b"g"},
)
assert mig.verify_zip(pak) == "full-stem-default-on"
@pytest.mark.parametrize(
"default, expected",
[
({"default": "off"}, "ok"), # the one safe, canonical shape
({"default": "OFF"}, "ok"), # case-insensitive
({"default": " off "}, "ok"), # surrounding whitespace tolerated
({}, "full-stem-default-not-off"), # MISSING — core defaults to True (ON)
({"default": ""}, "full-stem-default-not-off"), # empty → ON in core
({"default": False}, "full-stem-default-not-off"), # boolean, not the string
({"default": True}, "full-stem-default-on"), # boolean truthy → plays
({"default": "false"}, "full-stem-default-not-off"), # off-ish but non-canonical
({"default": "0"}, "full-stem-default-not-off"),
({"default": "no"}, "full-stem-default-not-off"),
({"default": "maybe"}, "full-stem-default-not-off"), # malformed
({"default": "on"}, "full-stem-default-on"),
({"default": "yes"}, "full-stem-default-on"),
({"default": "1"}, "full-stem-default-on"),
],
)
def test_verify_requires_an_explicit_off_on_a_retained_mixdown(tmp_path, default, expected):
"""Beside instrument stems, `full` is safe only with an explicit normalized
`off`. Core defaults an ABSENT `default` to ON and treats empty/unknown as
ON, so a missing or blank default is the double-audio hazard itself, not a
lesser one `verify` must not certify it."""
m = _manifest(
stems=[
{"id": "full", "file": "stems/full.ogg", **default},
{"id": "guitar", "file": "stems/guitar.ogg", "default": "on"},
]
)
del m["original_audio"]
pak = _write_pack(
tmp_path / f"{tmp_path.name}.feedpak",
m,
files={"stems/full.ogg": b"M", "stems/guitar.ogg": b"g"},
)
assert mig.verify_zip(pak) == expected
def test_verify_ignores_default_on_a_sole_full_stem(tmp_path: Path):
"""A single `full` stem IS the audio — the len>1 gate means its default is
not policed, so an on/absent default is fine (off would mute the pack)."""
for default in ({"default": "on"}, {}, {"default": ""}):
m = _manifest(stems=[{"id": "full", "file": "stems/full.ogg", **default}])
del m["original_audio"]
pak = _write_pack(
tmp_path / f"{tmp_path.name}-{len(default)}.feedpak",
m,
files={"stems/full.ogg": b"M"},
)
assert mig.verify_zip(pak) == "ok"
def test_verify_rejects_an_unmigrated_pack(tmp_path: Path):
pak = _write_pack(tmp_path / "song.feedpak", _manifest())
assert mig.verify_zip(pak) == "still-has-key"
# ── Unsafe manifest paths must not be laundered into playable audio ─────────
@pytest.mark.parametrize(
"rel", ["../outside.ogg", "/etc/passwd", "a/../../x.ogg", "C:/x.ogg", "a\\b.ogg"]
)
def test_migrate_refuses_an_unsafe_full_mix_path(tmp_path: Path, rel: str):
"""Core's loader REFUSES a full-mix path that escapes the pack — such a pack
simply has no full mix, and the audio is inert. Migrating it into
`stems/full.ogg` would take content the reader deliberately rejected and hand
it back as a valid, playable stem. Report it; never promote it."""
pak = _write_pack(
tmp_path / "song.feedpak",
_manifest(original_audio=rel),
files={rel: b"EVIL", "stems/guitar.ogg": b"g"},
)
before = pak.read_bytes()
assert mig.migrate_zip(pak, dry_run=False) == "unsafe-path"
assert pak.read_bytes() == before
def test_safe_relpath_accepts_ordinary_pack_paths():
assert mig.is_safe_relpath("stems/full.ogg")
assert mig.is_safe_relpath("original/full.ogg")
assert not mig.is_safe_relpath("")
assert not mig.is_safe_relpath("a//b.ogg")
# ── Damaged packs must not abort the run ────────────────────────────────────
def test_a_corrupt_archive_is_reported_not_fatal(tmp_path: Path, capsys):
"""A real library has damage in it — a truncated download, an archive left
half-written by an interrupted converter. One of those must not kill a
50,000-pack run and throw away the summary: the pack is reported, skipped,
and everything else still migrates."""
good = _write_pack(tmp_path / "good.feedpak", _manifest())
bad = tmp_path / "bad.feedpak"
bad.write_bytes(b"this is not a zip file at all")
rc = mig.main([str(tmp_path)])
out = capsys.readouterr().out
assert rc == 1 # a problem pack fails the run's exit code
assert "corrupt-zip" in out
assert "migrated" in out
assert mig.verify_zip(good) == "ok" # the healthy pack still got migrated
assert bad.read_bytes() == b"this is not a zip file at all" # untouched
# ── Directory-form (authoring) packs are discovered, not silently skipped ────
def _write_dir_pack(path: Path, manifest: dict, files: dict[str, bytes] | None = None) -> Path:
"""Build a directory-form pack (`song.sloppak/`), the authoring shape."""
files = files or {
"original/full.ogg": b"MIXDOWN",
"stems/guitar.ogg": b"g",
"stems/drums.ogg": b"d",
"arrangements/lead.json": b"{}",
}
path.mkdir()
(path / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
for name, data in files.items():
p = path / name
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(data)
return path
def test_iter_packs_discovers_directory_form_packs(tmp_path: Path):
"""A `song.sloppak/` directory is a pack; os.walk must yield it whole and
NOT descend into it (its stems/ are contents, not packs)."""
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
z = _write_pack(tmp_path / "other.feedpak", _manifest())
found = set(mig.iter_packs(tmp_path))
assert d in found and z in found
# Nothing inside the directory pack was yielded as its own pack.
assert not any(d in p.parents for p in found)
def test_iter_packs_yields_a_directly_passed_dir_pack(tmp_path: Path):
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
assert list(mig.iter_packs(d)) == [d]
def test_directory_form_pack_is_reported_not_silently_skipped(tmp_path: Path, capsys):
"""The migrator rewrites single-file packs atomically; a directory can't be
swapped that way, so it is surfaced as a problem rather than vanishing from
the run (the silent-skip this guards against) or being rewritten unsafely."""
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
good = _write_pack(tmp_path / "good.feedpak", _manifest())
rc = mig.main([str(tmp_path)])
out = capsys.readouterr().out
assert rc == 1 # a reported problem fails the exit code
assert "dir-form-unsupported" in out
assert mig.verify_zip(good) == "ok" # the zip pack still migrated
# The directory pack is untouched: legacy key intact, mixdown not moved.
manifest = yaml.safe_load((d / "manifest.yaml").read_text())
assert manifest.get("original_audio") == "original/full.ogg"
assert (d / "original" / "full.ogg").read_bytes() == b"MIXDOWN"
def test_verify_reports_directory_form_packs(tmp_path: Path):
d = _write_dir_pack(tmp_path / "song.sloppak", _manifest())
assert mig.verify_pack(d) == "dir-form-unsupported"
-277
View File
@@ -1,277 +0,0 @@
"""The sloppak loader's handling of a pack's complete mixdown (#933).
The mixdown is a stem: feedpak spec §5.3 RESERVES the id `full` for it. It is a
mixdown, not a layer it already contains every instrument, so a reader that
sums `stems` must never include it in that sum, and `load_song()` therefore
lifts it OUT of `LoadedSloppak.stems` and onto `LoadedSloppak.full_mix`.
Also covers the DEPRECATED `original_audio:` manifest key a key this repo
invented (#583) before the spec reserved `full`, which every pack in the wild
still carries. We read it as a fallback so those packs keep their full mix; we
never write it. Those tests are the deprecation contract: they go when the key
does (#945).
"""
from __future__ import annotations
import json
from pathlib import Path
import yaml
import sloppak as sloppak_mod
def _write_dir_sloppak(
root: Path,
manifest_extras: dict,
*,
write_legacy_full_mix: bool = False,
stems: list[dict] | None = None,
) -> Path:
"""Build a minimal directory-form sloppak that load_song will accept.
Uses the tmp_path leaf name to make the sloppak filename unique per test,
avoiding the module-level ``resolve_source_dir`` cache being poisoned by a
previous test that happened to share the same "song.sloppak" filename.
"""
pak = root / f"{root.name}.sloppak"
pak.mkdir()
arr_dir = pak / "arrangements"
arr_dir.mkdir()
arr = {
"name": "Lead",
"tuning": [0, 0, 0, 0, 0, 0],
"capo": 0,
"notes": [],
"chords": [],
"anchors": [],
"handshapes": [],
"templates": [],
"beats": [],
"sections": [],
}
(arr_dir / "lead.json").write_text(json.dumps(arr))
manifest = {
"title": "Test",
"artist": "Tester",
"album": "",
"year": 2026,
"duration": 10.0,
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
"stems": (
stems
if stems is not None
else [{"id": "guitar", "file": "stems/guitar.ogg", "default": True}]
),
}
manifest.update(manifest_extras)
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
if write_legacy_full_mix:
orig_dir = pak / "original"
orig_dir.mkdir()
# The loader only checks presence (is_file); contents are irrelevant.
(orig_dir / "full.ogg").write_bytes(b"OggS-not-real")
return pak
def _load(pak_path: Path, tmp_path: Path):
dlc_root = pak_path.parent
cache = tmp_path / "cache"
cache.mkdir()
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
def _separated(**extra) -> list[dict]:
"""A separated pack that RETAINS its mixdown, as spec §5.3 asks writers to."""
return [
{"id": "full", "file": "stems/full.ogg", "default": False, **extra},
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
{"id": "drums", "file": "stems/drums.ogg", "default": True},
]
# ── The `full` stem is the mixdown (spec §5.3) ───────────────────────────────
def test_full_stem_is_surfaced_as_the_mixdown(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
loaded = _load(pak, tmp_path)
# Manifest-relative, so the WS builds its URL exactly as it builds a stem's.
assert loaded.full_mix == "stems/full.ogg"
def test_full_stem_is_removed_from_the_stem_list(tmp_path: Path):
"""The regression this whole change exists to prevent.
Every consumer sums `stems` into one mix and renders one fader per entry. The
mixdown already contains every instrument, so leaving it in the list doubles
the entire song and muting `guitar` would still leave guitar audible inside
it. That exact trap is why the packer invented `original_audio` rather than
putting the mixdown where the format says it goes.
"""
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
loaded = _load(pak, tmp_path)
assert [s["id"] for s in loaded.stems] == ["guitar", "drums"]
def test_single_mix_pack_keeps_full_as_its_only_stem(tmp_path: Path):
"""A pack whose ONLY stem is `full` is a single-mix pack, not a separated one.
There are no instruments to be pristine against, so the mixdown stays the sole
playable stem and nothing is surfaced separately. Anything else would strip the
stem list of the most common pack shape in the library and leave it silent.
"""
pak = _write_dir_sloppak(
tmp_path, {}, stems=[{"id": "full", "file": "stems/full.ogg", "default": True}]
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
assert [s["id"] for s in loaded.stems] == ["full"]
def test_every_full_entry_is_removed_not_just_the_first(tmp_path: Path):
"""A malformed pack listing `full` twice must not leave one behind.
Removing the mixdown by object identity would drop only the entry we surface
and leave its duplicate in the stem list a whole copy of the song, summed
with the instruments. That is the exact bug this partition prevents, so a
duplicate must not smuggle it back in.
"""
pak = _write_dir_sloppak(
tmp_path,
{},
stems=[
{"id": "full", "file": "stems/full.ogg", "default": False},
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
{"id": "full", "file": "original/full.ogg", "default": True},
],
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix == "stems/full.ogg"
assert [s["id"] for s in loaded.stems] == ["guitar"]
def test_separated_pack_without_a_full_stem_has_no_mixdown(tmp_path: Path):
"""Stems only, mixdown discarded — the pre-1.15.0 shape. Nothing to surface."""
pak = _write_dir_sloppak(
tmp_path,
{},
stems=[
{"id": "guitar", "file": "stems/guitar.ogg", "default": True},
{"id": "drums", "file": "stems/drums.ogg", "default": True},
],
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
assert [s["id"] for s in loaded.stems] == ["guitar", "drums"]
def test_single_mix_pack_ignores_a_lingering_deprecated_key(tmp_path: Path):
"""`full` is the pack's only stem AND the old key is still there.
The stem wins, and it stays the sole playable stem falling back to the key
would surface the mixdown twice: once as the stem the player is already
playing, and once as a "pristine" track for it to cross over to.
"""
pak = _write_dir_sloppak(
tmp_path,
{"original_audio": "original/full.ogg"},
stems=[{"id": "full", "file": "stems/full.ogg", "default": True}],
write_legacy_full_mix=True,
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
assert [s["id"] for s in loaded.stems] == ["full"]
def test_full_stem_wins_over_the_deprecated_key(tmp_path: Path):
"""A migrated pack that still carries the old key must use the stem."""
pak = _write_dir_sloppak(
tmp_path,
{"original_audio": "original/full.ogg"},
stems=_separated(),
write_legacy_full_mix=True,
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix == "stems/full.ogg"
# ── The library index must not advertise the mixdown as an instrument ────────
def test_extract_meta_excludes_the_mixdown_from_stem_ids(tmp_path: Path):
"""The library's stem chips / stem_count come from here, and must agree with
load_song() otherwise the filter offers a "full" chip beside guitar+drums
and counts a third stem that no mixer will ever show."""
pak = _write_dir_sloppak(tmp_path, {}, stems=_separated())
meta = sloppak_mod.extract_meta(pak)
assert meta["stem_ids"] == ["guitar", "drums"]
assert meta["stem_count"] == 2
def test_extract_meta_keeps_full_for_a_single_mix_pack(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {}, stems=[{"id": "full", "file": "stems/full.ogg", "default": True}]
)
meta = sloppak_mod.extract_meta(pak)
assert meta["stem_ids"] == ["full"]
assert meta["stem_count"] == 1
# ── DEPRECATED `original_audio:` fallback — delete with the key (#945) ───────
def test_legacy_key_still_provides_the_full_mix(tmp_path: Path):
"""Every pack written before the spec reserved `full` looks like this. Dropping
the read would silently take the pristine mix away from all of them."""
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix == "original/full.ogg"
# The legacy mixdown lives OUTSIDE `stems`, so the stem list is untouched.
assert [s["id"] for s in loaded.stems] == ["guitar"]
def test_legacy_key_absent_means_no_full_mix(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {}, write_legacy_full_mix=True)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
def test_legacy_key_none_when_file_missing(tmp_path: Path):
# Manifest points at a full mix that isn't on disk — disabled silently.
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_legacy_full_mix=False
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
def test_legacy_key_none_when_value_blank(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": " "}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
# ── Security / path-traversal branches (legacy key only — a stem `file` is
# resolved through the same /api/sloppak/.../file/ guard as every other stem)
def test_legacy_key_none_when_path_escapes_sloppak(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "../outside.ogg"}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
def test_legacy_key_none_when_path_is_absolute(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "/etc/passwd"}, write_legacy_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.full_mix is None
+119
View File
@@ -0,0 +1,119 @@
"""End-to-end test for the sloppak loader recognising an `original_audio:`
manifest key (the single full-mix file shipped alongside the separate stems)
and surfacing the manifest-relative path on the LoadedSloppak."""
from __future__ import annotations
import json
from pathlib import Path
import yaml
import sloppak as sloppak_mod
def _write_dir_sloppak(root: Path, manifest_extras: dict, *, write_full_mix: bool) -> Path:
"""Build a minimal directory-form sloppak that load_song will accept.
Uses the tmp_path leaf name to make the sloppak filename unique per test,
avoiding the module-level ``resolve_source_dir`` cache being poisoned by a
previous test that happened to share the same "song.sloppak" filename.
"""
pak = root / f"{root.name}.sloppak"
pak.mkdir()
arr_dir = pak / "arrangements"
arr_dir.mkdir()
arr = {
"name": "Lead",
"tuning": [0, 0, 0, 0, 0, 0],
"capo": 0,
"notes": [],
"chords": [],
"anchors": [],
"handshapes": [],
"templates": [],
"beats": [],
"sections": [],
}
(arr_dir / "lead.json").write_text(json.dumps(arr))
manifest = {
"title": "Test",
"artist": "Tester",
"album": "",
"year": 2026,
"duration": 10.0,
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
"stems": [{"id": "guitar", "file": "stems/guitar.ogg", "default": True}],
}
manifest.update(manifest_extras)
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
if write_full_mix:
orig_dir = pak / "original"
orig_dir.mkdir()
# The loader only checks presence (is_file); contents are irrelevant.
(orig_dir / "full.ogg").write_bytes(b"OggS-not-real")
return pak
def _load(pak_path: Path, tmp_path: Path):
dlc_root = pak_path.parent
cache = tmp_path / "cache"
cache.mkdir()
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
# ── Happy path ───────────────────────────────────────────────────────────────
def test_load_song_attaches_original_audio_when_manifest_opts_in(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_full_mix=True
)
loaded = _load(pak, tmp_path)
# Stored as the manifest-relative string so server.py can build the URL the
# same way it builds stem URLs.
assert loaded.original_audio == "original/full.ogg"
# ── Absent / degraded branches ───────────────────────────────────────────────
def test_load_song_original_audio_none_when_manifest_silent(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {}, write_full_mix=True)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
def test_load_song_original_audio_none_when_file_missing(tmp_path: Path):
# Manifest points at a full mix that isn't on disk — disabled silently.
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "original/full.ogg"}, write_full_mix=False
)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
def test_load_song_original_audio_none_when_value_blank(tmp_path: Path):
pak = _write_dir_sloppak(tmp_path, {"original_audio": " "}, write_full_mix=True)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
# ── Security / path-traversal branches ──────────────────────────────────────
def test_load_song_original_audio_none_when_path_escapes_sloppak(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "../outside.ogg"}, write_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
def test_load_song_original_audio_none_when_path_is_absolute(tmp_path: Path):
pak = _write_dir_sloppak(
tmp_path, {"original_audio": "/etc/passwd"}, write_full_mix=True
)
loaded = _load(pak, tmp_path)
assert loaded.original_audio is None
-309
View File
@@ -1,309 +0,0 @@
"""The unpack cache is bounded, and reading part of a song doesn't explode it.
`sloppak_cache/` holds every song ever unpacked, fully decompressed. Stems are
already-compressed audio, so an unpacked song is ~1.1x its zip the cache is a
second copy of the library. It used to have no cap, no LRU, and no cleanup at
all: a tester reached 60 GB from an 1800-song library because one caller looped
the library calling load_song() (rig_builder's library-wide tone batch), which
unpacks the WHOLE pack stems included to read a few KB of tone JSON.
Pins, so neither half can silently come back:
- resolve_source_dir() evicts LRU songs to stay under the cap,
- it never evicts the song the caller just asked for,
- an evicted song is dropped from _source_cache too (otherwise the media route
keeps serving a path that no longer exists and 404s every stem instead of
re-unpacking),
- get_cached_source_dir() self-heals if the cache dir is deleted by hand,
- read_member_bytes() reads one file WITHOUT unpacking anything.
"""
import importlib
import zipfile
import pytest
import yaml
import sloppak as sloppak_mod
STEM = b"\x00" * (400 * 1024) # 400 KB of "audio" — the bulk of a real pack
ARR = b'{"tones": {"definitions": [{"Key": "clean"}]}}'
def _zip_pack(path, stem_bytes=STEM):
with zipfile.ZipFile(path, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({
"title": path.stem,
"arrangements": [{"file": "arrangements/lead.json", "name": "Lead"}],
"stems": [{"id": "full", "file": "stems/audio.ogg"}],
}))
zf.writestr("arrangements/lead.json", ARR)
zf.writestr("stems/audio.ogg", stem_bytes)
return path
@pytest.fixture(autouse=True)
def _fresh_module_state():
# _source_cache is module state and would leak across tests.
importlib.reload(sloppak_mod)
yield
importlib.reload(sloppak_mod)
def _cap_mb(monkeypatch, mb):
monkeypatch.setenv("FEEDBACK_SLOPPAK_CACHE_MAX_MB", str(mb))
def test_read_member_bytes_does_not_unpack(tmp_path, monkeypatch):
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
data = sloppak_mod.read_member_bytes(pack, "arrangements/lead.json")
assert data == ARR
assert list(cache.iterdir()) == [], (
"reading one member must not unpack the pack — this is the whole point: "
"load_song() would have written the 400 KB stem to disk to get 45 bytes of JSON"
)
def test_read_member_bytes_missing_member_is_none(tmp_path):
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
assert sloppak_mod.read_member_bytes(pack, "arrangements/nope.json") is None
assert sloppak_mod.read_member_bytes(pack, "") is None
def test_unpack_cache_evicts_lru_to_stay_under_cap(tmp_path, monkeypatch):
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1) # 1 MB — holds ~2 of our 400 KB packs
for i in range(6):
_zip_pack(dlc / f"song{i}.feedpak")
for i in range(6):
sloppak_mod.resolve_source_dir(f"song{i}.feedpak", dlc, cache)
total = sum(f.stat().st_size for f in cache.rglob("*") if f.is_file())
assert total <= 1 * 1024 * 1024, (
f"unpack cache ran to {total/1e6:.1f} MB against a 1 MB cap — this is the "
"unbounded growth that reached 60 GB in the field"
)
# The most recent song must survive; the oldest must not.
names = {d.name for d in cache.iterdir()}
assert "song5.feedpak" in names, "the song just resolved must never be evicted"
assert "song0.feedpak" not in names, "the least-recently-used song should go first"
def test_eviction_drops_the_source_cache_entry(tmp_path, monkeypatch):
"""An evicted song must not keep being handed out by get_cached_source_dir().
media.py only falls back to resolve_source_dir() when this returns None. If a
stale path survives, every stem 404s for the rest of the process instead of
re-unpacking a silently broken song, not a slow one.
"""
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1)
for i in range(6):
_zip_pack(dlc / f"song{i}.feedpak")
for i in range(6):
sloppak_mod.resolve_source_dir(f"song{i}.feedpak", dlc, cache)
evicted = sloppak_mod.get_cached_source_dir("song0.feedpak")
assert evicted is None, "an evicted song must be dropped from _source_cache"
# ...and asking for it again just re-unpacks it. Self-healing, not broken.
again = sloppak_mod.resolve_source_dir("song0.feedpak", dlc, cache)
assert (again / "stems" / "audio.ogg").is_file()
def test_get_cached_source_dir_self_heals_after_manual_delete(tmp_path, monkeypatch):
"""Telling a user to delete sloppak_cache/ to reclaim disk must be safe."""
import shutil
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 0) # eviction off — isolate the delete
_zip_pack(dlc / "song.feedpak")
src = sloppak_mod.resolve_source_dir("song.feedpak", dlc, cache)
assert sloppak_mod.get_cached_source_dir("song.feedpak") == src
shutil.rmtree(src) # the user clears the folder
assert sloppak_mod.get_cached_source_dir("song.feedpak") is None, (
"a path that no longer exists must not be served — the caller would 404 "
"every stem instead of re-unpacking"
)
assert (sloppak_mod.resolve_source_dir("song.feedpak", dlc, cache)
/ "stems" / "audio.ogg").is_file()
def test_cap_of_zero_disables_eviction(tmp_path, monkeypatch):
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 0)
for i in range(4):
_zip_pack(dlc / f"song{i}.feedpak")
for i in range(4):
sloppak_mod.resolve_source_dir(f"song{i}.feedpak", dlc, cache)
assert len(list(cache.iterdir())) == 4, "cap 0 must mean 'never evict'"
def test_read_member_bytes_normalizes_non_canonical_names(tmp_path):
"""A manifest may name a member './arrangements/lead.json' — valid, and it
resolved fine once unpacked. Reading the zip member by the raw string would
KeyError and silently report no tones. Same trap read_cover_bytes already hit."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
assert sloppak_mod.read_member_bytes(pack, "./arrangements/lead.json") == ARR
assert sloppak_mod.read_member_bytes(pack, "stems/../arrangements/lead.json") == ARR
assert sloppak_mod.read_member_bytes(pack, "arrangements\\lead.json") == ARR
def test_read_member_bytes_rejects_zip_slip(tmp_path):
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = _zip_pack(dlc / "song.feedpak")
assert sloppak_mod.read_member_bytes(pack, "../../etc/passwd") is None
assert sloppak_mod.read_member_bytes(pack, "/etc/passwd") is None
assert sloppak_mod.read_member_bytes(pack, ".") is None
def test_eviction_never_deletes_an_in_flight_unpack(tmp_path, monkeypatch):
"""Two unpacks run concurrently (_UNPACK_MAX_CONCURRENCY = 2). One finishing
must not rmtree the other's half-written dir — that resolver would then cache
an incomplete song and serve a broken pack.
Sized so the sweep genuinely has to reach the in-flight directory: each pack
is ~700 KB against a 1 MB cap, so once `keep` is protected the sweep must
delete EVERY other dir to get under the cap including the one being written.
(A naive version of this test passes even without the guard, because a
freshly-created dir is the most-recently-used and the sweep never gets to it.)
"""
import threading
big = b"\x00" * (700 * 1024)
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1)
for i in range(3):
_zip_pack(dlc / f"song{i}.feedpak", stem_bytes=big)
victim = cache / "song2.feedpak"
started = threading.Event()
release = threading.Event()
real_unpack = sloppak_mod._unpack_zip
def slow_unpack(zip_path, dest):
real_unpack(zip_path, dest) # dir now exists — "half written"
if dest == victim:
started.set()
release.wait(5) # hold it open while the other sweeps
monkeypatch.setattr(sloppak_mod, "_unpack_zip", slow_unpack)
t = threading.Thread(target=sloppak_mod.resolve_source_dir,
args=("song2.feedpak", dlc, cache))
t.start()
assert started.wait(5), "victim unpack did not start"
# song0 lands and sweeps: keep=song0, cache holds song0+song2 = 1.4 MB > 1 MB,
# so the sweep MUST try to delete song2 — which is still being written.
sloppak_mod.resolve_source_dir("song0.feedpak", dlc, cache)
in_flight_survived = victim.is_dir()
release.set()
t.join(5)
assert in_flight_survived, (
"eviction deleted a directory another thread was still unpacking into — "
"that resolver caches an incomplete song and serves a broken pack"
)
def test_read_member_bytes_finds_backslash_members(tmp_path):
"""Windows-authored packs store members as 'arrangements\\lead.json'.
_unpack_zip() normalizes those on extract, so unpack-then-read found them.
An exact getinfo() would not and we'd silently report the song has no tones."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = dlc / "win.feedpak"
with zipfile.ZipFile(pack, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({"title": "w"}))
zf.writestr("arrangements\\lead.json", ARR) # backslash member name
assert sloppak_mod.read_member_bytes(pack, "arrangements/lead.json") == ARR
def test_read_member_bytes_finds_non_canonical_STORED_names(tmp_path):
"""The archive itself may store './arrangements/lead.json'. _unpack_zip()
normalizes stored names on extract, so unpack-then-read resolved it. Both the
requested path and the stored name must be normalized, or the tones vanish."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = dlc / "odd.feedpak"
with zipfile.ZipFile(pack, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({"title": "o"}))
zf.writestr("./arrangements/lead.json", ARR) # stored non-canonically
assert sloppak_mod.read_member_bytes(pack, "arrangements/lead.json") == ARR
def test_read_member_bytes_matches_unpack_last_write_wins(tmp_path):
"""If a pack stores two names that normalize to the same file, _unpack_zip
writes them in order and the LAST one is what ends up on disk. Reading the
raw member by exact name would hand back the first stale arrangement data
that no unpacked read would ever have produced."""
dlc = tmp_path / "dlc"
dlc.mkdir()
pack = dlc / "dupe.feedpak"
with zipfile.ZipFile(pack, "w") as zf:
zf.writestr("manifest.yaml", yaml.safe_dump({"title": "d"}))
zf.writestr("arrangements/lead.json", b'{"tones": {"definitions": [{"Key": "STALE"}]}}')
zf.writestr("./arrangements/lead.json", ARR) # normalizes to the same path
assert sloppak_mod.read_member_bytes(pack, "arrangements/lead.json") == ARR
def test_failed_unpack_does_not_leave_the_dir_un_evictable(tmp_path, monkeypatch):
"""A dir marked in-flight is skipped by eviction. If a failed unpack leaves the
marker behind, that dir becomes permanently un-evictable a slow leak of
exactly the thing this cap exists to prevent."""
dlc = tmp_path / "dlc"
dlc.mkdir()
cache = tmp_path / "cache"
cache.mkdir()
_cap_mb(monkeypatch, 1)
_zip_pack(dlc / "boom.feedpak")
def blow_up(zip_path, dest):
dest.mkdir(parents=True, exist_ok=True)
raise OSError("disk full")
monkeypatch.setattr(sloppak_mod, "_unpack_zip", blow_up)
with pytest.raises(OSError):
sloppak_mod.resolve_source_dir("boom.feedpak", dlc, cache)
assert not sloppak_mod._unpacking, (
"a failed unpack left its destination marked in-flight — eviction will "
"skip it forever"
)
-54
View File
@@ -452,57 +452,3 @@ def test_award_xp_negative_reversal_clamps_at_zero(server):
db.award_xp(50, "minigames")
assert db.award_xp(-50, "minigames") == 0 # exact reversal
assert db.award_xp(-999, "minigames") == 0 # over-reverse clamps at 0
# ── Wall-clock play-time accrual (career hours odometer) ─────────────────────
def test_seconds_accrue_on_scored_and_position_posts(client):
r = client.post("/api/stats", json={"filename": "s.archive", "score": 400,
"accuracy": 0.6, "seconds": 120})
assert r.status_code == 200
assert r.json()["stats"]["seconds_total"] == pytest.approx(120)
# Position-only touch accrues too.
r2 = client.post("/api/stats", json={"filename": "s.archive",
"lastPlayPosition": 12.5, "seconds": 30})
assert r2.json()["stats"]["seconds_total"] == pytest.approx(150)
# A POST without seconds leaves the total alone.
r3 = client.post("/api/stats", json={"filename": "s.archive", "lastPlayPosition": 20.0})
assert r3.json()["stats"]["seconds_total"] == pytest.approx(150)
def test_seconds_only_post_accrues_without_touching_position(client):
client.post("/api/stats", json={"filename": "s.archive", "lastPlayPosition": 42.0})
r = client.post("/api/stats", json={"filename": "s.archive", "seconds": 90})
assert r.status_code == 200
row = r.json()["stats"]
assert row["seconds_total"] == pytest.approx(90)
# No plays counted, resume position untouched (song:ended must not
# overwrite Continue with the end-of-song offset).
assert row["plays"] == 0
assert row["last_position"] == pytest.approx(42.0)
# Recency must come from the seconds-only POST itself — prove it on a
# FRESH row (the position touch above already stamps last_played_at,
# which would make an assertion here vacuous).
r2 = client.post("/api/stats", json={"filename": "fresh.archive", "seconds": 30})
assert r2.json()["stats"]["last_played_at"]
# Still counts as playing today for the streak.
assert r.json()["progress"]["current_streak"] == 1
@pytest.mark.parametrize("bad", [True, "soon", -5, 0, 6 * 3600 + 1])
def test_seconds_validation_rejects_junk(client, bad):
r = client.post("/api/stats", json={"filename": "s.archive",
"lastPlayPosition": 1.0, "seconds": bad})
assert r.status_code == 400
@pytest.mark.parametrize("token", ["NaN", "Infinity"])
def test_seconds_validation_rejects_nonfinite(client, token):
# json= cannot serialize non-finite floats; python's json.loads (and thus
# the server's body parse) accepts the bare tokens, so send raw.
r = client.post(
"/api/stats",
content=f'{{"filename": "s.archive", "lastPlayPosition": 1.0, "seconds": {token}}}',
headers={"Content-Type": "application/json"},
)
assert r.status_code == 400
-203
View File
@@ -1,203 +0,0 @@
"""Regression tests for the feedpak spec-conformance gate.
The gate (tools/check_spec_conformance.py) is what keeps the app from drifting
away from the feedpak spec, so the gate itself must not be weakenable by a
quiet refactor: these tests pin its load-bearing behaviours read/write
classification, the closed allowlist, and the duplicate/malformed-entry
rejections. If one of these fails, the spec's protection regressed.
"""
import importlib.util
import textwrap
from pathlib import Path
import pytest
_SPEC_GATE = Path(__file__).resolve().parent.parent / "tools" / "check_spec_conformance.py"
_spec = importlib.util.spec_from_file_location("check_spec_conformance", _SPEC_GATE)
gate = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(gate)
def _touch(tmp_path, source):
p = tmp_path / "mod.py"
p.write_text(textwrap.dedent(source), encoding="utf-8")
return gate.keys_touched(p)
# ---------------------------------------------------------------- keys_touched
def test_get_is_a_read(tmp_path):
reads, writes = _touch(tmp_path, 'x = manifest.get("title")')
assert reads == {"title"} and writes == set()
def test_subscript_load_is_a_read(tmp_path):
reads, writes = _touch(tmp_path, 'x = manifest["artist"]')
assert reads == {"artist"} and writes == set()
def test_subscript_store_is_a_write_not_a_read(tmp_path):
# The original scan ignored ctx and scored this as a read (lib/songmeta.py
# pattern). A regression here reopens the emitted-key blind spot.
reads, writes = _touch(tmp_path, 'manifest["year"] = 1999')
assert writes == {"year"} and reads == set()
def test_setdefault_is_a_write(tmp_path):
# lib/gp2notation.py stamps feedpak_version this way; a subscript-only scan
# missed it entirely.
reads, writes = _touch(tmp_path, 'manifest.setdefault("feedpak_version", "1.2.0")')
assert writes == {"feedpak_version"} and reads == set()
def test_load_manifest_wrapped_get_is_seen(tmp_path):
# lib/enrichment.py idiom: (load_manifest(p) or {}).get("key")
reads, writes = _touch(
tmp_path, 'rel = (sloppak_mod.load_manifest(p) or {}).get("original_audio")'
)
assert "original_audio" in reads
def test_flow_aware_receiver_any_name(tmp_path):
# lib/routers/chart.py binds `m = load_manifest(p) or {}` — a fixed name
# list missed it and the module's reads went entirely unscanned. Locals
# assigned from load_manifest must be receivers whatever they're called.
reads, writes = _touch(
tmp_path,
"""
pak_info = sloppak_mod.load_manifest(p) or {}
x = pak_info.get("stems")
pak_info["genres"] = ["metal"]
""",
)
assert reads == {"stems"} and writes == {"genres"}
def test_plain_dict_named_m_is_not_a_receiver(tmp_path):
# Flow-awareness must not make every short local a manifest: `m` bound to
# something other than load_manifest stays out of the scan.
reads, writes = _touch(tmp_path, 'm = {}\nx = m.get("title")')
assert reads == set() and writes == set()
def test_unrelated_dicts_are_ignored(tmp_path):
reads, writes = _touch(tmp_path, 'x = config.get("title"); settings["artist"] = 1')
assert reads == set() and writes == set()
def test_non_literal_keys_are_ignored(tmp_path):
reads, writes = _touch(tmp_path, 'x = manifest.get(key_var); manifest[key_var] = 1')
assert reads == set() and writes == set()
def test_manifest_key_read_helper_is_seen(tmp_path):
# lib/routers/song.py uses this helper for gap-fill proposals. If helpers
# are invisible, adding a new literal key through that path bypasses both
# key-coverage and readers-complete.
reads, writes = _touch(
tmp_path,
'_gap_fill_manifest_absent(manifest, "album")\n'
'_gap_fill_manifest_absent(manifest, dynamic_key)\n',
)
assert reads == {"album"} and writes == set()
# ------------------------------------------------------------ exceptions file
def test_duplicate_exception_key_is_rejected():
doc = """
exceptions:
- key: original_audio
issue: https://example.com/1
- key: original_audio
issue: https://example.com/2
"""
with pytest.raises(SystemExit):
gate._parse_exceptions(textwrap.dedent(doc), "test")
@pytest.mark.parametrize("doc", [
"- just\n- a\n- list\n", # list at top level
"exceptions: not-a-list\n", # scalar where list expected
"exceptions:\n - just-a-string\n", # non-mapping entry
"exceptions: [\n", # invalid YAML
])
def test_malformed_exceptions_fail_legibly(doc):
# Malformed shapes must exit with a ::error::, not an AttributeError
# traceback — CI output has to say what to fix.
with pytest.raises(SystemExit):
gate._parse_exceptions(doc, "test")
def test_exception_without_issue_is_rejected():
# No tracking issue, no exception — entries are debt and debt is tracked.
doc = """
exceptions:
- key: original_audio
"""
with pytest.raises(SystemExit):
gate._parse_exceptions(textwrap.dedent(doc), "test")
# --------------------------------------------------------- allowlist is CLOSED
def _yml(tmp_path, name, keys):
p = tmp_path / name
entries = "".join(
f" - key: {k}\n issue: https://example.com/{k}\n" for k in keys
)
p.write_text("exceptions:\n" + (entries or " []\n"), encoding="utf-8")
return p
def test_allowlist_may_not_grow(tmp_path, monkeypatch):
# THE core property: adding an entry must fail, or the FEP process has an
# in-repo bypass and the gate is a speed bump with a signed excuse note.
baseline = _yml(tmp_path, "base.yml", ["original_audio"])
current = _yml(tmp_path, "current.yml", ["original_audio", "sneaky_new_key"])
monkeypatch.setattr(gate, "EXCEPTIONS_FILE", current)
assert gate.check_allowlist_closed(baseline, bootstrap=False) is False
def test_allowlist_may_shrink(tmp_path, monkeypatch):
baseline = _yml(tmp_path, "base.yml", ["original_audio"])
current = _yml(tmp_path, "current.yml", [])
monkeypatch.setattr(gate, "EXCEPTIONS_FILE", current)
assert gate.check_allowlist_closed(baseline, bootstrap=False) is True
def test_allowlist_steady_state_passes(tmp_path, monkeypatch):
baseline = _yml(tmp_path, "base.yml", ["original_audio"])
current = _yml(tmp_path, "current.yml", ["original_audio"])
monkeypatch.setattr(gate, "EXCEPTIONS_FILE", current)
assert gate.check_allowlist_closed(baseline, bootstrap=False) is True
def test_bootstrap_skips_the_diff(tmp_path, monkeypatch):
current = _yml(tmp_path, "current.yml", ["original_audio"])
monkeypatch.setattr(gate, "EXCEPTIONS_FILE", current)
assert gate.check_allowlist_closed(None, bootstrap=True) is True
# ------------------------------------------------------------ readers-complete
def test_readers_list_matches_the_codebase():
# If this fails, a module started touching feedpak manifests without being
# added to READERS — its keys are going unchecked. Same check CI runs.
assert gate.check_readers_complete() is True
def test_no_undeclared_keys_beyond_the_grandfathered(monkeypatch):
# Every key core touches is either spec-declared or grandfathered with a
# tracking issue. New keys go through the FEP process, full stop.
reads, writes = set(), set()
for rel in gate.READERS:
r, w = gate.keys_touched(gate.REPO / rel)
reads |= r
writes |= w
grandfathered = set(gate.load_exceptions())
# Not asserting against the spec here (no spec checkout in unit tests) —
# asserting the *shape*: the only non-spec keys tolerated are grandfathered,
# and today that is exactly {original_audio}.
assert grandfathered == {"original_audio"}
assert "original_audio" in reads
-3
View File
@@ -275,7 +275,4 @@ def test_freqs_to_midis_rejects_garbage():
from tunings import freqs_to_midis
assert freqs_to_midis([82.41, 0]) is None # non-positive
assert freqs_to_midis([82.41, "x"]) is None # non-numeric
assert freqs_to_midis([float("nan")]) is None # non-finite (would raise in int(round(...)))
assert freqs_to_midis([float("inf")]) is None # non-finite
assert freqs_to_midis([float("-inf")]) is None # non-finite
assert freqs_to_midis([]) == [] # vacuously fine
-537
View File
@@ -1,537 +0,0 @@
#!/usr/bin/env python3
"""feedpak spec-conformance gate.
feedpak is an open, versioned format with its own normative spec, JSON Schemas,
and reference validator (https://github.com/got-feedback/feedpak-spec). That
makes the spec a contract with everyone outside this repo: third-party packers,
converters, and players build against it. When core reads a manifest key the
spec never defined, the contract quietly breaks a spec-compliant pack stops
being a fully-working pack, and the format's real definition migrates into our
source tree. See #933 for the instance that motivated this gate.
We cannot mechanically prove core *interprets* a key the way the spec means. We
can prove four surface properties, and those cover the drift that actually
happens:
1. key-coverage every manifest key core reads OR WRITES is declared by the
spec. (Guarded by check_readers_complete(), so the list of
scanned modules cannot quietly fall behind the codebase.)
2. allowlist-closed feedpak-spec-exceptions.yml never grows. It grandfathers
keys that predate this gate; it is not a way to merge a
new one. The only route for a new key is the FEP process.
3. forward core ingests the spec's own example packs.
4. reverse packs committed here satisfy the spec's reference validator.
Dev/CI tooling only: never imported on the serve or Docker path (constitution
Principle I same category as scripts/build-tailwind.sh). `jsonschema` is
therefore a CI-only dependency, not a runtime requirement.
Usage:
python tools/check_spec_conformance.py --spec <path-to-feedpak-spec-checkout>
Exit status is 0 only when every layer passes.
"""
from __future__ import annotations
import argparse
import ast
import json
import re
import subprocess
import sys
import tempfile
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
# Modules that read or write a feedpak manifest dict. Explicit rather than
# globbed, because `manifest` is an overloaded name in this codebase: the
# loose-folder format (lib/loosefolder.py) and the diagnostics bundle
# (lib/diagnostics_bundle.py) both have their own unrelated `manifest`, and
# scanning those would flag *their* keys as feedpak drift.
#
# A hand-maintained list is itself a blind spot, so check_readers_complete()
# below re-derives the set and fails if this list has fallen behind. A missing
# file here is a hard error too, so a rename cannot silently disable the scan.
READERS = [
"lib/sloppak.py",
"lib/enrichment.py",
"lib/songmeta.py",
"lib/gp2notation.py", # rewrites manifest.yaml; stamps feedpak_version
"lib/routers/ws_highway.py", # reads `authors` off a feedpak manifest
"lib/routers/chart.py", # Get-info panel: binds `m = load_manifest(...)`
"lib/routers/song.py", # enrichment gap-fill: reads the manifest directly
]
# Where check_readers_complete() looks for modules READERS may have missed.
READER_SEARCH = ["lib/**/*.py", "server.py"]
# A module is handling a *feedpak* manifest (rather than some other manifest) if
# it shows one of these signals. lib/loosefolder.py and lib/diagnostics_bundle.py
# score zero on all of them, which is what keeps their keys out of the scan.
FEEDPAK_SIGNALS = re.compile(r"import sloppak|from sloppak|load_manifest|manifest\.yaml|feedpak")
# Locals assumed to hold a manifest dict by NAME. This is only the fallback for
# manifests that arrive as function parameters (ws_highway's `manifest` arg);
# locals ASSIGNED from load_manifest(...) are discovered flow-aware in
# keys_touched(), whatever they are called — chart.py's `m` taught us that a
# name list alone silently misses real readers.
MANIFEST_VARS = {"manifest", "mf"}
# Helper functions that take `(manifest, "literal_key", ...)` and read the
# manifest for that key. Keep this narrow: only helpers whose first argument is
# the manifest dict and whose second argument is a top-level manifest key belong
# here.
MANIFEST_KEY_READ_HELPERS = {"_gap_fill_manifest_absent"}
# Packs committed to this repo, checked against the spec's reference validator.
PACK_GLOBS = ["content/starter/*.feedpak", "docs/**/*.sloppak", "docs/**/*.feedpak"]
EXCEPTIONS_FILE = REPO / "feedpak-spec-exceptions.yml"
# How a new manifest key gets into core. There is no in-repo shortcut, by design:
# the spec's own governance says "a change is not part of the format until it
# lands here", and the FEP process is how it lands.
FEP = (
"New manifest keys go through the feedpak Enhancement Proposal process "
"(https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md): land a PR on "
"feedpak-spec that updates the normative spec, the JSON Schemas, an example, and the "
"changelog together — then re-run this PR's checks; the gate verifies against the "
"spec's HEAD, so once your key is in the spec, this PR goes green. It matters beyond "
"this PR: the whole repo is checked against the living spec, so non-conformance that "
"slips in shows up as red CI on every teammate's PR until it's resolved — sorting it "
"out here keeps everyone else unblocked."
)
def _fail(msg: str) -> None:
print(f"::error::{msg}")
def _manifest_locals(tree: ast.AST) -> set[str]:
"""Names of locals assigned from `load_manifest(...)` anywhere in `tree`.
Flow-aware receiver discovery: chart.py binds `m = load_manifest(p) or {}`,
and a fixed name list (`manifest`, `mf`) silently missed it the module's
reads went entirely unscanned. Whatever the local is called, an assignment
whose right-hand side mentions load_manifest marks it as a manifest dict.
"""
names: set[str] = set()
for node in ast.walk(tree):
if not isinstance(node, (ast.Assign, ast.AnnAssign)):
continue
try:
rhs = ast.unparse(node.value) if node.value else ""
except Exception:
continue
if "load_manifest" not in rhs:
continue
targets = node.targets if isinstance(node, ast.Assign) else [node.target]
for t in targets:
if isinstance(t, ast.Name):
names.add(t.id)
return names
def _is_manifest_receiver(node: ast.expr, receivers: set[str]) -> bool:
"""True when `node` evaluates to a manifest dict.
Covers named receivers (fixed names + flow-discovered locals) plus the
inline wrapped form used in lib/enrichment.py:
`(sloppak_mod.load_manifest(p) or {}).get("key")`.
"""
if isinstance(node, ast.Name) and node.id in receivers:
return True
try:
src = ast.unparse(node)
except Exception:
return False
return "load_manifest" in src
def keys_touched(path: Path) -> tuple[set[str], set[str]]:
"""Literal top-level manifest keys `path` reads and writes, separately.
Writes matter as much as reads: `manifest["k"] = v` means core *emits* `k`
into a pack it ships, so an undeclared key there puts non-spec surface into
the wild the same drift, pointed outward. `manifest["k"]` in a subscript
is a read only when its context is a Load; an `ast.walk` that ignores `ctx`
would score `manifest["year"] = ...` (lib/songmeta.py) as a read.
"""
reads: set[str] = set()
writes: set[str] = set()
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
receivers = MANIFEST_VARS | _manifest_locals(tree)
for node in ast.walk(tree):
if (
isinstance(node, ast.Call)
and isinstance(node.func, ast.Attribute)
# `setdefault("k", v)` writes k when absent — lib/gp2notation.py
# stamps feedpak_version that way, and a subscript-only scan misses
# it entirely, letting an emitted key slip past the gate.
and node.func.attr in ("get", "setdefault")
and _is_manifest_receiver(node.func.value, receivers)
and node.args
and isinstance(node.args[0], ast.Constant)
and isinstance(node.args[0].value, str)
):
bucket = writes if node.func.attr == "setdefault" else reads
bucket.add(node.args[0].value)
elif (
isinstance(node, ast.Call)
and isinstance(node.func, ast.Name)
and node.func.id in MANIFEST_KEY_READ_HELPERS
and len(node.args) >= 2
and _is_manifest_receiver(node.args[0], receivers)
and isinstance(node.args[1], ast.Constant)
and isinstance(node.args[1].value, str)
):
reads.add(node.args[1].value)
elif (
isinstance(node, ast.Subscript)
and _is_manifest_receiver(node.value, receivers)
and isinstance(node.slice, ast.Constant)
and isinstance(node.slice.value, str)
):
target = writes if isinstance(node.ctx, ast.Store) else reads
target.add(node.slice.value)
return reads, writes
def _parse_exceptions(text: str, origin: str) -> dict[str, str]:
"""Parse an exceptions document into {key: tracking issue}."""
import yaml # runtime dep (PyYAML is already in requirements.txt)
try:
data = yaml.safe_load(text) or {}
except yaml.YAMLError as e:
_fail(f"{origin}: not valid YAML — {e}")
sys.exit(1)
# A malformed shape (list/string at top level, non-mapping entry) must fail
# with a CI-legible error, not an AttributeError traceback.
if not isinstance(data, dict):
_fail(f"{origin}: top level must be a mapping with an 'exceptions' list, got {type(data).__name__}")
sys.exit(1)
entries = data.get("exceptions") or []
if not isinstance(entries, list):
_fail(f"{origin}: 'exceptions' must be a list, got {type(entries).__name__}")
sys.exit(1)
out: dict[str, str] = {}
for entry in entries:
if not isinstance(entry, dict):
_fail(f"{origin}: each exception must be a mapping with 'key' and 'issue', got {type(entry).__name__}")
sys.exit(1)
key, issue = entry.get("key"), entry.get("issue")
if not key or not issue:
_fail(f"{origin}: every exception needs both 'key' and 'issue'")
sys.exit(1)
# A duplicate would silently take the last issue link, quietly retargeting
# the debt this file exists to track. Fail instead.
if key in out:
_fail(
f"{origin}: '{key}' is listed more than once. "
f"Keep one entry per key so the tracking issue is unambiguous."
)
sys.exit(1)
out[key] = issue
return out
def load_exceptions() -> dict[str, str]:
"""Map of grandfathered key -> tracking issue URL, as of this working tree."""
if not EXCEPTIONS_FILE.exists():
return {}
return _parse_exceptions(
EXCEPTIONS_FILE.read_text(encoding="utf-8"), EXCEPTIONS_FILE.name
)
def check_allowlist_closed(baseline: Path | None, bootstrap: bool) -> bool:
"""The allowlist is CLOSED: it may shrink, never grow.
`feedpak-spec-exceptions.yml` grandfathers keys that predate this gate. It is
not a way to merge a new one. Without this check the gate would be a speed
bump with a signed excuse note anyone could append an entry and route
around the FEP process from inside this repo, which is exactly the drift that
produced #933.
So: removing an entry is fine (that's the debt being paid down); adding one
fails the build, and the error points at the FEP process instead.
"""
if bootstrap:
print(" allowlist-closed: bootstrapping (no baseline on the base branch) — skipped")
return True
if baseline is None:
print(" allowlist-closed: no baseline supplied (local run) — skipped")
return True
if not baseline.is_file():
_fail(
f"--baseline-exceptions {baseline} does not exist. CI derives this from the base "
f"branch; for a local run, omit the flag to skip the allowlist diff."
)
return False
base_keys = set(
_parse_exceptions(baseline.read_text(encoding="utf-8"), f"{EXCEPTIONS_FILE.name} (base)")
)
now_keys = set(load_exceptions())
added = sorted(now_keys - base_keys)
removed = sorted(base_keys - now_keys)
for key in added:
_fail(
f"{EXCEPTIONS_FILE.name}: this PR adds an exception for '{key}', and the allowlist "
f"can't take new entries — it only grandfathers keys that predate the gate. {FEP}"
)
if removed:
print(f" allowlist shrank (debt paid down): {', '.join(removed)}")
print(f" allowlist-closed: {'FAILED' if added else 'OK'}")
return not added
def check_readers_complete() -> bool:
"""READERS must not fall behind the codebase.
The key-coverage scan is only as good as the list of modules it scans, and a
hand-maintained list rots: `lib/routers/ws_highway.py` and
`lib/gp2notation.py` both touched feedpak manifests for a while without being
on it. So re-derive the set any module that both touches manifest keys and
shows a feedpak signal must be listed and fail if one is missing.
This is a guard on the gate itself, not on the format.
"""
listed = set(READERS)
missing: list[str] = []
for pattern in READER_SEARCH:
for path in sorted(REPO.glob(pattern)):
rel = path.relative_to(REPO).as_posix()
if rel in listed:
continue
src = path.read_text(encoding="utf-8", errors="replace")
if not FEEDPAK_SIGNALS.search(src):
continue
# Same scanner the coverage check uses — a separate "does it touch
# keys" regex diverged from it once already (`m = load_manifest(...)`
# in chart.py matched neither `manifest` nor `mf`, so the module
# went unlisted AND unscanned). One detector, one truth.
try:
reads, writes = keys_touched(path)
except SyntaxError:
continue
if reads or writes:
missing.append(rel)
for rel in missing:
_fail(
f"{rel} touches feedpak manifest keys but is not in READERS "
f"({Path(__file__).name}) — its keys are going unchecked. Add it."
)
print(f" scanning {len(listed)} modules; readers-complete: {'FAILED' if missing else 'OK'}")
return not missing
def check_key_coverage(spec: Path) -> bool:
"""Layer 1 — core must not read or write a manifest key the spec does not declare."""
schema = json.loads((spec / "schemas" / "manifest.schema.json").read_text(encoding="utf-8"))
declared = set(schema.get("properties") or {})
if not declared:
_fail("spec manifest.schema.json declares no properties — wrong path or bad checkout?")
return False
reads: set[str] = set()
writes: set[str] = set()
for rel in READERS:
path = REPO / rel
if not path.exists():
_fail(f"reader {rel} not found — was it renamed? Update READERS in {Path(__file__).name}.")
return False
try:
r, w = keys_touched(path)
except SyntaxError as e:
# A module that doesn't parse can't be scanned — but it also can't
# pass pytest, so this is belt-and-braces for a CI-legible message
# rather than a traceback if this job runs first.
_fail(f"could not scan {rel}: {e}")
return False
reads |= r
writes |= w
exceptions = load_exceptions()
ok = True
def _undeclared(keys: set[str]) -> list[str]:
return sorted((keys - declared) - set(exceptions))
for key in _undeclared(reads):
_fail(f"core reads manifest key '{key}', which the feedpak spec does not define. {FEP}")
ok = False
for key in _undeclared(writes):
_fail(
f"core writes manifest key '{key}', which the feedpak spec does not define — that "
f"puts non-spec surface into every pack we emit. {FEP}"
)
ok = False
# A stale exception is its own bug: it means the spec caught up and nobody
# cleaned up, so the allowlist slowly becomes a place drift hides.
touched = reads | writes
for key, issue in exceptions.items():
if key in declared:
_fail(
f"'{key}' is listed in {EXCEPTIONS_FILE.name} but the spec now declares it. "
f"Remove the exception and close {issue}."
)
ok = False
elif key not in touched:
_fail(
f"'{key}' is listed in {EXCEPTIONS_FILE.name} but core no longer reads or writes "
f"it. Remove the exception."
)
ok = False
print(f" spec declares {len(declared)} keys; core reads {len(reads)}, writes {len(writes)}")
if exceptions:
print(f" grandfathered (tracked debt): {', '.join(sorted(exceptions))}")
print(f" key-coverage: {'OK' if ok else 'FAILED'}")
return ok
def check_forward(spec: Path) -> bool:
"""Layer 3 — core must ingest every example pack the spec ships."""
examples_dir = spec / "examples"
if not examples_dir.is_dir():
_fail(f"{examples_dir} is missing — wrong path or bad checkout?")
return False
# rglob, not iterdir: the contract is "every example pack the spec ships", so
# a pack nested under examples/<group>/ must not slip through.
#
# Deliberately NOT filtered by is_file(): a feedpak is dual-form — a zip
# (`foo.feedpak`) *or* a directory (`foo.feedpak/`) — and the spec's own
# examples ship as directories today. An is_file() guard here would silently
# match zero packs. Matching on the suffix covers both forms, and rglob does
# not smuggle in a pack's innards because files inside a pack don't carry a
# pack suffix.
examples = sorted(
p for p in examples_dir.rglob("*")
if p.suffix in (".feedpak", ".sloppak")
)
if not examples:
_fail("spec ships no example packs — wrong path or bad checkout?")
return False
sys.path.insert(0, str(REPO / "lib"))
try:
import sloppak # noqa: E402 (path must be set first — flat imports, no package)
except Exception as e:
_fail(
f"could not import core's sloppak loader ({type(e).__name__}: {e}). "
f"Are requirements.txt deps installed?"
)
return False
ok = True
with tempfile.TemporaryDirectory() as tmp:
cache = Path(tmp)
for pack in examples:
try:
loaded = sloppak.load_song(pack.name, pack.parent, cache)
except Exception as e:
_fail(
f"core failed to load the spec's own example pack {pack.name}: "
f"{type(e).__name__}: {e}. A spec-valid pack must load."
)
ok = False
continue
if not loaded.song.arrangements:
_fail(f"core loaded {pack.name} but found no arrangements")
ok = False
continue
print(f" loaded {pack.name}: {len(loaded.song.arrangements)} arrangement(s)")
print(f" forward: {'OK' if ok else 'FAILED'}")
return ok
def check_reverse(spec: Path) -> bool:
"""Layer 4 — packs committed here must pass the spec's reference validator."""
packs = sorted({p for g in PACK_GLOBS for p in REPO.glob(g)})
if not packs:
print(" reverse: no committed packs — skipped")
return True
try:
proc = subprocess.run(
[sys.executable, str(spec / "tools" / "validate.py"), *[str(p) for p in packs]],
capture_output=True,
text=True,
# The validator takes seconds for all committed packs; a pathological
# pack or validator bug must fail the job, not hang the runner until
# the Actions-level timeout.
timeout=300,
)
except subprocess.TimeoutExpired:
_fail("the spec's reference validator did not finish within 300s — pathological pack or validator bug?")
print(" reverse: FAILED")
return False
sys.stdout.write("".join(f" {ln}\n" for ln in proc.stdout.splitlines() if ln.strip()))
if proc.returncode != 0:
_fail(
"a pack committed to this repo does not satisfy the feedpak spec "
"(see the reference validator output above)."
)
if proc.stderr.strip():
sys.stderr.write(proc.stderr)
print(f" reverse: {'OK' if proc.returncode == 0 else 'FAILED'}")
return proc.returncode == 0
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument(
"--spec",
required=True,
type=Path,
help="path to a feedpak-spec checkout (CI checks out the spec repo's HEAD)",
)
ap.add_argument(
"--baseline-exceptions",
type=Path,
help="the exceptions file as it exists on the base branch. Supplied by CI so the "
"allowlist can be proven to have not grown. Omit for a local run.",
)
ap.add_argument(
"--bootstrap-allowlist",
action="store_true",
help="the base branch has no exceptions file yet (this PR introduces the gate), so "
"there is nothing to diff against. CI passes this only in that case.",
)
args = ap.parse_args()
spec = args.spec.resolve()
if not (spec / "schemas" / "manifest.schema.json").exists():
_fail(f"{spec} does not look like a feedpak-spec checkout")
return 1
print("[1/4] key-coverage — core reads/writes only keys the spec declares")
# Both run, always: a stale READERS list and an undeclared key are separate
# failures, and reporting only the first would hide the second. Hence two
# calls and an explicit `and` over the results, not a short-circuiting one.
readers_ok = check_readers_complete()
coverage_ok = check_key_coverage(spec)
ok1 = readers_ok and coverage_ok
print("[2/4] allowlist-closed — the grandfather list may shrink, never grow")
ok2 = check_allowlist_closed(args.baseline_exceptions, args.bootstrap_allowlist)
print("[3/4] forward — core ingests the spec's example packs")
ok3 = check_forward(spec)
print("[4/4] reverse — committed packs satisfy the reference validator")
ok4 = check_reverse(spec)
if ok1 and ok2 and ok3 and ok4:
print("\nfeedpak spec conformance: OK")
return 0
print("\nfeedpak spec conformance: FAILED")
return 1
if __name__ == "__main__":
sys.exit(main())
-445
View File
@@ -1,445 +0,0 @@
#!/usr/bin/env python3
"""Migrate packs off the deprecated `original_audio:` key — the full mix is a stem.
Before feedpak 1.15.0 reserved the stem id `full`, spec §5.3 said the mixdown was
"commonly replaced" by the per-instrument stems when a pack was split so after
separation it had nowhere to live. This repo worked around that by inventing a
top-level `original_audio:` manifest key pointing at a parallel `original/`
directory (#583). The key was never in the spec, and #933 removed core's
dependence on it: the mixdown is a stem, and its id is `full`.
This rewrites a pack into the shape the spec now defines:
original/full.ogg -> stems/full.ogg (entry moved)
original_audio: original/full.ogg -> stems: [{id: full, file: stems/full.ogg,
default: 'off'}, ...]
`default: 'off'` is what makes the retained mixdown safe: a reader that honours
`default` (normative since feedpak 1.0.0) will not play it on open, so it never
doubles the mix even in a reader that predates the reserved id.
Nothing else in the pack is touched every other key, file and stem is preserved
verbatim, and `feedpak_version` is stamped to the version the result conforms to.
The rewrite is atomic per pack: a new archive is built beside the original and
renamed over it only on success, so an interrupted run leaves every pack either
fully migrated or untouched never truncated.
Idempotent: a pack that already carries a `full` stem and no `original_audio:` is
reported as `skip` and left alone, so a partial run can simply be re-run.
Usage:
python tools/migrate_full_mix_stem.py --dry-run /path/to/packs # report only
python tools/migrate_full_mix_stem.py /path/to/packs # migrate
python tools/migrate_full_mix_stem.py --verify /path/to/packs # check results
Exit status is 0 only when every pack ended up in the migrated shape (or was
already there).
"""
from __future__ import annotations
import argparse
import os
import shutil
import sys
import tempfile
import threading
import time
import zipfile
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
import yaml
# The version this migration brings a pack up to: the one that reserved `full`.
TARGET_FEEDPAK_VERSION = "1.15.0"
FULL_MIX_STEM_ID = "full"
LEGACY_KEY = "original_audio"
# Where the mixdown lands. §2.1's conventional layout — readers resolve through
# the manifest and never care about the path, but a pack that says `stems/` and
# means it is the one a human can read.
CANONICAL_FULL_MIX_PATH = "stems/full.ogg"
PACK_EXTS = (".feedpak", ".sloppak")
class Skip(Exception):
"""Pack needs no migration."""
def is_safe_relpath(rel: str) -> bool:
"""True when `rel` is a manifest path the spec allows (§2.2 rule 2).
POSIX-style relative: forward slashes, no leading `/`, no `..` segments, no
empty segments, no colon (which excludes drive letters and NTFS alternate
data streams), no backslashes.
This is a TRUST BOUNDARY, not a tidiness check. Core's loader refuses a
full-mix path that escapes the pack and reports the pack as having no full
mix the audio is inert. A migration that moved such an entry into
`stems/full.ogg` would take content the reader deliberately rejected and
hand it back as a valid, playable stem. So a pack like this is reported, not
migrated.
"""
if not rel or rel.startswith("/") or "\\" in rel or ":" in rel:
return False
parts = rel.split("/")
return all(p and p != ".." for p in parts)
def plan_manifest(manifest: dict) -> tuple[dict, str]:
"""Return (new_manifest, relpath_of_audio_to_move); "" = no file needs moving.
Raises Skip when the pack needs no migration. Pure no I/O so the part
with the decisions in it is testable without building archives.
"""
raw_stems = manifest.get("stems")
stems: list = raw_stems if isinstance(raw_stems, list) else []
has_full_stem = any(
isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID for s in stems
)
legacy_rel = manifest.get(LEGACY_KEY)
legacy_rel = legacy_rel.strip() if isinstance(legacy_rel, str) else ""
if not legacy_rel:
# Nothing invented to undo: either the pack already keeps its mixdown as
# the `full` stem, or it never carried one.
raise Skip("already migrated" if has_full_stem else "no original_audio key")
if has_full_stem:
# The mixdown is already a stem and the dead key merely lingers beside it.
# Drop the key; move nothing. But do NOT trust its `default`: a mixdown
# left enabled beside instrument stems is the double-audio hazard this
# migration exists to remove, and a reader that honours `default` would
# play the whole song on top of the stems on open. Force it off — unless
# `full` is the only stem, in which case it IS the audio.
others = [
s
for s in stems
if isinstance(s, dict) and str(s.get("id", "")) != FULL_MIX_STEM_ID
]
new_stems = []
for s in stems:
if isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID and others:
s = {**s, "default": "off"}
new_stems.append(s)
to_move = ""
else:
# `default` decides whether a reader plays this on open, and that is the
# whole safety margin: alongside per-instrument stems the mixdown must be
# OFF (a reader that sums the list would otherwise double the song), but
# when it is the pack's only stem it IS the audio and must be ON.
entry = {
"id": FULL_MIX_STEM_ID,
"file": CANONICAL_FULL_MIX_PATH,
"default": "off" if stems else "on",
}
# First in the list, matching the spec's §5.3 example.
new_stems = [entry, *stems]
# If the key already pointed at the canonical path, only the manifest is wrong.
to_move = "" if legacy_rel == CANONICAL_FULL_MIX_PATH else legacy_rel
out: dict = {}
for k, v in manifest.items():
if k == LEGACY_KEY:
continue # the invented key disappears
out[k] = new_stems if k == "stems" else v
out.setdefault("stems", new_stems) # a pack that had no stems list gets one
out["feedpak_version"] = TARGET_FEEDPAK_VERSION
return out, to_move
def migrate_zip(path: Path, dry_run: bool) -> str:
"""Rewrite one zipped pack in place. Returns a one-word status."""
with zipfile.ZipFile(path) as zf:
try:
raw = zf.read("manifest.yaml")
except KeyError:
return "no-manifest"
manifest = yaml.safe_load(raw) or {}
try:
new_manifest, old_rel = plan_manifest(manifest)
except Skip:
return "skip"
names = set(zf.namelist())
if old_rel:
if not is_safe_relpath(old_rel):
# Core refuses this path and plays no full mix for the pack. Do
# not launder it into a valid stem — see is_safe_relpath().
return "unsafe-path"
if old_rel not in names:
# The key points at audio that isn't in the archive. Core already
# treats that as "no full mix"; migrating would fabricate a stem
# entry for a file that does not exist and break every reader.
return "missing-audio"
if CANONICAL_FULL_MIX_PATH in names:
return "target-occupied"
else:
# Manifest-only rewrite (a stale key beside a mixdown that is already
# a stem, or a key that already pointed at the canonical path). Check
# the file the resulting `full` stem will actually NAME — not the
# canonical path, which an already-migrated pack is free not to use:
# §2.2 says readers resolve through the manifest, so a valid pack may
# keep its mixdown anywhere.
full_file = next(
(
s.get("file")
for s in new_manifest.get("stems", [])
if isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID
),
None,
)
if full_file not in names:
return "missing-audio"
if dry_run:
return "would-migrate"
# Build the replacement beside the original, on the same filesystem, so
# the final rename is atomic and an interrupted run can't truncate a pack.
tmp_fd, tmp_name = tempfile.mkstemp(
dir=str(path.parent), prefix=f".{path.name}.", suffix=".tmp"
)
os.close(tmp_fd)
tmp_path = Path(tmp_name)
try:
with zipfile.ZipFile(tmp_path, "w", zipfile.ZIP_DEFLATED) as out:
for item in zf.infolist():
if item.filename == "manifest.yaml":
out.writestr(
item,
yaml.safe_dump(
new_manifest, sort_keys=False, allow_unicode=True
),
)
continue
data = zf.read(item.filename)
if old_rel and item.filename == old_rel:
# Same bytes, same compression, new name: the mixdown moves
# from original/ into stems/ where the format says audio goes.
moved = zipfile.ZipInfo(
CANONICAL_FULL_MIX_PATH, date_time=item.date_time
)
moved.compress_type = item.compress_type
moved.external_attr = item.external_attr
out.writestr(moved, data)
continue
out.writestr(item, data)
except BaseException:
tmp_path.unlink(missing_ok=True)
raise
shutil.copystat(path, tmp_path)
os.replace(tmp_path, path) # atomic
return "migrated"
def verify_zip(path: Path) -> str:
"""Confirm a pack is in the migrated shape and its mixdown is really there."""
with zipfile.ZipFile(path) as zf:
try:
manifest = yaml.safe_load(zf.read("manifest.yaml")) or {}
except KeyError:
return "no-manifest"
if LEGACY_KEY in manifest:
return "still-has-key"
stems = manifest.get("stems") or []
full = next(
(
s
for s in stems
if isinstance(s, dict) and str(s.get("id", "")) == FULL_MIX_STEM_ID
),
None,
)
if full is None:
return "no-full-stem"
if full.get("file") not in set(zf.namelist()):
return "full-stem-missing-file"
# A retained mixdown that plays on open would double the mix in any reader
# that sums the stem list — the whole hazard this migration must not create.
# Beside instrument stems, `full` MUST carry an explicit, normalized "off":
# - core (lib/sloppak.py) defaults an ABSENT `default` to True (ON) and
# treats an empty / unrecognized string as ON, so a missing or blank
# default is not merely non-canonical — core would play the mixdown on
# open, doubling the song. It is the exact hazard, not a lesser one.
# - the migrator always writes the literal "off", so requiring it also
# certifies the pack is in the shape this tool produces — the most
# portable spelling, understood even by a reader that only knows
# "on"/"off" and would choke on a boolean or `false`/`0`/`no`.
# So: `on`-ish values are reported as actively-playing; everything that is
# not a normalized "off" (missing, empty, boolean, `false`/`no`/`0`,
# malformed) is reported as an unsafe/non-canonical default.
if len(stems) > 1:
default = str(full.get("default", "")).strip().lower()
if default in ("true", "on", "yes", "1"):
return "full-stem-default-on"
if default != "off":
return "full-stem-default-not-off"
return "ok"
def migrate_pack(path: Path, dry_run: bool) -> str:
"""Dispatch by pack form. ZIP-file packs are rewritten in place; directory
(authoring) packs are REPORTED, not rewritten.
A single-file pack is replaced atomically a fully-built temp archive
swapped in with one os.replace(), so an interrupted run leaves it either
fully migrated or untouched. A directory can't be swapped that way (no
atomic replace of a populated directory), so an in-place rewrite could leave
an authoring pack half-migrated. Rather than risk that, directory packs are
surfaced as `dir-form-unsupported` (a problem status, so the run's exit code
and summary flag them) for the operator to re-pack or migrate as a `.feedpak`.
"""
if path.is_dir():
return "dir-form-unsupported"
return migrate_zip(path, dry_run)
def verify_pack(path: Path) -> str:
"""Verify a pack; directory (authoring) packs are reported, see migrate_pack."""
if path.is_dir():
return "dir-form-unsupported"
return verify_zip(path)
def iter_packs(root: Path):
"""Yield every pack under `root`. A pack is a suffix-named ZIP FILE or a
suffix-named DIRECTORY (the authoring form) both are discovered so a
directory-form pack is never silently walked past. A directory pack is
yielded whole, not descended into: its `stems/` and `arrangements/` are pack
contents, not packs. (migrate/verify then report directory packs rather than
rewriting them in place see migrate_pack.)"""
if root.is_file():
yield root
return
# A directory whose OWN name is a pack suffix is a single directory-form
# pack passed directly, not a tree of packs to search.
if root.name.endswith(PACK_EXTS):
yield root
return
for dirpath, dirnames, filenames in os.walk(root):
for fn in sorted(filenames):
if fn.endswith(PACK_EXTS):
yield Path(dirpath) / fn
for dn in sorted(dn for dn in dirnames if dn.endswith(PACK_EXTS)):
yield Path(dirpath) / dn
# Don't descend INTO a pack directory — its contents aren't packs.
dirnames[:] = [dn for dn in dirnames if not dn.endswith(PACK_EXTS)]
def main(argv: list[str] | None = None) -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("root", type=Path, help="pack, or directory of packs")
ap.add_argument("--dry-run", action="store_true", help="report, change nothing")
ap.add_argument("--verify", action="store_true", help="check the migrated shape")
ap.add_argument("--jobs", type=int, default=8, help="parallel packs (default 8)")
args = ap.parse_args(argv)
if not args.root.exists():
print(f"error: {args.root} does not exist", file=sys.stderr)
return 2
packs = list(iter_packs(args.root))
if not packs:
print(f"no packs found under {args.root}", file=sys.stderr)
return 2
action = verify_pack if args.verify else (lambda p: migrate_pack(p, args.dry_run))
def work(pack: Path) -> str:
"""Never raise. One unreadable pack must not kill a 50,000-pack run.
A library this size has damage in it a truncated download, an archive
left half-written by an interrupted converter. Letting that propagate
aborts the whole job partway through and throws away the summary, which
is exactly when you most need to know what happened. Report it as a
problem status instead: the pack is untouched, the run continues, and the
final report names it.
"""
try:
return action(pack)
except zipfile.BadZipFile:
return "corrupt-zip"
except OSError as e:
return f"io-error ({e.__class__.__name__})"
except Exception as e: # malformed YAML, unexpected manifest shape, …
return f"error ({e.__class__.__name__})"
counts: dict[str, int] = {}
problems: list[tuple[str, Path]] = []
# A real run rewrites every archive under `root` — tens of thousands of packs
# and hundreds of gigabytes. Printing only a final summary means hours of
# silence, in which a stall and steady progress look identical. Emit a
# heartbeat instead: rate and ETA come from the packs actually finished, so
# it stays honest when the disk slows down. stderr, so `> report.txt` keeps
# the summary clean.
total = len(packs)
started = time.monotonic()
# The heartbeat runs on its OWN CLOCK, in its own thread.
#
# Two weaker designs were tried and both go quiet exactly when you need them
# to speak. Ticking every N packs ties the cadence to how slow a pack is: 500
# packs is a blink in a --dry-run and many minutes in a real migration, so the
# run that most needs watching says the least. Ticking on time but only when a
# pack *finishes* is no better: if every worker is grinding on a huge archive,
# nothing completes, so nothing prints — and a stall becomes indistinguishable
# from progress, which is the one thing a progress meter must never allow.
#
# A daemon thread on a fixed interval reports regardless. If the count stops
# advancing between beats, you are looking at a stall, and you can see it.
HEARTBEAT_SECONDS = 10.0
done = 0 # only the main loop writes it; the beat thread only reads
stop_beat = threading.Event()
def heartbeat() -> None:
while not stop_beat.wait(HEARTBEAT_SECONDS):
elapsed = time.monotonic() - started
rate = done / elapsed if elapsed > 0 else 0.0
eta = (total - done) / rate if rate > 0 else 0.0
print(
f" {done}/{total} ({100 * done / total:.1f}%) "
f"{rate:.1f} packs/s eta {eta / 60:.0f}m "
f"[{len(problems)} problem(s)]",
file=sys.stderr,
flush=True,
)
print(
f"{total} pack(s) under {args.root}"
f"{'verifying' if args.verify else 'dry run' if args.dry_run else 'migrating'} "
f"with {max(1, args.jobs)} job(s)",
file=sys.stderr,
flush=True,
)
beat = threading.Thread(target=heartbeat, daemon=True)
beat.start()
# as_completed, not pool.map: map yields in SUBMISSION order, so the counter
# would stall behind one slow pack while later ones were already done — a
# progress meter that lies about progress. Count each pack as it finishes.
try:
with ThreadPoolExecutor(max_workers=max(1, args.jobs)) as pool:
futures = {pool.submit(work, p): p for p in packs}
for fut in as_completed(futures):
pack = futures[fut]
status = fut.result()
counts[status] = counts.get(status, 0) + 1
if status not in ("migrated", "skip", "would-migrate", "ok"):
problems.append((status, pack))
done += 1
finally:
stop_beat.set()
beat.join(timeout=1)
print(f"\n{len(packs)} pack(s) under {args.root}")
for status, n in sorted(counts.items(), key=lambda kv: -kv[1]):
print(f" {n:>7} {status}")
if problems:
print(f"\n{len(problems)} pack(s) need a look:", file=sys.stderr)
for status, pack in problems[:20]:
print(f" {status:<22} {pack}", file=sys.stderr)
if len(problems) > 20:
print(f" … and {len(problems) - 20} more", file=sys.stderr)
return 1 if problems else 0
if __name__ == "__main__":
raise SystemExit(main())