feat(career): extract the whole setlist before the gig starts (no more waiting between songs) (#971)
ship-ci / ci (push) Waiting to run

* feat(career): extract the whole setlist before the gig starts

A feedpak is a zip, and the first play of one pays for its extraction into
sloppak_cache. Inside a set that cost landed BETWEEN songs: the player finished
a number and then sat there waiting for the next one to unpack, mid-gig.

A setlist is a known list up front, so unpack it all while the poster is still on
screen. New POST /gigs/prepare walks the set through resolve_source_dir; the
poster's Play button shows "Preparing set…" while it runs.

Best-effort by design, at every level:
  - a corrupt pak in the set does not sink the prepare (it is reported in
    `failed`; the play itself surfaces the error exactly as it does outside a
    gig — slow beats blocked)
  - a host without the library resolvers degrades to a no-op rather than 500
  - a failed request just falls through to the old lazy extraction

Ordering matters and is pinned: the set is unpacked BEFORE the stage is borrowed
(venue/viz overwritten) and before the queue starts, so a proposal cancelled
while unpacking leaves nothing half-applied to unwind.

Tests unpack REAL zips rather than mocking the extractor: every song of the set
lands on disk before the first note, a re-prepare does not duplicate the unpack,
one bad pak still leaves the good one prepared, and no-library / empty-setlist
degrade cleanly. 18/18.

NB the other half of the gig report — the per-song results popup interrupting
the set (and worse, claimAutoExit'ing so the queue would not advance until it was
dismissed) — is fixed in the note_detect plugin repo, which is not part of this
checkout.

* fix(career): bound the prepare request; validate the setlist (PR #971 review)

Both CodeRabbit findings were right.

1. A HUNG PREPARE COULD BLOCK THE GIG FOREVER.

   `await fetch(...)` only rejects on a network ERROR. A server that accepts the
   connection and then never answers hangs indefinitely — and the gig would never
   start. That makes this optimisation the exact thing the PR promises it can
   never be: the reason you cannot play.

   The request is now bounded by an AbortController (PREPARE_TIMEOUT_MS, generous
   because unpacking a setlist is real work — but a CEILING, not a wait). Past it
   we start the gig and let the first play extract lazily, as it always did. The
   Play button is restored in a `finally`, so a timeout cannot strand the poster
   on "Preparing set…" with Play disabled — which would have been the same bug
   wearing a different hat.

2. THE `songs` BODY WAS UNVALIDATED.

   A str is iterable: "abc" would have prepared three one-character "songs". And
   the endpoint unpacks zips, so an arbitrary caller could ask for unbounded work.
   Now list-only, string entries, blanks dropped, capped at MAX_GIG_SONGS.

Tests: the fetch is abortable and the button is re-enabled on EVERY path
including the abort; non-list bodies, non-string/blank entries, and an
oversized setlist. 50 career tests, JS 5/5, eslint clean.

* fix(career): path-traversal guard on prepare; a cap test that actually tests the cap

CodeRabbit again, and the first one is a real hole I put there.

1. PATH TRAVERSAL. sloppak.resolve_source_dir() does a bare `dlc_root / filename`
   with NO containment guard — so `../../x` walks straight out of the library, and
   my new endpoint handed it attacker-supplied filenames. Every filename now goes
   through _resolve_dlc_path first, the same check every other filename-bound
   handler applies. Pinned: `..`, backslash traversal, an absolute POSIX path and
   a Windows drive path are all refused, and nothing outside the library is
   unpacked.

2. THE CAP TEST WAS VACUOUS. It asserted `prepared == 0` against a fixture with no
   library — where the endpoint exits before extraction — so it passed whether or
   not MAX_GIG_SONGS existed. It now runs against a real library and asserts the
   endpoint CONSIDERED at most MAX_GIG_SONGS of the 82 it was handed. Verified to
   fail when the cap is removed.

   Same class of mistake as the notedetect gigBlock: a test that passes for the
   wrong reason. Worth saying out loud since it is twice in one day.

3. E702 — semicolon-joined statements in the new tests, split.

51 career tests; full suite green.
This commit is contained in:
Byron Gamatos
2026-07-15 00:36:20 +02:00
committed by GitHub
parent 917d81c2d2
commit 1702afa379
4 changed files with 380 additions and 2 deletions
+61
View File
@@ -46,6 +46,8 @@ from pathlib import Path
from fastapi import Body, HTTPException
from fastapi.responses import FileResponse
import sloppak
from dlc_paths import _resolve_dlc_path
from progression import instrument_for_arrangement
PLUGIN_ID = "career"
@@ -53,6 +55,9 @@ VENUE_ID_RE = re.compile(r"^[a-z0-9_-]{1,40}$")
PACK_FILENAME_RE = re.compile(r"^[a-z0-9_-]{1,64}\.(mp4|webm|mp3|json)$")
REQUIRED_LOOPS = ("bored", "neutral", "engaged", "ecstatic")
DOWNLOAD_CHUNK = 1024 * 256
# A setlist is a handful of songs; this endpoint unpacks zips, so cap the work an
# arbitrary caller can ask for.
MAX_GIG_SONGS = 32
_lock = threading.Lock()
_state = {
@@ -734,6 +739,62 @@ def setup(app, context):
"snapshot": snapshot})
return {"ok": True}
@app.post(f"/api/plugins/{PLUGIN_ID}/gigs/prepare")
def prepare_gig(body: dict = Body(...)):
"""Unpack every song of the set BEFORE the gig starts.
A feedpak is a zip: the first play of one pays for its extraction into
sloppak_cache. Inside a set that cost landed BETWEEN songs — the player
finished a number and then sat waiting for the next one to unpack, mid-
gig. A set is a known list up front, so extract it all while the player
is still looking at the poster.
Idempotent and cheap on a warm cache: resolve_source_dir() returns the
already-unpacked dir without rewriting it. Best-effort per song — one
bad feedpak must not block the set from starting (the play itself will
surface the error, exactly as it does outside a gig).
"""
raw = (body or {}).get("songs")
# A str is iterable: without the list check, "abc" would prepare three
# one-character "songs". Cap the count too — this endpoint unpacks zips,
# so an oversized list is real work, and a setlist is a handful of songs.
if not isinstance(raw, list):
return {"ok": True, "prepared": 0, "failed": []}
files = [f for f in raw if isinstance(f, str) and f.strip()][:MAX_GIG_SONGS]
if not files:
return {"ok": True, "prepared": 0, "failed": []}
# .get, not []: a host that doesn't hand us the resolvers (or has no
# library configured) must degrade to "extract lazily, as before" — this
# is an optimisation, and it is never allowed to be the thing that stops
# a gig from starting.
get_dlc = context.get("get_dlc_dir")
get_cache = context.get("get_sloppak_cache_dir")
dlc_root = get_dlc() if callable(get_dlc) else None
cache_root = get_cache() if callable(get_cache) else None
if dlc_root is None or cache_root is None:
return {"ok": False, "prepared": 0, "failed": files, "error": "no library"}
root = Path(dlc_root)
prepared, failed = 0, []
for fn in files:
# CONTAINMENT FIRST. resolve_source_dir() does a bare
# `dlc_root / filename` with no guard, so a crafted `../..` would
# walk straight out of the library. Every other filename-bound
# handler validates through _resolve_dlc_path; so does this one.
safe = _resolve_dlc_path(root, fn)
if safe is None:
_state["log"].warning("career: gig pre-extract rejected unsafe path %r", fn)
failed.append(fn)
continue
try:
sloppak.resolve_source_dir(fn, root, Path(cache_root))
prepared += 1
except Exception as exc: # noqa: BLE001 — one bad pak can't sink the set
_state["log"].warning("career: gig pre-extract failed for %s: %s", fn, exc)
failed.append(fn)
return {"ok": True, "prepared": prepared, "failed": failed}
@app.post(f"/api/plugins/{PLUGIN_ID}/gigs/propose")
def propose_gig(body: dict = Body(...)):
inst = str((body or {}).get("instrument") or "")
+48 -2
View File
@@ -12,6 +12,10 @@
'use strict';
const API = '/api/plugins/career';
// Unpacking a setlist is real work (zips, possibly on a slow/network drive),
// so this is generous — but it is a CEILING, not a wait. Past it we start the
// gig and let the first play extract lazily, as it always did.
const PREPARE_TIMEOUT_MS = 60000;
const VENUE_OVERRIDE_KEY = 'feedBack-career-venue';
const NO_VENUE = '__none__';
const PREV_VIZ_KEY = 'feedBack-career-prev-viz';
@@ -1123,10 +1127,52 @@
sfx('page');
}
function startGig() {
// Unpack the whole set before the first note.
//
// A feedpak is a zip, and the first play of one pays for its extraction. In
// a set that cost landed BETWEEN songs: the player finished a number and
// then sat there waiting for the next one to unpack, mid-gig. The setlist is
// known up front, so warm it all while the poster is still on screen.
//
// Best-effort by design: a library that won't pre-extract must not stop the
// gig from starting — the play itself surfaces the error the same way it
// does outside a gig. Slow is better than blocked.
async function prepareGigSongs(prop, btn) {
const label = btn && btn.textContent;
if (btn) { btn.disabled = true; btn.textContent = 'Preparing set…'; }
// A bare `await fetch(...)` only rejects on a network ERROR — a server
// that accepts the connection and then never answers hangs forever, and
// the gig would never start. That would make this optimisation the very
// thing it promises never to be: the reason you cannot play. Give up
// waiting and let the first play extract lazily, exactly as before.
const ctrl = new AbortController();
const timer = setTimeout(() => ctrl.abort(), PREPARE_TIMEOUT_MS);
try {
await fetch(`${API}/gigs/prepare`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ songs: prop.songs.map((s) => s.filename) }),
signal: ctrl.signal,
});
} catch (_) {
// abort, offline, non-2xx — all the same: start the gig anyway.
} finally {
clearTimeout(timer);
if (btn) { btn.disabled = false; if (label) btn.textContent = label; }
}
}
async function startGig(btn) {
const prop = _ppGigProposal;
const q = window.feedBack && window.feedBack.playQueue;
if (!prop || !q || typeof q.start !== 'function' || typeof window.playSong !== 'function') return;
// Extract the setlist BEFORE the stage is borrowed and the queue starts,
// so a failure here leaves nothing half-applied to unwind.
await prepareGigSongs(prop, btn);
// The poster's Play could have been cancelled while we were unpacking.
if (_ppGigProposal !== prop) return;
// The gig BORROWS the stage: stash whatever venue/viz the user had so
// the set ending gives it back (unlike "Play here", which is an
// explicit persistent choice on the venue card).
@@ -1424,7 +1470,7 @@
}
const gigBtn = e.target.closest('[data-pp-gig]');
if (gigBtn) { bookGig(gigBtn.dataset.ppGig); return; }
if (e.target.closest('[data-pp-gig-play]')) { startGig(); return; }
if (e.target.closest('[data-pp-gig-play]')) { startGig(e.target.closest('[data-pp-gig-play]')); return; }
if (e.target.closest('[data-pp-gig-reroll]')) {
if (_ppGigProposal) bookGig(_ppGigProposal.genre_key);
return;