Stream the rig library to the client

The core rig reader landed its reading half but not its exposing half:
the bindings shipped (`base_rig`, per-change `rig`) while the library
they are references INTO stayed server-side on LoadedSloppak.rigs with
no consumer at all. A client could see that a change binds
"grand-piano" and had no way to learn what that was.

Add a `rigs` WS message carrying the library whole and verbatim, and put
each drum part's resolved `tones` binding on its `song_info.drum_parts`
entry — the drum half had the same gap, since the loader resolved the
binding in P3 and the wire payload was still `{id, name}`.

Sent whenever the pack ships a library, deliberately NOT gated on this
arrangement having tone changes: a pack can bind sound to its drum parts
alone via `drum_tones` and still need the library.

Verbatim matters here. Core selects no realization and applies no
`intent.gm` floor, so the consumer that voices the part needs the whole
block to make that choice — including the unknown `role`/`engine`/`kind`
values and `ext` namespaces §7.9 requires a Reader to preserve.

Client side: `hwState.rigs` / `toneBaseRig`, `getRigs()` /
`getToneBaseRig()`, and `bundle.rigs` / `bundle.toneBaseRig` so
renderers reach them the same way they reach every other chart array,
with both reset paths clearing them between songs.

Packs that bind no rig are byte-identical on the wire — no `rigs`
message, no `base_rig`, no `rig`, no `tones` on a drum part.

Signed-off-by: gionnibgud <gionnibgud@gmail.com>
This commit is contained in:
gionnibgud
2026-07-23 12:29:21 +02:00
parent eef58c88c3
commit b9435891eb
5 changed files with 286 additions and 5 deletions
+11
View File
@@ -8,6 +8,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **The rig library reaches the client.** Completes the core reader for source
rigs: a new `rigs` WebSocket message carries the pack's `rigs.json` library
verbatim, and each entry in `song_info.drum_parts` now carries its `tones`
binding. Previously the *references* shipped (`base_rig`, per-change `rig`)
with nothing to resolve them against — the library loaded server-side and had
no consumer. The library is sent whenever a pack ships one rather than being
gated on the arrangement having tone changes, since a pack can bind sound to
its drum parts alone. `highway.js` exposes `getRigs()` / `getToneBaseRig()`
and adds `bundle.rigs` / `bundle.toneBaseRig` for renderers. Core still
selects no realization and applies no `intent.gm` floor — that belongs to
whatever voices the part. Packs that bind no rig send byte-identical frames.
- **Core reader for source rigs (feedpak 1.18.0).** A pack can declare what a
MIDI part should sound like by binding a rig; core now reads that binding and
hands it to the client instead of dropping it. Three parts: the
+2 -1
View File
@@ -684,13 +684,14 @@ The highway WebSocket at `/ws/highway/{filename}?arrangement={index}` streams th
| Message | Shape | Description |
|---------|-------|-------------|
| `loading` | `{ type: 'loading', stage }` | Status/progress message during extraction or conversion |
| `song_info` | `{ type, title, artist, arrangement, arrangement_index, arrangements, duration, tuning, capo, centOffset, format, audio_url, audio_error, stems }` | Song metadata. `arrangements` is the full list for the switcher. `audio_url` is `null` when audio is unavailable, in which case `audio_error` is non-null; otherwise `audio_error` is `null`. `stems` is always present — an empty array for non-sloppak songs or sloppak songs with no split stems. `tuning` is an array (6 for guitar, 4 for bass). `centOffset` is a float (cents) from the RS2014 `<centOffset>` field — commonly `-1200.0` for extended-range bass (one octave down), small non-zero values for true-tuned content (e.g. A443 ≈ +11.8 cents), `0.0` when absent. Available via `getSongInfo().centOffset`. |
| `song_info` | `{ type, title, artist, arrangement, arrangement_index, arrangements, duration, tuning, capo, centOffset, format, audio_url, audio_error, stems }` | Song metadata. `arrangements` is the full list for the switcher. `audio_url` is `null` when audio is unavailable, in which case `audio_error` is non-null; otherwise `audio_error` is `null`. `stems` is always present — an empty array for non-sloppak songs or sloppak songs with no split stems. `tuning` is an array (6 for guitar, 4 for bass). `centOffset` is a float (cents) from the RS2014 `<centOffset>` field — commonly `-1200.0` for extended-range bass (one octave down), small non-zero values for true-tuned content (e.g. A443 ≈ +11.8 cents), `0.0` when absent. Available via `getSongInfo().centOffset`. `drum_parts` (present for sloppaks, `[]` otherwise) lists the song's drum parts primary-first as `{id, name}`, plus **`tones`** when that part binds a sound (feedpak 1.18.0 §5.1/§5.2 — the entry's own `tones` or the song-level `drum_tones` fallback, already resolved server-side); the key is omitted when the part binds nothing. Resolve its `base_rig`/`rig` ids against the `rigs` message below. |
| `beats` | `{ type, data: [{ time, measure }] }` | Beat timestamps with measure numbers |
| `sections` | `{ type, data: [{ time, name }] }` | Named sections (Intro, Verse, Chorus, etc.) |
| `anchors` | `{ type, data: [{ time, fret, width }] }` | Fret zoom anchors |
| `chord_templates` | `{ type, data: [{ name, frets: [6] }] }` | Named chord shapes |
| `lyrics` | `{ type, data: [{ w, t, d }], source }` | Syllables: `w`=word, `t`=time, `d`=duration. `-` joins to previous, `+` = line break. `source` is one of `"xml"`, `"whisperx"`, `"user"` — UI can use it to render an "auto-transcribed" badge for `whisperx`. Sloppaks always include `source` (legacy sloppaks without a `lyrics_source` manifest key default to `"xml"` at load time). Loose folders set it based on which extractor matched. Absent only when no lyrics fired the message at all |
| `tone_changes` | `{ type: 'tone_changes', base, base_rig?, data: [{ t, name, rig? }] }` | Optional — tone change events relative to the arrangement base tone; only sent if tones were found. Note the time key is **`t`**, not `time` (both the sloppak path and the legacy XML path emit `t`). `base_rig` and each entry's `rig` are the pack's **rig bindings** — ids into [`rigs.json`](https://github.com/got-feedback/feedpak-spec/blob/main/spec/feedpak-v1.md#79-rigsjson) (feedpak §6.9/§7.9), carried through verbatim and **not** resolved by core: selecting a realization and applying the `intent.gm` floor belong to whatever voices the part. Both are **omitted entirely** when the chart binds no rig, so consumers predating the rig model see the payload they always did. |
| `rigs` | `{ type: 'rigs', version, data: [ <rig>, ... ] }` | Optional — the pack's **rig library** (`rigs.json`, feedpak §7.9): the engine-agnostic signal chains that `tone_changes`' `base_rig`/`rig` and each drum part's `tones` reference by `id`. Sent whenever the pack ships one, **not** gated on this arrangement having tone changes (a pack can bind sound to its drum parts alone). Rigs are **verbatim** — core selects no realization and applies no `intent.gm` floor, so whatever voices the part reads the whole block, including unknown `role`/`engine`/`kind` values and `ext` namespaces. |
| `notes` | `{ type, data: [{ t, s, f, sus, ho, po, sl, bn, ... }] }` | Single notes |
| `chords` | `{ type, data: [{ t, notes: [{ s, f, sus, ... }] }] }` | Chord events |
| `phrases` | `{ type, data: [{ start_time, end_time, max_difficulty, levels: [{ difficulty, notes, chords, anchors, handshapes }] }], total }` | Optional — per-phrase difficulty ladder for master-difficulty slider (feedBack#48). Only sent when the source chart carries multi-level phrase data (phrase-aware sloppak). Sent in chunks (`data` is a batch, `total` is the full count across messages) to avoid multi-MB single frames. Absent for GP imports and legacy sloppak; consumers must treat missing message as "single fixed difficulty — slider disabled". |
+23 -1
View File
@@ -581,8 +581,14 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
# as the `drum_tab`/`drum_hits` messages below. Always a list
# (empty when the pack has no drums, and a single entry for a
# legacy one-drum pack), so a part picker can bind unconditionally.
# `tones` is the part's sound binding (feedpak 1.18.0 §5.1/§5.2 —
# entry `tones` or the song-level `drum_tones` fallback, already
# resolved by the loader). Omitted when the part binds nothing, so
# a pack without bindings sends the payload it always did; pair it
# with the `rigs` message below to resolve `base_rig`/`rig` ids.
"drum_parts": [
{"id": p["id"], "name": p["name"]}
{"id": p["id"], "name": p["name"],
**({"tones": p["tones"]} if p.get("tones") else {})}
for p in (loaded_slop.drum_parts or [])
] if is_slop and loaded_slop is not None else [],
"has_notation": bool(
@@ -797,6 +803,22 @@ async def highway_ws(websocket: WebSocket, filename: str, arrangement: int = -1,
if base_rig:
payload["base_rig"] = base_rig
await websocket.send_json(payload)
# The pack's RIG LIBRARY (rigs.json, spec §7.9) — the thing the
# `base_rig` / `rig` ids above (and the drum parts' `tones`) are
# references INTO. Sent whole and verbatim: core does not select a
# realization or apply the `intent.gm` floor, so the consumer that
# voices the part needs the entire block to make that choice.
#
# Deliberately NOT gated on this arrangement having tone changes —
# a pack can bind sound to its drum parts alone (`drum_tones`) and
# still need the library. Sent whenever the pack ships one.
if loaded_slop is not None and loaded_slop.rigs is not None:
await websocket.send_json({
"type": "rigs",
"version": int(loaded_slop.rigs.get("version", 1)),
"data": loaded_slop.rigs.get("rigs") or [],
})
else:
xml_paths = sorted(_xml_walk("*.xml"))
+21 -3
View File
@@ -239,6 +239,14 @@ function createHighway() {
hwState.lyricsSource = "";
hwState.toneChanges = [];
hwState.toneBase = "";
// Rig id bound to the base tone ("" when the chart binds none).
hwState.toneBaseRig = "";
// The pack's rig library (rigs.json, feedpak-spec §7.9) — the engine-
// agnostic signal chains that `toneBase`/`toneChanges[].rig` and each drum
// part's `tones` reference by `id`. Empty array when the pack ships none.
// Verbatim from the server: core resolves no realization and applies no
// `intent.gm` floor, so whatever voices a part reads the whole block.
hwState.rigs = [];
// Drum-tab payload (sloppak-spec §5.3). When the active sloppak's
// manifest carries a `drum_tab:` key, the server streams a `drum_tab`
// metadata message followed by chunked `drum_hits`. `drumTab.hits` is
@@ -574,6 +582,10 @@ function createHighway() {
b.lyricsSource = hwState.lyricsSource;
b.toneChanges = hwState.toneChanges;
b.toneBase = hwState.toneBase;
b.toneBaseRig = hwState.toneBaseRig;
// Live reference, like the other chart arrays — swaps identity when
// the song changes, so a field-identity cache stays valid.
b.rigs = hwState.rigs;
// Drum tab payload (or null when the active arrangement has
// no drum_tab). Live reference — renderers MUST treat as
// read-only. Plugins should prefer this over decoding the
@@ -1686,7 +1698,7 @@ function createHighway() {
hwState._resizeHandler = () => this.resize();
window.addEventListener('resize', hwState._resizeHandler);
hwState.ready = false;
hwState.notes = []; hwState.chords = []; hwState.handShapes = []; hwState.beats = []; hwState.sections = []; hwState.anchors = []; hwState.chordTemplates = []; hwState.lyrics = []; hwState.lyricsSource = ""; hwState.toneChanges = []; hwState.toneBase = ""; hwState.drumTab = null;
hwState.notes = []; hwState.chords = []; hwState.handShapes = []; hwState.beats = []; hwState.sections = []; hwState.anchors = []; hwState.chordTemplates = []; hwState.lyrics = []; hwState.lyricsSource = ""; hwState.toneChanges = []; hwState.toneBase = ""; hwState.toneBaseRig = ""; hwState.rigs = []; hwState.drumTab = null;
hwState.stringCount = 6; // default until song_info arrives
// Reset phrase ladder + filter (feedBack#48). _mastery
// persists across arrangement switches — the slider's
@@ -2391,7 +2403,8 @@ function createHighway() {
// having to hook the raw WS themselves.
hwState.lyricsSource = msg.source || "";
break;
case 'tone_changes': hwState.toneChanges = msg.data; hwState.toneBase = msg.base || ""; break;
case 'tone_changes': hwState.toneChanges = msg.data; hwState.toneBase = msg.base || ""; hwState.toneBaseRig = msg.base_rig || ""; break;
case 'rigs': hwState.rigs = msg.data || []; break;
case 'notes': hwState.notes = hwState.notes.concat(msg.data); break;
case 'chords': hwState.chords = hwState.chords.concat(msg.data); break;
case 'handshapes': hwState.handShapes = hwState.handShapes.concat(msg.data); break;
@@ -2664,6 +2677,11 @@ function createHighway() {
getChordTemplates() { return hwState._xfChordTemplates !== null ? hwState._xfChordTemplates : hwState.chordTemplates; },
getToneChanges() { return hwState.toneChanges; },
getToneBase() { return hwState.toneBase; },
// Rig id bound to the base tone ("" when the chart binds none).
getToneBaseRig() { return hwState.toneBaseRig; },
// The pack's rig library, verbatim (feedpak-spec §7.9). Index by `id`
// to resolve a `base_rig` / `rig` / drum-part `tones` reference.
getRigs() { return hwState.rigs; },
getSections() { return hwState.sections; },
// Timed lyric syllables for the active song: [{t: start, d: length,
// w: word}], same array the highway WS populates. Exposed so overlay
@@ -2817,7 +2835,7 @@ function createHighway() {
// Close old WS but keep audio + animation running
if (hwState.ws) { hwState.ws.close(); hwState.ws = null; }
hwState.ready = false;
hwState.notes = []; hwState.chords = []; hwState.handShapes = []; hwState.beats = []; hwState.sections = []; hwState.anchors = []; hwState.chordTemplates = []; hwState.lyrics = []; hwState.lyricsSource = ""; hwState.toneChanges = []; hwState.toneBase = ""; hwState.drumTab = null;
hwState.notes = []; hwState.chords = []; hwState.handShapes = []; hwState.beats = []; hwState.sections = []; hwState.anchors = []; hwState.chordTemplates = []; hwState.lyrics = []; hwState.lyricsSource = ""; hwState.toneChanges = []; hwState.toneBase = ""; hwState.toneBaseRig = ""; hwState.rigs = []; hwState.drumTab = null;
hwState.stringCount = 6; // default until song_info arrives
// Drop any per-song offset from the previous load so setTime
// calls that fire before the next song_info arrives don't
+229
View File
@@ -0,0 +1,229 @@
"""Integration test for the rig library on the highway WebSocket.
The `tone_changes` frame carries rig *references* (`base_rig`, per-change
`rig`) and each drum part carries its `tones` binding — all of them ids that
mean nothing without the library they point into. This asserts the library
actually reaches the client, verbatim, and that packs binding no rig send the
payload they always did.
Core does not resolve any of it: selecting a realization and applying the
`intent.gm` floor belong to whatever voices the part, so the whole block ships
untouched (spec §7.9).
"""
from __future__ import annotations
import importlib
import json
import sys
import pytest
import yaml
from fastapi.testclient import TestClient
RIGS = {
"version": 2,
"rigs": [
{
"id": "grand-piano",
"name": "Grand Piano",
"instrument": "keys",
"blocks": [
{
"role": "source",
"intent": {"kind": "instrument", "gm": {"program": 0}},
"realizations": [
{"engine": "soundfont", "format": "sf2",
"ref": "sounds/grand.sf2", "bank": 0, "program": 0},
],
# Unknown namespace — §7.9 says a Reader MUST preserve it.
"ext": {"vendor.custom": {"anything": [1, 2, 3]}},
},
],
},
{"id": "std-kit", "name": "Standard Kit", "instrument": "drums",
"blocks": [{"role": "source",
"intent": {"kind": "instrument",
"gm": {"percussion": True, "kit": 0}}}]},
],
}
TONES = {
"base": "Grand",
"base_rig": "grand-piano",
"changes": [{"t": 12.5, "name": "Rhodes", "rig": "grand-piano"}],
}
DRUM_TAB = {
"version": 1,
"name": "Drums",
"kit": [{"id": "kick", "name": "Kick"}],
"hits": [{"t": 1.0, "p": "kick", "v": 100}],
}
def _write_sloppak(dlc_dir, *, rigs=None, tones=None, drum_tones=None):
pak = dlc_dir / "rigtest.sloppak"
pak.mkdir(parents=True)
arr_dir = pak / "arrangements"
arr_dir.mkdir()
arr = {
"name": "Lead", "tuning": [0, 0, 0, 0, 0, 0], "capo": 0,
"notes": [{"t": 1.0, "s": 0, "f": 3, "sus": 0}],
"chords": [], "anchors": [], "handshapes": [], "templates": [],
"beats": [{"time": 0.0, "measure": 1}], "sections": [],
}
if tones is not None:
arr["tones"] = tones
(arr_dir / "lead.json").write_text(json.dumps(arr))
manifest = {
"title": "Rig Test", "artist": "Tester", "album": "", "year": 2026,
"duration": 30.0,
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
"stems": [{"id": "full", "file": "stems/full.ogg", "default": True}],
"drum_tab": "drum_tab.json",
}
if rigs is not None:
manifest["rigs"] = "rigs.json"
(pak / "rigs.json").write_text(json.dumps(rigs))
if drum_tones is not None:
manifest["drum_tones"] = drum_tones
(pak / "drum_tab.json").write_text(json.dumps(DRUM_TAB))
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
return pak
@pytest.fixture()
def make_client(tmp_path, monkeypatch):
"""Factory: build the DLC dir first, then import a fresh server bound to it."""
def _make():
monkeypatch.setenv("CONFIG_DIR", str(tmp_path / "config"))
monkeypatch.setenv("DLC_DIR", str(tmp_path / "dlc"))
monkeypatch.setenv("FEEDBACK_SYNC_STARTUP", "1")
sys.modules.pop("server", None)
server = importlib.import_module("server")
monkeypatch.setattr(server, "load_plugins", lambda *a, **kw: None)
monkeypatch.setattr(server, "startup_scan", lambda: None)
monkeypatch.setattr(server, "SLOPPAK_CACHE_DIR", tmp_path / "cache")
import appstate as _appstate
monkeypatch.setattr(_appstate, "sloppak_cache_dir", tmp_path / "cache")
return server
(tmp_path / "dlc").mkdir()
yield _make
server = sys.modules.get("server")
conn = getattr(getattr(server, "meta_db", None), "conn", None)
if conn is not None:
getattr(sys.modules.get("server"), "_join_background_db_threads", lambda: None)()
conn.close()
def _frames(client, limit=300):
"""Every typed frame through `ready` (drains fully so teardown is clean)."""
out = []
with client.websocket_connect("/ws/highway/rigtest.sloppak?arrangement=0") as ws:
for _ in range(limit):
msg = ws.receive_json()
if msg.get("error"):
raise AssertionError(f"WS error frame: {msg}")
if msg.get("type") == "loading":
continue
out.append(msg)
if msg.get("type") == "ready":
break
return out
def _one(frames, kind):
hits = [f for f in frames if f.get("type") == kind]
return hits[0] if hits else None
# ── The library reaches the client ───────────────────────────────────────────
def test_rigs_library_streams_verbatim(make_client):
server = make_client()
_write_sloppak(server._get_dlc_dir(), rigs=RIGS, tones=TONES)
with TestClient(server.app) as client:
frames = _frames(client)
rigs = _one(frames, "rigs")
assert rigs is not None, [f["type"] for f in frames]
assert rigs["version"] == 2
# Verbatim — including the `ext` namespace and the soundfont realization a
# Reader that can't render it must still preserve.
assert rigs["data"] == RIGS["rigs"]
def test_tone_changes_reference_the_library(make_client):
"""The bindings and the library they point into both arrive, and the ids
line up — that pairing is the whole point of the reader."""
server = make_client()
_write_sloppak(server._get_dlc_dir(), rigs=RIGS, tones=TONES)
with TestClient(server.app) as client:
frames = _frames(client)
tc = _one(frames, "tone_changes")
assert tc["base_rig"] == "grand-piano"
assert tc["data"] == [{"t": 12.5, "name": "Rhodes", "rig": "grand-piano"}]
known = {r["id"] for r in _one(frames, "rigs")["data"]}
assert tc["base_rig"] in known
assert all(c["rig"] in known for c in tc["data"])
def test_rigs_stream_even_without_arrangement_tones(make_client):
"""A pack can bind sound to its drums alone, so the library must NOT be
gated on this arrangement having tone changes."""
server = make_client()
_write_sloppak(server._get_dlc_dir(), rigs=RIGS, tones=None,
drum_tones={"base": "Kit", "base_rig": "std-kit"})
with TestClient(server.app) as client:
frames = _frames(client)
assert _one(frames, "tone_changes") is None # no arrangement tones
assert _one(frames, "rigs") is not None # library still sent
def test_drum_part_carries_its_tones_binding(make_client):
server = make_client()
_write_sloppak(server._get_dlc_dir(), rigs=RIGS,
drum_tones={"base": "Kit", "base_rig": "std-kit"})
with TestClient(server.app) as client:
frames = _frames(client)
parts = _one(frames, "song_info")["drum_parts"]
assert len(parts) == 1
assert parts[0]["tones"] == {"base": "Kit", "base_rig": "std-kit"}
assert parts[0]["tones"]["base_rig"] in {r["id"] for r in _one(frames, "rigs")["data"]}
# ── Packs that bind nothing are unchanged ────────────────────────────────────
def test_no_rigs_message_when_pack_ships_no_library(make_client):
server = make_client()
_write_sloppak(server._get_dlc_dir(), rigs=None, tones=None)
with TestClient(server.app) as client:
frames = _frames(client)
assert _one(frames, "rigs") is None
def test_unbound_pack_keeps_the_legacy_payload_shape(make_client):
"""A chart with tones but no rig bindings must send the exact frame it did
before the rig model existed — no `base_rig`, no `rig`, no `tones` on the
drum part."""
server = make_client()
_write_sloppak(server._get_dlc_dir(), rigs=None,
tones={"base": "Clean", "changes": [{"t": 5.0, "name": "Lead"}]})
with TestClient(server.app) as client:
frames = _frames(client)
tc = _one(frames, "tone_changes")
assert "base_rig" not in tc
assert tc["data"] == [{"t": 5.0, "name": "Lead"}]
assert all("rig" not in c for c in tc["data"])
assert all("tones" not in p for p in _one(frames, "song_info")["drum_parts"])