mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-09-11 16:14:30 +00:00
Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a39e03559d | ||
|
|
c7fb074111 | ||
|
|
fb06e288e1 | ||
|
|
313348a1ff | ||
|
|
21b8a6cd49 |
@@ -35,6 +35,8 @@ plugins/minigames/__pycache__/
|
||||
!plugins/tuner/
|
||||
!plugins/tuner/**
|
||||
plugins/tuner/__pycache__/
|
||||
!plugins/input_setup/
|
||||
!plugins/input_setup/**
|
||||
node_modules/
|
||||
test-results/
|
||||
playwright-report/
|
||||
|
||||
@@ -100,7 +100,9 @@ do not collide in `sys.modules`.
|
||||
The whole point of Slopsmith is that a user points it at an existing
|
||||
song library folder and it Just Works. The library is scanned and
|
||||
indexed in `meta.db` (SQLite via `MetadataDB`). The open Sloppak
|
||||
format (`lib/sloppak.py`, `docs/sloppak-spec.md`) is the preferred
|
||||
format (`lib/sloppak.py`; specified at
|
||||
[got-feedback/feedback-feedpak-spec](https://github.com/got-feedback/feedback-feedpak-spec),
|
||||
published as feedpak — same format) is the preferred
|
||||
format and the home for new features; loose-folder XML charts
|
||||
(`lib/loosefolder.py`) are also discovered and played as a first-class
|
||||
format. Both must keep playing across releases.
|
||||
|
||||
@@ -199,11 +199,6 @@ usually an unrelated plugin's per-frame DOM work.
|
||||
|
||||
Slopsmith supports two ways for a plugin to participate in the main player's visuals. They coexist; the setRenderer contract is the default for any viz that draws a highway-shaped surface, and overlays handle layered decorations on top.
|
||||
|
||||
**Deeper references** (in [`docs/`](docs/)):
|
||||
- [`realtime-scoring-pipeline.md`](docs/realtime-scoring-pipeline.md) — end-to-end trace from audio sample → lit gem, across native engine, IPC, `note_detect`, highway, renderer.
|
||||
- [`note-state-provider.md`](docs/note-state-provider.md) — full contract for `setNoteStateProvider` / `bundle.getNoteState` (key formats, sustain semantics, alpha ownership).
|
||||
- [`visualization-feedback-guide.md`](docs/visualization-feedback-guide.md) — practical walkthrough with a minimal 2D Canvas example + troubleshooting decision tree.
|
||||
|
||||
**Pick the right shape:**
|
||||
- Replacing the whole highway drawing on the existing highway canvas (your renderer owns its rendering context / resources; `createHighway()` still owns the canvas element and the rAF loop)? → **setRenderer** (section 1). Enters the viz picker. Works in both the main player and per-panel under splitscreen.
|
||||
- Adding a layer on top of whichever viz is active? → **Overlay** (section 2). Navbar toggle, not in the picker.
|
||||
@@ -367,8 +362,6 @@ A previous standalone-pane contract (`window.createMyVisualization({ container }
|
||||
|
||||
#### 3. Note-state provider — for scorers that want renderers to "light up" notes (slopsmith#254)
|
||||
|
||||
> **Full reference**: [`docs/note-state-provider.md`](docs/note-state-provider.md) covers the contract in depth (key formats, sustain `'active'` vs `'hit'` semantics, alpha-ownership, common pitfalls). The summary below is the API at a glance.
|
||||
|
||||
A scoring plugin (note_detect) can publish a per-note judgment so whichever renderer is active draws the **gem itself** lit on a correct hit, and keeps a sustain trail glowing while it's still being played correctly — instead of a separate overlay ring floating near the note.
|
||||
|
||||
```js
|
||||
@@ -539,7 +532,13 @@ lyrics.json Syllable-level lyrics (optional)
|
||||
|
||||
Sloppak is the preferred format for new features. The [Stems plugin](https://github.com/topkoa/slopsmith-plugin-stems) provides live stem mixing for sloppak songs.
|
||||
|
||||
**Full developer reference:** [docs/sloppak-spec.md](docs/sloppak-spec.md) — manifest schema, arrangement wire format, and how to extend the format with new data types (drum tab, key/scale annotations, etc.).
|
||||
**Full developer reference:** the authoritative format spec now lives in its own repo —
|
||||
[got-feedback/feedback-feedpak-spec](https://github.com/got-feedback/feedback-feedpak-spec)
|
||||
([`spec/feedpak-v1.md`](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md)):
|
||||
manifest schema, arrangement wire format, and how to extend the format with new data types (drum
|
||||
tab, key/scale annotations, etc.). Published as **feedpak**; this codebase still uses the legacy
|
||||
**sloppak** name internally — same on-disk format. [docs/sloppak-spec.md](docs/sloppak-spec.md) is
|
||||
a local pointer + code map.
|
||||
|
||||
**Key code:**
|
||||
- `lib/sloppak.py` — format detection, zip/directory resolution, metadata extraction, song loading
|
||||
|
||||
@@ -153,6 +153,18 @@ The legacy chart-coupled surface — `highway.setNoteStateProvider(fn)`, the sin
|
||||
|
||||
Diagnostics live under `slopsmith.note_detection_capability.v1` and contain provider ids/labels/kinds, binding summaries (requester, provider, redacted context, target size), availability, and the last bounded outcome (event, binding, provider, MIDI number, hit flag) — never raw audio buffers, sample data, device labels, or song identity.
|
||||
|
||||
## MIDI-Input Domain
|
||||
|
||||
The MIDI-input slice (spec 012, issues #873/#880) promotes `midi-input` as a **core-owned** provider-coordinator implemented by [static/capabilities/midi-input.js](../static/capabilities/midi-input.js) — the MIDI analog of `audio-input`. It is deliberately separate from `audio-input` (whose source/`open` contract is audio-frame-centric: channel shapes, sample buffers) because MIDI carries discrete messages, not audio; and it is **not** owned by any feature plugin, so the device-access boundary outlives the input-setup wizard (exactly as `audio-input` is `core.audio.session`-owned). Consumers — the `input_setup` onboarding wizard, the `piano`/keys and `drums` plugins, and (as a follow-up, #881) note-detection's Web-MIDI provider — converge here on ONE device-access boundary: one permission prompt, one source list, one redaction boundary, retiring private per-plugin `navigator.requestMIDIAccess()` calls.
|
||||
|
||||
Native providers register source summaries with `providerId`, a stable `sourceId`, a derived redaction-safe `logicalSourceKey` (`providerId::sourceId`), `kind: "midi"`, a label, and `availability`. The public command surface is `inspect`, `list-sources`, `discover`, `select-source`, `open-source`, and `close-source`; provider operations are `source.enumerate`, `source.describe`, `source.open`, and `source.close`. `inspect`, `list-sources`, and `select-source` are prompt-free and never request MIDI access. Unlike audio (where `getUserMedia` gates labels and `open-source` is the prompt), Web-MIDI's `requestMIDIAccess()` gates the whole input list, so **`discover` is the permission boundary** and records `denied`/`unavailable` outcomes; `open-source` then attaches a shared listener session and never re-prompts.
|
||||
|
||||
Selected input is persisted by `logicalSourceKey` (`slopsmith.midiInput.selectedLogicalSourceKey`) when browser storage is available. Compatible requesters share one open session per source; each later calls `close-source`, and the provider receives `source.close` only after the last requester releases. Live MIDI message delivery (for the "play a note / hit a pad" calibration check) is exposed to in-page consumers through the public `window.slopsmith.midiInput` session handle only — never as raw capability events or diagnostics.
|
||||
|
||||
The reserved `midi-control` domain is the planned **sibling** for control mappings (CC/pitchbend/note → action routing) and will consume `midi-input` for device access (spec 013 / #882); this slice carves the device control plane out so `midi-control` can stay mappings-only. `midi-control` stays RESERVED (documentation-only) until a concrete mapping consumer + tests exist, per the future-domain governance.
|
||||
|
||||
Diagnostics live under `slopsmith.midi_input.diagnostics.v1` and contain provider ids, source ids/keys/kinds/availability, the selected key, and open-session keys — **device labels are redacted** and no raw MIDI messages are ever included.
|
||||
|
||||
## Capability Roles
|
||||
|
||||
Use capability declarations for provider/requester/observer relationships:
|
||||
|
||||
@@ -108,7 +108,7 @@ These domains are planned but should stay out of the runtime graph until a host
|
||||
| `ui.player-overlays` | exclusive-owner | safe | Overlay contributions layered over player or highway surfaces. | Overlay placement and z-order rules that coexist with legacy overlays. |
|
||||
| `plugins` | exclusive-owner | privileged | Plugin enable/disable/install/update workflows. | Visible user confirmation, rollback, and disabled-handler enforcement. |
|
||||
| `jobs` | multi-provider | privileged | Long-running jobs, cancellation, status, failures. | Scheduling limits, cancellation semantics, and user-visible failures. |
|
||||
| `midi-control` | multi-provider | sensitive | MIDI device providers and control mappings. | Device consent and redacted diagnostics. |
|
||||
| `midi-control` | multi-provider | sensitive | MIDI control mappings only (CC/pitchbend/note → action routing), consuming `midi-input` for device access. Device discovery/selection/open is split out to the delivered `midi-input` domain (spec 012). | A concrete mapping/routing workflow on top of the `midi-input` device plane (#882). |
|
||||
| `audio-input` | multi-provider | sensitive | Audio input device providers, source selection, open/close lifecycle, shared sessions, and redacted failure diagnostics. | Promoted by the audio graph/session slice and implemented by the audio-input control-plane slice. |
|
||||
| `tempo-clock` | multi-provider | safe | Tempo/clock provider registration and consumers. | A concrete tempo source and consumer workflow. |
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ Core domains also have a review scope. **Active contract** domains are wired to
|
||||
|
||||
| note-detection | provider-coordinator | sensitive | inspect, register-provider, unregister-provider, open-binding, close-binding, set-target, clear-target | pitch.estimate, verify.target | Detection-binding control plane (spec 009): providers (midi/engine/js) serve primitives; each requester binds its own redacted tuning context; consumers own judgment, hit/miss flow as observability events. Legacy `highway.setNoteStateProvider` is an accounted shim. Diagnostics carry provider/binding summaries and bounded outcomes — no raw audio, sample data, device labels, or song identity. |
|
||||
|
||||
| midi-input | provider-coordinator | sensitive | inspect, list-sources, discover, select-source, open-source, close-source | source.enumerate, source.describe, source.open, source.close | Core-owned MIDI device control plane (spec 012), the MIDI analog of `audio-input`. Inspect/list/select are prompt-free; `discover` is the Web-MIDI permission boundary (`requestMIDIAccess()` gates the whole input list) and records denied/unavailable outcomes; `open-source` attaches a shared listener session and never re-prompts. Selection persists by redaction-safe `logicalSourceKey`. Diagnostics redact device labels and never include raw MIDI messages or live handles. |
|
||||
|
||||
Privileged commands are roadmap-only until they have: a visible user confirmation path, diagnostics redaction rules, failure recovery, and tests that prove disabled or incompatible participants cannot execute handlers.
|
||||
|
||||
## Expected Future Domains
|
||||
@@ -38,7 +40,7 @@ These domains are expected future capability contracts, not current runtime grap
|
||||
| ui.player-overlays | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs overlay placement rules that coexist with legacy highway overlays. |
|
||||
| plugins | exclusive-owner | privileged | enable, disable, install-missing, update, inspect | Needs explicit user confirmation for writes/install/update. |
|
||||
| jobs | multi-provider | privileged | register, inspect, cancel | Needs scheduling limits, cancellation semantics, and user-visible failures. |
|
||||
| midi-control | multi-provider | sensitive | register, inspect | Needs device consent and redacted diagnostics. |
|
||||
| midi-control | multi-provider | sensitive | list-mappings, get-mapping, set-mapping, delete-mapping, activate-mapping, inspect | Mappings ONLY — CC/pitchbend/note → semantic action routing (spec 013). Device discovery/selection/open is NOT this domain's job: it consumes the delivered `midi-input` domain for device access. Needs a concrete mapping consumer (the MIDI control plugin / drums learn-mode) + redacted diagnostics (no raw MIDI streams) before promotion. |
|
||||
| tempo-clock | multi-provider | safe | register, inspect | Needs a concrete provider and consumer workflow. |
|
||||
|
||||
Planned domains should also stay out of the runtime graph until Slopsmith ships the corresponding user-facing workflows.
|
||||
|
||||
@@ -1,180 +0,0 @@
|
||||
# Note-state provider API
|
||||
|
||||
The contract that lets a scorer (today: `note_detect`) tell every renderer (today: built-in 2D, bundled 3D, anything you build) when a chart note has been **hit / actively held / missed**.
|
||||
|
||||
Introduced in [slopsmith#254](https://github.com/byrongamatos/slopsmith/issues/254). See [`realtime-scoring-pipeline.md`](realtime-scoring-pipeline.md) for how data flows into the provider; this doc focuses on the API itself.
|
||||
|
||||
**Audience**: plugin authors who want to either (a) **register** as the scorer (replacing `note_detect`), or (b) **read** per-note state from a custom visualization.
|
||||
|
||||
## The API in one minute
|
||||
|
||||
Two endpoints on the `highway` object, one bundle field exposed to renderers:
|
||||
|
||||
```js
|
||||
// Producer side (a scorer plugin)
|
||||
highway.setNoteStateProvider((note, chartTime) => {
|
||||
// return null / 'hit' / 'active' / 'miss' / { state, alpha, color }
|
||||
});
|
||||
highway.getNoteStateProvider(); // current provider, or null
|
||||
highway.setNoteStateProvider(null); // clear
|
||||
|
||||
// Consumer side (inside your renderer's draw)
|
||||
const st = bundle.getNoteState(note, chartTime); // same return shape
|
||||
```
|
||||
|
||||
**One provider at a time** — last writer wins. There's no event-emitter / multi-subscriber pattern; the contract is deliberately singular so the renderer's per-frame call is allocation-free and unambiguous. If two scorer plugins coexist, the second one's `setNoteStateProvider` call replaces the first.
|
||||
|
||||
The provider is called **per visible note, per frame, from the renderer's `draw()` loop**. Keep it cheap — a typical highway can have 30+ visible gems per frame across 60 fps. The bundled `note_detect` provider does one Map lookup + a few arithmetic operations.
|
||||
|
||||
## Return values
|
||||
|
||||
| Return | Meaning | When to use |
|
||||
|---|---|---|
|
||||
| `null` / `undefined` / `false` / `0` / `""` | No state for this note this frame. Renderer draws it normally. | The vast majority of returns. Default for unstruck notes, expired effects, notes the scorer doesn't know about. |
|
||||
| `'hit'` | Note was struck cleanly. | Right after a successful detection, while the brief post-strike glow is visible. |
|
||||
| `'active'` | A sustained note is *currently being held on-pitch*. | While a sustain trail should glow. Re-emit every frame the hold is still valid; stop returning state when the sustain ends. |
|
||||
| `'miss'` | Note expired without a clean detection. | After the scoring window closed and no hit was registered. |
|
||||
| `{ state, alpha, color }` | Full object form. Same `state`. Adds `alpha` (0..1 brightness multiplier) and optional `color` override. | Whenever you want a custom fade or to tint with something other than the renderer's default string colour. |
|
||||
|
||||
### `alpha`
|
||||
|
||||
A brightness multiplier in `[0, 1]`. **The provider owns the fade.** The renderer doesn't track timestamps or decay; it just renders whatever brightness you return.
|
||||
|
||||
- Returning a bare string (`'hit'`) is equivalent to `{ state: 'hit', alpha: 1, color: null }`.
|
||||
- Returning `{ state: 'hit', alpha: 0 }` is the same as returning `null` — the highway clamps it and drops the result before the renderer sees it. Use this as a clean way to signal "stop rendering" without changing your control flow.
|
||||
- Out-of-range numbers are clamped to `[0, 1]`. Non-finite (`NaN`, `Infinity`) falls back to `1`.
|
||||
|
||||
### `color`
|
||||
|
||||
Optional CSS-style colour string (e.g., `'#7ef'`, `'rgb(255, 90, 90)'`). When present, the renderer uses it instead of the default string colour for the lit gem / sustain trail. When absent or non-string, the renderer keeps its default palette.
|
||||
|
||||
## What the highway does to your return value
|
||||
|
||||
[`static/highway.js`](../static/highway.js) line 290:
|
||||
|
||||
```js
|
||||
function _noteState(note, chartTime) {
|
||||
if (!_noteStateProvider) return null;
|
||||
let raw;
|
||||
try { raw = _noteStateProvider(note, chartTime); } catch (e) { return null; }
|
||||
if (!raw) return null;
|
||||
const state = typeof raw === 'string' ? raw : raw.state;
|
||||
if (state !== 'hit' && state !== 'active' && state !== 'miss') return null;
|
||||
const alpha = (raw && typeof raw === 'object' && Number.isFinite(raw.alpha))
|
||||
? Math.max(0, Math.min(1, raw.alpha))
|
||||
: 1;
|
||||
if (alpha <= 0) return null;
|
||||
const color = (raw && typeof raw === 'object' && typeof raw.color === 'string') ? raw.color : null;
|
||||
return { state, alpha, color };
|
||||
}
|
||||
```
|
||||
|
||||
Practical implications:
|
||||
|
||||
- **Provider exceptions are swallowed.** If your provider throws, the renderer sees `null` for that note this frame. The next frame's call is independent. Cleanest for the renderer (no draw stall), trickiest for the provider author (no console signal that something's wrong). Log inside your provider if you need to diagnose.
|
||||
- **Invalid `state` strings return `null`.** Typos like `'Hit'` or `'hits'` are silently rejected. The renderer keeps drawing the note dim.
|
||||
- **`alpha === 0` returns `null`.** The dim-render path is what you want here anyway; this is the cleanest exit.
|
||||
- **Renderers receive the normalized object** (`{ state, alpha, color }`) or `null`. They never see your raw return value, so feel free to return shorthands.
|
||||
|
||||
## How to call `bundle.getNoteState` from a renderer
|
||||
|
||||
The bundle passed to `draw(bundle)` carries a stable reference (`getNoteState: _noteState` — same function every frame, never reallocated). You can cache it on the renderer instance:
|
||||
|
||||
```js
|
||||
init(canvas, bundle) {
|
||||
this.ctx = canvas.getContext('2d');
|
||||
this.getNoteState = bundle.getNoteState; // safe to cache the function ref
|
||||
},
|
||||
draw(bundle) {
|
||||
for (const n of bundle.notes) {
|
||||
const st = this.getNoteState(n, n.t);
|
||||
if (st && st.state === 'hit') /* paint bright */;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Two pitfalls when calling it
|
||||
|
||||
1. **Pass `chord.t`, not `cn.t`, for chord constituents.** Chord constituents are keyed in the producer by the chord's time, not the constituent's. (Most chart formats give the constituent the same `t` as the chord anyway, but the contract is explicit and you should follow it.)
|
||||
```js
|
||||
for (const ch of bundle.chords) {
|
||||
for (const cn of ch.notes) {
|
||||
const st = bundle.getNoteState(cn, ch.t); // ← ch.t
|
||||
}
|
||||
}
|
||||
```
|
||||
2. **Don't cache the return value across frames.** It encodes a fade. Re-call every frame.
|
||||
|
||||
## How to register as a provider
|
||||
|
||||
```js
|
||||
function myNoteStateFor(note, chartTime) {
|
||||
// note: { t, s, f, sus, … } — chart note object (or a chord constituent)
|
||||
// chartTime: number — `t` for single notes, the chord's `t` for chord constituents
|
||||
const verdict = myStore.lookup(`${chartTime}_${note.s}_${note.f}`);
|
||||
if (!verdict) return null;
|
||||
const age = performance.now() - verdict.timestamp;
|
||||
if (age > 800) return null; // owned fade — drop after 800 ms
|
||||
return {
|
||||
state: verdict.hit ? 'hit' : 'miss',
|
||||
alpha: Math.max(0, 1 - age / 800), // linear fade
|
||||
};
|
||||
}
|
||||
|
||||
// Mount on every highway in the session. In a typical plugin:
|
||||
// - call once on `song:ready`, after `createHighway()` has wired the panel
|
||||
// - call again whenever a new panel mounts (splitscreen)
|
||||
highway.setNoteStateProvider(myNoteStateFor);
|
||||
```
|
||||
|
||||
To play nicely with other plugins (or with `note_detect`'s default registration), check what's already there:
|
||||
|
||||
```js
|
||||
const existing = highway.getNoteStateProvider();
|
||||
if (existing == null) {
|
||||
highway.setNoteStateProvider(myNoteStateFor);
|
||||
} else {
|
||||
// Decide: chain (fallback to existing if you return null),
|
||||
// replace (existing.foo behaviour is lost), or abort.
|
||||
}
|
||||
```
|
||||
|
||||
## Keying conventions used by `note_detect`
|
||||
|
||||
If your provider is replacing or extending `note_detect`'s, match its key format:
|
||||
|
||||
| Note shape | Key in `noteResults` |
|
||||
|---|---|
|
||||
| Single note at `t = 12.34`, string `s`, fret `f` | `"12.34_{s}_{f}"` |
|
||||
| Chord constituent (chord at `t = 12.34`, constituent string `s`, fret `f`) | `"12.34_{s}_{f}"` — keyed by **chord time**, not constituent time |
|
||||
| Chord-frame verdict for that chord | `"12.34_chord"` |
|
||||
|
||||
The string interpolation uses the chart's `t` directly (no rounding, no string coercion beyond `${}`). Don't `.toFixed(2)` it — small floating-point drift between chord and constituent `t` values would create a key mismatch.
|
||||
|
||||
## Sustain state — `'hit'` vs `'active'`
|
||||
|
||||
Both are "lit" states; the difference is *who owns the fade*:
|
||||
|
||||
- `'hit'`: the gem was struck cleanly. The provider returns this once per frame, decaying `alpha` over a few hundred ms, then returns `null` when the glow should end.
|
||||
- `'active'`: a sustained note (one with `sus > 0`) is currently being held on-pitch. The provider keeps returning `{ state: 'active', alpha: 1 }` every frame as long as the hold is valid. When the hold ends (pitch slipped, sustain expired, song paused), the provider stops returning state.
|
||||
|
||||
A renderer typically treats both the same way visually — bright gem, bright sustain trail — but `'active'` differs in *duration*: it stays for as long as the hold lasts (potentially seconds), while `'hit'` is a short post-strike glow.
|
||||
|
||||
`note_detect` tracks the on-pitch state of held sustains in a `_susActiveUntil` Map (key → `performance.now()` when the grace window ends). A typical grace window is 30 ms (one or two pitch frames), so a brief audio gap during a sustained note doesn't kill the glow.
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
1. **Provider returns truthy forever → gem stays lit forever.** Always return `null` when the effect should end. The renderer has no other way to know.
|
||||
2. **Provider tracks fade in renderer state instead.** Don't. The provider is consulted by *every* renderer and you'd end up with mismatched fades on each. Owns-its-own-decay is the design.
|
||||
3. **Renderer passes a per-constituent `t` instead of the chord's.** Silent failure — the lookup misses, all chord gems render dim. See the second pitfall above.
|
||||
4. **Calling `setNoteStateProvider` once at module load.** The highway might not exist yet, or a later splitscreen panel might create another highway. Re-register on every `song:ready` and every new panel mount.
|
||||
5. **Renderer caches the bundle.** Bundle arrays are live references but the object itself is constructed per frame for some fields. Don't hold `let myBundle = bundle` across frames. Caching the function ref `bundle.getNoteState` is fine — it's stable.
|
||||
6. **Provider does heavy work per call.** It runs O(visible notes × FPS) — typically 1000+ calls/sec during dense passages. A Map lookup is fine; a regex / JSON parse / sort is not.
|
||||
|
||||
## Where to look next
|
||||
|
||||
- [`realtime-scoring-pipeline.md`](realtime-scoring-pipeline.md) — full data flow from audio → verdict → provider.
|
||||
- [`visualization-feedback-guide.md`](visualization-feedback-guide.md) — practical "how do I add this to my viz" walkthrough with a minimal 2D Canvas example.
|
||||
- [`../plugins/note_detect/`](../plugins/note_detect/) — the canonical producer. Search `screen.js` for `setNoteStateProvider` and `noteStateFor`.
|
||||
- [`../plugins/highway_3d/screen.js`](../plugins/highway_3d/screen.js) — canonical custom-renderer consumer. Search for `getNoteState`.
|
||||
- [`../CLAUDE.md`](../CLAUDE.md) § "Note-state provider" — the short version of this doc embedded in the plugin-system overview.
|
||||
@@ -1,227 +0,0 @@
|
||||
# Real-time scoring pipeline
|
||||
|
||||
This doc traces how a single audio sample becomes a lit-up gem on the highway. It exists because the path crosses five layers — native engine, IPC bridge, `note_detect` plugin, highway core, renderer — and the contract between them is currently spread across `slopsmith/CLAUDE.md`, `slopsmith-desktop/`, and four different plugin source files.
|
||||
|
||||
**Audience**: plugin authors building a visualization that wants to react to detected hits, and maintainers debugging why a gem isn't lighting up.
|
||||
|
||||
**Companion docs**
|
||||
- [`note-state-provider.md`](note-state-provider.md) — the `setNoteStateProvider` / `bundle.getNoteState` API contract, in isolation.
|
||||
- [`visualization-feedback-guide.md`](visualization-feedback-guide.md) — a practical "how do I add hit feedback to my custom viz" walkthrough.
|
||||
- `slopsmith-desktop/docs/audio-engine-architecture.md` (companion repo) — internals of the JUCE engine + ML detector that produce the verdicts this pipeline consumes.
|
||||
|
||||
## Pipeline at a glance
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ slopsmith-desktop (native, JUCE + ONNX + Basic Pitch) │
|
||||
│ │
|
||||
│ audio device → ring buffer → ┌─ PitchDetector (YIN) │
|
||||
│ ├─ MlNoteDetector (Basic Pitch / ONNX) │
|
||||
│ ├─ ChordScorer (FFT + harmonic comb) │
|
||||
│ └─ NoteVerifier (background thread) │
|
||||
│ │ │
|
||||
└───────────────────────────────────────┼──────────────────────────────┘
|
||||
│ IPC (Electron preload)
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ note_detect plugin (slopsmith/plugins/note_detect/screen.js) │
|
||||
│ │
|
||||
│ drain audio:getNoteVerdicts(songTime, playing) │
|
||||
│ (or fall back to browser matchNotes + audio:scoreChord) │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ noteResults Map keyed by `${time}_${string}_${fret}` │
|
||||
│ │ │
|
||||
│ ├─► window.slopsmith.emit('note:hit' | 'note:miss', judgment) │
|
||||
│ │ │
|
||||
│ └─► highway.setNoteStateProvider(noteStateFor) │
|
||||
└───────────────────────────────────────┬──────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ highway core (slopsmith/static/highway.js) │
|
||||
│ │
|
||||
│ bundle.getNoteState(note, chartTime) → null │
|
||||
│ │ 'hit' | 'active' | 'miss' │
|
||||
│ │ { state, alpha, color } │
|
||||
└───────────────────────────────────────┬──────────────────────────────┘
|
||||
│ per visible note, per frame
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Renderer (built-in 2D, bundled 3D, or your custom viz) │
|
||||
│ │
|
||||
│ draw(bundle) { │
|
||||
│ for (const n of bundle.notes) { │
|
||||
│ const st = bundle.getNoteState(n, n.t); │
|
||||
│ // light gem, sustain, sparkle … │
|
||||
│ } │
|
||||
│ } │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Two facts shape every other detail:
|
||||
|
||||
1. **Verdicts are produced once and consumed many times.** A renderer must not call `audio:scoreChord` itself. Scoring runs once in the engine (or in `note_detect`'s browser fallback) and the resulting per-note state is published through a single, idempotent provider. Renderers read the provider in their `draw()` loop and that's it.
|
||||
2. **The chord-time semantic is load-bearing.** Chord constituents are keyed in `noteResults` as `${chordTime}_${string}_${fret}`. A renderer that passes a per-constituent timestamp instead of the chord's `t` will look up nothing and silently render dim gems. See [`note-state-provider.md`](note-state-provider.md) for the rule.
|
||||
|
||||
## Layer 1 — Native engine
|
||||
|
||||
In normal desktop sessions, scoring runs in C++ on a JUCE audio thread + a background verifier thread, not in JavaScript. The architecture doc in the companion repo covers internals; the surface area visible to JS is:
|
||||
|
||||
| IPC method | Purpose | Cadence on the JS side |
|
||||
|---|---|---|
|
||||
| `audio:setChart(chart)` | Push the active arrangement once per song / arrangement switch. | Once per `playSong()`. |
|
||||
| `audio:getNoteVerdicts(songTime, playing)` | Drain finalized per-note verdicts AND push the renderer's playhead. | ~20 Hz (the `detectInterval` tick). |
|
||||
| `audio:getPitchDetection()` | Monophonic dominant pitch (YIN or ML, whichever is ready). | ~20 Hz. |
|
||||
| `audio:scoreChord(ctx)` | One-shot polyphonic score against the engine's input ring. | Per chord event during matching (fallback path). |
|
||||
| `audio:detectNotes()` | Full ML active-pitch set: `{notes: [{midi, confidence, onsetMs, onsetSeq}], sampleRate}`. | Optional; only the diagnostic / ML-aware code paths use it. |
|
||||
| `audio:isMlNoteDetection()` | `true` when Basic Pitch is loaded and ready. | Once at startup. |
|
||||
|
||||
All of these are feature-detected (`typeof audio.getNoteVerdicts === 'function'`) so the renderer keeps working against a downlevel addon.
|
||||
|
||||
Handlers live in [`audio-bridge.ts`](../../slopsmith-desktop/src/main/audio-bridge.ts) (in the `slopsmith-desktop` repo). Look for the `ipcMain.handle('audio:setChart', ...)`, `'audio:getNoteVerdicts'`, etc. blocks around lines 500–560.
|
||||
|
||||
## Layer 2 — `note_detect` plugin
|
||||
|
||||
`note_detect` is the only consumer of those IPC methods today. Everything downstream consumes `note_detect`'s output, not the engine's output directly. There are **two paths** through this layer; the engine path is preferred on desktop, the browser path is the fallback.
|
||||
|
||||
### Engine-verifier path (modern, desktop only)
|
||||
|
||||
On `playSong()`, `note_detect` calls `audio:setChart(arrangementChart)` and then polls `audio:getNoteVerdicts(songTime, playing)` on the detect tick. The verifier in C++ has already scored each chart note against the audio ring (using harmonic-comb + onset detection); each call returns the verdicts that have finalized since the last drain.
|
||||
|
||||
Each verdict has the shape:
|
||||
|
||||
```js
|
||||
{
|
||||
id: string, // chart-note id assigned at setChart time
|
||||
detected: boolean, // hit or miss
|
||||
detectedSongTime: number, // when the engine saw the onset
|
||||
centsError: number, // cents off the expected pitch
|
||||
snr: number // confidence-ish signal:noise ratio
|
||||
}
|
||||
```
|
||||
|
||||
`note_detect` translates each verdict into its internal `judgment` shape and writes it into `noteResults`, keyed `${chartNote.t}_${chartNote.s}_${chartNote.f}` for single notes, and `${chord.t}_${cn.s}_${cn.f}` for each chord constituent. Chord-level verdicts also write a chord-time-only entry `${chord.t}_chord` used for chord-frame tint.
|
||||
|
||||
### Browser fallback path (legacy / non-desktop)
|
||||
|
||||
If the desktop bridge isn't available (web build, downlevel addon, audio engine stopped), the same plugin runs the entire pipeline in the browser:
|
||||
|
||||
1. `getUserMedia` → `AudioContext` → `ChannelSplitterNode` → `ScriptProcessor` (per-channel mono).
|
||||
2. Per-frame YIN pitch detect + onset gate.
|
||||
3. `matchNotes()` — windowed match between detected events and chart notes (timing tolerance + pitch tolerance from settings).
|
||||
4. For chord events: `_ndScoreChord()` runs a renderer-side FFT + band-energy scorer on the accumulated buffer.
|
||||
|
||||
Either way, the result lands in `noteResults` with the same keys, and the downstream consumers don't care which path produced it.
|
||||
|
||||
### Publishing
|
||||
|
||||
Two channels:
|
||||
|
||||
```js
|
||||
// 1. Global event (for stats panels, journaling plugins, etc.)
|
||||
window.slopsmith.emit(judgment.hit ? 'note:hit' : 'note:miss', judgment);
|
||||
|
||||
// 2. Per-note state provider (for the highway + renderers)
|
||||
highway.setNoteStateProvider(noteStateFor);
|
||||
```
|
||||
|
||||
The provider function `noteStateFor(note, chartTime)` reads `noteResults` and returns one of:
|
||||
|
||||
| Return | Meaning |
|
||||
|---|---|
|
||||
| `null` (or anything falsy) | No state — renderer draws this note normally. |
|
||||
| `'hit'` | Note was struck cleanly. Fade owned by the provider. |
|
||||
| `'active'` | Sustained note currently still being held on-pitch. |
|
||||
| `'miss'` | Note expired without a clean detection. |
|
||||
| `{ state, alpha, color }` | Object form — same `state` field, plus explicit `alpha` (0..1) and optional `color` override. |
|
||||
|
||||
The full contract — including the keying rules and sustain semantics — is in [`note-state-provider.md`](note-state-provider.md).
|
||||
|
||||
## Layer 3 — Highway core
|
||||
|
||||
The highway exposes the provider to renderers via the per-frame `bundle` passed to `draw(bundle)`. The relevant lines in [`static/highway.js`](../static/highway.js):
|
||||
|
||||
- The provider slot itself: `let _noteStateProvider = null;` at line 153.
|
||||
- The normalizer that the bundle exposes: `function _noteState(note, chartTime) { … }` at line 290. This catches exceptions thrown by the provider, validates the `state` is one of `'hit'/'active'/'miss'`, clamps `alpha` to `[0, 1]`, drops zero-alpha returns, and returns `null` otherwise.
|
||||
- The bundle field: `getNoteState: _noteState` (line 514) — a **stable reference**, no per-frame allocation. Custom renderers can safely cache it.
|
||||
- The API method: `setNoteStateProvider(fn)` on the highway's exported object (line 2974). Last writer wins; passing `null` clears the provider.
|
||||
|
||||
The built-in 2D renderer consumes the provider in three places — `drawNote` (line 1445), `drawSustains` (line 1364), and the chord-frame path (line 1655) — so the default visuals work as soon as `note_detect` is enabled, without any per-plugin wiring.
|
||||
|
||||
## Layer 4 — Renderer
|
||||
|
||||
In your renderer's `draw(bundle)`, the typical pattern is:
|
||||
|
||||
```js
|
||||
draw(bundle) {
|
||||
for (const n of bundle.notes) {
|
||||
const state = bundle.getNoteState(n, n.t); // n.t for single notes
|
||||
if (state) {
|
||||
// state.state is 'hit' | 'active' | 'miss'
|
||||
// state.alpha is 0..1 (provider-owned decay)
|
||||
// state.color is optional string override
|
||||
drawLitGem(n, state);
|
||||
} else {
|
||||
drawDimGem(n);
|
||||
}
|
||||
}
|
||||
for (const ch of bundle.chords) {
|
||||
for (const cn of ch.notes) {
|
||||
const state = bundle.getNoteState(cn, ch.t); // ch.t, NOT cn.t
|
||||
// …
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Two patterns the bundled renderers use that you can copy:
|
||||
|
||||
- **2D highway** ([`static/highway.js`](../static/highway.js) `drawNote` at line 1445): `'hit'`/`'active'` → bright string color + additive halo + sparkle. `'miss'` → faint red wash. Sustain trail brightens with `'active'`.
|
||||
- **3D highway** ([`plugins/highway_3d/screen.js`](../plugins/highway_3d/screen.js) — search for `getNoteState`): outline mesh + body switch from dim material (`mStr[s]`) to bright (`mGlow[s]`); sustained notes keep the glow while `'active'`; a contained sparkle is queued on the 2D overlay layer.
|
||||
|
||||
For a from-scratch viz, [`visualization-feedback-guide.md`](visualization-feedback-guide.md) has a minimal Canvas2D example.
|
||||
|
||||
## Concrete trace: strumming a C major at chart time 12.34 s
|
||||
|
||||
Chart event: a `chord` at `t = 12.34` with three constituents — strings 1 (B), 2 (G), 3 (D) at frets `(1, 0, 2)`.
|
||||
|
||||
1. **t ≈ 12.30 s**: the player strums. The audio thread captures samples into `inputFrameRing`. The Basic Pitch ML detector publishes `B3`, `G3`, `D3` as active pitches; the `NoteVerifier` thread, which has had the chart since `playSong()`, opens scoring windows around `t = 12.34` for each chord-note (`12.34 ± timingTolerance`).
|
||||
2. **t ≈ 12.34 s**: Each open-window note's harmonic comb confirms the expected pitch present. The verifier closes the windows shortly after `12.34 + timingTolerance` and pushes three verdicts onto the drain queue: `{detected: true, detectedSongTime: 12.31, centsError: +4, snr: 5.3}` for each.
|
||||
3. **t ≈ 12.39 s**: `note_detect`'s detect tick calls `audio:getNoteVerdicts(12.39, true)`. It receives the three verdicts, builds three `judgment` objects, writes:
|
||||
- `noteResults["12.34_1_1"] = { hit: true, … }`
|
||||
- `noteResults["12.34_2_0"] = { hit: true, … }`
|
||||
- `noteResults["12.34_3_2"] = { hit: true, … }`
|
||||
- `noteResults["12.34_chord"] = { hit: true, hitStrings: 3, totalStrings: 3, … }`
|
||||
- Emits `window.slopsmith.emit('note:hit', judgment)` for the chord.
|
||||
4. **Next rAF**: the renderer's `draw(bundle)` iterates `bundle.chords`. For the chord at `t = 12.34` it calls `bundle.getNoteState(cn, 12.34)` for each constituent. Each call resolves `noteStateFor(cn, 12.34)`, which looks up `noteResults["12.34_${cn.s}_${cn.f}"]`, finds a recent `hit`, returns `{ state: 'hit', alpha: ~1, color: null }`.
|
||||
5. **2D renderer**: each gem in the chord renders with bright string color + halo + sparkle. The chord-frame tint goes green.
|
||||
6. **3D renderer (if active)**: outline + body swap to `mGlow[s]`; a sparkle entry is queued for each gem on the 2D overlay layer; the chord frame box tints green.
|
||||
7. **Over the next ~300 ms**: `noteStateFor` returns a decaying `alpha` (provider-owned fade). When `alpha` drops below the floor, it returns `null` and the gems revert to dim.
|
||||
|
||||
If any string had been missed (say string 3 silent because of an over-aggressive mute), step 2 would have closed that window with `detected: false`. `noteResults["12.34_3_2"]` would be a `miss`, the chord-frame verdict would still be `hit` (any-constituent-hit rule per [PR #316](https://github.com/byrongamatos/slopsmith/pull/316)), and the renderer would paint string 3 with the miss wash while the others lit up.
|
||||
|
||||
## Where to look next
|
||||
|
||||
If you want to…
|
||||
|
||||
- **Build a custom viz that reacts to hits** → [`visualization-feedback-guide.md`](visualization-feedback-guide.md).
|
||||
- **Understand the provider contract in full** (semantics of `alpha`, key formats, sustain rules) → [`note-state-provider.md`](note-state-provider.md).
|
||||
- **Dig into the engine internals** (NoteVerifier thread, ML detector loading, ring buffer) → `slopsmith-desktop/docs/audio-engine-architecture.md` in the companion repo.
|
||||
- **Tune detection quality** (false misses, latency, A/V offset) → [`note-detect-tuning.md`](note-detect-tuning.md).
|
||||
- **Read the canonical setRenderer contract** → [`../CLAUDE.md`](../CLAUDE.md) § "Visualization plugins".
|
||||
|
||||
## Key files
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `slopsmith-desktop/src/audio/AudioEngine.cpp:1447` | Audio device callback — entry point for every input sample. |
|
||||
| `slopsmith-desktop/src/audio/AudioEngine.h:196` | `inputFrameRing` declaration (lock-free SPSC, 8192 samples). |
|
||||
| `slopsmith-desktop/src/audio/NoteVerifier.cpp:152` | Background verifier thread loop. |
|
||||
| `slopsmith-desktop/src/audio/NoteVerifier.cpp:52` | `setChart()` — push the active arrangement. |
|
||||
| `slopsmith-desktop/src/main/audio-bridge.ts` | IPC handlers (look for `audio:setChart`, `audio:getNoteVerdicts`, `audio:scoreChord`). |
|
||||
| `slopsmith/plugins/note_detect/screen.js` | `noteResults` map, `noteStateFor`, `setNoteStateProvider` registration, verdict drain loop. |
|
||||
| `slopsmith/static/highway.js:153` | `_noteStateProvider` slot. |
|
||||
| `slopsmith/static/highway.js:290` | `_noteState()` normalizer (catches exceptions, clamps alpha, validates state). |
|
||||
| `slopsmith/static/highway.js:514` | `getNoteState: _noteState` bundle field. |
|
||||
| `slopsmith/static/highway.js:2974` | `setNoteStateProvider` on the public highway API. |
|
||||
@@ -4,7 +4,7 @@ A `.sloppak` is just a zip of plain files: some YAML, some JSON, some OGG audio,
|
||||
|
||||
This guide walks through the most common edits, aimed at musicians who are comfortable with a text editor and Audacity but don't live on the command line.
|
||||
|
||||
> For the format **schema** (what every field means, how the wire format works, how to extend the format with new data types), see [sloppak-spec.md](sloppak-spec.md). This document is the **how-do-I-actually-edit-mine** companion.
|
||||
> For the format **schema** (what every field means, how the wire format works, how to extend the format with new data types), see the authoritative [feedpak spec](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md) (the local [sloppak-spec.md](sloppak-spec.md) is now a pointer to it). This document is the **how-do-I-actually-edit-mine** companion.
|
||||
|
||||
---
|
||||
|
||||
@@ -242,7 +242,7 @@ For 4-string bass, only indices 0–3 are meaningful; leave 4 and 5 at `0`.
|
||||
|
||||
### What *not* to put in `manifest.yaml`
|
||||
|
||||
Don't add per-machine settings (audio device picks, MIDI port IDs), UI state, or your own play counts. The sloppak holds the song's authored data — anything that varies by user or machine lives in Slopsmith's config dir or the metadata DB. See [sloppak-spec.md §5.7](sloppak-spec.md#57-dont-break-the-manifest-contract) for the full list.
|
||||
Don't add per-machine settings (audio device picks, MIDI port IDs), UI state, or your own play counts. The sloppak holds the song's authored data — anything that varies by user or machine lives in Slopsmith's config dir or the metadata DB. See [feedpak spec §9.5](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md#95-what-does-not-belong-in-a-feedpak) for the full list.
|
||||
|
||||
---
|
||||
|
||||
@@ -261,6 +261,6 @@ For your own use, you can skip this entirely — Slopsmith reads the directory f
|
||||
|
||||
## Out of scope (for now)
|
||||
|
||||
- **Authoring a sloppak from scratch** (no Guitar Pro / MusicXML source file) — that's a developer task. Start at [sloppak-spec.md §4.2](sloppak-spec.md#42-writing-python-server-side).
|
||||
- **Editing notes / chords in `arrangements/*.json`** — technically possible but extremely tedious by hand: hundreds of objects with short field names per song. The fields are documented in [sloppak-spec.md §3](sloppak-spec.md#3-arrangement-json--the-wire-format), but for any real chart edit you want the [Arrangement Editor plugin](https://github.com/got-feedback/feedback-plugin-editor).
|
||||
- **Authoring a sloppak from scratch** (no Guitar Pro / MusicXML source file) — that's a developer task. Start at [feedpak spec §8 (Reading and writing)](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md#8-reading-and-writing).
|
||||
- **Editing notes / chords in `arrangements/*.json`** — technically possible but extremely tedious by hand: hundreds of objects with short field names per song. The fields are documented in [feedpak spec §6 (Arrangement JSON)](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md#6-arrangement-json), but for any real chart edit you want the [Arrangement Editor plugin](https://github.com/got-feedback/feedback-plugin-editor).
|
||||
- **Loudness normalization / advanced stem processing** — out of scope here; standard Audacity or ffmpeg workflows apply to any OGG file before you drop it into `stems/`.
|
||||
|
||||
+31
-921
@@ -1,939 +1,49 @@
|
||||
# Sloppak Format — Developer Guide
|
||||
# Sloppak / feedpak Format — moved
|
||||
|
||||
Sloppak is Slopsmith's open, hand-editable song format. This guide is for developers who want to **read**, **write**, or **extend** the format — including adding new data types like drum tabs, vocal pitches, lighting cues, key/scale annotations, or anything else a future visualization plugin might need.
|
||||
The full format specification that used to live here has moved to its own repository and is now
|
||||
the **authoritative, versioned reference**:
|
||||
|
||||
> If you're a **user** wanting to modify an existing sloppak — record your own rhythm stem, fix metadata, swap cover art, replace a Demucs split — see [sloppak-hand-editing.md](sloppak-hand-editing.md). That guide is the practical, step-by-step companion to this developer reference.
|
||||
> **📖 https://github.com/got-feedback/feedback-feedpak-spec**
|
||||
> — normative spec ([`spec/feedpak-v1.md`](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md)),
|
||||
> JSON Schemas, examples, and a reference validator.
|
||||
|
||||
The authoritative format reference lives in code (`lib/sloppak.py`, `lib/song.py`); this doc explains the why, the how, and the conventions you should follow when adding to it.
|
||||
Update bookmarks to point there. This page is a thin pointer kept at the original path so existing
|
||||
links keep resolving.
|
||||
|
||||
---
|
||||
## Naming: `sloppak` here, `feedpak` in the spec
|
||||
|
||||
## 1. Format at a glance
|
||||
The published format is named **feedpak** (extension `.feedpak`, manifest key `feedpak_version`).
|
||||
This codebase still uses the legacy **sloppak** name internally — `lib/sloppak.py`, the
|
||||
`.sloppak` extension, `SLOPSMITH_*` env vars, etc. **They describe the same on-disk format.** The
|
||||
rename is repo/public-facing only for now (see the top-level workspace `CLAUDE.md`), so when the
|
||||
spec says `feedpak` / `feedpak_version`, the packs this server reads and writes today are the same
|
||||
structure under the `.sloppak` name. The internal rename is a separate, later effort.
|
||||
|
||||
A sloppak exists in **two interchangeable forms**:
|
||||
## Hand-editing a pack
|
||||
|
||||
| Form | What it is | Used for |
|
||||
|---|---|---|
|
||||
| **Directory** | A folder named `*.sloppak/` containing the files below | Authoring, hand editing, plugin development |
|
||||
| **Zip archive** | A `.sloppak` file (zip with the same files inside) | Distribution |
|
||||
For the practical "how do I edit my own pack" walkthrough (record your own stem, fix metadata,
|
||||
swap cover art, replace a stem split), see the companion guide that stays in this repo:
|
||||
[sloppak-hand-editing.md](sloppak-hand-editing.md).
|
||||
|
||||
Both forms hold identical contents. Slopsmith resolves either transparently — zip files are unpacked to a cache the first time they're opened (see `resolve_source_dir()` in [lib/sloppak.py](../lib/sloppak.py)).
|
||||
## Where the format maps to code (this repo)
|
||||
|
||||
### Directory layout
|
||||
|
||||
```
|
||||
my-song.sloppak/
|
||||
├── manifest.yaml # Required — all metadata + file index
|
||||
├── arrangements/
|
||||
│ ├── lead.json # One JSON per playable arrangement
|
||||
│ ├── rhythm.json
|
||||
│ └── bass.json
|
||||
├── stems/
|
||||
│ ├── full.ogg # Mixed audio (initial single-stem output; may be absent after stem splitting)
|
||||
│ ├── guitar.ogg # Optional individual stems
|
||||
│ ├── bass.ogg
|
||||
│ ├── drums.ogg
|
||||
│ ├── vocals.ogg
|
||||
│ └── other.ogg
|
||||
├── lyrics.json # Optional — syllable-level lyrics
|
||||
└── cover.jpg # Optional — album art
|
||||
```
|
||||
|
||||
Three rules to remember:
|
||||
|
||||
1. **`manifest.yaml` is the index.** Nothing inside the sloppak is auto-discovered — every file path is listed in the manifest. This makes the format predictable: no scanning, no guessing. (One historical exception: the cover-art handler in `server.py` falls back to `cover.jpg` when `manifest.cover` is missing. New code should not add similar filename fallbacks.)
|
||||
2. **Filenames in `manifest.yaml` are POSIX paths**, relative to the sloppak root (forward slashes, no leading `/`).
|
||||
3. **YAML for the manifest, JSON for everything else.** YAML is hand-editable for users; JSON is fast-parsed and easy to round-trip in code.
|
||||
|
||||
---
|
||||
|
||||
## 2. `manifest.yaml` reference
|
||||
|
||||
Minimal valid manifest:
|
||||
|
||||
```yaml
|
||||
title: "Black Hole Sun"
|
||||
artist: "Soundgarden"
|
||||
duration: 320.5
|
||||
arrangements:
|
||||
- id: lead
|
||||
name: Lead
|
||||
file: arrangements/lead.json
|
||||
tuning: [0, 0, 0, 0, 0, 0]
|
||||
capo: 0
|
||||
stems:
|
||||
- id: full
|
||||
file: stems/full.ogg
|
||||
default: true
|
||||
```
|
||||
|
||||
Full set of currently-recognized top-level keys:
|
||||
|
||||
| Key | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `title` | string | yes | Song title |
|
||||
| `artist` | string | yes | Artist name |
|
||||
| `album` | string | no | Album |
|
||||
| `year` | int | no | Release year |
|
||||
| `duration` | float | yes | Song length in seconds |
|
||||
| `arrangements` | list | yes | Playable arrangements (see §2.1) |
|
||||
| `stems` | list | yes | Audio stems (see §2.2) |
|
||||
| `stem_separation` | object | no | Structured metadata when stems were produced by an automated separation engine (currently `demucs`). Shape: `{engine, model, version}`. See §2.2 for fields + semver semantics per [slopsmith#357](https://github.com/got-feedback/feedback/issues/357). Omitted for single-stem sloppaks (`stems: [{id: full, ...}]`) and for hand-edited / user-recorded stems |
|
||||
| `lyrics` | string | no | Path to lyrics JSON |
|
||||
| `lyrics_source` | string | no | Where the lyrics came from: `xml` (vocals XML from the chart source), `whisperx` (auto-transcribed), or `user` (hand-edited). Absent on legacy sloppaks — readers should treat missing as `xml` |
|
||||
| `lyric_transcription` | object | no | Structured metadata when lyrics came from an automated engine (currently `whisperx`). Same shape as the parent `stem_separation` block defined by [slopsmith#357](https://github.com/got-feedback/feedback/issues/357) — see §2.3 for fields and semver semantics. Omitted for authored lyrics (`xml`/`user`) |
|
||||
| `vocal_pitch` | string | no | Path to per-syllable pitch JSON (`{"version": 1, "notes": [{t, d, midi}, ...]}`). Consumed by [slopsmith-plugin-lyrics-karaoke](https://github.com/got-feedback/feedback-plugin-lyrics-karaoke) to render karaoke note bars. See §2.4 |
|
||||
| `pitch_extraction` | object | no | Structured metadata when pitch was extracted by an automated engine (currently `crepe` via the demucs server's `/pitch` endpoint). Same shape as `stem_separation` / `lyric_transcription`. Omitted for hand-edited pitch tracks |
|
||||
| `cover` | string | no | Path to cover image |
|
||||
| `preview` | string | no | Path to a short preview audio clip (OGG) at the sloppak root. Populated when the source carries a separate short browser-preview clip (decoded to `preview.ogg`); absent otherwise. Consumed by [`slopsmith-plugin-song-preview`](https://github.com/got-feedback/feedback-plugin-song-preview) for hover-to-listen previews in the library |
|
||||
| `song_timeline` | string | no | Path to a `song_timeline.json` file carrying song-wide beats and sections (see §5.3). When present, its data takes priority over any beats/sections embedded in arrangement JSONs. Older readers ignore the key and fall back to reading beats/sections from the first arrangement JSON as before |
|
||||
| `drum_tab` | string | no | Path to `drum_tab.json` — per-piece drum hits (see §5.3). Implemented end-to-end as of slopsmith#344 |
|
||||
|
||||
Unknown keys are **silently ignored** by the loader. This is deliberate — it's the extensibility hook (see §5).
|
||||
|
||||
### 2.1. `arrangements[]`
|
||||
|
||||
Each entry describes one playable arrangement and points at its JSON file:
|
||||
|
||||
```yaml
|
||||
arrangements:
|
||||
- id: lead # filesystem-safe stable ID, used for filenames
|
||||
name: Lead # display name (Lead/Rhythm/Bass/Combo are sorted first)
|
||||
file: arrangements/lead.json
|
||||
tuning: [0, 0, 0, 0, 0, 0] # six semitone offsets from E A D G B E
|
||||
capo: 0
|
||||
centOffset: 0.0 # optional float, cents; default 0.0
|
||||
```
|
||||
|
||||
- `tuning` is a list of semitone offsets from standard `E2 A2 D2 G3 B3 E4`. **Six elements is the standard six-string convention** and the only length `lib/tunings.py` produces friendly names for; 5- and 7-string content is accepted by the loader and falls through to a numeric label. For bass, the four bass strings are at indices 0–3; the other two slots are `0`. Consumers should not hard-code `len(tuning) == 6`.
|
||||
- `name` controls the sort order in the UI: `Lead > Combo > Rhythm > Bass > everything else`.
|
||||
- `centOffset` is a pitch-shift value in cents. Commonly `-1200.0` for extended-range bass arrangements tuned one octave down; small non-zero values for songs mastered at a non-A440 reference pitch (e.g. A443 ≈ +11.8 cents). Absent / `0.0` means no shift. Exposed to plugins via `getSongInfo().centOffset`.
|
||||
- Manifest-level `tuning`, `capo`, and `centOffset` **override** anything embedded in the arrangement JSON. The arrangement JSON's own values are fallbacks.
|
||||
- `notation` (optional string) — path to a `notation_<id>.json` file carrying standard musical notation data for this arrangement (see §5.3). When present, the loader surfaces it on `LoadedSloppak.notation_by_id[id]` and the highway WS streams `notation_info` + `notation_measures` messages. The `file:` key may be omitted when `notation:` is present — the loader creates a stub arrangement so the notation file can be the sole data source.
|
||||
|
||||
### 2.2. `stems[]`
|
||||
|
||||
```yaml
|
||||
stems:
|
||||
- id: full
|
||||
file: stems/full.ogg
|
||||
default: true # plays by default when the song opens
|
||||
- id: guitar
|
||||
file: stems/guitar.ogg
|
||||
default: true
|
||||
- id: drums
|
||||
file: stems/drums.ogg
|
||||
default: false
|
||||
```
|
||||
|
||||
- `id` is referenced by the Stems plugin and any other consumer; keep it stable.
|
||||
- `default` accepts `true`/`false`, or strings (`"on"`/`"off"`/`"true"`/etc.) for hand-edited manifests.
|
||||
- A freshly converted sloppak from `lib/sloppak_convert.py` starts with a single `{id: full, file: stems/full.ogg, ...}` entry. After stem-splitting (Demucs), `full.ogg` is removed and the manifest is rewritten with per-instrument entries (`guitar`, `bass`, `drums`, `vocals`, `other`). The format requires only that `stems` is non-empty — there's no specific filename or id that must always be present.
|
||||
|
||||
When stems were produced by an automated separation engine (Demucs), an optional `stem_separation` block records which engine + model produced them. Per [slopsmith#357](https://github.com/got-feedback/feedback/issues/357):
|
||||
|
||||
```yaml
|
||||
stem_separation:
|
||||
engine: demucs # stable engine id; only `demucs` today
|
||||
model: htdemucs_6s # specific model name (htdemucs_6s / htdemucs_ft / htdemucs / mdx_extra / ...)
|
||||
version: 1.0.0 # semver for slopsmith's stem-artifact contract
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `engine` — stable identifier for the separation engine. Currently always `demucs`. New engines (e.g. a hypothetical `spleeter`) would get their own stable id.
|
||||
- `model` — the engine-specific model id used for this split. For Demucs this is the `-n` flag value.
|
||||
- `version` — semver for Slopsmith's stem-artifact contract (independent of upstream Demucs / model versions). Bump per the same semantics #357 defines: patch = metadata-only fixes, minor = backward-compatible additions, major = stem set / packing / post-processing changed and existing splits should be regenerated.
|
||||
|
||||
Omitted for single-stem sloppaks (`stems: [{id: full, ...}]` — no automated separation ran) and for hand-edited / user-recorded stems. The RFC reserves a separate `stem_authoring` sibling block for the hand-edit case; that's deferred to a follow-up.
|
||||
|
||||
A remote Demucs server can use this block as part of a cache key so that changing the model or major version naturally produces a cache miss. Local plugin jobs should preserve this metadata in job state and in any copied/downloaded manifests.
|
||||
|
||||
### 2.3. `lyrics`
|
||||
|
||||
If present, points at a JSON file containing a flat list of syllable objects:
|
||||
|
||||
```json
|
||||
[
|
||||
{"t": 12.34, "d": 0.18, "w": "Hel"},
|
||||
{"t": 12.52, "d": 0.22, "w": "lo-"},
|
||||
{"t": 13.10, "d": 0.30, "w": "world"}
|
||||
]
|
||||
```
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `t` | Time in seconds |
|
||||
| `d` | Duration in seconds |
|
||||
| `w` | Syllable text. Trailing `-` joins to the next syllable as one word; trailing `+` marks the last syllable of a line (renderer wraps after it). Both are suffixes on a real syllable — not standalone entries. See `static/highway.js` for the rendering: `raw.endsWith('+')` flags end-of-line, and `sylText` strips the trailing marker before drawing |
|
||||
|
||||
When lyrics are present, the optional top-level `lyrics_source` key records where they came from. The assembler sets it to `xml` when the lyrics were parsed from the source chart's vocals XML; the WhisperX auto-transcription fallback (`scripts/transcribe_lyrics.py`, or `--auto-lyrics` on the split scripts) sets it to `whisperx`. Hand-edited lyrics should bump it to `user` so UI consumers can render a different badge (or no badge) than for machine-generated lyrics. The key is absent on sloppaks produced before this field existed — readers should treat missing as `xml` for backward compatibility.
|
||||
|
||||
When `lyrics_source` is `whisperx` (or any future automated engine), an optional `lyric_transcription` block records which engine + model produced the file. Shape mirrors the parent `stem_separation` RFC ([slopsmith#357](https://github.com/got-feedback/feedback/issues/357)):
|
||||
|
||||
```yaml
|
||||
lyric_transcription:
|
||||
engine: whisperx # stable engine id
|
||||
model: medium # the WhisperX model size that ran (tiny/base/small/medium/large-v2/large-v3)
|
||||
version: 1.0.0 # semver for slopsmith's lyric-transcription artifact contract
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `engine` — stable identifier for the transcription engine; currently always `whisperx`.
|
||||
- `model` — the engine-specific model id used for this transcription.
|
||||
- `version` — semver for Slopsmith's lyric-transcription artifact contract (independent of upstream Whisper / WhisperX versions). Bump per the same semantics #357 defines for stems: patch = metadata-only fixes, minor = backward-compatible additions, major = output shape changed and existing transcriptions should be regenerated.
|
||||
|
||||
Omitted for authored lyrics (`xml` / `user`). A remote WhisperX server can use this block as part of a cache key the same way #357 envisions for stems — caches should miss whenever any of the three fields change, ensuring stale transcriptions don't get returned after a model bump.
|
||||
|
||||
### 2.4. `vocal_pitch`
|
||||
|
||||
If present, points at a JSON file holding per-syllable pitch data — the karaoke companion to `lyrics`. Consumed by [slopsmith-plugin-lyrics-karaoke](https://github.com/got-feedback/feedback-plugin-lyrics-karaoke) to render karaoke-style note bars over the lyric text. Shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"notes": [
|
||||
{"t": 12.34, "d": 0.40, "midi": 64},
|
||||
{"t": 12.78, "d": 0.55, "midi": 67}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `version` | Schema version of this `vocal_pitch.json` file (currently the integer `1`). Bump on a breaking change to the `notes` entry shape. This is *not* the same as the top-level `pitch_extraction.version` block below, which is a semver string used as a cache-key for the extractor engine |
|
||||
| `notes` | List of pitch entries, one per syllable that the extractor could lock onto. `t` + `d` mirror the matching `lyrics.json` entry; `midi` is the MIDI note number (60 = middle C). Syllables the extractor couldn't pitch (silent / sub-confidence) are omitted from this list — it may be shorter than `lyrics.json` |
|
||||
|
||||
When pitch came from an automated engine (the demucs server's `/pitch` endpoint, which runs CREPE), the optional top-level `pitch_extraction` block records which engine + model produced the file. Same shape and semver-string semantics as `stem_separation` / `lyric_transcription` — distinct from the in-file integer `version` field above:
|
||||
|
||||
```yaml
|
||||
pitch_extraction:
|
||||
engine: crepe
|
||||
model: v1
|
||||
version: 1.0.0
|
||||
```
|
||||
|
||||
Omitted for hand-edited pitch tracks. As with the other two automated-artifact blocks, a remote pitch server can use this for cache-key invalidation.
|
||||
|
||||
The sloppak assembler runs pitch extraction automatically when `pitch_extraction.enabled` is set in its config AND a server URL is configured (either `pitch_extraction.server_url` or the shared `demucs_server_url`) AND the sloppak has lyrics + a `stems/vocals.ogg` after the split pass — either because `_maybe_transcribe_lyrics` just produced them via WhisperX OR because they were already on disk (from the source chart's vocals XML, hand-authoring, or an earlier build). Pitch is *not* coupled to `whisperx.enabled` — setting `pitch_extraction.enabled=true` alone (with WhisperX off) is enough to retro-generate pitch over any existing on-disk lyrics. Sloppaks built before this field existed simply don't carry it — readers should treat missing `vocal_pitch` as "no pitch data, fall back to whatever the karaoke plugin's local-extraction path produces (if any)".
|
||||
|
||||
---
|
||||
|
||||
## 3. Arrangement JSON — the wire format
|
||||
|
||||
Arrangement JSON files use the **wire format** produced by `arrangement_to_wire()` — the on-disk representation of a complete arrangement. Slopsmith's `/ws/highway/{filename}` endpoint transports similar data as a sequence of typed messages (`notes`, `chords`, `anchors`, `chord_templates`, `phrases`, …) rather than as one identical top-level JSON object. In practice, the WebSocket stream reuses the same per-object field names where applicable, but it should not be treated as a byte-for-byte match for `arrangements/*.json`.
|
||||
|
||||
The authoritative serializer/deserializer is in [lib/song.py](../lib/song.py):
|
||||
|
||||
- `arrangement_to_wire(arr) → dict` — write
|
||||
- `arrangement_from_wire(dict) → Arrangement` — read
|
||||
|
||||
### 3.1. Top-level shape
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Lead",
|
||||
"tuning": [0, 0, 0, 0, 0, 0],
|
||||
"capo": 0,
|
||||
"centOffset": 0.0, /* optional, float cents, default 0.0 */
|
||||
"notes": [ /* see 3.2 */ ],
|
||||
"chords": [ /* see 3.3 */ ],
|
||||
"anchors": [ /* see 3.4 */ ],
|
||||
"handshapes": [ /* see 3.5 */ ],
|
||||
"templates": [ /* see 3.6 */ ],
|
||||
"phrases": [ /* optional, see 3.7 */ ],
|
||||
"tones": { /* optional, see 3.9 */ },
|
||||
"beats": [ /* see 3.8, only on first arrangement */ ],
|
||||
"sections": [ /* see 3.8, only on first arrangement */ ]
|
||||
}
|
||||
```
|
||||
|
||||
`beats` and `sections` are **song-level** but live on the first arrangement's JSON for legacy reasons — `lib/sloppak.py` hoists them to the `Song` object on load. If you author multiple arrangements, only put them in one file. **New sloppaks should use `song_timeline.json` instead** (see §2 and §5.3) — when the manifest carries a `song_timeline:` key pointing at a schema-valid file, its beats/sections **replace** whatever the arrangement JSONs loaded (the override is applied after arrangement loading, so a valid `song_timeline.json` always wins). Arrangement-JSON beats/sections remain supported for backward compatibility with all existing sloppaks and are the fallback when the file is absent or invalid.
|
||||
|
||||
### 3.2. Notes
|
||||
|
||||
Field names are short on purpose — these get streamed thousands of times per song. Don't expand them.
|
||||
|
||||
```json
|
||||
{
|
||||
"t": 12.345, // time (s)
|
||||
"s": 2, // string (0 = lowest)
|
||||
"f": 7, // fret (0 = open, 24 = max)
|
||||
"sus": 0.5, // sustain (s, 0 = none)
|
||||
"sl": 9, // pitched slide-to fret (-1 = no slide)
|
||||
"slu": -1, // unpitched slide-to fret (-1 = no slide)
|
||||
"bn": 1.0, // bend amount in semitones
|
||||
"ho": false, // hammer-on
|
||||
"po": false, // pull-off
|
||||
"hm": false, // natural harmonic
|
||||
"hp": false, // pinch harmonic
|
||||
"pm": false, // palm mute
|
||||
"mt": false, // string mute
|
||||
"vb": false, // vibrato
|
||||
"tr": false, // tremolo
|
||||
"ac": false, // accent
|
||||
"tp": false, // tap
|
||||
"ln": false, // link-next (chord linking metadata; renderers may ignore — runtime linking is derived from proximity)
|
||||
"fhm": false, // fret-hand mute
|
||||
"plk": false, // pluck (pop, bass)
|
||||
"slp": false, // slap (bass)
|
||||
"rh": -1, // right-hand fingering (-1 = unset)
|
||||
"pkd": -1, // pick direction (-1 = unset, 0 = down, 1 = up)
|
||||
"ig": false // ignore (chart-author flag — note is rendered but not scored / sequenced)
|
||||
}
|
||||
```
|
||||
|
||||
Default values: numbers → `0` or `-1` (slides / `rh` / `pkd`), bools → `false`. Omit fields equal to their default if you're authoring by hand — the parser fills them in. **Encoders should default-omit the newer technique keys** (`ln`, `fhm`, `plk`, `slp`, `rh`, `pkd`, `ig`) — the highway streams notes thousands of times per song, so trimming the common case keeps the WebSocket payload tight. The pre-existing keys are still emitted unconditionally to preserve the legacy wire contract.
|
||||
|
||||
### 3.3. Chords
|
||||
|
||||
A chord groups note-shaped objects under a single time:
|
||||
|
||||
```json
|
||||
{
|
||||
"t": 30.0,
|
||||
"id": 12, // index into templates[]
|
||||
"hd": false, // high-density flag
|
||||
"notes": [
|
||||
{"s": 0, "f": 3, "sus": 0.0, ...},
|
||||
{"s": 1, "f": 5, "sus": 0.0, ...}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Chord notes use the same field set as standalone notes, **except `t` is omitted** (the chord carries the time). The fingering / shape lookup is `chord.id → templates[id]`.
|
||||
|
||||
### 3.4. Anchors
|
||||
|
||||
Where the fretting hand sits. Drives the highway zoom box.
|
||||
|
||||
```json
|
||||
{"time": 12.0, "fret": 5, "width": 4}
|
||||
```
|
||||
|
||||
### 3.5. Hand shapes
|
||||
|
||||
Spans during which a chord shape is held:
|
||||
|
||||
```json
|
||||
{"chord_id": 12, "start_time": 30.0, "end_time": 31.5, "arp": false}
|
||||
```
|
||||
|
||||
- `chord_id` (`int`, default `0`) — index into `templates[]`; identifies which chord template the span is holding.
|
||||
- `start_time` (`float`, default `0.0`) — start of the span in seconds.
|
||||
- `end_time` (`float`, default `0.0`) — end of the span in seconds.
|
||||
- `arp` (`bool`, default `false`, allowed values `true`/`false`) — whether this hand shape should be treated as an arpeggio span rather than a fully-strummed chord hold.
|
||||
|
||||
### 3.6. Chord templates
|
||||
|
||||
Named shapes referenced by `chord.id` and `handshape.chord_id`:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Em7",
|
||||
"displayName": "Em7",
|
||||
"arp": false,
|
||||
"fingers": [-1, 2, 1, -1, -1, -1],
|
||||
"frets": [ 0, 2, 2, 0, 0, 0]
|
||||
}
|
||||
```
|
||||
|
||||
- `name` (`string`, default `""`) — canonical template name used by the parser / authoring data.
|
||||
- `displayName` (`string`, default `name`) — label shown in the UI; source XML may use this for display-specific variants such as `-arp`.
|
||||
- `arp` (`bool`, default `false`, allowed values `true`/`false`) — whether the template is flagged as arpeggiated. Parsed from explicit XML attributes (`arpeggio` / `arp`, any common casing) or inferred from `displayName` markers such as `-arp`.
|
||||
- `fingers` (`int[6]`, default `[-1, -1, -1, -1, -1, -1]`) — fretting-hand finger numbers, lowest string first. `-1` = unused string, `0` = open string / no fretting finger, `1..4` = index/middle/ring/pinky.
|
||||
- `frets` (`int[6]`, default `[-1, -1, -1, -1, -1, -1]`) — fret numbers, lowest string first. `-1` = unused string, `0` = open string, positive values = fretted note.
|
||||
|
||||
### 3.7. Phrases (optional, multi-difficulty data)
|
||||
|
||||
Sources that carry per-phrase difficulty ladders (phrase-aware arrangement XML) include this. GP imports and legacy sloppaks omit it:
|
||||
|
||||
```json
|
||||
"phrases": [
|
||||
{
|
||||
"start_time": 0.0,
|
||||
"end_time": 12.5,
|
||||
"max_difficulty": 4,
|
||||
"levels": [
|
||||
{ "difficulty": 0, "notes": [...], "chords": [...], "anchors": [...], "handshapes": [...] },
|
||||
{ "difficulty": 1, "notes": [...], "chords": [...], "anchors": [...], "handshapes": [...] },
|
||||
...
|
||||
]
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
If you're writing a converter that doesn't have multi-difficulty data, **omit the `phrases` key entirely** (don't emit `"phrases": []`). A missing key signals "no ladder, disable the master-difficulty slider"; an empty list is the same in current code but reads ambiguously.
|
||||
|
||||
### 3.8. Beats and sections
|
||||
|
||||
```json
|
||||
"beats": [{"time": 0.5, "measure": 1}, {"time": 1.0, "measure": -1}, ...],
|
||||
"sections": [{"name": "verse", "number": 1, "time": 12.5}, ...]
|
||||
```
|
||||
|
||||
`measure: -1` = sub-beat (not a downbeat). Section `name` follows the usual song-structure conventions (`intro`, `verse`, `chorus`, `bridge`, `solo`, `outro`, …).
|
||||
|
||||
### 3.9. Tones (optional)
|
||||
|
||||
`tones` carries the arrangement's guitar tones — the amp/pedal/cabinet gear and the in-song tone switches. It's populated when the source chart carries tone data (`lib/tones.py`); a sloppak authored from scratch may omit it entirely.
|
||||
|
||||
```json
|
||||
"tones": {
|
||||
"base": "Clean Rhythm",
|
||||
"changes": [
|
||||
{"t": 12.5, "name": "Lead Drive"},
|
||||
{"t": 48.0, "name": "Clean Rhythm"}
|
||||
],
|
||||
"definitions": [
|
||||
{
|
||||
"Name": "Clean Rhythm",
|
||||
"Key": "Tone_A",
|
||||
"GearList": { /* raw gear blocks: Amp, PrePedal1-4, … */ }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `base` (string) — the tone in effect before the first change.
|
||||
- `changes` (list, time-sorted) — `{"t": seconds, "name": str}` tone switches. The highway draws a marker at each. Omit when the arrangement never switches tone.
|
||||
- `definitions` (list) — the **raw tone objects** (`Name`, `Key`, `GearList`), copied verbatim from the source chart's tone manifest. The Tones plugin parses these into the rendered signal chain (it owns the gear-name/image map, so the data is stored unparsed here).
|
||||
|
||||
All three sub-keys are individually optional; an arrangement with none of them simply omits `tones`. Readers that don't know about tones ignore the key (the loader preserves it verbatim).
|
||||
|
||||
---
|
||||
|
||||
## 4. Reading and writing sloppaks programmatically
|
||||
|
||||
### 4.1. Reading (Python, server-side)
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from sloppak import load_song, load_manifest
|
||||
|
||||
# Quick metadata only (parses manifest, skips arrangement JSONs)
|
||||
manifest = load_manifest(Path("song.sloppak"))
|
||||
|
||||
# Full song load (manifest + all arrangements + lyrics)
|
||||
loaded = load_song("song.sloppak", dlc_root=Path("/dlc"), unpack_cache_root=Path("/cache"))
|
||||
print(loaded.song.title, len(loaded.song.arrangements))
|
||||
print(loaded.stems) # [{"id": "full", "file": "stems/full.ogg", "default": True}]
|
||||
print(loaded.manifest) # raw dict — read your custom keys here
|
||||
```
|
||||
|
||||
### 4.2. Writing (Python, server-side)
|
||||
|
||||
There's no general-purpose writer in `lib/` yet. The current writer lives in [lib/sloppak_convert.py](../lib/sloppak_convert.py) inside the sloppak assembly function — it's the single source of truth for "how a sloppak gets built." If you need to write sloppaks from a new source, copy the structure of that function:
|
||||
|
||||
1. Build a `work_dir/` in temp.
|
||||
2. Write `arrangements/{id}.json` per arrangement using `arrangement_to_wire()`.
|
||||
3. Encode audio to OGG into `stems/`.
|
||||
4. Optionally write `lyrics.json`, `cover.jpg`.
|
||||
5. Compose the `manifest` dict and dump as YAML with `yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True)`.
|
||||
6. Either `shutil.copytree(work_dir, out)` for directory form, or `_zip_dir(work_dir, out)` for zip form.
|
||||
|
||||
Always use `yaml.safe_dump` (not `yaml.dump`) and pass `sort_keys=False` so the human-readable order is preserved.
|
||||
|
||||
### 4.3. Reading (JavaScript, plugin-side)
|
||||
|
||||
Plugins typically don't read the sloppak file directly — they consume the `/ws/highway/{filename}` WebSocket stream (see `CLAUDE.md` for the message protocol), which produces the same shapes. If you specifically need raw manifest access from the browser, expose it through a custom backend route in your plugin's `routes.py` and fetch it.
|
||||
|
||||
---
|
||||
|
||||
## 5. Extending the format — adding new data
|
||||
|
||||
Sloppak is designed to be extended without breaking older readers. The conventions below come from how `lyrics`, `stems`, and the optional `phrases` ladder were each added.
|
||||
|
||||
### 5.1. The golden rule: **manifest opt-in, file off to the side**
|
||||
|
||||
New data types should follow this pattern:
|
||||
|
||||
1. **Drop a new file** alongside the standard ones (e.g., `drums.json`, `keys.json`, `lighting.json`).
|
||||
2. **Add a manifest key** that *points at* that file (e.g., `drum_tab: drums.json`).
|
||||
3. **Make consumers gate on the manifest key**: if the key is absent, do nothing. Never auto-discover by filename — that breaks the "manifest is the index" rule.
|
||||
|
||||
So a sloppak with drum tabs would look like:
|
||||
|
||||
```yaml
|
||||
# manifest.yaml
|
||||
title: "Song"
|
||||
artist: "Band"
|
||||
duration: 240.0
|
||||
arrangements: [...]
|
||||
stems: [...]
|
||||
drum_tab: drum_tab.json # ← new key
|
||||
```
|
||||
|
||||
```
|
||||
my-song.sloppak/
|
||||
├── manifest.yaml
|
||||
├── arrangements/...
|
||||
├── stems/...
|
||||
└── drum_tab.json # ← new file
|
||||
```
|
||||
|
||||
Older Slopsmith readers ignore the unknown `drum_tab` key (the loader uses `manifest.get("drum_tab")` / unknown keys pass through). Your plugin checks for it and renders accordingly. **Zero coordination needed with core.**
|
||||
|
||||
### 5.2. Naming conventions for new keys and files
|
||||
|
||||
- **Manifest keys**: `snake_case`, descriptive, singular when the value is one thing (`lyrics`, `cover`, `drum_tab`), plural when it's a list (`stems`, `arrangements`).
|
||||
- **File names**: lowercase, hyphenated or underscored, JSON for structured data, OGG for audio, JPG/PNG for images.
|
||||
- **Inside JSON**: short field names for hot-path data that gets streamed thousands of times (`t`, `s`, `f` — see §3.2). Long names are fine for one-off metadata.
|
||||
- **Time fields**: always `t` or `time` (not `start`, not `timestamp`) — and always **seconds as floats**, not ms or ticks. Be consistent with the existing wire format.
|
||||
- **Indexes / IDs**: stable, filesystem-safe, lowercase. Don't reuse a source format's internal numeric IDs unless you have to.
|
||||
|
||||
### 5.3. Worked examples for the kinds of additions you mentioned
|
||||
|
||||
#### Drum tab
|
||||
|
||||
`drum_tab.json` carries per-piece hits authored on top of the song's audio.
|
||||
Implemented end-to-end as of slopsmith#344 (drums-from-scratch): the loader
|
||||
in `lib/sloppak.py` parses it, `lib/drums.py` defines the canonical piece-id
|
||||
vocabulary, and `/ws/highway/{filename}` streams it as `drum_tab` + chunked
|
||||
`drum_hits` messages.
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"name": "Drums",
|
||||
"kit": [
|
||||
{"id": "kick", "name": "Kick"},
|
||||
{"id": "snare", "name": "Snare"},
|
||||
{"id": "hh_closed", "name": "Hi-hat (closed)"},
|
||||
{"id": "hh_open", "name": "Hi-hat (open)"},
|
||||
{"id": "crash_r", "name": "Crash (right)"},
|
||||
{"id": "ride", "name": "Ride"}
|
||||
],
|
||||
"hits": [
|
||||
{"t": 0.500, "p": "kick", "v": 110},
|
||||
{"t": 0.750, "p": "snare", "v": 92},
|
||||
{"t": 0.750, "p": "hh_closed", "v": 70},
|
||||
{"t": 1.000, "p": "snare", "v": 60, "g": true},
|
||||
{"t": 1.250, "p": "snare", "v": 105, "f": true},
|
||||
{"t": 4.000, "p": "crash_r", "v": 120, "k": 0.080}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Manifest:
|
||||
|
||||
```yaml
|
||||
drum_tab: drum_tab.json
|
||||
```
|
||||
|
||||
##### Hit fields
|
||||
|
||||
| key | type | meaning |
|
||||
| --- | --- | --- |
|
||||
| `t` | float seconds | hit time, required, monotonic in `hits[]` |
|
||||
| `p` | string | piece-id from the closed list below; required |
|
||||
| `v` | int 1-127 | velocity (default 100) |
|
||||
| `g` | bool | ghost note (renders smaller / outline-only) |
|
||||
| `f` | bool | flam (renders a small leading ghost glyph 30 ms early) |
|
||||
| `k` | float seconds | cymbal-choke tail duration (renders a fade-out) |
|
||||
|
||||
##### Canonical piece-id vocabulary
|
||||
|
||||
A closed list lives in `lib/drums.py::PIECES`. Open/closed hi-hat are
|
||||
**distinct piece-ids**, not articulation flags — hit detection must reject
|
||||
a closed-hat strike on an open-hat note, which it can only do if the
|
||||
articulation is part of the piece-id.
|
||||
|
||||
| piece-id | category | default GM MIDI | default shape |
|
||||
| --- | --- | --- | --- |
|
||||
| `kick` | kick | 35, 36 | bar (full-width across all non-kick lanes) |
|
||||
| `snare` | drum | 38, 40 | rectangle |
|
||||
| `snare_xstick` | drum | 37 | hatched rectangle |
|
||||
| `tom_hi` | drum | 50, 48 | rectangle |
|
||||
| `tom_mid` | drum | 47, 45 | rectangle |
|
||||
| `tom_low` | drum | 43 | rectangle |
|
||||
| `tom_floor` | drum | 41 | rectangle |
|
||||
| `hh_closed` | cymbal | 42 | filled circle |
|
||||
| `hh_open` | cymbal | 46 | ring (outline) circle |
|
||||
| `hh_pedal` | cymbal | 44 | small circle with × |
|
||||
| `stack` | cymbal | 30 | jagged circle (no GM standard — reuses 30 from extended-percussion range) |
|
||||
| `crash_l` | cymbal | 49 | circle |
|
||||
| `crash_r` | cymbal | 57 | circle |
|
||||
| `splash` | cymbal | 55 | small circle |
|
||||
| `china` | cymbal | 52 | jagged circle |
|
||||
| `ride` | cymbal | 51, 59 | circle |
|
||||
| `ride_bell` | cymbal | 53 | circle with centre dot |
|
||||
| `bell` | cymbal | 80 | circle with centre dot (no GM standard — reuses "Mute Triangle") |
|
||||
|
||||
Unknown piece-ids round-trip through the loader (forward-compat); the
|
||||
client just renders them as a default rectangle.
|
||||
|
||||
##### Wire format
|
||||
|
||||
Streamed as two highway-WS message types:
|
||||
|
||||
```json
|
||||
{ "type": "drum_tab", "version": 1, "name": "Drums",
|
||||
"kit": [{"id": "kick", "name": "Kick"}, ...], "total": 1234 }
|
||||
```
|
||||
|
||||
…followed by one or more chunks of 500 hits:
|
||||
|
||||
```json
|
||||
{ "type": "drum_hits", "data": [{"t": 0.5, "p": "kick", "v": 110}, ...],
|
||||
"total": 1234 }
|
||||
```
|
||||
|
||||
##### Design notes
|
||||
|
||||
- `kit[]` is the legend — fixed metadata, separated from hot-path data.
|
||||
- `hits[]` uses short field names because this list can be thousands long.
|
||||
- `v` defaults to 100; ghost / flam / choke flags are all optional.
|
||||
- Older sloppaks whose drums are encoded as guitar notes (`midi = string*24 + fret`) still play — the drums plugin keeps a legacy decoder that reads the standard `notes` stream and synthesises `drum_hits` from it.
|
||||
|
||||
#### Song timeline (beats and sections as a top-level file)
|
||||
|
||||
`song_timeline.json` moves song-wide beats and sections out of the first
|
||||
arrangement JSON and into a dedicated file. Implemented in `lib/sloppak.py`
|
||||
alongside the notation format: the loader reads the manifest's optional
|
||||
`song_timeline:` key, validates the file, and populates `Song.beats` /
|
||||
`Song.sections` from it, taking priority over any beats/sections embedded
|
||||
in arrangement JSONs.
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"beats": [
|
||||
{"time": 0.500, "measure": 1},
|
||||
{"time": 1.000, "measure": -1},
|
||||
{"time": 1.500, "measure": -1},
|
||||
{"time": 2.000, "measure": 2}
|
||||
],
|
||||
"sections": [
|
||||
{"name": "intro", "number": 1, "time": 0.0},
|
||||
{"name": "verse", "number": 1, "time": 16.0},
|
||||
{"name": "chorus", "number": 1, "time": 32.0}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Manifest:
|
||||
|
||||
```yaml
|
||||
song_timeline: song_timeline.json
|
||||
```
|
||||
|
||||
| Field in `beats[]` | Type | Notes |
|
||||
|---|---|---|
|
||||
| `time` | float seconds | Beat timestamp. Matches the existing arrangement-JSON wire convention |
|
||||
| `measure` | int | 1-based downbeat number. `-1` = sub-beat (not a downbeat) |
|
||||
|
||||
| Field in `sections[]` | Type | Notes |
|
||||
|---|---|---|
|
||||
| `name` | string | song-structure convention: `intro`, `verse`, `chorus`, `bridge`, `solo`, `outro`, … |
|
||||
| `number` | int | Section repeat number |
|
||||
| `time` | float seconds | Section start |
|
||||
|
||||
**Backward compatibility.** Sloppaks without `song_timeline:` continue to
|
||||
work — the loader falls through to reading beats/sections from the first
|
||||
arrangement JSON exactly as before. No migration is needed.
|
||||
|
||||
**New sloppaks** should put beats/sections here and leave arrangement JSONs
|
||||
free of timeline data. This is especially important for notation-only
|
||||
arrangements (see below) where there may be no arrangement JSON at all.
|
||||
|
||||
---
|
||||
|
||||
#### Notation format (standard musical notation per arrangement)
|
||||
|
||||
The notation format promotes keys, piano, violin, and any other
|
||||
staff-notation instrument to first-class status with their own data
|
||||
structure, separate from the guitar wire format. Implemented in
|
||||
`lib/sloppak.py` and `lib/notation.py`; the highway WS streams
|
||||
`notation_info` + `notation_measures` messages when notation data is
|
||||
present for the active arrangement.
|
||||
|
||||
**Architecture: per-arrangement, not song-wide.** Unlike `drum_tab`
|
||||
(one drum track per song, top-level manifest key), notation is
|
||||
per-instrument. A song could carry both `notation_keys.json` and
|
||||
`notation_violin.json`. The manifest key lives on the **arrangement
|
||||
entry**, not at the top level.
|
||||
|
||||
```yaml
|
||||
arrangements:
|
||||
- id: keys
|
||||
name: Keys
|
||||
type: piano
|
||||
notation: notation_keys.json # per-arrangement sub-key
|
||||
# file: is optional when notation: is present
|
||||
```
|
||||
|
||||
```text
|
||||
my-song.sloppak/
|
||||
├── manifest.yaml
|
||||
├── song_timeline.json
|
||||
├── notation_keys.json
|
||||
└── stems/
|
||||
└── full.ogg
|
||||
```
|
||||
|
||||
**`notation_<id>.json` — file schema:**
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"instrument": "piano",
|
||||
"staves": [
|
||||
{"id": "rh", "clef": "G2", "label": "Right Hand"},
|
||||
{"id": "lh", "clef": "F4", "label": "Left Hand"}
|
||||
],
|
||||
"measures": [
|
||||
{
|
||||
"idx": 1,
|
||||
"t": 0.0,
|
||||
"ts": [4, 4],
|
||||
"ks": 0,
|
||||
"tempo": 120.0,
|
||||
"staves": {
|
||||
"rh": {
|
||||
"voices": [{"v": 1, "beats": [
|
||||
{"t": 0.000, "dur": 4, "notes": [{"midi": 64}]},
|
||||
{"t": 0.500, "dur": 4, "notes": [{"midi": 67}]}
|
||||
]}]
|
||||
},
|
||||
"lh": {
|
||||
"voices": [{"v": 1, "beats": [
|
||||
{"t": 0.000, "dur": 1, "notes": [{"midi": 52}, {"midi": 60}]}
|
||||
]}]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Top-level fields:**
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `version` | int | Always `1`. Bump on breaking schema change |
|
||||
| `instrument` | string | Mirrors arrangement `type`: `piano`, `violin`, `guitar`, etc. Makes the file self-describing |
|
||||
| `rights` | string | Optional copyright / rights text (MusicXML `<rights>`). Omit when absent |
|
||||
| `lyricist` | string | Optional lyricist credit (MusicXML `<creator type="lyricist">`). Omit when absent |
|
||||
| `arranger` | string | Optional arranger credit (MusicXML `<creator type="arranger">`). Omit when absent |
|
||||
| `staves` | list | Static staff definitions. Each has `id` (stable, referenced by `measures[].staves` keys), `clef` (see below), and optional `label` |
|
||||
| `measures` | list | Ordered measure data — the hot path |
|
||||
|
||||
**Clef vocabulary** (defined in `lib/notation.py::CLEFS`):
|
||||
|
||||
| Value | Meaning |
|
||||
|---|---|
|
||||
| `G2` | Treble clef — guitar, violin, flute, piano RH |
|
||||
| `F4` | Bass clef — bass guitar, cello, piano LH |
|
||||
| `C3` | Alto clef — viola |
|
||||
| `C4` | Tenor clef — cello upper register, trombone |
|
||||
| `neutral` | Unpitched / percussion staff |
|
||||
|
||||
**Measure fields:**
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `idx` | int | 1-based measure number |
|
||||
| `t` | float | Time in seconds at measure downbeat |
|
||||
| `ts` | int[2] | Time signature `[numerator, denominator]`. Omit if unchanged |
|
||||
| `beat_groups` | int[] | Beat grouping for compound and irregular meters, as a list of integers. Each integer is the count of time-signature denominator units in that primary beat group. The sum must equal the time-signature numerator. E.g. 6/8 → `[3, 3]`; 9/8 → `[3, 3, 3]`; 7/8 → `[2, 2, 3]`; 5/8 → `[2, 3]` or `[3, 2]`. Omit for simple meters (2/4, 3/4, 4/4) where grouping is unambiguous. Renderers translate this to their own beam-grouping API at render time — this field is renderer-agnostic. |
|
||||
| `ks` | int | Key signature: semitones from C, −7 to +7 (negative = flats, positive = sharps). Omit if unchanged |
|
||||
| `tempo` | float | BPM. Omit if unchanged |
|
||||
| `pickup` | bool | `true` when this measure is an anacrusis (pickup / upbeat) shorter than the time signature implies (MusicXML `implicit="yes"`). Renderers suppress the measure number and start counting from the next full measure. Omit when false |
|
||||
| `staves` | object | Keyed by staff `id`. Each staff has optional `clef` (omit if unchanged) and `voices` |
|
||||
|
||||
**Beat fields** (inside `staves → voices → beats`):
|
||||
|
||||
| Field | Default | Notes |
|
||||
|---|---|---|
|
||||
| `t` | required | Time in seconds |
|
||||
| `dur` | required | Duration denominator: `1`=whole, `2`=half, `4`=quarter, `8`=eighth, `16`=sixteenth, `32`=thirty-second |
|
||||
| `dot` | omit | Augmentation dots: `1`=dotted, `2`=double-dotted |
|
||||
| `rest` | omit | `true` if this beat is a rest; `notes` is omitted |
|
||||
| `tu` | omit | Tuplet: `[numerator, denominator]`, e.g. `[3, 2]` for triplet |
|
||||
| `beat_pos` | omit | Exact position within the measure as a rational `[numerator, denominator]` pair, where the denominator is the time-signature denominator. E.g. beat 2 in 6/8 (the second dotted quarter) = `[3, 8]`. Avoids floating-point imprecision when deriving beat position from tempo and absolute time. Omit if not set by the importer. Renderers that do not recognise this field derive position from `t` and the tempo map as before. |
|
||||
| `notes` | omit | List of note objects (omit for rests) |
|
||||
| `dyn` | omit | Dynamic: `ppp`, `pp`, `p`, `mp`, `mf`, `f`, `ff`, `fff` |
|
||||
| `slr` | omit | Slur start |
|
||||
| `slre` | omit | Slur end |
|
||||
| `grace` | omit | Grace-note beat, typed: `"a"` = acciaccatura (slashed, steals time from the previous note; MusicXML `<grace slash="yes">`), `"p"` = appoggiatura (unslashed, steals time from the following note; `<grace>`). The beat's `dur` is the grace note's written duration. Vocabulary in `lib/notation.py::GRACE_TYPES` |
|
||||
| `arp` | omit | `true` when the beat's chord is arpeggiated (rolled; MusicXML `<arpeggiate>`) |
|
||||
| `ferm` | omit | `true` when the beat carries a fermata (MusicXML `<fermata>`) |
|
||||
| `spd` / `sph` / `spu` | omit | Sustain pedal: pedal **d**own / **h**old-through-this-beat / **u**p. This is the only pedal encoding — there is deliberately no separate `ped` field. MusicXML mapping: `<pedal type="start">` → `spd`, `<pedal type="change">` → `spu` + `spd` on the same beat (re-pedal), `<pedal type="stop">` → `spu`; beats inside an active pedal span carry `sph` |
|
||||
| Additional beat effects | omit | `cre`, `dec`, `vib`, `vibw`, `fade`, `pm`, `lr`, `slap`, `pop`, `tap`, `su`, `sd`, `rasg`, `golpe`, `wah`, `txt`, `chrd` — all optional, omit when absent |
|
||||
|
||||
**Note fields** (inside `beats → notes`):
|
||||
|
||||
| Field | Default | Notes |
|
||||
|---|---|---|
|
||||
| `midi` | required | MIDI pitch 0–127. Unambiguous — no string/fret/tuning indirection |
|
||||
| `tied` | omit | Tied from the previous beat |
|
||||
| `acc` | omit | Accidental override: `null`/omit = derive from key sig; `0` = force natural (♮); `−2`/`−1`/`1`/`2` = double-flat/flat/sharp/double-sharp |
|
||||
| `stem` | omit | Force stem direction: `"up"` or `"down"` (MusicXML `<stem>`). Omit to let the renderer decide. Vocabulary in `lib/notation.py::STEM_DIRECTIONS` |
|
||||
| Additional note effects | omit | `stc`, `ten`, `ac`, `hac`, `vib`, `vibw`, `dead`, `ghost`, `fng`, `rfng`, `str`, `harm`, `bend`, `slide`, `trill`, `ho`, `po`, `tp`, `barre` — all optional |
|
||||
|
||||
**Wire format.** `song_info` carries `has_notation: bool`. Notation data
|
||||
is streamed as two highway-WS message types after `sections`, before `anchors`:
|
||||
|
||||
```json
|
||||
{"type": "notation_info", "version": 1, "instrument": "piano",
|
||||
"staves": [...], "total": 64}
|
||||
```
|
||||
|
||||
…followed by one or more chunks of 32 measures:
|
||||
|
||||
```json
|
||||
{"type": "notation_measures", "data": [...], "total": 64}
|
||||
```
|
||||
|
||||
`total` is the measure count across **all** chunks. Clients accumulate `data` arrays until the accumulated measure count reaches `total` (an individual chunk's `data.length` says nothing — every full chunk of a multi-chunk stream is shorter than `total`). The `anchors` frame that follows the notation block is a secondary end-of-block signal.
|
||||
|
||||
**`lib/notation.py`** is the vocabulary library: `SCHEMA_VERSION`, `CLEFS`, `DURATIONS`, `validate_notation()`, `measure_to_wire()`, `measures_to_wire()`.
|
||||
|
||||
**Legacy fallback.** Sloppaks that carry keys as guitar wire format (Clone Hero converted content) continue to work — the notation plugin checks for the `notation` key on the arrangement entry. When absent, it falls back to decoding guitar wire format notes via `midi = s * 24 + f`.
|
||||
|
||||
**v1 non-features (accepted limitations).** The following are deliberately
|
||||
out of schema v1; they ship, if ever, as **additive v1.x patches** (new
|
||||
optional fields old consumers ignore — the permissive validator passes
|
||||
unknown fields through by design):
|
||||
|
||||
- Microtonal pitch (anything finer than the ±2 semitone `acc` vocabulary).
|
||||
- Figured bass.
|
||||
- Mid-measure key-signature, time-signature, or clef changes (all three are
|
||||
measure-granular in v1).
|
||||
- Ottava lines (`ott`), repeat/volta barline semantics (`barline`),
|
||||
ornaments beyond trills (mordents, turns), tremolo (`trem`), and notated
|
||||
glissando lines (`glis`).
|
||||
|
||||
Importers MUST drop these source features with a logged warning rather than
|
||||
approximate them into wrong notation; renderers MUST NOT invent semantics
|
||||
for field names from this list before a v1.x patch specifies them.
|
||||
|
||||
---
|
||||
|
||||
#### Key / scale annotations (for theory-aware visualizations)
|
||||
|
||||
`keys.json` mirroring the `sections[]` shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"events": [
|
||||
{"t": 0.0, "key": "Em", "scale": "natural_minor"},
|
||||
{"t": 64.5, "key": "G", "scale": "major"},
|
||||
{"t": 142.0, "key": "Em", "scale": "natural_minor"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Manifest:
|
||||
|
||||
```yaml
|
||||
keys: keys.json
|
||||
```
|
||||
|
||||
Each entry implicitly applies until the next event. Same model as `sections[]`.
|
||||
|
||||
#### Vocal pitch contour (a different shape, a different key)
|
||||
|
||||
The canonical `vocal_pitch` key + file (defined in §2.4) is the
|
||||
per-syllable note format consumed by the karaoke plugin —
|
||||
`{version: 1, notes: [{t, d, midi}]}`. If you want to ship a finer-
|
||||
grained pitch *contour* (one sample every 20 ms, Hz instead of MIDI),
|
||||
that's a different shape and should ride on its own manifest key so
|
||||
the two don't collide:
|
||||
|
||||
```yaml
|
||||
vocal_pitch_contour: vocal_pitch_contour.json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"samples": [
|
||||
{"t": 0.000, "hz": 220.5},
|
||||
{"t": 0.020, "hz": 222.1}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Per §5.1, manifest keys are cheap — reach for a new one when the
|
||||
schema diverges, don't overload an existing key with a second shape.
|
||||
|
||||
### 5.4. `version` field — always include it
|
||||
|
||||
Every new file should have `"version": 1` at the top. It's free insurance: when you change the schema later, `version: 2` consumers can branch on it. Old consumers without that branch ignore the file (or fall back gracefully).
|
||||
|
||||
### 5.5. Stay backward-compatible
|
||||
|
||||
If you change a field that already shipped:
|
||||
|
||||
- **Adding fields** is always safe (older readers ignore them).
|
||||
- **Removing fields** breaks older readers. Don't.
|
||||
- **Repurposing fields** (changing meaning or units) is the worst — bump `version` and branch.
|
||||
|
||||
If you're tempted to remove or repurpose: leave the old field, add a new one, and sunset the old one over a release or two.
|
||||
|
||||
### 5.6. When to put data inside an arrangement vs. its own file
|
||||
|
||||
- **Inside arrangement JSON** (`arrangements/lead.json`):
|
||||
- Data that is *per-arrangement* and *per-instrument* (notes, chords, anchors, hand-shapes — guitar specifics).
|
||||
- Data that meaningfully differs between Lead and Rhythm versions of the same song.
|
||||
- **Its own file** (and pointed-at via manifest key):
|
||||
- Data that is *song-wide* (lyrics, beats, sections, tempo map, drum tab, lighting, key/scale changes).
|
||||
- Data that may be authored or generated independently of the playable arrangement (a stem split, an AI-generated drum tab).
|
||||
|
||||
Beats and sections historically lived inside the first arrangement JSON (early arrangement XML put them there). The `song_timeline.json` file (see §5.3) is the correct home for new sloppaks — the loader reads it first and it takes priority. New song-wide data should always be its own file.
|
||||
|
||||
### 5.7. Don't break the manifest contract
|
||||
|
||||
A few things that should *not* end up in `manifest.yaml`:
|
||||
|
||||
- **Per-machine settings** (DMX universes, IPs, output device picks) — those go in `${CONFIG_DIR}/...json`, not the sloppak.
|
||||
- **UI state** (last zoom level, panel sizes) — `localStorage` only.
|
||||
- **User progress / play counts** — Slopsmith stores these in its metadata DB, not in the sloppak.
|
||||
|
||||
The sloppak holds **the song's authored data**. Anything that varies by user or by machine is out.
|
||||
|
||||
---
|
||||
|
||||
## 6. Quick reference — file types you'll touch
|
||||
|
||||
| File | Format | Schema lives in | Authority |
|
||||
|---|---|---|---|
|
||||
| `manifest.yaml` | YAML | `lib/sloppak.py` (`load_manifest`, `extract_meta`) | This doc + the loader |
|
||||
| `arrangements/*.json` | JSON | `lib/song.py` (`arrangement_to_wire`, `arrangement_from_wire`) | The wire-format functions |
|
||||
| `lyrics.json` | JSON (flat list) | `lib/sloppak.py` (passed through to `Song.lyrics`) | This doc §2.3 |
|
||||
| `song_timeline.json` | JSON | `lib/sloppak.py` (loader) | This doc §5.3 |
|
||||
| `notation_<id>.json` | JSON | `lib/notation.py` (`validate_notation`, `measures_to_wire`) | This doc §5.3 |
|
||||
| `stems/*.ogg` | OGG Vorbis | — | Convention: `q:a 5` for size/quality balance |
|
||||
| `cover.jpg` | JPEG | — | Convention: square, 500–1500 px on a side |
|
||||
| Your new file | JSON (preferred) | Your plugin's spec doc | You |
|
||||
|
||||
---
|
||||
|
||||
## 7. Testing your extension
|
||||
|
||||
If you add a new file type or manifest key:
|
||||
|
||||
1. **Round-trip test**: write a sample, load it, write it back, compare. Add to `tests/test_sloppak.py`.
|
||||
2. **Backward-compat test**: load a sloppak that *doesn't* have your new key — your code must not crash, and the song must still play.
|
||||
3. **Hand-edit test**: open the directory form in a text editor, change a field by hand, reload Slopsmith. The format is meant to be hand-editable; your additions should preserve that.
|
||||
4. **Both forms**: test with both the directory form and the zipped form. The unpack cache is invalidated based on mtime and size, so you can repackage and reload without restarting the server.
|
||||
|
||||
The full pytest suite (`pytest`) must stay green before any PR.
|
||||
|
||||
---
|
||||
|
||||
## 8. Where to look in the code
|
||||
The spec is implementation-independent; this table is the feedback-specific bridge from format
|
||||
concepts to the code that reads and writes them. It is **not** part of the format.
|
||||
|
||||
| For… | Read |
|
||||
|---|---|
|
||||
| Format detection, source resolution, zip unpacking | [lib/sloppak.py](../lib/sloppak.py) |
|
||||
| Data classes (`Note`, `Chord`, `Arrangement`, `Song`, `Phrase`) | [lib/song.py](../lib/song.py) |
|
||||
| Wire-format helpers (`*_to_wire` / `*_from_wire`) | [lib/song.py](../lib/song.py) |
|
||||
| The reference sloppak writer | [lib/sloppak_convert.py](../lib/sloppak_convert.py) |
|
||||
| Drum tab vocabulary and wire helpers | [lib/drums.py](../lib/drums.py) |
|
||||
| The reference pack writer (assembly pipeline) | [lib/sloppak_convert.py](../lib/sloppak_convert.py) |
|
||||
| Drum-tab vocabulary and wire helpers | [lib/drums.py](../lib/drums.py) |
|
||||
| Notation vocabulary and wire helpers | [lib/notation.py](../lib/notation.py) |
|
||||
| Live streaming over WebSocket (consumes the same shapes) | `server.py` (`/ws/highway/{filename}`) |
|
||||
| The plugin system (where new viz consumers go) | [CLAUDE.md](../CLAUDE.md) — Plugin System section |
|
||||
| The plugin system (where new visualization consumers go) | [CLAUDE.md](../CLAUDE.md) |
|
||||
| Tests | [tests/test_sloppak.py](../tests/test_sloppak.py), [tests/test_sloppak_convert.py](../tests/test_sloppak_convert.py) |
|
||||
|
||||
> **Note on older section references.** Some inline code comments in this repo cite section
|
||||
> numbers from the previous version of this document (e.g. "sloppak-spec §5.3"). The external spec
|
||||
> renumbered its sections, so those citations are approximate — find the topic by name in the
|
||||
> [feedpak spec](https://github.com/got-feedback/feedback-feedpak-spec/blob/main/spec/feedpak-v1.md)
|
||||
> rather than by the old number.
|
||||
|
||||
@@ -1,203 +0,0 @@
|
||||
# Adding hit feedback to a visualization
|
||||
|
||||
A practical guide to making your custom visualization light up when notes are hit, dim when they're missed, and stay glowing while sustained notes are held correctly.
|
||||
|
||||
**Audience**: you're building a visualization plugin (per [`../CLAUDE.md`](../CLAUDE.md) § "Visualization plugins") and you want it to react to detected hits the same way the built-in 2D and 3D highways do.
|
||||
|
||||
**Prerequisites**
|
||||
- You can already get a `setRenderer`-style plugin loading. (If not, start with [`../CLAUDE.md`](../CLAUDE.md) — that contract is the entry point.)
|
||||
- You've skimmed [`realtime-scoring-pipeline.md`](realtime-scoring-pipeline.md). You don't need to understand the engine internals, just the shape of `bundle.getNoteState`.
|
||||
- `note_detect` is enabled in your test setup. Without it, the provider stays unregistered and `bundle.getNoteState` returns `null` for everything — your viz will work but you won't see feedback.
|
||||
|
||||
## The whole contract, in 5 lines
|
||||
|
||||
```js
|
||||
draw(bundle) {
|
||||
for (const n of bundle.notes) {
|
||||
const st = bundle.getNoteState(n, n.t); // for single notes, pass n.t
|
||||
// st is null | { state: 'hit' | 'active' | 'miss', alpha: 0..1, color: string|null }
|
||||
}
|
||||
for (const ch of bundle.chords) {
|
||||
for (const cn of ch.notes) {
|
||||
const st = bundle.getNoteState(cn, ch.t); // for chord constituents, pass CHORD time
|
||||
// …
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
That's it. The full semantics — when the provider fires, who owns fade timing, what the keys are — are in [`note-state-provider.md`](note-state-provider.md). This doc is about how to use them.
|
||||
|
||||
## Minimal 2D Canvas example
|
||||
|
||||
A complete `setRenderer`-style plugin that draws each chart note as a circle, lit up when hit, with a green halo for active sustains and a red wash for misses. ~80 lines, no dependencies. Drop it in `plugins/my_viz/screen.js` along with a matching `plugin.json` (`"type": "visualization"`).
|
||||
|
||||
```js
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
window.slopsmithViz_my_viz = function () {
|
||||
let ctx = null;
|
||||
let W = 0, H = 0;
|
||||
|
||||
return {
|
||||
contextType: '2d',
|
||||
|
||||
init(canvas, bundle) {
|
||||
ctx = canvas.getContext('2d');
|
||||
W = canvas.width;
|
||||
H = canvas.height;
|
||||
},
|
||||
|
||||
resize(w, h) {
|
||||
W = w; H = h;
|
||||
},
|
||||
|
||||
draw(bundle) {
|
||||
if (!ctx) return;
|
||||
ctx.clearRect(0, 0, W, H);
|
||||
|
||||
const now = bundle.currentTime;
|
||||
const stringCount = bundle.stringCount || 6;
|
||||
|
||||
// Project a chart time onto x. Notes ahead of `now` slide
|
||||
// in from the right; notes behind `now` slide off left.
|
||||
const xFor = (t) => W * 0.5 + (t - now) * 100;
|
||||
const yFor = (s) => H * (0.2 + 0.6 * (s / (stringCount - 1)));
|
||||
|
||||
// Single notes.
|
||||
for (const n of bundle.notes) {
|
||||
const x = xFor(n.t);
|
||||
if (x < -50 || x > W + 50) continue; // off-screen cull
|
||||
const y = yFor(n.s);
|
||||
const st = bundle.getNoteState(n, n.t);
|
||||
drawGem(ctx, x, y, n, st);
|
||||
}
|
||||
|
||||
// Chord constituents — pass CHORD time, not constituent time.
|
||||
for (const ch of bundle.chords) {
|
||||
const x = xFor(ch.t);
|
||||
if (x < -50 || x > W + 50) continue;
|
||||
for (const cn of ch.notes) {
|
||||
const y = yFor(cn.s);
|
||||
const st = bundle.getNoteState(cn, ch.t);
|
||||
drawGem(ctx, x, y, cn, st);
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
destroy() {
|
||||
ctx = null;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
function drawGem(ctx, x, y, note, st) {
|
||||
const r = 12;
|
||||
ctx.beginPath();
|
||||
ctx.arc(x, y, r, 0, Math.PI * 2);
|
||||
|
||||
if (!st) {
|
||||
// Dim — no judgment yet (or expired).
|
||||
ctx.fillStyle = '#445';
|
||||
ctx.fill();
|
||||
return;
|
||||
}
|
||||
|
||||
// Lit. `alpha` is provider-owned fade.
|
||||
const a = st.alpha;
|
||||
if (st.state === 'hit' || st.state === 'active') {
|
||||
ctx.fillStyle = st.color || `rgba(120, 220, 140, ${a})`;
|
||||
ctx.fill();
|
||||
// Halo grows with brightness — purely cosmetic.
|
||||
ctx.beginPath();
|
||||
ctx.arc(x, y, r + 6 * a, 0, Math.PI * 2);
|
||||
ctx.strokeStyle = `rgba(120, 220, 140, ${a * 0.5})`;
|
||||
ctx.lineWidth = 2;
|
||||
ctx.stroke();
|
||||
} else if (st.state === 'miss') {
|
||||
ctx.fillStyle = `rgba(220, 90, 90, ${a})`;
|
||||
ctx.fill();
|
||||
}
|
||||
}
|
||||
})();
|
||||
```
|
||||
|
||||
Things this example deliberately skips:
|
||||
|
||||
- **Sustain trails.** Add a `ctx.fillRect` from `xFor(n.t)` to `xFor(n.t + n.sus)` and switch its colour based on the `'active'` state.
|
||||
- **Inverted / lefty.** Check `bundle.inverted` and `bundle.lefty` and mirror your y-axis / x-axis transforms.
|
||||
- **Difficulty filter awareness.** `bundle.notes` is already filtered — you get this for free.
|
||||
- **Capo / tuning offsets.** Not needed for visualization; the gems just show the chart positions.
|
||||
|
||||
## Case study — how the 3D highway does it
|
||||
|
||||
[`../plugins/highway_3d/screen.js`](../plugins/highway_3d/screen.js) is the canonical custom renderer. It uses the same `bundle.getNoteState(note, chartTime)` call, but applied to Three.js materials.
|
||||
|
||||
The cliff notes (search the file for `getNoteState` to find each call site):
|
||||
|
||||
- For each visible note, the 3D highway calls `bundle.getNoteState(n, n.t)` (or `ch.t` for chord constituents).
|
||||
- A non-null return switches the gem's outline mesh from `mStr[s]` (dim, dull string colour) to `mGlow[s]` (bright emissive). The body mesh is swapped the same way.
|
||||
- Sustain trails use the bright material while `'active'`, the dim one otherwise.
|
||||
- A separate sparkle layer (`drawNotedetectSizzle()` on the 2D overlay canvas) queues a few crackling arcs and dots projected through the camera, fading with `st.alpha`. The 3D highway runs this on its 2D *overlay* canvas, not in the 3D scene, so the sparkles don't blow out the bloom or get hidden behind world geometry.
|
||||
- Chord-frame tint: when the chord-level entry (`${chord.t}_chord`) is `hit`, the chord-frame box tints green; on miss, it tints red. Per-constituent verdicts still drive the individual gem visuals so a partial-hit chord shows the frame green with individual constituents lit or dim.
|
||||
|
||||
Two implementation details worth copying:
|
||||
|
||||
1. The 3D highway caches `bundle.getNoteState` once per frame — `const getNs = bundle.getNoteState;` — and uses the cached ref for the inner loops. The function reference is stable across frames, so this is safe and saves a property read per gem.
|
||||
2. It null-guards every consumption (`if (st && st.state === 'hit')`) rather than relying on `st.state` to be defined. `null` is the common case — a single visible chord with 30 unstruck notes ahead of the playhead produces 30 null returns and 0 lit returns.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "I see no lit gems at all"
|
||||
|
||||
In order of likelihood:
|
||||
|
||||
1. **`note_detect` isn't running.** Open DevTools console and run:
|
||||
```js
|
||||
window.highway && window.highway.getNoteStateProvider()
|
||||
```
|
||||
If it returns `null`, no scorer is registered. Click the "Detect" button in the player controls (or enable it via your test fixture).
|
||||
2. **Your renderer isn't calling `bundle.getNoteState`.** Add a `console.log(st)` inside the loop. If you never see anything other than `null`, the provider is registered but you might be on a song with no scoring events yet (try strumming and watch the console).
|
||||
3. **You're passing the wrong `chartTime`.** Chord constituents need `chord.t`, not `cn.t`. Easy to miss if your inner loop reuses a variable named `t`.
|
||||
4. **Your renderer's `draw()` isn't firing.** Make sure the highway has had a `ready` message — `bundle.isReady` should be `true`. The factory gates `draw()` calls behind the ready flag.
|
||||
|
||||
### "Single notes light up but chord notes don't"
|
||||
|
||||
You're almost certainly passing `cn.t` instead of `ch.t` to `getNoteState`. The lookup map is keyed by the chord's time. Most chord constituents have the same `t` as the chord, but the contract still requires the chord time explicitly — and there are chart formats where constituent `t` values drift slightly.
|
||||
|
||||
### "Notes light up but never fade — they stay green forever"
|
||||
|
||||
The provider — not your renderer — owns fade timing. If you've replaced `note_detect` with your own provider and it returns a truthy state forever, gems stay lit forever. Return `null` when the effect should end. See [`note-state-provider.md`](note-state-provider.md) § "`alpha`" for how to encode a fade.
|
||||
|
||||
### "Sustains don't glow while held"
|
||||
|
||||
The provider needs to return `'active'` (not `'hit'`) for as long as the held pitch is detected. Check whether the producer in use (`note_detect` on desktop) is tracking sustain state — it does, via `_susActiveUntil`, but only when monophonic pitch detection is enabled and on-pitch within the configured cents tolerance. If your audio is noisy or the cents tolerance is tight, sustains may decay to `null` between frames.
|
||||
|
||||
### "My splitscreen panels all light up the same notes at the same time"
|
||||
|
||||
Two things might be going on:
|
||||
|
||||
1. **Two panels are scoring the same audio input.** Each `note_detect` instance currently shares the engine's single-source detection. See [issue #375](https://github.com/byrongamatos/slopsmith/issues/375) for the RFC on per-source detection. Until that lands, the fact that both panels light identically is expected.
|
||||
2. **Each panel's highway has its own provider slot.** If you're building a plugin that replaces the provider, register it on each panel's highway instance, not on a global. Splitscreen creates one highway per panel — `setNoteStateProvider` is per-highway.
|
||||
|
||||
### "My renderer prints `TypeError: bundle.getNoteState is not a function`"
|
||||
|
||||
The bundle field has been part of the contract since slopsmith#254. If `bundle.getNoteState` is missing, the host slopsmith you're running against predates that issue — you're on an old build. Check the version with `cat VERSION` in the slopsmith repo; anything ≥ 0.2 should have it. The field is set to a stable reference, never undefined.
|
||||
|
||||
### "I want to draw something on hit that isn't a gem (a particle burst, a sound, anything)"
|
||||
|
||||
Two options:
|
||||
|
||||
1. **Subscribe to events from inside your renderer.** `window.slopsmith.on('note:hit', judgment => { … })`. The judgment object has the chord/note info, timing/pitch error, etc. Fires once per detection, not per frame. Good for one-shots (sounds, screen shakes, achievement triggers).
|
||||
2. **Watch for state transitions in your draw loop.** Cache the last-seen state per note key in your renderer; when this frame's state changes from `null` to truthy, you know an event just landed. Good for cumulative effects (particle bursts that need to know their start time).
|
||||
|
||||
The provider doesn't itself give you an event stream — it gives you a sample-per-frame view of current state. Pick the right tool for your effect.
|
||||
|
||||
## Where to look next
|
||||
|
||||
- [`realtime-scoring-pipeline.md`](realtime-scoring-pipeline.md) — full data flow, top to bottom.
|
||||
- [`note-state-provider.md`](note-state-provider.md) — the producer contract (write your own scorer).
|
||||
- [`../CLAUDE.md`](../CLAUDE.md) — the canonical `setRenderer` / overlay contract.
|
||||
- [`../plugins/highway_3d/screen.js`](../plugins/highway_3d/screen.js) — case-study renderer.
|
||||
- [`note-detect-tuning.md`](note-detect-tuning.md) — what to do when detection itself is flaky (your viz works fine but the underlying hits / misses look wrong).
|
||||
@@ -6,11 +6,6 @@ The whole renderer is **one file** — `screen.js`, wrapped in an IIFE, register
|
||||
|
||||
**Styling (slopsmith `styles` capability).** This plugin owns its Tailwind CSS: it ships `assets/plugin.css` and declares `"styles": "assets/plugin.css"` in `plugin.json`, so core's prebuilt `static/tailwind.min.css` no longer scans it (it's excluded from core's content globs). The frontend injects `assets/plugin.css` as a `<link>` when the renderer activates. This is the one maintainer-time build step: after you add/change a Tailwind class in `screen.js` or `settings.html`, run `bash build-tailwind.sh` (pinned `tailwindcss@3.4.19`, `corePlugins.preflight=false` — utilities only) and **bump the `version` in `plugin.json`** so the injected `<link>`'s `?v=` cache-buster fetches the fresh file. The generated `assets/plugin.css` is committed; end users never build. See [docs/plugin-styles.md](../../docs/plugin-styles.md).
|
||||
|
||||
**For the contracts this plugin implements** (setRenderer, the note-state provider, the real-time scoring pipeline), see the repo-root docs:
|
||||
- [`../../docs/realtime-scoring-pipeline.md`](../../docs/realtime-scoring-pipeline.md) — end-to-end audio → lit gem trace. This plugin is one of the renderers in the final step.
|
||||
- [`../../docs/note-state-provider.md`](../../docs/note-state-provider.md) — the `bundle.getNoteState(note, chartTime)` contract this plugin consumes for hit/active/miss feedback.
|
||||
- [`../../docs/visualization-feedback-guide.md`](../../docs/visualization-feedback-guide.md) — practical "how do I add hit feedback to a viz" walkthrough, using this plugin as a case study.
|
||||
|
||||
> **Navigation note:** This guide references functions by name and uses the existing banner comments (`/* ── Scene initialisation ─ */`, etc.) as section anchors. Line numbers are deliberately avoided so this stays correct as the file evolves. Use `Grep` for the function name or banner text to jump to a section.
|
||||
|
||||
## File structure at a glance
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"id": "input_setup",
|
||||
"name": "Input Setup",
|
||||
"version": "0.1.0",
|
||||
"bundled": true,
|
||||
"private": false,
|
||||
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"],
|
||||
"script": "screen.js",
|
||||
"settings": { "html": "settings.html" },
|
||||
"description": "Per-instrument input-device selection and calibration, used during onboarding and re-launchable from Settings.",
|
||||
"category": "practice",
|
||||
"capabilities": {
|
||||
"input-calibration": {
|
||||
"roles": ["owner"],
|
||||
"commands": ["run", "status", "inspect"],
|
||||
"events": ["calibration-started", "calibration-done", "calibration-skipped"],
|
||||
"kind": "command",
|
||||
"mode": "active",
|
||||
"compatibility": "none",
|
||||
"ownership": "exclusive-owner",
|
||||
"safety": "safe",
|
||||
"description": "Owns the per-instrument input-setup wizard workflow; onboarding and Settings dispatch through the runtime.",
|
||||
"version": 1
|
||||
},
|
||||
"audio-input": {
|
||||
"roles": ["requester"],
|
||||
"requests": ["list-sources", "select-source", "open-source"],
|
||||
"mode": "active",
|
||||
"compatibility": "degrade-noop",
|
||||
"ownership": "requester-only",
|
||||
"safety": "sensitive",
|
||||
"description": "Picks the guitar/bass audio input device through the core audio-input domain.",
|
||||
"version": 1
|
||||
},
|
||||
"midi-input": {
|
||||
"roles": ["requester"],
|
||||
"requests": ["discover", "list-sources", "select-source", "open-source", "close-source"],
|
||||
"mode": "active",
|
||||
"compatibility": "degrade-noop",
|
||||
"ownership": "requester-only",
|
||||
"safety": "sensitive",
|
||||
"description": "Picks the keys/drums MIDI device through the core midi-input domain (Web-MIDI provider ships built-in with the domain).",
|
||||
"version": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,353 @@
|
||||
/*
|
||||
* input_setup — per-instrument input-device selection & calibration.
|
||||
*
|
||||
* Bundled core plugin (constitution P-II vanilla JS). It:
|
||||
* 1. supplies a Web-MIDI source provider to the core `midi-input` domain;
|
||||
* 2. owns the `input-calibration` capability domain (run / status / inspect);
|
||||
* 3. renders the onboarding input-setup wizard (one pass per instrument):
|
||||
* - guitar/bass → pick via `audio-input`, then launch note_detect's
|
||||
* Calibration Wizard (note-detection is a deferred surface — JS API);
|
||||
* - keys/drums → pick via `midi-input`, then a live "play a note /
|
||||
* hit a pad" confirmation.
|
||||
*
|
||||
* Idempotent (plugin-runtime-idempotent.v1): re-hydration is a no-op.
|
||||
*/
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
window.slopsmith = window.slopsmith || {};
|
||||
if (window.slopsmithInputSetup && window.slopsmithInputSetup.version === 1) return;
|
||||
|
||||
const capabilities = window.slopsmith.capabilities;
|
||||
const DONE_KEY = (inst) => `input_setup.done.${inst}`;
|
||||
const INSTRUMENTS = {
|
||||
guitar: { label: 'Guitar', mode: 'audio' },
|
||||
bass: { label: 'Bass', mode: 'audio' },
|
||||
keys: { label: 'Keys / Piano', mode: 'midi' },
|
||||
piano: { label: 'Keys / Piano', mode: 'midi' },
|
||||
drums: { label: 'Drums', mode: 'midi' },
|
||||
};
|
||||
|
||||
const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, (c) => (
|
||||
{ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c]));
|
||||
|
||||
function _isDone(inst) { try { return window.localStorage.getItem(DONE_KEY(inst)) === '1'; } catch (_) { return false; } }
|
||||
function _markDone(inst, v) { try { if (v) window.localStorage.setItem(DONE_KEY(inst), '1'); else window.localStorage.removeItem(DONE_KEY(inst)); } catch (_) { /* private mode */ } }
|
||||
|
||||
// The Web-MIDI source provider now ships built-in with the core midi-input
|
||||
// domain (static/capabilities/midi-input.js), so input_setup is a pure
|
||||
// consumer — it just discovers/selects/opens through `window.slopsmith.midiInput`.
|
||||
|
||||
// ── audio-input helper (guitar/bass device context) ─────────────────────
|
||||
async function _audioSources() {
|
||||
if (!capabilities || typeof capabilities.command !== 'function') return { sources: [], selected: null };
|
||||
try {
|
||||
const r = await capabilities.command('audio-input', 'list-sources', { requester: 'input_setup' });
|
||||
const p = (r && r.payload) || {};
|
||||
let sources = Array.isArray(p.sources) ? p.sources : [];
|
||||
// Exclude MIDI devices some plugins export into audio-input
|
||||
// (e.g. keys-highway-3d's pseudonymized 'midi-input-N'): they aren't
|
||||
// audio inputs and the cryptic labels confuse this guitar/bass picker.
|
||||
sources = sources.filter((s) => s
|
||||
&& !/midi/i.test(String(s.providerId || ''))
|
||||
&& !/^midi-input/i.test(String(s.label || '')));
|
||||
// De-dupe by display label — the desktop engine enumerates the same
|
||||
// device under several driver types, so the same name can repeat.
|
||||
const seen = new Set();
|
||||
sources = sources.filter((s) => {
|
||||
const key = String(s.label || '').toLowerCase();
|
||||
if (seen.has(key)) return false;
|
||||
seen.add(key);
|
||||
return true;
|
||||
});
|
||||
const selected = sources.find((s) => s && s.selected) || null;
|
||||
return { sources, selected };
|
||||
} catch (_) { return { sources: [], selected: null }; }
|
||||
}
|
||||
|
||||
// ── Wizard UI ────────────────────────────────────────────────────────────
|
||||
// Renders sequential per-instrument panels into `host`. Resolves the
|
||||
// returned promise to { completed:[...], skipped:[...] } when finished.
|
||||
function _runWizard(opts) {
|
||||
opts = opts || {};
|
||||
const instruments = (Array.isArray(opts.instruments) ? opts.instruments : [])
|
||||
.map((i) => String(i).toLowerCase()).filter((i) => INSTRUMENTS[i]);
|
||||
// De-dupe keys/piano (same MIDI flow under one label).
|
||||
const seen = new Set();
|
||||
const queue = instruments.filter((i) => { const k = INSTRUMENTS[i].label; if (seen.has(k)) return false; seen.add(k); return true; });
|
||||
|
||||
const completed = [];
|
||||
const skipped = [];
|
||||
let idx = 0;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
const host = opts.host;
|
||||
if (!host) { resolve({ completed, skipped }); return; }
|
||||
|
||||
function finish() {
|
||||
_emitOwner('calibration-done', { completed: completed.slice(), skipped: skipped.slice() });
|
||||
if (typeof opts.onComplete === 'function') { try { opts.onComplete({ completed, skipped }); } catch (_) {} }
|
||||
resolve({ completed, skipped });
|
||||
}
|
||||
// Per-panel teardown run on EVERY exit (Continue or the generic "Skip
|
||||
// for now"), so an opened MIDI session/listener never leaks past the
|
||||
// panel that opened it.
|
||||
let _activeCleanup = null;
|
||||
function next() {
|
||||
if (idx >= queue.length) { finish(); return; }
|
||||
renderPanel(queue[idx]);
|
||||
}
|
||||
function advance(inst, didComplete) {
|
||||
if (_activeCleanup) { try { _activeCleanup(); } catch (_) {} _activeCleanup = null; }
|
||||
if (didComplete) { _markDone(inst, true); if (!completed.includes(inst)) completed.push(inst); }
|
||||
else { if (!skipped.includes(inst)) skipped.push(inst); }
|
||||
idx += 1;
|
||||
next();
|
||||
}
|
||||
|
||||
function shell(inst, bodyHtml, footHtml) {
|
||||
const meta = INSTRUMENTS[inst];
|
||||
host.innerHTML =
|
||||
'<div class="space-y-4">' +
|
||||
'<div><div class="text-xs uppercase tracking-wider text-fb-textDim">Input setup — step ' + (idx + 1) + ' of ' + queue.length + '</div>' +
|
||||
'<h3 class="text-lg font-bold text-fb-text mt-0.5">Set up your ' + esc(meta.label) + '</h3></div>' +
|
||||
'<div data-is-body>' + bodyHtml + '</div>' +
|
||||
'<div class="flex justify-between items-center pt-1">' +
|
||||
'<button type="button" data-is-skip class="text-sm text-fb-textDim hover:text-fb-text">Skip for now</button>' +
|
||||
'<div data-is-foot>' + (footHtml || '') + '</div></div></div>';
|
||||
host.querySelector('[data-is-skip]').addEventListener('click', () => advance(inst, false));
|
||||
}
|
||||
|
||||
// ── per-instrument panels ───────────────────────────────────────
|
||||
async function renderPanel(inst) {
|
||||
const meta = INSTRUMENTS[inst];
|
||||
if (meta.mode === 'audio') return renderAudioPanel(inst);
|
||||
return renderMidiPanel(inst);
|
||||
}
|
||||
|
||||
// Guitar/bass: show the audio source (audio-input) and launch the
|
||||
// note_detect Calibration Wizard for the deep work.
|
||||
async function renderAudioPanel(inst) {
|
||||
const { sources, selected } = await _audioSources();
|
||||
const opts2 = sources.map((s) =>
|
||||
'<option value="' + esc(s.logicalSourceKey || s.sourceId || '') + '"' + (s.selected ? ' selected' : '') + '>' + esc(s.label || 'Input') + '</option>').join('');
|
||||
const hasDetector = !!(window.noteDetect && typeof window.noteDetect.launchCalibration === 'function');
|
||||
const body =
|
||||
'<p class="text-sm text-fb-textDim">Pick your audio input, then run the calibration to set levels, channel and latency.</p>' +
|
||||
(sources.length
|
||||
? '<label class="block text-xs uppercase tracking-wider text-fb-textDim mt-3 mb-1">Audio input</label>' +
|
||||
'<select data-is-audio class="w-full bg-gray-800/50 border border-gray-700 rounded-md px-2 py-1.5 text-sm text-fb-text outline-none">' + opts2 + '</select>'
|
||||
: '<p class="text-sm text-fb-accent mt-2">No audio input detected yet — plug in your interface, or skip and set this up later.</p>') +
|
||||
(hasDetector ? '' : '<p class="text-xs text-fb-textDim mt-3">The note detector isn’t loaded here — you can calibrate later from the player.</p>');
|
||||
const foot =
|
||||
'<button type="button" data-is-cal class="bg-fb-primary hover:bg-fb-primaryHi text-white px-5 py-2 rounded-md font-medium">' +
|
||||
(hasDetector ? 'Calibrate' : 'Continue') + '</button>';
|
||||
shell(inst, body, foot);
|
||||
|
||||
const sel = host.querySelector('[data-is-audio]');
|
||||
const commitAudio = (key) => {
|
||||
if (!capabilities || !key) return;
|
||||
capabilities.command('audio-input', 'select-source', { requester: 'input_setup', payload: { logicalSourceKey: key } }).catch(() => {});
|
||||
};
|
||||
if (sel) {
|
||||
sel.addEventListener('change', () => commitAudio(sel.value));
|
||||
// The <select> shows its first option by default, but no `change`
|
||||
// fires for that implicit pick — so on a first run with nothing yet
|
||||
// selected, audio-input would calibrate against the wrong/no source.
|
||||
// Commit the shown option up-front so the displayed device is the
|
||||
// one calibrated (idempotent if it was already selected).
|
||||
if (!selected) commitAudio(sel.value);
|
||||
}
|
||||
// Tell the tuner tables / note_detect which instrument this is.
|
||||
try { fetch('/api/settings', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ instrument: inst }) }); } catch (_) {}
|
||||
|
||||
host.querySelector('[data-is-cal]').addEventListener('click', () => {
|
||||
if (hasDetector) {
|
||||
window.noteDetect.launchCalibration({
|
||||
instrument: inst,
|
||||
onDone: () => advance(inst, true),
|
||||
onCancel: () => { /* stay on this panel; user can skip or retry */ },
|
||||
});
|
||||
} else {
|
||||
advance(inst, true);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Keys/drums: pick a MIDI device via midi-input and confirm a live hit.
|
||||
async function renderMidiPanel(inst) {
|
||||
const mi = window.slopsmith.midiInput;
|
||||
// Availability is the midi-input DOMAIN being present, not the
|
||||
// Web-MIDI browser API — the domain coordinates providers (the
|
||||
// built-in Web-MIDI one, plus any native/desktop adapter), so
|
||||
// gating on navigator.requestMIDIAccess would hide a usable
|
||||
// non-Web-MIDI provider before discover() is ever called.
|
||||
const midiAvailable = !!(mi && mi.version === 1);
|
||||
if (!midiAvailable) {
|
||||
shell(inst,
|
||||
'<p class="text-sm text-fb-accent">MIDI input isn’t available here. Connect a MIDI keyboard/e-kit in a supported environment, or skip for now.</p>',
|
||||
'<button type="button" data-is-skip2 class="bg-fb-primary hover:bg-fb-primaryHi text-white px-5 py-2 rounded-md font-medium">Continue</button>');
|
||||
host.querySelector('[data-is-skip2]').addEventListener('click', () => advance(inst, false));
|
||||
return;
|
||||
}
|
||||
const verb = inst === 'drums' ? 'hit a pad' : 'play a note';
|
||||
shell(inst,
|
||||
'<p class="text-sm text-fb-textDim">Connect your MIDI device, pick it below, then ' + verb + ' to confirm it’s working.</p>' +
|
||||
'<div class="mt-3 flex items-center gap-2">' +
|
||||
'<button type="button" data-is-scan class="text-sm text-fb-primary hover:text-fb-primaryHi">Scan for MIDI devices</button></div>' +
|
||||
'<div data-is-midi-wrap class="hidden mt-2">' +
|
||||
'<select data-is-midi class="w-full bg-gray-800/50 border border-gray-700 rounded-md px-2 py-1.5 text-sm text-fb-text outline-none"></select>' +
|
||||
'<p data-is-test class="text-sm text-fb-textDim mt-2">Waiting for input…</p></div>',
|
||||
'<button type="button" data-is-next disabled class="bg-fb-primary disabled:opacity-40 text-white px-5 py-2 rounded-md font-medium">Continue</button>');
|
||||
|
||||
const wrap = host.querySelector('[data-is-midi-wrap]');
|
||||
const select = host.querySelector('[data-is-midi]');
|
||||
const testEl = host.querySelector('[data-is-test]');
|
||||
const nextBtn = host.querySelector('[data-is-next]');
|
||||
let activeKey = null;
|
||||
let listener = null;
|
||||
let activeHandle = null;
|
||||
let openSeq = 0;
|
||||
|
||||
async function openSelected() {
|
||||
// Tear down the previous device + RESET the confirmation
|
||||
// state, so a hit on a prior device can't leave Continue
|
||||
// enabled for a newly-selected device that hasn't been heard.
|
||||
const myGen = ++openSeq;
|
||||
if (activeHandle && listener) { try { activeHandle.removeListener(listener); } catch (_) {} }
|
||||
if (activeKey) { try { mi.close({ requester: 'input_setup', logicalSourceKey: activeKey }); } catch (_) {} }
|
||||
activeHandle = null;
|
||||
listener = null;
|
||||
nextBtn.disabled = true;
|
||||
_markDone(inst, false);
|
||||
// Capture the requested key in a local: a newer openSelected()
|
||||
// overwrites the shared `activeKey`, so comparing it after the
|
||||
// awaits would let a stale open bind the wrong device.
|
||||
const requestedKey = select.value;
|
||||
activeKey = requestedKey;
|
||||
if (!requestedKey) { testEl.textContent = ''; return; }
|
||||
testEl.textContent = 'Waiting for input…';
|
||||
await mi.select(requestedKey);
|
||||
const res = await mi.open({ requester: 'input_setup', logicalSourceKey: requestedKey });
|
||||
// Discard a stale open if a newer openSelected() superseded us.
|
||||
if (myGen !== openSeq) { try { if (res) mi.close({ requester: 'input_setup', logicalSourceKey: requestedKey }); } catch (_) {} return; }
|
||||
if (!res || !res.handle) { testEl.textContent = 'Could not open this device.'; activeKey = null; return; }
|
||||
activeHandle = res.handle;
|
||||
listener = (data) => {
|
||||
// 0x90 = note-on (any channel); velocity > 0.
|
||||
if (data && (data[0] & 0xf0) === 0x90 && data[2] > 0) {
|
||||
testEl.innerHTML = '<span class="text-fb-primary font-semibold">✓ Got it</span> — device is working.';
|
||||
nextBtn.disabled = false;
|
||||
_markDone(inst, true);
|
||||
}
|
||||
};
|
||||
activeHandle.addListener(listener);
|
||||
}
|
||||
|
||||
host.querySelector('[data-is-scan]').addEventListener('click', async () => {
|
||||
await mi.discover();
|
||||
// Show every source the midi-input domain surfaces — not just
|
||||
// the built-in Web-MIDI provider — so a native/desktop MIDI
|
||||
// adapter registered with the domain is selectable too.
|
||||
const sources = window.slopsmith.midiInput.listSources() || [];
|
||||
if (!sources.length) { testEl && (testEl.textContent = ''); wrap.classList.remove('hidden'); select.innerHTML = '<option>No MIDI devices found</option>'; select.disabled = true; return; }
|
||||
wrap.classList.remove('hidden');
|
||||
select.disabled = false;
|
||||
select.innerHTML = sources.map((s) => '<option value="' + esc(s.logicalSourceKey) + '"' + (s.selected ? ' selected' : '') + '>' + esc(s.label) + '</option>').join('');
|
||||
openSelected();
|
||||
});
|
||||
select.addEventListener('change', openSelected);
|
||||
// Close the open session/listener on ANY exit (Continue or the
|
||||
// generic Skip), so a scanned+selected device doesn't keep its
|
||||
// Web-MIDI input live after the panel advances.
|
||||
_activeCleanup = () => {
|
||||
if (activeHandle && listener) { try { activeHandle.removeListener(listener); } catch (_) {} }
|
||||
if (activeKey) { try { mi.close({ requester: 'input_setup', logicalSourceKey: activeKey }); } catch (_) {} }
|
||||
activeHandle = null; listener = null; activeKey = null;
|
||||
};
|
||||
nextBtn.addEventListener('click', () => advance(inst, true));
|
||||
}
|
||||
|
||||
_emitOwner('calibration-started', { instruments: queue.slice() });
|
||||
next();
|
||||
});
|
||||
}
|
||||
|
||||
function _emitOwner(event, detail) {
|
||||
try { capabilities && capabilities.emitEvent && capabilities.emitEvent('input-calibration', event, detail || {}); } catch (_) {}
|
||||
}
|
||||
|
||||
// ── input-calibration owner domain ───────────────────────────────────────
|
||||
function _statusPayload(instruments) {
|
||||
const list = (Array.isArray(instruments) && instruments.length ? instruments : Object.keys(INSTRUMENTS))
|
||||
.map((i) => String(i).toLowerCase());
|
||||
const status = {};
|
||||
list.forEach((i) => { if (INSTRUMENTS[i]) status[i] = _isDone(i) ? 'done' : 'needs-setup'; });
|
||||
return status;
|
||||
}
|
||||
|
||||
if (capabilities && typeof capabilities.registerOwner === 'function') {
|
||||
capabilities.registerOwner('input-calibration', {
|
||||
pluginId: 'input_setup',
|
||||
kind: 'command',
|
||||
safety: 'safe',
|
||||
commands: ['run', 'status', 'inspect'],
|
||||
events: ['calibration-started', 'calibration-done', 'calibration-skipped'],
|
||||
description: 'Per-instrument input-setup wizard workflow (audio via audio-input + note_detect; MIDI via midi-input).',
|
||||
handlers: {
|
||||
inspect: () => ({ outcome: 'handled', payload: { available: true, status: _statusPayload() } }),
|
||||
status: (ctx) => ({ outcome: 'handled', payload: { status: _statusPayload((ctx.payload || {}).instruments) } }),
|
||||
// `run` is fire-and-launch: an interactive wizard far exceeds the
|
||||
// ~250ms handler timeout, so it starts the overlay and returns
|
||||
// immediately. Completion is signaled by the `calibration-done`
|
||||
// event (mirrors audio-monitoring `start`). A second `run` while
|
||||
// one is open is a no-op (single overlay).
|
||||
run: (ctx) => {
|
||||
const instruments = ((ctx.payload || {}).instruments) || [];
|
||||
if (!document.getElementById('input-setup-overlay')) launch(instruments);
|
||||
return { outcome: 'handled', payload: { started: true, instruments } };
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// ── public surface (onboarding + Settings re-entry) ──────────────────────
|
||||
function mount(container, options) {
|
||||
options = options || {};
|
||||
return _runWizard({ host: container, instruments: options.instruments || [], onComplete: options.onComplete, onSkip: options.onSkip });
|
||||
}
|
||||
|
||||
function launch(instruments) {
|
||||
const overlay = document.createElement('div');
|
||||
overlay.id = 'input-setup-overlay';
|
||||
overlay.className = 'fixed inset-0 z-[210] bg-black/60 backdrop-blur-sm flex items-center justify-center p-4';
|
||||
overlay.innerHTML = '<div class="bg-fb-card rounded-xl border border-fb-border/50 w-full max-w-lg p-6" data-is-host></div>';
|
||||
document.body.appendChild(overlay);
|
||||
const host = overlay.querySelector('[data-is-host]');
|
||||
return _runWizard({ host, instruments: instruments || [] }).then((r) => { overlay.remove(); return r; });
|
||||
}
|
||||
|
||||
window.slopsmithInputSetup = {
|
||||
version: 1,
|
||||
mount,
|
||||
launch,
|
||||
status: (instruments) => _statusPayload(instruments),
|
||||
};
|
||||
|
||||
// Settings-panel re-entry (settings.html "Set up input devices" button).
|
||||
// Re-runs the wizard for the player's selected instrument paths, falling
|
||||
// back to all instruments when progression isn't available.
|
||||
window._inputSetupRelaunch = async function () {
|
||||
let instruments = [];
|
||||
try {
|
||||
const r = await fetch('/api/progression');
|
||||
if (r.ok) {
|
||||
const d = await r.json();
|
||||
const paths = Array.isArray(d.paths) ? d.paths : [];
|
||||
instruments = paths.map((p) => (typeof p === 'string' ? p : (p && p.id))).filter(Boolean);
|
||||
}
|
||||
} catch (_) { /* offline — fall back below */ }
|
||||
if (!instruments.length) instruments = ['guitar', 'bass', 'keys', 'drums'];
|
||||
launch(instruments);
|
||||
};
|
||||
})();
|
||||
@@ -0,0 +1,16 @@
|
||||
<div class="space-y-4 py-2">
|
||||
<div class="bg-dark-900/50 p-3 rounded-xl border border-gray-800/50">
|
||||
<h3 class="text-sm font-medium text-gray-200">Input devices & calibration</h3>
|
||||
<p class="text-[11px] text-gray-500 mt-1">
|
||||
Re-run the input setup wizard for your instrument paths — pick your audio
|
||||
input or MIDI device and confirm it's working. Guitar/bass also opens the
|
||||
calibration wizard.
|
||||
</p>
|
||||
<button type="button"
|
||||
onclick="window._inputSetupRelaunch && window._inputSetupRelaunch()"
|
||||
class="mt-3 px-4 py-2 bg-accent hover:bg-accent-light text-white text-sm font-medium rounded-lg">
|
||||
Set up input devices
|
||||
</button>
|
||||
<p id="input-setup-settings-status" class="text-[11px] text-gray-500 mt-2"></p>
|
||||
</div>
|
||||
</div>
|
||||
@@ -0,0 +1,110 @@
|
||||
# Spec 012 — MIDI-Input Control-Plane Capability Domain
|
||||
|
||||
**Status:** active (control-plane slice) · **Issues:** #873 (impl), #880 (this spec) · **Base:** `release/v0.3.0`
|
||||
|
||||
## Summary
|
||||
|
||||
`midi-input` is a **core-owned provider-coordinator** capability domain for MIDI
|
||||
device discovery, selection, and open/close session lifecycle — the MIDI analog
|
||||
of `audio-input` (spec 006). It gives every MIDI consumer in Slopsmith (the
|
||||
`input_setup` onboarding wizard, the `piano`/keys and `drums` plugins, and — as
|
||||
a follow-up — note-detection's Web-MIDI provider) **one device-access boundary**:
|
||||
one permission prompt, one source list, one redaction boundary.
|
||||
|
||||
## Motivation
|
||||
|
||||
Today each MIDI consumer calls `navigator.requestMIDIAccess()` privately
|
||||
(piano, drums, plugin-midi, note-detection's `midi` provider kind), so there is
|
||||
no shared source list, no single permission prompt, and no common redaction of
|
||||
device labels. The onboarding input-setup step (#874/#876/#877) needs a single
|
||||
governed surface to pick and verify a MIDI device per instrument.
|
||||
|
||||
## Why not reuse `audio-input`
|
||||
|
||||
`audio-input`'s source/`source.open` contract is audio-frame-centric:
|
||||
`channelSummary`/`channelCount`/`channelShape`, `requiredChannelShape`, and
|
||||
redaction keyed to audio handles/buffers/samples. MIDI carries discrete messages
|
||||
and has no channel shape. Folding MIDI in would overload the audio contract and
|
||||
its redaction boundary. A sibling domain keeps both contracts clean and lets
|
||||
each evolve independently — the same reasoning that made `audio-input` and
|
||||
`audio-monitoring` siblings rather than one domain.
|
||||
|
||||
## Why core-owned (not plugin-owned)
|
||||
|
||||
An input control plane outlives any one feature; `audio-input` is
|
||||
`core.audio.session`-owned, not owned by a feature plugin. If `input_setup`
|
||||
owned `midi-input`, the domain's lifetime would be coupled to the wizard, and
|
||||
migrating ownership later (every consumer, persistence key, diagnostics schema
|
||||
references the owner) is costly. The domain is `core.midi-input`.
|
||||
|
||||
## Contract
|
||||
|
||||
- **Owner:** `core.midi-input`, kind `provider-coordinator`, safety `sensitive`.
|
||||
- **Public commands:** `inspect`, `list-sources`, `discover`, `select-source`,
|
||||
`open-source`, `close-source`.
|
||||
- **Provider operations:** `source.enumerate`, `source.describe`, `source.open`,
|
||||
`source.close`.
|
||||
- **Events:** `provider-registered`, `provider-unregistered`,
|
||||
`availability-changed`, `sources-changed`, `source-selected`, `source-opened`,
|
||||
`source-closed`.
|
||||
|
||||
### Sources & identity
|
||||
|
||||
Providers register source summaries with `providerId`, a stable `sourceId`, a
|
||||
derived **redaction-safe** `logicalSourceKey` (`providerId::sourceId`),
|
||||
`kind: "midi"`, a label, and `availability`. Persistence and diagnostics use the
|
||||
`logicalSourceKey`, never the human device label.
|
||||
|
||||
### Permission model (Web-MIDI nuance)
|
||||
|
||||
`requestMIDIAccess()` gates the **whole input list**, so **`discover` is the
|
||||
permission boundary** (not `open-source`, as it is for audio). `inspect` /
|
||||
`list-sources` / `select-source` are **prompt-free** and never request access.
|
||||
`discover` records `denied` / `unavailable` outcomes; `open-source` attaches a
|
||||
shared listener session to an already-discovered source and never re-prompts.
|
||||
|
||||
### Sessions
|
||||
|
||||
One shared open session per source across requesters (refcounted); the provider
|
||||
receives `source.close` only after the last requester releases. Live MIDI
|
||||
message delivery (for the "play a note / hit a pad" calibration check) is exposed
|
||||
to in-page consumers via the public `window.slopsmith.midiInput` session handle
|
||||
**only** — never as raw capability events or in diagnostics.
|
||||
|
||||
### Persistence & redaction
|
||||
|
||||
Selected source persists under `slopsmith.midiInput.selectedLogicalSourceKey`.
|
||||
Diagnostics (`slopsmith.midi_input.diagnostics.v1`) carry provider ids, source
|
||||
ids/keys/kinds/availability, the selected key, and open-session keys; device
|
||||
**labels are redacted** and **no raw MIDI messages** are ever included.
|
||||
|
||||
## Split from `midi-control`
|
||||
|
||||
The reserved `midi-control` domain is narrowed to **control mappings only**
|
||||
(CC/pitchbend/note → action routing) and will consume `midi-input` for device
|
||||
access. This spec carves out the device control plane so `midi-control` can stay
|
||||
mappings-only (#882).
|
||||
|
||||
## Consumers (separate issues)
|
||||
|
||||
- `input_setup` onboarding wizard — keys/drums device pick + verify (#876/#877).
|
||||
- `piano` / `drums` plugins — consume `midi-input` instead of private
|
||||
`requestMIDIAccess()` (via the sub-flow issues; legacy retired through bridges).
|
||||
- note-detection's Web-MIDI provider migrates onto `midi-input` (#881).
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Owner registers; appears in the Capability Inspector with the commands above.
|
||||
- `discover` is the only command that triggers `requestMIDIAccess()`;
|
||||
`inspect`/`list-sources`/`select-source` never prompt.
|
||||
- Selection persists across reload by `logicalSourceKey`.
|
||||
- Diagnostics contain no device labels or raw MIDI messages.
|
||||
- A consumer can `discover` → `select-source` → `open-source` → receive live
|
||||
note-on for the calibration check → `close-source` (session refcount releases).
|
||||
|
||||
## Out of scope (follow-ups)
|
||||
|
||||
- `midi-control` mapping/routing domain (#882).
|
||||
- note-detection provider migration onto `midi-input` (#881).
|
||||
- Retiring per-plugin `requestMIDIAccess()` in piano/drums via compatibility
|
||||
bridges (tracked with the sub-flow issues).
|
||||
@@ -0,0 +1,75 @@
|
||||
# Spec 013 — `midi-control` Mappings Domain (the midi-input/midi-control split)
|
||||
|
||||
**Status:** documented future contract (RESERVED — not in the runtime graph) ·
|
||||
**Issue:** #882 · **Depends on:** spec 012 (`midi-input`, delivered) · **Base:** `feedback/main`
|
||||
|
||||
## Summary
|
||||
|
||||
`midi-control` is the planned sibling of `midi-input`: it owns **MIDI control
|
||||
mappings** — routing CC / pitchbend / note messages to *semantic actions* (drum
|
||||
lane, transport command, effect parameter, etc.) — and **consumes `midi-input`**
|
||||
for device access. It does **not** discover, select, or open devices; that is
|
||||
`midi-input`'s job (spec 012, delivered).
|
||||
|
||||
This spec records the **split** so the boundary is unambiguous and the contract
|
||||
is ready for whoever builds the runtime slice. Per project governance
|
||||
(`docs/capability-safety-matrix.md`, `docs/capability-roadmap.md`), a future
|
||||
domain stays **documentation-only until a PR ships its host workflow, a concrete
|
||||
consumer, and tests** — so `midi-control` remains `RESERVED` in
|
||||
`static/capabilities.js` `RESERVED_FUTURE_DOMAINS` until then. This spec does not
|
||||
register a runtime domain.
|
||||
|
||||
## Why split it out
|
||||
|
||||
Before `midi-input` existed, "MIDI" meant two conflated concerns: getting bytes
|
||||
from a device, and mapping those bytes to actions. The reserved `midi-control`
|
||||
entry originally covered both. With `midi-input` delivered as the device control
|
||||
plane, `midi-control` is narrowed to **mappings only** — mirroring how
|
||||
`audio-input` (devices) is separate from `audio-effects`/`audio-mix` (what you do
|
||||
with the signal). Keeping them separate prevents a future god-domain and lets the
|
||||
device plane stabilize independently of mapping semantics.
|
||||
|
||||
## Boundary (normative)
|
||||
|
||||
- **`midi-input` owns:** device discovery (`discover`), source list, selection,
|
||||
open/close sessions, the Web-MIDI permission boundary, redacted device
|
||||
diagnostics. The raw MIDI message stream is delivered to in-page consumers via
|
||||
its session handle.
|
||||
- **`midi-control` will own:** named mappings from MIDI events (note / CC /
|
||||
pitchbend, optionally channel-scoped) to semantic actions, mapping persistence,
|
||||
active-mapping selection, and "learn" capture. It **consumes** a `midi-input`
|
||||
session for the live stream; it never calls `requestMIDIAccess` or enumerates
|
||||
devices.
|
||||
|
||||
## Proposed contract (for the future implementation slice)
|
||||
|
||||
- **Owner:** `core.midi-control` (or a first-party MIDI-control plugin),
|
||||
`multi-provider`, safety `sensitive`.
|
||||
- **Commands:** `list-mappings`, `get-mapping`, `set-mapping`, `delete-mapping`,
|
||||
`activate-mapping`, `inspect`.
|
||||
- **Mapping shape (sketch):** `{ id, label, trigger: { type: 'note'|'cc'|'pitchbend',
|
||||
number?, channel? }, action: { domain?, command?|actionId, params? } }`.
|
||||
- **Learn mode:** open a `midi-input` session, capture the next matching event,
|
||||
and bind it to the pending action (the per-plugin "learn" UIs in drums today
|
||||
are the reference behaviour to generalise).
|
||||
- **Diagnostics:** `slopsmith.midi_control.diagnostics.v1` — mapping summaries +
|
||||
bounded recent activations; **no raw MIDI streams, no device labels**.
|
||||
|
||||
## Intended consumers (promotion trigger)
|
||||
|
||||
The domain should be promoted out of RESERVED when a concrete consumer needs
|
||||
shared mappings, e.g.:
|
||||
- the generic **MIDI control plugin** (`feedback-plugin-midi`) — today an ad-hoc
|
||||
event→action mapper; the canonical first adopter.
|
||||
- **drums** note→lane mapping + "learn mode" (`feedback-plugin-drums`,
|
||||
`feedback-plugin-drum-highway-3d`) — currently per-plugin; could adopt
|
||||
`midi-control` to share mapping logic once the contract is proven.
|
||||
|
||||
Until such a consumer-driven slice exists (with host workflow + tests), this
|
||||
remains a documented contract only.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Any runtime registration / handlers (governance: no premature domain).
|
||||
- Migrating the drums/keys per-plugin mapping now — deferred to the consumer slice.
|
||||
- The device plane — owned by `midi-input` (spec 012, done).
|
||||
+27
-11
@@ -79,6 +79,19 @@
|
||||
const MAX_DECISIONS = 100;
|
||||
const MAX_SNAPSHOT_BYTES = 64 * 1024;
|
||||
const DEFAULT_HANDLER_TIMEOUT_MS = 250;
|
||||
// Per-(capability, command) handler-timeout overrides. A few commands front a
|
||||
// user-action / OS-permission prompt (fader reads that await a UI; MIDI/mic
|
||||
// device access) that legitimately runs far longer than the default budget,
|
||||
// so the dispatch/command surface must not fail them at 250 ms while the
|
||||
// operation is still completing through the provider.
|
||||
const COMMAND_TIMEOUTS_MS = {
|
||||
'audio-mix': { 'get-fader-value': 2100, 'set-fader-value': 2100 },
|
||||
'midi-input': { 'discover': 15000, 'open-source': 15000 },
|
||||
};
|
||||
function _commandTimeoutFor(capability, commandName) {
|
||||
const byCap = COMMAND_TIMEOUTS_MS[capability];
|
||||
return byCap ? byCap[commandName] : undefined;
|
||||
}
|
||||
const RESERVED_FUTURE_DOMAINS = new Set([
|
||||
'ui.navigation',
|
||||
'ui.plugin-screens',
|
||||
@@ -803,15 +816,18 @@
|
||||
}
|
||||
|
||||
function _withTimeout(promise, timeoutMs, participant) {
|
||||
return Promise.race([
|
||||
promise,
|
||||
new Promise(resolve => {
|
||||
setTimeout(() => resolve({
|
||||
outcome: 'failed',
|
||||
reason: `Handler ${participant.pluginId} timed out after ${timeoutMs} ms`,
|
||||
}), timeoutMs);
|
||||
}),
|
||||
]);
|
||||
// Capture + clear the timer once the race settles — otherwise a handler
|
||||
// that resolves first leaves a live setTimeout (up to timeoutMs) that
|
||||
// keeps the event loop alive and, for long overrides (MIDI permission
|
||||
// commands at 15s), accumulates delayed callbacks across repeated calls.
|
||||
let timer;
|
||||
const timeout = new Promise(resolve => {
|
||||
timer = setTimeout(() => resolve({
|
||||
outcome: 'failed',
|
||||
reason: `Handler ${participant.pluginId} timed out after ${timeoutMs} ms`,
|
||||
}), timeoutMs);
|
||||
});
|
||||
return Promise.race([promise, timeout]).finally(() => clearTimeout(timer));
|
||||
}
|
||||
|
||||
function _normalizeDecision(participant, result) {
|
||||
@@ -960,7 +976,7 @@
|
||||
if (typeof handler !== 'function') continue;
|
||||
let decision;
|
||||
try {
|
||||
const timeoutMs = Number(commandContext.timeoutMs || DEFAULT_HANDLER_TIMEOUT_MS);
|
||||
const timeoutMs = Number(commandContext.timeoutMs || _commandTimeoutFor(capabilityName, commandName) || DEFAULT_HANDLER_TIMEOUT_MS);
|
||||
const result = await _withTimeout(Promise.resolve(handler(commandContext)), timeoutMs, participant);
|
||||
decision = _normalizeDecision(participant, result);
|
||||
} catch (err) {
|
||||
@@ -1376,7 +1392,7 @@
|
||||
target: source.target || source.args?.target || null,
|
||||
payload: source.args || source.payload || {},
|
||||
claim: source.claim,
|
||||
timeoutMs: source.timeoutMs || (capability === 'audio-mix' && (commandName === 'get-fader-value' || commandName === 'set-fader-value') ? 2100 : undefined),
|
||||
timeoutMs: source.timeoutMs || _commandTimeoutFor(capability, commandName),
|
||||
});
|
||||
const status = _dispatchStatus(result);
|
||||
_emitEvent(capability, 'dispatched', { command: commandName, status, result, source: source.source || source.requester || 'dispatch' });
|
||||
|
||||
@@ -0,0 +1,408 @@
|
||||
// Core MIDI-input capability domain (spec 012 control plane).
|
||||
//
|
||||
// The MIDI analog of `audio-input`: a core-owned provider-coordinator over MIDI
|
||||
// device discovery, selection, and open/close session lifecycle. It is NOT
|
||||
// owned by any feature plugin (an input plane outlives any feature, exactly as
|
||||
// `audio-input` is `core.audio.session`-owned), and it is deliberately separate
|
||||
// from `audio-input` (whose source/open contract is audio-frame-centric:
|
||||
// channel shapes, sample buffers) — MIDI carries discrete messages, not audio.
|
||||
//
|
||||
// Consumers (the input_setup wizard, the piano/keys and drums plugins, and —
|
||||
// later — note-detection's Web-MIDI provider) converge on ONE device-access
|
||||
// boundary here: one permission prompt, one source list, one redaction
|
||||
// boundary, replacing private per-plugin `navigator.requestMIDIAccess()` calls.
|
||||
//
|
||||
// Web-MIDI nuance vs audio: `requestMIDIAccess()` gates the whole input LIST, so
|
||||
// `discover` (not `open-source`) is the permission boundary for MIDI. `inspect`
|
||||
// / `list-sources` / `select-source` stay prompt-free and never request access.
|
||||
//
|
||||
// Live message delivery (needed by the "play a note / hit a pad" calibration
|
||||
// check) is exposed to in-page consumers through the public global's session
|
||||
// handle, never as raw capability events or diagnostics.
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
window.slopsmith = window.slopsmith || {};
|
||||
const capabilities = window.slopsmith.capabilities;
|
||||
if (!capabilities || capabilities.version !== 1) return;
|
||||
if (window.slopsmith.midiInput && window.slopsmith.midiInput.version === 1) return;
|
||||
|
||||
const STORAGE_KEY = 'slopsmith.midiInput.selectedLogicalSourceKey';
|
||||
|
||||
// providerId → { id, label, participantId, handlers:{ enumerate, open, close } }
|
||||
// handlers are LIVE functions supplied in-page via the public global; they
|
||||
// never travel through the capability `command` payload.
|
||||
const providers = new Map();
|
||||
// logicalSourceKey → { sourceId, providerId, logicalSourceKey, kind, label, availability }
|
||||
const sources = new Map();
|
||||
// logicalSourceKey → { refs:Set<requester>, handle } — one shared open
|
||||
// session per source; the provider is closed only after the last release.
|
||||
const sessions = new Map();
|
||||
// logicalSourceKey → Promise — in-flight provider.open() calls, so concurrent
|
||||
// opens for the same source coalesce onto one provider session instead of each
|
||||
// calling provider.open() (which, for Web-MIDI, would overwrite the shared
|
||||
// input.onmidimessage handler and orphan the earlier session/handle).
|
||||
const opening = new Map();
|
||||
let selectedKey = _readStorage();
|
||||
let lastOutcome = null;
|
||||
|
||||
// ── outcome helpers (mirror note-detection.js) ──────────────────────────
|
||||
function _handled(payload = {}) { lastOutcome = { outcome: 'handled' }; return { outcome: 'handled', payload }; }
|
||||
function _degraded(reason, payload = {}) { lastOutcome = { outcome: 'degraded', reason }; return { outcome: 'degraded', reason, payload }; }
|
||||
function _denied(reason, payload = {}) { lastOutcome = { outcome: 'denied', reason }; return { outcome: 'denied', reason, payload }; }
|
||||
function _unavailable(reason, payload = {}) { lastOutcome = { outcome: 'unavailable', reason }; return { outcome: 'unavailable', reason, payload }; }
|
||||
|
||||
function _emit(name, detail) {
|
||||
try { capabilities.emitEvent('midi-input', name, detail || {}); }
|
||||
catch (_) { /* eventing must not break input */ }
|
||||
}
|
||||
|
||||
function _readStorage() {
|
||||
try { return window.localStorage.getItem(STORAGE_KEY) || null; }
|
||||
catch (_) { return null; }
|
||||
}
|
||||
function _writeStorage(key) {
|
||||
try { if (key) window.localStorage.setItem(STORAGE_KEY, key); else window.localStorage.removeItem(STORAGE_KEY); return true; }
|
||||
catch (_) { return false; }
|
||||
}
|
||||
|
||||
function _str(v, fallback) { const s = (v == null ? '' : String(v)).trim(); return s || fallback; }
|
||||
|
||||
// A stable, redaction-safe key for persistence: provider + a stable source
|
||||
// id, NOT the human device label.
|
||||
function _logicalKey(providerId, sourceId) { return `${providerId}::${sourceId}`; }
|
||||
|
||||
// ── snapshots ───────────────────────────────────────────────────────────
|
||||
// listShape keeps labels (this feeds the picker UI); diagShape strips them
|
||||
// (device labels are PII-adjacent).
|
||||
function _sourceListShape() {
|
||||
return Array.from(sources.values()).map((s) => ({
|
||||
logicalSourceKey: s.logicalSourceKey,
|
||||
sourceId: s.sourceId,
|
||||
providerId: s.providerId,
|
||||
kind: s.kind,
|
||||
label: s.label,
|
||||
availability: s.availability,
|
||||
selected: s.logicalSourceKey === selectedKey,
|
||||
open: sessions.has(s.logicalSourceKey),
|
||||
}));
|
||||
}
|
||||
|
||||
function _snapshot(extra = {}) {
|
||||
return {
|
||||
available: providers.size > 0,
|
||||
providers: Array.from(providers.values()).map((p) => ({ id: p.id, label: p.label })),
|
||||
sources: _sourceListShape(),
|
||||
selected: selectedKey,
|
||||
openSessions: Array.from(sessions.keys()),
|
||||
lastOutcome: lastOutcome ? { ...lastOutcome } : null,
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
function _contributeDiagnostics() {
|
||||
const diagnostics = window.slopsmith && window.slopsmith.diagnostics;
|
||||
if (!diagnostics || typeof diagnostics.contribute !== 'function') return;
|
||||
try {
|
||||
const snap = _snapshot();
|
||||
// Redact device labels everywhere — keep ids/kind/availability for
|
||||
// operational observability only. No raw MIDI messages ever.
|
||||
const redacted = {
|
||||
...snap,
|
||||
providers: snap.providers.map(({ label: _l, ...safe }) => safe),
|
||||
sources: snap.sources.map(({ label: _l, ...safe }) => safe),
|
||||
};
|
||||
diagnostics.contribute('midi-input-capability', {
|
||||
schema: 'slopsmith.midi_input.diagnostics.v1',
|
||||
...redacted,
|
||||
});
|
||||
} catch (_) { /* diagnostics must not break input */ }
|
||||
}
|
||||
|
||||
// ── provider registry (live handlers via the public global) ─────────────
|
||||
function _registerProvider(input = {}) {
|
||||
const providerId = _str(input.providerId || input.id, '');
|
||||
if (!providerId) return null;
|
||||
const handlers = {
|
||||
enumerate: typeof input.enumerate === 'function' ? input.enumerate : null,
|
||||
open: typeof input.open === 'function' ? input.open : null,
|
||||
close: typeof input.close === 'function' ? input.close : null,
|
||||
};
|
||||
const participantId = _str(input.participantId, providerId);
|
||||
const wasAvailable = providers.size > 0;
|
||||
providers.set(providerId, { id: providerId, label: _str(input.label, providerId), participantId, handlers });
|
||||
// Mirror a serializable declaration into the capability graph so the
|
||||
// Inspector/diagnostics can reason about the provider relationship.
|
||||
try {
|
||||
capabilities.registerParticipant(participantId, {
|
||||
'midi-input': {
|
||||
roles: ['provider'],
|
||||
operations: ['source.enumerate', 'source.describe', 'source.open', 'source.close'],
|
||||
mode: 'active',
|
||||
safety: 'sensitive',
|
||||
runtime: true,
|
||||
description: `MIDI input provider ${_str(input.label, providerId)}.`,
|
||||
provider_policy: { providerId },
|
||||
},
|
||||
});
|
||||
} catch (_) { /* declaration is best-effort */ }
|
||||
_emit('provider-registered', { providerId });
|
||||
if (!wasAvailable) _emit('availability-changed', { available: true });
|
||||
_contributeDiagnostics();
|
||||
return { providerId };
|
||||
}
|
||||
|
||||
function _unregisterProvider(providerId) {
|
||||
providerId = _str(providerId, '');
|
||||
const provider = providers.get(providerId);
|
||||
if (!provider) return false;
|
||||
// Drop the provider's sources + any open sessions.
|
||||
for (const [key, s] of Array.from(sources.entries())) {
|
||||
if (s.providerId === providerId) {
|
||||
_closeSessionInternal(key, 'provider-unregistered');
|
||||
sources.delete(key);
|
||||
}
|
||||
}
|
||||
providers.delete(providerId);
|
||||
if (typeof capabilities.unregisterParticipant === 'function') {
|
||||
try { capabilities.unregisterParticipant(provider.participantId, 'midi-input'); } catch (_) { /* best-effort */ }
|
||||
}
|
||||
_emit('provider-unregistered', { providerId });
|
||||
if (providers.size === 0) _emit('availability-changed', { available: false });
|
||||
_contributeDiagnostics();
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── discovery (the Web-MIDI permission boundary) ────────────────────────
|
||||
async function _discover() {
|
||||
if (providers.size === 0) return _unavailable('No MIDI provider registered', _snapshot());
|
||||
let found = 0;
|
||||
for (const provider of providers.values()) {
|
||||
if (!provider.handlers.enumerate) continue;
|
||||
let list;
|
||||
try { list = await provider.handlers.enumerate(); }
|
||||
catch (e) {
|
||||
// requestMIDIAccess rejection = permission denied / unsupported.
|
||||
return _denied(_str(e && e.message, 'MIDI access denied'), _snapshot());
|
||||
}
|
||||
const fresh = new Set();
|
||||
for (const raw of (Array.isArray(list) ? list : [])) {
|
||||
const sourceId = _str(raw.sourceId || raw.id, '');
|
||||
if (!sourceId) continue;
|
||||
const key = _logicalKey(provider.id, sourceId);
|
||||
fresh.add(key);
|
||||
sources.set(key, {
|
||||
sourceId,
|
||||
providerId: provider.id,
|
||||
logicalSourceKey: key,
|
||||
kind: 'midi',
|
||||
label: _str(raw.label || raw.name, 'MIDI input'),
|
||||
availability: _str(raw.availability, 'available'),
|
||||
});
|
||||
found += 1;
|
||||
}
|
||||
// Reconcile: drop this provider's sources that vanished since the
|
||||
// last enumeration (e.g. a device unplugged, firing statechange).
|
||||
// Without this, list-sources keeps showing disconnected devices and
|
||||
// selecting/opening them later fails on stale state.
|
||||
for (const [key, s] of Array.from(sources.entries())) {
|
||||
if (s.providerId === provider.id && !fresh.has(key)) {
|
||||
// Close any live session but KEEP the selectedKey preference
|
||||
// — the device may be replugged and should re-select.
|
||||
_closeSessionInternal(key, 'device-removed');
|
||||
sources.delete(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Restore a previously-selected source if it reappeared.
|
||||
if (selectedKey && !sources.has(selectedKey)) { /* keep the preference; it may return later */ }
|
||||
_emit('sources-changed', { count: found });
|
||||
_contributeDiagnostics();
|
||||
return _handled(_snapshot({ discovered: found }));
|
||||
}
|
||||
|
||||
function _selectSource(ctx = {}) {
|
||||
const payload = ctx.payload || {};
|
||||
const key = _str(payload.logicalSourceKey, '');
|
||||
if (!key) return _degraded('select-source requires a logicalSourceKey', _snapshot());
|
||||
if (!sources.has(key)) return _degraded(`Unknown MIDI source: ${key}`, _snapshot());
|
||||
selectedKey = key;
|
||||
_writeStorage(key);
|
||||
_emit('source-selected', { logicalSourceKey: key });
|
||||
_contributeDiagnostics();
|
||||
return _handled(_snapshot({ selected: key }));
|
||||
}
|
||||
|
||||
async function _openSource(ctx = {}) {
|
||||
const payload = ctx.payload || {};
|
||||
const requester = _str(ctx.source || ctx.requester || payload.requester, 'unknown');
|
||||
const key = _str(payload.logicalSourceKey, selectedKey || '');
|
||||
if (!key) return _degraded('No MIDI source selected', _snapshot());
|
||||
const source = sources.get(key);
|
||||
if (!source) return _degraded(`Unknown MIDI source: ${key}`, _snapshot());
|
||||
const provider = providers.get(source.providerId);
|
||||
if (!provider || !provider.handlers.open) return _unavailable('Provider cannot open MIDI input', _snapshot());
|
||||
|
||||
// Share one open session per source across requesters.
|
||||
let session = sessions.get(key);
|
||||
if (session) {
|
||||
session.refs.add(requester);
|
||||
return _handled(_snapshot({ sessionId: session.sessionId, shared: true }));
|
||||
}
|
||||
// Coalesce concurrent opens for the same source: if a provider.open() is
|
||||
// already in flight for this key, await it and adopt the resulting session
|
||||
// rather than opening the device a second time.
|
||||
if (opening.has(key)) {
|
||||
try { await opening.get(key); } catch (_) { /* fall through to retry below */ }
|
||||
session = sessions.get(key);
|
||||
if (session) {
|
||||
session.refs.add(requester);
|
||||
return _handled(_snapshot({ sessionId: session.sessionId, shared: true }));
|
||||
}
|
||||
}
|
||||
let handle;
|
||||
const openPromise = provider.handlers.open(source.sourceId, { requester });
|
||||
opening.set(key, openPromise);
|
||||
try { handle = await openPromise; }
|
||||
catch (e) { return _denied(_str(e && e.message, 'Could not open MIDI input'), _snapshot()); }
|
||||
finally { if (opening.get(key) === openPromise) opening.delete(key); }
|
||||
// A concurrent open may have won the race while we awaited; adopt its
|
||||
// session and release our redundant handle so we don't orphan a device.
|
||||
const existing = sessions.get(key);
|
||||
if (existing) {
|
||||
if (provider.handlers.close) {
|
||||
try { provider.handlers.close(source.sourceId, handle); } catch (_) { /* best-effort */ }
|
||||
}
|
||||
existing.refs.add(requester);
|
||||
return _handled(_snapshot({ sessionId: existing.sessionId, shared: true }));
|
||||
}
|
||||
session = { sessionId: `mis-${key}`, refs: new Set([requester]), handle };
|
||||
sessions.set(key, session);
|
||||
_emit('source-opened', { logicalSourceKey: key, requester });
|
||||
_contributeDiagnostics();
|
||||
return _handled(_snapshot({ sessionId: session.sessionId }));
|
||||
}
|
||||
|
||||
function _closeSessionInternal(key, reason) {
|
||||
const session = sessions.get(key);
|
||||
if (!session) return;
|
||||
const source = sources.get(key);
|
||||
const provider = source && providers.get(source.providerId);
|
||||
if (provider && provider.handlers.close) {
|
||||
try { provider.handlers.close(source.sourceId, session.handle); } catch (_) { /* best-effort */ }
|
||||
}
|
||||
sessions.delete(key);
|
||||
_emit('source-closed', { logicalSourceKey: key, reason: reason || 'closed' });
|
||||
}
|
||||
|
||||
function _closeSource(ctx = {}) {
|
||||
const payload = ctx.payload || {};
|
||||
const requester = _str(ctx.source || ctx.requester || payload.requester, 'unknown');
|
||||
const key = _str(payload.logicalSourceKey, selectedKey || '');
|
||||
const session = sessions.get(key);
|
||||
if (!session) return _handled(_snapshot({ closed: key, alreadyClosed: true }));
|
||||
session.refs.delete(requester);
|
||||
if (session.refs.size === 0) {
|
||||
_closeSessionInternal(key, 'released');
|
||||
_contributeDiagnostics();
|
||||
}
|
||||
return _handled(_snapshot({ closed: key }));
|
||||
}
|
||||
|
||||
capabilities.registerOwner('midi-input', {
|
||||
pluginId: 'core.midi-input',
|
||||
kind: 'provider-coordinator',
|
||||
safety: 'sensitive',
|
||||
commands: [
|
||||
'inspect', 'list-sources', 'discover',
|
||||
'select-source', 'open-source', 'close-source',
|
||||
],
|
||||
operations: ['source.enumerate', 'source.describe', 'source.open', 'source.close'],
|
||||
events: [
|
||||
'provider-registered', 'provider-unregistered', 'availability-changed',
|
||||
'sources-changed', 'source-selected', 'source-opened', 'source-closed',
|
||||
],
|
||||
description: 'Core-owned MIDI device control plane: discovery, selection, and shared open/close sessions. `discover` is the Web-MIDI permission boundary.',
|
||||
handlers: {
|
||||
inspect: () => _handled(_snapshot()),
|
||||
'list-sources': () => _handled(_snapshot()), // prompt-free; never requests access
|
||||
discover: (ctx) => _discover(ctx), // permission boundary
|
||||
'select-source': (ctx) => _selectSource(ctx), // prompt-free
|
||||
'open-source': (ctx) => _openSource(ctx),
|
||||
'close-source': (ctx) => _closeSource(ctx),
|
||||
},
|
||||
});
|
||||
|
||||
// ── public global (live surface for in-page consumers) ──────────────────
|
||||
// Providers register live handlers here; consumers (input_setup, piano,
|
||||
// drums) get a live session handle for the "play a note" check.
|
||||
window.slopsmith.midiInput = {
|
||||
version: 1,
|
||||
snapshot: _snapshot,
|
||||
listSources: () => _sourceListShape(),
|
||||
getSelected: () => selectedKey,
|
||||
registerProvider: _registerProvider,
|
||||
unregisterProvider: _unregisterProvider,
|
||||
discover: () => _discover(),
|
||||
select: (logicalSourceKey) => _selectSource({ payload: { logicalSourceKey } }),
|
||||
// Returns { outcome, sessionId, handle } where handle is the provider's
|
||||
// live MIDI input wrapper (exposes addListener/removeListener). The live
|
||||
// handle is surfaced ONLY through this in-page global, never through the
|
||||
// serializable `open-source` command payload. Use for calibration
|
||||
// note/pad checks.
|
||||
open: async (opts = {}) => {
|
||||
const result = await _openSource({ source: opts.requester || 'in-page', payload: opts });
|
||||
const key = _str(opts.logicalSourceKey, selectedKey || '');
|
||||
const session = sessions.get(key);
|
||||
return { ...result, handle: session ? session.handle : null, sessionId: session ? session.sessionId : null };
|
||||
},
|
||||
close: (opts = {}) => _closeSource({ source: opts.requester || 'in-page', payload: opts }),
|
||||
};
|
||||
|
||||
// ── built-in Web-MIDI provider ──────────────────────────────────────────
|
||||
// Ship a default Web-MIDI source provider so every consumer (piano, drums,
|
||||
// input_setup, …) gets MIDI devices from the domain without any one plugin
|
||||
// having to register the provider. Guarded by Web-MIDI support;
|
||||
// requestMIDIAccess() is the permission boundary, called lazily on discover.
|
||||
(function _registerBuiltinWebMidiProvider() {
|
||||
if (typeof navigator === 'undefined' || typeof navigator.requestMIDIAccess !== 'function') return;
|
||||
const BLOCK = /(midi through|thru|iac)/i; // loopback / passthrough ports
|
||||
let access = null;
|
||||
_registerProvider({
|
||||
providerId: 'web-midi',
|
||||
label: 'Web MIDI',
|
||||
// Distinct from the domain owner's participant id ('core.midi-input').
|
||||
// unregisterProvider() unregisters the provider's participant, so
|
||||
// sharing the owner's id would tear the whole domain's owner down on
|
||||
// a provider swap/hot-reload, leaving midi-input with no owner.
|
||||
participantId: 'core.midi-input.web-midi',
|
||||
enumerate: async () => {
|
||||
access = await navigator.requestMIDIAccess({ sysex: false });
|
||||
try { access.onstatechange = () => { _discover(); }; } catch (_) { /* best-effort */ }
|
||||
const out = [];
|
||||
access.inputs.forEach((input) => {
|
||||
if (BLOCK.test(input.name || '')) return;
|
||||
out.push({ sourceId: input.id, label: input.name || 'MIDI input', availability: 'available' });
|
||||
});
|
||||
return out;
|
||||
},
|
||||
open: async (sourceId) => {
|
||||
if (!access) access = await navigator.requestMIDIAccess({ sysex: false });
|
||||
const input = access.inputs.get(sourceId);
|
||||
if (!input) throw new Error('MIDI input not found');
|
||||
const listeners = new Set();
|
||||
input.onmidimessage = (e) => { listeners.forEach((fn) => { try { fn(e.data); } catch (_) { /* listener isolation */ } }); };
|
||||
return {
|
||||
addListener: (fn) => { if (typeof fn === 'function') listeners.add(fn); },
|
||||
removeListener: (fn) => listeners.delete(fn),
|
||||
_input: input,
|
||||
};
|
||||
},
|
||||
close: (sourceId, handle) => {
|
||||
if (handle && handle._input) { try { handle._input.onmidimessage = null; } catch (_) { /* best-effort */ } }
|
||||
},
|
||||
});
|
||||
})();
|
||||
|
||||
_contributeDiagnostics();
|
||||
})();
|
||||
@@ -31,6 +31,7 @@
|
||||
<script src="/static/capabilities/library-card-actions.js"></script>
|
||||
<script src="/static/capabilities/visualization.js"></script>
|
||||
<script src="/static/capabilities/note-detection.js"></script>
|
||||
<script src="/static/capabilities/midi-input.js"></script>
|
||||
</head>
|
||||
<body class="bg-dark-900 text-gray-200 font-display">
|
||||
|
||||
|
||||
+115
-32
@@ -1,7 +1,9 @@
|
||||
/* ── Consolidated tour menu — one floating button + popover for all tours ── */
|
||||
/* Palette aligned with the app's dark theme: #4080e0 accent, #e8c040 gold,
|
||||
#181830 dark background, #cbd5e1 / #94a3b8 / #64748b text scale. Per
|
||||
CLAUDE.md → Frontend Conventions. */
|
||||
/* Palette aligned with the v3 fee[dB]ack design tokens (tailwind.config.js):
|
||||
fb-card #1e293b surfaces, fb-cardMuted #0b1220 wells, fb-primary #0ea5e9
|
||||
accent, fb-border #334155 hairlines, fb-text #f8fafc / fb-textDim #94a3b8
|
||||
text, fb-gold #e8c040 badge. Plain CSS (no Tailwind classes) so it needs no
|
||||
stylesheet rebuild. Per CLAUDE.md → Frontend Conventions. */
|
||||
|
||||
.slopsmith-tour-menu-btn {
|
||||
position: fixed;
|
||||
@@ -14,20 +16,20 @@
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
border-radius: 50%;
|
||||
background: #181830;
|
||||
border: 1.5px solid #4080e0;
|
||||
color: #cbd5e1;
|
||||
background: #1e293b;
|
||||
border: 1.5px solid #0ea5e9;
|
||||
color: #f8fafc;
|
||||
font-size: 15px;
|
||||
font-weight: bold;
|
||||
cursor: pointer;
|
||||
box-shadow: 0 0 6px #4080e066;
|
||||
box-shadow: 0 0 6px #0ea5e966;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
transition: box-shadow 0.2s, transform 0.15s;
|
||||
}
|
||||
.slopsmith-tour-menu-btn:hover {
|
||||
box-shadow: 0 0 12px #4080e0aa;
|
||||
box-shadow: 0 0 12px #0ea5e9aa;
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
.slopsmith-tour-menu-btn.has-unseen {
|
||||
@@ -41,12 +43,12 @@
|
||||
width: 10px;
|
||||
height: 10px;
|
||||
background: #e8c040;
|
||||
border: 2px solid #181830;
|
||||
border: 2px solid #1e293b;
|
||||
border-radius: 50%;
|
||||
}
|
||||
@keyframes tour-pulse {
|
||||
0%, 100% { box-shadow: 0 0 6px #4080e066; }
|
||||
50% { box-shadow: 0 0 16px #4080e0cc; }
|
||||
0%, 100% { box-shadow: 0 0 6px #0ea5e966; }
|
||||
50% { box-shadow: 0 0 16px #0ea5e9cc; }
|
||||
}
|
||||
|
||||
.slopsmith-tour-menu-popover {
|
||||
@@ -62,12 +64,12 @@
|
||||
room for the trigger button + its bottom inset. */
|
||||
max-height: calc(100vh - 80px);
|
||||
overflow-y: auto;
|
||||
background: #181830;
|
||||
border: 1px solid #4080e044;
|
||||
background: #1e293b;
|
||||
border: 1px solid #0ea5e944;
|
||||
border-radius: 8px;
|
||||
padding: 6px;
|
||||
font-size: 13px;
|
||||
color: #cbd5e1;
|
||||
color: #f8fafc;
|
||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.5);
|
||||
}
|
||||
.slopsmith-tour-menu-popover .tour-menu-header {
|
||||
@@ -75,13 +77,13 @@
|
||||
font-size: 11px;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.08em;
|
||||
color: #64748b;
|
||||
border-bottom: 1px solid #1e1e3a;
|
||||
color: #94a3b8;
|
||||
border-bottom: 1px solid #334155;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.slopsmith-tour-menu-popover .tour-menu-empty {
|
||||
padding: 10px;
|
||||
color: #64748b;
|
||||
color: #94a3b8;
|
||||
font-style: italic;
|
||||
text-align: center;
|
||||
}
|
||||
@@ -95,23 +97,23 @@
|
||||
background: transparent;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
color: #cbd5e1;
|
||||
color: #f8fafc;
|
||||
font-size: 13px;
|
||||
text-align: left;
|
||||
cursor: pointer;
|
||||
transition: background 0.12s;
|
||||
}
|
||||
.slopsmith-tour-menu-popover .tour-menu-item:hover {
|
||||
background: #1a1a30;
|
||||
color: #fff;
|
||||
background: #334155;
|
||||
color: #f8fafc;
|
||||
}
|
||||
/* Keyboard focus gets an explicit ring instead of relying on the hover
|
||||
background — matches the :focus-visible treatment elsewhere in the
|
||||
app (style.css). Pointer focus is left alone. */
|
||||
.slopsmith-tour-menu-popover .tour-menu-item:focus-visible {
|
||||
background: #1a1a30;
|
||||
color: #fff;
|
||||
outline: 2px solid #4080e0;
|
||||
background: #334155;
|
||||
color: #f8fafc;
|
||||
outline: 2px solid #0ea5e9;
|
||||
outline-offset: -2px;
|
||||
}
|
||||
.slopsmith-tour-menu-popover .tour-menu-item-label {
|
||||
@@ -127,11 +129,11 @@
|
||||
}
|
||||
.slopsmith-tour-menu-popover .tour-menu-item-status.is-new {
|
||||
background: #e8c040;
|
||||
color: #181830;
|
||||
color: #0f172a;
|
||||
}
|
||||
.slopsmith-tour-menu-popover .tour-menu-item-status.is-seen {
|
||||
background: #1e1e3a;
|
||||
color: #64748b;
|
||||
background: #0b1220;
|
||||
color: #94a3b8;
|
||||
}
|
||||
|
||||
/* ── First-visit toast — anchored above the menu button ── */
|
||||
@@ -141,12 +143,12 @@
|
||||
bottom: 56px;
|
||||
right: 12px;
|
||||
z-index: 202;
|
||||
background: #181830;
|
||||
border: 1px solid #4080e044;
|
||||
background: #1e293b;
|
||||
border: 1px solid #0ea5e944;
|
||||
border-radius: 8px;
|
||||
padding: 10px 14px;
|
||||
font-size: 13px;
|
||||
color: #cbd5e1;
|
||||
color: #f8fafc;
|
||||
max-width: 240px;
|
||||
line-height: 1.4;
|
||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.5);
|
||||
@@ -159,7 +161,7 @@
|
||||
.slopsmith-tour-prompt .tour-prompt-more {
|
||||
margin-top: 4px;
|
||||
font-size: 11px;
|
||||
color: #64748b;
|
||||
color: #94a3b8;
|
||||
}
|
||||
.slopsmith-tour-prompt .tour-prompt-buttons {
|
||||
display: flex;
|
||||
@@ -178,11 +180,92 @@
|
||||
opacity: 0.85;
|
||||
}
|
||||
.slopsmith-tour-prompt button[data-action="start"] {
|
||||
background: #4080e0;
|
||||
background: #0ea5e9;
|
||||
color: #fff;
|
||||
font-weight: 600;
|
||||
}
|
||||
.slopsmith-tour-prompt button[data-action="dismiss"] {
|
||||
background: #1e1e3a;
|
||||
background: #334155;
|
||||
color: #94a3b8;
|
||||
}
|
||||
|
||||
/* ── Shepherd bubble theme — override the vendored light default ──────────── */
|
||||
/* The vendored static/vendor/shepherd.css ships Shepherd's stock LIGHT theme
|
||||
(white bubble, black text, blue buttons), which clashes with the dark v3 UI.
|
||||
These overrides load after it (index.html order) and recolor the spotlight
|
||||
bubbles to the fb-* tokens. The vendored file is left untouched so it stays
|
||||
upgradable. */
|
||||
.shepherd-element {
|
||||
background: #1e293b;
|
||||
border: 1px solid #334155;
|
||||
border-radius: 10px;
|
||||
box-shadow: 0 8px 28px rgba(0, 0, 0, 0.55);
|
||||
max-width: 360px;
|
||||
}
|
||||
.shepherd-content {
|
||||
background: #1e293b;
|
||||
border-radius: 10px;
|
||||
}
|
||||
.shepherd-text {
|
||||
color: #cbd5e1;
|
||||
font-size: 0.9rem;
|
||||
line-height: 1.45;
|
||||
padding: 0.85em 0.9em;
|
||||
}
|
||||
.shepherd-title {
|
||||
color: #f8fafc;
|
||||
font-size: 0.95rem;
|
||||
font-weight: 700;
|
||||
}
|
||||
/* Title row: drop the light-grey header fill the stock theme paints behind a
|
||||
titled step, so the header blends into the card. */
|
||||
.shepherd-has-title .shepherd-content .shepherd-header {
|
||||
background: transparent;
|
||||
padding: 0.85em 0.9em 0;
|
||||
}
|
||||
/* Arrow must match the bubble surface (stock paints it white, and grey behind
|
||||
a bottom-placed titled step). */
|
||||
.shepherd-arrow:before {
|
||||
background: #1e293b;
|
||||
}
|
||||
.shepherd-element.shepherd-has-title[data-popper-placement^="bottom"] > .shepherd-arrow:before {
|
||||
background-color: #1e293b;
|
||||
}
|
||||
.shepherd-footer {
|
||||
padding: 0 0.75rem 0.75rem;
|
||||
}
|
||||
/* Primary button (Next / Done) → fb-primary. */
|
||||
.shepherd-button {
|
||||
background: #0ea5e9;
|
||||
color: #fff;
|
||||
border-radius: 6px;
|
||||
font-weight: 600;
|
||||
padding: 0.4rem 1.1rem;
|
||||
}
|
||||
.shepherd-button:not(:disabled):hover {
|
||||
background: #38bdf8;
|
||||
color: #fff;
|
||||
}
|
||||
/* Secondary button (Back / Skip) → muted slate. */
|
||||
.shepherd-button.shepherd-button-secondary {
|
||||
background: #334155;
|
||||
color: #f8fafc;
|
||||
}
|
||||
.shepherd-button.shepherd-button-secondary:not(:disabled):hover {
|
||||
background: #475569;
|
||||
color: #f8fafc;
|
||||
}
|
||||
.shepherd-cancel-icon {
|
||||
color: #94a3b8;
|
||||
}
|
||||
.shepherd-cancel-icon:hover,
|
||||
.shepherd-has-title .shepherd-content .shepherd-cancel-icon:hover {
|
||||
color: #f8fafc;
|
||||
}
|
||||
.shepherd-has-title .shepherd-content .shepherd-cancel-icon {
|
||||
color: #94a3b8;
|
||||
}
|
||||
/* Dim the page a touch more, matching the onboarding overlay (bg-black/60). */
|
||||
.shepherd-modal-overlay-container.shepherd-modal-is-visible {
|
||||
opacity: 0.6;
|
||||
}
|
||||
|
||||
+24
-1
@@ -127,7 +127,13 @@
|
||||
}
|
||||
|
||||
function _unseenRelevant(screenId) {
|
||||
return _relevantPlugins(screenId).filter(p => !hasSeen(p.id) && !hasDismissed(p.id));
|
||||
// Drives the toast prompt + button "has-unseen" pulse. A tour registered
|
||||
// with autoPrompt:false is excluded — it's started programmatically by
|
||||
// its owner (e.g. the first-run home tour), so nagging via toast/pulse
|
||||
// would double up. It still lists in the menu and runs on demand.
|
||||
return _relevantPlugins(screenId).filter(p =>
|
||||
!hasSeen(p.id) && !hasDismissed(p.id) &&
|
||||
(_registry[p.id] ? _registry[p.id].autoPrompt !== false : true));
|
||||
}
|
||||
|
||||
// ── Menu UI ────────────────────────────────────────────────────────────
|
||||
@@ -615,7 +621,24 @@
|
||||
onStart: opts.onStart || null,
|
||||
onComplete: opts.onComplete || null,
|
||||
screens: Array.isArray(opts.screens) ? opts.screens.slice() : null,
|
||||
// autoPrompt:false opts the tour OUT of the unseen toast + button
|
||||
// pulse (it's driven programmatically by its owner, e.g. the
|
||||
// first-run home tour started from onboarding). It still lists in the
|
||||
// menu and runs via start(). Defaults to the legacy always-prompt.
|
||||
autoPrompt: opts.autoPrompt !== false,
|
||||
};
|
||||
// A `name` registers a CLIENT/CORE-owned tour — one that isn't a
|
||||
// server-discovered plugin with a tour.json — into the consolidated menu
|
||||
// catalog so it appears in the "?" menu. Never clobber a richer entry the
|
||||
// /api/plugins pass already supplied for a real plugin of the same id.
|
||||
if (opts.name && !_tourPlugins[pluginId]) {
|
||||
_tourPlugins[pluginId] = {
|
||||
id: pluginId,
|
||||
name: opts.name,
|
||||
has_screen: true,
|
||||
is_viz: false,
|
||||
};
|
||||
}
|
||||
// If the override changes the relevance for the current screen, refresh.
|
||||
_updateMenuVisibility();
|
||||
}
|
||||
|
||||
+1
-1
@@ -291,7 +291,7 @@
|
||||
const host = document.getElementById('v3-badge-instrument');
|
||||
if (!host) return;
|
||||
host.innerHTML =
|
||||
'<div class="relative">' +
|
||||
'<div id="v3-instrument-wrap" class="relative">' +
|
||||
'<button type="button" data-inst-toggle title="Instrument: ' + esc(settings.string_count + '-str ' + tuningLabel()) + '" ' +
|
||||
'class="bg-fb-card border border-fb-border/50 rounded-2xl h-[92px] w-16 flex flex-col items-center justify-center gap-2 hover:ring-1 hover:ring-fb-primary/40 transition">' +
|
||||
guitarIcon +
|
||||
|
||||
@@ -134,7 +134,7 @@
|
||||
const bars = Array.from({ length: segs }, (_, i) =>
|
||||
'<span class="flex-1 h-1.5 rounded-full ' + (i < filled ? 'bg-fb-primary' : 'bg-gray-500/40') + '"></span>').join('');
|
||||
continueCard =
|
||||
'<button id="v3-continue" class="group relative text-left rounded-xl overflow-hidden border border-fb-border/50 bg-fb-card aspect-square self-start flex flex-col justify-end">' +
|
||||
'<button id="v3-continue" data-tour="continue" class="group relative text-left rounded-xl overflow-hidden border border-fb-border/50 bg-fb-card aspect-square self-start flex flex-col justify-end">' +
|
||||
songArt(cont.art_url, 'absolute inset-0 w-full h-full object-cover opacity-60 group-hover:opacity-70 transition') +
|
||||
'<div class="absolute inset-0 bg-gradient-to-t from-black/90 via-black/40 to-transparent"></div>' +
|
||||
tuningChip(cont.tuning_name, 'absolute top-3 right-3') +
|
||||
@@ -146,7 +146,7 @@
|
||||
'<span class="absolute top-3 left-3 text-fb-text/80 group-hover:text-fb-text">▶</span></button>';
|
||||
} else if (pick) {
|
||||
continueCard =
|
||||
'<button id="v3-pick" data-fn="' + esc(pick.filename) + '" class="group relative text-left rounded-xl overflow-hidden border border-fb-border/50 bg-fb-card aspect-square self-start flex flex-col justify-end">' +
|
||||
'<button id="v3-pick" data-tour="continue" data-fn="' + esc(pick.filename) + '" class="group relative text-left rounded-xl overflow-hidden border border-fb-border/50 bg-fb-card aspect-square self-start flex flex-col justify-end">' +
|
||||
songArt(libArtUrl(pick), 'absolute inset-0 w-full h-full object-cover opacity-60 group-hover:opacity-70 transition') +
|
||||
'<div class="absolute inset-0 bg-gradient-to-t from-black/90 via-black/40 to-transparent"></div>' +
|
||||
tuningChip(pick.tuning_name, 'absolute top-3 right-3') +
|
||||
@@ -157,7 +157,7 @@
|
||||
'<span class="absolute top-3 left-3 text-fb-text/80 group-hover:text-fb-text">▶</span></button>';
|
||||
} else {
|
||||
continueCard =
|
||||
'<div class="rounded-xl border border-fb-border/50 bg-fb-card/60 aspect-square self-start flex flex-col items-center justify-center text-center p-4">' +
|
||||
'<div data-tour="continue" class="rounded-xl border border-fb-border/50 bg-fb-card/60 aspect-square self-start flex flex-col items-center justify-center text-center p-4">' +
|
||||
'<div class="text-fb-textDim text-sm mb-3">Pick a song to get started</div>' +
|
||||
'<button id="v3-continue-pick" class="bg-fb-card hover:bg-fb-card/70 border border-fb-border/50 text-fb-text text-sm px-4 py-2 rounded-md">Browse library</button></div>';
|
||||
}
|
||||
@@ -188,7 +188,7 @@
|
||||
'<a href="' + esc(changelogUrl) + '" target="_blank" rel="noopener" class="text-fb-primary hover:text-fb-primaryHi">Patch Notes for ' + esc(ver) + '</a>?</p>' : '') +
|
||||
// Featured grid: hero + continue
|
||||
'<div class="grid lg:grid-cols-3 gap-6 mt-6">' +
|
||||
'<div class="lg:col-span-2 relative rounded-xl overflow-hidden min-h-[480px] flex items-center bg-fb-bg">' +
|
||||
'<div id="v3-hero" class="lg:col-span-2 relative rounded-xl overflow-hidden min-h-[480px] flex items-center bg-fb-bg">' +
|
||||
// Hero artwork (neon note-highway), right-anchored. Placeholder
|
||||
// cropped from the design mock — swap static/v3/brand/hero.png for
|
||||
// the designer's high-res original (same path) when available.
|
||||
|
||||
@@ -94,6 +94,7 @@
|
||||
<script src="/static/capabilities/library-card-actions.js"></script>
|
||||
<script src="/static/capabilities/visualization.js"></script>
|
||||
<script src="/static/capabilities/note-detection.js"></script>
|
||||
<script src="/static/capabilities/midi-input.js"></script>
|
||||
</head>
|
||||
<body class="h-screen flex overflow-hidden bg-fb-sidebar text-fb-text font-display">
|
||||
|
||||
@@ -874,6 +875,10 @@
|
||||
<script src="/static/v3/songs.js"></script>
|
||||
<script src="/static/v3/lessons.js"></script>
|
||||
<script src="/static/v3/dashboard.js"></script>
|
||||
<!-- First-run home tour: spotlights the home cards via the shared tour
|
||||
engine (tour-engine.js, loaded above). Auto-runs once after onboarding
|
||||
(triggered from profile.js finish()); replayable from the "?" menu. -->
|
||||
<script src="/static/v3/onboarding-tour.js"></script>
|
||||
<script src="/static/v3/feedbarcade.js"></script>
|
||||
<script src="/static/v3/player-chrome.js"></script>
|
||||
<script>
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
/*
|
||||
* fee[dB]ack v0.3.0 — first-run home tour.
|
||||
*
|
||||
* Registers a spotlight tour over the home-page cards with the shared tour
|
||||
* engine (window.slopsmithTour / Shepherd) and auto-runs it once, the first
|
||||
* time the user lands on the home page after completing onboarding. It stays
|
||||
* replayable forever from the per-screen "?" tour menu (registered with
|
||||
* screens: ['v3-home']).
|
||||
*
|
||||
* Anchors (all stable, on-screen while #v3-home is active):
|
||||
* #v3-hero hero / Start Playing (dashboard.js)
|
||||
* [data-tour="continue"] continue / pick a song (dashboard.js, 3 variants)
|
||||
* #v3-instrument-wrap instrument selector badge (badges.js, topbar)
|
||||
* #v3-tuner-wrap tuner badge (badges.js, topbar)
|
||||
* #v3-audio-routing audio routing card (dashboard.js)
|
||||
* [data-v3-open-profile] profile badge (profile.js, topbar)
|
||||
* #v3-nav left sidebar navigation (index.html)
|
||||
*/
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
var TOUR_ID = 'home-onboarding';
|
||||
|
||||
// Each step dims the page and spotlights one element (shape: 'spotlight').
|
||||
// waitFor blocks the step until its target exists, so the async dashboard
|
||||
// re-render kicked off by 'v3:profile-updated' can't race the first step.
|
||||
function buildSteps() {
|
||||
return [
|
||||
{
|
||||
id: 'hero', shape: 'spotlight', position: 'bottom',
|
||||
selector: '#v3-hero', waitFor: '#v3-hero',
|
||||
title: 'Welcome to fee[dB]ack',
|
||||
content: 'This is your home base. Hit Start Playing to drop straight into a song from your library.',
|
||||
},
|
||||
{
|
||||
id: 'continue', shape: 'spotlight', position: 'left',
|
||||
selector: '[data-tour="continue"]', waitFor: '[data-tour="continue"]',
|
||||
title: 'Pick up where you left off',
|
||||
content: 'Your last song resumes right here in one click. Before you’ve played anything, it’s a quick random pick to get you going.',
|
||||
},
|
||||
{
|
||||
id: 'instrument', shape: 'spotlight', position: 'bottom',
|
||||
selector: '#v3-instrument-wrap', waitFor: '#v3-instrument-wrap',
|
||||
title: 'Choose your instrument',
|
||||
content: 'Set your instrument, string count and tuning here. The highway, tuner and scoring all adapt to this selection.',
|
||||
},
|
||||
{
|
||||
id: 'tuner', shape: 'spotlight', position: 'bottom',
|
||||
selector: '#v3-tuner-wrap', waitFor: '#v3-tuner-wrap',
|
||||
title: 'Tune up first',
|
||||
content: 'Open the tuner and match each string until the meter centers — accurate tuning means accurate scoring.',
|
||||
},
|
||||
{
|
||||
id: 'audio', shape: 'spotlight', position: 'top',
|
||||
selector: '#v3-audio-routing', waitFor: '#v3-audio-routing',
|
||||
title: 'Your signal path',
|
||||
content: 'Input → amp / NAM / IR → output, at a glance. Set up and monitor your gear from this card.',
|
||||
},
|
||||
{
|
||||
id: 'profile', shape: 'spotlight', position: 'bottom',
|
||||
selector: '[data-v3-open-profile]', waitFor: '[data-v3-open-profile]',
|
||||
title: 'Track your progress',
|
||||
content: 'Your profile, avatar and rank live here. Watch your accuracy climb and level up as you play.',
|
||||
},
|
||||
{
|
||||
id: 'nav', shape: 'spotlight', position: 'right',
|
||||
selector: '#v3-nav', waitFor: '#v3-nav',
|
||||
title: 'Find everything here',
|
||||
content: 'Browse your full library, lessons and plugins anytime. You can replay this tour from the ? button in the corner.',
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
function register() {
|
||||
var t = window.slopsmithTour;
|
||||
if (!t || typeof t.register !== 'function') return false;
|
||||
t.register(TOUR_ID, {
|
||||
name: 'Welcome tour', // label in the "?" tour menu
|
||||
screens: ['v3-home'], // relevant only on the home screen
|
||||
autoPrompt: false, // we auto-run it from onboarding; no toast nag
|
||||
buildSteps: buildSteps,
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
// Auto-run once after a genuine onboarding completion. profile.js gates the
|
||||
// call on !editing (a profile edit must not relaunch it); we additionally
|
||||
// honour the engine's own seen/dismissed state so it never repeats and a
|
||||
// dismissal isn't nagged. Stays replayable from the "?" menu either way.
|
||||
function startFirstRun() {
|
||||
var t = window.slopsmithTour;
|
||||
if (!t || typeof t.start !== 'function') return;
|
||||
try {
|
||||
if (t.hasSeen(TOUR_ID) || t.hasDismissed(TOUR_ID)) return;
|
||||
} catch (e) { /* private mode — fall through and attempt once */ }
|
||||
// Make sure the home screen is in view so the spotlight targets exist;
|
||||
// the per-step waitFor handles the async dashboard render.
|
||||
if (typeof window.showScreen === 'function') {
|
||||
try { window.showScreen('v3-home'); } catch (e) { /* best-effort */ }
|
||||
}
|
||||
// Defer a frame so the 'v3:profile-updated' dashboard re-render has a
|
||||
// chance to begin before Shepherd starts polling for the first target.
|
||||
var raf = window.requestAnimationFrame || function (fn) { return setTimeout(fn, 16); };
|
||||
raf(function () { try { t.start(TOUR_ID); } catch (e) { /* degrade */ } });
|
||||
}
|
||||
|
||||
window.v3OnboardingTour = { startFirstRun: startFirstRun };
|
||||
|
||||
// tour-engine.js assigns window.slopsmithTour at script-eval time, so if it
|
||||
// is loaded before us register() succeeds immediately; otherwise retry once
|
||||
// the DOM (and the engine) are ready.
|
||||
if (!register()) {
|
||||
document.addEventListener('DOMContentLoaded', register, { once: true });
|
||||
}
|
||||
})();
|
||||
+155
-18
@@ -153,6 +153,55 @@
|
||||
// diagnostic sloppak at 100% — or skip and reach Mastery Rank 1 anyway).
|
||||
// The profile POST always lands before the step-3 choice so onboarded=1 is
|
||||
// never blocked by the calibration decision. Editing keeps the single form.
|
||||
// Run the input-device setup wizard (the input_setup plugin's
|
||||
// `input-calibration` domain) for the chosen instrument paths — BETWEEN path
|
||||
// selection and the calibration challenge — so the diagnostic runs against a
|
||||
// calibrated input. Capability-idiomatic: dispatch `run` (fire-and-launch)
|
||||
// and await the `calibration-done` event. Degrades gracefully when the
|
||||
// plugin/runtime is absent so onboarding can never be stranded.
|
||||
// Wait (bounded) for the bundled input_setup plugin to finish registering
|
||||
// its input-calibration owner. Plugins load asynchronously, so onboarding
|
||||
// can reach this step before the plugin is ready — without this wait the
|
||||
// mandatory wizard is skipped by a load-order race (it dispatches, gets a
|
||||
// no-owner outcome, and falls through to the calibration challenge). The
|
||||
// public global is set at the end of the plugin's screen.js, after the
|
||||
// owner is registered, so it is a reliable readiness signal.
|
||||
function waitForInputSetup(timeoutMs) {
|
||||
const ready = () => !!(window.slopsmithInputSetup && typeof window.slopsmithInputSetup.launch === 'function');
|
||||
return new Promise((resolve) => {
|
||||
if (ready()) { resolve(true); return; }
|
||||
const t0 = Date.now();
|
||||
const iv = setInterval(() => {
|
||||
if (ready()) { clearInterval(iv); resolve(true); }
|
||||
else if (Date.now() - t0 >= timeoutMs) { clearInterval(iv); resolve(false); }
|
||||
}, 100);
|
||||
});
|
||||
}
|
||||
|
||||
async function runInputSetup(paths) {
|
||||
const instruments = (Array.isArray(paths) ? paths : []).map((p) => String(p).toLowerCase());
|
||||
if (!instruments.length) return;
|
||||
// Don't let a plugin-load race skip the mandatory input-setup step.
|
||||
if (!(await waitForInputSetup(8000))) return;
|
||||
const caps = window.slopsmith && window.slopsmith.capabilities;
|
||||
if (!caps || typeof caps.command !== 'function') {
|
||||
try { await window.slopsmithInputSetup.launch(instruments); } catch (e) { /* proceed */ }
|
||||
return;
|
||||
}
|
||||
await new Promise((resolve) => {
|
||||
let settled = false;
|
||||
let unsub = null;
|
||||
const done = () => { if (settled) return; settled = true; try { unsub && unsub(); } catch (e) { /* noop */ } resolve(); };
|
||||
try { unsub = typeof caps.subscribe === 'function' ? caps.subscribe('input-calibration:calibration-done', done) : null; } catch (e) { unsub = null; }
|
||||
// `run` is fire-and-launch; completion arrives via the event above.
|
||||
// A non-handled outcome (no owner / plugin absent / error) means
|
||||
// nothing was launched, so proceed immediately.
|
||||
caps.command('input-calibration', 'run', { requester: 'onboarding', payload: { instruments } })
|
||||
.then((r) => { if (!r || r.outcome !== 'handled') done(); })
|
||||
.catch(() => done());
|
||||
});
|
||||
}
|
||||
|
||||
function show(profile, opts) {
|
||||
opts = opts || {};
|
||||
const editing = !!opts.editing;
|
||||
@@ -160,7 +209,7 @@
|
||||
|
||||
const stepDots = editing ? '' :
|
||||
'<div class="flex justify-center gap-1.5 mt-3" id="v3-ob-dots">' +
|
||||
[1, 2, 3].map((n) => '<span data-dot="' + n + '" class="w-2 h-2 rounded-full bg-fb-border"></span>').join('') +
|
||||
[1, 2, 3, 4].map((n) => '<span data-dot="' + n + '" class="w-2 h-2 rounded-full bg-fb-border"></span>').join('') +
|
||||
'</div>';
|
||||
|
||||
const overlay = document.createElement('div');
|
||||
@@ -184,15 +233,24 @@
|
||||
'<button type="button" id="v3-ob-upload-btn" class="text-sm text-fb-primary hover:text-fb-primaryHi">Upload your own</button>' +
|
||||
'<input type="file" id="v3-ob-upload" accept="image/*" class="hidden">' +
|
||||
'<span id="v3-ob-preview"></span></div></div></div>' +
|
||||
// Step 2 — instrument paths (first-run only; tiles filled on entry).
|
||||
// Step 2 — song directory (where the user's songs live).
|
||||
'<div id="v3-ob-step2" class="hidden">' +
|
||||
'<label class="block text-xs uppercase tracking-wider text-fb-textDim mb-2">Song directory</label>' +
|
||||
'<p class="text-sm text-fb-textDim mb-3">Choose the folder where your songs are stored. We’ll scan it to build your library. You can change this later in Settings.</p>' +
|
||||
'<div class="flex gap-2">' +
|
||||
'<input id="v3-ob-songdir" type="text" placeholder="Path to your songs folder" ' +
|
||||
'class="flex-1 bg-gray-800/50 border border-gray-700 rounded-md px-3 py-2 text-sm text-fb-text outline-none focus:border-fb-primary focus:ring-1 focus:ring-fb-primary">' +
|
||||
'<button type="button" id="v3-ob-songdir-browse" class="hidden px-3 py-2 rounded-md text-sm bg-gray-800/50 border border-gray-700 text-fb-text hover:border-fb-primary whitespace-nowrap">Browse…</button>' +
|
||||
'</div></div>' +
|
||||
// Step 3 — instrument paths (first-run only; tiles filled on entry).
|
||||
'<div id="v3-ob-step3" class="hidden">' +
|
||||
'<label class="block text-xs uppercase tracking-wider text-fb-textDim mb-2">Pick your instrument path(s)</label>' +
|
||||
'<p class="text-sm text-fb-textDim mb-3">Each path levels up by completing challenges — together they make up your Mastery Rank. You can add more later.</p>' +
|
||||
'<div id="v3-ob-paths" class="grid grid-cols-3 gap-2"></div></div>' +
|
||||
// Step 3 — calibration offer (first-run only).
|
||||
'<div id="v3-ob-step3" class="hidden">' +
|
||||
// Step 4 — calibration offer (first-run only).
|
||||
'<div id="v3-ob-step4" class="hidden">' +
|
||||
'<label class="block text-xs uppercase tracking-wider text-fb-textDim mb-2">Calibration challenge</label>' +
|
||||
'<p class="text-sm text-fb-textDim">Prove your setup: play the <span class="text-fb-text">Slopsmith Diagnostic</span> with note detection and finish at <span class="text-fb-text font-semibold">100% accuracy</span> to reach <span class="text-fb-text font-semibold">Mastery Rank 1</span>.</p>' +
|
||||
'<p class="text-sm text-fb-textDim">Prove your setup: play the <span class="text-fb-text">fee[dB]ack Diagnostic</span> with note detection and finish at <span class="text-fb-text font-semibold">100% accuracy</span> to reach <span class="text-fb-text font-semibold">Mastery Rank 1</span>.</p>' +
|
||||
'<p class="text-sm text-fb-textDim mt-2">Not ready? Skip it and you’ll start at Rank 1 anyway — you can still play it later from the Progress screen.</p></div>' +
|
||||
'<p id="v3-ob-error" class="text-sm text-fb-accent hidden"></p>' +
|
||||
'<div class="flex justify-end gap-3">' +
|
||||
@@ -211,9 +269,12 @@
|
||||
const skipBtn = overlay.querySelector('#v3-ob-skip');
|
||||
let selected = null; // { type:'default', value } | { type:'upload', value:url }
|
||||
let step = 1; // first-run wizard step (editing stays on 1)
|
||||
let selectedPaths = []; // step-2 picks
|
||||
let songDir = ''; // step-2 song directory pick
|
||||
let selectedPaths = []; // step-3 picks
|
||||
let pathsAvailable = false; // any tiles rendered? (false → don't strand the user)
|
||||
let diagnosticFilename = null; // from /api/progression (step-3 "Play it now")
|
||||
let diagnosticFilename = null; // from /api/progression (step-4 "Play it now")
|
||||
const songDirEl = overlay.querySelector('#v3-ob-songdir');
|
||||
const songDirBrowse = overlay.querySelector('#v3-ob-songdir-browse');
|
||||
|
||||
if (editing && profile) {
|
||||
nameEl.value = profile.display_name || '';
|
||||
@@ -229,6 +290,10 @@
|
||||
const haveAvatar = !!selected || (editing && profile && !!profile.avatar_url);
|
||||
submit.disabled = !(nameEl.value.trim().length >= 1 && haveAvatar);
|
||||
} else if (step === 2) {
|
||||
// Song directory — require a non-empty path to proceed; "Skip
|
||||
// for now" is available for users who'll set it later.
|
||||
submit.disabled = !songDir.trim();
|
||||
} else if (step === 3) {
|
||||
// ≥1 path required — unless none could be offered (offline /
|
||||
// empty content), where blocking would strand onboarding.
|
||||
submit.disabled = pathsAvailable && selectedPaths.length < 1;
|
||||
@@ -240,7 +305,7 @@
|
||||
function setStep(n) {
|
||||
step = n;
|
||||
errEl.classList.add('hidden');
|
||||
for (let i = 1; i <= 3; i++) {
|
||||
for (let i = 1; i <= 4; i++) {
|
||||
overlay.querySelector('#v3-ob-step' + i).classList.toggle('hidden', i !== n);
|
||||
}
|
||||
overlay.querySelectorAll('#v3-ob-dots [data-dot]').forEach((d) => {
|
||||
@@ -250,11 +315,14 @@
|
||||
const subtitle = overlay.querySelector('#v3-ob-subtitle');
|
||||
if (subtitle) {
|
||||
subtitle.textContent = n === 1 ? 'Set up your player profile'
|
||||
: n === 2 ? 'Choose your instrument paths'
|
||||
: n === 2 ? 'Point us at your songs'
|
||||
: n === 3 ? 'Choose your instrument paths'
|
||||
: 'One last thing — calibrate your setup';
|
||||
}
|
||||
submit.textContent = n === 3 ? 'Play it now' : 'Next';
|
||||
skipBtn.classList.toggle('hidden', n !== 3);
|
||||
submit.textContent = n === 4 ? 'Play it now' : 'Next';
|
||||
// Skip is offered on the song-directory step (configure later) and
|
||||
// the calibration challenge.
|
||||
skipBtn.classList.toggle('hidden', !(n === 2 || n === 4));
|
||||
refreshSubmit();
|
||||
}
|
||||
|
||||
@@ -347,6 +415,43 @@
|
||||
|
||||
if (editing) overlay.querySelector('#v3-ob-cancel')?.addEventListener('click', () => overlay.remove());
|
||||
|
||||
// ── Song directory (step 2) ──────────────────────────────────────────
|
||||
if (songDirEl) {
|
||||
songDirEl.addEventListener('input', () => { songDir = songDirEl.value.trim(); refreshSubmit(); });
|
||||
}
|
||||
// Native folder picker on desktop; web users type/paste the path.
|
||||
const _desktop = window.slopsmithDesktop;
|
||||
if (songDirBrowse && _desktop && typeof _desktop.pickDirectory === 'function') {
|
||||
songDirBrowse.classList.remove('hidden');
|
||||
songDirBrowse.addEventListener('click', async () => {
|
||||
try {
|
||||
const picked = await _desktop.pickDirectory();
|
||||
if (picked) { songDirEl.value = picked; songDir = picked; refreshSubmit(); }
|
||||
} catch (e) { /* user cancelled / unavailable */ }
|
||||
});
|
||||
}
|
||||
// Save the song directory to settings + kick a library scan so the
|
||||
// user's songs appear. Throws (with a message) on an invalid folder.
|
||||
async function saveSongDir() {
|
||||
const dir = ((songDirEl && songDirEl.value) || '').trim();
|
||||
if (!dir) return; // skipped — leave unconfigured (settable later)
|
||||
const res = await fetch('/api/settings', {
|
||||
method: 'POST', headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ dlc_dir: dir }),
|
||||
});
|
||||
// /api/settings reports an invalid folder as a 200 with an `error`
|
||||
// field (a bare dict return, not a non-2xx status), so a res.ok-only
|
||||
// check would treat the failure as success and advance without saving.
|
||||
// Inspect the body too.
|
||||
let data = null;
|
||||
try { data = await res.json(); } catch (e) { /* non-JSON body */ }
|
||||
if (!res.ok || (data && data.error)) {
|
||||
throw new Error((data && data.error) || 'That folder couldn’t be set — check the path and try again.');
|
||||
}
|
||||
// Non-fatal: scan kicks off the library build in the background.
|
||||
try { await fetch('/api/rescan', { method: 'POST' }); } catch (e) { /* best-effort */ }
|
||||
}
|
||||
|
||||
async function postProfile() {
|
||||
const res = await fetch('/api/profile', {
|
||||
method: 'POST', headers: { 'Content-Type': 'application/json' },
|
||||
@@ -357,7 +462,8 @@
|
||||
return body;
|
||||
}
|
||||
|
||||
async function finish() {
|
||||
async function finish(finishOpts) {
|
||||
finishOpts = finishOpts || {};
|
||||
overlay.remove();
|
||||
await fetchProgress();
|
||||
if (window.v3Progression && typeof window.v3Progression.refresh === 'function') {
|
||||
@@ -366,6 +472,16 @@
|
||||
renderBadge();
|
||||
renderProfileScreen();
|
||||
if (window.slopsmith && window.slopsmith.emit) window.slopsmith.emit('v3:profile-updated', _profile);
|
||||
// First-run only: after a genuine onboarding completion (not a
|
||||
// profile edit), kick off the one-time home tour — but NOT when we're
|
||||
// about to launch the diagnostic ("Play it now"), which navigates to
|
||||
// the player; the tour would otherwise spotlight hidden home elements
|
||||
// and steal focus. The Skip path stays on home, so it runs there.
|
||||
// The engine's seen/dismissed state keeps it once; replayable from "?".
|
||||
if (!editing && !finishOpts.launchingSong &&
|
||||
window.v3OnboardingTour && typeof window.v3OnboardingTour.startFirstRun === 'function') {
|
||||
try { window.v3OnboardingTour.startFirstRun(); } catch (e) { /* never block onboarding */ }
|
||||
}
|
||||
}
|
||||
|
||||
submit.addEventListener('click', async () => {
|
||||
@@ -380,12 +496,23 @@
|
||||
}
|
||||
if (step === 1) {
|
||||
setStep(2);
|
||||
loadPathTiles();
|
||||
setTimeout(() => { try { songDirEl && songDirEl.focus(); } catch (e) { /* noop */ } }, 50);
|
||||
return;
|
||||
}
|
||||
if (step === 2) {
|
||||
// Save the song directory + kick a library scan, then continue
|
||||
// to instrument paths. "Skip for now" leaves it unconfigured.
|
||||
submit.disabled = true;
|
||||
try {
|
||||
await saveSongDir();
|
||||
setStep(3);
|
||||
loadPathTiles();
|
||||
} catch (e) { showErr(e.message || 'Could not set the song directory.'); refreshSubmit(); }
|
||||
return;
|
||||
}
|
||||
if (step === 3) {
|
||||
// Create the profile (onboarded=1) BEFORE the calibration choice
|
||||
// so closing the overlay at step 3 can never lose the profile.
|
||||
// so closing the overlay at the challenge can never lose the profile.
|
||||
submit.disabled = true;
|
||||
try {
|
||||
_profile = await postProfile();
|
||||
@@ -403,19 +530,29 @@
|
||||
throw new Error(msg);
|
||||
}
|
||||
}
|
||||
setStep(3);
|
||||
// New step: input-device selection + calibration, between
|
||||
// path selection and the note-detect calibration challenge.
|
||||
await runInputSetup(selectedPaths);
|
||||
setStep(4);
|
||||
} catch (e) { showErr(e.message || 'Could not save profile.'); refreshSubmit(); }
|
||||
return;
|
||||
}
|
||||
// Step 3 — "Play it now": leave calibration pending (it completes
|
||||
// Step 4 — "Play it now": leave calibration pending (it completes
|
||||
// through the normal scored-stats path) and launch the diagnostic.
|
||||
const target = diagnosticFilename;
|
||||
await finish();
|
||||
await finish({ launchingSong: !!target });
|
||||
if (target && typeof window.playSong === 'function') window.playSong(target);
|
||||
});
|
||||
|
||||
skipBtn.addEventListener('click', async () => {
|
||||
// Step 3 — skip: Mastery Rank 1 immediately, calibration stays
|
||||
// Step 2 — skip the song directory (the user can set it later in
|
||||
// Settings). Proceed straight to instrument paths.
|
||||
if (step === 2) {
|
||||
setStep(3);
|
||||
loadPathTiles();
|
||||
return;
|
||||
}
|
||||
// Step 4 — skip: Mastery Rank 1 immediately, calibration stays
|
||||
// replayable from the Progress screen.
|
||||
skipBtn.disabled = true;
|
||||
try {
|
||||
|
||||
@@ -103,7 +103,7 @@
|
||||
'<div class="flex items-center justify-between gap-3 flex-wrap">' +
|
||||
'<div class="min-w-0">' +
|
||||
'<h3 class="text-lg font-bold text-fb-text">Calibration challenge</h3>' +
|
||||
'<p class="text-sm text-fb-textDim mt-1">Play the <span class="text-fb-text">Slopsmith Diagnostic</span> with note detection and finish at ' +
|
||||
'<p class="text-sm text-fb-textDim mt-1">Play the <span class="text-fb-text">fee[dB]ack Diagnostic</span> with note detection and finish at ' +
|
||||
'<span class="text-fb-text font-semibold">100% accuracy</span>' +
|
||||
(pending ? ' to reach Mastery Rank 1.' : ' to prove your setup (you skipped this — rank already granted).') + '</p></div>' +
|
||||
'<div class="flex items-center gap-2 shrink-0">' +
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const vm = require('node:vm');
|
||||
const { createWindow, ROOT } = require('./capabilities_test_harness');
|
||||
|
||||
const CAPABILITIES_JS = path.join(ROOT, 'static', 'capabilities.js');
|
||||
const MIDI_INPUT_JS = path.join(ROOT, 'static', 'capabilities', 'midi-input.js');
|
||||
|
||||
function loadMidiInput(options = {}) {
|
||||
const window = createWindow(options);
|
||||
const context = vm.createContext(window);
|
||||
vm.runInContext(fs.readFileSync(CAPABILITIES_JS, 'utf8'), context, { filename: CAPABILITIES_JS });
|
||||
vm.runInContext(fs.readFileSync(MIDI_INPUT_JS, 'utf8'), context, { filename: MIDI_INPUT_JS });
|
||||
return window;
|
||||
}
|
||||
|
||||
// A fake provider whose enumerate/open/close are observable by the test.
|
||||
function fakeProvider(window, overrides = {}) {
|
||||
const calls = { enumerate: 0, open: [], close: [] };
|
||||
window.slopsmith.midiInput.registerProvider({
|
||||
providerId: 'web-midi',
|
||||
label: 'Web MIDI',
|
||||
participantId: 'input_setup',
|
||||
enumerate: async () => { calls.enumerate += 1; return overrides.sources || [{ sourceId: 'dev1', label: 'My Keyboard' }]; },
|
||||
open: async (sourceId) => { calls.open.push(sourceId); return { addListener() {}, removeListener() {}, _id: sourceId }; },
|
||||
close: (sourceId, handle) => { calls.close.push(sourceId); },
|
||||
...overrides.handlers,
|
||||
});
|
||||
return calls;
|
||||
}
|
||||
|
||||
test('midi-input registers an active sensitive provider-coordinator', () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
const pipeline = api.inspect('midi-input');
|
||||
assert.ok(pipeline, 'midi-input pipeline exists');
|
||||
const owner = (pipeline.participants || []).find(p => p.pluginId === 'core.midi-input');
|
||||
assert.ok(owner, 'core.midi-input owner registered');
|
||||
assert.equal(owner.safety, 'sensitive');
|
||||
assert.equal(owner.kind, 'provider-coordinator');
|
||||
for (const cmd of ['inspect', 'list-sources', 'discover', 'select-source', 'open-source', 'close-source']) {
|
||||
assert.ok(owner.commands.includes(cmd), `owner exposes ${cmd}`);
|
||||
}
|
||||
assert.equal(window.slopsmith.midiInput.version, 1);
|
||||
});
|
||||
|
||||
test('list-sources and select-source are prompt-free (never enumerate)', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
const calls = fakeProvider(window);
|
||||
const listed = await api.dispatch({ capability: 'midi-input', command: 'list-sources', source: 'tester' });
|
||||
assert.equal(listed.outcome, 'handled');
|
||||
assert.equal(calls.enumerate, 0, 'list-sources must not request MIDI access');
|
||||
});
|
||||
|
||||
test('discover is the permission boundary and surfaces sources', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
const calls = fakeProvider(window);
|
||||
const r = await api.dispatch({ capability: 'midi-input', command: 'discover', source: 'tester' });
|
||||
assert.equal(r.outcome, 'handled');
|
||||
assert.equal(calls.enumerate, 1, 'discover requests MIDI access exactly once');
|
||||
const sources = window.slopsmith.midiInput.listSources();
|
||||
assert.equal(sources.length, 1);
|
||||
assert.equal(sources[0].logicalSourceKey, 'web-midi::dev1');
|
||||
assert.equal(sources[0].kind, 'midi');
|
||||
});
|
||||
|
||||
test('re-discovery drops sources for devices that vanished', async () => {
|
||||
const window = loadMidiInput();
|
||||
let devices = [{ sourceId: 'dev1', label: 'A' }, { sourceId: 'dev2', label: 'B' }];
|
||||
window.slopsmith.midiInput.registerProvider({
|
||||
providerId: 'web-midi', label: 'Web MIDI',
|
||||
enumerate: async () => devices,
|
||||
open: async () => ({ addListener() {}, removeListener() {} }),
|
||||
close: () => {},
|
||||
});
|
||||
await window.slopsmith.midiInput.discover();
|
||||
assert.equal(window.slopsmith.midiInput.listSources().length, 2);
|
||||
devices = [{ sourceId: 'dev1', label: 'A' }]; // dev2 unplugged
|
||||
await window.slopsmith.midiInput.discover();
|
||||
const keys = window.slopsmith.midiInput.listSources().map((s) => s.logicalSourceKey);
|
||||
assert.equal(keys.length, 1, 'vanished device is dropped from the source list');
|
||||
assert.equal(keys[0], 'web-midi::dev1');
|
||||
});
|
||||
|
||||
test('discover with no provider reports unavailable', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
const r = await api.dispatch({ capability: 'midi-input', command: 'discover', source: 'tester' });
|
||||
assert.equal(r.outcome, 'unavailable');
|
||||
});
|
||||
|
||||
test('discover surfaces denied when MIDI access is rejected', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
fakeProvider(window, { handlers: { enumerate: async () => { throw new Error('SecurityError: permission denied'); } } });
|
||||
const r = await api.dispatch({ capability: 'midi-input', command: 'discover', source: 'tester' });
|
||||
assert.equal(r.outcome, 'denied');
|
||||
assert.match(r.reason, /denied/i);
|
||||
});
|
||||
|
||||
test('select-source persists by logicalSourceKey', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
fakeProvider(window);
|
||||
await api.dispatch({ capability: 'midi-input', command: 'discover', source: 'tester' });
|
||||
const sel = await api.dispatch({ capability: 'midi-input', command: 'select-source', source: 'tester', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
assert.equal(sel.outcome, 'handled');
|
||||
assert.equal(window.__storage.get('slopsmith.midiInput.selectedLogicalSourceKey'), 'web-midi::dev1');
|
||||
assert.ok(window.slopsmith.midiInput.listSources()[0].selected);
|
||||
});
|
||||
|
||||
test('open/close share one session and release on the last requester', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
const calls = fakeProvider(window);
|
||||
await window.slopsmith.midiInput.discover();
|
||||
await window.slopsmith.midiInput.select('web-midi::dev1');
|
||||
const a = await api.dispatch({ capability: 'midi-input', command: 'open-source', source: 'reqA', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
const b = await api.dispatch({ capability: 'midi-input', command: 'open-source', source: 'reqB', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
assert.equal(a.outcome, 'handled');
|
||||
assert.equal(b.outcome, 'handled');
|
||||
assert.equal(calls.open.length, 1, 'provider.open called once for a shared session');
|
||||
// First release keeps the session open; second closes it.
|
||||
await api.dispatch({ capability: 'midi-input', command: 'close-source', source: 'reqA', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
assert.equal(calls.close.length, 0, 'session stays open while a requester holds it');
|
||||
await api.dispatch({ capability: 'midi-input', command: 'close-source', source: 'reqB', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
assert.equal(calls.close.length, 1, 'provider.close after the last release');
|
||||
});
|
||||
|
||||
test('concurrent opens for one source coalesce onto a single provider.open', async () => {
|
||||
const window = loadMidiInput();
|
||||
const api = window.slopsmith.capabilities;
|
||||
// A provider whose open() stays pending until we release it, so both
|
||||
// dispatches are genuinely in flight at the same time.
|
||||
let release;
|
||||
const gate = new Promise((r) => { release = r; });
|
||||
const calls = { open: 0, close: 0 };
|
||||
window.slopsmith.midiInput.registerProvider({
|
||||
providerId: 'web-midi', label: 'Web MIDI',
|
||||
enumerate: async () => [{ sourceId: 'dev1', label: 'My Keyboard' }],
|
||||
open: async () => { calls.open += 1; await gate; return { addListener() {}, removeListener() {} }; },
|
||||
close: () => { calls.close += 1; },
|
||||
});
|
||||
await window.slopsmith.midiInput.discover();
|
||||
await window.slopsmith.midiInput.select('web-midi::dev1');
|
||||
const p1 = api.dispatch({ capability: 'midi-input', command: 'open-source', source: 'reqA', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
const p2 = api.dispatch({ capability: 'midi-input', command: 'open-source', source: 'reqB', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
release();
|
||||
const [a, b] = await Promise.all([p1, p2]);
|
||||
assert.equal(a.outcome, 'handled');
|
||||
assert.equal(b.outcome, 'handled');
|
||||
assert.equal(calls.open, 1, 'provider.open called exactly once despite concurrent opens');
|
||||
// Both requesters joined the single shared session: it survives the first
|
||||
// release and only closes on the last, with exactly one provider.close.
|
||||
await api.dispatch({ capability: 'midi-input', command: 'close-source', source: 'reqA', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
assert.equal(calls.close, 0, 'shared session stays open while reqB holds it');
|
||||
await api.dispatch({ capability: 'midi-input', command: 'close-source', source: 'reqB', payload: { logicalSourceKey: 'web-midi::dev1' } });
|
||||
assert.equal(calls.close, 1, 'provider.close once after the last requester releases');
|
||||
});
|
||||
|
||||
test('public open() surfaces the live handle (in-page only)', async () => {
|
||||
const window = loadMidiInput();
|
||||
fakeProvider(window);
|
||||
await window.slopsmith.midiInput.discover();
|
||||
await window.slopsmith.midiInput.select('web-midi::dev1');
|
||||
const res = await window.slopsmith.midiInput.open({ requester: 'input_setup', logicalSourceKey: 'web-midi::dev1' });
|
||||
assert.equal(res.outcome, 'handled');
|
||||
assert.ok(res.handle && typeof res.handle.addListener === 'function', 'live handle exposed via public global');
|
||||
});
|
||||
|
||||
// Load the domain with a Web-MIDI-capable navigator so the built-in provider
|
||||
// self-registers (the shared harness has no navigator, so it normally skips).
|
||||
function loadWithWebMidi(inputs) {
|
||||
const window = createWindow();
|
||||
window.navigator = {
|
||||
requestMIDIAccess: async () => ({
|
||||
onstatechange: null,
|
||||
inputs: new Map(inputs.map((i) => [i.id, { id: i.id, name: i.name, onmidimessage: null }])),
|
||||
}),
|
||||
};
|
||||
const context = vm.createContext(window);
|
||||
vm.runInContext(fs.readFileSync(CAPABILITIES_JS, 'utf8'), context, { filename: CAPABILITIES_JS });
|
||||
vm.runInContext(fs.readFileSync(MIDI_INPUT_JS, 'utf8'), context, { filename: MIDI_INPUT_JS });
|
||||
return window;
|
||||
}
|
||||
|
||||
test('built-in Web-MIDI provider self-registers + discovers, filtering loopback ports', async () => {
|
||||
const window = loadWithWebMidi([
|
||||
{ id: 'kb1', name: 'My Keyboard' },
|
||||
{ id: 'thru', name: 'Midi Through Port-0' }, // loopback → filtered out
|
||||
]);
|
||||
const api = window.slopsmith.capabilities;
|
||||
assert.ok(api.inspect('midi-input').participants.some(p => p.pluginId === 'core.midi-input'),
|
||||
'built-in provider registered without any plugin');
|
||||
const r = await api.dispatch({ capability: 'midi-input', command: 'discover', source: 'tester' });
|
||||
assert.equal(r.outcome, 'handled');
|
||||
const sources = window.slopsmith.midiInput.listSources();
|
||||
assert.equal(sources.length, 1, 'loopback/passthrough ports are filtered');
|
||||
assert.equal(sources[0].logicalSourceKey, 'web-midi::kb1');
|
||||
});
|
||||
|
||||
test('diagnostics are redaction-safe (no device labels, no raw messages)', async () => {
|
||||
const window = loadMidiInput();
|
||||
fakeProvider(window);
|
||||
await window.slopsmith.midiInput.discover();
|
||||
const contrib = window.slopsmith.diagnostics.snapshotContributions()['midi-input-capability'];
|
||||
assert.ok(contrib, 'midi-input contributes diagnostics');
|
||||
assert.equal(contrib.schema, 'slopsmith.midi_input.diagnostics.v1');
|
||||
const serialized = JSON.stringify(contrib);
|
||||
assert.ok(!serialized.includes('My Keyboard'), 'device labels are redacted from diagnostics');
|
||||
for (const s of contrib.sources) assert.ok(!('label' in s), 'source entries carry no label');
|
||||
});
|
||||
Reference in New Issue
Block a user