feat(onboarding): input-device setup step + core-owned midi-input domain (#526)

* feat(capabilities): add core-owned midi-input control-plane domain (#873, #880)

The MIDI analog of audio-input: a core-owned provider-coordinator over MIDI
device discovery, selection, and shared open/close sessions. Separate from
audio-input (whose source/open contract is audio-frame-centric) and not owned
by any feature plugin, so the device-access boundary outlives the input-setup
wizard. `discover` is the Web-MIDI permission boundary; selection persists by
redaction-safe logicalSourceKey; diagnostics redact device labels and never
carry raw MIDI messages.

- static/capabilities/midi-input.js + load-order wiring in both shells
- spec 012 + capability-domains/safety-matrix entries; midi-control narrowed
  to mappings-only (split)
- 9 domain tests against the real runtime

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(input_setup): bundled plugin owning input-calibration + Web-MIDI provider (#872)

Bundled core plugin that supplies the Web-MIDI source provider to the core
midi-input domain, owns the input-calibration workflow domain (run/status/
inspect), and renders the per-instrument wizard (guitar/bass -> audio-input +
note_detect; keys/drums -> midi-input live note/pad test). Idempotent
hydration; redaction-safe. .gitignore allowlists the in-tree plugin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(onboarding): input-device setup step between paths and calibration (#874)

After instrument-path selection and before the note-detect calibration
challenge, dispatch input-calibration `run` (fire-and-launch) and await the
`calibration-done` event. Fail-soft: a non-handled outcome (plugin/runtime
absent) advances immediately so onboarding can never be stranded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(midi-input): ship a built-in Web-MIDI provider in the core domain

Move the Web-MIDI source provider out of input_setup and into the core
midi-input domain so every consumer (piano, drums, input_setup) gets MIDI
devices from the domain without depending on any one plugin being loaded.
input_setup is now a pure midi-input requester (manifest role updated).
Prepares piano/drums full consumption (#876/#877). +1 domain test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(input_setup): Settings panel to re-run input setup (#878)

Adds a settings.html with a "Set up input devices" button (window
._inputSetupRelaunch) that re-runs the wizard for the player's selected
instrument paths (from /api/progression; falls back to all instruments). Makes
the calibration wizard re-launchable outside first-run onboarding.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(midi-control): formalize the midi-input/midi-control split (#882)

Narrow the reserved midi-control domain to mappings ONLY (CC/pitchbend/note →
action routing), consuming the delivered midi-input domain for device access.
Adds spec 013 defining the contract + intended consumers (feedback-plugin-midi,
drums learn-mode), updates the safety-matrix row, and cross-references it from
capability-domains. Per governance, midi-control stays RESERVED (no runtime
domain) until a concrete mapping consumer + tests exist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(onboarding): wait for input_setup before the calibration step (#874)

The input-setup wizard is a mandatory onboarding step, but plugins load
asynchronously — in the desktop app (40+ plugins) the user can reach path
selection and click Next before input_setup has registered its
input-calibration owner. The dispatch then got a no-owner outcome and
onboarding fell through to the calibration challenge, silently skipping the
wizard. Now wait (bounded, 8s) for the plugin's public global before
dispatching; fall through only if it never appears. Race-verified.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(onboarding): add Song directory step after name+avatar (#874)

New first-run step (now step 2 of 4: name+avatar → song directory → paths →
calibration challenge) where the player sets their songs folder, fixing the
"folder not configured" error on a fresh install. Saves to settings (dlc_dir)
and kicks a library scan; persists to config.json so it survives restart. A
native folder picker is offered on desktop (window.slopsmithDesktop
.pickDirectory); web users type/paste the path. "Skip for now" leaves it
unconfigured (settable later in Settings).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(input_setup): filter MIDI entries out of the guitar audio-input picker (#876)

Other plugins export pseudonymized MIDI sources ('midi-input-N') into the
audio-input domain; they aren't audio inputs and the cryptic labels confused
the guitar/bass device dropdown. Filter them out so only real audio inputs show.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(input_setup): de-dupe audio input picker entries (#876)

The desktop audio engine enumerates the same device under multiple driver
types, so the guitar audio-input dropdown showed repeated entries. De-dupe by
display label (paired with the desktop fix that surfaces real device names).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(midi-input): drop vanished devices on re-discovery; reset setup confirm on switch

Codex preflight findings:
- midi-input domain `_discover()` only upserted enumerated sources, so an
  unplugged device (statechange re-discovery) lingered in list-sources and later
  open/select hit stale state. Reconcile each provider's sources against the
  fresh enumeration (close any live session, keep the selectedKey preference).
- input_setup MIDI panel left "Continue" enabled (and the instrument marked
  done) after switching the device selection following a prior hit. Reset the
  waiting state + disable Continue on every selection change, and discard a
  stale open if the selection changed mid-await. +1 reconciliation test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(midi-input): coalesce concurrent opens; commit shown audio source pre-calibration

Codex re-review (round 2):
- midi-input domain: two concurrent open-source calls for the same source both
  passed the `sessions.get` guard and each called provider.open(), which for the
  built-in Web-MIDI provider overwrites the shared input.onmidimessage handler
  and orphans the earlier session — leaving the device silent. Coalesce in-flight
  opens onto one provider session (await the pending open, adopt its session;
  re-check after open and release a redundant handle if another open won). +test.
- input_setup: the guitar/bass audio <select> shows its first option by default
  but fires no `change`, so on a first run with nothing selected, audio-input was
  never told before launchCalibration(). Commit the shown option on render.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(midi-input): longer timeout for MIDI permission commands; stale-open guard in wizard

Codex re-review (round 3):
- The advertised command surface ran `discover`/`open-source` through the 250 ms
  default handler timeout, but those front a real Web-MIDI permission prompt /
  device open that commonly takes longer, so dispatch returned `failed` while the
  operation was still completing. Add per-(capability,command) timeout overrides
  (15 s for those two), folding the existing audio-mix special-case into the same
  table so both the command() and dispatch() paths honor it.
- input_setup MIDI panel: openSelected() compared the mutable shared `activeKey`
  after its awaits, so a device switch mid-open could bind the old device's
  listener / close the wrong session. Capture the requested key in a local and
  use a generation guard to discard a superseded open.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(onboarding): detect 200-with-error song-dir saves; close MIDI session on skip

Codex re-review (round 4):
- /api/settings reports an invalid folder as a 200 response with an `error` body
  (a bare dict return, not a non-2xx status), so saveSongDir's res.ok-only check
  treated the failure as success and advanced onboarding without saving. Parse
  the body and throw on `error` too.
- input_setup: the opened MIDI test session was only closed on the Continue
  button, so using the generic "Skip for now" after scanning leaked the listener
  and kept the Web-MIDI input live. Run teardown on every panel exit via a
  per-panel cleanup hook invoked by advance().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(input_setup): don't hard-code Web MIDI in the device wizard

Codex re-review (round 5): the MIDI panel gated availability on
navigator.requestMIDIAccess and filtered sources to providerId === 'web-midi',
which defeats the midi-input domain's provider-coordinator abstraction — a
native/desktop MIDI adapter registered with the domain would be reported
unavailable and hidden from the picker. Gate availability on the domain
(window.slopsmith.midiInput) and show every source it surfaces.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Byron Gamatos
2026-06-19 16:31:16 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 313348a1ff
commit fb06e288e1
15 changed files with 1396 additions and 19 deletions
+12
View File
@@ -153,6 +153,18 @@ The legacy chart-coupled surface — `highway.setNoteStateProvider(fn)`, the sin
Diagnostics live under `slopsmith.note_detection_capability.v1` and contain provider ids/labels/kinds, binding summaries (requester, provider, redacted context, target size), availability, and the last bounded outcome (event, binding, provider, MIDI number, hit flag) — never raw audio buffers, sample data, device labels, or song identity.
## MIDI-Input Domain
The MIDI-input slice (spec 012, issues #873/#880) promotes `midi-input` as a **core-owned** provider-coordinator implemented by [static/capabilities/midi-input.js](../static/capabilities/midi-input.js) — the MIDI analog of `audio-input`. It is deliberately separate from `audio-input` (whose source/`open` contract is audio-frame-centric: channel shapes, sample buffers) because MIDI carries discrete messages, not audio; and it is **not** owned by any feature plugin, so the device-access boundary outlives the input-setup wizard (exactly as `audio-input` is `core.audio.session`-owned). Consumers — the `input_setup` onboarding wizard, the `piano`/keys and `drums` plugins, and (as a follow-up, #881) note-detection's Web-MIDI provider — converge here on ONE device-access boundary: one permission prompt, one source list, one redaction boundary, retiring private per-plugin `navigator.requestMIDIAccess()` calls.
Native providers register source summaries with `providerId`, a stable `sourceId`, a derived redaction-safe `logicalSourceKey` (`providerId::sourceId`), `kind: "midi"`, a label, and `availability`. The public command surface is `inspect`, `list-sources`, `discover`, `select-source`, `open-source`, and `close-source`; provider operations are `source.enumerate`, `source.describe`, `source.open`, and `source.close`. `inspect`, `list-sources`, and `select-source` are prompt-free and never request MIDI access. Unlike audio (where `getUserMedia` gates labels and `open-source` is the prompt), Web-MIDI's `requestMIDIAccess()` gates the whole input list, so **`discover` is the permission boundary** and records `denied`/`unavailable` outcomes; `open-source` then attaches a shared listener session and never re-prompts.
Selected input is persisted by `logicalSourceKey` (`slopsmith.midiInput.selectedLogicalSourceKey`) when browser storage is available. Compatible requesters share one open session per source; each later calls `close-source`, and the provider receives `source.close` only after the last requester releases. Live MIDI message delivery (for the "play a note / hit a pad" calibration check) is exposed to in-page consumers through the public `window.slopsmith.midiInput` session handle only — never as raw capability events or diagnostics.
The reserved `midi-control` domain is the planned **sibling** for control mappings (CC/pitchbend/note → action routing) and will consume `midi-input` for device access (spec 013 / #882); this slice carves the device control plane out so `midi-control` can stay mappings-only. `midi-control` stays RESERVED (documentation-only) until a concrete mapping consumer + tests exist, per the future-domain governance.
Diagnostics live under `slopsmith.midi_input.diagnostics.v1` and contain provider ids, source ids/keys/kinds/availability, the selected key, and open-session keys — **device labels are redacted** and no raw MIDI messages are ever included.
## Capability Roles
Use capability declarations for provider/requester/observer relationships:
+1 -1
View File
@@ -108,7 +108,7 @@ These domains are planned but should stay out of the runtime graph until a host
| `ui.player-overlays` | exclusive-owner | safe | Overlay contributions layered over player or highway surfaces. | Overlay placement and z-order rules that coexist with legacy overlays. |
| `plugins` | exclusive-owner | privileged | Plugin enable/disable/install/update workflows. | Visible user confirmation, rollback, and disabled-handler enforcement. |
| `jobs` | multi-provider | privileged | Long-running jobs, cancellation, status, failures. | Scheduling limits, cancellation semantics, and user-visible failures. |
| `midi-control` | multi-provider | sensitive | MIDI device providers and control mappings. | Device consent and redacted diagnostics. |
| `midi-control` | multi-provider | sensitive | MIDI control mappings only (CC/pitchbend/note → action routing), consuming `midi-input` for device access. Device discovery/selection/open is split out to the delivered `midi-input` domain (spec 012). | A concrete mapping/routing workflow on top of the `midi-input` device plane (#882). |
| `audio-input` | multi-provider | sensitive | Audio input device providers, source selection, open/close lifecycle, shared sessions, and redacted failure diagnostics. | Promoted by the audio graph/session slice and implemented by the audio-input control-plane slice. |
| `tempo-clock` | multi-provider | safe | Tempo/clock provider registration and consumers. | A concrete tempo source and consumer workflow. |
+3 -1
View File
@@ -21,6 +21,8 @@ Core domains also have a review scope. **Active contract** domains are wired to
| note-detection | provider-coordinator | sensitive | inspect, register-provider, unregister-provider, open-binding, close-binding, set-target, clear-target | pitch.estimate, verify.target | Detection-binding control plane (spec 009): providers (midi/engine/js) serve primitives; each requester binds its own redacted tuning context; consumers own judgment, hit/miss flow as observability events. Legacy `highway.setNoteStateProvider` is an accounted shim. Diagnostics carry provider/binding summaries and bounded outcomes — no raw audio, sample data, device labels, or song identity. |
| midi-input | provider-coordinator | sensitive | inspect, list-sources, discover, select-source, open-source, close-source | source.enumerate, source.describe, source.open, source.close | Core-owned MIDI device control plane (spec 012), the MIDI analog of `audio-input`. Inspect/list/select are prompt-free; `discover` is the Web-MIDI permission boundary (`requestMIDIAccess()` gates the whole input list) and records denied/unavailable outcomes; `open-source` attaches a shared listener session and never re-prompts. Selection persists by redaction-safe `logicalSourceKey`. Diagnostics redact device labels and never include raw MIDI messages or live handles. |
Privileged commands are roadmap-only until they have: a visible user confirmation path, diagnostics redaction rules, failure recovery, and tests that prove disabled or incompatible participants cannot execute handlers.
## Expected Future Domains
@@ -38,7 +40,7 @@ These domains are expected future capability contracts, not current runtime grap
| ui.player-overlays | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs overlay placement rules that coexist with legacy highway overlays. |
| plugins | exclusive-owner | privileged | enable, disable, install-missing, update, inspect | Needs explicit user confirmation for writes/install/update. |
| jobs | multi-provider | privileged | register, inspect, cancel | Needs scheduling limits, cancellation semantics, and user-visible failures. |
| midi-control | multi-provider | sensitive | register, inspect | Needs device consent and redacted diagnostics. |
| midi-control | multi-provider | sensitive | list-mappings, get-mapping, set-mapping, delete-mapping, activate-mapping, inspect | Mappings ONLY — CC/pitchbend/note → semantic action routing (spec 013). Device discovery/selection/open is NOT this domain's job: it consumes the delivered `midi-input` domain for device access. Needs a concrete mapping consumer (the MIDI control plugin / drums learn-mode) + redacted diagnostics (no raw MIDI streams) before promotion. |
| tempo-clock | multi-provider | safe | register, inspect | Needs a concrete provider and consumer workflow. |
Planned domains should also stay out of the runtime graph until Slopsmith ships the corresponding user-facing workflows.