fix(audio-input): return bound device from open-source + live input-meter gate in input-setup

A3: openInputSource() surfaces the provider's bound device on the command return
(payload.bound) for the trusted caller, kept out of the redacted event/diagnostics.
B: input-setup guitar/bass step gains a Test-input live meter gating Continue on
heard signal, reading note_detect getInputLevel() (same stream the scorer binds).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
byrongamatos 2026-06-29 00:22:52 +02:00
parent 271fedda55
commit db0b55a301
4 changed files with 161 additions and 11 deletions

View File

@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **Input-setup wizard's guitar/bass step now has a live "we can hear you" check, so it can't be passed against a silent/wrong device.** The MIDI step already gated Continue on a live note; the audio step could be completed with zero proof any audio was arriving — which is exactly how a wrong-mic pick (see the desktop binding fix) sailed through onboarding. The guitar/bass panel (`plugins/input_setup/screen.js` `renderAudioPanel`) gains a **Test input** button that opens the *selected* device and shows a live input meter, with a **Continue** button that stays disabled until real signal is heard (two consecutive ticks above an idle-noise threshold → "✓ We can hear you"). **Calibrate** stays always-enabled alongside it — it's the tool that fixes a too-low input, so it must never be gated. The meter reads `window.noteDetect.getInputLevel()` — the *same* level the scorer sees off the bound capture stream (not a second, independently-opened stream), so it can't green-light a device detection isn't actually hearing. Changing the device while testing re-commits the selection and cycles the capture so the meter always reflects the shown device, never a stale one; the gate resets on device change, the panel restores the prior detection on/off state on exit, and hands the capture off cleanly to the Calibration Wizard. Degrades to the prior calibrate-only flow when the detector doesn't expose `getInputLevel`/enable/disable. Pairs with feedback-plugin-notedetect's new `getInputLevel()` getter and the desktop wrong-mic binding fix. Runtime verification needs a desktop build with a real interface (the web testbed has no enumerated audio-input sources).
- **Playlists get content-dependent covers + custom art.** Playlist cards were a tiny `🎵` emoji on an empty square. Now a playlist's cover reflects its contents: **empty → the icon**, **a few songs → the first song's album art**, **4+ songs → a 2×2 art mosaic**. You can also **upload a custom cover** (a "Cover" button in the playlist detail view → image picker; "Remove cover" reverts to the content view). `MetadataDB.list_playlists()` now returns each playlist's first few song `art_urls`; `GET /api/playlists` and `GET /api/playlists/{id}` add `cover_url` when a custom cover exists. New routes `POST` / `GET` / `DELETE /api/playlists/{id}/cover` store a small PNG thumbnail under `CONFIG_DIR/playlist_covers/` (PIL-converted, like song-art upload); the cover is removed with the playlist. Frontend: `playlistCoverHtml(p)` in `static/v3/playlists.js`. Tests: `tests/test_playlists_api.py` (art_urls + cover roundtrip / reject-non-image / delete-cleanup), `tests/js/v3_playlist_cover.test.js`.
- **v3 Songs: "Add to playlist" is now on each song's ⋮ "More" menu.** Previously a song could only be added to a playlist through select-mode (the checkbox → batch bar). The per-card overflow menu now has an **Add to playlist** row that targets that one song, reusing the same picker (choose a listed number or type a new name to create it). The select-mode batch flow and the single-song menu now share one extracted `addFilenamesToPlaylist(filenames)` helper in `static/v3/songs.js` (both grid and tree rows, since they share `openCardMenu`). Tests: `tests/js/v3_add_to_playlist_menu.test.js`.
- **Resume where you left off — leaving a song now snapshots your place so an exit is recoverable, not a restart-from-zero.** Exiting the player (`showScreen()` teardown, before audio unload) writes `{song, arrangement, position, speed}` to `localStorage` (`feedBack.resumeSession`), and a non-blocking **"Resume practice"** pill offers it back on the next non-player screen (and on the next app launch). Clicking Resume re-enters the song, restores the arrangement + playback speed, and seeks to the saved position via the existing `_audioSeek` funnel; `playSong()` gains a `{ resume: {position, speed} }` option that arms a `song:ready`-consumed restore instead of the normal autostart, so the two never fight over playback. The snapshot is deliberately conservative — ignored for a song you barely started (< 3s) or had basically finished (within 5s of the end), cleared on natural song-end and once consumed, and expired after 24h. The pill is self-contained (inline-styled, body-appended, works identically in the classic and v3 shells with no Tailwind rebuild), never blocks, and a dismiss forgets the current snapshot for the session. This pairs with the Escape focus fix: now that Escape reliably leaves regardless of focus, an *accidental* exit is one tap to undo. Public surface: `window.resumeLastSession()` / `window.feedBack.resumeLastSession`. (The broader nav-state work returning to a song after wandering into Settings Tone Builder is a separate, larger track; this lands the player-session slice.) Tests: `tests/browser/resume-session.spec.ts` (snapshot guards, staleness, pill show/hide/dismiss, resume consumption).
@ -40,6 +41,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **v3 library: exact artist/album filters + scroll/page-depth restore** (feedBack#857). The v3 Songs toolbar gains Artist and Album dropdowns (Album populates from the selected artist and stays disabled until one is chosen), backed by new exact, case-insensitive (`COLLATE NOCASE`) `artist` / `album` query params threaded through `MetadataDB._build_where``query_page` / `query_artists` / `query_stats` and the `/api/library`, `/api/library/artists`, `/api/library/stats` endpoints (the free-text `q` search stays fuzzy and composes with the exact filters). The artist/album catalog is fetched independently of the active artist/album selection so the dropdowns always list the full set for the current provider/search. The toolbar is now sticky so filter controls stay reachable when browsing deep libraries, and returning from the player restores the previous scroll position **and** the loaded infinite-scroll page depth via a `sessionStorage` snapshot keyed by a filter/sort/view state hash (invalidated whenever those change, so a filter change still resets to the top). Tests: `tests/test_library_filters.py` (backend artist/album filters), `tests/js/v3_songs_scroll.test.js` (state-hash + snapshot helpers).
### Fixed
- **`audio-input` `open-source` now reports the device the engine *actually* bound, so the wrong-mic case can be caught instead of trusting the pick blind.** A provider's `source.open` handler may return `payload: { boundType, boundName }`; `openInputSource()` (`static/capabilities/audio-session.js`) surfaces it on the command's **return** value as `payload.bound = { type, name }` for the trusted in-process caller (the input-setup confirmation gate, note_detect), enabling an honest "Now listening to: <device>" readout and detection of a silent substitution (picked BlackHole, got the internal mic). The raw device name is PII, so it is deliberately kept **out** of the emitted `source-opened` event and the diagnostics snapshot (which stay redacted) — mirroring how `list-sources` already returns the device `label` verbatim to the UI but pseudonymizes it in diagnostics. Purely additive: the open-session summary shape is unchanged and `bound` is omitted when the provider reports nothing. Pairs with feedBack-desktop's switch to a stable name-based input identity + fail-loud open. Tests: `tests/js/audio_session_input.test.js` (read-back surfaced to the caller, absent from event + snapshot).
- **Input-setup wizard no longer collapses an audio device's driver-type variants into one entry.** On Windows the desktop engine enumerates the same interface once per host API (ASIO / Windows Audio / DirectSound), and the wizard's audio picker (`plugins/input_setup/screen.js`) de-duped the source list by display **label** — so the variants (which share a name) collapsed to a single choice, silently keeping whichever sorted first (often *not* the low-latency ASIO one the player wants). The audio-input capability already collapses true duplicates by `logicalSourceKey` (`_visibleInputSources` in `static/capabilities/audio-session.js`), and the variants each have a **distinct** key, so the wizard's extra label-collapse was redundant for real dupes and destructive for these — it also could drop the variant that was actually `selected`. Removed it; the picker now lists every selectable input. Pairs with feedBack-desktop's change to label each source with its driver type (e.g. "Focusrite (ASIO)") so the now-distinct entries are legible.
- **v3 Songs grid now refreshes after a Settings rescan / DLC-folder change — no app restart needed.** On a fresh install, pointing at a DLC folder in Settings and running a scan left the Songs section empty until a restart (the scan *did* populate the library — `_background_scan` re-reads `config.json` fresh — but the v3 grid never reloaded). The Settings **Rescan / Full Rescan** handlers only refreshed the classic (v2) library via `loadLibrary()`; the v3 grid (`static/v3/songs.js`) had no listener for a scan it didn't start itself (only its own upload path self-refreshed via `watchUploadScan`), so its cached, pre-DLC (empty) DOM/snapshot survived a sidebar return until a full reload. The rescan handlers now emit a **`library:changed`** event (`static/app.js`); the v3 grid listens and **reloads if it's the active screen, else marks itself dirty** so the next entry does a full re-fetch instead of restoring the stale snapshot (a `_libraryDirty` short-circuit ahead of every cached-DOM fast-path in `onV3SongsScreenEnter`). Tests: `tests/js/v3_library_refresh.test.js` (the emit + the reload/dirty wiring).
- **Edit Metadata modal: the Year is now editable.** You could set a year when authoring a pak but the Songs → Edit Metadata modal had no Year field, so it could never be changed afterward. The backend (`POST /api/song/<f>/meta`) already accepted and normalized `year` (writes it into the file via `songmeta`, survives a rescan) — only the UI omitted it. Added a **Year** input to `openEditModal()` (populated from the song's existing year) and included `year` in `saveEditModal()`'s POST body (`static/app.js`). Both the v3 card menu and the legacy edit button already pass the year through, so both surfaces get the field.

View File

@ -126,32 +126,116 @@
return renderMidiPanel(inst);
}
// Guitar/bass: show the audio source (audio-input) and launch the
// note_detect Calibration Wizard for the deep work.
// Guitar/bass: show the audio source (audio-input), a live "we can
// hear you" meter, then launch the note_detect Calibration Wizard for
// the deep work.
async function renderAudioPanel(inst) {
const { sources, selected } = await _audioSources();
const opts2 = sources.map((s) =>
'<option value="' + esc(s.logicalSourceKey || s.sourceId || '') + '"' + (s.selected ? ' selected' : '') + '>' + esc(s.label || 'Input') + '</option>').join('');
const hasDetector = !!(window.noteDetect && typeof window.noteDetect.launchCalibration === 'function');
// The live-meter gate needs the detector to expose its input level
// (getInputLevel) and the enable/disable capture lifecycle. When any
// is missing (older detector / none) we degrade to the prior
// calibrate-only flow rather than show a meter we can't feed.
const nd = window.noteDetect;
const canMeter = !!(hasDetector && nd
&& typeof nd.getInputLevel === 'function'
&& typeof nd.enable === 'function'
&& typeof nd.disable === 'function'
&& typeof nd.isEnabled === 'function'
&& sources.length);
const body =
'<p class="text-sm text-fb-textDim">Pick your audio input, then run the calibration to set levels, channel and latency.</p>' +
'<p class="text-sm text-fb-textDim">Pick your audio input, then ' +
(canMeter ? 'play a note so we can confirm we hear it — then calibrate or continue.' : 'run the calibration to set levels, channel and latency.') + '</p>' +
(sources.length
? '<label class="block text-xs uppercase tracking-wider text-fb-textDim mt-3 mb-1">Audio input</label>' +
'<select data-is-audio class="w-full bg-gray-800/50 border border-gray-700 rounded-md px-2 py-1.5 text-sm text-fb-text outline-none">' + opts2 + '</select>'
: '<p class="text-sm text-fb-accent mt-2">No audio input detected yet — plug in your interface, or skip and set this up later.</p>') +
(canMeter
? '<div class="mt-3"><button type="button" data-is-test class="text-sm text-fb-primary hover:text-fb-primaryHi">Test input</button>' +
'<div data-is-meter-wrap class="hidden mt-2">' +
'<div class="h-2 w-full bg-gray-800 rounded overflow-hidden"><div data-is-meter class="h-full w-0 bg-fb-primary" style="transition:width 80ms linear"></div></div>' +
'<p data-is-test-msg class="text-sm text-fb-textDim mt-1.5">Play your ' + esc(String(INSTRUMENTS[inst].label).toLowerCase()) + ' — waiting for signal…</p></div></div>'
: '') +
(hasDetector ? '' : '<p class="text-xs text-fb-textDim mt-3">The note detector isnt loaded here — you can calibrate later from the player.</p>');
const foot =
'<button type="button" data-is-cal class="bg-fb-primary hover:bg-fb-primaryHi text-white px-5 py-2 rounded-md font-medium">' +
(hasDetector ? 'Calibrate' : 'Continue') + '</button>';
// canMeter: a gated "Continue" (enabled once we hear you) PLUS an
// always-enabled "Calibrate" — never gate Calibrate, since it's the
// tool that fixes a too-low input that produces no signal.
const foot = canMeter
? '<button type="button" data-is-cal class="border border-gray-600 hover:border-gray-400 text-fb-text px-4 py-2 rounded-md font-medium mr-2">Calibrate</button>' +
'<button type="button" data-is-cont disabled class="bg-fb-primary hover:bg-fb-primaryHi disabled:opacity-40 text-white px-5 py-2 rounded-md font-medium">Continue</button>'
: '<button type="button" data-is-cal class="bg-fb-primary hover:bg-fb-primaryHi text-white px-5 py-2 rounded-md font-medium">' +
(hasDetector ? 'Calibrate' : 'Continue') + '</button>';
shell(inst, body, foot);
const sel = host.querySelector('[data-is-audio]');
const commitAudio = (key) => {
if (!capabilities || !key) return;
capabilities.command('audio-input', 'select-source', { requester: 'input_setup', payload: { logicalSourceKey: key } }).catch(() => {});
if (!capabilities || !key) return Promise.resolve();
// Awaitable: the meter must reopen on the *committed* selection,
// so callers can sequence enable() after the choice persists.
return capabilities.command('audio-input', 'select-source', { requester: 'input_setup', payload: { logicalSourceKey: key } }).catch(() => {});
};
// ── live "we can hear you" meter ────────────────────────────────
const meterWrap = host.querySelector('[data-is-meter-wrap]');
const meterBar = host.querySelector('[data-is-meter]');
const testMsg = host.querySelector('[data-is-test-msg]');
const contBtn = host.querySelector('[data-is-cont]');
// Idle hiss sits well under this; a plucked string clears it easily.
const SIGNAL_THRESHOLD = 0.04;
let pollTimer = null;
let testing = false;
let weEnabled = false; // did WE turn detection on (so we restore it)?
let handingOff = false; // Calibrate owns the capture from here on
function stopPoll() { if (pollTimer) { clearInterval(pollTimer); pollTimer = null; } }
function resetGate() {
if (contBtn) contBtn.disabled = true;
_markDone(inst, false);
if (meterBar) meterBar.style.width = '0%';
if (testMsg) testMsg.textContent = 'Play your ' + String(INSTRUMENTS[inst].label).toLowerCase() + ' — waiting for signal…';
}
function startPoll() {
stopPoll();
let heard = 0;
pollTimer = setInterval(() => {
let lvl = 0;
try { lvl = (nd.getInputLevel() || {}).level || 0; } catch (_) { lvl = 0; }
if (meterBar) meterBar.style.width = Math.min(100, Math.round(lvl * 140)) + '%';
if (lvl >= SIGNAL_THRESHOLD) {
// Require two consecutive ticks so a single click/pop
// doesn't false-pass the gate.
if (++heard >= 2 && contBtn && contBtn.disabled) {
if (testMsg) testMsg.innerHTML = '<span class="text-fb-primary font-semibold">✓ We can hear you</span> — your input is working.';
contBtn.disabled = false;
_markDone(inst, true);
}
} else { heard = 0; }
}, 100);
}
async function startTest() {
stopPoll();
resetGate();
if (meterWrap) meterWrap.classList.remove('hidden');
// Persist the shown choice, THEN (re)open it: enable() is a no-op
// when detection is already on, so a device change while testing
// must cycle disable→enable or the meter would read the stale
// device — false confidence, the exact bug this gate prevents.
await commitAudio(sel.value);
try {
if (nd.isEnabled()) { await nd.disable(); await nd.enable(); weEnabled = false; }
else { await nd.enable(); weEnabled = true; }
} catch (_) { /* fail-soft: meter just won't move; Calibrate still works */ }
testing = true;
startPoll();
}
if (sel) {
sel.addEventListener('change', () => commitAudio(sel.value));
sel.addEventListener('change', async () => {
if (testing) { await startTest(); }
else { await commitAudio(sel.value); resetGate(); }
});
// The <select> shows its first option by default, but no `change`
// fires for that implicit pick — so on a first run with nothing yet
// selected, audio-input would calibrate against the wrong/no source.
@ -159,10 +243,27 @@
// one calibrated (idempotent if it was already selected).
if (!selected) commitAudio(sel.value);
}
const testBtn = host.querySelector('[data-is-test]');
if (testBtn) testBtn.addEventListener('click', () => { startTest(); });
if (contBtn) contBtn.addEventListener('click', () => advance(inst, true));
// Restore detection state on ANY exit (Continue / Skip), unless we've
// handed the capture off to the Calibration Wizard, which owns it.
_activeCleanup = () => {
stopPoll();
testing = false;
if (weEnabled && !handingOff) { try { nd.disable(); } catch (_) {} }
weEnabled = false;
};
// Tell the tuner tables / note_detect which instrument this is.
try { fetch('/api/settings', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ instrument: inst }) }); } catch (_) {}
host.querySelector('[data-is-cal]').addEventListener('click', () => {
// Hand the (possibly already-open) capture to the Calibration
// Wizard — it calls enable() itself and manages teardown — so our
// cleanup must not disable detection out from under it.
handingOff = true;
stopPoll();
if (hasDetector) {
// Hide our own full-screen overlay while note_detect's
// Calibration Wizard runs on top. That wizard goes
@ -182,7 +283,15 @@
window.noteDetect.launchCalibration({
instrument: inst,
onDone: () => { restore(); advance(inst, true); },
onCancel: () => { restore(); /* stay on this panel; user can skip or retry */ },
onCancel: () => {
restore();
// Back on this panel: reclaim cleanup ownership so a
// later Skip/Continue still restores detection state.
// The wizard's enable() may have left detection on; if
// we didn't turn it on ourselves, leave it as-is.
handingOff = false;
if (nd && typeof nd.isEnabled === 'function') { try { weEnabled = weEnabled && nd.isEnabled(); } catch (_) {} }
},
});
} else {
advance(inst, true);

View File

@ -1766,6 +1766,16 @@
const providerResult = _providerOutcome(raw);
openSession.state = providerResult.outcome === 'handled' ? 'open' : (providerResult.status || providerResult.outcome);
openSession.reason = providerResult.reason;
// Read-back: the provider (desktop renderer) reports which device the
// native engine ACTUALLY bound. Surface it to the in-process caller so
// the wizard can show "Now listening to: <device>" and catch a silent
// mismatch (picked BlackHole, got the internal mic). Kept OUT of the
// redacted summary/event below: that flows into diagnostics, where a raw
// device name (e.g. "Byron's AirPods") is PII — but it is fine to return
// verbatim to the trusted same-renderer caller, exactly as list-sources
// already returns the device `label` verbatim.
const boundInfo = _plainObject(providerResult.payload);
const bound = { type: _string(boundInfo.boundType, ''), name: _string(boundInfo.boundName, '') };
if (providerResult.outcome === 'handled') {
openSession.openedAt = _now();
currentSession.openInputSessions.set(key, openSession);
@ -1773,7 +1783,7 @@
_recordOutcome({ domain: 'audio-input', operation: 'open-source', participantId: requesterId, requesterId, providerId: provider.providerId, sourceId: selected.sourceId, logicalSourceKey: selected.logicalSourceKey, openSessionId: openSession.openSessionId, outcome: 'handled', status: 'open' });
capabilities.emitEvent('audio-input', 'source-opened', summary);
_touch();
return _handled(summary);
return _handled((bound.name || bound.type) ? { ...summary, bound } : summary);
}
const summary = _redactedOpenSession(openSession, _newPseudonymizer());
_recordOutcome({ domain: 'audio-input', operation: 'open-source', participantId: requesterId, requesterId, providerId: provider.providerId, sourceId: selected.sourceId, logicalSourceKey: selected.logicalSourceKey, openSessionId: openSession.openSessionId, outcome: providerResult.outcome, status: openSession.state, reason: providerResult.reason });

View File

@ -281,6 +281,35 @@ test('open-source and close-source record outcomes events and no live handles',
assert.equal(encoded.includes('token=abc'), false);
});
test('open-source returns the actually-bound device for read-back without leaking it into diagnostics', async () => {
const window = loadAudioSession();
const api = window.feedBack.capabilities;
const openedEvents = captureEvents(window, 'audio-input:source-opened');
await registerSource(api, {
sourceId: 'readback-source',
logicalSourceKey: 'test:readback',
operationHandlers: {
'source.open': () => ({ outcome: 'handled', status: 'open', payload: { boundType: 'CoreAudio', boundName: 'BlackHole 16ch', requestedName: 'BlackHole 16ch' } }),
'source.close': () => ({ outcome: 'handled', status: 'closed' }),
},
});
await api.dispatch({ capability: 'audio-input', command: 'select-source', source: 'user', payload: { logicalSourceKey: 'test:readback' } });
const open = await api.dispatch({ capability: 'audio-input', command: 'open-source', source: 'note_detect', payload: { requesterId: 'note_detect', purpose: 'note-detection' } });
// The trusted in-process caller (input_setup's confirmation gate) gets the real bound device,
// so it can show "Now listening to: <device>" and catch a silent wrong-mic substitution.
assert.equal(open.outcome, 'handled');
assert.equal(open.payload.bound.type, 'CoreAudio');
assert.equal(open.payload.bound.name, 'BlackHole 16ch');
// ...but a raw device name is PII: it must NOT reach the emitted event or the diagnostics snapshot.
assert.equal(openedEvents.length, 1);
assert.equal('bound' in openedEvents[0], false);
const encoded = JSON.stringify(window.feedBack.audioSession.snapshot());
assert.equal(encoded.includes('BlackHole'), false);
});
test('open-source reports no-owner no-handler unsupported failed and malformed provider data distinctly', async () => {
const window = loadAudioSession();
const api = window.feedBack.capabilities;