mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-08-11 11:19:24 +00:00
Clean release snapshot
This commit is contained in:
byrongamatos
@@ -0,0 +1,243 @@
|
||||
# Note Failure Feedback — Implementation Plan
|
||||
|
||||
Depends on: `docs/NOTE_FAILURE_SPEC.md` (read that first)
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Detection Plugin Foundation
|
||||
|
||||
**Goal:** Working note detection plugin streaming detected notes via WebSocket.
|
||||
|
||||
This phase was previously tracked in a separate NOTE_DETECTION_PLUGIN_PLAN
|
||||
document (in the `slopsmith-plugin-notedetect` repository). The relevant scope
|
||||
is summarized here to avoid relying on an internal git-only reference:
|
||||
|
||||
- [ ] Plugin skeleton: `slopsmith-plugin-notedetect/` with plugin.json, routes.py, screen.js
|
||||
- [ ] Port TonalRecall YIN detection (aubio + sounddevice) to routes.py
|
||||
- [ ] WebSocket at `/api/plugins/note_detect/stream` streaming `{ note, freq, confidence, time }`
|
||||
- [ ] Device selection UI in screen.html
|
||||
- [ ] requirements.txt: aubio, sounddevice, numpy
|
||||
|
||||
**Exit criterion:** With plugin active and guitar plugged in, playing a note
|
||||
causes a JSON event to appear in the browser console.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Note Matching Core
|
||||
|
||||
**Goal:** Client-side matching of detected notes to chart notes. No rendering yet —
|
||||
console logging only.
|
||||
|
||||
**Files:**
|
||||
- `screen.js` in the notedetect plugin
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Implement `expectedFreq(string, fret, tuningOffsets, capo, stringCount, arrangementName)` using
|
||||
base open-string frequencies, semitone offsets from `highway.getSongInfo().tuning`,
|
||||
and semitone math (`2^(semitones/12)`) rather than assuming 6-string standard
|
||||
tuning; use `highway.getStringCount()` as the authoritative string count because
|
||||
tuning may be padded to length 6 for RS XML sources even for bass/extended-range
|
||||
arrangements; include `highway.getSongInfo().capo` as additional semitones if
|
||||
the intent is expected sounding pitch; pass `highway.getSongInfo().arrangement` as
|
||||
`arrangementName` to disambiguate 5-string bass vs 5-string guitar (matching the
|
||||
spec's `getBaseTuning` helper)
|
||||
- [ ] Implement `NoteJudgmentTracker` class with:
|
||||
- `addDetection(detected)` — correlate with nearest unmatched chart note
|
||||
- `update(currentTime)` — expire pending notes whose match window has passed
|
||||
- `getJudgmentsInRange(tStart, tEnd)` — return judgments in time range
|
||||
- `reset()` — clear all state
|
||||
- [ ] Connect to detection WebSocket, feed events into tracker
|
||||
- [ ] Initialize tracker with `highway.getNotes()` and `highway.getChords()` on song ready
|
||||
- [ ] Console.log each judgment as it resolves (HIT/MISSED/EARLY/LATE/SHARP/FLAT)
|
||||
- [ ] Re-initialize tracker on `song:ready` (fires on every new song **and** on every
|
||||
arrangement switch — no need to hook `highway.reconnect` or other internals);
|
||||
do **not** use `song:loaded` — note/chord arrays are still empty at that point (data
|
||||
arrives incrementally and only completes at `song:ready`)
|
||||
|
||||
**Exit criterion:** Playing along with a song, console shows correct HIT/MISSED
|
||||
judgments with timing and pitch error values.
|
||||
|
||||
**Estimated scope:** ~200 lines JS
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Hit/Miss Highway Overlay
|
||||
|
||||
**Goal:** Visual feedback on the highway — green glow for hits, red X for misses.
|
||||
|
||||
**Files:**
|
||||
- `screen.js` in the notedetect plugin (draw hook)
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Register `highway.addDrawHook()` that reads judgments from the tracker
|
||||
- [ ] **Hit rendering:** Green glow ring behind notes at the now-line, fading over
|
||||
`hitGlowDuration` seconds. Use `highway.project()` and `highway.fretX()` for
|
||||
positioning. Additive blend via `ctx.globalCompositeOperation = 'lighter'`.
|
||||
- [ ] **Miss rendering:** Red `✕` marker at the note's string/fret position, drawn
|
||||
in the "past" region below the now-line. Do **not** rely on
|
||||
`highway.project(negative_offset)` for long-lived placement — the current
|
||||
renderer returns `null` for offsets more than ~50ms into the past. Instead,
|
||||
anchor at the now-line (`highway.project(0)`) and map elapsed time since the
|
||||
miss to a linear below-now-line Y position (configurable pixels/second), fading
|
||||
the marker after `missMarkerDuration` seconds.
|
||||
- [ ] **String pulse:** Brief red tint on the missed note's string (200ms fade on
|
||||
the string line segment near the now-line).
|
||||
- [ ] Handle lefty mode: use `highway.fillTextUnmirrored()` for text markers.
|
||||
- [ ] Cleanup: `highway.removeDrawHook()` on plugin destroy.
|
||||
|
||||
**Exit criterion:** Playing a song, you see green flashes on hit notes and red X
|
||||
markers scrolling past on missed notes.
|
||||
|
||||
**Estimated scope:** ~150 lines JS
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Diagnostic Labels (Timing + Pitch)
|
||||
|
||||
**Goal:** Show *why* a note was missed — too early, too late, sharp, flat.
|
||||
|
||||
**Files:**
|
||||
- `screen.js` in the notedetect plugin
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Extend draw hook to render timing indicators:
|
||||
- EARLY: orange `↑` + "-XXms" label above the miss marker
|
||||
- LATE: orange `↓` + "+XXms" label below the miss marker
|
||||
- Only shown when timing error exceeds `timingThresholdMs`
|
||||
- [ ] Extend draw hook to render pitch indicators:
|
||||
- SHARP: blue `♯` + "+XX¢" label
|
||||
- FLAT: blue `♭` + "-XX¢" label
|
||||
- Only shown when pitch error exceeds `pitchThresholdCents`
|
||||
- [ ] Compound states: stack timing label on top, pitch label below
|
||||
- [ ] Add settings UI in plugin settings panel for threshold configuration
|
||||
- [ ] Ensure labels don't overlap — offset vertically when multiple notes
|
||||
miss at close timestamps
|
||||
|
||||
**Exit criterion:** Playing intentionally early/late or bending sharp/flat
|
||||
shows the correct diagnostic labels.
|
||||
|
||||
**Estimated scope:** ~100 lines JS, ~30 lines settings HTML
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Loop Iteration Tracking
|
||||
|
||||
**Goal:** Track performance across loop iterations, show summary on each wrap.
|
||||
|
||||
**Files:**
|
||||
- `screen.js` in the notedetect plugin
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Detect loop wrap: `currentTime < previousTime - 0.5` in the frame update
|
||||
- [ ] On wrap: snapshot `{ hits, misses, total, percentage }` to `loopHistory[]`
|
||||
- [ ] Reset judgments for notes in `[loopA, loopB]` range (keep tracker alive
|
||||
for notes outside the loop)
|
||||
- [ ] Render loop summary overlay (top-center, semi-transparent background):
|
||||
```
|
||||
Loop N | X/Y notes (Z%) | Best: W%
|
||||
```
|
||||
Displayed for 1.5s, then fades.
|
||||
- [ ] Track `bestIteration` across all iterations for "Best" display
|
||||
- [ ] Emit `loop:complete` event via `window.slopsmith.emit()` so other plugins
|
||||
(practice journal) can record the data
|
||||
- [ ] Reset loop history when loop boundaries change or loop is cleared
|
||||
|
||||
**Exit criterion:** Looping a 4-bar phrase, you see iteration count and accuracy
|
||||
flash briefly at each loop wrap. Best score persists across iterations.
|
||||
|
||||
**Estimated scope:** ~120 lines JS
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Section Grading
|
||||
|
||||
**Goal:** Grade each song section (intro, verse, chorus, solo) and surface weak spots.
|
||||
|
||||
**Files:**
|
||||
- `screen.js` in the notedetect plugin
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Use `highway.getSections()` to identify section boundaries
|
||||
- [ ] Track hits/misses per section as notes are judged
|
||||
- [ ] At section boundaries (when `currentTime` crosses a section end),
|
||||
briefly flash the section grade:
|
||||
- A: 90%+, B: 75%+, C: 60%+, D: 40%+, F: below 40%
|
||||
- Color: green (A/B), yellow (C), red (D/F)
|
||||
- [ ] After song completes (or at any point via a hotkey), show a section
|
||||
summary panel listing all sections with grades
|
||||
- [ ] Highlight lowest-scoring section with a "Loop this section" button
|
||||
that sets A-B points to that section's boundaries
|
||||
- [ ] Emit `note:sectionGrade` event for other plugins
|
||||
|
||||
**Exit criterion:** Playing through a song, section grades flash at each
|
||||
transition. Lowest section is highlighted for targeted practice.
|
||||
|
||||
**Estimated scope:** ~150 lines JS, ~40 lines HTML
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Polish + Settings
|
||||
|
||||
**Goal:** Configurable thresholds, visual polish, performance.
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Full settings panel in plugin settings HTML:
|
||||
- Match window slider (100-500ms)
|
||||
- Pitch tolerance slider (20-100 cents)
|
||||
- Toggle timing/pitch labels
|
||||
- Toggle loop summary
|
||||
- Miss marker duration slider
|
||||
- [ ] Performance: ensure draw hook stays under 1ms per frame
|
||||
- Pre-compute judgment positions, don't recalculate in draw loop
|
||||
- Binary search over judgments by time (same pattern as `drawNotes`)
|
||||
- [ ] Smooth animations: glow/fade using eased alpha, not linear
|
||||
- [ ] Color-blind accessible palette option (use shapes not just colors)
|
||||
- [ ] Persist settings in plugin-local storage (e.g. `localStorage` prefixed with
|
||||
plugin id) — do **not** use `/api/settings` for this; the current server only
|
||||
persists a fixed set of known keys and will silently discard `notedetect_feedback`
|
||||
|
||||
**Estimated scope:** ~100 lines JS, ~60 lines HTML
|
||||
|
||||
---
|
||||
|
||||
## Dependency Graph
|
||||
|
||||
```
|
||||
Phase 0 (detection plugin)
|
||||
↓
|
||||
Phase 1 (matching core)
|
||||
↓
|
||||
Phase 2 (hit/miss overlay) ← Minimum viable feature
|
||||
↓
|
||||
Phase 3 (diagnostic labels)
|
||||
↓
|
||||
Phase 4 (loop tracking) ← Core practice value
|
||||
↓
|
||||
Phase 5 (section grading)
|
||||
↓
|
||||
Phase 6 (polish)
|
||||
```
|
||||
|
||||
Phases 3-5 are independent of each other and can be done in any order after Phase 2.
|
||||
Phase 6 should be last.
|
||||
|
||||
---
|
||||
|
||||
## Risk / Open Questions
|
||||
|
||||
1. **Latency budget:** Detection → WebSocket → matching → render adds latency.
|
||||
If total pipeline > 100ms, the match window needs to compensate with asymmetric
|
||||
tolerance (more lenient for "late" detections). Measure in Phase 1.
|
||||
|
||||
2. **Chord matching granularity:** Current plan matches chord notes individually.
|
||||
Should a chord be "missed" if 4/6 notes hit? Propose: grade chords as
|
||||
percentage, treat as HIT if ≥50% of notes matched. Revisit after Phase 2 testing.
|
||||
|
||||
3. **Tempo-scaled thresholds:** At 200 BPM, a 200ms match window covers almost
|
||||
an entire beat. Should thresholds scale with tempo? Propose: don't over-engineer
|
||||
this initially. Fixed thresholds work for most tempos. Revisit if users report
|
||||
issues at extreme tempos.
|
||||
|
||||
4. **Detection plugin availability:** Everything in Phases 1-6 degrades gracefully
|
||||
if the detection WebSocket isn't connected — the draw hook simply has no
|
||||
judgments to render, and the highway looks exactly as it does today.
|
||||
Reference in New Issue
Block a user