From b10bad351c22b0d5f6e667a7d79b98f2021cfde8 Mon Sep 17 00:00:00 2001 From: byrongamatos Date: Sun, 5 Jul 2026 22:12:02 +0200 Subject: [PATCH] =?UTF-8?q?fix(gp=5Fautosync):=20slope-constrained=20DTW?= =?UTF-8?q?=20steps=20=E2=80=94=20stop=20path=20collapse=20on=20riff-based?= =?UTF-8?q?=20songs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit librosa.sequence.dtw's default step sizes permit pure horizontal/vertical moves; on songs whose chroma is self-similar for long stretches the flat cost surface let the path collapse (minutes of score onto one audio frame), so auto-sync produced monotonic-but-garbage sync points and the per-bar warp imported charts badly out of sync while reporting success. Use the standard music-sync step pattern [[1,1],[1,2],[2,1]] (local tempo ratio bounded to 0.5x-2x), falling back to unconstrained steps if the global length ratio makes it infeasible. Validated on the reported song (138 BPM tab, YouTube audio): coarse points now track 1:1, refine holds slopes 0.77-1.04, warped downbeats hit onset peaks at 3.3x background energy. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 3 +++ lib/gp_autosync.py | 25 ++++++++++++++++++++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7dfe03c..8a6c462 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed +- **Auto-sync: DTW step constraint — riff-based songs no longer produce garbage sync points.** `librosa.sequence.dtw`'s default step pattern allows unbounded horizontal/vertical path runs, and on music with long self-similar chroma stretches (riff-driven stoner/doom, drone sections) the flat cost surface let the warping path collapse — minutes of score mapped onto a single audio frame, so the per-bar warp imported charts wildly out of sync while reporting success (observed on a real 138 BPM tab: effective displayed tempo 159 BPM, three sync points sharing one audio timestamp). `_dtw_align` now uses the standard music-sync slope-constrained step pattern (`[[1,1],[1,2],[2,1]]`, local tempo ratio bounded to 0.5x–2x), which makes the degenerate path impossible, with a fallback to unconstrained steps when the global length ratio makes the constrained pattern infeasible (e.g. a tab aligned against a full-concert video). Validated on the failing song: coarse points track the recording 1:1, refined downbeats land on onset peaks at 3.3x background energy. + ### Added - **"Colorblind (deuteranope)" highway string-color preset.** Adds a one-click preset to the shared "Highway String Colors" picker, sitting next to the existing Okabe–Ito "Colorblind-friendly" preset — contributed by a deuteranopic player who found the Okabe–Ito set still hard to separate. It retunes the six main strings (red / yellow-green / blue / orange / teal / deep-purple) and keeps that set's 7/8-string colors, and applies to **both** the 2D and 3D highways via the shared picker. Frontend-only, additive: `static/app.js` (`HWC_PRESETS`). - **`lib/gp_autosync.py`: piecewise time-warp helpers + a working `refine_sync()`.** `auto_sync()` has always computed per-bar sync points (DTW), but consumers could only apply the scalar bar-1 `audio_offset`, so any tempo difference between the recording and the tab's authored tempo accumulated audibly over the song. New librosa-free helpers expose the full mapping: `bar_start_times(gp_path)` (per-bar score times on the same axis as the sync points — GPIF bar-resolution map for `.gp`/`.gpx`, per-tick integration for GP3/4/5), `build_warp_anchors(points, bar_starts)` (strictly-monotonic `(score, audio)` anchor pairs), `warp_time(t, anchors)` (piecewise-linear map with edge-slope extrapolation for count-ins/tails), `warp_song_times(song, warp)` (retimes a `lib.song.Song` in place: notes + sustains, chords, beats, sections, anchors, handshapes, per-phrase difficulty levels, tone changes, tempo overrides), and `gp_has_expandable_repeats(gp_path)` (detects GP3/4/5 repeat/volta/direction markup whose playback expansion the as-written sync points cannot map — callers fall back to offset-only sync). Also implements `refine_sync()`, which the editor plugin's refine-sync endpoint has imported since the snapshot but which never existed in core (the Refine button 500'd): it densifies the coarse DTW points to every Nth bar and re-times each with a local onset phase sweep (sweep radius clamped under half a beat so periodic material can't lock a full beat off; short scoring grid + median residual snap). Synthetic click-track validation: ~13ms mean / ~40ms max error from ±180ms coarse input across 117–123 BPM recordings of a 120 BPM tab. Tests: `tests/test_gp_autosync_warp.py`. diff --git a/lib/gp_autosync.py b/lib/gp_autosync.py index cd689be..f3b2547 100644 --- a/lib/gp_autosync.py +++ b/lib/gp_autosync.py @@ -538,9 +538,32 @@ def _dtw_align( Returns wp where wp[i] = [score_frame_index, audio_frame_index]. """ import librosa + import numpy as np cs = _safe_normalise(chroma_score) ca = _safe_normalise(chroma_audio) - _D, wp = librosa.sequence.dtw(cs, ca, metric='cosine') + # Slope-constrained step pattern ([[1,1],[1,2],[2,1]], Müller's standard + # music-sync config): every step advances BOTH axes, bounding the local + # tempo ratio to 0.5x-2x. librosa's default steps allow pure + # horizontal/vertical runs, and on riff-based music (long self-similar + # chroma stretches, e.g. stoner/doom) the flat cost surface let the path + # collapse — whole minutes of score mapped onto a single audio frame, + # producing garbage sync points. The constrained pattern makes that + # degenerate path impossible. + steps = np.array([[1, 1], [1, 2], [2, 1]]) + weights = np.array([1.0, 1.0, 1.0]) + try: + _D, wp = librosa.sequence.dtw( + cs, ca, metric='cosine', + step_sizes_sigma=steps, weights_mul=weights, + ) + except Exception as exc: + # The constrained pattern needs the global length ratio within its + # 0.5x-2x slope bounds; a pathological pairing (e.g. a 3-minute tab + # against a 20-minute video) is infeasible and librosa raises. Fall + # back to the unconstrained path rather than failing the whole sync. + _log.warning("gp_autosync: constrained DTW infeasible (%s) — " + "falling back to unconstrained steps", exc) + _D, wp = librosa.sequence.dtw(cs, ca, metric='cosine') return wp[::-1] # reverse to forward order # ── Sync point extraction from DTW path ──────────────────────────────────────