mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-07-28 23:52:16 +00:00
fix(gp_autosync): slope-constrained DTW steps — stop path collapse on riff-based songs (#791)
librosa.sequence.dtw's default step sizes permit pure horizontal/vertical moves; on songs whose chroma is self-similar for long stretches the flat cost surface let the path collapse (minutes of score onto one audio frame), so auto-sync produced monotonic-but-garbage sync points and the per-bar warp imported charts badly out of sync while reporting success. Use the standard music-sync step pattern [[1,1],[1,2],[2,1]] (local tempo ratio bounded to 0.5x-2x), falling back to unconstrained steps if the global length ratio makes it infeasible. Validated on the reported song (138 BPM tab, YouTube audio): coarse points now track 1:1, refine holds slopes 0.77-1.04, warped downbeats hit onset peaks at 3.3x background energy. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
d567fd5597
commit
1a8540935b
@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **Auto-sync: DTW step constraint — riff-based songs no longer produce garbage sync points.** `librosa.sequence.dtw`'s default step pattern allows unbounded horizontal/vertical path runs, and on music with long self-similar chroma stretches (riff-driven stoner/doom, drone sections) the flat cost surface let the warping path collapse — minutes of score mapped onto a single audio frame, so the per-bar warp imported charts wildly out of sync while reporting success (observed on a real 138 BPM tab: effective displayed tempo 159 BPM, three sync points sharing one audio timestamp). `_dtw_align` now uses the standard music-sync slope-constrained step pattern (`[[1,1],[1,2],[2,1]]`, local tempo ratio bounded to 0.5x–2x), which makes the degenerate path impossible, with a fallback to unconstrained steps when the global length ratio makes the constrained pattern infeasible (e.g. a tab aligned against a full-concert video). Validated on the failing song: coarse points track the recording 1:1, refined downbeats land on onset peaks at 3.3x background energy.
|
||||
|
||||
### Added
|
||||
- **"Colorblind (deuteranope)" highway string-color preset.** Adds a one-click preset to the shared "Highway String Colors" picker, sitting next to the existing Okabe–Ito "Colorblind-friendly" preset — contributed by a deuteranopic player who found the Okabe–Ito set still hard to separate. It retunes the six main strings (red / yellow-green / blue / orange / teal / deep-purple) and keeps that set's 7/8-string colors, and applies to **both** the 2D and 3D highways via the shared picker. Frontend-only, additive: `static/app.js` (`HWC_PRESETS`).
|
||||
- **`lib/gp_autosync.py`: piecewise time-warp helpers + a working `refine_sync()`.** `auto_sync()` has always computed per-bar sync points (DTW), but consumers could only apply the scalar bar-1 `audio_offset`, so any tempo difference between the recording and the tab's authored tempo accumulated audibly over the song. New librosa-free helpers expose the full mapping: `bar_start_times(gp_path)` (per-bar score times on the same axis as the sync points — GPIF bar-resolution map for `.gp`/`.gpx`, per-tick integration for GP3/4/5), `build_warp_anchors(points, bar_starts)` (strictly-monotonic `(score, audio)` anchor pairs), `warp_time(t, anchors)` (piecewise-linear map with edge-slope extrapolation for count-ins/tails), `warp_song_times(song, warp)` (retimes a `lib.song.Song` in place: notes + sustains, chords, beats, sections, anchors, handshapes, per-phrase difficulty levels, tone changes, tempo overrides), and `gp_has_expandable_repeats(gp_path)` (detects GP3/4/5 repeat/volta/direction markup whose playback expansion the as-written sync points cannot map — callers fall back to offset-only sync). Also implements `refine_sync()`, which the editor plugin's refine-sync endpoint has imported since the snapshot but which never existed in core (the Refine button 500'd): it densifies the coarse DTW points to every Nth bar and re-times each with a local onset phase sweep (sweep radius clamped under half a beat so periodic material can't lock a full beat off; short scoring grid + median residual snap). Synthetic click-track validation: ~13ms mean / ~40ms max error from ±180ms coarse input across 117–123 BPM recordings of a 120 BPM tab. Tests: `tests/test_gp_autosync_warp.py`.
|
||||
|
||||
@ -538,9 +538,32 @@ def _dtw_align(
|
||||
Returns wp where wp[i] = [score_frame_index, audio_frame_index].
|
||||
"""
|
||||
import librosa
|
||||
import numpy as np
|
||||
cs = _safe_normalise(chroma_score)
|
||||
ca = _safe_normalise(chroma_audio)
|
||||
_D, wp = librosa.sequence.dtw(cs, ca, metric='cosine')
|
||||
# Slope-constrained step pattern ([[1,1],[1,2],[2,1]], Müller's standard
|
||||
# music-sync config): every step advances BOTH axes, bounding the local
|
||||
# tempo ratio to 0.5x-2x. librosa's default steps allow pure
|
||||
# horizontal/vertical runs, and on riff-based music (long self-similar
|
||||
# chroma stretches, e.g. stoner/doom) the flat cost surface let the path
|
||||
# collapse — whole minutes of score mapped onto a single audio frame,
|
||||
# producing garbage sync points. The constrained pattern makes that
|
||||
# degenerate path impossible.
|
||||
steps = np.array([[1, 1], [1, 2], [2, 1]])
|
||||
weights = np.array([1.0, 1.0, 1.0])
|
||||
try:
|
||||
_D, wp = librosa.sequence.dtw(
|
||||
cs, ca, metric='cosine',
|
||||
step_sizes_sigma=steps, weights_mul=weights,
|
||||
)
|
||||
except Exception as exc:
|
||||
# The constrained pattern needs the global length ratio within its
|
||||
# 0.5x-2x slope bounds; a pathological pairing (e.g. a 3-minute tab
|
||||
# against a 20-minute video) is infeasible and librosa raises. Fall
|
||||
# back to the unconstrained path rather than failing the whole sync.
|
||||
_log.warning("gp_autosync: constrained DTW infeasible (%s) — "
|
||||
"falling back to unconstrained steps", exc)
|
||||
_D, wp = librosa.sequence.dtw(cs, ca, metric='cosine')
|
||||
return wp[::-1] # reverse to forward order
|
||||
|
||||
# ── Sync point extraction from DTW path ──────────────────────────────────────
|
||||
|
||||
Loading…
Reference in New Issue
Block a user