Compare commits

..
Author SHA1 Message Date
ChrisBeWithYouandClaude Fable 5 30f10c8045 feat(gp2rs): emit lyrics + vocal_pitch sidecars from GP vocal tracks
GP files with a vocal track previously produced only the legacy vocals
arrangement XML; the melody sidecar converter existed but was never
wired, so imported paks had no karaoke-ready pitch data.

- convert_file() now calls _emit_vocal_sidecars() after writing the
  vocals XML (best-effort: a sidecar failure never breaks conversion,
  matching the notation-sidecar posture)
- lyrics sidecar is projected from the just-built vocals XML so timing
  stays in lockstep; the XML lyric convention's trailing '+' (join) is
  mapped to the spec 7.1 '-' (join) - a passthrough would have turned
  every joined word into a line break
- convert_vocal_track_to_pitch_sidecar() gains require_lyric=False for
  lyric-less melody tracks (pitch sidecar only)
- attach_vocal_sidecars_to_sloppak() added as the assembly-step helper
  (companion to attach_notation_to_sloppak): writes the sidecars into a
  directory-form pak and sets manifest lyrics/lyrics_source/vocal_pitch
  with no-clobber guards
- gp2rs_gpx registered in the spec-conformance READERS list

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ChrisBeWithYou <chris@rifflarr.local>
2026-07-23 00:32:55 -05:00
8297afc449 feat(tools): per-platform VST3 slicing for rig content packs (#1025)
ship-ci / ci (push) Waiting to run
Rebased onto merged main (was stacked on #1023/#1024, whose venue work is
now in main) so it no longer carries a stale content_packs.py that would
revert 1023's build_pack fixes.

- build_vst_pack: slice a fat .vst3 tree to one platform (keep its binary
  dir + shared bundle files, drop the two foreign platform dirs and src/
  build trees). Pins create_system=3 like build_pack — without it the same
  tree hashes differently on a Windows runner (native .vst3 are built there),
  breaking the precomputable-hash guarantee exactly where it matters.
- Publish wiring: 'python tools/content_packs.py <vst-root> --vst --version N
  --publish' builds+uploads vst-<plat>-vN releases for mac/win/linux and emits
  a platform-keyed {url,sha256,bytes} manifest — the shape rig_builder's
  data/vst_packs.json consumes. publish() refactored onto a shared
  _publish_release helper (venue behaviour unchanged).
- Tests: slice keeps target+shared/drops foreign, per-platform binary,
  reproducibility, unknown-platform reject, and a simulated-win32 guard that
  fails if the create_system pin is dropped. selfcheck covers the VST path.

Original build_vst_pack by Matthew Harris Glover; reworked for the create_system
fix, publish wiring, and rebase.

Signed-off-by: byrongamatos <xasiklas@gmail.com>
Co-authored-by: byrongamatos <xasiklas@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:01:51 +02:00
5 changed files with 625 additions and 23 deletions
+225 -9
View File
@@ -12,6 +12,7 @@ Both are called transparently by gp2rs.py when the file extension is .gpx.
Do not call this module directly; use gp2rs.list_tracks / gp2rs.convert_file. Do not call this module directly; use gp2rs.list_tracks / gp2rs.convert_file.
""" """
import json
import logging import logging
import re import re
import struct import struct
@@ -1186,6 +1187,7 @@ def convert_vocal_track_to_pitch_sidecar(
*, *,
tempo_bpm: float = 120.0, tempo_bpm: float = 120.0,
audio_offset: float = 0.0, audio_offset: float = 0.0,
require_lyric: bool = True,
) -> dict: ) -> dict:
""" """
Extract per-syllable pitch from a GPX vocal track as a vocal_pitch.json dict. Extract per-syllable pitch from a GPX vocal track as a vocal_pitch.json dict.
@@ -1196,14 +1198,17 @@ def convert_vocal_track_to_pitch_sidecar(
{"version": 1, "notes": [{"t": float, "d": float, "midi": int}, ...]} {"version": 1, "notes": [{"t": float, "d": float, "midi": int}, ...]}
This is complementary to convert_vocal_track() which produces arrangement XML. This is complementary to convert_vocal_track() which produces arrangement
NOTE: nothing in this module calls this helper yet — convert_file() does not XML. convert_file() calls both for every vocal track and writes the result
invoke it, so no vocal_pitch.json is emitted automatically. A caller wanting as a ``<stem>.vocal_pitch.json`` sidecar next to the vocals XML (see
the pitch ribbon must call this itself and persist the returned dict (e.g. _emit_vocal_sidecars); the sloppak assembly step then attaches it via
write it as vocal_pitch.json into the sloppak). When wiring vocals into a attach_vocal_sidecars_to_sloppak.
sloppak, call both:
- convert_vocal_track() → vocals arrangement XML (karaoke highway) ``require_lyric`` (default True) keeps the feedpak-spec §7.2 alignment:
- convert_vocal_track_to_pitch_sidecar() → vocal_pitch.json (pitch ribbon) only beats carrying a lyric emit a note, so the pitch ribbon mirrors
lyrics.json token-for-token. Pass False for a lyric-less vocal track
(authored melody, no lyric text) to emit every pitched beat instead —
there are no lyric tokens to stay aligned with.
Pitch source is the tab author's authored notes (exact), not AI audio Pitch source is the tab author's authored notes (exact), not AI audio
analysis — so this is more accurate than pYIN/CREPE for well-authored tabs. analysis — so this is more accurate than pYIN/CREPE for well-authored tabs.
@@ -1258,13 +1263,15 @@ def convert_vocal_track_to_pitch_sidecar(
# Only emit notes that have a lyric — unvoiced beats # Only emit notes that have a lyric — unvoiced beats
# (rests, instrumental fills) are excluded so the # (rests, instrumental fills) are excluded so the
# pitch ribbon stays aligned with lyric tokens. # pitch ribbon stays aligned with lyric tokens.
# (Relaxed via require_lyric=False for lyric-less
# vocal tracks, where every pitched beat counts.)
lyric_el = beat_el.find('Lyrics') lyric_el = beat_el.find('Lyrics')
has_lyric = ( has_lyric = (
lyric_el is not None lyric_el is not None
and lyric_el.find('Line') is not None and lyric_el.find('Line') is not None
and (lyric_el.find('Line').text or '').strip() and (lyric_el.find('Line').text or '').strip()
) )
if not has_lyric: if require_lyric and not has_lyric:
voice_time += dur voice_time += dur
continue continue
@@ -1682,6 +1689,26 @@ def convert_file(
raise ValueError(f"unsafe output filename from track name: {track['name']!r}") raise ValueError(f"unsafe output filename from track name: {track['name']!r}")
filepath.write_text(xml_str, encoding="utf-8") filepath.write_text(xml_str, encoding="utf-8")
output_files.append(str(filepath)) output_files.append(str(filepath))
# Vocal tracks additionally get karaoke sidecars next to the XML
# (`<stem>.lyrics.json` + `<stem>.vocal_pitch.json`, feedpak spec
# §7.1/§7.2) so the sloppak assembly step can attach the `lyrics` /
# `vocal_pitch` manifest keys without re-walking the GP file —
# same pattern as the keys notation sidecar below. Best-effort: a
# sidecar bug must never break the vocals XML conversion itself.
try:
_emit_vocal_sidecars(
filepath, xml_str,
root, track, raw_idx,
masterbars, bars_by_id, voices_dict, beats_dict,
notes_dict, rhythms_dict,
tempo_bpm=tempo_bpm, audio_offset=audio_offset,
)
except Exception:
_log.exception(
"gp2rs_gpx: vocal sidecar emission failed for track %r "
"— vocals XML is unaffected", track['name'],
)
continue continue
# Iterate all masterbars and collect notes for this track # Iterate all masterbars and collect notes for this track
@@ -2403,6 +2430,195 @@ def _build_vocals_xml(
return dom.toprettyxml(indent=' ', encoding=None) return dom.toprettyxml(indent=' ', encoding=None)
# ---------------------------------------------------------------------------
# Vocal karaoke sidecars + manifest wiring (feedpak spec §7.1/§7.2)
# Mirrors the gp2notation sidecar pattern: convert_file writes the payloads
# next to the vocals XML (arrangement ids / the pak don't exist yet at convert
# time), and the sloppak assembly step moves them into the pak root + manifest
# via attach_vocal_sidecars_to_sloppak.
# ---------------------------------------------------------------------------
def lyrics_sidecar_path(xml_path: str | Path) -> Path:
"""``Voice_Vocals.xml`` → ``Voice_Vocals.lyrics.json`` (next to the XML)."""
p = Path(xml_path)
return p.with_name(p.stem + ".lyrics.json")
def vocal_pitch_sidecar_path(xml_path: str | Path) -> Path:
"""``Voice_Vocals.xml`` → ``Voice_Vocals.vocal_pitch.json`` (next to the XML)."""
p = Path(xml_path)
return p.with_name(p.stem + ".vocal_pitch.json")
def _vocals_xml_to_lyrics(xml_str: str) -> list[dict]:
"""Project a ``<vocals>`` arrangement XML into the flat lyrics.json shape
(feedpak spec §7.1): ``[{"t": float, "d": float, "w": str}, ...]``.
Deriving from the XML (rather than re-walking the GP tree) guarantees the
two stay in lockstep — same tie extension, same rounding, same beats.
Suffix conversion: the XML lyric convention and feedpak disagree on ``+``.
In the vocals XML a trailing ``+`` means "connect to next token" (a join),
while feedpak ``+`` marks the last syllable of a LINE — so a pass-through
would turn every joined syllable into a line break. Joins map to feedpak's
trailing ``-`` instead; a trailing ``-`` already means the same thing in
both. Line-end ``+`` markers are never emitted: GP stores lyrics per beat
with no line structure, so there is nothing to derive them from.
"""
try:
root = ET.fromstring(xml_str)
except ET.ParseError:
return []
lyrics: list[dict] = []
for v in root.iter('vocal'):
w = (v.get('lyric') or '').strip()
if w.endswith('+'):
w = w[:-1] + '-'
# A bare joiner token isn't a syllable (spec: suffixes ride on real
# syllables, never standalone entries) — skip it.
if not w or w in ('-', '+'):
continue
try:
t = float(v.get('time', ''))
d = float(v.get('length', ''))
except (TypeError, ValueError):
continue
lyrics.append({'t': round(t, 3), 'd': round(d, 3), 'w': w})
return lyrics
def _emit_vocal_sidecars(
xml_path: Path,
xml_str: str,
root: ET.Element,
track: dict,
raw_idx: int,
masterbars: list,
bars_by_id: dict,
voices_dict: dict,
beats_dict: dict,
notes_dict: dict,
rhythms_dict: dict,
*,
tempo_bpm: float = 120.0,
audio_offset: float = 0.0,
) -> list[Path]:
"""Write the karaoke sidecars for one converted vocal track.
``<stem>.lyrics.json`` — only when the track actually carries lyric text
(derived from the vocals XML just built, so timings match exactly).
``<stem>.vocal_pitch.json`` — whenever the track has pitched beats. With
lyrics present the notes stay lyric-aligned (spec §7.2: one entry per
syllable); for a lyric-less melody track the lyric gate is dropped so the
authored pitch still ships.
Returns the sidecar paths written (possibly empty).
"""
written: list[Path] = []
lyrics = _vocals_xml_to_lyrics(xml_str)
if lyrics:
side = lyrics_sidecar_path(xml_path)
side.write_text(json.dumps(lyrics, separators=(",", ":")),
encoding="utf-8")
written.append(side)
pitch = convert_vocal_track_to_pitch_sidecar(
root, track, raw_idx,
masterbars, bars_by_id, voices_dict, beats_dict,
notes_dict, rhythms_dict,
tempo_bpm=tempo_bpm, audio_offset=audio_offset,
require_lyric=bool(lyrics),
)
if pitch.get('notes'):
side = vocal_pitch_sidecar_path(xml_path)
side.write_text(json.dumps(pitch, separators=(",", ":")),
encoding="utf-8")
written.append(side)
return written
def attach_vocal_sidecars_to_sloppak(
sloppak_dir: str | Path,
*,
lyrics: list | None = None,
vocal_pitch: dict | None = None,
lyrics_source: str = "authored",
) -> list[Path]:
"""Write ``lyrics.json`` / ``vocal_pitch.json`` into a directory-form
sloppak and point the top-level manifest ``lyrics`` / ``lyrics_source`` /
``vocal_pitch`` keys at them (feedpak spec §7.1/§7.2).
Vocal companion to gp2notation.attach_notation_to_sloppak, with the same
manifest round-trip caveat (PyYAML ``safe_load`` + ``safe_dump`` — key
order survives, comments don't). GP-derived payloads are ``authored``
provenance, so no ``lyric_transcription`` / ``pitch_extraction`` blocks
are written (the spec reserves those for automated engines).
Never clobbers: a payload whose manifest key is already set (or whose
target file already exists) is skipped, so a pak that already carries
lyrics/pitch — hand-edited or machine-extracted — is left alone.
Raises ``ValueError`` on a malformed payload, an unknown
``lyrics_source``, or a manifest that isn't a mapping. Returns the paths
actually written.
"""
import yaml
if lyrics_source not in ("authored", "transcribed", "user"):
raise ValueError(
f"lyrics_source must be authored/transcribed/user, got {lyrics_source!r}")
if lyrics is not None and not (
isinstance(lyrics, list) and all(
isinstance(e, dict)
and isinstance(e.get('w'), str)
and isinstance(e.get('t'), (int, float))
and isinstance(e.get('d'), (int, float))
for e in lyrics
)
):
raise ValueError("lyrics must be a list of {t, d, w} syllable dicts")
if vocal_pitch is not None and not (
isinstance(vocal_pitch, dict)
and isinstance(vocal_pitch.get('notes'), list)
):
raise ValueError("vocal_pitch must be a dict with a `notes` list")
pak = Path(sloppak_dir)
manifest_path = pak / "manifest.yaml"
manifest = yaml.safe_load(manifest_path.read_text(encoding="utf-8"))
if not isinstance(manifest, dict):
raise ValueError(f"{manifest_path} is not a mapping")
written: list[Path] = []
if lyrics and not manifest.get("lyrics") and not (pak / "lyrics.json").exists():
(pak / "lyrics.json").write_text(
json.dumps(lyrics, separators=(",", ":")), encoding="utf-8")
manifest["lyrics"] = "lyrics.json"
manifest["lyrics_source"] = lyrics_source
written.append(pak / "lyrics.json")
if (vocal_pitch and vocal_pitch.get("notes")
and not manifest.get("vocal_pitch")
and not (pak / "vocal_pitch.json").exists()):
(pak / "vocal_pitch.json").write_text(
json.dumps(vocal_pitch, separators=(",", ":")), encoding="utf-8")
manifest["vocal_pitch"] = "vocal_pitch.json"
written.append(pak / "vocal_pitch.json")
if written:
# Stamp the format version while we're rewriting the manifest (spec
# §4), without downgrading an existing declared version.
from sloppak import FEEDPAK_VERSION
manifest.setdefault("feedpak_version", FEEDPAK_VERSION)
manifest_path.write_text(
yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True),
encoding="utf-8",
)
return written
def _gpx_tuning(track: dict) -> list[int]: def _gpx_tuning(track: dict) -> list[int]:
"""Compute RS tuning offsets (semitones from standard) from GPX string pitches.""" """Compute RS tuning offsets (semitones from standard) from GPX string pitches."""
from gp2rs import STANDARD_TUNING_GUITAR, STANDARD_TUNING_BASS from gp2rs import STANDARD_TUNING_GUITAR, STANDARD_TUNING_BASS
+91
View File
@@ -0,0 +1,91 @@
"""VST-pack slicing (tools/content_packs.build_vst_pack): each platform pack
keeps only its own binaries + the shared bundle files, drops the rest, and is
reproducible."""
import zipfile
from pathlib import Path
from tools import content_packs
def _fake_vst_tree(root: Path):
# One fat .vst3 with all three platform binaries + shared files, plus a
# src/ build tree that must never ship.
c = root / "amps" / "Foo.vst3" / "Contents"
(c / "MacOS").mkdir(parents=True)
(c / "x86_64-win").mkdir(parents=True)
(c / "x86_64-linux").mkdir(parents=True)
(c / "Resources").mkdir(parents=True)
(c / "MacOS" / "Foo").write_bytes(b"mac-binary")
(c / "x86_64-win" / "Foo.vst3").write_bytes(b"win-binary")
(c / "x86_64-linux" / "Foo.so").write_bytes(b"linux-binary")
(c / "Info.plist").write_bytes(b"<plist/>")
(c / "Resources" / "moduleinfo.json").write_bytes(b"{}")
(root / "src" / "build").mkdir(parents=True)
(root / "src" / "build" / "junk.o").write_bytes(b"objfile")
def _names(zip_path):
with zipfile.ZipFile(zip_path) as zf:
return set(zf.namelist())
def test_slice_keeps_target_platform_and_shared_drops_foreign(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
content_packs.build_vst_pack(root, tmp_path / "mac.zip", "mac")
names = _names(tmp_path / "mac.zip")
base = "amps/Foo.vst3/Contents"
assert f"{base}/MacOS/Foo" in names # target binary kept
assert f"{base}/Info.plist" in names # shared kept
assert f"{base}/Resources/moduleinfo.json" in names # shared kept
assert f"{base}/x86_64-win/Foo.vst3" not in names # foreign dropped
assert f"{base}/x86_64-linux/Foo.so" not in names # foreign dropped
assert not any(n.startswith("src/") for n in names) # build trees never ship
def test_each_platform_gets_its_own_binary(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
wanted = {"mac": "MacOS/Foo", "win": "x86_64-win/Foo.vst3", "linux": "x86_64-linux/Foo.so"}
for plat, rel in wanted.items():
content_packs.build_vst_pack(root, tmp_path / f"{plat}.zip", plat)
names = _names(tmp_path / f"{plat}.zip")
assert f"amps/Foo.vst3/Contents/{rel}" in names
others = [v for k, v in wanted.items() if k != plat]
for o in others:
assert f"amps/Foo.vst3/Contents/{o}" not in names
def test_slice_is_reproducible(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
a = content_packs.build_vst_pack(root, tmp_path / "a.zip", "linux")
b = content_packs.build_vst_pack(root, tmp_path / "b.zip", "linux")
assert a == b and a["sha256"]
def test_slice_pins_create_system_for_cross_runner_reproducibility(tmp_path, monkeypatch):
# ZipInfo defaults create_system from the host OS (0 on Windows, 3 on Unix),
# and it lands in the central directory — so without an explicit pin the same
# tree hashes differently on a Windows runner, breaking the precomputable-hash
# guarantee exactly where it matters (native .vst3 are built on Windows). A
# same-machine reproducibility test can't catch that; simulate win32 and
# assert the pin forces 3 regardless.
monkeypatch.setattr(zipfile.sys, "platform", "win32")
root = tmp_path / "vst"
_fake_vst_tree(root)
content_packs.build_vst_pack(root, tmp_path / "w.zip", "linux")
with zipfile.ZipFile(tmp_path / "w.zip") as zf:
assert all(i.create_system == 3 for i in zf.infolist())
def test_unknown_platform_rejected(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
try:
content_packs.build_vst_pack(root, tmp_path / "x.zip", "bsd")
except ValueError as e:
assert "unknown platform" in str(e)
else:
raise AssertionError("build_vst_pack accepted an unknown platform")
+191
View File
@@ -7,6 +7,7 @@ the PR; here we pin the input-validation guards and the conversion helpers
that are easy to drive without a fixture. that are easy to drive without a fixture.
""" """
import json
import struct import struct
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
@@ -29,6 +30,10 @@ from gp2rs_gpx import (
_GPX_MAX_DECOMPRESSED, _GPX_MAX_DECOMPRESSED,
_find_piano_pairs, _find_piano_pairs,
convert_vocal_track_to_pitch_sidecar, convert_vocal_track_to_pitch_sidecar,
_vocals_xml_to_lyrics,
attach_vocal_sidecars_to_sloppak,
lyrics_sidecar_path,
vocal_pitch_sidecar_path,
_collect_tone_events, _collect_tone_events,
_inject_tones, _inject_tones,
_resolve_pending_slides, _resolve_pending_slides,
@@ -386,6 +391,192 @@ def test_vocal_pitch_sidecar_skips_beat_without_lyric():
assert out == {"version": 1, "notes": []} assert out == {"version": 1, "notes": []}
def test_vocal_pitch_sidecar_require_lyric_false_emits_melody():
# Lyric-less vocal track: with the gate relaxed the authored pitch still
# ships (there are no lyric tokens to stay aligned with).
out = convert_vocal_track_to_pitch_sidecar(
**_vocal_sidecar_args(with_lyric=False), require_lyric=False)
assert out == {"version": 1, "notes": [{"t": 0.0, "d": 0.5, "midi": 60}]}
# ── _vocals_xml_to_lyrics ───────────────────────────────────────────────────
def test_vocals_xml_to_lyrics_shape_and_suffixes():
xml = (
'<vocals count="4">'
'<vocal time="0.000" note="60" length="0.500" lyric="Hel-"/>'
'<vocal time="0.500" note="62" length="0.500" lyric="lo"/>'
'<vocal time="1.000" note="64" length="0.250" lyric="sing+"/>'
'<vocal time="1.250" note="64" length="0.250" lyric="ing"/>'
'</vocals>'
)
out = _vocals_xml_to_lyrics(xml)
assert out == [
# "-" means the same join in both conventions — passed through.
{"t": 0.0, "d": 0.5, "w": "Hel-"},
{"t": 0.5, "d": 0.5, "w": "lo"},
# XML "+" is a JOIN; feedpak "+" is a LINE END — joins become "-".
{"t": 1.0, "d": 0.25, "w": "sing-"},
{"t": 1.25, "d": 0.25, "w": "ing"},
]
def test_vocals_xml_to_lyrics_skips_bare_joiners_and_bad_xml():
xml = (
'<vocals count="2">'
'<vocal time="0.000" note="0" length="0.500" lyric="+"/>'
'<vocal time="0.500" note="60" length="0.500" lyric="la"/>'
'</vocals>'
)
assert _vocals_xml_to_lyrics(xml) == [{"t": 0.5, "d": 0.5, "w": "la"}]
assert _vocals_xml_to_lyrics("not xml <<<") == []
# ── convert_file end-to-end: vocal karaoke sidecars ─────────────────────────
# A vocal track must emit `<stem>.lyrics.json` + `<stem>.vocal_pitch.json`
# next to the vocals XML; non-vocal tracks must not; a lyric-less vocal track
# emits the pitch sidecar only.
_GPIF_VOCAL = """
<GPIF>
<Score><Title>T</Title><Artist>A</Artist></Score>
<Tracks>
<Track id="0"><Name>Vocals</Name>
<Property name="Tuning"><Pitches>60</Pitches></Property></Track>
</Tracks>
<MasterBars><MasterBar><Time>4/4</Time><Bars>0</Bars></MasterBar></MasterBars>
<Bars><Bar id="0"><Voices>0</Voices></Bar></Bars>
<Voices><Voice id="0"><Beats>0 1</Beats></Voice></Voices>
<Beats>
<Beat id="0"><Rhythm ref="r0"/><Lyrics><Line>Hel-</Line></Lyrics><Notes>0</Notes></Beat>
<Beat id="1"><Rhythm ref="r0"/><Lyrics><Line>lo</Line></Lyrics><Notes>1</Notes></Beat>
</Beats>
<Notes>
<Note id="0">
<Property name="String"><String>0</String></Property>
<Property name="Fret"><Fret>0</Fret></Property></Note>
<Note id="1">
<Property name="String"><String>0</String></Property>
<Property name="Fret"><Fret>2</Fret></Property></Note>
</Notes>
<Rhythms><Rhythm id="r0"><NoteValue>Quarter</NoteValue></Rhythm></Rhythms>
</GPIF>
"""
# Same melody, no <Lyrics> anywhere.
_GPIF_VOCAL_NO_LYRICS = _GPIF_VOCAL.replace(
"<Lyrics><Line>Hel-</Line></Lyrics>", "").replace(
"<Lyrics><Line>lo</Line></Lyrics>", "")
def test_convert_file_vocal_track_emits_both_sidecars(tmp_path, monkeypatch):
monkeypatch.setattr(gp2rs_gpx, "_load_gpif",
lambda _p: ET.fromstring(_GPIF_VOCAL))
out_files = convert_file("dummy.gpx", str(tmp_path), track_indices=[0])
assert len(out_files) == 1
assert ET.parse(out_files[0]).getroot().tag == "vocals"
# lyrics.json sidecar: flat [{t, d, w}] (spec §7.1), timings from the XML.
lyr = json.loads(lyrics_sidecar_path(out_files[0]).read_text(encoding="utf-8"))
assert lyr == [
{"t": 0.0, "d": 0.5, "w": "Hel-"},
{"t": 0.5, "d": 0.5, "w": "lo"},
]
# vocal_pitch.json sidecar: {version, notes:[{t, d, midi}]} (spec §7.2),
# lyric-aligned (one note per syllable) at the authored pitches.
pitch = json.loads(vocal_pitch_sidecar_path(out_files[0]).read_text(encoding="utf-8"))
assert pitch == {"version": 1, "notes": [
{"t": 0.0, "d": 0.5, "midi": 60},
{"t": 0.5, "d": 0.5, "midi": 62},
]}
def test_convert_file_no_vocal_track_no_sidecars(tmp_path, monkeypatch):
monkeypatch.setattr(gp2rs_gpx, "_load_gpif",
lambda _p: ET.fromstring(_GPIF_GUITAR_ASCENDING))
convert_file("dummy.gp", str(tmp_path),
track_indices=[0], arrangement_names={0: "Lead"})
assert not list(tmp_path.glob("*.lyrics.json"))
assert not list(tmp_path.glob("*.vocal_pitch.json"))
def test_convert_file_lyricless_vocal_track_pitch_sidecar_only(tmp_path, monkeypatch):
monkeypatch.setattr(gp2rs_gpx, "_load_gpif",
lambda _p: ET.fromstring(_GPIF_VOCAL_NO_LYRICS))
out_files = convert_file("dummy.gpx", str(tmp_path), track_indices=[0])
assert len(out_files) == 1
# No lyric text anywhere -> no lyrics.json; the authored melody still
# ships as vocal_pitch.json (lyric gate relaxed for lyric-less tracks).
assert not lyrics_sidecar_path(out_files[0]).exists()
pitch = json.loads(vocal_pitch_sidecar_path(out_files[0]).read_text(encoding="utf-8"))
assert pitch == {"version": 1, "notes": [
{"t": 0.0, "d": 0.5, "midi": 60},
{"t": 0.5, "d": 0.5, "midi": 62},
]}
# ── attach_vocal_sidecars_to_sloppak ────────────────────────────────────────
_LYRICS_PAYLOAD = [{"t": 0.0, "d": 0.5, "w": "Hel-"}, {"t": 0.5, "d": 0.5, "w": "lo"}]
_PITCH_PAYLOAD = {"version": 1, "notes": [{"t": 0.0, "d": 0.5, "midi": 60}]}
def _make_pak(tmp_path, manifest: dict):
import yaml
pak = tmp_path / "pak"
pak.mkdir()
(pak / "manifest.yaml").write_text(
yaml.safe_dump(manifest, sort_keys=False), encoding="utf-8")
return pak
def test_attach_vocal_sidecars_writes_files_and_manifest(tmp_path):
import yaml
pak = _make_pak(tmp_path, {"title": "T", "arrangements": []})
written = attach_vocal_sidecars_to_sloppak(
pak, lyrics=_LYRICS_PAYLOAD, vocal_pitch=_PITCH_PAYLOAD)
assert written == [pak / "lyrics.json", pak / "vocal_pitch.json"]
assert json.loads((pak / "lyrics.json").read_text(encoding="utf-8")) == _LYRICS_PAYLOAD
assert json.loads((pak / "vocal_pitch.json").read_text(encoding="utf-8")) == _PITCH_PAYLOAD
manifest = yaml.safe_load((pak / "manifest.yaml").read_text(encoding="utf-8"))
assert manifest["lyrics"] == "lyrics.json"
assert manifest["lyrics_source"] == "authored" # GP tab = authored chart
assert manifest["vocal_pitch"] == "vocal_pitch.json"
# No automated-engine provenance for authored payloads (spec §7.1.1/§7.2.1).
assert "lyric_transcription" not in manifest
assert "pitch_extraction" not in manifest
assert "feedpak_version" in manifest
def test_attach_vocal_sidecars_never_clobbers(tmp_path):
import yaml
pak = _make_pak(tmp_path, {
"title": "T",
"lyrics": "existing_lyrics.json",
"lyrics_source": "user",
})
written = attach_vocal_sidecars_to_sloppak(
pak, lyrics=_LYRICS_PAYLOAD, vocal_pitch=_PITCH_PAYLOAD)
# lyrics already claimed by the manifest -> skipped entirely; pitch is new.
assert written == [pak / "vocal_pitch.json"]
assert not (pak / "lyrics.json").exists()
manifest = yaml.safe_load((pak / "manifest.yaml").read_text(encoding="utf-8"))
assert manifest["lyrics"] == "existing_lyrics.json"
assert manifest["lyrics_source"] == "user"
def test_attach_vocal_sidecars_rejects_bad_payloads(tmp_path):
pak = _make_pak(tmp_path, {"title": "T"})
with pytest.raises(ValueError):
attach_vocal_sidecars_to_sloppak(pak, lyrics=[{"t": 0.0}]) # missing d/w
with pytest.raises(ValueError):
attach_vocal_sidecars_to_sloppak(pak, vocal_pitch={"version": 1}) # no notes
with pytest.raises(ValueError):
attach_vocal_sidecars_to_sloppak(
pak, lyrics=_LYRICS_PAYLOAD, lyrics_source="whisperx") # not spec enum
# ── _collect_tone_events ──────────────────────────────────────────────────── # ── _collect_tone_events ────────────────────────────────────────────────────
def _tone_args(banks, tempo_map=((0, 120.0),)): def _tone_args(banks, tempo_map=((0, 120.0),)):
+1
View File
@@ -58,6 +58,7 @@ READERS = [
"lib/enrichment.py", "lib/enrichment.py",
"lib/songmeta.py", "lib/songmeta.py",
"lib/gp2notation.py", # rewrites manifest.yaml; stamps feedpak_version "lib/gp2notation.py", # rewrites manifest.yaml; stamps feedpak_version
"lib/gp2rs_gpx.py", # attach_vocal_sidecars_to_sloppak: lyrics/vocal_pitch keys
"lib/routers/ws_highway.py", # reads `authors` off a feedpak manifest "lib/routers/ws_highway.py", # reads `authors` off a feedpak manifest
"lib/routers/chart.py", # Get-info panel: binds `m = load_manifest(...)` "lib/routers/chart.py", # Get-info panel: binds `m = load_manifest(...)`
"lib/routers/song.py", # enrichment gap-fill: reads the manifest directly "lib/routers/song.py", # enrichment gap-fill: reads the manifest directly
+107 -4
View File
@@ -74,6 +74,52 @@ def build_pack(src_dir: Path, out_zip: Path) -> dict:
return {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)} return {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)}
# rig_builder ships "fat" .vst3 bundles carrying all three platforms inside
# Contents/. A pack for one platform keeps that platform's binary dir + the
# shared bundle files, and drops the other two.
VST_PLATFORM_DIRS = {"mac": "MacOS", "win": "x86_64-win", "linux": "x86_64-linux"}
def build_vst_pack(vst_root: Path, out_zip: Path, platform: str) -> dict:
"""Reproducibly zip the VST tree keeping only `platform`'s binaries.
Slices each fat .vst3: everything is kept except the two foreign platform
dirs (MacOS / x86_64-win / x86_64-linux) and the vst/src build trees. Arc
names are relative to vst_root so the download endpoint extracts straight
into <plugin>/vst/. Same reproducible-build guarantees as build_pack.
"""
if platform not in VST_PLATFORM_DIRS:
raise ValueError(f"unknown platform {platform!r} (want mac/win/linux)")
foreign = set(VST_PLATFORM_DIRS.values()) - {VST_PLATFORM_DIRS[platform]}
files = []
for p in sorted(vst_root.rglob("*"), key=lambda q: q.as_posix()):
if not p.is_file():
continue
rel = p.relative_to(vst_root)
if rel.parts and rel.parts[0] == "src": # skip C++/JUCE build trees
continue
if set(rel.parts) & foreign: # drop foreign-platform binaries
continue
files.append((p, rel))
if not files:
raise ValueError(f"no VST files to pack for {platform} in {vst_root}")
out_zip.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(out_zip, "w", zipfile.ZIP_STORED) as zf:
for p, rel in files:
info = zipfile.ZipInfo(rel.as_posix(), date_time=(1980, 1, 1, 0, 0, 0))
info.compress_type = zipfile.ZIP_STORED
# Pin create_system like build_pack: ZipInfo defaults it from the
# host OS (0 on Windows, 3 on Unix), which would otherwise make the
# same pack hash differently across runners. VST packs are the most
# likely to be built on Windows (native .vst3), so without this pin
# the precomputable-hash guarantee breaks exactly where it's needed.
info.create_system = 3
info.external_attr = 0o644 << 16
zf.writestr(info, p.read_bytes())
data = out_zip.read_bytes()
return {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)}
def manifest_entry(out_zip: Path, url: str) -> dict: def manifest_entry(out_zip: Path, url: str) -> dict:
"""Pack info as the download-path expects it: {url, sha256, bytes}.""" """Pack info as the download-path expects it: {url, sha256, bytes}."""
return {"url": url, return {"url": url,
@@ -96,25 +142,47 @@ def pack_url(pack_id: str, version: int, repo: str = REPO) -> str:
f"{pack_tag(pack_id, version)}/{pack_asset(pack_id, version)}") f"{pack_tag(pack_id, version)}/{pack_asset(pack_id, version)}")
def publish(pack_id: str, version: int, zip_path: Path, repo: str = REPO) -> None: # VST packs use the same immutable per-pack convention, keyed by platform:
# tag `vst-<plat>-v<N>`, asset `vst-<plat>-pack-v<N>.zip`. The manifest they
# emit is keyed by platform (mac/win/linux) — the shape the rig_builder plugin's
# data/vst_packs.json consumes.
def vst_tag(platform: str, version: int) -> str:
return f"vst-{platform}-v{version}"
def vst_asset(platform: str, version: int) -> str:
return f"vst-{platform}-pack-v{version}.zip"
def vst_url(platform: str, version: int, repo: str = REPO) -> str:
return (f"https://github.com/{repo}/releases/download/"
f"{vst_tag(platform, version)}/{vst_asset(platform, version)}")
def _publish_release(tag: str, zip_path: Path, title: str, notes: str,
repo: str = REPO) -> None:
"""Create the per-pack release if missing, then upload the versioned zip. """Create the per-pack release if missing, then upload the versioned zip.
Tags are immutable: a media change means a new version (v1 → v2), never a Tags are immutable: a media change means a new version (v1 → v2), never a
re-upload — so no --clobber. gh errors if the asset already exists, which is re-upload — so no --clobber. gh errors if the asset already exists, which is
the right guard against overwriting a published, referenced pack. the right guard against overwriting a published, referenced pack.
""" """
tag = pack_tag(pack_id, version)
if subprocess.run(["gh", "release", "view", tag, "--repo", repo], if subprocess.run(["gh", "release", "view", tag, "--repo", repo],
capture_output=True).returncode != 0: capture_output=True).returncode != 0:
subprocess.run( subprocess.run(
["gh", "release", "create", tag, "--repo", repo, "--latest=false", ["gh", "release", "create", tag, "--repo", repo, "--latest=false",
"--title", f"{pack_id.capitalize()} venue pack v{version}", "--title", title, "--notes", notes],
"--notes", "Opt-in career venue pack. Not a code release."],
check=True) check=True)
subprocess.run( subprocess.run(
["gh", "release", "upload", tag, str(zip_path), "--repo", repo], check=True) ["gh", "release", "upload", tag, str(zip_path), "--repo", repo], check=True)
def publish(pack_id: str, version: int, zip_path: Path, repo: str = REPO) -> None:
_publish_release(pack_tag(pack_id, version), zip_path,
f"{pack_id.capitalize()} venue pack v{version}",
"Opt-in career venue pack. Not a code release.", repo)
def _pack_id(src_dir: Path) -> str: def _pack_id(src_dir: Path) -> str:
return src_dir.name return src_dir.name
@@ -129,6 +197,10 @@ def main(argv=None) -> int:
help="write zips here + a file:// manifest.json; no upload") help="write zips here + a file:// manifest.json; no upload")
ap.add_argument("--publish", action="store_true", ap.add_argument("--publish", action="store_true",
help="create/upload the per-pack release; emit release URLs") help="create/upload the per-pack release; emit release URLs")
ap.add_argument("--vst", action="store_true",
help="slice one rig VST root (src[0]) into per-platform "
"vst-<plat>-v<N> packs; manifest keyed by platform "
"(the shape rig_builder's data/vst_packs.json wants)")
ap.add_argument("--manifest", type=Path, ap.add_argument("--manifest", type=Path,
help="write the {id: {url,sha256,bytes}} map here (default: stdout)") help="write the {id: {url,sha256,bytes}} map here (default: stdout)")
ap.add_argument("--selfcheck", action="store_true", help="run the round-trip demo and exit") ap.add_argument("--selfcheck", action="store_true", help="run the round-trip demo and exit")
@@ -141,6 +213,20 @@ def main(argv=None) -> int:
out_dir = args.local if args.local else Path(args.src[0]).parent / "_packs" out_dir = args.local if args.local else Path(args.src[0]).parent / "_packs"
manifest = {} manifest = {}
if args.vst:
vst_root = args.src[0]
for plat in VST_PLATFORM_DIRS:
zip_path = out_dir / vst_asset(plat, args.version)
build_vst_pack(vst_root, zip_path, plat)
if args.publish:
_publish_release(vst_tag(plat, args.version), zip_path,
f"Rig VST pack ({plat}) v{args.version}",
"Opt-in per-platform rig VST pack. Not a code release.")
url = vst_url(plat, args.version)
else:
url = (out_dir.resolve() / zip_path.name).as_uri()
manifest[plat] = manifest_entry(zip_path, url)
else:
for src in args.src: for src in args.src:
pid = _pack_id(src) pid = _pack_id(src)
zip_path = out_dir / pack_asset(pid, args.version) zip_path = out_dir / pack_asset(pid, args.version)
@@ -183,6 +269,23 @@ def _selfcheck() -> int:
with zipfile.ZipFile(zip_path) as zf: with zipfile.ZipFile(zip_path) as zf:
names = zf.namelist() names = zf.namelist()
assert set(names) == {"manifest.json", "bored.mp4"}, names assert set(names) == {"manifest.json", "bored.mp4"}, names
# VST slice: keep target platform + shared, drop foreign, reproducible.
c = td / "vst" / "Foo.vst3" / "Contents"
for d in ("MacOS", "x86_64-win", "x86_64-linux", "Resources"):
(c / d).mkdir(parents=True)
(c / "MacOS" / "Foo").write_bytes(b"mac")
(c / "x86_64-linux" / "Foo.so").write_bytes(b"linux")
(c / "Info.plist").write_bytes(b"<plist/>")
vzip = td / vst_asset("linux", 1)
vinfo = build_vst_pack(td / "vst", vzip, "linux")
assert vinfo == build_vst_pack(td / "vst", td / "v2.zip", "linux"), \
"vst slice is not reproducible"
with zipfile.ZipFile(vzip) as zf:
vnames = set(zf.namelist())
assert "Foo.vst3/Contents/x86_64-linux/Foo.so" in vnames
assert "Foo.vst3/Contents/Info.plist" in vnames
assert not any("MacOS" in n for n in vnames), vnames
print("content_packs selfcheck: ok") print("content_packs selfcheck: ok")
return 0 return 0