Compare commits

..
Author SHA1 Message Date
ChrisBeWithYouandClaude Fable 5 30f10c8045 feat(gp2rs): emit lyrics + vocal_pitch sidecars from GP vocal tracks
GP files with a vocal track previously produced only the legacy vocals
arrangement XML; the melody sidecar converter existed but was never
wired, so imported paks had no karaoke-ready pitch data.

- convert_file() now calls _emit_vocal_sidecars() after writing the
  vocals XML (best-effort: a sidecar failure never breaks conversion,
  matching the notation-sidecar posture)
- lyrics sidecar is projected from the just-built vocals XML so timing
  stays in lockstep; the XML lyric convention's trailing '+' (join) is
  mapped to the spec 7.1 '-' (join) - a passthrough would have turned
  every joined word into a line break
- convert_vocal_track_to_pitch_sidecar() gains require_lyric=False for
  lyric-less melody tracks (pitch sidecar only)
- attach_vocal_sidecars_to_sloppak() added as the assembly-step helper
  (companion to attach_notation_to_sloppak): writes the sidecars into a
  directory-form pak and sets manifest lyrics/lyrics_source/vocal_pitch
  with no-clobber guards
- gp2rs_gpx registered in the spec-conformance READERS list

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ChrisBeWithYou <chris@rifflarr.local>
2026-07-23 00:32:55 -05:00
8297afc449 feat(tools): per-platform VST3 slicing for rig content packs (#1025)
ship-ci / ci (push) Waiting to run
Rebased onto merged main (was stacked on #1023/#1024, whose venue work is
now in main) so it no longer carries a stale content_packs.py that would
revert 1023's build_pack fixes.

- build_vst_pack: slice a fat .vst3 tree to one platform (keep its binary
  dir + shared bundle files, drop the two foreign platform dirs and src/
  build trees). Pins create_system=3 like build_pack — without it the same
  tree hashes differently on a Windows runner (native .vst3 are built there),
  breaking the precomputable-hash guarantee exactly where it matters.
- Publish wiring: 'python tools/content_packs.py <vst-root> --vst --version N
  --publish' builds+uploads vst-<plat>-vN releases for mac/win/linux and emits
  a platform-keyed {url,sha256,bytes} manifest — the shape rig_builder's
  data/vst_packs.json consumes. publish() refactored onto a shared
  _publish_release helper (venue behaviour unchanged).
- Tests: slice keeps target+shared/drops foreign, per-platform binary,
  reproducibility, unknown-platform reject, and a simulated-win32 guard that
  fails if the create_system pin is dropped. selfcheck covers the VST path.

Original build_vst_pack by Matthew Harris Glover; reworked for the create_system
fix, publish wiring, and rebase.

Signed-off-by: byrongamatos <xasiklas@gmail.com>
Co-authored-by: byrongamatos <xasiklas@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:01:51 +02:00
5 changed files with 625 additions and 23 deletions
+225 -9
View File
@@ -12,6 +12,7 @@ Both are called transparently by gp2rs.py when the file extension is .gpx.
Do not call this module directly; use gp2rs.list_tracks / gp2rs.convert_file.
"""
import json
import logging
import re
import struct
@@ -1186,6 +1187,7 @@ def convert_vocal_track_to_pitch_sidecar(
*,
tempo_bpm: float = 120.0,
audio_offset: float = 0.0,
require_lyric: bool = True,
) -> dict:
"""
Extract per-syllable pitch from a GPX vocal track as a vocal_pitch.json dict.
@@ -1196,14 +1198,17 @@ def convert_vocal_track_to_pitch_sidecar(
{"version": 1, "notes": [{"t": float, "d": float, "midi": int}, ...]}
This is complementary to convert_vocal_track() which produces arrangement XML.
NOTE: nothing in this module calls this helper yet — convert_file() does not
invoke it, so no vocal_pitch.json is emitted automatically. A caller wanting
the pitch ribbon must call this itself and persist the returned dict (e.g.
write it as vocal_pitch.json into the sloppak). When wiring vocals into a
sloppak, call both:
- convert_vocal_track() → vocals arrangement XML (karaoke highway)
- convert_vocal_track_to_pitch_sidecar() → vocal_pitch.json (pitch ribbon)
This is complementary to convert_vocal_track() which produces arrangement
XML. convert_file() calls both for every vocal track and writes the result
as a ``<stem>.vocal_pitch.json`` sidecar next to the vocals XML (see
_emit_vocal_sidecars); the sloppak assembly step then attaches it via
attach_vocal_sidecars_to_sloppak.
``require_lyric`` (default True) keeps the feedpak-spec §7.2 alignment:
only beats carrying a lyric emit a note, so the pitch ribbon mirrors
lyrics.json token-for-token. Pass False for a lyric-less vocal track
(authored melody, no lyric text) to emit every pitched beat instead —
there are no lyric tokens to stay aligned with.
Pitch source is the tab author's authored notes (exact), not AI audio
analysis — so this is more accurate than pYIN/CREPE for well-authored tabs.
@@ -1258,13 +1263,15 @@ def convert_vocal_track_to_pitch_sidecar(
# Only emit notes that have a lyric — unvoiced beats
# (rests, instrumental fills) are excluded so the
# pitch ribbon stays aligned with lyric tokens.
# (Relaxed via require_lyric=False for lyric-less
# vocal tracks, where every pitched beat counts.)
lyric_el = beat_el.find('Lyrics')
has_lyric = (
lyric_el is not None
and lyric_el.find('Line') is not None
and (lyric_el.find('Line').text or '').strip()
)
if not has_lyric:
if require_lyric and not has_lyric:
voice_time += dur
continue
@@ -1682,6 +1689,26 @@ def convert_file(
raise ValueError(f"unsafe output filename from track name: {track['name']!r}")
filepath.write_text(xml_str, encoding="utf-8")
output_files.append(str(filepath))
# Vocal tracks additionally get karaoke sidecars next to the XML
# (`<stem>.lyrics.json` + `<stem>.vocal_pitch.json`, feedpak spec
# §7.1/§7.2) so the sloppak assembly step can attach the `lyrics` /
# `vocal_pitch` manifest keys without re-walking the GP file —
# same pattern as the keys notation sidecar below. Best-effort: a
# sidecar bug must never break the vocals XML conversion itself.
try:
_emit_vocal_sidecars(
filepath, xml_str,
root, track, raw_idx,
masterbars, bars_by_id, voices_dict, beats_dict,
notes_dict, rhythms_dict,
tempo_bpm=tempo_bpm, audio_offset=audio_offset,
)
except Exception:
_log.exception(
"gp2rs_gpx: vocal sidecar emission failed for track %r "
"— vocals XML is unaffected", track['name'],
)
continue
# Iterate all masterbars and collect notes for this track
@@ -2403,6 +2430,195 @@ def _build_vocals_xml(
return dom.toprettyxml(indent=' ', encoding=None)
# ---------------------------------------------------------------------------
# Vocal karaoke sidecars + manifest wiring (feedpak spec §7.1/§7.2)
# Mirrors the gp2notation sidecar pattern: convert_file writes the payloads
# next to the vocals XML (arrangement ids / the pak don't exist yet at convert
# time), and the sloppak assembly step moves them into the pak root + manifest
# via attach_vocal_sidecars_to_sloppak.
# ---------------------------------------------------------------------------
def lyrics_sidecar_path(xml_path: str | Path) -> Path:
"""``Voice_Vocals.xml`` → ``Voice_Vocals.lyrics.json`` (next to the XML)."""
p = Path(xml_path)
return p.with_name(p.stem + ".lyrics.json")
def vocal_pitch_sidecar_path(xml_path: str | Path) -> Path:
"""``Voice_Vocals.xml`` → ``Voice_Vocals.vocal_pitch.json`` (next to the XML)."""
p = Path(xml_path)
return p.with_name(p.stem + ".vocal_pitch.json")
def _vocals_xml_to_lyrics(xml_str: str) -> list[dict]:
"""Project a ``<vocals>`` arrangement XML into the flat lyrics.json shape
(feedpak spec §7.1): ``[{"t": float, "d": float, "w": str}, ...]``.
Deriving from the XML (rather than re-walking the GP tree) guarantees the
two stay in lockstep — same tie extension, same rounding, same beats.
Suffix conversion: the XML lyric convention and feedpak disagree on ``+``.
In the vocals XML a trailing ``+`` means "connect to next token" (a join),
while feedpak ``+`` marks the last syllable of a LINE — so a pass-through
would turn every joined syllable into a line break. Joins map to feedpak's
trailing ``-`` instead; a trailing ``-`` already means the same thing in
both. Line-end ``+`` markers are never emitted: GP stores lyrics per beat
with no line structure, so there is nothing to derive them from.
"""
try:
root = ET.fromstring(xml_str)
except ET.ParseError:
return []
lyrics: list[dict] = []
for v in root.iter('vocal'):
w = (v.get('lyric') or '').strip()
if w.endswith('+'):
w = w[:-1] + '-'
# A bare joiner token isn't a syllable (spec: suffixes ride on real
# syllables, never standalone entries) — skip it.
if not w or w in ('-', '+'):
continue
try:
t = float(v.get('time', ''))
d = float(v.get('length', ''))
except (TypeError, ValueError):
continue
lyrics.append({'t': round(t, 3), 'd': round(d, 3), 'w': w})
return lyrics
def _emit_vocal_sidecars(
xml_path: Path,
xml_str: str,
root: ET.Element,
track: dict,
raw_idx: int,
masterbars: list,
bars_by_id: dict,
voices_dict: dict,
beats_dict: dict,
notes_dict: dict,
rhythms_dict: dict,
*,
tempo_bpm: float = 120.0,
audio_offset: float = 0.0,
) -> list[Path]:
"""Write the karaoke sidecars for one converted vocal track.
``<stem>.lyrics.json`` — only when the track actually carries lyric text
(derived from the vocals XML just built, so timings match exactly).
``<stem>.vocal_pitch.json`` — whenever the track has pitched beats. With
lyrics present the notes stay lyric-aligned (spec §7.2: one entry per
syllable); for a lyric-less melody track the lyric gate is dropped so the
authored pitch still ships.
Returns the sidecar paths written (possibly empty).
"""
written: list[Path] = []
lyrics = _vocals_xml_to_lyrics(xml_str)
if lyrics:
side = lyrics_sidecar_path(xml_path)
side.write_text(json.dumps(lyrics, separators=(",", ":")),
encoding="utf-8")
written.append(side)
pitch = convert_vocal_track_to_pitch_sidecar(
root, track, raw_idx,
masterbars, bars_by_id, voices_dict, beats_dict,
notes_dict, rhythms_dict,
tempo_bpm=tempo_bpm, audio_offset=audio_offset,
require_lyric=bool(lyrics),
)
if pitch.get('notes'):
side = vocal_pitch_sidecar_path(xml_path)
side.write_text(json.dumps(pitch, separators=(",", ":")),
encoding="utf-8")
written.append(side)
return written
def attach_vocal_sidecars_to_sloppak(
sloppak_dir: str | Path,
*,
lyrics: list | None = None,
vocal_pitch: dict | None = None,
lyrics_source: str = "authored",
) -> list[Path]:
"""Write ``lyrics.json`` / ``vocal_pitch.json`` into a directory-form
sloppak and point the top-level manifest ``lyrics`` / ``lyrics_source`` /
``vocal_pitch`` keys at them (feedpak spec §7.1/§7.2).
Vocal companion to gp2notation.attach_notation_to_sloppak, with the same
manifest round-trip caveat (PyYAML ``safe_load`` + ``safe_dump`` — key
order survives, comments don't). GP-derived payloads are ``authored``
provenance, so no ``lyric_transcription`` / ``pitch_extraction`` blocks
are written (the spec reserves those for automated engines).
Never clobbers: a payload whose manifest key is already set (or whose
target file already exists) is skipped, so a pak that already carries
lyrics/pitch — hand-edited or machine-extracted — is left alone.
Raises ``ValueError`` on a malformed payload, an unknown
``lyrics_source``, or a manifest that isn't a mapping. Returns the paths
actually written.
"""
import yaml
if lyrics_source not in ("authored", "transcribed", "user"):
raise ValueError(
f"lyrics_source must be authored/transcribed/user, got {lyrics_source!r}")
if lyrics is not None and not (
isinstance(lyrics, list) and all(
isinstance(e, dict)
and isinstance(e.get('w'), str)
and isinstance(e.get('t'), (int, float))
and isinstance(e.get('d'), (int, float))
for e in lyrics
)
):
raise ValueError("lyrics must be a list of {t, d, w} syllable dicts")
if vocal_pitch is not None and not (
isinstance(vocal_pitch, dict)
and isinstance(vocal_pitch.get('notes'), list)
):
raise ValueError("vocal_pitch must be a dict with a `notes` list")
pak = Path(sloppak_dir)
manifest_path = pak / "manifest.yaml"
manifest = yaml.safe_load(manifest_path.read_text(encoding="utf-8"))
if not isinstance(manifest, dict):
raise ValueError(f"{manifest_path} is not a mapping")
written: list[Path] = []
if lyrics and not manifest.get("lyrics") and not (pak / "lyrics.json").exists():
(pak / "lyrics.json").write_text(
json.dumps(lyrics, separators=(",", ":")), encoding="utf-8")
manifest["lyrics"] = "lyrics.json"
manifest["lyrics_source"] = lyrics_source
written.append(pak / "lyrics.json")
if (vocal_pitch and vocal_pitch.get("notes")
and not manifest.get("vocal_pitch")
and not (pak / "vocal_pitch.json").exists()):
(pak / "vocal_pitch.json").write_text(
json.dumps(vocal_pitch, separators=(",", ":")), encoding="utf-8")
manifest["vocal_pitch"] = "vocal_pitch.json"
written.append(pak / "vocal_pitch.json")
if written:
# Stamp the format version while we're rewriting the manifest (spec
# §4), without downgrading an existing declared version.
from sloppak import FEEDPAK_VERSION
manifest.setdefault("feedpak_version", FEEDPAK_VERSION)
manifest_path.write_text(
yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True),
encoding="utf-8",
)
return written
def _gpx_tuning(track: dict) -> list[int]:
"""Compute RS tuning offsets (semitones from standard) from GPX string pitches."""
from gp2rs import STANDARD_TUNING_GUITAR, STANDARD_TUNING_BASS
+91
View File
@@ -0,0 +1,91 @@
"""VST-pack slicing (tools/content_packs.build_vst_pack): each platform pack
keeps only its own binaries + the shared bundle files, drops the rest, and is
reproducible."""
import zipfile
from pathlib import Path
from tools import content_packs
def _fake_vst_tree(root: Path):
# One fat .vst3 with all three platform binaries + shared files, plus a
# src/ build tree that must never ship.
c = root / "amps" / "Foo.vst3" / "Contents"
(c / "MacOS").mkdir(parents=True)
(c / "x86_64-win").mkdir(parents=True)
(c / "x86_64-linux").mkdir(parents=True)
(c / "Resources").mkdir(parents=True)
(c / "MacOS" / "Foo").write_bytes(b"mac-binary")
(c / "x86_64-win" / "Foo.vst3").write_bytes(b"win-binary")
(c / "x86_64-linux" / "Foo.so").write_bytes(b"linux-binary")
(c / "Info.plist").write_bytes(b"<plist/>")
(c / "Resources" / "moduleinfo.json").write_bytes(b"{}")
(root / "src" / "build").mkdir(parents=True)
(root / "src" / "build" / "junk.o").write_bytes(b"objfile")
def _names(zip_path):
with zipfile.ZipFile(zip_path) as zf:
return set(zf.namelist())
def test_slice_keeps_target_platform_and_shared_drops_foreign(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
content_packs.build_vst_pack(root, tmp_path / "mac.zip", "mac")
names = _names(tmp_path / "mac.zip")
base = "amps/Foo.vst3/Contents"
assert f"{base}/MacOS/Foo" in names # target binary kept
assert f"{base}/Info.plist" in names # shared kept
assert f"{base}/Resources/moduleinfo.json" in names # shared kept
assert f"{base}/x86_64-win/Foo.vst3" not in names # foreign dropped
assert f"{base}/x86_64-linux/Foo.so" not in names # foreign dropped
assert not any(n.startswith("src/") for n in names) # build trees never ship
def test_each_platform_gets_its_own_binary(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
wanted = {"mac": "MacOS/Foo", "win": "x86_64-win/Foo.vst3", "linux": "x86_64-linux/Foo.so"}
for plat, rel in wanted.items():
content_packs.build_vst_pack(root, tmp_path / f"{plat}.zip", plat)
names = _names(tmp_path / f"{plat}.zip")
assert f"amps/Foo.vst3/Contents/{rel}" in names
others = [v for k, v in wanted.items() if k != plat]
for o in others:
assert f"amps/Foo.vst3/Contents/{o}" not in names
def test_slice_is_reproducible(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
a = content_packs.build_vst_pack(root, tmp_path / "a.zip", "linux")
b = content_packs.build_vst_pack(root, tmp_path / "b.zip", "linux")
assert a == b and a["sha256"]
def test_slice_pins_create_system_for_cross_runner_reproducibility(tmp_path, monkeypatch):
# ZipInfo defaults create_system from the host OS (0 on Windows, 3 on Unix),
# and it lands in the central directory — so without an explicit pin the same
# tree hashes differently on a Windows runner, breaking the precomputable-hash
# guarantee exactly where it matters (native .vst3 are built on Windows). A
# same-machine reproducibility test can't catch that; simulate win32 and
# assert the pin forces 3 regardless.
monkeypatch.setattr(zipfile.sys, "platform", "win32")
root = tmp_path / "vst"
_fake_vst_tree(root)
content_packs.build_vst_pack(root, tmp_path / "w.zip", "linux")
with zipfile.ZipFile(tmp_path / "w.zip") as zf:
assert all(i.create_system == 3 for i in zf.infolist())
def test_unknown_platform_rejected(tmp_path):
root = tmp_path / "vst"
_fake_vst_tree(root)
try:
content_packs.build_vst_pack(root, tmp_path / "x.zip", "bsd")
except ValueError as e:
assert "unknown platform" in str(e)
else:
raise AssertionError("build_vst_pack accepted an unknown platform")
+191
View File
@@ -7,6 +7,7 @@ the PR; here we pin the input-validation guards and the conversion helpers
that are easy to drive without a fixture.
"""
import json
import struct
import xml.etree.ElementTree as ET
@@ -29,6 +30,10 @@ from gp2rs_gpx import (
_GPX_MAX_DECOMPRESSED,
_find_piano_pairs,
convert_vocal_track_to_pitch_sidecar,
_vocals_xml_to_lyrics,
attach_vocal_sidecars_to_sloppak,
lyrics_sidecar_path,
vocal_pitch_sidecar_path,
_collect_tone_events,
_inject_tones,
_resolve_pending_slides,
@@ -386,6 +391,192 @@ def test_vocal_pitch_sidecar_skips_beat_without_lyric():
assert out == {"version": 1, "notes": []}
def test_vocal_pitch_sidecar_require_lyric_false_emits_melody():
# Lyric-less vocal track: with the gate relaxed the authored pitch still
# ships (there are no lyric tokens to stay aligned with).
out = convert_vocal_track_to_pitch_sidecar(
**_vocal_sidecar_args(with_lyric=False), require_lyric=False)
assert out == {"version": 1, "notes": [{"t": 0.0, "d": 0.5, "midi": 60}]}
# ── _vocals_xml_to_lyrics ───────────────────────────────────────────────────
def test_vocals_xml_to_lyrics_shape_and_suffixes():
xml = (
'<vocals count="4">'
'<vocal time="0.000" note="60" length="0.500" lyric="Hel-"/>'
'<vocal time="0.500" note="62" length="0.500" lyric="lo"/>'
'<vocal time="1.000" note="64" length="0.250" lyric="sing+"/>'
'<vocal time="1.250" note="64" length="0.250" lyric="ing"/>'
'</vocals>'
)
out = _vocals_xml_to_lyrics(xml)
assert out == [
# "-" means the same join in both conventions — passed through.
{"t": 0.0, "d": 0.5, "w": "Hel-"},
{"t": 0.5, "d": 0.5, "w": "lo"},
# XML "+" is a JOIN; feedpak "+" is a LINE END — joins become "-".
{"t": 1.0, "d": 0.25, "w": "sing-"},
{"t": 1.25, "d": 0.25, "w": "ing"},
]
def test_vocals_xml_to_lyrics_skips_bare_joiners_and_bad_xml():
xml = (
'<vocals count="2">'
'<vocal time="0.000" note="0" length="0.500" lyric="+"/>'
'<vocal time="0.500" note="60" length="0.500" lyric="la"/>'
'</vocals>'
)
assert _vocals_xml_to_lyrics(xml) == [{"t": 0.5, "d": 0.5, "w": "la"}]
assert _vocals_xml_to_lyrics("not xml <<<") == []
# ── convert_file end-to-end: vocal karaoke sidecars ─────────────────────────
# A vocal track must emit `<stem>.lyrics.json` + `<stem>.vocal_pitch.json`
# next to the vocals XML; non-vocal tracks must not; a lyric-less vocal track
# emits the pitch sidecar only.
_GPIF_VOCAL = """
<GPIF>
<Score><Title>T</Title><Artist>A</Artist></Score>
<Tracks>
<Track id="0"><Name>Vocals</Name>
<Property name="Tuning"><Pitches>60</Pitches></Property></Track>
</Tracks>
<MasterBars><MasterBar><Time>4/4</Time><Bars>0</Bars></MasterBar></MasterBars>
<Bars><Bar id="0"><Voices>0</Voices></Bar></Bars>
<Voices><Voice id="0"><Beats>0 1</Beats></Voice></Voices>
<Beats>
<Beat id="0"><Rhythm ref="r0"/><Lyrics><Line>Hel-</Line></Lyrics><Notes>0</Notes></Beat>
<Beat id="1"><Rhythm ref="r0"/><Lyrics><Line>lo</Line></Lyrics><Notes>1</Notes></Beat>
</Beats>
<Notes>
<Note id="0">
<Property name="String"><String>0</String></Property>
<Property name="Fret"><Fret>0</Fret></Property></Note>
<Note id="1">
<Property name="String"><String>0</String></Property>
<Property name="Fret"><Fret>2</Fret></Property></Note>
</Notes>
<Rhythms><Rhythm id="r0"><NoteValue>Quarter</NoteValue></Rhythm></Rhythms>
</GPIF>
"""
# Same melody, no <Lyrics> anywhere.
_GPIF_VOCAL_NO_LYRICS = _GPIF_VOCAL.replace(
"<Lyrics><Line>Hel-</Line></Lyrics>", "").replace(
"<Lyrics><Line>lo</Line></Lyrics>", "")
def test_convert_file_vocal_track_emits_both_sidecars(tmp_path, monkeypatch):
monkeypatch.setattr(gp2rs_gpx, "_load_gpif",
lambda _p: ET.fromstring(_GPIF_VOCAL))
out_files = convert_file("dummy.gpx", str(tmp_path), track_indices=[0])
assert len(out_files) == 1
assert ET.parse(out_files[0]).getroot().tag == "vocals"
# lyrics.json sidecar: flat [{t, d, w}] (spec §7.1), timings from the XML.
lyr = json.loads(lyrics_sidecar_path(out_files[0]).read_text(encoding="utf-8"))
assert lyr == [
{"t": 0.0, "d": 0.5, "w": "Hel-"},
{"t": 0.5, "d": 0.5, "w": "lo"},
]
# vocal_pitch.json sidecar: {version, notes:[{t, d, midi}]} (spec §7.2),
# lyric-aligned (one note per syllable) at the authored pitches.
pitch = json.loads(vocal_pitch_sidecar_path(out_files[0]).read_text(encoding="utf-8"))
assert pitch == {"version": 1, "notes": [
{"t": 0.0, "d": 0.5, "midi": 60},
{"t": 0.5, "d": 0.5, "midi": 62},
]}
def test_convert_file_no_vocal_track_no_sidecars(tmp_path, monkeypatch):
monkeypatch.setattr(gp2rs_gpx, "_load_gpif",
lambda _p: ET.fromstring(_GPIF_GUITAR_ASCENDING))
convert_file("dummy.gp", str(tmp_path),
track_indices=[0], arrangement_names={0: "Lead"})
assert not list(tmp_path.glob("*.lyrics.json"))
assert not list(tmp_path.glob("*.vocal_pitch.json"))
def test_convert_file_lyricless_vocal_track_pitch_sidecar_only(tmp_path, monkeypatch):
monkeypatch.setattr(gp2rs_gpx, "_load_gpif",
lambda _p: ET.fromstring(_GPIF_VOCAL_NO_LYRICS))
out_files = convert_file("dummy.gpx", str(tmp_path), track_indices=[0])
assert len(out_files) == 1
# No lyric text anywhere -> no lyrics.json; the authored melody still
# ships as vocal_pitch.json (lyric gate relaxed for lyric-less tracks).
assert not lyrics_sidecar_path(out_files[0]).exists()
pitch = json.loads(vocal_pitch_sidecar_path(out_files[0]).read_text(encoding="utf-8"))
assert pitch == {"version": 1, "notes": [
{"t": 0.0, "d": 0.5, "midi": 60},
{"t": 0.5, "d": 0.5, "midi": 62},
]}
# ── attach_vocal_sidecars_to_sloppak ────────────────────────────────────────
_LYRICS_PAYLOAD = [{"t": 0.0, "d": 0.5, "w": "Hel-"}, {"t": 0.5, "d": 0.5, "w": "lo"}]
_PITCH_PAYLOAD = {"version": 1, "notes": [{"t": 0.0, "d": 0.5, "midi": 60}]}
def _make_pak(tmp_path, manifest: dict):
import yaml
pak = tmp_path / "pak"
pak.mkdir()
(pak / "manifest.yaml").write_text(
yaml.safe_dump(manifest, sort_keys=False), encoding="utf-8")
return pak
def test_attach_vocal_sidecars_writes_files_and_manifest(tmp_path):
import yaml
pak = _make_pak(tmp_path, {"title": "T", "arrangements": []})
written = attach_vocal_sidecars_to_sloppak(
pak, lyrics=_LYRICS_PAYLOAD, vocal_pitch=_PITCH_PAYLOAD)
assert written == [pak / "lyrics.json", pak / "vocal_pitch.json"]
assert json.loads((pak / "lyrics.json").read_text(encoding="utf-8")) == _LYRICS_PAYLOAD
assert json.loads((pak / "vocal_pitch.json").read_text(encoding="utf-8")) == _PITCH_PAYLOAD
manifest = yaml.safe_load((pak / "manifest.yaml").read_text(encoding="utf-8"))
assert manifest["lyrics"] == "lyrics.json"
assert manifest["lyrics_source"] == "authored" # GP tab = authored chart
assert manifest["vocal_pitch"] == "vocal_pitch.json"
# No automated-engine provenance for authored payloads (spec §7.1.1/§7.2.1).
assert "lyric_transcription" not in manifest
assert "pitch_extraction" not in manifest
assert "feedpak_version" in manifest
def test_attach_vocal_sidecars_never_clobbers(tmp_path):
import yaml
pak = _make_pak(tmp_path, {
"title": "T",
"lyrics": "existing_lyrics.json",
"lyrics_source": "user",
})
written = attach_vocal_sidecars_to_sloppak(
pak, lyrics=_LYRICS_PAYLOAD, vocal_pitch=_PITCH_PAYLOAD)
# lyrics already claimed by the manifest -> skipped entirely; pitch is new.
assert written == [pak / "vocal_pitch.json"]
assert not (pak / "lyrics.json").exists()
manifest = yaml.safe_load((pak / "manifest.yaml").read_text(encoding="utf-8"))
assert manifest["lyrics"] == "existing_lyrics.json"
assert manifest["lyrics_source"] == "user"
def test_attach_vocal_sidecars_rejects_bad_payloads(tmp_path):
pak = _make_pak(tmp_path, {"title": "T"})
with pytest.raises(ValueError):
attach_vocal_sidecars_to_sloppak(pak, lyrics=[{"t": 0.0}]) # missing d/w
with pytest.raises(ValueError):
attach_vocal_sidecars_to_sloppak(pak, vocal_pitch={"version": 1}) # no notes
with pytest.raises(ValueError):
attach_vocal_sidecars_to_sloppak(
pak, lyrics=_LYRICS_PAYLOAD, lyrics_source="whisperx") # not spec enum
# ── _collect_tone_events ────────────────────────────────────────────────────
def _tone_args(banks, tempo_map=((0, 120.0),)):
+1
View File
@@ -58,6 +58,7 @@ READERS = [
"lib/enrichment.py",
"lib/songmeta.py",
"lib/gp2notation.py", # rewrites manifest.yaml; stamps feedpak_version
"lib/gp2rs_gpx.py", # attach_vocal_sidecars_to_sloppak: lyrics/vocal_pitch keys
"lib/routers/ws_highway.py", # reads `authors` off a feedpak manifest
"lib/routers/chart.py", # Get-info panel: binds `m = load_manifest(...)`
"lib/routers/song.py", # enrichment gap-fill: reads the manifest directly
+117 -14
View File
@@ -74,6 +74,52 @@ def build_pack(src_dir: Path, out_zip: Path) -> dict:
return {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)}
# rig_builder ships "fat" .vst3 bundles carrying all three platforms inside
# Contents/. A pack for one platform keeps that platform's binary dir + the
# shared bundle files, and drops the other two.
VST_PLATFORM_DIRS = {"mac": "MacOS", "win": "x86_64-win", "linux": "x86_64-linux"}
def build_vst_pack(vst_root: Path, out_zip: Path, platform: str) -> dict:
"""Reproducibly zip the VST tree keeping only `platform`'s binaries.
Slices each fat .vst3: everything is kept except the two foreign platform
dirs (MacOS / x86_64-win / x86_64-linux) and the vst/src build trees. Arc
names are relative to vst_root so the download endpoint extracts straight
into <plugin>/vst/. Same reproducible-build guarantees as build_pack.
"""
if platform not in VST_PLATFORM_DIRS:
raise ValueError(f"unknown platform {platform!r} (want mac/win/linux)")
foreign = set(VST_PLATFORM_DIRS.values()) - {VST_PLATFORM_DIRS[platform]}
files = []
for p in sorted(vst_root.rglob("*"), key=lambda q: q.as_posix()):
if not p.is_file():
continue
rel = p.relative_to(vst_root)
if rel.parts and rel.parts[0] == "src": # skip C++/JUCE build trees
continue
if set(rel.parts) & foreign: # drop foreign-platform binaries
continue
files.append((p, rel))
if not files:
raise ValueError(f"no VST files to pack for {platform} in {vst_root}")
out_zip.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(out_zip, "w", zipfile.ZIP_STORED) as zf:
for p, rel in files:
info = zipfile.ZipInfo(rel.as_posix(), date_time=(1980, 1, 1, 0, 0, 0))
info.compress_type = zipfile.ZIP_STORED
# Pin create_system like build_pack: ZipInfo defaults it from the
# host OS (0 on Windows, 3 on Unix), which would otherwise make the
# same pack hash differently across runners. VST packs are the most
# likely to be built on Windows (native .vst3), so without this pin
# the precomputable-hash guarantee breaks exactly where it's needed.
info.create_system = 3
info.external_attr = 0o644 << 16
zf.writestr(info, p.read_bytes())
data = out_zip.read_bytes()
return {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)}
def manifest_entry(out_zip: Path, url: str) -> dict:
"""Pack info as the download-path expects it: {url, sha256, bytes}."""
return {"url": url,
@@ -96,25 +142,47 @@ def pack_url(pack_id: str, version: int, repo: str = REPO) -> str:
f"{pack_tag(pack_id, version)}/{pack_asset(pack_id, version)}")
def publish(pack_id: str, version: int, zip_path: Path, repo: str = REPO) -> None:
# VST packs use the same immutable per-pack convention, keyed by platform:
# tag `vst-<plat>-v<N>`, asset `vst-<plat>-pack-v<N>.zip`. The manifest they
# emit is keyed by platform (mac/win/linux) — the shape the rig_builder plugin's
# data/vst_packs.json consumes.
def vst_tag(platform: str, version: int) -> str:
return f"vst-{platform}-v{version}"
def vst_asset(platform: str, version: int) -> str:
return f"vst-{platform}-pack-v{version}.zip"
def vst_url(platform: str, version: int, repo: str = REPO) -> str:
return (f"https://github.com/{repo}/releases/download/"
f"{vst_tag(platform, version)}/{vst_asset(platform, version)}")
def _publish_release(tag: str, zip_path: Path, title: str, notes: str,
repo: str = REPO) -> None:
"""Create the per-pack release if missing, then upload the versioned zip.
Tags are immutable: a media change means a new version (v1 → v2), never a
re-upload — so no --clobber. gh errors if the asset already exists, which is
the right guard against overwriting a published, referenced pack.
"""
tag = pack_tag(pack_id, version)
if subprocess.run(["gh", "release", "view", tag, "--repo", repo],
capture_output=True).returncode != 0:
subprocess.run(
["gh", "release", "create", tag, "--repo", repo, "--latest=false",
"--title", f"{pack_id.capitalize()} venue pack v{version}",
"--notes", "Opt-in career venue pack. Not a code release."],
"--title", title, "--notes", notes],
check=True)
subprocess.run(
["gh", "release", "upload", tag, str(zip_path), "--repo", repo], check=True)
def publish(pack_id: str, version: int, zip_path: Path, repo: str = REPO) -> None:
_publish_release(pack_tag(pack_id, version), zip_path,
f"{pack_id.capitalize()} venue pack v{version}",
"Opt-in career venue pack. Not a code release.", repo)
def _pack_id(src_dir: Path) -> str:
return src_dir.name
@@ -129,6 +197,10 @@ def main(argv=None) -> int:
help="write zips here + a file:// manifest.json; no upload")
ap.add_argument("--publish", action="store_true",
help="create/upload the per-pack release; emit release URLs")
ap.add_argument("--vst", action="store_true",
help="slice one rig VST root (src[0]) into per-platform "
"vst-<plat>-v<N> packs; manifest keyed by platform "
"(the shape rig_builder's data/vst_packs.json wants)")
ap.add_argument("--manifest", type=Path,
help="write the {id: {url,sha256,bytes}} map here (default: stdout)")
ap.add_argument("--selfcheck", action="store_true", help="run the round-trip demo and exit")
@@ -141,16 +213,30 @@ def main(argv=None) -> int:
out_dir = args.local if args.local else Path(args.src[0]).parent / "_packs"
manifest = {}
for src in args.src:
pid = _pack_id(src)
zip_path = out_dir / pack_asset(pid, args.version)
build_pack(src, zip_path)
if args.publish:
publish(pid, args.version, zip_path)
url = pack_url(pid, args.version)
else:
url = (out_dir.resolve() / zip_path.name).as_uri()
manifest[pid] = manifest_entry(zip_path, url)
if args.vst:
vst_root = args.src[0]
for plat in VST_PLATFORM_DIRS:
zip_path = out_dir / vst_asset(plat, args.version)
build_vst_pack(vst_root, zip_path, plat)
if args.publish:
_publish_release(vst_tag(plat, args.version), zip_path,
f"Rig VST pack ({plat}) v{args.version}",
"Opt-in per-platform rig VST pack. Not a code release.")
url = vst_url(plat, args.version)
else:
url = (out_dir.resolve() / zip_path.name).as_uri()
manifest[plat] = manifest_entry(zip_path, url)
else:
for src in args.src:
pid = _pack_id(src)
zip_path = out_dir / pack_asset(pid, args.version)
build_pack(src, zip_path)
if args.publish:
publish(pid, args.version, zip_path)
url = pack_url(pid, args.version)
else:
url = (out_dir.resolve() / zip_path.name).as_uri()
manifest[pid] = manifest_entry(zip_path, url)
out = json.dumps(manifest, indent=2)
if args.manifest:
@@ -183,6 +269,23 @@ def _selfcheck() -> int:
with zipfile.ZipFile(zip_path) as zf:
names = zf.namelist()
assert set(names) == {"manifest.json", "bored.mp4"}, names
# VST slice: keep target platform + shared, drop foreign, reproducible.
c = td / "vst" / "Foo.vst3" / "Contents"
for d in ("MacOS", "x86_64-win", "x86_64-linux", "Resources"):
(c / d).mkdir(parents=True)
(c / "MacOS" / "Foo").write_bytes(b"mac")
(c / "x86_64-linux" / "Foo.so").write_bytes(b"linux")
(c / "Info.plist").write_bytes(b"<plist/>")
vzip = td / vst_asset("linux", 1)
vinfo = build_vst_pack(td / "vst", vzip, "linux")
assert vinfo == build_vst_pack(td / "vst", td / "v2.zip", "linux"), \
"vst slice is not reproducible"
with zipfile.ZipFile(vzip) as zf:
vnames = set(zf.namelist())
assert "Foo.vst3/Contents/x86_64-linux/Foo.so" in vnames
assert "Foo.vst3/Contents/Info.plist" in vnames
assert not any("MacOS" in n for n in vnames), vnames
print("content_packs selfcheck: ok")
return 0