mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-08-13 12:20:09 +00:00
feat(core): read .jsonc data files (strip C-style comments) (feedpak-spec §8)
feedpak-spec §8 (FEP #3 / PR #13) allows .jsonc files (JSON with C-style // line and /* */ block comments) anywhere .json is specified. A Reader MUST strip comments before parsing. Core's sloppak/feedpak readers parsed every side file with bare json.loads, so a .jsonc arrangement / notation / drum_tab / song_timeline / lyrics / keys would fail to load. - lib/jsonc.py (new): shared parse_jsonc(text) + load_json(path). String-aware regex (mirrors the spec reference validator in feedpak-spec/tools/validate.py) — keeps comment-like text inside JSON string literals. load_json auto-detects .jsonc by suffix; plain .json goes straight through json.loads. - lib/sloppak.py: import load_json; replace the 6 json.loads(...read_text...) side-file read sites (arrangement, notation, drum_tab, song_timeline, lyrics, keys) with load_json(<path>). Removed the now-unused `import json`. - scripts/lift_keys_notation.py: import load_json; replace the 3 read sites (song_timeline, arrangement beats fallback, arrangement lift). `import json` stays (json.dumps write at the notation sidecar emit). Additive (MINOR) change: older readers parse .jsonc as plain JSON and ignore comments via the spec's forward-compatibility rules, so no existing pack needs regeneration. Tests: tests/test_sloppak_jsonc_load.py (16 tests) — parse_jsonc unit cases (line/block/multiline/string-boundary/malformed/plain), and end-to-end loads for all 6 side-file types via .jsonc with comments, plus the lift helper reading .jsonc song_timeline + .jsonc arrangement beats, plus the string-boundary preservation rule through the full loader. 122 sloppak/lift tests pass.
This commit is contained in:
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
- **`.jsonc` support for feedpak data files** (feedpak-spec §8, FEP #3 / PR #13). Hand-edited packs may now use the `.jsonc` extension (JSON with C-style `//` line and `/* */` block comments) for any data file the manifest points at — arrangements, notation sidecars, `drum_tab`, `song_timeline`, `lyrics`, and `keys`. New shared `lib/jsonc.py` provides `parse_jsonc(text)` + `load_json(path)` (auto-detects `.jsonc` by suffix, string-aware so comment-like text inside JSON string values is preserved) and is now used by every reader in `lib/sloppak.py` (six side-file sites) and `scripts/lift_keys_notation.py` (three arrangement / song_timeline read sites). The strip regex mirrors the reference validator in `feedpak-spec/tools/validate.py`. This is an additive (MINOR) change: older readers parse `.jsonc` files as plain JSON and ignore comments via the spec's forward-compatibility rules, so no existing pack needs regeneration. Tests: `tests/test_sloppak_jsonc_load.py` (covers all six side-file types, the lift helper, and the string-boundary preservation rule end-to-end).
|
||||||
- **Guitar Pro → notation importer (`lib/gp2notation.py`)** (slopsmith#825 WS4b, epic #828). Piano/keys tracks imported from Guitar Pro (GPIF: `.gpx` GP6 / `.gp` GP7-8) now produce real Sloppak Notation Format data (sloppak-spec §5.3) alongside the `midi = string*24 + fret` guitar wire encoding. `gp2rs_gpx.convert_file` writes a `<stem>.notation.json` sidecar next to each keys arrangement XML (best-effort — a notation bug never breaks the RS-XML conversion), and `gp2notation.attach_notation_to_sloppak()` is the assembly-side helper that renames it into `notation_<id>.json` + adds the per-arrangement `notation:` manifest sub-key. Voice→staff routing salvages the logic from PR #703 (whose `stf` wire-field approach this supersedes): GP voice position 0 → `rh` staff (`G2`), positions ≥ 1 → `lh` (`F4`); a forced-LH track (the merged `Piano LH` partner from `_find_piano_pairs`, or a standalone track named `… LH`) routes everything to `lh` — preserving authored hand crossings instead of inferring hands from pitch. Emits measures with absolute `t` from the bar-indexed tempo map, change-only `ts`/`tempo`/`ks`, and `beat_groups` for compound/irregular meters (6/8 → `[3,3]`, 9/8 → `[3,3,3]`, 5/8 → `[2,3]`, 7/8 → `[2,2,3]` — cf. the slopsmith#261 denominator pitfalls); beats carry `dur`/`dot`/`tu`/`rest` from GP rhythms and notes carry absolute `midi` (String+Fret resolves via the string template's concert pitches, Tone+Octave via `(octave+1)*12 + step`) with `tied` continuations kept as real beats (engraving needs the tied notehead — unlike the RS-XML walk, which drops them and extends sustain). Timing reuses the `gp2rs_gpx` machinery (bar-indexed tempo map, per-beat rhythm durations, `_note_midi`) so notation lines up with the RS XML the highway plays — with one deliberate divergence: double dots advance time ×1.75 (vs. the RS-XML walk's single-dot ×1.5 approximation) so a written `dot: 2` agrees with the emitted beat times; sharing the walk itself stays tracked in slopsmith#618. Tests: `tests/test_gp2notation.py`.
|
- **Guitar Pro → notation importer (`lib/gp2notation.py`)** (slopsmith#825 WS4b, epic #828). Piano/keys tracks imported from Guitar Pro (GPIF: `.gpx` GP6 / `.gp` GP7-8) now produce real Sloppak Notation Format data (sloppak-spec §5.3) alongside the `midi = string*24 + fret` guitar wire encoding. `gp2rs_gpx.convert_file` writes a `<stem>.notation.json` sidecar next to each keys arrangement XML (best-effort — a notation bug never breaks the RS-XML conversion), and `gp2notation.attach_notation_to_sloppak()` is the assembly-side helper that renames it into `notation_<id>.json` + adds the per-arrangement `notation:` manifest sub-key. Voice→staff routing salvages the logic from PR #703 (whose `stf` wire-field approach this supersedes): GP voice position 0 → `rh` staff (`G2`), positions ≥ 1 → `lh` (`F4`); a forced-LH track (the merged `Piano LH` partner from `_find_piano_pairs`, or a standalone track named `… LH`) routes everything to `lh` — preserving authored hand crossings instead of inferring hands from pitch. Emits measures with absolute `t` from the bar-indexed tempo map, change-only `ts`/`tempo`/`ks`, and `beat_groups` for compound/irregular meters (6/8 → `[3,3]`, 9/8 → `[3,3,3]`, 5/8 → `[2,3]`, 7/8 → `[2,2,3]` — cf. the slopsmith#261 denominator pitfalls); beats carry `dur`/`dot`/`tu`/`rest` from GP rhythms and notes carry absolute `midi` (String+Fret resolves via the string template's concert pitches, Tone+Octave via `(octave+1)*12 + step`) with `tied` continuations kept as real beats (engraving needs the tied notehead — unlike the RS-XML walk, which drops them and extends sustain). Timing reuses the `gp2rs_gpx` machinery (bar-indexed tempo map, per-beat rhythm durations, `_note_midi`) so notation lines up with the RS XML the highway plays — with one deliberate divergence: double dots advance time ×1.75 (vs. the RS-XML walk's single-dot ×1.5 approximation) so a written `dot: 2` agrees with the emitted beat times; sharing the walk itself stays tracked in slopsmith#618. Tests: `tests/test_gp2notation.py`.
|
||||||
- **Legacy keys → notation lifter (`scripts/lift_keys_notation.py`)** (slopsmith#825 WS4c, epic #828). One-time batch converter that lifts existing **directory-form** piano/keys sloppaks from the legacy guitar wire encoding (`midi = s*24 + f`) into real Sloppak Notation Format files (sloppak-spec §5.3). Candidates are arrangements whose name matches `\b(keys|piano|keyboard|synth)\b` (case-insensitive); each gets a `notation_<id>.json` plus the per-arrangement `notation:` manifest sub-key. Measures derive from the song-level `beats` downbeats (`measure >= 0`; `song_timeline.json` preferred, first-arrangement fallback), with per-measure tempo from downbeat spacing (emitted only on a > 1 BPM change). Durations come from the wire sustain (`sus`, legacy `l` alias) when present, else the gap to the next onset in the same hand — quantized to the nearest plain/single-dotted `{1,2,4,8,16,32}` denominator at the local tempo, floored at a 32nd. Hands are split heuristically: onsets within 10 ms form a group; a group spanning > 12 semitones splits at its largest internal interval gap (low side → `lh`), otherwise the whole group goes by mean pitch vs middle C — single-staff output when everything lands on one hand. Idempotent (arrangements already carrying `notation:` are skipped; an orphan `notation_<id>.json` without the manifest key is refused, not overwritten) with `--dry-run` support; every payload is checked via `notation.validate_notation` before write. Honest caveat: the manifest is round-tripped through PyYAML (`safe_load` + `safe_dump(sort_keys=False)`) — key order survives, YAML comments/custom formatting do not (the script warns when comments are present). Zip-form `.sloppak` files are reported and skipped. Tests: `tests/test_lift_keys_notation.py`.
|
- **Legacy keys → notation lifter (`scripts/lift_keys_notation.py`)** (slopsmith#825 WS4c, epic #828). One-time batch converter that lifts existing **directory-form** piano/keys sloppaks from the legacy guitar wire encoding (`midi = s*24 + f`) into real Sloppak Notation Format files (sloppak-spec §5.3). Candidates are arrangements whose name matches `\b(keys|piano|keyboard|synth)\b` (case-insensitive); each gets a `notation_<id>.json` plus the per-arrangement `notation:` manifest sub-key. Measures derive from the song-level `beats` downbeats (`measure >= 0`; `song_timeline.json` preferred, first-arrangement fallback), with per-measure tempo from downbeat spacing (emitted only on a > 1 BPM change). Durations come from the wire sustain (`sus`, legacy `l` alias) when present, else the gap to the next onset in the same hand — quantized to the nearest plain/single-dotted `{1,2,4,8,16,32}` denominator at the local tempo, floored at a 32nd. Hands are split heuristically: onsets within 10 ms form a group; a group spanning > 12 semitones splits at its largest internal interval gap (low side → `lh`), otherwise the whole group goes by mean pitch vs middle C — single-staff output when everything lands on one hand. Idempotent (arrangements already carrying `notation:` are skipped; an orphan `notation_<id>.json` without the manifest key is refused, not overwritten) with `--dry-run` support; every payload is checked via `notation.validate_notation` before write. Honest caveat: the manifest is round-tripped through PyYAML (`safe_load` + `safe_dump(sort_keys=False)`) — key order survives, YAML comments/custom formatting do not (the script warns when comments are present). Zip-form `.sloppak` files are reported and skipped. Tests: `tests/test_lift_keys_notation.py`.
|
||||||
- **Notation schema v1 freeze — completeness batch** (slopsmith#822, epic #828). Adds the low-hanging-fruit fields ahead of content production: top-level credits `rights`/`lyricist`/`arranger`; measure `pickup` (anacrusis); beat `arp` (arpeggiate), `ferm` (fermata), and **typed grace notes** — `grace: "a"` (acciaccatura, MusicXML `grace/@slash=yes`) / `"p"` (appoggiatura); note `stem` (`"up"`/`"down"` force). Pedal is settled as the existing `spd`/`sph`/`spu` trio with a documented MusicXML `<pedal start|change|stop>` mapping — no separate `ped` field. A new "v1 non-features" spec subsection pins the accepted limitations (microtonal, figured bass, mid-measure key/time/clef changes, `ott`/`barline`/ornaments/`trem`/`glis`) as additive-v1.x territory. `lib/notation.py` gains the `GRACE_TYPES`, `STEM_DIRECTIONS`, and `DYNAMICS` vocabularies; the validator stays permissive by design.
|
- **Notation schema v1 freeze — completeness batch** (slopsmith#822, epic #828). Adds the low-hanging-fruit fields ahead of content production: top-level credits `rights`/`lyricist`/`arranger`; measure `pickup` (anacrusis); beat `arp` (arpeggiate), `ferm` (fermata), and **typed grace notes** — `grace: "a"` (acciaccatura, MusicXML `grace/@slash=yes`) / `"p"` (appoggiatura); note `stem` (`"up"`/`"down"` force). Pedal is settled as the existing `spd`/`sph`/`spu` trio with a documented MusicXML `<pedal start|change|stop>` mapping — no separate `ped` field. A new "v1 non-features" spec subsection pins the accepted limitations (microtonal, figured bass, mid-measure key/time/clef changes, `ott`/`barline`/ornaments/`trem`/`glis`) as additive-v1.x territory. `lib/notation.py` gains the `GRACE_TYPES`, `STEM_DIRECTIONS`, and `DYNAMICS` vocabularies; the validator stays permissive by design.
|
||||||
|
|||||||
@@ -0,0 +1,56 @@
|
|||||||
|
"""JSONC support — JSON with C-style comments.
|
||||||
|
|
||||||
|
Per feedpak-spec §8: when a manifest pointer resolves to a ``.jsonc`` file, a
|
||||||
|
Reader MUST strip ``//`` line comments and ``/* */`` block comments before
|
||||||
|
parsing the JSON content. This module implements that stripping in a single
|
||||||
|
shared place so every sloppak/feedpak reader in this repo parses ``.jsonc``
|
||||||
|
the same way (string-aware so comment-like text inside JSON strings survives).
|
||||||
|
|
||||||
|
The regex mirrors the reference implementation in ``feedpak-spec/tools/validate.py``.
|
||||||
|
``load_json(path)`` auto-detects ``.jsonc`` by suffix; plain ``.json`` (and any
|
||||||
|
other extension) goes straight through ``json.loads``. Use it as a drop-in
|
||||||
|
replacement for ``json.loads(path.read_text(encoding="utf-8"))``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Match JSON string literals (preserved), // line comments, and /* block */
|
||||||
|
# comments. A single combined alternation processed by `sub` with a callback
|
||||||
|
# that keeps strings and replaces comments with the empty string — so
|
||||||
|
# comment-like text inside a string literal is never stripped.
|
||||||
|
_JSONC_STRIP_RE = re.compile(
|
||||||
|
r'"(?:[^"\\]|\\.)*"|' # string literal — keep as-is
|
||||||
|
r'//.*|' # // line comment — strip
|
||||||
|
r'/\*[\s\S]*?\*/', # /* block comment */ — strip
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_jsonc(text: str) -> object:
|
||||||
|
"""Parse a JSONC string, stripping C-style comments before JSON parsing.
|
||||||
|
|
||||||
|
Handles ``//`` line comments and ``/* */`` block comments, respecting
|
||||||
|
string boundaries so that comment-like text inside strings is preserved.
|
||||||
|
Raises ``json.JSONDecodeError`` on malformed JSON (after stripping).
|
||||||
|
"""
|
||||||
|
stripped = _JSONC_STRIP_RE.sub(
|
||||||
|
lambda m: m.group(0) if m.group(0).startswith('"') else '',
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
return json.loads(stripped)
|
||||||
|
|
||||||
|
|
||||||
|
def load_json(path: Path) -> object:
|
||||||
|
"""Read and parse a JSON/JSONC file by path.
|
||||||
|
|
||||||
|
Files ending in ``.jsonc`` are stripped of comments via :func:`parse_jsonc`;
|
||||||
|
all other files are parsed as plain JSON. UTF-8 encoded, matching every
|
||||||
|
other reader in this repo.
|
||||||
|
"""
|
||||||
|
raw = path.read_text(encoding="utf-8")
|
||||||
|
if path.name.lower().endswith(".jsonc"):
|
||||||
|
return parse_jsonc(raw)
|
||||||
|
return json.loads(raw)
|
||||||
+7
-7
@@ -13,7 +13,6 @@ See the format spec in the project's sloppak plan for the full layout.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
import logging
|
||||||
import math
|
import math
|
||||||
import shutil
|
import shutil
|
||||||
@@ -31,6 +30,7 @@ FEEDPAK_VERSION = "1.2.0"
|
|||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
|
from jsonc import load_json
|
||||||
from safepath import safe_join
|
from safepath import safe_join
|
||||||
from song import (
|
from song import (
|
||||||
Song,
|
Song,
|
||||||
@@ -405,7 +405,7 @@ def load_song(
|
|||||||
if not arr_path.exists():
|
if not arr_path.exists():
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
data = json.loads(arr_path.read_text(encoding="utf-8"))
|
data = load_json(arr_path)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.debug("sloppak: failed to parse arrangement %r: %s", rel, e)
|
log.debug("sloppak: failed to parse arrangement %r: %s", rel, e)
|
||||||
continue
|
continue
|
||||||
@@ -471,7 +471,7 @@ def load_song(
|
|||||||
raw_nt = None
|
raw_nt = None
|
||||||
if nt_path is not None and nt_path.exists():
|
if nt_path is not None and nt_path.exists():
|
||||||
try:
|
try:
|
||||||
raw_nt = json.loads(nt_path.read_text(encoding="utf-8"))
|
raw_nt = load_json(nt_path)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.warning("sloppak: failed to parse notation %r: %s", notation_rel, e)
|
log.warning("sloppak: failed to parse notation %r: %s", notation_rel, e)
|
||||||
if raw_nt is not None:
|
if raw_nt is not None:
|
||||||
@@ -510,7 +510,7 @@ def load_song(
|
|||||||
dt_path = None
|
dt_path = None
|
||||||
if dt_path is not None and dt_path.exists():
|
if dt_path is not None and dt_path.exists():
|
||||||
try:
|
try:
|
||||||
raw = json.loads(dt_path.read_text(encoding="utf-8"))
|
raw = load_json(dt_path)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.warning("sloppak: failed to parse drum_tab %r: %s", drum_tab_rel, e)
|
log.warning("sloppak: failed to parse drum_tab %r: %s", drum_tab_rel, e)
|
||||||
raw = None
|
raw = None
|
||||||
@@ -570,7 +570,7 @@ def load_song(
|
|||||||
st_path = None
|
st_path = None
|
||||||
if st_path is not None and st_path.exists():
|
if st_path is not None and st_path.exists():
|
||||||
try:
|
try:
|
||||||
raw = json.loads(st_path.read_text(encoding="utf-8"))
|
raw = load_json(st_path)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.warning("sloppak: failed to parse song_timeline %r: %s", song_timeline_rel, e)
|
log.warning("sloppak: failed to parse song_timeline %r: %s", song_timeline_rel, e)
|
||||||
raw = None
|
raw = None
|
||||||
@@ -668,7 +668,7 @@ def load_song(
|
|||||||
lyr_path = None
|
lyr_path = None
|
||||||
if lyr_path is not None and lyr_path.exists():
|
if lyr_path is not None and lyr_path.exists():
|
||||||
try:
|
try:
|
||||||
raw = json.loads(lyr_path.read_text(encoding="utf-8"))
|
raw = load_json(lyr_path)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.debug("sloppak: failed to parse lyrics %r: %s", lyrics_rel, e)
|
log.debug("sloppak: failed to parse lyrics %r: %s", lyrics_rel, e)
|
||||||
raw = None
|
raw = None
|
||||||
@@ -755,7 +755,7 @@ def load_song(
|
|||||||
k_path = None
|
k_path = None
|
||||||
if k_path is not None and k_path.exists():
|
if k_path is not None and k_path.exists():
|
||||||
try:
|
try:
|
||||||
raw = json.loads(k_path.read_text(encoding="utf-8"))
|
raw = load_json(k_path)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.warning("sloppak: failed to parse keys %r: %s", keys_rel, e)
|
log.warning("sloppak: failed to parse keys %r: %s", keys_rel, e)
|
||||||
raw = None
|
raw = None
|
||||||
|
|||||||
@@ -60,6 +60,8 @@ import yaml # noqa: E402
|
|||||||
|
|
||||||
import notation as notation_mod # noqa: E402, F401 (re-exported for tests)
|
import notation as notation_mod # noqa: E402, F401 (re-exported for tests)
|
||||||
|
|
||||||
|
from jsonc import load_json # noqa: E402
|
||||||
|
|
||||||
# The wire→notation heuristic core lives in ``lib/notation_lift.py`` so it can
|
# The wire→notation heuristic core lives in ``lib/notation_lift.py`` so it can
|
||||||
# be reused in-process (e.g. by the Arrangement Editor's notation save path)
|
# be reused in-process (e.g. by the Arrangement Editor's notation save path)
|
||||||
# rather than being copy-pasted out of this one-time CLI. Re-exported here so
|
# rather than being copy-pasted out of this one-time CLI. Re-exported here so
|
||||||
@@ -114,7 +116,7 @@ def _load_song_beats(pak: Path, manifest: dict) -> list[dict]:
|
|||||||
st_path = _safe_child(pak, st_rel)
|
st_path = _safe_child(pak, st_rel)
|
||||||
if st_path is not None and st_path.is_file():
|
if st_path is not None and st_path.is_file():
|
||||||
try:
|
try:
|
||||||
data = json.loads(st_path.read_text(encoding="utf-8"))
|
data = load_json(st_path)
|
||||||
# An empty beats list is not an authoritative timeline — fall
|
# An empty beats list is not an authoritative timeline — fall
|
||||||
# through to the arrangement JSONs rather than ending up with
|
# through to the arrangement JSONs rather than ending up with
|
||||||
# zero downbeats and skipping the whole sloppak.
|
# zero downbeats and skipping the whole sloppak.
|
||||||
@@ -134,7 +136,7 @@ def _load_song_beats(pak: Path, manifest: dict) -> list[dict]:
|
|||||||
if arr_path is None or not arr_path.is_file():
|
if arr_path is None or not arr_path.is_file():
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
data = json.loads(arr_path.read_text(encoding="utf-8"))
|
data = load_json(arr_path)
|
||||||
except (OSError, ValueError):
|
except (OSError, ValueError):
|
||||||
continue
|
continue
|
||||||
if isinstance(data, dict):
|
if isinstance(data, dict):
|
||||||
@@ -219,7 +221,7 @@ def lift_sloppak(pak: Path, *, dry_run: bool = False) -> list[str]:
|
|||||||
continue
|
continue
|
||||||
|
|
||||||
try:
|
try:
|
||||||
arr_data = json.loads(arr_path.read_text(encoding="utf-8"))
|
arr_data = load_json(arr_path)
|
||||||
except (OSError, ValueError) as e:
|
except (OSError, ValueError) as e:
|
||||||
log.warning("%s/%s: unreadable arrangement JSON (%s) — skipped",
|
log.warning("%s/%s: unreadable arrangement JSON (%s) — skipped",
|
||||||
pak.name, arr_id, e)
|
pak.name, arr_id, e)
|
||||||
|
|||||||
@@ -0,0 +1,335 @@
|
|||||||
|
"""Tests for .jsonc support in the sloppak/feedpak loaders (feedpak-spec §8).
|
||||||
|
|
||||||
|
When a manifest pointer resolves to a ``.jsonc`` file, the reader MUST strip
|
||||||
|
C-style comments (``//`` line and ``/* */`` block) before parsing. These tests
|
||||||
|
exercise every side-file reader in ``lib/sloppak.py`` (arrangement, notation,
|
||||||
|
drum_tab, song_timeline, lyrics, keys) plus the arrangement/song_timeline
|
||||||
|
reads in ``scripts/lift_keys_notation.py`` against ``.jsonc`` inputs, and pin
|
||||||
|
the string-boundary rule (comment-like text inside a JSON string survives).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
import sloppak as sloppak_mod
|
||||||
|
from jsonc import parse_jsonc
|
||||||
|
from scripts.lift_keys_notation import _load_song_beats
|
||||||
|
|
||||||
|
|
||||||
|
# ── jsonc.parse_jsonc unit tests ─────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_parse_jsonc_strips_line_comment():
|
||||||
|
assert parse_jsonc('{"a": 1 // c\n}') == {"a": 1}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_jsonc_strips_block_comment():
|
||||||
|
assert parse_jsonc('{"a": /* x */ 2}') == {"a": 2}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_jsonc_strips_multiline_block_comment():
|
||||||
|
text = '{\n /* multi\n line */\n "a": 1\n}'
|
||||||
|
assert parse_jsonc(text) == {"a": 1}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_jsonc_preserves_comment_like_text_in_strings():
|
||||||
|
# ``//`` and ``/*`` inside a JSON string literal must NOT be treated as
|
||||||
|
# comments — the string-aware regex keeps them verbatim.
|
||||||
|
text = '{"url": "https://x/y", "note": "// not a comment /* still not */"}'
|
||||||
|
out = parse_jsonc(text)
|
||||||
|
assert out["url"] == "https://x/y"
|
||||||
|
assert out["note"] == "// not a comment /* still not */"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_jsonc_rejects_malformed():
|
||||||
|
with pytest.raises(json.JSONDecodeError):
|
||||||
|
parse_jsonc('{"a": // comment breaks value\n}')
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_jsonc_plain_json_passes_through():
|
||||||
|
assert parse_jsonc('{"a": 1}') == {"a": 1}
|
||||||
|
|
||||||
|
|
||||||
|
# ── Fixture builder ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _base_arrangement(*, beats=None) -> dict:
|
||||||
|
return {
|
||||||
|
"name": "Lead", "tuning": [0, 0, 0, 0, 0, 0], "capo": 0,
|
||||||
|
"notes": [], "chords": [], "anchors": [], "handshapes": [],
|
||||||
|
"templates": [], "beats": beats or [], "sections": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _write_sloppak(root: Path, *, manifest_extras: dict, side_files: dict) -> Path:
|
||||||
|
"""Build a minimal directory-form sloppak.
|
||||||
|
|
||||||
|
``side_files`` maps a pak-relative filename (e.g. ``"arrangements/lead.jsonc"``)
|
||||||
|
to the exact text to write — so the caller controls comments / extensions.
|
||||||
|
"""
|
||||||
|
pak = root / f"{root.name}.sloppak"
|
||||||
|
pak.mkdir()
|
||||||
|
(pak / "arrangements").mkdir()
|
||||||
|
# A default lead arrangement file the manifest references; can be overridden
|
||||||
|
# via side_files if the caller wants a .jsonc arrangement.
|
||||||
|
if "arrangements/lead.json" not in side_files and "arrangements/lead.jsonc" not in side_files:
|
||||||
|
(pak / "arrangements" / "lead.json").write_text(
|
||||||
|
json.dumps(_base_arrangement())
|
||||||
|
)
|
||||||
|
for rel, text in side_files.items():
|
||||||
|
target = pak / rel
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_text(text, encoding="utf-8")
|
||||||
|
manifest = {
|
||||||
|
"title": "Test", "artist": "Tester", "album": "", "year": 2026,
|
||||||
|
"duration": 10.0,
|
||||||
|
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
|
||||||
|
"stems": [{"id": "full", "file": "stems/full.ogg", "default": True}],
|
||||||
|
}
|
||||||
|
manifest.update(manifest_extras)
|
||||||
|
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
|
||||||
|
return pak
|
||||||
|
|
||||||
|
|
||||||
|
def _load(pak_path: Path, tmp_path: Path):
|
||||||
|
dlc_root = pak_path.parent
|
||||||
|
cache = tmp_path / "cache"
|
||||||
|
cache.mkdir()
|
||||||
|
return sloppak_mod.load_song(pak_path.name, dlc_root, cache)
|
||||||
|
|
||||||
|
|
||||||
|
# ── Arrangement .jsonc ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_load_arrangement_jsonc_with_comments(tmp_path: Path):
|
||||||
|
arr_text = (
|
||||||
|
'// lead chart\n'
|
||||||
|
'{\n'
|
||||||
|
' "name": "Lead",\n'
|
||||||
|
' "tuning": [0, 0, 0, 0, 0, 0],\n'
|
||||||
|
' "capo": 0,\n'
|
||||||
|
' /* no notes yet */\n'
|
||||||
|
' "notes": [{"t": 0.5, "s": 0, "f": 5, "sus": 0}],\n'
|
||||||
|
' "chords": [], "anchors": [], "handshapes": [], "templates": [],\n'
|
||||||
|
' "beats": [], "sections": []\n'
|
||||||
|
'}'
|
||||||
|
)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={
|
||||||
|
"arrangements": [{"id": "lead", "name": "Lead",
|
||||||
|
"file": "arrangements/lead.jsonc"}]
|
||||||
|
},
|
||||||
|
side_files={"arrangements/lead.jsonc": arr_text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
assert len(loaded.song.arrangements) == 1
|
||||||
|
arr = loaded.song.arrangements[0]
|
||||||
|
assert arr.name == "Lead"
|
||||||
|
assert len(arr.notes) == 1
|
||||||
|
assert arr.notes[0].fret == 5
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_arrangement_jsonc_malformed_is_skipped(tmp_path: Path):
|
||||||
|
# A .jsonc file that is still invalid after stripping comments must be
|
||||||
|
# skipped gracefully (the loader's existing permissive path), not crash.
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={
|
||||||
|
"arrangements": [{"id": "lead", "name": "Lead",
|
||||||
|
"file": "arrangements/lead.jsonc"}]
|
||||||
|
},
|
||||||
|
side_files={"arrangements/lead.jsonc": '{"notes": // broken\n}'},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
# The malformed arrangement is skipped; load still returns a song shell.
|
||||||
|
assert len(loaded.song.arrangements) == 0
|
||||||
|
|
||||||
|
|
||||||
|
# ── Notation .jsonc ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_load_notation_jsonc_with_comments(tmp_path: Path):
|
||||||
|
from tests.test_sloppak_notation_load import VALID_NOTATION
|
||||||
|
payload = VALID_NOTATION
|
||||||
|
text = (
|
||||||
|
'/* notation for keys */\n'
|
||||||
|
+ json.dumps(payload)
|
||||||
|
)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={
|
||||||
|
"arrangements": [{"id": "lead", "name": "Lead",
|
||||||
|
"file": "arrangements/lead.json",
|
||||||
|
"notation": "notation_lead.jsonc"}]
|
||||||
|
},
|
||||||
|
side_files={"notation_lead.jsonc": text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
assert loaded.notation_by_id is not None
|
||||||
|
assert "lead" in loaded.notation_by_id
|
||||||
|
|
||||||
|
|
||||||
|
# ── drum_tab .jsonc ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_load_drum_tab_jsonc_with_comments(tmp_path: Path):
|
||||||
|
payload = {
|
||||||
|
"version": 1, "name": "Drums",
|
||||||
|
"kit": [{"id": "kick", "name": "Kick"}],
|
||||||
|
"hits": [{"t": 0.5, "p": "kick", "v": 110}],
|
||||||
|
}
|
||||||
|
text = '// drum tab\n' + json.dumps(payload)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={"drum_tab": "drum_tab.jsonc"},
|
||||||
|
side_files={"drum_tab.jsonc": text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
assert loaded.drum_tab is not None
|
||||||
|
assert loaded.drum_tab["hits"][0]["p"] == "kick"
|
||||||
|
|
||||||
|
|
||||||
|
# ── song_timeline .jsonc ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_load_song_timeline_jsonc_with_comments(tmp_path: Path):
|
||||||
|
payload = {
|
||||||
|
"beats": [{"time": 0.0, "measure": 0}, {"time": 0.5, "measure": 0}],
|
||||||
|
"sections": [{"name": "intro", "number": 0, "time": 0.0}],
|
||||||
|
}
|
||||||
|
text = '/* timeline */\n' + json.dumps(payload)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={"song_timeline": "song_timeline.jsonc"},
|
||||||
|
side_files={"song_timeline.jsonc": text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
assert loaded.song_timeline is not None
|
||||||
|
assert len(loaded.song.beats) == 2
|
||||||
|
assert loaded.song.sections[0].name == "intro"
|
||||||
|
|
||||||
|
|
||||||
|
# ── lyrics .jsonc ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_load_lyrics_jsonc_with_comments(tmp_path: Path):
|
||||||
|
payload = [
|
||||||
|
{"w": "Hel", "t": 0.0, "d": 0.2},
|
||||||
|
{"w": "lo", "t": 0.2, "d": 0.3},
|
||||||
|
]
|
||||||
|
text = '// syllable lyrics\n' + json.dumps(payload)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={"lyrics": "lyrics.jsonc"},
|
||||||
|
side_files={"lyrics.jsonc": text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
assert len(loaded.song.lyrics) == 2
|
||||||
|
assert loaded.song.lyrics[0]["w"] == "Hel"
|
||||||
|
|
||||||
|
|
||||||
|
# ── keys .jsonc ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_load_keys_jsonc_with_comments(tmp_path: Path):
|
||||||
|
payload = {
|
||||||
|
"version": 1,
|
||||||
|
"events": [{"t": 0.0, "key": "Em", "scale": "natural_minor"}],
|
||||||
|
}
|
||||||
|
text = '/* key/scale track */\n' + json.dumps(payload)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={"keys": "keys.jsonc"},
|
||||||
|
side_files={"keys.jsonc": text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
assert loaded.keys is not None
|
||||||
|
assert loaded.keys["events"][0]["key"] == "Em"
|
||||||
|
|
||||||
|
|
||||||
|
# ── Comment-like text inside strings is preserved end-to-end ─────────────────
|
||||||
|
|
||||||
|
def test_load_arrangement_jsonc_preserves_comment_like_string_values(tmp_path: Path):
|
||||||
|
# A note whose fret label (if it had one) or a string field contains ``//``
|
||||||
|
# must survive the strip. We put a ``//`` inside the arrangement name and a
|
||||||
|
# ``/* */`` inside a beat section name to exercise both comment shapes.
|
||||||
|
arr_text = (
|
||||||
|
'{\n'
|
||||||
|
' "name": "Lead // solo",\n'
|
||||||
|
' "tuning": [0, 0, 0, 0, 0, 0], "capo": 0,\n'
|
||||||
|
' "notes": [], "chords": [], "anchors": [], "handshapes": [], "templates": [],\n'
|
||||||
|
' "beats": [],\n'
|
||||||
|
' "sections": [{"name": "verse /* important */", "number": 0, "time": 0.0}]\n'
|
||||||
|
'}'
|
||||||
|
)
|
||||||
|
pak = _write_sloppak(
|
||||||
|
tmp_path,
|
||||||
|
manifest_extras={
|
||||||
|
"arrangements": [{"id": "lead", "name": "Lead",
|
||||||
|
"file": "arrangements/lead.jsonc"}]
|
||||||
|
},
|
||||||
|
side_files={"arrangements/lead.jsonc": arr_text},
|
||||||
|
)
|
||||||
|
loaded = _load(pak, tmp_path)
|
||||||
|
arr = loaded.song.arrangements[0]
|
||||||
|
# The manifest's ``name: Lead`` overrides the arrangement JSON's name, so
|
||||||
|
# assert on the section name instead — it flows straight from the parsed
|
||||||
|
# .jsonc with no manifest override, proving the comment-like text inside
|
||||||
|
# the string survived the strip.
|
||||||
|
assert arr.name == "Lead"
|
||||||
|
assert loaded.song.sections[0].name == "verse /* important */"
|
||||||
|
|
||||||
|
|
||||||
|
# ── lift_keys_notation reads .jsonc arrangements / song_timeline ─────────────
|
||||||
|
|
||||||
|
def test_lift_keys_notation_loads_song_beats_from_jsonc_timeline(tmp_path: Path):
|
||||||
|
"""``_load_song_beats`` (the helper the lifter uses to find downbeats) must
|
||||||
|
read a ``.jsonc`` song_timeline when the manifest points at one."""
|
||||||
|
timeline_text = (
|
||||||
|
'// timeline\n'
|
||||||
|
+ json.dumps({
|
||||||
|
"beats": [{"time": 0.0, "measure": 0}, {"time": 0.5, "measure": 0},
|
||||||
|
{"time": 1.0, "measure": 1}],
|
||||||
|
"sections": [],
|
||||||
|
})
|
||||||
|
)
|
||||||
|
pak = tmp_path / "song.sloppak"
|
||||||
|
pak.mkdir()
|
||||||
|
(pak / "arrangements").mkdir()
|
||||||
|
(pak / "arrangements" / "keys.json").write_text(json.dumps(_base_arrangement()))
|
||||||
|
(pak / "song_timeline.jsonc").write_text(timeline_text, encoding="utf-8")
|
||||||
|
manifest = {
|
||||||
|
"title": "T", "artist": "A", "duration": 4.0,
|
||||||
|
"arrangements": [{"id": "keys", "name": "Keys",
|
||||||
|
"file": "arrangements/keys.json"}],
|
||||||
|
"song_timeline": "song_timeline.jsonc",
|
||||||
|
"stems": [{"id": "full", "file": "stems/full.ogg"}],
|
||||||
|
}
|
||||||
|
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
|
||||||
|
beats = _load_song_beats(pak, manifest)
|
||||||
|
assert len(beats) == 3
|
||||||
|
assert beats[2]["measure"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_lift_keys_notation_falls_back_to_jsonc_arrangement_beats(tmp_path: Path):
|
||||||
|
"""When no song_timeline is present, ``_load_song_beats`` falls back to the
|
||||||
|
first arrangement JSON carrying beats — and that arrangement may be ``.jsonc``."""
|
||||||
|
arr_text = (
|
||||||
|
'/* keys */\n'
|
||||||
|
+ json.dumps({
|
||||||
|
**_base_arrangement(),
|
||||||
|
"beats": [{"time": 0.0, "measure": 0}, {"time": 1.0, "measure": 1}],
|
||||||
|
})
|
||||||
|
)
|
||||||
|
pak = tmp_path / "song.sloppak"
|
||||||
|
pak.mkdir()
|
||||||
|
(pak / "arrangements").mkdir()
|
||||||
|
(pak / "arrangements" / "keys.jsonc").write_text(arr_text, encoding="utf-8")
|
||||||
|
manifest = {
|
||||||
|
"title": "T", "artist": "A", "duration": 4.0,
|
||||||
|
"arrangements": [{"id": "keys", "name": "Keys",
|
||||||
|
"file": "arrangements/keys.jsonc"}],
|
||||||
|
"stems": [{"id": "full", "file": "stems/full.ogg"}],
|
||||||
|
}
|
||||||
|
(pak / "manifest.yaml").write_text(yaml.safe_dump(manifest, sort_keys=False))
|
||||||
|
beats = _load_song_beats(pak, manifest)
|
||||||
|
assert len(beats) == 2
|
||||||
|
assert beats[1]["measure"] == 1
|
||||||
Reference in New Issue
Block a user