got-feedBack/feedBack
1
0
Fork 0
mirror of https://github.com/got-feedBack/feedBack.git synced 2026-08-11 19:29:33 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

ci: close two blind spots in the key scan

Browse Source 
Review found the gate was scanning less than it claimed.

READERS missed two modules that genuinely touch feedpak manifests:
lib/routers/ws_highway.py (reads `authors`) and lib/gp2notation.py (loads
manifest.yaml, stamps feedpak_version, writes the file back). Keys touched
there were going entirely unchecked.

The scan also only recognised writes done via subscript, so
`manifest.setdefault("k", v)` — exactly how gp2notation.py stamps
feedpak_version — was invisible. setdefault with a literal key now counts as
a write.

The deeper problem is that READERS is hand-maintained, and a hand-maintained
list rots; that is how both modules went unnoticed. check_readers_complete()
now re-derives the set: any module under lib/ (or server.py) that both
touches manifest keys and shows a feedpak signal must be listed, or the build
fails. It is a guard on the gate itself.

The list stays explicit rather than becoming a glob, because `manifest` is
overloaded here: lib/loosefolder.py (the loose-folder manifest.json) and
lib/diagnostics_bundle.py (the diagnostics bundle manifest) have their own
unrelated manifests, and scanning those would flag *their* keys as feedpak
drift. Both score zero on the feedpak signals, which is what keeps them out.

Now scanning 5 modules: 20 reads, 2 writes, all spec-declared.

Signed-off-by: topkoa <topkoa@gmail.com>
This commit is contained in:
topkoa
2026-07-13 00:23:36 -04:00
parent 32d723b774
commit d806d12c22
1 changed files with 436 additions and 377 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/check_spec_conformance.py
+436 -377
View File
@@ -1,377 +1,436 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""feedpak spec-conformance gate. """feedpak spec-conformance gate.
feedpak is an open, versioned format with its own normative spec, JSON Schemas, feedpak is an open, versioned format with its own normative spec, JSON Schemas,
and reference validator (https://github.com/got-feedback/feedpak-spec). That and reference validator (https://github.com/got-feedback/feedpak-spec). That
makes the spec a contract with everyone outside this repo: third-party packers, makes the spec a contract with everyone outside this repo: third-party packers,
converters, and players build against it. When core reads a manifest key the converters, and players build against it. When core reads a manifest key the
spec never defined, the contract quietly breaks — a spec-compliant pack stops spec never defined, the contract quietly breaks — a spec-compliant pack stops
being a fully-working pack, and the format's real definition migrates into our being a fully-working pack, and the format's real definition migrates into our
source tree. See #933 for the instance that motivated this gate. source tree. See #933 for the instance that motivated this gate.
We cannot mechanically prove core *interprets* a key the way the spec means. We We cannot mechanically prove core *interprets* a key the way the spec means. We
can prove three surface properties, and those cover the drift that actually can prove four surface properties, and those cover the drift that actually
happens: happens:
1. key-coverage — every manifest key core reads is declared by the spec. 1. key-coverage — every manifest key core reads OR WRITES is declared by the
2. forward — core ingests the spec's own example packs. spec. (Guarded by check_readers_complete(), so the list of
3. reverse — packs committed here satisfy the spec's reference validator. scanned modules cannot quietly fall behind the codebase.)
2. allowlist-closed— feedpak-spec-exceptions.yml never grows. It grandfathers
Dev/CI tooling only: never imported on the serve or Docker path (constitution keys that predate this gate; it is not a way to merge a
Principle I — same category as scripts/build-tailwind.sh). `jsonschema` is new one. The only route for a new key is the FEP process.
therefore a CI-only dependency, not a runtime requirement. 3. forward — core ingests the spec's own example packs.
4. reverse — packs committed here satisfy the spec's reference validator.
Usage:
python tools/check_spec_conformance.py --spec <path-to-feedpak-spec-checkout> Dev/CI tooling only: never imported on the serve or Docker path (constitution
Principle I — same category as scripts/build-tailwind.sh). `jsonschema` is
Exit status is 0 only when every layer passes. therefore a CI-only dependency, not a runtime requirement.
"""
from __future__ import annotations Usage:
python tools/check_spec_conformance.py --spec <path-to-feedpak-spec-checkout>
import argparse
import ast Exit status is 0 only when every layer passes.
import json """
import subprocess from __future__ import annotations
import sys
import tempfile import argparse
from pathlib import Path import ast
import json
REPO = Path(__file__).resolve().parent.parent import re
import subprocess
# Modules that read a feedpak manifest dict. Listed explicitly rather than import sys
# globbed so that adding a new reader is a deliberate act that shows up in import tempfile
# review — a new reader is exactly when key drift gets introduced. A missing from pathlib import Path
# file here is a hard error, so a rename cannot silently disable the scan.
READERS = [ REPO = Path(__file__).resolve().parent.parent
"lib/sloppak.py",
"lib/enrichment.py", # Modules that read or write a feedpak manifest dict. Explicit rather than
"lib/songmeta.py", # globbed, because `manifest` is an overloaded name in this codebase: the
] # loose-folder format (lib/loosefolder.py) and the diagnostics bundle
# (lib/diagnostics_bundle.py) both have their own unrelated `manifest`, and
# Locals that hold a manifest dict. The loaders use a uniform idiom # scanning those would flag *their* keys as feedpak drift.
# (`manifest.get("key")`), so binding by name is sufficient today. See #
# "Limitations" in docs/feedpak-spec-gate.md for the hardening path. # A hand-maintained list is itself a blind spot, so check_readers_complete()
MANIFEST_VARS = {"manifest", "mf"} # below re-derives the set and fails if this list has fallen behind. A missing
# file here is a hard error too, so a rename cannot silently disable the scan.
# Packs committed to this repo, checked against the spec's reference validator. READERS = [
PACK_GLOBS = ["content/starter/*.feedpak", "docs/**/*.sloppak", "docs/**/*.feedpak"] "lib/sloppak.py",
"lib/enrichment.py",
EXCEPTIONS_FILE = REPO / "feedpak-spec-exceptions.yml" "lib/songmeta.py",
"lib/gp2notation.py", # rewrites manifest.yaml; stamps feedpak_version
# How a new manifest key gets into core. There is no in-repo shortcut, by design: "lib/routers/ws_highway.py", # reads `authors` off a feedpak manifest
# the spec's own governance says "a change is not part of the format until it ]
# lands here", and the FEP process is how it lands.
FEP = ( # Where check_readers_complete() looks for modules READERS may have missed.
"A new manifest key must go through the feedpak Enhancement Proposal process " READER_SEARCH = ["lib/**/*.py", "server.py"]
"(https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md): land a PR on "
"feedpak-spec that updates the normative spec, the JSON Schemas, an example, and the " # A module is handling a *feedpak* manifest (rather than some other manifest) if
"changelog together — then bump .feedpak-spec-ref to the merged SHA in this PR." # it shows one of these signals. lib/loosefolder.py and lib/diagnostics_bundle.py
) # score zero on all of them, which is what keeps their keys out of the scan.
FEEDPAK_SIGNALS = re.compile(r"import sloppak|from sloppak|load_manifest|manifest\.yaml|feedpak")
def _fail(msg: str) -> None: # Does this module touch manifest keys at all?
print(f"::error::{msg}") KEY_OPS = re.compile(r"(manifest|mf)\.(get|setdefault)\(|(manifest|mf)\[")
# Locals that hold a manifest dict. The loaders use a uniform idiom
def _is_manifest_receiver(node: ast.expr) -> bool: # (`manifest.get("key")`), so binding by name is sufficient today. See
"""True when `node` evaluates to a manifest dict. # "Limitations" in docs/feedpak-spec-gate.md for the hardening path.
MANIFEST_VARS = {"manifest", "mf"}
Covers the plain `manifest.get(...)` idiom plus the wrapped form used in
lib/enrichment.py: `(sloppak_mod.load_manifest(p) or {}).get("key")`. # Packs committed to this repo, checked against the spec's reference validator.
""" PACK_GLOBS = ["content/starter/*.feedpak", "docs/**/*.sloppak", "docs/**/*.feedpak"]
if isinstance(node, ast.Name) and node.id in MANIFEST_VARS:
return True EXCEPTIONS_FILE = REPO / "feedpak-spec-exceptions.yml"
try:
src = ast.unparse(node) # How a new manifest key gets into core. There is no in-repo shortcut, by design:
except Exception: # the spec's own governance says "a change is not part of the format until it
return False # lands here", and the FEP process is how it lands.
return "load_manifest" in src FEP = (
"A new manifest key must go through the feedpak Enhancement Proposal process "
"(https://github.com/got-feedback/feedpak-spec/blob/main/CONTRIBUTING.md): land a PR on "
def keys_touched(path: Path) -> tuple[set[str], set[str]]: "feedpak-spec that updates the normative spec, the JSON Schemas, an example, and the "
"""Literal top-level manifest keys `path` reads and writes, separately. "changelog together — then bump .feedpak-spec-ref to the merged SHA in this PR."
)
Writes matter as much as reads: `manifest["k"] = v` means core *emits* `k`
into a pack it ships, so an undeclared key there puts non-spec surface into
the wild — the same drift, pointed outward. `manifest["k"]` in a subscript def _fail(msg: str) -> None:
is a read only when its context is a Load; an `ast.walk` that ignores `ctx` print(f"::error::{msg}")
would score `manifest["year"] = ...` (lib/songmeta.py) as a read.
"""
reads: set[str] = set() def _is_manifest_receiver(node: ast.expr) -> bool:
writes: set[str] = set() """True when `node` evaluates to a manifest dict.
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
for node in ast.walk(tree): Covers the plain `manifest.get(...)` idiom plus the wrapped form used in
if ( lib/enrichment.py: `(sloppak_mod.load_manifest(p) or {}).get("key")`.
isinstance(node, ast.Call) """
and isinstance(node.func, ast.Attribute) if isinstance(node, ast.Name) and node.id in MANIFEST_VARS:
and node.func.attr == "get" return True
and _is_manifest_receiver(node.func.value) try:
and node.args src = ast.unparse(node)
and isinstance(node.args[0], ast.Constant) except Exception:
and isinstance(node.args[0].value, str) return False
): return "load_manifest" in src
reads.add(node.args[0].value)
elif (
isinstance(node, ast.Subscript) def keys_touched(path: Path) -> tuple[set[str], set[str]]:
and _is_manifest_receiver(node.value) """Literal top-level manifest keys `path` reads and writes, separately.
and isinstance(node.slice, ast.Constant)
and isinstance(node.slice.value, str) Writes matter as much as reads: `manifest["k"] = v` means core *emits* `k`
): into a pack it ships, so an undeclared key there puts non-spec surface into
target = writes if isinstance(node.ctx, ast.Store) else reads the wild — the same drift, pointed outward. `manifest["k"]` in a subscript
target.add(node.slice.value) is a read only when its context is a Load; an `ast.walk` that ignores `ctx`
return reads, writes would score `manifest["year"] = ...` (lib/songmeta.py) as a read.
"""
reads: set[str] = set()
def _parse_exceptions(text: str, origin: str) -> dict[str, str]: writes: set[str] = set()
"""Parse an exceptions document into {key: tracking issue}.""" tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
import yaml # runtime dep (PyYAML is already in requirements.txt) for node in ast.walk(tree):
if (
data = yaml.safe_load(text) or {} isinstance(node, ast.Call)
out: dict[str, str] = {} and isinstance(node.func, ast.Attribute)
for entry in data.get("exceptions") or []: # `setdefault("k", v)` writes k when absent — lib/gp2notation.py
key, issue = entry.get("key"), entry.get("issue") # stamps feedpak_version that way, and a subscript-only scan misses
if not key or not issue: # it entirely, letting an emitted key slip past the gate.
_fail(f"{origin}: every exception needs both 'key' and 'issue'") and node.func.attr in ("get", "setdefault")
sys.exit(1) and _is_manifest_receiver(node.func.value)
# A duplicate would silently take the last issue link, quietly retargeting and node.args
# the debt this file exists to track. Fail instead. and isinstance(node.args[0], ast.Constant)
if key in out: and isinstance(node.args[0].value, str)
_fail( ):
f"{origin}: '{key}' is listed more than once. " bucket = writes if node.func.attr == "setdefault" else reads
f"Keep one entry per key so the tracking issue is unambiguous." bucket.add(node.args[0].value)
) elif (
sys.exit(1) isinstance(node, ast.Subscript)
out[key] = issue and _is_manifest_receiver(node.value)
return out and isinstance(node.slice, ast.Constant)
and isinstance(node.slice.value, str)
):
def load_exceptions() -> dict[str, str]: target = writes if isinstance(node.ctx, ast.Store) else reads
"""Map of grandfathered key -> tracking issue URL, as of this working tree.""" target.add(node.slice.value)
if not EXCEPTIONS_FILE.exists(): return reads, writes
return {}
return _parse_exceptions(
EXCEPTIONS_FILE.read_text(encoding="utf-8"), EXCEPTIONS_FILE.name def _parse_exceptions(text: str, origin: str) -> dict[str, str]:
) """Parse an exceptions document into {key: tracking issue}."""
import yaml # runtime dep (PyYAML is already in requirements.txt)
def check_allowlist_closed(baseline: Path | None, bootstrap: bool) -> bool: data = yaml.safe_load(text) or {}
"""The allowlist is CLOSED: it may shrink, never grow. out: dict[str, str] = {}
for entry in data.get("exceptions") or []:
`feedpak-spec-exceptions.yml` grandfathers keys that predate this gate. It is key, issue = entry.get("key"), entry.get("issue")
not a way to merge a new one. Without this check the gate would be a speed if not key or not issue:
bump with a signed excuse note — anyone could append an entry and route _fail(f"{origin}: every exception needs both 'key' and 'issue'")
around the FEP process from inside this repo, which is exactly the drift that sys.exit(1)
produced #933. # A duplicate would silently take the last issue link, quietly retargeting
# the debt this file exists to track. Fail instead.
So: removing an entry is fine (that's the debt being paid down); adding one if key in out:
fails the build, and the error points at the FEP process instead. _fail(
""" f"{origin}: '{key}' is listed more than once. "
if bootstrap: f"Keep one entry per key so the tracking issue is unambiguous."
print(" allowlist-closed: bootstrapping (no baseline on the base branch) — skipped") )
return True sys.exit(1)
if baseline is None: out[key] = issue
print(" allowlist-closed: no baseline supplied (local run) — skipped") return out
return True
base_keys = set( def load_exceptions() -> dict[str, str]:
_parse_exceptions(baseline.read_text(encoding="utf-8"), f"{EXCEPTIONS_FILE.name} (base)") """Map of grandfathered key -> tracking issue URL, as of this working tree."""
) if not EXCEPTIONS_FILE.exists():
now_keys = set(load_exceptions()) return {}
added = sorted(now_keys - base_keys) return _parse_exceptions(
removed = sorted(base_keys - now_keys) EXCEPTIONS_FILE.read_text(encoding="utf-8"), EXCEPTIONS_FILE.name
)
for key in added:
_fail(
f"{EXCEPTIONS_FILE.name}: this PR ADDS an exception for '{key}'. The allowlist is " def check_allowlist_closed(baseline: Path | None, bootstrap: bool) -> bool:
f"closed — it grandfathers keys that predate this gate and may only shrink. {FEP}" """The allowlist is CLOSED: it may shrink, never grow.
)
if removed: `feedpak-spec-exceptions.yml` grandfathers keys that predate this gate. It is
print(f" allowlist shrank (debt paid down): {', '.join(removed)}") not a way to merge a new one. Without this check the gate would be a speed
print(f" allowlist-closed: {'FAILED' if added else 'OK'}") bump with a signed excuse note — anyone could append an entry and route
return not added around the FEP process from inside this repo, which is exactly the drift that
produced #933.
def check_key_coverage(spec: Path) -> bool: So: removing an entry is fine (that's the debt being paid down); adding one
"""Layer 1 — core must not read or write a manifest key the spec does not declare.""" fails the build, and the error points at the FEP process instead.
schema = json.loads((spec / "schemas" / "manifest.schema.json").read_text(encoding="utf-8")) """
declared = set(schema.get("properties") or {}) if bootstrap:
if not declared: print(" allowlist-closed: bootstrapping (no baseline on the base branch) — skipped")
_fail("spec manifest.schema.json declares no properties — wrong path or bad checkout?") return True
return False if baseline is None:
print(" allowlist-closed: no baseline supplied (local run) — skipped")
reads: set[str] = set() return True
writes: set[str] = set()
for rel in READERS: base_keys = set(
path = REPO / rel _parse_exceptions(baseline.read_text(encoding="utf-8"), f"{EXCEPTIONS_FILE.name} (base)")
if not path.exists(): )
_fail(f"reader {rel} not found — was it renamed? Update READERS in {Path(__file__).name}.") now_keys = set(load_exceptions())
return False added = sorted(now_keys - base_keys)
r, w = keys_touched(path) removed = sorted(base_keys - now_keys)
reads |= r
writes |= w for key in added:
_fail(
exceptions = load_exceptions() f"{EXCEPTIONS_FILE.name}: this PR ADDS an exception for '{key}'. The allowlist is "
ok = True f"closed — it grandfathers keys that predate this gate and may only shrink. {FEP}"
)
def _undeclared(keys: set[str]) -> list[str]: if removed:
return sorted((keys - declared) - set(exceptions)) print(f" allowlist shrank (debt paid down): {', '.join(removed)}")
print(f" allowlist-closed: {'FAILED' if added else 'OK'}")
for key in _undeclared(reads): return not added
_fail(f"core reads manifest key '{key}', which the feedpak spec does not define. {FEP}")
ok = False
def check_readers_complete() -> bool:
for key in _undeclared(writes): """READERS must not fall behind the codebase.
_fail(
f"core writes manifest key '{key}', which the feedpak spec does not define — that " The key-coverage scan is only as good as the list of modules it scans, and a
f"puts non-spec surface into every pack we emit. {FEP}" hand-maintained list rots: `lib/routers/ws_highway.py` and
) `lib/gp2notation.py` both touched feedpak manifests for a while without being
ok = False on it. So re-derive the set — any module that both touches manifest keys and
shows a feedpak signal must be listed — and fail if one is missing.
# A stale exception is its own bug: it means the spec caught up and nobody
# cleaned up, so the allowlist slowly becomes a place drift hides. This is a guard on the gate itself, not on the format.
touched = reads | writes """
for key, issue in exceptions.items(): listed = set(READERS)
if key in declared: missing: list[str] = []
_fail( for pattern in READER_SEARCH:
f"'{key}' is listed in {EXCEPTIONS_FILE.name} but the spec now declares it. " for path in sorted(REPO.glob(pattern)):
f"Remove the exception and close {issue}." rel = path.relative_to(REPO).as_posix()
) if rel in listed:
ok = False continue
elif key not in touched: src = path.read_text(encoding="utf-8", errors="replace")
_fail( if KEY_OPS.search(src) and FEEDPAK_SIGNALS.search(src):
f"'{key}' is listed in {EXCEPTIONS_FILE.name} but core no longer reads or writes " missing.append(rel)
f"it. Remove the exception."
) for rel in missing:
ok = False _fail(
f"{rel} touches feedpak manifest keys but is not in READERS "
print(f" spec declares {len(declared)} keys; core reads {len(reads)}, writes {len(writes)}") f"({Path(__file__).name}) — its keys are going unchecked. Add it."
if exceptions: )
print(f" allowlisted (pending spec): {', '.join(sorted(exceptions))}") print(f" scanning {len(listed)} modules; readers-complete: {'FAILED' if missing else 'OK'}")
print(f" key-coverage: {'OK' if ok else 'FAILED'}") return not missing
return ok
def check_key_coverage(spec: Path) -> bool:
def check_forward(spec: Path) -> bool: """Layer 1 — core must not read or write a manifest key the spec does not declare."""
"""Layer 2 — core must ingest every example pack the spec ships.""" schema = json.loads((spec / "schemas" / "manifest.schema.json").read_text(encoding="utf-8"))
examples_dir = spec / "examples" declared = set(schema.get("properties") or {})
if not examples_dir.is_dir(): if not declared:
_fail(f"{examples_dir} is missing — wrong path or bad checkout?") _fail("spec manifest.schema.json declares no properties — wrong path or bad checkout?")
return False return False
# rglob, not iterdir: the contract is "every example pack the spec ships", so
# a pack nested under examples/<group>/ must not slip through. reads: set[str] = set()
# writes: set[str] = set()
# Deliberately NOT filtered by is_file(): a feedpak is dual-form — a zip for rel in READERS:
# (`foo.feedpak`) *or* a directory (`foo.feedpak/`) — and the spec's own path = REPO / rel
# examples ship as directories today. An is_file() guard here would silently if not path.exists():
# match zero packs. Matching on the suffix covers both forms, and rglob does _fail(f"reader {rel} not found — was it renamed? Update READERS in {Path(__file__).name}.")
# not smuggle in a pack's innards because files inside a pack don't carry a return False
# pack suffix. r, w = keys_touched(path)
examples = sorted( reads |= r
p for p in examples_dir.rglob("*") writes |= w
if p.suffix in (".feedpak", ".sloppak")
) exceptions = load_exceptions()
if not examples: ok = True
_fail("spec ships no example packs — wrong path or bad checkout?")
return False def _undeclared(keys: set[str]) -> list[str]:
return sorted((keys - declared) - set(exceptions))
sys.path.insert(0, str(REPO / "lib"))
try: for key in _undeclared(reads):
import sloppak # noqa: E402 (path must be set first — flat imports, no package) _fail(f"core reads manifest key '{key}', which the feedpak spec does not define. {FEP}")
except Exception as e: ok = False
_fail(
f"could not import core's sloppak loader ({type(e).__name__}: {e}). " for key in _undeclared(writes):
f"Are requirements.txt deps installed?" _fail(
) f"core writes manifest key '{key}', which the feedpak spec does not define — that "
return False f"puts non-spec surface into every pack we emit. {FEP}"
)
ok = True ok = False
with tempfile.TemporaryDirectory() as tmp:
cache = Path(tmp) # A stale exception is its own bug: it means the spec caught up and nobody
for pack in examples: # cleaned up, so the allowlist slowly becomes a place drift hides.
try: touched = reads | writes
loaded = sloppak.load_song(pack.name, pack.parent, cache) for key, issue in exceptions.items():
except Exception as e: if key in declared:
_fail( _fail(
f"core failed to load the spec's own example pack {pack.name}: " f"'{key}' is listed in {EXCEPTIONS_FILE.name} but the spec now declares it. "
f"{type(e).__name__}: {e}. A spec-valid pack must load." f"Remove the exception and close {issue}."
) )
ok = False ok = False
continue elif key not in touched:
if not loaded.song.arrangements: _fail(
_fail(f"core loaded {pack.name} but found no arrangements") f"'{key}' is listed in {EXCEPTIONS_FILE.name} but core no longer reads or writes "
ok = False f"it. Remove the exception."
continue )
print(f" loaded {pack.name}: {len(loaded.song.arrangements)} arrangement(s)") ok = False
print(f" forward: {'OK' if ok else 'FAILED'}")
return ok print(f" spec declares {len(declared)} keys; core reads {len(reads)}, writes {len(writes)}")
if exceptions:
print(f" allowlisted (pending spec): {', '.join(sorted(exceptions))}")
def check_reverse(spec: Path) -> bool: print(f" key-coverage: {'OK' if ok else 'FAILED'}")
"""Layer 3 — packs committed here must pass the spec's reference validator.""" return ok
packs = sorted({p for g in PACK_GLOBS for p in REPO.glob(g)})
if not packs:
print(" reverse: no committed packs — skipped") def check_forward(spec: Path) -> bool:
return True """Layer 2 — core must ingest every example pack the spec ships."""
examples_dir = spec / "examples"
proc = subprocess.run( if not examples_dir.is_dir():
[sys.executable, str(spec / "tools" / "validate.py"), *[str(p) for p in packs]], _fail(f"{examples_dir} is missing — wrong path or bad checkout?")
capture_output=True, return False
text=True, # rglob, not iterdir: the contract is "every example pack the spec ships", so
) # a pack nested under examples/<group>/ must not slip through.
sys.stdout.write("".join(f" {ln}\n" for ln in proc.stdout.splitlines() if ln.strip())) #
if proc.returncode != 0: # Deliberately NOT filtered by is_file(): a feedpak is dual-form — a zip
_fail( # (`foo.feedpak`) *or* a directory (`foo.feedpak/`) — and the spec's own
"a pack committed to this repo does not satisfy the feedpak spec " # examples ship as directories today. An is_file() guard here would silently
"(see the reference validator output above)." # match zero packs. Matching on the suffix covers both forms, and rglob does
) # not smuggle in a pack's innards because files inside a pack don't carry a
if proc.stderr.strip(): # pack suffix.
sys.stderr.write(proc.stderr) examples = sorted(
print(f" reverse: {'OK' if proc.returncode == 0 else 'FAILED'}") p for p in examples_dir.rglob("*")
return proc.returncode == 0 if p.suffix in (".feedpak", ".sloppak")
)
if not examples:
def main() -> int: _fail("spec ships no example packs — wrong path or bad checkout?")
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0]) return False
ap.add_argument(
"--spec", sys.path.insert(0, str(REPO / "lib"))
required=True, try:
type=Path, import sloppak # noqa: E402 (path must be set first — flat imports, no package)
help="path to a feedpak-spec checkout (CI pins the SHA in .feedpak-spec-ref)", except Exception as e:
) _fail(
ap.add_argument( f"could not import core's sloppak loader ({type(e).__name__}: {e}). "
"--baseline-exceptions", f"Are requirements.txt deps installed?"
type=Path, )
help="the exceptions file as it exists on the base branch. Supplied by CI so the " return False
"allowlist can be proven to have not grown. Omit for a local run.",
) ok = True
ap.add_argument( with tempfile.TemporaryDirectory() as tmp:
"--bootstrap-allowlist", cache = Path(tmp)
action="store_true", for pack in examples:
help="the base branch has no exceptions file yet (this PR introduces the gate), so " try:
"there is nothing to diff against. CI passes this only in that case.", loaded = sloppak.load_song(pack.name, pack.parent, cache)
) except Exception as e:
args = ap.parse_args() _fail(
f"core failed to load the spec's own example pack {pack.name}: "
spec = args.spec.resolve() f"{type(e).__name__}: {e}. A spec-valid pack must load."
if not (spec / "schemas" / "manifest.schema.json").exists(): )
_fail(f"{spec} does not look like a feedpak-spec checkout") ok = False
return 1 continue
if not loaded.song.arrangements:
print("[1/4] key-coverage — core reads/writes only keys the spec declares") _fail(f"core loaded {pack.name} but found no arrangements")
ok1 = check_key_coverage(spec) ok = False
print("[2/4] allowlist-closed — the grandfather list may shrink, never grow") continue
ok2 = check_allowlist_closed(args.baseline_exceptions, args.bootstrap_allowlist) print(f" loaded {pack.name}: {len(loaded.song.arrangements)} arrangement(s)")
print("[3/4] forward — core ingests the spec's example packs") print(f" forward: {'OK' if ok else 'FAILED'}")
ok3 = check_forward(spec) return ok
print("[4/4] reverse — committed packs satisfy the reference validator")
ok4 = check_reverse(spec)
def check_reverse(spec: Path) -> bool:
if ok1 and ok2 and ok3 and ok4: """Layer 3 — packs committed here must pass the spec's reference validator."""
print("\nfeedpak spec conformance: OK") packs = sorted({p for g in PACK_GLOBS for p in REPO.glob(g)})
return 0 if not packs:
print("\nfeedpak spec conformance: FAILED") print(" reverse: no committed packs — skipped")
return 1 return True
proc = subprocess.run(
if __name__ == "__main__": [sys.executable, str(spec / "tools" / "validate.py"), *[str(p) for p in packs]],
sys.exit(main()) capture_output=True,
text=True,
)
sys.stdout.write("".join(f" {ln}\n" for ln in proc.stdout.splitlines() if ln.strip()))
if proc.returncode != 0:
_fail(
"a pack committed to this repo does not satisfy the feedpak spec "
"(see the reference validator output above)."
)
if proc.stderr.strip():
sys.stderr.write(proc.stderr)
print(f" reverse: {'OK' if proc.returncode == 0 else 'FAILED'}")
return proc.returncode == 0
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument(
"--spec",
required=True,
type=Path,
help="path to a feedpak-spec checkout (CI pins the SHA in .feedpak-spec-ref)",
)
ap.add_argument(
"--baseline-exceptions",
type=Path,
help="the exceptions file as it exists on the base branch. Supplied by CI so the "
"allowlist can be proven to have not grown. Omit for a local run.",
)
ap.add_argument(
"--bootstrap-allowlist",
action="store_true",
help="the base branch has no exceptions file yet (this PR introduces the gate), so "
"there is nothing to diff against. CI passes this only in that case.",
)
args = ap.parse_args()
spec = args.spec.resolve()
if not (spec / "schemas" / "manifest.schema.json").exists():
_fail(f"{spec} does not look like a feedpak-spec checkout")
return 1
print("[1/4] key-coverage — core reads/writes only keys the spec declares")
ok1 = check_readers_complete() & check_key_coverage(spec)
print("[2/4] allowlist-closed — the grandfather list may shrink, never grow")
ok2 = check_allowlist_closed(args.baseline_exceptions, args.bootstrap_allowlist)
print("[3/4] forward — core ingests the spec's example packs")
ok3 = check_forward(spec)
print("[4/4] reverse — committed packs satisfy the reference validator")
ok4 = check_reverse(spec)
if ok1 and ok2 and ok3 and ok4:
print("\nfeedpak spec conformance: OK")
return 0
print("\nfeedpak spec conformance: FAILED")
return 1
if __name__ == "__main__":
sys.exit(main())