feedBack/docs/feedpak-spec-gate.md

The feedpak spec-conformance gate

tools/check_spec_conformance.py, run in CI as the feedpak-spec job.

Why

feedpak is published as an open format: its own repo (got-feedback/feedpak-spec), a normative spec, JSON Schemas, and a reference validator. That is a promise to everyone outside this codebase — third-party packers, converters, and players build against the spec, and the spec is meant to be the complete and authoritative description of a pack.

The moment core reads a manifest key the spec doesn't define, that promise breaks silently:

• A spec-compliant pack is no longer guaranteed to be a fully-working pack.
• The reference validator can't warn authors about a key it has never heard of — it will happily green-light the key, and every misspelling of it.
• The format's real definition drifts into our source tree. In the case that motivated this gate (#933), third-party tooling started emitting an original/ directory that no code anywhere requires — the convention was reverse-engineered from an example in a code comment.

The rule this gate enforces: any manifest key core reads or writes must be in the spec before core ships code that depends on it. Spec first, implementation second. Writes are not exempt — a key core writes lands in every pack we emit, so an undeclared one seeds the ecosystem with non-spec data.

Note that "get it into the spec" is not automatically the right fix for an existing violation — for original_audio it isn't. The spec already carries the pre-separation mixdown as a stem ({id: full, file: stems/full.ogg}), so that key added a second, redundant location for audio to a format that already had one, and the resolution is to remove it rather than bless it. The gate takes no position on which way a violation resolves; it only insists that one of the two happens deliberately, in the open, before the code merges.

What it checks

We can't mechanically prove core interprets a key the way the spec means. We can prove four surface properties, and they cover the drift that actually occurs.

Layer	Check	Catches
1. key-coverage	Every manifest key core reads or writes is declared in the spec's manifest.schema.json.	Core growing a key the spec never defined — the #933 class.
2. allowlist-closed	feedpak-spec-exceptions.yml has not grown relative to the base branch.	Someone routing around the FEP process by allowlisting their own new key.
3. forward	Core's load_song() ingests every example pack the spec ships.	The spec adding or tightening something core ignores or breaks on.
4. reverse	Every pack committed to this repo passes the spec's tools/validate.py.	Core (or a contributor) committing a pack the spec would reject.

Layer 1 works by walking the AST of the modules listed in READERS and collecting every literal key touched on a manifest dict (manifest.get("x"), manifest["x"], and the wrapped (load_manifest(p) or {}).get("x") form used in lib/enrichment.py).

Reads and writes are both checked, and reported differently. A key core writes (manifest["x"] = v, as lib/songmeta.py does) is spec surface pointed outward: it puts a key into every pack we emit, so an undeclared one seeds the ecosystem with non-spec data. Subscripts are classified by AST context — Store is a write, Load is a read — so manifest["year"] = ... is not miscounted as a read.

When it fails

You added a manifest key the spec doesn't define. There is exactly one way forward, and it is not in this repo.

Land the key in the spec through the feedpak Enhancement Proposal (FEP) process (feedpak-spec/CONTRIBUTING.md):

1. Open a FEP issue on got-feedback/feedpak-spec — the problem, the proposed on-disk shape (manifest key and/or side-file), backward compatibility, and the version bump it implies.
2. Discuss, until it has a clear shape and rough consensus.
3. Land one PR there that updates the normative spec (spec/feedpak-v1.md), the relevant JSON Schema(s), an example in examples/ that exercises it, and the changelog — together. A PR touching only one of those is incomplete.
4. Back here, bump .feedpak-spec-ref to that merged SHA, in the same PR as your code. The gate goes green, because the key is now genuinely part of the format.

That is deliberately the only route. There is no in-repo escape hatch — no experimental prefix, no self-serve allowlist. If your PR is blocked, the answer is a FEP, not a workaround. The person merging has to stop and decide whether the change is worth taking through the format process, which is the whole point.

The spec's own governance says the same thing:

This repository defines the format only. Applications that read or write feedpak ... track this spec as a dependency; they do not drive it. A change is not part of the format until it lands here. — feedpak-spec/GOVERNANCE.md

feedpak-spec-exceptions.yml is a closed grandfather list, not a hatch

It exists solely because original_audio predates the gate. CI fails any PR that adds an entry (layer 2 diffs it against the base branch), so the list can only ever shrink. Entries are debt, each carries a tracking issue, and each disappears when the underlying key is removed from core. The gate also fails on a stale entry — the spec caught up, or core stopped touching the key — so the file cannot quietly become somewhere drift accumulates.

Deleting an entry does not, by itself, get you past the gate: layer 1 still fails while core reads the key. The entry goes when the code goes.

Pinning

.feedpak-spec-ref holds the SHA of the feedpak-spec commit this repo is verified against. Pinned rather than tracking the spec's default branch on purpose — a change over there must never turn CI red on an unrelated PR here.

When the spec moves, bump the SHA in its own PR. If that PR is red, the spec changed in a way core doesn't satisfy — exactly the signal we want, delivered as a reviewable PR rather than a surprise on someone else's branch.

Limitations

Known, and worth fixing in follow-ups rather than blocking on:

• Layer 1 is name-heuristic. It recognises manifest dicts bound to locals named in MANIFEST_VARS (manifest, mf) plus the load_manifest(...) call form. This works because the loaders use a uniform idiom, but it is fragile against a refactor that renames the local. The readers-complete guard limits the blast radius — a module that touches manifest keys can't go unscanned — but a renamed local inside a scanned module would still slip. The hardening step is to route all manifest access through a single declared KNOWN_MANIFEST_KEYS registry in lib/sloppak.py; the gate then compares registry against schema exactly instead of inferring.
• Layer 1 covers top-level keys only. Nested structure (arrangements[].file, .id, .notation) isn't checked. Extending to it means walking the schema's $ref subschemas.
• Layer 1 recognises get, setdefault, and subscripts as key access. update() and pop() aren't used against a feedpak manifest anywhere in the tree, so they're deliberately not special-cased rather than speculatively handled — but note KEY_OPS (which drives readers-complete) doesn't look for them either.
• Layer 4 can't catch unknown keys, because manifest.schema.json sets additionalProperties: true and the reference validator deliberately "treats unknown keys/files as forward-compatible". Fixing this properly belongs in the spec (tighten the schema, or give the validator a --strict mode). Until then, layer 1 is the only thing standing between us and the next original_audio.