R0: module-migration rails (src/ serving, live-edit cache, scriptType loading, governance) (#812)
ship-ci / ci (push) Has been cancelled

Host enablement for the plugin ES-module migration: sandboxed /api/plugins/{id}/src/ serving, no-cache+weak-ETag/304 live-edit caching on src/+screen.js+assets, scriptType:module loader injection + scriptType/minHost manifest passthrough; constitution v1.2.0 + module playbook + signed size-exemptions register + maintainer/CI-only ESLint gate; rerunnable perf-baseline harness. Reviewed by Codex (local), Copilot, and CodeRabbit.
This commit is contained in:
Byron Gamatos
2026-07-08 10:14:40 +02:00
committed by GitHub
parent a18a818e8b
commit 950e348357
15 changed files with 2520 additions and 19 deletions
+67
View File
@@ -0,0 +1,67 @@
# Perf baseline — module-migration refactor
The refactor promises "measured runtime wins, no hand-waved perf claims" and
"screen-entry and frame-time no worse." This is the baseline to hold it to.
Rerun the harness after every phase (R0 → R3c) and compare.
## Running it
```
# 1. start core against a library with real charts (see caveat below)
CONFIG_DIR=… DLC_DIR=/path/to/songs PYTHONPATH=lib \
python3 -m uvicorn server:app --host 127.0.0.1 --port 8000
# 2. capture (maintainer/CI-only; uses the committed Playwright chromium)
node scripts/perf-baseline.mjs --base http://127.0.0.1:8000 --n 60 --soak 30
```
The script prints a markdown block; paste it under "Results" below with the date
and the commit it was taken at.
## What it measures
- **Server latency** — p50/p95/p99 over N requests for `/api/version`,
`/api/plugins`, `/api/library`, `/api/library/artists`.
- **Cold boot → interactive** — full page load to `networkidle`.
- **JS heap** — `performance.memory.usedJSHeapSize` after load and after an idle
soak (a leak signal across a session).
- **Plugin-script shape** — how many plugin `<script>`s the loader injected (a
"the app booted with its plugins" sanity signal).
**Not yet captured — needs a seeded library with charts** (fill in when run
against a real environment): playback **frame-time p95** on the 2D and 3D
highway, and **screen-entry** (plugin inject → interactive) for
editor / notedetect / highway_3d with a chart loaded. These are the
perf-sensitive numbers that gate the `highway.js` split (R3c); the harness has
the hooks, they just need real songs in `DLC_DIR`.
## Results
### R0 baseline — 2026-07-08 (branch `feat/r0-plugin-module-rails`)
> ⚠️ A quick capture (`--n 50 --soak 8`) against an **empty** library (no charts
> in `DLC_DIR`), so the `/api/library*` and boot numbers are floor values —
> re-take on a seeded environment with the recommended `--n 60 --soak 30` for the
> real R0 baseline before comparing R1+ against it. Recorded here to prove the
> harness and lock the methodology.
Server latency (ms), n=50:
| Endpoint | status | p50 | p95 | p99 |
|---|---|---|---|---|
| `/api/version` | 200 | 0.9 | 1.8 | 22.3 |
| `/api/plugins` | 200 | 1.6 | 2.1 | 3.4 |
| `/api/library?limit=60` | 200 | 1.4 | 1.7 | 2.9 |
| `/api/library/artists` | 200 | 1.3 | 1.8 | 2.7 |
Client:
| Metric | Value |
|---|---|
| Cold boot → networkidle | 1268 ms |
| JS heap after load | 10.1 MB |
| JS heap after idle soak | 10.1 MB (no idle growth) |
| Plugin scripts injected | 12 |
No plugin has migrated yet, so all 12 are classic. When the R1 pilot (stems)
lands, cold-boot / heap should not regress.
+103
View File
@@ -0,0 +1,103 @@
# Plugin ES-module migration playbook
How to move a plugin off a single global-scope `screen.js` IIFE onto a native
ES-module graph — **no build step, no framework, no bundler**. This is the
mechanism the monolith-killing refactor uses; the host rails for it shipped in
R0 (see `.specify/memory/constitution.md` Principle II + the "Module load
contract" in Operating Constraints).
## The shape
```
my-plugin/
plugin.json + "scriptType": "module" ← opt in
screen.js import './src/main.js'; ← the entire file
src/
state.js (0) module state + accessors
util/… (1) pure helpers — real-import testable
…/… (2..4) model → render/audio/io → input
globals.js (5) THE ONLY file that writes window.*
main.js (5) boot: wire modules, register screen:changed
assets/… worklets / WASM / images (unchanged, served as today)
```
`screen.js` becomes a one-line static `import`. The host injects it as
`<script type="module">`, whose load event fires **only after the whole
static-import graph fetches and evaluates** — so the loader's
completion-by-`onload` + `_loadingPluginId` window + `playSong` wrapper-chain
order are all preserved. (A classic IIFE that fired a fire-and-forget
`import()` would break that contract — don't do that; use `scriptType:"module"`.)
## Non-negotiable rules
1. **Source-served, no build.** Modules are plain source files fetched from
`/api/plugins/<id>/src/<path>`. No bundler, transpiler, or TypeScript.
2. **Layering points downward** — `state → util → commands/model →
render/audio/io → input → globals/main`. A lint check (`import-x/no-cycle`)
enforces acyclicity; extract bottom-up so each move only imports
already-extracted layers.
3. **`globals.js` is the only writer of `window.*`.** The deliberate global
surface shrinks to one auditable file; everything else is module-scoped.
4. **Import-time purity.** `node --test` runs a module's top-level code on
import, so a module you want to unit-test must be side-effect-free at import:
no `document` / `window` / `localStorage` at module top level — lift init
into an exported `init()` called by `main.js`. (Constitution Principle V's
"no implicit IO at import time", applied to the frontend.) Tests are `.mjs`
and use real `import`, retiring the regex/`extractFunction` harness.
5. **Assets resolve via `import.meta.url`.** `document.currentScript` is `null`
inside a module. `assets/` lives at the plugin root, so a `src/` module must
climb out of `src/`: from `src/main.js`, `new URL('../assets/x.js',
import.meta.url)` (deeper modules need more `../`). Simpler and
depth-independent: the absolute route `/api/plugins/<id>/assets/x.js`.
Worklets run in a *separate* module graph (`AudioWorkletGlobalScope`) and
cannot share modules with `src/`.
6. **Re-init comes from `screen:changed`, not re-execution.** The host loads
`screen.js` once per version and `showScreen` re-injects nothing, so module
top-level code does **not** re-run when the user re-enters the screen at the
same version. Keep per-visit setup/teardown in a `window.feedBack.on(
'screen:changed', …)` handler — exactly as classic plugins (tuner,
minigames) already do. Do not rely on the IIFE re-running.
7. **Inline `onclick=` keeps working** during migration via `globals.js` (which
keeps every referenced symbol on `window`); retire inline handlers to
module-side `addEventListener` opportunistically, never as a blocking step.
## The live-edit loop
The host serves `screen.js`, `src/**`, and `assets/**` with
`Cache-Control: no-cache` + a weak `ETag` and honors `If-None-Match` → `304`.
So: edit a `src/` file → **refresh the browser** → the edited module returns
`200` and reloads while every unchanged module `304`s. There is no hot-reload;
the loop is edit → refresh → see change, exactly as before. The `?v=<version>`
query on `screen.js` is the legacy version buster; it does **not** propagate
into the `src/` graph and does not need to — ETag/mtime is the correctness
authority for the whole graph.
## Host-version floor (`minHost`)
A migrated plugin *requires* a host new enough to serve `src/` and inject
`type=module`. Declare the floor with `"minHost": "X.Y.Z"` in `plugin.json`.
(R0 plumbs the field through `/api/plugins`; enforcement — refuse-with-message
on an older host — is deferred, so bundled plugins are unaffected. Community
plugins should state the floor and not migrate below it.)
## Migration mechanics
- **Move-only PRs.** One slice extracts one module: cut code, add
imports/exports, update `globals.js` — zero behavior change. Behavior fixes
are separate PRs. (Init-lifts for import purity are the one non-pure move —
budget them.)
- **Bottom-up, layer by layer.** Within a layer, independent modules are
independent PRs (a DAG, not a chain); use a git worktree per branch.
- Tests move with their subject and convert to real `.mjs` imports in the same
PR (assertions unchanged).
- Size norm: no source file over **1,500 lines**; legitimate exceptions
(hot renderers, etc.) go in the signed register at `docs/size-exemptions.md`.
## Verifying a migration
`node --test <plugin>/tests/*.mjs`; load the plugin on the `:8000` testbed and
confirm it boots (`<script type=module>` in DevTools, the `src/` graph in
Network); edit a `src/` file → refresh → change visible (`200` on the edited
file, `304` on the rest); leave and re-enter the screen at the same version →
it re-inits via `screen:changed`. The R1 pilots (stems, then studio) certify
this end-to-end before the flagship repos migrate.
+62
View File
@@ -0,0 +1,62 @@
# Size-exemption register
The working norm (constitution Principle II; enforced by the `max-lines` lint
gate) is **no source file over 1,500 lines**. A few files are allowed to exceed
it because splitting them would do more harm than good — hot per-frame
renderers, C++, offline generators, cohesive registries. This register is the
list of those exceptions: each row is a **deliberate, signed** decision with a
ceiling, a rationale, and a review trigger. Without it, "no file over 1,500
without a *signed* exemption" is unenforceable.
**Rules**
- One row per file: a ceiling, a rationale, a signer, a review trigger.
- The `max-lines` per-file ceilings in `eslint.config.js` mirror this table —
keep them in sync (this register is canonical).
- Files with a scheduled split **plan** are *not* exempt — they live in
"Planned, not exempt" at the bottom so nothing falls between the two states.
- **Signers** (decided 2026-07-08): **Byron** signs core + bundled rows;
**Christian** signs the authored-plugin row (virtuoso, its own repo/track).
## Permanent exemptions (structural rationale)
| Repo / file | Lines (7-07) | Ceiling | Rationale | Signer | Review |
|---|---|---|---|---|---|
| core `static/highway.js` → residual `renderer-2d.js` (post-split) | ~2,400–2,900 est. | **3,000** | 60 fps hot path; no module boundary inside the per-frame loop | Byron | after the highway.js split |
| core `plugins/highway_3d/` → residual renderer | sized at split; likely **>3,000** | set at split, flagged now | same hot-path rule; the draw core can't be cut without behavior risk | Byron | after the highway_3d split |
| core `static/capabilities.js` | 1,538 | 1,600 | cohesive registry + `window.feedBack` bus, 38 lines over; a split spends credibility for nothing | Byron | R4 |
| tutorials `builtin/reading-the-highway/generate.py` | 1,818 | 2,000 | offline content generator, never imported at runtime, deps not in runtime requirements | Byron | if a 3rd builtin pack appears |
| desktop `src/audio/NodeAddon.cpp` | 3,542 | as-is | C++, outside the ESM/routes playbooks; under active use-after-free crash work — do not churn | Byron | after crash-class work settles |
| desktop `src/audio/AudioEngine.cpp` | 2,977 | as-is | same | Byron | same |
| desktop `src/vst-host/main.cpp` | 1,928 | as-is | same | Byron | same |
| virtuoso `screen.js` (authored, own track) | 25,741 | as-is until its own split | authored plugin on a separate roadmap; migrates on its own schedule | Christian | virtuoso split kickoff |
## Split-when-touched (no scheduled train; row retires when split)
| Repo / file | Lines | Ceiling | Rationale | Signer | Review |
|---|---|---|---|---|---|
| core `lib/gp2rs_gpx.py` | 2,540 | as-is | import converter, off the serve-path hot loop | Byron | when next touched |
| core `lib/gp2rs.py` | 2,055 | as-is | same | Byron | when next touched |
| core `lib/song.py` | 1,689 | as-is | data models + wire format; cohesive | Byron | when next touched |
| core `lib/gp_autosync.py` | 1,572 | as-is | under active dev (#787/#791) — don't collide | Byron | after in-flight work lands |
| core `plugins/capability_inspector/screen.js` | 1,752 | as-is | bundled diagnostics plugin, low churn | Byron | when next touched |
| core `plugins/folder_library/screen.js` | 1,672 | as-is | bundled plugin, low churn | Byron | when next touched |
## Temporary rows (cleared by a scheduled PR)
| Repo / file | Lines | Cleared by |
|---|---|---|
| core `plugins/__init__.py` | ~2,470 (grew under R0) | the `plugins/_routes.py` + `plugins/_registry.py` split (rides the server.py router work) |
## Watch list (under the norm — no row needed, re-census each phase)
`musicxml-import/mxml2notation.py` (1,456) · core `static/capabilities/audio-effects.js`
(1,436) · `studio routes.py` (1,399) · `update-manager screen.js` (1,492 — zero headroom).
## Planned, NOT exempt (owned by split plans — listed so nothing falls between states)
core `static/app.js` (11,821) · `static/highway.js` (4,154, whole file) · `server.py`
(13,948) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
(2,974) · `plugins/highway_3d/screen.js` (15,656) · `plugins/keys_highway_3d/screen.js`
(3,780) · `plugins/drum_highway_3d/screen.js` (3,597) — and every monolith with a PR
train in the refactor plan. Test files (e.g. `tests/test_plugins.py`) are out of scope
by policy — the norm governs source files.