mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-07-19 19:31:26 +00:00
R0: module-migration rails (src/ serving, live-edit cache, scriptType loading, governance) (#812)
Some checks failed
ship-ci / ci (push) Has been cancelled
Some checks failed
ship-ci / ci (push) Has been cancelled
Host enablement for the plugin ES-module migration: sandboxed /api/plugins/{id}/src/ serving, no-cache+weak-ETag/304 live-edit caching on src/+screen.js+assets, scriptType:module loader injection + scriptType/minHost manifest passthrough; constitution v1.2.0 + module playbook + signed size-exemptions register + maintainer/CI-only ESLint gate; rerunnable perf-baseline harness. Reviewed by Codex (local), Copilot, and CodeRabbit.
This commit is contained in:
parent
a18a818e8b
commit
950e348357
25
.github/workflows/ci.yml
vendored
25
.github/workflows/ci.yml
vendored
@ -123,3 +123,28 @@ jobs:
|
||||
sys.exit(1)
|
||||
print(f"Validated {len(manifests)} manifest(s) — OK")
|
||||
EOF
|
||||
|
||||
lint:
|
||||
# Maintainer/CI-only size + module-hygiene gate (constitution Principle I:
|
||||
# dev tooling, never on the serve/Docker path — same category as
|
||||
# scripts/build-tailwind.sh). max-lines WARNS (the 1,500-line size ratchet;
|
||||
# non-blocking), while import-x/no-unresolved + no-cycle HARD-ERROR on the
|
||||
# ES-module graphs the refactor produces. Exemptions: docs/size-exemptions.md.
|
||||
name: lint
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
# npm ci runs third-party postinstall scripts; don't leave the token in
|
||||
# git config for them (this job never pushes).
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: ESLint (size norm + module hygiene)
|
||||
run: npm run lint
|
||||
|
||||
@ -48,11 +48,30 @@ output is committed, so Docker / desktop / end users never build. New
|
||||
features extend `app.js` and the existing globals (`window.playSong`,
|
||||
`window.showScreen`, `window.createHighway`, `window.feedBack`).
|
||||
|
||||
Native ES modules are a first-class, build-free extension mechanism.
|
||||
Because `<script type="module">` and `import` are browser features — not
|
||||
a bundler — a large source file MAY be split into an `import`-ed module
|
||||
graph of plain source files, with **no build step and no framework**. A
|
||||
plugin opts in with `"scriptType": "module"` in `plugin.json`: its
|
||||
`screen.js` becomes a one-line `import './src/main.js'`, and the host
|
||||
serves the `src/` subtree from the sandboxed `/api/plugins/<id>/src/…`
|
||||
route and injects the entry as `<script type="module">`. The classic
|
||||
global-scope `screen.js` path remains fully supported; both coexist, and
|
||||
module scripts are still source-served — the no-bundler, no-transpiler,
|
||||
build-free-at-serve rule is unchanged. Core's own `static/` tree may
|
||||
migrate to the same module-graph shape (`static/js/…`) over time under
|
||||
this rule.
|
||||
|
||||
**Non-negotiable rules**
|
||||
|
||||
- Do not introduce a frontend framework, JSX, or a JS build pipeline in
|
||||
core. Plugins MAY ship their own bundled assets but core MUST remain
|
||||
source-served.
|
||||
- ES-module plugins remain source-served: no bundler or transpiler, and
|
||||
their own asset URLs (worklets, WASM, images) resolve via
|
||||
`import.meta.url` — never `document.currentScript`, which is `null`
|
||||
inside a module. `scriptType:"module"` and the optional `minHost`
|
||||
version floor are the only new `plugin.json` keys the module path adds.
|
||||
- Because the core Tailwind stylesheet is prebuilt, it contains only the
|
||||
classes present in core source at build time. Core's committed
|
||||
`static/tailwind.min.css` MUST stay in sync with source — CI enforces
|
||||
@ -214,6 +233,15 @@ no `..`, no absolute paths).
|
||||
runs first). Plugins MUST tolerate dependent globals being absent
|
||||
at load time and check at runtime
|
||||
(`typeof window.X === 'function'`).
|
||||
- **Module load contract**: a `scriptType:"module"` plugin is injected
|
||||
as `<script type="module">`, whose load event fires only after its
|
||||
whole static-import graph fetches and evaluates — so the loader's
|
||||
completion-by-`onload` guarantee (and the `playSong` wrapper-chain
|
||||
order above) is preserved exactly. The host loads `screen.js` once per
|
||||
version and `showScreen` re-injects nothing, so a plugin's per-visit
|
||||
re-initialization comes from its `screen:changed` handler, not from
|
||||
screen.js re-running; ES-module plugins inherit this unchanged (module
|
||||
top-level code does not re-execute on same-version re-mount).
|
||||
|
||||
## Development Workflow
|
||||
|
||||
@ -256,4 +284,4 @@ no `..`, no absolute paths).
|
||||
higher-numbered principle's escape hatch is to live in a plugin
|
||||
with its own bundled assets.
|
||||
|
||||
**Version**: 1.1.0 | **Ratified**: 2026-05-09 | **Last Amended**: 2026-06-01
|
||||
**Version**: 1.2.0 | **Ratified**: 2026-05-09 | **Last Amended**: 2026-07-08
|
||||
|
||||
@ -8,6 +8,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Plugins can ship an ES-module `src/` tree (module-migration rails, R0).** The host gains three things so a plugin can move off a single global-scope `screen.js` IIFE onto native ES modules with **no build step**: (1) a new sandboxed `GET /api/plugins/{id}/src/{path}` route that serves a plugin's `src/` source subtree, containment-checked by the same `safe_join` guard as `assets/` (traversal/absolute/NUL → 404); (2) the live-edit cache contract — `Cache-Control: no-cache` + a weak mtime/size `ETag` + `If-None-Match`→`304` — applied to `src/`, `screen.js`, and `assets/` (previously `screen.js` sent no cache headers and `assets/` emitted an ETag but never revalidated), so an edited module reloads on refresh while unchanged ones `304`; and (3) `scriptType`/`minHost` passthrough from `plugin.json` to `/api/plugins`, with the loader injecting a plugin that declares `"scriptType":"module"` as `<script type="module">` (its screen.js becomes `import './src/main.js'`). A `<script type=module>` fires its load event only after its whole static-import graph evaluates, preserving the loader's completion-by-`onload` + `_loadingPluginId` contract. Classic plugins are unaffected; `minHost` is passthrough-only for now (enforcement deferred). Tests: `tests/test_plugin_src_route.py` (serve/media-type/traversal/304/no-stale-304/screen.js+assets conditional), `tests/js/plugin_loader_script_type.test.js` (guarded module injection).
|
||||
- **Module-migration governance & rails (R0).** Constitution amended to **v1.2.0**: Principle II now names native ES modules as a first-class, *build-free* extension mechanism (the `scriptType:"module"` load path, both plugins and — over time — core's `static/js/`), keeping the no-bundler/no-transpiler/source-served rule intact; Operating Constraints gains a "Module load contract" clause (a `<script type=module>` load event awaits the whole static-import graph, so completion-by-`onload` is preserved; per-visit re-init comes from the `screen:changed` event, not screen.js re-execution). Mirrored into `CLAUDE.md`. New `docs/plugin-modules.md` (the migration playbook — layering, import-time purity, `import.meta.url` assets, the ETag live-edit loop) and `docs/size-exemptions.md` (the signed 1,500-line size-norm register; Byron signs core/bundled rows, Christian the authored virtuoso row). Adds a **maintainer/CI-only** ESLint gate (`eslint.config.js` + a `lint` CI job): `max-lines` warns at 1,500 as a non-blocking ratchet (ceilings for exempt files mirror the register), and `import-x/no-unresolved` + `import-x/no-cycle` hard-error on ES-module graphs — dormant until module code lands, never on the serve/Docker path.
|
||||
- **Perf-baseline harness (R0).** `scripts/perf-baseline.mjs` (maintainer-only, Playwright-driven) captures server p50/p95/p99 latency, cold boot-to-interactive, JS-heap after an idle soak, and the injected plugin-script shape (how many are `type=module`), so every refactor phase can be checked for "screen-entry and frame-time no worse." Methodology + the R0 baseline live in `docs/perf-baseline.md`; playback frame-time and chart-loaded screen-entry rows need a seeded library and are re-taken per environment.
|
||||
- **Sort and filter the library by your personal difficulty rating — now visible at a glance, not just in the edit drawer.** `song_user_meta.user_difficulty` (the 1–5 planning rating, settable manually or seeded by a plugin like the community `difficulty_tagger`) already existed but was only readable by opening a song's per-song details drawer. The library API gains `sort=difficulty` / `sort=difficulty-desc` — a correlated subquery over `song_user_meta`, following the same unrated-rows-sort-to-the-bottom-in-both-directions pattern as the existing `mastery` sort — and library cards now show the rating as a `◆N` badge (v2 grid/tree views and the v3 grid alike), next to the tuning and lyrics badges. The classic tree view's `query_artists` batch-attaches `user_difficulty` the same way `query_page` already did for the grid, so the badge actually renders there too instead of staying dark. Tests: `tests/test_library_filters.py::test_difficulty_sort_pushes_unrated_to_bottom`, `tests/test_library_filters.py::test_tree_view_songs_carry_user_difficulty`.
|
||||
- **`lib/midi_import.py`: `convert_midi_tempo_map` — MIDI imports can finally carry
|
||||
their bars.** The keys/drums note converters always computed a tempo-aware
|
||||
|
||||
@ -117,6 +117,8 @@ Notes:
|
||||
|
||||
**Frontend scripts** — `screen.js` runs in the global scope via a `<script>` tag. It can access `window.playSong`, `window.showScreen`, `window.createHighway`, the `<audio>` element, and the `window.feedBack` event emitter.
|
||||
|
||||
**ES-module plugins (`scriptType:"module"`)** — a plugin may instead ship a native ES-module graph with **no build step**: set `"scriptType": "module"` in `plugin.json`, make `screen.js` a one-line `import './src/main.js'`, and put the module tree under `src/` (served by the sandboxed `/api/plugins/<id>/src/{path}` route). The host injects it as `<script type="module">`, whose `onload` fires only after the whole static-import graph evaluates — so the loader's completion-by-`onload` + `_loadingPluginId` + `playSong` wrapper-chain ordering all hold. Resolve your own asset URLs (worklets, WASM) with `import.meta.url` — `document.currentScript` is `null` in a module. Module top-level code does **not** re-run when the user re-enters the screen at the same version (the host loads screen.js once and `showScreen` re-injects nothing), so keep per-visit re-init in a `screen:changed` handler, exactly as classic plugins do. Classic global-scope `screen.js` remains fully supported. See `docs/plugin-modules.md`.
|
||||
|
||||
**The playSong wrapper chain** — Plugins commonly wrap `window.playSong` to hook into song playback. Plugins load alphabetically, so the last-loaded (alphabetically later) wrapper runs first, while the alphabetically first plugin runs closest to the original. Be aware that `await` calls in inner wrappers yield to the event loop — WebSocket messages can arrive before outer wrappers finish setup.
|
||||
|
||||
## Plugin Best Practices
|
||||
|
||||
67
docs/perf-baseline.md
Normal file
67
docs/perf-baseline.md
Normal file
@ -0,0 +1,67 @@
|
||||
# Perf baseline — module-migration refactor
|
||||
|
||||
The refactor promises "measured runtime wins, no hand-waved perf claims" and
|
||||
"screen-entry and frame-time no worse." This is the baseline to hold it to.
|
||||
Rerun the harness after every phase (R0 → R3c) and compare.
|
||||
|
||||
## Running it
|
||||
|
||||
```
|
||||
# 1. start core against a library with real charts (see caveat below)
|
||||
CONFIG_DIR=… DLC_DIR=/path/to/songs PYTHONPATH=lib \
|
||||
python3 -m uvicorn server:app --host 127.0.0.1 --port 8000
|
||||
|
||||
# 2. capture (maintainer/CI-only; uses the committed Playwright chromium)
|
||||
node scripts/perf-baseline.mjs --base http://127.0.0.1:8000 --n 60 --soak 30
|
||||
```
|
||||
|
||||
The script prints a markdown block; paste it under "Results" below with the date
|
||||
and the commit it was taken at.
|
||||
|
||||
## What it measures
|
||||
|
||||
- **Server latency** — p50/p95/p99 over N requests for `/api/version`,
|
||||
`/api/plugins`, `/api/library`, `/api/library/artists`.
|
||||
- **Cold boot → interactive** — full page load to `networkidle`.
|
||||
- **JS heap** — `performance.memory.usedJSHeapSize` after load and after an idle
|
||||
soak (a leak signal across a session).
|
||||
- **Plugin-script shape** — how many plugin `<script>`s the loader injected (a
|
||||
"the app booted with its plugins" sanity signal).
|
||||
|
||||
**Not yet captured — needs a seeded library with charts** (fill in when run
|
||||
against a real environment): playback **frame-time p95** on the 2D and 3D
|
||||
highway, and **screen-entry** (plugin inject → interactive) for
|
||||
editor / notedetect / highway_3d with a chart loaded. These are the
|
||||
perf-sensitive numbers that gate the `highway.js` split (R3c); the harness has
|
||||
the hooks, they just need real songs in `DLC_DIR`.
|
||||
|
||||
## Results
|
||||
|
||||
### R0 baseline — 2026-07-08 (branch `feat/r0-plugin-module-rails`)
|
||||
|
||||
> ⚠️ A quick capture (`--n 50 --soak 8`) against an **empty** library (no charts
|
||||
> in `DLC_DIR`), so the `/api/library*` and boot numbers are floor values —
|
||||
> re-take on a seeded environment with the recommended `--n 60 --soak 30` for the
|
||||
> real R0 baseline before comparing R1+ against it. Recorded here to prove the
|
||||
> harness and lock the methodology.
|
||||
|
||||
Server latency (ms), n=50:
|
||||
|
||||
| Endpoint | status | p50 | p95 | p99 |
|
||||
|---|---|---|---|---|
|
||||
| `/api/version` | 200 | 0.9 | 1.8 | 22.3 |
|
||||
| `/api/plugins` | 200 | 1.6 | 2.1 | 3.4 |
|
||||
| `/api/library?limit=60` | 200 | 1.4 | 1.7 | 2.9 |
|
||||
| `/api/library/artists` | 200 | 1.3 | 1.8 | 2.7 |
|
||||
|
||||
Client:
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Cold boot → networkidle | 1268 ms |
|
||||
| JS heap after load | 10.1 MB |
|
||||
| JS heap after idle soak | 10.1 MB (no idle growth) |
|
||||
| Plugin scripts injected | 12 |
|
||||
|
||||
No plugin has migrated yet, so all 12 are classic. When the R1 pilot (stems)
|
||||
lands, cold-boot / heap should not regress.
|
||||
103
docs/plugin-modules.md
Normal file
103
docs/plugin-modules.md
Normal file
@ -0,0 +1,103 @@
|
||||
# Plugin ES-module migration playbook
|
||||
|
||||
How to move a plugin off a single global-scope `screen.js` IIFE onto a native
|
||||
ES-module graph — **no build step, no framework, no bundler**. This is the
|
||||
mechanism the monolith-killing refactor uses; the host rails for it shipped in
|
||||
R0 (see `.specify/memory/constitution.md` Principle II + the "Module load
|
||||
contract" in Operating Constraints).
|
||||
|
||||
## The shape
|
||||
|
||||
```
|
||||
my-plugin/
|
||||
plugin.json + "scriptType": "module" ← opt in
|
||||
screen.js import './src/main.js'; ← the entire file
|
||||
src/
|
||||
state.js (0) module state + accessors
|
||||
util/… (1) pure helpers — real-import testable
|
||||
…/… (2..4) model → render/audio/io → input
|
||||
globals.js (5) THE ONLY file that writes window.*
|
||||
main.js (5) boot: wire modules, register screen:changed
|
||||
assets/… worklets / WASM / images (unchanged, served as today)
|
||||
```
|
||||
|
||||
`screen.js` becomes a one-line static `import`. The host injects it as
|
||||
`<script type="module">`, whose load event fires **only after the whole
|
||||
static-import graph fetches and evaluates** — so the loader's
|
||||
completion-by-`onload` + `_loadingPluginId` window + `playSong` wrapper-chain
|
||||
order are all preserved. (A classic IIFE that fired a fire-and-forget
|
||||
`import()` would break that contract — don't do that; use `scriptType:"module"`.)
|
||||
|
||||
## Non-negotiable rules
|
||||
|
||||
1. **Source-served, no build.** Modules are plain source files fetched from
|
||||
`/api/plugins/<id>/src/<path>`. No bundler, transpiler, or TypeScript.
|
||||
2. **Layering points downward** — `state → util → commands/model →
|
||||
render/audio/io → input → globals/main`. A lint check (`import-x/no-cycle`)
|
||||
enforces acyclicity; extract bottom-up so each move only imports
|
||||
already-extracted layers.
|
||||
3. **`globals.js` is the only writer of `window.*`.** The deliberate global
|
||||
surface shrinks to one auditable file; everything else is module-scoped.
|
||||
4. **Import-time purity.** `node --test` runs a module's top-level code on
|
||||
import, so a module you want to unit-test must be side-effect-free at import:
|
||||
no `document` / `window` / `localStorage` at module top level — lift init
|
||||
into an exported `init()` called by `main.js`. (Constitution Principle V's
|
||||
"no implicit IO at import time", applied to the frontend.) Tests are `.mjs`
|
||||
and use real `import`, retiring the regex/`extractFunction` harness.
|
||||
5. **Assets resolve via `import.meta.url`.** `document.currentScript` is `null`
|
||||
inside a module. `assets/` lives at the plugin root, so a `src/` module must
|
||||
climb out of `src/`: from `src/main.js`, `new URL('../assets/x.js',
|
||||
import.meta.url)` (deeper modules need more `../`). Simpler and
|
||||
depth-independent: the absolute route `/api/plugins/<id>/assets/x.js`.
|
||||
Worklets run in a *separate* module graph (`AudioWorkletGlobalScope`) and
|
||||
cannot share modules with `src/`.
|
||||
6. **Re-init comes from `screen:changed`, not re-execution.** The host loads
|
||||
`screen.js` once per version and `showScreen` re-injects nothing, so module
|
||||
top-level code does **not** re-run when the user re-enters the screen at the
|
||||
same version. Keep per-visit setup/teardown in a `window.feedBack.on(
|
||||
'screen:changed', …)` handler — exactly as classic plugins (tuner,
|
||||
minigames) already do. Do not rely on the IIFE re-running.
|
||||
7. **Inline `onclick=` keeps working** during migration via `globals.js` (which
|
||||
keeps every referenced symbol on `window`); retire inline handlers to
|
||||
module-side `addEventListener` opportunistically, never as a blocking step.
|
||||
|
||||
## The live-edit loop
|
||||
|
||||
The host serves `screen.js`, `src/**`, and `assets/**` with
|
||||
`Cache-Control: no-cache` + a weak `ETag` and honors `If-None-Match` → `304`.
|
||||
So: edit a `src/` file → **refresh the browser** → the edited module returns
|
||||
`200` and reloads while every unchanged module `304`s. There is no hot-reload;
|
||||
the loop is edit → refresh → see change, exactly as before. The `?v=<version>`
|
||||
query on `screen.js` is the legacy version buster; it does **not** propagate
|
||||
into the `src/` graph and does not need to — ETag/mtime is the correctness
|
||||
authority for the whole graph.
|
||||
|
||||
## Host-version floor (`minHost`)
|
||||
|
||||
A migrated plugin *requires* a host new enough to serve `src/` and inject
|
||||
`type=module`. Declare the floor with `"minHost": "X.Y.Z"` in `plugin.json`.
|
||||
(R0 plumbs the field through `/api/plugins`; enforcement — refuse-with-message
|
||||
on an older host — is deferred, so bundled plugins are unaffected. Community
|
||||
plugins should state the floor and not migrate below it.)
|
||||
|
||||
## Migration mechanics
|
||||
|
||||
- **Move-only PRs.** One slice extracts one module: cut code, add
|
||||
imports/exports, update `globals.js` — zero behavior change. Behavior fixes
|
||||
are separate PRs. (Init-lifts for import purity are the one non-pure move —
|
||||
budget them.)
|
||||
- **Bottom-up, layer by layer.** Within a layer, independent modules are
|
||||
independent PRs (a DAG, not a chain); use a git worktree per branch.
|
||||
- Tests move with their subject and convert to real `.mjs` imports in the same
|
||||
PR (assertions unchanged).
|
||||
- Size norm: no source file over **1,500 lines**; legitimate exceptions
|
||||
(hot renderers, etc.) go in the signed register at `docs/size-exemptions.md`.
|
||||
|
||||
## Verifying a migration
|
||||
|
||||
`node --test <plugin>/tests/*.mjs`; load the plugin on the `:8000` testbed and
|
||||
confirm it boots (`<script type=module>` in DevTools, the `src/` graph in
|
||||
Network); edit a `src/` file → refresh → change visible (`200` on the edited
|
||||
file, `304` on the rest); leave and re-enter the screen at the same version →
|
||||
it re-inits via `screen:changed`. The R1 pilots (stems, then studio) certify
|
||||
this end-to-end before the flagship repos migrate.
|
||||
62
docs/size-exemptions.md
Normal file
62
docs/size-exemptions.md
Normal file
@ -0,0 +1,62 @@
|
||||
# Size-exemption register
|
||||
|
||||
The working norm (constitution Principle II; enforced by the `max-lines` lint
|
||||
gate) is **no source file over 1,500 lines**. A few files are allowed to exceed
|
||||
it because splitting them would do more harm than good — hot per-frame
|
||||
renderers, C++, offline generators, cohesive registries. This register is the
|
||||
list of those exceptions: each row is a **deliberate, signed** decision with a
|
||||
ceiling, a rationale, and a review trigger. Without it, "no file over 1,500
|
||||
without a *signed* exemption" is unenforceable.
|
||||
|
||||
**Rules**
|
||||
- One row per file: a ceiling, a rationale, a signer, a review trigger.
|
||||
- The `max-lines` per-file ceilings in `eslint.config.js` mirror this table —
|
||||
keep them in sync (this register is canonical).
|
||||
- Files with a scheduled split **plan** are *not* exempt — they live in
|
||||
"Planned, not exempt" at the bottom so nothing falls between the two states.
|
||||
- **Signers** (decided 2026-07-08): **Byron** signs core + bundled rows;
|
||||
**Christian** signs the authored-plugin row (virtuoso, its own repo/track).
|
||||
|
||||
## Permanent exemptions (structural rationale)
|
||||
|
||||
| Repo / file | Lines (7-07) | Ceiling | Rationale | Signer | Review |
|
||||
|---|---|---|---|---|---|
|
||||
| core `static/highway.js` → residual `renderer-2d.js` (post-split) | ~2,400–2,900 est. | **3,000** | 60 fps hot path; no module boundary inside the per-frame loop | Byron | after the highway.js split |
|
||||
| core `plugins/highway_3d/` → residual renderer | sized at split; likely **>3,000** | set at split, flagged now | same hot-path rule; the draw core can't be cut without behavior risk | Byron | after the highway_3d split |
|
||||
| core `static/capabilities.js` | 1,538 | 1,600 | cohesive registry + `window.feedBack` bus, 38 lines over; a split spends credibility for nothing | Byron | R4 |
|
||||
| tutorials `builtin/reading-the-highway/generate.py` | 1,818 | 2,000 | offline content generator, never imported at runtime, deps not in runtime requirements | Byron | if a 3rd builtin pack appears |
|
||||
| desktop `src/audio/NodeAddon.cpp` | 3,542 | as-is | C++, outside the ESM/routes playbooks; under active use-after-free crash work — do not churn | Byron | after crash-class work settles |
|
||||
| desktop `src/audio/AudioEngine.cpp` | 2,977 | as-is | same | Byron | same |
|
||||
| desktop `src/vst-host/main.cpp` | 1,928 | as-is | same | Byron | same |
|
||||
| virtuoso `screen.js` (authored, own track) | 25,741 | as-is until its own split | authored plugin on a separate roadmap; migrates on its own schedule | Christian | virtuoso split kickoff |
|
||||
|
||||
## Split-when-touched (no scheduled train; row retires when split)
|
||||
|
||||
| Repo / file | Lines | Ceiling | Rationale | Signer | Review |
|
||||
|---|---|---|---|---|---|
|
||||
| core `lib/gp2rs_gpx.py` | 2,540 | as-is | import converter, off the serve-path hot loop | Byron | when next touched |
|
||||
| core `lib/gp2rs.py` | 2,055 | as-is | same | Byron | when next touched |
|
||||
| core `lib/song.py` | 1,689 | as-is | data models + wire format; cohesive | Byron | when next touched |
|
||||
| core `lib/gp_autosync.py` | 1,572 | as-is | under active dev (#787/#791) — don't collide | Byron | after in-flight work lands |
|
||||
| core `plugins/capability_inspector/screen.js` | 1,752 | as-is | bundled diagnostics plugin, low churn | Byron | when next touched |
|
||||
| core `plugins/folder_library/screen.js` | 1,672 | as-is | bundled plugin, low churn | Byron | when next touched |
|
||||
|
||||
## Temporary rows (cleared by a scheduled PR)
|
||||
|
||||
| Repo / file | Lines | Cleared by |
|
||||
|---|---|---|
|
||||
| core `plugins/__init__.py` | ~2,470 (grew under R0) | the `plugins/_routes.py` + `plugins/_registry.py` split (rides the server.py router work) |
|
||||
|
||||
## Watch list (under the norm — no row needed, re-census each phase)
|
||||
|
||||
`musicxml-import/mxml2notation.py` (1,456) · core `static/capabilities/audio-effects.js`
|
||||
(1,436) · `studio routes.py` (1,399) · `update-manager screen.js` (1,492 — zero headroom).
|
||||
|
||||
## Planned, NOT exempt (owned by split plans — listed so nothing falls between states)
|
||||
|
||||
core `static/app.js` (11,821) · `static/highway.js` (4,154, whole file) · `server.py`
|
||||
(13,948) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
|
||||
(2,974) · `plugins/highway_3d/screen.js` (15,656) · `plugins/keys_highway_3d/screen.js`
|
||||
(3,780) · `plugins/drum_highway_3d/screen.js` (3,597) — and every monolith with a PR
|
||||
train in the refactor plan. Test files (e.g. `tests/test_plugins.py`) are out of scope
|
||||
by policy — the norm governs source files.
|
||||
66
eslint.config.js
Normal file
66
eslint.config.js
Normal file
@ -0,0 +1,66 @@
|
||||
// Flat ESLint config — MAINTAINER / CI ONLY. Never runs on the serve or Docker
|
||||
// path (constitution Principle I: dev-only tooling is exempt, same category as
|
||||
// scripts/build-tailwind.sh). It enforces the module-migration guardrails:
|
||||
//
|
||||
// * max-lines — the 1,500-line size norm, as a WARNING ratchet. Legacy
|
||||
// monoliths warn (the "this is over the norm, split it" signal) and shrink
|
||||
// as the refactor lands; warnings do not fail CI. Genuinely-large files are
|
||||
// exempted below, mirroring the signed register in docs/size-exemptions.md.
|
||||
// * import-x/no-unresolved + no-cycle — module hygiene, scoped to the real
|
||||
// ES-module graphs the refactor produces (a plugin's src/ tree, .mjs
|
||||
// tests). no-unresolved (a HARD error) catches broken import paths;
|
||||
// no-cycle enforces the downward-only layering rule. Core's classic scripts
|
||||
// have no import graph, so both are dormant today and become live gates the
|
||||
// moment module code appears — validated against the first real module
|
||||
// plugin (R1 pilot).
|
||||
|
||||
const importX = require('eslint-plugin-import-x');
|
||||
|
||||
// Per-file size ceilings — a mirror of docs/size-exemptions.md (canonical).
|
||||
// Keep in sync; each entry corresponds to a signed row in the register.
|
||||
const SIZE_EXEMPTIONS = [
|
||||
{ files: ['**/static/capabilities.js'], max: 1600 },
|
||||
{ files: ['**/plugins/capability_inspector/screen.js'], max: 100000 },
|
||||
{ files: ['**/plugins/folder_library/screen.js'], max: 100000 },
|
||||
];
|
||||
|
||||
const sizeRule = (max) => ['warn', { max, skipBlankLines: false, skipComments: false }];
|
||||
|
||||
module.exports = [
|
||||
{
|
||||
ignores: [
|
||||
'node_modules/**',
|
||||
'static/vendor/**',
|
||||
'plugins/**/assets/vendor/**',
|
||||
'**/*.min.js',
|
||||
'static/tailwind.min.css',
|
||||
],
|
||||
},
|
||||
// Size norm across all first-party JS. Classic scripts are parsed as
|
||||
// scripts (no import/export); module files get their own block below.
|
||||
{
|
||||
files: ['**/*.js', '**/*.cjs'],
|
||||
languageOptions: { ecmaVersion: 'latest', sourceType: 'script' },
|
||||
rules: { 'max-lines': sizeRule(1500) },
|
||||
},
|
||||
// ES-module graphs (a plugin's src/ tree, .mjs tests): module parsing + the
|
||||
// acyclic-imports hard gate + the size norm. A migrated bundled plugin's
|
||||
// entry `import './src/main.js'` screen.js must parse as a module — add its
|
||||
// glob here in that plugin's migration PR (classic screen.js stays a script).
|
||||
{
|
||||
files: ['**/src/**/*.js', '**/*.mjs'],
|
||||
languageOptions: { ecmaVersion: 'latest', sourceType: 'module' },
|
||||
plugins: { 'import-x': importX },
|
||||
// v4 flat-config resolver (resolver-next + createNodeResolver). Without
|
||||
// it the import rules silently skip imports they can't resolve.
|
||||
settings: { 'import-x/resolver-next': [importX.createNodeResolver()] },
|
||||
rules: {
|
||||
'max-lines': sizeRule(1500),
|
||||
'import-x/no-unresolved': 'error',
|
||||
'import-x/no-cycle': 'error',
|
||||
},
|
||||
},
|
||||
// Signed size exemptions (docs/size-exemptions.md) — raise the ceiling so
|
||||
// registered files don't warn below it.
|
||||
...SIZE_EXEMPTIONS.map(({ files, max }) => ({ files, rules: { 'max-lines': sizeRule(max) } })),
|
||||
];
|
||||
1759
package-lock.json
generated
1759
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@ -8,9 +8,12 @@
|
||||
"test:headed": "playwright test --headed",
|
||||
"test:debug": "playwright test --debug",
|
||||
"test:js": "node --test tests/js/*.test.js 'tests/plugins/*/js/*.test.js'",
|
||||
"install:playwright": "playwright install chromium"
|
||||
"install:playwright": "playwright install chromium",
|
||||
"lint": "eslint ."
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "^1.59.1"
|
||||
"@playwright/test": "^1.59.1",
|
||||
"eslint": "^9.39.4",
|
||||
"eslint-plugin-import-x": "^4.17.1"
|
||||
}
|
||||
}
|
||||
|
||||
@ -18,6 +18,54 @@ from safepath import safe_join
|
||||
log = logging.getLogger("feedBack.plugins")
|
||||
|
||||
|
||||
def _plugin_media_type(path: Path) -> str:
|
||||
"""Best-effort Content-Type for a served plugin file. `.js`/`.css` must come
|
||||
back as JavaScript/CSS so `<script type=module>` / `addModule()` / a `<link>`
|
||||
accept them; `mimetypes.guess_type` can miss these on a stripped platform
|
||||
registry, so fall back explicitly (mirrors the assets/ route)."""
|
||||
media_type = mimetypes.guess_type(path.name)[0]
|
||||
if media_type is None and path.suffix == ".js":
|
||||
return "application/javascript"
|
||||
if media_type is None and path.suffix == ".css":
|
||||
return "text/css"
|
||||
return media_type or "application/octet-stream"
|
||||
|
||||
|
||||
def _plugin_file_etag(path: Path) -> str | None:
|
||||
"""Weak ETag from mtime+size — cheap, stable across reads, changes on edit.
|
||||
This is what makes the live-edit loop work for module graphs: a conditional
|
||||
GET revalidates and 304s unchanged files on refresh instead of re-downloading
|
||||
the whole `src/` tree. Returns None if the file can't be stat'd."""
|
||||
try:
|
||||
st = path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
return f'W/"{st.st_mtime_ns:x}-{st.st_size:x}"'
|
||||
|
||||
|
||||
def _if_none_match(request: Request, etag: str) -> bool:
|
||||
"""True when the client's If-None-Match already holds `etag`."""
|
||||
# ponytail: we serve one weak ETag; the browser echoes it back verbatim, so
|
||||
# a direct compare is enough (comma-split tolerates a proxy concatenation).
|
||||
return etag in [t.strip() for t in request.headers.get("if-none-match", "").split(",")]
|
||||
|
||||
|
||||
def _plugin_file_response(request: Request, path: Path, media_type: str) -> Response:
|
||||
"""Serve a plugin source/asset file with the live-edit cache contract:
|
||||
`Cache-Control: no-cache` (browser may store but MUST revalidate) + a weak
|
||||
ETag, and a bodyless 304 when the client's If-None-Match already matches.
|
||||
Starlette's `FileResponse` emits an ETag but never evaluates If-None-Match
|
||||
itself, so the conditional handling has to live here."""
|
||||
headers = {"Cache-Control": "no-cache"}
|
||||
etag = _plugin_file_etag(path)
|
||||
if etag:
|
||||
headers["ETag"] = etag
|
||||
if _if_none_match(request, etag):
|
||||
return Response(status_code=304, headers=headers)
|
||||
# FileResponse sets etag/last-modified via setdefault, so the ETag above wins.
|
||||
return FileResponse(path, media_type=media_type, headers=headers)
|
||||
|
||||
|
||||
PLUGINS_DIR = Path(__file__).parent
|
||||
# Holds only *ready* (loaded) plugins — those whose dependencies installed
|
||||
# and whose routes registered. A plugin GRADUATES from PENDING_PLUGINS into
|
||||
@ -1373,6 +1421,12 @@ def load_plugins(app: FastAPI, context: dict, progress_cb=None, route_setup_fn=N
|
||||
"version": manifest.get("version"),
|
||||
"has_screen": bool(manifest.get("screen")),
|
||||
"has_script": bool(manifest.get("script")),
|
||||
# Module-migration (R0): `scriptType:"module"` tells the loader to
|
||||
# inject screen.js as <script type="module">; `minHost` is the
|
||||
# min core version a migrated plugin needs (passthrough only in R0 —
|
||||
# enforcement is deferred to R4, master §4b). None when unset.
|
||||
"script_type": manifest.get("scriptType"),
|
||||
"min_host": manifest.get("minHost"),
|
||||
"has_settings": bool(manifest.get("settings")),
|
||||
"settings_category": _settings_category,
|
||||
# Drives the v3 shell's immersive (full-screen) mode for this
|
||||
@ -2089,6 +2143,11 @@ def register_plugin_api(app: FastAPI):
|
||||
"fallback": p.get("fallback", False),
|
||||
"has_screen": p["has_screen"],
|
||||
"has_script": p["has_script"],
|
||||
# Module-migration passthrough (R0). Re-read from the manifest
|
||||
# like `version` above so stubbed test entries (built without
|
||||
# _nav_entry) don't need the key.
|
||||
"script_type": (p.get("_manifest") or {}).get("scriptType"),
|
||||
"min_host": (p.get("_manifest") or {}).get("minHost"),
|
||||
"has_settings": p["has_settings"],
|
||||
# v3 immersive screen opt-in (full-screen plugin UI).
|
||||
"fullscreen": p.get("fullscreen", False),
|
||||
@ -2142,6 +2201,9 @@ def register_plugin_api(app: FastAPI):
|
||||
"fallback": False,
|
||||
"has_screen": e.get("has_screen", False),
|
||||
"has_script": e.get("has_script", False),
|
||||
# Pending entries come from _nav_entry, so they carry these.
|
||||
"script_type": e.get("script_type"),
|
||||
"min_host": e.get("min_host"),
|
||||
"has_settings": e.get("has_settings", False),
|
||||
"settings_category": e.get("settings_category"),
|
||||
"fullscreen": e.get("fullscreen", False),
|
||||
@ -2307,7 +2369,7 @@ def register_plugin_api(app: FastAPI):
|
||||
return HTMLResponse("", status_code=404)
|
||||
|
||||
@app.get("/api/plugins/{plugin_id}/screen.js")
|
||||
def plugin_screen_js(plugin_id: str):
|
||||
def plugin_screen_js(request: Request, plugin_id: str):
|
||||
with PLUGINS_LOCK:
|
||||
snapshot = list(LOADED_PLUGINS)
|
||||
for p in snapshot:
|
||||
@ -2315,8 +2377,11 @@ def register_plugin_api(app: FastAPI):
|
||||
if p.get("status", "ready") != "ready":
|
||||
break
|
||||
script_file = p["_dir"] / p["_manifest"].get("script", "screen.js")
|
||||
if script_file.exists():
|
||||
return Response(script_file.read_text(encoding="utf-8"), media_type="application/javascript")
|
||||
if script_file.is_file():
|
||||
# no-cache + ETag/304 so an edited screen.js reloads on
|
||||
# refresh while an unchanged one revalidates cheaply — the
|
||||
# same live-edit contract the src/ module graph relies on.
|
||||
return _plugin_file_response(request, script_file, "application/javascript")
|
||||
return Response("", status_code=404)
|
||||
|
||||
@app.get("/api/plugins/{plugin_id}/settings.html")
|
||||
@ -2377,7 +2442,7 @@ def register_plugin_api(app: FastAPI):
|
||||
return Response("{}", status_code=404, media_type="application/json")
|
||||
|
||||
@app.get("/api/plugins/{plugin_id}/assets/{asset_path:path}")
|
||||
def plugin_asset(plugin_id: str, asset_path: str):
|
||||
def plugin_asset(request: Request, plugin_id: str, asset_path: str):
|
||||
"""Serve a static file a plugin bundles under its own ``assets/``
|
||||
directory (e.g. an AudioWorklet module, WASM, or image). Unlike the
|
||||
fixed screen.js/settings.html handlers above, this is a generic
|
||||
@ -2399,16 +2464,35 @@ def register_plugin_api(app: FastAPI):
|
||||
log.warning("Plugin %r: asset path rejected: %r", plugin_id, asset_path)
|
||||
break
|
||||
if target.is_file():
|
||||
media_type = mimetypes.guess_type(target.name)[0]
|
||||
# .js must come back as JavaScript so addModule() / <script>
|
||||
# accept it; guess_type can miss this on some platforms.
|
||||
if media_type is None and target.suffix == ".js":
|
||||
media_type = "application/javascript"
|
||||
# .css must come back as text/css so a <link rel=stylesheet>
|
||||
# (the styles capability) is honoured; guess_type can miss it
|
||||
# on a stripped platform mimetypes registry, same as .js.
|
||||
elif media_type is None and target.suffix == ".css":
|
||||
media_type = "text/css"
|
||||
return FileResponse(target, media_type=media_type or "application/octet-stream")
|
||||
# no-cache + ETag/304 so a live-edited worklet/asset reloads
|
||||
# on refresh (bare FileResponse emits an ETag but never 304s).
|
||||
return _plugin_file_response(request, target, _plugin_media_type(target))
|
||||
break
|
||||
return Response("", status_code=404)
|
||||
|
||||
@app.get("/api/plugins/{plugin_id}/src/{src_path:path}")
|
||||
def plugin_src(request: Request, plugin_id: str, src_path: str):
|
||||
"""Serve a file from a plugin's ES-module source tree under ``src/``.
|
||||
|
||||
This is the R0 host capability that lets a migrated plugin's
|
||||
``screen.js`` (a one-line ``import './src/main.js'``) load its whole
|
||||
module graph. Containment mirrors the assets/ route exactly —
|
||||
``safe_join`` against ``<plugin>/src`` rejects ``..``, absolute paths,
|
||||
and NUL bytes — and the live-edit cache contract (no-cache + ETag/304)
|
||||
makes an edited module reload on refresh while unchanged ones 304.
|
||||
Read-only; the src/ tree is source files, never executed server-side.
|
||||
"""
|
||||
with PLUGINS_LOCK:
|
||||
snapshot = list(LOADED_PLUGINS)
|
||||
for p in snapshot:
|
||||
if p["id"] == plugin_id:
|
||||
if p.get("status", "ready") != "ready":
|
||||
break
|
||||
target = safe_join(p["_dir"] / "src", src_path)
|
||||
if target is None:
|
||||
log.warning("Plugin %r: src path rejected: %r", plugin_id, src_path)
|
||||
break
|
||||
if target.is_file():
|
||||
return _plugin_file_response(request, target, _plugin_media_type(target))
|
||||
break
|
||||
return Response("", status_code=404)
|
||||
|
||||
94
scripts/perf-baseline.mjs
Normal file
94
scripts/perf-baseline.mjs
Normal file
@ -0,0 +1,94 @@
|
||||
// Perf-baseline harness for the module-migration refactor (R0).
|
||||
//
|
||||
// Rerun this after every phase (R0 → R3c) to prove the split does not regress
|
||||
// screen-entry, frame-time, memory, or server latency. It writes a markdown
|
||||
// results block to stdout; paste it into docs/perf-baseline.md (or redirect).
|
||||
//
|
||||
// Usage:
|
||||
// node scripts/perf-baseline.mjs --base http://127.0.0.1:8000 [--n 60] [--soak 30]
|
||||
//
|
||||
// Maintainer/CI-only dev tooling (uses the committed @playwright/test browser);
|
||||
// never part of the serve or Docker path. Metrics that need a seeded library
|
||||
// with charts (playback frame-time, screen-entry into a live highway) are
|
||||
// clearly labelled — run those against an environment with real songs.
|
||||
|
||||
import { createRequire } from 'module';
|
||||
const require = createRequire(import.meta.url);
|
||||
const { chromium } = require('@playwright/test');
|
||||
|
||||
const args = new Map();
|
||||
for (let i = 2; i < process.argv.length; i += 2) args.set(process.argv[i].replace(/^--/, ''), process.argv[i + 1]);
|
||||
const BASE = args.get('base') || 'http://127.0.0.1:8000';
|
||||
const N = parseInt(args.get('n') || '60', 10);
|
||||
const SOAK_S = parseInt(args.get('soak') || '30', 10);
|
||||
|
||||
const pct = (xs, p) => {
|
||||
if (!xs.length) return null;
|
||||
const s = [...xs].sort((a, b) => a - b);
|
||||
return s[Math.min(s.length - 1, Math.floor((p / 100) * s.length))];
|
||||
};
|
||||
const ms = (x) => (x == null ? '—' : `${x.toFixed(1)}`);
|
||||
|
||||
// ── Server latency: p50/p95/p99 over N requests per endpoint ──────────────────
|
||||
async function serverLatency(paths) {
|
||||
const rows = [];
|
||||
for (const path of paths) {
|
||||
const t = [];
|
||||
let status = 0;
|
||||
for (let i = 0; i < N; i++) {
|
||||
const t0 = performance.now();
|
||||
try {
|
||||
const r = await fetch(BASE + path);
|
||||
status = r.status;
|
||||
await r.arrayBuffer();
|
||||
} catch { status = -1; }
|
||||
t.push(performance.now() - t0);
|
||||
}
|
||||
rows.push({ path, status, p50: pct(t, 50), p95: pct(t, 95), p99: pct(t, 99) });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
// ── Client: cold boot-to-interactive + idle memory after a soak ───────────────
|
||||
async function clientMetrics() {
|
||||
const browser = await chromium.launch();
|
||||
const page = await browser.newPage();
|
||||
const t0 = Date.now();
|
||||
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 60000 });
|
||||
const bootMs = Date.now() - t0;
|
||||
|
||||
// performance.memory is Chromium-only; JS heap after settle.
|
||||
const mem0 = await page.evaluate(() => (performance.memory ? performance.memory.usedJSHeapSize : null));
|
||||
await page.waitForTimeout(SOAK_S * 1000);
|
||||
const mem1 = await page.evaluate(() => (performance.memory ? performance.memory.usedJSHeapSize : null));
|
||||
|
||||
const scripts = await page.evaluate(() =>
|
||||
document.querySelectorAll('script[data-plugin-id]').length);
|
||||
|
||||
await browser.close();
|
||||
return { bootMs, memStartMB: mem0 && mem0 / 1048576, memSoakMB: mem1 && mem1 / 1048576, scripts };
|
||||
}
|
||||
|
||||
const server = await serverLatency([
|
||||
'/api/version',
|
||||
'/api/plugins',
|
||||
'/api/library?limit=60',
|
||||
'/api/library/artists',
|
||||
]);
|
||||
const client = await clientMetrics();
|
||||
|
||||
const now = new Date().toISOString();
|
||||
let out = `\n<!-- generated by scripts/perf-baseline.mjs @ ${now} against ${BASE} (n=${N}, soak=${SOAK_S}s) -->\n\n`;
|
||||
out += `### Server latency (ms)\n\n| Endpoint | status | p50 | p95 | p99 |\n|---|---|---|---|---|\n`;
|
||||
for (const r of server) out += `| \`${r.path}\` | ${r.status} | ${ms(r.p50)} | ${ms(r.p95)} | ${ms(r.p99)} |\n`;
|
||||
out += `\n### Client\n\n| Metric | Value |\n|---|---|\n`;
|
||||
out += `| Cold boot → networkidle | ${client.bootMs} ms |\n`;
|
||||
out += `| JS heap after load | ${client.memStartMB ? client.memStartMB.toFixed(1) + ' MB' : '—'} |\n`;
|
||||
out += `| JS heap after ${SOAK_S}s idle soak | ${client.memSoakMB ? client.memSoakMB.toFixed(1) + ' MB' : '—'} |\n`;
|
||||
out += `| Plugin scripts injected | ${client.scripts} |\n`;
|
||||
out += `\n> **Requires a seeded library** (not captured by this run): playback frame-time p95\n`;
|
||||
out += `> on the 2D + 3D highway, and screen-entry (plugin inject → interactive) for\n`;
|
||||
out += `> editor / notedetect / highway_3d with a real chart loaded. Run this harness\n`;
|
||||
out += `> against an environment with charts in \`DLC_DIR\` to fill those in.\n`;
|
||||
|
||||
console.log(out);
|
||||
@ -11567,6 +11567,14 @@ async function loadPlugins() {
|
||||
// URL ?v=mtime convention elsewhere in this file).
|
||||
const v = encodeURIComponent(wantedVersion);
|
||||
script.src = `/api/plugins/${plugin.id}/screen.js${v ? `?v=${v}` : ''}`;
|
||||
// Module-migration (R0): a migrated plugin declares
|
||||
// scriptType:"module" and its screen.js is `import
|
||||
// './src/main.js'`. A <script type="module"> fires load
|
||||
// only after its whole static-import graph evaluates, so
|
||||
// the await-onload completion + _loadingPluginId contract
|
||||
// below is preserved (a classic-IIFE dynamic import()
|
||||
// would not). Classic plugins are unaffected.
|
||||
if (plugin.script_type === 'module') script.type = 'module';
|
||||
script.dataset.pluginId = plugin.id;
|
||||
script.dataset.pluginVersion = wantedVersion;
|
||||
window.feedBack._loadingPluginId = plugin.id;
|
||||
|
||||
59
tests/js/plugin_loader_script_type.test.js
Normal file
59
tests/js/plugin_loader_script_type.test.js
Normal file
@ -0,0 +1,59 @@
|
||||
// Guards the R0 module-migration loader change in static/app.js: a migrated
|
||||
// plugin (manifest scriptType:"module", surfaced as plugin.script_type) must be
|
||||
// injected as <script type="module"> so its screen.js `import './src/main.js'`
|
||||
// graph loads, while classic plugins stay untouched.
|
||||
//
|
||||
// The injection is a single line inside the large async loadPlugins() closure
|
||||
// (it depends on loadedScripts, _removePluginScriptTags, and the
|
||||
// _loadingPluginId completion window), so a faithful behavioural harness would
|
||||
// need to stub the whole loader. Instead this asserts the *structural*
|
||||
// contract in source — the guard exists, is gated (not unconditional), and sits
|
||||
// inside the screen.js injection block before appendChild. The behavioural proof
|
||||
// is the R0 end-to-end live-edit check (a real module plugin booting in-browser).
|
||||
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const APP_JS = path.join(__dirname, '..', '..', 'static', 'app.js');
|
||||
const src = fs.readFileSync(APP_JS, 'utf8');
|
||||
|
||||
// Isolate the screen.js <script> injection block: from where its src is built
|
||||
// to where the element is appended.
|
||||
function injectionBlock() {
|
||||
const start = src.indexOf('/api/plugins/${plugin.id}/screen.js');
|
||||
assert.ok(start !== -1, 'screen.js injection src not found — loader moved?');
|
||||
const end = src.indexOf('document.body.appendChild(script)', start);
|
||||
assert.ok(end !== -1, 'appendChild(script) not found after screen.js src');
|
||||
return src.slice(start, end);
|
||||
}
|
||||
|
||||
test('module plugins are injected as <script type="module">', () => {
|
||||
const block = injectionBlock();
|
||||
assert.match(
|
||||
block,
|
||||
/if\s*\(\s*plugin\.script_type\s*===\s*['"]module['"]\s*\)\s*script\.type\s*=\s*['"]module['"]\s*;/,
|
||||
'expected a guarded `script.type = "module"` keyed on plugin.script_type === "module"',
|
||||
);
|
||||
});
|
||||
|
||||
test('the module type is gated, never set unconditionally', () => {
|
||||
const block = injectionBlock();
|
||||
// Every assignment of script.type in the block must be on the same line as
|
||||
// the plugin.script_type guard (i.e. no bare `script.type = 'module'`).
|
||||
for (const line of block.split('\n')) {
|
||||
if (/script\.type\s*=/.test(line)) {
|
||||
assert.match(line, /plugin\.script_type\s*===\s*['"]module['"]/,
|
||||
`unguarded script.type assignment: ${line.trim()}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('the module guard sits before appendChild, after the src assignment', () => {
|
||||
const guardAt = src.indexOf('script.type = \'module\'');
|
||||
const srcAt = src.indexOf('/api/plugins/${plugin.id}/screen.js');
|
||||
const appendAt = src.indexOf('document.body.appendChild(script)', srcAt);
|
||||
assert.ok(guardAt > srcAt && guardAt < appendAt,
|
||||
'the module guard must live inside the screen.js injection block');
|
||||
});
|
||||
140
tests/test_plugin_src_route.py
Normal file
140
tests/test_plugin_src_route.py
Normal file
@ -0,0 +1,140 @@
|
||||
"""Tests for the plugin `src/` module-serving route and the live-edit cache
|
||||
contract added in R0 (module-migration rails).
|
||||
|
||||
Covers:
|
||||
* GET /api/plugins/{id}/src/{path} serves a plugin's ES-module source tree
|
||||
with the right Content-Type, including nested paths.
|
||||
* Path containment: `..`, absolute, and NUL are rejected (404) — the same
|
||||
`safe_join` guard the assets/ route uses.
|
||||
* The live-edit cache contract: no-cache + a weak ETag, a bodyless 304 on
|
||||
matching If-None-Match, and no stale 304 after an in-place edit.
|
||||
* screen.js and assets/ now also emit an ETag and honor If-None-Match
|
||||
(previously screen.js sent no headers and assets/ never returned 304).
|
||||
|
||||
The routes read the module-global `plugins.LOADED_PLUGINS`, so each test
|
||||
registers a fake ready plugin directly (save/restore that global) and drives
|
||||
`register_plugin_api` on a fresh FastAPI app — no full server import needed.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
import plugins
|
||||
|
||||
|
||||
PLUGIN_ID = "srctest"
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(tmp_path):
|
||||
"""A TestClient with `register_plugin_api` wired and a single fake ready
|
||||
plugin whose dir (`tmp_path`) holds a src/ tree, an asset, and a screen.js.
|
||||
Restores LOADED_PLUGINS afterward."""
|
||||
(tmp_path / "src").mkdir()
|
||||
(tmp_path / "src" / "main.js").write_text("import './util/x.js';\nexport const boot = 1;\n")
|
||||
(tmp_path / "src" / "util").mkdir()
|
||||
(tmp_path / "src" / "util" / "x.js").write_text("export const x = 42;\n")
|
||||
(tmp_path / "src" / "theme.css").write_text(".a{color:red}\n")
|
||||
(tmp_path / "assets").mkdir()
|
||||
(tmp_path / "assets" / "worklet.js").write_text("// worklet\n")
|
||||
(tmp_path / "screen.js").write_text("import './src/main.js';\n")
|
||||
|
||||
saved = list(plugins.LOADED_PLUGINS)
|
||||
plugins.LOADED_PLUGINS.clear()
|
||||
plugins.LOADED_PLUGINS.append({
|
||||
"id": PLUGIN_ID,
|
||||
"status": "ready",
|
||||
"_dir": tmp_path,
|
||||
"_manifest": {"script": "screen.js", "scriptType": "module"},
|
||||
})
|
||||
app = FastAPI()
|
||||
plugins.register_plugin_api(app)
|
||||
c = TestClient(app, raise_server_exceptions=True)
|
||||
try:
|
||||
yield c, tmp_path
|
||||
finally:
|
||||
c.close()
|
||||
plugins.LOADED_PLUGINS.clear()
|
||||
plugins.LOADED_PLUGINS.extend(saved)
|
||||
|
||||
|
||||
def test_src_file_served_with_js_media_type(client):
|
||||
c, _ = client
|
||||
r = c.get(f"/api/plugins/{PLUGIN_ID}/src/main.js")
|
||||
assert r.status_code == 200
|
||||
# Either application/javascript or text/javascript is a valid module-script
|
||||
# MIME (guess_type returns text/javascript on newer platforms); browsers
|
||||
# accept both for <script type=module>.
|
||||
assert "javascript" in r.headers["content-type"]
|
||||
assert "export const boot" in r.text
|
||||
assert r.headers["cache-control"] == "no-cache"
|
||||
assert r.headers.get("etag")
|
||||
|
||||
|
||||
def test_src_nested_path_and_css_media_type(client):
|
||||
c, _ = client
|
||||
assert c.get(f"/api/plugins/{PLUGIN_ID}/src/util/x.js").status_code == 200
|
||||
r = c.get(f"/api/plugins/{PLUGIN_ID}/src/theme.css")
|
||||
assert r.status_code == 200
|
||||
assert r.headers["content-type"].startswith("text/css")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", [
|
||||
"..%2f..%2fplugin.json", # escape the src/ dir
|
||||
"..%2f..%2f..%2fetc%2fpasswd",
|
||||
"%2fetc%2fpasswd", # absolute
|
||||
"util%2f..%2f..%2fscreen.js",
|
||||
])
|
||||
def test_src_traversal_rejected(client, bad):
|
||||
c, _ = client
|
||||
assert c.get(f"/api/plugins/{PLUGIN_ID}/src/{bad}").status_code == 404
|
||||
|
||||
|
||||
def test_src_missing_is_404(client):
|
||||
c, _ = client
|
||||
assert c.get(f"/api/plugins/{PLUGIN_ID}/src/nope.js").status_code == 404
|
||||
|
||||
|
||||
def test_src_conditional_304(client):
|
||||
c, _ = client
|
||||
r1 = c.get(f"/api/plugins/{PLUGIN_ID}/src/main.js")
|
||||
etag = r1.headers["etag"]
|
||||
r2 = c.get(f"/api/plugins/{PLUGIN_ID}/src/main.js", headers={"If-None-Match": etag})
|
||||
assert r2.status_code == 304
|
||||
assert r2.content == b""
|
||||
|
||||
|
||||
def test_src_no_stale_304_after_edit(client):
|
||||
c, root = client
|
||||
etag = c.get(f"/api/plugins/{PLUGIN_ID}/src/main.js").headers["etag"]
|
||||
(root / "src" / "main.js").write_text("export const boot = 2; // edited, longer body\n")
|
||||
r = c.get(f"/api/plugins/{PLUGIN_ID}/src/main.js", headers={"If-None-Match": etag})
|
||||
assert r.status_code == 200
|
||||
assert "boot = 2" in r.text
|
||||
assert r.headers["etag"] != etag
|
||||
|
||||
|
||||
def test_screen_js_now_conditional(client):
|
||||
c, _ = client
|
||||
r1 = c.get(f"/api/plugins/{PLUGIN_ID}/screen.js")
|
||||
assert r1.status_code == 200
|
||||
assert r1.headers["cache-control"] == "no-cache"
|
||||
etag = r1.headers["etag"]
|
||||
r2 = c.get(f"/api/plugins/{PLUGIN_ID}/screen.js", headers={"If-None-Match": etag})
|
||||
assert r2.status_code == 304
|
||||
|
||||
|
||||
def test_asset_now_conditional(client):
|
||||
c, _ = client
|
||||
r1 = c.get(f"/api/plugins/{PLUGIN_ID}/assets/worklet.js")
|
||||
assert r1.status_code == 200
|
||||
etag = r1.headers["etag"]
|
||||
r2 = c.get(f"/api/plugins/{PLUGIN_ID}/assets/worklet.js", headers={"If-None-Match": etag})
|
||||
assert r2.status_code == 304
|
||||
|
||||
|
||||
def test_unready_plugin_src_is_404(client):
|
||||
c, _ = client
|
||||
plugins.LOADED_PLUGINS[0]["status"] = "installing"
|
||||
assert c.get(f"/api/plugins/{PLUGIN_ID}/src/main.js").status_code == 404
|
||||
Loading…
Reference in New Issue
Block a user