got-feedBack/feedBack
1
0
Fork 0
mirror of https://github.com/got-feedBack/feedBack.git synced 2026-09-13 01:28:31 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

docs: extract plugin contracts into modular docs and slim CLAUDE.md

Browse Source 
CLAUDE.md had grown to 545 lines / 50 KB — most of it plugin-author
content that other AI tools (Cursor, Copilot, Codex, Aider) and humans
without AI never reach. Extract the plugin surface into 10 focused
docs and a JSON Schema for plugin.json, then slim CLAUDE.md to a
156-line navigable index.

New docs (~999 lines total, all self-contained):

  docs/PLUGIN_AUTHORING.md            — entry point and quickstart
  docs/plugin-manifest.md             — plugin.json field reference
  docs/plugin-visualization-contracts.md — setRenderer / overlay / note-state
  docs/plugin-audio-mixer.md          — fader registration
  docs/plugin-logging.md              — context["log"] + env vars
  docs/plugin-diagnostics.md          — server_files / callable
  docs/plugin-keyboard-shortcuts.md   — registerShortcut + scopes
  docs/plugin-sibling-imports.md      — load_sibling pattern
  docs/websocket-protocol.md          — /ws/highway message reference
  docs/testing-plugins.md             — pytest fixtures + Playwright

  schema/plugin.schema.json           — Draft 2020-12 schema for
                                        plugin.json; license enum
                                        mirrors CONTRIBUTING's curated
                                        allowlist. Backs CI validation
                                        and the plugin-validate skill.

CLAUDE.md slim (581 lines changed, -485):

  - Removed ~300 lines of plugin-author prose (now in docs/).
  - Kept architecture quick reference, running the app, testing,
    git workflow, versioning, song formats, frontend/backend
    conventions, plugin authoring INDEX (table → docs/), first-hour
    pitfalls, "For AI agents" footer.
  - Anchor stubs preserved next to the new index entries so deep
    links from specs/001-slopsmith-platform/analyze.md still resolve.

Verification:
  python -c "import json,glob,jsonschema; s=json.load(open('schema/plugin.schema.json')); [jsonschema.validate(json.load(open(p)), s) for p in sorted(glob.glob('plugins/*/plugin.json'))]"
  # ok — validates highway_3d, app_tour_library, app_tour_settings
Signed-off-by: Miguel_LZPF <mgcdreamer@gmail.com>
This commit is contained in:
Miguel_LZPF
2026-06-18 00:38:50 -07:00
committed by Bret Mogilefsky
parent cad78857fb
commit 1214a6c0a0
12 changed files with 1013 additions and 625 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+14 -625
View File
@@ -1,633 +1,22 @@
# Slopsmith — AI Agent Guide # CLAUDE.md
Slopsmith is a self-hosted web app for browsing, playing, and practicing interactive music notation, built around its own open `.sloppak` chart format. Charts come from importing Guitar Pro (GP5/GP8) or MusicXML, or from authoring in the built-in editor. It runs as a Docker container with a FastAPI backend (`server.py`), vanilla JavaScript frontend (`static/`), shared Python libraries (`lib/`), and an extensive plugin system (`plugins/`). There are no frontend frameworks — everything is plain JS, HTML, and Tailwind CSS. Claude Code memory file. This repo's canonical project orientation lives in [`AGENTS.md`](AGENTS.md) - the cross-tool standard read by Cursor, Copilot, Codex, Aider, Cline, and others. The line below uses Claude Code's `@`-import to inline AGENTS.md into this memory file, so there's a single source of truth and no drift between two near-duplicate files.
## Architecture Quick Reference @AGENTS.md
``` ## Claude-specific surfaces
server.py FastAPI app — library API, WebSocket highway, plugin loading
static/
app.js Main frontend — screens, library views, player, plugin loader
highway.js Canvas note highway renderer (createHighway factory)
index.html Single-page app shell
style.css Custom CSS loaded alongside Tailwind
lib/
song.py Core data models (Note, Chord, Arrangement, Song)
sloppak.py Sloppak format support
loosefolder.py Loose-folder XML chart support
audio.py OGG/MP3 audio handling
retune.py Pitch-shifting logic
tunings.py Tuning name/offset utilities
gp2rs.py Guitar Pro to arrangement XML conversion
gp2midi.py Guitar Pro to MIDI
plugins/
__init__.py Plugin discovery, loading, requirements install
<plugin_name>/ Each plugin is its own directory (often a git submodule)
tests/
test_*.py pytest test suite
```
## Plugin System The rest of this file is content that *only* makes sense for Claude Code (other AI tools have their own incompatible automation mechanisms). Skills, subagent, rule, and settings live under [`.claude/`](.claude/):
Plugins are the primary extension point. Each plugin lives in `plugins/<name>/` with a `plugin.json` manifest. Plugins are typically their own git repositories — see [CONTRIBUTING.md](CONTRIBUTING.md) for the licensing policy (curated plugins should be AGPL-3.0 or AGPL-compatible: MIT, BSD, Apache-2.0). - [`.claude/skills/plugin-scaffold/`](.claude/skills/plugin-scaffold/SKILL.md) - generates a new plugin skeleton (visualization / overlay / settings-only / routes-only).
- [`.claude/skills/plugin-validate/`](.claude/skills/plugin-validate/SKILL.md) - validates `plugin.json` against `schema/plugin.schema.json` locally before push.
- [`.claude/skills/speckit-*/`](.claude/skills/) - spec-kit skills (auto-generated from `.specify/`; don't edit manually).
- [`.claude/rules/plugin-author.md`](.claude/rules/plugin-author.md) - glob-scoped to `plugins/**`; encodes the contracts from `docs/PLUGIN_AUTHORING.md` so suggestions don't drift from them.
- [`.claude/agents/slopsmith-reviewer.md`](.claude/agents/slopsmith-reviewer.md) - plugin-aware code-review subagent. Invoke with `@slopsmith-reviewer`.
- [`.claude/settings.json`](.claude/settings.json) - repo defaults (no hooks enabled by default; commented opt-in example for `plugin.json` validation on save).
```json See [`.claude/README.md`](.claude/README.md) for conventions when adding more.
{
"id": "my_plugin",
"name": "My Plugin",
"version": "1.0.0",
"private": false,
"type": "visualization",
"nav": { "label": "My Plugin", "screen": "plugin-my_plugin" },
"screen": "screen.html",
"script": "screen.js",
"styles": "assets/plugin.css",
"routes": "routes.py",
"settings": {
"html": "settings.html",
"server_files": ["my_plugin.db", "my_plugin_models/"]
},
"diagnostics": {
"server_files": ["my_plugin.diag.json"],
"callable": "diagnostics:collect"
}
}
```
All fields except `id` and `name` are optional. Plugins can have any combination of frontend (screen/script), backend (routes), and settings. ## Why this file is short
`version` and `private` are advisory metadata — the plugin loader does not currently consume them, but plugins commonly include them for publishing/tooling purposes. Everything you'd expect to find here - architecture, running the app, testing, conventions, plugin authoring, first-hour pitfalls - is imported above via `@AGENTS.md`. Updates go in `AGENTS.md`. This file only carries Claude-Code-specific automation references.
`description`, `category`, and `icon` are **optional, additive v3 Pedalboard metadata** (surfaced in `/api/plugins`, consumed by the v3 Plugins page `static/v3/plugins-page.js`). `description` is a short one-sentence summary shown under the pedal name. `category` (`audio | creation | practice | game | tools`, free-form; unknown/absent → curated default → `"other"`) picks which pedalboard the plugin sits on. `icon` is an assets-relative thumbnail path (e.g. `"assets/thumb.png"`, ~square ~256×256, same containment rule as `styles`, served via `/api/plugins/<id>/assets/...`); if omitted the loader auto-detects `assets/thumb.png`, and plugins with no thumbnail get a default pedal graphic. All three are backward-compatible — omit them and the plugin still loads. See [docs/plugin-v3-ui.md](docs/plugin-v3-ui.md).
`styles` is the **opt-in** for self-hosted CSS (Principle II — prebuilt Tailwind, no Play CDN). Core's `static/tailwind.min.css` only contains classes scanned from core source at build time, so a plugin installed at runtime (community / NAS) that uses classes core didn't scan — especially arbitrary values like `text-[11px]` — renders unstyled. Declaring `styles` makes the frontend inject one versioned `<link rel="stylesheet">` into `<head>` (covering the plugin's screen *and* its settings panel) pointing at the plugin's own compiled stylesheet. The value is a **plugin-root-relative path that must live under `assets/`** (e.g. `"assets/plugin.css"`) so it serves through the sandboxed `/api/plugins/<id>/assets/...` route. Build it with `corePlugins: { preflight: false }` (utilities only — core ships the single base reset; don't duplicate it) and **never** the Tailwind Play CDN. Plugins that use only core-guaranteed utilities, or ship no Tailwind, omit `styles` and are byte-for-byte unaffected. Full authoring guide + scaffold: [docs/plugin-styles.md](docs/plugin-styles.md).
`settings.server_files` is the **opt-in** for the unified Settings export/import flow (slopsmith#113). It's a list of relpaths under `context["config_dir"]` that the plugin wants included in user-triggered backups. A trailing `/` denotes a directory (recurse). Plugins that omit this field have no server-side files exported; their state lives entirely in browser `localStorage`, which is bundled wholesale on every export. Rules:
- Relpaths only. Absolute paths, drive letters, `..` segments, and backslashes are rejected at load time with a `[Plugin]` warning.
- The same allowlist is consulted at both export and import: a bundle that references a file the *importing host*'s manifest no longer declares is skipped with a warning (handles plugin updates between export and import). A bundle that references a file your host's manifest never declared is also skipped — no surprise writes.
- Files are encoded as `{"encoding": "json", "data": <parsed>}` for `.json` files that parse cleanly (diff-friendly), `{"encoding": "base64", "data": "..."}` otherwise (sqlite, model blobs, IRs).
- Plugins own their internal data migration. Importing a bundle whose data schema predates your current code restores bytes verbatim — your plugin must cope at next load.
- Symlinks are skipped on export and never followed on import.
`diagnostics` is the **opt-in** for the troubleshooting bundle (slopsmith#166 — Settings → Export Diagnostics). Two independent fields:
- `diagnostics.server_files` — same allowlist semantics as `settings.server_files`: relpaths under `context["config_dir"]`, no `..`, no abs paths, no backslashes, no leading dots. Files listed here are copied verbatim into `plugins/<plugin_id>/<relpath>` inside the bundle. Use this for snapshot-style state (small DB excerpts, model lists, last-error files).
- `diagnostics.callable` — `"<module>:<function>"` (e.g. `"diagnostics:collect"`). Resolved lazily via `load_sibling` when the user clicks Export, then called as `func({"plugin_id": "...", "config_dir": Path(...)})`. Return `dict`/`list` → written to `plugins/<id>/callable.json`; `bytes` → `callable.bin`; `str` → `callable.txt`. Exceptions are caught and appended to the bundle's `manifest.notes` — a buggy plugin never crashes the export.
Plugins that omit the field contribute nothing to the bundle from the backend side. Frontend plugins can independently push state via `window.slopsmith.diagnostics.contribute(plugin_id, payload)` from their `screen.js` before the user hits Export. Bundle layout + per-file schemas: [docs/diagnostics-bundle-spec.md](docs/diagnostics-bundle-spec.md).
Best practices:
- Embed your own `schema` field (e.g. `"my_plugin.diag.v1"`) in JSON returned by `callable` so future tooling can dispatch by version.
- Keep payloads small (< 100 KB). Diagnostics are not a backup channel — that's `settings.server_files`.
- Don't include user secrets, API keys, or session tokens. The bundle is shared with maintainers / posted to GitHub issues.
`type` is an optional role hint (slopsmith#36). Supported values:
- `"visualization"` — plugin provides a highway renderer. Declaring this makes the plugin eligible for the main-player viz picker AND splitscreen's per-panel picker. Must pair with a `window.slopsmithViz_<id>` factory exporting the setRenderer contract below.
- Absent → no declared role; plugin is loaded and its script runs, but it doesn't appear in role-specific UIs.
**Backend routes** — `routes.py` must export a `setup(app, context)` function. The `context` dict provides:
- `config_dir` — persistent config path
- `get_dlc_dir()` — returns the DLC folder Path
- `extract_meta()` — metadata extraction callable
- `meta_db` — shared MetadataDB instance
- `library_providers` — shared library provider registry for source-aware browsing
- `register_library_provider(provider)` — register a plugin-provided library source. Providers expose `id`, `label`, optional `kind`/`capabilities`, and callable `query_page`, `query_artists`, `query_stats`, and `tuning_names` methods. Providers with `art.read` may also expose `get_art(song_id)` returning one of: a `Response` object (any media type, served as-is); raw `bytes` or `bytearray` (**assumed PNG** — use a `Response` or a `dict` with `content`+`media_type` keys for JPEG/WebP or other formats); a URL string (http/https → 302 redirect; other schemes are rejected with 400); a filesystem path string or `Path` (served as a file with auto-detected media type); or a `dict` with a `url`, `path`, or `content` key. Providers with `song.sync` may expose `sync_song(song_id)` returning `None` (success with no local file) or a `dict` — the dict is passed through as the JSON response and should include `filename`/`local_filename` if a local playable file was produced.
- `unregister_library_provider(provider_id)` — remove a plugin-provided library source by id. The built-in `local` provider cannot be removed.
- `get_sloppak_cache_dir()` — sloppak cache path
- `load_sibling(name)` — loads a sibling module from this plugin's directory under a unique, namespaced module name. See "Sibling imports" below.
- `log` — stdlib `logging.Logger` namespaced to `slopsmith.plugin.<id>`. Pre-configured with the app-wide level, format (including JSON mode), and correlation IDs. Use this for all backend plugin output instead of `print()`. See "Backend plugin logging" below.
**Sibling imports — use `load_sibling`, not bare imports** (slopsmith#33). The plugin loader inserts each plugin's directory onto `sys.path` so `from extractor import X` works, but Python caches imports by **module name** in `sys.modules`. Two plugins that each ship a top-level `extractor.py` (or any other generic name — `util.py`, `client.py`, `parser.py`, `config.py`, …) collide: whichever loads first wins, and the other plugin's `from extractor import X` either gets the wrong module or fails with `cannot import name 'X' from 'extractor'`.
The fix is `context["load_sibling"](name)`, which loads the sibling under a namespaced module name (`plugin_<id>.<name>`, where plugin_id is bijectively encoded so reverse-DNS-style ids like `com.example.foo` work without colliding: `_` -> `_5f_`, `.` -> `_2e_`) so each plugin gets its own copy:
```python
def setup(app, context):
helper = context["load_sibling"]("helper")
HelperClass = helper.HelperClass
# …
```
Notes:
- `name` is a bare module name — no `.py` suffix, no slashes, no `.`. The helper raises `ValueError` for path traversal / format issues and `ImportError` for missing files.
- Both single-file siblings (`extractor.py`) and package-form siblings (`extractor/__init__.py`) work. Package form wins when both exist (matches CPython's import-resolution precedence).
- Relative imports between siblings work — `from .shared import X` in a top-level helper, `from ..shared import X` from inside a sibling package. The synthetic parent package `plugin_<id>` carries the plugin directory in its `__path__`.
- `from . import sibling` (attribute-style) also resolves: loaded children are exposed as attributes on the parent package.
- Repeat calls return the cached module. Concurrent first-time calls are serialized via per-module locks so no caller observes a half-initialized module.
- Bare `import sibling` from `routes.py` still works during the transition period, but the loader prints a startup warning when it detects two plugins shipping a same-named top-level module — covering both `.py` files and package directories. Migrate to `load_sibling` to silence the warning and immunize your plugin from future ecosystem collisions. (Don't mix bare imports and `load_sibling` for the same module — they'd execute the file twice and split module-level state.)
**Frontend scripts** — `screen.js` runs in the global scope via a `<script>` tag. It can access `window.playSong`, `window.showScreen`, `window.createHighway`, the `<audio>` element, and the `window.slopsmith` event emitter.
**The playSong wrapper chain** — Plugins commonly wrap `window.playSong` to hook into song playback. Plugins load alphabetically, so the last-loaded (alphabetically later) wrapper runs first, while the alphabetically first plugin runs closest to the original. Be aware that `await` calls in inner wrappers yield to the event loop — WebSocket messages can arrive before outer wrappers finish setup.
## Plugin Best Practices
### v3 UI (fee[dB]ack v0.3.0) — player-chrome contract
v0.3.0 ships a redesigned UI behind a flag (`SLOPSMITH_UI=v3` or the `/v3` route);
the classic UI (v2) stays the default until 0.3.0 ships, so **plugins must work in
both**. v3 reuses the same engine (`server.py`, `app.js`, `highway.js`, `playSong`,
`showScreen`, capabilities, library providers, the `window.slopsmithViz_<id>` /
`setRenderer` contract), so a plugin's **backend, capabilities, `nav`/`screen`,
visualization renderers, diagnostics, and settings export work unchanged** — v3
surfaces `nav` in its sidebar and mounts screens exactly as v2 does.
**The only thing that changed is the player chrome.** If your plugin injects a
control into it, you must adapt:
- v2's wide always-visible `#player-controls` bar is, in v3, a **minimal
auto-hiding transport** (fades ~2.5 s after the pointer stills during playback)
plus a hover-reveal left icon rail. So injecting into `#player-controls` the
legacy way means your control **auto-hides**, and the legacy insertion anchors
(`insertBefore` the `span.text-gray-700` separator, or `button:last-child` / ✕
Close) **don't exist in v3** → it lands wrong / unreachable.
- **Detect v3** with `window.slopsmith.uiVersion === 'v3'` and **mount into
`window.slopsmith.ui.playerControlSlot()`** (a stable, always-reachable container
— the "Plugins" rail popover) instead of `#player-controls`. Drop the dead
anchors (append), and guard re-injection against the *actual* container
(`controls.contains(myBtn)`), not a hard-coded `#player-controls`.
- A host `MutationObserver` re-homes legacy `#player-controls` children into the
slot as a fallback, but it **breaks plugins that guard on
`#player-controls.contains()`** (the moved node fails the check → re-inject every
song). Mount into the slot yourself; don't rely on the shim.
- v3 uses `fb-*` tokens (`fb-card`, `fb-text`, `fb-textDim`, `fb-primary`,
`fb-border`) vs v2's `dark-*`/`accent`; legacy classes still render acceptably.
Keep `#player` overlay `z-index` ≤ the chrome layers (transport/HUD 20, rail 30,
popovers 40).
Full guide + the canonical snippet: **[docs/plugin-v3-ui.md](docs/plugin-v3-ui.md)**.
Verify any player-injecting plugin in **both** `/` (v2) and `/v3`.
### Performance — never run DOM queries on a per-frame path
Plugins share the main thread with the highway's 60 fps render loop, and during
playback the highway + note detectors mutate the DOM ~60×/s — so anything that
*reacts* to DOM changes runs that often too. Work that looks cheap in isolation
becomes the dominant cost when it runs every frame. A profiled "the 3D highway is
laggy" report turned out to be **three plugins doing per-frame `querySelectorAll`**
(~18% of main-thread CPU + NodeList GC churn), not the renderer. The GPU was idle.
- **Never call `querySelector` / `querySelectorAll` inside `draw()`, a
`requestAnimationFrame` loop, a short `setInterval`, or a `MutationObserver`
callback.** Resolve the element(s) **once** when your UI mounts and cache the
references; re-resolve only when the cached node is gone (`!el.isConnected`).
`querySelectorAll` also allocates a fresh `NodeList` every call → GC pressure at
60 fps. (notedetect #75 — a VU meter that `querySelector`'d its bar every tick.)
- **Scope `MutationObserver`s narrowly — never `observe(document.body, { subtree:
true })` just to notice your own UI's container mount.** A body-subtree observer
fires on *every* DOM mutation anywhere, including the per-frame highway churn, so
a callback that then scans the document is a per-frame full-DOM scan. Observe the
specific container; if it's swapped on screen changes, observe a stable parent,
or **cheaply early-bail** (one `getElementById` / a screen-state check) *before*
the expensive work. (sloppak-converter #32 — a body-subtree observer re-ran
whole-document inject sweeps on every frame of playback.)
- **Stop playback-tied loops when their UI is hidden.** An rAF/interval meter (VU,
etc.) that keeps drawing while its panel is closed is pure waste — gate it on
visibility, or stop and restart it on open/close.
- **Per-instance, not global.** Under splitscreen a viz/detector plugin runs
multiple instances. Cache refs and resolve panels against your *own* instance's
container, never a global `document.querySelector` that could grab a sibling
instance's node. (notedetect #75 follow-up.)
These are cheap to get right up front and expensive to retrofit. Profile the
**main thread**, not the GPU, when a renderer "feels laggy" — the offender is
usually an unrelated plugin's per-frame DOM work.
### Visualization plugins — two complementary contracts
Slopsmith supports two ways for a plugin to participate in the main player's visuals. They coexist; the setRenderer contract is the default for any viz that draws a highway-shaped surface, and overlays handle layered decorations on top.
**Pick the right shape:**
- Replacing the whole highway drawing on the existing highway canvas (your renderer owns its rendering context / resources; `createHighway()` still owns the canvas element and the rAF loop)? → **setRenderer** (section 1). Enters the viz picker. Works in both the main player and per-panel under splitscreen.
- Adding a layer on top of whichever viz is active? → **Overlay** (section 2). Navbar toggle, not in the picker.
#### 1. setRenderer contract (slopsmith#36) — preferred
Plugins that want to replace the main highway's draw function (per panel, per session) export a renderer factory on `window.slopsmithViz_<id>` where `<id>` matches the `id` in `plugin.json` (`type: "visualization"` required). The factory returns an object matching this shape:
```js
window.slopsmithViz_my_viz = function () {
return {
// Required canvas context type. Default '2d' if omitted.
// highway.js reads this BEFORE calling init() so it can
// replace the underlying <canvas> element if the current
// one is locked to a different context type (see "Canvas
// context-type swapping" below).
contextType: '2d', // or 'webgl2'
init(canvas, bundle) {
// One-time setup. Own your getContext() call here —
// acquire '2d' or 'webgl2' depending on the renderer.
// The canvas you receive is guaranteed to either be
// unbound or already bound to your declared contextType.
this.ctx = canvas.getContext('2d');
},
draw(bundle) {
// Called each requestAnimationFrame tick by the factory.
// `bundle` is a snapshot with: currentTime, songInfo, isReady,
// notes, chords, anchors (all difficulty-filter-aware),
// beats, sections, chordTemplates, stringCount, lyrics,
// toneChanges, toneBase, mastery, hasPhraseData, inverted,
// lefty, renderScale, lyricsVisible, the 2D coordinate
// helpers project and fretX, and getNoteState (see below).
// `stringCount` is the active arrangement's string count (4
// for bass, 6 for guitar, 7+ for extended-range GP imports —
// size string-indexed geometry against this, not a hardcoded
// 6). If your renderer needs lefty-aware text rendering, check
// bundle.lefty and apply the mirror transform yourself —
// a bundle-level helper isn't provided because it would
// need your renderer's own context, not the factory's.
//
// bundle.getNoteState(note, chartTime) (slopsmith#254) — call
// this per visible chart note / chord-note to find out whether
// a scorer (note_detect) has flagged it 'hit' / 'active' (a
// sustain currently being held correctly) / 'miss', so the gem
// itself can light up / a held sustain can glow instead of
// relying on an overlay ring. Returns null when no provider is
// registered or it reports nothing for this note; otherwise
// { state: 'hit'|'active'|'miss', alpha: 0..1, color: string|null }.
// For chord notes pass the chord's time (note_detect keys its
// judgments by `${time}_${string}_${fret}`). 'hit' and 'active'
// are both "lit" — a renderer may treat them identically; the
// provider owns all fade timing via `alpha` and by simply
// ceasing to return state when the effect should end.
},
resize(w, h) {
// Optional. Canvas dims already updated; re-create WebGL
// framebuffers / reset 2D transforms here.
},
destroy() {
// Optional. Release resources, remove DOM nodes, null refs.
// Called before setRenderer() swaps to another renderer
// and on highway.stop().
},
};
};
```
Selecting this plugin in the main-player viz picker — or in splitscreen's per-panel picker — calls `highway.setRenderer(factory())` on the existing highway instance. The built-in 2D highway is the default renderer and is restored by passing nullish — `setRenderer(null)` and `setRenderer(undefined)` both work (the implementation gates on `r == null`). Splitscreen panels create one `createHighway()` per panel and each independently consults the picker, so N panels can run different renderers (or N copies of the same renderer with different arrangements) without coordination.
**Lifecycle contract.** The factory returns a single renderer instance that may go through multiple `init() → ... → destroy()` cycles as the user navigates between songs or screens. Specifically:
- `init(canvas, bundle)` runs when the highway has a canvas and the renderer takes over drawing. This is when to acquire `getContext()`, build shaders / meshes / DOM nodes, and register listeners.
- `draw(bundle)` runs on every rAF frame once the WebSocket `ready` message has fired and until the renderer is replaced or the highway stops. It is **not** called during the loading / reconnect window (between `api.init()` + `stop()` and the next `ready`) — that would hand the renderer half-populated chart arrays. Renderers that want to show a "loading" state can read `bundle.isReady` inside a future-widened contract, but today the factory gates `draw` behind the ready flag and `isReady` is only informational once it does fire.
- `destroy()` runs when the renderer is replaced via another `setRenderer(...)` call, OR when `highway.stop()` is called (e.g. the user navigates away from the player). It releases everything `init()` acquired.
- **After `destroy()`, the same instance may receive another `init()` call** — this happens on `playSong()` which does `stop()` → `init()` to reuse the same canvas element for the next song. Renderers must tolerate `init()` being called again on an instance that was previously destroyed. Practically: null your refs in destroy, re-acquire them in init.
- `destroy()` is skipped when it would run on an un-init'd renderer — if a caller does `setRenderer(x)` before the highway ever init'd (possible when restoring a saved picker selection at page load), `x.destroy()` is not called until `x.init()` has run at least once.
- `resize(w, h)` is optional; runs after init and whenever the canvas dimensions change.
**Key rules:**
- The factory **returns a fresh object on each call** — important for splitscreen, where multiple panels will each get an independent instance.
- The renderer **owns its own rendering context** (2D or WebGL). Factory will not call getContext for you.
- **Canvas context-type swapping.** Browsers lock a `<canvas>` to the first context type successfully acquired for its lifetime: once `getContext('2d')` succeeds, `getContext('webgl2')` on that same canvas returns `null`, and vice versa. To let arbitrary 2D ⇄ WebGL renderer swaps work mid-session, `highway.setRenderer()` reads the next renderer's `contextType` before calling its `init()` and, if it differs from the type currently bound, replaces the underlying `<canvas>` element with a fresh one via `oldCanvas.cloneNode(false)` followed by `oldCanvas.replaceWith(newCanvas)`. The factory then calls the renderer's `init(newCanvas, bundle)` with the fresh element so its `getContext()` succeeds. Practical implications:
- **What survives the swap.** `cloneNode(false)` preserves *every HTML attribute* on the element — `id`, `class`, inline `style`, all `data-*` and `aria-*` attributes, `role`, `tabindex`, the attribute form of `width`/`height`, and anything else a plugin attached. DOM position is preserved by `replaceWith()`, so siblings, parents, and surrounding layout are unaffected.
- **What does NOT survive.** Event listeners attached via `addEventListener` are NOT cloned, and expando properties set imperatively on the JavaScript object (such as the bound rendering context, or any `canvas._myPlugin = …`-style data a plugin attached) are not carried over either. The bound rendering context being left behind on the detached element is exactly what allows the new canvas to start fresh and accept a different `getContext()` call. Note: `canvas.width`/`canvas.height` *are* reflected HTML attributes, so those values do survive the clone; `api.resize()` re-applies the backing-store dimensions on the new element after the swap regardless.
- Renderers must **declare `contextType`** on the returned instance (`'2d'` or `'webgl2'`; absent → `'2d'`). Factories may also expose it as a static (`window.slopsmithViz_<id>.contextType = 'webgl2'`) so core can read it before constructing the renderer — used today by Auto-mode evaluation.
- Plugins that hold a stale reference to the highway canvas across renderer swaps — including any code that registered listeners directly on the canvas element rather than on `window`/`document` — should listen for the `highway:canvas-replaced` event on `window.slopsmith` and re-acquire / re-register. `window.slopsmith.emit` dispatches a `CustomEvent`, so the payload `{ oldCanvas, newCanvas, contextType }` lives on `event.detail`, not on the event object itself:
```js
window.slopsmith.on('highway:canvas-replaced', (event) => {
const { oldCanvas, newCanvas, contextType } = event.detail;
// re-acquire / re-register against newCanvas
});
```
Plugins that re-query `document.getElementById('highway')` lazily inside their own event handlers don't need this listener — they pick up the new element automatically (it keeps `id="highway"`).
- **`highway:visibility`** — fired on `window.slopsmith` whenever the highway canvas transitions between displayed and hidden. Detection is DOM-based via `canvas.offsetParent === null` (catches `display:none` on the canvas or any ancestor — e.g. splitscreen's `#highway` hide) or whatever a host explicitly sets via `highway.setVisible(bool)`. While `visible === false`, core skips the rAF `renderer.draw(bundle)` call AND the default 2D draw, so renderers don't have to no-op themselves. The event is emitted only on transitions (including the first one after `init()`), not every frame. Payload `{ visible, canvas }` lives on `event.detail`:
```js
window.slopsmith.on('highway:visibility', (event) => {
const { visible, canvas } = event.detail;
// Toggle any sibling DOM your renderer mounts. The 3D Highway
// renderer hides its `.h3d-wrap` overlay here so `display:none`
// on `#highway` actually hides the visible output.
});
```
Renderers that only paint to the slopsmith canvas don't need this listener — the rAF skip is enough. Renderers that mount sibling DOM (separate WebGL contexts, overlays, etc.) do.
- **`highway.setVisible(bool | null)`** — forces the visibility state regardless of `offsetParent`. Pass `null` to clear the override and resume DOM-based detection. Useful when the host hides the highway via `visibility:hidden`, `opacity:0`, transforms, or clipping rather than `display:none`. The override re-emits any resulting transition immediately rather than waiting for the next rAF tick.
- Default-renderer ctx is closure-cached. The replace path nulls the closure ctx so stale draw paths short-circuit; the next default-renderer `init()` re-acquires the 2D context from the new canvas cleanly.
- `draw(bundle)` receives difficulty-filtered arrays — never read from `_filteredNotes` or other internals.
- `_drawHooks` fire for the default 2D renderer (the factory calls them at the end of each frame). Custom WebGL renderers that maintain a 2D overlay canvas (like the bundled 3D highway) also call `window.highway.fireDrawHooks(ctx, W, H)` on that overlay so overlay plugins continue to work regardless of which renderer is active. Custom renderers without a 2D overlay context should not attempt to fire hooks.
**Auto mode — `matchesArrangement(songInfo)` (optional).**
The viz picker prepends an "Auto (match arrangement)" entry that is the default selection on fresh installs. When Auto is active, core evaluates registered viz factories on every `song:ready` and swaps the renderer to the first factory whose `matchesArrangement(songInfo)` predicate returns truthy. No match → the built-in 2D highway.
Declare the predicate as a static on the factory (not the instance) so core can evaluate it without constructing a throwaway renderer:
```js
window.slopsmithViz_piano = function () { /* ... */ };
window.slopsmithViz_piano.matchesArrangement = function (songInfo) {
return /keys|piano|synth/i.test((songInfo && songInfo.arrangement) || '');
};
```
- `songInfo` is the highway's live song_info snapshot — `arrangement`, `tuning`, `capo`, `centOffset`, `arrangement_index`, `filename`, `artist`, `title`, etc. May be `{}` before the first song loads.
- Factories without `matchesArrangement` are skipped during auto-selection — the correct default for arrangement-agnostic viz (tabview, jumpingtab) that only make sense as manual picks.
- Explicit picker selections override Auto and are persisted to `localStorage.vizSelection`, so the pinned choice survives page reloads until the user switches back to "Auto" (which also persists). Picking "Auto" re-evaluates against the current song immediately. In contexts where `localStorage` is unavailable (private mode, sandboxed iframes, some test runners) persistence falls back to the current picker `<option>` value, which still overrides Auto for as long as the page stays loaded.
- When an Auto-selected renderer fails and core emits `viz:reverted`, the picker falls back to the built-in default and disables auto-switching until the user re-selects Auto.
- First match wins (picker order), so the registration order of plugins is the tiebreaker. Keep predicates narrow to avoid stealing songs from more specialized viz.
**WebGL viz in Auto mode.** Auto evaluation runs on every `song:ready` regardless of which renderer is active. Auto-installing a WebGL renderer when the canvas is currently 2D — or reverting from a WebGL Auto pick to the default 2D — works without a reload because `setRenderer` swaps the canvas element when `contextType` differs (see "Canvas context-type swapping" above). WebGL viz can therefore safely declare `matchesArrangement` and rely on Auto. For 3D Highway specifically, `_canRun3D()` in app.js still gates Auto from picking it on machines without WebGL2 — that fallback is independent of canvas swapping.
**Per-instance settings for host plugins (slopsmith#849).** A viz provider may declare per-instance controls a consuming host (e.g. splitscreen's per-panel popover) renders generically, by adding a `settings` array to its `capabilities.visualization` manifest block: `[{ key, label, type: "toggle" | "range" | "select", default, min?, max?, step?, options? }]`. This is the capability-native, declarative replacement for the ad-hoc `factory.panelControls` static. The validated list is surfaced through the visualization host's `list-providers` snapshot, so a host reads it without knowing the plugin. **A provider that declares `settings` MUST implement `applySetting(key, value)` on its renderer instance** — the host calls it on the specific per-panel instance, which is inherently per-panel (no canvas→panel resolution, no shared global localStorage keys). `getSetting(key)` is optional (the host falls back to the declared `default`); the host owns persistence. `factory.panelControls` remains read as a legacy fallback for hosts that still consume it, but new viz should declare `settings` + `applySetting`.
#### 2. Overlay contract — for add-on layers
Plugins that add a layer on top of whichever visualization is active — HUDs, fretboard diagrams, chord labels, practice feedback — don't replace the renderer. They manage their own canvas, their own rAF loop, and a toggle button somewhere visible (typically a navbar pill), reading public highway state via the getters:
- `highway.getTime()` / `highway.getBeats()` — current playback position
- `highway.getNotes()` / `highway.getChords()` — raw arrays containing every note/chord in the chart regardless of the current difficulty level
- `highway.getFilteredNotes()` / `highway.getFilteredChords()` — difficulty-filtered variants. Returns the master-difficulty-filtered arrays when the song has phrase-level data (slider active); falls through to the raw arrays for songs with a single difficulty level (slider disabled). Plugins that process only the notes the player is currently expected to play should use these instead of `getNotes()` / `getChords()`
- `highway.hasPhraseData()` — returns `true` when the current song has phrase-level difficulty ladder data (i.e. the mastery slider is active and `getFilteredNotes()` / `getFilteredChords()` return a filtered subset). Use this to gate logic that only makes sense when difficulty filtering is available
- `highway.getPhrases()` — phrase timing windows `[{ index, start_time, end_time, max_difficulty }]` for the current song's difficulty ladder. Returns `null` when phrase data is absent (GP imports, single-difficulty charts). Read-only; do not mutate. Pair with `hasPhraseData()` to gate phrase-aware logic.
- `highway.getMastery()` — current master-difficulty slider value as a fraction `0..1`. Reflects the same value the mastery slider is set to; meaningful only when `hasPhraseData()` is true.
- `highway.getChordTemplates()` — chord shape lookup table; index by `chord.id` from `getChords()` to get `{ name, fingers, frets }`. `fingers` and `frets` are per-string arrays (length matches the tuning's string count); within `fingers`, `-1` = unused, `0` = open string, `n > 0` = finger number. arrangement XML sources populate real fingerings; GP imports currently emit all `-1` since pre-import sources don't carry finger data. Not filter-aware: templates are static metadata, every `chord_id` referenced by `getChords()` is guaranteed valid
- `highway.getSongInfo()` — tuning, arrangement, capo
- `highway.getStringCount()` — number of strings on the active arrangement (4 for bass, 6 for guitar, 7+ for extended-range GP imports). Derived server-side as `max(notes-max-string + 1, name-based fallback, len(tuning))` where the tuning length only contributes when it isn't the arrangement XML padded 6-string form (sloppak / GP-imported sources carry trimmed tuning lengths). The name-based fallback is 4 for arrangements containing "bass" (case-insensitive) and 6 otherwise. This combination handles partial-string-usage charts (a 6-string lead that never plays string 5), extended-range GP imports (5-string bass, 7-string guitar), and sloppaks that explicitly encode the instrument range — without requiring plugins to do their own arrangement-name matching
- `highway.getLefty()` / `highway.getInverted()` — mirror + invert state
Overlays do NOT appear in the viz picker and do NOT declare `"type": "visualization"` in `plugin.json`. They coexist with whichever renderer (default 2D, 3D highway, piano, ...) the user has picked.
**Key rules:**
- **Own your rAF + canvas** — don't piggyback on `_drawHooks` or on `createHighway`'s rendering context. Draw hooks fire for the default 2D renderer and for custom renderers that explicitly call `window.highway.fireDrawHooks(ctx, W, H)` (e.g. the bundled 3D highway fires them on its 2D overlay canvas), but not for every custom renderer.
- **Re-read state every frame** — overlay output must track whatever the current renderer is drawing. Don't cache note positions across frames.
- **Respect lefty + invert toggles** — if the overlay depicts strings or frets, mirror using the same transforms the active renderer would.
- **If you position with `highway.project` / `highway.fretX` (the 2D-highway geometry), gate on `highway.isDefaultRenderer()`** — those helpers describe the *built-in 2D* highway's depth curve and fret zoom. When a custom renderer (3D highway, piano, …) is active your draw hook still fires (on that renderer's 2D overlay layer), but those coordinates won't match its scene — markers land in arbitrary places. Skip rendering when `isDefaultRenderer()` is false; the custom renderer owns that feedback. Renderer-agnostic overlays (fretboard diagram, chord-label HUD — they use `getNotes()`/`getChordTemplates()` + their own layout) don't need this guard.
- **Clean up on toggle-off** — cancel rAF and remove/hide the overlay canvas so inactive overlays aren't wasting frames.
Reference: [fretboard plugin](https://github.com/got-feedback/feedback-plugin-fretboard) — canonical overlay implementation (navbar toggle, own canvas, 80ms active-note window).
**Why two?** setRenderer plugs into an existing highway — main-player or splitscreen-panel — reusing its WebSocket and data parsing, so the common "I want a different look for the same data" case is zero boilerplate AND multi-instance for free. Overlays compose with whatever renderer is active — they decorate rather than replace, so multiple can stack (fretboard + chord labels + practice feedback) without fighting over the canvas.
A previous standalone-pane contract (`window.createMyVisualization({ container })` with its own WebSocket per pane) was used by splitscreen pre-Wave-C. It's been retired now that splitscreen calls `setRenderer` on per-panel `createHighway()` instances; if you find references in older plugin docs or external integration guides, those describe the legacy path.
#### 3. Note-state provider — for scorers that want renderers to "light up" notes (slopsmith#254)
A scoring plugin (note_detect) can publish a per-note judgment so whichever renderer is active draws the **gem itself** lit on a correct hit, and keeps a sustain trail glowing while it's still being played correctly — instead of a separate overlay ring floating near the note.
```js
// In the plugin (after resolving the highway instance):
highway.setNoteStateProvider((note, chartTime) => {
// `note` is the chart note object ({ t, s, f, sus, ... }); for chord
// notes `chartTime` is the chord's time. Return one of:
// - falsy → no special state (render normally)
// - 'hit' — struck correctly; renderer lights the gem
// - 'active' — a sustained note is right now being held correctly
// - 'miss' — missed; renderer may red-wash the gem
// - { state: <one of the above>, alpha?: 0..1, color?: '#rrggbb' }
// You own all fade timing: return a decaying `alpha` for a struck-note
// glow, `alpha: 1` (or a bare string) for a held sustain, and stop
// returning state when the effect should end. Keep it cheap — it's
// called per visible note per renderer per frame.
});
// On teardown: highway.setNoteStateProvider(null);
```
- Only one provider is active at a time (last `setNoteStateProvider` wins). `highway.getNoteStateProvider()` returns the current one (or null).
- The built-in 2D highway consults it in `drawNote` / `drawSustains` / the chord-frame path: 'hit'/'active' → bright string colour + additive halo + a contained "sizzle" (crackling sparks, throbbing core, a shockwave ring on a fresh strike) on the gem and a bright (vs dim) sustain trail; 'miss' → faint red wash. The bundled **3D highway** reads the same data via `bundle.getNoteState` (bright string-tinted outline + bright body + glowing sustain + a contained sparkle hugging the note rect on hit/active; red outline + suppressed body on miss). Custom renderers that want it call `bundle.getNoteState(note, chartTime)` — it null-guards and returns the normalized `{ state, alpha, color }` (or null).
- This is orthogonal to the overlay contract: note_detect remains an overlay (HUD, diagnostic miss markers, the "currently detected" indicator) *and* a scorer that feeds this provider. A renderer that ignores `getNoteState` simply doesn't light gems — nothing breaks.
### Audio mixer fader registration (slopsmith#87)
Plugins that produce audio outside the song's `<audio>` element (NAM amp output, synth voices, etc.) can register a labeled fader so users can balance them against the song from one mixer popover in the player controls.
```js
function _registerFader() {
const api = window.slopsmith && window.slopsmith.audio;
if (!api) return;
api.registerFader({
id: 'my_plugin', // unique key
label: 'My Plugin', // shown above the fader
unit: 'dB', // optional suffix shown next to the value (e.g. '%', 'dB')
min: 0, max: 2, step: 0.05,
defaultValue: 1.0,
getValue: () => _myCurrentVolume, // read current value
setValue: (v) => _setMyVolume(v), // write + persist + apply
});
}
if (window.slopsmith && window.slopsmith.audio) {
_registerFader();
} else {
window.addEventListener('slopsmith:audio:ready', _registerFader, { once: true });
}
```
The plugin owns persistence — the registry calls `getValue()` when the popover opens, and also after each `setValue()` during slider drags to re-sync the displayed value. Keep `getValue()` cheap and side-effect-free, and make sure `setValue()` updates whatever backing state `getValue()` reads synchronously. Pair `setValue` with whatever your plugin already does internally (write the GainNode, persist to localStorage, update any in-plugin label). Use `unregisterFader(id)` when your plugin is teardown-able and you want the strip to disappear; otherwise keep it registered so the user's setting persists across toggle states.
### Backend plugin logging
Use `context["log"]` for all backend plugin output. It is a stdlib `logging.Logger` namespaced to `slopsmith.plugin.<id>`, pre-configured with the app-wide level, format (including JSON mode), and correlation IDs. Never use `print()` — it bypasses correlation context and log rotation.
```python
def setup(app, context):
log = context["log"]
log.info("plugin ready")
log.warning("optional dependency %r not found, feature disabled", dep)
try:
risky_init()
except Exception:
log.exception("unhandled error during setup") # auto-captures traceback
```
For CLI entry points (scripts that also run as `__main__`), add a stdlib fallback so the logger works without the server pipeline:
```python
if __name__ == "__main__":
import logging
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
```
### Diagnostics contribution from frontend (slopsmith#166)
Plugins that hold useful debug state in the browser (active model name, last user input, internal counters) can push it into the diagnostics bundle by calling `window.slopsmith.diagnostics.contribute(plugin_id, payload)` at any time. The contribution API is idempotent — repeated calls overwrite the previous value. Whatever was last contributed before the user hits Export Diagnostics is what lands in `plugins/<plugin_id>/client.json`.
```js
window.slopsmith.diagnostics.contribute('my_plugin', {
schema: 'my_plugin.client_diag.v1',
active_preset: getActivePreset(),
last_error: _lastError,
});
```
Loaded from `static/diagnostics.js` ASAP in `<head>` so the console-wrap is in place before any other script runs. Available on the `window.slopsmith.diagnostics` namespace alongside `snapshotConsole()`, `snapshotHardware()`, `snapshotUa()`, `snapshotLocalStorage()`, `snapshotContributions()`. Keep your payload small (< 100 KB) and don't include secrets — bundles are shared with maintainers.
### Keyboard Shortcuts
Plugins can register keyboard shortcuts via the global `window.registerShortcut()` function. Shortcuts appear in the `?` help panel.
```js
window.registerShortcut({
key: 'k', // key value (e.key) or key code (e.code)
description: 'Toggle my view', // shown in the help panel
scope: 'player', // 'global' | 'player' | 'library' | 'settings' | 'plugin-{id}'
condition: () => _isMyViewActive, // optional guard
handler: (e) => _myAction() // called when shortcut triggers
});
```
**Scope** controls when the shortcut is active:
- `global` — works on any screen
- `player` — only on the player screen
- `library` — only on the home/favorites screens
- `settings` — only on the settings screen
- `plugin-{id}` — only when your plugin's screen is active
**Panel-scoped shortcuts:** For plugins that create multiple panels (e.g., splitscreen), shortcuts are automatically scoped to the active panel. Use `const panel = window.createShortcutPanel(id)` to create a panel (it returns the panel object — keep the reference so you can call `panel.clearShortcuts()` during cleanup) and `window.setActiveShortcutPanel(id)` to switch between them. Each panel has its own shortcut registry, so multiple panels can have the same key without collisions.
**Condition** is an optional guard function. If it returns false, the shortcut is skipped even if in scope.
**Key matching:** The handler matches against both `e.key` (character produced) and `e.code` (physical key). Use `e.key` for letters/symbols that depend on keyboard layout, and `e.code` for special keys (e.g. `Space`, `ArrowLeft`).
**Built-in shortcuts:**
| Key | Description |
|-----|-------------|
| `?` | Show keyboard shortcuts panel (global) |
| `Space` | Play/Pause (player only) |
| `←` / `→` | Seek ±5 seconds (player only) |
| `Escape` | Back to library (player only) |
| `[` / `]` | Audio offset ±10ms (Shift: ±50ms) (player only) |
**Debugging:** Open browser console and type `_listShortcuts()` to inspect registered shortcuts.
### General plugin guidelines
- Wrap your plugin code in an IIFE: `(function () { 'use strict'; ... })();`
- Use `localStorage` for user-facing settings, prefixed with your plugin id
- If hooking `window.playSong`, always call the original and `await` it
- If hooking `window.showScreen`, clean up your state when leaving the player screen
- Use `window.slopsmith.emit()` / `window.slopsmith.on()` for inter-plugin communication
- Use `window.registerShortcut()` to add keyboard shortcuts. Clean up with `window.unregisterShortcut(key, scope)` — pass the same scope you registered with, since the default is `'global'` and won't match `player`/`library`/`settings`/`plugin-*` bindings. For panel-scoped shortcuts, prefer `panel.clearShortcuts()`.
## Song Formats
Slopsmith supports two song formats:
### Loose folder (XML charts)
A directory containing arrangement XML plus an audio file (and optional `manifest.json` + album art). Discovered, indexed, and played directly — see `lib/loosefolder.py`. Metadata follows a `manifest.json` → XML tags → folder-name priority chain. Songs are tagged `format: "loose"` in the library.
### Sloppak (open format)
An open, hand-editable song package designed for Slopsmith. Exists in two interchangeable forms:
- **Zip archive** (`.sloppak` file) — distribution form
- **Directory** (`.sloppak/` folder) — authoring form
**Contents:**
```
manifest.yaml Song metadata (title, artist, album, duration, tuning, arrangement IDs, ...)
arrangements/
lead.json Note/chord/anchor data in wire format (see song.py)
rhythm.json Files here are driven by manifest.yaml arrangement entries
... (e.g. arrangements/<arrangement-id>.json)
stems/
full.ogg Mixed audio (always present)
guitar.ogg Individual stems (optional, from Demucs split)
bass.ogg
drums.ogg
vocals.ogg
piano.ogg
other.ogg
cover.jpg Album art (optional)
lyrics.json Syllable-level lyrics (optional)
```
Sloppak is the preferred format for new features. The [Stems plugin](https://github.com/topkoa/slopsmith-plugin-stems) provides live stem mixing for sloppak songs.
**Full developer reference:** [docs/sloppak-spec.md](docs/sloppak-spec.md) — manifest schema, arrangement wire format, and how to extend the format with new data types (drum tab, key/scale annotations, etc.).
**Key code:**
- `lib/sloppak.py` — format detection, zip/directory resolution, metadata extraction, song loading
- `lib/sloppak_convert.py` — sloppak assembly pipeline, Demucs stem splitting
- `lib/song.py` — shared data models (`Note`, `Chord`, `Arrangement`, `Song`) and wire format serialization used by both formats
## Frontend Conventions
- **No frameworks** — vanilla JS, fetch API, DOM manipulation
- **Globals** — `highway`, `audio`, `playSong()`, `showScreen()`, `createHighway()`, `window.slopsmith`
- **Storage** — `localStorage` for all user preferences
- **Styling** — Tailwind CSS utility classes, dark theme (`bg-dark-600`, `text-gray-300`, accent `#4080e0`, gold `#e8c040`). Tailwind is served as a **prebuilt** stylesheet (`static/tailwind.min.css`, regenerated by `bash scripts/build-tailwind.sh`), **never** the runtime Play CDN — the CDN's on-the-fly JIT rescanned the DOM on the main thread and dropped ~26% of frames with the 3D highway (slopsmith-desktop#110). The committed CSS only contains classes the build scanner saw, so CI (`tailwind-fresh`) rebuilds and diffs it; run the build script and commit when you add new classes. A plugin that uses classes not guaranteed in core (notably arbitrary values like `w-[37px]`) MUST ship its own compiled stylesheet via the `styles` manifest key, built with `corePlugins.preflight = false` (utilities only — core ships the one base reset). Plugins MUST NOT load the Tailwind Play CDN or any runtime CSS JIT. See constitution Principle II.
- **Naming** — camelCase for JS functions, kebab-case for CSS classes, snake_case for plugin IDs
- **Player layout** — `#player` is `display:flex; flex-direction:column; position:fixed; inset:0`. `#highway` is `flex:1`. `#player-controls` sits at the bottom. Hiding the highway collapses the layout — use `margin-top: auto` on controls if you need to hide it.
## Backend Conventions
- **Framework** — FastAPI with uvicorn
- **Imports** — flat imports from `lib/` (no package `__init__.py`): `from song import Song`
- **Database** — SQLite via MetadataDB class with `threading.Lock` for thread safety
- **WebSocket** — JSON frames, try/except `WebSocketDisconnect`
- **Error handling** — graceful fallbacks (audio conversion errors don't crash the song, missing art returns placeholder)
- **Type hints** — used sparingly (`Path | None`, `dict`, `list`)
- **Docstrings** — minimal; code is self-documenting
## Testing
```bash
pytest # Run all tests
pytest tests/test_song.py -v # Specific file
pytest -k "round_trip" -v # Pattern match
```
- Framework: pytest
- Config: `pyproject.toml` sets `pythonpath = [".", "lib"]` and `testpaths = ["tests"]`
- CI: GitHub Actions runs pytest on push/PR to main (Python 3.12)
- Test dependencies: `requirements-test.txt`
## Tuning the note_detect plugin
Detection quality is hard to judge by eye — a player UI that "feels worse" after a code change isn't a regression you can defend in review. The plugin ships with a record-replay-sweep workflow so changes to the detector, the matcher, or the user's environment (A/V offset, latency, channel) can be measured against a single reference take.
Quick orientation:
- **Reference recording** lives in the gear popover on the player (gated behind Settings → Note Detection → "Detection tuning (advanced)"). Arm before pressing Play; auto-saves a WAV to `static/note_detect_recordings/` on song-end. The directory is bind-mounted, so the host-side harness can read it without a copy step.
- **Benchmark sloppak** ships in-tree at [docs/benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak](docs/benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak) — 8 sections each isolating a different failure mode (low-freq mono, sustained holds, hammer/pull, power chords, dense open chords, bends). Drop it directly into your sloppak DLC folder to install (don't rename — slopsmith keys off the `.sloppak` suffix even though the file is a zip under the hood). The unzipped form lands at `static/sloppak_cache/note_detect_benchmark_v1.sloppak/` after first play. Builder: [docs/benchmarks/note_detect_v1/build_benchmark.py](docs/benchmarks/note_detect_v1/build_benchmark.py).
- **Headless harness** at [`tools/harness.js`](https://github.com/got-feedback/feedback-plugin-notedetect/blob/main/tools/harness.js) in the note_detect plugin's own repo (cloned into `plugins/note_detect/` locally) runs the same `processFrame` / `matchNotes` / `checkMisses` code path off Node, in seconds per run. Same `note_detect.diagnostic.v1` schema as the in-app Download Diagnostic button.
- **A/V auto-calibrate** (Settings → Note Detection) reads `timing_error_ms_hits.median` and proposes the av-offset that drives it to zero. Iterative: usually converges in 2–3 Apply rounds.
**Always record at 1.0× playback speed** — half-speed takes produce all-miss garbage because chart times are absolute. **Always use `timing_error_ms_hits` (not all-matched) as a calibration signal** — the all-matched median pins near a constant when the offset is wrong, because the matcher silently snaps to neighbouring chart notes.
Full developer reference (workflow recipes, harness flag table, diagnostic schema, common pitfalls): [docs/note-detect-tuning.md](docs/note-detect-tuning.md).
## Versioning
- **`VERSION`** (repo root) — single source of truth; plain semver string (e.g. `0.2.4`). Bind-mounted into the container and copied by the Dockerfile so it's always available at `/app/VERSION`.
- **`GET /api/version`** — returns `{"version": "<contents of VERSION>", "source_url": "...", "license_url": "..."}`. The version drives the navbar badge; `source_url` / `license_url` populate the Settings → About links. `source_url` is configurable via the `APP_SOURCE_URL` env var (default `https://github.com/got-feedback/feedback`); `license_url` falls back to `source_url + "/blob/main/LICENSE"` (GitHub-style, default branch `main`) and is overridable via the `APP_LICENSE_URL` env var — set it explicitly when the source is hosted on a non-GitHub forge (GitLab/Gitea/self-hosted) or under a non-`main` default branch. Both env values must be `http(s)`; non-http(s) values are rejected and fall back to the safe default to prevent `javascript:`/`data:` hrefs.
- **Auto-sync** — `.github/workflows/sync-version.yml` rewrites `VERSION` via a `repository_dispatch` (`desktop-released`) fired from `slopsmith-desktop`'s release job. As an explicit automation-only exception to the "Never push directly to main" rule in Git Workflow below, the sync job commits straight to `main` as `github-actions[bot]` (version bumps are mechanical; the PR round-trip adds no signal). Human contributors must still go through feature branches + PRs. No manual VERSION edits needed. Use the workflow's `workflow_dispatch` trigger with `version: X.Y.Z` for manual runs (recovery / out-of-band bumps).
- **`CHANGELOG.md`** — follows [Keep a Changelog](https://keepachangelog.com/) format. Update the `[Unreleased]` section with each PR; when `slopsmith-desktop` cuts a release, rename `[Unreleased]` to the new version + date (the VERSION bump itself is automated).
## Git Workflow
- **Never push directly to main** — always create a feature branch and open a PR
- **Upstream remote** — set `upstream` to the canonical Slopsmith repository; `origin` is your fork
- **Plugins are gitlinks** — each plugin in `plugins/` is typically its own git repo (submodule or clone). Branch switches on the main repo can clobber plugin directories. Use `git update-index --assume-unchanged` for plugin dirs if needed.
- **Commit style** — short imperative subject line, blank line, then body explaining *why*
## WebSocket Protocol Reference
The highway WebSocket at `/ws/highway/{filename}?arrangement={index}` streams these messages in order:
| Message | Shape | Description |
|---------|-------|-------------|
| `loading` | `{ type: 'loading', stage }` | Status/progress message during extraction or conversion |
| `song_info` | `{ type, title, artist, arrangement, arrangement_index, arrangements, duration, tuning, capo, centOffset, format, audio_url, audio_error, stems }` | Song metadata. `arrangements` is the full list for the switcher. `audio_url` is `null` when audio is unavailable, in which case `audio_error` is non-null; otherwise `audio_error` is `null`. `stems` is always present — an empty array for non-sloppak songs or sloppak songs with no split stems. `tuning` is an array (6 for guitar, 4 for bass). `centOffset` is a float (cents) from the RS2014 `<centOffset>` field — commonly `-1200.0` for extended-range bass (one octave down), small non-zero values for true-tuned content (e.g. A443 ≈ +11.8 cents), `0.0` when absent. Available via `getSongInfo().centOffset`. |
| `beats` | `{ type, data: [{ time, measure }] }` | Beat timestamps with measure numbers |
| `sections` | `{ type, data: [{ time, name }] }` | Named sections (Intro, Verse, Chorus, etc.) |
| `anchors` | `{ type, data: [{ time, fret, width }] }` | Fret zoom anchors |
| `chord_templates` | `{ type, data: [{ name, frets: [6] }] }` | Named chord shapes |
| `lyrics` | `{ type, data: [{ w, t, d }], source }` | Syllables: `w`=word, `t`=time, `d`=duration. `-` joins to previous, `+` = line break. `source` is one of `"xml"`, `"whisperx"`, `"user"` — UI can use it to render an "auto-transcribed" badge for `whisperx`. Sloppaks always include `source` (legacy sloppaks without a `lyrics_source` manifest key default to `"xml"` at load time). Loose folders set it based on which extractor matched. Absent only when no lyrics fired the message at all |
| `tone_changes` | `{ type: 'tone_changes', base, data: [{ time, name }] }` | Optional — tone change events relative to the arrangement base tone; only sent if tones were found |
| `notes` | `{ type, data: [{ t, s, f, sus, ho, po, sl, bn, ... }] }` | Single notes |
| `chords` | `{ type, data: [{ t, notes: [{ s, f, sus, ... }] }] }` | Chord events |
| `phrases` | `{ type, data: [{ start_time, end_time, max_difficulty, levels: [{ difficulty, notes, chords, anchors, handshapes }] }], total }` | Optional — per-phrase difficulty ladder for master-difficulty slider (slopsmith#48). Only sent when the source chart carries multi-level phrase data (phrase-aware sloppak). Sent in chunks (`data` is a batch, `total` is the full count across messages) to avoid multi-MB single frames. Absent for GP imports and legacy sloppak; consumers must treat missing message as "single fixed difficulty — slider disabled". |
| `ready` | `{ type: 'ready' }` | All data sent — safe to finalize and start rendering |
Message delivery is incremental. You may receive `loading` updates and `lyrics` before note/chord payloads; `tone_changes` comes after `lyrics` when present and may be omitted entirely. Do not finalize rendering until you receive `ready`.
## Common Pitfalls
1. **playSong wrapper race condition** — The wrapper chain runs outermost-first (last-loaded wrapper runs first). If an inner plugin (e.g. `3dhighway`) does `await import(CDN)`, it yields to the event loop. WebSocket messages (`song_info`, `ready`) can arrive before outer plugins set their callbacks. Use `getSongInfo()` as a fallback rather than relying solely on `_onReady`.
2. **Plugin gitlinks** — Plugins are separate git repos cloned into `plugins/`. Switching branches on the main repo can delete or clobber these directories. Be careful with `git checkout` and `git clean`.
3. **Highway flex layout** — `#highway` has `flex:1` in the player. Hiding it with `display:none` removes the flex child, causing `#player-controls` to float to the top. If you hide the highway, add `margin-top: auto` to the controls div to keep it at the bottom.
4. **Multiple WebSocket connections** — The server supports many simultaneous WebSocket connections to the same song. Split screen panels, lyrics panes, and jumping tab panes each open their own. This is by design — don't try to multiplex.
5. **Plugin load order** — Plugins load alphabetically by directory name. This determines the `playSong` wrapper chain order and which plugin's UI elements appear first. If your plugin depends on another's globals, check at runtime (`typeof window.X === 'function'`), not at load time.
docs/PLUGIN_AUTHORING.md
+73
View File
@@ -0,0 +1,73 @@
# Plugin Authoring Guide
Slopsmith's plugin system is the primary extension point. Each plugin lives in `plugins/<name>/` with a `plugin.json` manifest and can provide any combination of frontend (HTML/JS), backend (Python routes), settings UI, diagnostics, and visualization renderers.
This guide is the entry point. Each topic below has a dedicated doc — read what's relevant to what you're building.
## Quickstart
```
plugins/my_plugin/
├── plugin.json Manifest (required) — see docs/plugin-manifest.md
├── screen.html Optional — markup mounted at #plugin-my_plugin
├── screen.js Optional — runs in global scope on page load
├── routes.py Optional — exports setup(app, context)
├── settings.html Optional — settings-panel HTML
└── requirements.txt Optional — pip deps auto-installed on load
```
The minimum viable plugin is a `plugin.json` with just `id` and `name`. Everything else is opt-in.
```json
{
"id": "my_plugin",
"name": "My Plugin",
"version": "0.1.0"
}
```
## Topics
| Topic | Doc | When to read |
|---|---|---|
| **Manifest reference** | [plugin-manifest.md](plugin-manifest.md) | Field-by-field reference for `plugin.json`. Read first. |
| **Visualization contracts** | [plugin-visualization-contracts.md](plugin-visualization-contracts.md) | Building a highway renderer (setRenderer), an overlay layer, or a note-state provider. |
| **Audio mixer faders** | [plugin-audio-mixer.md](plugin-audio-mixer.md) | Plugin produces audio outside the song `<audio>` element. |
| **Backend logging** | [plugin-logging.md](plugin-logging.md) | Plugin has a `routes.py`. Use `context["log"]`, never `print()`. |
| **Diagnostics contribution** | [plugin-diagnostics.md](plugin-diagnostics.md) | Adding plugin state to the Export Diagnostics bundle. |
| **Keyboard shortcuts** | [plugin-keyboard-shortcuts.md](plugin-keyboard-shortcuts.md) | Registering keys via `window.registerShortcut()`. |
| **Sibling Python imports** | [plugin-sibling-imports.md](plugin-sibling-imports.md) | Multi-file backend plugins. Use `context["load_sibling"]`. |
| **WebSocket protocol** | [websocket-protocol.md](websocket-protocol.md) | Plugins that read the highway stream directly. |
| **Testing plugins** | [testing-plugins.md](testing-plugins.md) | Conftest fixtures and Playwright patterns for plugin tests. |
| **Diagnostics bundle spec** | [diagnostics-bundle-spec.md](diagnostics-bundle-spec.md) | Existing in-depth spec — what's inside a diagnostics export. |
| **Sloppak format spec** | [sloppak-spec.md](sloppak-spec.md) | Existing in-depth spec — for plugins that read/write sloppaks. |
## General guidelines
- Wrap your plugin code in an IIFE: `(function () { 'use strict'; ... })();`
- Use `localStorage` for user-facing settings, prefixed with your plugin id.
- If hooking `window.playSong`, always call the original and `await` it.
- If hooking `window.showScreen`, clean up your state when leaving the player screen.
- Use `window.slopsmith.emit()` / `window.slopsmith.on()` for inter-plugin communication.
- Use `window.registerShortcut()` to add keyboard shortcuts. Clean up with `window.unregisterShortcut(key, scope)` — pass the same scope you registered with, since the default is `'global'` and won't match `player`/`library`/`settings`/`plugin-*` bindings. For panel-scoped shortcuts, prefer `panel.clearShortcuts()`.
## Plugin frontend globals available at runtime
- `window.playSong(filename, arrangementIdx)` — load and play a song
- `window.showScreen(name)` — navigate between screens
- `window.createHighway()` — factory for the highway renderer (used by main player and splitscreen panels)
- `window.slopsmith` — event emitter (`emit`, `on`, `off`)
- `window.slopsmith.audio` — audio mixer fader registry
- `window.slopsmith.diagnostics` — diagnostics namespace (`contribute`, `snapshotConsole`, etc.)
- `window.registerShortcut` / `window.unregisterShortcut` / `window.createShortcutPanel` — keyboard shortcuts API
- `highway` global — set when the player is active. Getters: `getTime`, `getNotes`, `getChords`, `getChordTemplates`, `getSongInfo`, `getStringCount`, `getLefty`, `getInverted`, `getBeats`, `isDefaultRenderer`, …
## Plugin load order
Plugins load alphabetically by directory name. This determines the `playSong` wrapper chain order (last-loaded wrapper runs first; alphabetically earliest plugin runs closest to the original) and which plugin's UI elements appear first.
If your plugin depends on another's globals, **check at runtime** with `typeof window.X === 'function'`, not at load time. Plugins are independent — assume any other plugin may be missing or disabled.
## Licensing for curated plugins
Plugins submitted for inclusion in the curated list must be AGPL-3.0 or AGPL-compatible (MIT, BSD, Apache-2.0). See [CONTRIBUTING.md](../CONTRIBUTING.md) for the full policy. The `plugin.json` schema enforces this via the `license` field enum — see [plugin-manifest.md](plugin-manifest.md).
docs/plugin-audio-mixer.md
+45
View File
@@ -0,0 +1,45 @@
# Audio mixer fader registration
Plugins that produce audio outside the song's `<audio>` element (NAM amp output, synth voices, etc.) can register a labeled fader so users can balance them against the song from one mixer popover in the player controls.
This is the slopsmith#87 contract.
## Registration
```js
function _registerFader() {
const api = window.slopsmith && window.slopsmith.audio;
if (!api) return;
api.registerFader({
id: 'my_plugin', // unique key
label: 'My Plugin', // shown above the fader
unit: 'dB', // optional suffix shown next to the value (e.g. '%', 'dB')
min: 0, max: 2, step: 0.05,
defaultValue: 1.0,
getValue: () => _myCurrentVolume, // read current value
setValue: (v) => _setMyVolume(v), // write + persist + apply
});
}
if (window.slopsmith && window.slopsmith.audio) {
_registerFader();
} else {
window.addEventListener('slopsmith:audio:ready', _registerFader, { once: true });
}
```
## Contract
- **Persistence is the plugin's responsibility.** The registry calls `getValue()` when the popover opens and after each `setValue()` during slider drags to re-sync the displayed value.
- **`getValue()` must be cheap and side-effect-free.**
- **`setValue()` must update whatever backing state `getValue()` reads synchronously** — pair it with whatever your plugin already does internally (write the GainNode, persist to `localStorage`, update any in-plugin label).
- **Use `unregisterFader(id)`** when your plugin is teardown-able and you want the strip to disappear; otherwise keep it registered so the user's setting persists across toggle states.
## Lifecycle event
`slopsmith:audio:ready` fires once on `window` when the audio mixer registry is ready. Guard registration against this event for plugins that load before the mixer.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-visualization-contracts.md](plugin-visualization-contracts.md) — visualization counterpart
docs/plugin-diagnostics.md
+86
View File
@@ -0,0 +1,86 @@
# Plugin diagnostics contribution
Slopsmith ships an **Export Diagnostics** feature (Settings → Export Diagnostics) that bundles a redacted set of host + plugin state into a zip the user can share with maintainers. Plugins have three independent opt-ins for contributing to this bundle.
The bundle layout and per-file schemas are documented in [diagnostics-bundle-spec.md](diagnostics-bundle-spec.md). This doc covers how plugins integrate.
## 1. `manifest.diagnostics.server_files` — opt-in file capture
Declare files under `context["config_dir"]` to copy verbatim into the bundle:
```json
{
"diagnostics": {
"server_files": ["my_plugin.diag.json", "my_plugin_models/active.json"]
}
}
```
Rules (same as `settings.server_files`):
- Relpaths only. No `..`, no abs paths, no backslashes, no leading dots.
- Files land at `plugins/<plugin_id>/<relpath>` inside the bundle.
- Encoded as `{"encoding": "json", "data": <parsed>}` if `.json` parses cleanly, base64 otherwise.
Use this for **snapshot-style state** — small DB excerpts, model lists, last-error files. Don't use it for backups (that's `settings.server_files`).
## 2. `manifest.diagnostics.callable` — opt-in dynamic capture
Declare a Python entry point that produces diagnostics at export time:
```json
{
"diagnostics": {
"callable": "diagnostics:collect"
}
}
```
The form is `"<module>:<function>"`. The module is resolved lazily via `load_sibling` when the user clicks Export, then called as:
```python
# plugins/my_plugin/diagnostics.py
def collect(ctx):
"""ctx is {"plugin_id": str, "config_dir": Path}."""
return {
"schema": "my_plugin.diag.v1",
"active_preset": _read_active_preset(ctx["config_dir"]),
"model_count": len(list((ctx["config_dir"] / "models").glob("*.pt"))),
}
```
Return-type handling:
- `dict` / `list` → written to `plugins/<id>/callable.json`
- `bytes` → `callable.bin`
- `str` → `callable.txt`
Exceptions are caught and appended to the bundle's `manifest.notes` — a buggy plugin never crashes the export.
## 3. Frontend contribution (slopsmith#166)
Plugins that hold useful debug state in the browser (active model name, last user input, internal counters) can push it into the bundle from `screen.js`:
```js
window.slopsmith.diagnostics.contribute('my_plugin', {
schema: 'my_plugin.client_diag.v1',
active_preset: getActivePreset(),
last_error: _lastError,
});
```
- **Idempotent.** Repeated calls overwrite the previous value.
- Whatever was last contributed before the user hits Export Diagnostics is what lands in `plugins/<plugin_id>/client.json`.
- Available namespace: `window.slopsmith.diagnostics.{contribute, snapshotConsole, snapshotHardware, snapshotUa, snapshotLocalStorage, snapshotContributions}`.
- Loaded from `static/diagnostics.js` ASAP in `<head>` so the console-wrap is in place before any other script runs.
## Best practices
- **Embed a `schema` field** (e.g. `"my_plugin.diag.v1"`) in JSON returned by `callable` so future tooling can dispatch by version.
- **Keep payloads small (< 100 KB).** Diagnostics are not a backup channel — that's `settings.server_files`.
- **Don't include secrets, API keys, or session tokens.** Bundles are shared with maintainers / posted to GitHub issues.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [diagnostics-bundle-spec.md](diagnostics-bundle-spec.md) — full bundle layout + per-file schemas
- [plugin-manifest.md](plugin-manifest.md) — declaring `diagnostics` fields
- [plugin-logging.md](plugin-logging.md) — `log.exception()` writes traceback into the diagnostics log capture
docs/plugin-keyboard-shortcuts.md
+64
View File
@@ -0,0 +1,64 @@
# Plugin keyboard shortcuts
Plugins can register keyboard shortcuts via the global `window.registerShortcut()` function. Shortcuts appear in the `?` help panel.
## Registration
```js
window.registerShortcut({
key: 'k', // key value (e.key) or key code (e.code)
description: 'Toggle my view', // shown in the help panel
scope: 'player', // 'global' | 'player' | 'library' | 'settings' | 'plugin-{id}'
condition: () => _isMyViewActive, // optional guard
handler: (e) => _myAction() // called when shortcut triggers
});
```
## Scope
Scope controls when the shortcut is active:
- **`global`** — works on any screen
- **`player`** — only on the player screen
- **`library`** — only on the home/favorites screens
- **`settings`** — only on the settings screen
- **`plugin-{id}`** — only when your plugin's screen is active
## Panel-scoped shortcuts
For plugins that create multiple panels (e.g., splitscreen), shortcuts are automatically scoped to the active panel. Use `const panel = window.createShortcutPanel(id)` to create a panel (keep the returned reference so you can call `panel.clearShortcuts()` during cleanup) and `window.setActiveShortcutPanel(id)` to switch between them. Each panel has its own shortcut registry, so multiple panels can have the same key without collisions.
## Condition
`condition` is an optional guard function. If it returns false, the shortcut is skipped even if in scope.
## Key matching
The handler matches against both `e.key` (character produced) and `e.code` (physical key). Use `e.key` for letters/symbols that depend on keyboard layout, and `e.code` for special keys (e.g. `Space`, `ArrowLeft`).
## Cleanup
Clean up with `window.unregisterShortcut(key, scope)`. **You must pass the same scope you registered with** — the default is `'global'` and won't match `player`/`library`/`settings`/`plugin-*` bindings.
For panel-scoped shortcuts, prefer `panel.clearShortcuts()` over per-key unregister calls.
## Built-in shortcuts
| Key | Description |
|-----|-------------|
| `?` | Show keyboard shortcuts panel (global) |
| `Space` | Play/Pause (player only) |
| `←` / `→` | Seek ±5 seconds (player only) |
| `Escape` | Back to library (player only) |
| `[` / `]` | Audio offset ±10ms (Shift: ±50ms) (player only) |
Don't override these. The `?` panel is the canonical reference for users.
## Debugging
Open the browser console and type `_listShortcuts()` to inspect every registered shortcut, its scope, and its source plugin.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [testing-plugins.md](testing-plugins.md) — Playwright tests for shortcut behaviour
docs/plugin-logging.md
+67
View File
@@ -0,0 +1,67 @@
# Plugin logging
Use `context["log"]` for all backend plugin output. It is a stdlib `logging.Logger` namespaced to `slopsmith.plugin.<id>`, pre-configured with the app-wide level, format (including JSON mode), and correlation IDs. **Never use `print()`** — it bypasses correlation context and log rotation.
## Backend usage
```python
def setup(app, context):
log = context["log"]
log.info("plugin ready")
log.warning("optional dependency %r not found, feature disabled", dep)
try:
risky_init()
except Exception:
log.exception("unhandled error during setup") # auto-captures traceback
```
## CLI entry-point fallback
For helper scripts that also run as `__main__`, add a stdlib fallback so the logger works without the server pipeline:
```python
if __name__ == "__main__":
import logging
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
```
## How it's wired
The plugin loader assigns each plugin a logger before calling `setup()`:
```python
context["log"] = logging.getLogger(f"slopsmith.plugin.{plugin_id}")
```
That logger inherits from the root `slopsmith` logger, which is configured by `logging_setup.configure_logging()` (called from `main.py` before uvicorn boots). It respects:
- **`LOG_LEVEL`** — `DEBUG | INFO | WARNING | ERROR` (default `INFO`)
- **`LOG_FORMAT`** — `json | text` (default `text` — coloured console)
- **`LOG_FILE`** — optional path for a persistent log file (e.g. `/config/slopsmith.log`)
- **Correlation IDs** — each HTTP request carries `X-Request-ID`; structlog injects it into every log line during that request's lifetime. WebSocket sessions get their own `ws_conn_id` contextvar bound at accept time.
## Verifying
Look for your plugin's logger name in the console output:
```
INFO slopsmith.plugin.my_plugin: plugin ready
```
Switch `LOG_FORMAT=json` to see structured output:
```json
{"timestamp": "...", "level": "info", "logger": "slopsmith.plugin.my_plugin", "event": "plugin ready", "request_id": "..."}
```
## Common mistakes
- **Using `print()` inside `setup()` or request handlers.** `print()` goes to stdout, bypasses level filtering, log rotation, JSON formatting, and correlation IDs. Audit your plugin with `grep -nR 'print(' plugins/<id>/`.
- **Creating a new logger with `logging.getLogger("my_plugin")`.** This creates an unparented logger that doesn't inherit slopsmith's configuration. Always use the one passed via `context["log"]`.
- **Logging at `INFO` in hot paths.** Default `LOG_LEVEL` is `INFO`, so `log.info(...)` inside a per-frame or per-message loop floods output. Use `log.debug(...)` for hot-path tracing.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-manifest.md](plugin-manifest.md) — the `routes.py` / `setup(app, context)` contract
- [plugin-diagnostics.md](plugin-diagnostics.md) — capturing logs into the diagnostics bundle
docs/plugin-manifest.md
+138
View File
@@ -0,0 +1,138 @@
# `plugin.json` manifest reference
Every plugin lives in `plugins/<name>/` and must declare a `plugin.json` manifest. JSON Schema for this format ships at [`schema/plugin.schema.json`](../schema/plugin.schema.json) and is enforced in CI for in-tree plugins.
## Full example
```json
{
"id": "my_plugin",
"name": "My Plugin",
"version": "1.0.0",
"private": false,
"type": "visualization",
"nav": { "label": "My Plugin", "screen": "plugin-my_plugin" },
"screen": "screen.html",
"script": "screen.js",
"routes": "routes.py",
"settings": {
"html": "settings.html",
"server_files": ["my_plugin.db", "my_plugin_models/"]
},
"diagnostics": {
"server_files": ["my_plugin.diag.json"],
"callable": "diagnostics:collect"
}
}
```
All fields except `id` and `name` are optional. Plugins can have any combination of frontend (screen/script), backend (routes), and settings.
## Fields
### `id` (required, string)
Snake-case identifier. Used to namespace `localStorage` keys, build the plugin's screen id (`plugin-<id>`), namespace the backend logger (`slopsmith.plugin.<id>`), and as the directory name in diagnostics bundles. Cannot contain slashes, dots are encoded by the sibling-import loader (see [plugin-sibling-imports.md](plugin-sibling-imports.md)).
### `name` (required, string)
Human-readable name shown in UI surfaces.
### `version` (string, optional)
Plain semver string. Advisory only — the plugin loader does not consume this. Plugins commonly include it for publishing/tooling purposes.
### `private` (boolean, optional)
Advisory metadata for plugin authors. Not consumed by the loader.
### `type` (string, optional — role hint, slopsmith#36)
Supported values:
- `"visualization"` — plugin provides a highway renderer. Declaring this makes the plugin eligible for the main-player viz picker AND splitscreen's per-panel picker. Must pair with a `window.slopsmithViz_<id>` factory exporting the setRenderer contract (see [plugin-visualization-contracts.md](plugin-visualization-contracts.md)).
- Absent → no declared role; plugin is loaded and its script runs, but it doesn't appear in role-specific UIs.
### `nav` (object, optional)
`{ "label": string, "screen": string }` — adds a navbar entry that calls `showScreen(<screen>)`. `screen` is typically `plugin-<id>`.
### `screen` (string, optional)
Path to HTML file (relative to plugin dir). Mounted at `#plugin-<id>` in the SPA.
### `script` (string, optional)
Path to JS file (relative to plugin dir). Loaded via `<script>` tag in global scope. Wrap in an IIFE.
### `routes` (string, optional)
Path to Python file exporting `setup(app, context)`. See "Backend routes" below.
### `settings` (object, optional)
`{ "html": string, "server_files": string[] }`
- **`settings.html`** — settings-panel HTML.
- **`settings.server_files`** — **opt-in** for the unified Settings export/import flow (slopsmith#113). List of relpaths under `context["config_dir"]` that the plugin wants included in user-triggered backups. A trailing `/` denotes a directory (recurse).
Rules:
- Relpaths only. Absolute paths, drive letters, `..` segments, and backslashes are rejected at load time with a `[Plugin]` warning.
- The same allowlist is consulted at both export and import: a bundle that references a file the *importing host*'s manifest no longer declares is skipped with a warning (handles plugin updates between export and import). A bundle that references a file your host's manifest never declared is also skipped — no surprise writes.
- Files are encoded as `{"encoding": "json", "data": <parsed>}` for `.json` files that parse cleanly (diff-friendly), `{"encoding": "base64", "data": "..."}` otherwise (sqlite, model blobs, IRs).
- Plugins own their internal data migration. Importing a bundle whose data schema predates your current code restores bytes verbatim — your plugin must cope at next load.
- Symlinks are skipped on export and never followed on import.
Plugins that omit this field have no server-side files exported; their state lives entirely in browser `localStorage`, which is bundled wholesale on every export.
### `diagnostics` (object, optional)
`{ "server_files": string[], "callable": string }`
**Opt-in** for the troubleshooting bundle (slopsmith#166 — Settings → Export Diagnostics). Two independent fields:
- **`diagnostics.server_files`** — same allowlist semantics as `settings.server_files`: relpaths under `context["config_dir"]`, no `..`, no abs paths, no backslashes, no leading dots. Files listed here are copied verbatim into `plugins/<plugin_id>/<relpath>` inside the bundle. Use this for snapshot-style state (small DB excerpts, model lists, last-error files).
- **`diagnostics.callable`** — `"<module>:<function>"` (e.g. `"diagnostics:collect"`). Resolved lazily via `load_sibling` when the user clicks Export, then called as `func({"plugin_id": "...", "config_dir": Path(...)})`. Return `dict`/`list` → written to `plugins/<id>/callable.json`; `bytes` → `callable.bin`; `str` → `callable.txt`. Exceptions are caught and appended to the bundle's `manifest.notes` — a buggy plugin never crashes the export.
See [plugin-diagnostics.md](plugin-diagnostics.md) for full diagnostics integration patterns.
### `license` (string, optional but recommended)
SPDX identifier. For curated plugins, must be AGPL-3.0-or-later or AGPL-compatible (MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0). See [CONTRIBUTING.md](../CONTRIBUTING.md).
## Backend routes — `setup(app, context)`
`routes.py` must export `setup(app, context)`. The `context` dict provides:
- `config_dir` — persistent config path (`Path`)
- `get_dlc_dir()` — returns the DLC folder `Path`
- `extract_meta()` — metadata extraction callable
- `meta_db` — shared `MetadataDB` instance
- `get_sloppak_cache_dir()` — sloppak cache `Path`
- `load_sibling(name)` — loads a sibling module from this plugin's directory under a unique, namespaced module name. See [plugin-sibling-imports.md](plugin-sibling-imports.md).
- `log` — stdlib `logging.Logger` namespaced to `slopsmith.plugin.<id>`. Pre-configured with the app-wide level, format (including JSON mode), and correlation IDs. Use this for all backend plugin output instead of `print()`. See [plugin-logging.md](plugin-logging.md).
Example:
```python
def setup(app, context):
log = context["log"]
extractor = context["load_sibling"]("extractor")
@app.get("/api/my_plugin/status")
def status():
return {"ready": True}
log.info("my_plugin ready")
```
## Validation
Run the local validator skill `/plugin-validate` (Claude Code) or the CI workflow `.github/workflows/validate-plugins.yml`. Both consume [`schema/plugin.schema.json`](../schema/plugin.schema.json).
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-logging.md](plugin-logging.md) — `context["log"]` pattern
- [plugin-sibling-imports.md](plugin-sibling-imports.md) — `load_sibling`
- [plugin-diagnostics.md](plugin-diagnostics.md) — diagnostics opt-in details
- [diagnostics-bundle-spec.md](diagnostics-bundle-spec.md) — full diagnostics bundle layout
docs/plugin-sibling-imports.md
+48
View File
@@ -0,0 +1,48 @@
# Plugin sibling imports — `load_sibling`
When a backend plugin spans multiple Python files, **use `context["load_sibling"]`** rather than bare `import` statements. This is the slopsmith#33 contract.
## The problem
The plugin loader inserts each plugin's directory onto `sys.path` so `from extractor import X` works. But Python caches imports by **module name** in `sys.modules`. Two plugins that each ship a top-level `extractor.py` (or any other generic name — `util.py`, `client.py`, `parser.py`, `config.py`, …) collide: whichever loads first wins, and the other plugin's `from extractor import X` either gets the wrong module or fails with `cannot import name 'X' from 'extractor'`.
## The fix
`context["load_sibling"](name)` loads the sibling under a namespaced module name `plugin_<id>.<name>`, so each plugin gets its own copy. The `<id>` portion is bijectively encoded so reverse-DNS-style ids like `com.example.foo` work without colliding: `_` → `_5f_`, `.` → `_2e_`.
```python
def setup(app, context):
extractor = context["load_sibling"]("extractor")
PsarcReader = extractor.PsarcReader
# …
```
## Notes
- **`name` is a bare module name** — no `.py` suffix, no slashes, no `.`. The helper raises `ValueError` for path traversal / format issues and `ImportError` for missing files.
- **Both single-file siblings (`extractor.py`) and package-form siblings (`extractor/__init__.py`) work.** Package form wins when both exist (matches CPython's import-resolution precedence).
- **Relative imports between siblings work** — `from .shared import X` in a top-level helper, `from ..shared import X` from inside a sibling package. The synthetic parent package `plugin_<id>` carries the plugin directory in its `__path__`.
- **`from . import sibling` (attribute-style) also resolves**: loaded children are exposed as attributes on the parent package.
- **Repeat calls return the cached module.** Concurrent first-time calls are serialized via per-module locks so no caller observes a half-initialized module.
- **Don't mix `load_sibling` and bare `import` for the same module** — they'd execute the file twice and split module-level state.
## Migration
Bare `import sibling` from `routes.py` still works during the transition period, but the loader prints a startup warning when it detects two plugins shipping a same-named top-level module — covering both `.py` files and package directories. Migrate to `load_sibling` to silence the warning and immunize your plugin from future ecosystem collisions.
## Verification
The collision-detection logic is encoded in `tests/test_plugins.py`. Run:
```bash
pytest tests/test_plugins.py -v
```
If you're authoring a plugin with a top-level helper, add a test that imports your plugin under a `reset_plugin_state` fixture to confirm clean import behaviour. See [testing-plugins.md](testing-plugins.md).
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-manifest.md](plugin-manifest.md) — declaring `routes.py`
- [plugin-logging.md](plugin-logging.md) — `context["log"]` (also exposed via `context`)
- [testing-plugins.md](testing-plugins.md) — `reset_plugin_state` fixture
docs/plugin-visualization-contracts.md
+198
View File
@@ -0,0 +1,198 @@
# Plugin visualization contracts
Slopsmith supports two ways for a plugin to participate in the main player's visuals. They coexist; the **setRenderer** contract is the default for any viz that draws a highway-shaped surface, and **overlays** handle layered decorations on top. A third orthogonal contract — **note-state provider** — lets scorers feed per-note judgments back into whichever renderer is active.
**Pick the right shape:**
- Replacing the whole highway drawing on the existing highway canvas (your renderer owns its rendering context / resources; `createHighway()` still owns the canvas element and the rAF loop)? → **setRenderer** (section 1). Enters the viz picker. Works in both the main player and per-panel under splitscreen.
- Adding a layer on top of whichever viz is active? → **Overlay** (section 2). Navbar toggle, not in the picker.
- Scoring plugin that wants renderers to "light up" notes on correct hits? → **Note-state provider** (section 3). Orthogonal — can coexist with overlay HUDs.
## 1. setRenderer contract (slopsmith#36) — preferred
Plugins that want to replace the main highway's draw function (per panel, per session) export a renderer factory on `window.slopsmithViz_<id>` where `<id>` matches the `id` in `plugin.json` (`type: "visualization"` required). The factory returns an object matching this shape:
```js
window.slopsmithViz_my_viz = function () {
return {
// Required canvas context type. Default '2d' if omitted.
// highway.js reads this BEFORE calling init() so it can
// replace the underlying <canvas> element if the current
// one is locked to a different context type (see "Canvas
// context-type swapping" below).
contextType: '2d', // or 'webgl2'
init(canvas, bundle) {
// One-time setup. Own your getContext() call here —
// acquire '2d' or 'webgl2' depending on the renderer.
// The canvas you receive is guaranteed to either be
// unbound or already bound to your declared contextType.
this.ctx = canvas.getContext('2d');
},
draw(bundle) {
// Called each requestAnimationFrame tick by the factory.
// `bundle` is a snapshot with: currentTime, songInfo, isReady,
// notes, chords, anchors (all difficulty-filter-aware),
// beats, sections, chordTemplates, stringCount, lyrics,
// toneChanges, toneBase, mastery, hasPhraseData, inverted,
// lefty, renderScale, lyricsVisible, the 2D coordinate
// helpers project and fretX, and getNoteState (see below).
// `stringCount` is the active arrangement's string count (4
// for bass, 6 for guitar, 7+ for extended-range GP imports —
// size string-indexed geometry against this, not a hardcoded
// 6). If your renderer needs lefty-aware text rendering, check
// bundle.lefty and apply the mirror transform yourself —
// a bundle-level helper isn't provided because it would
// need your renderer's own context, not the factory's.
//
// bundle.getNoteState(note, chartTime) (slopsmith#254) — call
// this per visible chart note / chord-note to find out whether
// a scorer (note_detect) has flagged it 'hit' / 'active' (a
// sustain currently being held correctly) / 'miss', so the gem
// itself can light up / a held sustain can glow instead of
// relying on an overlay ring. Returns null when no provider is
// registered or it reports nothing for this note; otherwise
// { state: 'hit'|'active'|'miss', alpha: 0..1, color: string|null }.
// For chord notes pass the chord's time (note_detect keys its
// judgments by `${time}_${string}_${fret}`). 'hit' and 'active'
// are both "lit" — a renderer may treat them identically; the
// provider owns all fade timing via `alpha` and by simply
// ceasing to return state when the effect should end.
},
resize(w, h) {
// Optional. Canvas dims already updated; re-create WebGL
// framebuffers / reset 2D transforms here.
},
destroy() {
// Optional. Release resources, remove DOM nodes, null refs.
// Called before setRenderer() swaps to another renderer
// and on highway.stop().
},
};
};
```
Selecting this plugin in the main-player viz picker — or in splitscreen's per-panel picker — calls `highway.setRenderer(factory())` on the existing highway instance. The built-in 2D highway is the default renderer and is restored by passing nullish — `setRenderer(null)` and `setRenderer(undefined)` both work (the implementation gates on `r == null`). Splitscreen panels create one `createHighway()` per panel and each independently consults the picker, so N panels can run different renderers (or N copies of the same renderer with different arrangements) without coordination.
### Lifecycle contract
The factory returns a single renderer instance that may go through multiple `init() → ... → destroy()` cycles as the user navigates between songs or screens. Specifically:
- `init(canvas, bundle)` runs when the highway has a canvas and the renderer takes over drawing. This is when to acquire `getContext()`, build shaders / meshes / DOM nodes, and register listeners.
- `draw(bundle)` runs on every rAF frame once the WebSocket `ready` message has fired and until the renderer is replaced or the highway stops. It is **not** called during the loading / reconnect window (between `api.init()` + `stop()` and the next `ready`) — that would hand the renderer half-populated chart arrays. Renderers that want to show a "loading" state can read `bundle.isReady` inside a future-widened contract, but today the factory gates `draw` behind the ready flag and `isReady` is only informational once it does fire.
- `destroy()` runs when the renderer is replaced via another `setRenderer(...)` call, OR when `highway.stop()` is called (e.g. the user navigates away from the player). It releases everything `init()` acquired.
- **After `destroy()`, the same instance may receive another `init()` call** — this happens on `playSong()` which does `stop()` → `init()` to reuse the same canvas element for the next song. Renderers must tolerate `init()` being called again on an instance that was previously destroyed. Practically: null your refs in destroy, re-acquire them in init.
- `destroy()` is skipped when it would run on an un-init'd renderer — if a caller does `setRenderer(x)` before the highway ever init'd (possible when restoring a saved picker selection at page load), `x.destroy()` is not called until `x.init()` has run at least once.
- `resize(w, h)` is optional; runs after init and whenever the canvas dimensions change.
### Key rules
- The factory **returns a fresh object on each call** — important for splitscreen, where multiple panels will each get an independent instance.
- The renderer **owns its own rendering context** (2D or WebGL). Factory will not call getContext for you.
- **Canvas context-type swapping.** Browsers lock a `<canvas>` to the first context type successfully acquired for its lifetime: once `getContext('2d')` succeeds, `getContext('webgl2')` on that same canvas returns `null`, and vice versa. To let arbitrary 2D ⇄ WebGL renderer swaps work mid-session, `highway.setRenderer()` reads the next renderer's `contextType` before calling its `init()` and, if it differs from the type currently bound, replaces the underlying `<canvas>` element with a fresh one via `oldCanvas.cloneNode(false)` followed by `oldCanvas.replaceWith(newCanvas)`. The factory then calls the renderer's `init(newCanvas, bundle)` with the fresh element so its `getContext()` succeeds. Practical implications:
- **What survives the swap.** `cloneNode(false)` preserves *every HTML attribute* on the element — `id`, `class`, inline `style`, all `data-*` and `aria-*` attributes, `role`, `tabindex`, the attribute form of `width`/`height`, and anything else a plugin attached. DOM position is preserved by `replaceWith()`, so siblings, parents, and surrounding layout are unaffected.
- **What does NOT survive.** Event listeners attached via `addEventListener` are NOT cloned, and expando properties set imperatively on the JavaScript object (such as the bound rendering context, or any `canvas._myPlugin = …`-style data a plugin attached) are not carried over either. The bound rendering context being left behind on the detached element is exactly what allows the new canvas to start fresh and accept a different `getContext()` call. Note: `canvas.width`/`canvas.height` *are* reflected HTML attributes, so those values do survive the clone; `api.resize()` re-applies the backing-store dimensions on the new element after the swap regardless.
- Renderers must **declare `contextType`** on the returned instance (`'2d'` or `'webgl2'`; absent → `'2d'`). Factories may also expose it as a static (`window.slopsmithViz_<id>.contextType = 'webgl2'`) so core can read it before constructing the renderer — used today by Auto-mode evaluation.
- Plugins that hold a stale reference to the highway canvas across renderer swaps — including any code that registered listeners directly on the canvas element rather than on `window`/`document` — should listen for the `highway:canvas-replaced` event on `window.slopsmith` and re-acquire / re-register. `window.slopsmith.emit` dispatches a `CustomEvent`, so the payload `{ oldCanvas, newCanvas, contextType }` lives on `event.detail`, not on the event object itself:
```js
window.slopsmith.on('highway:canvas-replaced', (event) => {
const { oldCanvas, newCanvas, contextType } = event.detail;
// re-acquire / re-register against newCanvas
});
```
Plugins that re-query `document.getElementById('highway')` lazily inside their own event handlers don't need this listener — they pick up the new element automatically (it keeps `id="highway"`).
- **`highway:visibility`** — fired on `window.slopsmith` whenever the highway canvas transitions between displayed and hidden. Detection is DOM-based via `canvas.offsetParent === null` (catches `display:none` on the canvas or any ancestor — e.g. splitscreen's `#highway` hide) or whatever a host explicitly sets via `highway.setVisible(bool)`. While `visible === false`, core skips the rAF `renderer.draw(bundle)` call AND the default 2D draw, so renderers don't have to no-op themselves. The event is emitted only on transitions (including the first one after `init()`), not every frame. Payload `{ visible, canvas }` lives on `event.detail`:
```js
window.slopsmith.on('highway:visibility', (event) => {
const { visible, canvas } = event.detail;
// Toggle any sibling DOM your renderer mounts. The 3D Highway
// renderer hides its `.h3d-wrap` overlay here so `display:none`
// on `#highway` actually hides the visible output.
});
```
Renderers that only paint to the slopsmith canvas don't need this listener — the rAF skip is enough. Renderers that mount sibling DOM (separate WebGL contexts, overlays, etc.) do.
- **`highway.setVisible(bool | null)`** — forces the visibility state regardless of `offsetParent`. Pass `null` to clear the override and resume DOM-based detection. Useful when the host hides the highway via `visibility:hidden`, `opacity:0`, transforms, or clipping rather than `display:none`. The override re-emits any resulting transition immediately rather than waiting for the next rAF tick.
- Default-renderer ctx is closure-cached. The replace path nulls the closure ctx so stale draw paths short-circuit; the next default-renderer `init()` re-acquires the 2D context from the new canvas cleanly.
- `draw(bundle)` receives difficulty-filtered arrays — never read from `_filteredNotes` or other internals.
- `_drawHooks` fire for the default 2D renderer (the factory calls them at the end of each frame). Custom WebGL renderers that maintain a 2D overlay canvas (like the bundled 3D highway) also call `window.highway.fireDrawHooks(ctx, W, H)` on that overlay so overlay plugins continue to work regardless of which renderer is active. Custom renderers without a 2D overlay context should not attempt to fire hooks.
### Auto mode — `matchesArrangement(songInfo)` (optional)
The viz picker prepends an "Auto (match arrangement)" entry that is the default selection on fresh installs. When Auto is active, core evaluates registered viz factories on every `song:ready` and swaps the renderer to the first factory whose `matchesArrangement(songInfo)` predicate returns truthy. No match → the built-in 2D highway.
Declare the predicate as a static on the factory (not the instance) so core can evaluate it without constructing a throwaway renderer:
```js
window.slopsmithViz_piano = function () { /* ... */ };
window.slopsmithViz_piano.matchesArrangement = function (songInfo) {
return /keys|piano|synth/i.test((songInfo && songInfo.arrangement) || '');
};
```
- `songInfo` is the highway's live song_info snapshot — `arrangement`, `tuning`, `capo`, `arrangement_index`, `filename`, `artist`, `title`, etc. May be `{}` before the first song loads.
- Factories without `matchesArrangement` are skipped during auto-selection — the correct default for arrangement-agnostic viz (tabview, jumpingtab) that only make sense as manual picks.
- Explicit picker selections override Auto and are persisted to `localStorage.vizSelection`, so the pinned choice survives page reloads until the user switches back to "Auto" (which also persists). Picking "Auto" re-evaluates against the current song immediately. In contexts where `localStorage` is unavailable (private mode, sandboxed iframes, some test runners) persistence falls back to the current picker `<option>` value, which still overrides Auto for as long as the page stays loaded.
- When an Auto-selected renderer fails and core emits `viz:reverted`, the picker falls back to the built-in default and disables auto-switching until the user re-selects Auto.
- First match wins (picker order), so the registration order of plugins is the tiebreaker. Keep predicates narrow to avoid stealing songs from more specialized viz.
**WebGL viz in Auto mode.** Auto evaluation runs on every `song:ready` regardless of which renderer is active. Auto-installing a WebGL renderer when the canvas is currently 2D — or reverting from a WebGL Auto pick to the default 2D — works without a reload because `setRenderer` swaps the canvas element when `contextType` differs (see "Canvas context-type swapping" above). WebGL viz can therefore safely declare `matchesArrangement` and rely on Auto. For 3D Highway specifically, `_canRun3D()` in app.js still gates Auto from picking it on machines without WebGL2 — that fallback is independent of canvas swapping.
## 2. Overlay contract — for add-on layers
Plugins that add a layer on top of whichever visualization is active — HUDs, fretboard diagrams, chord labels, practice feedback — don't replace the renderer. They manage their own canvas, their own rAF loop, and a toggle button somewhere visible (typically a navbar pill), reading public highway state via the getters:
- `highway.getTime()` / `highway.getBeats()` — current playback position
- `highway.getNotes()` / `highway.getChords()` — difficulty-filter-aware arrays
- `highway.getChordTemplates()` — chord shape lookup table; index by `chord.id` from `getChords()` to get `{ name, fingers, frets }`. `fingers` and `frets` are per-string arrays (length matches the tuning's string count); within `fingers`, `-1` = unused, `0` = open string, `n > 0` = finger number. RS XML sources populate real fingerings; GP imports currently emit all `-1` since pre-import sources don't carry finger data. Not filter-aware: templates are static metadata, every `chord_id` referenced by `getChords()` is guaranteed valid.
- `highway.getSongInfo()` — tuning, arrangement, capo
- `highway.getStringCount()` — number of strings on the active arrangement (4 for bass, 6 for guitar, 7+ for extended-range GP imports). Derived server-side as `max(notes-max-string + 1, name-based fallback, len(tuning))` where the tuning length only contributes when it isn't the RS-XML padded 6-string form (sloppak / GP-imported sources carry trimmed tuning lengths). The name-based fallback is 4 for arrangements containing "bass" (case-insensitive) and 6 otherwise. This combination handles partial-string-usage charts (a 6-string lead that never plays string 5), extended-range GP imports (5-string bass, 7-string guitar), and sloppaks that explicitly encode the instrument range — without requiring plugins to do their own arrangement-name matching.
- `highway.getLefty()` / `highway.getInverted()` — mirror + invert state
Overlays do NOT appear in the viz picker and do NOT declare `"type": "visualization"` in `plugin.json`. They coexist with whichever renderer (default 2D, 3D highway, piano, ...) the user has picked.
### Key rules
- **Own your rAF + canvas** — don't piggyback on `_drawHooks` or on `createHighway`'s rendering context. Draw hooks fire for the default 2D renderer and for custom renderers that explicitly call `window.highway.fireDrawHooks(ctx, W, H)` (e.g. the bundled 3D highway fires them on its 2D overlay canvas), but not for every custom renderer.
- **Re-read state every frame** — overlay output must track whatever the current renderer is drawing. Don't cache note positions across frames.
- **Respect lefty + invert toggles** — if the overlay depicts strings or frets, mirror using the same transforms the active renderer would.
- **If you position with `highway.project` / `highway.fretX` (the 2D-highway geometry), gate on `highway.isDefaultRenderer()`** — those helpers describe the *built-in 2D* highway's depth curve and fret zoom. When a custom renderer (3D highway, piano, …) is active your draw hook still fires (on that renderer's 2D overlay layer), but those coordinates won't match its scene — markers land in arbitrary places. Skip rendering when `isDefaultRenderer()` is false; the custom renderer owns that feedback. Renderer-agnostic overlays (fretboard diagram, chord-label HUD — they use `getNotes()`/`getChordTemplates()` + their own layout) don't need this guard.
- **Clean up on toggle-off** — cancel rAF and remove/hide the overlay canvas so inactive overlays aren't wasting frames.
Reference: [fretboard plugin](https://github.com/byrongamatos/slopsmith-plugin-fretboard) — canonical overlay implementation (navbar toggle, own canvas, 80ms active-note window).
### Why two contracts?
setRenderer plugs into an existing highway — main-player or splitscreen-panel — reusing its WebSocket and data parsing, so the common "I want a different look for the same data" case is zero boilerplate AND multi-instance for free. Overlays compose with whatever renderer is active — they decorate rather than replace, so multiple can stack (fretboard + chord labels + practice feedback) without fighting over the canvas.
A previous standalone-pane contract (`window.createMyVisualization({ container })` with its own WebSocket per pane) was used by splitscreen pre-Wave-C. It's been retired now that splitscreen calls `setRenderer` on per-panel `createHighway()` instances; if you find references in older plugin docs or external integration guides, those describe the legacy path.
## 3. Note-state provider — for scorers that want renderers to "light up" notes (slopsmith#254)
A scoring plugin (note_detect) can publish a per-note judgment so whichever renderer is active draws the **gem itself** lit on a correct hit, and keeps a sustain trail glowing while it's still being played correctly — instead of a separate overlay ring floating near the note.
```js
// In the plugin (after resolving the highway instance):
highway.setNoteStateProvider((note, chartTime) => {
// `note` is the chart note object ({ t, s, f, sus, ... }); for chord
// notes `chartTime` is the chord's time. Return one of:
// - falsy → no special state (render normally)
// - 'hit' — struck correctly; renderer lights the gem
// - 'active' — a sustained note is right now being held correctly
// - 'miss' — missed; renderer may red-wash the gem
// - { state: <one of the above>, alpha?: 0..1, color?: '#rrggbb' }
// You own all fade timing: return a decaying `alpha` for a struck-note
// glow, `alpha: 1` (or a bare string) for a held sustain, and stop
// returning state when the effect should end. Keep it cheap — it's
// called per visible note per renderer per frame.
});
// On teardown: highway.setNoteStateProvider(null);
```
- Only one provider is active at a time (last `setNoteStateProvider` wins). `highway.getNoteStateProvider()` returns the current one (or null).
- The built-in 2D highway consults it in `drawNote` / `drawSustains` / the chord-frame path: 'hit'/'active' → bright string colour + additive halo + a contained "sizzle" (crackling sparks, throbbing core, a shockwave ring on a fresh strike) on the gem and a bright (vs dim) sustain trail; 'miss' → faint red wash. The bundled **3D highway** reads the same data via `bundle.getNoteState` (bright string-tinted outline + bright body + glowing sustain + a contained sparkle hugging the note rect on hit/active; red outline + suppressed body on miss). Custom renderers that want it call `bundle.getNoteState(note, chartTime)` — it null-guards and returns the normalized `{ state, alpha, color }` (or null).
- This is orthogonal to the overlay contract: note_detect remains an overlay (HUD, diagnostic miss markers, the "currently detected" indicator) *and* a scorer that feeds this provider. A renderer that ignores `getNoteState` simply doesn't light gems — nothing breaks.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-manifest.md](plugin-manifest.md) — declaring `"type": "visualization"`
- [websocket-protocol.md](websocket-protocol.md) — the WebSocket stream that drives `bundle`
- [plugin-audio-mixer.md](plugin-audio-mixer.md) — companion contract for audio-producing plugins
docs/testing-plugins.md
+104
View File
@@ -0,0 +1,104 @@
# Testing plugins
Slopsmith has three test surfaces — Python unit/integration tests (pytest), JS plugin-API contract tests (Node), and end-to-end browser tests (Playwright). This doc covers what to use when.
## Test layout
```
tests/
├── conftest.py Shared pytest fixtures (isolate_logging)
├── test_plugins.py Plugin loader + load_sibling + collision tests (includes reset_plugin_state)
├── test_song.py Wire-format serialization round-trips
├── test_*.py Per-feature backend tests
├── js/ Node --test JS plugin-API contract tests
└── browser/ Playwright end-to-end tests
```
## Running tests
```bash
pytest # All Python tests
pytest tests/test_plugins.py -v # Specific file
pytest -k "load_sibling" -v # Pattern match
npm run test:js # Node-native JS contract tests
npm run install:playwright # One-time: install Chromium
npm test # Playwright browser tests
npm run test:headed # Playwright with visible browser
npm run test:debug # Playwright inspector
```
CI runs `pytest` on every push/PR to `main` (Python 3.12).
## Shared fixtures
### `isolate_logging` (in `conftest.py`)
Saves and restores handlers, level, and `propagate` flag on the `slopsmith`, `uvicorn`, `uvicorn.error`, and `uvicorn.access` loggers, plus calls `structlog.reset_defaults()`. **Import into any test module that calls `configure_logging()`** so mutations don't bleed across tests.
```python
def test_my_log_thing(isolate_logging):
configure_logging()
# ... assertions
```
### `reset_plugin_state` (in `test_plugins.py`)
Local fixture used by plugin-loader tests. Saves and restores:
- `plugins.LOADED_PLUGINS`
- any `plugin_*` keys in `sys.modules`
- the bare names the tests simulate (`util`, `extractor`) in `sys.modules`
- `sys.path` (the loader mutates it)
Also unsets `SLOPSMITH_PLUGINS_DIR` for the test's duration via `monkeypatch` so a CI env that pre-sets it can't leak real user plugins into a tmp-path-driven test.
If you're authoring a plugin that ships top-level helper modules, model new tests on the patterns in `test_plugins.py` — use `reset_plugin_state` to guarantee a clean import slate.
## Backend test patterns
- **Wire-format round-trips.** `test_song.py` is the model: pure serialization tests, no fixtures, narrative docstring. Pattern: build a `Song`/`Arrangement`/`Note`, serialize, deserialize, assert equal.
- **Loader behaviour.** `test_plugins.py` uses tmp-path-driven plugin roots + `reset_plugin_state` + `_make_plugin` / `_run_load_plugins` helpers. Adopt these helpers for any new loader-touching tests.
- **Async / WebSocket.** Tests for FastAPI WebSocket endpoints use `httpx.AsyncClient` and `app.websocket_connect()`. See `test_audio.py` and `test_highway_3d_routes.py` for examples.
## JS contract tests (`tests/js/`)
Plain `node --test` files. No browser, no server. Cover the plugin-API surface as exposed on `window.slopsmith` and related globals. Run with `npm run test:js`.
Use these when a plugin needs to assert against the API contract without spinning up a real browser. They run in seconds; CI can grow these without slowing.
## Browser tests (`tests/browser/`)
Playwright + Chromium. `playwright.config.ts` runs serially (`workers: 1`) and boots a real Slopsmith instance via Docker Compose. Traces / video on failure.
What's currently covered: page load and keyboard shortcuts. **No plugin E2E patterns yet** — if you're adding the first one for your plugin, set the precedent. Suggested structure:
```ts
import { test, expect } from '@playwright/test';
test('my plugin loads its navbar entry', async ({ page }) => {
await page.goto('/');
await expect(page.getByRole('link', { name: 'My Plugin' })).toBeVisible();
});
test('my plugin shortcut fires', async ({ page }) => {
await page.goto('/#plugin-my_plugin');
await page.keyboard.press('k');
await expect(page.getByTestId('my-plugin-toggle')).toHaveAttribute('aria-pressed', 'true');
});
```
Prefer `data-testid` over text-based selectors so refactors don't break tests.
## Debugging
- **Browser console shortcut inventory.** `_listShortcuts()` prints every registered shortcut, its scope, and source.
- **`LOG_FORMAT=json pytest`** — get machine-readable test logs.
- **`pytest -s`** — disable output capture (handy for `print()`-style debug, though prefer `log.debug()` in production code).
- **Playwright trace viewer** — `npx playwright show-trace test-results/.../trace.zip` after a failure.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-logging.md](plugin-logging.md) — `context["log"]` and why `print()` breaks the logging pipeline
- [plugin-sibling-imports.md](plugin-sibling-imports.md) — `load_sibling` and the collision tests
- [plugin-keyboard-shortcuts.md](plugin-keyboard-shortcuts.md) — `window.registerShortcut` and `_listShortcuts()`
docs/websocket-protocol.md
+44
View File
@@ -0,0 +1,44 @@
# Highway WebSocket protocol reference
The highway WebSocket at `/ws/highway/{filename}?arrangement={index}` streams a song's chart data to the player. Plugins that drive their own highway, replace the renderer (see [plugin-visualization-contracts.md](plugin-visualization-contracts.md)), or consume the stream directly all read these frames.
## Message order
Each connection receives the following JSON frames, roughly in this order:
| Message | Shape | Description |
|---------|-------|-------------|
| `loading` | `{ type: 'loading', stage }` | Status/progress message during extraction or conversion. |
| `song_info` | `{ type, title, artist, arrangement, arrangement_index, arrangements, duration, tuning, capo, format, audio_url, audio_error, stems }` | Song metadata. `arrangements` is the full list for the switcher. `audio_url` is `null` when audio is unavailable, in which case `audio_error` is non-null; otherwise `audio_error` is `null`. `stems` is always present — an empty array for non-sloppak songs or sloppak songs with no split stems. `tuning` is an array (6 for guitar, 4 for bass). |
| `beats` | `{ type, data: [{ time, measure }] }` | Beat timestamps with measure numbers. |
| `sections` | `{ type, data: [{ time, name }] }` | Named sections (Intro, Verse, Chorus, etc.). |
| `anchors` | `{ type, data: [{ time, fret, width }] }` | Fret zoom anchors. |
| `chord_templates` | `{ type, data: [{ name, frets: [6] }] }` | Named chord shapes. |
| `lyrics` | `{ type, data: [{ w, t, d }] }` | Syllables: `w`=word, `t`=time, `d`=duration. `-` joins to previous, `+` = line break. |
| `tone_changes` | `{ type: 'tone_changes', base, data: [{ time, name }] }` | Optional — tone change events relative to the arrangement base tone; only sent if tones were found. |
| `notes` | `{ type, data: [{ t, s, f, sus, ho, po, sl, bn, ... }] }` | Single notes. |
| `chords` | `{ type, data: [{ t, notes: [{ s, f, sus, ... }] }] }` | Chord events. |
| `phrases` | `{ type, data: [{ start_time, end_time, max_difficulty, levels: [...] }], total }` | Optional — per-phrase difficulty ladder for master-difficulty slider (slopsmith#48). Only sent when the source chart carries multi-level phrase data (PSARC / phrase-aware sloppak). Sent in chunks (`data` is a batch, `total` is the full count across messages) to avoid multi-MB single frames. Absent for GP imports and legacy sloppak; consumers must treat missing message as "single fixed difficulty — slider disabled". |
| `ready` | `{ type: 'ready' }` | All data sent — safe to finalize and start rendering. |
## Delivery guarantees
Message delivery is **incremental**. You may receive `loading` updates and `lyrics` before note/chord payloads. `tone_changes` comes after `lyrics` when present and may be omitted entirely. **Do not finalize rendering until you receive `ready`.**
## Consumer notes
- **Tuning array length follows arrangement.** 6 strings for guitar, 4 for bass, more for extended-range GP imports. Use `highway.getStringCount()` for the authoritative count rather than `tuning.length` (which can be the RS-XML padded 6-string form).
- **Chord templates are static.** Every `chord_id` referenced by `chords` is guaranteed to be present in `chord_templates`.
- **Notes carry technique fields.** `ho` = hammer-on, `po` = pull-off, `sl` = slide target, `bn` = bend amount. The full set is defined in `lib/song.py`.
- **Multiple connections are supported.** Split-screen panels, lyrics panes, and jumping-tab panes each open their own WebSocket. By design — don't try to multiplex.
## Phrase payload (slopsmith#48)
`phrases.data` is an array of phrase objects. Each phrase has `start_time`, `end_time`, `max_difficulty`, and `levels`. Each `level` has `difficulty`, `notes`, `chords`, `anchors`, `handshapes` — fully scoped to that phrase. The master-difficulty slider applies a single difficulty across all phrases by selecting the matching level.
## Related
- [PLUGIN_AUTHORING.md](PLUGIN_AUTHORING.md) — guide index
- [plugin-visualization-contracts.md](plugin-visualization-contracts.md) — bundle shape passed to `draw()`
- [sloppak-spec.md](sloppak-spec.md) — sloppak format these messages are derived from
- `lib/song.py` — server-side `Note`, `Chord`, `Arrangement`, `Song` data models
schema/plugin.schema.json
+132
View File
@@ -0,0 +1,132 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/byrongamatos/slopsmith/blob/main/schema/plugin.schema.json",
"title": "Slopsmith plugin manifest",
"description": "Schema for plugins/<id>/plugin.json. See docs/plugin-manifest.md for prose. License enum mirrors the curated-plugin allowlist in CONTRIBUTING.md (keep them in sync; tests/test_plugin_schema.py asserts subset).",
"type": "object",
"additionalProperties": false,
"required": ["id", "name"],
"properties": {
"id": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)*$",
"description": "Snake-case identifier. Used to namespace localStorage, the plugin screen id (plugin-<id>), the backend logger (slopsmith.plugin.<id>), and the diagnostics bundle directory. Reverse-DNS form (com.example.foo) is supported via the load_sibling encoding."
},
"name": {
"type": "string",
"minLength": 1,
"description": "Human-readable name shown in UI surfaces."
},
"version": {
"type": "string",
"description": "Plain semver string. Advisory only; the plugin loader does not consume this."
},
"private": {
"type": "boolean",
"description": "Advisory metadata for plugin authors. Not consumed by the loader."
},
"bundled": {
"type": "boolean",
"description": "True for plugins shipped in-tree with Slopsmith. Used by tooling that distinguishes built-in plugins from user-installed ones."
},
"type": {
"type": "string",
"enum": ["visualization"],
"description": "Role hint (slopsmith#36). 'visualization' makes the plugin eligible for the viz picker; must pair with a window.slopsmithViz_<id> factory."
},
"license": {
"type": "string",
"description": "SPDX identifier. For curated plugins, must match CONTRIBUTING.md's allowlist.",
"enum": [
"AGPL-3.0-only",
"AGPL-3.0-or-later",
"GPL-3.0-only",
"GPL-3.0-or-later",
"LGPL-3.0-only",
"LGPL-3.0-or-later",
"MIT",
"BSD-2-Clause",
"BSD-3-Clause",
"Apache-2.0",
"ISC",
"Unlicense",
"CC0-1.0",
"0BSD"
]
},
"nav": {
"type": "object",
"additionalProperties": false,
"required": ["label", "screen"],
"properties": {
"label": { "type": "string", "minLength": 1 },
"screen": { "type": "string", "pattern": "^[a-z][a-z0-9_-]*$" }
},
"description": "Adds a navbar entry that calls showScreen(<screen>). Typically screen is 'plugin-<id>'."
},
"screen": {
"type": "string",
"description": "Relative path to HTML mounted at #plugin-<id>."
},
"script": {
"type": "string",
"description": "Relative path to JS loaded in global scope on page load. Wrap your code in an IIFE."
},
"routes": {
"type": "string",
"description": "Relative path to Python module exporting setup(app, context). See docs/plugin-manifest.md."
},
"tour": {
"type": "string",
"description": "Relative path to tour JSON for the in-app onboarding tour."
},
"settings": {
"type": "object",
"additionalProperties": false,
"properties": {
"html": {
"type": "string",
"description": "Relative path to settings-panel HTML."
},
"server_files": {
"type": "array",
"items": {
"type": "string",
"not": { "pattern": "^/|^[a-zA-Z]:|\\\\|(^|/)\\.\\.(/|$)" },
"description": "Relpath under context['config_dir']. No abs paths, no '..', no backslashes. Trailing '/' denotes a directory (recurse)."
},
"uniqueItems": true,
"description": "Opt-in for Settings export/import (slopsmith#113). Files included in user-triggered backups. See docs/plugin-manifest.md."
}
}
},
"diagnostics": {
"type": "object",
"additionalProperties": false,
"properties": {
"server_files": {
"type": "array",
"items": {
"type": "string",
"not": { "pattern": "^/|^[a-zA-Z]:|\\\\|(^|/)\\.\\.(/|$)" }
},
"uniqueItems": true,
"description": "Files copied verbatim into plugins/<id>/<relpath> inside the diagnostics bundle."
},
"callable": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*:[A-Za-z_][A-Za-z0-9_]*$",
"description": "<module>:<function> resolved via load_sibling at export time. Called with {'plugin_id', 'config_dir'} dict; return dict/list/bytes/str."
}
},
"description": "Opt-in for the Export Diagnostics bundle (slopsmith#166). See docs/plugin-diagnostics.md."
}
},
"allOf": [
{
"$comment": "type=visualization requires a script (the window.slopsmithViz_<id> factory)",
"if": { "properties": { "type": { "const": "visualization" } }, "required": ["type"] },
"then": { "required": ["script"] }
}
]
}