mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-09-11 20:54:29 +00:00
Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4b8ec441b6 | ||
|
|
3b862ba117 | ||
|
|
cbc65458e3 | ||
|
|
7c87538d6b | ||
|
|
c8701991cb | ||
|
|
514461167e | ||
|
|
0dcc9136b6 | ||
|
|
6da01c55a4 |
+2
-1
@@ -26,7 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
next root-level module can't ship broken.
|
||||
|
||||
### Added
|
||||
- **`routers/` — extracting `server.py`'s route layer, cheapest-first (R3).** Each PR moves a cohesive route group into a `fastapi.APIRouter` under `lib/routers/`, mounted with `app.include_router(...)` at its original site (FastAPI matches in registration order; the full route table stays byte-identical). Bodies are verbatim — only the decorator receiver (`@app` → `@router`) and singleton reads (`meta_db` → `appstate.meta_db`, resolved at call time) change. So far: `audio_effects` (5), `artist_aliases` (5), `loops` (3), `playlists` (12 + custom covers). The `config_dir` path constant now rides the `appstate` seam (env-derived, so the pop-and-reimport fixtures reconfigure it for free), and the shared request-field sanitizer `_clean_str` moved to `lib/reqfields.py`. The next cut is picked by a dependency-closure scan that ranks groups by how many `monkeypatch.setattr(server, …)` targets they'd drag along.
|
||||
- **Perf harness now measures 2D-highway frame time (R3c gate).** `scripts/perf-baseline.mjs` gains a `--song` mode that reports per-frame draw-cost p50/p95/p99 (draw-tagged via `highway.addDrawHook`), the metric that gates the `highway.js` split. Maintainer/CI-only; baseline recorded in `docs/perf-baseline.md`.
|
||||
- **`routers/` — extracting `server.py`'s route layer, cheapest-first (R3).** Each PR moves a cohesive route group into a `fastapi.APIRouter` under `lib/routers/`, mounted with `app.include_router(...)` at its original site (FastAPI matches in registration order; the full route table stays byte-identical). Bodies are verbatim — only the decorator receiver (`@app` → `@router`) and singleton reads (`meta_db` → `appstate.meta_db`, resolved at call time) change. So far: `audio_effects` (5), `artist_aliases` (5), `loops` (3), `playlists` (12 + covers), `ws_highway` (the 902-line highway chart WebSocket), `chart` (split/unsplit/work/fileinfo — unblocked by the DLC-path substrate). The DLC library-path resolution (`_get_dlc_dir`, pure `_resolve_dlc_path`) moved to `lib/dlc_paths.py`, reading paths through the seam; `config_dir`/`dlc_dir`/`dlc_dir_env` now ride the `appstate` seam (env-derived, so the pop-and-reimport fixtures reconfigure it for free), and the shared request-field sanitizer `_clean_str` moved to `lib/reqfields.py`. The next cut is picked by a dependency-closure scan that ranks groups by how many `monkeypatch.setattr(server, …)` targets they'd drag along.
|
||||
- **`routers/` — the first extracted route module (R3).** The five audio-effects mapping
|
||||
endpoints move out of `server.py` into `lib/routers/audio_effects.py` as a
|
||||
`fastapi.APIRouter`, mounted with `app.include_router(...)` **at the point in the file
|
||||
|
||||
@@ -37,6 +37,31 @@ the hooks, they just need real songs in `DLC_DIR`.
|
||||
|
||||
## Results
|
||||
|
||||
### R3c pre-lift baseline — 2026-07-10 (2D highway draw cost)
|
||||
|
||||
The gate for the `highway.js` split. Captured on a seeded library (the 33 MB
|
||||
Arcturus feedpak) with the new `--song` mode, which measures **per-frame draw
|
||||
cost** — rAF callbacks are tagged via `highway.addDrawHook`, so only frames the
|
||||
highway actually painted count (the other ~half are cheap no-op loops that would
|
||||
otherwise mask a regression). Any `highway.js` change must re-run this on the
|
||||
same machine and stay within noise of these numbers.
|
||||
|
||||
```bash
|
||||
node scripts/perf-baseline.mjs --base http://127.0.0.1:8300 \
|
||||
--song "Arcturus - The Sham Mirrors - Kinetic.feedpak"
|
||||
```
|
||||
|
||||
| run | draw frames | p50 | p95 | p99 | max |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | 53/106 | 2.2 | 3.2 | 3.6 | 3.6 |
|
||||
| 2 | 50/100 | 2.2 | 2.9 | 3.2 | 3.2 |
|
||||
| 3 | 53/106 | 2.1 | 2.7 | 3.5 | 3.5 |
|
||||
|
||||
**p50 ≈ 2.2 ms · p95 spread 2.7–3.2 ms** (3 runs × 10 s playback, headless
|
||||
chromium on the dev box). The `H`-container lift changes each closure-slot read
|
||||
to a `H.<slot>` property load; this is the number that proves it doesn't cost the
|
||||
hot loop.
|
||||
|
||||
### R0 baseline — 2026-07-08 (branch `feat/r0-plugin-module-rails`)
|
||||
|
||||
> ⚠️ A quick capture (`--n 50 --soak 8`) against an **empty** library (no charts
|
||||
|
||||
@@ -55,8 +55,8 @@ without a *signed* exemption" is unenforceable.
|
||||
## Planned, NOT exempt (owned by split plans — listed so nothing falls between states)
|
||||
|
||||
core `static/app.js` (11,852) · `static/highway.js` (4,168, whole file) · `server.py`
|
||||
(9,085 — was 14,037; ratcheted by the R3 `MetadataDB` + `AudioEffectsMappingDB`
|
||||
extractions and four `routers/` modules) ·
|
||||
(7,880 — was 14,037; ratcheted by the R3 `MetadataDB` + `AudioEffectsMappingDB`
|
||||
extractions and seven `routers/` modules) ·
|
||||
`lib/metadata_db.py` (4,373 — new in R3; the `MetadataDB` class alone is 4,018 lines
|
||||
and is a monolith in its own right, to be split per-table once the router train
|
||||
lands) · `static/v3/songs.js` (4,134) · `static/capabilities/audio-session.js`
|
||||
|
||||
+16
-1
@@ -70,8 +70,23 @@ audio_effect_mappings = None
|
||||
# `setattr(server, …)` in a few tests, so those slots (when added) need their
|
||||
# tests retargeted to appstate in the same PR.
|
||||
config_dir = None
|
||||
dlc_dir = None # the DLC_DIR env value as a Path (Path("") if unset)
|
||||
dlc_dir_env = None # the raw DLC_DIR env string, "" if unset — distinguishes
|
||||
# "unset" from Path("")→"." (see dlc_paths._get_dlc_dir)
|
||||
# Cache/asset dirs. static_dir + sloppak_cache_dir are patched via
|
||||
# `setattr(server, …)` in a few tests, so a router reading them here needs those
|
||||
# setattr sites retargeted to `setattr(appstate, …)` in the same PR (ws_highway
|
||||
# retargets the 3 test_highway_ws_* SLOPPAK sites). config_dir-derived dirs are
|
||||
# reconfigured for free on a setenv+reimport.
|
||||
static_dir = None
|
||||
sloppak_cache_dir = None
|
||||
audio_cache_dir = None
|
||||
|
||||
_SLOTS = frozenset({"meta_db", "audio_effect_mappings", "config_dir"})
|
||||
_SLOTS = frozenset({
|
||||
"meta_db", "audio_effect_mappings",
|
||||
"config_dir", "dlc_dir", "dlc_dir_env",
|
||||
"static_dir", "sloppak_cache_dir", "audio_cache_dir",
|
||||
})
|
||||
|
||||
|
||||
def configure(**kwargs) -> None:
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
"""DLC library path resolution — where the song files live, plus safe containment.
|
||||
|
||||
Extracted from ``server.py`` (R3). ``_resolve_dlc_path`` is pure and moved
|
||||
verbatim. ``_get_dlc_dir`` reads the env-derived paths through the ``appstate``
|
||||
seam (``server.py`` configures ``dlc_dir``/``dlc_dir_env``/``config_dir`` at
|
||||
import, fresh on every re-import), so this module does no import-time IO and the
|
||||
pop-and-reimport fixtures keep working. ``server.py`` re-exports both names, so
|
||||
existing ``server._get_dlc_dir`` / ``server._resolve_dlc_path`` references
|
||||
(tests, other handlers) resolve unchanged.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import appstate
|
||||
|
||||
|
||||
def _get_dlc_dir(cfg: dict | None = None) -> Path | None:
|
||||
# Only consider DLC_DIR if the env var was non-empty. `Path("")` collapses
|
||||
# to `.` and reports `.is_dir() == True`, which would silently shadow the
|
||||
# config.json fallback. Checking the raw env string preserves
|
||||
# `DLC_DIR=.` as a valid opt-in for cwd while keeping unset/empty out.
|
||||
if appstate.dlc_dir_env and appstate.dlc_dir.is_dir():
|
||||
return appstate.dlc_dir
|
||||
if cfg is None:
|
||||
config_file = appstate.config_dir / "config.json"
|
||||
if config_file.exists():
|
||||
try:
|
||||
cfg = json.loads(config_file.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
pass
|
||||
if isinstance(cfg, dict):
|
||||
raw = str(cfg.get("dlc_dir", "")).strip()
|
||||
if raw:
|
||||
p = Path(raw)
|
||||
if p.is_dir():
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
def _resolve_dlc_path(dlc: Path, filename: str) -> Path | None:
|
||||
"""Resolve `filename` under DLC_DIR and refuse anything that escapes.
|
||||
|
||||
`filename` arrives from `:path` route params and can contain `..`
|
||||
segments. The Sloppak and archive paths happen to fail safely later
|
||||
because their loaders raise on missing/invalid files, but loose-
|
||||
folder format detection (`is_loose_song`) globs and parses XML on
|
||||
disk first, which lets a crafted path trigger filesystem reads
|
||||
outside DLC_DIR before any guard fires. Centralise the containment
|
||||
check so every filename-bound handler validates before touching the
|
||||
filesystem.
|
||||
|
||||
Containment here is LEXICAL (normalize `.`/`..` WITHOUT following
|
||||
symlinks), not `safe_join`'s `.resolve()`-based check — because users
|
||||
commonly mount their song library through a directory JUNCTION/symlink
|
||||
(a library shared across app installs; the desktop app's own mounts).
|
||||
`.resolve()` follows that junction to its real target, sees it sits
|
||||
outside DLC_DIR, and wrongly rejects every song reached through it — the
|
||||
scanner's `rglob` indexes those songs, but art/load then 403/404s (broken
|
||||
covers, unplayable songs). Lexical normalization still rejects the only
|
||||
escapes a `:path` filename can express — `..` traversal and absolute
|
||||
paths — which the traversal tests pin. `safe_join` stays strict (it is
|
||||
the zip-slip / plugin-asset guard, where following a symlink out IS the
|
||||
defense); the loose-folder art handler keeps its own per-file symlink
|
||||
re-check for defence-in-depth.
|
||||
|
||||
Returns the validated Path (not necessarily link-resolved), or None if
|
||||
the filename is empty, contains a NUL, or escapes the DLC root.
|
||||
"""
|
||||
if not filename:
|
||||
return None
|
||||
# Backslashes → forward slashes so a Windows-style `..\\x` traversal is
|
||||
# rejected identically on POSIX (mirrors safe_join's normalisation).
|
||||
safe = filename.replace("\\", "/")
|
||||
if "\x00" in safe:
|
||||
return None
|
||||
# Reject drive-letter / absolute paths in BOTH conventions. A POSIX "/x" is
|
||||
# caught by the containment check below (the `/` operator discards `root`),
|
||||
# but a Windows drive-absolute "C:/x" is treated as a relative "C:" dir on
|
||||
# POSIX and would otherwise slip in as `<root>/C:/x` — so the contract must
|
||||
# hold cross-platform (a shared library is reached from either OS).
|
||||
from pathlib import PurePosixPath, PureWindowsPath
|
||||
if (PurePosixPath(safe).is_absolute()
|
||||
or PureWindowsPath(safe).is_absolute()
|
||||
or PureWindowsPath(safe).drive):
|
||||
return None
|
||||
try:
|
||||
root = dlc.resolve()
|
||||
# normpath collapses `.`/`..`/duplicate separators purely lexically —
|
||||
# it never touches the filesystem, so an in-library junction component
|
||||
# is preserved (allowed) while `..`/absolute segments still escape and
|
||||
# get caught by the containment check below.
|
||||
candidate = Path(os.path.normpath(root / safe))
|
||||
if not candidate.is_relative_to(root):
|
||||
return None
|
||||
except (ValueError, OSError):
|
||||
return None
|
||||
return candidate
|
||||
@@ -0,0 +1,115 @@
|
||||
"""Chart-level endpoints — split/unsplit a chart from its work, resolve work
|
||||
membership, and the context-menu "Get info" file inspector.
|
||||
|
||||
Extracted verbatim from ``server.py`` (R3); edits: ``@app`` -> ``@router``,
|
||||
``meta_db`` -> ``appstate.meta_db``. DLC path resolution comes from
|
||||
``dlc_paths``; sloppak/loose detection from the shared lib modules.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException
|
||||
|
||||
import appstate
|
||||
from dlc_paths import _get_dlc_dir, _resolve_dlc_path
|
||||
import sloppak as sloppak_mod
|
||||
import loosefolder as loosefolder_mod
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/api/chart/{filename:path}/split")
|
||||
def api_split_chart(filename: str):
|
||||
"""'These aren't the same song' — split this chart out as its own singleton
|
||||
work. Under /api/chart (NOT /api/song) so the DELETE /api/song/{path}
|
||||
catch-all can't shadow it."""
|
||||
key = appstate.meta_db._canonical_song_filename(filename)
|
||||
appstate.meta_db.split_chart(key)
|
||||
return {"ok": True, "filename": key}
|
||||
|
||||
|
||||
@router.post("/api/chart/{filename:path}/unsplit")
|
||||
def api_unsplit_chart(filename: str):
|
||||
"""Undo a split — rejoin the chart to its work."""
|
||||
key = appstate.meta_db._canonical_song_filename(filename)
|
||||
appstate.meta_db.unsplit_chart(key)
|
||||
return {"ok": True, "filename": key}
|
||||
|
||||
|
||||
@router.get("/api/chart/{filename:path}/work")
|
||||
def api_get_chart_work(filename: str):
|
||||
"""Resolve a chart's work membership: {work_key, chart_count}. For openers
|
||||
on rows that came from an ungrouped query (the tree view) — grouped grid
|
||||
rows already carry both fields inline."""
|
||||
return appstate.meta_db.chart_work(filename)
|
||||
|
||||
|
||||
@router.get("/api/chart/{filename:path}/fileinfo")
|
||||
def api_chart_fileinfo(filename: str):
|
||||
"""The context menu's "Get info": where the file lives + what the pack
|
||||
contains. Under /api/chart — the GET /api/song/{path} catch-all would
|
||||
swallow a /api/song/…/fileinfo suffix. Read-only; demo-mode blocks it
|
||||
because it exposes filesystem paths."""
|
||||
dlc = _get_dlc_dir()
|
||||
if not dlc:
|
||||
raise HTTPException(status_code=404, detail="not configured")
|
||||
p = _resolve_dlc_path(dlc, filename)
|
||||
if p is None:
|
||||
raise HTTPException(status_code=403, detail="forbidden")
|
||||
if not p.exists():
|
||||
raise HTTPException(status_code=404, detail="not found")
|
||||
# Restrict to actual charts — sloppak or loose song. Without this the route
|
||||
# would stat ANY file the user happens to keep under DLC_DIR (e.g. notes),
|
||||
# leaking its path/size; the app only recognises these two song formats.
|
||||
is_pak = sloppak_mod.is_sloppak(p)
|
||||
is_loose = loosefolder_mod.is_loose_song(p)
|
||||
if not (is_pak or is_loose):
|
||||
raise HTTPException(status_code=404, detail="not a chart")
|
||||
st = p.stat()
|
||||
info = {
|
||||
"filename": filename,
|
||||
"path": str(p),
|
||||
"folder": str(p.parent),
|
||||
"format": "sloppak" if is_pak else "loose",
|
||||
# Directory-form songs report the tree's total (covers loose folders
|
||||
# and dir-form paks); zip-form paks report the archive size. Symlinked
|
||||
# entries are skipped so a link inside the folder can't pull in — or
|
||||
# leak the size of — a file outside it.
|
||||
"size": (st.st_size if p.is_file()
|
||||
else sum(f.stat().st_size for f in p.rglob("*")
|
||||
if f.is_file() and not f.is_symlink())),
|
||||
"mtime": st.st_mtime,
|
||||
}
|
||||
if is_pak:
|
||||
try:
|
||||
m = sloppak_mod.load_manifest(p) or {}
|
||||
except Exception:
|
||||
m = {}
|
||||
arrs = [str(a.get("name", a.get("id", ""))) for a in (m.get("arrangements") or [])
|
||||
if isinstance(a, dict)]
|
||||
stems = [str(s.get("id", "")) for s in (m.get("stems") or []) if isinstance(s, dict)]
|
||||
try:
|
||||
has_cover = sloppak_mod.read_cover_bytes(p, m) is not None
|
||||
except Exception:
|
||||
has_cover = False
|
||||
# The optional identity/catalog keys, listed only when present — the
|
||||
# Get-info panel's "what this pack carries vs what's missing" readout.
|
||||
identity = {k: m.get(k) for k in
|
||||
("mbid", "isrc", "genres", "track", "disc", "album_artist",
|
||||
"feedpak_version", "language")
|
||||
if m.get(k) not in (None, "", [])}
|
||||
info["manifest"] = {
|
||||
"title": str(m.get("title", "")), "artist": str(m.get("artist", "")),
|
||||
"album": str(m.get("album", "")), "year": str(m.get("year", "") or ""),
|
||||
"arrangements": arrs, "stems": stems,
|
||||
"has_cover": has_cover, "has_lyrics": bool(m.get("lyrics")),
|
||||
"authors": [a.get("name", "") if isinstance(a, dict) else str(a)
|
||||
for a in (m.get("authors") or [])],
|
||||
"identity": identity,
|
||||
}
|
||||
# The enrichment verdict, so Get info can say "Matched (auto, 96%)" /
|
||||
# "Pinned by you" / "Not matched" alongside the file facts.
|
||||
row = appstate.meta_db.get_enrichment(filename)
|
||||
if row:
|
||||
info["match"] = {k: row.get(k) for k in
|
||||
("match_state", "match_source", "match_score",
|
||||
"canon_artist", "canon_title", "canon_album", "canon_year")}
|
||||
return info
|
||||
@@ -6,6 +6,9 @@ Extracted verbatim from ``server.py`` (R3). Edits: ``@app`` -> ``@router``,
|
||||
``reqfields``. See ``appstate.py``.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter
|
||||
@@ -14,6 +17,8 @@ from fastapi.responses import FileResponse, JSONResponse
|
||||
import appstate
|
||||
from reqfields import _clean_str
|
||||
|
||||
log = logging.getLogger("feedBack.server")
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Cache policy for the custom-cover file response: revalidate every time so a
|
||||
@@ -204,15 +209,40 @@ async def api_set_playlist_cover(pid: int, data: dict):
|
||||
return JSONResponse({"error": "Invalid base64"}, status_code=400)
|
||||
cover = _playlist_cover_path(pid)
|
||||
cover.parent.mkdir(parents=True, exist_ok=True)
|
||||
# Decode/validate the image — a bad payload is a CLIENT error (400), and the
|
||||
# message stays generic so it can't echo internals.
|
||||
try:
|
||||
from PIL import Image
|
||||
img = Image.open(io.BytesIO(img_data)).convert("RGB")
|
||||
img.thumbnail((640, 640)) # covers stay small
|
||||
tmp = cover.with_suffix(".png.tmp")
|
||||
img.save(str(tmp), "PNG")
|
||||
except Exception:
|
||||
return JSONResponse({"error": "Invalid image"}, status_code=400)
|
||||
# Persist. A save/replace failure is a SERVER error (500, logged, no
|
||||
# filesystem detail leaked) — the pre-split handler mislabeled these as 400
|
||||
# and echoed the exception. A unique temp name in the cover dir (not a shared
|
||||
# `{pid}.png.tmp`) means two concurrent uploads can't clobber each other's
|
||||
# temp file; the atomic replace publishes. Re-check the playlist still exists
|
||||
# just before publishing so a delete that raced the decode above can't leave
|
||||
# an orphan cover — cheap belt-and-braces; FeedBack is single-user
|
||||
# (Principle I), so a full per-playlist lock would be for a race the
|
||||
# deployment model precludes.
|
||||
tmp = None
|
||||
try:
|
||||
# mkstemp is inside the try too: an unwritable dir / full disk raises
|
||||
# here, and that's the same class of persistence failure as save/replace.
|
||||
fd, tmp_name = tempfile.mkstemp(prefix=f".{pid}.", suffix=".png.tmp", dir=str(cover.parent))
|
||||
tmp = Path(tmp_name)
|
||||
with os.fdopen(fd, "wb") as f:
|
||||
img.save(f, "PNG")
|
||||
if appstate.meta_db.get_playlist(pid) is None:
|
||||
tmp.unlink(missing_ok=True)
|
||||
return JSONResponse({"error": "not found"}, status_code=404)
|
||||
tmp.replace(cover)
|
||||
except Exception as e:
|
||||
return JSONResponse({"error": f"Invalid image: {e}"}, status_code=400)
|
||||
except Exception:
|
||||
if tmp is not None:
|
||||
tmp.unlink(missing_ok=True)
|
||||
log.exception("playlist cover save failed (pid=%s)", pid)
|
||||
return JSONResponse({"error": "could not save cover"}, status_code=500)
|
||||
return {"ok": True, "cover_url": _playlist_cover_url(pid)}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
"""Wishlist / "wanted" API (feedBack#636) — songs the user wants but doesn't own.
|
||||
|
||||
Extracted verbatim from ``server.py`` (R3); edits: ``@app`` -> ``@router``,
|
||||
``meta_db`` -> ``appstate.meta_db``, ``_clean_str`` from ``reqfields``.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
import appstate
|
||||
from reqfields import _clean_str
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/api/wanted")
|
||||
def api_list_wanted():
|
||||
"""The wishlist — songs the user wants but doesn't own yet (newest first)."""
|
||||
return {"wanted": appstate.meta_db.list_wanted()}
|
||||
|
||||
|
||||
@router.post("/api/wanted")
|
||||
def api_add_wanted(data: dict):
|
||||
"""Add a not-owned song to the wishlist. `artist`/`title` are required (at
|
||||
least one non-empty); `source`/`source_ref`/`note` are optional. Idempotent
|
||||
on identity so producers (find_more ownership-diff, manual add) can re-post."""
|
||||
if not isinstance(data, dict):
|
||||
return JSONResponse({"error": "body must be an object"}, status_code=400)
|
||||
artist = _clean_str(data.get("artist"))
|
||||
title = _clean_str(data.get("title"))
|
||||
if not artist and not title:
|
||||
return JSONResponse({"error": "artist or title required"}, status_code=400)
|
||||
row = appstate.meta_db.add_wanted(
|
||||
artist=artist, title=title,
|
||||
source=_clean_str(data.get("source")) or "manual",
|
||||
source_ref=_clean_str(data.get("source_ref")),
|
||||
note=_clean_str(data.get("note")),
|
||||
)
|
||||
return {"ok": True, "wanted": row}
|
||||
|
||||
|
||||
@router.delete("/api/wanted/{wanted_id}")
|
||||
def api_remove_wanted(wanted_id: int):
|
||||
"""Remove a wishlist entry by id."""
|
||||
return {"ok": appstate.meta_db.remove_wanted(wanted_id)}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -6,6 +6,16 @@
|
||||
//
|
||||
// Usage:
|
||||
// node scripts/perf-baseline.mjs --base http://127.0.0.1:8000 [--n 60] [--soak 30]
|
||||
// node scripts/perf-baseline.mjs --base http://127.0.0.1:8300 --song "Arcturus ….feedpak"
|
||||
//
|
||||
// With --song, it additionally measures the 2D highway's PER-FRAME DRAW cost:
|
||||
// it wraps requestAnimationFrame before any page script runs, tags the frames
|
||||
// in which the highway actually painted (via highway.addDrawHook), starts
|
||||
// playback, and reports draw-frame p50/p95/p99 over --frames seconds. Tagging
|
||||
// matters — roughly half the rAF callbacks belong to other cheap loops, and
|
||||
// averaging them in hides a renderer regression behind ~0.1 ms no-op frames.
|
||||
// This is the metric that gates the highway.js split (R3c); run it before AND
|
||||
// after each highway change on the same machine.
|
||||
//
|
||||
// Maintainer/CI-only dev tooling (uses the committed @playwright/test browser);
|
||||
// never part of the serve or Docker path. Metrics that need a seeded library
|
||||
@@ -21,6 +31,9 @@ for (let i = 2; i < process.argv.length; i += 2) args.set(process.argv[i].replac
|
||||
const BASE = args.get('base') || 'http://127.0.0.1:8000';
|
||||
const N = parseInt(args.get('n') || '60', 10);
|
||||
const SOAK_S = parseInt(args.get('soak') || '30', 10);
|
||||
const SONG = args.get('song') || null;
|
||||
const FRAME_S = parseInt(args.get('frames') || '10', 10);
|
||||
const RUNS = parseInt(args.get('runs') || '3', 10); // repeat frame sampling to show spread
|
||||
|
||||
const pct = (xs, p) => {
|
||||
if (!xs.length) return null;
|
||||
@@ -69,6 +82,64 @@ async function clientMetrics() {
|
||||
return { bootMs, memStartMB: mem0 && mem0 / 1048576, memSoakMB: mem1 && mem1 / 1048576, scripts };
|
||||
}
|
||||
|
||||
// ── 2D highway per-frame draw cost (needs --song + a seeded library) ──────────
|
||||
async function frameTimeOnce(browser) {
|
||||
const page = await browser.newPage();
|
||||
let f, t2, notes;
|
||||
try {
|
||||
// Wrap rAF before any page script; mark frames the highway actually drew.
|
||||
await page.addInitScript(() => {
|
||||
window.__f = [];
|
||||
window.__drew = false;
|
||||
const raf = window.requestAnimationFrame.bind(window);
|
||||
window.requestAnimationFrame = (cb) => raf((t) => {
|
||||
window.__drew = false;
|
||||
const t0 = performance.now();
|
||||
try { cb(t); } finally { window.__f.push({ ms: performance.now() - t0, drew: window.__drew }); }
|
||||
});
|
||||
});
|
||||
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 60000 });
|
||||
await page.evaluate(() => document.getElementById('v3-onboarding')?.remove());
|
||||
await page.evaluate((s) => window.playSong(s), SONG);
|
||||
await page.waitForFunction(() => (window.highway?.getNotes?.() || []).length >= 0 && window.highway?.getSongInfo?.(),
|
||||
null, { timeout: 45000 }).catch(() => {});
|
||||
await page.waitForFunction(() => (window.highway?.getNotes?.() || []).length > 0, null, { timeout: 45000 });
|
||||
await page.evaluate(() => window.highway.addDrawHook(() => { window.__drew = true; }));
|
||||
// Start playback so draw() leaves its paused-throttle path; confirm the clock advances.
|
||||
await page.evaluate(async () => { const a = window.highway.getAudioElement?.(); if (a) a.muted = true; await a?.play?.(); });
|
||||
await page.waitForTimeout(1500);
|
||||
const t1 = await page.evaluate(() => window.highway.getTime());
|
||||
await page.evaluate(() => { window.__f.length = 0; });
|
||||
await page.waitForTimeout(FRAME_S * 1000);
|
||||
({ f, t2, notes } = await page.evaluate(() => ({
|
||||
f: window.__f.slice(), t2: window.highway.getTime(), notes: window.highway.getNotes().length,
|
||||
})));
|
||||
if (!(t2 > t1 + 1)) throw new Error(`clock did not advance (${t1}→${t2}) — measured the paused path`);
|
||||
} finally {
|
||||
// Always close, even when an await above throws — otherwise a failing
|
||||
// run leaks its page until the final browser.close().
|
||||
await page.close();
|
||||
}
|
||||
const drew = f.filter((x) => x.drew).map((x) => x.ms);
|
||||
return { drew, total: f.length, notes };
|
||||
}
|
||||
|
||||
async function frameTime() {
|
||||
const browser = await chromium.launch({ args: ['--autoplay-policy=no-user-gesture-required'] });
|
||||
const rows = [];
|
||||
for (let i = 0; i < RUNS; i++) {
|
||||
try {
|
||||
const r = await frameTimeOnce(browser);
|
||||
rows.push({
|
||||
p50: pct(r.drew, 50), p95: pct(r.drew, 95), p99: pct(r.drew, 99),
|
||||
max: Math.max(...r.drew), n: r.drew.length, total: r.total, notes: r.notes,
|
||||
});
|
||||
} catch (e) { rows.push({ error: String(e.message || e) }); }
|
||||
}
|
||||
await browser.close();
|
||||
return rows;
|
||||
}
|
||||
|
||||
const server = await serverLatency([
|
||||
'/api/version',
|
||||
'/api/plugins',
|
||||
@@ -86,9 +157,21 @@ out += `| Cold boot → networkidle | ${client.bootMs} ms |\n`;
|
||||
out += `| JS heap after load | ${client.memStartMB ? client.memStartMB.toFixed(1) + ' MB' : '—'} |\n`;
|
||||
out += `| JS heap after ${SOAK_S}s idle soak | ${client.memSoakMB ? client.memSoakMB.toFixed(1) + ' MB' : '—'} |\n`;
|
||||
out += `| Plugin scripts injected | ${client.scripts} |\n`;
|
||||
out += `\n> **Requires a seeded library** (not captured by this run): playback frame-time p95\n`;
|
||||
out += `> on the 2D + 3D highway, and screen-entry (plugin inject → interactive) for\n`;
|
||||
out += `> editor / notedetect / highway_3d with a real chart loaded. Run this harness\n`;
|
||||
out += `> against an environment with charts in \`DLC_DIR\` to fill those in.\n`;
|
||||
if (SONG) {
|
||||
const frames = await frameTime();
|
||||
out += `\n### 2D highway per-frame draw cost — \`${SONG}\` (${RUNS} runs × ${FRAME_S}s)\n\n`;
|
||||
out += `| run | draw frames | p50 | p95 | p99 | max |\n|---|---|---|---|---|---|\n`;
|
||||
for (let i = 0; i < frames.length; i++) {
|
||||
const r = frames[i];
|
||||
if (r.error) { out += `| ${i + 1} | — | \`${r.error}\` | | | |\n`; continue; }
|
||||
out += `| ${i + 1} | ${r.n}/${r.total} | ${ms(r.p50)} | ${ms(r.p95)} | ${ms(r.p99)} | ${ms(r.max)} |\n`;
|
||||
}
|
||||
const p95s = frames.filter((r) => !r.error).map((r) => r.p95);
|
||||
if (p95s.length) out += `\n**p95 spread across runs: ${ms(Math.min(...p95s))}–${ms(Math.max(...p95s))} ms**\n`;
|
||||
} else {
|
||||
out += `\n> **Requires a seeded library** (not captured by this run): playback frame-time p95\n`;
|
||||
out += `> on the 2D highway — pass \`--song "<filename in DLC_DIR>"\` to capture it.\n`;
|
||||
out += `> (3D highway_3d + screen-entry timings are a separate R4 concern.)\n`;
|
||||
}
|
||||
|
||||
console.log(out);
|
||||
|
||||
@@ -16,6 +16,8 @@ import pytest
|
||||
import yaml
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from routers.ws_highway import _sanitize_authors
|
||||
|
||||
|
||||
# ── _sanitize_authors unit tests ────────────────────────────────────────────
|
||||
|
||||
@@ -35,7 +37,7 @@ def server_mod(monkeypatch, tmp_path):
|
||||
|
||||
|
||||
def test_sanitize_authors_valid(server_mod):
|
||||
out = server_mod._sanitize_authors(
|
||||
out = _sanitize_authors(
|
||||
{
|
||||
"authors": [
|
||||
{"name": "Azure", "role": "charter", "email": "a@b.c", "url": "x"},
|
||||
@@ -53,7 +55,7 @@ def test_sanitize_authors_valid(server_mod):
|
||||
|
||||
|
||||
def test_sanitize_authors_skips_malformed(server_mod):
|
||||
out = server_mod._sanitize_authors(
|
||||
out = _sanitize_authors(
|
||||
{
|
||||
"authors": [
|
||||
{"name": ""}, # blank name → skipped
|
||||
@@ -69,7 +71,7 @@ def test_sanitize_authors_skips_malformed(server_mod):
|
||||
|
||||
@pytest.mark.parametrize("manifest", [None, {}, {"authors": None}, {"authors": "x"}, "nope"])
|
||||
def test_sanitize_authors_absent_or_nonlist(server_mod, manifest):
|
||||
assert server_mod._sanitize_authors(manifest) == []
|
||||
assert _sanitize_authors(manifest) == []
|
||||
|
||||
|
||||
# ── song_info WS integration ────────────────────────────────────────────────
|
||||
@@ -118,6 +120,9 @@ def make_client(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(server, "load_plugins", lambda *a, **kw: None)
|
||||
monkeypatch.setattr(server, "startup_scan", lambda: None)
|
||||
monkeypatch.setattr(server, "SLOPPAK_CACHE_DIR", tmp_path / "cache")
|
||||
# ws_highway reads the cache dir through the appstate seam now.
|
||||
import appstate as _appstate
|
||||
monkeypatch.setattr(_appstate, "sloppak_cache_dir", tmp_path / "cache")
|
||||
return server
|
||||
|
||||
(tmp_path / "dlc").mkdir()
|
||||
|
||||
@@ -96,6 +96,9 @@ def make_client(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(server, "load_plugins", lambda *a, **kw: None)
|
||||
monkeypatch.setattr(server, "startup_scan", lambda: None)
|
||||
monkeypatch.setattr(server, "SLOPPAK_CACHE_DIR", tmp_path / "cache")
|
||||
# ws_highway reads the cache dir through the appstate seam now.
|
||||
import appstate as _appstate
|
||||
monkeypatch.setattr(_appstate, "sloppak_cache_dir", tmp_path / "cache")
|
||||
return server
|
||||
|
||||
(tmp_path / "dlc").mkdir()
|
||||
|
||||
@@ -123,6 +123,9 @@ def make_client(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(server, "load_plugins", lambda *a, **kw: None)
|
||||
monkeypatch.setattr(server, "startup_scan", lambda: None)
|
||||
monkeypatch.setattr(server, "SLOPPAK_CACHE_DIR", tmp_path / "cache")
|
||||
# ws_highway reads the cache dir through the appstate seam now.
|
||||
import appstate as _appstate
|
||||
monkeypatch.setattr(_appstate, "sloppak_cache_dir", tmp_path / "cache")
|
||||
return server
|
||||
|
||||
(tmp_path / "dlc").mkdir()
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Playlist cover upload: client vs server errors, no temp litter, no leak.
|
||||
|
||||
The pre-split handler caught decode AND persistence failures in one `except`,
|
||||
returned 400 for both, and echoed the exception (`Invalid image: {e}`) — so a
|
||||
disk/permission failure was mislabeled as a client error and could leak a
|
||||
filesystem path. These pin the split.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import io
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from PIL import Image
|
||||
|
||||
import appstate
|
||||
from metadata_db import MetadataDB
|
||||
from routers import playlists
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(tmp_path):
|
||||
prev = (appstate.meta_db, appstate.config_dir)
|
||||
db = MetadataDB(tmp_path)
|
||||
appstate.configure(meta_db=db, config_dir=tmp_path)
|
||||
app_ = __import__("fastapi").FastAPI()
|
||||
app_.include_router(playlists.router)
|
||||
try:
|
||||
yield TestClient(app_), tmp_path
|
||||
finally:
|
||||
db.conn.close()
|
||||
appstate.configure(meta_db=prev[0], config_dir=prev[1])
|
||||
|
||||
|
||||
def _png_b64():
|
||||
buf = io.BytesIO()
|
||||
Image.new("RGB", (8, 8), (10, 20, 30)).save(buf, "PNG")
|
||||
return base64.b64encode(buf.getvalue()).decode()
|
||||
|
||||
|
||||
def _make_playlist(client):
|
||||
return client.post("/api/playlists", json={"name": "P"}).json()["id"]
|
||||
|
||||
|
||||
def test_valid_cover_saves_and_leaves_no_temp(client):
|
||||
c, tmp_path = client
|
||||
pid = _make_playlist(c)
|
||||
r = c.post(f"/api/playlists/{pid}/cover", json={"image": _png_b64()})
|
||||
assert r.status_code == 200
|
||||
cover_dir = tmp_path / "playlist_covers"
|
||||
assert (cover_dir / f"{pid}.png").exists()
|
||||
# The atomic-publish temp file must not linger.
|
||||
assert not list(cover_dir.glob("*.tmp"))
|
||||
|
||||
|
||||
def test_undecodable_image_is_a_400_without_leaking(client):
|
||||
c, _ = client
|
||||
pid = _make_playlist(c)
|
||||
# valid base64, not a valid image
|
||||
r = c.post(f"/api/playlists/{pid}/cover", json={"image": base64.b64encode(b"not an image").decode()})
|
||||
assert r.status_code == 400
|
||||
body = r.json()["error"]
|
||||
assert body == "Invalid image" # generic — no exception detail echoed
|
||||
assert "playlist_covers" not in body # no filesystem path leak
|
||||
|
||||
|
||||
def test_save_failure_is_a_500_not_a_400(client, monkeypatch):
|
||||
"""A persistence failure (here: Image.save raising) must be a logged 500,
|
||||
not a 400 — the whole point of the decode/persist split. Negative-checks
|
||||
against the pre-fix behavior, which returned 400 for exactly this."""
|
||||
c, tmp_path = client
|
||||
pid = _make_playlist(c)
|
||||
payload = _png_b64() # build BEFORE patching save
|
||||
|
||||
def boom(self, fp, *a, **k):
|
||||
raise OSError("disk full")
|
||||
|
||||
monkeypatch.setattr(Image.Image, "save", boom)
|
||||
r = c.post(f"/api/playlists/{pid}/cover", json={"image": payload})
|
||||
|
||||
assert r.status_code == 500
|
||||
assert "disk full" not in r.json()["error"] # no internal detail
|
||||
assert not list((tmp_path / "playlist_covers").glob("*.tmp")) # temp cleaned up
|
||||
|
||||
|
||||
def test_temp_creation_failure_is_a_500(client, monkeypatch):
|
||||
"""mkstemp raising (unwritable dir / full disk) must hit the same logged 500
|
||||
path as a save failure, not escape as an unhandled server error."""
|
||||
import tempfile as _tempfile
|
||||
|
||||
c, _ = client
|
||||
pid = _make_playlist(c)
|
||||
payload = _png_b64()
|
||||
|
||||
def boom(*a, **k):
|
||||
raise OSError("read-only file system")
|
||||
|
||||
monkeypatch.setattr(_tempfile, "mkstemp", boom)
|
||||
r = c.post(f"/api/playlists/{pid}/cover", json={"image": payload})
|
||||
assert r.status_code == 500
|
||||
assert "read-only" not in r.json()["error"]
|
||||
|
||||
|
||||
def test_upload_to_missing_playlist_is_404(client):
|
||||
c, _ = client
|
||||
r = c.post("/api/playlists/9999/cover", json={"image": _png_b64()})
|
||||
assert r.status_code == 404
|
||||
@@ -0,0 +1,110 @@
|
||||
"""A client that disconnects mid-stream is routine, not a logged error.
|
||||
|
||||
The highway WS streams ~15 message batches before its keep-alive loop. A
|
||||
disconnect during that streaming used to fall through to the blanket
|
||||
`except Exception` and log `highway_ws unhandled error`. The dedicated
|
||||
`except WebSocketDisconnect: return` makes it quiet.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import importlib
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
from fastapi import WebSocketDisconnect
|
||||
|
||||
|
||||
def _write_sloppak(dlc_root):
|
||||
pak = dlc_root / "disc.sloppak"
|
||||
pak.mkdir(parents=True)
|
||||
(pak / "arrangements").mkdir()
|
||||
(pak / "arrangements" / "lead.json").write_text(json.dumps({
|
||||
"notes": [], "chords": [], "anchors": [], "handshapes": [],
|
||||
"templates": [], "beats": [{"time": 0.0, "measure": 1}],
|
||||
"sections": [{"name": "intro", "number": 1, "time": 0.0}],
|
||||
}))
|
||||
(pak / "manifest.yaml").write_text(yaml.safe_dump({
|
||||
"title": "Disc", "artist": "T", "album": "", "year": 2026, "duration": 10.0,
|
||||
"arrangements": [{"id": "lead", "name": "Lead", "file": "arrangements/lead.json"}],
|
||||
"stems": [],
|
||||
}, sort_keys=False))
|
||||
return pak
|
||||
|
||||
|
||||
class _DisconnectingWS:
|
||||
"""Accepts, then raises WebSocketDisconnect on the first streamed send —
|
||||
i.e. a client that drops mid-stream."""
|
||||
def __init__(self):
|
||||
self.sends = 0
|
||||
|
||||
async def accept(self):
|
||||
pass
|
||||
|
||||
async def send_json(self, data):
|
||||
self.sends += 1
|
||||
# Raise only on the FIRST send. If the handler wrongly kept streaming
|
||||
# after catching the disconnect, later sends would succeed and `sends`
|
||||
# would climb past 1 — the exactly-one assertion catches that.
|
||||
if self.sends == 1:
|
||||
raise WebSocketDisconnect(code=1001)
|
||||
|
||||
async def receive_text(self):
|
||||
raise WebSocketDisconnect(code=1001)
|
||||
|
||||
async def close(self):
|
||||
pass
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def server(tmp_path, monkeypatch):
|
||||
(tmp_path / "dlc").mkdir()
|
||||
monkeypatch.setenv("CONFIG_DIR", str(tmp_path / "config"))
|
||||
monkeypatch.setenv("DLC_DIR", str(tmp_path / "dlc"))
|
||||
monkeypatch.setenv("FEEDBACK_SKIP_STARTUP_TASKS", "1")
|
||||
sys.modules.pop("server", None)
|
||||
mod = importlib.import_module("server")
|
||||
yield mod, tmp_path / "dlc"
|
||||
conn = getattr(getattr(mod, "meta_db", None), "conn", None)
|
||||
if conn is not None:
|
||||
getattr(mod, "_join_background_db_threads", lambda: None)()
|
||||
conn.close()
|
||||
sys.modules.pop("server", None)
|
||||
|
||||
|
||||
class _Capture(logging.Handler):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.messages = []
|
||||
|
||||
def emit(self, record):
|
||||
self.messages.append(record.getMessage())
|
||||
|
||||
|
||||
def test_midstream_disconnect_is_not_logged_as_error(server):
|
||||
"""The first streamed send (`loading`) raises WebSocketDisconnect; the
|
||||
handler must return quietly, not log `highway_ws unhandled error`.
|
||||
|
||||
A raw handler on `feedBack.server` is used rather than pytest's `caplog`
|
||||
because `configure_logging()` (run at server import) reroutes that logger
|
||||
through the structlog pipeline, which caplog's fixture doesn't observe.
|
||||
"""
|
||||
_server, dlc = server
|
||||
_write_sloppak(dlc)
|
||||
from routers.ws_highway import highway_ws
|
||||
|
||||
cap = _Capture()
|
||||
cap.setLevel(logging.ERROR)
|
||||
lg = logging.getLogger("feedBack.server")
|
||||
lg.addHandler(cap)
|
||||
try:
|
||||
ws = _DisconnectingWS()
|
||||
asyncio.run(highway_ws(ws, "disc.sloppak", arrangement=0)) # must not raise
|
||||
finally:
|
||||
lg.removeHandler(cap)
|
||||
|
||||
assert ws.sends == 1, f"handler kept streaming after the disconnect ({ws.sends} sends)"
|
||||
unhandled = [m for m in cap.messages if "highway_ws unhandled error" in m]
|
||||
assert not unhandled, f"disconnect logged as unhandled error: {unhandled}"
|
||||
Reference in New Issue
Block a user