lib/scan.py (326). server.py 2,098 -> 1,870.
The background scan, its spawn ProcessPoolExecutor, and the kick/runner plumbing that
serialises passes. Bodies VERBATIM except the seam reads.
Everything shared is read LATE off appstate — the same contract every module in
lib/routers/ uses, and it is not cosmetic: tests monkeypatch CONFIG_DIR and swap meta_db,
so a value captured at import time pins the wrong one for the life of the process.
CONFIG_DIR -> appstate.config_dir
meta_db -> appstate.meta_db
_default_settings -> appstate.default_settings()
_stat_for_cache -> appstate.stat_for_cache()
━━━ THE SCAN STATUS IS REBOUND, NOT MUTATED ━━━
_background_scan does `global _scan_status; _scan_status = {**INIT, ...}` at every stage
transition. It REPLACES the dict; it never updates it in place. So nothing may hold that
dict by value — a reference captured once goes permanently stale at the first stage change
and would report "listing" forever while the scan ran to completion.
Hence `scan.status()`, a getter, and hence appstate publishes scan_status as a CALLABLE.
appstate.py already said so in a comment; this is the code that makes it true. (Same for
the plugin_context entry, which was already `lambda: dict(_scan_status)` — late-bound, so
it survives the move unchanged. The contract test from #898 covers it.)
━━━ appstate.server_root: A TRAP CLOSED PERMANENTLY ━━━
_background_scan seeds the builtin content, which needs the directory holding server.py.
`Path(__file__).resolve().parent` is correct in server.py and silently WRONG anywhere under
lib/ — it yields lib/, which holds no docs/ or data/ — and it fails by finding NOTHING
rather than by raising, so the seeds would just quietly never run.
lib/builtin_content.py (#900) closed that by taking the root as a parameter. This adds the
other half: server.py publishes it ONCE as appstate.server_root, so no module under lib/
ever has a reason to derive it. Documented at the slot.
pyflakes caught two more missing imports on the way in (loosefolder_mod, enrichment) —
each a NameError on a live scan path, and the suite would have handed them over one failure
at a time. It stays part of every server.py slice.
TESTS. The two scan fixtures (test_settings_api::scan_module,
test_feedpak_extension::scan_server) patched server._make_scan_executor to swap the spawn
pool for an in-process ThreadPool; they now patch it on lib/scan.py. Worth noting WHY that
still works: the fixtures re-import `server` per test, but `scan` stays cached in
sys.modules — and it picks up the fresh CONFIG_DIR anyway, because the appstate reads are
late-bound. The seam is doing exactly the job it was built for.
━━━ TEST ISOLATION: A REGRESSION THE CARVE ITSELF CREATED (Codex [P2]) ━━━
background_scan() deliberately NEVER sets running=False — ownership of that flag lives in
_scan_runner, so a kick_scan() racing the terminal write cannot observe a stale False and
start a second runner. Correct in production.
But the scan fixtures call background_scan() DIRECTLY, skipping the runner. That was
harmless while the state lived on `server`, which the fixtures RE-IMPORT per test. It is
NOT harmless now: `scan` stays cached in sys.modules across sys.modules.pop("server"), so
the status dict OUTLIVES the test. One direct call leaves the shared scanner marked
"running" forever, and every later scan or rescan returns "already in progress" and quietly
does nothing.
Verified: after a direct call, kick_scan() returns False and starts no scan at all.
The suite passed anyway, on ordering luck — which is exactly how this class of bug ships.
tests/conftest.py::reset_scan_state now snapshots and restores lib/scan.py's module state
around the two fixtures that drive it directly.
pytest 2398, pyflakes 0, Codex 0.
Refs #48
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GET/POST /api/settings, /api/settings/reset, and the two-phase atomic
export/import bundle (/api/settings/export|import) move to lib/routers/settings.py
with their exclusive helpers (the relpath allowlist validator, the atomic writer,
the library-DB snapshot + sqlite integrity gate, the config-type validator, the
bundle schema). Bodies verbatim except @app->@router and the seam reads:
meta_db->appstate.meta_db, CONFIG_DIR->appstate.config_dir,
_running_version->appstate.running_version(), and _default_settings->
appstate.default_settings (the canonical defaults builder stays in server.py —
the scan + artist-links code share it — and is injected as a new seam callable).
server.py: 5,539 -> 4,478 (-1,061).
Verified: pyflakes clean (bar the pre-existing File/safe_join/tuning_name/ET);
route table IDENTICAL (143); full pytest 2397 passed (154 settings cases incl the
export→import round-trip + library-DB snapshot/restore + relpath-allowlist SSRF/
traversal guards, retargeted onto the settings module). eslint 0.
BEHAVIORAL — needs an on-device settings export→import round-trip sign-off before
merge (do not merge on green CI alone).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move-only. The library metadata cache -- the `MetadataDB` class (4,018 lines)
plus the query helpers it owns (keyset paging cursors, the tuning grouping key,
smart-arrangement naming, tag normalisation, the startup DB-restore swap) --
moves out of server.py into a flat `lib/` module. Every moved block is
byte-identical to its server.py original; server.py is exactly origin/main
minus the six cut ranges, minus the now-dead `import contextlib`, plus the
import-back block and the constructor call site.
server.py: 14,037 -> 9,705 lines.
The one non-verbatim change is the seam that lets the class leave server.py:
`MetadataDB.__init__` now takes `config_dir` explicitly instead of reading the
module-level CONFIG_DIR, so `lib/metadata_db.py` does no IO at import
(Principle V). The `meta_db` singleton stays in server.py, so `server.meta_db`
(282 refs) and `server.app` (67 refs) resolve unchanged and no route moves.
None of the 114 `monkeypatch.setattr(server, ...)` targets moved.
Logging still goes through the `feedBack.server` logger, so log filters and
caplog assertions resolve to the same logger object.
`tests/test_settings_export_library_db.py` imports `_apply_pending_db_restore`
from metadata_db (the test moves with its subject); no other test changed.
Verified: pyflakes clean on the new module (zero undefined names, zero unused
imports) and no new undefined name in server.py; pytest 2341 passed;
node --test 1030 passed; eslint 0 errors; uvicorn boot smoke serves
/api/version, /api/library, and all three migrated plugins' src/ module graphs
(stems, studio, editor -> 200).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The v3 Settings "Save" button posts dlc_dir together with demucs_server_url,
default_arrangement and av_offset_ms in one request. POST /api/settings
validated dlc_dir first and early-returned "DLC directory not found" before it
ever processed demucs_server_url, so on a machine whose DLC path doesn't resolve
(fresh install, unplugged/network drive, a path carried over from another
machine) setting the Demucs server address silently failed — reported in
got-feedBack/feedBack-demucs-server#3 (macOS 07-05 nightly).
- server: a non-resolving dlc_dir is now recorded as a warning and skipped
rather than aborting the whole POST, so the co-submitted keys still persist.
The bad path is surfaced via a new additive `warnings` field and folded into
`message` so the settings status line still shows it.
- client (v3): the Demucs input now autosaves on blur/enter via a single-key
persistSetting POST, like every other v3 setting, so it never depends on the
coupled Save button.
- tests: cover the decoupling and the unchanged happy path.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* settings: add host instrument profiles
Signed-off-by: ChrisBeWithYou <christian.a.cowan@gmail.com>
* settings: add instrument pathway selection
Signed-off-by: ChrisBeWithYou <christian.a.cowan@gmail.com>
* fix(settings): profile-aware saves/resets/switch, provider tunings, bass-5
Five regressions from the instrument-profiles rework:
1. save_settings canonicalized profiles on EVERY save -> empty/unrelated POST
froze default profiles into config.json (broke
test_empty_post_preserves_all_existing_keys). Gate on the save touching
instrument settings; GET already virtualizes profiles.
2. pathway is profile-mirrored, so the Gameplay reset (flat-key delete) was a
no-op. reset_settings now resets pathway inside the persisted profiles too.
3. Per-profile tuning validation rejected provider/custom tunings (tuner
plugin, /api/tunings). _valid_tuning_for_key now accepts a name unknown to
every built-in table while still rejecting a built-in misapplied to the
wrong key.
4. First-migration overwrote an explicit active_instrument_profile with the
legacy-inferred one, so a fresh-config switch to 'bass' was lost. Use
setdefault so an explicit request wins.
5. Pre-existing test_instrument_fields_persist used bass-5 + 'Drop D' (a
4-string tuning). Updated to the valid 'Drop A'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(settings): partial-merge instrument_profiles; clamp tuning on string-count switch
Two partial-update follow-ups:
- save_settings normalized a POSTed instrument_profiles by FILLING every omitted
profile with defaults and replacing wholesale, so a one-profile update reset
the others. Validate each PROVIDED profile individually and merge the partial
over the persisted set inside the lock — /api/settings is partial-merge.
- the string-count picker posted only string_count, so the backend silently
reset a now-invalid tuning to Standard while the UI kept the old value
(settings/tuner desync). Clamp + post the valid tuning too, mirroring the
instrument-switch path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Signed-off-by: ChrisBeWithYou <christian.a.cowan@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the flaky pytest segfault (exit 139): the background scan and enrichment daemon threads (_scan_runner/_enrich_runner) use the shared MetadataDB connection, but test fixtures closed that connection in teardown without stopping them. A daemon thread mid-query on a freed SQLite conn is a native use-after-free → SIGSEGV. The app's startup kicks a scan, so almost any app-booting fixture was vulnerable. It only surfaced now because got-feedback/feedBack#728 added a push trigger, so ci/test runs on every push to main.
Fix: server.py retains the scan/enrich thread handles and adds _join_background_db_threads(); every test fixture now joins the workers before conn.close(). Verified: the full suite runs to completion (no segfault) where it previously crashed at ~25%.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sharing earned Feats on the (forthcoming) public wall is strictly opt-in,
default OFF, with a binding data-minimization contract.
- Onboarding (static/v3/profile.js): a new opt-in step (now a 5-step wizard)
after song-directory / before paths — publishes only display name + earned
Feats, never songs/skills/scores; off by default.
- Settings (plugins/achievements/settings.html, System tab via
settings.category): the same toggle + a "Remove me from the wall" button
(POST remove-me — wipes local synced state offline + enqueues removal).
- Core (server.py): achievements_enabled (bool, default false) in
_default_settings + /api/settings validation + _RESETTABLE_SETTINGS_KEYS;
mirrored to localStorage in app.js loadSettings().
- Data-minimization gate: engine.build_wall_payload is the single explicit-dict
serializer; key-set is EXACTLY {display_name, player_hash, achievement_id,
unlocked_at}, achievement_id always a Feat id. Enqueue is gated on
opted-in AND profile identity (reused player_hash); competency never
enqueues (integration law).
Verified natively: settings round-trip + validation + remove-me; opted-in
activity enqueues exactly one 4-field Feat payload; Playwright confirms the
5-step wizard + opt-in card (default unchecked), zero console errors.
29 plugin tests + new settings tests pass.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the single long scrolling v3 settings screen with a horizontal tab
bar (Gameplay / Audio / Graphics / Keybinds / Progression / Mic / Plugins /
System) over card rows (icon + title + description, control on the right) with
a per-category Reset.
- static/v3/index.html: tab bar + card-row markup (ids keep hydrating through
the unchanged app.js loadSettings()/persistSetting() path).
- static/v3/settings.js (new): tab switching + active-tab persistence
(localStorage 'v3-settings-tab'), per-category reset, read-only Keybinds
reference from window.getAllShortcuts().
- static/v3/v3.css: plain CSS, no Tailwind rebuild.
- Per-plugin settings tab: new optional settings.category in plugin.json →
plugins/__init__.py surfaces settings_category; app.js mounts each plugin
<details> into #plugin-settings-<category> (fallback: Plugins tab).
highway_3d ships category: "graphics".
- New gameplay settings: countdown_before_song (wired end-to-end, default off);
miss_penalty + fail_behavior (persist-only stubs); "Note highway speed"
surfaces existing master_difficulty.
- New POST /api/settings/reset clears whitelisted keys back to defaults.
Tests: test_settings_api.py, test_plugins.py::test_settings_category_parsed_from_manifest,
tests/browser/settings-tabbed.spec.ts. 179 passed locally.
Ported from the pre-rename feat/v3-settings-tabbed WIP onto current main
(slopsmith→feedBack rename applied; settings-screen markup conflict resolved
in favour of the new tabbed layout — all prior setting ids preserved).
Closes#579
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Update GitHub repo references from feedback* to feedBack*
* rename: slopsmith -> feedBack, byron -> got-feedBack
Renames across the entire codebase:
- slopsmith/Slopsmith/SLOPSMITH/SlopSmith -> feedBack/FeedBack/FEEDBACK/FeedBack
- byron/Byron/Byrongamatos -> got-feedBack/got-feedBack/got-feedBack
- /home/byron/ -> /opt/got-feedBack/
- byron@ougsoft.com -> hi@got-feedBack.org
- github.com/byrongamatos/ -> github.com/got-feedback/
- com.byron. -> com.got-feedback.
- SLOPSMITH_ env vars -> FEEDBACK_ with backward-compat fallback
- Protocol/storage strings migrated with read-old/write-new pattern
- window.slopsmith JS API -> window.feedBack (canonical) + backward-compat alias
Refs: #rename-slopsmith
* rename: complete regen against current main + fix backward-compat alias
Regenerated the slopsmith->feedBack / byron->got-feedBack rename on top of
current main (3 commits had landed since the branch: #572/#554/#574),
resolving the four content conflicts in favour of main's newer content
(autoplay/auto-exit, accuracy-badge, Virtuoso re-home, feedpak badge).
Completion fixes on top of the mechanical rename:
- Re-apply rename to post-branch content the original rename never saw:
window.slopsmith(.Tour) consumers in lessons.js / notifications.js /
onboarding-tour.js, and the matching JS + python tests (autoplay_exit,
progression_*, test_feedpak_extension FEEDBACK_* env vars). The test env
vars now match server.py (which reads FEEDBACK_SYNC_STARTUP /
FEEDBACK_SKIP_STARTUP_TASKS), so the sync-startup test exercises the real
path again.
- Restore the window.slopsmith backward-compat alias dropped during conflict
resolution, and move the bus aliases to AFTER the _feedBackExisting merge
block so they reference the fully-assembled object (also fixes the
loop_api.test.js API-surface regex, which the original PR latently broke).
- Drop the stray empty data/web_library.db (runtime DB lives in CONFIG_DIR)
and gitignore it.
- Fix stale tone-source test: feed[dB]ack -> fee[dB]ack to match shipped
source labels.
Verified locally (org CI billing-blocked): JS 819/819 pass; pytest 1669
passed / 1683 collected with 0 import errors; zero residual slopsmith/byron
except the two intentional window.slopsmith aliases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* rename: implement advertised backward-compat + prune dead community plugins
Address gaps where PR #537's "Backward compatibility" section was advertised
but not implemented, and clean up the community plugin list.
Env vars (FEEDBACK_* canonical, legacy SLOPSMITH_* honoured):
- New lib/env_compat.py (getenv_compat / env_flag_compat) + tests. server.py
(_env_flag + all FEEDBACK_* reads), diagnostics_hardware, gp2midi and
tailwind_rebuild now resolve the legacy alias, so existing SLOPSMITH_UI /
SLOPSMITH_PLUGINS_DIR / etc. deployments keep working.
- Fix the rename collapsing plugins/__init__.py and minigames/routes.py from
`FEEDBACK_PLUGINS_DIR or SLOPSMITH_PLUGINS_DIR` into a redundant
`FEEDBACK_ or FEEDBACK_` (the fallback was silently lost).
Storage (app.js update-channel):
- Read feedBack-update-channel, fall back to legacy slopsmith-update-channel,
and clear the legacy key on write — so a user's update-channel preference
survives the rename instead of resetting to "stable".
Community plugin list (README): the rename rewrote third-party repo URLs we
don't own. Probed every one; their owners never renamed, so:
- Restore the 13 live community plugins to their real slopsmith-* names.
- Prune 6 that are 404 to the public (topkoa splitscreen/stems, OmikronApex
tuner, Jafz2001 nam-rig-builder, DeathlySin song-preview, Erikcb91 shuffle).
- Fix a pre-existing Guitar Theory clone-command typo (nam-tone -> guitar-theory).
Verified: env_compat 7/7, JS 819/819, pytest 1690 collected / 0 import errors,
rename-sensitive + startup suites green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: byrongamatos <xasiklas@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the vestigial encrypted-archive ingestion path that the loader can
no longer service:
- scanner no longer lists .psarc files; library is sloppaks + loose folders
- upload endpoint accepts .sloppak only (drop .psarc + magic-byte branch)
- delete endpoint handles sloppak/loose only
- drop the psarc_platform (pc/mac) setting, its validation and tests
- default CONFIG_DIR no longer references an external game data folder
- drop the now-unused pycryptodome dependency (no module imports it)
- correct stale comments that described scanning as archive decryption
Updates test_settings_api / test_settings_export to match (scan fixtures
now build .sloppak stubs).
