Go to file
Byron Gamatos 8f014e6a30
refactor(server): carve the library scanner into lib/scan.py (R3b) (#901)
lib/scan.py (326). server.py 2,098 -> 1,870.

The background scan, its spawn ProcessPoolExecutor, and the kick/runner plumbing that
serialises passes. Bodies VERBATIM except the seam reads.

Everything shared is read LATE off appstate — the same contract every module in
lib/routers/ uses, and it is not cosmetic: tests monkeypatch CONFIG_DIR and swap meta_db,
so a value captured at import time pins the wrong one for the life of the process.

    CONFIG_DIR        -> appstate.config_dir
    meta_db           -> appstate.meta_db
    _default_settings -> appstate.default_settings()
    _stat_for_cache   -> appstate.stat_for_cache()

━━━ THE SCAN STATUS IS REBOUND, NOT MUTATED ━━━

_background_scan does `global _scan_status; _scan_status = {**INIT, ...}` at every stage
transition. It REPLACES the dict; it never updates it in place. So nothing may hold that
dict by value — a reference captured once goes permanently stale at the first stage change
and would report "listing" forever while the scan ran to completion.

Hence `scan.status()`, a getter, and hence appstate publishes scan_status as a CALLABLE.
appstate.py already said so in a comment; this is the code that makes it true. (Same for
the plugin_context entry, which was already `lambda: dict(_scan_status)` — late-bound, so
it survives the move unchanged. The contract test from #898 covers it.)

━━━ appstate.server_root: A TRAP CLOSED PERMANENTLY ━━━

_background_scan seeds the builtin content, which needs the directory holding server.py.
`Path(__file__).resolve().parent` is correct in server.py and silently WRONG anywhere under
lib/ — it yields lib/, which holds no docs/ or data/ — and it fails by finding NOTHING
rather than by raising, so the seeds would just quietly never run.

lib/builtin_content.py (#900) closed that by taking the root as a parameter. This adds the
other half: server.py publishes it ONCE as appstate.server_root, so no module under lib/
ever has a reason to derive it. Documented at the slot.

pyflakes caught two more missing imports on the way in (loosefolder_mod, enrichment) —
each a NameError on a live scan path, and the suite would have handed them over one failure
at a time. It stays part of every server.py slice.

TESTS. The two scan fixtures (test_settings_api::scan_module,
test_feedpak_extension::scan_server) patched server._make_scan_executor to swap the spawn
pool for an in-process ThreadPool; they now patch it on lib/scan.py. Worth noting WHY that
still works: the fixtures re-import `server` per test, but `scan` stays cached in
sys.modules — and it picks up the fresh CONFIG_DIR anyway, because the appstate reads are
late-bound. The seam is doing exactly the job it was built for.

━━━ TEST ISOLATION: A REGRESSION THE CARVE ITSELF CREATED (Codex [P2]) ━━━

background_scan() deliberately NEVER sets running=False — ownership of that flag lives in
_scan_runner, so a kick_scan() racing the terminal write cannot observe a stale False and
start a second runner. Correct in production.

But the scan fixtures call background_scan() DIRECTLY, skipping the runner. That was
harmless while the state lived on `server`, which the fixtures RE-IMPORT per test. It is
NOT harmless now: `scan` stays cached in sys.modules across sys.modules.pop("server"), so
the status dict OUTLIVES the test. One direct call leaves the shared scanner marked
"running" forever, and every later scan or rescan returns "already in progress" and quietly
does nothing.

Verified: after a direct call, kick_scan() returns False and starts no scan at all.

The suite passed anyway, on ordering luck — which is exactly how this class of bug ships.
tests/conftest.py::reset_scan_state now snapshots and restores lib/scan.py's module state
around the two fixtures that drive it directly.

pytest 2398, pyflakes 0, Codex 0.

Refs #48

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:26:02 +02:00
.claude/skills Clean release snapshot 2026-06-16 18:47:13 +02:00
.github R0: module-migration rails (src/ serving, live-edit cache, scriptType loading, governance) (#812) 2026-07-08 10:14:40 +02:00
.specify refactor(ui)!: remove the classic v2 shell — v3 is the only UI (R3a) (#871) 2026-07-11 16:33:03 +02:00
content/starter Delete content/starter/beethoven-ode_to_joy.feedpak 2026-07-03 23:42:45 +02:00
data/progression Clean release snapshot 2026-06-16 18:47:13 +02:00
docs refactor(ui)!: remove the classic v2 shell — v3 is the only UI (R3a) (#871) 2026-07-11 16:33:03 +02:00
lib refactor(server): carve the library scanner into lib/scan.py (R3b) (#901) 2026-07-12 01:26:02 +02:00
plugins fix(plugins): make a module plugin actually re-evaluate on reload (#879) (#897) 2026-07-12 00:19:59 +02:00
scripts perf(harness): add 2D-highway frame-time measurement + capture the R3c gate (H0) (#848) 2026-07-10 23:15:07 +02:00
specs rename: slopsmith → feedBack, byron → got-feedBack (#537) 2026-06-23 11:03:01 +02:00
static fix(plugins): make a module plugin actually re-evaluate on reload (#879) (#897) 2026-07-12 00:19:59 +02:00
tests refactor(server): carve the library scanner into lib/scan.py (R3b) (#901) 2026-07-12 01:26:02 +02:00
.dockerignore fix(build): move appstate.py + routers/ under lib/ so the desktop app ships them (#836) 2026-07-10 17:16:34 +02:00
.gitattributes Purge external-format terminology from code, tests and docs 2026-06-16 19:36:53 +01:00
.gitignore Bundle drum_highway_3d + keys_highway_3d as in-tree core plugins (#693) 2026-07-02 00:25:31 +02:00
build-proxmox-ct.sh rename: slopsmith → feedBack, byron → got-feedBack (#537) 2026-06-23 11:03:01 +02:00
CHANGELOG.md refactor(ui)!: remove the classic v2 shell — v3 is the only UI (R3a) (#871) 2026-07-11 16:33:03 +02:00
CLAUDE.md refactor(ui)!: remove the classic v2 shell — v3 is the only UI (R3a) (#871) 2026-07-11 16:33:03 +02:00
CONTRIBUTING.md rename: slopsmith → feedBack, byron → got-feedBack (#537) 2026-06-23 11:03:01 +02:00
docker-compose.nas.yml rename: slopsmith → feedBack, byron → got-feedBack (#537) 2026-06-23 11:03:01 +02:00
docker-compose.yml fix(build): move appstate.py + routers/ under lib/ so the desktop app ships them (#836) 2026-07-10 17:16:34 +02:00
Dockerfile fix(build): move appstate.py + routers/ under lib/ so the desktop app ships them (#836) 2026-07-10 17:16:34 +02:00
eslint.config.js refactor(app): carve the plugin loader out of app.js into static/js/ (R3a) (#878) 2026-07-11 18:18:00 +02:00
LICENSE Clean release snapshot 2026-06-16 18:47:13 +02:00
main.py rename: slopsmith → feedBack, byron → got-feedBack (#537) 2026-06-23 11:03:01 +02:00
package-lock.json R0: module-migration rails (src/ serving, live-edit cache, scriptType loading, governance) (#812) 2026-07-08 10:14:40 +02:00
package.json R0: module-migration rails (src/ serving, live-edit cache, scriptType loading, governance) (#812) 2026-07-08 10:14:40 +02:00
playwright.config.ts Clean release snapshot 2026-06-16 18:47:13 +02:00
pyproject.toml Clean release snapshot 2026-06-16 18:47:13 +02:00
requirements-test.txt Clean release snapshot 2026-06-16 18:47:13 +02:00
requirements.txt Make the library loader sloppak/loose-only 2026-06-16 19:09:53 +01:00
server.py refactor(server): carve the library scanner into lib/scan.py (R3b) (#901) 2026-07-12 01:26:02 +02:00
SUPPORTERS.md rename: slopsmith → feedBack, byron → got-feedBack (#537) 2026-06-23 11:03:01 +02:00
tailwind.config.js refactor(ui)!: remove the classic v2 shell — v3 is the only UI (R3a) (#871) 2026-07-11 16:33:03 +02:00
TODO.md docs: correct plugin URL casing after the feedBack rename (#576) 2026-06-23 11:20:23 +02:00
uv.lock Clean release snapshot 2026-06-16 18:47:13 +02:00
VERSION Bump version to 0.3.0-alpha.1 2026-07-03 13:41:50 +02:00