2 Commits

Author	SHA1	Message	Date
Byron Gamatos	0b4b174d33	perf(scan): skip full library re-stat when the tree is unchanged (#979) ... * perf(scan): skip full library re-stat when the tree is unchanged Startup scans globbed the whole DLC tree twice (*.feedpak, *.wem) and stat()'d every file to detect changes — ~100k filesystem round trips on a 50k-song library, and painful on a slow NTFS-3G FUSE mount (the "big drive churns on every launch" report). Adds/removes/renames of songs all bump the mtime of the containing directory (verified on the target mount), so after a full pass we persist {reldir: mtime_ns} for every library dir (scan_dir_signature.json, keyed by DLC path). The next scan re-stats only those dirs — a handful vs 100k ops — and skips the entire listing/stat pass when none changed. Blind spot: a pack rewritten in place under the same name bumps the file mtime but not its dir's. Rare for a song library, and the manual Refresh (/api/rescan + /api/rescan/full) now passes force=True to always do the full pass. force threads through kick_scan -> _scan_runner and coalesces like the rescan-pending flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scan): track directory-form songs' own dir in the signature CodeRabbit: _library_dirs recorded only each song's parent. For a directory-form song (loose-song folder or directory sloppak bundle), adding/removing/replacing a file INSIDE the folder bumps that folder's own mtime, not its parent's — so the fast path would skip a rescan it should run. Record the song's own dir when f.is_dir(). File-form sloppaks (a single .feedpak zip) aren't dirs, so the flat file library is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 21:23:10 +02:00
Byron Gamatos	8f014e6a30	refactor(server): carve the library scanner into lib/scan.py (R3b) (#901) ... lib/scan.py (326). server.py 2,098 -> 1,870. The background scan, its spawn ProcessPoolExecutor, and the kick/runner plumbing that serialises passes. Bodies VERBATIM except the seam reads. Everything shared is read LATE off appstate — the same contract every module in lib/routers/ uses, and it is not cosmetic: tests monkeypatch CONFIG_DIR and swap meta_db, so a value captured at import time pins the wrong one for the life of the process. CONFIG_DIR -> appstate.config_dir meta_db -> appstate.meta_db _default_settings -> appstate.default_settings() _stat_for_cache -> appstate.stat_for_cache() ━━━ THE SCAN STATUS IS REBOUND, NOT MUTATED ━━━ _background_scan does `global _scan_status; _scan_status = {**INIT, ...}` at every stage transition. It REPLACES the dict; it never updates it in place. So nothing may hold that dict by value — a reference captured once goes permanently stale at the first stage change and would report "listing" forever while the scan ran to completion. Hence `scan.status()`, a getter, and hence appstate publishes scan_status as a CALLABLE. appstate.py already said so in a comment; this is the code that makes it true. (Same for the plugin_context entry, which was already `lambda: dict(_scan_status)` — late-bound, so it survives the move unchanged. The contract test from #898 covers it.) ━━━ appstate.server_root: A TRAP CLOSED PERMANENTLY ━━━ _background_scan seeds the builtin content, which needs the directory holding server.py. `Path(__file__).resolve().parent` is correct in server.py and silently WRONG anywhere under lib/ — it yields lib/, which holds no docs/ or data/ — and it fails by finding NOTHING rather than by raising, so the seeds would just quietly never run. lib/builtin_content.py (#900) closed that by taking the root as a parameter. This adds the other half: server.py publishes it ONCE as appstate.server_root, so no module under lib/ ever has a reason to derive it. Documented at the slot. pyflakes caught two more missing imports on the way in (loosefolder_mod, enrichment) — each a NameError on a live scan path, and the suite would have handed them over one failure at a time. It stays part of every server.py slice. TESTS. The two scan fixtures (test_settings_api::scan_module, test_feedpak_extension::scan_server) patched server._make_scan_executor to swap the spawn pool for an in-process ThreadPool; they now patch it on lib/scan.py. Worth noting WHY that still works: the fixtures re-import `server` per test, but `scan` stays cached in sys.modules — and it picks up the fresh CONFIG_DIR anyway, because the appstate reads are late-bound. The seam is doing exactly the job it was built for. ━━━ TEST ISOLATION: A REGRESSION THE CARVE ITSELF CREATED (Codex [P2]) ━━━ background_scan() deliberately NEVER sets running=False — ownership of that flag lives in _scan_runner, so a kick_scan() racing the terminal write cannot observe a stale False and start a second runner. Correct in production. But the scan fixtures call background_scan() DIRECTLY, skipping the runner. That was harmless while the state lived on `server`, which the fixtures RE-IMPORT per test. It is NOT harmless now: `scan` stays cached in sys.modules across sys.modules.pop("server"), so the status dict OUTLIVES the test. One direct call leaves the shared scanner marked "running" forever, and every later scan or rescan returns "already in progress" and quietly does nothing. Verified: after a direct call, kick_scan() returns False and starts no scan at all. The suite passed anyway, on ordering luck — which is exactly how this class of bug ships. tests/conftest.py::reset_scan_state now snapshots and restores lib/scan.py's module state around the two fixtures that drive it directly. pytest 2398, pyflakes 0, Codex 0. Refs #48 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 01:26:02 +02:00