mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-07-21 20:31:21 +00:00
lib/scan.py (326). server.py 2,098 -> 1,870.
The background scan, its spawn ProcessPoolExecutor, and the kick/runner plumbing that
serialises passes. Bodies VERBATIM except the seam reads.
Everything shared is read LATE off appstate — the same contract every module in
lib/routers/ uses, and it is not cosmetic: tests monkeypatch CONFIG_DIR and swap meta_db,
so a value captured at import time pins the wrong one for the life of the process.
CONFIG_DIR -> appstate.config_dir
meta_db -> appstate.meta_db
_default_settings -> appstate.default_settings()
_stat_for_cache -> appstate.stat_for_cache()
━━━ THE SCAN STATUS IS REBOUND, NOT MUTATED ━━━
_background_scan does `global _scan_status; _scan_status = {**INIT, ...}` at every stage
transition. It REPLACES the dict; it never updates it in place. So nothing may hold that
dict by value — a reference captured once goes permanently stale at the first stage change
and would report "listing" forever while the scan ran to completion.
Hence `scan.status()`, a getter, and hence appstate publishes scan_status as a CALLABLE.
appstate.py already said so in a comment; this is the code that makes it true. (Same for
the plugin_context entry, which was already `lambda: dict(_scan_status)` — late-bound, so
it survives the move unchanged. The contract test from #898 covers it.)
━━━ appstate.server_root: A TRAP CLOSED PERMANENTLY ━━━
_background_scan seeds the builtin content, which needs the directory holding server.py.
`Path(__file__).resolve().parent` is correct in server.py and silently WRONG anywhere under
lib/ — it yields lib/, which holds no docs/ or data/ — and it fails by finding NOTHING
rather than by raising, so the seeds would just quietly never run.
lib/builtin_content.py (#900) closed that by taking the root as a parameter. This adds the
other half: server.py publishes it ONCE as appstate.server_root, so no module under lib/
ever has a reason to derive it. Documented at the slot.
pyflakes caught two more missing imports on the way in (loosefolder_mod, enrichment) —
each a NameError on a live scan path, and the suite would have handed them over one failure
at a time. It stays part of every server.py slice.
TESTS. The two scan fixtures (test_settings_api::scan_module,
test_feedpak_extension::scan_server) patched server._make_scan_executor to swap the spawn
pool for an in-process ThreadPool; they now patch it on lib/scan.py. Worth noting WHY that
still works: the fixtures re-import `server` per test, but `scan` stays cached in
sys.modules — and it picks up the fresh CONFIG_DIR anyway, because the appstate reads are
late-bound. The seam is doing exactly the job it was built for.
━━━ TEST ISOLATION: A REGRESSION THE CARVE ITSELF CREATED (Codex [P2]) ━━━
background_scan() deliberately NEVER sets running=False — ownership of that flag lives in
_scan_runner, so a kick_scan() racing the terminal write cannot observe a stale False and
start a second runner. Correct in production.
But the scan fixtures call background_scan() DIRECTLY, skipping the runner. That was
harmless while the state lived on `server`, which the fixtures RE-IMPORT per test. It is
NOT harmless now: `scan` stays cached in sys.modules across sys.modules.pop("server"), so
the status dict OUTLIVES the test. One direct call leaves the shared scanner marked
"running" forever, and every later scan or rescan returns "already in progress" and quietly
does nothing.
Verified: after a direct call, kick_scan() returns False and starts no scan at all.
The suite passed anyway, on ordering luck — which is exactly how this class of bug ships.
tests/conftest.py::reset_scan_state now snapshots and restores lib/scan.py's module state
around the two fixtures that drive it directly.
pytest 2398, pyflakes 0, Codex 0.
Refs #48
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
155 lines
7.5 KiB
Python
155 lines
7.5 KiB
Python
"""Shared application state — the seam that lets route modules reach core
|
|
singletons without importing ``server``.
|
|
|
|
``server.py`` is the host: it owns the FastAPI ``app``, constructs the DB
|
|
singletons, and runs the lifecycle. As routes move out into ``routers/`` (R3),
|
|
those modules need ``meta_db`` and friends — but they must not ``import
|
|
server``, or the import graph goes circular the moment ``server`` imports them
|
|
back.
|
|
|
|
So ``server`` **injects** its singletons here once, at the point it builds them::
|
|
|
|
# server.py
|
|
meta_db = MetadataDB(CONFIG_DIR)
|
|
appstate.configure(meta_db=meta_db, ...)
|
|
|
|
and a router reads them back as **module attributes, at call time**::
|
|
|
|
# routers/artists.py
|
|
import appstate
|
|
|
|
@router.get("/api/artist/{name}/page")
|
|
def artist_page(name):
|
|
return appstate.meta_db.artist_page(name)
|
|
|
|
This is the Python analogue of the injected `configureX({...})` seams the
|
|
frontend refactor uses (stems' ``configureStreaming``, studio's
|
|
``configureAudioGraph``, the editor's ``src/host.js``), and of the plugin
|
|
``setup(app, context)`` contract in Principle III: dependencies flow one way,
|
|
``server -> routers -> appstate``, and nothing imports back up.
|
|
|
|
Two properties this shape buys, both load-bearing:
|
|
|
|
* **``import appstate`` performs no IO and constructs nothing.** ``server``
|
|
still owns construction, so the ~49 test fixtures that do
|
|
``sys.modules.pop("server")`` + re-import (to rebuild ``meta_db`` under a
|
|
patched ``CONFIG_DIR``) keep working untouched — a singleton *owned* here
|
|
would survive that pop and go stale.
|
|
* **Reads are late-bound.** Routers must use ``appstate.meta_db``, never
|
|
``from appstate import meta_db`` — a ``from`` import freezes the binding at
|
|
its current value, so a later ``configure()`` (or a
|
|
``monkeypatch.setattr(appstate, "meta_db", fake)``) would not reach the
|
|
router. This is the same read-only-binding trap as ES ``import``.
|
|
|
|
Defaults are ``None`` on purpose: they are inert but *type-honest*, so a router
|
|
that runs before ``configure()`` fails loudly on ``NoneType`` instead of
|
|
quietly operating on a stand-in.
|
|
|
|
Slots are added here only when a router actually needs one — this is a seam,
|
|
not a grab-bag for everything in ``server.py``.
|
|
|
|
**Why this lives in ``lib/`` and not the repo root.** Because it constructs
|
|
nothing and does no import-time IO, it satisfies Principle V's rule for ``lib/``
|
|
modules — and ``lib/`` is the only core directory every packaging path already
|
|
copies: the Dockerfile (``COPY lib/``), ``docker-compose.yml``, and
|
|
feedback-desktop's ``bundle-slopsmith.sh`` (``cp -r lib``). All three also put
|
|
both the bundle root and ``lib/`` on ``sys.path``. A root-level module ships in
|
|
Docker but is silently dropped from the packaged desktop app, whose bundler
|
|
copies a hardcoded file list — that regression is what moved this file here.
|
|
"""
|
|
|
|
# The singletons routers may read. Every name here must also be a `_SLOTS` key.
|
|
meta_db = None
|
|
audio_effect_mappings = None
|
|
# The tuning-provider registry instance (built-ins + plugin-contributed). A
|
|
# stable object mutated in place via register()/unregister() — injected here by
|
|
# reference so routers read the same registry plugins populate.
|
|
tuning_providers = None
|
|
# The library-provider registry instance + the local provider, constructed in
|
|
# server.py (LocalLibraryProvider needs meta_db) and injected by reference. The
|
|
# classes live in lib/library_registry.py; plugins register their own providers
|
|
# through the registry via plugin_context.
|
|
library_providers = None
|
|
local_library_provider = None
|
|
|
|
# Config paths. server.py derives these from the environment (fresh on every
|
|
# import, so the ~49 pop-and-reimport fixtures keep working) and injects them
|
|
# here. Routers read them as `appstate.config_dir` etc. — a module attribute at
|
|
# call time. NOTE: config_dir/dlc_dir are env-derived, so a `setenv`+reimport
|
|
# test reconfigures them for free; STATIC_DIR/SLOPPAK_CACHE_DIR are patched via
|
|
# `setattr(server, …)` in a few tests, so those slots (when added) need their
|
|
# tests retargeted to appstate in the same PR.
|
|
config_dir = None
|
|
dlc_dir = None # the DLC_DIR env value as a Path (Path("") if unset)
|
|
dlc_dir_env = None # the raw DLC_DIR env string, "" if unset — distinguishes
|
|
# "unset" from Path("")→"." (see dlc_paths._get_dlc_dir)
|
|
# Cache/asset dirs. static_dir + sloppak_cache_dir are patched via
|
|
# `setattr(server, …)` in a few tests, so a router reading them here needs those
|
|
# setattr sites retargeted to `setattr(appstate, …)` in the same PR (ws_highway
|
|
# retargets the 3 test_highway_ws_* SLOPPAK sites). config_dir-derived dirs are
|
|
# reconfigured for free on a setenv+reimport.
|
|
static_dir = None
|
|
sloppak_cache_dir = None
|
|
audio_cache_dir = None
|
|
|
|
# Injected callables (not values): server owns the impl + its state, routers call
|
|
# through the seam. get_progression_content wraps a lazy content cache that stays
|
|
# in server.py (its `setattr(server, "_progression_content")` test is untouched).
|
|
get_progression_content = None
|
|
builtin_diagnostic_filename = None
|
|
running_version = None
|
|
# Art helpers that stay in server.py (shared with the art/delete routes) but are
|
|
# also called by the enrichment worker in lib/enrichment.py — injected as
|
|
# callables to keep enrichment acyclic. art_cache_dir is server's ART_CACHE_DIR.
|
|
art_cache_dir = None
|
|
song_pack_art_exists = None
|
|
art_override_paths = None
|
|
art_safe_name = None
|
|
# The canonical settings-defaults builder — stays in server.py (shared with the
|
|
# scan/artist-links code) but the settings router calls it through the seam.
|
|
default_settings = None
|
|
# Scan/ingest seam for the song routes (routers/song.py). kick_scan/
|
|
# invalidate_song_caches/stat_for_cache stay in server.py (scan lifecycle owns
|
|
# them); scan_status is a GETTER (the underlying dict is reassigned, so a value
|
|
# would go stale) — call appstate.scan_status() to read the live status.
|
|
kick_scan = None
|
|
invalidate_song_caches = None
|
|
stat_for_cache = None
|
|
scan_status = None
|
|
|
|
# The directory containing server.py: the repo root in dev, resources/feedBack when
|
|
# bundled — the tree that actually holds docs/ and data/.
|
|
#
|
|
# It is published HERE, by server.py, precisely so no module under lib/ ever computes it.
|
|
# `Path(__file__).resolve().parent` is correct in server.py and silently WRONG anywhere in
|
|
# lib/ (it yields lib/, which has no docs/ or data/), and it fails by finding nothing
|
|
# rather than by raising — the builtin-content seeds would just quietly never run. See
|
|
# lib/builtin_content.py's header. Read it; never re-derive it.
|
|
server_root = None
|
|
|
|
_SLOTS = frozenset({
|
|
"meta_db", "audio_effect_mappings", "tuning_providers",
|
|
"library_providers", "local_library_provider",
|
|
"config_dir", "dlc_dir", "dlc_dir_env",
|
|
"static_dir", "sloppak_cache_dir", "audio_cache_dir",
|
|
"get_progression_content", "builtin_diagnostic_filename",
|
|
"running_version",
|
|
"art_cache_dir", "song_pack_art_exists", "art_override_paths", "art_safe_name",
|
|
"default_settings",
|
|
"kick_scan", "invalidate_song_caches", "stat_for_cache", "scan_status",
|
|
"server_root",
|
|
})
|
|
|
|
|
|
def configure(**kwargs) -> None:
|
|
"""Publish `server`'s singletons into this module. Called once per
|
|
`server` import (and again on re-import), so it must be idempotent."""
|
|
unknown = set(kwargs) - _SLOTS
|
|
if unknown:
|
|
raise TypeError(
|
|
f"appstate.configure() got unknown slot(s): {sorted(unknown)}. "
|
|
f"Known slots: {sorted(_SLOTS)}. Add the name to _SLOTS if a router "
|
|
f"genuinely needs it."
|
|
)
|
|
globals().update(kwargs)
|