perf(scan): skip full library re-stat when the tree is unchanged (#979)
ship-ci / ci (push) Waiting to run

* perf(scan): skip full library re-stat when the tree is unchanged

Startup scans globbed the whole DLC tree twice (*.feedpak, *.wem) and
stat()'d every file to detect changes — ~100k filesystem round trips on
a 50k-song library, and painful on a slow NTFS-3G FUSE mount (the "big
drive churns on every launch" report).

Adds/removes/renames of songs all bump the mtime of the containing
directory (verified on the target mount), so after a full pass we persist
{reldir: mtime_ns} for every library dir (scan_dir_signature.json, keyed
by DLC path). The next scan re-stats only those dirs — a handful vs 100k
ops — and skips the entire listing/stat pass when none changed.

Blind spot: a pack rewritten in place under the same name bumps the file
mtime but not its dir's. Rare for a song library, and the manual Refresh
(/api/rescan + /api/rescan/full) now passes force=True to always do the
full pass. force threads through kick_scan -> _scan_runner and coalesces
like the rescan-pending flag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scan): track directory-form songs' own dir in the signature

CodeRabbit: _library_dirs recorded only each song's parent. For a
directory-form song (loose-song folder or directory sloppak bundle),
adding/removing/replacing a file INSIDE the folder bumps that folder's
own mtime, not its parent's — so the fast path would skip a rescan it
should run. Record the song's own dir when f.is_dir(). File-form
sloppaks (a single .feedpak zip) aren't dirs, so the flat file library
is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Byron Gamatos
2026-07-15 21:23:10 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 2f2a095e4c
commit 0b4b174d33
3 changed files with 217 additions and 7 deletions
+5 -2
View File
@@ -1115,7 +1115,10 @@ async def startup_status_stream(request: Request):
@app.post("/api/rescan")
def trigger_rescan():
"""Manually trigger a library rescan."""
if not scan.kick_scan():
# force=True: a manual Refresh must skip the directory-signature fast path —
# it is the escape hatch for the one change dir mtimes can't see (a pack
# rewritten in place under the same name).
if not scan.kick_scan(force=True):
return {"message": "Scan already in progress"}
return {"message": "Rescan started"}
@@ -1133,7 +1136,7 @@ def trigger_full_rescan():
# delete_missing() prunes anything genuinely gone at the end.
meta_db.conn.execute("UPDATE songs SET mtime = -1")
meta_db.conn.commit()
if not scan.kick_scan():
if not scan.kick_scan(force=True):
return {"message": "Scan already in progress"}
return {"message": "Full rescan started"}