diff --git a/CHANGELOG.md b/CHANGELOG.md index a72eb9d..53d8dcd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - **Keyset (cursor) pagination for the library grid — the data layer for an upcoming virtualized grid, and a latent paging bug fixed along the way.** Every library sort now carries a unique `filename` tiebreak, making the order **total** — which fixes a latent bug where rows sharing a sort key (e.g. two songs by the same artist) could be skipped or duplicated across `OFFSET` pages. `GET /api/library` gains an opaque `after` cursor + a `next_cursor` in the response: passing the cursor back fetches the next page with a **WHERE-seek** instead of `OFFSET`, so deep paging is O(page) regardless of depth. The seek is NULL-aware and exactly `OFFSET`-equivalent (verified across artist/title/recent, ascending + descending, including the legacy `dir=desc` shape and NULL sort keys); unknown/compound sorts and bad cursors fall back to `OFFSET`, and only the local provider is handed a cursor (collections/remote page by `OFFSET`). New composite `(artist NOCASE, filename)` / `(title NOCASE, filename)` / `(mtime, filename)` indexes cover the order. This is stage 1 of the virtualized-grid project (got-feedback/feedBack#636 item 3); the DOM-recycling render window builds on it next. Tests: `tests/test_library_keyset.py` (keyset==OFFSET parity, stable tiebreak, dir=desc, NULL keys, cursor fallback). +- **Smart collections — save a set of library filters as a live, auto-updating source.** A collection is a saved `/api/library` query (e.g. "Drop-D tunings", "sloppak only", "recently added") that stays live: it's registered as a **library provider**, so it shows up in the v3 Songs source picker and inherits the whole grid UI — paging, stats, the A–Z rail, art — for free, with **no new screen**. Storage reuses the playlist subsystem (a `playlists.rules` JSON blob = a smart collection; membership is the live filter result, not stored songs, and collections are excluded from the manual-playlist list + read-only to playlist mutations). New `GET`/`POST`/`PUT`/`DELETE /api/collections`; a per-collection `SmartCollectionProvider` delegates `query_page`/`query_stats`/`query_artists` to the local DB with the stored rules applied; providers are re-registered from a boot scan so collections survive a restart. Rules mirror the raw `/api/library` query params (unknown keys dropped, never 500). Frontend: a "+ Save as collection" action in the v3 filter drawer (shown when filters are active) names the current filter set and switches to it. The charrette's "the homelab primitive FeedBack was missing" pick (got-feedback/feedBack#636 item 2); richer rule fields (accuracy, genre, difficulty) follow as the metadata work lands. Tests: `tests/test_collections_api.py`, `tests/js/v3_collections.test.js`. - **The settings backup now includes your library database + custom art — your scores, favorites, playlists, and play history are no longer the one thing a backup can't save.** `GET /api/settings/export` gains an additive `core_server_files` section carrying a **consistent snapshot of `web_library.db`** (taken via the SQLite online-backup API, so it's a complete single file even while the server is running) plus any custom **playlist covers** and **avatar** (`CONFIG_DIR/playlist_covers/`, `CONFIG_DIR/avatars/`). On `POST /api/settings/import` the database is **staged** to `web_library.db.restore` rather than written over the live, open DB; it's swapped in at the next startup (`_apply_pending_db_restore`, before the connection opens), which also clears the old WAL sidecars so a stale `-wal` can't be replayed onto the restored file — the import response sets `restart_required: true` and warns accordingly. Custom art is written immediately. The bundle stays backward-compatible (older servers ignore the new section). Came out of the library design charrette (dev-ops lens's top "protect irreplaceable data" pick, got-feedback/feedBack#636). _Known gap:_ custom uploaded **song** art is still commingled with the rebuildable thumbnail cache in `art_cache/`, so it isn't bundled yet (a tracked follow-up). Tests: `tests/test_settings_export_library_db.py` (snapshot consistency, staged-not-live restore, sidecar clearing, traversal rejection, full round-trip). - **A persisted wishlist — keep a list of songs you want but don't own yet.** New `wanted` table + `GET`/`POST`/`DELETE /api/wanted` give FeedBack the *arr-style "Wanted/Monitored" primitive it was missing: an entry is a *not-owned* song (artist/title/source/source_ref/note), so it lives in its own table rather than the playlist subsystem (which references owned local files). The API is idempotent on identity (case-insensitive artist+title, plus source+source_ref), so a producer — the `find_more` ownership-diff, or a manual add — can re-post without duplicating. Newest-first. Backend primitive for the charrette's wishlist finding (got-feedback/feedBack#636 item 4); the consuming UI lives in the producing plugin. Tests: `tests/test_wanted_api.py`. - **Practice-aware library home — a "Repertoire" meter + a "Keep practicing" shelf on the v3 Songs page.** The library opened cold into a flat sorted grid; now the unfiltered grid front door leads with two practice-aware surfaces built entirely from data already on hand (no new endpoints or stored state). A **Repertoire meter** shows how much of your library you can actually play — *"Repertoire: 12 of 80 songs · 7 in progress"* with a progress bar — counting songs at or above the same mastery threshold the green accuracy badge uses (≥ 90% best accuracy) over the unfiltered library total. A **"Keep practicing" shelf** is a horizontal row of your recently-played-but-not-yet-mastered songs (newest first, click to play) — the practice-accuracy-driven "continue" rail a media server can't do. Both reuse `/api/stats/best` (already loaded for the card badges) + `/api/stats/recent`; they show **only** on the grid view when you aren't searching/filtering/selecting, refresh after a song is scored, and collapse to nothing on an empty library. Soft-gamification only — descriptive encouragement (goal-gradient / endowed-progress), never content-gating, decay, or nagging. Frontend-only: `static/v3/songs.js` (`renderLibraryHome`/`_repertoireCounts`), `static/v3/v3.css`. Came out of the library design charrette (the UX + gamification lenses' top pick). Tests: `tests/js/v3_keep_practicing.test.js`. diff --git a/server.py b/server.py index c546291..cbb9f25 100644 --- a/server.py +++ b/server.py @@ -657,6 +657,15 @@ class MetadataDB: ) """) self.conn.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_playlists_system_key ON playlists(system_key) WHERE system_key IS NOT NULL") + # Smart collections (feedBack#636 item 2): a playlist row whose `rules` + # JSON is non-NULL is a smart/dynamic collection — its membership is the + # LIVE result of those library filter params, not a stored song list. + # It surfaces as a registered library provider (the v3 source picker), + # so it inherits the whole Songs UI. Additive, idempotent migration. + try: + self.conn.execute("ALTER TABLE playlists ADD COLUMN rules TEXT") + except sqlite3.OperationalError: + pass # Wishlist / "wanted" (feedBack#636 item 4): a persisted, actionable # list of songs the user does NOT own yet — the *arr "Wanted/Monitored" # analogue. Unlike playlists (which reference owned local songs by @@ -1497,6 +1506,7 @@ class MetadataDB: from urllib.parse import quote rows = self.conn.execute( "SELECT id, name, system_key, created_at, updated_at FROM playlists " + "WHERE rules IS NULL " # smart collections live in the source picker, not here "ORDER BY (system_key IS NULL), name COLLATE NOCASE" ).fetchall() out = [] @@ -1568,14 +1578,79 @@ class MetadataDB: self.conn.commit() return cur.rowcount > 0 + # ── Smart collections (feedBack#636 item 2) ─────────────────────────── + @staticmethod + def _collection_row(r) -> dict: + rules = {} + if r[3]: + try: + parsed = json.loads(r[3]) + if isinstance(parsed, dict): + rules = parsed + except (ValueError, TypeError): + rules = {} + return {"id": r[0], "name": r[1], "system_key": r[2], "rules": rules, + "created_at": r[4], "updated_at": r[5]} + + def is_collection(self, pid: int) -> bool: + row = self.conn.execute( + "SELECT rules IS NOT NULL FROM playlists WHERE id = ?", (pid,) + ).fetchone() + return bool(row and row[0]) + + def list_collections(self) -> list[dict]: + rows = self.conn.execute( + "SELECT id, name, system_key, rules, created_at, updated_at FROM playlists " + "WHERE rules IS NOT NULL ORDER BY name COLLATE NOCASE" + ).fetchall() + return [self._collection_row(r) for r in rows] + + def get_collection(self, pid: int) -> dict | None: + r = self.conn.execute( + "SELECT id, name, system_key, rules, created_at, updated_at FROM playlists " + "WHERE id = ? AND rules IS NOT NULL", (pid,) + ).fetchone() + return self._collection_row(r) if r else None + + def create_collection(self, name: str, rules: dict) -> dict: + with self._lock: + cur = self.conn.execute( + "INSERT INTO playlists (name, system_key, rules, created_at, updated_at) " + "VALUES (?, NULL, ?, datetime('now'), datetime('now'))", + (name, json.dumps(rules or {})), + ) + self.conn.commit() + pid = cur.lastrowid + return self.get_collection(pid) + + def update_collection(self, pid: int, name: str | None = None, + rules: dict | None = None) -> dict | None: + if not self.is_collection(pid): + return None + with self._lock: + if name is not None: + self.conn.execute("UPDATE playlists SET name = ? WHERE id = ?", (name, pid)) + if rules is not None: + self.conn.execute("UPDATE playlists SET rules = ? WHERE id = ?", + (json.dumps(rules or {}), pid)) + self.conn.execute("UPDATE playlists SET updated_at = datetime('now') WHERE id = ?", (pid,)) + self.conn.commit() + return self.get_collection(pid) + def get_playlist(self, pid: int) -> dict | None: # A path-param int outside SQLite's 64-bit range raises OverflowError at # bind time (→ 500). Treat it as a miss; every mutating playlist handler # gates on this first, so the guard covers them too. if not isinstance(pid, int) or not (-(2**63) <= pid < 2**63): return None + # `rules IS NULL` excludes smart collections (#636 item 2): they share + # the playlists table but their membership is rules-based, so every + # manual-playlist mutation (add/remove/reorder/cover) that gates on + # get_playlist uniformly 404s on a collection id — collections are + # managed only through /api/collections. head = self.conn.execute( - "SELECT id, name, system_key, created_at, updated_at FROM playlists WHERE id = ?", (pid,) + "SELECT id, name, system_key, created_at, updated_at FROM playlists " + "WHERE id = ? AND rules IS NULL", (pid,) ).fetchone() if not head: return None @@ -2800,7 +2875,126 @@ class LibraryProviderRegistry: library_providers = LibraryProviderRegistry() -library_providers.register(LocalLibraryProvider(meta_db)) +_local_library_provider = LocalLibraryProvider(meta_db) +library_providers.register(_local_library_provider) + + +# Keys `_library_filter_args` (and a smart collection's stored `rules`) accept. +_LIBRARY_FILTER_PARAM_KEYS = frozenset(( + "q", "favorites", "format", "artist", "album", + "arrangements_has", "arrangements_lacks", "stems_has", "stems_lacks", + "has_lyrics", "tunings", +)) +# Rules mirror the raw /api/library query params (so the provider can feed them +# straight through `_library_filter_args`, and the frontend can build a rule from +# the same query string it already constructs). Multi-value filters are CSV +# strings; `favorites` is 0/1; the rest are plain strings. +_RULE_CSV_KEYS = frozenset(( + "tunings", "arrangements_has", "arrangements_lacks", "stems_has", "stems_lacks", +)) +_RULE_STR_KEYS = frozenset(("q", "format", "artist", "album", "has_lyrics", "sort")) + + +def _sanitize_collection_rules(raw) -> dict: + """Normalize rules to the raw query-param format, keeping only known keys. A + list for a multi-value filter is joined to CSV; `favorites` becomes 0/1. + Unknown keys are dropped so a rule survives a filter-vocab change rather than + 500-ing. Applied at API ingress AND when a provider loads a persisted row, so + a hand-edited / imported bad value (e.g. an int where a string is expected, + or a list for `sort`) can never crash a query.""" + if not isinstance(raw, dict): + return {} + out: dict = {} + for k, v in raw.items(): + if k in _RULE_CSV_KEYS: + if isinstance(v, list): + vals = [str(x) for x in v if isinstance(x, (str, int)) and not isinstance(x, bool)] + elif isinstance(v, str): + vals = [s for s in (p.strip() for p in v.split(",")) if s] + else: + continue + if vals: + out[k] = ",".join(vals) + elif k == "favorites": + if v: + out[k] = 1 + elif k in _RULE_STR_KEYS: + if isinstance(v, (str, int)) and not isinstance(v, bool): + s = str(v).strip() + if s: + out[k] = s + return out + + +class SmartCollectionProvider: + """A saved library filter, surfaced as a source (#636 item 2). Browse/stats + delegate to the local DB with the collection's stored `rules` applied — so + selecting it in the v3 source picker shows exactly that filtered slice with + the whole Songs UI (paging, stats, A–Z rail, art) for free. P1: the rules + ARE the query (live in-collection search is a P2 nicety). The matched songs + are local rows, so `kind="local"` keeps the client's play/art paths on the + local (not remote-sync) branch and art delegates straight through.""" + kind = "local" + capabilities = ("library.read", "art.read") + + def __init__(self, collection: dict, local: "LocalLibraryProvider"): + self._local = local + self.update(collection) + + def update(self, collection: dict) -> None: + self.id = f"collection:{collection['id']}" + self.collection_id = collection["id"] + self.label = collection.get("name") or "Collection" + # Re-sanitize on load: persisted JSON may predate the current vocab or + # have been hand-edited; never let a bad value reach a query. + self._rules = _sanitize_collection_rules(collection.get("rules") or {}) + + def _filter_kwargs(self) -> dict: + return _library_filter_args(**{k: v for k, v in self._rules.items() + if k in _LIBRARY_FILTER_PARAM_KEYS}) + + def _sort(self, fallback: str) -> str: + # A collection may pin its own sort (e.g. "recently added"); query_page + # falls back safely for an unknown value, so no validation needed here. + return self._rules.get("sort") or fallback + + def query_page(self, *, page=0, size=24, sort="artist", direction="asc", + naming_mode="legacy", **_ignore): + return self._local._db.query_page( + page=page, size=size, sort=self._sort(sort), direction=direction, + naming_mode=naming_mode, **self._filter_kwargs()) + + def query_artists(self, *, letter="", page=0, size=50, naming_mode="legacy", **_ignore): + return self._local._db.query_artists( + letter=letter, page=page, size=size, naming_mode=naming_mode, + **self._filter_kwargs()) + + def query_stats(self, *, sort="artist", want_sort_letters=False, + naming_mode="legacy", **_ignore): + return self._local._db.query_stats( + sort=self._sort(sort), want_sort_letters=want_sort_letters, + naming_mode=naming_mode, **self._filter_kwargs()) + + def tuning_names(self): + return self._local.tuning_names() + + async def get_art(self, song_id: str): + return await self._local.get_art(song_id) + + +def _sync_collection_provider(collection: dict) -> None: + """Register (or replace) the provider for one collection.""" + library_providers.register( + SmartCollectionProvider(collection, _local_library_provider), replace=True) + + +def _unregister_collection_provider(pid: int) -> None: + library_providers.unregister(f"collection:{pid}") + + +# Boot scan: surface every saved collection as a source. +for _c in meta_db.list_collections(): + _sync_collection_provider(_c) def register_library_provider(provider: object, *, replace: bool = False, owner_plugin_id: str | None = None) -> object: @@ -5675,6 +5869,53 @@ def api_delete_playlist_cover(pid: int): return {"ok": True} +# ── Smart collections API (feedBack#636 item 2) ─────────────────────────────── +# (rule schema + `_sanitize_collection_rules` are defined with the provider.) + +@app.get("/api/collections") +def api_list_collections(): + """Smart/dynamic collections (saved live library filters).""" + return {"collections": meta_db.list_collections()} + + +@app.post("/api/collections") +def api_create_collection(data: dict): + """Create a collection from a name + a set of library filter rules. It + immediately appears as a source in the library provider picker.""" + if not isinstance(data, dict): + return JSONResponse({"error": "body must be an object"}, status_code=400) + name = _clean_str(data.get("name")) + if not name: + return JSONResponse({"error": "name required"}, status_code=400) + col = meta_db.create_collection(name, _sanitize_collection_rules(data.get("rules"))) + _sync_collection_provider(col) + return {"ok": True, "collection": col} + + +@app.put("/api/collections/{pid}") +def api_update_collection(pid: int, data: dict): + """Rename a collection and/or replace its rules.""" + if not isinstance(data, dict): + return JSONResponse({"error": "body must be an object"}, status_code=400) + name = _clean_str(data.get("name")) or None + rules = _sanitize_collection_rules(data["rules"]) if "rules" in data else None + col = meta_db.update_collection(pid, name=name, rules=rules) + if col is None: + return JSONResponse({"error": "collection not found"}, status_code=404) + _sync_collection_provider(col) + return {"ok": True, "collection": col} + + +@app.delete("/api/collections/{pid}") +def api_delete_collection(pid: int): + """Delete a collection and unregister its provider.""" + if not meta_db.is_collection(pid): + return JSONResponse({"error": "collection not found"}, status_code=404) + meta_db.delete_playlist(pid) + _unregister_collection_provider(pid) + return {"ok": True} + + @app.post("/api/saved/toggle") def api_toggle_saved(data: dict): """Add/remove a song on the reserved Saved-for-Later playlist.""" diff --git a/static/v3/songs.js b/static/v3/songs.js index 06180d3..c46b3d9 100644 --- a/static/v3/songs.js +++ b/static/v3/songs.js @@ -202,6 +202,50 @@ return p; } + // The active filter set as a smart-collection rule object (raw query-param + // format the backend stores). Mirrors queryParams' filter fields, minus + // provider/page/size. Empty object → nothing worth saving as a collection. + function currentFilterRules() { + const f = state.filters, r = {}; + if (state.q) r.q = state.q; + if (state.format) r.format = state.format; + if (state.artist) r.artist = state.artist; + if (state.album) r.album = state.album; + if (f.arr_has.length) r.arrangements_has = f.arr_has.join(','); + if (f.arr_lacks.length) r.arrangements_lacks = f.arr_lacks.join(','); + if (f.stem_has.length) r.stems_has = f.stem_has.join(','); + if (f.stem_lacks.length) r.stems_lacks = f.stem_lacks.join(','); + if (f.lyrics) r.has_lyrics = f.lyrics; + if (f.tunings.length) r.tunings = f.tunings.join(','); + if (state.sort && state.sort !== 'artist') r.sort = state.sort; + return r; + } + + // Save the current filter set as a smart collection (a saved live query that + // shows up as a source in the picker). #636 item 2. + async function saveCurrentAsCollection() { + const rules = currentFilterRules(); + if (!Object.keys(rules).length) return; + const name = ((await window.uiPrompt({ + title: 'Save as collection', + label: 'A live view of the current filters, in the source picker.', + okLabel: 'Save', + placeholder: 'Collection name', + })) || '').trim(); + if (!name) return; + try { + const res = await fetch('/api/collections', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ name, rules }), + }); + if (!res.ok) return; + const col = (await res.json()).collection; + closeDrawer(); + if (col && col.id != null) state.provider = 'collection:' + col.id; + await render(); // rebuilds the toolbar (provider picker now lists + selects it) + } catch (e) { /* offline / aborted — leave the drawer as-is */ } + } + function albumsForArtist(name) { const a = (state.artistCatalog || []).find((x) => x.name === name); return a ? (a.albums || []) : []; @@ -1162,6 +1206,11 @@ } return triPill('tuning', val, label + ' (' + t.count + ')', f.tunings.includes(val) ? 'has' : 'any'); }).join('') || 'No tunings') + + // Collections always replay against the LOCAL library, so only offer + // "save" when browsing local with a non-empty filter set. + (state.provider === 'local' && Object.keys(currentFilterRules()).length + ? '
' + : '') + '