mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-09-14 08:50:03 +00:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
59bcf338a3 | ||
|
|
03e1c1d57e |
@@ -0,0 +1,90 @@
|
||||
name: Content packs
|
||||
|
||||
# Build & publish opt-in career venue packs as per-pack, versioned, immutable
|
||||
# releases (convention: tag `venue-<id>-v<N>`, asset `<id>-pack-v<N>.zip`),
|
||||
# then open a PR bumping venues.json url/sha256/bytes. This is automation so
|
||||
# publishing packs is never a person's manual job (SLIM-NIGHTLY item 1b).
|
||||
#
|
||||
# Immutable tags → publishing is a deliberate, versioned act, so this runs on
|
||||
# manual dispatch (not push): a media change means a new version, a human call.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
venues:
|
||||
description: "Space-separated venue ids to (re)publish, e.g. 'club arena'"
|
||||
required: true
|
||||
default: "club"
|
||||
version:
|
||||
description: "Pack version N (tag venue-<id>-vN). Bump for new media."
|
||||
required: true
|
||||
default: "1"
|
||||
|
||||
concurrency:
|
||||
group: content-packs
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
# Media lives in-tree today; add `lfs: true` once SLIM-NIGHTLY item 4
|
||||
# moves venue-packs/** to Git LFS.
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Build & publish packs
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
# Never interpolate dispatch inputs straight into the shell — a crafted
|
||||
# value would execute on the runner with this job's write token. Pass
|
||||
# via env, validate the formats, and use a Bash argument array.
|
||||
VENUES: ${{ github.event.inputs.venues }}
|
||||
VERSION: ${{ github.event.inputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
[[ "$VERSION" =~ ^[1-9][0-9]*$ ]] || { echo "::error::version must be a positive integer"; exit 1; }
|
||||
[[ "$VENUES" =~ ^[a-z0-9][a-z0-9-]*(\ [a-z0-9][a-z0-9-]*)*$ ]] || { echo "::error::venues must be space-separated venue ids"; exit 1; }
|
||||
read -r -a venues <<< "$VENUES"
|
||||
dirs=()
|
||||
for v in "${venues[@]}"; do
|
||||
dirs+=("plugins/career/venue-packs/$v")
|
||||
done
|
||||
python tools/content_packs.py "${dirs[@]}" \
|
||||
--version "$VERSION" \
|
||||
--publish \
|
||||
--manifest /tmp/packs-manifest.json
|
||||
cat /tmp/packs-manifest.json
|
||||
|
||||
- name: Apply url/sha256/bytes to venues.json
|
||||
run: |
|
||||
python - <<'PY'
|
||||
import json, pathlib
|
||||
manifest = json.load(open("/tmp/packs-manifest.json"))
|
||||
vpath = pathlib.Path("plugins/career/venues.json")
|
||||
data = json.loads(vpath.read_text())
|
||||
for v in data["venues"]:
|
||||
m = manifest.get(v["id"])
|
||||
if m and v.get("pack"):
|
||||
v["pack"].update(url=m["url"], sha256=m["sha256"], bytes=m["bytes"])
|
||||
vpath.write_text(json.dumps(data, indent=4) + "\n")
|
||||
PY
|
||||
|
||||
- name: Open manifest-bump PR
|
||||
uses: peter-evans/create-pull-request@v6
|
||||
with:
|
||||
commit-message: "career: refresh venue pack manifest (v${{ github.event.inputs.version }})"
|
||||
title: "career: refresh venue pack manifest (v${{ github.event.inputs.version }})"
|
||||
body: |
|
||||
Automated by the content-packs workflow after publishing
|
||||
`${{ github.event.inputs.venues }}` v${{ github.event.inputs.version }}
|
||||
to their `venue-<id>-v${{ github.event.inputs.version }}` releases.
|
||||
Bumps `venues.json` pack url/sha256/bytes to match the uploaded zips.
|
||||
branch: content-packs/manifest-bump
|
||||
delete-branch: true
|
||||
@@ -8,6 +8,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Opt-in career venue packs (#122)** — higher-tier venue crowd media
|
||||
(`club`, `arena`) is no longer bundled; the app downloads each pack on demand
|
||||
from its release when you reach the venue (sha256-verified), keeping the
|
||||
starter `bar` venue bundled for offline play. Trims ~678 MB from the desktop
|
||||
download; an unpublished pack shows "coming soon" and plays on the standard
|
||||
stage until its release lands.
|
||||
- **Session-sync relay WebSocket — `/ws/sync/{session_id}` (#1030).** A
|
||||
deliberately dumb JSON fan-out room: a text frame received from one client is
|
||||
forwarded verbatim to every other client on the same session id; the server
|
||||
interprets nothing (message schemas are owned by consumers). Rooms are created
|
||||
on first join and garbage-collected when the last socket leaves — no history,
|
||||
no replay, no persistence, so a host that crashes and rejoins the same id
|
||||
resumes publishing to reconnecting subscribers with no server-side
|
||||
coordination. Session ids are client-generated (`[A-Za-z0-9_-]{4,64}`); DoS
|
||||
hygiene for a LAN-exposed port via frame-size (16 KB), per-room (16 sockets),
|
||||
total-room (32), and per-socket rate (120 msg/s sustained, 240 burst) caps —
|
||||
over-limit sockets are closed with a policy code and the room carries on, and
|
||||
a peer that dies — or stalls: fan-out sends are bounded by a 5 s timeout —
|
||||
mid-fan-out is dropped without disturbing delivery to the rest. `main.py`
|
||||
also caps inbound WS frames at the transport (`ws_max_size=64 KB`, down from
|
||||
uvicorn's 16 MB default) so oversized frames never materialize server-side. First consumer: splitscreen's "pop out to LAN" follower mode
|
||||
(feedBack-plugin-splitscreen#21), which relays playhead/playstate/song-change
|
||||
frames from a host window to view-only followers on other LAN devices.
|
||||
Implementation in `lib/routers/ws_sync.py`; tests in `tests/test_ws_sync.py`.
|
||||
- **Drum-part picker (feedpak 1.17.0 "drums as arrangements").** When a song
|
||||
carries several drum charts, a **Drum part** selector appears beside the
|
||||
arrangement switcher (advanced settings) so a player can choose which drummer
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
"""Session-sync relay WebSocket — /ws/sync/{session_id} (feedBack#1030).
|
||||
|
||||
A deliberately dumb fan-out room: a JSON text frame received from one client
|
||||
is forwarded verbatim to every OTHER client connected to the same session id.
|
||||
The server interprets nothing beyond the limits below — message schemas are
|
||||
owned entirely by consumers. First consumer: splitscreen's LAN follower mode
|
||||
(feedBack-plugin-splitscreen#21), which relays playhead/playstate/song-change
|
||||
frames from a host window to view-only followers on other LAN devices.
|
||||
|
||||
Design points (full spec in the issue):
|
||||
|
||||
- Rooms are created on first join and garbage-collected when the last socket
|
||||
leaves. No history, no replay, no persistence — a late joiner simply waits
|
||||
for the next frame. Consumers that need state on join re-send it themselves
|
||||
(splitscreen answers every follower ``hello`` with a fresh ``config``).
|
||||
- That statelessness is what makes consumer crash-recovery work: a host that
|
||||
relaunches and rejoins the same session id resumes publishing to its
|
||||
reconnecting subscribers with no server-side coordination, and an idle room
|
||||
is indistinguishable from a nonexistent one.
|
||||
- ``session_id`` is client-generated and opaque (``[A-Za-z0-9_-]{4,64}``);
|
||||
consumers pick their own id policy (splitscreen uses a short typeable,
|
||||
persistent room key).
|
||||
- DoS hygiene for a port that may be LAN-exposed: frame-size cap, per-room and
|
||||
total-room caps, and a per-socket inbound token-bucket rate cap. Over-limit
|
||||
sockets are closed with a policy code; the room carries on. A peer that dies
|
||||
mid-fan-out is dropped without wedging delivery to the rest.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
|
||||
from fastapi import APIRouter, WebSocket
|
||||
|
||||
log = logging.getLogger("feedBack.server")
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_SESSION_ID_RE = re.compile(r"^[A-Za-z0-9_-]{4,64}$")
|
||||
|
||||
# Limits. Sized generously above the first consumer's needs (splitscreen
|
||||
# publishes time frames at ~15-20 Hz to a handful of viewers) while bounding
|
||||
# what an open LAN port can be made to do. All module-level so tests (and a
|
||||
# desperate operator) can override them.
|
||||
MAX_FRAME_BYTES = 16 * 1024
|
||||
MAX_CLIENTS_PER_ROOM = 16
|
||||
MAX_ROOMS = 32
|
||||
RATE_MSGS_PER_SEC = 120.0 # sustained inbound frames per socket
|
||||
RATE_BURST = 240.0 # token-bucket burst headroom
|
||||
# A peer that stops draining its socket would leave send_text() pending
|
||||
# forever — and since publishers await the fan-out gather, one stalled peer
|
||||
# would stall every publisher's receive loop behind it. Bounding the send
|
||||
# turns the stall into an eviction through the normal failed-send drop path.
|
||||
SEND_TIMEOUT_SECONDS = 5.0
|
||||
|
||||
# RFC 6455 close codes.
|
||||
_WS_UNSUPPORTED_DATA = 1003 # binary frame on a text-only relay
|
||||
_WS_POLICY_VIOLATION = 1008 # invalid session id / rate cap exceeded
|
||||
_WS_MSG_TOO_BIG = 1009
|
||||
_WS_TRY_AGAIN_LATER = 1013 # room or server at capacity
|
||||
|
||||
# session_id → {socket: per-socket send lock}. The lock serializes concurrent
|
||||
# fan-out sends to the same peer (two publishers relaying at once must not
|
||||
# interleave writes on a third socket's transport).
|
||||
_rooms: dict[str, dict[WebSocket, asyncio.Lock]] = {}
|
||||
|
||||
|
||||
async def _send_locked(peer: WebSocket, lock: asyncio.Lock, text: str) -> None:
|
||||
async with lock:
|
||||
await asyncio.wait_for(peer.send_text(text), timeout=SEND_TIMEOUT_SECONDS)
|
||||
|
||||
|
||||
@router.websocket("/ws/sync/{session_id}")
|
||||
async def sync_ws(websocket: WebSocket, session_id: str):
|
||||
"""Join the fan-out room *session_id*; relay every inbound text frame."""
|
||||
await websocket.accept()
|
||||
|
||||
if not _SESSION_ID_RE.fullmatch(session_id):
|
||||
await websocket.close(code=_WS_POLICY_VIOLATION, reason="invalid session id")
|
||||
return
|
||||
|
||||
# Capacity checks and insertion run with no await between them, so
|
||||
# concurrent joiners on the event loop can't race past the caps.
|
||||
room = _rooms.get(session_id)
|
||||
if room is None:
|
||||
if len(_rooms) >= MAX_ROOMS:
|
||||
await websocket.close(code=_WS_TRY_AGAIN_LATER, reason="too many active sessions")
|
||||
return
|
||||
room = _rooms[session_id] = {}
|
||||
log.debug("ws_sync: room %s created", session_id)
|
||||
elif len(room) >= MAX_CLIENTS_PER_ROOM:
|
||||
await websocket.close(code=_WS_TRY_AGAIN_LATER, reason="session full")
|
||||
return
|
||||
room[websocket] = asyncio.Lock()
|
||||
|
||||
tokens = RATE_BURST
|
||||
last_refill = time.monotonic()
|
||||
|
||||
try:
|
||||
while True:
|
||||
message = await websocket.receive()
|
||||
if message["type"] == "websocket.disconnect":
|
||||
break
|
||||
text = message.get("text")
|
||||
if text is None:
|
||||
await websocket.close(code=_WS_UNSUPPORTED_DATA, reason="text frames only")
|
||||
break
|
||||
if len(text.encode("utf-8", errors="ignore")) > MAX_FRAME_BYTES:
|
||||
await websocket.close(code=_WS_MSG_TOO_BIG, reason="frame too large")
|
||||
break
|
||||
|
||||
now = time.monotonic()
|
||||
tokens = min(RATE_BURST, tokens + (now - last_refill) * RATE_MSGS_PER_SEC)
|
||||
last_refill = now
|
||||
tokens -= 1.0
|
||||
if tokens < 0:
|
||||
await websocket.close(code=_WS_POLICY_VIOLATION, reason="rate cap exceeded")
|
||||
break
|
||||
|
||||
peers = [(ws, lock) for ws, lock in room.items() if ws is not websocket]
|
||||
if not peers:
|
||||
continue
|
||||
results = await asyncio.gather(
|
||||
*(_send_locked(ws, lock, text) for ws, lock in peers),
|
||||
return_exceptions=True,
|
||||
)
|
||||
# A peer that failed mid-send is dropped from the room here; its
|
||||
# own handler finishes cleanup (the finally below) when its
|
||||
# receive loop observes the disconnect.
|
||||
for (peer, _lock), result in zip(peers, results):
|
||||
if isinstance(result, Exception):
|
||||
room.pop(peer, None)
|
||||
finally:
|
||||
room.pop(websocket, None)
|
||||
# Guard against deleting a NEW room another joiner created after this
|
||||
# one emptied (only possible for a dict that is no longer ours).
|
||||
if not room and _rooms.get(session_id) is room:
|
||||
del _rooms[session_id]
|
||||
log.debug("ws_sync: room %s closed", session_id)
|
||||
@@ -44,6 +44,13 @@ def run() -> None:
|
||||
# record — including early startup messages — passes through the same
|
||||
# structured pipeline.
|
||||
log_config=None,
|
||||
# Cap inbound WebSocket frames at the transport, before uvicorn
|
||||
# materializes them in memory (its default is 16 MB). No client sends
|
||||
# large frames to this server: the highway WS receives only small
|
||||
# control messages, and the /ws/sync relay enforces its own tighter
|
||||
# 16 KB application cap (routers/ws_sync.py MAX_FRAME_BYTES) — this is
|
||||
# the defense-in-depth bound above it.
|
||||
ws_max_size=64 * 1024,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -104,6 +104,13 @@ def _bundled(venue_id):
|
||||
return (_bundled_venue_dir(venue_id) / "manifest.json").is_file()
|
||||
|
||||
|
||||
def _pack_published(pack):
|
||||
"""A remote pack is downloadable only once a publish has stamped its real
|
||||
size — the committed manifest carries a 0-byte placeholder (and an all-zero
|
||||
sha) until then, so don't offer a download that can't succeed yet."""
|
||||
return bool(pack and (pack.get("bytes") or 0) > 0)
|
||||
|
||||
|
||||
def _stars():
|
||||
"""(total, per-song dict, detail rows). Accuracy is a 0..1 fraction."""
|
||||
db = _state["meta_db"]
|
||||
@@ -664,7 +671,7 @@ def setup(app, context):
|
||||
"unlocked": stars_total >= v["star_threshold"],
|
||||
"installed": _installed(v["id"]),
|
||||
"bundled": _bundled(v["id"]),
|
||||
"has_pack": _bundled(v["id"]) or bool(v.get("pack")),
|
||||
"has_pack": _bundled(v["id"]) or _pack_published(v.get("pack")),
|
||||
"download": dl,
|
||||
})
|
||||
return {
|
||||
@@ -934,7 +941,7 @@ def setup(app, context):
|
||||
if venue is None:
|
||||
raise HTTPException(404, "Unknown venue.")
|
||||
pack = venue.get("pack")
|
||||
if not pack:
|
||||
if not _pack_published(pack):
|
||||
raise HTTPException(404, "No pack published for this venue yet.")
|
||||
stars_total, _, _ = _stars()
|
||||
if stars_total < venue["star_threshold"]:
|
||||
|
||||
@@ -17,14 +17,22 @@
|
||||
"name": "Velvet Room",
|
||||
"description": "A proper club stage. People actually came to hear you.",
|
||||
"star_threshold": 50,
|
||||
"pack": null
|
||||
"pack": {
|
||||
"url": "https://github.com/got-feedBack/feedBack/releases/download/venue-club-v1/club-pack-v1.zip",
|
||||
"sha256": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"bytes": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arena",
|
||||
"name": "Feedback Arena",
|
||||
"description": "Ten thousand seats. Try not to think about it.",
|
||||
"star_threshold": 150,
|
||||
"pack": null
|
||||
"pack": {
|
||||
"url": "https://github.com/got-feedBack/feedBack/releases/download/venue-arena-v1/arena-pack-v1.zip",
|
||||
"sha256": "8898c7912c84123559ecfd8b0afd3be19da9a0a4b04aa9dbb5aa1e7f4e3e1669",
|
||||
"bytes": 351284599
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -49,7 +49,7 @@ import demo_mode
|
||||
import scan
|
||||
import tailwind_rebuild
|
||||
# Extracted route modules. They import `appstate`, never `server` — one-way graph.
|
||||
from routers import audio_effects, artist_aliases, loops, playlists, ws_highway, chart, wanted, library_extras, shop, progression, profile, stats, version, diagnostics
|
||||
from routers import audio_effects, artist_aliases, loops, playlists, ws_highway, ws_sync, chart, wanted, library_extras, shop, progression, profile, stats, version, diagnostics
|
||||
from routers import tunings as tunings_router
|
||||
import enrichment
|
||||
from routers import art as art_router
|
||||
@@ -1618,6 +1618,12 @@ app.include_router(media_router.router)
|
||||
app.include_router(ws_highway.router)
|
||||
|
||||
|
||||
# ── Session-sync relay WebSocket (feedBack#1030) ─────────────────────────────
|
||||
# Dumb JSON fan-out rooms for cross-device followers (splitscreen LAN mode).
|
||||
# Implementation in lib/routers/ws_sync.py.
|
||||
app.include_router(ws_sync.router)
|
||||
|
||||
|
||||
# ── Audio serving ─────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -83,10 +83,25 @@ def test_download_without_published_pack_404s(client):
|
||||
|
||||
def test_download_locked_venue_403s(client, monkeypatch):
|
||||
club = career_routes._venue("club")
|
||||
monkeypatch.setitem(club, "pack", {"url": "http://x/pack.zip", "sha256": "0" * 64})
|
||||
monkeypatch.setitem(club, "pack", {"url": "http://x/pack.zip", "sha256": "0" * 64, "bytes": 123})
|
||||
assert client.post("/api/plugins/career/packs/club/download").status_code == 403
|
||||
|
||||
|
||||
def test_placeholder_pack_is_not_offered_until_published(client, monkeypatch):
|
||||
# A committed manifest carries a 0-byte placeholder until its release is
|
||||
# published. Such a pack must not be offered (has_pack False) and its
|
||||
# download must 404 — else the UI shows a button that can only fail.
|
||||
monkeypatch.setattr(career_routes, "_bundled", lambda vid: False)
|
||||
club = career_routes._venue("club")
|
||||
monkeypatch.setitem(club, "pack",
|
||||
{"url": "http://x/c.zip", "sha256": "0" * 64, "bytes": 0})
|
||||
by_id = {v["id"]: v for v in client.get("/api/plugins/career/state").json()["venues"]}
|
||||
assert by_id["club"]["has_pack"] is False # placeholder → not offered
|
||||
assert by_id["arena"]["has_pack"] is True # arena ships real bytes
|
||||
# Even forced, an unpublished pack won't start a download.
|
||||
assert client.post("/api/plugins/career/packs/club/download").status_code == 404
|
||||
|
||||
|
||||
def test_bundled_bar_pack_is_installed_and_served(client):
|
||||
state = client.get("/api/plugins/career/state").json()
|
||||
bar = {v["id"]: v for v in state["venues"]}["bar"]
|
||||
@@ -167,9 +182,53 @@ def test_download_worker_end_to_end(client, tmp_path):
|
||||
assert "sha256" in bad["error"]
|
||||
|
||||
|
||||
def test_content_packs_build_roundtrips_through_download(client, tmp_path):
|
||||
# tools/content_packs.py must produce a zip the real career worker accepts:
|
||||
# build_pack → manifest_entry → _download_pack → installed.
|
||||
from tools import content_packs
|
||||
|
||||
src = tmp_path / "bar"
|
||||
src.mkdir()
|
||||
for s in career_routes.REQUIRED_LOOPS:
|
||||
(src / f"{s}.mp4").write_bytes(b"fake-" + s.encode())
|
||||
(src / "cheer.mp4").write_bytes(b"fake-cheer")
|
||||
(src / "manifest.json").write_text(json.dumps({
|
||||
"venue": "bar", "version": 1,
|
||||
"loops": {s: f"{s}.mp4" for s in career_routes.REQUIRED_LOOPS},
|
||||
"stingers": {"cheer": "cheer.mp4"},
|
||||
}))
|
||||
out_dir = tmp_path / "packs"
|
||||
zip_path = out_dir / content_packs.pack_asset("bar", 1)
|
||||
info = content_packs.build_pack(src, zip_path)
|
||||
entry = content_packs.manifest_entry(zip_path, zip_path.resolve().as_uri())
|
||||
assert entry["sha256"] == info["sha256"] and entry["bytes"] == info["bytes"]
|
||||
|
||||
progress = {"status": "running", "bytes_done": 0, "bytes_total": 0, "error": None}
|
||||
career_routes._download_pack("bar", entry, progress)
|
||||
assert progress["status"] == "done", progress["error"]
|
||||
assert career_routes._installed("bar")
|
||||
|
||||
|
||||
def test_content_packs_rejects_files_the_downloader_would_refuse(tmp_path):
|
||||
# A stray file (e.g. macOS .DS_Store) must fail the build, not get published
|
||||
# and then break every client's download at _validate_pack_dir.
|
||||
from tools import content_packs
|
||||
|
||||
src = tmp_path / "bar"
|
||||
src.mkdir()
|
||||
(src / "bored.mp4").write_bytes(b"fake")
|
||||
(src / ".DS_Store").write_bytes(b"junk")
|
||||
try:
|
||||
content_packs.build_pack(src, tmp_path / "bar-pack-v1.zip")
|
||||
except ValueError as e:
|
||||
assert "downloader will reject" in str(e)
|
||||
else:
|
||||
raise AssertionError("build_pack accepted a .DS_Store the downloader rejects")
|
||||
|
||||
|
||||
def test_double_download_409s(client, monkeypatch):
|
||||
bar = career_routes._venue("bar")
|
||||
monkeypatch.setitem(bar, "pack", {"url": "http://x/pack.zip", "sha256": "0" * 64})
|
||||
monkeypatch.setitem(bar, "pack", {"url": "http://x/pack.zip", "sha256": "0" * 64, "bytes": 123})
|
||||
# Pretend one is already running.
|
||||
career_routes._state["downloads"]["bar"] = {"status": "running"}
|
||||
assert client.post("/api/plugins/career/packs/bar/download").status_code == 409
|
||||
|
||||
@@ -0,0 +1,234 @@
|
||||
"""Tests for the session-sync relay WebSocket (/ws/sync/{session_id}).
|
||||
|
||||
Behavior tests run against a minimal FastAPI app carrying just the router
|
||||
(fast — no full-server import); one integration test imports the real server
|
||||
to pin that the route is actually mounted there.
|
||||
|
||||
Covers the feedBack#1030 acceptance list: bidirectional fan-out, late join,
|
||||
sender never echoed, room garbage collection, and the limit closes (invalid
|
||||
session id, binary frames, frame size, room size, room count, rate cap) —
|
||||
including that one client tripping a limit doesn't disturb the others.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import importlib
|
||||
import sys
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
from starlette.websockets import WebSocketDisconnect
|
||||
|
||||
from routers import ws_sync
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_rooms():
|
||||
ws_sync._rooms.clear()
|
||||
yield
|
||||
ws_sync._rooms.clear()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client():
|
||||
app = FastAPI()
|
||||
app.include_router(ws_sync.router)
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
|
||||
|
||||
def _expect_close(ws, code):
|
||||
with pytest.raises(WebSocketDisconnect) as exc:
|
||||
ws.receive_text()
|
||||
assert exc.value.code == code
|
||||
|
||||
|
||||
# ── Fan-out semantics ────────────────────────────────────────────────────────
|
||||
|
||||
def test_two_clients_relay_both_directions_and_no_echo(client):
|
||||
with client.websocket_connect("/ws/sync/ROOM01") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM01") as b:
|
||||
a.send_text('{"type":"time","t":1.5}')
|
||||
assert b.receive_text() == '{"type":"time","t":1.5}'
|
||||
b.send_text('{"type":"hello"}')
|
||||
# A's first inbound frame is B's hello — NOT an echo of its own send.
|
||||
assert a.receive_text() == '{"type":"hello"}'
|
||||
|
||||
|
||||
def test_late_joiner_receives_subsequent_frames(client):
|
||||
with client.websocket_connect("/ws/sync/ROOM02") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM02") as b:
|
||||
a.send_text("f1")
|
||||
assert b.receive_text() == "f1"
|
||||
with client.websocket_connect("/ws/sync/ROOM02") as c:
|
||||
a.send_text("f2")
|
||||
assert b.receive_text() == "f2"
|
||||
assert c.receive_text() == "f2"
|
||||
|
||||
|
||||
def test_rooms_are_isolated(client):
|
||||
with client.websocket_connect("/ws/sync/ROOMA1") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOMB1") as b, \
|
||||
client.websocket_connect("/ws/sync/ROOMA1") as a2:
|
||||
a.send_text("for-room-a")
|
||||
assert a2.receive_text() == "for-room-a"
|
||||
# B (other room) got nothing: prove it by relaying within B's room.
|
||||
with client.websocket_connect("/ws/sync/ROOMB1") as b2:
|
||||
b2.send_text("for-room-b")
|
||||
assert b.receive_text() == "for-room-b"
|
||||
|
||||
|
||||
def test_client_disconnect_does_not_disrupt_remaining(client):
|
||||
with client.websocket_connect("/ws/sync/ROOM03") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM03") as b:
|
||||
with client.websocket_connect("/ws/sync/ROOM03") as c:
|
||||
a.send_text("before")
|
||||
assert b.receive_text() == "before"
|
||||
assert c.receive_text() == "before"
|
||||
# C is gone; relay between A and B continues.
|
||||
a.send_text("after")
|
||||
assert b.receive_text() == "after"
|
||||
|
||||
|
||||
def test_room_garbage_collected_when_last_client_leaves(client):
|
||||
with client.websocket_connect("/ws/sync/ROOM04") as a:
|
||||
with client.websocket_connect("/ws/sync/ROOM04") as b:
|
||||
a.send_text("x")
|
||||
assert b.receive_text() == "x"
|
||||
assert "ROOM04" in ws_sync._rooms
|
||||
assert "ROOM04" not in ws_sync._rooms
|
||||
assert ws_sync._rooms == {}
|
||||
|
||||
|
||||
# ── Limit enforcement ────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize("bad_id", ["abc", "x" * 65, "has space", "bad$id", "nope!"])
|
||||
def test_invalid_session_id_closed_with_policy_code(client, bad_id):
|
||||
with client.websocket_connect(f"/ws/sync/{bad_id}") as ws:
|
||||
_expect_close(ws, 1008)
|
||||
assert ws_sync._rooms == {}
|
||||
|
||||
|
||||
def test_binary_frame_closes_with_unsupported_data(client):
|
||||
with client.websocket_connect("/ws/sync/ROOM05") as ws:
|
||||
ws.send_bytes(b"\x00\x01")
|
||||
_expect_close(ws, 1003)
|
||||
|
||||
|
||||
def test_oversized_frame_closes_sender_only(client):
|
||||
with client.websocket_connect("/ws/sync/ROOM06") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM06") as b, \
|
||||
client.websocket_connect("/ws/sync/ROOM06") as c:
|
||||
a.send_text("x" * (ws_sync.MAX_FRAME_BYTES + 1))
|
||||
_expect_close(a, 1009)
|
||||
# The room carries on without A.
|
||||
b.send_text("still-alive")
|
||||
assert c.receive_text() == "still-alive"
|
||||
|
||||
|
||||
def test_room_client_cap(client, monkeypatch):
|
||||
monkeypatch.setattr(ws_sync, "MAX_CLIENTS_PER_ROOM", 2)
|
||||
with client.websocket_connect("/ws/sync/ROOM07") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM07") as b, \
|
||||
client.websocket_connect("/ws/sync/ROOM07") as c:
|
||||
_expect_close(c, 1013)
|
||||
a.send_text("two-is-fine")
|
||||
assert b.receive_text() == "two-is-fine"
|
||||
|
||||
|
||||
def test_total_room_cap(client, monkeypatch):
|
||||
monkeypatch.setattr(ws_sync, "MAX_ROOMS", 1)
|
||||
with client.websocket_connect("/ws/sync/ROOM08"):
|
||||
with client.websocket_connect("/ws/sync/ROOM09") as overflow:
|
||||
_expect_close(overflow, 1013)
|
||||
# Joining the EXISTING room is still fine at the room cap.
|
||||
with client.websocket_connect("/ws/sync/ROOM08"):
|
||||
pass
|
||||
|
||||
|
||||
def test_rate_cap_closes_flooding_sender(client, monkeypatch):
|
||||
monkeypatch.setattr(ws_sync, "RATE_BURST", 3.0)
|
||||
monkeypatch.setattr(ws_sync, "RATE_MSGS_PER_SEC", 0.0)
|
||||
with client.websocket_connect("/ws/sync/ROOM10") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM10") as b:
|
||||
for i in range(3):
|
||||
a.send_text(f"burst-{i}")
|
||||
for i in range(3):
|
||||
assert b.receive_text() == f"burst-{i}"
|
||||
a.send_text("one-too-many")
|
||||
_expect_close(a, 1008)
|
||||
# The over-limit frame was dropped, not relayed, and B lives on.
|
||||
with client.websocket_connect("/ws/sync/ROOM10") as c:
|
||||
c.send_text("fresh-socket")
|
||||
assert b.receive_text() == "fresh-socket"
|
||||
|
||||
|
||||
class _StalledPeer:
|
||||
"""A fake room member whose send never completes (peer stopped draining)."""
|
||||
|
||||
async def send_text(self, text):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
|
||||
def test_stalled_peer_is_evicted_and_healthy_peers_still_receive(client, monkeypatch):
|
||||
monkeypatch.setattr(ws_sync, "SEND_TIMEOUT_SECONDS", 0.2)
|
||||
with client.websocket_connect("/ws/sync/ROOM11") as a, \
|
||||
client.websocket_connect("/ws/sync/ROOM11") as b:
|
||||
# Wait for both handlers to have registered in the room, then inject
|
||||
# the stalled peer directly (a real stalled TCP peer isn't
|
||||
# constructible under TestClient).
|
||||
deadline = time.monotonic() + 2.0
|
||||
while len(ws_sync._rooms.get("ROOM11", {})) < 2:
|
||||
assert time.monotonic() < deadline, "room never filled"
|
||||
time.sleep(0.01)
|
||||
stalled = _StalledPeer()
|
||||
ws_sync._rooms["ROOM11"][stalled] = asyncio.Lock()
|
||||
|
||||
# Healthy delivery is not blocked behind the stalled peer, and by the
|
||||
# time a second frame has round-tripped, the first fan-out's timeout
|
||||
# has fired and evicted it.
|
||||
a.send_text("f1")
|
||||
assert b.receive_text() == "f1"
|
||||
a.send_text("f2")
|
||||
assert b.receive_text() == "f2"
|
||||
assert stalled not in ws_sync._rooms["ROOM11"]
|
||||
|
||||
|
||||
def test_main_run_caps_uvicorn_ws_max_size():
|
||||
"""main.py must bound inbound WS frames at the transport (uvicorn defaults
|
||||
to 16 MB, which would let a client materialize frames far past the relay's
|
||||
16 KB application cap before the handler ever sees them)."""
|
||||
import unittest.mock
|
||||
|
||||
import main
|
||||
|
||||
with (
|
||||
unittest.mock.patch("logging_setup.configure_logging"),
|
||||
unittest.mock.patch("uvicorn.run") as mock_run,
|
||||
):
|
||||
main.run()
|
||||
|
||||
kwargs = mock_run.call_args.kwargs
|
||||
assert kwargs.get("ws_max_size") == 64 * 1024
|
||||
assert kwargs["ws_max_size"] >= ws_sync.MAX_FRAME_BYTES
|
||||
|
||||
|
||||
# ── Real-app integration ─────────────────────────────────────────────────────
|
||||
|
||||
def test_route_mounted_on_real_server(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("CONFIG_DIR", str(tmp_path / "config"))
|
||||
monkeypatch.setenv("DLC_DIR", str(tmp_path / "dlc"))
|
||||
monkeypatch.setenv("FEEDBACK_SYNC_STARTUP", "1")
|
||||
sys.modules.pop("server", None)
|
||||
server = importlib.import_module("server")
|
||||
monkeypatch.setattr(server, "load_plugins", lambda *a, **kw: None)
|
||||
monkeypatch.setattr(server, "startup_scan", lambda: None)
|
||||
|
||||
with TestClient(server.app) as client:
|
||||
with client.websocket_connect("/ws/sync/REALAPP") as a, \
|
||||
client.websocket_connect("/ws/sync/REALAPP") as b:
|
||||
a.send_text('{"type":"time","t":0}')
|
||||
assert b.receive_text() == '{"type":"time","t":0}'
|
||||
@@ -0,0 +1,191 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build & publish opt-in content packs (career venue media, rig VST slices).
|
||||
|
||||
Flat-zips a pack directory, sha256s it, and emits the ``{url, sha256, bytes}``
|
||||
block that the career and rig_builder download paths consume
|
||||
(``plugins/career/routes.py`` ``_download_pack``). Two modes:
|
||||
|
||||
--local <dir> write zips + a file:// manifest (dev/CI/tests; no network)
|
||||
--publish create/upload each pack's per-pack release; emit release URLs
|
||||
|
||||
The zip is flat (files at the archive root) to satisfy career's zip-slip guard
|
||||
(``PACK_FILENAME_RE``) and ``_validate_pack_dir``. This module is the reusable
|
||||
core the content-packs CI workflow calls, so building packs is automation —
|
||||
never a person's manual job.
|
||||
|
||||
Run ``python tools/content_packs.py --selfcheck`` for the built-in round-trip.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
REPO = "got-feedBack/feedBack" # where the content-packs release lives (public)
|
||||
|
||||
# Must mirror career's download-time whitelist (plugins/career/routes.py
|
||||
# PACK_FILENAME_RE). If the builder packs a name the downloader rejects (e.g. a
|
||||
# stray .DS_Store), the published pack fails _validate_pack_dir for every client.
|
||||
PACK_FILENAME_RE = re.compile(r"^[a-z0-9_-]{1,64}\.(mp4|webm|mp3|json)$")
|
||||
|
||||
|
||||
def build_pack(src_dir: Path, out_zip: Path) -> dict:
|
||||
"""Flat-zip every file directly under src_dir; return {sha256, bytes}.
|
||||
|
||||
Only regular files at the top level are included (venue packs are flat).
|
||||
Subdirectories are skipped — a nested tree would trip career's zip-slip
|
||||
guard on download anyway.
|
||||
|
||||
The build is REPRODUCIBLE: identical file contents always yield a
|
||||
byte-identical zip (fixed name order, fixed mtime, fixed permissions,
|
||||
ZIP_STORED). So a sha256 computed on any machine matches the zip the CI
|
||||
workflow or another contributor produces — anyone can precompute the
|
||||
manifest values without having to be the one who uploads the asset.
|
||||
"""
|
||||
files = sorted((p for p in src_dir.iterdir() if p.is_file()),
|
||||
key=lambda p: p.name)
|
||||
if not files:
|
||||
raise ValueError(f"no files to pack in {src_dir}")
|
||||
bad = [p.name for p in files if not PACK_FILENAME_RE.fullmatch(p.name)]
|
||||
if bad:
|
||||
raise ValueError(
|
||||
f"{src_dir}: files the downloader will reject: {bad} "
|
||||
f"(allowed: {PACK_FILENAME_RE.pattern})")
|
||||
out_zip.parent.mkdir(parents=True, exist_ok=True)
|
||||
with zipfile.ZipFile(out_zip, "w", zipfile.ZIP_STORED) as zf:
|
||||
# ZIP_STORED: the media (mp4/mp3) and .vst3 binaries are already
|
||||
# compressed; deflating just burns CPU for ~0 gain.
|
||||
for p in files:
|
||||
# Fixed mtime (the zip epoch, 1980-01-01) + fixed perms so the
|
||||
# bytes don't depend on the checkout's file timestamps.
|
||||
info = zipfile.ZipInfo(p.name, date_time=(1980, 1, 1, 0, 0, 0))
|
||||
info.compress_type = zipfile.ZIP_STORED
|
||||
# Pin create_system: ZipInfo defaults it from the host OS (0 on
|
||||
# Windows, 3 on Unix), which would otherwise make the same pack
|
||||
# hash differently across runners. 3 = Unix.
|
||||
info.create_system = 3
|
||||
info.external_attr = 0o644 << 16
|
||||
zf.writestr(info, p.read_bytes())
|
||||
data = out_zip.read_bytes()
|
||||
return {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)}
|
||||
|
||||
|
||||
def manifest_entry(out_zip: Path, url: str) -> dict:
|
||||
"""Pack info as the download-path expects it: {url, sha256, bytes}."""
|
||||
return {"url": url,
|
||||
"sha256": hashlib.sha256(out_zip.read_bytes()).hexdigest(),
|
||||
"bytes": out_zip.stat().st_size}
|
||||
|
||||
|
||||
# Per-pack, versioned, immutable release convention (matches what the team
|
||||
# already published, e.g. tag `venue-arena-v1` / asset `arena-pack-v1.zip`).
|
||||
def pack_tag(pack_id: str, version: int) -> str:
|
||||
return f"venue-{pack_id}-v{version}"
|
||||
|
||||
|
||||
def pack_asset(pack_id: str, version: int) -> str:
|
||||
return f"{pack_id}-pack-v{version}.zip"
|
||||
|
||||
|
||||
def pack_url(pack_id: str, version: int, repo: str = REPO) -> str:
|
||||
return (f"https://github.com/{repo}/releases/download/"
|
||||
f"{pack_tag(pack_id, version)}/{pack_asset(pack_id, version)}")
|
||||
|
||||
|
||||
def publish(pack_id: str, version: int, zip_path: Path, repo: str = REPO) -> None:
|
||||
"""Create the per-pack release if missing, then upload the versioned zip.
|
||||
|
||||
Tags are immutable: a media change means a new version (v1 → v2), never a
|
||||
re-upload — so no --clobber. gh errors if the asset already exists, which is
|
||||
the right guard against overwriting a published, referenced pack.
|
||||
"""
|
||||
tag = pack_tag(pack_id, version)
|
||||
if subprocess.run(["gh", "release", "view", tag, "--repo", repo],
|
||||
capture_output=True).returncode != 0:
|
||||
subprocess.run(
|
||||
["gh", "release", "create", tag, "--repo", repo, "--latest=false",
|
||||
"--title", f"{pack_id.capitalize()} venue pack v{version}",
|
||||
"--notes", "Opt-in career venue pack. Not a code release."],
|
||||
check=True)
|
||||
subprocess.run(
|
||||
["gh", "release", "upload", tag, str(zip_path), "--repo", repo], check=True)
|
||||
|
||||
|
||||
def _pack_id(src_dir: Path) -> str:
|
||||
return src_dir.name
|
||||
|
||||
|
||||
def main(argv=None) -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("src", nargs="*", type=Path,
|
||||
help="pack source dirs (e.g. plugins/career/venue-packs/club)")
|
||||
ap.add_argument("--version", type=int, default=1,
|
||||
help="pack version (tag venue-<id>-v<N>); default 1")
|
||||
ap.add_argument("--local", type=Path, metavar="DIR",
|
||||
help="write zips here + a file:// manifest.json; no upload")
|
||||
ap.add_argument("--publish", action="store_true",
|
||||
help="create/upload the per-pack release; emit release URLs")
|
||||
ap.add_argument("--manifest", type=Path,
|
||||
help="write the {id: {url,sha256,bytes}} map here (default: stdout)")
|
||||
ap.add_argument("--selfcheck", action="store_true", help="run the round-trip demo and exit")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.selfcheck:
|
||||
return _selfcheck()
|
||||
if not args.src or (not args.local and not args.publish):
|
||||
ap.error("need one or more src dirs and either --local or --publish")
|
||||
|
||||
out_dir = args.local if args.local else Path(args.src[0]).parent / "_packs"
|
||||
manifest = {}
|
||||
for src in args.src:
|
||||
pid = _pack_id(src)
|
||||
zip_path = out_dir / pack_asset(pid, args.version)
|
||||
build_pack(src, zip_path)
|
||||
if args.publish:
|
||||
publish(pid, args.version, zip_path)
|
||||
url = pack_url(pid, args.version)
|
||||
else:
|
||||
url = (out_dir.resolve() / zip_path.name).as_uri()
|
||||
manifest[pid] = manifest_entry(zip_path, url)
|
||||
|
||||
out = json.dumps(manifest, indent=2)
|
||||
if args.manifest:
|
||||
args.manifest.write_text(out + "\n", encoding="utf-8")
|
||||
else:
|
||||
print(out)
|
||||
return 0
|
||||
|
||||
|
||||
def _selfcheck() -> int:
|
||||
"""Build a pack and confirm build_pack/manifest_entry agree on the digest."""
|
||||
import tempfile
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
td = Path(td)
|
||||
src = td / "bar"
|
||||
src.mkdir()
|
||||
(src / "manifest.json").write_text('{"venue":"bar"}')
|
||||
(src / "bored.mp4").write_bytes(b"\x00fake-video")
|
||||
zip_path = td / pack_asset("bar", 1)
|
||||
info = build_pack(src, zip_path)
|
||||
# Reproducible: a second build (into a different path) is byte-identical.
|
||||
info2 = build_pack(src, td / "again.zip")
|
||||
assert info2["sha256"] == info["sha256"], "build is not reproducible"
|
||||
entry = manifest_entry(zip_path, pack_url("bar", 1))
|
||||
assert entry["sha256"] == info["sha256"], "digest mismatch"
|
||||
assert entry["bytes"] == info["bytes"]
|
||||
assert entry["url"] == (
|
||||
f"https://github.com/{REPO}/releases/download/venue-bar-v1/bar-pack-v1.zip")
|
||||
# Round-trip: the zip must be flat (names == basenames).
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
names = zf.namelist()
|
||||
assert set(names) == {"manifest.json", "bored.mp4"}, names
|
||||
print("content_packs selfcheck: ok")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user