mirror of
https://github.com/got-feedBack/feedBack.git
synced 2026-08-14 12:47:10 +00:00
Clean release snapshot
This commit is contained in:
@@ -0,0 +1,260 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-analyze"
|
||||||
|
description: "Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation."
|
||||||
|
argument-hint: "Optional focus areas for analysis"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/analyze.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before analysis)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_analyze` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Goal.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Identify inconsistencies, duplications, ambiguities, and underspecified items across the three core artifacts (`spec.md`, `plan.md`, `tasks.md`) before implementation. This command MUST run only after `/speckit-tasks` has successfully produced a complete `tasks.md`.
|
||||||
|
|
||||||
|
## Operating Constraints
|
||||||
|
|
||||||
|
**STRICTLY READ-ONLY**: Do **not** modify any files. Output a structured analysis report. Offer an optional remediation plan (user must explicitly approve before any follow-up editing commands would be invoked manually).
|
||||||
|
|
||||||
|
**Constitution Authority**: The project constitution (`.specify/memory/constitution.md`) is **non-negotiable** within this analysis scope. Constitution conflicts are automatically CRITICAL and require adjustment of the spec, plan, or tasks—not dilution, reinterpretation, or silent ignoring of the principle. If a principle itself needs to change, that must occur in a separate, explicit constitution update outside `/speckit-analyze`.
|
||||||
|
|
||||||
|
## Execution Steps
|
||||||
|
|
||||||
|
### 1. Initialize Analysis Context
|
||||||
|
|
||||||
|
Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` once from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS. Derive absolute paths:
|
||||||
|
|
||||||
|
- SPEC = FEATURE_DIR/spec.md
|
||||||
|
- PLAN = FEATURE_DIR/plan.md
|
||||||
|
- TASKS = FEATURE_DIR/tasks.md
|
||||||
|
|
||||||
|
Abort with an error message if any required file is missing (instruct the user to run missing prerequisite command).
|
||||||
|
For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
|
### 2. Load Artifacts (Progressive Disclosure)
|
||||||
|
|
||||||
|
Load only the minimal necessary context from each artifact:
|
||||||
|
|
||||||
|
**From spec.md:**
|
||||||
|
|
||||||
|
- Overview/Context
|
||||||
|
- Functional Requirements
|
||||||
|
- Success Criteria (measurable outcomes — e.g., performance, security, availability, user success, business impact)
|
||||||
|
- User Stories
|
||||||
|
- Edge Cases (if present)
|
||||||
|
|
||||||
|
**From plan.md:**
|
||||||
|
|
||||||
|
- Architecture/stack choices
|
||||||
|
- Data Model references
|
||||||
|
- Phases
|
||||||
|
- Technical constraints
|
||||||
|
|
||||||
|
**From tasks.md:**
|
||||||
|
|
||||||
|
- Task IDs
|
||||||
|
- Descriptions
|
||||||
|
- Phase grouping
|
||||||
|
- Parallel markers [P]
|
||||||
|
- Referenced file paths
|
||||||
|
|
||||||
|
**From constitution:**
|
||||||
|
|
||||||
|
- Load `.specify/memory/constitution.md` for principle validation
|
||||||
|
|
||||||
|
### 3. Build Semantic Models
|
||||||
|
|
||||||
|
Create internal representations (do not include raw artifacts in output):
|
||||||
|
|
||||||
|
- **Requirements inventory**: For each Functional Requirement (FR-###) and Success Criterion (SC-###), record a stable key. Use the explicit FR-/SC- identifier as the primary key when present, and optionally also derive an imperative-phrase slug for readability (e.g., "User can upload file" → `user-can-upload-file`). Include only Success Criteria items that require buildable work (e.g., load-testing infrastructure, security audit tooling), and exclude post-launch outcome metrics and business KPIs (e.g., "Reduce support tickets by 50%").
|
||||||
|
- **User story/action inventory**: Discrete user actions with acceptance criteria
|
||||||
|
- **Task coverage mapping**: Map each task to one or more requirements or stories (inference by keyword / explicit reference patterns like IDs or key phrases)
|
||||||
|
- **Constitution rule set**: Extract principle names and MUST/SHOULD normative statements
|
||||||
|
|
||||||
|
### 4. Detection Passes (Token-Efficient Analysis)
|
||||||
|
|
||||||
|
Focus on high-signal findings. Limit to 50 findings total; aggregate remainder in overflow summary.
|
||||||
|
|
||||||
|
#### A. Duplication Detection
|
||||||
|
|
||||||
|
- Identify near-duplicate requirements
|
||||||
|
- Mark lower-quality phrasing for consolidation
|
||||||
|
|
||||||
|
#### B. Ambiguity Detection
|
||||||
|
|
||||||
|
- Flag vague adjectives (fast, scalable, secure, intuitive, robust) lacking measurable criteria
|
||||||
|
- Flag unresolved placeholders (TODO, TKTK, ???, `<placeholder>`, etc.)
|
||||||
|
|
||||||
|
#### C. Underspecification
|
||||||
|
|
||||||
|
- Requirements with verbs but missing object or measurable outcome
|
||||||
|
- User stories missing acceptance criteria alignment
|
||||||
|
- Tasks referencing files or components not defined in spec/plan
|
||||||
|
|
||||||
|
#### D. Constitution Alignment
|
||||||
|
|
||||||
|
- Any requirement or plan element conflicting with a MUST principle
|
||||||
|
- Missing mandated sections or quality gates from constitution
|
||||||
|
|
||||||
|
#### E. Coverage Gaps
|
||||||
|
|
||||||
|
- Requirements with zero associated tasks
|
||||||
|
- Tasks with no mapped requirement/story
|
||||||
|
- Success Criteria requiring buildable work (performance, security, availability) not reflected in tasks
|
||||||
|
|
||||||
|
#### F. Inconsistency
|
||||||
|
|
||||||
|
- Terminology drift (same concept named differently across files)
|
||||||
|
- Data entities referenced in plan but absent in spec (or vice versa)
|
||||||
|
- Task ordering contradictions (e.g., integration tasks before foundational setup tasks without dependency note)
|
||||||
|
- Conflicting requirements (e.g., one requires Next.js while other specifies Vue)
|
||||||
|
|
||||||
|
### 5. Severity Assignment
|
||||||
|
|
||||||
|
Use this heuristic to prioritize findings:
|
||||||
|
|
||||||
|
- **CRITICAL**: Violates constitution MUST, missing core spec artifact, or requirement with zero coverage that blocks baseline functionality
|
||||||
|
- **HIGH**: Duplicate or conflicting requirement, ambiguous security/performance attribute, untestable acceptance criterion
|
||||||
|
- **MEDIUM**: Terminology drift, missing non-functional task coverage, underspecified edge case
|
||||||
|
- **LOW**: Style/wording improvements, minor redundancy not affecting execution order
|
||||||
|
|
||||||
|
### 6. Produce Compact Analysis Report
|
||||||
|
|
||||||
|
Output a Markdown report (no file writes) with the following structure:
|
||||||
|
|
||||||
|
## Specification Analysis Report
|
||||||
|
|
||||||
|
| ID | Category | Severity | Location(s) | Summary | Recommendation |
|
||||||
|
|----|----------|----------|-------------|---------|----------------|
|
||||||
|
| A1 | Duplication | HIGH | spec.md:L120-134 | Two similar requirements ... | Merge phrasing; keep clearer version |
|
||||||
|
|
||||||
|
(Add one row per finding; generate stable IDs prefixed by category initial.)
|
||||||
|
|
||||||
|
**Coverage Summary Table:**
|
||||||
|
|
||||||
|
| Requirement Key | Has Task? | Task IDs | Notes |
|
||||||
|
|-----------------|-----------|----------|-------|
|
||||||
|
|
||||||
|
**Constitution Alignment Issues:** (if any)
|
||||||
|
|
||||||
|
**Unmapped Tasks:** (if any)
|
||||||
|
|
||||||
|
**Metrics:**
|
||||||
|
|
||||||
|
- Total Requirements
|
||||||
|
- Total Tasks
|
||||||
|
- Coverage % (requirements with >=1 task)
|
||||||
|
- Ambiguity Count
|
||||||
|
- Duplication Count
|
||||||
|
- Critical Issues Count
|
||||||
|
|
||||||
|
### 7. Provide Next Actions
|
||||||
|
|
||||||
|
At end of report, output a concise Next Actions block:
|
||||||
|
|
||||||
|
- If CRITICAL issues exist: Recommend resolving before `/speckit-implement`
|
||||||
|
- If only LOW/MEDIUM: User may proceed, but provide improvement suggestions
|
||||||
|
- Provide explicit command suggestions: e.g., "Run /speckit-specify with refinement", "Run /speckit-plan to adjust architecture", "Manually edit tasks.md to add coverage for 'performance-metrics'"
|
||||||
|
|
||||||
|
### 8. Offer Remediation
|
||||||
|
|
||||||
|
Ask the user: "Would you like me to suggest concrete remediation edits for the top N issues?" (Do NOT apply them automatically.)
|
||||||
|
|
||||||
|
### 9. Check for extension hooks
|
||||||
|
|
||||||
|
After reporting, check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_analyze` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Operating Principles
|
||||||
|
|
||||||
|
### Context Efficiency
|
||||||
|
|
||||||
|
- **Minimal high-signal tokens**: Focus on actionable findings, not exhaustive documentation
|
||||||
|
- **Progressive disclosure**: Load artifacts incrementally; don't dump all content into analysis
|
||||||
|
- **Token-efficient output**: Limit findings table to 50 rows; summarize overflow
|
||||||
|
- **Deterministic results**: Rerunning without changes should produce consistent IDs and counts
|
||||||
|
|
||||||
|
### Analysis Guidelines
|
||||||
|
|
||||||
|
- **NEVER modify files** (this is read-only analysis)
|
||||||
|
- **NEVER hallucinate missing sections** (if absent, report them accurately)
|
||||||
|
- **Prioritize constitution violations** (these are always CRITICAL)
|
||||||
|
- **Use examples over exhaustive rules** (cite specific instances, not generic patterns)
|
||||||
|
- **Report zero issues gracefully** (emit success report with coverage statistics)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
$ARGUMENTS
|
||||||
@@ -0,0 +1,372 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-checklist"
|
||||||
|
description: "Generate a custom checklist for the current feature based on user requirements."
|
||||||
|
argument-hint: "Domain or focus area for the checklist"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/checklist.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## Checklist Purpose: "Unit Tests for English"
|
||||||
|
|
||||||
|
**CRITICAL CONCEPT**: Checklists are **UNIT TESTS FOR REQUIREMENTS WRITING** - they validate the quality, clarity, and completeness of requirements in a given domain.
|
||||||
|
|
||||||
|
**NOT for verification/testing**:
|
||||||
|
|
||||||
|
- ❌ NOT "Verify the button clicks correctly"
|
||||||
|
- ❌ NOT "Test error handling works"
|
||||||
|
- ❌ NOT "Confirm the API returns 200"
|
||||||
|
- ❌ NOT checking if code/implementation matches the spec
|
||||||
|
|
||||||
|
**FOR requirements quality validation**:
|
||||||
|
|
||||||
|
- ✅ "Are visual hierarchy requirements defined for all card types?" (completeness)
|
||||||
|
- ✅ "Is 'prominent display' quantified with specific sizing/positioning?" (clarity)
|
||||||
|
- ✅ "Are hover state requirements consistent across all interactive elements?" (consistency)
|
||||||
|
- ✅ "Are accessibility requirements defined for keyboard navigation?" (coverage)
|
||||||
|
- ✅ "Does the spec define what happens when logo image fails to load?" (edge cases)
|
||||||
|
|
||||||
|
**Metaphor**: If your spec is code written in English, the checklist is its unit test suite. You're testing whether the requirements are well-written, complete, unambiguous, and ready for implementation - NOT whether the implementation works.
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before checklist generation)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_checklist` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Execution Steps.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Execution Steps
|
||||||
|
|
||||||
|
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS list.
|
||||||
|
- All file paths must be absolute.
|
||||||
|
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
|
2. **Clarify intent (dynamic)**: Derive up to THREE initial contextual clarifying questions (no pre-baked catalog). They MUST:
|
||||||
|
- Be generated from the user's phrasing + extracted signals from spec/plan/tasks
|
||||||
|
- Only ask about information that materially changes checklist content
|
||||||
|
- Be skipped individually if already unambiguous in `$ARGUMENTS`
|
||||||
|
- Prefer precision over breadth
|
||||||
|
|
||||||
|
Generation algorithm:
|
||||||
|
1. Extract signals: feature domain keywords (e.g., auth, latency, UX, API), risk indicators ("critical", "must", "compliance"), stakeholder hints ("QA", "review", "security team"), and explicit deliverables ("a11y", "rollback", "contracts").
|
||||||
|
2. Cluster signals into candidate focus areas (max 4) ranked by relevance.
|
||||||
|
3. Identify probable audience & timing (author, reviewer, QA, release) if not explicit.
|
||||||
|
4. Detect missing dimensions: scope breadth, depth/rigor, risk emphasis, exclusion boundaries, measurable acceptance criteria.
|
||||||
|
5. Formulate questions chosen from these archetypes:
|
||||||
|
- Scope refinement (e.g., "Should this include integration touchpoints with X and Y or stay limited to local module correctness?")
|
||||||
|
- Risk prioritization (e.g., "Which of these potential risk areas should receive mandatory gating checks?")
|
||||||
|
- Depth calibration (e.g., "Is this a lightweight pre-commit sanity list or a formal release gate?")
|
||||||
|
- Audience framing (e.g., "Will this be used by the author only or peers during PR review?")
|
||||||
|
- Boundary exclusion (e.g., "Should we explicitly exclude performance tuning items this round?")
|
||||||
|
- Scenario class gap (e.g., "No recovery flows detected—are rollback / partial failure paths in scope?")
|
||||||
|
|
||||||
|
Question formatting rules:
|
||||||
|
- If presenting options, generate a compact table with columns: Option | Candidate | Why It Matters
|
||||||
|
- Limit to A–E options maximum; omit table if a free-form answer is clearer
|
||||||
|
- Never ask the user to restate what they already said
|
||||||
|
- Avoid speculative categories (no hallucination). If uncertain, ask explicitly: "Confirm whether X belongs in scope."
|
||||||
|
|
||||||
|
Defaults when interaction impossible:
|
||||||
|
- Depth: Standard
|
||||||
|
- Audience: Reviewer (PR) if code-related; Author otherwise
|
||||||
|
- Focus: Top 2 relevance clusters
|
||||||
|
|
||||||
|
Output the questions (label Q1/Q2/Q3). After answers: if ≥2 scenario classes (Alternate / Exception / Recovery / Non-Functional domain) remain unclear, you MAY ask up to TWO more targeted follow‑ups (Q4/Q5) with a one-line justification each (e.g., "Unresolved recovery path risk"). Do not exceed five total questions. Skip escalation if user explicitly declines more.
|
||||||
|
|
||||||
|
3. **Understand user request**: Combine `$ARGUMENTS` + clarifying answers:
|
||||||
|
- Derive checklist theme (e.g., security, review, deploy, ux)
|
||||||
|
- Consolidate explicit must-have items mentioned by user
|
||||||
|
- Map focus selections to category scaffolding
|
||||||
|
- Infer any missing context from spec/plan/tasks (do NOT hallucinate)
|
||||||
|
|
||||||
|
4. **Load feature context**: Read from FEATURE_DIR:
|
||||||
|
- spec.md: Feature requirements and scope
|
||||||
|
- plan.md (if exists): Technical details, dependencies
|
||||||
|
- tasks.md (if exists): Implementation tasks
|
||||||
|
|
||||||
|
**Context Loading Strategy**:
|
||||||
|
- Load only necessary portions relevant to active focus areas (avoid full-file dumping)
|
||||||
|
- Prefer summarizing long sections into concise scenario/requirement bullets
|
||||||
|
- Use progressive disclosure: add follow-on retrieval only if gaps detected
|
||||||
|
- If source docs are large, generate interim summary items instead of embedding raw text
|
||||||
|
|
||||||
|
5. **Generate checklist** - Create "Unit Tests for Requirements":
|
||||||
|
- Create `FEATURE_DIR/checklists/` directory if it doesn't exist
|
||||||
|
- Generate unique checklist filename:
|
||||||
|
- Use short, descriptive name based on domain (e.g., `ux.md`, `api.md`, `security.md`)
|
||||||
|
- Format: `[domain].md`
|
||||||
|
- File handling behavior:
|
||||||
|
- If file does NOT exist: Create new file and number items starting from CHK001
|
||||||
|
- If file exists: Append new items to existing file, continuing from the last CHK ID (e.g., if last item is CHK015, start new items at CHK016)
|
||||||
|
- Never delete or replace existing checklist content - always preserve and append
|
||||||
|
|
||||||
|
**CORE PRINCIPLE - Test the Requirements, Not the Implementation**:
|
||||||
|
Every checklist item MUST evaluate the REQUIREMENTS THEMSELVES for:
|
||||||
|
- **Completeness**: Are all necessary requirements present?
|
||||||
|
- **Clarity**: Are requirements unambiguous and specific?
|
||||||
|
- **Consistency**: Do requirements align with each other?
|
||||||
|
- **Measurability**: Can requirements be objectively verified?
|
||||||
|
- **Coverage**: Are all scenarios/edge cases addressed?
|
||||||
|
|
||||||
|
**Category Structure** - Group items by requirement quality dimensions:
|
||||||
|
- **Requirement Completeness** (Are all necessary requirements documented?)
|
||||||
|
- **Requirement Clarity** (Are requirements specific and unambiguous?)
|
||||||
|
- **Requirement Consistency** (Do requirements align without conflicts?)
|
||||||
|
- **Acceptance Criteria Quality** (Are success criteria measurable?)
|
||||||
|
- **Scenario Coverage** (Are all flows/cases addressed?)
|
||||||
|
- **Edge Case Coverage** (Are boundary conditions defined?)
|
||||||
|
- **Non-Functional Requirements** (Performance, Security, Accessibility, etc. - are they specified?)
|
||||||
|
- **Dependencies & Assumptions** (Are they documented and validated?)
|
||||||
|
- **Ambiguities & Conflicts** (What needs clarification?)
|
||||||
|
|
||||||
|
**HOW TO WRITE CHECKLIST ITEMS - "Unit Tests for English"**:
|
||||||
|
|
||||||
|
❌ **WRONG** (Testing implementation):
|
||||||
|
- "Verify landing page displays 3 episode cards"
|
||||||
|
- "Test hover states work on desktop"
|
||||||
|
- "Confirm logo click navigates home"
|
||||||
|
|
||||||
|
✅ **CORRECT** (Testing requirements quality):
|
||||||
|
- "Are the exact number and layout of featured episodes specified?" [Completeness]
|
||||||
|
- "Is 'prominent display' quantified with specific sizing/positioning?" [Clarity]
|
||||||
|
- "Are hover state requirements consistent across all interactive elements?" [Consistency]
|
||||||
|
- "Are keyboard navigation requirements defined for all interactive UI?" [Coverage]
|
||||||
|
- "Is the fallback behavior specified when logo image fails to load?" [Edge Cases]
|
||||||
|
- "Are loading states defined for asynchronous episode data?" [Completeness]
|
||||||
|
- "Does the spec define visual hierarchy for competing UI elements?" [Clarity]
|
||||||
|
|
||||||
|
**ITEM STRUCTURE**:
|
||||||
|
Each item should follow this pattern:
|
||||||
|
- Question format asking about requirement quality
|
||||||
|
- Focus on what's WRITTEN (or not written) in the spec/plan
|
||||||
|
- Include quality dimension in brackets [Completeness/Clarity/Consistency/etc.]
|
||||||
|
- Reference spec section `[Spec §X.Y]` when checking existing requirements
|
||||||
|
- Use `[Gap]` marker when checking for missing requirements
|
||||||
|
|
||||||
|
**EXAMPLES BY QUALITY DIMENSION**:
|
||||||
|
|
||||||
|
Completeness:
|
||||||
|
- "Are error handling requirements defined for all API failure modes? [Gap]"
|
||||||
|
- "Are accessibility requirements specified for all interactive elements? [Completeness]"
|
||||||
|
- "Are mobile breakpoint requirements defined for responsive layouts? [Gap]"
|
||||||
|
|
||||||
|
Clarity:
|
||||||
|
- "Is 'fast loading' quantified with specific timing thresholds? [Clarity, Spec §NFR-2]"
|
||||||
|
- "Are 'related episodes' selection criteria explicitly defined? [Clarity, Spec §FR-5]"
|
||||||
|
- "Is 'prominent' defined with measurable visual properties? [Ambiguity, Spec §FR-4]"
|
||||||
|
|
||||||
|
Consistency:
|
||||||
|
- "Do navigation requirements align across all pages? [Consistency, Spec §FR-10]"
|
||||||
|
- "Are card component requirements consistent between landing and detail pages? [Consistency]"
|
||||||
|
|
||||||
|
Coverage:
|
||||||
|
- "Are requirements defined for zero-state scenarios (no episodes)? [Coverage, Edge Case]"
|
||||||
|
- "Are concurrent user interaction scenarios addressed? [Coverage, Gap]"
|
||||||
|
- "Are requirements specified for partial data loading failures? [Coverage, Exception Flow]"
|
||||||
|
|
||||||
|
Measurability:
|
||||||
|
- "Are visual hierarchy requirements measurable/testable? [Acceptance Criteria, Spec §FR-1]"
|
||||||
|
- "Can 'balanced visual weight' be objectively verified? [Measurability, Spec §FR-2]"
|
||||||
|
|
||||||
|
**Scenario Classification & Coverage** (Requirements Quality Focus):
|
||||||
|
- Check if requirements exist for: Primary, Alternate, Exception/Error, Recovery, Non-Functional scenarios
|
||||||
|
- For each scenario class, ask: "Are [scenario type] requirements complete, clear, and consistent?"
|
||||||
|
- If scenario class missing: "Are [scenario type] requirements intentionally excluded or missing? [Gap]"
|
||||||
|
- Include resilience/rollback when state mutation occurs: "Are rollback requirements defined for migration failures? [Gap]"
|
||||||
|
|
||||||
|
**Traceability Requirements**:
|
||||||
|
- MINIMUM: ≥80% of items MUST include at least one traceability reference
|
||||||
|
- Each item should reference: spec section `[Spec §X.Y]`, or use markers: `[Gap]`, `[Ambiguity]`, `[Conflict]`, `[Assumption]`
|
||||||
|
- If no ID system exists: "Is a requirement & acceptance criteria ID scheme established? [Traceability]"
|
||||||
|
|
||||||
|
**Surface & Resolve Issues** (Requirements Quality Problems):
|
||||||
|
Ask questions about the requirements themselves:
|
||||||
|
- Ambiguities: "Is the term 'fast' quantified with specific metrics? [Ambiguity, Spec §NFR-1]"
|
||||||
|
- Conflicts: "Do navigation requirements conflict between §FR-10 and §FR-10a? [Conflict]"
|
||||||
|
- Assumptions: "Is the assumption of 'always available podcast API' validated? [Assumption]"
|
||||||
|
- Dependencies: "Are external podcast API requirements documented? [Dependency, Gap]"
|
||||||
|
- Missing definitions: "Is 'visual hierarchy' defined with measurable criteria? [Gap]"
|
||||||
|
|
||||||
|
**Content Consolidation**:
|
||||||
|
- Soft cap: If raw candidate items > 40, prioritize by risk/impact
|
||||||
|
- Merge near-duplicates checking the same requirement aspect
|
||||||
|
- If >5 low-impact edge cases, create one item: "Are edge cases X, Y, Z addressed in requirements? [Coverage]"
|
||||||
|
|
||||||
|
**🚫 ABSOLUTELY PROHIBITED** - These make it an implementation test, not a requirements test:
|
||||||
|
- ❌ Any item starting with "Verify", "Test", "Confirm", "Check" + implementation behavior
|
||||||
|
- ❌ References to code execution, user actions, system behavior
|
||||||
|
- ❌ "Displays correctly", "works properly", "functions as expected"
|
||||||
|
- ❌ "Click", "navigate", "render", "load", "execute"
|
||||||
|
- ❌ Test cases, test plans, QA procedures
|
||||||
|
- ❌ Implementation details (frameworks, APIs, algorithms)
|
||||||
|
|
||||||
|
**✅ REQUIRED PATTERNS** - These test requirements quality:
|
||||||
|
- ✅ "Are [requirement type] defined/specified/documented for [scenario]?"
|
||||||
|
- ✅ "Is [vague term] quantified/clarified with specific criteria?"
|
||||||
|
- ✅ "Are requirements consistent between [section A] and [section B]?"
|
||||||
|
- ✅ "Can [requirement] be objectively measured/verified?"
|
||||||
|
- ✅ "Are [edge cases/scenarios] addressed in requirements?"
|
||||||
|
- ✅ "Does the spec define [missing aspect]?"
|
||||||
|
|
||||||
|
6. **Structure Reference**: Generate the checklist following the canonical template in `.specify/templates/checklist-template.md` for title, meta section, category headings, and ID formatting. If template is unavailable, use: H1 title, purpose/created meta lines, `##` category sections containing `- [ ] CHK### <requirement item>` lines with globally incrementing IDs starting at CHK001.
|
||||||
|
|
||||||
|
7. **Report**: Output full path to checklist file, item count, and summarize whether the run created a new file or appended to an existing one. Summarize:
|
||||||
|
- Focus areas selected
|
||||||
|
- Depth level
|
||||||
|
- Actor/timing
|
||||||
|
- Any explicit user-specified must-have items incorporated
|
||||||
|
|
||||||
|
**Important**: Each `/speckit-checklist` command invocation uses a short, descriptive checklist filename and either creates a new file or appends to an existing one. This allows:
|
||||||
|
|
||||||
|
- Multiple checklists of different types (e.g., `ux.md`, `test.md`, `security.md`)
|
||||||
|
- Simple, memorable filenames that indicate checklist purpose
|
||||||
|
- Easy identification and navigation in the `checklists/` folder
|
||||||
|
|
||||||
|
To avoid clutter, use descriptive types and clean up obsolete checklists when done.
|
||||||
|
|
||||||
|
## Example Checklist Types & Sample Items
|
||||||
|
|
||||||
|
**UX Requirements Quality:** `ux.md`
|
||||||
|
|
||||||
|
Sample items (testing the requirements, NOT the implementation):
|
||||||
|
|
||||||
|
- "Are visual hierarchy requirements defined with measurable criteria? [Clarity, Spec §FR-1]"
|
||||||
|
- "Is the number and positioning of UI elements explicitly specified? [Completeness, Spec §FR-1]"
|
||||||
|
- "Are interaction state requirements (hover, focus, active) consistently defined? [Consistency]"
|
||||||
|
- "Are accessibility requirements specified for all interactive elements? [Coverage, Gap]"
|
||||||
|
- "Is fallback behavior defined when images fail to load? [Edge Case, Gap]"
|
||||||
|
- "Can 'prominent display' be objectively measured? [Measurability, Spec §FR-4]"
|
||||||
|
|
||||||
|
**API Requirements Quality:** `api.md`
|
||||||
|
|
||||||
|
Sample items:
|
||||||
|
|
||||||
|
- "Are error response formats specified for all failure scenarios? [Completeness]"
|
||||||
|
- "Are rate limiting requirements quantified with specific thresholds? [Clarity]"
|
||||||
|
- "Are authentication requirements consistent across all endpoints? [Consistency]"
|
||||||
|
- "Are retry/timeout requirements defined for external dependencies? [Coverage, Gap]"
|
||||||
|
- "Is versioning strategy documented in requirements? [Gap]"
|
||||||
|
|
||||||
|
**Performance Requirements Quality:** `performance.md`
|
||||||
|
|
||||||
|
Sample items:
|
||||||
|
|
||||||
|
- "Are performance requirements quantified with specific metrics? [Clarity]"
|
||||||
|
- "Are performance targets defined for all critical user journeys? [Coverage]"
|
||||||
|
- "Are performance requirements under different load conditions specified? [Completeness]"
|
||||||
|
- "Can performance requirements be objectively measured? [Measurability]"
|
||||||
|
- "Are degradation requirements defined for high-load scenarios? [Edge Case, Gap]"
|
||||||
|
|
||||||
|
**Security Requirements Quality:** `security.md`
|
||||||
|
|
||||||
|
Sample items:
|
||||||
|
|
||||||
|
- "Are authentication requirements specified for all protected resources? [Coverage]"
|
||||||
|
- "Are data protection requirements defined for sensitive information? [Completeness]"
|
||||||
|
- "Is the threat model documented and requirements aligned to it? [Traceability]"
|
||||||
|
- "Are security requirements consistent with compliance obligations? [Consistency]"
|
||||||
|
- "Are security failure/breach response requirements defined? [Gap, Exception Flow]"
|
||||||
|
|
||||||
|
## Anti-Examples: What NOT To Do
|
||||||
|
|
||||||
|
**❌ WRONG - These test implementation, not requirements:**
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- [ ] CHK001 - Verify landing page displays 3 episode cards [Spec §FR-001]
|
||||||
|
- [ ] CHK002 - Test hover states work correctly on desktop [Spec §FR-003]
|
||||||
|
- [ ] CHK003 - Confirm logo click navigates to home page [Spec §FR-010]
|
||||||
|
- [ ] CHK004 - Check that related episodes section shows 3-5 items [Spec §FR-005]
|
||||||
|
```
|
||||||
|
|
||||||
|
**✅ CORRECT - These test requirements quality:**
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- [ ] CHK001 - Are the number and layout of featured episodes explicitly specified? [Completeness, Spec §FR-001]
|
||||||
|
- [ ] CHK002 - Are hover state requirements consistently defined for all interactive elements? [Consistency, Spec §FR-003]
|
||||||
|
- [ ] CHK003 - Are navigation requirements clear for all clickable brand elements? [Clarity, Spec §FR-010]
|
||||||
|
- [ ] CHK004 - Is the selection criteria for related episodes documented? [Gap, Spec §FR-005]
|
||||||
|
- [ ] CHK005 - Are loading state requirements defined for asynchronous episode data? [Gap]
|
||||||
|
- [ ] CHK006 - Can "visual hierarchy" requirements be objectively measured? [Measurability, Spec §FR-001]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key Differences:**
|
||||||
|
|
||||||
|
- Wrong: Tests if the system works correctly
|
||||||
|
- Correct: Tests if the requirements are written correctly
|
||||||
|
- Wrong: Verification of behavior
|
||||||
|
- Correct: Validation of requirement quality
|
||||||
|
- Wrong: "Does it do X?"
|
||||||
|
- Correct: "Is X clearly specified?"
|
||||||
|
|
||||||
|
## Post-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (after checklist generation)**:
|
||||||
|
Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_checklist` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
@@ -0,0 +1,254 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-clarify"
|
||||||
|
description: "Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec."
|
||||||
|
argument-hint: "Optional areas to clarify in the spec"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/clarify.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before clarification)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_clarify` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
Goal: Detect and reduce ambiguity or missing decision points in the active feature specification and record the clarifications directly in the spec file.
|
||||||
|
|
||||||
|
Note: This clarification workflow is expected to run (and be completed) BEFORE invoking `/speckit-plan`. If the user explicitly states they are skipping clarification (e.g., exploratory spike), you may proceed, but must warn that downstream rework risk increases.
|
||||||
|
|
||||||
|
Execution steps:
|
||||||
|
|
||||||
|
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --paths-only` from repo root **once** (combined `--json --paths-only` mode / `-Json -PathsOnly`). Parse minimal JSON payload fields:
|
||||||
|
- `FEATURE_DIR`
|
||||||
|
- `FEATURE_SPEC`
|
||||||
|
- (Optionally capture `IMPL_PLAN`, `TASKS` for future chained flows.)
|
||||||
|
- If JSON parsing fails, abort and instruct user to re-run `/speckit-specify` or verify feature branch environment.
|
||||||
|
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
|
2. Load the current spec file. Perform a structured ambiguity & coverage scan using this taxonomy. For each category, mark status: Clear / Partial / Missing. Produce an internal coverage map used for prioritization (do not output raw map unless no questions will be asked).
|
||||||
|
|
||||||
|
Functional Scope & Behavior:
|
||||||
|
- Core user goals & success criteria
|
||||||
|
- Explicit out-of-scope declarations
|
||||||
|
- User roles / personas differentiation
|
||||||
|
|
||||||
|
Domain & Data Model:
|
||||||
|
- Entities, attributes, relationships
|
||||||
|
- Identity & uniqueness rules
|
||||||
|
- Lifecycle/state transitions
|
||||||
|
- Data volume / scale assumptions
|
||||||
|
|
||||||
|
Interaction & UX Flow:
|
||||||
|
- Critical user journeys / sequences
|
||||||
|
- Error/empty/loading states
|
||||||
|
- Accessibility or localization notes
|
||||||
|
|
||||||
|
Non-Functional Quality Attributes:
|
||||||
|
- Performance (latency, throughput targets)
|
||||||
|
- Scalability (horizontal/vertical, limits)
|
||||||
|
- Reliability & availability (uptime, recovery expectations)
|
||||||
|
- Observability (logging, metrics, tracing signals)
|
||||||
|
- Security & privacy (authN/Z, data protection, threat assumptions)
|
||||||
|
- Compliance / regulatory constraints (if any)
|
||||||
|
|
||||||
|
Integration & External Dependencies:
|
||||||
|
- External services/APIs and failure modes
|
||||||
|
- Data import/export formats
|
||||||
|
- Protocol/versioning assumptions
|
||||||
|
|
||||||
|
Edge Cases & Failure Handling:
|
||||||
|
- Negative scenarios
|
||||||
|
- Rate limiting / throttling
|
||||||
|
- Conflict resolution (e.g., concurrent edits)
|
||||||
|
|
||||||
|
Constraints & Tradeoffs:
|
||||||
|
- Technical constraints (language, storage, hosting)
|
||||||
|
- Explicit tradeoffs or rejected alternatives
|
||||||
|
|
||||||
|
Terminology & Consistency:
|
||||||
|
- Canonical glossary terms
|
||||||
|
- Avoided synonyms / deprecated terms
|
||||||
|
|
||||||
|
Completion Signals:
|
||||||
|
- Acceptance criteria testability
|
||||||
|
- Measurable Definition of Done style indicators
|
||||||
|
|
||||||
|
Misc / Placeholders:
|
||||||
|
- TODO markers / unresolved decisions
|
||||||
|
- Ambiguous adjectives ("robust", "intuitive") lacking quantification
|
||||||
|
|
||||||
|
For each category with Partial or Missing status, add a candidate question opportunity unless:
|
||||||
|
- Clarification would not materially change implementation or validation strategy
|
||||||
|
- Information is better deferred to planning phase (note internally)
|
||||||
|
|
||||||
|
3. Generate (internally) a prioritized queue of candidate clarification questions (maximum 5). Do NOT output them all at once. Apply these constraints:
|
||||||
|
- Maximum of 5 total questions across the whole session.
|
||||||
|
- Each question must be answerable with EITHER:
|
||||||
|
- A short multiple‑choice selection (2–5 distinct, mutually exclusive options), OR
|
||||||
|
- A one-word / short‑phrase answer (explicitly constrain: "Answer in <=5 words").
|
||||||
|
- Only include questions whose answers materially impact architecture, data modeling, task decomposition, test design, UX behavior, operational readiness, or compliance validation.
|
||||||
|
- Ensure category coverage balance: attempt to cover the highest impact unresolved categories first; avoid asking two low-impact questions when a single high-impact area (e.g., security posture) is unresolved.
|
||||||
|
- Exclude questions already answered, trivial stylistic preferences, or plan-level execution details (unless blocking correctness).
|
||||||
|
- Favor clarifications that reduce downstream rework risk or prevent misaligned acceptance tests.
|
||||||
|
- If more than 5 categories remain unresolved, select the top 5 by (Impact * Uncertainty) heuristic.
|
||||||
|
|
||||||
|
4. Sequential questioning loop (interactive):
|
||||||
|
- Present EXACTLY ONE question at a time.
|
||||||
|
- For multiple‑choice questions:
|
||||||
|
- **Analyze all options** and determine the **most suitable option** based on:
|
||||||
|
- Best practices for the project type
|
||||||
|
- Common patterns in similar implementations
|
||||||
|
- Risk reduction (security, performance, maintainability)
|
||||||
|
- Alignment with any explicit project goals or constraints visible in the spec
|
||||||
|
- Present your **recommended option prominently** at the top with clear reasoning (1-2 sentences explaining why this is the best choice).
|
||||||
|
- Format as: `**Recommended:** Option [X] - <reasoning>`
|
||||||
|
- Then render all options as a Markdown table:
|
||||||
|
|
||||||
|
| Option | Description |
|
||||||
|
|--------|-------------|
|
||||||
|
| A | <Option A description> |
|
||||||
|
| B | <Option B description> |
|
||||||
|
| C | <Option C description> (add D/E as needed up to 5) |
|
||||||
|
| Short | Provide a different short answer (<=5 words) (Include only if free-form alternative is appropriate) |
|
||||||
|
|
||||||
|
- After the table, add: `You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.`
|
||||||
|
- For short‑answer style (no meaningful discrete options):
|
||||||
|
- Provide your **suggested answer** based on best practices and context.
|
||||||
|
- Format as: `**Suggested:** <your proposed answer> - <brief reasoning>`
|
||||||
|
- Then output: `Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.`
|
||||||
|
- After the user answers:
|
||||||
|
- If the user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer.
|
||||||
|
- Otherwise, validate the answer maps to one option or fits the <=5 word constraint.
|
||||||
|
- If ambiguous, ask for a quick disambiguation (count still belongs to same question; do not advance).
|
||||||
|
- Once satisfactory, record it in working memory (do not yet write to disk) and move to the next queued question.
|
||||||
|
- Stop asking further questions when:
|
||||||
|
- All critical ambiguities resolved early (remaining queued items become unnecessary), OR
|
||||||
|
- User signals completion ("done", "good", "no more"), OR
|
||||||
|
- You reach 5 asked questions.
|
||||||
|
- Never reveal future queued questions in advance.
|
||||||
|
- If no valid questions exist at start, immediately report no critical ambiguities.
|
||||||
|
|
||||||
|
5. Integration after EACH accepted answer (incremental update approach):
|
||||||
|
- Maintain in-memory representation of the spec (loaded once at start) plus the raw file contents.
|
||||||
|
- For the first integrated answer in this session:
|
||||||
|
- Ensure a `## Clarifications` section exists (create it just after the highest-level contextual/overview section per the spec template if missing).
|
||||||
|
- Under it, create (if not present) a `### Session YYYY-MM-DD` subheading for today.
|
||||||
|
- Append a bullet line immediately after acceptance: `- Q: <question> → A: <final answer>`.
|
||||||
|
- Then immediately apply the clarification to the most appropriate section(s):
|
||||||
|
- Functional ambiguity → Update or add a bullet in Functional Requirements.
|
||||||
|
- User interaction / actor distinction → Update User Stories or Actors subsection (if present) with clarified role, constraint, or scenario.
|
||||||
|
- Data shape / entities → Update Data Model (add fields, types, relationships) preserving ordering; note added constraints succinctly.
|
||||||
|
- Non-functional constraint → Add/modify measurable criteria in Success Criteria > Measurable Outcomes (convert vague adjective to metric or explicit target).
|
||||||
|
- Edge case / negative flow → Add a new bullet under Edge Cases / Error Handling (or create such subsection if template provides placeholder for it).
|
||||||
|
- Terminology conflict → Normalize term across spec; retain original only if necessary by adding `(formerly referred to as "X")` once.
|
||||||
|
- If the clarification invalidates an earlier ambiguous statement, replace that statement instead of duplicating; leave no obsolete contradictory text.
|
||||||
|
- Save the spec file AFTER each integration to minimize risk of context loss (atomic overwrite).
|
||||||
|
- Preserve formatting: do not reorder unrelated sections; keep heading hierarchy intact.
|
||||||
|
- Keep each inserted clarification minimal and testable (avoid narrative drift).
|
||||||
|
|
||||||
|
6. Validation (performed after EACH write plus final pass):
|
||||||
|
- Clarifications session contains exactly one bullet per accepted answer (no duplicates).
|
||||||
|
- Total asked (accepted) questions ≤ 5.
|
||||||
|
- Updated sections contain no lingering vague placeholders the new answer was meant to resolve.
|
||||||
|
- No contradictory earlier statement remains (scan for now-invalid alternative choices removed).
|
||||||
|
- Markdown structure valid; only allowed new headings: `## Clarifications`, `### Session YYYY-MM-DD`.
|
||||||
|
- Terminology consistency: same canonical term used across all updated sections.
|
||||||
|
|
||||||
|
7. Write the updated spec back to `FEATURE_SPEC`.
|
||||||
|
|
||||||
|
8. Report completion (after questioning loop ends or early termination):
|
||||||
|
- Number of questions asked & answered.
|
||||||
|
- Path to updated spec.
|
||||||
|
- Sections touched (list names).
|
||||||
|
- Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact).
|
||||||
|
- If any Outstanding or Deferred remain, recommend whether to proceed to `/speckit-plan` or run `/speckit-clarify` again later post-plan.
|
||||||
|
- Suggested next command.
|
||||||
|
|
||||||
|
Behavior rules:
|
||||||
|
|
||||||
|
- If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding.
|
||||||
|
- If spec file missing, instruct user to run `/speckit-specify` first (do not create a new spec here).
|
||||||
|
- Never exceed 5 total asked questions (clarification retries for a single question do not count as new questions).
|
||||||
|
- Avoid speculative tech stack questions unless the absence blocks functional clarity.
|
||||||
|
- Respect user early termination signals ("stop", "done", "proceed").
|
||||||
|
- If no questions asked due to full coverage, output a compact coverage summary (all categories Clear) then suggest advancing.
|
||||||
|
- If quota reached with unresolved high-impact categories remaining, explicitly flag them under Deferred with rationale.
|
||||||
|
|
||||||
|
Context for prioritization: $ARGUMENTS
|
||||||
|
|
||||||
|
## Post-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (after clarification)**:
|
||||||
|
Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_clarify` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
@@ -0,0 +1,157 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-constitution"
|
||||||
|
description: "Create or update the project constitution from interactive or provided principle inputs, ensuring all dependent templates stay in sync."
|
||||||
|
argument-hint: "Principles or values for the project constitution"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/constitution.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before constitution update)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_constitution` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
You are updating the project constitution at `.specify/memory/constitution.md`. This file is a TEMPLATE containing placeholder tokens in square brackets (e.g. `[PROJECT_NAME]`, `[PRINCIPLE_1_NAME]`). Your job is to (a) collect/derive concrete values, (b) fill the template precisely, and (c) propagate any amendments across dependent artifacts.
|
||||||
|
|
||||||
|
**Note**: If `.specify/memory/constitution.md` does not exist yet, it should have been initialized from `.specify/templates/constitution-template.md` during project setup. If it's missing, copy the template first.
|
||||||
|
|
||||||
|
Follow this execution flow:
|
||||||
|
|
||||||
|
1. Load the existing constitution at `.specify/memory/constitution.md`.
|
||||||
|
- Identify every placeholder token of the form `[ALL_CAPS_IDENTIFIER]`.
|
||||||
|
**IMPORTANT**: The user might require less or more principles than the ones used in the template. If a number is specified, respect that - follow the general template. You will update the doc accordingly.
|
||||||
|
|
||||||
|
2. Collect/derive values for placeholders:
|
||||||
|
- If user input (conversation) supplies a value, use it.
|
||||||
|
- Otherwise infer from existing repo context (README, docs, prior constitution versions if embedded).
|
||||||
|
- For governance dates: `RATIFICATION_DATE` is the original adoption date (if unknown ask or mark TODO), `LAST_AMENDED_DATE` is today if changes are made, otherwise keep previous.
|
||||||
|
- `CONSTITUTION_VERSION` must increment according to semantic versioning rules:
|
||||||
|
- MAJOR: Backward incompatible governance/principle removals or redefinitions.
|
||||||
|
- MINOR: New principle/section added or materially expanded guidance.
|
||||||
|
- PATCH: Clarifications, wording, typo fixes, non-semantic refinements.
|
||||||
|
- If version bump type ambiguous, propose reasoning before finalizing.
|
||||||
|
|
||||||
|
3. Draft the updated constitution content:
|
||||||
|
- Replace every placeholder with concrete text (no bracketed tokens left except intentionally retained template slots that the project has chosen not to define yet—explicitly justify any left).
|
||||||
|
- Preserve heading hierarchy and comments can be removed once replaced unless they still add clarifying guidance.
|
||||||
|
- Ensure each Principle section: succinct name line, paragraph (or bullet list) capturing non‑negotiable rules, explicit rationale if not obvious.
|
||||||
|
- Ensure Governance section lists amendment procedure, versioning policy, and compliance review expectations.
|
||||||
|
|
||||||
|
4. Consistency propagation checklist (convert prior checklist into active validations):
|
||||||
|
- Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align with updated principles.
|
||||||
|
- Read `.specify/templates/spec-template.md` for scope/requirements alignment—update if constitution adds/removes mandatory sections or constraints.
|
||||||
|
- Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or removed principle-driven task types (e.g., observability, versioning, testing discipline).
|
||||||
|
- Read each command file in `.specify/templates/commands/*.md` (including this one) to verify no outdated references (agent-specific names like CLAUDE only) remain when generic guidance is required.
|
||||||
|
- Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific guidance files if present). Update references to principles changed.
|
||||||
|
|
||||||
|
5. Produce a Sync Impact Report (prepend as an HTML comment at top of the constitution file after update):
|
||||||
|
- Version change: old → new
|
||||||
|
- List of modified principles (old title → new title if renamed)
|
||||||
|
- Added sections
|
||||||
|
- Removed sections
|
||||||
|
- Templates requiring updates (✅ updated / ⚠ pending) with file paths
|
||||||
|
- Follow-up TODOs if any placeholders intentionally deferred.
|
||||||
|
|
||||||
|
6. Validation before final output:
|
||||||
|
- No remaining unexplained bracket tokens.
|
||||||
|
- Version line matches report.
|
||||||
|
- Dates ISO format YYYY-MM-DD.
|
||||||
|
- Principles are declarative, testable, and free of vague language ("should" → replace with MUST/SHOULD rationale where appropriate).
|
||||||
|
|
||||||
|
7. Write the completed constitution back to `.specify/memory/constitution.md` (overwrite).
|
||||||
|
|
||||||
|
8. Output a final summary to the user with:
|
||||||
|
- New version and bump rationale.
|
||||||
|
- Any files flagged for manual follow-up.
|
||||||
|
- Suggested commit message (e.g., `docs: amend constitution to vX.Y.Z (principle additions + governance update)`).
|
||||||
|
|
||||||
|
Formatting & Style Requirements:
|
||||||
|
|
||||||
|
- Use Markdown headings exactly as in the template (do not demote/promote levels).
|
||||||
|
- Wrap long rationale lines to keep readability (<100 chars ideally) but do not hard enforce with awkward breaks.
|
||||||
|
- Keep a single blank line between sections.
|
||||||
|
- Avoid trailing whitespace.
|
||||||
|
|
||||||
|
If the user supplies partial updates (e.g., only one principle revision), still perform validation and version decision steps.
|
||||||
|
|
||||||
|
If critical info missing (e.g., ratification date truly unknown), insert `TODO(<FIELD_NAME>): explanation` and include in the Sync Impact Report under deferred items.
|
||||||
|
|
||||||
|
Do not create a new template; always operate on the existing `.specify/memory/constitution.md` file.
|
||||||
|
|
||||||
|
## Post-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (after constitution update)**:
|
||||||
|
Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_constitution` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
@@ -0,0 +1,210 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-implement"
|
||||||
|
description: "Execute the implementation plan by processing and executing all tasks defined in tasks.md"
|
||||||
|
argument-hint: "Optional implementation guidance or task filter"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/implement.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before implementation)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_implement` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
|
2. **Check checklists status** (if FEATURE_DIR/checklists/ exists):
|
||||||
|
- Scan all checklist files in the checklists/ directory
|
||||||
|
- For each checklist, count:
|
||||||
|
- Total items: All lines matching `- [ ]` or `- [X]` or `- [x]`
|
||||||
|
- Completed items: Lines matching `- [X]` or `- [x]`
|
||||||
|
- Incomplete items: Lines matching `- [ ]`
|
||||||
|
- Create a status table:
|
||||||
|
|
||||||
|
```text
|
||||||
|
| Checklist | Total | Completed | Incomplete | Status |
|
||||||
|
|-----------|-------|-----------|------------|--------|
|
||||||
|
| ux.md | 12 | 12 | 0 | ✓ PASS |
|
||||||
|
| test.md | 8 | 5 | 3 | ✗ FAIL |
|
||||||
|
| security.md | 6 | 6 | 0 | ✓ PASS |
|
||||||
|
```
|
||||||
|
|
||||||
|
- Calculate overall status:
|
||||||
|
- **PASS**: All checklists have 0 incomplete items
|
||||||
|
- **FAIL**: One or more checklists have incomplete items
|
||||||
|
|
||||||
|
- **If any checklist is incomplete**:
|
||||||
|
- Display the table with incomplete item counts
|
||||||
|
- **STOP** and ask: "Some checklists are incomplete. Do you want to proceed with implementation anyway? (yes/no)"
|
||||||
|
- Wait for user response before continuing
|
||||||
|
- If user says "no" or "wait" or "stop", halt execution
|
||||||
|
- If user says "yes" or "proceed" or "continue", proceed to step 3
|
||||||
|
|
||||||
|
- **If all checklists are complete**:
|
||||||
|
- Display the table showing all checklists passed
|
||||||
|
- Automatically proceed to step 3
|
||||||
|
|
||||||
|
3. Load and analyze the implementation context:
|
||||||
|
- **REQUIRED**: Read tasks.md for the complete task list and execution plan
|
||||||
|
- **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
|
||||||
|
- **IF EXISTS**: Read data-model.md for entities and relationships
|
||||||
|
- **IF EXISTS**: Read contracts/ for API specifications and test requirements
|
||||||
|
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
||||||
|
- **IF EXISTS**: Read .specify/memory/constitution.md for governance constraints
|
||||||
|
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
||||||
|
|
||||||
|
4. **Project Setup Verification**:
|
||||||
|
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
||||||
|
|
||||||
|
**Detection & Creation Logic**:
|
||||||
|
- Check if the following command succeeds to determine if the repository is a git repo (create/verify .gitignore if so):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git rev-parse --git-dir 2>/dev/null
|
||||||
|
```
|
||||||
|
|
||||||
|
- Check if Dockerfile* exists or Docker in plan.md → create/verify .dockerignore
|
||||||
|
- Check if .eslintrc* exists → create/verify .eslintignore
|
||||||
|
- Check if eslint.config.* exists → ensure the config's `ignores` entries cover required patterns
|
||||||
|
- Check if .prettierrc* exists → create/verify .prettierignore
|
||||||
|
- Check if .npmrc or package.json exists → create/verify .npmignore (if publishing)
|
||||||
|
- Check if terraform files (*.tf) exist → create/verify .terraformignore
|
||||||
|
- Check if .helmignore needed (helm charts present) → create/verify .helmignore
|
||||||
|
|
||||||
|
**If ignore file already exists**: Verify it contains essential patterns, append missing critical patterns only
|
||||||
|
**If ignore file missing**: Create with full pattern set for detected technology
|
||||||
|
|
||||||
|
**Common Patterns by Technology** (from plan.md tech stack):
|
||||||
|
- **Node.js/JavaScript/TypeScript**: `node_modules/`, `dist/`, `build/`, `*.log`, `.env*`
|
||||||
|
- **Python**: `__pycache__/`, `*.pyc`, `.venv/`, `venv/`, `dist/`, `*.egg-info/`
|
||||||
|
- **Java**: `target/`, `*.class`, `*.jar`, `.gradle/`, `build/`
|
||||||
|
- **C#/.NET**: `bin/`, `obj/`, `*.user`, `*.suo`, `packages/`
|
||||||
|
- **Go**: `*.exe`, `*.test`, `vendor/`, `*.out`
|
||||||
|
- **Ruby**: `.bundle/`, `log/`, `tmp/`, `*.gem`, `vendor/bundle/`
|
||||||
|
- **PHP**: `vendor/`, `*.log`, `*.cache`, `*.env`
|
||||||
|
- **Rust**: `target/`, `debug/`, `release/`, `*.rs.bk`, `*.rlib`, `*.prof*`, `.idea/`, `*.log`, `.env*`
|
||||||
|
- **Kotlin**: `build/`, `out/`, `.gradle/`, `.idea/`, `*.class`, `*.jar`, `*.iml`, `*.log`, `.env*`
|
||||||
|
- **C++**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.so`, `*.a`, `*.exe`, `*.dll`, `.idea/`, `*.log`, `.env*`
|
||||||
|
- **C**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.a`, `*.so`, `*.exe`, `*.dll`, `autom4te.cache/`, `config.status`, `config.log`, `.idea/`, `*.log`, `.env*`
|
||||||
|
- **Swift**: `.build/`, `DerivedData/`, `*.swiftpm/`, `Packages/`
|
||||||
|
- **R**: `.Rproj.user/`, `.Rhistory`, `.RData`, `.Ruserdata`, `*.Rproj`, `packrat/`, `renv/`
|
||||||
|
- **Universal**: `.DS_Store`, `Thumbs.db`, `*.tmp`, `*.swp`, `.vscode/`, `.idea/`
|
||||||
|
|
||||||
|
**Tool-Specific Patterns**:
|
||||||
|
- **Docker**: `node_modules/`, `.git/`, `Dockerfile*`, `.dockerignore`, `*.log*`, `.env*`, `coverage/`
|
||||||
|
- **ESLint**: `node_modules/`, `dist/`, `build/`, `coverage/`, `*.min.js`
|
||||||
|
- **Prettier**: `node_modules/`, `dist/`, `build/`, `coverage/`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
|
||||||
|
- **Terraform**: `.terraform/`, `*.tfstate*`, `*.tfvars`, `.terraform.lock.hcl`
|
||||||
|
- **Kubernetes/k8s**: `*.secret.yaml`, `secrets/`, `.kube/`, `kubeconfig*`, `*.key`, `*.crt`
|
||||||
|
|
||||||
|
5. Parse tasks.md structure and extract:
|
||||||
|
- **Task phases**: Setup, Tests, Core, Integration, Polish
|
||||||
|
- **Task dependencies**: Sequential vs parallel execution rules
|
||||||
|
- **Task details**: ID, description, file paths, parallel markers [P]
|
||||||
|
- **Execution flow**: Order and dependency requirements
|
||||||
|
|
||||||
|
6. Execute implementation following the task plan:
|
||||||
|
- **Phase-by-phase execution**: Complete each phase before moving to the next
|
||||||
|
- **Respect dependencies**: Run sequential tasks in order, parallel tasks [P] can run together
|
||||||
|
- **Follow TDD approach**: Execute test tasks before their corresponding implementation tasks
|
||||||
|
- **File-based coordination**: Tasks affecting the same files must run sequentially
|
||||||
|
- **Validation checkpoints**: Verify each phase completion before proceeding
|
||||||
|
|
||||||
|
7. Implementation execution rules:
|
||||||
|
- **Setup first**: Initialize project structure, dependencies, configuration
|
||||||
|
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
||||||
|
- **Core development**: Implement models, services, CLI commands, endpoints
|
||||||
|
- **Integration work**: Database connections, middleware, logging, external services
|
||||||
|
- **Polish and validation**: Unit tests, performance optimization, documentation
|
||||||
|
|
||||||
|
8. Progress tracking and error handling:
|
||||||
|
- Report progress after each completed task
|
||||||
|
- Halt execution if any non-parallel task fails
|
||||||
|
- For parallel tasks [P], continue with successful tasks, report failed ones
|
||||||
|
- Provide clear error messages with context for debugging
|
||||||
|
- Suggest next steps if implementation cannot proceed
|
||||||
|
- **IMPORTANT** For completed tasks, make sure to mark the task off as [X] in the tasks file.
|
||||||
|
|
||||||
|
9. Completion validation:
|
||||||
|
- Verify all required tasks are completed
|
||||||
|
- Check that implemented features match the original specification
|
||||||
|
- Validate that tests pass and coverage meets requirements
|
||||||
|
- Confirm the implementation follows the technical plan
|
||||||
|
- Report final status with summary of completed work
|
||||||
|
|
||||||
|
Note: This command assumes a complete task breakdown exists in tasks.md. If tasks are incomplete or missing, suggest running `/speckit-tasks` first to regenerate the task list.
|
||||||
|
|
||||||
|
10. **Check for extension hooks**: After completion validation, check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_implement` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-plan"
|
||||||
|
description: "Execute the implementation planning workflow using the plan template to generate design artifacts."
|
||||||
|
argument-hint: "Optional guidance for the planning phase"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/plan.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before planning)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_plan` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
|
2. **Load context**: Read FEATURE_SPEC and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
|
||||||
|
|
||||||
|
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
||||||
|
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
||||||
|
- Fill Constitution Check section from constitution
|
||||||
|
- Evaluate gates (ERROR if violations unjustified)
|
||||||
|
- Phase 0: Generate research.md (resolve all NEEDS CLARIFICATION)
|
||||||
|
- Phase 1: Generate data-model.md, contracts/, quickstart.md
|
||||||
|
- Phase 1: Update agent context by running the agent script
|
||||||
|
- Re-evaluate Constitution Check post-design
|
||||||
|
|
||||||
|
4. **Stop and report**: Command ends after Phase 2 planning. Report branch, IMPL_PLAN path, and generated artifacts.
|
||||||
|
|
||||||
|
5. **Check for extension hooks**: After reporting, check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_plan` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Phases
|
||||||
|
|
||||||
|
### Phase 0: Outline & Research
|
||||||
|
|
||||||
|
1. **Extract unknowns from Technical Context** above:
|
||||||
|
- For each NEEDS CLARIFICATION → research task
|
||||||
|
- For each dependency → best practices task
|
||||||
|
- For each integration → patterns task
|
||||||
|
|
||||||
|
2. **Generate and dispatch research agents**:
|
||||||
|
|
||||||
|
```text
|
||||||
|
For each unknown in Technical Context:
|
||||||
|
Task: "Research {unknown} for {feature context}"
|
||||||
|
For each technology choice:
|
||||||
|
Task: "Find best practices for {tech} in {domain}"
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Consolidate findings** in `research.md` using format:
|
||||||
|
- Decision: [what was chosen]
|
||||||
|
- Rationale: [why chosen]
|
||||||
|
- Alternatives considered: [what else evaluated]
|
||||||
|
|
||||||
|
**Output**: research.md with all NEEDS CLARIFICATION resolved
|
||||||
|
|
||||||
|
### Phase 1: Design & Contracts
|
||||||
|
|
||||||
|
**Prerequisites:** `research.md` complete
|
||||||
|
|
||||||
|
1. **Extract entities from feature spec** → `data-model.md`:
|
||||||
|
- Entity name, fields, relationships
|
||||||
|
- Validation rules from requirements
|
||||||
|
- State transitions if applicable
|
||||||
|
|
||||||
|
2. **Define interface contracts** (if project has external interfaces) → `/contracts/`:
|
||||||
|
- Identify what interfaces the project exposes to users or other systems
|
||||||
|
- Document the contract format appropriate for the project type
|
||||||
|
- Examples: public APIs for libraries, command schemas for CLI tools, endpoints for web services, grammars for parsers, UI contracts for applications
|
||||||
|
- Skip if project is purely internal (build scripts, one-off tools, etc.)
|
||||||
|
|
||||||
|
3. **Agent context update**:
|
||||||
|
- Update the plan reference between the `<!-- SPECKIT START -->` and `<!-- SPECKIT END -->` markers in `CLAUDE.md` to point to the plan file created in step 1 (the IMPL_PLAN path)
|
||||||
|
|
||||||
|
**Output**: data-model.md, /contracts/*, quickstart.md, updated agent context file
|
||||||
|
|
||||||
|
## Key rules
|
||||||
|
|
||||||
|
- Use absolute paths for filesystem operations; use project-relative paths for references in documentation and agent context files
|
||||||
|
- ERROR on gate failures or unresolved clarifications
|
||||||
@@ -0,0 +1,330 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-specify"
|
||||||
|
description: "Create or update the feature specification from a natural language feature description."
|
||||||
|
argument-hint: "Describe the feature you want to specify"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/specify.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before specification)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_specify` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
The text the user typed after `/speckit-specify` in the triggering message **is** the feature description. Assume you always have it available in this conversation even if `$ARGUMENTS` appears literally below. Do not ask the user to repeat it unless they provided an empty command.
|
||||||
|
|
||||||
|
Given that feature description, do this:
|
||||||
|
|
||||||
|
1. **Generate a concise short name** (2-4 words) for the feature:
|
||||||
|
- Analyze the feature description and extract the most meaningful keywords
|
||||||
|
- Create a 2-4 word short name that captures the essence of the feature
|
||||||
|
- Use action-noun format when possible (e.g., "add-user-auth", "fix-payment-bug")
|
||||||
|
- Preserve technical terms and acronyms (OAuth2, API, JWT, etc.)
|
||||||
|
- Keep it concise but descriptive enough to understand the feature at a glance
|
||||||
|
- Examples:
|
||||||
|
- "I want to add user authentication" → "user-auth"
|
||||||
|
- "Implement OAuth2 integration for the API" → "oauth2-api-integration"
|
||||||
|
- "Create a dashboard for analytics" → "analytics-dashboard"
|
||||||
|
- "Fix payment processing timeout bug" → "fix-payment-timeout"
|
||||||
|
|
||||||
|
2. **Branch creation** (optional, via hook):
|
||||||
|
|
||||||
|
If a `before_specify` hook ran successfully in the Pre-Execution Checks above, it will have created/switched to a git branch and output JSON containing `BRANCH_NAME` and `FEATURE_NUM`. Note these values for reference, but the branch name does **not** dictate the spec directory name.
|
||||||
|
|
||||||
|
If the user explicitly provided `GIT_BRANCH_NAME`, pass it through to the hook so the branch script uses the exact value as the branch name (bypassing all prefix/suffix generation).
|
||||||
|
|
||||||
|
3. **Create the spec feature directory**:
|
||||||
|
|
||||||
|
Specs live under the default `specs/` directory unless the user explicitly provides `SPECIFY_FEATURE_DIRECTORY`.
|
||||||
|
|
||||||
|
**Resolution order for `SPECIFY_FEATURE_DIRECTORY`**:
|
||||||
|
1. If the user explicitly provided `SPECIFY_FEATURE_DIRECTORY` (e.g., via environment variable, argument, or configuration), use it as-is
|
||||||
|
2. Otherwise, auto-generate it under `specs/`:
|
||||||
|
- Check `.specify/init-options.json` for `branch_numbering`
|
||||||
|
- If `"timestamp"`: prefix is `YYYYMMDD-HHMMSS` (current timestamp)
|
||||||
|
- If `"sequential"` or absent: prefix is `NNN` (next available 3-digit number after scanning existing directories in `specs/`)
|
||||||
|
- Construct the directory name: `<prefix>-<short-name>` (e.g., `003-user-auth` or `20260319-143022-user-auth`)
|
||||||
|
- Set `SPECIFY_FEATURE_DIRECTORY` to `specs/<directory-name>`
|
||||||
|
|
||||||
|
**Create the directory and spec file**:
|
||||||
|
- `mkdir -p SPECIFY_FEATURE_DIRECTORY`
|
||||||
|
- Copy `.specify/templates/spec-template.md` to `SPECIFY_FEATURE_DIRECTORY/spec.md` as the starting point
|
||||||
|
- Set `SPEC_FILE` to `SPECIFY_FEATURE_DIRECTORY/spec.md`
|
||||||
|
- Persist the resolved path to `.specify/feature.json`:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"feature_directory": "<resolved feature dir>"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Write the actual resolved directory path value (for example, `specs/003-user-auth`), not the literal string `SPECIFY_FEATURE_DIRECTORY`.
|
||||||
|
This allows downstream commands (`/speckit-plan`, `/speckit-tasks`, etc.) to locate the feature directory without relying on git branch name conventions.
|
||||||
|
|
||||||
|
**IMPORTANT**:
|
||||||
|
- You must only create one feature per `/speckit-specify` invocation
|
||||||
|
- The spec directory name and the git branch name are independent — they may be the same but that is the user's choice
|
||||||
|
- The spec directory and file are always created by this command, never by the hook
|
||||||
|
|
||||||
|
4. Load `.specify/templates/spec-template.md` to understand required sections.
|
||||||
|
|
||||||
|
5. Follow this execution flow:
|
||||||
|
1. Parse user description from arguments
|
||||||
|
If empty: ERROR "No feature description provided"
|
||||||
|
2. Extract key concepts from description
|
||||||
|
Identify: actors, actions, data, constraints
|
||||||
|
3. For unclear aspects:
|
||||||
|
- Make informed guesses based on context and industry standards
|
||||||
|
- Only mark with [NEEDS CLARIFICATION: specific question] if:
|
||||||
|
- The choice significantly impacts feature scope or user experience
|
||||||
|
- Multiple reasonable interpretations exist with different implications
|
||||||
|
- No reasonable default exists
|
||||||
|
- **LIMIT: Maximum 3 [NEEDS CLARIFICATION] markers total**
|
||||||
|
- Prioritize clarifications by impact: scope > security/privacy > user experience > technical details
|
||||||
|
4. Fill User Scenarios & Testing section
|
||||||
|
If no clear user flow: ERROR "Cannot determine user scenarios"
|
||||||
|
5. Generate Functional Requirements
|
||||||
|
Each requirement must be testable
|
||||||
|
Use reasonable defaults for unspecified details (document assumptions in Assumptions section)
|
||||||
|
6. Define Success Criteria
|
||||||
|
Create measurable, technology-agnostic outcomes
|
||||||
|
Include both quantitative metrics (time, performance, volume) and qualitative measures (user satisfaction, task completion)
|
||||||
|
Each criterion must be verifiable without implementation details
|
||||||
|
7. Identify Key Entities (if data involved)
|
||||||
|
8. Return: SUCCESS (spec ready for planning)
|
||||||
|
|
||||||
|
6. Write the specification to SPEC_FILE using the template structure, replacing placeholders with concrete details derived from the feature description (arguments) while preserving section order and headings.
|
||||||
|
|
||||||
|
7. **Specification Quality Validation**: After writing the initial spec, validate it against quality criteria:
|
||||||
|
|
||||||
|
a. **Create Spec Quality Checklist**: Generate a checklist file at `SPECIFY_FEATURE_DIRECTORY/checklists/requirements.md` using the checklist template structure with these validation items:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Specification Quality Checklist: [FEATURE NAME]
|
||||||
|
|
||||||
|
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||||
|
**Created**: [DATE]
|
||||||
|
**Feature**: [Link to spec.md]
|
||||||
|
|
||||||
|
## Content Quality
|
||||||
|
|
||||||
|
- [ ] No implementation details (languages, frameworks, APIs)
|
||||||
|
- [ ] Focused on user value and business needs
|
||||||
|
- [ ] Written for non-technical stakeholders
|
||||||
|
- [ ] All mandatory sections completed
|
||||||
|
|
||||||
|
## Requirement Completeness
|
||||||
|
|
||||||
|
- [ ] No [NEEDS CLARIFICATION] markers remain
|
||||||
|
- [ ] Requirements are testable and unambiguous
|
||||||
|
- [ ] Success criteria are measurable
|
||||||
|
- [ ] Success criteria are technology-agnostic (no implementation details)
|
||||||
|
- [ ] All acceptance scenarios are defined
|
||||||
|
- [ ] Edge cases are identified
|
||||||
|
- [ ] Scope is clearly bounded
|
||||||
|
- [ ] Dependencies and assumptions identified
|
||||||
|
|
||||||
|
## Feature Readiness
|
||||||
|
|
||||||
|
- [ ] All functional requirements have clear acceptance criteria
|
||||||
|
- [ ] User scenarios cover primary flows
|
||||||
|
- [ ] Feature meets measurable outcomes defined in Success Criteria
|
||||||
|
- [ ] No implementation details leak into specification
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Items marked incomplete require spec updates before `/speckit-clarify` or `/speckit-plan`
|
||||||
|
```
|
||||||
|
|
||||||
|
b. **Run Validation Check**: Review the spec against each checklist item:
|
||||||
|
- For each item, determine if it passes or fails
|
||||||
|
- Document specific issues found (quote relevant spec sections)
|
||||||
|
|
||||||
|
c. **Handle Validation Results**:
|
||||||
|
|
||||||
|
- **If all items pass**: Mark checklist complete and proceed to step 8
|
||||||
|
|
||||||
|
- **If items fail (excluding [NEEDS CLARIFICATION])**:
|
||||||
|
1. List the failing items and specific issues
|
||||||
|
2. Update the spec to address each issue
|
||||||
|
3. Re-run validation until all items pass (max 3 iterations)
|
||||||
|
4. If still failing after 3 iterations, document remaining issues in checklist notes and warn user
|
||||||
|
|
||||||
|
- **If [NEEDS CLARIFICATION] markers remain**:
|
||||||
|
1. Extract all [NEEDS CLARIFICATION: ...] markers from the spec
|
||||||
|
2. **LIMIT CHECK**: If more than 3 markers exist, keep only the 3 most critical (by scope/security/UX impact) and make informed guesses for the rest
|
||||||
|
3. For each clarification needed (max 3), present options to user in this format:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Question [N]: [Topic]
|
||||||
|
|
||||||
|
**Context**: [Quote relevant spec section]
|
||||||
|
|
||||||
|
**What we need to know**: [Specific question from NEEDS CLARIFICATION marker]
|
||||||
|
|
||||||
|
**Suggested Answers**:
|
||||||
|
|
||||||
|
| Option | Answer | Implications |
|
||||||
|
|--------|--------|--------------|
|
||||||
|
| A | [First suggested answer] | [What this means for the feature] |
|
||||||
|
| B | [Second suggested answer] | [What this means for the feature] |
|
||||||
|
| C | [Third suggested answer] | [What this means for the feature] |
|
||||||
|
| Custom | Provide your own answer | [Explain how to provide custom input] |
|
||||||
|
|
||||||
|
**Your choice**: _[Wait for user response]_
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **CRITICAL - Table Formatting**: Ensure markdown tables are properly formatted:
|
||||||
|
- Use consistent spacing with pipes aligned
|
||||||
|
- Each cell should have spaces around content: `| Content |` not `|Content|`
|
||||||
|
- Header separator must have at least 3 dashes: `|--------|`
|
||||||
|
- Test that the table renders correctly in markdown preview
|
||||||
|
5. Number questions sequentially (Q1, Q2, Q3 - max 3 total)
|
||||||
|
6. Present all questions together before waiting for responses
|
||||||
|
7. Wait for user to respond with their choices for all questions (e.g., "Q1: A, Q2: Custom - [details], Q3: B")
|
||||||
|
8. Update the spec by replacing each [NEEDS CLARIFICATION] marker with the user's selected or provided answer
|
||||||
|
9. Re-run validation after all clarifications are resolved
|
||||||
|
|
||||||
|
d. **Update Checklist**: After each validation iteration, update the checklist file with current pass/fail status
|
||||||
|
|
||||||
|
8. **Report completion** to the user with:
|
||||||
|
- `SPECIFY_FEATURE_DIRECTORY` — the feature directory path
|
||||||
|
- `SPEC_FILE` — the spec file path
|
||||||
|
- Checklist results summary
|
||||||
|
- Readiness for the next phase (`/speckit-clarify` or `/speckit-plan`)
|
||||||
|
|
||||||
|
9. **Check for extension hooks**: After reporting completion, check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_specify` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
**NOTE:** Branch creation is handled by the `before_specify` hook (git extension). Spec directory and file creation are always handled by this core command.
|
||||||
|
|
||||||
|
## Quick Guidelines
|
||||||
|
|
||||||
|
- Focus on **WHAT** users need and **WHY**.
|
||||||
|
- Avoid HOW to implement (no tech stack, APIs, code structure).
|
||||||
|
- Written for business stakeholders, not developers.
|
||||||
|
- DO NOT create any checklists that are embedded in the spec. That will be a separate command.
|
||||||
|
|
||||||
|
### Section Requirements
|
||||||
|
|
||||||
|
- **Mandatory sections**: Must be completed for every feature
|
||||||
|
- **Optional sections**: Include only when relevant to the feature
|
||||||
|
- When a section doesn't apply, remove it entirely (don't leave as "N/A")
|
||||||
|
|
||||||
|
### For AI Generation
|
||||||
|
|
||||||
|
When creating this spec from a user prompt:
|
||||||
|
|
||||||
|
1. **Make informed guesses**: Use context, industry standards, and common patterns to fill gaps
|
||||||
|
2. **Document assumptions**: Record reasonable defaults in the Assumptions section
|
||||||
|
3. **Limit clarifications**: Maximum 3 [NEEDS CLARIFICATION] markers - use only for critical decisions that:
|
||||||
|
- Significantly impact feature scope or user experience
|
||||||
|
- Have multiple reasonable interpretations with different implications
|
||||||
|
- Lack any reasonable default
|
||||||
|
4. **Prioritize clarifications**: scope > security/privacy > user experience > technical details
|
||||||
|
5. **Think like a tester**: Every vague requirement should fail the "testable and unambiguous" checklist item
|
||||||
|
6. **Common areas needing clarification** (only if no reasonable default exists):
|
||||||
|
- Feature scope and boundaries (include/exclude specific use cases)
|
||||||
|
- User types and permissions (if multiple conflicting interpretations possible)
|
||||||
|
- Security/compliance requirements (when legally/financially significant)
|
||||||
|
|
||||||
|
**Examples of reasonable defaults** (don't ask about these):
|
||||||
|
|
||||||
|
- Data retention: Industry-standard practices for the domain
|
||||||
|
- Performance targets: Standard web/mobile app expectations unless specified
|
||||||
|
- Error handling: User-friendly messages with appropriate fallbacks
|
||||||
|
- Authentication method: Standard session-based or OAuth2 for web apps
|
||||||
|
- Integration patterns: Use project-appropriate patterns (REST/GraphQL for web services, function calls for libraries, CLI args for tools, etc.)
|
||||||
|
|
||||||
|
### Success Criteria Guidelines
|
||||||
|
|
||||||
|
Success criteria must be:
|
||||||
|
|
||||||
|
1. **Measurable**: Include specific metrics (time, percentage, count, rate)
|
||||||
|
2. **Technology-agnostic**: No mention of frameworks, languages, databases, or tools
|
||||||
|
3. **User-focused**: Describe outcomes from user/business perspective, not system internals
|
||||||
|
4. **Verifiable**: Can be tested/validated without knowing implementation details
|
||||||
|
|
||||||
|
**Good examples**:
|
||||||
|
|
||||||
|
- "Users can complete checkout in under 3 minutes"
|
||||||
|
- "System supports 10,000 concurrent users"
|
||||||
|
- "95% of searches return results in under 1 second"
|
||||||
|
- "Task completion rate improves by 40%"
|
||||||
|
|
||||||
|
**Bad examples** (implementation-focused):
|
||||||
|
|
||||||
|
- "API response time is under 200ms" (too technical, use "Users see results instantly")
|
||||||
|
- "Database can handle 1000 TPS" (implementation detail, use user-facing metric)
|
||||||
|
- "React components render efficiently" (framework-specific)
|
||||||
|
- "Redis cache hit rate above 80%" (technology-specific)
|
||||||
@@ -0,0 +1,202 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-tasks"
|
||||||
|
description: "Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts."
|
||||||
|
argument-hint: "Optional task generation constraints"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/tasks.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before tasks generation)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_tasks` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
1. **Setup**: Run `.specify/scripts/bash/setup-tasks.sh --json` from repo root and parse FEATURE_DIR, TASKS_TEMPLATE, and AVAILABLE_DOCS list. `FEATURE_DIR` and `TASKS_TEMPLATE` must be absolute paths when provided. `AVAILABLE_DOCS` is a list of document names/relative paths available under `FEATURE_DIR` (for example `research.md` or `contracts/`). For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
|
2. **Load design documents**: Read from FEATURE_DIR:
|
||||||
|
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
|
||||||
|
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
||||||
|
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
||||||
|
|
||||||
|
3. **Execute task generation workflow**:
|
||||||
|
- Load plan.md and extract tech stack, libraries, project structure
|
||||||
|
- Load spec.md and extract user stories with their priorities (P1, P2, P3, etc.)
|
||||||
|
- If data-model.md exists: Extract entities and map to user stories
|
||||||
|
- If contracts/ exists: Map interface contracts to user stories
|
||||||
|
- If research.md exists: Extract decisions for setup tasks
|
||||||
|
- Generate tasks organized by user story (see Task Generation Rules below)
|
||||||
|
- Generate dependency graph showing user story completion order
|
||||||
|
- Create parallel execution examples per user story
|
||||||
|
- Validate task completeness (each user story has all needed tasks, independently testable)
|
||||||
|
|
||||||
|
4. **Generate tasks.md**: Read the tasks template from TASKS_TEMPLATE (from the JSON output above) and use it as structure. If TASKS_TEMPLATE is empty, fall back to `.specify/templates/tasks-template.md`. Fill with:
|
||||||
|
- Correct feature name from plan.md
|
||||||
|
- Phase 1: Setup tasks (project initialization)
|
||||||
|
- Phase 2: Foundational tasks (blocking prerequisites for all user stories)
|
||||||
|
- Phase 3+: One phase per user story (in priority order from spec.md)
|
||||||
|
- Each phase includes: story goal, independent test criteria, tests (if requested), implementation tasks
|
||||||
|
- Final Phase: Polish & cross-cutting concerns
|
||||||
|
- All tasks must follow the strict checklist format (see Task Generation Rules below)
|
||||||
|
- Clear file paths for each task
|
||||||
|
- Dependencies section showing story completion order
|
||||||
|
- Parallel execution examples per story
|
||||||
|
- Implementation strategy section (MVP first, incremental delivery)
|
||||||
|
|
||||||
|
5. **Report**: Output path to generated tasks.md and summary:
|
||||||
|
- Total task count
|
||||||
|
- Task count per user story
|
||||||
|
- Parallel opportunities identified
|
||||||
|
- Independent test criteria for each story
|
||||||
|
- Suggested MVP scope (typically just User Story 1)
|
||||||
|
- Format validation: Confirm ALL tasks follow the checklist format (checkbox, ID, labels, file paths)
|
||||||
|
|
||||||
|
6. **Check for extension hooks**: After tasks.md is generated, check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_tasks` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
Context for task generation: $ARGUMENTS
|
||||||
|
|
||||||
|
The tasks.md should be immediately executable - each task must be specific enough that an LLM can complete it without additional context.
|
||||||
|
|
||||||
|
## Task Generation Rules
|
||||||
|
|
||||||
|
**CRITICAL**: Tasks MUST be organized by user story to enable independent implementation and testing.
|
||||||
|
|
||||||
|
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
||||||
|
|
||||||
|
### Checklist Format (REQUIRED)
|
||||||
|
|
||||||
|
Every task MUST strictly follow this format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
- [ ] [TaskID] [P?] [Story?] Description with file path
|
||||||
|
```
|
||||||
|
|
||||||
|
**Format Components**:
|
||||||
|
|
||||||
|
1. **Checkbox**: ALWAYS start with `- [ ]` (markdown checkbox)
|
||||||
|
2. **Task ID**: Sequential number (T001, T002, T003...) in execution order
|
||||||
|
3. **[P] marker**: Include ONLY if task is parallelizable (different files, no dependencies on incomplete tasks)
|
||||||
|
4. **[Story] label**: REQUIRED for user story phase tasks only
|
||||||
|
- Format: [US1], [US2], [US3], etc. (maps to user stories from spec.md)
|
||||||
|
- Setup phase: NO story label
|
||||||
|
- Foundational phase: NO story label
|
||||||
|
- User Story phases: MUST have story label
|
||||||
|
- Polish phase: NO story label
|
||||||
|
5. **Description**: Clear action with exact file path
|
||||||
|
|
||||||
|
**Examples**:
|
||||||
|
|
||||||
|
- ✅ CORRECT: `- [ ] T001 Create project structure per implementation plan`
|
||||||
|
- ✅ CORRECT: `- [ ] T005 [P] Implement authentication middleware in src/middleware/auth.py`
|
||||||
|
- ✅ CORRECT: `- [ ] T012 [P] [US1] Create User model in src/models/user.py`
|
||||||
|
- ✅ CORRECT: `- [ ] T014 [US1] Implement UserService in src/services/user_service.py`
|
||||||
|
- ❌ WRONG: `- [ ] Create User model` (missing ID and Story label)
|
||||||
|
- ❌ WRONG: `T001 [US1] Create model` (missing checkbox)
|
||||||
|
- ❌ WRONG: `- [ ] [US1] Create User model` (missing Task ID)
|
||||||
|
- ❌ WRONG: `- [ ] T001 [US1] Create model` (missing file path)
|
||||||
|
|
||||||
|
### Task Organization
|
||||||
|
|
||||||
|
1. **From User Stories (spec.md)** - PRIMARY ORGANIZATION:
|
||||||
|
- Each user story (P1, P2, P3...) gets its own phase
|
||||||
|
- Map all related components to their story:
|
||||||
|
- Models needed for that story
|
||||||
|
- Services needed for that story
|
||||||
|
- Interfaces/UI needed for that story
|
||||||
|
- If tests requested: Tests specific to that story
|
||||||
|
- Mark story dependencies (most stories should be independent)
|
||||||
|
|
||||||
|
2. **From Contracts**:
|
||||||
|
- Map each interface contract → to the user story it serves
|
||||||
|
- If tests requested: Each interface contract → contract test task [P] before implementation in that story's phase
|
||||||
|
|
||||||
|
3. **From Data Model**:
|
||||||
|
- Map each entity to the user story(ies) that need it
|
||||||
|
- If entity serves multiple stories: Put in earliest story or Setup phase
|
||||||
|
- Relationships → service layer tasks in appropriate story phase
|
||||||
|
|
||||||
|
4. **From Setup/Infrastructure**:
|
||||||
|
- Shared infrastructure → Setup phase (Phase 1)
|
||||||
|
- Foundational/blocking tasks → Foundational phase (Phase 2)
|
||||||
|
- Story-specific setup → within that story's phase
|
||||||
|
|
||||||
|
### Phase Structure
|
||||||
|
|
||||||
|
- **Phase 1**: Setup (project initialization)
|
||||||
|
- **Phase 2**: Foundational (blocking prerequisites - MUST complete before user stories)
|
||||||
|
- **Phase 3+**: User Stories in priority order (P1, P2, P3...)
|
||||||
|
- Within each story: Tests (if requested) → Models → Services → Endpoints → Integration
|
||||||
|
- Each phase should be a complete, independently testable increment
|
||||||
|
- **Final Phase**: Polish & Cross-Cutting Concerns
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
---
|
||||||
|
name: "speckit-taskstoissues"
|
||||||
|
description: "Convert existing tasks into actionable, dependency-ordered GitHub issues for the feature based on available design artifacts."
|
||||||
|
argument-hint: "Optional filter or label for GitHub issues"
|
||||||
|
compatibility: "Requires spec-kit project structure with .specify/ directory"
|
||||||
|
metadata:
|
||||||
|
author: "github-spec-kit"
|
||||||
|
source: "templates/commands/taskstoissues.md"
|
||||||
|
user-invocable: true
|
||||||
|
disable-model-invocation: false
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
## User Input
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ARGUMENTS
|
||||||
|
```
|
||||||
|
|
||||||
|
You **MUST** consider the user input before proceeding (if not empty).
|
||||||
|
|
||||||
|
## Pre-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (before tasks-to-issues conversion)**:
|
||||||
|
- Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.before_taskstoissues` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Pre-Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Pre-Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
|
||||||
|
Wait for the result of the hook command before proceeding to the Outline.
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
|
|
||||||
|
## Outline
|
||||||
|
|
||||||
|
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
1. From the executed script, extract the path to **tasks**.
|
||||||
|
1. Get the Git remote by running:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git config --get remote.origin.url
|
||||||
|
```
|
||||||
|
|
||||||
|
> [!CAUTION]
|
||||||
|
> ONLY PROCEED TO NEXT STEPS IF THE REMOTE IS A GITHUB URL
|
||||||
|
|
||||||
|
1. For each task in the list, use the GitHub MCP server to create a new issue in the repository that is representative of the Git remote.
|
||||||
|
|
||||||
|
> [!CAUTION]
|
||||||
|
> UNDER NO CIRCUMSTANCES EVER CREATE ISSUES IN REPOSITORIES THAT DO NOT MATCH THE REMOTE URL
|
||||||
|
|
||||||
|
## Post-Execution Checks
|
||||||
|
|
||||||
|
**Check for extension hooks (after tasks-to-issues conversion)**:
|
||||||
|
Check if `.specify/extensions.yml` exists in the project root.
|
||||||
|
- If it exists, read it and look for entries under the `hooks.after_taskstoissues` key
|
||||||
|
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
|
||||||
|
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
|
||||||
|
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
|
||||||
|
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
|
||||||
|
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
|
||||||
|
- When constructing slash commands from hook command names, replace dots (`.`) with hyphens (`-`). For example, `speckit.git.commit` → `/speckit-git-commit`.
|
||||||
|
- For each executable hook, output the following based on its `optional` flag:
|
||||||
|
- **Optional hook** (`optional: true`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Optional Hook**: {extension}
|
||||||
|
Command: `/{command}`
|
||||||
|
Description: {description}
|
||||||
|
|
||||||
|
Prompt: {prompt}
|
||||||
|
To execute: `/{command}`
|
||||||
|
```
|
||||||
|
- **Mandatory hook** (`optional: false`):
|
||||||
|
```
|
||||||
|
## Extension Hooks
|
||||||
|
|
||||||
|
**Automatic Hook**: {extension}
|
||||||
|
Executing: `/{command}`
|
||||||
|
EXECUTE_COMMAND: {command}
|
||||||
|
```
|
||||||
|
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
*
|
||||||
|
|
||||||
|
!.dockerignore
|
||||||
|
!Dockerfile
|
||||||
|
!dockerfile
|
||||||
|
!requirements.txt
|
||||||
|
!server.py
|
||||||
|
!main.py
|
||||||
|
!VERSION
|
||||||
|
!tailwind.config.js
|
||||||
|
|
||||||
|
!lib/
|
||||||
|
!lib/**
|
||||||
|
|
||||||
|
# Mirror the static/* policy from .gitignore: ship the static tree, but not
|
||||||
|
# the cached audio/art artifacts a developer's runtime would generate
|
||||||
|
# (static/*.ogg, *.mp3, *.wav, art/, audio_*/). Those bloat the image and
|
||||||
|
# make builds non-reproducible across machines.
|
||||||
|
!static/
|
||||||
|
!static/**
|
||||||
|
static/*.ogg
|
||||||
|
static/*.mp3
|
||||||
|
static/*.wav
|
||||||
|
static/art/
|
||||||
|
static/audio_*/
|
||||||
|
static/sloppak_cache/
|
||||||
|
# note-detection tuning flow auto-writes WAVs and JSONL captures here.
|
||||||
|
# Not in .gitignore yet — see docs/note-detect-tuning.md — but still a
|
||||||
|
# generated-at-runtime directory that shouldn't bake into the image.
|
||||||
|
static/note_detect_recordings/
|
||||||
|
|
||||||
|
# All in-tree core plugins ship in the image. Each plugin directory must be
|
||||||
|
# explicitly re-allowed after the blanket `plugins/*/` exclusion below.
|
||||||
|
!plugins/
|
||||||
|
plugins/*/
|
||||||
|
!plugins/__init__.py
|
||||||
|
!plugins/app_tour_library/
|
||||||
|
!plugins/app_tour_library/**
|
||||||
|
!plugins/app_tour_settings/
|
||||||
|
!plugins/app_tour_settings/**
|
||||||
|
!plugins/capability_inspector/
|
||||||
|
!plugins/capability_inspector/**
|
||||||
|
!plugins/highway_3d/
|
||||||
|
!plugins/highway_3d/**
|
||||||
|
!plugins/minigames/
|
||||||
|
!plugins/minigames/**
|
||||||
|
!plugins/tuner/
|
||||||
|
!plugins/tuner/**
|
||||||
|
|
||||||
|
!data/
|
||||||
|
!data/**
|
||||||
|
|
||||||
|
|
||||||
|
# Ship only the built-in diagnostic sloppak artifacts (seeded into
|
||||||
|
# DLC_DIR/diagnostics-builtin/ at scan time). The builder script + README
|
||||||
|
# under docs/diagnostics/ are dev-only and stay out of the image.
|
||||||
|
!docs/
|
||||||
|
docs/*/
|
||||||
|
!docs/diagnostics/
|
||||||
|
docs/diagnostics/*
|
||||||
|
!docs/diagnostics/*.sloppak
|
||||||
|
|
||||||
|
# Exclude common developer artifacts even inside the whitelisted trees so a
|
||||||
|
# venv / cache / build output never gets baked into the image.
|
||||||
|
**/__pycache__/
|
||||||
|
**/*.pyc
|
||||||
|
**/.pytest_cache/
|
||||||
|
**/.mypy_cache/
|
||||||
|
**/.ruff_cache/
|
||||||
|
**/node_modules/
|
||||||
|
**/.venv/
|
||||||
|
**/venv/
|
||||||
|
**/.env
|
||||||
|
# (the Dockerfile builder stage rebuilds from source). A contributor
|
||||||
|
# into the build context.
|
||||||
|
**/bin/
|
||||||
|
**/obj/
|
||||||
|
**/.env.*
|
||||||
|
**/.DS_Store
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# Shell scripts must use LF line endings — bash interprets a \r in the
|
||||||
|
# shebang as part of the path, producing "/usr/bin/env: 'bash\r': No such
|
||||||
|
# file or directory" on WSL2 / Linux when checked out with autocrlf=true.
|
||||||
|
*.sh text eol=lf
|
||||||
|
|
||||||
|
# Keep the minified Three.js bundle byte-identical across platforms.
|
||||||
|
# Without this, autocrlf=true would rewrite the bundle's newlines on
|
||||||
|
# Windows checkouts and the SHA-256 wouldn't match the upstream tag —
|
||||||
|
# making the "pinned r170" claim in static/vendor/three/VERSION
|
||||||
|
# dishonest. Scoped narrowly to the bundle (not the whole vendor dir)
|
||||||
|
# so LICENSE / VERSION stay diff-readable for future version bumps.
|
||||||
|
static/vendor/three/three.module.min.js binary
|
||||||
|
|
||||||
|
# PSARC test fixtures are zlib-compressed binary archives — diffs and
|
||||||
|
# autocrlf rewrites would corrupt them.
|
||||||
|
tests/fixtures/*.psarc binary
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
blank_issues_enabled: true
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
name: ci
|
||||||
|
|
||||||
|
# Runs only as a reusable workflow invoked by ship-ci.yml (for PRs into main
|
||||||
|
# and release/**). It deliberately has no standalone pull_request trigger: a
|
||||||
|
# direct run would publish unprefixed "<job>" checks, but the org rulesets
|
||||||
|
# require the "ci / <job>" names produced when ship-ci.yml calls this workflow.
|
||||||
|
on:
|
||||||
|
workflow_call:
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
pull-requests: read
|
||||||
|
checks: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
|
||||||
|
test:
|
||||||
|
name: test
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: '3.12'
|
||||||
|
cache: 'pip'
|
||||||
|
|
||||||
|
- name: Guard against print() / traceback.print_exc() in server.py, lib/, and bundled plugin routes
|
||||||
|
run: |
|
||||||
|
# git grep: tracked files only — no .pyc / __pycache__ noise from
|
||||||
|
# later pytest runs. Covers both audited patterns from #155 / #242.
|
||||||
|
# `(^|[^A-Za-z0-9_])` anchor avoids matching suffixes like `myprint(`;
|
||||||
|
# POSIX leaves `\b` undefined, so we use an explicit character class.
|
||||||
|
hits=$(git grep -nE '(^|[^A-Za-z0-9_])(print|traceback\.print_exc)[[:space:]]*\(' \
|
||||||
|
-- server.py lib/ \
|
||||||
|
$(git ls-files 'plugins/*/routes.py') || true)
|
||||||
|
if [ -n "$hits" ]; then
|
||||||
|
echo "$hits"
|
||||||
|
first=$(printf '%s\n' "$hits" | head -n1)
|
||||||
|
file=$(printf '%s' "$first" | cut -d: -f1)
|
||||||
|
line=$(printf '%s' "$first" | cut -d: -f2)
|
||||||
|
echo "::error file=${file},line=${line}::print() or traceback.print_exc() found in server.py, lib/, or a bundled plugin routes.py. Use the slopsmith logger (lib/logging_setup.py) — see issues #155 / #242."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: |
|
||||||
|
python -m pip install --upgrade pip
|
||||||
|
pip install -r requirements.txt -r requirements-test.txt
|
||||||
|
|
||||||
|
- name: Run pytest
|
||||||
|
run: pytest
|
||||||
|
|
||||||
|
- name: Run JS plugin-API tests
|
||||||
|
run: node --test tests/js/*.test.js 'tests/plugins/*/js/*.test.js'
|
||||||
|
|
||||||
|
tailwind-fresh:
|
||||||
|
# Guard that the committed static/tailwind.min.css is in sync with source.
|
||||||
|
# The Play CDN's runtime JIT was removed (slopsmith-desktop#110); a prebuilt
|
||||||
|
# stylesheet only contains classes the scanner saw at build time, so stale
|
||||||
|
# CSS silently ships unstyled elements. Rebuild and fail on any diff.
|
||||||
|
name: tailwind-fresh
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: '20'
|
||||||
|
|
||||||
|
- name: Rebuild Tailwind CSS
|
||||||
|
run: bash scripts/build-tailwind.sh
|
||||||
|
|
||||||
|
- name: Verify committed static/tailwind.min.css is fresh
|
||||||
|
run: |
|
||||||
|
# Hard-fail (matches the print() guard convention) — do NOT auto-commit.
|
||||||
|
if ! git diff --quiet -- static/tailwind.min.css; then
|
||||||
|
echo "::error file=static/tailwind.min.css::static/tailwind.min.css is stale. Run 'bash scripts/build-tailwind.sh' and commit the regenerated file."
|
||||||
|
git diff -- static/tailwind.min.css
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
manifest-validation:
|
||||||
|
name: manifest-validation
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: '3.12'
|
||||||
|
|
||||||
|
- name: Validate plugin manifests
|
||||||
|
run: |
|
||||||
|
python - <<'EOF'
|
||||||
|
import json, sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
errors = []
|
||||||
|
manifests = sorted(Path("plugins").glob("*/plugin.json"))
|
||||||
|
|
||||||
|
for manifest in manifests:
|
||||||
|
try:
|
||||||
|
data = json.loads(manifest.read_text())
|
||||||
|
except json.JSONDecodeError as e:
|
||||||
|
errors.append(f"{manifest}: invalid JSON — {e}")
|
||||||
|
continue
|
||||||
|
for field in ("id", "name"):
|
||||||
|
if field not in data:
|
||||||
|
errors.append(f"{manifest}: missing required field '{field}'")
|
||||||
|
pid = data.get("id", "")
|
||||||
|
if pid and pid != pid.lower():
|
||||||
|
errors.append(f"{manifest}: 'id' must be lowercase, got '{pid}'")
|
||||||
|
# The plugin loader treats each plugins/<dir> as a Python module,
|
||||||
|
# so the manifest 'id' must match its directory name.
|
||||||
|
dirname = manifest.parent.name
|
||||||
|
if pid and pid != dirname:
|
||||||
|
errors.append(f"{manifest}: 'id' ({pid!r}) must match directory name ({dirname!r})")
|
||||||
|
|
||||||
|
if errors:
|
||||||
|
for e in errors:
|
||||||
|
print(f"::error::{e}")
|
||||||
|
sys.exit(1)
|
||||||
|
print(f"Validated {len(manifests)} manifest(s) — OK")
|
||||||
|
EOF
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
name: Nightly
|
||||||
|
|
||||||
|
on:
|
||||||
|
schedule:
|
||||||
|
- cron: '0 2 * * *'
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
setup:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
outputs:
|
||||||
|
branch: ${{ steps.branch.outputs.branch }}
|
||||||
|
date: ${{ steps.date.outputs.date }}
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Find active release branch
|
||||||
|
id: branch
|
||||||
|
env:
|
||||||
|
GH_TOKEN: ${{ github.token }}
|
||||||
|
run: |
|
||||||
|
branch=$(gh api "repos/${{ github.repository }}/git/matching-refs/heads/release/v" \
|
||||||
|
--jq '[.[].ref | ltrimstr("refs/heads/")] | map(ltrimstr("refs/heads/")) | .[]' \
|
||||||
|
| sort -V | tail -1 || true)
|
||||||
|
if [[ -z "$branch" ]]; then
|
||||||
|
branch="main"
|
||||||
|
fi
|
||||||
|
echo "branch=$branch" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "Active branch: $branch"
|
||||||
|
|
||||||
|
- name: Get date
|
||||||
|
id: date
|
||||||
|
run: echo "date=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
build-docker:
|
||||||
|
needs: setup
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
packages: write
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
ref: ${{ needs.setup.outputs.branch }}
|
||||||
|
persist-credentials: false
|
||||||
|
|
||||||
|
- name: Set up Docker Buildx
|
||||||
|
uses: docker/setup-buildx-action@v3
|
||||||
|
|
||||||
|
- name: Log in to GHCR
|
||||||
|
uses: docker/login-action@v3
|
||||||
|
with:
|
||||||
|
registry: ghcr.io
|
||||||
|
username: ${{ github.actor }}
|
||||||
|
password: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
|
- name: Build and push Docker image
|
||||||
|
uses: docker/build-push-action@v6
|
||||||
|
with:
|
||||||
|
context: .
|
||||||
|
platforms: linux/amd64,linux/arm64
|
||||||
|
push: true
|
||||||
|
tags: |
|
||||||
|
ghcr.io/slopsmith/slopsmith:nightly
|
||||||
|
ghcr.io/slopsmith/slopsmith:nightly-${{ needs.setup.outputs.date }}
|
||||||
|
cache-from: type=gha
|
||||||
|
cache-to: type=gha,mode=max
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
name: release
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
tags: ['v*']
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
packages: write
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
docker:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
|
||||||
|
- name: Log in to GHCR
|
||||||
|
uses: docker/login-action@v3
|
||||||
|
with:
|
||||||
|
registry: ghcr.io
|
||||||
|
username: ${{ github.actor }}
|
||||||
|
password: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
|
- name: Extract version from tag
|
||||||
|
id: meta
|
||||||
|
run: |
|
||||||
|
tag="${GITHUB_REF_NAME}" # e.g. v0.3.0 or v0.3.0-beta.1
|
||||||
|
version="${tag#v}" # strip leading v
|
||||||
|
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||||
|
# Emit the full tag list here so the build step never receives a
|
||||||
|
# blank tag line (the previous inline `… || ''` conditional left an
|
||||||
|
# empty entry for pre-release tags). :latest is added only for
|
||||||
|
# stable releases (no pre-release suffix).
|
||||||
|
{
|
||||||
|
echo "tags<<TAGS_EOF"
|
||||||
|
echo "ghcr.io/${GITHUB_REPOSITORY}:${version}"
|
||||||
|
if [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||||
|
echo "ghcr.io/${GITHUB_REPOSITORY}:latest"
|
||||||
|
fi
|
||||||
|
echo "TAGS_EOF"
|
||||||
|
} >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: Build and push Docker image
|
||||||
|
uses: docker/build-push-action@v6
|
||||||
|
with:
|
||||||
|
context: .
|
||||||
|
push: true
|
||||||
|
tags: ${{ steps.meta.outputs.tags }}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
name: ship-ci
|
||||||
|
|
||||||
|
# PRs into both main and release/** run CI through this wrapper so the check
|
||||||
|
# runs are named "ci / <job>" (reusable-workflow caller prefix), matching the
|
||||||
|
# org rulesets' required contexts. ci.yml itself only triggers via
|
||||||
|
# workflow_call — it never runs standalone, which would emit unprefixed
|
||||||
|
# "<job>" checks that the rulesets can't match.
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
branches: [main, 'release/**']
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
pull-requests: read
|
||||||
|
checks: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
ci:
|
||||||
|
uses: ./.github/workflows/ci.yml
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
name: Sync VERSION from desktop release
|
||||||
|
|
||||||
|
# Updates the VERSION file in this repo whenever slopsmith-desktop
|
||||||
|
# publishes a new tagged release. slopsmith-desktop's build.yml
|
||||||
|
# dispatches the `desktop-released` event at the end of a successful
|
||||||
|
# tag build (see docs in CLAUDE.md). A `workflow_dispatch` trigger is
|
||||||
|
# kept for manual testing / recovery.
|
||||||
|
#
|
||||||
|
# Related issue: #81.
|
||||||
|
|
||||||
|
on:
|
||||||
|
repository_dispatch:
|
||||||
|
types: [desktop-released]
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
version:
|
||||||
|
description: 'Version to sync (vX.Y.Z or X.Y.Z)'
|
||||||
|
required: true
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: write
|
||||||
|
|
||||||
|
# Serialize runs so a rapid-fire pair of dispatches can't produce a
|
||||||
|
# non-fast-forward push race. Later runs queue behind earlier ones.
|
||||||
|
concurrency:
|
||||||
|
group: sync-version
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
sync:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
# Always operate on main regardless of trigger branch. repository_dispatch
|
||||||
|
# already runs against the default branch, but workflow_dispatch can be
|
||||||
|
# launched from any branch in the UI — pinning ref: main keeps both
|
||||||
|
# paths committing to the same place.
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
ref: main
|
||||||
|
|
||||||
|
- name: Determine target version
|
||||||
|
id: v
|
||||||
|
# Pass untrusted payloads through env vars instead of ${{ }}
|
||||||
|
# expansion inside the shell body — inline expansion makes the
|
||||||
|
# script vulnerable to command injection if a dispatch client
|
||||||
|
# sent a value like `$(...)` or a quote break.
|
||||||
|
env:
|
||||||
|
RAW_DISPATCH: ${{ github.event.client_payload.version }}
|
||||||
|
RAW_INPUT: ${{ inputs.version }}
|
||||||
|
EVENT_NAME: ${{ github.event_name }}
|
||||||
|
run: |
|
||||||
|
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
|
||||||
|
raw="$RAW_INPUT"
|
||||||
|
else
|
||||||
|
raw="$RAW_DISPATCH"
|
||||||
|
fi
|
||||||
|
# Accept both "vX.Y.Z" and "X.Y.Z" on the wire.
|
||||||
|
version="${raw#v}"
|
||||||
|
# Anchored semver guard — rejects payloads like
|
||||||
|
# "soundfonts-v1" (seen in the desktop release list) or any
|
||||||
|
# stray text. The emitter side validates too, but we don't
|
||||||
|
# trust cross-repo inputs.
|
||||||
|
if ! [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||||
|
echo "::error::Invalid version payload: '$raw'"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: Compare with current VERSION
|
||||||
|
id: cmp
|
||||||
|
env:
|
||||||
|
TARGET: ${{ steps.v.outputs.version }}
|
||||||
|
run: |
|
||||||
|
current=$(tr -d '[:space:]' < VERSION)
|
||||||
|
if [ "$current" = "$TARGET" ]; then
|
||||||
|
echo "No change (already at $current)."
|
||||||
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
||||||
|
else
|
||||||
|
echo "Bumping $current -> $TARGET."
|
||||||
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "previous=$current" >> "$GITHUB_OUTPUT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
- name: Commit and push
|
||||||
|
if: steps.cmp.outputs.changed == 'true'
|
||||||
|
env:
|
||||||
|
TARGET: ${{ steps.v.outputs.version }}
|
||||||
|
run: |
|
||||||
|
printf '%s\n' "$TARGET" > VERSION
|
||||||
|
git config user.name 'github-actions[bot]'
|
||||||
|
git config user.email 'github-actions[bot]@users.noreply.github.com'
|
||||||
|
git add VERSION
|
||||||
|
git commit -m "chore: sync VERSION to $TARGET (desktop release)"
|
||||||
|
# Explicit HEAD:main push — if workflow_dispatch was somehow
|
||||||
|
# launched with ref: main overridden in the UI, this still
|
||||||
|
# lands on main rather than pushing to whatever the tracking
|
||||||
|
# branch was.
|
||||||
|
git push origin HEAD:main
|
||||||
+48
@@ -0,0 +1,48 @@
|
|||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
*.egg-info/
|
||||||
|
*.log
|
||||||
|
.env*
|
||||||
|
.DS_Store
|
||||||
|
.vscode/
|
||||||
|
static/*.ogg
|
||||||
|
static/*.mp3
|
||||||
|
static/*.wav
|
||||||
|
static/art/
|
||||||
|
static/audio_*
|
||||||
|
plugins/*/
|
||||||
|
!plugins/__init__.py
|
||||||
|
# Core plugins ship in-tree. They're plugins-as-plugins (plugin loader
|
||||||
|
# treats them identically to user-installed ones) but are bundled with
|
||||||
|
# the default container image and marked `"bundled": true` in their
|
||||||
|
# manifest. Add new core plugins as `!plugins/<id>/` exceptions.
|
||||||
|
!plugins/highway_3d/
|
||||||
|
!plugins/highway_3d/**
|
||||||
|
plugins/highway_3d/__pycache__/
|
||||||
|
!plugins/app_tour_library/
|
||||||
|
!plugins/app_tour_library/**
|
||||||
|
!plugins/app_tour_settings/
|
||||||
|
!plugins/app_tour_settings/**
|
||||||
|
!plugins/capability_inspector/
|
||||||
|
!plugins/capability_inspector/**
|
||||||
|
!plugins/minigames/
|
||||||
|
!plugins/minigames/**
|
||||||
|
plugins/minigames/__pycache__/
|
||||||
|
!plugins/tuner/
|
||||||
|
!plugins/tuner/**
|
||||||
|
plugins/tuner/__pycache__/
|
||||||
|
node_modules/
|
||||||
|
test-results/
|
||||||
|
playwright-report/
|
||||||
|
|
||||||
|
/dlc
|
||||||
|
|
||||||
|
.vscode
|
||||||
|
Thumbs.db
|
||||||
|
*.tmp
|
||||||
|
*.swp
|
||||||
|
.idea/
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"ai": "claude",
|
||||||
|
"ai_skills": true,
|
||||||
|
"branch_numbering": "sequential",
|
||||||
|
"context_file": "CLAUDE.md",
|
||||||
|
"here": true,
|
||||||
|
"integration": "claude",
|
||||||
|
"script": "sh",
|
||||||
|
"speckit_version": "0.8.8.dev0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"version": "0.8.8.dev0",
|
||||||
|
"integration_state_schema": 1,
|
||||||
|
"installed_integrations": [
|
||||||
|
"claude"
|
||||||
|
],
|
||||||
|
"integration_settings": {
|
||||||
|
"claude": {
|
||||||
|
"script": "sh",
|
||||||
|
"invoke_separator": "-"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"integration": "claude",
|
||||||
|
"default_integration": "claude"
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"integration": "claude",
|
||||||
|
"version": "0.8.8.dev0",
|
||||||
|
"installed_at": "2026-05-09T19:47:45.677817+00:00",
|
||||||
|
"files": {
|
||||||
|
".claude/skills/speckit-analyze/SKILL.md": "2eef0fbff6cad15c9d4714d8986192387811c971a82a1135ab0404f3db0c5e90",
|
||||||
|
".claude/skills/speckit-checklist/SKILL.md": "26419fc118dcd9c4e1e977460696a04b7757b8fb0a2d1ff9c64732669deb7977",
|
||||||
|
".claude/skills/speckit-clarify/SKILL.md": "f2560f9f2007b4e995130f0c42633f08837a76a35d94e84091713a6f39bb1064",
|
||||||
|
".claude/skills/speckit-constitution/SKILL.md": "c1a044aba243ca6aff627fb5e4404feb6f1108d4f7dd174631bee3ae477d6c15",
|
||||||
|
".claude/skills/speckit-implement/SKILL.md": "6029565c1a56de8919d1846b187cd644f734a0e30a6067a709803e6bc0d2abf7",
|
||||||
|
".claude/skills/speckit-plan/SKILL.md": "8141ebbce228ad0b422a84e3b995d2bd85de917b96eadd02b5fcb56fb23f2594",
|
||||||
|
".claude/skills/speckit-specify/SKILL.md": "caadc05119eca453709a0425ed88d253883f9c55da4c13a4898367653a859483",
|
||||||
|
".claude/skills/speckit-tasks/SKILL.md": "54c4665be61818ed50aa528bb4c51db3627079b2c67d47f2b01046268288c4a5",
|
||||||
|
".claude/skills/speckit-taskstoissues/SKILL.md": "99bf5ffd90dcb57b63007c7f659a5160a18ce6feb82889895808e2d277abe83b"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
{
|
||||||
|
"integration": "speckit",
|
||||||
|
"version": "0.8.8.dev0",
|
||||||
|
"installed_at": "2026-05-09T19:47:45.689935+00:00",
|
||||||
|
"files": {
|
||||||
|
".specify/scripts/bash/create-new-feature.sh": "bcf4964ca0c6c78717bb42d9e66b8c7e5ee82779cd96afc5aa7b08b75abe5790",
|
||||||
|
".specify/scripts/bash/setup-plan.sh": "0d1d7a66de157b0be1385bb91aa71e5bf95550217abf47a73270dab0dc52895a",
|
||||||
|
".specify/scripts/bash/check-prerequisites.sh": "aff361639c504b95a2901493f5022788adc01a6792fd37f132de8f57782e4b80",
|
||||||
|
".specify/scripts/bash/common.sh": "dd638316259e699fd466542c77ef16af5eb198efe0447c081f86b890db414ba8",
|
||||||
|
".specify/scripts/bash/setup-tasks.sh": "e8d050c63c5afb664a8b671b0b0155513fb9cab0567b335e16b9eb035482aad2",
|
||||||
|
".specify/templates/tasks-template.md": "fb7a30a6e8e7319b7134bd52a26dd52fb7dd9106ab8fa08b6fb551d704dac498",
|
||||||
|
".specify/templates/checklist-template.md": "c37695297e5d3153d64f82c21223509940b13932046c7961c42d1d669516130c",
|
||||||
|
".specify/templates/spec-template.md": "785dc50d856dd92d6515eca0761e16dce0c9ba0a3cd07154fd33eae77932422a",
|
||||||
|
".specify/templates/plan-template.md": "5ad267630e370c73fe957dafa61bf76d633f3aea9d2f0b5195087d729cdd1e41",
|
||||||
|
".specify/templates/constitution-template.md": "ce7549540fa45543cca797a150201d868e64495fdff39dc38246fb17bd4024b3"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,257 @@
|
|||||||
|
# Slopsmith Constitution
|
||||||
|
|
||||||
|
> Slopsmith is a self-hosted, single-user web app for browsing, playing, and
|
||||||
|
> practicing interactive music notation, built around its own open `.sloppak`
|
||||||
|
> chart format (charts imported from Guitar Pro / MusicXML or authored in the
|
||||||
|
> built-in editor). This constitution captures the non-negotiable principles
|
||||||
|
> that govern its core (`server.py`, `lib/`, `static/`) and that all in-tree
|
||||||
|
> plugins (`plugins/<name>/`) inherit by default. It is a *retrospective*
|
||||||
|
> document — the codebase came first, the principles below were distilled from
|
||||||
|
> `CLAUDE.md`, `README.md`, and the shape of the existing implementation.
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
### I. Self-Hosted, Single-User, Docker-First
|
||||||
|
|
||||||
|
Slopsmith targets one user running one container against a personal
|
||||||
|
song library folder. There is no multi-tenant model, no
|
||||||
|
authentication, no rate limiting, and no shared backend. Deployment is
|
||||||
|
expressed as a single `docker compose up -d` against the bundled
|
||||||
|
`Dockerfile`; everything required (vgmstream, FFmpeg, FluidSynth,
|
||||||
|
Python) is baked into the image so the host machine needs
|
||||||
|
only Docker. Native (non-Docker) launch is supported for development
|
||||||
|
but not the primary supported path.
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- Do not introduce a user/account model, multi-tenant data partitioning,
|
||||||
|
or auth middleware. All endpoints assume one trusted local user.
|
||||||
|
- New runtime dependencies (binaries, Python modules) must be installable
|
||||||
|
inside the existing `Dockerfile`. If something cannot run in the
|
||||||
|
container, it does not ship in core.
|
||||||
|
- `DLC_DIR` and `CONFIG_DIR` are the only required configuration inputs.
|
||||||
|
Adding a new mandatory path/env var is a constitutional change.
|
||||||
|
|
||||||
|
### II. Vanilla Frontend — No Frameworks
|
||||||
|
|
||||||
|
The frontend (`static/app.js`, `static/highway.js`, `static/index.html`,
|
||||||
|
`static/style.css`) is plain JavaScript with the `fetch` API, direct DOM
|
||||||
|
manipulation, and the Canvas 2D / WebGL2 APIs. The only style framework
|
||||||
|
is Tailwind CSS, served as a prebuilt static stylesheet
|
||||||
|
(`static/tailwind.min.css`, regenerated by `scripts/build-tailwind.sh`)
|
||||||
|
— never the runtime Play CDN, whose on-the-fly JIT rescans the DOM on
|
||||||
|
the main thread and caused sustained frame drops with the 3D highway
|
||||||
|
(slopsmith-desktop#110). No React, Vue, Svelte, bundler, transpiler, or
|
||||||
|
TypeScript appears in the core static tree, and no build step runs on
|
||||||
|
the serve path: the Tailwind build is a maintainer-only one-shot whose
|
||||||
|
output is committed, so Docker / desktop / end users never build. New
|
||||||
|
features extend `app.js` and the existing globals (`window.playSong`,
|
||||||
|
`window.showScreen`, `window.createHighway`, `window.slopsmith`).
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- Do not introduce a frontend framework, JSX, or a JS build pipeline in
|
||||||
|
core. Plugins MAY ship their own bundled assets but core MUST remain
|
||||||
|
source-served.
|
||||||
|
- Because the core Tailwind stylesheet is prebuilt, it contains only the
|
||||||
|
classes present in core source at build time. Core's committed
|
||||||
|
`static/tailwind.min.css` MUST stay in sync with source — CI enforces
|
||||||
|
this by rebuilding and diffing. A plugin that uses Tailwind classes
|
||||||
|
not guaranteed in core (notably arbitrary values like `w-[37px]`)
|
||||||
|
MUST ship its own compiled stylesheet via the `styles` capability,
|
||||||
|
built with `corePlugins.preflight = false` so it emits only utilities
|
||||||
|
and does not re-apply the base reset core already provides. Plugins
|
||||||
|
MUST NOT load the Tailwind Play CDN (or any runtime CSS JIT) — the
|
||||||
|
same no-CDN, build-free-at-serve rule that binds core binds plugins.
|
||||||
|
- New UI state lives in `localStorage` (or a backend endpoint), not in a
|
||||||
|
framework store.
|
||||||
|
- Naming: camelCase JS, kebab-case CSS, snake_case plugin IDs. Player
|
||||||
|
layout invariants (`#player` flex-column, `#highway` flex:1,
|
||||||
|
`#player-controls` at the bottom) MUST be preserved.
|
||||||
|
|
||||||
|
### III. Plugins Are the Extension Point — Isolated by `load_sibling`
|
||||||
|
|
||||||
|
Functionality that is not part of the irreducible "browse + play charts"
|
||||||
|
loop ships as a plugin under `plugins/<name>/`, not as core code. Each
|
||||||
|
plugin is its own directory (typically a separate git repo), discovered
|
||||||
|
at startup via `plugin.json`, and free to add nav links, screens,
|
||||||
|
settings panels, and `/api/plugins/<id>/...` routes. Plugins MUST
|
||||||
|
isolate their backend Python imports via `context["load_sibling"]` so
|
||||||
|
two plugins shipping a generic `extractor.py` / `util.py` / `client.py`
|
||||||
|
do not collide in `sys.modules`.
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- Generic features (practice journal, setlist, metronome, tone player,
|
||||||
|
tab view, MIDI control, stem mixing, editors, etc.) belong in a plugin
|
||||||
|
repo, not in `lib/` or `server.py`.
|
||||||
|
- Plugin backend modules MUST use `context["load_sibling"]("name")` for
|
||||||
|
sibling imports. Bare `import sibling` works during transition but
|
||||||
|
triggers a startup warning when a name collides.
|
||||||
|
- Plugins MUST register routes under `/api/plugins/<plugin_id>/...`,
|
||||||
|
use `window.slopsmith.emit/on` for cross-plugin communication, and
|
||||||
|
prefix their `localStorage` keys with their plugin id.
|
||||||
|
- Plugins inherit this constitution and may layer additional rules in
|
||||||
|
their own `CLAUDE.md`, but MUST NOT relax core principles (e.g. a
|
||||||
|
plugin cannot require a frontend framework in core).
|
||||||
|
|
||||||
|
### IV. Backwards-Compatible Chart Library
|
||||||
|
|
||||||
|
The whole point of Slopsmith is that a user points it at an existing
|
||||||
|
song library folder and it Just Works. The library is scanned and
|
||||||
|
indexed in `meta.db` (SQLite via `MetadataDB`). The open Sloppak
|
||||||
|
format (`lib/sloppak.py`, `docs/sloppak-spec.md`) is the preferred
|
||||||
|
format and the home for new features; loose-folder XML charts
|
||||||
|
(`lib/loosefolder.py`) are also discovered and played as a first-class
|
||||||
|
format. Both must keep playing across releases.
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- Never modify, move, or delete files inside the user's library folder
|
||||||
|
without an explicit user-initiated action (retune, edit metadata).
|
||||||
|
- A library scan MUST be non-blocking: the user can browse already-
|
||||||
|
scanned songs while import continues. Scans MUST tolerate corrupt or
|
||||||
|
partial files without aborting the batch.
|
||||||
|
- Schema migrations on `meta.db` MUST be additive and idempotent;
|
||||||
|
unrecognized columns from a newer build MUST not crash an older one.
|
||||||
|
- Existing arrangement IDs, sloppak manifests, and the highway
|
||||||
|
WebSocket message shape are stable contracts. Breaking changes
|
||||||
|
require a CHANGELOG entry under "Migration notes".
|
||||||
|
|
||||||
|
### V. Pure-Function Core Libraries, Tested
|
||||||
|
|
||||||
|
The shared Python in `lib/` (`song.py`, `tunings.py`, `sloppak.py`,
|
||||||
|
`loosefolder.py`, `gp2rs.py`, `gp2midi.py`, `retune.py`, etc.) is written
|
||||||
|
as flat-importable, side-effect-light modules — no `__init__.py`
|
||||||
|
package, no implicit IO at import time, no global mutable state beyond
|
||||||
|
the explicit `MetadataDB` and config singletons in `server.py`. The
|
||||||
|
pytest suite under `tests/` covers pure-function helpers (note/tempo/
|
||||||
|
tick math, tuning lookups, song wire format) and runs in CI on every
|
||||||
|
push and PR to `main` against Python 3.12.
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- New `lib/` modules MUST be importable with `from <module> import X`
|
||||||
|
(flat imports, `pyproject.toml` sets `pythonpath = [".", "lib"]`).
|
||||||
|
- Pure helpers added to `lib/` SHOULD ship with pytest coverage in
|
||||||
|
`tests/test_<module>.py`. Network/filesystem-heavy code is exempt
|
||||||
|
but should be split so the pure parts are testable.
|
||||||
|
- CI MUST stay green on `main`. A failing test on `main` is a P0.
|
||||||
|
|
||||||
|
### VI. Observability Over Chattiness
|
||||||
|
|
||||||
|
All backend output goes through the stdlib `logging` pipeline configured
|
||||||
|
by `lib/logging_setup.py`, controlled by `LOG_LEVEL` / `LOG_FORMAT` /
|
||||||
|
`LOG_FILE`. Plugins receive a pre-configured `context["log"]` namespaced
|
||||||
|
to `slopsmith.plugin.<id>` and MUST use it instead of `print`. HTTP
|
||||||
|
responses carry a `X-Request-ID` header from `CorrelationIdMiddleware`
|
||||||
|
and the same id appears as `request_id` in JSON log lines. The
|
||||||
|
"Settings → Export Diagnostics" bundle (`lib/diagnostics_bundle.py`)
|
||||||
|
collects logs, hardware, plugin inventory, browser console, and per-
|
||||||
|
plugin contributed diagnostics into a single redacted zip.
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- New backend code MUST use `logging.getLogger(...)` (or the plugin
|
||||||
|
`context["log"]`). `print()` and `traceback.print_exc()` are legacy
|
||||||
|
and being migrated; do not add new ones.
|
||||||
|
- Diagnostic redaction is on by default. Anything that ships in the
|
||||||
|
bundle (DLC paths, song filenames, IPs, bearer tokens) MUST be
|
||||||
|
redacted via `lib/diagnostics_redact.py` rules before export.
|
||||||
|
- Plugins that contribute diagnostics MUST embed a versioned
|
||||||
|
`schema` field (e.g. `"my_plugin.diag.v1"`) and keep payloads
|
||||||
|
under 100 KB. Diagnostics is not a backup channel — that is
|
||||||
|
`settings.server_files`.
|
||||||
|
|
||||||
|
### VII. Versioned, Migration-Aware Settings
|
||||||
|
|
||||||
|
User configuration lives in two places: server-side under `CONFIG_DIR`
|
||||||
|
(SQLite `meta.db`, `config.yaml`, plugin opted-in files) and client-
|
||||||
|
side in browser `localStorage`. Both can be exported and re-imported
|
||||||
|
as a single bundle (`POST /api/settings/import`,
|
||||||
|
`GET /api/settings/export`, slopsmith#113). Import is two-phase:
|
||||||
|
phase-1 validates the entire bundle (schema, paths, encoding) and
|
||||||
|
phase-2 commits each file atomically via temp+rename. Plugins opt
|
||||||
|
their server-side files into the bundle via
|
||||||
|
`settings.server_files` in `plugin.json` (relpaths under `CONFIG_DIR`,
|
||||||
|
no `..`, no absolute paths).
|
||||||
|
|
||||||
|
**Non-negotiable rules**
|
||||||
|
|
||||||
|
- Server-side settings imports MUST be all-or-nothing on safety-critical
|
||||||
|
failures (path traversal, schema mismatch, decode failure). Plugin
|
||||||
|
state mismatches between export and import are recoverable warnings,
|
||||||
|
not hard failures.
|
||||||
|
- Plugins are responsible for their own internal data migration.
|
||||||
|
Importing a bundle whose schema predates the running plugin's code
|
||||||
|
MUST restore bytes verbatim — the plugin copes at next load.
|
||||||
|
- The `VERSION` file is the single source of truth for the running
|
||||||
|
release; it is auto-bumped from `slopsmith-desktop` releases via
|
||||||
|
`.github/workflows/sync-version.yml`. Manual edits are reserved for
|
||||||
|
out-of-band recovery only.
|
||||||
|
|
||||||
|
## Operating Constraints
|
||||||
|
|
||||||
|
- **Concurrency model**: FastAPI + uvicorn, sync handlers for the bulk
|
||||||
|
of routes, WebSockets for the highway data stream
|
||||||
|
(`/ws/highway/{filename}`) and retune progress (`/ws/retune`).
|
||||||
|
`MetadataDB` uses a `threading.Lock`; long-running work (rescan,
|
||||||
|
retune, sloppak assembly) is dispatched via background threads or
|
||||||
|
separate processes, never inline on the request path.
|
||||||
|
- **Data flow**:
|
||||||
|
`library folder → scan (sloppak.py / loosefolder.py) → MetadataDB (SQLite) →
|
||||||
|
/api/library* → static/app.js → /ws/highway → static/highway.js`.
|
||||||
|
Sloppak arrangements are served from `arrangements/<id>.json`.
|
||||||
|
- **Frontend layout invariants**: `#player` is `display:flex;
|
||||||
|
flex-direction:column; position:fixed; inset:0`; `#highway` is
|
||||||
|
`flex:1`; `#player-controls` sits at the bottom. Hiding the highway
|
||||||
|
collapses the layout — use `margin-top: auto` on controls if you
|
||||||
|
need to hide it.
|
||||||
|
- **Plugin load order**: alphabetical by directory name. The
|
||||||
|
`playSong` wrapper chain runs outermost-first (last-loaded wrapper
|
||||||
|
runs first). Plugins MUST tolerate dependent globals being absent
|
||||||
|
at load time and check at runtime
|
||||||
|
(`typeof window.X === 'function'`).
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
- **Branching**: never push directly to `main`. Always feature branch
|
||||||
|
+ PR. Exception: the automated `VERSION` bump from
|
||||||
|
`slopsmith-desktop`'s release job, which commits to `main` as
|
||||||
|
`github-actions[bot]`.
|
||||||
|
- **Reviews**: PRs run the local Codex review loop
|
||||||
|
(`feedback_codex_preflight.md`) and the GitHub Copilot review pass
|
||||||
|
(`feedback_copilot_review.md`) before being eligible to merge.
|
||||||
|
After pushing a fix, the CodeRabbit loop
|
||||||
|
(`feedback_coderabbit_review.md`) runs to silence.
|
||||||
|
- **Testing**: `pytest` for backend (`requirements-test.txt`),
|
||||||
|
Playwright for browser interactions (`tests/browser/`), CI runs both
|
||||||
|
on every push/PR to `main`.
|
||||||
|
- **CHANGELOG**: every PR updates `[Unreleased]`. Releases rename
|
||||||
|
`[Unreleased]` to `[X.Y.Z] - YYYY-MM-DD` (the VERSION bump itself is
|
||||||
|
automated).
|
||||||
|
- **Plugin gitlinks**: plugins under `plugins/` are typically separate
|
||||||
|
git repos. Branch switches on the main repo can clobber plugin
|
||||||
|
directories. Use `git update-index --assume-unchanged` and avoid
|
||||||
|
`git clean -fd` near `plugins/`.
|
||||||
|
|
||||||
|
## Governance
|
||||||
|
|
||||||
|
- This constitution governs the core repo (`server.py`, `lib/`,
|
||||||
|
`static/`, `tests/`, `.github/`, `Dockerfile`, `docker-compose.yml`).
|
||||||
|
- Plugins inherit these principles by default. A plugin MAY add
|
||||||
|
stricter rules in its own `CLAUDE.md` / `README.md`, but MUST NOT
|
||||||
|
weaken a core principle. A plugin requesting a relaxation (e.g.
|
||||||
|
shipping its own SQLite DB outside `CONFIG_DIR`) requires an
|
||||||
|
explicit constitutional amendment in this file.
|
||||||
|
- Amendments require: (a) a PR that updates this file alongside the
|
||||||
|
code change, (b) an entry in `CHANGELOG.md` under "Migration notes"
|
||||||
|
if user-visible, and (c) a corresponding update to `CLAUDE.md` so
|
||||||
|
AI agents and humans see the same source of truth.
|
||||||
|
- The principles are listed in priority order. When two principles
|
||||||
|
conflict (e.g. "vanilla frontend" vs. a plugin that wants to ship
|
||||||
|
React), the lower-numbered principle wins by default; the
|
||||||
|
higher-numbered principle's escape hatch is to live in a plugin
|
||||||
|
with its own bundled assets.
|
||||||
|
|
||||||
|
**Version**: 1.1.0 | **Ratified**: 2026-05-09 | **Last Amended**: 2026-06-01
|
||||||
Executable
+190
@@ -0,0 +1,190 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
# Consolidated prerequisite checking script
|
||||||
|
#
|
||||||
|
# This script provides unified prerequisite checking for Spec-Driven Development workflow.
|
||||||
|
# It replaces the functionality previously spread across multiple scripts.
|
||||||
|
#
|
||||||
|
# Usage: ./check-prerequisites.sh [OPTIONS]
|
||||||
|
#
|
||||||
|
# OPTIONS:
|
||||||
|
# --json Output in JSON format
|
||||||
|
# --require-tasks Require tasks.md to exist (for implementation phase)
|
||||||
|
# --include-tasks Include tasks.md in AVAILABLE_DOCS list
|
||||||
|
# --paths-only Only output path variables (no validation)
|
||||||
|
# --help, -h Show help message
|
||||||
|
#
|
||||||
|
# OUTPUTS:
|
||||||
|
# JSON mode: {"FEATURE_DIR":"...", "AVAILABLE_DOCS":["..."]}
|
||||||
|
# Text mode: FEATURE_DIR:... \n AVAILABLE_DOCS: \n ✓/✗ file.md
|
||||||
|
# Paths only: REPO_ROOT: ... \n BRANCH: ... \n FEATURE_DIR: ... etc.
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Parse command line arguments
|
||||||
|
JSON_MODE=false
|
||||||
|
REQUIRE_TASKS=false
|
||||||
|
INCLUDE_TASKS=false
|
||||||
|
PATHS_ONLY=false
|
||||||
|
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--json)
|
||||||
|
JSON_MODE=true
|
||||||
|
;;
|
||||||
|
--require-tasks)
|
||||||
|
REQUIRE_TASKS=true
|
||||||
|
;;
|
||||||
|
--include-tasks)
|
||||||
|
INCLUDE_TASKS=true
|
||||||
|
;;
|
||||||
|
--paths-only)
|
||||||
|
PATHS_ONLY=true
|
||||||
|
;;
|
||||||
|
--help|-h)
|
||||||
|
cat << 'EOF'
|
||||||
|
Usage: check-prerequisites.sh [OPTIONS]
|
||||||
|
|
||||||
|
Consolidated prerequisite checking for Spec-Driven Development workflow.
|
||||||
|
|
||||||
|
OPTIONS:
|
||||||
|
--json Output in JSON format
|
||||||
|
--require-tasks Require tasks.md to exist (for implementation phase)
|
||||||
|
--include-tasks Include tasks.md in AVAILABLE_DOCS list
|
||||||
|
--paths-only Only output path variables (no prerequisite validation)
|
||||||
|
--help, -h Show this help message
|
||||||
|
|
||||||
|
EXAMPLES:
|
||||||
|
# Check task prerequisites (plan.md required)
|
||||||
|
./check-prerequisites.sh --json
|
||||||
|
|
||||||
|
# Check implementation prerequisites (plan.md + tasks.md required)
|
||||||
|
./check-prerequisites.sh --json --require-tasks --include-tasks
|
||||||
|
|
||||||
|
# Get feature paths only (no validation)
|
||||||
|
./check-prerequisites.sh --paths-only
|
||||||
|
|
||||||
|
EOF
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "ERROR: Unknown option '$arg'. Use --help for usage information." >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Source common functions
|
||||||
|
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
source "$SCRIPT_DIR/common.sh"
|
||||||
|
|
||||||
|
# Get feature paths and validate branch
|
||||||
|
_paths_output=$(get_feature_paths) || { echo "ERROR: Failed to resolve feature paths" >&2; exit 1; }
|
||||||
|
eval "$_paths_output"
|
||||||
|
unset _paths_output
|
||||||
|
check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
|
||||||
|
|
||||||
|
# If paths-only mode, output paths and exit (support JSON + paths-only combined)
|
||||||
|
if $PATHS_ONLY; then
|
||||||
|
if $JSON_MODE; then
|
||||||
|
# Minimal JSON paths payload (no validation performed)
|
||||||
|
if has_jq; then
|
||||||
|
jq -cn \
|
||||||
|
--arg repo_root "$REPO_ROOT" \
|
||||||
|
--arg branch "$CURRENT_BRANCH" \
|
||||||
|
--arg feature_dir "$FEATURE_DIR" \
|
||||||
|
--arg feature_spec "$FEATURE_SPEC" \
|
||||||
|
--arg impl_plan "$IMPL_PLAN" \
|
||||||
|
--arg tasks "$TASKS" \
|
||||||
|
'{REPO_ROOT:$repo_root,BRANCH:$branch,FEATURE_DIR:$feature_dir,FEATURE_SPEC:$feature_spec,IMPL_PLAN:$impl_plan,TASKS:$tasks}'
|
||||||
|
else
|
||||||
|
printf '{"REPO_ROOT":"%s","BRANCH":"%s","FEATURE_DIR":"%s","FEATURE_SPEC":"%s","IMPL_PLAN":"%s","TASKS":"%s"}\n' \
|
||||||
|
"$(json_escape "$REPO_ROOT")" "$(json_escape "$CURRENT_BRANCH")" "$(json_escape "$FEATURE_DIR")" "$(json_escape "$FEATURE_SPEC")" "$(json_escape "$IMPL_PLAN")" "$(json_escape "$TASKS")"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "REPO_ROOT: $REPO_ROOT"
|
||||||
|
echo "BRANCH: $CURRENT_BRANCH"
|
||||||
|
echo "FEATURE_DIR: $FEATURE_DIR"
|
||||||
|
echo "FEATURE_SPEC: $FEATURE_SPEC"
|
||||||
|
echo "IMPL_PLAN: $IMPL_PLAN"
|
||||||
|
echo "TASKS: $TASKS"
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Validate required directories and files
|
||||||
|
if [[ ! -d "$FEATURE_DIR" ]]; then
|
||||||
|
echo "ERROR: Feature directory not found: $FEATURE_DIR" >&2
|
||||||
|
echo "Run /speckit.specify first to create the feature structure." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$IMPL_PLAN" ]]; then
|
||||||
|
echo "ERROR: plan.md not found in $FEATURE_DIR" >&2
|
||||||
|
echo "Run /speckit.plan first to create the implementation plan." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check for tasks.md if required
|
||||||
|
if $REQUIRE_TASKS && [[ ! -f "$TASKS" ]]; then
|
||||||
|
echo "ERROR: tasks.md not found in $FEATURE_DIR" >&2
|
||||||
|
echo "Run /speckit.tasks first to create the task list." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Build list of available documents
|
||||||
|
docs=()
|
||||||
|
|
||||||
|
# Always check these optional docs
|
||||||
|
[[ -f "$RESEARCH" ]] && docs+=("research.md")
|
||||||
|
[[ -f "$DATA_MODEL" ]] && docs+=("data-model.md")
|
||||||
|
|
||||||
|
# Check contracts directory (only if it exists and has files)
|
||||||
|
if [[ -d "$CONTRACTS_DIR" ]] && [[ -n "$(ls -A "$CONTRACTS_DIR" 2>/dev/null)" ]]; then
|
||||||
|
docs+=("contracts/")
|
||||||
|
fi
|
||||||
|
|
||||||
|
[[ -f "$QUICKSTART" ]] && docs+=("quickstart.md")
|
||||||
|
|
||||||
|
# Include tasks.md if requested and it exists
|
||||||
|
if $INCLUDE_TASKS && [[ -f "$TASKS" ]]; then
|
||||||
|
docs+=("tasks.md")
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Output results
|
||||||
|
if $JSON_MODE; then
|
||||||
|
# Build JSON array of documents
|
||||||
|
if has_jq; then
|
||||||
|
if [[ ${#docs[@]} -eq 0 ]]; then
|
||||||
|
json_docs="[]"
|
||||||
|
else
|
||||||
|
json_docs=$(printf '%s\n' "${docs[@]}" | jq -R . | jq -s .)
|
||||||
|
fi
|
||||||
|
jq -cn \
|
||||||
|
--arg feature_dir "$FEATURE_DIR" \
|
||||||
|
--argjson docs "$json_docs" \
|
||||||
|
'{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs}'
|
||||||
|
else
|
||||||
|
if [[ ${#docs[@]} -eq 0 ]]; then
|
||||||
|
json_docs="[]"
|
||||||
|
else
|
||||||
|
json_docs=$(for d in "${docs[@]}"; do printf '"%s",' "$(json_escape "$d")"; done)
|
||||||
|
json_docs="[${json_docs%,}]"
|
||||||
|
fi
|
||||||
|
printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s}\n' "$(json_escape "$FEATURE_DIR")" "$json_docs"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# Text output
|
||||||
|
echo "FEATURE_DIR:$FEATURE_DIR"
|
||||||
|
echo "AVAILABLE_DOCS:"
|
||||||
|
|
||||||
|
# Show status of each potential document
|
||||||
|
check_file "$RESEARCH" "research.md"
|
||||||
|
check_file "$DATA_MODEL" "data-model.md"
|
||||||
|
check_dir "$CONTRACTS_DIR" "contracts/"
|
||||||
|
check_file "$QUICKSTART" "quickstart.md"
|
||||||
|
|
||||||
|
if $INCLUDE_TASKS; then
|
||||||
|
check_file "$TASKS" "tasks.md"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
Executable
+645
@@ -0,0 +1,645 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Common functions and variables for all scripts
|
||||||
|
|
||||||
|
# Find repository root by searching upward for .specify directory
|
||||||
|
# This is the primary marker for spec-kit projects
|
||||||
|
find_specify_root() {
|
||||||
|
local dir="${1:-$(pwd)}"
|
||||||
|
# Normalize to absolute path to prevent infinite loop with relative paths
|
||||||
|
# Use -- to handle paths starting with - (e.g., -P, -L)
|
||||||
|
dir="$(cd -- "$dir" 2>/dev/null && pwd)" || return 1
|
||||||
|
local prev_dir=""
|
||||||
|
while true; do
|
||||||
|
if [ -d "$dir/.specify" ]; then
|
||||||
|
echo "$dir"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
# Stop if we've reached filesystem root or dirname stops changing
|
||||||
|
if [ "$dir" = "/" ] || [ "$dir" = "$prev_dir" ]; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
prev_dir="$dir"
|
||||||
|
dir="$(dirname "$dir")"
|
||||||
|
done
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get repository root, prioritizing .specify directory over git
|
||||||
|
# This prevents using a parent git repo when spec-kit is initialized in a subdirectory
|
||||||
|
get_repo_root() {
|
||||||
|
# First, look for .specify directory (spec-kit's own marker)
|
||||||
|
local specify_root
|
||||||
|
if specify_root=$(find_specify_root); then
|
||||||
|
echo "$specify_root"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Fallback to git if no .specify found
|
||||||
|
if git rev-parse --show-toplevel >/dev/null 2>&1; then
|
||||||
|
git rev-parse --show-toplevel
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Final fallback to script location for non-git repos
|
||||||
|
local script_dir="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
(cd "$script_dir/../../.." && pwd)
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get current branch, with fallback for non-git repositories
|
||||||
|
get_current_branch() {
|
||||||
|
# First check if SPECIFY_FEATURE environment variable is set
|
||||||
|
if [[ -n "${SPECIFY_FEATURE:-}" ]]; then
|
||||||
|
echo "$SPECIFY_FEATURE"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Then check git if available at the spec-kit root (not parent)
|
||||||
|
local repo_root=$(get_repo_root)
|
||||||
|
if has_git; then
|
||||||
|
git -C "$repo_root" rev-parse --abbrev-ref HEAD
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# For non-git repos, try to find the latest feature directory
|
||||||
|
local specs_dir="$repo_root/specs"
|
||||||
|
|
||||||
|
if [[ -d "$specs_dir" ]]; then
|
||||||
|
local latest_feature=""
|
||||||
|
local highest=0
|
||||||
|
local latest_timestamp=""
|
||||||
|
|
||||||
|
for dir in "$specs_dir"/*; do
|
||||||
|
if [[ -d "$dir" ]]; then
|
||||||
|
local dirname=$(basename "$dir")
|
||||||
|
if [[ "$dirname" =~ ^([0-9]{8}-[0-9]{6})- ]]; then
|
||||||
|
# Timestamp-based branch: compare lexicographically
|
||||||
|
local ts="${BASH_REMATCH[1]}"
|
||||||
|
if [[ "$ts" > "$latest_timestamp" ]]; then
|
||||||
|
latest_timestamp="$ts"
|
||||||
|
latest_feature=$dirname
|
||||||
|
fi
|
||||||
|
elif [[ "$dirname" =~ ^([0-9]{3,})- ]]; then
|
||||||
|
local number=${BASH_REMATCH[1]}
|
||||||
|
number=$((10#$number))
|
||||||
|
if [[ "$number" -gt "$highest" ]]; then
|
||||||
|
highest=$number
|
||||||
|
# Only update if no timestamp branch found yet
|
||||||
|
if [[ -z "$latest_timestamp" ]]; then
|
||||||
|
latest_feature=$dirname
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ -n "$latest_feature" ]]; then
|
||||||
|
echo "$latest_feature"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "main" # Final fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if we have git available at the spec-kit root level
|
||||||
|
# Returns true only if git is installed and the repo root is inside a git work tree
|
||||||
|
# Handles both regular repos (.git directory) and worktrees/submodules (.git file)
|
||||||
|
has_git() {
|
||||||
|
# First check if git command is available (before calling get_repo_root which may use git)
|
||||||
|
command -v git >/dev/null 2>&1 || return 1
|
||||||
|
local repo_root=$(get_repo_root)
|
||||||
|
# Check if .git exists (directory or file for worktrees/submodules)
|
||||||
|
[ -e "$repo_root/.git" ] || return 1
|
||||||
|
# Verify it's actually a valid git work tree
|
||||||
|
git -C "$repo_root" rev-parse --is-inside-work-tree >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Strip a single optional path segment (e.g. gitflow "feat/004-name" -> "004-name").
|
||||||
|
# Only when the full name is exactly two slash-free segments; otherwise returns the raw name.
|
||||||
|
spec_kit_effective_branch_name() {
|
||||||
|
local raw="$1"
|
||||||
|
if [[ "$raw" =~ ^([^/]+)/([^/]+)$ ]]; then
|
||||||
|
printf '%s\n' "${BASH_REMATCH[2]}"
|
||||||
|
else
|
||||||
|
printf '%s\n' "$raw"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
check_feature_branch() {
|
||||||
|
local raw="$1"
|
||||||
|
local has_git_repo="$2"
|
||||||
|
|
||||||
|
# For non-git repos, we can't enforce branch naming but still provide output
|
||||||
|
if [[ "$has_git_repo" != "true" ]]; then
|
||||||
|
echo "[specify] Warning: Git repository not detected; skipped branch validation" >&2
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
local branch
|
||||||
|
branch=$(spec_kit_effective_branch_name "$raw")
|
||||||
|
|
||||||
|
# Accept sequential prefix (3+ digits) but exclude malformed timestamps
|
||||||
|
# Malformed: 7-or-8 digit date + 6-digit time with no trailing slug (e.g. "2026031-143022" or "20260319-143022")
|
||||||
|
local is_sequential=false
|
||||||
|
if [[ "$branch" =~ ^[0-9]{3,}- ]] && [[ ! "$branch" =~ ^[0-9]{7}-[0-9]{6}- ]] && [[ ! "$branch" =~ ^[0-9]{7,8}-[0-9]{6}$ ]]; then
|
||||||
|
is_sequential=true
|
||||||
|
fi
|
||||||
|
if [[ "$is_sequential" != "true" ]] && [[ ! "$branch" =~ ^[0-9]{8}-[0-9]{6}- ]]; then
|
||||||
|
echo "ERROR: Not on a feature branch. Current branch: $raw" >&2
|
||||||
|
echo "Feature branches should be named like: 001-feature-name, 1234-feature-name, or 20260319-143022-feature-name" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Safely read .specify/feature.json's "feature_directory" value.
|
||||||
|
# Prints the raw value (possibly relative) to stdout, or empty string if the file
|
||||||
|
# is missing, unparseable, or does not contain the key. Always returns 0 so callers
|
||||||
|
# under `set -e` cannot be aborted by parser failure.
|
||||||
|
# Parser order mirrors the historical get_feature_paths behavior: jq -> python3 -> grep/sed.
|
||||||
|
read_feature_json_feature_directory() {
|
||||||
|
local repo_root="$1"
|
||||||
|
local fj="$repo_root/.specify/feature.json"
|
||||||
|
[[ -f "$fj" ]] || { printf '%s' ''; return 0; }
|
||||||
|
|
||||||
|
local _fd=''
|
||||||
|
if command -v jq >/dev/null 2>&1; then
|
||||||
|
if ! _fd=$(jq -r '.feature_directory // empty' "$fj" 2>/dev/null); then
|
||||||
|
_fd=''
|
||||||
|
fi
|
||||||
|
elif command -v python3 >/dev/null 2>&1; then
|
||||||
|
# Use Python so pretty-printed/multi-line JSON still parses correctly.
|
||||||
|
if ! _fd=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); v=d.get('feature_directory'); print(v if v else '')" "$fj" 2>/dev/null); then
|
||||||
|
_fd=''
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# Last-resort single-line grep/sed fallback. The `|| true` guards against
|
||||||
|
# grep returning 1 (no match) aborting under `set -e` / `pipefail`.
|
||||||
|
_fd=$( { grep -E '"feature_directory"[[:space:]]*:' "$fj" 2>/dev/null || true; } \
|
||||||
|
| head -n 1 \
|
||||||
|
| sed -E 's/^[^:]*:[[:space:]]*"([^"]*)".*$/\1/' )
|
||||||
|
fi
|
||||||
|
|
||||||
|
printf '%s' "$_fd"
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Returns 0 when .specify/feature.json lists feature_directory that exists as a directory
|
||||||
|
# and matches the resolved active FEATURE_DIR (so /speckit.plan can skip git branch pattern checks).
|
||||||
|
# Delegates parsing to read_feature_json_feature_directory, which is safe under `set -e`.
|
||||||
|
feature_json_matches_feature_dir() {
|
||||||
|
local repo_root="$1"
|
||||||
|
local active_feature_dir="$2"
|
||||||
|
|
||||||
|
local _fd
|
||||||
|
_fd=$(read_feature_json_feature_directory "$repo_root")
|
||||||
|
|
||||||
|
[[ -n "$_fd" ]] || return 1
|
||||||
|
[[ "$_fd" != /* ]] && _fd="$repo_root/$_fd"
|
||||||
|
[[ -d "$_fd" ]] || return 1
|
||||||
|
|
||||||
|
local norm_json norm_active
|
||||||
|
norm_json="$(cd -- "$_fd" 2>/dev/null && pwd -P)" || return 1
|
||||||
|
norm_active="$(cd -- "$active_feature_dir" 2>/dev/null && pwd -P)" || return 1
|
||||||
|
|
||||||
|
[[ "$norm_json" == "$norm_active" ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
# Find feature directory by numeric prefix instead of exact branch match
|
||||||
|
# This allows multiple branches to work on the same spec (e.g., 004-fix-bug, 004-add-feature)
|
||||||
|
find_feature_dir_by_prefix() {
|
||||||
|
local repo_root="$1"
|
||||||
|
local branch_name
|
||||||
|
branch_name=$(spec_kit_effective_branch_name "$2")
|
||||||
|
local specs_dir="$repo_root/specs"
|
||||||
|
|
||||||
|
# Extract prefix from branch (e.g., "004" from "004-whatever" or "20260319-143022" from timestamp branches)
|
||||||
|
local prefix=""
|
||||||
|
if [[ "$branch_name" =~ ^([0-9]{8}-[0-9]{6})- ]]; then
|
||||||
|
prefix="${BASH_REMATCH[1]}"
|
||||||
|
elif [[ "$branch_name" =~ ^([0-9]{3,})- ]]; then
|
||||||
|
prefix="${BASH_REMATCH[1]}"
|
||||||
|
else
|
||||||
|
# If branch doesn't have a recognized prefix, fall back to exact match
|
||||||
|
echo "$specs_dir/$branch_name"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Search for directories in specs/ that start with this prefix
|
||||||
|
local matches=()
|
||||||
|
if [[ -d "$specs_dir" ]]; then
|
||||||
|
for dir in "$specs_dir"/"$prefix"-*; do
|
||||||
|
if [[ -d "$dir" ]]; then
|
||||||
|
matches+=("$(basename "$dir")")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Handle results
|
||||||
|
if [[ ${#matches[@]} -eq 0 ]]; then
|
||||||
|
# No match found - return the branch name path (will fail later with clear error)
|
||||||
|
echo "$specs_dir/$branch_name"
|
||||||
|
elif [[ ${#matches[@]} -eq 1 ]]; then
|
||||||
|
# Exactly one match - perfect!
|
||||||
|
echo "$specs_dir/${matches[0]}"
|
||||||
|
else
|
||||||
|
# Multiple matches - this shouldn't happen with proper naming convention
|
||||||
|
echo "ERROR: Multiple spec directories found with prefix '$prefix': ${matches[*]}" >&2
|
||||||
|
echo "Please ensure only one spec directory exists per prefix." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
get_feature_paths() {
|
||||||
|
local repo_root=$(get_repo_root)
|
||||||
|
local current_branch=$(get_current_branch)
|
||||||
|
local has_git_repo="false"
|
||||||
|
|
||||||
|
if has_git; then
|
||||||
|
has_git_repo="true"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Resolve feature directory. Priority:
|
||||||
|
# 1. SPECIFY_FEATURE_DIRECTORY env var (explicit override)
|
||||||
|
# 2. .specify/feature.json "feature_directory" key (persisted by /speckit.specify)
|
||||||
|
# 3. Branch-name-based prefix lookup (legacy fallback)
|
||||||
|
local feature_dir
|
||||||
|
if [[ -n "${SPECIFY_FEATURE_DIRECTORY:-}" ]]; then
|
||||||
|
feature_dir="$SPECIFY_FEATURE_DIRECTORY"
|
||||||
|
# Normalize relative paths to absolute under repo root
|
||||||
|
[[ "$feature_dir" != /* ]] && feature_dir="$repo_root/$feature_dir"
|
||||||
|
elif [[ -f "$repo_root/.specify/feature.json" ]]; then
|
||||||
|
# Shared, set -e-safe parser: jq -> python3 -> grep/sed. Returns empty on
|
||||||
|
# missing/unparseable/unset so we fall through to the branch-prefix lookup.
|
||||||
|
local _fd
|
||||||
|
_fd=$(read_feature_json_feature_directory "$repo_root")
|
||||||
|
if [[ -n "$_fd" ]]; then
|
||||||
|
feature_dir="$_fd"
|
||||||
|
# Normalize relative paths to absolute under repo root
|
||||||
|
[[ "$feature_dir" != /* ]] && feature_dir="$repo_root/$feature_dir"
|
||||||
|
elif ! feature_dir=$(find_feature_dir_by_prefix "$repo_root" "$current_branch"); then
|
||||||
|
echo "ERROR: Failed to resolve feature directory" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
elif ! feature_dir=$(find_feature_dir_by_prefix "$repo_root" "$current_branch"); then
|
||||||
|
echo "ERROR: Failed to resolve feature directory" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Use printf '%q' to safely quote values, preventing shell injection
|
||||||
|
# via crafted branch names or paths containing special characters
|
||||||
|
printf 'REPO_ROOT=%q\n' "$repo_root"
|
||||||
|
printf 'CURRENT_BRANCH=%q\n' "$current_branch"
|
||||||
|
printf 'HAS_GIT=%q\n' "$has_git_repo"
|
||||||
|
printf 'FEATURE_DIR=%q\n' "$feature_dir"
|
||||||
|
printf 'FEATURE_SPEC=%q\n' "$feature_dir/spec.md"
|
||||||
|
printf 'IMPL_PLAN=%q\n' "$feature_dir/plan.md"
|
||||||
|
printf 'TASKS=%q\n' "$feature_dir/tasks.md"
|
||||||
|
printf 'RESEARCH=%q\n' "$feature_dir/research.md"
|
||||||
|
printf 'DATA_MODEL=%q\n' "$feature_dir/data-model.md"
|
||||||
|
printf 'QUICKSTART=%q\n' "$feature_dir/quickstart.md"
|
||||||
|
printf 'CONTRACTS_DIR=%q\n' "$feature_dir/contracts"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if jq is available for safe JSON construction
|
||||||
|
has_jq() {
|
||||||
|
command -v jq >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Escape a string for safe embedding in a JSON value (fallback when jq is unavailable).
|
||||||
|
# Handles backslash, double-quote, and JSON-required control character escapes (RFC 8259).
|
||||||
|
json_escape() {
|
||||||
|
local s="$1"
|
||||||
|
s="${s//\\/\\\\}"
|
||||||
|
s="${s//\"/\\\"}"
|
||||||
|
s="${s//$'\n'/\\n}"
|
||||||
|
s="${s//$'\t'/\\t}"
|
||||||
|
s="${s//$'\r'/\\r}"
|
||||||
|
s="${s//$'\b'/\\b}"
|
||||||
|
s="${s//$'\f'/\\f}"
|
||||||
|
# Escape any remaining U+0001-U+001F control characters as \uXXXX.
|
||||||
|
# (U+0000/NUL cannot appear in bash strings and is excluded.)
|
||||||
|
# LC_ALL=C ensures ${#s} counts bytes and ${s:$i:1} yields single bytes,
|
||||||
|
# so multi-byte UTF-8 sequences (first byte >= 0xC0) pass through intact.
|
||||||
|
local LC_ALL=C
|
||||||
|
local i char code
|
||||||
|
for (( i=0; i<${#s}; i++ )); do
|
||||||
|
char="${s:$i:1}"
|
||||||
|
printf -v code '%d' "'$char" 2>/dev/null || code=256
|
||||||
|
if (( code >= 1 && code <= 31 )); then
|
||||||
|
printf '\\u%04x' "$code"
|
||||||
|
else
|
||||||
|
printf '%s' "$char"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
check_file() { [[ -f "$1" ]] && echo " ✓ $2" || echo " ✗ $2"; }
|
||||||
|
check_dir() { [[ -d "$1" && -n $(ls -A "$1" 2>/dev/null) ]] && echo " ✓ $2" || echo " ✗ $2"; }
|
||||||
|
|
||||||
|
# Resolve a template name to a file path using the priority stack:
|
||||||
|
# 1. .specify/templates/overrides/
|
||||||
|
# 2. .specify/presets/<preset-id>/templates/ (sorted by priority from .registry)
|
||||||
|
# 3. .specify/extensions/<ext-id>/templates/
|
||||||
|
# 4. .specify/templates/ (core)
|
||||||
|
resolve_template() {
|
||||||
|
local template_name="$1"
|
||||||
|
local repo_root="$2"
|
||||||
|
local base="$repo_root/.specify/templates"
|
||||||
|
|
||||||
|
# Priority 1: Project overrides
|
||||||
|
local override="$base/overrides/${template_name}.md"
|
||||||
|
[ -f "$override" ] && echo "$override" && return 0
|
||||||
|
|
||||||
|
# Priority 2: Installed presets (sorted by priority from .registry)
|
||||||
|
local presets_dir="$repo_root/.specify/presets"
|
||||||
|
if [ -d "$presets_dir" ]; then
|
||||||
|
local registry_file="$presets_dir/.registry"
|
||||||
|
if [ -f "$registry_file" ] && command -v python3 >/dev/null 2>&1; then
|
||||||
|
# Read preset IDs sorted by priority (lower number = higher precedence).
|
||||||
|
# The python3 call is wrapped in an if-condition so that set -e does not
|
||||||
|
# abort the function when python3 exits non-zero (e.g. invalid JSON).
|
||||||
|
local sorted_presets=""
|
||||||
|
if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" python3 -c "
|
||||||
|
import json, sys, os
|
||||||
|
try:
|
||||||
|
with open(os.environ['SPECKIT_REGISTRY']) as f:
|
||||||
|
data = json.load(f)
|
||||||
|
presets = data.get('presets', {})
|
||||||
|
for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10) if isinstance(x[1], dict) else 10):
|
||||||
|
if isinstance(meta, dict) and meta.get('enabled', True) is not False:
|
||||||
|
print(pid)
|
||||||
|
except Exception:
|
||||||
|
sys.exit(1)
|
||||||
|
" 2>/dev/null); then
|
||||||
|
if [ -n "$sorted_presets" ]; then
|
||||||
|
# python3 succeeded and returned preset IDs — search in priority order
|
||||||
|
while IFS= read -r preset_id; do
|
||||||
|
local candidate="$presets_dir/$preset_id/templates/${template_name}.md"
|
||||||
|
[ -f "$candidate" ] && echo "$candidate" && return 0
|
||||||
|
done <<< "$sorted_presets"
|
||||||
|
fi
|
||||||
|
# python3 succeeded but registry has no presets — nothing to search
|
||||||
|
else
|
||||||
|
# python3 failed (missing, or registry parse error) — fall back to unordered directory scan
|
||||||
|
for preset in "$presets_dir"/*/; do
|
||||||
|
[ -d "$preset" ] || continue
|
||||||
|
local candidate="$preset/templates/${template_name}.md"
|
||||||
|
[ -f "$candidate" ] && echo "$candidate" && return 0
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# Fallback: alphabetical directory order (no python3 available)
|
||||||
|
for preset in "$presets_dir"/*/; do
|
||||||
|
[ -d "$preset" ] || continue
|
||||||
|
local candidate="$preset/templates/${template_name}.md"
|
||||||
|
[ -f "$candidate" ] && echo "$candidate" && return 0
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Priority 3: Extension-provided templates
|
||||||
|
local ext_dir="$repo_root/.specify/extensions"
|
||||||
|
if [ -d "$ext_dir" ]; then
|
||||||
|
for ext in "$ext_dir"/*/; do
|
||||||
|
[ -d "$ext" ] || continue
|
||||||
|
# Skip hidden directories (e.g. .backup, .cache)
|
||||||
|
case "$(basename "$ext")" in .*) continue;; esac
|
||||||
|
local candidate="$ext/templates/${template_name}.md"
|
||||||
|
[ -f "$candidate" ] && echo "$candidate" && return 0
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Priority 4: Core templates
|
||||||
|
local core="$base/${template_name}.md"
|
||||||
|
[ -f "$core" ] && echo "$core" && return 0
|
||||||
|
|
||||||
|
# Template not found in any location.
|
||||||
|
# Return 1 so callers can distinguish "not found" from "found".
|
||||||
|
# Callers running under set -e should use: TEMPLATE=$(resolve_template ...) || true
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Resolve a template name to composed content using composition strategies.
|
||||||
|
# Reads strategy metadata from preset manifests and composes content
|
||||||
|
# from multiple layers using prepend, append, or wrap strategies.
|
||||||
|
#
|
||||||
|
# Usage: CONTENT=$(resolve_template_content "template-name" "$REPO_ROOT")
|
||||||
|
# Returns composed content string on stdout; exit code 1 if not found.
|
||||||
|
resolve_template_content() {
|
||||||
|
local template_name="$1"
|
||||||
|
local repo_root="$2"
|
||||||
|
local base="$repo_root/.specify/templates"
|
||||||
|
|
||||||
|
# Collect all layers (highest priority first)
|
||||||
|
local -a layer_paths=()
|
||||||
|
local -a layer_strategies=()
|
||||||
|
|
||||||
|
# Priority 1: Project overrides (always "replace")
|
||||||
|
local override="$base/overrides/${template_name}.md"
|
||||||
|
if [ -f "$override" ]; then
|
||||||
|
layer_paths+=("$override")
|
||||||
|
layer_strategies+=("replace")
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Priority 2: Installed presets (sorted by priority from .registry)
|
||||||
|
local presets_dir="$repo_root/.specify/presets"
|
||||||
|
if [ -d "$presets_dir" ]; then
|
||||||
|
local registry_file="$presets_dir/.registry"
|
||||||
|
local sorted_presets=""
|
||||||
|
if [ -f "$registry_file" ] && command -v python3 >/dev/null 2>&1; then
|
||||||
|
if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" python3 -c "
|
||||||
|
import json, sys, os
|
||||||
|
try:
|
||||||
|
with open(os.environ['SPECKIT_REGISTRY']) as f:
|
||||||
|
data = json.load(f)
|
||||||
|
presets = data.get('presets', {})
|
||||||
|
for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10) if isinstance(x[1], dict) else 10):
|
||||||
|
if isinstance(meta, dict) and meta.get('enabled', True) is not False:
|
||||||
|
print(pid)
|
||||||
|
except Exception:
|
||||||
|
sys.exit(1)
|
||||||
|
" 2>/dev/null); then
|
||||||
|
if [ -n "$sorted_presets" ]; then
|
||||||
|
local yaml_warned=false
|
||||||
|
while IFS= read -r preset_id; do
|
||||||
|
# Read strategy and file path from preset manifest
|
||||||
|
local strategy="replace"
|
||||||
|
local manifest_file=""
|
||||||
|
local manifest="$presets_dir/$preset_id/preset.yml"
|
||||||
|
if [ -f "$manifest" ] && command -v python3 >/dev/null 2>&1; then
|
||||||
|
# Requires PyYAML; falls back to replace/convention if unavailable
|
||||||
|
local result
|
||||||
|
local py_stderr
|
||||||
|
py_stderr=$(mktemp)
|
||||||
|
result=$(SPECKIT_MANIFEST="$manifest" SPECKIT_TMPL="$template_name" python3 -c "
|
||||||
|
import sys, os
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
except ImportError:
|
||||||
|
print('yaml_missing', file=sys.stderr)
|
||||||
|
print('replace\t')
|
||||||
|
sys.exit(0)
|
||||||
|
try:
|
||||||
|
with open(os.environ['SPECKIT_MANIFEST']) as f:
|
||||||
|
data = yaml.safe_load(f)
|
||||||
|
for t in data.get('provides', {}).get('templates', []):
|
||||||
|
if t.get('name') == os.environ['SPECKIT_TMPL'] and t.get('type', 'template') == 'template':
|
||||||
|
print(t.get('strategy', 'replace') + '\t' + t.get('file', ''))
|
||||||
|
sys.exit(0)
|
||||||
|
print('replace\t')
|
||||||
|
except Exception:
|
||||||
|
print('replace\t')
|
||||||
|
" 2>"$py_stderr")
|
||||||
|
local parse_status=$?
|
||||||
|
if [ $parse_status -eq 0 ] && [ -n "$result" ]; then
|
||||||
|
IFS=$'\t' read -r strategy manifest_file <<< "$result"
|
||||||
|
strategy=$(printf '%s' "$strategy" | tr '[:upper:]' '[:lower:]')
|
||||||
|
fi
|
||||||
|
if [ "$yaml_warned" = false ] && grep -q 'yaml_missing' "$py_stderr" 2>/dev/null; then
|
||||||
|
echo "Warning: PyYAML not available; composition strategies may be ignored" >&2
|
||||||
|
yaml_warned=true
|
||||||
|
fi
|
||||||
|
rm -f "$py_stderr"
|
||||||
|
fi
|
||||||
|
# Try manifest file path first, then convention path
|
||||||
|
local candidate=""
|
||||||
|
if [ -n "$manifest_file" ]; then
|
||||||
|
# Reject absolute paths and parent traversal
|
||||||
|
case "$manifest_file" in
|
||||||
|
/*|*../*|../*) manifest_file="" ;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
if [ -n "$manifest_file" ]; then
|
||||||
|
local mf="$presets_dir/$preset_id/$manifest_file"
|
||||||
|
[ -f "$mf" ] && candidate="$mf"
|
||||||
|
fi
|
||||||
|
if [ -z "$candidate" ]; then
|
||||||
|
local cf="$presets_dir/$preset_id/templates/${template_name}.md"
|
||||||
|
[ -f "$cf" ] && candidate="$cf"
|
||||||
|
fi
|
||||||
|
if [ -n "$candidate" ]; then
|
||||||
|
layer_paths+=("$candidate")
|
||||||
|
layer_strategies+=("$strategy")
|
||||||
|
fi
|
||||||
|
done <<< "$sorted_presets"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# python3 failed — fall back to unordered directory scan (replace only)
|
||||||
|
for preset in "$presets_dir"/*/; do
|
||||||
|
[ -d "$preset" ] || continue
|
||||||
|
local candidate="$preset/templates/${template_name}.md"
|
||||||
|
if [ -f "$candidate" ]; then
|
||||||
|
layer_paths+=("$candidate")
|
||||||
|
layer_strategies+=("replace")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# No python3 or registry — fall back to unordered directory scan (replace only)
|
||||||
|
for preset in "$presets_dir"/*/; do
|
||||||
|
[ -d "$preset" ] || continue
|
||||||
|
local candidate="$preset/templates/${template_name}.md"
|
||||||
|
if [ -f "$candidate" ]; then
|
||||||
|
layer_paths+=("$candidate")
|
||||||
|
layer_strategies+=("replace")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Priority 3: Extension-provided templates (always "replace")
|
||||||
|
local ext_dir="$repo_root/.specify/extensions"
|
||||||
|
if [ -d "$ext_dir" ]; then
|
||||||
|
for ext in "$ext_dir"/*/; do
|
||||||
|
[ -d "$ext" ] || continue
|
||||||
|
case "$(basename "$ext")" in .*) continue;; esac
|
||||||
|
local candidate="$ext/templates/${template_name}.md"
|
||||||
|
if [ -f "$candidate" ]; then
|
||||||
|
layer_paths+=("$candidate")
|
||||||
|
layer_strategies+=("replace")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Priority 4: Core templates (always "replace")
|
||||||
|
local core="$base/${template_name}.md"
|
||||||
|
if [ -f "$core" ]; then
|
||||||
|
layer_paths+=("$core")
|
||||||
|
layer_strategies+=("replace")
|
||||||
|
fi
|
||||||
|
|
||||||
|
local count=${#layer_paths[@]}
|
||||||
|
[ "$count" -eq 0 ] && return 1
|
||||||
|
|
||||||
|
# Check if any layer uses a non-replace strategy
|
||||||
|
local has_composition=false
|
||||||
|
for s in "${layer_strategies[@]}"; do
|
||||||
|
[ "$s" != "replace" ] && has_composition=true && break
|
||||||
|
done
|
||||||
|
|
||||||
|
# If the top (highest-priority) layer is replace, it wins entirely —
|
||||||
|
# lower layers are irrelevant regardless of their strategies.
|
||||||
|
if [ "${layer_strategies[0]}" = "replace" ]; then
|
||||||
|
cat "${layer_paths[0]}"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$has_composition" = false ]; then
|
||||||
|
cat "${layer_paths[0]}"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Find the effective base: scan from highest priority (index 0) downward
|
||||||
|
# to find the nearest replace layer. Only compose layers above that base.
|
||||||
|
local base_idx=-1
|
||||||
|
local i
|
||||||
|
for (( i=0; i<count; i++ )); do
|
||||||
|
if [ "${layer_strategies[$i]}" = "replace" ]; then
|
||||||
|
base_idx=$i
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ $base_idx -lt 0 ]; then
|
||||||
|
return 1 # no base layer found
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Read the base content; compose layers above the base (higher priority)
|
||||||
|
local content
|
||||||
|
content=$(cat "${layer_paths[$base_idx]}"; printf x)
|
||||||
|
content="${content%x}"
|
||||||
|
|
||||||
|
for (( i=base_idx-1; i>=0; i-- )); do
|
||||||
|
local path="${layer_paths[$i]}"
|
||||||
|
local strat="${layer_strategies[$i]}"
|
||||||
|
local layer_content
|
||||||
|
# Preserve trailing newlines
|
||||||
|
layer_content=$(cat "$path"; printf x)
|
||||||
|
layer_content="${layer_content%x}"
|
||||||
|
|
||||||
|
case "$strat" in
|
||||||
|
replace) content="$layer_content" ;;
|
||||||
|
prepend) content="$(printf '%s\n\n%s' "$layer_content" "$content")" ;;
|
||||||
|
append) content="$(printf '%s\n\n%s' "$content" "$layer_content")" ;;
|
||||||
|
wrap)
|
||||||
|
case "$layer_content" in
|
||||||
|
*'{CORE_TEMPLATE}'*) ;;
|
||||||
|
*) echo "Error: wrap strategy missing {CORE_TEMPLATE} placeholder" >&2; return 1 ;;
|
||||||
|
esac
|
||||||
|
while [[ "$layer_content" == *'{CORE_TEMPLATE}'* ]]; do
|
||||||
|
local before="${layer_content%%\{CORE_TEMPLATE\}*}"
|
||||||
|
local after="${layer_content#*\{CORE_TEMPLATE\}}"
|
||||||
|
layer_content="${before}${content}${after}"
|
||||||
|
done
|
||||||
|
content="$layer_content"
|
||||||
|
;;
|
||||||
|
*) echo "Error: unknown strategy '$strat'" >&2; return 1 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
printf '%s' "$content"
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
Executable
+413
@@ -0,0 +1,413 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
JSON_MODE=false
|
||||||
|
DRY_RUN=false
|
||||||
|
ALLOW_EXISTING=false
|
||||||
|
SHORT_NAME=""
|
||||||
|
BRANCH_NUMBER=""
|
||||||
|
USE_TIMESTAMP=false
|
||||||
|
ARGS=()
|
||||||
|
i=1
|
||||||
|
while [ $i -le $# ]; do
|
||||||
|
arg="${!i}"
|
||||||
|
case "$arg" in
|
||||||
|
--json)
|
||||||
|
JSON_MODE=true
|
||||||
|
;;
|
||||||
|
--dry-run)
|
||||||
|
DRY_RUN=true
|
||||||
|
;;
|
||||||
|
--allow-existing-branch)
|
||||||
|
ALLOW_EXISTING=true
|
||||||
|
;;
|
||||||
|
--short-name)
|
||||||
|
if [ $((i + 1)) -gt $# ]; then
|
||||||
|
echo 'Error: --short-name requires a value' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
i=$((i + 1))
|
||||||
|
next_arg="${!i}"
|
||||||
|
# Check if the next argument is another option (starts with --)
|
||||||
|
if [[ "$next_arg" == --* ]]; then
|
||||||
|
echo 'Error: --short-name requires a value' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
SHORT_NAME="$next_arg"
|
||||||
|
;;
|
||||||
|
--number)
|
||||||
|
if [ $((i + 1)) -gt $# ]; then
|
||||||
|
echo 'Error: --number requires a value' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
i=$((i + 1))
|
||||||
|
next_arg="${!i}"
|
||||||
|
if [[ "$next_arg" == --* ]]; then
|
||||||
|
echo 'Error: --number requires a value' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
BRANCH_NUMBER="$next_arg"
|
||||||
|
;;
|
||||||
|
--timestamp)
|
||||||
|
USE_TIMESTAMP=true
|
||||||
|
;;
|
||||||
|
--help|-h)
|
||||||
|
echo "Usage: $0 [--json] [--dry-run] [--allow-existing-branch] [--short-name <name>] [--number N] [--timestamp] <feature_description>"
|
||||||
|
echo ""
|
||||||
|
echo "Options:"
|
||||||
|
echo " --json Output in JSON format"
|
||||||
|
echo " --dry-run Compute branch name and paths without creating branches, directories, or files"
|
||||||
|
echo " --allow-existing-branch Switch to branch if it already exists instead of failing"
|
||||||
|
echo " --short-name <name> Provide a custom short name (2-4 words) for the branch"
|
||||||
|
echo " --number N Specify branch number manually (overrides auto-detection)"
|
||||||
|
echo " --timestamp Use timestamp prefix (YYYYMMDD-HHMMSS) instead of sequential numbering"
|
||||||
|
echo " --help, -h Show this help message"
|
||||||
|
echo ""
|
||||||
|
echo "Examples:"
|
||||||
|
echo " $0 'Add user authentication system' --short-name 'user-auth'"
|
||||||
|
echo " $0 'Implement OAuth2 integration for API' --number 5"
|
||||||
|
echo " $0 --timestamp --short-name 'user-auth' 'Add user authentication'"
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
ARGS+=("$arg")
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
i=$((i + 1))
|
||||||
|
done
|
||||||
|
|
||||||
|
FEATURE_DESCRIPTION="${ARGS[*]}"
|
||||||
|
if [ -z "$FEATURE_DESCRIPTION" ]; then
|
||||||
|
echo "Usage: $0 [--json] [--dry-run] [--allow-existing-branch] [--short-name <name>] [--number N] [--timestamp] <feature_description>" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Trim whitespace and validate description is not empty (e.g., user passed only whitespace)
|
||||||
|
FEATURE_DESCRIPTION=$(echo "$FEATURE_DESCRIPTION" | sed -E 's/^[[:space:]]+|[[:space:]]+$//g')
|
||||||
|
if [ -z "$FEATURE_DESCRIPTION" ]; then
|
||||||
|
echo "Error: Feature description cannot be empty or contain only whitespace" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Function to get highest number from specs directory
|
||||||
|
get_highest_from_specs() {
|
||||||
|
local specs_dir="$1"
|
||||||
|
local highest=0
|
||||||
|
|
||||||
|
if [ -d "$specs_dir" ]; then
|
||||||
|
for dir in "$specs_dir"/*; do
|
||||||
|
[ -d "$dir" ] || continue
|
||||||
|
dirname=$(basename "$dir")
|
||||||
|
# Match sequential prefixes (>=3 digits), but skip timestamp dirs.
|
||||||
|
if echo "$dirname" | grep -Eq '^[0-9]{3,}-' && ! echo "$dirname" | grep -Eq '^[0-9]{8}-[0-9]{6}-'; then
|
||||||
|
number=$(echo "$dirname" | grep -Eo '^[0-9]+')
|
||||||
|
number=$((10#$number))
|
||||||
|
if [ "$number" -gt "$highest" ]; then
|
||||||
|
highest=$number
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "$highest"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Function to get highest number from git branches
|
||||||
|
get_highest_from_branches() {
|
||||||
|
git branch -a 2>/dev/null | sed 's/^[* ]*//; s|^remotes/[^/]*/||' | _extract_highest_number
|
||||||
|
}
|
||||||
|
|
||||||
|
# Extract the highest sequential feature number from a list of ref names (one per line).
|
||||||
|
# Shared by get_highest_from_branches and get_highest_from_remote_refs.
|
||||||
|
_extract_highest_number() {
|
||||||
|
local highest=0
|
||||||
|
while IFS= read -r name; do
|
||||||
|
[ -z "$name" ] && continue
|
||||||
|
if echo "$name" | grep -Eq '^[0-9]{3,}-' && ! echo "$name" | grep -Eq '^[0-9]{8}-[0-9]{6}-'; then
|
||||||
|
number=$(echo "$name" | grep -Eo '^[0-9]+' || echo "0")
|
||||||
|
number=$((10#$number))
|
||||||
|
if [ "$number" -gt "$highest" ]; then
|
||||||
|
highest=$number
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo "$highest"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Function to get highest number from remote branches without fetching (side-effect-free)
|
||||||
|
get_highest_from_remote_refs() {
|
||||||
|
local highest=0
|
||||||
|
|
||||||
|
for remote in $(git remote 2>/dev/null); do
|
||||||
|
local remote_highest
|
||||||
|
remote_highest=$(GIT_TERMINAL_PROMPT=0 git ls-remote --heads "$remote" 2>/dev/null | sed 's|.*refs/heads/||' | _extract_highest_number)
|
||||||
|
if [ "$remote_highest" -gt "$highest" ]; then
|
||||||
|
highest=$remote_highest
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "$highest"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Function to check existing branches (local and remote) and return next available number.
|
||||||
|
# When skip_fetch is true, queries remotes via ls-remote (read-only) instead of fetching.
|
||||||
|
check_existing_branches() {
|
||||||
|
local specs_dir="$1"
|
||||||
|
local skip_fetch="${2:-false}"
|
||||||
|
|
||||||
|
if [ "$skip_fetch" = true ]; then
|
||||||
|
# Side-effect-free: query remotes via ls-remote
|
||||||
|
local highest_remote=$(get_highest_from_remote_refs)
|
||||||
|
local highest_branch=$(get_highest_from_branches)
|
||||||
|
if [ "$highest_remote" -gt "$highest_branch" ]; then
|
||||||
|
highest_branch=$highest_remote
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# Fetch all remotes to get latest branch info (suppress errors if no remotes)
|
||||||
|
git fetch --all --prune >/dev/null 2>&1 || true
|
||||||
|
local highest_branch=$(get_highest_from_branches)
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Get highest number from ALL specs (not just matching short name)
|
||||||
|
local highest_spec=$(get_highest_from_specs "$specs_dir")
|
||||||
|
|
||||||
|
# Take the maximum of both
|
||||||
|
local max_num=$highest_branch
|
||||||
|
if [ "$highest_spec" -gt "$max_num" ]; then
|
||||||
|
max_num=$highest_spec
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Return next number
|
||||||
|
echo $((max_num + 1))
|
||||||
|
}
|
||||||
|
|
||||||
|
# Function to clean and format a branch name
|
||||||
|
clean_branch_name() {
|
||||||
|
local name="$1"
|
||||||
|
echo "$name" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/-\+/-/g' | sed 's/^-//' | sed 's/-$//'
|
||||||
|
}
|
||||||
|
|
||||||
|
# Resolve repository root using common.sh functions which prioritize .specify over git
|
||||||
|
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
source "$SCRIPT_DIR/common.sh"
|
||||||
|
|
||||||
|
REPO_ROOT=$(get_repo_root)
|
||||||
|
|
||||||
|
# Check if git is available at this repo root (not a parent)
|
||||||
|
if has_git; then
|
||||||
|
HAS_GIT=true
|
||||||
|
else
|
||||||
|
HAS_GIT=false
|
||||||
|
fi
|
||||||
|
|
||||||
|
cd "$REPO_ROOT"
|
||||||
|
|
||||||
|
SPECS_DIR="$REPO_ROOT/specs"
|
||||||
|
if [ "$DRY_RUN" != true ]; then
|
||||||
|
mkdir -p "$SPECS_DIR"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Function to generate branch name with stop word filtering and length filtering
|
||||||
|
generate_branch_name() {
|
||||||
|
local description="$1"
|
||||||
|
|
||||||
|
# Common stop words to filter out
|
||||||
|
local stop_words="^(i|a|an|the|to|for|of|in|on|at|by|with|from|is|are|was|were|be|been|being|have|has|had|do|does|did|will|would|should|could|can|may|might|must|shall|this|that|these|those|my|your|our|their|want|need|add|get|set)$"
|
||||||
|
|
||||||
|
# Convert to lowercase and split into words
|
||||||
|
local clean_name=$(echo "$description" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/ /g')
|
||||||
|
|
||||||
|
# Filter words: remove stop words and words shorter than 3 chars (unless they're uppercase acronyms in original)
|
||||||
|
local meaningful_words=()
|
||||||
|
for word in $clean_name; do
|
||||||
|
# Skip empty words
|
||||||
|
[ -z "$word" ] && continue
|
||||||
|
|
||||||
|
# Keep words that are NOT stop words AND (length >= 3 OR are potential acronyms)
|
||||||
|
if ! echo "$word" | grep -qiE "$stop_words"; then
|
||||||
|
if [ ${#word} -ge 3 ]; then
|
||||||
|
meaningful_words+=("$word")
|
||||||
|
elif echo "$description" | grep -q "\b${word^^}\b"; then
|
||||||
|
# Keep short words if they appear as uppercase in original (likely acronyms)
|
||||||
|
meaningful_words+=("$word")
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# If we have meaningful words, use first 3-4 of them
|
||||||
|
if [ ${#meaningful_words[@]} -gt 0 ]; then
|
||||||
|
local max_words=3
|
||||||
|
if [ ${#meaningful_words[@]} -eq 4 ]; then max_words=4; fi
|
||||||
|
|
||||||
|
local result=""
|
||||||
|
local count=0
|
||||||
|
for word in "${meaningful_words[@]}"; do
|
||||||
|
if [ $count -ge $max_words ]; then break; fi
|
||||||
|
if [ -n "$result" ]; then result="$result-"; fi
|
||||||
|
result="$result$word"
|
||||||
|
count=$((count + 1))
|
||||||
|
done
|
||||||
|
echo "$result"
|
||||||
|
else
|
||||||
|
# Fallback to original logic if no meaningful words found
|
||||||
|
local cleaned=$(clean_branch_name "$description")
|
||||||
|
echo "$cleaned" | tr '-' '\n' | grep -v '^$' | head -3 | tr '\n' '-' | sed 's/-$//'
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Generate branch name
|
||||||
|
if [ -n "$SHORT_NAME" ]; then
|
||||||
|
# Use provided short name, just clean it up
|
||||||
|
BRANCH_SUFFIX=$(clean_branch_name "$SHORT_NAME")
|
||||||
|
else
|
||||||
|
# Generate from description with smart filtering
|
||||||
|
BRANCH_SUFFIX=$(generate_branch_name "$FEATURE_DESCRIPTION")
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Warn if --number and --timestamp are both specified
|
||||||
|
if [ "$USE_TIMESTAMP" = true ] && [ -n "$BRANCH_NUMBER" ]; then
|
||||||
|
>&2 echo "[specify] Warning: --number is ignored when --timestamp is used"
|
||||||
|
BRANCH_NUMBER=""
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Determine branch prefix
|
||||||
|
if [ "$USE_TIMESTAMP" = true ]; then
|
||||||
|
FEATURE_NUM=$(date +%Y%m%d-%H%M%S)
|
||||||
|
BRANCH_NAME="${FEATURE_NUM}-${BRANCH_SUFFIX}"
|
||||||
|
else
|
||||||
|
# Determine branch number
|
||||||
|
if [ -z "$BRANCH_NUMBER" ]; then
|
||||||
|
if [ "$DRY_RUN" = true ] && [ "$HAS_GIT" = true ]; then
|
||||||
|
# Dry-run: query remotes via ls-remote (side-effect-free, no fetch)
|
||||||
|
BRANCH_NUMBER=$(check_existing_branches "$SPECS_DIR" true)
|
||||||
|
elif [ "$DRY_RUN" = true ]; then
|
||||||
|
# Dry-run without git: local spec dirs only
|
||||||
|
HIGHEST=$(get_highest_from_specs "$SPECS_DIR")
|
||||||
|
BRANCH_NUMBER=$((HIGHEST + 1))
|
||||||
|
elif [ "$HAS_GIT" = true ]; then
|
||||||
|
# Check existing branches on remotes
|
||||||
|
BRANCH_NUMBER=$(check_existing_branches "$SPECS_DIR")
|
||||||
|
else
|
||||||
|
# Fall back to local directory check
|
||||||
|
HIGHEST=$(get_highest_from_specs "$SPECS_DIR")
|
||||||
|
BRANCH_NUMBER=$((HIGHEST + 1))
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Force base-10 interpretation to prevent octal conversion (e.g., 010 → 8 in octal, but should be 10 in decimal)
|
||||||
|
FEATURE_NUM=$(printf "%03d" "$((10#$BRANCH_NUMBER))")
|
||||||
|
BRANCH_NAME="${FEATURE_NUM}-${BRANCH_SUFFIX}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# GitHub enforces a 244-byte limit on branch names
|
||||||
|
# Validate and truncate if necessary
|
||||||
|
MAX_BRANCH_LENGTH=244
|
||||||
|
if [ ${#BRANCH_NAME} -gt $MAX_BRANCH_LENGTH ]; then
|
||||||
|
# Calculate how much we need to trim from suffix
|
||||||
|
# Account for prefix length: timestamp (15) + hyphen (1) = 16, or sequential (3) + hyphen (1) = 4
|
||||||
|
PREFIX_LENGTH=$(( ${#FEATURE_NUM} + 1 ))
|
||||||
|
MAX_SUFFIX_LENGTH=$((MAX_BRANCH_LENGTH - PREFIX_LENGTH))
|
||||||
|
|
||||||
|
# Truncate suffix at word boundary if possible
|
||||||
|
TRUNCATED_SUFFIX=$(echo "$BRANCH_SUFFIX" | cut -c1-$MAX_SUFFIX_LENGTH)
|
||||||
|
# Remove trailing hyphen if truncation created one
|
||||||
|
TRUNCATED_SUFFIX=$(echo "$TRUNCATED_SUFFIX" | sed 's/-$//')
|
||||||
|
|
||||||
|
ORIGINAL_BRANCH_NAME="$BRANCH_NAME"
|
||||||
|
BRANCH_NAME="${FEATURE_NUM}-${TRUNCATED_SUFFIX}"
|
||||||
|
|
||||||
|
>&2 echo "[specify] Warning: Branch name exceeded GitHub's 244-byte limit"
|
||||||
|
>&2 echo "[specify] Original: $ORIGINAL_BRANCH_NAME (${#ORIGINAL_BRANCH_NAME} bytes)"
|
||||||
|
>&2 echo "[specify] Truncated to: $BRANCH_NAME (${#BRANCH_NAME} bytes)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
FEATURE_DIR="$SPECS_DIR/$BRANCH_NAME"
|
||||||
|
SPEC_FILE="$FEATURE_DIR/spec.md"
|
||||||
|
|
||||||
|
if [ "$DRY_RUN" != true ]; then
|
||||||
|
if [ "$HAS_GIT" = true ]; then
|
||||||
|
branch_create_error=""
|
||||||
|
if ! branch_create_error=$(git checkout -q -b "$BRANCH_NAME" 2>&1); then
|
||||||
|
current_branch="$(git rev-parse --abbrev-ref HEAD 2>/dev/null || true)"
|
||||||
|
# Check if branch already exists
|
||||||
|
if git branch --list "$BRANCH_NAME" | grep -q .; then
|
||||||
|
if [ "$ALLOW_EXISTING" = true ]; then
|
||||||
|
# If we're already on the branch, continue without another checkout.
|
||||||
|
if [ "$current_branch" = "$BRANCH_NAME" ]; then
|
||||||
|
:
|
||||||
|
# Otherwise switch to the existing branch instead of failing.
|
||||||
|
elif ! switch_branch_error=$(git checkout -q "$BRANCH_NAME" 2>&1); then
|
||||||
|
>&2 echo "Error: Failed to switch to existing branch '$BRANCH_NAME'. Please resolve any local changes or conflicts and try again."
|
||||||
|
if [ -n "$switch_branch_error" ]; then
|
||||||
|
>&2 printf '%s\n' "$switch_branch_error"
|
||||||
|
fi
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
elif [ "$USE_TIMESTAMP" = true ]; then
|
||||||
|
>&2 echo "Error: Branch '$BRANCH_NAME' already exists. Rerun to get a new timestamp or use a different --short-name."
|
||||||
|
exit 1
|
||||||
|
else
|
||||||
|
>&2 echo "Error: Branch '$BRANCH_NAME' already exists. Please use a different feature name or specify a different number with --number."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
>&2 echo "Error: Failed to create git branch '$BRANCH_NAME'."
|
||||||
|
if [ -n "$branch_create_error" ]; then
|
||||||
|
>&2 printf '%s\n' "$branch_create_error"
|
||||||
|
else
|
||||||
|
>&2 echo "Please check your git configuration and try again."
|
||||||
|
fi
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
>&2 echo "[specify] Warning: Git repository not detected; skipped branch creation for $BRANCH_NAME"
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$FEATURE_DIR"
|
||||||
|
|
||||||
|
if [ ! -f "$SPEC_FILE" ]; then
|
||||||
|
TEMPLATE=$(resolve_template "spec-template" "$REPO_ROOT") || true
|
||||||
|
if [ -n "$TEMPLATE" ] && [ -f "$TEMPLATE" ]; then
|
||||||
|
cp "$TEMPLATE" "$SPEC_FILE"
|
||||||
|
else
|
||||||
|
echo "Warning: Spec template not found; created empty spec file" >&2
|
||||||
|
touch "$SPEC_FILE"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Inform the user how to persist the feature variable in their own shell
|
||||||
|
printf '# To persist: export SPECIFY_FEATURE=%q\n' "$BRANCH_NAME" >&2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if $JSON_MODE; then
|
||||||
|
if command -v jq >/dev/null 2>&1; then
|
||||||
|
if [ "$DRY_RUN" = true ]; then
|
||||||
|
jq -cn \
|
||||||
|
--arg branch_name "$BRANCH_NAME" \
|
||||||
|
--arg spec_file "$SPEC_FILE" \
|
||||||
|
--arg feature_num "$FEATURE_NUM" \
|
||||||
|
'{BRANCH_NAME:$branch_name,SPEC_FILE:$spec_file,FEATURE_NUM:$feature_num,DRY_RUN:true}'
|
||||||
|
else
|
||||||
|
jq -cn \
|
||||||
|
--arg branch_name "$BRANCH_NAME" \
|
||||||
|
--arg spec_file "$SPEC_FILE" \
|
||||||
|
--arg feature_num "$FEATURE_NUM" \
|
||||||
|
'{BRANCH_NAME:$branch_name,SPEC_FILE:$spec_file,FEATURE_NUM:$feature_num}'
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
if [ "$DRY_RUN" = true ]; then
|
||||||
|
printf '{"BRANCH_NAME":"%s","SPEC_FILE":"%s","FEATURE_NUM":"%s","DRY_RUN":true}\n' "$(json_escape "$BRANCH_NAME")" "$(json_escape "$SPEC_FILE")" "$(json_escape "$FEATURE_NUM")"
|
||||||
|
else
|
||||||
|
printf '{"BRANCH_NAME":"%s","SPEC_FILE":"%s","FEATURE_NUM":"%s"}\n' "$(json_escape "$BRANCH_NAME")" "$(json_escape "$SPEC_FILE")" "$(json_escape "$FEATURE_NUM")"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "BRANCH_NAME: $BRANCH_NAME"
|
||||||
|
echo "SPEC_FILE: $SPEC_FILE"
|
||||||
|
echo "FEATURE_NUM: $FEATURE_NUM"
|
||||||
|
if [ "$DRY_RUN" != true ]; then
|
||||||
|
printf '# To persist in your shell: export SPECIFY_FEATURE=%q\n' "$BRANCH_NAME"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
Executable
+75
@@ -0,0 +1,75 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Parse command line arguments
|
||||||
|
JSON_MODE=false
|
||||||
|
ARGS=()
|
||||||
|
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--json)
|
||||||
|
JSON_MODE=true
|
||||||
|
;;
|
||||||
|
--help|-h)
|
||||||
|
echo "Usage: $0 [--json]"
|
||||||
|
echo " --json Output results in JSON format"
|
||||||
|
echo " --help Show this help message"
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
ARGS+=("$arg")
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Get script directory and load common functions
|
||||||
|
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
source "$SCRIPT_DIR/common.sh"
|
||||||
|
|
||||||
|
# Get all paths and variables from common functions
|
||||||
|
_paths_output=$(get_feature_paths) || { echo "ERROR: Failed to resolve feature paths" >&2; exit 1; }
|
||||||
|
eval "$_paths_output"
|
||||||
|
unset _paths_output
|
||||||
|
|
||||||
|
# If feature.json pins an existing feature directory, branch naming is not required.
|
||||||
|
if ! feature_json_matches_feature_dir "$REPO_ROOT" "$FEATURE_DIR"; then
|
||||||
|
check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Ensure the feature directory exists
|
||||||
|
mkdir -p "$FEATURE_DIR"
|
||||||
|
|
||||||
|
# Copy plan template if it exists
|
||||||
|
TEMPLATE=$(resolve_template "plan-template" "$REPO_ROOT") || true
|
||||||
|
if [[ -n "$TEMPLATE" ]] && [[ -f "$TEMPLATE" ]]; then
|
||||||
|
cp "$TEMPLATE" "$IMPL_PLAN"
|
||||||
|
echo "Copied plan template to $IMPL_PLAN"
|
||||||
|
else
|
||||||
|
echo "Warning: Plan template not found"
|
||||||
|
# Create a basic plan file if template doesn't exist
|
||||||
|
touch "$IMPL_PLAN"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Output results
|
||||||
|
if $JSON_MODE; then
|
||||||
|
if has_jq; then
|
||||||
|
jq -cn \
|
||||||
|
--arg feature_spec "$FEATURE_SPEC" \
|
||||||
|
--arg impl_plan "$IMPL_PLAN" \
|
||||||
|
--arg specs_dir "$FEATURE_DIR" \
|
||||||
|
--arg branch "$CURRENT_BRANCH" \
|
||||||
|
--arg has_git "$HAS_GIT" \
|
||||||
|
'{FEATURE_SPEC:$feature_spec,IMPL_PLAN:$impl_plan,SPECS_DIR:$specs_dir,BRANCH:$branch,HAS_GIT:$has_git}'
|
||||||
|
else
|
||||||
|
printf '{"FEATURE_SPEC":"%s","IMPL_PLAN":"%s","SPECS_DIR":"%s","BRANCH":"%s","HAS_GIT":"%s"}\n' \
|
||||||
|
"$(json_escape "$FEATURE_SPEC")" "$(json_escape "$IMPL_PLAN")" "$(json_escape "$FEATURE_DIR")" "$(json_escape "$CURRENT_BRANCH")" "$(json_escape "$HAS_GIT")"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "FEATURE_SPEC: $FEATURE_SPEC"
|
||||||
|
echo "IMPL_PLAN: $IMPL_PLAN"
|
||||||
|
echo "SPECS_DIR: $FEATURE_DIR"
|
||||||
|
echo "BRANCH: $CURRENT_BRANCH"
|
||||||
|
echo "HAS_GIT: $HAS_GIT"
|
||||||
|
fi
|
||||||
|
|
||||||
Executable
+96
@@ -0,0 +1,96 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Parse command line arguments
|
||||||
|
JSON_MODE=false
|
||||||
|
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--json) JSON_MODE=true ;;
|
||||||
|
--help|-h)
|
||||||
|
echo "Usage: $0 [--json]"
|
||||||
|
echo " --json Output results in JSON format"
|
||||||
|
echo " --help Show this help message"
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*) echo "ERROR: Unknown option '$arg'" >&2; exit 1 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Source common functions
|
||||||
|
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
source "$SCRIPT_DIR/common.sh"
|
||||||
|
|
||||||
|
# Get feature paths
|
||||||
|
_paths_output=$(get_feature_paths) || { echo "ERROR: Failed to resolve feature paths" >&2; exit 1; }
|
||||||
|
eval "$_paths_output"
|
||||||
|
unset _paths_output
|
||||||
|
|
||||||
|
# Validate branch
|
||||||
|
# If feature.json pins an existing feature directory, branch naming is not required.
|
||||||
|
if ! feature_json_matches_feature_dir "$REPO_ROOT" "$FEATURE_DIR"; then
|
||||||
|
check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$IMPL_PLAN" ]]; then
|
||||||
|
echo "ERROR: plan.md not found in $FEATURE_DIR" >&2
|
||||||
|
echo "Run /speckit.plan first to create the implementation plan." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$FEATURE_SPEC" ]]; then
|
||||||
|
echo "ERROR: spec.md not found in $FEATURE_DIR" >&2
|
||||||
|
echo "Run /speckit.specify first to create the feature structure." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Build available docs list
|
||||||
|
docs=()
|
||||||
|
[[ -f "$RESEARCH" ]] && docs+=("research.md")
|
||||||
|
[[ -f "$DATA_MODEL" ]] && docs+=("data-model.md")
|
||||||
|
if [[ -d "$CONTRACTS_DIR" ]] && [[ -n "$(ls -A "$CONTRACTS_DIR" 2>/dev/null)" ]]; then
|
||||||
|
docs+=("contracts/")
|
||||||
|
fi
|
||||||
|
[[ -f "$QUICKSTART" ]] && docs+=("quickstart.md")
|
||||||
|
|
||||||
|
# Resolve tasks template through override stack
|
||||||
|
TASKS_TEMPLATE=$(resolve_template "tasks-template" "$REPO_ROOT") || true
|
||||||
|
if [[ -z "$TASKS_TEMPLATE" ]] || [[ ! -f "$TASKS_TEMPLATE" ]]; then
|
||||||
|
echo "ERROR: Could not resolve required tasks-template from the template override stack for $REPO_ROOT" >&2
|
||||||
|
echo "Template 'tasks-template' was not found in any supported location (overrides, presets, extensions, or shared core). Add an override at .specify/templates/overrides/tasks-template.md, or run 'specify init' / reinstall shared infra to restore the core .specify/templates/tasks-template.md template." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Output results
|
||||||
|
if $JSON_MODE; then
|
||||||
|
if has_jq; then
|
||||||
|
if [[ ${#docs[@]} -eq 0 ]]; then
|
||||||
|
json_docs="[]"
|
||||||
|
else
|
||||||
|
json_docs=$(printf '%s\n' "${docs[@]}" | jq -R . | jq -s .)
|
||||||
|
fi
|
||||||
|
jq -cn \
|
||||||
|
--arg feature_dir "$FEATURE_DIR" \
|
||||||
|
--argjson docs "$json_docs" \
|
||||||
|
--arg tasks_template "${TASKS_TEMPLATE:-}" \
|
||||||
|
'{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs,TASKS_TEMPLATE:$tasks_template}'
|
||||||
|
else
|
||||||
|
if [[ ${#docs[@]} -eq 0 ]]; then
|
||||||
|
json_docs="[]"
|
||||||
|
else
|
||||||
|
json_docs=$(for d in "${docs[@]}"; do printf '"%s",' "$(json_escape "$d")"; done)
|
||||||
|
json_docs="[${json_docs%,}]"
|
||||||
|
fi
|
||||||
|
printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s,"TASKS_TEMPLATE":"%s"}\n' \
|
||||||
|
"$(json_escape "$FEATURE_DIR")" "$json_docs" "$(json_escape "${TASKS_TEMPLATE:-}")"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "FEATURE_DIR: $FEATURE_DIR"
|
||||||
|
echo "TASKS_TEMPLATE: ${TASKS_TEMPLATE:-not found}"
|
||||||
|
echo "AVAILABLE_DOCS:"
|
||||||
|
check_file "$RESEARCH" "research.md"
|
||||||
|
check_file "$DATA_MODEL" "data-model.md"
|
||||||
|
check_dir "$CONTRACTS_DIR" "contracts/"
|
||||||
|
check_file "$QUICKSTART" "quickstart.md"
|
||||||
|
fi
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# [CHECKLIST TYPE] Checklist: [FEATURE NAME]
|
||||||
|
|
||||||
|
**Purpose**: [Brief description of what this checklist covers]
|
||||||
|
**Created**: [DATE]
|
||||||
|
**Feature**: [Link to spec.md or relevant documentation]
|
||||||
|
|
||||||
|
**Note**: This checklist is generated by the `/speckit-checklist` command based on feature context and requirements.
|
||||||
|
|
||||||
|
<!--
|
||||||
|
============================================================================
|
||||||
|
IMPORTANT: The checklist items below are SAMPLE ITEMS for illustration only.
|
||||||
|
|
||||||
|
The /speckit-checklist command MUST replace these with actual items based on:
|
||||||
|
- User's specific checklist request
|
||||||
|
- Feature requirements from spec.md
|
||||||
|
- Technical context from plan.md
|
||||||
|
- Implementation details from tasks.md
|
||||||
|
|
||||||
|
DO NOT keep these sample items in the generated checklist file.
|
||||||
|
============================================================================
|
||||||
|
-->
|
||||||
|
|
||||||
|
## [Category 1]
|
||||||
|
|
||||||
|
- [ ] CHK001 First checklist item with clear action
|
||||||
|
- [ ] CHK002 Second checklist item
|
||||||
|
- [ ] CHK003 Third checklist item
|
||||||
|
|
||||||
|
## [Category 2]
|
||||||
|
|
||||||
|
- [ ] CHK004 Another category item
|
||||||
|
- [ ] CHK005 Item with specific criteria
|
||||||
|
- [ ] CHK006 Final item in this category
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Check items off as completed: `[x]`
|
||||||
|
- Add comments or findings inline
|
||||||
|
- Link to relevant resources or documentation
|
||||||
|
- Items are numbered sequentially for easy reference
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# [PROJECT_NAME] Constitution
|
||||||
|
<!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
### [PRINCIPLE_1_NAME]
|
||||||
|
<!-- Example: I. Library-First -->
|
||||||
|
[PRINCIPLE_1_DESCRIPTION]
|
||||||
|
<!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
|
||||||
|
|
||||||
|
### [PRINCIPLE_2_NAME]
|
||||||
|
<!-- Example: II. CLI Interface -->
|
||||||
|
[PRINCIPLE_2_DESCRIPTION]
|
||||||
|
<!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
|
||||||
|
|
||||||
|
### [PRINCIPLE_3_NAME]
|
||||||
|
<!-- Example: III. Test-First (NON-NEGOTIABLE) -->
|
||||||
|
[PRINCIPLE_3_DESCRIPTION]
|
||||||
|
<!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
|
||||||
|
|
||||||
|
### [PRINCIPLE_4_NAME]
|
||||||
|
<!-- Example: IV. Integration Testing -->
|
||||||
|
[PRINCIPLE_4_DESCRIPTION]
|
||||||
|
<!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
|
||||||
|
|
||||||
|
### [PRINCIPLE_5_NAME]
|
||||||
|
<!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
|
||||||
|
[PRINCIPLE_5_DESCRIPTION]
|
||||||
|
<!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
|
||||||
|
|
||||||
|
## [SECTION_2_NAME]
|
||||||
|
<!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
|
||||||
|
|
||||||
|
[SECTION_2_CONTENT]
|
||||||
|
<!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
|
||||||
|
|
||||||
|
## [SECTION_3_NAME]
|
||||||
|
<!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
|
||||||
|
|
||||||
|
[SECTION_3_CONTENT]
|
||||||
|
<!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
|
||||||
|
|
||||||
|
## Governance
|
||||||
|
<!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
|
||||||
|
|
||||||
|
[GOVERNANCE_RULES]
|
||||||
|
<!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
|
||||||
|
|
||||||
|
**Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
|
||||||
|
<!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
# Implementation Plan: [FEATURE]
|
||||||
|
|
||||||
|
**Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link]
|
||||||
|
**Input**: Feature specification from `/specs/[###-feature-name]/spec.md`
|
||||||
|
|
||||||
|
**Note**: This template is filled in by the `/speckit-plan` command. See `.specify/templates/plan-template.md` for the execution workflow.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
[Extract from feature spec: primary requirement + technical approach from research]
|
||||||
|
|
||||||
|
## Technical Context
|
||||||
|
|
||||||
|
<!--
|
||||||
|
ACTION REQUIRED: Replace the content in this section with the technical details
|
||||||
|
for the project. The structure here is presented in advisory capacity to guide
|
||||||
|
the iteration process.
|
||||||
|
-->
|
||||||
|
|
||||||
|
**Language/Version**: [e.g., Python 3.11, Swift 5.9, Rust 1.75 or NEEDS CLARIFICATION]
|
||||||
|
**Primary Dependencies**: [e.g., FastAPI, UIKit, LLVM or NEEDS CLARIFICATION]
|
||||||
|
**Storage**: [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
|
||||||
|
**Testing**: [e.g., pytest, XCTest, cargo test or NEEDS CLARIFICATION]
|
||||||
|
**Target Platform**: [e.g., Linux server, iOS 15+, WASM or NEEDS CLARIFICATION]
|
||||||
|
**Project Type**: [e.g., library/cli/web-service/mobile-app/compiler/desktop-app or NEEDS CLARIFICATION]
|
||||||
|
**Performance Goals**: [domain-specific, e.g., 1000 req/s, 10k lines/sec, 60 fps or NEEDS CLARIFICATION]
|
||||||
|
**Constraints**: [domain-specific, e.g., <200ms p95, <100MB memory, offline-capable or NEEDS CLARIFICATION]
|
||||||
|
**Scale/Scope**: [domain-specific, e.g., 10k users, 1M LOC, 50 screens or NEEDS CLARIFICATION]
|
||||||
|
|
||||||
|
## Constitution Check
|
||||||
|
|
||||||
|
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||||
|
|
||||||
|
[Gates determined based on constitution file]
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
### Documentation (this feature)
|
||||||
|
|
||||||
|
```text
|
||||||
|
specs/[###-feature]/
|
||||||
|
├── plan.md # This file (/speckit-plan command output)
|
||||||
|
├── research.md # Phase 0 output (/speckit-plan command)
|
||||||
|
├── data-model.md # Phase 1 output (/speckit-plan command)
|
||||||
|
├── quickstart.md # Phase 1 output (/speckit-plan command)
|
||||||
|
├── contracts/ # Phase 1 output (/speckit-plan command)
|
||||||
|
└── tasks.md # Phase 2 output (/speckit-tasks command - NOT created by /speckit-plan)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Source Code (repository root)
|
||||||
|
<!--
|
||||||
|
ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
|
||||||
|
for this feature. Delete unused options and expand the chosen structure with
|
||||||
|
real paths (e.g., apps/admin, packages/something). The delivered plan must
|
||||||
|
not include Option labels.
|
||||||
|
-->
|
||||||
|
|
||||||
|
```text
|
||||||
|
# [REMOVE IF UNUSED] Option 1: Single project (DEFAULT)
|
||||||
|
src/
|
||||||
|
├── models/
|
||||||
|
├── services/
|
||||||
|
├── cli/
|
||||||
|
└── lib/
|
||||||
|
|
||||||
|
tests/
|
||||||
|
├── contract/
|
||||||
|
├── integration/
|
||||||
|
└── unit/
|
||||||
|
|
||||||
|
# [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected)
|
||||||
|
backend/
|
||||||
|
├── src/
|
||||||
|
│ ├── models/
|
||||||
|
│ ├── services/
|
||||||
|
│ └── api/
|
||||||
|
└── tests/
|
||||||
|
|
||||||
|
frontend/
|
||||||
|
├── src/
|
||||||
|
│ ├── components/
|
||||||
|
│ ├── pages/
|
||||||
|
│ └── services/
|
||||||
|
└── tests/
|
||||||
|
|
||||||
|
# [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected)
|
||||||
|
api/
|
||||||
|
└── [same as backend above]
|
||||||
|
|
||||||
|
ios/ or android/
|
||||||
|
└── [platform-specific structure: feature modules, UI flows, platform tests]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Structure Decision**: [Document the selected structure and reference the real
|
||||||
|
directories captured above]
|
||||||
|
|
||||||
|
## Complexity Tracking
|
||||||
|
|
||||||
|
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||||
|
|
||||||
|
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||||
|
|-----------|------------|-------------------------------------|
|
||||||
|
| [e.g., 4th project] | [current need] | [why 3 projects insufficient] |
|
||||||
|
| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] |
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
# Feature Specification: [FEATURE NAME]
|
||||||
|
|
||||||
|
**Feature Branch**: `[###-feature-name]`
|
||||||
|
**Created**: [DATE]
|
||||||
|
**Status**: Draft
|
||||||
|
**Input**: User description: "$ARGUMENTS"
|
||||||
|
|
||||||
|
## User Scenarios & Testing *(mandatory)*
|
||||||
|
|
||||||
|
<!--
|
||||||
|
IMPORTANT: User stories should be PRIORITIZED as user journeys ordered by importance.
|
||||||
|
Each user story/journey must be INDEPENDENTLY TESTABLE - meaning if you implement just ONE of them,
|
||||||
|
you should still have a viable MVP (Minimum Viable Product) that delivers value.
|
||||||
|
|
||||||
|
Assign priorities (P1, P2, P3, etc.) to each story, where P1 is the most critical.
|
||||||
|
Think of each story as a standalone slice of functionality that can be:
|
||||||
|
- Developed independently
|
||||||
|
- Tested independently
|
||||||
|
- Deployed independently
|
||||||
|
- Demonstrated to users independently
|
||||||
|
-->
|
||||||
|
|
||||||
|
### User Story 1 - [Brief Title] (Priority: P1)
|
||||||
|
|
||||||
|
[Describe this user journey in plain language]
|
||||||
|
|
||||||
|
**Why this priority**: [Explain the value and why it has this priority level]
|
||||||
|
|
||||||
|
**Independent Test**: [Describe how this can be tested independently - e.g., "Can be fully tested by [specific action] and delivers [specific value]"]
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
||||||
|
2. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 2 - [Brief Title] (Priority: P2)
|
||||||
|
|
||||||
|
[Describe this user journey in plain language]
|
||||||
|
|
||||||
|
**Why this priority**: [Explain the value and why it has this priority level]
|
||||||
|
|
||||||
|
**Independent Test**: [Describe how this can be tested independently]
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### User Story 3 - [Brief Title] (Priority: P3)
|
||||||
|
|
||||||
|
[Describe this user journey in plain language]
|
||||||
|
|
||||||
|
**Why this priority**: [Explain the value and why it has this priority level]
|
||||||
|
|
||||||
|
**Independent Test**: [Describe how this can be tested independently]
|
||||||
|
|
||||||
|
**Acceptance Scenarios**:
|
||||||
|
|
||||||
|
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
[Add more user stories as needed, each with an assigned priority]
|
||||||
|
|
||||||
|
### Edge Cases
|
||||||
|
|
||||||
|
<!--
|
||||||
|
ACTION REQUIRED: The content in this section represents placeholders.
|
||||||
|
Fill them out with the right edge cases.
|
||||||
|
-->
|
||||||
|
|
||||||
|
- What happens when [boundary condition]?
|
||||||
|
- How does system handle [error scenario]?
|
||||||
|
|
||||||
|
## Requirements *(mandatory)*
|
||||||
|
|
||||||
|
<!--
|
||||||
|
ACTION REQUIRED: The content in this section represents placeholders.
|
||||||
|
Fill them out with the right functional requirements.
|
||||||
|
-->
|
||||||
|
|
||||||
|
### Functional Requirements
|
||||||
|
|
||||||
|
- **FR-001**: System MUST [specific capability, e.g., "allow users to create accounts"]
|
||||||
|
- **FR-002**: System MUST [specific capability, e.g., "validate email addresses"]
|
||||||
|
- **FR-003**: Users MUST be able to [key interaction, e.g., "reset their password"]
|
||||||
|
- **FR-004**: System MUST [data requirement, e.g., "persist user preferences"]
|
||||||
|
- **FR-005**: System MUST [behavior, e.g., "log all security events"]
|
||||||
|
|
||||||
|
*Example of marking unclear requirements:*
|
||||||
|
|
||||||
|
- **FR-006**: System MUST authenticate users via [NEEDS CLARIFICATION: auth method not specified - email/password, SSO, OAuth?]
|
||||||
|
- **FR-007**: System MUST retain user data for [NEEDS CLARIFICATION: retention period not specified]
|
||||||
|
|
||||||
|
### Key Entities *(include if feature involves data)*
|
||||||
|
|
||||||
|
- **[Entity 1]**: [What it represents, key attributes without implementation]
|
||||||
|
- **[Entity 2]**: [What it represents, relationships to other entities]
|
||||||
|
|
||||||
|
## Success Criteria *(mandatory)*
|
||||||
|
|
||||||
|
<!--
|
||||||
|
ACTION REQUIRED: Define measurable success criteria.
|
||||||
|
These must be technology-agnostic and measurable.
|
||||||
|
-->
|
||||||
|
|
||||||
|
### Measurable Outcomes
|
||||||
|
|
||||||
|
- **SC-001**: [Measurable metric, e.g., "Users can complete account creation in under 2 minutes"]
|
||||||
|
- **SC-002**: [Measurable metric, e.g., "System handles 1000 concurrent users without degradation"]
|
||||||
|
- **SC-003**: [User satisfaction metric, e.g., "90% of users successfully complete primary task on first attempt"]
|
||||||
|
- **SC-004**: [Business metric, e.g., "Reduce support tickets related to [X] by 50%"]
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
<!--
|
||||||
|
ACTION REQUIRED: The content in this section represents placeholders.
|
||||||
|
Fill them out with the right assumptions based on reasonable defaults
|
||||||
|
chosen when the feature description did not specify certain details.
|
||||||
|
-->
|
||||||
|
|
||||||
|
- [Assumption about target users, e.g., "Users have stable internet connectivity"]
|
||||||
|
- [Assumption about scope boundaries, e.g., "Mobile support is out of scope for v1"]
|
||||||
|
- [Assumption about data/environment, e.g., "Existing authentication system will be reused"]
|
||||||
|
- [Dependency on existing system/service, e.g., "Requires access to the existing user profile API"]
|
||||||
@@ -0,0 +1,251 @@
|
|||||||
|
---
|
||||||
|
|
||||||
|
description: "Task list template for feature implementation"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Tasks: [FEATURE NAME]
|
||||||
|
|
||||||
|
**Input**: Design documents from `/specs/[###-feature-name]/`
|
||||||
|
**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/
|
||||||
|
|
||||||
|
**Tests**: The examples below include test tasks. Tests are OPTIONAL - only include them if explicitly requested in the feature specification.
|
||||||
|
|
||||||
|
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
|
||||||
|
|
||||||
|
## Format: `[ID] [P?] [Story] Description`
|
||||||
|
|
||||||
|
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||||
|
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
|
||||||
|
- Include exact file paths in descriptions
|
||||||
|
|
||||||
|
## Path Conventions
|
||||||
|
|
||||||
|
- **Single project**: `src/`, `tests/` at repository root
|
||||||
|
- **Web app**: `backend/src/`, `frontend/src/`
|
||||||
|
- **Mobile**: `api/src/`, `ios/src/` or `android/src/`
|
||||||
|
- Paths shown below assume single project - adjust based on plan.md structure
|
||||||
|
|
||||||
|
<!--
|
||||||
|
============================================================================
|
||||||
|
IMPORTANT: The tasks below are SAMPLE TASKS for illustration purposes only.
|
||||||
|
|
||||||
|
The /speckit-tasks command MUST replace these with actual tasks based on:
|
||||||
|
- User stories from spec.md (with their priorities P1, P2, P3...)
|
||||||
|
- Feature requirements from plan.md
|
||||||
|
- Entities from data-model.md
|
||||||
|
- Endpoints from contracts/
|
||||||
|
|
||||||
|
Tasks MUST be organized by user story so each story can be:
|
||||||
|
- Implemented independently
|
||||||
|
- Tested independently
|
||||||
|
- Delivered as an MVP increment
|
||||||
|
|
||||||
|
DO NOT keep these sample tasks in the generated tasks.md file.
|
||||||
|
============================================================================
|
||||||
|
-->
|
||||||
|
|
||||||
|
## Phase 1: Setup (Shared Infrastructure)
|
||||||
|
|
||||||
|
**Purpose**: Project initialization and basic structure
|
||||||
|
|
||||||
|
- [ ] T001 Create project structure per implementation plan
|
||||||
|
- [ ] T002 Initialize [language] project with [framework] dependencies
|
||||||
|
- [ ] T003 [P] Configure linting and formatting tools
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Foundational (Blocking Prerequisites)
|
||||||
|
|
||||||
|
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
|
||||||
|
|
||||||
|
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
|
||||||
|
|
||||||
|
Examples of foundational tasks (adjust based on your project):
|
||||||
|
|
||||||
|
- [ ] T004 Setup database schema and migrations framework
|
||||||
|
- [ ] T005 [P] Implement authentication/authorization framework
|
||||||
|
- [ ] T006 [P] Setup API routing and middleware structure
|
||||||
|
- [ ] T007 Create base models/entities that all stories depend on
|
||||||
|
- [ ] T008 Configure error handling and logging infrastructure
|
||||||
|
- [ ] T009 Setup environment configuration management
|
||||||
|
|
||||||
|
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: User Story 1 - [Title] (Priority: P1) 🎯 MVP
|
||||||
|
|
||||||
|
**Goal**: [Brief description of what this story delivers]
|
||||||
|
|
||||||
|
**Independent Test**: [How to verify this story works on its own]
|
||||||
|
|
||||||
|
### Tests for User Story 1 (OPTIONAL - only if tests requested) ⚠️
|
||||||
|
|
||||||
|
> **NOTE: Write these tests FIRST, ensure they FAIL before implementation**
|
||||||
|
|
||||||
|
- [ ] T010 [P] [US1] Contract test for [endpoint] in tests/contract/test_[name].py
|
||||||
|
- [ ] T011 [P] [US1] Integration test for [user journey] in tests/integration/test_[name].py
|
||||||
|
|
||||||
|
### Implementation for User Story 1
|
||||||
|
|
||||||
|
- [ ] T012 [P] [US1] Create [Entity1] model in src/models/[entity1].py
|
||||||
|
- [ ] T013 [P] [US1] Create [Entity2] model in src/models/[entity2].py
|
||||||
|
- [ ] T014 [US1] Implement [Service] in src/services/[service].py (depends on T012, T013)
|
||||||
|
- [ ] T015 [US1] Implement [endpoint/feature] in src/[location]/[file].py
|
||||||
|
- [ ] T016 [US1] Add validation and error handling
|
||||||
|
- [ ] T017 [US1] Add logging for user story 1 operations
|
||||||
|
|
||||||
|
**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: User Story 2 - [Title] (Priority: P2)
|
||||||
|
|
||||||
|
**Goal**: [Brief description of what this story delivers]
|
||||||
|
|
||||||
|
**Independent Test**: [How to verify this story works on its own]
|
||||||
|
|
||||||
|
### Tests for User Story 2 (OPTIONAL - only if tests requested) ⚠️
|
||||||
|
|
||||||
|
- [ ] T018 [P] [US2] Contract test for [endpoint] in tests/contract/test_[name].py
|
||||||
|
- [ ] T019 [P] [US2] Integration test for [user journey] in tests/integration/test_[name].py
|
||||||
|
|
||||||
|
### Implementation for User Story 2
|
||||||
|
|
||||||
|
- [ ] T020 [P] [US2] Create [Entity] model in src/models/[entity].py
|
||||||
|
- [ ] T021 [US2] Implement [Service] in src/services/[service].py
|
||||||
|
- [ ] T022 [US2] Implement [endpoint/feature] in src/[location]/[file].py
|
||||||
|
- [ ] T023 [US2] Integrate with User Story 1 components (if needed)
|
||||||
|
|
||||||
|
**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: User Story 3 - [Title] (Priority: P3)
|
||||||
|
|
||||||
|
**Goal**: [Brief description of what this story delivers]
|
||||||
|
|
||||||
|
**Independent Test**: [How to verify this story works on its own]
|
||||||
|
|
||||||
|
### Tests for User Story 3 (OPTIONAL - only if tests requested) ⚠️
|
||||||
|
|
||||||
|
- [ ] T024 [P] [US3] Contract test for [endpoint] in tests/contract/test_[name].py
|
||||||
|
- [ ] T025 [P] [US3] Integration test for [user journey] in tests/integration/test_[name].py
|
||||||
|
|
||||||
|
### Implementation for User Story 3
|
||||||
|
|
||||||
|
- [ ] T026 [P] [US3] Create [Entity] model in src/models/[entity].py
|
||||||
|
- [ ] T027 [US3] Implement [Service] in src/services/[service].py
|
||||||
|
- [ ] T028 [US3] Implement [endpoint/feature] in src/[location]/[file].py
|
||||||
|
|
||||||
|
**Checkpoint**: All user stories should now be independently functional
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
[Add more user story phases as needed, following the same pattern]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase N: Polish & Cross-Cutting Concerns
|
||||||
|
|
||||||
|
**Purpose**: Improvements that affect multiple user stories
|
||||||
|
|
||||||
|
- [ ] TXXX [P] Documentation updates in docs/
|
||||||
|
- [ ] TXXX Code cleanup and refactoring
|
||||||
|
- [ ] TXXX Performance optimization across all stories
|
||||||
|
- [ ] TXXX [P] Additional unit tests (if requested) in tests/unit/
|
||||||
|
- [ ] TXXX Security hardening
|
||||||
|
- [ ] TXXX Run quickstart.md validation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependencies & Execution Order
|
||||||
|
|
||||||
|
### Phase Dependencies
|
||||||
|
|
||||||
|
- **Setup (Phase 1)**: No dependencies - can start immediately
|
||||||
|
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
|
||||||
|
- **User Stories (Phase 3+)**: All depend on Foundational phase completion
|
||||||
|
- User stories can then proceed in parallel (if staffed)
|
||||||
|
- Or sequentially in priority order (P1 → P2 → P3)
|
||||||
|
- **Polish (Final Phase)**: Depends on all desired user stories being complete
|
||||||
|
|
||||||
|
### User Story Dependencies
|
||||||
|
|
||||||
|
- **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
|
||||||
|
- **User Story 2 (P2)**: Can start after Foundational (Phase 2) - May integrate with US1 but should be independently testable
|
||||||
|
- **User Story 3 (P3)**: Can start after Foundational (Phase 2) - May integrate with US1/US2 but should be independently testable
|
||||||
|
|
||||||
|
### Within Each User Story
|
||||||
|
|
||||||
|
- Tests (if included) MUST be written and FAIL before implementation
|
||||||
|
- Models before services
|
||||||
|
- Services before endpoints
|
||||||
|
- Core implementation before integration
|
||||||
|
- Story complete before moving to next priority
|
||||||
|
|
||||||
|
### Parallel Opportunities
|
||||||
|
|
||||||
|
- All Setup tasks marked [P] can run in parallel
|
||||||
|
- All Foundational tasks marked [P] can run in parallel (within Phase 2)
|
||||||
|
- Once Foundational phase completes, all user stories can start in parallel (if team capacity allows)
|
||||||
|
- All tests for a user story marked [P] can run in parallel
|
||||||
|
- Models within a story marked [P] can run in parallel
|
||||||
|
- Different user stories can be worked on in parallel by different team members
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Parallel Example: User Story 1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Launch all tests for User Story 1 together (if tests requested):
|
||||||
|
Task: "Contract test for [endpoint] in tests/contract/test_[name].py"
|
||||||
|
Task: "Integration test for [user journey] in tests/integration/test_[name].py"
|
||||||
|
|
||||||
|
# Launch all models for User Story 1 together:
|
||||||
|
Task: "Create [Entity1] model in src/models/[entity1].py"
|
||||||
|
Task: "Create [Entity2] model in src/models/[entity2].py"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Strategy
|
||||||
|
|
||||||
|
### MVP First (User Story 1 Only)
|
||||||
|
|
||||||
|
1. Complete Phase 1: Setup
|
||||||
|
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
|
||||||
|
3. Complete Phase 3: User Story 1
|
||||||
|
4. **STOP and VALIDATE**: Test User Story 1 independently
|
||||||
|
5. Deploy/demo if ready
|
||||||
|
|
||||||
|
### Incremental Delivery
|
||||||
|
|
||||||
|
1. Complete Setup + Foundational → Foundation ready
|
||||||
|
2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
|
||||||
|
3. Add User Story 2 → Test independently → Deploy/Demo
|
||||||
|
4. Add User Story 3 → Test independently → Deploy/Demo
|
||||||
|
5. Each story adds value without breaking previous stories
|
||||||
|
|
||||||
|
### Parallel Team Strategy
|
||||||
|
|
||||||
|
With multiple developers:
|
||||||
|
|
||||||
|
1. Team completes Setup + Foundational together
|
||||||
|
2. Once Foundational is done:
|
||||||
|
- Developer A: User Story 1
|
||||||
|
- Developer B: User Story 2
|
||||||
|
- Developer C: User Story 3
|
||||||
|
3. Stories complete and integrate independently
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- [P] tasks = different files, no dependencies
|
||||||
|
- [Story] label maps task to specific user story for traceability
|
||||||
|
- Each user story should be independently completable and testable
|
||||||
|
- Verify tests fail before implementing
|
||||||
|
- Commit after each task or logical group
|
||||||
|
- Stop at any checkpoint to validate story independently
|
||||||
|
- Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
schema_version: "1.0"
|
||||||
|
workflow:
|
||||||
|
id: "speckit"
|
||||||
|
name: "Full SDD Cycle"
|
||||||
|
version: "1.0.0"
|
||||||
|
author: "GitHub"
|
||||||
|
description: "Runs specify → plan → tasks → implement with review gates"
|
||||||
|
|
||||||
|
requires:
|
||||||
|
speckit_version: ">=0.7.2"
|
||||||
|
integrations:
|
||||||
|
any: ["copilot", "claude", "gemini"]
|
||||||
|
|
||||||
|
inputs:
|
||||||
|
spec:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
prompt: "Describe what you want to build"
|
||||||
|
integration:
|
||||||
|
type: string
|
||||||
|
default: "copilot"
|
||||||
|
prompt: "Integration to use (e.g. claude, copilot, gemini)"
|
||||||
|
scope:
|
||||||
|
type: string
|
||||||
|
default: "full"
|
||||||
|
enum: ["full", "backend-only", "frontend-only"]
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- id: specify
|
||||||
|
command: speckit.specify
|
||||||
|
integration: "{{ inputs.integration }}"
|
||||||
|
input:
|
||||||
|
args: "{{ inputs.spec }}"
|
||||||
|
|
||||||
|
- id: review-spec
|
||||||
|
type: gate
|
||||||
|
message: "Review the generated spec before planning."
|
||||||
|
options: [approve, reject]
|
||||||
|
on_reject: abort
|
||||||
|
|
||||||
|
- id: plan
|
||||||
|
command: speckit.plan
|
||||||
|
integration: "{{ inputs.integration }}"
|
||||||
|
input:
|
||||||
|
args: "{{ inputs.spec }}"
|
||||||
|
|
||||||
|
- id: review-plan
|
||||||
|
type: gate
|
||||||
|
message: "Review the plan before generating tasks."
|
||||||
|
options: [approve, reject]
|
||||||
|
on_reject: abort
|
||||||
|
|
||||||
|
- id: tasks
|
||||||
|
command: speckit.tasks
|
||||||
|
integration: "{{ inputs.integration }}"
|
||||||
|
input:
|
||||||
|
args: "{{ inputs.spec }}"
|
||||||
|
|
||||||
|
- id: implement
|
||||||
|
command: speckit.implement
|
||||||
|
integration: "{{ inputs.integration }}"
|
||||||
|
input:
|
||||||
|
args: "{{ inputs.spec }}"
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"schema_version": "1.0",
|
||||||
|
"workflows": {
|
||||||
|
"speckit": {
|
||||||
|
"name": "Full SDD Cycle",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"description": "Runs specify \u2192 plan \u2192 tasks \u2192 implement with review gates",
|
||||||
|
"source": "bundled",
|
||||||
|
"installed_at": "2026-05-09T19:47:45.710028+00:00",
|
||||||
|
"updated_at": "2026-05-09T19:47:45.710035+00:00"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+118
@@ -0,0 +1,118 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||||
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **Guitar Pro → notation importer (`lib/gp2notation.py`)** (slopsmith#825 WS4b, epic #828). Piano/keys tracks imported from Guitar Pro (GPIF: `.gpx` GP6 / `.gp` GP7-8) now produce real Sloppak Notation Format data (sloppak-spec §5.3) alongside the `midi = string*24 + fret` guitar wire encoding. `gp2rs_gpx.convert_file` writes a `<stem>.notation.json` sidecar next to each keys arrangement XML (best-effort — a notation bug never breaks the RS-XML conversion), and `gp2notation.attach_notation_to_sloppak()` is the assembly-side helper that renames it into `notation_<id>.json` + adds the per-arrangement `notation:` manifest sub-key. Voice→staff routing salvages the logic from PR #703 (whose `stf` wire-field approach this supersedes): GP voice position 0 → `rh` staff (`G2`), positions ≥ 1 → `lh` (`F4`); a forced-LH track (the merged `Piano LH` partner from `_find_piano_pairs`, or a standalone track named `… LH`) routes everything to `lh` — preserving authored hand crossings instead of inferring hands from pitch. Emits measures with absolute `t` from the bar-indexed tempo map, change-only `ts`/`tempo`/`ks`, and `beat_groups` for compound/irregular meters (6/8 → `[3,3]`, 9/8 → `[3,3,3]`, 5/8 → `[2,3]`, 7/8 → `[2,2,3]` — cf. the slopsmith#261 denominator pitfalls); beats carry `dur`/`dot`/`tu`/`rest` from GP rhythms and notes carry absolute `midi` (String+Fret resolves via the string template's concert pitches, Tone+Octave via `(octave+1)*12 + step`) with `tied` continuations kept as real beats (engraving needs the tied notehead — unlike the RS-XML walk, which drops them and extends sustain). Timing reuses the `gp2rs_gpx` machinery (bar-indexed tempo map, per-beat rhythm durations, `_note_midi`) so notation lines up with the RS XML the highway plays — with one deliberate divergence: double dots advance time ×1.75 (vs. the RS-XML walk's single-dot ×1.5 approximation) so a written `dot: 2` agrees with the emitted beat times; sharing the walk itself stays tracked in slopsmith#618. Tests: `tests/test_gp2notation.py`.
|
||||||
|
- **Legacy keys → notation lifter (`scripts/lift_keys_notation.py`)** (slopsmith#825 WS4c, epic #828). One-time batch converter that lifts existing **directory-form** piano/keys sloppaks from the legacy guitar wire encoding (`midi = s*24 + f`) into real Sloppak Notation Format files (sloppak-spec §5.3). Candidates are arrangements whose name matches `\b(keys|piano|keyboard|synth)\b` (case-insensitive); each gets a `notation_<id>.json` plus the per-arrangement `notation:` manifest sub-key. Measures derive from the song-level `beats` downbeats (`measure >= 0`; `song_timeline.json` preferred, first-arrangement fallback), with per-measure tempo from downbeat spacing (emitted only on a > 1 BPM change). Durations come from the wire sustain (`sus`, legacy `l` alias) when present, else the gap to the next onset in the same hand — quantized to the nearest plain/single-dotted `{1,2,4,8,16,32}` denominator at the local tempo, floored at a 32nd. Hands are split heuristically: onsets within 10 ms form a group; a group spanning > 12 semitones splits at its largest internal interval gap (low side → `lh`), otherwise the whole group goes by mean pitch vs middle C — single-staff output when everything lands on one hand. Idempotent (arrangements already carrying `notation:` are skipped; an orphan `notation_<id>.json` without the manifest key is refused, not overwritten) with `--dry-run` support; every payload is checked via `notation.validate_notation` before write. Honest caveat: the manifest is round-tripped through PyYAML (`safe_load` + `safe_dump(sort_keys=False)`) — key order survives, YAML comments/custom formatting do not (the script warns when comments are present). Zip-form `.sloppak` files are reported and skipped. Tests: `tests/test_lift_keys_notation.py`.
|
||||||
|
- **Notation schema v1 freeze — completeness batch** (slopsmith#822, epic #828). Adds the low-hanging-fruit fields ahead of content production: top-level credits `rights`/`lyricist`/`arranger`; measure `pickup` (anacrusis); beat `arp` (arpeggiate), `ferm` (fermata), and **typed grace notes** — `grace: "a"` (acciaccatura, MusicXML `grace/@slash=yes`) / `"p"` (appoggiatura); note `stem` (`"up"`/`"down"` force). Pedal is settled as the existing `spd`/`sph`/`spu` trio with a documented MusicXML `<pedal start|change|stop>` mapping — no separate `ped` field. A new "v1 non-features" spec subsection pins the accepted limitations (microtonal, figured bass, mid-measure key/time/clef changes, `ott`/`barline`/ornaments/`trem`/`glis`) as additive-v1.x territory. `lib/notation.py` gains the `GRACE_TYPES`, `STEM_DIRECTIONS`, and `DYNAMICS` vocabularies; the validator stays permissive by design.
|
||||||
|
- **Notation format — standard musical notation as a first-class sloppak type.** Promotes keys, piano, violin, and any other staff-notation instrument out of the guitar wire format and into their own data structure, following the same promotion path used for drums (slopsmith#344). New `lib/notation.py` defines the canonical vocabulary (`CLEFS`, `DURATIONS`, `SCHEMA_VERSION`), a permissive `validate_notation()` check, and `measures_to_wire()` / `measure_to_wire()` wire helpers. `lib/sloppak.py::load_song` reads a new per-arrangement `notation:` sub-key from each arrangement entry in the manifest (Option B: per-arrangement, not song-wide), applies path-traversal guards, validates the parsed JSON via `validate_notation()`, and surfaces all notation payloads on `LoadedSloppak.notation_by_id` (a `dict[str, dict]` keyed by arrangement id). A failed or missing notation file for one arrangement does not abort or skip the arrangement itself — partial-failure isolation mirrors the drum tab loader. `file:` is now optional when `notation:` is present: the loader creates a minimal stub arrangement so a notation-only arrangement entry does not require a guitar wire format JSON. `/ws/highway/{filename}` gains two new message types — `notation_info` (staves, instrument, total measure count) and chunked `notation_measures` (32 measures per chunk) — streamed after `sections` and before `anchors`; `song_info` carries a new `has_notation: bool` flag so viz pickers can auto-activate the notation plugin regardless of arrangement name. The notation file schema is measure-structured (`measure → staff → voice → beat → note`), uses MIDI for pitch (no string/fret/tuning indirection), and carries the full set of effects that alphaTab can render. See `docs/sloppak-spec.md` §5.3 for the full schema. Open questions resolved per the piano/keys epic (slopsmith#828 / #822): Option B (per-arrangement `notation:` sub-key) and `file:`-optional-when-`notation:`-present are the endorsed design.
|
||||||
|
- **`song_timeline.json` — beats and sections as a top-level file.** A new optional top-level file pointed at by a new manifest key (`song_timeline: song_timeline.json`) provides the correct home for song-wide beats and sections, replacing the legacy convention of embedding them in the first arrangement JSON. The loader in `lib/sloppak.py` reads and validates the file (must be a dict with `beats` and `sections` as lists), clears and repopulates `Song.beats` / `Song.sections` from it when present, and stores the raw dict on `LoadedSloppak.song_timeline`. The existing arrangement-JSON fallback is fully preserved: all existing sloppaks that omit `song_timeline:` continue to load without any change. This is a prerequisite for notation-only sloppaks, which may have no arrangement JSON at all and therefore no carrier for beats/sections data. New sloppaks should put beats/sections in `song_timeline.json` only. See `docs/sloppak-spec.md` §2 and §5.3.
|
||||||
|
- **`note-detection` capability domain promoted — control plane (spec 009)** (slopsmith#727/#728, epic #828). New core host `static/capabilities/note-detection.js`: provider registry (kinds `midi`/`engine`/`js`, primitives `pitch.estimate`/`verify.target`), requester-owned context-scoped detection bindings (`open-binding`/`close-binding`/`set-target`/`clear-target` — each binding carries its own redacted tuning context, independent of the host's loaded song, per spec-009 FR-003), and hit/miss/verdict observability events (consumers own judgment). The legacy chart-coupled `highway.setNoteStateProvider` surface keeps working and is wrapped for compatibility-shim hit accounting. Diagnostics (`slopsmith.note_detection_capability.v1`) carry provider/binding summaries and bounded outcomes — no raw audio, device labels, or song identity. Migrating the chart path, Step Mode verify, minigames YIN, and the engine verifier onto bindings is the remainder of the spec-009 slice.
|
||||||
|
- **`visualization` capability domain promoted (cap:6)** (slopsmith#828). New core host `static/capabilities/visualization.js` registers a provider-coordinator owning the highway renderer surface: commands `inspect` / `list-providers` / `select-renderer` / `clear-renderer` (selection delegates to the existing picker so persistence, WebGL2 gating, and fallback stay single-sourced), events `providers-refreshed` / `renderer-changed` / `renderer-ready` / `renderer-failed`. Legacy discovery (`type: "visualization"` manifests, `window.slopsmithViz_*` globals) keeps working unchanged and is accounted as compatibility shims with hit counts. `static/app.js` attributes every renderer change (auto-match / user-select / fallback) and auto-match outcomes into the domain. Diagnostics (`slopsmith.visualization_capability.v1`) carry provider ids/labels/context types, active renderer + selection source, last auto-match outcome, and last failure — no song filenames/titles. Per-panel (splitscreen) selection is a tracked follow-up.
|
||||||
|
- **Viz picker routes notation arrangements** (slopsmith#826, epic #828). `window.slopsmith.currentSong` gains `hasNotation` (sibling of `hasDrumTab`) from the `song_info` frame's `has_notation` flag, so notation viz plugins (Staff View, Keys Highway 3D) can gate `matchesArrangement` on data presence instead of arrangement-name heuristics. When a notation-only arrangement (no wire notes — `file:` omitted per sloppak-spec §5.3) falls through Auto with no notation plugin installed, the built-in highway still takes the canvas but the Auto label reads "no notation view installed" and a one-shot dismissable hint points at the visualization picker — never a silently blank board.
|
||||||
|
- **Keys instrument path in progression** (slopsmith#828). New `data/progression/paths/keys.json` (5 levels / 15 challenges at parity with the guitar path) plus keys-flavoured daily/weekly quest pool entries (`d.keys-one` "Ivory Tower", `w.keys-three` "Grand Recital"). `lib/progression.py::instrument_for_arrangement()` now attributes `type: piano|keys` arrangements — and names matching `keys`/`piano`/`keyboard`/`synth` on a word boundary — to the new `keys` instrument, so scored keys runs advance the path automatically. Purely content + attribution: no schema or API changes.
|
||||||
|
- **v3 library: exact artist/album filters + scroll/page-depth restore** (slopsmith#857). The v3 Songs toolbar gains Artist and Album dropdowns (Album populates from the selected artist and stays disabled until one is chosen), backed by new exact, case-insensitive (`COLLATE NOCASE`) `artist` / `album` query params threaded through `MetadataDB._build_where` → `query_page` / `query_artists` / `query_stats` and the `/api/library`, `/api/library/artists`, `/api/library/stats` endpoints (the free-text `q` search stays fuzzy and composes with the exact filters). The artist/album catalog is fetched independently of the active artist/album selection so the dropdowns always list the full set for the current provider/search. The toolbar is now sticky so filter controls stay reachable when browsing deep libraries, and returning from the player restores the previous scroll position **and** the loaded infinite-scroll page depth via a `sessionStorage` snapshot keyed by a filter/sort/view state hash (invalidated whenever those change, so a filter change still resets to the top). Tests: `tests/test_library_filters.py` (backend artist/album filters), `tests/js/v3_songs_scroll.test.js` (state-hash + snapshot helpers).
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **v3 library scroll-restore no longer breaks the classic v2 UI or drops off-screen searches** (slopsmith#857). Two regressions in the scroll-restore work above: (1) `playSong` remapped `home`-launched songs to return to the `#v3-songs` screen unconditionally, but `static/app.js` is shared with the v2 UI (served at `/v2` / `SLOPSMITH_UI=v2`) where that screen does not exist — Esc-from-player then called `showScreen('v3-songs')`, which threw on the missing element and stranded the user on a blank screen with playback still running; the remap now applies only when `#v3-songs` is present. (2) The Songs screen-entry fast-path skips reloading to preserve scroll, but the global topbar search routed through it, so once Songs had been visited, searching from another screen navigated there without applying the new query; the screen now tracks the state hash each fetch reflects and refetches when it has drifted, keeping the scroll-preserving no-op only when nothing changed.
|
||||||
|
- **An active custom highway renderer is no longer starved of `draw()` when it hides the canvas** (#819). The per-frame draw gate in `static/highway.js` bailed on `if (!_lastVisible) return`, which conflated two different "hidden" states: a genuine off-screen canvas (`offsetParent === null` — navigate-away / `display:none` splitscreen panel, #246) versus a renderer-set *override-hide* (`setVisible(false)`, where an opaque overlay covers the canvas but the active renderer keeps painting its own surface). The gate now only pauses everything for the off-screen case (and still pauses the default 2D renderer on an override-hide); the **active custom renderer** keeps receiving `draw()` through its own override-hide. The `highway:visibility` event still fires before the gate, so sibling overlay renderers (e.g. 3D Highway's `.h3d-wrap`) still pause. This is the core-side root cause behind the Tab View cursor freezing in single-player (slopsmith#734; worked around plugin-side in slopsmith-plugin-tabview#25).
|
||||||
|
- **Screensaver no longer kicks in during windowed-mode playback** (#686). While a song is playing, `static/app.js` now holds a [Screen Wake Lock](https://developer.mozilla.org/en-US/docs/Web/API/Screen_Wake_Lock_API) (`navigator.wakeLock.request('screen')`) so the OS display/screensaver stays awake even though only audio + the highway animation are active and the keyboard/mouse are idle. The lock is acquired on `song:play`/`song:resume` and released on `song:pause`/`song:ended`/`song:stop` (kept only while actually playing), and re-acquired on `visibilitychange` when the tab refocuses (the API auto-releases a lock whenever the page is hidden). Both the HTML5 `<audio>` and JUCE desktop playback paths emit the same `song:*` events, so the fix covers both. In slopsmith-desktop (Electron), where `navigator.wakeLock` is unreliable, it also drives a native `powerSaveBlocker` bridge via the optional `window.slopsmithDesktop.power.setScreenAwake` hook when present; both calls degrade silently where unsupported. Note: the browser Wake Lock API is secure-context only, so in a plain browser this is active on `localhost` / HTTPS only — a session opened over plain HTTP to a LAN IP (e.g. `http://192.168.1.100:8000`) won't keep the screen awake; front it with HTTPS or use the desktop app (see README → reverse-proxy notes).
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **3D highway: realistic curved metal frets.** The fret wires are now bowed `TubeGeometry` (the middle strings push away from the camera so the row of frets reads as wrapping a cylindrical neck — a depth cue) rendered with a lit `MeshStandardMaterial` instead of the old flat, straight `MeshBasicMaterial` boxes, so the scene's ambient + directional light glints across the rounded surface for a polished-steel look. The existing per-frame highlight is preserved unchanged: frets inside the active anchor still turn gold (`0xD8A636`), which under the metallic shading reads as brass. Metalness is kept moderate (0.4, not full-metal) because the scene has no envMap — a PBR full-metal surface would reflect black — with a dim emissive floor so frets stay legible down the fogged neck. Backported from the `highway_babylon` plugin's "hit-zone fret bars". All knobs (`FRET_BOW_DZ`, metalness/roughness/emissive) are tunable constants. `plugins/highway_3d` v3.25.0.
|
||||||
|
- **3D highway: section + tone HUD cards now default OFF.** The v0.3.0 player chrome carries a persistent "Up Next" pill, making the in-canvas section card redundant by default (it doubled the readout, slopsmith feedback); the tone HUD follows the same less-is-more default. Both remain available in Settings → 3D Highway (visibility/position/size unchanged); users who previously toggled either explicitly keep their stored preference — only the untouched default flips. `plugins/highway_3d` v3.24.1.
|
||||||
|
- **Perf**: replace runtime Tailwind Play CDN with a prebuilt static stylesheet (`static/tailwind.min.css`). The Play CDN's runtime JIT scanned the DOM ~1.8x/sec on the main thread (~37 ms blocking spans), dropping ~26% of frames in long playback sessions with the 3D highway as default. Theme extensions (dark/accent/gold colors, Inter font) move to `tailwind.config.js`; regen via `bash scripts/build-tailwind.sh`. No runtime build step — the generated CSS is committed. Fixes slopsmith-desktop#110.
|
||||||
|
- **Perf**: reduce per-frame allocations in the 2D highway chord + lyric render paths. `_ensureChordRenderCache` now also caches `sortedNotes` / `nonZeroNotes` / `nonZeroFrets` / `allMuted` / `hasMultipleNotes` (computed once per chord, invalidated on `src` / `_inverted` / `chordTemplates` change — the third key catches a stale `isOpen`-derived classification when the WS `chord_templates` message lands after the final `chords` chunk), so `drawChords` no longer re-sorts / re-filters / spreads min-max per visible chord per frame. The in-chord unison bend classification is folded inline (no `chordPositions.filter` × 2 per frame). `drawLyrics` memoizes `ctx.measureText` results in a two-level `Map<fontSize, Map<text, width>>` so cache hits don't allocate a composite string key. Lit-sustain shimmer in `drawSustains` swaps the 4 per-note-per-frame `Math.random()` calls for a 64-entry precomputed jitter LUT (xorshift32-seeded — the LUT contents are reload-stable and test-reproducible; rendered shimmer is deterministic per `createHighway()` instance, since the seed includes that instance's `_frameIdx`) indexed by `(frameIdx + n.s + ⌊n.t·60⌋)`, visually indistinguishable and allocation-free.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **Player progression: Mastery Rank, instrument-path challenges, daily/weekly quests, Decibels currency, cosmetics shop (spec 010).** Onboarding gains two steps: pick one or more **instrument paths** (Guitar / Bass / Drums — data-driven, more can ship as content) and a **calibration challenge** offer (play the bundled Slopsmith Diagnostic with note detection at 100% accuracy — or skip; either way you reach **Mastery Rank 1**, and a skipped calibration can still be completed later from the Progress screen). Each path levels by completing a content-defined number of **challenges** (any order) from that level's set; Mastery Rank = onboarding rank + the sum of path levels, starting at 0 on a fresh install. The existing unified XP backend is untouched but the frontend renames it to **Decibels (dB)** — a spendable currency earned ONLY by playing (songs, FeedBarcade rounds, quest rewards; no real-money path exists or may be added) — with spend tracked in a separate wallet so lifetime earnings stay monotonic. Rotating **daily/weekly quests** (deterministic per period, lazy instantiation, local-midnight / Monday resets) award dB and feed `quest_completed` challenges. A new **Progress** screen (rank hero, per-path checklists, quest countdowns, add-a-path) and **Shop** screen (themes via CSS-variable swaps under `html[data-fb-theme]`, avatar frames; atomic balance-checked purchases — 402 on insufficient dB, 409 on re-buy) join the v3 nav, and the topbar badge now shows Rank + challenge-set progress + dB balance. All definitions live in `data/progression/` JSON (paths/levels/challenges, quest pools, shop catalog) — adding a rank, challenge, quest, or cosmetic is a content edit + restart, never code; invalid content degrades to logged warnings. New tables (additive + idempotent): `progression_state`, `player_paths`, `challenge_progress`, `quest_state`, `wallet`, `shop_owned`, `shop_equipped`. New endpoints: `GET /api/progression`, `POST /api/progression/paths|onboarding|events` (events whitelists `minigame_run`; `song_completed` stays server-derived inside `POST /api/stats`, which now resolves the instrument server-side and reports an additive `progression` outcome key), `GET /api/shop`, `POST /api/shop/buy|equip`; equipped cosmetics ride along on `GET /api/profile`. A new **`progression` capability domain** (core-owned, kind: command, safety: safe — `inspect`, `record-event`, `list-shop`, `buy-item`/`equip-item` gated on user action) emits `challenge-completed`/`quest-completed`/`path-level-up`/`rank-changed`/`db-changed`/`calibration-completed`/`cosmetic-equipped`, mirrored as `progression:*` window events, with a redaction-safe diagnostics contributor; backend plugins get the symmetric `record_progression_event` context hook (the bundled minigames hub reports runs through it, guarded for standalone). Spec: `specs/010-progression-domain/`. Tests: `tests/test_progression.py`, `tests/test_progression_api.py`. **Migration notes:** existing XP totals carry over as lifetime dB (balance = lifetime − spent); resetting a per-source XP contribution (e.g. a minigames profile reset) after spending can clamp the spendable balance to 0 until new dB is earned; drums-path v1 content uses currently-satisfiable goals (arcade rounds, quests, any-instrument plays) until drums scoring lands.
|
||||||
|
- **3D highway score FX (notedetect game-scoring layer).** The bundled `highway_3d` renderer now visualizes the scoring layer shipped in slopsmith-plugin-notedetect ≥1.13: floating **"+N" score pops** above each judged gem (sourced from the note-state provider's new `{ points, mult, popKey }` verdict fields — chord members share the chord-level `popKey`, so a chord pops once, not once per string), plus session-level FX from the new `notedetect:fx` event — a particle burst at the strike line on streak milestones (25/50/every 100), an expanding ring pulse on multiplier tier-ups (×2/×3/×4), and a brief red wash when a ≥10 streak breaks. Colors and the pop font follow the user's notedetect scoring-UI skin (`slopsmith_notedetect_skin`: neon/esports/metal, refreshed live on the `notedetect:skin` bus event). Everything renders on the existing 2D overlay canvas from fixed-size slot pools — no Three.js geometry, no text-sprite cache traffic, near-zero cost when idle — and degrades to a silent no-op with older notedetect builds (the new fields/events simply never arrive). Splitscreen panels scope FX to their own detector instance via the bubbling per-panel `notedetect:fx` dispatch. `plugins/highway_3d` v3.24.1.
|
||||||
|
- **3D highway: slide direction arrows + gem-follow animation.** Slide notes now show a ›/‹ arrow indicating which way the slide goes — on the note/gem itself, as an early preview on the neck before the note arrives, and (optionally) chained further ahead for multi-leg slides — each independently toggleable in Settings → 3D Highway (`slideArrowApproachVisible`, `slideArrowNeckVisible`, `slideArrowChainPreviewVisible`). The note gem also now visually glides from its starting fret to the slide's destination over the note's sustain and holds there through the brief post-sustain linger, instead of snapping back to the starting fret — most noticeable on unpitched "slide to nothing" notes. `plugins/highway_3d` v3.25.2.
|
||||||
|
- **3D highway: up to 3 upcoming-note ghost previews per string, with fade-in/grow.** Each string now previews up to 3 upcoming notes (was 1) on a fixed 0.6 s fade-in/grow ramp, so tight same-string runs no longer pop in at full size right before impact and the player can read note order ahead of time. `isBlocked` (the pre-impact ghost suppression in a note's last 150 ms) is now scoped to chord notes only — for lead notes it had been blinking the ghost out right before each sustained note in dense runs. (Slide notes stay excluded too, per the slide-arrow work above, since their gem glides off the start fret.) `plugins/highway_3d` v3.26.0.
|
||||||
|
- **Enable/disable plugins from the v3 Pedalboard (footswitch backend).** Every `/api/plugins` entry now carries an `enabled` boolean (default `true`), and a new `POST /api/plugins/{plugin_id}/enabled` endpoint (`{"enabled": <bool>}` → `{"id", "enabled"}`) persists the choice to `CONFIG_DIR/plugin_state.json` (only non-default `enabled:false` entries are stored; a missing/corrupt file is tolerated and never crashes startup). The loader **skips disabled plugins at startup** — no requirements install, no `routes.setup()`, no screen/nav/capabilities — while still surfacing them in `/api/plugins` as a disabled entry (`status:"disabled"`, `enabled:false`) so the UI can show an "off" pedal you can switch back on. Toggling persists immediately and flips the in-memory flag so the next `/api/plugins` reflects it at once (a runtime-disabled plugin's already-mounted routes/screen remain until the next restart; re-enabling a startup-skipped plugin mounts on restart). A disabled plugin is **excluded from the capability pipeline** — its capability metadata is emptied in `/api/plugins`. Guard rails keep `capability_inspector` and `app_tour_*` always enabled (disable → `400`); unknown id → `404`; missing/non-boolean `enabled` → `400`. Backend only; the v3 Pedalboard frontend consumes this contract. Docs: `docs/plugin-v3-ui.md`.
|
||||||
|
- **fee[dB]ack v0.3.0 rebrand + UI redesign (opt-in, isolated).** The visible product is being renamed from **Slopsmith** to **fee[dB]ack** (the `[dB]` is a decibel pun on the practice "feedback" loop) alongside a full dashboard-style UI redesign. **Rebrand scope is the app + docs wordmark only** — the repository, Python package, ghcr Docker image, `CONFIG_DIR`, and `SLOPSMITH_*` env vars all keep the `slopsmith` name, so existing deployments and data are unaffected. The redesigned UI is additive and served behind a feature flag: `SLOPSMITH_UI=v3` flips the `/` route to the new `static/v3/` shell, and `GET /v3` always serves it; the default `/` stays byte-identical to 0.2.9 until 0.3.0 flips the default. This release adds the `static/v3/` scaffold (navy app shell + styled fee[dB]ack wordmark, brand SVG + favicon + PWA manifest with 192/512 icons), an additive `fb` Tailwind color palette (legacy `dark`/`accent`/`gold` retained) with `static/v3/**` in the content globs, and the regenerated `static/tailwind.min.css`. Vanilla JS, prebuilt Tailwind, no Play CDN (Principle II). Shell wiring, screens, profile/scoring backends, and capability-runtime integration land in subsequent v0.3.0 changes.
|
||||||
|
- **fee[dB]ack v0.3.0 app shell (sidebar + topbar + routing).** The v3 shell (`static/v3/index.html`) is now a re-chromed copy of the legacy app: the new left **sidebar** (HOME / LIBRARY groups) and **topbar** (secondary nav, search, Support, badge-cluster mount points) replace the hidden legacy navbar, and new `#v3-*` screens (dashboard/plugins/profile/playlists/saved) are added — while all legacy screens (`#home` library, `#favorites`, `#settings`, `#player`, `#audio`, plugin nav containers) are kept verbatim so `static/app.js` boots **unmodified** and the whole engine (player/highway, plugin loader, capabilities, audio, library, settings) is reused as-is. Navigation is the shared `window.showScreen` across `#v3-*`, reused legacy, and `#plugin-*` screens, with a responsive hamburger and a `localStorage`/`v3:`-namespaced shell. Plugin nav is mirrored into the sidebar from `/api/plugins` (UI placement is a deferred capability domain, so this uses the legacy loader, not capability dispatch). `static/v3/shell.js` wraps `window.showScreen` via the idempotent rehydration pattern to keep sidebar/topbar active-state in sync.
|
||||||
|
- **fee[dB]ack v0.3.0 player profile + first-run onboarding + unified XP + streak.** Adds a single-user core **profile** (`profile`/`profile_progress`/`xp_profile` tables in `web_library.db`, additive + idempotent): display name + avatar, a stable `player_hash` (SHA-256 of the first name + a once-generated salt — stable across later renames; a future-leaderboard label, never auth), and a **streak** (any session on a calendar day keeps it; a missed day resets to 1). New endpoints: `GET/POST /api/profile`, `POST /api/profile/avatar` (base64, re-encoded to a ≤512px PNG under `CONFIG_DIR/avatars/`), `GET /api/profile/avatar/{name}` (safe-joined), `GET /api/profile/avatars` (bundled defaults under `static/v3/avatars/`), `GET /api/profile/progress` (one call for the badge), and `POST /api/xp/award`. **Unified XP:** `lib/xp.py` is the single XP curve (same math the minigames plugin shipped); the core `xp_profile` store is the one source of truth the profile badge reads, exposed to plugins via `context["award_xp"]`/`get_xp_progress`/`seed_xp`. The bundled **minigames** plugin now delegates XP to the core store (seeding once from its existing `profile.json` so earned levels carry over) — so song-play, minigames, and tutorials all feed one level. Frontend: a blocking first-run onboarding overlay (name + avatar grid + upload), the topbar profile badge (avatar, 🔥 streak, level + XP bar), and the `#v3-profile` screen. Tests: `tests/test_xp.py`, `tests/test_profile_api.py`.
|
||||||
|
- **fee[dB]ack v0.3.0 song-stats store (best score + accuracy, plays, resume position).** A core `song_stats` table (`web_library.db`, additive + idempotent, PK `(filename, arrangement)`) records per-song/arrangement best/last score + accuracy, play count, and last position. Endpoints: `POST /api/stats` (scored session → `plays += 1`, `best_*` = max, `last_*` = new, plus **unified-XP award** `xp_for_run(score)` and a **streak** bump, both behind try/except so a side-effect failure never drops the stat write; or position-only `lastPlayPosition` → resume touch with no `plays` change), `GET /api/stats/{filename}` (aggregated across arrangements), `GET /api/stats/recent` (joined to song title/artist/art for "Jump back in"). Scoring stays frontend-driven: `static/v3/stats-recorder.js` tallies the `note:hit`/`note:miss` events the optional `slopsmith-plugin-notedetect` already emits (and also accepts an explicit `note_detect:session-ended` summary), then POSTs on song end; it also persists resume position on pause/stop. No note-detect edit required — note-detection is a deferred capability domain, so the recorder uses those legacy events and degrades to "no accuracy" when the plugin isn't installed. Score/accuracy math is shared with the server via `lib/song_score.py`. Tests: `tests/test_song_score.py`, `tests/test_song_stats_api.py`.
|
||||||
|
- **fee[dB]ack v0.3.0 playlists, Saved for Later, and Continue-Playing.** Core playlist management (`playlists` + `playlist_songs` tables in `web_library.db`, additive + idempotent): create/rename/delete, add/remove/reorder songs, plus a reserved **Saved for Later** system playlist (created on first use; protected from rename/delete). Endpoints: `GET/POST /api/playlists`, `GET/PATCH/DELETE /api/playlists/{id}`, `POST /api/playlists/{id}/songs`, `DELETE /api/playlists/{id}/songs/{filename}`, `POST /api/playlists/{id}/reorder`, `POST /api/saved/toggle`, and `GET /api/session/continue` (derives the resume song + last position from `song_stats`, no new table). Frontend `static/v3/playlists.js` renders the `#v3-playlists` list + detail (drag-reorder, play, remove) and `#v3-saved`, and exposes `window.v3Saved.toggle()` for a "Save for later" affordance on song cards. Favorites reuse the existing favorites screen/API. Core REST, no capability domain. Tests: `tests/test_playlists_api.py`.
|
||||||
|
- **fee[dB]ack v0.3.0 Dashboard / Home.** The `#v3-home` dashboard (matching the v0.3.0 design target) composes the new backends: a "Welcome back, {name}!" banner with a patch-notes link (`/api/version`), a hero card (Start Playing / Create Lobby), a **Continue-Playing** card (`/api/session/continue` → art, tuning chip, 4-segment progress; click resumes via `playSong` + best-effort seek), a stats row (audio-routing widget placeholder until prompt 18, library count from `/api/library/stats`, plugins count from `/api/plugins` where `status==="ready"`), and a **Recently Played** grid (`/api/stats/recent`) with per-song accuracy badges (good/mid/low ramp). Each widget fetches + renders independently and degrades gracefully (missing/empty endpoint → placeholder, never blocks first paint). `static/v3/dashboard.js`; re-renders on return to Home and on profile update.
|
||||||
|
- **fee[dB]ack v0.3.0 tuner + instrument topbar badges.** The topbar gains an **instrument selector** (guitar/bass + string count + tuning + reference pitch) persisted via additive `/api/settings` fields (`reference_pitch` clamped 430–450, `instrument`, `string_count` 4–8, `tuning` name or semitone offsets); changing it emits `instrument:changed` so the note_detect scorer can re-tune (consumed once the external plugin adopts it). A **live tuner badge** stays idle until the user enables the mic (explicit gesture; `getUserMedia`), then shows a YIN note readout with a cents needle (green within ±5¢) using a new dependency-free `static/v3/tuner-core.js` (YIN + frequency→note/cents, honoring the reference pitch); clicking opens the full `slopsmith-plugin-tuner` screen when installed. CPU-friendly (~20 Hz, paused when the tab is hidden, respects `prefers-reduced-motion`). Tests: `tests/test_settings_instrument.py`, `tests/js/tuner_core.test.js`.
|
||||||
|
- **fee[dB]ack v0.3.0 audio-routing widget (dashboard).** The dashboard's audio stat tile now reads the live audio session **through the capability runtime** — `audio-mix inspect` (route + faders + required kinds), `audio-input list-sources` (selected/available input), `audio-monitoring inspect` — and renders **Audio Input → VST/NAM/IR → Audio Output** with per-node state dots and a Connected/Not Connected line. It never touches `audio-mixer.js` internals or `nam_tone` routes directly; "Not Connected" is the honest browser default (no native route), and it degrades on `no-owner`/`no-handler`/`failed` or absent capabilities. Refreshes on `instrument:changed`, play/stop, capability audio events, and each Home visit. `static/v3/audio-routing.js`.
|
||||||
|
- **fee[dB]ack v0.3.0 Plugins page.** The `#v3-plugins` screen renders the enriched `/api/plugins`: a "{N} active" header (`status==="ready"`), a card grid per plugin (icon, name, version, status pill with the error on failed, capability summary badges — declared domains / validation warnings / unsupported versions / shim hits / bundled / type), an **Open →** action that navigates to the plugin's injected `#plugin-<id>` screen, and All/Bundled/Visualizations filters. Surfaces a deep-link to the bundled **Capability Inspector** rather than re-implementing the graph. No new backend. `static/v3/plugins-page.js`.
|
||||||
|
- **fee[dB]ack v0.3.0 Songs / Library screen (`#v3-songs`).** A native vanilla-JS library browser over the existing `/api/library*` endpoints: provider selector (via the `library` capability, not DOM scraping), grid + tree views, sort, format filter, a tri-state filter drawer (arrangements / stems / lyrics / tunings), topbar-driven search (debounced), infinite scroll, fb song cards with **accuracy badges** (good/mid/low ramp, batched via a new `GET /api/stats/best`), favorite + save-for-later affordances, and upload (reuses the existing uploader). The "Songs" sidebar nav now opens this screen. No regression to `/api/library*`. `static/v3/songs.js`.
|
||||||
|
- **`ui.library-card-injection` capability + native song-card actions (fee[dB]ack v0.3.0).** New core capability (`static/capabilities/library-card-actions.js`, owner `core.ui.library-card-injection`, exposed as `window.slopsmith.libraryCardActions`) lets plugins **register** per-song library-card actions (id, label, placement, applicability, enabled state, run handler) with `action-registered`/`action-result` events — replacing the legacy `.song-card` DOM-injection pattern (roadmap domain #9, now delivered as a frontend host). The native Songs grid renders registered actions in each card's "⋮" menu; the built-in **Edit metadata** and **Convert to E Standard (retune)** actions ship through it (`static/v3/card-actions-core.js`, calling the existing `openEditModal`/`retuneSong` globals). Songs cards also gain **arrangement chips** (play a specific arrangement) and a **multi-select** mode with batch **Add to playlist** / **Save for Later**. Recipe in `docs/capability-recipes.md`; tests in `tests/js/library_card_actions.test.js`. Migrating the external card-action plugins (Sloppak Converter, Find More, editor) onto `register(...)` is a follow-up.
|
||||||
|
- **`centOffset` exposed via `getSongInfo()`** — the arrangement `<centOffset>` field (float, cents) is now parsed from all chart sources (loose folder XML, sloppak wire format) and sent as `centOffset` in the `song_info` WebSocket message. Plugins can read `getSongInfo().centOffset` to obtain the arrangement's pitch-shift offset — commonly `-1200.0` for extended-range bass (one octave down) or a small non-zero value for true-tuned content (e.g. A443 ≈ +11.8 cents). Defaults to `0.0` when absent.
|
||||||
|
- **`highway.getPhrases()` and `highway.getMastery()` public plugin API** — exposes phrase timing windows (`[{ index, start_time, end_time, max_difficulty }]`) and the current mastery slider value (`0..1`) as documented, stable plugin API. Both values were already in memory and reachable via internal names; this surfaces them with intent so plugins can implement section-aware logic (e.g. tracking accuracy per phrase, suppressing difficulty changes during a hard solo) without reaching into undocumented internals. Returns `null` when the song has no phrase data (GP imports, single-difficulty charts). Pair with the existing `hasPhraseData()` to gate phrase-aware code paths.
|
||||||
|
- **Tailwind freshness guard + wider plugin scan.** A new `tailwind-fresh` CI job (`.github/workflows/tests.yml`) rebuilds `static/tailwind.min.css` with the pinned `tailwindcss@3.4.19` and hard-fails on any diff, so the committed prebuilt stylesheet can no longer silently lag source (after PR #411 removed the runtime Play CDN, a stale file shipped unstyled elements with no guard). The `tailwind.config.js` plugin content glob is widened to `./plugins/**/*.{js,html}`, which also scans non-`screen.js` plugin JS (e.g. `plugins/app_tour_*/script.js`) that was previously invisible to the build. Regenerating under the wider glob is a no-op for runtime behaviour — it only adds classes that were already used in source. Groundwork for the plugin `styles` capability (constitution 1.1.0, Principle II): runtime-installed plugins ship their own compiled CSS rather than relying on core's build-time scan.
|
||||||
|
- **Plugin capability pipelines** — adds the first versioned capability coordination layer for plugin authors and support tooling. `/api/plugins` now exposes validated capability declarations, validation warnings, unsupported-version metadata, UI/runtime domain declarations, and compatibility shim summaries for legacy `nav` / `screen` / `settings` / `routes` / visualization surfaces. The browser runtime now tracks manifest participants separately from live handlers, explicit dispatch outcomes (`no-owner`, `no-handler`, `unsupported-command`, `incompatible-version`), claim lifecycle cleanup, manual override precedence, deterministic ownership conflicts, multi-provider ordering, shim hit counts, and a redaction-safe diagnostics snapshot capped at 64 KB. A bundled Capability Inspector plugin shows the live graph, and new docs cover the manifest schema, recipes, safety matrix, lifecycle cleanup, and diagnostics contract.
|
||||||
|
- **Audio graph/session capability slice** — promotes `audio-mix`, `audio-input`, `audio-monitoring`, and coordinated `stems` diagnostics into the capability runtime. The new audio session host records song route/fader state, redaction-safe input sources, monitoring lifecycle outcomes, stem automation claims/overrides/orphans, and compatibility bridge hits for legacy faders, song volume, Stems master volume, 3D Highway analyser taps, audio startup barriers, and input source handoffs. `core.audio.session` coordinates `stems` without replacing the Stems plugin as the owner of actual stem playback/state.
|
||||||
|
- **Audio-mix control plane** — makes `audio-mix` the player mixer source of truth. Core now exposes `list-faders`, `get-fader-value`, `set-fader-value`, `inspect-route`, and `inspect-analyser` through the capability runtime, routes native and compatibility-backed fader provider operations with a 2-second timeout, reports committed values back to the mixer UI, suppresses matching legacy faders when a native participant owns the same logical control, and expands audio-session diagnostics/Capability Inspector rendering for fader availability, source modes, bridge hits, route/analyser summaries, and timeout failures.
|
||||||
|
- **Audio-input control plane** — makes `audio-input` the redaction-safe source of truth for instrument input discovery and lifecycle. Core now exposes `list-sources`, `select-source`, `open-source`, and `close-source` through the capability runtime, persists selected logical sources, keeps inspect/list/select prompt-free, routes provider `source.open`/`source.close` operations with bounded outcomes, shares compatible open sessions across requesters, suppresses compatibility-backed duplicate sources when a native provider owns the same logical key, and expands audio-session diagnostics/Capability Inspector rendering for selected input, open sessions, bridge hits, storage status, and permission/device failures without exposing raw device labels or live audio handles.
|
||||||
|
- **Audio-monitoring control plane** — makes `audio-monitoring` the shared live-monitoring coordinator. Core now exposes provider registration/list/selection, explicit user-action `start`, requester-counted `stop`, prompt-free `inspect`/`monitoring.status`, and `set-direct-monitor` through the capability runtime. Monitoring starts integrate with selected `audio-input` readiness, background requesters can only attach to active compatible sessions, active sessions survive song/playback stops without auto-resuming after reload, native providers suppress compatibility-backed legacy monitor paths, and diagnostics/Capability Inspector now show providers, sessions, requesters, direct-monitor state, bridge hits, and distinct safe outcomes (`provider-selection-required`, `user-action-required`, `incompatible`, `unavailable`, `stopped`, etc.) without exposing raw audio/device data.
|
||||||
|
- **Playback control plane** — promotes `playback` to an active core capability domain for song transport, timing, loop, route, requester/observer, bridge, and diagnostics state. Core now exposes `inspect`, user-authorized `start`, `pause`, `resume`, `stop`, `seek`, `set-loop`, and `clear-loop` through the capability runtime while `static/app.js` keeps raw `<audio>`/JUCE handles private behind a redaction-safe adapter. Playback diagnostics use pseudonymous targets in exported bundles, local display labels only in the Capability Inspector, bounded recent outcomes/events, and bridge accounting for `window.playSong`, legacy `song:*` events, `window.slopsmith` transport helpers, loop helpers, and browser/native route handoff.
|
||||||
|
- **3D highway — Tone HUD, fret dividers, chord-diagram toggle, FPS counter.** The bundled `plugins/highway_3d` gains an amber **Tone-change HUD** (shows the active tone and the next scheduled tone change; position / size / visibility configurable in settings), a **fret-dividers** toggle (vertical dividers on the highway, on by default, via `h3dBgSetFretDividersVisible`), a **chord-diagram visibility** toggle (`h3dBgSetChordDiagramVisible`), and an **FPS counter** setting migrated to `BG_DEFAULTS.fpsVisible` (drops the legacy `h3d_showFps` localStorage key). Chord-diagram position is restricted to `tl`/`tr`; legacy `bl`/`br` values are coerced on load. Perf: accent-halo shell descriptors are pre-built per string in `initScene()` and the chord-verdict cache key is encoded as a number, eliminating per-frame allocations in the `drawNote()` and chord hot paths.
|
||||||
|
- **Sloppak assembly preserves a short preview clip.** When a source chart carries a separate short browser-preview audio clip alongside the full song, the sloppak assembler now decodes it to `preview.ogg` at the sloppak root and records it under a new top-level `preview:` manifest key (POSIX relpath, same shape as `lyrics`/`cover`). A failed preview decode is logged at debug and skipped without aborting the overall build. Sources with no separate preview are unaffected. Older sloppak readers ignore the unknown `preview` key, so the change is purely additive (sloppak-spec.md §5.5 backward-compat). Documented in `docs/sloppak-spec.md` §2 alongside the other optional top-level keys. Enables [`slopsmith-plugin-song-preview`](https://github.com/byrongamatos/slopsmith-plugin-song-preview) to render hover-to-listen previews for sloppaks without seeking into the full audio.
|
||||||
|
- **Generic plugin asset route** — `GET /api/plugins/{plugin_id}/assets/{path}` serves arbitrary static files a plugin bundles under its own `assets/` directory (AudioWorklet modules, WASM, images, etc.), so plugins can self-host browser-fetchable assets without a CDN (Principle II). Containment is enforced by `lib/safepath.safe_join` against `<plugin>/assets/`, so `..` traversal, absolute paths, and NUL bytes cannot escape `assets/` to reach a plugin's Python modules. `.js` is served as `application/javascript`. First consumer: the stems plugin's pitch-preserving time-stretch worklet.
|
||||||
|
- **Minigames framework — bundled as a core plugin (`plugins/minigames/`).** Promotes the upstream [`slopsmith-plugin-minigames`](https://github.com/byrongamatos/slopsmith-plugin-minigames) repo into the core bundle so every Slopsmith install gets the framework out of the box (same promotion path used for `highway_3d`). The plugin adds a top-level **Minigames** nav link (alongside Library / Favorites / Upload — not buried in the Plugins dropdown), a library-style card grid of installed minigame plugins, and a shared profile layer (XP, level, per-game leaderboards, cross-minigame unlocks) persisted under `CONFIG_DIR/minigames/` and opted into the settings export. Other plugins that want to ship a minigame add a `minigame` block to their `plugin.json` and call `window.slopsmithMinigames.register(spec)`; the SDK exposes scoring (`createContinuous` runs a self-contained YIN tracker; `createDiscrete` / `createChord` wrap `note_detect`'s `createNoteDetector`), HUD primitives, run persistence, and a scheduler so individual minigames don't need their own DSP or backend. Backend endpoints live under `/api/plugins/minigames/{runs,profile,registry}`. The framework is plugin-shaped (not core code) per Principle III, but bundled so it ships with every install. First consumer: [`slopsmith-plugin-flappy-bend`](https://github.com/byrongamatos/slopsmith-plugin-flappy-bend), shipped separately.
|
||||||
|
- **Alpha-build heads-up banner** — when `/api/version` reports a version string containing "alpha" (case-insensitive), an amber banner appears at the top of the library section warning users that the build is in active development and may have bugs or breaking changes. The banner stays hidden on stable / beta / RC builds. No persistence or dismiss state — it's a passive notice, not a modal.
|
||||||
|
- **Drum vocabulary expanded to 18 pieces** — adds `stack` (MIDI 30, from GM's extended-percussion range, unused by real drum-kit MIDIs) and `bell` (MIDI 80 "Mute Triangle", also unused in real drum-kit MIDIs) to `lib/drums.py` PIECES. Inserted in the iteration order so the editor / highway lane ordering is *hi-hat → stack → crash → … → ride bell → bell*. Both are cymbals; default shape `circle_jagged` (stack) / `circle_dot` (bell). Old drum tabs round-trip unchanged — the schema is permissive and existing piece-ids are untouched.
|
||||||
|
- **GP / MIDI drum import surfaces unmapped notes** — `convert_drum_track_to_drumtab` (`lib/gp2rs.py`) and `convert_drum_track_from_midi` (`lib/midi_import.py`) gain an optional keyword-only `out_unmapped` parameter. Callers that pass an empty dict receive a per-MIDI record of every silently-skipped percussion note (`{midi: {"count": int, "times": [float, ...]}}`, times capped at 100 samples per note). This lets the editor plugin show a warning + manual-mapping UI on import instead of silently dropping unmapped notes. Default behavior unchanged for callers that don't opt in.
|
||||||
|
|
||||||
|
- **Drum support from scratch** — drums are now a first-class arrangement type with their own JSON payload on disk and their own WS stream to the highway. New `lib/drums.py` defines the closed piece-id vocabulary (kick, snare, snare_xstick, hh_closed/open/pedal, tom_hi/mid/low/floor, crash_l/r, splash, china, ride, ride_bell), default GM-MIDI mappings, three preset lane configurations, and a permissive `drum_tab.json` validator. `lib/sloppak.py::load_song` reads the manifest's optional top-level `drum_tab:` key, parses + validates the JSON, and surfaces it on `LoadedSloppak.drum_tab`; the load stays permissive so a missing or malformed tab silently disables drums rather than failing the sloppak load. `/ws/highway/{filename}` gains two new message types — `drum_tab` (metadata + kit legend) and chunked `drum_hits` (500 hits per frame, same chunking as notes) — exposed to renderers via `bundle.drumTab`. `song_info` carries a `has_drum_tab` flag so viz pickers can auto-activate the drums highway regardless of which guitar arrangement is selected. `lib/gp2rs.py::convert_drum_track_to_drumtab` converts a Guitar Pro drum track to a `drum_tab.json` dict, preserving velocity verbatim, mapping hi-hat openness through the canonical piece-ids, and flagging flam / ghost / cymbal-choke articulations from GP effects. `lib/midi_import.py` gains `list_drum_tracks` + `convert_drum_track_from_midi` (channel-9 only) with heuristic flam-collapse (≤30 ms same-piece) and choke detection (cymbal note-off ≤120 ms). `docs/sloppak-spec.md` §5.3 promotes drum_tab from worked-example to canonical with the closed piece-id table and wire format. Sloppaks without a drum_tab are unaffected; legacy drums-as-guitar-notes sloppaks keep playing via the drums plugin's fallback decoder.
|
||||||
|
- **Loose folder support** — a directory containing an audio file + arrangement XMLs, with optional `manifest.json` and album art, is now discovered, indexed, and playable as a first-class library format alongside Sloppak. The scanner walks `DLC_DIR` for non-preview audio files and treats each parent directory that also contains XMLs as a loose song. Metadata follows a `manifest.json` → XML tags → folder-name priority chain (see `lib/loosefolder.py`). Songs are tagged `format: "loose"`, render an amber `FOLDER` badge in the library, and are filterable via the new "Folder" option in the format dropdown. Audio uses the shared vgmstream/`convert_wem` pipeline, cached under `AUDIO_CACHE_DIR`. The chart `<offset>` from the first non-vocals XML is now propagated to the frontend via `song_info.offset` and applied in `highway.setTime()` so loose folders authored against non-silence-padded audio stay in sync. Pairs with the companion `slopsmith-plugin-loosefolder` plugin which adds an in-player Fix Sync UI for nudging and saving offset corrections.
|
||||||
|
- Highway note-state hook (#254). New `highway.setNoteStateProvider(fn)` lets a scorer plugin publish a per-note judgment (`'hit'` / `'active'` for a sustain currently held correctly / `'miss'`, or `{ state, alpha, color }`) so the renderer lights up the **gem itself** on a correct hit and keeps a sustain trail glowing while it's still being played right — instead of a separate overlay ring near the note. The built-in 2D highway honors it in `drawNote` / `drawSustains` / the chord-frame path (bright string colour + additive halo on hits, bright vs dim sustain trail, faint red wash on misses); the bundled 3D highway reads the same data via `bundle.getNoteState` (bright string-tinted outline + bright body + glowing sustain on hit/active, red outline + suppressed body on miss). Custom renderers opt in by calling `bundle.getNoteState(note, chartTime)`. note_detect registers the provider (and still owns its HUD / diagnostic miss markers / "currently detected" indicator); renderers that ignore the hook simply don't light gems. On a confirmed hit/active the renderers add a contained "sparkle/sizzle" on the note — the 2D highway: additive throbbing halo + flickering hot core + crackling spark lines (+ an expanding shockwave ring on a fresh strike) on the gem and a glowing/jittery sustain trail; the 3D highway: a few twinkling bright dots and short crackling arc segments hugging the note's rectangle (no bloom past the note), drawn on its overlay and projected through the camera so they ride the note. Also adds `highway.isDefaultRenderer()` so overlays that position with the 2D-highway helpers (`project` / `fretX`) can skip rendering when a custom renderer is active — fixes note_detect's miss markers appearing in random places over the 3D highway. New 3D-highway setting **Show note preview on the fretboard** (on by default) toggles the board-projection ghost — the translucent preview of the upcoming note on the fretboard surface. (Note: the companion change in the **note_detect plugin repo** turns its full-screen green/red edge flash off by default and adds a toggle to re-enable it — ships separately with note_detect, not in this slopsmith release.)
|
||||||
|
- Diagnostic bundle export (#166). New "Export Diagnostics" + "Preview Bundle" buttons in Settings produce a single redacted zip combining server logs (tail of `LOG_FILE`), system info (Python/OS/version), hardware probe (CPU model + cores + freq + RAM, GPU via `nvidia-smi`/`rocm-smi`/`system_profiler`, container/Electron/bare runtime detection), full plugin inventory with git SHA + remote URL (read directly from `.git/HEAD` so it works in minimal runtime images without `git` installed) + orphan/failed-to-load detection, the browser console transcript (all levels: log/info/warn/error/debug + window.onerror + unhandledrejection, 500-entry ring buffer), browser hardware (WebGL/WebGPU adapter info, navigator + userAgentData), filtered localStorage, and per-plugin contributed diagnostics. Top-level `manifest.json` lists every file with its versioned schema id (`system.hardware.v1`, `client.console.v1`, etc.) so AI agents can dispatch by schema. Redaction is on by default: DLC paths, song filenames (`<song:HASH8>` stable per-bundle), IPv4/IPv6 addresses, bearer tokens, and `key=`/`token=` query strings are replaced. Plugins opt their backend diagnostics in via a new `diagnostics` manifest field (`server_files` allowlist mirroring `settings.server_files` semantics, plus an optional `callable: "<module>:<function>"` resolved lazily via `load_sibling`). Frontend plugins push diagnostics via `window.slopsmith.diagnostics.contribute(plugin_id, payload)`. Three new endpoints: `POST /api/diagnostics/export`, `GET /api/diagnostics/preview`, `GET /api/diagnostics/hardware`. Full bundle format spec in `docs/diagnostics-bundle-spec.md`.
|
||||||
|
- Structured logging bootstrap (phase 1 of #155). Three new environment variables control server log output: `LOG_LEVEL` (default `INFO`), `LOG_FORMAT` (`text` for coloured console, `json` for one-JSON-object-per-line suitable for Loki/ELK/Promtail), and `LOG_FILE` (optional path, rotated at 10 MB with 5 backups). HTTP responses now include a `X-Request-ID` correlation header (via `CorrelationIdMiddleware`); the same request ID appears as `request_id` in structured log lines emitted via the stdlib `logging` / `structlog` APIs during that request.
|
||||||
|
- Structured logging migration completed (phase 2 of #155, #159, #242). The 42 `print()` calls and 6 `traceback.print_exc()` calls across `server.py` and `lib/` have been migrated to levelled `slopsmith.*` loggers. Silent `except: pass` blocks in `lib/sloppak.py` and `lib/sloppak_convert.py` now surface as `log.warning` / `log.debug` with the exception attached. WebSocket handlers (`highway_ws`, `ws_retune`) bind a `ws_conn_id` contextvar at accept time so every log line within a session carries a connection ID. A CI grep guard in `.github/workflows/tests.yml` fails the build if either `print(` or `traceback.print_exc(` reappears in `server.py` or `lib/`.
|
||||||
|
- **Lyrics Karaoke plugin** — end-to-end karaoke setup for Sloppak songs in one workflow. The setup screen shows a per-song checklist (vocals stem / synced lyrics / per-syllable pitch) and a single "Build Karaoke" button that runs whatever's missing: Whisper alignment of pasted lyric text against the vocals stem, then `librosa.pyin` pitch extraction. Both artifacts persist inside the Sloppak (`lyrics.json`, `vocal_pitch.json`). In the player, a "Karaoke" toggle swaps the text-lyrics overlay for a horizontal pitch ribbon (one bar per syllable, vertically positioned by pitch, sweeping playhead).
|
||||||
|
- Settings export/import (#113). Two buttons on the Settings page bundle server config, browser localStorage, and opted-in plugin server-side files into a single versioned JSON file for backup, migration, or sharing a calibrated setup. Server-side import is all-or-nothing for safety-critical failures: phase-1 validates the entire bundle (schema, path-traversal, encoding) before any disk writes; phase-2 commits each file via temp+rename. Plugin-state mismatches between export and import are handled leniently: files referenced for a plugin that isn't loaded are skipped with a warning, files referenced for a plugin whose manifest no longer declares them are skipped with a warning, and localStorage is merged (not cleared) so first-run defaults from plugins installed after the export are preserved. Path-traversal, absolute paths, schema mismatch, and decode failures remain hard refusals. Plugins opt their server-side files in by declaring `settings.server_files` in `plugin.json` (list of relpaths under `CONFIG_DIR`; trailing `/` denotes a directory).
|
||||||
|
- Library filtering by parts present or missing (#129, #69). New right-side Filters drawer (single button next to the format/sort row, with active-filter count badge and dismissible chips below) lets you require or exclude arrangements (Lead/Rhythm/Bass/Combo), specific stems on Sloppaks (drums/bass/vocals/piano/other), lyrics, and tuning. Multi-select within an axis is OR (Lead OR Rhythm); cross-axis is AND. State persists across reloads. New endpoint `GET /api/library/tuning-names` returns distinct tunings present in the library, ordered by musical distance.
|
||||||
|
- Sort library by year (#128). Two new options in the sort dropdown: "Year (newest)" and "Year (oldest)". Songs without a year are pushed to the bottom for both directions.
|
||||||
|
- **`highway.getLyrics()` accessor.** `createHighway()` now exposes the parsed timed lyric syllables (`[{t, d, w}]`) via `getLyrics()`, mirroring `getBeats()`/`getSections()`, so overlay plugins can render karaoke without opening a second highway WebSocket. Pure accessor; no behavior change.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **Perf (3D highway, slopsmith#226)**: pre-warm `plugins/highway_3d/screen.js` object pools at board init. Previously the pool factory grew lazily on first `.get()` past the high-water mark, allocating a fresh `T.Mesh` mid-rAF on dense 7/8-string charts and stalling those frames; the meshes were then permanently added to `noteG` (the pool only hides on `reset()`, never removes), bloating the scene graph for the rest of the session. Pre-warming spends the cost up front. Fold the per-frame `updateStringHighlights()` per-string loop with the post-call `mGlow`/`mAccentCore` emissive writes — one walk over the per-string scratch arrays instead of two. Replace `longestConsecutiveRun`'s per-call array allocations with a `{start, len}` index pair (trades two per-call sub-array allocations for one small 2-key object — net reduction in per-visible-chord allocation churn). Opt-in perf bench harness via `?h3dbench=1` URL param: `console.log` p50/p95/max for six update() segments every 5 seconds; when the URL flag is absent the mark helpers are bound to empty functions at renderer-instance init (each `createHighway()` panel re-checks the flag), so the hot-path calls are no-ops with negligible overhead (typically JIT-inlined).
|
||||||
|
- **License**: Relicensed to AGPL-3.0-only. Prior versions claimed MIT in the README, but the bundled desktop build statically links JUCE 8 (AGPL-3.0), so AGPL terms have effectively governed the desktop distribution since JUCE was added. AGPL-3.0-only is now the canonical license for the project — see [LICENSE](LICENSE) and [CONTRIBUTING.md](CONTRIBUTING.md) (DCO sign-off + plugin licensing policy). Bundled and vendored third-party code keeps its original license.
|
||||||
|
- Tuning sort is now ordered by musical distance from E Standard (#22) instead of alphabetical: E Standard first, then Drop D / F Standard at distance 2, then Eb Standard / F# Standard at distance 6, etc. Within a magnitude tier, down-tuned variants come before up-tuned, then alphabetical.
|
||||||
|
- Settings page restructured into separate "Slopsmith" (core) and "Plugins" sections, with each plugin's settings rendered as a collapsible panel (collapsed by default). "Plugin Updates" moved into the Plugins section.
|
||||||
|
- **Lyrics Sync** is now a redirect stub. Its alignment + save endpoints moved into the new Lyrics Karaoke plugin alongside the pitch extraction. Existing nav entries and bookmarks land on a "moved" page that auto-redirects to the merged plugin.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
- **Path traversal in archive extractors and library path resolution.** `lib/sloppak.py::_unpack_zip` and `server.py::_resolve_dlc_path` previously concatenated attacker-controlled entry names or filenames directly onto the extraction or library directory, so a crafted sloppak zip member or library filename with `..` segments, an absolute path, or backslash separators could write or read outside the intended directory. Any code path that unpacks a user-supplied archive (library upload, click-to-play, retune) or resolves a library path was reachable. Both locations now delegate to a new `lib/safepath.py::safe_join` helper that resolves each destination once and rejects entries that don't fall under the target directory; rejected entries are logged and skipped, the rest of the archive still extracts. The stem-split paths in `lib/sloppak_convert.py::split_stems` and `scripts/split_stems.py` previously called `ZipFile.extractall()` directly on user-supplied sloppaks; both now delegate to the same hardened `lib/sloppak.py::_unpack_zip` so every sloppak-unzip site in the codebase shares one containment guarantee. Tests in `tests/test_archive_traversal.py` and `tests/test_safepath.py` pin the contract for `../`, deep traversal, absolute paths, mixed `subdir/../../` forms, Windows-style separators, NUL bytes, names that resolve to the unpack root, and symlinked roots.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- E Standard retune now stays metadata-consistent across a chart's arrangement files (slopsmith-plugin-notedetect#50). Previously the retune path could shift the audio and update manifests while leaving some arrangement metadata untouched, so `load_song()` later exposed the *original* tuning at runtime. `lib/retune.py` now updates every arrangement's tuning metadata consistently before applying the E Standard tuning, and raises on a partial update instead of silently packing split tuning metadata. EStd files generated before this fix should be re-converted so their metadata is consistent.
|
||||||
|
- Keyboard shortcut help now opens from the Player/3D Highway context when Linux/Electron reports Shift+Slash as `key="/"`, including while player controls such as the visualization picker are focused (#598).
|
||||||
|
- 3D Highway left-handed mode now has regression coverage for fret-axis mirroring, board rebuilds on runtime lefty changes, and mirrored camera state including the lookahead target and shoulder offset; the maintainer guide no longer claims the renderer ignores `bundle.lefty` (#321).
|
||||||
|
- Chord-level `fretHandMute` is now parsed into each note's `fret_hand_mute` (wire `fhm`) instead of being folded into `mute` (`mt`), matching `_parse_note` and preserving wire-format fidelity for both the template-expanded (synthetic-note) and explicit-`chordNote` paths. The 3D highway renders the fret-hand-mute X for `mt` *or* `fhm` notes, so the muted-chord overlay still shows. Also fixes the per-note fret-connector label vanishing exactly at the hit line (the fade now holds full opacity through `dt = 0`).
|
||||||
|
- `gp2rs` now respects the time-signature denominator when emitting ebeat subdivisions, fixing misaligned beat grids in 6/8 and other non-quarter-note meters.
|
||||||
|
- Settings dropdowns (Default Arrangement, Platform Filter) now persist immediately when changed. Previously a dropdown selection was only written to `config.json` when an unrelated "Save" button (Library Folder or Demucs Server) was clicked, so picking a default arrangement and navigating away silently discarded it. Both `<select>` controls now POST the single changed field on `change` via the partial-merge `/api/settings` endpoint, matching the auto-save behaviour of the A/V Sync Offset and mastery sliders. The text inputs (Library Folder Path, Demucs Server URL) keep their explicit Save buttons. Autosaves are sent through a single client-side queue (one request in flight at a time, in selection order), and the `POST /api/settings` handler now serializes its read-modify-write of `config.json` under a lock so concurrent partial updates can no longer overwrite each other and drop a key. The config write is also atomic (temp + rename) and `/api/settings/import` shares the same lock, so readers never observe a half-written file and a settings import can't race a concurrent partial save.
|
||||||
|
- Demucs stem split failing on Windows desktop with `OSError: Could not load this library: libtorchcodec_core4.dll` or `ImportError: TorchCodec is required for save_with_torchcodec`. The demucs subprocess now bootstraps a `torchaudio.save` → `soundfile.write` shim before importing demucs, sidestepping the torchcodec dependency entirely. The override stays in place across torchaudio versions — soundfile's WAV writes are behaviorally equivalent for demucs's float32 outputs.
|
||||||
|
- Splitscreen pop-out windows briefly flashed the library/song grid before showing the popped panel. A popup loads the full app (whose default screen, `#home`, is the library) and only swaps to the player once the splitscreen plugin loads; app init now detects `?ssFollower=1` and switches to the player screen up front, so the popup shows player chrome the whole time.
|
||||||
|
- Sloppak assembly dropped all tone data — affected sloppaks showed no signal chain in the Tones plugin and no tone-change markers on the highway. The assembler (`lib/sloppak_convert.py`) now lifts each arrangement's tones from the source chart via the new `lib/tones.py` helper and embeds them inline in the arrangement JSON under a `tones` key (`base`, `changes`, `definitions` — see `docs/sloppak-spec.md` §3.9). The highway WebSocket reads `base`/`changes` for sloppaks, and the Tones plugin (≥ 1.1.0) reads `definitions` to render the gear chain. Sloppaks built before this release carry no tone data and must be rebuilt from their source chart to gain it.
|
||||||
|
- Tab View (slopsmith-plugin-tabview ≥ 3.0.1): the bottom row of tablature was permanently hidden behind the player controls bar (#336). The overlay reserved 60px at the *top* (clearing the transparent HUD) and extended all the way to the bottom of `#player`, where the opaque `#player-controls` (z-index 10) drew over the last row. The overlay now measures `#player-hud` and `#player-controls` dynamically and insets both edges; a `ResizeObserver` on the controls bar re-runs the inset when it wraps to a second row on narrow viewports.
|
||||||
|
- Tab View (slopsmith-plugin-tabview ≥ 3.0.1): the cursor highlight led playback by roughly one beat (#336). alphaTab snaps `tickPosition` to the start of the *next* beat, so the cursor would race ahead by 500ms+ at typical tempos. The plugin now sends `tickPosition` one beat earlier so the snap lands on the current beat, and the highlight overlay tracks the bar cursor (`.at-cursor-bar`) instead of the next-beat cursor (`.at-cursor-beat`).
|
||||||
|
|
||||||
|
### Migration notes
|
||||||
|
- **Constitution amended to 1.1.0 (Principle II — Vanilla Frontend).** Prebuilt Tailwind (`static/tailwind.min.css`) is now codified as non-negotiable: no Play CDN / runtime CSS JIT anywhere, core or plugin. Plugin authors: a plugin that uses Tailwind classes not guaranteed in core — especially arbitrary values like `w-[37px]` — MUST ship its own compiled stylesheet via the new `styles` manifest key, built with `corePlugins.preflight = false`. Plugins that use only core-guaranteed utilities, or that ship no Tailwind at all, need no change. Contributors: after adding any Tailwind class to core or a bundled plugin, run `bash scripts/build-tailwind.sh` and commit the regenerated CSS, or the `tailwind-fresh` CI job fails.
|
||||||
|
- The library filters depend on three new columns (`stem_ids`, `tuning_name`, `tuning_sort_key`) that are populated as songs are scanned. If filters look empty after upgrading, run **Settings → Full Rescan** to repopulate; alternatively the periodic background rescan picks them up over time.
|
||||||
|
|
||||||
|
## [0.2.4] - 2026-04-22
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- Version badge in navbar (`/api/version` endpoint + `VERSION` file)
|
||||||
|
- `CHANGELOG.md` and semantic versioning
|
||||||
|
- Step Mode plugin
|
||||||
|
- `gp2midi` improvements and expanded test coverage
|
||||||
|
- Note Detection plugin factory-pattern refactor with multi-instance/splitscreen support
|
||||||
|
- Per-panel note detection in Split Screen plugin with M/L/R channel routing for multi-input interfaces
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- `SLOPPAK_CACHE_DIR` moved to `CONFIG_DIR` for AppImage compatibility
|
||||||
|
- Improved error message when plugin requirements fail to install
|
||||||
@@ -0,0 +1,633 @@
|
|||||||
|
# Slopsmith — AI Agent Guide
|
||||||
|
|
||||||
|
Slopsmith is a self-hosted web app for browsing, playing, and practicing interactive music notation, built around its own open `.sloppak` chart format. Charts come from importing Guitar Pro (GP5/GP8) or MusicXML, or from authoring in the built-in editor. It runs as a Docker container with a FastAPI backend (`server.py`), vanilla JavaScript frontend (`static/`), shared Python libraries (`lib/`), and an extensive plugin system (`plugins/`). There are no frontend frameworks — everything is plain JS, HTML, and Tailwind CSS.
|
||||||
|
|
||||||
|
## Architecture Quick Reference
|
||||||
|
|
||||||
|
```
|
||||||
|
server.py FastAPI app — library API, WebSocket highway, plugin loading
|
||||||
|
static/
|
||||||
|
app.js Main frontend — screens, library views, player, plugin loader
|
||||||
|
highway.js Canvas note highway renderer (createHighway factory)
|
||||||
|
index.html Single-page app shell
|
||||||
|
style.css Custom CSS loaded alongside Tailwind
|
||||||
|
lib/
|
||||||
|
song.py Core data models (Note, Chord, Arrangement, Song)
|
||||||
|
sloppak.py Sloppak format support
|
||||||
|
loosefolder.py Loose-folder XML chart support
|
||||||
|
audio.py OGG/MP3 audio handling
|
||||||
|
retune.py Pitch-shifting logic
|
||||||
|
tunings.py Tuning name/offset utilities
|
||||||
|
gp2rs.py Guitar Pro to arrangement XML conversion
|
||||||
|
gp2midi.py Guitar Pro to MIDI
|
||||||
|
plugins/
|
||||||
|
__init__.py Plugin discovery, loading, requirements install
|
||||||
|
<plugin_name>/ Each plugin is its own directory (often a git submodule)
|
||||||
|
tests/
|
||||||
|
test_*.py pytest test suite
|
||||||
|
```
|
||||||
|
|
||||||
|
## Plugin System
|
||||||
|
|
||||||
|
Plugins are the primary extension point. Each plugin lives in `plugins/<name>/` with a `plugin.json` manifest. Plugins are typically their own git repositories — see [CONTRIBUTING.md](CONTRIBUTING.md) for the licensing policy (curated plugins should be AGPL-3.0 or AGPL-compatible: MIT, BSD, Apache-2.0).
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "my_plugin",
|
||||||
|
"name": "My Plugin",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"private": false,
|
||||||
|
"type": "visualization",
|
||||||
|
"nav": { "label": "My Plugin", "screen": "plugin-my_plugin" },
|
||||||
|
"screen": "screen.html",
|
||||||
|
"script": "screen.js",
|
||||||
|
"styles": "assets/plugin.css",
|
||||||
|
"routes": "routes.py",
|
||||||
|
"settings": {
|
||||||
|
"html": "settings.html",
|
||||||
|
"server_files": ["my_plugin.db", "my_plugin_models/"]
|
||||||
|
},
|
||||||
|
"diagnostics": {
|
||||||
|
"server_files": ["my_plugin.diag.json"],
|
||||||
|
"callable": "diagnostics:collect"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
All fields except `id` and `name` are optional. Plugins can have any combination of frontend (screen/script), backend (routes), and settings.
|
||||||
|
|
||||||
|
`version` and `private` are advisory metadata — the plugin loader does not currently consume them, but plugins commonly include them for publishing/tooling purposes.
|
||||||
|
|
||||||
|
`description`, `category`, and `icon` are **optional, additive v3 Pedalboard metadata** (surfaced in `/api/plugins`, consumed by the v3 Plugins page `static/v3/plugins-page.js`). `description` is a short one-sentence summary shown under the pedal name. `category` (`audio | creation | practice | game | tools`, free-form; unknown/absent → curated default → `"other"`) picks which pedalboard the plugin sits on. `icon` is an assets-relative thumbnail path (e.g. `"assets/thumb.png"`, ~square ~256×256, same containment rule as `styles`, served via `/api/plugins/<id>/assets/...`); if omitted the loader auto-detects `assets/thumb.png`, and plugins with no thumbnail get a default pedal graphic. All three are backward-compatible — omit them and the plugin still loads. See [docs/plugin-v3-ui.md](docs/plugin-v3-ui.md).
|
||||||
|
|
||||||
|
`styles` is the **opt-in** for self-hosted CSS (Principle II — prebuilt Tailwind, no Play CDN). Core's `static/tailwind.min.css` only contains classes scanned from core source at build time, so a plugin installed at runtime (community / NAS) that uses classes core didn't scan — especially arbitrary values like `text-[11px]` — renders unstyled. Declaring `styles` makes the frontend inject one versioned `<link rel="stylesheet">` into `<head>` (covering the plugin's screen *and* its settings panel) pointing at the plugin's own compiled stylesheet. The value is a **plugin-root-relative path that must live under `assets/`** (e.g. `"assets/plugin.css"`) so it serves through the sandboxed `/api/plugins/<id>/assets/...` route. Build it with `corePlugins: { preflight: false }` (utilities only — core ships the single base reset; don't duplicate it) and **never** the Tailwind Play CDN. Plugins that use only core-guaranteed utilities, or ship no Tailwind, omit `styles` and are byte-for-byte unaffected. Full authoring guide + scaffold: [docs/plugin-styles.md](docs/plugin-styles.md).
|
||||||
|
|
||||||
|
`settings.server_files` is the **opt-in** for the unified Settings export/import flow (slopsmith#113). It's a list of relpaths under `context["config_dir"]` that the plugin wants included in user-triggered backups. A trailing `/` denotes a directory (recurse). Plugins that omit this field have no server-side files exported; their state lives entirely in browser `localStorage`, which is bundled wholesale on every export. Rules:
|
||||||
|
- Relpaths only. Absolute paths, drive letters, `..` segments, and backslashes are rejected at load time with a `[Plugin]` warning.
|
||||||
|
- The same allowlist is consulted at both export and import: a bundle that references a file the *importing host*'s manifest no longer declares is skipped with a warning (handles plugin updates between export and import). A bundle that references a file your host's manifest never declared is also skipped — no surprise writes.
|
||||||
|
- Files are encoded as `{"encoding": "json", "data": <parsed>}` for `.json` files that parse cleanly (diff-friendly), `{"encoding": "base64", "data": "..."}` otherwise (sqlite, model blobs, IRs).
|
||||||
|
- Plugins own their internal data migration. Importing a bundle whose data schema predates your current code restores bytes verbatim — your plugin must cope at next load.
|
||||||
|
- Symlinks are skipped on export and never followed on import.
|
||||||
|
|
||||||
|
`diagnostics` is the **opt-in** for the troubleshooting bundle (slopsmith#166 — Settings → Export Diagnostics). Two independent fields:
|
||||||
|
- `diagnostics.server_files` — same allowlist semantics as `settings.server_files`: relpaths under `context["config_dir"]`, no `..`, no abs paths, no backslashes, no leading dots. Files listed here are copied verbatim into `plugins/<plugin_id>/<relpath>` inside the bundle. Use this for snapshot-style state (small DB excerpts, model lists, last-error files).
|
||||||
|
- `diagnostics.callable` — `"<module>:<function>"` (e.g. `"diagnostics:collect"`). Resolved lazily via `load_sibling` when the user clicks Export, then called as `func({"plugin_id": "...", "config_dir": Path(...)})`. Return `dict`/`list` → written to `plugins/<id>/callable.json`; `bytes` → `callable.bin`; `str` → `callable.txt`. Exceptions are caught and appended to the bundle's `manifest.notes` — a buggy plugin never crashes the export.
|
||||||
|
|
||||||
|
Plugins that omit the field contribute nothing to the bundle from the backend side. Frontend plugins can independently push state via `window.slopsmith.diagnostics.contribute(plugin_id, payload)` from their `screen.js` before the user hits Export. Bundle layout + per-file schemas: [docs/diagnostics-bundle-spec.md](docs/diagnostics-bundle-spec.md).
|
||||||
|
|
||||||
|
Best practices:
|
||||||
|
- Embed your own `schema` field (e.g. `"my_plugin.diag.v1"`) in JSON returned by `callable` so future tooling can dispatch by version.
|
||||||
|
- Keep payloads small (< 100 KB). Diagnostics are not a backup channel — that's `settings.server_files`.
|
||||||
|
- Don't include user secrets, API keys, or session tokens. The bundle is shared with maintainers / posted to GitHub issues.
|
||||||
|
|
||||||
|
`type` is an optional role hint (slopsmith#36). Supported values:
|
||||||
|
- `"visualization"` — plugin provides a highway renderer. Declaring this makes the plugin eligible for the main-player viz picker AND splitscreen's per-panel picker. Must pair with a `window.slopsmithViz_<id>` factory exporting the setRenderer contract below.
|
||||||
|
- Absent → no declared role; plugin is loaded and its script runs, but it doesn't appear in role-specific UIs.
|
||||||
|
|
||||||
|
**Backend routes** — `routes.py` must export a `setup(app, context)` function. The `context` dict provides:
|
||||||
|
- `config_dir` — persistent config path
|
||||||
|
- `get_dlc_dir()` — returns the DLC folder Path
|
||||||
|
- `extract_meta()` — metadata extraction callable
|
||||||
|
- `meta_db` — shared MetadataDB instance
|
||||||
|
- `library_providers` — shared library provider registry for source-aware browsing
|
||||||
|
- `register_library_provider(provider)` — register a plugin-provided library source. Providers expose `id`, `label`, optional `kind`/`capabilities`, and callable `query_page`, `query_artists`, `query_stats`, and `tuning_names` methods. Providers with `art.read` may also expose `get_art(song_id)` returning one of: a `Response` object (any media type, served as-is); raw `bytes` or `bytearray` (**assumed PNG** — use a `Response` or a `dict` with `content`+`media_type` keys for JPEG/WebP or other formats); a URL string (http/https → 302 redirect; other schemes are rejected with 400); a filesystem path string or `Path` (served as a file with auto-detected media type); or a `dict` with a `url`, `path`, or `content` key. Providers with `song.sync` may expose `sync_song(song_id)` returning `None` (success with no local file) or a `dict` — the dict is passed through as the JSON response and should include `filename`/`local_filename` if a local playable file was produced.
|
||||||
|
- `unregister_library_provider(provider_id)` — remove a plugin-provided library source by id. The built-in `local` provider cannot be removed.
|
||||||
|
- `get_sloppak_cache_dir()` — sloppak cache path
|
||||||
|
- `load_sibling(name)` — loads a sibling module from this plugin's directory under a unique, namespaced module name. See "Sibling imports" below.
|
||||||
|
- `log` — stdlib `logging.Logger` namespaced to `slopsmith.plugin.<id>`. Pre-configured with the app-wide level, format (including JSON mode), and correlation IDs. Use this for all backend plugin output instead of `print()`. See "Backend plugin logging" below.
|
||||||
|
|
||||||
|
**Sibling imports — use `load_sibling`, not bare imports** (slopsmith#33). The plugin loader inserts each plugin's directory onto `sys.path` so `from extractor import X` works, but Python caches imports by **module name** in `sys.modules`. Two plugins that each ship a top-level `extractor.py` (or any other generic name — `util.py`, `client.py`, `parser.py`, `config.py`, …) collide: whichever loads first wins, and the other plugin's `from extractor import X` either gets the wrong module or fails with `cannot import name 'X' from 'extractor'`.
|
||||||
|
|
||||||
|
The fix is `context["load_sibling"](name)`, which loads the sibling under a namespaced module name (`plugin_<id>.<name>`, where plugin_id is bijectively encoded so reverse-DNS-style ids like `com.example.foo` work without colliding: `_` -> `_5f_`, `.` -> `_2e_`) so each plugin gets its own copy:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def setup(app, context):
|
||||||
|
extractor = context["load_sibling"]("extractor")
|
||||||
|
PsarcReader = extractor.PsarcReader
|
||||||
|
# …
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
- `name` is a bare module name — no `.py` suffix, no slashes, no `.`. The helper raises `ValueError` for path traversal / format issues and `ImportError` for missing files.
|
||||||
|
- Both single-file siblings (`extractor.py`) and package-form siblings (`extractor/__init__.py`) work. Package form wins when both exist (matches CPython's import-resolution precedence).
|
||||||
|
- Relative imports between siblings work — `from .shared import X` in a top-level helper, `from ..shared import X` from inside a sibling package. The synthetic parent package `plugin_<id>` carries the plugin directory in its `__path__`.
|
||||||
|
- `from . import sibling` (attribute-style) also resolves: loaded children are exposed as attributes on the parent package.
|
||||||
|
- Repeat calls return the cached module. Concurrent first-time calls are serialized via per-module locks so no caller observes a half-initialized module.
|
||||||
|
- Bare `import sibling` from `routes.py` still works during the transition period, but the loader prints a startup warning when it detects two plugins shipping a same-named top-level module — covering both `.py` files and package directories. Migrate to `load_sibling` to silence the warning and immunize your plugin from future ecosystem collisions. (Don't mix bare imports and `load_sibling` for the same module — they'd execute the file twice and split module-level state.)
|
||||||
|
|
||||||
|
**Frontend scripts** — `screen.js` runs in the global scope via a `<script>` tag. It can access `window.playSong`, `window.showScreen`, `window.createHighway`, the `<audio>` element, and the `window.slopsmith` event emitter.
|
||||||
|
|
||||||
|
**The playSong wrapper chain** — Plugins commonly wrap `window.playSong` to hook into song playback. Plugins load alphabetically, so the last-loaded (alphabetically later) wrapper runs first, while the alphabetically first plugin runs closest to the original. Be aware that `await` calls in inner wrappers yield to the event loop — WebSocket messages can arrive before outer wrappers finish setup.
|
||||||
|
|
||||||
|
## Plugin Best Practices
|
||||||
|
|
||||||
|
### v3 UI (fee[dB]ack v0.3.0) — player-chrome contract
|
||||||
|
|
||||||
|
v0.3.0 ships a redesigned UI behind a flag (`SLOPSMITH_UI=v3` or the `/v3` route);
|
||||||
|
the classic UI (v2) stays the default until 0.3.0 ships, so **plugins must work in
|
||||||
|
both**. v3 reuses the same engine (`server.py`, `app.js`, `highway.js`, `playSong`,
|
||||||
|
`showScreen`, capabilities, library providers, the `window.slopsmithViz_<id>` /
|
||||||
|
`setRenderer` contract), so a plugin's **backend, capabilities, `nav`/`screen`,
|
||||||
|
visualization renderers, diagnostics, and settings export work unchanged** — v3
|
||||||
|
surfaces `nav` in its sidebar and mounts screens exactly as v2 does.
|
||||||
|
|
||||||
|
**The only thing that changed is the player chrome.** If your plugin injects a
|
||||||
|
control into it, you must adapt:
|
||||||
|
|
||||||
|
- v2's wide always-visible `#player-controls` bar is, in v3, a **minimal
|
||||||
|
auto-hiding transport** (fades ~2.5 s after the pointer stills during playback)
|
||||||
|
plus a hover-reveal left icon rail. So injecting into `#player-controls` the
|
||||||
|
legacy way means your control **auto-hides**, and the legacy insertion anchors
|
||||||
|
(`insertBefore` the `span.text-gray-700` separator, or `button:last-child` / ✕
|
||||||
|
Close) **don't exist in v3** → it lands wrong / unreachable.
|
||||||
|
- **Detect v3** with `window.slopsmith.uiVersion === 'v3'` and **mount into
|
||||||
|
`window.slopsmith.ui.playerControlSlot()`** (a stable, always-reachable container
|
||||||
|
— the "Plugins" rail popover) instead of `#player-controls`. Drop the dead
|
||||||
|
anchors (append), and guard re-injection against the *actual* container
|
||||||
|
(`controls.contains(myBtn)`), not a hard-coded `#player-controls`.
|
||||||
|
- A host `MutationObserver` re-homes legacy `#player-controls` children into the
|
||||||
|
slot as a fallback, but it **breaks plugins that guard on
|
||||||
|
`#player-controls.contains()`** (the moved node fails the check → re-inject every
|
||||||
|
song). Mount into the slot yourself; don't rely on the shim.
|
||||||
|
- v3 uses `fb-*` tokens (`fb-card`, `fb-text`, `fb-textDim`, `fb-primary`,
|
||||||
|
`fb-border`) vs v2's `dark-*`/`accent`; legacy classes still render acceptably.
|
||||||
|
Keep `#player` overlay `z-index` ≤ the chrome layers (transport/HUD 20, rail 30,
|
||||||
|
popovers 40).
|
||||||
|
|
||||||
|
Full guide + the canonical snippet: **[docs/plugin-v3-ui.md](docs/plugin-v3-ui.md)**.
|
||||||
|
Verify any player-injecting plugin in **both** `/` (v2) and `/v3`.
|
||||||
|
|
||||||
|
### Performance — never run DOM queries on a per-frame path
|
||||||
|
|
||||||
|
Plugins share the main thread with the highway's 60 fps render loop, and during
|
||||||
|
playback the highway + note detectors mutate the DOM ~60×/s — so anything that
|
||||||
|
*reacts* to DOM changes runs that often too. Work that looks cheap in isolation
|
||||||
|
becomes the dominant cost when it runs every frame. A profiled "the 3D highway is
|
||||||
|
laggy" report turned out to be **three plugins doing per-frame `querySelectorAll`**
|
||||||
|
(~18% of main-thread CPU + NodeList GC churn), not the renderer. The GPU was idle.
|
||||||
|
|
||||||
|
- **Never call `querySelector` / `querySelectorAll` inside `draw()`, a
|
||||||
|
`requestAnimationFrame` loop, a short `setInterval`, or a `MutationObserver`
|
||||||
|
callback.** Resolve the element(s) **once** when your UI mounts and cache the
|
||||||
|
references; re-resolve only when the cached node is gone (`!el.isConnected`).
|
||||||
|
`querySelectorAll` also allocates a fresh `NodeList` every call → GC pressure at
|
||||||
|
60 fps. (notedetect #75 — a VU meter that `querySelector`'d its bar every tick.)
|
||||||
|
|
||||||
|
- **Scope `MutationObserver`s narrowly — never `observe(document.body, { subtree:
|
||||||
|
true })` just to notice your own UI's container mount.** A body-subtree observer
|
||||||
|
fires on *every* DOM mutation anywhere, including the per-frame highway churn, so
|
||||||
|
a callback that then scans the document is a per-frame full-DOM scan. Observe the
|
||||||
|
specific container; if it's swapped on screen changes, observe a stable parent,
|
||||||
|
or **cheaply early-bail** (one `getElementById` / a screen-state check) *before*
|
||||||
|
the expensive work. (sloppak-converter #32 — a body-subtree observer re-ran
|
||||||
|
whole-document inject sweeps on every frame of playback.)
|
||||||
|
|
||||||
|
- **Stop playback-tied loops when their UI is hidden.** An rAF/interval meter (VU,
|
||||||
|
etc.) that keeps drawing while its panel is closed is pure waste — gate it on
|
||||||
|
visibility, or stop and restart it on open/close.
|
||||||
|
|
||||||
|
- **Per-instance, not global.** Under splitscreen a viz/detector plugin runs
|
||||||
|
multiple instances. Cache refs and resolve panels against your *own* instance's
|
||||||
|
container, never a global `document.querySelector` that could grab a sibling
|
||||||
|
instance's node. (notedetect #75 follow-up.)
|
||||||
|
|
||||||
|
These are cheap to get right up front and expensive to retrofit. Profile the
|
||||||
|
**main thread**, not the GPU, when a renderer "feels laggy" — the offender is
|
||||||
|
usually an unrelated plugin's per-frame DOM work.
|
||||||
|
|
||||||
|
### Visualization plugins — two complementary contracts
|
||||||
|
|
||||||
|
Slopsmith supports two ways for a plugin to participate in the main player's visuals. They coexist; the setRenderer contract is the default for any viz that draws a highway-shaped surface, and overlays handle layered decorations on top.
|
||||||
|
|
||||||
|
**Pick the right shape:**
|
||||||
|
- Replacing the whole highway drawing on the existing highway canvas (your renderer owns its rendering context / resources; `createHighway()` still owns the canvas element and the rAF loop)? → **setRenderer** (section 1). Enters the viz picker. Works in both the main player and per-panel under splitscreen.
|
||||||
|
- Adding a layer on top of whichever viz is active? → **Overlay** (section 2). Navbar toggle, not in the picker.
|
||||||
|
|
||||||
|
#### 1. setRenderer contract (slopsmith#36) — preferred
|
||||||
|
|
||||||
|
Plugins that want to replace the main highway's draw function (per panel, per session) export a renderer factory on `window.slopsmithViz_<id>` where `<id>` matches the `id` in `plugin.json` (`type: "visualization"` required). The factory returns an object matching this shape:
|
||||||
|
|
||||||
|
```js
|
||||||
|
window.slopsmithViz_my_viz = function () {
|
||||||
|
return {
|
||||||
|
// Required canvas context type. Default '2d' if omitted.
|
||||||
|
// highway.js reads this BEFORE calling init() so it can
|
||||||
|
// replace the underlying <canvas> element if the current
|
||||||
|
// one is locked to a different context type (see "Canvas
|
||||||
|
// context-type swapping" below).
|
||||||
|
contextType: '2d', // or 'webgl2'
|
||||||
|
init(canvas, bundle) {
|
||||||
|
// One-time setup. Own your getContext() call here —
|
||||||
|
// acquire '2d' or 'webgl2' depending on the renderer.
|
||||||
|
// The canvas you receive is guaranteed to either be
|
||||||
|
// unbound or already bound to your declared contextType.
|
||||||
|
this.ctx = canvas.getContext('2d');
|
||||||
|
},
|
||||||
|
draw(bundle) {
|
||||||
|
// Called each requestAnimationFrame tick by the factory.
|
||||||
|
// `bundle` is a snapshot with: currentTime, songInfo, isReady,
|
||||||
|
// notes, chords, anchors (all difficulty-filter-aware),
|
||||||
|
// beats, sections, chordTemplates, stringCount, lyrics,
|
||||||
|
// toneChanges, toneBase, mastery, hasPhraseData, inverted,
|
||||||
|
// lefty, renderScale, lyricsVisible, the 2D coordinate
|
||||||
|
// helpers project and fretX, and getNoteState (see below).
|
||||||
|
// `stringCount` is the active arrangement's string count (4
|
||||||
|
// for bass, 6 for guitar, 7+ for extended-range GP imports —
|
||||||
|
// size string-indexed geometry against this, not a hardcoded
|
||||||
|
// 6). If your renderer needs lefty-aware text rendering, check
|
||||||
|
// bundle.lefty and apply the mirror transform yourself —
|
||||||
|
// a bundle-level helper isn't provided because it would
|
||||||
|
// need your renderer's own context, not the factory's.
|
||||||
|
//
|
||||||
|
// bundle.getNoteState(note, chartTime) (slopsmith#254) — call
|
||||||
|
// this per visible chart note / chord-note to find out whether
|
||||||
|
// a scorer (note_detect) has flagged it 'hit' / 'active' (a
|
||||||
|
// sustain currently being held correctly) / 'miss', so the gem
|
||||||
|
// itself can light up / a held sustain can glow instead of
|
||||||
|
// relying on an overlay ring. Returns null when no provider is
|
||||||
|
// registered or it reports nothing for this note; otherwise
|
||||||
|
// { state: 'hit'|'active'|'miss', alpha: 0..1, color: string|null }.
|
||||||
|
// For chord notes pass the chord's time (note_detect keys its
|
||||||
|
// judgments by `${time}_${string}_${fret}`). 'hit' and 'active'
|
||||||
|
// are both "lit" — a renderer may treat them identically; the
|
||||||
|
// provider owns all fade timing via `alpha` and by simply
|
||||||
|
// ceasing to return state when the effect should end.
|
||||||
|
},
|
||||||
|
resize(w, h) {
|
||||||
|
// Optional. Canvas dims already updated; re-create WebGL
|
||||||
|
// framebuffers / reset 2D transforms here.
|
||||||
|
},
|
||||||
|
destroy() {
|
||||||
|
// Optional. Release resources, remove DOM nodes, null refs.
|
||||||
|
// Called before setRenderer() swaps to another renderer
|
||||||
|
// and on highway.stop().
|
||||||
|
},
|
||||||
|
};
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Selecting this plugin in the main-player viz picker — or in splitscreen's per-panel picker — calls `highway.setRenderer(factory())` on the existing highway instance. The built-in 2D highway is the default renderer and is restored by passing nullish — `setRenderer(null)` and `setRenderer(undefined)` both work (the implementation gates on `r == null`). Splitscreen panels create one `createHighway()` per panel and each independently consults the picker, so N panels can run different renderers (or N copies of the same renderer with different arrangements) without coordination.
|
||||||
|
|
||||||
|
**Lifecycle contract.** The factory returns a single renderer instance that may go through multiple `init() → ... → destroy()` cycles as the user navigates between songs or screens. Specifically:
|
||||||
|
|
||||||
|
- `init(canvas, bundle)` runs when the highway has a canvas and the renderer takes over drawing. This is when to acquire `getContext()`, build shaders / meshes / DOM nodes, and register listeners.
|
||||||
|
- `draw(bundle)` runs on every rAF frame once the WebSocket `ready` message has fired and until the renderer is replaced or the highway stops. It is **not** called during the loading / reconnect window (between `api.init()` + `stop()` and the next `ready`) — that would hand the renderer half-populated chart arrays. Renderers that want to show a "loading" state can read `bundle.isReady` inside a future-widened contract, but today the factory gates `draw` behind the ready flag and `isReady` is only informational once it does fire.
|
||||||
|
- `destroy()` runs when the renderer is replaced via another `setRenderer(...)` call, OR when `highway.stop()` is called (e.g. the user navigates away from the player). It releases everything `init()` acquired.
|
||||||
|
- **After `destroy()`, the same instance may receive another `init()` call** — this happens on `playSong()` which does `stop()` → `init()` to reuse the same canvas element for the next song. Renderers must tolerate `init()` being called again on an instance that was previously destroyed. Practically: null your refs in destroy, re-acquire them in init.
|
||||||
|
- `destroy()` is skipped when it would run on an un-init'd renderer — if a caller does `setRenderer(x)` before the highway ever init'd (possible when restoring a saved picker selection at page load), `x.destroy()` is not called until `x.init()` has run at least once.
|
||||||
|
- `resize(w, h)` is optional; runs after init and whenever the canvas dimensions change.
|
||||||
|
|
||||||
|
**Key rules:**
|
||||||
|
- The factory **returns a fresh object on each call** — important for splitscreen, where multiple panels will each get an independent instance.
|
||||||
|
- The renderer **owns its own rendering context** (2D or WebGL). Factory will not call getContext for you.
|
||||||
|
- **Canvas context-type swapping.** Browsers lock a `<canvas>` to the first context type successfully acquired for its lifetime: once `getContext('2d')` succeeds, `getContext('webgl2')` on that same canvas returns `null`, and vice versa. To let arbitrary 2D ⇄ WebGL renderer swaps work mid-session, `highway.setRenderer()` reads the next renderer's `contextType` before calling its `init()` and, if it differs from the type currently bound, replaces the underlying `<canvas>` element with a fresh one via `oldCanvas.cloneNode(false)` followed by `oldCanvas.replaceWith(newCanvas)`. The factory then calls the renderer's `init(newCanvas, bundle)` with the fresh element so its `getContext()` succeeds. Practical implications:
|
||||||
|
- **What survives the swap.** `cloneNode(false)` preserves *every HTML attribute* on the element — `id`, `class`, inline `style`, all `data-*` and `aria-*` attributes, `role`, `tabindex`, the attribute form of `width`/`height`, and anything else a plugin attached. DOM position is preserved by `replaceWith()`, so siblings, parents, and surrounding layout are unaffected.
|
||||||
|
- **What does NOT survive.** Event listeners attached via `addEventListener` are NOT cloned, and expando properties set imperatively on the JavaScript object (such as the bound rendering context, or any `canvas._myPlugin = …`-style data a plugin attached) are not carried over either. The bound rendering context being left behind on the detached element is exactly what allows the new canvas to start fresh and accept a different `getContext()` call. Note: `canvas.width`/`canvas.height` *are* reflected HTML attributes, so those values do survive the clone; `api.resize()` re-applies the backing-store dimensions on the new element after the swap regardless.
|
||||||
|
- Renderers must **declare `contextType`** on the returned instance (`'2d'` or `'webgl2'`; absent → `'2d'`). Factories may also expose it as a static (`window.slopsmithViz_<id>.contextType = 'webgl2'`) so core can read it before constructing the renderer — used today by Auto-mode evaluation.
|
||||||
|
- Plugins that hold a stale reference to the highway canvas across renderer swaps — including any code that registered listeners directly on the canvas element rather than on `window`/`document` — should listen for the `highway:canvas-replaced` event on `window.slopsmith` and re-acquire / re-register. `window.slopsmith.emit` dispatches a `CustomEvent`, so the payload `{ oldCanvas, newCanvas, contextType }` lives on `event.detail`, not on the event object itself:
|
||||||
|
```js
|
||||||
|
window.slopsmith.on('highway:canvas-replaced', (event) => {
|
||||||
|
const { oldCanvas, newCanvas, contextType } = event.detail;
|
||||||
|
// re-acquire / re-register against newCanvas
|
||||||
|
});
|
||||||
|
```
|
||||||
|
Plugins that re-query `document.getElementById('highway')` lazily inside their own event handlers don't need this listener — they pick up the new element automatically (it keeps `id="highway"`).
|
||||||
|
- **`highway:visibility`** — fired on `window.slopsmith` whenever the highway canvas transitions between displayed and hidden. Detection is DOM-based via `canvas.offsetParent === null` (catches `display:none` on the canvas or any ancestor — e.g. splitscreen's `#highway` hide) or whatever a host explicitly sets via `highway.setVisible(bool)`. While `visible === false`, core skips the rAF `renderer.draw(bundle)` call AND the default 2D draw, so renderers don't have to no-op themselves. The event is emitted only on transitions (including the first one after `init()`), not every frame. Payload `{ visible, canvas }` lives on `event.detail`:
|
||||||
|
```js
|
||||||
|
window.slopsmith.on('highway:visibility', (event) => {
|
||||||
|
const { visible, canvas } = event.detail;
|
||||||
|
// Toggle any sibling DOM your renderer mounts. The 3D Highway
|
||||||
|
// renderer hides its `.h3d-wrap` overlay here so `display:none`
|
||||||
|
// on `#highway` actually hides the visible output.
|
||||||
|
});
|
||||||
|
```
|
||||||
|
Renderers that only paint to the slopsmith canvas don't need this listener — the rAF skip is enough. Renderers that mount sibling DOM (separate WebGL contexts, overlays, etc.) do.
|
||||||
|
- **`highway.setVisible(bool | null)`** — forces the visibility state regardless of `offsetParent`. Pass `null` to clear the override and resume DOM-based detection. Useful when the host hides the highway via `visibility:hidden`, `opacity:0`, transforms, or clipping rather than `display:none`. The override re-emits any resulting transition immediately rather than waiting for the next rAF tick.
|
||||||
|
- Default-renderer ctx is closure-cached. The replace path nulls the closure ctx so stale draw paths short-circuit; the next default-renderer `init()` re-acquires the 2D context from the new canvas cleanly.
|
||||||
|
- `draw(bundle)` receives difficulty-filtered arrays — never read from `_filteredNotes` or other internals.
|
||||||
|
- `_drawHooks` fire for the default 2D renderer (the factory calls them at the end of each frame). Custom WebGL renderers that maintain a 2D overlay canvas (like the bundled 3D highway) also call `window.highway.fireDrawHooks(ctx, W, H)` on that overlay so overlay plugins continue to work regardless of which renderer is active. Custom renderers without a 2D overlay context should not attempt to fire hooks.
|
||||||
|
|
||||||
|
**Auto mode — `matchesArrangement(songInfo)` (optional).**
|
||||||
|
|
||||||
|
The viz picker prepends an "Auto (match arrangement)" entry that is the default selection on fresh installs. When Auto is active, core evaluates registered viz factories on every `song:ready` and swaps the renderer to the first factory whose `matchesArrangement(songInfo)` predicate returns truthy. No match → the built-in 2D highway.
|
||||||
|
|
||||||
|
Declare the predicate as a static on the factory (not the instance) so core can evaluate it without constructing a throwaway renderer:
|
||||||
|
|
||||||
|
```js
|
||||||
|
window.slopsmithViz_piano = function () { /* ... */ };
|
||||||
|
window.slopsmithViz_piano.matchesArrangement = function (songInfo) {
|
||||||
|
return /keys|piano|synth/i.test((songInfo && songInfo.arrangement) || '');
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
- `songInfo` is the highway's live song_info snapshot — `arrangement`, `tuning`, `capo`, `centOffset`, `arrangement_index`, `filename`, `artist`, `title`, etc. May be `{}` before the first song loads.
|
||||||
|
- Factories without `matchesArrangement` are skipped during auto-selection — the correct default for arrangement-agnostic viz (tabview, jumpingtab) that only make sense as manual picks.
|
||||||
|
- Explicit picker selections override Auto and are persisted to `localStorage.vizSelection`, so the pinned choice survives page reloads until the user switches back to "Auto" (which also persists). Picking "Auto" re-evaluates against the current song immediately. In contexts where `localStorage` is unavailable (private mode, sandboxed iframes, some test runners) persistence falls back to the current picker `<option>` value, which still overrides Auto for as long as the page stays loaded.
|
||||||
|
- When an Auto-selected renderer fails and core emits `viz:reverted`, the picker falls back to the built-in default and disables auto-switching until the user re-selects Auto.
|
||||||
|
- First match wins (picker order), so the registration order of plugins is the tiebreaker. Keep predicates narrow to avoid stealing songs from more specialized viz.
|
||||||
|
|
||||||
|
**WebGL viz in Auto mode.** Auto evaluation runs on every `song:ready` regardless of which renderer is active. Auto-installing a WebGL renderer when the canvas is currently 2D — or reverting from a WebGL Auto pick to the default 2D — works without a reload because `setRenderer` swaps the canvas element when `contextType` differs (see "Canvas context-type swapping" above). WebGL viz can therefore safely declare `matchesArrangement` and rely on Auto. For 3D Highway specifically, `_canRun3D()` in app.js still gates Auto from picking it on machines without WebGL2 — that fallback is independent of canvas swapping.
|
||||||
|
|
||||||
|
**Per-instance settings for host plugins (slopsmith#849).** A viz provider may declare per-instance controls a consuming host (e.g. splitscreen's per-panel popover) renders generically, by adding a `settings` array to its `capabilities.visualization` manifest block: `[{ key, label, type: "toggle" | "range" | "select", default, min?, max?, step?, options? }]`. This is the capability-native, declarative replacement for the ad-hoc `factory.panelControls` static. The validated list is surfaced through the visualization host's `list-providers` snapshot, so a host reads it without knowing the plugin. **A provider that declares `settings` MUST implement `applySetting(key, value)` on its renderer instance** — the host calls it on the specific per-panel instance, which is inherently per-panel (no canvas→panel resolution, no shared global localStorage keys). `getSetting(key)` is optional (the host falls back to the declared `default`); the host owns persistence. `factory.panelControls` remains read as a legacy fallback for hosts that still consume it, but new viz should declare `settings` + `applySetting`.
|
||||||
|
|
||||||
|
#### 2. Overlay contract — for add-on layers
|
||||||
|
|
||||||
|
Plugins that add a layer on top of whichever visualization is active — HUDs, fretboard diagrams, chord labels, practice feedback — don't replace the renderer. They manage their own canvas, their own rAF loop, and a toggle button somewhere visible (typically a navbar pill), reading public highway state via the getters:
|
||||||
|
|
||||||
|
- `highway.getTime()` / `highway.getBeats()` — current playback position
|
||||||
|
- `highway.getNotes()` / `highway.getChords()` — raw arrays containing every note/chord in the chart regardless of the current difficulty level
|
||||||
|
- `highway.getFilteredNotes()` / `highway.getFilteredChords()` — difficulty-filtered variants. Returns the master-difficulty-filtered arrays when the song has phrase-level data (slider active); falls through to the raw arrays for songs with a single difficulty level (slider disabled). Plugins that process only the notes the player is currently expected to play should use these instead of `getNotes()` / `getChords()`
|
||||||
|
- `highway.hasPhraseData()` — returns `true` when the current song has phrase-level difficulty ladder data (i.e. the mastery slider is active and `getFilteredNotes()` / `getFilteredChords()` return a filtered subset). Use this to gate logic that only makes sense when difficulty filtering is available
|
||||||
|
- `highway.getPhrases()` — phrase timing windows `[{ index, start_time, end_time, max_difficulty }]` for the current song's difficulty ladder. Returns `null` when phrase data is absent (GP imports, single-difficulty charts). Read-only; do not mutate. Pair with `hasPhraseData()` to gate phrase-aware logic.
|
||||||
|
- `highway.getMastery()` — current master-difficulty slider value as a fraction `0..1`. Reflects the same value the mastery slider is set to; meaningful only when `hasPhraseData()` is true.
|
||||||
|
- `highway.getChordTemplates()` — chord shape lookup table; index by `chord.id` from `getChords()` to get `{ name, fingers, frets }`. `fingers` and `frets` are per-string arrays (length matches the tuning's string count); within `fingers`, `-1` = unused, `0` = open string, `n > 0` = finger number. RS XML sources populate real fingerings; GP imports currently emit all `-1` since pre-import sources don't carry finger data. Not filter-aware: templates are static metadata, every `chord_id` referenced by `getChords()` is guaranteed valid
|
||||||
|
- `highway.getSongInfo()` — tuning, arrangement, capo
|
||||||
|
- `highway.getStringCount()` — number of strings on the active arrangement (4 for bass, 6 for guitar, 7+ for extended-range GP imports). Derived server-side as `max(notes-max-string + 1, name-based fallback, len(tuning))` where the tuning length only contributes when it isn't the RS-XML padded 6-string form (sloppak / GP-imported sources carry trimmed tuning lengths). The name-based fallback is 4 for arrangements containing "bass" (case-insensitive) and 6 otherwise. This combination handles partial-string-usage charts (a 6-string lead that never plays string 5), extended-range GP imports (5-string bass, 7-string guitar), and sloppaks that explicitly encode the instrument range — without requiring plugins to do their own arrangement-name matching
|
||||||
|
- `highway.getLefty()` / `highway.getInverted()` — mirror + invert state
|
||||||
|
|
||||||
|
Overlays do NOT appear in the viz picker and do NOT declare `"type": "visualization"` in `plugin.json`. They coexist with whichever renderer (default 2D, 3D highway, piano, ...) the user has picked.
|
||||||
|
|
||||||
|
**Key rules:**
|
||||||
|
- **Own your rAF + canvas** — don't piggyback on `_drawHooks` or on `createHighway`'s rendering context. Draw hooks fire for the default 2D renderer and for custom renderers that explicitly call `window.highway.fireDrawHooks(ctx, W, H)` (e.g. the bundled 3D highway fires them on its 2D overlay canvas), but not for every custom renderer.
|
||||||
|
- **Re-read state every frame** — overlay output must track whatever the current renderer is drawing. Don't cache note positions across frames.
|
||||||
|
- **Respect lefty + invert toggles** — if the overlay depicts strings or frets, mirror using the same transforms the active renderer would.
|
||||||
|
- **If you position with `highway.project` / `highway.fretX` (the 2D-highway geometry), gate on `highway.isDefaultRenderer()`** — those helpers describe the *built-in 2D* highway's depth curve and fret zoom. When a custom renderer (3D highway, piano, …) is active your draw hook still fires (on that renderer's 2D overlay layer), but those coordinates won't match its scene — markers land in arbitrary places. Skip rendering when `isDefaultRenderer()` is false; the custom renderer owns that feedback. Renderer-agnostic overlays (fretboard diagram, chord-label HUD — they use `getNotes()`/`getChordTemplates()` + their own layout) don't need this guard.
|
||||||
|
- **Clean up on toggle-off** — cancel rAF and remove/hide the overlay canvas so inactive overlays aren't wasting frames.
|
||||||
|
|
||||||
|
Reference: [fretboard plugin](https://github.com/byrongamatos/slopsmith-plugin-fretboard) — canonical overlay implementation (navbar toggle, own canvas, 80ms active-note window).
|
||||||
|
|
||||||
|
**Why two?** setRenderer plugs into an existing highway — main-player or splitscreen-panel — reusing its WebSocket and data parsing, so the common "I want a different look for the same data" case is zero boilerplate AND multi-instance for free. Overlays compose with whatever renderer is active — they decorate rather than replace, so multiple can stack (fretboard + chord labels + practice feedback) without fighting over the canvas.
|
||||||
|
|
||||||
|
A previous standalone-pane contract (`window.createMyVisualization({ container })` with its own WebSocket per pane) was used by splitscreen pre-Wave-C. It's been retired now that splitscreen calls `setRenderer` on per-panel `createHighway()` instances; if you find references in older plugin docs or external integration guides, those describe the legacy path.
|
||||||
|
|
||||||
|
#### 3. Note-state provider — for scorers that want renderers to "light up" notes (slopsmith#254)
|
||||||
|
|
||||||
|
A scoring plugin (note_detect) can publish a per-note judgment so whichever renderer is active draws the **gem itself** lit on a correct hit, and keeps a sustain trail glowing while it's still being played correctly — instead of a separate overlay ring floating near the note.
|
||||||
|
|
||||||
|
```js
|
||||||
|
// In the plugin (after resolving the highway instance):
|
||||||
|
highway.setNoteStateProvider((note, chartTime) => {
|
||||||
|
// `note` is the chart note object ({ t, s, f, sus, ... }); for chord
|
||||||
|
// notes `chartTime` is the chord's time. Return one of:
|
||||||
|
// - falsy → no special state (render normally)
|
||||||
|
// - 'hit' — struck correctly; renderer lights the gem
|
||||||
|
// - 'active' — a sustained note is right now being held correctly
|
||||||
|
// - 'miss' — missed; renderer may red-wash the gem
|
||||||
|
// - { state: <one of the above>, alpha?: 0..1, color?: '#rrggbb' }
|
||||||
|
// You own all fade timing: return a decaying `alpha` for a struck-note
|
||||||
|
// glow, `alpha: 1` (or a bare string) for a held sustain, and stop
|
||||||
|
// returning state when the effect should end. Keep it cheap — it's
|
||||||
|
// called per visible note per renderer per frame.
|
||||||
|
});
|
||||||
|
// On teardown: highway.setNoteStateProvider(null);
|
||||||
|
```
|
||||||
|
|
||||||
|
- Only one provider is active at a time (last `setNoteStateProvider` wins). `highway.getNoteStateProvider()` returns the current one (or null).
|
||||||
|
- The built-in 2D highway consults it in `drawNote` / `drawSustains` / the chord-frame path: 'hit'/'active' → bright string colour + additive halo + a contained "sizzle" (crackling sparks, throbbing core, a shockwave ring on a fresh strike) on the gem and a bright (vs dim) sustain trail; 'miss' → faint red wash. The bundled **3D highway** reads the same data via `bundle.getNoteState` (bright string-tinted outline + bright body + glowing sustain + a contained sparkle hugging the note rect on hit/active; red outline + suppressed body on miss). Custom renderers that want it call `bundle.getNoteState(note, chartTime)` — it null-guards and returns the normalized `{ state, alpha, color }` (or null).
|
||||||
|
- This is orthogonal to the overlay contract: note_detect remains an overlay (HUD, diagnostic miss markers, the "currently detected" indicator) *and* a scorer that feeds this provider. A renderer that ignores `getNoteState` simply doesn't light gems — nothing breaks.
|
||||||
|
|
||||||
|
### Audio mixer fader registration (slopsmith#87)
|
||||||
|
|
||||||
|
Plugins that produce audio outside the song's `<audio>` element (NAM amp output, synth voices, etc.) can register a labeled fader so users can balance them against the song from one mixer popover in the player controls.
|
||||||
|
|
||||||
|
```js
|
||||||
|
function _registerFader() {
|
||||||
|
const api = window.slopsmith && window.slopsmith.audio;
|
||||||
|
if (!api) return;
|
||||||
|
api.registerFader({
|
||||||
|
id: 'my_plugin', // unique key
|
||||||
|
label: 'My Plugin', // shown above the fader
|
||||||
|
unit: 'dB', // optional suffix shown next to the value (e.g. '%', 'dB')
|
||||||
|
min: 0, max: 2, step: 0.05,
|
||||||
|
defaultValue: 1.0,
|
||||||
|
getValue: () => _myCurrentVolume, // read current value
|
||||||
|
setValue: (v) => _setMyVolume(v), // write + persist + apply
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
if (window.slopsmith && window.slopsmith.audio) {
|
||||||
|
_registerFader();
|
||||||
|
} else {
|
||||||
|
window.addEventListener('slopsmith:audio:ready', _registerFader, { once: true });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The plugin owns persistence — the registry calls `getValue()` when the popover opens, and also after each `setValue()` during slider drags to re-sync the displayed value. Keep `getValue()` cheap and side-effect-free, and make sure `setValue()` updates whatever backing state `getValue()` reads synchronously. Pair `setValue` with whatever your plugin already does internally (write the GainNode, persist to localStorage, update any in-plugin label). Use `unregisterFader(id)` when your plugin is teardown-able and you want the strip to disappear; otherwise keep it registered so the user's setting persists across toggle states.
|
||||||
|
|
||||||
|
### Backend plugin logging
|
||||||
|
|
||||||
|
Use `context["log"]` for all backend plugin output. It is a stdlib `logging.Logger` namespaced to `slopsmith.plugin.<id>`, pre-configured with the app-wide level, format (including JSON mode), and correlation IDs. Never use `print()` — it bypasses correlation context and log rotation.
|
||||||
|
|
||||||
|
```python
|
||||||
|
def setup(app, context):
|
||||||
|
log = context["log"]
|
||||||
|
log.info("plugin ready")
|
||||||
|
log.warning("optional dependency %r not found, feature disabled", dep)
|
||||||
|
try:
|
||||||
|
risky_init()
|
||||||
|
except Exception:
|
||||||
|
log.exception("unhandled error during setup") # auto-captures traceback
|
||||||
|
```
|
||||||
|
|
||||||
|
For CLI entry points (scripts that also run as `__main__`), add a stdlib fallback so the logger works without the server pipeline:
|
||||||
|
|
||||||
|
```python
|
||||||
|
if __name__ == "__main__":
|
||||||
|
import logging
|
||||||
|
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Diagnostics contribution from frontend (slopsmith#166)
|
||||||
|
|
||||||
|
Plugins that hold useful debug state in the browser (active model name, last user input, internal counters) can push it into the diagnostics bundle by calling `window.slopsmith.diagnostics.contribute(plugin_id, payload)` at any time. The contribution API is idempotent — repeated calls overwrite the previous value. Whatever was last contributed before the user hits Export Diagnostics is what lands in `plugins/<plugin_id>/client.json`.
|
||||||
|
|
||||||
|
```js
|
||||||
|
window.slopsmith.diagnostics.contribute('my_plugin', {
|
||||||
|
schema: 'my_plugin.client_diag.v1',
|
||||||
|
active_preset: getActivePreset(),
|
||||||
|
last_error: _lastError,
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Loaded from `static/diagnostics.js` ASAP in `<head>` so the console-wrap is in place before any other script runs. Available on the `window.slopsmith.diagnostics` namespace alongside `snapshotConsole()`, `snapshotHardware()`, `snapshotUa()`, `snapshotLocalStorage()`, `snapshotContributions()`. Keep your payload small (< 100 KB) and don't include secrets — bundles are shared with maintainers.
|
||||||
|
|
||||||
|
### Keyboard Shortcuts
|
||||||
|
|
||||||
|
Plugins can register keyboard shortcuts via the global `window.registerShortcut()` function. Shortcuts appear in the `?` help panel.
|
||||||
|
|
||||||
|
```js
|
||||||
|
window.registerShortcut({
|
||||||
|
key: 'k', // key value (e.key) or key code (e.code)
|
||||||
|
description: 'Toggle my view', // shown in the help panel
|
||||||
|
scope: 'player', // 'global' | 'player' | 'library' | 'settings' | 'plugin-{id}'
|
||||||
|
condition: () => _isMyViewActive, // optional guard
|
||||||
|
handler: (e) => _myAction() // called when shortcut triggers
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Scope** controls when the shortcut is active:
|
||||||
|
- `global` — works on any screen
|
||||||
|
- `player` — only on the player screen
|
||||||
|
- `library` — only on the home/favorites screens
|
||||||
|
- `settings` — only on the settings screen
|
||||||
|
- `plugin-{id}` — only when your plugin's screen is active
|
||||||
|
|
||||||
|
**Panel-scoped shortcuts:** For plugins that create multiple panels (e.g., splitscreen), shortcuts are automatically scoped to the active panel. Use `const panel = window.createShortcutPanel(id)` to create a panel (it returns the panel object — keep the reference so you can call `panel.clearShortcuts()` during cleanup) and `window.setActiveShortcutPanel(id)` to switch between them. Each panel has its own shortcut registry, so multiple panels can have the same key without collisions.
|
||||||
|
|
||||||
|
**Condition** is an optional guard function. If it returns false, the shortcut is skipped even if in scope.
|
||||||
|
|
||||||
|
**Key matching:** The handler matches against both `e.key` (character produced) and `e.code` (physical key). Use `e.key` for letters/symbols that depend on keyboard layout, and `e.code` for special keys (e.g. `Space`, `ArrowLeft`).
|
||||||
|
|
||||||
|
**Built-in shortcuts:**
|
||||||
|
|
||||||
|
| Key | Description |
|
||||||
|
|-----|-------------|
|
||||||
|
| `?` | Show keyboard shortcuts panel (global) |
|
||||||
|
| `Space` | Play/Pause (player only) |
|
||||||
|
| `←` / `→` | Seek ±5 seconds (player only) |
|
||||||
|
| `Escape` | Back to library (player only) |
|
||||||
|
| `[` / `]` | Audio offset ±10ms (Shift: ±50ms) (player only) |
|
||||||
|
|
||||||
|
**Debugging:** Open browser console and type `_listShortcuts()` to inspect registered shortcuts.
|
||||||
|
|
||||||
|
### General plugin guidelines
|
||||||
|
|
||||||
|
- Wrap your plugin code in an IIFE: `(function () { 'use strict'; ... })();`
|
||||||
|
- Use `localStorage` for user-facing settings, prefixed with your plugin id
|
||||||
|
- If hooking `window.playSong`, always call the original and `await` it
|
||||||
|
- If hooking `window.showScreen`, clean up your state when leaving the player screen
|
||||||
|
- Use `window.slopsmith.emit()` / `window.slopsmith.on()` for inter-plugin communication
|
||||||
|
- Use `window.registerShortcut()` to add keyboard shortcuts. Clean up with `window.unregisterShortcut(key, scope)` — pass the same scope you registered with, since the default is `'global'` and won't match `player`/`library`/`settings`/`plugin-*` bindings. For panel-scoped shortcuts, prefer `panel.clearShortcuts()`.
|
||||||
|
|
||||||
|
## Song Formats
|
||||||
|
|
||||||
|
Slopsmith supports two song formats:
|
||||||
|
|
||||||
|
### Loose folder (XML charts)
|
||||||
|
A directory containing arrangement XML plus an audio file (and optional `manifest.json` + album art). Discovered, indexed, and played directly — see `lib/loosefolder.py`. Metadata follows a `manifest.json` → XML tags → folder-name priority chain. Songs are tagged `format: "loose"` in the library.
|
||||||
|
|
||||||
|
### Sloppak (open format)
|
||||||
|
An open, hand-editable song package designed for Slopsmith. Exists in two interchangeable forms:
|
||||||
|
- **Zip archive** (`.sloppak` file) — distribution form
|
||||||
|
- **Directory** (`.sloppak/` folder) — authoring form
|
||||||
|
|
||||||
|
**Contents:**
|
||||||
|
```
|
||||||
|
manifest.yaml Song metadata (title, artist, album, duration, tuning, arrangement IDs, ...)
|
||||||
|
arrangements/
|
||||||
|
lead.json Note/chord/anchor data in wire format (see song.py)
|
||||||
|
rhythm.json Files here are driven by manifest.yaml arrangement entries
|
||||||
|
... (e.g. arrangements/<arrangement-id>.json)
|
||||||
|
stems/
|
||||||
|
full.ogg Mixed audio (always present)
|
||||||
|
guitar.ogg Individual stems (optional, from Demucs split)
|
||||||
|
bass.ogg
|
||||||
|
drums.ogg
|
||||||
|
vocals.ogg
|
||||||
|
piano.ogg
|
||||||
|
other.ogg
|
||||||
|
cover.jpg Album art (optional)
|
||||||
|
lyrics.json Syllable-level lyrics (optional)
|
||||||
|
```
|
||||||
|
|
||||||
|
Sloppak is the preferred format for new features. The [Stems plugin](https://github.com/topkoa/slopsmith-plugin-stems) provides live stem mixing for sloppak songs.
|
||||||
|
|
||||||
|
**Full developer reference:** [docs/sloppak-spec.md](docs/sloppak-spec.md) — manifest schema, arrangement wire format, and how to extend the format with new data types (drum tab, key/scale annotations, etc.).
|
||||||
|
|
||||||
|
**Key code:**
|
||||||
|
- `lib/sloppak.py` — format detection, zip/directory resolution, metadata extraction, song loading
|
||||||
|
- `lib/sloppak_convert.py` — sloppak assembly pipeline, Demucs stem splitting
|
||||||
|
- `lib/song.py` — shared data models (`Note`, `Chord`, `Arrangement`, `Song`) and wire format serialization used by both formats
|
||||||
|
|
||||||
|
## Frontend Conventions
|
||||||
|
|
||||||
|
- **No frameworks** — vanilla JS, fetch API, DOM manipulation
|
||||||
|
- **Globals** — `highway`, `audio`, `playSong()`, `showScreen()`, `createHighway()`, `window.slopsmith`
|
||||||
|
- **Storage** — `localStorage` for all user preferences
|
||||||
|
- **Styling** — Tailwind CSS utility classes, dark theme (`bg-dark-600`, `text-gray-300`, accent `#4080e0`, gold `#e8c040`). Tailwind is served as a **prebuilt** stylesheet (`static/tailwind.min.css`, regenerated by `bash scripts/build-tailwind.sh`), **never** the runtime Play CDN — the CDN's on-the-fly JIT rescanned the DOM on the main thread and dropped ~26% of frames with the 3D highway (slopsmith-desktop#110). The committed CSS only contains classes the build scanner saw, so CI (`tailwind-fresh`) rebuilds and diffs it; run the build script and commit when you add new classes. A plugin that uses classes not guaranteed in core (notably arbitrary values like `w-[37px]`) MUST ship its own compiled stylesheet via the `styles` manifest key, built with `corePlugins.preflight = false` (utilities only — core ships the one base reset). Plugins MUST NOT load the Tailwind Play CDN or any runtime CSS JIT. See constitution Principle II.
|
||||||
|
- **Naming** — camelCase for JS functions, kebab-case for CSS classes, snake_case for plugin IDs
|
||||||
|
- **Player layout** — `#player` is `display:flex; flex-direction:column; position:fixed; inset:0`. `#highway` is `flex:1`. `#player-controls` sits at the bottom. Hiding the highway collapses the layout — use `margin-top: auto` on controls if you need to hide it.
|
||||||
|
|
||||||
|
## Backend Conventions
|
||||||
|
|
||||||
|
- **Framework** — FastAPI with uvicorn
|
||||||
|
- **Imports** — flat imports from `lib/` (no package `__init__.py`): `from song import Song`
|
||||||
|
- **Database** — SQLite via MetadataDB class with `threading.Lock` for thread safety
|
||||||
|
- **WebSocket** — JSON frames, try/except `WebSocketDisconnect`
|
||||||
|
- **Error handling** — graceful fallbacks (audio conversion errors don't crash the song, missing art returns placeholder)
|
||||||
|
- **Type hints** — used sparingly (`Path | None`, `dict`, `list`)
|
||||||
|
- **Docstrings** — minimal; code is self-documenting
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pytest # Run all tests
|
||||||
|
pytest tests/test_song.py -v # Specific file
|
||||||
|
pytest -k "round_trip" -v # Pattern match
|
||||||
|
```
|
||||||
|
|
||||||
|
- Framework: pytest
|
||||||
|
- Config: `pyproject.toml` sets `pythonpath = [".", "lib"]` and `testpaths = ["tests"]`
|
||||||
|
- CI: GitHub Actions runs pytest on push/PR to main (Python 3.12)
|
||||||
|
- Test dependencies: `requirements-test.txt`
|
||||||
|
|
||||||
|
## Tuning the note_detect plugin
|
||||||
|
|
||||||
|
Detection quality is hard to judge by eye — a player UI that "feels worse" after a code change isn't a regression you can defend in review. The plugin ships with a record-replay-sweep workflow so changes to the detector, the matcher, or the user's environment (A/V offset, latency, channel) can be measured against a single reference take.
|
||||||
|
|
||||||
|
Quick orientation:
|
||||||
|
- **Reference recording** lives in the gear popover on the player (gated behind Settings → Note Detection → "Detection tuning (advanced)"). Arm before pressing Play; auto-saves a WAV to `static/note_detect_recordings/` on song-end. The directory is bind-mounted, so the host-side harness can read it without a copy step.
|
||||||
|
- **Benchmark sloppak** ships in-tree at [docs/benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak](docs/benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak) — 8 sections each isolating a different failure mode (low-freq mono, sustained holds, hammer/pull, power chords, dense open chords, bends). Drop it directly into your sloppak DLC folder to install (don't rename — slopsmith keys off the `.sloppak` suffix even though the file is a zip under the hood). The unzipped form lands at `static/sloppak_cache/note_detect_benchmark_v1.sloppak/` after first play. Builder: [docs/benchmarks/note_detect_v1/build_benchmark.py](docs/benchmarks/note_detect_v1/build_benchmark.py).
|
||||||
|
- **Headless harness** at [`tools/harness.js`](https://github.com/byrongamatos/slopsmith-plugin-notedetect/blob/main/tools/harness.js) in the note_detect plugin's own repo (cloned into `plugins/note_detect/` locally) runs the same `processFrame` / `matchNotes` / `checkMisses` code path off Node, in seconds per run. Same `note_detect.diagnostic.v1` schema as the in-app Download Diagnostic button.
|
||||||
|
- **A/V auto-calibrate** (Settings → Note Detection) reads `timing_error_ms_hits.median` and proposes the av-offset that drives it to zero. Iterative: usually converges in 2–3 Apply rounds.
|
||||||
|
|
||||||
|
**Always record at 1.0× playback speed** — half-speed takes produce all-miss garbage because chart times are absolute. **Always use `timing_error_ms_hits` (not all-matched) as a calibration signal** — the all-matched median pins near a constant when the offset is wrong, because the matcher silently snaps to neighbouring chart notes.
|
||||||
|
|
||||||
|
Full developer reference (workflow recipes, harness flag table, diagnostic schema, common pitfalls): [docs/note-detect-tuning.md](docs/note-detect-tuning.md).
|
||||||
|
|
||||||
|
## Versioning
|
||||||
|
|
||||||
|
- **`VERSION`** (repo root) — single source of truth; plain semver string (e.g. `0.2.4`). Bind-mounted into the container and copied by the Dockerfile so it's always available at `/app/VERSION`.
|
||||||
|
- **`GET /api/version`** — returns `{"version": "<contents of VERSION>", "source_url": "...", "license_url": "..."}`. The version drives the navbar badge; `source_url` / `license_url` populate the Settings → About links. `source_url` is configurable via the `APP_SOURCE_URL` env var (default `https://github.com/byrongamatos/slopsmith`); `license_url` falls back to `source_url + "/blob/main/LICENSE"` (GitHub-style, default branch `main`) and is overridable via the `APP_LICENSE_URL` env var — set it explicitly when the source is hosted on a non-GitHub forge (GitLab/Gitea/self-hosted) or under a non-`main` default branch. Both env values must be `http(s)`; non-http(s) values are rejected and fall back to the safe default to prevent `javascript:`/`data:` hrefs.
|
||||||
|
- **Auto-sync** — `.github/workflows/sync-version.yml` rewrites `VERSION` via a `repository_dispatch` (`desktop-released`) fired from `slopsmith-desktop`'s release job. As an explicit automation-only exception to the "Never push directly to main" rule in Git Workflow below, the sync job commits straight to `main` as `github-actions[bot]` (version bumps are mechanical; the PR round-trip adds no signal). Human contributors must still go through feature branches + PRs. No manual VERSION edits needed. Use the workflow's `workflow_dispatch` trigger with `version: X.Y.Z` for manual runs (recovery / out-of-band bumps).
|
||||||
|
- **`CHANGELOG.md`** — follows [Keep a Changelog](https://keepachangelog.com/) format. Update the `[Unreleased]` section with each PR; when `slopsmith-desktop` cuts a release, rename `[Unreleased]` to the new version + date (the VERSION bump itself is automated).
|
||||||
|
|
||||||
|
## Git Workflow
|
||||||
|
|
||||||
|
- **Never push directly to main** — always create a feature branch and open a PR
|
||||||
|
- **Upstream remote** — set `upstream` to the canonical Slopsmith repository; `origin` is your fork
|
||||||
|
- **Plugins are gitlinks** — each plugin in `plugins/` is typically its own git repo (submodule or clone). Branch switches on the main repo can clobber plugin directories. Use `git update-index --assume-unchanged` for plugin dirs if needed.
|
||||||
|
- **Commit style** — short imperative subject line, blank line, then body explaining *why*
|
||||||
|
|
||||||
|
## WebSocket Protocol Reference
|
||||||
|
|
||||||
|
The highway WebSocket at `/ws/highway/{filename}?arrangement={index}` streams these messages in order:
|
||||||
|
|
||||||
|
| Message | Shape | Description |
|
||||||
|
|---------|-------|-------------|
|
||||||
|
| `loading` | `{ type: 'loading', stage }` | Status/progress message during extraction or conversion |
|
||||||
|
| `song_info` | `{ type, title, artist, arrangement, arrangement_index, arrangements, duration, tuning, capo, centOffset, format, audio_url, audio_error, stems }` | Song metadata. `arrangements` is the full list for the switcher. `audio_url` is `null` when audio is unavailable, in which case `audio_error` is non-null; otherwise `audio_error` is `null`. `stems` is always present — an empty array for non-sloppak songs or sloppak songs with no split stems. `tuning` is an array (6 for guitar, 4 for bass). `centOffset` is a float (cents) from the RS2014 `<centOffset>` field — commonly `-1200.0` for extended-range bass (one octave down), small non-zero values for true-tuned content (e.g. A443 ≈ +11.8 cents), `0.0` when absent. Available via `getSongInfo().centOffset`. |
|
||||||
|
| `beats` | `{ type, data: [{ time, measure }] }` | Beat timestamps with measure numbers |
|
||||||
|
| `sections` | `{ type, data: [{ time, name }] }` | Named sections (Intro, Verse, Chorus, etc.) |
|
||||||
|
| `anchors` | `{ type, data: [{ time, fret, width }] }` | Fret zoom anchors |
|
||||||
|
| `chord_templates` | `{ type, data: [{ name, frets: [6] }] }` | Named chord shapes |
|
||||||
|
| `lyrics` | `{ type, data: [{ w, t, d }], source }` | Syllables: `w`=word, `t`=time, `d`=duration. `-` joins to previous, `+` = line break. `source` is one of `"xml"`, `"whisperx"`, `"user"` — UI can use it to render an "auto-transcribed" badge for `whisperx`. Sloppaks always include `source` (legacy sloppaks without a `lyrics_source` manifest key default to `"xml"` at load time). Loose folders set it based on which extractor matched. Absent only when no lyrics fired the message at all |
|
||||||
|
| `tone_changes` | `{ type: 'tone_changes', base, data: [{ time, name }] }` | Optional — tone change events relative to the arrangement base tone; only sent if tones were found |
|
||||||
|
| `notes` | `{ type, data: [{ t, s, f, sus, ho, po, sl, bn, ... }] }` | Single notes |
|
||||||
|
| `chords` | `{ type, data: [{ t, notes: [{ s, f, sus, ... }] }] }` | Chord events |
|
||||||
|
| `phrases` | `{ type, data: [{ start_time, end_time, max_difficulty, levels: [{ difficulty, notes, chords, anchors, handshapes }] }], total }` | Optional — per-phrase difficulty ladder for master-difficulty slider (slopsmith#48). Only sent when the source chart carries multi-level phrase data (phrase-aware sloppak). Sent in chunks (`data` is a batch, `total` is the full count across messages) to avoid multi-MB single frames. Absent for GP imports and legacy sloppak; consumers must treat missing message as "single fixed difficulty — slider disabled". |
|
||||||
|
| `ready` | `{ type: 'ready' }` | All data sent — safe to finalize and start rendering |
|
||||||
|
|
||||||
|
Message delivery is incremental. You may receive `loading` updates and `lyrics` before note/chord payloads; `tone_changes` comes after `lyrics` when present and may be omitted entirely. Do not finalize rendering until you receive `ready`.
|
||||||
|
|
||||||
|
## Common Pitfalls
|
||||||
|
|
||||||
|
1. **playSong wrapper race condition** — The wrapper chain runs outermost-first (last-loaded wrapper runs first). If an inner plugin (e.g. `3dhighway`) does `await import(CDN)`, it yields to the event loop. WebSocket messages (`song_info`, `ready`) can arrive before outer plugins set their callbacks. Use `getSongInfo()` as a fallback rather than relying solely on `_onReady`.
|
||||||
|
|
||||||
|
2. **Plugin gitlinks** — Plugins are separate git repos cloned into `plugins/`. Switching branches on the main repo can delete or clobber these directories. Be careful with `git checkout` and `git clean`.
|
||||||
|
|
||||||
|
3. **Highway flex layout** — `#highway` has `flex:1` in the player. Hiding it with `display:none` removes the flex child, causing `#player-controls` to float to the top. If you hide the highway, add `margin-top: auto` to the controls div to keep it at the bottom.
|
||||||
|
|
||||||
|
4. **Multiple WebSocket connections** — The server supports many simultaneous WebSocket connections to the same song. Split screen panels, lyrics panes, and jumping tab panes each open their own. This is by design — don't try to multiplex.
|
||||||
|
|
||||||
|
5. **Plugin load order** — Plugins load alphabetically by directory name. This determines the `playSong` wrapper chain order and which plugin's UI elements appear first. If your plugin depends on another's globals, check at runtime (`typeof window.X === 'function'`), not at load time.
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# Contributing to Slopsmith
|
||||||
|
|
||||||
|
Thanks for wanting to contribute! This document covers the legal and workflow expectations for code, plugins, and documentation contributions.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Slopsmith is licensed under [AGPL-3.0-only](LICENSE). Contributions you submit (PRs, patches, documentation, plugin entries in the curated list) are licensed inbound under the same terms — **inbound = outbound**. By opening a pull request, you agree that your contribution may be distributed under AGPL-3.0-only as part of Slopsmith.
|
||||||
|
|
||||||
|
## Developer Certificate of Origin (DCO)
|
||||||
|
|
||||||
|
We use the [Developer Certificate of Origin](https://developercertificate.org/) (DCO) to track contribution provenance. Every commit must be signed off:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git commit -s -m "your commit message"
|
||||||
|
```
|
||||||
|
|
||||||
|
This appends a line to your commit message like:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Signed-off-by: Jane Developer <jane@example.com>
|
||||||
|
```
|
||||||
|
|
||||||
|
The sign-off certifies that you wrote the code (or have the right to submit it) and that you're contributing it under AGPL-3.0-only as the LICENSE file describes. The full text of the certification is at [developercertificate.org](https://developercertificate.org/).
|
||||||
|
|
||||||
|
If you forget to sign off, amend the most recent commit with `git commit --amend -s` (or rebase + sign off older commits) and force-push to your PR branch.
|
||||||
|
|
||||||
|
## Plugin licensing
|
||||||
|
|
||||||
|
Plugins live in their own repositories and are loaded at runtime — see the [Plugin System section in CLAUDE.md](CLAUDE.md) for the technical contract, and [Plugin Best Practices](CLAUDE.md) for the conventions every plugin should follow (v2/v3 player chrome, the visualization contracts, and the **performance rules** — no per-frame DOM queries or broad `document.body` `MutationObserver`s — that keep the 60 fps highway smooth). Plugins are not subject to AGPL by being loaded into Slopsmith (the loader runs them as separate Python modules / browser scripts), but for the **curated plugin list** to accept your plugin we ask that it be released under an AGPL-3.0-compatible license:
|
||||||
|
|
||||||
|
- AGPL-3.0-only or AGPL-3.0-or-later
|
||||||
|
- GPL-3.0-only or GPL-3.0-or-later
|
||||||
|
- LGPL-3.0-only or LGPL-3.0-or-later
|
||||||
|
- MIT
|
||||||
|
- BSD-2-Clause or BSD-3-Clause
|
||||||
|
- Apache-2.0
|
||||||
|
- ISC
|
||||||
|
- Unlicense / CC0-1.0 / 0BSD
|
||||||
|
|
||||||
|
Plugins under GPL-2.0-only, LGPL-2.1-only, CDDL, EPL, or proprietary terms will not be added to the curated list. You're still free to publish and self-distribute them — Slopsmith will load any plugin a user installs locally — but they won't be promoted from the main project.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
Standard PR workflow described in [CLAUDE.md → Git Workflow](CLAUDE.md):
|
||||||
|
- Never push directly to `main`.
|
||||||
|
- Create a feature branch on your fork.
|
||||||
|
- Open a PR against `byrongamatos/slopsmith:main`.
|
||||||
|
- Keep commits scoped and well-described; short imperative subject + `Signed-off-by` trailer.
|
||||||
|
|
||||||
|
## Questions
|
||||||
|
|
||||||
|
Open an issue or start a [Discussion](https://github.com/byrongamatos/slopsmith/discussions) if you're unsure whether a contribution fits — much better to ask early than to find out after the work is done.
|
||||||
+226
@@ -0,0 +1,226 @@
|
|||||||
|
# ── Stage 1b: Build native vgmstream-cli for target arch ─────────────────────
|
||||||
|
FROM python:3.12-slim AS vgmstream-builder
|
||||||
|
ARG VGMSTREAM_REF=r2083
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
build-essential \
|
||||||
|
cmake \
|
||||||
|
pkg-config \
|
||||||
|
yasm \
|
||||||
|
libmpg123-dev \
|
||||||
|
libvorbis-dev \
|
||||||
|
libspeex-dev \
|
||||||
|
libopus-dev \
|
||||||
|
git \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN git clone --depth 1 --branch "${VGMSTREAM_REF}" https://github.com/vgmstream/vgmstream.git /tmp/vgmstream
|
||||||
|
|
||||||
|
RUN cmake -S /tmp/vgmstream -B /tmp/vgmstream/build \
|
||||||
|
-DCMAKE_BUILD_TYPE=Release \
|
||||||
|
-DBUILD_V123=OFF \
|
||||||
|
-DBUILD_AUDACIOUS=OFF \
|
||||||
|
-DBUILD_SHARED_LIBS=OFF \
|
||||||
|
-DUSE_FFMPEG=OFF \
|
||||||
|
&& cmake --build /tmp/vgmstream/build --config Release --target vgmstream_cli -j"$(nproc)" \
|
||||||
|
&& mkdir -p /out \
|
||||||
|
&& cp /tmp/vgmstream/build/cli/vgmstream-cli /out/vgmstream-cli
|
||||||
|
|
||||||
|
# ── Stage 1c: Fetch static ffmpeg ─────────────────────────────────────────
|
||||||
|
# Throwaway stage — only the ffmpeg/ffprobe binaries cross into the final
|
||||||
|
# image via COPY. Doing the download here (rather than in stage 2) means
|
||||||
|
# the final image never has to install curl, which transitively pulls in
|
||||||
|
# libcurl4t64 → librtmp1 → libgnutls30t64 (and therefore gnutls28 with
|
||||||
|
# its unfixed HIGH CVEs). Alpine is used because it's tiny and the
|
||||||
|
# download tools don't need any of Debian's TLS baggage.
|
||||||
|
#
|
||||||
|
# Source: BtbN/FFmpeg-Builds (GPL static build, 7.1 series).
|
||||||
|
# BtbN publishes dated release tags (autobuild-YYYY-MM-DD-HH-MM) that
|
||||||
|
# yield immutable URLs — the versioned tarballs never disappear, unlike
|
||||||
|
# JVS rolling releases. Includes libvorbis (confirmed --enable-libvorbis
|
||||||
|
# in the configure line), so Sloppak's .ogg output path is unaffected.
|
||||||
|
#
|
||||||
|
# To bump: pick a new autobuild-* tag from
|
||||||
|
# https://github.com/BtbN/FFmpeg-Builds/releases
|
||||||
|
# download the two linux gpl-7.1 tarballs, re-run
|
||||||
|
# sha256sum ffmpeg-*-linux{64,arm64}-gpl-7.1.tar.xz
|
||||||
|
# and update FFMPEG_RELEASE + both SHA256 ARGs below.
|
||||||
|
FROM alpine:3.20 AS ffmpeg-fetcher
|
||||||
|
ARG TARGETARCH
|
||||||
|
ARG FFMPEG_RELEASE=autobuild-2026-06-01-15-02
|
||||||
|
ARG FFMPEG_BUILD_AMD64=ffmpeg-n7.1.4-7-gadcf20da26-linux64-gpl-7.1.tar.xz
|
||||||
|
ARG FFMPEG_BUILD_ARM64=ffmpeg-n7.1.4-7-gadcf20da26-linuxarm64-gpl-7.1.tar.xz
|
||||||
|
ARG FFMPEG_SHA256_AMD64=afde55344990650c117fbb7cb36b38d2ab6790b06beb06a9c43a9300c9ce277a
|
||||||
|
ARG FFMPEG_SHA256_ARM64=03c8a7d9a7cf48d017a22a7c31acfdc8e76c5cb193923f883b0338c7baf0bd28
|
||||||
|
RUN apk add --no-cache curl xz \
|
||||||
|
&& arch="${TARGETARCH:-$(apk --print-arch)}" \
|
||||||
|
&& case "$arch" in \
|
||||||
|
arm64|aarch64) FFMPEG_TARBALL="${FFMPEG_BUILD_ARM64}"; FFMPEG_SHA256="${FFMPEG_SHA256_ARM64}" ;; \
|
||||||
|
amd64|x86_64) FFMPEG_TARBALL="${FFMPEG_BUILD_AMD64}"; FFMPEG_SHA256="${FFMPEG_SHA256_AMD64}" ;; \
|
||||||
|
*) echo "Unsupported arch: $arch" >&2; exit 1 ;; \
|
||||||
|
esac \
|
||||||
|
&& curl -fsSL "https://github.com/BtbN/FFmpeg-Builds/releases/download/${FFMPEG_RELEASE}/${FFMPEG_TARBALL}" -o /tmp/ffmpeg.tar.xz \
|
||||||
|
&& echo "${FFMPEG_SHA256} /tmp/ffmpeg.tar.xz" | sha256sum -c - \
|
||||||
|
&& mkdir -p /tmp/ffmpeg-extract /out \
|
||||||
|
&& tar -xJf /tmp/ffmpeg.tar.xz -C /tmp/ffmpeg-extract --strip-components=1 \
|
||||||
|
&& cp /tmp/ffmpeg-extract/bin/ffmpeg /tmp/ffmpeg-extract/bin/ffprobe /out/ \
|
||||||
|
&& cp /tmp/ffmpeg-extract/LICENSE.txt /out/LICENSE.txt \
|
||||||
|
&& rm -rf /tmp/ffmpeg-extract /tmp/ffmpeg.tar.xz
|
||||||
|
|
||||||
|
# ── Stage 1d: Build the Tailwind stylesheet over the FULL plugin set ──────
|
||||||
|
# The committed static/tailwind.min.css is generated against only the in-tree
|
||||||
|
# plugins. Rather than ship it as-is (leaving baked-in plugins' classes
|
||||||
|
# unstyled now that the Play CDN's runtime JIT is gone — slopsmith#411),
|
||||||
|
# rebuild it here, after static/ + plugins/ are present, so the sheet covers
|
||||||
|
# whatever plugins are baked into the image. Runs in a throwaway node stage so
|
||||||
|
# this build-time toolchain never lands in the final image; the runtime node
|
||||||
|
# added later in the final stage (for on-install regeneration) is a separate,
|
||||||
|
# deliberate inclusion. Reuses the repo's tailwind.config.js (theme, safelist,
|
||||||
|
# highway_3d exclusion) for parity with scripts/build-tailwind.sh.
|
||||||
|
# (Runtime-installed plugins are handled separately by the server's rebuild.)
|
||||||
|
FROM node:22-slim AS tailwind-builder
|
||||||
|
WORKDIR /build
|
||||||
|
COPY tailwind.config.js ./
|
||||||
|
COPY static/ ./static/
|
||||||
|
COPY plugins/ ./plugins/
|
||||||
|
RUN npx -y tailwindcss@3.4.19 \
|
||||||
|
-c tailwind.config.js \
|
||||||
|
-i static/_tailwind.src.css \
|
||||||
|
-o static/tailwind.min.css \
|
||||||
|
--minify
|
||||||
|
|
||||||
|
# ── Stage 2: Final image ────────────────────────────────────────────────
|
||||||
|
FROM python:3.12-slim
|
||||||
|
# Re-declare the ffmpeg ARGs so their values are available to LABEL below.
|
||||||
|
# ARG values don't cross stage boundaries in multi-stage builds; defaults
|
||||||
|
# must be repeated here to take effect when no --build-arg is supplied.
|
||||||
|
ARG FFMPEG_RELEASE=autobuild-2026-06-01-15-02
|
||||||
|
ARG FFMPEG_BUILD_AMD64=ffmpeg-n7.1.4-7-gadcf20da26-linux64-gpl-7.1.tar.xz
|
||||||
|
ARG FFMPEG_BUILD_ARM64=ffmpeg-n7.1.4-7-gadcf20da26-linuxarm64-gpl-7.1.tar.xz
|
||||||
|
|
||||||
|
# Apply latest security updates to base packages (clears glibc deb13u3 and
|
||||||
|
# similar). Done first so any subsequent installs resolve against the
|
||||||
|
# patched versions rather than the stale ones baked into the base image.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get -y upgrade \
|
||||||
|
&& apt-get clean \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*
|
||||||
|
|
||||||
|
# Runtime packages.
|
||||||
|
#
|
||||||
|
# NOTE: ffmpeg is intentionally NOT installed via apt. The apt `ffmpeg`
|
||||||
|
# package drags in the full codec + TLS + graphics dependency tree
|
||||||
|
# (mbedtls, gnutls28, mesa, x264, tiff, openjpeg2, libcaca, harfbuzz,
|
||||||
|
# cairo, openldap, libcdio…), almost all of which has unfixed CVEs and
|
||||||
|
# none of which Slopsmith uses. We pull a static ffmpeg binary further
|
||||||
|
# down instead.
|
||||||
|
#
|
||||||
|
# vgmstream-cli is also built with -DUSE_FFMPEG=OFF (see stage 1b), so
|
||||||
|
# we don't need the libav* runtime libraries either — the Wwise Vorbis
|
||||||
|
# audio streams vgmstream handles are decoded natively. Dropping
|
||||||
|
# libav* also drops their transitive deps on mbedtls and gnutls28.
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
fluidsynth \
|
||||||
|
fluid-soundfont-gm \
|
||||||
|
libsndfile1 \
|
||||||
|
# Runtime shared libraries for the natively-built vgmstream-cli.
|
||||||
|
# `BUILD_SHARED_LIBS=OFF` in the builder stage only static-links
|
||||||
|
# vgmstream's own libs; the external codec dependencies it linked
|
||||||
|
# against (mpg123, vorbis, speex, opus) are still dynamic and need
|
||||||
|
# their runtime packages here.
|
||||||
|
libmpg123-0 \
|
||||||
|
libvorbisfile3 \
|
||||||
|
libspeex1 \
|
||||||
|
libopus0 \
|
||||||
|
# Shared libs the copied-in `node` binary links against. Normally present
|
||||||
|
# transitively, but install explicitly so the runtime regeneration path
|
||||||
|
# can't break with a dynamic-linker error if a future base/dep change stops
|
||||||
|
# pulling them in.
|
||||||
|
libstdc++6 \
|
||||||
|
libgcc-s1 \
|
||||||
|
&& apt-get clean \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*
|
||||||
|
|
||||||
|
# Node + the pinned Tailwind CLI for RUNTIME stylesheet regeneration. When a
|
||||||
|
# plugin is installed into SLOPSMITH_PLUGINS_DIR at runtime (or discovered
|
||||||
|
# there on startup), the server rebuilds static/tailwind.min.css so the
|
||||||
|
# plugin's classes are styled — the image-baked sheet only covered in-tree
|
||||||
|
# plugins (see lib/tailwind_rebuild.py). tailwindcss is installed globally so
|
||||||
|
# the rebuild runs offline, with no npx fetch at install time. node/npm are
|
||||||
|
# copied from the existing tailwind-builder stage (same node:20-slim base) to
|
||||||
|
# avoid pulling that image a second time.
|
||||||
|
COPY --from=tailwind-builder /usr/local/bin/node /usr/local/bin/node
|
||||||
|
COPY --from=tailwind-builder /usr/local/lib/node_modules/npm /usr/local/lib/node_modules/npm
|
||||||
|
RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm \
|
||||||
|
&& ln -sf /usr/local/lib/node_modules/npm/bin/npx-cli.js /usr/local/bin/npx \
|
||||||
|
&& npm install -g tailwindcss@3.4.19 \
|
||||||
|
&& npm cache clean --force
|
||||||
|
|
||||||
|
# Static ffmpeg + ffprobe binaries from the throwaway fetcher stage above.
|
||||||
|
# BtbN GPL builds statically link their codec deps and don't pull in
|
||||||
|
# GnuTLS/mbedTLS, mesa, x264, cairo, etc. No CVE surface from the system
|
||||||
|
# codec stack; ~80 MB on disk.
|
||||||
|
#
|
||||||
|
# NOTE (GPL): the static ffmpeg binary is licensed under GPL v2+.
|
||||||
|
# LICENSE.txt from the BtbN tarball is copied into /usr/share/doc/ffmpeg/
|
||||||
|
# so the license text is present in the runtime image.
|
||||||
|
#
|
||||||
|
# If this image is redistributed publicly, the GPL requires that the
|
||||||
|
# Corresponding Source for this ffmpeg build also be made available.
|
||||||
|
# BtbN publishes full build configuration and source references at:
|
||||||
|
# https://github.com/BtbN/FFmpeg-Builds (tag: FFMPEG_RELEASE ARG)
|
||||||
|
# Ensure your redistribution method meets GPL conveyance requirements —
|
||||||
|
# either by pointing recipients to BtbN's source or by hosting it yourself.
|
||||||
|
COPY --from=ffmpeg-fetcher /out/ffmpeg /out/ffprobe /usr/local/bin/
|
||||||
|
COPY --from=ffmpeg-fetcher /out/LICENSE.txt /usr/share/doc/ffmpeg/LICENSE.txt
|
||||||
|
RUN chmod +x /usr/local/bin/ffmpeg /usr/local/bin/ffprobe
|
||||||
|
# Record provenance so the exact BtbN source can be located for GPL compliance
|
||||||
|
# or debugging. Inspect with: docker inspect <image> | grep -A5 ffmpeg
|
||||||
|
LABEL org.slopsmith.ffmpeg.release="${FFMPEG_RELEASE}" \
|
||||||
|
org.slopsmith.ffmpeg.source.amd64="${FFMPEG_BUILD_AMD64}" \
|
||||||
|
org.slopsmith.ffmpeg.source.arm64="${FFMPEG_BUILD_ARM64}" \
|
||||||
|
org.slopsmith.ffmpeg.upstream="https://github.com/BtbN/FFmpeg-Builds"
|
||||||
|
|
||||||
|
# Native vgmstream-cli built against the image's own libraries
|
||||||
|
COPY --from=vgmstream-builder /out/vgmstream-cli /usr/local/bin/vgmstream-cli
|
||||||
|
RUN chmod +x /usr/local/bin/vgmstream-cli
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Upgrade pip itself before installing requirements — clears the pip CVEs
|
||||||
|
# (CVE-2025-8869, CVE-2026-6357, CVE-2026-1703) that ship with the base.
|
||||||
|
# Pinned for reproducibility; bump PIP_VERSION when a newer release is needed.
|
||||||
|
ARG PIP_VERSION=26.1.1
|
||||||
|
RUN pip install --no-cache-dir "pip==${PIP_VERSION}"
|
||||||
|
|
||||||
|
COPY requirements.txt .
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
COPY lib/ /app/lib/
|
||||||
|
COPY static/ /app/static/
|
||||||
|
COPY plugins/ /app/plugins/
|
||||||
|
COPY data/ /app/data/
|
||||||
|
# Replace the committed sheet with the bundled-plugin-aware build from stage 1d.
|
||||||
|
COPY --from=tailwind-builder /build/static/tailwind.min.css /app/static/tailwind.min.css
|
||||||
|
# tailwind.config.js + _tailwind.src.css let the server regenerate the sheet
|
||||||
|
# when a plugin is installed at runtime (see update_manager on-install hook).
|
||||||
|
COPY tailwind.config.js /app/tailwind.config.js
|
||||||
|
COPY server.py /app/
|
||||||
|
COPY main.py /app/
|
||||||
|
COPY VERSION /app/
|
||||||
|
# Built-in diagnostic sloppaks seeded into DLC_DIR/diagnostics-builtin/ at scan
|
||||||
|
# time (_seed_builtin_diagnostic_sloppaks in server.py). server.py resolves the
|
||||||
|
# source relative to its own dir, so it must live under /app/docs/diagnostics/.
|
||||||
|
# Only the .sloppak artifacts are needed at runtime — not the builder script.
|
||||||
|
COPY docs/diagnostics/*.sloppak /app/docs/diagnostics/
|
||||||
|
|
||||||
|
ENV PYTHONPATH=/app/lib:/app
|
||||||
|
|
||||||
|
EXPOSE 8000
|
||||||
|
|
||||||
|
# main.py calls configure_logging() before uvicorn.run(..., log_config=None),
|
||||||
|
# which prevents uvicorn from applying its default dictConfig. This ensures
|
||||||
|
# the structlog pipeline is active for ALL uvicorn messages — including the
|
||||||
|
# early lifecycle lines ("Started server process", "Waiting for application
|
||||||
|
# startup") that fire before the ASGI startup hook.
|
||||||
|
CMD python main.py
|
||||||
@@ -0,0 +1,661 @@
|
|||||||
|
GNU AFFERO GENERAL PUBLIC LICENSE
|
||||||
|
Version 3, 19 November 2007
|
||||||
|
|
||||||
|
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||||
|
Everyone is permitted to copy and distribute verbatim copies
|
||||||
|
of this license document, but changing it is not allowed.
|
||||||
|
|
||||||
|
Preamble
|
||||||
|
|
||||||
|
The GNU Affero General Public License is a free, copyleft license for
|
||||||
|
software and other kinds of works, specifically designed to ensure
|
||||||
|
cooperation with the community in the case of network server software.
|
||||||
|
|
||||||
|
The licenses for most software and other practical works are designed
|
||||||
|
to take away your freedom to share and change the works. By contrast,
|
||||||
|
our General Public Licenses are intended to guarantee your freedom to
|
||||||
|
share and change all versions of a program--to make sure it remains free
|
||||||
|
software for all its users.
|
||||||
|
|
||||||
|
When we speak of free software, we are referring to freedom, not
|
||||||
|
price. Our General Public Licenses are designed to make sure that you
|
||||||
|
have the freedom to distribute copies of free software (and charge for
|
||||||
|
them if you wish), that you receive source code or can get it if you
|
||||||
|
want it, that you can change the software or use pieces of it in new
|
||||||
|
free programs, and that you know you can do these things.
|
||||||
|
|
||||||
|
Developers that use our General Public Licenses protect your rights
|
||||||
|
with two steps: (1) assert copyright on the software, and (2) offer
|
||||||
|
you this License which gives you legal permission to copy, distribute
|
||||||
|
and/or modify the software.
|
||||||
|
|
||||||
|
A secondary benefit of defending all users' freedom is that
|
||||||
|
improvements made in alternate versions of the program, if they
|
||||||
|
receive widespread use, become available for other developers to
|
||||||
|
incorporate. Many developers of free software are heartened and
|
||||||
|
encouraged by the resulting cooperation. However, in the case of
|
||||||
|
software used on network servers, this result may fail to come about.
|
||||||
|
The GNU General Public License permits making a modified version and
|
||||||
|
letting the public access it on a server without ever releasing its
|
||||||
|
source code to the public.
|
||||||
|
|
||||||
|
The GNU Affero General Public License is designed specifically to
|
||||||
|
ensure that, in such cases, the modified source code becomes available
|
||||||
|
to the community. It requires the operator of a network server to
|
||||||
|
provide the source code of the modified version running there to the
|
||||||
|
users of that server. Therefore, public use of a modified version, on
|
||||||
|
a publicly accessible server, gives the public access to the source
|
||||||
|
code of the modified version.
|
||||||
|
|
||||||
|
An older license, called the Affero General Public License and
|
||||||
|
published by Affero, was designed to accomplish similar goals. This is
|
||||||
|
a different license, not a version of the Affero GPL, but Affero has
|
||||||
|
released a new version of the Affero GPL which permits relicensing under
|
||||||
|
this license.
|
||||||
|
|
||||||
|
The precise terms and conditions for copying, distribution and
|
||||||
|
modification follow.
|
||||||
|
|
||||||
|
TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
0. Definitions.
|
||||||
|
|
||||||
|
"This License" refers to version 3 of the GNU Affero General Public License.
|
||||||
|
|
||||||
|
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||||
|
works, such as semiconductor masks.
|
||||||
|
|
||||||
|
"The Program" refers to any copyrightable work licensed under this
|
||||||
|
License. Each licensee is addressed as "you". "Licensees" and
|
||||||
|
"recipients" may be individuals or organizations.
|
||||||
|
|
||||||
|
To "modify" a work means to copy from or adapt all or part of the work
|
||||||
|
in a fashion requiring copyright permission, other than the making of an
|
||||||
|
exact copy. The resulting work is called a "modified version" of the
|
||||||
|
earlier work or a work "based on" the earlier work.
|
||||||
|
|
||||||
|
A "covered work" means either the unmodified Program or a work based
|
||||||
|
on the Program.
|
||||||
|
|
||||||
|
To "propagate" a work means to do anything with it that, without
|
||||||
|
permission, would make you directly or secondarily liable for
|
||||||
|
infringement under applicable copyright law, except executing it on a
|
||||||
|
computer or modifying a private copy. Propagation includes copying,
|
||||||
|
distribution (with or without modification), making available to the
|
||||||
|
public, and in some countries other activities as well.
|
||||||
|
|
||||||
|
To "convey" a work means any kind of propagation that enables other
|
||||||
|
parties to make or receive copies. Mere interaction with a user through
|
||||||
|
a computer network, with no transfer of a copy, is not conveying.
|
||||||
|
|
||||||
|
An interactive user interface displays "Appropriate Legal Notices"
|
||||||
|
to the extent that it includes a convenient and prominently visible
|
||||||
|
feature that (1) displays an appropriate copyright notice, and (2)
|
||||||
|
tells the user that there is no warranty for the work (except to the
|
||||||
|
extent that warranties are provided), that licensees may convey the
|
||||||
|
work under this License, and how to view a copy of this License. If
|
||||||
|
the interface presents a list of user commands or options, such as a
|
||||||
|
menu, a prominent item in the list meets this criterion.
|
||||||
|
|
||||||
|
1. Source Code.
|
||||||
|
|
||||||
|
The "source code" for a work means the preferred form of the work
|
||||||
|
for making modifications to it. "Object code" means any non-source
|
||||||
|
form of a work.
|
||||||
|
|
||||||
|
A "Standard Interface" means an interface that either is an official
|
||||||
|
standard defined by a recognized standards body, or, in the case of
|
||||||
|
interfaces specified for a particular programming language, one that
|
||||||
|
is widely used among developers working in that language.
|
||||||
|
|
||||||
|
The "System Libraries" of an executable work include anything, other
|
||||||
|
than the work as a whole, that (a) is included in the normal form of
|
||||||
|
packaging a Major Component, but which is not part of that Major
|
||||||
|
Component, and (b) serves only to enable use of the work with that
|
||||||
|
Major Component, or to implement a Standard Interface for which an
|
||||||
|
implementation is available to the public in source code form. A
|
||||||
|
"Major Component", in this context, means a major essential component
|
||||||
|
(kernel, window system, and so on) of the specific operating system
|
||||||
|
(if any) on which the executable work runs, or a compiler used to
|
||||||
|
produce the work, or an object code interpreter used to run it.
|
||||||
|
|
||||||
|
The "Corresponding Source" for a work in object code form means all
|
||||||
|
the source code needed to generate, install, and (for an executable
|
||||||
|
work) run the object code and to modify the work, including scripts to
|
||||||
|
control those activities. However, it does not include the work's
|
||||||
|
System Libraries, or general-purpose tools or generally available free
|
||||||
|
programs which are used unmodified in performing those activities but
|
||||||
|
which are not part of the work. For example, Corresponding Source
|
||||||
|
includes interface definition files associated with source files for
|
||||||
|
the work, and the source code for shared libraries and dynamically
|
||||||
|
linked subprograms that the work is specifically designed to require,
|
||||||
|
such as by intimate data communication or control flow between those
|
||||||
|
subprograms and other parts of the work.
|
||||||
|
|
||||||
|
The Corresponding Source need not include anything that users
|
||||||
|
can regenerate automatically from other parts of the Corresponding
|
||||||
|
Source.
|
||||||
|
|
||||||
|
The Corresponding Source for a work in source code form is that
|
||||||
|
same work.
|
||||||
|
|
||||||
|
2. Basic Permissions.
|
||||||
|
|
||||||
|
All rights granted under this License are granted for the term of
|
||||||
|
copyright on the Program, and are irrevocable provided the stated
|
||||||
|
conditions are met. This License explicitly affirms your unlimited
|
||||||
|
permission to run the unmodified Program. The output from running a
|
||||||
|
covered work is covered by this License only if the output, given its
|
||||||
|
content, constitutes a covered work. This License acknowledges your
|
||||||
|
rights of fair use or other equivalent, as provided by copyright law.
|
||||||
|
|
||||||
|
You may make, run and propagate covered works that you do not
|
||||||
|
convey, without conditions so long as your license otherwise remains
|
||||||
|
in force. You may convey covered works to others for the sole purpose
|
||||||
|
of having them make modifications exclusively for you, or provide you
|
||||||
|
with facilities for running those works, provided that you comply with
|
||||||
|
the terms of this License in conveying all material for which you do
|
||||||
|
not control copyright. Those thus making or running the covered works
|
||||||
|
for you must do so exclusively on your behalf, under your direction
|
||||||
|
and control, on terms that prohibit them from making any copies of
|
||||||
|
your copyrighted material outside their relationship with you.
|
||||||
|
|
||||||
|
Conveying under any other circumstances is permitted solely under
|
||||||
|
the conditions stated below. Sublicensing is not allowed; section 10
|
||||||
|
makes it unnecessary.
|
||||||
|
|
||||||
|
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||||
|
|
||||||
|
No covered work shall be deemed part of an effective technological
|
||||||
|
measure under any applicable law fulfilling obligations under article
|
||||||
|
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||||
|
similar laws prohibiting or restricting circumvention of such
|
||||||
|
measures.
|
||||||
|
|
||||||
|
When you convey a covered work, you waive any legal power to forbid
|
||||||
|
circumvention of technological measures to the extent such circumvention
|
||||||
|
is effected by exercising rights under this License with respect to
|
||||||
|
the covered work, and you disclaim any intention to limit operation or
|
||||||
|
modification of the work as a means of enforcing, against the work's
|
||||||
|
users, your or third parties' legal rights to forbid circumvention of
|
||||||
|
technological measures.
|
||||||
|
|
||||||
|
4. Conveying Verbatim Copies.
|
||||||
|
|
||||||
|
You may convey verbatim copies of the Program's source code as you
|
||||||
|
receive it, in any medium, provided that you conspicuously and
|
||||||
|
appropriately publish on each copy an appropriate copyright notice;
|
||||||
|
keep intact all notices stating that this License and any
|
||||||
|
non-permissive terms added in accord with section 7 apply to the code;
|
||||||
|
keep intact all notices of the absence of any warranty; and give all
|
||||||
|
recipients a copy of this License along with the Program.
|
||||||
|
|
||||||
|
You may charge any price or no price for each copy that you convey,
|
||||||
|
and you may offer support or warranty protection for a fee.
|
||||||
|
|
||||||
|
5. Conveying Modified Source Versions.
|
||||||
|
|
||||||
|
You may convey a work based on the Program, or the modifications to
|
||||||
|
produce it from the Program, in the form of source code under the
|
||||||
|
terms of section 4, provided that you also meet all of these conditions:
|
||||||
|
|
||||||
|
a) The work must carry prominent notices stating that you modified
|
||||||
|
it, and giving a relevant date.
|
||||||
|
|
||||||
|
b) The work must carry prominent notices stating that it is
|
||||||
|
released under this License and any conditions added under section
|
||||||
|
7. This requirement modifies the requirement in section 4 to
|
||||||
|
"keep intact all notices".
|
||||||
|
|
||||||
|
c) You must license the entire work, as a whole, under this
|
||||||
|
License to anyone who comes into possession of a copy. This
|
||||||
|
License will therefore apply, along with any applicable section 7
|
||||||
|
additional terms, to the whole of the work, and all its parts,
|
||||||
|
regardless of how they are packaged. This License gives no
|
||||||
|
permission to license the work in any other way, but it does not
|
||||||
|
invalidate such permission if you have separately received it.
|
||||||
|
|
||||||
|
d) If the work has interactive user interfaces, each must display
|
||||||
|
Appropriate Legal Notices; however, if the Program has interactive
|
||||||
|
interfaces that do not display Appropriate Legal Notices, your
|
||||||
|
work need not make them do so.
|
||||||
|
|
||||||
|
A compilation of a covered work with other separate and independent
|
||||||
|
works, which are not by their nature extensions of the covered work,
|
||||||
|
and which are not combined with it such as to form a larger program,
|
||||||
|
in or on a volume of a storage or distribution medium, is called an
|
||||||
|
"aggregate" if the compilation and its resulting copyright are not
|
||||||
|
used to limit the access or legal rights of the compilation's users
|
||||||
|
beyond what the individual works permit. Inclusion of a covered work
|
||||||
|
in an aggregate does not cause this License to apply to the other
|
||||||
|
parts of the aggregate.
|
||||||
|
|
||||||
|
6. Conveying Non-Source Forms.
|
||||||
|
|
||||||
|
You may convey a covered work in object code form under the terms
|
||||||
|
of sections 4 and 5, provided that you also convey the
|
||||||
|
machine-readable Corresponding Source under the terms of this License,
|
||||||
|
in one of these ways:
|
||||||
|
|
||||||
|
a) Convey the object code in, or embodied in, a physical product
|
||||||
|
(including a physical distribution medium), accompanied by the
|
||||||
|
Corresponding Source fixed on a durable physical medium
|
||||||
|
customarily used for software interchange.
|
||||||
|
|
||||||
|
b) Convey the object code in, or embodied in, a physical product
|
||||||
|
(including a physical distribution medium), accompanied by a
|
||||||
|
written offer, valid for at least three years and valid for as
|
||||||
|
long as you offer spare parts or customer support for that product
|
||||||
|
model, to give anyone who possesses the object code either (1) a
|
||||||
|
copy of the Corresponding Source for all the software in the
|
||||||
|
product that is covered by this License, on a durable physical
|
||||||
|
medium customarily used for software interchange, for a price no
|
||||||
|
more than your reasonable cost of physically performing this
|
||||||
|
conveying of source, or (2) access to copy the
|
||||||
|
Corresponding Source from a network server at no charge.
|
||||||
|
|
||||||
|
c) Convey individual copies of the object code with a copy of the
|
||||||
|
written offer to provide the Corresponding Source. This
|
||||||
|
alternative is allowed only occasionally and noncommercially, and
|
||||||
|
only if you received the object code with such an offer, in accord
|
||||||
|
with subsection 6b.
|
||||||
|
|
||||||
|
d) Convey the object code by offering access from a designated
|
||||||
|
place (gratis or for a charge), and offer equivalent access to the
|
||||||
|
Corresponding Source in the same way through the same place at no
|
||||||
|
further charge. You need not require recipients to copy the
|
||||||
|
Corresponding Source along with the object code. If the place to
|
||||||
|
copy the object code is a network server, the Corresponding Source
|
||||||
|
may be on a different server (operated by you or a third party)
|
||||||
|
that supports equivalent copying facilities, provided you maintain
|
||||||
|
clear directions next to the object code saying where to find the
|
||||||
|
Corresponding Source. Regardless of what server hosts the
|
||||||
|
Corresponding Source, you remain obligated to ensure that it is
|
||||||
|
available for as long as needed to satisfy these requirements.
|
||||||
|
|
||||||
|
e) Convey the object code using peer-to-peer transmission, provided
|
||||||
|
you inform other peers where the object code and Corresponding
|
||||||
|
Source of the work are being offered to the general public at no
|
||||||
|
charge under subsection 6d.
|
||||||
|
|
||||||
|
A separable portion of the object code, whose source code is excluded
|
||||||
|
from the Corresponding Source as a System Library, need not be
|
||||||
|
included in conveying the object code work.
|
||||||
|
|
||||||
|
A "User Product" is either (1) a "consumer product", which means any
|
||||||
|
tangible personal property which is normally used for personal, family,
|
||||||
|
or household purposes, or (2) anything designed or sold for incorporation
|
||||||
|
into a dwelling. In determining whether a product is a consumer product,
|
||||||
|
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||||
|
product received by a particular user, "normally used" refers to a
|
||||||
|
typical or common use of that class of product, regardless of the status
|
||||||
|
of the particular user or of the way in which the particular user
|
||||||
|
actually uses, or expects or is expected to use, the product. A product
|
||||||
|
is a consumer product regardless of whether the product has substantial
|
||||||
|
commercial, industrial or non-consumer uses, unless such uses represent
|
||||||
|
the only significant mode of use of the product.
|
||||||
|
|
||||||
|
"Installation Information" for a User Product means any methods,
|
||||||
|
procedures, authorization keys, or other information required to install
|
||||||
|
and execute modified versions of a covered work in that User Product from
|
||||||
|
a modified version of its Corresponding Source. The information must
|
||||||
|
suffice to ensure that the continued functioning of the modified object
|
||||||
|
code is in no case prevented or interfered with solely because
|
||||||
|
modification has been made.
|
||||||
|
|
||||||
|
If you convey an object code work under this section in, or with, or
|
||||||
|
specifically for use in, a User Product, and the conveying occurs as
|
||||||
|
part of a transaction in which the right of possession and use of the
|
||||||
|
User Product is transferred to the recipient in perpetuity or for a
|
||||||
|
fixed term (regardless of how the transaction is characterized), the
|
||||||
|
Corresponding Source conveyed under this section must be accompanied
|
||||||
|
by the Installation Information. But this requirement does not apply
|
||||||
|
if neither you nor any third party retains the ability to install
|
||||||
|
modified object code on the User Product (for example, the work has
|
||||||
|
been installed in ROM).
|
||||||
|
|
||||||
|
The requirement to provide Installation Information does not include a
|
||||||
|
requirement to continue to provide support service, warranty, or updates
|
||||||
|
for a work that has been modified or installed by the recipient, or for
|
||||||
|
the User Product in which it has been modified or installed. Access to a
|
||||||
|
network may be denied when the modification itself materially and
|
||||||
|
adversely affects the operation of the network or violates the rules and
|
||||||
|
protocols for communication across the network.
|
||||||
|
|
||||||
|
Corresponding Source conveyed, and Installation Information provided,
|
||||||
|
in accord with this section must be in a format that is publicly
|
||||||
|
documented (and with an implementation available to the public in
|
||||||
|
source code form), and must require no special password or key for
|
||||||
|
unpacking, reading or copying.
|
||||||
|
|
||||||
|
7. Additional Terms.
|
||||||
|
|
||||||
|
"Additional permissions" are terms that supplement the terms of this
|
||||||
|
License by making exceptions from one or more of its conditions.
|
||||||
|
Additional permissions that are applicable to the entire Program shall
|
||||||
|
be treated as though they were included in this License, to the extent
|
||||||
|
that they are valid under applicable law. If additional permissions
|
||||||
|
apply only to part of the Program, that part may be used separately
|
||||||
|
under those permissions, but the entire Program remains governed by
|
||||||
|
this License without regard to the additional permissions.
|
||||||
|
|
||||||
|
When you convey a copy of a covered work, you may at your option
|
||||||
|
remove any additional permissions from that copy, or from any part of
|
||||||
|
it. (Additional permissions may be written to require their own
|
||||||
|
removal in certain cases when you modify the work.) You may place
|
||||||
|
additional permissions on material, added by you to a covered work,
|
||||||
|
for which you have or can give appropriate copyright permission.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, for material you
|
||||||
|
add to a covered work, you may (if authorized by the copyright holders of
|
||||||
|
that material) supplement the terms of this License with terms:
|
||||||
|
|
||||||
|
a) Disclaiming warranty or limiting liability differently from the
|
||||||
|
terms of sections 15 and 16 of this License; or
|
||||||
|
|
||||||
|
b) Requiring preservation of specified reasonable legal notices or
|
||||||
|
author attributions in that material or in the Appropriate Legal
|
||||||
|
Notices displayed by works containing it; or
|
||||||
|
|
||||||
|
c) Prohibiting misrepresentation of the origin of that material, or
|
||||||
|
requiring that modified versions of such material be marked in
|
||||||
|
reasonable ways as different from the original version; or
|
||||||
|
|
||||||
|
d) Limiting the use for publicity purposes of names of licensors or
|
||||||
|
authors of the material; or
|
||||||
|
|
||||||
|
e) Declining to grant rights under trademark law for use of some
|
||||||
|
trade names, trademarks, or service marks; or
|
||||||
|
|
||||||
|
f) Requiring indemnification of licensors and authors of that
|
||||||
|
material by anyone who conveys the material (or modified versions of
|
||||||
|
it) with contractual assumptions of liability to the recipient, for
|
||||||
|
any liability that these contractual assumptions directly impose on
|
||||||
|
those licensors and authors.
|
||||||
|
|
||||||
|
All other non-permissive additional terms are considered "further
|
||||||
|
restrictions" within the meaning of section 10. If the Program as you
|
||||||
|
received it, or any part of it, contains a notice stating that it is
|
||||||
|
governed by this License along with a term that is a further
|
||||||
|
restriction, you may remove that term. If a license document contains
|
||||||
|
a further restriction but permits relicensing or conveying under this
|
||||||
|
License, you may add to a covered work material governed by the terms
|
||||||
|
of that license document, provided that the further restriction does
|
||||||
|
not survive such relicensing or conveying.
|
||||||
|
|
||||||
|
If you add terms to a covered work in accord with this section, you
|
||||||
|
must place, in the relevant source files, a statement of the
|
||||||
|
additional terms that apply to those files, or a notice indicating
|
||||||
|
where to find the applicable terms.
|
||||||
|
|
||||||
|
Additional terms, permissive or non-permissive, may be stated in the
|
||||||
|
form of a separately written license, or stated as exceptions;
|
||||||
|
the above requirements apply either way.
|
||||||
|
|
||||||
|
8. Termination.
|
||||||
|
|
||||||
|
You may not propagate or modify a covered work except as expressly
|
||||||
|
provided under this License. Any attempt otherwise to propagate or
|
||||||
|
modify it is void, and will automatically terminate your rights under
|
||||||
|
this License (including any patent licenses granted under the third
|
||||||
|
paragraph of section 11).
|
||||||
|
|
||||||
|
However, if you cease all violation of this License, then your
|
||||||
|
license from a particular copyright holder is reinstated (a)
|
||||||
|
provisionally, unless and until the copyright holder explicitly and
|
||||||
|
finally terminates your license, and (b) permanently, if the copyright
|
||||||
|
holder fails to notify you of the violation by some reasonable means
|
||||||
|
prior to 60 days after the cessation.
|
||||||
|
|
||||||
|
Moreover, your license from a particular copyright holder is
|
||||||
|
reinstated permanently if the copyright holder notifies you of the
|
||||||
|
violation by some reasonable means, this is the first time you have
|
||||||
|
received notice of violation of this License (for any work) from that
|
||||||
|
copyright holder, and you cure the violation prior to 30 days after
|
||||||
|
your receipt of the notice.
|
||||||
|
|
||||||
|
Termination of your rights under this section does not terminate the
|
||||||
|
licenses of parties who have received copies or rights from you under
|
||||||
|
this License. If your rights have been terminated and not permanently
|
||||||
|
reinstated, you do not qualify to receive new licenses for the same
|
||||||
|
material under section 10.
|
||||||
|
|
||||||
|
9. Acceptance Not Required for Having Copies.
|
||||||
|
|
||||||
|
You are not required to accept this License in order to receive or
|
||||||
|
run a copy of the Program. Ancillary propagation of a covered work
|
||||||
|
occurring solely as a consequence of using peer-to-peer transmission
|
||||||
|
to receive a copy likewise does not require acceptance. However,
|
||||||
|
nothing other than this License grants you permission to propagate or
|
||||||
|
modify any covered work. These actions infringe copyright if you do
|
||||||
|
not accept this License. Therefore, by modifying or propagating a
|
||||||
|
covered work, you indicate your acceptance of this License to do so.
|
||||||
|
|
||||||
|
10. Automatic Licensing of Downstream Recipients.
|
||||||
|
|
||||||
|
Each time you convey a covered work, the recipient automatically
|
||||||
|
receives a license from the original licensors, to run, modify and
|
||||||
|
propagate that work, subject to this License. You are not responsible
|
||||||
|
for enforcing compliance by third parties with this License.
|
||||||
|
|
||||||
|
An "entity transaction" is a transaction transferring control of an
|
||||||
|
organization, or substantially all assets of one, or subdividing an
|
||||||
|
organization, or merging organizations. If propagation of a covered
|
||||||
|
work results from an entity transaction, each party to that
|
||||||
|
transaction who receives a copy of the work also receives whatever
|
||||||
|
licenses to the work the party's predecessor in interest had or could
|
||||||
|
give under the previous paragraph, plus a right to possession of the
|
||||||
|
Corresponding Source of the work from the predecessor in interest, if
|
||||||
|
the predecessor has it or can get it with reasonable efforts.
|
||||||
|
|
||||||
|
You may not impose any further restrictions on the exercise of the
|
||||||
|
rights granted or affirmed under this License. For example, you may
|
||||||
|
not impose a license fee, royalty, or other charge for exercise of
|
||||||
|
rights granted under this License, and you may not initiate litigation
|
||||||
|
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||||
|
any patent claim is infringed by making, using, selling, offering for
|
||||||
|
sale, or importing the Program or any portion of it.
|
||||||
|
|
||||||
|
11. Patents.
|
||||||
|
|
||||||
|
A "contributor" is a copyright holder who authorizes use under this
|
||||||
|
License of the Program or a work on which the Program is based. The
|
||||||
|
work thus licensed is called the contributor's "contributor version".
|
||||||
|
|
||||||
|
A contributor's "essential patent claims" are all patent claims
|
||||||
|
owned or controlled by the contributor, whether already acquired or
|
||||||
|
hereafter acquired, that would be infringed by some manner, permitted
|
||||||
|
by this License, of making, using, or selling its contributor version,
|
||||||
|
but do not include claims that would be infringed only as a
|
||||||
|
consequence of further modification of the contributor version. For
|
||||||
|
purposes of this definition, "control" includes the right to grant
|
||||||
|
patent sublicenses in a manner consistent with the requirements of
|
||||||
|
this License.
|
||||||
|
|
||||||
|
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||||
|
patent license under the contributor's essential patent claims, to
|
||||||
|
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||||
|
propagate the contents of its contributor version.
|
||||||
|
|
||||||
|
In the following three paragraphs, a "patent license" is any express
|
||||||
|
agreement or commitment, however denominated, not to enforce a patent
|
||||||
|
(such as an express permission to practice a patent or covenant not to
|
||||||
|
sue for patent infringement). To "grant" such a patent license to a
|
||||||
|
party means to make such an agreement or commitment not to enforce a
|
||||||
|
patent against the party.
|
||||||
|
|
||||||
|
If you convey a covered work, knowingly relying on a patent license,
|
||||||
|
and the Corresponding Source of the work is not available for anyone
|
||||||
|
to copy, free of charge and under the terms of this License, through a
|
||||||
|
publicly available network server or other readily accessible means,
|
||||||
|
then you must either (1) cause the Corresponding Source to be so
|
||||||
|
available, or (2) arrange to deprive yourself of the benefit of the
|
||||||
|
patent license for this particular work, or (3) arrange, in a manner
|
||||||
|
consistent with the requirements of this License, to extend the patent
|
||||||
|
license to downstream recipients. "Knowingly relying" means you have
|
||||||
|
actual knowledge that, but for the patent license, your conveying the
|
||||||
|
covered work in a country, or your recipient's use of the covered work
|
||||||
|
in a country, would infringe one or more identifiable patents in that
|
||||||
|
country that you have reason to believe are valid.
|
||||||
|
|
||||||
|
If, pursuant to or in connection with a single transaction or
|
||||||
|
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||||
|
covered work, and grant a patent license to some of the parties
|
||||||
|
receiving the covered work authorizing them to use, propagate, modify
|
||||||
|
or convey a specific copy of the covered work, then the patent license
|
||||||
|
you grant is automatically extended to all recipients of the covered
|
||||||
|
work and works based on it.
|
||||||
|
|
||||||
|
A patent license is "discriminatory" if it does not include within
|
||||||
|
the scope of its coverage, prohibits the exercise of, or is
|
||||||
|
conditioned on the non-exercise of one or more of the rights that are
|
||||||
|
specifically granted under this License. You may not convey a covered
|
||||||
|
work if you are a party to an arrangement with a third party that is
|
||||||
|
in the business of distributing software, under which you make payment
|
||||||
|
to the third party based on the extent of your activity of conveying
|
||||||
|
the work, and under which the third party grants, to any of the
|
||||||
|
parties who would receive the covered work from you, a discriminatory
|
||||||
|
patent license (a) in connection with copies of the covered work
|
||||||
|
conveyed by you (or copies made from those copies), or (b) primarily
|
||||||
|
for and in connection with specific products or compilations that
|
||||||
|
contain the covered work, unless you entered into that arrangement,
|
||||||
|
or that patent license was granted, prior to 28 March 2007.
|
||||||
|
|
||||||
|
Nothing in this License shall be construed as excluding or limiting
|
||||||
|
any implied license or other defenses to infringement that may
|
||||||
|
otherwise be available to you under applicable patent law.
|
||||||
|
|
||||||
|
12. No Surrender of Others' Freedom.
|
||||||
|
|
||||||
|
If conditions are imposed on you (whether by court order, agreement or
|
||||||
|
otherwise) that contradict the conditions of this License, they do not
|
||||||
|
excuse you from the conditions of this License. If you cannot convey a
|
||||||
|
covered work so as to satisfy simultaneously your obligations under this
|
||||||
|
License and any other pertinent obligations, then as a consequence you may
|
||||||
|
not convey it at all. For example, if you agree to terms that obligate you
|
||||||
|
to collect a royalty for further conveying from those to whom you convey
|
||||||
|
the Program, the only way you could satisfy both those terms and this
|
||||||
|
License would be to refrain entirely from conveying the Program.
|
||||||
|
|
||||||
|
13. Remote Network Interaction; Use with the GNU General Public License.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, if you modify the
|
||||||
|
Program, your modified version must prominently offer all users
|
||||||
|
interacting with it remotely through a computer network (if your version
|
||||||
|
supports such interaction) an opportunity to receive the Corresponding
|
||||||
|
Source of your version by providing access to the Corresponding Source
|
||||||
|
from a network server at no charge, through some standard or customary
|
||||||
|
means of facilitating copying of software. This Corresponding Source
|
||||||
|
shall include the Corresponding Source for any work covered by version 3
|
||||||
|
of the GNU General Public License that is incorporated pursuant to the
|
||||||
|
following paragraph.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, you have
|
||||||
|
permission to link or combine any covered work with a work licensed
|
||||||
|
under version 3 of the GNU General Public License into a single
|
||||||
|
combined work, and to convey the resulting work. The terms of this
|
||||||
|
License will continue to apply to the part which is the covered work,
|
||||||
|
but the work with which it is combined will remain governed by version
|
||||||
|
3 of the GNU General Public License.
|
||||||
|
|
||||||
|
14. Revised Versions of this License.
|
||||||
|
|
||||||
|
The Free Software Foundation may publish revised and/or new versions of
|
||||||
|
the GNU Affero General Public License from time to time. Such new versions
|
||||||
|
will be similar in spirit to the present version, but may differ in detail to
|
||||||
|
address new problems or concerns.
|
||||||
|
|
||||||
|
Each version is given a distinguishing version number. If the
|
||||||
|
Program specifies that a certain numbered version of the GNU Affero General
|
||||||
|
Public License "or any later version" applies to it, you have the
|
||||||
|
option of following the terms and conditions either of that numbered
|
||||||
|
version or of any later version published by the Free Software
|
||||||
|
Foundation. If the Program does not specify a version number of the
|
||||||
|
GNU Affero General Public License, you may choose any version ever published
|
||||||
|
by the Free Software Foundation.
|
||||||
|
|
||||||
|
If the Program specifies that a proxy can decide which future
|
||||||
|
versions of the GNU Affero General Public License can be used, that proxy's
|
||||||
|
public statement of acceptance of a version permanently authorizes you
|
||||||
|
to choose that version for the Program.
|
||||||
|
|
||||||
|
Later license versions may give you additional or different
|
||||||
|
permissions. However, no additional obligations are imposed on any
|
||||||
|
author or copyright holder as a result of your choosing to follow a
|
||||||
|
later version.
|
||||||
|
|
||||||
|
15. Disclaimer of Warranty.
|
||||||
|
|
||||||
|
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||||
|
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||||
|
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||||
|
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||||
|
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||||
|
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||||
|
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||||
|
|
||||||
|
16. Limitation of Liability.
|
||||||
|
|
||||||
|
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||||
|
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||||
|
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||||
|
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||||
|
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||||
|
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||||
|
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||||
|
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||||
|
SUCH DAMAGES.
|
||||||
|
|
||||||
|
17. Interpretation of Sections 15 and 16.
|
||||||
|
|
||||||
|
If the disclaimer of warranty and limitation of liability provided
|
||||||
|
above cannot be given local legal effect according to their terms,
|
||||||
|
reviewing courts shall apply local law that most closely approximates
|
||||||
|
an absolute waiver of all civil liability in connection with the
|
||||||
|
Program, unless a warranty or assumption of liability accompanies a
|
||||||
|
copy of the Program in return for a fee.
|
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
How to Apply These Terms to Your New Programs
|
||||||
|
|
||||||
|
If you develop a new program, and you want it to be of the greatest
|
||||||
|
possible use to the public, the best way to achieve this is to make it
|
||||||
|
free software which everyone can redistribute and change under these terms.
|
||||||
|
|
||||||
|
To do so, attach the following notices to the program. It is safest
|
||||||
|
to attach them to the start of each source file to most effectively
|
||||||
|
state the exclusion of warranty; and each file should have at least
|
||||||
|
the "copyright" line and a pointer to where the full notice is found.
|
||||||
|
|
||||||
|
<one line to give the program's name and a brief idea of what it does.>
|
||||||
|
Copyright (C) <year> <name of author>
|
||||||
|
|
||||||
|
This program is free software: you can redistribute it and/or modify
|
||||||
|
it under the terms of the GNU Affero General Public License as published by
|
||||||
|
the Free Software Foundation, either version 3 of the License, or
|
||||||
|
(at your option) any later version.
|
||||||
|
|
||||||
|
This program is distributed in the hope that it will be useful,
|
||||||
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
GNU Affero General Public License for more details.
|
||||||
|
|
||||||
|
You should have received a copy of the GNU Affero General Public License
|
||||||
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
Also add information on how to contact you by electronic and paper mail.
|
||||||
|
|
||||||
|
If your software can interact with users remotely through a computer
|
||||||
|
network, you should also make sure that it provides a way for users to
|
||||||
|
get its source. For example, if your program is a web application, its
|
||||||
|
interface could display a "Source" link that leads users to an archive
|
||||||
|
of the code. There are many ways you could offer source, and different
|
||||||
|
solutions will be better for different programs; see section 13 for the
|
||||||
|
specific requirements.
|
||||||
|
|
||||||
|
You should also get your employer (if you work as a programmer) or school,
|
||||||
|
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||||
|
For more information on this, and how to apply and follow the GNU AGPL, see
|
||||||
|
<https://www.gnu.org/licenses/>.
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# fee[dB]ack
|
||||||
|
|
||||||
|
## Plugins
|
||||||
|
|
||||||
|
| Plugin | Description | Install |
|
||||||
|
|------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
|
||||||
|
| [Create from Tab](https://github.com/byrongamatos/slopsmith-plugin-ug) | Search Ultimate Guitar for GP tabs and convert to playable songs | `git clone ...slopsmith-plugin-ug.git ultimate_guitar` |
|
||||||
|
| [Import Tab](https://github.com/byrongamatos/slopsmith-plugin-tabimport) | Drag and drop Guitar Pro files to create songs | `git clone ...slopsmith-plugin-tabimport.git tab_import` |
|
||||||
|
| [Practice Journal](https://github.com/byrongamatos/slopsmith-plugin-practice) | Auto-track practice time, speed, loops. Dashboard with charts | `git clone ...slopsmith-plugin-practice.git practice_journal` |
|
||||||
|
| [Setlist Builder](https://github.com/byrongamatos/slopsmith-plugin-setlist) | Create ordered playlists with sequential playback | `git clone ...slopsmith-plugin-setlist.git setlist` |
|
||||||
|
| [Metronome](https://github.com/byrongamatos/slopsmith-plugin-metronome) | Audible click and visual beat flash synced to song tempo | `git clone ...slopsmith-plugin-metronome.git metronome` |
|
||||||
|
| [Tone Player](https://github.com/byrongamatos/slopsmith-plugin-tones) | View amp/pedal/cab signal chains with gear artwork | `git clone ...slopsmith-plugin-tones.git tones` |
|
||||||
|
| [Fretboard View](https://github.com/byrongamatos/slopsmith-plugin-fretboard) | Live fretboard overlay showing active notes in real-time | `git clone ...slopsmith-plugin-fretboard.git fretboard` |
|
||||||
|
| [Tab View](https://github.com/byrongamatos/slopsmith-plugin-tabview) | Scrolling guitar tablature notation via alphaTab | `git clone ...slopsmith-plugin-tabview.git tab_view` |
|
||||||
|
| [MIDI Amp Control](https://github.com/byrongamatos/slopsmith-plugin-midi) | Auto-switch amp/modeler presets via MIDI on tone changes | `git clone ...slopsmith-plugin-midi.git midi_amp` |
|
||||||
|
| [Section Map](https://github.com/byrongamatos/slopsmith-plugin-sectionmap) | Color-coded song structure minimap with clickable navigation | `git clone ...slopsmith-plugin-sectionmap.git section_map` |
|
||||||
|
| [Arrangement Editor](https://github.com/byrongamatos/slopsmith-plugin-editor) | DAW-like visual editor for creating and editing song note charts | `git clone ...slopsmith-plugin-editor.git editor` |
|
||||||
|
| [MIDI Capo](https://github.com/masc0t/slopsmith-plugin-midi-capo) | MIDI capo control for real-time transposition | `git clone ...slopsmith-plugin-midi-capo.git midi_capo` |
|
||||||
|
| [Note Detection](https://github.com/byrongamatos/slopsmith-plugin-notedetect) | Real-time pitch detection and scoring against highway notes | `git clone ...slopsmith-plugin-notedetect.git note_detect` |
|
||||||
|
| [Find More](https://github.com/masc0t/slopsmith-plugin-find-more) | Search for more songs by the same artist | `git clone ...slopsmith-plugin-find-more.git find_more` |
|
||||||
|
| [Piano Highway](https://github.com/byrongamatos/slopsmith-plugin-piano) | Scrolling piano/keyboard view for Keys arrangements with MIDI input | `git clone ...slopsmith-plugin-piano.git piano` |
|
||||||
|
| [Studio](https://github.com/byrongamatos/slopsmith-plugin-studio) | Collaborative band recording and multi-track mixing | `git clone ...slopsmith-plugin-studio.git studio` |
|
||||||
|
| [Drum Highway](https://github.com/byrongamatos/slopsmith-plugin-drums) | Lane-based drum highway with MIDI drum pad input and built-in sounds | `git clone ...slopsmith-plugin-drums.git drums` |
|
||||||
|
| [Split Screen](https://github.com/topkoa/slopsmith-plugin-splitscreen) | 2-4 highway panels side-by-side for multi-arrangement practice | `git clone ...slopsmith-plugin-splitscreen.git splitscreen` |
|
||||||
|
| [Stems Mixer](https://github.com/topkoa/slopsmith-plugin-stems) | Per-stem mute/volume controls for .sloppak songs | `git clone ...slopsmith-plugin-stems.git stems` |
|
||||||
|
| [Invert Highway](https://github.com/masc0t/slopsmith-plugin-invert-highway) | Flip the highway note direction | `git clone ...slopsmith-plugin-invert-highway.git invert_highway` |
|
||||||
|
| [Jumping Tab](https://github.com/renanboni/slopsmith-plugin-jumpingtab) | Yousician-style 2D horizontal tab with trajectory arcs and hopping ball | `git clone ...slopsmith-plugin-jumpingtab.git jumpingtab` |
|
||||||
|
| [Step Mode](https://github.com/byrongamatos/slopsmith-plugin-stepmode) | Step-by-step practice mode — highway freezes at each note until played (via Note Detection) or Space | `git clone ...slopsmith-plugin-stepmode.git step_mode` |
|
||||||
|
| [Lyrics Sync](https://github.com/byrongamatos/slopsmith-plugin-lyrics-sync) | Generate synced LRC lyrics from text + vocals stem via Whisper alignment | `git clone ...slopsmith-plugin-lyrics-sync.git lyrics_sync` |
|
||||||
|
| [Lyrics Karaoke](https://github.com/byrongamatos/slopsmith-plugin-lyrics-karaoke) | Per-syllable karaoke pitch ribbon for sloppak songs (Whisper alignment + librosa pYIN) | `git clone ...slopsmith-plugin-lyrics-karaoke.git lyrics_karaoke` |
|
||||||
|
| [NAM Tone Engine](https://github.com/byrongamatos/slopsmith-plugin-nam-tone) | In-browser amp modeling with NAM WASM, cabinet IRs, tone auto-switching | `git clone ...slopsmith-plugin-nam-tone.git nam_tone` |
|
||||||
|
| [Guitar Theory Lab](https://github.com/topkoa/slopsmith-plugin-guitar-theory) | Explore scales, chords, intervals, tunings, and voicings on a fully interactive fretboard | `git clone ...slopsmith-plugin-nam-tone.git guitar-theory-lab` |
|
||||||
|
| [Themes](https://github.com/masc0t/slopsmith-plugin-themes) | Offers several basic recolorings of the interface | `git clone ...slopsmith-plugin-themes.git themes` |
|
||||||
|
| [Update Manager](https://github.com/masc0t/slopsmith-update-manager) | Installs, updates, and uninstalls other plugins and the slopsmith core itself | `git clone ...slopsmith-update-manager.git update_manager` |
|
||||||
|
| [Tuner](https://github.com/OmikronApex/slopsmith-plugin-tuner) | Floating tuner with customizable tunings | `git clone ...slopsmith-plugin-tuner.git tuner` |
|
||||||
|
| [Simplify Chords](https://github.com/bkranendonk/slopsmith-plugin-simplify-chords) | Changes complex chords on the note highway to simpler ones. Inspired by Ultimate Guitar's Simplify button. | `git clone ...slopsmith-plugin-simplify-chords.git simplify-chords` |
|
||||||
|
| [Key Bindings](https://github.com/jackipicco/slopsmith-plugin-key-bindings) | Highway key bindings for keyboard and TV remote | `git clone ...slopsmith-plugin-key-bindings.git key_bindings` |
|
||||||
|
| [Folder Organizer](https://github.com/Elit3d/slopsmith-plugin-folder-organizer) | Organize your sloppak DLC songs into a folder tree view, grouped by subfolder name | `git clone ...slopsmith-plugin-folder-organizer.git folder-organizer` |
|
||||||
|
| [SlopScale](https://github.com/ChrisBeWithYou/slopsmith-plugin-slopscale) | Scale, arpeggio, and sweep-arpeggio practice routines with 3D highway, 2D highway, and tab renderers. Pathway selector, CAGED shape-run arpeggios, and generated audio backing. | `git clone ...slopsmith-plugin-slopscale.git slopscale` |
|
||||||
|
| [NAM Rig Builder](https://github.com/Jafz2001/slopsmith-plugin-nam-rig-builder) | Map tones to chained NAM neural-amp rigs (tone3000 captures + IRs) — full pedal→amp→cab playback, per-stage bypass, and a gear catalog | `git clone ...slopsmith-plugin-nam-rig-builder.git nam_rig_builder` |
|
||||||
|
| [Audio Preview](https://github.com/saleemk/slopsmith-plugin-audio-preview) | Quick audio previews from library cards with configurable start time, volume, and duration | `git clone ...slopsmith-plugin-audio-preview.git audio_preview` |
|
||||||
|
| [Song Mastery](https://github.com/jamesgaiser/slopsmith-plugin-song-mastery) | Auto-adjusts difficulty based on your rolling note accuracy and saves the slider position per song | `git clone ...slopsmith-plugin-song-mastery.git song_mastery` |
|
||||||
|
| [Song Preview](https://github.com/DeathlySin/slopsmith-plugin-song-preview) | Quickly hear previews of songs in your library with a clean visual indicator of what's playing. Supports .sloppak and loose folders song formats, with the visual indicator matching up to whatever theme you are using! | `git clone ...slopsmith-plugin-song-preview.git song_preview` |
|
||||||
|
| [Mobile Note Highway](https://github.com/saleemk/slopsmith-plugin-mobile-note-highway) | Touch-optimized player with collapsible controls, highway gestures, and device-adaptive layouts for phones and tablets | `git clone ...slopsmith-plugin-mobile-note-highway.git mobile_note_highway` |
|
||||||
|
| [Shuffle](https://github.com/Erikcb91/Slopsmith-Shuffle-Mode) | Random playback from your library — artist & tuning filters, auto-advance with countdown popup, note_detect compatible | `git clone https://github.com/Erikcb91/Slopsmith-Shuffle-Mode.git shuffle` |
|
||||||
|
|
||||||
|
Install any plugin by cloning it into your `plugins/` directory and restarting:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd plugins
|
||||||
|
git clone https://github.com/byrongamatos/slopsmith-plugin-ug.git ultimate_guitar
|
||||||
|
docker compose restart
|
||||||
|
```
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# Supporters
|
||||||
|
|
||||||
|
Slopsmith's development is supported by these generous people. Thank you. ❤️
|
||||||
|
|
||||||
|
Want to be listed here? See [Support Slopsmith](README.md#support-slopsmith).
|
||||||
|
|
||||||
|
## Patrons
|
||||||
|
|
||||||
|
_Be the first._
|
||||||
|
|
||||||
|
## Sustainers
|
||||||
|
|
||||||
|
_Be the first._
|
||||||
|
|
||||||
|
## Supporters
|
||||||
|
|
||||||
|
_Be the first._
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Note Failure Feedback
|
||||||
|
|
||||||
|
Some rhythm-practice tools mark missed or failed notes with **exclamation marks** (!) on the note highway at the position where the missed note was. On chords, the exclamation mark sits directly above the chord "bars," which can make them hard to see on songs with dense six-string chords — and such markers often can't be resized or recolored.
|
||||||
|
|
||||||
|
A few related things worth knowing:
|
||||||
|
|
||||||
|
- The markers are persistent visual flags on the note track after the note passes, not a popup or audio cue.
|
||||||
|
- For stricter feedback, a zero error-tolerance mode can require a clean run before a looped section advances.
|
||||||
|
- Forgiving note detection means an occasional genuinely missed note won't get
|
||||||
|
flagged, especially in fast passages — a tighter scoring mode helps when you
|
||||||
|
want stricter grading.
|
||||||
|
|
||||||
|
Goal: reproduce and improve on that behavior. When a user loops over the same
|
||||||
|
5-note lick repeatedly, the highway should show note misses with diagnostic
|
||||||
|
detail — which note was missed and *how* it was missed (too late / too early /
|
||||||
|
too sharp / too flat / not played).
|
||||||
|
|
||||||
|
## Docs
|
||||||
|
|
||||||
|
- **[Technical Spec](docs/NOTE_FAILURE_SPEC.md)** — architecture, matching
|
||||||
|
algorithm, rendering design, data structures, integration points
|
||||||
|
- **[Implementation Plan](docs/NOTE_FAILURE_PLAN.md)** — 7 phases from
|
||||||
|
detection foundation through section grading and polish
|
||||||
|
- **Note Detection Plugin Plan** — see the
|
||||||
|
[slopsmith-plugin-notedetect](https://github.com/topkoa/slopsmith-plugin-notedetect)
|
||||||
|
repository (Phase 0 foundation)
|
||||||
@@ -0,0 +1,667 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# shellcheck shell=bash
|
||||||
|
# =============================================================================
|
||||||
|
# build-proxmox-ct.sh – Build a Proxmox LXC rootfs from WSL2 (no lxc-start)
|
||||||
|
#
|
||||||
|
# Run this from your project root (where server.py, lib/, etc. live).
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# sudo bash build-proxmox-ct.sh [TARGETARCH] [OUTPUT_NAME]
|
||||||
|
#
|
||||||
|
# Examples:
|
||||||
|
# sudo bash build-proxmox-ct.sh amd64 slopsmith-ct
|
||||||
|
# sudo bash build-proxmox-ct.sh arm64 slopsmith-ct
|
||||||
|
#
|
||||||
|
# The resulting container ships empty; mount or copy your .sloppak /
|
||||||
|
# loose-folder library into /dlc inside the CT after import.
|
||||||
|
#
|
||||||
|
# Environment variables:
|
||||||
|
# SKIP_HASH_CHECK=1 Bypass SHA256 verification — for unpinned hashes OR
|
||||||
|
# to override mismatches when an upstream artifact rolls.
|
||||||
|
# Use with caution.
|
||||||
|
# KEEP_BUILD_DIR=1 Retain ${BUILD_BASE} after a successful build
|
||||||
|
# FORCE_REBUILD=1 Delete an existing rootfs without prompting (for CI)
|
||||||
|
#
|
||||||
|
# Prerequisites (install in WSL):
|
||||||
|
# sudo apt install debootstrap systemd-container tar zstd curl unzip git
|
||||||
|
#
|
||||||
|
# On Proxmox, after transfer:
|
||||||
|
# pct restore <VMID> slopsmith-ct.tar.zst --storage local-lvm --rootfs 8 --unprivileged 1
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
TARGETARCH="${1:-amd64}"
|
||||||
|
OUTPUT_NAME="${2:-slopsmith-ct}"
|
||||||
|
|
||||||
|
# OUTPUT_NAME is a positional arg that flows into BUILD_BASE (interpolated into
|
||||||
|
# `mkdir -p` / `rm -rf` paths) and into the final tarball name. Reject anything
|
||||||
|
# outside a safe filename charset so an input like `../../etc` can't escape
|
||||||
|
# /tmp or shape the tarball path.
|
||||||
|
if [[ ! "$OUTPUT_NAME" =~ ^[A-Za-z0-9._-]+$ ]]; then
|
||||||
|
echo "[ERROR] OUTPUT_NAME must match ^[A-Za-z0-9._-]+\$ (got: '${OUTPUT_NAME}')" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# debootstrap requires a real Linux filesystem (ext4/tmpfs/etc.) — it creates
|
||||||
|
# device nodes that NTFS/FUSE mounts (/mnt/c, /mnt/d …) cannot represent.
|
||||||
|
# We default the build dir to /tmp (a Linux fs on every WSL2 setup we've
|
||||||
|
# seen, even when /tmp isn't strictly tmpfs) and copy the final tarball back.
|
||||||
|
# Namespace BUILD_BASE by OUTPUT_NAME + TARGETARCH so concurrent invocations
|
||||||
|
# (or stale leftovers from a prior build of a different artifact) don't
|
||||||
|
# collide on /tmp/proxmox-ct-build/rootfs. BUILD_BASE can still be overridden
|
||||||
|
# via the environment for users who want a known, reusable path.
|
||||||
|
BUILD_BASE="${BUILD_BASE:-/tmp/proxmox-ct-build-${OUTPUT_NAME}-${TARGETARCH}}"
|
||||||
|
# Safety net: BUILD_BASE feeds rm -rf in the cleanup trap, the rebuild flow,
|
||||||
|
# and (indirectly) the rootfs build. Refuse obviously dangerous values up
|
||||||
|
# front so a stray BUILD_BASE=/ or BUILD_BASE='' can never `rm -rf` the host.
|
||||||
|
case "$BUILD_BASE" in
|
||||||
|
""|/|//|/.*|.|./*|../*) echo "[ERROR] Refusing dangerous BUILD_BASE='${BUILD_BASE}'." >&2; exit 1 ;;
|
||||||
|
esac
|
||||||
|
if [[ "${BUILD_BASE}" != /* ]]; then
|
||||||
|
echo "[ERROR] BUILD_BASE must be an absolute path (got: '${BUILD_BASE}')." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Normalize so that things like /tmp/../etc resolve against the real prefix
|
||||||
|
# check below — without this, a path-traversal payload would slip past the
|
||||||
|
# /tmp/* match and the cleanup branches could rm-rf an unintended host path.
|
||||||
|
BUILD_BASE=$(realpath -m -- "$BUILD_BASE")
|
||||||
|
ROOTFS="${BUILD_BASE}/rootfs"
|
||||||
|
|
||||||
|
if (( ${#BUILD_BASE} < 6 )); then
|
||||||
|
echo "[ERROR] BUILD_BASE='${BUILD_BASE}' is too short — refusing for safety." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [[ "${BUILD_BASE}" != /tmp/* && "${I_KNOW_WHAT_IM_DOING:-0}" != "1" ]]; then
|
||||||
|
echo "[ERROR] BUILD_BASE='${BUILD_BASE}' resolves outside /tmp." >&2
|
||||||
|
echo " Re-run with I_KNOW_WHAT_IM_DOING=1 to use a non-/tmp path." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$BUILD_BASE"
|
||||||
|
|
||||||
|
# vgmstream tag and the immutable commit it MUST resolve to. A tag is a
|
||||||
|
# movable Git ref upstream — pinning the expected SHA and verifying after
|
||||||
|
# clone catches a re-tag without losing the shallow-clone optimisation.
|
||||||
|
VGMSTREAM_REF="r2083"
|
||||||
|
VGMSTREAM_COMMIT="57df2e179d929532094f4e4dd42ce5395514622b"
|
||||||
|
VGMSTREAM_REPO="https://github.com/vgmstream/vgmstream.git"
|
||||||
|
# Static ffmpeg binaries from BtbN/FFmpeg-Builds (GPL, 7.1 series).
|
||||||
|
# To bump: pick a new autobuild-* tag from
|
||||||
|
# https://github.com/BtbN/FFmpeg-Builds/releases
|
||||||
|
# download the two linux gpl-7.1 tarballs, re-run
|
||||||
|
# sha256sum ffmpeg-*-linux{64,arm64}-gpl-7.1.tar.xz
|
||||||
|
# and update FFMPEG_RELEASE + both builds + hashes below.
|
||||||
|
FFMPEG_RELEASE="autobuild-2026-06-01-15-02"
|
||||||
|
FFMPEG_BUILD_AMD64="ffmpeg-n7.1.4-7-gadcf20da26-linux64-gpl-7.1.tar.xz"
|
||||||
|
FFMPEG_BUILD_ARM64="ffmpeg-n7.1.4-7-gadcf20da26-linuxarm64-gpl-7.1.tar.xz"
|
||||||
|
FFMPEG_SHA256_AMD64=afde55344990650c117fbb7cb36b38d2ab6790b06beb06a9c43a9300c9ce277a
|
||||||
|
FFMPEG_SHA256_ARM64=03c8a7d9a7cf48d017a22a7c31acfdc8e76c5cb193923f883b0338c7baf0bd28
|
||||||
|
|
||||||
|
APP_DIR="/app"
|
||||||
|
VENV_DIR="/opt/app-venv"
|
||||||
|
PIP_VERSION="26.1.1"
|
||||||
|
DLC_DIR="/dlc"
|
||||||
|
CONFIG_DIR="/config"
|
||||||
|
SVC_USER="slopsmith"
|
||||||
|
|
||||||
|
# Coloured logging
|
||||||
|
info() { echo -e "\033[1;34m[INFO]\033[0m $*"; }
|
||||||
|
ok() { echo -e "\033[1;32m[OK]\033[0m $*"; }
|
||||||
|
warn() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
|
||||||
|
die() { echo -e "\033[1;31m[ERROR]\033[0m $*" >&2; exit 1; }
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
local rc=$?
|
||||||
|
if [[ $rc -ne 0 && -d "${BUILD_BASE:-}" ]]; then
|
||||||
|
warn "Build failed (exit $rc). Partial rootfs left at ${BUILD_BASE} for inspection."
|
||||||
|
# Use printf directly: warn() pipes through `echo -e`, which would
|
||||||
|
# re-interpret the backslash escapes that `printf %q` emits and
|
||||||
|
# silently break the suggested cleanup command.
|
||||||
|
printf "\033[1;33m[WARN]\033[0m Run: sudo rm -rf %q\n" "${BUILD_BASE}"
|
||||||
|
elif [[ $rc -eq 0 && -d "${BUILD_BASE:-}" && "${KEEP_BUILD_DIR:-0}" != "1" ]]; then
|
||||||
|
info "Removing build directory ${BUILD_BASE} (set KEEP_BUILD_DIR=1 to retain)."
|
||||||
|
rm -rf "${BUILD_BASE}"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
# Verify a downloaded file against a pinned SHA256 hash.
|
||||||
|
# Skips verification when the expected hash is empty (not yet pinned).
|
||||||
|
verify_sha256() {
|
||||||
|
local file="$1" expected="$2" label="${3:-$1}"
|
||||||
|
if [[ -z "$expected" ]]; then
|
||||||
|
if [[ "${SKIP_HASH_CHECK:-0}" != "1" ]]; then
|
||||||
|
die "No SHA256 pinned for ${label}. Pin the hash or set SKIP_HASH_CHECK=1 to proceed."
|
||||||
|
fi
|
||||||
|
warn "No SHA256 pinned for ${label} — skipping verification (SKIP_HASH_CHECK=1)."
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
local actual
|
||||||
|
actual=$(sha256sum "$file" | awk '{print $1}')
|
||||||
|
if [[ "$actual" != "$expected" ]]; then
|
||||||
|
if [[ "${SKIP_HASH_CHECK:-0}" == "1" ]]; then
|
||||||
|
warn "SHA256 mismatch for ${label} (expected ${expected}, got ${actual}) — continuing because SKIP_HASH_CHECK=1."
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
die "SHA256 mismatch for ${label}:\n" \
|
||||||
|
" expected: ${expected}\n" \
|
||||||
|
" got: ${actual}\n" \
|
||||||
|
" Refresh the pinned hash, or set SKIP_HASH_CHECK=1 to bypass."
|
||||||
|
fi
|
||||||
|
ok "SHA256 verified for ${label}."
|
||||||
|
}
|
||||||
|
|
||||||
|
[[ $EUID -eq 0 ]] || die "Run as root: sudo bash $0"
|
||||||
|
|
||||||
|
case "$TARGETARCH" in
|
||||||
|
arm64) RID="linux-arm64" ; DEBIAN_ARCH="arm64" ;;
|
||||||
|
amd64) RID="linux-x64" ; DEBIAN_ARCH="amd64" ;;
|
||||||
|
*) die "Unsupported TARGETARCH: ${TARGETARCH}. Expected: amd64 | arm64" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# arm64 cross-builds require qemu-user-static + a registered binfmt handler.
|
||||||
|
# Checking the binary alone isn't enough — without a registered+enabled handler
|
||||||
|
# debootstrap/nspawn fail later with "exec format error" after significant
|
||||||
|
# wasted setup time.
|
||||||
|
if [[ "$TARGETARCH" == "arm64" && "$(uname -m)" != "aarch64" ]]; then
|
||||||
|
if ! command -v qemu-aarch64-static &>/dev/null; then
|
||||||
|
die "arm64 builds on a non-arm64 host require qemu-user-static.\n" \
|
||||||
|
" Install with: sudo apt install qemu-user-static binfmt-support\n" \
|
||||||
|
" Then re-run this script."
|
||||||
|
fi
|
||||||
|
binfmt_reg=""
|
||||||
|
for f in /proc/sys/fs/binfmt_misc/qemu-aarch64 \
|
||||||
|
/proc/sys/fs/binfmt_misc/qemu-aarch64-static; do
|
||||||
|
[[ -f "$f" ]] && grep -q '^enabled' "$f" 2>/dev/null && { binfmt_reg="$f"; break; }
|
||||||
|
done
|
||||||
|
if [[ -z "$binfmt_reg" ]]; then
|
||||||
|
die "arm64 binfmt handler not registered or not enabled.\n" \
|
||||||
|
" Register with: sudo apt install qemu-user-static binfmt-support\n" \
|
||||||
|
" Or: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes\n" \
|
||||||
|
" Then verify: grep ^enabled /proc/sys/fs/binfmt_misc/qemu-aarch64*"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Confirm required tools
|
||||||
|
for cmd in debootstrap systemd-nspawn curl unzip git tar zstd; do
|
||||||
|
command -v "$cmd" &>/dev/null || die "'$cmd' not found. Run: sudo apt install debootstrap systemd-container curl unzip git tar zstd"
|
||||||
|
done
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# Pre-flight: verify the pinned BtbN FFmpeg release still exists
|
||||||
|
# =============================================================================
|
||||||
|
# BtbN only keeps ~10 days of autobuilds. A stale FFMPEG_RELEASE means
|
||||||
|
# the build will 404 deep into step 5b after significant setup work.
|
||||||
|
# Fail fast with actionable instructions instead.
|
||||||
|
info "Checking ffmpeg release availability …"
|
||||||
|
case "$TARGETARCH" in
|
||||||
|
arm64) _preflight_tarball="${FFMPEG_BUILD_ARM64}" ;;
|
||||||
|
amd64) _preflight_tarball="${FFMPEG_BUILD_AMD64}" ;;
|
||||||
|
esac
|
||||||
|
_preflight_url="https://github.com/BtbN/FFmpeg-Builds/releases/download/${FFMPEG_RELEASE}/${_preflight_tarball}"
|
||||||
|
_http_code=$(curl -sL -o /dev/null -w '%{http_code}' --head "${_preflight_url}" || true)
|
||||||
|
if [[ ! "$_http_code" =~ ^2[0-9]{2}$ ]]; then
|
||||||
|
die "Pinned ffmpeg release is no longer available (HTTP ${_http_code}).
|
||||||
|
|
||||||
|
URL: ${_preflight_url}
|
||||||
|
|
||||||
|
BtbN/FFmpeg-Builds only keeps ~10 days of autobuilds.
|
||||||
|
To fix, update these variables in build-proxmox-ct.sh:
|
||||||
|
|
||||||
|
1. Pick a current release tag from:
|
||||||
|
https://github.com/BtbN/FFmpeg-Builds/releases
|
||||||
|
|
||||||
|
2. Update FFMPEG_RELEASE to the new tag
|
||||||
|
(e.g. autobuild-YYYY-MM-DD-HH-MM)
|
||||||
|
|
||||||
|
3. Update FFMPEG_BUILD_AMD64 and FFMPEG_BUILD_ARM64
|
||||||
|
to the new *-linux64-gpl-7.1.tar.xz and
|
||||||
|
*-linuxarm64-gpl-7.1.tar.xz filenames
|
||||||
|
|
||||||
|
4. Update FFMPEG_SHA256_AMD64 and FFMPEG_SHA256_ARM64
|
||||||
|
from the checksums.sha256 file in that release
|
||||||
|
|
||||||
|
Also update the same ARGs in Dockerfile"
|
||||||
|
fi
|
||||||
|
ok "ffmpeg release verified (HTTP ${_http_code})."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# Helper: run a command inside the rootfs via systemd-nspawn
|
||||||
|
# --quiet suppresses nspawn chatter so apt/cmake output is the only
|
||||||
|
# thing we see during the build. The host's /etc/resolv.conf is
|
||||||
|
# bind-mounted read-only so DNS works inside nspawn.
|
||||||
|
# =============================================================================
|
||||||
|
r() {
|
||||||
|
systemd-nspawn \
|
||||||
|
--quiet \
|
||||||
|
--directory="$ROOTFS" \
|
||||||
|
--bind-ro=/etc/resolv.conf:/etc/resolv.conf \
|
||||||
|
-- bash -c "set -e; $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 1. Bootstrap a minimal Debian Trixie rootfs
|
||||||
|
# =============================================================================
|
||||||
|
info "Bootstrapping Debian Trixie (${DEBIAN_ARCH}) rootfs at ${ROOTFS} …"
|
||||||
|
if [[ -d "$ROOTFS" ]]; then
|
||||||
|
if [[ "${FORCE_REBUILD:-0}" == "1" ]]; then
|
||||||
|
info "FORCE_REBUILD=1 — removing existing rootfs at ${ROOTFS}."
|
||||||
|
rm -rf "$ROOTFS" || die "Failed to remove existing rootfs at ${ROOTFS}."
|
||||||
|
elif [[ -t 0 ]]; then
|
||||||
|
warn "Existing rootfs found at ${ROOTFS} – remove it to rebuild from scratch."
|
||||||
|
read -rp " Delete and rebuild? [y/N] " yn
|
||||||
|
if [[ "$yn" =~ ^[Yy]$ ]]; then
|
||||||
|
rm -rf "$ROOTFS" || die "Failed to remove existing rootfs at ${ROOTFS}."
|
||||||
|
else
|
||||||
|
die "Aborting."
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
die "Existing rootfs at ${ROOTFS}; rerun with FORCE_REBUILD=1 to overwrite."
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
debootstrap \
|
||||||
|
--arch="$DEBIAN_ARCH" \
|
||||||
|
--include=ca-certificates,curl,gnupg \
|
||||||
|
trixie \
|
||||||
|
"$ROOTFS" \
|
||||||
|
https://deb.debian.org/debian
|
||||||
|
|
||||||
|
ok "Bootstrap complete."
|
||||||
|
|
||||||
|
# DNS during the build is supplied by the host's /etc/resolv.conf, which
|
||||||
|
# r() bind-mounts read-only into nspawn. The rootfs's own /etc/resolv.conf
|
||||||
|
# gets replaced with a systemd-resolved stub symlink in step 10(d).
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 2. System packages (mirrors Stage 2 apt block)
|
||||||
|
# =============================================================================
|
||||||
|
info "Installing system packages …"
|
||||||
|
# systemd-sysv + systemd-resolved are explicit because the final container
|
||||||
|
# enables systemd-networkd/systemd-resolved units in step 10 and rewrites
|
||||||
|
# /etc/resolv.conf to the resolved stub — a minimal debootstrap does not
|
||||||
|
# guarantee these binaries on its own, which would yield broken DNS in the
|
||||||
|
# imported CT.
|
||||||
|
#
|
||||||
|
# NOTE: ffmpeg is NOT installed via apt — a static binary is copied in
|
||||||
|
# step 5b instead, avoiding the huge codec + TLS dependency tree.
|
||||||
|
# vgmstream-cli is built from source in step 5, needing runtime libs:
|
||||||
|
# libmpg123-0, libvorbisfile3, libspeex1, libopus0.
|
||||||
|
#
|
||||||
|
# Node.js: Debian Trixie ships Node 20 by default. We use extrepo to enable
|
||||||
|
# the official NodeSource Node 22.x repository for LTS support.
|
||||||
|
r "apt-get update -qq \
|
||||||
|
&& apt-get -y upgrade \
|
||||||
|
&& apt-get install -y --no-install-recommends \
|
||||||
|
systemd-sysv systemd-resolved \
|
||||||
|
python3 python3-pip python3-venv \
|
||||||
|
fluidsynth \
|
||||||
|
fluid-soundfont-gm \
|
||||||
|
libsndfile1 \
|
||||||
|
libmpg123-0 \
|
||||||
|
libvorbisfile3 \
|
||||||
|
libspeex1 \
|
||||||
|
libopus0 \
|
||||||
|
libstdc++6 \
|
||||||
|
libgcc-s1 \
|
||||||
|
extrepo \
|
||||||
|
&& apt-get clean && rm -rf /var/lib/apt/lists/*"
|
||||||
|
ok "System packages installed."
|
||||||
|
|
||||||
|
info "Enabling Node.js 22.x repository via extrepo …"
|
||||||
|
r "extrepo enable node_22.x \
|
||||||
|
&& apt-get update -qq \
|
||||||
|
&& apt-get install -y --no-install-recommends nodejs \
|
||||||
|
&& apt-get clean && rm -rf /var/lib/apt/lists/*"
|
||||||
|
ok "Node.js 22.x installed."
|
||||||
|
|
||||||
|
# Build-time dependencies for compiling vgmstream-cli from source (step 5).
|
||||||
|
# Purged after the build in step 5c to keep the rootfs lean.
|
||||||
|
info "Installing vgmstream build dependencies (temporary) …"
|
||||||
|
# `git` intentionally absent — vgmstream is cloned host-side and the build
|
||||||
|
# itself uses cmake only, so the container never invokes git. Adding it back
|
||||||
|
# would pull in libcurl/libexpat unnecessarily.
|
||||||
|
r "apt-get update -qq && apt-get install -y --no-install-recommends \
|
||||||
|
build-essential cmake pkg-config yasm \
|
||||||
|
libmpg123-dev libvorbis-dev libspeex-dev libopus-dev \
|
||||||
|
&& apt-get clean && rm -rf /var/lib/apt/lists/*"
|
||||||
|
ok "Build dependencies installed."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 5. Build vgmstream-cli from source (mirrors Dockerfile stage 1b)
|
||||||
|
# =============================================================================
|
||||||
|
info "Building vgmstream-cli from source (${VGMSTREAM_REF}) …"
|
||||||
|
# Clone into /root/ — systemd-nspawn mounts a private tmpfs over /tmp,
|
||||||
|
# hiding anything placed at ${ROOTFS}/tmp/ by the host.
|
||||||
|
rm -rf "${ROOTFS}/root/vgmstream"
|
||||||
|
git clone --depth 1 --branch "${VGMSTREAM_REF}" "${VGMSTREAM_REPO}" "${ROOTFS}/root/vgmstream"
|
||||||
|
# Verify the tag still points at the pinned commit — a re-tag upstream
|
||||||
|
# would otherwise silently change what we ship.
|
||||||
|
_vgmstream_head=$(git -C "${ROOTFS}/root/vgmstream" rev-parse HEAD)
|
||||||
|
if [[ "${_vgmstream_head}" != "${VGMSTREAM_COMMIT}" ]]; then
|
||||||
|
die "vgmstream tag ${VGMSTREAM_REF} resolves to ${_vgmstream_head}, expected ${VGMSTREAM_COMMIT}. Tag may have moved upstream — verify and update VGMSTREAM_COMMIT."
|
||||||
|
fi
|
||||||
|
|
||||||
|
r "cmake -S /root/vgmstream -B /root/vgmstream/build \
|
||||||
|
-DCMAKE_BUILD_TYPE=Release \
|
||||||
|
-DBUILD_V123=OFF \
|
||||||
|
-DBUILD_AUDACIOUS=OFF \
|
||||||
|
-DBUILD_SHARED_LIBS=OFF \
|
||||||
|
-DUSE_FFMPEG=OFF \
|
||||||
|
&& cmake --build /root/vgmstream/build --config Release --target vgmstream_cli -j\$(nproc) \
|
||||||
|
&& cp /root/vgmstream/build/cli/vgmstream-cli /usr/local/bin/vgmstream-cli \
|
||||||
|
&& chmod +x /usr/local/bin/vgmstream-cli"
|
||||||
|
rm -rf "${ROOTFS}/root/vgmstream"
|
||||||
|
ok "vgmstream-cli built."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 5b. Static ffmpeg (mirrors Dockerfile stage 1c — BtbN GPL build)
|
||||||
|
# =============================================================================
|
||||||
|
info "Installing static ffmpeg …"
|
||||||
|
case "$TARGETARCH" in
|
||||||
|
arm64) FFMPEG_TARBALL="${FFMPEG_BUILD_ARM64}"; FFMPEG_SHA256="${FFMPEG_SHA256_ARM64}" ;;
|
||||||
|
amd64) FFMPEG_TARBALL="${FFMPEG_BUILD_AMD64}"; FFMPEG_SHA256="${FFMPEG_SHA256_AMD64}" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
curl -fsSL "https://github.com/BtbN/FFmpeg-Builds/releases/download/${FFMPEG_RELEASE}/${FFMPEG_TARBALL}" \
|
||||||
|
-o "${BUILD_BASE}/ffmpeg.tar.xz"
|
||||||
|
verify_sha256 "${BUILD_BASE}/ffmpeg.tar.xz" "${FFMPEG_SHA256}" "ffmpeg-static (${TARGETARCH})"
|
||||||
|
|
||||||
|
mkdir -p "${BUILD_BASE}/ffmpeg-extract"
|
||||||
|
tar -xJf "${BUILD_BASE}/ffmpeg.tar.xz" -C "${BUILD_BASE}/ffmpeg-extract" --strip-components=1
|
||||||
|
cp "${BUILD_BASE}/ffmpeg-extract/bin/ffmpeg" "${ROOTFS}/usr/local/bin/ffmpeg"
|
||||||
|
cp "${BUILD_BASE}/ffmpeg-extract/bin/ffprobe" "${ROOTFS}/usr/local/bin/ffprobe"
|
||||||
|
chmod +x "${ROOTFS}/usr/local/bin/ffmpeg" "${ROOTFS}/usr/local/bin/ffprobe"
|
||||||
|
|
||||||
|
# GPL compliance — ship the license text AND a written offer pointing
|
||||||
|
# at where the corresponding source can be obtained, so redistributing
|
||||||
|
# the resulting CT template doesn't strand recipients without access to
|
||||||
|
# the source for the GPL-licensed binary. LICENSE.txt alone covers the
|
||||||
|
# license terms; the SOURCE file satisfies the "corresponding source"
|
||||||
|
# availability requirement of GPLv3 §6 / GPLv2 §3.
|
||||||
|
mkdir -p "${ROOTFS}/usr/share/doc/ffmpeg"
|
||||||
|
cp "${BUILD_BASE}/ffmpeg-extract/LICENSE.txt" "${ROOTFS}/usr/share/doc/ffmpeg/LICENSE.txt"
|
||||||
|
cat > "${ROOTFS}/usr/share/doc/ffmpeg/SOURCE" <<EOF
|
||||||
|
This ffmpeg/ffprobe binary is a static GPL build from the BtbN/FFmpeg-Builds
|
||||||
|
project (https://github.com/BtbN/FFmpeg-Builds). The exact build artefact
|
||||||
|
shipped here is:
|
||||||
|
|
||||||
|
Release: ${FFMPEG_RELEASE}
|
||||||
|
Tarball: ${FFMPEG_TARBALL}
|
||||||
|
SHA-256: ${FFMPEG_SHA256}
|
||||||
|
|
||||||
|
Corresponding source code (per GPL):
|
||||||
|
- Build recipe + scripts: https://github.com/BtbN/FFmpeg-Builds/tree/${FFMPEG_RELEASE}
|
||||||
|
- Upstream FFmpeg source: https://github.com/FFmpeg/FFmpeg (the FFmpeg
|
||||||
|
commit baked into this build is identified by the n7.1.x version tag
|
||||||
|
in the tarball filename, e.g. n7.1.4-5-ged860ef7d9 → upstream commit
|
||||||
|
ed860ef7d9).
|
||||||
|
|
||||||
|
If either URL becomes unavailable, the maintainer of this CT template will
|
||||||
|
make the corresponding source available on request for at least three years,
|
||||||
|
per GPLv2 §3(b) / GPLv3 §6(b).
|
||||||
|
EOF
|
||||||
|
|
||||||
|
rm -rf "${BUILD_BASE}/ffmpeg-extract" "${BUILD_BASE}/ffmpeg.tar.xz"
|
||||||
|
ok "Static ffmpeg installed."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 5c. Clean up build-time dependencies
|
||||||
|
# =============================================================================
|
||||||
|
info "Removing build-time dependencies …"
|
||||||
|
r "apt-get purge -y --auto-remove \
|
||||||
|
build-essential cmake pkg-config yasm \
|
||||||
|
libmpg123-dev libvorbis-dev libspeex-dev libopus-dev \
|
||||||
|
&& apt-get clean && rm -rf /var/lib/apt/lists/*"
|
||||||
|
ok "Build dependencies removed."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 5d. Tailwind CLI for runtime stylesheet regeneration
|
||||||
|
# =============================================================================
|
||||||
|
# When a plugin is installed into SLOPSMITH_PLUGINS_DIR at runtime (or
|
||||||
|
# discovered there on startup), the server rebuilds static/tailwind.min.css
|
||||||
|
# so the plugin's classes are styled — the image-baked sheet only covers
|
||||||
|
# in-tree plugins (see lib/tailwind_rebuild.py). tailwindcss is installed
|
||||||
|
# globally so the rebuild runs offline, with no npx fetch at install time.
|
||||||
|
# nodejs + npm were installed in step 2 via apt.
|
||||||
|
info "Installing Tailwind CLI globally …"
|
||||||
|
r "npm install -g tailwindcss@3.4.19 \
|
||||||
|
&& npm cache clean --force"
|
||||||
|
ok "Tailwind CLI installed."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 6. Python application
|
||||||
|
# =============================================================================
|
||||||
|
info "Setting up Python application …"
|
||||||
|
mkdir -p \
|
||||||
|
"${ROOTFS}${APP_DIR}/lib" \
|
||||||
|
"${ROOTFS}${APP_DIR}/static" \
|
||||||
|
"${ROOTFS}${APP_DIR}/plugins"
|
||||||
|
|
||||||
|
for d in lib static plugins; do
|
||||||
|
if [[ -d "$d" ]]; then
|
||||||
|
cp -r "${d}/." "${ROOTFS}${APP_DIR}/${d}/"
|
||||||
|
info " Copied ${d}/"
|
||||||
|
else
|
||||||
|
# main.py imports logging_setup from lib/; server.py imports plugins.
|
||||||
|
# Without these the rootfs boots and the service crashes immediately.
|
||||||
|
if [[ "$d" == "lib" || "$d" == "plugins" ]]; then
|
||||||
|
die " '${d}/' not found — required for the service to import."
|
||||||
|
fi
|
||||||
|
warn " Local '${d}/' not found – skipping."
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
for f in requirements.txt server.py VERSION main.py tailwind.config.js; do
|
||||||
|
if [[ -f "$f" ]]; then
|
||||||
|
cp "$f" "${ROOTFS}${APP_DIR}/"
|
||||||
|
info " Copied ${f}"
|
||||||
|
else
|
||||||
|
# main.py imports `server:app`, so without server.py the service unit
|
||||||
|
# would boot but fail on first request — make it fail-fast at build.
|
||||||
|
if [[ "$f" == "requirements.txt" || "$f" == "main.py" || "$f" == "server.py" ]]; then
|
||||||
|
die " '${f}' not found — required for the service to start."
|
||||||
|
fi
|
||||||
|
warn " '${f}' not found – skipping."
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
info "Creating Python venv and installing dependencies …"
|
||||||
|
r "python3 -m venv ${VENV_DIR} \
|
||||||
|
&& ${VENV_DIR}/bin/pip install --no-cache-dir 'pip==${PIP_VERSION}' \
|
||||||
|
&& ${VENV_DIR}/bin/pip install --no-cache-dir -r ${APP_DIR}/requirements.txt"
|
||||||
|
ok "Python venv + dependencies installed."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 6b. Build Tailwind stylesheet over the full plugin set
|
||||||
|
# =============================================================================
|
||||||
|
# The committed static/tailwind.min.css is generated against only the in-tree
|
||||||
|
# plugins. Rebuild it here after static/ + plugins/ are in place so the sheet
|
||||||
|
# covers whatever plugins are baked into the rootfs. Mirrors Dockerfile
|
||||||
|
# stage 1d. Runtime-installed plugins are handled by the server's rebuild.
|
||||||
|
info "Building Tailwind stylesheet …"
|
||||||
|
r "cd ${APP_DIR} \
|
||||||
|
&& tailwindcss \
|
||||||
|
-c tailwind.config.js \
|
||||||
|
-i static/_tailwind.src.css \
|
||||||
|
-o static/tailwind.min.css \
|
||||||
|
--minify"
|
||||||
|
ok "Tailwind stylesheet built."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 7. Data directories + assets
|
||||||
|
# =============================================================================
|
||||||
|
info "Populating data directories …"
|
||||||
|
# The library folder (${DLC_DIR}) is created empty — mount or copy your
|
||||||
|
# .sloppak / loose-folder library into it after the CT is imported.
|
||||||
|
mkdir -p "${ROOTFS}${CONFIG_DIR}" "${ROOTFS}${DLC_DIR}"
|
||||||
|
|
||||||
|
if [[ -d "config" ]]; then
|
||||||
|
cp -r config/. "${ROOTFS}${CONFIG_DIR}/"
|
||||||
|
info " Copied config/"
|
||||||
|
else
|
||||||
|
warn " config/ not found."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 8. Environment variables
|
||||||
|
# =============================================================================
|
||||||
|
info "Writing /etc/environment …"
|
||||||
|
cat > "${ROOTFS}/etc/environment" <<EOF
|
||||||
|
PATH=${VENV_DIR}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||||
|
PYTHONPATH=${APP_DIR}/lib:${APP_DIR}
|
||||||
|
DLC_DIR=${DLC_DIR}
|
||||||
|
CONFIG_DIR=${CONFIG_DIR}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 9. systemd service for uvicorn
|
||||||
|
# =============================================================================
|
||||||
|
info "Creating service user '${SVC_USER}' …"
|
||||||
|
r "useradd --system --home-dir ${APP_DIR} --shell /usr/sbin/nologin ${SVC_USER}"
|
||||||
|
ok "User '${SVC_USER}' created."
|
||||||
|
|
||||||
|
info "Installing slopsmith-server.service …"
|
||||||
|
mkdir -p "${ROOTFS}/etc/systemd/system"
|
||||||
|
cat > "${ROOTFS}/etc/systemd/system/slopsmith-server.service" <<EOF
|
||||||
|
[Unit]
|
||||||
|
Description=Slopsmith uvicorn server
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
User=${SVC_USER}
|
||||||
|
# Default port (8000) is non-privileged; uncomment the next line only if
|
||||||
|
# you set PORT<1024 in /etc/environment so the unit can bind it.
|
||||||
|
# AmbientCapabilities=CAP_NET_BIND_SERVICE
|
||||||
|
WorkingDirectory=${APP_DIR}
|
||||||
|
EnvironmentFile=/etc/environment
|
||||||
|
ExecStart=${VENV_DIR}/bin/python3 main.py
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Enable by symlinking (avoids running systemctl inside nspawn)
|
||||||
|
mkdir -p "${ROOTFS}/etc/systemd/system/multi-user.target.wants"
|
||||||
|
ln -sf /etc/systemd/system/slopsmith-server.service \
|
||||||
|
"${ROOTFS}/etc/systemd/system/multi-user.target.wants/slopsmith-server.service"
|
||||||
|
ok "Service enabled."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 10. Proxmox-specific tweaks
|
||||||
|
# =============================================================================
|
||||||
|
info "Applying Proxmox CT compatibility tweaks …"
|
||||||
|
|
||||||
|
# (a) Ensure a working /etc/hostname and /etc/hosts.
|
||||||
|
# Use OUTPUT_NAME (already validated to a safe filename charset) so the
|
||||||
|
# template's identity matches the artifact name. This is just a sane
|
||||||
|
# fallback — `pct restore --hostname …` (or the Proxmox UI) will overwrite
|
||||||
|
# /etc/hostname when the CT is created from this template.
|
||||||
|
DEFAULT_HOSTNAME="${OUTPUT_NAME//_/-}" # underscores aren't valid in hostnames
|
||||||
|
echo "${DEFAULT_HOSTNAME}" > "${ROOTFS}/etc/hostname"
|
||||||
|
cat > "${ROOTFS}/etc/hosts" <<EOF
|
||||||
|
127.0.0.1 localhost
|
||||||
|
127.0.1.1 ${DEFAULT_HOSTNAME}
|
||||||
|
::1 localhost ip6-localhost ip6-loopback
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# (b) Clear machine-id so Proxmox generates a fresh one on first boot
|
||||||
|
# A pre-filled machine-id can cause network/systemd conflicts across clones.
|
||||||
|
echo -n > "${ROOTFS}/etc/machine-id"
|
||||||
|
[[ -f "${ROOTFS}/var/lib/dbus/machine-id" ]] && echo -n > "${ROOTFS}/var/lib/dbus/machine-id"
|
||||||
|
|
||||||
|
# (c) DHCP networking via systemd-networkd (Proxmox expects this for unprivileged CTs)
|
||||||
|
mkdir -p "${ROOTFS}/etc/systemd/network"
|
||||||
|
cat > "${ROOTFS}/etc/systemd/network/20-eth0.network" <<EOF
|
||||||
|
[Match]
|
||||||
|
Name=eth0
|
||||||
|
|
||||||
|
[Network]
|
||||||
|
DHCP=yes
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Enable via symlinks on the host – systemctl inside nspawn needs a running
|
||||||
|
# init which WSL doesn't provide.
|
||||||
|
|
||||||
|
mkdir -p "${ROOTFS}/etc/systemd/system/multi-user.target.wants"
|
||||||
|
for svc in systemd-networkd systemd-resolved; do
|
||||||
|
unit_src=""
|
||||||
|
for d in /lib/systemd/system /usr/lib/systemd/system; do
|
||||||
|
if [[ -e "${ROOTFS}${d}/${svc}.service" ]]; then
|
||||||
|
unit_src="${d}/${svc}.service"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [[ -z "$unit_src" ]]; then
|
||||||
|
die "${svc}.service unit not found in rootfs — DNS/networking would be broken in the imported CT."
|
||||||
|
fi
|
||||||
|
ln -sf "${unit_src}" "${ROOTFS}/etc/systemd/system/multi-user.target.wants/${svc}.service"
|
||||||
|
done
|
||||||
|
|
||||||
|
# (d) Ensure correct permissions on key dirs.
|
||||||
|
# -h preserves symlinks: a Python venv keeps /opt/app-venv/bin/python3 as a
|
||||||
|
# symlink to /usr/bin/python3, and a plain `chown -R` would chase it and
|
||||||
|
# rewrite the system interpreter's ownership inside the rootfs.
|
||||||
|
SVC_UID="$(r "id -u ${SVC_USER}" | tr -d '\r')"
|
||||||
|
SVC_GID="$(r "id -g ${SVC_USER}" | tr -d '\r')"
|
||||||
|
chown -hR "${SVC_UID}:${SVC_GID}" \
|
||||||
|
"${ROOTFS}${APP_DIR}" "${ROOTFS}${CONFIG_DIR}" \
|
||||||
|
"${ROOTFS}${DLC_DIR}" "${ROOTFS}${VENV_DIR}"
|
||||||
|
|
||||||
|
# (e) Fix resolv.conf to use the systemd-resolved stub. MUST run after the
|
||||||
|
# last r() invocation: r() bind-mounts the host /etc/resolv.conf onto the
|
||||||
|
# rootfs path, and systemd-nspawn follows symlinks when resolving the bind
|
||||||
|
# target — pointing /etc/resolv.conf at /run/systemd/resolve/stub-resolv.conf
|
||||||
|
# before that would make subsequent r() calls try to bind onto a path that
|
||||||
|
# doesn't exist during the build.
|
||||||
|
rm -f "${ROOTFS}/etc/resolv.conf"
|
||||||
|
ln -sf /run/systemd/resolve/stub-resolv.conf "${ROOTFS}/etc/resolv.conf"
|
||||||
|
|
||||||
|
ok "Proxmox tweaks applied."
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# 11. Package as a Proxmox-importable .tar.zst
|
||||||
|
# =============================================================================
|
||||||
|
OUTPUT_FILE="${OUTPUT_NAME}.tar.zst"
|
||||||
|
info "Creating ${OUTPUT_FILE} …"
|
||||||
|
|
||||||
|
# Proxmox pct restore expects a plain rootfs tarball (no ./rootfs/ prefix).
|
||||||
|
tar \
|
||||||
|
--numeric-owner \
|
||||||
|
--xattrs \
|
||||||
|
--acls \
|
||||||
|
-C "$ROOTFS" \
|
||||||
|
-c . \
|
||||||
|
| zstd -T0 -9 > "$OUTPUT_FILE"
|
||||||
|
|
||||||
|
ok "Template ready: $(pwd)/${OUTPUT_FILE} ($(du -sh "$OUTPUT_FILE" | cut -f1))"
|
||||||
|
|
||||||
|
# =============================================================================
|
||||||
|
# Done
|
||||||
|
# =============================================================================
|
||||||
|
cat <<DONE
|
||||||
|
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
Build complete!
|
||||||
|
|
||||||
|
Transfer to Proxmox:
|
||||||
|
scp ${OUTPUT_FILE} root@<proxmox-host>:/var/lib/vz/template/cache/
|
||||||
|
|
||||||
|
Import on Proxmox (pick an unused VMID, e.g. 200):
|
||||||
|
pct restore 200 /var/lib/vz/template/cache/${OUTPUT_FILE} \\
|
||||||
|
--storage local-lvm \\
|
||||||
|
--rootfs 8 \\
|
||||||
|
--memory 2048 \\
|
||||||
|
--cores 2 \\
|
||||||
|
--net0 name=eth0,bridge=vmbr0,ip=dhcp \\
|
||||||
|
--unprivileged 1 \\
|
||||||
|
--start 1
|
||||||
|
|
||||||
|
Then check the server:
|
||||||
|
pct exec 200 -- systemctl status slopsmith-server
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
DONE
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
{
|
||||||
|
"id": "bass",
|
||||||
|
"name": "Bass",
|
||||||
|
"icon": "bass",
|
||||||
|
"order": 2,
|
||||||
|
"levels": [
|
||||||
|
{
|
||||||
|
"level": 1,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "bass.l1.first-groove",
|
||||||
|
"title": "First Groove",
|
||||||
|
"description": "Finish any bass song with note detection on.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l1.locked-in",
|
||||||
|
"title": "Locked In",
|
||||||
|
"description": "Score 80%+ accuracy on a bass song.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "min_accuracy": 0.8, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l1.daily-grind",
|
||||||
|
"title": "Daily Grind",
|
||||||
|
"description": "Complete 3 daily quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "daily", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 2,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "bass.l2.five-songs",
|
||||||
|
"title": "In the Pocket",
|
||||||
|
"description": "Finish 5 bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l2.tight",
|
||||||
|
"title": "Tight",
|
||||||
|
"description": "Score 90%+ accuracy on a bass song.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "min_accuracy": 0.9, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l2.arcade-debut",
|
||||||
|
"title": "Arcade Debut",
|
||||||
|
"description": "Play 3 FeedBarcade rounds.",
|
||||||
|
"goal": { "type": "minigame_run", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 3,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "bass.l3.repertoire",
|
||||||
|
"title": "Repertoire",
|
||||||
|
"description": "Finish 10 different bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "distinct": true, "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l3.consistent",
|
||||||
|
"title": "Consistent",
|
||||||
|
"description": "Score 85%+ accuracy on 5 bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "min_accuracy": 0.85, "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l3.weekly-warrior",
|
||||||
|
"title": "Weekly Warrior",
|
||||||
|
"description": "Complete 2 weekly quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "weekly", "target": 2 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 4,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "bass.l4.streak-week",
|
||||||
|
"title": "Seven-Day Streak",
|
||||||
|
"description": "Reach a 7-day play streak.",
|
||||||
|
"goal": { "type": "streak_reached", "days": 7 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l4.precision",
|
||||||
|
"title": "Precision",
|
||||||
|
"description": "Score 95%+ accuracy on 3 bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "min_accuracy": 0.95, "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l4.marathon",
|
||||||
|
"title": "Marathon",
|
||||||
|
"description": "Finish 25 bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "target": 25 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 5,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "bass.l5.collector",
|
||||||
|
"title": "Collector",
|
||||||
|
"description": "Earn 5,000 lifetime Decibels.",
|
||||||
|
"goal": { "type": "db_earned", "amount": 5000 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l5.virtuoso",
|
||||||
|
"title": "Low-End Virtuoso",
|
||||||
|
"description": "Score 95%+ accuracy on 10 different bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "min_accuracy": 0.95, "distinct": true, "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "bass.l5.dedicated",
|
||||||
|
"title": "Dedicated",
|
||||||
|
"description": "Finish 50 bass songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "target": 50 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
{
|
||||||
|
"id": "drums",
|
||||||
|
"name": "Drums",
|
||||||
|
"icon": "drums",
|
||||||
|
"order": 3,
|
||||||
|
"levels": [
|
||||||
|
{
|
||||||
|
"level": 1,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "drums.l1.arcade-rounds",
|
||||||
|
"title": "Hit the Arcade",
|
||||||
|
"description": "Play 3 FeedBarcade rounds.",
|
||||||
|
"goal": { "type": "minigame_run", "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "drums.l1.daily-pair",
|
||||||
|
"title": "Daily Pair",
|
||||||
|
"description": "Complete 2 daily quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "daily", "target": 2 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "drums.l1.session-time",
|
||||||
|
"title": "Session Time",
|
||||||
|
"description": "Finish 3 songs on any instrument.",
|
||||||
|
"goal": { "type": "songs_played_total", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 2,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "drums.l2.arcade-regular",
|
||||||
|
"title": "Arcade Regular",
|
||||||
|
"description": "Play 10 FeedBarcade rounds.",
|
||||||
|
"goal": { "type": "minigame_run", "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "drums.l2.weekly-one",
|
||||||
|
"title": "Weekly Beat",
|
||||||
|
"description": "Complete a weekly quest.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "weekly", "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "drums.l2.session-grind",
|
||||||
|
"title": "Session Grind",
|
||||||
|
"description": "Finish 10 songs on any instrument.",
|
||||||
|
"goal": { "type": "songs_played_total", "target": 10 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 3,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "drums.l3.streak-five",
|
||||||
|
"title": "Five-Day Streak",
|
||||||
|
"description": "Reach a 5-day play streak.",
|
||||||
|
"goal": { "type": "streak_reached", "days": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "drums.l3.arcade-devotee",
|
||||||
|
"title": "Arcade Devotee",
|
||||||
|
"description": "Play 25 FeedBarcade rounds.",
|
||||||
|
"goal": { "type": "minigame_run", "target": 25 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "drums.l3.collector",
|
||||||
|
"title": "Collector",
|
||||||
|
"description": "Earn 2,500 lifetime Decibels.",
|
||||||
|
"goal": { "type": "db_earned", "amount": 2500 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
{
|
||||||
|
"id": "guitar",
|
||||||
|
"name": "Guitar",
|
||||||
|
"icon": "guitar",
|
||||||
|
"order": 1,
|
||||||
|
"levels": [
|
||||||
|
{
|
||||||
|
"level": 1,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "guitar.l1.first-notes",
|
||||||
|
"title": "First Notes",
|
||||||
|
"description": "Finish any guitar song with note detection on.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l1.clean-run",
|
||||||
|
"title": "Clean Run",
|
||||||
|
"description": "Score 80%+ accuracy on a guitar song.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "min_accuracy": 0.8, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l1.daily-grind",
|
||||||
|
"title": "Daily Grind",
|
||||||
|
"description": "Complete 3 daily quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "daily", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 2,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "guitar.l2.five-songs",
|
||||||
|
"title": "Warming Up",
|
||||||
|
"description": "Finish 5 guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l2.sharpshooter",
|
||||||
|
"title": "Sharpshooter",
|
||||||
|
"description": "Score 90%+ accuracy on a guitar song.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "min_accuracy": 0.9, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l2.arcade-debut",
|
||||||
|
"title": "Arcade Debut",
|
||||||
|
"description": "Play 3 FeedBarcade rounds.",
|
||||||
|
"goal": { "type": "minigame_run", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 3,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "guitar.l3.repertoire",
|
||||||
|
"title": "Repertoire",
|
||||||
|
"description": "Finish 10 different guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "distinct": true, "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l3.consistent",
|
||||||
|
"title": "Consistent",
|
||||||
|
"description": "Score 85%+ accuracy on 5 guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "min_accuracy": 0.85, "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l3.weekly-warrior",
|
||||||
|
"title": "Weekly Warrior",
|
||||||
|
"description": "Complete 2 weekly quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "weekly", "target": 2 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 4,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "guitar.l4.streak-week",
|
||||||
|
"title": "Seven-Day Streak",
|
||||||
|
"description": "Reach a 7-day play streak.",
|
||||||
|
"goal": { "type": "streak_reached", "days": 7 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l4.precision",
|
||||||
|
"title": "Precision",
|
||||||
|
"description": "Score 95%+ accuracy on 3 guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "min_accuracy": 0.95, "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l4.marathon",
|
||||||
|
"title": "Marathon",
|
||||||
|
"description": "Finish 25 guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "target": 25 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 5,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "guitar.l5.collector",
|
||||||
|
"title": "Collector",
|
||||||
|
"description": "Earn 5,000 lifetime Decibels.",
|
||||||
|
"goal": { "type": "db_earned", "amount": 5000 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l5.virtuoso",
|
||||||
|
"title": "Virtuoso",
|
||||||
|
"description": "Score 95%+ accuracy on 10 different guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "min_accuracy": 0.95, "distinct": true, "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "guitar.l5.dedicated",
|
||||||
|
"title": "Dedicated",
|
||||||
|
"description": "Finish 50 guitar songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "guitar", "target": 50 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
{
|
||||||
|
"id": "keys",
|
||||||
|
"name": "Keys",
|
||||||
|
"icon": "keys",
|
||||||
|
"order": 4,
|
||||||
|
"levels": [
|
||||||
|
{
|
||||||
|
"level": 1,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "keys.l1.first-notes",
|
||||||
|
"title": "First Notes",
|
||||||
|
"description": "Finish any keys song with your MIDI keyboard connected.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l1.clean-run",
|
||||||
|
"title": "Clean Run",
|
||||||
|
"description": "Score 80%+ accuracy on a keys song.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "min_accuracy": 0.8, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l1.daily-grind",
|
||||||
|
"title": "Daily Grind",
|
||||||
|
"description": "Complete 3 daily quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "daily", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 2,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "keys.l2.five-songs",
|
||||||
|
"title": "Warming Up",
|
||||||
|
"description": "Finish 5 keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l2.sharpshooter",
|
||||||
|
"title": "Sharpshooter",
|
||||||
|
"description": "Score 90%+ accuracy on a keys song.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "min_accuracy": 0.9, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l2.arcade-debut",
|
||||||
|
"title": "Arcade Debut",
|
||||||
|
"description": "Play 3 FeedBarcade rounds.",
|
||||||
|
"goal": { "type": "minigame_run", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 3,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "keys.l3.repertoire",
|
||||||
|
"title": "Repertoire",
|
||||||
|
"description": "Finish 10 different keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "distinct": true, "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l3.consistent",
|
||||||
|
"title": "Consistent",
|
||||||
|
"description": "Score 85%+ accuracy on 5 keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "min_accuracy": 0.85, "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l3.weekly-warrior",
|
||||||
|
"title": "Weekly Warrior",
|
||||||
|
"description": "Complete 2 weekly quests.",
|
||||||
|
"goal": { "type": "quest_completed", "period": "weekly", "target": 2 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 4,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "keys.l4.streak-week",
|
||||||
|
"title": "Seven-Day Streak",
|
||||||
|
"description": "Reach a 7-day play streak.",
|
||||||
|
"goal": { "type": "streak_reached", "days": 7 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l4.precision",
|
||||||
|
"title": "Precision",
|
||||||
|
"description": "Score 95%+ accuracy on 3 keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "min_accuracy": 0.95, "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l4.marathon",
|
||||||
|
"title": "Marathon",
|
||||||
|
"description": "Finish 25 keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "target": 25 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"level": 5,
|
||||||
|
"required": 2,
|
||||||
|
"challenges": [
|
||||||
|
{
|
||||||
|
"id": "keys.l5.collector",
|
||||||
|
"title": "Collector",
|
||||||
|
"description": "Earn 5,000 lifetime Decibels.",
|
||||||
|
"goal": { "type": "db_earned", "amount": 5000 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l5.virtuoso",
|
||||||
|
"title": "Virtuoso",
|
||||||
|
"description": "Score 95%+ accuracy on 10 different keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "min_accuracy": 0.95, "distinct": true, "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "keys.l5.dedicated",
|
||||||
|
"title": "Dedicated",
|
||||||
|
"description": "Finish 50 keys songs.",
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "target": 50 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
{
|
||||||
|
"daily": {
|
||||||
|
"count": 3,
|
||||||
|
"pool": [
|
||||||
|
{
|
||||||
|
"id": "d.play-two",
|
||||||
|
"title": "Quick Set",
|
||||||
|
"description": "Finish 2 songs with note detection on.",
|
||||||
|
"reward_db": 50,
|
||||||
|
"goal": { "type": "song_completed", "target": 2 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.play-three",
|
||||||
|
"title": "Triple Bill",
|
||||||
|
"description": "Finish 3 songs with note detection on.",
|
||||||
|
"reward_db": 75,
|
||||||
|
"goal": { "type": "song_completed", "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.accuracy-80",
|
||||||
|
"title": "Dialed In",
|
||||||
|
"description": "Score 80%+ accuracy on a song.",
|
||||||
|
"reward_db": 60,
|
||||||
|
"goal": { "type": "song_completed", "min_accuracy": 0.8, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.accuracy-90",
|
||||||
|
"title": "Nailed It",
|
||||||
|
"description": "Score 90%+ accuracy on a song.",
|
||||||
|
"reward_db": 80,
|
||||||
|
"goal": { "type": "song_completed", "min_accuracy": 0.9, "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.two-different",
|
||||||
|
"title": "Mix It Up",
|
||||||
|
"description": "Finish 2 different songs.",
|
||||||
|
"reward_db": 60,
|
||||||
|
"goal": { "type": "song_completed", "distinct": true, "target": 2 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.arcade-one",
|
||||||
|
"title": "Coin-Op",
|
||||||
|
"description": "Play a FeedBarcade round.",
|
||||||
|
"reward_db": 40,
|
||||||
|
"goal": { "type": "minigame_run", "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.arcade-three",
|
||||||
|
"title": "High Score Hunt",
|
||||||
|
"description": "Play 3 FeedBarcade rounds.",
|
||||||
|
"reward_db": 70,
|
||||||
|
"goal": { "type": "minigame_run", "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.bass-one",
|
||||||
|
"title": "Bottom End",
|
||||||
|
"description": "Finish a bass song.",
|
||||||
|
"reward_db": 60,
|
||||||
|
"goal": { "type": "song_completed", "instrument": "bass", "target": 1 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "d.keys-one",
|
||||||
|
"title": "Ivory Tower",
|
||||||
|
"description": "Finish a keys song.",
|
||||||
|
"reward_db": 60,
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "target": 1 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"weekly": {
|
||||||
|
"count": 2,
|
||||||
|
"pool": [
|
||||||
|
{
|
||||||
|
"id": "w.songs-ten",
|
||||||
|
"title": "Ten-Song Week",
|
||||||
|
"description": "Finish 10 songs this week.",
|
||||||
|
"reward_db": 250,
|
||||||
|
"goal": { "type": "song_completed", "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "w.five-different",
|
||||||
|
"title": "Crate Digger",
|
||||||
|
"description": "Finish 5 different songs this week.",
|
||||||
|
"reward_db": 200,
|
||||||
|
"goal": { "type": "song_completed", "distinct": true, "target": 5 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "w.accuracy-90-x3",
|
||||||
|
"title": "Precision Week",
|
||||||
|
"description": "Score 90%+ accuracy on 3 songs this week.",
|
||||||
|
"reward_db": 300,
|
||||||
|
"goal": { "type": "song_completed", "min_accuracy": 0.9, "target": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "w.arcade-ten",
|
||||||
|
"title": "Arcade Marathon",
|
||||||
|
"description": "Play 10 FeedBarcade rounds this week.",
|
||||||
|
"reward_db": 200,
|
||||||
|
"goal": { "type": "minigame_run", "target": 10 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "w.streak-three",
|
||||||
|
"title": "Keep the Streak",
|
||||||
|
"description": "Reach a 3-day play streak.",
|
||||||
|
"reward_db": 150,
|
||||||
|
"goal": { "type": "streak_reached", "days": 3 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "w.keys-three",
|
||||||
|
"title": "Grand Recital",
|
||||||
|
"description": "Finish 3 keys songs this week.",
|
||||||
|
"reward_db": 150,
|
||||||
|
"goal": { "type": "song_completed", "instrument": "keys", "target": 3 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,109 @@
|
|||||||
|
{
|
||||||
|
"items": [
|
||||||
|
{
|
||||||
|
"id": "theme.sunset-strat",
|
||||||
|
"slot": "theme",
|
||||||
|
"name": "Sunset Strat",
|
||||||
|
"description": "Warm amber stage lights over a dusk-dark body.",
|
||||||
|
"cost": 500,
|
||||||
|
"payload": {
|
||||||
|
"colors": {
|
||||||
|
"bg": "#1c1210",
|
||||||
|
"sidebar": "#211512",
|
||||||
|
"card": "#2e1d18",
|
||||||
|
"cardMuted": "#160e0b",
|
||||||
|
"primary": "#f97316",
|
||||||
|
"primaryHi": "#fb923c",
|
||||||
|
"accent": "#ef4444",
|
||||||
|
"text": "#fdf6ef",
|
||||||
|
"textDim": "#b8a193",
|
||||||
|
"border": "#4a3328",
|
||||||
|
"good": "#22c55e",
|
||||||
|
"mid": "#eab308",
|
||||||
|
"low": "#ef4444",
|
||||||
|
"gold": "#e8c040"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "theme.moss-amp",
|
||||||
|
"slot": "theme",
|
||||||
|
"name": "Moss Amp",
|
||||||
|
"description": "Mellow greens like a vintage tolex cab.",
|
||||||
|
"cost": 500,
|
||||||
|
"payload": {
|
||||||
|
"colors": {
|
||||||
|
"bg": "#0f1a14",
|
||||||
|
"sidebar": "#12211a",
|
||||||
|
"card": "#1b2e23",
|
||||||
|
"cardMuted": "#0a140e",
|
||||||
|
"primary": "#10b981",
|
||||||
|
"primaryHi": "#34d399",
|
||||||
|
"accent": "#ef4444",
|
||||||
|
"text": "#f0fdf4",
|
||||||
|
"textDim": "#8fb3a0",
|
||||||
|
"border": "#2e4a3a",
|
||||||
|
"good": "#22c55e",
|
||||||
|
"mid": "#eab308",
|
||||||
|
"low": "#ef4444",
|
||||||
|
"gold": "#e8c040"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "theme.violet-fuzz",
|
||||||
|
"slot": "theme",
|
||||||
|
"name": "Violet Fuzz",
|
||||||
|
"description": "Deep purple haze with a neon bite.",
|
||||||
|
"cost": 750,
|
||||||
|
"payload": {
|
||||||
|
"colors": {
|
||||||
|
"bg": "#16101f",
|
||||||
|
"sidebar": "#1a1226",
|
||||||
|
"card": "#281a38",
|
||||||
|
"cardMuted": "#100a17",
|
||||||
|
"primary": "#a855f7",
|
||||||
|
"primaryHi": "#c084fc",
|
||||||
|
"accent": "#f43f5e",
|
||||||
|
"text": "#faf5ff",
|
||||||
|
"textDim": "#a795bd",
|
||||||
|
"border": "#43305c",
|
||||||
|
"good": "#22c55e",
|
||||||
|
"mid": "#eab308",
|
||||||
|
"low": "#ef4444",
|
||||||
|
"gold": "#e8c040"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "frame.gold-record",
|
||||||
|
"slot": "avatar_frame",
|
||||||
|
"name": "Gold Record",
|
||||||
|
"description": "A solid gold ring for certified players.",
|
||||||
|
"cost": 300,
|
||||||
|
"payload": {
|
||||||
|
"frame_style": "box-shadow: 0 0 0 3px #e8c040, 0 0 12px rgba(232, 192, 64, 0.6)"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "frame.sky-halo",
|
||||||
|
"slot": "avatar_frame",
|
||||||
|
"name": "Sky Halo",
|
||||||
|
"description": "A cool sky-blue glow.",
|
||||||
|
"cost": 300,
|
||||||
|
"payload": {
|
||||||
|
"frame_style": "box-shadow: 0 0 0 3px #38bdf8, 0 0 12px rgba(56, 189, 248, 0.6)"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "frame.redline",
|
||||||
|
"slot": "avatar_frame",
|
||||||
|
"name": "Redline",
|
||||||
|
"description": "Pushed into the red, on purpose.",
|
||||||
|
"cost": 450,
|
||||||
|
"payload": {
|
||||||
|
"frame_style": "box-shadow: 0 0 0 3px #ef4444, 0 0 14px rgba(239, 68, 68, 0.7)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
services:
|
||||||
|
web:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
ports:
|
||||||
|
- "8000:8000"
|
||||||
|
volumes:
|
||||||
|
# Song library folder on NAS
|
||||||
|
- /volume1/music/slopsmith:/dlc
|
||||||
|
# Persistent config, cache, favorites, loops, practice data
|
||||||
|
- slopsmith-config:/config
|
||||||
|
environment:
|
||||||
|
- DLC_DIR=/dlc
|
||||||
|
- CONFIG_DIR=/config
|
||||||
|
# Logging (optional)
|
||||||
|
# - LOG_LEVEL=INFO # DEBUG | INFO | WARNING | ERROR (default: INFO)
|
||||||
|
# - LOG_FORMAT=json # json | text (default: text)
|
||||||
|
# - LOG_FILE=/config/slopsmith.log # also write to a persistent file
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
slopsmith-config:
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
services:
|
||||||
|
web:
|
||||||
|
build: .
|
||||||
|
ports:
|
||||||
|
- "8000:8000"
|
||||||
|
volumes:
|
||||||
|
# Mount your song library folder (adjust path for your system)
|
||||||
|
- ${LIBRARY_PATH:-./library}:/dlc
|
||||||
|
# Persistent config and cache
|
||||||
|
- slopsmith-config:/config
|
||||||
|
# Mount source for live reload during development
|
||||||
|
- ./static:/app/static
|
||||||
|
- ./server.py:/app/server.py
|
||||||
|
- ./VERSION:/app/VERSION
|
||||||
|
- ./ug_browser.py:/app/ug_browser.py
|
||||||
|
- ./lib:/app/lib
|
||||||
|
- ./plugins:/app/plugins
|
||||||
|
- ./.git:/app/.git:ro
|
||||||
|
environment:
|
||||||
|
- PYTHONPATH=/app/lib:/app
|
||||||
|
- DLC_DIR=/dlc
|
||||||
|
- CONFIG_DIR=/config
|
||||||
|
# Write note_detect recordings to the host-visible bind mount (./static)
|
||||||
|
# rather than the /config Docker volume: recorded takes (and training
|
||||||
|
# bundles) then land in static/note_detect_recordings/ on the host, where
|
||||||
|
# the offline harness can read them directly.
|
||||||
|
- STATIC_DIR=/app/static
|
||||||
|
# Logging (optional)
|
||||||
|
# - LOG_LEVEL=DEBUG # DEBUG | INFO | WARNING | ERROR (default: INFO)
|
||||||
|
# - LOG_FORMAT=json # json | text (default: text — coloured console)
|
||||||
|
# - LOG_FILE=/config/slopsmith.log # also write to a persistent file
|
||||||
|
dns:
|
||||||
|
- 8.8.8.8
|
||||||
|
- 1.1.1.1
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
slopsmith-config:
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
@@ -0,0 +1,243 @@
|
|||||||
|
# Note Failure Feedback — Implementation Plan
|
||||||
|
|
||||||
|
Depends on: `docs/NOTE_FAILURE_SPEC.md` (read that first)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 0: Detection Plugin Foundation
|
||||||
|
|
||||||
|
**Goal:** Working note detection plugin streaming detected notes via WebSocket.
|
||||||
|
|
||||||
|
This phase was previously tracked in a separate NOTE_DETECTION_PLUGIN_PLAN
|
||||||
|
document (in the `slopsmith-plugin-notedetect` repository). The relevant scope
|
||||||
|
is summarized here to avoid relying on an internal git-only reference:
|
||||||
|
|
||||||
|
- [ ] Plugin skeleton: `slopsmith-plugin-notedetect/` with plugin.json, routes.py, screen.js
|
||||||
|
- [ ] Port TonalRecall YIN detection (aubio + sounddevice) to routes.py
|
||||||
|
- [ ] WebSocket at `/api/plugins/note_detect/stream` streaming `{ note, freq, confidence, time }`
|
||||||
|
- [ ] Device selection UI in screen.html
|
||||||
|
- [ ] requirements.txt: aubio, sounddevice, numpy
|
||||||
|
|
||||||
|
**Exit criterion:** With plugin active and guitar plugged in, playing a note
|
||||||
|
causes a JSON event to appear in the browser console.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Note Matching Core
|
||||||
|
|
||||||
|
**Goal:** Client-side matching of detected notes to chart notes. No rendering yet —
|
||||||
|
console logging only.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- `screen.js` in the notedetect plugin
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Implement `expectedFreq(string, fret, tuningOffsets, capo, stringCount, arrangementName)` using
|
||||||
|
base open-string frequencies, semitone offsets from `highway.getSongInfo().tuning`,
|
||||||
|
and semitone math (`2^(semitones/12)`) rather than assuming 6-string standard
|
||||||
|
tuning; use `highway.getStringCount()` as the authoritative string count because
|
||||||
|
tuning may be padded to length 6 for RS XML sources even for bass/extended-range
|
||||||
|
arrangements; include `highway.getSongInfo().capo` as additional semitones if
|
||||||
|
the intent is expected sounding pitch; pass `highway.getSongInfo().arrangement` as
|
||||||
|
`arrangementName` to disambiguate 5-string bass vs 5-string guitar (matching the
|
||||||
|
spec's `getBaseTuning` helper)
|
||||||
|
- [ ] Implement `NoteJudgmentTracker` class with:
|
||||||
|
- `addDetection(detected)` — correlate with nearest unmatched chart note
|
||||||
|
- `update(currentTime)` — expire pending notes whose match window has passed
|
||||||
|
- `getJudgmentsInRange(tStart, tEnd)` — return judgments in time range
|
||||||
|
- `reset()` — clear all state
|
||||||
|
- [ ] Connect to detection WebSocket, feed events into tracker
|
||||||
|
- [ ] Initialize tracker with `highway.getNotes()` and `highway.getChords()` on song ready
|
||||||
|
- [ ] Console.log each judgment as it resolves (HIT/MISSED/EARLY/LATE/SHARP/FLAT)
|
||||||
|
- [ ] Re-initialize tracker on `song:ready` (fires on every new song **and** on every
|
||||||
|
arrangement switch — no need to hook `highway.reconnect` or other internals);
|
||||||
|
do **not** use `song:loaded` — note/chord arrays are still empty at that point (data
|
||||||
|
arrives incrementally and only completes at `song:ready`)
|
||||||
|
|
||||||
|
**Exit criterion:** Playing along with a song, console shows correct HIT/MISSED
|
||||||
|
judgments with timing and pitch error values.
|
||||||
|
|
||||||
|
**Estimated scope:** ~200 lines JS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Hit/Miss Highway Overlay
|
||||||
|
|
||||||
|
**Goal:** Visual feedback on the highway — green glow for hits, red X for misses.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- `screen.js` in the notedetect plugin (draw hook)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Register `highway.addDrawHook()` that reads judgments from the tracker
|
||||||
|
- [ ] **Hit rendering:** Green glow ring behind notes at the now-line, fading over
|
||||||
|
`hitGlowDuration` seconds. Use `highway.project()` and `highway.fretX()` for
|
||||||
|
positioning. Additive blend via `ctx.globalCompositeOperation = 'lighter'`.
|
||||||
|
- [ ] **Miss rendering:** Red `✕` marker at the note's string/fret position, drawn
|
||||||
|
in the "past" region below the now-line. Do **not** rely on
|
||||||
|
`highway.project(negative_offset)` for long-lived placement — the current
|
||||||
|
renderer returns `null` for offsets more than ~50ms into the past. Instead,
|
||||||
|
anchor at the now-line (`highway.project(0)`) and map elapsed time since the
|
||||||
|
miss to a linear below-now-line Y position (configurable pixels/second), fading
|
||||||
|
the marker after `missMarkerDuration` seconds.
|
||||||
|
- [ ] **String pulse:** Brief red tint on the missed note's string (200ms fade on
|
||||||
|
the string line segment near the now-line).
|
||||||
|
- [ ] Handle lefty mode: use `highway.fillTextUnmirrored()` for text markers.
|
||||||
|
- [ ] Cleanup: `highway.removeDrawHook()` on plugin destroy.
|
||||||
|
|
||||||
|
**Exit criterion:** Playing a song, you see green flashes on hit notes and red X
|
||||||
|
markers scrolling past on missed notes.
|
||||||
|
|
||||||
|
**Estimated scope:** ~150 lines JS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: Diagnostic Labels (Timing + Pitch)
|
||||||
|
|
||||||
|
**Goal:** Show *why* a note was missed — too early, too late, sharp, flat.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- `screen.js` in the notedetect plugin
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Extend draw hook to render timing indicators:
|
||||||
|
- EARLY: orange `↑` + "-XXms" label above the miss marker
|
||||||
|
- LATE: orange `↓` + "+XXms" label below the miss marker
|
||||||
|
- Only shown when timing error exceeds `timingThresholdMs`
|
||||||
|
- [ ] Extend draw hook to render pitch indicators:
|
||||||
|
- SHARP: blue `♯` + "+XX¢" label
|
||||||
|
- FLAT: blue `♭` + "-XX¢" label
|
||||||
|
- Only shown when pitch error exceeds `pitchThresholdCents`
|
||||||
|
- [ ] Compound states: stack timing label on top, pitch label below
|
||||||
|
- [ ] Add settings UI in plugin settings panel for threshold configuration
|
||||||
|
- [ ] Ensure labels don't overlap — offset vertically when multiple notes
|
||||||
|
miss at close timestamps
|
||||||
|
|
||||||
|
**Exit criterion:** Playing intentionally early/late or bending sharp/flat
|
||||||
|
shows the correct diagnostic labels.
|
||||||
|
|
||||||
|
**Estimated scope:** ~100 lines JS, ~30 lines settings HTML
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: Loop Iteration Tracking
|
||||||
|
|
||||||
|
**Goal:** Track performance across loop iterations, show summary on each wrap.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- `screen.js` in the notedetect plugin
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Detect loop wrap: `currentTime < previousTime - 0.5` in the frame update
|
||||||
|
- [ ] On wrap: snapshot `{ hits, misses, total, percentage }` to `loopHistory[]`
|
||||||
|
- [ ] Reset judgments for notes in `[loopA, loopB]` range (keep tracker alive
|
||||||
|
for notes outside the loop)
|
||||||
|
- [ ] Render loop summary overlay (top-center, semi-transparent background):
|
||||||
|
```
|
||||||
|
Loop N | X/Y notes (Z%) | Best: W%
|
||||||
|
```
|
||||||
|
Displayed for 1.5s, then fades.
|
||||||
|
- [ ] Track `bestIteration` across all iterations for "Best" display
|
||||||
|
- [ ] Emit `loop:complete` event via `window.slopsmith.emit()` so other plugins
|
||||||
|
(practice journal) can record the data
|
||||||
|
- [ ] Reset loop history when loop boundaries change or loop is cleared
|
||||||
|
|
||||||
|
**Exit criterion:** Looping a 4-bar phrase, you see iteration count and accuracy
|
||||||
|
flash briefly at each loop wrap. Best score persists across iterations.
|
||||||
|
|
||||||
|
**Estimated scope:** ~120 lines JS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: Section Grading
|
||||||
|
|
||||||
|
**Goal:** Grade each song section (intro, verse, chorus, solo) and surface weak spots.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- `screen.js` in the notedetect plugin
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Use `highway.getSections()` to identify section boundaries
|
||||||
|
- [ ] Track hits/misses per section as notes are judged
|
||||||
|
- [ ] At section boundaries (when `currentTime` crosses a section end),
|
||||||
|
briefly flash the section grade:
|
||||||
|
- A: 90%+, B: 75%+, C: 60%+, D: 40%+, F: below 40%
|
||||||
|
- Color: green (A/B), yellow (C), red (D/F)
|
||||||
|
- [ ] After song completes (or at any point via a hotkey), show a section
|
||||||
|
summary panel listing all sections with grades
|
||||||
|
- [ ] Highlight lowest-scoring section with a "Loop this section" button
|
||||||
|
that sets A-B points to that section's boundaries
|
||||||
|
- [ ] Emit `note:sectionGrade` event for other plugins
|
||||||
|
|
||||||
|
**Exit criterion:** Playing through a song, section grades flash at each
|
||||||
|
transition. Lowest section is highlighted for targeted practice.
|
||||||
|
|
||||||
|
**Estimated scope:** ~150 lines JS, ~40 lines HTML
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 6: Polish + Settings
|
||||||
|
|
||||||
|
**Goal:** Configurable thresholds, visual polish, performance.
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Full settings panel in plugin settings HTML:
|
||||||
|
- Match window slider (100-500ms)
|
||||||
|
- Pitch tolerance slider (20-100 cents)
|
||||||
|
- Toggle timing/pitch labels
|
||||||
|
- Toggle loop summary
|
||||||
|
- Miss marker duration slider
|
||||||
|
- [ ] Performance: ensure draw hook stays under 1ms per frame
|
||||||
|
- Pre-compute judgment positions, don't recalculate in draw loop
|
||||||
|
- Binary search over judgments by time (same pattern as `drawNotes`)
|
||||||
|
- [ ] Smooth animations: glow/fade using eased alpha, not linear
|
||||||
|
- [ ] Color-blind accessible palette option (use shapes not just colors)
|
||||||
|
- [ ] Persist settings in plugin-local storage (e.g. `localStorage` prefixed with
|
||||||
|
plugin id) — do **not** use `/api/settings` for this; the current server only
|
||||||
|
persists a fixed set of known keys and will silently discard `notedetect_feedback`
|
||||||
|
|
||||||
|
**Estimated scope:** ~100 lines JS, ~60 lines HTML
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependency Graph
|
||||||
|
|
||||||
|
```
|
||||||
|
Phase 0 (detection plugin)
|
||||||
|
↓
|
||||||
|
Phase 1 (matching core)
|
||||||
|
↓
|
||||||
|
Phase 2 (hit/miss overlay) ← Minimum viable feature
|
||||||
|
↓
|
||||||
|
Phase 3 (diagnostic labels)
|
||||||
|
↓
|
||||||
|
Phase 4 (loop tracking) ← Core practice value
|
||||||
|
↓
|
||||||
|
Phase 5 (section grading)
|
||||||
|
↓
|
||||||
|
Phase 6 (polish)
|
||||||
|
```
|
||||||
|
|
||||||
|
Phases 3-5 are independent of each other and can be done in any order after Phase 2.
|
||||||
|
Phase 6 should be last.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risk / Open Questions
|
||||||
|
|
||||||
|
1. **Latency budget:** Detection → WebSocket → matching → render adds latency.
|
||||||
|
If total pipeline > 100ms, the match window needs to compensate with asymmetric
|
||||||
|
tolerance (more lenient for "late" detections). Measure in Phase 1.
|
||||||
|
|
||||||
|
2. **Chord matching granularity:** Current plan matches chord notes individually.
|
||||||
|
Should a chord be "missed" if 4/6 notes hit? Propose: grade chords as
|
||||||
|
percentage, treat as HIT if ≥50% of notes matched. Revisit after Phase 2 testing.
|
||||||
|
|
||||||
|
3. **Tempo-scaled thresholds:** At 200 BPM, a 200ms match window covers almost
|
||||||
|
an entire beat. Should thresholds scale with tempo? Propose: don't over-engineer
|
||||||
|
this initially. Fixed thresholds work for most tempos. Revisit if users report
|
||||||
|
issues at extreme tempos.
|
||||||
|
|
||||||
|
4. **Detection plugin availability:** Everything in Phases 1-6 degrades gracefully
|
||||||
|
if the detection WebSocket isn't connected — the draw hook simply has no
|
||||||
|
judgments to render, and the highway looks exactly as it does today.
|
||||||
@@ -0,0 +1,390 @@
|
|||||||
|
# Note Failure Feedback — Technical Spec
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
When a user loops over a lick, **show note misses on the highway** with diagnostic
|
||||||
|
detail: which note was missed, and *how* it was missed (timing vs pitch).
|
||||||
|
|
||||||
|
Many rhythm-practice tools show a `!` marker at the missed note position after it
|
||||||
|
passes. We improve on this by showing *why* the note was missed — too early, too
|
||||||
|
late, wrong pitch, or not played at all.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
This feature depends on the **note detection plugin** (`slopsmith-plugin-notedetect`),
|
||||||
|
which provides real-time pitch detection via server-side aubio/YIN over WebSocket.
|
||||||
|
The detection plugin streams `DetectedNote` events; this spec describes the
|
||||||
|
**matching, judgment, and rendering** layer that consumes those events.
|
||||||
|
|
||||||
|
Without the detection plugin active, no miss/hit feedback is shown — the highway
|
||||||
|
renders exactly as it does today.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Guitar → USB Adapter → sounddevice (server)
|
||||||
|
↓
|
||||||
|
aubio YIN detection
|
||||||
|
↓
|
||||||
|
WebSocket: detected notes
|
||||||
|
↓
|
||||||
|
┌───────────────────────┐
|
||||||
|
│ Note Matcher │ ← THIS SPEC
|
||||||
|
│ (client-side JS) │
|
||||||
|
│ │
|
||||||
|
│ Chart notes (highway) │
|
||||||
|
│ × Detected notes (WS) │
|
||||||
|
│ = Match/Miss/Extra │
|
||||||
|
└───────────────────────┘
|
||||||
|
↓
|
||||||
|
Highway draw hook overlay
|
||||||
|
(hit glow, miss markers, diagnostics)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
|
||||||
|
1. **Chart notes** arrive via existing highway WebSocket (`/ws/highway/{filename}`).
|
||||||
|
Wire format: `{ t, s, f, sus, bn, ho, po, ... }` (see `lib/song.py:note_to_wire`)
|
||||||
|
|
||||||
|
2. **Detected notes** arrive via detection plugin WebSocket
|
||||||
|
(`/api/plugins/note_detect/stream`).
|
||||||
|
Wire format: `{ note: "A2", freq: 110.0, confidence: 0.92, time: 1.234 }`
|
||||||
|
|
||||||
|
> **Plugin naming note:** The detection plugin's repository is named
|
||||||
|
> `slopsmith-plugin-notedetect`, but the plugin registers with the id
|
||||||
|
> `note_detect` (snake_case). Its HTTP/WebSocket routes therefore appear
|
||||||
|
> under `/api/plugins/note_detect/…`. There is no `window.slopsmithPlugin_*`
|
||||||
|
> global pattern in Slopsmith — to check whether the detection plugin is
|
||||||
|
> available at runtime, attempt a fetch to `/api/plugins/note_detect/status`
|
||||||
|
> (or similar) or consult the `/api/plugins` list. Use the repo name only
|
||||||
|
> in documentation links.
|
||||||
|
|
||||||
|
3. **Note Matcher** (new, client-side) correlates these two streams in real-time.
|
||||||
|
|
||||||
|
4. **Draw hook** renders results on the highway via `highway.addDrawHook()`.
|
||||||
|
|
||||||
|
> **⚠ Limitation:** `addDrawHook()` is only invoked by the **default 2D renderer**.
|
||||||
|
> If the user has switched to a custom renderer (e.g., a WebGL 3D highway plugin),
|
||||||
|
> draw hooks are not called and this overlay will be invisible. Implementers should
|
||||||
|
> note this in the plugin's UI (e.g., a warning banner when a non-default renderer
|
||||||
|
> is detected) and may want to explore a renderer-agnostic overlay approach
|
||||||
|
> (own canvas + own rAF loop, reading public highway state via getters) as a
|
||||||
|
> future improvement.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Note Matching Algorithm
|
||||||
|
|
||||||
|
### Match Window
|
||||||
|
|
||||||
|
A detected note matches a chart note when:
|
||||||
|
|
||||||
|
| Criterion | Threshold | Notes |
|
||||||
|
|----------------|------------------------|---------------------------------------------|
|
||||||
|
| **Time** | ±200ms (configurable) | Centered on chart note time |
|
||||||
|
| **Pitch** | ±50 cents | Accounts for imperfect intonation |
|
||||||
|
| **String** | Pitch-only (for now) | `DetectedNote` carries no string field; exact-string matching requires the detection plugin to be extended to emit a string estimate. Treat string as always-unknown until that extension lands. |
|
||||||
|
|
||||||
|
### Expected Frequency Calculation
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// Open-string base frequencies (Hz), string index 0 = lowest string.
|
||||||
|
// Select by highway.getStringCount() + arrangement name from highway.getSongInfo():
|
||||||
|
// 4-string → BASS_TUNING (E1 A1 D2 G2)
|
||||||
|
// 5-string bass → BASS5_TUNING (B0 E1 A1 D2 G2)
|
||||||
|
// 6-string (default)→ GUITAR_TUNING (E2 A2 D3 G3 B3 E4)
|
||||||
|
// 7-string → GUITAR7_TUNING (B1 E2 A2 D3 G3 B3 E4)
|
||||||
|
const GUITAR_TUNING = [82.41, 110.00, 146.83, 196.00, 246.94, 329.63];
|
||||||
|
const BASS_TUNING = [41.20, 55.00, 73.42, 98.00];
|
||||||
|
const BASS5_TUNING = [30.87, 41.20, 55.00, 73.42, 98.00];
|
||||||
|
const GUITAR7_TUNING = [61.74, 82.41, 110.00, 146.83, 196.00, 246.94, 329.63];
|
||||||
|
|
||||||
|
function getBaseTuning(stringCount, arrangementName) {
|
||||||
|
const isBass = /bass/i.test(arrangementName || '');
|
||||||
|
if (stringCount === 4) return BASS_TUNING;
|
||||||
|
if (stringCount === 5 && isBass) return BASS5_TUNING;
|
||||||
|
if (stringCount === 7) return GUITAR7_TUNING;
|
||||||
|
return GUITAR_TUNING; // 6-string or unknown
|
||||||
|
}
|
||||||
|
|
||||||
|
function expectedFreq(string, fret, tuningOffsets, capo = 0, stringCount, arrangementName = '') {
|
||||||
|
// tuningOffsets: per-string semitone offsets from standard (from song metadata).
|
||||||
|
// stringCount: REQUIRED — pass highway.getStringCount(). Do NOT default to
|
||||||
|
// tuningOffsets.length: RS XML sources pad the tuning array to length 6 even for
|
||||||
|
// 4-string bass, which would cause incorrect base-tuning selection.
|
||||||
|
// arrangementName: pass highway.getSongInfo().arrangement to resolve ambiguous
|
||||||
|
// 5-string cases (5-string bass vs 5-string extended guitar).
|
||||||
|
const BASE = getBaseTuning(stringCount, arrangementName);
|
||||||
|
if (string < 0 || string >= stringCount || string >= BASE.length) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const semitones = tuningOffsets[string] + capo;
|
||||||
|
const base = BASE[string] * Math.pow(2, semitones / 12);
|
||||||
|
return base * Math.pow(2, fret / 12);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Match States
|
||||||
|
|
||||||
|
Each chart note resolves to a **judgment** with two independent axes:
|
||||||
|
|
||||||
|
- **Timing axis** (`timingState`): `'OK'` if within `timingThresholdMs`, `'EARLY'` if matched
|
||||||
|
more than `timingThresholdMs` before chart time, `'LATE'` if more than `timingThresholdMs`
|
||||||
|
after. `null` if the note was never matched (MISSED).
|
||||||
|
- **Pitch axis** (`pitchState`): `'OK'` if detected pitch is within `pitchThresholdCents` of
|
||||||
|
expected, `'SHARP'` if above by more than `pitchThresholdCents`, `'FLAT'` if below. `null`
|
||||||
|
if unmatched.
|
||||||
|
- **`hit`**: `true` when both axes are `'OK'`; `false` for MISSED or any off-axis result.
|
||||||
|
|
||||||
|
The axes combine independently (e.g., `LATE + FLAT`, `EARLY + SHARP`). The state diagram
|
||||||
|
below shows possible terminal values per axis:
|
||||||
|
|
||||||
|
```
|
||||||
|
PENDING → hit=true, timingState='OK', pitchState='OK'
|
||||||
|
→ hit=false, timingState=null, pitchState=null (MISSED — window expired)
|
||||||
|
→ hit=false, timingState='EARLY', pitchState=... (too early)
|
||||||
|
→ hit=false, timingState='LATE', pitchState=... (too late)
|
||||||
|
→ hit=false, timingState='OK', pitchState='SHARP'
|
||||||
|
→ hit=false, timingState='OK', pitchState='FLAT'
|
||||||
|
```
|
||||||
|
|
||||||
|
Timing and pitch thresholds are read from configuration (`timingThresholdMs`,
|
||||||
|
`pitchThresholdCents`) — never hard-coded.
|
||||||
|
|
||||||
|
### Judgment Data Structure
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// Per-note judgment, attached after the note passes the now-line.
|
||||||
|
// Compound judgments (e.g. LATE + FLAT) are expressed as separate
|
||||||
|
// timingState / pitchState fields; never concatenate them into `state`.
|
||||||
|
{
|
||||||
|
chartNote: { t, s, f, ... }, // Original chart note
|
||||||
|
|
||||||
|
// Overall outcome — top-level quick check
|
||||||
|
hit: false, // true iff timing AND pitch are both clean
|
||||||
|
|
||||||
|
// Timing axis: null if no detection arrived (pure MISSED)
|
||||||
|
timingState: 'EARLY' | 'LATE' | 'OK' | null,
|
||||||
|
timingError: -120, // Milliseconds (negative = early); null if no detection
|
||||||
|
|
||||||
|
// Pitch axis: null if no detection arrived (same condition as timingState).
|
||||||
|
// Pitch is evaluated independently for any matched detection — a LATE note
|
||||||
|
// can also be FLAT (both axes are set even when timingState ≠ 'OK').
|
||||||
|
pitchState: 'SHARP' | 'FLAT' | 'OK' | null,
|
||||||
|
pitchError: +15, // Cents (positive = sharp); null if no detection
|
||||||
|
|
||||||
|
// Raw detection data (null if no detection arrived)
|
||||||
|
detectedFreq: 112.3, // What was actually played
|
||||||
|
expectedFreq: 110.0, // What should have been played
|
||||||
|
detectedAt: 1.354, // When the detection arrived
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Precedence / rendering rules:**
|
||||||
|
- `hit: true` → green glow; both timing/pitch states will be `'OK'`. A judgment is a hit when `|timingError| ≤ timingThresholdMs` **and** `|pitchError| ≤ pitchThresholdCents` (see §Configuration).
|
||||||
|
- `timingState: null` (no detection) → pure miss (`✕`); skip pitch display.
|
||||||
|
- Non-null `timingState` + non-null `pitchState` → compound: render timing
|
||||||
|
indicator on top, pitch indicator below.
|
||||||
|
- Emitted `note:hit` / `note:miss` events carry the full judgment object so
|
||||||
|
subscribers can inspect either axis independently.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Highway Rendering
|
||||||
|
|
||||||
|
### Hit Feedback
|
||||||
|
|
||||||
|
Notes matched within `timingThresholdMs` (default 100 ms) **and**
|
||||||
|
`pitchThresholdCents` (default 20 ¢) get a **green glow ring** that fades over
|
||||||
|
`hitGlowDuration` (default 0.5 s). This is the combination that sets `hit: true`
|
||||||
|
on the judgment object. The existing note rendering is unchanged — the glow is
|
||||||
|
drawn *behind* the note at the now-line position as it passes.
|
||||||
|
|
||||||
|
```
|
||||||
|
[existing note bubble]
|
||||||
|
└── green glow ring (additive blend, fades)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Miss Markers
|
||||||
|
|
||||||
|
Missed notes get a persistent marker that continues downward past the now-line
|
||||||
|
and remains visible for 2 seconds (configurable). The marker stays at the
|
||||||
|
note's string/fret position on the "past" portion of the highway (below
|
||||||
|
now-line).
|
||||||
|
|
||||||
|
**Positioning rule:** do **not** rely on `highway.project(tOffset)` for the full
|
||||||
|
miss-marker lifetime below the now-line. That helper returns `null` for offsets
|
||||||
|
more than ~50ms into the past, so it cannot place markers that persist for
|
||||||
|
seconds after the note passes. Instead, define a dedicated mapping anchored at
|
||||||
|
the now-line:
|
||||||
|
|
||||||
|
- `tOffset = 0` starts at the now-line (use `highway.project(0)` to get this Y).
|
||||||
|
- For the past region (`tOffset < 0` up to `-missMarkerDuration`), place the
|
||||||
|
marker below the now-line using a configurable linear pixels-per-second mapping,
|
||||||
|
clamped to the visible past area.
|
||||||
|
- The existing `highway.project()` may still be used for positions at/above the
|
||||||
|
now-line (approaching notes in the last ~50ms), but once a marker has crossed
|
||||||
|
into the past region its Y is governed by this below-now-line mapping.
|
||||||
|
|
||||||
|
| State | Visual |
|
||||||
|
|--------|-------------------------------------------------------------|
|
||||||
|
| MISSED | Red `✕` at note position + red tint on string segment |
|
||||||
|
| EARLY | Orange `↑` (up arrow) + timing offset label (e.g., "-120ms")|
|
||||||
|
| LATE | Orange `↓` (down arrow) + timing offset label ("+85ms") |
|
||||||
|
| SHARP | Blue `♯` + cents label ("+35¢") |
|
||||||
|
| FLAT | Blue `♭` + cents label ("-42¢") |
|
||||||
|
|
||||||
|
Compound states stack vertically: timing indicator on top, pitch indicator below.
|
||||||
|
|
||||||
|
### Miss markers on the string area
|
||||||
|
|
||||||
|
Below the now-line, all strings for the active arrangement are always visible
|
||||||
|
(use `highway.getStringCount()` — 4 for bass, 6 for guitar, 7+ for extended-range).
|
||||||
|
For a missed note,
|
||||||
|
the relevant string segment between the now-line and ~20px below it gets a brief
|
||||||
|
red pulse (200ms fade).
|
||||||
|
|
||||||
|
### Loop Iteration Summary
|
||||||
|
|
||||||
|
When A-B looping is active, at the end of each loop iteration (when playback
|
||||||
|
wraps from B back to A), show a brief overlay:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────┐
|
||||||
|
│ Loop 3/∞ │
|
||||||
|
│ 5/7 notes hit (71%)│
|
||||||
|
│ Best: 6/7 (86%) │
|
||||||
|
└─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Displayed for 1.5s, then fades. Does not block the highway.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## State Management
|
||||||
|
|
||||||
|
### NoteJudgmentTracker
|
||||||
|
|
||||||
|
Client-side class that manages the correlation between chart notes and detections.
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
class NoteJudgmentTracker {
|
||||||
|
constructor(chartNotes, chartChords, tuning) { ... }
|
||||||
|
|
||||||
|
// Called when a detected note arrives from the detection WebSocket
|
||||||
|
addDetection(detected) { ... }
|
||||||
|
|
||||||
|
// Called each frame; checks for expired match windows
|
||||||
|
update(currentTime) { ... }
|
||||||
|
|
||||||
|
// Returns judgments for notes in the visible time range
|
||||||
|
getJudgmentsInRange(tStart, tEnd) { ... }
|
||||||
|
|
||||||
|
// Reset (on song change, loop restart, arrangement switch)
|
||||||
|
reset() { ... }
|
||||||
|
|
||||||
|
// Stats for the current loop iteration
|
||||||
|
getLoopStats() { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Memory Management
|
||||||
|
|
||||||
|
- Judgments older than 10 seconds behind current time are pruned each frame.
|
||||||
|
- Detection buffer holds last 5 seconds of raw detections.
|
||||||
|
- On loop wrap (B→A), archive current iteration stats, reset judgments for
|
||||||
|
the loop range, keep detections flowing.
|
||||||
|
|
||||||
|
### Loop-Aware Behavior
|
||||||
|
|
||||||
|
The tracker must handle A-B looping:
|
||||||
|
|
||||||
|
1. Detect loop wrap: `currentTime < previousTime - 0.5` (jumped backward).
|
||||||
|
2. On wrap: snapshot current stats to `loopHistory[]`, reset judgments
|
||||||
|
for notes in `[loopA, loopB]` range.
|
||||||
|
3. `getLoopStats()` returns current iteration + best historical iteration.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
### Existing Highway API Used
|
||||||
|
|
||||||
|
| API | Purpose |
|
||||||
|
|------------------------------|------------------------------------------|
|
||||||
|
| `highway.addDrawHook(fn)` | Register the overlay renderer |
|
||||||
|
| `highway.removeDrawHook(fn)` | Cleanup on plugin unload |
|
||||||
|
| `highway.getTime()` | Current chart time (audio-aligned) |
|
||||||
|
| `highway.getAvOffset()` | A/V offset in ms; visual render clock = `getTime() + getAvOffset()/1000` — use this when computing `tOffset` for `project()` calls inside draw hooks, otherwise markers appear shifted when the user has calibrated A/V latency |
|
||||||
|
| `highway.getNotes()` | All chart notes (for matching) |
|
||||||
|
| `highway.getChords()` | All chart chords (match individual notes)|
|
||||||
|
| `highway.getSections()` | Section boundaries (for section grading) |
|
||||||
|
| `highway.getSongInfo()` | Tuning offsets for frequency calculation |
|
||||||
|
| `highway.project(tOffset)` | Convert time offset to Y position |
|
||||||
|
| `highway.fretX(fret, scale, w)` | Convert fret to X position using `scale` from `highway.project(tOffset)` |
|
||||||
|
| `highway.fillTextUnmirrored` | Text that stays readable in lefty mode |
|
||||||
|
|
||||||
|
### Existing App.js Used
|
||||||
|
|
||||||
|
| Global | Purpose |
|
||||||
|
|------------------------------|------------------------------------------|
|
||||||
|
| `loopA`, `loopB` | Current A-B loop boundaries |
|
||||||
|
| `audio.currentTime` | Actual audio playback position |
|
||||||
|
|
||||||
|
### New Events Emitted (via `window.slopsmith.emit`)
|
||||||
|
|
||||||
|
| Event | Payload |
|
||||||
|
|------------------------------|------------------------------------------|
|
||||||
|
| `note:hit` | full `Judgment` object (see §Judgment Data Structure) |
|
||||||
|
| `note:miss` | full `Judgment` object |
|
||||||
|
| `loop:complete` | `{ iteration, stats }` |
|
||||||
|
| `note:sectionGrade` | `{ section, grade, hits, total }` |
|
||||||
|
|
||||||
|
`note:hit` and `note:miss` always carry the complete `Judgment` object so
|
||||||
|
subscribers can inspect `timingState`, `pitchState`, timing/pitch errors, and
|
||||||
|
raw detection data independently, without the emitter having to pre-select fields.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Configuration (plugin settings)
|
||||||
|
|
||||||
|
There are three distinct threshold tiers — keep them conceptually separate:
|
||||||
|
|
||||||
|
| Tier | Setting(s) | Role |
|
||||||
|
|------|-----------|------|
|
||||||
|
| **Match window** | `matchWindowMs`, `pitchToleranceCents` | Outer gate: a detection is only correlated to a chart note if it falls within these limits. Outside → ignored (extra note, not an attempt). |
|
||||||
|
| **Hit threshold** | `timingThresholdMs`, `pitchThresholdCents` | Sets `hit: true`. A matched note is a clean hit when `|timingError| ≤ timingThresholdMs` **and** `|pitchError| ≤ pitchThresholdCents`. Also triggers the green glow. Must be ≤ match window values. |
|
||||||
|
| **Label threshold** | (same keys) | Same values double as the boundary at which EARLY/LATE/SHARP/FLAT labels appear. Within the hit threshold = `'OK'` state; outside = labeled state. |
|
||||||
|
|
||||||
|
| Setting | Default | Description |
|
||||||
|
|------------------------|---------|----------------------------------------------------------------|
|
||||||
|
| `matchWindowMs` | 200 | Outer time tolerance for correlating a detection to a chart note (ms) |
|
||||||
|
| `pitchToleranceCents` | 50 | Outer pitch tolerance for correlation (cents) |
|
||||||
|
| `timingThresholdMs` | 100 | `|timingError| ≤ this` → `timingState: 'OK'` and `hit` eligible; also defines the EARLY/LATE label boundary |
|
||||||
|
| `pitchThresholdCents` | 20 | `|pitchError| ≤ this` → `pitchState: 'OK'` and `hit` eligible; also defines the SHARP/FLAT label boundary |
|
||||||
|
| `showTimingErrors` | true | Show EARLY/LATE labels when `timingState` is non-OK |
|
||||||
|
| `showPitchErrors` | true | Show SHARP/FLAT labels when `pitchState` is non-OK |
|
||||||
|
| `missMarkerDuration` | 2.0 | How long miss markers stay visible (sec) |
|
||||||
|
| `showLoopSummary` | true | Show stats on loop wrap |
|
||||||
|
| `hitGlowDuration` | 0.5 | Green glow fade time (sec) |
|
||||||
|
|
||||||
|
Persist these settings in plugin-local storage (e.g. `localStorage` prefixed
|
||||||
|
with the plugin id). Do **not** assume they can be saved through Slopsmith's
|
||||||
|
`/api/settings` endpoint under a `notedetect_feedback` key — the current server
|
||||||
|
only persists a fixed set of known settings keys. If backend support for a
|
||||||
|
dedicated persisted key is added later, this plugin may migrate to `/api/settings`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What This Does NOT Cover
|
||||||
|
|
||||||
|
- **Audio input / pitch detection** — handled by the detection plugin
|
||||||
|
- **Device selection UI** — handled by the detection plugin
|
||||||
|
- **Score persistence / history** — future work (practice journal plugin)
|
||||||
|
- **Difficulty scaling** — automatic dynamic-difficulty scaling is not implemented
|
||||||
|
- **Chord grading** — chords are graded per-note (each note in the chord
|
||||||
|
is independently matched), not as a single unit
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# Slopsmith Note Detect Bass Benchmark — v1
|
||||||
|
|
||||||
|
A bass-focused companion to the guitar benchmarks
|
||||||
|
(note_detect_v1 + note_detect_v2). Tests `note_detect` against bass-
|
||||||
|
specific idioms: walking lines, octave jumps, root+fifth patterns,
|
||||||
|
double-stops, and long low-E holds that stress YIN's accumulator at
|
||||||
|
~41 Hz.
|
||||||
|
|
||||||
|
- **Tempo**: 90 BPM
|
||||||
|
- **Tuning**: E standard 4-string (E1 A1 D2 G2, no capo)
|
||||||
|
- **Audio**: metronome click track only — play *over* the click.
|
||||||
|
- **Duration**: 181 s
|
||||||
|
|
||||||
|
## Sections
|
||||||
|
|
||||||
|
| Section | Tests |
|
||||||
|
|---|---|
|
||||||
|
| A. Open strings (slow walk) | Mono detection on each open string, low → high → low |
|
||||||
|
| B. 5th-fret (slow walk) | Fretted-note detection across the 4 strings |
|
||||||
|
| C. Sustained notes | 3 × 4-second held roots |
|
||||||
|
| D. Octave walks | Root ↔ octave alternation, 2 strings + 2 frets up |
|
||||||
|
| E. Walking bassline | A minor pentatonic ascending + descending |
|
||||||
|
| F. Root + fifth pattern | Classic rock bass pattern (8 events) |
|
||||||
|
| G. Double-stops | 2-string voicings — the chord-scorer test for bass |
|
||||||
|
| H. Long low-E holds | 3 × 5-second E1 holds, stresses YIN under-buffer regime |
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Diagnostic JSON schema is `note_detect.diagnostic.v1`. Filter
|
||||||
|
`benchmark_hint` to bucket bass vs guitar runs.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Built by `docs/benchmarks/note_detect_bass_v1/build_benchmark.py`.
|
||||||
@@ -0,0 +1,503 @@
|
|||||||
|
"""Builds the Note Detect Bass Benchmark sloppak (v1).
|
||||||
|
|
||||||
|
A bass-focused companion to the guitar benchmarks (note_detect_v1 +
|
||||||
|
note_detect_v2). Same 90 BPM click, similar half-note pacing as v2,
|
||||||
|
but the sections are built around what bass actually plays: single-
|
||||||
|
note lines, octave jumps, walking patterns, sustained roots, and
|
||||||
|
two-string double-stops (the closest bass gets to "chords").
|
||||||
|
|
||||||
|
Why a separate bass benchmark instead of toggling string count on
|
||||||
|
the guitar one:
|
||||||
|
|
||||||
|
- Tuning is different — 4-string bass open MIDI is [28, 33, 38, 43]
|
||||||
|
(E1, A1, D2, G2) vs the guitar's [40, 45, 50, 55, 59, 64]. The
|
||||||
|
benchmark needs to produce notes the player can actually play on
|
||||||
|
the instrument they have plugged in.
|
||||||
|
- Bass idioms are different from guitar idioms. Strumming sections
|
||||||
|
don't apply; walking bass + octave patterns do.
|
||||||
|
- Low-frequency detection is materially harder for YIN — E1 at
|
||||||
|
~41 Hz needs more accumulated samples for confident detection
|
||||||
|
than guitar E2 at ~82 Hz. The benchmark should exercise that
|
||||||
|
regime explicitly so we can spot regressions there.
|
||||||
|
|
||||||
|
How to run inside the slopsmith container:
|
||||||
|
|
||||||
|
docker cp docs/benchmarks/note_detect_bass_v1/build_benchmark.py \\
|
||||||
|
slopsmith-web-1:/tmp/build_benchmark_bass.py
|
||||||
|
docker exec slopsmith-web-1 python /tmp/build_benchmark_bass.py \\
|
||||||
|
/app/static/sloppak_cache/note_detect_benchmark_bass_v1.sloppak
|
||||||
|
|
||||||
|
After regenerating, copy the zip output to the tracked path with the
|
||||||
|
`.sloppak` (not `.sloppak.zip`) suffix.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import shutil
|
||||||
|
import struct
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import wave
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
# ── Benchmark parameters ────────────────────────────────────────────────
|
||||||
|
BPM = 90.0
|
||||||
|
SECONDS_PER_BEAT = 60.0 / BPM
|
||||||
|
BEATS_PER_BAR = 4
|
||||||
|
BAR_S = BEATS_PER_BAR * SECONDS_PER_BEAT
|
||||||
|
INTRO_BARS = 2
|
||||||
|
OUTRO_BARS = 2
|
||||||
|
EXERCISE_BARS = 8
|
||||||
|
|
||||||
|
# 4-string bass open MIDI per string, low → high.
|
||||||
|
# Matches lib/tunings convention used by note_detect when the
|
||||||
|
# arrangement is 'bass' and stringCount is 4.
|
||||||
|
OPEN_MIDI = [28, 33, 38, 43] # E1 A1 D2 G2
|
||||||
|
N_STRINGS = 4
|
||||||
|
|
||||||
|
SR = 44100
|
||||||
|
|
||||||
|
|
||||||
|
# ── Click-track audio generator ────────────────────────────────────────
|
||||||
|
def _sine_burst(freq_hz, duration_s, amplitude):
|
||||||
|
n = int(SR * duration_s)
|
||||||
|
out = []
|
||||||
|
fade = max(1, int(0.004 * SR))
|
||||||
|
for i in range(n):
|
||||||
|
env = 1.0
|
||||||
|
if i < fade:
|
||||||
|
env = i / fade
|
||||||
|
elif i >= n - fade:
|
||||||
|
env = (n - 1 - i) / fade
|
||||||
|
s = math.sin(2 * math.pi * freq_hz * (i / SR)) * amplitude * env
|
||||||
|
out.append(max(-1.0, min(1.0, s)))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def write_click_wav(path: Path, duration_s: float):
|
||||||
|
total_samples = int(SR * duration_s)
|
||||||
|
pcm = [0] * total_samples
|
||||||
|
beat = 0
|
||||||
|
t = 0.0
|
||||||
|
while t < duration_s:
|
||||||
|
is_downbeat = (beat % BEATS_PER_BAR == 0)
|
||||||
|
freq = 1200 if is_downbeat else 800
|
||||||
|
amp = 0.6 if is_downbeat else 0.35
|
||||||
|
burst = _sine_burst(freq, 0.040, amp)
|
||||||
|
start = int(t * SR)
|
||||||
|
for i, s in enumerate(burst):
|
||||||
|
j = start + i
|
||||||
|
if 0 <= j < total_samples:
|
||||||
|
pcm[j] = int(max(-1.0, min(1.0, pcm[j] / 32767 + s)) * 32767)
|
||||||
|
t += SECONDS_PER_BEAT
|
||||||
|
beat += 1
|
||||||
|
|
||||||
|
pcm = [struct.pack('<h', v) for v in pcm]
|
||||||
|
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with wave.open(str(path), 'wb') as w:
|
||||||
|
w.setnchannels(1)
|
||||||
|
w.setsampwidth(2)
|
||||||
|
w.setframerate(SR)
|
||||||
|
w.writeframes(bytes(b''.join(pcm)))
|
||||||
|
|
||||||
|
|
||||||
|
# ── Chart helpers ─────────────────────────────────────────────────────
|
||||||
|
def note(t, s, f, sus=0.0, **flags):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
's': s,
|
||||||
|
'f': f,
|
||||||
|
'sus': round(sus, 3),
|
||||||
|
'sl': flags.get('sl', -1),
|
||||||
|
'slu': flags.get('slu', -1),
|
||||||
|
'bn': flags.get('bn', 0.0),
|
||||||
|
'ho': flags.get('ho', False),
|
||||||
|
'po': flags.get('po', False),
|
||||||
|
'hm': flags.get('hm', False),
|
||||||
|
'hp': flags.get('hp', False),
|
||||||
|
'pm': flags.get('pm', False),
|
||||||
|
'mt': flags.get('mt', False),
|
||||||
|
'vb': flags.get('vb', False),
|
||||||
|
'tr': flags.get('tr', False),
|
||||||
|
'ac': flags.get('ac', False),
|
||||||
|
'tp': flags.get('tp', False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord(t, id_, notes):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
'id': id_,
|
||||||
|
'hd': False,
|
||||||
|
'notes': notes,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord_note(s, f, sus=0.0, **flags):
|
||||||
|
n = note(0.0, s, f, sus, **flags)
|
||||||
|
n.pop('t')
|
||||||
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
# ── Exercises ─────────────────────────────────────────────────────────
|
||||||
|
# Bass idioms: single notes dominate, occasional double-stops (root +
|
||||||
|
# fifth on adjacent higher string two frets up, or root + octave two
|
||||||
|
# strings + two frets up), long sustains. Half-note pacing throughout
|
||||||
|
# for the same "give the player time to land cleanly" reasoning as
|
||||||
|
# guitar v2.
|
||||||
|
|
||||||
|
def exercise_open_strings_slow(t0):
|
||||||
|
"""All 4 open strings, low → high → low. Tests the lowest end of
|
||||||
|
YIN's range (E1 = 41 Hz) where the under-buffering threshold
|
||||||
|
kicks in."""
|
||||||
|
seq = [0, 1, 2, 3, 3, 2, 1, 0]
|
||||||
|
notes_out = []
|
||||||
|
for i, s in enumerate(seq):
|
||||||
|
notes_out.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, 0,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 4, 3)
|
||||||
|
return notes_out, [], 'Open strings (slow walk)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_fretted_5th_slow(t0):
|
||||||
|
"""5th fret on each string, ascending. Maps to A1 / D2 / G2 / C3
|
||||||
|
— comfortable register for hand position, no stretch."""
|
||||||
|
seq = [(s, 5) for s in range(N_STRINGS)]
|
||||||
|
notes_out = []
|
||||||
|
for i, (s, f) in enumerate(seq):
|
||||||
|
notes_out.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, f,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 4, 3)
|
||||||
|
return notes_out, [], 'Fretted positions (5th fret, slow walk)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_sustained(t0):
|
||||||
|
"""Three 4-second sustained roots across the range. Tests the
|
||||||
|
`_sustainStillHeld` active-glow path on bass tonalities."""
|
||||||
|
sus = 4.0
|
||||||
|
targets = [(0, 5), (1, 7), (2, 5)] # A1, E2, G2 — spread across mid-range
|
||||||
|
notes_out = []
|
||||||
|
for i, (s, f) in enumerate(targets):
|
||||||
|
notes_out.append(note(t0 + i * (sus + 1.0), s, f, sus=sus))
|
||||||
|
return notes_out, [], 'Sustained notes (3 holds, 4 s each)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_octave_walk(t0):
|
||||||
|
"""Octave jumps — common bass pattern (root note + octave on the
|
||||||
|
string two above). Pairs: (0,0)↔(2,2) = E1↔E2 octave. Plays root,
|
||||||
|
octave, root, octave at half-note pacing."""
|
||||||
|
pairs = [
|
||||||
|
(0, 0, 2, 2), # E1 ↔ E2
|
||||||
|
(1, 0, 3, 2), # A1 ↔ A2
|
||||||
|
]
|
||||||
|
notes_out = []
|
||||||
|
t = 0.0
|
||||||
|
for (sa, fa, sb, fb) in pairs:
|
||||||
|
notes_out.append(note(t0 + t, sa, fa, sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
t += 2 * SECONDS_PER_BEAT
|
||||||
|
notes_out.append(note(t0 + t, sb, fb, sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
t += 2 * SECONDS_PER_BEAT
|
||||||
|
notes_out.append(note(t0 + t, sa, fa, sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
t += 2 * SECONDS_PER_BEAT
|
||||||
|
notes_out.append(note(t0 + t, sb, fb, sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
t += 2 * SECONDS_PER_BEAT
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 2, 3)
|
||||||
|
return notes_out, [], 'Octave walks (root ↔ octave)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_walking_line(t0):
|
||||||
|
"""Walking bassline — root, third, fifth, sixth ascending, then
|
||||||
|
descending. Classic 4-bar walking pattern in A minor pentatonic
|
||||||
|
starting on A string open. Tests detection across a fretted run."""
|
||||||
|
# A1, C2, D2, E2 (ascend), E2, D2, C2, A1 (descend)
|
||||||
|
pattern = [
|
||||||
|
(1, 0), # A1
|
||||||
|
(1, 3), # C2
|
||||||
|
(1, 5), # D2
|
||||||
|
(1, 7), # E2
|
||||||
|
(1, 7), # E2
|
||||||
|
(1, 5), # D2
|
||||||
|
(1, 3), # C2
|
||||||
|
(1, 0), # A1
|
||||||
|
]
|
||||||
|
notes_out = []
|
||||||
|
for i, (s, f) in enumerate(pattern):
|
||||||
|
notes_out.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, f,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 4, 3)
|
||||||
|
return notes_out, [], 'Walking bassline (A minor pentatonic)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_root_fifth_pattern(t0):
|
||||||
|
"""Root + fifth alternation — single most common bass pattern in
|
||||||
|
rock / country. Plays (root, fifth, root, fifth) on each of two
|
||||||
|
voicings. The fifth sits on the next-higher string, 2 frets up
|
||||||
|
from the root — a one-finger reach with no string skip."""
|
||||||
|
# Root on (0, 0) = E1, fifth = (1, 2) = B1 (A string fret 2)
|
||||||
|
# Then root on (1, 0) = A1, fifth = (2, 2) = E2 (D string fret 2)
|
||||||
|
pattern = [
|
||||||
|
(0, 0), (1, 2), (0, 0), (1, 2),
|
||||||
|
(1, 0), (2, 2), (1, 0), (2, 2),
|
||||||
|
]
|
||||||
|
notes_out = []
|
||||||
|
for i, (s, f) in enumerate(pattern):
|
||||||
|
notes_out.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, f,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 4, 3)
|
||||||
|
return notes_out, [], 'Root + fifth pattern'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_double_stops(t0):
|
||||||
|
"""Two-string "chord" events — closest bass gets to chords.
|
||||||
|
Root + fifth simultaneously on adjacent strings, repeated 8
|
||||||
|
times at half-note pacing. Lets the chord scorer exercise the
|
||||||
|
2-string code path with bass-range frequencies."""
|
||||||
|
# Voicing: E1 + B1 (root + fifth on E + A strings)
|
||||||
|
voicing = [(0, 0), (1, 2)]
|
||||||
|
strums = 8
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
# Sloppak wire spec keeps chord-template fingers/frets in six-slot
|
||||||
|
# arrays even for bass (docs/sloppak-spec.md §chord-template), so we
|
||||||
|
# pad the unused two slots with -1; the chord notes themselves stay
|
||||||
|
# on strings 0–1.
|
||||||
|
template = {
|
||||||
|
'name': 'E5 (bass)', 'displayName': 'E5', 'arp': False,
|
||||||
|
'fingers': [-1, -1, -1, -1, -1, -1],
|
||||||
|
'frets': [ 0, 2, -1, -1, -1, -1],
|
||||||
|
}
|
||||||
|
for i in range(strums):
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in voicing]
|
||||||
|
chords_out.append(chord(t0 + i * 2 * SECONDS_PER_BEAT, 0, chord_notes))
|
||||||
|
return [], (chords_out, [template]), 'Double-stops (root + fifth, 8 strums)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_low_e_long_holds(t0):
|
||||||
|
"""Three long-held low E (open string, lowest note on the
|
||||||
|
instrument). Specifically targets YIN's under-buffering regime
|
||||||
|
— E1 at 41 Hz needs roughly 4096 samples for a confident lock
|
||||||
|
at 44.1 kHz, so the detector should spend ~95 ms accumulating
|
||||||
|
before it can report. Holds of 5 s each give the scorer huge
|
||||||
|
runway; if the detector can't lock here it can't lock anywhere."""
|
||||||
|
sus = 5.0
|
||||||
|
notes_out = []
|
||||||
|
for i in range(3):
|
||||||
|
notes_out.append(note(t0 + i * (sus + 0.5), 0, 0, sus=sus))
|
||||||
|
return notes_out, [], 'Long low-E holds (5 s each)'
|
||||||
|
|
||||||
|
|
||||||
|
EXERCISES = [
|
||||||
|
('A. Open strings (slow)', exercise_open_strings_slow),
|
||||||
|
('B. 5th-fret (slow)', exercise_fretted_5th_slow),
|
||||||
|
('C. Sustained notes', exercise_sustained),
|
||||||
|
('D. Octave walks', exercise_octave_walk),
|
||||||
|
('E. Walking bassline', exercise_walking_line),
|
||||||
|
('F. Root + fifth pattern', exercise_root_fifth_pattern),
|
||||||
|
('G. Double-stops (root + 5)', exercise_double_stops),
|
||||||
|
('H. Long low-E holds', exercise_low_e_long_holds),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ── Driver ─────────────────────────────────────────────────────────────
|
||||||
|
def build(out_dir: Path):
|
||||||
|
notes_all = []
|
||||||
|
chords_all = []
|
||||||
|
templates_all = []
|
||||||
|
sections = []
|
||||||
|
beats = []
|
||||||
|
|
||||||
|
t = INTRO_BARS * BAR_S
|
||||||
|
for label, fn in EXERCISES:
|
||||||
|
sections.append({'name': label, 'number': len(sections) + 1, 'time': round(t, 3)})
|
||||||
|
result = fn(t)
|
||||||
|
ns, ch_or_tuple, _desc = result
|
||||||
|
notes_all.extend(ns)
|
||||||
|
if isinstance(ch_or_tuple, tuple):
|
||||||
|
cs, tmpls = ch_or_tuple
|
||||||
|
# Rebase section-local chord template ids — see v1/v2
|
||||||
|
# builders for the full explanation. Bass v1 only has one
|
||||||
|
# chord exercise today (double-stops), but applying the
|
||||||
|
# same offset pattern future-proofs the driver against the
|
||||||
|
# day someone adds a second chord exercise that also uses
|
||||||
|
# local-zero-based ids.
|
||||||
|
offset = len(templates_all)
|
||||||
|
for c in cs:
|
||||||
|
c['id'] = c.get('id', 0) + offset
|
||||||
|
chords_all.extend(cs)
|
||||||
|
templates_all.extend(tmpls)
|
||||||
|
else:
|
||||||
|
chords_all.extend(ch_or_tuple)
|
||||||
|
t += EXERCISE_BARS * BAR_S
|
||||||
|
|
||||||
|
end_t = t + OUTRO_BARS * BAR_S
|
||||||
|
|
||||||
|
bar_count = 0
|
||||||
|
bt = 0.0
|
||||||
|
while bt < end_t:
|
||||||
|
is_downbeat = abs(bt % BAR_S) < 1e-3
|
||||||
|
if is_downbeat:
|
||||||
|
bar_count += 1
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': bar_count})
|
||||||
|
else:
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': -1})
|
||||||
|
bt += SECONDS_PER_BEAT
|
||||||
|
|
||||||
|
anchors = [{'time': 0.0, 'fret': 1, 'width': 12}]
|
||||||
|
for sec in sections:
|
||||||
|
anchors.append({'time': sec['time'], 'fret': 1, 'width': 12})
|
||||||
|
|
||||||
|
arrangement = {
|
||||||
|
'name': 'Bass',
|
||||||
|
# Pad to 6 slots even on bass — slopsmith's `tuning_name()` only
|
||||||
|
# recognises named tunings (E Standard, Drop D, etc.) on 6-element
|
||||||
|
# arrays, so a 4-element array shows up in the library card as the
|
||||||
|
# raw numeric form ("0 0 0 0") instead of "E Standard". The
|
||||||
|
# arrangement name ("Bass") + note positions still drive the
|
||||||
|
# detector's bass-specific behaviour; this just makes the library
|
||||||
|
# display friendly.
|
||||||
|
'tuning': [0] * 6,
|
||||||
|
'capo': 0,
|
||||||
|
'notes': sorted(notes_all, key=lambda n: n['t']),
|
||||||
|
'chords': sorted(chords_all, key=lambda c: c['t']),
|
||||||
|
'anchors': anchors,
|
||||||
|
'handshapes': [],
|
||||||
|
'templates': templates_all,
|
||||||
|
'beats': beats,
|
||||||
|
'sections': sections,
|
||||||
|
}
|
||||||
|
|
||||||
|
manifest = {
|
||||||
|
'title': 'Note Detect Bass Benchmark v1',
|
||||||
|
'artist': 'Slopsmith',
|
||||||
|
'album': 'Note Detection Benchmark',
|
||||||
|
'year': 2026,
|
||||||
|
'duration': round(end_t, 3),
|
||||||
|
'arrangements': [
|
||||||
|
{
|
||||||
|
'id': 'bass',
|
||||||
|
'name': 'Bass',
|
||||||
|
'file': 'arrangements/bass.json',
|
||||||
|
# Pad to 6 slots — see arrangement-level comment.
|
||||||
|
'tuning': [0] * 6,
|
||||||
|
'capo': 0,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
'stems': [
|
||||||
|
{'id': 'full', 'file': 'stems/full.ogg', 'default': True},
|
||||||
|
],
|
||||||
|
'benchmark': {
|
||||||
|
'id': 'slopsmith-note-detect-benchmark-bass',
|
||||||
|
'version': 1,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
out_dir = Path(out_dir)
|
||||||
|
if out_dir.exists():
|
||||||
|
# Defensive — see v1 builder. Only rmtree something that looks
|
||||||
|
# like a sloppak so a typo on the CLI doesn't nuke an unrelated
|
||||||
|
# directory.
|
||||||
|
if not (out_dir.suffix == '.sloppak'
|
||||||
|
or (out_dir / 'manifest.yaml').exists()):
|
||||||
|
raise RuntimeError(
|
||||||
|
f"refusing to rmtree {out_dir!r}: does not look like a sloppak "
|
||||||
|
f"(no .sloppak suffix, no manifest.yaml)."
|
||||||
|
)
|
||||||
|
shutil.rmtree(out_dir)
|
||||||
|
out_dir.mkdir(parents=True)
|
||||||
|
(out_dir / 'arrangements').mkdir()
|
||||||
|
(out_dir / 'stems').mkdir()
|
||||||
|
|
||||||
|
(out_dir / 'manifest.yaml').write_text(
|
||||||
|
yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
(out_dir / 'arrangements' / 'bass.json').write_text(
|
||||||
|
json.dumps(arrangement, separators=(',', ':')),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
|
||||||
|
wav_path = out_dir / 'stems' / 'full.wav'
|
||||||
|
write_click_wav(wav_path, end_t)
|
||||||
|
ogg_path = out_dir / 'stems' / 'full.ogg'
|
||||||
|
subprocess.run(
|
||||||
|
['ffmpeg', '-y', '-loglevel', 'error',
|
||||||
|
'-i', str(wav_path),
|
||||||
|
'-c:a', 'libvorbis', '-q:a', '5',
|
||||||
|
str(ogg_path)],
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
wav_path.unlink()
|
||||||
|
|
||||||
|
(out_dir / 'BENCHMARK.md').write_text(_benchmark_readme(end_t), encoding='utf-8')
|
||||||
|
_build_zip(out_dir)
|
||||||
|
|
||||||
|
print(f'Built {out_dir}')
|
||||||
|
print(f' {out_dir}.zip')
|
||||||
|
print(f' Duration: {end_t:.1f} s')
|
||||||
|
print(f' Notes: {len(arrangement["notes"])}')
|
||||||
|
print(f' Chords: {len(arrangement["chords"])}')
|
||||||
|
print(f' Templates:{len(arrangement["templates"])}')
|
||||||
|
|
||||||
|
|
||||||
|
def _build_zip(src_dir: Path):
|
||||||
|
"""Pack with fixed dates / attrs for zip-metadata reproducibility.
|
||||||
|
See v1 guitar builder docstring for full caveats."""
|
||||||
|
import zipfile
|
||||||
|
zip_path = src_dir.with_suffix(src_dir.suffix + '.zip')
|
||||||
|
if zip_path.exists():
|
||||||
|
zip_path.unlink()
|
||||||
|
with zipfile.ZipFile(zip_path, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
|
||||||
|
for p in sorted(src_dir.rglob('*')):
|
||||||
|
if p.is_file():
|
||||||
|
rel = p.relative_to(src_dir).as_posix()
|
||||||
|
info = zipfile.ZipInfo(filename=rel, date_time=(1980, 1, 1, 0, 0, 0))
|
||||||
|
info.compress_type = zipfile.ZIP_DEFLATED
|
||||||
|
info.external_attr = (0o644 & 0xFFFF) << 16
|
||||||
|
info.create_system = 3 # POSIX — see v1 builder for why
|
||||||
|
zf.writestr(info, p.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
def _benchmark_readme(duration_s):
|
||||||
|
return f"""# Slopsmith Note Detect Bass Benchmark — v1
|
||||||
|
|
||||||
|
A bass-focused companion to the guitar benchmarks
|
||||||
|
(note_detect_v1 + note_detect_v2). Tests `note_detect` against bass-
|
||||||
|
specific idioms: walking lines, octave jumps, root+fifth patterns,
|
||||||
|
double-stops, and long low-E holds that stress YIN's accumulator at
|
||||||
|
~41 Hz.
|
||||||
|
|
||||||
|
- **Tempo**: {BPM:g} BPM
|
||||||
|
- **Tuning**: E standard 4-string (E1 A1 D2 G2, no capo)
|
||||||
|
- **Audio**: metronome click track only — play *over* the click.
|
||||||
|
- **Duration**: {duration_s:.0f} s
|
||||||
|
|
||||||
|
## Sections
|
||||||
|
|
||||||
|
| Section | Tests |
|
||||||
|
|---|---|
|
||||||
|
| A. Open strings (slow walk) | Mono detection on each open string, low → high → low |
|
||||||
|
| B. 5th-fret (slow walk) | Fretted-note detection across the 4 strings |
|
||||||
|
| C. Sustained notes | 3 × 4-second held roots |
|
||||||
|
| D. Octave walks | Root ↔ octave alternation, 2 strings + 2 frets up |
|
||||||
|
| E. Walking bassline | A minor pentatonic ascending + descending |
|
||||||
|
| F. Root + fifth pattern | Classic rock bass pattern (8 events) |
|
||||||
|
| G. Double-stops | 2-string voicings — the chord-scorer test for bass |
|
||||||
|
| H. Long low-E holds | 3 × 5-second E1 holds, stresses YIN under-buffer regime |
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Diagnostic JSON schema is `note_detect.diagnostic.v1`. Filter
|
||||||
|
`benchmark_hint` to bucket bass vs guitar runs.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Built by `docs/benchmarks/note_detect_bass_v1/build_benchmark.py`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
out = Path(sys.argv[1]) if len(sys.argv) > 1 else Path('./note_detect_benchmark_bass_v1.sloppak')
|
||||||
|
build(out)
|
||||||
Binary file not shown.
@@ -0,0 +1,46 @@
|
|||||||
|
# Slopsmith Note Detect Benchmark — v1
|
||||||
|
|
||||||
|
A short test piece for tuning Slopsmith's `note_detect` plugin. Eight
|
||||||
|
exercises, each isolating a specific detection failure mode. Run with
|
||||||
|
**Detect** enabled, play through, then export the diagnostic JSON
|
||||||
|
(Settings → Plugins → Note Detection → Download Diagnostic JSON, or
|
||||||
|
the button on the end-of-session summary modal).
|
||||||
|
|
||||||
|
- **Tempo**: 90 BPM
|
||||||
|
- **Tuning**: E standard (no capo)
|
||||||
|
- **Audio**: metronome click track only (downbeat = louder + higher
|
||||||
|
tone). Play *over* the click — `note_detect` listens to your guitar
|
||||||
|
signal, not the audio in this file.
|
||||||
|
- **Duration**: 139 s
|
||||||
|
|
||||||
|
## Sections
|
||||||
|
|
||||||
|
| Section | Tests | Watch in the diagnostic |
|
||||||
|
|---|---|---|
|
||||||
|
| A. Open strings (low→high→low) | Basic mono detection on each open string | `pure` (mic/audio chain), per-string accuracy |
|
||||||
|
| B. 5th-fret positions | Fretted-note detection across all 6 strings | per-string variance |
|
||||||
|
| C. 12th-fret octaves | Higher-frequency detection — YIN's octave-up risk | `sharp` bin spiking |
|
||||||
|
| D. Sustained notes (4 s) | The `active` held-on-pitch glow | `sharp`/`flat` drift while held |
|
||||||
|
| E. Hammer-on / pull-off | Transient detection without a fresh pick attack | `pure` (no transient registered) |
|
||||||
|
| F. Power chords (2-string) | Chord leniency on sparse voicings | `chordPartial` |
|
||||||
|
| G. Open major chords | Chord leniency on dense voicings (E, A, D, G) | `chordPartial` |
|
||||||
|
| H. Bends (half- + whole-step) | Single-note pitch tolerance with pitch in motion | `sharp` bin |
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Share the JSON (schema `note_detect.diagnostic.v1`). It includes:
|
||||||
|
|
||||||
|
- Hit/miss totals split single-note vs chord
|
||||||
|
- Primary-cause bin per miss (pure / chord-partial / early / late / sharp / flat)
|
||||||
|
- Per-string hit rate
|
||||||
|
- Signed timing- and pitch-error percentiles (p10 / median / p90)
|
||||||
|
- Detection settings snapshot (method, tolerances, leniency)
|
||||||
|
- Per-judgment event log (capped at 2000 events) with the chart note's
|
||||||
|
technique flags so each miss can be re-binned by `SUS`/`B`/`H`/etc. offline
|
||||||
|
- `benchmark_hint`: `{title, artist, arrangement}` — filter on these
|
||||||
|
to bucket reports from different runs of this benchmark.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Built by `docs/benchmarks/note_detect_v1/build_benchmark.py` in the
|
||||||
|
slopsmith repo. Tweak the exercise list there and regenerate.
|
||||||
@@ -0,0 +1,601 @@
|
|||||||
|
"""Builds the Note Detect Benchmark sloppak (v1).
|
||||||
|
|
||||||
|
A reproducible, distributable test piece for the note_detect plugin: 8
|
||||||
|
short exercises designed to isolate specific failure modes (open-string
|
||||||
|
mono, fretted positions, octaves, sustained held notes, hammer-on /
|
||||||
|
pull-off, sparse power chords, dense open chords, bends).
|
||||||
|
|
||||||
|
How to run inside the slopsmith container (recommended — has ffmpeg +
|
||||||
|
pyyaml already):
|
||||||
|
|
||||||
|
docker cp docs/benchmarks/note_detect_v1/build_benchmark.py \
|
||||||
|
slopsmith-web-1:/tmp/build_benchmark.py
|
||||||
|
docker exec slopsmith-web-1 python /tmp/build_benchmark.py \
|
||||||
|
/app/static/sloppak_cache/note_detect_benchmark_v1.sloppak
|
||||||
|
|
||||||
|
The output sloppak lands under `static/sloppak_cache/` on the host
|
||||||
|
(bind-mounted into the container). Copy / zip it from there.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import shutil
|
||||||
|
import struct
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import wave
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import yaml # bundled with the slopsmith image
|
||||||
|
|
||||||
|
# ── Benchmark parameters ────────────────────────────────────────────────
|
||||||
|
BPM = 90.0
|
||||||
|
SECONDS_PER_BEAT = 60.0 / BPM # 0.6667
|
||||||
|
BEATS_PER_BAR = 4
|
||||||
|
BAR_S = BEATS_PER_BAR * SECONDS_PER_BEAT # 2.667
|
||||||
|
INTRO_BARS = 2 # silence before the first event
|
||||||
|
OUTRO_BARS = 2 # tail after the last
|
||||||
|
EXERCISE_BARS = 6 # length of each exercise
|
||||||
|
|
||||||
|
# Standard E-tuning open MIDI per string, low → high (matches lib/tunings
|
||||||
|
# convention used by note_detect when arrangement is 'guitar').
|
||||||
|
OPEN_MIDI = [40, 45, 50, 55, 59, 64] # E2 A2 D3 G3 B3 E4
|
||||||
|
|
||||||
|
SR = 44100 # sample rate for the click WAV
|
||||||
|
|
||||||
|
|
||||||
|
# ── Click-track audio generator ────────────────────────────────────────
|
||||||
|
def _sine_burst(freq_hz, duration_s, amplitude):
|
||||||
|
"""Short sine burst with a linear attack/release envelope so the
|
||||||
|
click reads as a tick, not a pop."""
|
||||||
|
n = int(SR * duration_s)
|
||||||
|
out = []
|
||||||
|
fade = max(1, int(0.004 * SR)) # 4 ms fade in + out
|
||||||
|
for i in range(n):
|
||||||
|
env = 1.0
|
||||||
|
if i < fade:
|
||||||
|
env = i / fade
|
||||||
|
elif i >= n - fade:
|
||||||
|
env = (n - 1 - i) / fade
|
||||||
|
s = math.sin(2 * math.pi * freq_hz * (i / SR)) * amplitude * env
|
||||||
|
out.append(s)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def write_click_wav(path: Path, total_duration_s: float):
|
||||||
|
"""A click on every beat; the downbeat (beat 0 of each bar) is louder
|
||||||
|
and a tone higher. Steady reference for the player; the chart's
|
||||||
|
event times sit on the same beat grid."""
|
||||||
|
n_total = int(math.ceil(total_duration_s * SR))
|
||||||
|
buf = [0.0] * n_total
|
||||||
|
|
||||||
|
click_dur = 0.045
|
||||||
|
downbeat_tone = 1500
|
||||||
|
upbeat_tone = 1000
|
||||||
|
downbeat_amp = 0.22
|
||||||
|
upbeat_amp = 0.12
|
||||||
|
|
||||||
|
beat_idx = 0
|
||||||
|
t = 0.0
|
||||||
|
while t < total_duration_s - click_dur:
|
||||||
|
is_downbeat = (beat_idx % BEATS_PER_BAR) == 0
|
||||||
|
click = _sine_burst(
|
||||||
|
downbeat_tone if is_downbeat else upbeat_tone,
|
||||||
|
click_dur,
|
||||||
|
downbeat_amp if is_downbeat else upbeat_amp,
|
||||||
|
)
|
||||||
|
i0 = int(t * SR)
|
||||||
|
for j, v in enumerate(click):
|
||||||
|
if i0 + j < n_total:
|
||||||
|
buf[i0 + j] += v
|
||||||
|
t += SECONDS_PER_BEAT
|
||||||
|
beat_idx += 1
|
||||||
|
|
||||||
|
# Soft clip to keep within 16-bit headroom even if a future tweak
|
||||||
|
# piles bursts up.
|
||||||
|
pcm = bytearray()
|
||||||
|
for v in buf:
|
||||||
|
s = max(-1.0, min(1.0, v))
|
||||||
|
pcm.extend(struct.pack('<h', int(s * 32700)))
|
||||||
|
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with wave.open(str(path), 'wb') as w:
|
||||||
|
w.setnchannels(1)
|
||||||
|
w.setsampwidth(2)
|
||||||
|
w.setframerate(SR)
|
||||||
|
w.writeframes(bytes(pcm))
|
||||||
|
|
||||||
|
|
||||||
|
# ── Chart helpers ─────────────────────────────────────────────────────
|
||||||
|
def note(t, s, f, sus=0.0, **flags):
|
||||||
|
"""Build a single-note dict in the sloppak wire format. Defaults
|
||||||
|
match the wire-format defaults from docs/sloppak-spec.md §3.2."""
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
's': s,
|
||||||
|
'f': f,
|
||||||
|
'sus': round(sus, 3),
|
||||||
|
'sl': flags.get('sl', -1),
|
||||||
|
'slu': flags.get('slu', -1),
|
||||||
|
'bn': flags.get('bn', 0.0),
|
||||||
|
'ho': flags.get('ho', False),
|
||||||
|
'po': flags.get('po', False),
|
||||||
|
'hm': flags.get('hm', False),
|
||||||
|
'hp': flags.get('hp', False),
|
||||||
|
'pm': flags.get('pm', False),
|
||||||
|
'mt': flags.get('mt', False),
|
||||||
|
'vb': flags.get('vb', False),
|
||||||
|
'tr': flags.get('tr', False),
|
||||||
|
'ac': flags.get('ac', False),
|
||||||
|
'tp': flags.get('tp', False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord(t, id_, notes):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
'id': id_,
|
||||||
|
'hd': False,
|
||||||
|
'notes': notes,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord_note(s, f, sus=0.0, **flags):
|
||||||
|
n = note(0.0, s, f, sus, **flags)
|
||||||
|
n.pop('t') # chord notes inherit the chord's time
|
||||||
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
# ── Exercises ─────────────────────────────────────────────────────────
|
||||||
|
# Each returns a 3-tuple `(notes, chords_or_with_templates, description)`.
|
||||||
|
# The middle slot is overloaded so single-note exercises don't have to
|
||||||
|
# carry a useless empty `templates` list:
|
||||||
|
# • Single-note exercises return `(notes, [], desc)` — second slot is
|
||||||
|
# just the (empty) chords list.
|
||||||
|
# • Chord exercises return `(notes, (chords, templates), desc)` — the
|
||||||
|
# driver unpacks the tuple when it sees one (see `build()`).
|
||||||
|
# Exercise start times are computed by the driver; helpers use `t0` as
|
||||||
|
# the exercise's bar-aligned start time, then place events relative to it.
|
||||||
|
|
||||||
|
def exercise_open_strings(t0):
|
||||||
|
"""Single notes — open strings, low → high → low, quarter notes."""
|
||||||
|
seq = [0, 1, 2, 3, 4, 5, 5, 4, 3, 2, 1, 0] # 12 notes = 3 bars at q-note
|
||||||
|
notes = []
|
||||||
|
for i, s in enumerate(seq):
|
||||||
|
notes.append(note(t0 + i * SECONDS_PER_BEAT, s, 0, sus=SECONDS_PER_BEAT * 0.9))
|
||||||
|
# Cap the last note's sustain into the trailing bar so it rings out
|
||||||
|
notes[-1]['sus'] = round(SECONDS_PER_BEAT * 3, 3)
|
||||||
|
return notes, [], 'Open strings (low→high→low)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_fretted_positions(t0):
|
||||||
|
"""Each string's 5th fret, ascending then descending. Tests basic
|
||||||
|
fretted-note detection across the range."""
|
||||||
|
seq = [(s, 5) for s in range(6)] + [(s, 5) for s in range(5, -1, -1)]
|
||||||
|
notes = []
|
||||||
|
for i, (s, f) in enumerate(seq):
|
||||||
|
notes.append(note(t0 + i * SECONDS_PER_BEAT, s, f, sus=SECONDS_PER_BEAT * 0.9))
|
||||||
|
notes[-1]['sus'] = round(SECONDS_PER_BEAT * 3, 3)
|
||||||
|
return notes, [], 'Fretted positions (5th fret on each string)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_octaves(t0):
|
||||||
|
"""12th-fret octaves on each string. Tests detection at higher
|
||||||
|
frequencies where YIN can lock onto the second harmonic."""
|
||||||
|
notes = []
|
||||||
|
# 6 notes, half-note each (2 beats), so the player has time to land
|
||||||
|
# cleanly. 6 × 2 = 12 beats = 3 bars.
|
||||||
|
for i, s in enumerate(range(6)):
|
||||||
|
notes.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, 12,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
notes[-1]['sus'] = round(SECONDS_PER_BEAT * 3, 3)
|
||||||
|
return notes, [], '12th-fret octaves'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_sustained(t0):
|
||||||
|
"""Four-second sustained notes. The renderer's `active` glow
|
||||||
|
requires the provider to keep returning state — exercises the
|
||||||
|
on-pitch hold check (`_sustainStillHeld`)."""
|
||||||
|
sus = 4.0
|
||||||
|
# Three targets spread across the range (low / mid / high). Held at
|
||||||
|
# 3 to keep the whole exercise within the section's 16 s slot —
|
||||||
|
# 4 events with a 4-s sustain at a 5-s cadence would end at t0+19
|
||||||
|
# and bleed 3 s into the next section's note-detect window, which
|
||||||
|
# contaminates the bin attribution we promise section-by-section.
|
||||||
|
targets = [(0, 5), (2, 7), (5, 5)]
|
||||||
|
notes = []
|
||||||
|
# One every 5 seconds (4-sec sustain + 1-sec gap). 3 events × 5 s
|
||||||
|
# = 14 s of music, comfortably inside EXERCISE_BARS * BAR_S = 16 s.
|
||||||
|
for i, (s, f) in enumerate(targets):
|
||||||
|
notes.append(note(t0 + i * (sus + 1.0), s, f, sus=sus))
|
||||||
|
return notes, [], 'Sustained notes (4 s each, on-pitch hold)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_hammer_pull(t0):
|
||||||
|
"""Open → hammer-on → pull-off. Hammer-ons and pull-offs have no
|
||||||
|
fresh pick attack, so transient detection is what's tested."""
|
||||||
|
notes = []
|
||||||
|
# Pattern per bar: D3 (s=1, f=5 — A-string fretted at 5) picked, HO
|
||||||
|
# to f=7 (E3), PO back to f=5 (D3). HO/PO flags ride the destination
|
||||||
|
# note, not the source — that's where the technique is performed.
|
||||||
|
# Use 4 bars.
|
||||||
|
for bar in range(4):
|
||||||
|
bt = t0 + bar * BAR_S
|
||||||
|
notes.append(note(bt + 0 * SECONDS_PER_BEAT, 1, 5, sus=0.4)) # pluck D3
|
||||||
|
notes.append(note(bt + 1 * SECONDS_PER_BEAT, 1, 7, sus=0.4, ho=True))
|
||||||
|
notes.append(note(bt + 2 * SECONDS_PER_BEAT, 1, 5, sus=0.4, po=True))
|
||||||
|
# rest on beat 4
|
||||||
|
return notes, [], 'Hammer-on / pull-off (no pick attack)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_power_chords(t0):
|
||||||
|
"""Two-string power chords. Sparse voicing tests whether the chord
|
||||||
|
leniency threshold is appropriate for 2-string chord events."""
|
||||||
|
# Wire format: s=0 is the lowest-pitched string (low E on guitar),
|
||||||
|
# s=5 the highest (high E). Two-string power-chord voicings, each
|
||||||
|
# rooted on the lower of the two strings:
|
||||||
|
# E5 — low E open + A fret 2 (E2 + B2)
|
||||||
|
# A5 — A open + D fret 2 (A2 + E3)
|
||||||
|
# D5 — D open + G fret 2 (D3 + A3)
|
||||||
|
# G5 — G open + B fret 3 (G3 + D4)
|
||||||
|
voicings = [
|
||||||
|
('E5', [(0, 0), (1, 2)]),
|
||||||
|
('A5', [(1, 0), (2, 2)]),
|
||||||
|
('D5', [(2, 0), (3, 2)]),
|
||||||
|
('G5', [(3, 0), (4, 3)]),
|
||||||
|
]
|
||||||
|
templates = []
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
# 8 chord events over 8 half-note slots (4 bars at half notes).
|
||||||
|
pattern = list(range(4)) + list(range(4)) # play each voicing twice
|
||||||
|
for slot, idx in enumerate(pattern):
|
||||||
|
name, sf = voicings[idx]
|
||||||
|
tmpl_id = idx
|
||||||
|
if slot < len(voicings): # only add each template once
|
||||||
|
frets = [-1] * 6
|
||||||
|
for (s, f) in sf:
|
||||||
|
frets[s] = f
|
||||||
|
templates.append({
|
||||||
|
'name': name,
|
||||||
|
'displayName': name,
|
||||||
|
'arp': False,
|
||||||
|
'fingers': [-1] * 6,
|
||||||
|
'frets': frets,
|
||||||
|
})
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in sf]
|
||||||
|
chords_out.append(chord(t0 + slot * 2 * SECONDS_PER_BEAT, tmpl_id, chord_notes))
|
||||||
|
return [], (chords_out, templates), 'Power chords (2-string sparse voicings)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_open_chords(t0):
|
||||||
|
"""Open major chords. Dense voicings test whether the leniency
|
||||||
|
threshold is too strict when the player can't reliably ring every
|
||||||
|
string."""
|
||||||
|
# Standard open-chord voicings, low → high string. Strings with `-1`
|
||||||
|
# in the template's frets list aren't part of the chord.
|
||||||
|
# E open: E0 A2 D2 G1 B0 e0 (all 6 strings)
|
||||||
|
# A open: — A0 D2 G2 B2 e0 (skip low E)
|
||||||
|
# D open: — — D0 G2 B3 e2 (skip low E + A)
|
||||||
|
# G open: E3 A2 D0 G0 B0 e3 (all 6 strings; common 6-string fingering)
|
||||||
|
voicings = [
|
||||||
|
('E', [(0, 0), (1, 2), (2, 2), (3, 1), (4, 0), (5, 0)]),
|
||||||
|
('A', [(1, 0), (2, 2), (3, 2), (4, 2), (5, 0)]),
|
||||||
|
('D', [(2, 0), (3, 2), (4, 3), (5, 2)]),
|
||||||
|
('G', [(0, 3), (1, 2), (2, 0), (3, 0), (4, 0), (5, 3)]),
|
||||||
|
]
|
||||||
|
templates = []
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
pattern = list(range(4)) + list(range(4))
|
||||||
|
for slot, idx in enumerate(pattern):
|
||||||
|
name, sf = voicings[idx]
|
||||||
|
# Local-zero-based template id. The driver in `build()` rebases
|
||||||
|
# these onto the global `templates_all` index before emitting
|
||||||
|
# the arrangement, so we don't need to pre-offset here — and
|
||||||
|
# in fact mustn't, since double-offsetting would point at
|
||||||
|
# template ids past the end of the list.
|
||||||
|
tmpl_id = idx
|
||||||
|
if slot < len(voicings):
|
||||||
|
frets = [-1] * 6
|
||||||
|
for (s, f) in sf:
|
||||||
|
frets[s] = f
|
||||||
|
templates.append({
|
||||||
|
'name': name,
|
||||||
|
'displayName': name,
|
||||||
|
'arp': False,
|
||||||
|
'fingers': [-1] * 6,
|
||||||
|
'frets': frets,
|
||||||
|
})
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in sf]
|
||||||
|
chords_out.append(chord(t0 + slot * 2 * SECONDS_PER_BEAT, tmpl_id, chord_notes))
|
||||||
|
return [], (chords_out, templates), 'Open major chords (E A D G — dense)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_bends(t0):
|
||||||
|
"""Half-step and whole-step bends. Bends shift pitch mid-note —
|
||||||
|
tests whether the single-note pitch tolerance is wide enough."""
|
||||||
|
notes = []
|
||||||
|
# Whole-step bend on G string fret 7 (D4 → E4): bn=2.0 semitones.
|
||||||
|
# Half-step bend on B string fret 8 (G4 → G#4): bn=1.0 semitone.
|
||||||
|
pattern = [
|
||||||
|
(3, 7, 2.0), # whole-step on G string
|
||||||
|
(4, 8, 1.0), # half-step on B string
|
||||||
|
(3, 7, 2.0),
|
||||||
|
(4, 8, 1.0),
|
||||||
|
]
|
||||||
|
for i, (s, f, bn) in enumerate(pattern):
|
||||||
|
# 4 bends, half-note each (2 beats), 4 × 2 = 8 beats = 2 bars.
|
||||||
|
notes.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, f,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6, bn=bn))
|
||||||
|
notes[-1]['sus'] = round(SECONDS_PER_BEAT * 3, 3)
|
||||||
|
return notes, [], 'Bends (half-step + whole-step)'
|
||||||
|
|
||||||
|
|
||||||
|
EXERCISES = [
|
||||||
|
('A. Open strings', exercise_open_strings),
|
||||||
|
('B. 5th-fret positions', exercise_fretted_positions),
|
||||||
|
('C. 12th-fret octaves', exercise_octaves),
|
||||||
|
('D. Sustained notes', exercise_sustained),
|
||||||
|
('E. Hammer / pull', exercise_hammer_pull),
|
||||||
|
('F. Power chords', exercise_power_chords),
|
||||||
|
('G. Open chords', exercise_open_chords),
|
||||||
|
('H. Bends', exercise_bends),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ── Driver ─────────────────────────────────────────────────────────────
|
||||||
|
def build(out_dir: Path):
|
||||||
|
notes_all = []
|
||||||
|
chords_all = []
|
||||||
|
templates_all = []
|
||||||
|
sections = []
|
||||||
|
beats = []
|
||||||
|
|
||||||
|
t = INTRO_BARS * BAR_S
|
||||||
|
for label, fn in EXERCISES:
|
||||||
|
sections.append({'name': label, 'number': len(sections) + 1, 'time': round(t, 3)})
|
||||||
|
result = fn(t)
|
||||||
|
ns, ch_or_tuple, _desc = result
|
||||||
|
notes_all.extend(ns)
|
||||||
|
if isinstance(ch_or_tuple, tuple):
|
||||||
|
cs, tmpls = ch_or_tuple
|
||||||
|
# Rebase section-local chord template ids onto the global
|
||||||
|
# `templates_all` list — see v2 builder for the full
|
||||||
|
# explanation. Multiple chord exercises in this benchmark
|
||||||
|
# (power, open) each use ids 0..N locally; without
|
||||||
|
# offsetting, open-chord events would silently point at
|
||||||
|
# power-chord templates.
|
||||||
|
offset = len(templates_all)
|
||||||
|
for c in cs:
|
||||||
|
c['id'] = c.get('id', 0) + offset
|
||||||
|
chords_all.extend(cs)
|
||||||
|
templates_all.extend(tmpls)
|
||||||
|
else:
|
||||||
|
chords_all.extend(ch_or_tuple)
|
||||||
|
t += EXERCISE_BARS * BAR_S
|
||||||
|
|
||||||
|
end_t = t + OUTRO_BARS * BAR_S
|
||||||
|
|
||||||
|
# Beats array — one entry per beat, measure markers on downbeats.
|
||||||
|
bar_count = 0
|
||||||
|
bt = 0.0
|
||||||
|
while bt < end_t:
|
||||||
|
is_downbeat = abs(bt % BAR_S) < 1e-3
|
||||||
|
if is_downbeat:
|
||||||
|
bar_count += 1
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': bar_count})
|
||||||
|
else:
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': -1})
|
||||||
|
bt += SECONDS_PER_BEAT
|
||||||
|
|
||||||
|
# Anchors — keep the highway zoom wide enough for everything on
|
||||||
|
# screen. One anchor at start, then per-exercise re-anchors so the
|
||||||
|
# camera doesn't drift to the wrong neighbourhood between sections.
|
||||||
|
anchors = [{'time': 0.0, 'fret': 1, 'width': 12}]
|
||||||
|
for sec in sections:
|
||||||
|
anchors.append({'time': sec['time'], 'fret': 1, 'width': 12})
|
||||||
|
|
||||||
|
arrangement = {
|
||||||
|
'name': 'Lead',
|
||||||
|
'tuning': [0, 0, 0, 0, 0, 0],
|
||||||
|
'capo': 0,
|
||||||
|
'notes': sorted(notes_all, key=lambda n: n['t']),
|
||||||
|
'chords': sorted(chords_all, key=lambda c: c['t']),
|
||||||
|
'anchors': anchors,
|
||||||
|
'handshapes': [],
|
||||||
|
'templates': templates_all,
|
||||||
|
'beats': beats,
|
||||||
|
'sections': sections,
|
||||||
|
}
|
||||||
|
|
||||||
|
manifest = {
|
||||||
|
'title': 'Note Detect Benchmark v1',
|
||||||
|
'artist': 'Slopsmith',
|
||||||
|
'album': 'Note Detection Benchmark',
|
||||||
|
'year': 2026,
|
||||||
|
'duration': round(end_t, 3),
|
||||||
|
'arrangements': [
|
||||||
|
{
|
||||||
|
'id': 'lead',
|
||||||
|
'name': 'Lead',
|
||||||
|
'file': 'arrangements/lead.json',
|
||||||
|
'tuning': [0, 0, 0, 0, 0, 0],
|
||||||
|
'capo': 0,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
'stems': [
|
||||||
|
{'id': 'full', 'file': 'stems/full.ogg', 'default': True},
|
||||||
|
],
|
||||||
|
# Non-standard key — picked up by future tooling that wants to
|
||||||
|
# detect "this is the benchmark, schema v1". The loader ignores it.
|
||||||
|
'benchmark': {
|
||||||
|
'id': 'slopsmith-note-detect-benchmark',
|
||||||
|
'version': 1,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── Write files ──
|
||||||
|
out_dir = Path(out_dir)
|
||||||
|
if out_dir.exists():
|
||||||
|
# Defensive: only blow away a directory that LOOKS like a
|
||||||
|
# sloppak (has a manifest.yaml at its root, or matches the
|
||||||
|
# `.sloppak` suffix this builder generates). A user who
|
||||||
|
# passes e.g. `python build_benchmark.py /tmp` by accident
|
||||||
|
# otherwise loses `/tmp` to a recursive delete.
|
||||||
|
if not (out_dir.suffix == '.sloppak'
|
||||||
|
or (out_dir / 'manifest.yaml').exists()):
|
||||||
|
raise RuntimeError(
|
||||||
|
f"refusing to rmtree {out_dir!r}: does not look like a sloppak "
|
||||||
|
f"(no .sloppak suffix, no manifest.yaml). Pass a path ending in "
|
||||||
|
f".sloppak or pointing at an existing sloppak directory."
|
||||||
|
)
|
||||||
|
shutil.rmtree(out_dir)
|
||||||
|
out_dir.mkdir(parents=True)
|
||||||
|
(out_dir / 'arrangements').mkdir()
|
||||||
|
(out_dir / 'stems').mkdir()
|
||||||
|
|
||||||
|
(out_dir / 'manifest.yaml').write_text(
|
||||||
|
yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
(out_dir / 'arrangements' / 'lead.json').write_text(
|
||||||
|
json.dumps(arrangement, separators=(',', ':')),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
|
||||||
|
# Click track. Write WAV first, then transcode to OGG via ffmpeg —
|
||||||
|
# the loader expects `stems/full.ogg`.
|
||||||
|
wav_path = out_dir / 'stems' / 'full.wav'
|
||||||
|
write_click_wav(wav_path, end_t)
|
||||||
|
ogg_path = out_dir / 'stems' / 'full.ogg'
|
||||||
|
subprocess.run(
|
||||||
|
['ffmpeg', '-y', '-loglevel', 'error',
|
||||||
|
'-i', str(wav_path),
|
||||||
|
'-c:a', 'libvorbis', '-q:a', '5',
|
||||||
|
str(ogg_path)],
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
wav_path.unlink() # ogg is canonical; wav was scaffolding
|
||||||
|
|
||||||
|
# Distribution README — ships inside the sloppak so other devs can
|
||||||
|
# follow the exercises without external docs. The loader ignores
|
||||||
|
# files it doesn't know about, so this travels with the package.
|
||||||
|
(out_dir / 'BENCHMARK.md').write_text(_benchmark_readme(end_t), encoding='utf-8')
|
||||||
|
|
||||||
|
# Zip-archive distribution form alongside the directory. Built with
|
||||||
|
# the stdlib zipfile module so paths use forward slashes regardless
|
||||||
|
# of the host OS — PowerShell's Compress-Archive on Windows produces
|
||||||
|
# backslash paths inside the zip, which the loader (running on
|
||||||
|
# Linux) then reads as literal filenames instead of directory
|
||||||
|
# separators and quietly drops every arrangement.
|
||||||
|
_build_zip(out_dir)
|
||||||
|
|
||||||
|
print(f'Built {out_dir}')
|
||||||
|
print(f' {out_dir}.zip')
|
||||||
|
print(f' Duration: {end_t:.1f} s')
|
||||||
|
print(f' Notes: {len(arrangement["notes"])}')
|
||||||
|
print(f' Chords: {len(arrangement["chords"])}')
|
||||||
|
print(f' Templates:{len(arrangement["templates"])}')
|
||||||
|
|
||||||
|
|
||||||
|
def _build_zip(src_dir: Path):
|
||||||
|
"""Pack `src_dir` into `<src_dir>.zip` with forward-slash paths.
|
||||||
|
|
||||||
|
Zip-level reproducibility: every entry uses a fixed `date_time` (the
|
||||||
|
zip spec's earliest legal value, 1980-01-01 00:00:00), a fixed
|
||||||
|
`external_attr` (rw-r--r--), and an explicit `ZipInfo` so the
|
||||||
|
archive metadata depends only on contents, not on when the build
|
||||||
|
ran. JSON / YAML / MD entries are byte-identical across rebuilds.
|
||||||
|
|
||||||
|
Caveat: the bundled `stems/full.ogg` is still non-deterministic
|
||||||
|
across rebuilds because libvorbis writes a random bitstream serial
|
||||||
|
number to every Ogg page (~1% of the file's bytes are container
|
||||||
|
framing, not audio). The audio PCM that the detector listens to is
|
||||||
|
deterministic; only the container headers differ. So a diff of the
|
||||||
|
tracked sloppak will always show OGG churn after `_build_zip`, but
|
||||||
|
the chart, manifest, and audible signal are stable. If a future PR
|
||||||
|
needs full byte-stability, it can either cache a hand-built OGG or
|
||||||
|
switch the stem to FLAC.
|
||||||
|
"""
|
||||||
|
import zipfile
|
||||||
|
zip_path = src_dir.with_suffix(src_dir.suffix + '.zip')
|
||||||
|
if zip_path.exists():
|
||||||
|
zip_path.unlink()
|
||||||
|
with zipfile.ZipFile(zip_path, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
|
||||||
|
for p in sorted(src_dir.rglob('*')):
|
||||||
|
if p.is_file():
|
||||||
|
# Force POSIX-style arcname so a Windows build still
|
||||||
|
# emits a Linux-loadable archive.
|
||||||
|
rel = p.relative_to(src_dir).as_posix()
|
||||||
|
info = zipfile.ZipInfo(filename=rel, date_time=(1980, 1, 1, 0, 0, 0))
|
||||||
|
info.compress_type = zipfile.ZIP_DEFLATED
|
||||||
|
# rw-r--r-- in the upper 16 bits where ZIP stores
|
||||||
|
# external attrs on POSIX. Avoids "executable" / weird
|
||||||
|
# permission bits leaking from the host filesystem.
|
||||||
|
info.external_attr = (0o644 & 0xFFFF) << 16
|
||||||
|
# Force POSIX (3) for the create-system byte so the
|
||||||
|
# zip's central-directory metadata doesn't drift when
|
||||||
|
# the same builder runs on Windows vs Linux. Python's
|
||||||
|
# default is host-dependent (3 on POSIX, 0 on Windows)
|
||||||
|
# and was the last source of zip-level non-determinism
|
||||||
|
# after the date_time + external_attr fixes.
|
||||||
|
info.create_system = 3
|
||||||
|
zf.writestr(info, p.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
def _benchmark_readme(duration_s):
|
||||||
|
return f"""# Slopsmith Note Detect Benchmark — v1
|
||||||
|
|
||||||
|
A short test piece for tuning Slopsmith's `note_detect` plugin. Eight
|
||||||
|
exercises, each isolating a specific detection failure mode. Run with
|
||||||
|
**Detect** enabled, play through, then export the diagnostic JSON
|
||||||
|
(Settings → Plugins → Note Detection → Download Diagnostic JSON, or
|
||||||
|
the button on the end-of-session summary modal).
|
||||||
|
|
||||||
|
- **Tempo**: {BPM:g} BPM
|
||||||
|
- **Tuning**: E standard (no capo)
|
||||||
|
- **Audio**: metronome click track only (downbeat = louder + higher
|
||||||
|
tone). Play *over* the click — `note_detect` listens to your guitar
|
||||||
|
signal, not the audio in this file.
|
||||||
|
- **Duration**: {duration_s:.0f} s
|
||||||
|
|
||||||
|
## Sections
|
||||||
|
|
||||||
|
| Section | Tests | Watch in the diagnostic |
|
||||||
|
|---|---|---|
|
||||||
|
| A. Open strings (low→high→low) | Basic mono detection on each open string | `pure` (mic/audio chain), per-string accuracy |
|
||||||
|
| B. 5th-fret positions | Fretted-note detection across all 6 strings | per-string variance |
|
||||||
|
| C. 12th-fret octaves | Higher-frequency detection — YIN's octave-up risk | `sharp` bin spiking |
|
||||||
|
| D. Sustained notes (4 s) | The `active` held-on-pitch glow | `sharp`/`flat` drift while held |
|
||||||
|
| E. Hammer-on / pull-off | Transient detection without a fresh pick attack | `pure` (no transient registered) |
|
||||||
|
| F. Power chords (2-string) | Chord leniency on sparse voicings | `chordPartial` |
|
||||||
|
| G. Open major chords | Chord leniency on dense voicings (E, A, D, G) | `chordPartial` |
|
||||||
|
| H. Bends (half- + whole-step) | Single-note pitch tolerance with pitch in motion | `sharp` bin |
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Share the JSON (schema `note_detect.diagnostic.v1`). It includes:
|
||||||
|
|
||||||
|
- Hit/miss totals split single-note vs chord
|
||||||
|
- Primary-cause bin per miss (pure / chord-partial / early / late / sharp / flat)
|
||||||
|
- Per-string hit rate
|
||||||
|
- Signed timing- and pitch-error percentiles (p10 / median / p90)
|
||||||
|
- Detection settings snapshot (method, tolerances, leniency)
|
||||||
|
- Per-judgment event log (capped at 2000 events) with the chart note's
|
||||||
|
technique flags so each miss can be re-binned by `SUS`/`B`/`H`/etc. offline
|
||||||
|
- `benchmark_hint`: `{{title, artist, arrangement}}` — filter on these
|
||||||
|
to bucket reports from different runs of this benchmark.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Built by `docs/benchmarks/note_detect_v1/build_benchmark.py` in the
|
||||||
|
slopsmith repo. Tweak the exercise list there and regenerate.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) != 2:
|
||||||
|
print('usage: build_benchmark.py <output-sloppak-dir>', file=sys.stderr)
|
||||||
|
sys.exit(2)
|
||||||
|
build(Path(sys.argv[1]))
|
||||||
Binary file not shown.
@@ -0,0 +1,37 @@
|
|||||||
|
# Slopsmith Note Detect Benchmark — v2
|
||||||
|
|
||||||
|
A slower-paced companion to v1, focused on what players can actually
|
||||||
|
land cleanly. Half-note spacing throughout (~1.33 s between events at
|
||||||
|
90 BPM), with multiple **strumming** sections — single chord voicings
|
||||||
|
repeated at half-note cadence — to exercise the chord scorer's
|
||||||
|
consistency across a sequence of strikes.
|
||||||
|
|
||||||
|
- **Tempo**: 90 BPM
|
||||||
|
- **Tuning**: E standard (no capo)
|
||||||
|
- **Audio**: metronome click track only — play *over* the click.
|
||||||
|
- **Duration**: 181 s
|
||||||
|
|
||||||
|
## Sections
|
||||||
|
|
||||||
|
| Section | Tests |
|
||||||
|
|---|---|
|
||||||
|
| A. Open strings (slow walk) | Basic mono detection, low → high → low at half-note pacing |
|
||||||
|
| B. 5th-fret (slow walk) | Fretted-note detection, ascending half-notes |
|
||||||
|
| C. Sustained notes | Long-hold pitch detection, 4 s each |
|
||||||
|
| D. E5 power chord strum | Chord scorer on a 2-string voicing, 8 strums |
|
||||||
|
| E. A5 / E5 alternating | Chord scorer on a voicing change, 8 strums total |
|
||||||
|
| F. E major strum | 6-string dense voicing, 8 strums |
|
||||||
|
| G. A major strum | 5-string voicing (skips low E), 8 strums |
|
||||||
|
| H. D major strum | 4-string voicing (skips low E + A), 8 strums |
|
||||||
|
|
||||||
|
No hammer/pull, no bends — those are next on the algorithm-tuning
|
||||||
|
list and aren't useful as benchmarks until that work lands.
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Share the diagnostic JSON (schema `note_detect.diagnostic.v1`).
|
||||||
|
Filter `benchmark_hint` to bucket v1 vs v2 runs.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Built by `docs/benchmarks/note_detect_v2/build_benchmark.py`.
|
||||||
@@ -0,0 +1,512 @@
|
|||||||
|
"""Builds the Note Detect Benchmark sloppak (v2).
|
||||||
|
|
||||||
|
A relaxed-pace test piece tuned for the player's strengths: half-note
|
||||||
|
spacing throughout, no hammer-on / pull-off section, no bend section,
|
||||||
|
no fast staccato. Adds explicit strumming sections (single chord
|
||||||
|
repeated at half-note cadence) so the chord scorer is exercised
|
||||||
|
across a sequence of strums on the same voicing — closer to how
|
||||||
|
chords actually appear in real songs than v1's single-stroke
|
||||||
|
voicings.
|
||||||
|
|
||||||
|
Goals vs v1:
|
||||||
|
- More breathing room between every event (half-notes, ~1.33 s at
|
||||||
|
90 BPM, instead of v1's quarter notes at ~0.667 s).
|
||||||
|
- More chord events overall, with strumming patterns.
|
||||||
|
- Drop the technique sections (HO/PO/bends) — the detector's
|
||||||
|
technique handling is the next algorithm focus, separate from
|
||||||
|
measuring "do basic single notes + chords score correctly?"
|
||||||
|
|
||||||
|
How to run inside the slopsmith container:
|
||||||
|
|
||||||
|
docker cp docs/benchmarks/note_detect_v2/build_benchmark.py \\
|
||||||
|
slopsmith-web-1:/tmp/build_benchmark_v2.py
|
||||||
|
docker exec slopsmith-web-1 python /tmp/build_benchmark_v2.py \\
|
||||||
|
/app/static/sloppak_cache/note_detect_benchmark_v2.sloppak
|
||||||
|
|
||||||
|
After regenerating, copy the zip output to the tracked path with the
|
||||||
|
`.sloppak` (not `.sloppak.zip`) suffix — same gotcha as v1:
|
||||||
|
|
||||||
|
cp static/sloppak_cache/note_detect_benchmark_v2.sloppak.zip \\
|
||||||
|
docs/benchmarks/note_detect_v2/note_detect_benchmark_v2.sloppak
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import shutil
|
||||||
|
import struct
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import wave
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
# ── Benchmark parameters ────────────────────────────────────────────────
|
||||||
|
BPM = 90.0
|
||||||
|
SECONDS_PER_BEAT = 60.0 / BPM
|
||||||
|
BEATS_PER_BAR = 4
|
||||||
|
BAR_S = BEATS_PER_BAR * SECONDS_PER_BEAT
|
||||||
|
INTRO_BARS = 2
|
||||||
|
OUTRO_BARS = 2
|
||||||
|
EXERCISE_BARS = 8 # v2 uses 8-bar sections (was 6 in v1) for extra breathing room.
|
||||||
|
|
||||||
|
# Standard E-tuning open MIDI per string, low → high.
|
||||||
|
OPEN_MIDI = [40, 45, 50, 55, 59, 64] # E2 A2 D3 G3 B3 E4
|
||||||
|
|
||||||
|
SR = 44100
|
||||||
|
|
||||||
|
|
||||||
|
# ── Click-track audio generator ────────────────────────────────────────
|
||||||
|
def _sine_burst(freq_hz, duration_s, amplitude):
|
||||||
|
n = int(SR * duration_s)
|
||||||
|
out = []
|
||||||
|
fade = max(1, int(0.004 * SR))
|
||||||
|
for i in range(n):
|
||||||
|
env = 1.0
|
||||||
|
if i < fade:
|
||||||
|
env = i / fade
|
||||||
|
elif i >= n - fade:
|
||||||
|
env = (n - 1 - i) / fade
|
||||||
|
s = math.sin(2 * math.pi * freq_hz * (i / SR)) * amplitude * env
|
||||||
|
out.append(max(-1.0, min(1.0, s)))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def write_click_wav(path: Path, duration_s: float):
|
||||||
|
"""Per-beat click track. Downbeats louder + higher pitch."""
|
||||||
|
total_samples = int(SR * duration_s)
|
||||||
|
pcm = [0] * total_samples
|
||||||
|
beat = 0
|
||||||
|
t = 0.0
|
||||||
|
while t < duration_s:
|
||||||
|
is_downbeat = (beat % BEATS_PER_BAR == 0)
|
||||||
|
freq = 1200 if is_downbeat else 800
|
||||||
|
amp = 0.6 if is_downbeat else 0.35
|
||||||
|
burst = _sine_burst(freq, 0.040, amp)
|
||||||
|
start = int(t * SR)
|
||||||
|
for i, s in enumerate(burst):
|
||||||
|
j = start + i
|
||||||
|
if 0 <= j < total_samples:
|
||||||
|
pcm[j] = int(max(-1.0, min(1.0, pcm[j] / 32767 + s)) * 32767)
|
||||||
|
t += SECONDS_PER_BEAT
|
||||||
|
beat += 1
|
||||||
|
|
||||||
|
pcm = [struct.pack('<h', v) for v in pcm]
|
||||||
|
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with wave.open(str(path), 'wb') as w:
|
||||||
|
w.setnchannels(1)
|
||||||
|
w.setsampwidth(2)
|
||||||
|
w.setframerate(SR)
|
||||||
|
w.writeframes(bytes(b''.join(pcm)))
|
||||||
|
|
||||||
|
|
||||||
|
# ── Chart helpers ─────────────────────────────────────────────────────
|
||||||
|
def note(t, s, f, sus=0.0, **flags):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
's': s,
|
||||||
|
'f': f,
|
||||||
|
'sus': round(sus, 3),
|
||||||
|
'sl': flags.get('sl', -1),
|
||||||
|
'slu': flags.get('slu', -1),
|
||||||
|
'bn': flags.get('bn', 0.0),
|
||||||
|
'ho': flags.get('ho', False),
|
||||||
|
'po': flags.get('po', False),
|
||||||
|
'hm': flags.get('hm', False),
|
||||||
|
'hp': flags.get('hp', False),
|
||||||
|
'pm': flags.get('pm', False),
|
||||||
|
'mt': flags.get('mt', False),
|
||||||
|
'vb': flags.get('vb', False),
|
||||||
|
'tr': flags.get('tr', False),
|
||||||
|
'ac': flags.get('ac', False),
|
||||||
|
'tp': flags.get('tp', False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord(t, id_, notes):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
'id': id_,
|
||||||
|
'hd': False,
|
||||||
|
'notes': notes,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord_note(s, f, sus=0.0, **flags):
|
||||||
|
n = note(0.0, s, f, sus, **flags)
|
||||||
|
n.pop('t')
|
||||||
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
# ── Exercises ─────────────────────────────────────────────────────────
|
||||||
|
# v2 single-note exercises: HALF-NOTE pacing (2 beats / 1.33 s between
|
||||||
|
# events). That's roughly half the density of v1's quarter-note pacing,
|
||||||
|
# giving the player time to mute, reset, and re-pluck cleanly.
|
||||||
|
#
|
||||||
|
# v2 chord exercises: each chord voicing is STRUMMED multiple times at
|
||||||
|
# the same half-note cadence. Two reasons:
|
||||||
|
# 1. Real songs strum chords; single-stroke voicings are an
|
||||||
|
# artificial test that doesn't exercise the chord scorer's
|
||||||
|
# consistency across repeated strikes.
|
||||||
|
# 2. Multiple strums per voicing give the user a forgiving runway —
|
||||||
|
# if they nail 3 of 4 strums of an E5 power chord, that's still
|
||||||
|
# mostly hits.
|
||||||
|
|
||||||
|
def exercise_open_strings_slow(t0):
|
||||||
|
"""Open strings, half-note pacing, low → high → low. Wide spacing
|
||||||
|
lets each string ring out before the next is plucked, so the
|
||||||
|
detector has clean steady-state pitch to lock onto."""
|
||||||
|
seq = [0, 1, 2, 3, 4, 5, 5, 4, 3, 2, 1, 0] # 12 strings, half-notes = 24 beats = 6 bars
|
||||||
|
notes_out = []
|
||||||
|
for i, s in enumerate(seq):
|
||||||
|
notes_out.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, 0,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
# Final note rings into the 2-bar tail of the section.
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 4, 3)
|
||||||
|
return notes_out, [], 'Open strings (slow walk)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_fretted_positions_slow(t0):
|
||||||
|
"""5th-fret on each string, half-note pacing, ascending. One
|
||||||
|
direction (no descent) so total runtime fits comfortably in 8 bars
|
||||||
|
with plenty of tail room."""
|
||||||
|
seq = [(s, 5) for s in range(6)] # 6 notes × 2 beats = 12 beats = 3 bars
|
||||||
|
notes_out = []
|
||||||
|
for i, (s, f) in enumerate(seq):
|
||||||
|
notes_out.append(note(t0 + i * 2 * SECONDS_PER_BEAT, s, f,
|
||||||
|
sus=SECONDS_PER_BEAT * 1.6))
|
||||||
|
notes_out[-1]['sus'] = round(SECONDS_PER_BEAT * 4, 3)
|
||||||
|
return notes_out, [], 'Fretted positions (slow walk, 5th fret)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_sustained(t0):
|
||||||
|
"""Three 4-second sustained notes (low E, D, high E — spread across
|
||||||
|
the range). 4 s ring + 1 s gap = 5 s per event × 3 events = 15 s,
|
||||||
|
comfortably inside an 8-bar (≈ 21.3 s) section."""
|
||||||
|
sus = 4.0
|
||||||
|
targets = [(0, 5), (2, 7), (5, 5)]
|
||||||
|
notes_out = []
|
||||||
|
for i, (s, f) in enumerate(targets):
|
||||||
|
notes_out.append(note(t0 + i * (sus + 1.0), s, f, sus=sus))
|
||||||
|
return notes_out, [], 'Sustained notes (3 holds, 4 s each)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_e5_strum(t0):
|
||||||
|
"""E5 power chord strummed at half-note cadence. 8 strums × 2
|
||||||
|
beats = 16 beats = 4 bars of strumming, plus 4 bars of tail."""
|
||||||
|
voicing = [(0, 0), (1, 2)] # low E open + A fret 2 = E5
|
||||||
|
strums = 8
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6 # ring through the next strum, not past it
|
||||||
|
template = {
|
||||||
|
'name': 'E5', 'displayName': 'E5', 'arp': False,
|
||||||
|
'fingers': [-1] * 6,
|
||||||
|
'frets': [0 if s == 0 else (2 if s == 1 else -1) for s in range(6)],
|
||||||
|
}
|
||||||
|
for i in range(strums):
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in voicing]
|
||||||
|
chords_out.append(chord(t0 + i * 2 * SECONDS_PER_BEAT, 0, chord_notes))
|
||||||
|
return [], (chords_out, [template]), 'E5 power chord — slow strum (8×)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_a5_e5_alternating(t0):
|
||||||
|
"""A5 / E5 alternating, half-note strums. 8 strums total (4 of
|
||||||
|
each), gives a "1 5 1 5" feel that's the simplest chord progression
|
||||||
|
a player can land — minimal hand movement between voicings."""
|
||||||
|
voicings = [
|
||||||
|
('A5', [(1, 0), (2, 2)]), # A open + D fret 2 = A5
|
||||||
|
('E5', [(0, 0), (1, 2)]), # E open + A fret 2 = E5
|
||||||
|
]
|
||||||
|
templates = []
|
||||||
|
for i, (name, sf) in enumerate(voicings):
|
||||||
|
frets = [-1] * 6
|
||||||
|
for (s, f) in sf:
|
||||||
|
frets[s] = f
|
||||||
|
templates.append({
|
||||||
|
'name': name, 'displayName': name, 'arp': False,
|
||||||
|
'fingers': [-1] * 6, 'frets': frets,
|
||||||
|
})
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
strums = 8
|
||||||
|
for i in range(strums):
|
||||||
|
idx = i % 2 # alternate A5 / E5
|
||||||
|
_, sf = voicings[idx]
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in sf]
|
||||||
|
chords_out.append(chord(t0 + i * 2 * SECONDS_PER_BEAT, idx, chord_notes))
|
||||||
|
return [], (chords_out, templates), 'A5 / E5 alternating strums (8×)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_e_open_strum(t0):
|
||||||
|
"""E major open chord, half-note strums. All 6 strings ringing —
|
||||||
|
the densest voicing in the benchmark, tests the chord scorer's
|
||||||
|
per-string differentiation on the full set."""
|
||||||
|
voicing = [(0, 0), (1, 2), (2, 2), (3, 1), (4, 0), (5, 0)]
|
||||||
|
strums = 8
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
template = {
|
||||||
|
'name': 'E', 'displayName': 'E', 'arp': False,
|
||||||
|
'fingers': [-1] * 6,
|
||||||
|
'frets': [0, 2, 2, 1, 0, 0],
|
||||||
|
}
|
||||||
|
for i in range(strums):
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in voicing]
|
||||||
|
chords_out.append(chord(t0 + i * 2 * SECONDS_PER_BEAT, 0, chord_notes))
|
||||||
|
return [], (chords_out, [template]), 'E major open chord — slow strum (8×)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_a_open_strum(t0):
|
||||||
|
"""A major open chord, half-note strums. 5 strings (skips low E).
|
||||||
|
Slightly easier than E for the player (less stretch) and tests
|
||||||
|
the scorer's behaviour on a missing-low-string voicing."""
|
||||||
|
voicing = [(1, 0), (2, 2), (3, 2), (4, 2), (5, 0)]
|
||||||
|
strums = 8
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
template = {
|
||||||
|
'name': 'A', 'displayName': 'A', 'arp': False,
|
||||||
|
'fingers': [-1] * 6,
|
||||||
|
'frets': [-1, 0, 2, 2, 2, 0],
|
||||||
|
}
|
||||||
|
for i in range(strums):
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in voicing]
|
||||||
|
chords_out.append(chord(t0 + i * 2 * SECONDS_PER_BEAT, 0, chord_notes))
|
||||||
|
return [], (chords_out, [template]), 'A major open chord — slow strum (8×)'
|
||||||
|
|
||||||
|
|
||||||
|
def exercise_d_open_strum(t0):
|
||||||
|
"""D major open chord, half-note strums. 4 strings (skips low E
|
||||||
|
and A). Tests the chord scorer on partial voicings — common in
|
||||||
|
real songs and an easy stretch for new players."""
|
||||||
|
voicing = [(2, 0), (3, 2), (4, 3), (5, 2)]
|
||||||
|
strums = 8
|
||||||
|
chords_out = []
|
||||||
|
sus = SECONDS_PER_BEAT * 1.6
|
||||||
|
template = {
|
||||||
|
'name': 'D', 'displayName': 'D', 'arp': False,
|
||||||
|
'fingers': [-1] * 6,
|
||||||
|
'frets': [-1, -1, 0, 2, 3, 2],
|
||||||
|
}
|
||||||
|
for i in range(strums):
|
||||||
|
chord_notes = [chord_note(s, f, sus=sus) for (s, f) in voicing]
|
||||||
|
chords_out.append(chord(t0 + i * 2 * SECONDS_PER_BEAT, 0, chord_notes))
|
||||||
|
return [], (chords_out, [template]), 'D major open chord — slow strum (8×)'
|
||||||
|
|
||||||
|
|
||||||
|
EXERCISES = [
|
||||||
|
('A. Open strings (slow)', exercise_open_strings_slow),
|
||||||
|
('B. 5th-fret (slow)', exercise_fretted_positions_slow),
|
||||||
|
('C. Sustained notes', exercise_sustained),
|
||||||
|
('D. E5 power chord strum', exercise_e5_strum),
|
||||||
|
('E. A5 / E5 alternating', exercise_a5_e5_alternating),
|
||||||
|
('F. E major strum', exercise_e_open_strum),
|
||||||
|
('G. A major strum', exercise_a_open_strum),
|
||||||
|
('H. D major strum', exercise_d_open_strum),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ── Driver ─────────────────────────────────────────────────────────────
|
||||||
|
def build(out_dir: Path):
|
||||||
|
notes_all = []
|
||||||
|
chords_all = []
|
||||||
|
templates_all = []
|
||||||
|
sections = []
|
||||||
|
beats = []
|
||||||
|
|
||||||
|
t = INTRO_BARS * BAR_S
|
||||||
|
for label, fn in EXERCISES:
|
||||||
|
sections.append({'name': label, 'number': len(sections) + 1, 'time': round(t, 3)})
|
||||||
|
result = fn(t)
|
||||||
|
ns, ch_or_tuple, _desc = result
|
||||||
|
notes_all.extend(ns)
|
||||||
|
if isinstance(ch_or_tuple, tuple):
|
||||||
|
cs, tmpls = ch_or_tuple
|
||||||
|
# Rebase section-local chord template ids onto the global
|
||||||
|
# `templates_all` list. Each exercise emits its chords
|
||||||
|
# with `tmpl_id` numbered from 0 within the exercise; if
|
||||||
|
# we naively appended both chords and templates without
|
||||||
|
# offsetting, later sections' chords would silently
|
||||||
|
# reference earlier sections' templates (e.g. an open
|
||||||
|
# chord pointing at a power-chord shape). Apply the
|
||||||
|
# offset to each chord's `id` field before extending the
|
||||||
|
# global lists.
|
||||||
|
offset = len(templates_all)
|
||||||
|
for c in cs:
|
||||||
|
c['id'] = c.get('id', 0) + offset
|
||||||
|
chords_all.extend(cs)
|
||||||
|
templates_all.extend(tmpls)
|
||||||
|
else:
|
||||||
|
chords_all.extend(ch_or_tuple)
|
||||||
|
t += EXERCISE_BARS * BAR_S
|
||||||
|
|
||||||
|
end_t = t + OUTRO_BARS * BAR_S
|
||||||
|
|
||||||
|
# Beats — measure markers on downbeats.
|
||||||
|
bar_count = 0
|
||||||
|
bt = 0.0
|
||||||
|
while bt < end_t:
|
||||||
|
is_downbeat = abs(bt % BAR_S) < 1e-3
|
||||||
|
if is_downbeat:
|
||||||
|
bar_count += 1
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': bar_count})
|
||||||
|
else:
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': -1})
|
||||||
|
bt += SECONDS_PER_BEAT
|
||||||
|
|
||||||
|
# Anchors — re-anchor on each section so the camera doesn't drift.
|
||||||
|
anchors = [{'time': 0.0, 'fret': 1, 'width': 12}]
|
||||||
|
for sec in sections:
|
||||||
|
anchors.append({'time': sec['time'], 'fret': 1, 'width': 12})
|
||||||
|
|
||||||
|
arrangement = {
|
||||||
|
'name': 'Lead',
|
||||||
|
'tuning': [0, 0, 0, 0, 0, 0],
|
||||||
|
'capo': 0,
|
||||||
|
'notes': sorted(notes_all, key=lambda n: n['t']),
|
||||||
|
'chords': sorted(chords_all, key=lambda c: c['t']),
|
||||||
|
'anchors': anchors,
|
||||||
|
'handshapes': [],
|
||||||
|
'templates': templates_all,
|
||||||
|
'beats': beats,
|
||||||
|
'sections': sections,
|
||||||
|
}
|
||||||
|
|
||||||
|
manifest = {
|
||||||
|
'title': 'Note Detect Benchmark v2',
|
||||||
|
'artist': 'Slopsmith',
|
||||||
|
'album': 'Note Detection Benchmark',
|
||||||
|
'year': 2026,
|
||||||
|
'duration': round(end_t, 3),
|
||||||
|
'arrangements': [
|
||||||
|
{
|
||||||
|
'id': 'lead',
|
||||||
|
'name': 'Lead',
|
||||||
|
'file': 'arrangements/lead.json',
|
||||||
|
'tuning': [0, 0, 0, 0, 0, 0],
|
||||||
|
'capo': 0,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
'stems': [
|
||||||
|
{'id': 'full', 'file': 'stems/full.ogg', 'default': True},
|
||||||
|
],
|
||||||
|
'benchmark': {
|
||||||
|
'id': 'slopsmith-note-detect-benchmark',
|
||||||
|
'version': 2,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── Write files ──
|
||||||
|
out_dir = Path(out_dir)
|
||||||
|
if out_dir.exists():
|
||||||
|
# Defensive — see v1 builder. Only rmtree something that looks
|
||||||
|
# like a sloppak so a typo on the CLI doesn't nuke an unrelated
|
||||||
|
# directory.
|
||||||
|
if not (out_dir.suffix == '.sloppak'
|
||||||
|
or (out_dir / 'manifest.yaml').exists()):
|
||||||
|
raise RuntimeError(
|
||||||
|
f"refusing to rmtree {out_dir!r}: does not look like a sloppak "
|
||||||
|
f"(no .sloppak suffix, no manifest.yaml)."
|
||||||
|
)
|
||||||
|
shutil.rmtree(out_dir)
|
||||||
|
out_dir.mkdir(parents=True)
|
||||||
|
(out_dir / 'arrangements').mkdir()
|
||||||
|
(out_dir / 'stems').mkdir()
|
||||||
|
|
||||||
|
(out_dir / 'manifest.yaml').write_text(
|
||||||
|
yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
(out_dir / 'arrangements' / 'lead.json').write_text(
|
||||||
|
json.dumps(arrangement, separators=(',', ':')),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
|
||||||
|
wav_path = out_dir / 'stems' / 'full.wav'
|
||||||
|
write_click_wav(wav_path, end_t)
|
||||||
|
ogg_path = out_dir / 'stems' / 'full.ogg'
|
||||||
|
subprocess.run(
|
||||||
|
['ffmpeg', '-y', '-loglevel', 'error',
|
||||||
|
'-i', str(wav_path),
|
||||||
|
'-c:a', 'libvorbis', '-q:a', '5',
|
||||||
|
str(ogg_path)],
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
wav_path.unlink()
|
||||||
|
|
||||||
|
(out_dir / 'BENCHMARK.md').write_text(_benchmark_readme(end_t), encoding='utf-8')
|
||||||
|
_build_zip(out_dir)
|
||||||
|
|
||||||
|
print(f'Built {out_dir}')
|
||||||
|
print(f' {out_dir}.zip')
|
||||||
|
print(f' Duration: {end_t:.1f} s')
|
||||||
|
print(f' Notes: {len(arrangement["notes"])}')
|
||||||
|
print(f' Chords: {len(arrangement["chords"])}')
|
||||||
|
print(f' Templates:{len(arrangement["templates"])}')
|
||||||
|
|
||||||
|
|
||||||
|
def _build_zip(src_dir: Path):
|
||||||
|
"""Pack with fixed dates / attrs for zip-metadata reproducibility.
|
||||||
|
See v1 builder docstring for full caveats (OGG framing has its own
|
||||||
|
non-determinism we don't try to fix here)."""
|
||||||
|
import zipfile
|
||||||
|
zip_path = src_dir.with_suffix(src_dir.suffix + '.zip')
|
||||||
|
if zip_path.exists():
|
||||||
|
zip_path.unlink()
|
||||||
|
with zipfile.ZipFile(zip_path, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
|
||||||
|
for p in sorted(src_dir.rglob('*')):
|
||||||
|
if p.is_file():
|
||||||
|
rel = p.relative_to(src_dir).as_posix()
|
||||||
|
info = zipfile.ZipInfo(filename=rel, date_time=(1980, 1, 1, 0, 0, 0))
|
||||||
|
info.compress_type = zipfile.ZIP_DEFLATED
|
||||||
|
info.external_attr = (0o644 & 0xFFFF) << 16
|
||||||
|
info.create_system = 3 # POSIX — see v1 builder for why
|
||||||
|
zf.writestr(info, p.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
def _benchmark_readme(duration_s):
|
||||||
|
return f"""# Slopsmith Note Detect Benchmark — v2
|
||||||
|
|
||||||
|
A slower-paced companion to v1, focused on what players can actually
|
||||||
|
land cleanly. Half-note spacing throughout (~1.33 s between events at
|
||||||
|
90 BPM), with multiple **strumming** sections — single chord voicings
|
||||||
|
repeated at half-note cadence — to exercise the chord scorer's
|
||||||
|
consistency across a sequence of strikes.
|
||||||
|
|
||||||
|
- **Tempo**: {BPM:g} BPM
|
||||||
|
- **Tuning**: E standard (no capo)
|
||||||
|
- **Audio**: metronome click track only — play *over* the click.
|
||||||
|
- **Duration**: {duration_s:.0f} s
|
||||||
|
|
||||||
|
## Sections
|
||||||
|
|
||||||
|
| Section | Tests |
|
||||||
|
|---|---|
|
||||||
|
| A. Open strings (slow walk) | Basic mono detection, low → high → low at half-note pacing |
|
||||||
|
| B. 5th-fret (slow walk) | Fretted-note detection, ascending half-notes |
|
||||||
|
| C. Sustained notes | Long-hold pitch detection, 4 s each |
|
||||||
|
| D. E5 power chord strum | Chord scorer on a 2-string voicing, 8 strums |
|
||||||
|
| E. A5 / E5 alternating | Chord scorer on a voicing change, 8 strums total |
|
||||||
|
| F. E major strum | 6-string dense voicing, 8 strums |
|
||||||
|
| G. A major strum | 5-string voicing (skips low E), 8 strums |
|
||||||
|
| H. D major strum | 4-string voicing (skips low E + A), 8 strums |
|
||||||
|
|
||||||
|
No hammer/pull, no bends — those are next on the algorithm-tuning
|
||||||
|
list and aren't useful as benchmarks until that work lands.
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Share the diagnostic JSON (schema `note_detect.diagnostic.v1`).
|
||||||
|
Filter `benchmark_hint` to bucket v1 vs v2 runs.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Built by `docs/benchmarks/note_detect_v2/build_benchmark.py`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
# ── CLI ───────────────────────────────────────────────────────────────
|
||||||
|
if __name__ == '__main__':
|
||||||
|
out = Path(sys.argv[1]) if len(sys.argv) > 1 else Path('./note_detect_benchmark_v2.sloppak')
|
||||||
|
build(out)
|
||||||
Binary file not shown.
@@ -0,0 +1,290 @@
|
|||||||
|
# Capability Domains
|
||||||
|
|
||||||
|
Capability domains are Slopsmith-wide coordination surfaces for core, bundled first-party plugins, external plugins, and future adapters. Plugins declare the runtime surfaces they use in `plugin.json`; core declares and owns host workflows directly in the runtime. These declarations let diagnostics and support tools reason about behavior without relying on private globals.
|
||||||
|
|
||||||
|
## Standards
|
||||||
|
|
||||||
|
Migrated plugins should declare standards explicitly:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Only declare `plugin-runtime-idempotent.v1` when repeated script hydration cannot duplicate wrappers, listeners, timers, DOM roots, diagnostics contributors, jobs, media nodes, or capability participants.
|
||||||
|
|
||||||
|
## UI Contributions
|
||||||
|
|
||||||
|
Legacy `nav`, `screen`, and `settings` fields still work through the existing plugin loader. PR1 keeps UI capability domains out of the runtime graph, so migrated plugins should not treat `ui.navigation`, `ui.plugin-screens`, or `settings` as active capability contracts yet. Their candidate manifest shape is reserved for a future UI-host PR:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ui": {
|
||||||
|
"ui.navigation": [{ "id": "my-plugin-nav", "region": "plugins", "label": "My Plugin" }],
|
||||||
|
"ui.plugin-screens": [{ "id": "my-plugin-screen", "region": "plugin-screens", "label": "My Plugin" }],
|
||||||
|
"settings": [{ "id": "my-plugin-settings", "region": "plugin-settings", "label": "My Plugin" }]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Core continues to load legacy UI fields normally. It does not emit PR1 compatibility shim entries for UI placement or visualization `type`; the PR that promotes those domains will own their shim accounting and tests.
|
||||||
|
|
||||||
|
## Runtime Domains
|
||||||
|
|
||||||
|
Declare non-UI runtime surfaces under `domains` or `runtime_domains`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"domains": {
|
||||||
|
"library": { "role": "provider" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
If a plugin still uses `routes`, the backend loader continues to load `routes.py` normally. PR1 does not expose that legacy surface as `backend.routes`; the backend route domain is deferred until a future PR has a concrete route/provider workflow and privilege review.
|
||||||
|
|
||||||
|
Plugins that call `context["register_library_provider"](...)` are attributed to the loading plugin id in `/api/library/providers` as `owner_plugin_id`. The browser library capability module at [static/capabilities/library.js](../static/capabilities/library.js) owns the `library` domain as a `provider-coordinator`: it refreshes `/api/library/providers`, registers the built-in `local` provider as `core.library.local`, and registers plugin-backed providers under their `owner_plugin_id` when one is known. Provider manifests should still declare the `library` capability so diagnostics and the bundled inspector can show intended relationships before the backend route code runs.
|
||||||
|
|
||||||
|
Route-only external plugins that participate in library workflows without registering a browsable provider should declare requester/observer intent instead of provider ownership when they adopt this contract in their own repositories. This PR documents the generic shape only: such plugins use `library` requester/observer `requests` and `observes` declarations and do not appear as providers, owners, or separate `backend.routes` domains.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"capabilities": {
|
||||||
|
"library": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["query-page", "query-artists", "query-stats", "tuning-names", "get-art", "sync-song"],
|
||||||
|
"description": "Adds a browsable library source and optional song sync.",
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "safe"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The frontend exposes the current source list through `window.slopsmith.capabilities.command('library', 'list-providers')`. Public owner commands (`list-providers`, `refresh-providers`, `get-current`, `select-provider`, `sync-song`, `inspect`) are distinct from provider operations (`query-page`, `query-artists`, `query-stats`, `tuning-names`, `get-art`, `sync-song`). The app-owned handler delegates to the existing provider registry and source selector, so plugins should not scrape the `#lib-provider` dropdown.
|
||||||
|
|
||||||
|
Capability declarations may include a short `description`. The bundled Capability Inspector shows that text on expanded domain owner cards; when it is omitted, the inspector falls back to a compact generated owner summary.
|
||||||
|
|
||||||
|
## Audio Graph/Session Domains
|
||||||
|
|
||||||
|
The audio graph/session slice promotes four player-audio domains into the runtime graph: `audio-mix`, `audio-input`, `audio-monitoring`, and `stems`. The browser module at [static/capabilities/audio-session.js](../static/capabilities/audio-session.js) owns the active session boundary, contributes diagnostics under `slopsmith.audio_session.diagnostics.v1`, and records compatibility bridge hits for legacy audio surfaces.
|
||||||
|
|
||||||
|
`audio-mix`, `audio-input`, and `audio-monitoring` are core-owned provider-coordinator domains. They expose bounded inspect/register/start/stop style commands, redaction-safe diagnostics, and bridge accounting for legacy fader, analyser, input, and monitoring handshakes.
|
||||||
|
|
||||||
|
For `audio-mix`, native fader providers register mix participants with stable `participantId`, `kind`, `sourceMode`, optional `logicalFaderKey`, and `fader` metadata. The public command surface is `inspect`, `list-faders`, `get-fader-value`, `set-fader-value`, `inspect-route`, `inspect-analyser`, `register-participant`, and `unregister-participant`; provider operations are `fader.get-value`, `fader.set-value`, `route.get-current`, and `analyser.get-summary`. Providers own persistence for plugin faders and must return committed values from set operations so the player mixer can display the value that actually applied.
|
||||||
|
|
||||||
|
Legacy `window.slopsmith.audio.registerFader(...)` remains supported as an audio-mix compatibility bridge. The bridge registers a compatibility-backed participant, wraps legacy `getValue`/`setValue` callbacks as provider operations, and preserves `window.slopsmith.audio.getFaders()` for external callers. If a native participant and a legacy fader share the same logical fader key, the native participant owns the visible control; the legacy participant is retained for diagnostics with `supersededBy` and an `overshadowed` bridge hit. Removal gates for the bridge are: native providers cover bundled mixer integrations, diagnostics show no unexpected legacy hits in normal playback, and repeated plugin hydration does not create duplicate faders.
|
||||||
|
|
||||||
|
Audio-mix diagnostics live under `slopsmith.audio_session.diagnostics.v1`. The `audio-mix` domain snapshot includes session state, participants, visible fader summaries, required participant-kind coverage, route summary, analyser summary, bridge hits, and bounded recent outcomes. Fader outcomes include operation name, participant id, fader id, status such as `committed`, `normalized`, `unavailable`, or `timeout`, and a bounded reason. Diagnostics must not include raw audio buffers, FFT arrays, device labels, stable hardware identifiers, secrets, or unredacted local paths; route/analyser payloads are summaries only.
|
||||||
|
|
||||||
|
For `audio-input`, native providers register source summaries with `sourceId`, `providerId`, `logicalSourceKey`, `kind`, redaction-safe label/pseudonym, `availability`, `channelSummary`, `sourceMode`, and provider operations. The public command surface is `inspect`, `list-sources`, `register-source`, `unregister-source`, `select-source`, `open-source`, and `close-source`; provider operations are `source.enumerate`, `source.describe`, `source.open`, and `source.close`. `inspect`, `list-sources`, and `select-source` are prompt-free and never open live input or call enumeration. `source.enumerate` runs only when explicitly requested by provider/user discovery. `open-source` is the permission boundary: it routes to `source.open`, attributes the requester, checks the selected source and requested channel shape, and records `handled`, `denied`, `degraded`, `failed`, `no-owner`, `no-handler`, `unsupported-command`, or `incompatible-version` outcomes.
|
||||||
|
|
||||||
|
Selected input is persisted by `logicalSourceKey` when browser storage is available. If storage is unavailable, the in-memory selection remains usable for the current session and diagnostics report the storage status. Start/stop/song switches preserve selected input independently of playback transport while clearing live open sessions. Compatible requesters share one open session per logical source and channel shape; requester references are released via `close-source`, and the provider receives `source.close` only after the last requester releases.
|
||||||
|
|
||||||
|
Compatibility-backed input sources should record `sourceMode: "compatibility"` plus `compatibilitySource` and, when applicable, an `audio-input.legacy-source` bridge hit. If a native provider and a compatibility-backed source share the same logical source key, the native source owns the visible source list and the compatibility source is retained in diagnostics with `supersededBy`. Removal gates for input bridges are: native providers cover bundled source discovery/open flows, diagnostics show no unexpected compatibility hits in normal playback, denied/unavailable/failure outcomes are distinguishable, repeated hydration does not create duplicate sources, and support snapshots contain no raw device labels, hardware ids, paths, secrets, live handles, buffers, samples, or waveform data.
|
||||||
|
|
||||||
|
For `audio-monitoring`, native providers register monitoring summaries with `providerId`, `logicalMonitoringKey`, redaction-safe label/pseudonym, `availability`, `sourceMode`, provider operations, `directMonitor`, and `latencySummary`. The public command surface is `inspect`, `list-providers`, `register-provider`, `unregister-provider`, `select-provider`, `start`, `stop`, and `set-direct-monitor`; provider operations are `monitoring.start`, `monitoring.stop`, `monitoring.status`, and `monitoring.set-direct-monitor`. `inspect`, `list-providers`, `select-provider`, and `monitoring.status` are prompt-free and must not open audio input or start monitoring.
|
||||||
|
|
||||||
|
Fresh monitoring start is a user-action boundary. A requester that calls `start` without `authorization: "user-action"` receives `user-action-required` unless it can attach to an already-active compatible monitoring session. Start dispatch opens the selected audio-input source through the `audio-input` domain, checks the requested channel shape, and then calls the provider's `monitoring.start` with a redaction-safe `sourceRef`, requester id, required channel shape, direct-monitor preference, and optional requester requirement. Outcomes distinguish `handled`, `degraded`, `denied`, `unavailable`, `failed`, `no-owner`, `no-handler`, `unsupported-command`, `incompatible`, `incompatible-version`, `provider-selection-required`, and `user-action-required`.
|
||||||
|
|
||||||
|
Monitoring sessions are keyed by provider, selected source, required channel shape, and direct-monitor policy. Compatible requesters share an active session without re-calling `monitoring.start`; each requester later calls `stop`, and the provider receives `monitoring.stop` only after the final requester releases it. Song switches and playback stops preserve active monitoring sessions for the current browser runtime, while page reload restores only the selected provider and direct-monitor preference; live monitoring stays stopped until a new explicit start.
|
||||||
|
|
||||||
|
Direct-monitor state is user-authoritative. `set-direct-monitor` updates the user's/default preference and applies provider control to active sessions only when the provider supports it. Requester `directMonitorRequirement` values are advisory constraints: when they conflict with the user's preference or provider support, the requester/session is marked degraded or unsupported, but the stored user/default preference is not changed.
|
||||||
|
|
||||||
|
Compatibility-backed monitoring providers should record `sourceMode: "compatibility"` plus `compatibilitySource` (which becomes the bridge id, defaulting to `audio-monitoring.legacy-provider` when unset) and, when applicable, the `audio-monitoring.audio-barrier` startup-barrier bridge hit. If a native provider and compatibility-backed provider share a logical monitoring key, the native provider owns the visible provider list and the compatibility provider is retained in diagnostics with `supersededBy`. Removal gates for monitoring bridges are: native providers cover bundled start/stop/status/direct-monitor flows, normal playback shows no unexpected legacy hits, background requesters cannot silently start live monitoring, repeated hydration does not duplicate providers or sessions, and support snapshots contain no raw device labels, hardware ids, paths, secrets, live handles, buffers, samples, waveform data, recordings, or provider-private payloads.
|
||||||
|
|
||||||
|
`stems` is different: `core.audio.session` is a coordinator, not the semantic owner of stem playback. The Stems plugin, or another active stem provider, remains the provider/owner of actual stem state, mute/restore mechanics, and per-song availability. The session coordinator records the active provider via `registerStemOwner(...)`, brokers claim/override/orphan diagnostics, and returns `no-owner` when no stem provider is available.
|
||||||
|
|
||||||
|
New bundled audio code should use the session host or native capability dispatch instead of adding new globals, private stem-state reads, direct analyser ownership, or plugin-specific handshakes. Existing legacy paths remain supported through named compatibility bridges until their migration notes and removal gates are satisfied.
|
||||||
|
|
||||||
|
## Audio Effects Domain
|
||||||
|
|
||||||
|
The audio-effects slice promotes `audio-effects` as a core-owned provider-coordinator domain implemented by [static/capabilities/audio-effects.js](../static/capabilities/audio-effects.js). The host owns provider selection, compatible executor selection, route state, fallback accounting, redaction-safe diagnostics, and the constrained chain-plan schema. Providers do not call executors. Provider code proposes plans and, for execution requests, returns a provider-private trusted asset map to the host; the host immediately hands that private request to a compatible executor such as trusted Desktop native audio or NAM Tone's browser/WASM executor.
|
||||||
|
|
||||||
|
The public command surface is `inspect`, `list-providers`, `list-executors`, `register-provider`, `unregister-provider`, `register-executor`, `unregister-executor`, `select-chain`, `resolve-plan`, `load-plan`, `inspect-route`, `list-mappings`, `upsert-mapping`, `delete-mapping`, `activate-mapping`, `clear-active-mapping`, `bypass`, `restore`, `fallback`, `activate-segment`, `set-stage-bypass`, `set-stage-parameter`, and `record-bridge-hit`. Provider operations are `chain.resolve`, `chain.inspect`, `mapping.list`, `mapping.upsert`, `mapping.delete`, `mapping.activate`, `mapping.clear-active`, `segment.activate`, `stage.set-bypass`, `stage.set-parameter`, `route.bypass`, and `route.restore`; executor operations are `loadChainPlan`, `activateSegment`, `setStageBypass`, and `setStageParameter`. Fresh chain selection and route bypass/restore require `authorization: "user-action"` or `authorization: "restore-selection"`; physical loading through `load-plan` requires `authorization: "user-action"`, `authorization: "restore-selection"`, or `authorization: "playback-session"`. Background requesters may inspect the current route and resolve an already selected compatible provider.
|
||||||
|
|
||||||
|
Core also owns the durable public mapping index at `/api/audio-effects/mappings`. A mapping answers "for this song/tone, this provider has an addressable effect plan"; it does not contain the provider's preset or chain data. Rows are keyed by `song_key + tone_key + provider_id`, carry an opaque `provider_ref`, and may be marked as the active mapping for that song/tone. Providers CRUD their own rows through the audio-effects host and resolve `provider_ref` inside their own storage when `chain.resolve` runs. This lets NAM Tone and Rig Builder coexist for the same song/tone while core owns arbitration and fallback order. `song_key` should be the playback domain's redaction-safe settings key when available; `filename` is optional legacy/debug context for migration and display.
|
||||||
|
|
||||||
|
Providers register stable `providerId`, `pluginId`, `routeKey`, priority, availability, source mode, operations, and operation handlers. Executors register stable `executorId`, `pluginId`, `routeKey`, priority, availability, source mode, supported provider ids, supported stage kinds, optional maximum stage count, operations, and handlers. The host chooses the highest-priority enabled provider for a route unless the caller requests a specific provider, then chooses the highest-priority compatible executor for that provider and resolved plan. Compatibility means both provider-compatible and plan-compatible: a browser/WASM NAM executor can advertise `providerIds: ["nam-tone"]`, `supportedKinds: ["nam", "ir"]`, and `maxStages: 2`, so it will not be asked to execute a Rig Builder VST/full-chain plan. If a selected provider has no compatible executor and the caller supplies a fallback provider, the host may fall back to that provider; if the caller explicitly requested the original provider, the host reports `unavailable` instead of silently changing providers. The initial default route is `desktop-main`, matching the desktop native executor path planned for full-chain NAM/IR/VST playback while still allowing browser executors for non-Desktop runtimes.
|
||||||
|
|
||||||
|
`chain.resolve` returns schema `slopsmith.audio_effects.chain_plan.v1`. A valid plan includes `planId`, `routeKey`, `providerId`, `stages`, optional `segments`, and optional redaction-safe summaries. Each stage exposes only stable opaque `stageId`, `kind` (`nam`, `ir`, `vst`, `utility`, or `bypass`), `role` (`pre-pedal`, `amp`, `cab`, `rack`, `master-pre`, etc.), opaque `assetRef`, optional opaque `stateRef`, bypass state, gain summary, and safe summary metadata. Raw file paths, URLs, model filenames, IR filenames, VST state blobs, native preset JSON, callbacks, handles, DOM nodes, audio buffers, samples, and waveform data are rejected or omitted.
|
||||||
|
|
||||||
|
Diagnostics live under `slopsmith.audio_effects.diagnostics.v1`. The snapshot includes provider summaries, executor summaries, route summaries, bridge hits, bounded recent outcomes, limits, and redaction notes. It intentionally omits full chain plans, stage asset references, provider-private mapping payloads, raw filenames, and song keys; diagnostics should explain which provider/executor/route failed without leaking local library structure or licensed asset names. Legacy NAM Tone/Rig Builder fetch interception, direct Desktop `loadPreset` calls, legacy tone controls, old `nam_tone.db` `tone_mappings` access, and MIDI/external effect handoffs are attributed through `audio-effects.legacy-nam-routing`, `audio-effects.legacy-native-load`, `audio-effects.legacy-tone-controls`, `audio-effects.legacy-tone-db`, and `audio-effects.legacy-midi-amp` bridge records while providers migrate.
|
||||||
|
|
||||||
|
## Playback Control Plane
|
||||||
|
|
||||||
|
The playback slice promotes `playback` as a core-owned command domain implemented by [static/capabilities/playback.js](../static/capabilities/playback.js). The public command surface is `inspect`, `start`, `pause`, `resume`, `stop`, `seek`, `set-loop`, `clear-loop`, `register-requester`, and `register-observer`. The domain emits `playback:*` lifecycle events for requests, loading, ready/start/pause/resume/seek/stop/end, route transitions, loop changes, failures/degraded states, superseded sessions, and compatibility bridge hits.
|
||||||
|
|
||||||
|
`static/app.js` remains the transport data plane. It registers a private playback adapter that can start songs, pause/resume/stop, seek, and manage loops, but the capability snapshot never exposes the `<audio>` element, JUCE player object, raw audio buffers, native route handles, samples, waveforms, recordings, local file paths, or URL payloads. Exported diagnostics use pseudonymous `target-*` ids for arrangement-scoped identity and hashed `settings-*` keys for per-song plugin settings; the local Capability Inspector may show visible title, artist, and arrangement labels for the active song.
|
||||||
|
|
||||||
|
Legacy playback surfaces remain supported during migration and are attributed through bridges such as `playback.window-play-song`, `playback.song-events`, `playback.window-slopsmith-transport`, `playback.loop-api`, and native route handoff records. Fresh audible `start` commands require `authorization: "user-action"`; background requesters may inspect or control an existing session, but user-priority pause/stop decisions block lower-priority automation until a user action resumes or starts a new session.
|
||||||
|
|
||||||
|
## Progression Domain
|
||||||
|
|
||||||
|
The progression slice (spec 010) promotes `progression` as a core-owned command domain implemented by [static/v3/progression-core.js](../static/v3/progression-core.js). Core owns the player's mastery rank (onboarding calibration + instrument-path levels), the challenge/quest engine, the Decibels wallet, and the cosmetics shop; all definitions are bundled content under `data/progression/` so new paths, levels, challenges, quests, and shop items are JSON edits, never code.
|
||||||
|
|
||||||
|
The public command surface is `inspect`, `record-event`, `list-shop`, `buy-item`, and `equip-item`. `record-event` accepts only whitelisted externally-postable event types (`minigame_run` in v1); `song_completed` is server-derived inside `/api/stats` so scored-session authority stays in one place and is denied at this surface. `buy-item` and `equip-item` require `authorization: "user-action"`. Backend plugins use the symmetric plugin-context hook `record_progression_event` (the minigames hub reports runs through it), which trusts backend code and skips the HTTP whitelist.
|
||||||
|
|
||||||
|
The domain emits `challenge-completed`, `quest-completed`, `path-level-up`, `rank-changed`, `db-changed`, `calibration-completed`, and `cosmetic-equipped` on the capability surface, mirrored as `progression:*` events on `window.slopsmith` for non-capability consumers. Diagnostics live under `slopsmith.progression.diag.v1` and contain content-load warnings, rank/path-level/quest counts, and wallet totals only — no song filenames or display names.
|
||||||
|
|
||||||
|
Decibels are earned exclusively by playing (songs, minigame runs, quest rewards); there is no real-money acquisition path and none may be added. The wallet tracks spend separately from the monotonic lifetime-earned total, so per-source XP resets and `db_earned` goals stay correct. A deferred release slice adds a `contributor` role so plugins can ship their own challenge/quest content (e.g. a drums plugin contributing drums challenges); content stays core-bundled until then.
|
||||||
|
|
||||||
|
## Visualization Domain
|
||||||
|
|
||||||
|
The visualization slice (cap:6) promotes `visualization` as a core-owned provider-coordinator implemented by [static/capabilities/visualization.js](../static/capabilities/visualization.js). Viz plugins are providers of the highway renderer surface; the core picker/auto-match machinery in `static/app.js` stays the selection workflow and attributes every renderer change into the domain.
|
||||||
|
|
||||||
|
The public command surface is `inspect`, `list-providers`, `select-renderer`, and `clear-renderer`. Selection delegates to the existing picker (`setViz`) so localStorage persistence, WebGL2 gating, and fallback semantics have exactly one implementation. The domain emits `providers-refreshed`, `renderer-changed` (with a `source` of `auto-match`, `user-select`, `fallback`, or `command:<requester>`), `renderer-ready`, and `renderer-failed`.
|
||||||
|
|
||||||
|
Provider discovery is still the legacy surface — `type: "visualization"` manifests populate the picker and `window.slopsmithViz_*` factory globals carry the renderer contract — and both are registered as compatibility shims (`visualization:type-visualization-manifest`, `visualization:window.slopsmithViz_*`) with hit accounting, so the Inspector shows exactly how much of the domain still rides the bridge. Plugins migrate by declaring a `visualization` provider capability in their manifests; the renderer factory contract (`init`/`draw`/`resize`/`destroy`, `contextType`, `matchesArrangement`) is unchanged.
|
||||||
|
|
||||||
|
**Per-instance provider settings (#849).** A provider may declare a `settings` array on its `visualization` capability — generic control descriptors (`{ key, label, type: "toggle" | "range" | "select", default, min/max/step, options }`) the capability-pipelines schema validates on *any* domain (the field lives on the shared `capabilityDeclaration`, not a visualization-only spot — see `docs/plugin-manifest.schema.json`). Descriptors flow through the **generic participant model**: the backend validates them for `/api/plugins` (`plugins/__init__.py`), `static/capabilities.js` normalizes + preserves them on the registered participant (so generic `inspect('visualization')` carries them), and the visualization owner reads them back from the participant by id — no app.js/picker side channel. They surface in the `list-providers` snapshot (each provider's `settings`, deep-frozen) plus a `provider_policy.hasSettings` flag in diagnostics, so a consuming host — splitscreen's per-panel control popover — can render the controls generically without per-plugin hardcoding. The visualization domain's **apply contract**: a provider that declares `settings` MUST implement `applySetting(key, value)` on its renderer instance (the host calls it on the specific per-panel instance, which is inherently per-panel — no canvas→panel resolution, no shared global keys); `getSetting(key)` is optional (the host falls back to the declared `default`). The host owns persistence. **This core slice lands the declarative surface + participant plumbing only** — no bundled provider declares `settings` yet (`highway_3d` still ships the legacy `factory.panelControls` static). The `highway_3d` migration and the splitscreen generic consumer are the remaining #849 follow-ups.
|
||||||
|
|
||||||
|
Diagnostics live under `slopsmith.visualization_capability.v1` and contain provider ids/labels/context types, the active renderer id and its selection source, the last auto-match outcome (resolved id + whether any predicate claimed the song), and the last failure (provider id + reason) — never song filenames, titles, or arrangement names. Per-panel selection (splitscreen #90) and per-panel provider settings (#849) are tracked follow-ups; the domain currently models the primary highway surface.
|
||||||
|
|
||||||
|
## Note-Detection Domain
|
||||||
|
|
||||||
|
The note-detection slice (spec 009, issues #727/#728) promotes `note-detection` as a core-owned provider-coordinator implemented by [static/capabilities/note-detection.js](../static/capabilities/note-detection.js). Doctrine per [specs/009-note-detection-domain/spec.md](../specs/009-note-detection-domain/spec.md): the domain exposes detection PRIMITIVES through requester-owned, context-scoped bindings — a monophonic pitch estimate and a polyphonic "is this note set ringing now?" verification — and consumers own all judgment semantics (hit windows, streaks, accuracy, tiers). Hit/miss/verdict results flow through the domain as observability events, never as domain-owned scoring.
|
||||||
|
|
||||||
|
The public command surface is `inspect`, `register-provider`, `unregister-provider`, `open-binding`, `close-binding`, `set-target`, and `clear-target`. Providers declare a kind — `midi` (a digital instrument producing exact verdicts, e.g. the keys highway's Web-MIDI input), `engine` (the desktop JUCE verifier), or `js` (the browser harmonic-comb / YIN fallback) — and the primitives they serve (`pitch.estimate`, `verify.target`). Each binding carries its requester's own redacted context summary (arrangement kind, string count, capo, MIDI range), independent of whatever song the host highway has loaded; concurrent bindings never perturb one another (spec 009 FR-003). With no provider registered, `open-binding` reports `unavailable` — consumers degrade, never block.
|
||||||
|
|
||||||
|
The legacy chart-coupled surface — `highway.setNoteStateProvider(fn)`, the single-global-detector path spec 009 retires — keeps working unchanged and is wrapped for compatibility-shim hit accounting (`note-detection:highway.setNoteStateProvider`). Migrating the chart `note_detect` consumer, Step Mode verify, minigames YIN scoring, and the `setVerifyTarget` bridge onto real bindings — and wiring per-binding tuning contexts into the engine verifier — is the remainder of the spec-009 slice and lands behind the Spec 003 migration gate.
|
||||||
|
|
||||||
|
Diagnostics live under `slopsmith.note_detection_capability.v1` and contain provider ids/labels/kinds, binding summaries (requester, provider, redacted context, target size), availability, and the last bounded outcome (event, binding, provider, MIDI number, hit flag) — never raw audio buffers, sample data, device labels, or song identity.
|
||||||
|
|
||||||
|
## Capability Roles
|
||||||
|
|
||||||
|
Use capability declarations for provider/requester/observer relationships:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"capabilities": {
|
||||||
|
"library": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["list-providers", "get-current", "inspect"],
|
||||||
|
"observes": ["providers-refreshed", "source-changed"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Future app-level workflows can then express intent through capability domains instead of hard-coding plugin-private implementation details.
|
||||||
|
|
||||||
|
Core registers manifest capability declarations from `/api/plugins` before plugin scripts hydrate. Runtime owners can then re-register the same participant with command handlers, event handlers, and current availability state. The merged participant view is visible through `window.slopsmith.capabilities.snapshotDiagnostics()` and `getDiagnostics()`.
|
||||||
|
|
||||||
|
Core domains include review metadata in diagnostics:
|
||||||
|
|
||||||
|
- `active`: wired to current Slopsmith behavior and expected to work as an integration point.
|
||||||
|
- `diagnostic`: support/inspection-only runtime surfaces.
|
||||||
|
|
||||||
|
PR1 includes only the delivered domains listed in [capability-roadmap.md](capability-roadmap.md): `pipeline`, `diagnostics`, and `library`. The follow-up audio graph/session slice promotes `audio-mix`, `audio-input`, `audio-monitoring`, and a coordinated `stems` surface. The playback slice promotes `playback` as an active transport control plane. The audio-effects slice promotes provider-selected effect-chain planning while leaving physical processor loading to compatible executors such as trusted Desktop native audio or a browser/WASM executor. The visualization slice promotes `visualization` as the highway renderer provider-coordinator, and the note-detection slice (spec 009) promotes `note-detection` as the detection-binding control plane. Backend routes, app UI, settings, and other hardware-facing domains remain documented in the roadmap and safety matrix until their own host workflow/provider slice exists.
|
||||||
|
|
||||||
|
Capability metadata is versioned by the `capability-pipelines.v1` standard. Invalid roles, commands, operations, requests, observes, emits, events, owner kinds, compatibility modes, ownership policies, safety classes, or version fields are excluded from the capability graph and surfaced through `capability_validation_warnings`; legacy plugin fields continue to load through their existing app paths. Plugins that declare a future `capability-pipelines` version are reported through `capability_unsupported_versions` and their runtime handlers are marked incompatible.
|
||||||
|
|
||||||
|
`diagnostics` and `pipeline` are adjacent support domains. `diagnostics` is the read-only snapshot/export surface: `snapshot` returns the redaction-safe state used by support bundles and the Capability Inspector. `pipeline` is the graph operations surface: `inspect`, `validate`, and `participant.set-enabled` operate on the capability graph itself and emit graph lifecycle events such as `resolved`, `runtime.validated`, and `participant.state-changed`.
|
||||||
|
|
||||||
|
Requesters should use the public claim/dispatch/release flow instead of mutating another plugin's globals:
|
||||||
|
|
||||||
|
```js
|
||||||
|
const api = window.slopsmith.capabilities;
|
||||||
|
const releaseClaim = api.claim({ capability: 'example.plugin-domain', claimId: 'example.automation-active', requester: 'example_requester' });
|
||||||
|
await api.dispatch({
|
||||||
|
capability: 'example.plugin-domain',
|
||||||
|
command: 'apply',
|
||||||
|
source: 'example_requester',
|
||||||
|
claim: { claimId: 'example.automation-active' },
|
||||||
|
args: { target: { kind: 'example-target' } },
|
||||||
|
});
|
||||||
|
releaseClaim();
|
||||||
|
```
|
||||||
|
|
||||||
|
The claim owner is inferred from the active owner participant for the capability, so requesters should identify themselves with `requester` or `source` instead of passing an `owner` field. `release` only needs the `claimId` and, when useful for disambiguation, the `capability`.
|
||||||
|
|
||||||
|
Manual user actions win over matching automation claims. When an owner records a user override for the same capability target, the registry reports the command as `overridden` and skips re-applying automation for that target. Owners keep restore snapshots for their own surfaces so requesters do not need to read private state.
|
||||||
|
|
||||||
|
When a requester disappears, the registry releases its active claims and clears restore snapshot references. When an owner or live handler disappears, matching claims become `orphaned` and non-dispatchable until the user or owning plugin resolves them. Runtime enable/disable state is lifecycle metadata, not a manual override.
|
||||||
|
|
||||||
|
## Owner Kinds And Dispatch Outcomes
|
||||||
|
|
||||||
|
Owner participants use a `kind` that describes how the domain is coordinated:
|
||||||
|
|
||||||
|
- `command`: one active owner handles public commands.
|
||||||
|
- `provider-coordinator`: one owner coordinates provider participants through provider operations.
|
||||||
|
- `event`: the owner primarily emits or coordinates events.
|
||||||
|
- `diagnostic`: read-only support and inspector surfaces.
|
||||||
|
- `privileged`: command execution needs an explicit enforcement plan before shipping.
|
||||||
|
|
||||||
|
Legacy `ownership` remains accepted in manifests for compatibility and diagnostics, but new domains should prefer `kind` plus participant roles. Ownership is derived for core owners where possible: `provider-coordinator` behaves like a multi-provider domain, diagnostics are diagnostic-only, privileged owners are privileged, and command/event owners are exclusive by default.
|
||||||
|
|
||||||
|
The compatibility ownership vocabulary remains:
|
||||||
|
|
||||||
|
- `exclusive-owner`: at most one active owner; duplicate owners produce a conflict and dispatch degrades.
|
||||||
|
- `multi-provider`: multiple providers may participate, but ordering must be deterministic through fixed priority or `before`/`after` constraints.
|
||||||
|
- `observer-only`: participants listen for events and should not handle commands.
|
||||||
|
- `requester-only`: participants request commands from another owner.
|
||||||
|
- `privileged`: command execution needs an explicit enforcement plan before shipping.
|
||||||
|
- `diagnostic-only`: read-only support and inspector surfaces.
|
||||||
|
|
||||||
|
Dispatch results use explicit outcomes: `handled`, `transformed`, `denied`, `failed`, `degraded`, `short-circuited`, `overridden`, `no-owner`, `no-handler`, `no-target`, `unsupported-command`, `incompatible`, `incompatible-version`, `unavailable`, `provider-selection-required`, `user-action-required`, `stale`, `cancelled`, and `stopped`. No-owner, no-handler, no-target, unsupported-command, incompatible, incompatible-version, provider-selection-required, user-action-required, stale, and cancelled decisions are recorded in diagnostics so support bundles explain why nothing happened.
|
||||||
|
|
||||||
|
## Deferred Core Adapters
|
||||||
|
|
||||||
|
UI placement and settings contributions are real Slopsmith surfaces, but they are not PR1 capability contracts (visualization is active as of the cap:6 slice; note-detection as of the spec-009 slice). Audio mixer/session domains are active as of the audio graph/session slice, playback is active as of the playback control-plane slice, and audio-effects is active as a provider/route/chain-plan coordinator; plugins should keep using current documented APIs for remaining areas until the corresponding domain PR ships the host workflow, command/event contract, compatibility shims, diagnostics fields, and tests.
|
||||||
|
|
||||||
|
The library provider workflow is the PR1 core adapter and is implemented natively as the `library` capability module. Provider refresh, selection, and sync run through `library` owner commands; backend provider registration remains the way providers enter the library registry, and the browser module turns that registry into provider participants. The app event bus continues to dispatch local `window.slopsmith` events for legacy listeners; playback now mirrors song transport, route, seek, and loop lifecycle into `playback`, and visualization attributes renderer selection/failure, while navigation, note, and route-only surfaces remain outside capability domains until their own slices land.
|
||||||
|
|
||||||
|
The direct `window.highway` object remains the renderer data plane. Per-frame reads such as notes, chords, beats, and renderer hooks should not be moved behind asynchronous capability commands until there is a dedicated chart/render facade.
|
||||||
|
|
||||||
|
## First-Party Management Plugins
|
||||||
|
|
||||||
|
Large management surfaces should prefer plugin-owned UI over crowding normal Settings. First-party management plugins can contribute screens and settings panels while core keeps shared services and diagnostics contracts centralized.
|
||||||
|
|
||||||
|
The bundled Capability Inspector plugin is the support surface for the current graph. It reads `window.slopsmith.capabilities.snapshotDiagnostics()`, filters by domain, and summarizes manifest participants, runtime participants, conflicts, unsupported versions, safety classes, expected legacy event surfaces, and compatibility shim hits without rendering raw runtime objects. Domains are grouped in review order: application/library, player/audio runtime, plugin-defined surfaces, then capability runtime. In the all-domains view, each domain starts collapsed with a domain-specific icon plus compact summary badges for participant-lane count, endpoint count, observed links, shimmed links, and status; badge labels live in tooltips/ARIA labels so the header stays scannable. Clicking the domain label expands or collapses the domain, opening the same graph view used by the single-domain filter. The graph places owner details and right-aligned command/event groups on the left, with short owner descriptions bottom-aligned as the final part of that pane. Participant usage is grouped the same way on the right, with observed or shimmed links between border-aligned endpoint ports. In multi-provider domains, links to provider participants use provider-family colors: purple for owner-to-provider command delegation and a lighter violet for provider events. Provider participants, including `library` sources, stay on the right lane and show a provider icon in their header. Headers show role-aware core/non-core origin badges such as Core owner, Core provider, or Non-core participant; owner headers place the origin badge directly after the owner icon, and the built-in local library provider is marked as core-origin. Observer and requester roles are implied by the command/event links rather than separate header badges. Participant cards are shown only when the plugin or runtime source has visible command or event usage for the current graph filter; domains with no such usage show zero participants, and attribution-only shims with no matching endpoint stay out of the lane. Command and event groups can collapse; when collapsed, all links for that side and group converge on the single group port. Hovering a participant, endpoint, or command/event group emphasizes the matching links and dims unrelated links; owner-side labels outside the current focus de-emphasize so the active source endpoints are easy to track. Expanded domain graphs progressively enhance to Cytoscape.js overlays that route bezier links between measured DOM endpoint ports, while keeping the HTML lanes as the fallback and readable data surface. Its Plugins-menu entry is hidden by default; enable **Capability Inspector → Show in Plugins menu** from Settings when reviewing or debugging capability behavior.
|
||||||
|
|
||||||
|
## Diagnostics Contract
|
||||||
|
|
||||||
|
Capability diagnostics use schema `slopsmith.capabilities.diagnostics.v1`. Snapshots are redaction-safe and capped at 64 KB by trimming older `recentDecisions` first while preserving current participants, active or orphaned claims, conflicts, domain review metadata, shim summaries, safety notes, and unsupported-version reports. Server diagnostics bundles include plugin manifest capability metadata, validation warnings, unsupported-version metadata, and compatibility shim summaries.
|
||||||
|
|
||||||
|
Compatibility shim entries include `shimId`, `source`, `capability`, `legacySurface`, `status`, `reason`, and optional hit fields. A shim with `hitCount > 0` means legacy behavior was observed, not merely declared. The `library` domain no longer uses compatibility shims for provider registration or source selection; provider attribution comes from `owner_plugin_id` and runtime provider participants. Future domains should add expected shim entries only in the PR that implements their actual legacy bridge.
|
||||||
|
|
||||||
|
## Expected Future Domains
|
||||||
|
|
||||||
|
Expected future domains live in [capability-roadmap.md](capability-roadmap.md) and [capability-safety-matrix.md](capability-safety-matrix.md) instead of the runtime graph. They are reserved names and candidate command shapes for future PRs, not current contracts. A future-domain PR should add the real host workflow, runtime registration, diagnostics redaction rules, tests, and compatibility shims in the same slice that makes the domain visible to plugins.
|
||||||
|
|
||||||
|
## Incremental Roadmap
|
||||||
|
|
||||||
|
Release slices should stay reviewable. The domain-level roadmap, PR1 domain set, deferred domains, shim policy, and future-domain PR checklist live in [capability-roadmap.md](capability-roadmap.md).
|
||||||
|
|
||||||
|
Future privileged domains must state user value, included and excluded commands, safety class, diagnostics fields, failure recovery, and tests proving disabled or incompatible participants cannot execute handlers before implementation begins.
|
||||||
|
|
||||||
|
## Rehydration Pattern
|
||||||
|
|
||||||
|
Plugins that wrap shared functions such as `window.playSong` or `window.showScreen` should store wrapper state on a stable `window.__slopsmith...Hooks` object. Re-running the script should replace the implementation object and return before installing another wrapper.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const hookState = window.__slopsmithMyPluginHooks || (window.__slopsmithMyPluginHooks = {});
|
||||||
|
hookState.impl = { afterPlaySong(filename) { /* current implementation */ } };
|
||||||
|
if (hookState.installed) return;
|
||||||
|
hookState.installed = true;
|
||||||
|
hookState.basePlaySong = window.playSong;
|
||||||
|
window.playSong = async function(filename, arrangement) {
|
||||||
|
await hookState.basePlaySong.call(this, filename, arrangement);
|
||||||
|
hookState.impl?.afterPlaySong?.(filename, arrangement);
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation Commands
|
||||||
|
|
||||||
|
From the `slopsmith/` directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node --check static/app.js
|
||||||
|
node --check static/capabilities.js
|
||||||
|
node --check static/diagnostics.js
|
||||||
|
node --check plugins/capability_inspector/screen.js
|
||||||
|
node --test tests/js/*.test.js
|
||||||
|
pytest tests/test_plugin_runtime_idempotence.py tests/test_plugins.py tests/test_diagnostics_bundle.py -q
|
||||||
|
```
|
||||||
@@ -0,0 +1,560 @@
|
|||||||
|
# Capability Authoring Recipes
|
||||||
|
|
||||||
|
Use these examples as small manifest fragments when migrating plugin-facing integrations to capability pipelines. The capability model is system-wide; these recipes focus on plugin manifests because core-owned domains are registered by Slopsmith itself. Each example is intentionally complete enough to pass the loader contract in [plugin-manifest.schema.json](plugin-manifest.schema.json).
|
||||||
|
|
||||||
|
> **Self-hosted CSS?** If your plugin uses Tailwind classes core doesn't ship (notably arbitrary values like `text-[11px]`), declare a `styles` key and bundle your own preflight-off stylesheet — see [plugin-styles.md](plugin-styles.md). That is separate from the capability-pipeline recipes below.
|
||||||
|
|
||||||
|
## Owner And Provider
|
||||||
|
|
||||||
|
A plugin that owns a domain and handles commands declares `owner` and `provider`. Use this for a single canonical implementation in a plugin-owned domain, such as a future stem-control capability.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "stems",
|
||||||
|
"name": "Stems",
|
||||||
|
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"stems": {
|
||||||
|
"roles": ["owner", "provider"],
|
||||||
|
"commands": ["mute", "restore", "inspect"],
|
||||||
|
"events": ["claim:created", "claim:released", "stems.ready"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"ownership": "exclusive-owner",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Requester And Observer
|
||||||
|
|
||||||
|
A plugin that requests work from another domain and listens for lifecycle events declares `requester` and `observer`.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "example_requester",
|
||||||
|
"name": "Example Requester",
|
||||||
|
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"example.plugin-domain": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"commands": ["apply", "restore", "inspect"],
|
||||||
|
"events": ["claim:created", "claim:released", "example.manual-override"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"ownership": "requester-only",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Observer Only
|
||||||
|
|
||||||
|
A plugin that only reads public events should declare `observer` and no command handlers. In PR1, this is most useful for participants that observe the delivered `library` workflow; future plugin-owned domains can use the same pattern once promoted.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "practice_hud",
|
||||||
|
"name": "Practice HUD",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"example.plugin-domain": {
|
||||||
|
"roles": ["observer"],
|
||||||
|
"commands": [],
|
||||||
|
"events": ["claim:created", "claim:released", "example.manual-override"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "degrade-noop",
|
||||||
|
"ownership": "observer-only",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Library Provider
|
||||||
|
|
||||||
|
A plugin that registers a remote client or generated library source declares itself as a `library` provider. The backend registration call is still made from `routes.py` with `context["register_library_provider"](...)`; the native browser library capability turns the provider registry into runtime provider participants. A thin server wrapper that only exposes the local library over HTTP should not declare `library` as a provider unless it also registers a provider in the library registry.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "remote_library_client",
|
||||||
|
"name": "Remote Library Client",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"library": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["query-page", "query-artists", "query-stats", "tuning-names", "get-art", "sync-song"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Library Requester And Observer
|
||||||
|
|
||||||
|
A route-only wrapper that uses the library capability without registering a browsable provider should declare requester/observer intent instead of provider ownership. This is a generic manifest shape for external plugins to adopt in their own repositories; it does not make the wrapper part of this PR's delivered domain set.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "library_route_wrapper",
|
||||||
|
"name": "Library Route Wrapper",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"library": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["list-providers", "get-current", "inspect"],
|
||||||
|
"observes": ["providers-refreshed", "source-changed"],
|
||||||
|
"description": "Uses the library source list through its own route surface without registering a provider.",
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Audio Mix Fader Provider
|
||||||
|
|
||||||
|
Existing plugins can keep using `window.slopsmith.audio.registerFader(spec)` while migrating. The compatibility bridge records the fader as an `audio-mix` participant. New bundled code should prefer a native participant declaration plus the audio-session helper once available in its integration point.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "delay_fx",
|
||||||
|
"name": "Delay FX",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"audio-mix": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["fader.get-value", "fader.set-value"],
|
||||||
|
"events": ["fader-value-changed", "fader-unavailable"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Native audio-mix fader providers should register a stable participant id and fader id, return the committed value from every set operation, and settle get/set operations within two seconds. The player mixer displays the committed value rather than the raw requested value. If the fader is temporarily unavailable, keep the participant registered with unavailable/disabled state so the mixer can render a disabled control and diagnostics can explain why it cannot be changed.
|
||||||
|
|
||||||
|
During migration, a plugin may still call `window.slopsmith.audio.registerFader(spec)`. Core maps that legacy fader into a compatibility-backed audio-mix participant and records bridge hits. If a native participant and a legacy fader represent the same logical source, the native participant owns the visible control and the legacy path is reported as compatibility-backed/overshadowed.
|
||||||
|
|
||||||
|
## Audio Effects Provider
|
||||||
|
|
||||||
|
Plugins that can provide guitar/bass processing chains should declare `audio-effects` as a provider and register at runtime with `window.slopsmith.audioEffects.registerProvider(...)`. The provider returns opaque chain plans; it must not expose local filenames, URLs, native preset JSON, VST state blobs, or raw handles through diagnostics or public route state.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "rig_builder",
|
||||||
|
"name": "Rig Builder",
|
||||||
|
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"audio-effects": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["chain.resolve", "chain.inspect", "segment.activate", "stage.set-bypass", "stage.set-parameter"],
|
||||||
|
"events": ["provider-registered", "route-selected", "plan-resolved", "changed", "fallback", "bridge-hit"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "shim-allowed",
|
||||||
|
"safety": "sensitive",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
```js
|
||||||
|
const effects = window.slopsmith && window.slopsmith.audioEffects;
|
||||||
|
effects.registerProvider({
|
||||||
|
providerId: 'rig-builder',
|
||||||
|
pluginId: 'rig_builder',
|
||||||
|
routeKey: 'desktop-main',
|
||||||
|
priority: 40,
|
||||||
|
operations: ['chain.resolve', 'segment.activate', 'stage.set-bypass', 'stage.set-parameter'],
|
||||||
|
operationHandlers: {
|
||||||
|
'chain.resolve': request => ({
|
||||||
|
outcome: 'handled',
|
||||||
|
plan: {
|
||||||
|
schema: 'slopsmith.audio_effects.chain_plan.v1',
|
||||||
|
planId: 'song-tone-plan',
|
||||||
|
routeKey: request.routeKey,
|
||||||
|
providerId: 'rig-builder',
|
||||||
|
stages: [
|
||||||
|
{ stageId: 'amp', kind: 'nam', role: 'amp', assetRef: 'rig-builder:asset:amp-main' },
|
||||||
|
{ stageId: 'cab', kind: 'ir', role: 'cab', assetRef: 'rig-builder:asset:cab-main' }
|
||||||
|
],
|
||||||
|
segments: [{ segmentId: 'base', stageIds: ['amp', 'cab'] }],
|
||||||
|
summary: { stageCount: 2 }
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
User-facing controls should dispatch through the domain instead of mutating another plugin's private state:
|
||||||
|
|
||||||
|
```js
|
||||||
|
await window.slopsmith.capabilities.dispatch({
|
||||||
|
capability: 'audio-effects',
|
||||||
|
command: 'select-chain',
|
||||||
|
source: 'rig_builder',
|
||||||
|
payload: { routeKey: 'desktop-main', providerId: 'rig-builder', authorization: 'user-action' }
|
||||||
|
});
|
||||||
|
|
||||||
|
const resolved = await window.slopsmith.capabilities.dispatch({
|
||||||
|
capability: 'audio-effects',
|
||||||
|
command: 'resolve-plan',
|
||||||
|
source: 'nam_tone',
|
||||||
|
payload: { routeKey: 'desktop-main', target: { settingsKey: 'settings-v1-...' } }
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Providers should store public song/tone routing through the host-owned mapping index and keep their own preset or chain rows private. The mapping's `provider_ref` is opaque to core: NAM Tone can use a preset id, Rig Builder can use a chain/preset id, and each provider resolves that reference in `chain.resolve`.
|
||||||
|
|
||||||
|
```js
|
||||||
|
await window.slopsmith.audioEffects.upsertMapping({
|
||||||
|
song_key: playbackTarget.settingsKey,
|
||||||
|
filename: playbackTarget.filename, // optional migration/debug context
|
||||||
|
tone_key: 'Dist',
|
||||||
|
provider_id: 'rig-builder',
|
||||||
|
provider_ref: 'chain:99',
|
||||||
|
label: 'Full Rig Builder chain',
|
||||||
|
source: 'manual',
|
||||||
|
active: true
|
||||||
|
});
|
||||||
|
|
||||||
|
const mappings = await window.slopsmith.audioEffects.listMappings({
|
||||||
|
song_key: playbackTarget.settingsKey,
|
||||||
|
tone_key: 'Dist'
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Only one mapping is active for a `song_key + tone_key` at a time, but multiple providers may have rows for the same song/tone. The active row decides which provider core asks first; provider fallback remains explicit through provider priority and `fallbackProviderId` during `loadPlan(...)`.
|
||||||
|
|
||||||
|
Browser or native executors should declare both provider scope and plan scope. A NAM-only browser executor should not claim Rig Builder plans just because it can load NAM files:
|
||||||
|
|
||||||
|
```js
|
||||||
|
window.slopsmith.audioEffects.registerExecutor({
|
||||||
|
executorId: 'nam-tone-browser-wasm',
|
||||||
|
pluginId: 'nam_tone',
|
||||||
|
routeKey: 'desktop-main',
|
||||||
|
providerIds: ['nam-tone'],
|
||||||
|
supportedKinds: ['nam', 'ir'],
|
||||||
|
maxStages: 2,
|
||||||
|
sourceMode: 'browser',
|
||||||
|
loadChainPlan: request => loadNamToneWasmPlan(request)
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Trusted Desktop can advertise broader support, while provider-specific browser executors should keep their `providerIds`, `supportedKinds`, and `maxStages` as narrow as the runtime actually supports.
|
||||||
|
|
||||||
|
The desktop executor is the trust boundary for physical loading. It should treat `assetRef` and `stateRef` values as opaque provider references, validate them through provider-owned lookup code, enforce local policy, then load or reject processor stages. Browser diagnostics should report route/provider/outcome summaries only.
|
||||||
|
|
||||||
|
## Audio Input And Monitoring Requester
|
||||||
|
|
||||||
|
Plugins that need live instrument input should declare requester/observer intent and let the host expose redaction-safe source identity. Diagnostics must not contain raw device labels, stable hardware ids, or audio buffers.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "note_detect",
|
||||||
|
"name": "Note Detect",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"audio-input": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["inspect", "list-sources", "select-source", "open-source", "close-source"],
|
||||||
|
"observes": ["source-registered", "source-selected", "source-opened", "source-open-degraded", "source-closed", "permission-denied"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "shim-allowed",
|
||||||
|
"ownership": "requester-only",
|
||||||
|
"safety": "sensitive",
|
||||||
|
"version": 1
|
||||||
|
},
|
||||||
|
"audio-monitoring": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["inspect", "list-providers", "select-provider", "start", "stop", "set-direct-monitor"],
|
||||||
|
"observes": ["provider-registered", "provider-selected", "provider-selection-required", "monitoring-started", "monitoring-degraded", "monitoring-unavailable", "monitoring-failed", "monitoring-denied", "monitoring-stopped", "direct-monitor-changed"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "shim-allowed",
|
||||||
|
"ownership": "requester-only",
|
||||||
|
"safety": "sensitive",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Requesters should list or inspect sources before opening them. `inspect`, `list-sources`, and `select-source` are prompt-free and must not call provider enumeration or open live input. When a requester needs audio, it dispatches `open-source` with a purpose and required channel shape. The requester identity is taken from the dispatch `source` (the authenticated caller) — a payload-supplied `requesterId` is ignored, so a requester cannot spoof another's identity or release a shared session it does not own. Compatible requesters share one open session; each requester later dispatches `close-source`, and the provider is closed only after the last requester releases it.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const api = window.slopsmith.capabilities;
|
||||||
|
await api.dispatch({ capability: 'audio-input', command: 'select-source', source: 'user', payload: { logicalSourceKey: 'browser:instrument:primary' } });
|
||||||
|
const opened = await api.dispatch({
|
||||||
|
capability: 'audio-input',
|
||||||
|
command: 'open-source',
|
||||||
|
source: 'note_detect', // identity for the open session; payload requesterId is ignored
|
||||||
|
payload: { purpose: 'note-detection', requiredChannelShape: 'mono' },
|
||||||
|
});
|
||||||
|
// Keep provider-owned streams/nodes private. Diagnostics receive only opened.payload summaries.
|
||||||
|
await api.dispatch({ capability: 'audio-input', command: 'close-source', source: 'note_detect', payload: { openSessionId: opened.payload.openSessionId } });
|
||||||
|
```
|
||||||
|
|
||||||
|
Monitoring is a separate lifecycle layered on top of input readiness. A fresh live monitoring start must come from an explicit user action; background requesters can attach only when an already-active compatible session exists.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const monitoring = await api.dispatch({
|
||||||
|
capability: 'audio-monitoring',
|
||||||
|
command: 'start',
|
||||||
|
source: 'note_detect',
|
||||||
|
payload: {
|
||||||
|
authorization: 'user-action',
|
||||||
|
requiredChannelShape: 'mono',
|
||||||
|
directMonitorRequirement: 'muted'
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
if (monitoring.outcome === 'user-action-required') {
|
||||||
|
// Show your own UI affordance; do not trigger a device prompt in the background.
|
||||||
|
}
|
||||||
|
|
||||||
|
await api.dispatch({
|
||||||
|
capability: 'audio-monitoring',
|
||||||
|
command: 'stop',
|
||||||
|
source: 'note_detect',
|
||||||
|
payload: { monitoringId: monitoring.payload && monitoring.payload.monitoringId },
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Audio Input Provider
|
||||||
|
|
||||||
|
Native input providers register redaction-safe source summaries with stable logical keys. Use `source.enumerate` only for an explicit user/provider discovery action; normal list/inspect/select flows should use already-registered summaries.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "desktop_audio",
|
||||||
|
"name": "Desktop Audio",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"audio-input": {
|
||||||
|
"roles": ["provider", "observer"],
|
||||||
|
"operations": ["source.enumerate", "source.open", "source.close"],
|
||||||
|
"events": ["source-registered", "source-opened", "source-closed", "source-open-degraded", "permission-denied"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "sensitive",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Provider source records should include `sourceId`, `providerId`, `logicalSourceKey`, `kind`, safe label or diagnostics pseudonym, availability, `channelSummary`, and supported operations/handlers. Do not put browser `MediaStream`, `AudioNode`, native handles, buffers, samples, waveform data, raw device labels, stable hardware ids, paths, or secrets in returned payloads; keep those in provider-private state.
|
||||||
|
|
||||||
|
## Audio Monitoring Provider
|
||||||
|
|
||||||
|
Native monitoring providers register a stable `logicalMonitoringKey` and keep actual audio streams, native handles, and device labels provider-private. The core host coordinates selected provider, requester sharing, direct-monitor policy, and diagnostics, but the provider owns the actual live monitor graph.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "desktop_audio",
|
||||||
|
"name": "Desktop Audio",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"audio-monitoring": {
|
||||||
|
"roles": ["provider", "observer"],
|
||||||
|
"operations": ["monitoring.start", "monitoring.stop", "monitoring.status", "monitoring.set-direct-monitor"],
|
||||||
|
"events": ["provider-registered", "monitoring-started", "monitoring-stopped", "direct-monitor-changed"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "sensitive",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Provider records should include `providerId`, `logicalMonitoringKey`, safe label or diagnostics pseudonym, `availability`, `sourceMode`, supported operations, `directMonitor` summary, and `latencySummary`. `monitoring.start` receives a redaction-safe `sourceRef`, `requesterId`, `requiredChannelShape`, `directMonitorPreference`, and optional `directMonitorRequirement`; it should return only status summaries such as active/degraded/denied/unavailable/failed. `monitoring.status` must be prompt-free and must not open audio input. `monitoring.set-direct-monitor` may apply the user's preference for active sessions; requester requirements must never mutate the user's stored preference.
|
||||||
|
|
||||||
|
## Stems Provider Behind Audio Session Coordination
|
||||||
|
|
||||||
|
The Stems plugin remains the provider/owner of actual stem playback state. `core.audio.session` coordinates dispatch, claims, overrides, orphan detection, and diagnostics, but it does not replace the provider.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "stems",
|
||||||
|
"name": "Stems",
|
||||||
|
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"stems": {
|
||||||
|
"roles": ["owner", "provider"],
|
||||||
|
"commands": ["mute", "restore", "inspect"],
|
||||||
|
"operations": ["stem.get-state", "stem.apply-automation", "stem.restore-automation"],
|
||||||
|
"events": ["owner-available", "automation-applied", "automation-restored", "automation-overridden", "claim-orphaned"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "shim-allowed",
|
||||||
|
"ownership": "exclusive-owner",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Playback Requester And Observer
|
||||||
|
|
||||||
|
Plugins that need to inspect or coordinate song transport should declare `playback` requester/observer intent and use the capability dispatch surface instead of wrapping `window.playSong` or scraping the `<audio>` element. Raw media handles stay private to core; diagnostics expose only pseudonymous targets, sanitized timing, route, loop, requester, observer, and recent outcome summaries.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "practice_hud",
|
||||||
|
"name": "Practice HUD",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"playback": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["inspect", "pause", "resume", "seek", "set-loop", "clear-loop"],
|
||||||
|
"observes": ["ready", "started", "paused", "resumed", "seeking", "seeked", "stopped", "loop-set", "loop-cleared"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "shim-allowed",
|
||||||
|
"ownership": "requester-only",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Fresh audible starts require a user action. Background plugins should call `inspect` first and attach to an existing compatible session; if a plugin needs to offer a play/start action, wire it to a visible user gesture and pass `authorization: "user-action"`.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const api = window.slopsmith.capabilities;
|
||||||
|
|
||||||
|
const state = await api.dispatch({
|
||||||
|
capability: 'playback',
|
||||||
|
command: 'inspect',
|
||||||
|
source: 'practice_hud',
|
||||||
|
args: {},
|
||||||
|
});
|
||||||
|
|
||||||
|
if (state.status !== 'idle') {
|
||||||
|
await api.dispatch({
|
||||||
|
capability: 'playback',
|
||||||
|
command: 'seek',
|
||||||
|
source: 'practice_hud',
|
||||||
|
args: { time: 42.0, reason: 'practice segment jump' },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
During migration, legacy uses of `window.playSong`, `song:*` events, `window.slopsmith.seek`, and loop helpers remain available and are recorded as playback bridge hits. Treat bridge hits as migration telemetry: native capability requests should eventually cover normal plugin workflows so unexpected legacy hits disappear from diagnostics.
|
||||||
|
|
||||||
|
## Progression Requester And Observer
|
||||||
|
|
||||||
|
Plugins that report gameplay outcomes or react to player progression (spec 010) should declare `progression` requester/observer intent and use capability dispatch instead of private fetches. Externally postable event types are whitelisted (`minigame_run` in v1); `song_completed` is server-derived inside `/api/stats` and is denied at this surface. Backend plugin code can use the plugin-context hook `record_progression_event` instead (the minigames hub does).
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "my_minigame",
|
||||||
|
"name": "My Minigame",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"progression": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["inspect", "record-event"],
|
||||||
|
"observes": ["challenge-completed", "quest-completed", "path-level-up", "rank-changed", "db-changed"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"ownership": "requester-only",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`buy-item` and `equip-item` require a visible user gesture (`authorization: "user-action"`). Decibels are play-earned only; plugins must not present any purchase path.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const api = window.slopsmith.capabilities;
|
||||||
|
|
||||||
|
const result = await api.dispatch({
|
||||||
|
capability: 'progression',
|
||||||
|
command: 'record-event',
|
||||||
|
source: 'my_minigame',
|
||||||
|
payload: { type: 'minigame_run', payload: { game_id: 'my-minigame', score: 420 } },
|
||||||
|
});
|
||||||
|
// result.payload lists challenges/quests completed by this event (toast UX).
|
||||||
|
|
||||||
|
window.slopsmith.on('progression:quest-completed', (e) => {
|
||||||
|
console.log('quest done:', e.detail.title, '+' + e.detail.reward_db + ' dB');
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Future Expansion Domains
|
||||||
|
|
||||||
|
Some domain names are reserved for expected future contracts, but they are not registered in the runtime graph yet. For example, `ui.player-panels` is documented as a likely panel-host surface, but Slopsmith does not currently expose a capability command for panel contributions. See [capability-roadmap.md](capability-roadmap.md) for the PR1 domain set and deferred-domain checklist.
|
||||||
|
|
||||||
|
Plugins should not declare future expansion domains until the corresponding host workflow ships. For current integrations, prefer active domains such as `library`, `playback`, `audio-mix`, `audio-input`, `audio-monitoring`, or `stems` intent matching the recipes above.
|
||||||
|
|
||||||
|
Invalid capability metadata is excluded from the capability graph, but legacy manifest fields still load through their existing app paths. The `library` workflow is native in PR1 and does not use compatibility shim metadata. Unsupported `capability-pipelines` versions are reported as incompatible and their runtime handlers must not execute.
|
||||||
|
|
||||||
|
## Library Card Action (`ui.library-card-injection`)
|
||||||
|
|
||||||
|
Delivered in fee[dB]ack v0.3.0 (frontend host). Plugins add per-song actions to
|
||||||
|
the library cards by REGISTERING them instead of DOM-injecting onto
|
||||||
|
`.song-card`. The library renders applicable actions in each card's action
|
||||||
|
menu, dispatches the handler on click, and emits `action-result` events;
|
||||||
|
the owner is visible in the Capability Inspector.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "my_card_action",
|
||||||
|
"name": "My Card Action",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"ui.library-card-injection": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["action.run"],
|
||||||
|
"events": ["action-registered", "action-result"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "safe",
|
||||||
|
"version": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Register the action from the plugin's `screen.js`:
|
||||||
|
|
||||||
|
```js
|
||||||
|
window.slopsmith.libraryCardActions.register({
|
||||||
|
id: 'my_card_action.run',
|
||||||
|
pluginId: 'my_card_action',
|
||||||
|
label: 'Do the thing',
|
||||||
|
placement: 'menu', // 'menu' | 'inline' | 'overlay'
|
||||||
|
order: 50,
|
||||||
|
applies: (song) => song.format === 'sloppak', // shown only when relevant
|
||||||
|
enabled: (song) => true,
|
||||||
|
run: async (song, ctx) => { // ctx.source identifies the surface
|
||||||
|
await fetch('/api/plugins/my_card_action/run', {
|
||||||
|
method: 'POST', headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ filename: song.filename }),
|
||||||
|
});
|
||||||
|
},
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
`register(spec)` returns an `unregister()` fn. The host owns rendering,
|
||||||
|
applicability, enabled state, and `action-result` events — plugins do not touch
|
||||||
|
library DOM. Legacy `.song-card` DOM injection still works in the 0.2.x UI;
|
||||||
|
migrate to this for the v0.3.0 native library.
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
# Capability Slice Review Preflight
|
||||||
|
|
||||||
|
Use this preflight before opening the next capability-slice PR. It captures the review patterns found while landing the audio/input/monitoring/playback slices, so a future slice should only need to analyze this document plus the immediately previous PR for newly discovered review themes.
|
||||||
|
|
||||||
|
## How To Use This
|
||||||
|
|
||||||
|
1. Read this checklist before creating the PR.
|
||||||
|
2. Scan the new slice for each pattern below.
|
||||||
|
3. Add focused regression tests for every pattern that applies to the new domain.
|
||||||
|
4. Then inspect only the last merged capability PR for new reviewer feedback that is not already covered here.
|
||||||
|
|
||||||
|
## Identity And Authority
|
||||||
|
|
||||||
|
- Command attribution must come from the capability dispatch caller (`requester` / `source`), not from payload fields such as `requesterId`.
|
||||||
|
- Payload identity is allowed only on explicit registration commands, such as `register-requester`, `register-observer`, or provider/source registration surfaces.
|
||||||
|
- Docs and examples must not tell callers to pass ignored identity fields in command payloads.
|
||||||
|
- Shared-session release paths must verify the releasing requester owns that attachment. A spoofed payload must not release another requester.
|
||||||
|
- User-action boundaries must be explicit. Fresh audible or live-input starts from background code should return `user-action-required`; background requesters may attach only to already-active compatible sessions when the contract allows it.
|
||||||
|
|
||||||
|
## Redaction And Diagnostics
|
||||||
|
|
||||||
|
- Treat all local storage, provider replies, adapter replies, bridge payloads, event details, command payloads, and dispatch caller strings as untrusted.
|
||||||
|
- Exported diagnostics must not contain raw filenames, titles, artists, paths, URLs, secrets, API keys, tokens, device labels, hardware ids, media/native handles, buffers, samples, waveforms, recordings, route-private objects, or provider-private objects.
|
||||||
|
- Redact before normalizing ids. Charset-only sanitizers can preserve path or token fragments such as `Users-me-plugin-token-abc`.
|
||||||
|
- Filter raw keys after normalizing camelCase to snake_case, so names like `accessToken`, `apiKey`, `nativeHandleRef`, `rawDeviceId`, and `mediaStream` are caught.
|
||||||
|
- Allow safe display fields only through an explicit allowlist. Do not let a general `label` exemption reintroduce raw device labels.
|
||||||
|
- Stored ids or persisted selections must be accepted only if they are already redaction-safe; otherwise ignore and clear them when possible.
|
||||||
|
- Bridge entries are exported verbatim enough to deserve the same redaction/bounding as command outcomes.
|
||||||
|
|
||||||
|
## Outcomes And Provider Results
|
||||||
|
|
||||||
|
- Propagate explicit provider/adapter outcomes when they are part of the domain contract. Do not collapse `no-handler`, `no-owner`, `unsupported-command`, `incompatible-version`, `overridden`, or similar actionable statuses into generic `degraded` or empty success.
|
||||||
|
- A provider list/enumerate command with providers but no matching handler should return `no-handler`, not `handled` with an empty list.
|
||||||
|
- Malformed provider or adapter results should be `failed` or another explicit contract outcome. Void/missing fields must not be treated as successful active/handled state.
|
||||||
|
- Command return values, recent outcomes, lifecycle events, and inspector display should agree on the same outcome/status names.
|
||||||
|
- If a command records an outcome on an early return, diagnostics must be refreshed immediately. In current hosts this usually means routing through the central outcome helper that calls the diagnostic touch/contribution path.
|
||||||
|
|
||||||
|
## Identifier Semantics
|
||||||
|
|
||||||
|
- Document whether each id is a public round-trip handle, an internal generated id, or a per-snapshot pseudonym.
|
||||||
|
- Do not compare raw caller ids against pseudonyms from diagnostics snapshots.
|
||||||
|
- If duplicate providers can share a logical key, disambiguate with all fields that define the selected winner, such as provider id, source mode, route kind, or channel shape.
|
||||||
|
- Generated internal ids that need intra-snapshot correlation may remain stable within that snapshot. Untrusted caller-supplied ids should be bounded, redacted, hashed, or pseudonymized before export.
|
||||||
|
- Use one pseudonymizer for a batch when returning multiple related records, so distinct raw ids do not all become the same `source-01`/`route-01` style value.
|
||||||
|
|
||||||
|
## Lifecycle And Teardown
|
||||||
|
|
||||||
|
- Session replacement must close or finalize provider-owned live resources before discarding the old session state.
|
||||||
|
- Stop paths and route/session switches should emit the same redaction-safe summary shape as normal close/ended paths.
|
||||||
|
- Prompt-free commands such as inspect, list, and status must not trigger provider enumeration, permission prompts, device opening, or route activation.
|
||||||
|
- Optional fields need precise fallback semantics. If an optional disambiguator is omitted, fallback only when the match is unambiguous; if the caller supplied an explicit but wrong disambiguator, fail instead of touching a different session.
|
||||||
|
|
||||||
|
## Schema And Docs
|
||||||
|
|
||||||
|
- Every emitted event name should have one payload shape. Avoid ad-hoc degraded/denied payloads that omit fields present in the success shape.
|
||||||
|
- Contract request shapes must match fields read by the runtime. Remove fields the runtime ignores, and document separate fields such as `storageStatus` instead of inventing extra enum values.
|
||||||
|
- Data-model docs must match actual implementation semantics, especially global-vs-provider-scoped ids, `supersededBy` meanings, restore statuses, and outcome enums.
|
||||||
|
- Recipes should use realistic dispatch caller values: `source: 'user'` for user preference changes, plugin ids for plugin work, and no payload identity when attribution comes from dispatch.
|
||||||
|
|
||||||
|
## Tests To Add Per Slice
|
||||||
|
|
||||||
|
- Anti-spoofing: payload `requesterId` must not override the dispatch caller for control commands.
|
||||||
|
- Redaction: unsafe caller ids, persisted values, bridge payloads, provider results, adapter results, nested payloads, and camelCase raw keys must not appear in exported diagnostics or events.
|
||||||
|
- Outcome preservation: provider/adapter `denied`, `failed`, `degraded`, `no-owner`, `no-handler`, `unsupported-command`, `incompatible-version`, and domain-specific outcomes should round-trip exactly when supported.
|
||||||
|
- Diagnostics freshness: early denied/no-owner/no-handler/failed returns must update the exported diagnostics contribution immediately.
|
||||||
|
- Schema consistency: denied/degraded/unavailable events should have the same summary shape as handled events for the same command family.
|
||||||
|
- Duplicate identity: native-vs-compatibility or multi-provider duplicates should resolve to the selected/canonical winner and reject non-selected hints.
|
||||||
|
- Teardown: stop/session-switch paths should close live provider resources and preserve diagnosable final state.
|
||||||
|
- Documentation grep: scan for stale `requesterId` payload examples, removed enum values, unsupported request fields, and bridge ids that do not exist at runtime.
|
||||||
|
|
||||||
|
## Validation Baseline
|
||||||
|
|
||||||
|
For each slice, run the focused suite for the domain plus the cross-domain regression suite it touches. Also run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git diff --check origin/main..HEAD
|
||||||
|
node --check static/capabilities.js
|
||||||
|
python3 -m py_compile tests/test_plugin_runtime_idempotence.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Add domain-specific `node --check` and `node --test` commands to the slice quickstart, including the core capability host and any inspector or adapter files changed by the slice.
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
# Capability Roadmap
|
||||||
|
|
||||||
|
This roadmap keeps the first capability PR reviewable while making the future domain plan explicit. PR1 ships the substrate and only the domains whose current behavior is implemented, diagnosed, and tested. Future domains stay planned or reserved until the PR that implements their host workflow also adds runtime registration, compatibility shims when needed, diagnostics, and tests.
|
||||||
|
|
||||||
|
[plugin-capability-inventory.md](plugin-capability-inventory.md) is the current plugin evidence pass for this roadmap. It inventories 41 included plugins, confirms that no included manifest currently declares `capabilities`, and maps legacy behavior to recommended future domains. Roadmap entries below should be read together with that inventory: domains listed here are planned names and migration targets, while the inventory explains which real plugins are likely to use or declare them.
|
||||||
|
|
||||||
|
## PR1 Domain Set
|
||||||
|
|
||||||
|
PR1 should include these delivered domains:
|
||||||
|
|
||||||
|
| Domain | Scope | Owner Kind | Safety | Why It Is In PR1 |
|
||||||
|
|--------|-------|------------|--------|------------------|
|
||||||
|
| `pipeline` | Core diagnostic surface | diagnostic | diagnostic-only | Exposes capability graph inspection, validation, and participant enablement diagnostics. |
|
||||||
|
| `diagnostics` | Core diagnostic surface | diagnostic | diagnostic-only | Lets support bundles explain capability state safely. |
|
||||||
|
| `library` | Core app workflow | provider-coordinator | safe | Models current local and plugin-provided library sources, source selection, and song sync. |
|
||||||
|
|
||||||
|
The core/runtime domains in PR1 are intentionally small: diagnostics snapshots, pipeline graph operations, and one concrete app workflow (`library`). Runtime claim and override mechanics are covered by focused behavior tests, but no plugin-owned proving domain is promoted into the runtime graph in PR1.
|
||||||
|
|
||||||
|
`diagnostics` and `pipeline` are support domains, not feature workflow domains. `diagnostics` is the read-only snapshot/export facade consumed by support bundles and the Capability Inspector. `pipeline` is the graph operations facade: resolve, inspect, validate, and enable or disable participants.
|
||||||
|
|
||||||
|
## PR1 Compatibility Shims
|
||||||
|
|
||||||
|
PR1 does not expose expected compatibility shims for `library`. Library is implemented as a native provider-coordinator domain; provider attribution comes from backend `owner_plugin_id` metadata and browser runtime provider participants.
|
||||||
|
|
||||||
|
Future domains should not add expected shim entries until their own implementation PR. A domain PR owns its compatibility story.
|
||||||
|
|
||||||
|
## Audio Graph/Session And Effects Slices
|
||||||
|
|
||||||
|
The audio graph/session and effects slices promote these domains after PR1:
|
||||||
|
|
||||||
|
| Domain | Scope | Owner Kind | Safety | Compatibility Bridges |
|
||||||
|
|--------|-------|------------|--------|-----------------------|
|
||||||
|
| `audio-mix` | Song volume, fader participants, route summary, analyser bridge accounting | provider-coordinator | safe | `audio-mix.fader-registry`, `audio-mix.song-volume`, `audio-mix.analyser` |
|
||||||
|
| `audio-input` | Redaction-safe input source registration, selection, open-session lifecycle, channel compatibility, and provider migration diagnostics | provider-coordinator | sensitive | `audio-input.legacy-source` |
|
||||||
|
| `audio-monitoring` | Monitoring provider selection, explicit user-action start/stop, direct-monitor policy, requester sharing, and audio startup barrier readiness | provider-coordinator | sensitive | `audio-monitoring.audio-barrier`, `audio-monitoring.legacy-provider` |
|
||||||
|
| `stems` | Stem automation claims and active provider status | coordinator plus plugin provider | safe | `stems.master-volume`, `stems.private-state` |
|
||||||
|
| `audio-effects` | Provider-selected effect routes, constrained chain plans, route/stage controls, fallback, and native-executor bridge accounting | provider-coordinator | sensitive | `audio-effects.legacy-tone-controls`, `audio-effects.legacy-nam-routing`, `audio-effects.legacy-midi-amp` |
|
||||||
|
|
||||||
|
`core.audio.session` is the runtime coordinator for all four domains. It owns `audio-mix`, `audio-input`, and `audio-monitoring`; for `stems`, it coordinates the active Stems provider without replacing the Stems plugin as the owner of actual stem playback/state.
|
||||||
|
|
||||||
|
The focused audio-mix control-plane slice promotes fader discovery, read/write operations, committed-value events, native-over-legacy duplicate handling, route/analyser inspection, and compatibility removal gates into `audio-mix`. During migration, `window.slopsmith.audio.registerFader(...)` remains available as a compatibility adapter, but the player mixer consumes the audio-mix control plane as its source of truth.
|
||||||
|
|
||||||
|
The focused audio-input control-plane slice promotes source listing, prompt-free selection/inspection, explicit provider enumeration, open/close dispatch, channel-shape compatibility, selected-source persistence, shared requester sessions, and redaction-safe failure diagnostics into `audio-input`. During migration, legacy browser, desktop, or plugin-specific input handoffs should be recorded as `audio-input.legacy-source` bridge hits. Native providers own the visible source when they share a logical source key with a compatibility-backed source; the compatibility source remains diagnostics-only until normal playback shows no unexpected legacy hits.
|
||||||
|
|
||||||
|
The focused audio-monitoring control-plane slice promotes monitoring provider registration, selected-provider persistence, explicit user-action live monitoring start, shared requester attachment, final-requester stop, provider disappearance/orphan diagnostics, direct-monitor preference/control summaries, prompt-free status inspection, and redaction-safe failure outcomes into `audio-monitoring`. Fresh monitoring starts from plugins/background code return `user-action-required`; background requesters may attach only to an already-active compatible monitoring session. Native providers own a logical monitoring path when they share it with compatibility-backed startup barrier or legacy monitoring surfaces, and the legacy path remains diagnostics-only until normal playback shows no unexpected bridge hits.
|
||||||
|
|
||||||
|
The focused audio-effects control-plane slice promotes provider registration, user-authorized chain selection, constrained chain-plan resolution, route bypass/restore, segment activation, stage bypass/parameter routing, fallback accounting, and redaction-safe diagnostics into `audio-effects`. Providers propose opaque chain plans; trusted desktop/native code validates the references and physically loads NAM, IR, VST, or utility stages. During migration, legacy tone controls, NAM Tone/Rig Builder native-preset route interception, and MIDI/external effect handoffs should be recorded as audio-effects bridge hits. Bridge removal gates are: NAM Tone and Rig Builder register native providers, normal playback resolves through `audio-effects` without fetch interception, failure paths degrade to NAM Tone or bypass cleanly, repeated hydration does not duplicate providers/routes, and support snapshots contain no local paths, filenames, model/IR names, URLs, raw native preset JSON, VST state blobs, handles, callbacks, DOM nodes, audio buffers, samples, or waveforms.
|
||||||
|
|
||||||
|
## Playback Control Plane Slice
|
||||||
|
|
||||||
|
The playback slice promotes `playback` from a deferred domain to an active exclusive-owner core domain. It owns transport commands (`start`, `pause`, `resume`, `stop`, `seek`, `set-loop`, `clear-loop`, `inspect`), lifecycle events (`playback:requested`, `playback:loading`, `playback:ready`, `playback:started`, `playback:paused`, `playback:resumed`, `playback:seeking`, `playback:seeked`, `playback:ended`, `playback:stopped`, route events, bridge hits, and loop events), and redaction-safe diagnostics for session, target, timing, route, loop, requester, observer, bridge, and recent outcome state.
|
||||||
|
|
||||||
|
The implementation deliberately keeps raw transport handles in `static/app.js`: the domain host registers a private adapter and receives sanitized snapshots instead of exposing the `<audio>` element, JUCE player, decoded audio buffers, waveform data, or native route handles. Playback targets expose a pseudonymous arrangement-scoped `targetId` plus a hashed per-song `settingsKey` so observers can store local per-song settings without reading raw filenames or paths. Compatibility bridges currently account for `window.playSong`, `window.slopsmith` transport helpers, legacy song events, loop helpers, media snapshots, route switching, and native-route handoff. Fresh audible starts require `authorization: "user-action"`; background requesters can inspect or control only an existing compatible session according to the command conflict policy.
|
||||||
|
|
||||||
|
Playback bridge removal gates are: bundled and first-party plugins use native playback dispatch for normal requester/observer workflows; normal play/pause/seek/loop/route smoke runs show no unexpected bridge hits beyond compatibility-only listeners; playback diagnostics distinguish denied, no-target, stale, cancelled, degraded, unavailable, failed, and stopped outcomes; repeated plugin hydration does not duplicate requesters, observers, wrappers, or bridge entries; and exported support snapshots contain no raw song filenames, paths, URLs, media handles, buffers, waveforms, samples, or recordings.
|
||||||
|
|
||||||
|
## Progression Domain Slice
|
||||||
|
|
||||||
|
The progression slice (spec 010) promotes `progression` as an active exclusive-owner core domain: mastery rank (onboarding calibration + per-path levels), the data-driven challenge/quest engine under `data/progression/`, the Decibels wallet (frontend rename of the unified XP store; spend tracked separately from lifetime earnings), and the cosmetics shop. Commands are `inspect`, `record-event` (whitelisted types; `minigame_run` in v1), `list-shop`, `buy-item`, and `equip-item` (buy/equip require explicit user action). Lifecycle events mirror as `progression:*` window events.
|
||||||
|
|
||||||
|
Deferred follow-up slices: a `contributor` role so plugins ship their own challenge/quest content (drums challenges from a drums-scoring plugin, quest-pool entries from minigame plugins), and drums scoring wiring so `song_completed {instrument: "drums"}` goals become satisfiable.
|
||||||
|
|
||||||
|
## Recommended Next Slices
|
||||||
|
|
||||||
|
The plugin inventory suggests this migration order after the audio graph/session and playback slices:
|
||||||
|
|
||||||
|
1. `jobs`: coordinate conversion, import, update, preview, and studio work with progress, cancellation, retry, and terminal failure semantics.
|
||||||
|
2. `note-detection` (control-plane slice done — see spec-009): continue with chart-path/engine migration onto bindings; audio-input coupling and calibration diagnostics are follow-up slices.
|
||||||
|
3. UI contribution host: migrate navigation, plugin screens, player controls, player panels, overlays, shortcuts, and guided tours under placement/lifecycle policy.
|
||||||
|
4. Backend and privileged capability cleanup: migrate routes, plugin lifecycle, media import/export, recording, external services, and subprocess-backed workflows with explicit user confirmation and diagnostics redaction.
|
||||||
|
|
||||||
|
## UI/UX Migration Path
|
||||||
|
|
||||||
|
This is the recommended order for UI/UX capability work only. It excludes audio semantics, backend route execution, media jobs, and plugin install/update behavior except where those systems need a visible contribution point.
|
||||||
|
|
||||||
|
| Order | Slice | Domains | Legacy Surfaces To Bridge | Migration Target | Removal Gate |
|
||||||
|
|-------|-------|---------|---------------------------|------------------|--------------|
|
||||||
|
| 1 | UI contribution substrate | `ui.navigation`, `ui.plugin-screens`, `settings`, `ui.player-controls`, `ui.player-overlays`, `ui.player-panels` | Manifest `nav`, `screen`, `settings`, direct DOM insertion, screen-specific globals | Shared contribution registry with regions, ordering, mount/unmount, visibility, focus, teardown, diagnostics, and compatibility shim accounting | Every legacy UI field is represented as a contribution in diagnostics, with no duplicate mounts after script rehydration. |
|
||||||
|
| 2 | App navigation and plugin screens | `ui.navigation`, `ui.plugin-screens` | `window.showScreen` wrappers, plugin nav entries, ad hoc screen initialization | Central screen host that owns navigation events, plugin screen lifecycle, current-screen state, and back/restore behavior | Plugins can register screens without wrapping `showScreen`; legacy wrappers are observed only as compatibility hits. |
|
||||||
|
| 3 | Settings contribution host | `settings` | Manifest `settings.html`, plugin settings panels, settings backup hints | Settings registry with panel metadata, redaction class, backup/import allowlist summary, visibility policy, and diagnostics | Settings UI can render from registered contributions; settings values remain plugin-owned and redacted. |
|
||||||
|
| 4 | Keyboard and command UX | `keyboard-shortcuts` plus UI host regions | `window.registerShortcut`, panel-scoped shortcut helpers, help panel entries | Shortcut contribution registry with scope, priority, conflict reporting, enable/disable state, and help metadata | Shortcut conflicts are diagnosable and panel-scoped shortcuts do not require private registries. |
|
||||||
|
| 5 | Player controls | `ui.player-controls` | Direct player control DOM edits, control popovers, button/slider globals | Ordered player-control regions with stable command buttons, popovers, sliders, disabled states, and contribution teardown | Player controls can be added/removed/reordered without plugins mutating the control bar directly. |
|
||||||
|
| 6 | Player overlays | `ui.player-overlays`, `tours` | Overlay canvases, tour overlays, highway visibility listeners, direct z-index management | Overlay host with anchors, z-order, hit-testing, renderer compatibility flags, visibility events, and cleanup | Fretboard, section map, tours, transpose, step mode, and similar overlays can coexist without private layering rules. |
|
||||||
|
| 7 | Player panels | `ui.player-panels` | Splitscreen panel DOM, panel-local highway instances, panel-local shortcuts | Panel host with layout slots, active-panel focus, per-panel renderer selection, per-panel shortcuts, visibility, and teardown | Splitscreen-style panels can be composed through host APIs instead of wrapping playback/screen globals. |
|
||||||
|
| 8 | Visualization UX | `visualization` | `type: "visualization"`, `window.slopsmithViz_*`, viz picker state, auto-match hooks | Renderer provider registry with picker integration, auto-match ordering, context-type metadata, fallback/revert events, and per-panel selection | Renderer selection and failure recovery are fully attributed in diagnostics; picker options no longer depend on global scans. |
|
||||||
|
| 9 | Library and guided UX extensions | `ui.library-card-injection`, `tours` | Library card buttons, tour registration globals, target selectors | Contribution APIs for library card actions and guided-tour steps with applicability, target resolution, and action-result events | Library actions and tours can be inspected, disabled, and tested independently of plugin-private DOM injection. |
|
||||||
|
| 10 | Theme and polish surfaces | `settings` or candidate `ui.theme` | Global theme settings, direct stylesheet/class mutation | Theme contribution metadata for tokens, selected theme, preview/apply/restore lifecycle, and diagnostics without user secrets | Themes are reversible and attributable, and visual changes do not depend on hidden global state. |
|
||||||
|
|
||||||
|
The UI contribution substrate should land first because every later UI/UX slice needs the same basic primitives: contribution identity, stable regions, deterministic ordering, mount/unmount, visibility, focus, teardown, diagnostics, and compatibility shim hit accounting. Specialized UI domains should stay small and should only add behavior that the shared substrate cannot express cleanly.
|
||||||
|
|
||||||
|
UI/UX migration should preserve current plugin fields during the transition. Core can translate manifest `nav`, `screen`, and `settings` into contribution records before plugin scripts hydrate, then let runtime plugins re-register richer metadata when their scripts load. The removal gate for each legacy UI API is not just a new command name; it is proof that repeated script hydration, screen switching, player navigation, and plugin disable/enable cycles do not duplicate DOM nodes, wrappers, listeners, shortcuts, canvases, or tours.
|
||||||
|
|
||||||
|
## Deferred Domains
|
||||||
|
|
||||||
|
These domains are planned but should stay out of the runtime graph until a host workflow exists:
|
||||||
|
|
||||||
|
| Domain | Expected Ownership | Expected Safety | Candidate Scope | Implementation Trigger |
|
||||||
|
|--------|--------------------|-----------------|-----------------|------------------------|
|
||||||
|
| `stems` | coordinated plugin provider | safe | Stem mute/restore, ownership claims, manual override events, and requester/observer coordination. | Promoted by the audio graph/session slice as a coordinated provider domain. |
|
||||||
|
| `ui.navigation` | exclusive-owner | safe | Navigation contributions and screen-change events. | A UI host PR that owns contribution placement and route/screen semantics. |
|
||||||
|
| `ui.plugin-screens` | exclusive-owner | safe | Plugin screen registration and lifecycle. | A screen host PR with mount/unmount and visibility policy. |
|
||||||
|
| `settings` | exclusive-owner | sensitive | Plugin settings contribution metadata without settings values. | A settings contribution PR with redaction rules and migration story. |
|
||||||
|
| `audio-mix` | multi-provider | safe | Mixer fader registration and current fader inspection. | Promoted by the audio graph/session slice. |
|
||||||
|
| `audio-monitoring` | multi-provider | sensitive | Monitoring provider selection, live start/stop lifecycle, shared requester sessions, direct-monitor state, and redacted failure diagnostics. | Promoted by the audio graph/session slice and implemented by the audio-monitoring control-plane slice. |
|
||||||
|
| `backend.routes` | multi-provider | privileged | Server route/provider participation and route inspection. | A backend domain PR with concrete core/provider workflow, privilege review, and route diagnostics. |
|
||||||
|
| `ui.player-controls` | exclusive-owner | safe | Player-control contributions and ordering. | A first-party player-control host and layout policy. |
|
||||||
|
| `ui.player-panels` | exclusive-owner | safe | Player panel contributions, mount/unmount, visibility, ordering. | A panel host with layout and focus rules. |
|
||||||
|
| `ui.player-overlays` | exclusive-owner | safe | Overlay contributions layered over player or highway surfaces. | Overlay placement and z-order rules that coexist with legacy overlays. |
|
||||||
|
| `plugins` | exclusive-owner | privileged | Plugin enable/disable/install/update workflows. | Visible user confirmation, rollback, and disabled-handler enforcement. |
|
||||||
|
| `jobs` | multi-provider | privileged | Long-running jobs, cancellation, status, failures. | Scheduling limits, cancellation semantics, and user-visible failures. |
|
||||||
|
| `midi-control` | multi-provider | sensitive | MIDI device providers and control mappings. | Device consent and redacted diagnostics. |
|
||||||
|
| `audio-input` | multi-provider | sensitive | Audio input device providers, source selection, open/close lifecycle, shared sessions, and redacted failure diagnostics. | Promoted by the audio graph/session slice and implemented by the audio-input control-plane slice. |
|
||||||
|
| `tempo-clock` | multi-provider | safe | Tempo/clock provider registration and consumers. | A concrete tempo source and consumer workflow. |
|
||||||
|
|
||||||
|
Deferred domains may remain documented or reserved, but they should not produce expected shims, inspector links, or runtime handlers before their implementation slice.
|
||||||
|
|
||||||
|
## Candidate Domains From Plugin Inventory
|
||||||
|
|
||||||
|
These candidate domains were surfaced by the included plugin inventory but are not yet part of the core deferred-domain table. They should be promoted only if a focused spec proves that the boundary is clearer than folding the behavior into an existing domain.
|
||||||
|
|
||||||
|
| Candidate Domain | Expected Ownership | Expected Safety | Candidate Scope | Initial Evidence |
|
||||||
|
|------------------|--------------------|-----------------|-----------------|------------------|
|
||||||
|
| `ui.library-card-injection` | exclusive-owner | safe | Library card actions, placement, applicability, enabled/disabled state, and action-result events. | Find More and Sloppak Converter add library-card actions that are separate from browsable library providers. |
|
||||||
|
| `tours` | exclusive-owner | safe | Guided tour registration, eligibility, target resolution, step lifecycle, and screen/navigation dependencies. | Library/settings tours, tutorials, and guided plugin walkthroughs use tour-specific lifecycle behavior. |
|
||||||
|
| `keyboard-shortcuts` | exclusive-owner | safe | Shortcut contribution registration, scope, conflict resolution, enable/disable state, and help-panel metadata. | Splitscreen, Step Mode, and practice-style plugins use or imply scoped shortcuts beyond normal UI placement. |
|
||||||
|
| `media-import-export` | multi-provider | privileged | Upload/import/export/conversion requests, accepted file types, generated artifacts, cleanup, and failure semantics. | Editor, Tab Import, Profile Import, Sloppak Converter, Studio, and Rig Builder all move user files through backend workflows. |
|
||||||
|
| `recording` | multi-provider | sensitive | Arm/start/stop capture, take upload/import, capture-source binding, latency metadata, and storage cleanup. | Studio and karaoke workflows need capture/session semantics distinct from raw audio input. |
|
||||||
|
| `practice-session` | multi-provider | safe | Practice session lifecycle, goals, score/progress events, chart segment focus, and journal persistence boundaries. | Practice Journal, Minigames, Guitar Theory, Flappy Bend, and Note Detect imply practice/progression state. |
|
||||||
|
| `collaboration` | multi-provider | sensitive | Room/session lifecycle, participant identity redaction, shared playback sync, conflict policy, and disconnect recovery. | Multiplayer is a distinct real-time coordination surface. |
|
||||||
|
| `external-services` | diagnostic or privileged metadata | privileged | Network/download/subprocess integration inventory, endpoint attribution, confirmation policy, and failure diagnostics. | Update Manager, Find More, Sloppak Converter, and media jobs reach outside local Slopsmith state. |
|
||||||
|
|
||||||
|
Candidate domains can also remain as safety metadata on existing domains. For example, `external-services` may be more useful as a cross-cutting review tag than as a dispatchable runtime capability.
|
||||||
|
|
||||||
|
## Domain Versioning
|
||||||
|
|
||||||
|
PR1 does not add per-domain versioning. The `capability-pipelines.v1` standard versions the overall manifest/runtime/diagnostics contract. Domain evolution follows compatibility rules:
|
||||||
|
|
||||||
|
- Adding optional commands, events, diagnostics fields, or participant metadata is non-breaking.
|
||||||
|
- Removing or renaming commands/events is breaking.
|
||||||
|
- Changing ownership semantics is breaking.
|
||||||
|
- Changing command payloads, return payloads, or dispatch outcomes incompatibly is breaking.
|
||||||
|
- A breaking change requires either a future `capability-pipelines` version or a clearly new domain name if parallel support is needed.
|
||||||
|
|
||||||
|
Per-domain versions should wait until Slopsmith has a concrete need for multiple incompatible versions of the same domain to coexist.
|
||||||
|
|
||||||
|
## Future Domain PR Checklist
|
||||||
|
|
||||||
|
A PR that promotes a deferred domain into the runtime graph should include:
|
||||||
|
|
||||||
|
1. User value and included/excluded command scope.
|
||||||
|
2. Host workflow or provider implementation.
|
||||||
|
3. Runtime domain review metadata.
|
||||||
|
4. Manifest and runtime registration path.
|
||||||
|
5. Compatibility shims only for legacy behavior the PR actually bridges.
|
||||||
|
6. Diagnostics fields and redaction rules.
|
||||||
|
7. Inspector behavior and meaningful labels/tooltips.
|
||||||
|
8. Tests for valid metadata, invalid metadata, unsupported versions, disabled participants, command outcomes, and shim hit accounting.
|
||||||
|
9. Documentation updates in the safety matrix and capability docs.
|
||||||
|
|
||||||
|
Before opening the PR, run the reusable review checklist in [capability-review-preflight.md](capability-review-preflight.md). It records the cross-cutting findings from previous capability reviews so each new slice can focus additional review research on the most recently merged PR.
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
# Capability Safety Matrix
|
||||||
|
|
||||||
|
Capability declarations include a safety class so reviewers can decide whether a domain can ship as a normal plugin contract or needs extra enforcement first.
|
||||||
|
|
||||||
|
Core domains also have a review scope. **Active contract** domains are wired to current Slopsmith behavior and should be tested as working integration points. Expected future domains are documented below, but are intentionally not registered in the runtime graph until Slopsmith ships the corresponding host UI or provider workflow.
|
||||||
|
|
||||||
|
| Domain | Owner Kind | Safety Class | Stable Commands | Provider Operations | Notes |
|
||||||
|
|--------|------------|--------------|-----------------|---------------------|-------|
|
||||||
|
| pipeline | diagnostic | diagnostic-only | resolve, inspect, validate, participant.set-enabled | none | Graph inspection, validation, and participant lifecycle diagnostics. |
|
||||||
|
| diagnostics | diagnostic | diagnostic-only | snapshot | none | Redaction-safe snapshot/export surface for support bundles and the Capability Inspector. |
|
||||||
|
| library | provider-coordinator | safe | list-providers, refresh-providers, select-provider, get-current, sync-song, inspect | query-page, query-artists, query-stats, tuning-names, get-art, sync-song | Library source selection and provider-owned song sync; provider ids are public UI labels, while provider internals stay backend-owned. |
|
||||||
|
| audio-mix | provider-coordinator | safe | inspect, list-faders, get-fader-value, set-fader-value, inspect-route, inspect-analyser, register-participant, unregister-participant | fader.get-value, fader.set-value, analyser.get-summary, route.get-current | Song route/fader/analyser summaries, committed fader values, native-over-legacy duplicate handling, and bridge accounting; no raw audio data is exposed. |
|
||||||
|
| audio-input | provider-coordinator | sensitive | inspect, list-sources, register-source, unregister-source, select-source, open-source, close-source | source.enumerate, source.describe, source.open, source.close | Source/device identity is redacted or pseudonymized per diagnostics snapshot. Inspect/list/select are prompt-free; `source.enumerate` runs only when explicitly requested; `open-source` is the permission boundary and records denied/unavailable/failed/incompatible/no-owner/no-handler outcomes without exposing live handles, buffers, samples, or raw device labels. |
|
||||||
|
| audio-monitoring | provider-coordinator | sensitive | inspect, list-providers, register-provider, unregister-provider, select-provider, start, stop, set-direct-monitor | monitoring.start, monitoring.stop, monitoring.status, monitoring.set-direct-monitor | Inspect/list/select/status are prompt-free. Fresh monitoring start requires explicit user action; background requesters may only attach to an active compatible session. Outcomes distinguish handled, stopped, denied, unavailable, degraded, failed, no-owner, no-handler, unsupported-command, incompatible, incompatible-version, provider-selection-required, and user-action-required. Diagnostics redact raw device labels, hardware ids, paths, secrets, live handles, buffers, samples, waveforms, and recordings. |
|
||||||
|
| stems | coordinator plus plugin provider | safe | inspect, mute, restore | stem.get-state, stem.apply-automation, stem.restore-automation | Core coordinates claims/overrides; the active Stems provider owns actual stem state/playback. |
|
||||||
|
| playback | exclusive-owner | safe | inspect, start, pause, resume, stop, seek, set-loop, clear-loop, register-requester, register-observer | none | Core owns the transport control plane while `app.js` keeps raw media handles private. Fresh audible starts require explicit user action. Diagnostics expose pseudonymous targets, sanitized route/timing/loop state, requester/observer summaries, bridge hits, bounded recent outcomes, and no audio elements, native handles, decoded buffers, samples, waveforms, or recordings. |
|
||||||
|
| progression | exclusive-owner | safe | inspect, record-event, list-shop, buy-item, equip-item | none | Core owns mastery rank, the challenge/quest engine, the Decibels wallet, and the cosmetics shop (spec 010). `record-event` accepts whitelisted types only (`minigame_run`); `song_completed` is server-derived in `/api/stats` and denied here. `buy-item`/`equip-item` require explicit user action. Decibels are play-earned only — no real-money path exists or may be added. Diagnostics (`slopsmith.progression.diag.v1`) carry content warnings, rank/level/quest counts, and wallet totals; no song filenames or display names. |
|
||||||
|
| audio-effects | provider-coordinator | sensitive | inspect, list-providers, register-provider, unregister-provider, select-chain, resolve-plan, inspect-route, bypass, restore, fallback, activate-segment, set-stage-bypass, set-stage-parameter, record-bridge-hit | chain.resolve, chain.inspect, segment.activate, stage.set-bypass, stage.set-parameter, route.bypass, route.restore | Core owns provider selection, route state, chain-plan schema validation, fallback accounting, and diagnostics. Providers propose opaque NAM/IR/VST/utility chain plans; trusted desktop/native code validates and loads processors. Chain selection and route bypass/restore require explicit user action or restored selection. Diagnostics omit raw paths, filenames, URLs, model/IR names, native preset JSON, VST state blobs, handles, callbacks, DOM nodes, audio buffers, samples, and waveforms. |
|
||||||
|
|
||||||
|
| visualization | provider-coordinator | safe | inspect, list-providers, select-renderer, clear-renderer | renderer.create, renderer.destroy | Highway renderer provider registry, picker-delegated selection, auto-match attribution, and failure fallback. `renderer.create` maps to the legacy `window.slopsmithViz_*` factory `init(canvas, ctx)` call; `renderer.destroy` maps to the factory `destroy()` teardown. Legacy `type: "visualization"` manifests and `window.slopsmithViz_*` globals are accounted compatibility shims. Diagnostics carry provider ids/labels, selection source, last auto-match outcome, and last failure — no song filenames, titles, or arrangement names. |
|
||||||
|
|
||||||
|
| note-detection | provider-coordinator | sensitive | inspect, register-provider, unregister-provider, open-binding, close-binding, set-target, clear-target | pitch.estimate, verify.target | Detection-binding control plane (spec 009): providers (midi/engine/js) serve primitives; each requester binds its own redacted tuning context; consumers own judgment, hit/miss flow as observability events. Legacy `highway.setNoteStateProvider` is an accounted shim. Diagnostics carry provider/binding summaries and bounded outcomes — no raw audio, sample data, device labels, or song identity. |
|
||||||
|
|
||||||
|
Privileged commands are roadmap-only until they have: a visible user confirmation path, diagnostics redaction rules, failure recovery, and tests that prove disabled or incompatible participants cannot execute handlers.
|
||||||
|
|
||||||
|
## Expected Future Domains
|
||||||
|
|
||||||
|
These domains are expected future capability contracts, not current runtime graph entries. They should stay documentation-only until a PR adds the corresponding host workflow and tests.
|
||||||
|
|
||||||
|
| Domain | Expected Ownership Policy | Expected Safety Class | Candidate Commands | Review Gate |
|
||||||
|
|--------|---------------------------|-----------------------|--------------------|-------------|
|
||||||
|
| ui.navigation | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, navigate, inspect | Needs a UI host PR with contribution placement and route/screen semantics. |
|
||||||
|
| ui.plugin-screens | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs a screen host PR with mount/unmount and visibility policy. |
|
||||||
|
| settings | exclusive-owner | sensitive | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs redaction rules and a migration story for settings metadata. |
|
||||||
|
| backend.routes | multi-provider | privileged | register, inspect | Needs a concrete backend route/provider workflow, privilege review, and route diagnostics. |
|
||||||
|
| ui.player-controls | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs a first-party player-control host. |
|
||||||
|
| ui.player-panels | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs a first-party panel host and layout policy. |
|
||||||
|
| ui.player-overlays | exclusive-owner | safe | register-contribution, mount, unmount, set-visible, reorder-by-policy, inspect | Needs overlay placement rules that coexist with legacy highway overlays. |
|
||||||
|
| plugins | exclusive-owner | privileged | enable, disable, install-missing, update, inspect | Needs explicit user confirmation for writes/install/update. |
|
||||||
|
| jobs | multi-provider | privileged | register, inspect, cancel | Needs scheduling limits, cancellation semantics, and user-visible failures. |
|
||||||
|
| midi-control | multi-provider | sensitive | register, inspect | Needs device consent and redacted diagnostics. |
|
||||||
|
| tempo-clock | multi-provider | safe | register, inspect | Needs a concrete provider and consumer workflow. |
|
||||||
|
|
||||||
|
Planned domains should also stay out of the runtime graph until Slopsmith ships the corresponding user-facing workflows.
|
||||||
|
|
||||||
|
When promoting a planned domain, use [capability-review-preflight.md](capability-review-preflight.md) before opening the PR. The preflight captures recurring review requirements for identity, redaction, outcome propagation, diagnostics freshness, schema consistency, and teardown.
|
||||||
@@ -0,0 +1,533 @@
|
|||||||
|
# Slopsmith Diagnostics Bundle — Format Specification
|
||||||
|
|
||||||
|
This document is the authoritative reference for the `slopsmith-diag-*.zip`
|
||||||
|
file produced by Settings → Export Diagnostics (slopsmith#166).
|
||||||
|
|
||||||
|
The bundle is consumed by humans (maintainers reading bug reports) **and**
|
||||||
|
AI agents (auto-triage, code-aware assistants). Every JSON file inside
|
||||||
|
the zip carries an explicit `schema` field so consumers can dispatch by
|
||||||
|
version without guessing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
A diagnostic bundle is a plain ZIP archive. The default filename is:
|
||||||
|
|
||||||
|
```
|
||||||
|
slopsmith-diag-<slopsmith-version>-<YYYYMMDD-HHMMSS>.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
Top-level layout:
|
||||||
|
|
||||||
|
```
|
||||||
|
slopsmith-diag-0.2.4-20260503-143022.zip
|
||||||
|
├── manifest.json AI-friendly index, schema 1
|
||||||
|
├── README.txt Human-friendly: what's in here, how to read
|
||||||
|
├── system/
|
||||||
|
│ ├── version.json slopsmith + python + OS
|
||||||
|
│ ├── env.json allowlisted env vars only (no secrets)
|
||||||
|
│ ├── hardware.json backend hardware (container-limited if Docker)
|
||||||
|
│ └── plugins.json loaded + orphan plugins, with git info
|
||||||
|
├── logs/
|
||||||
|
│ ├── server.log tail of LOG_FILE (last ~5 MB), redacted if requested
|
||||||
|
│ ├── server.pretty.log human-readable companion when LOG_FORMAT=json (auto-detected)
|
||||||
|
│ └── server.log.meta.json
|
||||||
|
├── client/
|
||||||
|
│ ├── console.json all console levels + window errors + rejections
|
||||||
|
│ ├── hardware.json browser-visible hardware: WebGL/WebGPU, host OS
|
||||||
|
│ ├── local_storage.json filtered
|
||||||
|
│ └── ua.json browser, screen, page URL on export
|
||||||
|
└── plugins/<plugin_id>/ per-plugin contributed diagnostics
|
||||||
|
```
|
||||||
|
|
||||||
|
Sections are conditional on the user's include toggles (system, hardware,
|
||||||
|
logs, console, plugins). Missing sections are not represented in
|
||||||
|
`manifest.json`'s `files` array.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `manifest.json` (bundle-level, schema `1`)
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": 1, // bundle schema; bump = breaking change
|
||||||
|
"exported_at": "2026-05-03T14:30:22Z",
|
||||||
|
"slopsmith_version": "0.2.4",
|
||||||
|
"runtime": "docker", // "docker" | "electron" | "bare"
|
||||||
|
"redacted": true, // were redactions applied?
|
||||||
|
"files": [
|
||||||
|
{ "path": "system/version.json", "kind": "json", "schema": "system.version.v1", "size": 312 },
|
||||||
|
{ "path": "logs/server.log", "kind": "text", "lines": 41203, "size": 5242880 }
|
||||||
|
],
|
||||||
|
"redactions": { // present when redacted=true
|
||||||
|
"paths_replaced": 142,
|
||||||
|
"ips_replaced": 3,
|
||||||
|
"song_names_replaced": 27,
|
||||||
|
"secrets_replaced": 1
|
||||||
|
},
|
||||||
|
"notes": [
|
||||||
|
"container masks host CPU/RAM in system/hardware.json — real host info lives in client/hardware.json"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Field semantics:
|
||||||
|
|
||||||
|
- `schema: 1` — top-level bundle schema. Increment only on breaking changes
|
||||||
|
to the layout (file moves, mandatory new sections). New optional fields
|
||||||
|
are NOT a schema bump; consumers must ignore unknown keys.
|
||||||
|
- `runtime` — single source of truth for "where was this bundle produced"
|
||||||
|
so an agent can pick the right interpretation rules. See
|
||||||
|
[Runtime kinds](#runtime-kinds).
|
||||||
|
- `files[].schema` — present only when the file's first-level JSON object
|
||||||
|
carries a string `schema` field (e.g. `"system.hardware.v1"`).
|
||||||
|
- `files[].kind` — `"json"` | `"text"` | `"binary"`.
|
||||||
|
- `notes` — human-readable callouts. Always present; may be empty.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Per-file schemas
|
||||||
|
|
||||||
|
### `system.version.v1` — `system/version.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "system.version.v1",
|
||||||
|
"slopsmith_version": "0.2.4",
|
||||||
|
"python": { "version": "3.12.4", "implementation": "CPython", "executable": "/usr/bin/python" },
|
||||||
|
"os": { "system": "Linux", "release": "6.5.0", "machine": "x86_64" },
|
||||||
|
"exported_at": "2026-05-03T14:30:22Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `system.env.v1` — `system/env.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "system.env.v1",
|
||||||
|
"vars": {
|
||||||
|
"LOG_LEVEL": "INFO",
|
||||||
|
"LOG_FORMAT": "json",
|
||||||
|
"SLOPSMITH_RUNTIME": "electron"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Allowlisted env var keys only (see `ENV_ALLOWLIST` in `lib/diagnostics_bundle.py`):
|
||||||
|
`LOG_LEVEL`, `LOG_FORMAT`, `LOG_FILE`, `SLOPSMITH_RUNTIME`, `PORT`, `HOST`,
|
||||||
|
`TZ`, `PYTHONUNBUFFERED`, `DEMUCS_SERVER_URL`. New entries require an
|
||||||
|
allowlist edit; secrets must never be added.
|
||||||
|
|
||||||
|
### `system.hardware.v1` — `system/hardware.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "system.hardware.v1",
|
||||||
|
"runtime": { "kind": "docker", "in_docker": true, "in_kubernetes": false },
|
||||||
|
"os": { "system": "Linux", "release": "6.5.0", "version": "...", "machine": "x86_64" },
|
||||||
|
"cpu": {
|
||||||
|
"brand": "AMD Ryzen 9 7950X 16-Core Processor",
|
||||||
|
"arch": "x86_64",
|
||||||
|
"cores_logical": 32,
|
||||||
|
"cores_physical": 16,
|
||||||
|
"freq_mhz_current": 4500,
|
||||||
|
"freq_mhz_max": 5700
|
||||||
|
},
|
||||||
|
"memory": { "total_bytes": 67108864000, "available_bytes": 42000000000 },
|
||||||
|
"gpu": [
|
||||||
|
{
|
||||||
|
"source": "nvidia-smi",
|
||||||
|
"name": "NVIDIA GeForce RTX 4070",
|
||||||
|
"driver": "550.54.14",
|
||||||
|
"memory_total_mb": 12282
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"notes": ["container masks host CPU/RAM"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`gpu` is a list (zero, one, or many entries). Source values in the wild:
|
||||||
|
`"nvidia-smi"`, `"rocm-smi"`, `"system_profiler"`. Container deployments
|
||||||
|
without NVIDIA Container Toolkit will have an empty list and a `notes`
|
||||||
|
entry explaining why.
|
||||||
|
|
||||||
|
### `system.plugins.v1` — `system/plugins.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "system.plugins.v1",
|
||||||
|
"plugins": [
|
||||||
|
{
|
||||||
|
"id": "stems",
|
||||||
|
"name": "Stems",
|
||||||
|
"version": "1.2.0",
|
||||||
|
"type": null,
|
||||||
|
"loaded": true,
|
||||||
|
"has_screen": true,
|
||||||
|
"has_script": true,
|
||||||
|
"has_settings": false,
|
||||||
|
"has_routes": true,
|
||||||
|
"diagnostics_declared": true,
|
||||||
|
"dir": "stems",
|
||||||
|
"standards": ["capability-pipelines.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"stems": { "roles": ["owner", "provider"], "commands": ["mute", "restore"] }
|
||||||
|
},
|
||||||
|
"capability_validation_warnings": [],
|
||||||
|
"capability_unsupported_versions": [],
|
||||||
|
"compatibility_shims": [],
|
||||||
|
"git": { "sha": "abc123d", "remote": "https://github.com/topkoa/slopsmith-plugin-stems.git" }
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"orphans": [
|
||||||
|
{
|
||||||
|
"id": "broken",
|
||||||
|
"name": "Broken Plugin",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"loaded": false,
|
||||||
|
"dir": "broken",
|
||||||
|
"path": "/home/user/.config/slopsmith/plugins/broken"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`orphans` covers plugin directories that contain a `plugin.json` but are
|
||||||
|
NOT in `LOADED_PLUGINS`. Two sub-cases:
|
||||||
|
|
||||||
|
- **Failed-to-load** (no `evicted` field): the plugin id is not loaded at
|
||||||
|
all — usually requirements install failure or manifest error. A plugin
|
||||||
|
appearing only in `orphans` without `evicted` is the single best
|
||||||
|
diagnostic signal for "user installed plugin X but it's not working".
|
||||||
|
- **Evicted/superseded** (`"evicted": true`): the plugin id IS loaded, but
|
||||||
|
from a *different* directory. Typical cause: bundled-wins logic discarded
|
||||||
|
an old user-installed clone in favour of the in-tree copy. Also covers
|
||||||
|
bundled plugin directories whose routes failed and whose server fell back
|
||||||
|
to a user copy (the bundled dir then has a different path from the loaded
|
||||||
|
entry). Check the server startup log for the specific failure reason.
|
||||||
|
|
||||||
|
`dir` is the bare directory name. `path` is the full resolved absolute path
|
||||||
|
to the orphan directory — the key disambiguator when the bundled copy and a
|
||||||
|
user-installed copy share the same directory name (e.g. both `highway_3d`).
|
||||||
|
In a redacted bundle `path` has home-dir and config-dir prefixes replaced
|
||||||
|
with placeholder tokens (e.g. `<HOME>/...`, `<CONFIG_DIR>/...`) so
|
||||||
|
filesystem paths and usernames do not leak.
|
||||||
|
|
||||||
|
Capability fields are redaction-safe manifest metadata. Invalid capability
|
||||||
|
declarations are excluded from `capabilities` and explained in
|
||||||
|
`capability_validation_warnings`; legacy surfaces still appear as
|
||||||
|
`compatibility_shims` so maintainers can see which old fields were bridged
|
||||||
|
into the capability model. Unsupported future `capability-pipelines` versions
|
||||||
|
appear in `capability_unsupported_versions` and should be treated as
|
||||||
|
non-executable runtime intent.
|
||||||
|
|
||||||
|
Client-side capability snapshots contributed under `plugins/capabilities/client.json`
|
||||||
|
use schema `slopsmith.capabilities.diagnostics.v1`. They include current
|
||||||
|
pipelines, participants, conflicts, missing providers, user overrides, active
|
||||||
|
or orphaned claims, claim lifecycle records, compatibility shim hit counts,
|
||||||
|
unsupported-version reports, and recent decisions. The runtime caps this
|
||||||
|
snapshot at 64 KB by trimming older `recentDecisions` first while preserving
|
||||||
|
current graph state.
|
||||||
|
|
||||||
|
### `logs.server.v1` — `logs/server.log.meta.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "logs.server.v1",
|
||||||
|
"log_file": "/data/log/slopsmith.log",
|
||||||
|
"exists": true,
|
||||||
|
"size_bytes": 8388608,
|
||||||
|
"tail_bytes": 5242880,
|
||||||
|
"truncated": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The companion `logs/server.log` is the raw text tail (UTF-8). When
|
||||||
|
`LOG_FORMAT=json`, every line is independently parseable as JSON.
|
||||||
|
When the file exceeds 5 MB, the partial first line is dropped before
|
||||||
|
serialization so log parsers don't choke.
|
||||||
|
|
||||||
|
When the tail is JSON-per-line (auto-detected by content, not by env
|
||||||
|
var), an additional `logs/server.pretty.log` companion is written:
|
||||||
|
human-readable lines of the form `<timestamp> [<LEVEL>] <event> k=v
|
||||||
|
k=v`. Mixed-format tails (a config flip mid-run) preserve non-JSON
|
||||||
|
lines verbatim. The original `server.log` is still emitted unchanged
|
||||||
|
for machine consumers. `server.log.meta.json:pretty_companion` is set
|
||||||
|
to `true` whenever `server.pretty.log` is present.
|
||||||
|
|
||||||
|
### `client.console.v1` — `client/console.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "client.console.v1",
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"t": 1714752622123,
|
||||||
|
"kind": "console", // "console" | "error" | "rejection"
|
||||||
|
"level": "warn", // "log" | "info" | "warn" | "error" | "debug"
|
||||||
|
"msg": "WebSocket disconnected: 1006",
|
||||||
|
"args": ["WebSocket disconnected: 1006"],
|
||||||
|
"ua": "Mozilla/5.0 ...",
|
||||||
|
"screen": { "width": 2560, "height": 1440, "devicePixelRatio": 1, "colorDepth": 24 }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"t": 1714752623456,
|
||||||
|
"kind": "rejection",
|
||||||
|
"level": "error",
|
||||||
|
"msg": "fetch failed",
|
||||||
|
"stack": "Error: ...\n at ...",
|
||||||
|
"ua": "...",
|
||||||
|
"screen": { ... }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Bounded ring buffer: 500 entries, ~250 KB cap. Each entry's `args` may
|
||||||
|
contain truncated stringifications of non-string console arguments —
|
||||||
|
depth limit 4, key cap 30, string truncation at 1024 chars, circular refs
|
||||||
|
serialized as `"[circular]"`.
|
||||||
|
|
||||||
|
### `client.hardware.v1` — `client/hardware.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "client.hardware.v1",
|
||||||
|
"runtime": {
|
||||||
|
"kind": "electron",
|
||||||
|
"electron": "28.1.0",
|
||||||
|
"chrome": "120.0.6099.109",
|
||||||
|
"node": "18.18.2",
|
||||||
|
"v8": "12.0.267.8",
|
||||||
|
"app_version": "0.2.4"
|
||||||
|
},
|
||||||
|
"navigator": {
|
||||||
|
"userAgent": "Mozilla/5.0 ...",
|
||||||
|
"platform": "Win32",
|
||||||
|
"hardwareConcurrency": 16,
|
||||||
|
"deviceMemory": 8,
|
||||||
|
"languages": ["en-US"]
|
||||||
|
},
|
||||||
|
"userAgentData": {
|
||||||
|
"platform": "Windows",
|
||||||
|
"platformVersion": "15.0.0",
|
||||||
|
"architecture": "x86",
|
||||||
|
"model": "",
|
||||||
|
"bitness": "64"
|
||||||
|
},
|
||||||
|
"screen": { "width": 2560, "height": 1440, "devicePixelRatio": 1, "colorDepth": 24 },
|
||||||
|
"webgl": {
|
||||||
|
"available": true,
|
||||||
|
"vendor": "Google Inc. (NVIDIA)",
|
||||||
|
"renderer": "ANGLE (NVIDIA, NVIDIA GeForce RTX 4070 Direct3D11 vs_5_0 ps_5_0)",
|
||||||
|
"version": "WebGL 2.0 (OpenGL ES 3.0 Chromium)",
|
||||||
|
"shading_language_version": "WebGL GLSL ES 3.00 (OpenGL ES GLSL ES 3.0 Chromium)",
|
||||||
|
"max_texture_size": 16384,
|
||||||
|
"redacted": false
|
||||||
|
},
|
||||||
|
"webgpu": {
|
||||||
|
"available": true,
|
||||||
|
"adapter_info": {
|
||||||
|
"vendor": "nvidia",
|
||||||
|
"architecture": "ada",
|
||||||
|
"device": "",
|
||||||
|
"description": ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`runtime.kind` rules:
|
||||||
|
|
||||||
|
- `"electron"` if `navigator.userAgent` contains `Electron/`. Versions
|
||||||
|
populated when the desktop launcher exposes `window.slopsmithElectron`
|
||||||
|
via a preload `contextBridge`.
|
||||||
|
- `"browser"` otherwise.
|
||||||
|
|
||||||
|
`webgl.redacted: true` indicates the browser refused to expose the real
|
||||||
|
renderer string (Firefox privacy mode, Safari ≥17). Treat the `vendor`
|
||||||
|
and `renderer` fields as advisory in that case.
|
||||||
|
|
||||||
|
### `client.local_storage.v1` — `client/local_storage.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "client.local_storage.v1",
|
||||||
|
"data": { "<key>": "<value as string>" }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Every key/value in browser `localStorage` at export time. Plugins
|
||||||
|
typically prefix their keys with their `plugin_id`.
|
||||||
|
|
||||||
|
### `client.ua.v1` — `client/ua.json`
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"schema": "client.ua.v1",
|
||||||
|
"userAgent": "...",
|
||||||
|
"url": "https://slopsmith.local/",
|
||||||
|
"screen": { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Plugin diagnostics — `plugins/<plugin_id>/...`
|
||||||
|
|
||||||
|
Per-plugin directory. Two ways to populate it:
|
||||||
|
|
||||||
|
1. `diagnostics.server_files` — relpaths under `config_dir`, copied
|
||||||
|
verbatim. Same allowlist semantics as `settings.server_files`.
|
||||||
|
2. `diagnostics.callable` — `<module>:<function>`; called with
|
||||||
|
`({"plugin_id", "config_dir"})`. Return values:
|
||||||
|
- `dict` / `list` → written to `plugins/<id>/callable.json`
|
||||||
|
- `bytes` → written to `plugins/<id>/callable.bin`
|
||||||
|
- `str` → written to `plugins/<id>/callable.txt`
|
||||||
|
- other types → discarded with a warning
|
||||||
|
Exceptions are caught and logged to the bundle's `manifest.notes`
|
||||||
|
— a buggy plugin never crashes the export.
|
||||||
|
|
||||||
|
Plugins are encouraged to embed their own `schema` field
|
||||||
|
(`"<plugin_id>.diag.v1"`) in any JSON they emit so future tooling can
|
||||||
|
dispatch by plugin schema.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Runtime kinds
|
||||||
|
|
||||||
|
`manifest.runtime` and `system/hardware.json:runtime.kind` and
|
||||||
|
`client/hardware.json:runtime.kind` may take these values:
|
||||||
|
|
||||||
|
| Kind | Backend sees… | Frontend sees… | Cross-correlate? |
|
||||||
|
|------------|-------------------------|----------------|------------------|
|
||||||
|
| `docker` | container-limited | host | NO — different machines |
|
||||||
|
| `electron` | host (Python is child) | host | YES — same machine |
|
||||||
|
| `bare` | host | host | YES — same machine |
|
||||||
|
|
||||||
|
Detection precedence (backend):
|
||||||
|
|
||||||
|
1. `SLOPSMITH_RUNTIME` env var (`"electron"`/`"docker"`/`"bare"`)
|
||||||
|
2. `/.dockerenv` exists OR `/proc/1/cgroup` mentions `docker`/
|
||||||
|
`containerd`/`kubepods` → `docker`
|
||||||
|
3. Parent process name matches `electron` or `Slopsmith` → `electron`
|
||||||
|
4. Default: `bare`
|
||||||
|
|
||||||
|
Detection (frontend): `Electron/` in user agent → `electron`, else
|
||||||
|
`browser`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Redaction
|
||||||
|
|
||||||
|
Applied to `logs/server.log` text and `client/console.json` entry
|
||||||
|
messages when `redact: true` (default). The bundle's
|
||||||
|
`manifest.json:redactions` reports per-token-class counts.
|
||||||
|
|
||||||
|
Token grammar (stable within a single bundle, salted differently
|
||||||
|
between bundles):
|
||||||
|
|
||||||
|
| Token | Source |
|
||||||
|
|--------------------|-----------------------------------------------------|
|
||||||
|
| `<DLC_DIR>` | configured DLC root path |
|
||||||
|
| `<HOME>` | user's home directory |
|
||||||
|
| `<CONFIG_DIR>` | slopsmith config directory |
|
||||||
|
| `<song:HASH8>` | song filename / basename (8-char salted SHA-256) |
|
||||||
|
| `<ip:HASH6>` | IPv4 / IPv6 address |
|
||||||
|
| `<redacted>` | bearer token, `key=`/`token=`/`api_key=` query strings |
|
||||||
|
|
||||||
|
`hardware.json` and `plugins.json` are NOT redacted (no PII).
|
||||||
|
`local_storage.json` always has values for keys matching secret-name
|
||||||
|
patterns (`api_key`, `token`, `secret`, `password`, `auth`, `bearer`,
|
||||||
|
etc.) replaced with `"<redacted>"` — this happens unconditionally,
|
||||||
|
regardless of the main redaction toggle, because plugin authors
|
||||||
|
commonly store tokens in localStorage.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Versioning policy
|
||||||
|
|
||||||
|
- **Bundle schema (`manifest.schema`)**: integer. Bump on breaking
|
||||||
|
layout changes (file relocations, removed required sections,
|
||||||
|
incompatible structural changes to existing schemas). Today: `1`.
|
||||||
|
- **Per-file schemas (`<area>.<name>.v<n>`)**: bumped independently.
|
||||||
|
A bundle MAY mix old and new file schemas during transitions.
|
||||||
|
- **Adding optional fields** to an existing schema is NOT a bump.
|
||||||
|
Consumers MUST ignore unknown keys.
|
||||||
|
- **Removing a field** is a bump.
|
||||||
|
|
||||||
|
Bundles older than the consumer's known schemas should be processed on
|
||||||
|
a best-effort basis (display what's recognized, warn about the rest).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## AI agent reading guide
|
||||||
|
|
||||||
|
Start at `manifest.json`. It lists every file with its schema id —
|
||||||
|
dispatch on schema, never on path or filename heuristics.
|
||||||
|
|
||||||
|
Common symptom → file map:
|
||||||
|
|
||||||
|
| Symptom | Files to inspect |
|
||||||
|
|----------------------------------|-----------------------------------------------------------------------------------------|
|
||||||
|
| Audio not playing | `system/plugins.json` (stems plugin loaded?), grep `logs/server.log` for `ffmpeg`/`vgmstream`, `client/console.json` for fetch errors |
|
||||||
|
| 3D highway slow / black | `client/hardware.json` (`webgl.renderer`, `webgpu.adapter_info`); `client/console.json` for WebGL warnings |
|
||||||
|
| Plugin error on load | grep `logs/server.log` for `Plugin %r`, check `system/plugins.json:orphans` for failed-to-load |
|
||||||
|
| WebSocket disconnects | `client/console.json` (`level: "warn"` / `"error"`) |
|
||||||
|
| "Works on my machine" | Diff `system/version.json` + `system/env.json` + `system/hardware.json` between bundles |
|
||||||
|
| Song-specific bug | grep `logs/server.log` for the song's `<song:HASH>` token (stable across the bundle) |
|
||||||
|
| Cross-platform crash | `manifest.runtime` + `system/hardware.json:runtime` + `client/hardware.json:runtime` |
|
||||||
|
| Cache / disk issue | `system/env.json:LOG_FILE`, `logs/server.log.meta.json:exists` |
|
||||||
|
|
||||||
|
When the bundle was redacted, the redaction token map is documented
|
||||||
|
above. Two log lines mentioning `<song:a3f1c2>` are about the same song
|
||||||
|
— but a bundle exported separately with the same song will use a
|
||||||
|
different token.
|
||||||
|
|
||||||
|
When `manifest.runtime == "docker"`, the backend `system/hardware.json`
|
||||||
|
reports container-limited values. Real host CPU / RAM / GPU live in
|
||||||
|
`client/hardware.json` only. Don't cross-correlate.
|
||||||
|
|
||||||
|
When `manifest.runtime == "electron"`, both halves describe the same
|
||||||
|
machine.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plugin contribution contract
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
// plugin.json
|
||||||
|
{
|
||||||
|
"id": "nam_tone",
|
||||||
|
"name": "NAM Tone",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"diagnostics": {
|
||||||
|
"server_files": ["nam_tone.db.diag.json"],
|
||||||
|
"callable": "diagnostics:collect"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Frontend plugins push diagnostics by calling
|
||||||
|
`window.slopsmith.diagnostics.contribute(plugin_id, payload)` before the
|
||||||
|
user clicks Export. The payload is written to `plugins/<id>/client.json`
|
||||||
|
(gated on the same "Plugin diagnostics" toggle as backend plugin files).
|
||||||
|
|
||||||
|
Backend callable signature:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# plugins/nam_tone/diagnostics.py
|
||||||
|
def collect(ctx: dict) -> dict | bytes | str:
|
||||||
|
"""ctx: {'plugin_id': 'nam_tone', 'config_dir': Path(...)}"""
|
||||||
|
return {
|
||||||
|
"schema": "nam_tone.diag.v1",
|
||||||
|
"models": [...],
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Best practices:
|
||||||
|
|
||||||
|
- Return small payloads (< 100 KB). Diagnostics are not a backup channel.
|
||||||
|
- Embed your own `schema` field in returned dicts.
|
||||||
|
- Never raise — but if you do, the export keeps going and notes the
|
||||||
|
failure.
|
||||||
|
- Don't include user secrets, API keys, or session tokens.
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
# Slopsmith diagnostic sloppaks
|
||||||
|
|
||||||
|
Generated, non-copyrighted mini-songs for technique-assessment style
|
||||||
|
checks. Report-only — they do not change gameplay settings or detection
|
||||||
|
thresholds.
|
||||||
|
|
||||||
|
## Basic Guitar (POC)
|
||||||
|
|
||||||
|
**Artifact:** `slopsmith-diagnostic-basic-guitar.sloppak`
|
||||||
|
|
||||||
|
**Contents (~55 s):**
|
||||||
|
|
||||||
|
- 3 s count-in (quiet click)
|
||||||
|
- Open thickest string, open next string
|
||||||
|
- Thickest string, 5th fret
|
||||||
|
- Repeated E5 power chords (thickest open + next string fret 2)
|
||||||
|
- Repeat pass: open, fretted, power chords again
|
||||||
|
|
||||||
|
Sections: Intro, Open Strings, Fretted Note, Power Chords, Repeat Check.
|
||||||
|
|
||||||
|
Manifest includes a custom `diagnostic:` tag (ignored by the loader today;
|
||||||
|
for future Technique Assessment integration).
|
||||||
|
|
||||||
|
## Rebuild
|
||||||
|
|
||||||
|
From the slopsmith repo root (requires `ffmpeg`; the slopsmith Docker image
|
||||||
|
has `libvorbis`, Homebrew ffmpeg may use the built-in `vorbis` encoder):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 docs/diagnostics/build_diagnostic_basic_guitar.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Builtin seeding
|
||||||
|
|
||||||
|
On library scan startup (and periodic rescans), the server copies bundled
|
||||||
|
diagnostic sloppaks into the user DLC folder when missing or when the
|
||||||
|
bundled source is newer:
|
||||||
|
|
||||||
|
`DLC_DIR/diagnostics-builtin/slopsmith-diagnostic-basic-guitar.sloppak`
|
||||||
|
|
||||||
|
Source: `docs/diagnostics/slopsmith-diagnostic-basic-guitar.sloppak` (next to
|
||||||
|
`server.py` in dev; must be included in the desktop bundle — see
|
||||||
|
`slopsmith-desktop/scripts/bundle-slopsmith.sh`).
|
||||||
|
|
||||||
|
Unlike `tutorials-builtin/`, `diagnostics-builtin/` **is** included in the
|
||||||
|
library scan. Tracks appear under **Slopsmith** /
|
||||||
|
**Technique Assessment Diagnostics**.
|
||||||
|
|
||||||
|
Existing destination files are not overwritten unless the bundled source
|
||||||
|
has a newer modification time. User files elsewhere (e.g. `diagnostics-test/`)
|
||||||
|
are never touched.
|
||||||
|
|
||||||
|
## Manual install / test
|
||||||
|
|
||||||
|
Normally seeding is automatic once a DLC folder is configured. To test a
|
||||||
|
custom copy or an unreleased build:
|
||||||
|
|
||||||
|
1. Copy `slopsmith-diagnostic-basic-guitar.sloppak` into your Slopsmith
|
||||||
|
DLC folder (e.g. `diagnostics-test/` or any scanned path).
|
||||||
|
2. Restart Slopsmith or trigger a library rescan if the song does not appear.
|
||||||
|
3. Load **Slopsmith Diagnostic — Basic Guitar**.
|
||||||
|
4. Play the **Diagnostic Guitar** arrangement.
|
||||||
|
5. Confirm the 3D highway shows open notes and power-chord gems.
|
||||||
|
6. Turn **Detect** on — note_detect should push the chart to the desktop
|
||||||
|
verifier on `song:ready` like any other sloppak.
|
||||||
|
|
||||||
|
## Future
|
||||||
|
|
||||||
|
- Bass diagnostic sloppak
|
||||||
|
- 7/8-string guitar variants
|
||||||
|
- Drums (`drum_tab.json`)
|
||||||
|
- Piano/keys (separate wire model)
|
||||||
|
- Detection Health “Run Basic Guitar Diagnostic” launch button (note_detect)
|
||||||
@@ -0,0 +1,441 @@
|
|||||||
|
"""Build the Slopsmith Diagnostic — Basic Guitar sloppak (POC).
|
||||||
|
|
||||||
|
A short, generated, non-copyrighted mini-song for technique-assessment
|
||||||
|
style checks: open strings, one fretted note, and repeated E5 power chords.
|
||||||
|
Click-track backing only — no external audio.
|
||||||
|
|
||||||
|
Run from the slopsmith repo root:
|
||||||
|
|
||||||
|
python3 docs/diagnostics/build_diagnostic_basic_guitar.py
|
||||||
|
|
||||||
|
Output (zip archive):
|
||||||
|
|
||||||
|
docs/diagnostics/slopsmith-diagnostic-basic-guitar.sloppak
|
||||||
|
|
||||||
|
Pattern matches docs/benchmarks/note_detect_v1/build_benchmark.py.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import shutil
|
||||||
|
import struct
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import wave
|
||||||
|
import zipfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
except ImportError:
|
||||||
|
yaml = None
|
||||||
|
|
||||||
|
|
||||||
|
def _yaml_scalar(v):
|
||||||
|
if isinstance(v, bool):
|
||||||
|
return 'true' if v else 'false'
|
||||||
|
if isinstance(v, int):
|
||||||
|
return str(v)
|
||||||
|
if isinstance(v, float):
|
||||||
|
return repr(v)
|
||||||
|
if v is None:
|
||||||
|
return 'null'
|
||||||
|
s = str(v)
|
||||||
|
if any(c in s for c in ':{}[]&*#?|-<>=!%@`"') or s.strip() != s:
|
||||||
|
return json.dumps(s, ensure_ascii=False)
|
||||||
|
return s
|
||||||
|
|
||||||
|
|
||||||
|
def _yaml_lines(obj, indent=0):
|
||||||
|
prefix = ' ' * indent
|
||||||
|
lines = []
|
||||||
|
if isinstance(obj, dict):
|
||||||
|
for k, v in obj.items():
|
||||||
|
if isinstance(v, dict):
|
||||||
|
lines.append(f'{prefix}{k}:')
|
||||||
|
lines.extend(_yaml_lines(v, indent + 1))
|
||||||
|
elif isinstance(v, list):
|
||||||
|
if not v:
|
||||||
|
lines.append(f'{prefix}{k}: []')
|
||||||
|
elif all(isinstance(x, dict) for x in v):
|
||||||
|
lines.append(f'{prefix}{k}:')
|
||||||
|
for item in v:
|
||||||
|
lines.append(f'{prefix} -')
|
||||||
|
for ik, iv in item.items():
|
||||||
|
if isinstance(iv, (dict, list)):
|
||||||
|
lines.append(f'{prefix} {ik}:')
|
||||||
|
lines.extend(_yaml_lines(iv, indent + 3))
|
||||||
|
else:
|
||||||
|
lines.append(f'{prefix} {ik}: {_yaml_scalar(iv)}')
|
||||||
|
else:
|
||||||
|
lines.append(f'{prefix}{k}:')
|
||||||
|
for item in v:
|
||||||
|
lines.append(f'{prefix} - {_yaml_scalar(item)}')
|
||||||
|
else:
|
||||||
|
lines.append(f'{prefix}{k}: {_yaml_scalar(v)}')
|
||||||
|
elif isinstance(obj, list):
|
||||||
|
for item in obj:
|
||||||
|
if isinstance(item, dict):
|
||||||
|
lines.append(f'{prefix}-')
|
||||||
|
for k, v in item.items():
|
||||||
|
if isinstance(v, (dict, list)):
|
||||||
|
lines.append(f'{prefix} {k}:')
|
||||||
|
lines.extend(_yaml_lines(v, indent + 2))
|
||||||
|
else:
|
||||||
|
lines.append(f'{prefix} {k}: {_yaml_scalar(v)}')
|
||||||
|
else:
|
||||||
|
lines.append(f'{prefix}- {_yaml_scalar(item)}')
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
|
def dump_manifest_yaml(manifest: dict) -> str:
|
||||||
|
if yaml is not None:
|
||||||
|
return yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True)
|
||||||
|
return '\n'.join(_yaml_lines(manifest)) + '\n'
|
||||||
|
|
||||||
|
|
||||||
|
# ── Chart timing ────────────────────────────────────────────────────────
|
||||||
|
BPM = 90.0
|
||||||
|
SECONDS_PER_BEAT = 60.0 / BPM
|
||||||
|
BEATS_PER_BAR = 4
|
||||||
|
BAR_S = BEATS_PER_BAR * SECONDS_PER_BEAT
|
||||||
|
|
||||||
|
COUNT_IN_S = 3.0 # silence / quiet count-in before first note
|
||||||
|
NOTE_SUS = 2.8 # single-note ring time
|
||||||
|
CHORD_SUS = 3.0 # power-chord ring time
|
||||||
|
OUTRO_S = 4.0 # tail after last event
|
||||||
|
|
||||||
|
SR = 44100
|
||||||
|
|
||||||
|
|
||||||
|
def note(t, s, f, sus=0.0, **flags):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
's': s,
|
||||||
|
'f': f,
|
||||||
|
'sus': round(sus, 3),
|
||||||
|
'sl': flags.get('sl', -1),
|
||||||
|
'slu': flags.get('slu', -1),
|
||||||
|
'bn': flags.get('bn', 0.0),
|
||||||
|
'ho': flags.get('ho', False),
|
||||||
|
'po': flags.get('po', False),
|
||||||
|
'hm': flags.get('hm', False),
|
||||||
|
'hp': flags.get('hp', False),
|
||||||
|
'pm': flags.get('pm', False),
|
||||||
|
'mt': flags.get('mt', False),
|
||||||
|
'vb': flags.get('vb', False),
|
||||||
|
'tr': flags.get('tr', False),
|
||||||
|
'ac': flags.get('ac', False),
|
||||||
|
'tp': flags.get('tp', False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord(t, id_, notes):
|
||||||
|
return {
|
||||||
|
't': round(t, 3),
|
||||||
|
'id': id_,
|
||||||
|
'hd': False,
|
||||||
|
'notes': notes,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def chord_note(s, f, sus=0.0, **flags):
|
||||||
|
n = note(0.0, s, f, sus, **flags)
|
||||||
|
n.pop('t')
|
||||||
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
def _sine_burst(freq_hz, duration_s, amplitude):
|
||||||
|
n = int(SR * duration_s)
|
||||||
|
out = []
|
||||||
|
fade = max(1, int(0.004 * SR))
|
||||||
|
for i in range(n):
|
||||||
|
env = 1.0
|
||||||
|
if i < fade:
|
||||||
|
env = i / fade
|
||||||
|
elif i >= n - fade:
|
||||||
|
env = (n - 1 - i) / fade
|
||||||
|
s = math.sin(2 * math.pi * freq_hz * (i / SR)) * amplitude * env
|
||||||
|
out.append(s)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def write_click_wav(path: Path, total_duration_s: float, count_in_s: float):
|
||||||
|
"""Metronome click on every beat; quieter during count-in."""
|
||||||
|
n_total = int(math.ceil(total_duration_s * SR))
|
||||||
|
buf = [0.0] * n_total
|
||||||
|
|
||||||
|
click_dur = 0.045
|
||||||
|
downbeat_tone = 1500
|
||||||
|
upbeat_tone = 1000
|
||||||
|
downbeat_amp = 0.22
|
||||||
|
upbeat_amp = 0.12
|
||||||
|
count_in_amp_scale = 0.35
|
||||||
|
|
||||||
|
beat_idx = 0
|
||||||
|
t = 0.0
|
||||||
|
while t < total_duration_s - click_dur:
|
||||||
|
is_downbeat = (beat_idx % BEATS_PER_BAR) == 0
|
||||||
|
amp = downbeat_amp if is_downbeat else upbeat_amp
|
||||||
|
if t < count_in_s:
|
||||||
|
amp *= count_in_amp_scale
|
||||||
|
click = _sine_burst(
|
||||||
|
downbeat_tone if is_downbeat else upbeat_tone,
|
||||||
|
click_dur,
|
||||||
|
amp,
|
||||||
|
)
|
||||||
|
i0 = int(t * SR)
|
||||||
|
for j, v in enumerate(click):
|
||||||
|
if i0 + j < n_total:
|
||||||
|
buf[i0 + j] += v
|
||||||
|
t += SECONDS_PER_BEAT
|
||||||
|
beat_idx += 1
|
||||||
|
|
||||||
|
pcm = bytearray()
|
||||||
|
for v in buf:
|
||||||
|
s = max(-1.0, min(1.0, v))
|
||||||
|
pcm.extend(struct.pack('<h', int(s * 32700)))
|
||||||
|
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with wave.open(str(path), 'wb') as w:
|
||||||
|
w.setnchannels(1)
|
||||||
|
w.setsampwidth(2)
|
||||||
|
w.setframerate(SR)
|
||||||
|
w.writeframes(bytes(pcm))
|
||||||
|
|
||||||
|
|
||||||
|
def _power_chord_e5(t):
|
||||||
|
"""E5: thickest string open + next string fret 2."""
|
||||||
|
return chord(
|
||||||
|
t,
|
||||||
|
0,
|
||||||
|
[
|
||||||
|
chord_note(0, 0, sus=CHORD_SUS),
|
||||||
|
chord_note(1, 2, sus=CHORD_SUS),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_chart():
|
||||||
|
notes = []
|
||||||
|
chords = []
|
||||||
|
sections = []
|
||||||
|
|
||||||
|
# ── Event times (seconds) ──
|
||||||
|
t_open_low = 4.0
|
||||||
|
t_open_next = 8.0
|
||||||
|
t_fret5 = 12.0
|
||||||
|
power_times_1 = [16.0, 20.0, 24.0, 28.0]
|
||||||
|
t_open_repeat = 32.0
|
||||||
|
t_fret5_repeat = 36.0
|
||||||
|
power_times_2 = [40.0, 44.0, 48.0]
|
||||||
|
|
||||||
|
notes.append(note(t_open_low, 0, 0, sus=NOTE_SUS))
|
||||||
|
notes.append(note(t_open_next, 1, 0, sus=NOTE_SUS))
|
||||||
|
notes.append(note(t_fret5, 0, 5, sus=NOTE_SUS))
|
||||||
|
notes.append(note(t_open_repeat, 0, 0, sus=NOTE_SUS))
|
||||||
|
notes.append(note(t_fret5_repeat, 0, 5, sus=NOTE_SUS))
|
||||||
|
|
||||||
|
for t in power_times_1 + power_times_2:
|
||||||
|
chords.append(_power_chord_e5(t))
|
||||||
|
|
||||||
|
last_event_t = max(power_times_2)
|
||||||
|
end_t = last_event_t + CHORD_SUS + OUTRO_S
|
||||||
|
|
||||||
|
sections = [
|
||||||
|
{'name': 'Intro', 'number': 1, 'time': 0.0},
|
||||||
|
{'name': 'Open Strings', 'number': 2, 'time': round(t_open_low, 3)},
|
||||||
|
{'name': 'Fretted Note', 'number': 3, 'time': round(t_fret5, 3)},
|
||||||
|
{'name': 'Power Chords', 'number': 4, 'time': round(power_times_1[0], 3)},
|
||||||
|
{'name': 'Repeat Check', 'number': 5, 'time': round(t_open_repeat, 3)},
|
||||||
|
]
|
||||||
|
|
||||||
|
beats = []
|
||||||
|
bar_count = 0
|
||||||
|
bt = 0.0
|
||||||
|
while bt < end_t:
|
||||||
|
if abs(bt % BAR_S) < 1e-3:
|
||||||
|
bar_count += 1
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': bar_count})
|
||||||
|
else:
|
||||||
|
beats.append({'time': round(bt, 3), 'measure': -1})
|
||||||
|
bt += SECONDS_PER_BEAT
|
||||||
|
|
||||||
|
anchors = [{'time': 0.0, 'fret': 0, 'width': 6}]
|
||||||
|
for sec in sections:
|
||||||
|
anchors.append({'time': sec['time'], 'fret': 0, 'width': 6})
|
||||||
|
|
||||||
|
templates = [{
|
||||||
|
'name': 'E5',
|
||||||
|
'displayName': 'E5',
|
||||||
|
'arp': False,
|
||||||
|
'fingers': [-1, -1, -1, -1, -1, -1],
|
||||||
|
'frets': [0, 2, -1, -1, -1, -1],
|
||||||
|
}]
|
||||||
|
|
||||||
|
arrangement = {
|
||||||
|
'name': 'Diagnostic Guitar',
|
||||||
|
'tuning': [0, 0, 0, 0, 0, 0],
|
||||||
|
'capo': 0,
|
||||||
|
'notes': sorted(notes, key=lambda n: n['t']),
|
||||||
|
'chords': sorted(chords, key=lambda c: c['t']),
|
||||||
|
'anchors': anchors,
|
||||||
|
'handshapes': [],
|
||||||
|
'templates': templates,
|
||||||
|
'beats': beats,
|
||||||
|
'sections': sections,
|
||||||
|
}
|
||||||
|
|
||||||
|
manifest = {
|
||||||
|
'title': 'Slopsmith Diagnostic — Basic Guitar',
|
||||||
|
'artist': 'Slopsmith',
|
||||||
|
'album': 'Technique Assessment Diagnostics',
|
||||||
|
'year': 2026,
|
||||||
|
'duration': round(end_t, 3),
|
||||||
|
'arrangements': [{
|
||||||
|
'id': 'lead',
|
||||||
|
'name': 'Diagnostic Guitar',
|
||||||
|
'file': 'arrangements/lead.json',
|
||||||
|
'tuning': [0, 0, 0, 0, 0, 0],
|
||||||
|
'capo': 0,
|
||||||
|
}],
|
||||||
|
'stems': [{
|
||||||
|
'id': 'full',
|
||||||
|
'file': 'stems/full.ogg',
|
||||||
|
'default': True,
|
||||||
|
}],
|
||||||
|
'diagnostic': {
|
||||||
|
'kind': 'technique-assessment-basic',
|
||||||
|
'instrument': 'guitar',
|
||||||
|
'string_count': 6,
|
||||||
|
'version': 1,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
return manifest, arrangement, end_t
|
||||||
|
|
||||||
|
|
||||||
|
def _build_zip(src_dir: Path, zip_path: Path):
|
||||||
|
if zip_path.exists():
|
||||||
|
zip_path.unlink()
|
||||||
|
with zipfile.ZipFile(zip_path, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
|
||||||
|
for p in sorted(src_dir.rglob('*')):
|
||||||
|
if p.is_file():
|
||||||
|
rel = p.relative_to(src_dir).as_posix()
|
||||||
|
info = zipfile.ZipInfo(filename=rel, date_time=(1980, 1, 1, 0, 0, 0))
|
||||||
|
info.compress_type = zipfile.ZIP_DEFLATED
|
||||||
|
info.external_attr = (0o644 & 0xFFFF) << 16
|
||||||
|
info.create_system = 3
|
||||||
|
zf.writestr(info, p.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
def build(output_zip: Path) -> dict:
|
||||||
|
manifest, arrangement, end_t = build_chart()
|
||||||
|
|
||||||
|
staging = output_zip.parent / '_diag_basic_guitar_staging'
|
||||||
|
if staging.exists():
|
||||||
|
shutil.rmtree(staging)
|
||||||
|
staging.mkdir(parents=True)
|
||||||
|
(staging / 'arrangements').mkdir()
|
||||||
|
(staging / 'stems').mkdir()
|
||||||
|
|
||||||
|
(staging / 'manifest.yaml').write_text(
|
||||||
|
dump_manifest_yaml(manifest),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
(staging / 'arrangements' / 'lead.json').write_text(
|
||||||
|
json.dumps(arrangement, separators=(',', ':')),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
|
||||||
|
wav_path = staging / 'stems' / 'full.wav'
|
||||||
|
write_click_wav(wav_path, end_t, COUNT_IN_S)
|
||||||
|
ogg_path = staging / 'stems' / 'full.ogg'
|
||||||
|
encoder_cmds = [
|
||||||
|
['-c:a', 'libvorbis', '-q:a', '5'],
|
||||||
|
# FFmpeg 8's built-in vorbis encoder requires stereo input.
|
||||||
|
['-strict', '-2', '-ac', '2', '-c:a', 'vorbis', '-q:a', '5'],
|
||||||
|
]
|
||||||
|
last_err = None
|
||||||
|
for enc_args in encoder_cmds:
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
['ffmpeg', '-y', '-loglevel', 'error',
|
||||||
|
'-i', str(wav_path),
|
||||||
|
*enc_args,
|
||||||
|
str(ogg_path)],
|
||||||
|
check=True,
|
||||||
|
stderr=subprocess.DEVNULL if enc_args != encoder_cmds[-1] else None,
|
||||||
|
)
|
||||||
|
last_err = None
|
||||||
|
break
|
||||||
|
except FileNotFoundError as e:
|
||||||
|
shutil.rmtree(staging, ignore_errors=True)
|
||||||
|
raise RuntimeError(
|
||||||
|
'ffmpeg not found — install ffmpeg to build the OGG stem.'
|
||||||
|
) from e
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
last_err = e
|
||||||
|
if last_err is not None:
|
||||||
|
shutil.rmtree(staging, ignore_errors=True)
|
||||||
|
raise RuntimeError(
|
||||||
|
'ffmpeg failed to encode stems/full.ogg — tried libvorbis and vorbis encoders.'
|
||||||
|
) from last_err
|
||||||
|
wav_path.unlink()
|
||||||
|
|
||||||
|
(staging / 'DIAGNOSTIC.md').write_text(
|
||||||
|
_diagnostic_readme(end_t),
|
||||||
|
encoding='utf-8',
|
||||||
|
)
|
||||||
|
|
||||||
|
_build_zip(staging, output_zip)
|
||||||
|
shutil.rmtree(staging, ignore_errors=True)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'output': output_zip,
|
||||||
|
'duration_s': end_t,
|
||||||
|
'notes': len(arrangement['notes']),
|
||||||
|
'chords': len(arrangement['chords']),
|
||||||
|
'sections': len(arrangement['sections']),
|
||||||
|
'stem': 'stems/full.ogg',
|
||||||
|
'size_bytes': output_zip.stat().st_size,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _diagnostic_readme(duration_s: float) -> str:
|
||||||
|
return f"""# Slopsmith Diagnostic — Basic Guitar
|
||||||
|
|
||||||
|
Short generated diagnostic track for technique-assessment style checks.
|
||||||
|
Non-copyrighted click-track backing only.
|
||||||
|
|
||||||
|
- Duration: {duration_s:.0f} s
|
||||||
|
- Tuning: E standard (6-string), capo 0
|
||||||
|
- Sections: Intro, Open Strings, Fretted Note, Power Chords, Repeat Check
|
||||||
|
|
||||||
|
Report-only — does not change gameplay settings.
|
||||||
|
|
||||||
|
Built by docs/diagnostics/build_diagnostic_basic_guitar.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
repo_root = Path(__file__).resolve().parents[2]
|
||||||
|
default_out = Path(__file__).resolve().parent / 'slopsmith-diagnostic-basic-guitar.sloppak'
|
||||||
|
out = Path(sys.argv[1]) if len(sys.argv) > 1 else default_out
|
||||||
|
if not out.is_absolute():
|
||||||
|
out = repo_root / out
|
||||||
|
|
||||||
|
stats = build(out)
|
||||||
|
print(f'Built {stats["output"]}')
|
||||||
|
print(f' Duration: {stats["duration_s"]:.1f} s')
|
||||||
|
print(f' Notes: {stats["notes"]}')
|
||||||
|
print(f' Chords: {stats["chords"]}')
|
||||||
|
print(f' Sections: {stats["sections"]}')
|
||||||
|
print(f' Stem: {stats["stem"]}')
|
||||||
|
print(f' Size: {stats["size_bytes"]} bytes')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
@@ -0,0 +1,231 @@
|
|||||||
|
# Note Detection Tuning Workflow
|
||||||
|
|
||||||
|
How to iterate on the `note_detect` plugin's detection quality with objective, repeatable measurements instead of "feels worse / feels better" guesswork. The same workflow works for tuning the user's environment (A/V offset, latency comp, channel selection) and for tuning the detector code itself (frame size, confidence thresholds, chord-scoring algorithm).
|
||||||
|
|
||||||
|
## Why this exists
|
||||||
|
|
||||||
|
Detection quality varies by guitar pickup, audio interface, monitor latency, the user's playing style, and the chart's note density. Eyeballing the player UI tells you whether something feels right, not whether a change improved or regressed scoring. The pieces below let you record once and replay many times against arbitrary parameter combinations:
|
||||||
|
|
||||||
|
- **Reference recording** — captures the exact PCM frames the live detector saw, so a single take can be re-scored against any settings.
|
||||||
|
- **Benchmark sloppak** — a known, distributable chart with isolated failure-mode sections.
|
||||||
|
- **Headless harness** — runs the same `processFrame` / `matchNotes` / `checkMisses` code path the browser uses, off Node, in seconds per run.
|
||||||
|
- **Diagnostic JSON** — both live (in-browser) and harness output share the `note_detect.diagnostic.v1` schema, so cross-comparison is trivial.
|
||||||
|
|
||||||
|
## The benchmark sloppak
|
||||||
|
|
||||||
|
The distributable sloppak ships in-tree at [docs/benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak](benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak) — drop it directly in your library folder (e.g. `<your-library>/sloppak/`) and it shows up in the library. The file is a zip under the hood but slopsmith's loader (`is_sloppak`) keys off the `.sloppak` suffix, so don't rename. After playing it once it ends up extracted under `static/sloppak_cache/note_detect_benchmark_v1.sloppak/`, which is where the harness reads its `arrangements/lead.json` from. 90 BPM, 8 numbered sections, ~2:20 total:
|
||||||
|
|
||||||
|
| Section | Notes | Isolates |
|
||||||
|
|---|---|---|
|
||||||
|
| A. Open strings | 12 single notes | low-frequency YIN behaviour (E2=82 Hz) |
|
||||||
|
| B. 5th-fret positions | 12 single notes | mid-range pitch accuracy |
|
||||||
|
| C. 12th-fret octaves | sparse single notes | high-frequency YIN behaviour |
|
||||||
|
| D. Sustained notes | long-hold single notes | sustain matching / pure-miss vs detected |
|
||||||
|
| E. Hammer / pull | legato pairs | technique-flag handling, attack ambiguity |
|
||||||
|
| F. Power chords | 8 chord events | 2-string chord scorer |
|
||||||
|
| G. Open chords | 8 chord events | dense chord scorer (5+ strings ringing) |
|
||||||
|
| H. Bends | bend pairs | pitch-tolerance edge behaviour |
|
||||||
|
|
||||||
|
Every chart note has `sus > 0` — so anything you tune against this benchmark exercises the sustain path, not staccato detection. (If we add a staccato section later, the cleanest split is by section name; don't categorize by `sus` value on the event log — see the "Common pitfalls" section.)
|
||||||
|
|
||||||
|
To rebuild after edits to the exercise list, follow the docstring at the top of `build_benchmark.py`. The script writes both an unzipped directory (`.sloppak/`) and a zipped archive (`.sloppak.zip`). The slopsmith library scanner (`lib/sloppak.py::is_sloppak()`) matches on the `.sloppak` suffix, **not** on `.sloppak.zip` — the directory form is usable as-is, but the zip output needs its suffix swapped before it'll be discovered. After regenerating, copy the zip output to the tracked path with the `.sloppak` suffix so it stays a drop-in install. Run from the slopsmith repo root so the relative paths resolve:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From the slopsmith repo root.
|
||||||
|
cp static/sloppak_cache/note_detect_benchmark_v1.sloppak.zip \
|
||||||
|
docs/benchmarks/note_detect_v1/note_detect_benchmark_v1.sloppak
|
||||||
|
```
|
||||||
|
|
||||||
|
Also update `docs/benchmarks/note_detect_v1/BENCHMARK.md` if you changed sections — it's the user-facing description that ships inside the sloppak, kept alongside the tracked file so contributors can see the section list without having to unzip.
|
||||||
|
|
||||||
|
## End-to-end iteration loop
|
||||||
|
|
||||||
|
The typical cycle for one tuning hypothesis:
|
||||||
|
|
||||||
|
1. **Enable tuning mode** (Settings → Note Detection → "Detection tuning (advanced)"). Off by default; turns on the dev surfaces (Reference Recording, Diagnostic JSON, miss-category breakdown).
|
||||||
|
2. **Arm a recording** from the gear popover next to the Detect button on the player. Arm before pressing Play.
|
||||||
|
3. **Play through the benchmark** (or any song) at **1.0× playback speed**. Half-speed playback breaks audio↔chart alignment and produces all-miss garbage — see Pitfalls.
|
||||||
|
4. **Auto-save fires on song end.** The WAV lands in `static/note_detect_recordings/note_detect_<slug>_<timestamp>.wav` (bind-mounted, so it's reachable from the host without a copy step).
|
||||||
|
5. **Run the headless harness** with a known config. Paths below assume the note_detect plugin is cloned into `plugins/note_detect/` (see the slopsmith README for the plugin-install flow — note_detect ships as a separate repo):
|
||||||
|
```bash
|
||||||
|
node plugins/note_detect/tools/harness.js \
|
||||||
|
--audio static/note_detect_recordings/note_detect_<…>.wav \
|
||||||
|
--chart static/sloppak_cache/note_detect_benchmark_v1.sloppak/arrangements/lead.json \
|
||||||
|
--out /tmp/run.json
|
||||||
|
```
|
||||||
|
Prints a one-liner: `<hits>/<total> hits (<%>) — breakdown {pure, chordPartial, early, late, sharp, flat}`.
|
||||||
|
6. **Sweep parameters** by re-running the harness with different flags (see "Harness flags" below). Compare bins side-by-side. The same recording can drive dozens of runs in seconds.
|
||||||
|
7. **Form a hypothesis, change code or settings, repeat.** Each PR or settings tweak should move at least one bin in the right direction. If you can't show that, you don't have evidence to ship it.
|
||||||
|
|
||||||
|
## Harness flags
|
||||||
|
|
||||||
|
All flags map 1:1 to a runtime setting; defaults mirror what a fresh plugin install ships with:
|
||||||
|
|
||||||
|
| Flag | Default | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `--audio <path>` | — | WAV/OGG/MP3 input. WAV is parsed natively; other formats need ffmpeg on PATH. |
|
||||||
|
| `--chart <path>` | — | The arrangement JSON (e.g. `arrangements/lead.json` from a sloppak directory). |
|
||||||
|
| `--out <path>` | — | Diagnostic JSON destination. |
|
||||||
|
| `--method yin\|hps` | `yin` | CREPE is not exercised by the harness (needs WebGL). |
|
||||||
|
| `--pitch-tolerance <cents>` | `50` | Outer match window for pitch. |
|
||||||
|
| `--pitch-hit-threshold <cents>` | `20` | Tighter band that counts as "clean" pitch. |
|
||||||
|
| `--timing-tolerance <s>` | `0.150` | Outer match window for timing. |
|
||||||
|
| `--timing-hit-threshold <s>` | `0.100` | Tighter band that counts as "clean" timing. |
|
||||||
|
| `--chord-hit-ratio <r>` | `0.6` | Fraction of strings that must ring for a chord hit (per-string energy bands). |
|
||||||
|
| `--latency <s>` | `0.080` | Detector pipeline latency compensation. |
|
||||||
|
| `--frame-size <n>` | `1024` | YIN buffer size in samples. Bigger = better low-freq detection, more latency. |
|
||||||
|
| `--sample-rate <hz>` | `44100` | Decode target. The WAV reader resamples if the file is different. |
|
||||||
|
| `--arrangement guitar\|bass` | `guitar` | Picks the open-string MIDI table. |
|
||||||
|
| `--string-count <n>` | `6` | Used by the string-fret → MIDI math. |
|
||||||
|
| `--av-offset-ms <ms>` | `0` | Same semantics as `setAvOffsetMs` — pass the user's main-Settings value when replaying their take. **Use `=` for negatives**: `--av-offset-ms=-100`. |
|
||||||
|
| `--verbose` | off | Logs progress to stderr. |
|
||||||
|
|
||||||
|
## Diagnostic JSON — the bits that matter for iteration
|
||||||
|
|
||||||
|
Schema `note_detect.diagnostic.v1`. Identical output from live (Settings → Download Diagnostic JSON) and harness. Key fields when comparing runs:
|
||||||
|
|
||||||
|
- `summary.hits / misses / accuracy` — top-line score.
|
||||||
|
- `miss_breakdown` — per-category miss bins:
|
||||||
|
- `pure` — detector never reported a confident matching pitch in the note's time window. Usually a detector or buffer issue.
|
||||||
|
- `chordPartial` — chord saw some strings but missed the per-string ratio.
|
||||||
|
- `early / late` — pitch was right but timing landed outside the inner hit threshold.
|
||||||
|
- `sharp / flat` — pitch was outside the pitch hit threshold (but inside the outer tolerance, otherwise it'd be `pure`).
|
||||||
|
- `timing_error_ms` — distribution over **all matched judgments**. Pinned near a constant when av-offset is wrong (matcher snaps to nearest chart note); use for diagnostics only, *not* as a calibration signal.
|
||||||
|
- `timing_error_ms_hits` — distribution over **only hits**. Responds linearly to av-offset. The A/V auto-calibrate feature keys off this.
|
||||||
|
- `pitch_error_cents` — same shape as timing but for pitch.
|
||||||
|
- `events[]` — per-judgment log (capped). Each entry: `{t, at, s, f, sus, hit, chord, ts, ps, te, pe, ex, dx, cnf, tf}`. The `cnf` field is the pitch-detection confidence at match time; `dx` is the detected MIDI; `ex` is the expected MIDI.
|
||||||
|
|
||||||
|
## A/V auto-calibrate — the iterative pattern
|
||||||
|
|
||||||
|
Settings → Note Detection → "A/V Sync — Auto-Calibrate" surfaces a button that reads `timing_error_ms_hits.median` and applies `setAvOffsetMs(currentOffset − median)`. Expected workflow:
|
||||||
|
|
||||||
|
1. If your current A/V offset is wildly off and you're getting almost no hits, **reset the main Settings A/V slider to 0 first**. The matcher snaps to wrong chart notes when offset is far off, which makes `te-hits` an unreliable signal.
|
||||||
|
2. Play a section with Detect on until you see at least 5 hits on the counter.
|
||||||
|
3. Click **Apply** — it sets the new offset and clears the timing samples so the next reading reflects only the new regime.
|
||||||
|
4. Play another section. Apply again. Usually converges in 2–3 rounds; the button greys out as "Already within 20 ms" when there's nothing useful left to suggest.
|
||||||
|
|
||||||
|
Crucially: **don't trust the suggestion at low hit counts.** Hits at a far-off offset come from coincidental near-matches to wrong chart notes, and their median is noise. The button gates on `n ≥ 5` but for noisy players a higher manual threshold is wise.
|
||||||
|
|
||||||
|
## Common pitfalls
|
||||||
|
|
||||||
|
- **Playback speed must be 1.0× during recording.** The recording captures audio at whatever pace it actually played, but the chart times are absolute. A half-speed take produces all-miss output because every chart event fires its match window before the audio has reached that note. Always confirm the speed slider before pressing Play.
|
||||||
|
- **Don't categorize event-log entries by `event.sus`.** `checkMisses` historically passed only `{s, f}` into miss judgments, so every pure-missed sustained note showed up as `sus=0` in the event log. The bug is fixed (full chart-note flows through now) but old recordings on older builds will mislead you. The reliable answer is to join event entries back against the source chart by `(t, s, f)` and read `sus` from there.
|
||||||
|
- **All-matched `timing_error_ms.median` is *not* a calibration signal.** When A/V offset is wrong, the matcher matches the user's pluck against whatever chart note is closest in time, not the intended one. The resulting te median is pinned near a constant regardless of the offset value. Always use `timing_error_ms_hits.median` for calibration math.
|
||||||
|
- **At a very wrong A/V offset, the auto-calibrate suggestion can point further wrong.** When few hits land, their te median is a property of which wrong chart notes happened to be reachable, not of the user's real skew. Start near zero or near a known reasonable value if you suspect the offset is far off.
|
||||||
|
- **Sweeping parameter X won't fix a problem that lives outside X.** If pure misses dominate at the default config and stay pinned across a 4× range of frame sizes or pitch tolerances, the bottleneck is not those parameters — likely the detector algorithm, the chord scorer, or the matching window logic. Recognise the ceiling and pivot to code changes.
|
||||||
|
- **Match the recording's sample rate when scoring chords.** The chord scorer is fully self-contained (its own FFT, not `AnalyserNode`) and runs in the harness identically to the browser path. But the harness defaults to `--sample-rate 44100` while most modern USB interfaces capture at 48000 — passing the WAV at the wrong rate resamples it linearly, which smears the FFT bins enough to swing chord-hit counts by 1–2 per take. Cross-validated against one contributor's 48 kHz recording, harness at `--sample-rate 48000 --frame-size 2048` reproduces his live chord-hit count within ±1 (9/16 vs his 10/16). Single-note scoring is less sensitive to this and the default sample rate is usually fine.
|
||||||
|
- **Bumping the latency-offset default doesn't generalise.** The right latency comp is heavily audio-chain-dependent (USB interface vs. on-board, ScriptProcessor buffering, OS audio path). A value that's perfect for one user over-corrects for another — bumping the default to match the best-tuned user we had data for regressed two of four fixtures. Leave latency at the conservative default and rely on the A/V auto-calibrate panel + the user-facing slider to dial it in per-chain.
|
||||||
|
|
||||||
|
## Recipes
|
||||||
|
|
||||||
|
### Live judgment streaming — watching a session in flight
|
||||||
|
|
||||||
|
When tuning mode is on, the plugin POSTs each judgment to `POST /api/plugins/note_detect/live-judgment` as it's produced. Backend appends one JSON line to `static/note_detect_recordings/live_<sessionId>.jsonl`. A fresh session id is minted on every `song:play`, so each take produces its own file paired with the recorded WAV (when arming) by timestamp.
|
||||||
|
|
||||||
|
The file is human-readable and updates while the song plays. Tail it with `Get-Content -Wait` on Windows or `tail -f` on macOS/Linux:
|
||||||
|
|
||||||
|
```jsonl
|
||||||
|
{"t":5.333,"s":0,"f":0,"hit":true,"ts":"OK","te":12,"pe":3,"cnf":0.94}
|
||||||
|
{"t":6.000,"s":1,"f":0,"hit":false,"ts":"EARLY","te":-180,"cnf":0.71}
|
||||||
|
{"t":6.667,"s":2,"f":0,"hit":true,"ts":"OK","te":-20,"pe":8}
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the lowest-friction way to share a session with a collaborator: they don't need to wait for the song to end, you don't need to upload anything — the file lives in the bind-mounted `static/` tree, so any host-side process can read it during play.
|
||||||
|
|
||||||
|
Limitations:
|
||||||
|
- Streaming is fire-and-forget; the POSTs don't block detection. A request failure is silently swallowed so the in-memory diagnostic stays the source of truth.
|
||||||
|
- File cap is 8 MB per session (a 3-minute song produces ~60 KB, so this is 100× headroom). Beyond the cap the route returns 413 and the in-memory log keeps growing.
|
||||||
|
- Disabled outside tuning mode — normal users pay no overhead.
|
||||||
|
|
||||||
|
### "Did my detector change improve things?" — the regression suite
|
||||||
|
|
||||||
|
For a single fixture, two ad-hoc harness runs work (see below). For real iteration where you want **all** your fixtures measured against a stored baseline, use the regression driver in the plugin:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd plugins/note_detect
|
||||||
|
|
||||||
|
# One-time: copy the example, edit paths to point at your recordings.
|
||||||
|
cp tools/regression-fixtures.example.json tools/regression-fixtures.json
|
||||||
|
|
||||||
|
# Capture a baseline (do this BEFORE making any code changes).
|
||||||
|
npm run regression:update
|
||||||
|
|
||||||
|
# ...make detector changes...
|
||||||
|
|
||||||
|
# Re-measure against the baseline. Exit code 1 if any fixture regresses.
|
||||||
|
npm run regression:vs-baseline
|
||||||
|
```
|
||||||
|
|
||||||
|
The driver iterates each fixture, runs `harness.js`, and prints a table of `hits/total · pure · chordPartial · Δhits-vs-baseline`. Both the fixtures file and the baseline are gitignored — they reference your local recordings, which aren't portable across contributors. Commit them in your fork if you want CI, otherwise treat them as local state.
|
||||||
|
|
||||||
|
The same workflow works on any tuning change — A/V offset sweep, frame-size sweep, algorithm experiments. Just make sure the baseline was captured *before* the change you want to measure.
|
||||||
|
|
||||||
|
### "Did my detector change improve things?" — ad hoc
|
||||||
|
|
||||||
|
Same recording, same chart, two harness runs. Recipe assumes you're at the slopsmith repo root *and* that the Note Detection plugin is cloned at `plugins/note_detect/` per the README. The detector source lives in that nested plugin repo, which slopsmith's `.gitignore` excludes via `plugins/*/`, so the stash dance has to run **inside** the plugin repo — `git stash` from the slopsmith root would either bail out or, worse, stash unrelated slopsmith edits.
|
||||||
|
|
||||||
|
The stash dance below uses **`git stash push -u -m "..."`** to give the stash a known name *and* include untracked files. `-u` matters: if your detector change added a new module or fixture, an untracked-file-blind stash would leave it on disk during the "before" run and contaminate the baseline. The script then asserts a stash was actually created before popping (so a clean worktree doesn't silently pop someone else's WIP), wraps each step in **`set -euo pipefail`** so a failed `git stash pop` (e.g., conflict) aborts before the "after" harness records an invalid result, and uses `trap` to surface any failure with a clear message.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
set -euo pipefail
|
||||||
|
PLUGIN_DIR=plugins/note_detect
|
||||||
|
HARNESS=$PLUGIN_DIR/tools/harness.js
|
||||||
|
STASH_MSG="harness-before-$$"
|
||||||
|
trap 'echo "harness recipe aborted — stash may still be in $PLUGIN_DIR (\"git -C $PLUGIN_DIR stash list\")" >&2' ERR
|
||||||
|
# Stash the detector edits inside the plugin repo, not the slopsmith root.
|
||||||
|
# -u also stashes untracked files (new modules, fixtures) so they don't
|
||||||
|
# leak into the "before" baseline. `|| true` only swallows the
|
||||||
|
# clean-worktree case, which the next line catches explicitly.
|
||||||
|
git -C "$PLUGIN_DIR" stash push -u -m "$STASH_MSG" || true
|
||||||
|
# Bail out cleanly if nothing was stashed — running the "before" against
|
||||||
|
# the same code as "after" would just produce identical numbers.
|
||||||
|
git -C "$PLUGIN_DIR" stash list | grep -q "$STASH_MSG" || { echo "no detector changes to stash in $PLUGIN_DIR — try again with edits in place"; exit 1; }
|
||||||
|
node $HARNESS --audio <wav> --chart <json> --out /tmp/before.json
|
||||||
|
# `stash pop` failures (e.g., conflicts that auto-merge can't resolve)
|
||||||
|
# now abort via set -e instead of silently rolling into the "after" run
|
||||||
|
# with a half-restored tree.
|
||||||
|
git -C "$PLUGIN_DIR" stash pop "$(git -C "$PLUGIN_DIR" stash list | grep "$STASH_MSG" | head -1 | cut -d: -f1)"
|
||||||
|
node $HARNESS --audio <wav> --chart <json> --out /tmp/after.json
|
||||||
|
node -e "
|
||||||
|
const fs = require('fs');
|
||||||
|
for (const [n, p] of [['before','/tmp/before.json'],['after','/tmp/after.json']]) {
|
||||||
|
const d = JSON.parse(fs.readFileSync(p,'utf8'));
|
||||||
|
console.log(n, d.summary, d.miss_breakdown);
|
||||||
|
}
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
If `summary.hits` went up *and* no miss-bin went up by more than ~1, ship it. If hits went up but `sharp/flat` went up too, you traded pure misses for pitch misses — investigate whether the tolerance shift makes sense.
|
||||||
|
|
||||||
|
### "Find the optimal A/V offset for this take"
|
||||||
|
|
||||||
|
Sweep:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
HARNESS=plugins/note_detect/tools/harness.js
|
||||||
|
for AV in -100 -50 0 50 100 150 200; do
|
||||||
|
echo "=== av=$AV ==="
|
||||||
|
node $HARNESS --audio <wav> --chart <json> --av-offset-ms=$AV --out /tmp/sw_$AV.json | tail -1
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
Pick the highest hit count, then narrow in with finer steps. Cross-reference with `timing_error_ms_hits.median` — at the optimum it'll be close to zero.
|
||||||
|
|
||||||
|
### "Categorize misses by chart section"
|
||||||
|
|
||||||
|
Join the event log against the chart's `sections[]` to bin per-section hit rate. Useful for finding which exercises in the benchmark sloppak a tuning change improves or regresses.
|
||||||
|
|
||||||
|
### "Why is this specific note pure-missed?"
|
||||||
|
|
||||||
|
Find the note's `t` in the chart, then grep the event log for entries near that time. If `cnf` is 0 for every nearby event, the detector never fired confidently — likely a YIN buffer / confidence issue. If `cnf > 0` but `dx` doesn't match `ex`, pitch detection is firing on a different note (octave error, harmonic, neighbour string).
|
||||||
|
|
||||||
|
## Reference
|
||||||
|
|
||||||
|
The Note Detection plugin lives in its own repository — these links go to the canonical source at github.com. If you've cloned the plugin into a local `plugins/note_detect/` next to this repo, the same files are at the equivalent path on disk.
|
||||||
|
|
||||||
|
- Plugin source: [`screen.js`](https://github.com/byrongamatos/slopsmith-plugin-notedetect/blob/main/screen.js) — `matchNotes`, `checkMisses`, `_diagTimingErrors` / `_diagTimingErrorsHits`, `getDiagnostic`.
|
||||||
|
- Routes: [`routes.py`](https://github.com/byrongamatos/slopsmith-plugin-notedetect/blob/main/routes.py) — the `/api/plugins/note_detect/recording` and `/api/plugins/note_detect/live-judgment` endpoints.
|
||||||
|
- Harness: [`tools/harness.js`](https://github.com/byrongamatos/slopsmith-plugin-notedetect/blob/main/tools/harness.js).
|
||||||
|
- Regression driver: [`tools/regression.js`](https://github.com/byrongamatos/slopsmith-plugin-notedetect/blob/main/tools/regression.js).
|
||||||
|
- Benchmark builder: [docs/benchmarks/note_detect_v1/build_benchmark.py](benchmarks/note_detect_v1/build_benchmark.py).
|
||||||
|
- Settings UI: [`settings.html`](https://github.com/byrongamatos/slopsmith-plugin-notedetect/blob/main/settings.html) — A/V auto-calibrate panel, tuning-mode toggle, diagnostic block.
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 75 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 212 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.9 MiB |
@@ -0,0 +1,257 @@
|
|||||||
|
# Plugin Capability Inventory
|
||||||
|
|
||||||
|
This report inventories the currently included plugins staged in `plugins/` and maps their observed behavior to Slopsmith capability domains. It is intended to inform the capability roadmap and the next migration specs now that PR1, the audio graph/session slice, playback, and audio-effects are active capability surfaces.
|
||||||
|
|
||||||
|
## Scope And Method
|
||||||
|
|
||||||
|
- Inventory source: 41 `plugin.json` manifests under `plugins/`, plus the first-party plugin repos that were migrated during the 008/playback and audio-effects work (`nam_tone`, `rig_builder`, Remote Library Client, and Remote Library Server).
|
||||||
|
- Verification pass: the original bundled-plugin scan found 25 plugins with backend `routes.py` and 14 plugins with `settings.html`. First-party plugin repos outside `plugins/` were checked separately from their current manifests and handoff docs.
|
||||||
|
- Most bundled plugin entries below are still inferred/recommended declarations. Current first-party manifests now declare active capability intent for `diagnostics`, `pipeline`, `library`, `audio-mix`, `audio-input`, `audio-monitoring`, `stems`, `playback`, `audio-effects`, `jobs`, and privileged capability inventory surfaces where their repos have already migrated.
|
||||||
|
- Manifest fields such as `nav`, `screen`, `settings`, `routes`, and `type: "visualization"` were treated as high-confidence evidence.
|
||||||
|
- Code patterns such as `window.slopsmithViz_*`, `window.playSong` wrappers, `window.showScreen` wrappers, `window.registerShortcut`, `window.slopsmithTour.register`, `window.slopsmith.audio.registerFader`, `highway.setNoteStateProvider`, and route/WebSocket handlers were treated as behavior evidence.
|
||||||
|
|
||||||
|
## Roadmap Baseline
|
||||||
|
|
||||||
|
The current roadmap already covers several surfaces implied by the plugin set:
|
||||||
|
|
||||||
|
| Roadmap State | Domains |
|
||||||
|
|---------------|---------|
|
||||||
|
| Active PR1 domains | `pipeline`, `diagnostics`, `library` |
|
||||||
|
| Active audio graph/session slice | `audio-mix`, `audio-input`, `audio-monitoring`, `stems` |
|
||||||
|
| Active playback slice | `playback` |
|
||||||
|
| Active audio-effects slice | `audio-effects` |
|
||||||
|
| Planned UI domains | `ui.navigation`, `ui.plugin-screens`, `ui.player-controls`, `ui.player-panels`, `ui.player-overlays`, `settings` |
|
||||||
|
| Planned player/runtime domains | `midi-control`, `tempo-clock` (`visualization` active as of the cap:6 slice; `note-detection` active as of the spec-009 slice) |
|
||||||
|
| Planned privileged domains | `backend.routes`, `plugins`, `jobs` |
|
||||||
|
|
||||||
|
The plugin inventory confirms these planned domains are directionally right. The main gaps are additional domain names or command scopes for library card injection, guided tours, keyboard shortcuts, media import/export, recording/capture, practice/session scoring, external services, and collaboration. `audio-effects` is no longer a missing candidate: it is an active provider-coordinator domain, with remaining work concentrated in provider adoption, bridge removal, and executor coverage.
|
||||||
|
|
||||||
|
## Executive Summary
|
||||||
|
|
||||||
|
- The most common plugin surface is a plugin screen backed by optional routes and settings. `ui.plugin-screens`, `backend.routes`, and `settings` should be treated as first-class migration targets.
|
||||||
|
- Player integrations are still heavily legacy-global today. `playback` now provides the active transport lifecycle/control plane, and `visualization` now provides the renderer provider/selection contract (legacy globals ride accounted shims), while `ui.player-overlays`, `ui.player-controls`, and `ui.player-panels` still need lifecycle and ordering contracts before wrappers can be retired.
|
||||||
|
- The audio domains promoted by the audio graph/session slice match real plugin behavior. `audio-mix`, `audio-input`, `audio-monitoring`, and `stems` are active; remaining work is mostly native provider adoption, bridge-hit cleanup, and cross-domain smoke coverage.
|
||||||
|
- `audio-effects` is active. Core owns provider/executor selection, route state, mapping index, constrained chain-plan validation, route gain/release, segment activation, stage bypass/parameter dispatch, fallback accounting, and redaction-safe diagnostics. Desktop owns trusted native execution. NAM Tone provides the baseline NAM+IR provider/executor path, while Rig Builder is the high-priority full-chain provider for NAM/VST/IR routes.
|
||||||
|
- `playback` is active as of the 008 slice, with redaction-safe target/settings keys and transport lifecycle diagnostics. Legacy `window.playSong` and song-event wrappers remain compatibility paths while plugins migrate requester/observer workflows.
|
||||||
|
- Long-running work is spread across conversion, update, import, preview, and studio plugins. The `jobs` domain should include progress, cancellation, terminal failure, and provider attribution.
|
||||||
|
- Several plugins perform privileged or externally mediated work: subprocesses, downloads, native audio bridges, plugin updates, and media conversion. These should stay out of broad capability activation until each surface has user confirmation, diagnostics redaction, and failure recovery.
|
||||||
|
|
||||||
|
## Current Manifest Declarations Since The First Pass
|
||||||
|
|
||||||
|
| Plugin Or Runtime | Current Declarations | Notes |
|
||||||
|
|-------------------|----------------------|-------|
|
||||||
|
| Core capability runtime | `pipeline`, `diagnostics`, `library`, `audio-mix`, `audio-input`, `audio-monitoring`, `stems`, `playback`, `audio-effects` | Active runtime owners/coordinators are registered by core, not by plugin manifests. |
|
||||||
|
| `capability_inspector` | `diagnostics`, `pipeline` | Bundled support surface for the graph and diagnostics snapshots. |
|
||||||
|
| `remote_library_client` | `library` provider | Declares provider operations for query, art, stats, tuning names, and sync. |
|
||||||
|
| `remote_library_server` | `library` requester/observer | Wraps the local provider for direct remote clients without claiming provider ownership. |
|
||||||
|
| `nam_tone` | `stems`, `audio-mix`, `audio-input`, `audio-monitoring`, `audio-effects`, `playback` | Baseline NAM+IR provider/executor and live guitar requester/observer path. |
|
||||||
|
| `rig_builder` | `library`, `audio-effects`, `playback`, `jobs`, privileged inventory, UI contributions | High-priority full-chain provider; library migration is complete for the Songs tab, while privileged routes and long-running work still need host-backed execution slices. |
|
||||||
|
| Desktop native executor | `audio-effects` executor | Trusted native execution for the `desktop-main` route: load/clear chain, activate segments, set stage bypass/parameters, set route gain, start audio, and release/mute routes. It does not own provider semantics. |
|
||||||
|
|
||||||
|
## Per-Plugin Mapping
|
||||||
|
|
||||||
|
| Plugin | Recommended Domains | Expected Roles | Roadmap Status | Confidence | Evidence |
|
||||||
|
|--------|---------------------|----------------|----------------|------------|----------|
|
||||||
|
| `app_tour_library` | `tours`, `ui.player-overlays` | provider, observer | Missing `tours`; overlay planned | High | Tour registration and screen-change observation. |
|
||||||
|
| `app_tour_settings` | `tours`, `ui.player-overlays` | provider, observer | Missing `tours`; overlay planned | High | Tour registration and screen-change observation. |
|
||||||
|
| `audio_engine` | `audio-monitoring`, `audio-effects`, `ui.plugin-screens`, `settings` | provider, requester | Audio monitoring/effects active; plugin adoption pending | Medium | Screen/settings surfaces and native/VST audio engine intent. |
|
||||||
|
| `drum_highway_3d` | `visualization`, `ui.player-overlays`, `settings`, `midi-control` | visualization provider, observer | Planned | High | `type: "visualization"`, WebGL renderer, settings surface, drum/MIDI use case. |
|
||||||
|
| `drums` | `visualization` | visualization provider | Planned | High | `type: "visualization"` and renderer script. |
|
||||||
|
| `editor` | `ui.plugin-screens`, `backend.routes`, `media-import-export`, `jobs` | screen provider, route provider, job provider | UI/routes/jobs planned; media domain missing | High | Screen plus backend routes for editing/import/export workflows. |
|
||||||
|
| `find_more` | `library`, `ui.plugin-screens`, `backend.routes`, `ui.library-card-injection`, `external-services` | requester/provider, route provider | Library active; card injection/external services missing | High | Screen/routes plus library discovery and card injection behavior. |
|
||||||
|
| `flappy_bend` | `ui.plugin-screens`, `backend.routes`, `practice-session` | screen provider, route provider | UI/routes planned; practice-session missing | Medium | Game screen and backend route surface. |
|
||||||
|
| `fretboard` | `ui.player-overlays` | overlay provider | Planned | High | Highway-state overlay pattern. |
|
||||||
|
| `guitar_theory` | `ui.plugin-screens`, `settings`, `practice-session` | screen provider, settings provider | UI/settings planned; practice-session missing | Medium | Screen/settings manifest surfaces for theory tools. |
|
||||||
|
| `highway_3d` | `visualization`, `ui.player-overlays`, `backend.routes`, `settings`, `audio-monitoring` | visualization provider, route provider, observer | Audio-monitoring active; visualization/UI/routes planned | High | `type: "visualization"`, WebGL renderer, routes/settings, analyser monitoring bridge. |
|
||||||
|
| `invert_highway` | `ui.player-overlays`, `settings`, `visualization` | overlay provider, observer | Planned | High | Settings surface and highway/playback wrapper behavior. |
|
||||||
|
| `jumpingtab` | `visualization`, `ui.player-overlays` | visualization provider, observer | Planned | High | `type: "visualization"`, renderer factory, highway visibility behavior. |
|
||||||
|
| `lyrics_karaoke` | `ui.plugin-screens`, `backend.routes`, `playback`, `recording` | screen provider, route provider, observer | Playback active; UI/routes planned; recording missing | High | Screen/routes plus karaoke timing and lyric/audio workflows. |
|
||||||
|
| `metronome` | `ui.player-overlays`, `audio-mix`, `playback`, `tempo-clock` | overlay provider, audio participant, observer | Audio-mix/playback active; overlay/tempo planned | High | Player overlay behavior, metronome audio, playback coupling. |
|
||||||
|
| `midi_amp` | `midi-control`, `ui.plugin-screens`, `backend.routes`, `settings`, `audio-effects` | MIDI provider, screen provider, route provider | MIDI/UI/routes planned; audio-effects active bridge target | High | Manifest id `midi_amp`, routes, settings, and MIDI amp workflow. |
|
||||||
|
| `minigames` | `ui.plugin-screens`, `backend.routes`, `settings`, `diagnostics`, `practice-session` | screen provider, route provider, diagnostics provider | Mostly planned/active; practice-session missing | High | Routes/settings, diagnostics files, minigame state. |
|
||||||
|
| `multiplayer` | `collaboration`, `ui.plugin-screens`, `backend.routes`, `playback`, `audio-mix` | collaboration provider, route provider, observer | Collaboration missing | Medium | Screen/routes and real-time multiplayer/audio mix behavior. |
|
||||||
|
| `rig_builder` | `library`, `playback`, `audio-effects`, `jobs`, `privileged-capabilities`, `ui.plugin-screens`, `backend.routes`, `media-import-export` | effects provider/requester/observer, screen provider, route provider | First-party manifest active; backend/jobs/UI hosts still incomplete | High | Full-chain NAM/VST/IR provider, core mapping writes, library provider routing, mega-chain playback, tone3000/import/export routes. |
|
||||||
|
| `nam_tone` | `audio-mix`, `audio-input`, `audio-monitoring`, `stems`, `audio-effects`, `playback`, `ui.plugin-screens`, `backend.routes`, `settings` | audio provider/requester/observer, effects provider/executor, screen provider, route provider | First-party manifest active; UI/routes/settings planned | High | Fader registration, input/monitoring graph, stem ducking, baseline NAM+IR provider/executor, model/IR routes/settings. |
|
||||||
|
| `note_detect` | `note-detection`, `audio-input`, `audio-monitoring`, `ui.player-overlays`, `backend.routes`, `settings`, `diagnostics` | note provider, audio requester, overlay provider | Audio input/monitoring active; note/UI/routes planned | High | `highway.setNoteStateProvider`, calibration/settings/routes, diagnostic workflow. |
|
||||||
|
| `piano` | `visualization` | visualization provider | Planned | High | `type: "visualization"` and renderer script. |
|
||||||
|
| `plugin_manager` | `plugins`, `ui.plugin-screens` | plugin lifecycle provider, screen provider | Planned | High | Plugin management screen and desktop bridge integration. |
|
||||||
|
| `practice_journal` | `practice-session`, `ui.plugin-screens`, `backend.routes` | practice provider, screen provider, route provider | Practice-session missing | High | Practice journal screen/routes. |
|
||||||
|
| `profileimport` | `media-import-export`, `ui.plugin-screens`, `backend.routes`, `jobs` | import provider, screen provider, route provider | Media domain missing; jobs planned | High | Profile import screen/routes. |
|
||||||
|
| `section_map` | `ui.player-overlays`, `playback` | overlay provider, observer | Planned | High | Highway section overlay behavior. |
|
||||||
|
| `setlist` | `library`, `playback`, `ui.plugin-screens`, `backend.routes` | requester/provider, screen provider, route provider | Library/playback active; UI/routes planned | High | Setlist screen/routes and song selection/playback workflow. |
|
||||||
|
| `sloppak_converter` | `media-import-export`, `jobs`, `library`, `ui.plugin-screens`, `backend.routes`, `ui.library-card-injection` | conversion provider, job provider, route provider | Library active; jobs/UI/routes planned; media/card missing | High | Converter routes, queue UI, library card actions, conversion jobs. |
|
||||||
|
| `slopscale` | `ui.plugin-screens`, `backend.routes`, `settings`, `visualization` | screen provider, route provider, observer | Planned | High | Routes/settings and 3D highway visualization observation. |
|
||||||
|
| `song_preview` | `playback`, `audio-mix`, `ui.plugin-screens`, `backend.routes`, `settings` | preview provider, route provider, audio participant | Playback/audio-mix active; UI/routes planned | Medium | Preview screen/routes/settings and audio preview behavior. |
|
||||||
|
| `splitscreen` | `ui.player-panels`, `ui.player-overlays`, `visualization`, `playback`, `keyboard-shortcuts`, `settings` | panel provider, observer, shortcut provider | Playback active; UI/visualization planned; shortcuts missing | High | Multi-highway panels, playback/screen wrappers, panel shortcuts/settings. |
|
||||||
|
| `stem_mixer` | `stems`, `audio-mix`, `ui.plugin-screens`, `backend.routes`, `settings`, `jobs` | stem provider, mixer provider, route provider | Audio active; jobs planned | High | Stems mixer routes/settings and stem/audio mix ownership. |
|
||||||
|
| `step_mode` | `ui.player-overlays`, `playback`, `settings`, `keyboard-shortcuts` | overlay provider, observer, shortcut provider | Shortcuts missing | Medium | Player overlay/settings and step-practice behavior. |
|
||||||
|
| `studio` | `audio-mix`, `audio-input`, `audio-monitoring`, `recording`, `media-import-export`, `jobs`, `ui.plugin-screens`, `backend.routes` | DAW provider, route provider, job provider | Audio active; jobs/UI/routes planned; recording/media missing | Medium | Studio screen/routes, multitrack recording/mixing workflows. |
|
||||||
|
| `tab_import` | `media-import-export`, `ui.plugin-screens`, `backend.routes`, `jobs` | import provider, route provider, job provider | Media missing; jobs planned | High | Tab import screen/routes. |
|
||||||
|
| `tabview` | `visualization`, `backend.routes` | visualization provider, route provider | Planned | High | `type: "visualization"` and backend tab routes. |
|
||||||
|
| `themes` | `settings`, `ui.theme` | theme provider | Settings planned; theme domain missing | Medium | Global settings/routes for theming. |
|
||||||
|
| `tones` | `audio-effects`, `playback`, `ui.plugin-screens`, `backend.routes` | tone provider, playback observer, route provider | Audio-effects/playback active; provider adoption pending | High | Tone screen/routes and playback wrapper behavior. |
|
||||||
|
| `transpose-chords` | `ui.player-overlays`, `visualization`, `playback` | overlay provider, highway observer | Planned | High | Chord/highway reader and playback wrapper behavior. |
|
||||||
|
| `tutorials` | `tours`, `ui.plugin-screens`, `backend.routes`, `settings` | tutorial provider, route provider | Tours missing | Medium | Tutorial screen/routes/settings and guided content. |
|
||||||
|
| `update_manager` | `plugins`, `jobs`, `ui.plugin-screens`, `backend.routes`, `external-services` | update provider, job provider, route provider | Plugins/jobs planned; external services missing | Medium | Update screen/routes and desktop/network integration. |
|
||||||
|
|
||||||
|
## Domain Coverage Summary
|
||||||
|
|
||||||
|
| Domain | Approximate Plugin Count | Roadmap Fit | Notes |
|
||||||
|
|--------|--------------------------|-------------|-------|
|
||||||
|
| `ui.plugin-screens` | 24 | Planned | Main extension surface; should include screen lifecycle, visibility, focus, and teardown. |
|
||||||
|
| `backend.routes` | 25 route files | Planned privileged | Needs route diagnostics, plugin attribution, and privilege review. |
|
||||||
|
| `settings` | 14 | Planned sensitive | Should cover contribution metadata and backup/import allowlists without exposing values. |
|
||||||
|
| `visualization` | 6 declared providers plus observers | Planned | Existing renderer factory contract is mature enough to formalize. |
|
||||||
|
| `ui.player-overlays` | 14 | Planned | Needs overlay placement, visibility, z-order, and coexistence policy. |
|
||||||
|
| `audio-mix` | 6+ | Active | Runtime control plane exists; migration work is native participant coverage and legacy fader bridge removal gates. |
|
||||||
|
| `audio-input` | 4+ | Active | Needs broader provider coverage across browser, Desktop, and native paths. |
|
||||||
|
| `audio-monitoring` | 5+ | Active | Needs broader provider coverage and cross-domain failure smoke tests. |
|
||||||
|
| `stems` | 3 | Active coordinated provider | Current coordinator/provider split matches plugin ownership. |
|
||||||
|
| `library` | 3+ | Active | Needs to account for library card actions separately from browsable providers. |
|
||||||
|
| `jobs` | 7+ | Planned privileged | Conversion/import/update/studio work all need a common job model. |
|
||||||
|
| `playback` | 9+ | Active | Wrapper chains should migrate to transport commands, requester/observer declarations, and lifecycle events. |
|
||||||
|
| `note-detection` | 1 | Planned sensitive | Current provider is high-impact enough for a focused spec. |
|
||||||
|
| `midi-control` | 2 | Planned sensitive | Needs consent, device redaction, and mapping diagnostics. |
|
||||||
|
| `tempo-clock` | 1+ | Planned | Metronome and practice tools imply clock source/consumer semantics. |
|
||||||
|
| `plugins` | 2 | Planned privileged | Plugin manager/update manager require confirmation and rollback. |
|
||||||
|
| `diagnostics` | 2+ | Active | Existing diagnostics contributions should become easier to inspect by domain. |
|
||||||
|
| `ui.library-card-injection` | 2+ | Missing | Library card buttons/actions are distinct from library source providers. |
|
||||||
|
| `tours` | 4 | Missing | Guided tours behave like UI overlays with screen navigation coupling. |
|
||||||
|
| `keyboard-shortcuts` | 2+ | Missing | Existing global shortcut registry needs contribution and conflict policy. |
|
||||||
|
| `media-import-export` | 6+ | Missing | Import/export/conversion is broader than `jobs` and often uses privileged backend routes. |
|
||||||
|
| `recording` | 2+ | Missing | Studio and karaoke workflows need capture/session semantics. |
|
||||||
|
| `audio-effects` | 5+ | Active | Core host, mapping index, provider/executor registry, route gain/release, segment/stage controls, fallback, and diagnostics exist; remaining work is provider adoption and bridge removal. |
|
||||||
|
| `practice-session` | 4+ | Missing | Practice journal, minigames, theory, and note detection imply scoring/progression state. |
|
||||||
|
| `collaboration` | 1 | Missing | Multiplayer needs its own trust, identity, and sync model. |
|
||||||
|
| `external-services` | 3+ | Missing or safety inventory | Network/download/subprocess integrations may be better tracked as safety metadata than as one capability. |
|
||||||
|
|
||||||
|
## Operation And Event Gaps
|
||||||
|
|
||||||
|
### High Priority
|
||||||
|
|
||||||
|
| Domain | Missing Or Under-Specified Surface |
|
||||||
|
|--------|------------------------------------|
|
||||||
|
| `playback` | Migrate legacy wrapper users onto active `start`, `pause`, `resume`, `stop`, `seek`, `set-loop`, `clear-loop`, and `inspect` commands plus playback lifecycle events. Keep compatibility bridge hits visible until normal playback/loop/route smoke runs are clean. |
|
||||||
|
| `audio-effects` | Finish provider adoption: NAM Tone remains the fallback provider/executor and owns the legacy player `Chain` control; Rig Builder owns full-chain routes while enabled/pending/failed/active; Desktop owns trusted native execution only. Remaining gaps are host-backed active mapping read paths, replacement of old fetch/native-load bridges, additional provider declarations for `tones`/`midi_amp`/`audio_engine`, and zero-leak diagnostics around asset refs. |
|
||||||
|
| `jobs` | Add `register-provider`, `enqueue`, `list`, `inspect`, `cancel`, `pause`, `resume`, and `retry`. Emit `queued`, `started`, `progress`, `log`, `completed`, `failed`, `cancelled`, and `provider-unavailable`. |
|
||||||
|
| `note-detection` | Add provider registration, active provider selection, note-state provider lifecycle, input binding, hit/miss/state events, calibration diagnostics, and performance-data redaction. |
|
||||||
|
|
||||||
|
### Medium Priority
|
||||||
|
|
||||||
|
| Domain | Missing Or Under-Specified Surface |
|
||||||
|
|--------|------------------------------------|
|
||||||
|
| `ui.plugin-screens` | Define contribution registration, mount/unmount, visibility, focus, navigation, teardown, and rehydration policy. |
|
||||||
|
| `ui.player-overlays` | Define surface, anchor, z-order, visibility, teardown, hit-testing, and renderer compatibility flags. |
|
||||||
|
| `ui.player-panels` | Define panel registration, per-panel renderer state, focus, shortcut scope, layout constraints, and teardown. |
|
||||||
|
| `ui.player-controls` | Define ordered contribution regions, command buttons, popovers, sliders, disabled states, and conflict policy. |
|
||||||
|
| `visualization` | Formalize provider registration, `contextType`, `matchesArrangement`, `panelControls`, per-panel selection, fallback/revert events, and renderer failure diagnostics. |
|
||||||
|
| `media-import-export` | Add import/export job requests, accepted file types, source trust metadata, generated artifact paths, and cleanup/failure semantics. |
|
||||||
|
| `audio-input` | Broaden native/browser provider adoption and test denied/unavailable/degraded/failure outcomes without raw device labels or hardware ids. |
|
||||||
|
| `audio-monitoring` | Broaden native/browser provider adoption, direct-monitor policy coverage, startup barrier accounting, and safe input-level summaries. |
|
||||||
|
|
||||||
|
### Lower Priority Or Cross-Cutting
|
||||||
|
|
||||||
|
| Domain | Missing Or Under-Specified Surface |
|
||||||
|
|--------|------------------------------------|
|
||||||
|
| `settings` | Add settings contribution metadata, export/import participation, settings schema hints, redaction class, and per-plugin backup diagnostics. |
|
||||||
|
| `plugins` | Add install/enable/disable/update commands with user confirmation, rollback, disabled-handler enforcement, and desktop bridge failure recovery. |
|
||||||
|
| `midi-control` | Add device enumerate/open/close, message send/listen, mapping registration, consent, and redacted diagnostics. |
|
||||||
|
| `tempo-clock` | Add tempo provider registration, BPM/time-signature changes, beat events, metronome tick state, and consumer subscription. |
|
||||||
|
| `keyboard-shortcuts` | Add shortcut contribution registration, scope, conflict resolution, enable/disable, and help-panel metadata. |
|
||||||
|
| `tours` | Add tour registration, eligibility, start/stop, step lifecycle, target resolution, and screen navigation dependency declarations. |
|
||||||
|
| `ui.library-card-injection` | Add card action registration, placement, enabled/disabled state, per-provider applicability, and action-result events. |
|
||||||
|
| `recording` | Add arm/start/stop capture, take upload/import, latency metadata, capture-source binding, and storage cleanup. |
|
||||||
|
| `practice-session` | Add session start/stop, goal registration, score/progress events, chart segment focus, and journal persistence boundaries. |
|
||||||
|
| `collaboration` | Add room/session lifecycle, participant identity redaction, shared playback sync, conflict policy, and disconnect recovery. |
|
||||||
|
|
||||||
|
## Audio Domain Notes
|
||||||
|
|
||||||
|
The audio graph/session and audio-effects slices should stay scoped to coordination, dispatch, and redaction-safe diagnostics. This inventory reinforces four follow-up requirements:
|
||||||
|
|
||||||
|
1. `audio-mix` has a control plane, but native provider coverage and duplicate native-over-legacy cleanup still need release gates before the legacy fader registry can become compatibility-only.
|
||||||
|
2. `stems` should remain coordinated by core but owned by the active Stems provider. Stem playback, mute/restore semantics, availability, and per-stem state belong to the provider.
|
||||||
|
3. `audio-input` and `audio-monitoring` should cover both browser and Desktop/native paths without leaking raw device labels, source ids, or capture details in diagnostics.
|
||||||
|
4. `audio-effects` should keep provider semantics out of Desktop native code. Core selects providers/executors and coordinates route lifecycle; providers resolve opaque chain plans and private trusted asset maps; Desktop only executes the validated native requests and must release/mute routes cleanly. Provider-positive UI should follow route ownership: the legacy player `Chain` control belongs to `nam_tone` and should stay hidden while Rig Builder owns tones.
|
||||||
|
|
||||||
|
## Recommended Roadmap Updates
|
||||||
|
|
||||||
|
1. Finish audio-effects provider adoption and bridge removal gates: NAM Tone and Rig Builder should read/write the core mapping index first, route playback through the active provider/executor path, and reserve legacy fetch/native-load/DB bridges for fallback/import only.
|
||||||
|
2. Move long-running and privileged plugin work behind host-owned `jobs` and privileged-operation inventories before migrating converter/import/update/studio/Rig Builder routes. This gives backend work a shared cancellation/progress/error and approval model.
|
||||||
|
3. Promote `note-detection` as its own sensitive provider domain. It touches audio input, monitoring, visualization feedback, calibration, and diagnostics.
|
||||||
|
4. Create a UI contribution host spec that includes `ui.navigation`, `ui.plugin-screens`, `ui.player-controls`, `ui.player-overlays`, `ui.player-panels`, `keyboard-shortcuts`, and possibly `tours`.
|
||||||
|
5. Add missing candidate domains or safety inventories for `media-import-export`, `recording`, `practice-session`, `collaboration`, `ui.library-card-injection`, and `external-services`.
|
||||||
|
6. Revisit active playback consumers and retire wrapper-only tone/overlay/visualizer integrations once requester/observer declarations and compatibility bridge counts show clean normal playback.
|
||||||
|
|
||||||
|
## Suggested Manifest Direction
|
||||||
|
|
||||||
|
When these plugins migrate, manifests should describe intent even before runtime handlers hydrate. For example, a visualization plugin might declare:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"standards": ["capability-pipelines.v1", "plugin-runtime-idempotent.v1"],
|
||||||
|
"capabilities": {
|
||||||
|
"visualization": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["renderer.create", "renderer.destroy", "renderer.inspect"],
|
||||||
|
"emits": ["renderer-ready", "renderer-failed"],
|
||||||
|
"mode": "optional",
|
||||||
|
"compatibility": "legacy-window-shim",
|
||||||
|
"safety": "safe"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
An audio plugin that participates in the active audio-session domains should declare requester/provider relationships more explicitly:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"capabilities": {
|
||||||
|
"audio-mix": {
|
||||||
|
"roles": ["provider"],
|
||||||
|
"operations": ["fader.get-value", "fader.set-value"],
|
||||||
|
"emits": ["fader-value-changed"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "legacy-window-shim",
|
||||||
|
"safety": "safe"
|
||||||
|
},
|
||||||
|
"stems": {
|
||||||
|
"roles": ["requester", "observer"],
|
||||||
|
"requests": ["mute", "restore", "inspect"],
|
||||||
|
"observes": ["owner-available", "automation-applied", "automation-restored", "automation-overridden", "claim-orphaned"],
|
||||||
|
"mode": "active",
|
||||||
|
"compatibility": "none",
|
||||||
|
"safety": "safe"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
For active domains, command and operation names should follow [capability-domains.md](capability-domains.md) and the relevant host module. For deferred domains, these examples remain direction markers rather than current contracts.
|
||||||
|
|
||||||
|
## Highway String Colors (data-plane API)
|
||||||
|
|
||||||
|
User-customizable per-string highway colors (the "Highway String Colors" setting in the 3D Highway plugin's panel) are **not** a capability domain. Consistent with `capability-domains.md` keeping highway-rendering and `visualization` surfaces off the capability graph until a dedicated render-facade slice lands, they are exposed as a synchronous **data-plane** API on `window.slopsmith.highwayColors` plus a change event. Visualization/overlay plugins (custom highways, minigames, fretboard widgets) should read colors from here so their gems/strings match the user's theme.
|
||||||
|
|
||||||
|
Colors are keyed by **named string slot**, not raw index, so a string keeps its color across arrangements (Low E stays Low E's color on a 6-string guitar, 4-string bass, or 7/8-string, where the extra low strings use the `low7`/`low8` slots). Slots: `highE`, `B`, `G`, `D`, `A`, `lowE`, `low7` (7-string Low B), `low8` (8-string Low F#).
|
||||||
|
|
||||||
|
`window.slopsmith.highwayColors` (`version: 1`):
|
||||||
|
|
||||||
|
| Member | Returns | Purpose |
|
||||||
|
|--------|---------|---------|
|
||||||
|
| `slots` | `[{key,label,sub}]` | Ordered named slots (stable `key`s). |
|
||||||
|
| `get()` | `{slot:hex}` | User overrides only (`{}` = pure defaults). |
|
||||||
|
| `getDefaults()` | `{slot:hex}` | Canonical default color per slot. |
|
||||||
|
| `getResolved()` | `{slot:hex}` | Defaults overlaid with overrides — colors in effect by name. |
|
||||||
|
| `keysForChart(sc, isBass)` | `[slotKey…]` | Which slot each chart string index maps to (index 0 = lowest). |
|
||||||
|
| `toEffective(sc?, isBass?)` | `[hex…]` | Per-string-**index** colors for an arrangement (resolved). Omit args for the loaded chart. |
|
||||||
|
| `getCurrent()` | `[hex…]` | Per-index colors actually applied to the live 2D highway right now. |
|
||||||
|
| `apply(slotMap)` | — | Set colors (persists + applies to both highways); `null`/`{}` reverts. |
|
||||||
|
| `encodeShare(name, map)` / `decodeShare(code)` | `string` / `{name,colors}` | The `SLOPHWY2.` copy/paste share format. |
|
||||||
|
| `onChange(fn)` / `offChange(fn)` | unsubscribe fn | `fn(resolvedMap)` fires on any color change (also on song load when the slot→index mapping shifts). |
|
||||||
|
|
||||||
|
The underlying change event is `window.slopsmith.emit('highway:stringColors', …)`; `onChange` wraps it and hands back the resolved map. The raw `window.highway.getStringColors()` data-plane accessor (per-index) remains available for renderers that only need the current applied array. When a `visualization` capability slice eventually lands, this facade is the natural thing to fold into it.
|
||||||
|
|
||||||
|
## Validation Notes
|
||||||
|
|
||||||
|
This report should be revisited after more bundled plugins adopt manifest capability declarations. The document now separates current first-party declarations from inferred legacy behavior; the next revision should turn each active-domain row into a removal-gate checklist with Capability Inspector smoke output, bridge-hit expectations, and support-bundle redaction checks.
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "https://slopsmith.local/contracts/plugin-manifest-capabilities.schema.json",
|
||||||
|
"title": "Slopsmith Plugin Manifest Capability Contract",
|
||||||
|
"type": "object",
|
||||||
|
"required": ["id", "name"],
|
||||||
|
"properties": {
|
||||||
|
"id": { "type": "string", "minLength": 1, "pattern": "^[A-Za-z0-9_.-]+$" },
|
||||||
|
"name": { "type": "string", "minLength": 1 },
|
||||||
|
"version": { "type": ["string", "null"] },
|
||||||
|
"private": { "type": "boolean" },
|
||||||
|
"standards": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"capability_api": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": { "standard": { "const": "capability-pipelines.v1" }, "version": { "const": 1 } },
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"capabilities": {
|
||||||
|
"type": "object",
|
||||||
|
"propertyNames": { "$ref": "#/$defs/domainName" },
|
||||||
|
"additionalProperties": { "$ref": "#/$defs/capabilityDeclaration" }
|
||||||
|
},
|
||||||
|
"ui_contributions": { "type": "object", "propertyNames": { "$ref": "#/$defs/domainName" }, "additionalProperties": { "$ref": "#/$defs/contributionList" } },
|
||||||
|
"ui": { "type": "object", "propertyNames": { "$ref": "#/$defs/domainName" }, "additionalProperties": { "$ref": "#/$defs/contributionList" } },
|
||||||
|
"runtime_domains": { "type": "object", "propertyNames": { "$ref": "#/$defs/domainName" }, "additionalProperties": { "$ref": "#/$defs/domainDeclaration" } },
|
||||||
|
"domains": { "type": "object", "propertyNames": { "$ref": "#/$defs/domainName" }, "additionalProperties": { "$ref": "#/$defs/domainDeclaration" } },
|
||||||
|
"settings_schema": { "type": "object" },
|
||||||
|
"nav": {}, "screen": {}, "script": {}, "routes": {}, "settings": {}, "diagnostics": {}, "type": { "type": "string" }, "tour": {},
|
||||||
|
"description": { "type": "string", "description": "Short one-sentence summary of the plugin, surfaced on the v3 Pedalboard Plugins page (clamped to ~2 lines). Optional and additive." },
|
||||||
|
"category": { "type": "string", "description": "Which pedalboard the plugin sits on in the v3 Plugins page. Suggested values: 'audio', 'creation', 'practice', 'game', 'tools'. Free-form; unknown/absent values fall back to a curated default then 'other'. Optional and additive." },
|
||||||
|
"icon": { "type": "string", "minLength": 1, "pattern": "^assets/(?!.*\\.\\.)[^\\\\?#]+$", "description": "Plugin-root-relative path under assets/ (e.g. 'assets/thumb.png') to a thumbnail (~square, ~256x256 PNG/SVG) shown as the pedal graphic on the v3 Plugins page. Same containment rule as `styles`. If omitted, the loader auto-detects assets/thumb.png; failing that the UI shows a default pedal graphic. Optional and additive." },
|
||||||
|
"styles": { "type": "string", "minLength": 1, "pattern": "^assets/(?!.*\\.\\.)[^\\\\?#]+$", "description": "Plugin-root-relative path under assets/ (e.g. 'assets/plugin.css') to a compiled, preflight-off Tailwind stylesheet the frontend injects as a <link>. Must stay under assets/ with no '..', backslash, or query/fragment. See docs/plugin-styles.md." }
|
||||||
|
},
|
||||||
|
"additionalProperties": true,
|
||||||
|
"$defs": {
|
||||||
|
"domainName": { "type": "string", "minLength": 1, "pattern": "^[A-Za-z0-9_.:-]+$" },
|
||||||
|
"capabilityDeclaration": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"roles": { "type": "array", "items": { "enum": ["owner", "provider", "observer", "requester", "transformer", "handler", "validator", "short-circuiter", "contributor"] }, "uniqueItems": true },
|
||||||
|
"commands": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"operations": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"requests": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"observes": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"emits": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"events": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true },
|
||||||
|
"kind": { "enum": ["command", "provider-coordinator", "event", "diagnostic", "privileged"] },
|
||||||
|
"mode": { "enum": ["active", "optional", "legacy-shim", "disabled"] },
|
||||||
|
"compatibility": { "enum": ["none", "shim-allowed", "degrade-noop", "required", "legacy-window-shim"] },
|
||||||
|
"ownership": { "enum": ["exclusive-owner", "multi-provider", "observer-only", "requester-only", "privileged", "diagnostic-only"] },
|
||||||
|
"safety": { "enum": ["safe", "privileged", "sensitive", "diagnostic-only"] },
|
||||||
|
"order": { "type": "object", "properties": { "fixed": { "type": "boolean" }, "before": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true }, "after": { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true } }, "additionalProperties": false },
|
||||||
|
"provider_policy": { "type": "object" },
|
||||||
|
"settings": { "type": "array", "description": "Declarative per-instance control descriptors (toggle / range / select) a participant exposes for a consuming host to render generically. Domain-agnostic in shape; how a host applies a chosen value is defined by each capability domain's contract (the visualization domain requires an applySetting(key, value) method on the renderer instance).", "items": { "type": "object", "required": ["key", "type"], "properties": { "key": { "type": "string", "minLength": 1 }, "label": { "type": "string" }, "type": { "enum": ["toggle", "range", "select"] }, "default": {}, "min": { "type": "number" }, "max": { "type": "number" }, "step": { "type": "number" }, "options": { "type": "array", "items": { "type": "object", "required": ["id"], "properties": { "id": { "type": "string", "minLength": 1 }, "label": { "type": "string" } }, "additionalProperties": false } } }, "additionalProperties": false } },
|
||||||
|
"description": { "type": "string" },
|
||||||
|
"summary": { "type": "string" },
|
||||||
|
"version": { "const": 1 }
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"domainDeclaration": { "oneOf": [{ "type": "object", "properties": { "role": { "type": "string", "minLength": 1 }, "roles": { "type": "array", "items": { "type": "string", "minLength": 1 } }, "ownership": { "enum": ["exclusive-owner", "multi-provider", "observer-only", "requester-only", "privileged", "diagnostic-only"] }, "safety": { "enum": ["safe", "privileged", "sensitive", "diagnostic-only"] }, "legacy_source": { "type": "string", "minLength": 1 } }, "additionalProperties": true }, { "type": "array" }] },
|
||||||
|
"contributionList": { "type": "array", "items": { "type": "object", "required": ["id"], "properties": { "id": { "type": "string", "minLength": 1 }, "region": { "type": "string", "minLength": 1 }, "label": { "type": "string" }, "order": { "type": ["number", "integer", "string"] } }, "additionalProperties": true } }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,142 @@
|
|||||||
|
# Plugin styling — the `styles` capability
|
||||||
|
|
||||||
|
> Building for the redesigned **v3 UI** (`SLOPSMITH_UI=v3` / `/v3`)? v3 uses `fb-*`
|
||||||
|
> design tokens and a restructured player chrome with a dedicated plugin-control
|
||||||
|
> slot. See **[plugin-v3-ui.md](plugin-v3-ui.md)** for the player-chrome contract
|
||||||
|
> plugins must follow in v3.
|
||||||
|
|
||||||
|
Slopsmith serves Tailwind as a **prebuilt** stylesheet
|
||||||
|
(`static/tailwind.min.css`), never the runtime Play CDN. The CDN's on-the-fly
|
||||||
|
JIT rescanned the DOM on the main thread and dropped ~26% of frames with the 3D
|
||||||
|
highway running (slopsmith-desktop#110). See **constitution Principle II**.
|
||||||
|
|
||||||
|
A prebuilt stylesheet only contains the classes the build scanner saw in **core
|
||||||
|
source at core build time**. That has a consequence for plugins:
|
||||||
|
|
||||||
|
- Core's build scans bundled plugins on disk, but **a plugin installed at
|
||||||
|
runtime** (community / NAS) was never scanned. Its classes — especially
|
||||||
|
arbitrary values like `text-[11px]`, `grid-cols-[1fr_auto]`,
|
||||||
|
`shadow-[0_0_8px_rgba(0,0,0,.5)]` — are **absent** from the served CSS, so its
|
||||||
|
UI renders unstyled.
|
||||||
|
|
||||||
|
The `styles` capability fixes this: your plugin ships its **own** compiled
|
||||||
|
stylesheet and declares it in the manifest. The frontend injects one versioned
|
||||||
|
`<link rel="stylesheet">` into `<head>` when your plugin activates, covering both
|
||||||
|
your screen and your settings panel.
|
||||||
|
|
||||||
|
> You only need this if you use Tailwind classes that aren't guaranteed in core —
|
||||||
|
> in practice, **any arbitrary-value class** (`w-[37px]`), or a custom class core
|
||||||
|
> doesn't ship. If you use only common core utilities (`flex`, `p-4`,
|
||||||
|
> `text-gray-300`, `bg-dark-600`), you can omit `styles` and rely on core's CSS.
|
||||||
|
|
||||||
|
## 1. Declare it in `plugin.json`
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "my_plugin",
|
||||||
|
"name": "My Plugin",
|
||||||
|
"version": "1.2.0",
|
||||||
|
"screen": "screen.html",
|
||||||
|
"script": "screen.js",
|
||||||
|
"styles": "assets/plugin.css"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`styles` is a **plugin-root-relative path that must live under `assets/`** (like
|
||||||
|
`screen`/`script`/`routes` are root-relative). It serves through the sandboxed
|
||||||
|
`/api/plugins/<id>/assets/...` route, so the file must be at
|
||||||
|
`<plugin>/assets/plugin.css`. The injected `<link>` is cache-busted with
|
||||||
|
`?v=<version>`, so **bump your manifest `version`** whenever you rebuild the CSS,
|
||||||
|
or browsers may serve a stale copy within a session.
|
||||||
|
|
||||||
|
## 2. Build the stylesheet — utilities only, `preflight: false`
|
||||||
|
|
||||||
|
Core already ships Tailwind's base reset (preflight) once. Your plugin must
|
||||||
|
**not** re-apply it, or it would double the reset and fight core's styles. Build
|
||||||
|
with `corePlugins: { preflight: false }` so your sheet emits **only the utility
|
||||||
|
classes your files use**.
|
||||||
|
|
||||||
|
`tailwind.config.js` (in your plugin repo):
|
||||||
|
|
||||||
|
```js
|
||||||
|
/** Plugin stylesheet build — utilities only, scanned from this plugin's files.
|
||||||
|
* Regenerate assets/plugin.css with: bash build-tailwind.sh */
|
||||||
|
module.exports = {
|
||||||
|
corePlugins: { preflight: false }, // core owns the single base reset
|
||||||
|
content: [
|
||||||
|
'./screen.js',
|
||||||
|
'./settings.html',
|
||||||
|
'./screen.html',
|
||||||
|
// add any other file that carries Tailwind classes (e.g. './tour.json')
|
||||||
|
],
|
||||||
|
theme: {
|
||||||
|
extend: {
|
||||||
|
// Re-declare any core theme tokens you reference so they compile here.
|
||||||
|
colors: {
|
||||||
|
dark: { 900: '#050508', 800: '#0a0a12', 700: '#10101e', 600: '#181830', 500: '#1e1e3a' },
|
||||||
|
accent: { DEFAULT: '#4080e0', light: '#60a0ff', dark: '#2060b0' },
|
||||||
|
gold: '#e8c040',
|
||||||
|
},
|
||||||
|
fontFamily: { display: ['"Inter"', 'system-ui', 'sans-serif'] },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
// Mirror only the dynamically-built classes your code generates at runtime
|
||||||
|
// (Tailwind can't see them textually). Drop this if you have none.
|
||||||
|
safelist: [
|
||||||
|
{ pattern: /^(bg|text|border)-(dark|accent)(-.+)?$/ },
|
||||||
|
],
|
||||||
|
plugins: [],
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Input CSS — **`@tailwind utilities;` only** (no `@tailwind base`, that's the
|
||||||
|
preflight you're disabling):
|
||||||
|
|
||||||
|
`_plugin.src.css`:
|
||||||
|
|
||||||
|
```css
|
||||||
|
@tailwind utilities;
|
||||||
|
```
|
||||||
|
|
||||||
|
Build script `build-tailwind.sh` (run at your plugin's release time — the output
|
||||||
|
is committed; end users never build):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")"
|
||||||
|
# Pin the same Tailwind 3.x core uses so output stays diff-stable.
|
||||||
|
exec npx -y tailwindcss@3.4.19 \
|
||||||
|
-c tailwind.config.js \
|
||||||
|
-i _plugin.src.css \
|
||||||
|
-o assets/plugin.css \
|
||||||
|
--minify
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash build-tailwind.sh # writes assets/plugin.css — commit it
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3. Rules (inherited from the constitution)
|
||||||
|
|
||||||
|
- **No Play CDN, no runtime CSS JIT** — anywhere, ever. Same rule that binds core.
|
||||||
|
- **`preflight: false`** — utilities only; core ships the one base reset.
|
||||||
|
- **`styles` under `assets/`** — it serves through the sandboxed asset route;
|
||||||
|
`..`, absolute paths, and NUL bytes are rejected by `safe_join`.
|
||||||
|
- **Bump `version` on every CSS rebuild** so the `?v=` cache-buster fetches fresh.
|
||||||
|
- Plugins without `styles` are unaffected and inject no `<link>`.
|
||||||
|
|
||||||
|
## How it works (for reference)
|
||||||
|
|
||||||
|
- The loader derives a manifest-only `has_styles` boolean and passes the `styles`
|
||||||
|
path through to `/api/plugins` — no plugin code is imported
|
||||||
|
(`plugins/__init__.py::_nav_entry`).
|
||||||
|
- The frontend (`static/app.js::_injectPluginStyles`) injects one
|
||||||
|
`<link rel="stylesheet" data-plugin-id data-plugin-version
|
||||||
|
href="/api/plugins/<id>/assets/plugin.css?v=<version>">` (the `styles` value —
|
||||||
|
e.g. `assets/plugin.css` — appended to `/api/plugins/<id>/`) into `<head>`, **before** the
|
||||||
|
screen markup so styles are present on first paint. It's deduped by version: a
|
||||||
|
plugin upgrade swaps the old `<link>` for the new one; re-activation never piles
|
||||||
|
up duplicates.
|
||||||
|
- The stylesheet is served by the existing
|
||||||
|
`/api/plugins/<id>/assets/<path>` route as `text/css`.
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
# Building plugins for the v3 UI (fee[dB]ack v0.3.0)
|
||||||
|
|
||||||
|
v0.3.0 ("fee[dB]ack") ships a redesigned UI **behind a flag** — `SLOPSMITH_UI=v3`
|
||||||
|
or the `/v3` route. The classic UI (v2) remains the default until 0.3.0 ships, so
|
||||||
|
plugins must work in **both**.
|
||||||
|
|
||||||
|
The good news: v3 **reuses the same engine** as v2 — same `server.py`, `app.js`,
|
||||||
|
`highway.js`, `playSong`, `showScreen`, capability registry, library providers,
|
||||||
|
and the `window.slopsmithViz_<id>` / `setRenderer` visualization contract. So your
|
||||||
|
plugin's **backend, capabilities, library providers, `nav`/`screen`, visualization
|
||||||
|
renderers, diagnostics, and settings export all work unchanged in v3.** v3 surfaces
|
||||||
|
your `nav` entry in the new sidebar (via `shell.js` `renderPluginNav`) and your
|
||||||
|
screen mounts exactly as before.
|
||||||
|
|
||||||
|
**The one thing that changed is the player chrome** — and only if your plugin
|
||||||
|
injects controls into it.
|
||||||
|
|
||||||
|
## What changed in the player
|
||||||
|
|
||||||
|
In v2, `#player-controls` was a wide, **always-visible** bottom bar. In v3 it
|
||||||
|
became a **minimal, auto-hiding centered transport** (it fades ~2.5 s after the
|
||||||
|
pointer goes still during playback), flanked by a **hover-reveal left icon rail**
|
||||||
|
with popovers.
|
||||||
|
|
||||||
|
So the legacy way of injecting a control breaks in v3 two ways:
|
||||||
|
|
||||||
|
1. **Auto-hide** — a button you append to `#player-controls` vanishes with the
|
||||||
|
transport after a couple seconds.
|
||||||
|
2. **Dead anchors** — legacy code commonly inserts before a `<span class="text-gray-700">`
|
||||||
|
separator or the `button:last-child` (the ✕ Close button). **Neither exists in
|
||||||
|
the v3 transport**, so your control lands at the wrong end or is unreachable.
|
||||||
|
|
||||||
|
## The contract: detect v3, mount into the plugin-control slot
|
||||||
|
|
||||||
|
The host exposes:
|
||||||
|
|
||||||
|
- `window.slopsmith.uiVersion === 'v3'` — detect v3 (absent / not `'v3'` in v2).
|
||||||
|
- `window.slopsmith.ui.playerControlSlot()` — returns a **stable, always-reachable
|
||||||
|
container** (the "Plugins" rail popover). In v3, append your control(s) here
|
||||||
|
instead of `#player-controls`.
|
||||||
|
|
||||||
|
Canonical pattern for any control you inject into the player:
|
||||||
|
|
||||||
|
```js
|
||||||
|
function playerSlot() {
|
||||||
|
return (window.slopsmith && window.slopsmith.uiVersion === 'v3'
|
||||||
|
&& window.slopsmith.ui && typeof window.slopsmith.ui.playerControlSlot === 'function')
|
||||||
|
? window.slopsmith.ui.playerControlSlot() : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function injectMyButton() {
|
||||||
|
const slot = playerSlot();
|
||||||
|
const controls = slot || document.getElementById('player-controls'); // v3 slot, else v2 bar
|
||||||
|
if (!controls) return;
|
||||||
|
if (myBtn && controls.contains(myBtn)) return; // guard the ACTUAL container
|
||||||
|
// Legacy inserts before a separator / the ✕ Close button; the v3 slot has no
|
||||||
|
// such anchor, so just append there.
|
||||||
|
const anchor = slot ? null : controls.querySelector('span.text-gray-700, button:last-child');
|
||||||
|
myBtn = document.createElement('button');
|
||||||
|
/* ... */
|
||||||
|
if (anchor) controls.insertBefore(myBtn, anchor); else controls.appendChild(myBtn);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- **Gate v3 behavior on `uiVersion`** so v2 is byte-for-byte unchanged.
|
||||||
|
- **Never** `insertBefore` the legacy `span.text-gray-700` separator or
|
||||||
|
`button:last-child` — they don't exist in the v3 transport. Append instead.
|
||||||
|
- **Guard idempotency against the actual container** (`controls.contains(myBtn)`),
|
||||||
|
not a hard-coded `#player-controls` — otherwise re-injection logic breaks in v3.
|
||||||
|
- **Dropdowns/panels** your control opens: position them via the trigger's
|
||||||
|
`getBoundingClientRect()` (portal to `document.body` or `#player`), **not**
|
||||||
|
relative to `#player-controls` — the trigger now lives in the rail popover.
|
||||||
|
- **Overlays/HUDs/canvases** you attach to `#player` keep working; just keep their
|
||||||
|
`z-index` **under the chrome layers**: transport/HUD `z-20`, rail `z-30`,
|
||||||
|
popovers `z-40`.
|
||||||
|
|
||||||
|
## Pedalboard metadata (icon, description, category)
|
||||||
|
|
||||||
|
The v3 **Plugins page** renders each plugin as a guitar **pedal** grouped onto
|
||||||
|
category **pedalboards**. To make your pedal look good, declare three optional,
|
||||||
|
additive manifest fields (all surfaced in `/api/plugins`):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "my_plugin",
|
||||||
|
"name": "My Plugin",
|
||||||
|
"description": "One short sentence shown under the pedal name.",
|
||||||
|
"category": "audio",
|
||||||
|
"icon": "assets/thumb.png"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- **`description`** — one short sentence (clamped to ~2 lines on the pedal).
|
||||||
|
- **`category`** — which board the pedal sits on. Suggested:
|
||||||
|
`audio | creation | practice | game | tools`. Unknown/absent → a curated default
|
||||||
|
then `"other"`.
|
||||||
|
- **`icon`** — assets-relative thumbnail (~square, ~256×256 PNG/SVG), served via
|
||||||
|
the existing sandboxed `/api/plugins/<id>/assets/...` route (same containment
|
||||||
|
rule as `styles`). **Shortcut:** if you omit `icon` but ship
|
||||||
|
`assets/thumb.png`, the loader auto-detects it — no manifest edit needed.
|
||||||
|
Plugins with no thumbnail get a default pedal graphic.
|
||||||
|
|
||||||
|
All three are backward-compatible: omit them and the plugin still loads and shows
|
||||||
|
a default pedal.
|
||||||
|
|
||||||
|
## The compatibility shim (don't rely on it)
|
||||||
|
|
||||||
|
So un-updated plugins still function, the host runs a `MutationObserver` that
|
||||||
|
re-homes any non-native `#player-controls` child into the slot. It's a safety net
|
||||||
|
— but it **breaks plugins that guard re-injection with
|
||||||
|
`#player-controls.contains(myBtn)`** (once the host moves the node out, the check
|
||||||
|
goes false and the plugin re-injects every song). **Mount into the slot yourself**
|
||||||
|
(the pattern above) to be correct; treat the shim as a fallback only.
|
||||||
|
|
||||||
|
## Styling
|
||||||
|
|
||||||
|
v3 uses `fb-*` design tokens (`fb-card`, `fb-text`, `fb-textDim`, `fb-primary`,
|
||||||
|
`fb-border`); v2 uses `dark-*` / `accent`. Legacy classes still **render
|
||||||
|
acceptably** in v3's dark theme, so a plugin that only uses core-guaranteed
|
||||||
|
utilities is functional in both. For polish, ship your own stylesheet via the
|
||||||
|
`styles` capability ([plugin-styles.md](plugin-styles.md)) declaring the tokens you
|
||||||
|
use — but the host slot already provides a styled container, so simple controls
|
||||||
|
need nothing special.
|
||||||
|
|
||||||
|
## Enabling / disabling plugins (Pedalboard footswitch)
|
||||||
|
|
||||||
|
The v3 **Pedalboard** Plugins page renders each plugin as a guitar pedal whose
|
||||||
|
"footswitch" turns the plugin on or off. The backend contract:
|
||||||
|
|
||||||
|
- **`enabled` field on every `/api/plugins` entry** — a boolean, default `true`.
|
||||||
|
Absent (older entries, stubbed test rows) is treated as enabled. The frontend
|
||||||
|
hides the nav and shows the footswitch unlit when `enabled` is `false`.
|
||||||
|
- **`POST /api/plugins/{plugin_id}/enabled`** — body `{"enabled": <bool>}`,
|
||||||
|
returns `{"id": "<id>", "enabled": <bool>}`.
|
||||||
|
- `400` if the body is missing/invalid or `enabled` isn't a real boolean
|
||||||
|
(`0`/`1`/strings are rejected).
|
||||||
|
- `400` if you try to disable an always-on plugin — `capability_inspector`
|
||||||
|
and any `app_tour_*` may never be disabled (disabling would brick the app or
|
||||||
|
the capability-graph review surface). Bundled plugins are otherwise
|
||||||
|
disable-able.
|
||||||
|
- `404` for an unknown plugin id (not loaded and not pending).
|
||||||
|
|
||||||
|
### Persistence
|
||||||
|
|
||||||
|
The choice is persisted under `CONFIG_DIR/plugin_state.json` as
|
||||||
|
`{"<plugin_id>": {"enabled": false}, ...}`. **Only non-default (`enabled:false`)
|
||||||
|
entries are stored** — re-enabling drops the key entirely, so the file stays
|
||||||
|
small and "absent ⇒ enabled" is the invariant. A missing or corrupt state file
|
||||||
|
is tolerated (logged, falls back to `{}`) and never crashes startup.
|
||||||
|
|
||||||
|
### Restart semantics
|
||||||
|
|
||||||
|
- Toggling **persists immediately** and flips the **in-memory** `enabled` flag,
|
||||||
|
so the very next `/api/plugins` (and thus the nav, the Pedalboard, and the
|
||||||
|
capability pipeline) reflects the change at once — no restart needed for the
|
||||||
|
UI to update.
|
||||||
|
- A plugin **disabled at runtime keeps its already-mounted routes/screen** until
|
||||||
|
the next restart; full hot-unload is out of scope. The frontend treats
|
||||||
|
`enabled:false` as "off" regardless.
|
||||||
|
- At **startup**, the loader **skips disabled plugins entirely** — it does not
|
||||||
|
install requirements, run `routes.setup()`, or register their screen, nav, or
|
||||||
|
capabilities. They still appear in `/api/plugins` as a disabled entry
|
||||||
|
(`status: "disabled"`, `enabled: false`) so the UI can show an "off" pedal you
|
||||||
|
can switch back on. **Re-enabling** a plugin that was skipped at startup
|
||||||
|
updates the flag immediately but the plugin only actually mounts on the next
|
||||||
|
restart.
|
||||||
|
|
||||||
|
### Capability pipeline
|
||||||
|
|
||||||
|
A disabled plugin is **excluded from the capability pipeline**: its
|
||||||
|
`capabilities`, `standards`, `capability_validation_warnings`,
|
||||||
|
`capability_unsupported_versions`, and `compatibility_shims` are emptied in the
|
||||||
|
`/api/plugins` response whenever `enabled` is `false` (covering both
|
||||||
|
startup-skipped and runtime-toggled-off plugins). Because the browser capability
|
||||||
|
registry registers any entry that carries a capability declaration regardless of
|
||||||
|
status, suppressing the metadata here is what actually keeps a disabled plugin
|
||||||
|
out of the capability graph.
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
- [ ] Backend / capabilities / library provider / `nav` + `screen` /
|
||||||
|
visualization renderer — **no change needed** (they work in v3 as-is).
|
||||||
|
- [ ] If you inject a control into the player: detect v3 and mount into
|
||||||
|
`window.slopsmith.ui.playerControlSlot()`; drop the dead separator /
|
||||||
|
`button:last-child` anchor; guard `contains()` against the actual container.
|
||||||
|
- [ ] Dropdowns positioned via `getBoundingClientRect()`, not `#player-controls`.
|
||||||
|
- [ ] `#player` overlays keep `z-index` ≤ the chrome layers (transport/HUD 20,
|
||||||
|
rail 30, popovers 40).
|
||||||
|
- [ ] Verify in **both** `/` (v2) and `/v3`.
|
||||||
@@ -0,0 +1,206 @@
|
|||||||
|
# Debugging Keyboard Shortcuts
|
||||||
|
|
||||||
|
This skill helps you debug keyboard shortcut issues in Slopsmith.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
1. **Start Slopsmith:**
|
||||||
|
```bash
|
||||||
|
cd ~/path/to/slopsmith
|
||||||
|
LIBRARY_PATH=/path/to/your/library docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Open browser:** http://localhost:8000
|
||||||
|
|
||||||
|
3. **Open DevTools:** Press `F12` or `Ctrl+Shift+I`
|
||||||
|
|
||||||
|
## Debugging Commands
|
||||||
|
|
||||||
|
Open the browser console and run these commands:
|
||||||
|
|
||||||
|
### Enable Debug Logging
|
||||||
|
```javascript
|
||||||
|
_setDebugShortcuts(true)
|
||||||
|
```
|
||||||
|
This will log every keypress and shortcut match attempt.
|
||||||
|
|
||||||
|
### List All Registered Shortcuts
|
||||||
|
```javascript
|
||||||
|
_listShortcuts()
|
||||||
|
```
|
||||||
|
Shows all shortcuts with their keys, scopes, and descriptions.
|
||||||
|
|
||||||
|
### Test a Specific Shortcut
|
||||||
|
```javascript
|
||||||
|
_testShortcut('Space')
|
||||||
|
```
|
||||||
|
Shows if a shortcut would be active in the current context.
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
### 1. Shortcut Not Triggering
|
||||||
|
|
||||||
|
**Check:**
|
||||||
|
- Are you on the right screen? (Player shortcuts only work on player screen)
|
||||||
|
- Is focus in an input field? (Shortcuts are disabled when typing)
|
||||||
|
- Is the key registered? Run `_listShortcuts()` to see all registered shortcuts
|
||||||
|
|
||||||
|
**Debug:**
|
||||||
|
```javascript
|
||||||
|
_setDebugShortcuts(true)
|
||||||
|
// Now press your key and watch the console
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Scope Issues
|
||||||
|
|
||||||
|
**Check current context:**
|
||||||
|
```javascript
|
||||||
|
// This shows which screen you're on
|
||||||
|
document.querySelector('.screen.active')?.id
|
||||||
|
```
|
||||||
|
|
||||||
|
**Common scopes:**
|
||||||
|
- `global` - Works on any screen
|
||||||
|
- `player` - Only on player screen
|
||||||
|
- `library` - On home, favorites, or settings screens
|
||||||
|
- `plugin-{id}` - Only on a specific plugin's screen
|
||||||
|
|
||||||
|
### 3. Key Matching Issues
|
||||||
|
|
||||||
|
The system matches on both `e.key` (character produced) and `e.code` (physical key):
|
||||||
|
|
||||||
|
- Use `e.key` for letters/symbols that depend on keyboard layout
|
||||||
|
- Use `e.code` for special keys (Space, ArrowLeft, Escape, etc.)
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```javascript
|
||||||
|
// Good for special keys
|
||||||
|
registerShortcut({ key: 'Space', ... }) // or 'ArrowLeft', 'Escape'
|
||||||
|
|
||||||
|
// Good for layout-dependent keys
|
||||||
|
registerShortcut({ key: '?', ... }) // or '[', ']', 'k'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Condition Not Met
|
||||||
|
|
||||||
|
If your shortcut has a condition function, it must return `true`:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
registerShortcut({
|
||||||
|
key: 'k',
|
||||||
|
description: 'My action',
|
||||||
|
scope: 'player',
|
||||||
|
condition: () => _isMyViewActive, // Must be true
|
||||||
|
handler: () => _myAction()
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Test it:**
|
||||||
|
```javascript
|
||||||
|
_testShortcut('k')
|
||||||
|
// Check if `conditionMet` is true
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing Your Changes
|
||||||
|
|
||||||
|
1. Make changes to `static/app.js`
|
||||||
|
2. Refresh the browser (changes are live-reloaded via Docker volume mount)
|
||||||
|
3. Run `_listShortcuts()` to verify your shortcut is registered
|
||||||
|
4. Press `?` to open the shortcuts help panel
|
||||||
|
5. Test your shortcut
|
||||||
|
|
||||||
|
## Built-in Shortcuts
|
||||||
|
|
||||||
|
Press `?` to see all shortcuts in the UI. Built-in shortcuts:
|
||||||
|
|
||||||
|
| Key | Scope | Description |
|
||||||
|
|-----|-------|-------------|
|
||||||
|
| `?` | Global | Show keyboard shortcuts |
|
||||||
|
| `Space` | Player | Play/Pause |
|
||||||
|
| `ArrowLeft` | Player | Seek back 5 seconds |
|
||||||
|
| `ArrowRight` | Player | Seek forward 5 seconds |
|
||||||
|
| `Escape` | Player | Back to library |
|
||||||
|
| `[` | Player | Offset audio back (Shift: 50ms, else 10ms) |
|
||||||
|
| `]` | Player | Offset audio forward (Shift: 50ms, else 10ms) |
|
||||||
|
|
||||||
|
## Adding Your Own Shortcuts
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
registerShortcut({
|
||||||
|
key: 'k', // Key to press
|
||||||
|
description: 'Toggle my view', // Shown in help panel
|
||||||
|
scope: 'player', // When it's active
|
||||||
|
condition: () => _isMyViewActive, // Optional guard
|
||||||
|
handler: (e) => _myAction() // What to do
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Panel-Scoped Shortcuts
|
||||||
|
|
||||||
|
For plugins that create multiple panels (e.g., splitscreen), shortcuts are automatically scoped to the active panel:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// Create panels (must exist before setActiveShortcutPanel can target them)
|
||||||
|
const panel1 = window.createShortcutPanel('panel-1');
|
||||||
|
const panel2 = window.createShortcutPanel('panel-2');
|
||||||
|
|
||||||
|
// Set active panel and register shortcuts
|
||||||
|
window.setActiveShortcutPanel('panel-1');
|
||||||
|
registerShortcut({
|
||||||
|
key: 'd',
|
||||||
|
description: 'Dock panel',
|
||||||
|
scope: 'global',
|
||||||
|
handler: () => _dockPanel()
|
||||||
|
});
|
||||||
|
|
||||||
|
// Switch to another panel
|
||||||
|
window.setActiveShortcutPanel('panel-2');
|
||||||
|
registerShortcut({
|
||||||
|
key: 'f',
|
||||||
|
description: 'Toggle fullscreen',
|
||||||
|
scope: 'global',
|
||||||
|
handler: () => _toggleFullscreen()
|
||||||
|
});
|
||||||
|
|
||||||
|
// Clean up when done — clear every panel you created
|
||||||
|
panel1.clearShortcuts();
|
||||||
|
panel2.clearShortcuts();
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important:** In splitscreen, `scope: 'player'` means "player screen in the current panel". Each panel can have its own player shortcuts without collisions.
|
||||||
|
|
||||||
|
**Truly global shortcuts:** Use `window.getGlobalShortcutContext()` for shortcuts that must work in all panels (exceptional case, logs warning).
|
||||||
|
|
||||||
|
## Network Issues
|
||||||
|
|
||||||
|
If shortcuts aren't working at all:
|
||||||
|
|
||||||
|
1. Check the **Network** tab in DevTools
|
||||||
|
2. Look for failed requests to `/api/plugins`
|
||||||
|
3. Check the **Console** tab for JavaScript errors
|
||||||
|
4. Verify the container is running:
|
||||||
|
```bash
|
||||||
|
docker compose ps
|
||||||
|
docker compose logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
## WebSocket Issues
|
||||||
|
|
||||||
|
Keyboard shortcuts don't require WebSocket, but if other features aren't working:
|
||||||
|
|
||||||
|
1. Check **Network** tab → "WS" filter
|
||||||
|
2. Look for WebSocket connections to `/ws/highway/...`
|
||||||
|
3. Should show status "101 Switching Protocols"
|
||||||
|
|
||||||
|
## Getting Help
|
||||||
|
|
||||||
|
If you're still stuck:
|
||||||
|
|
||||||
|
1. Enable debug mode: `_setDebugShortcuts(true)`
|
||||||
|
2. Reproduce the issue
|
||||||
|
3. Copy the console output
|
||||||
|
4. Share it along with:
|
||||||
|
- Which screen you're on
|
||||||
|
- What key you're pressing
|
||||||
|
- What you expect to happen
|
||||||
|
- What actually happens
|
||||||
@@ -0,0 +1,266 @@
|
|||||||
|
# Sloppak Hand-Editing — User Guide
|
||||||
|
|
||||||
|
A `.sloppak` is just a zip of plain files: some YAML, some JSON, some OGG audio, maybe a JPEG. That means you can open one up and change it. Want to record your own rhythm guitar take and use that instead of the mix? Fix an artist typo? Swap the cover art? Replace a Demucs split that bled drums into the "other" stem? You don't need to rebuild the whole sloppak from its source — just edit the file.
|
||||||
|
|
||||||
|
This guide walks through the most common edits, aimed at musicians who are comfortable with a text editor and Audacity but don't live on the command line.
|
||||||
|
|
||||||
|
> For the format **schema** (what every field means, how the wire format works, how to extend the format with new data types), see [sloppak-spec.md](sloppak-spec.md). This document is the **how-do-I-actually-edit-mine** companion.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The two forms — directory and zip
|
||||||
|
|
||||||
|
A sloppak exists in two interchangeable forms:
|
||||||
|
|
||||||
|
| Form | What it is | When to use it |
|
||||||
|
|---|---|---|
|
||||||
|
| **Directory** | A folder named `something.sloppak/` with the files loose inside | **Authoring** — easy to edit, no zip/unzip cycle |
|
||||||
|
| **Zip** | A `something.sloppak` file (zip with the same files inside) | **Distributing** — single file to share |
|
||||||
|
|
||||||
|
Slopsmith reads both. You can drop either one straight into your DLC folder and it'll show up in the library.
|
||||||
|
|
||||||
|
### Unzipping for editing
|
||||||
|
|
||||||
|
Slopsmith's converter ships sloppaks in zip form. To edit one, unzip it:
|
||||||
|
|
||||||
|
- **Windows:** rename `mysong.sloppak` → `mysong.zip`, right-click → Extract All. Then rename the resulting folder back to `mysong.sloppak/` (with the trailing slash / folder form). Or use [7-Zip](https://www.7-zip.org/) and unzip without renaming.
|
||||||
|
- **macOS:** rename `.sloppak` → `.zip`, double-click. Or use The Unarchiver.
|
||||||
|
- **Linux:** `unzip mysong.sloppak -d mysong.sloppak/`.
|
||||||
|
|
||||||
|
Once you have the directory form, you can edit any file inside and Slopsmith will pick it up — no re-zipping required for your own use.
|
||||||
|
|
||||||
|
### Cache: when changes don't appear
|
||||||
|
|
||||||
|
The first time Slopsmith opens a zip-form sloppak, it extracts a working copy into its config directory's cache: `${CONFIG_DIR}/sloppak_cache/<safe-id>` (in the standard Docker setup that's inside the `slopsmith-config` volume, mounted at `/config` in the container). The `<safe-id>` is the sloppak filename with each path separator (`/` or `\`) replaced by `__` and each space replaced by `_`. So `My-Song.sloppak` stays `My-Song.sloppak`, and `Artist/My Song.sloppak` becomes `Artist__My_Song.sloppak`.
|
||||||
|
|
||||||
|
You almost never need to touch this cache directly. If you edit the **original zip** in your DLC folder, Slopsmith re-extracts automatically when the zip's modification time or size changes — just save your edits and reload.
|
||||||
|
|
||||||
|
If a change still isn't appearing, the simplest reset is to remove the matching cache folder so Slopsmith rebuilds it on the next song load. In a default Docker install that's `docker exec <container> rm -rf /config/sloppak_cache/<safe-id>` (or the equivalent for your setup).
|
||||||
|
|
||||||
|
If you'd rather skip the cache layer entirely, **drop the directory form straight into your DLC folder** — Slopsmith uses it in place and there's nothing to invalidate.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Record and add your own rhythm stem
|
||||||
|
|
||||||
|
The use case: the converted rhythm guitar sounds muddy (Demucs has a tough time isolating fingerpicked acoustic, hi-gain palm mutes, etc.), and you'd rather play and record your own take.
|
||||||
|
|
||||||
|
### Step 1 — Set up your reference
|
||||||
|
|
||||||
|
1. Unzip the sloppak (see §1).
|
||||||
|
2. Look in `stems/` and open the reference audio in [Audacity](https://www.audacityteam.org/) (File → Open). Two cases:
|
||||||
|
- **`stems/full.ogg` is present** (pre-Demucs-split sloppak, or one where you kept `full.ogg` as a fallback — see §3). Open it directly; it's the original mixed audio.
|
||||||
|
- **No `full.ogg`, only per-instrument stems** (`guitar.ogg`, `bass.ogg`, `drums.ogg`, …). This is the default after Demucs splitting, since the converter deletes `full.ogg` once split stems exist. Select **all** the per-instrument stems and open them together — Audacity loads each as its own track aligned at `t=0`, and playing all of them at once reconstructs the full mix.
|
||||||
|
3. Note the **sample rate** displayed in Audacity's status bar (typically `44100 Hz`). Your recording must match this.
|
||||||
|
|
||||||
|
### Step 2 — Record your take aligned to the mix
|
||||||
|
|
||||||
|
1. In Audacity, with the reference track(s) open, add a new audio track (Tracks → Add New → Mono/Stereo Track).
|
||||||
|
2. Set Audacity to play the reference through your headphones (so you can hear what you're playing along to) while recording your own input.
|
||||||
|
3. Hit Record and play your rhythm part along with the reference from `t=0`. Critical: **start recording at the very beginning of the song.** If you punch in late, alignment will be off when you drop it in.
|
||||||
|
4. Stop when the song ends. Trim any silence/click at the very start of your recorded track so its first sample lines up with `t=0` of the reference (zoom in tight and check visually against the kick or first guitar hit).
|
||||||
|
|
||||||
|
### Step 3 — Export as OGG
|
||||||
|
|
||||||
|
1. **Solo** your recorded track (mute every reference track).
|
||||||
|
2. File → Export → Export as OGG Vorbis.
|
||||||
|
3. Quality slider: **5** (matches what the converter uses). Save as `rhythm_custom.ogg`.
|
||||||
|
4. Confirm in the export dialog that the sample rate is the same `44100 Hz` you noted in Step 1.
|
||||||
|
|
||||||
|
### Step 4 — Drop it in and update the manifest
|
||||||
|
|
||||||
|
1. Copy `rhythm_custom.ogg` into the sloppak's `stems/` folder.
|
||||||
|
2. Open `manifest.yaml` in any text editor (Notepad++, VS Code, BBEdit, gedit — all fine; just **don't use Word**).
|
||||||
|
3. Find the `stems:` block. Two things matter here:
|
||||||
|
- **Order:** Slopsmith's base `<audio>` element always plays the **first** stem listed in `stems[]`, regardless of `default:` flags. So if you want your custom stem to be what the player plays out-of-the-box (and what users without the Stems plugin will hear), put it **first**.
|
||||||
|
- **`default:` flags:** consulted by the [Stems plugin](https://github.com/topkoa/slopsmith-plugin-stems) to decide which faders start un-muted. They do **not** affect what the base `<audio>` element plays — that's purely the first-stem rule above.
|
||||||
|
|
||||||
|
Example for a Demucs-split sloppak where you re-recorded the rhythm guitar:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
stems:
|
||||||
|
- id: rhythm_custom # listed first → base <audio> plays this
|
||||||
|
file: stems/rhythm_custom.ogg
|
||||||
|
default: true
|
||||||
|
- id: guitar
|
||||||
|
file: stems/guitar.ogg
|
||||||
|
default: false # Stems plugin starts this fader muted
|
||||||
|
- id: bass
|
||||||
|
file: stems/bass.ogg
|
||||||
|
default: true
|
||||||
|
- id: drums
|
||||||
|
file: stems/drums.ogg
|
||||||
|
default: true
|
||||||
|
# … other stems unchanged …
|
||||||
|
```
|
||||||
|
|
||||||
|
Example for a pre-split sloppak (only `full.ogg` exists):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
stems:
|
||||||
|
- id: rhythm_custom # listed first → base <audio> plays this
|
||||||
|
file: stems/rhythm_custom.ogg
|
||||||
|
default: true
|
||||||
|
- id: full
|
||||||
|
file: stems/full.ogg
|
||||||
|
default: false # Stems plugin starts the full mix muted
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Save the file. **Mind the indentation** — two spaces, no tabs. YAML is fussy about this.
|
||||||
|
|
||||||
|
### Step 5 — Reload and verify
|
||||||
|
|
||||||
|
Reload the song in Slopsmith. The [Stems plugin](https://github.com/topkoa/slopsmith-plugin-stems) will show a fader for `rhythm_custom` next to the others. If you don't see it, check the cache notes in §1.
|
||||||
|
|
||||||
|
### Common gotchas
|
||||||
|
|
||||||
|
- **Sample-rate mismatch** → choppy/pitched-wrong playback. Re-export from Audacity at exactly the rate the other stems use.
|
||||||
|
- **Mono vs stereo mismatch** is fine for playback but levels can feel different — match what the other stems use if you want consistent behavior in the mixer.
|
||||||
|
- **Silence padding at the start** of your recording → your stem will play late. Trim it tight in Audacity before exporting.
|
||||||
|
- **Tabs in `manifest.yaml`** → Slopsmith will refuse to load the song. Use two spaces.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Replace a bad Demucs stem
|
||||||
|
|
||||||
|
Demucs is good but not perfect. `htdemucs_6s` will occasionally bleed snare into `other.ogg` or leave drum overtones in the bass track. Fixing it works the same way as adding a custom stem — you're just overwriting an existing one.
|
||||||
|
|
||||||
|
### Option A: overwrite in place
|
||||||
|
|
||||||
|
1. Source or record a clean replacement and export it as OGG with the same sample rate.
|
||||||
|
2. Save it directly over the bad file (e.g. `stems/other.ogg`).
|
||||||
|
3. Reload — no manifest change needed.
|
||||||
|
|
||||||
|
### Option B: keep the original, add a replacement
|
||||||
|
|
||||||
|
Useful if you want to A/B them:
|
||||||
|
|
||||||
|
1. Save your new file as `stems/other_v2.ogg`.
|
||||||
|
2. In `manifest.yaml`, change the `file:` path on that stem's entry:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- id: other
|
||||||
|
file: stems/other_v2.ogg # was stems/other.ogg
|
||||||
|
default: true
|
||||||
|
```
|
||||||
|
|
||||||
|
3. The old `other.ogg` stays in the folder but is no longer referenced. Delete it later if you want.
|
||||||
|
|
||||||
|
### Removing a stem entirely
|
||||||
|
|
||||||
|
If you want to drop a stem (e.g. `piano.ogg` is empty for this song):
|
||||||
|
|
||||||
|
1. Delete the file from `stems/`.
|
||||||
|
2. **Also remove** its entry from `manifest.yaml stems[]`. Leaving an orphan manifest entry pointing at a missing file produces a 404 in the player.
|
||||||
|
|
||||||
|
### A word on `full.ogg`
|
||||||
|
|
||||||
|
A converted sloppak starts with just `stems/full.ogg`. After Demucs splits it, the converter rewrites the manifest to list the per-instrument stems and removes `full.ogg`. If you're hand-editing and want to *keep* `full.ogg` as a fallback (mixed audio in case all the individual stems are muted), that's fine — leave the file in place and add a manifest entry with `default: false`. Don't delete `full.ogg` unless the per-instrument stems sum cleanly to a full mix.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Edit metadata, cover art, lyrics, tuning
|
||||||
|
|
||||||
|
All of these are tweaks to either `manifest.yaml` or files it points at. Open `manifest.yaml` in a text editor for the next three sections.
|
||||||
|
|
||||||
|
### Title, artist, album, year
|
||||||
|
|
||||||
|
Top-level keys in `manifest.yaml`. Just edit the strings:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
title: "Black Hole Sun"
|
||||||
|
artist: "Soundgarden"
|
||||||
|
album: "Superunknown"
|
||||||
|
year: 1994
|
||||||
|
duration: 320.5
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep the quotes if the value already has them (especially when there's an apostrophe or colon). Reload the song — the library card updates next time the library refreshes.
|
||||||
|
|
||||||
|
### Cover art
|
||||||
|
|
||||||
|
Drop a square JPEG or PNG (500–1500 px on a side is the sweet spot) into the sloppak root and point the manifest at it:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
cover: cover.jpg
|
||||||
|
```
|
||||||
|
|
||||||
|
If the manifest doesn't have a `cover:` line, add one. The converter normally produces `cover.jpg` already; this is mostly relevant if you want to replace it with a better image.
|
||||||
|
|
||||||
|
### Lyrics
|
||||||
|
|
||||||
|
`lyrics.json` is a flat JSON list of syllable objects:
|
||||||
|
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{"t": 12.34, "d": 0.18, "w": "Hel"},
|
||||||
|
{"t": 12.52, "d": 0.22, "w": "lo-"},
|
||||||
|
{"t": 13.10, "d": 0.30, "w": "world"}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| `t` | Time the syllable starts, in seconds (float) |
|
||||||
|
| `d` | Duration in seconds |
|
||||||
|
| `w` | The syllable text. A trailing `-` joins it to the next syllable as one word. A trailing `+` marks the last syllable of a line (the renderer wraps after it). Both are suffixes on a real syllable — not standalone entries |
|
||||||
|
|
||||||
|
Common hand-edits:
|
||||||
|
- **Karaoke timing is off** — bump `t` values up or down a few hundredths of a second.
|
||||||
|
- **Wrong word** — edit `w`.
|
||||||
|
- **Missing line break** — append `+` to the last syllable of the line that should end there (e.g. change `"w": "world"` to `"w": "world+"`). Don't insert a standalone `"+"` entry — that creates an empty syllable that still consumes word-spacing in the renderer.
|
||||||
|
|
||||||
|
It's plain JSON — edit in any text editor.
|
||||||
|
|
||||||
|
### Tuning
|
||||||
|
|
||||||
|
Per-arrangement, in `manifest.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
arrangements:
|
||||||
|
- id: lead
|
||||||
|
name: Lead
|
||||||
|
file: arrangements/lead.json
|
||||||
|
tuning: [0, 0, 0, 0, 0, 0] # E standard
|
||||||
|
capo: 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Each number is **semitones from E A D G B E**, lowest string first. Common tunings:
|
||||||
|
|
||||||
|
| Tuning | Offsets |
|
||||||
|
|---|---|
|
||||||
|
| E Standard | `[0, 0, 0, 0, 0, 0]` |
|
||||||
|
| Eb Standard | `[-1, -1, -1, -1, -1, -1]` |
|
||||||
|
| D Standard | `[-2, -2, -2, -2, -2, -2]` |
|
||||||
|
| Drop D | `[-2, 0, 0, 0, 0, 0]` |
|
||||||
|
| Drop C | `[-4, -2, -2, -2, -2, -2]` |
|
||||||
|
| DADGAD | `[-2, 0, 0, 0, -2, -2]` |
|
||||||
|
|
||||||
|
The manifest tuning overrides whatever's stored inside `arrangements/lead.json` — so fixing it here is enough; you don't need to touch the arrangement JSON.
|
||||||
|
|
||||||
|
For 4-string bass, only indices 0–3 are meaningful; leave 4 and 5 at `0`.
|
||||||
|
|
||||||
|
### What *not* to put in `manifest.yaml`
|
||||||
|
|
||||||
|
Don't add per-machine settings (audio device picks, MIDI port IDs), UI state, or your own play counts. The sloppak holds the song's authored data — anything that varies by user or machine lives in Slopsmith's config dir or the metadata DB. See [sloppak-spec.md §5.7](sloppak-spec.md#57-dont-break-the-manifest-contract) for the full list.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Re-zipping for distribution
|
||||||
|
|
||||||
|
If you want to share your modified sloppak with someone else, re-zip it:
|
||||||
|
|
||||||
|
1. Open the `mysong.sloppak/` directory.
|
||||||
|
2. Select **everything inside** — `manifest.yaml`, `arrangements/`, `stems/`, `lyrics.json`, `cover.jpg`.
|
||||||
|
3. Zip the **contents**, not the parent folder. (If you zip the folder, the zip will have a top-level `mysong.sloppak/` directory inside, which Slopsmith won't parse — the manifest must be at the zip root.)
|
||||||
|
4. Rename `mysong.zip` → `mysong.sloppak`.
|
||||||
|
|
||||||
|
For your own use, you can skip this entirely — Slopsmith reads the directory form straight from your DLC folder.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Out of scope (for now)
|
||||||
|
|
||||||
|
- **Authoring a sloppak from scratch** (no Guitar Pro / MusicXML source file) — that's a developer task. Start at [sloppak-spec.md §4.2](sloppak-spec.md#42-writing-python-server-side).
|
||||||
|
- **Editing notes / chords in `arrangements/*.json`** — technically possible but extremely tedious by hand: hundreds of objects with short field names per song. The fields are documented in [sloppak-spec.md §3](sloppak-spec.md#3-arrangement-json--the-wire-format), but for any real chart edit you want the [Arrangement Editor plugin](https://github.com/byrongamatos/slopsmith-plugin-editor).
|
||||||
|
- **Loudness normalization / advanced stem processing** — out of scope here; standard Audacity or ffmpeg workflows apply to any OGG file before you drop it into `stems/`.
|
||||||
@@ -0,0 +1,939 @@
|
|||||||
|
# Sloppak Format — Developer Guide
|
||||||
|
|
||||||
|
Sloppak is Slopsmith's open, hand-editable song format. This guide is for developers who want to **read**, **write**, or **extend** the format — including adding new data types like drum tabs, vocal pitches, lighting cues, key/scale annotations, or anything else a future visualization plugin might need.
|
||||||
|
|
||||||
|
> If you're a **user** wanting to modify an existing sloppak — record your own rhythm stem, fix metadata, swap cover art, replace a Demucs split — see [sloppak-hand-editing.md](sloppak-hand-editing.md). That guide is the practical, step-by-step companion to this developer reference.
|
||||||
|
|
||||||
|
The authoritative format reference lives in code (`lib/sloppak.py`, `lib/song.py`); this doc explains the why, the how, and the conventions you should follow when adding to it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Format at a glance
|
||||||
|
|
||||||
|
A sloppak exists in **two interchangeable forms**:
|
||||||
|
|
||||||
|
| Form | What it is | Used for |
|
||||||
|
|---|---|---|
|
||||||
|
| **Directory** | A folder named `*.sloppak/` containing the files below | Authoring, hand editing, plugin development |
|
||||||
|
| **Zip archive** | A `.sloppak` file (zip with the same files inside) | Distribution |
|
||||||
|
|
||||||
|
Both forms hold identical contents. Slopsmith resolves either transparently — zip files are unpacked to a cache the first time they're opened (see `resolve_source_dir()` in [lib/sloppak.py](../lib/sloppak.py)).
|
||||||
|
|
||||||
|
### Directory layout
|
||||||
|
|
||||||
|
```
|
||||||
|
my-song.sloppak/
|
||||||
|
├── manifest.yaml # Required — all metadata + file index
|
||||||
|
├── arrangements/
|
||||||
|
│ ├── lead.json # One JSON per playable arrangement
|
||||||
|
│ ├── rhythm.json
|
||||||
|
│ └── bass.json
|
||||||
|
├── stems/
|
||||||
|
│ ├── full.ogg # Mixed audio (initial single-stem output; may be absent after stem splitting)
|
||||||
|
│ ├── guitar.ogg # Optional individual stems
|
||||||
|
│ ├── bass.ogg
|
||||||
|
│ ├── drums.ogg
|
||||||
|
│ ├── vocals.ogg
|
||||||
|
│ └── other.ogg
|
||||||
|
├── lyrics.json # Optional — syllable-level lyrics
|
||||||
|
└── cover.jpg # Optional — album art
|
||||||
|
```
|
||||||
|
|
||||||
|
Three rules to remember:
|
||||||
|
|
||||||
|
1. **`manifest.yaml` is the index.** Nothing inside the sloppak is auto-discovered — every file path is listed in the manifest. This makes the format predictable: no scanning, no guessing. (One historical exception: the cover-art handler in `server.py` falls back to `cover.jpg` when `manifest.cover` is missing. New code should not add similar filename fallbacks.)
|
||||||
|
2. **Filenames in `manifest.yaml` are POSIX paths**, relative to the sloppak root (forward slashes, no leading `/`).
|
||||||
|
3. **YAML for the manifest, JSON for everything else.** YAML is hand-editable for users; JSON is fast-parsed and easy to round-trip in code.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. `manifest.yaml` reference
|
||||||
|
|
||||||
|
Minimal valid manifest:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
title: "Black Hole Sun"
|
||||||
|
artist: "Soundgarden"
|
||||||
|
duration: 320.5
|
||||||
|
arrangements:
|
||||||
|
- id: lead
|
||||||
|
name: Lead
|
||||||
|
file: arrangements/lead.json
|
||||||
|
tuning: [0, 0, 0, 0, 0, 0]
|
||||||
|
capo: 0
|
||||||
|
stems:
|
||||||
|
- id: full
|
||||||
|
file: stems/full.ogg
|
||||||
|
default: true
|
||||||
|
```
|
||||||
|
|
||||||
|
Full set of currently-recognized top-level keys:
|
||||||
|
|
||||||
|
| Key | Type | Required | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `title` | string | yes | Song title |
|
||||||
|
| `artist` | string | yes | Artist name |
|
||||||
|
| `album` | string | no | Album |
|
||||||
|
| `year` | int | no | Release year |
|
||||||
|
| `duration` | float | yes | Song length in seconds |
|
||||||
|
| `arrangements` | list | yes | Playable arrangements (see §2.1) |
|
||||||
|
| `stems` | list | yes | Audio stems (see §2.2) |
|
||||||
|
| `stem_separation` | object | no | Structured metadata when stems were produced by an automated separation engine (currently `demucs`). Shape: `{engine, model, version}`. See §2.2 for fields + semver semantics per [slopsmith#357](https://github.com/byrongamatos/slopsmith/issues/357). Omitted for single-stem sloppaks (`stems: [{id: full, ...}]`) and for hand-edited / user-recorded stems |
|
||||||
|
| `lyrics` | string | no | Path to lyrics JSON |
|
||||||
|
| `lyrics_source` | string | no | Where the lyrics came from: `xml` (vocals XML from the chart source), `whisperx` (auto-transcribed), or `user` (hand-edited). Absent on legacy sloppaks — readers should treat missing as `xml` |
|
||||||
|
| `lyric_transcription` | object | no | Structured metadata when lyrics came from an automated engine (currently `whisperx`). Same shape as the parent `stem_separation` block defined by [slopsmith#357](https://github.com/byrongamatos/slopsmith/issues/357) — see §2.3 for fields and semver semantics. Omitted for authored lyrics (`xml`/`user`) |
|
||||||
|
| `vocal_pitch` | string | no | Path to per-syllable pitch JSON (`{"version": 1, "notes": [{t, d, midi}, ...]}`). Consumed by [slopsmith-plugin-lyrics-karaoke](https://github.com/byrongamatos/slopsmith-plugin-lyrics-karaoke) to render karaoke note bars. See §2.4 |
|
||||||
|
| `pitch_extraction` | object | no | Structured metadata when pitch was extracted by an automated engine (currently `crepe` via the demucs server's `/pitch` endpoint). Same shape as `stem_separation` / `lyric_transcription`. Omitted for hand-edited pitch tracks |
|
||||||
|
| `cover` | string | no | Path to cover image |
|
||||||
|
| `preview` | string | no | Path to a short preview audio clip (OGG) at the sloppak root. Populated when the source carries a separate short browser-preview clip (decoded to `preview.ogg`); absent otherwise. Consumed by [`slopsmith-plugin-song-preview`](https://github.com/byrongamatos/slopsmith-plugin-song-preview) for hover-to-listen previews in the library |
|
||||||
|
| `song_timeline` | string | no | Path to a `song_timeline.json` file carrying song-wide beats and sections (see §5.3). When present, its data takes priority over any beats/sections embedded in arrangement JSONs. Older readers ignore the key and fall back to reading beats/sections from the first arrangement JSON as before |
|
||||||
|
| `drum_tab` | string | no | Path to `drum_tab.json` — per-piece drum hits (see §5.3). Implemented end-to-end as of slopsmith#344 |
|
||||||
|
|
||||||
|
Unknown keys are **silently ignored** by the loader. This is deliberate — it's the extensibility hook (see §5).
|
||||||
|
|
||||||
|
### 2.1. `arrangements[]`
|
||||||
|
|
||||||
|
Each entry describes one playable arrangement and points at its JSON file:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
arrangements:
|
||||||
|
- id: lead # filesystem-safe stable ID, used for filenames
|
||||||
|
name: Lead # display name (Lead/Rhythm/Bass/Combo are sorted first)
|
||||||
|
file: arrangements/lead.json
|
||||||
|
tuning: [0, 0, 0, 0, 0, 0] # six semitone offsets from E A D G B E
|
||||||
|
capo: 0
|
||||||
|
centOffset: 0.0 # optional float, cents; default 0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
- `tuning` is a list of semitone offsets from standard `E2 A2 D2 G3 B3 E4`. **Six elements is the standard six-string convention** and the only length `lib/tunings.py` produces friendly names for; 5- and 7-string content is accepted by the loader and falls through to a numeric label. For bass, the four bass strings are at indices 0–3; the other two slots are `0`. Consumers should not hard-code `len(tuning) == 6`.
|
||||||
|
- `name` controls the sort order in the UI: `Lead > Combo > Rhythm > Bass > everything else`.
|
||||||
|
- `centOffset` is a pitch-shift value in cents. Commonly `-1200.0` for extended-range bass arrangements tuned one octave down; small non-zero values for songs mastered at a non-A440 reference pitch (e.g. A443 ≈ +11.8 cents). Absent / `0.0` means no shift. Exposed to plugins via `getSongInfo().centOffset`.
|
||||||
|
- Manifest-level `tuning`, `capo`, and `centOffset` **override** anything embedded in the arrangement JSON. The arrangement JSON's own values are fallbacks.
|
||||||
|
- `notation` (optional string) — path to a `notation_<id>.json` file carrying standard musical notation data for this arrangement (see §5.3). When present, the loader surfaces it on `LoadedSloppak.notation_by_id[id]` and the highway WS streams `notation_info` + `notation_measures` messages. The `file:` key may be omitted when `notation:` is present — the loader creates a stub arrangement so the notation file can be the sole data source.
|
||||||
|
|
||||||
|
### 2.2. `stems[]`
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
stems:
|
||||||
|
- id: full
|
||||||
|
file: stems/full.ogg
|
||||||
|
default: true # plays by default when the song opens
|
||||||
|
- id: guitar
|
||||||
|
file: stems/guitar.ogg
|
||||||
|
default: true
|
||||||
|
- id: drums
|
||||||
|
file: stems/drums.ogg
|
||||||
|
default: false
|
||||||
|
```
|
||||||
|
|
||||||
|
- `id` is referenced by the Stems plugin and any other consumer; keep it stable.
|
||||||
|
- `default` accepts `true`/`false`, or strings (`"on"`/`"off"`/`"true"`/etc.) for hand-edited manifests.
|
||||||
|
- A freshly converted sloppak from `lib/sloppak_convert.py` starts with a single `{id: full, file: stems/full.ogg, ...}` entry. After stem-splitting (Demucs), `full.ogg` is removed and the manifest is rewritten with per-instrument entries (`guitar`, `bass`, `drums`, `vocals`, `other`). The format requires only that `stems` is non-empty — there's no specific filename or id that must always be present.
|
||||||
|
|
||||||
|
When stems were produced by an automated separation engine (Demucs), an optional `stem_separation` block records which engine + model produced them. Per [slopsmith#357](https://github.com/byrongamatos/slopsmith/issues/357):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
stem_separation:
|
||||||
|
engine: demucs # stable engine id; only `demucs` today
|
||||||
|
model: htdemucs_6s # specific model name (htdemucs_6s / htdemucs_ft / htdemucs / mdx_extra / ...)
|
||||||
|
version: 1.0.0 # semver for slopsmith's stem-artifact contract
|
||||||
|
```
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
- `engine` — stable identifier for the separation engine. Currently always `demucs`. New engines (e.g. a hypothetical `spleeter`) would get their own stable id.
|
||||||
|
- `model` — the engine-specific model id used for this split. For Demucs this is the `-n` flag value.
|
||||||
|
- `version` — semver for Slopsmith's stem-artifact contract (independent of upstream Demucs / model versions). Bump per the same semantics #357 defines: patch = metadata-only fixes, minor = backward-compatible additions, major = stem set / packing / post-processing changed and existing splits should be regenerated.
|
||||||
|
|
||||||
|
Omitted for single-stem sloppaks (`stems: [{id: full, ...}]` — no automated separation ran) and for hand-edited / user-recorded stems. The RFC reserves a separate `stem_authoring` sibling block for the hand-edit case; that's deferred to a follow-up.
|
||||||
|
|
||||||
|
A remote Demucs server can use this block as part of a cache key so that changing the model or major version naturally produces a cache miss. Local plugin jobs should preserve this metadata in job state and in any copied/downloaded manifests.
|
||||||
|
|
||||||
|
### 2.3. `lyrics`
|
||||||
|
|
||||||
|
If present, points at a JSON file containing a flat list of syllable objects:
|
||||||
|
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{"t": 12.34, "d": 0.18, "w": "Hel"},
|
||||||
|
{"t": 12.52, "d": 0.22, "w": "lo-"},
|
||||||
|
{"t": 13.10, "d": 0.30, "w": "world"}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| `t` | Time in seconds |
|
||||||
|
| `d` | Duration in seconds |
|
||||||
|
| `w` | Syllable text. Trailing `-` joins to the next syllable as one word; trailing `+` marks the last syllable of a line (renderer wraps after it). Both are suffixes on a real syllable — not standalone entries. See `static/highway.js` for the rendering: `raw.endsWith('+')` flags end-of-line, and `sylText` strips the trailing marker before drawing |
|
||||||
|
|
||||||
|
When lyrics are present, the optional top-level `lyrics_source` key records where they came from. The assembler sets it to `xml` when the lyrics were parsed from the source chart's vocals XML; the WhisperX auto-transcription fallback (`scripts/transcribe_lyrics.py`, or `--auto-lyrics` on the split scripts) sets it to `whisperx`. Hand-edited lyrics should bump it to `user` so UI consumers can render a different badge (or no badge) than for machine-generated lyrics. The key is absent on sloppaks produced before this field existed — readers should treat missing as `xml` for backward compatibility.
|
||||||
|
|
||||||
|
When `lyrics_source` is `whisperx` (or any future automated engine), an optional `lyric_transcription` block records which engine + model produced the file. Shape mirrors the parent `stem_separation` RFC ([slopsmith#357](https://github.com/byrongamatos/slopsmith/issues/357)):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
lyric_transcription:
|
||||||
|
engine: whisperx # stable engine id
|
||||||
|
model: medium # the WhisperX model size that ran (tiny/base/small/medium/large-v2/large-v3)
|
||||||
|
version: 1.0.0 # semver for slopsmith's lyric-transcription artifact contract
|
||||||
|
```
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
- `engine` — stable identifier for the transcription engine; currently always `whisperx`.
|
||||||
|
- `model` — the engine-specific model id used for this transcription.
|
||||||
|
- `version` — semver for Slopsmith's lyric-transcription artifact contract (independent of upstream Whisper / WhisperX versions). Bump per the same semantics #357 defines for stems: patch = metadata-only fixes, minor = backward-compatible additions, major = output shape changed and existing transcriptions should be regenerated.
|
||||||
|
|
||||||
|
Omitted for authored lyrics (`xml` / `user`). A remote WhisperX server can use this block as part of a cache key the same way #357 envisions for stems — caches should miss whenever any of the three fields change, ensuring stale transcriptions don't get returned after a model bump.
|
||||||
|
|
||||||
|
### 2.4. `vocal_pitch`
|
||||||
|
|
||||||
|
If present, points at a JSON file holding per-syllable pitch data — the karaoke companion to `lyrics`. Consumed by [slopsmith-plugin-lyrics-karaoke](https://github.com/byrongamatos/slopsmith-plugin-lyrics-karaoke) to render karaoke-style note bars over the lyric text. Shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"notes": [
|
||||||
|
{"t": 12.34, "d": 0.40, "midi": 64},
|
||||||
|
{"t": 12.78, "d": 0.55, "midi": 67}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| `version` | Schema version of this `vocal_pitch.json` file (currently the integer `1`). Bump on a breaking change to the `notes` entry shape. This is *not* the same as the top-level `pitch_extraction.version` block below, which is a semver string used as a cache-key for the extractor engine |
|
||||||
|
| `notes` | List of pitch entries, one per syllable that the extractor could lock onto. `t` + `d` mirror the matching `lyrics.json` entry; `midi` is the MIDI note number (60 = middle C). Syllables the extractor couldn't pitch (silent / sub-confidence) are omitted from this list — it may be shorter than `lyrics.json` |
|
||||||
|
|
||||||
|
When pitch came from an automated engine (the demucs server's `/pitch` endpoint, which runs CREPE), the optional top-level `pitch_extraction` block records which engine + model produced the file. Same shape and semver-string semantics as `stem_separation` / `lyric_transcription` — distinct from the in-file integer `version` field above:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
pitch_extraction:
|
||||||
|
engine: crepe
|
||||||
|
model: v1
|
||||||
|
version: 1.0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
Omitted for hand-edited pitch tracks. As with the other two automated-artifact blocks, a remote pitch server can use this for cache-key invalidation.
|
||||||
|
|
||||||
|
The sloppak assembler runs pitch extraction automatically when `pitch_extraction.enabled` is set in its config AND a server URL is configured (either `pitch_extraction.server_url` or the shared `demucs_server_url`) AND the sloppak has lyrics + a `stems/vocals.ogg` after the split pass — either because `_maybe_transcribe_lyrics` just produced them via WhisperX OR because they were already on disk (from the source chart's vocals XML, hand-authoring, or an earlier build). Pitch is *not* coupled to `whisperx.enabled` — setting `pitch_extraction.enabled=true` alone (with WhisperX off) is enough to retro-generate pitch over any existing on-disk lyrics. Sloppaks built before this field existed simply don't carry it — readers should treat missing `vocal_pitch` as "no pitch data, fall back to whatever the karaoke plugin's local-extraction path produces (if any)".
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Arrangement JSON — the wire format
|
||||||
|
|
||||||
|
Arrangement JSON files use the **wire format** produced by `arrangement_to_wire()` — the on-disk representation of a complete arrangement. Slopsmith's `/ws/highway/{filename}` endpoint transports similar data as a sequence of typed messages (`notes`, `chords`, `anchors`, `chord_templates`, `phrases`, …) rather than as one identical top-level JSON object. In practice, the WebSocket stream reuses the same per-object field names where applicable, but it should not be treated as a byte-for-byte match for `arrangements/*.json`.
|
||||||
|
|
||||||
|
The authoritative serializer/deserializer is in [lib/song.py](../lib/song.py):
|
||||||
|
|
||||||
|
- `arrangement_to_wire(arr) → dict` — write
|
||||||
|
- `arrangement_from_wire(dict) → Arrangement` — read
|
||||||
|
|
||||||
|
### 3.1. Top-level shape
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "Lead",
|
||||||
|
"tuning": [0, 0, 0, 0, 0, 0],
|
||||||
|
"capo": 0,
|
||||||
|
"centOffset": 0.0, /* optional, float cents, default 0.0 */
|
||||||
|
"notes": [ /* see 3.2 */ ],
|
||||||
|
"chords": [ /* see 3.3 */ ],
|
||||||
|
"anchors": [ /* see 3.4 */ ],
|
||||||
|
"handshapes": [ /* see 3.5 */ ],
|
||||||
|
"templates": [ /* see 3.6 */ ],
|
||||||
|
"phrases": [ /* optional, see 3.7 */ ],
|
||||||
|
"tones": { /* optional, see 3.9 */ },
|
||||||
|
"beats": [ /* see 3.8, only on first arrangement */ ],
|
||||||
|
"sections": [ /* see 3.8, only on first arrangement */ ]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`beats` and `sections` are **song-level** but live on the first arrangement's JSON for legacy reasons — `lib/sloppak.py` hoists them to the `Song` object on load. If you author multiple arrangements, only put them in one file. **New sloppaks should use `song_timeline.json` instead** (see §2 and §5.3) — when the manifest carries a `song_timeline:` key pointing at a schema-valid file, its beats/sections **replace** whatever the arrangement JSONs loaded (the override is applied after arrangement loading, so a valid `song_timeline.json` always wins). Arrangement-JSON beats/sections remain supported for backward compatibility with all existing sloppaks and are the fallback when the file is absent or invalid.
|
||||||
|
|
||||||
|
### 3.2. Notes
|
||||||
|
|
||||||
|
Field names are short on purpose — these get streamed thousands of times per song. Don't expand them.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"t": 12.345, // time (s)
|
||||||
|
"s": 2, // string (0 = lowest)
|
||||||
|
"f": 7, // fret (0 = open, 24 = max)
|
||||||
|
"sus": 0.5, // sustain (s, 0 = none)
|
||||||
|
"sl": 9, // pitched slide-to fret (-1 = no slide)
|
||||||
|
"slu": -1, // unpitched slide-to fret (-1 = no slide)
|
||||||
|
"bn": 1.0, // bend amount in semitones
|
||||||
|
"ho": false, // hammer-on
|
||||||
|
"po": false, // pull-off
|
||||||
|
"hm": false, // natural harmonic
|
||||||
|
"hp": false, // pinch harmonic
|
||||||
|
"pm": false, // palm mute
|
||||||
|
"mt": false, // string mute
|
||||||
|
"vb": false, // vibrato
|
||||||
|
"tr": false, // tremolo
|
||||||
|
"ac": false, // accent
|
||||||
|
"tp": false, // tap
|
||||||
|
"ln": false, // link-next (chord linking metadata; renderers may ignore — runtime linking is derived from proximity)
|
||||||
|
"fhm": false, // fret-hand mute
|
||||||
|
"plk": false, // pluck (pop, bass)
|
||||||
|
"slp": false, // slap (bass)
|
||||||
|
"rh": -1, // right-hand fingering (-1 = unset)
|
||||||
|
"pkd": -1, // pick direction (-1 = unset, 0 = down, 1 = up)
|
||||||
|
"ig": false // ignore (chart-author flag — note is rendered but not scored / sequenced)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Default values: numbers → `0` or `-1` (slides / `rh` / `pkd`), bools → `false`. Omit fields equal to their default if you're authoring by hand — the parser fills them in. **Encoders should default-omit the newer technique keys** (`ln`, `fhm`, `plk`, `slp`, `rh`, `pkd`, `ig`) — the highway streams notes thousands of times per song, so trimming the common case keeps the WebSocket payload tight. The pre-existing keys are still emitted unconditionally to preserve the legacy wire contract.
|
||||||
|
|
||||||
|
### 3.3. Chords
|
||||||
|
|
||||||
|
A chord groups note-shaped objects under a single time:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"t": 30.0,
|
||||||
|
"id": 12, // index into templates[]
|
||||||
|
"hd": false, // high-density flag
|
||||||
|
"notes": [
|
||||||
|
{"s": 0, "f": 3, "sus": 0.0, ...},
|
||||||
|
{"s": 1, "f": 5, "sus": 0.0, ...}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Chord notes use the same field set as standalone notes, **except `t` is omitted** (the chord carries the time). The fingering / shape lookup is `chord.id → templates[id]`.
|
||||||
|
|
||||||
|
### 3.4. Anchors
|
||||||
|
|
||||||
|
Where the fretting hand sits. Drives the highway zoom box.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"time": 12.0, "fret": 5, "width": 4}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.5. Hand shapes
|
||||||
|
|
||||||
|
Spans during which a chord shape is held:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"chord_id": 12, "start_time": 30.0, "end_time": 31.5, "arp": false}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `chord_id` (`int`, default `0`) — index into `templates[]`; identifies which chord template the span is holding.
|
||||||
|
- `start_time` (`float`, default `0.0`) — start of the span in seconds.
|
||||||
|
- `end_time` (`float`, default `0.0`) — end of the span in seconds.
|
||||||
|
- `arp` (`bool`, default `false`, allowed values `true`/`false`) — whether this hand shape should be treated as an arpeggio span rather than a fully-strummed chord hold.
|
||||||
|
|
||||||
|
### 3.6. Chord templates
|
||||||
|
|
||||||
|
Named shapes referenced by `chord.id` and `handshape.chord_id`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "Em7",
|
||||||
|
"displayName": "Em7",
|
||||||
|
"arp": false,
|
||||||
|
"fingers": [-1, 2, 1, -1, -1, -1],
|
||||||
|
"frets": [ 0, 2, 2, 0, 0, 0]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `name` (`string`, default `""`) — canonical template name used by the parser / authoring data.
|
||||||
|
- `displayName` (`string`, default `name`) — label shown in the UI; source XML may use this for display-specific variants such as `-arp`.
|
||||||
|
- `arp` (`bool`, default `false`, allowed values `true`/`false`) — whether the template is flagged as arpeggiated. Parsed from explicit XML attributes (`arpeggio` / `arp`, any common casing) or inferred from `displayName` markers such as `-arp`.
|
||||||
|
- `fingers` (`int[6]`, default `[-1, -1, -1, -1, -1, -1]`) — fretting-hand finger numbers, lowest string first. `-1` = unused string, `0` = open string / no fretting finger, `1..4` = index/middle/ring/pinky.
|
||||||
|
- `frets` (`int[6]`, default `[-1, -1, -1, -1, -1, -1]`) — fret numbers, lowest string first. `-1` = unused string, `0` = open string, positive values = fretted note.
|
||||||
|
|
||||||
|
### 3.7. Phrases (optional, multi-difficulty data)
|
||||||
|
|
||||||
|
Sources that carry per-phrase difficulty ladders (phrase-aware arrangement XML) include this. GP imports and legacy sloppaks omit it:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"phrases": [
|
||||||
|
{
|
||||||
|
"start_time": 0.0,
|
||||||
|
"end_time": 12.5,
|
||||||
|
"max_difficulty": 4,
|
||||||
|
"levels": [
|
||||||
|
{ "difficulty": 0, "notes": [...], "chords": [...], "anchors": [...], "handshapes": [...] },
|
||||||
|
{ "difficulty": 1, "notes": [...], "chords": [...], "anchors": [...], "handshapes": [...] },
|
||||||
|
...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
If you're writing a converter that doesn't have multi-difficulty data, **omit the `phrases` key entirely** (don't emit `"phrases": []`). A missing key signals "no ladder, disable the master-difficulty slider"; an empty list is the same in current code but reads ambiguously.
|
||||||
|
|
||||||
|
### 3.8. Beats and sections
|
||||||
|
|
||||||
|
```json
|
||||||
|
"beats": [{"time": 0.5, "measure": 1}, {"time": 1.0, "measure": -1}, ...],
|
||||||
|
"sections": [{"name": "verse", "number": 1, "time": 12.5}, ...]
|
||||||
|
```
|
||||||
|
|
||||||
|
`measure: -1` = sub-beat (not a downbeat). Section `name` follows the usual song-structure conventions (`intro`, `verse`, `chorus`, `bridge`, `solo`, `outro`, …).
|
||||||
|
|
||||||
|
### 3.9. Tones (optional)
|
||||||
|
|
||||||
|
`tones` carries the arrangement's guitar tones — the amp/pedal/cabinet gear and the in-song tone switches. It's populated when the source chart carries tone data (`lib/tones.py`); a sloppak authored from scratch may omit it entirely.
|
||||||
|
|
||||||
|
```json
|
||||||
|
"tones": {
|
||||||
|
"base": "Clean Rhythm",
|
||||||
|
"changes": [
|
||||||
|
{"t": 12.5, "name": "Lead Drive"},
|
||||||
|
{"t": 48.0, "name": "Clean Rhythm"}
|
||||||
|
],
|
||||||
|
"definitions": [
|
||||||
|
{
|
||||||
|
"Name": "Clean Rhythm",
|
||||||
|
"Key": "Tone_A",
|
||||||
|
"GearList": { /* raw gear blocks: Amp, PrePedal1-4, … */ }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `base` (string) — the tone in effect before the first change.
|
||||||
|
- `changes` (list, time-sorted) — `{"t": seconds, "name": str}` tone switches. The highway draws a marker at each. Omit when the arrangement never switches tone.
|
||||||
|
- `definitions` (list) — the **raw tone objects** (`Name`, `Key`, `GearList`), copied verbatim from the source chart's tone manifest. The Tones plugin parses these into the rendered signal chain (it owns the gear-name/image map, so the data is stored unparsed here).
|
||||||
|
|
||||||
|
All three sub-keys are individually optional; an arrangement with none of them simply omits `tones`. Readers that don't know about tones ignore the key (the loader preserves it verbatim).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Reading and writing sloppaks programmatically
|
||||||
|
|
||||||
|
### 4.1. Reading (Python, server-side)
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pathlib import Path
|
||||||
|
from sloppak import load_song, load_manifest
|
||||||
|
|
||||||
|
# Quick metadata only (parses manifest, skips arrangement JSONs)
|
||||||
|
manifest = load_manifest(Path("song.sloppak"))
|
||||||
|
|
||||||
|
# Full song load (manifest + all arrangements + lyrics)
|
||||||
|
loaded = load_song("song.sloppak", dlc_root=Path("/dlc"), unpack_cache_root=Path("/cache"))
|
||||||
|
print(loaded.song.title, len(loaded.song.arrangements))
|
||||||
|
print(loaded.stems) # [{"id": "full", "file": "stems/full.ogg", "default": True}]
|
||||||
|
print(loaded.manifest) # raw dict — read your custom keys here
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2. Writing (Python, server-side)
|
||||||
|
|
||||||
|
There's no general-purpose writer in `lib/` yet. The current writer lives in [lib/sloppak_convert.py](../lib/sloppak_convert.py) inside the sloppak assembly function — it's the single source of truth for "how a sloppak gets built." If you need to write sloppaks from a new source, copy the structure of that function:
|
||||||
|
|
||||||
|
1. Build a `work_dir/` in temp.
|
||||||
|
2. Write `arrangements/{id}.json` per arrangement using `arrangement_to_wire()`.
|
||||||
|
3. Encode audio to OGG into `stems/`.
|
||||||
|
4. Optionally write `lyrics.json`, `cover.jpg`.
|
||||||
|
5. Compose the `manifest` dict and dump as YAML with `yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True)`.
|
||||||
|
6. Either `shutil.copytree(work_dir, out)` for directory form, or `_zip_dir(work_dir, out)` for zip form.
|
||||||
|
|
||||||
|
Always use `yaml.safe_dump` (not `yaml.dump`) and pass `sort_keys=False` so the human-readable order is preserved.
|
||||||
|
|
||||||
|
### 4.3. Reading (JavaScript, plugin-side)
|
||||||
|
|
||||||
|
Plugins typically don't read the sloppak file directly — they consume the `/ws/highway/{filename}` WebSocket stream (see `CLAUDE.md` for the message protocol), which produces the same shapes. If you specifically need raw manifest access from the browser, expose it through a custom backend route in your plugin's `routes.py` and fetch it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Extending the format — adding new data
|
||||||
|
|
||||||
|
Sloppak is designed to be extended without breaking older readers. The conventions below come from how `lyrics`, `stems`, and the optional `phrases` ladder were each added.
|
||||||
|
|
||||||
|
### 5.1. The golden rule: **manifest opt-in, file off to the side**
|
||||||
|
|
||||||
|
New data types should follow this pattern:
|
||||||
|
|
||||||
|
1. **Drop a new file** alongside the standard ones (e.g., `drums.json`, `keys.json`, `lighting.json`).
|
||||||
|
2. **Add a manifest key** that *points at* that file (e.g., `drum_tab: drums.json`).
|
||||||
|
3. **Make consumers gate on the manifest key**: if the key is absent, do nothing. Never auto-discover by filename — that breaks the "manifest is the index" rule.
|
||||||
|
|
||||||
|
So a sloppak with drum tabs would look like:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# manifest.yaml
|
||||||
|
title: "Song"
|
||||||
|
artist: "Band"
|
||||||
|
duration: 240.0
|
||||||
|
arrangements: [...]
|
||||||
|
stems: [...]
|
||||||
|
drum_tab: drum_tab.json # ← new key
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
my-song.sloppak/
|
||||||
|
├── manifest.yaml
|
||||||
|
├── arrangements/...
|
||||||
|
├── stems/...
|
||||||
|
└── drum_tab.json # ← new file
|
||||||
|
```
|
||||||
|
|
||||||
|
Older Slopsmith readers ignore the unknown `drum_tab` key (the loader uses `manifest.get("drum_tab")` / unknown keys pass through). Your plugin checks for it and renders accordingly. **Zero coordination needed with core.**
|
||||||
|
|
||||||
|
### 5.2. Naming conventions for new keys and files
|
||||||
|
|
||||||
|
- **Manifest keys**: `snake_case`, descriptive, singular when the value is one thing (`lyrics`, `cover`, `drum_tab`), plural when it's a list (`stems`, `arrangements`).
|
||||||
|
- **File names**: lowercase, hyphenated or underscored, JSON for structured data, OGG for audio, JPG/PNG for images.
|
||||||
|
- **Inside JSON**: short field names for hot-path data that gets streamed thousands of times (`t`, `s`, `f` — see §3.2). Long names are fine for one-off metadata.
|
||||||
|
- **Time fields**: always `t` or `time` (not `start`, not `timestamp`) — and always **seconds as floats**, not ms or ticks. Be consistent with the existing wire format.
|
||||||
|
- **Indexes / IDs**: stable, filesystem-safe, lowercase. Don't reuse a source format's internal numeric IDs unless you have to.
|
||||||
|
|
||||||
|
### 5.3. Worked examples for the kinds of additions you mentioned
|
||||||
|
|
||||||
|
#### Drum tab
|
||||||
|
|
||||||
|
`drum_tab.json` carries per-piece hits authored on top of the song's audio.
|
||||||
|
Implemented end-to-end as of slopsmith#344 (drums-from-scratch): the loader
|
||||||
|
in `lib/sloppak.py` parses it, `lib/drums.py` defines the canonical piece-id
|
||||||
|
vocabulary, and `/ws/highway/{filename}` streams it as `drum_tab` + chunked
|
||||||
|
`drum_hits` messages.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"name": "Drums",
|
||||||
|
"kit": [
|
||||||
|
{"id": "kick", "name": "Kick"},
|
||||||
|
{"id": "snare", "name": "Snare"},
|
||||||
|
{"id": "hh_closed", "name": "Hi-hat (closed)"},
|
||||||
|
{"id": "hh_open", "name": "Hi-hat (open)"},
|
||||||
|
{"id": "crash_r", "name": "Crash (right)"},
|
||||||
|
{"id": "ride", "name": "Ride"}
|
||||||
|
],
|
||||||
|
"hits": [
|
||||||
|
{"t": 0.500, "p": "kick", "v": 110},
|
||||||
|
{"t": 0.750, "p": "snare", "v": 92},
|
||||||
|
{"t": 0.750, "p": "hh_closed", "v": 70},
|
||||||
|
{"t": 1.000, "p": "snare", "v": 60, "g": true},
|
||||||
|
{"t": 1.250, "p": "snare", "v": 105, "f": true},
|
||||||
|
{"t": 4.000, "p": "crash_r", "v": 120, "k": 0.080}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Manifest:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
drum_tab: drum_tab.json
|
||||||
|
```
|
||||||
|
|
||||||
|
##### Hit fields
|
||||||
|
|
||||||
|
| key | type | meaning |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `t` | float seconds | hit time, required, monotonic in `hits[]` |
|
||||||
|
| `p` | string | piece-id from the closed list below; required |
|
||||||
|
| `v` | int 1-127 | velocity (default 100) |
|
||||||
|
| `g` | bool | ghost note (renders smaller / outline-only) |
|
||||||
|
| `f` | bool | flam (renders a small leading ghost glyph 30 ms early) |
|
||||||
|
| `k` | float seconds | cymbal-choke tail duration (renders a fade-out) |
|
||||||
|
|
||||||
|
##### Canonical piece-id vocabulary
|
||||||
|
|
||||||
|
A closed list lives in `lib/drums.py::PIECES`. Open/closed hi-hat are
|
||||||
|
**distinct piece-ids**, not articulation flags — hit detection must reject
|
||||||
|
a closed-hat strike on an open-hat note, which it can only do if the
|
||||||
|
articulation is part of the piece-id.
|
||||||
|
|
||||||
|
| piece-id | category | default GM MIDI | default shape |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `kick` | kick | 35, 36 | bar (full-width across all non-kick lanes) |
|
||||||
|
| `snare` | drum | 38, 40 | rectangle |
|
||||||
|
| `snare_xstick` | drum | 37 | hatched rectangle |
|
||||||
|
| `tom_hi` | drum | 50, 48 | rectangle |
|
||||||
|
| `tom_mid` | drum | 47, 45 | rectangle |
|
||||||
|
| `tom_low` | drum | 43 | rectangle |
|
||||||
|
| `tom_floor` | drum | 41 | rectangle |
|
||||||
|
| `hh_closed` | cymbal | 42 | filled circle |
|
||||||
|
| `hh_open` | cymbal | 46 | ring (outline) circle |
|
||||||
|
| `hh_pedal` | cymbal | 44 | small circle with × |
|
||||||
|
| `stack` | cymbal | 30 | jagged circle (no GM standard — reuses 30 from extended-percussion range) |
|
||||||
|
| `crash_l` | cymbal | 49 | circle |
|
||||||
|
| `crash_r` | cymbal | 57 | circle |
|
||||||
|
| `splash` | cymbal | 55 | small circle |
|
||||||
|
| `china` | cymbal | 52 | jagged circle |
|
||||||
|
| `ride` | cymbal | 51, 59 | circle |
|
||||||
|
| `ride_bell` | cymbal | 53 | circle with centre dot |
|
||||||
|
| `bell` | cymbal | 80 | circle with centre dot (no GM standard — reuses "Mute Triangle") |
|
||||||
|
|
||||||
|
Unknown piece-ids round-trip through the loader (forward-compat); the
|
||||||
|
client just renders them as a default rectangle.
|
||||||
|
|
||||||
|
##### Wire format
|
||||||
|
|
||||||
|
Streamed as two highway-WS message types:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "type": "drum_tab", "version": 1, "name": "Drums",
|
||||||
|
"kit": [{"id": "kick", "name": "Kick"}, ...], "total": 1234 }
|
||||||
|
```
|
||||||
|
|
||||||
|
…followed by one or more chunks of 500 hits:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "type": "drum_hits", "data": [{"t": 0.5, "p": "kick", "v": 110}, ...],
|
||||||
|
"total": 1234 }
|
||||||
|
```
|
||||||
|
|
||||||
|
##### Design notes
|
||||||
|
|
||||||
|
- `kit[]` is the legend — fixed metadata, separated from hot-path data.
|
||||||
|
- `hits[]` uses short field names because this list can be thousands long.
|
||||||
|
- `v` defaults to 100; ghost / flam / choke flags are all optional.
|
||||||
|
- Older sloppaks whose drums are encoded as guitar notes (`midi = string*24 + fret`) still play — the drums plugin keeps a legacy decoder that reads the standard `notes` stream and synthesises `drum_hits` from it.
|
||||||
|
|
||||||
|
#### Song timeline (beats and sections as a top-level file)
|
||||||
|
|
||||||
|
`song_timeline.json` moves song-wide beats and sections out of the first
|
||||||
|
arrangement JSON and into a dedicated file. Implemented in `lib/sloppak.py`
|
||||||
|
alongside the notation format: the loader reads the manifest's optional
|
||||||
|
`song_timeline:` key, validates the file, and populates `Song.beats` /
|
||||||
|
`Song.sections` from it, taking priority over any beats/sections embedded
|
||||||
|
in arrangement JSONs.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"beats": [
|
||||||
|
{"time": 0.500, "measure": 1},
|
||||||
|
{"time": 1.000, "measure": -1},
|
||||||
|
{"time": 1.500, "measure": -1},
|
||||||
|
{"time": 2.000, "measure": 2}
|
||||||
|
],
|
||||||
|
"sections": [
|
||||||
|
{"name": "intro", "number": 1, "time": 0.0},
|
||||||
|
{"name": "verse", "number": 1, "time": 16.0},
|
||||||
|
{"name": "chorus", "number": 1, "time": 32.0}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Manifest:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
song_timeline: song_timeline.json
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field in `beats[]` | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `time` | float seconds | Beat timestamp. Matches the existing arrangement-JSON wire convention |
|
||||||
|
| `measure` | int | 1-based downbeat number. `-1` = sub-beat (not a downbeat) |
|
||||||
|
|
||||||
|
| Field in `sections[]` | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `name` | string | song-structure convention: `intro`, `verse`, `chorus`, `bridge`, `solo`, `outro`, … |
|
||||||
|
| `number` | int | Section repeat number |
|
||||||
|
| `time` | float seconds | Section start |
|
||||||
|
|
||||||
|
**Backward compatibility.** Sloppaks without `song_timeline:` continue to
|
||||||
|
work — the loader falls through to reading beats/sections from the first
|
||||||
|
arrangement JSON exactly as before. No migration is needed.
|
||||||
|
|
||||||
|
**New sloppaks** should put beats/sections here and leave arrangement JSONs
|
||||||
|
free of timeline data. This is especially important for notation-only
|
||||||
|
arrangements (see below) where there may be no arrangement JSON at all.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Notation format (standard musical notation per arrangement)
|
||||||
|
|
||||||
|
The notation format promotes keys, piano, violin, and any other
|
||||||
|
staff-notation instrument to first-class status with their own data
|
||||||
|
structure, separate from the guitar wire format. Implemented in
|
||||||
|
`lib/sloppak.py` and `lib/notation.py`; the highway WS streams
|
||||||
|
`notation_info` + `notation_measures` messages when notation data is
|
||||||
|
present for the active arrangement.
|
||||||
|
|
||||||
|
**Architecture: per-arrangement, not song-wide.** Unlike `drum_tab`
|
||||||
|
(one drum track per song, top-level manifest key), notation is
|
||||||
|
per-instrument. A song could carry both `notation_keys.json` and
|
||||||
|
`notation_violin.json`. The manifest key lives on the **arrangement
|
||||||
|
entry**, not at the top level.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
arrangements:
|
||||||
|
- id: keys
|
||||||
|
name: Keys
|
||||||
|
type: piano
|
||||||
|
notation: notation_keys.json # per-arrangement sub-key
|
||||||
|
# file: is optional when notation: is present
|
||||||
|
```
|
||||||
|
|
||||||
|
```text
|
||||||
|
my-song.sloppak/
|
||||||
|
├── manifest.yaml
|
||||||
|
├── song_timeline.json
|
||||||
|
├── notation_keys.json
|
||||||
|
└── stems/
|
||||||
|
└── full.ogg
|
||||||
|
```
|
||||||
|
|
||||||
|
**`notation_<id>.json` — file schema:**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"instrument": "piano",
|
||||||
|
"staves": [
|
||||||
|
{"id": "rh", "clef": "G2", "label": "Right Hand"},
|
||||||
|
{"id": "lh", "clef": "F4", "label": "Left Hand"}
|
||||||
|
],
|
||||||
|
"measures": [
|
||||||
|
{
|
||||||
|
"idx": 1,
|
||||||
|
"t": 0.0,
|
||||||
|
"ts": [4, 4],
|
||||||
|
"ks": 0,
|
||||||
|
"tempo": 120.0,
|
||||||
|
"staves": {
|
||||||
|
"rh": {
|
||||||
|
"voices": [{"v": 1, "beats": [
|
||||||
|
{"t": 0.000, "dur": 4, "notes": [{"midi": 64}]},
|
||||||
|
{"t": 0.500, "dur": 4, "notes": [{"midi": 67}]}
|
||||||
|
]}]
|
||||||
|
},
|
||||||
|
"lh": {
|
||||||
|
"voices": [{"v": 1, "beats": [
|
||||||
|
{"t": 0.000, "dur": 1, "notes": [{"midi": 52}, {"midi": 60}]}
|
||||||
|
]}]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Top-level fields:**
|
||||||
|
|
||||||
|
| Field | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `version` | int | Always `1`. Bump on breaking schema change |
|
||||||
|
| `instrument` | string | Mirrors arrangement `type`: `piano`, `violin`, `guitar`, etc. Makes the file self-describing |
|
||||||
|
| `rights` | string | Optional copyright / rights text (MusicXML `<rights>`). Omit when absent |
|
||||||
|
| `lyricist` | string | Optional lyricist credit (MusicXML `<creator type="lyricist">`). Omit when absent |
|
||||||
|
| `arranger` | string | Optional arranger credit (MusicXML `<creator type="arranger">`). Omit when absent |
|
||||||
|
| `staves` | list | Static staff definitions. Each has `id` (stable, referenced by `measures[].staves` keys), `clef` (see below), and optional `label` |
|
||||||
|
| `measures` | list | Ordered measure data — the hot path |
|
||||||
|
|
||||||
|
**Clef vocabulary** (defined in `lib/notation.py::CLEFS`):
|
||||||
|
|
||||||
|
| Value | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| `G2` | Treble clef — guitar, violin, flute, piano RH |
|
||||||
|
| `F4` | Bass clef — bass guitar, cello, piano LH |
|
||||||
|
| `C3` | Alto clef — viola |
|
||||||
|
| `C4` | Tenor clef — cello upper register, trombone |
|
||||||
|
| `neutral` | Unpitched / percussion staff |
|
||||||
|
|
||||||
|
**Measure fields:**
|
||||||
|
|
||||||
|
| Field | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `idx` | int | 1-based measure number |
|
||||||
|
| `t` | float | Time in seconds at measure downbeat |
|
||||||
|
| `ts` | int[2] | Time signature `[numerator, denominator]`. Omit if unchanged |
|
||||||
|
| `beat_groups` | int[] | Beat grouping for compound and irregular meters, as a list of integers. Each integer is the count of time-signature denominator units in that primary beat group. The sum must equal the time-signature numerator. E.g. 6/8 → `[3, 3]`; 9/8 → `[3, 3, 3]`; 7/8 → `[2, 2, 3]`; 5/8 → `[2, 3]` or `[3, 2]`. Omit for simple meters (2/4, 3/4, 4/4) where grouping is unambiguous. Renderers translate this to their own beam-grouping API at render time — this field is renderer-agnostic. |
|
||||||
|
| `ks` | int | Key signature: semitones from C, −7 to +7 (negative = flats, positive = sharps). Omit if unchanged |
|
||||||
|
| `tempo` | float | BPM. Omit if unchanged |
|
||||||
|
| `pickup` | bool | `true` when this measure is an anacrusis (pickup / upbeat) shorter than the time signature implies (MusicXML `implicit="yes"`). Renderers suppress the measure number and start counting from the next full measure. Omit when false |
|
||||||
|
| `staves` | object | Keyed by staff `id`. Each staff has optional `clef` (omit if unchanged) and `voices` |
|
||||||
|
|
||||||
|
**Beat fields** (inside `staves → voices → beats`):
|
||||||
|
|
||||||
|
| Field | Default | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `t` | required | Time in seconds |
|
||||||
|
| `dur` | required | Duration denominator: `1`=whole, `2`=half, `4`=quarter, `8`=eighth, `16`=sixteenth, `32`=thirty-second |
|
||||||
|
| `dot` | omit | Augmentation dots: `1`=dotted, `2`=double-dotted |
|
||||||
|
| `rest` | omit | `true` if this beat is a rest; `notes` is omitted |
|
||||||
|
| `tu` | omit | Tuplet: `[numerator, denominator]`, e.g. `[3, 2]` for triplet |
|
||||||
|
| `beat_pos` | omit | Exact position within the measure as a rational `[numerator, denominator]` pair, where the denominator is the time-signature denominator. E.g. beat 2 in 6/8 (the second dotted quarter) = `[3, 8]`. Avoids floating-point imprecision when deriving beat position from tempo and absolute time. Omit if not set by the importer. Renderers that do not recognise this field derive position from `t` and the tempo map as before. |
|
||||||
|
| `notes` | omit | List of note objects (omit for rests) |
|
||||||
|
| `dyn` | omit | Dynamic: `ppp`, `pp`, `p`, `mp`, `mf`, `f`, `ff`, `fff` |
|
||||||
|
| `slr` | omit | Slur start |
|
||||||
|
| `slre` | omit | Slur end |
|
||||||
|
| `grace` | omit | Grace-note beat, typed: `"a"` = acciaccatura (slashed, steals time from the previous note; MusicXML `<grace slash="yes">`), `"p"` = appoggiatura (unslashed, steals time from the following note; `<grace>`). The beat's `dur` is the grace note's written duration. Vocabulary in `lib/notation.py::GRACE_TYPES` |
|
||||||
|
| `arp` | omit | `true` when the beat's chord is arpeggiated (rolled; MusicXML `<arpeggiate>`) |
|
||||||
|
| `ferm` | omit | `true` when the beat carries a fermata (MusicXML `<fermata>`) |
|
||||||
|
| `spd` / `sph` / `spu` | omit | Sustain pedal: pedal **d**own / **h**old-through-this-beat / **u**p. This is the only pedal encoding — there is deliberately no separate `ped` field. MusicXML mapping: `<pedal type="start">` → `spd`, `<pedal type="change">` → `spu` + `spd` on the same beat (re-pedal), `<pedal type="stop">` → `spu`; beats inside an active pedal span carry `sph` |
|
||||||
|
| Additional beat effects | omit | `cre`, `dec`, `vib`, `vibw`, `fade`, `pm`, `lr`, `slap`, `pop`, `tap`, `su`, `sd`, `rasg`, `golpe`, `wah`, `txt`, `chrd` — all optional, omit when absent |
|
||||||
|
|
||||||
|
**Note fields** (inside `beats → notes`):
|
||||||
|
|
||||||
|
| Field | Default | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `midi` | required | MIDI pitch 0–127. Unambiguous — no string/fret/tuning indirection |
|
||||||
|
| `tied` | omit | Tied from the previous beat |
|
||||||
|
| `acc` | omit | Accidental override: `null`/omit = derive from key sig; `0` = force natural (♮); `−2`/`−1`/`1`/`2` = double-flat/flat/sharp/double-sharp |
|
||||||
|
| `stem` | omit | Force stem direction: `"up"` or `"down"` (MusicXML `<stem>`). Omit to let the renderer decide. Vocabulary in `lib/notation.py::STEM_DIRECTIONS` |
|
||||||
|
| Additional note effects | omit | `stc`, `ten`, `ac`, `hac`, `vib`, `vibw`, `dead`, `ghost`, `fng`, `rfng`, `str`, `harm`, `bend`, `slide`, `trill`, `ho`, `po`, `tp`, `barre` — all optional |
|
||||||
|
|
||||||
|
**Wire format.** `song_info` carries `has_notation: bool`. Notation data
|
||||||
|
is streamed as two highway-WS message types after `sections`, before `anchors`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"type": "notation_info", "version": 1, "instrument": "piano",
|
||||||
|
"staves": [...], "total": 64}
|
||||||
|
```
|
||||||
|
|
||||||
|
…followed by one or more chunks of 32 measures:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"type": "notation_measures", "data": [...], "total": 64}
|
||||||
|
```
|
||||||
|
|
||||||
|
`total` is the measure count across **all** chunks. Clients accumulate `data` arrays until the accumulated measure count reaches `total` (an individual chunk's `data.length` says nothing — every full chunk of a multi-chunk stream is shorter than `total`). The `anchors` frame that follows the notation block is a secondary end-of-block signal.
|
||||||
|
|
||||||
|
**`lib/notation.py`** is the vocabulary library: `SCHEMA_VERSION`, `CLEFS`, `DURATIONS`, `validate_notation()`, `measure_to_wire()`, `measures_to_wire()`.
|
||||||
|
|
||||||
|
**Legacy fallback.** Sloppaks that carry keys as guitar wire format (Clone Hero converted content) continue to work — the notation plugin checks for the `notation` key on the arrangement entry. When absent, it falls back to decoding guitar wire format notes via `midi = s * 24 + f`.
|
||||||
|
|
||||||
|
**v1 non-features (accepted limitations).** The following are deliberately
|
||||||
|
out of schema v1; they ship, if ever, as **additive v1.x patches** (new
|
||||||
|
optional fields old consumers ignore — the permissive validator passes
|
||||||
|
unknown fields through by design):
|
||||||
|
|
||||||
|
- Microtonal pitch (anything finer than the ±2 semitone `acc` vocabulary).
|
||||||
|
- Figured bass.
|
||||||
|
- Mid-measure key-signature, time-signature, or clef changes (all three are
|
||||||
|
measure-granular in v1).
|
||||||
|
- Ottava lines (`ott`), repeat/volta barline semantics (`barline`),
|
||||||
|
ornaments beyond trills (mordents, turns), tremolo (`trem`), and notated
|
||||||
|
glissando lines (`glis`).
|
||||||
|
|
||||||
|
Importers MUST drop these source features with a logged warning rather than
|
||||||
|
approximate them into wrong notation; renderers MUST NOT invent semantics
|
||||||
|
for field names from this list before a v1.x patch specifies them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Key / scale annotations (for theory-aware visualizations)
|
||||||
|
|
||||||
|
`keys.json` mirroring the `sections[]` shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"events": [
|
||||||
|
{"t": 0.0, "key": "Em", "scale": "natural_minor"},
|
||||||
|
{"t": 64.5, "key": "G", "scale": "major"},
|
||||||
|
{"t": 142.0, "key": "Em", "scale": "natural_minor"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Manifest:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
keys: keys.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Each entry implicitly applies until the next event. Same model as `sections[]`.
|
||||||
|
|
||||||
|
#### Vocal pitch contour (a different shape, a different key)
|
||||||
|
|
||||||
|
The canonical `vocal_pitch` key + file (defined in §2.4) is the
|
||||||
|
per-syllable note format consumed by the karaoke plugin —
|
||||||
|
`{version: 1, notes: [{t, d, midi}]}`. If you want to ship a finer-
|
||||||
|
grained pitch *contour* (one sample every 20 ms, Hz instead of MIDI),
|
||||||
|
that's a different shape and should ride on its own manifest key so
|
||||||
|
the two don't collide:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
vocal_pitch_contour: vocal_pitch_contour.json
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"samples": [
|
||||||
|
{"t": 0.000, "hz": 220.5},
|
||||||
|
{"t": 0.020, "hz": 222.1}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Per §5.1, manifest keys are cheap — reach for a new one when the
|
||||||
|
schema diverges, don't overload an existing key with a second shape.
|
||||||
|
|
||||||
|
### 5.4. `version` field — always include it
|
||||||
|
|
||||||
|
Every new file should have `"version": 1` at the top. It's free insurance: when you change the schema later, `version: 2` consumers can branch on it. Old consumers without that branch ignore the file (or fall back gracefully).
|
||||||
|
|
||||||
|
### 5.5. Stay backward-compatible
|
||||||
|
|
||||||
|
If you change a field that already shipped:
|
||||||
|
|
||||||
|
- **Adding fields** is always safe (older readers ignore them).
|
||||||
|
- **Removing fields** breaks older readers. Don't.
|
||||||
|
- **Repurposing fields** (changing meaning or units) is the worst — bump `version` and branch.
|
||||||
|
|
||||||
|
If you're tempted to remove or repurpose: leave the old field, add a new one, and sunset the old one over a release or two.
|
||||||
|
|
||||||
|
### 5.6. When to put data inside an arrangement vs. its own file
|
||||||
|
|
||||||
|
- **Inside arrangement JSON** (`arrangements/lead.json`):
|
||||||
|
- Data that is *per-arrangement* and *per-instrument* (notes, chords, anchors, hand-shapes — guitar specifics).
|
||||||
|
- Data that meaningfully differs between Lead and Rhythm versions of the same song.
|
||||||
|
- **Its own file** (and pointed-at via manifest key):
|
||||||
|
- Data that is *song-wide* (lyrics, beats, sections, tempo map, drum tab, lighting, key/scale changes).
|
||||||
|
- Data that may be authored or generated independently of the playable arrangement (a stem split, an AI-generated drum tab).
|
||||||
|
|
||||||
|
Beats and sections historically lived inside the first arrangement JSON (early arrangement XML put them there). The `song_timeline.json` file (see §5.3) is the correct home for new sloppaks — the loader reads it first and it takes priority. New song-wide data should always be its own file.
|
||||||
|
|
||||||
|
### 5.7. Don't break the manifest contract
|
||||||
|
|
||||||
|
A few things that should *not* end up in `manifest.yaml`:
|
||||||
|
|
||||||
|
- **Per-machine settings** (DMX universes, IPs, output device picks) — those go in `${CONFIG_DIR}/...json`, not the sloppak.
|
||||||
|
- **UI state** (last zoom level, panel sizes) — `localStorage` only.
|
||||||
|
- **User progress / play counts** — Slopsmith stores these in its metadata DB, not in the sloppak.
|
||||||
|
|
||||||
|
The sloppak holds **the song's authored data**. Anything that varies by user or by machine is out.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Quick reference — file types you'll touch
|
||||||
|
|
||||||
|
| File | Format | Schema lives in | Authority |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `manifest.yaml` | YAML | `lib/sloppak.py` (`load_manifest`, `extract_meta`) | This doc + the loader |
|
||||||
|
| `arrangements/*.json` | JSON | `lib/song.py` (`arrangement_to_wire`, `arrangement_from_wire`) | The wire-format functions |
|
||||||
|
| `lyrics.json` | JSON (flat list) | `lib/sloppak.py` (passed through to `Song.lyrics`) | This doc §2.3 |
|
||||||
|
| `song_timeline.json` | JSON | `lib/sloppak.py` (loader) | This doc §5.3 |
|
||||||
|
| `notation_<id>.json` | JSON | `lib/notation.py` (`validate_notation`, `measures_to_wire`) | This doc §5.3 |
|
||||||
|
| `stems/*.ogg` | OGG Vorbis | — | Convention: `q:a 5` for size/quality balance |
|
||||||
|
| `cover.jpg` | JPEG | — | Convention: square, 500–1500 px on a side |
|
||||||
|
| Your new file | JSON (preferred) | Your plugin's spec doc | You |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Testing your extension
|
||||||
|
|
||||||
|
If you add a new file type or manifest key:
|
||||||
|
|
||||||
|
1. **Round-trip test**: write a sample, load it, write it back, compare. Add to `tests/test_sloppak.py`.
|
||||||
|
2. **Backward-compat test**: load a sloppak that *doesn't* have your new key — your code must not crash, and the song must still play.
|
||||||
|
3. **Hand-edit test**: open the directory form in a text editor, change a field by hand, reload Slopsmith. The format is meant to be hand-editable; your additions should preserve that.
|
||||||
|
4. **Both forms**: test with both the directory form and the zipped form. The unpack cache is invalidated based on mtime and size, so you can repackage and reload without restarting the server.
|
||||||
|
|
||||||
|
The full pytest suite (`pytest`) must stay green before any PR.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Where to look in the code
|
||||||
|
|
||||||
|
| For… | Read |
|
||||||
|
|---|---|
|
||||||
|
| Format detection, source resolution, zip unpacking | [lib/sloppak.py](../lib/sloppak.py) |
|
||||||
|
| Data classes (`Note`, `Chord`, `Arrangement`, `Song`, `Phrase`) | [lib/song.py](../lib/song.py) |
|
||||||
|
| Wire-format helpers (`*_to_wire` / `*_from_wire`) | [lib/song.py](../lib/song.py) |
|
||||||
|
| The reference sloppak writer | [lib/sloppak_convert.py](../lib/sloppak_convert.py) |
|
||||||
|
| Drum tab vocabulary and wire helpers | [lib/drums.py](../lib/drums.py) |
|
||||||
|
| Notation vocabulary and wire helpers | [lib/notation.py](../lib/notation.py) |
|
||||||
|
| Live streaming over WebSocket (consumes the same shapes) | `server.py` (`/ws/highway/{filename}`) |
|
||||||
|
| The plugin system (where new viz consumers go) | [CLAUDE.md](../CLAUDE.md) — Plugin System section |
|
||||||
|
| Tests | [tests/test_sloppak.py](../tests/test_sloppak.py), [tests/test_sloppak_convert.py](../tests/test_sloppak_convert.py) |
|
||||||
+523
@@ -0,0 +1,523 @@
|
|||||||
|
"""Audio extraction and conversion for Rocksmith CDLC."""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
log = logging.getLogger("slopsmith.lib.audio")
|
||||||
|
|
||||||
|
# Maximum length of any single decoder-error fragment that we surface to
|
||||||
|
# the client. ffmpeg can emit multi-kB build-configuration / version
|
||||||
|
# banners on failure, which would make the WebSocket `audio_error`
|
||||||
|
# payload huge and bury the actionable bit.
|
||||||
|
_MAX_DECODER_DETAIL_CHARS = 500
|
||||||
|
|
||||||
|
|
||||||
|
def _basename_any_path(raw: str) -> str:
|
||||||
|
"""Cross-platform basename: recognises both `/` and `\\` as
|
||||||
|
separators regardless of host, and strips trailing separators before
|
||||||
|
splitting so a directory match collapses to its final segment
|
||||||
|
instead of the empty string.
|
||||||
|
|
||||||
|
`os.path.basename` is platform-specific — on POSIX it only treats
|
||||||
|
`/` as a separator, so a Windows path emitted by a decoder running
|
||||||
|
inside a cross-platform error log would leak through verbatim."""
|
||||||
|
candidate = raw.rstrip("/\\")
|
||||||
|
if not candidate:
|
||||||
|
return raw
|
||||||
|
last = max(candidate.rfind("/"), candidate.rfind("\\"))
|
||||||
|
base = candidate[last + 1:] if last >= 0 else candidate
|
||||||
|
return base or raw
|
||||||
|
|
||||||
|
|
||||||
|
# Lines that match this regex are version / build banner output that
|
||||||
|
# ffmpeg (and friends) emit before the actual error message. They're
|
||||||
|
# never actionable on their own — the useful error is somewhere after.
|
||||||
|
_BANNER_LINE_RE = re.compile(
|
||||||
|
r"""^\s*(
|
||||||
|
ffmpeg\sversion # e.g. "ffmpeg version 4.4.2-..."
|
||||||
|
| built\swith # " built with gcc ..."
|
||||||
|
| configuration: # " configuration: ..."
|
||||||
|
| lib(av\w+|sw\w+|postproc) # " libavutil 56.70.100"
|
||||||
|
| Stream\smapping: # ffmpeg's "Stream mapping:" header
|
||||||
|
| Input\s\# # "Input #0, wav, from ..."
|
||||||
|
| Output\s\# # "Output #0, mp3, ..."
|
||||||
|
| Duration: # " Duration: ..."
|
||||||
|
| Press\s\[q\] # interactive prompts
|
||||||
|
)""",
|
||||||
|
re.VERBOSE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _truncate_detail(text: str, limit: int = _MAX_DECODER_DETAIL_CHARS) -> str:
|
||||||
|
"""Shrink a multi-line decoder stderr blob to one actionable line
|
||||||
|
under `limit` characters.
|
||||||
|
|
||||||
|
ffmpeg-style failures start with a multi-line version / build /
|
||||||
|
config banner and put the actual error after it, so naive
|
||||||
|
"first non-empty line" picks the banner. Skip lines matching
|
||||||
|
`_BANNER_LINE_RE` and prefer the first remaining non-empty line.
|
||||||
|
If every line matched the banner pattern (shouldn't happen in
|
||||||
|
practice but cover the case), fall back to the first non-empty
|
||||||
|
line so we don't end up emitting an empty string."""
|
||||||
|
lines = [ln for ln in text.splitlines() if ln.strip()]
|
||||||
|
actionable = next(
|
||||||
|
(ln for ln in lines if not _BANNER_LINE_RE.match(ln)),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if actionable is None:
|
||||||
|
actionable = lines[0] if lines else text.strip()
|
||||||
|
actionable = actionable.strip()
|
||||||
|
if len(actionable) <= limit:
|
||||||
|
return actionable
|
||||||
|
return actionable[:limit - 1].rstrip() + "…"
|
||||||
|
|
||||||
|
|
||||||
|
# Unquoted absolute paths: stop at whitespace or common delimiters. The
|
||||||
|
# quote characters are excluded so the quoted-path pass (below) can
|
||||||
|
# claim those matches instead.
|
||||||
|
#
|
||||||
|
# Drive-letter Windows paths support both separator conventions —
|
||||||
|
# `C:\Users\…` and `C:/Users/…`. Native Windows APIs and many tools
|
||||||
|
# (PowerShell, .NET, ffmpeg with -i C:/...) emit the forward-slash form,
|
||||||
|
# and the unquoted-path branch alone can't handle that case without it
|
||||||
|
# because the body `[^\s"'`<>|]+` would never have matched the
|
||||||
|
# colon-then-slash prefix.
|
||||||
|
_UNQUOTED_ABS_PATH_RE = re.compile(
|
||||||
|
r"""(?:
|
||||||
|
(?:[A-Za-z]:[\\/] | \\\\) # Windows: C:\… / C:/… or UNC \\host\…
|
||||||
|
| / # POSIX: leading /
|
||||||
|
)
|
||||||
|
[^\s"'`<>|]+
|
||||||
|
""",
|
||||||
|
re.VERBOSE,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Quoted absolute paths (single, double, or backtick quotes). Captures
|
||||||
|
# the opening quote so the replacement can keep the quoting wrapper
|
||||||
|
# intact while collapsing the inner path to its basename. Allows spaces
|
||||||
|
# in the path — that's the whole reason quotes get used in stderr
|
||||||
|
# output (`C:\Program Files\…`, `/Users/Alice/My Secrets/…`).
|
||||||
|
_QUOTED_ABS_PATH_RE = re.compile(
|
||||||
|
r"""(['"`])
|
||||||
|
((?:[A-Za-z]:[\\/] | \\\\ | /)
|
||||||
|
[^'"`\n]+)
|
||||||
|
\1
|
||||||
|
""",
|
||||||
|
re.VERBOSE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_unquoted_match(match: re.Match) -> str:
|
||||||
|
return _basename_any_path(match.group(0))
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_quoted_match(match: re.Match) -> str:
|
||||||
|
quote = match.group(1)
|
||||||
|
inner = match.group(2)
|
||||||
|
return f"{quote}{_basename_any_path(inner)}{quote}"
|
||||||
|
|
||||||
|
|
||||||
|
def _bundled_bin_dir() -> Path | None:
|
||||||
|
"""Resolve the desktop bundle's resources/bin/ directory if we're
|
||||||
|
running inside one. Layout: resources/slopsmith/lib/audio.py →
|
||||||
|
resources/bin/. Gate on vgmstream-cli's presence so we don't
|
||||||
|
misidentify random parent dirs (e.g. Docker's `/bin`, dev
|
||||||
|
layouts where parents[2] resolves to the repo root) — vgmstream-cli
|
||||||
|
is bundled on every desktop platform and isn't a typical system
|
||||||
|
binary, so it's a precise signature for the desktop layout."""
|
||||||
|
bundled = Path(__file__).resolve().parents[2] / "bin"
|
||||||
|
if any((bundled / n).is_file() for n in ("vgmstream-cli", "vgmstream-cli.exe")):
|
||||||
|
return bundled
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _bundled_or_path(name: str) -> str | None:
|
||||||
|
"""Prefer the bundled binary on desktop, fall back to PATH lookup.
|
||||||
|
|
||||||
|
Necessary because Electron's child PATH on macOS / Linux puts
|
||||||
|
user-installed binaries (Homebrew `/opt/homebrew/bin`, /usr/local)
|
||||||
|
before our `resources/bin`, so `shutil.which` alone picks up the
|
||||||
|
user's binary — which may have been built without the features
|
||||||
|
we rely on (e.g. Homebrew ffmpeg formulas that omit libvorbis)."""
|
||||||
|
bundled = _bundled_bin_dir()
|
||||||
|
if bundled is not None:
|
||||||
|
for fname in (name, f"{name}.exe"):
|
||||||
|
cand = bundled / fname
|
||||||
|
if cand.is_file():
|
||||||
|
return str(cand)
|
||||||
|
return shutil.which(name)
|
||||||
|
|
||||||
|
|
||||||
|
def _repo_root() -> Path:
|
||||||
|
"""Return the repository root for local binary fallbacks."""
|
||||||
|
return Path(__file__).resolve().parent.parent
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_executable(candidate: str | None) -> str | None:
|
||||||
|
"""Resolve either a command name on PATH or an explicit executable path.
|
||||||
|
|
||||||
|
Explicit paths must refer to a regular file (directories often satisfy
|
||||||
|
os.access(..., X_OK) on POSIX but cannot be exec'd by subprocess)."""
|
||||||
|
if not candidate:
|
||||||
|
return None
|
||||||
|
if os.path.sep in candidate or (os.path.altsep and os.path.altsep in candidate):
|
||||||
|
path = Path(candidate).expanduser()
|
||||||
|
if path.is_file() and os.access(path, os.X_OK):
|
||||||
|
return str(path.resolve())
|
||||||
|
return None
|
||||||
|
return shutil.which(candidate)
|
||||||
|
|
||||||
|
|
||||||
|
def _vgmstream_cmd(resolution_notes: list[str] | None = None) -> str | None:
|
||||||
|
"""Return the best available vgmstream-cli executable.
|
||||||
|
|
||||||
|
Resolution order:
|
||||||
|
1. `VGMSTREAM_CLI` env var (explicit override — must beat everything
|
||||||
|
else so a user can force a known-good binary when the bundled or
|
||||||
|
system one is broken)
|
||||||
|
2. Bundled `resources/bin/vgmstream-cli` (desktop)
|
||||||
|
3. `vgmstream-cli` on PATH
|
||||||
|
4. Repo-local build outputs (autotools `.libs/`, CMake `build/cli/`,
|
||||||
|
and the in-tree `vgmstream/cli/` location), checked for both Unix
|
||||||
|
and Windows (`.exe`) names so a local `cmake --build` discovered
|
||||||
|
off-PATH still works.
|
||||||
|
|
||||||
|
`vgmstream123` is intentionally excluded: it is a player-style frontend
|
||||||
|
with a different argument schema and cannot be invoked with the
|
||||||
|
`-o <wav> <wem>` interface the rest of this module assumes.
|
||||||
|
|
||||||
|
`resolution_notes` (optional): when provided, the resolver appends
|
||||||
|
human-readable warnings about resolution-time problems (e.g. a
|
||||||
|
`VGMSTREAM_CLI` value that didn't resolve). Callers that surface
|
||||||
|
decode failures to the user can fold these into the final error
|
||||||
|
message so the user understands why their override was ignored
|
||||||
|
instead of seeing only the generic "no decoder found" guidance."""
|
||||||
|
env_value = os.environ.get("VGMSTREAM_CLI")
|
||||||
|
explicit = _resolve_executable(env_value)
|
||||||
|
if explicit:
|
||||||
|
return explicit
|
||||||
|
if env_value:
|
||||||
|
# The env var is documented as an explicit override, so silently
|
||||||
|
# falling through when it's set to a stale or non-executable path
|
||||||
|
# is misleading. Log a warning so the user sees why their override
|
||||||
|
# didn't take, but don't raise — we still want the next fallback
|
||||||
|
# to succeed if e.g. PATH has a working binary.
|
||||||
|
log.warning(
|
||||||
|
"VGMSTREAM_CLI=%r is set but does not resolve to an "
|
||||||
|
"executable file; falling through to other candidates",
|
||||||
|
env_value,
|
||||||
|
)
|
||||||
|
if resolution_notes is not None:
|
||||||
|
# Don't echo the env's full value back to the user — it's an
|
||||||
|
# absolute path; the basename + "ignored" is enough to point
|
||||||
|
# them at their misconfiguration without leaking layout.
|
||||||
|
# `_basename_any_path` (not `os.path.basename`) so a Windows
|
||||||
|
# value on a POSIX host still collapses correctly.
|
||||||
|
resolution_notes.append(
|
||||||
|
f"VGMSTREAM_CLI={_basename_any_path(env_value) or '<set>'!r}"
|
||||||
|
" is not an executable file and was ignored"
|
||||||
|
)
|
||||||
|
|
||||||
|
bundled_dir = _bundled_bin_dir()
|
||||||
|
if bundled_dir is not None:
|
||||||
|
for fname in ("vgmstream-cli", "vgmstream-cli.exe"):
|
||||||
|
cand = bundled_dir / fname
|
||||||
|
# Same exec check we apply to env/repo-local candidates —
|
||||||
|
# a present-but-not-executable file (lost +x after a tar
|
||||||
|
# extract, marked unreadable, etc.) would otherwise be
|
||||||
|
# returned here and block the perfectly fine PATH binary
|
||||||
|
# below from getting a chance.
|
||||||
|
if cand.is_file() and os.access(cand, os.X_OK):
|
||||||
|
return str(cand)
|
||||||
|
|
||||||
|
on_path = shutil.which("vgmstream-cli")
|
||||||
|
if on_path:
|
||||||
|
return on_path
|
||||||
|
|
||||||
|
root = _repo_root()
|
||||||
|
for rel in (
|
||||||
|
"vgmstream/build/cli/vgmstream-cli",
|
||||||
|
"vgmstream/build/cli/vgmstream-cli.exe",
|
||||||
|
"vgmstream/cli/vgmstream-cli",
|
||||||
|
"vgmstream/cli/vgmstream-cli.exe",
|
||||||
|
"vgmstream/cli/.libs/vgmstream-cli",
|
||||||
|
"vgmstream/cli/.libs/vgmstream-cli.exe",
|
||||||
|
):
|
||||||
|
resolved = _resolve_executable(str(root / rel))
|
||||||
|
if resolved:
|
||||||
|
return resolved
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _ffmpeg_cmd() -> str | None:
|
||||||
|
"""Return the path to ffmpeg, preferring the bundled binary."""
|
||||||
|
return _bundled_or_path("ffmpeg")
|
||||||
|
|
||||||
|
|
||||||
|
def _ffmpeg_wav_to_ogg(ffmpeg: str, wav: Path, out_ogg: Path) -> subprocess.CompletedProcess:
|
||||||
|
"""Encode WAV → Ogg Vorbis. Prefers libvorbis (external, full quality);
|
||||||
|
if the ffmpeg build lacks it (some Homebrew formulas no longer set
|
||||||
|
--enable-libvorbis), retries with ffmpeg's built-in `vorbis` encoder
|
||||||
|
under `-strict experimental`. Same .ogg container either way; the
|
||||||
|
built-in path produces a lower-quality file but always works."""
|
||||||
|
r = subprocess.run(
|
||||||
|
[ffmpeg, "-y", "-i", str(wav), "-c:a", "libvorbis", "-q:a", "5", str(out_ogg)],
|
||||||
|
capture_output=True,
|
||||||
|
)
|
||||||
|
if r.returncode == 0 and out_ogg.exists() and out_ogg.stat().st_size >= 100:
|
||||||
|
return r
|
||||||
|
if b"Unknown encoder 'libvorbis'" not in (r.stderr or b""):
|
||||||
|
return r
|
||||||
|
return subprocess.run(
|
||||||
|
[ffmpeg, "-y", "-i", str(wav),
|
||||||
|
"-c:a", "vorbis", "-strict", "experimental", "-q:a", "5", str(out_ogg)],
|
||||||
|
capture_output=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_paths(text: str, *paths: str) -> str:
|
||||||
|
"""Replace absolute filesystem paths in `text` with their basenames.
|
||||||
|
|
||||||
|
Decoder error strings get joined into the RuntimeError that
|
||||||
|
`convert_wem` raises, and slopsmith surfaces that text in the
|
||||||
|
browser as `audio_error`. Leaking install / user / DLC paths to the
|
||||||
|
client is a needless info disclosure, so before any decoder error
|
||||||
|
leaves this module we strip absolute paths down to their final
|
||||||
|
segment.
|
||||||
|
|
||||||
|
Two-pass approach:
|
||||||
|
1. Replace each *known* path (decoder binary, input WEM, intended
|
||||||
|
output) verbatim so its basename survives even when the path
|
||||||
|
contains characters the generic regex's character class
|
||||||
|
excludes (e.g. quoted arguments).
|
||||||
|
2. Run the generic absolute-path regex over the remainder so
|
||||||
|
paths the decoder emitted itself ("could not open
|
||||||
|
/unrelated/private/file") also get redacted to their
|
||||||
|
basename. Decoders sometimes log paths the caller never
|
||||||
|
passed in (e.g. plugin search paths, dynamic loader paths),
|
||||||
|
and those are exactly the ones the caller can't enumerate."""
|
||||||
|
out = text
|
||||||
|
for p in paths:
|
||||||
|
if not p:
|
||||||
|
continue
|
||||||
|
out = out.replace(p, _basename_any_path(p))
|
||||||
|
# Quoted paths first so the unquoted pass doesn't claim part of a
|
||||||
|
# quoted match — paths with spaces only survive when quoted, so we
|
||||||
|
# need that branch to win there.
|
||||||
|
out = _QUOTED_ABS_PATH_RE.sub(_scrub_quoted_match, out)
|
||||||
|
out = _UNQUOTED_ABS_PATH_RE.sub(_scrub_unquoted_match, out)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_wem_to_wav(vgmstream: str, wem_path: str, wav_path: str) -> tuple[bool, str]:
|
||||||
|
"""Decode a WEM file to WAV using vgmstream and return status + detail.
|
||||||
|
|
||||||
|
Catches launch-time OSError (wrong architecture, missing dynamic loader,
|
||||||
|
permission errors a stat-check can't predict) so callers can record the
|
||||||
|
failure and fall through to ffmpeg / ww2ogg instead of crashing. The
|
||||||
|
returned detail is scrubbed of absolute paths because callers fold it
|
||||||
|
into the user-facing decode error."""
|
||||||
|
try:
|
||||||
|
r = subprocess.run(
|
||||||
|
[vgmstream, "-o", wav_path, wem_path],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
# `errors='replace'` — without this, a vgmstream build that
|
||||||
|
# emits non-UTF-8 bytes (corrupt input, locale mismatch) makes
|
||||||
|
# subprocess.run raise UnicodeDecodeError and bypass the
|
||||||
|
# failure-aggregation path the caller relies on.
|
||||||
|
errors="replace",
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
# Log the full path server-side for ops, but keep the client-facing
|
||||||
|
# detail path-neutral. OSError/TimeoutExpired stringify with the
|
||||||
|
# command path or filename in many cases (e.g. exec-format errors
|
||||||
|
# quote the filename, TimeoutExpired stringifies the cmd list), so
|
||||||
|
# the exc text itself also needs scrubbing.
|
||||||
|
log.warning("vgmstream launch failed (%s): %s", vgmstream, exc)
|
||||||
|
scrubbed = _scrub_paths(str(exc), vgmstream, wem_path, wav_path)
|
||||||
|
return False, f"failed to invoke {_basename_any_path(vgmstream)}: {_truncate_detail(scrubbed)}"
|
||||||
|
|
||||||
|
if r.returncode == 0 and os.path.exists(wav_path) and os.path.getsize(wav_path) > 0:
|
||||||
|
return True, ""
|
||||||
|
|
||||||
|
# Strip both streams before choosing which to report — a whitespace-only
|
||||||
|
# stderr would otherwise suppress a useful stdout message and the caller
|
||||||
|
# would see only "exit code N".
|
||||||
|
err = (r.stderr or "").strip()
|
||||||
|
out = (r.stdout or "").strip()
|
||||||
|
detail = err or out or f"exit code {r.returncode}"
|
||||||
|
return False, _truncate_detail(_scrub_paths(detail, vgmstream, wem_path, wav_path))
|
||||||
|
|
||||||
|
|
||||||
|
def find_wem_files(extracted_dir: str) -> list[str]:
|
||||||
|
"""Find WEM audio files, sorted largest first (full song before preview)."""
|
||||||
|
wem_files = list(Path(extracted_dir).rglob("*.wem"))
|
||||||
|
wem_files.sort(key=lambda p: p.stat().st_size, reverse=True)
|
||||||
|
return [str(f) for f in wem_files]
|
||||||
|
|
||||||
|
|
||||||
|
def convert_wem(wem_path: str, output_base: str) -> str:
|
||||||
|
"""
|
||||||
|
Convert a WEM file to a playable format.
|
||||||
|
Returns path to the converted audio file.
|
||||||
|
"""
|
||||||
|
# `errors` holds *attempted-decoder* failures (something ran, didn't
|
||||||
|
# work); `resolution_notes` holds *configuration* warnings (e.g.
|
||||||
|
# a stale VGMSTREAM_CLI). Keeping them separate matters for the
|
||||||
|
# final branch: if every decoder is missing entirely we want the
|
||||||
|
# actionable "install vgmstream-cli" guidance, not "Failed to
|
||||||
|
# decode" — but the resolution note should still ride along on
|
||||||
|
# either path so a misconfigured user understands why their
|
||||||
|
# override was ignored.
|
||||||
|
errors: list[str] = []
|
||||||
|
resolution_notes: list[str] = []
|
||||||
|
|
||||||
|
# Try vgmstream-cli → WAV → MP3 (best browser compatibility).
|
||||||
|
vgmstream = _vgmstream_cmd(resolution_notes=resolution_notes)
|
||||||
|
if vgmstream:
|
||||||
|
wav = output_base + ".wav"
|
||||||
|
ok, detail = _decode_wem_to_wav(vgmstream, wem_path, wav)
|
||||||
|
if ok:
|
||||||
|
ffmpeg = _ffmpeg_cmd()
|
||||||
|
if ffmpeg:
|
||||||
|
mp3 = output_base + ".mp3"
|
||||||
|
# Same OSError/timeout protection as the direct-fallback
|
||||||
|
# ffmpeg calls below — vgmstream decoded fine, but a
|
||||||
|
# wrong-arch / missing-loader ffmpeg would otherwise
|
||||||
|
# raise raw out of convert_wem instead of letting us
|
||||||
|
# fall back to returning the decoded WAV.
|
||||||
|
try:
|
||||||
|
r2 = subprocess.run(
|
||||||
|
[ffmpeg, "-y", "-i", wav, "-b:a", "192k", mp3],
|
||||||
|
capture_output=True,
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
log.warning("ffmpeg MP3-transcode launch failed (%s): %s", ffmpeg, exc)
|
||||||
|
r2 = None
|
||||||
|
if r2 is not None and r2.returncode == 0 and os.path.exists(mp3):
|
||||||
|
os.remove(wav)
|
||||||
|
return mp3
|
||||||
|
return wav
|
||||||
|
errors.append(f"vgmstream: {detail}")
|
||||||
|
|
||||||
|
# Try ffmpeg directly (some builds handle Wwise). Wrap subprocess.run
|
||||||
|
# in try/except like _decode_wem_to_wav does — a wrong-architecture
|
||||||
|
# or broken-loader ffmpeg binary would otherwise raise OSError out of
|
||||||
|
# convert_wem and the browser would receive the raw exception text
|
||||||
|
# (including absolute paths) instead of the scrubbed aggregated
|
||||||
|
# decoder error, while also skipping the ww2ogg fallback below.
|
||||||
|
ffmpeg = _ffmpeg_cmd()
|
||||||
|
if ffmpeg:
|
||||||
|
mp3 = output_base + ".mp3"
|
||||||
|
try:
|
||||||
|
r = subprocess.run(
|
||||||
|
[ffmpeg, "-y", "-i", wem_path, "-b:a", "192k", mp3],
|
||||||
|
capture_output=True,
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
log.warning("ffmpeg launch failed (%s): %s", ffmpeg, exc)
|
||||||
|
errors.append(
|
||||||
|
f"ffmpeg mp3: failed to invoke {_basename_any_path(ffmpeg)}: "
|
||||||
|
+ _truncate_detail(_scrub_paths(str(exc), ffmpeg, wem_path, mp3))
|
||||||
|
)
|
||||||
|
r = None
|
||||||
|
if r is not None:
|
||||||
|
if r.returncode == 0 and os.path.exists(mp3) and os.path.getsize(mp3) > 0:
|
||||||
|
return mp3
|
||||||
|
stderr = (r.stderr or b'').decode(errors='replace').strip() or f"exit code {r.returncode}"
|
||||||
|
errors.append(
|
||||||
|
f"ffmpeg mp3: {_truncate_detail(_scrub_paths(stderr, ffmpeg, wem_path, mp3))}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Try WAV output as fallback
|
||||||
|
wav = output_base + ".wav"
|
||||||
|
try:
|
||||||
|
r = subprocess.run(
|
||||||
|
[ffmpeg, "-y", "-i", wem_path, wav],
|
||||||
|
capture_output=True,
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
log.warning("ffmpeg launch failed (%s): %s", ffmpeg, exc)
|
||||||
|
errors.append(
|
||||||
|
f"ffmpeg wav: failed to invoke {_basename_any_path(ffmpeg)}: "
|
||||||
|
+ _truncate_detail(_scrub_paths(str(exc), ffmpeg, wem_path, wav))
|
||||||
|
)
|
||||||
|
r = None
|
||||||
|
if r is not None:
|
||||||
|
if r.returncode == 0 and os.path.exists(wav) and os.path.getsize(wav) > 0:
|
||||||
|
return wav
|
||||||
|
stderr = (r.stderr or b'').decode(errors='replace').strip() or f"exit code {r.returncode}"
|
||||||
|
errors.append(
|
||||||
|
f"ffmpeg wav: {_truncate_detail(_scrub_paths(stderr, ffmpeg, wem_path, wav))}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Try ww2ogg — same launch-failure protection as ffmpeg above.
|
||||||
|
# `shutil.which` confirms the file is executable, not that the kernel
|
||||||
|
# can actually exec it (wrong arch / missing loader still raise here).
|
||||||
|
ww2ogg = shutil.which("ww2ogg")
|
||||||
|
if ww2ogg:
|
||||||
|
ogg = output_base + ".ogg"
|
||||||
|
try:
|
||||||
|
r = subprocess.run(
|
||||||
|
[ww2ogg, wem_path, "-o", ogg],
|
||||||
|
capture_output=True,
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
log.warning("ww2ogg launch failed (%s): %s", ww2ogg, exc)
|
||||||
|
errors.append(
|
||||||
|
f"ww2ogg: failed to invoke {_basename_any_path(ww2ogg)}: "
|
||||||
|
+ _truncate_detail(_scrub_paths(str(exc), ww2ogg, wem_path, ogg))
|
||||||
|
)
|
||||||
|
r = None
|
||||||
|
if r is not None:
|
||||||
|
if r.returncode == 0 and os.path.exists(ogg) and os.path.getsize(ogg) > 0:
|
||||||
|
return ogg
|
||||||
|
stderr = (r.stderr or b'').decode(errors='replace').strip() or f"exit code {r.returncode}"
|
||||||
|
errors.append(
|
||||||
|
f"ww2ogg: {_truncate_detail(_scrub_paths(stderr, ww2ogg, wem_path, ogg))}"
|
||||||
|
)
|
||||||
|
|
||||||
|
_INSTALL_GUIDANCE = (
|
||||||
|
"Install vgmstream-cli:\n"
|
||||||
|
" Manjaro/Arch: yay -S vgmstream-cli-bin\n"
|
||||||
|
" Or set VGMSTREAM_CLI to a built binary, e.g. vgmstream/cli/vgmstream-cli"
|
||||||
|
)
|
||||||
|
|
||||||
|
user_msg_prefix = " | ".join(resolution_notes) + (" | " if resolution_notes else "")
|
||||||
|
|
||||||
|
if errors:
|
||||||
|
# Something ran and failed. `wem_path` is the on-disk input
|
||||||
|
# path, often deep inside the user's DLC dir — log the full
|
||||||
|
# path for ops, but keep the client-facing error to just the
|
||||||
|
# filename. If vgmstream itself was never resolved (only ffmpeg
|
||||||
|
# or ww2ogg tried-and-failed), append the install guidance —
|
||||||
|
# ffmpeg is commonly present and often can't decode Wwise WEMs,
|
||||||
|
# so without this hint a user missing the primary decoder
|
||||||
|
# never sees "install vgmstream-cli" guidance.
|
||||||
|
suffix = ""
|
||||||
|
if not vgmstream:
|
||||||
|
suffix = f" | (Hint: {_INSTALL_GUIDANCE})"
|
||||||
|
log.warning("Decode failed for %s: %s",
|
||||||
|
wem_path, " | ".join([*resolution_notes, *errors]))
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Failed to decode WEM {_basename_any_path(wem_path)}: "
|
||||||
|
+ user_msg_prefix + " | ".join(errors) + suffix
|
||||||
|
)
|
||||||
|
|
||||||
|
# No decoder ran at all — give the actionable guidance, with the
|
||||||
|
# resolution note prefixed so a user who *did* set VGMSTREAM_CLI
|
||||||
|
# (incorrectly) understands why their override didn't help.
|
||||||
|
raise RuntimeError(
|
||||||
|
user_msg_prefix + "No WEM audio decoder found. " + _INSTALL_GUIDANCE
|
||||||
|
)
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,239 @@
|
|||||||
|
"""Backend hardware probe for diagnostic bundles.
|
||||||
|
|
||||||
|
Produces a `system.hardware.v1`-shaped dict — see
|
||||||
|
docs/diagnostics-bundle-spec.md.
|
||||||
|
|
||||||
|
All probes are best-effort and never raise. Missing tools, missing
|
||||||
|
permissions, container masking — every case yields a structured note in
|
||||||
|
the output rather than a 500 on the export endpoint.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import platform
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SCHEMA = "system.hardware.v1"
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_run(cmd: list[str], timeout: float = 2.0) -> tuple[int, str, str]:
|
||||||
|
try:
|
||||||
|
proc = subprocess.run(
|
||||||
|
cmd,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=timeout,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
return proc.returncode, proc.stdout or "", proc.stderr or ""
|
||||||
|
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
|
||||||
|
return 127, "", ""
|
||||||
|
|
||||||
|
|
||||||
|
def detect_runtime() -> dict:
|
||||||
|
"""Cheap runtime-kind detection (env var + cgroup checks, no subprocess).
|
||||||
|
|
||||||
|
Exported as a public function so callers that don't need the full
|
||||||
|
hardware probe can still obtain the runtime kind without paying for
|
||||||
|
nvidia-smi / psutil CPU probes.
|
||||||
|
"""
|
||||||
|
out: dict = {"kind": "bare", "in_docker": False, "in_kubernetes": False}
|
||||||
|
env_runtime = os.environ.get("SLOPSMITH_RUNTIME", "").strip().lower()
|
||||||
|
if env_runtime in ("electron", "docker", "bare"):
|
||||||
|
out["kind"] = env_runtime
|
||||||
|
if Path("/.dockerenv").exists():
|
||||||
|
out["in_docker"] = True
|
||||||
|
if out["kind"] == "bare":
|
||||||
|
out["kind"] = "docker"
|
||||||
|
cgroup = Path("/proc/1/cgroup")
|
||||||
|
if cgroup.exists():
|
||||||
|
try:
|
||||||
|
txt = cgroup.read_text(errors="ignore")
|
||||||
|
if "docker" in txt or "containerd" in txt or "kubepods" in txt:
|
||||||
|
out["in_docker"] = True
|
||||||
|
if out["kind"] == "bare":
|
||||||
|
out["kind"] = "docker"
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
if os.environ.get("KUBERNETES_SERVICE_HOST"):
|
||||||
|
out["in_kubernetes"] = True
|
||||||
|
if out["kind"] == "bare":
|
||||||
|
try:
|
||||||
|
import psutil # type: ignore
|
||||||
|
|
||||||
|
parent = psutil.Process(os.getppid()).name().lower()
|
||||||
|
if "electron" in parent or "slopsmith" in parent:
|
||||||
|
out["kind"] = "electron"
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_os() -> dict:
|
||||||
|
return {
|
||||||
|
"system": platform.system(),
|
||||||
|
"release": platform.release(),
|
||||||
|
"version": platform.version(),
|
||||||
|
"machine": platform.machine(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_cpu(notes: list[str]) -> dict:
|
||||||
|
out: dict = {
|
||||||
|
"brand": None,
|
||||||
|
"arch": platform.machine(),
|
||||||
|
"cores_logical": os.cpu_count(),
|
||||||
|
"cores_physical": None,
|
||||||
|
"freq_mhz_current": None,
|
||||||
|
"freq_mhz_max": None,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
import psutil # type: ignore
|
||||||
|
|
||||||
|
out["cores_physical"] = psutil.cpu_count(logical=False)
|
||||||
|
freq = psutil.cpu_freq()
|
||||||
|
if freq:
|
||||||
|
out["freq_mhz_current"] = round(freq.current) if freq.current else None
|
||||||
|
out["freq_mhz_max"] = round(freq.max) if freq.max else None
|
||||||
|
except Exception as e:
|
||||||
|
notes.append(f"psutil cpu probe failed: {e}")
|
||||||
|
try:
|
||||||
|
import cpuinfo # type: ignore
|
||||||
|
|
||||||
|
info = cpuinfo.get_cpu_info() or {}
|
||||||
|
out["brand"] = info.get("brand_raw") or info.get("brand") or None
|
||||||
|
except Exception as e:
|
||||||
|
notes.append(f"py-cpuinfo probe failed: {e}")
|
||||||
|
# Fallback: platform.processor() is reliable on Windows + some Linux,
|
||||||
|
# useless on macOS Apple Silicon (returns 'arm').
|
||||||
|
proc = platform.processor()
|
||||||
|
if proc and proc.lower() not in ("arm", "i386"):
|
||||||
|
out["brand"] = proc
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_memory(notes: list[str]) -> dict:
|
||||||
|
out: dict = {"total_bytes": None, "available_bytes": None}
|
||||||
|
try:
|
||||||
|
import psutil # type: ignore
|
||||||
|
|
||||||
|
vm = psutil.virtual_memory()
|
||||||
|
out["total_bytes"] = int(vm.total)
|
||||||
|
out["available_bytes"] = int(vm.available)
|
||||||
|
except Exception as e:
|
||||||
|
notes.append(f"psutil memory probe failed: {e}")
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_gpu_nvidia() -> list[dict]:
|
||||||
|
rc, stdout, _ = _safe_run(
|
||||||
|
[
|
||||||
|
"nvidia-smi",
|
||||||
|
"--query-gpu=name,driver_version,memory.total",
|
||||||
|
"--format=csv,noheader,nounits",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
if rc != 0 or not stdout.strip():
|
||||||
|
return []
|
||||||
|
gpus: list[dict] = []
|
||||||
|
for line in stdout.strip().splitlines():
|
||||||
|
parts = [p.strip() for p in line.split(",")]
|
||||||
|
if len(parts) < 3:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
mem_mb = int(parts[2])
|
||||||
|
except ValueError:
|
||||||
|
mem_mb = None
|
||||||
|
gpus.append({
|
||||||
|
"source": "nvidia-smi",
|
||||||
|
"name": parts[0],
|
||||||
|
"driver": parts[1],
|
||||||
|
"memory_total_mb": mem_mb,
|
||||||
|
})
|
||||||
|
return gpus
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_gpu_rocm() -> list[dict]:
|
||||||
|
rc, stdout, _ = _safe_run(
|
||||||
|
["rocm-smi", "--showproductname", "--showdriverversion", "--json"]
|
||||||
|
)
|
||||||
|
if rc != 0 or not stdout.strip():
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
data = json.loads(stdout)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return []
|
||||||
|
gpus: list[dict] = []
|
||||||
|
for card_id, card in (data or {}).items():
|
||||||
|
if not isinstance(card, dict):
|
||||||
|
continue
|
||||||
|
gpu: dict = {
|
||||||
|
"source": "rocm-smi",
|
||||||
|
"id": card_id,
|
||||||
|
"name": card.get("Card series") or card.get("Card model") or "AMD GPU",
|
||||||
|
}
|
||||||
|
driver = card.get("Driver version") or card.get("driver_version")
|
||||||
|
if driver:
|
||||||
|
gpu["driver"] = driver
|
||||||
|
gpus.append(gpu)
|
||||||
|
return gpus
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_gpu_macos() -> list[dict]:
|
||||||
|
if platform.system() != "Darwin":
|
||||||
|
return []
|
||||||
|
rc, stdout, _ = _safe_run(
|
||||||
|
["system_profiler", "SPDisplaysDataType", "-json"], timeout=4.0
|
||||||
|
)
|
||||||
|
if rc != 0 or not stdout.strip():
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
data = json.loads(stdout)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return []
|
||||||
|
gpus: list[dict] = []
|
||||||
|
for card in data.get("SPDisplaysDataType", []) or []:
|
||||||
|
gpus.append({
|
||||||
|
"source": "system_profiler",
|
||||||
|
"name": card.get("sppci_model") or card.get("_name") or "GPU",
|
||||||
|
"vendor": card.get("spdisplays_vendor"),
|
||||||
|
"metal_support": card.get("spdisplays_metalfamily"),
|
||||||
|
})
|
||||||
|
return gpus
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_gpus(notes: list[str]) -> list[dict]:
|
||||||
|
gpus: list[dict] = []
|
||||||
|
gpus.extend(_probe_gpu_nvidia())
|
||||||
|
gpus.extend(_probe_gpu_rocm())
|
||||||
|
gpus.extend(_probe_gpu_macos())
|
||||||
|
if not gpus:
|
||||||
|
notes.append(
|
||||||
|
"no GPU probes succeeded — nvidia-smi/rocm-smi/system_profiler absent or denied"
|
||||||
|
)
|
||||||
|
return gpus
|
||||||
|
|
||||||
|
|
||||||
|
def collect() -> dict:
|
||||||
|
"""Build a `system.hardware.v1` dict. Never raises."""
|
||||||
|
notes: list[str] = []
|
||||||
|
runtime = detect_runtime()
|
||||||
|
cpu = _probe_cpu(notes)
|
||||||
|
memory = _probe_memory(notes)
|
||||||
|
gpu = _probe_gpus(notes)
|
||||||
|
if runtime["in_docker"]:
|
||||||
|
notes.append(
|
||||||
|
"container masks host CPU/RAM — values reflect container limits, not host"
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"schema": SCHEMA,
|
||||||
|
"runtime": runtime,
|
||||||
|
"os": _probe_os(),
|
||||||
|
"cpu": cpu,
|
||||||
|
"memory": memory,
|
||||||
|
"gpu": gpu,
|
||||||
|
"notes": notes,
|
||||||
|
}
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
"""Redaction primitives for diagnostic bundles.
|
||||||
|
|
||||||
|
A `Redactor` carries the per-bundle salt and substitution caches so that
|
||||||
|
identical inputs (e.g. the same song path appearing in 50 log lines)
|
||||||
|
produce identical output tokens (`<song:a3f1c2>`). Different bundles get
|
||||||
|
different salts so tokens cannot be cross-correlated between exports.
|
||||||
|
|
||||||
|
Stable token grammar (see docs/diagnostics-bundle-spec.md):
|
||||||
|
<DLC_DIR> — DLC root path
|
||||||
|
<HOME> — user's home directory
|
||||||
|
<CONFIG_DIR> — slopsmith config dir
|
||||||
|
<song:hash8> — song filename / basename (8 hex chars)
|
||||||
|
<ip:hash6> — IPv4 / IPv6 address (6 hex chars)
|
||||||
|
<redacted> — bearer tokens, key=/token= query strings
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import re
|
||||||
|
import secrets
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
_IPV4_RE = re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b")
|
||||||
|
_IPV6_RE = re.compile(
|
||||||
|
r"(?<![A-Fa-f0-9:])"
|
||||||
|
r"(?:[A-Fa-f0-9]{1,4}:){2,7}[A-Fa-f0-9]{1,4}"
|
||||||
|
r"(?![A-Fa-f0-9:])"
|
||||||
|
)
|
||||||
|
_BEARER_RE = re.compile(r"(?i)\bBearer\s+[A-Za-z0-9._\-+/=]+")
|
||||||
|
_URL_USERINFO_RE = re.compile(r"(?i)(https?://)[^@/\s]+@")
|
||||||
|
_QSTRING_SECRET_RE = re.compile(
|
||||||
|
r"(?i)\b(api[_-]?key|key|token|secret|password|pwd|auth)=([^\s&\"']+)"
|
||||||
|
)
|
||||||
|
_SONG_FILENAME_RE = re.compile(
|
||||||
|
r"\b[\w()'\-+&,.!?\[\]]+\.(?:psarc|sloppak|wem|ogg|mp3|wav)\b",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Redactor:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
dlc_dir: Path | None = None,
|
||||||
|
home_dir: Path | None = None,
|
||||||
|
config_dir: Path | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._salt = secrets.token_hex(8)
|
||||||
|
self._dlc_dir = self._normalize(dlc_dir)
|
||||||
|
self._home_dir = self._normalize(home_dir)
|
||||||
|
self._config_dir = self._normalize(config_dir)
|
||||||
|
self._song_cache: dict[str, str] = {}
|
||||||
|
self._ip_cache: dict[str, str] = {}
|
||||||
|
self.counts: dict[str, int] = {
|
||||||
|
"paths_replaced": 0,
|
||||||
|
"ips_replaced": 0,
|
||||||
|
"song_names_replaced": 0,
|
||||||
|
"secrets_replaced": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _normalize(p: Path | None) -> str | None:
|
||||||
|
if p is None:
|
||||||
|
return None
|
||||||
|
# Resolve only when the path exists, so callers can pass a
|
||||||
|
# synthetic prefix (tests, container-mapped paths) without
|
||||||
|
# having Path.resolve() rewrite a missing /dlc/songs to
|
||||||
|
# C:\dlc\songs on Windows.
|
||||||
|
try:
|
||||||
|
if p.exists():
|
||||||
|
s = str(p.resolve())
|
||||||
|
else:
|
||||||
|
s = str(p)
|
||||||
|
except (OSError, RuntimeError):
|
||||||
|
s = str(p)
|
||||||
|
return s if s and s != "." else None
|
||||||
|
|
||||||
|
def _hash(self, value: str, n: int) -> str:
|
||||||
|
h = hashlib.sha256()
|
||||||
|
h.update(self._salt.encode())
|
||||||
|
h.update(value.encode())
|
||||||
|
return h.hexdigest()[:n]
|
||||||
|
|
||||||
|
def _replace_path_prefix(self, text: str, prefix: str | None, token: str) -> str:
|
||||||
|
if not prefix:
|
||||||
|
return text
|
||||||
|
# Match both forward- and backslash variants — Windows paths
|
||||||
|
# appear with backslashes in tracebacks, Linux with slashes.
|
||||||
|
candidates = {prefix, prefix.replace("/", "\\"), prefix.replace("\\", "/")}
|
||||||
|
replaced = text
|
||||||
|
for cand in candidates:
|
||||||
|
if not cand:
|
||||||
|
continue
|
||||||
|
count = replaced.count(cand)
|
||||||
|
if count:
|
||||||
|
replaced = replaced.replace(cand, token)
|
||||||
|
self.counts["paths_replaced"] += count
|
||||||
|
return replaced
|
||||||
|
|
||||||
|
def _redact_song(self, m: re.Match) -> str:
|
||||||
|
name = m.group(0)
|
||||||
|
token = self._song_cache.get(name)
|
||||||
|
if token is None:
|
||||||
|
token = f"<song:{self._hash(name, 8)}>"
|
||||||
|
self._song_cache[name] = token
|
||||||
|
self.counts["song_names_replaced"] += 1
|
||||||
|
return token
|
||||||
|
|
||||||
|
def _redact_ip(self, m: re.Match) -> str:
|
||||||
|
ip = m.group(0)
|
||||||
|
# Skip obvious non-IPs: dotted version numbers, sloppy fragments.
|
||||||
|
if ip.count(".") == 3:
|
||||||
|
try:
|
||||||
|
if not all(0 <= int(p) <= 255 for p in ip.split(".")):
|
||||||
|
return ip
|
||||||
|
except ValueError:
|
||||||
|
return ip
|
||||||
|
token = self._ip_cache.get(ip)
|
||||||
|
if token is None:
|
||||||
|
token = f"<ip:{self._hash(ip, 6)}>"
|
||||||
|
self._ip_cache[ip] = token
|
||||||
|
self.counts["ips_replaced"] += 1
|
||||||
|
return token
|
||||||
|
|
||||||
|
def _redact_secret_qstring(self, m: re.Match) -> str:
|
||||||
|
self.counts["secrets_replaced"] += 1
|
||||||
|
return f"{m.group(1)}=<redacted>"
|
||||||
|
|
||||||
|
def _redact_bearer(self, _m: re.Match) -> str:
|
||||||
|
self.counts["secrets_replaced"] += 1
|
||||||
|
return "Bearer <redacted>"
|
||||||
|
|
||||||
|
def _redact_url_userinfo(self, m: re.Match) -> str:
|
||||||
|
self.counts["secrets_replaced"] += 1
|
||||||
|
return f"{m.group(1)}<redacted>@"
|
||||||
|
|
||||||
|
def redact_text(self, text: str) -> str:
|
||||||
|
if not isinstance(text, str) or not text:
|
||||||
|
return text
|
||||||
|
# Path prefixes first (longest-match) so song-name regex never
|
||||||
|
# eats a path component.
|
||||||
|
text = self._replace_path_prefix(text, self._dlc_dir, "<DLC_DIR>")
|
||||||
|
text = self._replace_path_prefix(text, self._config_dir, "<CONFIG_DIR>")
|
||||||
|
text = self._replace_path_prefix(text, self._home_dir, "<HOME>")
|
||||||
|
text = _SONG_FILENAME_RE.sub(self._redact_song, text)
|
||||||
|
text = _IPV6_RE.sub(self._redact_ip, text)
|
||||||
|
text = _IPV4_RE.sub(self._redact_ip, text)
|
||||||
|
# URL userinfo before query-string secrets so user:pass@ is caught
|
||||||
|
# even when the URL also has token= in the query string.
|
||||||
|
text = _URL_USERINFO_RE.sub(self._redact_url_userinfo, text)
|
||||||
|
text = _QSTRING_SECRET_RE.sub(self._redact_secret_qstring, text)
|
||||||
|
text = _BEARER_RE.sub(self._redact_bearer, text)
|
||||||
|
return text
|
||||||
|
|
||||||
|
def redact_lines(self, lines):
|
||||||
|
for line in lines:
|
||||||
|
yield self.redact_text(line)
|
||||||
+289
@@ -0,0 +1,289 @@
|
|||||||
|
"""Drum kit vocabulary, presets, and drum_tab.json helpers.
|
||||||
|
|
||||||
|
The canonical drum payload in a sloppak is a top-level `drum_tab.json` file
|
||||||
|
referenced from `manifest.yaml` via the `drum_tab:` key (see
|
||||||
|
`docs/sloppak-spec.md` §5.3). This module is the source of truth for:
|
||||||
|
|
||||||
|
- the closed list of drum piece-ids that a `drum_tab.json` may reference,
|
||||||
|
- their default GM percussion MIDI notes and visual category,
|
||||||
|
- preset lane configurations for the drums plugin,
|
||||||
|
- a permissive validator + short-key wire helper used by both the writer
|
||||||
|
side (importers) and the reader side (sloppak loader + highway WS).
|
||||||
|
|
||||||
|
The schema is intentionally extensible: unknown piece-ids round-trip through
|
||||||
|
the loader so a newer sloppak can still play on an older client that just
|
||||||
|
doesn't have visuals for the new piece. Validation is strict only on the
|
||||||
|
top-level shape (`version`, `kit`, `hits` types).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import math
|
||||||
|
|
||||||
|
log = logging.getLogger("slopsmith.lib.drums")
|
||||||
|
|
||||||
|
|
||||||
|
# ── Piece vocabulary ──────────────────────────────────────────────────────────
|
||||||
|
#
|
||||||
|
# Each entry pins a closed piece-id to its default General MIDI percussion
|
||||||
|
# note(s), a category (kick/drum/cymbal — drives default shape rendering), and
|
||||||
|
# a default colour. The drums plugin reads this map on startup and uses the
|
||||||
|
# defaults to seed the user's lane configuration; users can override colours
|
||||||
|
# and shapes per lane in localStorage.
|
||||||
|
|
||||||
|
PIECES: dict[str, dict] = {
|
||||||
|
# Kick — full-width bar across all non-kick lanes.
|
||||||
|
"kick": {"midi": [35, 36], "category": "kick", "shape": "bar", "color": "#f59e0b"},
|
||||||
|
|
||||||
|
# Drums proper — rectangles. Toms ordered hi→floor.
|
||||||
|
"snare": {"midi": [38, 40], "category": "drum", "shape": "rect", "color": "#ef4444"},
|
||||||
|
"snare_xstick": {"midi": [37], "category": "drum", "shape": "rect_hatched", "color": "#dc2626"},
|
||||||
|
"tom_hi": {"midi": [50, 48], "category": "drum", "shape": "rect", "color": "#eab308"},
|
||||||
|
"tom_mid": {"midi": [47, 45], "category": "drum", "shape": "rect", "color": "#ca8a04"},
|
||||||
|
"tom_low": {"midi": [43], "category": "drum", "shape": "rect", "color": "#a16207"},
|
||||||
|
"tom_floor": {"midi": [41], "category": "drum", "shape": "rect", "color": "#854d0e"},
|
||||||
|
|
||||||
|
# Cymbals — circles. Open/closed hi-hat are distinct piece-ids, not a
|
||||||
|
# per-hit articulation flag, because hit detection must reject a
|
||||||
|
# closed-hat strike on an open-hat note (and vice versa).
|
||||||
|
"hh_closed": {"midi": [42], "category": "cymbal", "shape": "circle_filled", "color": "#22d3ee"},
|
||||||
|
"hh_open": {"midi": [46], "category": "cymbal", "shape": "circle_ring", "color": "#06b6d4"},
|
||||||
|
"hh_pedal": {"midi": [44], "category": "cymbal", "shape": "circle_small_x", "color": "#0891b2"},
|
||||||
|
# Stack — two cymbals stacked for a trashy/choked effect. GM has no
|
||||||
|
# standard for it; we reuse 30 (in GM's extended-percussion range,
|
||||||
|
# unused by real drum-kit MIDIs).
|
||||||
|
"stack": {"midi": [30], "category": "cymbal", "shape": "circle_jagged", "color": "#94a3b8"},
|
||||||
|
"crash_l": {"midi": [49], "category": "cymbal", "shape": "circle", "color": "#84cc16"},
|
||||||
|
"crash_r": {"midi": [57], "category": "cymbal", "shape": "circle", "color": "#65a30d"},
|
||||||
|
"splash": {"midi": [55], "category": "cymbal", "shape": "circle_small", "color": "#a3e635"},
|
||||||
|
"china": {"midi": [52], "category": "cymbal", "shape": "circle_jagged", "color": "#4d7c0f"},
|
||||||
|
"ride": {"midi": [51, 59], "category": "cymbal", "shape": "circle", "color": "#3b82f6"},
|
||||||
|
"ride_bell": {"midi": [53], "category": "cymbal", "shape": "circle_dot", "color": "#1d4ed8"},
|
||||||
|
# Bell cymbal — a small mounted bell, distinct from the ride's bell.
|
||||||
|
# No GM standard; we reuse 80 ("Mute Triangle"), unused in real
|
||||||
|
# drum-kit MIDIs.
|
||||||
|
"bell": {"midi": [80], "category": "cymbal", "shape": "circle_dot", "color": "#fde047"},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# Reverse map MIDI note → piece-id. First piece-id whose `midi` list contains
|
||||||
|
# the note wins (PIECES is iteration-ordered so the "preferred" piece-id for a
|
||||||
|
# shared MIDI is the one declared earlier). Built once at import time.
|
||||||
|
_MIDI_TO_PIECE: dict[int, str] = {}
|
||||||
|
for _pid, _meta in PIECES.items():
|
||||||
|
for _m in _meta["midi"]:
|
||||||
|
_MIDI_TO_PIECE.setdefault(_m, _pid)
|
||||||
|
|
||||||
|
|
||||||
|
def midi_to_piece(midi: int) -> str | None:
|
||||||
|
"""Return the canonical piece-id for a GM percussion MIDI note, or None
|
||||||
|
if the note isn't mapped (e.g. cowbell, tambourine — extensible later)."""
|
||||||
|
return _MIDI_TO_PIECE.get(int(midi))
|
||||||
|
|
||||||
|
|
||||||
|
def piece_to_default_midi(piece: str) -> list[int]:
|
||||||
|
"""Return the GM MIDI notes that map to `piece` by default. Empty list for
|
||||||
|
unknown piece-ids — callers should treat that as "unmapped" rather than
|
||||||
|
crashing, so a newer sloppak's unknown piece round-trips silently."""
|
||||||
|
entry = PIECES.get(piece)
|
||||||
|
return list(entry["midi"]) if entry else []
|
||||||
|
|
||||||
|
|
||||||
|
def piece_default_shape(piece: str) -> str:
|
||||||
|
"""Default rendering shape for a piece-id. `"rect"` fallback so an
|
||||||
|
unknown piece still draws something the user can see."""
|
||||||
|
entry = PIECES.get(piece)
|
||||||
|
return entry["shape"] if entry else "rect"
|
||||||
|
|
||||||
|
|
||||||
|
def piece_default_color(piece: str) -> str:
|
||||||
|
"""Default colour for a piece-id. Neutral grey fallback for unknown."""
|
||||||
|
entry = PIECES.get(piece)
|
||||||
|
return entry["color"] if entry else "#9ca3af"
|
||||||
|
|
||||||
|
|
||||||
|
def piece_category(piece: str) -> str:
|
||||||
|
"""Category (`kick`/`drum`/`cymbal`) — `"drum"` fallback for unknown."""
|
||||||
|
entry = PIECES.get(piece)
|
||||||
|
return entry["category"] if entry else "drum"
|
||||||
|
|
||||||
|
|
||||||
|
# ── Preset lane configurations ────────────────────────────────────────────────
|
||||||
|
#
|
||||||
|
# Each preset is a list of `lane` dicts. A lane carries:
|
||||||
|
# - `pieces`: list of piece-ids that route to this lane (multiple → shared)
|
||||||
|
# - `label`: short header text
|
||||||
|
# Visual fields (color, shape, weight) are optional; the renderer falls back
|
||||||
|
# to the per-piece defaults above. The drums plugin layers user customisation
|
||||||
|
# on top of these.
|
||||||
|
|
||||||
|
PRESET_RB4 = [
|
||||||
|
{"pieces": ["kick"], "label": "Ki"},
|
||||||
|
{"pieces": ["snare", "snare_xstick"], "label": "Sn"},
|
||||||
|
{"pieces": ["hh_closed", "hh_open", "hh_pedal"], "label": "HH"},
|
||||||
|
{"pieces": ["tom_hi", "tom_mid"], "label": "T"},
|
||||||
|
{"pieces": ["tom_low", "tom_floor"], "label": "FT"},
|
||||||
|
{"pieces": ["crash_l", "crash_r", "splash", "china", "stack"], "label": "Cr"},
|
||||||
|
{"pieces": ["ride", "ride_bell", "bell"], "label": "Ri"},
|
||||||
|
]
|
||||||
|
|
||||||
|
# 8-lane layout matching the legacy drums plugin v3 (HH / Sn / T1 / T2 / T3 /
|
||||||
|
# Cr / Ri / Ki) so existing sloppaks keep their familiar lane order when the
|
||||||
|
# rewrite ships.
|
||||||
|
PRESET_PHASESHIFT8 = [
|
||||||
|
{"pieces": ["hh_closed", "hh_open", "hh_pedal"], "label": "HH"},
|
||||||
|
{"pieces": ["snare", "snare_xstick"], "label": "Sn"},
|
||||||
|
{"pieces": ["tom_hi"], "label": "T1"},
|
||||||
|
{"pieces": ["tom_mid"], "label": "T2"},
|
||||||
|
{"pieces": ["tom_low", "tom_floor"], "label": "T3"},
|
||||||
|
{"pieces": ["crash_l", "crash_r", "splash", "china", "stack"], "label": "Cr"},
|
||||||
|
{"pieces": ["ride", "ride_bell", "bell"], "label": "Ri"},
|
||||||
|
{"pieces": ["kick"], "label": "Ki"},
|
||||||
|
]
|
||||||
|
|
||||||
|
# One lane per piece-id — for users with a full e-kit who want every piece on
|
||||||
|
# its own column. Order roughly mirrors a physical kit left→right.
|
||||||
|
PRESET_EKIT_FULL = [
|
||||||
|
{"pieces": ["hh_pedal"], "label": "HH-p"},
|
||||||
|
{"pieces": ["hh_closed"], "label": "HH-c"},
|
||||||
|
{"pieces": ["hh_open"], "label": "HH-o"},
|
||||||
|
{"pieces": ["snare_xstick"], "label": "Sn-x"},
|
||||||
|
{"pieces": ["snare"], "label": "Sn"},
|
||||||
|
{"pieces": ["tom_hi"], "label": "T1"},
|
||||||
|
{"pieces": ["tom_mid"], "label": "T2"},
|
||||||
|
{"pieces": ["tom_low"], "label": "T3"},
|
||||||
|
{"pieces": ["tom_floor"], "label": "FT"},
|
||||||
|
{"pieces": ["stack"], "label": "Stk"},
|
||||||
|
{"pieces": ["crash_l"], "label": "Cr-L"},
|
||||||
|
{"pieces": ["splash"], "label": "Sp"},
|
||||||
|
{"pieces": ["china"], "label": "Ch"},
|
||||||
|
{"pieces": ["ride"], "label": "Ri"},
|
||||||
|
{"pieces": ["ride_bell"], "label": "Ri-B"},
|
||||||
|
{"pieces": ["bell"], "label": "Bl"},
|
||||||
|
{"pieces": ["crash_r"], "label": "Cr-R"},
|
||||||
|
{"pieces": ["kick"], "label": "Ki"},
|
||||||
|
]
|
||||||
|
|
||||||
|
PRESETS: dict[str, list[dict]] = {
|
||||||
|
"rb4": PRESET_RB4,
|
||||||
|
"phase_shift_8": PRESET_PHASESHIFT8,
|
||||||
|
"ekit_full": PRESET_EKIT_FULL,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── drum_tab.json schema helpers ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
# Default velocity when a hit omits `v`. Matches spec §5.3 ("v is optional,
|
||||||
|
# defaults to 100 — keeps simple charts terse").
|
||||||
|
DEFAULT_VELOCITY = 100
|
||||||
|
|
||||||
|
# Current `version` written by importers. Readers MUST accept any version they
|
||||||
|
# recognise; an unknown version is logged at DEBUG level on every call to
|
||||||
|
# validate_drum_tab() and the payload is still passed
|
||||||
|
# through (per Principle IV, additive evolution).
|
||||||
|
SCHEMA_VERSION = 1
|
||||||
|
|
||||||
|
|
||||||
|
def validate_drum_tab(data: object) -> tuple[bool, str]:
|
||||||
|
"""Light schema check for a parsed `drum_tab.json` payload.
|
||||||
|
|
||||||
|
Returns `(ok, reason)`. Accepts both `version: 1` (current) and absent
|
||||||
|
`version` (treat as 1) for forward-compat with hand-edited tabs.
|
||||||
|
`hits[]` is required and must be a list; individual hits are NOT
|
||||||
|
validated here — per-hit filtering happens in `hit_to_wire()` /
|
||||||
|
`hits_to_wire()` at WS-stream time, so a single malformed hit cannot
|
||||||
|
disqualify the whole tab.
|
||||||
|
"""
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return False, "drum_tab payload must be a JSON object"
|
||||||
|
hits = data.get("hits")
|
||||||
|
if not isinstance(hits, list):
|
||||||
|
return False, "drum_tab.hits must be a list"
|
||||||
|
kit = data.get("kit", [])
|
||||||
|
if kit is not None and not isinstance(kit, list):
|
||||||
|
return False, "drum_tab.kit must be a list (or omitted)"
|
||||||
|
ver = data.get("version", SCHEMA_VERSION)
|
||||||
|
if isinstance(ver, bool) or not isinstance(ver, int):
|
||||||
|
return False, "drum_tab.version must be an integer"
|
||||||
|
if ver != SCHEMA_VERSION:
|
||||||
|
log.debug("drum_tab: unknown schema version %r — passing through", ver)
|
||||||
|
return True, ""
|
||||||
|
|
||||||
|
|
||||||
|
def hit_to_wire(hit: dict) -> dict | None:
|
||||||
|
"""Normalise one hit dict into the short-key wire form streamed by
|
||||||
|
`/ws/highway/{filename}`. Returns None on a malformed hit (missing `t`
|
||||||
|
or `p`) so the loader can drop just that entry without aborting the
|
||||||
|
whole tab.
|
||||||
|
|
||||||
|
Wire keys (all optional except `t`, `p`):
|
||||||
|
t float seconds required, monotonic
|
||||||
|
p string piece-id required, free-form (validated against PIECES
|
||||||
|
by the client; unknown ids render as `"rect"`)
|
||||||
|
v int 1-127 velocity (omitted when absent; client defaults
|
||||||
|
to DEFAULT_VELOCITY)
|
||||||
|
g bool ghost note
|
||||||
|
f bool flam
|
||||||
|
k float seconds cymbal-choke tail duration
|
||||||
|
"""
|
||||||
|
if not isinstance(hit, dict):
|
||||||
|
return None
|
||||||
|
t_raw = hit.get("t")
|
||||||
|
if isinstance(t_raw, bool):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
t = float(t_raw) # type: ignore[arg-type]
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
if not math.isfinite(t):
|
||||||
|
return None
|
||||||
|
p = hit.get("p")
|
||||||
|
if not isinstance(p, str) or not p:
|
||||||
|
return None
|
||||||
|
out: dict = {"t": round(t, 3), "p": p}
|
||||||
|
v = hit.get("v")
|
||||||
|
if not isinstance(v, bool) and isinstance(v, (int, float)) and math.isfinite(v) and 1 <= int(v) <= 127:
|
||||||
|
out["v"] = int(v)
|
||||||
|
if bool(hit.get("g")):
|
||||||
|
out["g"] = True
|
||||||
|
if bool(hit.get("f")):
|
||||||
|
out["f"] = True
|
||||||
|
k = hit.get("k")
|
||||||
|
if not isinstance(k, bool) and isinstance(k, (int, float)) and math.isfinite(k) and k > 0:
|
||||||
|
out["k"] = round(float(k), 3)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def hits_to_wire(hits: list[dict]) -> list[dict]:
|
||||||
|
"""Vectorised `hit_to_wire` — drops malformed entries, sorts by time."""
|
||||||
|
out: list[dict] = []
|
||||||
|
for h in hits:
|
||||||
|
w = hit_to_wire(h)
|
||||||
|
if w is not None:
|
||||||
|
out.append(w)
|
||||||
|
out.sort(key=lambda h: h["t"])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def normalise_kit(kit: list | None) -> list[dict]:
|
||||||
|
"""Normalise the `kit[]` legend: each entry becomes `{"id": str, "name":
|
||||||
|
str}`. Unknown piece-ids are kept (forward-compat) with a title-cased
|
||||||
|
fallback name. Returns an empty list for missing/empty kit (the client
|
||||||
|
will derive the kit from the union of `hits[].p` in that case)."""
|
||||||
|
if not isinstance(kit, list):
|
||||||
|
return []
|
||||||
|
out: list[dict] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
for entry in kit:
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
continue
|
||||||
|
pid = entry.get("id")
|
||||||
|
if not isinstance(pid, str) or not pid or pid in seen:
|
||||||
|
continue
|
||||||
|
seen.add(pid)
|
||||||
|
name = entry.get("name")
|
||||||
|
if not isinstance(name, str) or not name:
|
||||||
|
name = pid.replace("_", " ").title()
|
||||||
|
out.append({"id": pid, "name": name})
|
||||||
|
return out
|
||||||
+307
@@ -0,0 +1,307 @@
|
|||||||
|
"""Generate MIDI and render audio from a Guitar Pro file."""
|
||||||
|
|
||||||
|
import glob
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
log = logging.getLogger("slopsmith.lib.gp2midi")
|
||||||
|
|
||||||
|
import guitarpro
|
||||||
|
from midiutil import MIDIFile
|
||||||
|
|
||||||
|
GP_TICKS_PER_QUARTER = 960
|
||||||
|
|
||||||
|
# Standard tuning MIDI values (GP string order: 1=high, 6=low)
|
||||||
|
STANDARD_6 = [64, 59, 55, 50, 45, 40] # e B G D A E
|
||||||
|
STANDARD_4 = [43, 38, 33, 28] # G D A E (bass)
|
||||||
|
|
||||||
|
|
||||||
|
def gp_to_midi(gp_path: str, output_midi: str, track_indices: list[int] | None = None,
|
||||||
|
force_standard_tuning: bool = False) -> str:
|
||||||
|
"""Convert Guitar Pro file to MIDI.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
gp_path: Path to .gp5/.gp4/.gp3 file
|
||||||
|
output_midi: Output .mid file path
|
||||||
|
track_indices: Which tracks to include (None = all non-percussion)
|
||||||
|
force_standard_tuning: If True, use E standard tuning for all instruments
|
||||||
|
(keeps fret numbers, changes the pitch of open strings)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to the MIDI file
|
||||||
|
"""
|
||||||
|
song = guitarpro.parse(gp_path)
|
||||||
|
|
||||||
|
if track_indices is None:
|
||||||
|
track_indices = list(range(len(song.tracks)))
|
||||||
|
|
||||||
|
midi = MIDIFile(
|
||||||
|
len(track_indices),
|
||||||
|
ticks_per_quarternote=GP_TICKS_PER_QUARTER,
|
||||||
|
)
|
||||||
|
|
||||||
|
for midi_track_idx, gp_track_idx in enumerate(track_indices):
|
||||||
|
track = song.tracks[gp_track_idx]
|
||||||
|
is_perc = track.isPercussionTrack
|
||||||
|
|
||||||
|
# MIDI channel: percussion must be 9, others avoid 9
|
||||||
|
if is_perc:
|
||||||
|
channel = 9
|
||||||
|
else:
|
||||||
|
channel = midi_track_idx if midi_track_idx < 9 else midi_track_idx + 1
|
||||||
|
channel = min(channel, 15)
|
||||||
|
|
||||||
|
midi.addTrackName(midi_track_idx, 0, track.name)
|
||||||
|
midi.addTempo(midi_track_idx, 0, song.tempo)
|
||||||
|
|
||||||
|
# Instrument and volume from GP channel data
|
||||||
|
gp_ch = track.channel
|
||||||
|
if gp_ch and not is_perc:
|
||||||
|
midi.addProgramChange(midi_track_idx, channel, 0, gp_ch.instrument)
|
||||||
|
elif not is_perc:
|
||||||
|
midi.addProgramChange(midi_track_idx, channel, 0, 29) # overdriven guitar
|
||||||
|
|
||||||
|
# Volume (CC7) and pan (CC10)
|
||||||
|
if gp_ch:
|
||||||
|
vol = min(127, gp_ch.volume)
|
||||||
|
pan = min(127, gp_ch.balance)
|
||||||
|
midi.addControllerEvent(midi_track_idx, channel, 0, 7, vol)
|
||||||
|
midi.addControllerEvent(midi_track_idx, channel, 0, 10, pan)
|
||||||
|
|
||||||
|
# Tempo changes
|
||||||
|
tempo_added = set()
|
||||||
|
for measure in track.measures:
|
||||||
|
for voice in measure.voices:
|
||||||
|
for beat in voice.beats:
|
||||||
|
if beat.effect and beat.effect.mixTableChange:
|
||||||
|
mtc = beat.effect.mixTableChange
|
||||||
|
if mtc.tempo and mtc.tempo.value > 0:
|
||||||
|
tick_time = beat.start / GP_TICKS_PER_QUARTER
|
||||||
|
if tick_time not in tempo_added:
|
||||||
|
midi.addTempo(midi_track_idx, tick_time, mtc.tempo.value)
|
||||||
|
tempo_added.add(tick_time)
|
||||||
|
|
||||||
|
# Notes
|
||||||
|
for measure in track.measures:
|
||||||
|
for voice in measure.voices:
|
||||||
|
for beat in voice.beats:
|
||||||
|
if not beat.notes:
|
||||||
|
continue
|
||||||
|
|
||||||
|
beat_time = beat.start / GP_TICKS_PER_QUARTER
|
||||||
|
|
||||||
|
dur_quarters = 4.0 / beat.duration.value
|
||||||
|
if beat.duration.isDotted:
|
||||||
|
dur_quarters *= 1.5
|
||||||
|
if beat.duration.tuplet.enters > 0 and beat.duration.tuplet.times > 0:
|
||||||
|
dur_quarters *= beat.duration.tuplet.times / beat.duration.tuplet.enters
|
||||||
|
|
||||||
|
for note in beat.notes:
|
||||||
|
if note.type == guitarpro.NoteType.rest:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if force_standard_tuning and not is_perc:
|
||||||
|
num_strings = len(track.strings)
|
||||||
|
std = STANDARD_4 if num_strings == 4 else STANDARD_6
|
||||||
|
string_midi = std[note.string - 1] if note.string - 1 < len(std) else track.strings[note.string - 1].value
|
||||||
|
else:
|
||||||
|
string_midi = track.strings[note.string - 1].value
|
||||||
|
pitch = string_midi + note.value
|
||||||
|
|
||||||
|
if note.type == guitarpro.NoteType.dead:
|
||||||
|
dur_q = 0.05
|
||||||
|
else:
|
||||||
|
dur_q = dur_quarters
|
||||||
|
|
||||||
|
velocity = note.velocity
|
||||||
|
if note.effect.ghostNote:
|
||||||
|
velocity = max(20, velocity // 2)
|
||||||
|
|
||||||
|
# Skip invalid notes that would cause midiutil to crash
|
||||||
|
if dur_q <= 0:
|
||||||
|
dur_q = 0.05
|
||||||
|
if pitch < 0 or pitch > 127:
|
||||||
|
continue
|
||||||
|
if velocity <= 0:
|
||||||
|
velocity = 1
|
||||||
|
|
||||||
|
midi.addNote(
|
||||||
|
midi_track_idx, channel,
|
||||||
|
pitch, beat_time, dur_q, velocity,
|
||||||
|
)
|
||||||
|
|
||||||
|
with open(output_midi, "wb") as f:
|
||||||
|
try:
|
||||||
|
midi.writeFile(f)
|
||||||
|
except IndexError:
|
||||||
|
# midiutil can crash with "pop from empty list" on malformed note events
|
||||||
|
# Retry with deinterleave disabled
|
||||||
|
f.seek(0)
|
||||||
|
f.truncate()
|
||||||
|
midi.close()
|
||||||
|
midi.writeFile(f)
|
||||||
|
|
||||||
|
return output_midi
|
||||||
|
|
||||||
|
|
||||||
|
def _find_soundfont() -> str | None:
|
||||||
|
"""Locate a .sf2 soundfont for MIDI rendering.
|
||||||
|
|
||||||
|
Precedence:
|
||||||
|
1. ``SLOPSMITH_SOUNDFONT`` env var (user override / desktop-app-supplied)
|
||||||
|
2. Bundled ``<RESOURCESPATH>/soundfonts/*.sf2`` (Electron desktop builds)
|
||||||
|
3. Common system locations per OS.
|
||||||
|
"""
|
||||||
|
override = os.environ.get("SLOPSMITH_SOUNDFONT")
|
||||||
|
if override:
|
||||||
|
if os.path.isfile(override):
|
||||||
|
return override
|
||||||
|
log.warning("SLOPSMITH_SOUNDFONT is set to %r but that file does not exist; falling back to other sources", override)
|
||||||
|
|
||||||
|
resources = os.environ.get("RESOURCESPATH")
|
||||||
|
if resources:
|
||||||
|
matches = sorted(glob.glob(os.path.join(resources, "soundfonts", "*.sf2")))
|
||||||
|
if matches:
|
||||||
|
return matches[0]
|
||||||
|
|
||||||
|
candidates: list[str] = []
|
||||||
|
if sys.platform.startswith("linux"):
|
||||||
|
candidates += [
|
||||||
|
"/usr/share/soundfonts/FluidR3_GM.sf2",
|
||||||
|
"/usr/share/soundfonts/FluidR3_GS.sf2",
|
||||||
|
"/usr/share/soundfonts/default.sf2",
|
||||||
|
"/usr/share/sounds/sf2/FluidR3_GM.sf2",
|
||||||
|
"/usr/share/sounds/sf2/default-GM.sf2",
|
||||||
|
]
|
||||||
|
elif sys.platform == "darwin":
|
||||||
|
candidates += [
|
||||||
|
"/opt/homebrew/share/sounds/sf2/FluidR3_GM.sf2",
|
||||||
|
"/opt/homebrew/share/soundfonts/FluidR3_GM.sf2",
|
||||||
|
"/usr/local/share/sounds/sf2/FluidR3_GM.sf2",
|
||||||
|
"/usr/local/share/soundfonts/FluidR3_GM.sf2",
|
||||||
|
]
|
||||||
|
elif sys.platform == "win32":
|
||||||
|
appdata = os.environ.get("APPDATA")
|
||||||
|
if appdata:
|
||||||
|
# "Slopsmith" matches slopsmith-desktop's Electron productName
|
||||||
|
# (app.getPath('userData') resolves to %APPDATA%\Slopsmith on Windows).
|
||||||
|
for pattern in (
|
||||||
|
os.path.join(appdata, "Slopsmith", "soundfonts", "*.sf2"),
|
||||||
|
os.path.join(appdata, "SoundFonts", "*.sf2"),
|
||||||
|
):
|
||||||
|
candidates += sorted(glob.glob(pattern))
|
||||||
|
|
||||||
|
for path in candidates:
|
||||||
|
if os.path.isfile(path):
|
||||||
|
return path
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _soundfont_install_hint() -> str:
|
||||||
|
if sys.platform.startswith("linux"):
|
||||||
|
return (
|
||||||
|
"Install a soundfont:\n"
|
||||||
|
" Arch/Manjaro: sudo pacman -S soundfont-fluid\n"
|
||||||
|
" Debian/Ubuntu: sudo apt install fluid-soundfont-gm\n"
|
||||||
|
" Fedora: sudo dnf install fluid-soundfont-gm"
|
||||||
|
)
|
||||||
|
if sys.platform == "darwin":
|
||||||
|
# Homebrew's fluid-synth formula doesn't bundle a soundfont; the user
|
||||||
|
# needs to fetch one separately (confirmed 2026-04 against the
|
||||||
|
# upstream formula).
|
||||||
|
return (
|
||||||
|
"Download a soundfont (e.g. GeneralUser GS from schristiancollins.com "
|
||||||
|
"or FluidR3_GM from musical-artifacts.com) and either place the .sf2 "
|
||||||
|
"file in /usr/local/share/sounds/sf2/ (Intel) or "
|
||||||
|
"/opt/homebrew/share/sounds/sf2/ (Apple Silicon), or set the "
|
||||||
|
"SLOPSMITH_SOUNDFONT environment variable to its full path."
|
||||||
|
)
|
||||||
|
if sys.platform == "win32":
|
||||||
|
return (
|
||||||
|
"Download a soundfont (e.g. GeneralUser GS from schristiancollins.com or "
|
||||||
|
"FluidR3_GM from musical-artifacts.com) and either place the .sf2 file in "
|
||||||
|
"%APPDATA%\\Slopsmith\\soundfonts\\ or set the SLOPSMITH_SOUNDFONT "
|
||||||
|
"environment variable to its full path."
|
||||||
|
)
|
||||||
|
return "Set SLOPSMITH_SOUNDFONT to the full path of a .sf2 file."
|
||||||
|
|
||||||
|
|
||||||
|
def _fluidsynth_install_hint() -> str:
|
||||||
|
if sys.platform.startswith("linux"):
|
||||||
|
return (
|
||||||
|
"Install fluidsynth:\n"
|
||||||
|
" Arch/Manjaro: sudo pacman -S fluidsynth\n"
|
||||||
|
" Debian/Ubuntu: sudo apt install fluidsynth\n"
|
||||||
|
" Fedora: sudo dnf install fluidsynth"
|
||||||
|
)
|
||||||
|
if sys.platform == "darwin":
|
||||||
|
return "Install fluidsynth with Homebrew: brew install fluid-synth"
|
||||||
|
if sys.platform == "win32":
|
||||||
|
return (
|
||||||
|
"Install fluidsynth (https://github.com/FluidSynth/fluidsynth/releases) and "
|
||||||
|
"ensure fluidsynth.exe is on your PATH."
|
||||||
|
)
|
||||||
|
return "Install fluidsynth and ensure it is on PATH."
|
||||||
|
|
||||||
|
|
||||||
|
def render_midi_to_audio(midi_path: str, output_path: str) -> str:
|
||||||
|
"""Render MIDI to OGG audio using fluidsynth."""
|
||||||
|
soundfont = _find_soundfont()
|
||||||
|
if not soundfont:
|
||||||
|
raise RuntimeError(
|
||||||
|
"No soundfont found. " + _soundfont_install_hint()
|
||||||
|
)
|
||||||
|
|
||||||
|
wav_path = output_path + ".wav"
|
||||||
|
ogg_path = output_path + ".ogg"
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
["fluidsynth", "-ni", "-T", "wav", "-F", wav_path, "-r", "44100", soundfont, midi_path],
|
||||||
|
capture_output=True, text=True, timeout=600,
|
||||||
|
)
|
||||||
|
except FileNotFoundError as e:
|
||||||
|
raise RuntimeError("fluidsynth not found. " + _fluidsynth_install_hint()) from e
|
||||||
|
|
||||||
|
if result.returncode != 0 or not os.path.exists(wav_path):
|
||||||
|
raise RuntimeError(f"fluidsynth failed: {result.stderr[-300:]}")
|
||||||
|
|
||||||
|
result = subprocess.run(
|
||||||
|
["ffmpeg", "-y", "-i", wav_path, "-q:a", "6", ogg_path],
|
||||||
|
capture_output=True, timeout=60,
|
||||||
|
)
|
||||||
|
if result.returncode == 0 and os.path.exists(ogg_path):
|
||||||
|
os.remove(wav_path)
|
||||||
|
return ogg_path
|
||||||
|
|
||||||
|
return wav_path
|
||||||
|
|
||||||
|
|
||||||
|
def gp_to_audio(gp_path: str, output_path: str,
|
||||||
|
track_indices: list[int] | None = None,
|
||||||
|
force_standard_tuning: bool = False) -> str:
|
||||||
|
"""Convert Guitar Pro file directly to audio.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
gp_path: Path to .gp5 file
|
||||||
|
output_path: Output audio file path (without extension)
|
||||||
|
track_indices: Which tracks (None = all including drums)
|
||||||
|
force_standard_tuning: Force E standard tuning (keeps frets, changes pitch)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to the audio file
|
||||||
|
"""
|
||||||
|
tmp_midi = tempfile.mktemp(suffix=".mid", prefix="rs_midi_")
|
||||||
|
try:
|
||||||
|
tuning_label = " (E Standard)" if force_standard_tuning else ""
|
||||||
|
log.info("Generating MIDI from %s%s", Path(gp_path).name, tuning_label)
|
||||||
|
gp_to_midi(gp_path, tmp_midi, track_indices, force_standard_tuning)
|
||||||
|
log.info("Rendering audio with FluidSynth...")
|
||||||
|
return render_midi_to_audio(tmp_midi, output_path)
|
||||||
|
finally:
|
||||||
|
if os.path.exists(tmp_midi):
|
||||||
|
os.remove(tmp_midi)
|
||||||
@@ -0,0 +1,529 @@
|
|||||||
|
"""Guitar Pro → Sloppak Notation Format importer (GPIF: .gpx GP6 / .gp GP7-8).
|
||||||
|
|
||||||
|
Builds the per-arrangement ``notation_<id>.json`` payload documented in
|
||||||
|
``docs/sloppak-spec.md`` §5.3 from a parsed GPIF score, so piano/keys tracks
|
||||||
|
imported from Guitar Pro carry real engraving data (measures → staves →
|
||||||
|
voices → beats → notes with absolute MIDI pitch) instead of only the
|
||||||
|
``midi = string*24 + fret`` guitar wire encoding.
|
||||||
|
|
||||||
|
Voice → staff routing (salvaged from the superseded PR #703 ``stf`` wire-field
|
||||||
|
approach): the GP author's voice position within a bar decides the hand —
|
||||||
|
voice position 0 lands on the ``rh`` staff (treble, ``G2``), voice positions
|
||||||
|
≥ 1 land on the ``lh`` staff (bass, ``F4``). A forced-LH track (the merged
|
||||||
|
``Piano LH`` partner of an LH/RH pair, or a standalone track whose name ends
|
||||||
|
in ``LH``) routes every voice to ``lh``. This preserves authored hand
|
||||||
|
crossings instead of inferring hands from pitch.
|
||||||
|
|
||||||
|
Timing reuses the same machinery as ``gp2rs_gpx.convert_file`` — the
|
||||||
|
bar-indexed tempo map, per-beat rhythm durations (dots + tuplets; see
|
||||||
|
``_beat_secs`` for the one deliberate double-dot divergence), and
|
||||||
|
``_note_midi`` — so the
|
||||||
|
notation beats line up with the RS-XML notes the highway plays (see
|
||||||
|
slopsmith#618 for the longer-term goal of sharing the note-building walk
|
||||||
|
itself, and slopsmith#261 for the time-signature-denominator pitfalls the
|
||||||
|
``beat_groups`` emission here exists to avoid re-introducing).
|
||||||
|
|
||||||
|
Where this plugs in: ``gp2rs_gpx.convert_file`` calls
|
||||||
|
``convert_track_to_notation`` for every keys track and writes the payload as
|
||||||
|
an ``<xml-stem>.notation.json`` sidecar next to the arrangement XML. The
|
||||||
|
sloppak *assembly* step (which assigns arrangement ids and writes
|
||||||
|
``manifest.yaml`` — today that lives in the editor plugin's create-mode save)
|
||||||
|
then renames the sidecar into place via ``attach_notation_to_sloppak``.
|
||||||
|
|
||||||
|
Analogous to ``gp2rs.convert_drum_track_to_drumtab`` for the drum tab format.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
import xml.etree.ElementTree as ET
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import notation as notation_mod
|
||||||
|
|
||||||
|
log = logging.getLogger("slopsmith.lib.gp2notation")
|
||||||
|
|
||||||
|
|
||||||
|
# GPX NoteValue string → notation duration denominator (sloppak-spec §5.3:
|
||||||
|
# 1=whole … 32=thirty-second). 64th/128th are below the schema floor; those
|
||||||
|
# beats are DROPPED with a warning (v1 non-features doctrine: drop, never
|
||||||
|
# approximate) — clamping the written value to 32 while time advances by the
|
||||||
|
# true 64th/128th span would emit self-contradictory notation (overlapping
|
||||||
|
# written durations).
|
||||||
|
_NOTE_VALUE_DEN: dict[str, int] = {
|
||||||
|
"Whole": 1, "Half": 2, "Quarter": 4, "Eighth": 8,
|
||||||
|
"16th": 16, "32nd": 32,
|
||||||
|
}
|
||||||
|
|
||||||
|
_SUB_FLOOR_NOTE_VALUES = frozenset({"64th", "128th"})
|
||||||
|
|
||||||
|
_STAFF_DEFS: dict[str, dict] = {
|
||||||
|
"rh": {"id": "rh", "clef": "G2", "label": "Right Hand"},
|
||||||
|
"lh": {"id": "lh", "clef": "F4", "label": "Left Hand"},
|
||||||
|
}
|
||||||
|
|
||||||
|
# Track names that force every voice onto the lh staff (e.g. the "Piano LH"
|
||||||
|
# half of an LH/RH pair imported standalone).
|
||||||
|
_LH_NAME_RE = re.compile(r"\blh\b\s*$", re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
def _children(root: ET.Element, tag: str) -> list[ET.Element]:
|
||||||
|
"""Children of ``root/<tag>``, or ``[]`` — explicit None check (an empty
|
||||||
|
Element is falsy, so ``find(...) or []`` would mis-handle it and trips
|
||||||
|
ElementTree's truth-value DeprecationWarning)."""
|
||||||
|
el = root.find(tag)
|
||||||
|
return list(el) if el is not None else []
|
||||||
|
|
||||||
|
|
||||||
|
def beat_groups_for(num: int, den: int) -> list[int] | None:
|
||||||
|
"""Return the spec ``beat_groups`` list for a time signature, or ``None``.
|
||||||
|
|
||||||
|
Simple meters (denominator < 8, e.g. 2/4, 3/4, 4/4) have unambiguous
|
||||||
|
grouping and omit the field. Compound meters built from dotted beats
|
||||||
|
group in threes (6/8 → [3, 3]; 9/8 → [3, 3, 3]; 12/8 → [3, 3, 3, 3]);
|
||||||
|
the common irregular meters get their conventional default (5/8 → [2, 3];
|
||||||
|
7/8 → [2, 2, 3]). Anything else is omitted — the renderer's default
|
||||||
|
grouping applies (sloppak-spec §5.3: the field is renderer-agnostic and
|
||||||
|
optional).
|
||||||
|
"""
|
||||||
|
if den < 8:
|
||||||
|
return None
|
||||||
|
if num > 3 and num % 3 == 0:
|
||||||
|
return [3] * (num // 3)
|
||||||
|
if num == 5:
|
||||||
|
return [2, 3]
|
||||||
|
if num == 7:
|
||||||
|
return [2, 2, 3]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _rhythm_fields(beat_el: ET.Element, rhythms_dict: dict) -> tuple[int, int, list[int] | None]:
|
||||||
|
"""Return ``(dur, dot, tu)`` notation fields for a GPIF beat.
|
||||||
|
|
||||||
|
- ``dur`` — duration denominator from the referenced Rhythm's NoteValue
|
||||||
|
(unknown values default to quarter, matching ``_beat_dur_secs``).
|
||||||
|
- ``dot`` — augmentation dots from ``<AugmentationDot count="N">``,
|
||||||
|
clamped to the schema's 0–2 range. Beat *times* advance via
|
||||||
|
``_beat_secs``, which applies the matching multiplier (×1.5 single,
|
||||||
|
×1.75 double), so the written dots and the emitted times agree.
|
||||||
|
- ``tu`` — ``[num, den]`` tuplet from ``<PrimaryTuplet>``, or ``None``.
|
||||||
|
"""
|
||||||
|
dur = 4
|
||||||
|
dot = 0
|
||||||
|
tu: list[int] | None = None
|
||||||
|
rref = beat_el.find("Rhythm")
|
||||||
|
if rref is not None:
|
||||||
|
rhythm = rhythms_dict.get(rref.get("ref", ""))
|
||||||
|
if rhythm is not None:
|
||||||
|
nv = rhythm.findtext("NoteValue", "Quarter")
|
||||||
|
dur = _NOTE_VALUE_DEN.get(nv, 4)
|
||||||
|
dot_el = rhythm.find("AugmentationDot")
|
||||||
|
if dot_el is not None:
|
||||||
|
try:
|
||||||
|
dot = max(1, min(2, int(dot_el.get("count", 1))))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
dot = 1
|
||||||
|
tuplet = rhythm.find("PrimaryTuplet")
|
||||||
|
if tuplet is not None:
|
||||||
|
try:
|
||||||
|
t_num = int(tuplet.get("num", 1))
|
||||||
|
t_den = int(tuplet.get("den", 1))
|
||||||
|
if t_num > 0 and t_den > 0 and (t_num, t_den) != (1, 1):
|
||||||
|
tu = [t_num, t_den]
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
pass
|
||||||
|
return dur, dot, tu
|
||||||
|
|
||||||
|
|
||||||
|
def _beat_secs(beat_el: ET.Element, rhythms_dict: dict, tempo_bpm: float) -> float:
|
||||||
|
"""Duration of a GPIF beat in seconds, honouring the full dot count.
|
||||||
|
|
||||||
|
Mirrors ``gp2rs_gpx._beat_dur_secs`` except for double dots: that helper
|
||||||
|
applies ×1.5 for any ``<AugmentationDot>`` regardless of its ``count``
|
||||||
|
attribute, which would make a written ``dot: 2`` disagree with the
|
||||||
|
emitted absolute beat times (overlapping engraving). Here a single dot
|
||||||
|
is ×1.5 and a double dot ×1.75, so the notation walk stays
|
||||||
|
self-consistent; for the rare double-dotted keys beat this intentionally
|
||||||
|
diverges from the RS-XML walk's single-dot approximation.
|
||||||
|
"""
|
||||||
|
from gp2rs_gpx import _NOTE_VALUE_QN
|
||||||
|
|
||||||
|
dur_qn = 0.25
|
||||||
|
rref = beat_el.find("Rhythm")
|
||||||
|
if rref is not None:
|
||||||
|
rhythm = rhythms_dict.get(rref.get("ref", ""))
|
||||||
|
if rhythm is not None:
|
||||||
|
nv = rhythm.findtext("NoteValue", "Quarter")
|
||||||
|
dur_qn = _NOTE_VALUE_QN.get(nv, 0.25)
|
||||||
|
dot_el = rhythm.find("AugmentationDot")
|
||||||
|
if dot_el is not None:
|
||||||
|
try:
|
||||||
|
count = int(dot_el.get("count", 1))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
count = 1
|
||||||
|
dur_qn *= 1.75 if count >= 2 else 1.5
|
||||||
|
tuplet = rhythm.find("PrimaryTuplet")
|
||||||
|
if tuplet is not None:
|
||||||
|
try:
|
||||||
|
num = int(tuplet.get("num", 1))
|
||||||
|
den = int(tuplet.get("den", 1))
|
||||||
|
if num and den:
|
||||||
|
dur_qn *= den / num
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
pass
|
||||||
|
return dur_qn * (60.0 / tempo_bpm)
|
||||||
|
|
||||||
|
|
||||||
|
def _masterbar_ks(mb: ET.Element) -> int | None:
|
||||||
|
"""Key signature (semitones from C, −7…+7) from a MasterBar, or None."""
|
||||||
|
key_el = mb.find("Key")
|
||||||
|
if key_el is None:
|
||||||
|
return None
|
||||||
|
raw = key_el.findtext("AccidentalCount")
|
||||||
|
if raw is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
ks = int(raw.strip())
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
return ks if -7 <= ks <= 7 else None
|
||||||
|
|
||||||
|
|
||||||
|
def _walk_track_beats(
|
||||||
|
root: ET.Element,
|
||||||
|
raw_idx: int,
|
||||||
|
string_pitches: list[int],
|
||||||
|
*,
|
||||||
|
audio_offset: float,
|
||||||
|
force_staff: str | None,
|
||||||
|
) -> list[dict[str, list[list[dict]]]]:
|
||||||
|
"""Walk one raw track bar-by-bar and bucket its beats per staff.
|
||||||
|
|
||||||
|
Returns one entry per masterbar: ``{staff_id: [voice_beats, ...]}`` where
|
||||||
|
each ``voice_beats`` is the ordered beat list of one GP voice. Timing
|
||||||
|
mirrors ``gp2rs_gpx.convert_file`` (bar-indexed tempo map applied at bar
|
||||||
|
starts, ``_beat_secs`` per beat) so notation lines up with the RS XML
|
||||||
|
(modulo the double-dot fix documented on ``_beat_secs``).
|
||||||
|
"""
|
||||||
|
# Local import keeps lib's flat-import convention and avoids a hard cycle
|
||||||
|
# (gp2rs_gpx imports this module lazily from inside convert_file).
|
||||||
|
from gp2rs_gpx import (
|
||||||
|
_build_tempo_map, _gpif_tempo, _note_is_tie, _note_midi, _notes_by_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
masterbars = _children(root, "MasterBars")
|
||||||
|
bars_by_id = {b.get("id"): b for b in _children(root, "Bars")}
|
||||||
|
voices_dict = {v.get("id"): v for v in _children(root, "Voices")}
|
||||||
|
beats_dict = {b.get("id"): b for b in _children(root, "Beats")}
|
||||||
|
rhythms_dict = {r.get("id"): r for r in _children(root, "Rhythms")}
|
||||||
|
# Same duplicate-id-tolerant note lookup convert_file uses.
|
||||||
|
notes_dict = _notes_by_id(root)
|
||||||
|
|
||||||
|
tempo_bpm = _gpif_tempo(root)
|
||||||
|
tempo_iter = iter(_build_tempo_map(root))
|
||||||
|
next_tempo_bar, next_tempo_bpm = next(tempo_iter, (999999, tempo_bpm))
|
||||||
|
cur_tempo = tempo_bpm
|
||||||
|
|
||||||
|
out: list[dict[str, list[list[dict]]]] = []
|
||||||
|
current_time = 0.0
|
||||||
|
|
||||||
|
for mb_idx, mb in enumerate(masterbars):
|
||||||
|
while mb_idx >= next_tempo_bar:
|
||||||
|
cur_tempo = next_tempo_bpm
|
||||||
|
next_tempo_bar, next_tempo_bpm = next(tempo_iter, (999999, cur_tempo))
|
||||||
|
|
||||||
|
time_sig = mb.findtext("Time", "4/4")
|
||||||
|
try:
|
||||||
|
num_b, den_b = [int(x) for x in time_sig.split("/")]
|
||||||
|
except ValueError:
|
||||||
|
num_b, den_b = 4, 4
|
||||||
|
# A malformed-but-parseable signature like "4/0" or "-3/4" would
|
||||||
|
# divide by zero / run time backwards below.
|
||||||
|
if num_b <= 0 or den_b <= 0:
|
||||||
|
log.warning("gp2notation: invalid time signature %r — assuming 4/4", time_sig)
|
||||||
|
num_b, den_b = 4, 4
|
||||||
|
bar_duration = num_b * (4.0 / den_b) * (60.0 / cur_tempo)
|
||||||
|
|
||||||
|
per_staff: dict[str, list[list[dict]]] = {}
|
||||||
|
bar_ids = mb.findtext("Bars", "").split()
|
||||||
|
bid = bar_ids[raw_idx] if raw_idx < len(bar_ids) else "-1"
|
||||||
|
bar = bars_by_id.get(bid) if bid not in ("-1", "") else None
|
||||||
|
if bar is not None:
|
||||||
|
for voice_pos, vid in enumerate(bar.findtext("Voices", "").split()):
|
||||||
|
if vid == "-1":
|
||||||
|
continue
|
||||||
|
voice = voices_dict.get(vid)
|
||||||
|
if voice is None:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# PR #703's voice→staff rule: GP voice position 0 = right
|
||||||
|
# hand (treble), positions ≥ 1 = left hand (bass); a forced
|
||||||
|
# staff (merged/standalone LH track) overrides both.
|
||||||
|
staff = force_staff or ("rh" if voice_pos == 0 else "lh")
|
||||||
|
|
||||||
|
voice_beats: list[dict] = []
|
||||||
|
voice_time = current_time
|
||||||
|
for beat_id in voice.findtext("Beats", "").split():
|
||||||
|
beat_el = beats_dict.get(beat_id)
|
||||||
|
if beat_el is None:
|
||||||
|
continue
|
||||||
|
dur_secs = _beat_secs(beat_el, rhythms_dict, cur_tempo)
|
||||||
|
# Sub-32nd rhythms can't be written in schema v1: drop the
|
||||||
|
# beat (warning) but advance time by its true span so the
|
||||||
|
# rest of the bar stays aligned with the RS-XML walk.
|
||||||
|
rref = beat_el.find("Rhythm")
|
||||||
|
rhythm = rhythms_dict.get(rref.get("ref", "")) if rref is not None else None
|
||||||
|
nv = rhythm.findtext("NoteValue", "Quarter") if rhythm is not None else "Quarter"
|
||||||
|
if nv in _SUB_FLOOR_NOTE_VALUES:
|
||||||
|
log.warning(
|
||||||
|
"gp2notation: dropping %s beat at %.3fs — below the "
|
||||||
|
"schema's 32nd floor (v1 non-feature)",
|
||||||
|
nv, voice_time + audio_offset,
|
||||||
|
)
|
||||||
|
voice_time += dur_secs
|
||||||
|
continue
|
||||||
|
dur, dot, tu = _rhythm_fields(beat_el, rhythms_dict)
|
||||||
|
|
||||||
|
beat_out: dict = {
|
||||||
|
"t": round(voice_time + audio_offset, 3),
|
||||||
|
"dur": dur,
|
||||||
|
}
|
||||||
|
if dot:
|
||||||
|
beat_out["dot"] = dot
|
||||||
|
if tu:
|
||||||
|
beat_out["tu"] = tu
|
||||||
|
|
||||||
|
notes_out: list[dict] = []
|
||||||
|
for nid in beat_el.findtext("Notes", "").strip().split():
|
||||||
|
note_el = notes_dict.get(nid)
|
||||||
|
if note_el is None:
|
||||||
|
continue
|
||||||
|
# GP pitch is absolute for piano-family tracks:
|
||||||
|
# String+Fret resolves via the track's string-template
|
||||||
|
# pitches (string_pitches[idx] + fret, concert pitch)
|
||||||
|
# and Tone+Octave is (octave+1)*12 + step semitone —
|
||||||
|
# both yield a real MIDI number, no tuning offset.
|
||||||
|
midi = _note_midi(note_el, string_pitches)
|
||||||
|
if midi is None or not 0 <= midi <= 127:
|
||||||
|
continue
|
||||||
|
note_out: dict = {"midi": midi}
|
||||||
|
# Unlike the RS-XML walk (which drops tie destinations
|
||||||
|
# and extends the origin's sustain), notation keeps
|
||||||
|
# tied continuations as their own beats — engraving
|
||||||
|
# needs the tied notehead.
|
||||||
|
if _note_is_tie(note_el):
|
||||||
|
note_out["tied"] = True
|
||||||
|
notes_out.append(note_out)
|
||||||
|
|
||||||
|
if notes_out:
|
||||||
|
beat_out["notes"] = notes_out
|
||||||
|
else:
|
||||||
|
# Authored rest, or every note failed pitch extraction.
|
||||||
|
beat_out["rest"] = True
|
||||||
|
voice_beats.append(beat_out)
|
||||||
|
voice_time += dur_secs
|
||||||
|
|
||||||
|
if voice_beats:
|
||||||
|
per_staff.setdefault(staff, []).append(voice_beats)
|
||||||
|
|
||||||
|
out.append(per_staff)
|
||||||
|
current_time += bar_duration
|
||||||
|
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def convert_track_to_notation(
|
||||||
|
root: ET.Element,
|
||||||
|
raw_idx: int,
|
||||||
|
string_pitches: list[int],
|
||||||
|
*,
|
||||||
|
instrument: str = "piano",
|
||||||
|
audio_offset: float = 0.0,
|
||||||
|
track_name: str = "",
|
||||||
|
lh_raw_idx: int | None = None,
|
||||||
|
lh_string_pitches: list[int] | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Convert one GPIF keys/piano track to a notation payload (spec §5.3).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
root: Parsed ``score.gpif`` element (``gp2rs_gpx._load_gpif`` output).
|
||||||
|
raw_idx: Raw track index into MasterBar ``Bars`` id lists (the same
|
||||||
|
index ``convert_file`` derives via ``filtered_to_raw``).
|
||||||
|
string_pitches: The track's string-template tuning (may be empty for
|
||||||
|
Tone+Octave-encoded tracks).
|
||||||
|
instrument: Self-describing instrument name for the payload.
|
||||||
|
audio_offset: Seconds added to every emitted time (audio sync).
|
||||||
|
track_name: Used only to detect a standalone forced-LH track
|
||||||
|
(name ending in "LH" → every voice routes to the lh staff).
|
||||||
|
lh_raw_idx / lh_string_pitches: When a Piano LH/RH pair was merged
|
||||||
|
(``gp2rs_gpx._find_piano_pairs``), the LH partner's raw index and
|
||||||
|
tuning — its beats are walked separately and forced onto ``lh``.
|
||||||
|
|
||||||
|
Returns the validated notation dict (``version``/``instrument``/
|
||||||
|
``staves``/``measures``). Raises ``ValueError`` if the built payload
|
||||||
|
fails ``notation.validate_notation`` (importer bug guard).
|
||||||
|
"""
|
||||||
|
from gp2rs_gpx import _build_tempo_map, _gpif_tempo
|
||||||
|
|
||||||
|
force_staff = "lh" if (track_name and _LH_NAME_RE.search(track_name)) else None
|
||||||
|
walked = _walk_track_beats(
|
||||||
|
root, raw_idx, string_pitches,
|
||||||
|
audio_offset=audio_offset, force_staff=force_staff,
|
||||||
|
)
|
||||||
|
if lh_raw_idx is not None:
|
||||||
|
lh_walked = _walk_track_beats(
|
||||||
|
root, lh_raw_idx, lh_string_pitches or [],
|
||||||
|
audio_offset=audio_offset, force_staff="lh",
|
||||||
|
)
|
||||||
|
# Merge the LH partner's voices into each measure's lh staff, after
|
||||||
|
# any voices the main track already routed there.
|
||||||
|
for main_bar, lh_bar in zip(walked, lh_walked):
|
||||||
|
for staff_id, voices in lh_bar.items():
|
||||||
|
main_bar.setdefault(staff_id, []).extend(voices)
|
||||||
|
|
||||||
|
masterbars = _children(root, "MasterBars")
|
||||||
|
tempo_bpm = _gpif_tempo(root)
|
||||||
|
tempo_iter = iter(_build_tempo_map(root))
|
||||||
|
next_tempo_bar, next_tempo_bpm = next(tempo_iter, (999999, tempo_bpm))
|
||||||
|
cur_tempo = tempo_bpm
|
||||||
|
|
||||||
|
measures: list[dict] = []
|
||||||
|
used_staves: set[str] = set()
|
||||||
|
current_time = 0.0
|
||||||
|
last_ts: tuple[int, int] | None = None
|
||||||
|
last_tempo: float | None = None
|
||||||
|
last_ks: int | None = None
|
||||||
|
|
||||||
|
for mb_idx, mb in enumerate(masterbars):
|
||||||
|
while mb_idx >= next_tempo_bar:
|
||||||
|
cur_tempo = next_tempo_bpm
|
||||||
|
next_tempo_bar, next_tempo_bpm = next(tempo_iter, (999999, cur_tempo))
|
||||||
|
|
||||||
|
time_sig = mb.findtext("Time", "4/4")
|
||||||
|
try:
|
||||||
|
num_b, den_b = [int(x) for x in time_sig.split("/")]
|
||||||
|
except ValueError:
|
||||||
|
num_b, den_b = 4, 4
|
||||||
|
if num_b <= 0 or den_b <= 0:
|
||||||
|
log.warning("gp2notation: invalid time signature %r — assuming 4/4", time_sig)
|
||||||
|
num_b, den_b = 4, 4
|
||||||
|
|
||||||
|
measure: dict = {
|
||||||
|
"idx": mb_idx + 1,
|
||||||
|
"t": round(current_time + audio_offset, 3),
|
||||||
|
}
|
||||||
|
if (num_b, den_b) != last_ts:
|
||||||
|
measure["ts"] = [num_b, den_b]
|
||||||
|
groups = beat_groups_for(num_b, den_b)
|
||||||
|
if groups:
|
||||||
|
measure["beat_groups"] = groups
|
||||||
|
last_ts = (num_b, den_b)
|
||||||
|
if cur_tempo != last_tempo:
|
||||||
|
measure["tempo"] = cur_tempo
|
||||||
|
last_tempo = cur_tempo
|
||||||
|
ks = _masterbar_ks(mb)
|
||||||
|
if ks is not None and ks != last_ks:
|
||||||
|
measure["ks"] = ks
|
||||||
|
last_ks = ks
|
||||||
|
|
||||||
|
staves_payload: dict[str, dict] = {}
|
||||||
|
for staff_id in ("rh", "lh"): # stable staff order
|
||||||
|
voices = (walked[mb_idx] if mb_idx < len(walked) else {}).get(staff_id)
|
||||||
|
if not voices:
|
||||||
|
continue
|
||||||
|
used_staves.add(staff_id)
|
||||||
|
staves_payload[staff_id] = {
|
||||||
|
"voices": [
|
||||||
|
{"v": v_num, "beats": beats}
|
||||||
|
for v_num, beats in enumerate(voices, start=1)
|
||||||
|
],
|
||||||
|
}
|
||||||
|
measure["staves"] = staves_payload
|
||||||
|
measures.append(measure)
|
||||||
|
current_time += num_b * (4.0 / den_b) * (60.0 / cur_tempo)
|
||||||
|
|
||||||
|
staves = [_STAFF_DEFS[s] for s in ("rh", "lh") if s in used_staves]
|
||||||
|
if not staves:
|
||||||
|
staves = [_STAFF_DEFS["rh"]] # empty track — still a valid grand-staff stub
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"version": notation_mod.SCHEMA_VERSION,
|
||||||
|
"instrument": instrument,
|
||||||
|
"staves": staves,
|
||||||
|
"measures": measures,
|
||||||
|
}
|
||||||
|
ok, reason = notation_mod.validate_notation(payload)
|
||||||
|
if not ok:
|
||||||
|
raise ValueError(f"gp2notation built an invalid payload: {reason}")
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
# ── Sidecar + manifest wiring ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def notation_sidecar_path(xml_path: str | Path) -> Path:
|
||||||
|
"""The notation sidecar written next to a converted arrangement XML.
|
||||||
|
|
||||||
|
``Foo_Keys.xml`` → ``Foo_Keys.notation.json``. Arrangement ids don't
|
||||||
|
exist yet at convert time (they're assigned when the sloppak manifest is
|
||||||
|
assembled), so the sidecar pairs with the XML by filename stem; the
|
||||||
|
assembly step renames it to ``notation_<id>.json`` via
|
||||||
|
``attach_notation_to_sloppak``.
|
||||||
|
"""
|
||||||
|
p = Path(xml_path)
|
||||||
|
return p.with_name(p.stem + ".notation.json")
|
||||||
|
|
||||||
|
|
||||||
|
def write_notation_sidecar(xml_path: str | Path, payload: dict) -> Path:
|
||||||
|
"""Validate and write the notation sidecar for a converted XML."""
|
||||||
|
ok, reason = notation_mod.validate_notation(payload)
|
||||||
|
if not ok:
|
||||||
|
raise ValueError(f"refusing to write invalid notation sidecar: {reason}")
|
||||||
|
side = notation_sidecar_path(xml_path)
|
||||||
|
side.write_text(json.dumps(payload, separators=(",", ":")), encoding="utf-8")
|
||||||
|
return side
|
||||||
|
|
||||||
|
|
||||||
|
def attach_notation_to_sloppak(sloppak_dir: str | Path, arr_id: str, payload: dict) -> Path:
|
||||||
|
"""Write ``notation_<arr_id>.json`` into a directory-form sloppak and add
|
||||||
|
the ``notation:`` sub-key to that arrangement's manifest entry.
|
||||||
|
|
||||||
|
Raises ``ValueError`` on an invalid payload, an unsafe/unknown
|
||||||
|
arrangement id, or a manifest without a matching arrangement entry.
|
||||||
|
Note: the manifest is round-tripped through PyYAML (``safe_load`` +
|
||||||
|
``safe_dump(sort_keys=False)``) — key order is preserved but comments
|
||||||
|
and custom formatting are lost.
|
||||||
|
"""
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
ok, reason = notation_mod.validate_notation(payload)
|
||||||
|
if not ok:
|
||||||
|
raise ValueError(f"invalid notation payload: {reason}")
|
||||||
|
if not arr_id or not re.fullmatch(r"[A-Za-z0-9_-]+", arr_id):
|
||||||
|
raise ValueError(f"unsafe arrangement id for notation filename: {arr_id!r}")
|
||||||
|
|
||||||
|
pak = Path(sloppak_dir)
|
||||||
|
manifest_path = pak / "manifest.yaml"
|
||||||
|
manifest = yaml.safe_load(manifest_path.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(manifest, dict):
|
||||||
|
raise ValueError(f"{manifest_path} is not a mapping")
|
||||||
|
|
||||||
|
entry = next(
|
||||||
|
(e for e in (manifest.get("arrangements") or [])
|
||||||
|
if isinstance(e, dict) and e.get("id") == arr_id),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if entry is None:
|
||||||
|
raise ValueError(f"no arrangement with id {arr_id!r} in {manifest_path}")
|
||||||
|
|
||||||
|
filename = f"notation_{arr_id}.json"
|
||||||
|
(pak / filename).write_text(
|
||||||
|
json.dumps(payload, separators=(",", ":")), encoding="utf-8"
|
||||||
|
)
|
||||||
|
entry["notation"] = filename
|
||||||
|
manifest_path.write_text(
|
||||||
|
yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return pak / filename
|
||||||
+1865
File diff suppressed because it is too large
Load Diff
+2141
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,453 @@
|
|||||||
|
"""
|
||||||
|
lib/gp8_audio_sync.py — Extract embedded audio and sync data from GP8 (.gp) files.
|
||||||
|
|
||||||
|
Guitar Pro 8 can embed a backing track (OGG audio) into a .gp file alongside
|
||||||
|
sync points that map bar positions to exact audio timestamps. This module
|
||||||
|
extracts both, giving Slopsmith:
|
||||||
|
|
||||||
|
1. A real backing track audio file (OGG) — no MIDI synthesis needed
|
||||||
|
2. A precise audio_offset (seconds) from the FramePadding value
|
||||||
|
3. A bar-indexed tempo map derived from sync point ModifiedTempo values,
|
||||||
|
which is more accurate than the tab's authored tempo automations for
|
||||||
|
files that have been manually synced to audio
|
||||||
|
|
||||||
|
Public API:
|
||||||
|
has_embedded_audio(gp_path) -> bool
|
||||||
|
extract_audio(gp_path, output_dir) -> str | None (path to .ogg file)
|
||||||
|
extract_sync(gp_path) -> GpSyncData | None
|
||||||
|
|
||||||
|
GpSyncData fields:
|
||||||
|
audio_offset float seconds to add to all RS note times (negative
|
||||||
|
means audio starts before bar 1)
|
||||||
|
sync_points list[SyncPoint] bar-indexed audio timestamps
|
||||||
|
audio_asset_id str filename stem of the OGG in Content/Assets/
|
||||||
|
|
||||||
|
SyncPoint fields:
|
||||||
|
bar int 0-based bar index in the score
|
||||||
|
time_secs float position in the audio file (seconds from start)
|
||||||
|
modified_tempo float actual BPM at this point in the recording
|
||||||
|
original_tempo float tab's authored BPM at this bar
|
||||||
|
|
||||||
|
Usage in convert_file():
|
||||||
|
sync = extract_sync(gp_path)
|
||||||
|
if sync:
|
||||||
|
audio_path = extract_audio(gp_path, output_dir)
|
||||||
|
xml = convert_file(..., audio_offset=sync.audio_offset)
|
||||||
|
else:
|
||||||
|
# fall back to gp2midi for GP3-5, or MIDI-less for GPX without audio
|
||||||
|
pass
|
||||||
|
"""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import xml.etree.ElementTree as ET
|
||||||
|
import zipfile
|
||||||
|
import io
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
_log = logging.getLogger("slopsmith.lib.gp8_audio_sync")
|
||||||
|
|
||||||
|
# GP8 embeds the backing track under Content/Assets/ as OGG *or* one of
|
||||||
|
# several other formats (MP3 is common — e.g. tracks rendered straight
|
||||||
|
# from a DAW). Earlier code only matched .ogg, so an MP3-backed file would
|
||||||
|
# report has_embedded_audio() True (via meta.json) yet extract nothing.
|
||||||
|
# Match any of these and transcode to OGG on extraction when needed.
|
||||||
|
_AUDIO_ASSET_EXTS = ('.ogg', '.mp3', '.m4a', '.aac', '.wav', '.flac', '.opus', '.wma')
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_gpif(data: bytes):
|
||||||
|
"""Parse GPIF XML bytes with defusedxml when available, stdlib otherwise.
|
||||||
|
|
||||||
|
Centralised so every caller hardens parsing the same way (no divergent
|
||||||
|
inline try/except blocks).
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import defusedxml.ElementTree as _safe_ET
|
||||||
|
return _safe_ET.fromstring(data)
|
||||||
|
except ImportError:
|
||||||
|
_log.warning(
|
||||||
|
'gp8_audio_sync: defusedxml not installed; parsing with stdlib '
|
||||||
|
'xml.etree (install defusedxml for hardened parsing)'
|
||||||
|
)
|
||||||
|
return ET.fromstring(data)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_audio_asset(zf, root=None) -> tuple[str, str | None]:
|
||||||
|
"""Resolve the embedded backing-track audio asset inside a .gp ZIP.
|
||||||
|
|
||||||
|
Matches ``BackingTrack/AssetId`` against the audio files under
|
||||||
|
``Content/Assets/`` (OGG, MP3, M4A, …) and falls back to the first
|
||||||
|
audio asset when the declared id is missing or unmatched. Returns
|
||||||
|
``(asset_stem, audio_zip_path)``, or ``('', None)`` when the archive
|
||||||
|
has no audio asset. Shared by ``extract_sync`` and ``extract_audio``
|
||||||
|
so the matching logic can't drift between them.
|
||||||
|
"""
|
||||||
|
audio_files = [
|
||||||
|
n for n in zf.namelist()
|
||||||
|
if n.startswith('Content/Assets/')
|
||||||
|
and n.lower().endswith(_AUDIO_ASSET_EXTS)
|
||||||
|
]
|
||||||
|
if not audio_files:
|
||||||
|
return '', None
|
||||||
|
|
||||||
|
# When the SAME backing track is present in several formats (same
|
||||||
|
# AssetId stem), prefer the OGG: it's copied out losslessly while any
|
||||||
|
# other format must be transcoded, so this preserves both quality and
|
||||||
|
# the pre-MP3-support behaviour. This only applies to genuine same-stem
|
||||||
|
# duplicates — the unmatched fallback below keeps ZIP order so an
|
||||||
|
# unrelated later OGG can't displace the archive's first asset.
|
||||||
|
def _prefer_ogg(candidates):
|
||||||
|
return next(
|
||||||
|
(n for n in candidates if n.lower().endswith('.ogg')),
|
||||||
|
candidates[0],
|
||||||
|
)
|
||||||
|
|
||||||
|
declared = ''
|
||||||
|
if root is None:
|
||||||
|
try:
|
||||||
|
root = _parse_gpif(zf.read('Content/score.gpif'))
|
||||||
|
except Exception:
|
||||||
|
root = None
|
||||||
|
if root is not None:
|
||||||
|
bt = root.find('BackingTrack')
|
||||||
|
if bt is not None:
|
||||||
|
aid = bt.find('AssetId')
|
||||||
|
declared = (aid.text or '').strip() if aid is not None else ''
|
||||||
|
|
||||||
|
if declared:
|
||||||
|
matched = [n for n in audio_files if Path(n).stem == declared]
|
||||||
|
if matched:
|
||||||
|
return declared, _prefer_ogg(matched)
|
||||||
|
_log.warning(
|
||||||
|
'gp8_audio_sync: declared AssetId %r not found; falling back to first audio asset',
|
||||||
|
declared,
|
||||||
|
)
|
||||||
|
# Fallback: the archive's first audio asset (ZIP order), unchanged from
|
||||||
|
# the original OGG-only behaviour.
|
||||||
|
return Path(audio_files[0]).stem, audio_files[0]
|
||||||
|
|
||||||
|
# GP8 uses 44100 Hz internally for FrameOffset values regardless of the
|
||||||
|
# OGG file's own sample rate. The embedded OGG is typically 48000 Hz
|
||||||
|
# (Rocksmith's preferred rate) and should be passed through as-is —
|
||||||
|
# do NOT resample it. The 44100 constant is only used here to convert
|
||||||
|
# FrameOffset integers to seconds for timing math; it never touches audio.
|
||||||
|
# Verified: 44100 gives <10ms sync error; 48000 gives ~530ms error.
|
||||||
|
_GP8_FRAME_RATE = 44100
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SyncPoint:
|
||||||
|
"""One GP8 sync point: a bar-to-audio-timestamp mapping."""
|
||||||
|
bar: int
|
||||||
|
time_secs: float # position in audio file (seconds from file start)
|
||||||
|
modified_tempo: float # actual recording BPM at this bar
|
||||||
|
original_tempo: float # tab's authored BPM at this bar
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class GpSyncData:
|
||||||
|
"""Sync data extracted from a GP8 file with an embedded backing track."""
|
||||||
|
audio_offset: float # seconds: negative = audio starts before bar 1
|
||||||
|
audio_asset_id: str # OGG filename stem in Content/Assets/
|
||||||
|
sync_points: list[SyncPoint] = field(default_factory=list)
|
||||||
|
|
||||||
|
def tempo_at_bar(self, bar: int) -> float:
|
||||||
|
"""Return the ModifiedTempo for the sync segment containing `bar`.
|
||||||
|
|
||||||
|
Uses the last sync point whose bar index is <= the requested bar,
|
||||||
|
which matches GP8's behaviour of holding each tempo until the next
|
||||||
|
sync point.
|
||||||
|
"""
|
||||||
|
result = self.sync_points[0].modified_tempo if self.sync_points else 120.0
|
||||||
|
for sp in self.sync_points:
|
||||||
|
if sp.bar <= bar:
|
||||||
|
result = sp.modified_tempo
|
||||||
|
else:
|
||||||
|
break
|
||||||
|
return result
|
||||||
|
|
||||||
|
def time_at_bar(self, bar: int, beats_per_bar: float = 4.0) -> float:
|
||||||
|
"""Interpolate the audio timestamp (seconds) for any bar index.
|
||||||
|
|
||||||
|
For bars between sync points, interpolates using the ModifiedTempo
|
||||||
|
of the preceding sync point — matching GP8's linear interpolation.
|
||||||
|
For bars before the first sync point, extrapolates backward.
|
||||||
|
"""
|
||||||
|
if not self.sync_points:
|
||||||
|
return 0.0
|
||||||
|
|
||||||
|
# Find the surrounding sync points
|
||||||
|
before = self.sync_points[0]
|
||||||
|
after = None
|
||||||
|
for sp in self.sync_points:
|
||||||
|
if sp.bar <= bar:
|
||||||
|
before = sp
|
||||||
|
else:
|
||||||
|
after = sp
|
||||||
|
break
|
||||||
|
|
||||||
|
# Between two sync points, interpolate linearly by bar index between
|
||||||
|
# the two known audio timestamps. This is exact regardless of the
|
||||||
|
# time signature (no beats_per_bar assumption) and matches GP8's
|
||||||
|
# straight-line interpolation between sync points.
|
||||||
|
if after is not None and after.bar > before.bar:
|
||||||
|
frac = (bar - before.bar) / (after.bar - before.bar)
|
||||||
|
return before.time_secs + frac * (after.time_secs - before.time_secs)
|
||||||
|
|
||||||
|
# Past the last sync point (or before the first): no second anchor, so
|
||||||
|
# extrapolate from `before` using its ModifiedTempo. beats_per_bar
|
||||||
|
# defaults to 4.0 — callers should pass the actual time-signature
|
||||||
|
# numerator for correct non-4/4 extrapolation here.
|
||||||
|
bars_since = bar - before.bar
|
||||||
|
seconds_per_bar = beats_per_bar * 60.0 / before.modified_tempo
|
||||||
|
return before.time_secs + bars_since * seconds_per_bar
|
||||||
|
|
||||||
|
|
||||||
|
def _open_gp_zip(gp_path: str):
|
||||||
|
"""Open a .gp ZIP container and return (raw_bytes, ZipFile)."""
|
||||||
|
with open(gp_path, 'rb') as fh:
|
||||||
|
raw = fh.read()
|
||||||
|
if raw[:2] != b'PK':
|
||||||
|
raise ValueError(f"{gp_path!r} is not a GP7/GP8 ZIP file (magic: {raw[:4]!r})")
|
||||||
|
return raw, zipfile.ZipFile(io.BytesIO(raw))
|
||||||
|
|
||||||
|
|
||||||
|
def has_embedded_audio(gp_path: str) -> bool:
|
||||||
|
"""Return True if the .gp file has an embedded backing track.
|
||||||
|
|
||||||
|
This is the canonical gate: it returns False for anything that isn't a
|
||||||
|
GP7/GP8 ZIP container with embedded audio (including GP3/4/5 files and
|
||||||
|
malformed inputs). Callers should check this first — `extract_sync` and
|
||||||
|
`extract_audio` return None for both "no embedded audio" and "not a
|
||||||
|
GP7/8 container", so they don't distinguish the two on their own.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
raw, zf = _open_gp_zip(gp_path)
|
||||||
|
with zf:
|
||||||
|
names = zf.namelist()
|
||||||
|
# meta.json has {"hasAudio": true} when audio is embedded
|
||||||
|
if 'meta.json' in names:
|
||||||
|
import json
|
||||||
|
meta = json.loads(zf.read('meta.json'))
|
||||||
|
if meta.get('hasAudio'):
|
||||||
|
return True
|
||||||
|
# Also check directly for any embedded audio asset (OGG, MP3, …)
|
||||||
|
return any(
|
||||||
|
n.startswith('Content/Assets/')
|
||||||
|
and n.lower().endswith(_AUDIO_ASSET_EXTS)
|
||||||
|
for n in names
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def extract_sync(gp_path: str) -> GpSyncData | None:
|
||||||
|
"""Extract sync data from a GP8 file.
|
||||||
|
|
||||||
|
Returns None if the file has no embedded audio or no sync points.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
raw, zf = _open_gp_zip(gp_path)
|
||||||
|
with zf:
|
||||||
|
if 'Content/score.gpif' not in zf.namelist():
|
||||||
|
return None
|
||||||
|
|
||||||
|
root = _parse_gpif(zf.read('Content/score.gpif'))
|
||||||
|
|
||||||
|
# Find the BackingTrack element for FramePadding and asset ID
|
||||||
|
bt = root.find('BackingTrack')
|
||||||
|
if bt is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# FramePadding: negative = audio starts before bar 1
|
||||||
|
frame_padding = 0
|
||||||
|
fp_el = bt.find('FramePadding')
|
||||||
|
if fp_el is not None and fp_el.text:
|
||||||
|
try:
|
||||||
|
frame_padding = int(fp_el.text.strip())
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
audio_offset = frame_padding / _GP8_FRAME_RATE
|
||||||
|
|
||||||
|
# Resolve the audio asset (AssetId match, first-asset fallback).
|
||||||
|
asset_name, ogg_match = _resolve_audio_asset(zf, root)
|
||||||
|
ogg_files = [ogg_match] if ogg_match else []
|
||||||
|
|
||||||
|
if not asset_name and not ogg_files:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Extract SyncPoint automations from MasterTrack
|
||||||
|
mt = root.find('MasterTrack')
|
||||||
|
sync_points: list[SyncPoint] = []
|
||||||
|
|
||||||
|
if mt is not None:
|
||||||
|
for auto in mt.findall('.//Automations/*'):
|
||||||
|
if auto.findtext('Type') != 'SyncPoint':
|
||||||
|
continue
|
||||||
|
val = auto.find('Value')
|
||||||
|
if val is None:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
bar = int(val.findtext('BarIndex') or 0)
|
||||||
|
# FrameOffset (inside Value) is the audio frame for this
|
||||||
|
# sync point. Default to 0 when absent — do NOT fall back
|
||||||
|
# to the automation's `Position`, which is an in-bar
|
||||||
|
# musical position (1/16384-note units), not a frame
|
||||||
|
# count, and would yield a nonsensical time_secs.
|
||||||
|
frame_offset = 0
|
||||||
|
fo_el = val.find('FrameOffset')
|
||||||
|
if fo_el is not None and fo_el.text:
|
||||||
|
frame_offset = int(fo_el.text.strip())
|
||||||
|
modified_tempo = float(val.findtext('ModifiedTempo') or 120)
|
||||||
|
original_tempo = float(val.findtext('OriginalTempo') or 120)
|
||||||
|
time_secs = frame_offset / _GP8_FRAME_RATE
|
||||||
|
sync_points.append(SyncPoint(
|
||||||
|
bar=bar,
|
||||||
|
time_secs=time_secs,
|
||||||
|
modified_tempo=modified_tempo,
|
||||||
|
original_tempo=original_tempo,
|
||||||
|
))
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
continue
|
||||||
|
|
||||||
|
sync_points.sort(key=lambda sp: sp.bar)
|
||||||
|
|
||||||
|
if not sync_points and not ogg_files:
|
||||||
|
return None
|
||||||
|
|
||||||
|
return GpSyncData(
|
||||||
|
audio_offset=audio_offset,
|
||||||
|
audio_asset_id=asset_name,
|
||||||
|
sync_points=sync_points,
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
_log.warning("gp8_audio_sync: failed to extract sync from %r: %s", gp_path, e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def extract_audio(gp_path: str, output_dir: str) -> str | None:
|
||||||
|
"""Extract the embedded backing-track audio to output_dir as OGG.
|
||||||
|
|
||||||
|
Returns the path to the extracted `.ogg` file, or None if no audio is
|
||||||
|
found (or a non-OGG asset couldn't be transcoded). The filename is
|
||||||
|
derived from the GP file stem with an `_audio` suffix:
|
||||||
|
e.g. my_song.gp -> my_song_audio.ogg.
|
||||||
|
|
||||||
|
GP8 embeds the backing track as OGG or another format (MP3 is common).
|
||||||
|
An OGG asset is copied out verbatim; any other format is transcoded to
|
||||||
|
OGG via ffmpeg so the editor/web pipeline — which expects OGG stems —
|
||||||
|
can use it unchanged.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
raw, zf = _open_gp_zip(gp_path)
|
||||||
|
with zf:
|
||||||
|
# Resolve the asset via the same AssetId logic extract_sync uses.
|
||||||
|
_asset_name, chosen = _resolve_audio_asset(zf)
|
||||||
|
if not chosen:
|
||||||
|
return None
|
||||||
|
|
||||||
|
out = Path(output_dir)
|
||||||
|
out.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Name the output after the GP file, not the UUID asset name.
|
||||||
|
stem = Path(gp_path).stem
|
||||||
|
out_path = out / f"{stem}_audio.ogg"
|
||||||
|
src_ext = Path(chosen).suffix.lower()
|
||||||
|
|
||||||
|
# Already OGG — copy the bytes out verbatim (lossless, fast).
|
||||||
|
if src_ext == '.ogg':
|
||||||
|
out_path.write_bytes(zf.read(chosen))
|
||||||
|
_log.info("gp8_audio_sync: extracted audio to %s", out_path)
|
||||||
|
return str(out_path)
|
||||||
|
|
||||||
|
# Non-OGG (e.g. MP3): stage + transcode inside a private temp dir,
|
||||||
|
# then atomically move the result into place. A private work dir
|
||||||
|
# keeps the function idempotent — a failed run can't clobber or
|
||||||
|
# delete a good *_audio.ogg from a previous successful run, and the
|
||||||
|
# staged source can't collide with a file the caller already put in
|
||||||
|
# output_dir.
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import tempfile
|
||||||
|
work = Path(tempfile.mkdtemp(prefix="gp8_audio_", dir=str(out)))
|
||||||
|
try:
|
||||||
|
src_path = work / f"src{src_ext or '.bin'}"
|
||||||
|
src_path.write_bytes(zf.read(chosen))
|
||||||
|
try:
|
||||||
|
from audio import _ffmpeg_cmd, _ffmpeg_wav_to_ogg
|
||||||
|
ffmpeg = _ffmpeg_cmd()
|
||||||
|
except Exception:
|
||||||
|
ffmpeg = None
|
||||||
|
if not ffmpeg:
|
||||||
|
_log.warning(
|
||||||
|
"gp8_audio_sync: embedded audio is %s but ffmpeg is "
|
||||||
|
"unavailable to transcode it to OGG", src_ext,
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
# `_ffmpeg_wav_to_ogg` runs `ffmpeg -i <in> ... <out.ogg>`; the
|
||||||
|
# input may be any format ffmpeg can decode despite the name.
|
||||||
|
tmp_ogg = work / "out.ogg"
|
||||||
|
r = _ffmpeg_wav_to_ogg(ffmpeg, src_path, tmp_ogg)
|
||||||
|
if (r.returncode == 0 and tmp_ogg.exists()
|
||||||
|
and tmp_ogg.stat().st_size >= 100):
|
||||||
|
# Atomic move into place — out_path is only ever touched on
|
||||||
|
# success (work dir is under output_dir, so same filesystem).
|
||||||
|
os.replace(tmp_ogg, out_path)
|
||||||
|
_log.info(
|
||||||
|
"gp8_audio_sync: transcoded embedded %s audio to %s",
|
||||||
|
src_ext, out_path,
|
||||||
|
)
|
||||||
|
return str(out_path)
|
||||||
|
_log.warning(
|
||||||
|
"gp8_audio_sync: ffmpeg failed to transcode embedded %s "
|
||||||
|
"audio to OGG (rc=%s)", src_ext, r.returncode,
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
finally:
|
||||||
|
shutil.rmtree(work, ignore_errors=True)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
_log.warning("gp8_audio_sync: failed to extract audio from %r: %s", gp_path, e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def build_tempo_map_from_sync(sync: GpSyncData) -> list[tuple[int, float]]:
|
||||||
|
"""
|
||||||
|
Build a bar-indexed tempo map from GP8 sync points.
|
||||||
|
|
||||||
|
Returns list of (bar_index, bpm) pairs sorted by bar_index, in the same
|
||||||
|
format as gp2rs_gpx._build_tempo_map(). This can be passed directly to
|
||||||
|
the bar iteration loop in convert_file() for accurate timing.
|
||||||
|
|
||||||
|
For GP8 files with audio sync, ModifiedTempo values are more accurate
|
||||||
|
than the tab's authored Tempo automations — they reflect the actual
|
||||||
|
recording tempo rather than the transcriber's approximation.
|
||||||
|
"""
|
||||||
|
if not sync.sync_points:
|
||||||
|
return [(0, 120.0)]
|
||||||
|
return [(sp.bar, sp.modified_tempo) for sp in sync.sync_points]
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
import sys
|
||||||
|
import logging as _logging
|
||||||
|
_logging.basicConfig(level=_logging.INFO)
|
||||||
|
path = sys.argv[1] if len(sys.argv) > 1 else None
|
||||||
|
if not path:
|
||||||
|
_log.error('Usage: python gp8_audio_sync.py <file.gp>')
|
||||||
|
sys.exit(1)
|
||||||
|
_log.info('has_embedded_audio: %s', has_embedded_audio(path))
|
||||||
|
sync = extract_sync(path)
|
||||||
|
if sync:
|
||||||
|
_log.info('audio_offset: %.4fs', sync.audio_offset)
|
||||||
|
_log.info('audio_asset: %s', sync.audio_asset_id)
|
||||||
|
_log.info('sync_points: %d', len(sync.sync_points))
|
||||||
|
for sp in sync.sync_points:
|
||||||
|
_log.info(
|
||||||
|
'bar=%-4d t=%.3fs modified_bpm=%.3f original_bpm=%.1f',
|
||||||
|
sp.bar, sp.time_secs, sp.modified_tempo, sp.original_tempo,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
_log.info('No sync data found')
|
||||||
+1077
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,175 @@
|
|||||||
|
"""Logging configuration for Slopsmith.
|
||||||
|
|
||||||
|
Call ``configure_logging()`` once at server startup, before any slopsmith
|
||||||
|
module imports that might emit log records.
|
||||||
|
|
||||||
|
Environment variables:
|
||||||
|
LOG_LEVEL — severity threshold for the ``slopsmith.*`` logger tree
|
||||||
|
(default: INFO). Also accepted: DEBUG, WARNING, ERROR.
|
||||||
|
LOG_FORMAT — "json" for structured output (Loki, ELK, Promtail);
|
||||||
|
"text" (default) for human-readable coloured console output.
|
||||||
|
LOG_FILE — optional path; when set, a RotatingFileHandler is added
|
||||||
|
alongside the console handler (max 10 MB, 5 backups).
|
||||||
|
The parent directory is created automatically if it does not
|
||||||
|
exist. If the file cannot be opened, a warning is printed
|
||||||
|
and the server continues with console-only logging.
|
||||||
|
Useful for persistent NAS deployments.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import logging.handlers
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
|
||||||
|
|
||||||
|
def _add_correlation_id(
|
||||||
|
logger: object, method_name: str, event_dict: dict
|
||||||
|
) -> dict:
|
||||||
|
"""Inject the current request correlation ID into the event dict."""
|
||||||
|
try:
|
||||||
|
from asgi_correlation_id import correlation_id
|
||||||
|
|
||||||
|
cid = correlation_id.get(None)
|
||||||
|
if cid:
|
||||||
|
event_dict["request_id"] = cid
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
return event_dict
|
||||||
|
|
||||||
|
|
||||||
|
def configure_logging() -> None:
|
||||||
|
"""Wire up the slopsmith logger hierarchy.
|
||||||
|
|
||||||
|
Safe to call multiple times; always reflects the current LOG_LEVEL,
|
||||||
|
LOG_FORMAT, and LOG_FILE environment variables.
|
||||||
|
"""
|
||||||
|
raw_level = os.environ.get("LOG_LEVEL", "INFO").upper()
|
||||||
|
level = getattr(logging, raw_level, None)
|
||||||
|
if not isinstance(level, int):
|
||||||
|
sys.stderr.write(
|
||||||
|
f"[slopsmith] WARNING: unrecognised LOG_LEVEL={raw_level!r};"
|
||||||
|
" falling back to INFO.\n"
|
||||||
|
)
|
||||||
|
level = logging.INFO
|
||||||
|
|
||||||
|
raw_fmt = os.environ.get("LOG_FORMAT", "text").lower()
|
||||||
|
if raw_fmt not in ("json", "text"):
|
||||||
|
sys.stderr.write(
|
||||||
|
f"[slopsmith] WARNING: unrecognised LOG_FORMAT={raw_fmt!r};"
|
||||||
|
" falling back to 'text'.\n"
|
||||||
|
)
|
||||||
|
raw_fmt = "text"
|
||||||
|
fmt = raw_fmt
|
||||||
|
|
||||||
|
log_file = os.environ.get("LOG_FILE", "").strip()
|
||||||
|
|
||||||
|
# Console renderer: coloured when text mode, JSON otherwise.
|
||||||
|
console_renderer = (
|
||||||
|
structlog.processors.JSONRenderer()
|
||||||
|
if fmt == "json"
|
||||||
|
else structlog.dev.ConsoleRenderer()
|
||||||
|
)
|
||||||
|
# File renderer: always plain (no ANSI escape sequences) so rotated log
|
||||||
|
# files are human-readable without a terminal. JSON mode reuses the same
|
||||||
|
# renderer because JSON output is already colour-free.
|
||||||
|
file_renderer = (
|
||||||
|
structlog.processors.JSONRenderer()
|
||||||
|
if fmt == "json"
|
||||||
|
else structlog.dev.ConsoleRenderer(colors=False)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Applied to all records — both structlog-native and stdlib (foreign) calls.
|
||||||
|
# Stdlib logging handles %-style format strings itself, so no
|
||||||
|
# PositionalArgumentsFormatter is needed here.
|
||||||
|
pre_chain: list = [
|
||||||
|
structlog.contextvars.merge_contextvars,
|
||||||
|
_add_correlation_id,
|
||||||
|
structlog.stdlib.add_log_level,
|
||||||
|
structlog.stdlib.add_logger_name,
|
||||||
|
structlog.processors.TimeStamper(fmt="iso"),
|
||||||
|
structlog.processors.StackInfoRenderer(),
|
||||||
|
]
|
||||||
|
|
||||||
|
structlog.configure(
|
||||||
|
processors=pre_chain + [structlog.stdlib.ProcessorFormatter.wrap_for_formatter],
|
||||||
|
logger_factory=structlog.stdlib.LoggerFactory(),
|
||||||
|
wrapper_class=structlog.stdlib.BoundLogger,
|
||||||
|
# Keep False so that every reconfigure() call takes effect immediately
|
||||||
|
# for any code that holds a structlog.get_logger() proxy. The small
|
||||||
|
# per-call overhead is acceptable given that logging is not on the hot
|
||||||
|
# path.
|
||||||
|
cache_logger_on_first_use=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _make_formatter(renderer: object) -> structlog.stdlib.ProcessorFormatter:
|
||||||
|
return structlog.stdlib.ProcessorFormatter(
|
||||||
|
processors=[
|
||||||
|
structlog.stdlib.ProcessorFormatter.remove_processors_meta,
|
||||||
|
# Format exc_info tuples to strings before the renderer so that
|
||||||
|
# JSONRenderer never encounters a non-serializable traceback object.
|
||||||
|
structlog.processors.ExceptionRenderer(),
|
||||||
|
renderer,
|
||||||
|
],
|
||||||
|
foreign_pre_chain=pre_chain,
|
||||||
|
)
|
||||||
|
|
||||||
|
console_formatter = _make_formatter(console_renderer)
|
||||||
|
console = logging.StreamHandler(sys.stdout)
|
||||||
|
console.setFormatter(console_formatter)
|
||||||
|
handlers: list[logging.Handler] = [console]
|
||||||
|
|
||||||
|
if log_file:
|
||||||
|
file_formatter = _make_formatter(file_renderer)
|
||||||
|
try:
|
||||||
|
Path(log_file).parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
fh = logging.handlers.RotatingFileHandler(
|
||||||
|
log_file,
|
||||||
|
maxBytes=10 * 1024 * 1024, # 10 MB
|
||||||
|
backupCount=5,
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
fh.setFormatter(file_formatter)
|
||||||
|
handlers.append(fh)
|
||||||
|
except OSError as exc:
|
||||||
|
sys.stderr.write(
|
||||||
|
f"[slopsmith] WARNING: could not open LOG_FILE={log_file!r}: {exc}"
|
||||||
|
" — continuing with console-only logging.\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
_uvicorn_names = ("uvicorn", "uvicorn.error", "uvicorn.access")
|
||||||
|
all_loggers = [logging.getLogger("slopsmith")] + [
|
||||||
|
logging.getLogger(n) for n in _uvicorn_names
|
||||||
|
]
|
||||||
|
|
||||||
|
# Collect all unique old handlers across every logger *before* any close so
|
||||||
|
# that a shared handler (slopsmith and uvicorn* were intentionally given the
|
||||||
|
# same objects) isn't closed while still attached to another logger tree.
|
||||||
|
old_handlers: set[logging.Handler] = set()
|
||||||
|
for lg in all_loggers:
|
||||||
|
old_handlers.update(lg.handlers)
|
||||||
|
|
||||||
|
# Detach first, then close each unique handler exactly once.
|
||||||
|
for lg in all_loggers:
|
||||||
|
for h in list(lg.handlers):
|
||||||
|
lg.removeHandler(h)
|
||||||
|
for h in old_handlers:
|
||||||
|
h.close()
|
||||||
|
|
||||||
|
# Install fresh handlers on the slopsmith root.
|
||||||
|
root = logging.getLogger("slopsmith")
|
||||||
|
for h in handlers:
|
||||||
|
root.addHandler(h)
|
||||||
|
root.setLevel(level)
|
||||||
|
root.propagate = False
|
||||||
|
|
||||||
|
# Route uvicorn output through the same pipeline so everything is uniform.
|
||||||
|
for name in _uvicorn_names:
|
||||||
|
lg = logging.getLogger(name)
|
||||||
|
lg.handlers = list(handlers)
|
||||||
|
lg.propagate = False
|
||||||
|
lg.setLevel(level)
|
||||||
@@ -0,0 +1,434 @@
|
|||||||
|
"""loosefolder.py — treat a directory of raw RS2014 assets as a playable song.
|
||||||
|
|
||||||
|
Expected layout (artist/album/song_dir structure is optional but used for
|
||||||
|
metadata inference when manifest.json is absent):
|
||||||
|
|
||||||
|
song_dir/
|
||||||
|
audio.wem (required)
|
||||||
|
lead.xml (at least one arrangement XML required)
|
||||||
|
rhythm.xml
|
||||||
|
bass.xml
|
||||||
|
manifest.json (optional)
|
||||||
|
album_art.jpg / cover.jpg / album.png (optional)
|
||||||
|
|
||||||
|
manifest.json fields (all optional — XML metadata fills in gaps):
|
||||||
|
{
|
||||||
|
"title": "Song Name",
|
||||||
|
"artist": "Artist Name",
|
||||||
|
"album": "Album Name",
|
||||||
|
"year": "2024",
|
||||||
|
"tuning_offsets": [0, 0, 0, 0, 0, 0]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import xml.etree.ElementTree as ET
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
AUDIO_NAMES = ["audio.wem", "song.wem"]
|
||||||
|
# Match every extension server.get_song_art is prepared to serve
|
||||||
|
# (jpeg/png/webp). Without `.jpeg`/`.webp`, loose folders shipping
|
||||||
|
# `cover.jpeg` or `album_art.webp` would never have their art surfaced.
|
||||||
|
_ART_STEMS = ["album_art", "cover", "album", "art", "folder"]
|
||||||
|
_ART_EXTS = [".jpg", ".jpeg", ".png", ".webp"]
|
||||||
|
ART_NAMES = [f"{stem}{ext}" for stem in _ART_STEMS for ext in _ART_EXTS]
|
||||||
|
|
||||||
|
# Arrangement type detection from filename keywords or <arrangement> tag
|
||||||
|
# Format: keyword -> (type, display_name, sort_priority)
|
||||||
|
ARR_TYPE_MAP = {
|
||||||
|
"lead": ("lead", "Lead", 0),
|
||||||
|
"rhythm": ("rhythm", "Rhythm", 2),
|
||||||
|
"bass": ("bass", "Bass", 3),
|
||||||
|
"combo": ("combo", "Combo", 1),
|
||||||
|
"chord": ("combo", "Combo", 1),
|
||||||
|
"humstrum": ("combo", "Combo", 1),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _iter_local(path: Path, pattern: str):
|
||||||
|
"""Yield regular files matching `pattern` in `path` that resolve
|
||||||
|
inside `path`.
|
||||||
|
|
||||||
|
Two guards in one helper:
|
||||||
|
* Reject directories (a folder named `audio.wem` or `lead.xml`
|
||||||
|
would otherwise be matched by glob and break downstream
|
||||||
|
readers / converters).
|
||||||
|
* Reject symlinks escaping the folder so a crafted CDLC can't
|
||||||
|
smuggle external content into the scan.
|
||||||
|
"""
|
||||||
|
root = path.resolve()
|
||||||
|
for match in path.glob(pattern):
|
||||||
|
if not match.is_file():
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
resolved = match.resolve()
|
||||||
|
resolved.relative_to(root)
|
||||||
|
except (OSError, ValueError):
|
||||||
|
continue
|
||||||
|
yield resolved
|
||||||
|
|
||||||
|
|
||||||
|
def _iter_local_xmls(path: Path):
|
||||||
|
"""Backwards-compatible wrapper for `_iter_local(path, '*.xml')`."""
|
||||||
|
yield from _iter_local(path, "*.xml")
|
||||||
|
|
||||||
|
|
||||||
|
def is_loose_song(path: Path) -> bool:
|
||||||
|
"""True if this directory looks like a playable loose song folder.
|
||||||
|
|
||||||
|
Requires both a non-preview WEM and at least one arrangement XML
|
||||||
|
that isn't a vocals or showlights track — otherwise highway_ws would
|
||||||
|
later fail when it tries to pick an arrangement from an empty list.
|
||||||
|
|
||||||
|
Classification looks at the XML root element rather than the
|
||||||
|
filename so a custom named `lead_vocals_fix.xml` (root `<song>`)
|
||||||
|
still counts as a playable arrangement.
|
||||||
|
"""
|
||||||
|
if not path.is_dir():
|
||||||
|
return False
|
||||||
|
# Require an actual file that resolves inside `path` — both
|
||||||
|
# rejects directories named `audio.wem` and refuses symlinks
|
||||||
|
# escaping the song folder.
|
||||||
|
def _named_audio_ok(name: str) -> bool:
|
||||||
|
p = path / name
|
||||||
|
if not p.is_file():
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
p.resolve().relative_to(path.resolve())
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
has_audio = (
|
||||||
|
any(_named_audio_ok(a) for a in AUDIO_NAMES)
|
||||||
|
or any("preview" not in f.stem.lower()
|
||||||
|
for f in _iter_local(path, "*.wem"))
|
||||||
|
)
|
||||||
|
if not has_audio:
|
||||||
|
return False
|
||||||
|
for xml in _iter_local_xmls(path):
|
||||||
|
try:
|
||||||
|
root_tag = ET.parse(str(xml)).getroot().tag
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if root_tag == "song":
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def find_audio(path: Path) -> Path | None:
|
||||||
|
"""Return the path to the best audio file in the folder.
|
||||||
|
Prefers known names, then falls back to any WEM that isn't a preview clip.
|
||||||
|
Only returns regular files that resolve inside `path` — a directory,
|
||||||
|
broken symlink, or symlink escaping the folder would otherwise be
|
||||||
|
returned and either break convert_wem or read external content.
|
||||||
|
"""
|
||||||
|
root = path.resolve()
|
||||||
|
|
||||||
|
def _in_folder(p: Path) -> bool:
|
||||||
|
try:
|
||||||
|
p.resolve().relative_to(root)
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
# Check known names first
|
||||||
|
for a in AUDIO_NAMES:
|
||||||
|
cand = path / a
|
||||||
|
if cand.is_file() and _in_folder(cand):
|
||||||
|
return cand
|
||||||
|
|
||||||
|
def _safe_size(f: Path) -> int:
|
||||||
|
# Treat unreadable files (broken symlinks, permission errors)
|
||||||
|
# as zero-byte so they sort last and never get picked.
|
||||||
|
try:
|
||||||
|
return f.stat().st_size
|
||||||
|
except OSError:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
candidates = sorted(
|
||||||
|
[f for f in _iter_local(path, "*.wem")
|
||||||
|
if "preview" not in f.stem.lower()],
|
||||||
|
key=_safe_size,
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
return candidates[0] if candidates else None
|
||||||
|
|
||||||
|
|
||||||
|
def find_art(path: Path) -> Path | None:
|
||||||
|
"""Return the path to the first recognised album art file in the folder.
|
||||||
|
|
||||||
|
Only matches regular files — a directory named `cover.jpg` would
|
||||||
|
otherwise be returned and trip the FileResponse / containment
|
||||||
|
checks downstream.
|
||||||
|
"""
|
||||||
|
return next((path / a for a in ART_NAMES if (path / a).is_file()), None)
|
||||||
|
|
||||||
|
|
||||||
|
def _arr_type_from_filename(stem: str) -> tuple:
|
||||||
|
"""Infer arrangement type from filename keywords."""
|
||||||
|
s = stem.lower()
|
||||||
|
for key, val in ARR_TYPE_MAP.items():
|
||||||
|
if key in s:
|
||||||
|
return val
|
||||||
|
return ("lead", "Lead", 0) # fallback
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_xml_meta(xml_path: Path) -> dict:
|
||||||
|
"""Parse a Rocksmith arrangement XML and return song-level metadata."""
|
||||||
|
try:
|
||||||
|
root = ET.parse(str(xml_path)).getroot()
|
||||||
|
if root.tag != "song":
|
||||||
|
return {}
|
||||||
|
|
||||||
|
def txt(tag, default=""):
|
||||||
|
el = root.find(tag)
|
||||||
|
return el.text.strip() if el is not None and el.text else default
|
||||||
|
|
||||||
|
# Tuning from attributes
|
||||||
|
tuning_el = root.find("tuning")
|
||||||
|
if tuning_el is not None:
|
||||||
|
offsets = [int(tuning_el.get(f"string{i}", 0)) for i in range(6)]
|
||||||
|
else:
|
||||||
|
offsets = [0] * 6
|
||||||
|
|
||||||
|
# Arrangement type — filename is more reliable than the XML tag
|
||||||
|
# because some authoring tools write "Lead" for all arrangements.
|
||||||
|
# We read the XML tag here and let _detect_arrangements decide
|
||||||
|
# which source to trust.
|
||||||
|
arr_tag = txt("arrangement", "").lower()
|
||||||
|
arr_from_tag = ARR_TYPE_MAP.get(arr_tag, None)
|
||||||
|
|
||||||
|
duration = 0.0
|
||||||
|
try:
|
||||||
|
duration = float(txt("songLength", "0"))
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
return {
|
||||||
|
"title": txt("title"),
|
||||||
|
"artist": txt("artistName"),
|
||||||
|
"album": txt("albumName"),
|
||||||
|
"year": txt("albumYear", ""),
|
||||||
|
"duration": duration,
|
||||||
|
"tuning_offsets": offsets,
|
||||||
|
"arr_from_tag": arr_from_tag, # may be None
|
||||||
|
}
|
||||||
|
except Exception:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def _detect_arrangements(path: Path) -> tuple[list[dict], dict]:
|
||||||
|
"""
|
||||||
|
Parse all arrangement XMLs.
|
||||||
|
Returns (arrangements_list, shared_meta).
|
||||||
|
shared_meta contains title/artist/album/year/duration/tuning_offsets
|
||||||
|
sourced from the highest-priority arrangement (lead > combo > rhythm >
|
||||||
|
bass) — picking the guitar tuning when both bass and lead are present.
|
||||||
|
"""
|
||||||
|
arrangements = []
|
||||||
|
# Track which arrangement priority sourced shared_meta so a later,
|
||||||
|
# higher-priority arrangement (lead < bass in sort order) overrides.
|
||||||
|
shared_meta = {}
|
||||||
|
shared_priority = None
|
||||||
|
|
||||||
|
for xml in sorted(_iter_local_xmls(path)):
|
||||||
|
# Trust the XML root over the filename — a custom named
|
||||||
|
# `lead_vocals_fix.xml` is still a real arrangement.
|
||||||
|
# `_parse_xml_meta` returns {} for any root other than <song>,
|
||||||
|
# which is how vocals/showlights tracks get filtered out.
|
||||||
|
stem = xml.stem.lower()
|
||||||
|
|
||||||
|
meta = _parse_xml_meta(xml)
|
||||||
|
if not meta:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Arrangement type: prefer filename keywords over the XML tag
|
||||||
|
# because some tools write "Lead" for all arrangements regardless
|
||||||
|
# of actual type. Only fall back to the XML tag when the filename
|
||||||
|
# gives no useful signal (i.e. no recognisable keyword found).
|
||||||
|
filename_type = _arr_type_from_filename(stem)
|
||||||
|
if filename_type[0] != "lead" or "lead" in stem:
|
||||||
|
# Filename gave a confident answer
|
||||||
|
arr_type, arr_name, priority = filename_type
|
||||||
|
else:
|
||||||
|
# Filename wasn't specific — try the XML tag
|
||||||
|
arr_from_tag = meta.get("arr_from_tag")
|
||||||
|
if arr_from_tag:
|
||||||
|
arr_type, arr_name, priority = arr_from_tag
|
||||||
|
else:
|
||||||
|
arr_type, arr_name, priority = filename_type
|
||||||
|
|
||||||
|
# Take song-level fields from the highest-priority arrangement
|
||||||
|
# (lowest `priority` number) so tuning reflects the main guitar
|
||||||
|
# instead of whatever sorted first alphabetically.
|
||||||
|
if meta.get("title") and (shared_priority is None or priority < shared_priority):
|
||||||
|
shared_meta = {k: meta[k] for k in
|
||||||
|
("title", "artist", "album", "year",
|
||||||
|
"duration", "tuning_offsets")}
|
||||||
|
shared_priority = priority
|
||||||
|
|
||||||
|
arrangements.append({
|
||||||
|
"type": arr_type,
|
||||||
|
"name": arr_name,
|
||||||
|
"file": xml.name,
|
||||||
|
"priority": priority,
|
||||||
|
})
|
||||||
|
|
||||||
|
arrangements.sort(key=lambda a: a["priority"])
|
||||||
|
for i, a in enumerate(arrangements):
|
||||||
|
a["index"] = i
|
||||||
|
del a["priority"]
|
||||||
|
|
||||||
|
return arrangements, shared_meta
|
||||||
|
|
||||||
|
|
||||||
|
def _has_lyrics(path: Path) -> bool:
|
||||||
|
"""Return True if any XML in the folder is a vocals track."""
|
||||||
|
for xml in _iter_local_xmls(path):
|
||||||
|
try:
|
||||||
|
if ET.parse(str(xml)).getroot().tag == "vocals":
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_duration(raw, fallback) -> float:
|
||||||
|
"""Coerce a manifest duration to a finite float, falling back on bad input.
|
||||||
|
|
||||||
|
Rejects NaN / Infinity so a manifest like `{"duration": "Infinity"}`
|
||||||
|
can't poison `meta_db` and then crash Starlette's JSON encoder when
|
||||||
|
the row is served back through `/api/song/...`.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
v = float(raw)
|
||||||
|
if math.isfinite(v):
|
||||||
|
return v
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
v = float(fallback or 0.0)
|
||||||
|
return v if math.isfinite(v) else 0.0
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_text(raw) -> str | None:
|
||||||
|
"""Return raw if it's a non-empty string, else None.
|
||||||
|
|
||||||
|
Manifest fields like `title` / `artist` / `album` can arrive as
|
||||||
|
JSON nulls, lists, or numbers (e.g. someone setting album to a
|
||||||
|
year by mistake). Returning None lets the caller fall back to
|
||||||
|
XML / folder inference instead of crashing the DB row write.
|
||||||
|
"""
|
||||||
|
if isinstance(raw, str) and raw:
|
||||||
|
return raw
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_tuning_offsets(raw, fallback) -> list[int]:
|
||||||
|
"""Validate manifest tuning_offsets: must be a list of 6 numeric values."""
|
||||||
|
if isinstance(raw, list) and len(raw) == 6:
|
||||||
|
try:
|
||||||
|
return [int(v) for v in raw]
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
pass
|
||||||
|
if isinstance(fallback, list) and len(fallback) == 6:
|
||||||
|
return list(fallback)
|
||||||
|
return [0] * 6
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_manifest_arrangements(raw) -> list[dict] | None:
|
||||||
|
"""Return raw if it's a well-formed arrangement list, else None.
|
||||||
|
|
||||||
|
Each entry must be a dict carrying at least `type`, `name`, `file`
|
||||||
|
(string-typed). Bad shapes get dropped so the parsed XML list wins.
|
||||||
|
"""
|
||||||
|
if not isinstance(raw, list) or not raw:
|
||||||
|
return None
|
||||||
|
out = []
|
||||||
|
for entry in raw:
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
return None
|
||||||
|
if not all(isinstance(entry.get(k), str) and entry.get(k)
|
||||||
|
for k in ("type", "name", "file")):
|
||||||
|
return None
|
||||||
|
out.append(entry)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def extract_meta(path: Path, dlc_root: Path | None = None) -> dict:
|
||||||
|
"""Return metadata dict for a loose song folder.
|
||||||
|
|
||||||
|
Priority chain:
|
||||||
|
1. manifest.json (explicit user-supplied data)
|
||||||
|
2. XML metadata (parsed from arrangement XMLs)
|
||||||
|
3. Folder name (last resort inference from directory structure)
|
||||||
|
|
||||||
|
`dlc_root` is used to bound the folder-name inference: artist/album
|
||||||
|
are only inferred from `path.relative_to(dlc_root)` components, so
|
||||||
|
a loose folder placed at `<DLC>/song/` doesn't accidentally surface
|
||||||
|
the user's home-directory name as the artist.
|
||||||
|
"""
|
||||||
|
# 1. Try manifest.json
|
||||||
|
manifest_path = path / "manifest.json"
|
||||||
|
manifest = {}
|
||||||
|
if manifest_path.exists():
|
||||||
|
try:
|
||||||
|
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if not isinstance(manifest, dict):
|
||||||
|
manifest = {}
|
||||||
|
|
||||||
|
# 2. Parse XMLs for song metadata + arrangements
|
||||||
|
arrangements, xml_meta = _detect_arrangements(path)
|
||||||
|
|
||||||
|
# 3. Folder inference — only from path components under DLC_DIR so
|
||||||
|
# absolute-path parts (`/home/<user>/...`) can never leak as artist.
|
||||||
|
rel_parts: tuple[str, ...] = ()
|
||||||
|
if dlc_root is not None:
|
||||||
|
try:
|
||||||
|
rel_parts = path.resolve().relative_to(dlc_root.resolve()).parts
|
||||||
|
except (ValueError, OSError):
|
||||||
|
rel_parts = ()
|
||||||
|
|
||||||
|
# Coerce manifest text fields — non-strings (lists, numbers, null)
|
||||||
|
# would otherwise propagate into meta_db rows and break DB writes.
|
||||||
|
title = _coerce_text(manifest.get("title")) or xml_meta.get("title") or path.name
|
||||||
|
artist = (_coerce_text(manifest.get("artist")) or xml_meta.get("artist")
|
||||||
|
or (rel_parts[-3] if len(rel_parts) >= 3 else ""))
|
||||||
|
album = (_coerce_text(manifest.get("album")) or xml_meta.get("album")
|
||||||
|
or (rel_parts[-2] if len(rel_parts) >= 2 else ""))
|
||||||
|
raw_year = manifest.get("year")
|
||||||
|
if isinstance(raw_year, (int, float)) and not isinstance(raw_year, bool):
|
||||||
|
manifest_year = str(int(raw_year))
|
||||||
|
else:
|
||||||
|
manifest_year = _coerce_text(raw_year) or ""
|
||||||
|
year = manifest_year or str(xml_meta.get("year", ""))
|
||||||
|
duration = _coerce_duration(manifest.get("duration"),
|
||||||
|
xml_meta.get("duration", 0))
|
||||||
|
tuning_offsets = _coerce_tuning_offsets(manifest.get("tuning_offsets"),
|
||||||
|
xml_meta.get("tuning_offsets"))
|
||||||
|
|
||||||
|
manifest_arr = _validate_manifest_arrangements(manifest.get("arrangements"))
|
||||||
|
if manifest_arr is not None:
|
||||||
|
arrangements = manifest_arr
|
||||||
|
|
||||||
|
audio = find_audio(path)
|
||||||
|
art = find_art(path)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"title": title,
|
||||||
|
"artist": artist,
|
||||||
|
"album": album,
|
||||||
|
"year": year,
|
||||||
|
"duration": duration,
|
||||||
|
"tuning_offsets": tuning_offsets,
|
||||||
|
"arrangements": arrangements,
|
||||||
|
"audio_path": str(audio) if audio else None,
|
||||||
|
"art_path": str(art) if art else None,
|
||||||
|
"has_lyrics": _has_lyrics(path),
|
||||||
|
}
|
||||||
@@ -0,0 +1,512 @@
|
|||||||
|
"""WhisperX-based lyric transcription for vocal stems.
|
||||||
|
|
||||||
|
Acts as a fallback path when a sloppak lacks `lyrics.json`. Operates on an
|
||||||
|
already-isolated vocal stem (a Demucs `vocals.ogg`) — does NOT separate
|
||||||
|
vocals from a mixed track; that's the caller's responsibility.
|
||||||
|
|
||||||
|
Output shape matches the on-disk `lyrics.json` shape documented at
|
||||||
|
`docs/sloppak-spec.md` §2.3:
|
||||||
|
|
||||||
|
[{"t": float, "d": float, "w": str}, ...]
|
||||||
|
|
||||||
|
`t` and `d` are seconds. `w` carries a `-` suffix when it joins to the
|
||||||
|
following syllable, and a `+` suffix when it's the last syllable on a
|
||||||
|
line (the frontend renderer in `static/highway.js` keys off
|
||||||
|
`raw.endsWith('+')` and strips the suffix before drawing — see
|
||||||
|
`docs/sloppak-spec.md` §2.3). Both markers are suffixes on real
|
||||||
|
syllables, never standalone tokens. WhisperX emits words, not
|
||||||
|
syllables; the mapper appends `+` to the previous word on segment-gap
|
||||||
|
heuristics and otherwise lets each word stand as its own syllable.
|
||||||
|
|
||||||
|
Engine selection
|
||||||
|
────────────────
|
||||||
|
Two transcription paths share a common output:
|
||||||
|
|
||||||
|
* `transcribe_vocals_remote(path, server_url, ...)` — POST the vocal
|
||||||
|
stem to the `/align` endpoint on a slopsmith-demucs-server (Byron's
|
||||||
|
reference server already hosts WhisperX alongside Demucs at the same
|
||||||
|
URL).
|
||||||
|
|
||||||
|
* `transcribe_vocals_local(path, ...)` — load WhisperX in-process. Heavy
|
||||||
|
(~3 GB of model weights for `large-v2` + the wav2vec2 aligner) and
|
||||||
|
slow on CPU. Deferred imports of `whisperx`, `torch`, and `soundfile`
|
||||||
|
keep the rest of slopsmith free of those dependencies.
|
||||||
|
|
||||||
|
Callers pick between them based on a `whisperx.server_url` config and
|
||||||
|
fall back as appropriate. This module does not read config — both
|
||||||
|
entry points are pure functions of their arguments.
|
||||||
|
|
||||||
|
Hallucination mitigation
|
||||||
|
────────────────────────
|
||||||
|
Whisper invents plausible-sounding lyrics on near-silent or purely
|
||||||
|
instrumental input. Two gates guard against that:
|
||||||
|
|
||||||
|
1. `vocals_has_signal(path, threshold)` — cheap RMS check before
|
||||||
|
inference. Skips songs where the vocal stem is below threshold
|
||||||
|
(Demucs returns near-silent vocals for instrumentals).
|
||||||
|
|
||||||
|
2. `min_word_score` post-filter — WhisperX's word alignment emits a
|
||||||
|
per-word confidence score; words below the threshold are dropped
|
||||||
|
from the output. Default 0.35 matches the value the reference
|
||||||
|
TabGrabber prototype settled on.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import gc
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable, Optional
|
||||||
|
|
||||||
|
log = logging.getLogger("slopsmith.lib.lyrics_transcribe")
|
||||||
|
|
||||||
|
ProgressCB = Optional[Callable[[float, str, str], None]]
|
||||||
|
|
||||||
|
|
||||||
|
# ── Availability probes ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def whisperx_available() -> bool:
|
||||||
|
"""Cheap probe — does this interpreter have whisperx importable?
|
||||||
|
|
||||||
|
The local transcription path imports whisperx lazily, so this probe lets
|
||||||
|
callers gate on availability without paying the full import cost
|
||||||
|
(which transitively pulls torch and may try to initialize CUDA).
|
||||||
|
|
||||||
|
Catches a broader exception set than just ImportError because
|
||||||
|
importing whisperx can fail with OSError (libsndfile or other
|
||||||
|
native libs missing), RuntimeError (torch CUDA init failure,
|
||||||
|
BLAS/LAPACK load problems), or essentially any exception the
|
||||||
|
deep transitive stack chooses to raise. The tests assert this
|
||||||
|
probe NEVER raises; falling back to False for any failure mode
|
||||||
|
keeps that contract while still surfacing the real error if a
|
||||||
|
later actual transcription tries to use the helper."""
|
||||||
|
try:
|
||||||
|
import whisperx # noqa: F401
|
||||||
|
return True
|
||||||
|
except (ImportError, OSError, RuntimeError) as e:
|
||||||
|
log.debug("whisperx_available: import failed (%s)", e)
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
# Last-resort catch so a transient/unexpected failure can't
|
||||||
|
# crash the caller. Logged at WARNING so it's visible in normal
|
||||||
|
# operation (vs the expected ImportError on installs without
|
||||||
|
# whisperx, which stays at DEBUG).
|
||||||
|
log.warning("whisperx_available: unexpected probe failure (%s)", e)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# ── Silence gate ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def vocals_has_signal(vocals_path: Path, threshold: float = 0.005) -> bool:
|
||||||
|
"""Return True if the vocal stem has RMS energy above `threshold`.
|
||||||
|
|
||||||
|
Cheap pre-check intended to short-circuit transcription on
|
||||||
|
instrumentals — Demucs separates instrumental tracks into a
|
||||||
|
near-silent vocals stem, and running Whisper on silence produces
|
||||||
|
hallucinated lyrics. The default threshold is conservative; a
|
||||||
|
truly silent stem reads ~1e-6, normal vocals well above 0.01.
|
||||||
|
|
||||||
|
Returns True when soundfile or numpy is missing OR fails to load
|
||||||
|
its native lib (best-effort gate, not a hard requirement). The
|
||||||
|
transcription itself will surface the real failure if those deps
|
||||||
|
are actually needed downstream.
|
||||||
|
|
||||||
|
Catching OSError matters because `import soundfile` performs a
|
||||||
|
ctypes load of `libsndfile` at import time — on a host without the
|
||||||
|
native lib installed, that raises `OSError` (not ImportError) and
|
||||||
|
would otherwise propagate up and break the surrounding
|
||||||
|
transcription run instead of just skipping the gate."""
|
||||||
|
try:
|
||||||
|
import numpy as np
|
||||||
|
import soundfile as sf
|
||||||
|
except (ImportError, OSError) as e:
|
||||||
|
log.debug("vocals_has_signal: soundfile/numpy unavailable (%s) — skipping gate", e)
|
||||||
|
return True
|
||||||
|
# Stream the file in blocks instead of loading the whole stem into
|
||||||
|
# memory. A 4-minute stereo vocal stem at 44.1kHz is ~84 MB as
|
||||||
|
# float32; multiply that across a batch of conversions and the
|
||||||
|
# allocations get noticeable. SoundFile.blocks() yields chunks
|
||||||
|
# without ever holding the full buffer, and we only need a
|
||||||
|
# running sum-of-squares + frame count to compute RMS at the end.
|
||||||
|
# Short-circuit threshold check inside the loop: once we've
|
||||||
|
# accumulated enough signal to clear the gate, no need to keep
|
||||||
|
# scanning the rest of the file.
|
||||||
|
sumsq = 0.0
|
||||||
|
nframes = 0
|
||||||
|
try:
|
||||||
|
with sf.SoundFile(str(vocals_path)) as fh:
|
||||||
|
for block in fh.blocks(blocksize=65536, dtype="float32", always_2d=False):
|
||||||
|
if block.size == 0:
|
||||||
|
continue
|
||||||
|
if block.ndim > 1:
|
||||||
|
block = block.mean(axis=1)
|
||||||
|
sumsq += float(np.sum(np.square(block)))
|
||||||
|
nframes += int(block.shape[0])
|
||||||
|
# Early exit once we know the gate will pass — no point
|
||||||
|
# reading the rest of a 4-minute file to confirm.
|
||||||
|
if nframes > 0 and (sumsq / nframes) >= (threshold * threshold):
|
||||||
|
log.debug("vocals_has_signal: %s passed early at %d frames",
|
||||||
|
vocals_path.name, nframes)
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
log.warning("vocals_has_signal: read of %s failed: %s", vocals_path, e)
|
||||||
|
return True
|
||||||
|
if nframes == 0:
|
||||||
|
return False
|
||||||
|
rms = float(np.sqrt(sumsq / nframes))
|
||||||
|
log.debug("vocals_has_signal: %s rms=%.6f threshold=%.6f", vocals_path.name, rms, threshold)
|
||||||
|
return rms >= threshold
|
||||||
|
|
||||||
|
|
||||||
|
# ── Output mapping ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
# Gap (in seconds) between WhisperX segments that triggers a `+` line break
|
||||||
|
# syllable in the sloppak output. Bumped from 1.5s (TabGrabber's value) to
|
||||||
|
# 3.0s after seeing the lower threshold produce short-burst phrasing on
|
||||||
|
# sung material — singers breathe at ~0.5-1.5s between phrases of the
|
||||||
|
# same verse, so the tighter cutoff fragmented every line into a few
|
||||||
|
# words. 3.0s captures stanza-level pauses (verse→chorus, end-of-bridge)
|
||||||
|
# while keeping intra-line breaths grouped on one rendered line. The
|
||||||
|
# highway renderer still has its own 4.0s safety fallback (see
|
||||||
|
# static/highway.js) that forces a wrap regardless, so this only
|
||||||
|
# controls when WE author breaks vs delegating to the renderer.
|
||||||
|
_LINE_BREAK_GAP_SECONDS = 3.0
|
||||||
|
|
||||||
|
# Floor on per-word duration in the sloppak output. WhisperX occasionally
|
||||||
|
# emits zero-length words for very short syllables; the highway overlay's
|
||||||
|
# fade timing expects a non-zero `d`, so clamp here.
|
||||||
|
_MIN_WORD_DURATION = 0.05
|
||||||
|
|
||||||
|
# Semver for the lyric-transcription artifact contract that gets stamped
|
||||||
|
# into the sloppak manifest's `lyric_transcription` block alongside the
|
||||||
|
# engine + model. Bump per the semantics defined in slopsmith#357 (the
|
||||||
|
# parent `stem_separation` RFC):
|
||||||
|
# * patch — metadata-only or implementation fixes; no regeneration
|
||||||
|
# * minor — backward-compatible additions
|
||||||
|
# * major — output shape / semantics changed; existing transcriptions
|
||||||
|
# should be regenerated
|
||||||
|
# Independent from any upstream WhisperX / Whisper / wav2vec2 version.
|
||||||
|
LYRIC_TRANSCRIPTION_SCHEMA_VERSION = "1.0.0"
|
||||||
|
LYRIC_TRANSCRIPTION_ENGINE = "whisperx"
|
||||||
|
|
||||||
|
|
||||||
|
def _whisperx_to_sloppak(aligned: dict, min_score: float) -> list[dict]:
|
||||||
|
"""Map WhisperX `aligned` output to sloppak `lyrics.json` shape.
|
||||||
|
|
||||||
|
`aligned` is the dict returned by `whisperx.align()`: a `segments`
|
||||||
|
list, each segment carrying a `words` list of `{word, start, end,
|
||||||
|
score}` dicts. Drops words below `min_score` (hallucination filter)
|
||||||
|
and marks line breaks on segment gaps that exceed
|
||||||
|
`_LINE_BREAK_GAP_SECONDS`.
|
||||||
|
|
||||||
|
Line-break encoding follows the frontend lyric renderer's convention
|
||||||
|
in `static/highway.js`: `+` is a SUFFIX on the last word of a line,
|
||||||
|
not a standalone token. A bare `{"w": "+"}` token would be parsed
|
||||||
|
as an empty syllable that ends a line — visible as a blank slot in
|
||||||
|
the overlay. Emitting `"world+"` instead keeps the syllable count
|
||||||
|
correct and the renderer strips the suffix when drawing.
|
||||||
|
|
||||||
|
Times are rounded to 3 decimals to match the on-disk `lyrics.json`
|
||||||
|
convention (docs/sloppak-spec.md §2.3)."""
|
||||||
|
out: list[dict] = []
|
||||||
|
# `prev_end` tracks the actual end of the last processed segment
|
||||||
|
# (NOT the last surviving word), so the gap heuristic measures
|
||||||
|
# against real audio timing. Segments whose only words get filtered
|
||||||
|
# out still advance the cursor — otherwise the next segment's gap
|
||||||
|
# would falsely measure all the way back to whatever survived
|
||||||
|
# several segments ago.
|
||||||
|
prev_end: float | None = None
|
||||||
|
for segment in aligned.get("segments", []) or []:
|
||||||
|
words = segment.get("words") or []
|
||||||
|
# Walk every word, regardless of whether it survives the
|
||||||
|
# confidence filter, so we can apply the line-break heuristic
|
||||||
|
# at the moment we actually emit a syllable. Doing the gap
|
||||||
|
# check at emit time (vs. once per segment) means a segment
|
||||||
|
# whose entire word list gets filtered can't strand a "pending"
|
||||||
|
# break that fires against an unrelated syllable in a later
|
||||||
|
# segment.
|
||||||
|
for w in words:
|
||||||
|
text = (w.get("word") or "").strip()
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
start = w.get("start")
|
||||||
|
end = w.get("end")
|
||||||
|
score = w.get("score")
|
||||||
|
# Drop words that fail confidence threshold. WhisperX
|
||||||
|
# occasionally emits words without a score (e.g. when
|
||||||
|
# alignment couldn't localize them); treat those as
|
||||||
|
# untrustworthy and drop too.
|
||||||
|
if not isinstance(score, (int, float)) or score < min_score:
|
||||||
|
continue
|
||||||
|
if not isinstance(start, (int, float)) or not isinstance(end, (int, float)):
|
||||||
|
continue
|
||||||
|
# Line break: suffix `+` on the previous emitted syllable
|
||||||
|
# if there's a comfortably large silence between the last
|
||||||
|
# processed-segment cursor and the current surviving word.
|
||||||
|
# Anchoring on `prev_end` (segment end), not `out[-1]`'s
|
||||||
|
# actual end, keeps the heuristic aligned with real audio
|
||||||
|
# timing — a long mid-segment pause within one phrase
|
||||||
|
# shouldn't force a break, and a trailing-words-filtered
|
||||||
|
# segment shouldn't falsely inflate the gap to the next one.
|
||||||
|
if (
|
||||||
|
prev_end is not None
|
||||||
|
and (float(start) - prev_end) > _LINE_BREAK_GAP_SECONDS
|
||||||
|
and out
|
||||||
|
and not out[-1]["w"].endswith("+")
|
||||||
|
):
|
||||||
|
out[-1]["w"] = out[-1]["w"] + "+"
|
||||||
|
duration = max(_MIN_WORD_DURATION, float(end) - float(start))
|
||||||
|
out.append({
|
||||||
|
"t": round(float(start), 3),
|
||||||
|
"d": round(duration, 3),
|
||||||
|
"w": text,
|
||||||
|
})
|
||||||
|
# Advance `prev_end` to the segment's actual end (or the latest
|
||||||
|
# numeric word end if the segment lacks an `end`). This runs
|
||||||
|
# for every segment — even empty / fully-filtered ones — so
|
||||||
|
# the next segment's gap measurement reflects real audio
|
||||||
|
# timing regardless of survivorship.
|
||||||
|
seg_end = segment.get("end")
|
||||||
|
if isinstance(seg_end, (int, float)):
|
||||||
|
prev_end = float(seg_end)
|
||||||
|
else:
|
||||||
|
word_ends = [
|
||||||
|
float(w["end"]) for w in words
|
||||||
|
if isinstance(w.get("end"), (int, float))
|
||||||
|
]
|
||||||
|
if word_ends:
|
||||||
|
prev_end = max(word_ends)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
# ── Local transcription ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _pick_compute_type(device: str) -> str:
|
||||||
|
"""Match TabGrabber's compute-type defaults: float16 on CUDA, int8 on CPU.
|
||||||
|
|
||||||
|
WhisperX accepts float16/float32/int8 on CUDA and int8/float32 on CPU.
|
||||||
|
int8 is the only viable choice for CPU inference at usable speeds."""
|
||||||
|
return "float16" if device == "cuda" else "int8"
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_device(device: str | None) -> str:
|
||||||
|
if device and device != "auto":
|
||||||
|
return device
|
||||||
|
try:
|
||||||
|
import torch
|
||||||
|
return "cuda" if torch.cuda.is_available() else "cpu"
|
||||||
|
except ImportError:
|
||||||
|
return "cpu"
|
||||||
|
|
||||||
|
|
||||||
|
def _free_gpu_memory() -> None:
|
||||||
|
"""Force a GC + CUDA cache flush.
|
||||||
|
|
||||||
|
Note that `del`-ing a local in a helper function only deletes the
|
||||||
|
helper's parameter binding, not the caller's reference — to actually
|
||||||
|
drop the model the caller must null its own variables (see the
|
||||||
|
finally block in `transcribe_vocals_local`). This helper only handles
|
||||||
|
the GC + CUDA side, which is the same regardless of who held the
|
||||||
|
references. Safe to call regardless of CUDA availability or whether
|
||||||
|
torch is even installed."""
|
||||||
|
gc.collect()
|
||||||
|
try:
|
||||||
|
import torch
|
||||||
|
if torch.cuda.is_available():
|
||||||
|
torch.cuda.empty_cache()
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def transcribe_vocals_local(
|
||||||
|
vocals_path: Path,
|
||||||
|
*,
|
||||||
|
model_size: str = "medium",
|
||||||
|
language: str | None = None,
|
||||||
|
device: str | None = None,
|
||||||
|
compute_type: str | None = None,
|
||||||
|
min_word_score: float = 0.35,
|
||||||
|
progress_cb: ProgressCB = None,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Run WhisperX in-process against a vocal stem.
|
||||||
|
|
||||||
|
Deferred whisperx import — callers gate on `whisperx_available()`
|
||||||
|
first to avoid the ImportError surfacing here. Heavy: first call
|
||||||
|
downloads ~1.5 GB of model weights for `medium` (~3 GB for
|
||||||
|
`large-v2`) into the WhisperX cache.
|
||||||
|
|
||||||
|
`model_size` is one of WhisperX's accepted sizes: tiny, base, small,
|
||||||
|
medium, large-v2, large-v3. Default `medium` balances accuracy and
|
||||||
|
first-run download size; bump to `large-v2` for production quality.
|
||||||
|
|
||||||
|
`language` is an ISO code (e.g. `"en"`); `None` lets WhisperX
|
||||||
|
autodetect from the audio.
|
||||||
|
|
||||||
|
`device` is `"cuda"` / `"cpu"` / `None` (auto-detect). `compute_type`
|
||||||
|
follows TabGrabber's defaults when `None`."""
|
||||||
|
try:
|
||||||
|
import whisperx
|
||||||
|
except ImportError as e:
|
||||||
|
raise RuntimeError(
|
||||||
|
"whisperx not installed. Install via `pip install whisperx`."
|
||||||
|
) from e
|
||||||
|
|
||||||
|
resolved_device = _resolve_device(device)
|
||||||
|
resolved_compute = compute_type or _pick_compute_type(resolved_device)
|
||||||
|
|
||||||
|
if progress_cb:
|
||||||
|
try:
|
||||||
|
progress_cb(0.05, "transcribing", f"Loading WhisperX ({model_size}, {resolved_device})")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Wrap every model lifecycle call in a single try/finally so a failure in
|
||||||
|
# load_audio / transcribe / load_align_model still frees the ASR model —
|
||||||
|
# otherwise a bad stem in the middle of a batch run strands GPU memory and
|
||||||
|
# the next song's load_model OOMs.
|
||||||
|
#
|
||||||
|
# Caller-side `= None` reassignment is the only way to actually drop the
|
||||||
|
# references here; a helper's `del m` only releases the helper's binding,
|
||||||
|
# leaving the caller's reference live and the GPU memory pinned until
|
||||||
|
# this function returns. That defeats the purpose of running gc + empty
|
||||||
|
# cache mid-batch — by the time the next song's transcribe_vocals_local
|
||||||
|
# fires, we want the previous model GONE, not held until the caller
|
||||||
|
# frame unwinds.
|
||||||
|
asr_model = align_model = align_metadata = None
|
||||||
|
try:
|
||||||
|
asr_model = whisperx.load_model(model_size, resolved_device, compute_type=resolved_compute)
|
||||||
|
audio = whisperx.load_audio(str(vocals_path))
|
||||||
|
|
||||||
|
if progress_cb:
|
||||||
|
try:
|
||||||
|
progress_cb(0.30, "transcribing", "Transcribing vocals")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
result = asr_model.transcribe(audio, language=language)
|
||||||
|
detected_lang = result.get("language") or language or "en"
|
||||||
|
|
||||||
|
if progress_cb:
|
||||||
|
try:
|
||||||
|
progress_cb(0.60, "transcribing", f"Aligning words ({detected_lang})")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
align_model, align_metadata = whisperx.load_align_model(
|
||||||
|
language_code=detected_lang, device=resolved_device
|
||||||
|
)
|
||||||
|
aligned = whisperx.align(
|
||||||
|
result["segments"], align_model, align_metadata, audio,
|
||||||
|
resolved_device, return_char_alignments=False,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
asr_model = None
|
||||||
|
align_model = None
|
||||||
|
align_metadata = None
|
||||||
|
_free_gpu_memory()
|
||||||
|
|
||||||
|
if progress_cb:
|
||||||
|
try:
|
||||||
|
progress_cb(0.90, "transcribing", "Building lyric tokens")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return _whisperx_to_sloppak(aligned, min_word_score)
|
||||||
|
|
||||||
|
|
||||||
|
# ── Remote transcription ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def transcribe_vocals_remote(
|
||||||
|
vocals_path: Path,
|
||||||
|
server_url: str,
|
||||||
|
*,
|
||||||
|
language: str | None = None,
|
||||||
|
api_key: str | None = None,
|
||||||
|
timeout: int = 300,
|
||||||
|
min_word_score: float = 0.35,
|
||||||
|
progress_cb: ProgressCB = None,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""POST the vocal stem to `{server_url}/align` and parse the response.
|
||||||
|
|
||||||
|
Expects the server to respond with a JSON object carrying a `words` (or
|
||||||
|
`segments`) field in WhisperX's native shape; `_whisperx_to_sloppak`
|
||||||
|
consumes that directly.
|
||||||
|
|
||||||
|
`min_word_score` is applied to native `segments` responses the same
|
||||||
|
way the local path applies it, so the hallucination guard doesn't
|
||||||
|
weaken when routing to a remote server. Pre-flattened `{"words": [...]}`
|
||||||
|
responses are passed through unfiltered (the server is assumed to
|
||||||
|
have applied its own gating before flattening).
|
||||||
|
|
||||||
|
Errors raise `RuntimeError` with a truncated server response, same
|
||||||
|
idiom Demucs uses, so the caller can log+continue without bringing
|
||||||
|
down the surrounding split job."""
|
||||||
|
import requests
|
||||||
|
|
||||||
|
server_url = server_url.rstrip("/")
|
||||||
|
if progress_cb:
|
||||||
|
try:
|
||||||
|
progress_cb(0.10, "transcribing", f"Uploading to WhisperX server ({server_url})")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
headers: dict[str, str] = {}
|
||||||
|
if api_key:
|
||||||
|
headers["Authorization"] = f"Bearer {api_key}"
|
||||||
|
|
||||||
|
params: dict[str, str] = {}
|
||||||
|
if language:
|
||||||
|
params["language"] = language
|
||||||
|
|
||||||
|
with open(vocals_path, "rb") as f:
|
||||||
|
resp = requests.post(
|
||||||
|
f"{server_url}/align",
|
||||||
|
files={"file": (vocals_path.name, f, "audio/ogg")},
|
||||||
|
params=params,
|
||||||
|
headers=headers or None,
|
||||||
|
timeout=timeout,
|
||||||
|
)
|
||||||
|
|
||||||
|
if resp.status_code != 200:
|
||||||
|
raise RuntimeError(f"WhisperX server error ({resp.status_code}): {resp.text[:300]}")
|
||||||
|
|
||||||
|
data = resp.json()
|
||||||
|
|
||||||
|
# Two response shapes are accepted, in this order of preference:
|
||||||
|
#
|
||||||
|
# 1. Native WhisperX `{"segments": [...]}` — let the standard
|
||||||
|
# mapper handle it (line breaks + score filter + clamps).
|
||||||
|
# 2. Pre-flattened sloppak shape `{"words": [{"t","d","w"}, ...]}`
|
||||||
|
# — pass through with rounding for parity with local path.
|
||||||
|
#
|
||||||
|
# Anything else is an error: surface enough of the response that
|
||||||
|
# `_maybe_transcribe_lyrics` can log it and move on.
|
||||||
|
if "segments" in data:
|
||||||
|
return _whisperx_to_sloppak(data, min_score=min_word_score)
|
||||||
|
if "words" in data:
|
||||||
|
raw_words = data["words"]
|
||||||
|
if not isinstance(raw_words, list):
|
||||||
|
raise RuntimeError(
|
||||||
|
f"WhisperX server returned non-list `words`: {type(raw_words).__name__}"
|
||||||
|
)
|
||||||
|
out: list[dict] = []
|
||||||
|
for w in raw_words:
|
||||||
|
# Defensive: a malformed server could ship strings, numbers,
|
||||||
|
# or partial dicts. Skip anything that isn't a dict with all
|
||||||
|
# three required keys so the loop doesn't crash on bad data —
|
||||||
|
# the worst case is a partial transcription, not a wedged job.
|
||||||
|
if not isinstance(w, dict):
|
||||||
|
continue
|
||||||
|
if "t" not in w or "d" not in w or "w" not in w:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
out.append({
|
||||||
|
"t": round(float(w["t"]), 3),
|
||||||
|
"d": round(float(w["d"]), 3),
|
||||||
|
"w": str(w["w"]),
|
||||||
|
})
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
# Bad numeric types on this entry; skip and continue.
|
||||||
|
continue
|
||||||
|
return out
|
||||||
|
raise RuntimeError(f"WhisperX server returned unrecognized shape: {str(data)[:300]}")
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user