Streamer mix outputs (PR1): one stream bus → a 2nd output device (#49)

* feat(audio): streamer mix outputs — one stream bus to a 2nd output device (PR1)

Built-in routing so a streamer can send a separate mix (game ± their guitar
tone) to a second output device for OBS/Discord capture, while still monitoring
locally — no VoiceMeeter/Reaper. PR1 of the design in docs/streamer-mix-outputs.md.

Architecture: this inverts the engine's proven Phase-2 multi-INPUT-device pattern
to the output side. A new StreamSink = its own AudioDeviceManager + drain callback
+ packed drop-oldest SPSC ring (a mirror of InputDeviceSlot). The PRODUCER is the
main output path (both the duplex callback and the split audioOutputCallback): it
snapshots the guitar monitor mix BEFORE backing is added, then composes the stream
submix (includeGuitar ? guitar : 0) + (includeBacking ? backing : 0) × gain and
packs it into the sink ring. The CONSUMER (streamSinkCallback) drains the ring to
the second device. Backing is rendered once on the master clock and fanned to the
stream ring (never re-advances the transport / touches backingLock). Default off →
zero behaviour change; the sink reopens across restarts (reopenDesiredStreamSink,
mirroring reopenDesiredExtraInputs).

Surface: NodeAddon setStreamOutputDevice/clearStreamOutput/setStreamBus/
setStreamBusGain/getStreamSinkLevel/isStreamOutputActive/getStreamUnderflowCount
→ audio:* IPC → preload → a new "Streaming & Extra Outputs" section on the Audio
page (device picker, game/guitar toggles, gain, a meter mirroring what OBS/Discord
receives; persisted to localStorage). v1 rejects a sample-rate-mismatched sink with
a clear error (async SRC is PR3).

Scope (PR1): ONE stream bus = game ± the guitar monitor mix. Per-source A/B mixes
(re-amped DI vs wet as separate OBS tracks) and per-bus mute that lets a local
monitor-kill (#47) NOT silence the stream are PR2 (see the doc). No virtual driver
shipped — route to a spare output / virtual cable / Go-Live capture.

NOT compiled or run on the author's box — this is native C++ (AudioEngine /
NodeAddon) that needs a desktop build. Renderer JS verified with node --check;
TS bridge/preload are additive (AudioModule is an index type so the calls
typecheck). Draft pending a build + a tester pass (see the PR checklist).

Refs got-feedback/feedBack-desktop#48 (tracking), #46/#47 (audio-engine family).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbexxfTt8q2tAn436MqGWF

* fix(audio): harden streamer-mix sink lifecycle/RT-safety (review on PR #49)

Addresses the P0/P1/P2/P3 findings from the Codex + manual review.

P0 — shutdown UAF: ~AudioEngine() never tore down the stream sink, and
StreamSink declared `manager` before `callback`/`ring`, so the manager
could be destroyed after the callback/ring it drives. stopAudio() now
closes the sink (and the dtor calls stopAudio()), and `manager` is
declared LAST so it destructs first even if a teardown path is missed.

P1 — stopAudio() ignored the sink: the 2nd output device kept running and
underflowing while "stopped". It now closes via closeStreamSinkDevice()
(intent preserved → startAudio() reopens, like extra inputs).

P1 — split-path producer buffers could realloc under a live callback:
streamGuitarScratch/streamMixScratch are now sized to a fixed capacity
(>= the ring) so a same/smaller-block device restart on either clock never
reallocates them mid-use.

P1 — split path read backingBuffer OUTSIDE backingLock (duplex held it):
composeAndPushStreamMix in audioOutputCallback now runs inside the lock
scope, so backingBuffer is read under the lock that guards its resize.

P1 — live setStreamOutputDevice() broke the SPSC single-writer invariant:
streamSinkAboutToStart() resets the ring while the producer might still be
writing. It now clears `active` before reconfiguring so the producer
stops, and only re-arms after a clean open.

P1 — failed open left stale state: a shared `fail()` path now closes the
device and drops the desired intent, so a deterministic failure (e.g. SR
mismatch) isn't retried every start and never reports active with no
device. The renderer keeps its own persisted choice.

P2 — streamSinkStopped() was empty: now marks the sink inactive (and
clears the meter) on an unplanned device loss, preserving intent.

P2 — no ring-capacity guard on the duplex path: composeAndPushStreamMix
skips (and counts) a block larger than the ring instead of wrapping.

P2 — gain NaN/Inf + bridge bool coercion: native sanitizeStreamGain()
(finite, clamped 0..8); the TS bridge requires real booleans (no
Boolean("false")===true) and a finite gain.

P3 — drop-oldest now counted via streamSink.overflowCount, exposed as
getStreamOverflowCount() through the addon/bridge/preload (mirrors
underflow) for drift diagnosis.

Still NOT compiled here (needs a desktop build). TS typechecks clean
(tsc --noEmit); renderer node --check clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(audio): make stream scratch fixed-capacity; document reconfig tail

Follow-up to the review-fix commit, closing the two residual edge cases
from the Codex re-review:

- Producer scratch (streamGuitarScratch/streamMixScratch) is now sized to a
  FIXED capacity == the ring and never grown with the block size. Oversized
  blocks are already skipped by the capacity guard, so a fixed cap is
  sufficient and means the buffers allocate exactly once — they can never
  realloc under a live split-mode producer for ANY later/hotplug block size
  (previously a larger restart block could still realloc).

- Reworded the setStreamOutputDevice() comment to stop overstating the
  active=false barrier: it prevents NEW producer pushes, but a block already
  in flight can finish one push before the (much slower) device reopen drives
  streamSinkAboutToStart's ring reset. Net worst case is one imperfect block
  on the stream bus (never the local monitor) during a manual device switch —
  atomic, no data race, no UAF. Documented as a known PR1 limitation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(audio): count oversized stream blocks (capacity guard before scratch guard)

Codex re-review nit: with the fixed-size scratch (== ring), an oversized
block tripped the undersized-scratch guard first and was dropped without
being counted. Check the ring-capacity guard FIRST so oversized duplex
blocks are always counted as stream overflows; keep the scratch guard
after it as cold-start/reconfig defense.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ChrisBeWithYou <chris@rifflarr.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: byrongamatos <xasiklas@gmail.com>
This commit is contained in:
ChrisBeWithYou
2026-06-29 00:49:25 +02:00
committed by GitHub
co-authored by Claude Opus 4.8 ChrisBeWithYou byrongamatos
parent 0eabbceb73
commit 06c68262a9
8 changed files with 967 additions and 0 deletions
+251
View File
@@ -0,0 +1,251 @@
# Streamer Mix Outputs — Design & PR Plan
**Status:** charrette complete (6-lane panel: audio-engine, compatibility, devops/operability,
sound-design, growth/marketing, streaming-UX) + a Discord-direct fast-follow round.
**Scope:** a built-in, advanced "streamer mix outputs" feature for the FeedBack **desktop** app —
multiple independent audio mixes routed to OBS *and/or* Discord, **without external routing tools**.
**Date:** 2026-06-28
This is desktop-shell-only (native engine + bridge + renderer Audio page). It does not touch the
web renderer or upstream core. All audio logic stays in C++.
---
## TL;DR
1. **This is far less work than it looks.** Every panelist who read the engine independently reached
the same conclusion: the native JUCE engine already solved the genuinely hard real-time pieces for
the *input* side. The feature is largely **"invert Phase-2 to the output side"** — a bus/submix
layer + extra output sinks. The three requested mixes map onto sources that already exist.
2. **The hard part is not mixing — it's getting a mix OUT to OBS/Discord without shipping a driver.**
That's the whole design tension, and it resolves to: route a bus to an output endpoint the capture
tool can already grab, and use **monitor-kill (PR #47)** so the FeedBack process emits *only* the
stream mix.
3. **OBS and Discord are one mechanism with two presets**, not two features. OBS can carry multiple
mixes (separate endpoints/tracks); **Discord is inherently one combined mix** per call.
4. **Ship one stream bus first** (serves both the Discord "play for friends" persona and the OBS
single-feed persona); the multi-mix A/B matrix is OBS-shaped and comes second.
---
## The three target use cases (the brief)
Streamer with an Axe FX III on ASIO — **ASIO in5 = dry DI**, **ASIO in1 = the wet tone** monitored
via their Axe FX *hardware* (≈0 latency to their ears):
1. **LOCAL (me):** hear the **game audio only** (they hear their guitar through their own rig).
2. **STREAM A → viewers:** **game + the DI (in5) re-amped through an in-app NAM/tone** (song-bundled
or independently chosen).
3. **STREAM B → viewers:** **game + the wet hardware tone (in1)**, no NAM.
---
## What the engine already has (verified in `feedback-desktop/src/audio`)
- The native engine **already mixes game + guitar in one domain**: it loads/plays the song backing
track (`AudioEngine`: `loadBackingTrack`/`startBacking`/`backingVolume`/`getBackingLevel`) **and**
processes guitar through a per-input **VST/NAM/IR** chain in `SourceChain`, summing → one output.
- **Dual `AudioDeviceManager`s** (duplex *and* split mode) on **independent clocks**.
- **Lock-free, drift-absorbing SPSC rings** (packed-LR uint64, drop-oldest) — already used to sum up
to **3 extra input devices**, each on its own hardware clock (`extraInputs[]` / `InputDeviceSlot`).
- **`kMaxSources` (8) pooled `SourceChain`s**, each with `selectedInputChannel` (pick ASIO ch 1 vs 5),
`deviceKey` (multi-device), and its own tone chain. → "DI→NAM" and "wet, no-NAM" are **each already
expressible as a source**.
- Safe live teardown handshake (`callbacksInFlight[]`/`pendingRelease[]`), fixed pools, no RT alloc,
`ScopedNoDenormals`.
**The gap (exactly):** one output device; output channels hard-capped at 2; **no bus/submix concept**;
**no master limiter** (the only limiter is inside `BackingLeveler`, on the backing pre-fader).
---
## Core architecture (consensus: audio-engine + devops + UX)
**Mixer = a fixed pool of N Buses. Each Bus = (tap-set + per-tap gain) → one Sink.**
- **Tap-set** = a bitmask over **already-existing** `SourceChain`s + a **backing/game** flag. No new
per-bus DSP — "DI→NAM vs wet" is just *which source* a bus includes.
- **Sink** = `OutputDeviceSlot`, a near-exact mirror of the existing `InputDeviceSlot` (own device
manager + output callback + packed SPSC ring + desired-vs-active intent).
- **Render once, sum N times.** Each source already renders once; buses select subsets. Cost is a
handful of MACs/block (~<1% CPU) — bus count does **not** multiply the NAM/IR/backing cost.
- **The one crux decision (must settle in PR1):** backing currently renders on the *listener's* clock
under `backingLock` and **advances the transport** there. N sinks each rendering backing would
**double-advance the playhead** (a correctness bug). → **Render backing once on the master clock and
fan it via ring to the stream sinks**; keep LOCAL's backing native-clock-pristine. Drop-oldest drift
on OBS/Discord-bound audio is fine.
- **Persistence:** extend `slopsmith-audio-settings.json` with a `buses[]` array (desired intent:
`{name, taps, gains, sinkDevice/channels}`), re-established on startup like
`reopenDesiredExtraInputs()`.
The three use cases map directly:
- **LOCAL** = `{ backing:1, sources:none, sink: primary ASIO/exclusive }` (player hears game; guitar
via their rig; ~0 monitor latency).
- **STREAM A** = `{ backing:1, source: DI(in5)→NAM, sink: stream endpoint }`.
- **STREAM B** = `{ backing:1, source: wet(in1), sink: stream endpoint }`.
---
## The hard part: reaching OBS / Discord with NO external tools
**Universal limit (both OBS App-Audio Capture AND Discord Go-Live):** capture is **per-process** — it
grabs the *whole* FeedBack process audio, **merged**. You cannot isolate two mixes from one process
this way. → **Monitor-kill (PR #47) is the key lever:** make the process emit *only* the stream mix
while the monitor sits on **ASIO/exclusive** (invisible to WASAPI process-capture). Then app-capture /
Go-Live grabs exactly the intended mix, zero install.
### No-driver sink matrix (per OS)
| OS | Best no-driver path | Multiple independent mixes? | Notes |
|---|---|---|---|
| **Windows** | Route bus → a distinct **WASAPI endpoint** (spare interface out / 2nd interface / onboard); OBS *Audio Output Capture* or Discord grabs it. For one mix: **monitor-kill + app-capture/Go-Live**, zero install. | Only with **spare endpoints** (one per mix) | Extra **ASIO** channels are great for *hardware* routing but not software OBS (ASIO exclusive/single-client; `obs-asio` unmaintained on OBS 30+). |
| **macOS** | A cheap **user-space AudioServerPlugin** (BlackHole-class, notarized) → capture/Discord-mic. Or physical channels. | Yes (via the plugin / aggregate) | **Discord screen-share audio is still broken on macOS (2025)** → Mac leans on the virtual device. Cleanest *driver* story (no kernel, no BSOD). |
| **Linux** | **PipeWire null-sinks** at runtime → OBS PipeWire capture / Discord. | **Yes, full N-way, free** | Cleanest platform. Discord screen-share audio works via PipeWire/Pulse (Jan-2025 Wayland); `venmic` for PipeWire-direct. |
### The virtual-device question (a separate, deliberate decision)
A **shipped virtual audio device** is the only universal, no-spare-hardware way to N independent mixes.
But the **Windows** variant is a **signed kernel/APO driver** — EV cert (~$250/yr), Partner Center,
per-Windows-update re-validation, BSOD blast radius, **and anti-cheat (Vanguard/EAC) flags custom audio
drivers** → disqualifying as a default for a practice app. **macOS** AudioServerPlugin is cheap/safe;
**Linux** is free (null-sink). **Recommendation:** do **not** ship a Windows driver as the default —
either recommend an existing vetted cable, or treat the driver as its own scoped project. Ship the
no-driver `extraOutputs[]` path now.
---
## OBS vs Discord — two targets, one mechanism
| | **OBS → Twitch/YouTube** | **Discord (Go Live / Screen Share)** |
|---|---|---|
| Funnel role | **Acquisition** — public, indexed, clippable, social proof to strangers | **Activation / retention / social glue** — friends-only, two-click, high-frequency; warm word-of-mouth; on-ramp to public streaming |
| # mixes it can carry | **Multiple** (separate endpoints → OBS tracks, e.g. Stream A on Track 3, Stream B on Track 4) | **One combined mix** by nature (one call = one screen-share + one mic) |
| Best transport | Audio Output Capture of a routed endpoint (or app-capture + monitor-kill) | **Route A: Go Live "share application audio"** — bypasses Discord's voice DSP, stereo, clean |
| Fidelity | Platform-grade | **Opus voice-grade** — "jam-grade," not audiophile |
**Discord fast-follow specifics:**
- **Route A (recommended):** Go Live → share the FeedBack **application** audio. Bypasses Krisp/AEC/AGC,
stereo, continuous. Windows ✓, macOS ✗ (screen-share audio broken), Linux ✓ (PipeWire/Pulse).
- **Route B (fallback, discouraged):** bus → virtual device → set as Discord **mic**. Runs music
through **Krisp noise-suppression + AEC + AGC + VAD + VOIP-mode Opus + mono fold** — all of which
**destroy music and are uncompensable from our side**. Only if not screen-sharing; requires a
*blocking* "turn OFF Krisp/Noise-Suppression, Echo Cancellation, Automatic Gain" checklist.
- **Discord is one mix only** — the UI must not imply two simultaneous Discord mixes (single-choice
tone radio, disable adding a second Discord bus).
---
## Latency & fidelity truths to SURFACE, not hide (sound-design)
- **The two guitars are the same performance at two delays.** The wet hardware tone (in1, ~0 latency
to the player's ears) and the in-app DI→NAM re-amp (buffered) **must never share a bus** (comb/flange).
**Every bus carries exactly ONE guitar; LOCAL carries ZERO** (verify no leak — strum hard, LOCAL
meter must not move).
- **Stream buses must delay the GAME by Δ = L_in + L_mon** so the guitar locks to the game on the
viewer's feed. **Δ is the same number the scorer already uses** — derive it from the source's
verifier offset (`setVerifierUserOffset`), don't invent a new slider. Add the bus's own chain
latency (needs a new `SignalChain` latency query; NAM≈0 today but a look-ahead VST would break it).
- **Every guitar-carrying bus needs a per-bus ZERO-LOOK-AHEAD limiter** at 1 to 1.5 dBTP (reuse
`BackingLeveler`'s limiter stage, drop its AGC). Look-ahead would re-introduce the desync. The
existing +6 dBFS sanitize scrub is *containment, not safety*.
- **Loudness:** OBS ~16 LUFS / 1.5 dBTP. Discord lower & steadier (~16 to 18, more steady-state
compression, **mono-safe centered guitar**) — give Discord's normalizer nothing to chase.
- **Score off the DI (in5)**, not the distorted wet (in1).
- **Hearing-safety in a VC:** **headphones only** (speakers → howl loop, worse with AEC off); disable
Discord join/leave + notification sounds (hard transients); never sum the VC return into the
protected monitor pre-limiter.
---
## Marketing / growth framing (gamification)
- **Primary headline (community-native):** *"Press Go Live and your friends hear the game and your
guitar — no setup, no extra apps."*
- **Secondary (creator tier):** *"Stream to Twitch with game + your tone in one screen — no VoiceMeeter."*
- **Audience tiers → the three mixes:** pro-rig (BYO wet tone), **mid-tier (re-amped DI = the volume
unlock)**, beginner ("no amp? we'll re-amp you so your stream sounds great").
- **Why streamer-first:** for an instrument game, every stream is a playable demo + social proof +
recruitment. Discord-direct is the **retention/social-glue + warm-lead** engine; OBS is **public
acquisition**. The hard engine work is shared, so it's "and," not "or."
- **Anti-overscope:** the smallest thing that earns the headline = **one separated stream-mix bus**,
monitor stays private. Defer multi-mix, ducking, overlays, clips.
- **Risks:** (1) **support burden** — audio setup is the #1 support sink, and "no external tools"
makes echoes/wrong-device *our* bug → mandatory per-bus meter + "what OBS/Discord hears" preview;
(2) **fidelity expectations** — Discord is Opus/"jam-grade," never market "audiophile over Discord";
(3) **ToS/licensing optics** — making it frictionless to broadcast **bundled/charted copyrighted
song audio** is the DMCA/Content-ID zone. **Default the stream mix to game-backing + the player's
own playing; never market "stream your favorite songs."** Deserves Christian's counsel lens.
---
## Ranked PR plan
> Testing reality: native changes need a ~7-min desktop rebuild; this is a Windows box (can't run
> mac/Linux). The **routing/fan-out/teardown logic** gets deterministic C++/JS unit tests (the
> `tests/` harness already does this for chordscorer/multi-source without real devices); device I/O
> stays manual-verify (OBS/Discord capture + meters + soak). All PRs are `feedback-desktop` only.
**PR 1 — MVP: one configurable stream bus → chosen output. ← the single first PR.**
- Refactor the current single output into a reusable `OutputDeviceSlot` (sink 0) as the first commit
(byte-identical), then add **one** extra sink + a minimal bus model (backing + a chosen, optionally
NAM'd source + gain). Producer fans the selected sources+backing into the bus ring; bridge calls
(`addOutputSink`/`setBusTaps`/`setBusGain`/`removeOutputSink`/`getBusMetrics`); persistence;
**per-bus level + underflow meter**. Reject mismatched-SR sinks with a clear error.
- **Pairs with PR #47 (monitor-kill).** Delivers "game + my re-amped DI (or wet) → a chosen output,
separate from what I monitor" — captured by **OBS Audio Output Capture** *or* **Discord Go Live**
(validate the Discord Go-Live path first; it's the simplest, zero extra device on Windows).
- Risk: **medium** (new RT consumer + device lifecycle, but copied from the proven `extraInputs`).
**PR 2 — Full matrix: N buses → N sinks + routing UI.** Per-bus tap selection (card-stack UI primary,
optional 3×3 grid), delivers the OBS 3-way (LOCAL + A + B) simultaneously + **per-bus zero-look-ahead
limiter** + per-bus meters. Mostly UI; RT core proven by PR1.
**PR 3 — Robustness:** per-sink **async SRC** (mismatched rates), **auto-reopen** on mid-stream device
loss, "stream silent for N sec" warning, the `SignalChain` latency query for delay-comp.
**PR 4 (defer) — Polish:** presets ("Play for friends in Discord" / "Game only" / "Re-amp" / "My amp"),
the OBS + Discord setup helpers (literal step copy, "what they hear" preview, Krisp-off checklist on
the mic route), per-bus mute/solo, naming.
**Driver track (separate, deliberate):** macOS AudioServerPlugin (cheap, worth it) + Linux null-sink
(free); **no Windows kernel driver** (recommend an existing cable or scope it as its own project).
Owned by devops + Christian (cost/risk/anti-cheat).
---
## Decisions (LOCKED 2026-06-28)
1. **Backing render topology → SEPARATE copies.** Keep LOCAL backing native-clock-pristine on the
player's own device; fan a *separate* backing render to the stream rings (drop-oldest drift on
OBS/Discord-bound audio is fine). Monitor never inherits stream compromises; no double-advanced
playhead.
2. **Same-interface routing → RING-FREE direct fan-out.** When buses share one interface (one clock),
sum straight through — sample-accurate, lowest latency, no inter-bus ring. Rings only for
genuinely separate devices / clocks.
3. **Discord surface → BOTH layers.** A destination *type* in the bus model (Discord is just another
"send this mix to ___" target) AND a friendly "Play for friends in Discord" preset on top. Not two
engines.
4. **Virtual audio device → MAC ONLY; NO Windows driver.**
- **macOS:** ship our own user-space virtual device (BlackHole-class, notarized) as an **optional,
substitutable** output — seamless out-of-box; user can disable it and use their own.
- **Windows:** **do NOT build/ship a driver.** The default is no-cable: OBS App-Audio Capture /
Discord Go-Live + **monitor-kill (#47)** → one stream mix, any interface, no spare output. Spare
output / 2nd interface routes a mix for OBS (and separate A/B). The "separate mix with no spare
output" niche uses the user's own free cable. A FeedBack-built Windows driver is rejected
(signed system driver = signing cost + anti-cheat flags + BSOD risk + per-update maintenance).
- **Reasoning correction baked in:** the Windows cable is niche because the **no-cable app-capture
path covers the common single-mix case** — NOT because "most users have multi-out ASIO" (many are
on 2-out interfaces with no spare). And note **spare ASIO outs can't cleanly feed OBS on the same
PC** (ASIO exclusive/single-client; obs-asio dead on OBS 30+) — they feed *hardware* capture / a
2nd PC, not same-machine OBS. So bring-your-own-cable everywhere works; we build only the Mac one.
---
## UI sketch (streaming-UX) — per-bus card stack (primary), grid on reveal
Lives in a **collapsed "Streaming & Extra Outputs"** section at the bottom of the Audio page (invisible
to the 95% who don't stream). A 3-question wizard forks first on **destination** (OBS / Discord / Both).
Buses shown as cards (🔵 "What you hear" / 🔴 "Stream → OBS" / 🟣 "Discord mix"), each with source
rows + level + a meter labeled *"this is exactly what OBS/Discord hears."* Presets cover the use cases
by name; an "Advanced routing" reveal exposes the full source×bus grid. Per-bus **OBS setup** / **Discord
setup** drawers print the literal capture steps for the route that bus targets, with a gentle test tone
+ mirrored meter as the "it's working" signal.
+323
View File
@@ -1202,6 +1202,10 @@ void AudioEngine::startAudio()
// them but kept the intent). This is what makes a stop/start cycle or a device // them but kept the intent). This is what makes a stop/start cycle or a device
// reconfigure transparently resume multi-input detection. // reconfigure transparently resume multi-input detection.
reopenDesiredExtraInputs(); reopenDesiredExtraInputs();
// Restore the streamer-mix output device too (same intent-survives-restart
// pattern). Best-effort — a failure leaves the sink inactive, never blocks.
reopenDesiredStreamSink();
} }
void AudioEngine::stopAudio() void AudioEngine::stopAudio()
@@ -1224,6 +1228,12 @@ void AudioEngine::stopAudio()
// are bound — the single-device path is unchanged. // are bound — the single-device path is unchanged.
for (int dk = 1; dk <= kMaxExtraInputDevices; ++dk) for (int dk = 1; dk <= kMaxExtraInputDevices; ++dk)
closeExtraInputDevice(dk - 1); closeExtraInputDevice(dk - 1);
// Tear down the streamer-mix OUTPUT device too — KEEP its desired intent so the
// next startAudio() reopens it (same pattern as extra inputs). Without this the
// 2nd output device keeps running and underflowing while the engine is "stopped",
// and the destructor (which calls stopAudio()) would be the only path that ever
// closes it — closing the device here also makes that destructor teardown safe.
closeStreamSinkDevice();
audioRunning.store(false, std::memory_order_relaxed); audioRunning.store(false, std::memory_order_relaxed);
currentBackingLevel.store(0.0f); currentBackingLevel.store(0.0f);
} }
@@ -1739,6 +1749,18 @@ void AudioEngine::audioDeviceAboutToStart(juce::AudioIODevice* device)
if (sourceMonitorScratch.getNumSamples() < bs) if (sourceMonitorScratch.getNumSamples() < bs)
sourceMonitorScratch.setSize(2, bs, false, false, true); sourceMonitorScratch.setSize(2, bs, false, false, true);
// Producer-side stream-mix scratch (duplex path runs here). Sized to a FIXED
// capacity equal to the ring and NEVER grown with the block size: in split mode
// the OUTPUT callback is the producer that uses these buffers while THIS (input)
// about-to-start can fire on a hotplug/resume — a block-size-driven realloc here
// would free memory out from under the live producer. A block larger than the
// ring can't be published anyway and is skipped by the capacity guard in
// composeAndPushStreamMix(), so a fixed cap is sufficient AND allocates exactly
// once: it can never realloc under a live producer, for any later block size.
constexpr int streamScratchCap = (int) kOutputRingFrames;
if (streamGuitarScratch.getNumSamples() < streamScratchCap) streamGuitarScratch.setSize(2, streamScratchCap, false, false, true);
if (streamMixScratch.getNumSamples() < streamScratchCap) streamMixScratch.setSize(2, streamScratchCap, false, false, true);
// Prepare each ACTIVE PRIMARY-device source's DSP and reset its rings for a // Prepare each ACTIVE PRIMARY-device source's DSP and reset its rings for a
// clean cold start. Inactive pooled chains stay unprepared (no threads). EXTRA- // clean cold start. Inactive pooled chains stay unprepared (no threads). EXTRA-
// device sources (deviceKey > 0) are prepared by their own extraInputAboutToStart // device sources (deviceKey > 0) are prepared by their own extraInputAboutToStart
@@ -1815,6 +1837,15 @@ void AudioEngine::audioOutputAboutToStart(juce::AudioIODevice* device)
if ((int) outputPullScratchL.size() < bs) outputPullScratchL.assign((size_t) bs, 0.0f); if ((int) outputPullScratchL.size() < bs) outputPullScratchL.assign((size_t) bs, 0.0f);
if ((int) outputPullScratchR.size() < bs) outputPullScratchR.assign((size_t) bs, 0.0f); if ((int) outputPullScratchR.size() < bs) outputPullScratchR.assign((size_t) bs, 0.0f);
// Producer-side stream-mix scratch (split path composes the stream submix in
// this output callback). Fixed capacity == the ring, never grown — a block
// restart on EITHER clock can't realloc these under a live producer, for any
// block size (oversized blocks are skipped, not buffered). See the matching
// note in audioDeviceAboutToStart().
constexpr int streamScratchCap = (int) kOutputRingFrames;
if (streamGuitarScratch.getNumSamples() < streamScratchCap) streamGuitarScratch.setSize(2, streamScratchCap, false, false, true);
if (streamMixScratch.getNumSamples() < streamScratchCap) streamMixScratch.setSize(2, streamScratchCap, false, false, true);
// NOTE: outputBackingBuffer is sized by audioDeviceAboutToStart() from the // NOTE: outputBackingBuffer is sized by audioDeviceAboutToStart() from the
// INPUT device's block size — it's the split-input DSP scratch, not an // INPUT device's block size — it's the split-input DSP scratch, not an
// output-side buffer. Don't touch it here: resizing from the output // output-side buffer. Don't touch it here: resizing from the output
@@ -1875,6 +1906,257 @@ void AudioEngine::audioOutputStopped()
// ring invariants being re-established, which is harder to reason about. // ring invariants being re-established, which is harder to reason about.
} }
// ── Streamer mix output sink (PR1) ──────────────────────────────────────────
// Producer (primary/output callback): compose the stream submix and pack it into
// the sink's ring. Consumer (streamSinkCallback): drain + write to the device.
// Mirrors the InputDeviceSlot ring discipline, inverted to the output side.
void AudioEngine::composeAndPushStreamMix(const juce::AudioBuffer<float>& guitarMix,
const juce::AudioBuffer<float>* backingBuf,
int backingFrames, float backingVol, int numSamples)
{
if (! streamSink.active.load(std::memory_order_acquire)) return;
// A block larger than the entire ring can't be published atomically (it would
// wrap and overwrite unread slots before writeIndex is bumped). Skip it and
// count an overflow. Checked FIRST (before the scratch guard) so an oversized
// block is always counted — the fixed-size scratch is exactly the ring, so an
// oversized block also trips the scratch guard below and would otherwise be
// dropped silently. The split path already rejects oversized devices at setup;
// this guards the duplex path, whose block size we don't pre-validate.
if (numSamples > (int) kOutputRingFrames)
{
streamSink.overflowCount.fetch_add(1, std::memory_order_relaxed);
return;
}
// Scratch not yet sized to the full ring (cold start before the producer's
// about-to-start ran, or a transient reconfig) — skip rather than alloc on RT.
// After warm-up the scratch is exactly the ring, so for an in-range block this
// never trips.
if (streamMixScratch.getNumSamples() < numSamples) return;
const bool ig = streamBusIncludeGuitar.load(std::memory_order_relaxed);
const bool ib = streamBusIncludeBacking.load(std::memory_order_relaxed);
const float gain = streamBusGain.load(std::memory_order_relaxed);
streamMixScratch.clear(0, 0, numSamples);
streamMixScratch.clear(1, 0, numSamples);
if (ig)
for (int ch = 0; ch < 2; ++ch)
streamMixScratch.addFrom(ch, 0, guitarMix,
juce::jmin(ch, guitarMix.getNumChannels() - 1), 0, numSamples);
if (ib && backingBuf != nullptr && backingFrames > 0)
{
const int n = juce::jmin(backingFrames, numSamples);
for (int ch = 0; ch < 2; ++ch)
streamMixScratch.addFrom(ch, 0, *backingBuf,
juce::jmin(ch, backingBuf->getNumChannels() - 1), 0, n, backingVol);
}
streamMixScratch.applyGain(0, 0, numSamples, gain);
streamMixScratch.applyGain(1, 0, numSamples, gain);
const float peak = juce::jmax(streamMixScratch.getMagnitude(0, 0, numSamples),
streamMixScratch.getMagnitude(1, 0, numSamples));
streamSinkLevel.store(peak, std::memory_order_relaxed);
packStereoIntoRing(streamMixScratch, numSamples, streamSink.ring, streamSink.writeIndex);
}
void AudioEngine::streamSinkCallback(float* const* outputData, int numOutputChannels, int numSamples)
{
const juce::ScopedNoDenormals noDenormals;
if (numOutputChannels <= 0) return;
juce::AudioBuffer<float> buffer(outputData, numOutputChannels, numSamples);
buffer.clear();
constexpr uint64_t kMask = (uint64_t) kOutputRingFrames - 1;
constexpr uint64_t kCap = (uint64_t) kOutputRingFrames;
const int scratchCap = (int) streamSink.pullScratchL.size();
const int outSamples = juce::jmin(numSamples, scratchCap);
uint64_t r = streamSink.readIndex.load(std::memory_order_relaxed);
const uint64_t w = streamSink.writeIndex.load(std::memory_order_acquire);
if (w < r) { r = w; streamSink.readIndex.store(r, std::memory_order_relaxed); }
if ((w - r) > kCap)
{
r = w - kCap;
streamSink.readIndex.store(r, std::memory_order_relaxed);
streamSink.overflowCount.fetch_add(1, std::memory_order_relaxed);
}
const uint64_t available = w - r;
const int pullCount = juce::jmin(outSamples, (int) available);
const int consumeCount = juce::jmin(numSamples, (int) available);
const int copyChannels = juce::jmin(numOutputChannels, 2);
for (int i = 0; i < pullCount; ++i)
{
const uint64_t slot = (r + (uint64_t) i) & kMask;
float l, rr;
unpackLR(streamSink.ring[(size_t) slot].load(std::memory_order_relaxed), l, rr);
buffer.setSample(0, i, l);
if (copyChannels > 1) buffer.setSample(1, i, rr);
}
if (pullCount < outSamples)
streamSink.underflowCount.fetch_add(1, std::memory_order_relaxed);
streamSink.readIndex.store(r + (uint64_t) consumeCount, std::memory_order_release);
}
void AudioEngine::streamSinkAboutToStart(juce::AudioIODevice* device)
{
if (device == nullptr) return;
const int bs = device->getCurrentBufferSizeSamples();
double sr = device->getCurrentSampleRate();
if (sr <= 0.0) sr = currentSampleRate.load(std::memory_order_relaxed);
streamSink.blockSize.store(bs, std::memory_order_relaxed);
streamSink.sampleRate.store(sr, std::memory_order_relaxed);
const int cap = juce::jmax(bs, 2048);
if ((int) streamSink.pullScratchL.size() < cap) streamSink.pullScratchL.assign((size_t) cap, 0.0f);
if ((int) streamSink.pullScratchR.size() < cap) streamSink.pullScratchR.assign((size_t) cap, 0.0f);
streamSink.writeIndex.store(0, std::memory_order_relaxed);
streamSink.readIndex.store(0, std::memory_order_relaxed);
streamSink.underflowCount.store(0, std::memory_order_relaxed);
streamSink.overflowCount.store(0, std::memory_order_relaxed);
for (auto& v : streamSink.ring) v.store(0, std::memory_order_relaxed);
}
void AudioEngine::streamSinkStopped()
{
// Fires on any stop of the stream device — an unplanned loss (unplug / driver
// reset) as well as our own teardown. Mark inactive so the producer stops
// filling a now-consumer-less ring and the meter clears; desiredTypeName/Name
// are deliberately left intact so reopenDesiredStreamSink() can restore it.
streamSink.active.store(false, std::memory_order_release);
streamSinkLevel.store(0.0f, std::memory_order_relaxed);
}
juce::String AudioEngine::setStreamOutputDevice(const juce::String& typeName, const juce::String& deviceName)
{
// Control-thread only. Opens an OUTPUT-only device on the stream sink's own
// AudioDeviceManager and attaches the drain callback. Mirrors applySplitSetup's
// output open. v1 requires the sink's nominal SR to match the engine rate (no
// async resampler yet — that's PR3); a mismatch is rejected with a clear error.
streamSink.desiredTypeName = typeName;
streamSink.desiredDeviceName = deviceName;
// Stop the producer from writing the ring while we (re)configure the device:
// setAudioDeviceSetup() below drives streamSinkAboutToStart(), which resets the
// ring indices and slots. Clearing `active` first stops the main callback from
// STARTING new pushes. A producer block already past the active-check can still
// finish one push, but the device close/reopen takes far longer than a single
// audio block, so that in-flight push completes well before about-to-start runs.
// Worst case is therefore one imperfect block on the STREAM bus (never the local
// monitor) during a manual device switch — atomic, no data race, no UAF. We only
// re-arm `active` after a fully clean open.
streamSink.active.store(false, std::memory_order_release);
// Any failure below: detach/close the half-open device AND drop the desired
// intent. A deterministic failure (e.g. SR mismatch) is then NOT retried on every
// startAudio(), and the engine never reports active with no device behind it. The
// renderer keeps its own persisted choice and re-applies it, so nothing is lost.
auto fail = [this](const juce::String& msg) -> juce::String {
closeStreamSinkDevice();
streamSink.desiredTypeName = {};
streamSink.desiredDeviceName = {};
return msg;
};
if (! streamSink.initialised)
{
streamSink.manager.initialise(0, 2, nullptr, false);
streamSink.initialised = true;
}
juce::AudioIODeviceType* outType = nullptr;
for (auto* t : streamSink.manager.getAvailableDeviceTypes())
if (t->getTypeName() == typeName) { outType = t; break; }
if (! outType) return fail("Stream output device type not found: " + typeName);
try {
if (auto* cur = streamSink.manager.getCurrentDeviceTypeObject())
{
if (cur->getTypeName() != typeName)
streamSink.manager.setCurrentAudioDeviceType(typeName, true);
}
else streamSink.manager.setCurrentAudioDeviceType(typeName, true);
} catch (...) { return fail("setCurrentAudioDeviceType threw for stream output type '" + typeName + "'"); }
juce::String resolved = deviceName;
if (resolved.isEmpty())
{
auto names = outType->getDeviceNames(false);
if (names.size() > 0) resolved = names[0];
}
juce::AudioDeviceManager::AudioDeviceSetup setup;
setup.inputDeviceName = "";
setup.outputDeviceName = resolved;
setup.sampleRate = currentSampleRate.load(std::memory_order_relaxed);
setup.bufferSize = outputBlockSize.load(std::memory_order_relaxed);
setup.useDefaultInputChannels = false;
setup.useDefaultOutputChannels = false;
setup.inputChannels.clear();
setup.outputChannels.setRange(0, 2, true);
juce::String err;
try { err = streamSink.manager.setAudioDeviceSetup(setup, true); }
catch (...) { return fail("stream output setAudioDeviceSetup threw"); }
if (err.isNotEmpty()) return fail("stream output setup: " + err);
auto* dev = streamSink.manager.getCurrentAudioDevice();
if (! dev) return fail("no stream output device after setup");
const double devSr = dev->getCurrentSampleRate();
const double engineSr = currentSampleRate.load(std::memory_order_relaxed);
if (engineSr > 0.0 && std::abs(devSr - engineSr) > 0.5)
return fail("Stream output sample rate (" + juce::String(devSr)
+ ") must match the engine rate (" + juce::String(engineSr)
+ "). Pick a device that supports " + juce::String(engineSr) + " Hz.");
streamSink.callback.engine = this;
if (! streamSink.callbackRegistered)
{
streamSink.manager.addAudioCallback(&streamSink.callback);
streamSink.callbackRegistered = true;
}
streamSink.active.store(true, std::memory_order_release);
fprintf(stderr, "[AudioEngine] stream output active: %s (%s)\n",
resolved.toRawUTF8(), typeName.toRawUTF8());
return {};
}
void AudioEngine::closeStreamSinkDevice()
{
// Detach the drain callback and close the device, leaving desiredTypeName/Name
// intact so startAudio()/reopenDesiredStreamSink() can restore it. active=false
// first so the producer stops pushing; removeAudioCallback() then blocks until
// the consumer callback is no longer in flight, so the ring/device go quiescent
// before close. Idempotent — safe when nothing is open.
streamSink.active.store(false, std::memory_order_release);
if (streamSink.callbackRegistered)
{
streamSink.manager.removeAudioCallback(&streamSink.callback);
streamSink.callbackRegistered = false;
}
try { streamSink.manager.closeAudioDevice(); } catch (...) {}
streamSinkLevel.store(0.0f, std::memory_order_relaxed);
}
void AudioEngine::clearStreamOutput()
{
closeStreamSinkDevice();
streamSink.desiredTypeName = {};
streamSink.desiredDeviceName = {};
}
void AudioEngine::reopenDesiredStreamSink()
{
// Copy the intent first: setStreamOutputDevice() mutates desiredTypeName/Name
// (and clears them on failure), so don't pass the members in by reference.
const juce::String t = streamSink.desiredTypeName;
const juce::String d = streamSink.desiredDeviceName;
if (d.isEmpty() && t.isEmpty()) return;
const juce::String err = setStreamOutputDevice(t, d);
if (err.isNotEmpty())
fprintf(stderr, "[AudioEngine] reopenDesiredStreamSink failed: %s\n", err.toRawUTF8());
}
void AudioEngine::audioDeviceIOCallbackWithContext( void AudioEngine::audioDeviceIOCallbackWithContext(
const float* const* inputData, int numInputChannels, const float* const* inputData, int numInputChannels,
float* const* outputData, int numOutputChannels, float* const* outputData, int numOutputChannels,
@@ -1942,11 +2224,23 @@ void AudioEngine::audioDeviceIOCallbackWithContext(
// Split defers all three to OutputCallback (output device's clock). // Split defers all three to OutputCallback (output device's clock).
if (duplex) if (duplex)
{ {
// Stream sink (producer): snapshot the guitar monitor mix BEFORE backing is
// added, so the stream submix can carry guitar independent of the local mix.
const bool streamActive = streamSink.active.load(std::memory_order_acquire);
int streamBackingFrames = 0;
float streamBackingVol = 0.0f;
bool streamBackingOn = false;
if (streamActive && streamGuitarScratch.getNumSamples() >= numSamples)
for (int ch = 0; ch < 2; ++ch)
streamGuitarScratch.copyFrom(ch, 0, buffer,
juce::jmin(ch, buffer.getNumChannels() - 1), 0, numSamples);
const juce::ScopedTryLock sl(backingLock); const juce::ScopedTryLock sl(backingLock);
if (sl.isLocked() && backingTransport && backingPlaying.load()) if (sl.isLocked() && backingTransport && backingPlaying.load())
{ {
const int outSamples = renderBackingBlockLocked(numSamples); const int outSamples = renderBackingBlockLocked(numSamples);
const float bVol = backingVolume.load(); const float bVol = backingVolume.load();
streamBackingFrames = outSamples; streamBackingVol = bVol; streamBackingOn = true;
const int mixChannels = juce::jmin(numOutputChannels, 2); const int mixChannels = juce::jmin(numOutputChannels, 2);
float backingLevelSq = 0.0f; float backingLevelSq = 0.0f;
for (int ch = 0; ch < mixChannels; ++ch) for (int ch = 0; ch < mixChannels; ++ch)
@@ -1971,6 +2265,13 @@ void AudioEngine::audioDeviceIOCallbackWithContext(
currentBackingLevel.store(0.0f); currentBackingLevel.store(0.0f);
} }
// Stream sink: compose + push the stream submix (guitar snapshot + backing)
// BEFORE the local master output gain, so the stream level is independent.
if (streamActive)
composeAndPushStreamMix(streamGuitarScratch,
streamBackingOn ? &backingBuffer : nullptr,
streamBackingFrames, streamBackingVol, numSamples);
// Apply output gain // Apply output gain
buffer.applyGain(outputGain.load()); buffer.applyGain(outputGain.load());
@@ -2576,6 +2877,17 @@ void AudioEngine::audioOutputCallback(const float* const* /*inputData*/,
s.readIndex.store(er + (uint64_t) eConsume, std::memory_order_release); s.readIndex.store(er + (uint64_t) eConsume, std::memory_order_release);
} }
// Stream sink (producer, split clock): snapshot the full guitar mix (primary
// ring + extra inputs) BEFORE backing is added.
const bool streamActive = streamSink.active.load(std::memory_order_acquire);
int streamBackingFrames = 0;
float streamBackingVol = 0.0f;
bool streamBackingOn = false;
if (streamActive && streamGuitarScratch.getNumSamples() >= numSamples)
for (int ch = 0; ch < 2; ++ch)
streamGuitarScratch.copyFrom(ch, 0, buffer,
juce::jmin(ch, buffer.getNumChannels() - 1), 0, numSamples);
{ {
const juce::ScopedTryLock sl(backingLock); const juce::ScopedTryLock sl(backingLock);
if (sl.isLocked() && backingTransport && backingPlaying.load()) if (sl.isLocked() && backingTransport && backingPlaying.load())
@@ -2583,6 +2895,7 @@ void AudioEngine::audioOutputCallback(const float* const* /*inputData*/,
// Shared with the duplex path so the two callbacks can't drift. // Shared with the duplex path so the two callbacks can't drift.
const int backingOut = renderBackingBlockLocked(numSamples); const int backingOut = renderBackingBlockLocked(numSamples);
const float bVol = backingVolume.load(); const float bVol = backingVolume.load();
streamBackingFrames = backingOut; streamBackingVol = bVol; streamBackingOn = true;
// RMS, computed identically to the duplex path so getBackingLevel() // RMS, computed identically to the duplex path so getBackingLevel()
// reports the same metric regardless of which device clock is active. // reports the same metric regardless of which device clock is active.
// copyChannels is already capped at 2, so it doubles as the mix count. // copyChannels is already capped at 2, so it doubles as the mix count.
@@ -2608,6 +2921,16 @@ void AudioEngine::audioOutputCallback(const float* const* /*inputData*/,
{ {
currentBackingLevel.store(0.0f); currentBackingLevel.store(0.0f);
} }
// Stream sink: compose + push the stream submix before the local master
// gain. Done INSIDE the backingLock scope so backingBuffer is read under the
// very lock that guards its resize (audio*AboutToStart) — matching the duplex
// path. When the tryLock failed (streamBackingOn=false) we pass a null backing
// pointer and never touch backingBuffer, so there is nothing to protect.
if (streamActive)
composeAndPushStreamMix(streamGuitarScratch,
streamBackingOn ? &backingBuffer : nullptr,
streamBackingFrames, streamBackingVol, numSamples);
} }
buffer.applyGain(outputGain.load()); buffer.applyGain(outputGain.load());
+100
View File
@@ -231,6 +231,31 @@ public:
float getBackingLevel() const { return currentBackingLevel.load(); } float getBackingLevel() const { return currentBackingLevel.load(); }
void resetPeaks(); void resetPeaks();
// ── Streamer mix output (PR1: one stream bus → one extra output device) ───
// An ADDITIONAL output device carrying an independent submix (game/backing +
// the guitar monitor mix) for OBS/Discord capture, separate from the local
// monitor output. Default off → zero behaviour change. Control-thread only.
// setStreamOutputDevice returns "" on success or an error string.
juce::String setStreamOutputDevice(const juce::String& typeName, const juce::String& deviceName);
void clearStreamOutput();
bool isStreamOutputActive() const { return streamSink.active.load(std::memory_order_acquire); }
juce::String getStreamOutputDeviceName() const { return streamSink.desiredDeviceName; }
// Bus content: include the backing/game, include the guitar monitor mix, and a
// linear output gain. All atomic — safe to set live. Gain is sanitised
// (finite, clamped 0..8) so a NaN/Inf from JS can never reach the stream ring.
void setStreamBus(bool includeBacking, bool includeGuitar, float gain)
{
streamBusIncludeBacking.store(includeBacking, std::memory_order_relaxed);
streamBusIncludeGuitar.store(includeGuitar, std::memory_order_relaxed);
streamBusGain.store(sanitizeStreamGain(gain), std::memory_order_relaxed);
}
void setStreamBusGain(float gain) { streamBusGain.store(sanitizeStreamGain(gain), std::memory_order_relaxed); }
float getStreamSinkLevel() const { return streamSinkLevel.load(std::memory_order_relaxed); }
uint64_t getStreamUnderflowCount() const { return streamSink.underflowCount.load(std::memory_order_relaxed); }
// Producer overflow (drop-oldest): the consumer fell a full ring behind and
// frames were skipped. Exposed alongside underflow for stream drift diagnosis.
uint64_t getStreamOverflowCount() const { return streamSink.overflowCount.load(std::memory_order_relaxed); }
// Latency // Latency
double getLatencyMs() const; double getLatencyMs() const;
@@ -614,5 +639,80 @@ private:
std::array<std::atomic<uint64_t>, kOutputRingFrames>& ring, std::array<std::atomic<uint64_t>, kOutputRingFrames>& ring,
std::atomic<uint64_t>& writeIndex); std::atomic<uint64_t>& writeIndex);
// ── Streamer mix output sink (PR1) ───────────────────────────────────────
// A second OUTPUT AudioDeviceManager on its OWN clock that drains a dedicated
// SPSC ring fed by the main output path's composed stream submix. This mirrors
// the InputDeviceSlot pattern INVERTED to the output side: the PRODUCER is the
// primary/output callback (composeAndPushStreamMix), the CONSUMER is this extra
// output device's callback (streamSinkCallback). Default off → no behaviour change.
struct StreamSinkCallback : juce::AudioIODeviceCallback
{
AudioEngine* engine = nullptr;
void audioDeviceIOCallbackWithContext(const float* const* inputData, int numInputChannels,
float* const* outputData, int numOutputChannels,
int numSamples,
const juce::AudioIODeviceCallbackContext&) override
{
juce::ignoreUnused(inputData, numInputChannels);
if (engine) engine->streamSinkCallback(outputData, numOutputChannels, numSamples);
}
void audioDeviceAboutToStart(juce::AudioIODevice* d) override { if (engine) engine->streamSinkAboutToStart(d); }
void audioDeviceStopped() override { if (engine) engine->streamSinkStopped(); }
};
struct StreamSink
{
StreamSinkCallback callback;
std::array<std::atomic<uint64_t>, kOutputRingFrames> ring{};
std::atomic<uint64_t> writeIndex{0};
std::atomic<uint64_t> readIndex{0};
std::atomic<uint64_t> underflowCount{0};
std::atomic<uint64_t> overflowCount{0};
std::atomic<bool> active{false};
std::atomic<double> sampleRate{48000.0};
std::atomic<int> blockSize{256};
std::vector<float> pullScratchL, pullScratchR; // sized in streamSinkAboutToStart
bool callbackRegistered = false;
bool initialised = false;
// Declared LAST so it DESTRUCTS FIRST (members tear down in reverse
// declaration order): the manager's dtor closes the device and detaches
// `callback` while `callback`/`ring` are still alive — no use-after-free
// even if an explicit teardown path is ever missed. stopAudio() /
// closeStreamSinkDevice() also tear it down explicitly before this.
juce::AudioDeviceManager manager;
// Persistent INTENT (control thread only): the device the user chose.
// Survives a stop/restart so reopenDesiredStreamSink() can re-open it.
juce::String desiredTypeName;
juce::String desiredDeviceName;
};
StreamSink streamSink;
std::atomic<bool> streamBusIncludeBacking{true};
std::atomic<bool> streamBusIncludeGuitar{true};
std::atomic<float> streamBusGain{1.0f};
std::atomic<float> streamSinkLevel{0.0f};
// Producer-side scratch (written by the primary/output callback): the guitar
// monitor-mix snapshot (pre-backing) and the composed stream submix. Sized in
// audioDeviceAboutToStart / audioOutputAboutToStart alongside the other scratch.
juce::AudioBuffer<float> streamGuitarScratch;
juce::AudioBuffer<float> streamMixScratch;
// Clamp a requested stream gain to a finite, sane range so a NaN/Inf (or a
// wild value) from the JS bridge can never be packed into the stream ring.
static float sanitizeStreamGain(float g) { return std::isfinite(g) ? juce::jlimit(0.0f, 8.0f, g) : 0.0f; }
void streamSinkCallback(float* const* outputData, int numOutputChannels, int numSamples);
void streamSinkAboutToStart(juce::AudioIODevice* device);
void streamSinkStopped();
void reopenDesiredStreamSink();
// Detach + close the stream-sink device but KEEP desiredTypeName/Name, so a
// stopAudio()/startAudio() cycle re-opens it (intent survives, like extra
// inputs). Also the single teardown used by the dtor and clearStreamOutput().
void closeStreamSinkDevice();
// Compose the stream submix from the captured guitar mix + the just-rendered
// backing block and pack it into the stream ring. Called from both output
// callbacks after backing render. `backingBuf` may be null (not playing).
void composeAndPushStreamMix(const juce::AudioBuffer<float>& guitarMix,
const juce::AudioBuffer<float>* backingBuf,
int backingFrames, float backingVol, int numSamples);
JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR(AudioEngine) JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR(AudioEngine)
}; };
+81
View File
@@ -1318,6 +1318,79 @@ static Napi::Value UnbindInputDevice(const Napi::CallbackInfo& info)
return Napi::Boolean::New(env, liveEngine->unbindInputDevice(info[0].As<Napi::Number>().Int32Value())); return Napi::Boolean::New(env, liveEngine->unbindInputDevice(info[0].As<Napi::Number>().Int32Value()));
} }
// ── Streamer mix output (PR1) ───────────────────────────────────────────────
// setStreamOutputDevice(typeName, deviceName) -> "" on success, else an error.
static Napi::Value SetStreamOutputDevice(const Napi::CallbackInfo& info)
{
auto env = info.Env();
auto liveEngine = snapshotEngine();
if (!liveEngine) return Napi::String::New(env, "no engine");
if (info.Length() < 2 || !info[0].IsString() || !info[1].IsString())
return Napi::String::New(env, "setStreamOutputDevice(typeName:string, deviceName:string)");
const std::string typeName = info[0].As<Napi::String>().Utf8Value();
const std::string devName = info[1].As<Napi::String>().Utf8Value();
return Napi::String::New(env,
liveEngine->setStreamOutputDevice(juce::String(typeName), juce::String(devName)).toStdString());
}
// clearStreamOutput() -> undefined
static Napi::Value ClearStreamOutput(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
if (liveEngine) liveEngine->clearStreamOutput();
return info.Env().Undefined();
}
// setStreamBus(includeBacking:boolean, includeGuitar:boolean, gain:number)
static Napi::Value SetStreamBus(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
if (liveEngine && info.Length() >= 3 && info[0].IsBoolean() && info[1].IsBoolean() && info[2].IsNumber())
liveEngine->setStreamBus(info[0].As<Napi::Boolean>().Value(),
info[1].As<Napi::Boolean>().Value(),
(float) info[2].As<Napi::Number>().DoubleValue());
return info.Env().Undefined();
}
// setStreamBusGain(gain:number)
static Napi::Value SetStreamBusGain(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
if (liveEngine && info.Length() >= 1 && info[0].IsNumber())
liveEngine->setStreamBusGain((float) info[0].As<Napi::Number>().DoubleValue());
return info.Env().Undefined();
}
// getStreamSinkLevel() -> number (peak 0..1+)
static Napi::Value GetStreamSinkLevel(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
return Napi::Number::New(info.Env(), liveEngine ? liveEngine->getStreamSinkLevel() : 0.0f);
}
// isStreamOutputActive() -> boolean
static Napi::Value IsStreamOutputActive(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
return Napi::Boolean::New(info.Env(), liveEngine ? liveEngine->isStreamOutputActive() : false);
}
// getStreamUnderflowCount() -> number
static Napi::Value GetStreamUnderflowCount(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
return Napi::Number::New(info.Env(),
(double) (liveEngine ? liveEngine->getStreamUnderflowCount() : 0ull));
}
// getStreamOverflowCount() -> number (consumer fell a full ring behind; frames dropped)
static Napi::Value GetStreamOverflowCount(const Napi::CallbackInfo& info)
{
auto liveEngine = snapshotEngine();
return Napi::Number::New(info.Env(),
(double) (liveEngine ? liveEngine->getStreamOverflowCount() : 0ull));
}
// setSourceInputChannel(sourceId, channel) // setSourceInputChannel(sourceId, channel)
static Napi::Value SetSourceInputChannel(const Napi::CallbackInfo& info) static Napi::Value SetSourceInputChannel(const Napi::CallbackInfo& info)
{ {
@@ -3166,6 +3239,14 @@ static Napi::Object InitModule(Napi::Env env, Napi::Object exports)
exports.Set("listInputDevices", Napi::Function::New(env, ListInputDevices)); exports.Set("listInputDevices", Napi::Function::New(env, ListInputDevices));
exports.Set("bindInputDevice", Napi::Function::New(env, BindInputDevice)); exports.Set("bindInputDevice", Napi::Function::New(env, BindInputDevice));
exports.Set("unbindInputDevice", Napi::Function::New(env, UnbindInputDevice)); exports.Set("unbindInputDevice", Napi::Function::New(env, UnbindInputDevice));
exports.Set("setStreamOutputDevice", Napi::Function::New(env, SetStreamOutputDevice));
exports.Set("clearStreamOutput", Napi::Function::New(env, ClearStreamOutput));
exports.Set("setStreamBus", Napi::Function::New(env, SetStreamBus));
exports.Set("setStreamBusGain", Napi::Function::New(env, SetStreamBusGain));
exports.Set("getStreamSinkLevel", Napi::Function::New(env, GetStreamSinkLevel));
exports.Set("isStreamOutputActive", Napi::Function::New(env, IsStreamOutputActive));
exports.Set("getStreamUnderflowCount", Napi::Function::New(env, GetStreamUnderflowCount));
exports.Set("getStreamOverflowCount", Napi::Function::New(env, GetStreamOverflowCount));
exports.Set("setSourceInputChannel", Napi::Function::New(env, SetSourceInputChannel)); exports.Set("setSourceInputChannel", Napi::Function::New(env, SetSourceInputChannel));
exports.Set("setSourceVerifierOffset", Napi::Function::New(env, SetSourceVerifierOffset)); exports.Set("setSourceVerifierOffset", Napi::Function::New(env, SetSourceVerifierOffset));
exports.Set("setSourceMonitorMute", Napi::Function::New(env, SetSourceMonitorMute)); exports.Set("setSourceMonitorMute", Napi::Function::New(env, SetSourceMonitorMute));
+55
View File
@@ -812,6 +812,61 @@ export function initAudioBridge(): void {
} }
}); });
// ── Streamer mix output (PR1) ───────────────────────────────────────────
ipcMain.handle('audio:setStreamOutputDevice', (_event, typeName: unknown, deviceName: unknown) => {
if (!audio || typeof audio.setStreamOutputDevice !== 'function') return 'unsupported';
if (typeof typeName !== 'string' || typeof deviceName !== 'string') return 'invalid arguments';
try {
return audio.setStreamOutputDevice(typeName, deviceName);
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
console.warn(`[audio] setStreamOutputDevice failed: ${msg}`);
return msg;
}
});
ipcMain.handle('audio:clearStreamOutput', () => {
if (audio && typeof audio.clearStreamOutput === 'function') audio.clearStreamOutput();
});
ipcMain.handle('audio:setStreamBus', (_event, includeBacking: unknown, includeGuitar: unknown, gain: unknown) => {
if (audio && typeof audio.setStreamBus === 'function') {
// Require real booleans (don't Boolean()-coerce — that turns the string
// "false" into true) and a finite gain (native also clamps/finite-guards).
const ib = typeof includeBacking === 'boolean' ? includeBacking : true;
const ig = typeof includeGuitar === 'boolean' ? includeGuitar : true;
const g = typeof gain === 'number' && Number.isFinite(gain) ? gain : 1.0;
audio.setStreamBus(ib, ig, g);
}
});
ipcMain.handle('audio:setStreamBusGain', (_event, gain: unknown) => {
if (audio && typeof audio.setStreamBusGain === 'function'
&& typeof gain === 'number' && Number.isFinite(gain)) {
audio.setStreamBusGain(gain);
}
});
ipcMain.handle('audio:getStreamSinkLevel', () => {
if (!audio || typeof audio.getStreamSinkLevel !== 'function') return 0;
return audio.getStreamSinkLevel();
});
ipcMain.handle('audio:isStreamOutputActive', () => {
if (!audio || typeof audio.isStreamOutputActive !== 'function') return false;
return audio.isStreamOutputActive();
});
ipcMain.handle('audio:getStreamUnderflowCount', () => {
if (!audio || typeof audio.getStreamUnderflowCount !== 'function') return 0;
return audio.getStreamUnderflowCount();
});
ipcMain.handle('audio:getStreamOverflowCount', () => {
if (!audio || typeof audio.getStreamOverflowCount !== 'function') return 0;
return audio.getStreamOverflowCount();
});
ipcMain.handle('audio:removeSource', (_event, id: unknown) => { ipcMain.handle('audio:removeSource', (_event, id: unknown) => {
if (!audio || typeof audio.removeSource !== 'function') return false; if (!audio || typeof audio.removeSource !== 'function') return false;
if (!validSourceId(id)) return false; if (!validSourceId(id)) return false;
+13
View File
@@ -349,6 +349,19 @@ const feedBackDesktopApi = {
ipcRenderer.invoke('audio:bindInputDevice', deviceKey, deviceName), ipcRenderer.invoke('audio:bindInputDevice', deviceKey, deviceName),
unbindInputDevice: (deviceKey: number): Promise<boolean> => unbindInputDevice: (deviceKey: number): Promise<boolean> =>
ipcRenderer.invoke('audio:unbindInputDevice', deviceKey), ipcRenderer.invoke('audio:unbindInputDevice', deviceKey),
// Streamer mix output (PR1): a second output device carrying an independent
// game ± guitar submix for OBS/Discord capture.
setStreamOutputDevice: (typeName: string, deviceName: string): Promise<string> =>
ipcRenderer.invoke('audio:setStreamOutputDevice', typeName, deviceName),
clearStreamOutput: (): Promise<void> => ipcRenderer.invoke('audio:clearStreamOutput'),
setStreamBus: (includeBacking: boolean, includeGuitar: boolean, gain: number): Promise<void> =>
ipcRenderer.invoke('audio:setStreamBus', includeBacking, includeGuitar, gain),
setStreamBusGain: (gain: number): Promise<void> =>
ipcRenderer.invoke('audio:setStreamBusGain', gain),
getStreamSinkLevel: (): Promise<number> => ipcRenderer.invoke('audio:getStreamSinkLevel'),
isStreamOutputActive: (): Promise<boolean> => ipcRenderer.invoke('audio:isStreamOutputActive'),
getStreamUnderflowCount: (): Promise<number> => ipcRenderer.invoke('audio:getStreamUnderflowCount'),
getStreamOverflowCount: (): Promise<number> => ipcRenderer.invoke('audio:getStreamOverflowCount'),
setSourceInputChannel: (id: number, channel: number): Promise<void> => setSourceInputChannel: (id: number, channel: number): Promise<void> =>
ipcRenderer.invoke('audio:setSourceInputChannel', id, channel), ipcRenderer.invoke('audio:setSourceInputChannel', id, channel),
setSourceVerifierOffset: (id: number, seconds: number): Promise<void> => setSourceVerifierOffset: (id: number, seconds: number): Promise<void> =>
+52
View File
@@ -125,6 +125,58 @@
</div> </div>
</div> </div>
<!-- Streaming & Extra Outputs (PR1: one stream bus → a second output device for
OBS/Discord capture, independent of your local monitor). Advanced; off by default. -->
<div class="mb-6">
<h3 class="text-lg font-semibold mb-3 text-slate-200">Streaming &amp; Extra Outputs</h3>
<p class="text-xs text-slate-400 mb-3">
Send a separate mix (game &plusmn; your guitar tone) to a second output device that
OBS or Discord can capture &mdash; while you keep monitoring here. Pick a spare output
or a virtual cable; on Discord, share that device or use Go&nbsp;Live.
</p>
<label class="flex items-center gap-2 cursor-pointer mb-3">
<input type="checkbox" id="ae-stream-enable"
class="w-4 h-4 rounded bg-slate-700 border-slate-600 accent-fuchsia-500">
<span class="text-sm text-slate-300">Send a stream mix to a second output</span>
</label>
<div id="ae-stream-config" class="space-y-3" style="display: none;">
<div class="grid grid-cols-1 md:grid-cols-2 gap-3">
<div>
<label class="block text-xs text-slate-400 mb-1">Stream Output Type</label>
<select id="ae-stream-type" class="w-full bg-slate-700 border border-slate-600 rounded px-3 py-2 text-sm"></select>
</div>
<div>
<label class="block text-xs text-slate-400 mb-1">Stream Output Device</label>
<select id="ae-stream-device" class="w-full bg-slate-700 border border-slate-600 rounded px-3 py-2 text-sm"></select>
</div>
</div>
<div class="flex flex-wrap gap-4">
<label class="flex items-center gap-2 cursor-pointer">
<input type="checkbox" id="ae-stream-game" checked
class="w-4 h-4 rounded bg-slate-700 border-slate-600 accent-fuchsia-500">
<span class="text-sm text-slate-300">Include game audio</span>
</label>
<label class="flex items-center gap-2 cursor-pointer">
<input type="checkbox" id="ae-stream-guitar" checked
class="w-4 h-4 rounded bg-slate-700 border-slate-600 accent-fuchsia-500">
<span class="text-sm text-slate-300">Include my guitar tone</span>
</label>
</div>
<div>
<label class="block text-xs text-slate-400 mb-1">Stream level</label>
<div class="h-4 bg-slate-800 rounded overflow-hidden">
<div id="ae-stream-meter" class="h-full bg-fuchsia-500 transition-all duration-75" style="width:0%"></div>
</div>
<div class="flex justify-between mt-1">
<input type="range" id="ae-stream-gain" min="-60" max="12" step="0.1" value="0" class="w-full accent-fuchsia-500">
<span id="ae-stream-gain-label" class="text-xs text-slate-400 ml-2 whitespace-nowrap">0.0 dB</span>
</div>
<p class="text-[11px] text-fuchsia-300/80 mt-1">This is what OBS/Discord will hear.</p>
</div>
<p id="ae-stream-status" class="text-xs text-slate-400"></p>
</div>
</div>
<!-- Noise Gate (AmpliTube-style: threshold, release, depth → native setNoiseGate) --> <!-- Noise Gate (AmpliTube-style: threshold, release, depth → native setNoiseGate) -->
<div class="mb-6"> <div class="mb-6">
<h3 class="text-lg font-semibold mb-3 text-slate-200">Noise Gate</h3> <h3 class="text-lg font-semibold mb-3 text-slate-200">Noise Gate</h3>
+92
View File
@@ -50,6 +50,7 @@ window.__feedBackDesktopAudioHooks = window.__feedBackDesktopAudioHooks || {};
const applyDeviceBtn = $('ae-apply-device'); const applyDeviceBtn = $('ae-apply-device');
const meterInput = $('ae-meter-input'); const meterInput = $('ae-meter-input');
const meterOutput = $('ae-meter-output'); const meterOutput = $('ae-meter-output');
let streamMeterEl = null; // assigned in setupStreaming(); read by the meter poll
const inputGainSlider = $('ae-input-gain'); const inputGainSlider = $('ae-input-gain');
const outputGainSlider = $('ae-output-gain'); const outputGainSlider = $('ae-output-gain');
const inputGainLabel = $('ae-input-gain-label'); const inputGainLabel = $('ae-input-gain-label');
@@ -1063,6 +1064,9 @@ window.__feedBackDesktopAudioHooks = window.__feedBackDesktopAudioHooks || {};
await refreshDeviceOptions(); await refreshDeviceOptions();
registerAudioSessionInputSources(); registerAudioSessionInputSources();
registerAudioSessionMixParticipants(); registerAudioSessionMixParticipants();
// Streamer mix output section (PR1) — wires after device types are known.
setupStreaming();
} }
function updateInputDeviceDropdown(typeInfo) { function updateInputDeviceDropdown(typeInfo) {
@@ -1090,6 +1094,88 @@ window.__feedBackDesktopAudioHooks = window.__feedBackDesktopAudioHooks || {};
updateOutputDeviceDropdown(typeInfo); updateOutputDeviceDropdown(typeInfo);
} }
// ── Streaming & Extra Outputs (PR1) ────────────────────────────────────────
// Wires the stream-output section: a second output device carrying a game ±
// guitar submix for OBS/Discord capture. Persists to localStorage. Desktop-only
// — without the native bridge methods the section disables itself.
function setupStreaming() {
const enable = $('ae-stream-enable');
const config = $('ae-stream-config');
const typeSel = $('ae-stream-type');
const devSel = $('ae-stream-device');
const gameCb = $('ae-stream-game');
const guitarCb = $('ae-stream-guitar');
const gain = $('ae-stream-gain');
const gainLabel = $('ae-stream-gain-label');
const statusEl = $('ae-stream-status');
streamMeterEl = $('ae-stream-meter');
if (!enable || !config || !typeSel || !devSel) return;
if (!api || typeof api.setStreamOutputDevice !== 'function') {
enable.disabled = true; // no native stream support in this build
if (statusEl) statusEl.textContent = 'Not available in this build.';
return;
}
const LS = 'ae.stream';
const load = () => { try { return JSON.parse(localStorage.getItem(LS) || '{}'); } catch (_) { return {}; } };
const save = () => { try { localStorage.setItem(LS, JSON.stringify(st)); } catch (_) {} };
const st = load();
const populateTypes = () => {
typeSel.innerHTML = '';
for (const t of currentDeviceTypes) {
if (!t.outputs || t.outputs.length === 0) continue;
const o = document.createElement('option');
o.value = t.name; o.textContent = t.name;
typeSel.appendChild(o);
}
if (st.type && selectHasValue(typeSel, st.type)) typeSel.value = st.type;
};
const populateDevices = () => {
devSel.innerHTML = '<option value="">Default</option>';
const t = currentDeviceTypes.find(x => x.name === typeSel.value);
for (const name of (t && t.outputs ? t.outputs : [])) {
const o = document.createElement('option');
o.value = name; o.textContent = name;
devSel.appendChild(o);
}
if (st.device && selectHasValue(devSel, st.device)) devSel.value = st.device;
};
const gainLinear = () => dbToLinearGain(parseFloat(gain.value));
const refreshGainLabel = () => { gainLabel.textContent = parseFloat(gain.value).toFixed(1) + ' dB'; };
const applyBus = () => { try { api.setStreamBus(gameCb.checked, guitarCb.checked, gainLinear()); } catch (_) {} };
const applyDevice = async () => {
if (!enable.checked) { try { await api.clearStreamOutput(); } catch (_) {} statusEl.textContent = ''; return; }
statusEl.textContent = 'Opening stream output…';
try {
const err = await api.setStreamOutputDevice(typeSel.value, devSel.value);
statusEl.textContent = err ? ('Error: ' + err)
: 'Stream output active — point OBS/Discord at this device.';
if (!err) applyBus();
} catch (e) { statusEl.textContent = 'Error: ' + (e && e.message ? e.message : String(e)); }
};
populateTypes();
populateDevices();
if (typeof st.game === 'boolean') gameCb.checked = st.game;
if (typeof st.guitar === 'boolean') guitarCb.checked = st.guitar;
if (typeof st.gain === 'number') gain.value = String(st.gain);
refreshGainLabel();
enable.checked = !!st.enabled;
config.style.display = enable.checked ? '' : 'none';
if (enable.checked) applyDevice();
enable.addEventListener('change', () => {
config.style.display = enable.checked ? '' : 'none';
st.enabled = enable.checked; save(); applyDevice();
});
typeSel.addEventListener('change', () => { st.type = typeSel.value; save(); populateDevices(); applyDevice(); });
devSel.addEventListener('change', () => { st.device = devSel.value; save(); applyDevice(); });
gameCb.addEventListener('change', () => { st.game = gameCb.checked; save(); applyBus(); });
guitarCb.addEventListener('change', () => { st.guitar = guitarCb.checked; save(); applyBus(); });
gain.addEventListener('input', () => { refreshGainLabel(); st.gain = parseFloat(gain.value); save(); applyBus(); });
}
// ── Signal Chain ────────────────────────────────────────────────────────── // ── Signal Chain ──────────────────────────────────────────────────────────
async function refreshChain() { async function refreshChain() {
const container = chainContainer || $('ae-chain'); const container = chainContainer || $('ae-chain');
@@ -1200,6 +1286,12 @@ window.__feedBackDesktopAudioHooks = window.__feedBackDesktopAudioHooks || {};
meterInput.style.width = inPct + '%'; meterInput.style.width = inPct + '%';
meterOutput.style.width = outPct + '%'; meterOutput.style.width = outPct + '%';
// Stream-output meter (PR1): mirrors what OBS/Discord receives.
if (streamMeterEl && typeof api.getStreamSinkLevel === 'function') {
try { streamMeterEl.style.width = toMeterPct(await api.getStreamSinkLevel()) + '%'; }
catch (_) { /* ignore */ }
}
// Clipping indicator // Clipping indicator
meterInput.className = levels.inputLevel > 0.95 meterInput.className = levels.inputLevel > 0.95
? 'h-full bg-red-500 transition-all duration-75' ? 'h-full bg-red-500 transition-all duration-75'