fix(audio): promote in-process VST3 to sandbox on editor-open (Windows crash) (#54)

Windows-only crash fix: opening an in-process VST3 editor faults via WndProc on the background message thread. OpenPluginEditor now promotes the slot to the out-of-process sandbox (state transferred via get/setStateInformation) via the new SignalChain::replaceProcessor, and opens the editor there.

Review hardening (multi-angle + Codex): state capture runs under the audio lock + SEH guard (SignalChain::captureVstStateForPromotion) so it can't race processBlock or fault the app; the transient sandbox pin is undone on promotion failure (isCrashedPlugin/removeCrashedPlugin) so a healthy plugin isn't stranded; replaceProcessor stages type/name/path for correct blocklist attribution; shared prepareForPlayback helper.

CI green incl. addon (windows-latest). Editor-open crash repro on a real Windows host still recommended as follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

src/audio/NodeAddon.cpp

@@ -2776,8 +2776,105 @@ static Napi::Value OpenPluginEditor(const Napi::CallbackInfo& info)
     {
         auto liveEngine = snapshotEngine();
         if (!liveEngine) return;
-        auto* slot = liveEngine->getSignalChain().getSlot(slotId);
+        auto& chain = liveEngine->getSignalChain();
+        auto* slot = chain.getSlot(slotId);
         if (!slot || !slot->processor) return;
+
+        // ── Windows editor-crash class fix ───────────────────────────────────
+        // An in-process VST3 editor is created on JUCE's BACKGROUND message
+        // thread (V8 owns the OS main thread inside a Node addon). On Windows a
+        // Qt-using / window-on-init plugin then faults via USER32->WndProc on
+        // WM_ACTIVATEAPP with NO host frame on the stack, so the SignalChain SEH
+        // guard can't catch it and the whole app dies (0xC0000005 / 0xC0000409).
+        // Fix: never open a VST3 editor in-process on Windows — promote the slot
+        // to the out-of-process sandbox (which hosts the editor on a real
+        // top-level message thread, the environment the plugin needs) and open
+        // it there. Compiled on every platform so the swap path keeps building;
+        // gated to Windows at runtime since the in-process editor is fine on
+        // macOS/Linux (no WndProc) and the sandbox hop is pure overhead there.
+        static constexpr bool kPromoteEditorToSandbox =
+           #if JUCE_WINDOWS
+            true;
+           #else
+            false;
+           #endif
+        if (kPromoteEditorToSandbox)
+        {
+            // Decide + snapshot state SAFELY. captureVstStateForPromotion runs
+            // hasEditor()/getStateInformation() under the audio lock and the SEH
+            // guard (see its contract), so they neither race process()'s
+            // processBlock nor fault the app — an UNguarded getStateInformation on
+            // the very plugins this promotion targets would reintroduce the editor
+            // crash on the message thread. It returns true only for a non-sandboxed
+            // in-process VST3 that actually has an editor.
+            juce::MemoryBlock state;
+            if (chain.captureVstStateForPromotion(slotId, state))
+            {
+                const juce::String path = slot->path;   // immutable; message-thread only
+                fprintf(stderr, "[AudioEngine] editor-open: promoting in-process VST3 to sandbox: slot %d '%s'\n",
+                        slotId, path.toRawUTF8());
+
+                juce::PluginDescription desc;
+                desc.fileOrIdentifier = path;
+                desc.name = juce::File(path).getFileNameWithoutExtension();
+
+                // tryLoadSandboxed only accepts a plugin that shouldSandbox()
+                // approves, so pin this path to the runtime sandbox list first.
+                // Remember whether it was ALREADY pinned: if the promotion fails
+                // we undo only OUR pin below, so a healthy, never-crashed plugin
+                // isn't left permanently forced to a sandbox that just proved
+                // unavailable (while a pre-existing/real blocklist entry stays).
+                const bool wasAlreadyPinned = slopsmith::sandbox::isCrashedPlugin(path);
+                slopsmith::sandbox::addCrashedPlugin(path);
+
+                bool promoted = false;
+                juce::String err;
+                auto sandboxed = slopsmith::sandbox::tryLoadSandboxed(
+                    desc, chain.getCurrentSampleRate(), chain.getCurrentBlockSize(), err);
+                if (sandboxed)
+                {
+                    if (state.getSize() > 0)
+                        sandboxed->setStateInformation(state.getData(), (int) state.getSize());
+                    if (chain.replaceProcessor(slotId, std::move(sandboxed)))
+                    {
+                        promoted = true;
+                        bool editorOpened = false;
+                        if (auto* slot2 = chain.getSlot(slotId))
+                            if (auto* sb = dynamic_cast<slopsmith::sandbox::SandboxedProcessor*>(slot2->processor.get()))
+                                editorOpened = sb->requestOpenEditor();
+                        fprintf(stderr, "[AudioEngine] editor-open: sandbox promotion OK for slot %d (editor %s)\n",
+                                slotId, editorOpened ? "opened" : "FAILED to open");
+                    }
+                    else
+                    {
+                        fprintf(stderr, "[AudioEngine] editor-open: replaceProcessor failed for slot %d\n", slotId);
+                    }
+                }
+                else
+                {
+                    fprintf(stderr, "[AudioEngine] editor-open: sandbox promotion failed for '%s': %s\n",
+                            path.toRawUTF8(), err.toRawUTF8());
+                }
+
+                // Undo our transient pin on failure so a plugin that never crashed
+                // isn't stranded on the (evidently unavailable) sandbox route.
+                if (! promoted && ! wasAlreadyPinned)
+                    slopsmith::sandbox::removeCrashedPlugin(path);
+
+                // Promoted or not, never fall through to the in-process editor on
+                // Windows — that is the WndProc/Qt crash path this branch exists
+                // to avoid.
+                return;
+            }
+            // Not promotable (non-VST / editor-less / already-sandboxed, or the
+            // guarded capture faulted and released the processor). Fall through to
+            // the in-process branch below, which is safe for all of those cases
+            // (an already-sandboxed slot opens its editor out-of-process; an
+            // editor-less or released processor simply opens no window).
+        }
+
+        // In-process editor: non-VST3, editor-less, already-sandboxed, or POSIX
+        // (where the in-process editor is safe).
         auto* processor = slot->processor.get();
         auto name = slot->name;
         juce::AudioProcessorEditor* editor = nullptr;

src/audio/Sandbox/SandboxFactory_shared.cpp

@@ -189,6 +189,35 @@ void addCrashedPlugin(const juce::String& pluginPath)
     }
 }
 
+bool isCrashedPlugin(const juce::String& pluginPath)
+{
+    if (pluginPath.isEmpty()) return false;
+    const auto canonical = juce::File(pluginPath).getFullPathName();
+    const std::lock_guard<std::mutex> lock(g_crashedPluginsMutex);
+    return g_crashedPlugins.contains(canonical, /*ignoreCase*/ true);
+}
+
+void removeCrashedPlugin(const juce::String& pluginPath)
+{
+    if (pluginPath.isEmpty()) return;
+    const auto canonical = juce::File(pluginPath).getFullPathName();
+    const std::lock_guard<std::mutex> lock(g_crashedPluginsMutex);
+    // Case-insensitive match, mirroring addCrashedPlugin/shouldSandbox's
+    // contains(..., ignoreCase=true). Iterate backwards to remove safely.
+    bool removed = false;
+    for (int i = g_crashedPlugins.size(); --i >= 0;)
+    {
+        if (g_crashedPlugins[i].equalsIgnoreCase(canonical))
+        {
+            g_crashedPlugins.remove(i);
+            removed = true;
+        }
+    }
+    if (removed)
+        VST_TRACE("removeCrashedPlugin: %s removed from runtime crash blocklist",
+                  canonical.toRawUTF8());
+}
+
 void setCrashedPlugins(const juce::StringArray& pluginPaths)
 {
     const std::lock_guard<std::mutex> lock(g_crashedPluginsMutex);

src/audio/Sandbox/SandboxedProcessor.h

@@ -236,6 +236,18 @@ void setCrashedPlugins(const juce::StringArray& pluginPaths);
 // offending plugin to the out-of-process sandbox.
 void addCrashedPlugin(const juce::String& pluginPath);
 
+// Query whether a path is currently on the runtime crash blocklist (matched
+// case-insensitively, same as shouldSandbox). Lets a caller that pins a plugin
+// only for a transient operation know whether the entry was already present.
+bool isCrashedPlugin(const juce::String& pluginPath);
+
+// Remove one plugin path from the runtime crash blocklist (case-insensitive).
+// Companion to addCrashedPlugin for callers that pin a plugin to force a
+// sandbox load for a single operation (e.g. promoting an editor) and must undo
+// the pin if that operation fails — so a plugin that never actually crashed
+// isn't left permanently forced to the sandbox for the session.
+void removeCrashedPlugin(const juce::String& pluginPath);
+
 // Resolve the path to slopsmith-vst-host.exe (sits next to the audio addon
 // .node). Returns a non-existent File if it can't be located. Exposed so the
 // out-of-process VST scan path can spawn the same host binary as the sandbox.

src/audio/SignalChain.cpp

@@ -384,6 +384,15 @@ void SignalChain::queueMidiMessage(int targetSlotId, const juce::MidiMessage& ms
     // If queue full, message silently dropped (acceptable for PC messages)
 }
 
+// Shared prepare sequence for a processor entering the live chain — used by
+// addProcessor and replaceProcessor so the channel config and prepare ordering
+// stay identical between them. Call from inside invokePlugin's fault guard.
+static void prepareForPlayback(juce::AudioProcessor& p, double sampleRate, int blockSize)
+{
+    p.setPlayConfigDetails(2, 2, sampleRate, blockSize);
+    p.prepareToPlay(sampleRate, blockSize);
+}
+
 int SignalChain::addProcessor(std::unique_ptr<juce::AudioProcessor> processor,
                                ProcessorSlot::Type type,
                                const juce::String& name,
@@ -403,8 +412,7 @@ int SignalChain::addProcessor(std::unique_ptr<juce::AudioProcessor> processor,
     // slot is dropped, rather than taking the app down.
     invokePlugin(*slot, [&](juce::AudioProcessor& p)
     {
-        p.setPlayConfigDetails(2, 2, currentSampleRate, currentBlockSize);
-        p.prepareToPlay(currentSampleRate, currentBlockSize);
+        prepareForPlayback(p, currentSampleRate, currentBlockSize);
     });
     if (! slot->processor) return -1;
 
@@ -421,6 +429,91 @@ void SignalChain::removeProcessor(int slotId)
     if (idx >= 0) slots.remove(idx);
 }
 
+bool SignalChain::replaceProcessor(int slotId, std::unique_ptr<juce::AudioProcessor> processor)
+{
+    if (!processor) return false;
+
+    // Copy the target slot's identity (type/name/path) onto the staging slot
+    // BEFORE preparing, so if the incoming processor faults during prepareToPlay
+    // invokePlugin's catch blocklists the RIGHT plugin path (addProcessor sets
+    // these before its own prepare for the same reason). Without this the staging
+    // path is empty and the fault is recorded against "".
+    ProcessorSlot staging;
+    {
+        const juce::ScopedLock sl(lock);
+        const int idx = findSlotIndex(slotId);
+        if (idx < 0) return false;           // nothing to replace
+        staging.type = slots[idx]->type;
+        staging.name = slots[idx]->name;
+        staging.path = slots[idx]->path;
+    }
+
+    // Prepare the incoming processor before it goes live, exactly as addProcessor
+    // does — under invokePlugin's SEH/signal guard so a fault in prepareToPlay is
+    // contained (the processor is dropped) rather than taking the app down.
+    staging.processor = std::move(processor);
+    invokePlugin(staging, [&](juce::AudioProcessor& p)
+    {
+        prepareForPlayback(p, currentSampleRate, currentBlockSize);
+    });
+    if (! staging.processor) return false;   // faulted during prepare → leave the slot as-is
+
+    std::unique_ptr<juce::AudioProcessor> old;
+    {
+        const juce::ScopedLock sl(lock);
+        const int idx = findSlotIndex(slotId);
+        if (idx < 0) return false;           // slot was removed underneath us
+        auto* slot = slots[idx];
+        old = std::move(slot->processor);
+        slot->processor = std::move(staging.processor);
+    }
+    // Tear the old processor down OUTSIDE the audio lock: releaseResources() (and
+    // a VST3 destructor) can block, and must never stall process() on it.
+    if (old)
+    {
+        old->releaseResources();
+        old.reset();
+    }
+    return true;
+}
+
+bool SignalChain::captureVstStateForPromotion(int slotId, juce::MemoryBlock& state)
+{
+    // Hold the audio lock across the whole check+snapshot: hasEditor() and
+    // getStateInformation() are plugin calls on a LIVE processor, and process()
+    // runs processBlock on that same instance under this lock (ScopedTryLock, so
+    // it simply drops a block here rather than deadlocking). Doing the snapshot
+    // off-lock would be a data race with the audio thread.
+    const juce::ScopedLock sl(lock);
+    const int idx = findSlotIndex(slotId);
+    if (idx < 0) return false;
+    auto* slot = slots[idx];
+    if (! slot->processor) return false;
+    if (slot->type != ProcessorSlot::Type::VST) return false;
+    // Already out-of-process — nothing to promote (its editor path is safe).
+    if (dynamic_cast<slopsmith::sandbox::SandboxedProcessor*>(slot->processor.get()) != nullptr)
+        return false;
+
+    // Run the plugin calls under invokePlugin's SEH/signal guard: a plugin that
+    // faults in hasEditor()/getStateInformation() is contained + blocklisted +
+    // released (leaving slot->processor null), never fatal.
+    bool promotable = false;
+    invokePlugin(*slot, [&](juce::AudioProcessor& p)
+    {
+        if (! p.hasEditor()) return;   // editor-less VST3 → nothing to open
+        p.getStateInformation(state);
+        promotable = true;
+    });
+    // slot->processor is null iff the guarded call faulted (invokePlugin released
+    // it). Don't promote from a released slot; the empty `state` is discarded.
+    if (! promotable || slot->processor == nullptr)
+    {
+        state.reset();
+        return false;
+    }
+    return true;
+}
+
 void SignalChain::moveProcessor(int fromIndex, int toIndex)
 {
     const juce::ScopedLock sl(lock);

src/audio/SignalChain.h

@@ -52,6 +52,27 @@ public:
                      const juce::String& name,
                      const juce::String& path);
     void removeProcessor(int slotId);
+    // Swap the processor in an existing slot IN PLACE — same slotId, position,
+    // name, path, type and routing are preserved; only the underlying processor
+    // changes. Used to promote an in-process VST3 to the out-of-process sandbox
+    // when its editor is opened (an in-process editor is the Windows WndProc/Qt
+    // crash path). Prepares the incoming processor under the SEH guard, then
+    // swaps under the audio lock; the old processor is torn down off the lock.
+    // Returns false if the slot is gone or the incoming processor faulted in
+    // prepareToPlay (in which case the existing processor is left untouched).
+    bool replaceProcessor(int slotId, std::unique_ptr<juce::AudioProcessor> processor);
+    // Snapshot a slot's state for sandbox promotion, SAFELY. Runs hasEditor()
+    // and getStateInformation() under the audio lock (so they can't race
+    // process()'s processBlock on the same instance) and under the SEH/signal
+    // guard (so a plugin that faults during the snapshot is contained +
+    // blocklisted, not fatal — an unguarded getStateInformation on the very
+    // plugins this promotion targets would reintroduce the editor crash on the
+    // message thread). Returns true and fills `state` only for a non-sandboxed
+    // in-process VST3 slot that actually has an editor; returns false (leaving
+    // `state` empty) for a missing/non-VST/editor-less/already-sandboxed slot or
+    // if the guarded calls faulted (processor released). A false result means
+    // "not promotable" — the caller may safely open the editor in-process.
+    bool captureVstStateForPromotion(int slotId, juce::MemoryBlock& state);
     void moveProcessor(int fromIndex, int toIndex);
     void setBypass(int slotId, bool bypassed);
     void setMultiBypass(const juce::Array<std::pair<int, bool>>& changes);
@@ -66,6 +87,10 @@ public:
     int getNumSlots() const;
     const ProcessorSlot* getSlot(int slotId) const;
     juce::Array<const ProcessorSlot*> getAllSlots() const;
+    // Current prepared playback format — used to prepare a processor that is
+    // swapped in mid-session (replaceProcessor) at the same rate as the chain.
+    double getCurrentSampleRate() const { return currentSampleRate; }
+    int getCurrentBlockSize() const { return currentBlockSize; }
 
     // Parameters for a specific slot
     struct ParamInfo