Finding 2 of the final whole-branch review: the sidecar's loopback
WebSocket accepted any connection with no origin/auth check, so any
local process (or webpage that guessed/scanned the ephemeral port)
could drive load-chain to load arbitrary native plugin bundles or read
arbitrary files via wavPath.
server.py's VstServer now generates a per-process secrets.token_urlsafe
token, prints it alongside the ready line, and rejects the WebSocket
handshake via websockets' process_request hook unless the connecting
client presents it as a `?token=` query param. Threaded through:
studio-server's vstSidecar.ts (extended ready-line regex) and
routes/vst.ts (/vst/start response), the CLI adapter, and the studio
client's useVstHost.ts (connects with the token in the WS URL).
vstBounce.ts (the offline render path) is unaffected — it invokes the
sidecar's `bounce` CLI subcommand directly and never opens a WebSocket.
Also folds in two smaller final-review findings that touch the same
files:
- vstSidecar.test.ts's mid-spawn-kill test used a fixed 300ms sleep to
wait for the child's PID file; replaced with a short-interval poll
(waitForFile) to remove a CI-contention flake risk.
- useVstHost.ts's handleDisconnect doc-comment claimed trackIndexRef was
kept "purely for the FX property panel's own direct loadChain calls"
— false (the FX panel never calls loadChain; only useVstPreview
does). Corrected to describe it as forward-defensive infrastructure
with no current FX-panel caller.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Finding 1 of the final whole-branch review: opening a plugin's native
editor and tweaking a parameter only ever mutated the sidecar's live
in-memory plugin instance — nothing read the updated state back and
wrote it into the track's .vstchain.json, so `hyperframes render`
silently bounced stale/default state instead of what the user heard in
preview.
There is no "editor closed" event on the wire (the native window closes
via its own OS chrome), so VstSection now polls vstHost.getState every
2.5s while an editor is presumed open (since the last "Open editor"
click, until the track changes or the section unmounts) and PUTs the
chain file via the existing writeChainFile helper when state actually
changed. A best-effort final flush runs from the same effect's cleanup
on unmount/navigate-away.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds an end-to-end integration test proving the whole VST render path
(composition HTML -> processCompositionAudio -> vst-host sidecar bounce
-> mixed output) works for real, using a deterministic builtin
pedalboard plugin. Skips when uv/vst-host aren't available.
Also adds packages/vst-host/README.md (install, WS protocol, exit
codes, manual E2E checklist) and closes a deferred Task 8 coverage gap
with a focused test for the lambda-render VST guard.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rounds 1 (621523c6f) and 2 (6e05901c9) each tried to enumerate, per track,
whether it was safe to re-stream after a sidecar disconnect/reconnect, and
each left a real gap on review (round 2's unsafeAfterDisconnectRef missed
tracks whose local AudioContext/AudioWorkletNode wiring was still in flight
when the disconnect fired). The sidecar's own trackIndex-assignment rule can
reassign colliding indices on any reload, and a WS disconnect doesn't reliably
tell the client whether the same stateful process answered the reconnect or
a fresh one did (server-side protocol flaw, out of scope here) — so no
client-side per-track tracking has proven complete.
Replace all of it with one suspendedRef in useVstPreview: the first
disconnect this hook instance ever observes permanently stops it from
loading ANY track — previously-loaded, previously-attempted, or never-seen
— for the rest of its mounted lifetime. Delete unsafeAfterDisconnectRef
entirely. Also fix the "Load chains" effect's cancellation branch to fully
teardownTrack() a load that resolves after a disconnect (unmuting the
element) instead of only closing its AudioContext, which previously left a
mid-flight track muted forever with no audio ever routed to it.
useVstHost's trackIndexRef mirror-clearing behavior is left unchanged: it's
still needed for the FX property panel's independent, direct loadChain
calls on the shared connection, which aren't gated by useVstPreview's
suspension.
Tradeoff, stated plainly: a single transient WS blip now permanently
disables VST preview for the rest of the studio session (recoverable only
by reloading the page) — accepted because two narrower recovery attempts
have each shipped with a real correctness gap, and a wrong/misrouted track
is worse than an honest, visible fallback to dry audio.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A WS disconnect does not imply the VST sidecar process was torn down —
startVstSidecar() is a singleton reused across reconnects in the common
case, so its server-side _tracks dict survives. useVstHost previously
cleared its trackIndex mirror on every disconnect on the false premise
that a reconnect always gets a fresh, empty sidecar, while useVstPreview's
crash-fallback auto-reconnect wiped its own "already loaded" bookkeeping
and silently re-issued load-chain for every previously-streamed track.
The server's real (never-reset) index-assignment rule could then hand
those re-loaded tracks a colliding trackIndex that the client's
freshly-recomputed mirror had no way to detect.
Stop clearing trackIndexRef on disconnect (it's a more accurate mirror
left as-is than reset to a false "confirmed empty" state), and track
which trackIds were streaming at disconnect time so the "Load chains"
effect never auto-reloads them again — they stay in dry fallback until
something explicitly re-triggers a load, matching this file's existing
"degrade to dry when index safety can't be guaranteed" pattern.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Task 13b addendum: VstSection never received a real vstHost (PropertyPanel
defaulted it to null, StudioRightPanel never passed one), so the FX panel
was decorative. Naively fixing that with a second useVstHost() call would
have opened an independent WebSocket racing useVstPreview's connection for
the sidecar's per-track numeric trackIndex state.
Fix: mount ONE useVstHost() instance in NLEProvider (alongside useVstPreview,
moved there from useTimelinePlayer to keep it under the studio 600-LOC file
cap) and thread it through NLEContext so both useVstPreview and
StudioRightPanel's FX panel share the same connection. Add a chain-loaded
broadcast (onChainLoaded) plus a trackIndex-mirroring rule in useVstHost
that replicates the sidecar's pop-then-reinsert indexing exactly, so
useVstPreview can resync an already-loaded track's index when any caller
reloads its chain, and revert both sides to dry playback (with a toast)
instead of silently misrouting PCM when the sidecar's own indexing rule
collides two tracks.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds VstRingBuffer (pure TS, unit-tested), the vst-stream AudioWorkletProcessor,
and useVstPreview — the transport bridge that mutes data-vst-chain <audio>
elements, streams the sidecar's processed PCM through a worklet, keeps it in
sync with play/pause/seek and a periodic drift check, and falls back to dry
audio with a toast + one auto-restart attempt if the sidecar disconnects.
Mounted from useTimelinePlayer with a one-line addition.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
loadChain and getState overwrote pendingTrackRef entries on a second
call for the same trackId, silently discarding the first call's
resolve/reject and leaving its promise hanging forever. Mirror scan()'s
existing supersede-and-reject pattern, scoped per trackId.
Implements the real VstHostApi that propertyPanelVstSection.tsx (Task 11)
consumes but didn't provide: lazily POSTs /api/vst/start, opens a WS to
the returned port with binaryType=arraybuffer, and dispatches JSON
control-plane events (scan/load-chain/get-state/error) to pending
request resolvers keyed by trackId, while forwarding raw binary PCM
frames to onPcmFrame subscribers untouched for Task 13 to decode.
Adds the property-panel UI for attaching a VST effect chain to an <audio>
element (Task 11 of the VST plugin support plan):
- packages/core/src/editing/affordances.ts: resolveEditingSections gains
vstFx (true only for facts.tag === "audio"), narrower than the existing
media flag.
- packages/studio/src/utils/vstChainFile.ts: ChainFileJson/ChainPluginJson
types + parseChainFile/serializeChainFile for the on-disk .vstchain.json
contract shared with the Python sidecar.
- packages/studio/src/components/editor/propertyPanelVstSection.tsx: new
VstSection component + the VstHostApi interface Task 12's useVstHost will
implement. Renders an install hint when vstHost is null; otherwise lets a
user add an effect (writes fx/<elementId>.vstchain.json via the existing
files PUT endpoint, then commits data-vst-chain via onSetAttribute), list
chain plugins, remove one, and open a plugin's editor.
- Wired into PropertyPanel.tsx gated on sections.vstFx, with an optional
vstHost prop (defaults to null) on PropertyPanelProps for Task 12 to fill in.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds registerVstRoutes (POST /vst/start, GET /vst/status) and the optional
startVstSidecar/getVstSidecarStatus StudioApiAdapter methods, wired into
both hosts (CLI embedded server + Studio Vite dev server).
Relocates the VST sidecar lifecycle module from packages/cli/src/vst/sidecar.ts
to packages/studio-server/src/vstSidecar.ts (exposed via a new
@hyperframes/studio-server/vst-sidecar subpath export) so both hosts can
import it without duplicating the resolver + spawn logic; the CLI module
becomes a thin re-export so hyperframes preview's existing import keeps
working unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
stopVstSidecar() only checked the `running` variable, which is set after
the sidecar's ready handshake. Calling it while a spawn was still in
flight (e.g. Ctrl-C during the sidecar's boot window) was a no-op,
orphaning the child process. Track the in-flight child in a new
`spawningChild` variable from spawn() time so stopVstSidecar() can kill
it before the handshake arrives.
startVstSidecar()'s singleton guard only checked the module-level
`running` variable, which is assigned inside the async ready-handshake
callback. A second concurrent call landing before the first child
became ready saw `running === null` and spawned its own child,
orphaning whichever process lost the ready race — exactly what
stopVstSidecar() and preview.ts's shutdown paths exist to prevent.
Track an in-flight `pending` promise, set synchronously before any
spawn/callback work, so concurrent callers share the same startup
attempt instead of double-spawning. Cleared via .finally() so a future
call after this attempt settles (success or failure) can retry.
Adds packages/cli/src/vst/sidecar.ts: resolves the vst-host sidecar
command (HF_VST_HOST_CMD override -> monorepo uv run -> bare command
on PATH), spawns `<cmd> serve --port 0`, and resolves once stdout
announces VST-HOST-LISTENING port=<N> (30s timeout with an
`uv tool install` hint). startVstSidecar()/getVstSidecar() are a
per-process singleton.
Wires sidecar teardown into all three `hyperframes preview` launch
paths: registerChildTreeShutdown's signal handler (dev + local studio
modes) and embedded mode's own SIGINT/SIGTERM cleanup + last-resort
exit handler, since embedded mode has no child process to register
with registerChildTreeShutdown. stopVstSidecar() is a safe no-op when
no sidecar is running, so this holds even though the sidecar itself is
started lazily by a later API route (Task 10), not by preview.ts.
processCompositionAudio's per-element loop races each track's async chain
concurrently via Promise.all, which rejects as soon as the first element
fails without waiting for siblings still in flight. A VstChainProcessingError
from one track therefore let another track's VST sidecar subprocess (up to
BOUNCE_TIMEOUT_MS = 10 minutes) keep running after the finally block deleted
workDir out from under it.
Thread an internal AbortController's signal through every element's chain
(extract/prepare/applyVstChainToWav), aborting it in a catch around
Promise.all before workDir is removed, and abort it on the caller's own
external signal too. applyVstChainToWav now accepts { signal } and kills its
spawned sidecar on abort, following runFfmpeg's existing spawn-abort
convention.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A missing VST plugin or chain file was caught by the per-element try/catch
shared with every soft-degrading failure mode (missing source asset,
download failure, etc): the track was silently dropped and MixResult.success
stayed true, so a render with a missing plugin completed successfully with
no visible error anywhere.
Introduce VstChainProcessingError to mark VST-specific failures and let the
per-element catch rethrow it instead of swallowing it into the warnings
list. The rethrow escapes the Promise.all and rejects
processCompositionAudio itself, which (traced through audioStage.ts's
uncaught await and observability.ts's observeRenderStage, which rethrows)
surfaces as a real render failure through the CLI's existing error-handling
path — the same one every other hard pipeline failure already uses. Every
non-VST failure mode is untouched.
Fixed the two "hard-fails" tests to assert the promise actually rejects
(rejects.toThrow) instead of only checking tracksProcessed/error on an
still-resolved, still-successful result — the previous assertions would
have passed even with the bug present.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Spawns the hyperframes-vst sidecar (packages/vst-host) to bounce a
track's dry WAV through its VST chain before the volume-envelope bake
in processCompositionAudio. A missing plugin or sidecar failure is a
hard failure for that track (named plugin + track id), never a silent
fallback to unprocessed audio.
An unrelated client sending {"cmd":"transport","action":"pause"} could
cancel another connection's active playback task, the same class of bug
the disconnect and play-takeover paths already guard against via
self._play_owner. pause now only cancels/clears the play task when the
sending connection is the current owner; otherwise it is a no-op.
Adds a regression test mirroring the existing disconnect-ownership test.
Track which websocket connection owns the running play task so an
unrelated client disconnecting no longer cancels another client's
active playback. Broaden _dispatch's exception handling so any error
(not just PluginMissingError) — e.g. a KeyError from a command
referencing a not-yet-loaded/unloaded/unknown trackId — replies with
{"event":"error","code":"bad_command"} instead of crashing the
connection with a 1011 close.
Adds two TDD regression tests reproducing both bugs against the
pre-fix code before implementing the fixes.
Adds server.py (VstServer) implementing the full control protocol over one
websocket: load/unload-chain, set-param, get-state, scan, open/close-editor,
and transport (play/pause/seek) with realtime-paced binary PCM streaming via
Task 4's TrackStream/wire-frame codec. Wires up `hyperframes-vst serve
[--port N]` in __main__.py. Adds pytest-asyncio dev dependency and
asyncio_mode = "auto" pytest config for the new async websocket tests.
Moves assetResolution.ts from packages/lint into @hyperframes/parsers behind
a ./asset-resolution subpath export, mirroring the ./ff-binaries shape, and
points lint's two importers at it. Pure move: no behavior change. Unblocks
studio-server consuming the same helpers without a dependency cycle (the CLI
already depends on lint, so importing lint from studio-server is not an option).
The cli, engine, and lint packages each carried their own copy of the
ffmpeg/ffprobe lookup, annotated fallow-ignore code-duplication, and the
copies had drifted: the engine copy handled Windows PATHEXT and executed
which/where without a shell but lacked the Homebrew-dirs fallback for
GUI-spawned processes; the cli copy had the opposite. One resolver in
@hyperframes/parsers (the dependency-graph bottom) now carries the union
of both hardenings, and all three packages delegate to it. Every
consumer gets strictly more robust resolution; env-override semantics
per call site are preserved via configuredMustExist.
A composition variable mirrored as --<slug> for a mounted sub-composition
(default or an explicit data-variable-values value) previously overrode any
same-named custom property the document already authored elsewhere (e.g. a
:root theme token), since the mirroring had no "already defined" guard —
unlike the two other emission paths, which already skip re-emitting when the
name collides with an authored definition. Extend that guard to the
sub-composition mount path in both the compiler (htmlBundler.ts) and the
runtime loader (compositionLoader.ts / getVariables.ts), so an authored
definition always wins; render-time --variables overrides still always win.
Auto-generated weekly digest (bun run changelog:weekly), hand-cleaned to
publish quality. Grouped into Features / Fixes / Performance / Docs &
Examples with readable one-liners and commit + PR links.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Packages Jake Moran's changelog-video pipeline (v1, validated end-to-end
by Home on the Jun 23-29 range) as a repo-native skill set that Claude
Code (.claude/skills/) and Codex CLI (.agents/skills/) auto-discover the
moment the repo is opened. No install step; run the skill against a
changelog markdown for a given git range and it produces a lint-clean,
seam-gate-green 1080x1080 MP4 (~45-60s, Annie VO, mock-UI visualizations,
caption rail) end-to-end.
Six skills added byte-identical in both mirror dirs:
- changelog-video (pipeline entry point)
- motion-doctrine (carries seam-stamp.mjs + seam-gate.mjs)
- cut-the-curve, captions-overlay, seam-craft, oversized-cursor
Layout:
- .claude/skills/ - Claude Code project-local auto-discover
- .agents/skills/ - Codex CLI project-local auto-discover (verified via
Magi's clean-home Codex 0.144.3 repro; NOT .codex/skills/)
Fonts, animated background (12 MB), house BGM (5 MB), lexicon, and
align-captions ship inside the skill dirs. .gitattributes routes only
.claude/skills/**/*.{mp4,mp3} + .agents/skills/**/*.{mp4,mp3} through
LFS — narrowly scoped so unrelated Player, Studio, registry, and
marketplace media stay put. HeyGen CLI auth is the one credential the
skill needs; Node >= 22, ffmpeg, and headless Chrome are documented
alongside in both READMEs.
.gitignore: rewrites .claude/ and .agents/ blocks to keep agent-installed
skill hygiene while re-including the six repo-native skill dirs plus
README.md.
CI:
- Extends changes.skills filter to match .claude/skills/**,
.agents/skills/**, scripts/lint-skills.ts, and scripts/check-skill-mirror.mjs.
- New 'Skills: project-native lint + mirror' job runs the extended
lint-skills.ts (schema-driven; required { name, description } + optional
{ license, allowed-tools, metadata }, name pattern check, description
length check) plus a new check-skill-mirror.mjs byte-integrity script
(24 mirrored files must match; README.md deliberately per-CLI).
- Wired into 'bun run lint' locally.
Frontmatter validator:
- Rejects unsupported top-level keys (catches category:-style drift).
- Requires name + description.
- Validates name pattern (^[a-z][a-z0-9-]{0,63}$) and description shape
(non-empty, <=1024 chars).
- Missing frontmatter block itself is a first-class error.
Also strips unsupported top-level 'category:' frontmatter from Jake's
motion-doctrine and cut-the-curve SKILL.mds (both mirrors), rewrites the
TTS invocation from ~/.claude/skills/media-use/... to the tracked
skills/hyperframes-media/scripts/heygen-tts.mjs, swaps npx hyperframes@latest
for the repo-local CLI in the gate step, and fixes a lint issue in Jake's
seam-gate.mjs (ternary-for-side-effect -> if/else).
Validated end-to-end by Home on Jun 23-29 (MP4 posted in C0ACCNHLG3U
thread 1784181166.041319). Independently reviewed R1/R2/R3 by Magi.
Co-authored-by: Jake Moran <jake@heygen.com>
## What changed
- adds a stage-aware review guide for plan, sketch, animation, and final-review states
- makes the review loop explicit: review frames → save feedback → reply in agent chat
- adds copyable agent handoff and approval messages
- keeps frame status agent-owned and presents planned frames before previews exist
- clarifies voiceover ownership, save state, feedback errors, and preview availability
- improves responsive layout and surfaces storyboard warnings without dominating the board
- adds unit coverage for review-stage and handoff-step derivation
## Why
Storyboard reviewers could leave comments, but the UI did not clearly explain when a pass was ready, what to inspect, or how saved feedback reaches the agent. This makes the human-to-agent iteration loop visible while preserving the existing file-based handoff.
## Impact
Users get a clear next action throughout the storyboard lifecycle and can copy the exact message needed to continue the agent workflow. The underlying storyboard and frame-comment formats are unchanged.
## Validation
- `bun run --filter @hyperframes/studio test --run src/components/storyboard/storyboardReviewStage.test.ts` — 13 tests passed
- `bun run --filter @hyperframes/studio typecheck`
- pre-commit: tracked artifacts, lint, formatting, Fallow audit, typecheck