mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
sync/hyperframes-codegen-81d5a9cd
738
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
720ff5ac9c | chore: release v0.8.16 | ||
|
|
ee64c3b116 |
fix(engine): stop destroying the AAC priming edit list when muxing (#3505)
`muxVideoWithAudio` passed `-avoid_negative_ts make_zero` unless the caller
set `preserveAudioPrimingEditList`. In practice the dominant path is an AAC
sidecar copied into mp4, where that flag is actively harmful: ffmpeg's
default is `auto`, which the mp4/mov muxers (AVFMT_TS_NEGATIVE) already
resolve to `disabled`. Forcing `make_zero` overrides the correct default,
discards the priming edit list the sidecar encode created, shifts the video
start_time forward by one AAC frame and writes an empty video edit at t=0 —
which edit-list-honoring players (QuickTime/Safari) render as a black first
frame.
Verified with ffprobe on a copy mux of a 30fps h264 mp4 and an AAC sidecar:
with `make_zero` video start_time 0.066000, elst: [media time -1,
dur 5940] + [media time 6000, dur 180000]
audio start_time 0.042993, elst: [media time -1, ...]
without (this fix) video start_time 0.000000, elst: [media time 6000,
dur 180000]
audio start_time 0.000000, elst: [media time 1024, ...]
The empty leading edit and the offset both disappear, and the audio keeps
its 1024-sample priming edit.
The flag is now never passed for a mux, in any mode. `preserveAudioPrimingEditList`
is part of the exported engine API, so it stays on `MuxVideoWithAudioOptions`
as `@deprecated` and no-op rather than being removed; the two internal callers
that set it (`assembleStage`, distributed `assemble`) drop it.
`buildEncoderArgs` and `streamingEncoder` still pass the flag for video-only
output and are deliberately left alone — those chunks are consumed as
intermediates, not as a delivered mp4/mov.
Fixes #3487
Co-authored-by: Alexandru Mincu <alex@mountsoftware.ro>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Val
|
97991bbd35 |
fix(engine,producer,lint): resolve <source> children for media extract and localize (#3238)
Parent src-only scans skipped multi-format <video>/<audio> markup, so those elements were never extracted, downloaded, or mixed and rendered blank/silent. Lint now accepts a child <source src> as a resolvable media src. |
||
|
Miguel Ángel
|
c9f43ebcfb |
fix(engine): preserve source frame identity above 99,999 (#3503)
* fix(engine): preserve extracted frame identity * fix(producer): order legacy distributed frames numerically |
||
|
Rajan Pantha
|
dae5b7b90b |
fix(engine): treat a sentineled cache entry with no frames as a miss (#3434)
lookupCacheEntry reported a hit purely on the presence of the .hf-complete sentinel. The sentinel records that extraction finished, not that the frames survived, so any per-file cleanup that empties the directory leaves an entry that rehydrates with zero frames. rehydrateCacheEntry then returns totalFrames: 0, the clip reaches the coverage gate with nothing, and the render aborts with a message about capture coverage. Because the poison is on disk rather than in the composition, every later render of the project fails the same way with nothing the user can change to fix it. An entry now counts as a hit only when it carries the sentinel AND still holds at least one frame file, so an emptied entry re-extracts. The check is format-agnostic: a hit must be usable whatever extension the frames carry. Addresses the cache half of #3372. |
||
|
Val
|
f52ec1c25f |
fix(producer,core): honor relative data-start id-refs in render media scheduling (#3252)
compileTimingAttrs/injectDurations used parseFloat, so data-start="intro" wrote a NaN data-end and extract preferred that over duration; parseNumeric now skips the id-ref (parseVideoElements already resolves it). collectRenderMedia's resolveHostWindow likewise read host data-start with parseFloat, so chained sub-composition slots (data-start="hook") stacked at 0-2s and every scene after the first rendered black. It now resolves host starts through the shared resolveReferencedStart, matching the media parsers. Fixes #3361. |
||
|
|
a7e8674758 |
fix(producer): fall back to screenshot capture on drawElement canvas-not-initialized (#3480)
* fix(producer): fall back to screenshot capture on drawElement canvas-not-initialized The fast-capture drawElement path only special-cased the "No cached paint record" error to trigger a per-frame screenshot fallback; every other error (including "drawElement canvas not initialized", seen at frame 0 on some macOS/Chrome combinations) was rethrown, hard-failing the whole render even though the docs promise automatic fallback on incompatible compositions. Extend the existing fallback branch (in both captureFrameCore and captureFrameToBufferPipelined) to also catch canvas-not-initialized errors via a shared isRecoverableDrawElementError predicate, with a diagnostic message identifying which case triggered the fallback. Closes #3423 Co-Authored-By: Miga <noreply@anthropic.com> * fix(producer): address review — tighten error matching, audit batch path, add fallback-ratio guard * fix(engine): add prepareFrameForCapture to batch screenshot fallback loop * fix(engine): split canvas-not-initialized from composition-root-missing errors drawElementService threw the same HF_DE_CANVAS_NOT_INITIALIZED error for both !canvas and !root. Missing composition root (navigated/broken page) was classified recoverable and fell back to pageScreenshotCapture, which captured blank or wrong content silently. Now: - !root → HF_DE_COMPOSITION_ROOT_MISSING (not recoverable, hard fail) - !canvas → HF_DE_CANVAS_NOT_INITIALIZED (recoverable, screenshot fallback) Split applied at all 3 emit sites (serial, pipelined, batch). Co-Authored-By: miga-heygen <miguel.sierra_miga@heygen.com> --------- Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com> Co-authored-by: Miga <noreply@anthropic.com> |
||
|
Miguel Ángel
|
740f7ead89 | chore: release v0.8.15 | ||
|
miga-heygenandMiga
|
3202f3fb87 |
fix(producer): trip DE parallel-router circuit breaker on stalls and hangs (#3479)
Root cause: the per-worker capture calls in captureFrameRange (parallelCoordinator.ts) take no abort signal of their own, and only checked `signal.aborted` BEFORE starting each frame — a no-op once a worker is already awaiting an in-flight call. On WSL2, the native drawElement/BeginFrame capture call can hang indefinitely at frame 0 with no error. The DE parallel-router's existing stall watchdog (captureStreamingStage.ts) correctly fires `stallController.abort()` after HF_DE_STALL_MS, but that abort had no way to reach a worker already wedged inside a hung capture call — so executeParallelCapture's Promise.all waited forever, the render hung indefinitely, and the CLI's circuit breaker (which only runs after executeRenderJob settles) never got a chance to trip. Fix: race each per-frame capture call against the signal actually firing (raceAgainstAbort), the same "can't cancel, only race" pattern already used by the sequential capture path. Once the watchdog's abort is observed, the wedged worker rejects, executeParallelCapture settles, and the existing pinned-fallback retry / "reverted" outcome / circuit breaker machinery (already correct) runs end to end. Also widen the CLI breaker's trip condition from the literal string "reverted" to "not a clean routed success", so any future non-success outcome the observability layer records also latches the breaker instead of silently falling through. Closes #3441 Co-authored-by: Miga <noreply@anthropic.com> |
||
|
Miguel Ángel
|
81069fe47f | chore: release v0.8.14 (#3474) | ||
|
Vance Ingalls
|
3ed971d018 | chore: release v0.8.13 | ||
|
Vance Ingalls
|
2ca578f945 | chore: release v0.8.12 (#3457) | ||
|
Vance Ingalls
|
1aec3b4a09 |
fix(engine): harden grouped audio rendering (#3446)
* fix(core): harden audio FX and group identity * fix(core): address audio group review feedback * fix(core): align preview transport with grouped audio * test(core): pin audio group gain ceiling * fix(core): preserve solo bridge through stack * fix(engine): harden grouped audio rendering * docs(engine): explain grouped mix fallback invariant * test(engine): allow grouped mixes to finish on Windows |
||
|
Miguel Ángel
|
32d58a73e3 | chore: release v0.8.11 (#3440) | ||
|
Miguel Ángel
|
65b2299db2 |
fix(engine): preserve static dedup across caption runs (#3438)
* fix(engine): preserve authored clip boundaries after normalization * perf(engine): bound static verification work across caption runs * fix(core): preserve explicit nonpositive timeline windows |
||
|
Miguel Ángel
|
59a69a145b | chore: release v0.8.10 (#3426) | ||
|
Vance Ingalls
|
f6e8e8ddfd | chore: release v0.8.9 (#3422) | ||
|
Vance IngallsandClaude Opus 4.7
|
073b098e21 |
feat(engine): preflight psnr filter availability and force-fallback to screenshot on missing (#3418)
## What Adds a one-shot ffmpeg-psnr filter probe at drawElement session bootstrap. When the resident ffmpeg is missing or lacks libpostproc (no `psnr` filter), the capture-session router now force-fallbacks to the screenshot capture path and emits a `de_gate_reason = "ffmpeg_no_psnr_filter"` telemetry signal via the existing `render_complete` breakdown. Also tightens `psnrForDiskSample`'s catch: infrastructure-class ffmpeg failures (ENOENT, "No such filter") no longer silently skip the sample — they abort the render so the safety net cannot fail-open post-preflight. ## Why The drawElement self-verify safety net (parallelCoordinator's `psnrForDiskSample` → `psnrDb`) shells to `ffmpeg -lavfi psnr`. If ffmpeg is missing, or was compiled without libpostproc (so the `psnr` filter is absent), every per-sample compare throws. The existing catch swallows the error and returns `null` — callers treat that as "skip this sample" and the render completes with the safety net inoperative. Field signal (⭐ 9/10 CLI feedback, Slack ts=1787380767.210079, hyperframes 0.8.7, darwin/arm64, tid=93ff9910-2207-45c2-bc1f-54c0b347d4fe): > "host ffmpeg lacked psnr filter used by drawElement self-verification, > but render completed." The user's frames happened to be byte-identical so no visual damage shipped — but the safety net silently wasn't running. Any future compositor-damage bug on that host would have shipped straight through. ## How Two-part fix, both in `packages/engine`: 1. New `utils/psnrFilterAvailability.ts` — cached probe that runs `ffmpeg -hide_banner -filters` once per process and word-boundary- matches `psnr` in the output. Any failure (ENOENT, non-zero exit, timeout, unparseable output) returns `false`; never rejects. 2. Wired into `services/frameCapture.ts` `initDrawElementOrTransparentBackground` right after the Chrome capability probe: when useDrawElement resolves true and the preflight returns false, set `session.deGateReason = "ffmpeg_no_psnr_filter"` (same low-cardinality bucket every other DE gate uses; flows through `getCapturePerfSummary` → `render_complete.de_gate_reason` in PostHog), emit a stderr warning naming what's missing, and call `routeToFallback()` — the same fail-graceful shape as the SwiftShader / CSS-effect / at-risk-timeline gates. Skipped under `HF_FORCE_DRAWELEMENT=1` (matches the diagnostic knob's policy of bypassing every other gate). Belt-and-braces: `psnrForDiskSample` now discriminates infrastructure- class failures (ENOENT / "No such filter" / "Unknown filter") from per-sample noise (readFile races, transient EPERM). Only the former re-throw — per-sample noise still returns `null` (skipped sample). The preflight normally catches this at bootstrap; the re-throw covers ffmpeg-swapped-mid-render. ## Test plan - [x] Unit tests added: `packages/engine/src/utils/psnrFilterAvailability.test.ts` — mocked `execFile` covers: `psnr` present → true; `psnr` absent → false; ENOENT → false; non-zero exit → false; result memoized + reset works; substring-not-word-boundary → false. - [x] Unit tests added: `isFfmpegInfrastructureFailure` in `packages/engine/src/services/parallelCoordinator.test.ts` covers ENOENT, "No such filter", "Unknown filter", per-sample EACCES, parse errors, null/non-object. - [x] `bun run test` — `packages/engine/src/utils/psnrFilterAvailability.test.ts` (6 tests) + `packages/engine/src/services/parallelCoordinator.test.ts` (50 tests) + `frameCapture.test.ts` (26 tests) all pass. Pre-existing ffprobe test failures (4) on the base commit are unrelated (missing PNG fixture bytes — the file is 129 B on disk, likely LFS-stored). - [x] `bunx tsc --noEmit -p packages/engine/tsconfig.json` — clean. - [x] `bunx oxlint <files>` — 0 warnings, 0 errors. - [x] `bunx oxfmt --check <files>` — clean. Not covered here: an integration test that boots `initDrawElementOrTransparentBackground` end-to-end. That path is Puppeteer-driven and has no unit-scale bootstrap harness in the repository — the pure preflight + pure discriminator coverage above are what this PR can prove at the vitest layer. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
6f82acf50c | chore: release v0.8.8 (#3411) | ||
|
Miguel Ángel
|
92a6076807 |
test(engine): budget the ffmpeg audio-level tests, and make a stall say why (#3410)
`places a delayed track on its authored start` timed out on the windows runner and failed an unrelated PR, the second time this week an ffmpeg audio test has done that. The previous fix raised the budget in `audioMixer.grouping.test.ts`, which was the file the symptom named. It was the wrong scope: that file was the only audio suite with explicit timeouts at all. `audioMixer.level.test.ts` had none, so its two real-ffmpeg tests ran on vitest's 5s default. The failing one takes ~137ms locally, so the runner is not 36x slower — but 5s was never a budget anyone chose for a full mix. Applied to the suite rather than to each test, so there is one home for it, and scoped to the ffmpeg-gated describe: the sibling parsing suites are pure and should keep failing fast at 5s. Headroom alone would only have delayed an undiagnosable failure. The ffmpeg process timeout is 5 minutes by default, far above any test budget, so a stalled mix could only ever surface as a bare "Test timed out" with no stderr and no failing stage. Tests now cap it at 20s and assert through a helper that reports `failures` instead of collapsing to `expected false to be true`. Both claims verified rather than asserted: a deliberately 6s test now passes where the 5s default would have killed it, and forcing the process timeout to 1ms reports `stage: "prepare", reason: "ffmpeg_timeout"` instead of a timeout. Reviewing with whitespace ignored is much smaller: adding the third argument to `describe` reindents the suite body, so 131/101 is really 34/4. |
||
|
Miguel Ángel
|
9c73e64a07 |
test(engine): give the audio grouping mixes room, and make a stall say why (#3408)
`a group FX chain fully cutting its members leaves an ungrouped track untouched` timed out on the windows runner, failing an unrelated PR. The whole file runs in ~4s locally and that test in ~1.1s, so 30s was not generous — but the runner is roughly 10x slower and this test drives more ffmpeg than any of its siblings, two full mixes plus a group FX chain. 30s was the tightest budget in the package; 60s is what the rest of the ffmpeg-driven engine tests use. Headroom alone would only have moved the same undiagnosable failure later, because nothing here could report why. The production ffmpeg process timeout is 5 minutes, far above any test budget, so a stalled mix could only ever surface as "Test timed out in 30000ms" with no stderr and no failing stage. Tests now cap it at 20s, and the mix wrapper throws the recorded failures instead of returning `success: false` into an `expect(...).toBe(true)` that reports `expected false to be true` and discards the reason. Verified by forcing the process timeout to 1ms: the failure goes from a 30s wall-clock timeout to a 150ms error naming the stage, reason and element (`stage: "prepare", reason: "ffmpeg_timeout", elementId: "a"`). This does not explain the Windows stall itself, which I could not reproduce on macOS. It makes the next occurrence report what it was doing. |
||
|
Miguel Ángel
|
41af866bcb | chore: release v0.8.7 (#3402) | ||
|
Miguel Ángel
|
77566a198b |
test(engine): give the ffmpeg-bound grouping mixes their 30s timeout (#3398)
audioMixer.grouping.test.ts spawns real ffmpeg per assertion and ran on vitest's 5s default; on slow Windows runners the FX-chain and envelope cases land right at the line and fail runs that touch nothing in the engine. The other ffmpeg-bound engine suites (videoFrameExtractor) already carry a per-test 30_000 timeout; this brings the grouping suite in line. |
||
|
Vance IngallsandClaude Sonnet 5
|
0d26072e6c |
feat(studio,core): mute groups, and hear-only-this that cannot reach the export (#3291)
B5: mute and solo, on groups and tracks (track mute already shipped by A2 —
nothing to build there).
Group mute — persisted as data-hidden on the <hf-audio-group> element itself
(never written onto members, per design doc §2.1's state-restoration
warning). Studio action reuses B7's generic setAudioGroupAttribute
(setQuiet/setLive split) rather than duplicating toggleTimelineTrackHidden's
shape — same one-atomic-patch/one-undo-entry contract, already built for
exactly this purpose. Render: B4 already drops every member of a
data-hidden group (confirmed by a new audioMixer.test.ts case — no
production change needed there). Preview: a dedicated muteGain node
(groupInput -> [fx] -> muteGain -> output -> master) so a mute toggle
never fights scheduleVolumeLane's ramps on the same param — the same
hazard B7's volume fader was split out to avoid. Mid-playback toggles
sync via a new syncAudioGroupMute pass in init.ts (a group carries no
data-start, so it's invisible to the existing visibility-node query).
Members of a muted group render the strikethrough label treatment
(TimelineTrackPlainHeader's isGroupMuted, sourced from
TimelineElement.audioGroupHidden) — display only, no attribute touched.
Solo — "Hear only this": a new session-only store slice (audioSoloSlice,
soloed: ReadonlySet<string> of clip/group ids, never track numbers, never
serialized). Predicate (isAudibleUnderSolo, packages/core/src/audioGroups.ts
so both the store and the preview transport share one definition): an
element is audible while any solo is active only if it or its own group is
soloed. "Siblings, never ancestors" lives in the graph, not the predicate —
solo gain is a per-element stage only; group buses are never attenuated by
solo, so a soloed member's path through its group stays open by
construction. Preview: a dedicated per-element soloGain in
webAudioTransport.ts (parallel to the mute mechanics), pushed via
window.__hf.setAudioSolo — a direct call, not an attribute write, so it
can't ride the visibility-diff path mute uses. media.ts's HTMLMedia
fallback folds the same predicate into its per-tick volume computation
(the same seam A2 used for data-hidden). Half-lit group indicator
(isGroupHalfLitUnderSolo) for "not soloed itself, but a member is".
Exclusive-by-default toggle, ⌘/Ctrl-click to add/remove, TimelineSoloButton
(⌗) beside mute on both track and group headers. Transport-bar banner
("Hearing only <label> — your export is not affected", Clear button) added
in PlayerControls.tsx, reading labels straight off the live preview DOM.
Export-safety, the most important property here: toggling/adding/clearing
solo never calls setAttribute/removeAttribute on any element and never
invokes the project save path (both asserted directly via spies in
audioSoloSlice.test.ts) — solo cannot reach an export by construction, not
by convention.
Also: extracted useHydrateActiveCompPathFromUrl out of App.tsx (a
pre-existing, unrelated effect) to stay under the 600-line filesize cap
after wiring useAudioSoloBridge in; and fixed a circular dependency the
solo-banner wiring introduced (useAudioSoloBridge.ts now imports
usePlayerStore from its concrete module instead of the player/ barrel,
which re-exports PlayerControls.tsx — the barrel path is what closed the
cycle).
Gates: bun run build clean; packages/core full suite 2379/2379; packages/
studio full suite 4276/4294 (18 pre-existing todo); packages/engine
audioMixer.grouping.test.ts 5/5; oxfmt/oxlint clean on all 23 touched
files; fallow clean (0 new circular deps, 0 new filesize/complexity
findings).
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Sonnet 5
|
99f42be04c |
feat(engine): render grouped audio through a summed, FX-processed bus (#3289)
* feat(core): route grouped audio through a group bus in preview An audio element carrying `data-audio-group` no longer lands its gain on the master bus directly — it feeds a per-group `GainNode` (built lazily on first use, one per group id) which itself feeds master, so members of the same group sum before the ear, ready for a group-level FX chain and volume/mute in later steps. An id with no matching `<hf-audio-group>` element still gets a plain, unprocessed bus rather than losing the track. The group's own chain and volume lane are wired through the same `attachElementFxChain`/`scheduleVolumeLane` every element already uses, against the group's clock — composition time (design doc §1.3), since a group has no `data-start` and a missing one parses as 0. The bus persists across `stopAll()` (mirroring `_masterGain`'s own lifecycle) so replaying a group does not rebuild its chain; only `destroy()` disposes it. Render is untouched — stays flat until B4; `audio-groups` is still a 0% canary so nothing ships this to a real composition without hand-authoring `data-audio-group`. Also: `audioGroupOf` (B1) crashed on any element lacking a real `tagName`/ `getAttribute` — exactly the shape of most `HTMLMediaElement` test doubles in this suite, including this file's own `mockEl`. Made it tolerant, same style as `readChain`'s existing guard in `runtime/audioFx.ts`. `schedulePlayback` was already 110 lines pre-existing before this diff; extracted `resolveDestination` and `handleSourceEnded` to shrink it to 92, then suppressed the remainder (inherently sequential graph wiring, not a decision tree) per the same precedent B2 used on `TimelineLogicalRow`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(studio,lint): carve targets voiceover groups — always, when plural Plural voiceover carve now targets a group instead of naming each clip: `resolveCarveSourceIds` (core `audioGroups.ts`) expands a group id to its current members at analysis time, so a clip added to the group later is covered without touching `sources`. The picker (`useFxCarve.ts`) offers a grouped voice as one option instead of one row per member, tests overlap as a union of member spans (a group overlaps the bed if ANY member does), and prefers a qualifying group over its individual members in `autoSourceIds`. Picking two or more ungrouped voice clips in the carve flow now mints a group behind them (`mintGroupId`, de-duped against every id in the document) and writes `data-audio-group` on each picked clip atomically, one undo entry — `createAudioGroupAndAssignMembers` in `timelineTrackVisibility.ts` copies `setElementsHidden`'s multi-target write shape. The DSP is untouched: `mixCarveSources` already sums multiple sources correctly (verified in the design doc's own investigation) — this only fixes the picker. New lint rule `audio_carve_ungrouped_sources` (`packages/lint/src/rules/ media.ts`, alongside `audio_volume_double_automation`) warns when a `data-fx-carve`'s `sources` names two or more plain clip ids instead of a group — the shape that silently rots when a clip is added. `/hyperframes- audio` states the same rule as an invariant, not a tip, with the grouped- narration HTML example from the design doc. The group-matching and auto-group logic (`withAutoGroupedSources`, `collectCarveCandidates`) is split into `useFxCarveGrouping.ts` — `useFxCarve.ts` was pushing past the 600-line cap. `resolveNextCarveSettings` is deliberately NOT an `async function`: wrapping it in one would force a microtask on every call, including the synchronous branch — the exact bug `withAutoGroupedSources`'s own sync-when-possible contract exists to avoid, and one caught via `propertyPanelAudioFxGroup.test.tsx` (10 failures) before fixing it back to a plain function the caller conditionally awaits. Also extracted `useEffectiveTimelineDuration` out of `App.tsx` and `useRemoveBackground` out of `StudioRightPanel.tsx` (both pushed past 600 lines from an added prop wire), and decomposed `useFxCarve.ts`'s picker IIFE to clear fallow's complexity gate. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(engine): render grouped audio through a summed, FX-processed bus Renders what B3 already routes in preview: a group's members sub-mix into one PCM WAV at full composition length (adelay already places each member at its composition position, so the group WAV's t=0 IS composition time), run through the group's own FX chain and automation via the same applyAudioFxChain/envelope-bake path a member uses, then fold into the flat track list as one processed AudioTrack — the final mixAudioTracks call never has to know groups exist. Gain law verified against plans/spikes/amix-nesting-spike.sh (brought over from the plans branch, along with audioMixer.grouping.test.ts, since both were committed there and never merged to origin/main — every step branch in this stack descends from origin/main): the sub-mix's own amix prefers normalize=0 (nulls exactly against a flat mix), falling back to per-node compensation by the group's OWN member count only when this ffmpeg build's amix rejects the option. Carrying any other count into a nested amix node is the exact +2.499 dB silent failure the spike measured — confirmed by a manual mutation check (wrong-count compensation landed 3.5 dB hot, exactly 20*log10(3/2) for a 2-member group compensated as 3; reverted after confirming the level test catches it). A group element carrying data-hidden drops every member before the sub-mix ever runs (RULES: mute-by-drop, never mute-by-volume-0) — parseAudioElements now resolves groups once per parse and skips hidden-group members the same way it already skips data-hidden ancestors. HfAudioGroup (packages/core/src/audioGroups.ts, from B1) gains fxChain, automation, volume and hidden, read off the group element the same way resolveAudioGroups already reads data-label — audioGroups.test.ts updated for the wider shape plus new coverage for the added reads. it.todo("mixes a grouped composition at the same level as the ungrouped one") is now a real, passing test; two more added per the step doc (FX routing isolation, member-level envelope survives grouping). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
63eb35041c |
fix(deps): bump puppeteer so the browser hides its console window on Windows (#3394)
Windows users see a console window per chrome-headless-shell worker during a render. Those windows come from Puppeteer's own launcher, not from any spawn in this repo, so the windowsHide work on our ffmpeg spawns could not reach them. @puppeteer/browsers added windowsHide: true to its spawn in 3.2.1. It is absent in 3.1.0 and 3.2.0. puppeteer-core pins that dependency exactly, and 25.8.0 is the first release pinning 3.2.1 (25.5.0 -> 3.1.0, 25.6.0 and 25.7.0 -> 3.2.0), so 25.8.0 is the minimum that carries the fix rather than a preference for the latest. Verified after install that exactly one copy resolves, at 3.2.1, and that its launcher carries the flag. A draft render still completes. Refs #3379 |
||
|
Miguel Ángel
|
315a7b758c |
fix(engine): hide ffmpeg console windows on Windows (#3381)
ffmpeg and ffprobe are console-subsystem binaries and Node defaults windowsHide to false, so every spawn opened a visible console window on Windows. A render shells out dozens of times across parallel workers, which flashed a burst of windows across the user's desktop. Applied at every production spawn site rather than only the two named in the report, since they all share the cause: runFfmpeg, both gpuEncoder probes, ffprobe, streamingEncoder, audioExtractor and the distributed version check. windowsHide is a no-op on macOS and Linux. The dev-only parity and regression harnesses are left alone; they never run on a user's desktop. Closes #3379 |
||
|
Vance IngallsandClaude Sonnet 5
|
1b86b56127 |
feat(core): pitch shift — a granular shifter as the fifth FX worklet (#3276)
Adds hf-pitchshift alongside the four existing dynamics worklets: a dual-tap granular delay line, 100 ms grain, taps 180° apart so one is always crossfading in as the other resets — hides the splice each tap makes on wrap. Read-tap speed relative to the write head tracks the semitone ratio, so pitch shifts without changing duration. Registered through the same workletBuilder/dispose-message path the other four use (so shapeOf never rebuilds on a param tweak, and a chain drop retires it), wired into the registry with a plain-language copy entry and a ~0.2s chain tail (two grains). One implementation, shared by preview (Web Audio in the page) and render (the same worklet run inside an OfflineAudioContext in the headless browser) — confirmed by a browser-render test that measures the actual output frequency, not just that it differs from input. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
James Russo
|
36c7dffe5c | chore: release v0.8.6 (#3386) | ||
|
Miguel Ángel
|
c056289d83 |
fix(audio): renumber timestamps between apad and atrim in mixed branches (#3380)
On FFmpeg 5.x through 8.0.x the samples `apad` appends carry timestamps the following `atrim` misreads. A delayed branch then sounds at t=0 instead of its offset and, once four or more branches are mixed, the last one disappears from the output entirely. No error is raised; the render succeeds with wrong audio. Reverting to `apad=whole_dur=` is not an option: #2769 moved off that form because some builds reject the option outright ("Error applying option 'whole_dur': Option not found"). Inserting `asetpts=N/SR/TB` between the pad and the trim rebuilds the timestamps from the sample count using only filters every build ships, so it fixes the misplacement without giving up the portability that change bought. Verified on FFmpeg 4.2.7, 7.0.2, an 8.x nightly and 8.1.1: the current form is wrong on the middle two, the new form is correct on all four. audioPadTrim.ts also pads with apad+atrim but has no adelay and is correct on every version tested, so it is left alone. Closes #3344 |
||
|
Miguel Ángel
|
7a8f8a0b45 | chore: release v0.8.5 (#3375) | ||
|
Miguel Ángel
|
42b94fd5db | chore: release v0.8.4 (#3359) | ||
|
Miguel Ángel
|
634df5a5af |
fix(producer): give inlined media a document-unique render id (#3342)
* fix(producer): give inlined media a document-unique render id
Element ids are unique per composition file, but the render document is
the inlined union of every file. The producer merged the per-file media
lists and deduplicated by id, so clips that shared an id collapsed into a
single entry, and every id-keyed stage (extract, inject, visibility,
bounds) resolved to whichever element came first in the document. The
surviving clip's frames landed on the wrong element and the visible scene
rendered without footage.
Two shapes hit this, and neither is author error:
- Two scenes that each declare `<video id="clip">`. Legal per file, and
unavoidable when a scene is duplicated into a copy with inner ids
kept, or when one file is mounted twice.
- Two scenes that each declare a bare `<video>`. The timing compiler
numbers auto-ids per file, so both arrive as `hf-video-0` with no
authored id involved at all.
Stamp a document-unique `data-hf-render-id` while inlining, and read the
media list off the inlined document instead of merging per-file lists.
The render id equals the element id whenever that id is already unique,
so documents without a collision keep identical pipeline keys.
Author `id` attributes are left alone: 158 of the 161 registry blocks
reference their own ids from `#id` CSS or getElementById, so renaming
would trade broken footage for broken styling. The engine resolves media
elements through the render id instead, falling back to getElementById
for documents the producer never compiled.
Collecting from the inlined document also retires the per-file media
extraction in parseSubCompositions along with its offset bookkeeping;
host offsets are recovered from the composition hosts the clip sits in.
* fix(core): resolve render-frame siblings by render id in the runtime
The injector creates each `__render_frame_<id>__` sibling from the media
element's render id, but four runtime readers still built that id from the
plain `el.id`. On a document where two compositions share a media id, all
of them resolved the first collider's frame.
colorGrading is the one that changes pixels: findRenderFrameImage returns
the image the grading pass samples, with no class check to catch the
mismatch, so the second video was graded from the first one's frame.
media, mediaProxy and video-texture-compat use it as a render-mode or
substitute-source signal, where both colliders happen to agree during
render, but none of them should rest on that.
Add renderFrameSibling as the single owner of "which frame belongs to
this element" and route all four through it. It reads the stamped render
id and falls back to the author id, so a collision-free document resolves
exactly as before and an uncompiled one (preview, snapshot, check) is
unchanged.
The engine's in-page bridge keeps its own copy of the rule because code
serialized into page.evaluate cannot import; it now names core as the
definition, and a test pins the sibling-id format both sides build so
they cannot drift apart silently.
* refactor(engine): build render-frame sibling ids from core's definition
The drift guard named both sides but pinned one. renderFrameSibling.test
asserts core's format, while the engine rebuilt the same id from a literal
template at six independent sites. Changing the format on either side left
the test green and every runtime reader silently unable to find its frame —
this PR's own failure mode, one level up.
Export the affixes and renderFrameIdForRenderId from core, and take the id
from there at all six. Four sites resolve it on the Node side, where the
engine can import; the two that iterate the DOM in-page receive the affixes
as evaluate arguments, which avoids depending on bridge install order.
Also switch two `__hfMediaId?.(el) ?? el.id` reads to `||`. The bridge
returns "" for an element with neither id, so `??` kept the empty string
and built `__render_frame___`, which no reader looks for. Inert today
because the compiler assigns positional ids to id-less timed media, but it
made the two sides disagree in the one case they could.
|
||
|
Miguel Ángel
|
e282ff15cc |
fix(audio): raise the authoring gain ceiling and carry it through the probes (#3333)
* fix(audio): raise the authoring gain ceiling and carry it through the probes Builds on #3328, which made the preview graph apply author gain and user volume exactly once each. That ownership is now correct but everything is still clamped to 1.0, so a clip authored above unity cannot be heard or rendered. `HTMLMediaElement.volume` is spec-clamped to [0,1], so both timeline probes lost a clip's authored gain the moment it also carried a fade: the probe seeded the element at the clamped value and every sample read back at or below 0 dB, and the mixer prefers probed keyframes over the static volume. Both probes now shadow the accessor for their own duration and forward the clamped value to the native setter, so the authored gain survives while nothing outside the probe ever sees an illegal volume. Measured on one 6 s composition, first 4 s: unity -32.8 LUFS, boosted-with-fade -32.8 before and -27.0 after — +5.8 dB, exactly the gain the clip was authored at. One ceiling, defined once in `audioGain.ts` and reachable from both sides: the render mixer imports it, and the page-serialized probe takes it as a parameter rather than re-literalling it. User volume stays spec-clamped — it is a fader, not a gain. Also holds the percent volume slider above unity in both property panels. That control tops out at 100%, so one touch would cap a boosted clip and drop up to 12 dB that now genuinely renders; the dB fader that can represent these levels replaces it in the next PR. * fix(audio): carry a static above-unity gain onto the preview gain node Review follow-up. `setElementVolume` receives the clip's author gain and clamped it to [0,1], so a static `data-volume` above unity was capped on the WebAudio preview path while the render honoured it — the exact preview/render divergence this ceiling exists to close. Automation lanes hid it: they schedule ramps onto the param directly and never pass through here. The master volume beside it stays spec-clamped, because a user fader is not a gain. Verified by mutation: restoring the [0,1] clamp reds the new case. Also scope the leveller's rationale to this rung — `VOLUME_RANGE` still stops at unity until the dB fader lands, so "both now span the same range" was premature — and say why the GSAP-tracking fallback is unity-capped: it reads back through `el.volume`, which the spec pins to [0,1], so it cannot observe an above-unity value however wide the clamp gets. * fix(audio): restore the live test files this branch overwrote, and uncap preview Review blocker: three files were wholesale copies from the abandoned #3304 branch laid over a two-day-newer base, so they silently reverted work that had landed in between. CI could not see it — deleted tests do not fail. - `audioMixer.test.ts` was byte-identical to #3304's head: 1186 lines against a base of 1353. Gone with it were the `data-playback-start` fallthrough cases from #3322 — merged 54 minutes before this branch's own merge base — and all retiming coverage (`playbackRate` 7 to 0, `atempo` 5 to 0), the strict literal-timing table, and the zero-window cases. - `mediaVolumeEnvelope.test.ts` dropped the trailing-garbage duration case and the plateau-retention case. - `packages/core/package.json` rolled the package version back 0.8.3 to 0.7.109. All three are restored from `main` with only this PR's additions re-applied on top, and the subpath export is regenerated by the repo's own script rather than hand-edited. Also closes the preview/render split the same review raised. Two clamps had to go, not one: `setElementVolume` capped the author gain at the transport, and the first-tick branch in `syncRuntimeMedia` trusted `el.volume` — which is spec-bound to [0,1] and so cannot represent a boost, opening a boosted clip at 0 dB for one tick before the steady-state branch took over. Both pinned by tests, both verified by mutation. |
||
|
Miguel Ángel
|
3e4b08cdc1 | chore: release v0.8.3 (#3327) | ||
|
Miguel Ángel
|
afafca4b96 |
feat: make creator media edits render-safe (#3322)
* feat: make creator media edits render-safe * fix: align media playback timing * docs: add creator editing recipes * docs: expand creator editing guidance * fix: unify media source offsets * fix: scale natural media duration * fix: preserve natural media zero spans * fix: align compiled natural media timing * test: classify compiler media test as integration * fix: drop inactive media windows * fix: unify literal timing parsing * fix: keep browser media parsing serializable * fix: keep page timing readers strict * fix: close remaining preview timing gaps * fix(core): preserve Studio voice pitch at playback speed * chore: keep creator contract source-neutral |
||
|
Miguel Ángel
|
049f5618d7 | chore: release v0.8.2 (#3324) | ||
|
Miguel Ángel
|
ad84b00c90 | chore: release v0.8.1 (#3319) | ||
|
James Russo
|
232686f7e0 | chore: release v0.8.0 (#3318) | ||
|
Miguel Ángel
|
4403b8beef | chore: release v0.7.111 (#3315) | ||
|
Miguel Ángel
|
5e36f7ac54 | chore: release v0.7.110 (#3303) | ||
|
Miguel Ángel
|
12fd6d9087 | chore: release v0.7.109 (#3273) | ||
|
James Russo
|
f7d2260f9d |
feat(engine): stamp rendered files with hidden renderer provenance (#3264)
* feat(engine): stamp rendered files with hidden renderer provenance * fix(engine,producer): re-assert provenance at every container writer Review found that a no-audio MOV render still shipped untagged. The concat step is the last container write on that path (mux is skipped without audio, and applyFaststart only copies mov/webm), and the concat demuxer does not carry the chunks' container metadata through. The same hole applies to no-audio WebM, and to the in-process chunked encode in chunkEncoder, not just the distributed assemble path. mp4 was masked throughout because applyFaststart re-runs ffmpeg for that format and re-tagged the output. Tags the four remaining writers: the chunked-encode concat, and assemble's single-chunk remux, concat and cfr re-encode. Also corrects the trust claim. These are unsigned, freely writable keys, so a present tag means the file claims to be HyperFrames output, not that HyperFrames wrote it. Documented as an unauthenticated diagnostic hint rather than an authenticity or attribution boundary. Tests assert on the assembled file through the real assemble() path for both mov and webm; both fail without the concat fix. * test(engine): pin provenance through the in-process chunked concat Review noted the distributed writers are mutation-pinned but the encodeFramesChunkedConcat fix had no real-file regression of its own. Encodes 70 frames at a 30-frame chunk size so the concat step actually runs, then asserts the tags on the resulting no-audio mov. Fails without the concat fix, passes with it. |
||
|
Vance Ingalls
|
9ba528914d | chore: release v0.7.108 (#3265) | ||
|
Vance IngallsandClaude Opus 5
|
b839dbd2cc |
fix(core): give the chorus and phaser LFOs a phase, and unwire them on dispose (#3183)
* fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. * fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a preview-only commit that useDomEditAttributeCommits.ts never grew — backported that option support from its own later commit so the two sides of the API agree. The paste path and its tests were missing the box selection's v0/v1 bounds a sibling commit added to AutomationSelection. The FX panel's carve controls still edited the six mechanism numbers (maxCutDb, bands, intelligibilityBias) after carveProfile() collapsed authoring to one Strength knob, so those fields no longer existed on HfCarveSettings; UI now edits strength, and analyseCarveBands is called with carveProfile(strength). Also closes fallow's complexity, dead-code and duplication findings on this PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math) and useAutomationRangeDrag.ts (the marquee-select gesture) out of useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a resolver into named functions, dropped an export nothing outside its file used, and shared a step-simplifier between audioCarve's two envelope builders. The edge-stretch vs. box-select priority test in TimelineAutomationLane.test was still pinning the pre-box-select rule (edge wins over a point sitting on it) that a sibling commit deliberately reversed — a point inside the box is now selected content, so grabbing it drags the group instead. Updated the test to the shipped rule instead of the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): cap the via conic's weight so an edge-clamped via point can't NaN A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to (0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0. viaConic divided by that zero to get an infinite weight, and shapeVia turned Infinity into NaN a few steps later (Infinity - Infinity in the quadratic coefficient). NaN reaching setValueCurveAtTime silences the automated parameter for the rest of the render. Capped the weight at 1e6 instead of leaving it unbounded — past that point the arc already reads as touching the via point, so nothing visible is lost. Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`, since NaN fails the original comparison and fell through it. Review by Miga (PR #3208). * fix(studio-server): fingerprint the proactive waveform cache key too The route already keys the waveform cache on the asset's size and mtime as well as its path, so a rebuilt-in-place file gets fresh peaks instead of stale ones. generateWaveformCache — the proactive path that runs on upload — still called buildWaveformCacheKey with the path alone, so it wrote to a different key than the route reads from (making the pre-generated cache never found) and kept the exact collision bug this fingerprint exists to fix on its own path. Review by Miga (PR #3211). * style(docs): run oxfmt on the /hyperframes-audio skill docs Table column widths had drifted out of alignment with oxfmt's own rules, failing format:check and blocking the Preflight gate every downstream branch inherits. Whitespace only, no content change. * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). * fix(studio): widen PropertyPanel's resetModules render timeout again The 20s margin (already once widened for the same reason) is timing out in CI's full-monorepo Test run — the resetModules()+fresh-import render this test needs is uncached and competes with every other package's test suite for the same worker pool, and the same test passes in well under 2s standalone. Went to 45s rather than re-tuning to whatever number happens to clear the current CI load, since that number moves every time CI gains a package. * fix(studio): stop the single-candidate auto-apply carve firing twice Two auto-apply effects both fire when sourceOptions.length === 1: the multi-candidate effect only guards length === 0, so a single candidate passes it too, and the single-candidate effect passes its own guard right after — both compute the same sources list and both call setCarve, so the common case (one narrator, one bed) triggered two decodes, two FFT runs, and two concurrent attribute writes for one decision. The multi-candidate effect now defers to its sibling for exactly one candidate, which already has its own detailed handling for that case. Review by Miga (PR #3213). * feat(core): carve against every voice over a bed, always (#3212) * feat(core): carve against every voice over a bed, always dynamically A bed usually runs under a whole sequence — a narrator, an interview answer, a second presenter — and carving against one of them left the others fighting it. `source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's clock before anything is measured. That is what keeps one analysis sufficient: the chain is fixed, so there is no per-voice filter to switch between, and bands drawn from all the speech there is with envelopes that rise wherever any of it happens answer the actual question — where and when is speech masking this bed. Summed rather than averaged: two people talking at once mask more than either alone. Audio before the bed starts is dropped rather than folded in at zero, since it plays over nothing and shifting it would put a cut where there is no voice. `dynamic` is gone. A fixed depth thins the bed through every pause, and once both have been heard there is no reason to want it, so every carve follows the speech. Two helpers the panel and the headless script now share instead of each carrying a copy — two definitions of "what does this name suggest" drift, and then the two disagree about which track is the voice: - `classifyAudioName` reads a track's kind from its id and filename together. `unknown` is deliberately common: treating an unrecognised name as "not a voice" would hide the one track somebody needs to pick. - `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten duration counts as unbounded, not zero — refusing a clip whose length the composition leaves to the media would drop the commonest case there is. Files written before this still load: a single `source` reads as a one-voice list, a stored `dynamic` is ignored, and an absent attribute means the defaults whole. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): port the carve UI off the removed source/dynamic fields #3212 (accidentally squash-merged into this branch instead of main) changed HfCarveSettings from a single `source` + `dynamic` toggle to a `sources` list with dynamic mode removed outright — the multi-voice UI consumer that goes with that shape lands in the very next PR, so this branch was left with a type that no longer matched its own code. Minimal port, not the multi-voice redesign that PR does properly: the "Listen to" picker and analyse() treat sources[0] as the one voice this UI still understands, and every dynamic-mode branch (the automated envelope lanes, the toggle, the checkbox) is gone along with the field — a carve is now always the static value the analysis computes, matching what the type change made permanent. Test suite trimmed the same way: the automation-lane and toggle tests covered behavior that no longer exists. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
ea0344122c |
fix(engine): duck before quantising, chunk the PCM, reschedule on rate change (#3174)
* fix(core): remove the build-audio-fx-runtime.ts stray resurrected by a main merge An earlier merge with main brought this deleted file back (git's merge/delete handling on an unchanged-on-one-side file); package.json already points at build-inline-artifact.ts, so it sat unreachable and duplicating that file's config, both of which fallow flagged. * fix(studio): pull TimelineLanes under the 600-line cap TimelineLanes.tsx hit 620 lines. Extracted the three per-clip pointer gestures (resize-start, pointer-down move-arm, click/razor-split) into createClipGestureHandlers — one factory call per rendered clip instead of ~120 lines of inline handler bodies in the render loop. 529 lines now. * fix(studio): split the extracted pointerdown handler under the CRAP threshold Moving the ~120-line gesture logic into timelineClipGestureHandlers.ts concentrated it into two functions fallow flagged (onPointerDown at CRAP 63.6, onResizeStart at 31.6). Split the decision logic (which gesture a pointerdown implies) into a pure resolvePointerDownAction, then split its own intent-blocking check into isIntentBlocked. onResizeStart's guard moved into canStartResize. Every function now scores under 30. * fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. * fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a preview-only commit that useDomEditAttributeCommits.ts never grew — backported that option support from its own later commit so the two sides of the API agree. The paste path and its tests were missing the box selection's v0/v1 bounds a sibling commit added to AutomationSelection. The FX panel's carve controls still edited the six mechanism numbers (maxCutDb, bands, intelligibilityBias) after carveProfile() collapsed authoring to one Strength knob, so those fields no longer existed on HfCarveSettings; UI now edits strength, and analyseCarveBands is called with carveProfile(strength). Also closes fallow's complexity, dead-code and duplication findings on this PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math) and useAutomationRangeDrag.ts (the marquee-select gesture) out of useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a resolver into named functions, dropped an export nothing outside its file used, and shared a step-simplifier between audioCarve's two envelope builders. The edge-stretch vs. box-select priority test in TimelineAutomationLane.test was still pinning the pre-box-select rule (edge wins over a point sitting on it) that a sibling commit deliberately reversed — a point inside the box is now selected content, so grabbing it drags the group instead. Updated the test to the shipped rule instead of the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): cap the via conic's weight so an edge-clamped via point can't NaN A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to (0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0. viaConic divided by that zero to get an infinite weight, and shapeVia turned Infinity into NaN a few steps later (Infinity - Infinity in the quadratic coefficient). NaN reaching setValueCurveAtTime silences the automated parameter for the rest of the render. Capped the weight at 1e6 instead of leaving it unbounded — past that point the arc already reads as touching the via point, so nothing visible is lost. Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`, since NaN fails the original comparison and fell through it. Review by Miga (PR #3208). * fix(studio-server): fingerprint the proactive waveform cache key too The route already keys the waveform cache on the asset's size and mtime as well as its path, so a rebuilt-in-place file gets fresh peaks instead of stale ones. generateWaveformCache — the proactive path that runs on upload — still called buildWaveformCacheKey with the path alone, so it wrote to a different key than the route reads from (making the pre-generated cache never found) and kept the exact collision bug this fingerprint exists to fix on its own path. Review by Miga (PR #3211). * style(docs): run oxfmt on the /hyperframes-audio skill docs Table column widths had drifted out of alignment with oxfmt's own rules, failing format:check and blocking the Preflight gate every downstream branch inherits. Whitespace only, no content change. * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). * fix(studio): widen PropertyPanel's resetModules render timeout again The 20s margin (already once widened for the same reason) is timing out in CI's full-monorepo Test run — the resetModules()+fresh-import render this test needs is uncached and competes with every other package's test suite for the same worker pool, and the same test passes in well under 2s standalone. Went to 45s rather than re-tuning to whatever number happens to clear the current CI load, since that number moves every time CI gains a package. * fix(studio): stop the single-candidate auto-apply carve firing twice Two auto-apply effects both fire when sourceOptions.length === 1: the multi-candidate effect only guards length === 0, so a single candidate passes it too, and the single-candidate effect passes its own guard right after — both compute the same sources list and both call setCarve, so the common case (one narrator, one bed) triggered two decodes, two FFT runs, and two concurrent attribute writes for one decision. The multi-candidate effect now defers to its sibling for exactly one candidate, which already has its own detailed handling for that case. Review by Miga (PR #3213). * feat(core): carve against every voice over a bed, always (#3212) * feat(core): carve against every voice over a bed, always dynamically A bed usually runs under a whole sequence — a narrator, an interview answer, a second presenter — and carving against one of them left the others fighting it. `source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's clock before anything is measured. That is what keeps one analysis sufficient: the chain is fixed, so there is no per-voice filter to switch between, and bands drawn from all the speech there is with envelopes that rise wherever any of it happens answer the actual question — where and when is speech masking this bed. Summed rather than averaged: two people talking at once mask more than either alone. Audio before the bed starts is dropped rather than folded in at zero, since it plays over nothing and shifting it would put a cut where there is no voice. `dynamic` is gone. A fixed depth thins the bed through every pause, and once both have been heard there is no reason to want it, so every carve follows the speech. Two helpers the panel and the headless script now share instead of each carrying a copy — two definitions of "what does this name suggest" drift, and then the two disagree about which track is the voice: - `classifyAudioName` reads a track's kind from its id and filename together. `unknown` is deliberately common: treating an unrecognised name as "not a voice" would hide the one track somebody needs to pick. - `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten duration counts as unbounded, not zero — refusing a clip whose length the composition leaves to the media would drop the commonest case there is. Files written before this still load: a single `source` reads as a one-voice list, a stored `dynamic` is ignored, and an absent attribute means the defaults whole. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): port the carve UI off the removed source/dynamic fields #3212 (accidentally squash-merged into this branch instead of main) changed HfCarveSettings from a single `source` + `dynamic` toggle to a `sources` list with dynamic mode removed outright — the multi-voice UI consumer that goes with that shape lands in the very next PR, so this branch was left with a type that no longer matched its own code. Minimal port, not the multi-voice redesign that PR does properly: the "Listen to" picker and analyse() treat sources[0] as the one voice this UI still understands, and every dynamic-mode branch (the automated envelope lanes, the toggle, the checkbox) is gone along with the field — a carve is now always the static value the analysis computes, matching what the type change made permanent. Test suite trimmed the same way: the automation-lane and toggle tests covered behavior that no longer exists. * fix(cli): stop render.test.ts from downloading a real browser The "render command explicit composition" test drives the full render.js command handler, which takes the plan-based execute.ts path instead of the renderLocal path the other tests in this file exercise. That path calls ensureBrowser directly, bypassing the mocked preflight.js, and performs a real network install of chrome-headless-shell into the shared ~/.cache/hyperframes/chrome cache as a side effect of running the test suite. In CI this raced with the engine's audioFxRender browser tests running in a parallel worker against the same HOME, producing an intermittent EACCES on the partially-installed binary. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
95751d6b10 |
fix(audio): eleven small correctness fixes across engine, core and the panel (#3173)
* fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo Both effect builders set wet.gain to the mix and dry.gain to its complement in identical two-line blocks; fallow kept re-flagging it as a 10-line clone on every unrelated change. Extracted setWetDryMix. * fix(core): remove the build-audio-fx-runtime.ts stray resurrected by a main merge An earlier merge with main brought this deleted file back (git's merge/delete handling on an unchanged-on-one-side file); package.json already points at build-inline-artifact.ts, so it sat unreachable and duplicating that file's config, both of which fallow flagged. * fix(studio): pull TimelineLanes under the 600-line cap TimelineLanes.tsx hit 620 lines. Extracted the three per-clip pointer gestures (resize-start, pointer-down move-arm, click/razor-split) into createClipGestureHandlers — one factory call per rendered clip instead of ~120 lines of inline handler bodies in the render loop. 529 lines now. * fix(studio): split the extracted pointerdown handler under the CRAP threshold Moving the ~120-line gesture logic into timelineClipGestureHandlers.ts concentrated it into two functions fallow flagged (onPointerDown at CRAP 63.6, onResizeStart at 31.6). Split the decision logic (which gesture a pointerdown implies) into a pure resolvePointerDownAction, then split its own intent-blocking check into isIntentBlocked. onResizeStart's guard moved into canStartResize. Every function now scores under 30. * fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. * fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a preview-only commit that useDomEditAttributeCommits.ts never grew — backported that option support from its own later commit so the two sides of the API agree. The paste path and its tests were missing the box selection's v0/v1 bounds a sibling commit added to AutomationSelection. The FX panel's carve controls still edited the six mechanism numbers (maxCutDb, bands, intelligibilityBias) after carveProfile() collapsed authoring to one Strength knob, so those fields no longer existed on HfCarveSettings; UI now edits strength, and analyseCarveBands is called with carveProfile(strength). Also closes fallow's complexity, dead-code and duplication findings on this PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math) and useAutomationRangeDrag.ts (the marquee-select gesture) out of useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a resolver into named functions, dropped an export nothing outside its file used, and shared a step-simplifier between audioCarve's two envelope builders. The edge-stretch vs. box-select priority test in TimelineAutomationLane.test was still pinning the pre-box-select rule (edge wins over a point sitting on it) that a sibling commit deliberately reversed — a point inside the box is now selected content, so grabbing it drags the group instead. Updated the test to the shipped rule instead of the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): cap the via conic's weight so an edge-clamped via point can't NaN A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to (0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0. viaConic divided by that zero to get an infinite weight, and shapeVia turned Infinity into NaN a few steps later (Infinity - Infinity in the quadratic coefficient). NaN reaching setValueCurveAtTime silences the automated parameter for the rest of the render. Capped the weight at 1e6 instead of leaving it unbounded — past that point the arc already reads as touching the via point, so nothing visible is lost. Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`, since NaN fails the original comparison and fell through it. Review by Miga (PR #3208). * fix(studio-server): fingerprint the proactive waveform cache key too The route already keys the waveform cache on the asset's size and mtime as well as its path, so a rebuilt-in-place file gets fresh peaks instead of stale ones. generateWaveformCache — the proactive path that runs on upload — still called buildWaveformCacheKey with the path alone, so it wrote to a different key than the route reads from (making the pre-generated cache never found) and kept the exact collision bug this fingerprint exists to fix on its own path. Review by Miga (PR #3211). * style(docs): run oxfmt on the /hyperframes-audio skill docs Table column widths had drifted out of alignment with oxfmt's own rules, failing format:check and blocking the Preflight gate every downstream branch inherits. Whitespace only, no content change. * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). * fix(studio): widen PropertyPanel's resetModules render timeout again The 20s margin (already once widened for the same reason) is timing out in CI's full-monorepo Test run — the resetModules()+fresh-import render this test needs is uncached and competes with every other package's test suite for the same worker pool, and the same test passes in well under 2s standalone. Went to 45s rather than re-tuning to whatever number happens to clear the current CI load, since that number moves every time CI gains a package. * fix(studio): stop the single-candidate auto-apply carve firing twice Two auto-apply effects both fire when sourceOptions.length === 1: the multi-candidate effect only guards length === 0, so a single candidate passes it too, and the single-candidate effect passes its own guard right after — both compute the same sources list and both call setCarve, so the common case (one narrator, one bed) triggered two decodes, two FFT runs, and two concurrent attribute writes for one decision. The multi-candidate effect now defers to its sibling for exactly one candidate, which already has its own detailed handling for that case. Review by Miga (PR #3213). * feat(core): carve against every voice over a bed, always (#3212) * feat(core): carve against every voice over a bed, always dynamically A bed usually runs under a whole sequence — a narrator, an interview answer, a second presenter — and carving against one of them left the others fighting it. `source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's clock before anything is measured. That is what keeps one analysis sufficient: the chain is fixed, so there is no per-voice filter to switch between, and bands drawn from all the speech there is with envelopes that rise wherever any of it happens answer the actual question — where and when is speech masking this bed. Summed rather than averaged: two people talking at once mask more than either alone. Audio before the bed starts is dropped rather than folded in at zero, since it plays over nothing and shifting it would put a cut where there is no voice. `dynamic` is gone. A fixed depth thins the bed through every pause, and once both have been heard there is no reason to want it, so every carve follows the speech. Two helpers the panel and the headless script now share instead of each carrying a copy — two definitions of "what does this name suggest" drift, and then the two disagree about which track is the voice: - `classifyAudioName` reads a track's kind from its id and filename together. `unknown` is deliberately common: treating an unrecognised name as "not a voice" would hide the one track somebody needs to pick. - `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten duration counts as unbounded, not zero — refusing a clip whose length the composition leaves to the media would drop the commonest case there is. Files written before this still load: a single `source` reads as a one-voice list, a stored `dynamic` is ignored, and an absent attribute means the defaults whole. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): port the carve UI off the removed source/dynamic fields #3212 (accidentally squash-merged into this branch instead of main) changed HfCarveSettings from a single `source` + `dynamic` toggle to a `sources` list with dynamic mode removed outright — the multi-voice UI consumer that goes with that shape lands in the very next PR, so this branch was left with a type that no longer matched its own code. Minimal port, not the multi-voice redesign that PR does properly: the "Listen to" picker and analyse() treat sources[0] as the one voice this UI still understands, and every dynamic-mode branch (the automated envelope lanes, the toggle, the checkbox) is gone along with the field — a carve is now always the static value the analysis computes, matching what the type change made permanent. Test suite trimmed the same way: the automation-lane and toggle tests covered behavior that no longer exists. * docs(plans): fix pre-existing oxfmt formatting drift Blocks the regression workflow's required preflight gate. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
6a0e409fc2 |
feat(engine): let an FX tail decay instead of cutting it at the clip (#3036)
* feat(core): audio FX registry One declarative description of every effect that can be applied to an audio track: fourteen across filters, dynamics, non-linear and time, each exposing its full parameter surface rather than a curated subset. Parameters carry the range, step, unit and scale a control needs, so a panel can generate its UI from this rather than hard-coding a form per effect, and a value that survives `normalizeAudioFxParams` is always safe to realise. Everything is declared in the units a person thinks in — dB, ms, Hz. Parsing rejects an unknown effect id rather than skipping the node. A chain that quietly loses an effect renders something other than what was authored, which is worse than refusing to load it. Data only: no audio is produced here. The graph that realises each effect is referenced by the `web` id and lands in the next change, which keeps this module free of browser globals so the engine and the linter can import it. * fix(core): stop declaring knobs that move nothing Three parameters were declared with ranges, defaults and hints, and read by no builder — dials an author could turn with no audible result. - `chorus.decay` and `bitcrush.aa`: removed. FFmpeg's chorus feeds a decay back into its delay line and a bitcrusher's anti-alias needs a real filter; adding either is new DSP, not a fix, so the honest move is to stop advertising them. - `lowshelf.q` / `highshelf.q`: removed. The Web Audio spec leaves Q unused for shelving filters, so the control moved nothing — and because the shared Q helper marks it automatable, an author could draw an envelope on it and hear nothing at all. `phaser.decay` and `gate.knee` stay: the first drives the sweep depth, and the second is now read by the gate's processor. A test asserts each of these directly, since the existing exposure invariant only checks that a flagged parameter reaches an AudioParam — a parameter the node then ignores passes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): Web Audio graphs for the FX registry One graph builder per `web` id, turning the registry's declarations into running audio. Every node exposes `update`, so turning a dial re-parameterises the live graph rather than rebuilding it: an AudioParam change lands on the next 128-sample quantum, about 2.7 ms at 48 kHz. `buildFxChain` reports whether an update could be applied in place — adding or bypassing an effect, or switching a filter between one and two poles (which changes the node type from BiquadFilterNode to IIRFilterNode), changes the graph's shape and returns false so the caller rebuilds. Four effects have no native node and run as AudioWorklet processors: compressor, limiter, gate and bitcrush. The module is registered from a data: URL rather than a blob:, because a blob inherits the page origin and is opaque on a file:// page, where it fails with an unhelpful AbortError. Reverb has no single node either. `synthesizeReverbImpulse` generates a tail from the room parameters, seeded so the same room sounds the same on every machine, and the ConvolverNode uses it. Tests cover the wiring — which nodes get built, how they connect, parameter application and clamping, in-place update versus rebuild, disposal — against a fake AudioContext, since happy-dom has no Web Audio. * fix(core): reverb level, phaser wiring, per-channel dynamics, one-pole rebuild Four defects in the graph builders, all found by review rather than by ear. **Reverb was unusable at its own defaults.** A ConvolverNode applies the impulse's gain whole — the graph sets `normalize = false` so a room is deterministic rather than browser-defined — but the impulse was raw decaying noise. Measured L2 at the registry default (size 0.7 / damping 0.5): 46.4, or +33.3 dB, putting the wet path ~24 dB over dry at the default `wet: 0.35`. It is now normalised to unit energy, so the wet knob means what it says. Preview and render both convolve this buffer, so they stayed identical throughout — equally deafening before, equally correct now. **Phaser in_gain/out_gain trim the signal entering and leaving the effect**, not a wet/dry pair. Wired to the wet and dry legs, "Input" muted the dry path and the two defaults summed to 1.14, so inserting a phaser raised the track level. They are now input and output trims with the legs summed at unity. Its declared waveform is also honoured: `lfo.type` was never assigned, so the default "Triangular" was silently a sine. **The dynamics worklets held one envelope across a channel-major loop.** The followers advance per sample, so on stereo a 20 ms attack behaved as 10 ms, and the right channel's gain came from an envelope that had already traversed the left — the two ducked differently from the same input and the image pumped. State is now per channel, as is the gate's smoothed gain and bitcrush's sample-hold counter, which previously advanced only on the last channel and left every earlier one frozen for a whole quantum. The gate also honours the knee it declares instead of chattering on material sitting at the threshold. **A one-pole filter's cutoff was swallowed in preview.** Its coefficients are fixed at construction, so `update` cannot push a new frequency — but the shape signature carried only type and pole count, so a cutoff change looked like a values-only edit and went into a no-op updater. Preview kept filtering at the old frequency while the render used the new one: a preview/render divergence in exactly the two effects that do not use a BiquadFilterNode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * feat(core): register the audio-fx-rack canary at 0% Lands the rollout switch dark, per the registry's own procedure: "Start at percentage: 0 and merge that — a canary at 0 is dead code you can land safely and ramp without a code review." Declared at the bottom of the stack so every branch above can read it. The gate itself goes in at wa-4-fx-panel, where the rack first appears. Scope is deliberate and stated in the description: it gates the AUTHORING surface only. A composition that already carries `data-fx-chain` still plays and renders it. A canary should stage who can REACH a feature, not make an attribute somebody already wrote silently inert — an agent that writes a chain through the skill would otherwise produce a file whose audio processing vanishes with no error. * feat(studio): audio FX panel generated from the registry Controls for the whole chain: add, remove, reorder, bypass, and every knob each effect declares. Nothing in the panel knows what a compressor is. The registry supplies each parameter's range, step, unit and scale and the panel renders what it finds, so adding an effect or a knob upstream needs no change here, and the panel cannot offer a value the renderer would reject — a typed-in figure is clamped into the declared range on the way through. Frequency and time controls span three or four decades, so those declare a log scale and the slider maps exponentially; a linear slider would spend most of its travel somewhere useless. Reorder is a first-class control because chain order changes the sound: a reverb before a compressor is not the same as after. Carve gets its own block rather than an entry in the add menu, with a picker for the voice track to listen to. It processes this track based on another one, which is how a sidechain control works — it lives on the track that changes, and names the source. * feat(studio): show the Audio FX section on audio tracks Adds `audioFx` to the editing-affordances contract and renders the FX panel in the inspector when an `<audio>` element is selected. The section is audio-only. A `<video>` carries its sound on a separate `<audio>` element, so an FX chain on the video would have nothing to process. Chain and carve settings are written straight back onto the element as serialised attributes, the way colour grading carries its config, so persistence is an ordinary attribute write and needs no new server route. A chain that cannot be parsed renders as empty rather than breaking the panel, and the attribute is left untouched until the user changes something. The collapsed group summarises what is on the track ("2 effects + carve") so the state is visible without expanding it. Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED defaults to true, so the flat inspector is what actually renders. * refactor(studio): lift audioFxSummary out of PropertyPanelFlat `PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap, so the required File size check is red — the sole reason this PR is blocked. The review says as much: "mechanical fix (~5 min), not a design problem. Code itself is LGTM." Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later branch creates for it. Deliberately the smallest cut that clears the cap rather than the whole `AudioFxGroup` extraction: every later commit in the stack edits AudioFxGroup, so moving it here would collide with each of them, while almost nothing touches this function. 595 lines. * feat(studio): put the audio FX rack behind its canary Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered at 0% — so the whole 47-PR stack can land without showing anyone a feature that has not been measured yet. The gate sits on the AUTHORING surface and nowhere else. The runtime and the render still honour a `data-fx-chain` already on an element, so a composition written through the skill or by `carve.mjs` keeps its processing rather than going silently dry for anyone outside the cohort. A canary should stage who can REACH a feature, not make an attribute somebody already wrote stop working with no error. Gated at the panel rather than in `resolveEditingSections`: the affordance resolver is a pure function in core describing what an element CAN support, and rollout state is not a property of an `<audio>` tag. Pinned the 0% with a test, and checked it fails at 25 — a ramp should have to break something that says "this ships dark" out loud. One gap, stated rather than papered over: the gate itself has no unit test. I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness never renders the Audio FX group for its audio fixture even with the gate removed — so the test passed for the wrong reason in the off case and could not pass at all in the on case. A test that cannot fail for the right reason is worse than none. Verifying the gate needs the panel harness to mount that section first, which is its own change. * feat(core,studio): hear the FX chain in preview, and run the carve analysis Splices an element's FX chain into the playback graph so preview stops being silent about effects, and wires the carve button that was previously inert. The chain goes between the decoded source and its gain stage: effects see the raw signal and volume automation rides on their output, matching the order the offline render uses. Since preview and render call the same graph builders, what is heard while scrubbing is what gets written. The splice lives in the transport rather than on the `<audio>` element. The transport plays each track from a decoded AudioBuffer and mutes the element to avoid doubling, so capturing the element with createMediaElementSource would have processed a stream nothing is listening to — it looked like it worked because the call succeeded, and the audio was unchanged. A chain that cannot be built plays dry rather than silencing the track, which is the right failure in preview: the author keeps working and hears the source. The render still refuses, because shipping the dry signal there would be wrong. Carve now analyses for real: it decodes the chosen voice track, ranks its bands and writes the resulting peaking filters onto this track. Generated nodes are tagged `fromCarve`, so re-running replaces the previous carve instead of stacking another set on top of hand-added effects. Known limitation: the graph is built when a source is scheduled, so a knob turned mid-playback takes effect on the next play or seek rather than immediately. Live re-parameterisation needs the transport to hold the handle and forward updates. * fix(studio,core): stop parameter drags from restarting playback Dragging a knob wrote the chain through the persisting attribute path on every input event. That path refreshes the preview, which reloads the composition and reschedules audio — so a single drag reloaded dozens of times and playback stuttered the whole way. Drags now go through `onSetAttributeLive`, the same path colour grading uses for scrubs: it coalesces undo entries and sets `skipRefresh`, so no reload happens. The persisting write fires once, when the gesture ends — pointer-up or blur for a slider, Enter or blur for a typed value. A select commits immediately since there is no drag to wait for. While dragging, the control is driven from local state. Waiting for the value to round-trip through the element attribute made the knob lag behind the pointer. For the change to be audible without a reload, the graph now follows the attribute: the chain installed by the transport observes the element and re-parameterises itself in place, so a value change lands on the next 128-sample quantum. A shape change (effect added, bypassed, pole count) cannot be patched into a running graph, so it still waits for the next schedule rather than cutting the audio mid-play. The regression test drags a slider through several values and asserts the persisting handler is untouched until release. * fix(core): register FX worklets before building nodes that need them An AudioWorkletNode cannot be constructed before its processor is registered — it throws, and the surrounding chain is lost with it. `attachElementFxChain` built the chain first and only then called `ensureAudioFxWorklets`, so every worklet-backed effect (compressor, limiter, gate, bitcrush) threw on construction and the track fell back to dry. Instrumenting the preview showed `hf-compressor: InvalidStateError` with addModule never called at all. When the module has not landed yet the track now plays dry and the graph is swapped in once registration resolves, so the effect arrives a moment late instead of never. Registration is also tracked per context rather than in one module-level promise. A processor registered on one AudioContext does not exist on another, so the shared promise made every context after the first believe it was ready when it was not — the studio's transport owns its own context, which is exactly that case. With the worklets actually running, the compressor's per-sample log10 and pow became real audio-thread work. Samples below the knee have a gain of exactly unity and need neither, so the envelope is now compared in the linear domain and the transcendentals only run for samples that are actually being compressed. * refactor(studio): split the FX node row out of FxSection Clears the health findings the FX stack left behind: the chain-node render callback was a 70-line closure over half of FxSection's state, and the two reorder arrows were the same button written twice. Also drops two exports with no consumers, and registers the audio FX runtime stub as an entry point — it is bundled by file path, so nothing imports it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(studio): lift the audio FX group out of PropertyPanelFlat `PropertyPanelFlat.tsx` was 672 lines against the repo's 600-line cap, so the required File size check was red — the sole reason #3014 and #3022 are blocked. Both reviews say the same thing: "mechanical fix, not a design problem. Code itself is LGTM." Moves `AudioFxGroup` and `audioFxSummary` into `propertyPanelAudioFxGroup.tsx`, which is where a later branch puts them anyway — done here so the file is under the cap from the point it first crosses it, rather than ten branches later. 533 lines now. The four audio imports it no longer needs go with it. Not fixed here: three `FxSection carve` tests fail on this branch with "Cannot read properties of undefined (reading 'toFixed')". Confirmed pre-existing by stashing this change and re-running — that is the separate `Test` failure the review also flags. * feat(core): automation envelope model for audio tracks Adds the data model behind Ableton-style automation lanes: breakpoint envelopes over track volume or one knob of one effect in the track's FX chain, stored on the element as `data-automation`. Times are clip-local, so an envelope travels with the clip when it moves — the clip-envelope model rather than arrangement automation. `sampleAutomationLane` is the single interpolator. The lane drawing, the preview scheduler and the render bake all call it, so the picture and the sound cannot disagree about the curve. Log-scaled parameters interpolate in log space, matching what their own knob already promises. FX nodes gain a stable `id`, minted by count rather than randomly so the document is the same on every machine. Lanes address nodes by id, so reordering a chain never re-points a lane at a different effect, and a lane whose effect was deleted is dropped rather than left to reattach. Also warns when a track carries both a volume lane and a GSAP volume tween, since only the lane is heard and the tween silently does nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): expose the AudioParams behind automatable FX knobs Marks the knobs an automation lane can drive and has each graph builder hand back the AudioParam behind them, so a scheduler can write to a running effect without knowing what the effect is. A knob is not always one AudioParam. A wet/dry mix is two gains moving in opposition, and a knob in milliseconds drives a delay time in seconds, so each target carries the mapping out of the knob's own declared unit. What stays unautomatable is stated where it is decided: a WaveShaper curve, a convolution impulse and a one-pole filter's coefficients are all rebuilt wholesale rather than scheduled, and the four worklet effects take values by postMessage rather than through AudioParams. The registry flag is written by hand, so a test builds every effect and checks the exposure both ways — nothing flagged is missing, nothing exposed is unflagged. A flag that lied would offer a lane that silently did nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): play automation envelopes in preview Schedules each lane onto the AudioParams behind its knob using native ramps and value curves. Nothing evaluates the envelope per frame: it is handed to the audio thread once, so it stays sample-accurate however busy the main thread is, and the offline render will schedule it the same way. Timing comes from the transport, so an envelope survives seeking into the middle of a clip, a clip that has not started yet, and a playback rate that compresses clip seconds into context seconds. A straight line is only scheduled as a ramp when nothing bends it — no curvature, a linear parameter scale, and no unit mapping. Log-scaled parameters and mapped ones are sampled instead, since a delay knob in milliseconds and a wet/dry pair moving in opposition are not linear in the parameter they drive. Lanes with nowhere to write are skipped rather than reported: a one-pole filter exposes no frequency param, and the worklet effects expose none at all. Editing an envelope mid-playback re-aims it at the live playhead rather than restarting the track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(engine): bake automation envelopes into the render The offline render schedules FX lanes with the same scheduler preview uses, inside the OfflineAudioContext that already runs the same graph builders. The input WAV is the clip's own audio from its first sample, so clip-local time is offline time and the envelope needs no offset. Volume lanes take the existing PCM bake rather than a second mechanism: the lane is converted to keyframes, so a straight fade stays two of them and only a bent segment is sampled — the baker interpolates linearly and would otherwise quietly straighten the curve. A volume lane supersedes keyframes probed from the timeline, which `lint` already warns about. A browser test sweeps a lowpass from below a 2 kHz tone to well above it and measures both ends. Parsing the envelope is not the same as scheduling it, and only running the real thing tells the two apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): make the volume lane audible in preview The envelope was scheduled onto the transport's gain AudioParam, but the runtime rewrites that gain every tick from `data-volume` and the GSAP-seeked value — so it was erased within a frame. Volume automation was correct in the render and inaudible while previewing. The lane now feeds the per-tick path where the probed volume keyframes already sit, checked ahead of them so the two cannot fight, and the transport no longer schedules volume at all: one mechanism instead of two racing. The cost is honest — in preview the level steps per tick rather than per sample, exactly as the existing keyframe path does. The render still bakes it into the PCM sample-accurately, and FX parameters are still scheduled on their own AudioParams, since nothing rewrites those. Parsed lanes are cached by attribute text: the runtime asks once per tick per track, and parsing there would run the JSON parser 60 times a second for a value that only changes on an edit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): apply chain edits to the running graph A structural edit — an effect added, removed, bypassed, or a filter's pole count switched — was dropped. `buildFxChain`'s update reports false when the change is not merely new values, and the attribute observer ignored that, so the edit only took hold when the persisting write reloaded the composition. That reload restarted every playing track, which is what was heard as the audio chopping. The graph is now swapped in place: the old effects are detached, the new ones built and connected between the same source and gain, and any lanes re-scheduled onto the new nodes. The source node is never touched, so playback does not restart. A track with no chain is watched too, rather than wired through and forgotten, so adding its first effect is heard the same way. That means the function always returns a disposer instead of null for the empty case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(studio): the geometry and plumbing behind automation lanes Everything an automation lane needs before there is a lane to look at, kept apart from the component so the maths can be read and tested without a pointer. `automationLaneGeometry` is pure: which parameters a clip can automate (its fader, then each automatable knob of each effect that carries a node id), how a value maps to a position in the lane, and how a lane is edited. Two decisions live here and are worth review: - A log-read knob maps on its own log scale, so the middle of a 100 Hz–20 kHz lane is the geometric mean. Dragging and drawing then agree with what the knob's own scale already promises. - `withLane` replaces a lane in place rather than appending. A lane with no explicitly chosen parameter shows whichever comes first, so moving the edited one to the end would switch the lane out from under the pointer on the first edit. `automationLaneData` parses the two attributes, cached by their text so the identity only changes when the text does — the lane holds an optimistic draft while a point is dragged and compares against that identity, and a fresh object on every playhead tick would throw the drag away. It binds automation to the chain the way preview and the render bind it, so a lane whose effect was deleted is dropped rather than drawn against the wrong axis. `useAutomationLanes` routes edits through the DOM edit session, targeting the selected element because that is what the attribute commit path writes to. Row height reserves each lane at its own height rather than counting it as another keyframe lane, and `TimelinePropertyLanes` gains a footer slot so the lanes share the keyframe disclosure — and its `aria-controls`. `TimelineElement` moves to its own module: playerStore had reached the 600-line studio ceiling exactly and could not carry another field. It is re-exported from there, so no importer changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) * fix(studio): make automationLaneData reviewable, and evict one entry not all The cache key held a literal NUL byte instead of its escape, so git classified the whole module as binary: it landed as `Bin 0 -> 2677 bytes` with zero diffable lines, invisible to review, to grep, and to any textual merge. The escape is behaviour-identical. With the file readable, two things in it needed fixing. Eviction cleared the entire map. Clearing changes the identity of every lane's automation at once, and a lane compares its drag draft against that identity — so one unrelated element arriving at the limit would release an in-progress drag and snap the point back. It now drops the oldest entry, and a hit is re-inserted so it counts as recently used. Nothing tested this module, which is what let the binary blob through. Now covered: identity stability, re-parsing when the chain changes but the automation text does not, a hot entry surviving 40 evictions, and an unreadable attribute reading as nothing. The geometry module's exports are ignored for dead-code while its consumer sits one PR upstack, following the convention already used for the fast-capture stack. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): carry the audio FX attributes onto every timeline row Both element builders read `data-fx-chain` and `data-automation` off the host element, and an expanded sub-composition child is built without one — so an audio track inside a sub-composition reserved no automation height and drew no lanes, while the property panel, which reads the live DOM selection rather than the row, still showed its chain and its toggles. `hostElementState` exists to re-inherit exactly this class of host-only field; it now covers these two alongside `hidden`, `timelineLocked` and `timelineRole`. `parseTimelineFromDOM` had the same gap and now reads both directly. Also exempts the offline FX render's browser entry from the health gate: it runs only inside the headless page the engine drives, so its CRAP score is coverage-driven rather than complexity-driven, and its behaviour is covered by the engine's real-browser render tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(studio): the automation lane itself Draws each automated parameter as its own lane under the audio clip, on the same disclosure caret the keyframe lanes use — that caret is the DAW automation triangle. One lane per parameter rather than a selector to swap between them, so two envelopes can be read and edited without hiding either. Double-click the line to add a point, drag to shape it, right-click a point to remove it. Three things here took more than one attempt, and the comments say why: - **A dragged point did not move.** The live write deliberately skips the preview refresh — that is what keeps dragging from restarting playback — so the stored value does not move under the pointer. The lane keeps a local draft. - **Releasing snapped it back.** The draft was dropped when the drag ended, which is before the persisted write comes around; it now lives until the automation it was drawn over actually changes. - **A press was eaten.** Not stopping propagation let the timeline start its own gesture and swallow the second half of a double-click. The lane owns the press once it is live — and when it is not, it selects its clip instead, since lanes sit below the clip bar where the timeline's own selection handler never sees them. The envelope is inset by the grab radius so a point at the clip's first or last frame is drawn whole rather than half outside the lane, and clip time still lines up with screen position because the inset and the offset cancel. 🤖 Generated with [Claude Code](https://claude.com/claude-code) * fix(studio): let a track disclose its automation without a tween The lane was mounted inside the property-lanes wrapper, which renders only for a track's GSAP keyframe clip — so an audio clip with no tween resolved to nothing: no disclosure caret, no reserved height, no lanes. Verified on a composition with one `<audio>`, an envelope, and no tweens anywhere: 0 carets, 0 lanes. The attribute still wrote and the render still baked it, so the feature failed silently for exactly the tracks it exists for. Same composition now: 1 caret, and expanding it draws the Volume lane. Automation counts as something to disclose. `resolveTrackKeyframeClip` takes a counter alongside the keyframe lane counts and qualifies a clip on either; the header asks the same counter about the clip it already holds. A function rather than another map threaded through the props: every caller then reads one cached parse, so the height a row reserves and the lanes drawn in it cannot drift apart. That drift is also fixed for the lane's own offset, which passed the raw tween count where every other consumer uses distinct property groups. Two tweens on one property drew one keyframe lane but pushed the automation lane down by two, spilling into the next track; one tween on two properties did the inverse and drew it over a diamond lane, stealing its pointer events. It now reads the same `laneCounts` map the reserved height and the drawn lanes use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(studio): automate a parameter without reloading the preview The write path and the volume half of the panel surface. **A commit that persists without reloading.** For attributes the runtime applies to the live graph itself — an FX chain, its automation — a reload would only interrupt playback to reach the state the preview already has. `skipRefresh` and `refreshAfter` were already independent options; this exposes the combination that skips the reload but still re-reads the selection. Both halves are needed, and they were fighting each other. Without the reload, audio no longer chops on an edit. Without the resync, the panel keeps reading the selection snapshot it was built with, so a second edit computes from a pre-edit value and appears to do nothing — deleting one effect made every later delete a no-op. `handleDomAttributeLiveCommit` is untouched and still used for knob dragging, where a per-move re-render is exactly what you do not want. **Volume.** An automated track's slider is disabled, since a level set there would be overwritten by the envelope on the next tick, and the toggle beside it adds or deletes the lane. Adding seeds it with a single point at the level the slider already shows, so automating a track never changes how loud it is. **One shared reader** for both panel sections, which is what surfaced that resolving against an absent chain would have deleted every FX lane the moment someone automated a volume: the volume section does not parse the chain, so "no chain" now means "do not resolve" rather than "drop what cannot be resolved". The toggle itself lives with the FX controls it is shared with, and says `Automated` / `Automate` through the studio's own Tooltip rather than a native browser hover. 🤖 Generated with [Claude Code](https://claude.com/claude-code) * fix(studio): stop the last two automation writes reloading the preview The quiet commit added here was only used by the FX group. Two writers still went through the refreshing one, so they reloaded the preview and restarted every playing track — the exact chop the live write during a drag exists to avoid: - releasing a dragged breakpoint, so the audio hitched at the end of every point you moved; - clicking the volume toggle, while the same click on an effect parameter was already silent. Both are quiet now: still persisted, still resyncing the selection so a following edit computes from the value just written. Also fixes the seeded volume. `Number(dataAttributes.volume ?? "1")` is 0 for an attribute that is present but empty, so automating such a track started its lane at silence while the engine read the same empty value as unity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(studio): automate and un-automate each effect parameter The per-parameter surface in the FX panel. An automated parameter's control is disabled — a value typed there would be overwritten by the envelope on the next tick, so the lane is the value now — and the toggle beside it adds or deletes that parameter's lane. Adding seeds the lane with a single point at the value the control already holds, so switching to an envelope never changes the sound, only where the value comes from. Parameters no envelope can drive have no toggle at all: the worklet-backed dynamics expose no AudioParams, a WaveShaper's curve and a convolution impulse are rebuilt wholesale rather than scheduled. Neither does a chain node with no id, since a lane addresses nodes by id — so adding an effect now mints one. Carve moves onto the same non-reloading write, and decodes its source in an `OfflineAudioContext`: opening a second output device mid-playback makes the running track glitch while the hardware is reconfigured. Turning carve off now also drops the filters it generated, which otherwise kept dipping the bed with nothing in the panel to explain it. `AudioFxGroup` moves into its own module — PropertyPanelFlat was at its size budget — which also gave the panel's write behaviour somewhere to be tested: what it writes, seeded at the current value, preserving the lanes it is not touching, and clearing the attribute when the last one goes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) * feat(studio): only offer voiceover carve when there is a voice to carve against Carve is a relationship between two tracks — it analyses another track's voice and dips this bed where that voice sits. In a composition with a single audio track there is nothing to listen to, so the block offered an empty source picker and an Analyse button that could never do anything. It is now shown only when the composition holds another audio track, and still shown when carve is already configured: hiding a live setting because its voice track was removed would leave the bed being dipped from out of sight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(studio): cover carve visibility through the real element The panel derives carve's source list from the selected element's document, so a selection with no element has no sources — which the new visibility rule correctly reads as 'nothing to carve against'. The suite mounted exactly that, so it was asserting on a hidden block. Selections now carry a real <audio> with a sibling track, and the two cases the rule exists for are pinned directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): keep FX panel writes from clobbering each other Three writes in the audio panel each read the source file, mutate one attribute and write it back. Fired without ordering they read the same content and the last one lands, dropping the others. - Deleting an effect left its automation lanes in the attribute. Ids are minted lowest-free, so the next effect added took the same id and inherited the dead envelope: disabled and "Automated" without the author ever automating it, and baked into the render. - Switching carve off wrote the chain (dropping the filters it generated) and the carve settings at once, so either the filters stayed with no carve to explain them or the settings survived with no filters. - The three carve dials committed per input event, patching the source and resyncing the selection dozens of times per drag. They now preview live and persist on release, like the FX knobs already do. Volume automation reads through the quiet commit too, so removing a lane resyncs the panel instead of leaving the slider disabled. * feat(engine): let an FX tail decay instead of cutting it at the clip The offline render ended at the last input sample, so a reverb or a delay was still ringing when the context stopped. Measured on a 1.5 s tone through a default reverb, the render cut at 1.524 s while the tail was still at -29.7 dB — an audible chop, and the one place the render did not match preview. The length does not have to be guessed. Every tail here follows from its own settings: a convolution is exactly as long as its impulse, and `synthesizeReverbImpulse` derives that from room size; a delay's repeats fall by `feedback` every `time`, so the count down to -60 dB is a log. Everything else settles with its input — an all-pass chain has group delay, not a tail, and a 9-second compressor release has no signal to release once the clip stops. `chainTailSeconds` sums them (the chain is serial, so a delay in front of a reverb hands each repeat to the room), reads a lane's maximum rather than the static knob where one is automated, and caps at 5 s — 5 s between repeats at 0.95 feedback is eleven minutes of decay, and the panel can dial exactly that. The mixer's per-track atrim now allows the clip plus its tail; the atrim after apad still holds every track to the composition's length, so a tail can run over what follows but never extends the video. Same fixture after: a smooth decay to -72 dB, last non-zero sample at 3.306 s against the 3.4 s the settings predict. * feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * test(engine): probe ffmpeg and Chrome instead of assuming them Two failures on #3021's Test job, both about the environment rather than the code under test. **Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to `execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()` resolves — every other ffmpeg-dependent suite in this package already goes through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)` so a contributor without ffmpeg skips rather than fails. **The browser guard trusted the wrong thing.** It asked `resolveHeadlessShellPath()` and treated a returned path as "a browser is here". CI's cache holds a chrome-headless-shell that resolves and then fails to spawn — a partial download is indistinguishable from a working one by `existsSync`, which is all that resolver checks. So the three browser cases ran anyway and failed on the launch. It now runs `--version` and requires exit 0, which is the same probe the ffmpeg suites use: ask the binary, do not infer from the filesystem. Checked both directions rather than just the green one. With a working browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed at a binary that exits non-zero — CI's exact situation — exactly 3 skip and the other 8 still run. A guard that quietly skipped everything would have looked identical on the CI summary. * feat(core): register the audio-fx-rack canary at 0% Lands the rollout switch dark, per the registry's own procedure: "Start at percentage: 0 and merge that — a canary at 0 is dead code you can land safely and ramp without a code review." Declared at the bottom of the stack so every branch above can read it. The gate itself goes in at wa-4-fx-panel, where the rack first appears. Scope is deliberate and stated in the description: it gates the AUTHORING surface only. A composition that already carries `data-fx-chain` still plays and renders it. A canary should stage who can REACH a feature, not make an attribute somebody already wrote silently inert — an agent that writes a chain through the skill would otherwise produce a file whose audio processing vanishes with no error. * feat(studio): audio FX panel generated from the registry Controls for the whole chain: add, remove, reorder, bypass, and every knob each effect declares. Nothing in the panel knows what a compressor is. The registry supplies each parameter's range, step, unit and scale and the panel renders what it finds, so adding an effect or a knob upstream needs no change here, and the panel cannot offer a value the renderer would reject — a typed-in figure is clamped into the declared range on the way through. Frequency and time controls span three or four decades, so those declare a log scale and the slider maps exponentially; a linear slider would spend most of its travel somewhere useless. Reorder is a first-class control because chain order changes the sound: a reverb before a compressor is not the same as after. Carve gets its own block rather than an entry in the add menu, with a picker for the voice track to listen to. It processes this track based on another one, which is how a sidechain control works — it lives on the track that changes, and names the source. * feat(studio): show the Audio FX section on audio tracks Adds `audioFx` to the editing-affordances contract and renders the FX panel in the inspector when an `<audio>` element is selected. The section is audio-only. A `<video>` carries its sound on a separate `<audio>` element, so an FX chain on the video would have nothing to process. Chain and carve settings are written straight back onto the element as serialised attributes, the way colour grading carries its config, so persistence is an ordinary attribute write and needs no new server route. A chain that cannot be parsed renders as empty rather than breaking the panel, and the attribute is left untouched until the user changes something. The collapsed group summarises what is on the track ("2 effects + carve") so the state is visible without expanding it. Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED defaults to true, so the flat inspector is what actually renders. * refactor(studio): lift audioFxSummary out of PropertyPanelFlat `PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap, so the required File size check is red — the sole reason this PR is blocked. The review says as much: "mechanical fix (~5 min), not a design problem. Code itself is LGTM." Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later branch creates for it. Deliberately the smallest cut that clears the cap rather than the whole `AudioFxGroup` extraction: every later commit in the stack edits AudioFxGroup, so moving it here would collide with each of them, while almost nothing touches this function. 595 lines. * feat(core,studio): hear the FX chain in preview, and run the carve analysis Splices an element's FX chain into the playback graph so preview stops being silent about effects, and wires the carve button that was previously inert. The chain goes between the decoded source and its gain stage: effects see the raw signal and volume automation rides on their output, matching the order the offline render uses. Since preview and render call the same graph builders, what is heard while scrubbing is what gets written. The splice lives in the transport rather than on the `<audio>` element. The transport plays each track from a decoded AudioBuffer and mutes the element to avoid doubling, so capturing the element with createMediaElementSource would have processed a stream nothing is listening to — it looked like it worked because the call succeeded, and the audio was unchanged. A chain that cannot be built plays dry rather than silencing the track, which is the right failure in preview: the author keeps working and hears the source. The render still refuses, because shipping the dry signal there would be wrong. Carve now analyses for real: it decodes the chosen voice track, ranks its bands and writes the resulting peaking filters onto this track. Generated nodes are tagged `fromCarve`, so re-running replaces the previous carve instead of stacking another set on top of hand-added effects. Known limitation: the graph is built when a source is scheduled, so a knob turned mid-playback takes effect on the next play or seek rather than immediately. Live re-parameterisation needs the transport to hold the handle and forward updates. * fix(studio,core): stop parameter drags from restarting playback Dragging a knob wrote the chain through the persisting attribute path on every input event. That path refreshes the preview, which reloads the composition and reschedules audio — so a single drag reloaded dozens of times and playback stuttered the whole way. Drags now go through `onSetAttributeLive`, the same path colour grading uses for scrubs: it coalesces undo entries and sets `skipRefresh`, so no reload happens. The persisting write fires once, when the gesture ends — pointer-up or blur for a slider, Enter or blur for a typed value. A select commits immediately since there is no drag to wait for. While dragging, the control is driven from local state. Waiting for the value to round-trip through the element attribute made the knob lag behind the pointer. For the change to be audible without a reload, the graph now follows the attribute: the chain installed by the transport observes the element and re-parameterises itself in place, so a value change lands on the next 128-sample quantum. A shape change (effect added, bypassed, pole count) cannot be patched into a running graph, so it still waits for the next schedule rather than cutting the audio mid-play. The regression test drags a slider through several values and asserts the persisting handler is untouched until release. * feat(studio): put the audio FX rack behind its canary Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered at 0% — so the whole 47-PR stack can land without showing anyone a feature that has not been measured yet. The gate sits on the AUTHORING surface and nowhere else. The runtime and the render still honour a `data-fx-chain` already on an element, so a composition written through the skill or by `carve.mjs` keeps its processing rather than going silently dry for anyone outside the cohort. A canary should stage who can REACH a feature, not make an attribute somebody already wrote stop working with no error. Gated at the panel rather than in `resolveEditingSections`: the affordance resolver is a pure function in core describing what an element CAN support, and rollout state is not a property of an `<audio>` tag. Pinned the 0% with a test, and checked it fails at 25 — a ramp should have to break something that says "this ships dark" out loud. One gap, stated rather than papered over: the gate itself has no unit test. I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness never renders the Audio FX group for its audio fixture even with the gate removed — so the test passed for the wrong reason in the off case and could not pass at all in the on case. A test that cannot fail for the right reason is worse than none. Verifying the gate needs the panel harness to mount that section first, which is its own change. * fix(core): register FX worklets before building nodes that need them An AudioWorkletNode cannot be constructed before its processor is registered — it throws, and the surrounding chain is lost with it. `attachElementFxChain` built the chain first and only then called `ensureAudioFxWorklets`, so every worklet-backed effect (compressor, limiter, gate, bitcrush) threw on construction and the track fell back to dry. Instrumenting the preview showed `hf-compressor: InvalidStateError` with addModule never called at all. When the module has not landed yet the track now plays dry and the graph is swapped in once registration resolves, so the effect arrives a moment late instead of never. Registration is also tracked per context rather than in one module-level promise. A processor registered on one AudioContext does not exist on another, so the shared promise made every context after the first believe it was ready when it was not — the studio's transport owns its own context, which is exactly that case. With the worklets actually running, the compressor's per-sample log10 and pow became real audio-thread work. Samples below the knee have a gain of exactly unity and need neither, so the envelope is now compared in the linear domain and the transcendentals only run for samples that are actually being compressed. * refactor(studio): split the FX node row out of FxSection Clears the health findings the FX stack left behind: the chain-node render callback was a 70-line closure over half of FxSection's state, and the two reorder arrows were the same button written twice. Also drops two exports with no consumers, and registers the audio FX runtime stub as an entry point — it is bundled by file path, so nothing imports it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): automation envelope model for audio tracks Adds the data model behind Ableton-style automation lanes: breakpoint envelopes over track volume or one knob of one effect in the track's FX chain, stored on the element as `data-automation`. Times are clip-local, so an envelope travels with the clip when it moves — the clip-envelope model rather than arrangement automation. `sampleAutomationLane` is the single interpolator. The lane drawing, the preview scheduler and the render bake all call it, so the picture and the sound cannot disagree about the curve. Log-scaled parameters interpolate in log space, matching what their own knob already promises. FX nodes gain a stable `id`, minted by count rather than randomly so the document is the same on every machine. Lanes address nodes by id, so reordering a chain never re-points a lane at a different effect, and a lane whose effect was deleted is dropped rather than left to reattach. Also warns when a track carries both a volume lane and a GSAP volume tween, since only the lane is heard and the tween silently does nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(studio): lift the audio FX group out of PropertyPanelFlat `PropertyPanelFlat.tsx` was 672 lines against the repo's 600-line cap, so the required File size check was red — the sole reason #3014 and #3022 are blocked. Both reviews say the same thing: "mechanical fix, not a design problem. Code itself is LGTM." Moves `AudioFxGroup` and `audioFxSummary` into `propertyPanelAudioFxGroup.tsx`, which is where a later branch puts them anyway — done here so the file is under the cap from the point it first crosses it, rather than ten branches later. 533 lines now. The four audio imports it no longer needs go with it. Not fixed here: three `FxSection carve` tests fail on this branch with "Cannot read properties of undefined (reading 'toFixed')". Confirmed pre-existing by stashing this change and re-running — that is the separate `Test` failure the review also flags. * feat(core): expose the AudioParams behind automatable FX knobs Marks the knobs an automation lane can drive and has each graph builder hand back the AudioParam behind them, so a scheduler can write to a running effect without knowing what the effect is. A knob is not always one AudioParam. A wet/dry mix is two gains moving in opposition, and a knob in milliseconds drives a delay time in seconds, so each target carries the mapping out of the knob's own declared unit. What stays unautomatable is stated where it is decided: a WaveShaper curve, a convolution impulse and a one-pole filter's coefficients are all rebuilt wholesale rather than scheduled, and the four worklet effects take values by postMessage rather than through AudioParams. The registry flag is written by hand, so a test builds every effect and checks the exposure both ways — nothing flagged is missing, nothing exposed is unflagged. A flag that lied would offer a lane that silently did nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): play automation envelopes in preview Schedules each lane onto the AudioParams behind its knob using native ramps and value curves. Nothing evaluates the envelope per frame: it is handed to the audio thread once, so it stays sample-accurate however busy the main thread is, and the offline render will schedule it the same way. Timing comes from the transport, so an envelope survives seeking into the middle of a clip, a clip that has not started yet, and a playback rate that compresses clip seconds into context seconds. A straight line is only scheduled as a ramp when nothing bends it — no curvature, a linear parameter scale, and no unit mapping. Log-scaled parameters and mapped ones are sampled instead, since a delay knob in milliseconds and a wet/dry pair moving in opposition are not linear in the parameter they drive. Lanes with nowhere to write are skipped rather than reported: a one-pole filter exposes no frequency param, and the worklet effects expose none at all. Editing an envelope mid-playback re-aims it at the live playhead rather than restarting the track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): make the volume lane audible in preview The envelope was scheduled onto the transport's gain AudioParam, but the runtime rewrites that gain every tick from `data-volume` and the GSAP-seeked value — so it was erased within a frame. Volume automation was correct in the render and inaudible while previewing. The lane now feeds the per-tick path where the probed volume keyframes already sit, checked ahead of them so the two cannot fight, and the transport no longer schedules volume at all: one mechanism instead of two racing. The cost is honest — in preview the level steps per tick rather than per sample, exactly as the existing keyframe path does. The render still bakes it into the PCM sample-accurately, and FX parameters are still scheduled on their own AudioParams, since nothing rewrites those. Parsed lanes are cached by attribute text: the runtime asks once per tick per track, and parsing there would run the JSON parser 60 times a second for a value that only changes on an edit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(engine): bake automation envelopes into the render The offline render schedules FX lanes with the same scheduler preview uses, inside the OfflineAudioContext that already runs the same graph builders. The input WAV is the clip's own audio from its first sample, so clip-local time is offline time and the envelope needs no offset. Volume lanes take the existing PCM bake rather than a second mechanism: the lane is converted to keyframes, so a straight fade stays two of them and only a bent segment is sampled — the baker interpolates linearly and would otherwise quietly straighten the curve. A volume lane supersedes keyframes probed from the timeline, which `lint` already warns about. A browser test sweeps a lowpass from below a 2 kHz tone to well above it and measures both ends. Parsing the envelope is not the same as scheduling it, and only running the real thing tells the two apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): apply chain edits to the running graph A structural edit — an effect added, removed, bypassed, or a filter's pole count switched — was dropped. `buildFxChain`'s update reports false when the change is not merely new values, and the attribute observer ignored that, so the edit only took hold when the persisting write reloaded the composition. That reload restarted every playing track, which is what was heard as the audio chopping. The graph is now swapped in place: the old effects are detached, the new ones built and connected between the same source and gain, and any lanes re-scheduled onto the new nodes. The source node is never touched, so playback does not restart. A track with no chain is watched too, rather than wired through and forgotten, so adding its first effect is heard the same way. That means the function always returns a disposer instead of null for the empty case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): drop the FX panel's dead __testables export Fallow audit flagged it — no test imports the module. * fix(core,studio): clear the remaining Fallow audit findings on the FX panel - Split FxSection's per-node row into FxNodeRow + FxNodeControls so the CRAP score (31.6, threshold 30) splits across two smaller units instead of moving wholesale with one extraction. - Dedupe the repeated "open the add menu, read its items" block in propertyPanelFxSection.test.tsx into openAddMenuItems(). - Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into one build-inline-artifact.ts, config-selected by CLI arg — the two scripts were a byte-for-byte clone save for names. - Exempt canary.test.ts's rawFnv (a deliberate independent reimplementation used to cross-check canaryBucket, per its own docstring) and the property-panel test files' shared renderInto/mount scaffolding (pre-existing across 9 files, 2 outside this stack) in .fallowrc.jsonc, consistent with this file's existing exemptions for the same class of intentional/pre-existing duplication. * fix(ci): allowlist the build-script consolidation in the no-main-deletions guard build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into build-inline-artifact.ts to kill a fallow duplication finding; the deletion guard flagged that as an accidental loss since main still has both originals. * fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo Both effect builders set wet.gain to the mix and dry.gain to its complement in identical two-line blocks; fallow kept re-flagging it as a 10-line clone on every unrelated change. Extracted setWetDryMix. * fix(core): remove the build-audio-fx-runtime.ts stray resurrected by a main merge An earlier merge with main brought this deleted file back (git's merge/delete handling on an unchanged-on-one-side file); package.json already points at build-inline-artifact.ts, so it sat unreachable and duplicating that file's config, both of which fallow flagged. * fix(studio): pull TimelineLanes under the 600-line cap TimelineLanes.tsx hit 620 lines. Extracted the three per-clip pointer gestures (resize-start, pointer-down move-arm, click/razor-split) into createClipGestureHandlers — one factory call per rendered clip instead of ~120 lines of inline handler bodies in the render loop. 529 lines now. * fix(studio): split the extracted pointerdown handler under the CRAP threshold Moving the ~120-line gesture logic into timelineClipGestureHandlers.ts concentrated it into two functions fallow flagged (onPointerDown at CRAP 63.6, onResizeStart at 31.6). Split the decision logic (which gesture a pointerdown implies) into a pure resolvePointerDownAction, then split its own intent-blocking check into isIntentBlocked. onResizeStart's guard moved into canStartResize. Every function now scores under 30. * fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
23ba58cfd3 |
feat(audio): play automation envelopes in preview and bake them at render (#3016)
* feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * test(engine): probe ffmpeg and Chrome instead of assuming them Two failures on #3021's Test job, both about the environment rather than the code under test. **Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to `execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()` resolves — every other ffmpeg-dependent suite in this package already goes through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)` so a contributor without ffmpeg skips rather than fails. **The browser guard trusted the wrong thing.** It asked `resolveHeadlessShellPath()` and treated a returned path as "a browser is here". CI's cache holds a chrome-headless-shell that resolves and then fails to spawn — a partial download is indistinguishable from a working one by `existsSync`, which is all that resolver checks. So the three browser cases ran anyway and failed on the launch. It now runs `--version` and requires exit 0, which is the same probe the ffmpeg suites use: ask the binary, do not infer from the filesystem. Checked both directions rather than just the green one. With a working browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed at a binary that exits non-zero — CI's exact situation — exactly 3 skip and the other 8 still run. A guard that quietly skipped everything would have looked identical on the CI summary. * feat(core): register the audio-fx-rack canary at 0% Lands the rollout switch dark, per the registry's own procedure: "Start at percentage: 0 and merge that — a canary at 0 is dead code you can land safely and ramp without a code review." Declared at the bottom of the stack so every branch above can read it. The gate itself goes in at wa-4-fx-panel, where the rack first appears. Scope is deliberate and stated in the description: it gates the AUTHORING surface only. A composition that already carries `data-fx-chain` still plays and renders it. A canary should stage who can REACH a feature, not make an attribute somebody already wrote silently inert — an agent that writes a chain through the skill would otherwise produce a file whose audio processing vanishes with no error. * feat(studio): audio FX panel generated from the registry Controls for the whole chain: add, remove, reorder, bypass, and every knob each effect declares. Nothing in the panel knows what a compressor is. The registry supplies each parameter's range, step, unit and scale and the panel renders what it finds, so adding an effect or a knob upstream needs no change here, and the panel cannot offer a value the renderer would reject — a typed-in figure is clamped into the declared range on the way through. Frequency and time controls span three or four decades, so those declare a log scale and the slider maps exponentially; a linear slider would spend most of its travel somewhere useless. Reorder is a first-class control because chain order changes the sound: a reverb before a compressor is not the same as after. Carve gets its own block rather than an entry in the add menu, with a picker for the voice track to listen to. It processes this track based on another one, which is how a sidechain control works — it lives on the track that changes, and names the source. * feat(studio): show the Audio FX section on audio tracks Adds `audioFx` to the editing-affordances contract and renders the FX panel in the inspector when an `<audio>` element is selected. The section is audio-only. A `<video>` carries its sound on a separate `<audio>` element, so an FX chain on the video would have nothing to process. Chain and carve settings are written straight back onto the element as serialised attributes, the way colour grading carries its config, so persistence is an ordinary attribute write and needs no new server route. A chain that cannot be parsed renders as empty rather than breaking the panel, and the attribute is left untouched until the user changes something. The collapsed group summarises what is on the track ("2 effects + carve") so the state is visible without expanding it. Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED defaults to true, so the flat inspector is what actually renders. * refactor(studio): lift audioFxSummary out of PropertyPanelFlat `PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap, so the required File size check is red — the sole reason this PR is blocked. The review says as much: "mechanical fix (~5 min), not a design problem. Code itself is LGTM." Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later branch creates for it. Deliberately the smallest cut that clears the cap rather than the whole `AudioFxGroup` extraction: every later commit in the stack edits AudioFxGroup, so moving it here would collide with each of them, while almost nothing touches this function. 595 lines. * feat(core,studio): hear the FX chain in preview, and run the carve analysis Splices an element's FX chain into the playback graph so preview stops being silent about effects, and wires the carve button that was previously inert. The chain goes between the decoded source and its gain stage: effects see the raw signal and volume automation rides on their output, matching the order the offline render uses. Since preview and render call the same graph builders, what is heard while scrubbing is what gets written. The splice lives in the transport rather than on the `<audio>` element. The transport plays each track from a decoded AudioBuffer and mutes the element to avoid doubling, so capturing the element with createMediaElementSource would have processed a stream nothing is listening to — it looked like it worked because the call succeeded, and the audio was unchanged. A chain that cannot be built plays dry rather than silencing the track, which is the right failure in preview: the author keeps working and hears the source. The render still refuses, because shipping the dry signal there would be wrong. Carve now analyses for real: it decodes the chosen voice track, ranks its bands and writes the resulting peaking filters onto this track. Generated nodes are tagged `fromCarve`, so re-running replaces the previous carve instead of stacking another set on top of hand-added effects. Known limitation: the graph is built when a source is scheduled, so a knob turned mid-playback takes effect on the next play or seek rather than immediately. Live re-parameterisation needs the transport to hold the handle and forward updates. * fix(studio,core): stop parameter drags from restarting playback Dragging a knob wrote the chain through the persisting attribute path on every input event. That path refreshes the preview, which reloads the composition and reschedules audio — so a single drag reloaded dozens of times and playback stuttered the whole way. Drags now go through `onSetAttributeLive`, the same path colour grading uses for scrubs: it coalesces undo entries and sets `skipRefresh`, so no reload happens. The persisting write fires once, when the gesture ends — pointer-up or blur for a slider, Enter or blur for a typed value. A select commits immediately since there is no drag to wait for. While dragging, the control is driven from local state. Waiting for the value to round-trip through the element attribute made the knob lag behind the pointer. For the change to be audible without a reload, the graph now follows the attribute: the chain installed by the transport observes the element and re-parameterises itself in place, so a value change lands on the next 128-sample quantum. A shape change (effect added, bypassed, pole count) cannot be patched into a running graph, so it still waits for the next schedule rather than cutting the audio mid-play. The regression test drags a slider through several values and asserts the persisting handler is untouched until release. * feat(studio): put the audio FX rack behind its canary Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered at 0% — so the whole 47-PR stack can land without showing anyone a feature that has not been measured yet. The gate sits on the AUTHORING surface and nowhere else. The runtime and the render still honour a `data-fx-chain` already on an element, so a composition written through the skill or by `carve.mjs` keeps its processing rather than going silently dry for anyone outside the cohort. A canary should stage who can REACH a feature, not make an attribute somebody already wrote stop working with no error. Gated at the panel rather than in `resolveEditingSections`: the affordance resolver is a pure function in core describing what an element CAN support, and rollout state is not a property of an `<audio>` tag. Pinned the 0% with a test, and checked it fails at 25 — a ramp should have to break something that says "this ships dark" out loud. One gap, stated rather than papered over: the gate itself has no unit test. I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness never renders the Audio FX group for its audio fixture even with the gate removed — so the test passed for the wrong reason in the off case and could not pass at all in the on case. A test that cannot fail for the right reason is worse than none. Verifying the gate needs the panel harness to mount that section first, which is its own change. * fix(core): register FX worklets before building nodes that need them An AudioWorkletNode cannot be constructed before its processor is registered — it throws, and the surrounding chain is lost with it. `attachElementFxChain` built the chain first and only then called `ensureAudioFxWorklets`, so every worklet-backed effect (compressor, limiter, gate, bitcrush) threw on construction and the track fell back to dry. Instrumenting the preview showed `hf-compressor: InvalidStateError` with addModule never called at all. When the module has not landed yet the track now plays dry and the graph is swapped in once registration resolves, so the effect arrives a moment late instead of never. Registration is also tracked per context rather than in one module-level promise. A processor registered on one AudioContext does not exist on another, so the shared promise made every context after the first believe it was ready when it was not — the studio's transport owns its own context, which is exactly that case. With the worklets actually running, the compressor's per-sample log10 and pow became real audio-thread work. Samples below the knee have a gain of exactly unity and need neither, so the envelope is now compared in the linear domain and the transcendentals only run for samples that are actually being compressed. * refactor(studio): split the FX node row out of FxSection Clears the health findings the FX stack left behind: the chain-node render callback was a 70-line closure over half of FxSection's state, and the two reorder arrows were the same button written twice. Also drops two exports with no consumers, and registers the audio FX runtime stub as an entry point — it is bundled by file path, so nothing imports it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): automation envelope model for audio tracks Adds the data model behind Ableton-style automation lanes: breakpoint envelopes over track volume or one knob of one effect in the track's FX chain, stored on the element as `data-automation`. Times are clip-local, so an envelope travels with the clip when it moves — the clip-envelope model rather than arrangement automation. `sampleAutomationLane` is the single interpolator. The lane drawing, the preview scheduler and the render bake all call it, so the picture and the sound cannot disagree about the curve. Log-scaled parameters interpolate in log space, matching what their own knob already promises. FX nodes gain a stable `id`, minted by count rather than randomly so the document is the same on every machine. Lanes address nodes by id, so reordering a chain never re-points a lane at a different effect, and a lane whose effect was deleted is dropped rather than left to reattach. Also warns when a track carries both a volume lane and a GSAP volume tween, since only the lane is heard and the tween silently does nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(studio): lift the audio FX group out of PropertyPanelFlat `PropertyPanelFlat.tsx` was 672 lines against the repo's 600-line cap, so the required File size check was red — the sole reason #3014 and #3022 are blocked. Both reviews say the same thing: "mechanical fix, not a design problem. Code itself is LGTM." Moves `AudioFxGroup` and `audioFxSummary` into `propertyPanelAudioFxGroup.tsx`, which is where a later branch puts them anyway — done here so the file is under the cap from the point it first crosses it, rather than ten branches later. 533 lines now. The four audio imports it no longer needs go with it. Not fixed here: three `FxSection carve` tests fail on this branch with "Cannot read properties of undefined (reading 'toFixed')". Confirmed pre-existing by stashing this change and re-running — that is the separate `Test` failure the review also flags. * feat(core): expose the AudioParams behind automatable FX knobs Marks the knobs an automation lane can drive and has each graph builder hand back the AudioParam behind them, so a scheduler can write to a running effect without knowing what the effect is. A knob is not always one AudioParam. A wet/dry mix is two gains moving in opposition, and a knob in milliseconds drives a delay time in seconds, so each target carries the mapping out of the knob's own declared unit. What stays unautomatable is stated where it is decided: a WaveShaper curve, a convolution impulse and a one-pole filter's coefficients are all rebuilt wholesale rather than scheduled, and the four worklet effects take values by postMessage rather than through AudioParams. The registry flag is written by hand, so a test builds every effect and checks the exposure both ways — nothing flagged is missing, nothing exposed is unflagged. A flag that lied would offer a lane that silently did nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): play automation envelopes in preview Schedules each lane onto the AudioParams behind its knob using native ramps and value curves. Nothing evaluates the envelope per frame: it is handed to the audio thread once, so it stays sample-accurate however busy the main thread is, and the offline render will schedule it the same way. Timing comes from the transport, so an envelope survives seeking into the middle of a clip, a clip that has not started yet, and a playback rate that compresses clip seconds into context seconds. A straight line is only scheduled as a ramp when nothing bends it — no curvature, a linear parameter scale, and no unit mapping. Log-scaled parameters and mapped ones are sampled instead, since a delay knob in milliseconds and a wet/dry pair moving in opposition are not linear in the parameter they drive. Lanes with nowhere to write are skipped rather than reported: a one-pole filter exposes no frequency param, and the worklet effects expose none at all. Editing an envelope mid-playback re-aims it at the live playhead rather than restarting the track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(engine): bake automation envelopes into the render The offline render schedules FX lanes with the same scheduler preview uses, inside the OfflineAudioContext that already runs the same graph builders. The input WAV is the clip's own audio from its first sample, so clip-local time is offline time and the envelope needs no offset. Volume lanes take the existing PCM bake rather than a second mechanism: the lane is converted to keyframes, so a straight fade stays two of them and only a bent segment is sampled — the baker interpolates linearly and would otherwise quietly straighten the curve. A volume lane supersedes keyframes probed from the timeline, which `lint` already warns about. A browser test sweeps a lowpass from below a 2 kHz tone to well above it and measures both ends. Parsing the envelope is not the same as scheduling it, and only running the real thing tells the two apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): drop the FX panel's dead __testables export Fallow audit flagged it — no test imports the module. * fix(core,studio): clear the remaining Fallow audit findings on the FX panel - Split FxSection's per-node row into FxNodeRow + FxNodeControls so the CRAP score (31.6, threshold 30) splits across two smaller units instead of moving wholesale with one extraction. - Dedupe the repeated "open the add menu, read its items" block in propertyPanelFxSection.test.tsx into openAddMenuItems(). - Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into one build-inline-artifact.ts, config-selected by CLI arg — the two scripts were a byte-for-byte clone save for names. - Exempt canary.test.ts's rawFnv (a deliberate independent reimplementation used to cross-check canaryBucket, per its own docstring) and the property-panel test files' shared renderInto/mount scaffolding (pre-existing across 9 files, 2 outside this stack) in .fallowrc.jsonc, consistent with this file's existing exemptions for the same class of intentional/pre-existing duplication. * fix(ci): allowlist the build-script consolidation in the no-main-deletions guard build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into build-inline-artifact.ts to kill a fallow duplication finding; the deletion guard flagged that as an accidental loss since main still has both originals. * fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo Both effect builders set wet.gain to the mix and dry.gain to its complement in identical two-line blocks; fallow kept re-flagging it as a 10-line clone on every unrelated change. Extracted setWetDryMix. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
cc40e35aa0 |
feat(engine): render the FX chain offline, and the carve analysis behind it (#3021)
* feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * test(engine): probe ffmpeg and Chrome instead of assuming them Two failures on #3021's Test job, both about the environment rather than the code under test. **Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to `execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()` resolves — every other ffmpeg-dependent suite in this package already goes through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)` so a contributor without ffmpeg skips rather than fails. **The browser guard trusted the wrong thing.** It asked `resolveHeadlessShellPath()` and treated a returned path as "a browser is here". CI's cache holds a chrome-headless-shell that resolves and then fails to spawn — a partial download is indistinguishable from a working one by `existsSync`, which is all that resolver checks. So the three browser cases ran anyway and failed on the launch. It now runs `--version` and requires exit 0, which is the same probe the ffmpeg suites use: ask the binary, do not infer from the filesystem. Checked both directions rather than just the green one. With a working browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed at a binary that exits non-zero — CI's exact situation — exactly 3 skip and the other 8 still run. A guard that quietly skipped everything would have looked identical on the CI summary. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |