ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-01 19:42:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(studio): show every automated knob at the playhead (#3210)

Browse Source 
* feat(studio): pure range ops for automation lane selections

Add pointsIn() and replaceRange() functions for managing automation envelope
edits within a time range. The key invariant: envelope values outside the
selection never move. Implemented by anchoring the boundaries at t0 and t1
by sampling the original lane, so cutting middle sections cannot reshape
the rest. Inner points from shape generators can suppress redundant anchors
at merge distance.

* fix(studio): budget replaceRange's inner points before capping, not after

* feat(studio): automation selection slice

* feat(studio): drag-select a time range on an automation lane

Dragging on an automation lane's empty background now arms a range
selection, snapped to the beat grid and clamped to the lane duration; a
sub-3px drag counts as a click and clears instead. Point drags and
Alt-drag segment bends still take priority, since the range arm only
runs where the existing point/segment hit-test already returned null.

useAutomationLanes binds the selection slice per element/lane so the
rect renders from the store, matching the read pattern the writes
already use.

* feat(studio): delete an automation selection from the keyboard

Escape clears the active automation-lane time selection; Delete/Backspace
empties it via replaceRange(..., inner: []), which pins anchor points at
both edges and leaves the envelope outside the selection untouched. Mounted
in TimelineLanes.tsx next to the useAutomationLanes() call that already
lives there. Also adds a stale-selection guard in TimelineAutomationLaneSlot
that clears the selection if its lane's target stops existing on the bound
element's automation (e.g. the automated effect was deleted).

* test(studio): cover the automation selection stale-target guard

* feat(studio): ramp, swell and dip generators for automation selections

* fix(studio): let an automation range keep Delete from the clip

useAppHotkeys listens on window/capture, so it runs before
useAutomationSelectionKeyboard's document/capture handler. With a range
selected, Delete fell straight through to the clip-delete branch and
destroyed the whole audio clip the lane belongs to; Backspace hit the
reset-keyframes branch on the way and wiped the clip's keyframes.

Guard both by returning early when automationSelection is set, mirroring
the selectedKeyframes precedent six lines above. No preventDefault: the
downstream handler still needs the key.

dispatchPlainKey is exported so the arbitration between keyframes, an
automation range and the clip can be pinned without standing up the hook.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: suppress unused export for automationShapes (consumed upstack)

* feat(studio): simplify dense automation runs

Implements Ramer-Douglas-Peucker point-thinning for audio automation
lane breakpoints, working in unit space for correct log-scaled
parameter handling.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore: add automationSimplify to fallow complexity ignore

The Ramer-Douglas-Peucker algorithm in automationSimplify.ts has
inherent complexity (12 cyclomatic / 20 cognitive) that is by design
and not refactorable. Added to health.ignore list and ignoreExports
list since it's consumed by the UI layer one PR upstack.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(studio): internal clipboard for automation ranges

* feat(studio): copy and paste automation ranges across lanes

Extends the automation-selection keyboard hook with Cmd/Ctrl+C (copy the
active range) and Cmd/Ctrl+V (paste onto the selected clip's lane, at the
selection's start or the playhead, chaining the selection to the pasted
span so a second paste lands right after the first). Paste falls through
untouched when no target lane resolves, so clip-level paste keeps working.

Also fixes a latent test-isolation bug: setup() never unmounted the
previous test's Host, so document keydown listeners leaked across tests
and could consume later events before the current test's own listener ran.

* feat(studio): shape and simplify menu on an automation selection

Right-click inside an active time-selection rectangle on an automation
lane now opens a menu offering the four utility shapes (Ramp up, Ramp
down, Swell, Dip) and Simplify, composing generateShape/simplifyPoints
with pointsIn/replaceRange from the prior selection tasks. A point's own
right-click still stops propagation and deletes it, unaffected.

* feat(studio): retime an automation selection

Add retimeRange pure operation that scales interior points proportionally
into a new time span, then uses replaceRange to update the lane while
preserving the envelope outside the union of old and new ranges.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(studio): repair the automation paste path and finish the key arbitration

Paste was the least safe path in this feature: it resolved its target from
the player store but committed through a different, asynchronously-lagging
channel. Six review findings against this branch, plus the Cmd+C/Cmd+V half
of the arbitration wa-15 started for Delete.

- Write channel: resolvePasteTarget bails unless the binding's
  commitTargetKey equals the element it resolved. useAutomationLanes exposes
  that key, resolved through resolveTimelineIdForSelection — the same
  resolver applyDomSelection uses — and read in the same render as the
  commit handlers, so a handler and the key cannot describe different
  moments. Before this, clicking clip B then immediately pasting serialized
  B's automation onto A and left B untouched.
- Chaining: the paste anchor comes from sel.t1, not sel.t0, so a second
  Cmd+V lands after the first instead of on top of it. The old comment
  claimed the new behaviour while the code did the opposite, and no test
  pressed Cmd+V twice.
- Empty copy: copyRange returns false rather than arming a clipboard whose
  every paste is a destructive flatten, and samples the range's edges so
  copying a smooth stretch yields a real segment instead of no points.
- Playhead: the playhead branch requires the playhead to be inside the clip
  rather than silently clamping an out-of-clip playhead to the clip's start.
- Keys: one chord helper normalizes with toLowerCase() and gates on
  !shiftKey && !altKey, matching useAppHotkeys. CapsLock no longer kills the
  shortcut and Ctrl+Alt+V no longer pastes where the app declines.
- Arbitration: useAppHotkeys consults automationOwnsKey before its c/v
  branch, so an active range keeps Cmd+C/Cmd+V from the clip clipboard the
  same way it keeps Delete. Without it Cmd+V duplicated the clip while the
  automation paste wrote the same file, and Cmd+C armed both clipboards.
  It returns without preventDefault — the downstream handler needs the key —
  and declines when the automation clipboard is empty so clip paste still
  works. dispatchModifierKey is exported to pin this, like dispatchPlainKey.
- Double-action: the hook now returns early on e.defaultPrevented.
  useAppHotkeys is on window/capture and deliberately lets a keyframe
  selection outrank a range on Delete; without this that press deleted the
  keyframes there AND emptied the range here.

- Project scoping: the clipboard scopes itself. Every entry point carries the
  project it speaks for and a mismatch empties the module, the shape
  keyframeSlice already uses to discard a request from a previous session.
  Scoping it inside the module rather than clearing it from the session seam
  is deliberate: the failure is silent and destructive — a range copied in
  project A pasted into B is remapped through A's captured sourceRange for an
  FX node B may not have, and the keystroke is consumed so clip paste never
  runs — so no future caller should be able to forget the guard. The mark
  isLastPasteSpan reads is scoped transitively, through the same check.
- Session reset: createTimelineResetState clears automationSelection. It is
  as ephemeral as selectedKeyframes, and a range surviving a project switch
  can match a same-keyed clip in the new project and redirect a paste through
  sel.elementKey === paste.elementKey to a stale t0.

Five of the six paste fixes above shipped without a test that fails without
them, which is how the branch reached review with a comment describing
chaining that the code did not do. Each now has one: a second Cmd+V landing
after the first, a commit-target mismatch declining, an out-of-clip playhead
declining, an empty-lane copy leaving an earlier clipboard intact, and Cmd+V
with CapsLock on. All five fail against this branch's parent.

* test(studio): probe retimeRange's actual guarantee, not sample-continuity past a moved edge

The failing test probed t=5.1, which sits inside the reshaped transition
segment between the new edge (t=5) and the existing point (t=6). When
growing past an existing breakpoint, the transition TO that point
legitimately reshapes — the edge moved (t=3→t=5) even though the far
point (t=6) did not.

The real guarantee: all BREAKPOINTS strictly outside the union keep
exact (t, v) values. Corrected test to:
1. Verify sample continuity on unaffected side: t=[0,1,1.9]
2. Verify the breakpoint at t=6 keeps exact value: (t:6, v:0)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(studio): stretch an automation selection by its edges

Add an edge-handle drag to a selection's rect: grabbing within 8px of
either edge retimes the selection via the already-landed retimeRange,
scaling interior points proportionally and clamping the dragged edge
against its partner and the clip's duration. Priority is point-drag >
curve-drag > edge-stretch > new-range-select, so a point sitting on an
edge still wins the press. Cursor shows col-resize while hovering or
dragging a handle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(studio): retime edge-stretch from a fixed points snapshot

moveEdge fed retimeRange the live draft on every pointermove while
origin.t0/t1 stayed pinned to the drag's start. retimeRange is a
relative transform that scales a lane's own current point positions,
so repeated pointermoves compounded the scale factor (interior points
drift toward the far edge) and could drop points that retimed past the
selection's original bound out of the next move's `inner` set entirely.

Snapshot lane.points at arm time (armBackgroundGesture) alongside the
existing frozen origin, and always retime from that snapshot in
moveEdge instead of the live draft. finishEdgeDrag is unchanged: it
already just persists the last (now-correct) preview.

Adds a regression test asserting a multi-pointermove edge-drag (both
edges) lands on the exact same final points as a single-shot drag to
the same target — the case that exposed the bug, since the existing
suite only ever tested a single move.

* fix(studio): clamp selection-start paste, sharpen clipboard test, cleanup

- useAutomationSelectionKeyboard: clamp the selection-start paste branch
  to [0, element.duration - clip.span], same as the playhead branch
  already does. An unclamped paste near a clip's end could write points
  past element.duration and leave the resulting selection's edge
  ungrabbable off the visible lane.
- automationClipboard.test.ts: swap the cross-parameter mapping test's
  target from fx.r.wet (numerically identical to VOLUME_RANGE) to the
  log-scaled fx.n1.frequency, so the test actually discriminates real
  unit-space mapping from a linear guess or a verbatim value copy.
- automationLaneSelection.ts: drop the lone `!` non-null assertion in
  decimateEvenly's budget-of-1 branch for a guarded pattern, matching
  the loop right below it and the repo's no-`!` convention.
- .fallowrc.jsonc: remove the two ignoreExports entries for
  AUTOMATION_SHAPES and simplifyPoints — both are now genuinely
  consumed (AutomationSelectionMenu.tsx, TimelineAutomationLane.tsx).
- AutomationSelectionMenu.tsx: port TrackGapContextMenu's viewport-edge
  clamping so a right-click near the bottom/right of the timeline
  doesn't render the shape/simplify menu partially off-screen.

* fix(studio): give edge-stretch the gesture contract the other four follow

Seven review findings against this branch, five of which were one defect:
edge-stretch was added as a fifth mutually-exclusive gesture on the lane
without joining the threshold / live-preview / revert-on-cancel contract the
point drag, curve bend, range drag and double-click all obey. Patching them
one at a time would have been more code and less coherent, so this makes the
stretch structurally parallel to its sibling range drag instead, and extracts
it to useAutomationEdgeStretch on the way out — the gestures file had ~60
lines of headroom under the 600-line studio cap, and shaving comments to fit
a refactor in is not a plan.

- Threshold. A press within the 8px halo of either edge used to persist a
  no-op commit and push an undo entry that changed nothing (commitDataAttribute
  has no unchanged-value short-circuit). Worse, it made the pre-existing "click
  the background to clear the selection" escape unreachable anywhere near an
  edge. Below 3px of travel — the same threshold the range drag uses — the
  press now clears the selection and writes nothing at all.
- Live preview. moveEdge never fired onRangeSelect and the hook discarded the
  drag's live position, so the highlight rect and both edge lines stayed pinned
  at the pre-drag bounds for the whole gesture and snapped into place on
  release: the user dragged an invisible handle. It now reports bounds on every
  move, exactly as the marquee drag does and for the same reason.
- Revert on cancel. pointercancel means the browser abandoned the gesture; it
  was routed to the same handler as pointerup, which persisted whatever partial
  retime it had reached. It now restores the arm-time snapshot through the
  preview channel — there is nothing persisted to undo — and puts the selection
  back. A new cancelDrag handler owns that, so a release and an abandonment are
  no longer the same event.
- Lost capture. capturePointer took the capture on e.target, i.e. whichever
  child the press landed on. A child that unmounts mid-drag takes the capture
  with it, silently, with no pointercancel — after which edgeDrag stayed
  non-null and every later button-less pointermove kept retiming and writing.
  Capture is now taken on the svg, which outlives every gesture on it, and a
  move reporting no buttons held ends the drag as a cancel.
- Hit priority. A breakpoint sitting exactly on the selection's edge used to
  win the press. Since replaceRange pins an anchor at the union bound and
  finishEdgeDrag leaves the selection edge at that same time, EVERY range
  operation — stretch, delete, shape insert — leaves a point exactly on the
  edge it just created: the second stretch of the same edge resolved to a
  point-drag, at the one height (on the envelope) where a user naturally grabs
  it. The feature was not repeatable. An active selection's edge now outranks a
  point on it; clearing the selection reaches the point again, which is tested.
- Clamp order. The dragged edge was bounded against its partner AFTER the
  0-floor, so a selection thinner than the minimum width yielded a negative t0,
  which core's cleanPoint then collapses onto a duplicate t=0 on the serialize
  round-trip — silent envelope corruption. The floor is now applied last. The
  minimum width is its own MIN_SELECTION_SEC rather than a borrowed
  POINT_MERGE_SEC: when two breakpoints are the same breakpoint is a different
  question from how thin a time selection may get.

One finding does not survive: edgeAt's `d0 <= d1` tiebreak was reported as
making the t1 edge ungrabbable on a narrow selection, but that comparison IS
nearest-wins, and a press right of the midpoint already resolved to t1. The
midpoint split here is the same rule written so it is legible rather than
inferred, and the test for it is labelled as characterizing behaviour, not
fixing it. What was genuinely unreachable inside a narrow halo — starting a
fresh range, or clearing the old one without Escape — the threshold above fixes.

Also settles what retimeRange does with a breakpoint sitting ON a dragged
edge, which was never decided: pointsIn is endpoint-inclusive, so it is
interior and travels with the stretch. It has to be, because the commonest
stretch of all is grabbing an edge to drag exactly that point outward, and
anchoring it would delete it and flatten the span instead. The price is that
the retimed point lands on the union's own boundary where a preservation anchor
would go, and anchor() stands down within a merge radius — one time cannot hold
two values — so the segment leaving the union reshapes. That is the one place
replaceRange's outside-never-moves invariant bends, and both halves are now
pinned: the exact points and the sampled slope for the on-edge case, and the
full two-sided invariant for a selection whose edges are off any breakpoint.
The earlier right-side probe at t=5.1 that caught this was deleted during
development as inherent; it was reporting the real behaviour.

* fix(core): make audio automation survive being rescheduled mid-playback

Anything landing inside a running value curve is refused unless the parameter is
cancelled first, and two paths were not cancelling: the chain observer wrote each
knob straight onto its AudioParam before rescheduling, and a bent segment read as
straight because only the curve exponent was checked, never the via point the
timeline actually writes. The first threw NotSupportedError into the console and
abandoned the rest of the envelope; the second played a dragged bend as a ramp.

Measured against Chrome, in a live context and in an offline one suspended
mid-curve: any cancel frees the span, and only a missing cancel is refused.
clearParamLane takes the strongest form on purpose, because curve-over-curve
refusals were reported with a cancel at the new schedule time already in place
and have never reproduced; emit keeps a ramp fallback as the backstop for
whatever that mechanism turns out to be.

Dynamic carve is what exercises all of it, so it lands here too:

- a `gain` primitive, so a carve can match levels as well as carve bands
- carve settings collapse to one `strength`, with carveProfile deriving the six
  numbers that always moved together anyway
- analyseCarveDynamics / analyseCarveDuck turn the analysis into envelopes, with
  a slow release so the bed does not snap back the instant a word ends
- worklets are awaited inside attach, so adding a compressor to a carved bed no
  longer kills its envelopes and freezes every later edit
- per-track failure detail in the render's audio stage, which was being discarded

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(studio): select automation points with a box, and stop them crossing

Replaces the time-range selection with a rectangle. A lane selection is a set of
breakpoints, not a span, so it now has value bounds as well as time bounds and a
point is caught only if it falls inside both — which is what lets you take the
peaks of an envelope and leave the dips between them. Delete, the group drag and
the rings drawn on caught points all read the one rule, so what looks selected is
exactly what those act on. Copy, paste, shape insert and simplify still work on
the box's time span, because they act on the envelope over a stretch of time.

Dragging is bounded by its neighbours in both the single and group cases. A point
cannot cross another, and cannot land exactly on one either: the lane collapses
points that share a `t`, keeping the later one, so arriving on top of a neighbour
deleted it. It stops a millisecond short, which is under a pixel at any zoom the
lane offers and keeps both points. Only stationary neighbours constrain a group,
per member rather than per end, since a box can select a non-contiguous set.

Edge-stretch is removed rather than fixed. Dragging a selection's edges to retime
the points inside it was the feature this branch opened for, and it is not wanted:
the hook, retimeRange, the edge handles, the col-resize cursor and the pointercancel
revert path all go, along with the ~360 lines of tests that pinned them.

Also: gesture-scoped coalescing keys, so one drag is one undo entry rather than a
fragmented chain of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(studio): put #3207's edge-stretch back, folded into the unified hook

The review blocks this PR for deleting a feature two PRs downstack:

  "#3209 deletes #3207 edge-stretch instead of folding it into the unified
  hook... merging the stack would ship #3207 and then silently remove its
  user-facing retime/edge-stretch feature."

Verified, all four claims: `useAutomationEdgeStretch.ts` (248 lines) and its
test were deleted, `retimeRange` was dropped from
`automationLaneSelection.ts`, and the consolidated hook has no edge
hit-test, arm/move/finish path, or resize cursor.

Restored: the module, `retimeRange`, the selection regressions, and the
lane wiring (`col-resize` cursor, `pointercancel` reverting a partial
retime rather than persisting it).

It is not a straight revert, because #3209 changed the selection from a
time range to a box. Edge-stretch now takes `{t0,t1,v0,v1}` and moves only
the time edges — the value extent rides through untouched, which keeps it
the same gesture it was.

**One arbitration call worth a second opinion.** #3207's rule was that a
selection's edge outranks a point sitting on it, because every range
operation leaves a breakpoint exactly on the edge it created — a
point-first rule made the second stretch of an edge resolve to a
point-drag. Under a box that rule now contradicts #3209's own test
("stops the group at a point it did not select"), which presses at t=0
v=1 — simultaneously the t0 edge and a selected point.

I inverted it: selected content wins, the edge stretches everywhere it is
not also selected content. The reasoning is that a box makes the point
visibly part of the selection, and dragging selected content has to move
it. That restores #3209's test and keeps the stretch usable along the rest
of the edge — but it is a product decision between two deliberate designs,
so flag it if #3207's original precedence was load-bearing.

954 player tests pass, including the 17 restored ones.

* feat(studio): show every automated knob at the playhead, and carve as one module

An automated parameter has two values: the number sitting in the chain, which is
only the seed a lane replaced, and the number the envelope is on right now. The
second is the true one, so the panel shows it — on the carve rack's readouts and
on every effect's own fader and number field. A rack that showed the seed stood
still while the carve was audibly working.

Off the clip it keeps sampling rather than falling back to the stored number: a
lane holds its first value backwards and its last forwards, so before the clip
starts it already knows what it will open on, and the stored seed is a value
nothing will ever play. Showing it made the fader jump the moment the clip came
under the playhead.

The playhead comes off the liveTime channel, throttled to 30 Hz — the RAF loop
deliberately keeps frames out of the store, so a panel watching only the store
would sit still for a whole take. PropertyPanel had that subscription inline;
it is now one shared hook with two callers.

Readouts reserve the width their parameter can need rather than what its current
value takes, because an updating value one character narrower shunted everything
after it sideways 30 times a second.

The carve's effects are presented as one module: an author switched on a carve,
and the peaking filters plus the level stage are how it is built, not six things
to remove one at a time. Opening it lists every member's settings as readouts,
since strength is what sets them. No carve control is offered on a track another
track already carves against — that track is the voice, not the bed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(engine): render audio FX in an OfflineAudioContext

Reads `data-fx-chain` off an audio element and runs the chain over the trimmed
WAV before volume automation is baked in — effects should see the raw signal,
and the envelope belongs on their output.

The processing happens in an OfflineAudioContext inside the headless browser
the engine already drives, running the same graph builders the studio previews
with. That is the point of the approach: one implementation per effect, so the
render agreeing with the preview is a property of the architecture rather than
a tolerance to police. Reimplementing each effect as an FFmpeg filter would
mean two implementations to keep in step, and for the dynamics processors and
modulated delays there is no filter that behaves the same way.

`build:audio-fx-runtime` bundles the graph builders into an injectable IIFE,
following the same pattern as the existing runtime artifacts, so the browser
runs exactly the code the studio does.

The page loads from a file:// URL rather than about:blank because AudioWorklet
is only exposed in a secure context — the compressor, limiter, gate and
bitcrush processors would otherwise fail to register with an opaque error.
file:// qualifies and needs no listening socket.

The chain is serialised into the attribute the way colour grading carries its
config, so there is no side-car file to resolve or lose.

An FX failure is fatal for the whole mix rather than a per-track soft failure.
Every other audio failure mode degrades gracefully — the track drops, siblings
continue — but substituting the dry signal for a processed one ships a render
that sounds plausible and is not what the author set up. Since the per-element
work races under Promise.all, an internal AbortController chained off the
caller's signal aborts in-flight siblings before workDir is removed.

* feat(core): voiceover carve analysis

Finds the bands a voice occupies so a music bed can be dipped there, letting
the voice sit in front without ducking the whole track.

Carve is a relationship between two tracks rather than an effect on one, so it
stays out of the FX chain. What it emits is an ordinary chain of peaking
filters, so a carve composes with whatever else is on the track and needs no
separate rendering path.

Selection is weighted toward intelligibility rather than raw voice energy.
Ranking purely by power lands on the fundamental almost every time, because
that is where a voice is loudest — but the masking that actually hurts a
voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The
bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights
toward 1-3 kHz.

Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB
across these bands — it falls off roughly 6 dB per octave above the fundamental
— so a weighting has to be on that scale to move anything at all. A
multiplicative weight of `1 - bias + bias * shaped` is bounded below by
`1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7
default, 3 dB at 0.5. That is no influence against a real voice — every bias
short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the
outcome the bias exists to prevent, while looking decisive against a fixture
whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth
up to 30 dB at full strength, and relative cut depths come from a dB difference
rather than a ratio of weighted linear powers.

The bias reweights ranking without overriding the spectrum — a band the voice
has no energy in is not worth carving, and scores -Infinity rather than
competing — so a strongly low-pitched voice can still select low at full bias.
What the tests hold is that biasing never selects lower than the unbiased
ranking, that the DEFAULT bias reaches the presence region on a voice with a
realistic tilt, and that bias 0 still follows raw power exactly.

Includes a radix-2 FFT rather than a dependency; one Welch-style averaged
spectrum over third-octave bands does not justify pulling in a DSP library.

* fix(engine): keep the FX render 16-bit, stereo, and correctly sized

Three defects in the offline FX path, none of which any test could see.

**Float output silently disabled sample-accurate volume automation.** The writer
emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope
into the samples and accepts only 16-bit PCM, returning null otherwise. So
enabling any effect downgraded that track to the ffmpeg expression path — capped
at 32 straight segments, quantising a curved envelope, and on a dense one falling
back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a
limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test
asserts the baker accepts the writer's own output and actually fades it.

**Everything was folded to mono.** `prepareAudioTrack` goes out of its way to
emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo
rematrix — and this folded it, then wrote one channel. So adding a single peaking
EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed
stereo. Channels now travel as one plane each, through an OfflineAudioContext of
the same width, and come back interleaved.

**Small results decoded the wrong length.** `new Float32Array(buf.buffer)`
discards byteOffset and byteLength, and Node pools small allocations: a 400-byte
payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples
decoded as 2048 samples of unrelated memory — and the empty-result guard could
not see it. The reader has the mirror-image fix: a float data chunk on an odd
boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now
copies instead of throwing RangeError on an unaligned view.

The tail limitation is now stated rather than mis-stated: the context is exactly
as long as the input, so a reverb or delay still ringing is cut there. The old
comment claimed the opposite. How far a tail may run past a clip's end changes
the clip's length in the mix, so it is a product decision, not one to make here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(producer): report an FX render failure as an audio error

`processCompositionAudio` reports per-track failures in its result, but an FX
failure it cannot degrade past — a browser that will not launch, a chain that
will not build — rejects instead. `runAudioStage` had no try, so that rejection
escaped to the orchestrator as an unclassified pipeline exception, losing the
stage/owner/retryable classification this stage exists to attach, and skipping
its abort check on the way out.

It now lands in `audioError` alongside every other cause, while an abort still
keeps its own shape rather than being reported as an audio problem.

Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh
clone typechecks packages/engine without building first. The bundle is built from
the stub, and the stub changes three times across this stack — so the artifact
differs per branch and would conflict on every restack. Its model,
position-edits-render-inline.ts, is committed only because it is stable. Building
before testing is this monorepo's existing contract (studio's tests need core's
dist too), so the gap is not specific to audio FX and is better closed by a build
ordering gate than by committing a per-branch artifact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(engine): skip the browser FX render cases when there is no browser

CI's `Test` job was red on this PR with four failures, all the same cause:

  Failed to launch the browser process: spawn
  /home/runner/.cache/hyperframes/chrome/chrome-headless-shell

The job installs ffmpeg and no browser, deliberately — every other suite
that needs an external binary already guards on it
(`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming
a Chrome, so they failed on an absent dependency rather than on anything
about the code.

Guards on `resolveHeadlessShellPath()` — the same resolver
`acquireBrowser` launches through, so the check cannot drift from the
thing it guards the way a hard-coded cache path would. A configured path
that does not exist throws; that is caught and read as "cannot run here".

Checked both directions rather than just the green one: with a browser all
11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a
missing binary exactly 3 skip and the other 8 still run. A guard that
silently skipped everything would have looked identical in CI.

They keep their value where it exists — every developer machine, and any
job that has run `hyperframes browser ensure`.

Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five
days and several force-pushes stale. None of the 17 open repo alerts are
in files this PR changes; it re-runs on this push.

* chore(engine): suppress the temp-file alert with the reason it is safe

CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file
(high) — the one new alert on #3021, and the reason its CodeQL check is
red.

It is a false positive, and the comment says why rather than just silencing
it: `path` is always inside a directory made by `mkdtempSync`, never a
name assembled directly under `tmpdir()`. Both callers are covered — the
browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`,
and the render output goes to the producer work dir, itself
`mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the
random suffix and creates the directory 0700 in one syscall, so the
predictable filename inside it cannot be pre-created or symlinked by
another user, which is the attack the rule is about. The analyzer sees the
dataflow reach `tmpdir()` and not the mkdtemp in between.

Suppressed inline rather than dismissed in the UI, so the justification
lives next to the code and the rule stays live for anything added later in
this file. Matches the repo's existing convention — `planV2.ts:222`
carries an `lgtm[js/insecure-temporary-file]` for a different reason on
the same rule.

Correcting myself: I first reported this alert as not real, having
intersected the PR's files against the default-branch alert list, which
does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns
it straight away.

* test(engine): probe ffmpeg and Chrome instead of assuming them

Two failures on #3021's Test job, both about the environment rather than
the code under test.

**Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to
`execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide
ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()`
resolves — every other ffmpeg-dependent suite in this package already goes
through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)`
so a contributor without ffmpeg skips rather than fails.

**The browser guard trusted the wrong thing.** It asked
`resolveHeadlessShellPath()` and treated a returned path as "a browser is
here". CI's cache holds a chrome-headless-shell that resolves and then
fails to spawn — a partial download is indistinguishable from a working
one by `existsSync`, which is all that resolver checks. So the three
browser cases ran anyway and failed on the launch.

It now runs `--version` and requires exit 0, which is the same probe the
ffmpeg suites use: ask the binary, do not infer from the filesystem.

Checked both directions rather than just the green one. With a working
browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed
at a binary that exits non-zero — CI's exact situation — exactly 3 skip
and the other 8 still run. A guard that quietly skipped everything would
have looked identical on the CI summary.

* feat(core): register the audio-fx-rack canary at 0%

Lands the rollout switch dark, per the registry's own procedure: "Start at
percentage: 0 and merge that — a canary at 0 is dead code you can land
safely and ramp without a code review."

Declared at the bottom of the stack so every branch above can read it. The
gate itself goes in at wa-4-fx-panel, where the rack first appears.

Scope is deliberate and stated in the description: it gates the AUTHORING
surface only. A composition that already carries `data-fx-chain` still
plays and renders it. A canary should stage who can REACH a feature, not
make an attribute somebody already wrote silently inert — an agent that
writes a chain through the skill would otherwise produce a file whose audio
processing vanishes with no error.

* feat(studio): audio FX panel generated from the registry

Controls for the whole chain: add, remove, reorder, bypass, and every knob each
effect declares.

Nothing in the panel knows what a compressor is. The registry supplies each
parameter's range, step, unit and scale and the panel renders what it finds, so
adding an effect or a knob upstream needs no change here, and the panel cannot
offer a value the renderer would reject — a typed-in figure is clamped into the
declared range on the way through.

Frequency and time controls span three or four decades, so those declare a log
scale and the slider maps exponentially; a linear slider would spend most of
its travel somewhere useless.

Reorder is a first-class control because chain order changes the sound: a
reverb before a compressor is not the same as after.

Carve gets its own block rather than an entry in the add menu, with a picker
for the voice track to listen to. It processes this track based on another one,
which is how a sidechain control works — it lives on the track that changes,
and names the source.

* feat(studio): show the Audio FX section on audio tracks

Adds `audioFx` to the editing-affordances contract and renders the FX panel in
the inspector when an `<audio>` element is selected.

The section is audio-only. A `<video>` carries its sound on a separate
`<audio>` element, so an FX chain on the video would have nothing to process.

Chain and carve settings are written straight back onto the element as
serialised attributes, the way colour grading carries its config, so
persistence is an ordinary attribute write and needs no new server route. A
chain that cannot be parsed renders as empty rather than breaking the panel,
and the attribute is left untouched until the user changes something.

The collapsed group summarises what is on the track ("2 effects + carve") so
the state is visible without expanding it.

Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED
defaults to true, so the flat inspector is what actually renders.

* refactor(studio): lift audioFxSummary out of PropertyPanelFlat

`PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap,
so the required File size check is red — the sole reason this PR is
blocked. The review says as much: "mechanical fix (~5 min), not a design
problem. Code itself is LGTM."

Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later
branch creates for it. Deliberately the smallest cut that clears the cap
rather than the whole `AudioFxGroup` extraction: every later commit in the
stack edits AudioFxGroup, so moving it here would collide with each of
them, while almost nothing touches this function.

595 lines.

* feat(core,studio): hear the FX chain in preview, and run the carve analysis

Splices an element's FX chain into the playback graph so preview stops being
silent about effects, and wires the carve button that was previously inert.

The chain goes between the decoded source and its gain stage: effects see the
raw signal and volume automation rides on their output, matching the order the
offline render uses. Since preview and render call the same graph builders,
what is heard while scrubbing is what gets written.

The splice lives in the transport rather than on the `<audio>` element. The
transport plays each track from a decoded AudioBuffer and mutes the element to
avoid doubling, so capturing the element with createMediaElementSource would
have processed a stream nothing is listening to — it looked like it worked
because the call succeeded, and the audio was unchanged.

A chain that cannot be built plays dry rather than silencing the track, which
is the right failure in preview: the author keeps working and hears the source.
The render still refuses, because shipping the dry signal there would be wrong.

Carve now analyses for real: it decodes the chosen voice track, ranks its bands
and writes the resulting peaking filters onto this track. Generated nodes are
tagged `fromCarve`, so re-running replaces the previous carve instead of
stacking another set on top of hand-added effects.

Known limitation: the graph is built when a source is scheduled, so a knob
turned mid-playback takes effect on the next play or seek rather than
immediately. Live re-parameterisation needs the transport to hold the handle
and forward updates.

* fix(studio,core): stop parameter drags from restarting playback

Dragging a knob wrote the chain through the persisting attribute path on every
input event. That path refreshes the preview, which reloads the composition and
reschedules audio — so a single drag reloaded dozens of times and playback
stuttered the whole way.

Drags now go through `onSetAttributeLive`, the same path colour grading uses for
scrubs: it coalesces undo entries and sets `skipRefresh`, so no reload happens.
The persisting write fires once, when the gesture ends — pointer-up or blur for
a slider, Enter or blur for a typed value. A select commits immediately since
there is no drag to wait for.

While dragging, the control is driven from local state. Waiting for the value to
round-trip through the element attribute made the knob lag behind the pointer.

For the change to be audible without a reload, the graph now follows the
attribute: the chain installed by the transport observes the element and
re-parameterises itself in place, so a value change lands on the next
128-sample quantum. A shape change (effect added, bypassed, pole count) cannot
be patched into a running graph, so it still waits for the next schedule rather
than cutting the audio mid-play.

The regression test drags a slider through several values and asserts the
persisting handler is untouched until release.

* feat(studio): put the audio FX rack behind its canary

Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered
at 0% — so the whole 47-PR stack can land without showing anyone a feature
that has not been measured yet.

The gate sits on the AUTHORING surface and nowhere else. The runtime and
the render still honour a `data-fx-chain` already on an element, so a
composition written through the skill or by `carve.mjs` keeps its
processing rather than going silently dry for anyone outside the cohort. A
canary should stage who can REACH a feature, not make an attribute somebody
already wrote stop working with no error.

Gated at the panel rather than in `resolveEditingSections`: the affordance
resolver is a pure function in core describing what an element CAN support,
and rollout state is not a property of an `<audio>` tag.

Pinned the 0% with a test, and checked it fails at 25 — a ramp should have
to break something that says "this ships dark" out loud.

One gap, stated rather than papered over: the gate itself has no unit test.
I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness
never renders the Audio FX group for its audio fixture even with the gate
removed — so the test passed for the wrong reason in the off case and could
not pass at all in the on case. A test that cannot fail for the right
reason is worse than none. Verifying the gate needs the panel harness to
mount that section first, which is its own change.

* fix(core): register FX worklets before building nodes that need them

An AudioWorkletNode cannot be constructed before its processor is registered —
it throws, and the surrounding chain is lost with it. `attachElementFxChain`
built the chain first and only then called `ensureAudioFxWorklets`, so every
worklet-backed effect (compressor, limiter, gate, bitcrush) threw on
construction and the track fell back to dry. Instrumenting the preview showed
`hf-compressor: InvalidStateError` with addModule never called at all.

When the module has not landed yet the track now plays dry and the graph is
swapped in once registration resolves, so the effect arrives a moment late
instead of never.

Registration is also tracked per context rather than in one module-level
promise. A processor registered on one AudioContext does not exist on another,
so the shared promise made every context after the first believe it was ready
when it was not — the studio's transport owns its own context, which is exactly
that case.

With the worklets actually running, the compressor's per-sample log10 and pow
became real audio-thread work. Samples below the knee have a gain of exactly
unity and need neither, so the envelope is now compared in the linear domain
and the transcendentals only run for samples that are actually being
compressed.

* refactor(studio): split the FX node row out of FxSection

Clears the health findings the FX stack left behind: the chain-node render
callback was a 70-line closure over half of FxSection's state, and the two
reorder arrows were the same button written twice.

Also drops two exports with no consumers, and registers the audio FX runtime
stub as an entry point — it is bundled by file path, so nothing imports it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(core): automation envelope model for audio tracks

Adds the data model behind Ableton-style automation lanes: breakpoint
envelopes over track volume or one knob of one effect in the track's FX
chain, stored on the element as `data-automation`.

Times are clip-local, so an envelope travels with the clip when it moves —
the clip-envelope model rather than arrangement automation.

`sampleAutomationLane` is the single interpolator. The lane drawing, the
preview scheduler and the render bake all call it, so the picture and the
sound cannot disagree about the curve. Log-scaled parameters interpolate in
log space, matching what their own knob already promises.

FX nodes gain a stable `id`, minted by count rather than randomly so the
document is the same on every machine. Lanes address nodes by id, so
reordering a chain never re-points a lane at a different effect, and a lane
whose effect was deleted is dropped rather than left to reattach.

Also warns when a track carries both a volume lane and a GSAP volume tween,
since only the lane is heard and the tween silently does nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(studio): lift the audio FX group out of PropertyPanelFlat

`PropertyPanelFlat.tsx` was 672 lines against the repo's 600-line cap, so
the required File size check was red — the sole reason #3014 and #3022 are
blocked. Both reviews say the same thing: "mechanical fix, not a design
problem. Code itself is LGTM."

Moves `AudioFxGroup` and `audioFxSummary` into
`propertyPanelAudioFxGroup.tsx`, which is where a later branch puts them
anyway — done here so the file is under the cap from the point it first
crosses it, rather than ten branches later.

533 lines now. The four audio imports it no longer needs go with it.

Not fixed here: three `FxSection carve` tests fail on this branch with
"Cannot read properties of undefined (reading 'toFixed')". Confirmed
pre-existing by stashing this change and re-running — that is the separate
`Test` failure the review also flags.

* feat(core): expose the AudioParams behind automatable FX knobs

Marks the knobs an automation lane can drive and has each graph builder hand
back the AudioParam behind them, so a scheduler can write to a running effect
without knowing what the effect is.

A knob is not always one AudioParam. A wet/dry mix is two gains moving in
opposition, and a knob in milliseconds drives a delay time in seconds, so
each target carries the mapping out of the knob's own declared unit.

What stays unautomatable is stated where it is decided: a WaveShaper curve, a
convolution impulse and a one-pole filter's coefficients are all rebuilt
wholesale rather than scheduled, and the four worklet effects take values by
postMessage rather than through AudioParams.

The registry flag is written by hand, so a test builds every effect and
checks the exposure both ways — nothing flagged is missing, nothing exposed
is unflagged. A flag that lied would offer a lane that silently did nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(core): play automation envelopes in preview

Schedules each lane onto the AudioParams behind its knob using native ramps
and value curves. Nothing evaluates the envelope per frame: it is handed to
the audio thread once, so it stays sample-accurate however busy the main
thread is, and the offline render will schedule it the same way.

Timing comes from the transport, so an envelope survives seeking into the
middle of a clip, a clip that has not started yet, and a playback rate that
compresses clip seconds into context seconds.

A straight line is only scheduled as a ramp when nothing bends it — no
curvature, a linear parameter scale, and no unit mapping. Log-scaled
parameters and mapped ones are sampled instead, since a delay knob in
milliseconds and a wet/dry pair moving in opposition are not linear in the
parameter they drive.

Lanes with nowhere to write are skipped rather than reported: a one-pole
filter exposes no frequency param, and the worklet effects expose none at
all. Editing an envelope mid-playback re-aims it at the live playhead rather
than restarting the track.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(core): make the volume lane audible in preview

The envelope was scheduled onto the transport's gain AudioParam, but the
runtime rewrites that gain every tick from `data-volume` and the GSAP-seeked
value — so it was erased within a frame. Volume automation was correct in the
render and inaudible while previewing.

The lane now feeds the per-tick path where the probed volume keyframes already
sit, checked ahead of them so the two cannot fight, and the transport no
longer schedules volume at all: one mechanism instead of two racing.

The cost is honest — in preview the level steps per tick rather than per
sample, exactly as the existing keyframe path does. The render still bakes it
into the PCM sample-accurately, and FX parameters are still scheduled on their
own AudioParams, since nothing rewrites those.

Parsed lanes are cached by attribute text: the runtime asks once per tick per
track, and parsing there would run the JSON parser 60 times a second for a
value that only changes on an edit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(engine): bake automation envelopes into the render

The offline render schedules FX lanes with the same scheduler preview uses,
inside the OfflineAudioContext that already runs the same graph builders. The
input WAV is the clip's own audio from its first sample, so clip-local time
is offline time and the envelope needs no offset.

Volume lanes take the existing PCM bake rather than a second mechanism: the
lane is converted to keyframes, so a straight fade stays two of them and only
a bent segment is sampled — the baker interpolates linearly and would
otherwise quietly straighten the curve. A volume lane supersedes keyframes
probed from the timeline, which `lint` already warns about.

A browser test sweeps a lowpass from below a 2 kHz tone to well above it and
measures both ends. Parsing the envelope is not the same as scheduling it,
and only running the real thing tells the two apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(core): apply chain edits to the running graph

A structural edit — an effect added, removed, bypassed, or a filter's pole
count switched — was dropped. `buildFxChain`'s update reports false when the
change is not merely new values, and the attribute observer ignored that, so
the edit only took hold when the persisting write reloaded the composition.
That reload restarted every playing track, which is what was heard as the
audio chopping.

The graph is now swapped in place: the old effects are detached, the new ones
built and connected between the same source and gain, and any lanes
re-scheduled onto the new nodes. The source node is never touched, so playback
does not restart.

A track with no chain is watched too, rather than wired through and forgotten,
so adding its first effect is heard the same way. That means the function
always returns a disposer instead of null for the empty case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(studio): drop the FX panel's dead __testables export

Fallow audit flagged it — no test imports the module.

* fix(core,studio): clear the remaining Fallow audit findings on the FX panel

- Split FxSection's per-node row into FxNodeRow + FxNodeControls so the
  CRAP score (31.6, threshold 30) splits across two smaller units instead
  of moving wholesale with one extraction.
- Dedupe the repeated "open the add menu, read its items" block in
  propertyPanelFxSection.test.tsx into openAddMenuItems().
- Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into
  one build-inline-artifact.ts, config-selected by CLI arg — the two
  scripts were a byte-for-byte clone save for names.
- Exempt canary.test.ts's rawFnv (a deliberate independent
  reimplementation used to cross-check canaryBucket, per its own
  docstring) and the property-panel test files' shared renderInto/mount
  scaffolding (pre-existing across 9 files, 2 outside this stack) in
  .fallowrc.jsonc, consistent with this file's existing exemptions for
  the same class of intentional/pre-existing duplication.

* fix(ci): allowlist the build-script consolidation in the no-main-deletions guard

build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into
build-inline-artifact.ts to kill a fallow duplication finding; the deletion
guard flagged that as an accidental loss since main still has both originals.

* fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo

Both effect builders set wet.gain to the mix and dry.gain to its complement
in identical two-line blocks; fallow kept re-flagging it as a 10-line clone
on every unrelated change. Extracted setWetDryMix.

* fix(core): remove the build-audio-fx-runtime.ts stray resurrected by a main merge

An earlier merge with main brought this deleted file back (git's merge/delete
handling on an unchanged-on-one-side file); package.json already points at
build-inline-artifact.ts, so it sat unreachable and duplicating that file's
config, both of which fallow flagged.

* fix(studio): pull TimelineLanes under the 600-line cap

TimelineLanes.tsx hit 620 lines. Extracted the three per-clip pointer
gestures (resize-start, pointer-down move-arm, click/razor-split) into
createClipGestureHandlers — one factory call per rendered clip instead of
~120 lines of inline handler bodies in the render loop. 529 lines now.

* fix(studio): split the extracted pointerdown handler under the CRAP threshold

Moving the ~120-line gesture logic into timelineClipGestureHandlers.ts
concentrated it into two functions fallow flagged (onPointerDown at CRAP
63.6, onResizeStart at 31.6). Split the decision logic (which gesture a
pointerdown implies) into a pure resolvePointerDownAction, then split
its own intent-blocking check into isIntentBlocked. onResizeStart's guard
moved into canStartResize. Every function now scores under 30.

* fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat

CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the
stack removed the last use of the type here without removing the import.

* fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened

useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a
preview-only commit that useDomEditAttributeCommits.ts never grew — backported
that option support from its own later commit so the two sides of the API
agree. The paste path and its tests were missing the box selection's v0/v1
bounds a sibling commit added to AutomationSelection. The FX panel's carve
controls still edited the six mechanism numbers (maxCutDb, bands,
intelligibilityBias) after carveProfile() collapsed authoring to one Strength
knob, so those fields no longer existed on HfCarveSettings; UI now edits
strength, and analyseCarveBands is called with carveProfile(strength).

Also closes fallow's complexity, dead-code and duplication findings on this
PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math)
and useAutomationRangeDrag.ts (the marquee-select gesture) out of
useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a
resolver into named functions, dropped an export nothing outside its file
used, and shared a step-simplifier between audioCarve's two envelope
builders.

The edge-stretch vs. box-select priority test in TimelineAutomationLane.test
was still pinning the pre-box-select rule (edge wins over a point sitting on
it) that a sibling commit deliberately reversed — a point inside the box is
now selected content, so grabbing it drags the group instead. Updated the
test to the shipped rule instead of the old one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(core): cap the via conic's weight so an edge-clamped via point can't NaN

A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to
(0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0.
viaConic divided by that zero to get an infinite weight, and shapeVia turned
Infinity into NaN a few steps later (Infinity - Infinity in the quadratic
coefficient). NaN reaching setValueCurveAtTime silences the automated
parameter for the rest of the render.

Capped the weight at 1e6 instead of leaving it unbounded — past that point
the arc already reads as touching the via point, so nothing visible is lost.
Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`,
since NaN fails the original comparison and fell through it.

Review by Miga (PR #3208).

* fix(studio): widen PropertyPanel's resetModules render timeout again

The 20s margin (already once widened for the same reason) is timing out in
CI's full-monorepo Test run — the resetModules()+fresh-import render this
test needs is uncached and competes with every other package's test suite
for the same worker pool, and the same test passes in well under 2s
standalone. Went to 45s rather than re-tuning to whatever number happens to
clear the current CI load, since that number moves every time CI gains a
package.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-08-13 01:31:57 -07:00
committed by GitHub
co-authored by Claude Opus 5
parent 69face9dc6
commit 7d38fed97f
19 changed files with 1711 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/studio/src/components/StudioRightPanel.tsx
+16 -1
View File
@@ -327,6 +327,21 @@ export function StudioRightPanel({
},
[projectId, refreshFileTree, showToast],
);
/**
* A dial being dragged writes to the preview and stops there.
*
* Every one of these panels previews on each pointermove and commits on
* release. Persisting the moves too put a fragment of the drag in the undo
* stack — and since those writes race, history could not coalesce them
* reliably, so undo took back a sliver of the gesture rather than the gesture.
* The release's own commit is what reaches the file and the undo stack.
*/
const setAttributeWhileDragging = useCallback(
(attr: string, value: string | null) =>
handleDomAttributeLiveCommit(attr, value, undefined, { previewOnly: true }),
[handleDomAttributeLiveCommit],
);
const handleHideAllSelected = () => {
const { elements } = usePlayerStore.getState();
const keys = timelineKeysForSelections(domEditGroupSelections, elements, activeCompPath);
@@ -361,7 +376,7 @@ export function StudioRightPanel({
onSetStyle={handleDomStyleCommit}
onSetAttribute={handleDomAttributeCommit}
onSetAttributes={handleDomAttributesCommit}
onSetAttributeLive={handleDomAttributeLiveCommit}
onSetAttributeLive={setAttributeWhileDragging}
onSetAttributeQuiet={handleDomAttributeQuietCommit}
onApplyColorGradingScope={handleApplyColorGradingScope}
onSetHtmlAttribute={handleDomHtmlAttributeCommit}
packages/studio/src/components/TimelineToolbar.test.tsx
+46
View File
@@ -118,3 +118,49 @@ describe("TimelineToolbar — motion path endpoints", () => {
act(() => root.unmount());
});
});
describe("TimelineToolbar — keyframes on audio tracks", () => {
const clip = (tag: string) => ({
id: "bgm",
key: "bgm",
tag,
start: 0,
duration: 10,
track: 1,
});
/** A session whose selection would otherwise offer the keyframe toggle. */
function sessionFor(tag: string) {
usePlayerStore.setState({ elements: [clip(tag)], selectedElementId: "bgm", currentTime: 1 });
const element = document.createElement(tag);
element.id = "bgm";
return {
domEditSelection: makeSelection("Element", element),
selectedGsapAnimations: [],
handleGsapAddAnimation: vi.fn(),
handleGsapConvertToKeyframes: vi.fn(),
handleGsapRemoveKeyframe: vi.fn(),
} satisfies NonNullable<React.ComponentProps<typeof TimelineToolbar>["domEditSession"]>;
}
it("offers no keyframe toggle for an audio clip", () => {
// An audio clip has no box on the canvas, so there is nothing to move or fade —
// and pressing this seeded a tween from the position properties, which put a
// position lane on a track that has no position. Audio is automated instead.
const { host, root } = renderToolbar(sessionFor("audio"));
const button = host.querySelector<HTMLButtonElement>(
'button[aria-label="Add keyframe at playhead"]',
);
expect(button?.disabled).toBe(true);
act(() => root.unmount());
});
it("still offers it for a visual clip", () => {
const { host, root } = renderToolbar(sessionFor("div"));
const button = host.querySelector<HTMLButtonElement>(
'button[aria-label="Add keyframe at playhead"]',
);
expect(button?.disabled).toBe(false);
act(() => root.unmount());
});
});
packages/studio/src/components/TimelineToolbar.tsx
+17
View File
@@ -86,8 +86,22 @@ function resolveKeyframeToggleState(
};
}
/**
* Can this element be keyframed at all?
*
* An audio clip cannot. It has no box on the canvas, so there is nothing to move,
* scale or fade — and "add a keyframe" on one seeds a tween from the position
* properties, which produced a position lane on a track that has no position. Audio
* is automated instead: volume and effect parameters, on their own lanes.
*/
function isKeyframeable(element: TimelineElement | undefined): boolean {
return element?.tag !== "audio";
}
function useKeyframeToggle(session?: DomEditSessionSlice) {
const currentTime = usePlayerStore((s) => s.currentTime);
const selectedElementId = usePlayerStore((s) => s.selectedElementId);
const elements = usePlayerStore((s) => s.elements);
const sessionRef = useRef(session);
sessionRef.current = session;
@@ -95,6 +109,9 @@ function useKeyframeToggle(session?: DomEditSessionSlice) {
sessionRef as React.RefObject<EnableKeyframesSession | undefined>,
);
const selected = elements.find((element) => (element.key ?? element.id) === selectedElementId);
if (!isKeyframeable(selected)) return { ...NO_KEYFRAME_TOGGLE, onToggle: undefined };
const toggleState = resolveKeyframeToggleState(session, currentTime);
return {
packages/studio/src/components/editor/PropertyPanel.tsx
+7 -23
View File
@@ -1,5 +1,5 @@
import { scopedElementKey } from "../../hooks/gsapKeyframeCacheHelpers";
import { memo, useEffect, useMemo, useRef, useState } from "react";
import { memo, useMemo, useRef, useState } from "react";
import { Move } from "../../icons/SystemIcons";
import { InspectorHeaderActions } from "./InspectorHeaderActions";
import { useStudioShellContext } from "../../contexts/StudioContext";
@@ -29,7 +29,8 @@ import { KeyframeNavigation } from "./KeyframeNavigation";
import { STUDIO_FLAT_INSPECTOR_ENABLED } from "./manualEditingAvailability";
import { PropertyPanelFlat } from "./PropertyPanelFlat";
import { createGsapLivePreview } from "./gsapLivePreview";
import { usePlayerStore, liveTime } from "../../player";
import { usePlayerStore } from "../../player";
import { useLivePlayheadTime } from "../../hooks/useLivePlayheadTime";
import { TimingSection } from "./propertyPanelTimingSection";
import { type PropertyPanelProps } from "./propertyPanelHelpers";
import { GestureRecordPanelButton } from "./GestureRecordControl";
@@ -114,31 +115,14 @@ export const PropertyPanel = memo(function PropertyPanel(props: PropertyPanelPro
const { showToast } = useStudioShellContext();
const [clipboardCopied, setClipboardCopied] = useState(false);
const clipboardTimerRef = useRef<ReturnType<typeof setTimeout>>(undefined);
const storeTime = usePlayerStore((s) => s.currentTime);
const isPlaying = usePlayerStore((s) => s.isPlaying);
const timelineElements = usePlayerStore((s) => s.elements);
const selectedElementId = usePlayerStore((s) => s.selectedElementId);
const selectedElementHidden = isSelectedElementHidden(timelineElements, selectedElementId);
const visibilityToggleLabel = selectedElementHidden ? "Show element" : "Hide element";
const liveTimeRef = useRef(storeTime);
const [, forceRender] = useState(0);
useEffect(() => {
if (!isPlaying) return;
let timerId: ReturnType<typeof setTimeout> | 0 = 0;
const unsub = liveTime.subscribe((t) => {
liveTimeRef.current = t;
if (!timerId)
timerId = setTimeout(() => {
timerId = 0;
forceRender((v) => v + 1);
}, 33);
});
return () => {
unsub();
if (timerId) clearTimeout(timerId);
};
}, [isPlaying]);
const currentTime = isPlaying ? liveTimeRef.current : storeTime;
// Live during playback, the store's when paused — see the hook. Shared with the
// audio FX panel, which follows the playhead for the same reason: a value the
// timeline drives has to be shown moving, not frozen at what the attribute says.
const currentTime = useLivePlayheadTime();
const cacheElementKey = element?.id ?? element?.selector ?? "";
const cacheEntry = usePlayerStore((s) => s.keyframeCache.get(cacheElementKey));
packages/studio/src/components/editor/PropertyPanelFlat.tsx
+2 -2
View File
@@ -6,6 +6,7 @@ import { slugifyDesignInput } from "../../utils/designInputTracking";
import { isTextEditableSelection } from "./domEditing";
import type { PropertyPanelFlatProps } from "./propertyPanelFlatProps";
import { formatPxMetricValue } from "./propertyPanelHelpers";
import { audioFxSummary } from "./audioFxSummary";
import { PropertyPanelFlatHeader } from "./PropertyPanelFlatHeader";
import { PropertyPanelFlatFooter } from "./PropertyPanelFlatFooter";
import { FlatGroupHeader } from "./propertyPanelFlatPrimitives";
@@ -14,8 +15,7 @@ import { FlatStyleSection } from "./propertyPanelFlatStyleSections";
import { FlatLayoutSection } from "./propertyPanelFlatLayoutSection";
import { FlatMotionSection } from "./propertyPanelFlatMotionSection";
import { isCanaryEnabled } from "../../telemetry/canary";
import { audioFxSummary } from "./audioFxSummary";
import { AudioFxGroup } from "./propertyPanelAudioFxGroup";
import { AudioFxGroup } from "./propertyPanelAudioFxGroup.js";
import { useVolumeAutomation } from "./useVolumeAutomation";
import { FlatMediaSection } from "./propertyPanelFlatMediaSection";
import { deriveElementTiming } from "./propertyPanelFlatTimingDerivation";
packages/studio/src/components/editor/audioFxSummary.test.ts
+75
View File
@@ -0,0 +1,75 @@
import { describe, expect, it } from "vitest";
import { audioFxSummary } from "./audioFxSummary";
import type { DomEditSelection } from "./domEditingTypes";
const el = (dataAttributes: Record<string, string>): DomEditSelection =>
({ dataAttributes }) as unknown as DomEditSelection;
const chain = (nodes: unknown[]) => JSON.stringify({ version: 1, nodes });
describe("audioFxSummary", () => {
it("counts a carve as one module, not as the filters behind it", () => {
// Six bands and a level stage reading "7 effects" is the misreading the
// grouping exists to prevent.
const summary = audioFxSummary(
el({
"fx-chain": chain([
{ type: "peaking", id: "n1", fromCarve: true, params: { frequency: 400 } },
{ type: "peaking", id: "n2", fromCarve: true, params: { frequency: 1600 } },
{ type: "gain", id: "n3", fromCarve: true, params: { gain: -6 } },
]),
"fx-carve": JSON.stringify({ source: "vo", strength: 0.25 }),
}),
);
expect(summary).toBe("carve");
});
it("counts hand-built effects alongside the module", () => {
expect(
audioFxSummary(
el({
"fx-chain": chain([
{ type: "peaking", id: "n1", fromCarve: true, params: { frequency: 400 } },
{ type: "lowpass", id: "n2", params: { frequency: 8000 } },
{ type: "delay", id: "n3", params: { time: 200 } },
]),
}),
),
).toBe("2 effects + carve");
});
it("says how many when there is no carve", () => {
expect(audioFxSummary(el({ "fx-chain": chain([{ type: "lowpass", id: "n1" }]) }))).toBe(
"1 effect",
);
});
it("names a carve that is on but has not compiled to filters yet", () => {
// Switching it on with no voice chosen leaves the control in this section with
// nothing behind it; the summary should still say the section holds one.
expect(audioFxSummary(el({ "fx-carve": JSON.stringify({ source: "", strength: 0.25 }) }))).toBe(
"carve",
);
});
it("ignores bypassed effects, as it always did", () => {
expect(
audioFxSummary(
el({
"fx-chain": chain([
{ type: "lowpass", id: "n1", enabled: false },
{ type: "delay", id: "n2" },
]),
}),
),
).toBe("1 effect");
});
it("says none for a track with neither", () => {
expect(audioFxSummary(el({}))).toBe("none");
});
it("says so when the chain cannot be read", () => {
expect(audioFxSummary(el({ "fx-chain": "{not json" }))).toBe("unreadable");
});
});
packages/studio/src/components/editor/audioFxSummary.ts
+18 -9
View File
@@ -1,27 +1,36 @@
/**
* What the collapsed Audio FX group says it holds.
*
* Its own module because `PropertyPanelFlat.tsx` is at the repo's 600-line
* budget, and this is the piece with no dependency on the panel around it.
* It has to describe the rack the author would see on opening it, which counts a
* carve as one module rather than as the filters it compiles to. Six peaking
* bands and a level stage reading "7 effects" invited exactly the misreading the
* grouping exists to prevent — that they are seven things to manage.
*/
import { parseAudioFxChain } from "@hyperframes/core/audio-fx";
import type { DomEditSelection } from "./domEditing";
import type { DomEditSelection } from "./domEditingTypes";
/** Chain length at a glance, so the collapsed group says whether anything is on. */
export function audioFxSummary(element: DomEditSelection): string {
const raw = element.dataAttributes?.["fx-chain"];
const carve = element.dataAttributes?.["fx-carve"];
let count = 0;
const carveAttr = element.dataAttributes?.["fx-carve"];
let handBuilt = 0;
let carveNodes = 0;
if (raw) {
try {
count = parseAudioFxChain(raw).nodes.filter((n) => n.enabled !== false).length;
for (const node of parseAudioFxChain(raw).nodes) {
if (node.enabled === false) continue;
if (node.fromCarve) carveNodes += 1;
else handBuilt += 1;
}
} catch {
return "unreadable";
}
}
const parts: string[] = [];
if (count > 0) parts.push(`${count} effect${count === 1 ? "" : "s"}`);
if (carve) parts.push("carve");
if (handBuilt > 0) parts.push(`${handBuilt} effect${handBuilt === 1 ? "" : "s"}`);
// One name for the module however many filters are behind it. Named when the
// carve is switched on at all, because the control is in this section whether or
// not it has compiled to anything yet.
if (carveNodes > 0 || carveAttr) parts.push("carve");
return parts.length > 0 ? parts.join(" + ") : "none";
}
packages/studio/src/components/editor/propertyPanelAudioFxGroup.test.tsx
+621 -8
View File
@@ -4,6 +4,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";
import { createRoot } from "react-dom/client";
import { AudioFxGroup } from "./propertyPanelAudioFxGroup.js";
import type { DomEditSelection } from "./domEditingTypes";
import { liveTime, usePlayerStore } from "../../player";
(globalThis as unknown as { IS_REACT_ACT_ENVIRONMENT: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
@@ -174,13 +175,7 @@ describe("AudioFxGroup carve", () => {
],
});
const carveOn = JSON.stringify({
source: "vo",
maxCutDb: 6,
bands: 3,
q: 1.4,
intelligibilityBias: 0.7,
});
const carveOn = JSON.stringify({ source: "vo", strength: 0.5, dynamic: false });
const carveToggle = (host: HTMLElement): HTMLButtonElement => {
const block = host.querySelector(".hf-fx-carve")!;
@@ -232,7 +227,8 @@ describe("AudioFxGroup carve", () => {
act(() => {
// React's value tracker swallows a plain assignment, so go through the
// prototype setter the way the other panel tests do.
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set?.call(dial, "0.5");
// A different value than the carve holds; setting the same one is not a change.
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set?.call(dial, "0.8");
dial?.dispatchEvent(new Event("input", { bubbles: true }));
});
expect(onSetAttributeLive.mock.calls.map((c) => c[0])).toEqual(["data-fx-carve"]);
@@ -250,6 +246,372 @@ describe("AudioFxGroup carve", () => {
});
});
describe("AudioFxGroup dynamic carve", () => {
const carvedChain = JSON.stringify({
version: 1,
nodes: [{ type: "lowpass", id: "n1", params: { frequency: 400, q: 0.9, poles: "2" } }],
});
// Strength 0 carves frequencies only — no level ducking — so the spectral
// cases measure just the spectral half. A case that wants the duck raises it.
const settings = (dynamic: boolean, over: Record<string, unknown> = {}) =>
JSON.stringify({ source: "vo", strength: 0, dynamic, ...over });
/** The value written for one attribute, whatever order the writes landed in. */
const writeFor = (calls: unknown[][], attr: string) =>
JSON.parse(String(calls.find((c) => c[0] === attr)![1]));
/** Choose a voice track the way the select does. */
const pickSource = (host: HTMLElement, id: string) => {
const select = host.querySelector<HTMLSelectElement>(".hf-fx-carve select")!;
Object.getOwnPropertyDescriptor(HTMLSelectElement.prototype, "value")?.set?.call(select, id);
select.dispatchEvent(new Event("change", { bubbles: true }));
};
const dynamicBox = (host: HTMLElement) =>
host.querySelector<HTMLInputElement>(".hf-fx-carve-dynamic")!;
/** A voice with a pause in it, decoded through a stubbed offline context. */
function stubDecode(): void {
const sampleRate = 48000;
const data = new Float32Array(sampleRate * 4);
for (let i = 0; i < data.length; i++) {
const t = i / sampleRate;
data[i] = t > 1 && t < 3 ? 0.7 * Math.sin(2 * Math.PI * 1000 * t) : 0;
}
vi.stubGlobal(
"fetch",
vi.fn(async () => ({ arrayBuffer: async () => new ArrayBuffer(8) })),
);
vi.stubGlobal(
"OfflineAudioContext",
class {
decodeAudioData = async () => ({ sampleRate, getChannelData: () => data });
},
);
}
afterEach(() => vi.unstubAllGlobals());
it("records the choice in the carve settings", () => {
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedChain,
"fx-carve": settings(false),
});
act(() => dynamicBox(host).click());
const write = onSetAttributeQuiet.mock.calls.find((c) => c[0] === "data-fx-carve");
expect(JSON.parse(String(write![1])).dynamic).toBe(true);
});
it("automates the carve filters' gain from the voice, in the bed's own time", async () => {
stubDecode();
// Voice starts 10s into the composition, bed at 0: the envelope is measured
// against the voice but read from the start of the bed, so it has to shift.
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedChain,
// No source yet: picking one is what applies the carve.
"fx-carve": settings(true, { source: "" }),
start: "0",
});
const vo = document.getElementById("vo")!;
vo.setAttribute("data-start", "10");
vo.setAttribute("src", "voice.wav");
await act(async () => {
pickSource(host, "vo");
});
// Chain first, then automation: a lane naming a node the chain does not
// carry yet is dropped when it is read back.
const order = onSetAttributeQuiet.mock.calls.map((c) => c[0]);
// The settings land first, then the filters they imply, then the envelopes.
expect(order.indexOf("data-fx-chain")).toBeLessThan(order.indexOf("data-automation"));
const carved = writeFor(onSetAttributeQuiet.mock.calls, "data-fx-chain").nodes;
const carveNode = carved.find((n: { fromCarve?: boolean }) => n.fromCarve);
expect(carveNode.id).toBeTruthy();
const lanes = writeFor(onSetAttributeQuiet.mock.calls, "data-automation").lanes;
const lane = lanes.find((l: { target: string }) => l.target === `fx.${carveNode.id}.gain`) as {
points: { t: number; v: number }[];
};
expect(lane).toBeTruthy();
// Flat at the bed's own start, before the voice exists at all.
expect(lane.points[0]).toMatchObject({ t: 0, v: 0 });
// The voice's pause is at 0-1s of its own clip, so 10-11s of the bed's.
expect(lane.points.find((p) => p.t > 10.5 && p.t < 11)?.v ?? 0).toBe(0);
// And it cuts once the voice speaks, a second later. Depth is per band and
// relative to that band's own peak in the voice, so the invariant is that the
// envelope gets most of the way to what the analysis put on the node — not a
// fixed number of dB, which changes with the band the analysis chose.
const bandGain = Number(carveNode.params?.gain ?? 0);
// At least half the depth the analysis put on the node; the exact floor
// depends on which band it chose and how the envelope was thinned.
expect(Math.min(...lane.points.map((p) => p.v))).toBeLessThanOrEqual(bandGain * 0.5);
// Ends back at no cut, so the bed is not left dipped for the rest of the clip.
expect(lane.points.at(-1)!.v).toBe(0);
});
it("adds a gain stage that ducks the bed under the voice, automated when dynamic", async () => {
// Carving frequencies cannot beat a bed that is simply louder than the
// voice. The level half rides a gain node the carve owns, so the track's own
// volume lane is left alone.
stubDecode();
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedChain,
"fx-carve": settings(true, { strength: 1, source: "" }),
start: "0",
});
const vo = document.getElementById("vo")!;
vo.setAttribute("data-start", "0");
vo.setAttribute("src", "voice.wav");
// The bed is measured too — "how far over the voice is it" needs both.
document.getElementById("bed")!.setAttribute("src", "bed.m4a");
await act(async () => {
pickSource(host, "vo");
});
const nodes = writeFor(onSetAttributeQuiet.mock.calls, "data-fx-chain").nodes;
const gain = nodes.find((n: { type: string }) => n.type === "gain");
expect(gain).toBeTruthy();
expect(gain.fromCarve).toBe(true);
// Dynamic hands the value to the envelope, so the static one stays at unity.
expect(gain.params.gain).toBe(0);
const lanes = writeFor(onSetAttributeQuiet.mock.calls, "data-automation").lanes;
const duckLane = lanes.find((l: { target: string }) => l.target === `fx.${gain.id}.gain`);
expect(duckLane).toBeTruthy();
expect(Math.min(...duckLane.points.map((p: { v: number }) => p.v))).toBeLessThan(0);
// Every carved band gets an envelope reaching that band's own analysed depth.
for (const node of nodes.filter((n: { type: string }) => n.type === "peaking")) {
const lane = lanes.find((l: { target: string }) => l.target === `fx.${node.id}.gain`) as
| { points: { v: number }[] }
| undefined;
expect(lane, `band ${node.id} has no envelope`).toBeTruthy();
const deepest = Math.min(...lane!.points.map((p) => p.v));
expect(deepest).toBeLessThanOrEqual(0);
expect(deepest).toBeGreaterThanOrEqual(node.params.gain - 0.2);
expect(deepest).toBeLessThanOrEqual(node.params.gain * 0.5);
}
// The author's own volume lane is not something a carve gets to touch.
expect(lanes.some((l: { target: string }) => l.target === "volume")).toBe(false);
});
it("holds one measured value when the carve is not dynamic", async () => {
stubDecode();
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedChain,
"fx-carve": settings(false, { strength: 1, source: "" }),
start: "0",
});
const vo = document.getElementById("vo")!;
vo.setAttribute("data-start", "0");
vo.setAttribute("src", "voice.wav");
// The bed is measured too — "how far over the voice is it" needs both.
document.getElementById("bed")!.setAttribute("src", "bed.m4a");
await act(async () => {
pickSource(host, "vo");
});
const nodes = writeFor(onSetAttributeQuiet.mock.calls, "data-fx-chain").nodes;
const gain = nodes.find((n: { type: string }) => n.type === "gain");
expect(gain.params.gain).toBeLessThan(0);
// Nothing to schedule: a static carve is a value, not an envelope.
expect(onSetAttributeQuiet.mock.calls.some((c) => c[0] === "data-automation")).toBe(false);
});
it("carves frequencies only when the duck is off", async () => {
stubDecode();
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedChain,
"fx-carve": settings(true, { strength: 0, source: "" }),
start: "0",
});
document.getElementById("vo")!.setAttribute("src", "voice.wav");
document.getElementById("bed")!.setAttribute("src", "bed.m4a");
await act(async () => {
pickSource(host, "vo");
});
const nodes = writeFor(onSetAttributeQuiet.mock.calls, "data-fx-chain").nodes;
expect(nodes.some((n: { type: string }) => n.type === "gain")).toBe(false);
});
it("applies as soon as a voice track is picked, with no second step", async () => {
// A carve with a source and no filters is a setting nobody applied. Choosing
// the voice is the whole gesture.
stubDecode();
const { host, onSetAttributeQuiet } = mount({
"fx-chain": JSON.stringify({ version: 1, nodes: [] }),
"fx-carve": JSON.stringify({ source: "", strength: 0.25, dynamic: true }),
start: "0",
});
document.getElementById("vo")!.setAttribute("src", "voice.wav");
document.getElementById("bed")!.setAttribute("src", "bed.m4a");
await act(async () => {
pickSource(host, "vo");
});
const written = onSetAttributeQuiet.mock.calls.map((c) => c[0]);
expect(written).toEqual(["data-fx-carve", "data-fx-chain", "data-automation"]);
const nodes = JSON.parse(
String(onSetAttributeQuiet.mock.calls.find((c) => c[0] === "data-fx-chain")![1]),
).nodes;
expect(nodes.every((n: { fromCarve?: boolean }) => n.fromCarve)).toBe(true);
});
it("re-applies when dynamic is switched on, not just when strength moves", async () => {
stubDecode();
const carvedAlready = JSON.stringify({
version: 1,
nodes: [
{
type: "peaking",
id: "n1",
fromCarve: true,
params: { frequency: 1000, gain: -6, q: 1.4 },
},
],
});
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedAlready,
"fx-carve": settings(false, { strength: 0.25 }),
start: "0",
});
document.getElementById("vo")!.setAttribute("src", "voice.wav");
document.getElementById("bed")!.setAttribute("src", "bed.m4a");
await act(async () => {
host.querySelector<HTMLInputElement>(".hf-fx-carve-dynamic")!.click();
});
// Static and dynamic are different chains, so the switch has to rebuild them.
expect(onSetAttributeQuiet.mock.calls.some((c) => c[0] === "data-fx-chain")).toBe(true);
});
it("re-applies an existing carve when strength moves", async () => {
// Strength is the whole control surface, so it has to act on what is already
// applied. Left to the button alone, a carve kept the filters and envelopes
// its old strength produced and the knob silently described nothing.
stubDecode();
const carvedAlready = JSON.stringify({
version: 1,
nodes: [
{
type: "peaking",
id: "n1",
fromCarve: true,
params: { frequency: 1000, gain: -6, q: 1.4 },
},
{ type: "gain", id: "n2", fromCarve: true, params: { gain: -6 } },
],
});
const { host, onSetAttributeQuiet } = mount({
"fx-chain": carvedAlready,
"fx-carve": settings(true, { strength: 0.25 }),
start: "0",
});
document.getElementById("vo")!.setAttribute("src", "voice.wav");
document.getElementById("bed")!.setAttribute("src", "bed.m4a");
const dial = host.querySelector<HTMLInputElement>(".hf-fx-carve input[type=range]")!;
await act(async () => {
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set?.call(dial, "1");
dial.dispatchEvent(new Event("input", { bubbles: true }));
dial.dispatchEvent(new PointerEvent("pointerup", { bubbles: true }));
});
const written = onSetAttributeQuiet.mock.calls.map((c) => c[0]);
expect(written).toContain("data-fx-carve");
// The settings land first, then the filters they imply, then the envelopes.
expect(written.indexOf("data-fx-carve")).toBeLessThan(written.indexOf("data-fx-chain"));
expect(written.indexOf("data-fx-chain")).toBeLessThan(written.indexOf("data-automation"));
const chainWrite = onSetAttributeQuiet.mock.calls.find((c) => c[0] === "data-fx-chain");
const nodes = JSON.parse(String(chainWrite![1])).nodes;
// Full strength: deeper than the 6 dB the quarter-strength carve had.
const deepest = Math.min(
...nodes
.filter((n: { type: string }) => n.type === "peaking")
.map((n: { params: { gain: number } }) => n.params.gain),
);
expect(deepest).toBeLessThan(-6);
});
it("does nothing but record the setting while no voice track is chosen", async () => {
// There is nothing to listen to, so there is nothing to derive. This is the
// one case that only writes the setting now that the apply button is gone.
stubDecode();
const { host, onSetAttributeQuiet } = mount({
"fx-chain": JSON.stringify({ version: 1, nodes: [] }),
"fx-carve": settings(true, { strength: 0.25, source: "" }),
start: "0",
});
const dial = host.querySelector<HTMLInputElement>(".hf-fx-carve input[type=range]")!;
await act(async () => {
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set?.call(dial, "1");
dial.dispatchEvent(new Event("input", { bubbles: true }));
dial.dispatchEvent(new PointerEvent("pointerup", { bubbles: true }));
});
expect(onSetAttributeQuiet.mock.calls.map((c) => c[0])).toEqual(["data-fx-carve"]);
});
it("does not re-analyse on every pixel of a drag", async () => {
// Only the release re-applies. Analysing per pointermove would decode both
// tracks on each pixel.
stubDecode();
const carvedAlready = JSON.stringify({
version: 1,
nodes: [
{
type: "peaking",
id: "n1",
fromCarve: true,
params: { frequency: 1000, gain: -6, q: 1.4 },
},
],
});
const { host, onSetAttributeQuiet, onSetAttributeLive } = mount({
"fx-chain": carvedAlready,
"fx-carve": settings(true, { strength: 0.25 }),
start: "0",
});
document.getElementById("vo")!.setAttribute("src", "voice.wav");
const dial = host.querySelector<HTMLInputElement>(".hf-fx-carve input[type=range]")!;
await act(async () => {
for (const v of ["0.4", "0.6", "0.8"]) {
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set?.call(dial, v);
dial.dispatchEvent(new Event("input", { bubbles: true }));
}
});
expect(onSetAttributeLive.mock.calls.every((c) => c[0] === "data-fx-carve")).toBe(true);
expect(onSetAttributeQuiet.mock.calls.some((c) => c[0] === "data-fx-chain")).toBe(false);
});
it("drops the envelopes when dynamic is switched back off", async () => {
// An automated gain ignores the panel's depth, so leaving the lanes behind
// would keep the filters following a voice with nothing saying they do.
const automation = JSON.stringify({
version: 1,
lanes: [
{ target: "fx.n2.gain", points: [{ t: 0, v: 0 }] },
{ target: "volume", points: [{ t: 0, v: 1 }] },
],
});
const withCarveNode = JSON.stringify({
version: 1,
nodes: [
{ type: "peaking", id: "n2", fromCarve: true, params: { frequency: 1000, gain: -6 } },
],
});
const { host, onSetAttributeQuiet } = mount({
"fx-chain": withCarveNode,
"fx-carve": settings(true),
automation,
});
await act(async () => {
dynamicBox(host).click();
});
const write = onSetAttributeQuiet.mock.calls.find((c) => c[0] === "data-automation");
expect(write).toBeTruthy();
const lanes = JSON.parse(String(write![1])).lanes;
expect(lanes.map((l: { target: string }) => l.target)).toEqual(["volume"]);
});
});
describe("AudioFxGroup successive edits", () => {
const three = JSON.stringify({
version: 1,
@@ -298,6 +660,72 @@ describe("AudioFxGroup successive edits", () => {
});
describe("AudioFxGroup carve visibility", () => {
/**
* A carve is a relationship: a bed is carved *against* a voice. The voice is the
* other end of it, so offering the same control there is offering to carve a
* track against itself by proxy — and switching it on left a setting that could
* never do anything.
*/
it("does not offer carve on a track another track is carving against", () => {
const bed = document.createElement("audio");
bed.id = "bed";
bed.setAttribute("src", "bed.m4a");
bed.setAttribute("data-fx-carve", JSON.stringify({ source: "vo", strength: 0.25 }));
document.body.append(bed);
const voice = document.createElement("audio");
voice.id = "vo";
voice.setAttribute("src", "voice.wav");
document.body.append(voice);
const host = document.createElement("div");
document.body.append(host);
const selection = {
dataAttributes: {},
id: "vo",
element: voice,
} as unknown as DomEditSelection;
act(() => {
createRoot(host).render(
<AudioFxGroup
element={selection}
onSetAttributeQuiet={vi.fn()}
onSetAttributeLive={vi.fn()}
/>,
);
});
expect(host.querySelector(".hf-fx-carve")).toBeNull();
});
it("still offers it on the bed doing the carving", () => {
const bed = document.createElement("audio");
bed.id = "bed";
bed.setAttribute("src", "bed.m4a");
bed.setAttribute("data-fx-carve", JSON.stringify({ source: "vo", strength: 0.25 }));
document.body.append(bed);
const voice = document.createElement("audio");
voice.id = "vo";
voice.setAttribute("src", "voice.wav");
document.body.append(voice);
const host = document.createElement("div");
document.body.append(host);
const selection = {
dataAttributes: { "fx-carve": JSON.stringify({ source: "vo", strength: 0.25 }) },
id: "bed",
element: bed,
} as unknown as DomEditSelection;
act(() => {
createRoot(host).render(
<AudioFxGroup
element={selection}
onSetAttributeQuiet={vi.fn()}
onSetAttributeLive={vi.fn()}
/>,
);
});
expect(host.querySelector(".hf-fx-carve")).not.toBeNull();
});
it("offers carve when the composition holds another audio track", () => {
const { host } = mount({ "fx-chain": CHAIN });
expect(host.querySelector(".hf-fx-carve")).toBeTruthy();
@@ -356,3 +784,188 @@ describe("AudioFxGroup deleting an effect", () => {
expect(onSetAttributeQuiet.mock.calls.some((c) => c[0] === "data-automation")).toBe(false);
});
});
describe("AudioFxGroup carve module readouts", () => {
/**
* A carve on a bed running 0–10 s, with a lane that ramps its 400 Hz band from
* no cut to −6 dB across the first five seconds. The stored gain is −1, which is
* deliberately not a value the lane ever passes through: it stands in for the
* seed a lane leaves behind, so a readout showing it can only mean the playhead
* was not consulted.
*/
const carved = {
start: "0",
duration: "10",
"fx-chain": JSON.stringify({
version: 1,
nodes: [
{
type: "peaking",
id: "n1",
fromCarve: true,
params: { frequency: 400, gain: -1, q: 1.4 },
},
],
}),
automation: JSON.stringify({
version: 1,
lanes: [
{
target: "fx.n1.gain",
points: [
{ t: 0, v: 0 },
{ t: 5, v: -6 },
],
},
],
}),
"fx-carve": JSON.stringify({ source: "vo", strength: 0.25, dynamic: true }),
};
/** Park the playhead somewhere, paused — a scrub is the same question as playback. */
const seek = (time: number) => {
act(() => {
usePlayerStore.setState({ currentTime: time, isPlaying: false });
});
};
const gainReadout = (host: HTMLElement): HTMLElement | null => {
for (const span of Array.from(host.querySelectorAll<HTMLElement>(".hf-fx-carve-member span"))) {
if (span.textContent?.startsWith("Gain")) return span;
}
return null;
};
const openModule = (host: HTMLElement) => {
const head = host.querySelector<HTMLButtonElement>(".hf-fx-carve-module .hf-fx-node-name");
act(() => head?.click());
};
afterEach(() => {
usePlayerStore.setState({ currentTime: 0, isPlaying: false });
});
it("shows the envelope's value at the playhead, not the stored seed", () => {
seek(2.5);
const { host } = mount(carved);
openModule(host);
const gain = gainReadout(host);
// Halfway along a 0 → −6 dB ramp.
expect(gain?.textContent).toContain("-3 dB");
expect(gain?.hasAttribute("data-automation-live")).toBe(true);
});
it("follows the playhead as it moves", () => {
seek(0);
const { host } = mount(carved);
openModule(host);
expect(gainReadout(host)?.textContent).toContain("0 dB");
seek(5);
expect(gainReadout(host)?.textContent).toContain("-6 dB");
seek(1);
expect(gainReadout(host)?.textContent).toContain("-1.2 dB");
});
it("follows the transport during playback, off the live-time channel", async () => {
// The RAF loop deliberately keeps every frame out of the store — it notifies
// `liveTime` instead — so a panel that only watched the store would sit still
// for the whole take and then jump when playback stopped.
seek(0);
const { host } = mount(carved);
openModule(host);
act(() => {
usePlayerStore.setState({ isPlaying: true });
});
act(() => liveTime.notify(4));
// Throttled to 30 Hz rather than rendered per frame, so the readout lands on
// the next tick and not in this one.
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 60));
});
expect(gainReadout(host)?.textContent).toContain("-4.8 dB");
act(() => liveTime.notify(5));
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 60));
});
expect(gainReadout(host)?.textContent).toContain("-6 dB");
});
it("reserves the same width whatever the value reads", () => {
// The readout updates 30 times a second while the transport runs, and a value
// one character narrower shunts everything after it sideways. So the width comes
// from what the parameter CAN read, not from what it currently does.
seek(2.5);
const { host } = mount(carved);
openModule(host);
const widthAt = (): string | undefined =>
gainReadout(host)?.querySelector<HTMLElement>(".tabular-nums")?.style.minWidth;
const narrow = widthAt();
expect(narrow).toBe("8ch"); // -40..40 dB at one decimal: "-12.5 dB"
seek(5); // -6 dB — two characters shorter than -3 dB was
expect(gainReadout(host)?.textContent).toContain("-6 dB");
expect(widthAt()).toBe(narrow);
});
it("moves a hand-built effect's own slider and number, not just the carve rack", () => {
// Every automated value follows its lane, whatever put the effect there. This
// one is a delay the author added and automated by hand: its control is locked
// (the lane owns the value), so the fader has no drag to fight and can simply
// show the truth.
const { host } = mount({
start: "0",
duration: "10",
"fx-chain": JSON.stringify({
version: 1,
nodes: [{ type: "delay", id: "n1", params: { time: 250, feedback: 0.35, mix: 0.4 } }],
}),
automation: JSON.stringify({
version: 1,
lanes: [
{
target: "fx.n1.mix",
points: [
{ t: 0, v: 0 },
{ t: 4, v: 1 },
],
},
],
}),
});
const mixRow = rowFor(host, "Mix");
const number = mixRow?.querySelector<HTMLInputElement>(".hf-fx-number");
const slider = mixRow?.querySelector<HTMLInputElement>(".hf-fx-slider");
expect(number?.disabled).toBe(true); // the lane owns it
seek(1); // a quarter along a 0 → 1 ramp
expect(Number(number?.value)).toBeCloseTo(0.25, 2);
const quarter = Number(slider?.value);
seek(3);
expect(Number(number?.value)).toBeCloseTo(0.75, 2);
expect(Number(slider?.value)).toBeGreaterThan(quarter);
});
it("shows the lane's own edge value off the clip, not the stored seed", () => {
// Past the bed's end, where a lane holds its last value — which is what would
// play if the playhead came back. The stored -1 dB is a seed the lane replaced
// and nothing will ever use it, so putting it on screen only made the fader
// jump when the clip came under the playhead.
seek(20);
const { host } = mount(carved);
openModule(host);
const gain = gainReadout(host);
expect(gain?.textContent).toContain("-6 dB"); // the ramp's last point
expect(gain?.hasAttribute("data-automation-live")).toBe(true);
expect(gain?.hasAttribute("data-automated")).toBe(true);
});
it("holds the lane's first value before the clip starts", () => {
// Same rule at the other end: a lane opens on its first point, so that is what
// the fader should read while the playhead is still upstream of the clip.
seek(-5);
const { host } = mount(carved);
openModule(host);
expect(gainReadout(host)?.textContent).toContain("0 dB");
});
});
packages/studio/src/components/editor/propertyPanelAudioFxGroup.tsx
+217 -16
View File
@@ -9,20 +9,30 @@
import { useState } from "react";
import {
defaultAudioFxParams,
HF_AUDIO_FX_ATTR,
mintAudioFxNodeId,
parseAudioFxChain,
serializeAudioFxChain,
type HfAudioFxChain,
type HfAudioFxNode,
} from "@hyperframes/core/audio-fx";
import {
analyseCarveBands,
analyseCarveDuck,
analyseCarveDynamics,
carveBandsToChain,
carveProfile,
HF_AUDIO_CARVE_ATTR,
normalizeCarveSettings,
type HfCarveSettings,
} from "@hyperframes/core/audio-carve";
import { fxAutomationTarget, type HfAutomation } from "@hyperframes/core/audio-automation";
import {
fxAutomationTarget,
sampleAutomationLane,
type HfAutomation,
type HfAutomationLane,
} from "@hyperframes/core/audio-automation";
import {
automatedTargetsOf,
automationAttrValue,
@@ -33,6 +43,7 @@ import {
withSeededLane,
} from "./propertyPanelAutomation";
import type { DomEditSelection } from "./domEditingTypes";
import { useLivePlayheadTime } from "../../hooks/useLivePlayheadTime";
/**
* Rate the carve source is decoded at. Analysis is self-consistent because it
@@ -41,6 +52,22 @@ import type { DomEditSelection } from "./domEditingTypes";
const DECODE_SAMPLE_RATE = 48000;
import { FxSection, type AudioTrackOption } from "./propertyPanelFxSection.js";
/** Where a clip starts on the timeline, in seconds. */
function clipStart(value: string | null | undefined): number {
const n = Number(value);
return Number.isFinite(n) ? n : 0;
}
/** Lanes belonging to nodes the carve generated, which a re-run replaces. */
function withoutCarveLanes(automation: HfAutomation, chain: HfAudioFxChain): HfAutomation {
const prefixes = chain.nodes.filter((n) => n.fromCarve && n.id).map((n) => `fx.${n.id}.`);
if (prefixes.length === 0) return automation;
return {
version: automation.version,
lanes: automation.lanes.filter((lane) => !prefixes.some((p) => lane.target.startsWith(p))),
};
}
/**
* Bridges the FX panel to the element/attribute world. Chain and carve are
* serialised onto the element the way colour grading carries its config, so
@@ -81,6 +108,37 @@ export function AudioFxGroup({
const automation = readPanelAutomation(element.dataAttributes?.["automation"], chain);
const automatedTargets = automatedTargetsOf(automation);
/**
* What every automated knob is worth at the playhead, so the rack shows the
* value the audio is actually using rather than the one the attribute stores.
*
* An automated parameter has two values: the number sitting in the chain, which
* is only a seed once a lane exists, and the number the envelope is on right now.
* The second is the true one, and a rack that shows the first reads as broken
* during playback — the carve is visibly working and the readouts do not move.
*
* Sampled while paused as well, because the same argument applies to a scrub:
* the playhead is somewhere, and the envelope has a value there.
*
* Sampled off the clip too, which is not obvious. A lane holds its first value
* backwards and its last value forwards, so before the clip starts it already
* knows what it will open on — while the stored number is a seed the lane
* replaced and nothing will ever play. Showing that seed put a value on screen
* that the automation never uses, and made the fader jump the moment the clip
* came under the playhead.
*/
const playhead = useLivePlayheadTime();
const localTime = playhead - clipStart(element.dataAttributes?.["start"]);
const liveAutomationValues = ((): Map<string, number> => {
const values = new Map<string, number>();
for (const lane of automation.lanes) {
const range = resolveAutomationRange(lane.target, chain);
if (!range) continue;
values.set(lane.target, sampleAutomationLane(lane, localTime, range.scale));
}
return values;
})();
// Written through the live path on purpose. It persists to the source just
// like the refreshing one, but skips the preview reload — and a reload
// restarts every playing track, which is heard as the audio chopping. The
@@ -127,6 +185,18 @@ export function AudioFxGroup({
* commit, which does not exist yet.
*/
const setCarve = async (next: HfCarveSettings | null): Promise<void> => {
// Envelopes the carve wrote outlive it otherwise, and an automated gain
// ignores the panel's own depth — so switching dynamic off would leave the
// filters still following the voice with nothing saying they do.
if (!next || (carve?.dynamic && !next.dynamic)) {
const carriedOver = withoutCarveLanes(automation, chain);
if (carriedOver.lanes.length !== automation.lanes.length) {
await onSetAttributeQuiet(
HF_AUDIO_AUTOMATION_ATTR,
automationAttrValue(carriedOver) || null,
);
}
}
if (!next) {
const kept = chain.nodes.filter((n) => !n.fromCarve);
if (kept.length !== chain.nodes.length) {
@@ -137,6 +207,20 @@ export function AudioFxGroup({
}
}
await onSetAttributeQuiet(HF_AUDIO_CARVE_ATTR, next ? JSON.stringify(next) : null);
// Every setting here describes the filters, so changing one rebuilds them.
// There is no apply button: a carve naming a voice with no filters behind it
// is a setting nobody applied, and the panel already knows everything it needs
// to. Picking the voice is what starts it; strength and dynamic re-derive what
// is already there. A carve with no source yet has nothing to analyse.
const changed =
next &&
next.source &&
(!carve ||
next.source !== carve.source ||
next.strength !== carve.strength ||
next.dynamic !== carve.dynamic);
if (next && changed) await analyse(next);
};
/** Every lane belonging to a node that is going away. */
@@ -158,6 +242,32 @@ export function AudioFxGroup({
}
})();
/**
* Is some other track carving against this one?
*
* A carve is a relationship — a bed is carved against a voice — and the voice is
* the far end of it. Offering the same control there offers to carve a track
* against itself by proxy, and switching it on left a setting with no source it
* could legally name. Read off the other elements' own carve attributes, because
* that is where the relationship is recorded.
*/
const carvedAgainstBy = ((): string | null => {
const doc = element.element?.ownerDocument;
if (!doc || !element.id) return null;
for (const other of Array.from(doc.querySelectorAll<HTMLElement>(`[${HF_AUDIO_CARVE_ATTR}]`))) {
if (other.id === element.id) continue;
try {
const raw = other.getAttribute(HF_AUDIO_CARVE_ATTR);
if (raw && normalizeCarveSettings(JSON.parse(raw)).source === element.id) {
return other.id || "another track";
}
} catch {
// An unreadable carve on some other element says nothing about this one.
}
}
return null;
})();
const sourceOptions: AudioTrackOption[] = (() => {
const doc = element.element?.ownerDocument;
if (!doc) return [];
@@ -173,16 +283,14 @@ export function AudioFxGroup({
* on this one. The bands replace any previous carve output but leave
* hand-added effects alone, so re-analysing does not discard other work.
*/
const analyse = async (): Promise<void> => {
if (!carve?.source) return;
const analyse = async (active: HfCarveSettings | null = carve): Promise<void> => {
if (!active?.source) return;
const doc = element.element?.ownerDocument;
const voice = doc?.getElementById(carve.source) as HTMLAudioElement | null;
const voice = doc?.getElementById(active.source) as HTMLAudioElement | null;
const src = voice?.getAttribute("src");
if (!src) return;
setAnalysing(true);
try {
const res = await fetch(new URL(src, doc!.baseURI).href);
const bytes = await res.arrayBuffer();
// Decoded in an OfflineAudioContext, not a live one. Opening a second
// output device mid-playback makes the running track glitch while the
// hardware is reconfigured; an offline context touches no device.
@@ -191,24 +299,116 @@ export function AudioFxGroup({
(window as unknown as { webkitOfflineAudioContext?: typeof OfflineAudioContext })
.webkitOfflineAudioContext;
if (!Ctor) return;
const decoder = new Ctor(1, 1, DECODE_SAMPLE_RATE);
const buffer = await decoder.decodeAudioData(bytes);
const bands = analyseCarveBands(
buffer.getChannelData(0),
buffer.sampleRate,
carveProfile(carve.strength),
);
const decode = async (relative: string): Promise<AudioBuffer> => {
const res = await fetch(new URL(relative, doc!.baseURI).href);
return new Ctor(1, 1, DECODE_SAMPLE_RATE).decodeAudioData(await res.arrayBuffer());
};
const buffer = await decode(src);
// Strength is what the author set; these are the numbers it means.
const profile = carveProfile(active.strength);
// The bed as well as the voice, when the carve is asked to match levels:
// "how far over the voice is this bed" cannot be answered by listening to
// one of them.
const bedSrc = profile.duckDb > 0 ? element.element?.getAttribute("src") : null;
const bedBuffer = bedSrc ? await decode(bedSrc).catch(() => null) : null;
const bands = analyseCarveBands(buffer.getChannelData(0), buffer.sampleRate, profile);
const carved = carveBandsToChain(bands);
// The level half of the carve, measured against the voice it has to sit
// under. Times come back relative to the voice clip; the gap between the
// two clips' starts is what aligns them.
const offset =
clipStart(voice?.getAttribute("data-start")) - clipStart(element.dataAttributes?.["start"]);
const duck = bedBuffer
? analyseCarveDuck(
buffer.getChannelData(0),
bedBuffer.getChannelData(0),
buffer.sampleRate,
profile,
offset,
)
: [];
// Static carve holds one value, so the level match becomes the duck the
// voice needs while it is actually speaking — the median of it, which
// ignores both the pauses and any single loudest bar.
const speaking = duck.filter((p) => p.v < 0).map((p) => p.v);
const staticDuckDb = speaking.length
? (speaking.sort((a, b) => a - b)[Math.floor(speaking.length / 2)] ?? 0)
: 0;
// Carve output is tagged so a re-run replaces it instead of stacking.
const kept = chain.nodes.filter((n) => !n.fromCarve);
// Ids, minted against the nodes already claiming one, because a dynamic
// carve automates these filters and a lane addresses its node by id.
let claimed: HfAudioFxChain = { version: 1, nodes: kept };
const mint = (node: HfAudioFxNode): HfAudioFxNode => {
const withId = { ...node, id: mintAudioFxNodeId(claimed), fromCarve: true };
claimed = { version: 1, nodes: [...claimed.nodes, withId] };
return withId;
};
const carvedNodes: HfAudioFxNode[] = carved.nodes.map(mint);
// The gain stage sits after the filters, and only exists when the carve was
// asked to make level room. Dynamic drives it from the envelope; static
// holds the one value above.
const duckNode =
duck.length > 0
? mint({
type: "gain",
enabled: true,
params: { ...defaultAudioFxParams("gain"), gain: active.dynamic ? 0 : staticDuckDb },
})
: null;
const next = {
version: 1,
nodes: [...carved.nodes.map((n) => ({ ...n, fromCarve: true })), ...kept],
nodes: [...carvedNodes, ...(duckNode ? [duckNode] : []), ...kept],
};
// Live, like every other chain write: the runtime swaps the graph in
// place, so a reload would only interrupt the audio to reach the same
// filters.
onSetAttributeQuiet(HF_AUDIO_FX_ATTR, serializeAudioFxChain(next));
//
// Awaited, because the automation write below is a second read-modify-write
// against the same file — fired together the later one would drop the
// earlier — and because a lane naming a node the chain does not have yet is
// pruned when it is read back.
await onSetAttributeQuiet(HF_AUDIO_FX_ATTR, serializeAudioFxChain(next));
/**
* One carve envelope as a lane on this bed's clock.
*
* Everything the analysis returns is timed from the start of the voice, so
* it shifts by the gap between the two clips; and a lane holds its first
* value backwards to the start of its own clip, so a bed that begins before
* the voice needs an explicit "no cut" at zero or it starts out ducked.
*/
const laneFor = (id: string, points: { t: number; v: number }[]): HfAutomationLane[] => {
const shifted = points
.map((p) => ({ t: Number((p.t + offset).toFixed(3)), v: p.v }))
.filter((p) => p.t >= 0);
if ((shifted[0]?.t ?? 0) > 0) shifted.unshift({ t: 0, v: 0 });
return shifted.length > 1
? [{ target: fxAutomationTarget(id, "gain"), points: shifted }]
: [];
};
// Dynamic carve: each filter's depth becomes an envelope of the voice's
// level in that band, so pauses leave the bed alone.
const lanes: HfAutomationLane[] = active.dynamic
? analyseCarveDynamics(buffer.getChannelData(0), buffer.sampleRate, bands).flatMap(
(dyn, i) => {
const id = carvedNodes[i]?.id;
if (!id) return [];
return laneFor(id, dyn.points);
},
)
: [];
// The level envelope rides the gain stage, on the same clock as the bands.
if (active.dynamic && duckNode?.id && duck.length > 0) {
lanes.push(...laneFor(duckNode.id, duck));
}
const carriedOver = withoutCarveLanes(automation, chain);
if (lanes.length > 0 || carriedOver.lanes.length !== automation.lanes.length) {
writeAutomation({ version: 1, lanes: [...carriedOver.lanes, ...lanes] });
}
} catch {
// Leave the chain as it was; the button simply re-enables.
} finally {
@@ -220,6 +420,7 @@ export function AudioFxGroup({
<FxSection
chain={chain}
automatedTargets={automatedTargets}
liveAutomationValues={liveAutomationValues}
onAutomateParam={automateParam}
onRemoveParamAutomation={removeParamAutomation}
onRemoveNodeAutomation={removeNodeAutomation}
@@ -242,7 +443,7 @@ export function AudioFxGroup({
onCarveChange={(next) => void setCarve(next)}
onCarvePreview={(next) => onSetAttributeLive(HF_AUDIO_CARVE_ATTR, JSON.stringify(next))}
sourceOptions={sourceOptions}
onAnalyseCarve={() => void analyse()}
carvedAgainstBy={carvedAgainstBy}
analysing={analysing}
/>
);
packages/studio/src/components/editor/propertyPanelFxControls.tsx
+16 -1
View File
@@ -42,6 +42,12 @@ function display(p: HfAudioFxNumberParam, value: number): string {
interface ParamRowProps {
param: HfAudioFxParam;
value: number | string;
/**
* The envelope's value at the playhead, when a lane drives this parameter and
* the playhead is over the clip. Shown in place of `value`, which by then is
* only the seed the lane replaced.
*/
liveValue?: number;
/** Fires continuously while dragging — cheap, not persisted. */
onChange(key: string, value: number | string): void;
/** Fires once when the gesture ends — this is the write that persists. */
@@ -100,6 +106,7 @@ export function AutomationToggle({
export function FxParamRow({
param,
value,
liveValue,
onChange,
onCommit,
disabled,
@@ -159,7 +166,11 @@ export function FxParamRow({
);
}
const shown = dragging ? local : value;
// An envelope's value at the playhead outranks the one stored in the chain,
// because it is the one the audio is using — the stored number is only the seed
// the lane replaced. Safe against the pointer: an automated control is locked
// (see `locked` below), so there is no drag for this to fight.
const shown = dragging ? local : (liveValue ?? value);
const numeric = typeof shown === "number" ? shown : Number(shown);
const current = Number.isFinite(numeric) ? numeric : param.default;
@@ -229,6 +240,8 @@ export function FxParamRow({
interface FxParamsProps {
def: HfAudioFxDef;
params: HfAudioFxParamValues;
/** What automated knobs are worth at the playhead, by parameter key. */
liveValues?: ReadonlyMap<string, number>;
onChange(params: HfAudioFxParamValues): void;
onCommit?(params: HfAudioFxParamValues): void;
disabled?: boolean;
@@ -243,6 +256,7 @@ interface FxParamsProps {
export function FxParams({
def,
params,
liveValues,
onChange,
onCommit,
disabled,
@@ -270,6 +284,7 @@ export function FxParams({
key={p.key}
param={p}
value={params[p.key] ?? p.default}
liveValue={liveValues?.get(p.key)}
onChange={set}
onCommit={commit}
disabled={disabled}
packages/studio/src/components/editor/propertyPanelFxSection.test.tsx
+126 -11
View File
@@ -28,8 +28,6 @@ const chainOf = (...types: string[]): HfAudioFxChain => ({
nodes: types.map((t) => ({ type: t, enabled: true, params: defaultAudioFxParams(t) })),
});
const noop = () => {};
function mount(overrides: Partial<Parameters<typeof FxSection>[0]> = {}) {
const onChainChange = vi.fn();
const onChainPreview = vi.fn();
@@ -42,7 +40,6 @@ function mount(overrides: Partial<Parameters<typeof FxSection>[0]> = {}) {
carve={overrides.carve ?? null}
onCarveChange={overrides.onCarveChange ?? onCarveChange}
sourceOptions={overrides.sourceOptions ?? [{ id: "vo", label: "Voiceover" }]}
onAnalyseCarve={overrides.onAnalyseCarve ?? noop}
analysing={overrides.analysing}
disabled={overrides.disabled}
automatedTargets={overrides.automatedTargets}
@@ -200,6 +197,123 @@ describe("FxSection chain", () => {
});
});
describe("FxSection carve module", () => {
/**
* A carve is one thing an author turned on, not the six filters it happens to
* compile to. Listed individually they read as hand-built effects: removable
* one at a time, reorderable, each with knobs that the next strength change
* overwrites without warning.
*/
const carved = {
version: 1,
nodes: [
{ type: "peaking", id: "n1", fromCarve: true, params: { frequency: 400, gain: -6, q: 1.4 } },
{ type: "peaking", id: "n2", fromCarve: true, params: { frequency: 1600, gain: -9, q: 1.4 } },
{ type: "gain", id: "n3", fromCarve: true, params: { gain: -6 } },
{ type: "lowpass", id: "n4", params: { frequency: 8000, q: 0.7, poles: "2" } },
],
} as unknown as HfAudioFxChain;
it("shows the carve's effects as one module, alongside hand-built ones", () => {
const { host } = mount({ chain: carved });
const rows = Array.from(host.querySelectorAll<HTMLElement>(".hf-fx-node"));
// One row for the carve, one for the low-pass the author added.
expect(rows).toHaveLength(2);
expect(host.querySelector(".hf-fx-carve-module")).not.toBeNull();
expect(host.querySelector(".hf-fx-carve-module")?.textContent).toContain("Voiceover carve");
});
it("says what the module contains, since its parts are not listed", () => {
const { host } = mount({ chain: carved });
const text = host.querySelector(".hf-fx-carve-module")?.textContent ?? "";
expect(text).toMatch(/2 bands/);
expect(text).toMatch(/level/);
});
it("removes every carve effect together, never one of them", () => {
const onChainChange = vi.fn();
const { host } = mount({ chain: carved, onChainChange });
const removes = Array.from(
host.querySelectorAll<HTMLButtonElement>(".hf-fx-carve-module .hf-fx-remove"),
);
expect(removes).toHaveLength(1);
act(() => removes[0]!.click());
const nodes = onChainChange.mock.calls[0]![0].nodes as { id: string }[];
expect(nodes.map((n) => n.id)).toEqual(["n4"]);
});
it("bypasses the whole module at once", () => {
const onChainChange = vi.fn();
const { host } = mount({ chain: carved, onChainChange });
const bypass = host.querySelector<HTMLButtonElement>(".hf-fx-carve-module .hf-fx-bypass")!;
act(() => bypass.click());
const nodes = onChainChange.mock.calls[0]![0].nodes as { id: string; enabled?: boolean }[];
expect(nodes.filter((n) => n.id !== "n4").every((n) => n.enabled === false)).toBe(true);
// The author's own effect is not touched.
expect(nodes.find((n) => n.id === "n4")?.enabled).not.toBe(false);
});
it("lists what each effect inside it is set to", () => {
// Grouped is not hidden: the carve compiles to real filters and an author has
// to be able to see where they landed. What they cannot do is edit them by
// hand — strength owns those numbers — so the settings read out rather than
// offering controls that the next adjustment would overwrite.
const { host } = mount({ chain: carved });
const module = host.querySelector<HTMLElement>(".hf-fx-carve-module")!;
act(() => module.querySelector<HTMLButtonElement>(".hf-fx-node-name")!.click());
const members = Array.from(module.querySelectorAll<HTMLElement>(".hf-fx-carve-member"));
expect(members).toHaveLength(3);
// Named by what tells them apart, the way the timeline lanes name them.
expect(members.map((m) => m.querySelector(".hf-fx-carve-member-name")?.textContent)).toEqual([
"Peaking EQ 400 Hz",
"Peaking EQ 1.6 kHz",
"Gain",
]);
// And every one of its settings is visible.
const first = members[0]!.textContent ?? "";
expect(first).toContain("Gain");
expect(first).toMatch(/-6(\.0)? dB/);
expect(first).toMatch(/1\.4/); // Q
});
it("reads its settings out rather than offering controls", () => {
const { host } = mount({ chain: carved });
const module = host.querySelector<HTMLElement>(".hf-fx-carve-module")!;
act(() => module.querySelector<HTMLButtonElement>(".hf-fx-node-name")!.click());
expect(module.querySelectorAll("input")).toHaveLength(0);
});
it("says which of them the timeline is driving", () => {
// A carve in dynamic mode automates every one of these, and that is where the
// values come from — so the module has to point at the lane rather than look
// like a static setting.
const { host } = mount({
chain: carved,
automatedTargets: new Set(["fx.n1.gain", "fx.n3.gain"]),
});
const module = host.querySelector<HTMLElement>(".hf-fx-carve-module")!;
act(() => module.querySelector<HTMLButtonElement>(".hf-fx-node-name")!.click());
const automated = Array.from(module.querySelectorAll("[data-automated]"));
expect(automated).toHaveLength(2);
});
it("keeps the summary readable while collapsed", () => {
const { host } = mount({ chain: carved });
const module = host.querySelector<HTMLElement>(".hf-fx-carve-module")!;
expect(module.querySelectorAll(".hf-fx-carve-member")).toHaveLength(0);
expect(module.textContent).toContain("2 bands + level");
});
it("offers no per-effect controls inside the module", () => {
// Reordering or editing one band is meaningless: the next strength change
// rewrites every one of them.
const { host } = mount({ chain: carved });
const module = host.querySelector(".hf-fx-carve-module")!;
expect(module.querySelectorAll(".hf-fx-move")).toHaveLength(0);
expect(module.querySelectorAll("input[type=range]")).toHaveLength(0);
});
});
describe("FxSection carve", () => {
it("is off by default and is not an entry in the chain", () => {
const { host } = mount();
@@ -229,16 +343,17 @@ describe("FxSection carve", () => {
expect(options).toContain("Narration");
});
it("will not analyse until a source is chosen", () => {
const { host } = mount({ carve: { ...DEFAULT_CARVE, source: "" } });
expect(host.querySelector<HTMLButtonElement>(".hf-fx-analyse")!.disabled).toBe(true);
it("offers no analyse button — picking a voice is the whole gesture", () => {
// A carve with a source and no filters is a setting nobody applied; the
// button was a second step for something the panel already knew to do.
const { host } = mount({ carve: { ...DEFAULT_CARVE, source: "vo" } });
expect(host.querySelector(".hf-fx-analyse")).toBeNull();
expect(host.textContent).not.toMatch(/Analyse/i);
});
it("analyses once a source is chosen", () => {
const onAnalyseCarve = vi.fn();
const { host } = mount({ carve: { ...DEFAULT_CARVE, source: "vo" }, onAnalyseCarve });
click(host.querySelector(".hf-fx-analyse"));
expect(onAnalyseCarve).toHaveBeenCalledTimes(1);
it("says when it is working, since there is no button to grey out", () => {
const { host } = mount({ carve: { ...DEFAULT_CARVE, source: "vo" }, analysing: true });
expect(host.querySelector(".hf-fx-carve-working")?.textContent).toMatch(/Analysing/i);
});
it("disables everything when the panel is read-only", () => {
packages/studio/src/components/editor/propertyPanelFxSection.tsx
+331 -32
View File
@@ -18,11 +18,15 @@ import {
type HfAudioFxDef,
type HfAudioFxGroup,
type HfAudioFxNode,
type HfAudioFxParam,
type HfAudioFxParamValues,
} from "@hyperframes/core/audio-fx";
import { DEFAULT_CARVE, type HfCarveSettings } from "@hyperframes/core/audio-carve";
import { fxAutomationTarget } from "@hyperframes/core/audio-automation";
import { FxParams, FxParamRow } from "./propertyPanelFxControls.js";
// Shared with the timeline's lane labels: a band is named by its frequency in
// both places, and two formatters would drift.
import { formatHz } from "../../player/components/automationLaneData";
const GROUP_ORDER: HfAudioFxGroup[] = ["filter", "dynamics", "nonlinear", "time"];
const GROUP_LABEL: Record<HfAudioFxGroup, string> = {
@@ -41,6 +45,7 @@ interface FxNodeRowProps {
node: HfAudioFxNode;
index: number;
automatedTargets?: ReadonlySet<string>;
liveAutomationValues?: ReadonlyMap<string, number>;
onAutomateParam?(nodeId: string, paramKey: string): void;
onRemoveParamAutomation?(nodeId: string, paramKey: string): void;
open: boolean;
@@ -148,6 +153,205 @@ function FxNodeHeader({
);
}
/**
* The carve's own effects, as one module.
*
* A carve is one thing the author switched on; the peaking filters and the level
* stage are how it is built. Listed individually they read as hand-built effects
* — removable one at a time, reorderable, each with knobs the next strength
* change silently overwrites. So the rack shows the unit, says what is inside it,
* and offers the two actions that mean anything for a whole module: bypass it,
* or remove it.
*/
/** What one effect inside the module is called: its own name, plus the band. */
function carveMemberName(node: HfAudioFxNode): string {
const def = getAudioFxDef(node.type);
const freq = node.params?.["frequency"];
const label = def?.label ?? node.type;
return typeof freq === "number" ? `${label} ${formatHz(freq)}` : label;
}
/** A parameter's value as the rack shows it: rounded to the step, with its unit. */
function formatParamValue(param: HfAudioFxParam, raw: number | string | undefined): string {
if (param.kind !== "number" || typeof raw !== "number") return String(raw ?? "");
const places = param.step >= 1 ? 0 : param.step >= 0.1 ? 1 : 2;
return `${Number(raw.toFixed(places))}${param.unit ? ` ${param.unit}` : ""}`;
}
/**
* Width to reserve for a parameter's value, in characters.
*
* Derived from what the parameter CAN read rather than what it currently reads, so
* the column never moves: an automated value updates 30 times a second, and
* `-1 dB` is two characters narrower than `-3.2 dB`, which was enough to shunt
* everything after it sideways on every frame. `ch` is exact here because the
* readouts are monospace and already `tabular-nums`.
*/
function paramValueWidthCh(param: HfAudioFxParam): number {
if (param.kind === "enum") {
return Math.max(1, ...param.options.map((option) => option.value.length));
}
const places = param.step >= 1 ? 0 : param.step >= 0.1 ? 1 : 2;
const digits = Math.max(
String(Math.floor(Math.abs(param.min))).length,
String(Math.floor(Math.abs(param.max))).length,
);
const sign = param.min < 0 ? 1 : 0;
const decimals = places > 0 ? places + 1 : 0;
const unit = param.unit ? param.unit.length + 1 : 0;
return sign + digits + decimals + unit;
}
/** One member of the module: what it is, and what every knob is set to. */
function FxCarveMember({
node,
automatedTargets,
liveAutomationValues,
}: {
node: HfAudioFxNode;
automatedTargets?: ReadonlySet<string>;
liveAutomationValues?: ReadonlyMap<string, number>;
}) {
const def = getAudioFxDef(node.type);
if (!def) return null;
const params = node.params ?? defaultAudioFxParams(node.type);
return (
<div className="hf-fx-carve-member flex flex-col gap-0.5 py-1 pl-3 pr-1.5">
<span className="hf-fx-carve-member-name truncate font-mono text-[9px] text-panel-text-1">
{carveMemberName(node)}
</span>
<div className="flex flex-wrap gap-x-3 gap-y-0.5">
{def.params.map((param) => {
const target = node.id ? fxAutomationTarget(node.id, param.key) : null;
const automated = Boolean(target && automatedTargets?.has(target));
// The envelope's value at the playhead when there is one, which is what
// the audio is using; the stored number is only the seed behind it.
const live = target ? liveAutomationValues?.get(target) : undefined;
const driven = automated && live !== undefined;
const value = formatParamValue(param, driven ? live : params[param.key]);
return (
<span
key={param.key}
className="flex items-baseline gap-1 font-mono text-[9px] text-panel-text-4"
{...(automated ? { "data-automated": "" } : {})}
{...(driven ? { "data-automation-live": "" } : {})}
>
<span className="text-panel-text-4">{param.label}</span>
<span
className="tabular-nums text-panel-text-1"
style={{ minWidth: `${paramValueWidthCh(param)}ch` }}
>
{value}
</span>
{/* The lane is where an automated value comes from, and where it is
edited — saying so is the difference between a stale readout and
a pointer to the thing that owns it. */}
{automated ? <span className="text-[#3CE6AC]">A</span> : null}
</span>
);
})}
</div>
</div>
);
}
/**
* The carve's own effects, as one module.
*
* A carve is one thing the author switched on; the peaking filters and the level
* stage are how it is built. Listed individually in the rack they read as
* hand-built effects — removable one at a time, reorderable, each with knobs the
* next strength change silently overwrites. So the rack shows the unit, and the
* unit owns the actions that mean anything for a whole module: bypass, remove.
*
* Grouped is not hidden. Opening it lists every effect inside with all of its
* settings, because an author has to be able to see where the analysis landed —
* as readouts rather than controls, since strength is what sets them and a knob
* here would be overwritten by the next adjustment. A value the timeline drives
* says so, and points at the lane that owns it.
*/
function FxCarveModule({
nodes,
automatedTargets,
liveAutomationValues,
open,
disabled,
onToggleOpen,
onToggleBypass,
onRemove,
}: {
nodes: HfAudioFxNode[];
automatedTargets?: ReadonlySet<string>;
liveAutomationValues?: ReadonlyMap<string, number>;
open: boolean;
disabled?: boolean;
onToggleOpen(): void;
onToggleBypass(): void;
onRemove(): void;
}) {
const bands = nodes.filter((n) => n.type === "peaking").length;
const hasLevel = nodes.some((n) => n.type === "gain");
const bypassed = nodes.every((n) => n.enabled === false);
const summary = [`${bands} band${bands === 1 ? "" : "s"}`, ...(hasLevel ? ["level"] : [])].join(
" + ",
);
return (
<div
className={`hf-fx-node hf-fx-carve-module rounded-[4px] border border-panel-border-input${
bypassed ? " opacity-50" : ""
}`}
data-fx-node="carve"
>
<div className="hf-fx-node-head flex min-h-7 items-center gap-1 px-1.5">
<button
type="button"
className="hf-fx-node-name min-w-0 flex-1 truncate text-left text-[11px] font-semibold text-panel-text-1 hover:text-panel-text-0"
aria-expanded={open}
onClick={onToggleOpen}
>
Voiceover carve
</button>
<span className="hf-fx-carve-summary shrink-0 font-mono text-[9px] text-panel-text-4">
{summary}
</span>
<button
type="button"
className="hf-fx-bypass rounded-[3px] border border-panel-border-input px-1.5 py-0.5 font-mono text-[9px] text-panel-text-4 hover:text-panel-text-0 disabled:opacity-40"
aria-pressed={bypassed}
title={bypassed ? "Enable carve" : "Bypass carve"}
disabled={disabled}
onClick={onToggleBypass}
>
{bypassed ? "Off" : "On"}
</button>
<button
type="button"
className="hf-fx-remove px-1 font-mono text-[11px] text-panel-text-4 hover:text-red-400 disabled:opacity-40"
title="Remove carve"
disabled={disabled}
onClick={onRemove}
>
&times;
</button>
</div>
{open ? (
// Divided rows rather than boxes: these are parts of one module, and a
// border around each would read as the separate effects this replaced.
<div className="hf-fx-carve-members divide-y divide-panel-border-input/60 border-t border-panel-border-input">
{nodes.map((node, i) => (
<FxCarveMember
key={node.id ?? `${node.type}-${i}`}
node={node}
automatedTargets={automatedTargets}
liveAutomationValues={liveAutomationValues}
/>
))}
</div>
) : null}
</div>
);
}
/**
* Which of an effect's knobs already have a lane.
*
@@ -174,6 +378,7 @@ function FxNodeParams({
index,
disabled,
automatedTargets,
liveAutomationValues,
onUpdate,
onPreview,
onAutomateParam,
@@ -184,16 +389,29 @@ function FxNodeParams({
index: number;
disabled: boolean;
automatedTargets?: ReadonlySet<string>;
liveAutomationValues?: ReadonlyMap<string, number>;
onUpdate(index: number, patch: Partial<HfAudioFxNode>): void;
onPreview(index: number, params: HfAudioFxParamValues): void;
onAutomateParam?(nodeId: string, paramKey: string): void;
onRemoveParamAutomation?(nodeId: string, paramKey: string): void;
}) {
const nodeId = node.id;
// Lanes address a node by id; the controls know their own parameter keys. This
// is the one place that translation belongs.
const liveValues = ((): Map<string, number> | undefined => {
if (!nodeId || !liveAutomationValues?.size) return undefined;
const byKey = new Map<string, number>();
for (const param of def.params) {
const live = liveAutomationValues.get(fxAutomationTarget(nodeId, param.key));
if (live !== undefined) byKey.set(param.key, live);
}
return byKey;
})();
return (
<FxParams
def={def}
params={node.params ?? defaultAudioFxParams(node.type)}
liveValues={liveValues}
disabled={disabled}
onChange={(params: HfAudioFxParamValues) => onPreview(index, params)}
onCommit={(params: HfAudioFxParamValues) => onUpdate(index, { params })}
@@ -213,6 +431,7 @@ function FxNodeRow({
node,
index,
automatedTargets,
liveAutomationValues,
onAutomateParam,
onRemoveParamAutomation,
open,
@@ -251,6 +470,7 @@ function FxNodeRow({
index={index}
disabled={Boolean(disabled) || bypassed}
automatedTargets={automatedTargets}
liveAutomationValues={liveAutomationValues}
onUpdate={onUpdate}
onPreview={onPreview}
onAutomateParam={onAutomateParam}
@@ -265,6 +485,15 @@ export interface FxSectionProps {
chain: HfAudioFxChain;
/** Targets this track already automates, as `fx.<nodeId>.<param>` strings. */
automatedTargets?: ReadonlySet<string>;
/**
* What each automated target is worth at the playhead, by the same key.
*
* An automated parameter's stored number is only the seed the lane replaced, so
* a rack that shows it stands still while the carve is audibly working. Absent,
* or missing a key, means there is no playhead over this clip and the stored
* value is the honest one.
*/
liveAutomationValues?: ReadonlyMap<string, number>;
/** Add a lane for one effect parameter, seeded at its current value. */
onAutomateParam?(nodeId: string, paramKey: string): void;
/** Delete one effect parameter's lane. */
@@ -281,10 +510,13 @@ export interface FxSectionProps {
/** Continuous updates while a carve slider is dragged. Without this every
* pointermove patched the source file and resynced the selection. */
onCarvePreview?(carve: HfCarveSettings): void;
/**
* Set when another track's carve listens to this one, naming it. The carve block
* is then not offered here at all: this track is the voice, not the bed.
*/
carvedAgainstBy?: string | null;
/** Other audio elements that could act as the carve source. */
sourceOptions: AudioTrackOption[];
/** Re-run analysis against the current source audio. */
onAnalyseCarve?(): void;
analysing?: boolean;
disabled?: boolean;
}
@@ -292,16 +524,17 @@ export interface FxSectionProps {
export function FxSection({
chain,
automatedTargets,
liveAutomationValues,
onAutomateParam,
onRemoveParamAutomation,
onRemoveNodeAutomation,
onChainChange,
onChainPreview,
carve,
carvedAgainstBy,
onCarveChange,
onCarvePreview,
sourceOptions,
onAnalyseCarve,
analysing,
disabled,
}: FxSectionProps) {
@@ -310,7 +543,10 @@ export function FxSection({
const previewCarve = onCarvePreview ?? onCarveChange;
// Nothing to carve against means nothing to show — see the block below.
const showCarve = sourceOptions.length > 0 || carve !== null;
// Not offered on the voice another track is already carving against — that
// track is the far end of someone else's relationship, and a carve of its own
// could only name a source it must not.
const showCarve = !carvedAgainstBy && (sourceOptions.length > 0 || carve !== null);
const [adding, setAdding] = useState(false);
const [openNode, setOpenNode] = useState<number | null>(0);
@@ -368,6 +604,24 @@ export function FxSection({
[chain.nodes, mutate, onRemoveNodeAutomation],
);
const [carveOpen, setCarveOpen] = useState(false);
const carveNodes = useMemo(() => chain.nodes.filter((n) => n.fromCarve), [chain.nodes]);
/** Remove the carve's effects together, with the envelopes they carried. */
const removeCarve = useCallback(() => {
for (const node of carveNodes) {
if (node.id) onRemoveNodeAutomation?.(node.id);
}
mutate(chain.nodes.filter((n) => !n.fromCarve));
setOpenNode(null);
}, [carveNodes, chain.nodes, mutate, onRemoveNodeAutomation]);
/** Bypass or enable every carve effect at once — the module is the unit. */
const toggleCarveBypass = useCallback(() => {
const bypassed = carveNodes.every((n) => n.enabled === false);
mutate(chain.nodes.map((n) => (n.fromCarve ? { ...n, enabled: bypassed } : n)));
}, [carveNodes, chain.nodes, mutate]);
const moveNode = useCallback(
(index: number, delta: number) => {
const target = index + delta;
@@ -389,24 +643,46 @@ export function FxSection({
No effects on this track.
</p>
) : (
chain.nodes.map((node, i) => (
<FxNodeRow
key={`${node.type}-${i}`}
node={node}
index={i}
automatedTargets={automatedTargets}
onAutomateParam={onAutomateParam}
onRemoveParamAutomation={onRemoveParamAutomation}
open={openNode === i}
last={i === chain.nodes.length - 1}
disabled={disabled}
onToggleOpen={() => setOpenNode(openNode === i ? null : i)}
onUpdate={updateNode}
onMove={moveNode}
onRemove={removeNode}
onPreview={previewNode}
/>
))
chain.nodes.map((node, i) => {
if (node.fromCarve) {
// The module stands in for the whole run of carve nodes, drawn once
// at the first of them.
const first = chain.nodes.findIndex((n) => n.fromCarve);
if (i !== first) return null;
return (
<FxCarveModule
key="carve-module"
nodes={carveNodes}
automatedTargets={automatedTargets}
liveAutomationValues={liveAutomationValues}
open={carveOpen}
disabled={disabled}
onToggleOpen={() => setCarveOpen((was) => !was)}
onToggleBypass={toggleCarveBypass}
onRemove={removeCarve}
/>
);
}
return (
<FxNodeRow
key={`${node.type}-${i}`}
node={node}
index={i}
automatedTargets={automatedTargets}
liveAutomationValues={liveAutomationValues}
onAutomateParam={onAutomateParam}
onRemoveParamAutomation={onRemoveParamAutomation}
open={openNode === i}
last={i === chain.nodes.length - 1}
disabled={disabled}
onToggleOpen={() => setOpenNode(openNode === i ? null : i)}
onUpdate={updateNode}
onMove={moveNode}
onRemove={removeNode}
onPreview={previewNode}
/>
);
})
)}
</div>
@@ -484,6 +760,12 @@ export function FxSection({
))}
</select>
</label>
{/* One knob for the whole effect. Depth, band count, width, the
intelligibility weighting and both level-match numbers move
together anyway — a gentle carve is shallow in few bands with
little ducking, a hard one is deeper in more with more — so the
panel sets the strength and `carveProfile` derives the six
numbers the analysis works in. */}
<FxParamRow
param={{
kind: "number",
@@ -492,23 +774,40 @@ export function FxSection({
unit: "",
min: 0,
max: 1,
step: 0.01,
step: 0.05,
default: DEFAULT_CARVE.strength,
hint: "How hard to carve. The six numbers the analysis actually uses — depth, band count, width and the rest — all move together from this one control.",
hint: "How hard to carve: deeper cuts, in more bands, and more room made by dropping the bed's level under the voice. At 0 it carves frequencies only. Moving this re-applies an existing carve; the button is for the first one, or after changing the voice track.",
}}
value={carve.strength}
disabled={disabled}
onChange={(_k, v) => previewCarve({ ...carve, strength: Number(v) })}
onCommit={(_k, v) => onCarveChange({ ...carve, strength: Number(v) })}
/>
<button
type="button"
className="hf-fx-analyse mt-1 w-full rounded-[3px] bg-panel-surface py-1 text-[10px] text-panel-text-1 hover:text-panel-text-0 disabled:opacity-40"
disabled={disabled || analysing || !carve.source || !onAnalyseCarve}
onClick={() => onAnalyseCarve?.()}
>
{analysing ? "Analysing…" : "Analyse and apply"}
</button>
{/* A static carve holds its cuts for the whole clip, pauses
included. Dynamic hands every value to an envelope of the voice's
own level, so the bed is only worked on while there is something
to make room for. Written as ordinary automation, which is why the
lanes show up in the timeline and can be edited afterwards. */}
<label className="hf-fx-row flex min-h-6 items-center gap-2">
<span className="hf-fx-label w-[86px] flex-shrink-0 truncate text-[10px] text-panel-text-4">
Dynamic
</span>
<input
type="checkbox"
className="hf-fx-carve-dynamic h-3 w-3 accent-panel-accent"
checked={carve.dynamic}
disabled={disabled}
onChange={(e) => onCarveChange({ ...carve, dynamic: e.target.checked })}
/>
<span className="min-w-0 flex-1 truncate text-[9px] text-panel-text-4">
follows the voice, flat where it is silent
</span>
</label>
{analysing ? (
<p className="hf-fx-carve-working py-1 text-center text-[10px] text-panel-text-4">
Analysing…
</p>
) : null}
</>
) : null}
</div>
packages/studio/src/hooks/useAutomationSelectionKeyboard.test.tsx
+82 -10
View File
@@ -20,8 +20,8 @@ import type { TimelineElement } from "../player/store/timelineElement";
* A selection box spanning the lane's whole value axis.
*
* What almost every test here is about is the time span — which breakpoints a
* Delete or a copy covers. Giving these an unbounded axis keeps the fixture
* out of the way of that.
* Delete or a copy covers. The box's value bounds have their own tests; giving
* these an unbounded axis keeps them testing the one thing they name.
*/
function wholeAxis<T extends { t0: number; t1: number }>(sel: T): T & { v0: number; v1: number } {
return { ...sel, v0: Number.NEGATIVE_INFINITY, v1: Number.POSITIVE_INFINITY };
@@ -117,7 +117,11 @@ describe("useAutomationSelectionKeyboard", () => {
return { onCommit };
};
it("Delete empties the selected range and pins anchors", () => {
it("Delete removes every breakpoint the selection covers", () => {
// Deleted, not emptied. Pinning anchors at the selection's edges keeps the
// envelope either side from moving, which is right for a shape insert or a
// paste — but answering "delete these points" with two NEW points at the edges
// reads as the delete not having worked.
usePlayerStore.setState({ elements: [bgmElement], selectedElementId: "bgm" });
usePlayerStore
.getState()
@@ -126,7 +130,69 @@ describe("useAutomationSelectionKeyboard", () => {
key("Delete");
const written = onCommit.mock.calls.at(-1)?.[0];
const points = written?.lanes?.[0]?.points ?? [];
expect(points.map((p: { t: number }) => p.t)).toEqual([0, 1, 3, 4]);
// The fixture lane is 0, 2, 4: only t=2 was inside.
expect(points.map((p: { t: number }) => p.t)).toEqual([0, 4]);
});
it("Delete leaves a point the box's value bounds exclude", () => {
// The box spans the whole clip but only its top, so Delete takes the one
// breakpoint up there and nothing else. A time range could not express this.
usePlayerStore.setState({ elements: [bgmElement], selectedElementId: "bgm" });
usePlayerStore.getState().setAutomationSelection({
elementKey: "bgm",
target: "volume",
t0: 0,
t1: 4,
v0: 0.9,
v1: 1,
});
const { onCommit } = setup({});
key("Delete");
const written = onCommit.mock.calls.at(-1)?.[0];
const points = written?.lanes?.[0]?.points ?? [];
// Fixture is (0, v=1), (2, v=0.5), (4, v=0): only the first was in the box.
expect(points.map((p: { t: number }) => p.t)).toEqual([2, 4]);
});
it("Delete takes points sitting exactly on the selection's edges", () => {
// Endpoint-inclusive, matching the copy path: a point the selection was dragged
// over is inside it, edge or not. Every range operation leaves a breakpoint
// exactly on an edge, so excluding them would leave those behind every time.
usePlayerStore.setState({ elements: [bgmElement], selectedElementId: "bgm" });
usePlayerStore
.getState()
.setAutomationSelection(wholeAxis({ elementKey: "bgm", target: "volume", t0: 2, t1: 4 }));
const { onCommit } = setup({});
key("Delete");
const points = onCommit.mock.calls.at(-1)?.[0]?.lanes?.[0]?.points ?? [];
expect(points.map((p: { t: number }) => p.t)).toEqual([0]);
});
it("Delete over a stretch with no breakpoints writes nothing at all", () => {
// A no-op rather than a write: emptying a span that had nothing in it used to
// push an undo entry that changed nothing but the anchors it invented.
usePlayerStore.setState({ elements: [bgmElement], selectedElementId: "bgm" });
usePlayerStore
.getState()
.setAutomationSelection(wholeAxis({ elementKey: "bgm", target: "volume", t0: 2.5, t1: 3.5 }));
const { onCommit } = setup({});
const e = new KeyboardEvent("keydown", { key: "Delete", bubbles: true, cancelable: true });
act(() => void document.dispatchEvent(e));
expect(onCommit).not.toHaveBeenCalled();
expect(e.defaultPrevented).toBe(false);
});
it("Delete clears the lane when the selection covers all of it", () => {
usePlayerStore.setState({ elements: [bgmElement], selectedElementId: "bgm" });
usePlayerStore
.getState()
.setAutomationSelection(wholeAxis({ elementKey: "bgm", target: "volume", t0: 0, t1: 6 }));
const { onCommit } = setup({});
key("Delete");
const written = onCommit.mock.calls.at(-1)?.[0];
// withLane drops a lane with no points left, so the attribute goes empty and
// the clip is back to its plain data-volume.
expect(written?.lanes ?? []).toEqual([]);
});
it("Escape clears the selection", () => {
@@ -194,8 +260,10 @@ describe("useAutomationSelectionKeyboard", () => {
target: "volume",
t0: 5,
t1: 7,
v0: VOLUME_RANGE.min,
v1: VOLUME_RANGE.max,
// Full height: everything the paste landed is selected, so Delete straight
// after undoes it in one press.
v0: 0,
v1: 1,
});
});
@@ -231,8 +299,10 @@ describe("useAutomationSelectionKeyboard", () => {
target: "volume",
t0: 4,
t1: 6,
v0: VOLUME_RANGE.min,
v1: VOLUME_RANGE.max,
// Full height: everything the paste landed is selected, so Delete straight
// after undoes it in one press.
v0: 0,
v1: 1,
});
});
@@ -267,8 +337,10 @@ describe("useAutomationSelectionKeyboard", () => {
target: "volume",
t0: 4,
t1: 6,
v0: VOLUME_RANGE.min,
v1: VOLUME_RANGE.max,
// Full height: everything the paste landed is selected, so Delete straight
// after undoes it in one press.
v0: 0,
v1: 1,
});
});
packages/studio/src/hooks/useAutomationSelectionKeyboard.ts
+29 -20
View File
@@ -1,8 +1,8 @@
/**
* Keyboard surface for the active automation selection: Escape clears,
* Delete/Backspace empties the range (anchors pinned, envelope outside
* untouched), Cmd/Ctrl+C copies it, Cmd/Ctrl+V pastes at the selection's
* start (or the playhead) onto the selected clip's lane. Sibling of
* Delete/Backspace deletes every breakpoint inside the selection box, Cmd/Ctrl+C
* copies its span, Cmd/Ctrl+V pastes at the selection's start (or the playhead) onto the
* selected clip's lane. Sibling of
* useKeyframeKeyboard and copies its contract: capture phase so playback
* shortcuts cannot swallow keys we act on, inert while any text input has
* focus, and a key is only consumed when it does something.
@@ -16,7 +16,7 @@
import { useEffect } from "react";
import { usePlayerStore, type TimelineElement } from "../player/store/playerStore";
import { laneFor, withLane } from "../player/components/automationLaneGeometry";
import { replaceRange } from "../player/components/automationLaneSelection";
import { pointInSelection, replaceRange } from "../player/components/automationLaneSelection";
import {
copyRange,
isLastPasteSpan,
@@ -87,11 +87,22 @@ function resolveSelectionContext(
}
/**
* The write that empties the active selection, or null when there is nothing
* to do: the clip is gone, its lane is read-only, the target no longer
* resolves to a range, or the lane already has no points in it. Split out of
* the keydown handler so each stays under the complexity a single branch of
* keyboard dispatch should carry.
* The write that deletes the breakpoints inside the active selection, or null when
* there is nothing to do: the clip is gone, its lane is read-only, the target no
* longer resolves to a range, or the selection covers no breakpoints.
*
* Deletes them outright rather than emptying the span behind anchor points. Anchors
* are what `replaceRange` exists for, and they are right for a shape insert or a
* paste — the envelope either side of the edit must not move. But Delete over a
* selection is the author saying "these points, gone", and answering that with two
* NEW points at the selection's edges reads as the delete not having worked. The
* envelope between the surviving neighbours re-interpolates, which is what deleting
* a breakpoint means everywhere else in the lane (right-clicking one does exactly
* this).
*
* Both axes of the selection box, edges included: what Delete removes is exactly
* what the lane drew a ring around. A point at the right time but outside the box's
* value bounds stays — which is the whole reason the box has them.
*/
function resolveDeleteWrite(
state: PlayerState,
@@ -99,14 +110,12 @@ function resolveDeleteWrite(
sel: AutomationSelection,
): { onCommit(next: HfAutomation): void; next: HfAutomation } | null {
const ctx = resolveSelectionContext(state, lanes, sel);
if (!ctx || ctx.lane.points.length === 0) return null;
const points = replaceRange({
lane: ctx.lane,
range: ctx.range,
t0: sel.t0,
t1: sel.t1,
inner: [],
});
if (!ctx) return null;
const points = ctx.lane.points.filter((p) => !pointInSelection(p, sel));
// Nothing inside is nothing to do — and it must stay a no-op rather than
// writing, or Delete over a smooth stretch would push an undo entry that
// changed nothing.
if (points.length === ctx.lane.points.length) return null;
return {
onCommit: ctx.binding.onCommit,
next: withLane(ctx.binding.automation, { target: sel.target, points }),
@@ -246,9 +255,9 @@ function handlePaste(
paste.binding.onCommit(withLane(paste.binding.automation, { target: paste.target, points }));
// Select the pasted span — the only feedback that it landed — and mark it, so
// an immediate second Cmd+V recognises this selection as the paste's own and
// chains right after it instead of overwriting it. Full-height box over the
// pasted span: this path only ever reads t0/t1, so the box's value bounds are
// cosmetic — but the parameter's own range keeps it a sensible box to draw.
// chains right after it instead of overwriting it.
// Full-height box over the pasted span: everything that landed is selected, so
// Delete straight after a paste undoes it in one press.
const mark = {
elementKey: paste.elementKey,
target: paste.target,
packages/studio/src/hooks/useDomEditAttributeCommits.ts
+13 -1
View File
@@ -394,7 +394,19 @@ export function useDomEditAttributeCommits({
},
onError: (error) => reportDomEditPersistFailure(domEditSelection, [op], error, showToast),
shouldResync: () => isLatestCommit(),
resync: () => refreshDomEditSelectionFromPreview(domEditSelection),
resync: () => {
refreshDomEditSelectionFromPreview(domEditSelection);
// The player store keeps its own copy of each element's attributes, and
// that copy is what the timeline's automation lanes draw from. Nothing
// else refreshes it: a commit patches the preview document and the file,
// and resyncs the dom-edit SELECTION for the panel. So every writer that
// did not also update the store by hand — the FX panel's automate and
// un-automate buttons, the keyboard Delete, a paste — changed the file and
// the audio while the lane went on drawing what it had, until a reload.
// One sink here rather than a sync in each writer, because three of them
// shipped without one.
syncStoredAutomationFromPreview(previewIframeRef.current?.contentDocument ?? null);
},
});
},
[
packages/studio/src/hooks/useDomEditCommits.test.tsx
+38
View File
@@ -1338,6 +1338,44 @@ describe("useDomEditCommits attribute persist handling", () => {
}
});
it("applies a preview-only write without persisting it", async () => {
// What a drag needs from every pointermove: the preview and the audio graph
// follow, the file does not. Persisting each move filled the undo stack with
// fragments of one gesture — and because those writes race, a follow-up's
// "before" often was not the previous entry's "after", so history refused to
// coalesce them and undo took back a few milliseconds of the drag.
const fetchSpy = stubPatchFetch({ ok: true, changed: true, matched: true });
const { iframe, element } = createPreviewElement();
const rendered = renderDomEditCommits(createSelection(element), iframe);
try {
await act(async () => {
await rendered.hook.handleDomAttributeLiveCommit("volume", "0.7", undefined, {
previewOnly: true,
});
});
expect(element.getAttribute("data-volume")).toBe("0.7");
expect(fetchSpy).not.toHaveBeenCalled();
} finally {
rendered.cleanup();
}
});
it("still persists a live write that does not ask to be preview-only", async () => {
const fetchSpy = stubPatchFetch({ ok: true, changed: true, matched: true });
const { iframe, element } = createPreviewElement();
const rendered = renderDomEditCommits(createSelection(element), iframe);
try {
await act(async () => {
await rendered.hook.handleDomAttributeLiveCommit("volume", "0.7");
});
expect(fetchSpy).toHaveBeenCalled();
} finally {
rendered.cleanup();
}
});
it("keeps a data-attribute commit on success", async () => {
stubPatchFetch({
ok: true,
packages/studio/src/hooks/useLivePlayheadTime.ts
+49
View File
@@ -0,0 +1,49 @@
/**
* The playhead in composition seconds, live while the transport runs.
*
* The RAF loop deliberately does not push every frame through the store — it
* notifies `liveTime` instead, so the playhead can move without re-rendering the
* app. A panel that wants to follow it therefore has to subscribe itself, and
* throttle: 30 Hz reads as continuous and costs an order of magnitude less than a
* render per frame.
*
* Paused, the store is the truth — a seek or a scrub lands there — so this returns
* that instead, which is what lets a readout follow the playhead while it is being
* dragged as well as while it is playing.
*/
import { useEffect, useRef, useState } from "react";
import { liveTime, usePlayerStore } from "../player";
/** Long enough to be much cheaper than a frame, short enough to read as motion. */
const THROTTLE_MS = 33;
export function useLivePlayheadTime(): number {
const storeTime = usePlayerStore((s) => s.currentTime);
const isPlaying = usePlayerStore((s) => s.isPlaying);
const liveRef = useRef(storeTime);
const [, forceRender] = useState(0);
// Paused, the ref tracks the store so the first frame of playback is never a
// stale value from the last time the transport ran.
if (!isPlaying) liveRef.current = storeTime;
useEffect(() => {
if (!isPlaying) return;
let timerId: ReturnType<typeof setTimeout> | 0 = 0;
const unsubscribe = liveTime.subscribe((t) => {
liveRef.current = t;
if (!timerId) {
timerId = setTimeout(() => {
timerId = 0;
forceRender((v) => v + 1);
}, THROTTLE_MS);
}
});
return () => {
unsubscribe();
if (timerId) clearTimeout(timerId);
};
}, [isPlaying]);
return isPlaying ? liveRef.current : storeTime;
}
packages/studio/src/hooks/usePreviewPersistence.ts
+7
View File
@@ -14,6 +14,7 @@ import {
import { trackStudioEvent } from "../utils/studioTelemetry";
import { applyUndoRestoreToPreview, type UndoRestoreFile } from "../utils/gsapUndoRestore";
import { usePlayerStore } from "../player";
import { syncStoredAutomationFromPreview } from "../player/lib/automationStoreSync";
/** The restore payload the undo/redo preview-sync consumes (from the history store). */
interface HistoryPreviewRestore {
@@ -207,7 +208,13 @@ export function usePreviewPersistence({
player.setElements([]);
player.setSelectedElementId(null);
player.setTimelineReady(false);
return;
}
// A soft restore patched the reverted attributes onto the live preview, but the
// player store keeps its own copy and that copy is what the automation lanes
// draw — so without this an undone envelope edit stayed invisible until a
// reload. The full path above clears the store and waits for discovery instead.
syncStoredAutomationFromPreview(previewIframeRef.current?.contentDocument ?? null);
},
[previewIframeRef, activeCompPathRef, reloadPreview],
);
packages/studio/src/player/components/automationLaneData.ts
+1 -1
View File
@@ -131,7 +131,7 @@ function orderLanes(lanes: HfAutomationLane[], chain: HfAudioFxChain | null): Hf
}
/** A frequency as an author reads it: 400 Hz, 1.6 kHz, 10 kHz. */
function formatHz(freq: number): string {
export function formatHz(freq: number): string {
if (freq < 1000) return `${Math.round(freq)} Hz`;
const k = freq / 1000;
return `${k >= 10 ? Math.round(k) : Number(k.toFixed(1))} kHz`;