Commit Graph
4002 Commits
Author SHA1 Message Date
Vance Ingalls 42b945ddae merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:12 -07:00
Vance Ingalls 830a21c1dc merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:11 -07:00
Vance Ingalls 9cb9a76001 merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:10 -07:00
Vance Ingalls 652fea0574 merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:10 -07:00
Vance Ingalls edff77fbf1 merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:09 -07:00
Vance Ingalls b0998a8aea merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:08 -07:00
Vance Ingalls c9c535c70e merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:08 -07:00
Vance Ingalls ff6db14fa7 merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:07 -07:00
Vance Ingalls 4b068fb529 merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:06 -07:00
Vance Ingalls 43b03b682a merge: pull in the wet/dry dedupe fix 2026-08-12 14:12:06 -07:00
Vance Ingalls f951af68fe merge: pull in the wet/dry dedupe fix 2026-08-12 14:11:16 -07:00
Vance Ingalls 9f6e04c33b merge: pull in the wet/dry dedupe fix 2026-08-12 14:11:15 -07:00
Vance Ingalls 0d92b36f35 merge: pull in the wet/dry dedupe fix
# Conflicts:
#	packages/core/src/audio/audioFxGraph.ts
2026-08-12 14:11:07 -07:00
Vance Ingalls 7cf032d2f6 fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo
Both effect builders set wet.gain to the mix and dry.gain to its complement
in identical two-line blocks; fallow kept re-flagging it as a 10-line clone
on every unrelated change. Extracted setWetDryMix.
2026-08-12 14:07:55 -07:00
Vance Ingalls 89206869e1 merge: pull in the main-merge conflict fix 2026-08-12 13:44:38 -07:00
Vance Ingalls dd81d79528 merge: pull in the main-merge conflict fix 2026-08-12 13:44:37 -07:00
Vance Ingalls b82bb88fef merge: pull in the main-merge conflict fix 2026-08-12 13:44:36 -07:00
Vance Ingalls e830e08a22 merge: pull in the main-merge conflict fix 2026-08-12 13:44:36 -07:00
Vance Ingalls 31f486222c merge: pull in the main-merge conflict fix 2026-08-12 13:44:35 -07:00
Vance Ingalls 7da9de505a merge: pull in the main-merge conflict fix 2026-08-12 13:44:34 -07:00
Vance Ingalls 943e8d7890 merge: pull in the main-merge conflict fix 2026-08-12 13:44:34 -07:00
Vance Ingalls df8d168e1a merge: pull in the main-merge conflict fix 2026-08-12 13:44:33 -07:00
Vance Ingalls cc70cd1384 merge: pull in the main-merge conflict fix 2026-08-12 13:44:33 -07:00
Vance Ingalls 0c163524ad merge: pull in the main-merge conflict fix 2026-08-12 13:44:20 -07:00
Vance Ingalls fe19f99dcf merge: pull in the main-merge conflict fix 2026-08-12 13:43:37 -07:00
Vance Ingalls 6e3c8d1a89 merge: pull in the main-merge conflict fix 2026-08-12 13:43:37 -07:00
Vance Ingalls 5ee0b17d12 merge: pull in the main-merge conflict fix 2026-08-12 13:43:21 -07:00
Vance Ingalls 6edd737b47 Merge remote-tracking branch 'origin/main' into wa-2-fx-preview
# Conflicts:
#	packages/studio/src/components/editor/PropertyPanelFlat.tsx
#	packages/studio/src/components/editor/propertyPanelFxControls.tsx
#	packages/studio/src/components/editor/propertyPanelFxSection.test.tsx
#	packages/studio/src/components/editor/propertyPanelFxSection.tsx
2026-08-12 13:42:44 -07:00
Vance IngallsandClaude Opus 5 c2996c8626 feat(studio): the FX panel, generated from the registry (#3022)
* feat(engine): render audio FX in an OfflineAudioContext

Reads `data-fx-chain` off an audio element and runs the chain over the trimmed
WAV before volume automation is baked in — effects should see the raw signal,
and the envelope belongs on their output.

The processing happens in an OfflineAudioContext inside the headless browser
the engine already drives, running the same graph builders the studio previews
with. That is the point of the approach: one implementation per effect, so the
render agreeing with the preview is a property of the architecture rather than
a tolerance to police. Reimplementing each effect as an FFmpeg filter would
mean two implementations to keep in step, and for the dynamics processors and
modulated delays there is no filter that behaves the same way.

`build:audio-fx-runtime` bundles the graph builders into an injectable IIFE,
following the same pattern as the existing runtime artifacts, so the browser
runs exactly the code the studio does.

The page loads from a file:// URL rather than about:blank because AudioWorklet
is only exposed in a secure context — the compressor, limiter, gate and
bitcrush processors would otherwise fail to register with an opaque error.
file:// qualifies and needs no listening socket.

The chain is serialised into the attribute the way colour grading carries its
config, so there is no side-car file to resolve or lose.

An FX failure is fatal for the whole mix rather than a per-track soft failure.
Every other audio failure mode degrades gracefully — the track drops, siblings
continue — but substituting the dry signal for a processed one ships a render
that sounds plausible and is not what the author set up. Since the per-element
work races under Promise.all, an internal AbortController chained off the
caller's signal aborts in-flight siblings before workDir is removed.

* feat(core): voiceover carve analysis

Finds the bands a voice occupies so a music bed can be dipped there, letting
the voice sit in front without ducking the whole track.

Carve is a relationship between two tracks rather than an effect on one, so it
stays out of the FX chain. What it emits is an ordinary chain of peaking
filters, so a carve composes with whatever else is on the track and needs no
separate rendering path.

Selection is weighted toward intelligibility rather than raw voice energy.
Ranking purely by power lands on the fundamental almost every time, because
that is where a voice is loudest — but the masking that actually hurts a
voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The
bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights
toward 1-3 kHz.

Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB
across these bands — it falls off roughly 6 dB per octave above the fundamental
— so a weighting has to be on that scale to move anything at all. A
multiplicative weight of `1 - bias + bias * shaped` is bounded below by
`1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7
default, 3 dB at 0.5. That is no influence against a real voice — every bias
short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the
outcome the bias exists to prevent, while looking decisive against a fixture
whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth
up to 30 dB at full strength, and relative cut depths come from a dB difference
rather than a ratio of weighted linear powers.

The bias reweights ranking without overriding the spectrum — a band the voice
has no energy in is not worth carving, and scores -Infinity rather than
competing — so a strongly low-pitched voice can still select low at full bias.
What the tests hold is that biasing never selects lower than the unbiased
ranking, that the DEFAULT bias reaches the presence region on a voice with a
realistic tilt, and that bias 0 still follows raw power exactly.

Includes a radix-2 FFT rather than a dependency; one Welch-style averaged
spectrum over third-octave bands does not justify pulling in a DSP library.

* fix(engine): keep the FX render 16-bit, stereo, and correctly sized

Three defects in the offline FX path, none of which any test could see.

**Float output silently disabled sample-accurate volume automation.** The writer
emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope
into the samples and accepts only 16-bit PCM, returning null otherwise. So
enabling any effect downgraded that track to the ffmpeg expression path — capped
at 32 straight segments, quantising a curved envelope, and on a dense one falling
back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a
limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test
asserts the baker accepts the writer's own output and actually fades it.

**Everything was folded to mono.** `prepareAudioTrack` goes out of its way to
emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo
rematrix — and this folded it, then wrote one channel. So adding a single peaking
EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed
stereo. Channels now travel as one plane each, through an OfflineAudioContext of
the same width, and come back interleaved.

**Small results decoded the wrong length.** `new Float32Array(buf.buffer)`
discards byteOffset and byteLength, and Node pools small allocations: a 400-byte
payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples
decoded as 2048 samples of unrelated memory — and the empty-result guard could
not see it. The reader has the mirror-image fix: a float data chunk on an odd
boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now
copies instead of throwing RangeError on an unaligned view.

The tail limitation is now stated rather than mis-stated: the context is exactly
as long as the input, so a reverb or delay still ringing is cut there. The old
comment claimed the opposite. How far a tail may run past a clip's end changes
the clip's length in the mix, so it is a product decision, not one to make here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(producer): report an FX render failure as an audio error

`processCompositionAudio` reports per-track failures in its result, but an FX
failure it cannot degrade past — a browser that will not launch, a chain that
will not build — rejects instead. `runAudioStage` had no try, so that rejection
escaped to the orchestrator as an unclassified pipeline exception, losing the
stage/owner/retryable classification this stage exists to attach, and skipping
its abort check on the way out.

It now lands in `audioError` alongside every other cause, while an abort still
keeps its own shape rather than being reported as an audio problem.

Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh
clone typechecks packages/engine without building first. The bundle is built from
the stub, and the stub changes three times across this stack — so the artifact
differs per branch and would conflict on every restack. Its model,
position-edits-render-inline.ts, is committed only because it is stable. Building
before testing is this monorepo's existing contract (studio's tests need core's
dist too), so the gap is not specific to audio FX and is better closed by a build
ordering gate than by committing a per-branch artifact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(engine): skip the browser FX render cases when there is no browser

CI's `Test` job was red on this PR with four failures, all the same cause:

  Failed to launch the browser process: spawn
  /home/runner/.cache/hyperframes/chrome/chrome-headless-shell

The job installs ffmpeg and no browser, deliberately — every other suite
that needs an external binary already guards on it
(`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming
a Chrome, so they failed on an absent dependency rather than on anything
about the code.

Guards on `resolveHeadlessShellPath()` — the same resolver
`acquireBrowser` launches through, so the check cannot drift from the
thing it guards the way a hard-coded cache path would. A configured path
that does not exist throws; that is caught and read as "cannot run here".

Checked both directions rather than just the green one: with a browser all
11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a
missing binary exactly 3 skip and the other 8 still run. A guard that
silently skipped everything would have looked identical in CI.

They keep their value where it exists — every developer machine, and any
job that has run `hyperframes browser ensure`.

Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five
days and several force-pushes stale. None of the 17 open repo alerts are
in files this PR changes; it re-runs on this push.

* chore(engine): suppress the temp-file alert with the reason it is safe

CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file
(high) — the one new alert on #3021, and the reason its CodeQL check is
red.

It is a false positive, and the comment says why rather than just silencing
it: `path` is always inside a directory made by `mkdtempSync`, never a
name assembled directly under `tmpdir()`. Both callers are covered — the
browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`,
and the render output goes to the producer work dir, itself
`mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the
random suffix and creates the directory 0700 in one syscall, so the
predictable filename inside it cannot be pre-created or symlinked by
another user, which is the attack the rule is about. The analyzer sees the
dataflow reach `tmpdir()` and not the mkdtemp in between.

Suppressed inline rather than dismissed in the UI, so the justification
lives next to the code and the rule stays live for anything added later in
this file. Matches the repo's existing convention — `planV2.ts:222`
carries an `lgtm[js/insecure-temporary-file]` for a different reason on
the same rule.

Correcting myself: I first reported this alert as not real, having
intersected the PR's files against the default-branch alert list, which
does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns
it straight away.

* test(engine): probe ffmpeg and Chrome instead of assuming them

Two failures on #3021's Test job, both about the environment rather than
the code under test.

**Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to
`execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide
ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()`
resolves — every other ffmpeg-dependent suite in this package already goes
through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)`
so a contributor without ffmpeg skips rather than fails.

**The browser guard trusted the wrong thing.** It asked
`resolveHeadlessShellPath()` and treated a returned path as "a browser is
here". CI's cache holds a chrome-headless-shell that resolves and then
fails to spawn — a partial download is indistinguishable from a working
one by `existsSync`, which is all that resolver checks. So the three
browser cases ran anyway and failed on the launch.

It now runs `--version` and requires exit 0, which is the same probe the
ffmpeg suites use: ask the binary, do not infer from the filesystem.

Checked both directions rather than just the green one. With a working
browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed
at a binary that exits non-zero — CI's exact situation — exactly 3 skip
and the other 8 still run. A guard that quietly skipped everything would
have looked identical on the CI summary.

* feat(core): register the audio-fx-rack canary at 0%

Lands the rollout switch dark, per the registry's own procedure: "Start at
percentage: 0 and merge that — a canary at 0 is dead code you can land
safely and ramp without a code review."

Declared at the bottom of the stack so every branch above can read it. The
gate itself goes in at wa-4-fx-panel, where the rack first appears.

Scope is deliberate and stated in the description: it gates the AUTHORING
surface only. A composition that already carries `data-fx-chain` still
plays and renders it. A canary should stage who can REACH a feature, not
make an attribute somebody already wrote silently inert — an agent that
writes a chain through the skill would otherwise produce a file whose audio
processing vanishes with no error.

* feat(studio): audio FX panel generated from the registry

Controls for the whole chain: add, remove, reorder, bypass, and every knob each
effect declares.

Nothing in the panel knows what a compressor is. The registry supplies each
parameter's range, step, unit and scale and the panel renders what it finds, so
adding an effect or a knob upstream needs no change here, and the panel cannot
offer a value the renderer would reject — a typed-in figure is clamped into the
declared range on the way through.

Frequency and time controls span three or four decades, so those declare a log
scale and the slider maps exponentially; a linear slider would spend most of
its travel somewhere useless.

Reorder is a first-class control because chain order changes the sound: a
reverb before a compressor is not the same as after.

Carve gets its own block rather than an entry in the add menu, with a picker
for the voice track to listen to. It processes this track based on another one,
which is how a sidechain control works — it lives on the track that changes,
and names the source.

* feat(studio): show the Audio FX section on audio tracks

Adds `audioFx` to the editing-affordances contract and renders the FX panel in
the inspector when an `<audio>` element is selected.

The section is audio-only. A `<video>` carries its sound on a separate
`<audio>` element, so an FX chain on the video would have nothing to process.

Chain and carve settings are written straight back onto the element as
serialised attributes, the way colour grading carries its config, so
persistence is an ordinary attribute write and needs no new server route. A
chain that cannot be parsed renders as empty rather than breaking the panel,
and the attribute is left untouched until the user changes something.

The collapsed group summarises what is on the track ("2 effects + carve") so
the state is visible without expanding it.

Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED
defaults to true, so the flat inspector is what actually renders.

* refactor(studio): lift audioFxSummary out of PropertyPanelFlat

`PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap,
so the required File size check is red — the sole reason this PR is
blocked. The review says as much: "mechanical fix (~5 min), not a design
problem. Code itself is LGTM."

Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later
branch creates for it. Deliberately the smallest cut that clears the cap
rather than the whole `AudioFxGroup` extraction: every later commit in the
stack edits AudioFxGroup, so moving it here would collide with each of
them, while almost nothing touches this function.

595 lines.

* feat(studio): put the audio FX rack behind its canary

Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered
at 0% — so the whole 47-PR stack can land without showing anyone a feature
that has not been measured yet.

The gate sits on the AUTHORING surface and nowhere else. The runtime and
the render still honour a `data-fx-chain` already on an element, so a
composition written through the skill or by `carve.mjs` keeps its
processing rather than going silently dry for anyone outside the cohort. A
canary should stage who can REACH a feature, not make an attribute somebody
already wrote stop working with no error.

Gated at the panel rather than in `resolveEditingSections`: the affordance
resolver is a pure function in core describing what an element CAN support,
and rollout state is not a property of an `<audio>` tag.

Pinned the 0% with a test, and checked it fails at 25 — a ramp should have
to break something that says "this ships dark" out loud.

One gap, stated rather than papered over: the gate itself has no unit test.
I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness
never renders the Audio FX group for its audio fixture even with the gate
removed — so the test passed for the wrong reason in the off case and could
not pass at all in the on case. A test that cannot fail for the right
reason is worse than none. Verifying the gate needs the panel harness to
mount that section first, which is its own change.

* fix(studio): drop the FX panel's dead __testables export

Fallow audit flagged it — no test imports the module.

* fix(core,studio): clear the remaining Fallow audit findings on the FX panel

- Split FxSection's per-node row into FxNodeRow + FxNodeControls so the
  CRAP score (31.6, threshold 30) splits across two smaller units instead
  of moving wholesale with one extraction.
- Dedupe the repeated "open the add menu, read its items" block in
  propertyPanelFxSection.test.tsx into openAddMenuItems().
- Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into
  one build-inline-artifact.ts, config-selected by CLI arg — the two
  scripts were a byte-for-byte clone save for names.
- Exempt canary.test.ts's rawFnv (a deliberate independent
  reimplementation used to cross-check canaryBucket, per its own
  docstring) and the property-panel test files' shared renderInto/mount
  scaffolding (pre-existing across 9 files, 2 outside this stack) in
  .fallowrc.jsonc, consistent with this file's existing exemptions for
  the same class of intentional/pre-existing duplication.

* fix(ci): allowlist the build-script consolidation in the no-main-deletions guard

build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into
build-inline-artifact.ts to kill a fallow duplication finding; the deletion
guard flagged that as an accidental loss since main still has both originals.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 13:34:08 -07:00
Vance Ingalls c86a1f059d merge: pull in the deletion-guard fix 2026-08-12 12:54:37 -07:00
Vance Ingalls 06150a9247 merge: pull in the deletion-guard fix 2026-08-12 12:54:36 -07:00
Vance Ingalls 79f02ec743 merge: pull in the deletion-guard fix 2026-08-12 12:54:36 -07:00
Vance Ingalls e14bd85cfc merge: pull in the deletion-guard fix 2026-08-12 12:54:35 -07:00
Vance Ingalls 4f1974e7d7 merge: pull in the deletion-guard fix 2026-08-12 12:54:35 -07:00
Vance Ingalls dc951b963d merge: pull in the deletion-guard fix 2026-08-12 12:54:34 -07:00
Vance Ingalls 46efb156ea merge: pull in the deletion-guard fix 2026-08-12 12:54:33 -07:00
Vance Ingalls fe1ef0ccdd merge: pull in the deletion-guard fix 2026-08-12 12:54:33 -07:00
Vance Ingalls 82f8d2078d merge: pull in the deletion-guard fix 2026-08-12 12:54:32 -07:00
Vance Ingalls b9c76842d2 merge: pull in the deletion-guard fix 2026-08-12 12:54:32 -07:00
Vance Ingalls 556afc7fd8 merge: pull in the deletion-guard fix 2026-08-12 12:54:25 -07:00
Vance Ingalls 3e2c2595fa merge: pull in the deletion-guard fix 2026-08-12 12:54:24 -07:00
Vance Ingalls e1011a856b merge: pull in the deletion-guard fix 2026-08-12 12:54:24 -07:00
Vance Ingalls 70693236e0 merge: pull in the deletion-guard fix 2026-08-12 12:54:23 -07:00
Vance Ingalls dad13c8f43 fix(ci): allowlist the build-script consolidation in the no-main-deletions guard
build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into
build-inline-artifact.ts to kill a fallow duplication finding; the deletion
guard flagged that as an accidental loss since main still has both originals.
2026-08-12 12:54:04 -07:00
Vance Ingalls 4d2a37b20e merge: pull in the main-catchup merge 2026-08-12 12:50:13 -07:00
Vance Ingalls 788adde693 merge: pull in the main-catchup merge 2026-08-12 12:50:12 -07:00
Vance Ingalls 80070abb49 merge: pull in the main-catchup merge 2026-08-12 12:50:11 -07:00
Vance Ingalls 34f0546e50 merge: pull in the main-catchup merge 2026-08-12 12:50:11 -07:00
Vance Ingalls a99d6d63a0 merge: pull in the main-catchup merge 2026-08-12 12:50:10 -07:00
Vance Ingalls 2fb958871e merge: pull in the main-catchup merge 2026-08-12 12:50:09 -07:00