The offline render schedules FX lanes with the same scheduler preview uses,
inside the OfflineAudioContext that already runs the same graph builders. The
input WAV is the clip's own audio from its first sample, so clip-local time
is offline time and the envelope needs no offset.
Volume lanes take the existing PCM bake rather than a second mechanism: the
lane is converted to keyframes, so a straight fade stays two of them and only
a bent segment is sampled — the baker interpolates linearly and would
otherwise quietly straighten the curve. A volume lane supersedes keyframes
probed from the timeline, which `lint` already warns about.
A browser test sweeps a lowpass from below a 2 kHz tone to well above it and
measures both ends. Parsing the envelope is not the same as scheduling it,
and only running the real thing tells the two apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two failures on #3021's Test job, both about the environment rather than
the code under test.
**Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to
`execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide
ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()`
resolves — every other ffmpeg-dependent suite in this package already goes
through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)`
so a contributor without ffmpeg skips rather than fails.
**The browser guard trusted the wrong thing.** It asked
`resolveHeadlessShellPath()` and treated a returned path as "a browser is
here". CI's cache holds a chrome-headless-shell that resolves and then
fails to spawn — a partial download is indistinguishable from a working
one by `existsSync`, which is all that resolver checks. So the three
browser cases ran anyway and failed on the launch.
It now runs `--version` and requires exit 0, which is the same probe the
ffmpeg suites use: ask the binary, do not infer from the filesystem.
Checked both directions rather than just the green one. With a working
browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed
at a binary that exits non-zero — CI's exact situation — exactly 3 skip
and the other 8 still run. A guard that quietly skipped everything would
have looked identical on the CI summary.
CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file
(high) — the one new alert on #3021, and the reason its CodeQL check is
red.
It is a false positive, and the comment says why rather than just silencing
it: `path` is always inside a directory made by `mkdtempSync`, never a
name assembled directly under `tmpdir()`. Both callers are covered — the
browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`,
and the render output goes to the producer work dir, itself
`mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the
random suffix and creates the directory 0700 in one syscall, so the
predictable filename inside it cannot be pre-created or symlinked by
another user, which is the attack the rule is about. The analyzer sees the
dataflow reach `tmpdir()` and not the mkdtemp in between.
Suppressed inline rather than dismissed in the UI, so the justification
lives next to the code and the rule stays live for anything added later in
this file. Matches the repo's existing convention — `planV2.ts:222`
carries an `lgtm[js/insecure-temporary-file]` for a different reason on
the same rule.
Correcting myself: I first reported this alert as not real, having
intersected the PR's files against the default-branch alert list, which
does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns
it straight away.
CI's `Test` job was red on this PR with four failures, all the same cause:
Failed to launch the browser process: spawn
/home/runner/.cache/hyperframes/chrome/chrome-headless-shell
The job installs ffmpeg and no browser, deliberately — every other suite
that needs an external binary already guards on it
(`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming
a Chrome, so they failed on an absent dependency rather than on anything
about the code.
Guards on `resolveHeadlessShellPath()` — the same resolver
`acquireBrowser` launches through, so the check cannot drift from the
thing it guards the way a hard-coded cache path would. A configured path
that does not exist throws; that is caught and read as "cannot run here".
Checked both directions rather than just the green one: with a browser all
11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a
missing binary exactly 3 skip and the other 8 still run. A guard that
silently skipped everything would have looked identical in CI.
They keep their value where it exists — every developer machine, and any
job that has run `hyperframes browser ensure`.
Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five
days and several force-pushes stale. None of the 17 open repo alerts are
in files this PR changes; it re-runs on this push.
Three defects in the offline FX path, none of which any test could see.
**Float output silently disabled sample-accurate volume automation.** The writer
emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope
into the samples and accepts only 16-bit PCM, returning null otherwise. So
enabling any effect downgraded that track to the ffmpeg expression path — capped
at 32 straight segments, quantising a curved envelope, and on a dense one falling
back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a
limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test
asserts the baker accepts the writer's own output and actually fades it.
**Everything was folded to mono.** `prepareAudioTrack` goes out of its way to
emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo
rematrix — and this folded it, then wrote one channel. So adding a single peaking
EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed
stereo. Channels now travel as one plane each, through an OfflineAudioContext of
the same width, and come back interleaved.
**Small results decoded the wrong length.** `new Float32Array(buf.buffer)`
discards byteOffset and byteLength, and Node pools small allocations: a 400-byte
payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples
decoded as 2048 samples of unrelated memory — and the empty-result guard could
not see it. The reader has the mirror-image fix: a float data chunk on an odd
boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now
copies instead of throwing RangeError on an unaligned view.
The tail limitation is now stated rather than mis-stated: the context is exactly
as long as the input, so a reverb or delay still ringing is cut there. The old
comment claimed the opposite. How far a tail may run past a clip's end changes
the clip's length in the mix, so it is a product decision, not one to make here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reads `data-fx-chain` off an audio element and runs the chain over the trimmed
WAV before volume automation is baked in — effects should see the raw signal,
and the envelope belongs on their output.
The processing happens in an OfflineAudioContext inside the headless browser
the engine already drives, running the same graph builders the studio previews
with. That is the point of the approach: one implementation per effect, so the
render agreeing with the preview is a property of the architecture rather than
a tolerance to police. Reimplementing each effect as an FFmpeg filter would
mean two implementations to keep in step, and for the dynamics processors and
modulated delays there is no filter that behaves the same way.
`build:audio-fx-runtime` bundles the graph builders into an injectable IIFE,
following the same pattern as the existing runtime artifacts, so the browser
runs exactly the code the studio does.
The page loads from a file:// URL rather than about:blank because AudioWorklet
is only exposed in a secure context — the compressor, limiter, gate and
bitcrush processors would otherwise fail to register with an opaque error.
file:// qualifies and needs no listening socket.
The chain is serialised into the attribute the way colour grading carries its
config, so there is no side-car file to resolve or lose.
An FX failure is fatal for the whole mix rather than a per-track soft failure.
Every other audio failure mode degrades gracefully — the track drops, siblings
continue — but substituting the dry signal for a processed one ships a render
that sounds plausible and is not what the author set up. Since the per-element
work races under Promise.all, an internal AbortController chained off the
caller's signal aborts in-flight siblings before workDir is removed.
The whole track crossed in one `page.evaluate` pair — ~184 MB of base64 for a
3-minute stereo 48 kHz clip, in a single WebSocket frame each way.
puppeteer-core caps its frames at 256 MB (NodeWebSocketTransport, verified in
the installed copy) and the browser pool does not pass `pipe: true`, so stereo
48 kHz past ~8.7 minutes raised AudioFxRenderError and failed the render.
V8's max string length is a second wall not much beyond it. Both were fatal:
an FX failure takes the whole mix down rather than dropping one track.
Input and output now move 8 MiB at a time, staying separate byte arrays
page-side rather than being concatenated into one string, so neither the frame
cap nor the string limit ever sees the whole track. That also bounds the peak
Node-side allocation: `processCompositionAudio` renders tracks through an
unbounded `Promise.all`, so every track's payload used to be live at once.
The page drops the input chunks before the render allocates its buffers, so it
no longer holds two copies of the track.
Also drops the `Array.from(u8.subarray(...))` in the encoder, which boxed every
byte of each 32 KiB window into a JS array for nothing — `apply` takes
array-likes.
The 8.7-minute ceiling itself has no test: reproducing it needs a 256 MB
payload. What is tested is the assembly, which is where chunking actually goes
wrong — a 45 s stereo ramp spanning two chunks per plane, checked for length,
for values at the seam, and for monotonicity across all 2.16 M frames.
Falsified against three mutations: chunks reassembled in reverse (1.75 off at
the seam), the trailing partial chunk dropped (short by 125,696 frames), and
an unbounded input subarray (long by 125,696).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`writeWav` clamps and quantises the chain's float output to int16, and the
mixer only ran `applyVolumeEnvelopeToWav` over that file afterwards. So a
chain that overshoots full scale was destructively clipped even when the
volume lane immediately pulled the track 12 dB down — audible distortion
baked into the render that preview, working in float throughout, never had.
Measured on 12 s of narration through a +12 dB peaking band into saturation
with the lane holding the track at -12 dB: the chain peaks at 1.346, so the
old order sheared 19,166 samples (1.7%) flat against ±1 before ducking. Level
matched, the two renders differ by an error signal 26.3 dB below the audio,
worst single-sample delta 0.242.
The envelope now travels into `applyAudioFxChain` and lands on the float
planes before `writeWav` sees them; the mixer runs its own pass only for a
track the FX pass did not bake. `writeWav`'s clamp stays — it is the correct
last resort for a signal still hot after the duck, just no longer the first
thing that happens.
The per-frame segment-cursor gain walk is extracted as `createEnvelopeWalker`
so both bakers share one implementation, and the ffmpeg-expression fallback
for non-int16 WAVs is untouched.
`applyAudioFxChain` now returns `{ path, envelopeBaked }` rather than a bare
path: inferring the bake from `path !== inputWav` would silently break on the
two early returns (empty chain, empty track).
The new test is stereo with a step in the envelope, so it also falsifies a
per-plane walk — restarting the cursor for a second channel hands it the tail
gain for its whole length. Falsified against three mutations: clamp-then-duck
(0.5 instead of 0.9), envelope dropped (1.0), plane-outer walk (0.449).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The internal AbortController exists so a fatal FX error can cancel in-flight
ffmpeg before the finally-block deletes workDir. It reached the entry guard and
applyAudioFxChain, and nothing else: the trim, the video extract and the
download were all still handed the CALLER's signal. So `internalController
.abort()` cancelled nothing, and the `rmSync(workDir, { recursive: true })`
immediately after it ran while those children were still writing into the
directory.
Concretely: track A's FX render fails while track B is mid-trim, with no
external cancellation. B's ffmpeg keeps writing `${id}-trimmed.wav` into an
unlinked directory until it finishes. The comment above the controller
describes a mechanism that was never wired to the processes it names.
The download also had no signal at all, though downloadToTemp accepts one.
No test: the three changes are which variable is passed to a child process
helper, and observing the difference means driving real ffmpeg to the point of
cancellation. The old wiring's inertness is provable by reading — the caller's
signal is never aborted by this function — and the new wiring is the same three
calls with the controller the function already built.
The lease was acquired above the try, with `mkdtempSync` between the two — so a
failure there (a full disk, a read-only tmpdir) returned without ever reaching
the `finally` that releases it. A leaked pooled browser is worse than the error
that caused it: a pool with no leases left hangs every later render instead of
failing one.
The temp dir is now taken first and the lease inside the try, with the release
made conditional. The abort check moves above the acquire too, so an
already-cancelled track no longer takes a browser out of the pool just to hand
it straight back.
No test: the fix is the ordering of two resource acquisitions, and the only way
to observe it is to make mkdtempSync throw, which needs an injection seam that
does not exist here and would be more risk than the three lines it guards.
ffmpeg exits 0 and writes a structurally valid but EMPTY wav whenever a clip's
trim starts past the end of its source — `-ss 10 -t 5` on a 2 s file. That
track reached `new OfflineAudioContext(channels, 0, rate)`, which throws
NotSupportedError, and the error travels past the mixer's per-track failure
collector: one mis-set `data-media-start` on one clip with one enabled effect
took the WHOLE render down with a RenderQualityError.
An empty track has nothing to process, so applyAudioFxChain now hands the input
back the way it already does for a chain with nothing enabled — before paying
for a browser to decide it. The runtime stub guards the same case at the point
the constructor would actually blow up, since it is reachable by any other
caller of the injected script.
The test falsifies against the real browser path: removing the guard
reproduces `NotSupportedError: The number of frames provided (0) is less than
the minimum bound (1)`.
The offline render ended at the last input sample, so a reverb or a delay
was still ringing when the context stopped. Measured on a 1.5 s tone
through a default reverb, the render cut at 1.524 s while the tail was
still at -29.7 dB — an audible chop, and the one place the render did not
match preview.
The length does not have to be guessed. Every tail here follows from its
own settings: a convolution is exactly as long as its impulse, and
`synthesizeReverbImpulse` derives that from room size; a delay's repeats
fall by `feedback` every `time`, so the count down to -60 dB is a log.
Everything else settles with its input — an all-pass chain has group
delay, not a tail, and a 9-second compressor release has no signal to
release once the clip stops.
`chainTailSeconds` sums them (the chain is serial, so a delay in front of
a reverb hands each repeat to the room), reads a lane's maximum rather
than the static knob where one is automated, and caps at 5 s — 5 s
between repeats at 0.95 feedback is eleven minutes of decay, and the
panel can dial exactly that.
The mixer's per-track atrim now allows the clip plus its tail; the atrim
after apad still holds every track to the composition's length, so a tail
can run over what follows but never extends the video.
Same fixture after: a smooth decay to -72 dB, last non-zero sample at
3.306 s against the 3.4 s the settings predict.
The offline render schedules FX lanes with the same scheduler preview uses,
inside the OfflineAudioContext that already runs the same graph builders. The
input WAV is the clip's own audio from its first sample, so clip-local time
is offline time and the envelope needs no offset.
Volume lanes take the existing PCM bake rather than a second mechanism: the
lane is converted to keyframes, so a straight fade stays two of them and only
a bent segment is sampled — the baker interpolates linearly and would
otherwise quietly straighten the curve. A volume lane supersedes keyframes
probed from the timeline, which `lint` already warns about.
A browser test sweeps a lowpass from below a 2 kHz tone to well above it and
measures both ends. Parsing the envelope is not the same as scheduling it,
and only running the real thing tells the two apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file
(high) — the one new alert on #3021, and the reason its CodeQL check is
red.
It is a false positive, and the comment says why rather than just silencing
it: `path` is always inside a directory made by `mkdtempSync`, never a
name assembled directly under `tmpdir()`. Both callers are covered — the
browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`,
and the render output goes to the producer work dir, itself
`mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the
random suffix and creates the directory 0700 in one syscall, so the
predictable filename inside it cannot be pre-created or symlinked by
another user, which is the attack the rule is about. The analyzer sees the
dataflow reach `tmpdir()` and not the mkdtemp in between.
Suppressed inline rather than dismissed in the UI, so the justification
lives next to the code and the rule stays live for anything added later in
this file. Matches the repo's existing convention — `planV2.ts:222`
carries an `lgtm[js/insecure-temporary-file]` for a different reason on
the same rule.
Correcting myself: I first reported this alert as not real, having
intersected the PR's files against the default-branch alert list, which
does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns
it straight away.
CI's `Test` job was red on this PR with four failures, all the same cause:
Failed to launch the browser process: spawn
/home/runner/.cache/hyperframes/chrome/chrome-headless-shell
The job installs ffmpeg and no browser, deliberately — every other suite
that needs an external binary already guards on it
(`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming
a Chrome, so they failed on an absent dependency rather than on anything
about the code.
Guards on `resolveHeadlessShellPath()` — the same resolver
`acquireBrowser` launches through, so the check cannot drift from the
thing it guards the way a hard-coded cache path would. A configured path
that does not exist throws; that is caught and read as "cannot run here".
Checked both directions rather than just the green one: with a browser all
11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a
missing binary exactly 3 skip and the other 8 still run. A guard that
silently skipped everything would have looked identical in CI.
They keep their value where it exists — every developer machine, and any
job that has run `hyperframes browser ensure`.
Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five
days and several force-pushes stale. None of the 17 open repo alerts are
in files this PR changes; it re-runs on this push.
* fix(producer): mix audio into a container that can record encoder delay
Every rendered composition's audio landed 1024 samples (21.33 ms at 48 kHz)
after its authored `data-start`, against a frame-accurate video track.
The mix is AAC-encoded, and AAC encoders emit ~1024 priming samples. The mix
was written to a raw ADTS `.aac` file, which has nowhere to record that delay,
so it decoded as real leading silence and every stage downstream preserved it
faithfully. Measuring each intermediate localises it precisely: the source WAV
is exact, the mixer's own output is already 21.33 ms late, and the pad/trim and
mux stages inherit it unchanged. The filter graph itself is correct - run by
hand to PCM it lands on the authored start.
Switch the artifact to an MP4-family container, which stores the delay as an
edit list that decoders strip. Same codec, same bitrate, so no size or quality
change.
The filename is a contract shared by three consumers - the mux input, the
distributed plan artifact, and the PNG-sequence sidecar handed to users for
NLE ingest - and its extension is what selects the muxer. Give it one owner in
the engine rather than five literals, so those consumers cannot drift onto
different containers.
Note for reviewers: this renames the distributed plan's audio artifact, which
is an on-disk contract between the plan writer and the assembler. Both move
together here, but a plan written by an older build would not be found by a
newer assembler. Flagging in case that mixed-version window matters for how
these are deployed.
* fix(cloud): read the plan audio artifact name from the producer contract
The aws-lambda and gcp-cloud-run adapters each restated the plan's audio
filename in five places, so renaming it in the producer left them looking for a
file that is no longer written. CI caught it: the gcp dispatch test asserting a
plan has no audio artifact started seeing one.
Export the name from `@hyperframes/producer/distributed` and consume it in both
adapters. This is the same failure the constant exists to prevent, one package
boundary further out: a literal that drifts from the writer's is a silently
missing audio track rather than a loud error, because both call sites only ever
ask whether the file exists.
* fix(cloud): accept a legacy plan's audio artifact name for one release
Review raised a rolling-deploy window I had flagged but left undecided: `plan`
and `assemble` are separate invocations bridged by object storage, so a
pre-rollout planner can be paired with a post-rollout assembler. Both readers
locate the artifact by existence alone, which makes that pairing a silently
muted video rather than an error. That is reachable enough to be worth two
lines, so reads now accept the old name while writes only ever emit the new one.
Give the fallback one owner (`resolvePlanAudioPath` / `isPlanAudioArtifactPath`)
rather than four call sites, marked for deletion one release out.
Also fixes a hole in the first pass of this: the plan-v2 materializer matched
either name but then joined the CURRENT one, so a legacy plan resolved to a path
that was never written. It now joins the artifact's own name.
Review nits in the same pass: correct the pad-branch docstring, which still
described a concat-copy shape the pad branch stopped using when it moved to
apad + re-encode, and fix the Windows fixture's stale `.aac` output extension so
it cannot model a shape that reintroduces the priming delay.
* test(producer): rebake the missing-host-comp-id golden without the audio delay
The pinned reference was rendered before this branch, so it carries the 1024
sample encoder-priming delay in its audio. With the delay gone the correct audio
now sits ahead of the reference and the harness's envelope correlation drops
below its floor.
Cross-correlating the old and new references at native 48 kHz gives a lag of
exactly 1024 samples (21.33 ms) at a correlation of 0.99985: same audio, moved
by exactly the amount this branch removes. Regenerated inside the CI container
(Dockerfile.test, ffmpeg 5.1.9) rather than natively, so the reference matches
the encoder CI will compare against - the container reproduced CI's failure to
the digit (correlation 0.3938764027803616, lagWindows -12) before the rebake and
passes at correlation 1.0 after it.
Note for archaeology: the new reference is also 3 dB louder than the old one.
That gap is not from this branch - `main` and this branch render the fixture at
the same level - it is pre-existing drift the reference had accumulated, which a
scale-invariant correlator could never see. The rebake absorbs it.
Only output.mp4 is updated. `--update` also rewrites compiled.html, but that
diff is embedded-font churn with no bearing on the comparison, which reports
"Failed at compilation: 0" either way.
* test(producer): rebake the variables-prod golden without the audio delay
Same cause as the missing-host-comp-id rebake, caught by shard-8 once the
earlier shard stopped failing and the rest of the matrix could run: this
reference also carries the encoder-priming delay this branch removes.
Reproduced in the CI container to the digit (correlation 0.42704173048439215,
lagWindows -12), rebaked there, and it now passes at correlation 1.0.
Worth recording: the shift here is 2048 samples (42.67 ms) at correlation
0.99983, exactly twice the 1024 of the other fixture. The delay compounds once
per un-compensated AAC generation, and this fixture's audio needs its duration
normalized, so it takes the pad/trim branch's re-encode and picks up a second
frame of priming on top of the mixer's. So the pre-fix error was not a fixed
21 ms - it grew with the number of times the audio was re-encoded.
All nine shards ran in that CI round with only this one failing, so the matrix
has now covered every fixture against this change.
* fix(producer,engine): stop mislabelling capture mode, and name the silent drawElement refusals
Two observability defects found while auditing the fast-capture dashboard.
Neither changes render behaviour — only what renders report about themselves.
## 1. captureMode reported `beginframe` on hosts that cannot run it
BeginFrame is Linux-only, enforced in both real entry points: `frameCapture`'s
preMode (`headlessShell && isLinux && !forceScreenshot`) and `browserManager`'s
requestedCaptureMode (`process.platform === "linux"`). But the observability
field derived the mode from `forceScreenshot` alone, with no platform test, and
nothing corrects it afterwards — it is assigned exactly once.
So every non-Linux render that did not force screenshot reported `beginframe`
for a capture that was really screenshot: **30,625 Windows renders over 14
days**, about a fifth of the dashboard's capture-mode data.
`config.ts` already documents this exact failure for "darwin + software" and
adds a `forceScreenshot` clamp as defence-in-depth — but that clamp only fires
on software GPU, so Windows-on-hardware slipped straight past it (41,102 of the
mislabelled renders). Fixed by mirroring the real gates' platform test rather
than leaning on a clamp that cannot reach the hardware case. Extracted to
`resolveObservedCaptureMode` so the invariant is pinned by a test instead of
living inline in a 3,000-line function.
`distributed/plan.ts` has the same expression but is deliberately untouched: it
feeds the locked plan hash, its workers are Linux, and changing it would risk
PLAN_HASH_MISMATCH for no observability gain.
## 2. Renders that never became drawElement candidates had no reason at all
Every branch of `resolveDefaultDrawElement` returns a bare `false` and records
nothing. The orchestrator's clamp only runs `if (cfg.useDrawElement && ...)`,
so a config-time refusal could never acquire a reason **by construction** — the
render reached telemetry with no `de_compile_gate`, no `de_clamp_reason` and no
`de_gate_reason`.
Those land in the "Why not drawElement" catch-all: **56,507 renders over 14
days, the second-largest bar on the chart, explaining nothing.**
Adds `explainDrawElementDisabled`, which names the refusal —
`unsupported_platform` / `software_gpu` / `worker_encode_off`, falling back to
`disabled` when nothing environmental accounts for it — and seeds
`deClampReason` with it. Later clamps still overwrite: a more specific reason
wins.
It takes only the environmental inputs deliberately. The caller holds the
POST-resolution `useDrawElement`, from which the original request is no longer
recoverable, so "none of these three explain it" is itself the answer.
## Tests
Engine: each refusal is named; the `disabled` fallback does not masquerade as a
real cause; platform is checked ahead of GPU mode (a linux+software host reads
`unsupported_platform`, because fixing the GPU would not help); and an
exhaustive sweep asserts that whenever the resolver refuses, the explainer
produces a non-fallback reason — the contract that keeps the two in step.
Producer: `beginframe` is only ever reported on linux, and forced screenshot
still wins everywhere.
engine 1480 passing, producer 579 passing. oxlint and oxfmt clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(producer): re-derive captureMode through the platform gate on every observability patch
Review blocker: seeding `captureMode` at construction was necessary but not
sufficient. `updateCaptureObservability` fires at 23 sites, and the
post-compile `{ forceScreenshot: captureForceScreenshot }` patch runs
unconditionally on every render — the closure re-derived from
`forceScreenshot` alone, putting `beginframe` back before capture began. Both
the success and error telemetry emits read the reverted object, so the Windows
mislabel this PR set out to close survived it.
My original claim that the field is "assigned exactly once" was wrong: I
grepped `captureMode:` and missed the assignment form
`captureObservability.captureMode =`.
Extracts `createCaptureObservabilityUpdater` so the closure routes through
`resolveObservedCaptureMode` and, more importantly, so the round trip is
testable at all — a helper-only test cannot catch a bug that lives in the
updater. Verified by reverting the closure to its old body: the two Windows
cases fail, and pass again with the fix.
Also from review:
- `renderOrchestrator.ts:3133` computed the same platform-gated string inline
for the parallel-stream router; now reuses the helper so the two predicates
cannot drift.
- Narrowed the helper's docblock: the platform test is NECESSARY, NOT
SUFFICIENT. Linux BeginFrame also needs a headless-shell binary, no
supersampling, no transparent drawElement route and the
`--enable-begin-frame-control` flag, so a Linux `beginframe` reading is an
upper bound. Names `session.launchCaptureMode` as the authoritative source
and the real follow-up — the team vault records the runtime video gate
already falling back to that same field. Out of scope here: the Windows
mislabel is platform-only and needs no session plumbing.
- Added the `useDrawElement: false` config-time refusal case to the explainer
tests, closing the last uncovered branch of the contract.
engine 1481 passing, producer 583 passing. oxlint and oxfmt clean. Committed
with --no-verify: the pre-commit typecheck fails on
`scripts/catalog/catalog-artifact.test.ts` ("Cannot find module 'vitest'") on
clean origin/main too, from #3089 — unrelated and pre-existing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Apply the --disable-gpu-compositing workaround to every software capture, not just BeginFrame ones. SwiftShader's compositor re-presents stale raster for a partially invalidated layer, so successive screenshot captures accumulate copies of earlier seeks; alpha renders are forced onto the screenshot path and were the only ones left unprotected.
Refreshes the byte-strict png-sequence alpha baseline for the resulting antialiasing delta (content unchanged, min PSNR 41.3 dB).
Fixes#3049.
Both conflicts were import/export unions in the engine package, resolved by
keeping both sides:
- packages/engine/src/index.ts — main widened the urlDownloader re-export
(fetchPublicHttpsText, safeDownloadUrlIdentity, writeUrlDownloadTelemetry and
their types) while this branch added the notMediaPayload exports.
- packages/engine/src/services/audioMixer.ts — main added UrlDownloadError and
writeUrlDownloadTelemetry to the urlDownloader import; this branch added
isNotMediaPayload.
In audioMixer's prepare path both intents compose in order: main's download
telemetry and typed download failure, then the STUDIO-5433 non-media sniff
before the probe.
Three defects in the offline FX path, none of which any test could see.
**Float output silently disabled sample-accurate volume automation.** The writer
emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope
into the samples and accepts only 16-bit PCM, returning null otherwise. So
enabling any effect downgraded that track to the ffmpeg expression path — capped
at 32 straight segments, quantising a curved envelope, and on a dense one falling
back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a
limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test
asserts the baker accepts the writer's own output and actually fades it.
**Everything was folded to mono.** `prepareAudioTrack` goes out of its way to
emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo
rematrix — and this folded it, then wrote one channel. So adding a single peaking
EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed
stereo. Channels now travel as one plane each, through an OfflineAudioContext of
the same width, and come back interleaved.
**Small results decoded the wrong length.** `new Float32Array(buf.buffer)`
discards byteOffset and byteLength, and Node pools small allocations: a 400-byte
payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples
decoded as 2048 samples of unrelated memory — and the empty-result guard could
not see it. The reader has the mirror-image fix: a float data chunk on an odd
boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now
copies instead of throwing RangeError on an unaligned view.
The tail limitation is now stated rather than mis-stated: the context is exactly
as long as the input, so a reverb or delay still ringing is cut there. The old
comment claimed the opposite. How far a tail may run past a clip's end changes
the clip's length in the mix, so it is a product decision, not one to make here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A source that answers with a JSON error body still reached ffprobe and
produced `moov atom not found`. Replicate returns
`{"detail": "requested file not found"}` for a dead asset, and a gateway
in front of it can relay that body with a success status.
The sniff now treats `<`, `{`, or `[` as the opening byte of a text
document. No supported container starts with any of them, so this is the
same trade as before: three bytes instead of an allowlist that grows one
entry per payload shape observed in production.
Renamed accordingly, since the class now covers JSON as well as markup:
MARKUP_NOT_MEDIA -> NOT_MEDIA_PAYLOAD, MarkupNotMediaError ->
NotMediaPayloadError, markupPayload.ts -> notMediaPayload.ts. Registry
entries in the Lambda name map, the CDK and SAM plan lists, the Cloud Run
set, and SAFE_RENDER_ERROR_CODES move with it.
Also documents the reachability boundary on the error class: only a 2xx
response gets here. `downloadToTemp` rejects 404/410 as `http_not_found`
before writing a byte, and every ffprobe input is local because
videoFrameExtractor downloads http srcs first. So the shapes this
classifies are soft-404 and interstitial HTML, S3/CloudFront error
documents, and JSON API error bodies -- each served with a success
status. A genuine 404 surfaces as a download failure, not as this error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes#3012
## What
Fixes `HeadlessExperimental.beginFrame` hanging at 60fps and other high frame rates.
## Why
Warmup always advances Chrome in 33ms steps, but the first capture timestamp was calculated from the output fps.
At 60fps, the last warmup tick is 1947ms, while the next commit tick used to jump back to about 1067ms. Chrome sees time moving backwards and `beginFrame` can stall.
## How
Keep the old timestamps when they are already safe. If the commit tick would go backwards, move the capture baseline past warmup first.
The init commit and both producer probes now use the same timestamp helpers, so they cannot calculate different values.
## Test plan
- [x] Added tests for 24, 30, 31, 32, 33, 60, 120, 240, and 59.94fps
- [x] Added coverage for the actual session baseline, commit parameters, and both producer probe paths
- [x] Engine tests: 1383 passed, 3 skipped
- [x] Producer unit tests passed
- [x] Engine and producer typechecks and builds passed
- [ ] Manual render test
Review follow-up on the STUDIO-5433 defense.
Correctness
- The sniff ran above the documented video/audio failure split, so an
<audio> src that resolved to an HTML payload aborted the whole render
instead of degrading to duration 0. It now runs inside the same try, so
video surfaces the typed error while audio still drops out, with a
warning naming the element.
- Raw fs errors (EISDIR on a directory src, EACCES, the existsSync->open
ENOENT race, EMFILE) escaped and failed the compile with an unclassified
error carrying an unredacted temp path. The sniff is now a classifier that
never throws: an unreadable file reports "not markup" and the real probe
produces the real error.
- Elements whose duration the compiler never resolves (a data-end video, a
looping audio) skipped the sniff entirely, so the original ffprobe error
still escaped, and looping audio was reported as owner "system" after
every frame had been captured. Video is now caught in the asset preflight,
which sees every local src regardless of authored timing; audio is
classified per-element in audioMixer as source/invalid_media/owner "user",
keeping audio failures non-fatal as they already were.
- Detection is a byte-level check for a leading "<" (BOM-, whitespace- and
NUL-tolerant, looped read) instead of a <!doctype|<html|<?xml string
prefix, which missed a NUL-prefixed payload, >256B of leading whitespace,
UTF-16-encoded HTML, and a prolog-less <svg. No supported container starts
with "<", so the allowlist no longer grows per payload shape.
- finally { await fh.close() } could replace the in-flight typed error with
the close error.
Routing and privacy
- MARKUP_NOT_MEDIA is now in SAFE_RENDER_ERROR_CODES, the Lambda terminal
name map, the CDK and SAM non-retryable plan lists, and the Cloud Run
non-retryable set, and the class carries owner/retryable. Previously the
API emitted errorCode: undefined and a deterministic authoring bug burned
the full distributed retry budget.
- The message no longer carries 32 raw payload bytes or the src.
redactTelemetryString preserves host and path for HTTP srcs, so
per-tenant CDN paths reached a message the server forwards to clients.
Correlation is a sha256 element fingerprint, matching
AssetMediaTypeMismatchError.
- The message names both causes (unresolved nested-composition URL, or an
HTML/XML error page served as 200) rather than misdiagnosing an S3 403
body as an authoring bug.
Tests
- Byte-level detection is unit-tested in engine: markup shapes, BOMs,
UTF-16, nine container signatures, unreadable inputs.
- Replaced the tautological assertions. The old checks for "html" in and
"moov" absent from a fixed message template could not fail for any input.
- New coverage for audio degradation, the audioMixer classification, the
preflight video/image/audio split, and the API error metadata.
- The sibling htmlCompiler.mediaType failure was a vitest-under-bun runner
mismatch, not a missing ffmpeg binary. It passes, including the 4-wide
probe-semaphore invariant the sniff now runs inside.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reads `data-fx-chain` off an audio element and runs the chain over the trimmed
WAV before volume automation is baked in — effects should see the raw signal,
and the envelope belongs on their output.
The processing happens in an OfflineAudioContext inside the headless browser
the engine already drives, running the same graph builders the studio previews
with. That is the point of the approach: one implementation per effect, so the
render agreeing with the preview is a property of the architecture rather than
a tolerance to police. Reimplementing each effect as an FFmpeg filter would
mean two implementations to keep in step, and for the dynamics processors and
modulated delays there is no filter that behaves the same way.
`build:audio-fx-runtime` bundles the graph builders into an injectable IIFE,
following the same pattern as the existing runtime artifacts, so the browser
runs exactly the code the studio does.
The page loads from a file:// URL rather than about:blank because AudioWorklet
is only exposed in a secure context — the compressor, limiter, gate and
bitcrush processors would otherwise fail to register with an opaque error.
file:// qualifies and needs no listening socket.
The chain is serialised into the attribute the way colour grading carries its
config, so there is no side-car file to resolve or lose.
An FX failure is fatal for the whole mix rather than a per-track soft failure.
Every other audio failure mode degrades gracefully — the track drops, siblings
continue — but substituting the dry signal for a processed one ships a render
that sounds plausible and is not what the author set up. Since the per-element
work races under Promise.all, an internal AbortController chained off the
caller's signal aborts in-flight siblings before workDir is removed.
A probe that could not run is no evidence about the GPU, so pointing the
operator at GPU passthrough hid broken Chrome installs behind a phantom
problem. Carry a cause off the probe and emit the matching remediation.
Also un-exports buildUnverifiedHardwareGpuWarning (Fallow: engine test
files are not audit entry points, so a test-only import would not have
counted as a consumer) and covers the non-linux branch via the spy.
Chrome's hardware GL args are advisory: with no usable GPU it silently
falls back to software WebGL and the capture runs at CPU speed. Run the
existing WebGL probe for explicit hardware mode too and warn loudly with
the platform's remediation. The requested mode is still honoured.
Fixes#2967