* test: ffmpeg failures name how ffmpeg ended and keep its last line
* fix(producer): an ffmpeg failure names how ffmpeg ended and keeps its last line
* refactor(producer): say why in names, not comments
* fix(engine): renders under WSL reach the GPU on Chrome 150 and later
Part of #5548.
* fix(engine): shorten the WSL browser env comment
* fix(engine): cite the Chromium bug in one line and allow the fallback test's removal
* fix(engine): drop WAYLAND_DISPLAY under WSL only when that is what reaches the GPU
* fix(engine): restore the WSL note from before the full-Chrome fallback
* test(engine): give the second worker's page load room on a busy Windows runner
* fix(engine): keep the Chromium bug comment under the line limit
* fix(engine): renders under WSL use the GPU through full Chrome when the headless shell cannot
Part of #5548.
* fix(engine): a missing or unreadable full Chrome cache never stops a WSL render
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8
VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.
The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused
v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): stop multi-worker renders hanging on media-heavy pages on Windows and macOS
Off Linux, parallel workers shared one pooled browser. Pages in one browser
share Chrome's six connections per host to the file server, so a page with
many videos and audio clips held all six and the next worker's page.goto
waited until it timed out. Every parallel worker now gets its own browser.
Part of #5548
* refactor(engine): shorten the worker browser comment
* fix(engine): say why WSL renders fall back to software and how to get the GPU
Under WSL the GPU check often falls back to SwiftShader and the only output
was the probe line, with Docker passthrough advice for --browser-gpu. Print
one note that says what was checked (can this user open the render node),
the exact fix, and the native Windows alternative. The CLI's WSL detector
moves to the engine so both use one.
Part of #5548
* fix(engine): keep WSL GPU notes true to the probe result and render mode
The WSL note now prints only when auto mode really selects software, and
says whether the GPU check found no GPU or could not finish. Explicit
--browser-gpu keeps its unverified and Chrome-install advice and adds the
WSL checks. The render-node check verifies group membership before
suggesting usermod.
#4564 routes SDR colour through `format=gbrp`. Without accurate_rnd,
swscale's yuv-to-gbrp conversion leaves the last 8 columns black when the
width is 16n + 8 (1000, 1080, 1112...), so every 1080x1920 portrait MP4
since 0.8.81 has had an 8 px black strip on the right. The same step in
SDR jpg frame extraction blanked the right edge of footage at those widths.
Set accurate_rnd on the yuv-to-gbrp scale in both chains. It is also the
more accurate rounding: on a testsrc2 frame, worst luma error against a
float reference drops from 1.85 to 0.90 codes.
Fixes#5551
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): bound checks of non-finishing compositions
* fix(cli): account for cold startup in deadline witness
* fix(cli): wait for profile owners before test cleanup
* test(cli): separate portable deadline and Linux process witnesses
* test(cli): preserve per-launch browser profile ownership
* fix(cli): start check deadline after browser readiness
* fix(engine): encode and tag ProRes mov output as BT.709
* test(engine): read the ProRes colour tags from the first frame
* test(engine): end the ProRes tag probe's options with --
The offline render now pads the source with silence for the whole render
(clip + chain tail + latency) instead of only the true-peak lookahead, so the
pitch shifter's ring drains instead of cutting off about 90 ms early.
A bitcrush node built part-way into a clip seeded its hold counter but started
its held value at 0, playing up to samples-1 frames of silence on a preview
seek. The held value is now primed from the first input sample; render windows
still line up with a whole render.
* fix(core): a scene rendered on its own takes its master chain from its own root
The standalone wrapper for a scene file reused the project index's root
attributes, so the index's data-fx-chain became the master bus. The wrapper
now carries the scene root's own chain, or none when the scene declares none,
as the audio-effects reference describes.
— Jerrai
* fix(engine): run long audio chains window by window instead of reading the file whole
applyAudioFxChain read the whole input WAV into memory, which Node refuses past
2 GiB (a stereo float mix beyond about 93 minutes), and the browser page then
held the track twice. Inputs above 512 MiB of samples are now read, rendered
and written in 5 minute windows, each with 10 s of preceding and 1 s of
following audio rendered and discarded so stateful effects arrive warm. The
output header is patched at the end. Inputs below the threshold keep the
existing whole-file path.
Parameter automation is clocked from the file start and is not windowed. An
input past 2 GiB that carries automation now fails with an AudioFxRenderError
naming that, instead of Node's raw file-size error.
— Jerrai
* chore(engine): trim the browser-lease comment to its live invariant
— Jerrai
* fix(core): a scene mounted as the index's own root takes its own master chain
When the index's top-level composition is itself the scene's mount, the standalone
extraction returned before the scene's chain was applied, so the index's chain still won.
— Jerrai
* fix(engine): windowed audio renders follow the file's clock and warm up for the whole chain
- Pass each window's start time into the page and on to buildFxChain, and seed the
pitch-shift worklet's tap phase from it, so chorus, phaser and pitch shift continue
across a window seam instead of restarting from time zero.
- Size the pre-roll from the chain (delay repeats, reverb room, dynamics release, to -60 dB),
never below the configured floor and never above a cap.
- Read a WAV data size of 0 or 0xFFFFFFFF as running to the end of the file, so audio past
4 GiB is no longer dropped.
- Close the input descriptor when the output cannot be opened, and name the real cause when
an input over 2 GiB cannot be streamed.
- Cover reverb, a slow-release compressor, chorus and pitch shift in the windowed-vs-whole
test, with an output-length check.
— Jerrai
* fix(engine): start each audio window on a render quantum and test the seams against a hard chain
Windows that begin off a 128-frame block see worklet blocks shifted against a whole render, which
left spikes in the tail after the last input sample. Align the window start, and make the
windowed-vs-whole test cover modulation, delay, reverb and a slow-release compressor on windows
whose start is not zero, failing above 0.001 where it previously tolerated 0.02.
— Jerrai
* fix(engine): keep a limiter's lookahead, seed bitcrush holds, close-check without /proc
- Pad the render source with the limiter's lookahead of silence so the last samples of a clip
leave the lookahead line instead of being dropped.
- Seed the bitcrush hold counter from the window's start so holds that do not divide the
render quantum stay in step across seams, and cover bitcrush in the windowed-vs-whole test.
- Check that the input descriptor is closed through the fs mock instead of /proc/self/fd,
which only exists on Linux.
— Jerrai
* test(engine): pick a bitcrush hold the window starts do not divide
The earlier hold of 3 divided both window starts, so an unseeded counter passed.
— Jerrai
* fix(engine): warn at the pre-roll cap, share tail formulas, read a zero data size alike
- Warn when a chain's memory exceeds the pre-roll cap, so cut repeats at window seams
are not silent.
- Export the delay and reverb tail formulas from audio-fx-tail and use them for the
window warm-up too.
- Both WAV readers now treat a declared data size of 0 or 0xFFFFFFFF as the rest of
the file.
— Jerrai
* feat(audio): composition master bus (data-fx-chain on the root)
A clip can belong to only one audio group, so there was no way to put one
effect, such as a limiter, across the whole mix. The data-fx-chain on the
top-level composition root now runs once over the sum of every clip and every
group, after each group's own chain and fader. No new attribute: a sub-composition
root's data-fx-chain was never applied to anything and still is not.
Render: when a master chain exists, mixAudioTracks writes a float WAV instead
of AAC, applyAudioFxChain runs the chain over it, and ffmpeg encodes AAC last,
so the chain reads unclipped samples. An unreadable chain is a mix failure, as
for a group. Preview: the transport splices the chain between the clips' sum and
the monitor, re-anchored per play generation and watched for edits.
Only the top-level root is read. No master automation: the root's data-automation
is not read as the bus's. No lint rule: data-fx-chain is valid on clips, groups
and roots, and the one place it does nothing (a sub-composition root) has always
been silent.
— Jerrai
* fix(core): preview routes every audible video through the master bus
A root data-fx-chain is attached only inside the Web Audio graph, but a plainly
audible <video> joined that graph only for its own chain, group, automation or a
volume over 1. So preview limited the music and left the talking head dry, and a
composition whose only audio is video got no master chain at all, while export
sums every audible element into the master. A master chain on the top-level
composition root is now a reason to join, the same as a video's own chain.
Elements under a root with no chain keep native output. Sub-composition roots
are still ignored.
— Jerrai
* fix(audio): master bus review round 2 (temp dir, tests, docs, Studio meter)
- audioMixer: a fatal FX error in the master pass or a group bus left the
work directory (the full float sum, about 1.4 GB an hour) on disk. The
final mix now runs in try/finally and the group rethrow removes it first.
- Tests: the "once over the sum" case now uses a limiter then a gain, so a
per-clip chain (-10.4) and a doubled one (-18) both fail it; a new case
proves the sum stays float until the chain reads it; the fader case uses
a group volume of 0.5 so the fader's position is exercised; the work-dir
removal is covered for both paths.
- Docs: "Three rules exist" (the master-bus lint rule is gone), a bad master
chain stops the render with audio_processing_failed, a sub-composition
file opened on its own has its root chain as the master bus, and a seek
clears a group or master limiter.
- Studio: TimelineElement.masterChain carries the root's data-fx-chain, so
the program meter shows for a video-only composition whose only audio
routing is the master chain, as preview now plays it.
- Comment blocks over four lines cut; webAudioTransport.setRate split so
its CRAP score is under the Fallow threshold.
— Jerrai
* fix(audio): master chain reaches tracks the render adds; honest plain-limiter copy
- Master chain: it was carried on each parsed AudioElement, so a track the
producer adds itself (appendAutoDetectedVideoAudio, the browser-discovered
audio in probeStage) never had it, and a composition whose only audio was
such a track exported with no master chain, no ceiling and no warning. It is
now a composition property: parseMasterFxChain(html) at compile, carried on
CompiledComposition and CompositionMetadata, passed to processCompositionAudio
by runAudioStage (in-process and distributed plan). AudioElement.masterFxChain
is removed so there is one source. Each forward (compile stage, local
orchestrator, distributed plan) has a test that fails when it is dropped.
- Plain limiter: it follows the level with no lookahead and Level after is
applied past the ceiling, so it cannot promise "nothing gets past" or "never
clips". Studio copy (description, panel text, knob label, summary) and the
hyperframes-audio skill now say a fast or inter-sample peak can pass it and
point to the true-peak limiter for a delivery limit. A copy test guards it.
— Jerrai
* fix(producer): keep master bus comments within the ratchet; run the plan test with the ffmpeg tests
Drop three one-line field comments that raised their files' comment share,
and list plan.masterBus.test.ts with the integration tests because plan()
reads the ffmpeg version.
— Jerrai
* chore(ci): repin the producer test-classification source in the reachability map
The reachability map pins the hash of test-classification.mjs, which now
lists one more integration test. The map's producer lane command already
covers every test under src, in both the unit and integration matrix lanes.
— Jerrai
* fix(engine): exports place sound and picture right when a source's audio and video start apart
* fix(engine): a late video stream plays to its end and distributed and HDR renders hide its lead
* test(engine): pin a late video stream's trimmed start, held tail and HDR trim
* fix(engine): a capture stall names the step Chrome stopped answering in
* fix(engine): a stall on a motion-blurred frame names the seek or injection it hung in
* fix(engine): exports no longer fail on a video clip outside the composition's length
A clip that starts at or after the composition's end (or ends before 0) is
never on screen, so extraction skips it. Its timeline window stayed as
authored, so the frame-coverage gate still expected its full length and
aborted the render with "captured 0 of expected N frames". Such a clip now
leaves extraction with an empty window, the same way a partly visible clip
is already trimmed to its visible part, so it owes no frames.
* fix(producer): drop clips outside the timeline before extraction, cloud renders included
The previous commit marked such a clip with end = start, but the window
resolver reads a zero-length slot as "use the source's natural length", and
the distributed plan still required an extracted entry for every declared
clip. Instead, the extract stage (shared by in-process and distributed
renders) removes clips that lie entirely outside [0, duration) from the
composition before probing and extraction, using one engine predicate that
the extractor's own early skip also uses, so a clip ending before 0 is no
longer downloaded or probed either.
* fix(engine): keep natural-length clips when dropping clips outside the timeline
A clip with no authored end carries end 0 (script-created media, or a src
set from a script), which the window resolver reads as "natural source
length". The outside-the-timeline check treated it as ending at 0 and dropped
it, so the clip froze on its first frame. Only an authored end (after the
start) at or before 0 now counts as before the timeline.
* fix(producer): only drop clips that start at or after the composition's end
A clip with no authored length carries end 0, the same value as an authored
end at 0, so "ends at or before 0" cannot be told apart from "natural
length" and a script-created clip with a negative start was dropped and
froze. Clips entirely before 0 keep main's behaviour; the filter now only
drops clips starting at or after the end, which never reads the end.
* test(producer): pin that a clip starting just before the end is still extracted
* fix(core): a sub-composition timeline with a non-function duration no longer aborts the runtime
* fix(core): every registered-timeline duration read survives a partial or throwing entry
* fix(core): a runtime that fails to start reports its real error instead of a half-built page
* fix(core): a timeline post that throws after start-up is named too, and a later good post clears it
* fix(cli): check skips a custom clock whose time() returns null instead of reading it as stuck at 0
* fix(core): a recorded runtime start-up error stays set instead of being cleared by a later post
* fix(core): the runtime failure message says what to fix
* fix(core): the runtime failure is recorded even when the thrown value cannot be printed
* chore(core): format and regenerate the position-edits bundle for the startup error recorder
* fix(cli): keyframes and validate close the browser when the runtime fails to start
* fix(studio): text in Studio reads without em or en dashes
* fix(studio): the dash check also catches HTML entities, and peak reads Unavailable
* test(studio): the dash check test covers every entity form
* fix(studio): messages Studio shows from lint, sdk, core and the renderer drop dashes too
* fix(producer): the gate check stops early on the engine's new fallback wording
* feat(core): add a lookahead true-peak limiter FX (truepeak)
The existing limiter is an envelope follower with no lookahead and no
inter-sample detection, so a -14 dB ceiling can leave the true peak several dB
over. truepeak estimates the true peak on a 4x polyphase interpolator, takes the
minimum required gain over a lookahead window and smooths it with a boxcar of
the same length, so the gain is already down when the peak leaves the delay
line. Release is exponential and can only lower the gain further; channels
share one gain.
The offline render lengthens the OfflineAudioContext by the chain's latency and
drops the lead-in, so a limited clip stays sample-aligned with its source.
Preview plays the lookahead (about 3.3 ms at the default).
— Jerrai
* docs(audio): truepeak ceiling is a 4x estimate; measure it with an ideal interpolator
The unit test's meter was a 96-tap Kaiser sinc, which rolls off near Nyquist and
reads full-band noise about 0.5 dB low, so the PR reported a 0.3 dB broadband
overshoot. Against an ideal 16x interpolation (zero-padded FFT, cross-checked with
a 16x soxr resample) hot full-band white noise ends 1.2 to 1.7 dB over the ceiling
(median 1.4 over 28 runs of 10 to 30 s). ffmpeg's ebur128 true peak is also a 4x
meter and reads the same files 0.1 dB over.
- The test meter is now the FFT interpolation. Tolerances are what it measures on
each fixed signal: tones 0.01 dB (they land on the ceiling), the dense program
0.5 dB (reads 0.445, the old 0.4 passed only because the meter read low), and a
new full-band noise case at -1, -6 and -14 dBTP with 1.35 dB (reads 1.09 to 1.31).
- audio-effects.mdx and the hyperframes-audio skill say the ceiling is a 4x
estimate, give the measured overshoot and recommend a 2 dB margin under a hard
delivery limit.
- skills-manifest.json regenerated.
— Jerrai
* fix(core): truepeak UI copy no longer promises a hard ceiling
The registry description and the rack summary said the true peak is never above
the ceiling. The limiter holds a 4x estimate, and full-band noise can land up to
about 1.7 dB above it, as the docs now say. The summary reads "held near", the
description names the estimate and the overshoot, and the rack blurb says "holds
... to a ceiling". No skill file changed, so the skills manifest is unchanged.
— Jerrai
* fix(core): Studio truepeak copy states the 4x-estimate margin
The ceiling control was labelled "Never exceed" and the module said the mix
"must not clip", so a Studio user never saw that dense full-band content can end
up to about 1.7 dB above the ceiling. The label is now "Target ceiling" with a
hint that names the 4x estimate and the headroom to leave, the blurb says
"toward a target ceiling", the rack summary says peaks are pulled down toward the
value, and the registry hint no longer says "highest true peak allowed". The
audio skill's "hard dBTP ceiling" wording is gone and the skills manifest is
regenerated. A copy test keeps the promise out.
— Jerrai
* fix(core): truepeak clears on a seek, and the review nits (fallow, comments)
- Unexport TRUE_PEAK_OVERSAMPLE, TRUE_PEAK_DETECTOR_DELAY and
truePeakLookaheadSamples: nothing imports them (Fallow audit).
- A persistent group bus keeps its chain across a seek, so the limiter's
delay line and held gain carried over. The worklet takes __hfReset, the
graph exposes FxChainHandle.reset(), and reanchor() calls it.
- The processor-loading harness is shared by the truepeak and worklet tests
(it was a 12-line clone); the test FFT no longer repeats audioCarve's
bit-reversal; two test bodies are under the CRAP threshold.
- Comment blocks cut to four lines or fewer (Comments check).
- Docs state the preview delay in ms.
— Jerrai
* fix(engine): copy border-width/style/color onto the replacement video frame
injectVideoFramesBatch substitutes each <video> with a sibling <img> holding
an extracted still frame, copying an allow-listed set of CSS properties from
the video's computed style onto the img. border-width/border-style/
border-color were absent from that allow-list, so any border authored
directly on a <video> never reached the replacement image and disappeared
from render/snapshot output. border-radius and clip-path were already on the
list and already clip a replaced element's content correctly without
needing overflow:hidden, confirmed by a real-Chromium test that exercises
the actual capture path and reads real screenshot pixels.
Also corrected a stale comment in frameCapture.ts claiming
detectCssEffectRisk can return "clip-path" as a risk value -- it can't; that
string only ever comes from a separate, animation-only at-risk-props gate
further down the same file.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* test(producer): regenerate style-9-prod golden for the border fix
style-9-prod's video has an authored 8px border + 16px border-radius
(design_review.md calls out the "floating" look) that never rendered before
this fix, so its committed golden baked in the pre-fix (borderless) output.
Regenerated via `tsx src/regression-harness.ts style-9-prod --update`;
verified the diff is exactly the border/radius becoming visible (extracted
frames before/after) and the fixture passes its own visual/audio checks
again (100/100 checkpoints, correlation 1.000).
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): measure the video's box before styling its replacement frame
injectVideoFramesBatch measured the video's used box (offsetLeft/Top/
Width/Height) after creating, inserting, and styling the replacement
<img> sibling -- once that copy loop started carrying border-width
(added for the border-visibility fix), the freshly-inserted, still
in-flow, now-bordered <img> competed for space in a flex row before
the video's own box was ever read. In a flex-centered layout (video
width:100%, sharing a row with the img sibling), that shrank the
measured video box by the exact width of the border, so the img ended
up sized from the video's post-shrink box instead of its authored one.
Moved the measurement to before the <img> is created/inserted/styled
at all, and set img.style.boxSizing = "border-box" explicitly after
the style-copy loop, since offsetWidth/offsetHeight (and the
getBoundingClientRect fallback) are always a border-box measurement
regardless of what box-sizing value that loop copies from the video's
own computed style.
Added a flex-centered regression test asserting the replacement img's
box stays identical to the video's box; confirmed it fails with the
old measurement order (a 16px border shrinks a 500px box to 484px) and
passes with the fix. Regenerated the one fixture (style-9-prod) whose
committed golden had baked in the shrunk-frame geometry.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* refactor(engine): consolidate video-box measurement into one object
Follow-up to c88890357: group the four measurement locals into a
single videoBox object instead of four loose consts, and trim the
surrounding comments down to their load-bearing content. No behavior
change -- same measurement order, same fallback conditions, same
applied style values; re-verified red (reverting the measurement order
still reproduces the 500px -> 484px shrink) and green.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* refactor(engine): trim the border-fix comments to the invariant
* test(producer): keep main's style-9-prod compiled.html; timings are unchanged
* test(producer): repin the classification source hash for test reachability
* fix(core): a color-graded bordered source keeps its canvas inside its border box
* fix(core): a graded source with an empty content box still shows its border
* fix(core): a border-only graded canvas stays hidden until the source has drawn
* fix(core): border-only graded canvas shows only while grading hides the source
* test(core): pin the border-only canvas hide after a context loss
* fix(core): grading canvas carries no border, so a graded source never paints it twice
* fix(core): graded injected frame drops its border once grading draws over it
* fix(core): a released grading entry gives the injected frame its border back
* fix(core): a released injected frame takes the video's current border style
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* perf(engine): re-renders reuse the extracted frames of remote videos
A remote video is downloaded into the render's own work dir, so its path and
mtime are new on every render. The extraction cache keyed on both, so a
re-render of a composition with a remote video never hit the cache and ran
ffmpeg over the whole clip again. Key a downloaded source on the SHA-256 of
its bytes instead. Keys of local sources are unchanged.
* fix(engine): key the video copies a render writes itself by content
The producer downloads a remote video at compile time and copies it into the
render's compiled dir, so the extractor never sees the https src. Key any
source inside the compiled dir or the extractor's own output dir by content,
and skip the cache for one that cannot be read.
* perf(engine): hash a render-written video only when its cache lookup runs
Distributed planning copies the whole project into the compiled dir and defers
long video extractions, which never look up the cache. Hash render-written
sources just before the lookup, once per file, so deferred ranges and repeated
sources are not hashed.
* fix(engine): cache only downloaded media by content, not distributed project copies
A distributed plan copies the whole project into the compiled dir, so keying
everything there by content would hash every project video on the planner.
Content-key the extractor's own downloads and the dirs the caller names; the
producer names its remote media dir. Hashing stops on abort, and a source
with no hash skips the cache.
* style(engine): keep the extraction cache's comment share at its base
* test(engine): pin no hashing with the cache off and the producer's remote-media dir
* test(engine): run the cache-off case through the shared render helper
* fix(engine): long exports with many clips no longer stall at 90% or hide a timeout
* fix(engine): stop queued frame extractions on cancel and time out distributed assembly
* refactor(engine): simplify the extraction slot check and its concurrency test
* test(engine): prove a cancelled extraction leaves the queue, skip the shell shim on Windows
* test(engine): wait for the held extraction's own record before cancelling the queued one
* fix(cli): snapshot, check, layout and thumbnails honour software WebGPU like render
* test(cli): stub the local WebGPU helper in the studio server tests
* fix(engine): webgpu compositions are recognised after the bundled runtime script
* fix(engine): a hung GPU probe falls back to software in 15 s instead of 120 s
* fix(engine): the GPU probe keeps the configured launch timeout and bounds only its page
* build(parsers): declare the html-attribute-spans subpath in its descriptor
* build(gcp-cloud-run): the BeginFrame contract stage copies parsers, which the engine now imports
* test(cli): the screenshot launch pin follows the shared local WebGPU helper
* fix(engine): the GPU probe's wait for Chrome's first page falls inside its 15 s limit
* test(engine): real-ffmpeg HLS tests get the 30 s budget other real-ffmpeg tests use
* test(engine): real-ffmpeg fixture cleanup gets the 30 s its setup already has
* fix(engine): recover audio duration from packet timestamps before clamping
An authored `data-duration` on an <audio> element could be silently
shortened when a source file's container-summary duration (mvhd/tkhd/mdhd)
undercounts the real audio length while the packet stream itself is
intact. Add `probeAudioDurationFromPackets()`, which re-derives the true
duration from the last packet's own pts_time + duration_time — figures
ffprobe computes from the stream's own timestamps, not the container's
summary header — and wire it into the compiler's existing clamp check as a
rescue attempt before accepting a shorter value. Works regardless of
codec/profile; verified against real HE-AAC/SBR fixtures and a
binary-patched "lying container" MP4 built for the regression test.
The rescue only fires for candidates the existing clamp predicate already
flagged (no added cost on the common path), runs in parallel across
candidates rather than serially, and requires the recovered duration to
exceed the existing clamp epsilon (absorbing the few milliseconds of
priming-delay drift raw packet timestamps carry relative to an
edit-list-corrected container duration) before overriding the shorter
value.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): move audio duration correction into extractAudioMetadata
The previous approach compared the compiler's own packet-scan probe
against the already-corrected duration returned by extractAudioMetadata,
which meant the correction only applied to one caller and the regression
fixture happened to exercise a pre-existing AAC-LC-specific refinement
instead of the new code path.
Move the fix to the single canonical place duration is derived: replace
the old AAC-LC-only packet-count refinement (a profile allowlist assuming
a fixed 1024-samples-per-frame constant) with a codec-agnostic mechanism
that decodes the final audio frame and reads its own timestamp plus real
sample count. This has no per-codec assumption, so it applies uniformly
regardless of codec or profile, and every caller of extractAudioMetadata
benefits automatically. The compiler's own clamp logic is now unchanged
from its pre-existing form.
Verified against a real, previously-untouched case: a FLAC-in-MP4 file
(a codec the old AAC-LC-only path never corrected) with its mvhd/tkhd/mdhd
duration atoms binary-patched to half their true value while every real
frame stays intact. Confirmed this reproduces on the prior code (by
literally substituting its actual content) and is fixed by the new code.
The decode probe reads CSV output (each line stands alone) rather than
JSON (a single top-level array), and retains only the output tail under a
small cap — a lying container's true duration can be arbitrarily larger
than its declared one, and JSON truncated from the front isn't valid JSON,
so an unbounded scan of a long file silently fell back to the wrong
duration once ffprobe's stdout cap was hit.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* test(producer): update style-7-prod golden for corrected AAC duration probe
The deleted AAC-LC packet-count refinement assumed every demuxed packet
decodes to exactly 1024 samples. For this suite's real audio fixture
(a 48kHz stereo AAC-LC MP4), that assumption overcounts: ffprobe reports
753 demuxed packets, but only 752 of them actually decode into a real
audio frame (verified via raw PCM sample count: 770048 samples / 48000Hz
= 16.0426666...s). The old mechanism's 753 * 1024 / 48000 = 16.064s was
therefore wrong; the new decode-based probe correctly derives ~16.043s
from the real final frame.
Regression-shards caught this as a compilation mismatch on the previous
commit. Verified the corrected golden against the real source: full
compilation, visual (0 failed frames), and audio (correlation 0.966)
checks all pass locally with the actual LFS-hosted golden video, and a
standalone compileForRender() check confirms all other audio-bearing
fixture suites in the repo are unaffected.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* perf(engine): seek the audio decode probe instead of scanning the whole file
probeDecodedAudioDuration ran a full, single-threaded decode of the
entire audio stream on every extractAudioMetadata call (once per audio
element in the HTML compiler, plus audio mixing/padding/encoding
steps). On real 30-minute files this took over a second, and could
approach ffprobe's 30s deadline on multi-hour files while holding a
shared probe slot the whole time.
Add -read_intervals so the probe seeks to ~1 second before the
container's already-known claimed duration and decodes only that
tail. This is safe because the correction only ever raises the
duration: an honest or over-claiming container's seek lands at or past
the real end and finds nothing extra, while an under-claiming
container (the only case this exists to fix) still has real frames
past the seek point. If the tail read finds nothing at all, fall back
to a full unseeked decode before giving up, protecting files whose
seek index is unreliable (an index-less or non-seekably-sourced
container) from silently missing a real correction — mirroring the
existing seek-then-fallback pattern already used by the sibling
final-video-frame probe in this file.
Also replaces the real-file regression test's runtime ffmpeg-spawning
fixture generation with two small, committed binary fixtures (an
honest FLAC-in-MP4 file and a copy with its container duration atoms
halved) plus the ffprobe-static package, so this test's real-file
proof actually runs in CI instead of being silently skipped when no
system ffmpeg is on PATH.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* refactor(engine): trim the audio duration probe comments to the invariant
* test(engine): cache tests queue the decode probe and count metadata probes only
* test(engine): probe for ffprobe with a literal argv so the argv contract classifies it
* test(producer): regenerate the speed-ramp golden for the single-encode mix
The audio probe now reads the mixed AAC as exactly 4.000 s instead of the
packet-count 4.032 s (which counted the priming frame), so the final mix
is stream-copied instead of re-encoded once more. Same waveform (sample
correlation 0.9996 vs the old golden), 3.1 dB less true-peak attenuation;
video frames are byte-identical.
* fix(engine): measure decoded audio from the stream start and trust it for raw ADTS
The decoded last-frame end is an absolute timestamp, so subtract the audio stream's start time
(format start as fallback) before comparing it with the start-relative container duration, and
start the tail read at start + duration - 1 s. MPEG-TS audio (1.4 s default start) and offset
MP4s were over-extended by their start time.
Raw ADTS (format aac) has no duration field; ffprobe estimates it from the opening bitrate and
overclaims after a quiet opening. Its decoded timestamps are sample-counted, so for raw ADTS the
decoded end now wins in both directions beyond the margin. Other containers stay upward-only so
an MP4 edit list is never extended.
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* fix(cli): name the unzip errors when every Windows unzip tool fails
* fix(cli): a cached empty or folder Chrome counts as missing, not found
* fix(engine): skip an empty cached Chrome the same way the CLI does
* fix(engine): keep scanning the Chrome cache past a build it cannot read
* test(engine): a folder at the cached Chrome path is skipped
* perf(engine): wait for a missing composition timeline once per render, not once per browser
A composition host that never registers a timeline (CSS, canvas or rAF animation
without data-no-timeline) made every capture session of a render wait the full
player-ready timeout: calibration, then the workers, then each static-frame check
page. The first session that times out now records the host ids in a memo shared
by the render's sessions; later sessions report the same timeout warning without
waiting again.
* perf(engine): the static-frame check page stops on a failed script and chunks share one memo
The verification page now gets the session's failed-script list, so it bails after
the short grace like the main page instead of waiting the full timeout. Distributed
chunks share one timeline-wait memo across their sessions. The poll takes an
options object instead of eight positional parameters.
* fix(engine): replay only a clean timeline timeout so a script failure still fails the render
A session that skipped the wait classified it at once, before a late page error
could arrive, so a render that used to stop on a script failure could ship. The
memo is now written only by a wait that ran its full length with no page errors
and no failed script loads; a replaying session that already sees a failed script
load reports a script failure.
* perf(producer): extract distributed video frames in each chunk
Plan v2 no longer extracts long constant-frame-rate videos. It ships
each source once and every chunk extracts only the frames it shows, so
Plan time stops growing with video length (#5260).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(aws-lambda): plan no longer bounds video length
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(producer): pin deferred compiled sources to every chunk
Each chunk's page loads every <video> src during session init and a
missing file fails the chunk, so a deferred source under compiled/ must
keep reaching every chunk, as it does on main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): extract long video ranges in segments and keep plan uploads flowing
A distributed render's plan step extracted a whole video in one ffmpeg process,
so a long clip hit ffmpegProcessTimeout (300 s) however much of Lambda's
15-minute limit was left, and every frame then uploaded in batches of 16 that
each waited for their slowest blob.
Ranges longer than two minutes now extract as two-minute segments that run side
by side, each numbered after the last. They sample the same absolute times as a
single pass, so the frames are byte-identical (checked on a 26-minute, 37,440
frame clip). Plan v2 publishing keeps 32 uploads in flight instead of batching.
On a 10 GB Lambda the 26-minute clip's plan step fell from 395 s to 200 s.
The Lambda docs now state the plan-step length bound.
Fixes#5260
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): split extraction on an integer frame count
The split compared and subtracted float seconds, so 200.3 - 80.3 became
120.00000000000001 s: a second segment got `-t 1.42e-14`, which ffmpeg
rejects, failing an ordinary two-minute range. The frame count now comes from
extractionFrameCountForDuration, the rule a single pass follows, and every
segment takes an exact frame count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): automatic worker count never exceeds what the V8 heap can feed
A 24 GB Mac exporting a 43 s portrait project with 22 videos picked 5 capture workers against a 4192 MB heap that feeds about 4, and the render died at 62% with JavaScript heap out of memory. The heap budget was only advisory; it now caps the automatic count (bound 'heap'). An explicit --workers N is untouched.
* test(engine): heap-cap test restores its mocks even when an assertion fails
Review findings on #5149: the os and v8 mocks were restored only if every assertion passed, and two comments still called the heap cap advisory.
* docs(engine): heap budget comments say it caps auto sizing
* fix(engine): motion blur works over video by holding each video frame across samples
Each output frame's injected video frame is now held for every sub-frame
sample and probe of that frame, so footage stays as shot and only the
layers over it blur. The refusal for sessions with a before-capture hook
is removed.
Fixes#5144
* fix(engine): trim held-video comments, pin the new test-classification hash
* fix(engine): a blurred frame at a sub-composition cut keeps its video
Each sample now decides which videos show at its own time, like every other
layer, and only a video on screen at the output frame's time keeps that
frame's picture. Before, the visible set was taken at the frame's time while
sub-composition hosts were hidden at the sample's time, so the samples just
before a cut showed no video and the cut frame dipped toward black. The
before-capture hook takes the held time as an optional third argument, and
the early injection is gone: the GPU re-render now always runs at the page's
own time.
* style(engine): one doc comment for the before-capture hook
* fix(engine): a video animating its own style stays in step when blurred
The injector copies a video's own style only when its frame changes, so the
first injection of each blurred frame now runs at the frame's own time
before the samples. A video sliding by its own transform no longer sits a
quarter frame behind with blur on.
* test(producer): a two-line header for the motion blur over video test
* test(producer): reach the cut fixture by path, not a URL fallow reads as an import
* feat(render): report encoding and assembling progress
* fix(render): close every progress stage and fail a render whose progress callback throws
* fix(render): send a repeated progress update once
* fix(render): keep Docker terminal output as before and drop the unneeded callback ending
* fix(render): keep one failure stage code per stage whatever its frame counts
* feat(core): support deterministic async frame sources
* fix(runtime): retain queued draws and disable unsafe dedup
* docs(core): mark frame-source hosts with data-no-timeline
A composition that follows the frame-source contract registers no GSAP
timeline, so lint reported missing_timeline_registry and every render
waited 45 s for timeline registration (17.6 s -> 1 min 47.9 s for the
14 s imported film sample). Document the existing data-no-timeline
opt-out for the host and a timeline-less root.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(runtime): avoid transport redraws after render seeks
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(engine): render WebGPU compositions on SwiftShader behind an opt-in
PRODUCER_ALLOW_SOFTWARE_WEBGPU=true lets a host without a GPU render a
data-requires-webgpu composition instead of refusing it. SwiftShader can
draw canvas WebGPU only through its Vulkan backend with GPU compositing
on, so an opted-in software launch adds the Vulkan feature and
--use-vulkan=swiftshader, keeps compositing on, and the adapter check
accepts the fallback adapter. Off by default; every other launch is
unchanged.
* test(engine): compare the opted-out launch without a cast
* test(engine): pin which adapter a capture session accepts for WebGPU
* test(engine): assert the adapter check's arguments with the call matcher
* docs(engine): fit the software WebGPU note in three lines
* fix(engine): a render fails when a composition script throws before its timeline binds
* fix(engine): an unrelated page error no longer fails a render whose timeline registers late
* fix(engine): only the composition's own scripts fail a render, local or distributed
* style(engine): format the script failure change
* fix(engine): a third-party browser rejection no longer fails a render
* refactor(engine): one page URL for navigation and error attribution
* fix(engine): a rejection after a hash change or in an iframe no longer fails a render
* fix(engine): a widget's error no longer fails a render once its script is inlined
* fix(core): name injected and inlined scripts without building code from library input
* fix(engine): a third-party embed's eval or inserted script no longer fails a render
* fix(producer): a widget's inline handler or a sub-composition no longer misreports a render
* refactor(engine): read each script response once and keep comments within 120 characters
* fix(engine): a scene that throws after registering its timeline no longer cuts a render short
* fix(engine): a composition script failure names the error and its real cause
* fix(engine): a failed VFX chain fails the render at once instead of shipping a missing layer
* fix(core): a VFX layer that is empty on one frame renders empty instead of failing or going stale
* fix(core): a VFX layer that grows from zero size renders instead of staying missing
* fix(core): a VFX layer hidden or late while empty renders once it has a size
* test(producer): cover a VFX layer that is 0×0 under a host that starts later
* test(producer): pin both VFX 0×0 marks, with a GSAP-shaped wipe and a real late host
* test(producer): a late VFX host with a GSAP wipe catches a walk that skips hidden hosts
* fix(engine): fail the render when an injected video frame cannot decode
* fix(engine): report BeginFrame capture errors at once, not after the stall watchdog
* fix(engine): skip the diagnostic screenshot for any session launched for BeginFrame
* fix(engine): preserve composition root background in alpha exports
initTransparentBackground() forced html, body, AND any
[data-composition-id] element (the composition root, or a nested
sub-composition root) to transparent for every alpha-carrying capture
session. That's correct for the HDR two-pass layered DOM pass, where
the HDR video itself is the backdrop — but for a plain alpha export
(MOV ProRes4444, webm+alpha) of a normal composition, an author's own
background painted on the root is real, intentional content, and it
was being silently dropped.
Add CaptureOptions.clearCompositionRootBackground, threaded through
session.options into every general-session call site (default false —
preserve the root's authored background). The two HDR-layered-DOM-
session call sites set it to true, keeping their existing (correct)
behavior. initTransparentBackground's injected stylesheet is now
idempotently rewritten rather than insert-once/skip, so a caller's
choice is never silently ignored by an earlier call on the same page.
Add a producer test fixture + transparency-test.ts check proving a
composition root's authored background survives into alpha output,
plus unit tests for both flag values and the rewrite-on-repeat-call
behavior directly.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* test(producer): register the new transparency fixture with the shard planner
transparency-regression-root-bg is exercised by transparency-test.ts
directly (asserts a real alpha channel), not by the sharded regression
harness, same as its sibling transparency-regression. The shard
planner hard-fails on any fixture directory it doesn't recognize as
either scheduled or explicitly excluded, so register it in the
excluded list with the same reason as its sibling.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): restore explicit transparent-background call for HDR layered sessions
The previous commit removed the two HDR-layered-DOM-session call
sites' own explicit initTransparentBackground() call, relying on
initializeSession()'s internal call (gated behind
session.options.format === "png") plus a new
clearCompositionRootBackground session option.
That was wrong: a plain HDR render's own DOM session captures
"jpeg", not "png" — the final HDR MP4/HEVC output doesn't itself
carry an alpha channel, only the layered-compositing step does. So
initializeSession()'s internal call never fired at all for this
common case, and the HDR video layer never got a transparent DOM
backdrop to composite into.
Reproduced directly against the hdr-regression fixture (part of the
regression-shards CI suite): a real local render showed a severe
PSNR collapse (~8-56dB) in the "Window B: z-order sandwich" segment,
where a DOM element behind the HDR video, the video itself, and a
translucent DOM overlay in front all need correct z-ordering — with
the composition root's own background no longer cleared, it painted
solid over the video slot instead of letting the video composite
through. Confirmed via bisection: reverting to the prior commit's
files reproduces 0 failed frames (passing); the removed-call version
reproduces the exact failure; restoring the explicit calls fixes it
again (100/100 checkpoints, 0 failed frames).
Restore the explicit, unconditional call in both HDR call sites,
right after initializeSession(). clearCompositionRootBackground still
does its job for the general/plain-alpha-export path (frameCapture.ts),
which is unaffected by this fix.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* refactor(engine): let the HDR layered pass own composition-root clearing
Drop the CaptureOptions.clearCompositionRootBackground field: every HDR layered
DOM session already calls initTransparentBackground right after init, so that
call is the one owner. Engine paths use the default, which keeps root backgrounds.
* chore(producer): drop comment additions the comment ratchet rejects
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): warn when a video input declares alpha but decodes opaque
- Problem: a video input whose alpha_mode=1 tag outlives its alpha plane (a remux drops the BlockAdditional sidecar while keeping the tag) composites as a solid rectangle, and nothing tells the user their file, not the renderer, is the problem (#3220 / #3226).
- Fix: during extraction preflight, when an input declares alpha (hasAlpha) and its codec can carry alpha, sample the decoded alpha plane (up to 3 frames at 8x8 rgba, bounded 30s, ~768-byte ceiling). If uniformly opaque, emit a non-blocking stderr warning naming the file with the re-export remedy. Inconclusive probes stay silent; opaque full-frame backgrounds are legitimate and never fail the render.
- Verification: 8 new unit tests (byte logic + message); engine suite 1483/1487 with 4 pre-existing failures on clean main (missing hdr-regression PNG fixture in this clone); typecheck/lint/format clean; real-fixture checks: lying-tag WebM warns, genuine-alpha WebM stays silent, both agree with ffmpeg alphaextract.
Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
* fix(engine): probe input alpha with the decoder that matches its codec
* style(engine): format the alpha probe test
* fix(engine): keep the alpha probe from failing or racing a render
---------
Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Miguel Ángel