* perf(engine): a remote source's tail decodes run many at a time, not one per half-core
* perf(producer): the plan logs where its extract stage spends its time
* perf(engine): a window that ends inside its source takes its frame count without a decode
* test(engine): pin the tail-decode slot test to the windows that decode
* style(engine): format the tail-probe window check
* test(engine): deferred counts match extraction on a B-frame NTSC source; VFR is not deferred
* style(engine): format the tail-probe tests
* perf(aws-lambda): a site stores each video's ffprobe JSON, and the plan reads it instead of probing
* test(aws-lambda): a site stores and hands out each video's probe
* style(aws-lambda): format the probe-facts upload and tests
* perf(engine): a site's stored probe also holds the final-frame timestamp the plan would probe for
* test(engine): drop an unused import
* test(aws-lambda): the stored probe holds the ffprobe JSON and the final frame
* test(aws-lambda): the stored probe holds the ffprobe JSON and the final frame
* perf(producer): a distributed chunk encodes while it captures
* perf(producer): the audio task logs where its mix spends its time
One line per audio stage: prepare, groups, mix, master, encode and the peak check, in ms.
* perf(engine): locked warmup ticks stop sleeping once the page is ready
* perf(engine): a window reaching a stored video's end takes its frame count from stored facts
* fix(producer): the cloud audio task logs its phases
* perf(engine): stored facts keep the last packet length so late-start windows skip the tail decode
* fix(engine): a probe spawns without waiting when no stored facts are declared
* chore(aws-lambda): shorten comments over the line limit
* fix(engine): warmup keeps pacing after a failed tick; stored final frames check ffprobe major
* test(engine): the recorded ffprobe major matches the build's own banner
* test(engine): the version-banner probes go through a named binary like the other spawns
* fix(engine): renames ride out Windows file locks instead of failing the render
* fix(producer): the true-peak and segment-manifest renames ride out Windows file locks too
* perf(engine): remote sources of a site keep their probe results for the whole render
* test(engine): probe a content-addressed URL once per render
* style(engine): format the audio probe split
* perf(aws-lambda): a render reads a video's audio from a small file stored beside it
A video's audio is interleaved with its pictures, so mixing it from the cloud read the whole video (2.5 GB for a 20-minute 4K film) at the Lambda's network speed. lambda sites create now stores each large video's audio packets as a separate object keyed by the video's hash; the audio task reads that when it exists.
* chore(producer): shorten the audio sidecar header comment
* fix(aws-lambda): no audio sidecar for a video whose audio starts late or that has several tracks
* test(producer): format the audio sidecar tests and end their ffprobe options with --
* fix(studio-server): keep transparency in proxies of VP8/VP9 alpha webm
A webm whose alpha lives in the VP8/VP9 side channel (alpha_mode=1,
pix_fmt yuv420p) was proxied to opaque H.264, and the VP8 proxy path
decoded it with ffmpeg's native decoder, which drops alpha. The alpha
rule and the alpha-reading decoder now live once in
@hyperframes/parsers/media-alpha, used by engine and studio-server.
* build(parsers): raise the packed size budget for the media-alpha subpath
parsers already sat at its packed budget on main; the shared alpha rule
adds 676 B of JS and 803 B of declarations (190958 B packed in CI).
* perf(engine): cloud renders cut a video's audio from its URL instead of downloading the file
* refactor(engine): keep the audio mixer's comment share at the base
* fix(engine): a cloud-held file's audio probe retries once, then fails instead of mixing unprobed
* fix(engine): a cancelled probe is not retried, and the encode test noise is seeded so it repeats
* test: queue runs on slow windows runners stop failing four timing tests
* test(cli): the keepalive process test budgets for a cold start on a slow runner
* fix(engine): long 4K video exports no longer time out while frames are still coming
* test(engine): the paced extraction test keeps a wide margin on a loaded runner
* fix(engine): long phone-video exports probe each source once instead of once per cut
* fix(engine): a failed source probe is retried, and probes share the extraction slots
* test(engine): synthesize the tone-mapped concurrency fixture with known input colour
* perf(producer): distributed chunks download only the videos they show
A project video under compiled/ that chunks extract their own frames from
now goes only to the chunks that show it, like video-sources/ files. A
chunk skips the readiness wait for a deferred video it does not show, so
it no longer logs media_load_failed for a file it was never given.
* fix(producer): cloud site uploads pack the files their id hashes, symlinks included
* perf(aws-lambda): chunks read only the byte ranges of a video they need
A chunk that extracts its own frames from a deferred project video now
reads it from S3 by byte range through a short-lived pre-signed URL,
instead of downloading and hashing the whole object first. S3 attests the
object's stored SHA-256 before the URL is signed; every other artifact is
still downloaded and verified as before.
* perf(aws-lambda): upload project files once to a shared content-addressed area
* chore(producer): keep planV2 comment share at the base level
* fix(engine): verify the TLS certificate of https video inputs and never surface their signed query
* fix(aws-lambda): renders re-upload a site's expired files instead of failing
* perf(aws-lambda): the plan step probes project videos in place instead of downloading them
* fix(engine): https video reads reconnect and time out, and redaction covers wrapped causes
Signed-URL reads now pass -rw_timeout, -reconnect and -reconnect_delay_max (both Lambda builds accept them) and ffmpeg also -reconnect_on_network_error, so a transient S3 error or a stalled read fails fast or recovers instead of waiting out the process timeout. Query redaction now descends into error.cause.
* fix(aws-lambda): share entry bundles and ratchet package budget
* fix(aws-lambda): site ids uploaded before files.json still render
A site id from an earlier sites create has only sites/<id>/project.tar.gz.
renderToLambda now falls back to that tarball, which the plan step still
reads, and when neither key exists it fails before starting with both keys
named and the fix (run sites create again).
* fix(aws-lambda): renders fail before upload on a stack too old to read sites
A stack deployed before files.json sites untars the site file in its plan
step and fails after four retries. renderToLambda now reads the state
machine definition first and stops with STACK_TOO_OLD and the redeploy
command when the plan task does not list SITE_MANIFEST_INVALID, which new
SAM and CDK stacks now treat as terminal. The README's upgrade section
drops the planProtocol v1 fallback, which hits the same failure.
* docs(aws-lambda): document shared content storage and the upgrade path
The deploy guide still described the tarball HeadObject short-circuit. It
now covers the cas/sha256/ store and its 30-day lifecycle rule, the
STACK_TOO_OLD, SITE_FILES_MISSING and SITE_NOT_FOUND errors, that old site
ids keep working, and that every project uploads once more after the
upgrade. A test keeps a troubleshooting entry for each pre-submit error.
* fix(producer): chunks log each video they read by URL, without its query
A chunk reading a video straight from the bucket left no trace in its log.
It now logs one line per remote source with the path and the URL minus its
query string, so signatures never reach the log.
* fix(engine): ffmpeg errors never keep part of a signed URL's query
A process's stderr tail was trimmed before its URLs were redacted, so a
cut inside a signed URL left a query fragment the URL pattern no longer
matched, and the fragment reached the error. The stderr tail is now
redacted before every trim, and the redactor also strips a signing
parameter cut off from its URL.
* fix(aws-lambda): comments name the site manifest instead of a file path
* fix(engine): stderr redaction survives chunk splits, and every ffmpeg spawn uses it
The stderr tail was redacted on each chunk and then cut, so once a cut
severed a long URL's scheme, the rest of its signature in the next chunk
matched neither pattern. The tail is now kept raw with 2x slack and
redacted only when the text is produced, then cut to size. The unused
producer audioExtractor (its own spawn, unredacted stderr) is deleted and
the parity harness's PCM decode goes through ManagedChildProcess; a test
fails on any ffmpeg/ffprobe spawn in engine, producer or aws-lambda that
does not.
* chore(producer): record the audioExtractor deletion as intended
* fix(engine): drop a signature fragment the stderr tail cut from its URL
The raw tail is kept at twice the cap, but that cut can itself land inside a
signature, and when redaction shrinks the rest below the cap the fragment
survived. After a cut, the severed token is dropped before redaction.
* refactor(engine): name the severed-token drop instead of commenting it
* test(engine): a cut-off URL leaves nothing behind; aws-lambda size budget to measured
* fix(producer): chunk URL-read tests use main's frame list; producer size budget to measured
* chore(producer): size budget at the measured size, no headroom
* chore(producer): size budget back to measured x1.05, the check's own rule
* fix(engine): SDR content in HLG exports sits at reference white, not peak
The sRGB to HLG conversion for DOM layers fed SDR-relative light straight into the HLG OETF, whose input is light relative to the HLG peak, so SDR white landed at 100% signal. Graded HLG footage (graded as SDR, then converted back) and every white overlay were pushed to peak and blew out. SDR white now sits at 75% HLG signal (BT.2408), the same model the PQ branch uses with 203 nits.
* docs(engine): shorten the SDR-white note in the HDR LUT
* docs(engine): cut the LUT generator's notes to its usage and drift rule
* fix(engine): SDR exports carry BT.709 transfer and primaries with every encoder
The SDR conversion filter set the BT.709 matrix on its frames but not the
transfer or primaries. Encoders that take colour from the frames rather than
from -color_* options (VideoToolbox on macOS) wrote color_space=bt709 with
color_transfer and color_primaries unknown. The capture and raw-input SDR
filters now end in the tagged BT.709 filter ProRes already uses.
* fix(engine): HDR exports tag BT.2020 with PQ or HLG on the frames too
VideoToolbox HDR encodes had the same gap as SDR: given only -color_* options
it wrote bt2020nc with transfer and primaries unknown, so players read the
export as SDR. Every HDR branch of the streamed and disk encoders now tags the
frames through one hdrFrameTagsFilter built from the encoder colour params.
* test(engine): colour-tag probes pass -- before the output path
* test: ffmpeg failures name how ffmpeg ended and keep its last line
* fix(producer): an ffmpeg failure names how ffmpeg ended and keeps its last line
* refactor(producer): say why in names, not comments
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8
VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.
The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused
v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#4564 routes SDR colour through `format=gbrp`. Without accurate_rnd,
swscale's yuv-to-gbrp conversion leaves the last 8 columns black when the
width is 16n + 8 (1000, 1080, 1112...), so every 1080x1920 portrait MP4
since 0.8.81 has had an 8 px black strip on the right. The same step in
SDR jpg frame extraction blanked the right edge of footage at those widths.
Set accurate_rnd on the yuv-to-gbrp scale in both chains. It is also the
more accurate rounding: on a testsrc2 frame, worst luma error against a
float reference drops from 1.85 to 0.90 codes.
Fixes#5551
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): encode and tag ProRes mov output as BT.709
* test(engine): read the ProRes colour tags from the first frame
* test(engine): end the ProRes tag probe's options with --
* fix(engine): exports place sound and picture right when a source's audio and video start apart
* fix(engine): a late video stream plays to its end and distributed and HDR renders hide its lead
* test(engine): pin a late video stream's trimmed start, held tail and HDR trim
* fix(studio): text in Studio reads without em or en dashes
* fix(studio): the dash check also catches HTML entities, and peak reads Unavailable
* test(studio): the dash check test covers every entity form
* fix(studio): messages Studio shows from lint, sdk, core and the renderer drop dashes too
* fix(producer): the gate check stops early on the engine's new fallback wording
* fix(engine): long exports with many clips no longer stall at 90% or hide a timeout
* fix(engine): stop queued frame extractions on cancel and time out distributed assembly
* refactor(engine): simplify the extraction slot check and its concurrency test
* test(engine): prove a cancelled extraction leaves the queue, skip the shell shim on Windows
* test(engine): wait for the held extraction's own record before cancelling the queued one
* fix(engine): recover audio duration from packet timestamps before clamping
An authored `data-duration` on an <audio> element could be silently
shortened when a source file's container-summary duration (mvhd/tkhd/mdhd)
undercounts the real audio length while the packet stream itself is
intact. Add `probeAudioDurationFromPackets()`, which re-derives the true
duration from the last packet's own pts_time + duration_time — figures
ffprobe computes from the stream's own timestamps, not the container's
summary header — and wire it into the compiler's existing clamp check as a
rescue attempt before accepting a shorter value. Works regardless of
codec/profile; verified against real HE-AAC/SBR fixtures and a
binary-patched "lying container" MP4 built for the regression test.
The rescue only fires for candidates the existing clamp predicate already
flagged (no added cost on the common path), runs in parallel across
candidates rather than serially, and requires the recovered duration to
exceed the existing clamp epsilon (absorbing the few milliseconds of
priming-delay drift raw packet timestamps carry relative to an
edit-list-corrected container duration) before overriding the shorter
value.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): move audio duration correction into extractAudioMetadata
The previous approach compared the compiler's own packet-scan probe
against the already-corrected duration returned by extractAudioMetadata,
which meant the correction only applied to one caller and the regression
fixture happened to exercise a pre-existing AAC-LC-specific refinement
instead of the new code path.
Move the fix to the single canonical place duration is derived: replace
the old AAC-LC-only packet-count refinement (a profile allowlist assuming
a fixed 1024-samples-per-frame constant) with a codec-agnostic mechanism
that decodes the final audio frame and reads its own timestamp plus real
sample count. This has no per-codec assumption, so it applies uniformly
regardless of codec or profile, and every caller of extractAudioMetadata
benefits automatically. The compiler's own clamp logic is now unchanged
from its pre-existing form.
Verified against a real, previously-untouched case: a FLAC-in-MP4 file
(a codec the old AAC-LC-only path never corrected) with its mvhd/tkhd/mdhd
duration atoms binary-patched to half their true value while every real
frame stays intact. Confirmed this reproduces on the prior code (by
literally substituting its actual content) and is fixed by the new code.
The decode probe reads CSV output (each line stands alone) rather than
JSON (a single top-level array), and retains only the output tail under a
small cap — a lying container's true duration can be arbitrarily larger
than its declared one, and JSON truncated from the front isn't valid JSON,
so an unbounded scan of a long file silently fell back to the wrong
duration once ffprobe's stdout cap was hit.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* test(producer): update style-7-prod golden for corrected AAC duration probe
The deleted AAC-LC packet-count refinement assumed every demuxed packet
decodes to exactly 1024 samples. For this suite's real audio fixture
(a 48kHz stereo AAC-LC MP4), that assumption overcounts: ffprobe reports
753 demuxed packets, but only 752 of them actually decode into a real
audio frame (verified via raw PCM sample count: 770048 samples / 48000Hz
= 16.0426666...s). The old mechanism's 753 * 1024 / 48000 = 16.064s was
therefore wrong; the new decode-based probe correctly derives ~16.043s
from the real final frame.
Regression-shards caught this as a compilation mismatch on the previous
commit. Verified the corrected golden against the real source: full
compilation, visual (0 failed frames), and audio (correlation 0.966)
checks all pass locally with the actual LFS-hosted golden video, and a
standalone compileForRender() check confirms all other audio-bearing
fixture suites in the repo are unaffected.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* perf(engine): seek the audio decode probe instead of scanning the whole file
probeDecodedAudioDuration ran a full, single-threaded decode of the
entire audio stream on every extractAudioMetadata call (once per audio
element in the HTML compiler, plus audio mixing/padding/encoding
steps). On real 30-minute files this took over a second, and could
approach ffprobe's 30s deadline on multi-hour files while holding a
shared probe slot the whole time.
Add -read_intervals so the probe seeks to ~1 second before the
container's already-known claimed duration and decodes only that
tail. This is safe because the correction only ever raises the
duration: an honest or over-claiming container's seek lands at or past
the real end and finds nothing extra, while an under-claiming
container (the only case this exists to fix) still has real frames
past the seek point. If the tail read finds nothing at all, fall back
to a full unseeked decode before giving up, protecting files whose
seek index is unreliable (an index-less or non-seekably-sourced
container) from silently missing a real correction — mirroring the
existing seek-then-fallback pattern already used by the sibling
final-video-frame probe in this file.
Also replaces the real-file regression test's runtime ffmpeg-spawning
fixture generation with two small, committed binary fixtures (an
honest FLAC-in-MP4 file and a copy with its container duration atoms
halved) plus the ffprobe-static package, so this test's real-file
proof actually runs in CI instead of being silently skipped when no
system ffmpeg is on PATH.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* refactor(engine): trim the audio duration probe comments to the invariant
* test(engine): cache tests queue the decode probe and count metadata probes only
* test(engine): probe for ffprobe with a literal argv so the argv contract classifies it
* test(producer): regenerate the speed-ramp golden for the single-encode mix
The audio probe now reads the mixed AAC as exactly 4.000 s instead of the
packet-count 4.032 s (which counted the priming frame), so the final mix
is stream-copied instead of re-encoded once more. Same waveform (sample
correlation 0.9996 vs the old golden), 3.1 dB less true-peak attenuation;
video frames are byte-identical.
* fix(engine): measure decoded audio from the stream start and trust it for raw ADTS
The decoded last-frame end is an absolute timestamp, so subtract the audio stream's start time
(format start as fallback) before comparing it with the start-relative container duration, and
start the tail read at start + duration - 1 s. MPEG-TS audio (1.4 s default start) and offset
MP4s were over-extended by their start time.
Raw ADTS (format aac) has no duration field; ffprobe estimates it from the opening bitrate and
overclaims after a quiet opening. Its decoded timestamps are sample-counted, so for raw ADTS the
decoded end now wins in both directions beyond the margin. Other containers stay upward-only so
an MP4 edit list is never extended.
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* feat(render): report encoding and assembling progress
* fix(render): close every progress stage and fail a render whose progress callback throws
* fix(render): send a repeated progress update once
* fix(render): keep Docker terminal output as before and drop the unneeded callback ending
* fix(render): keep one failure stage code per stage whatever its frame counts
* fix(engine): warn when a video input declares alpha but decodes opaque
- Problem: a video input whose alpha_mode=1 tag outlives its alpha plane (a remux drops the BlockAdditional sidecar while keeping the tag) composites as a solid rectangle, and nothing tells the user their file, not the renderer, is the problem (#3220 / #3226).
- Fix: during extraction preflight, when an input declares alpha (hasAlpha) and its codec can carry alpha, sample the decoded alpha plane (up to 3 frames at 8x8 rgba, bounded 30s, ~768-byte ceiling). If uniformly opaque, emit a non-blocking stderr warning naming the file with the re-export remedy. Inconclusive probes stay silent; opaque full-frame backgrounds are legitimate and never fail the render.
- Verification: 8 new unit tests (byte logic + message); engine suite 1483/1487 with 4 pre-existing failures on clean main (missing hdr-regression PNG fixture in this clone); typecheck/lint/format clean; real-fixture checks: lying-tag WebM warns, genuine-alpha WebM stays silent, both agree with ffmpeg alphaextract.
Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
* fix(engine): probe input alpha with the decoder that matches its codec
* style(engine): format the alpha probe test
* fix(engine): keep the alpha probe from failing or racing a render
---------
Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* fix: hide console windows spawned by ffmpeg and nvidia-smi on Windows
Node's `windowsHide` defaults to false, so every child process that is a
console application pops a window on the user's desktop. During a render
or a regression run that is a stream of black windows stealing focus.
The repo already sets `windowsHide: true` across cli, studio-server and
most of engine; these were the remaining gaps, all on paths that run
during a normal render:
- core/mediaGradeAnalyzer: ffprobe + ffmpeg, once per analyzed media file
- engine/browserManager: `nvidia-smi` VRAM probe. This one goes through a
shell (`execSync`), so it flashes a cmd.exe window even on machines with
no NVIDIA GPU, where the command exists only to fail
- producer: the regression harness ffmpeg runner, PSNR/audio comparison
helpers, fixture synthesis, and the ffprobe call in audioPadTrim
No behavior change on POSIX, where the flag is ignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(producer): escape PSNR stats_file for ffmpeg's two-pass filtergraph parse
An ffmpeg filtergraph argument is unescaped twice: once when the graph is
split into filters and their options, then again when the option value is
read. `psnrAtFrames` escaped `\` and `:` a single time, which only survives
the first pass.
On POSIX that is invisible, because mkdtemp under tmpdir() produces neither
character. On Windows the stats path is always `C:\Users\...`, so it reaches
the option parser with a bare `:`, which starts a new option and fails the
whole graph:
[AVFilterGraph] No option name near '\Users\...\psnr.log'
Error parsing filterchain '[rv][gv]psnr=...:stats_file=C\:\Users\...'
Every suite that got as far as quality validation died there, so a Windows
regression run reported 0 passes with only 2 genuine compilation failures.
Escaping for both passes fixes it and stays a no-op on POSIX.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(producer): pin PSNR stats path escaping with a colon and backslash path
* fix(producer): keep the PSNR stats path out of the filtergraph
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): forced-SDR renders show tone-mapped HDR footage as Studio does
Forced-SDR extraction tone-maps HDR frames with zscale, but the frames kept the
BT.709 transfer tag, so Chrome re-curved them (a tone-mapped 388d96 painted as
2d8790), and JPEG frames kept the BT.709 matrix every JPEG reader decodes as
BT.601. Tone-mapped frames now get the same sRGB tag and BT.601 JPEG conversion
as other SDR frames.
macOS used VideoToolbox, which does not tone-map: its frames stay PQ and Chrome
tone-maps them itself, darker than the hable curve Linux and Studio use. macOS
now takes the zscale path whenever ffmpeg has zscale, found through the shared
ffmpeg filter probe; without it the old path stays and the render warns once.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): forced-SDR renders tone-map HDR footage that lacks matrix or primaries tags
zscale finds no path between colourspaces without them, so the extraction
failed. Untagged HDR footage is BT.2020 in practice, and now gets those tags.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): forced-SDR tone map also tags the transfer ffprobe read from the container
From ffmpeg 8.1 ffprobe reports a container-only PQ or HLG tag; the frames
still lack it, so zscale failed where macOS VideoToolbox used to render.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* test(engine): gate the container-only transfer case on the ffprobe that reads it
* docs(engine): shorten the filter-probe comment to the comment limit
* fix(engine): keep valid per-frame colour tags when tone-mapping HDR footage
* test(engine): build the mixed HLG and PQ fixture without B-frames so its timestamps are valid
* test(engine): build the trimmed mixed fixture without B-frames too
---------
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): declare SDR extracted frames as sRGB so Chrome does not darken footage
* fix(engine): encode SDR mp4 frames with the BT.709 matrix they are tagged with
Chrome hands the SDR encoder BT.601 full-range JPEG captures (or RGB PNGs).
The encoders only converted the range, so swscale kept the BT.601 matrix under
the BT.709 tag (ffmpeg 5.1 to 7.0: CSS #C83C28 delivered as #D44824) or
converted YUV to YUV through a low-precision step (ffmpeg 8+: #101010 delivered
as #0B0D0D). Decode to RGB with the capture matrix, then encode with BT.709.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): tag SDR frames plain sRGB so every ffmpeg writes the sRGB chunk
ffmpeg writes a PNG sRGB chunk only for BT.709 primaries and writes cICP only
from 6.1. On ffmpeg 5.1 (Debian bookworm) a BT.601-tagged source therefore got
cHRM plus gAMA 1/2.2, which Chrome decodes as a pure 2.2 power: 0x10 shadows
rendered as 0x07. Declaring BT.709 primaries too gives the sRGB chunk on every
version, so Chrome shows the frame code values for every SDR source. The test
now reads the PNG chunks instead of ffprobe, which reports no transfer for an
sRGB chunk on ffmpeg 5.1.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): extract SDR jpg frames as the BT.601 full range JPEG readers assume
The default frame format for opaque SDR sources is JPEG. ffmpeg converted the
BT.709 source to full-range YUV without changing the matrix, and every JPEG
reader (Chrome included) decodes BT.601, so footage shifted: 0xC83C28 showed as
0xBB2D29. The old mp4 encode made the inverse mistake and hid it; with the
encode fixed, it would reach the output. Decode through RGB with the source
matrix, then encode BT.601 full range, keeping 4:2:0.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): round the SDR mp4 colour conversion accurately
swscale's default rounding moved saturated colours up to 3 codes (pure red 254
delivered as 251 on ffmpeg 5.1 to 7.0); accurate_rnd brings that to 1. The
chunk encoder now decides the HDR-or-SDR filter once, and the 10-bit GPU pad
test sets hdr, which its name always claimed.
* style(engine): keep the colour filter reasons in names and tests, not source comments
The comment ratchet caps a file's comment share: the new filter module and the
extractor carry their intent in constant names, and the encode test states why
the conversion goes through RGB.
* fix(engine): keep colour edges sharp through the SDR colour conversions
Going through RGB upsampled chroma with swscale's default filter, so ffmpeg 4.4
to 7.x smeared colour across edges by about 2 px (red 254 next to blue came out
8 inside the blue). Nearest-neighbour resampling maps the half-resolution chroma
back one to one, keeps the flat-colour results, and makes accurate_rnd redundant.
The pixel tests now also check a red to blue edge.
* test(producer): regenerate goldens whose colours the old mp4 matrix bent
These four fixtures compare at 40 to 45 dB, and their goldens were rendered
with the BT.601-under-BT.709 encode, so the corrected colours score 34 to 37 dB.
Regenerated in the Dockerfile.test image. Each new golden is closer to the
colour its page declares, e.g. #00d4aa was 00bcab and is now 00d3aa, and
#087443 was 006643 and is now 087241.
* fix(engine): keep VFR frame timing when extraction applies colour filters
ffmpeg before 6.1 ignores frame durations in CFR resampling once any -vf is
set, so a VFR window ending on a still lost frames. Filtered VFR extraction
now resamples in one process exactly as before and filters in a second.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): encode raw SDR layered frames in BT.709
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* test(engine): keep the new colour tests inside the format, comment and argv checks
* fix(engine): keep HDR and unusual pixel formats out of the two-process VFR path
The raw handoff dropped HDR10 light levels and misread semi-planar formats, and
decoder options rejected some ffprobe colour names. Only SDR sources in formats
nut carries unchanged now take it, their colour tags come back through setparams,
and a process that cannot start releases the other.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): read untagged footage with the matrix Chrome plays it with
Chrome decodes an untagged matrix as BT.709 from 720 coded lines up and as
BT.601 below; extraction assumed BT.601 at every size, so HD footage without
colour tags shifted hue against Studio's own preview.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): untagged VP9 and AV1 footage keeps BT.601, as Chrome plays it
Chrome applies the 720-line rule only to codecs it decodes through ffmpeg
(H.264, VP8); untagged VP9, VP9 alpha and AV1 play as BT.601 at every size.
Also widens the raw-pipe pixel formats to all verified ones, and reports the
pipeline process that failed first by settle order, not by per-process clocks.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* test(producer): regenerate goldens for corrected page colours and untagged SD footage
* docs(engine): say which untagged codecs the matrix rule was measured on
---------
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
readEngineVersion resolved ../../package.json relative to import.meta.url.
That path is correct from packages/engine/src/utils/ in the repo and
overshoots the package root from <pkg>/dist/ in the published CLI, which
bundles the engine. The require threw MODULE_NOT_FOUND, a bare catch
swallowed it, and every render stamped the 0.0.0-dev fallback.
No single relative literal is correct in both layouts, so resolve by walking
up to our own package root instead of by a fixed depth.
The name check is the fix, not a refinement. A bare "first package.json
found while walking up" resolves to whatever sits above in a hoisted install
and returns a confident wrong version - the same class of defect as the
sentinel, and harder to notice because the answer looks plausible. Only a
package.json whose own name is hyperframes or @hyperframes/* may answer.
A test pins this: removing the name check fails that case and only that case.
Also replaces the sentinel. 0.0.0-dev parses as a version, compares against
one, and sorts below every release, so a total resolution failure was
indistinguishable from a dev build - which is why it shipped unnoticed. The
fallback is now "unresolved", deliberately not semver-shaped, and every
failure path warns with the reason. A failed lookup still never breaks a
render.
The old comment claimed the engine ships as raw TS and exports
./package.json. The published package has no exports field and bundles the
engine, so that premise does not hold where the code runs; it is replaced
with the two layouts and why they conflict.
Refs #4365
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(engine): hold the download attempt deadline so slow runners cannot trip it
urlDownloader tests passed a real 1s deadline that a slow Windows attempt outlasted.
The deadline timer is now held; the one test that needs it fires it by hand.
* fix(engine): name the underlying error when a download fails, without its URL
* test(engine): fire the stalled-body deadlines by hand instead of racing a real 20 ms timer
When ffprobe omits the video stream's own `duration` field,
`extractMediaMetadata` falls back to the container duration
(`format.duration`), which includes the longest stream in the
container. Stock and looping clips commonly have an audio track
2x the video length, inflating `videoStreamDurationSeconds` and
causing the downstream coverage gate to expect twice the frames
FFmpeg can actually produce — the "50% cluster" failure pattern.
Cross-check with `nb_frames / fps` when the stream duration is
absent. If the frame-derived duration is >10% shorter than the
container, use it instead. This corrects the inflation at probe
time without requiring a second ffprobe pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(gcp-cloud-run): run the Cloud Run image on Bun 1.4.2 so remote downloads cannot hang
* fix(engine): make the download deadline abort the body pipeline
* fix(engine): classify a single-keyframe video as problematic, not healthy
analyzeKeyframeIntervals short-circuited to isProblematic:false whenever
it found fewer than 2 keyframes, but exactly one keyframe is the worst
case for seek performance, not a healthy result: every seek past frame 0
must decode the entire file.
The single-keyframe branch treats the rest of the video stream's own
duration (not a container duration, which a longer audio track can
inflate) as one uninterrupted GOP. ffprobe's frame timestamps are
absolute presentation timestamps rather than relative to stream start,
so a re-muxed or trimmed clip with a nonzero start time is normalized
the same way extractFinalVideoFrameTimestamp already does for the same
quirk. The cache key now includes both values, matching that sibling
function's convention, since a stale cache entry from a different
duration/start pair would otherwise win silently.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* refactor(engine): share stream-window normalization between keyframe check and final-frame lookup
extractFinalVideoFrameTimestamp and analyzeKeyframeIntervals each
independently computed the same stream-start fallback, finite guard, and
cache-key format from the same metadata shape. Extracted resolveStreamWindow
so the two can't drift out of sync with each other.
Also: match ffprobe's real trailing-comma csv output in one keyframe mock,
add a test at the exact 2s threshold boundary, and comment the
noUncheckedIndexedAccess-only fallback at the single-keyframe branch.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* style(engine): trim long comments to one-to-three lines
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
`readonly string[][]` parses as `ReadonlyArray<string[]>`, so `calls.push`
was a TS2339 in 5214552cd. It slipped through because the engine tsconfig
excludes `src/**/*.test.ts` and vitest strips types; caught by running tsc
on the test file explicitly. `Array<readonly string[]>` is what the
neighbouring "one ps call" test already uses.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every render image (packages/cli/src/docker/Dockerfile.render,
packages/gcp-cloud-run/Dockerfile, examples/k8s-jobs/Dockerfile.example,
Dockerfile.test) is node:22-bookworm-slim with no procps, so `ps` is ENOENT
in containers and sampleProcessRss would return [] on every tick: the whole
cloud fleet would report chrome_memory_samples: 0, indistinguishable from
"no Chrome pids". Linux now reads VmRSS from /proc/<pid>/status with a
per-pid try/catch, so one renderer that exited between discovery and
sampling omits only itself (with `ps` a wholly-dead batch fails the call).
No child process per tick on the primary server platform. Same precedent as
the CLI's orphanCleanup.ts / processTree.ts. `ps` stays for darwin and
other POSIX; `tasklist` for win32. The reader is an optional fourth
parameter with a default, so Task 2/3 callers are unaffected.
Scope note: spec section 2.5 records the Target-closed population as
docker=false throughout, so this closes a render_complete completeness gap
on cloud, not a coverage gap on the crash class this phase targets.
Verification: 15/15 vitest; four linux-branch mutants (per-pid catch
rethrows, regex loses /m, kb <= 0 weakened, path /status -> /stat) each
fail at least one test, restored green. Real-process smoke check on darwin:
sampleProcessRss([process.pid, <exited pid>]) returns one sample for the
live pid (BSD ps exits 0 with partial output when a valid-range pid is
gone). The linux path is unit-tested against a /proc/<pid>/status fixture
only; no Linux host or VM was available in this session to execute it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review of 76d8e2682 found three surviving mutants: removing the
`Number.isInteger(p) && p > 0` filter, dropping the `new Set` dedup, and
weakening `kb <= 0` to `kb < 0` all passed the plan's 8 tests. The filter is
the guard that matters most: Task 2 feeds `browser.process()?.pid`, and a
single NaN or undefined reaching `ps -p` makes ps exit non-zero, which the
never-rejects catch turns into `[]` for every pid in the sample.
Mutation check: each of the three mutants now fails exactly one test
("drops invalid pids and duplicates before calling ps" for the filter and
dedup, "returns [] when the memory column is not a positive number" for the
tasklist guard); restored source is 10/10 green.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
First piece of the long-form render Phase -1 telemetry: the Chrome memory
sampler needs resident set size per pid without adding pidusage or
systeminformation. `ps -o pid=,rss=` covers POSIX in one call for every pid;
`tasklist /FO CSV` covers win32 one pid at a time. The function never rejects
and omits pids that exited between discovery and sampling, so a failed sample
can never reach the capture loop.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Deliverable one of issue #4010: average K sub-frame captures into each output
frame so rotation, scale and opacity blur the way After Effects does, which the
translation-only catalog component (#4008) cannot do. Opt in per render with
CaptureOptions.motionBlur; absent means the capture path is unchanged.
The blocker was quantizeTimeToFrame flooring every seek onto the output frame
grid, which collapsed all K samples onto one instant and made the averaged frame
byte-identical to the unblurred one. quantizeSeekTime now owns that decision at
all three quantize sites (runtime player, its no-timeline fallback, and the init
transport) and refines the grid by an integer subdivision count, which can only
contain the output grid, never move a frame-time instant off it. The finer grid
rounds rather than floors: a sample time round-trips through the page bridge as
a float, and past a few million ticks that error exceeds the floor epsilon.
Accumulation sits inside captureFrameCore AFTER the static-dedup early return,
so a static frame still costs one reused buffer and only moving frames pay K. It
is not a before-capture hook, which would have disabled dedup outright.
Callback invariant: one eventful seek per output frame at the frame time, then K
suppressed sample seeks, then a suppressed restore to the frame time. The
restore matters because a suppressed seek still moves the playhead, so without
it the next frame's eventful seek would swallow the callbacks inside the shutter
window.
Scope of this deliverable: screenshot capture mode and PNG output. beginframe,
drawelement and JPEG encoding are rejected with a named error rather than
silently rendering unblurred. Video content is rejected for the same reason it
is a documented limitation: injected frames are extracted per output frame and
cannot follow a sub-frame seek.
Blend defaults to sRGB, matching AE's non-linearised 8-bpc working space and
the catalog component. Linear light is an explicit opt-in.
Samples are folded into a running average as they are captured rather than held
until the end, so peak memory is one accumulator plus one decoded sample instead
of all K: at 1080p that is a constant 41 MB rather than 141 MB at 16 samples and
540 MB at the 64 maximum, on every frame of the render.
PNG chunk CRC32 now has one owner in utils/crc32.ts, shared by the new encoder
and ffprobe's metadata reader. The encoder previously imported zlib's crc32 by
name, which throws at module evaluation on Node 22.0 and 22.1; ffprobe already
carried the guarded version and the comment explaining why. The sRGB transfer
function is likewise exported once and shared.
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Report HDR auto-promotion consistently from the render media set, including remote media, and make its regression coverage portable across supported paths.
Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com>
Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com>
* fix(build): stop generated files from being read half-written during the build
Two unrelated pull requests kept failing CI for reasons that had nothing to do
with their changes.
Typecheck failed with "TS1002: Unterminated string literal" pointing at a
generated file. The root build compiles several packages at the same time, and
more than one of them regenerates files under a package's src/generated while a
sibling's tsc is already reading them. A plain write empties the file and then
refills it, so for a few milliseconds what is on disk is the first half of the
new file. Whichever build read it in that gap saw a truncated file and stopped.
It never happened locally because locally nothing else is reading.
Every generator now writes the new version under a temporary name and then
renames it over the target, which the filesystem does in one step. A reader
either gets the whole previous version or the whole new one; there is no moment
where it can see half of either. Content that has not changed is not rewritten
at all, so repeat builds no longer touch these files. Four generators were
affected and all four now go through one shared helper, which is where the rule
lives from now on.
The build also rebuilt the core package a second time, in parallel with the
packages that read it. That second rebuild was already redundant, and removing
it takes the writers out of the window entirely. Renaming is what makes the file
safe; dropping the duplicate build makes the build shorter as well.
Separately, a linter performance test failed on four of the last ten failed runs
on main. It timed a scan with a stopwatch and demanded the result come in under
two seconds; one run took 3.7s. That is a statement about how busy the shared
runner was, not about the code, because a stopwatch also counts the time the
machine spent running someone else's job. The test now counts only the processor
time this process actually used, and checks that the cost grows in step with the
input rather than against a fixed number of milliseconds. Measured on a machine
under load, the stopwatch ratio for the same input reached 45x while the
processor-time ratio stayed at 20x.
Both fixes come with a check that fails if the fix is removed.
* fix(build): keep the shared write helper inside the core package
The helper the generators call had been placed in the repo-root scripts folder.
Every container image that builds a package copies the packages folders whole
but cherry-picks root scripts one file at a time, so the image builds failed on
a missing module. The repo already shows both halves of that convention: the one
root script a package build imports has a matching copy line in the image, and
cross-package imports into the core package need none.
The helper now lives with the core package's other build scripts, and the one
generator outside that package reaches it the way the engine package already
reaches core.
* test(engine): keep the probe cache bound test off the filesystem
The eviction test wrote, stat'ed and deleted 129 temp files to exercise an
in-memory LRU rule. Its runtime tracked filesystem contention rather than
the code under test, and on a busy Windows runner the file churn alone
pushed a ~300ms test past the 5s timeout, failing unrelated pull requests.
Cache identity comes from stat, so the test now synthesises stat results
and never touches disk. While here, the test also asserts the LRU touch:
re-probing an entry before the bound is hit must keep it resident and
evict the next-oldest one instead. The previous shape never hit the cache
during the fill, so that branch was untested.
* test(lint): keep the scaling check inside the scanner's linear range
The CPU-ratio version sampled 320k characters, where the output string's
own growth dominates and the whole test cost seconds of CPU; on a shared
runner that tripped the default test timeout, the failure this change
exists to remove. Sample 10k and 80k characters instead, where 8x input
measures ~8x and a quadratic scan still measures 60x or more, and give
the test an explicit timeout so a slow runner reports the ratio.
* test(core): snapshot file identity through a descriptor
The before/after inode and mtime checks read the file through a path stat
and then exercised the writer on the same path, which reads as a
check-then-use race to static analysis. Snapshot through an open
descriptor instead; the assertions are unchanged.
* fix(producer): transport safe extraction failure metadata
* refactor(producer): generalize public error metadata
* test(producer): use vendor-neutral media hosts
Miguel Ángel