Commit Graph
112 Commits
Author SHA1 Message Date
Miguel Ángel ea742c5018 fix(engine): exports carry full colour tags with every encoder, SDR and HDR (#5742)
* fix(engine): SDR exports carry BT.709 transfer and primaries with every encoder

The SDR conversion filter set the BT.709 matrix on its frames but not the
transfer or primaries. Encoders that take colour from the frames rather than
from -color_* options (VideoToolbox on macOS) wrote color_space=bt709 with
color_transfer and color_primaries unknown. The capture and raw-input SDR
filters now end in the tagged BT.709 filter ProRes already uses.

* fix(engine): HDR exports tag BT.2020 with PQ or HLG on the frames too

VideoToolbox HDR encodes had the same gap as SDR: given only -color_* options
it wrote bt2020nc with transfer and primaries unknown, so players read the
export as SDR. Every HDR branch of the streamed and disk encoders now tags the
frames through one hdrFrameTagsFilter built from the encoder colour params.

* test(engine): colour-tag probes pass -- before the output path
2026-10-18 22:15:31 +00:00
Miguel Ángel b98a1ac6e0 test(engine): process tracker tests spawn node, not shell tools, so Windows runs stop timing out (#5832) 2026-10-18 22:13:31 +00:00
Miguel Ángel c03532b285 fix(producer): an ffmpeg failure names how ffmpeg ended and keeps its last line (#5678)
* test: ffmpeg failures name how ffmpeg ended and keep its last line

* fix(producer): an ffmpeg failure names how ffmpeg ended and keeps its last line

* refactor(producer): say why in names, not comments
2026-10-16 01:38:20 +00:00
James RussoandClaude Opus 5.5 6afcb48395 fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8 (#5572)
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8

VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.

The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused

v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:45:43 +00:00
James RussoandClaude Opus 5.5 401354cd8f fix(engine): keep the rightmost 8 columns of SDR MP4 at widths like 1080 (#5559)
#4564 routes SDR colour through `format=gbrp`. Without accurate_rnd,
swscale's yuv-to-gbrp conversion leaves the last 8 columns black when the
width is 16n + 8 (1000, 1080, 1112...), so every 1080x1920 portrait MP4
since 0.8.81 has had an 8 px black strip on the right. The same step in
SDR jpg frame extraction blanked the right edge of footage at those widths.

Set accurate_rnd on the yuv-to-gbrp scale in both chains. It is also the
more accurate rounding: on a testsrc2 frame, worst luma error against a
float reference drops from 1.85 to 0.90 codes.

Fixes #5551

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:50:09 +00:00
Diego López c8e105dc92 fix(engine): encode and tag ProRes mov output as BT.709 (#5382)
* fix(engine): encode and tag ProRes mov output as BT.709

* test(engine): read the ProRes colour tags from the first frame

* test(engine): end the ProRes tag probe's options with --
2026-10-13 00:07:44 +00:00
Miguel Ángel b211fc7cb8 fix(engine): MP4, MOV and TS exports keep sound and picture in sync when streams start apart (#5455)
* fix(engine): exports place sound and picture right when a source's audio and video start apart

* fix(engine): a late video stream plays to its end and distributed and HDR renders hide its lead

* test(engine): pin a late video stream's trimmed start, held tail and HDR trim
2026-10-11 13:54:33 +00:00
Miguel Ángel 9825614d91 fix(studio): Studio's own text reads without em or en dashes (#5414)
* fix(studio): text in Studio reads without em or en dashes

* fix(studio): the dash check also catches HTML entities, and peak reads Unavailable

* test(studio): the dash check test covers every entity form

* fix(studio): messages Studio shows from lint, sdk, core and the renderer drop dashes too

* fix(producer): the gate check stops early on the engine's new fallback wording
2026-10-10 19:08:30 +00:00
Miguel Ángel 3d6de0581e fix(engine): audio duration probe reads to the end on Windows; cli tests skip bun's disk cache (#5395) 2026-10-10 03:05:46 +00:00
Miguel Ángel 4b68f73fe7 fix(engine): long exports with many clips no longer stall at 90% or hide a timeout (#5375)
* fix(engine): long exports with many clips no longer stall at 90% or hide a timeout

* fix(engine): stop queued frame extractions on cancel and time out distributed assembly

* refactor(engine): simplify the extraction slot check and its concurrency test

* test(engine): prove a cancelled extraction leaves the queue, skip the shell shim on Windows

* test(engine): wait for the held extraction's own record before cancelling the queued one
2026-10-10 02:43:44 +00:00
miga-heygenandMiguel Ángel 9460c983e0 fix(engine): recover audio duration by decoding the final frame (#3986)
* fix(engine): recover audio duration from packet timestamps before clamping

An authored `data-duration` on an <audio> element could be silently
shortened when a source file's container-summary duration (mvhd/tkhd/mdhd)
undercounts the real audio length while the packet stream itself is
intact. Add `probeAudioDurationFromPackets()`, which re-derives the true
duration from the last packet's own pts_time + duration_time — figures
ffprobe computes from the stream's own timestamps, not the container's
summary header — and wire it into the compiler's existing clamp check as a
rescue attempt before accepting a shorter value. Works regardless of
codec/profile; verified against real HE-AAC/SBR fixtures and a
binary-patched "lying container" MP4 built for the regression test.

The rescue only fires for candidates the existing clamp predicate already
flagged (no added cost on the common path), runs in parallel across
candidates rather than serially, and requires the recovered duration to
exceed the existing clamp epsilon (absorbing the few milliseconds of
priming-delay drift raw packet timestamps carry relative to an
edit-list-corrected container duration) before overriding the shorter
value.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* fix(engine): move audio duration correction into extractAudioMetadata

The previous approach compared the compiler's own packet-scan probe
against the already-corrected duration returned by extractAudioMetadata,
which meant the correction only applied to one caller and the regression
fixture happened to exercise a pre-existing AAC-LC-specific refinement
instead of the new code path.

Move the fix to the single canonical place duration is derived: replace
the old AAC-LC-only packet-count refinement (a profile allowlist assuming
a fixed 1024-samples-per-frame constant) with a codec-agnostic mechanism
that decodes the final audio frame and reads its own timestamp plus real
sample count. This has no per-codec assumption, so it applies uniformly
regardless of codec or profile, and every caller of extractAudioMetadata
benefits automatically. The compiler's own clamp logic is now unchanged
from its pre-existing form.

Verified against a real, previously-untouched case: a FLAC-in-MP4 file
(a codec the old AAC-LC-only path never corrected) with its mvhd/tkhd/mdhd
duration atoms binary-patched to half their true value while every real
frame stays intact. Confirmed this reproduces on the prior code (by
literally substituting its actual content) and is fixed by the new code.

The decode probe reads CSV output (each line stands alone) rather than
JSON (a single top-level array), and retains only the output tail under a
small cap — a lying container's true duration can be arbitrarily larger
than its declared one, and JSON truncated from the front isn't valid JSON,
so an unbounded scan of a long file silently fell back to the wrong
duration once ffprobe's stdout cap was hit.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* test(producer): update style-7-prod golden for corrected AAC duration probe

The deleted AAC-LC packet-count refinement assumed every demuxed packet
decodes to exactly 1024 samples. For this suite's real audio fixture
(a 48kHz stereo AAC-LC MP4), that assumption overcounts: ffprobe reports
753 demuxed packets, but only 752 of them actually decode into a real
audio frame (verified via raw PCM sample count: 770048 samples / 48000Hz
= 16.0426666...s). The old mechanism's 753 * 1024 / 48000 = 16.064s was
therefore wrong; the new decode-based probe correctly derives ~16.043s
from the real final frame.

Regression-shards caught this as a compilation mismatch on the previous
commit. Verified the corrected golden against the real source: full
compilation, visual (0 failed frames), and audio (correlation 0.966)
checks all pass locally with the actual LFS-hosted golden video, and a
standalone compileForRender() check confirms all other audio-bearing
fixture suites in the repo are unaffected.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* perf(engine): seek the audio decode probe instead of scanning the whole file

probeDecodedAudioDuration ran a full, single-threaded decode of the
entire audio stream on every extractAudioMetadata call (once per audio
element in the HTML compiler, plus audio mixing/padding/encoding
steps). On real 30-minute files this took over a second, and could
approach ffprobe's 30s deadline on multi-hour files while holding a
shared probe slot the whole time.

Add -read_intervals so the probe seeks to ~1 second before the
container's already-known claimed duration and decodes only that
tail. This is safe because the correction only ever raises the
duration: an honest or over-claiming container's seek lands at or past
the real end and finds nothing extra, while an under-claiming
container (the only case this exists to fix) still has real frames
past the seek point. If the tail read finds nothing at all, fall back
to a full unseeked decode before giving up, protecting files whose
seek index is unreliable (an index-less or non-seekably-sourced
container) from silently missing a real correction — mirroring the
existing seek-then-fallback pattern already used by the sibling
final-video-frame probe in this file.

Also replaces the real-file regression test's runtime ffmpeg-spawning
fixture generation with two small, committed binary fixtures (an
honest FLAC-in-MP4 file and a copy with its container duration atoms
halved) plus the ffprobe-static package, so this test's real-file
proof actually runs in CI instead of being silently skipped when no
system ffmpeg is on PATH.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* refactor(engine): trim the audio duration probe comments to the invariant

* test(engine): cache tests queue the decode probe and count metadata probes only

* test(engine): probe for ffprobe with a literal argv so the argv contract classifies it

* test(producer): regenerate the speed-ramp golden for the single-encode mix

The audio probe now reads the mixed AAC as exactly 4.000 s instead of the
packet-count 4.032 s (which counted the priming frame), so the final mix
is stream-copied instead of re-encoded once more. Same waveform (sample
correlation 0.9996 vs the old golden), 3.1 dB less true-peak attenuation;
video frames are byte-identical.

* fix(engine): measure decoded audio from the stream start and trust it for raw ADTS

The decoded last-frame end is an absolute timestamp, so subtract the audio stream's start time
(format start as fallback) before comparing it with the start-relative container duration, and
start the tail read at start + duration - 1 s. MPEG-TS audio (1.4 s default start) and offset
MP4s were over-extended by their start time.

Raw ADTS (format aac) has no duration field; ffprobe estimates it from the opening bitrate and
overclaims after a quiet opening. Its decoded timestamps are sample-counted, so for raw ADTS the
decoded end now wins in both directions beyond the margin. Other containers stay upward-only so
an MP4 edit list is never extended.

---------

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-09 21:58:32 +00:00
Abdullah M. Nazif 730a330800 fix(engine): invalidate cached media metadata after file changes (#5258) 2026-10-08 20:04:48 +00:00
Abdullah M. Nazif 95d5494baa fix(engine): preserve out-of-range HDR object positions (#5266) 2026-10-08 20:00:37 +00:00
Miguel Ángel cffc3aa4e1 feat(render): report encoding and assembling progress (#5137)
* feat(render): report encoding and assembling progress

* fix(render): close every progress stage and fail a render whose progress callback throws

* fix(render): send a repeated progress update once

* fix(render): keep Docker terminal output as before and drop the unneeded callback ending

* fix(render): keep one failure stage code per stage whatever its frame counts
2026-10-07 04:29:35 +00:00
Santhi PrakashandMiguel Ángel cad7dbed0d fix(engine): warn when a video input declares alpha but decodes opaque (#3771)
* fix(engine): warn when a video input declares alpha but decodes opaque

- Problem: a video input whose alpha_mode=1 tag outlives its alpha plane (a remux drops the BlockAdditional sidecar while keeping the tag) composites as a solid rectangle, and nothing tells the user their file, not the renderer, is the problem (#3220 / #3226).
- Fix: during extraction preflight, when an input declares alpha (hasAlpha) and its codec can carry alpha, sample the decoded alpha plane (up to 3 frames at 8x8 rgba, bounded 30s, ~768-byte ceiling). If uniformly opaque, emit a non-blocking stderr warning naming the file with the re-export remedy. Inconclusive probes stay silent; opaque full-frame backgrounds are legitimate and never fail the render.
- Verification: 8 new unit tests (byte logic + message); engine suite 1483/1487 with 4 pre-existing failures on clean main (missing hdr-regression PNG fixture in this clone); typecheck/lint/format clean; real-fixture checks: lying-tag WebM warns, genuine-alpha WebM stays silent, both agree with ffmpeg alphaextract.

Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>

* fix(engine): probe input alpha with the decoder that matches its codec

* style(engine): format the alpha probe test

* fix(engine): keep the alpha probe from failing or racing a render

---------

Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-02 04:59:16 -07:00
9d6ca45857 fix: unbreak the regression suite on Windows (PSNR filtergraph escaping + console window popups) (#3839)
* fix: hide console windows spawned by ffmpeg and nvidia-smi on Windows

Node's `windowsHide` defaults to false, so every child process that is a
console application pops a window on the user's desktop. During a render
or a regression run that is a stream of black windows stealing focus.

The repo already sets `windowsHide: true` across cli, studio-server and
most of engine; these were the remaining gaps, all on paths that run
during a normal render:

- core/mediaGradeAnalyzer: ffprobe + ffmpeg, once per analyzed media file
- engine/browserManager: `nvidia-smi` VRAM probe. This one goes through a
  shell (`execSync`), so it flashes a cmd.exe window even on machines with
  no NVIDIA GPU, where the command exists only to fail
- producer: the regression harness ffmpeg runner, PSNR/audio comparison
  helpers, fixture synthesis, and the ffprobe call in audioPadTrim

No behavior change on POSIX, where the flag is ignored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(producer): escape PSNR stats_file for ffmpeg's two-pass filtergraph parse

An ffmpeg filtergraph argument is unescaped twice: once when the graph is
split into filters and their options, then again when the option value is
read. `psnrAtFrames` escaped `\` and `:` a single time, which only survives
the first pass.

On POSIX that is invisible, because mkdtemp under tmpdir() produces neither
character. On Windows the stats path is always `C:\Users\...`, so it reaches
the option parser with a bare `:`, which starts a new option and fails the
whole graph:

    [AVFilterGraph] No option name near '\Users\...\psnr.log'
    Error parsing filterchain '[rv][gv]psnr=...:stats_file=C\:\Users\...'

Every suite that got as far as quality validation died there, so a Windows
regression run reported 0 passes with only 2 genuine compilation failures.
Escaping for both passes fixes it and stays a no-op on POSIX.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(producer): pin PSNR stats path escaping with a colon and backslash path

* fix(producer): keep the PSNR stats path out of the filtergraph

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-02 04:14:36 -07:00
Miguel ÁngelandKenneth Shortrede 3a64ea867c fix(engine): forced-SDR renders show tone-mapped HDR footage as Studio does (#4600)
* fix(engine): forced-SDR renders show tone-mapped HDR footage as Studio does

Forced-SDR extraction tone-maps HDR frames with zscale, but the frames kept the
BT.709 transfer tag, so Chrome re-curved them (a tone-mapped 388d96 painted as
2d8790), and JPEG frames kept the BT.709 matrix every JPEG reader decodes as
BT.601. Tone-mapped frames now get the same sRGB tag and BT.601 JPEG conversion
as other SDR frames.

macOS used VideoToolbox, which does not tone-map: its frames stay PQ and Chrome
tone-maps them itself, darker than the hable curve Linux and Studio use. macOS
now takes the zscale path whenever ffmpeg has zscale, found through the shared
ffmpeg filter probe; without it the old path stays and the render warns once.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): forced-SDR renders tone-map HDR footage that lacks matrix or primaries tags

zscale finds no path between colourspaces without them, so the extraction
failed. Untagged HDR footage is BT.2020 in practice, and now gets those tags.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): forced-SDR tone map also tags the transfer ffprobe read from the container

From ffmpeg 8.1 ffprobe reports a container-only PQ or HLG tag; the frames
still lack it, so zscale failed where macOS VideoToolbox used to render.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* test(engine): gate the container-only transfer case on the ffprobe that reads it

* docs(engine): shorten the filter-probe comment to the comment limit

* fix(engine): keep valid per-frame colour tags when tone-mapping HDR footage

* test(engine): build the mixed HLG and PQ fixture without B-frames so its timestamps are valid

* test(engine): build the trimmed mixed fixture without B-frames too

---------

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
2026-09-28 08:21:13 +00:00
Miguel ÁngelandKenneth Shortrede c9b3d9c962 fix(engine): SDR renders keep footage and page colours as in the source (#4564)
* fix(engine): declare SDR extracted frames as sRGB so Chrome does not darken footage

* fix(engine): encode SDR mp4 frames with the BT.709 matrix they are tagged with

Chrome hands the SDR encoder BT.601 full-range JPEG captures (or RGB PNGs).
The encoders only converted the range, so swscale kept the BT.601 matrix under
the BT.709 tag (ffmpeg 5.1 to 7.0: CSS #C83C28 delivered as #D44824) or
converted YUV to YUV through a low-precision step (ffmpeg 8+: #101010 delivered
as #0B0D0D). Decode to RGB with the capture matrix, then encode with BT.709.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): tag SDR frames plain sRGB so every ffmpeg writes the sRGB chunk

ffmpeg writes a PNG sRGB chunk only for BT.709 primaries and writes cICP only
from 6.1. On ffmpeg 5.1 (Debian bookworm) a BT.601-tagged source therefore got
cHRM plus gAMA 1/2.2, which Chrome decodes as a pure 2.2 power: 0x10 shadows
rendered as 0x07. Declaring BT.709 primaries too gives the sRGB chunk on every
version, so Chrome shows the frame code values for every SDR source. The test
now reads the PNG chunks instead of ffprobe, which reports no transfer for an
sRGB chunk on ffmpeg 5.1.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): extract SDR jpg frames as the BT.601 full range JPEG readers assume

The default frame format for opaque SDR sources is JPEG. ffmpeg converted the
BT.709 source to full-range YUV without changing the matrix, and every JPEG
reader (Chrome included) decodes BT.601, so footage shifted: 0xC83C28 showed as
0xBB2D29. The old mp4 encode made the inverse mistake and hid it; with the
encode fixed, it would reach the output. Decode through RGB with the source
matrix, then encode BT.601 full range, keeping 4:2:0.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): round the SDR mp4 colour conversion accurately

swscale's default rounding moved saturated colours up to 3 codes (pure red 254
delivered as 251 on ffmpeg 5.1 to 7.0); accurate_rnd brings that to 1. The
chunk encoder now decides the HDR-or-SDR filter once, and the 10-bit GPU pad
test sets hdr, which its name always claimed.

* style(engine): keep the colour filter reasons in names and tests, not source comments

The comment ratchet caps a file's comment share: the new filter module and the
extractor carry their intent in constant names, and the encode test states why
the conversion goes through RGB.

* fix(engine): keep colour edges sharp through the SDR colour conversions

Going through RGB upsampled chroma with swscale's default filter, so ffmpeg 4.4
to 7.x smeared colour across edges by about 2 px (red 254 next to blue came out
8 inside the blue). Nearest-neighbour resampling maps the half-resolution chroma
back one to one, keeps the flat-colour results, and makes accurate_rnd redundant.
The pixel tests now also check a red to blue edge.

* test(producer): regenerate goldens whose colours the old mp4 matrix bent

These four fixtures compare at 40 to 45 dB, and their goldens were rendered
with the BT.601-under-BT.709 encode, so the corrected colours score 34 to 37 dB.
Regenerated in the Dockerfile.test image. Each new golden is closer to the
colour its page declares, e.g. #00d4aa was 00bcab and is now 00d3aa, and
#087443 was 006643 and is now 087241.

* fix(engine): keep VFR frame timing when extraction applies colour filters

ffmpeg before 6.1 ignores frame durations in CFR resampling once any -vf is
set, so a VFR window ending on a still lost frames. Filtered VFR extraction
now resamples in one process exactly as before and filters in a second.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): encode raw SDR layered frames in BT.709

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* test(engine): keep the new colour tests inside the format, comment and argv checks

* fix(engine): keep HDR and unusual pixel formats out of the two-process VFR path

The raw handoff dropped HDR10 light levels and misread semi-planar formats, and
decoder options rejected some ffprobe colour names. Only SDR sources in formats
nut carries unchanged now take it, their colour tags come back through setparams,
and a process that cannot start releases the other.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): read untagged footage with the matrix Chrome plays it with

Chrome decodes an untagged matrix as BT.709 from 720 coded lines up and as
BT.601 below; extraction assumed BT.601 at every size, so HD footage without
colour tags shifted hue against Studio's own preview.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): untagged VP9 and AV1 footage keeps BT.601, as Chrome plays it

Chrome applies the 720-line rule only to codecs it decodes through ffmpeg
(H.264, VP8); untagged VP9, VP9 alpha and AV1 play as BT.601 at every size.
Also widens the raw-pipe pixel formats to all verified ones, and reports the
pipeline process that failed first by settle order, not by per-process clocks.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* test(producer): regenerate goldens for corrected page colours and untagged SD footage

* docs(engine): say which untagged codecs the matrix rule was measured on

---------

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
2026-09-27 11:16:35 +00:00
WaterrrForeverandClaude Opus 5 250c9bd509 fix(engine): resolve the version by walking to our own package root (#4409)
readEngineVersion resolved ../../package.json relative to import.meta.url.
That path is correct from packages/engine/src/utils/ in the repo and
overshoots the package root from <pkg>/dist/ in the published CLI, which
bundles the engine. The require threw MODULE_NOT_FOUND, a bare catch
swallowed it, and every render stamped the 0.0.0-dev fallback.

No single relative literal is correct in both layouts, so resolve by walking
up to our own package root instead of by a fixed depth.

The name check is the fix, not a refinement. A bare "first package.json
found while walking up" resolves to whatever sits above in a hoisted install
and returns a confident wrong version - the same class of defect as the
sentinel, and harder to notice because the answer looks plausible. Only a
package.json whose own name is hyperframes or @hyperframes/* may answer.
A test pins this: removing the name check fails that case and only that case.

Also replaces the sentinel. 0.0.0-dev parses as a version, compares against
one, and sorts below every release, so a total resolution failure was
indistinguishable from a dev build - which is why it shipped unnoticed. The
fallback is now "unresolved", deliberately not semver-shaped, and every
failure path warns with the reason. A failed lookup still never breaks a
render.

The old comment claimed the engine ships as raw TS and exports
./package.json. The published package has no exports field and bundles the
engine, so that premise does not hold where the code runs; it is replaced
with the two layouts and why they conflict.

Refs #4365

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-25 02:29:30 +08:00
Miguel Ángel cbaddec51a fix(engine): download tests stop racing real deadlines, and failures name their cause (#4419)
* test(engine): hold the download attempt deadline so slow runners cannot trip it

urlDownloader tests passed a real 1s deadline that a slow Windows attempt outlasted.

The deadline timer is now held; the one test that needs it fires it by hand.

* fix(engine): name the underlying error when a download fails, without its URL

* test(engine): fire the stalled-body deadlines by hand instead of racing a real 20 ms timer
2026-09-24 10:58:53 -04:00
xiayeandClaude Opus 4.6 ee4c00db6a fix(engine): cross-check nb_frames when video stream duration is absent
When ffprobe omits the video stream's own `duration` field,
`extractMediaMetadata` falls back to the container duration
(`format.duration`), which includes the longest stream in the
container. Stock and looping clips commonly have an audio track
2x the video length, inflating `videoStreamDurationSeconds` and
causing the downstream coverage gate to expect twice the frames
FFmpeg can actually produce — the "50% cluster" failure pattern.

Cross-check with `nb_frames / fps` when the stream duration is
absent. If the frame-derived duration is >10% shorter than the
container, use it instead. This corrects the inflation at probe
time without requiring a second ffprobe pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-23 15:37:35 -07:00
Val 30f548f62b fix: a stalled remote asset download no longer hangs a Cloud Run render (#4343)
* fix(gcp-cloud-run): run the Cloud Run image on Bun 1.4.2 so remote downloads cannot hang

* fix(engine): make the download deadline abort the body pipeline
2026-09-23 16:38:58 +00:00
miga-heygenandMiguel Ángel 29378b6c7f fix(engine): classify a single-keyframe video as problematic, not healthy (#3998)
* fix(engine): classify a single-keyframe video as problematic, not healthy

analyzeKeyframeIntervals short-circuited to isProblematic:false whenever
it found fewer than 2 keyframes, but exactly one keyframe is the worst
case for seek performance, not a healthy result: every seek past frame 0
must decode the entire file.

The single-keyframe branch treats the rest of the video stream's own
duration (not a container duration, which a longer audio track can
inflate) as one uninterrupted GOP. ffprobe's frame timestamps are
absolute presentation timestamps rather than relative to stream start,
so a re-muxed or trimmed clip with a nonzero start time is normalized
the same way extractFinalVideoFrameTimestamp already does for the same
quirk. The cache key now includes both values, matching that sibling
function's convention, since a stale cache entry from a different
duration/start pair would otherwise win silently.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* refactor(engine): share stream-window normalization between keyframe check and final-frame lookup

extractFinalVideoFrameTimestamp and analyzeKeyframeIntervals each
independently computed the same stream-start fallback, finite guard, and
cache-key format from the same metadata shape. Extracted resolveStreamWindow
so the two can't drift out of sync with each other.

Also: match ffprobe's real trailing-comma csv output in one keyframe mock,
add a test at the exact 2s threshold boundary, and comment the
noUncheckedIndexedAccess-only fallback at the single-keyframe branch.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* style(engine): trim long comments to one-to-three lines

---------

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-09-23 02:05:51 -04:00
Miguel Ángel 607775bb3a fix(engine): make urlDownloader retry tests independent of wall-clock time (#4155) 2026-09-19 03:07:27 -04:00
Vance IngallsandClaude Fable 5.1 1862c34399 test(engine): fix readonly array type in the pid-filter test
`readonly string[][]` parses as `ReadonlyArray<string[]>`, so `calls.push`
was a TS2339 in 5214552cd. It slipped through because the engine tsconfig
excludes `src/**/*.test.ts` and vitest strips types; caught by running tsc
on the test file explicitly. `Array<readonly string[]>` is what the
neighbouring "one ps call" test already uses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 3cee3b94f7 fix(engine): read /proc/<pid>/status for RSS on linux instead of shelling out to ps
Every render image (packages/cli/src/docker/Dockerfile.render,
packages/gcp-cloud-run/Dockerfile, examples/k8s-jobs/Dockerfile.example,
Dockerfile.test) is node:22-bookworm-slim with no procps, so `ps` is ENOENT
in containers and sampleProcessRss would return [] on every tick: the whole
cloud fleet would report chrome_memory_samples: 0, indistinguishable from
"no Chrome pids". Linux now reads VmRSS from /proc/<pid>/status with a
per-pid try/catch, so one renderer that exited between discovery and
sampling omits only itself (with `ps` a wholly-dead batch fails the call).
No child process per tick on the primary server platform. Same precedent as
the CLI's orphanCleanup.ts / processTree.ts. `ps` stays for darwin and
other POSIX; `tasklist` for win32. The reader is an optional fourth
parameter with a default, so Task 2/3 callers are unaffected.

Scope note: spec section 2.5 records the Target-closed population as
docker=false throughout, so this closes a render_complete completeness gap
on cloud, not a coverage gap on the crash class this phase targets.

Verification: 15/15 vitest; four linux-branch mutants (per-pid catch
rethrows, regex loses /m, kb <= 0 weakened, path /status -> /stat) each
fail at least one test, restored green. Real-process smoke check on darwin:
sampleProcessRss([process.pid, <exited pid>]) returns one sample for the
live pid (BSD ps exits 0 with partial output when a valid-range pid is
gone). The linux path is unit-tested against a /proc/<pid>/status fixture
only; no Linux host or VM was available in this session to execute it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 645cbdce4e test(engine): pin the pid filter, dedup and tasklist memory guard in sampleProcessRss
Review of 76d8e2682 found three surviving mutants: removing the
`Number.isInteger(p) && p > 0` filter, dropping the `new Set` dedup, and
weakening `kb <= 0` to `kb < 0` all passed the plan's 8 tests. The filter is
the guard that matters most: Task 2 feeds `browser.process()?.pid`, and a
single NaN or undefined reaching `ps -p` makes ps exit non-zero, which the
never-rejects catch turns into `[]` for every pid in the sample.

Mutation check: each of the three mutants now fails exactly one test
("drops invalid pids and duplicates before calling ps" for the filter and
dedup, "returns [] when the memory column is not a positive number" for the
tasklist guard); restored source is 10/10 green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 bc70156f00 feat(engine): zero-dependency per-pid RSS sampler (ps / tasklist)
First piece of the long-form render Phase -1 telemetry: the Chrome memory
sampler needs resident set size per pid without adding pidusage or
systeminformation. `ps -o pid=,rss=` covers POSIX in one call for every pid;
`tasklist /FO CSV` covers win32 one pid at a time. The function never rejects
and omits pids that exited between discovery and sampling, so a failed sample
can never reach the capture loop.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:00 -07:00
miga-heygenandMiguel Ángel 0a1c6f5a45 feat(engine): sub-frame multi-sample motion blur, capture-level accumulation (#4019)
Deliverable one of issue #4010: average K sub-frame captures into each output
frame so rotation, scale and opacity blur the way After Effects does, which the
translation-only catalog component (#4008) cannot do. Opt in per render with
CaptureOptions.motionBlur; absent means the capture path is unchanged.

The blocker was quantizeTimeToFrame flooring every seek onto the output frame
grid, which collapsed all K samples onto one instant and made the averaged frame
byte-identical to the unblurred one. quantizeSeekTime now owns that decision at
all three quantize sites (runtime player, its no-timeline fallback, and the init
transport) and refines the grid by an integer subdivision count, which can only
contain the output grid, never move a frame-time instant off it. The finer grid
rounds rather than floors: a sample time round-trips through the page bridge as
a float, and past a few million ticks that error exceeds the floor epsilon.

Accumulation sits inside captureFrameCore AFTER the static-dedup early return,
so a static frame still costs one reused buffer and only moving frames pay K. It
is not a before-capture hook, which would have disabled dedup outright.

Callback invariant: one eventful seek per output frame at the frame time, then K
suppressed sample seeks, then a suppressed restore to the frame time. The
restore matters because a suppressed seek still moves the playhead, so without
it the next frame's eventful seek would swallow the callbacks inside the shutter
window.

Scope of this deliverable: screenshot capture mode and PNG output. beginframe,
drawelement and JPEG encoding are rejected with a named error rather than
silently rendering unblurred. Video content is rejected for the same reason it
is a documented limitation: injected frames are extracted per output frame and
cannot follow a sub-frame seek.

Blend defaults to sRGB, matching AE's non-linearised 8-bpc working space and
the catalog component. Linear light is an explicit opt-in.

Samples are folded into a running average as they are captured rather than held
until the end, so peak memory is one accumulator plus one decoded sample instead
of all K: at 1080p that is a constant 41 MB rather than 141 MB at 16 samples and
540 MB at the 64 maximum, on every frame of the render.

PNG chunk CRC32 now has one owner in utils/crc32.ts, shared by the new encoder
and ffprobe's metadata reader. The encoder previously imported zlib's crc32 by
name, which throws at module evaluation on Node 22.0 and 22.1; ffprobe already
carried the guarded version and the comment explaining why. The sRGB transfer
function is likewise exported once and shared.

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-09-16 20:10:06 -04:00
heygengenesis[bot]andmiguel.sierra 5d2478c6fa fix: report HDR auto-promotion (#3985)
Report HDR auto-promotion consistently from the render media set, including remote media, and make its regression coverage portable across supported paths.

Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com>
Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com>
2026-09-16 06:13:24 +00:00
James Russo 96637aeb1c fix(capture): share the render host policy and block localhost aliases (#3844) 2026-09-10 07:59:30 -04:00
Miguel Ángel db51d5479d fix(ci): generated files never read half-written, lint scaling test runner-proof, probe cache test off disk (#3834)
* fix(build): stop generated files from being read half-written during the build

Two unrelated pull requests kept failing CI for reasons that had nothing to do
with their changes.

Typecheck failed with "TS1002: Unterminated string literal" pointing at a
generated file. The root build compiles several packages at the same time, and
more than one of them regenerates files under a package's src/generated while a
sibling's tsc is already reading them. A plain write empties the file and then
refills it, so for a few milliseconds what is on disk is the first half of the
new file. Whichever build read it in that gap saw a truncated file and stopped.
It never happened locally because locally nothing else is reading.

Every generator now writes the new version under a temporary name and then
renames it over the target, which the filesystem does in one step. A reader
either gets the whole previous version or the whole new one; there is no moment
where it can see half of either. Content that has not changed is not rewritten
at all, so repeat builds no longer touch these files. Four generators were
affected and all four now go through one shared helper, which is where the rule
lives from now on.

The build also rebuilt the core package a second time, in parallel with the
packages that read it. That second rebuild was already redundant, and removing
it takes the writers out of the window entirely. Renaming is what makes the file
safe; dropping the duplicate build makes the build shorter as well.

Separately, a linter performance test failed on four of the last ten failed runs
on main. It timed a scan with a stopwatch and demanded the result come in under
two seconds; one run took 3.7s. That is a statement about how busy the shared
runner was, not about the code, because a stopwatch also counts the time the
machine spent running someone else's job. The test now counts only the processor
time this process actually used, and checks that the cost grows in step with the
input rather than against a fixed number of milliseconds. Measured on a machine
under load, the stopwatch ratio for the same input reached 45x while the
processor-time ratio stayed at 20x.

Both fixes come with a check that fails if the fix is removed.

* fix(build): keep the shared write helper inside the core package

The helper the generators call had been placed in the repo-root scripts folder.
Every container image that builds a package copies the packages folders whole
but cherry-picks root scripts one file at a time, so the image builds failed on
a missing module. The repo already shows both halves of that convention: the one
root script a package build imports has a matching copy line in the image, and
cross-package imports into the core package need none.

The helper now lives with the core package's other build scripts, and the one
generator outside that package reaches it the way the engine package already
reaches core.

* test(engine): keep the probe cache bound test off the filesystem

The eviction test wrote, stat'ed and deleted 129 temp files to exercise an
in-memory LRU rule. Its runtime tracked filesystem contention rather than
the code under test, and on a busy Windows runner the file churn alone
pushed a ~300ms test past the 5s timeout, failing unrelated pull requests.

Cache identity comes from stat, so the test now synthesises stat results
and never touches disk. While here, the test also asserts the LRU touch:
re-probing an entry before the bound is hit must keep it resident and
evict the next-oldest one instead. The previous shape never hit the cache
during the fill, so that branch was untested.

* test(lint): keep the scaling check inside the scanner's linear range

The CPU-ratio version sampled 320k characters, where the output string's
own growth dominates and the whole test cost seconds of CPU; on a shared
runner that tripped the default test timeout, the failure this change
exists to remove. Sample 10k and 80k characters instead, where 8x input
measures ~8x and a quadratic scan still measures 60x or more, and give
the test an explicit timeout so a slow runner reports the ratio.

* test(core): snapshot file identity through a descriptor

The before/after inode and mtime checks read the file through a path stat
and then exercised the writer on the same path, which reads as a
check-then-use race to static analysis. Snapshot through an open
descriptor instead; the assertions are unchanged.
2026-09-10 03:08:58 +00:00
Vance Ingalls b22afd6bbc fix(engine): reject malformed JPEG input before encoding 2026-09-08 23:51:00 -07:00
Miguel Ángel e1fda7866e fix(engine): use supported VideoToolbox quality options (#3687) 2026-09-08 18:12:07 +00:00
James Russo e5d1d9bf0e fix(engine): avoid transform regex backtracking (#3704) 2026-09-05 11:53:43 -04:00
James Russo c0887b650d fix(producer): transport safe extraction failure metadata (#3592)
* fix(producer): transport safe extraction failure metadata

* refactor(producer): generalize public error metadata

* test(producer): use vendor-neutral media hosts
2026-09-01 21:35:40 -04:00
James Russo 0f7eebd7e4 fix(encoder): signal host interruptions for retry (#3578)
* fix(encoder): signal host interruptions for retry

* fix(encoder): cover all render interruption paths

* fix(encoder): classify HDR pre-extraction drains
2026-08-31 22:07:43 -04:00
miga-heygenandSanthi Prakash 05275c1e8c fix(producer): assert render artifact duration and frame count before commit (#3506)
* fix(producer): assert render artifact duration and frame count before commit

Refuse to publish an artifact that is significantly shorter or has fewer frames

than the capture pipeline just reported. Adds a duration/frame-count gate on top

of the existing readable-non-empty check inside ArtifactTransaction.validate(),

keyed off the values the orchestrator already carries. Closes #3395.

* fix(producer): wire ffprobe frame count into the artifact duration probe

The frame-count gate added in #3395 accepts an expectedFrames value from
the orchestrator, but defaultArtifactDurationProbe was still returning
only durationSeconds - so the wire was half-built and the assertion
short-circuited on undefined for every real render. Forward meta.frames
from ffprobe so the field-packet case the issue names (container duration
correct, stream shorter) is actually caught by the frame-count check,
not just the duration one.

extractMediaMetadata now populates a new frames field from the video
stream's nb_frames tag, returning undefined when the demuxer did not
report one (fragmented MP4, malformed streams, muxes that require
-count_packets). Callers that gate on the count must treat undefined as
no answer; the assertion already does.

The previous CI run (#32589981916) cancelled shard-6 at the 1h job
timeout after bun install failed to extract the aws-cdk-lib tarball
mid-Docker-build - a cache flake, not a code regression. Pushing a
follow-up commit retriggers CI against the now-populated cache layer;
the regression should clear without further code changes.

---------

Co-authored-by: Santhi Prakash <b.santhiprakash@gmail.com>
2026-08-28 00:19:27 +00:00
Vance IngallsandClaude Opus 4.7 073b098e21 feat(engine): preflight psnr filter availability and force-fallback to screenshot on missing (#3418)
## What

Adds a one-shot ffmpeg-psnr filter probe at drawElement session bootstrap.
When the resident ffmpeg is missing or lacks libpostproc (no `psnr` filter),
the capture-session router now force-fallbacks to the screenshot capture
path and emits a `de_gate_reason = "ffmpeg_no_psnr_filter"` telemetry
signal via the existing `render_complete` breakdown.

Also tightens `psnrForDiskSample`'s catch: infrastructure-class ffmpeg
failures (ENOENT, "No such filter") no longer silently skip the sample —
they abort the render so the safety net cannot fail-open post-preflight.

## Why

The drawElement self-verify safety net (parallelCoordinator's
`psnrForDiskSample` → `psnrDb`) shells to `ffmpeg -lavfi psnr`. If ffmpeg
is missing, or was compiled without libpostproc (so the `psnr` filter is
absent), every per-sample compare throws. The existing catch swallows the
error and returns `null` — callers treat that as "skip this sample" and
the render completes with the safety net inoperative.

Field signal (⭐ 9/10 CLI feedback, Slack ts=1787380767.210079,
hyperframes 0.8.7, darwin/arm64, tid=93ff9910-2207-45c2-bc1f-54c0b347d4fe):

> "host ffmpeg lacked psnr filter used by drawElement self-verification,
> but render completed."

The user's frames happened to be byte-identical so no visual damage
shipped — but the safety net silently wasn't running. Any future
compositor-damage bug on that host would have shipped straight through.

## How

Two-part fix, both in `packages/engine`:

1. New `utils/psnrFilterAvailability.ts` — cached probe that runs
   `ffmpeg -hide_banner -filters` once per process and word-boundary-
   matches `psnr` in the output. Any failure (ENOENT, non-zero exit,
   timeout, unparseable output) returns `false`; never rejects.

2. Wired into `services/frameCapture.ts` `initDrawElementOrTransparentBackground`
   right after the Chrome capability probe: when useDrawElement resolves
   true and the preflight returns false, set
   `session.deGateReason = "ffmpeg_no_psnr_filter"` (same low-cardinality
   bucket every other DE gate uses; flows through `getCapturePerfSummary`
   → `render_complete.de_gate_reason` in PostHog), emit a stderr warning
   naming what's missing, and call `routeToFallback()` — the same
   fail-graceful shape as the SwiftShader / CSS-effect / at-risk-timeline
   gates. Skipped under `HF_FORCE_DRAWELEMENT=1` (matches the diagnostic
   knob's policy of bypassing every other gate).

Belt-and-braces: `psnrForDiskSample` now discriminates infrastructure-
class failures (ENOENT / "No such filter" / "Unknown filter") from
per-sample noise (readFile races, transient EPERM). Only the former
re-throw — per-sample noise still returns `null` (skipped sample). The
preflight normally catches this at bootstrap; the re-throw covers
ffmpeg-swapped-mid-render.

## Test plan

- [x] Unit tests added:
  `packages/engine/src/utils/psnrFilterAvailability.test.ts` — mocked
  `execFile` covers: `psnr` present → true; `psnr` absent → false; ENOENT
  → false; non-zero exit → false; result memoized + reset works;
  substring-not-word-boundary → false.
- [x] Unit tests added:
  `isFfmpegInfrastructureFailure` in
  `packages/engine/src/services/parallelCoordinator.test.ts` covers
  ENOENT, "No such filter", "Unknown filter", per-sample EACCES, parse
  errors, null/non-object.
- [x] `bun run test` — `packages/engine/src/utils/psnrFilterAvailability.test.ts`
  (6 tests) + `packages/engine/src/services/parallelCoordinator.test.ts`
  (50 tests) + `frameCapture.test.ts` (26 tests) all pass. Pre-existing
  ffprobe test failures (4) on the base commit are unrelated (missing PNG
  fixture bytes — the file is 129 B on disk, likely LFS-stored).
- [x] `bunx tsc --noEmit -p packages/engine/tsconfig.json` — clean.
- [x] `bunx oxlint <files>` — 0 warnings, 0 errors.
- [x] `bunx oxfmt --check <files>` — clean.

Not covered here: an integration test that boots
`initDrawElementOrTransparentBackground` end-to-end. That path is
Puppeteer-driven and has no unit-scale bootstrap harness in the
repository — the pure preflight + pure discriminator coverage above are
what this PR can prove at the vitest layer.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-22 04:23:34 -07:00
Miguel Ángel 315a7b758c fix(engine): hide ffmpeg console windows on Windows (#3381)
ffmpeg and ffprobe are console-subsystem binaries and Node defaults
windowsHide to false, so every spawn opened a visible console window on
Windows. A render shells out dozens of times across parallel workers,
which flashed a burst of windows across the user's desktop.

Applied at every production spawn site rather than only the two named in
the report, since they all share the cause: runFfmpeg, both gpuEncoder
probes, ffprobe, streamingEncoder, audioExtractor and the distributed
version check. windowsHide is a no-op on macOS and Linux.

The dev-only parity and regression harnesses are left alone; they never
run on a user's desktop.

Closes #3379
2026-08-21 11:37:21 -04:00
James Russo f7d2260f9d feat(engine): stamp rendered files with hidden renderer provenance (#3264)
* feat(engine): stamp rendered files with hidden renderer provenance

* fix(engine,producer): re-assert provenance at every container writer

Review found that a no-audio MOV render still shipped untagged. The concat
step is the last container write on that path (mux is skipped without audio,
and applyFaststart only copies mov/webm), and the concat demuxer does not
carry the chunks' container metadata through.

The same hole applies to no-audio WebM, and to the in-process chunked encode
in chunkEncoder, not just the distributed assemble path. mp4 was masked
throughout because applyFaststart re-runs ffmpeg for that format and re-tagged
the output.

Tags the four remaining writers: the chunked-encode concat, and assemble's
single-chunk remux, concat and cfr re-encode.

Also corrects the trust claim. These are unsigned, freely writable keys, so a
present tag means the file claims to be HyperFrames output, not that
HyperFrames wrote it. Documented as an unauthenticated diagnostic hint rather
than an authenticity or attribution boundary.

Tests assert on the assembled file through the real assemble() path for both
mov and webm; both fail without the concat fix.

* test(engine): pin provenance through the in-process chunked concat

Review noted the distributed writers are mutation-pinned but the
encodeFramesChunkedConcat fix had no real-file regression of its own.

Encodes 70 frames at a 30-frame chunk size so the concat step actually runs,
then asserts the tags on the resulting no-audio mov. Fails without the concat
fix, passes with it.
2026-08-13 16:32:09 -07:00
Vance Ingalls c03cc2c52c Merge branch 'main' into via/studio-5433-html-sniff-defense
Both conflicts were import/export unions in the engine package, resolved by
keeping both sides:

- packages/engine/src/index.ts — main widened the urlDownloader re-export
  (fetchPublicHttpsText, safeDownloadUrlIdentity, writeUrlDownloadTelemetry and
  their types) while this branch added the notMediaPayload exports.
- packages/engine/src/services/audioMixer.ts — main added UrlDownloadError and
  writeUrlDownloadTelemetry to the urlDownloader import; this branch added
  isNotMediaPayload.

In audioMixer's prepare path both intents compose in order: main's download
telemetry and typed download failure, then the STUDIO-5433 non-media sniff
before the probe.
2026-08-07 13:51:49 -07:00
Vance IngallsandClaude Opus 5 349c066a83 fix(producer): classify JSON error bodies as non-media sources too
A source that answers with a JSON error body still reached ffprobe and
produced `moov atom not found`. Replicate returns
`{"detail": "requested file not found"}` for a dead asset, and a gateway
in front of it can relay that body with a success status.

The sniff now treats `<`, `{`, or `[` as the opening byte of a text
document. No supported container starts with any of them, so this is the
same trade as before: three bytes instead of an allowlist that grows one
entry per payload shape observed in production.

Renamed accordingly, since the class now covers JSON as well as markup:
MARKUP_NOT_MEDIA -> NOT_MEDIA_PAYLOAD, MarkupNotMediaError ->
NotMediaPayloadError, markupPayload.ts -> notMediaPayload.ts. Registry
entries in the Lambda name map, the CDK and SAM plan lists, the Cloud Run
set, and SAFE_RENDER_ERROR_CODES move with it.

Also documents the reachability boundary on the error class: only a 2xx
response gets here. `downloadToTemp` rejects 404/410 as `http_not_found`
before writing a byte, and every ffprobe input is local because
videoFrameExtractor downloads http srcs first. So the shapes this
classifies are soft-404 and interstitial HTML, S3/CloudFront error
documents, and JSON API error bodies -- each served with a success
status. A genuine 404 surfaces as a download failure, not as this error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 03:16:11 -07:00
James Russo bbdfee1166 fix(engine): validate remote download integrity (#2938) 2026-08-04 18:02:01 -07:00
Vance IngallsandClaude Opus 5 f3689c1481 fix(producer): scope and harden the markup-payload sniff
Review follow-up on the STUDIO-5433 defense.

Correctness

- The sniff ran above the documented video/audio failure split, so an
  <audio> src that resolved to an HTML payload aborted the whole render
  instead of degrading to duration 0. It now runs inside the same try, so
  video surfaces the typed error while audio still drops out, with a
  warning naming the element.
- Raw fs errors (EISDIR on a directory src, EACCES, the existsSync->open
  ENOENT race, EMFILE) escaped and failed the compile with an unclassified
  error carrying an unredacted temp path. The sniff is now a classifier that
  never throws: an unreadable file reports "not markup" and the real probe
  produces the real error.
- Elements whose duration the compiler never resolves (a data-end video, a
  looping audio) skipped the sniff entirely, so the original ffprobe error
  still escaped, and looping audio was reported as owner "system" after
  every frame had been captured. Video is now caught in the asset preflight,
  which sees every local src regardless of authored timing; audio is
  classified per-element in audioMixer as source/invalid_media/owner "user",
  keeping audio failures non-fatal as they already were.
- Detection is a byte-level check for a leading "<" (BOM-, whitespace- and
  NUL-tolerant, looped read) instead of a <!doctype|<html|<?xml string
  prefix, which missed a NUL-prefixed payload, >256B of leading whitespace,
  UTF-16-encoded HTML, and a prolog-less <svg. No supported container starts
  with "<", so the allowlist no longer grows per payload shape.
- finally { await fh.close() } could replace the in-flight typed error with
  the close error.

Routing and privacy

- MARKUP_NOT_MEDIA is now in SAFE_RENDER_ERROR_CODES, the Lambda terminal
  name map, the CDK and SAM non-retryable plan lists, and the Cloud Run
  non-retryable set, and the class carries owner/retryable. Previously the
  API emitted errorCode: undefined and a deterministic authoring bug burned
  the full distributed retry budget.
- The message no longer carries 32 raw payload bytes or the src.
  redactTelemetryString preserves host and path for HTTP srcs, so
  per-tenant CDN paths reached a message the server forwards to clients.
  Correlation is a sha256 element fingerprint, matching
  AssetMediaTypeMismatchError.
- The message names both causes (unresolved nested-composition URL, or an
  HTML/XML error page served as 200) rather than misdiagnosing an S3 403
  body as an authoring bug.

Tests

- Byte-level detection is unit-tested in engine: markup shapes, BOMs,
  UTF-16, nine container signatures, unreadable inputs.
- Replaced the tautological assertions. The old checks for "html" in and
  "moov" absent from a fixed message template could not fail for any input.
- New coverage for audio degradation, the audioMixer classification, the
  preflight video/image/audio split, and the API error metadata.
- The sibling htmlCompiler.mediaType failure was a vitest-under-bun runner
  mismatch, not a missing ffmpeg binary. It passes, including the 4-wide
  probe-semaphore invariant the sniff now runs inside.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 17:38:51 -07:00
James Russo 9792c32950 fix(producer): reject asset media type mismatches (#2937)
* fix(producer): reject asset media type mismatches

* fix(engine): document read-only AVIF probe

* fix(engine): bound read-only AVIF brand probe

* fix(producer): make media preflight lifecycle-safe

* fix(producer): reconcile runtime media before preflight

* fix(engine): avoid writable file-open detection

* fix(producer): close runtime media preflight gaps
2026-08-03 18:16:41 -07:00
James Russo 2339757377 fix: bound HDR and video extraction resources (#2955)
* fix: bound HDR and video extraction resources

* fix: trim negative video extraction preroll

* fix: skip invisible video extraction windows

* fix: preserve negative-start loop and held tails

* fix: cap finite video slots to source duration

* fix: bound held-tail frame extraction

* fix: plan from playable video duration

* fix: preserve open-ended held video tails

* fix: resolve held tails from decoded frames

* fix: normalize final-frame probe timestamps

* fix: handle unseekable final-frame sources

* fix: dedupe final-frame probes per render

* refactor: clarify output dynamic range contract
2026-08-02 20:21:54 -07:00
Vance IngallsandClaude Opus 5 9e275423e2 fix(engine): reject stdin input, decode stdout correctly, bound its size
Three issues in runFfprobe's process and stream handling.

A filePath of exactly "-" hung for 30 seconds. `--` stops option parsing,
so "-intro.mp4" is safe, but ffprobe rewrites "-" to `fd:` AFTER option
parsing and reads stdin — and stdin was an inherited pipe the parent
never writes to and never ends. The probe ran to the deadline and failed
with an empty diagnostic, because ffprobe never errored so stderr was
blank: 30010 ms and no message, against 28 ms for a normal missing-file
error. Rejected up front, and the child now gets stdio ["ignore", ...]
so no future invocation can block on stdin either.

stdout was decoded per chunk. `stdout += data.toString()` decodes each
64 KiB pipe chunk independently, so a multi-byte character straddling a
boundary became U+FFFD on both sides — verified: 200 KB of 3-byte
characters produced 15 replacements and a string 9 characters longer
than the source. -show_format output above ~64 KiB with non-ASCII tag
text returns silently mangled values, since JSON.parse still succeeds.
Now accumulated through StringDecoder.

Note on testing that one: U+FFFD is valid JSON string content, and
nothing on extractMediaMetadata's public surface exposes a tag value, so
there is no assertion that fails against the old implementation. Rather
than add a test that cannot fail, it is stated here and the bound below
is what the new test covers.

stdout was unbounded. stderr is capped by ManagedChildProcess but stdout
was not, and analyzeKeyframeIntervals emits one line per frame — an
all-intra ProRes proxy can produce an arbitrarily large string. Capped
at 8M characters, which real -show_streams JSON is nowhere near.

Tests: "-" rejected without spawning, the stdio shape, and the size
bound. Reverting the stdin guards fails 1. The first draft of the bound
checked before appending, so a single oversized chunk passed — the test
caught it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 19:53:01 -07:00
Vance IngallsandClaude Opus 5 361fd49926 fix(engine): allowlist AAC-LC for the packet refinement, not deny HE-AAC
The previous gate was a HE-AAC DENYLIST, so every other profile still
got the 1024-sample formula. ffprobe reports codec_name "aac" for all of
them; the framing lives in the profile:

  LC            1024 samples/frame   <- the only one this maths fits
  HE-AAC v1/v2  2048 output samples against a doubled sample_rate
  LD            512
  ELD           480
  Main/SSR/LTP  1024 nominally, unverified here
  xHE-AAC       variable

LD and ELD therefore had their already-correct container duration
overwritten with a value 2x / ~2.13x too large, and an unknown or
missing profile fell through — so an unrecognised HE spelling preserved
the exact truncation the previous commit set out to close.

Now an affirmative match on LC. Skipping the refinement is harmless:
format.duration is already correct before it runs.

Tests: 11 non-LC profiles (including LD, ELD, xHE-AAC, empty and
unrecognised) assert the container duration is kept AND that the second
probe is not launched; LC still refines, with whitespace tolerated. The
pre-existing duration table asserted that an UNPROFILED "aac" stream
refines — the behaviour under review — so it now states LC explicitly
and adds an unprofiled row that must not refine.

Also strengthened the `--` separator test while it was failing: it
compared a flattened count of 3 across three spawns, which one call
emitting three terminators would satisfy. Now asserts the last two argv
entries per call.

Reverting the allowlist fails 8.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 19:52:26 -07:00
Vance IngallsandClaude Opus 5 242a42f6c9 fix(engine): make the AAC duration refinement safe, cancellable and LC-only
The packet-count probe is a refinement — durationSeconds is already
correct from format.duration before it runs — but it was written as if
it were load-bearing.

It could fail the whole call. No try/catch, and `-count_packets` demuxes
the entire container against runFfprobe's fixed 30s deadline, so a long
AAC file on slow or network storage timed out and extractAudioMetadata
rejected. htmlCompiler catches that under the comment "Source file has
no audio stream", returns duration 0, drops the audio element, and the
render ships silent with no warning. Now caught, keeping the container
duration.

It ignored the caller's AbortSignal. Only the first probe received it,
so aborting during the packet probe let the child run to completion and
the call resolved with full metadata after cancellation — while
audioPadTrim's comment claims the wrapper preserves cancellation. The
signal is forwarded, and an abort still propagates rather than being
swallowed as a refinement failure.

It halved HE-AAC durations. ffprobe reports codec_name "aac" for
HE-AAC v1/v2 as well — the marker is in the profile field — and with SBR
each packet carries 2048 output samples against the doubled output
sample_rate, so the 1024 assumption computed exactly half. A 10:00
podcast became 5:00 and htmlCompiler truncated the audio there. Gated on
profile, with `profile` added to FFProbeStream.

Tests: probe failure, junk output, three HE-AAC profile spellings (which
also assert the second probe is not attempted), and that plain AAC-LC is
still refined. Reverting the guards fails 5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 19:52:25 -07:00