Commit Graph
117 Commits
Author SHA1 Message Date
Miguel Ángel 54bcdc616f fix(producer): a local render extracts each video at about the size it is shown (#7609) 2026-11-12 17:16:59 +00:00
Miguel Ángel 395c845cd3 fix(engine): render work is sized by the CPUs the process may use (#7612)
* fix(engine): render work is sized by the CPUs the process may use

* fix(engine): keep the usable CPU count comment under the line limit
2026-11-12 17:07:45 +00:00
Miguel Ángel cc47f7ea74 fix(engine): a video shown at exactly half its size is extracted at half size (#7607) 2026-11-12 14:00:10 +00:00
Michael WangandClaude Opus 5.5 60fb9d2c5e fix(producer,engine): harden render paths, render slots and local listeners (#7508)
* fix(producer): build the render job before taking a render slot

A request whose options fail the strict render-request validator now gets
a 400 on both the blocking and streaming routes, and its inline project
directory is removed. Validate workers up front like the other overrides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): write compiled sub-compositions only inside the compiled directory

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): keep media work files inside their directory for any element id

Map a media element id to one plain path segment before it names a frame
directory or an intermediate audio file. Ids that already are one plain
segment map to themselves, so existing paths stay the same; the producer
and the distributed plan derive the same segment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): listen on loopback by default in the engine file server

The file server feeds the co-located capture browser, which loads it from
localhost. A caller that wants another interface passes hostname.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): refuse cross-origin browser posts to the standalone render server

A web page on another origin can no longer start a render or lint through
the server's POST routes. Requests without an Origin header (CLI tools,
server-to-server callers) and same-origin pages are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(engine): trim comments on the media path segment and file server host

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): trust the browser's Sec-Fetch-Site verdict for render server posts

A same-origin page behind a reverse proxy that rewrites the Host header sent an
Origin that no longer matched Host, so the cross-origin guard refused it. The
guard now uses Sec-Fetch-Site when the browser sends it and falls back to the
Origin and Host comparison only for browsers that do not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): keep generated media path names apart from plain ids and within filename limits

A plain media id that already ends like a generated name now takes the hashed
branch too, so it can no longer share frame folders or audio work files with
the unsafe id that produced that name. The readable part of a generated name
is capped at 64 UTF-8 bytes before the digest, so long or multibyte ids that
contain separators stay within the 255-byte filename limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 20:26:24 +00:00
Miguel Ángel 235c80bae0 fix(engine): a video clip shows a new source frame on every frame it is on screen (#7471)
* fix(engine): a clip shows a new source frame on every output frame it covers

* refactor(engine): shorten the frame count doc to two lines

* style(engine): collapse the single-pass ffmpeg call the formatter wants on one line
2026-11-10 02:16:59 +00:00
Miguel Ángel 4d0fa2baa8 fix(engine): short clips from Apple-encoded videos keep their last frame (#7469)
* fix(engine): short clips from Apple-encoded videos keep their last frame

* refactor(engine): name the end-of-source hold and frame bound instead of commenting them

* fix(engine): frame ranges past a source's end stay empty with the held final frame

* test(engine): move the final-frame test into its own file
2026-11-10 01:38:38 +00:00
Miguel Ángel b559a98d37 fix(engine): hlg footage with a large header sei also tone-maps at hlg's white (#7349)
* fix(engine): hlg footage with a large header sei also tone-maps at hlg's white

* fix(engine): say in the read's own doc line why it streams

* test(engine): the header-SEI HLG fixture encodes without b-frames, so it muxes on windows
2026-11-07 21:46:28 +00:00
Miguel Ángel 5ee9e928d5 fix(core): hlg footage tone-maps to sdr at the bt.2408 reference white (#7234)
* fix(core): hlg footage tone-maps to sdr at the bt.2408 reference white

* fix(core): an HLG clip whose stream leaves its transfer unset keeps HLG's white

An SPS with an unspecified transfer takes the stream's tag, as its frames do; a failed keyframe read is not cached; the grey test needs ffmpeg 8.
2026-11-05 13:04:56 +00:00
Miguel Ángel ac1948a88b fix(engine): a full disk during frame extraction says so, not that the video is broken (#7161)
* fix(engine): a full disk during frame extraction says so, not that the video is broken

* fix(engine): a pass that filled the disk is not decoded again in software
2026-11-04 14:45:56 +00:00
Miguel Ángel 58428b44bc fix(engine): passthrough keeps frame count and accepts films whose audio outlasts the picture (#6920)
* fix(engine): passthrough output follows the page frames and holds a short video's last frame

* test(engine): a film whose audio outlasts its video still qualifies for passthrough

* test(engine): passthrough encodes one output frame per page frame when the video is shorter

* test(engine): end the ffprobe options before the file in the passthrough frame count test
2026-11-01 10:42:32 +00:00
Miguel Ángel 98520366a6 feat(producer): opt-in video layer passthrough for a lone full-canvas video (#6914)
* feat(engine): composite a source video inside the streaming encoder

* feat(engine): opt-in flag and rule for compositing a lone full-timeline video

* feat(producer): opt-in video layer passthrough for a lone full-canvas video

* feat(producer): launch the probe, drop its page and capture PNG when passthrough is on

* refactor(engine): trim the passthrough doc comments to one line

* fix(producer): refuse passthrough when a wrapper or sub-composition paints a background

* chore(producer): mark the in-page layout check as one deliberate unit

* fix(engine): a video left out of capture no longer blocks static-frame dedup

* refactor(engine): shorten two long comments in the static-frame eligibility scan
2026-11-01 05:17:00 +00:00
Miguel Ángel 94849bf1e0 refactor(engine): one builder for the video frame filter chain (#6908)
* refactor(engine): one builder for the video frame filter chain

* refactor(engine): keep the filter chain builder module-private until it has a caller
2026-11-01 00:54:54 +00:00
Miguel Ángel 32de0196d5 fix(producer): a failed video extraction names which clip files failed (#6905)
* fix(producer): a failed video extraction names which clip files failed

* fix(producer): the extraction source label hides scheme payloads and drive paths

* style(producer): format the extraction source label
2026-11-01 00:09:14 +00:00
Miguel Ángel 47fe68562f fix(engine): nested video clips on a frame boundary export every source frame (#6893)
* fix(engine): nested video clips on a frame boundary export every source frame

* fix(engine): drop comment from ffmpeg time formatter
2026-10-31 20:08:23 +00:00
Miguel Ángel 60f711686c fix(producer): auto renders with color-graded media stay SDR instead of promoting to HDR (#6671)
* fix(producer): auto renders with color-graded media stay SDR instead of promoting to HDR

* fix(producer): keep auto renders SDR when a graded canvas is present
2026-10-31 16:16:06 +00:00
Miguel Ángel 536209da07 fix(producer): report extract_video progress while ffmpeg extracts frames (#6822)
Frames ffmpeg has written now reach the render's stage progress, so a long source video no longer leaves the render silent between extract_video and capture.
2026-10-30 11:23:59 +00:00
Miguel Ángel 10a4ed56cb fix(engine): a video clip starting between frames shows its media on the right frame (#6759)
* fix(engine): a video clip starting between frames shows its media on the right frame

* chore(engine): trim the helper comment to one line

* chore(engine): drop the helper comment

* test(engine): extract once per start in the off-grid clip test

* fix(engine): a looped clip keeps its loop origin when the start is off the frame grid

* test(engine): three off-grid starts cover the failure modes and keep the flake gate under its limit
2026-10-30 09:01:21 +00:00
Miguel Ángel 54c98b5906 fix(engine): a video on a non-square pixel grid renders at its display shape (#6772)
* fix(engine): a video on a non-square pixel grid renders at its display shape

* chore(engine): drop comments the names already carry

* fix(engine): a rotated clip with a non-square pixel grid keeps its display shape

* test(engine): the frame probes end their options with -- like every ffprobe call
2026-10-30 03:00:25 +00:00
Miguel Ángel a3ff1c4902 revert(engine): late-starting videos play to their last frame (#6746)
Reverts #6736. Its end-of-clip read fails intermittently on Windows, and the fallback
then expects frames past the clip's end.
2026-10-29 14:14:21 +00:00
Miguel Ángel 22b53c0115 fix(engine): late-starting videos play to their last frame (#6736)
* fix(engine): late-starting videos play to their last frame

A video whose first frame is after timestamp 0 could end early (frozen
tail) or late: ffprobe's stream duration does not run from the first
frame in every container (fragmented MP4, Matroska). Such a video's end
is now read from its last packet, once, where its facts are probed.

* test(engine): the WebM-with-audio case measures its first frame from the file

* test(engine): the first-frame probe ends its arguments with the input file
2026-10-29 12:05:27 +00:00
Miguel Ángel 437784038b fix(engine): late-starting MP4 and WebM videos render every frame the preview shows (#6664)
* fix(engine): late-starting MP4 and WebM videos render every frame the preview shows

The browser counts an MP4 or WebM from timestamp 0, but FFmpeg seeks from the file's
earliest stream start. A file whose first frame comes after 0 (a fragment cut from a
longer stream, an edit-list offset, a WebM with a start offset) was extracted early and
short, so renders tripped the video coverage gate. Every video seek now maps media time
to FFmpeg's timeline, and the video begins where the browser shows it.

* fix(engine): audio from late-starting MP4 and WebM plays when the preview plays it

The mixer seeked and trimmed audio by FFmpeg's clock, which starts at the file's earliest
stream. In an MP4 or WebM whose streams all start after 0, the browser plays that audio
later, so the render heard it early by the same amount. The mixer now maps media time onto
FFmpeg's clock with the same offset frame extraction uses.

* fix(engine): a late-starting file's picture shows on the clock its audio plays on

* test(producer): vfr-screen-recording baseline shows each frame at its timestamp
2026-10-28 13:09:53 +00:00
Miguel Ángel 313792408d fix(producer): HDR sources are tone-mapped for formats that cannot carry HDR (#6597)
* fix(producer): HDR sources are tone-mapped for formats that cannot carry HDR

* refactor(producer): one owner for which output formats can carry HDR

* refactor(producer): drop comments the helper names already carry

* fix(engine): SDR clips are not lifted to HDR when the render tone-maps to SDR
2026-10-27 23:09:15 +00:00
Miguel Ángel 3b2058a2b7 fix(engine): extraction waits out the decode up to a deep seek in a long-GOP video (#6583)
* fix(engine): extraction waits out the decode up to a deep seek in a long-GOP video

* chore(engine): shorten the keyframe gap doc comment

* fix(engine): cancelling an extraction during the keyframe probe reports a cancel

* fix(engine): a segment stopped by another's failure reads as stopped, not cancelled
2026-10-27 20:09:05 +00:00
Miguel Ángel 019b17b4b7 fix(engine): a failed video extraction says what ffmpeg reported, without paths or URLs (#6527) 2026-10-27 07:29:03 +00:00
Miguel Ángel 3ec054ab6f fix(engine): renders on a Mac decode a clip in software when the media engine refuses it (#6488)
* fix(engine): renders on a Mac decode a clip in software when the media engine refuses it

* refactor(engine): name the software decode retry condition instead of commenting it
2026-10-27 02:52:25 +00:00
Miguel Ángel e3e93aa7fe fix(core): paused preview and render show the same, correct video frame (#6432)
* feat(core): one function gives the media time that shows the frame starting at an instant

* fix(core): paused preview and render show the same, correct video frame

* fix(core): register the media-clock subpath in its descriptor and keep the reason beside the test

* fix(core): media seeks lead by 10 us so Chrome's timestamp rounding never shows the prior frame

* fix(core): generated pages lead media seeks by a literal, on every generator branch
2026-10-26 17:48:51 +00:00
Miguel Ángel 37f14fdcb1 fix(snapshot): video frames keep the colours render gives the same footage (#6425)
* fix(snapshot): video frames keep the colours render gives the same footage

* fix(snapshot): say so once when video colours cannot be matched to render

* test(snapshot): cut the HEVC fixture's encode time so the flake gate stays inside its timeout

* style(snapshot): format the HEVC fixture line
2026-10-26 17:27:13 +00:00
Miguel Ángel e51ec11caa perf(engine): on a Mac, video frames for a render decode on the media engine (#5878)
Frame extraction decoded every source in software, so a long 4K60 HEVC clip
spent much of its CPU on decoding. On macOS, extraction now asks ffmpeg for
VideoToolbox when this ffmpeg has it and the source cannot carry alpha; ffmpeg
falls back to software for a stream VideoToolbox cannot decode, and the frames
written are byte-identical.
2026-10-26 01:33:24 +00:00
Miguel Ángel d49e527b32 fix(engine): ffmpeg steps and downloads stop on no progress, not while still advancing (#6258)
* fix(engine): ffmpeg steps and downloads stop on no progress, not while still advancing

* test(cli): an ffmpeg step that stops advancing reports inactivity, not a deadline
2026-10-25 23:04:13 +00:00
Miguel Ángel 382a14d73e perf(engine): vp9 and vp8 sources without an alpha plane extract as jpeg (#6284)
A VP8/VP9 source with no alpha tag has one frame decoded with the alpha-aware decoder when its
metadata is read; if that frame has no alpha plane, its frames extract as jpeg instead of png.
Tagged alpha, a decoded alpha plane, a failed probe, or a plan written without the field keep png.
2026-10-25 08:32:49 +00:00
Miguel Ángel da4386f0d0 perf(aws-lambda): cloud renders plan without reading video bytes and start chunks sooner (#6194)
* perf(engine): a remote source's tail decodes run many at a time, not one per half-core

* perf(producer): the plan logs where its extract stage spends its time

* perf(engine): a window that ends inside its source takes its frame count without a decode

* test(engine): pin the tail-decode slot test to the windows that decode

* style(engine): format the tail-probe window check

* test(engine): deferred counts match extraction on a B-frame NTSC source; VFR is not deferred

* style(engine): format the tail-probe tests

* perf(aws-lambda): a site stores each video's ffprobe JSON, and the plan reads it instead of probing

* test(aws-lambda): a site stores and hands out each video's probe

* style(aws-lambda): format the probe-facts upload and tests

* perf(engine): a site's stored probe also holds the final-frame timestamp the plan would probe for

* test(engine): drop an unused import

* test(aws-lambda): the stored probe holds the ffprobe JSON and the final frame

* test(aws-lambda): the stored probe holds the ffprobe JSON and the final frame

* perf(producer): a distributed chunk encodes while it captures

* perf(producer): the audio task logs where its mix spends its time

One line per audio stage: prepare, groups, mix, master, encode and the peak check, in ms.

* perf(engine): locked warmup ticks stop sleeping once the page is ready

* perf(engine): a window reaching a stored video's end takes its frame count from stored facts

* fix(producer): the cloud audio task logs its phases

* perf(engine): stored facts keep the last packet length so late-start windows skip the tail decode

* fix(engine): a probe spawns without waiting when no stored facts are declared

* chore(aws-lambda): shorten comments over the line limit

* fix(engine): warmup keeps pacing after a failed tick; stored final frames check ffprobe major

* test(engine): the recorded ffprobe major matches the build's own banner

* test(engine): the version-banner probes go through a named binary like the other spawns
2026-10-24 14:30:16 +00:00
Miguel Ángel bd4cae7e35 fix(engine): an opaque ProRes clip extracts to jpg frames, only ProRes with alpha stays png (#6130)
* fix(engine): an opaque ProRes clip extracts to jpg frames, only ProRes with alpha stays png

ProRes was always cached as lossless PNG in case a missed tag hid alpha. ProRes alpha shows in its pixel format (yuva444p), so the guess only costs size: a 4K 4:2:2 clip wrote about 17 MB a frame, 18 GB for 36 s. VP8 and VP9 keep PNG because their alpha rides a side channel ffprobe can miss.

* test(engine): ProRes extraction format follows the alpha plane

* chore(engine): format the ProRes extraction change
2026-10-23 14:14:06 +00:00
Miguel Ángel 69738b7ce4 fix(engine): render into a folder whose name holds a percent sign (#6116)
* fix(engine): render into a folder whose name holds a percent sign

ffmpeg reads an image sequence path as a printf template, so a folder like
"12% Up" was parsed as a format code and every video render failed. The
folder part of each frame-sequence path now goes through one helper that
doubles the percent sign.

* fix(engine): drop the helper doc line the comment ratchet rejects
2026-10-23 07:33:25 +00:00
Miguel Ángel 207c8a4daf fix(engine): a muted video no longer plays at full volume in renders that skip the browser probe (#6044)
* fix(engine): a muted video no longer plays at full volume in renders that skip the browser probe

A <video data-has-audio="true" muted> with an audio stream is silent in
preview and in renders that run the browser probe, but a render that
skipped the probe mixed it at volume 1.0, ignoring data-volume. The audio
parser drops muted videos; the video parser kept hasAudio from
data-has-audio alone, so the auto-detect fallback re-added the sound.
The video parser now asks the same isAudibleVideoElement check.

* style(producer): format the muted-video parity test
2026-10-21 18:58:45 +00:00
Miguel Ángel d67e81c7f1 fix(studio-server): keep transparency in proxies of VP8/VP9 alpha webm (#6025)
* fix(studio-server): keep transparency in proxies of VP8/VP9 alpha webm

A webm whose alpha lives in the VP8/VP9 side channel (alpha_mode=1,
pix_fmt yuv420p) was proxied to opaque H.264, and the VP8 proxy path
decoded it with ffmpeg's native decoder, which drops alpha. The alpha
rule and the alpha-reading decoder now live once in
@hyperframes/parsers/media-alpha, used by engine and studio-server.

* build(parsers): raise the packed size budget for the media-alpha subpath

parsers already sat at its packed budget on main; the shared alpha rule
adds 676 B of JS and 803 B of declarations (190958 B packed in CI).
2026-10-21 13:49:30 +00:00
Miguel Ángel 7cc5dd382c perf(producer): cloud render chunks hand Chrome each video near the size it is shown (#6001)
* perf(producer): cloud render chunks hand Chrome each video near the size it is shown

A chunk now measures the largest box each video is drawn in over its own frames,
in device pixels, before extracting that video's frames, and halves the frames
while they still cover that box. Graded videos, pages with a canvas, and
object-fit none/scale-down keep full-size frames.

* docs(engine): shorten the shown-box measurement comment

* refactor(engine): name the halving filter instead of commenting it

* test(producer): end ffprobe options before the frame path in the shown-size tests
2026-10-21 05:32:19 +00:00
Miguel Ángel 472ee43c3a fix(engine): long phone-video exports neither time out nor stall before the first frame (#5921)
* fix(engine): long 4K video exports no longer time out while frames are still coming

* test(engine): the paced extraction test keeps a wide margin on a loaded runner

* fix(engine): long phone-video exports probe each source once instead of once per cut

* fix(engine): a failed source probe is retried, and probes share the extraction slots

* test(engine): synthesize the tone-mapped concurrency fixture with known input colour
2026-10-20 01:14:42 +00:00
Miguel Ángel 992df8b203 perf(cloud): move project bytes once through shared content storage (#5772)
* perf(producer): distributed chunks download only the videos they show

A project video under compiled/ that chunks extract their own frames from
now goes only to the chunks that show it, like video-sources/ files. A
chunk skips the readiness wait for a deferred video it does not show, so
it no longer logs media_load_failed for a file it was never given.

* fix(producer): cloud site uploads pack the files their id hashes, symlinks included

* perf(aws-lambda): chunks read only the byte ranges of a video they need

A chunk that extracts its own frames from a deferred project video now
reads it from S3 by byte range through a short-lived pre-signed URL,
instead of downloading and hashing the whole object first. S3 attests the
object's stored SHA-256 before the URL is signed; every other artifact is
still downloaded and verified as before.

* perf(aws-lambda): upload project files once to a shared content-addressed area

* chore(producer): keep planV2 comment share at the base level

* fix(engine): verify the TLS certificate of https video inputs and never surface their signed query

* fix(aws-lambda): renders re-upload a site's expired files instead of failing

* perf(aws-lambda): the plan step probes project videos in place instead of downloading them

* fix(engine): https video reads reconnect and time out, and redaction covers wrapped causes

Signed-URL reads now pass -rw_timeout, -reconnect and -reconnect_delay_max (both Lambda builds accept them) and ffmpeg also -reconnect_on_network_error, so a transient S3 error or a stalled read fails fast or recovers instead of waiting out the process timeout. Query redaction now descends into error.cause.

* fix(aws-lambda): share entry bundles and ratchet package budget

* fix(aws-lambda): site ids uploaded before files.json still render

A site id from an earlier sites create has only sites/<id>/project.tar.gz.
renderToLambda now falls back to that tarball, which the plan step still
reads, and when neither key exists it fails before starting with both keys
named and the fix (run sites create again).

* fix(aws-lambda): renders fail before upload on a stack too old to read sites

A stack deployed before files.json sites untars the site file in its plan
step and fails after four retries. renderToLambda now reads the state
machine definition first and stops with STACK_TOO_OLD and the redeploy
command when the plan task does not list SITE_MANIFEST_INVALID, which new
SAM and CDK stacks now treat as terminal. The README's upgrade section
drops the planProtocol v1 fallback, which hits the same failure.

* docs(aws-lambda): document shared content storage and the upgrade path

The deploy guide still described the tarball HeadObject short-circuit. It
now covers the cas/sha256/ store and its 30-day lifecycle rule, the
STACK_TOO_OLD, SITE_FILES_MISSING and SITE_NOT_FOUND errors, that old site
ids keep working, and that every project uploads once more after the
upgrade. A test keeps a troubleshooting entry for each pre-submit error.

* fix(producer): chunks log each video they read by URL, without its query

A chunk reading a video straight from the bucket left no trace in its log.
It now logs one line per remote source with the path and the URL minus its
query string, so signatures never reach the log.

* fix(engine): ffmpeg errors never keep part of a signed URL's query

A process's stderr tail was trimmed before its URLs were redacted, so a
cut inside a signed URL left a query fragment the URL pattern no longer
matched, and the fragment reached the error. The stderr tail is now
redacted before every trim, and the redactor also strips a signing
parameter cut off from its URL.

* fix(aws-lambda): comments name the site manifest instead of a file path

* fix(engine): stderr redaction survives chunk splits, and every ffmpeg spawn uses it

The stderr tail was redacted on each chunk and then cut, so once a cut
severed a long URL's scheme, the rest of its signature in the next chunk
matched neither pattern. The tail is now kept raw with 2x slack and
redacted only when the text is produced, then cut to size. The unused
producer audioExtractor (its own spawn, unredacted stderr) is deleted and
the parity harness's PCM decode goes through ManagedChildProcess; a test
fails on any ffmpeg/ffprobe spawn in engine, producer or aws-lambda that
does not.

* chore(producer): record the audioExtractor deletion as intended

* fix(engine): drop a signature fragment the stderr tail cut from its URL

The raw tail is kept at twice the cap, but that cut can itself land inside a
signature, and when redaction shrinks the rest below the cap the fragment
survived. After a cut, the severed token is dropped before redaction.

* refactor(engine): name the severed-token drop instead of commenting it

* test(engine): a cut-off URL leaves nothing behind; aws-lambda size budget to measured

* fix(producer): chunk URL-read tests use main's frame list; producer size budget to measured

* chore(producer): size budget at the measured size, no headroom

* chore(producer): size budget back to measured x1.05, the check's own rule
2026-10-19 19:23:51 +00:00
Miguel Ángel b46ebedeae fix(engine): a 29.97 fps camera clip exports smoothly at 30 fps (#5849)
* fix(engine): a 29.97 fps camera clip exports smoothly at 30 fps

Extraction samples each output frame from the last source frame starting at
or before it. Millisecond container ticks move each frame by up to half a
millisecond, while 29.97 fps drifts against 30 fps by 0.033 ms per frame, so
the rounding decided the slot and the export alternated repeats and skips.
Constant-rate frames are now re-spaced on the stream's own frame grid from
the first decoded frame before sampling. The cache schema moves to v8 so
exports stop reusing frames extracted the old way.

* fix(engine): every extraction process snaps frames to one absolute grid

The grid was anchored on each process's first decoded frame, which carries its
own millisecond rounding, so a segment or distributed chunk could place the
29.97 -> 30 repeat elsewhere and skip a frame at its seam. Frames are now
snapped to the stream's own grid in file time (this process's seek plus the
video's lead), the same in every process.
2026-10-18 16:37:20 +00:00
James RussoandClaude Opus 5.5 6afcb48395 fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8 (#5572)
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8

VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.

The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused

v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:45:43 +00:00
James RussoandClaude Opus 5.5 401354cd8f fix(engine): keep the rightmost 8 columns of SDR MP4 at widths like 1080 (#5559)
#4564 routes SDR colour through `format=gbrp`. Without accurate_rnd,
swscale's yuv-to-gbrp conversion leaves the last 8 columns black when the
width is 16n + 8 (1000, 1080, 1112...), so every 1080x1920 portrait MP4
since 0.8.81 has had an 8 px black strip on the right. The same step in
SDR jpg frame extraction blanked the right edge of footage at those widths.

Set accurate_rnd on the yuv-to-gbrp scale in both chains. It is also the
more accurate rounding: on a testsrc2 frame, worst luma error against a
float reference drops from 1.85 to 0.90 codes.

Fixes #5551

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:50:09 +00:00
Miguel Ángel b211fc7cb8 fix(engine): MP4, MOV and TS exports keep sound and picture in sync when streams start apart (#5455)
* fix(engine): exports place sound and picture right when a source's audio and video start apart

* fix(engine): a late video stream plays to its end and distributed and HDR renders hide its lead

* test(engine): pin a late video stream's trimmed start, held tail and HDR trim
2026-10-11 13:54:33 +00:00
Miguel Ángel 81f60dbc98 fix(producer): exports no longer fail on a video clip outside the composition's length (#5431)
* fix(engine): exports no longer fail on a video clip outside the composition's length

A clip that starts at or after the composition's end (or ends before 0) is
never on screen, so extraction skips it. Its timeline window stayed as
authored, so the frame-coverage gate still expected its full length and
aborted the render with "captured 0 of expected N frames". Such a clip now
leaves extraction with an empty window, the same way a partly visible clip
is already trimmed to its visible part, so it owes no frames.

* fix(producer): drop clips outside the timeline before extraction, cloud renders included

The previous commit marked such a clip with end = start, but the window
resolver reads a zero-length slot as "use the source's natural length", and
the distributed plan still required an extracted entry for every declared
clip. Instead, the extract stage (shared by in-process and distributed
renders) removes clips that lie entirely outside [0, duration) from the
composition before probing and extraction, using one engine predicate that
the extractor's own early skip also uses, so a clip ending before 0 is no
longer downloaded or probed either.

* fix(engine): keep natural-length clips when dropping clips outside the timeline

A clip with no authored end carries end 0 (script-created media, or a src
set from a script), which the window resolver reads as "natural source
length". The outside-the-timeline check treated it as ending at 0 and dropped
it, so the clip froze on its first frame. Only an authored end (after the
start) at or before 0 now counts as before the timeline.

* fix(producer): only drop clips that start at or after the composition's end

A clip with no authored length carries end 0, the same value as an authored
end at 0, so "ends at or before 0" cannot be told apart from "natural
length" and a script-created clip with a negative start was dropped and
froze. Clips entirely before 0 keep main's behaviour; the filter now only
drops clips starting at or after the end, which never reads the end.

* test(producer): pin that a clip starting just before the end is still extracted
2026-10-11 00:02:51 +00:00
Miguel Ángel b6467d9131 perf(engine): re-renders reuse the extracted frames of remote videos (#5408)
* perf(engine): re-renders reuse the extracted frames of remote videos

A remote video is downloaded into the render's own work dir, so its path and
mtime are new on every render. The extraction cache keyed on both, so a
re-render of a composition with a remote video never hit the cache and ran
ffmpeg over the whole clip again. Key a downloaded source on the SHA-256 of
its bytes instead. Keys of local sources are unchanged.

* fix(engine): key the video copies a render writes itself by content

The producer downloads a remote video at compile time and copies it into the
render's compiled dir, so the extractor never sees the https src. Key any
source inside the compiled dir or the extractor's own output dir by content,
and skip the cache for one that cannot be read.

* perf(engine): hash a render-written video only when its cache lookup runs

Distributed planning copies the whole project into the compiled dir and defers
long video extractions, which never look up the cache. Hash render-written
sources just before the lookup, once per file, so deferred ranges and repeated
sources are not hashed.

* fix(engine): cache only downloaded media by content, not distributed project copies

A distributed plan copies the whole project into the compiled dir, so keying
everything there by content would hash every project video on the planner.
Content-key the extractor's own downloads and the dirs the caller names; the
producer names its remote media dir. Hashing stops on abort, and a source
with no hash skips the cache.

* style(engine): keep the extraction cache's comment share at its base

* test(engine): pin no hashing with the cache off and the producer's remote-media dir

* test(engine): run the cache-off case through the shared render helper
2026-10-10 08:09:39 +00:00
Miguel Ángel 4b68f73fe7 fix(engine): long exports with many clips no longer stall at 90% or hide a timeout (#5375)
* fix(engine): long exports with many clips no longer stall at 90% or hide a timeout

* fix(engine): stop queued frame extractions on cancel and time out distributed assembly

* refactor(engine): simplify the extraction slot check and its concurrency test

* test(engine): prove a cancelled extraction leaves the queue, skip the shell shim on Windows

* test(engine): wait for the held extraction's own record before cancelling the queued one
2026-10-10 02:43:44 +00:00
James RussoandClaude Opus 5.5 04f2bcd67a perf(producer): extract distributed video frames in each chunk (#5290)
* perf(producer): extract distributed video frames in each chunk

Plan v2 no longer extracts long constant-frame-rate videos. It ships
each source once and every chunk extracts only the frames it shows, so
Plan time stops growing with video length (#5260).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(aws-lambda): plan no longer bounds video length

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(producer): pin deferred compiled sources to every chunk

Each chunk's page loads every <video> src during session init and a
missing file fails the chunk, so a deferred source under compiled/ must
keep reaching every chunk, as it does on main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:50:01 +00:00
James RussoandClaude Opus 5.5 de1ef5a05a fix(engine): extract long video ranges in segments and keep plan uploads flowing (#5280)
* fix(engine): extract long video ranges in segments and keep plan uploads flowing

A distributed render's plan step extracted a whole video in one ffmpeg process,
so a long clip hit ffmpegProcessTimeout (300 s) however much of Lambda's
15-minute limit was left, and every frame then uploaded in batches of 16 that
each waited for their slowest blob.

Ranges longer than two minutes now extract as two-minute segments that run side
by side, each numbered after the last. They sample the same absolute times as a
single pass, so the frames are byte-identical (checked on a 26-minute, 37,440
frame clip). Plan v2 publishing keeps 32 uploads in flight instead of batching.
On a 10 GB Lambda the 26-minute clip's plan step fell from 395 s to 200 s.

The Lambda docs now state the plan-step length bound.

Fixes #5260

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(engine): split extraction on an integer frame count

The split compared and subtracted float seconds, so 200.3 - 80.3 became
120.00000000000001 s: a second segment got `-t 1.42e-14`, which ffmpeg
rejects, failing an ordinary two-minute range. The frame count now comes from
extractionFrameCountForDuration, the rule a single pass follows, and every
segment takes an exact frame count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 19:07:46 +00:00
Santhi PrakashandMiguel Ángel cad7dbed0d fix(engine): warn when a video input declares alpha but decodes opaque (#3771)
* fix(engine): warn when a video input declares alpha but decodes opaque

- Problem: a video input whose alpha_mode=1 tag outlives its alpha plane (a remux drops the BlockAdditional sidecar while keeping the tag) composites as a solid rectangle, and nothing tells the user their file, not the renderer, is the problem (#3220 / #3226).
- Fix: during extraction preflight, when an input declares alpha (hasAlpha) and its codec can carry alpha, sample the decoded alpha plane (up to 3 frames at 8x8 rgba, bounded 30s, ~768-byte ceiling). If uniformly opaque, emit a non-blocking stderr warning naming the file with the re-export remedy. Inconclusive probes stay silent; opaque full-frame backgrounds are legitimate and never fail the render.
- Verification: 8 new unit tests (byte logic + message); engine suite 1483/1487 with 4 pre-existing failures on clean main (missing hdr-regression PNG fixture in this clone); typecheck/lint/format clean; real-fixture checks: lying-tag WebM warns, genuine-alpha WebM stays silent, both agree with ffmpeg alphaextract.

Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>

* fix(engine): probe input alpha with the decoder that matches its codec

* style(engine): format the alpha probe test

* fix(engine): keep the alpha probe from failing or racing a render

---------

Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-02 04:59:16 -07:00
Vance IngallsandClaude Opus 5.5 3c299acc7f feat(core): audible videos in preview web audio, audio groups, meters and carve (2/8) (#4814)
* feat(core): add isAudibleVideoElement predicate and audible media selector

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(runtime): route processed audible videos through web audio in preview

An audible <video> joins the Web Audio schedule, hidden-toggle and
group-capture passes when it needs a gain el.volume cannot express: a
boost above unity, an fx chain, automation, or an audio group. A plain
audible video at unity stays on native output. The clock master stays
audio-only; a video whose capture fails stays on native output (never
decoded, never failed closed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(audio): let audible videos join audio groups in preview and render

resolveAudioGroups, audioGroupOf, group render ids, the engine mixer and
the audio_group_no_members lint rule accept an audible <video>. A hidden
group drops its video members from the mix; muted videos stay inert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(editing): offer the audio fx section on audible videos

EditableElementFacts gains hasAudio; studio and sdk derive it from muted and data-has-audio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): show audio meters when a project has an audible video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): offer audible videos as carve voices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): let audible video clips join audio groups

Drops the video refusal; grouping and group audition count audible videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): honor hidden audio groups for auto-detected video audio

Muted videos that declare data-has-audio are no longer mixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep authored src, loop and honest status when keeping cutout sound

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(core): treat only data-has-audio=true as audible in runtime and lint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(core): trim the web audio route and group-key comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(runtime): capture ramped clips so a group's bus carries them in preview

A rate lane skipped Web Audio entirely, so a ramped video or audio clip in a
group played native output without the group's fader, mute or FX while the
export applied them. Only the decoded-buffer fallback cannot follow a rate
curve; the media-element capture can, so the skip now guards that fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): show meters only for videos Web Audio carries

A plain audible video at unity stays on native output, which the meters
cannot read, so it no longer advertises them. The runtime's join rule and the
meter gate now share audibleVideoNeedsWebAudio (core/audible-video).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 02:28:30 +00:00
37f30b1d06 feat: keep a video's sound on the clip by default (1/8) (#4813)
* feat(cli): one owner for init media placeholders, audio stays on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): init keeps a video's sound on the clip; silent files scaffold muted

Supersedes the patchVideoSrc changes in #3977 and #3942 and carries their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: miga-heygen <miga-heygen@users.noreply.github.com>
Co-Authored-By: kapelame <kapelame@users.noreply.github.com>

* feat(registry): example templates keep the a-roll's sound on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): video audio rules recommend keeping sound on the video

Update video_missing_muted and video_audio_double_source message and
fixHint text to recommend the new default: keep audio on the video with
data-has-audio="true" (not muted). Tests verify the new wording is in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): a video is either audible (data-has-audio) or muted, never both

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): follow the compiler's audio rule in the audio-track toggle

Confirm before background removal drops a video's sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): hyperframes-core teaches sound on the video by default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): workflows and scaffold guidance keep a video's sound on the clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: footage with sound keeps it on the video clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): init from-file a-roll is timed, muted strip is attribute-exact

The from-file a-roll video gets data-start="0" so a scaffold lints clean, and
the orphaned audio-only slot comment is dropped. setVideoAudio now walks the
tag attribute by attribute, so a class value containing the word muted
survives. The ffprobe-missing warning says the video is assumed to have sound
and how to switch to muted, and DEFAULT_META notes the failure surfaces at
render until lint support lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep has-audio in step with the muted toggle, test cutout confirm

Muting a video now clears has-audio and unmuting sets it to true, in both the
Design and flat media panels, matching the Has audio track toggle. Audio
elements are unchanged. Adds coverage for both panels plus the cutout
window.confirm guard: declined leaves src alone, accepted swaps src and mutes,
a silent video never prompts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): updateElementInHtml hasAudio only applies to videos, true or false

An audio element or an undefined hasAudio no longer picks up a muted attribute;
only an explicit false on a video writes muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: video_missing_muted rule text, silent-footage notes, consistent audible examples

lint.mdx and cli.mdx describe the current video_missing_muted rule instead of
the retired unmuted-video wording. The editing recipes say how to adapt for
silent footage and require the outgoing clip's sound to end at a J-cut. The
Remotion media example shows the audible mapping consistently, data-attributes
states that has-audio is required unless muted, and the hairline frame note no
longer claims the video is muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): encode init media filenames and probe --audio duration

Function replacers stop dollar-sign pattern expansion in filenames; src is percent-encoded.
--audio now passes the probed track length as the composition duration in both init paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): audio-state-aware media rules and time-aware double-source check

video_missing_muted honors any explicit data-has-audio.
video_audio_double_source only considers audible videos and only fires
when the same-src clips overlap in time.
duplicate_audio_track now covers audible videos.
audio_file_without_element no longer claims silence when an audible video
carries the sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): audio-stream check for remotion ports, followable J-cut recipe

Remotion Video maps to data-has-audio only when the file has an audio stream.
The J/L-cut recipe now shows both shots with muted pictures and separate audio.
embedded-captions removes the audio.mp3 scratch file it extracts.
skills-manifest.json regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(lint): trim comments to satisfy the comment ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): atomic muted edits, cutout keeps sound on a linked audio

Muted and Has audio track commit one batched write (one persist, one
undo entry) and remove data-has-audio instead of writing an empty
marker. Unmuting a video probes its src: audible -> true, silent ->
false, probe failed -> absent. Remove background on an audible video
inserts a linked sibling <audio> with the original sound in the same
write, replacing the drop-the-sound confirm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): skip hidden videos when auto-adding video audio

A video hidden in Studio is dropped by the mixer, so its src never reached
existingAudioSrcs and the render re-added it at full volume. The video parser
now records the shared hidden predicate and the auto-detect step skips hidden
videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep useDomEditSession under the 600-line cap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: miga-heygen <miga-heygen@users.noreply.github.com>
Co-authored-by: kapelame <kapelame@users.noreply.github.com>
2026-10-01 23:56:36 +00:00