Commit Graph
76 Commits
Author SHA1 Message Date
Miguel Ángel b559a98d37 fix(engine): hlg footage with a large header sei also tone-maps at hlg's white (#7349)
* fix(engine): hlg footage with a large header sei also tone-maps at hlg's white

* fix(engine): say in the read's own doc line why it streams

* test(engine): the header-SEI HLG fixture encodes without b-frames, so it muxes on windows
2026-11-07 21:46:28 +00:00
Miguel Ángel 5ee9e928d5 fix(core): hlg footage tone-maps to sdr at the bt.2408 reference white (#7234)
* fix(core): hlg footage tone-maps to sdr at the bt.2408 reference white

* fix(core): an HLG clip whose stream leaves its transfer unset keeps HLG's white

An SPS with an unspecified transfer takes the stream's tag, as its frames do; a failed keyframe read is not cached; the grey test needs ffmpeg 8.
2026-11-05 13:04:56 +00:00
Miguel Ángel 32de0196d5 fix(producer): a failed video extraction names which clip files failed (#6905)
* fix(producer): a failed video extraction names which clip files failed

* fix(producer): the extraction source label hides scheme payloads and drive paths

* style(producer): format the extraction source label
2026-11-01 00:09:14 +00:00
Miguel Ángel 47fe68562f fix(engine): nested video clips on a frame boundary export every source frame (#6893)
* fix(engine): nested video clips on a frame boundary export every source frame

* fix(engine): drop comment from ffmpeg time formatter
2026-10-31 20:08:23 +00:00
Miguel Ángel 60f711686c fix(producer): auto renders with color-graded media stay SDR instead of promoting to HDR (#6671)
* fix(producer): auto renders with color-graded media stay SDR instead of promoting to HDR

* fix(producer): keep auto renders SDR when a graded canvas is present
2026-10-31 16:16:06 +00:00
Miguel Ángel 279d40dcd7 test(engine): the rotated-clip fixture builds on ffmpeg older than 6 (#6811) 2026-10-30 09:27:00 +00:00
Miguel Ángel 10a4ed56cb fix(engine): a video clip starting between frames shows its media on the right frame (#6759)
* fix(engine): a video clip starting between frames shows its media on the right frame

* chore(engine): trim the helper comment to one line

* chore(engine): drop the helper comment

* test(engine): extract once per start in the off-grid clip test

* fix(engine): a looped clip keeps its loop origin when the start is off the frame grid

* test(engine): three off-grid starts cover the failure modes and keep the flake gate under its limit
2026-10-30 09:01:21 +00:00
Miguel Ángel 54c98b5906 fix(engine): a video on a non-square pixel grid renders at its display shape (#6772)
* fix(engine): a video on a non-square pixel grid renders at its display shape

* chore(engine): drop comments the names already carry

* fix(engine): a rotated clip with a non-square pixel grid keeps its display shape

* test(engine): the frame probes end their options with -- like every ffprobe call
2026-10-30 03:00:25 +00:00
Miguel Ángel a3ff1c4902 revert(engine): late-starting videos play to their last frame (#6746)
Reverts #6736. Its end-of-clip read fails intermittently on Windows, and the fallback
then expects frames past the clip's end.
2026-10-29 14:14:21 +00:00
Miguel Ángel 22b53c0115 fix(engine): late-starting videos play to their last frame (#6736)
* fix(engine): late-starting videos play to their last frame

A video whose first frame is after timestamp 0 could end early (frozen
tail) or late: ffprobe's stream duration does not run from the first
frame in every container (fragmented MP4, Matroska). Such a video's end
is now read from its last packet, once, where its facts are probed.

* test(engine): the WebM-with-audio case measures its first frame from the file

* test(engine): the first-frame probe ends its arguments with the input file
2026-10-29 12:05:27 +00:00
Miguel Ángel 313792408d fix(producer): HDR sources are tone-mapped for formats that cannot carry HDR (#6597)
* fix(producer): HDR sources are tone-mapped for formats that cannot carry HDR

* refactor(producer): one owner for which output formats can carry HDR

* refactor(producer): drop comments the helper names already carry

* fix(engine): SDR clips are not lifted to HDR when the render tone-maps to SDR
2026-10-27 23:09:15 +00:00
Miguel Ángel 3b2058a2b7 fix(engine): extraction waits out the decode up to a deep seek in a long-GOP video (#6583)
* fix(engine): extraction waits out the decode up to a deep seek in a long-GOP video

* chore(engine): shorten the keyframe gap doc comment

* fix(engine): cancelling an extraction during the keyframe probe reports a cancel

* fix(engine): a segment stopped by another's failure reads as stopped, not cancelled
2026-10-27 20:09:05 +00:00
Miguel Ángel e3e93aa7fe fix(core): paused preview and render show the same, correct video frame (#6432)
* feat(core): one function gives the media time that shows the frame starting at an instant

* fix(core): paused preview and render show the same, correct video frame

* fix(core): register the media-clock subpath in its descriptor and keep the reason beside the test

* fix(core): media seeks lead by 10 us so Chrome's timestamp rounding never shows the prior frame

* fix(core): generated pages lead media seeks by a literal, on every generator branch
2026-10-26 17:48:51 +00:00
Miguel Ángel d49e527b32 fix(engine): ffmpeg steps and downloads stop on no progress, not while still advancing (#6258)
* fix(engine): ffmpeg steps and downloads stop on no progress, not while still advancing

* test(cli): an ffmpeg step that stops advancing reports inactivity, not a deadline
2026-10-25 23:04:13 +00:00
Miguel Ángel 382a14d73e perf(engine): vp9 and vp8 sources without an alpha plane extract as jpeg (#6284)
A VP8/VP9 source with no alpha tag has one frame decoded with the alpha-aware decoder when its
metadata is read; if that frame has no alpha plane, its frames extract as jpeg instead of png.
Tagged alpha, a decoded alpha plane, a failed probe, or a plan written without the field keep png.
2026-10-25 08:32:49 +00:00
Jay Hemnani c2a6b15f6f fix(core): resolve a relative data-start inside the clip's own composition (#5336)
* fix(core): resolve a relative data-start inside the clip's own composition

* fix(engine): render-side data-start references resolve in the clip's own composition

The engine's reference resolver still looked an id up document-wide, so
with two different compositions sharing an id the producer placed media
from the first one while the runtime used the clip's own. Scope it the
same way as the runtime's startResolver, with the same document-wide
fallback, and correct the SDK comments that called the runtime lookup global.

* fix(parsers): keep getAttribute-only reference adapters working

The scoped lookup needs parentElement, closest and querySelectorAll, but
RefResolverEl is exported from @hyperframes/parsers and @hyperframes/engine.
Make them optional and fall back to the document-wide lookup when an adapter
lacks them.

* refactor: look up a scoped reference with one id query
2026-10-25 00:23:39 +00:00
Diego López e3881bc589 test(engine): synthesize the untagged AV1 fixture with whichever AV1 encoder ffmpeg has (#5379) 2026-10-24 08:15:40 +00:00
Abdullah M. Nazif bd4d8c6cb5 test: use supported FFmpeg flags for VFR fixtures (#5352) 2026-10-24 04:10:24 +00:00
Miguel Ángel bd4cae7e35 fix(engine): an opaque ProRes clip extracts to jpg frames, only ProRes with alpha stays png (#6130)
* fix(engine): an opaque ProRes clip extracts to jpg frames, only ProRes with alpha stays png

ProRes was always cached as lossless PNG in case a missed tag hid alpha. ProRes alpha shows in its pixel format (yuva444p), so the guess only costs size: a 4K 4:2:2 clip wrote about 17 MB a frame, 18 GB for 36 s. VP8 and VP9 keep PNG because their alpha rides a side channel ffprobe can miss.

* test(engine): ProRes extraction format follows the alpha plane

* chore(engine): format the ProRes extraction change
2026-10-23 14:14:06 +00:00
Miguel Ángel 69738b7ce4 fix(engine): render into a folder whose name holds a percent sign (#6116)
* fix(engine): render into a folder whose name holds a percent sign

ffmpeg reads an image sequence path as a printf template, so a folder like
"12% Up" was parsed as a format code and every video render failed. The
folder part of each frame-sequence path now goes through one helper that
doubles the percent sign.

* fix(engine): drop the helper doc line the comment ratchet rejects
2026-10-23 07:33:25 +00:00
Miguel Ángel 7cc5dd382c perf(producer): cloud render chunks hand Chrome each video near the size it is shown (#6001)
* perf(producer): cloud render chunks hand Chrome each video near the size it is shown

A chunk now measures the largest box each video is drawn in over its own frames,
in device pixels, before extracting that video's frames, and halves the frames
while they still cover that box. Graded videos, pages with a canvas, and
object-fit none/scale-down keep full-size frames.

* docs(engine): shorten the shown-box measurement comment

* refactor(engine): name the halving filter instead of commenting it

* test(producer): end ffprobe options before the frame path in the shown-size tests
2026-10-21 05:32:19 +00:00
Miguel Ángel f98ef09be4 test: queue runs on slow windows runners stop failing four timing tests (#5949)
* test: queue runs on slow windows runners stop failing four timing tests

* test(cli): the keepalive process test budgets for a cold start on a slow runner
2026-10-20 09:12:43 +00:00
Miguel Ángel 472ee43c3a fix(engine): long phone-video exports neither time out nor stall before the first frame (#5921)
* fix(engine): long 4K video exports no longer time out while frames are still coming

* test(engine): the paced extraction test keeps a wide margin on a loaded runner

* fix(engine): long phone-video exports probe each source once instead of once per cut

* fix(engine): a failed source probe is retried, and probes share the extraction slots

* test(engine): synthesize the tone-mapped concurrency fixture with known input colour
2026-10-20 01:14:42 +00:00
Miguel Ángel b46ebedeae fix(engine): a 29.97 fps camera clip exports smoothly at 30 fps (#5849)
* fix(engine): a 29.97 fps camera clip exports smoothly at 30 fps

Extraction samples each output frame from the last source frame starting at
or before it. Millisecond container ticks move each frame by up to half a
millisecond, while 29.97 fps drifts against 30 fps by 0.033 ms per frame, so
the rounding decided the slot and the export alternated repeats and skips.
Constant-rate frames are now re-spaced on the stream's own frame grid from
the first decoded frame before sampling. The cache schema moves to v8 so
exports stop reusing frames extracted the old way.

* fix(engine): every extraction process snaps frames to one absolute grid

The grid was anchored on each process's first decoded frame, which carries its
own millisecond rounding, so a segment or distributed chunk could place the
29.97 -> 30 repeat elsewhere and skip a frame at its seam. Frames are now
snapped to the stream's own grid in file time (this process's seek plus the
video's lead), the same in every process.
2026-10-18 16:37:20 +00:00
Miguel Ángel 14e2ccf3bb test(engine): the warm extraction cache test checks ffmpeg did not run, not the clock (#5613) 2026-10-14 17:51:02 +00:00
James RussoandClaude Opus 5.5 6afcb48395 fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8 (#5572)
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8

VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.

The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused

v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:45:43 +00:00
James RussoandClaude Opus 5.5 401354cd8f fix(engine): keep the rightmost 8 columns of SDR MP4 at widths like 1080 (#5559)
#4564 routes SDR colour through `format=gbrp`. Without accurate_rnd,
swscale's yuv-to-gbrp conversion leaves the last 8 columns black when the
width is 16n + 8 (1000, 1080, 1112...), so every 1080x1920 portrait MP4
since 0.8.81 has had an 8 px black strip on the right. The same step in
SDR jpg frame extraction blanked the right edge of footage at those widths.

Set accurate_rnd on the yuv-to-gbrp scale in both chains. It is also the
more accurate rounding: on a testsrc2 frame, worst luma error against a
float reference drops from 1.85 to 0.90 codes.

Fixes #5551

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:50:09 +00:00
Miguel Ángel 6a83959b0f test(engine): real-ffmpeg tests and fixture cleanup get a 30 s budget (#5368)
* test(engine): real-ffmpeg HLS tests get the 30 s budget other real-ffmpeg tests use

* test(engine): real-ffmpeg fixture cleanup gets the 30 s its setup already has
2026-10-09 23:09:10 +00:00
James RussoandClaude Opus 5.5 04f2bcd67a perf(producer): extract distributed video frames in each chunk (#5290)
* perf(producer): extract distributed video frames in each chunk

Plan v2 no longer extracts long constant-frame-rate videos. It ships
each source once and every chunk extracts only the frames it shows, so
Plan time stops growing with video length (#5260).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(aws-lambda): plan no longer bounds video length

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(producer): pin deferred compiled sources to every chunk

Each chunk's page loads every <video> src during session init and a
missing file fails the chunk, so a deferred source under compiled/ must
keep reaching every chunk, as it does on main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:50:01 +00:00
James RussoandClaude Opus 5.5 de1ef5a05a fix(engine): extract long video ranges in segments and keep plan uploads flowing (#5280)
* fix(engine): extract long video ranges in segments and keep plan uploads flowing

A distributed render's plan step extracted a whole video in one ffmpeg process,
so a long clip hit ffmpegProcessTimeout (300 s) however much of Lambda's
15-minute limit was left, and every frame then uploaded in batches of 16 that
each waited for their slowest blob.

Ranges longer than two minutes now extract as two-minute segments that run side
by side, each numbered after the last. They sample the same absolute times as a
single pass, so the frames are byte-identical (checked on a 26-minute, 37,440
frame clip). Plan v2 publishing keeps 32 uploads in flight instead of batching.
On a 10 GB Lambda the 26-minute clip's plan step fell from 395 s to 200 s.

The Lambda docs now state the plan-step length bound.

Fixes #5260

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(engine): split extraction on an integer frame count

The split compared and subtracted float seconds, so 200.3 - 80.3 became
120.00000000000001 s: a second segment got `-t 1.42e-14`, which ffmpeg
rejects, failing an ordinary two-minute range. The frame count now comes from
extractionFrameCountForDuration, the rule a single pass follows, and every
segment takes an exact frame count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 19:07:46 +00:00
Vance IngallsandClaude Opus 5.5 3c299acc7f feat(core): audible videos in preview web audio, audio groups, meters and carve (2/8) (#4814)
* feat(core): add isAudibleVideoElement predicate and audible media selector

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(runtime): route processed audible videos through web audio in preview

An audible <video> joins the Web Audio schedule, hidden-toggle and
group-capture passes when it needs a gain el.volume cannot express: a
boost above unity, an fx chain, automation, or an audio group. A plain
audible video at unity stays on native output. The clock master stays
audio-only; a video whose capture fails stays on native output (never
decoded, never failed closed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(audio): let audible videos join audio groups in preview and render

resolveAudioGroups, audioGroupOf, group render ids, the engine mixer and
the audio_group_no_members lint rule accept an audible <video>. A hidden
group drops its video members from the mix; muted videos stay inert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(editing): offer the audio fx section on audible videos

EditableElementFacts gains hasAudio; studio and sdk derive it from muted and data-has-audio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): show audio meters when a project has an audible video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): offer audible videos as carve voices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): let audible video clips join audio groups

Drops the video refusal; grouping and group audition count audible videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): honor hidden audio groups for auto-detected video audio

Muted videos that declare data-has-audio are no longer mixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep authored src, loop and honest status when keeping cutout sound

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(core): treat only data-has-audio=true as audible in runtime and lint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(core): trim the web audio route and group-key comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(runtime): capture ramped clips so a group's bus carries them in preview

A rate lane skipped Web Audio entirely, so a ramped video or audio clip in a
group played native output without the group's fader, mute or FX while the
export applied them. Only the decoded-buffer fallback cannot follow a rate
curve; the media-element capture can, so the skip now guards that fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): show meters only for videos Web Audio carries

A plain audible video at unity stays on native output, which the meters
cannot read, so it no longer advertises them. The runtime's join rule and the
meter gate now share audibleVideoNeedsWebAudio (core/audible-video).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 02:28:30 +00:00
37f30b1d06 feat: keep a video's sound on the clip by default (1/8) (#4813)
* feat(cli): one owner for init media placeholders, audio stays on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): init keeps a video's sound on the clip; silent files scaffold muted

Supersedes the patchVideoSrc changes in #3977 and #3942 and carries their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: miga-heygen <miga-heygen@users.noreply.github.com>
Co-Authored-By: kapelame <kapelame@users.noreply.github.com>

* feat(registry): example templates keep the a-roll's sound on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): video audio rules recommend keeping sound on the video

Update video_missing_muted and video_audio_double_source message and
fixHint text to recommend the new default: keep audio on the video with
data-has-audio="true" (not muted). Tests verify the new wording is in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): a video is either audible (data-has-audio) or muted, never both

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): follow the compiler's audio rule in the audio-track toggle

Confirm before background removal drops a video's sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): hyperframes-core teaches sound on the video by default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): workflows and scaffold guidance keep a video's sound on the clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: footage with sound keeps it on the video clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): init from-file a-roll is timed, muted strip is attribute-exact

The from-file a-roll video gets data-start="0" so a scaffold lints clean, and
the orphaned audio-only slot comment is dropped. setVideoAudio now walks the
tag attribute by attribute, so a class value containing the word muted
survives. The ffprobe-missing warning says the video is assumed to have sound
and how to switch to muted, and DEFAULT_META notes the failure surfaces at
render until lint support lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep has-audio in step with the muted toggle, test cutout confirm

Muting a video now clears has-audio and unmuting sets it to true, in both the
Design and flat media panels, matching the Has audio track toggle. Audio
elements are unchanged. Adds coverage for both panels plus the cutout
window.confirm guard: declined leaves src alone, accepted swaps src and mutes,
a silent video never prompts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): updateElementInHtml hasAudio only applies to videos, true or false

An audio element or an undefined hasAudio no longer picks up a muted attribute;
only an explicit false on a video writes muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: video_missing_muted rule text, silent-footage notes, consistent audible examples

lint.mdx and cli.mdx describe the current video_missing_muted rule instead of
the retired unmuted-video wording. The editing recipes say how to adapt for
silent footage and require the outgoing clip's sound to end at a J-cut. The
Remotion media example shows the audible mapping consistently, data-attributes
states that has-audio is required unless muted, and the hairline frame note no
longer claims the video is muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): encode init media filenames and probe --audio duration

Function replacers stop dollar-sign pattern expansion in filenames; src is percent-encoded.
--audio now passes the probed track length as the composition duration in both init paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): audio-state-aware media rules and time-aware double-source check

video_missing_muted honors any explicit data-has-audio.
video_audio_double_source only considers audible videos and only fires
when the same-src clips overlap in time.
duplicate_audio_track now covers audible videos.
audio_file_without_element no longer claims silence when an audible video
carries the sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): audio-stream check for remotion ports, followable J-cut recipe

Remotion Video maps to data-has-audio only when the file has an audio stream.
The J/L-cut recipe now shows both shots with muted pictures and separate audio.
embedded-captions removes the audio.mp3 scratch file it extracts.
skills-manifest.json regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(lint): trim comments to satisfy the comment ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): atomic muted edits, cutout keeps sound on a linked audio

Muted and Has audio track commit one batched write (one persist, one
undo entry) and remove data-has-audio instead of writing an empty
marker. Unmuting a video probes its src: audible -> true, silent ->
false, probe failed -> absent. Remove background on an audible video
inserts a linked sibling <audio> with the original sound in the same
write, replacing the drop-the-sound confirm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): skip hidden videos when auto-adding video audio

A video hidden in Studio is dropped by the mixer, so its src never reached
existingAudioSrcs and the render re-added it at full volume. The video parser
now records the shared hidden predicate and the auto-detect step skips hidden
videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep useDomEditSession under the 600-line cap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: miga-heygen <miga-heygen@users.noreply.github.com>
Co-authored-by: kapelame <kapelame@users.noreply.github.com>
2026-10-01 23:56:36 +00:00
Miguel ÁngelandKenneth Shortrede 3a64ea867c fix(engine): forced-SDR renders show tone-mapped HDR footage as Studio does (#4600)
* fix(engine): forced-SDR renders show tone-mapped HDR footage as Studio does

Forced-SDR extraction tone-maps HDR frames with zscale, but the frames kept the
BT.709 transfer tag, so Chrome re-curved them (a tone-mapped 388d96 painted as
2d8790), and JPEG frames kept the BT.709 matrix every JPEG reader decodes as
BT.601. Tone-mapped frames now get the same sRGB tag and BT.601 JPEG conversion
as other SDR frames.

macOS used VideoToolbox, which does not tone-map: its frames stay PQ and Chrome
tone-maps them itself, darker than the hable curve Linux and Studio use. macOS
now takes the zscale path whenever ffmpeg has zscale, found through the shared
ffmpeg filter probe; without it the old path stays and the render warns once.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): forced-SDR renders tone-map HDR footage that lacks matrix or primaries tags

zscale finds no path between colourspaces without them, so the extraction
failed. Untagged HDR footage is BT.2020 in practice, and now gets those tags.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): forced-SDR tone map also tags the transfer ffprobe read from the container

From ffmpeg 8.1 ffprobe reports a container-only PQ or HLG tag; the frames
still lack it, so zscale failed where macOS VideoToolbox used to render.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* test(engine): gate the container-only transfer case on the ffprobe that reads it

* docs(engine): shorten the filter-probe comment to the comment limit

* fix(engine): keep valid per-frame colour tags when tone-mapping HDR footage

* test(engine): build the mixed HLG and PQ fixture without B-frames so its timestamps are valid

* test(engine): build the trimmed mixed fixture without B-frames too

---------

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
2026-09-28 08:21:13 +00:00
Miguel Ángel e8313b2b39 fix(engine): renders show the video frame the preview shows at each output frame (#4596)
* fix(engine): renders show the video frame the preview shows below the source frame rate

* refactor(engine): one condition owns CFR sampling at the output rate

* fix(engine): webm and mkv clips keep every frame when rendered at their own rate

* fix(engine): only fine timebases get the half-tick sampling shift, so coarse ones stay exact

* docs(engine): the sampling comment states the half-tick bound it keeps

* fix(engine): move the extraction cache to v6 so main v5 entries are not reused

* docs(engine): keep extraction comments within their share

* test(producer): regenerate four 25-to-30 fps baselines for the exact frame pick

* test(engine): name the v6 schema prefix in its test
2026-09-28 01:38:09 +00:00
Miguel ÁngelandKenneth Shortrede c9b3d9c962 fix(engine): SDR renders keep footage and page colours as in the source (#4564)
* fix(engine): declare SDR extracted frames as sRGB so Chrome does not darken footage

* fix(engine): encode SDR mp4 frames with the BT.709 matrix they are tagged with

Chrome hands the SDR encoder BT.601 full-range JPEG captures (or RGB PNGs).
The encoders only converted the range, so swscale kept the BT.601 matrix under
the BT.709 tag (ffmpeg 5.1 to 7.0: CSS #C83C28 delivered as #D44824) or
converted YUV to YUV through a low-precision step (ffmpeg 8+: #101010 delivered
as #0B0D0D). Decode to RGB with the capture matrix, then encode with BT.709.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): tag SDR frames plain sRGB so every ffmpeg writes the sRGB chunk

ffmpeg writes a PNG sRGB chunk only for BT.709 primaries and writes cICP only
from 6.1. On ffmpeg 5.1 (Debian bookworm) a BT.601-tagged source therefore got
cHRM plus gAMA 1/2.2, which Chrome decodes as a pure 2.2 power: 0x10 shadows
rendered as 0x07. Declaring BT.709 primaries too gives the sRGB chunk on every
version, so Chrome shows the frame code values for every SDR source. The test
now reads the PNG chunks instead of ffprobe, which reports no transfer for an
sRGB chunk on ffmpeg 5.1.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): extract SDR jpg frames as the BT.601 full range JPEG readers assume

The default frame format for opaque SDR sources is JPEG. ffmpeg converted the
BT.709 source to full-range YUV without changing the matrix, and every JPEG
reader (Chrome included) decodes BT.601, so footage shifted: 0xC83C28 showed as
0xBB2D29. The old mp4 encode made the inverse mistake and hid it; with the
encode fixed, it would reach the output. Decode through RGB with the source
matrix, then encode BT.601 full range, keeping 4:2:0.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): round the SDR mp4 colour conversion accurately

swscale's default rounding moved saturated colours up to 3 codes (pure red 254
delivered as 251 on ffmpeg 5.1 to 7.0); accurate_rnd brings that to 1. The
chunk encoder now decides the HDR-or-SDR filter once, and the 10-bit GPU pad
test sets hdr, which its name always claimed.

* style(engine): keep the colour filter reasons in names and tests, not source comments

The comment ratchet caps a file's comment share: the new filter module and the
extractor carry their intent in constant names, and the encode test states why
the conversion goes through RGB.

* fix(engine): keep colour edges sharp through the SDR colour conversions

Going through RGB upsampled chroma with swscale's default filter, so ffmpeg 4.4
to 7.x smeared colour across edges by about 2 px (red 254 next to blue came out
8 inside the blue). Nearest-neighbour resampling maps the half-resolution chroma
back one to one, keeps the flat-colour results, and makes accurate_rnd redundant.
The pixel tests now also check a red to blue edge.

* test(producer): regenerate goldens whose colours the old mp4 matrix bent

These four fixtures compare at 40 to 45 dB, and their goldens were rendered
with the BT.601-under-BT.709 encode, so the corrected colours score 34 to 37 dB.
Regenerated in the Dockerfile.test image. Each new golden is closer to the
colour its page declares, e.g. #00d4aa was 00bcab and is now 00d3aa, and
#087443 was 006643 and is now 087241.

* fix(engine): keep VFR frame timing when extraction applies colour filters

ffmpeg before 6.1 ignores frame durations in CFR resampling once any -vf is
set, so a VFR window ending on a still lost frames. Filtered VFR extraction
now resamples in one process exactly as before and filters in a second.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): encode raw SDR layered frames in BT.709

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* test(engine): keep the new colour tests inside the format, comment and argv checks

* fix(engine): keep HDR and unusual pixel formats out of the two-process VFR path

The raw handoff dropped HDR10 light levels and misread semi-planar formats, and
decoder options rejected some ffprobe colour names. Only SDR sources in formats
nut carries unchanged now take it, their colour tags come back through setparams,
and a process that cannot start releases the other.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): read untagged footage with the matrix Chrome plays it with

Chrome decodes an untagged matrix as BT.709 from 720 coded lines up and as
BT.601 below; extraction assumed BT.601 at every size, so HD footage without
colour tags shifted hue against Studio's own preview.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* fix(engine): untagged VP9 and AV1 footage keeps BT.601, as Chrome plays it

Chrome applies the 720-line rule only to codecs it decodes through ffmpeg
(H.264, VP8); untagged VP9, VP9 alpha and AV1 play as BT.601 at every size.
Also widens the raw-pipe pixel formats to all verified ones, and reports the
pipeline process that failed first by settle order, not by per-process clocks.

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>

* test(producer): regenerate goldens for corrected page colours and untagged SD footage

* docs(engine): say which untagged codecs the matrix rule was measured on

---------

Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
2026-09-27 11:16:35 +00:00
Miguel Ángel 3def3514d1 fix(engine): export draws a video in exactly the frames it is shown in (#4565)
* fix(engine): export draws a video in exactly the frames export visibility shows it

* test(producer): guard that chunk frame planning snaps clip windows like export

* refactor(engine): set the render fps once per frame table and name its clip window

* chore(engine): trim frame table comments
2026-09-27 03:05:57 +00:00
Miguel Ángel 984ad28c7c fix(core): clips that meet within float rounding count as meeting in edits and playback (#4535)
* fix(core): clips that meet within float rounding count as meeting in edits and playback

* fix(engine): hand a shared clip boundary to the incoming clip in export, as preview does
2026-09-26 22:14:14 +00:00
Miguel Ángel 3f3f3b7f10 fix(engine): audio source-not-found names the file and shares one src resolver with lint (#4172)
* fix(engine): audio source-not-found names the file and shares one src resolver with lint

* fix(engine): audio source-not-found prints the resolved path with forward slashes on Windows

* fix(parsers): the shared src resolver ignores a media fragment like the browser
2026-09-19 10:07:22 -04:00
Miguel Ángel bdecae066a feat(engine): render speed ramps with the same source-time mapping as the preview (#4143)
* feat(engine): render speed ramps with the same source-time mapping as the preview

The frame extractor, frame lookup table, coverage check, compile-time duration and audio mixer
read a clip's rate lane through sourceTimeAt, so a rendered frame at t shows the source frame
the preview shows. Audio bakes a ramp as consecutive source slices stretched by their mean
rate, keeping pitch. The compile-time attribute reader now handles single-quoted and
entity-encoded JSON attributes. Adds a speed-ramp regression fixture.

* test(producer): regenerate the speed-ramp golden audio with the current engine

The stored baseline predates the per-slice audio fix, so its audio drifted 1 to 3 percent in
pitch from the expected source-time track and the CI render correlated at 0.61. Rendered in the
CI test image, the new baseline tracks the expected pitch to 0.1 percent median, and the compare
now reports visual 0 failed frames and audio correlation 1.000.

* refactor(core): drop data-preserve-pitch, pitch is preserved by construction

The engine always time-stretches audio with atempo, so an opt-out read only by
the preview made preview and render disagree. Remove the attribute and its
reader so both paths keep pitch.
2026-09-19 08:01:17 -04:00
Miguel Ángel 7f452ad276 feat(core): speed ramps as a rate lane in data-automation (#4129)
* feat(core): speed ramps as a rate lane in data-automation

A clip can carry a rate lane in data-automation. sourceTimeAt integrates it to source time and is the one mapping the runtime uses for media seeking, playback rate, natural duration, child timelines and the audio-master clock. Rate is clamped to 0.1-10 in one place, pitch is preserved unless data-preserve-pitch is false, and ramped clips play through the media element. Lint warns on a rate lane outside video and audio.

* fix(core): format the speed-ramp files and expect the unified rate clamp in the engine tests

* test(engine): read the playback-rate clamp from core instead of a literal
2026-09-19 06:27:42 -04:00
heygengenesis[bot]andmiguel.sierra ef48485b91 fix(engine): tone-map HDR frames for forced SDR (#3981)
Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com>
Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com>
2026-09-16 06:32:17 +00:00
heygengenesis[bot]andmiguel.sierra ae18e3a327 test(engine): cover held tails in HDR preflight (#3963)
Add regression coverage for finite SDR video slots that start past EOF in mixed HDR timelines, and verify lint exclusions with successful local duration probes.

Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com>
Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com>
2026-09-16 01:27:09 +00:00
heygengenesis[bot]andmiguel.sierra f3bd4234f5 fix(engine): hold finite video slots past EOF (#3962)
Allow explicit finite non-looping video slots to use the existing held-final-frame extraction path when data-media-start is at or beyond source duration. Warn during project lint for local static slots that will use this behavior.

Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com>
Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com>
2026-09-15 19:10:33 +00:00
Val 97991bbd35 fix(engine,producer,lint): resolve <source> children for media extract and localize (#3238)
Parent src-only scans skipped multi-format <video>/<audio> markup, so those
elements were never extracted, downloaded, or mixed and rendered blank/silent.
Lint now accepts a child <source src> as a resolvable media src.
2026-08-26 20:17:12 +00:00
Val f52ec1c25f fix(producer,core): honor relative data-start id-refs in render media scheduling (#3252)
compileTimingAttrs/injectDurations used parseFloat, so data-start="intro"
wrote a NaN data-end and extract preferred that over duration; parseNumeric
now skips the id-ref (parseVideoElements already resolves it).

collectRenderMedia's resolveHostWindow likewise read host data-start with
parseFloat, so chained sub-composition slots (data-start="hook") stacked at
0-2s and every scene after the first rendered black. It now resolves host
starts through the shared resolveReferencedStart, matching the media parsers.

Fixes #3361.
2026-08-26 05:36:24 +00:00
Miguel Ángel afafca4b96 feat: make creator media edits render-safe (#3322)
* feat: make creator media edits render-safe

* fix: align media playback timing

* docs: add creator editing recipes

* docs: expand creator editing guidance

* fix: unify media source offsets

* fix: scale natural media duration

* fix: preserve natural media zero spans

* fix: align compiled natural media timing

* test: classify compiler media test as integration

* fix: drop inactive media windows

* fix: unify literal timing parsing

* fix: keep browser media parsing serializable

* fix: keep page timing readers strict

* fix: close remaining preview timing gaps

* fix(core): preserve Studio voice pitch at playback speed

* chore: keep creator contract source-neutral
2026-08-18 10:17:02 -04:00
James Russo 127eb19371 fix(producer): correct short VFR frame coverage (#2936)
* fix(producer): correct short VFR frame coverage

* fix(engine): keep VFR extraction seek-local

* fix(engine): match ffmpeg decimal frame boundaries

* fix(engine): preserve exact extraction frame rates

* fix(engine): key frame cache by exact rate
2026-08-03 18:42:03 -07:00
James Russo 2339757377 fix: bound HDR and video extraction resources (#2955)
* fix: bound HDR and video extraction resources

* fix: trim negative video extraction preroll

* fix: skip invisible video extraction windows

* fix: preserve negative-start loop and held tails

* fix: cap finite video slots to source duration

* fix: bound held-tail frame extraction

* fix: plan from playable video duration

* fix: preserve open-ended held video tails

* fix: resolve held tails from decoded frames

* fix: normalize final-frame probe timestamps

* fix: handle unseekable final-frame sources

* fix: dedupe final-frame probes per render

* refactor: clarify output dynamic range contract
2026-08-02 20:21:54 -07:00
James 9b63646c8a fix(producer): type video extraction failures 2026-07-26 19:47:33 +00:00
Miguel Ángel 2e8f871bc8 fix(video): hold final frame through composition 2026-07-16 02:15:01 +00:00