* fix: hide console windows spawned by ffmpeg and nvidia-smi on Windows
Node's `windowsHide` defaults to false, so every child process that is a
console application pops a window on the user's desktop. During a render
or a regression run that is a stream of black windows stealing focus.
The repo already sets `windowsHide: true` across cli, studio-server and
most of engine; these were the remaining gaps, all on paths that run
during a normal render:
- core/mediaGradeAnalyzer: ffprobe + ffmpeg, once per analyzed media file
- engine/browserManager: `nvidia-smi` VRAM probe. This one goes through a
shell (`execSync`), so it flashes a cmd.exe window even on machines with
no NVIDIA GPU, where the command exists only to fail
- producer: the regression harness ffmpeg runner, PSNR/audio comparison
helpers, fixture synthesis, and the ffprobe call in audioPadTrim
No behavior change on POSIX, where the flag is ignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(producer): escape PSNR stats_file for ffmpeg's two-pass filtergraph parse
An ffmpeg filtergraph argument is unescaped twice: once when the graph is
split into filters and their options, then again when the option value is
read. `psnrAtFrames` escaped `\` and `:` a single time, which only survives
the first pass.
On POSIX that is invisible, because mkdtemp under tmpdir() produces neither
character. On Windows the stats path is always `C:\Users\...`, so it reaches
the option parser with a bare `:`, which starts a new option and fails the
whole graph:
[AVFilterGraph] No option name near '\Users\...\psnr.log'
Error parsing filterchain '[rv][gv]psnr=...:stats_file=C\:\Users\...'
Every suite that got as far as quality validation died there, so a Windows
regression run reported 0 passes with only 2 genuine compilation failures.
Escaping for both passes fixes it and stays a no-op on POSIX.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(producer): pin PSNR stats path escaping with a colon and backslash path
* fix(producer): keep the PSNR stats path out of the filtergraph
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* fix(engine): pin capture surface scale to 1 device pixel per CSS pixel
Capture reads the browser's compositor surface (`Page.captureScreenshot`
with `fromSurface: true`, and `HeadlessExperimental.beginFrame` likewise),
and that surface is rasterized at the host display's backing scale.
`page.setViewport({ deviceScaleFactor })` goes through
`Emulation.setDeviceMetricsOverride`, which only moves
`window.devicePixelRatio` on the page side — it does not resize the
surface.
On a HiDPI host every render therefore came out at twice the composition:
a 1920x1080 composition encoded to a 3840x2160 file, and no flag could
bring it down, because the pipeline's own deviceScaleFactor was already 1.
Supersampling stacked on top of it, so `--resolution landscape-4k`
produced 7680x4320 rather than 3840x2160.
Pinning the surface with `--force-device-scale-factor=1` leaves the
deliberate supersampling path (the capture clip's `scale`, from
`resolveDeviceScaleFactor`) as the only thing that multiplies output
pixels. The flag is a no-op on Linux, where the surface is already 1x, so
CI and cloud output are unchanged.
* fix(engine): drop the capture-scale comments the ratchet rejects
---------
Co-authored-by: Leo Xu <xuliyou@bytedance.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
* feat(core): add isAudibleVideoElement predicate and audible media selector
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(runtime): route processed audible videos through web audio in preview
An audible <video> joins the Web Audio schedule, hidden-toggle and
group-capture passes when it needs a gain el.volume cannot express: a
boost above unity, an fx chain, automation, or an audio group. A plain
audible video at unity stays on native output. The clock master stays
audio-only; a video whose capture fails stays on native output (never
decoded, never failed closed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(audio): let audible videos join audio groups in preview and render
resolveAudioGroups, audioGroupOf, group render ids, the engine mixer and
the audio_group_no_members lint rule accept an audible <video>. A hidden
group drops its video members from the mix; muted videos stay inert.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(editing): offer the audio fx section on audible videos
EditableElementFacts gains hasAudio; studio and sdk derive it from muted and data-has-audio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(studio): show audio meters when a project has an audible video
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(studio): offer audible videos as carve voices
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(studio): let audible video clips join audio groups
Drops the video refusal; grouping and group audition count audible videos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(engine): honor hidden audio groups for auto-detected video audio
Muted videos that declare data-has-audio are no longer mixed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(studio): keep authored src, loop and honest status when keeping cutout sound
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(core): treat only data-has-audio=true as audible in runtime and lint
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(core): trim the web audio route and group-key comments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(runtime): capture ramped clips so a group's bus carries them in preview
A rate lane skipped Web Audio entirely, so a ramped video or audio clip in a
group played native output without the group's fader, mute or FX while the
export applied them. Only the decoded-buffer fallback cannot follow a rate
curve; the media-element capture can, so the skip now guards that fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(studio): show meters only for videos Web Audio carries
A plain audible video at unity stays on native output, which the meters
cannot read, so it no longer advertises them. The runtime's join rule and the
meter gate now share audibleVideoNeedsWebAudio (core/audible-video).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(cli): one owner for init media placeholders, audio stays on the video
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(cli): init keeps a video's sound on the clip; silent files scaffold muted
Supersedes the patchVideoSrc changes in #3977 and #3942 and carries their tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: miga-heygen <miga-heygen@users.noreply.github.com>
Co-Authored-By: kapelame <kapelame@users.noreply.github.com>
* feat(registry): example templates keep the a-roll's sound on the video
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(lint): video audio rules recommend keeping sound on the video
Update video_missing_muted and video_audio_double_source message and
fixHint text to recommend the new default: keep audio on the video with
data-has-audio="true" (not muted). Tests verify the new wording is in place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(parsers): a video is either audible (data-has-audio) or muted, never both
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(studio): follow the compiler's audio rule in the audio-track toggle
Confirm before background removal drops a video's sound.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(skills): hyperframes-core teaches sound on the video by default
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(skills): workflows and scaffold guidance keep a video's sound on the clip
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: footage with sound keeps it on the video clip
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): init from-file a-roll is timed, muted strip is attribute-exact
The from-file a-roll video gets data-start="0" so a scaffold lints clean, and
the orphaned audio-only slot comment is dropped. setVideoAudio now walks the
tag attribute by attribute, so a class value containing the word muted
survives. The ffprobe-missing warning says the video is assumed to have sound
and how to switch to muted, and DEFAULT_META notes the failure surfaces at
render until lint support lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(studio): keep has-audio in step with the muted toggle, test cutout confirm
Muting a video now clears has-audio and unmuting sets it to true, in both the
Design and flat media panels, matching the Has audio track toggle. Audio
elements are unchanged. Adds coverage for both panels plus the cutout
window.confirm guard: declined leaves src alone, accepted swaps src and mutes,
a silent video never prompts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(parsers): updateElementInHtml hasAudio only applies to videos, true or false
An audio element or an undefined hasAudio no longer picks up a muted attribute;
only an explicit false on a video writes muted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: video_missing_muted rule text, silent-footage notes, consistent audible examples
lint.mdx and cli.mdx describe the current video_missing_muted rule instead of
the retired unmuted-video wording. The editing recipes say how to adapt for
silent footage and require the outgoing clip's sound to end at a J-cut. The
Remotion media example shows the audible mapping consistently, data-attributes
states that has-audio is required unless muted, and the hairline frame note no
longer claims the video is muted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): encode init media filenames and probe --audio duration
Function replacers stop dollar-sign pattern expansion in filenames; src is percent-encoded.
--audio now passes the probed track length as the composition duration in both init paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(lint): audio-state-aware media rules and time-aware double-source check
video_missing_muted honors any explicit data-has-audio.
video_audio_double_source only considers audible videos and only fires
when the same-src clips overlap in time.
duplicate_audio_track now covers audible videos.
audio_file_without_element no longer claims silence when an audible video
carries the sound.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(skills): audio-stream check for remotion ports, followable J-cut recipe
Remotion Video maps to data-has-audio only when the file has an audio stream.
The J/L-cut recipe now shows both shots with muted pictures and separate audio.
embedded-captions removes the audio.mp3 scratch file it extracts.
skills-manifest.json regenerated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(lint): trim comments to satisfy the comment ratchet
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(studio): atomic muted edits, cutout keeps sound on a linked audio
Muted and Has audio track commit one batched write (one persist, one
undo entry) and remove data-has-audio instead of writing an empty
marker. Unmuting a video probes its src: audible -> true, silent ->
false, probe failed -> absent. Remove background on an audible video
inserts a linked sibling <audio> with the original sound in the same
write, replacing the drop-the-sound confirm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(producer): skip hidden videos when auto-adding video audio
A video hidden in Studio is dropped by the mixer, so its src never reached
existingAudioSrcs and the render re-added it at full volume. The video parser
now records the shared hidden predicate and the auto-detect step skips hidden
videos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(studio): keep useDomEditSession under the 600-line cap
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: miga-heygen <miga-heygen@users.noreply.github.com>
Co-authored-by: kapelame <kapelame@users.noreply.github.com>
* feat(cli): add hyperframes clean to find and remove what HyperFrames left on disk
`hyperframes clean` lists leftover render work dirs, debug renders, the history of projects
that no longer exist, idle video proxies and extracted video frames, with sizes, and removes
them. `--dry-run` removes nothing, `--json` is for agents, `--snapshots` also removes
snapshot folders (otherwise listed as reclaimable). It never touches outputs, sources or
anything a running render or preview still owns.
* docs(cli): say that clean does not search a project's subfolders
* fix(cli): clean test compares folder names on Windows and the skills manifest is regenerated
* fix(studio-server): the proxy cache sweep removes only files the transcoder named
It removed any idle .mp4 or .webm in .transcode-cache, so a user file placed there, or reached through
a symlinked cache folder, could go. Proxies are always <sha256>.<ext> and their temp files
.tmp-<uuid>-<proxy name>; nothing else is swept.
* style(producer): keep a renderDirOwner doc comment within 120 columns
Ctrl+C during a render left its whole work dir of captured frames (and a
staging dir) in the renders folder. Puppeteer's default SIGINT handler kills
Chrome and exits the process before the render's own cancel path can remove
them. The engine keeps that default unless the host owns Ctrl+C; the CLI's
render cancellation scope claims it while a render runs, so the render aborts
and cleans up after itself.
A render killed outright (SIGKILL, crash) cannot clean up, so each work dir and
staging dir now records its owner process (pid, hostname, boot, and on Linux
the pid namespace), and the next render into the same folder removes
the ones whose owner has exited.
Cancelling a render while its browsers start no longer leaves anything
running that kept a batch render alive after Ctrl+C: the BeginFrame warm-up
loop, the probe browser and file server, or a capture browser whose launch
finished after the cancel (now closed before the worker settles, since
Puppeteer's exit hook skips every other live browser).
* fix(engine): forced-SDR renders show tone-mapped HDR footage as Studio does
Forced-SDR extraction tone-maps HDR frames with zscale, but the frames kept the
BT.709 transfer tag, so Chrome re-curved them (a tone-mapped 388d96 painted as
2d8790), and JPEG frames kept the BT.709 matrix every JPEG reader decodes as
BT.601. Tone-mapped frames now get the same sRGB tag and BT.601 JPEG conversion
as other SDR frames.
macOS used VideoToolbox, which does not tone-map: its frames stay PQ and Chrome
tone-maps them itself, darker than the hable curve Linux and Studio use. macOS
now takes the zscale path whenever ffmpeg has zscale, found through the shared
ffmpeg filter probe; without it the old path stays and the render warns once.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): forced-SDR renders tone-map HDR footage that lacks matrix or primaries tags
zscale finds no path between colourspaces without them, so the extraction
failed. Untagged HDR footage is BT.2020 in practice, and now gets those tags.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): forced-SDR tone map also tags the transfer ffprobe read from the container
From ffmpeg 8.1 ffprobe reports a container-only PQ or HLG tag; the frames
still lack it, so zscale failed where macOS VideoToolbox used to render.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* test(engine): gate the container-only transfer case on the ffprobe that reads it
* docs(engine): shorten the filter-probe comment to the comment limit
* fix(engine): keep valid per-frame colour tags when tone-mapping HDR footage
* test(engine): build the mixed HLG and PQ fixture without B-frames so its timestamps are valid
* test(engine): build the trimmed mixed fixture without B-frames too
---------
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): renders show the video frame the preview shows below the source frame rate
* refactor(engine): one condition owns CFR sampling at the output rate
* fix(engine): webm and mkv clips keep every frame when rendered at their own rate
* fix(engine): only fine timebases get the half-tick sampling shift, so coarse ones stay exact
* docs(engine): the sampling comment states the half-tick bound it keeps
* fix(engine): move the extraction cache to v6 so main v5 entries are not reused
* docs(engine): keep extraction comments within their share
* test(producer): regenerate four 25-to-30 fps baselines for the exact frame pick
* test(engine): name the v6 schema prefix in its test
* fix(engine): declare SDR extracted frames as sRGB so Chrome does not darken footage
* fix(engine): encode SDR mp4 frames with the BT.709 matrix they are tagged with
Chrome hands the SDR encoder BT.601 full-range JPEG captures (or RGB PNGs).
The encoders only converted the range, so swscale kept the BT.601 matrix under
the BT.709 tag (ffmpeg 5.1 to 7.0: CSS #C83C28 delivered as #D44824) or
converted YUV to YUV through a low-precision step (ffmpeg 8+: #101010 delivered
as #0B0D0D). Decode to RGB with the capture matrix, then encode with BT.709.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): tag SDR frames plain sRGB so every ffmpeg writes the sRGB chunk
ffmpeg writes a PNG sRGB chunk only for BT.709 primaries and writes cICP only
from 6.1. On ffmpeg 5.1 (Debian bookworm) a BT.601-tagged source therefore got
cHRM plus gAMA 1/2.2, which Chrome decodes as a pure 2.2 power: 0x10 shadows
rendered as 0x07. Declaring BT.709 primaries too gives the sRGB chunk on every
version, so Chrome shows the frame code values for every SDR source. The test
now reads the PNG chunks instead of ffprobe, which reports no transfer for an
sRGB chunk on ffmpeg 5.1.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): extract SDR jpg frames as the BT.601 full range JPEG readers assume
The default frame format for opaque SDR sources is JPEG. ffmpeg converted the
BT.709 source to full-range YUV without changing the matrix, and every JPEG
reader (Chrome included) decodes BT.601, so footage shifted: 0xC83C28 showed as
0xBB2D29. The old mp4 encode made the inverse mistake and hid it; with the
encode fixed, it would reach the output. Decode through RGB with the source
matrix, then encode BT.601 full range, keeping 4:2:0.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): round the SDR mp4 colour conversion accurately
swscale's default rounding moved saturated colours up to 3 codes (pure red 254
delivered as 251 on ffmpeg 5.1 to 7.0); accurate_rnd brings that to 1. The
chunk encoder now decides the HDR-or-SDR filter once, and the 10-bit GPU pad
test sets hdr, which its name always claimed.
* style(engine): keep the colour filter reasons in names and tests, not source comments
The comment ratchet caps a file's comment share: the new filter module and the
extractor carry their intent in constant names, and the encode test states why
the conversion goes through RGB.
* fix(engine): keep colour edges sharp through the SDR colour conversions
Going through RGB upsampled chroma with swscale's default filter, so ffmpeg 4.4
to 7.x smeared colour across edges by about 2 px (red 254 next to blue came out
8 inside the blue). Nearest-neighbour resampling maps the half-resolution chroma
back one to one, keeps the flat-colour results, and makes accurate_rnd redundant.
The pixel tests now also check a red to blue edge.
* test(producer): regenerate goldens whose colours the old mp4 matrix bent
These four fixtures compare at 40 to 45 dB, and their goldens were rendered
with the BT.601-under-BT.709 encode, so the corrected colours score 34 to 37 dB.
Regenerated in the Dockerfile.test image. Each new golden is closer to the
colour its page declares, e.g. #00d4aa was 00bcab and is now 00d3aa, and
#087443 was 006643 and is now 087241.
* fix(engine): keep VFR frame timing when extraction applies colour filters
ffmpeg before 6.1 ignores frame durations in CFR resampling once any -vf is
set, so a VFR window ending on a still lost frames. Filtered VFR extraction
now resamples in one process exactly as before and filters in a second.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): encode raw SDR layered frames in BT.709
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* test(engine): keep the new colour tests inside the format, comment and argv checks
* fix(engine): keep HDR and unusual pixel formats out of the two-process VFR path
The raw handoff dropped HDR10 light levels and misread semi-planar formats, and
decoder options rejected some ffprobe colour names. Only SDR sources in formats
nut carries unchanged now take it, their colour tags come back through setparams,
and a process that cannot start releases the other.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): read untagged footage with the matrix Chrome plays it with
Chrome decodes an untagged matrix as BT.709 from 720 coded lines up and as
BT.601 below; extraction assumed BT.601 at every size, so HD footage without
colour tags shifted hue against Studio's own preview.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): untagged VP9 and AV1 footage keeps BT.601, as Chrome plays it
Chrome applies the 720-line rule only to codecs it decodes through ffmpeg
(H.264, VP8); untagged VP9, VP9 alpha and AV1 play as BT.601 at every size.
Also widens the raw-pipe pixel formats to all verified ones, and reports the
pipeline process that failed first by settle order, not by per-process clocks.
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* test(producer): regenerate goldens for corrected page colours and untagged SD footage
* docs(engine): say which untagged codecs the matrix rule was measured on
---------
Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com>
* fix(engine): export draws a video in exactly the frames export visibility shows it
* test(producer): guard that chunk frame planning snaps clip windows like export
* refactor(engine): set the render fps once per frame table and name its clip window
* chore(engine): trim frame table comments
* fix(core): clips that meet within float rounding count as meeting in edits and playback
* fix(engine): hand a shared clip boundary to the incoming clip in export, as preview does
* fix(producer): give each parallel segmented-capture worker its own browser
Segmented capture with two or more workers failed at the first frame with
"Invalid JPEG input at frame 0: empty frame". Every worker opened its page in
one pooled Chrome, and beginFrame capture does not survive a shared browser:
a worker's first beginFrame returned no screenshot. Fresh worker sessions now
follow shouldDisableBrowserPoolForParallelWorker, the rule the parallel disk
path already uses.
The default session factory was also built once per worker, so with a probe
session every worker reused that same page. It is now built once per run, so
only the first worker takes the probe.
Workers are now created inside the try, so when a later worker fails to
launch, the finally still closes the browsers opened before it.
* fix(producer): close a segmented worker session whose initialization fails
The default segmented session factory opened a fresh browser and then
initialized it with nothing owning it on failure, so a page-load or init
timeout leaked that Chrome. It now closes the session and rethrows, as the
streaming stage does.
