* fix(producer): build the render job before taking a render slot
A request whose options fail the strict render-request validator now gets
a 400 on both the blocking and streaming routes, and its inline project
directory is removed. Validate workers up front like the other overrides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(producer): write compiled sub-compositions only inside the compiled directory
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(engine): keep media work files inside their directory for any element id
Map a media element id to one plain path segment before it names a frame
directory or an intermediate audio file. Ids that already are one plain
segment map to themselves, so existing paths stay the same; the producer
and the distributed plan derive the same segment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(engine): listen on loopback by default in the engine file server
The file server feeds the co-located capture browser, which loads it from
localhost. A caller that wants another interface passes hostname.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(producer): refuse cross-origin browser posts to the standalone render server
A web page on another origin can no longer start a render or lint through
the server's POST routes. Requests without an Origin header (CLI tools,
server-to-server callers) and same-origin pages are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(engine): trim comments on the media path segment and file server host
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(producer): trust the browser's Sec-Fetch-Site verdict for render server posts
A same-origin page behind a reverse proxy that rewrites the Host header sent an
Origin that no longer matched Host, so the cross-origin guard refused it. The
guard now uses Sec-Fetch-Site when the browser sends it and falls back to the
Origin and Host comparison only for browsers that do not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(engine): keep generated media path names apart from plain ids and within filename limits
A plain media id that already ends like a generated name now takes the hashed
branch too, so it can no longer share frame folders or audio work files with
the unsafe id that produced that name. The readable part of a generated name
is capped at 64 UTF-8 bytes before the digest, so long or multibyte ids that
contain separators stay within the 255-byte filename limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): staging folders beside a project path no longer fail past 260 characters on Windows
mkdtemp does not get the long-path prefix that mkdir does, so every staging folder
made beside a project or output path failed once its full path passed 260 characters.
They now come from one helper, createStagingDir in @hyperframes/core/atomic-file, and a
scan test fails on a new mkdtemp beside a path.
* fix(cli): give mkdtemp the long-path form instead of replacing it
CodeQL treats mkdtemp as the safe way to make a folder under the system temp dir and
flagged every file written into a folder from the mkdir-based helper. createStagingDir
now calls mkdtemp with the long-path form of the path on Windows, so both hold.
* fix(cli): leave non-Windows paths alone when making a staging folder
The long-path form is now chosen by the shape of the path, not process.platform,
so tests that pretend to be Windows keep working and the comment share does not rise.
* fix(core): a staging folder keeps a Windows path that already names its namespace
* fix(core): a staging folder keeps a Windows device path as the caller gave it
* fix(core): the staging folder helper's comment fits the line limit
* fix(engine): read a video's length from its packets when its headers state none
* fix(engine): ignore the cut first line of a long packet scan and count N/A as a frame
* chore(engine): trim the packet scan comments and put its doc back on the function
* style(engine): format the packet duration test
* fix(engine): GPU renders use the NVENC preset names the host's ffmpeg lists
* fix(engine): one ffmpeg stdout reader for encoder detection and preset listing
* ci(engine): raise the engine unpacked size budget for the NVENC preset reader
* fix(producer): auto renders with color-graded media stay SDR instead of promoting to HDR
* fix(producer): keep auto renders SDR when a graded canvas is present
* fix(cli): capture's public-address check uses the engine's list; 192.0/16 is not all reserved
Only 192.0.0.0/24 and 192.0.2.0/24 are special-use. The capture proxy blocked all of
192.0.0.0/16, so sites on WordPress VIP (192.0.64.0/18, e.g. nasa.gov at 192.0.66.108)
failed every capture with net::ERR_TUNNEL_CONNECTION_FAILED. isPublicIPv4 now reads the
engine's isBlockedNetworkHost, so capture and asset downloads share one list.
— Jerrai (cutroom-debug)
* refactor(engine): one special-use network list, in a light subpath capture imports
Importing all of @hyperframes/engine into capture's publicNetwork slowed the lookup child's
start-up past publicNetwork.process.test's 6 s mid-lookup exit. The list and
isBlockedNetworkHost move to engine/src/utils/networkPolicy.ts, exported as
@hyperframes/engine/network-policy; urlDownloader and the engine index read it there.
— Jerrai (cutroom-debug)
* test(engine): pin 192.0.0.0/24 and 192.0.2.0/24 blocked, the rest of 192.0/16 public
— Jerrai (cutroom-debug)
* fix(cli): alias the engine's network-policy subpath in the CLI bundle
esbuild's root alias rewrote @hyperframes/engine/network-policy to src/index.ts/network-policy;
the exact-subpath list now carries it. The publicNetwork docstring is cut to keep the file's
comment share under its base.
— Jerrai (cutroom-debug)
* fix(engine): a video on a non-square pixel grid renders at its display shape
* chore(engine): drop comments the names already carry
* fix(engine): a rotated clip with a non-square pixel grid keeps its display shape
* test(engine): the frame probes end their options with -- like every ffprobe call
* fix(engine): late-starting videos play to their last frame
A video whose first frame is after timestamp 0 could end early (frozen
tail) or late: ffprobe's stream duration does not run from the first
frame in every container (fragmented MP4, Matroska). Such a video's end
is now read from its last packet, once, where its facts are probed.
* test(engine): the WebM-with-audio case measures its first frame from the file
* test(engine): the first-frame probe ends its arguments with the input file
* fix(engine): late-starting MP4 and WebM videos render every frame the preview shows
The browser counts an MP4 or WebM from timestamp 0, but FFmpeg seeks from the file's
earliest stream start. A file whose first frame comes after 0 (a fragment cut from a
longer stream, an edit-list offset, a WebM with a start offset) was extracted early and
short, so renders tripped the video coverage gate. Every video seek now maps media time
to FFmpeg's timeline, and the video begins where the browser shows it.
* fix(engine): audio from late-starting MP4 and WebM plays when the preview plays it
The mixer seeked and trimmed audio by FFmpeg's clock, which starts at the file's earliest
stream. In an MP4 or WebM whose streams all start after 0, the browser plays that audio
later, so the render heard it early by the same amount. The mixer now maps media time onto
FFmpeg's clock with the same offset frame extraction uses.
* fix(engine): a late-starting file's picture shows on the clock its audio plays on
* test(producer): vfr-screen-recording baseline shows each frame at its timestamp
* perf(engine,cli): a machine's GPU encoder probe runs once, not on every render
* refactor(engine): one shared probe per process, no answer kept on disk
* ci(windows): probe that a render opens no visible window
* ci(windows): the probe names what opened each window
* ci(windows): the probe spawns each binary alone, with and without detached
* ci(windows): the probe logs the processes started during the render
* ci(windows): the probe also spawns Chrome one consoleless hop down
* ci(windows): the probe logs each child started without windowsHide
* ci(windows): the probe logs every child process with its windowsHide and detached
* fix(engine): Chrome keeps the CLI's console on Windows, so a render opens no window
* fix(cli): every child the CLI starts on Windows is hidden unless it opts out
* ci(windows): the probe bisects Chrome's flags and keeps each command's output
* ci(windows): the probe names the Chrome process behind each window
* ci(windows): the probe loads a page in Chrome directly and one hidden hop down
* ci(windows): the probe prints the Chrome main command line and renders with the reporter's flags
* ci(windows): the probe narrows the CLI's Chrome arguments to the ones that open a window
* ci(windows): the probe launches the CLI's Chrome arguments one hidden hop down
* ci(windows): the probe launches Chrome through the engine's browser pool
* ci(windows): the probe varies what the engine passes to Puppeteer
* fix(engine): renders on Windows launch Chrome with a Puppeteer that keeps it attached
* refactor(cli): hideChildConsoles copies its arguments and keeps one comment line
* fix(engine): extraction waits out the decode up to a deep seek in a long-GOP video
* chore(engine): shorten the keyframe gap doc comment
* fix(engine): cancelling an extraction during the keyframe probe reports a cancel
* fix(engine): a segment stopped by another's failure reads as stopped, not cancelled
* fix: phone videos with an undecodable first audio track check, preview and render
* fix(engine): a rate-lane clip bakes its ramp from the decodable audio track
* test: phone-clip tests end ffprobe options before the input
* fix(engine): a scratch folder Windows refuses to delete no longer fails a finished audio mix
* fix(engine): audio scratch files Windows refuses to delete no longer fail a render
* fix(engine): a rename Windows keeps refusing falls back to a copy so the audio mix finishes
* fix(engine): audio scratch left behind is counted in the render log
* style(engine): wrap the long rename fallback test call
* refactor(engine): split the rename fallback decision out of the retry loop
* fix(engine): the rename copy fallback is opt-in and scratch leftovers are counted per render
* style(engine): keep renameWithRetry's comment share at the base
* style(engine): drop the blank line left by the comment cut
* fix(producer): convert HEIC images to JPEG at compile time
Tiled HEIC photos probe as several video streams, so the media-type
preflight rejected them as image:video, and Chrome cannot decode HEIC.
Sniff the ftyp brands of every <img>, join and rotate the tile grid with
ffmpeg, verify the declared size, and swap in a cached JPEG.
* fix(producer): shorten the HEIC prep comments
* fix(producer): name host tool failures and keep a failed HEIC preview image as authored
Also: HEIC tests run in the integration lane; the primary grid is picked by default disposition.
* chore(producer): raise the package size budget for the HEIC prep
Frame extraction decoded every source in software, so a long 4K60 HEVC clip
spent much of its CPU on decoding. On macOS, extraction now asks ffmpeg for
VideoToolbox when this ffmpeg has it and the source cannot carry alpha; ffmpeg
falls back to software for a stream VideoToolbox cannot decode, and the frames
written are byte-identical.
* fix(engine): ffmpeg steps and downloads stop on no progress, not while still advancing
* test(cli): an ffmpeg step that stops advancing reports inactivity, not a deadline
A VP8/VP9 source with no alpha tag has one frame decoded with the alpha-aware decoder when its
metadata is read; if that frame has no alpha plane, its frames extract as jpeg instead of png.
Tagged alpha, a decoded alpha plane, a failed probe, or a plan written without the field keep png.
* perf(engine): a remote source's tail decodes run many at a time, not one per half-core
* perf(producer): the plan logs where its extract stage spends its time
* perf(engine): a window that ends inside its source takes its frame count without a decode
* test(engine): pin the tail-decode slot test to the windows that decode
* style(engine): format the tail-probe window check
* test(engine): deferred counts match extraction on a B-frame NTSC source; VFR is not deferred
* style(engine): format the tail-probe tests
* perf(aws-lambda): a site stores each video's ffprobe JSON, and the plan reads it instead of probing
* test(aws-lambda): a site stores and hands out each video's probe
* style(aws-lambda): format the probe-facts upload and tests
* perf(engine): a site's stored probe also holds the final-frame timestamp the plan would probe for
* test(engine): drop an unused import
* test(aws-lambda): the stored probe holds the ffprobe JSON and the final frame
* test(aws-lambda): the stored probe holds the ffprobe JSON and the final frame
* perf(producer): a distributed chunk encodes while it captures
* perf(producer): the audio task logs where its mix spends its time
One line per audio stage: prepare, groups, mix, master, encode and the peak check, in ms.
* perf(engine): locked warmup ticks stop sleeping once the page is ready
* perf(engine): a window reaching a stored video's end takes its frame count from stored facts
* fix(producer): the cloud audio task logs its phases
* perf(engine): stored facts keep the last packet length so late-start windows skip the tail decode
* fix(engine): a probe spawns without waiting when no stored facts are declared
* chore(aws-lambda): shorten comments over the line limit
* fix(engine): warmup keeps pacing after a failed tick; stored final frames check ffprobe major
* test(engine): the recorded ffprobe major matches the build's own banner
* test(engine): the version-banner probes go through a named binary like the other spawns
* fix(engine): renames ride out Windows file locks instead of failing the render
* fix(producer): the true-peak and segment-manifest renames ride out Windows file locks too
* perf(engine): remote sources of a site keep their probe results for the whole render
* test(engine): probe a content-addressed URL once per render
* style(engine): format the audio probe split
* perf(aws-lambda): a render reads a video's audio from a small file stored beside it
A video's audio is interleaved with its pictures, so mixing it from the cloud read the whole video (2.5 GB for a 20-minute 4K film) at the Lambda's network speed. lambda sites create now stores each large video's audio packets as a separate object keyed by the video's hash; the audio task reads that when it exists.
* chore(producer): shorten the audio sidecar header comment
* fix(aws-lambda): no audio sidecar for a video whose audio starts late or that has several tracks
* test(producer): format the audio sidecar tests and end their ffprobe options with --
* fix(studio-server): keep transparency in proxies of VP8/VP9 alpha webm
A webm whose alpha lives in the VP8/VP9 side channel (alpha_mode=1,
pix_fmt yuv420p) was proxied to opaque H.264, and the VP8 proxy path
decoded it with ffmpeg's native decoder, which drops alpha. The alpha
rule and the alpha-reading decoder now live once in
@hyperframes/parsers/media-alpha, used by engine and studio-server.
* build(parsers): raise the packed size budget for the media-alpha subpath
parsers already sat at its packed budget on main; the shared alpha rule
adds 676 B of JS and 803 B of declarations (190958 B packed in CI).
* perf(engine): cloud renders cut a video's audio from its URL instead of downloading the file
* refactor(engine): keep the audio mixer's comment share at the base
* fix(engine): a cloud-held file's audio probe retries once, then fails instead of mixing unprobed
* fix(engine): a cancelled probe is not retried, and the encode test noise is seeded so it repeats
* test: queue runs on slow windows runners stop failing four timing tests
* test(cli): the keepalive process test budgets for a cold start on a slow runner
* fix(engine): long 4K video exports no longer time out while frames are still coming
* test(engine): the paced extraction test keeps a wide margin on a loaded runner
* fix(engine): long phone-video exports probe each source once instead of once per cut
* fix(engine): a failed source probe is retried, and probes share the extraction slots
* test(engine): synthesize the tone-mapped concurrency fixture with known input colour
* perf(producer): distributed chunks download only the videos they show
A project video under compiled/ that chunks extract their own frames from
now goes only to the chunks that show it, like video-sources/ files. A
chunk skips the readiness wait for a deferred video it does not show, so
it no longer logs media_load_failed for a file it was never given.
* fix(producer): cloud site uploads pack the files their id hashes, symlinks included
* perf(aws-lambda): chunks read only the byte ranges of a video they need
A chunk that extracts its own frames from a deferred project video now
reads it from S3 by byte range through a short-lived pre-signed URL,
instead of downloading and hashing the whole object first. S3 attests the
object's stored SHA-256 before the URL is signed; every other artifact is
still downloaded and verified as before.
* perf(aws-lambda): upload project files once to a shared content-addressed area
* chore(producer): keep planV2 comment share at the base level
* fix(engine): verify the TLS certificate of https video inputs and never surface their signed query
* fix(aws-lambda): renders re-upload a site's expired files instead of failing
* perf(aws-lambda): the plan step probes project videos in place instead of downloading them
* fix(engine): https video reads reconnect and time out, and redaction covers wrapped causes
Signed-URL reads now pass -rw_timeout, -reconnect and -reconnect_delay_max (both Lambda builds accept them) and ffmpeg also -reconnect_on_network_error, so a transient S3 error or a stalled read fails fast or recovers instead of waiting out the process timeout. Query redaction now descends into error.cause.
* fix(aws-lambda): share entry bundles and ratchet package budget
* fix(aws-lambda): site ids uploaded before files.json still render
A site id from an earlier sites create has only sites/<id>/project.tar.gz.
renderToLambda now falls back to that tarball, which the plan step still
reads, and when neither key exists it fails before starting with both keys
named and the fix (run sites create again).
* fix(aws-lambda): renders fail before upload on a stack too old to read sites
A stack deployed before files.json sites untars the site file in its plan
step and fails after four retries. renderToLambda now reads the state
machine definition first and stops with STACK_TOO_OLD and the redeploy
command when the plan task does not list SITE_MANIFEST_INVALID, which new
SAM and CDK stacks now treat as terminal. The README's upgrade section
drops the planProtocol v1 fallback, which hits the same failure.
* docs(aws-lambda): document shared content storage and the upgrade path
The deploy guide still described the tarball HeadObject short-circuit. It
now covers the cas/sha256/ store and its 30-day lifecycle rule, the
STACK_TOO_OLD, SITE_FILES_MISSING and SITE_NOT_FOUND errors, that old site
ids keep working, and that every project uploads once more after the
upgrade. A test keeps a troubleshooting entry for each pre-submit error.
* fix(producer): chunks log each video they read by URL, without its query
A chunk reading a video straight from the bucket left no trace in its log.
It now logs one line per remote source with the path and the URL minus its
query string, so signatures never reach the log.
* fix(engine): ffmpeg errors never keep part of a signed URL's query
A process's stderr tail was trimmed before its URLs were redacted, so a
cut inside a signed URL left a query fragment the URL pattern no longer
matched, and the fragment reached the error. The stderr tail is now
redacted before every trim, and the redactor also strips a signing
parameter cut off from its URL.
* fix(aws-lambda): comments name the site manifest instead of a file path
* fix(engine): stderr redaction survives chunk splits, and every ffmpeg spawn uses it
The stderr tail was redacted on each chunk and then cut, so once a cut
severed a long URL's scheme, the rest of its signature in the next chunk
matched neither pattern. The tail is now kept raw with 2x slack and
redacted only when the text is produced, then cut to size. The unused
producer audioExtractor (its own spawn, unredacted stderr) is deleted and
the parity harness's PCM decode goes through ManagedChildProcess; a test
fails on any ffmpeg/ffprobe spawn in engine, producer or aws-lambda that
does not.
* chore(producer): record the audioExtractor deletion as intended
* fix(engine): drop a signature fragment the stderr tail cut from its URL
The raw tail is kept at twice the cap, but that cut can itself land inside a
signature, and when redaction shrinks the rest below the cap the fragment
survived. After a cut, the severed token is dropped before redaction.
* refactor(engine): name the severed-token drop instead of commenting it
* test(engine): a cut-off URL leaves nothing behind; aws-lambda size budget to measured
* fix(producer): chunk URL-read tests use main's frame list; producer size budget to measured
* chore(producer): size budget at the measured size, no headroom
* chore(producer): size budget back to measured x1.05, the check's own rule
* fix(engine): SDR content in HLG exports sits at reference white, not peak
The sRGB to HLG conversion for DOM layers fed SDR-relative light straight into the HLG OETF, whose input is light relative to the HLG peak, so SDR white landed at 100% signal. Graded HLG footage (graded as SDR, then converted back) and every white overlay were pushed to peak and blew out. SDR white now sits at 75% HLG signal (BT.2408), the same model the PQ branch uses with 203 nits.
* docs(engine): shorten the SDR-white note in the HDR LUT
* docs(engine): cut the LUT generator's notes to its usage and drift rule
* fix(engine): SDR exports carry BT.709 transfer and primaries with every encoder
The SDR conversion filter set the BT.709 matrix on its frames but not the
transfer or primaries. Encoders that take colour from the frames rather than
from -color_* options (VideoToolbox on macOS) wrote color_space=bt709 with
color_transfer and color_primaries unknown. The capture and raw-input SDR
filters now end in the tagged BT.709 filter ProRes already uses.
* fix(engine): HDR exports tag BT.2020 with PQ or HLG on the frames too
VideoToolbox HDR encodes had the same gap as SDR: given only -color_* options
it wrote bt2020nc with transfer and primaries unknown, so players read the
export as SDR. Every HDR branch of the streamed and disk encoders now tags the
frames through one hdrFrameTagsFilter built from the encoder colour params.
* test(engine): colour-tag probes pass -- before the output path
* test: ffmpeg failures name how ffmpeg ended and keep its last line
* fix(producer): an ffmpeg failure names how ffmpeg ended and keeps its last line
* refactor(producer): say why in names, not comments
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8
VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.
The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused
v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#4564 routes SDR colour through `format=gbrp`. Without accurate_rnd,
swscale's yuv-to-gbrp conversion leaves the last 8 columns black when the
width is 16n + 8 (1000, 1080, 1112...), so every 1080x1920 portrait MP4
since 0.8.81 has had an 8 px black strip on the right. The same step in
SDR jpg frame extraction blanked the right edge of footage at those widths.
Set accurate_rnd on the yuv-to-gbrp scale in both chains. It is also the
more accurate rounding: on a testsrc2 frame, worst luma error against a
float reference drops from 1.85 to 0.90 codes.
Fixes#5551
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): encode and tag ProRes mov output as BT.709
* test(engine): read the ProRes colour tags from the first frame
* test(engine): end the ProRes tag probe's options with --
* fix(engine): exports place sound and picture right when a source's audio and video start apart
* fix(engine): a late video stream plays to its end and distributed and HDR renders hide its lead
* test(engine): pin a late video stream's trimmed start, held tail and HDR trim
* fix(studio): text in Studio reads without em or en dashes
* fix(studio): the dash check also catches HTML entities, and peak reads Unavailable
* test(studio): the dash check test covers every entity form
* fix(studio): messages Studio shows from lint, sdk, core and the renderer drop dashes too
* fix(producer): the gate check stops early on the engine's new fallback wording
* fix(engine): long exports with many clips no longer stall at 90% or hide a timeout
* fix(engine): stop queued frame extractions on cancel and time out distributed assembly
* refactor(engine): simplify the extraction slot check and its concurrency test
* test(engine): prove a cancelled extraction leaves the queue, skip the shell shim on Windows
* test(engine): wait for the held extraction's own record before cancelling the queued one