* perf(producer): distributed chunks download only the videos they show
A project video under compiled/ that chunks extract their own frames from
now goes only to the chunks that show it, like video-sources/ files. A
chunk skips the readiness wait for a deferred video it does not show, so
it no longer logs media_load_failed for a file it was never given.
* fix(producer): cloud site uploads pack the files their id hashes, symlinks included
* perf(aws-lambda): chunks read only the byte ranges of a video they need
A chunk that extracts its own frames from a deferred project video now
reads it from S3 by byte range through a short-lived pre-signed URL,
instead of downloading and hashing the whole object first. S3 attests the
object's stored SHA-256 before the URL is signed; every other artifact is
still downloaded and verified as before.
* perf(aws-lambda): upload project files once to a shared content-addressed area
* chore(producer): keep planV2 comment share at the base level
* fix(engine): verify the TLS certificate of https video inputs and never surface their signed query
* fix(aws-lambda): renders re-upload a site's expired files instead of failing
* perf(aws-lambda): the plan step probes project videos in place instead of downloading them
* fix(engine): https video reads reconnect and time out, and redaction covers wrapped causes
Signed-URL reads now pass -rw_timeout, -reconnect and -reconnect_delay_max (both Lambda builds accept them) and ffmpeg also -reconnect_on_network_error, so a transient S3 error or a stalled read fails fast or recovers instead of waiting out the process timeout. Query redaction now descends into error.cause.
* fix(aws-lambda): share entry bundles and ratchet package budget
* fix(aws-lambda): site ids uploaded before files.json still render
A site id from an earlier sites create has only sites/<id>/project.tar.gz.
renderToLambda now falls back to that tarball, which the plan step still
reads, and when neither key exists it fails before starting with both keys
named and the fix (run sites create again).
* fix(aws-lambda): renders fail before upload on a stack too old to read sites
A stack deployed before files.json sites untars the site file in its plan
step and fails after four retries. renderToLambda now reads the state
machine definition first and stops with STACK_TOO_OLD and the redeploy
command when the plan task does not list SITE_MANIFEST_INVALID, which new
SAM and CDK stacks now treat as terminal. The README's upgrade section
drops the planProtocol v1 fallback, which hits the same failure.
* docs(aws-lambda): document shared content storage and the upgrade path
The deploy guide still described the tarball HeadObject short-circuit. It
now covers the cas/sha256/ store and its 30-day lifecycle rule, the
STACK_TOO_OLD, SITE_FILES_MISSING and SITE_NOT_FOUND errors, that old site
ids keep working, and that every project uploads once more after the
upgrade. A test keeps a troubleshooting entry for each pre-submit error.
* fix(producer): chunks log each video they read by URL, without its query
A chunk reading a video straight from the bucket left no trace in its log.
It now logs one line per remote source with the path and the URL minus its
query string, so signatures never reach the log.
* fix(engine): ffmpeg errors never keep part of a signed URL's query
A process's stderr tail was trimmed before its URLs were redacted, so a
cut inside a signed URL left a query fragment the URL pattern no longer
matched, and the fragment reached the error. The stderr tail is now
redacted before every trim, and the redactor also strips a signing
parameter cut off from its URL.
* fix(aws-lambda): comments name the site manifest instead of a file path
* fix(engine): stderr redaction survives chunk splits, and every ffmpeg spawn uses it
The stderr tail was redacted on each chunk and then cut, so once a cut
severed a long URL's scheme, the rest of its signature in the next chunk
matched neither pattern. The tail is now kept raw with 2x slack and
redacted only when the text is produced, then cut to size. The unused
producer audioExtractor (its own spawn, unredacted stderr) is deleted and
the parity harness's PCM decode goes through ManagedChildProcess; a test
fails on any ffmpeg/ffprobe spawn in engine, producer or aws-lambda that
does not.
* chore(producer): record the audioExtractor deletion as intended
* fix(engine): drop a signature fragment the stderr tail cut from its URL
The raw tail is kept at twice the cap, but that cut can itself land inside a
signature, and when redaction shrinks the rest below the cap the fragment
survived. After a cut, the severed token is dropped before redaction.
* refactor(engine): name the severed-token drop instead of commenting it
* test(engine): a cut-off URL leaves nothing behind; aws-lambda size budget to measured
* fix(producer): chunk URL-read tests use main's frame list; producer size budget to measured
* chore(producer): size budget at the measured size, no headroom
* chore(producer): size budget back to measured x1.05, the check's own rule
* fix(engine): SDR content in HLG exports sits at reference white, not peak
The sRGB to HLG conversion for DOM layers fed SDR-relative light straight into the HLG OETF, whose input is light relative to the HLG peak, so SDR white landed at 100% signal. Graded HLG footage (graded as SDR, then converted back) and every white overlay were pushed to peak and blew out. SDR white now sits at 75% HLG signal (BT.2408), the same model the PQ branch uses with 203 nits.
* docs(engine): shorten the SDR-white note in the HDR LUT
* docs(engine): cut the LUT generator's notes to its usage and drift rule
* perf(producer): assemble chunks that carry audio under one faststart index
Reads each chunk's moov, merges the sample tables into one moov at the front, and writes
the chunks' mdat payloads after it unchanged: the same bytes storage can copy into place.
The assembler no longer needs the plan audio for such plans.
* perf(aws-lambda): build the final mp4 inside S3 with its index up front
The Lambda plan asks chunks to carry their audio. Assemble writes the faststart header as
part 1 of a multipart upload and copies each chunk's mdat payload with UploadPartCopy, so
the video bytes never pass through the function; any failure aborts the upload.
* fix(engine): a chunk's first frame shows its own time, not the composition's opening frame
* fix(producer): stitch silent AAC tails and clean failed outputs
* fix(producer): open stitched chunks read-only with an explicit mode
* chore(aws-lambda): raise the package size budget for S3-side MP4 assembly
handler.js and index.js each bundle the new S3 stitch code (+5,550 bytes apiece) and s3Stitch.d.ts adds 970: +3,479 packed and +12,070 unpacked over main, measured with the CI pack. The budget rises by exactly that delta.
* fix(aws-lambda): v1 plans give chunks their audio too, so both protocols match
* perf(producer): plans whose chunks carry audio encode it once, at the frames' exact length
* test(producer): terminate the plan audio ffprobe argv like every other probe
* fix(engine): SDR exports carry BT.709 transfer and primaries with every encoder
The SDR conversion filter set the BT.709 matrix on its frames but not the
transfer or primaries. Encoders that take colour from the frames rather than
from -color_* options (VideoToolbox on macOS) wrote color_space=bt709 with
color_transfer and color_primaries unknown. The capture and raw-input SDR
filters now end in the tagged BT.709 filter ProRes already uses.
* fix(engine): HDR exports tag BT.2020 with PQ or HLG on the frames too
VideoToolbox HDR encodes had the same gap as SDR: given only -color_* options
it wrote bt2020nc with transfer and primaries unknown, so players read the
export as SDR. Every HDR branch of the streamed and disk encoders now tags the
frames through one hdrFrameTagsFilter built from the encoder colour params.
* test(engine): colour-tag probes pass -- before the output path
* fix(engine): a 29.97 fps camera clip exports smoothly at 30 fps
Extraction samples each output frame from the last source frame starting at
or before it. Millisecond container ticks move each frame by up to half a
millisecond, while 29.97 fps drifts against 30 fps by 0.033 ms per frame, so
the rounding decided the slot and the export alternated repeats and skips.
Constant-rate frames are now re-spaced on the stream's own frame grid from
the first decoded frame before sampling. The cache schema moves to v8 so
exports stop reusing frames extracted the old way.
* fix(engine): every extraction process snaps frames to one absolute grid
The grid was anchored on each process's first decoded frame, which carries its
own millisecond rounding, so a segment or distributed chunk could place the
29.97 -> 30 repeat elsewhere and skip a frame at its seam. Frames are now
snapped to the stream's own grid in file time (this process's seek plus the
video's lead), the same in every process.
* fix(cli): check and snapshot wait for a slow CDN script as long as render does
check, snapshot, layout, motion-shot and validate gave the page 10 s to reach
DOMContentLoaded while render gives it the engine's page-navigation timeout
(60 s), so a parser-blocking script from a slow CDN failed the diagnostics
with "Navigation timeout of 10000 ms exceeded" on projects that render fine.
The separate 10 s budgets are gone: every caller takes the engine's value
(and its env override), raised only by a command's own longer --timeout.
The cold-Chrome allowance for sending the first request is unchanged.
* fix(cli): check and snapshot read only the page-load budget from the engine
resolvePageNavigationTimeout() owns the page-navigation env read for render
and the diagnostics, so check and snapshot no longer resolve the whole render
config, which printed render's Windows streaming-encode warning. validate's
comment and timeout hint now say render shares the budget.
* perf(engine): long renders encode their audio in parallel segments
* fix(engine): parallel audio encode counts packets from whole samples
The expected packet count came from a floating-point duration, so frame-aligned lengths such as 147.52 s (one ulp above 7080960 samples) expected one packet more than the encoder writes, the last segment check threw, and the render lost its audio. Round to whole samples first; a 147.52 s test fails without the fix.
* fix(gcp-cloud-run): the BeginFrame contract image resolves the engine's core import
The engine config imports the distributed fps set from @hyperframes/core, and the contract stage of the GCP image copies only parsers and engine sources, so the import failed at build time. The set now lives in a small core subpath (@hyperframes/core/distributed-fps) that the engine imports like its other light core imports, and the contract stage copies core like parsers. The core root export is unchanged.
* style(core): format package.json
* perf(producer): assemble in one pass and keep its disk use under one output copy
The distributed assembler joined the chunks into a full-size file, muxed the
audio into a second one and rewrote that again for faststart, so local disk
peaked near four copies of the video and a 30-minute 1080p render came close
to Lambda's 10 GB /tmp. The join, audio mux and faststart now run as one
ffmpeg pass over the concat demuxer; the audio pad/trim takes its frame count
from the chunks. Video and audio packets are unchanged. Assemble and the
Lambda assemble handler log per-stage timings.
* refactor(producer): split assemble into plan check, cfr re-encode and final write
Keeps assemble under the complexity gate and drops an orphaned doc comment in chunkEncoder. No behaviour change.
The integer rates Lambda and Cloud Run accept were written out by hand in
about 25 places across core, engine, producer, the adapters and the CLI.
They now come from DISTRIBUTED_FPS / DistributedFps / isDistributedFps in
@hyperframes/core. Engine config snapshots accept 25 like the rest.
## What a user can do now
Render a video at 720p or 480p: `hyperframes render --resolution 720p` (or `480p`, or the presets `landscape-720p`, `portrait-480p`, `square-720p` and so on) gives a 1280×720 or 854×480 file from a 1080p composition, and the same presets work through the studio-server render route. Before, any preset smaller than the composition was refused ("Downsampling via --resolution is not supported").
## How
- `parsers/src/types.ts`: six presets, `{landscape,portrait,square}-{720p,480p}`, and the `720p` / `480p` aliases (aspect-agnostic, like `1080p` and `4k`). 480p at 16:9 is 854×480: 480 × 16/9 is 853.33, and 4:2:0 encoders need even sides.
- `checkOutputResolutionCompatibility` now allows a preset smaller than the composition. A smaller preset may be off by under one pixel on its long side (the 854); a larger one must still match exactly and be a whole multiple, as before. The aspect remap that `--resolution 1080p` / `4k` apply on their own (`suggestMatchingPreset`, used by the CLI preflight and the compile stage) returns only a preset in the asked tier, never another: with the looser match for smaller presets, a 1922×1080 composition asked for `4k` would otherwise have been remapped to 480p. It is refused instead, as before.
- `resolveDeviceScaleFactor` returns the long side's ratio, so a smaller preset renders at a device scale factor under 1: Chrome lays the page out at composition size and rasterizes each frame straight at 1280×720 or 854×480, the way 4K already renders at 2. For 480p that is 854/1920: the width lands on 854 exactly and the height's 480.375 rounds to 480, so no pad column is added. Output sizes round as Chrome rounds the screenshot.
- The streaming encoder was given the composition's size instead of the output size on both streaming paths (`renderOrchestrator.ts`). 4K never noticed, as both sizes are even; at 480p the encoder skipped its even-size pad and libx264 refused an odd frame (`width not divisible by 2 (853x480)`). It now gets the output size, as the disk-frame encode already did.
- The paths that refused DPR > 1 (BeginFrame, drawElement, the Linux fast-capture gate) now refuse any DPR other than 1, since none of them honor a scale; scaled renders take the screenshot path, as 4K does.
- `render/outputScaleRoute.ts`: the layered compositor (HDR content, and shader transitions in webm and mov) sizes its layers and encoder from the composition and never scales, so a scaled output on that route now stops with a named error, checked where every route and replan is settled (`syncCapturePlan`, beside motion blur's route check). Before, a 1080p request from a 4K composition with shader transitions as mov would have encoded 3840×2160. HDR with `--resolution` was already refused in preflight; 4K on this route had the same gap on main and gets the same error.
## What I measured
- Real renders from source on a Linux box (`bun src/cli.ts render registry/examples/nyt-graph --resolution 720p|480p|1080p`), checked with ffprobe: 720p `h264, 1280x720, yuv420p, 15.0 s`; 480p `h264, 854x480, yuv420p, 15.0 s`; 1080p unchanged at 1920x1080. A frame of each, looked at: same chart and layout, crisp text, and the 480p right-edge column is the page's own color (250,248,242), not a pad.
- Before the encoder-size fix the 480p render failed with `[libx264] width not divisible by 2 (853x480)`; after it, it renders.
- `outputScaleRoute.test.ts`: the real route choice for a 4K composition with shader transitions as mov (`outputSupportsPageSideShaderCompositing`, `shouldUseLayeredComposite`, `createCapturePlan`, `resolveDeviceScaleFactor`) lands on `hdr_layered` at scale 0.5 and is refused. CLI preflight and the compile stage both refuse `4k` on a 1922×1080 composition.
- fails without the fix: without the same-tier rule, the preflight and compile tests for 1922×1080 at `4k` pass the render through at 854×480; without the route check, the mov test does not throw.
- At d33c4458: producer `outputScaleRoute` + `compileStage` 18/18 three runs in a row, `shared` (bun) 9/9, cli `render` 104/104 once and its new test three runs in a row; tsc clean in producer and cli. Earlier at e72f6507: parsers `outputResolutionCompatibility` 20/20, core `index` 26/26, producer `resolveDeviceScaleFactor` 14/14, cli `render` + `parseOutputResolution` 122/122, studio-server `render` 53/53. The new cases fail on main (the presets don't exist; smaller presets are refused). tsc clean in parsers, engine, producer and cli; oxlint and oxfmt clean.
## What I did NOT exercise
- Distributed (Lambda / Cloud Run) renders at 720p/480p: the chunk worker's sizes round the same way, but I did not run a distributed render.
- Studio's render queue keeps its own copy of the rule (`RenderQueue.tsx` `scaleApplies`) and offers no 720p/480p option; unchanged here.
compositionRequiresWebGpu read only the first data-composition-id tag, the
page root. In bundled HTML a mounted composition's root keeps
data-requires-webgpu but carries data-hf-inner-root instead of a composition
id, so a pasted-in WebGPU block launched without WebGPU and failed with
'WebGPU unavailable' even with PRODUCER_ALLOW_SOFTWARE_WEBGPU set.
* test: ffmpeg failures name how ffmpeg ended and keep its last line
* fix(producer): an ffmpeg failure names how ffmpeg ended and keeps its last line
* refactor(producer): say why in names, not comments
* fix(build): ship NodeNext-resolvable declarations from every package
tsc copies relative specifiers verbatim, so packages whose sources use
bundler-style extensionless imports (core, inspector) published .d.ts files
that a module/moduleResolution nodenext consumer cannot resolve. Core already
rewrote its emitted JavaScript for this, but not its declarations.
Move that rewrite to scripts/rewrite-relative-import-extensions.mjs, make it
cover .d.ts too (adding .js or /index.js, and failing the build when a
declaration import has no declaration), and run it after every tsc emit:
core, engine, sdk, producer, aws-lambda, gcp-cloud-run and inspector.
verify:packed-manifests now typechecks the clean packed consumer with
skipLibCheck false and type-only imports of every typed subpath, so an
unresolvable import inside a published declaration fails the gate and is
attributed to its package.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(ci): allow the moved core rewrite-esm-extensions script deletion
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: rebuild the edit-accuracy bench when the import-extension rewriter changes
The package builds now run scripts/rewrite-relative-import-extensions.mjs, so main's
edit_accuracy paths filter has to include it (check-edit-accuracy-filter test).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore: release v0.8.159
* chore: v0.8.159 notes list the first-frame fix
* chore: v0.8.159 notes list the sandboxed GPU check fix
* chore: v0.8.159 notes list the hidden-scene fix and the offline font test
* fix(engine): renders under WSL reach the GPU on Chrome 150 and later
Part of #5548.
* fix(engine): shorten the WSL browser env comment
* fix(engine): cite the Chromium bug in one line and allow the fallback test's removal
* fix(engine): drop WAYLAND_DISPLAY under WSL only when that is what reaches the GPU
* fix(engine): restore the WSL note from before the full-Chrome fallback
* test(engine): give the second worker's page load room on a busy Windows runner
* fix(engine): keep the Chromium bug comment under the line limit
* fix(engine): renders under WSL use the GPU through full Chrome when the headless shell cannot
Part of #5548.
* fix(engine): a missing or unreadable full Chrome cache never stops a WSL render
* fix(engine): sample VFR video by timestamp so it does not freeze on FFmpeg 8
VFR sources were resampled with `-fps_mode cfr -r`, which on FFmpeg 8 honours
each frame's duration. In mp4 files with B-frames those durations are
decode-order gaps, so a frame can claim 0.4s while the next one starts 17ms
later, and the extracted video froze on it. VFR now uses the same fps filter
sampling as CFR (the frame on screen at each output tick), bounded by
-frames:v so a window that ends on a still keeps its rounded-up frame count.
The two-process raw pipe that worked around FFmpeg 5.1 dropping durations
under -vf is no longer needed and is removed. The vfr-screen-recording golden
is regenerated: FFmpeg 5.1's cfr mapping it was rendered with ran one to two
source frames late.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): move the extraction cache to v7 so v6 VFR frames are not reused
v6 entries for VFR sources hold frames sampled by packet duration, which
FFmpeg 8 freezes and FFmpeg 5.1 lags. The key is unchanged by the fix, so
a warm cache served them after the upgrade.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): stop multi-worker renders hanging on media-heavy pages on Windows and macOS
Off Linux, parallel workers shared one pooled browser. Pages in one browser
share Chrome's six connections per host to the file server, so a page with
many videos and audio clips held all six and the next worker's page.goto
waited until it timed out. Every parallel worker now gets its own browser.
Part of #5548
* refactor(engine): shorten the worker browser comment
Miguel Ángel