Files
hyperframes/packages/producer/README.md
T
PaprikaChenandClaude Opus 5.5 1a9fb8acb6 feat: opt-in 4:4:4 chroma for SDR MP4 masters (#5820)
* feat(engine): decouple transparent background from png capture format

PNG capture used to imply a cleared page background. A new
CaptureOptions.transparentBackground lets lossless opaque PNG capture
(4:4:4 MP4 output) keep the page background. Absent, it still follows
format === "png", so alpha output and motion blur are unchanged.

Opaque PNG skips drawElement, whose verification and worker-encode
paths are JPEG-only.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(engine): add 4:4:4 chroma option to getEncoderPreset

getEncoderPreset takes an optional chromaSubsampling ("420" by
default). "444" returns yuv444p for SDR MP4, so libx264 encodes the
High 4:4:4 Predictive profile and odd sizes need no even padding.
WebM, MOV and HDR presets ignore it. The ChromaSubsampling type is
exported for the producer.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(producer): chromaSubsampling render option with lossless png capture

RenderConfig.chromaSubsampling ("420" by default) opts an SDR MP4
render into 4:4:4. "444" encodes libx264 yuv444p and captures every
frame as opaque PNG: Chrome's quality-100 JPEG is already 4:2:0 on
beginFrame and screenshot capture, and drawElement can fall back to
screenshot mid-render, so PNG is the only capture that keeps the
colour. drawElement is turned off for these renders.

The probe and main capture options share resolveCaptureSurface, so
they cannot disagree on format, quality or background.

Formats other than mp4, GPU encoding and HDR output are rejected
before any work starts, auto-detected HDR sources fail before capture,
and distributed requests refuse 444. The default render is unchanged.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(producer,studio-server): accept chromaSubsampling on render routes

The shared parseEncoderOptions validates chromaSubsampling and answers
400 for "444" with a non-mp4 format, gpu, or force-hdr, so a request
that would come back 4:2:0 never starts a render. The Studio render
route, both Studio adapters and the producer HTTP server forward it.
The producer server also refuses "444" with outputDynamicRange "hdr",
which parseEncoderOptions cannot see.

No Studio UI control; the option is route-level like crf and hdrMode.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): add --chroma-subsampling flag

`hyperframes render --chroma-subsampling 444` opts an SDR MP4 render
into full-resolution chroma (yuv444p with lossless PNG capture). The
default stays 420. The plan rejects 444 with a non-mp4 format, --gpu
or --hdr before anything renders, with a usage error that names the
conflicting flag.

The value reaches the local render request and, only when it is 444,
the Docker argv, so default Docker invocations are unchanged.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document 4:4:4 chroma subsampling

Add a "4:4:4 masters" section to the rendering guide: when to use
--chroma-subsampling 444, why it captures PNG, the measured chroma gain
(large for text and UI, none for footage and gradients), its limits,
and a player compatibility table to be filled in from testing.

Document the flag in the CLI reference, the built-in rendering docs and
the hyperframes-cli skill reference (with the regenerated skills
manifest), and the chromaSubsampling option in the producer docs.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: add 4:4:4 player compatibility results

Fill the 4:4:4 compatibility table from testing H.264 High 4:4:4
Predictive output. Chrome 155, Edge 155, VLC 3.0, Safari on iOS 26.6
and Safari 26.6 on macOS play it, and so does AVFoundation on macOS
26.6, the framework QuickTime Player plays through. Firefox 157 on
Windows and both Windows media players do not.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: note the measured render cost of 4:4:4 output

Replace the vague "slower capture" note with measured numbers: a text
and UI composition rendered in about twice the default time, full-frame
footage in about three and a half times, because PNG encoding cost grows
with image detail.

Refs #5555

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): reject chromaSubsampling 444 in distributed render config

Distributed configs have no chroma field and validateDistributedRenderConfig
ignored unknown keys, so a JS caller passing chromaSubsampling: "444" to the
Lambda or Cloud Run SDK got a 4:2:0 file with no error. Reject it, and make
the rendering guide say so precisely.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(producer): check a 444 render reaches the encoder as yuv444p

Every helper had tests, but nothing checked the orchestrator passes
chromaSubsampling through, so dropping it from the encoder preset or capture
surface would turn 444 renders back into 4:2:0 with all tests green. Render a
small fixture with "444" and with the default and check pix_fmt with ffprobe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(producer): move the 444 drawElement gate into a helper

Keeps runCompileStage under the fallow complexity threshold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: keep comment share at or below the merge-base

comment-ratchet failed on 11 files this PR touched. Keep the 4:4:4
rationale once in chromaSubsampling.ts, move the distributed-render
reason into its error message, and cut the rest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(producer): check a 444 render captures full-resolution chroma

The encoder check alone passes when chromaSubsampling is dropped from the
capture surface: frames are captured as JPEG and encoded yuv444p with
4:2:0 colour. Put a red/blue edge at an odd x, inside a 4:2:0 chroma
pair, and require the two pixels to differ in Cb.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(producer): trim the chromaSubsampling.ts header to the package comment share

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: finish switching to capturesAlpha from #6262

The merge that adopted main's transparent / capturesAlpha left these files
uncommitted, so resolveCaptureSurface still returned transparentBackground.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(engine): use transparent / capturesAlpha from #6262

main added CaptureOptions.transparent and capturesAlpha (#6262), which do
what this PR's transparentBackground did. Use them, keep the PNG check in
drawElementWorkerEncodeEligible, and re-pin test-classification.mjs and
the skills manifest against current main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(producer): cut the 4:4:4 comment blocks to one to three lines

The reasoning lives in the PR description with the measurements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: reconcile 4:4:4 chroma with latest main

Keep main's video-layer capture overrides, template entry resolution,
ProRes profile parsing and blur-differential tests alongside the chroma
option, and fold the encoder sub-parsers into one loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: keep main's compile-stage tests and regenerate the skills manifest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 00:20:55 +00:00

12 KiB
Raw Blame History

@hyperframes/producer

Full HTML-to-video rendering pipeline: capture frames with Chrome's BeginFrame API, encode with FFmpeg, mix audio — all in one call.

Install

npm install @hyperframes/producer

Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded), FFmpeg

Usage

Render a video

import { createRenderJob, executeRenderJob } from "@hyperframes/producer";

const job = createRenderJob({
  fps: 30,
  quality: "standard",
});

await executeRenderJob(job, "./my-project", "./output.mp4", (currentJob) => {
  console.log(`${Math.round(currentJob.progress)}%`);
});

Run as an HTTP server

The producer can also run as a render server, accepting render requests over HTTP:

import { startServer } from "@hyperframes/producer";

await startServer({ port: 8080 });
// POST /render or /render/stream with projectDir, html, or previewUrl (public HTTPS only) and render options

Both HTTP routes accept optional pageMotionBlur: "off" to disable authored page motion blur. Omit it to preserve the composition's behavior. Other explicit values are rejected before a render starts. This is separate from engine sub-frame motionBlur sampling.

Configuration

RenderConfig controls the render pipeline:

Option Default Description
entryFile index.html Entry composition inside the project directory. Its authored dimensions set the canvas size.
fps required Frame rate as an integer or exact { num, den } rational.
quality required Encoder preset ("draft", "standard", "high").
pageMotionBlur authored setting Set "off" to disable page motion blur; omit to preserve authored behavior.
format "mp4" Output container — "mp4", "webm", "mov", "gif", "png-sequence", or "hls". See Transparent Video Output and HLS Output below.
hlsSegmentSeconds 4 HLS target segment length in whole seconds. Ignored for every other format.
videoFrameFormat "auto" Source video frame extraction format — "auto", "jpg", or "png". Use "png" for UI recordings, screen captures, and color-sensitive source videos.
chromaSubsampling "420" MP4 chroma, "420" or "444". "444" encodes yuv444p from lossless PNG capture for text/UI masters; SDR MP4 and software encoder only. Many players cannot decode it.

Transparent Video Output

The producer can render HTML compositions to formats that carry a true alpha channel — not chroma key. The same composition that renders an opaque MP4 renders a layerable overlay when you set format.

format Codec / pixel format Alpha Audio Use case
"mp4" (default) H.264 (yuv420p) or H.265 + HDR10 No AAC Streaming, social, default deliverable
"webm" VP9 + yuva420p True alpha Opus Web playback as overlay (<video> over background); supported in Chrome, Edge, Firefox
"mov" ProRes 4444 + yuva444p10le True alpha AAC Editor ingest (Premiere, Final Cut Pro, DaVinci Resolve)
"png-sequence" Numbered RGBA PNGs in a directory Lossless alpha Sidecar audio.m4a After Effects / Nuke / Fusion, or pipelines that post-process frames before encoding
"hls" H.264 (yuv420p) in MPEG-TS No AAC rendition VOD streaming — see HLS Output

Example

import { createRenderJob, executeRenderJob } from "@hyperframes/producer";

const job = createRenderJob({
  fps: 30,
  quality: "standard",
  format: "webm", // "mp4" | "webm" | "mov" | "png-sequence"
});

await executeRenderJob(job, "./my-project", "./output.webm");

What "transparent background" means here

The producer captures Chrome screenshots with the page background forced transparent (html, body, [data-composition-id] { background: transparent !important }) and the CDP default background override set to RGBA 0,0,0,0. The captured PNGs carry a real alpha channel and that channel is preserved end-to-end:

  • VP9 (webm) is encoded with -pix_fmt yuva420p, -auto-alt-ref 0, -cpu-used 4 by default, and alpha_mode=1 metadata. Tune the speed/quality tradeoff with PRODUCER_VP9_CPU_USED (-8 to 8) or local CLI --vp9-cpu-used.
  • ProRes 4444 (mov) uses -pix_fmt yuva444p10le and carries a separate alpha plane. Chrome's RGBA capture, including alpha, is 8 bits per channel; the 10-bit pixel format adds no capture precision.
  • PNG sequences are written without re-encoding (zero-padded frame_NNNNNN.png).

This is not chroma keying. There is no green/blue background to remove and no "key" tolerance to tune — pixels that were transparent in the browser are transparent in the output.

Caveats

  • Linux + alpha forces screenshot capture. Chrome's BeginFrame compositor (the default deterministic capture path on Linux headless-shell) does not preserve alpha; the orchestrator falls back to Page.captureScreenshot, which is slower per frame. macOS and Windows already use screenshot mode by default, so they are unaffected.
  • HDR + alpha is not supported. Setting hdr: true together with an alpha-capable format logs a warning and falls back to SDR. Use format: "mp4" for HDR10 output.
  • png-sequence does not produce a single muxed file. When the composition contains audio elements, an audio.m4a sidecar is written alongside the PNGs in outputPath.
  • Safari + WebM alpha is incomplete. For broad browser playback of an alpha video, ship format: "mov" to your editor and re-encode for the codec your distribution target supports.

Authoring transparent compositions

Don't paint a fullscreen background in your HTML. The default body background is overridden to transparent automatically — any body { background: ... }, #root { background: ... }, or [data-composition-id] { background: ... } rule is force-overridden during alpha rendering. Backgrounds on inner elements (cards, scenes, components) are kept.

HLS Output

format: "hls" packages the same H.264 + AAC encode as mp4 as an HLS VOD stream. outputPath is a directory:

master.m3u8      # references both renditions
video.m3u8       # video media playlist
video_00000.ts   # MPEG-TS segments, one IDR frame at the start of each
audio.m3u8       # written only when the composition has audio
audio_00000.ts
const job = createRenderJob({
  fps: 30,
  quality: "standard",
  format: "hls",
  hlsSegmentSeconds: 4,
});
await executeRenderJob(job, "./my-project", "./renders/stream");

Segments are exactly hlsSegmentSeconds long except the last: the encoder's GOP is locked to round(fps × hlsSegmentSeconds) frames so the packaging pass can stream-copy (-c copy) and still cut on keyframes. There is no second encode.

Constraints:

  • SDR only. hdrMode: "force-hdr" is rejected; HDR10 would need fMP4 segments and HEVC. Use format: "mp4" for HDR.
  • Software encoder only. useGpu: true is rejected — GPU encoders ignore the forced-keyframe lock that fixed-length segments depend on.
  • Not available in distributed rendering, including Lambda and Cloud Run.
  • MPEG-TS carries no mov-family metadata, so the render provenance tags written into MP4 output are absent.

Distributed rendering

For renders too large for a single machine, the producer ships a public set of distributed-render primitives. They are pure functions over local file paths — networking and orchestration live in adapter packages (Temporal, AWS Lambda + Step Functions, Cloud Run Jobs, K8s Jobs).

Plan v2 is recommended for new integrations. It publishes an immutable manifest plus content-addressed artifacts and materializes only each worker's declared dependencies:

import { planV2, renderChunkV2, assembleV2 } from "@hyperframes/producer/distributed";

// Controller-side: produce a v2 manifest + local content-addressed store.
const planResult = await planV2(
  projectDir,
  { fps: 30, width: 1920, height: 1080, format: "mp4" },
  "/tmp/plan-v2",
);

const chunk = await renderChunkV2("/tmp/plan-v2", 0, "/tmp/chunks/0.mp4");

// Controller-side: stitch chunks into the final deliverable.
await assembleV2("/tmp/plan-v2", ["/tmp/chunks/0.mp4", "/tmp/chunks/1.mp4"], "/tmp/output.mp4");

Cloud adapters should use planV2WithPublisher() so artifacts publish directly to object storage. The legacy plan() / renderChunk() / assemble() v1 layout remains supported, and cloud SDKs still interpret an omitted protocol as v1 for backwards compatibility.

The activity functions plus their result types are also re-exported from @hyperframes/producer so callers that pin the main package don't need a separate subpath import. Supported formats: mp4 SDR, mov ProRes 4444, and png-sequence. webm and HDR mp4 trip a typed FormatNotSupportedInDistributedError — use the in-process renderer (executeRenderJob) for those.

How it works

  1. Serve — spins up a local file server for the HTML composition
  2. Capture — opens the page in headless Chrome, seeks frame-by-frame via HeadlessExperimental.beginFrame (or Page.captureScreenshot for transparent / non-Linux renders), captures screenshots
  3. Encode — pipes frames through FFmpeg (with GPU encoder detection and chunked concat). Skipped for format: "png-sequence".
  4. Mix — extracts <audio> elements and mixes them into the final video. For png-sequence, audio is written as an audio.m4a sidecar.
  5. Finalize — applies faststart for streaming-friendly MP4 (no-op for WebM, MOV, and png-sequence). For hls, stream-copies the encode into playlists and MPEG-TS segments instead.

Where each step lives

executeRenderJob in src/services/renderOrchestrator.ts runs the stages in src/services/render/stages/, in this order:

  • compileStage.ts, probeStage.ts, extractVideosStage.ts, audioStage.ts
  • src/services/render/captureCost.ts sizes the worker count (runCaptureCalibration, then resolveRenderWorkerCount)
  • createCapturePlan in src/services/render/capturePlan.ts picks the capture kind: streaming, disk, segmented or HDR layered
  • capture: captureStreamingStage.ts, captureSegmentedStage.ts and captureHdrStage.ts encode as they capture; disk capture (captureStage.ts) encodes afterwards in encodeStage.ts
  • assembleStage.ts, except for PNG sequences and GIF, which are written directly

The job's producerConfig is used when present; otherwise resolveConfig in packages/engine/src/config.ts builds it. Capture, encoding and browser primitives are engine services (see packages/engine/README.md).

Documentation

Full documentation: hyperframes.heygen.com/packages/producer