Files
hyperframes/packages/producer
James RussoandClaude Opus 4.7 21f5066832 feat(producer): enable webm in distributed mode via concat-copy (#951)
* feat(producer): enable webm in distributed mode via concat-copy

PR 8.2 of the WebM distributed-rendering plan (v1.5 backlog #1; see
DISTRIBUTED-RENDERING-PLAN.md §7.2). Wires libvpx-vp9 webm through the
distributed pipeline now that PR 8.1 proved concat-copy works.

Architectural decision: Path A (concat-copy) — based on PR 8.1's smoke
test result (9/9 tests pass for both yuv420p and yuva420p VP9 streams).
The simpler architecture wins; no re-encode in assemble, no encode-
parallelism loss.

Changes:

- plan.ts:
  - DistributedRenderConfig.format and PlanResult.format now include
    "webm" — type-level acceptance matches the runtime gate.
  - rejectUnsupportedDistributedFormat() no longer trips on webm. HDR
    mp4 remains the only refused configuration.
  - resolveEncoderTriple() returns libvpx-vp9-software + yuva420p +
    preset="good" for format="webm". yuva420p preserves alpha — the
    format's main reason for existing for web delivery.
  - codec= remains rejected for non-mp4 formats (mov is always ProRes
    4444; webm is always libvpx-vp9). The error message lists all four
    distributed-supported formats.
  - FormatNotSupportedInDistributedError docstring updated to reflect
    the new reality (only HDR is unsupported).

- freezePlan.ts: LockedRenderConfig.encoder gains "libvpx-vp9-software".
  Mirrors libx265-software / prores-software / png-sequence in shape;
  the chunk worker reads this discriminant to decide encode args.

- renderChunk.ts: drops the now-incorrect cast that excluded webm from
  buildSyntheticRenderJob's format input; tightens the preset-format
  cast to include webm.

- assemble.ts: docstring + comment updates. The mp4/mov concat-copy
  path is format-agnostic — webm uses the exact same code (applyFaststart
  is a no-op for webm via the existing chunkEncoder.ts gate;
  muxVideoWithAudio already routes webm to libopus audio).

- planFormatBanlist.test.ts: webm-rejection tests removed; replaced with
  "accepts webm" tests + a HDR+webm combo test that verifies HDR is the
  trip regardless of format.

- plan.test.ts: new describe block pins the webm wiring contract:
  format="webm" produces an encoder=libvpx-vp9-software /
  pixelFormat=yuva420p planDir with closedGop=true and gopSize=chunkSize.

- webm-concat-copy.test.ts (smoke): extended with a yuva420p variant
  that proves the alpha pixel format the distributed pipeline actually
  emits also round-trips through concat-copy. 9/9 tests pass locally.

§8 format support matrix in DISTRIBUTED-RENDERING-PLAN.md is intentionally
left unchanged at this PR — it flips to ✓ in PR 8.4 once the end-to-end
fixture (PR 8.3) is green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(producer): include webm in plan-time needsAlpha + strengthen alpha smoke

PR review feedback from Miguel and Vai on #951 caught a real bug:
`plan.ts`'s `needsAlpha` disjunction excluded `"webm"`, so the plan
stage froze `forceScreenshot: false` into the `LockedRenderConfig`
even though distributed webm uses `yuva420p`. Every chunk worker
captured opaque RGB via BeginFrame (which doesn't preserve alpha on
Linux headless-shell), and libvpx-vp9 encoded uniformly-opaque alpha
that the encoder then dropped — producing un-keyable webm.

Two changes:

1. **plan.ts**: include `"webm"` in `needsAlpha`. Matches the
   in-process renderer's logic at `renderOrchestrator.ts:1469`
   (`const needsAlpha = isWebm || isMov || isPngSequence`); the two
   sites must stay in sync since the distributed pipeline's PSNR
   regression compares against the in-process baseline.

2. **Smoke test (yuva420p describe)**: source frames now use a real
   alpha gradient (`geq=a='X*255/W'` on top of `testsrc2`) instead of
   `testsrc2 + format=rgba` which was uniformly opaque. The decode-
   pix_fmt assertion is dropped (ffprobe reports `yuv420p` for
   VP9-with-alpha because the alpha lives in a Matroska
   `BlockAdditional` sidecar) and replaced with two stronger checks:
   - `TAG:ALPHA_MODE=1` is present on the stream — proves the
     encoder was actually configured for alpha
   - alpha plane variance after `-c:v libvpx-vp9 -i ... -pix_fmt rgba
     -vf extractplanes=a,signalstats` — proves the alpha sub-stream
     round-trips through concat-copy with spatially-varying content,
     not uniform/dropped alpha
   - decode-test gate is now exit-code-only (was `exitCode || stderr`
     which would flake on chatty ffmpeg `-v error` builds emitting
     non-fatal DTS/container notes)

These checks would have caught the `needsAlpha` bug before review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(aws-lambda): widen narrow format types to include webm

CI on PR #951 was failing at typecheck/build because the producer's
`DistributedRenderConfig.format` widened to include webm in this PR
but the aws-lambda package's narrow `"mp4" | "mov" | "png-sequence"`
type literals in `events.ts`, `handler.ts`, and `validateConfig.ts`
hadn't kept up. `renderToLambda.ts:87` passed `config.format` (now
including webm) into a parameter typed against the narrow union,
producing TS2345.

This widening originally landed in PR #952 (test fixture PR) but
needs to be atomic with the producer's widening here to keep each
PR independently typecheck-clean.

Also refactor `formatExtension` from a switch dispatch to a
`Record<DistributedFormat, string>` lookup. Adding the webm case
tipped the switch's CRAP to the 30.0 fallow threshold; the lookup
table drops cyclomatic from 5 to 1 with the same compile-time
exhaustiveness guarantee (TS errors on missing entries when
`DistributedFormat` adds a new format). The runtime
`_exhaustive: never` throw was only protecting against a string
slipping past TS; `validateConfig.ts`'s `ALLOWED_FORMATS` already
gates untrusted input at the SDK boundary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 02:46:21 -04:00
..
2026-05-19 02:33:31 +00:00

@hyperframes/producer

Full HTML-to-video rendering pipeline: capture frames with Chrome's BeginFrame API, encode with FFmpeg, mix audio — all in one call.

Install

npm install @hyperframes/producer

Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded), FFmpeg

Usage

Render a video

import { createRenderJob, executeRenderJob } from "@hyperframes/producer";

const job = createRenderJob({
  inputPath: "./my-composition.html",
  outputPath: "./output.mp4",
  width: 1920,
  height: 1080,
  fps: 30,
});

const result = await executeRenderJob(job, (progress) => {
  console.log(`${Math.round(progress.percent * 100)}%`);
});

console.log(result.outputPath); // ./output.mp4

Run as an HTTP server

The producer can also run as a render server, accepting render requests over HTTP:

import { startServer } from "@hyperframes/producer";

await startServer({ port: 8080 });
// POST /render with a RenderConfig body

Configuration

RenderConfig controls the render pipeline:

Option Default Description
inputPath Path to the HTML composition
outputPath Output video file path (or directory, for format: "png-sequence")
width 1920 Frame width in pixels
height 1080 Frame height in pixels
fps 30 Frames per second (24, 30, or 60)
quality "standard" Encoder preset ("draft", "standard", "high")
format "mp4" Output container — "mp4", "webm", "mov", or "png-sequence". See Transparent Video Output below.

Transparent Video Output

The producer can render HTML compositions to formats that carry a true alpha channel — not chroma key. The same composition that renders an opaque MP4 renders a layerable overlay when you set format.

format Codec / pixel format Alpha Audio Use case
"mp4" (default) H.264 (yuv420p) or H.265 + HDR10 No AAC Streaming, social, default deliverable
"webm" VP9 + yuva420p True alpha Opus Web playback as overlay (<video> over background); supported in Chrome, Edge, Firefox
"mov" ProRes 4444 + yuva444p10le True alpha + 10-bit AAC Editor ingest (Premiere, Final Cut Pro, DaVinci Resolve)
"png-sequence" Numbered RGBA PNGs in a directory Lossless alpha Sidecar audio.aac After Effects / Nuke / Fusion, or pipelines that post-process frames before encoding

Example

import { createRenderJob, executeRenderJob } from "@hyperframes/producer";

const job = createRenderJob({
  inputPath: "./my-composition.html",
  outputPath: "./output.webm", // or a directory for "png-sequence"
  width: 1080,
  height: 1920,
  fps: 30,
  format: "webm", // "mp4" | "webm" | "mov" | "png-sequence"
});

await executeRenderJob(job);

What "transparent background" means here

The producer captures Chrome screenshots with the page background forced transparent (html, body, [data-composition-id] { background: transparent !important }) and the CDP default background override set to RGBA 0,0,0,0. The captured PNGs carry a real alpha channel and that channel is preserved end-to-end:

  • VP9 (webm) is encoded with -pix_fmt yuva420p, -auto-alt-ref 0, and alpha_mode=1 metadata.
  • ProRes 4444 (mov) is encoded with -pix_fmt yuva444p10le.
  • PNG sequences are written without re-encoding (zero-padded frame_NNNNNN.png).

This is not chroma keying. There is no green/blue background to remove and no "key" tolerance to tune — pixels that were transparent in the browser are transparent in the output.

Caveats

  • Linux + alpha forces screenshot capture. Chrome's BeginFrame compositor (the default deterministic capture path on Linux headless-shell) does not preserve alpha; the orchestrator falls back to Page.captureScreenshot, which is slower per frame. macOS and Windows already use screenshot mode by default, so they are unaffected.
  • HDR + alpha is not supported. Setting hdr: true together with an alpha-capable format logs a warning and falls back to SDR. Use format: "mp4" for HDR10 output.
  • png-sequence does not produce a single muxed file. When the composition contains audio elements, an audio.aac sidecar is written alongside the PNGs in outputPath.
  • Safari + WebM alpha is incomplete. For broad browser playback of an alpha video, ship format: "mov" to your editor and re-encode for the codec your distribution target supports.

Authoring transparent compositions

Don't paint a fullscreen background in your HTML. The default body background is overridden to transparent automatically — any body { background: ... }, #root { background: ... }, or [data-composition-id] { background: ... } rule is force-overridden during alpha rendering. Backgrounds on inner elements (cards, scenes, components) are kept.

Distributed rendering

For renders too large for a single machine, the producer ships a public set of distributed-render primitives. They are pure functions over local file paths — networking and orchestration live in adapter packages (Temporal, AWS Lambda + Step Functions, Cloud Run Jobs, K8s Jobs).

import { plan, renderChunk, assemble } from "@hyperframes/producer/distributed";

// Controller-side: produce a self-contained planDir + content-addressed planHash.
const planResult = await plan(
  projectDir,
  { fps: 30, width: 1920, height: 1080, format: "mp4" },
  "/tmp/plan",
);

// Worker-side: render one chunk. Byte-identical retries on the same
// `(planDir, chunkIndex)` — Temporal / Step Functions retry policies are safe
// to point at this.
const chunk = await renderChunk("/tmp/plan", 0, "/tmp/chunks/0.mp4");

// Controller-side: stitch chunks into the final deliverable.
await assemble(
  "/tmp/plan",
  ["/tmp/chunks/0.mp4", "/tmp/chunks/1.mp4"],
  "/tmp/plan/audio.aac",
  "/tmp/output.mp4",
);

The three activity functions plus their result types are also re-exported from @hyperframes/producer so callers that pin the main package don't need a separate subpath import. Supported formats: mp4 SDR, mov ProRes 4444, and png-sequence. webm and HDR mp4 trip a typed FormatNotSupportedInDistributedError — use the in-process renderer (executeRenderJob) for those.

How it works

  1. Serve — spins up a local file server for the HTML composition
  2. Capture — opens the page in headless Chrome, seeks frame-by-frame via HeadlessExperimental.beginFrame (or Page.captureScreenshot for transparent / non-Linux renders), captures screenshots
  3. Encode — pipes frames through FFmpeg (with GPU encoder detection and chunked concat). Skipped for format: "png-sequence".
  4. Mix — extracts <audio> elements and mixes them into the final video. For png-sequence, audio is written as an audio.aac sidecar.
  5. Finalize — applies faststart for streaming-friendly MP4 (no-op for WebM, MOV, and png-sequence)

Documentation

Full documentation: hyperframes.heygen.com/packages/producer