mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
Youssef Toufik
fe017b48c7
Extends RenderConfig.format with "png-sequence" and patches two correctness
gaps so the existing "webm" / "mov" values actually preserve the alpha
channel end-to-end.
Engine fixes:
- screenshotService.pageScreenshotCapture: drop optimizeForSpeed for PNG
captures. The fast path uses an alpha-unaware codec that crushes real
alpha values; kept for opaque jpeg captures where it is harmless.
- frameCapture: replace the inline setDefaultBackgroundColorOverride
block (which fired pre-navigation and was reset by page.goto) with a
proper initTransparentBackground() call inside initializeSession,
after the window.__hf readiness poll. This also injects the
html/body/[data-composition-id]{background:transparent !important}
stylesheet so compositions with custom body / #root backgrounds do not
defeat the override. Wired into both screenshot-mode and beginframe-mode
branches.
Producer:
- RenderConfig.format extended to "mp4" | "webm" | "mov" | "png-sequence"
with full JSDoc.
- Streaming encode is bypassed for png-sequence (frames go straight to
disk). FORMAT_EXT extended.
- New Stage-5 png-sequence branch: mkdir outputPath, copy captured PNGs as
frame_NNNNNN.png, copy audio.aac sidecar when audio is present.
- Stage-6 mux/faststart and the debug copy are wrapped in !isPngSequence.
- README.md: new "Transparent Video Output" section.
Tests:
- New fixture tests/transparency-regression/ tagged "transparency".
- New tsx script src/transparency-test.ts asserts pixel-level alpha for
webm + png-sequence outputs. Wired as "test:transparency".
- Default "test" / "test:update" scripts pass --exclude-tags transparency
so the golden-MP4 harness ignores the new fixture.
Verified locally on macOS arm64: typecheck clean across engine + producer,
producer renderOrchestrator vitest 10/10, transparency-test passes for
both webm and png-sequence with end-to-end pixel assertions.
126 lines
7.3 KiB
Markdown
126 lines
7.3 KiB
Markdown
# @hyperframes/producer
|
|
|
|
Full HTML-to-video rendering pipeline: capture frames with Chrome's BeginFrame API, encode with FFmpeg, mix audio — all in one call.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
npm install @hyperframes/producer
|
|
```
|
|
|
|
**Requirements:** Node.js >= 22, Chrome/Chromium (auto-downloaded), FFmpeg
|
|
|
|
## Usage
|
|
|
|
### Render a video
|
|
|
|
```typescript
|
|
import { createRenderJob, executeRenderJob } from "@hyperframes/producer";
|
|
|
|
const job = createRenderJob({
|
|
inputPath: "./my-composition.html",
|
|
outputPath: "./output.mp4",
|
|
width: 1920,
|
|
height: 1080,
|
|
fps: 30,
|
|
});
|
|
|
|
const result = await executeRenderJob(job, (progress) => {
|
|
console.log(`${Math.round(progress.percent * 100)}%`);
|
|
});
|
|
|
|
console.log(result.outputPath); // ./output.mp4
|
|
```
|
|
|
|
### Run as an HTTP server
|
|
|
|
The producer can also run as a render server, accepting render requests over HTTP:
|
|
|
|
```typescript
|
|
import { startServer } from "@hyperframes/producer";
|
|
|
|
await startServer({ port: 8080 });
|
|
// POST /render with a RenderConfig body
|
|
```
|
|
|
|
### Configuration
|
|
|
|
`RenderConfig` controls the render pipeline:
|
|
|
|
| Option | Default | Description |
|
|
| ------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `inputPath` | — | Path to the HTML composition |
|
|
| `outputPath` | — | Output video file path (or directory, for `format: "png-sequence"`) |
|
|
| `width` | 1920 | Frame width in pixels |
|
|
| `height` | 1080 | Frame height in pixels |
|
|
| `fps` | 30 | Frames per second (24, 30, or 60) |
|
|
| `quality` | `"standard"` | Encoder preset (`"draft"`, `"standard"`, `"high"`) |
|
|
| `format` | `"mp4"` | Output container — `"mp4"`, `"webm"`, `"mov"`, or `"png-sequence"`. See [Transparent Video Output](#transparent-video-output) below. |
|
|
|
|
## Transparent Video Output
|
|
|
|
The producer can render HTML compositions to formats that carry a **true alpha channel** — not chroma key. The same composition that renders an opaque MP4 renders a layerable overlay when you set `format`.
|
|
|
|
| `format` | Codec / pixel format | Alpha | Audio | Use case |
|
|
| ----------------- | --------------------------------- | ----------------------- | ------------------- | --------------------------------------------------------------------------------------- |
|
|
| `"mp4"` (default) | H.264 (yuv420p) or H.265 + HDR10 | No | AAC | Streaming, social, default deliverable |
|
|
| `"webm"` | VP9 + yuva420p | **True alpha** | Opus | Web playback as overlay (`<video>` over background); supported in Chrome, Edge, Firefox |
|
|
| `"mov"` | ProRes 4444 + yuva444p10le | **True alpha + 10-bit** | AAC | Editor ingest (Premiere, Final Cut Pro, DaVinci Resolve) |
|
|
| `"png-sequence"` | Numbered RGBA PNGs in a directory | **Lossless alpha** | Sidecar `audio.aac` | After Effects / Nuke / Fusion, or pipelines that post-process frames before encoding |
|
|
|
|
### Example
|
|
|
|
```typescript
|
|
import { createRenderJob, executeRenderJob } from "@hyperframes/producer";
|
|
|
|
const job = createRenderJob({
|
|
inputPath: "./my-composition.html",
|
|
outputPath: "./output.webm", // or a directory for "png-sequence"
|
|
width: 1080,
|
|
height: 1920,
|
|
fps: 30,
|
|
format: "webm", // "mp4" | "webm" | "mov" | "png-sequence"
|
|
});
|
|
|
|
await executeRenderJob(job);
|
|
```
|
|
|
|
### What "transparent background" means here
|
|
|
|
The producer captures Chrome screenshots with the page background forced transparent (`html, body, [data-composition-id] { background: transparent !important }`) and the CDP default background override set to RGBA 0,0,0,0. The captured PNGs carry a real alpha channel and that channel is preserved end-to-end:
|
|
|
|
- VP9 (`webm`) is encoded with `-pix_fmt yuva420p`, `-auto-alt-ref 0`, and `alpha_mode=1` metadata.
|
|
- ProRes 4444 (`mov`) is encoded with `-pix_fmt yuva444p10le`.
|
|
- PNG sequences are written without re-encoding (zero-padded `frame_NNNNNN.png`).
|
|
|
|
This is not chroma keying. There is no green/blue background to remove and no "key" tolerance to tune — pixels that were transparent in the browser are transparent in the output.
|
|
|
|
### Caveats
|
|
|
|
- **Linux + alpha forces screenshot capture.** Chrome's BeginFrame compositor (the default deterministic capture path on Linux headless-shell) does not preserve alpha; the orchestrator falls back to `Page.captureScreenshot`, which is slower per frame. macOS and Windows already use screenshot mode by default, so they are unaffected.
|
|
- **HDR + alpha is not supported.** Setting `hdr: true` together with an alpha-capable format logs a warning and falls back to SDR. Use `format: "mp4"` for HDR10 output.
|
|
- **`png-sequence` does not produce a single muxed file.** When the composition contains audio elements, an `audio.aac` sidecar is written alongside the PNGs in `outputPath`.
|
|
- **Safari + WebM alpha is incomplete.** For broad browser playback of an alpha video, ship `format: "mov"` to your editor and re-encode for the codec your distribution target supports.
|
|
|
|
### Authoring transparent compositions
|
|
|
|
Don't paint a fullscreen background in your HTML. The default body background is overridden to transparent automatically — any `body { background: ... }`, `#root { background: ... }`, or `[data-composition-id] { background: ... }` rule is force-overridden during alpha rendering. Backgrounds on inner elements (cards, scenes, components) are kept.
|
|
|
|
## How it works
|
|
|
|
1. **Serve** — spins up a local file server for the HTML composition
|
|
2. **Capture** — opens the page in headless Chrome, seeks frame-by-frame via `HeadlessExperimental.beginFrame` (or `Page.captureScreenshot` for transparent / non-Linux renders), captures screenshots
|
|
3. **Encode** — pipes frames through FFmpeg (with GPU encoder detection and chunked concat). Skipped for `format: "png-sequence"`.
|
|
4. **Mix** — extracts `<audio>` elements and mixes them into the final video. For `png-sequence`, audio is written as an `audio.aac` sidecar.
|
|
5. **Finalize** — applies faststart for streaming-friendly MP4 (no-op for WebM, MOV, and `png-sequence`)
|
|
|
|
## Documentation
|
|
|
|
Full documentation: [hyperframes.heygen.com/packages/producer](https://hyperframes.heygen.com/packages/producer)
|
|
|
|
## Related packages
|
|
|
|
- [`@hyperframes/core`](../core) — types, parsers, frame adapters
|
|
- [`@hyperframes/engine`](../engine) — lower-level capture and encode primitives
|
|
- [`hyperframes`](../cli) — CLI
|