mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
## Summary This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI. The branch now does four things: - forwards `--hdr` through the Docker render path in the CLI - adds and expands HDR documentation across the docs site - adds first-class HDR still-image support to the engine/producer pipeline - adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags ## What changed ### CLI and docs - `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI - added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs - documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes ### Engine and producer HDR image support - added `ImageElement` support to the engine composition model and parsing path - threaded image elements through producer compilation and orchestration - probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source - included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order - integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays - forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic - skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows ### HDR metadata robustness - added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs - this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ ### Regression coverage and fixture cleanup - added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end - added `hdr-pq`, a focused HDR PQ regression fixture for the video path - updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only` - removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI - added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests ## Why The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking. The practical issue this closes is: - local host runs could pass while CI failed `hdr-image-only` - the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering - root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment - parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments ## Test plan ### Local targeted checks ```bash bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts ``` ### Producer regression runs on host ```bash bun run --cwd packages/core build:hyperframes-runtime:modular bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only ``` Observed result: - `fast` shard: 7 passed, 0 failed - `hdr` shard: 2 passed, 0 failed ### CI-equivalent Docker verification ```bash docker build -f Dockerfile.test -t hyperframes-producer:test . docker run --rm \ --security-opt seccomp=unconfined \ --shm-size=4g \ -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \ hyperframes-producer:test \ --sequential hdr-pq hdr-image-only ``` Observed result: - `hdr-image-only`: passed - `hdr-pq`: passed - shard summary: 2 passed, 0 failed ### Specific regression fixed Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with: - missing `"[Render] HDR source detected — output: PQ ..."` log line - full-frame visual mismatch across all 100 checkpoints - PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
347 lines
12 KiB
Plaintext
347 lines
12 KiB
Plaintext
---
|
||
title: Rendering
|
||
description: "Render compositions to MP4, MOV, or WebM locally or in Docker."
|
||
---
|
||
|
||
Render your Hyperframes [compositions](/concepts/compositions) to MP4, MOV, or WebM with the [CLI](/packages/cli). The rendering pipeline is frame-by-frame and seek-driven — see [Deterministic Rendering](/concepts/determinism) for how this works under the hood.
|
||
|
||
## Getting Started
|
||
|
||
<Steps>
|
||
<Step title="Verify your environment">
|
||
Run the diagnostics command to check for required dependencies:
|
||
|
||
```bash Terminal
|
||
npx hyperframes doctor
|
||
```
|
||
|
||
Expected output:
|
||
|
||
```
|
||
✓ Node.js v22.x
|
||
✓ FFmpeg 7.x
|
||
✓ FFprobe 7.x
|
||
✓ Chrome (bundled)
|
||
✓ Docker available
|
||
```
|
||
</Step>
|
||
<Step title="Preview your composition">
|
||
Before rendering, preview your composition in the browser to verify it looks correct:
|
||
|
||
```bash Terminal
|
||
npx hyperframes preview
|
||
```
|
||
</Step>
|
||
<Step title="Render to MP4">
|
||
Run the render command from your project directory:
|
||
|
||
```bash Terminal
|
||
npx hyperframes render --output output.mp4
|
||
```
|
||
|
||
Expected output:
|
||
|
||
```
|
||
⠋ Rendering composition "root" (30fps, standard quality)
|
||
✓ Captured 240 frames in 8.2s
|
||
✓ Encoded to output.mp4 (8.0s, 1920x1080, 4.2MB)
|
||
```
|
||
</Step>
|
||
</Steps>
|
||
|
||
## Rendering Modes
|
||
|
||
<Tabs>
|
||
<Tab title="Local Mode">
|
||
### Local Mode (default)
|
||
|
||
Uses Puppeteer (bundled Chromium) and your system's FFmpeg. Fast for iteration during development.
|
||
|
||
**Requires:** FFmpeg installed on your system. See [Troubleshooting](/guides/troubleshooting) if FFmpeg is not found.
|
||
|
||
```bash Terminal
|
||
npx hyperframes render --output output.mp4
|
||
```
|
||
|
||
**Pros:**
|
||
- Fast startup, no container overhead
|
||
- Uses your system GPU for hardware-accelerated encoding (with `--gpu`)
|
||
- Best for iterative development
|
||
|
||
**Cons:**
|
||
- Output may vary across platforms due to font and Chrome version differences
|
||
- Not suitable for CI/CD pipelines that require reproducibility
|
||
</Tab>
|
||
<Tab title="Docker Mode">
|
||
### Docker Mode
|
||
|
||
[Deterministic](/concepts/determinism) output with an exact Chrome version and font set. Use this for production renders and CI pipelines.
|
||
|
||
**Requires:** Docker installed and running.
|
||
|
||
```bash Terminal
|
||
npx hyperframes render --docker --output output.mp4
|
||
```
|
||
|
||
**Pros:**
|
||
- Identical output on every platform — same Chrome, same fonts, same FFmpeg
|
||
- The same pipeline used in production
|
||
- Ideal for CI/CD and automated workflows
|
||
|
||
**Cons:**
|
||
- Slower startup due to container initialization
|
||
- No GPU acceleration inside the container
|
||
|
||
<Note>
|
||
Docker mode uses `chrome-headless-shell` with [BeginFrame](/concepts/determinism#how-it-works) control for frame-perfect, deterministic capture.
|
||
</Note>
|
||
</Tab>
|
||
</Tabs>
|
||
|
||
## When to Use Each Mode
|
||
|
||
| Scenario | Recommended Mode |
|
||
|----------|-----------------|
|
||
| Local development and iteration | Local |
|
||
| CI/CD pipeline | Docker |
|
||
| Sharing renders with a team | Docker |
|
||
| Quick preview export | Local |
|
||
| AI agent-driven rendering | Docker |
|
||
| Benchmarking performance | Local |
|
||
|
||
## Options
|
||
|
||
| Flag | Values | Default | Description |
|
||
|------|--------|---------|-------------|
|
||
| `--output` | path | `renders/<name>.mp4` | Output file path |
|
||
| `--format` | mp4, mov, webm | mp4 | Output format (see [Transparent Video](#transparent-video) below) |
|
||
| `--fps` | 24, 30, 60 | 30 | Frames per second |
|
||
| `--quality` | draft, standard, high | standard | Encoding quality preset |
|
||
| `--crf` | 0–51 | — | Override CRF (lower = higher quality). Cannot combine with `--video-bitrate` |
|
||
| `--video-bitrate` | e.g. `10M`, `5000k` | — | Target bitrate encoding. Cannot combine with `--crf` |
|
||
| `--workers` | 1-8 or `auto` | auto | Parallel render workers (see [Workers](#workers) below) |
|
||
| `--max-concurrent-renders` | 1-10 | 2 | Max simultaneous renders via the producer server (see [Concurrent Renders](#concurrent-renders) below) |
|
||
| `--gpu` | — | off | GPU encoding (NVENC, VideoToolbox, VAAPI) |
|
||
| `--hdr` | — | off | Detect HDR sources and output HDR10 (MP4 only). See [HDR Rendering](/guides/hdr) |
|
||
| `--docker` | — | off | Use Docker for [deterministic rendering](/concepts/determinism) |
|
||
| `--quiet` | — | off | Suppress verbose output |
|
||
|
||
## Quality and Encoding
|
||
|
||
The `--quality` flag selects a preset that controls the H.264 CRF (Constant Rate Factor) and encoder speed:
|
||
|
||
| Preset | CRF | x264 Preset | Best For |
|
||
|--------|-----|-------------|----------|
|
||
| `draft` | 28 | ultrafast | Quick previews, iteration |
|
||
| `standard` | 18 | medium | General use — visually lossless at 1080p |
|
||
| `high` | 15 | slow | Final delivery, near-lossless quality |
|
||
|
||
For finer control, use `--crf` or `--video-bitrate` to override the preset:
|
||
|
||
```bash
|
||
# Near-lossless quality (CRF 15 = very high quality, large file)
|
||
npx hyperframes render --crf 15 --output pristine.mp4
|
||
|
||
# Target a specific bitrate (useful for size-constrained delivery)
|
||
npx hyperframes render --video-bitrate 10M --output controlled.mp4
|
||
```
|
||
|
||
**Tip**: The default `standard` preset (CRF 18) is visually lossless at 1080p — most people cannot distinguish it from the source. Use `--quality draft` for faster iteration, or `--quality high` / `--crf 10` when file size is no concern.
|
||
|
||
## Workers
|
||
|
||
Each render worker launches a **separate Chrome browser process** to capture frames in parallel. More workers can speed up rendering, but each one consumes ~256 MB of RAM and significant CPU.
|
||
|
||
### Default behavior
|
||
|
||
By default, Hyperframes uses **half of your CPU cores, capped at 4**:
|
||
|
||
| Machine | CPU cores | Default workers |
|
||
|---------|-----------|----------------|
|
||
| MacBook Air (M1) | 8 | 4 |
|
||
| MacBook Pro (M3) | 12 | 4 (capped) |
|
||
| 4-core laptop | 4 | 2 |
|
||
| 2-core VM | 2 | 1 |
|
||
|
||
This is intentionally conservative. Each worker spawns its own Chrome process, so the per-worker overhead is significant. Fewer workers avoids resource contention with FFmpeg encoding and your other applications.
|
||
|
||
### Choosing a worker count
|
||
|
||
```bash Terminal
|
||
# Explicit worker count
|
||
npx hyperframes render --workers 1 --output output.mp4
|
||
|
||
# Let Hyperframes pick based on your CPU
|
||
npx hyperframes render --workers auto --output output.mp4
|
||
|
||
# Maximum parallelism (use with caution on laptops)
|
||
npx hyperframes render --workers 8 --output output.mp4
|
||
```
|
||
|
||
<Tip>
|
||
Start with the default. If renders feel slow and your system has headroom (check Activity Monitor / `htop`), try increasing `--workers`. If you see high memory pressure or fan noise, reduce it.
|
||
</Tip>
|
||
|
||
### When to use 1 worker
|
||
|
||
- Short compositions (under 2 seconds / 60 frames) — parallelism overhead exceeds the benefit
|
||
- Low-memory machines (4 GB or less)
|
||
- Running renders alongside other heavy processes (video editing, large builds)
|
||
|
||
### When to increase workers
|
||
|
||
- Long compositions (30+ seconds) on a machine with 8+ cores and 16+ GB RAM
|
||
- Dedicated render machines or CI runners
|
||
- Docker mode on a well-provisioned host
|
||
|
||
## Concurrent Renders
|
||
|
||
When multiple render requests hit the producer server simultaneously (common with AI agents), each render spawns its own set of Chrome worker processes. Too many concurrent renders can exhaust CPU and cause failures.
|
||
|
||
The producer server uses a **request-level semaphore** to queue renders. Only `maxConcurrentRenders` renders execute at a time — additional requests wait in a FIFO queue until a slot opens.
|
||
|
||
### Configuration
|
||
|
||
```bash Terminal
|
||
# CLI flag
|
||
npx hyperframes render --max-concurrent-renders 2 --output output.mp4
|
||
|
||
# Environment variable (for the producer server)
|
||
PRODUCER_MAX_CONCURRENT_RENDERS=2
|
||
```
|
||
|
||
The default is **2** concurrent renders, which works well on 8-core machines where each render uses 2-3 workers.
|
||
|
||
### Queue status
|
||
|
||
The producer server exposes a `GET /render/queue` endpoint that returns the current state:
|
||
|
||
```json
|
||
{
|
||
"maxConcurrentRenders": 2,
|
||
"activeRenders": 1,
|
||
"queuedRenders": 3
|
||
}
|
||
```
|
||
|
||
AI agents can poll this endpoint to decide whether to submit a render or wait.
|
||
|
||
### SSE queue events
|
||
|
||
When using the streaming endpoint (`POST /render/stream`), queued requests receive a `queued` event before rendering begins:
|
||
|
||
```json
|
||
{"type": "queued", "requestId": "...", "position": 2}
|
||
```
|
||
|
||
This lets agents report "waiting in queue" to users rather than appearing stuck.
|
||
|
||
### Choosing a concurrency limit
|
||
|
||
| Machine | CPU cores | Recommended limit |
|
||
|---------|-----------|------------------|
|
||
| 4-core VM | 4 | 1 |
|
||
| 8-core workstation | 8 | 2 |
|
||
| 16-core server | 16 | 3-4 |
|
||
| 32-core render box | 32 | 5-6 |
|
||
|
||
<Tip>
|
||
When in doubt, use 1. Renders will queue up and execute sequentially, but each one gets full CPU and finishes as fast as possible. This is better than 3 renders fighting for CPU and all finishing slowly — or failing.
|
||
</Tip>
|
||
|
||
## Transparent Video
|
||
|
||
Hyperframes supports rendering with a transparent background — useful for overlays, lower thirds, subscribe cards, and any element you want to composite over other footage in a video editor.
|
||
|
||
### Recommended format: MOV (ProRes 4444)
|
||
|
||
```bash Terminal
|
||
npx hyperframes render --format mov --output overlay.mov
|
||
```
|
||
|
||
**MOV with ProRes 4444** is the industry standard for transparent video. It works in all major video editors:
|
||
|
||
- CapCut
|
||
- Final Cut Pro
|
||
- Adobe Premiere Pro
|
||
- DaVinci Resolve
|
||
- After Effects
|
||
|
||
<Warning>
|
||
ProRes MOV files are large (typically 5-40 MB for short clips) because ProRes is a high-quality intermediate codec optimized for editing, not delivery. This is expected — the same tradeoff Remotion and professional pipelines make.
|
||
</Warning>
|
||
|
||
### Format comparison
|
||
|
||
| Format | Codec | Transparency | Video editors | Browsers | File size |
|
||
|--------|-------|-------------|---------------|----------|-----------|
|
||
| **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No | Large |
|
||
| **WebM** | VP9 | Yes | None (shows black background) | Chrome, Firefox | Small |
|
||
| **MP4** | H.264 | No | All | All | Small |
|
||
|
||
<Note>
|
||
**WebM VP9 alpha** is technically supported but all major video editors ignore the alpha channel and render transparent areas as black. Only Chromium-based browsers (Chrome, Arc, Brave, Edge) decode VP9 alpha correctly. Safari does not support it. Use MOV for editor workflows and WebM only for browser-based playback.
|
||
</Note>
|
||
|
||
### How it works
|
||
|
||
When you render with `--format mov` or `--format webm`, Hyperframes:
|
||
|
||
1. Captures each frame as a **PNG with alpha channel** (instead of JPEG for MP4)
|
||
2. Sets Chrome's page background to transparent via `Emulation.setDefaultBackgroundColorOverride`
|
||
3. Encodes with an alpha-capable codec (ProRes 4444 for MOV, VP9 for WebM)
|
||
|
||
Your composition's HTML should **not** set a `background` on `html` or `body` — leave it unset so the transparent background comes through.
|
||
|
||
### Authoring transparent compositions
|
||
|
||
```html
|
||
<style>
|
||
/* Do NOT set background on html/body — leave them transparent */
|
||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||
|
||
[data-composition-id="my-overlay"] {
|
||
position: relative;
|
||
width: 1920px;
|
||
height: 1080px;
|
||
overflow: hidden;
|
||
/* No background here either */
|
||
}
|
||
</style>
|
||
```
|
||
|
||
Only the visible elements (cards, text, images) will appear in the final video. Everything else will be transparent.
|
||
|
||
### Verifying transparency
|
||
|
||
- **In a browser:** Open the MOV file — it won't play (ProRes is not a browser codec). Instead, render a WebM copy and open it in Chrome on a checkerboard background page.
|
||
- **In a video editor:** Import the MOV file and place it on a track above other footage. Transparent areas should show the footage below.
|
||
- **Online tool:** Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify your MOV or WebM has working transparency.
|
||
|
||
## Tips
|
||
|
||
<Tip>
|
||
Use `draft` quality during development for fast previews. Switch to `standard` or `high` for final output.
|
||
</Tip>
|
||
|
||
- Use `npx hyperframes benchmark` to find optimal settings for your system
|
||
- Docker mode is slower but guarantees [identical output](/concepts/determinism) across platforms
|
||
- For compositions with many frames, `--gpu` can significantly speed up local encoding
|
||
|
||
## Next Steps
|
||
|
||
<CardGroup cols={2}>
|
||
<Card title="Deterministic Rendering" icon="lock" href="/concepts/determinism">
|
||
Understand the determinism guarantees
|
||
</Card>
|
||
<Card title="HDR Rendering" icon="sun" href="/guides/hdr">
|
||
Render HDR10 MP4 from HDR video and image sources
|
||
</Card>
|
||
<Card title="CLI Reference" icon="terminal" href="/packages/cli">
|
||
Full list of CLI commands and flags
|
||
</Card>
|
||
<Card title="Troubleshooting" icon="wrench" href="/guides/troubleshooting">
|
||
Fix common rendering issues
|
||
</Card>
|
||
</CardGroup>
|