Files
hyperframes/docs/guides/hdr.mdx
T
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346)
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
2026-04-20 12:16:24 -07:00

195 lines
9.2 KiB
Plaintext

---
title: HDR Rendering
description: "Render compositions to HDR10 MP4 (BT.2020 PQ or HLG, 10-bit H.265) when sources contain HDR video or images."
---
Hyperframes can render to HDR10 MP4 (H.265 10-bit, BT.2020) when your composition references HDR video or HDR still images. HDR is opt-in via the `--hdr` flag — it auto-detects HDR sources and falls back to SDR when none are present.
<Note>
The `--hdr` flag does not *force* HDR. It enables HDR detection. If your composition contains only SDR media, the flag is a no-op and you get a normal SDR render.
</Note>
## Quickstart
<Steps>
<Step title="Add an HDR source to your composition">
Hyperframes detects HDR from the source's color space metadata. The most reliable HDR sources are:
- **HDR video** tagged BT.2020 with PQ (`smpte2084`) or HLG (`arib-std-b67`) transfer
- **HDR still images** as 16-bit PNG with BT.2020 PQ encoding
See [Source Media](#source-media-requirements) for full details.
</Step>
<Step title="Render with --hdr">
```bash Terminal
npx hyperframes render --hdr --output output.mp4
```
HDR output requires `--format mp4`. If you also pass `--format mov` or `--format webm`, Hyperframes logs a warning and falls back to SDR.
</Step>
<Step title="Verify the output is HDR">
Use `ffprobe` to confirm the encoded stream carries HDR color tagging and HDR10 metadata:
```bash Terminal
ffprobe -v error -show_streams output.mp4 | grep -E 'color_transfer|color_primaries|color_space'
```
See [Verifying HDR output](#verifying-hdr-output) for what to look for.
</Step>
</Steps>
## How HDR Mode Works
When `--hdr` is set, the producer:
<Steps>
<Step title="Probes every video and image source">
Runs `ffprobe` on each `<video>` and `<img>` source to read its color space (primaries, transfer function, matrix). Probing is gated on `--hdr` to avoid `ffprobe` overhead on SDR-only renders.
</Step>
<Step title="Picks the dominant HDR transfer">
If any source uses PQ (`smpte2084`), the output uses **PQ**. Otherwise, if any source uses HLG (`arib-std-b67`), the output uses **HLG**. If no HDR sources are found, the flag is a no-op and you get an SDR render.
</Step>
<Step title="Encodes to H.265 10-bit BT.2020">
The video encoder switches to `libx265` with `-pix_fmt yuv420p10le`, color tagging `colorprim=bt2020:transfer=<smpte2084|arib-std-b67>:colormatrix=bt2020nc`, and HDR10 static metadata (`master-display` and `max-cll`). Without that metadata, players (QuickTime, YouTube, HDR TVs) tone-map the stream as if it were SDR BT.2020 — which looks wrong.
</Step>
<Step title="Composites HDR sources natively, converts SDR overlays">
HDR videos and images are extracted as 16-bit linear-light pixels via FFmpeg, kept out of the DOM screenshot, and composited server-side at full bit depth. SDR DOM overlays (text, shapes, UI from your HTML) are converted from **sRGB** to **BT.2020** before being layered on top, so colors do not shift.
</Step>
</Steps>
## Source Media Requirements
### HDR video
Hyperframes recognizes HDR video from its `ffprobe` color space metadata:
| Indicator | Recognized as HDR |
|-----------|-------------------|
| `color_primaries` contains `bt2020` | Yes |
| `color_space` contains `bt2020` | Yes |
| `color_transfer = smpte2084` (PQ) | Yes — PQ |
| `color_transfer = arib-std-b67` (HLG) | Yes — HLG |
| All else (e.g. `bt709`, `smpte170m`) | No — treated as SDR |
A valid HDR source is any MP4 whose stream metadata reports BT.2020 primaries plus PQ or HLG transfer. Hyperframes will detect it automatically:
```bash Terminal
ffprobe -v error -show_streams assets/clip.mp4 | grep color
# color_primaries=bt2020
# color_transfer=smpte2084
# color_space=bt2020nc
```
### HDR still images
Hyperframes supports HDR still images delivered as **16-bit PNGs** tagged with BT.2020 primaries and PQ transfer. Drop them into the composition as a normal `<img>`:
```html index.html
<img class="clip" data-start="0" data-duration="3"
src="./assets/hdr-photo.png" />
```
When `--hdr` is set, the image is decoded once to 16-bit linear-light RGB and composited natively into the HDR output.
<Note>
HDR `<img>` decoding is limited to **16-bit PNG**. JPEG, WebP, AVIF, and APNG are not recognized as HDR sources — they load through the normal SDR DOM path. For HDR motion, use a `<video>` element.
</Note>
### SDR sources mixed with HDR
You can freely mix SDR and HDR media in the same composition:
- **SDR videos** stay in the DOM screenshot path and get the sRGB → BT.2020 conversion described above
- **HDR videos** are extracted natively at 16-bit and composited underneath the SDR DOM layer
- **SDR images and DOM elements** (text, shapes, gradients, GSAP animations) are converted from sRGB to BT.2020
This is the same pipeline that handles compositions where, for example, an HDR drone clip plays under an SDR lower-third with animated text.
## Output Format Requirements
| Output format | HDR supported |
|---------------|---------------|
| `mp4` | Yes — H.265 10-bit BT.2020, HDR10 metadata |
| `mov` | No — falls back to SDR |
| `webm` | No — falls back to SDR |
If you set `--hdr` together with `--format mov` or `--format webm`, Hyperframes logs a message and produces the equivalent SDR render. There is no error — the render still completes — so check the logs (or your verification step) to confirm you got HDR.
## Verifying HDR Output
Use `ffprobe` to confirm both the color tagging and the HDR10 static metadata are present:
```bash Terminal
ffprobe -v error -show_streams -select_streams v:0 output.mp4 \
| grep -E 'codec_name|pix_fmt|color_transfer|color_primaries|color_space'
```
For a PQ HDR10 render you should see:
```
codec_name=hevc
pix_fmt=yuv420p10le
color_space=bt2020nc
color_primaries=bt2020
color_transfer=smpte2084
```
Then check the HDR10 SEI / container boxes:
```bash Terminal
ffprobe -v error -show_frames -read_intervals "%+#1" \
-show_entries frame=side_data_list output.mp4
```
You should see entries for **Mastering display metadata** and **Content light level metadata**. Without them, HDR-aware players will treat the file as SDR BT.2020 and the colors will look washed-out or wrong on an HDR display.
For HLG renders the only difference is `color_transfer=arib-std-b67` — the rest of the checks are the same.
## Docker Rendering
`--hdr` is forwarded into the Docker render pipeline, so you can produce HDR10 MP4 output from the containerized renderer:
```bash Terminal
npx hyperframes render --hdr --docker --output output.mp4
```
The container runs the same probe → composite → encode pipeline as the local renderer. Verify the output with the same `ffprobe` checks described in [Verifying HDR output](#verifying-hdr-output).
<Note>
Docker HDR rendering currently runs **CPU-side** for the SDR DOM layer (the container falls back to software WebGL because GPU passthrough is not configured by default). Frame capture is therefore slower than local headed Chrome — measure your own composition with `--quiet` off and compare wall-clock times before sizing CI runners. The encoded HDR10 metadata and pixel data are identical to a local render.
</Note>
## Limitations
- **MP4 only** — `--hdr` with `--format mov` or `--format webm` falls back to SDR
- **HDR images: 16-bit PNG only** — other formats (JPEG, WebP, AVIF, APNG) are not decoded as HDR and fall through the SDR DOM path
- **Player support** — the [`<hyperframes-player>`](/packages/player) web component plays back the encoded MP4 in the browser and inherits whatever HDR support the host browser provides; it does not implement its own HDR pipeline
- **Headed Chrome HDR DOM capture** — the engine ships a separate WebGPU-based capture path for rendering CSS-animated DOM directly into HDR (`initHdrReadback`, `launchHdrBrowser`). It requires headed Chrome with `--enable-unsafe-webgpu` and is not used by the default render pipeline. See [Engine: HDR](/packages/engine#hdr-apis) if you are building a custom integration.
## Common Pitfalls
| Symptom | Likely cause |
|---------|--------------|
| Output looks identical to SDR | Source media is SDR — `--hdr` is a no-op without an HDR source. Run `ffprobe` on your inputs |
| Output is "kind of HDR" but tone-mapped wrong on YouTube/QuickTime | Missing HDR10 static metadata on the encoded stream. Verify with the ffprobe snippet above |
| Docker render is much slower than local | Expected — the container falls back to software WebGL for SDR DOM capture. Pixel output is the same |
| Used `--format webm` and got SDR | Expected — HDR output is MP4 only |
| HDR `<img>` looks SDR / washed out | Source is not a 16-bit PNG. Re-export as 16-bit PNG (BT.2020 PQ) or use a `<video>` element instead |
## Next Steps
<CardGroup cols={2}>
<Card title="Rendering" icon="film" href="/guides/rendering">
Local vs Docker, quality presets, workers
</Card>
<Card title="CLI" icon="terminal" href="/packages/cli">
Full `render` command reference including `--hdr`
</Card>
<Card title="Engine: HDR APIs" icon="gear" href="/packages/engine#hdr-apis">
Public HDR utilities exported from `@hyperframes/engine`
</Card>
<Card title="Common Mistakes" icon="triangle-exclamation" href="/guides/common-mistakes">
Pitfalls that affect render output
</Card>
</CardGroup>