feat(engine): add HDR video output pipeline (#265)

## Summary

Adds the ability to render HDR video output (H.265 10-bit, BT.2020) from HyperFrames compositions. When the renderer detects HDR source video, it automatically switches to the HDR output pipeline — no flags needed.

## What it does

- **Auto-detection** — Probes each video source with `ffprobe`. If any has bt2020/PQ/HLG color metadata, the output switches to H.265 10-bit with correct color tags. SDR-only compositions are unaffected (H.264, bt709).
- **HLG pass-through** — Native HLG pixels from FFmpeg extraction are piped directly to the encoder without conversion. This avoids brightness loss from HLG→linear→PQ conversion (which requires an OOTF system gamma we can't reliably apply).
- **Encoder HDR support** — Both chunk and streaming encoders accept HDR presets: `libx265`, `yuv420p10le`, BT.2020 color primaries, `hvc1` codec tag (required for Apple playback).
- **WebGPU HDR capture (gated)** — A complete WebGPU float16 readback pipeline is implemented and tested but gated behind headed Chrome (headless doesn't expose WebGPU). Ready for future use with WebGPU canvas content.
- **HDR utilities** — `detectTransfer()` (PQ vs HLG), `getHdrEncoderColorParams()`, `analyzeCompositionHdr()`. 15 unit tests.

## Key design decisions

| Decision | Why |
|----------|-----|
| No `--hdr` flag | SDR content encoded as HDR causes orange shift in browsers. Auto-detect eliminates this. |
| HLG pass-through (not HLG→PQ) | Conversion loses brightness without OOTF. Pass-through matches source exactly. |
| `hvc1` codec tag | Apple QuickTime requires `hvc1` (not `hev1`) for HEVC playback. |
| 1-hour streaming timeout | HDR capture at ~6fps needs more time than the default 10-minute FFmpeg timeout. |

## Files changed

| File | What changed |
|------|-------------|
| `packages/engine/src/utils/hdr.ts` | **NEW** — HDR detection, transfer types, encoder params (15 tests) |
| `packages/engine/src/services/hdrCapture.ts` | **NEW** — WebGPU readback, HLG conversion, PQ encode |
| `packages/engine/src/services/streamingEncoder.ts` | HDR presets, raw rgb48le input, color tags |
| `packages/engine/src/services/chunkEncoder.ts` | HDR presets, conditional color tags |
| `packages/producer/src/services/renderOrchestrator.ts` | Auto-detection loop, HDR pass-through capture path |

## How to test

Render a composition with an HDR video source. The output should be H.265 10-bit with HDR metadata visible in `ffprobe` (bt2020, arib-std-b67 or smpte2084). Plays correctly in QuickTime and on HDR displays.

## Stack position

**2 of 6** — Stacked on #258 (SDR/HDR normalization). Provides the encoder infrastructure that phases 1-5 build on.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
This commit is contained in:
Vance Ingalls
2026-04-19 15:10:59 -07:00
committed by GitHub
parent d1f992570a
commit 5a3fde19d4
19 changed files with 1794 additions and 318 deletions
+35 -9
View File
@@ -10,6 +10,7 @@ import { copyFileSync, existsSync, mkdirSync, readdirSync, statSync, writeFileSy
import { join, dirname } from "path";
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
import { type GpuEncoder, getCachedGpuEncoder, getGpuEncoderName } from "../utils/gpuEncoder.js";
import { type HdrTransfer } from "../utils/hdr.js";
import { runFfmpeg } from "../utils/runFfmpeg.js";
import type { EncoderOptions, EncodeResult, MuxResult } from "./chunkEncoder.types.js";
@@ -21,15 +22,24 @@ export const ENCODER_PRESETS = {
high: { preset: "slow", quality: 15, codec: "h264" as const },
};
export interface EncoderPreset {
preset: string;
quality: number;
codec: "h264" | "h265" | "vp9" | "prores";
pixelFormat: string;
hdr?: { transfer: HdrTransfer };
}
/**
* Get encoder preset for a given quality and output format.
* WebM uses VP9 with alpha-capable pixel format; MP4 uses h264;
* WebM uses VP9 with alpha-capable pixel format; MP4 uses h264 (or h265 for HDR);
* MOV uses ProRes 4444 with alpha for editor-compatible transparency.
*/
export function getEncoderPreset(
quality: "draft" | "standard" | "high",
format: "mp4" | "webm" | "mov" = "mp4",
): { preset: string; quality: number; codec: "h264" | "vp9" | "prores"; pixelFormat: string } {
hdr?: { transfer: HdrTransfer },
): EncoderPreset {
const base = ENCODER_PRESETS[quality];
if (format === "webm") {
return {
@@ -47,6 +57,15 @@ export function getEncoderPreset(
pixelFormat: "yuva444p10le",
};
}
if (hdr) {
return {
preset: base.preset === "ultrafast" ? "fast" : base.preset,
quality: base.quality,
codec: "h265",
pixelFormat: "yuv420p10le",
hdr,
};
}
return { ...base, pixelFormat: "yuv420p" };
}
@@ -109,9 +128,8 @@ export function buildEncoderArgs(
if (bitrate) args.push("-b:v", bitrate);
else args.push("-crf", String(quality));
// Encoder-specific params: anti-banding + bt709 color space.
// Encoder-specific params: anti-banding + color space tagging.
// aq-mode=3 redistributes bits to dark flat areas (gradients).
// colorprim/transfer/colormatrix embed bt709 in the H.264/H.265 VUI.
const xParamsFlag = codec === "h264" ? "-x264-params" : "-x265-params";
const colorParams = "colorprim=bt709:transfer=bt709:colormatrix=bt709";
if (preset === "ultrafast") {
@@ -120,6 +138,10 @@ export function buildEncoderArgs(
args.push(xParamsFlag, `aq-mode=3:aq-strength=0.8:deblock=1,1:${colorParams}`);
}
}
// Apple devices require hvc1 tag for HEVC playback (default hev1 won't open in QuickTime)
if (codec === "h265") {
args.push("-tag:v", "hvc1");
}
} else if (codec === "vp9") {
args.push("-c:v", "libvpx-vp9", "-b:v", bitrate || "0", "-crf", String(quality));
args.push("-deadline", preset === "ultrafast" ? "realtime" : "good");
@@ -134,8 +156,12 @@ export function buildEncoderArgs(
return [...args, "-y", outputPath];
}
// BT.709 color space metadata — Chrome screenshots are sRGB which maps to bt709.
// Tags the output so players interpret colors correctly across devices.
// Color space metadata — tags the output so players interpret colors correctly.
// Chrome screenshots are always sRGB/bt709 pixels regardless of --hdr flag.
// We tag truthfully as bt709 even for HDR output — the --hdr flag gives
// H.265 + 10-bit encoding (better quality/compression) without lying about
// the color space. Tagging as bt2020 when pixels are bt709 causes browsers
// to apply the wrong color transform, producing visible orange/warm shifts.
if (codec === "h264" || codec === "h265") {
args.push(
"-colorspace:v",
@@ -148,15 +174,15 @@ export function buildEncoderArgs(
"tv",
);
// Convert full-range RGB input (Chrome screenshots) to limited/TV range for H.264.
// VAAPI already has a -vf chain for hwupload; prepend range conversion to it.
// Range conversion: Chrome's full-range RGB → limited/TV range.
if (gpuEncoder === "vaapi") {
// Replace the existing VAAPI -vf with one that includes range conversion
const vfIdx = args.indexOf("-vf");
if (vfIdx !== -1) {
args[vfIdx + 1] = `scale=in_range=pc:out_range=tv,${args[vfIdx + 1]}`;
}
} else if (!shouldUseGpu) {
// Range conversion: Chrome screenshots are full-range RGB.
// The scale filter handles both 8-bit and 10-bit correctly.
args.push("-vf", "scale=in_range=pc:out_range=tv");
}