mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-10 22:20:14 +00:00
feat(engine): wire options.hdr through chunkEncoder + dynamic SDR→HDR transfer (#370)
## Summary
Three independent fixes that share a common thread: HDR config flowing correctly from `EngineConfig` down through every encoder. The headline fix: disk-based HDR encodes via `chunkEncoder` were silently producing BT.709-tagged output despite `options.hdr` being set.
## Why
`Chunk 3` of `plans/hdr-followups.md`. The streaming encoder was correct but `chunkEncoder.buildEncoderArgs` hard-coded BT.709 color tags and the `bt709` VUI block in `-x265-params`, even when callers passed an HDR `EncoderOptions`. Today this is harmless because `renderOrchestrator` routes native-HDR content to `streamingEncoder` and only feeds `chunkEncoder` sRGB Chrome screenshots — but the contract was a lie, and any future caller that wired HDR through `chunkEncoder` would silently get SDR output.
## What changed
**3A — `chunkEncoder` respects `options.hdr` (BT.2020 + mastering metadata).** When `options.hdr` is set, the libx265 software path emits `bt2020nc` plus the matching transfer (`smpte2084` for PQ, `arib-std-b67` for HLG) at the codec level *and* embeds master-display + max-cll SEI in `-x265-params` via `getHdrEncoderColorParams`. libx264 still tags BT.709 inside `-x264-params` (libx264 has no HDR support) but the codec-level color flags flip so the container describes pixels truthfully. GPU H.265 (nvenc/videotoolbox/qsv/vaapi) gets the BT.2020 tags but no `-x265-params` block, so static mastering metadata is omitted — acceptable for previews, not HDR-aware delivery.
**3B — `convertSdrToHdr` accepts a target transfer.** `videoFrameExtractor.convertSdrToHdr` was hard-coded to `transfer=arib-std-b67` (HLG) regardless of the surrounding composition's dominant transfer. `extractAllVideoFrames` now calls `analyzeCompositionHdr` first, then passes the dominant transfer (`"pq"` or `"hlg"`) into `convertSdrToHdr` so an SDR clip mixed into a PQ timeline gets converted with `smpte2084`, not `arib-std-b67`.
**3C — `EngineConfig.hdr` type matches its declared shape.** The IIFE for the `hdr` field returned `undefined` when `PRODUCER_HDR_TRANSFER` wasn't `"hlg"` or `"pq"`, but the field is typed as `{ transfer: HdrTransfer } | false`. Returning `false` matches the type and avoids a downstream `undefined` check.
## Test plan
- [x] `chunkEncoder.test.ts`: replaced the previous "HDR options ignored" assertions with 8 new specs covering BT.2020 + transfer tagging, master-display/max-cll embedding, libx264 fallback behavior, GPU H.265 + HDR (tags but no x265-params), and range conversion for both SDR and HDR CPU paths.
- [x] All 313 engine unit tests pass (5 new HDR specs).
- [x] `ffprobe` an HDR composition rendered through the chunk encoder path: shows `bt2020nc` color matrix, `smpte2084` transfer, and mastering display metadata.
## Stack
Chunk 3 of `plans/hdr-followups.md`. Independent of Chunks 1/4 (touches separate code paths).
This commit is contained in:
@@ -10,7 +10,11 @@ import { existsSync, mkdirSync, readdirSync, rmSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { parseHTML } from "linkedom";
|
||||
import { extractVideoMetadata, type VideoMetadata } from "../utils/ffprobe.js";
|
||||
import { isHdrColorSpace as isHdrColorSpaceUtil } from "../utils/hdr.js";
|
||||
import {
|
||||
analyzeCompositionHdr,
|
||||
isHdrColorSpace as isHdrColorSpaceUtil,
|
||||
type HdrTransfer,
|
||||
} from "../utils/hdr.js";
|
||||
import { downloadToTemp, isHttpUrl } from "../utils/urlDownloader.js";
|
||||
import { runFfmpeg } from "../utils/runFfmpeg.js";
|
||||
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
|
||||
@@ -250,21 +254,28 @@ export async function extractVideoFramesRange(
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert an SDR video to HDR color space (HLG / BT.2020) so it can be
|
||||
* composited alongside HDR content without looking washed out.
|
||||
* Convert an SDR (BT.709) video to BT.2020 wide-gamut so it can be composited
|
||||
* alongside HDR content without looking washed out.
|
||||
*
|
||||
* Uses zscale for color space conversion with a nominal peak luminance of
|
||||
* 600 nits — high enough that SDR content doesn't appear too dark next to
|
||||
* HDR, matching the approach used by HeyGen's Rio pipeline.
|
||||
* Uses FFmpeg's `colorspace` filter to remap BT.709 → BT.2020 (no real tone
|
||||
* mapping — just a primaries swap so the input fits inside the wider HDR
|
||||
* gamut), then re-tags the stream with the caller's target HDR transfer
|
||||
* function (PQ for HDR10, HLG for broadcast HDR). The output transfer must
|
||||
* match the dominant transfer of the surrounding HDR content; otherwise the
|
||||
* downstream encoder will tag the final video with the wrong curve.
|
||||
*/
|
||||
async function convertSdrToHdr(
|
||||
inputPath: string,
|
||||
outputPath: string,
|
||||
targetTransfer: HdrTransfer,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout">>,
|
||||
): Promise<void> {
|
||||
const timeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;
|
||||
|
||||
// smpte2084 = PQ (HDR10), arib-std-b67 = HLG.
|
||||
const colorTrc = targetTransfer === "pq" ? "smpte2084" : "arib-std-b67";
|
||||
|
||||
const args = [
|
||||
"-i",
|
||||
inputPath,
|
||||
@@ -273,7 +284,7 @@ async function convertSdrToHdr(
|
||||
"-color_primaries",
|
||||
"bt2020",
|
||||
"-color_trc",
|
||||
"arib-std-b67",
|
||||
colorTrc,
|
||||
"-colorspace",
|
||||
"bt2020nc",
|
||||
"-c:v",
|
||||
@@ -401,8 +412,15 @@ export async function extractAllVideoFrames(
|
||||
}),
|
||||
);
|
||||
|
||||
const hasAnyHdr = videoColorSpaces.some(isHdrColorSpaceUtil);
|
||||
if (hasAnyHdr) {
|
||||
const hdrInfo = analyzeCompositionHdr(videoColorSpaces);
|
||||
if (hdrInfo.hasHdr && hdrInfo.dominantTransfer) {
|
||||
// dominantTransfer is "majority wins" — if a composition mixes PQ and HLG
|
||||
// sources (rare but legal), the minority transfer's videos get converted
|
||||
// with the wrong curve. We treat this as caller-error: a single composition
|
||||
// should not mix PQ and HLG sources, the orchestrator picks one transfer
|
||||
// for the whole render, and any source not on that curve is normalized to
|
||||
// it. If you need both transfers, render two separate compositions.
|
||||
const targetTransfer = hdrInfo.dominantTransfer;
|
||||
const convertDir = join(options.outputDir, "_hdr_normalized");
|
||||
mkdirSync(convertDir, { recursive: true });
|
||||
|
||||
@@ -410,12 +428,13 @@ export async function extractAllVideoFrames(
|
||||
if (signal?.aborted) break;
|
||||
const cs = videoColorSpaces[i] ?? null;
|
||||
if (!isHdrColorSpaceUtil(cs)) {
|
||||
// SDR video in a mixed timeline — convert to HDR color space
|
||||
// SDR video in a mixed timeline — convert to the dominant HDR transfer
|
||||
// so the encoder tags the final video correctly (PQ vs HLG).
|
||||
const entry = resolvedVideos[i];
|
||||
if (!entry) continue;
|
||||
const convertedPath = join(convertDir, `${entry.video.id}_hdr.mp4`);
|
||||
try {
|
||||
await convertSdrToHdr(entry.videoPath, convertedPath, signal, config);
|
||||
await convertSdrToHdr(entry.videoPath, convertedPath, targetTransfer, signal, config);
|
||||
entry.videoPath = convertedPath;
|
||||
} catch (err) {
|
||||
errors.push({
|
||||
|
||||
Reference in New Issue
Block a user