mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-07 10:06:21 +00:00
fix(cli): forward --hdr through Docker render + HDR docs (#346)
## Summary This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI. The branch now does four things: - forwards `--hdr` through the Docker render path in the CLI - adds and expands HDR documentation across the docs site - adds first-class HDR still-image support to the engine/producer pipeline - adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags ## What changed ### CLI and docs - `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI - added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs - documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes ### Engine and producer HDR image support - added `ImageElement` support to the engine composition model and parsing path - threaded image elements through producer compilation and orchestration - probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source - included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order - integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays - forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic - skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows ### HDR metadata robustness - added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs - this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ ### Regression coverage and fixture cleanup - added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end - added `hdr-pq`, a focused HDR PQ regression fixture for the video path - updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only` - removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI - added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests ## Why The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking. The practical issue this closes is: - local host runs could pass while CI failed `hdr-image-only` - the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering - root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment - parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments ## Test plan ### Local targeted checks ```bash bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts ``` ### Producer regression runs on host ```bash bun run --cwd packages/core build:hyperframes-runtime:modular bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only ``` Observed result: - `fast` shard: 7 passed, 0 failed - `hdr` shard: 2 passed, 0 failed ### CI-equivalent Docker verification ```bash docker build -f Dockerfile.test -t hyperframes-producer:test . docker run --rm \ --security-opt seccomp=unconfined \ --shm-size=4g \ -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \ hyperframes-producer:test \ --sequential hdr-pq hdr-image-only ``` Observed result: - `hdr-image-only`: passed - `hdr-pq`: passed - shard summary: 2 passed, 0 failed ### Specific regression fixed Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with: - missing `"[Render] HDR source detected — output: PQ ..."` log line - full-frame visual mismatch across all 100 checkpoints - PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
This commit is contained in:
Vance Ingalls
@@ -8,6 +8,8 @@ import {
|
||||
blitRgb48leAffine,
|
||||
parseTransformMatrix,
|
||||
roundedRectAlpha,
|
||||
resampleRgb48leObjectFit,
|
||||
normalizeObjectFit,
|
||||
} from "./alphaBlit.js";
|
||||
|
||||
// ── PNG construction helpers ─────────────────────────────────────────────────
|
||||
@@ -991,3 +993,155 @@ describe("blitRgb48leAffine with borderRadius", () => {
|
||||
expect(Buffer.compare(canvas1, canvas2)).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
// ── normalizeObjectFit ──────────────────────────────────────────────────────
|
||||
|
||||
describe("normalizeObjectFit", () => {
|
||||
it("returns supported values verbatim", () => {
|
||||
expect(normalizeObjectFit("fill")).toBe("fill");
|
||||
expect(normalizeObjectFit("cover")).toBe("cover");
|
||||
expect(normalizeObjectFit("contain")).toBe("contain");
|
||||
expect(normalizeObjectFit("none")).toBe("none");
|
||||
expect(normalizeObjectFit("scale-down")).toBe("scale-down");
|
||||
});
|
||||
|
||||
it("trims whitespace and lowercases input", () => {
|
||||
expect(normalizeObjectFit(" COVER ")).toBe("cover");
|
||||
});
|
||||
|
||||
it("falls back to fill for unsupported values", () => {
|
||||
expect(normalizeObjectFit(undefined)).toBe("fill");
|
||||
expect(normalizeObjectFit("")).toBe("fill");
|
||||
expect(normalizeObjectFit("inherit")).toBe("fill");
|
||||
expect(normalizeObjectFit("garbage")).toBe("fill");
|
||||
});
|
||||
});
|
||||
|
||||
// ── resampleRgb48leObjectFit ────────────────────────────────────────────────
|
||||
|
||||
function readRgb16(buf: Buffer, width: number, x: number, y: number): [number, number, number] {
|
||||
const off = (y * width + x) * 6;
|
||||
return [buf.readUInt16LE(off), buf.readUInt16LE(off + 2), buf.readUInt16LE(off + 4)];
|
||||
}
|
||||
|
||||
describe("resampleRgb48leObjectFit", () => {
|
||||
it("returns the same buffer unchanged for identity fill resample", () => {
|
||||
const src = makeHdrFrame(4, 4, 40000, 30000, 20000);
|
||||
const out = resampleRgb48leObjectFit(src, 4, 4, 4, 4, "fill");
|
||||
|
||||
// Fast path returns the same Buffer reference, not a copy
|
||||
expect(out).toBe(src);
|
||||
});
|
||||
|
||||
it("returns the source untouched on degenerate dimensions", () => {
|
||||
const src = makeHdrFrame(4, 4, 1, 2, 3);
|
||||
expect(resampleRgb48leObjectFit(src, 0, 4, 8, 8, "cover")).toBe(src);
|
||||
expect(resampleRgb48leObjectFit(src, 4, 4, 0, 8, "cover")).toBe(src);
|
||||
});
|
||||
|
||||
it("fills a larger box with stretched content (fit=fill)", () => {
|
||||
const src = makeHdrFrame(2, 2, 50000, 40000, 30000);
|
||||
const out = resampleRgb48leObjectFit(src, 2, 2, 8, 4, "fill");
|
||||
|
||||
expect(out.length).toBe(8 * 4 * 6);
|
||||
// Every output pixel should be the source color (uniform input → uniform output)
|
||||
for (let y = 0; y < 4; y++) {
|
||||
for (let x = 0; x < 8; x++) {
|
||||
const [r, g, b] = readRgb16(out, 8, x, y);
|
||||
expect(r).toBe(50000);
|
||||
expect(g).toBe(40000);
|
||||
expect(b).toBe(30000);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("covers the destination box (cover) — fills entire box, no black bars", () => {
|
||||
// 4×2 source into a 6×6 dst: cover scales by 6/2 = 3 → rendered 12×6, cropped horizontally
|
||||
const src = makeHdrFrame(4, 2, 65000, 0, 0);
|
||||
const out = resampleRgb48leObjectFit(src, 4, 2, 6, 6, "cover");
|
||||
|
||||
// No pillarbox/letterbox black anywhere
|
||||
for (let y = 0; y < 6; y++) {
|
||||
for (let x = 0; x < 6; x++) {
|
||||
const [r] = readRgb16(out, 6, x, y);
|
||||
expect(r).toBe(65000);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("contains the source (contain) and letterboxes with opaque black", () => {
|
||||
// 4×2 source into a 6×6 dst: contain scales by 6/4 = 1.5 → rendered 6×3, vertically centered
|
||||
const src = makeHdrFrame(4, 2, 65000, 65000, 65000);
|
||||
const out = resampleRgb48leObjectFit(src, 4, 2, 6, 6, "contain");
|
||||
|
||||
// Top and bottom rows should be black (letterbox)
|
||||
for (const y of [0, 5]) {
|
||||
for (let x = 0; x < 6; x++) {
|
||||
expect(readRgb16(out, 6, x, y)).toEqual([0, 0, 0]);
|
||||
}
|
||||
}
|
||||
// Middle band (rows 2–3) should be the source color
|
||||
for (const y of [2, 3]) {
|
||||
for (let x = 0; x < 6; x++) {
|
||||
const [r, g, b] = readRgb16(out, 6, x, y);
|
||||
expect(r).toBe(65000);
|
||||
expect(g).toBe(65000);
|
||||
expect(b).toBe(65000);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("none preserves source size and centers it on a black background", () => {
|
||||
// 2×2 source into a 6×6 dst with default object-position 50%/50%
|
||||
const src = makeHdrFrame(2, 2, 40000, 30000, 20000);
|
||||
const out = resampleRgb48leObjectFit(src, 2, 2, 6, 6, "none");
|
||||
|
||||
// Center 2×2 region (rows 2–3, cols 2–3) holds the source
|
||||
for (let y = 2; y < 4; y++) {
|
||||
for (let x = 2; x < 4; x++) {
|
||||
const [r, g, b] = readRgb16(out, 6, x, y);
|
||||
expect(r).toBe(40000);
|
||||
expect(g).toBe(30000);
|
||||
expect(b).toBe(20000);
|
||||
}
|
||||
}
|
||||
// Corners should be black
|
||||
expect(readRgb16(out, 6, 0, 0)).toEqual([0, 0, 0]);
|
||||
expect(readRgb16(out, 6, 5, 5)).toEqual([0, 0, 0]);
|
||||
});
|
||||
|
||||
it("respects object-position for none-fit alignment", () => {
|
||||
// 2×2 source into a 6×6 dst, anchored top-left
|
||||
const src = makeHdrFrame(2, 2, 40000, 30000, 20000);
|
||||
const out = resampleRgb48leObjectFit(src, 2, 2, 6, 6, "none", "0% 0%");
|
||||
|
||||
// Top-left 2×2 block holds the source
|
||||
for (let y = 0; y < 2; y++) {
|
||||
for (let x = 0; x < 2; x++) {
|
||||
const [r] = readRgb16(out, 6, x, y);
|
||||
expect(r).toBe(40000);
|
||||
}
|
||||
}
|
||||
// Bottom-right corner stays black
|
||||
expect(readRgb16(out, 6, 5, 5)).toEqual([0, 0, 0]);
|
||||
// Just below the source band should be black
|
||||
expect(readRgb16(out, 6, 0, 2)).toEqual([0, 0, 0]);
|
||||
expect(readRgb16(out, 6, 2, 0)).toEqual([0, 0, 0]);
|
||||
});
|
||||
|
||||
it("scale-down behaves like none when source fits in dst", () => {
|
||||
const src = makeHdrFrame(2, 2, 40000, 30000, 20000);
|
||||
const noneOut = resampleRgb48leObjectFit(src, 2, 2, 6, 6, "none");
|
||||
const sdOut = resampleRgb48leObjectFit(src, 2, 2, 6, 6, "scale-down");
|
||||
|
||||
expect(Buffer.compare(noneOut, sdOut)).toBe(0);
|
||||
});
|
||||
|
||||
it("scale-down behaves like contain when source overflows dst", () => {
|
||||
const src = makeHdrFrame(8, 4, 40000, 30000, 20000);
|
||||
const containOut = resampleRgb48leObjectFit(src, 8, 4, 6, 6, "contain");
|
||||
const sdOut = resampleRgb48leObjectFit(src, 8, 4, 6, 6, "scale-down");
|
||||
|
||||
expect(Buffer.compare(containOut, sdOut)).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user