mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
Vance Ingalls
2e1a1d91a2
## Summary
Two correctness fixes in the HDR transform & clipping pipeline: `parseTransformMatrix` now handles `matrix3d(...)` (GSAP's default `force3D: true`), and shader-transitions sets every non-first scene to `opacity: 0` at `t=0` so the engine doesn't over-composite at the start.
## Why
`Chunk 4` of `plans/hdr-followups.md`. Transform extraction and border-radius computation existed but were dead — an HDR video with `rotation: 45` rendered un-rotated, and 3-scene compositions ghosted at `t=0` because every scene defaulted to CSS `opacity: 1` and contributed to the first frame.
## What changed
**Matrix3d support in `parseTransformMatrix`.** `DOMMatrix.toString()` emits `matrix3d` whenever any ancestor in the chain has used a 3D transform — most importantly GSAP's default `force3D: true`, which converts `translate(...)` into `translate3d(..., 0)`. Without this, every GSAP-driven transform was silently dropped during HDR compositing because `videoFrameInjector.getViewportMatrix()` would return `matrix3d(...)` and the blit path would parse it as `null` and fall back to identity. The 16-value column-major form is converted to its 2D affine projection (indices 0, 1, 4, 5, 12, 13 → m11, m12, m21, m22, m41, m42); Z, perspective, and out-of-plane rotation components are dropped.
**Initial-state opacity in `initEngineMode`.** The browser preview branch uses a GL canvas overlay during transitions, so scene opacity at `t=0` doesn't matter visually. The engine branch reads scene opacity directly via `queryElementStacking()` to decide which layers to composite. Without an explicit initial-state tween, every scene defaulted to CSS `opacity: 1` and contributed to the very first frame, causing ghosting/overlap until the first transition fired. `tl.set()` at position 0 anchors the initial state in the timeline graph so reverse seeks from inside a later transition restore it correctly.
These two fixes together make `el.transform` and `el.borderRadius` (already wired in Chunk 7A's `compositeHdrFrame`) actually flow through the GSAP-animated case, and keep the engine's per-frame compositing aligned with what the user sees in browser preview.
## Test plan
- [x] 6 new `alphaBlit.test.ts` cases (identity matrix3d, translate3d, scale + translate3d, rotateZ, malformed arg count, non-finite values).
- [x] Existing `hdr-regression` Window H already CSS-sets `#scene-b { opacity: 0 }` as a fallback; the new `tl.set` is redundant for that case but harmless and removes the need for compositions to remember the CSS workaround.
- [x] Manual: rotated HDR video (`rotation: 45`) appears rotated; `border-radius: 50%` clips to circle; 3-scene composition has no overlap at `t=0`.
## Stack
Chunk 4 of `plans/hdr-followups.md`. Window F of the regression suite documents the bug; the next PR in the stack tightens the `maxFrameFailures` budget to 0.
HDR Regression Suite
HDR10 (BT.2020 PQ) regression suite with four back-to-back windows (A–D) covering the highest-value HDR compositing shapes. 10s / 300 frames at 30fps.
Windows
| # | Window | Pipeline aspect under test |
|---|---|---|
| A | Baseline HDR + direct opacity | HDR pass-through with a GSAP opacity tween directly on the <video> element. |
| B | Z-order sandwich (DOM → HDR → DOM) | Orange background, HDR video in the middle, blue overlay on top. Tests z-ordered layer compositing. |
| C | Transform + border-radius | HDR <video> with transform: rotate() scale() + border-radius clipping. Tests affine blit pipeline. |
| D | Shader transition (HDR → HDR image) | Shader transition between an HDR video and an HDR PQ image. Tests HDR image transfer cache + shader path. |
Fixtures
src/hdr-clip.mp4— short HEVC Main10 / BT.2020 PQ clip with a moving bright gradient (seeNOTICE.mdfor attribution).src/hdr-photo-pq.png— 256x144 16-bit RGB PNG with a hand-injectedcICPchunk (primaries=BT.2020, transfer=SMPTE ST 2084, matrix=GBR, range=full).
To regenerate the PNG fixture:
python3 packages/producer/tests/hdr-regression/scripts/generate-hdr-photo-pq.py
Running
cd packages/producer
bun run test hdr-regression
bun run test:update hdr-regression
In CI it runs in the hdr shard alongside hdr-hlg-regression
(see .github/workflows/regression.yml).