mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
## Summary This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI. The branch now does four things: - forwards `--hdr` through the Docker render path in the CLI - adds and expands HDR documentation across the docs site - adds first-class HDR still-image support to the engine/producer pipeline - adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags ## What changed ### CLI and docs - `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI - added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs - documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes ### Engine and producer HDR image support - added `ImageElement` support to the engine composition model and parsing path - threaded image elements through producer compilation and orchestration - probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source - included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order - integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays - forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic - skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows ### HDR metadata robustness - added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs - this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ ### Regression coverage and fixture cleanup - added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end - added `hdr-pq`, a focused HDR PQ regression fixture for the video path - updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only` - removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI - added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests ## Why The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking. The practical issue this closes is: - local host runs could pass while CI failed `hdr-image-only` - the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering - root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment - parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments ## Test plan ### Local targeted checks ```bash bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts ``` ### Producer regression runs on host ```bash bun run --cwd packages/core build:hyperframes-runtime:modular bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only ``` Observed result: - `fast` shard: 7 passed, 0 failed - `hdr` shard: 2 passed, 0 failed ### CI-equivalent Docker verification ```bash docker build -f Dockerfile.test -t hyperframes-producer:test . docker run --rm \ --security-opt seccomp=unconfined \ --shm-size=4g \ -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \ hyperframes-producer:test \ --sequential hdr-pq hdr-image-only ``` Observed result: - `hdr-image-only`: passed - `hdr-pq`: passed - shard summary: 2 passed, 0 failed ### Specific regression fixed Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with: - missing `"[Render] HDR source detected — output: PQ ..."` log line - full-frame visual mismatch across all 100 checkpoints - PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
262 lines
11 KiB
Plaintext
262 lines
11 KiB
Plaintext
---
|
||
title: Common Mistakes
|
||
description: "Pitfalls that break Hyperframes compositions."
|
||
---
|
||
|
||
These are mistakes that cannot be caught by the linter. For automated checks, run `npx hyperframes lint` (see [CLI](/packages/cli#lint)).
|
||
|
||
<Warning>
|
||
The first two mistakes — animating video element dimensions and controlling media playback in scripts — are the most common causes of broken compositions. If your video looks wrong, check these first.
|
||
</Warning>
|
||
|
||
<AccordionGroup>
|
||
<Accordion title="Animating video element dimensions">
|
||
**Symptom:** Video frames stop updating, or browser performance drops severely.
|
||
|
||
**Cause:** GSAP animating `width`, `height`, `top`, `left` directly on a `<video>` element can cause the browser to stop rendering frames.
|
||
|
||
**Before (broken):**
|
||
|
||
```javascript index.html
|
||
// Animating the video element directly — causes frame rendering to stop
|
||
tl.to("#el-video", { width: 500, height: 280, top: 700, left: 1400 }, 26);
|
||
```
|
||
|
||
**After (fixed):**
|
||
|
||
```html index.html
|
||
<!-- Wrap the video in a div and animate the wrapper -->
|
||
<div id="pip-wrapper" style="position: absolute; width: 1920px; height: 1080px;">
|
||
<video id="el-video" data-start="0" data-track-index="0"
|
||
src="./assets/video.mp4" style="width: 100%; height: 100%;"></video>
|
||
</div>
|
||
```
|
||
|
||
```javascript index.html
|
||
// Animate the wrapper — the video fills it at 100%
|
||
tl.to("#pip-wrapper", { width: 500, height: 280, top: 700, left: 1400 }, 26);
|
||
```
|
||
|
||
Use a non-timed wrapper `<div>` for visual effects like picture-in-picture. Animate the wrapper; let the video fill it via CSS.
|
||
</Accordion>
|
||
|
||
<Accordion title="Controlling media playback in scripts">
|
||
**Symptom:** Audio/video playback is out of sync, or plays when it should not.
|
||
|
||
**Cause:** Calling `video.play()`, `video.pause()`, or setting `audio.currentTime` in your scripts. The [framework owns all media playback](/reference/html-schema#framework-managed-behavior).
|
||
|
||
**Before (broken):**
|
||
|
||
```javascript index.html
|
||
// Conflicts with framework media sync
|
||
document.getElementById("el-video").play();
|
||
document.getElementById("el-audio").currentTime = 5;
|
||
```
|
||
|
||
**After (fixed):**
|
||
|
||
```javascript index.html
|
||
// Don't control media playback at all. The framework handles it.
|
||
// Use GSAP for visual animations only:
|
||
tl.to("#el-video", { opacity: 1, duration: 0.5 }, 0);
|
||
```
|
||
|
||
The framework reads [`data-start`](/concepts/data-attributes#timing-attributes), [`data-media-start`](/concepts/data-attributes#media-attributes), and [`data-volume`](/concepts/data-attributes#media-attributes) to control when and how media plays. See [Compositions: Two Layers](/concepts/compositions#two-layers-primitives-and-scripts) for the separation between HTML primitives and scripts.
|
||
</Accordion>
|
||
|
||
<Accordion title="Composition duration shorter than video">
|
||
**Symptom:** Video plays for a few seconds then stops. Timeline shows 8-10 seconds even though the video is minutes long.
|
||
|
||
**Cause:** The composition duration equals the [GSAP timeline duration](/guides/gsap-animation#timeline-duration-and-composition-duration), not `data-duration` on the video. If your last GSAP animation ends at 8 seconds, the composition is 8 seconds long — regardless of how long the video source is.
|
||
|
||
**Before (broken):**
|
||
|
||
```javascript index.html
|
||
// Timeline is only 7.8s long — video cuts off after 7.8 seconds
|
||
tl.to("#lower-third", { left: -640, duration: 0.6 }, 7.2);
|
||
```
|
||
|
||
**After (fixed):**
|
||
|
||
```javascript index.html
|
||
tl.to("#lower-third", { left: -640, duration: 0.6 }, 7.2);
|
||
|
||
// Extend the timeline to 283 seconds to match the video length
|
||
tl.set({}, {}, 283);
|
||
```
|
||
|
||
`tl.set({}, {}, TIME)` adds a zero-duration tween at the specified time, extending the timeline without affecting any elements.
|
||
|
||
<Tip>
|
||
A quick check: run `npx hyperframes compositions` to see the resolved duration of each composition. If it is shorter than expected, your timeline needs extending.
|
||
</Tip>
|
||
</Accordion>
|
||
|
||
<Accordion title="Missing class='clip' on timed elements">
|
||
**Symptom:** Elements are always visible, ignoring their `data-start` and `data-duration` timing.
|
||
|
||
**Cause:** The [`class="clip"`](/concepts/data-attributes#element-visibility) attribute tells the runtime to manage the element's visibility lifecycle. Without it, the element is always rendered.
|
||
|
||
**Before (broken):**
|
||
|
||
```html index.html
|
||
<!-- Missing class="clip" — this element is always visible -->
|
||
<h1 id="title" data-start="2" data-duration="5" data-track-index="0">
|
||
Hello World
|
||
</h1>
|
||
```
|
||
|
||
**After (fixed):**
|
||
|
||
```html index.html
|
||
<!-- With class="clip", the runtime shows this only from 2s to 7s -->
|
||
<h1 id="title" class="clip" data-start="2" data-duration="5" data-track-index="0">
|
||
Hello World
|
||
</h1>
|
||
```
|
||
|
||
<Note>
|
||
The linter catches this one: `npx hyperframes lint` will flag timed elements missing `class="clip"`.
|
||
</Note>
|
||
</Accordion>
|
||
|
||
<Accordion title="Oversized source images">
|
||
**Symptom:** Preview stutters during scenes with images on screen. Render is slower than expected.
|
||
|
||
**Cause:** Source images at much higher resolution than the canvas. Chrome decodes images to raw RGBA bitmaps before displaying them, and bitmap size is `width × height × 4` bytes — independent of file size on disk. A 7000×5000 JPEG is 140MB decoded, even if the file is only 2MB.
|
||
|
||
Displaying such an image in a 384×1080 region wastes memory and forces the compositor to resample a huge texture every frame.
|
||
|
||
**Before (bloated):**
|
||
|
||
```html index.html
|
||
<!-- 7000x5000 source, ~140MB decoded -->
|
||
<img class="clip" data-start="0" data-duration="3"
|
||
src="./assets/hero-scene.jpg" />
|
||
```
|
||
|
||
**After (sized to the canvas):**
|
||
|
||
```bash Terminal
|
||
# Resize a batch of images to fit within 3840x3840, preserving aspect ratio
|
||
mkdir -p assets/resized
|
||
mogrify -path assets/resized -resize 3840x3840\> assets/*.jpg
|
||
```
|
||
|
||
```html index.html
|
||
<!-- ~3840x2560 source, ~40MB decoded -->
|
||
<img class="clip" data-start="0" data-duration="3"
|
||
src="./assets/resized/hero-scene.jpg" />
|
||
```
|
||
|
||
**Rule of thumb:** source images at most 2x the canvas dimensions. For a 1920×1080 composition, 3840×2160 is already plenty. See [Performance: Image sizing](/guides/performance#image-sizing).
|
||
</Accordion>
|
||
|
||
<Accordion title="Heavy backdrop-filter stacks">
|
||
**Symptom:** Specific scenes drop to 5-10fps in preview. The composition is fine elsewhere.
|
||
|
||
**Cause:** `backdrop-filter: blur()` on large elements, especially stacked at high radii. Each blur layer forces the compositor to sample pixels behind the element, run a blur kernel, and composite the result. Stacked layers multiply the cost.
|
||
|
||
**Before (expensive):**
|
||
|
||
```css
|
||
/* 8 layers per side = 16 blur passes every frame */
|
||
.pb-1 { backdrop-filter: blur(1px); }
|
||
.pb-2 { backdrop-filter: blur(2px); }
|
||
.pb-3 { backdrop-filter: blur(4px); }
|
||
.pb-4 { backdrop-filter: blur(8px); }
|
||
.pb-5 { backdrop-filter: blur(16px); }
|
||
.pb-6 { backdrop-filter: blur(32px); }
|
||
.pb-7 { backdrop-filter: blur(64px); }
|
||
.pb-8 { backdrop-filter: blur(128px); }
|
||
```
|
||
|
||
**After (3 tuned layers):**
|
||
|
||
```css
|
||
/* Fewer passes with hand-picked radii — visually similar, much cheaper */
|
||
.pb-1 { backdrop-filter: blur(4px); }
|
||
.pb-2 { backdrop-filter: blur(16px); }
|
||
.pb-3 { backdrop-filter: blur(48px); }
|
||
```
|
||
|
||
**Guidelines:**
|
||
|
||
- Keep stacked `backdrop-filter` layers to 2-3 per region
|
||
- Avoid radii above 64px over large areas — the biggest radii dominate the total cost
|
||
- For a static blur effect, pre-render it into a PNG once and overlay with a regular `<img>`
|
||
|
||
See [Performance: backdrop-filter: blur()](/guides/performance#backdrop-filter-blur) for the full breakdown.
|
||
</Accordion>
|
||
|
||
<Accordion title="Expected HDR output but got SDR">
|
||
**Symptom:** Rendered with `--hdr`, but the output looks the same as SDR or `ffprobe` reports `color_transfer=bt709`.
|
||
|
||
**Cause:** `--hdr` is a *detection* flag, not a *force* flag. Hyperframes only switches to HDR encoding when a source `<video>` or `<img>` is tagged with BT.2020 / PQ / HLG color metadata. Two common reasons HDR is not engaged:
|
||
|
||
1. **All sources are SDR.** `--hdr` is a no-op on SDR-only compositions. Verify with `ffprobe`:
|
||
|
||
```bash Terminal
|
||
ffprobe -v error -show_streams source.mp4 | grep color_transfer
|
||
# Want: smpte2084 (PQ) or arib-std-b67 (HLG)
|
||
# SDR: bt709, smpte170m, bt470bg, etc.
|
||
```
|
||
|
||
2. **Wrong output format.** HDR output requires MP4. `--format mov` and `--format webm` fall back to SDR — Hyperframes logs a warning when this happens.
|
||
|
||
`--docker` works the same as local rendering — `--hdr` is forwarded into the container and produces the same HDR10 MP4 output (slower, since the container falls back to software WebGL for SDR DOM capture).
|
||
|
||
See [HDR Rendering](/guides/hdr) for the full source requirements and verification steps.
|
||
</Accordion>
|
||
|
||
<Accordion title="Timeline key doesn't match data-composition-id">
|
||
**Symptom:** Animations don't play. The composition appears static.
|
||
|
||
**Cause:** The key used in `window.__timelines` must exactly match the [`data-composition-id`](/concepts/data-attributes#composition-attributes) attribute on the composition root element.
|
||
|
||
**Before (broken):**
|
||
|
||
```javascript index.html
|
||
// Mismatch: HTML says "my-video", script registers "root"
|
||
// <div data-composition-id="my-video" ...>
|
||
window.__timelines["root"] = tl;
|
||
```
|
||
|
||
**After (fixed):**
|
||
|
||
```javascript index.html
|
||
// Key matches the data-composition-id attribute
|
||
// <div data-composition-id="my-video" ...>
|
||
window.__timelines["my-video"] = tl;
|
||
```
|
||
</Accordion>
|
||
</AccordionGroup>
|
||
|
||
## Debugging Checklist
|
||
|
||
When something does not work, check in this order:
|
||
|
||
1. **Run the linter:** `npx hyperframes lint` — catches most structural issues
|
||
2. **Timeline registered?** Is `window.__timelines["<id>"]` set? Does the key match [`data-composition-id`](/concepts/data-attributes#composition-attributes)?
|
||
3. **GSAP-only animations?** Only animate visual properties (opacity, transform, color) — see [GSAP Animation](/guides/gsap-animation#key-rules)
|
||
4. **Timeline long enough?** Add `tl.set({}, {}, DURATION)` at the end — see [Timeline Duration](/guides/gsap-animation#timeline-duration-and-composition-duration)
|
||
5. **Console errors?** Open browser console — runtime errors show as `[Browser:ERROR]`
|
||
6. **Still stuck?** See [Troubleshooting](/guides/troubleshooting) for environment and rendering issues
|
||
|
||
## Next Steps
|
||
|
||
<CardGroup cols={2}>
|
||
<Card title="Troubleshooting" icon="wrench" href="/guides/troubleshooting">
|
||
Fix environment and rendering issues
|
||
</Card>
|
||
<Card title="GSAP Animation" icon="wand-magic-sparkles" href="/guides/gsap-animation">
|
||
Review animation rules and patterns
|
||
</Card>
|
||
<Card title="HTML Schema Reference" icon="code" href="/reference/html-schema">
|
||
Full attribute reference and checklist
|
||
</Card>
|
||
<Card title="Data Attributes" icon="database" href="/concepts/data-attributes">
|
||
Timing, media, and composition attributes
|
||
</Card>
|
||
</CardGroup>
|