mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
* feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes - 15 GLSL→TypeScript shader transitions on rgb48le buffers - Dual-scene compositing with scene detection via window.__hf.transitions - --hdr flag gates ffprobe probing (zero overhead on SDR compositions) - Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT - Buffer.from() copy in writeFrame() fixes streaming encoder race condition - SDR rendering fixes (three stacked bugs) - Object.assign fix for window.__hf preservation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: tighten shader smoke thresholds + assert .scene contract - Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268. * fix(hdr): restore VIRTUAL_TIME_SHIM and applyRenderModeHints in renderOrchestrator Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes") accidentally removed two pieces of the deterministic rendering pipeline: 1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`, which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven animations advance only when `window.__hf.seek(t)` is called. 2. The `applyRenderModeHints` function and its post-`compileForRender` call site, which auto-forces screenshot capture mode for compositions the compiler flagged as needing it (RAF, iframes, etc.). Without (1), RAF animations advanced by wall-clock between the main-loop seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat` lost its automatic fallback to screenshot mode and the child-document motion stopped being captured. Both helpers are still produced by `htmlCompiler` and exercised by `renderOrchestrator.test.ts` — the orchestrator just stopped calling them. Restored: - Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js` - Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer` call sites (probe + main render) - Re-add `applyRenderModeHints` (matching the test expectations) and call it immediately after `compileForRender` - Persist `renderModeHints` in `summary.json` and the "Compiled composition metadata" log line Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression failures on `feat/hdr-layered-compositing`. Made-with: Cursor * test(engine): expand sampleRgb48le coverage + audit Uint16Array alignment Adds: - 8 new sampleRgb48le bilinear-interpolation tests covering boundary pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers. - uint16-alignment-audit.test.ts documenting the alignment requirement for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE. Background: ~105 hot-loop sites in shader transitions still use readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut overhead but requires guaranteed even byteOffsets — these tests document the contract before any future refactor lands. * fix(engine,producer): mask DOM layers during HDR layered compositing The HDR layered compositor blits z-ordered layers over a shared canvas. DOM layers used a full-page screenshot from `captureAlphaPng`, which captures *every* painted pixel on the page — root background, sibling-scene content, overlay UI elements that aren't part of the current layer. Those opaque pixels were then blitted over the canvas, overwriting any HDR content composited beneath in earlier layers. The previous workaround toggled `display:none` on hide ids via `hideVideoElements`/`showVideoElements`. That correctly hid native videos but did nothing about the root composition's background or about overlay elements that the layer grouping considered part of a different layer. This commit replaces the workaround with a precise CSS mask installed before each DOM screenshot: 1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and re-shows the layer's elements (and their descendants and their injected `__render_frame_*` siblings) with `visibility: visible !important`. CSS visibility is *not* multiplicative through descendants — a child with `visibility: visible` overrides an ancestor's `visibility: hidden`, so deeply nested layer content still paints even though every intermediate ancestor is hidden by the mass-hide rule. 2. Non-layer data-start ids are inline-hidden with `visibility: hidden !important`. Inline `!important` beats stylesheet `!important`, so this overrides the show rule for elements that fall under a show selector but should NOT paint — most importantly HDR videos and other-layer SDR videos that live as descendants of `#root`. 3. `removeDomLayerMask` tears the stylesheet down and clears the inline `visibility`/`opacity` properties so subsequent video frame injection gets a clean slate. Crucially the mask only sets `visibility`, never `opacity`. CSS opacity *is* multiplicative — `opacity: 0` on `#root` would zero out every descendant including layer videos, even with `visibility: visible`. We also extend `initTransparentBackground` to force the composition root (`[data-composition-id]`) transparent in addition to `html`/`body`, because compositions almost always set `#root { background: ... }` and that background paints across the whole viewport otherwise. Both compositing paths use the new helpers: - The per-layer DOM branch (`compositeToBuffer`) for normal frames. - The transition path (single DOM screenshot per scene) so transition frames also get a clean per-scene capture. Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`: per-layer pixel-add accounting, dumps of every captured DOM PNG, and a periodic raw `rgb48le` snapshot of the composite buffer. These were essential to diagnosing the root-overwrite bug and stay zero-cost in normal renders. Also stops the workDir / per-video frame-dir cleanup when `KEEP_TEMP=1` so the dumps survive past frame N. Made-with: Cursor * fix(engine): preserve GSAP-applied opacity across DOM-layer captures SDR clips inside an HDR composition were rendering at full opacity even when the user had animated their wrapper opacity (e.g. fade-in or yoyo). Two bugs in the per-layer screenshot path conspired to drop the GSAP-applied opacity on the floor: 1. removeDomLayerMask was unconditionally calling `el.style.removeProperty("opacity")` on every wrapper after each layer capture. applyDomLayerMask only ever sets `visibility`, so the only inline opacity present is the value GSAP wrote. Stripping it between layer captures means that on the next capture (at the same timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline is already at that time — the opacity is never restored, and the wrapper renders fully opaque. 2. injectVideoFramesBatch was reading the source <video>'s computed opacity via `parseFloat(computedStyle.opacity) || 1` and copying it onto the injected <img>. Because syncVideoFrameVisibility forces the <video> to `opacity: 0 !important` to hide it during capture, the computed value is always 0, which `|| 1` then silently flips to full opacity. The <img> is a sibling of the <video> inside the same wrapper, so it should inherit opacity from the wrapper directly instead of having a value hard-set on it. Fix both: drop the opacity removal in removeDomLayerMask, skip opacity when copying visual properties from <video> to <img>, and explicitly clear any stale inline opacity on the <img> so it inherits from the wrapper that GSAP is animating. Made-with: Cursor * fix(producer): correct hdrLayerStartTimes typo to hdrVideoStartTimes The diagnostic logging block in executeRenderJob's HDR layer composite path referenced an undeclared `hdrLayerStartTimes` map. The correct variable, declared and populated earlier in the same function, is `hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer masking work and broke the producer build/typecheck on CI. Made-with: Cursor * fix(engine): restore video opacity copy to injected frame img Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on the assumption that the <img> sibling would inherit GSAP's opacity from a shared wrapper. That breaks any composition where GSAP animates opacity directly on the <video> element itself: the <img> has no animated ancestor and renders at full opacity throughout any fade, even when the user's intent is partial or zero opacity. The CI `style-7-prod` and `style-8-prod` regressions caught this: the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut because the <img> inherited opacity 1 regardless of GSAP's tween. Restore the old explicit copy from `computedStyle.opacity` to the <img>'s inline opacity, with the `|| 1` fallback intentionally preserved. The fallback is load-bearing: GSAP's seek does not re-apply tweens that have already completed, so post-fade frames read opacity 0 from the stale `opacity: 0 !important` we apply to hide the native <video>. The `|| 1` recovers the tween's end-state opacity 1 for those frames, matching the final on-screen intent and the existing baseline renders. Handles both DOM shapes: - GSAP on wrapper: video's own computed opacity is 1, img set to 1, wrapper's opacity applies via stacking as before. - GSAP on <video>: video's computed opacity is the tween value, copied to img directly since they are siblings. Fixes: - style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33) - style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@hyperframes/shader-transitions
WebGL shader transitions for HyperFrames compositions. Renders GPU-accelerated scene-to-scene transitions using fragment shaders, driven by GSAP timelines.
Install
npm install @hyperframes/shader-transitions
Or load directly via CDN:
<script src="https://cdn.jsdelivr.net/npm/@hyperframes/shader-transitions/dist/index.global.js"></script>
Usage
import { init } from "@hyperframes/shader-transitions";
const tl = init({
bgColor: "#0a0a0a",
accentColor: "#ff6b2b",
scenes: ["scene-1", "scene-2", "scene-3"],
transitions: [
{ time: 3, shader: "domain-warp", duration: 0.8 },
{ time: 8, shader: "light-leak", duration: 0.7 },
],
});
The init() function captures each scene to a WebGL texture at transition time, crossfades between them using the selected shader, and returns a GSAP timeline. If WebGL is unavailable, it falls back to hard cuts.
With an existing timeline
Pass your own GSAP timeline to layer transitions onto it:
const tl = gsap.timeline({ paused: true });
// ... add your scene animations ...
init({
bgColor: "#000",
scenes: ["intro", "demo", "outro"],
transitions: [
{ time: 5, shader: "cinematic-zoom" },
{ time: 12, shader: "glitch", duration: 0.5 },
],
timeline: tl,
});
Available shaders
| Shader | Description |
|---|---|
domain-warp |
Organic noise-based warp with glowing edge |
ridged-burn |
Ridged noise burn with sparks and heat glow |
whip-pan |
Horizontal motion blur simulating a fast camera pan |
sdf-iris |
Circular iris wipe with glowing ring edge |
ripple-waves |
Concentric ripple distortion radiating from center |
gravitational-lens |
Warping gravity well with chromatic aberration |
cinematic-zoom |
Radial zoom blur with chromatic fringing |
chromatic-split |
RGB channel separation expanding from center |
glitch |
Digital glitch with block displacement and scanlines |
swirl-vortex |
Spiral rotation with noise-based warping |
thermal-distortion |
Heat shimmer rising from the bottom |
flash-through-white |
Flash to white then reveal the next scene |
cross-warp-morph |
Noise-driven morph blending both scenes |
light-leak |
Warm cinematic light leak with lens flare |
API
init(config): GsapTimeline
| Option | Type | Required | Description |
|---|---|---|---|
bgColor |
string |
yes | Fallback background color (hex) for scene capture. Use the composition's body/canvas background — individual scenes set their own background-color via CSS. |
accentColor |
string |
no | Accent color (hex) for shader glow effects |
scenes |
string[] |
yes | Element IDs of each scene, in order |
transitions |
TransitionConfig[] |
yes | Transition definitions (see below) |
timeline |
GsapTimeline |
no | Existing timeline to attach transitions to |
compositionId |
string |
no | Override the data-composition-id for timeline registration |
TransitionConfig
| Option | Type | Default | Description |
|---|---|---|---|
time |
number |
— | Start time in seconds |
shader |
ShaderName |
— | Shader name from the table above |
duration |
number |
0.7 |
Transition duration in seconds |
ease |
string |
"power2.inOut" |
GSAP easing function |
SHADER_NAMES
Array of all available shader name strings, useful for validation or building UIs.
import { SHADER_NAMES } from "@hyperframes/shader-transitions";
// ["domain-warp", "ridged-burn", "whip-pan", ...]
Distribution
| Format | File | Use case |
|---|---|---|
| ESM | dist/index.js |
Bundlers (Vite, webpack, etc.) |
| CJS | dist/index.cjs |
Node.js / require() |
| IIFE | dist/index.global.js |
<script> tag, CDN (global: HyperShader) |
All formats include source maps. TypeScript definitions included.
Related packages
@hyperframes/core-- types, parsers, runtime@hyperframes/engine-- rendering enginehyperframes-- CLI
License
MIT