mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-06 02:01:07 +00:00
Vance Ingalls
e37ebe7999
## What Two fixes from adversarial testing of the DE parallel router (10 hostile comps, routed vs screenshot-baseline PSNR). The router itself held — both bugs are in general drawElement fast capture, and one slipped past self-verify. ### 1. Compile gate: ancestor background-image (`producer`) `drawElementService`'s per-frame ancestor fill replicates what lies behind the captured subtree by walking up the DOM for the nearest non-transparent **`backgroundColor`**. A background-**image** (`linear-gradient`, `url()`) on `body`/`html`/a wrapper reads as transparent in that scan, so a deeper ancestor's solid color paints instead wherever the subtree leaves pixels uncovered. Measured repro: body `linear-gradient` + html solid color + an element shrinking late in the comp → DE paints the html purple instead of the body gradient. 30.9 dB min frame vs baseline, visually unmistakable. Identical damage single-worker and parallel — general DE bug, in every wild DE render matching this (very common) authoring pattern. Fix: `detectAncestorBackgroundImage()` in the compiler (DOM-aware — inline styles on the root's ancestor chain + `<style>` rules resolved via `querySelectorAll`, so class-selected wrappers are covered; backgrounds *inside* the root are deliberately not matched). New compile gate `ancestor_background_image`, same shape as the 3D/mix-blend gates, bypass `HF_FAST_CAPTURE_ANCESTOR_BG=true`. ### 2. Self-verify tail sample (`engine`) The verify grid sampled at `(i+1)/(k+1)` → [20/40/60/80]% of the timeline. The damage above starts at ~79% and peaks after the last sample — verification **passed** on output that bottomed at 30.9 dB (threshold 32 dB would have caught it, it just never looked there). Fix: `computeDeVerifySampleFractions()` — first k−1 samples evenly spaced, last pinned at 95%. Default k=4 grid becomes [25/50/75/95]%. Kills the whole late-onset damage class, not just this repro. ## Validation - Repro comp (body gradient + shrink reveal): now gates → baseline route, 54.7 dB avg vs ground truth (was 41.6 avg / 30.9 min with the purple surround) - Control comp (nested stacked fades, routed): still routes, verify grid `[90, 180, 270, 342] of 360`, passes, 60.6 dB avg — unchanged - Full adversarial matrix context: 6/10 comps routed clean (49–68 dB min), blend/3D gated correctly, animated-canvas damage caught by verify at 16.5 dB with clean revert, video comps route legitimately (frames pre-extracted) - Tests: 7 new detection cases (`htmlCompiler.test.ts`), 5 new grid cases (`frameCapture-verifySampleFractions.test.ts`); `compileStage.test.ts` + `frameCapture.test.ts` suites green 🤖 Generated with [Claude Code](https://claude.com/claude-code)
@hyperframes/engine
Seekable web-page-to-video rendering engine built on Puppeteer and FFmpeg.
Framework-agnostic: works with GSAP, Lottie, Three.js, CSS animations, or any web content that implements the window.__hf seek protocol.
Install
npm install @hyperframes/engine
Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded by Puppeteer), FFmpeg
What it does
The engine opens your HTML composition in a headless Chrome instance, seeks frame-by-frame using Chrome's HeadlessExperimental.beginFrame API, captures screenshots, and encodes them into video with FFmpeg.
Key services
| Service | Description |
|---|---|
| browserManager | Launches and pools headless Chrome instances (chrome-headless-shell) |
| frameCapture | Manages capture sessions — seek, screenshot, buffer lifecycle |
| screenshotService | BeginFrame-based capture with CDP (Chrome DevTools Protocol) |
| chunkEncoder | FFmpeg encoding with chunked concat, GPU detection, faststart |
| streamingEncoder | Pipe frames to FFmpeg in real time (no intermediate PNGs on disk) |
| audioMixer | Parse <audio> elements and mix audio tracks via FFmpeg |
| videoFrameExtractor | Extract frames from <video> elements for compositing |
| parallelCoordinator | Split frame ranges across worker processes |
| fileServer | Serve local HTML files to the browser via Hono |
Usage
import {
acquireBrowser,
releaseBrowser,
createCaptureSession,
initializeSession,
captureFrame,
closeCaptureSession,
} from "@hyperframes/engine";
// 1. Launch browser
const browser = await acquireBrowser({ captureMode: "beginFrame" });
// 2. Open a capture session
const session = createCaptureSession({
browser: browser.browser,
url: "http://localhost:3000/my-composition.html",
width: 1920,
height: 1080,
fps: 30,
});
await initializeSession(session);
// 3. Capture frames
for (let i = 0; i < totalFrames; i++) {
await captureFrame(session, i, `/tmp/frames/frame-${i}.png`);
}
// 4. Clean up
await closeCaptureSession(session);
await releaseBrowser(browser);
Most users should use @hyperframes/producer or the hyperframes CLI instead of calling the engine directly.
Documentation
Full documentation: hyperframes.heygen.com/packages/engine
Related packages
@hyperframes/core— types, parsers, frame adapters@hyperframes/producer— high-level render pipeline built on this enginehyperframes— CLI