mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
Vance IngallsandClaude Opus 4.7
073b098e21
## What Adds a one-shot ffmpeg-psnr filter probe at drawElement session bootstrap. When the resident ffmpeg is missing or lacks libpostproc (no `psnr` filter), the capture-session router now force-fallbacks to the screenshot capture path and emits a `de_gate_reason = "ffmpeg_no_psnr_filter"` telemetry signal via the existing `render_complete` breakdown. Also tightens `psnrForDiskSample`'s catch: infrastructure-class ffmpeg failures (ENOENT, "No such filter") no longer silently skip the sample — they abort the render so the safety net cannot fail-open post-preflight. ## Why The drawElement self-verify safety net (parallelCoordinator's `psnrForDiskSample` → `psnrDb`) shells to `ffmpeg -lavfi psnr`. If ffmpeg is missing, or was compiled without libpostproc (so the `psnr` filter is absent), every per-sample compare throws. The existing catch swallows the error and returns `null` — callers treat that as "skip this sample" and the render completes with the safety net inoperative. Field signal (⭐ 9/10 CLI feedback, Slack ts=1787380767.210079, hyperframes 0.8.7, darwin/arm64, tid=93ff9910-2207-45c2-bc1f-54c0b347d4fe): > "host ffmpeg lacked psnr filter used by drawElement self-verification, > but render completed." The user's frames happened to be byte-identical so no visual damage shipped — but the safety net silently wasn't running. Any future compositor-damage bug on that host would have shipped straight through. ## How Two-part fix, both in `packages/engine`: 1. New `utils/psnrFilterAvailability.ts` — cached probe that runs `ffmpeg -hide_banner -filters` once per process and word-boundary- matches `psnr` in the output. Any failure (ENOENT, non-zero exit, timeout, unparseable output) returns `false`; never rejects. 2. Wired into `services/frameCapture.ts` `initDrawElementOrTransparentBackground` right after the Chrome capability probe: when useDrawElement resolves true and the preflight returns false, set `session.deGateReason = "ffmpeg_no_psnr_filter"` (same low-cardinality bucket every other DE gate uses; flows through `getCapturePerfSummary` → `render_complete.de_gate_reason` in PostHog), emit a stderr warning naming what's missing, and call `routeToFallback()` — the same fail-graceful shape as the SwiftShader / CSS-effect / at-risk-timeline gates. Skipped under `HF_FORCE_DRAWELEMENT=1` (matches the diagnostic knob's policy of bypassing every other gate). Belt-and-braces: `psnrForDiskSample` now discriminates infrastructure- class failures (ENOENT / "No such filter" / "Unknown filter") from per-sample noise (readFile races, transient EPERM). Only the former re-throw — per-sample noise still returns `null` (skipped sample). The preflight normally catches this at bootstrap; the re-throw covers ffmpeg-swapped-mid-render. ## Test plan - [x] Unit tests added: `packages/engine/src/utils/psnrFilterAvailability.test.ts` — mocked `execFile` covers: `psnr` present → true; `psnr` absent → false; ENOENT → false; non-zero exit → false; result memoized + reset works; substring-not-word-boundary → false. - [x] Unit tests added: `isFfmpegInfrastructureFailure` in `packages/engine/src/services/parallelCoordinator.test.ts` covers ENOENT, "No such filter", "Unknown filter", per-sample EACCES, parse errors, null/non-object. - [x] `bun run test` — `packages/engine/src/utils/psnrFilterAvailability.test.ts` (6 tests) + `packages/engine/src/services/parallelCoordinator.test.ts` (50 tests) + `frameCapture.test.ts` (26 tests) all pass. Pre-existing ffprobe test failures (4) on the base commit are unrelated (missing PNG fixture bytes — the file is 129 B on disk, likely LFS-stored). - [x] `bunx tsc --noEmit -p packages/engine/tsconfig.json` — clean. - [x] `bunx oxlint <files>` — 0 warnings, 0 errors. - [x] `bunx oxfmt --check <files>` — clean. Not covered here: an integration test that boots `initDrawElementOrTransparentBackground` end-to-end. That path is Puppeteer-driven and has no unit-scale bootstrap harness in the repository — the pure preflight + pure discriminator coverage above are what this PR can prove at the vitest layer. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@hyperframes/engine
Seekable web-page-to-video rendering engine built on Puppeteer and FFmpeg.
Framework-agnostic: works with GSAP, Lottie, Three.js, CSS animations, or any web content that implements the window.__hf seek protocol.
Install
npm install @hyperframes/engine
Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded by Puppeteer), FFmpeg
What it does
The engine opens your HTML composition in a headless Chrome instance, seeks frame-by-frame using Chrome's HeadlessExperimental.beginFrame API, captures screenshots, and encodes them into video with FFmpeg.
Key services
| Service | Description |
|---|---|
| browserManager | Launches and pools headless Chrome instances (chrome-headless-shell) |
| frameCapture | Manages capture sessions — seek, screenshot, buffer lifecycle |
| screenshotService | BeginFrame-based capture with CDP (Chrome DevTools Protocol) |
| chunkEncoder | FFmpeg encoding with chunked concat, GPU detection, faststart |
| streamingEncoder | Pipe frames to FFmpeg in real time (no intermediate PNGs on disk) |
| audioMixer | Parse <audio> elements and mix audio tracks via FFmpeg |
| videoFrameExtractor | Extract frames from <video> elements for compositing |
| parallelCoordinator | Split frame ranges across worker processes |
| fileServer | Serve local HTML files to the browser via Hono |
Usage
import {
acquireBrowser,
createCaptureSession,
initializeSession,
captureFrame,
closeCaptureSession,
} from "@hyperframes/engine";
// 1. Launch browser
const browserLease = await acquireBrowser({ captureMode: "beginFrame" });
// 2. Open a capture session
const session = createCaptureSession({
browser: browserLease.browser,
url: "http://localhost:3000/my-composition.html",
width: 1920,
height: 1080,
fps: 30,
});
await initializeSession(session);
// 3. Capture frames
for (let i = 0; i < totalFrames; i++) {
await captureFrame(session, i, `/tmp/frames/frame-${i}.png`);
}
// 4. Clean up
await closeCaptureSession(session);
await browserLease.release();
Most users should use @hyperframes/producer or the hyperframes CLI instead of calling the engine directly.
Documentation
Full documentation: hyperframes.heygen.com/packages/engine
Related packages
@hyperframes/core— types, parsers, frame adapters@hyperframes/producer— high-level render pipeline built on this enginehyperframes— CLI