mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
James Russo
57cdf7d80e
## What Adds a content-addressed cache for extracted video frames, keyed on the tuple `(path, mtime, size, mediaStart, duration, fps, format)`. Repeat renders of the same composition (studio edit → re-render, preview → final) skip the ffmpeg extraction entirely. ## Why Video frame extraction is the dominant non-capture phase for video-heavy compositions. Studio iteration workflows extract the same frames over and over — each render burns ffmpeg time that adds no value. Validated on `/tmp/hf-fixtures/cfr-sdr-cache`: ``` Cold (miss): extractMs=69, videoExtractMs=70, totalElapsedMs=2052 Warm (hit): extractMs=1, videoExtractMs=2, totalElapsedMs=1964 cacheHits: 0→1, cacheMisses: 1→0 ``` The fixture is tiny (3s CFR SDR @ 30fps), so the wall-clock delta is small; the extraction-time delta (69→1ms, 98%) scales linearly with source length. For heavy-iteration workflows (a user rendering the same composition while tuning encoding params), extraction time goes to zero on every repeat render. Depends on #444 (instrumentation surface) and #445 (segment-scope HDR preflight — otherwise cache keys would be unstable across renders on mixed-HDR compositions). ## How - New `packages/engine/src/services/extractionCache.ts`: - SHA-256 key over a stable JSON encoding of `(path, mtime_ms, size, mediaStart, duration, fps, format)`. Infinity duration is normalized to `-1` so unresolved natural-duration sources still produce stable keys. - Truncates to 16 hex chars in the entry directory name — 64 bits of entropy is plenty at cache scale and keeps `ls` output short. - `hfcache-v2-` schema prefix — bumping it invalidates old entries (callers own gc policy; the cache owns keys). - `.hf-complete` dotfile sentinel. An entry dir without the sentinel is treated as a miss (covers crash-mid-extract and abandoned writes); the next render re-extracts over the partial frames with `-y`. - `FRAME_FILENAME_PREFIX = "frame_"` shared with the extractor — future refactors only need to touch one place to rename frames. - `EngineConfig.extractCacheDir` (env: `HYPERFRAMES_EXTRACT_CACHE_DIR`) gates the feature. Undefined disables caching — extraction runs into the render's workDir and cleanup removes it on render end, preserving the prior behaviour exactly. No default root is chosen by the engine; the caller (CLI, app, studio) owns the location policy. - `ExtractedFrames.ownedByLookup` flag prevents `FrameLookupTable.cleanup` from rm'ing a shared cache dir at render end. Set to `true` on both hits and misses (misses own the directory they wrote into, but hand it over to the cache rather than deleting it). - Phase 3 extractor flow: 1. Snapshot `(videoPath, mediaStart, start, end)` per resolved video BEFORE Phase 2a/2b preflight mutates them — so cache keys are stable across renders that use workDir-local normalized files (those files have fresh mtimes every render). 2. Compute key, `lookupCacheEntry`. 3. On hit: rebuild `ExtractedFrames` from the cache dir plus the Phase 2-probed `VideoMetadata` — no re-ffprobe. 4. On miss: `ensureCacheEntryDir`, extract with `extractVideoFramesRange(..., outputDirOverride)`, then `markCacheEntryComplete` (the sentinel write is the last step so a crash leaves the dir un-sentineled). - `extractVideoFramesRange` gains an `outputDirOverride` parameter so cache-miss writes land directly in the keyed dir (no `join(outputDir, videoId)` wrapping). ## Test plan - [x] 19 unit tests in `extractionCache.test.ts` covering key determinism, mtime/size invalidation, format/fps/mediaStart/duration invalidation, Infinity normalization, sentinel semantics, missing-file tolerance - [x] 2 integration tests in `videoFrameExtractor.test.ts`: - "reuses extracted frames on a warm cache hit" — asserts `cacheHits=1`, `extractMs<50ms` on second call against a CFR SDR fixture - "invalidates the cache when fps changes" — different fps on second call forces a new miss - [x] End-to-end validation with `HYPERFRAMES_EXTRACT_CACHE_DIR` set, two runs of the same fixture - [x] Lint + format (oxlint + oxfmt) - [x] Typecheck (engine + producer)
@hyperframes/engine
Seekable web-page-to-video rendering engine built on Puppeteer and FFmpeg.
Framework-agnostic: works with GSAP, Lottie, Three.js, CSS animations, or any web content that implements the window.__hf seek protocol.
Install
npm install @hyperframes/engine
Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded by Puppeteer), FFmpeg
What it does
The engine opens your HTML composition in a headless Chrome instance, seeks frame-by-frame using Chrome's HeadlessExperimental.beginFrame API, captures screenshots, and encodes them into video with FFmpeg.
Key services
| Service | Description |
|---|---|
| browserManager | Launches and pools headless Chrome instances (chrome-headless-shell) |
| frameCapture | Manages capture sessions — seek, screenshot, buffer lifecycle |
| screenshotService | BeginFrame-based capture with CDP (Chrome DevTools Protocol) |
| chunkEncoder | FFmpeg encoding with chunked concat, GPU detection, faststart |
| streamingEncoder | Pipe frames to FFmpeg in real time (no intermediate PNGs on disk) |
| audioMixer | Parse <audio> elements and mix audio tracks via FFmpeg |
| videoFrameExtractor | Extract frames from <video> elements for compositing |
| parallelCoordinator | Split frame ranges across worker processes |
| fileServer | Serve local HTML files to the browser via Hono |
Usage
import {
acquireBrowser,
releaseBrowser,
createCaptureSession,
initializeSession,
captureFrame,
closeCaptureSession,
} from "@hyperframes/engine";
// 1. Launch browser
const browser = await acquireBrowser({ captureMode: "beginFrame" });
// 2. Open a capture session
const session = createCaptureSession({
browser: browser.browser,
url: "http://localhost:3000/my-composition.html",
width: 1920,
height: 1080,
fps: 30,
});
await initializeSession(session);
// 3. Capture frames
for (let i = 0; i < totalFrames; i++) {
await captureFrame(session, i, `/tmp/frames/frame-${i}.png`);
}
// 4. Clean up
await closeCaptureSession(session);
await releaseBrowser(browser);
Most users should use @hyperframes/producer or the hyperframes CLI instead of calling the engine directly.
Documentation
Full documentation: hyperframes.heygen.com/packages/engine
Related packages
@hyperframes/core— types, parsers, frame adapters@hyperframes/producer— high-level render pipeline built on this enginehyperframes— CLI