mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
Miguel Ángel
1e05d78378
## Summary
- **Enable browser pool by default** (`enableBrowserPool: true`) — parallel capture workers now share a single Chrome process via reference-counted pool instead of each spawning their own (~256MB each). A 6-worker render drops from 7+ browser parent processes to 1 shared pool.
- **Add launch-promise deduplication** in `acquireBrowser` — when multiple workers race into the pool simultaneously (via `Promise.all`), they await the same launch Promise instead of each triggering a separate Chrome spawn. Same pattern as the existing `_autoBrowserGpuModeCache` for GPU probes.
- **Add `connected` health check** on pool hit — if Chrome crashes mid-render, subsequent acquires detect the dead browser and launch fresh instead of returning a stale reference.
- **Add `drainBrowserPool()`** for explicit cleanup between independent render jobs.
- **CLI studio server** now uses the shared pool instead of its own redundant `enableBrowserPool: false` singleton, so thumbnail generation shares Chrome with render workers.
## Problem
The engine had a reference-counted browser pool (`browserManager.ts:73-75`) but it was **disabled by default** (`enableBrowserPool: false`). This meant:
1. **Every parallel worker spawned its own Chrome** — a `--workers 6` render launched 7+ independent Chrome processes (1 probe + 6 workers), each ~256MB.
2. **The pool had a race condition** — even if manually enabled, concurrent workers calling `acquireBrowser()` via `Promise.all` could all see `pooledBrowser === null` before the first launch completed, spawning N Chromes instead of 1.
3. **No crash recovery** — if Chrome died, the pool still held the dead reference. Subsequent acquires got a disconnected browser.
4. **CLI studio server ran its own singleton** — `studioServer.ts` explicitly set `enableBrowserPool: false` and managed a separate browser, so thumbnails and renders could never share.
Over time, orphaned Chrome processes accumulated across renders and previews. We observed **344 headless Chrome processes** consuming **569% CPU and 20% memory** on a dev machine.
## Before / After (6-worker parallel render)
| Metric | Before (pool off) | After (pool on) |
|--------|-------------------|-----------------|
| Browser parent processes | 7+ (1 probe + 6 workers) | **2** (1 GPU probe + 1 shared) |
| Total Chrome processes (with helpers) | 40-50+ | **14** |
| Memory during capture | ~20%+ | **4.6%** |
| Render time (1200 frames, 30fps) | ~64s | **53s** (~17% faster) |
| Post-render orphans | Accumulated over time | **0** |
## Changes
| File | Change |
|------|--------|
| `engine/src/config.ts` | `enableBrowserPool` default `false` → `true` |
| `engine/src/services/browserManager.ts` | Extract `launchBrowser()`, add `_pooledBrowserLaunchPromise` dedup, add `connected` check on pool hit, add `drainBrowserPool()` and `_resetBrowserPoolForTests()` |
| `engine/src/index.ts` | Export `drainBrowserPool` |
| `engine/src/services/browserManager.test.ts` | Pool dedup and drain tests |
| `cli/src/server/studioServer.ts` | Remove `enableBrowserPool: false` override — thumbnails now share the pool |
| `producer/src/services/browserManager.ts` | Re-export `drainBrowserPool` |
## Backward compatibility
- `PRODUCER_ENABLE_BROWSER_POOL=false` env var disables pooling (same as before).
- Callers passing `{ enableBrowserPool: false }` explicitly still get isolated browsers.
- Tests that set `enableBrowserPool: false` in their config fixtures continue to work.
## Test plan
- [x] Engine tests pass (597/597)
- [x] Producer tests pass (406/407, 1 pre-existing flaky test in `pngDecodeBlitWorkerPool`)
- [x] Build passes (lint, format, typecheck all green via lefthook pre-commit)
- [x] Manual render: `shortform-financial` with `--workers 6` → 1200 frames in 53s, 0 orphaned Chrome processes after completion
- [x] Process monitoring during render confirmed 2 browser parents (1 GPU probe + 1 shared pool) instead of 7+
@hyperframes/engine
Seekable web-page-to-video rendering engine built on Puppeteer and FFmpeg.
Framework-agnostic: works with GSAP, Lottie, Three.js, CSS animations, or any web content that implements the window.__hf seek protocol.
Install
npm install @hyperframes/engine
Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded by Puppeteer), FFmpeg
What it does
The engine opens your HTML composition in a headless Chrome instance, seeks frame-by-frame using Chrome's HeadlessExperimental.beginFrame API, captures screenshots, and encodes them into video with FFmpeg.
Key services
| Service | Description |
|---|---|
| browserManager | Launches and pools headless Chrome instances (chrome-headless-shell) |
| frameCapture | Manages capture sessions — seek, screenshot, buffer lifecycle |
| screenshotService | BeginFrame-based capture with CDP (Chrome DevTools Protocol) |
| chunkEncoder | FFmpeg encoding with chunked concat, GPU detection, faststart |
| streamingEncoder | Pipe frames to FFmpeg in real time (no intermediate PNGs on disk) |
| audioMixer | Parse <audio> elements and mix audio tracks via FFmpeg |
| videoFrameExtractor | Extract frames from <video> elements for compositing |
| parallelCoordinator | Split frame ranges across worker processes |
| fileServer | Serve local HTML files to the browser via Hono |
Usage
import {
acquireBrowser,
releaseBrowser,
createCaptureSession,
initializeSession,
captureFrame,
closeCaptureSession,
} from "@hyperframes/engine";
// 1. Launch browser
const browser = await acquireBrowser({ captureMode: "beginFrame" });
// 2. Open a capture session
const session = createCaptureSession({
browser: browser.browser,
url: "http://localhost:3000/my-composition.html",
width: 1920,
height: 1080,
fps: 30,
});
await initializeSession(session);
// 3. Capture frames
for (let i = 0; i < totalFrames; i++) {
await captureFrame(session, i, `/tmp/frames/frame-${i}.png`);
}
// 4. Clean up
await closeCaptureSession(session);
await releaseBrowser(browser);
Most users should use @hyperframes/producer or the hyperframes CLI instead of calling the engine directly.
Documentation
Full documentation: hyperframes.heygen.com/packages/engine
Related packages
@hyperframes/core— types, parsers, frame adapters@hyperframes/producer— high-level render pipeline built on this enginehyperframes— CLI