Files
hyperframes/packages/engine/src/services/browserManager.ts
T
Miguel Ángel 395fb9c084 feat: add browser GPU render mode (#571)
## Problem

HyperFrames already had `--gpu`, but that flag only controlled FFmpeg hardware encoding. The browser capture path still forced Chrome/WebGL through SwiftShader software GL via `--use-angle=swiftshader`, so WebGL-heavy local renders could leave the biggest bottleneck on the CPU path.

That made the existing flag naming easy to misread: `--gpu` sounded like it accelerated the whole render, but it did not change the browser frame-capture backend.

## What this fixes

- Enables host browser GPU acceleration automatically for local CLI renders.
- Adds `--no-browser-gpu` as the local opt-out for software Chrome/WebGL capture.
- Keeps `--browser-gpu` as an explicit local browser-GPU request.
- Adds `browserGpuMode: "software" | "hardware"` to engine config, with `PRODUCER_BROWSER_GPU_MODE` env support for lower-level producer users.
- Keeps Docker browser capture on the deterministic software path.
- Maps hardware browser GPU mode to platform-native Chrome backends:
  - macOS: Metal-backed ANGLE
  - Windows: D3D11-backed ANGLE
  - Linux: EGL
- Blocks explicit `--browser-gpu --docker` with a clear error because Docker browser GPU passthrough is not cross-platform.
- Clarifies docs so `--gpu` means FFmpeg encoder GPU and browser GPU means Chrome/WebGL capture GPU.
- Keeps encoder backend selection auto-detected from FFmpeg capabilities:
  - NVIDIA: NVENC
  - macOS: VideoToolbox
  - Linux: VAAPI
  - Intel: QSV

## Why two flags

There are two separate GPU surfaces in the render pipeline:

1. Browser GPU controls Chrome frame capture.
   - Affects WebGL, canvas, CSS rendering, compositing, and screenshot capture inside the browser.
   - This is enabled automatically for local CLI renders.
   - Use `--no-browser-gpu` when you want the software browser baseline.

2. `--gpu` controls FFmpeg video encoding.
   - Affects the final encode step after frames have already been captured.
   - The concrete encoder is auto-detected from the host FFmpeg build and hardware.
   - It can be faster for some machines/codecs, but it is not equivalent to browser rendering acceleration.

The controls stay independent because users may want:

- `hyperframes render` for the fast local default with browser GPU capture.
- `hyperframes render --no-browser-gpu` for the software-browser local baseline.
- `hyperframes render --gpu` for browser GPU capture plus hardware FFmpeg encoding.
- `hyperframes render --no-browser-gpu --gpu` for software browser capture plus hardware FFmpeg encoding.
- `hyperframes render --docker` for deterministic browser capture.

## Why `--gpu` does not imply browser GPU

Keeping `--gpu` scoped to FFmpeg encoding avoids a semantic break and keeps the risk profile explicit:

- `--gpu` already means encoder acceleration. Expanding it to also change Chrome capture would silently alter behavior for users who only wanted hardware encoding.
- Browser GPU and encoder GPU have different portability. Encoder GPU can work in Docker when the host exposes the right devices; browser GPU passthrough is not cross-platform, so this PR intentionally blocks explicit `--browser-gpu --docker`.
- The Apple presentation benchmark shows why the controls should stay separate: browser GPU capture was the useful improvement, while macOS VideoToolbox via `--gpu` was slower and produced larger output for this `standard` H.264 run.

If HyperFrames later wants a single umbrella acceleration control, it should be explicit, for example `--acceleration browser|encoder|all` or `--gpu=browser|encoder|all`, rather than changing the meaning of the existing boolean `--gpu`.

## Root cause

`buildChromeArgs()` always injected `--use-gl=angle --use-angle=swiftshader`. `disableGpu` only appended `--disable-gpu`; it did not provide a hardware-GPU mode. That made the public `--gpu` flag look broader than it was, because render capture stayed software-backed even when encoder GPU was requested.

## Verification

### Local checks

- `bun install`
- `bun run build:hyperframes-runtime`
- `bun run --filter @hyperframes/engine test src/config.test.ts src/services/browserManager.test.ts`
- `bun run --filter @hyperframes/cli test src/utils/dockerRunArgs.test.ts src/commands/render.test.ts`
- `bun run --filter @hyperframes/cli typecheck`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run --filter @hyperframes/producer typecheck`
- `cd packages/producer && bunx vitest run src/services/renderOrchestrator.test.ts`
- `bunx oxlint packages/cli/src/commands/render.ts packages/cli/src/commands/render.test.ts packages/cli/src/utils/dockerRunArgs.ts packages/cli/src/utils/dockerRunArgs.test.ts packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/browserManager.ts packages/engine/src/services/browserManager.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt --check ...` on changed source/docs files
- `git diff --check`
- `bun packages/cli/src/cli.ts render --help | rg -n "browser-gpu|no-browser-gpu|GPU"`
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --output /tmp/hf-auto-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict`
  - Render plan prints `GPU: browser GPU (auto)`.
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --no-browser-gpu --output /tmp/hf-software-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict`
  - Render plan does not print browser GPU.
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --docker --browser-gpu --output /tmp/should-not-render.mp4`
  - Exits 1 with `Browser GPU is local-only`.
- `buildDockerRunArgs()` regression coverage asserts Docker container args include `--no-browser-gpu`, preventing nested container renders from re-enabling browser GPU through the local CLI default.
- `resolveBrowserGpuForCli()` regression coverage asserts `PRODUCER_BROWSER_GPU_MODE=software` opts out when no CLI browser-GPU flag is supplied, while explicit `--browser-gpu` / `--no-browser-gpu` still win.
- `ffmpeg -v error -i /tmp/hf-auto-browser-gpu-smoke.mp4 -f null -`
- `ffmpeg -v error -i /tmp/hf-software-browser-gpu-smoke.mp4 -f null -`
- `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-browser-gpu-smoke.mp4` -> H.264, 1920x1080, 24fps, 5.0s

### Apple presentation benchmark

Rendered `/Users/miguel07code/Downloads/apple-presentation.zip` as supplied after extracting to `/tmp/hf-apple-profile/apple-presentation`.

Fixed settings:

- 1920x1080
- 30fps
- `standard` quality
- 4240 frames
- 141.32s duration
- 8-worker cap; render auto-calibration used 6 capture workers
- macOS host detected FFmpeg GPU encoder: `videotoolbox`

| Mode | Equivalent flags after this PR | Wall time | vs software-browser baseline | Speed | Capture | Encode | Output |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Software browser + CPU encode | `--no-browser-gpu` | 120.77s | baseline | 1.17x | 97.87s | 10.04s | 8.38MB |
| Browser GPU + CPU encode | default local render | 70.10s | 42.0% faster | 2.02x | 50.72s | 9.91s | 8.39MB |
| Software browser + encoder GPU | `--no-browser-gpu --gpu` | 133.16s | 10.3% slower | 1.06x | 103.58s | 18.31s | 25.43MB |
| Browser GPU + encoder GPU | `--gpu` | 74.12s | 38.6% faster | 1.91x | 46.69s | 17.93s | 25.45MB |

Result: browser GPU capture is the meaningful improvement for this WebGL/browser-capture-heavy presentation. VideoToolbox encoding was slower and produced larger files for this current `standard` H.264 path, so `--gpu` should stay separate and opt-in.

Why `--gpu` plus browser GPU was slower than browser GPU alone: the combined run captured about 4.0s faster than browser GPU alone, but VideoToolbox encoding was about 8.0s slower than CPU x264 encoding, so the encode loss outweighed the capture gain.

### VideoToolbox flag check

I also isolated the encode stage against the already-captured Apple frames to check whether macOS GPU encoding only needed special flags.

`ffmpeg -h encoder=h264_videotoolbox` does not expose a CRF/CQ-style quality option like x264. It exposes bitrate-oriented and VideoToolbox-specific options such as `-b:v`, `-realtime`, `-profile`, `-coder`, `-prio_speed`, `-power_efficient`, and `-allow_sw`. That means our current `-q:v` mapping is not equivalent to x264 CRF and can produce very different bitrate/size behavior.

Measured full-frame encode variants on this host:

| VideoToolbox variant | Encode wall time | Output size | Bitrate |
| --- | ---: | ---: | ---: |
| Current `-q:v 64 -allow_sw 1` | 18.76s | 25.31MB | 1.43 Mbps |
| Current without `-allow_sw 1` | 18.21s | 25.31MB | 1.43 Mbps |
| `-b:v 500k -maxrate 750k -bufsize 1000k -profile high -coder cabac -realtime 1 -prio_speed 1 -power_efficient 0` | 20.58s | 7.42MB | 0.42 Mbps |
| Same with `-b:v 1500k` | 20.84s | 16.70MB | 0.95 Mbps |
| `-b:v 500k -profile baseline -coder cavlc -realtime 1 -prio_speed 1 -power_efficient 0` | 18.11s | 8.94MB | 0.51 Mbps |

Conclusion: VideoToolbox can be made size/bitrate-predictable with explicit `--video-bitrate`, but the tested speed-oriented flags did not make it faster than CPU x264 wall time for this render. That reinforces keeping `--gpu` encoder acceleration explicit and separate from browser GPU capture.

Artifacts from the local benchmark:

- `/tmp/hf-apple-profile/results/cpu.mp4`
- `/tmp/hf-apple-profile/results/browser-gpu.mp4`
- `/tmp/hf-apple-profile/results/encoder-gpu.mp4`
- `/tmp/hf-apple-profile/results/full-gpu.mp4`
- `/tmp/hf-apple-profile/results/summary.json`

All four benchmark MP4s completed `ffprobe` and full `ffmpeg -f null` decode checks.

### Pixel comparison

Compared decoded MP4 output between software-browser and browser-GPU renders:

- Apple presentation:
  - 4240 frames compared
  - 636 exact matching decoded frame hashes
  - 3604 different decoded frame hashes
  - Average PSNR: 57.79 dB
- `css-spinner-render-compat` clean fixture:
  - 120 frames compared
  - 0 exact matching decoded frame hashes
  - Average PSNR: 61.57 dB

Interpretation: browser GPU output is not strict hash/pixel-identical to the software-browser path after lossy H.264 encode, but the measured deltas are visually tiny. Above 50 dB PSNR is typically visually indistinguishable for normal video review. Use `--no-browser-gpu` or Docker when strict cross-run/cross-machine reproducibility matters more than local speed.

### Browser verification

- Started HyperFrames Studio preview for `packages/producer/tests/css-spinner-render-compat/src`.
- Used `agent-browser` to open `http://localhost:5191#project/src` and verify the composition loaded in Studio.
- Screenshots:
  - `/tmp/hf-gpu-browser-proof/preview-loaded.png`
  - `/tmp/hf-gpu-browser-proof/preview-playing.png`
  - `/tmp/hf-gpu-browser-proof/preview-frame-60.png`
- Agent-browser recordings:
  - `/tmp/hf-gpu-browser-proof/preview-playback.webm`
  - `/tmp/hf-gpu-browser-proof/preview-seek.webm`

## Notes

- Browser GPU is enabled automatically for local CLI renders and disabled in Docker.
- `--no-browser-gpu` is the opt-out for software Chrome/WebGL capture.
- `--gpu` remains encoder-only and opt-in.
- The Apple presentation zip has existing lint errors around unmanaged nested videos and imperative media `play()` calls. The benchmark still compares the same supplied source across modes, but it should not be treated as a clean deterministic-composition fixture.
2026-04-30 06:46:14 +02:00

334 lines
11 KiB
TypeScript

/**
* Browser Manager
*
* Manages Puppeteer browser lifecycle: Chrome executable resolution,
* launch args, pooled browser acquisition/release.
*/
import type { Browser, PuppeteerNode } from "puppeteer-core";
import { existsSync, readdirSync } from "fs";
import { join } from "path";
import { homedir } from "os";
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
let _puppeteer: PuppeteerNode | undefined;
async function getPuppeteer(): Promise<PuppeteerNode> {
if (_puppeteer) return _puppeteer;
try {
const mod = await import("puppeteer" as string);
_puppeteer = mod.default;
} catch {
const mod = await import("puppeteer-core");
_puppeteer = mod.default;
}
if (!_puppeteer) throw new Error("Neither puppeteer nor puppeteer-core found");
return _puppeteer;
}
// "beginframe" = atomic compositor control via HeadlessExperimental.beginFrame (Linux only)
// "screenshot" = renderSeek + Page.captureScreenshot (all platforms)
export type CaptureMode = "beginframe" | "screenshot";
export interface AcquiredBrowser {
browser: Browser;
captureMode: CaptureMode;
}
/**
* Resolve chrome-headless-shell binary for deterministic BeginFrame rendering.
* Checks config.chromePath, then PRODUCER_HEADLESS_SHELL_PATH env var,
* then scans Puppeteer's managed cache at ~/.cache/puppeteer/chrome-headless-shell/.
*/
export function resolveHeadlessShellPath(
config?: Partial<Pick<EngineConfig, "chromePath">>,
): string | undefined {
if (config?.chromePath) {
return config.chromePath;
}
if (process.env.PRODUCER_HEADLESS_SHELL_PATH) {
return process.env.PRODUCER_HEADLESS_SHELL_PATH;
}
const baseDir = join(homedir(), ".cache", "puppeteer", "chrome-headless-shell");
if (!existsSync(baseDir)) return undefined;
try {
const versions = readdirSync(baseDir).sort().reverse(); // newest first
for (const version of versions) {
const candidates = [
join(baseDir, version, "chrome-headless-shell-linux64", "chrome-headless-shell"),
join(baseDir, version, "chrome-headless-shell-mac-arm64", "chrome-headless-shell"),
join(baseDir, version, "chrome-headless-shell-mac-x64", "chrome-headless-shell"),
join(baseDir, version, "chrome-headless-shell-win64", "chrome-headless-shell.exe"),
];
for (const binary of candidates) {
if (existsSync(binary)) return binary;
}
}
} catch {
// ignore
}
return undefined;
}
let pooledBrowser: Browser | null = null;
let pooledBrowserRefCount = 0;
let pooledCaptureMode: CaptureMode = "screenshot";
// Preserve the producer-era export so re-export shims keep the same public API.
export const ENABLE_BROWSER_POOL = DEFAULT_CONFIG.enableBrowserPool;
// Flags only meaningful when Chrome's compositor is driven by
// HeadlessExperimental.beginFrame. If we fall back to screenshot mode they
// must be stripped — `--enable-begin-frame-control` in particular makes the
// compositor wait for frames we'll never send, producing blank screenshots.
const BEGINFRAME_ONLY_FLAGS = new Set([
"--deterministic-mode",
"--enable-begin-frame-control",
"--disable-new-content-rendering-timeout",
"--run-all-compositor-stages-before-draw",
"--disable-threaded-animation",
"--disable-threaded-scrolling",
"--disable-checker-imaging",
"--disable-image-animation-resync",
"--enable-surface-synchronization",
]);
function stripBeginFrameFlags(args: string[]): string[] {
return args.filter((a) => !BEGINFRAME_ONLY_FLAGS.has(a));
}
/**
* Probe whether the browser still speaks HeadlessExperimental.beginFrame.
*
* Recent chrome-headless-shell builds (observed on 147) expose the domain
* well enough that HeadlessExperimental.enable succeeds but drop the
* beginFrame method itself — the capture loop then dies on first frame with
* `'HeadlessExperimental.beginFrame' wasn't found`. So we probe BOTH: enable
* + one cheap beginFrame raced against a 2s timeout. In beginframe-control
* mode the command completes as soon as the compositor acks, so a real
* supported browser returns well under the timeout.
*
* Any failure (method missing, timeout, protocol error) is treated as
* unsupported. Real errors after launch would surface in the warmup loop and
* fall out through the caller's try/catch.
*/
async function probeBeginFrameSupport(browser: Browser): Promise<boolean> {
let page;
try {
page = await browser.newPage();
const client = await page.createCDPSession();
await client.send("HeadlessExperimental.enable");
const beginFrame = client.send("HeadlessExperimental.beginFrame", {
frameTimeTicks: 0,
interval: 33,
noDisplayUpdates: true,
});
const timeout = new Promise<never>((_, reject) =>
setTimeout(() => reject(new Error("beginFrame probe timeout")), 2000),
);
await Promise.race([beginFrame, timeout]);
await client.detach().catch(() => {});
return true;
} catch {
return false;
} finally {
await page?.close().catch(() => {});
}
}
export async function acquireBrowser(
chromeArgs: string[],
config?: Partial<
Pick<
EngineConfig,
"browserTimeout" | "protocolTimeout" | "enableBrowserPool" | "chromePath" | "forceScreenshot"
>
>,
): Promise<AcquiredBrowser> {
const enablePool = config?.enableBrowserPool ?? DEFAULT_CONFIG.enableBrowserPool;
if (enablePool && pooledBrowser) {
pooledBrowserRefCount += 1;
return { browser: pooledBrowser, captureMode: pooledCaptureMode };
}
// Config chromePath overrides env var / auto-detection.
const headlessShell = resolveHeadlessShellPath(config);
// BeginFrame requires chrome-headless-shell AND Linux (crashes on macOS/Windows).
const isLinux = process.platform === "linux";
const forceScreenshot = config?.forceScreenshot ?? DEFAULT_CONFIG.forceScreenshot;
let captureMode: CaptureMode;
let executablePath: string | undefined;
if (headlessShell && isLinux && !forceScreenshot) {
captureMode = "beginframe";
executablePath = headlessShell;
} else {
// Screenshot mode with renderSeek: works on all platforms.
captureMode = "screenshot";
executablePath = headlessShell ?? undefined;
}
const ppt = await getPuppeteer();
const browserTimeout = config?.browserTimeout ?? DEFAULT_CONFIG.browserTimeout;
const protocolTimeout = config?.protocolTimeout ?? DEFAULT_CONFIG.protocolTimeout;
let browser = await ppt.launch({
headless: true,
args: chromeArgs,
defaultViewport: null,
executablePath,
timeout: browserTimeout,
protocolTimeout,
});
// Probe HeadlessExperimental.beginFrame — recent chrome-headless-shell
// builds (observed on 147) dropped the method while keeping the flags
// valid, so `--enable-begin-frame-control` leaves the compositor waiting
// for beginFrames the engine can no longer send. Auto-fall back to
// screenshot mode with the appropriate flags.
if (captureMode === "beginframe") {
const supported = await probeBeginFrameSupport(browser).catch(() => true);
if (!supported) {
await browser.close().catch(() => {});
console.warn(
"[BrowserManager] HeadlessExperimental.beginFrame unavailable in this Chromium build; falling back to screenshot mode.",
);
captureMode = "screenshot";
browser = await ppt.launch({
headless: true,
args: stripBeginFrameFlags(chromeArgs),
defaultViewport: null,
executablePath,
timeout: browserTimeout,
protocolTimeout,
});
}
}
if (enablePool) {
pooledBrowser = browser;
pooledBrowserRefCount = 1;
pooledCaptureMode = captureMode;
}
return { browser, captureMode };
}
export async function releaseBrowser(
browser: Browser,
config?: Partial<Pick<EngineConfig, "enableBrowserPool">>,
): Promise<void> {
const enablePool = config?.enableBrowserPool ?? DEFAULT_CONFIG.enableBrowserPool;
if (!enablePool) {
await browser.close().catch(() => {});
return;
}
if (pooledBrowser && pooledBrowser === browser) {
pooledBrowserRefCount = Math.max(0, pooledBrowserRefCount - 1);
if (pooledBrowserRefCount === 0) {
await browser.close().catch(() => {});
pooledBrowser = null;
}
return;
}
await browser.close().catch(() => {});
}
export interface BuildChromeArgsOptions {
width: number;
height: number;
captureMode?: CaptureMode;
platform?: NodeJS.Platform;
}
export function buildChromeArgs(
options: BuildChromeArgsOptions,
config?: Partial<Pick<EngineConfig, "browserGpuMode" | "disableGpu" | "chromePath">>,
): string[] {
const platform = options.platform ?? process.platform;
const gpuDisabled = config?.disableGpu ?? DEFAULT_CONFIG.disableGpu;
const browserGpuMode = gpuDisabled
? "software"
: (config?.browserGpuMode ?? DEFAULT_CONFIG.browserGpuMode);
// Chrome flags tuned for headless rendering performance. The set below is a
// fairly standard "headless-for-capture" configuration — similar profiles
// appear in Puppeteer's defaults, Playwright, Remotion, and Chrome's own
// headless-shell guidance.
const chromeArgs = [
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-dev-shm-usage",
"--enable-webgl",
"--ignore-gpu-blocklist",
...getBrowserGpuArgs(browserGpuMode, platform),
"--font-render-hinting=none",
"--force-color-profile=srgb",
`--window-size=${options.width},${options.height}`,
// Prevent Chrome from throttling background tabs/timers — critical when the
// page is offscreen during headless capture
"--disable-background-timer-throttling",
"--disable-backgrounding-occluded-windows",
"--disable-renderer-backgrounding",
"--disable-background-media-suspend",
// Reduce overhead from unused Chrome features
"--disable-breakpad",
"--disable-component-extensions-with-background-pages",
"--disable-default-apps",
"--disable-extensions",
"--disable-hang-monitor",
"--disable-ipc-flooding-protection",
"--disable-popup-blocking",
"--disable-sync",
"--disable-component-update",
"--disable-domain-reliability",
"--disable-print-preview",
"--no-pings",
"--no-zygote",
// Memory
"--force-gpu-mem-available-mb=4096",
"--disk-cache-size=268435456",
// Disable features that add overhead
"--disable-features=AudioServiceOutOfProcess,IsolateOrigins,site-per-process,Translate,BackForwardCache,IntensiveWakeUpThrottling",
];
// BeginFrame flags — only when using chrome-headless-shell on Linux
if (options.captureMode !== "screenshot") {
chromeArgs.push(
"--deterministic-mode",
"--enable-begin-frame-control",
"--disable-new-content-rendering-timeout",
"--run-all-compositor-stages-before-draw",
"--disable-threaded-animation",
"--disable-threaded-scrolling",
"--disable-checker-imaging",
"--disable-image-animation-resync",
"--enable-surface-synchronization",
);
}
if (gpuDisabled) {
chromeArgs.push("--disable-gpu");
}
return chromeArgs;
}
function getBrowserGpuArgs(
mode: EngineConfig["browserGpuMode"],
platform: NodeJS.Platform,
): string[] {
if (mode === "software") {
return ["--use-gl=angle", "--use-angle=swiftshader"];
}
switch (platform) {
case "darwin":
return ["--use-gl=angle", "--use-angle=metal", "--enable-gpu-rasterization"];
case "win32":
return ["--use-gl=angle", "--use-angle=d3d11", "--enable-gpu-rasterization"];
case "linux":
return ["--use-gl=egl", "--enable-gpu-rasterization"];
default:
return ["--enable-gpu-rasterization"];
}
}