5 Commits

Include renames

Author	SHA1	Message	Date
James	67bb56c703	feat(engine): browserGpuMode "auto" — probe WebGL once, fall back to software ... When the host doesn't have a usable GPU (CI containers, eval rigs without GPU passthrough, dev VMs), Chrome's hardware-mode WebGL flags (`--use-gl=egl/metal/d3d11`) silently leave WebGL unavailable — `getContext("webgl")` returns null, three.js' WebGLRenderer dies, the canvas stays black. Surfaced today by Abhay's c2v-eval failing on a docker render of an hf bundle that uses three.js + a custom fragment shader. The fix that's been there: `--use-gl=angle --use-angle=swiftshader` (CPU software WebGL, ~5-50× slower but pixel-identical). The engine already exposed `browserGpuMode: "software"` for this. The gap was discovery — users had to know to pass `--no-browser-gpu` on no-GPU hosts. This change adds `browserGpuMode: "auto"` (now the CLI default for local renders): on first launch in the process, probe Chrome with hardware args, check `canvas.getContext("webgl") !== null`, cache the result. ~1-2 s on first render, free on every subsequent render in the same worker. Hardware GPUs keep their fast path; no-GPU hosts get SwiftShader without ceremony. Behaviour matrix: - No flag, no env, local → "auto" (NEW default) - `--browser-gpu` → "hardware" (force; errors if no GPU) - `--no-browser-gpu` → "software" (force SwiftShader) - `PRODUCER_BROWSER_GPU_MODE` → "hardware" / "software" / "auto" / unset - Docker mode → forced "software" (unchanged) Engine-config default stays "software" (conservative for embedders); the "auto" default lives in the CLI's `resolveBrowserGpuForCli` so producer embedders aren't surprised by a probe-on-launch. Also adds `--enable-unsafe-swiftshader` to the software flag set — Chrome 120+ deprecated implicit SwiftShader fallback and emits a deprecation warning unless the flag is set explicitly. Despite the "unsafe" name this is exactly the pre-deprecation behaviour; the rename is about Chrome's threat model on the open web, not about the rendering itself. Verification: - Engine 535/535 + CLI 256/256 (incl. new probe tests + tri-state CLI test) - Empirical: probe on this no-GPU devbox returns "software" in 240 ms, cached 0 ms on subsequent calls - Format / lint / typecheck clean across all packages Refs the Abhay/Slack thread on c2v-eval rendering without a GPU node.	2026-05-06 17:33:55 +00:00
Miguel Ángel	dde26cf62d	feat: default streaming encode for sequential renders (#579)	2026-05-01 04:22:26 +02:00
Miguel Ángel	395fb9c084	feat: add browser GPU render mode (#571) ... ## Problem HyperFrames already had `--gpu`, but that flag only controlled FFmpeg hardware encoding. The browser capture path still forced Chrome/WebGL through SwiftShader software GL via `--use-angle=swiftshader`, so WebGL-heavy local renders could leave the biggest bottleneck on the CPU path. That made the existing flag naming easy to misread: `--gpu` sounded like it accelerated the whole render, but it did not change the browser frame-capture backend. ## What this fixes - Enables host browser GPU acceleration automatically for local CLI renders. - Adds `--no-browser-gpu` as the local opt-out for software Chrome/WebGL capture. - Keeps `--browser-gpu` as an explicit local browser-GPU request. - Adds `browserGpuMode: "software" | "hardware"` to engine config, with `PRODUCER_BROWSER_GPU_MODE` env support for lower-level producer users. - Keeps Docker browser capture on the deterministic software path. - Maps hardware browser GPU mode to platform-native Chrome backends: - macOS: Metal-backed ANGLE - Windows: D3D11-backed ANGLE - Linux: EGL - Blocks explicit `--browser-gpu --docker` with a clear error because Docker browser GPU passthrough is not cross-platform. - Clarifies docs so `--gpu` means FFmpeg encoder GPU and browser GPU means Chrome/WebGL capture GPU. - Keeps encoder backend selection auto-detected from FFmpeg capabilities: - NVIDIA: NVENC - macOS: VideoToolbox - Linux: VAAPI - Intel: QSV ## Why two flags There are two separate GPU surfaces in the render pipeline: 1. Browser GPU controls Chrome frame capture. - Affects WebGL, canvas, CSS rendering, compositing, and screenshot capture inside the browser. - This is enabled automatically for local CLI renders. - Use `--no-browser-gpu` when you want the software browser baseline. 2. `--gpu` controls FFmpeg video encoding. - Affects the final encode step after frames have already been captured. - The concrete encoder is auto-detected from the host FFmpeg build and hardware. - It can be faster for some machines/codecs, but it is not equivalent to browser rendering acceleration. The controls stay independent because users may want: - `hyperframes render` for the fast local default with browser GPU capture. - `hyperframes render --no-browser-gpu` for the software-browser local baseline. - `hyperframes render --gpu` for browser GPU capture plus hardware FFmpeg encoding. - `hyperframes render --no-browser-gpu --gpu` for software browser capture plus hardware FFmpeg encoding. - `hyperframes render --docker` for deterministic browser capture. ## Why `--gpu` does not imply browser GPU Keeping `--gpu` scoped to FFmpeg encoding avoids a semantic break and keeps the risk profile explicit: - `--gpu` already means encoder acceleration. Expanding it to also change Chrome capture would silently alter behavior for users who only wanted hardware encoding. - Browser GPU and encoder GPU have different portability. Encoder GPU can work in Docker when the host exposes the right devices; browser GPU passthrough is not cross-platform, so this PR intentionally blocks explicit `--browser-gpu --docker`. - The Apple presentation benchmark shows why the controls should stay separate: browser GPU capture was the useful improvement, while macOS VideoToolbox via `--gpu` was slower and produced larger output for this `standard` H.264 run. If HyperFrames later wants a single umbrella acceleration control, it should be explicit, for example `--acceleration browser|encoder|all` or `--gpu=browser|encoder|all`, rather than changing the meaning of the existing boolean `--gpu`. ## Root cause `buildChromeArgs()` always injected `--use-gl=angle --use-angle=swiftshader`. `disableGpu` only appended `--disable-gpu`; it did not provide a hardware-GPU mode. That made the public `--gpu` flag look broader than it was, because render capture stayed software-backed even when encoder GPU was requested. ## Verification ### Local checks - `bun install` - `bun run build:hyperframes-runtime` - `bun run --filter @hyperframes/engine test src/config.test.ts src/services/browserManager.test.ts` - `bun run --filter @hyperframes/cli test src/utils/dockerRunArgs.test.ts src/commands/render.test.ts` - `bun run --filter @hyperframes/cli typecheck` - `bun run --filter @hyperframes/engine typecheck` - `bun run --filter @hyperframes/producer typecheck` - `cd packages/producer && bunx vitest run src/services/renderOrchestrator.test.ts` - `bunx oxlint packages/cli/src/commands/render.ts packages/cli/src/commands/render.test.ts packages/cli/src/utils/dockerRunArgs.ts packages/cli/src/utils/dockerRunArgs.test.ts packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/browserManager.ts packages/engine/src/services/browserManager.test.ts packages/producer/src/services/renderOrchestrator.test.ts` - `bunx oxfmt --check ...` on changed source/docs files - `git diff --check` - `bun packages/cli/src/cli.ts render --help | rg -n "browser-gpu|no-browser-gpu|GPU"` - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --output /tmp/hf-auto-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict` - Render plan prints `GPU: browser GPU (auto)`. - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --no-browser-gpu --output /tmp/hf-software-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict` - Render plan does not print browser GPU. - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --docker --browser-gpu --output /tmp/should-not-render.mp4` - Exits 1 with `Browser GPU is local-only`. - `buildDockerRunArgs()` regression coverage asserts Docker container args include `--no-browser-gpu`, preventing nested container renders from re-enabling browser GPU through the local CLI default. - `resolveBrowserGpuForCli()` regression coverage asserts `PRODUCER_BROWSER_GPU_MODE=software` opts out when no CLI browser-GPU flag is supplied, while explicit `--browser-gpu` / `--no-browser-gpu` still win. - `ffmpeg -v error -i /tmp/hf-auto-browser-gpu-smoke.mp4 -f null -` - `ffmpeg -v error -i /tmp/hf-software-browser-gpu-smoke.mp4 -f null -` - `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-browser-gpu-smoke.mp4` -> H.264, 1920x1080, 24fps, 5.0s ### Apple presentation benchmark Rendered `/Users/miguel07code/Downloads/apple-presentation.zip` as supplied after extracting to `/tmp/hf-apple-profile/apple-presentation`. Fixed settings: - 1920x1080 - 30fps - `standard` quality - 4240 frames - 141.32s duration - 8-worker cap; render auto-calibration used 6 capture workers - macOS host detected FFmpeg GPU encoder: `videotoolbox` | Mode | Equivalent flags after this PR | Wall time | vs software-browser baseline | Speed | Capture | Encode | Output | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | Software browser + CPU encode | `--no-browser-gpu` | 120.77s | baseline | 1.17x | 97.87s | 10.04s | 8.38MB | | Browser GPU + CPU encode | default local render | 70.10s | 42.0% faster | 2.02x | 50.72s | 9.91s | 8.39MB | | Software browser + encoder GPU | `--no-browser-gpu --gpu` | 133.16s | 10.3% slower | 1.06x | 103.58s | 18.31s | 25.43MB | | Browser GPU + encoder GPU | `--gpu` | 74.12s | 38.6% faster | 1.91x | 46.69s | 17.93s | 25.45MB | Result: browser GPU capture is the meaningful improvement for this WebGL/browser-capture-heavy presentation. VideoToolbox encoding was slower and produced larger files for this current `standard` H.264 path, so `--gpu` should stay separate and opt-in. Why `--gpu` plus browser GPU was slower than browser GPU alone: the combined run captured about 4.0s faster than browser GPU alone, but VideoToolbox encoding was about 8.0s slower than CPU x264 encoding, so the encode loss outweighed the capture gain. ### VideoToolbox flag check I also isolated the encode stage against the already-captured Apple frames to check whether macOS GPU encoding only needed special flags. `ffmpeg -h encoder=h264_videotoolbox` does not expose a CRF/CQ-style quality option like x264. It exposes bitrate-oriented and VideoToolbox-specific options such as `-b:v`, `-realtime`, `-profile`, `-coder`, `-prio_speed`, `-power_efficient`, and `-allow_sw`. That means our current `-q:v` mapping is not equivalent to x264 CRF and can produce very different bitrate/size behavior. Measured full-frame encode variants on this host: | VideoToolbox variant | Encode wall time | Output size | Bitrate | | --- | ---: | ---: | ---: | | Current `-q:v 64 -allow_sw 1` | 18.76s | 25.31MB | 1.43 Mbps | | Current without `-allow_sw 1` | 18.21s | 25.31MB | 1.43 Mbps | | `-b:v 500k -maxrate 750k -bufsize 1000k -profile high -coder cabac -realtime 1 -prio_speed 1 -power_efficient 0` | 20.58s | 7.42MB | 0.42 Mbps | | Same with `-b:v 1500k` | 20.84s | 16.70MB | 0.95 Mbps | | `-b:v 500k -profile baseline -coder cavlc -realtime 1 -prio_speed 1 -power_efficient 0` | 18.11s | 8.94MB | 0.51 Mbps | Conclusion: VideoToolbox can be made size/bitrate-predictable with explicit `--video-bitrate`, but the tested speed-oriented flags did not make it faster than CPU x264 wall time for this render. That reinforces keeping `--gpu` encoder acceleration explicit and separate from browser GPU capture. Artifacts from the local benchmark: - `/tmp/hf-apple-profile/results/cpu.mp4` - `/tmp/hf-apple-profile/results/browser-gpu.mp4` - `/tmp/hf-apple-profile/results/encoder-gpu.mp4` - `/tmp/hf-apple-profile/results/full-gpu.mp4` - `/tmp/hf-apple-profile/results/summary.json` All four benchmark MP4s completed `ffprobe` and full `ffmpeg -f null` decode checks. ### Pixel comparison Compared decoded MP4 output between software-browser and browser-GPU renders: - Apple presentation: - 4240 frames compared - 636 exact matching decoded frame hashes - 3604 different decoded frame hashes - Average PSNR: 57.79 dB - `css-spinner-render-compat` clean fixture: - 120 frames compared - 0 exact matching decoded frame hashes - Average PSNR: 61.57 dB Interpretation: browser GPU output is not strict hash/pixel-identical to the software-browser path after lossy H.264 encode, but the measured deltas are visually tiny. Above 50 dB PSNR is typically visually indistinguishable for normal video review. Use `--no-browser-gpu` or Docker when strict cross-run/cross-machine reproducibility matters more than local speed. ### Browser verification - Started HyperFrames Studio preview for `packages/producer/tests/css-spinner-render-compat/src`. - Used `agent-browser` to open `http://localhost:5191#project/src` and verify the composition loaded in Studio. - Screenshots: - `/tmp/hf-gpu-browser-proof/preview-loaded.png` - `/tmp/hf-gpu-browser-proof/preview-playing.png` - `/tmp/hf-gpu-browser-proof/preview-frame-60.png` - Agent-browser recordings: - `/tmp/hf-gpu-browser-proof/preview-playback.webm` - `/tmp/hf-gpu-browser-proof/preview-seek.webm` ## Notes - Browser GPU is enabled automatically for local CLI renders and disabled in Docker. - `--no-browser-gpu` is the opt-out for software Chrome/WebGL capture. - `--gpu` remains encoder-only and opt-in. - The Apple presentation zip has existing lint errors around unmanaged nested videos and imperative media `play()` calls. The benchmark still compares the same supplied source across modes, but it should not be treated as a clean deterministic-composition fixture.	2026-04-30 06:46:14 +02:00
Miguel Ángel	b98093aa1c	fix: remove hidden audio gain in renders (#362) ... ## Summary This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources. It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation. Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first. ## Root Cause The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior: - `packages/engine/src/config.ts` defaulted `audioGain` to `1.35` - `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter: - `[mixed]volume=${masterOutputGain}[out]` - with the default config, that meant every render got multiplied by `1.35` That exactly matches the issue reporter's measured scalar boost. While investigating the workaround, I also found a second correctness bug: - `processCompositionAudio()` used `element.volume || 1.0` - that coerced `0` to `1.0` - so `data-volume="0"` did not actually mute the track in rendered output Separately, the repo-level Studio typecheck could fail before any build step because: - `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player` - `packages/player/package.json` points TypeScript at built `dist/*` outputs - in a fresh workspace, those built outputs may not exist yet - Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck ## What Changed 1. Set the engine default `audioGain` back to unity (`1`) 2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0` 3. Added regression coverage for both behaviors 4. Updated the producer-side config fixture to reflect the corrected default 5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build ## Why These Changes Are Needed This is not a UX preference issue; it is a correctness and API contract issue. - The docs describe `data-volume` as a direct 0-1 control. - Rendered output should preserve source levels unless the author explicitly changes them. - Hidden global gain makes output non-deterministic from the author's perspective. - `data-volume="0"` must mean silence, not full-volume playback. - Local workspace typecheck should not require unrelated package build artifacts to exist first. Leaving the current behavior in place means: - voice recordings near normal peak levels can clip during render - authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output - mute semantics in docs and code diverge - local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed ## Testing ### Focused regression tests Ran: - `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts` Result: - `10 passed` These tests specifically verify: - default resolved `audioGain` is `1` - a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph - the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`) ### Broader package verification Ran: - `bun run --filter @hyperframes/engine test` - `bun run --filter @hyperframes/engine build` - `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts` - `bun run --filter @hyperframes/producer typecheck` - `bun run --filter @hyperframes/studio typecheck` - `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts` - `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json` - `bunx lefthook run pre-commit` Results: - full engine test suite passed (`309 passed`) - engine build passed - touched producer test file passed (`7 passed`) - producer typecheck passed - studio typecheck passed - oxlint passed with `0 warnings, 0 errors` - formatting passed - pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker ## Known Verification Limitation There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow. ## User Impact After this change: - rendered audio matches source level by default - authors no longer need to compensate for a hidden +2.6 dB boost - `data-volume="0"` correctly mutes rendered audio - the documented volume contract matches engine behavior again - local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts Closes #361.	2026-04-21 20:33:25 +02:00
Vance IngallsandClaude Opus 4.6	9f8e5ba5a1	initial code (#2) ... * feat: initial code port from hyperframes-internal Port all OSS-ready packages from the internal monorepo: - @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime - @hyperframes/cli — CLI for creating, previewing, and rendering compositions - @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg) - @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg) - @hyperframes/ui-player — browser-based video player component - @hyperframes/studio — composition editor (React frontend + Hono backend) Includes regression test suite with Docker-based test harness. All HeyGen-internal references, deployment infrastructure, and proprietary assets have been removed. Package names migrated from @app/* to @hyperframes/*. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scrub internal codenames and stale references from OSS port - Replace static.heygen.ai runtime URLs in test fixtures - Remove internal CDN publish script (publish-hyperframe-runtime.ts) - Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime with neutral names (studio, hyperframe-runtime, __hyperframeRuntime) - Fix stale Vault API / localhost references in docs - Remove broken deprecated_studio link Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining internal codenames and stale references - Delete stale producer README.md and PIPELINE.md (referenced nonexistent files) - Replace "Cerberus" codename with "HyperFrames" in test design reviews - Replace magic-edit postMessage identifiers with hf-preview/hf-parent - Rename debug-magic-edit-timeline.ts to debug-timeline.ts - Replace "Motion Cut" with "HyperFrames" in Timeline comments - Fix studio/CLI references to nonexistent archive package (use local data/projects/ dir, stub render proxy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-21 22:43:56 -07:00