## Summary First slice of `P0-1` from the player perf proposal: lays the foundation for a player perf gate so later PRs can plug in fps / scrub / drift / parity scenarios without rebuilding infrastructure. Ships one smoke scenario (`03-load`, cold + warm composition load) to prove the gate end-to-end on real numbers. ## Why There was no automated way to catch player perf regressions. Every perf concern in the existing proposal — composition load time, sustained FPS, scrub p95, mirror-clock drift, live-vs-seek parity — needs the same plumbing: a same-origin harness, a Puppeteer runner, a baseline file, a gate that emits structured results, and a CI workflow that runs the right scenarios on the right changes. Building that up-front in one reviewable PR lets every subsequent perf PR (`P0-1b`, `P0-1c`, and beyond) be a 100-line scenario file plus a baseline entry instead of re-litigating the framework. ## What changed ### Harness — `packages/player/tests/perf/server.ts` - `Bun.serve` on a free port, single same-origin host for the player IIFE bundle, hyperframe runtime, GSAP from `node_modules`, and fixture HTML. - Same-origin matters: cross-origin would force every probe through `postMessage`, hiding bugs and inflating numbers in ways production never sees. Tests should measure the path the studio editor actually takes. - Routes: - `/player.js` → built IIFE bundle (rebuilt on demand). - `/vendor/runtime.js`, `/vendor/gsap.min.js` → resolved from `node_modules` so fixtures don't need to ship copies. - `/fixtures/*` → fixture HTML. ### Runner — `packages/player/tests/perf/runner.ts` - `puppeteer-core` thin wrappers (`launchBrowser`, `loadHostPage`). - Uses the system Chrome detected by `setup-chrome` in CI rather than the bundled puppeteer revision — keeps the action smaller, lets us pin Chrome version policy at the workflow level, and matches what users actually run. ### Gate — `packages/player/tests/perf/perf-gate.ts` + `baseline.json` - Loads `baseline.json` (initial budgets: cold/warm comp load, fps, scrub p95 isolated/inline, drift max/p95) with a 10% `allowedRegressionRatio`. - Per-metric direction (`lower-is-better` / `higher-is-better`) so the same evaluator handles latency and throughput. - Returns a structured `GateReport` consumed by both the CLI (table output) and `metrics.json` (CI artifact). - Two modes: `measure` (log only — used during the rollout) and `enforce` (fail the build) — flip per-metric once we trust the signal, without touching the harness. ### CLI orchestrator — `packages/player/tests/perf/index.ts` - Parses `--mode` / `--scenarios` / `--runs` / `--fixture` in both space- and equals-separated form (so `--scenarios fps,scrub` and `--scenarios=fps,scrub` both work — matches what humans type and what GitHub Actions emits). - Runs scenarios, runs the gate, and **always** writes `results/metrics.json` with schema version, git SHA, metrics, and gate rows — so failed runs are still investigable from the artifact alone. ### Fixture + smoke scenario - `fixtures/gsap-heavy/index.html`: 200 stagger-animated tiles, no media. Heavy enough to make load time meaningful, light enough to be deterministic. - `scenarios/03-load.ts`: cold + warm composition load. Measures from navigation start to player `ready` event, reports p95 across runs. ### CI — `.github/workflows/player-perf.yml` - `paths-filter` on `player` / `core` / `runtime` — perf only runs when something that could move the needle actually changed. - Sets up bun + node + chrome, runs perf in `measure` mode on a shard matrix (so future scenarios shard naturally), uploads `metrics.json` artifacts, and a summary job aggregates shard results into a single PR comment. ### Wiring - `packages/player`: `puppeteer-core`, `gsap`, `@types/bun` devDeps; typecheck extended to cover the perf `tsconfig`; new `perf` script. - Root `package.json`: `player:perf` workspace script so `bun run player:perf` runs the whole suite locally with the same flags CI uses. - `.gitignore`: `packages/player/tests/perf/results/`. - Separate `tests/perf/tsconfig.json` so test code doesn't pollute the package `rootDir` while still being typechecked. ## Test plan - [x] Local: `bun run player:perf` passes — cold p95 ≈ 386 ms, warm p95 ≈ 375 ms, both well under the seeded baselines. - [x] Typecheck, lint, format pass on the perf workspace. - [x] Existing player unit tests (71/71) still green. - [ ] First CI run after merge will be the real signal: confirms `setup-chrome` works on hosted runners, the shard matrix wires up, and `metrics.json` artifacts upload. ## Stack Step `P0-1a` of the player perf proposal. The next two slices are content-only — they don't touch the harness: - `P0-1b` (#400): adds `02-fps`, `04-scrub`, `05-drift` scenarios on a 10-video-grid fixture. - `P0-1c` (#401): adds `06-parity` (live playback vs. synchronously-seeked reference, compared via SSIM). Wiring this gate up first means each follow-up is a self-contained scenario file + baseline row + workflow shard.
Write HTML. Render video. Built for agents.
Hyperframes is an open-source video rendering framework that lets you create, preview, and render HTML-based video compositions — with first-class support for AI agents.
Quick Start
Option 1: With an AI coding agent (recommended)
Install the HyperFrames skills, then describe the video you want:
npx skills add heygen-com/hyperframes
This teaches your agent (Claude Code, Cursor, Gemini CLI, Codex) how to write correct compositions and GSAP animations. In Claude Code, the skills register as slash commands — invoke /hyperframes to author compositions, /hyperframes-cli for CLI commands, and /gsap for animation help.
For Codex specifically, the same skills are also exposed as an OpenAI Codex plugin — sparse-install just the plugin surface:
codex plugin marketplace add heygen-com/hyperframes --sparse .codex-plugin --sparse skills --sparse assets
Try it: example prompts
Copy any of these into your agent to get started. The /hyperframes prefix loads the skill context explicitly so you get correct output the first time.
Cold start — describe what you want:
Using
/hyperframes, create a 10-second product intro with a fade-in title, a background video, and background music.
Warm start — turn existing context into a video:
Take a look at this GitHub repo https://github.com/heygen-com/hyperframes and explain its uses and architecture to me using
/hyperframes.
Summarize the attached PDF into a 45-second pitch video using
/hyperframes.
Turn this CSV into an animated bar chart race using
/hyperframes.
Format-specific:
Make a 9:16 TikTok-style hook video about [topic] using
/hyperframes, with bouncy captions synced to a TTS narration.
Iterate — talk to the agent like a video editor:
Make the title 2x bigger, swap to dark mode, and add a fade-out at the end.
Add a lower third at 0:03 with my name and title.
The agent handles scaffolding, animation, and rendering. See the prompting guide for more patterns.
Option 2: Start a project manually
npx hyperframes init my-video
cd my-video
npx hyperframes preview # preview in browser (live reload)
npx hyperframes render # render to MP4
hyperframes init installs skills automatically, so you can hand off to your AI agent at any point.
Requirements: Node.js >= 22, FFmpeg
Why Hyperframes?
- HTML-native — compositions are HTML files with data attributes. No React, no proprietary DSL.
- AI-first — agents already speak HTML. The CLI is non-interactive by default, designed for agent-driven workflows.
- Deterministic rendering — same input = identical output. Built for automated pipelines.
- Frame Adapter pattern — bring your own animation runtime (GSAP, Lottie, CSS, Three.js).
Hyperframes vs Remotion
Hyperframes is inspired by Remotion — we used Remotion at HeyGen in production, learned a ton from it, and kept attribution comments in the source for the patterns it pioneered (Chrome launch flags, image2pipe → FFmpeg streaming, frame buffering). Both tools drive headless Chrome and both are deterministic. They differ on one decision: what the primary author writes. Remotion's bet is React components; Hyperframes' bet is HTML.
| Hyperframes | Remotion | |
|---|---|---|
| Authoring | HTML + CSS + GSAP | React components (TSX) |
| Build step | None; index.html plays as-is |
Required (bundler) |
| Library-clock animations (GSAP, Anime.js, Motion One) | Seekable, frame-accurate | Plays at wall-clock during render |
| Arbitrary HTML / CSS passthrough | Paste and animate | Rewrite as JSX |
| Distributed rendering | Single-machine today | Lambda, production-ready |
Licensing: fully open source vs source-available
Hyperframes is completely open source under Apache 2.0 — an OSI-approved license. Use it commercially at any scale, with no per-render fees, no seat caps, no company-size thresholds.
Remotion is source-available, not open source. The code is on GitHub under a custom Remotion License that requires a paid company license above small-team thresholds. It's a great product with a real team behind it — but if open-source licensing matters to you (OSI compliance, redistribution rights, no per-use fees), that's a first-order decision point.
Full write-up with benchmarks, an honest list of where each tool wins, and a GSAP side-by-side: Hyperframes vs Remotion guide.
How It Works
Define your video as HTML with data attributes:
<div id="stage" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
<video
id="clip-1"
data-start="0"
data-duration="5"
data-track-index="0"
src="intro.mp4"
muted
playsinline
></video>
<img
id="overlay"
class="clip"
data-start="2"
data-duration="3"
data-track-index="1"
src="logo.png"
/>
<audio
id="bg-music"
data-start="0"
data-duration="9"
data-track-index="2"
data-volume="0.5"
src="music.wav"
></audio>
</div>
Preview instantly in the browser. Render to MP4 locally or in Docker.
Catalog
50+ ready-to-use blocks and components — social overlays, shader transitions, data visualizations, and cinematic effects:
npx hyperframes add flash-through-white # shader transition
npx hyperframes add instagram-follow # social overlay
npx hyperframes add data-chart # animated chart
Browse the full catalog at hyperframes.heygen.com/catalog.
Documentation
Full documentation at hyperframes.heygen.com/introduction — Quickstart | Guides | API Reference | Catalog
Packages
| Package | Description |
|---|---|
hyperframes |
CLI — create, preview, lint, and render compositions |
@hyperframes/core |
Types, parsers, generators, linter, runtime, frame adapters |
@hyperframes/engine |
Seekable page-to-video capture engine (Puppeteer + FFmpeg) |
@hyperframes/producer |
Full rendering pipeline (capture + encode + audio mix) |
@hyperframes/studio |
Browser-based composition editor UI |
@hyperframes/player |
Embeddable <hyperframes-player> web component |
@hyperframes/shader-transitions |
WebGL shader transitions for compositions |
Skills
HyperFrames ships skills that teach AI agents framework-specific patterns that generic docs don't cover.
npx skills add heygen-com/hyperframes
| Skill | What it teaches |
|---|---|
hyperframes |
HTML composition authoring, captions, TTS, audio-reactive animation, transitions |
hyperframes-cli |
CLI commands: init, lint, preview, render, transcribe, tts, doctor |
hyperframes-registry |
Block and component installation via hyperframes add |
website-to-hyperframes |
Capture a URL and turn it into a video — full website-to-video pipeline |
gsap |
GSAP animation API, timelines, easing, ScrollTrigger, plugins, React/Vue/Svelte, performance |
Contributing
See CONTRIBUTING.md for guidelines.
Cloning the repo
The repo uses Git LFS for golden regression-test baselines under packages/producer/tests/**/output.mp4 (~240 MB of .mp4 files). If you're cloning the full repo for development, install Git LFS first:
# macOS
brew install git-lfs
# Ubuntu/Debian
sudo apt install git-lfs
# Windows
winget install GitHub.GitLFS
# (or install Git for Windows, which bundles Git LFS as an optional component)
# Then (once, per machine)
git lfs install
If you hit git-lfs filter-process: command not found during git clone or npx skills add heygen-com/hyperframes, install Git LFS and retry. You can also skip LFS content if you only need the source files:
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/heygen-com/hyperframes.git
