ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-11-14 21:29:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
8b8dcf543e9de69f0bea9348a7aeffdc2de2ca49
hyperframes/README.md
T
Miguel Ángel 68bd52ac6d feat: add init tailwind flag (#577) 
## Problem

Users who want Tailwind utilities in a plain HyperFrames composition currently have to know which Tailwind browser script to add and where to place it. The first pass added `--tailwind`, but review caught three production-facing gaps: the CDN version was major-only, the insertion helper could silently no-op on compact HTML, and the render pipeline did not explicitly wait for Tailwind's async browser compilation before capturing frame 0.

There is also a version-specific agent risk: HyperFrames `init --tailwind` uses Tailwind v4.2 through `@tailwindcss/browser@4.2.4`, while `packages/studio` still uses Tailwind v3. Without a dedicated skill, agents can easily mix v3 `tailwind.config.js` / `@tailwind` patterns into v4 browser-runtime composition HTML.

## What this fixes

- Adds `hyperframes init --tailwind`.
- Pins the Tailwind browser runtime to `@tailwindcss/browser@4.2.4/dist/index.global.js` with SRI and `crossorigin="anonymous"`.
- Injects a `window.__tailwindReady` promise next to the browser runtime.
- Makes frame capture wait for `window.__tailwindReady` in both screenshot and BeginFrame capture modes before capturing frame 0.
- Inserts Tailwind support before `</head>` case-insensitively, including single-line/minified heads, and falls back to prepending when there is no head tag.
- Skips recursive Tailwind injection under `.git`, `dist`, and `node_modules`.
- Tracks whether init used Tailwind in the existing `init_template` telemetry event.
- Adds a first-party `/tailwind` skill for Tailwind v4.2 browser-runtime HyperFrames composition work.
- Updates README, docs, generated project agent files, CLI skill guidance, and plugin metadata so the Tailwind skill is discoverable.
- Documents the browser-runtime tradeoff and production/offline guidance.

## Root cause

`scaffoldProject()` copied the selected example and patched media placeholders, then immediately wrote project metadata and `package.json`. There was no optional post-copy step for framework-specific HTML support. The initial Tailwind post-copy step also treated the browser runtime like a static script, but Tailwind compiles utilities asynchronously after scanning the DOM, so the capture engine needed an explicit readiness contract.

On the agent side, the repo exposed HyperFrames, CLI, GSAP, registry, and runtime adapter skills, but had no Tailwind-specific instruction to separate the v4 browser-runtime composition path from Studio's v3 internal setup.

## Verification

### Local checks

- `bunx vitest run packages/cli/src/commands/init.test.ts`
- `bun run --filter @hyperframes/cli test src/commands/init.test.ts`
- `bun run --filter @hyperframes/cli typecheck`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run lint:skills`
- `bun run lint`
- `npx skills add . --list` showed 12 local skills, including `tailwind`.
- `bunx oxfmt --check packages/cli/src/commands/init.ts packages/cli/src/commands/init.test.ts packages/cli/src/telemetry/events.ts packages/engine/src/services/frameCapture.ts docs/packages/cli.mdx`
- `bunx oxfmt --check README.md docs/quickstart.mdx docs/packages/cli.mdx CLAUDE.md packages/cli/src/templates/_shared/CLAUDE.md packages/cli/src/templates/_shared/AGENTS.md skills/hyperframes-cli/SKILL.md skills/tailwind/SKILL.md .codex-plugin/plugin.json .cursor-plugin/plugin.json`
- `bunx oxlint packages/cli/src/commands/init.ts packages/cli/src/commands/init.test.ts packages/cli/src/telemetry/events.ts packages/engine/src/services/frameCapture.ts`
- `git diff --check`
- Lefthook pre-commit: lint/format/typecheck for code commit; format for docs/skill commit
- Lefthook commit-msg: commitlint

Generated-project render proof at `/tmp/hf-tailwind-render-proof`:

- `bun packages/cli/src/cli.ts init /tmp/hf-tailwind-render-proof --example blank --tailwind --non-interactive --skip-skills`
- Added a temporary Tailwind-only card using `flex`, `h-full`, `w-full`, `items-center`, `justify-center`, `bg-slate-950`, `rounded-3xl`, `bg-white`, `px-20`, `py-12`, `text-8xl`, `font-black`, `text-black`, and `shadow-2xl`.
- `bun packages/cli/src/cli.ts lint /tmp/hf-tailwind-render-proof` → 0 errors, 0 warnings.
- `bun packages/cli/src/cli.ts validate /tmp/hf-tailwind-render-proof` → 0 errors, 0 regular warnings; the temp proof still reports validator contrast warnings even though the rendered/browser pixels show black text on white background.
- `bun packages/cli/src/cli.ts render /tmp/hf-tailwind-render-proof --workers 1 --fps 24 --quality draft --output /tmp/hf-tailwind-render-proof-artifacts/output.mp4`
- Render compiler inlined both GSAP and `https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4.2.4/dist/index.global.js`.
- `ffprobe -v error -select_streams v:0 -show_entries stream=codec_name,width,height,r_frame_rate,duration -of default=noprint_wrappers=1 /tmp/hf-tailwind-render-proof-artifacts/output.mp4` → H.264, 1920x1080, 24fps, 10s.
- Extracted frame-0 proof: `/tmp/hf-tailwind-render-proof-artifacts/frame-000.png`.

### Browser verification

- Started Studio preview for `/tmp/hf-tailwind-render-proof`.
- Used `agent-browser` to open `http://localhost:5194`.
- Verified the Tailwind-styled composition rendered in Studio preview.
- Captured screenshot: `/tmp/hf-tailwind-render-proof-artifacts/browser/tailwind-preview.png`.
- Captured agent-browser-driven recording: `/tmp/hf-tailwind-render-proof-artifacts/browser/tailwind-preview.webm`.
- Served the PR worktree locally and used `agent-browser` to open the new Tailwind skill proof page.
- Verified the browser-visible skill content includes `@tailwindcss/browser@4.2.4`.
- Captured screenshot: `/Users/miguel07code/.codex/worktrees/pr-577-tailwind-comments/tmp/agent-browser-proof/tailwind-skill.png`.
- Captured agent-browser-driven recording: `/Users/miguel07code/.codex/worktrees/pr-577-tailwind-comments/tmp/agent-browser-proof/tailwind-skill.webm`.

## Notes

- This still intentionally uses Tailwind's browser runtime rather than adding a generated Tailwind build pipeline. That keeps `hyperframes init --tailwind` small and compatible with the current no-install generated project workflow.
- The `/tailwind` skill cites official Tailwind v4 docs plus community skill references, but its instructions are HyperFrames-specific and tuned for the pinned v4.2 browser runtime.
- Browser proof artifacts are local-only under `/tmp/hf-tailwind-render-proof-artifacts/` and `tmp/agent-browser-proof/` and intentionally not committed.
2026-05-01 00:07:00 +02:00

234 lines
13 KiB
Markdown
Raw Blame History

<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/logo/dark.svg">
<source media="(prefers-color-scheme: light)" srcset="docs/logo/light.svg">
<img alt="HyperFrames" src="docs/logo/light.svg" width="300">
</picture>
</p>
<p align="center">
<a href="https://www.npmjs.com/package/hyperframes"><img src="https://img.shields.io/npm/v/hyperframes.svg?style=flat" alt="npm version"></a>
<a href="https://www.npmjs.com/package/hyperframes"><img src="https://img.shields.io/npm/dm/hyperframes.svg?style=flat" alt="npm downloads"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" alt="License"></a>
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/node-%3E%3D22-brightgreen" alt="Node.js"></a>
<a href="https://discord.gg/EbK98HBPdk"><img src="https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&logoColor=white" alt="Discord"></a>
</p>
<p align="center"><b>Write HTML. Render video. Built for agents.</b></p>
<p align="center">
<img src="https://static.heygen.ai/hyperframes-oss/docs/images/readme-demo.gif" alt="HyperFrames demo — HTML code on the left transforms into a rendered video on the right" width="800">
</p>
Hyperframes is an open-source video rendering framework that lets you create, preview, and render HTML-based video compositions — with first-class support for AI agents.
## Quick Start
### Option 1: With an AI coding agent (recommended)
Install the HyperFrames skills, then describe the video you want:
```bash
npx skills add heygen-com/hyperframes
```
This teaches your agent (Claude Code, Cursor, Gemini CLI, Codex) how to write correct compositions, GSAP timelines, Tailwind v4 browser-runtime styles, and first-party adapter animations. In Claude Code, the skills register as slash commands — invoke `/hyperframes` to author compositions, `/hyperframes-cli` for CLI commands, `/tailwind` for `init --tailwind` projects, `/gsap` for timeline animation help, or the adapter skills (`/animejs`, `/css-animations`, `/lottie`, `/three`, `/waapi`) when a composition uses those runtimes.
For Claude Design, open [`docs/guides/claude-design-hyperframes.md`](https://github.com/heygen-com/hyperframes/blob/main/docs/guides/claude-design-hyperframes.md) on GitHub and click the download button (↓) to save it, then attach the file to your Claude Design chat. It produces a valid first draft; refine in any AI coding agent. See the [Claude Design guide](https://hyperframes.heygen.com/guides/claude-design).
For Codex specifically, the same skills are also exposed as an [OpenAI Codex plugin](./.codex-plugin/plugin.json) — sparse-install just the plugin surface:
```bash
codex plugin marketplace add heygen-com/hyperframes --sparse .codex-plugin --sparse skills --sparse assets
```
For Claude Code, the repo also ships a [Claude Code plugin manifest](./.claude-plugin/plugin.json): test it locally with `claude --plugin-dir .`. The manifest intentionally omits `skills` because Claude Code auto-discovers the root `skills/` directory by convention, and for marketplace submission use the title `HyperFrames by HeyGen` plus the black/white icon assets at [`assets/claude-code-icon-dark.svg`](./assets/claude-code-icon-dark.svg) and [`assets/claude-code-icon-light.svg`](./assets/claude-code-icon-light.svg) for the two theme slots.
For Cursor, the same skills are packaged as a [Cursor plugin](./.cursor-plugin/plugin.json) — install from the Cursor Marketplace, or sideload by cloning this repo and pointing **Settings → Plugins → Load unpacked** at the repo root.
#### Try it: example prompts
Copy any of these into your agent to get started. The `/hyperframes` prefix loads the skill context explicitly so you get correct output the first time.
**Cold start — describe what you want:**
> Using `/hyperframes`, create a 10-second product intro with a fade-in title, a background video, and background music.
**Warm start — turn existing context into a video:**
> Take a look at this GitHub repo https://github.com/heygen-com/hyperframes and explain its uses and architecture to me using `/hyperframes`.
> Summarize the attached PDF into a 45-second pitch video using `/hyperframes`.
> Turn this CSV into an animated bar chart race using `/hyperframes`.
**Format-specific:**
> Make a 9:16 TikTok-style hook video about [topic] using `/hyperframes`, with bouncy captions synced to a TTS narration.
**Iterate — talk to the agent like a video editor:**
> Make the title 2x bigger, swap to dark mode, and add a fade-out at the end.
> Add a lower third at 0:03 with my name and title.
The agent handles scaffolding, animation, and rendering. See the [prompting guide](https://hyperframes.heygen.com/guides/prompting) for more patterns.
### Option 2: Start a project manually
```bash
npx hyperframes init my-video
cd my-video
npx hyperframes preview # preview in browser (live reload)
npx hyperframes render # render to MP4
```
`hyperframes init` installs skills automatically, so you can hand off to your AI agent at any point.
**Requirements:** Node.js >= 22, FFmpeg
## Why Hyperframes?
- **HTML-native** — compositions are HTML files with data attributes. No React, no proprietary DSL.
- **AI-first** — agents already speak HTML. The CLI is non-interactive by default, designed for agent-driven workflows.
- **Deterministic rendering** — same input = identical output. Built for automated pipelines.
- **Frame Adapter pattern** — bring your own animation runtime (GSAP, Lottie, CSS, Three.js).
## Hyperframes vs Remotion
Hyperframes is inspired by [Remotion](https://www.remotion.dev) — we used Remotion at HeyGen in production, learned a ton from it, and kept attribution comments in the source for the patterns it pioneered (Chrome launch flags, image2pipe → FFmpeg streaming, frame buffering). Both tools drive headless Chrome and both are deterministic. They differ on one decision: **what the primary author writes.** Remotion's bet is React components; Hyperframes' bet is HTML.
| | **Hyperframes** | **Remotion** |
| ----------------------------------------------------- | ------------------------------ | --------------------------------- |
| Authoring | HTML + CSS + GSAP | React components (TSX) |
| Build step | None; `index.html` plays as-is | Required (bundler) |
| Library-clock animations (GSAP, Anime.js, Motion One) | Seekable, frame-accurate | Plays at wall-clock during render |
| Arbitrary HTML / CSS passthrough | Paste and animate | Rewrite as JSX |
| Distributed rendering | Single-machine today | Lambda, production-ready |
### Licensing: fully open source vs source-available
**Hyperframes is completely open source under [Apache 2.0](LICENSE)** — an OSI-approved license. Use it commercially at any scale, with no per-render fees, no seat caps, no company-size thresholds.
**Remotion is [source-available, not open source](https://www.remotion.pro/license).** The code is on GitHub under a custom Remotion License that requires a paid company license above small-team thresholds. It's a great product with a real team behind it — but if open-source licensing matters to you (OSI compliance, redistribution rights, no per-use fees), that's a first-order decision point.
Full write-up with benchmarks, an honest list of where each tool wins, and a GSAP side-by-side: **[Hyperframes vs Remotion guide](https://hyperframes.heygen.com/guides/hyperframes-vs-remotion)**.
## How It Works
Define your video as HTML with data attributes:
```html
<div id="stage" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
<video
id="clip-1"
data-start="0"
data-duration="5"
data-track-index="0"
src="intro.mp4"
muted
playsinline
></video>
<img
id="overlay"
class="clip"
data-start="2"
data-duration="3"
data-track-index="1"
src="logo.png"
/>
<audio
id="bg-music"
data-start="0"
data-duration="9"
data-track-index="2"
data-volume="0.5"
src="music.wav"
></audio>
</div>
```
Preview instantly in the browser. Render to MP4 locally or in Docker.
## Catalog
50+ ready-to-use blocks and components — social overlays, shader transitions, data visualizations, and cinematic effects:
```bash
npx hyperframes add flash-through-white # shader transition
npx hyperframes add instagram-follow # social overlay
npx hyperframes add data-chart # animated chart
```
Browse the full catalog at **[hyperframes.heygen.com/catalog](https://hyperframes.heygen.com/catalog/blocks/data-chart)**.
## Documentation
Full documentation at **[hyperframes.heygen.com/introduction](https://hyperframes.heygen.com/introduction)** — [Quickstart](https://hyperframes.heygen.com/quickstart) | [Guides](https://hyperframes.heygen.com/guides/gsap-animation) | [API Reference](https://hyperframes.heygen.com/packages/core) | [Catalog](https://hyperframes.heygen.com/catalog/blocks/data-chart)
## Packages
| Package | Description |
| ---------------------------------------------------------------- | ----------------------------------------------------------- |
| [`hyperframes`](packages/cli) | CLI — create, preview, lint, and render compositions |
| [`@hyperframes/core`](packages/core) | Types, parsers, generators, linter, runtime, frame adapters |
| [`@hyperframes/engine`](packages/engine) | Seekable page-to-video capture engine (Puppeteer + FFmpeg) |
| [`@hyperframes/producer`](packages/producer) | Full rendering pipeline (capture + encode + audio mix) |
| [`@hyperframes/studio`](packages/studio) | Browser-based composition editor UI |
| [`@hyperframes/player`](packages/player) | Embeddable `<hyperframes-player>` web component |
| [`@hyperframes/shader-transitions`](packages/shader-transitions) | WebGL shader transitions for compositions |
## Skills
HyperFrames ships [skills](https://github.com/vercel-labs/skills) that teach AI agents framework-specific patterns that generic docs don't cover.
```bash
npx skills add heygen-com/hyperframes
```
| Skill | What it teaches |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `hyperframes` | HTML composition authoring, captions, TTS, audio-reactive animation, transitions |
| `hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts, doctor |
| `hyperframes-registry` | Block and component installation via `hyperframes add` |
| `website-to-hyperframes` | Capture a URL and turn it into a video — full website-to-video pipeline |
| `remotion-to-hyperframes` | Translate a Remotion (React) composition into a HyperFrames HTML composition |
| `gsap` | GSAP timelines for HyperFrames: paused registration, deterministic seeking, easing, sequencing, performance |
| `animejs` | Anime.js animations and timelines registered on `window.__hfAnime` for deterministic HyperFrames seeking |
| `css-animations` | CSS keyframe animation patterns that HyperFrames can discover, pause, and seek |
| `lottie` | `lottie-web` and dotLottie players registered on `window.__hfLottie` with local assets and paused playback |
| `three` | Three.js scenes that render from HyperFrames `hf-seek` events and `window.__hfThreeTime` instead of wall-clock time |
| `waapi` | Web Animations API `element.animate()` patterns seeked through `document.getAnimations()` |
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
### Cloning the repo
The repo uses [Git LFS](https://git-lfs.com) for golden regression-test baselines under `packages/producer/tests/**/output.mp4` (~240 MB of `.mp4` files). If you're cloning the full repo for development, install Git LFS first:
```bash
# macOS
brew install git-lfs
# Ubuntu/Debian
sudo apt install git-lfs
# Windows
winget install GitHub.GitLFS
# (or install Git for Windows, which bundles Git LFS as an optional component)
# Then (once, per machine)
git lfs install
```
If you hit `git-lfs filter-process: command not found` during `git clone` or `npx skills add heygen-com/hyperframes`, install Git LFS and retry. You can also skip LFS content if you only need the source files:
```bash
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/heygen-com/hyperframes.git
```
## License
[Apache 2.0](LICENSE)
Reference in New Issue View Git Blame Copy Permalink