mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 16:42:27 +00:00
James RussoandClaude Opus 4.8
646ffff927
* feat(cli): support OpenRouter as an alternative vision provider for capture captioning `hyperframes capture` could only enrich asset descriptions with Gemini vision, which requires a Google API key. Add OpenRouter as an alternative so users without Google access can caption via any vision-capable model through one unified key. Provider is selected by which key is present: OPENROUTER_API_KEY → OpenRouter (OpenAI-style /chat/completions with an image_url data URI), else GEMINI_API_KEY/GOOGLE_API_KEY → Gemini (unchanged), else DOM-only as before. OpenRouter wins if both are set. Default model is google/gemini-3.1-flash-lite (the OpenRouter analog of the Gemini path's existing 3.1-flash-lite tier), overridable via HYPERFRAMES_OPENROUTER_MODEL. Both vision call sites — the image loop and the rasterized-SVG loop — route through a single `captionOne` dispatcher, so the new provider works for SVGs too (the original PR #840 only patched the image loop, which would have left OpenRouter-only users with crashing SVG captioning). The OpenRouter path checks res.ok and surfaces the status/body on failure. Reimplements #840 (which was unmergeable: saved with a UTF-8 BOM + CRLF so GitHub rendered it as a binary diff, used `any`, reused the Gemini model env var, and had a hallucinated default model id). - Adds unit tests for the OpenRouter path (happy path, graceful degradation on non-OK status, no-key skip). - Documents OPENROUTER_API_KEY in the website-to-video guide and the CLI capture reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cli): fix typecheck in OpenRouter caption test — capture request without `as` The test cast `fetchMock.mock.calls[0]` to a tuple (TS2352: `[] | undefined` doesn't overlap `[string, RequestInit]`), which failed the Typecheck CI job. Capture the url/init inside the typed mock and assert via `new Headers()` + `typeof` narrowing instead — no `as` assertions (which the repo bans anyway). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
240 lines
9.5 KiB
Plaintext
240 lines
9.5 KiB
Plaintext
---
|
||
title: Website to Video
|
||
description: "Capture any website and turn it into a production video with a single prompt."
|
||
---
|
||
|
||
Give your AI agent a URL and a creative direction. It captures the site, extracts the brand identity, writes a script and storyboard, generates voiceover, builds animated compositions, and delivers a renderable video.
|
||
|
||
```
|
||
"Create a 20-second product launch video from https://linear.app.
|
||
Make it feel like an Apple keynote announcement."
|
||
```
|
||
|
||
## Getting Started
|
||
|
||
<Steps>
|
||
<Step title="Install skills">
|
||
Skills teach your AI agent how to capture websites and create HyperFrames compositions. Install once — they persist across sessions.
|
||
|
||
```bash
|
||
npx skills add heygen-com/hyperframes
|
||
```
|
||
|
||
Works with [Claude Code](https://claude.ai/claude-code), [Cursor](https://cursor.sh), [Gemini CLI](https://github.com/google-gemini/gemini-cli), and [Codex CLI](https://github.com/openai/codex).
|
||
</Step>
|
||
<Step title="Prompt your agent">
|
||
Open your agent in any directory and describe the video you want:
|
||
|
||
```
|
||
Create a 25-second product launch video from https://example.com. Bold, cinematic, dark theme energy.
|
||
```
|
||
|
||
The agent loads the skill when they see a URL and a video request, and runs the full pipeline — capture, design, script, storyboard, voiceover, build, validate.
|
||
|
||
<Note>
|
||
Agents also trigger this skill automatically when they see a URL and a video request.
|
||
</Note>
|
||
</Step>
|
||
<Step title="Preview">
|
||
```bash
|
||
npx hyperframes preview
|
||
```
|
||
|
||
Opens the video in your browser. Edits reload automatically.
|
||
</Step>
|
||
<Step title="Render to MP4">
|
||
```bash
|
||
npx hyperframes render --output my-video.mp4
|
||
```
|
||
|
||
```
|
||
✓ Captured 750 frames in 12.4s
|
||
✓ Encoded to my-video.mp4 (25.0s, 1920×1080, 6.8MB)
|
||
```
|
||
</Step>
|
||
</Steps>
|
||
|
||
<Note>
|
||
You don't need to run `npx hyperframes capture` manually — the skill instructs the agent to capture as the first step. The capture command is documented [below](#capture-command) for advanced use.
|
||
</Note>
|
||
|
||
## How the Pipeline Works
|
||
|
||
The skill follows the [Hyperframes pipeline](/guides/pipeline): seven steps, each producing a named artifact that feeds the next.
|
||
|
||
| Step | Output | What happens |
|
||
|------|--------|-------------|
|
||
| **Capture** | `capture/` | Extract screenshots, design tokens, fonts, assets, animations |
|
||
| **Design** | `DESIGN.md` | Brand reference — colors, typography, do's and don'ts |
|
||
| **Script** | `SCRIPT.md` | Narration text with hook, story, proof, CTA |
|
||
| **Storyboard** | `STORYBOARD.md` | Per-beat creative direction — mood, assets, animations, transitions |
|
||
| **VO + Timing** | `narration.wav` + `transcript.json` | TTS audio with word-level timestamps |
|
||
| **Build** | `compositions/*.html` | Animated HTML compositions, one per beat |
|
||
| **Validate** | Snapshot PNGs | Visual verification before delivery |
|
||
|
||
See [the pipeline guide](/guides/pipeline) for a detailed walkthrough of each step, the contents of every generated file, and how to iterate without re-running the whole pipeline. The structure is useful for any Hyperframes project, not just website captures.
|
||
|
||
## Video Types
|
||
|
||
The prompt determines the format. Include a duration and creative direction:
|
||
|
||
| Type | Duration | Example |
|
||
|------|----------|---------|
|
||
| Social ad | 10–15s | _"15-second Instagram reel. Energetic, fast cuts."_ |
|
||
| Product launch | 20–30s | _"25-second product launch. Apple keynote energy."_ |
|
||
| Product tour | 30–60s | _"45-second tour showing the top 3 features."_ |
|
||
| Brand reel | 15–30s | _"20-second brand video. Celebrate the design."_ |
|
||
| Feature announcement | 15–25s | _"Feature announcement highlighting the new AI agents."_ |
|
||
| Teaser | 8–15s | _"10-second teaser. Super minimal. Just the hook."_ |
|
||
|
||
<Tip>
|
||
Creative direction matters more than format. _"Playful, hand-crafted feel"_ or _"dark, developer-focused, show code"_ shapes the storyboard and drives every visual decision the agent makes.
|
||
</Tip>
|
||
|
||
## Enriching Captures with Gemini Vision
|
||
|
||
By default, captures describe assets using DOM context — alt text, nearby headings, CSS classes. Add a vision API key for richer AI-powered descriptions.
|
||
|
||
Create a `.env` file in your project root with **either** a [Gemini API key](https://aistudio.google.com/apikey):
|
||
|
||
```bash
|
||
echo "GEMINI_API_KEY=your-key-here" > .env
|
||
```
|
||
|
||
…**or**, if you don't have Google access, an [OpenRouter key](https://openrouter.ai/keys) — a single API that fronts many vision models:
|
||
|
||
```bash
|
||
echo "OPENROUTER_API_KEY=your-key-here" > .env
|
||
```
|
||
|
||
OpenRouter is used when its key is present (it takes priority if both are set). The default model is `google/gemini-3.1-flash-lite`; override it with `HYPERFRAMES_OPENROUTER_MODEL` (any vision-capable OpenRouter model), just as `HYPERFRAMES_GEMINI_MODEL` overrides the Gemini default.
|
||
|
||
<Tabs>
|
||
<Tab title="Without Gemini">
|
||
```
|
||
- hero-bg.png — 582KB, section: "Hero", above fold
|
||
```
|
||
The agent knows the file exists and where it was on the page, but not what it looks like.
|
||
</Tab>
|
||
<Tab title="With Gemini">
|
||
```
|
||
- hero-bg.png — 582KB, A gradient wave in purple and blue sweeps
|
||
across a dark background, creating an aurora-like effect.
|
||
```
|
||
The agent knows what the image actually shows, enabling better creative decisions in the storyboard.
|
||
</Tab>
|
||
</Tabs>
|
||
|
||
| Tier | Rate limit | Cost per image |
|
||
|------|-----------|----------------|
|
||
| Free | 5 RPM | Free |
|
||
| Paid | 2,000 RPM | ~$0.001 |
|
||
|
||
A typical capture with 40 images costs about **$0.04** on the paid tier.
|
||
|
||
## Capture Command
|
||
|
||
The skill runs capture automatically, but you can run it directly for pre-caching, debugging, or using the data outside of video production.
|
||
|
||
```bash
|
||
npx hyperframes capture https://stripe.com
|
||
```
|
||
|
||
```
|
||
◇ Captured Stripe | Financial Infrastructure → captures/stripe-com
|
||
|
||
Screenshots: 12
|
||
Assets: 45
|
||
Sections: 15
|
||
Fonts: sohne-var
|
||
```
|
||
|
||
| Flag | Default | Description |
|
||
|------|---------|-------------|
|
||
| `-o, --output` | `captures/<hostname>` | Output directory |
|
||
| `--timeout` | `120000` | Page load timeout in ms |
|
||
| `--skip-assets` | `false` | Skip downloading images and fonts |
|
||
| `--max-screenshots` | `24` | Maximum screenshot count |
|
||
| `--json` | `false` | Output structured JSON for programmatic use |
|
||
|
||
### What Gets Captured
|
||
|
||
| Data | Description |
|
||
|------|-------------|
|
||
| **Screenshots** | Viewport captures at every scroll depth — dynamic count based on page height |
|
||
| **Colors** | Pixel-sampled dominant colors + computed styles, including oklch/lab conversion |
|
||
| **Fonts** | CSS font families + downloaded woff2 files |
|
||
| **Assets** | Images, SVGs with semantic names, Lottie animations, video previews |
|
||
| **Text** | All visible text in DOM order |
|
||
| **Animations** | Web Animations API, scroll-triggered animations, WebGL shaders |
|
||
| **Sections** | Page structure with headings, types, background colors |
|
||
| **CTAs** | Buttons and links detected by class names and text patterns |
|
||
|
||
## Snapshot Command
|
||
|
||
Capture key frames from a built video as PNGs — verify compositions without a full render:
|
||
|
||
```bash
|
||
npx hyperframes snapshot my-project --at 2.9,10.4,18.7
|
||
```
|
||
|
||
| Flag | Default | Description |
|
||
|------|---------|-------------|
|
||
| `--frames` | `5` | Number of evenly-spaced frames |
|
||
| `--at` | — | Comma-separated timestamps in seconds |
|
||
| `--timeout` | `5000` | Ms to wait for runtime to initialize |
|
||
|
||
## Iterating
|
||
|
||
You don't need to re-run the full pipeline to make changes:
|
||
|
||
- **Edit the storyboard** — `STORYBOARD.md` is the creative north star. Change a beat's mood or assets, then ask the agent to rebuild just that beat.
|
||
- **Edit a composition** — open `compositions/beat-3-proof.html` directly and tweak animations, colors, or layout.
|
||
- **Rebuild one beat** — _"Rebuild beat 2 with more energy. Use the product screenshot as full-bleed background."_
|
||
|
||
See the [pipeline guide](/guides/pipeline#iterating) for more re-entry patterns.
|
||
|
||
## Troubleshooting
|
||
|
||
<AccordionGroup>
|
||
<Accordion title="Capture times out">
|
||
Increase the timeout for sites with Cloudflare or heavy client-side rendering:
|
||
|
||
```bash
|
||
npx hyperframes capture https://example.com --timeout 180000
|
||
```
|
||
</Accordion>
|
||
<Accordion title="Few assets captured">
|
||
Sites using frameworks like Framer lazy-load images via IntersectionObserver. The capture scrolls through the page to trigger loading, but very long pages may miss images near the bottom. Adding a Gemini key improves descriptions of captured assets, but doesn't increase the count.
|
||
</Accordion>
|
||
<Accordion title="Colors look wrong">
|
||
The capture uses pixel sampling combined with DOM computed styles. Dark sites should show dark colors in the palette. Check the scroll screenshots in `captures/<name>/screenshots/` to see what the capture actually saw.
|
||
</Accordion>
|
||
<Accordion title="Agent doesn't find the skill">
|
||
Verify skills are installed:
|
||
|
||
```bash
|
||
npx skills add heygen-com/hyperframes
|
||
```
|
||
|
||
Lead your prompt with _"Use the /website-to-video skill"_ for the most reliable results. Agents also discover it automatically when they see a URL and a video request.
|
||
</Accordion>
|
||
</AccordionGroup>
|
||
|
||
## Next Steps
|
||
|
||
<CardGroup cols={2}>
|
||
<Card title="The Pipeline" icon="list-check" href="/guides/pipeline">
|
||
The canonical 7-step structure this workflow follows.
|
||
</Card>
|
||
<Card title="Quickstart" icon="rocket" href="/quickstart">
|
||
New to HyperFrames? Start here.
|
||
</Card>
|
||
<Card title="GSAP Animation" icon="wand-magic-sparkles" href="/guides/gsap-animation">
|
||
Animation patterns used in compositions.
|
||
</Card>
|
||
<Card title="Rendering" icon="film" href="/guides/rendering">
|
||
Render to MP4, MOV, or WebM.
|
||
</Card>
|
||
</CardGroup>
|