fix: double-audio bug + lint rules + docs guide + capture improvements

Double-audio bug fix: - scaffolding.ts: stop writing index.html in captures/ (root cause — runtime discovered scaffold + real index.html as two compositions) - New lint rule: multiple_root_compositions — errors if >1 root HTML - New lint rule: duplicate_audio_track — warns on overlapping audio Capture improvements (from testing 30+ websites): - Catalog runs BEFORE extractHtml (which mutates DOM — converts img src to data URLs). HeyKuba: 2 images → 78. - networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets) - Lazy-load image wait, CSS background-image cataloging - SVG naming from class/id/parent (not just aria-label) - Gemini batch 5→20, pause 12s→2s, maxOutputTokens 300→500 - Asset descriptions sorted: captioned first Docs: - New guide: guides/website-to-video.mdx (full tutorial) - CLI docs: added capture and snapshot commands - docs.json: website-to-video in Guides nav C
2026-08-31 02:41:44 +00:00 · 2026-04-16 22:58:50 -04:00
parent 3256551a5e
commit a77a6cbbf7
5 changed files with 362 additions and 49 deletions
@@ -69,6 +69,7 @@
          {
            "group": "Guides",
            "pages": [
+"guides/website-to-video",
              "guides/prompting",
              "guides/gsap-animation",
              "guides/rendering",
@@ -0,0 +1,225 @@
+---
+title: Website to Video
+description: "Capture any website and turn it into a production video with a single prompt."
+---
+
+Give your AI agent a URL and a creative direction. It captures the site, extracts the brand identity, writes a script and storyboard, generates voiceover, builds animated compositions, and delivers a renderable video.
+
+```
+"Create a 20-second product launch video from https://linear.app.
+ Make it feel like an Apple keynote announcement."
+```
+
+## Getting Started
+
+<Steps>
+  <Step title="Install skills">
+    Skills teach your AI agent how to capture websites and create HyperFrames compositions. Install once — they persist across sessions.
+
+    ```bash
+    npx skills add heygen-com/hyperframes
+    npx skills add greensock/gsap-skills
+    ```
+
+    Works with [Claude Code](https://claude.ai/claude-code), [Cursor](https://cursor.sh), [Gemini CLI](https://github.com/google-gemini/gemini-cli), and [Codex CLI](https://github.com/openai/codex).
+  </Step>
+  <Step title="Prompt your agent">
+    Open your agent in any directory and describe the video you want:
+
+    ```
+    Create a 25-second product launch video from https://stripe.com.
+    Bold, cinematic, financial infrastructure energy.
+    ```
+
+    The agent discovers the `/website-to-hyperframes` skill and runs the full pipeline automatically — capture, design, script, storyboard, voiceover, build, validate.
+  </Step>
+  <Step title="Preview">
+    ```bash
+    npx hyperframes preview
+    ```
+
+    Opens the video in your browser. Edits reload automatically.
+  </Step>
+  <Step title="Render to MP4">
+    ```bash
+    npx hyperframes render --output my-video.mp4
+    ```
+
+    ```
+    ✓ Captured 750 frames in 12.4s
+    ✓ Encoded to my-video.mp4 (25.0s, 1920×1080, 6.8MB)
+    ```
+  </Step>
+</Steps>
+
+<Note>
+  You don't need to run `npx hyperframes capture` manually — the skill instructs the agent to capture as the first step. The capture command is documented [below](#capture-command) for advanced use.
+</Note>
+
+## How the Pipeline Works
+
+The skill runs 7 steps. Each produces an artifact that feeds the next:
+
+| Step | Output | What happens |
+|------|--------|-------------|
+| **Capture** | `captures/<name>/` | Extract screenshots, design tokens, fonts, assets, animations |
+| **Design** | `DESIGN.md` | Brand reference — colors, typography, do's and don'ts |
+| **Script** | `SCRIPT.md` | Narration text with hook, story, proof, CTA |
+| **Storyboard** | `STORYBOARD.md` | Per-beat creative direction — mood, assets, animations, transitions |
+| **VO + Timing** | `narration.wav` + `transcript.json` | TTS audio with word-level timestamps |
+| **Build** | `compositions/*.html` | Animated HTML compositions, one per beat |
+| **Validate** | Snapshot PNGs | Visual verification before delivery |
+
+## Video Types
+
+The prompt determines the format. Include a duration and creative direction:
+
+| Type | Duration | Example |
+|------|----------|---------|
+| Social ad | 10–15s | _"15-second Instagram reel. Energetic, fast cuts."_ |
+| Product launch | 20–30s | _"25-second product launch. Apple keynote energy."_ |
+| Product tour | 30–60s | _"45-second tour showing the top 3 features."_ |
+| Brand reel | 15–30s | _"20-second brand video. Celebrate the design."_ |
+| Feature announcement | 15–25s | _"Feature announcement highlighting the new AI agents."_ |
+| Teaser | 8–15s | _"10-second teaser. Super minimal. Just the hook."_ |
+
+<Tip>
+  Creative direction matters more than format. _"Playful, hand-crafted feel"_ or _"dark, developer-focused, show code"_ shapes the storyboard and drives every visual decision the agent makes.
+</Tip>
+
+## Enriching Captures with Gemini Vision
+
+By default, captures describe assets using DOM context — alt text, nearby headings, CSS classes. Add a [Gemini API key](https://aistudio.google.com/apikey) for richer AI-powered descriptions using vision.
+
+Create a `.env` file in your project root:
+
+```bash
+echo "GEMINI_API_KEY=your-key-here" > .env
+```
+
+<Tabs>
+  <Tab title="Without Gemini">
+    ```
+    - hero-bg.png — 582KB, section: "Hero", above fold
+    ```
+    The agent knows the file exists and where it was on the page, but not what it looks like.
+  </Tab>
+  <Tab title="With Gemini">
+    ```
+    - hero-bg.png — 582KB, A gradient wave in purple and blue sweeps
+      across a dark background, creating an aurora-like effect.
+    ```
+    The agent knows what the image actually shows, enabling better creative decisions in the storyboard.
+  </Tab>
+</Tabs>
+
+| Tier | Rate limit | Cost per image |
+|------|-----------|----------------|
+| Free | 5 RPM | Free |
+| Paid | 2,000 RPM | ~$0.001 |
+
+A typical capture with 40 images costs about **$0.04** on the paid tier.
+
+## Capture Command
+
+The skill runs capture automatically, but you can run it directly for pre-caching, debugging, or using the data outside of video production.
+
+```bash
+npx hyperframes capture https://stripe.com
+```
+
+```
+◇  Captured Stripe | Financial Infrastructure → captures/stripe-com
+
+  Screenshots: 12
+  Assets: 45
+  Sections: 15
+  Fonts: sohne-var
+```
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `-o, --output` | `captures/<hostname>` | Output directory |
+| `--timeout` | `120000` | Page load timeout in ms |
+| `--skip-assets` | `false` | Skip downloading images and fonts |
+| `--max-screenshots` | `24` | Maximum screenshot count |
+| `--json` | `false` | Output structured JSON for programmatic use |
+
+### What Gets Captured
+
+| Data | Description |
+|------|-------------|
+| **Screenshots** | Viewport captures at every scroll depth — dynamic count based on page height |
+| **Colors** | Pixel-sampled dominant colors + computed styles, including oklch/lab conversion |
+| **Fonts** | CSS font families + downloaded woff2 files |
+| **Assets** | Images, SVGs with semantic names, Lottie animations, video previews |
+| **Text** | All visible text in DOM order |
+| **Animations** | Web Animations API, scroll-triggered animations, WebGL shaders |
+| **Sections** | Page structure with headings, types, background colors |
+| **CTAs** | Buttons and links detected by class names and text patterns |
+
+## Snapshot Command
+
+Capture key frames from a built video as PNGs — verify compositions without a full render:
+
+```bash
+npx hyperframes snapshot my-project --at 2.9,10.4,18.7
+```
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--frames` | `5` | Number of evenly-spaced frames |
+| `--at` | — | Comma-separated timestamps in seconds |
+| `--timeout` | `5000` | Ms to wait for runtime to initialize |
+
+## Iterating
+
+You don't need to re-run the full pipeline to make changes:
+
+- **Edit the storyboard** — `STORYBOARD.md` is the creative north star. Change a beat's mood or assets, then ask the agent to rebuild just that beat.
+- **Edit a composition** — open `compositions/beat-3-proof.html` directly and tweak animations, colors, or layout.
+- **Rebuild one beat** — _"Rebuild beat 2 with more energy. Use the product screenshot as full-bleed background."_
+
+## Troubleshooting
+
+<AccordionGroup>
+  <Accordion title="Capture times out">
+    Increase the timeout for sites with Cloudflare or heavy client-side rendering:
+
+    ```bash
+    npx hyperframes capture https://example.com --timeout 180000
+    ```
+  </Accordion>
+  <Accordion title="Few assets captured">
+    Sites using frameworks like Framer lazy-load images via IntersectionObserver. The capture scrolls through the page to trigger loading, but very long pages may miss images near the bottom. Adding a Gemini key improves descriptions of captured assets, but doesn't increase the count.
+  </Accordion>
+  <Accordion title="Colors look wrong">
+    The capture uses pixel sampling combined with DOM computed styles. Dark sites should show dark colors in the palette. Check the scroll screenshots in `captures/<name>/screenshots/` to see what the capture actually saw.
+  </Accordion>
+  <Accordion title="Agent doesn't find the skill">
+    Verify skills are installed:
+
+    ```bash
+    npx skills add heygen-com/hyperframes
+    ```
+
+    The skill triggers automatically when the agent sees a URL and a video request. To invoke explicitly: _"Use the /website-to-hyperframes skill."_
+  </Accordion>
+</AccordionGroup>
+
+## Next Steps
+
+<CardGroup cols={2}>
+  <Card title="Quickstart" icon="rocket" href="/quickstart">
+    New to HyperFrames? Start here.
+  </Card>
+  <Card title="GSAP Animation" icon="wand-magic-sparkles" href="/guides/gsap-animation">
+    Animation patterns used in compositions.
+  </Card>
+  <Card title="Rendering" icon="film" href="/guides/rendering">
+    Render to MP4, MOV, or WebM.
+  </Card>
+  <Card title="CLI Reference" icon="terminal" href="/packages/cli">
+    Full command reference.
+  </Card>
+</CardGroup>
@@ -14,11 +14,13 @@ npx hyperframes <command>
 ## When to Use

 **Use the CLI when you want to:**
- Create a new composition project from an example
- Preview compositions with live hot reload during development
- Render compositions to MP4 (locally or in Docker)
- Lint compositions for structural issues
- Check your environment for missing dependencies
+- Capture a website for video production (`capture`)
+- Create a new composition project from an example (`init`)
+- Preview compositions with live hot reload (`preview`)
+- Render compositions to MP4 locally or in Docker (`render`)
+- Lint compositions for structural issues (`lint`)
+- Capture key frames as PNG screenshots (`snapshot`)
+- Check your environment for missing dependencies (`doctor`)

 **Use a different package if you want to:**
 - Render programmatically from Node.js code — use the [producer](/packages/producer)
@@ -321,6 +323,39 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
    <Tip>
      Combine `tts` with `transcribe` to generate narration and word-level timestamps for captions in a single workflow: generate the audio with `tts`, then transcribe the output with `transcribe` to get word-level timing.
    </Tip>
+    ### `capture`
+
+    Capture a website — extract screenshots, design tokens, fonts, assets, and animations for video production:
+
+    ```bash
+    npx hyperframes capture https://stripe.com
+    npx hyperframes capture https://linear.app -o captures/linear
+    npx hyperframes capture https://example.com --json
+    ```
+
+    ```
+    ◇  Captured Stripe | Financial Infrastructure → captures/stripe-com
+
+      Screenshots: 12
+      Assets: 45
+      Sections: 15
+      Fonts: sohne-var
+    ```
+
+    | Flag | Description |
+    |------|-------------|
+    | `-o, --output` | Output directory (default: `captures/<hostname>`) |
+    | `--timeout` | Page load timeout in ms (default: 120000) |
+    | `--skip-assets` | Skip downloading images and fonts |
+    | `--max-screenshots` | Maximum screenshot count (default: 24) |
+    | `--json` | Output structured JSON for programmatic use |
+
+    The capture command extracts everything an AI agent needs to understand a website's visual identity: viewport screenshots at every scroll depth, color palette (pixel-sampled + DOM computed), font files, images with semantic names, SVGs, Lottie animations, video previews, WebGL shaders, visible text, and page structure.
+
+    Output is a self-contained directory with a `CLAUDE.md` file that any AI agent can read to understand the captured site. Used by the `/website-to-hyperframes` skill as step 1 of the video production pipeline.
+
+    Set `GEMINI_API_KEY` in a `.env` file for AI-powered image descriptions via Gemini 2.5 Flash vision (~$0.001/image). See the [Website to Video](/guides/website-to-video#enriching-captures-with-gemini-vision) guide for details.
+
  </Tab>
  <Tab title="Preview">
    ### `preview`
@@ -375,6 +410,32 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
    - **Info** (`ℹ`) — informational notices, shown only with `--verbose`

    The linter detects missing attributes, missing adapter libraries (GSAP, Lottie, Three.js), structural problems, and more. See [Common Mistakes](/guides/common-mistakes) for details on each rule.
+
+    ### `snapshot`
+
+    Capture key frames from a composition as PNG screenshots — verify visual output without a full render:
+
+    ```bash
+    npx hyperframes snapshot my-project --at 2.9,10.4,18.7
+    npx hyperframes snapshot my-project --frames 10
+    ```
+
+    ```
+    ◆  Capturing 3 frames at [2.9s, 10.4s, 18.7s] from my-project
+
+    ◇  3 snapshots saved to snapshots/
+       snapshots/frame-00-at-2.9s.png
+       snapshots/frame-01-at-10.4s.png
+       snapshots/frame-02-at-18.7s.png
+    ```
+
+    | Flag | Description |
+    |------|-------------|
+    | `--frames` | Number of evenly-spaced frames to capture (default: 5) |
+    | `--at` | Comma-separated timestamps in seconds (e.g., `3.0,10.5,18.0`) |
+    | `--timeout` | Ms to wait for runtime to initialize (default: 5000) |
+
+    The snapshot command bundles the project, serves it locally, launches headless Chrome, seeks to each timestamp, and captures a 1920×1080 PNG. Useful for visual verification during the build step of the [website-to-video](/guides/website-to-video) workflow.
  </Tab>
  <Tab title="Build">
    ### `render`
@@ -61,51 +61,13 @@ export async function generateProjectScaffold(
  progress: (stage: string, detail?: string) => void,
  warnings: string[],
 ): Promise<void> {
-  // Ensure capture output is a valid HyperFrames project (index.html + meta.json)
-  const indexPath = join(outputDir, "index.html");
+  // Capture output is a DATA folder, not a video project.
+  // The agent builds index.html + compositions/ during step 6.
+  // We only write meta.json (project metadata) — NOT index.html.
+  // Writing index.html here caused a double-audio bug: the runtime
+  // discovered both the scaffold and the agent's real index.html as
+  // valid compositions, playing two audio tracks offset in time.
  const metaPath = join(outputDir, "meta.json");
-  if (!existsSync(indexPath)) {
-    writeFileSync(
-      indexPath,
-      `<!doctype html>
-<html lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=1920, height=1080" />
-    <script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
-    <style>
-      * { margin: 0; padding: 0; box-sizing: border-box; }
-      html, body { margin: 0; width: 1920px; height: 1080px; overflow: hidden; background: #000; }
-    </style>
-  </head>
-  <body>
-    <!-- Root composition wrapper — AGENT: update data-duration to match total video length -->
-    <div data-composition-id="main" data-width="1920" data-height="1080" data-start="0" data-duration="28">
-
-      <!-- SCENE SLOTS — AGENT: adjust count, durations, and IDs to match your scene plan -->
-      <div id="scene-1" data-composition-src="compositions/scene-1.html" data-start="0" data-duration="7" data-track-index="1" data-width="1920" data-height="1080"></div>
-      <div id="scene-2" data-composition-src="compositions/scene-2.html" data-start="7" data-duration="7" data-track-index="1" data-width="1920" data-height="1080"></div>
-      <div id="scene-3" data-composition-src="compositions/scene-3.html" data-start="14" data-duration="7" data-track-index="1" data-width="1920" data-height="1080"></div>
-      <div id="scene-4" data-composition-src="compositions/scene-4.html" data-start="21" data-duration="7" data-track-index="1" data-width="1920" data-height="1080"></div>
-
-      <!-- NARRATION — AGENT: update src after generating TTS -->
-      <audio id="narration" data-start="0" data-duration="28" data-track-index="0" data-volume="1" src="narration.wav"></audio>
-
-      <!-- CAPTIONS (optional — only add if user requests captions/subtitles) -->
-
-    </div>
-
-    <script>
-      window.__timelines = window.__timelines || {};
-      var tl = gsap.timeline({ paused: true });
-      window.__timelines["main"] = tl;
-    </script>
-  </body>
-</html>
-`,
-      "utf-8",
-    );
-  }
  if (!existsSync(metaPath)) {
    const hostname = new URL(url).hostname.replace(/^www\./, "");
    writeFileSync(
@@ -53,6 +53,8 @@ export function lintProject(project: ProjectDir): ProjectLintResult {
  const projectFindings = [
    ...lintProjectAudioFiles(project.dir, allHtmlSources),
    ...lintAudioSrcNotFound(project.dir, allHtmlSources),
+    ...lintMultipleRootCompositions(results),
+    ...lintDuplicateAudioTracks(allHtmlSources),
  ];
  if (projectFindings.length > 0) {
    // Append project-level findings to the root index.html result
@@ -154,6 +156,68 @@ function lintAudioSrcNotFound(projectDir: string, htmlSources: string[]): Hyperf
  return findings;
 }

+/**
+ * Error if multiple root-level HTML files exist (not in compositions/).
+ * Catches the double-audio bug where a scaffold and the real index.html
+ * both register as root compositions.
+ */
+function lintMultipleRootCompositions(
+  results: Array<{ file: string; result: HyperframeLintResult }>,
+): HyperframeLintFinding[] {
+  const findings: HyperframeLintFinding[] = [];
+  const rootFiles = results.map((r) => r.file).filter((f) => !f.startsWith("compositions/"));
+
+  if (rootFiles.length > 1) {
+    findings.push({
+      code: "multiple_root_compositions",
+      severity: "error",
+      message: `Multiple root-level HTML files found: ${rootFiles.join(", ")}. The runtime may discover both as composition entry points, causing duplicate audio playback.`,
+      fixHint:
+        "A project should have exactly one root index.html. Remove or rename extra root-level HTML files.",
+    });
+  }
+  return findings;
+}
+
+/**
+ * Warn if multiple <audio> elements on the same data-track-index overlap in time.
+ * This causes layered audio playback.
+ */
+function lintDuplicateAudioTracks(htmlSources: string[]): HyperframeLintFinding[] {
+  const findings: HyperframeLintFinding[] = [];
+  const audioRe =
+    /<audio\b[^>]*\bdata-track-index\s*=\s*["'](\d+)["'][^>]*\bdata-start\s*=\s*["']([^"']+)["'][^>]*\bdata-duration\s*=\s*["']([^"']+)["'][^>]*>/gi;
+
+  const tracks: Array<{ trackIndex: number; start: number; end: number; src: string }> = [];
+  for (const html of htmlSources) {
+    let match: RegExpExecArray | null;
+    while ((match = audioRe.exec(html)) !== null) {
+      const trackIndex = parseInt(match[1]!, 10);
+      const start = parseFloat(match[2]!);
+      const duration = parseFloat(match[3]!);
+      const srcMatch = match[0].match(/\bsrc\s*=\s*["']([^"']+)["']/);
+      tracks.push({ trackIndex, start, end: start + duration, src: srcMatch?.[1] ?? "unknown" });
+    }
+  }
+
+  for (let i = 0; i < tracks.length; i++) {
+    for (let j = i + 1; j < tracks.length; j++) {
+      const a = tracks[i]!;
+      const b = tracks[j]!;
+      if (a.trackIndex !== b.trackIndex) continue;
+      if (a.start < b.end && b.start < a.end) {
+        findings.push({
+          code: "duplicate_audio_track",
+          severity: "warning",
+          message: `Multiple <audio> elements on track ${a.trackIndex} overlap (${a.src} at ${a.start}-${a.end.toFixed(1)}s, ${b.src} at ${b.start}-${b.end.toFixed(1)}s). This causes layered audio playback.`,
+          fixHint: "Use non-overlapping time windows or different track indices.",
+        });
+      }
+    }
+  }
+  return findings;
+}
+
 /**
 * Determine whether a render should be blocked based on lint results and strict mode.
 * --strict blocks on errors; --strict-all blocks on errors or warnings.