mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
87f4c77e2f
* feat(cli): add website capture with AI-powered DESIGN.md generation
Adds `hyperframes capture <url>` command that extracts a complete design
system from any website, producing AI-agent-ready output:
- Full-page screenshot (lazy-load aware, nav at top)
- AI-generated DESIGN.md via Claude API (colors, typography, elevation,
components, do's/don'ts) with programmatic asset catalog (136+ assets
with HTML context annotations like img[src], css url(), link[rel=preload])
- CSS-purged compositions (87% size reduction via PurgeCSS)
- HTML-prettified compositions (one-tag-per-line for AI readability)
- CLAUDE.md + .cursorrules auto-generated for AI agent instructions
- Asset deduplication (srcset variants) and tracking pixel filtering
* feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement
- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update session handoff with slack research findings
- key finding: team already wants DESIGN.md integration (James, Bin, Vance)
- skills quality matters enormously - must invoke /hyperframes-compose
- eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide)
- templates at templates/ need study before finalizing skill
- session handoff updated with critical next steps
* refactor(cli): simplify capture pipeline, remove replica generator
* feat(capture): add Lottie detection and WebGL shader extraction
Captures Lottie animations via network interception and WebGL shader
source via gl.shaderSource hooking during site crawl. Updates
website-to-hyperframes skill with asset planning guidance, Lottie/shader
reading instructions, and stronger creative direction for scene planning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(capture): clean pipeline + shader-first creative workflow
Capture pipeline:
- Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps)
- Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md)
- Remove dead canvas-to-image code in htmlExtractor (post canvas removal)
- Parallelize image downloads (batches of 5 via Promise.allSettled)
- Fix pre-existing TS error (match[1] guard in font downloader)
- Default capture output to captures/<hostname>
Skill creative overhaul:
- Add shader transition selection to creative director step (Step 4)
- Add shader wiring instructions to engineer step (Step 5)
- Replace 4-line energy modifiers with visual vocabulary table
- Strip rigid scene-by-scene templates from video-recipes.md
- Strip example fill data from scene plan tables
- Add "read transition refs before planning" instruction
- Add creative ambition language ("how the hell did they make this")
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add skill architecture redesign spec
Comprehensive redesign of website-to-hyperframes skill and capture
pipeline based on code review findings and Claude Code architecture
research. Key changes: remove AI auto-generation, restructure skill
into phases, embed shader boilerplate in scaffold, fix color format.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add implementation plan for skill architecture redesign
13-task plan covering: capture pipeline cleanup (remove AI generation,
fix colors to HEX, add asset descriptions, shader-ready scaffold),
skill restructuring (4 phases with artifact gates), and compose skill
Visual Identity Gate upgrade.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(capture): remove AI auto-generation and SDK dependencies
* fix(capture): convert extracted colors to HEX format
* refactor(capture): remove AI key path, add asset descriptions generator
* refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions
* feat(capture): pre-wire shader transitions in index.html scaffold
* chore: remove duplicate visual-styles.md (canonical is in hyperframes/)
* refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator
* feat(skill): add Phase 1 understand reference
* feat(skill): add Phase 2 design reference with full DESIGN.md schema
* feat(skill): add Phase 3 creative direction reference
* feat(skill): add Phase 4 build reference with inline shader example
* feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md
* docs: update CLAUDE.md skill references for phase-based workflow
* fix: address code review findings
- Remove orphaned `false` argument in generateAgentPrompt call (critical:
was shifting hasLottie, hasShaders, catalogedAssets parameters)
- Add HSL color handling in rgbToHex via temp element resolution
- Remove build artifact commit section from phase-4-build.md
- Fix __GSAP_TIMELINE reference to __timelines
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(capture): regex double-escape + simplify scaffold + fix asset descriptions
- Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\()
so browser receives valid regex patterns via page.evaluate()
- Simplify index.html scaffold: scene slots + audio + timeline + comment pointing
to shader-setup.md reference (no broken inline shader boilerplate)
- Fix asset descriptions: use CatalogedAsset.contexts/notes instead of
nonexistent htmlContext field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: code review — 16 bugs, 7-step skill rewrite, cleanup
Code fixes:
- snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero
for --frames 1, port bind error handling, rAF-based render settle
- index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini
batch/rate-limit comments, fix video preview viewport y-coordinate
- tokenExtractor.ts: remove dead seen[si] dedup code
- gsap.ts: index ALL classes for inline-style transform conflict detection
Skill architecture rewrite (4-phase → 7-step):
- Replace phase-1 through phase-4 with step-1 through step-7
- Add techniques.md (10 visual techniques with code patterns)
- Fix /hyperframes-compose → /hyperframes (skill doesn't exist)
- Fix captures/arc-browser reference → shader-setup.md (file doesn't exist)
- Fix step-7 hardcoded captures/stripe path
- Document Gemini API free/paid rate limits in step-1
Cleanup:
- CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step
- .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast)
- Delete old phase-*.md, animation-recreation.md, tts-integration.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove dev artifacts, research docs, wrong lockfiles
Remove files that shouldn't ship in this PR:
- docs/research/ (aura analysis, prompt catalogs)
- docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes)
- docs/superpowers/ planning and spec docs
- pnpm-lock.yaml at root and cli (repo uses bun, not pnpm)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention
Main PR #283 removed the full skills table from CLAUDE.md and moved it
to AGENTS.md. Align with that decision: use main's slim dev-focused
format, fix pnpm→bun references, add one-line /website-to-hyperframes
pointer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): add capture command to help groups
The capture command was registered in cli.ts but missing from
the help groups, so it wouldn't appear in `hyperframes --help`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: format skill reference files (oxfmt)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: regenerate bun.lock after rebase
The lockfile was stale after rebasing onto main — bun install
--frozen-lockfile failed in CI because new dependencies (google/genai,
patchright, purgecss) weren't reflected in the lockfile.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review comments + improve capture quality
Review fixes (16 comments from jrusso1020 + vanceingalls):
- screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots
- screenshotCapture: dynamic screenshot count based on page height (30% overlap)
- snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard
- htmlExtractor: fix code injection via parameterized evaluate
- index.ts: video preview re-measures position after scroll, .env file loading
- capture.ts: BLOCKED.md on timeout failures
- gsap.ts: 5 inline-style lint tests added (all pass)
- Remove Playwright, patchright deps; @google/genai to optionalDependencies
- Gitignore: generic patterns instead of 20 hardcoded directories
- Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance)
Capture quality improvements (tested on 10+ websites):
- Color extraction: canvas-based oklch/lab resolver, pixel sampling via
elementFromPoint, broad sweep for accent colors, gradient/shadow extraction
- Section detection: broadened selectors for div-based layouts, height cap
to skip page-level wrappers, parent bg walkup for dark sites
- Font downloads: cap 6 per family / 30 total (Cal.com: 306→30)
- CTA detection: text pattern matching + nav context filtering
- Heading text: innerText with whitespace normalization
- Gemini captioning: maxOutputTokens 100→300, .env auto-loading
- .env.example updated with GEMINI_API_KEY docs
- TTS ranking: Kokoro first with Python 3.10+ note
* fix: address PR review comments + improve capture quality
Review round 2 fixes (jrusso1020 + vanceingalls):
- verify/index.ts: add path traversal guard (relative + isAbsolute)
- verify/index.ts: fix sections[i] undefined typecheck error (CI green)
- index.ts: escape Lottie JSON with \u003c to prevent </script> breakout
- step-4-storyboard: fix technique count contradiction (2-3 per beat, not
across whole video)
- step-6-build: perspective tilt uses gsap.set() instead of CSS transform
(avoids GSAP overwrite conflict)
- step-1-capture: reorder — command first, Gemini note after (zero-config
is the default path, API key is optional enhancement)
- step-7-validate: add tsx fallback for snapshot command
- step-3-script: vary hook patterns, don't default to number every time
- assetDownloader: exempt SVGs from 10KB minimum filter (company logos
like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets)
Note: adm-zip was NOT removed (reviewer #3) — it's still in
packages/cli/package.json:30. The root package.json had patchright
and purgecss removed, not adm-zip.
Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms
zero references in the entire codebase. The @anthropic-ai/sdk dependency
was removed earlier in this branch.
* refactor(capture): split index.ts (1175 to 566 lines) into modules
Mechanical extraction, zero logic changes.
New files:
- mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots
- contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions
- scaffolding.ts (135 lines): .env loading, project scaffold generation
Also fixes false-positive BLOCKED.md with structural Cloudflare detection.
Tested on 20 websites, pre/post output identical.
* chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss)
The --split feature auto-generates compositions from captured HTML — a
different approach from the /website-to-hyperframes skill workflow where
agents build compositions from scratch using the storyboard.
No skill file, no step reference, and no test session ever used --split.
Removes 923 lines of unused code + purgecss dependency.
Backed up to ~/Desktop/capture-split-backup/ for reference.
* fix(security): add ssrf protection, lottie injection fix, oom guard
- assetDownloader: add isPrivateUrl() guard blocking private IP ranges
(127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata
endpoints, localhost, and non-HTTP schemes
- mediaCapture: fix Lottie JSON injection by loading shell HTML first
then passing animation data via parameterized page.evaluate()
- index.ts: check Content-Length header before response.buffer() in
Lottie network interception to avoid OOM on multi-GB responses
* fix(capture): security fixes, timeout, sub-agent dispatch instructions
Security (from miguel-heygen review):
- assetDownloader: export isPrivateUrl() SSRF guard
- htmlExtractor: add isPrivateUrl check before CSS fetch
- mediaCapture: add isPrivateUrl check before Lottie fetch
- mediaCapture: fix previewPage leak (try/finally)
- mediaCapture: skip Lottie files > 2MB for preview (CDP limit)
- contentExtractor: skip images > 4MB for Gemini captioning
- index.ts: check Content-Length before response.buffer() (OOM guard)
- snapshot.ts: register error handler before server.listen()
Capture improvements:
- Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare)
- step-6-build: sub-agent dispatch template with explicit rules:
pass file PATHS not contents, use local fonts not Google Fonts,
verify ../assets/ references after each beat
* fix(capture): catalog before DOM mutation, networkidle2, faster Gemini
Critical: asset cataloger now runs BEFORE extractHtml which converts img
src to data URLs. Framer sites like heykuba.com went from 2 to 78 images.
- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load wait: scroll to bottom, wait for img.complete
- CSS background-image cataloging for Framer/Webflow
- SVG naming: checks class, id, parent, inner text (not just aria-label)
- Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img)
- maxOutputTokens 300->500, descriptions sorted captioned-first
- Remove tsx fallback from step-1 (reviewer nit, published CLI has it)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
6.0 KiB
6.0 KiB
| name | description |
|---|---|
| website-to-hyperframes | Capture a website and create a HyperFrames video from it. Use when: (1) a user provides a URL and wants a video, (2) someone says "capture this site", "turn this into a video", "make a promo from my site", (3) the user wants a social ad, product tour, or any video based on an existing website, (4) the user shares a link and asks for any kind of video content. Even if the user just pastes a URL — this is the skill to use. |
Website to HyperFrames
Capture a website, then produce a professional video from it.
Users say things like:
- "Capture https://... and make me a 25-second product launch video"
- "Turn this website into a 15-second social ad for Instagram"
- "Create a 30-second product tour from https://..."
The workflow has 7 steps. Each produces an artifact that gates the next.
Step 1: Capture & Understand
Read: references/step-1-capture.md
Run the capture, read the extracted data, and build a working summary using the write-down-and-forget method.
Gate: Print your site summary (name, top colors, fonts, key assets, one-sentence vibe).
Step 2: Write DESIGN.md
Read: references/step-2-design.md
Write a simple brand reference for the captured website. 6 sections, ~90 lines. This is a cheat sheet, not the creative plan — that comes in Step 4.
Gate: DESIGN.md exists in the project directory.
Step 3: Write SCRIPT
Read: references/step-3-script.md
Write the narration script. The story backbone. Scene durations come from the narration, not from guessing.
Gate: SCRIPT.md exists in the project directory.
Step 4: Write STORYBOARD
Read: references/step-4-storyboard.md
Write per-beat creative direction: mood, camera, animations, transitions, assets, depth layers, SFX. This is the creative north star — the document the engineer follows to build each composition.
Gate: STORYBOARD.md exists with beat-by-beat direction and an asset audit table.
Step 5: Generate VO + Map Timing
Read: references/step-5-vo.md
Generate TTS audio, transcribe for word-level timestamps, and map timestamps to beats. Update STORYBOARD.md with real durations.
Gate: narration.wav (or .mp3) + transcript.json exist. Beat timings in STORYBOARD.md updated.
Step 6: Build Compositions
Read: The /hyperframes skill (invoke it — every rule matters)
Read: references/step-6-build.md
Build each composition following the storyboard. After each one: self-review for layout, asset placement, and animation quality.
Gate: Every composition has been self-reviewed. No overlapping elements, no misplaced assets, no static images without motion.
Step 7: Validate & Deliver
Read: references/step-7-validate.md
Lint, validate, preview. Create a HANDOFF.md for multi-session continuity.
Gate: npx hyperframes lint and npx hyperframes validate pass with zero errors.
Quick Reference
Video Types
| Type | Duration | Beats | Narration |
|---|---|---|---|
| Social ad (IG/TikTok) | 10-15s | 3-4 | Optional hook sentence |
| Product demo | 30-60s | 5-8 | Full narration |
| Feature announcement | 15-30s | 3-5 | Full narration |
| Brand reel | 20-45s | 4-6 | Optional, music focus |
| Launch teaser | 10-20s | 2-4 | Minimal, high energy |
Format
- Landscape: 1920x1080 (default)
- Portrait: 1080x1920 (Instagram Stories, TikTok)
- Square: 1080x1080 (Instagram feed)
Reference Files
| File | When to read |
|---|---|
| step-1-capture.md | Step 1 — reading captured data |
| step-2-design.md | Step 2 — writing DESIGN.md |
| step-3-script.md | Step 3 — writing the narration script |
| step-4-storyboard.md | Step 4 — per-beat creative direction |
| step-5-vo.md | Step 5 — TTS, transcription, timing |
| step-6-build.md | Step 6 — building compositions with self-review |
| step-7-validate.md | Step 7 — lint, validate, preview, handoff |
| techniques.md | Steps 4 & 6 — 10 visual techniques with code patterns (SVG drawing, Canvas 2D, 3D, typography, Lottie, video, typing, variable fonts, MotionPath, transitions) |