ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-12 15:20:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(skill): website-to-hyperframes — concept-first authoring + per-beat read protocol

Browse Source 
Rewrite of the website-to-hyperframes skill that came out of 11
evaluation rounds. The honest read of those evals: prose-only
guidance had hit its ceiling — sub-agents kept reporting "0 errors,
looks good" without doing the work, producing slideshow-quality
videos with mismatched brand colors, missing logos, and beats that
didn't serve the storyboard. This restructure addresses the
failure modes that real videos showed, not theoretical ones.

**Step structure (replaces 7-step layout with concept-first 6-step)**

Old: capture → design → script → storyboard → vo → build → validate
New: capture → design → brief → storyboard → vo → build → validate

The brief step (Step 2) is new: a conversation-shaped step that
aligns message + audience + arc before any beat-writing happens.
Concept-first throughout — message → arc → beats that serve the arc
→ which assets and techniques bring each beat to life.

**Step 0 (capture)**

- "View the contact sheets — carefully, every cell, not a glance"
  closes the failure mode where agents reported "viewed the contact
  sheet" after one scroll and later wrote beats referencing assets
  that didn't exist or missed the brand logo.
- Names the right artifacts to read in order (tokens.json →
  design-styles.json → asset-descriptions.md → fonts-manifest.json),
  with read-on-demand guidance for the rest.

**Step 1 (design)**

- DESIGN.md authoring guide. Restored component CSS sections
  (Component Stylings, Spacing & Layout, Depth & Elevation) that
  earlier batches over-collapsed.

**Step 2 (brief)**

- Strategy/messaging step. Clear instruction for "Surprise me" /
  minimal direction: state the minimum context (where the video
  runs, who it's for) and proceed bold.

**Step 3 (storyboard + script)**

- Concept gate at the top — answer "what makes this video distinct"
  before writing beat 1.
- Brand-floor MUST rules (logo in opener + closer; signature visual
  somewhere in the video).
- Captured assets (SVG logos, illustrations, hero art, gradients)
  are first-class beat content alongside composed UIs — many of
  them carry beats outright. The constraint is only that you start
  from the message, not the asset inventory.

**Step 4 (vo)**

- TTS ranking: HeyGen first (auto word timestamps), ElevenLabs
  second, Kokoro free. Audio timing reconciliation gate: if actual
  audio duration ≠ storyboard planned ±15%, rescale beats or trim
  script before Step 5.

**Step 5 (build) + beat-builder-guide.md**

- Sub-agent template now pastes brand values inline rather than
  telling the sub-agent to re-read DESIGN.md. Targeted file reads
  with specific sections + line ranges.
- "Patterns that ARE shots" affirmative list (captured logo
  draw-on, hero illustration push-in, captured screenshot with
  parallax layers, kinetic typography over captured asset).
- Webpage-mimicry patterns (full CSS browser chrome, parked-camera
  composition, ±2px breathing motion) marked ⚠ rather than ❌ —
  fine when the storyboard genuinely calls for them as the subject.
- Required cinematography per beat: shot type, camera move, depth
  strategy, purpose.

**Step 6 (validate) — per-beat read protocol**

This replaces the previous "spawn verify-beats CLI" gate. A grep
of composition HTML can catch structural lies (missing hex codes,
wrong asset paths) but it can't catch boring beats, off-screen
logos, GSAP timelines that only cover the first 2 seconds, or
camera moves that don't match the storyboard. Those failures only
surface when somebody opens the file and reads it.

Per-beat verdict template names the brand hex codes used, captured
asset paths referenced, headline `font-size`, GSAP timeline
coverage, and storyboard alignment. Critic sub-agent scores a
"Captured asset utilization" dimension specifically so the eval
captures whether captured SVGs/illustrations carried beats or got
recreated as divs.

**Asset bundle**

- 20 Pixabay-licensed SFX files with `CREDITS.md` documenting
  provenance. SFX assignment moved to Step 3 (creative decision)
  so Step 5 implements rather than improvises.
- Capabilities reference + html-in-canvas-patterns updated:
  Three.js 0.181.2 + ESM jsm imports, mulberry32 seeded PRNG for
  deterministic shatter, 24-effect text-animation catalog
  referenced (catalog itself lands in the hyperframes-skill PR).
- Visual vocabulary rewritten: replaces user-word lookup tables
  with brand-first derivation across 6 axes; user words land as
  modifiers, not replacements.
This commit is contained in:
ukimsanov
2026-05-21 11:08:51 -07:00
parent 2c9544f6d4
commit a1ffb6e7bf
39 changed files with 3276 additions and 864 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/website-to-hyperframes/SKILL.md
+118 -49
View File
@@ -6,7 +6,31 @@ description: |
# Website to HyperFrames
Capture a website, then produce a professional video from it.
Capture a website, then produce a professional video from it — collaboratively with the user.
**Take your time on thinking and reviewing.** Quality matters more than speed. Read every reference file the steps point to. Look at every snapshot carefully. If a composition looks weak, revise it before moving on. A polished video is worth more than a rushed one delivered 5 minutes faster. That said: don't sit idle on stuck commands — escalate immediately if a process hangs (see Step 4 for escalation order).
**This is a collaborative workflow by default.** At key moments (marked 💬), you stop and ask the user what they want and refine based on their feedback.
**Autonomous mode exception:** If the user says "decide for me", "just build it", "surprise me", or gives any signal they don't want to be asked questions — skip ALL 💬 gates. Make all creative decisions yourself (video type, style, voice, storyboard), and present the finished result for feedback at the end. Do not ask four separate questions across four separate steps. Read the room once and commit.
**Sub-agent mode (default):** Step 5 dispatches one sub-agent per beat. Each sub-agent reads [beat-builder-guide.md](references/beat-builder-guide.md), builds, lints, snapshots, and verifies its own beat before reporting back. The main agent assembles the final video and does a final check.
**No sub-agents:** If the user says "no sub-agents", "build it yourself", or the runtime doesn't support parallel agents — the main agent builds all compositions sequentially using the same beat-builder-guide workflow. Same quality, just slower.
**This skill requires image-viewing capability** for the validate step (Step 6). If your agent cannot view PNG files, the snapshot review will be blind. Contact sheets (Step 0 and Step 6) are designed to minimize the number of images needed — but some visual verification is unavoidable.
---
## The Creative Tension Principle
Before writing the first beat of any storyboard, answer this in one sentence:
> **"What makes this video different from a generic [video type] for any [industry] brand?"**
If you can't answer it, you haven't thought enough. A product demo for a fintech tool and a product demo for a design tool should not share the same visual DNA. The answer comes from this specific brand's captured assets, its visual language, and what the user said they want — not from a lookup table.
This principle applies at every creative decision point: picking a visual style, choosing transitions, writing beats, building compositions. Every choice should be traceable to something specific about this brand, not just to "this is what I do for cinematic videos."
Users say things like:
@@ -14,79 +38,109 @@ Users say things like:
- "Turn this website into a 15-second social ad for Instagram"
- "Create a 30-second product tour from https://..."
The workflow has 7 steps. Each produces an artifact that gates the next.
---
## Step -1: What we're actually making (REQUIRED before Step 0)
You're not making _a video_. You're making something that **stops scrollers** in the first 1.5 seconds and **feels alive in every single frame** — with motion, depth, momentum, like things exist in a physical world. **Think about how to go viral.** Slow intros are for cinematic trailers; videos shipping anywhere social or feed-based need a hook that beats the 1.5-second scroll threshold.
**Use the captured assets.** Open every SVG in `capture/assets/svgs/`. Open every illustration in `capture/assets/`. Read the descriptions in `capture/extracted/asset-descriptions.md`. Many of these will carry beats outright — the brand logo SVG drawing itself stroke-by-stroke, the hero illustration breathing as ambient depth, the captured gradient as a full-bleed background, the brand mark stamped onto every scene as identity. These are video gold. The capture exists because the brand's actual visual identity matters; using it is what makes the video feel like _this_ brand and not a generic dark cinematic template.
**Compose when there's no captured asset that fits.** For product UI sections where a clean captured asset doesn't exist (kanban boards, chat threads, dashboards, terminals, counters, code editors), build them from divs/SVG/CSS rather than pasting a screenshot. SVG path drawing, kinetic typography, counter animations via `tl.set()`, layered panels, shader-driven gradients — these are all part of HyperFrames' toolkit when no captured asset earns the beat.
**The screenshot trap is specifically about raw product-UI screenshots being pasted as full-bleed beat content.** Captured SVGs, illustrations, logos, hero art, gradients, photography — none of these are screenshots. They're brand assets. Use them. The trap is the "full-bleed dashboard.png + Ken Burns + voiceover" slideshow pattern, not the careful use of captured brand artifacts.
**This is a VIDEO, not a webpage rebuilt in divs.** Composing from divs is the right _medium_ (no screenshots) — but the WRONG outcome is to build a webpage-style layout and animate it 2 pixels. Videos use cinematic grammar: framing, depth, camera movement, scale, atmosphere. A kanban in a video is not "a kanban board centered in the frame at 80% scale with cards breathing 1px" — it's a SHOT: extreme close-up on a card sliding home, then the camera pulls back to reveal the full board, ambient particles + glow + depth give it weight. The composed divs are the subject; the cinematography is what makes it feel like film.
**Specific anti-patterns to refuse, every time:**
- **macOS window chrome** (traffic-light dots, address bars, browser tabs, breadcrumbs) unless the beat IS about the window/browser as the subject
- **Centered layout with chrome around it** — sidebar + header + content area + footer — that's a screenshot reproduced in CSS, not a shot
- **"Breathing" micro-animations** (y: ±1–2px, scale: 1.01) — invisible at 1080p/4K video scale, useless as motion, a sign the sub-agent ran out of ideas
- **Page-level navigation** — sidebars, headers, footers, breadcrumbs, "back" arrows unless the beat is specifically demonstrating navigation
- **"Settled" beats** where nothing moves except a counter pulse — every beat must have continuous, _visible_ motion across the entire duration
**Video grammar to USE in every beat:**
- **Frame the beat as a SHOT** — close-up / medium / wide / over-the-shoulder / Dutch angle. Pick deliberately, not "centered."
- **Camera motion is a primary element** — dolly in, push, parallax pan, orbit, pull-back. The camera moves THROUGH the composition; the composition doesn't sit still in front of the camera.
- **Scale as energy** — enter at 1.4× and settle to 1.0; extreme close-up that pulls back to wide; the subject grows or shrinks through the beat
- **Depth layers** — ambient background atmosphere, focal midground subject, accent foreground element — each moving at different parallax speed
- **Light as choreography** — glow tracks the subject, key moments lit with bloom, transitions happen through light shifts
- **Real motion magnitudes** — 30–100px movements, 0→1 opacity reveals, 0.7→1.0 scale changes — values that READ at video scale. Tiny micro-movements feel like a still image with twitches.
---
## Step 1: Capture & Understand
## Step 0: Capture & Understand the Brand
**Read:** [references/step-1-capture.md](references/step-1-capture.md)
**Read:** [references/step-0-capture.md](references/step-0-capture.md)
Run the capture, read the extracted data, and build a working summary using the write-down-and-forget method.
Capture the site, then read the extracted data to understand the **brand and product** — what it does, who it's for, what voice it speaks in, what mood it lives in. The captured assets are a brand toolkit for later, not the building blocks the video is made from.
**Gate:** Print your site summary (name, top colors, fonts, key assets, one-sentence vibe).
**Gate:** Site summary printed — strategy-first (what the product does, who it's for, brand voice) before the asset / color / font inventory.
---
## Step 2: Write DESIGN.md
## Step 1: Brand Identity
**Read:** [references/step-2-design.md](references/step-2-design.md)
**Read:** [references/step-1-design.md](references/step-1-design.md)
Write a simple brand reference for the captured website. 6 sections, ~90 lines. This is a cheat sheet, not the creative plan — that comes in Step 4.
Write DESIGN.md — a brand cheat sheet covering the visual identity: colors, typography, component styles, layout principles. Use `design-styles.json` for exact computed values.
**Gate:** `DESIGN.md` exists in the project directory.
**Speed option:** For fast-pacing videos (billboard-per-beat), DESIGN.md can be a 50-line summary of colors + fonts + do's/don'ts — not a 300-line document. The sub-agent prompt in Step 5 pastes brand values directly, so DESIGN.md depth only matters for complex compositions.
**Gate:** `DESIGN.md` exists (any length) with at minimum: color palette, font choices, and do's/don'ts.
---
## Step 3: Write SCRIPT
## Step 2: Strategy & Messaging
**Read:** [references/step-3-script.md](references/step-3-script.md)
**Read:** [references/step-2-brief.md](references/step-2-brief.md), [references/visual-vocabulary.md](references/visual-vocabulary.md), [references/capabilities.md](references/capabilities.md) (scan the Table of Contents — deep-dive sections only as needed)
Write the narration script. The story backbone. Scene durations come from the narration, not from guessing.
Align with the user on **what the video must communicate** before talking visuals or assets. Parse the user's prompt — they probably already gave you the video type and style. Ask only what's missing: the ONE thing this video must say, the narrative arc, and the audience.
**Gate:** `SCRIPT.md` exists in the project directory.
**Gate:** Video type, duration, format, and — critically — the message and narrative arc are locked. Without those, Step 3 can't write a concept-first storyboard.
---
## Step 4: Write STORYBOARD
## Step 3: Storyboard + Script 💬
**Read:** [references/step-4-storyboard.md](references/step-4-storyboard.md)
**Read:** [references/step-3-storyboard.md](references/step-3-storyboard.md)
Write per-beat creative direction: mood, camera, animations, transitions, assets, depth layers, SFX. This is the creative north star — the document the engineer follows to build each composition.
Write the storyboard concept-first: message → narrative arc → beats that serve the arc → techniques per beat → brand accents pass at the end. Then write the narration script to match. Present both to the user with a beat-by-beat summary. Iterate until they approve.
**Gate:** `STORYBOARD.md` exists with beat-by-beat direction and an asset audit table.
**Gate:** `STORYBOARD.md` + `SCRIPT.md` exist AND the user has approved the plan.
---
## Step 5: Generate VO + Map Timing
## Step 4: VO, Timing + Captions 💬
**Read:** [references/step-5-vo.md](references/step-5-vo.md)
**Read:** [references/step-4-vo.md](references/step-4-vo.md)
Generate TTS audio, transcribe for word-level timestamps, and map timestamps to beats. Update STORYBOARD.md with real durations.
If Step 2 said no narration — ask about background music, then skip to Step 5. Otherwise: ask the user which TTS provider (HeyGen TTS, ElevenLabs, or Kokoro), generate audio, transcribe, map timestamps to beats. Then ask about captions.
**Gate:** `narration.wav` (or .mp3) + `transcript.json` exist. Beat timings in STORYBOARD.md updated.
**Gate:** Either (a) no narration was requested and storyboard has manual beat timings, or (b) `narration.wav` + `transcript.json` exist and beat timings updated with real durations.
---
## Step 6: Build Compositions
## Step 5: Build Compositions
**Read:** The `hyperframes` skill (load it — every rule matters)
**Read:** [references/step-6-build.md](references/step-6-build.md)
**Read:** [references/step-5-build.md](references/step-5-build.md)
Build each composition following the storyboard. After each one: self-review for layout, asset placement, and animation quality.
Build index.html and compositions following the architecture and pacing chosen in the storyboard (Step 3). Sub-agents run `hyperframes lint` and `hyperframes snapshot` on each beat before reporting back.
**Gate:** Every composition has been self-reviewed. No overlapping elements, no misplaced assets, no static images without motion.
**Gate:** **The main agent does NOT trust sub-agents' chat reports.** After every sub-agent completes, the main agent opens each `compositions/beat-N.html` and reads it top-to-bottom. For each beat: does the GSAP timeline use the data attributes correctly, do the brand colors from DESIGN.md actually appear in the CSS, are the captured assets the storyboard called for actually referenced, is the headline at video-readable size, does the beat serve the storyboard arc? Anything off — fix it inline or re-dispatch the sub-agent with the specific problem quoted.
---
## Step 7: Validate & Deliver
## Step 6: Validate & Deliver
**Read:** [references/step-7-validate.md](references/step-7-validate.md)
**Read:** [references/step-6-validate.md](references/step-6-validate.md)
Lint, validate, snapshot, preview. Deliver the localhost Studio project URL
(`http://localhost:<port>/#project/<project-name>`) to the user first — only
render to MP4 on explicit request. Do not treat `index.html` as the project
handoff link; it is source-code context only.
Lint, validate, take snapshots scaled to video length (formula: `max(beats × 3, ceil(duration_seconds / 2))`), and review each one. Fix issues before delivering. Deliver the localhost Studio project URL — only render to MP4 on explicit user request.
**Deliver something you're proud of.** Before handing off, ask yourself: would I post this on social media with my name on it? If not, fix what's wrong.
**Gate:** `npx hyperframes lint` and `npx hyperframes validate` pass with zero errors, and the final response includes the active Studio project URL.
@@ -96,13 +150,17 @@ handoff link; it is source-code context only.
### Video Types
| Type | Duration | Beats | Narration |
| --------------------- | -------- | ----- | ---------------------- |
| Social ad (IG/TikTok) | 10-15s | 3-4 | Optional hook sentence |
| Product demo | 30-60s | 5-8 | Full narration |
| Feature announcement | 15-30s | 3-5 | Full narration |
| Brand reel | 20-45s | 4-6 | Optional, music focus |
| Launch teaser | 10-20s | 2-4 | Minimal, high energy |
Typical constraints by video type — use as a starting point, not a formula. Beat count should follow from the content and the narration, not from a target range.
| Type | Typical duration | Duration driver | Narration |
| --------------------- | ---------------- | ------------------ | --------------------- |
| Social ad (IG/TikTok) | 10–15s | Platform limit | Optional |
| Product demo | 30–60s | Script length | Full narration |
| Feature announcement | 15–30s | Feature complexity | Full narration |
| Brand reel | 20–45s | Music track | Optional, music focus |
| Launch teaser | 10–20s | Hook energy | Minimal |
Beat count is not in this table intentionally — it should come from the storyboard, not from "social ad = 3-4 beats." A social ad for a complex product might need 5 well-timed beats. A brand reel with one strong visual thesis might need 3.
### Format
@@ -110,15 +168,26 @@ handoff link; it is source-code context only.
- **Portrait**: 1080x1920 (Instagram Stories, TikTok)
- **Square**: 1080x1080 (Instagram feed)
### User Interaction Points
| Step | What to ask | Why |
| ---------------------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Step 2 (Strategy) | Message, narrative arc, audience, video type, style, format | The story is what every downstream choice flows from. Without it, beats are arbitrary. |
| Step 3 (Storyboard + Script) | Beat-by-beat approval, script review | Cheapest place to iterate. 30s to change a beat, 5min to rebuild a composition. |
| Step 4 (VO) | TTS provider choice, API key if needed | Voice quality makes or breaks the video. User may have provider preferences. |
### Reference Files
| File | When to read |
| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [step-1-capture.md](references/step-1-capture.md) | Step 1 — reading captured data |
| [step-2-design.md](references/step-2-design.md) | Step 2 — writing DESIGN.md |
| [step-3-script.md](references/step-3-script.md) | Step 3 — writing the narration script |
| [step-4-storyboard.md](references/step-4-storyboard.md) | Step 4 — per-beat creative direction |
| [step-5-vo.md](references/step-5-vo.md) | Step 5 — TTS, transcription, timing |
| [step-6-build.md](references/step-6-build.md) | Step 6 — building compositions with self-review |
| [step-7-validate.md](references/step-7-validate.md) | Step 7 — lint, validate, snapshot, preview |
| [techniques.md](../hyperframes/references/techniques.md) | Steps 4 & 6 — 11 visual techniques with code patterns (SVG drawing, Canvas 2D, 3D, typography, Lottie, video, typing, variable fonts, MotionPath, transitions, audio-reactive) |
| File | When to read |
| ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| [step-0-capture.md](references/step-0-capture.md) | Step 0 — capture, understand the brand and product, write strategy-first site summary |
| [step-1-design.md](references/step-1-design.md) | Step 1 — write DESIGN.md brand cheat sheet (6 sections, 250-350 lines) |
| [step-2-brief.md](references/step-2-brief.md) | Step 2 — align on message, narrative arc, audience with user |
| [capabilities.md](references/capabilities.md) | Steps 2 & 5 — full inventory of what HyperFrames can do (24 sections). Scan the TOC during the brief, deep-dive specific sections during build |
| [visual-vocabulary.md](references/visual-vocabulary.md) | Step 2 & 3 — translate subjective terms to concrete techniques. Composable building blocks, not rigid presets |
| [step-3-storyboard.md](references/step-3-storyboard.md) | Step 3 — storyboard + script (combined) with user review gate |
| [step-4-vo.md](references/step-4-vo.md) | Step 4 — TTS provider choice, generation, timing |
| [step-5-build.md](references/step-5-build.md) | Step 5 — build index.html + compositions |
| [step-6-validate.md](references/step-6-validate.md) | Step 6 — lint, validate, snapshots (scaled to video length), preview |
| [techniques.md](../hyperframes/references/techniques.md) | Steps 3 & 5 — 20 visual techniques with code patterns (adapt, don't copy-paste) |
| [html-in-canvas-patterns.md](../hyperframes/references/html-in-canvas-patterns.md) | Step 5 — complete code patterns for HTML-in-Canvas effects (lives in the hyperframes skill) |