ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,137 Commits 856 Branches 404 Tags
6d2569c6bbc869acd7a175509d0c5813e6fd45a1
Commit Graph
25 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
ukimsanovandClaude Opus 4.7 ce4bcc47c0 fix(cli): use multi-strategy runtime resolver for snapshot 
Use loadRuntimeSource() from runtimeSource.ts instead of a single
hardcoded path. Tries: build from source (dev), inlined constant
(production), pre-built artifact (fallback). Fixes snapshot in dev
mode where __dirname is src/commands/ not dist/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 14:36:34 -04:00
ukimsanov 8cc2084669 fix(cli): snapshot resolves runtime from own dist, not monorepo layout 
The snapshot command resolved the HyperFrames runtime IIFE via a
relative path that walked up three directories to packages/core/dist/.
This only worked in the monorepo dev layout — npm/npx installs have
a flat dist/ folder with cli.js and the runtime side by side.

Without the runtime, window.__player was never created and the
snapshot fell back to seeking every __timelines entry to the same
absolute time. Sub-composition timelines expect relative time
(offset from their data-start), so all beats rendered beat-1 content.

Fix: resolve("hyperframe.runtime.iife.js") from __dirname (the dist/
folder itself), where the build already copies the runtime IIFE.
 2026-04-26 00:53:06 -04:00
ukimsanovandClaude Opus 4.6 67ed767641 fix(docs): use claude.ai/design link, remove raw download option 
- Link to claude.ai/design instead of claude.ai
- Remove raw.githubusercontent download links (just GitHub with ↓ button)
- Fix stale SKILL.md link text in prompting guide

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-25 17:02:58 -04:00
ukimsanovandClaude Opus 4.6 a8f0752bf6 fix(shader-transitions): address review — NaN guard, canvas sync, shared defaults 
- Validate data-width/data-height: fall back to defaults if NaN or <= 0
- Sync existing #gl-canvas dimensions on reuse (if init called twice)
- Import DEFAULT_WIDTH/DEFAULT_HEIGHT in capture.ts instead of
  hardcoding 1920/1080 in parameter defaults

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 21:41:42 -04:00
ukimsanovandClaude Opus 4.6 c81ef50e2f feat(shader-transitions): support any aspect ratio + update skill 
Add dynamic resolution to shader transitions and update Skeleton A
to use shaders on vertical compositions.

shader-transitions changes:
- webgl.ts: read dimensions from params instead of hardcoded constants
- capture.ts: accept width/height for html2canvas
- hyper-shader.ts: read data-width/data-height from composition root

skill + docs changes:
- Skeleton A now has 1 shader at hero reveal (s3→s4 midpoint)
- Removed "no shaders on vertical" limitation from docs
- Updated claude-design.mdx known limitations section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 21:32:09 -04:00
ukimsanovandClaude Opus 4.6 6b84971f6b feat(shader-transitions): support any aspect ratio (vertical, square) 
Read data-width/data-height from the composition root instead of
hardcoding 1920x1080. Enables shader transitions on vertical
(1080x1920) and square (1080x1080) compositions.

Changes across 3 files:
- webgl.ts: WIDTH/HEIGHT constants → DEFAULT_WIDTH/DEFAULT_HEIGHT,
  createContext and renderShader accept width/height params
- capture.ts: captureScene and captureIncomingScene accept
  width/height params for html2canvas
- hyper-shader.ts: reads data-width/data-height from root element,
  passes dimensions to all webgl and capture calls

GLSL shaders unchanged — they already use u_resolution uniform for
all coordinate math and work at any aspect ratio.

Backwards compatible: all params default to 1920x1080 when not
provided or when data-width/data-height are missing from the DOM.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 21:19:49 -04:00
ukimsanovandClaude Opus 4.6 c6da00d9c8 docs: fix SKILL.md download UX across all references 
Raw GitHub URLs serve as text/plain — clicking opens a text tab
instead of downloading. Updated all 5 references across 4 files:

- claude-design.mdx (2 refs): "right-click → Save Link As"
- prompting.mdx (1 ref): "right-click → Save Link As"
- README.md (1 ref): "click download button on GitHub"
- quickstart.mdx (1 ref): "click download button on GitHub"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 21:06:27 -04:00
ukimsanovandClaude Opus 4.6 95ca0200d4 docs: update all Claude Design references for template-first skill 
Update docs, quickstart, prompting guide, and README to reflect:
- Template-first approach (attach file, not paste URL)
- Claude Design produces drafts, refine in any AI coding agent
- Known limitations (vertical shaders, seeking, no linting)
- Practical example prompts (feature announcement, founder pitch)
- Removed outdated references (invisible bridges, fetch-the-skills-tree)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 20:52:40 -04:00
ukimsanovandClaude Opus 4.6 18136d4eb5 fix(shader-transitions): add inWindow guard to .catch fallback handler 
Address Vance's review on #456: the .catch handler was missing the
same tl.time() window check that .then has. Late-rejecting captures
(Safari + SVG-filter compositions) could fire gsap.to/fromTo on
scenes the playhead already left, causing flash-to-black mid-scene.

Wraps the CSS crossfade fallback in the same inWindow guard so
stale catch handlers are no-ops, matching the .then behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 20:17:13 -04:00
ukimsanov ee7c6c3c24 refactor(claude-design-skill): template-first rewrite with bug fixes 
Replace the rule-heavy Claude Design skill with a template-first
approach. Instead of teaching all rules from scratch (863 lines),
provide pre-valid skeletons where structural rules are embedded.
Claude Design fills in palette, content, and animations.

Key changes:
- Pre-valid HTML skeletons for social reel (vertical), launch teaser,
  product explainer, and cinematic title
- Mixed transitions: 2-3 shader transitions at key moments, rest
  hard cuts (matches professional video practice)
- autoAlpha toggles for non-anchor scenes (fixes invisible middle
  scenes caused by HyperShader's blanket opacity reset)
- Explicit first-anchor opacity fix (HyperShader browser mode never
  auto-shows the first anchor scene)
- No shaders on vertical (1080x1920) — WebGL canvas is hardcoded
  to 1920x1080 in webgl.ts
- Inline animation patterns (counter, stroke draw, stagger, float,
  bar chart, orbit, highlight sweep, safe CSS grain)
- Claude Design → Claude Code handoff workflow documented in
  README template and delivery step
- Troubleshooting table for black preview, invisible scenes, seeking
- Self-review checklist split: structural validity, brand accuracy,
  animation baseline
- Skill description updated per review feedback on PR #353 to
  disambiguate from the hyperframes skill on cross-surface routing

Bugs fixed by this rewrite (confirmed across demov4-1 through
demov4-6 and demov5-1 through demov5-5):
- First shader anchor invisible in every composition
- Non-anchor scenes killed by blanket querySelectorAll(".scene")
  opacity reset
- visibility toggles insufficient (opacity:0 persists after reset)
- Shaders on vertical compositions produce distorted transitions
 2026-04-23 20:12:20 -04:00
ukimsanov 41b4017273 docs(shader-transitions): clarify allowTaint caveat in capture comment 
Address Copilot review comment on #456: the old `allowTaint` doc comment
said the resulting canvas is "still usable as a WebGL texture via
gl.texImage2D (no pixel read-back required)", which is wrong. A tainted
canvas CANNOT be uploaded to WebGL — the spec requires SecurityError on
non-origin-clean sources with no opt-out. That's exactly what we observe
in Safari + SVG-filter compositions, and what hyper-shader.ts's catch
handler now handles via CSS crossfade.

Update the comment to correctly describe the flag's effect: it only
moves the failure point from html2canvas to texImage2D; the end-user
UX is the same (smooth CSS fade in either case). The flag remains
defensively correct for the non-taint branches where it genuinely
helps (cross-origin images with `Access-Control-Allow-Origin`).

No code change — comment only.

Made-with: Cursor
 2026-04-23 16:20:32 -04:00
ukimsanov b3b458fda4 fix(shader-transitions): harden capture against visibility, scrub, Safari taint 
Three interrelated fixes for the live-player path in
@hyperframes/shader-transitions (Studio preview, <hyperframes-player>
embeds, Claude Design in-pane iframe). Zero changes to engine mode —
initEngineMode is byte-identical; producer render pipeline and CLI
hyperframes render produce byte-identical output.

1. captureIncomingScene now forces visibility:visible during capture.
   The HF runtime sets visibility:hidden on [data-start] elements outside
   their playback window. With centered shader timing (transition.time =
   boundary - duration/2), html2canvas captures the incoming scene while
   it's still hidden → blank texture → visible blink mid-transition.
   Fix saves, overrides, captures, and restores visibility only for the
   capture window. Empirically validated via a direct html2canvas probe:
   captures of visibility:hidden elements return blank; with override,
   they return real content.

2. post-capture.dom guards on tl.time() window before mutating DOM.
   On scrub across multiple shader transitions, tl.call() fires several
   transitions' callbacks in rapid succession; each launches async
   html2canvas; each .then() unconditionally set all .scene opacities to
   0, enabled shader canvas, and pointed state at that transition. The
   last to resolve won — often for a transition the playhead had left.
   Result: scenes stuck opacity:0 mid-scene; blank screen until the next
   transition's end.call ran. Fix: check tl.time() is still inside
   [T, T+dur] before applying state; otherwise skip.

3. .catch fallback does CSS crossfade instead of hard cut.
   When capture fails (Safari canvas taint from SVG data URLs, CORS
   errors, extreme DOM complexity) the old catch snapped all scenes to
   opacity:0 then set incoming to opacity:1 — jarring instant jump. Fix
   uses gsap.to/fromTo on opacity over the intended transition duration;
   smooth 0.5s fade is strictly better UX. Hard cut preserved as
   last-resort if elements are missing.

Also adds defensive useCORS: true and allowTaint: true to the
html2canvas call. No behavior change in Chrome (capture normally
succeeds); adds resilience for cross-origin images with CORS headers
and SVG-tainted canvases respectively.

Known limitations (out of scope, follow-up tracked):

- Safari + cross-origin iframe: html2canvas is 10-12x slower than Chrome
  due to WebKit's DocumentCloner.cloneNode perf (html2canvas#3108),
  causing perceptible per-transition freezes (1.5-2s each) in Claude
  Design's in-pane preview. Needs pre-capture architecture (cache
  incoming-scene textures at init) to eliminate per-transition cost.
- SVG filter data URLs fundamentally taint html2canvas output in Safari;
  WebGL's texImage2D has no framework opt-out (WebGL spec). Addressed
  at the composition level via the Claude Design skill's anti-pattern 4
  in a parallel PR.

Made-with: Cursor
 2026-04-23 15:15:58 -04:00
ukimsanov 0730f88dc7 fix(cli): address snapshot PR review feedback 
Incorporates four review points on #348:

1. Fix typecheck error — cast the dynamic `@hyperframes/engine` import to a
   typed shape and drop the `as typeof x` cast on a `null`-unioned variable
   that TS rejected. CI `Typecheck` was failing on main because the cast
   conflicted with the declared union.

2. Clear stale injected <img> overlays — always call
   `syncVideoFrameVisibility(page, activeIds)` on every seek (including
   `active.length === 0`), so injected frames from a previous timestamp
   don't leak into later snapshots. The runtime's visibility toggles act
   on the native <video> but not on its injected sibling, which Copilot
   correctly flagged as a leakage source.

3. URL-decode the served video path before resolving to a filesystem path.
   Files whose names contain spaces or other URL-encoded characters were
   skipped because `new URL().pathname` preserves `%20` while the file
   server decodes inbound requests and the file on disk lives at the
   decoded name.

4. Mirror `packages/core/src/runtime/media.ts` media math so clips with a
   non-1 `defaultPlaybackRate` get the correct active window and relTime.
   Specifically: clamp `defaultPlaybackRate` to [0.1, 5], apply
   `(t - start) * rate + mediaStart`, and adjust the duration-fallback
   branch by `/ rate` when only source duration is known.

5. Kill FFmpeg on a 30s timeout so a pathological clip cannot wedge
   snapshot indefinitely. Matches the default in
   `@hyperframes/engine`'s `runFfmpeg`.

Re-verified against the same 4-timestamp A/B on launch-video-2:
MD5s match the ffmpeg-from-render ground truth
  12.5s → ef9684e36fea53a0db7adf7cfcaacad3  (Stripe)
  16.0s → 487494ca16344d55d7181408dc439a56  (Framer)
  20.5s → 8835c34ad2a45755a1c98a7e079427a1  (HeyGen 3D)
  32.5s → 34dc9450f2bd661c12039d7aa82a30b0  (GitHub finale)

No-video projects (basecamp-tour, linear-brand, commissioned/github)
still produce identical output to the pre-fix path. Latency unchanged.

Made-with: Cursor
 2026-04-20 10:30:29 -04:00
ukimsanov a3e7c6176f fix(cli): inject real video frames in snapshot to match render 
The snapshot command previously just called `tl.seek(t)` + `page.screenshot`
and trusted Chrome to advance `<video>`-element decoders. Chrome headless
silently ignores `video.currentTime = X` writes — the setter is accepted
but the decoder never moves. Result: every snapshot of a composition that
uses body-level `<video data-start>` elements renders the same frame
regardless of the requested timestamp (the z-topmost video's first-frame
paints through, because all clips share `position: absolute; inset: 0`
and visibility:hidden doesn't always prevent the GPU surface from
contributing to the composite).

The render pipeline has already solved this: for each body-level video it
extracts the needed frame via FFmpeg and overlays it as an <img> sibling
via `injectVideoFramesBatch` (packages/engine/src/services/screenshot
Service.ts). This commit ports that same primitive into `snapshot`:

1. Added `extractVideoFrameToBuffer(videoPath, t)` — one FFmpeg spawn per
   active video, `-ss` keyframe seek (~100-200 ms), writes a temp PNG.
2. After the existing seek + settle, enumerate `<video data-start>`
   elements that are active at the target time, resolve each one's
   `currentSrc` URL back to a filesystem path under `projectDir`, extract
   the frame, and call `injectVideoFramesBatch`.
3. Then screenshot — as before.

Non-breaking: when no body-level `<video data-start>` elements exist (every
other project in the repo — basecamp, linear, stripe, github component), the
new block short-circuits on `active.length === 0` and behaves identically
to the pre-fix path. Verified against three no-video projects: bit-identical
snapshot output, no latency regression.

Measured on macOS M2 (4 frames, cold):
  launch-video-2 (11 timed videos): 6.48s → 6.16s   (-5%)
  basecamp-tour  (no timed videos): 5.67s → 4.87s   (-14%)

Proof: Pre-fix MD5 at t=12.5, 16.0, 20.5, 32.5 — all 4 identical (wrong frame)
  Post-fix MD5 at same timestamps  — all 4 distinct, match ffmpeg-from-render
Made-with: Cursor
 2026-04-20 09:36:33 -04:00
ukimsanovandClaude Opus 4.6 4e87f28cd9 fix: address PR #339 review — 7 issues 
1. Catalog failure safety: warn when catalog is empty or throws, so
   capture doesn't silently produce zero images
2. Dead path: extract-audio-data.py → skills/gsap/scripts/ (was
   skills/hyperframes/scripts/)
3. --json fonts compat: emit both `fonts` (string[]) and
   `fontsDetailed` (FontToken[]) to avoid breaking external consumers
4. Restore .cursorrules writing alongside AGENTS.md + CLAUDE.md
5. .gitignore: remove over-broad `projects/` and `videos/` entries,
   keep scoped `cursor-tests/` and `launch-video*/`
6. agentPromptGenerator: mark unused params as reserved with comments,
   remove _animations from buildPrompt
7. Cookie filter: threshold 20 → 8 chars to preserve footer copy like
   "© 2026 Stripe" (16 chars) and "Privacy & Terms" (15 chars)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-19 15:34:15 -04:00
ukimsanov 517e327294 docs(skill): render step is on-demand, not automatic 
Preview is the delivery — scrub through it, iterate on tweaks, only render
once the user explicitly asks ("render it", "make the final", "I'm happy").
Rendering takes minutes per pass and is wasted work if the user wants any
changes after seeing it.

- step-7-validate: rename Render section to "Render (on-demand only)" and
  make the trigger criteria explicit
- SKILL.md: step-7 summary updated to "Deliver the preview to the user
  first — only render to MP4 on explicit request"

Made-with: Cursor
 2026-04-19 08:43:59 -04:00
ukimsanov 1a69cde4be docs(skill): shader/audio/render/snapshot guidance from regression evidence 
Adds decision guidance and conventions the 8-site regression test exposed as
blind spots — agents had the capability but never reached for it.

step-1-capture:
- Clarify capture goes into <project-dir>/capture/ so capture artifacts stay
  isolated from later build files (SCRIPT/STORYBOARD/DESIGN/compositions/)
- 7/8 regression tests already did this; codify as the convention

step-4-storyboard:
- Add "When to pick which" decision table for shader vs CSS vs hard cut
  transitions. Shader transitions were available but used in 0/8 tests —
  every test defaulted to CSS. The table frames shaders as "reveals, reaction
  shots, brand moments" vs CSS as "connective tissue"
- Update technique count (10 → 11)

step-6-build:
- Mid-scene activity table gets a new row for audio-reactive logo/CTA
  animation (bass pulse, treble glow). Audio-reactive was used in 0/8 tests
  despite narration being present in all of them

step-7-validate:
- Snapshot section: explicit "use hyperframes snapshot, don't roll custom"
  with the default naming pattern spelled out. Stripe's run used custom
  ffmpeg naming (beat-6-cta-at-20.5s.png) instead of frame-XX-at-Ys.png
- New render section: require --output renders/<project>.mp4 so final MP4s
  have predictable names. Without this, 7/8 tests produced wildly different
  filenames (preview.mp4, cal_2026-04-19_20-29-21.mp4, basecamp.mp4, etc.)

techniques.md:
- New technique #11: Audio-Reactive Animation. Covers the sampling pattern
  (per-frame tl.call, not single tween), when to use (music/dramatic VO
  videos), intensity ranges (3-5% for text/logos, 10-30% for backgrounds),
  and anti-patterns (equalizer bars, waveforms, strobing). Cross-references
  skills/hyperframes/references/audio-reactive.md for the full API

Made-with: Cursor
 2026-04-19 08:41:53 -04:00
ukimsanov d8f1af1ef1 feat(capture): write AGENTS.md alongside CLAUDE.md + skill refinements from regression tests 
Capture pipeline:
- agentPromptGenerator now writes AGENTS.md + CLAUDE.md (drop legacy
  .cursorrules), matching the dual-file convention already used by the
  _shared templates in hyperframes init. AGENTS.md is picked up natively by
  Cursor, Codex, Gemini CLI, Windsurf, Aider, and Jules; CLAUDE.md covers
  Claude Code. Both files share the same content — a capture data inventory
  that points agents to the website-to-hyperframes skill.

website-to-hyperframes skill refinements (derived from 8-site regression test):
- Drop slash-command phrasing throughout SKILL.md and step-6-build.md so the
  skill works identically across Claude Code (slash), Cursor (auto-discover
  by description), and other agents.
- Remove stale HANDOFF.md references from SKILL.md step-7 summary and
  reference table — matches the intent of the prior step-7 cleanup.
- step-5-vo: specify narration.txt filename convention (pronunciation-
  substituted spoken text; distinct from SCRIPT.md the creative doc).
- step-6 self-review adds three rules derived from actual lint warnings
  observed across the 8 regression runs:
    - Every <template> root needs data-start + data-duration (catches
      root_composition_missing_data_start/duration, seen in 4/8 runs).
    - Caption exits need a hard tl.set kill after tl.to(opacity:0), or
      per-word karaoke tweens can leave captions stuck on screen
      (caption_exit_missing_hard_kill).
    - No duplicate media nodes with identical src + start + duration, or
      the compiler discovers them twice (duplicate_media_discovery_risk).

Housekeeping:
- .gitignore: add cursor-tests/, basecamp-video/, projects/, videos/ —
  local regression-test scratch dirs that should never be committed.
- Remove two broken symlinks from .claude/skills/ that pointed to paths
  which never existed in the repo (.claude/skills/ is already gitignored).

Made-with: Cursor
 2026-04-18 23:17:29 -04:00
ukimsanov 92a5ef419b feat(capture): improve capture quality + clean up CLAUDE.md 
Capture improvements:
- Font weights via document.fonts API + DOM sampling (variable font detection)
- Section background-image extraction (no more false #FFFFFF on hero photos)
- Detected libraries surfaced in CLAUDE.md brand summary
- Structured visible-text.txt with [tag] prefixes, cookie/nav noise filtered
- tokens.json cleaned: removed images/paragraphs/icons (duplicated elsewhere),
  filtered sections to heading-only, trimmed cssVariables to design-relevant
- Removed redundant scroll pass in htmlExtractor (2-5s faster per capture)
- Font cap at 20 families, Placeholder/Fallback fonts filtered

CLAUDE.md rewrite:
- Removed prescriptive tone ("use exact strings" → "rephrase freely")
- Removed fluff sections (How to Create, DESIGN.md warning, Example Prompts,
  Source Patterns)
- asset-descriptions.md promoted to first data row
- Removed assets-catalog.json from inventory

Skill fixes:
- Dead shader refs → point to packages/shader-transitions/README.md
- Google Fonts import in techniques.md → local @font-face placeholder
- Added Stripe DESIGN.md as light-brand example
- Removed HANDOFF.md generation from step-7
- Updated step-1 for new font weight + visible-text formats
 2026-04-18 18:48:37 -04:00
ukimsanov 5cb3726f27 Merge remote-tracking branch 'origin/main' into feat/capture-improvements-v2 2026-04-18 17:16:42 -04:00
ukimsanov 65011f3b57 Merge remote-tracking branch 'origin/main' into feat/capture-improvements-v2 2026-04-17 13:39:44 -04:00
ukimsanovandClaude Opus 4.6 274db7a5ef fix: address PR #299 review — lint correctness, docs, Gemini benchmark 
- lintMultipleRootCompositions: scan filesystem for HTML files with
  data-composition-id (was filtering results array — always 1 entry)
- lintDuplicateAudioTracks: order-independent attribute extraction,
  dedup by (src,start,duration,trackIndex), Infinity fallback for
  missing data-duration (matches runtime behavior)
- 10 new tests for both lint rules
- docs: explicit skill invocation, remove gsap-skills, fix indentation
- Gemini: env override (HYPERFRAMES_GEMINI_MODEL), benchmark data in
  code comment (49 imgs: 3.1-lite ~507ms/img, 2.5-lite ~230ms/img)
- cli.mdx: version-agnostic "Gemini vision" reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-17 10:15:26 -04:00
ularkim de5b53c08b feat(capture): switch Gemini 2.5 Flash → 3.1 Flash Lite 
Gemini 3.1 Flash Lite Preview: 2.5x faster TTFT, 45% faster output,
slightly cheaper ($0.25/M vs $0.30/M input), near-2.5-Flash quality.
Descriptions are actually more detailed in testing.
 2026-04-16 22:58:50 -04:00
ularkim a77a6cbbf7 fix: double-audio bug + lint rules + docs guide + capture improvements 
Double-audio bug fix:
- scaffolding.ts: stop writing index.html in captures/ (root cause —
  runtime discovered scaffold + real index.html as two compositions)
- New lint rule: multiple_root_compositions — errors if >1 root HTML
- New lint rule: duplicate_audio_track — warns on overlapping audio

Capture improvements (from testing 30+ websites):
- Catalog runs BEFORE extractHtml (which mutates DOM — converts img src
  to data URLs). HeyKuba: 2 images → 78.
- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load image wait, CSS background-image cataloging
- SVG naming from class/id/parent (not just aria-label)
- Gemini batch 5→20, pause 12s→2s, maxOutputTokens 300→500
- Asset descriptions sorted: captioned first

Docs:
- New guide: guides/website-to-video.mdx (full tutorial)
- CLI docs: added capture and snapshot commands
- docs.json: website-to-video in Guides nav

C
 2026-04-16 22:58:50 -04:00
Ular KimsanovandClaude Opus 4.6 87f4c77e2f feat: website capture pipeline + 7-step video production skill (#284) 
* feat(cli): add website capture with AI-powered DESIGN.md generation

Adds `hyperframes capture <url>` command that extracts a complete design
system from any website, producing AI-agent-ready output:

- Full-page screenshot (lazy-load aware, nav at top)
- AI-generated DESIGN.md via Claude API (colors, typography, elevation,
  components, do's/don'ts) with programmatic asset catalog (136+ assets
  with HTML context annotations like img[src], css url(), link[rel=preload])
- CSS-purged compositions (87% size reduction via PurgeCSS)
- HTML-prettified compositions (one-tag-per-line for AI readability)
- CLAUDE.md + .cursorrules auto-generated for AI agent instructions
- Asset deduplication (srcset variants) and tracking pixel filtering

* feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement

- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update session handoff with slack research findings

- key finding: team already wants DESIGN.md integration (James, Bin, Vance)
- skills quality matters enormously - must invoke /hyperframes-compose
- eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide)
- templates at templates/ need study before finalizing skill
- session handoff updated with critical next steps

* refactor(cli): simplify capture pipeline, remove replica generator

* feat(capture): add Lottie detection and WebGL shader extraction

Captures Lottie animations via network interception and WebGL shader
source via gl.shaderSource hooking during site crawl. Updates
website-to-hyperframes skill with asset planning guidance, Lottie/shader
reading instructions, and stronger creative direction for scene planning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(capture): clean pipeline + shader-first creative workflow

Capture pipeline:
- Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps)
- Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md)
- Remove dead canvas-to-image code in htmlExtractor (post canvas removal)
- Parallelize image downloads (batches of 5 via Promise.allSettled)
- Fix pre-existing TS error (match[1] guard in font downloader)
- Default capture output to captures/<hostname>

Skill creative overhaul:
- Add shader transition selection to creative director step (Step 4)
- Add shader wiring instructions to engineer step (Step 5)
- Replace 4-line energy modifiers with visual vocabulary table
- Strip rigid scene-by-scene templates from video-recipes.md
- Strip example fill data from scene plan tables
- Add "read transition refs before planning" instruction
- Add creative ambition language ("how the hell did they make this")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add skill architecture redesign spec

Comprehensive redesign of website-to-hyperframes skill and capture
pipeline based on code review findings and Claude Code architecture
research. Key changes: remove AI auto-generation, restructure skill
into phases, embed shader boilerplate in scaffold, fix color format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add implementation plan for skill architecture redesign

13-task plan covering: capture pipeline cleanup (remove AI generation,
fix colors to HEX, add asset descriptions, shader-ready scaffold),
skill restructuring (4 phases with artifact gates), and compose skill
Visual Identity Gate upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(capture): remove AI auto-generation and SDK dependencies

* fix(capture): convert extracted colors to HEX format

* refactor(capture): remove AI key path, add asset descriptions generator

* refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions

* feat(capture): pre-wire shader transitions in index.html scaffold

* chore: remove duplicate visual-styles.md (canonical is in hyperframes/)

* refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator

* feat(skill): add Phase 1 understand reference

* feat(skill): add Phase 2 design reference with full DESIGN.md schema

* feat(skill): add Phase 3 creative direction reference

* feat(skill): add Phase 4 build reference with inline shader example

* feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md

* docs: update CLAUDE.md skill references for phase-based workflow

* fix: address code review findings

- Remove orphaned `false` argument in generateAgentPrompt call (critical:
  was shifting hasLottie, hasShaders, catalogedAssets parameters)
- Add HSL color handling in rgbToHex via temp element resolution
- Remove build artifact commit section from phase-4-build.md
- Fix __GSAP_TIMELINE reference to __timelines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(capture): regex double-escape + simplify scaffold + fix asset descriptions

- Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\()
  so browser receives valid regex patterns via page.evaluate()
- Simplify index.html scaffold: scene slots + audio + timeline + comment pointing
  to shader-setup.md reference (no broken inline shader boilerplate)
- Fix asset descriptions: use CatalogedAsset.contexts/notes instead of
  nonexistent htmlContext field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: code review — 16 bugs, 7-step skill rewrite, cleanup

Code fixes:
- snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero
  for --frames 1, port bind error handling, rAF-based render settle
- index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini
  batch/rate-limit comments, fix video preview viewport y-coordinate
- tokenExtractor.ts: remove dead seen[si] dedup code
- gsap.ts: index ALL classes for inline-style transform conflict detection

Skill architecture rewrite (4-phase → 7-step):
- Replace phase-1 through phase-4 with step-1 through step-7
- Add techniques.md (10 visual techniques with code patterns)
- Fix /hyperframes-compose → /hyperframes (skill doesn't exist)
- Fix captures/arc-browser reference → shader-setup.md (file doesn't exist)
- Fix step-7 hardcoded captures/stripe path
- Document Gemini API free/paid rate limits in step-1

Cleanup:
- CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step
- .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast)
- Delete old phase-*.md, animation-recreation.md, tts-integration.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dev artifacts, research docs, wrong lockfiles

Remove files that shouldn't ship in this PR:
- docs/research/ (aura analysis, prompt catalogs)
- docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes)
- docs/superpowers/ planning and spec docs
- pnpm-lock.yaml at root and cli (repo uses bun, not pnpm)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention

Main PR #283 removed the full skills table from CLAUDE.md and moved it
to AGENTS.md. Align with that decision: use main's slim dev-focused
format, fix pnpm→bun references, add one-line /website-to-hyperframes
pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add capture command to help groups

The capture command was registered in cli.ts but missing from
the help groups, so it wouldn't appear in `hyperframes --help`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format skill reference files (oxfmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: regenerate bun.lock after rebase

The lockfile was stale after rebasing onto main — bun install
--frozen-lockfile failed in CI because new dependencies (google/genai,
patchright, purgecss) weren't reflected in the lockfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments + improve capture quality

Review fixes (16 comments from jrusso1020 + vanceingalls):
- screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots
- screenshotCapture: dynamic screenshot count based on page height (30% overlap)
- snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard
- htmlExtractor: fix code injection via parameterized evaluate
- index.ts: video preview re-measures position after scroll, .env file loading
- capture.ts: BLOCKED.md on timeout failures
- gsap.ts: 5 inline-style lint tests added (all pass)
- Remove Playwright, patchright deps; @google/genai to optionalDependencies
- Gitignore: generic patterns instead of 20 hardcoded directories
- Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance)

Capture quality improvements (tested on 10+ websites):
- Color extraction: canvas-based oklch/lab resolver, pixel sampling via
  elementFromPoint, broad sweep for accent colors, gradient/shadow extraction
- Section detection: broadened selectors for div-based layouts, height cap
  to skip page-level wrappers, parent bg walkup for dark sites
- Font downloads: cap 6 per family / 30 total (Cal.com: 306→30)
- CTA detection: text pattern matching + nav context filtering
- Heading text: innerText with whitespace normalization
- Gemini captioning: maxOutputTokens 100→300, .env auto-loading
- .env.example updated with GEMINI_API_KEY docs
- TTS ranking: Kokoro first with Python 3.10+ note

* fix: address PR review comments + improve capture quality

Review round 2 fixes (jrusso1020 + vanceingalls):
- verify/index.ts: add path traversal guard (relative + isAbsolute)
- verify/index.ts: fix sections[i] undefined typecheck error (CI green)
- index.ts: escape Lottie JSON with \u003c to prevent </script> breakout
- step-4-storyboard: fix technique count contradiction (2-3 per beat, not
  across whole video)
- step-6-build: perspective tilt uses gsap.set() instead of CSS transform
  (avoids GSAP overwrite conflict)
- step-1-capture: reorder — command first, Gemini note after (zero-config
  is the default path, API key is optional enhancement)
- step-7-validate: add tsx fallback for snapshot command
- step-3-script: vary hook patterns, don't default to number every time
- assetDownloader: exempt SVGs from 10KB minimum filter (company logos
  like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets)

Note: adm-zip was NOT removed (reviewer #3) — it's still in
packages/cli/package.json:30. The root package.json had patchright
and purgecss removed, not adm-zip.

Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms
zero references in the entire codebase. The @anthropic-ai/sdk dependency
was removed earlier in this branch.

* refactor(capture): split index.ts (1175 to 566 lines) into modules

Mechanical extraction, zero logic changes.

New files:
- mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots
- contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions
- scaffolding.ts (135 lines): .env loading, project scaffold generation

Also fixes false-positive BLOCKED.md with structural Cloudflare detection.
Tested on 20 websites, pre/post output identical.

* chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss)

The --split feature auto-generates compositions from captured HTML — a
different approach from the /website-to-hyperframes skill workflow where
agents build compositions from scratch using the storyboard.

No skill file, no step reference, and no test session ever used --split.
Removes 923 lines of unused code + purgecss dependency.

Backed up to ~/Desktop/capture-split-backup/ for reference.

* fix(security): add ssrf protection, lottie injection fix, oom guard

- assetDownloader: add isPrivateUrl() guard blocking private IP ranges
  (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata
  endpoints, localhost, and non-HTTP schemes
- mediaCapture: fix Lottie JSON injection by loading shell HTML first
  then passing animation data via parameterized page.evaluate()
- index.ts: check Content-Length header before response.buffer() in
  Lottie network interception to avoid OOM on multi-GB responses

* fix(capture): security fixes, timeout, sub-agent dispatch instructions

Security (from miguel-heygen review):
- assetDownloader: export isPrivateUrl() SSRF guard
- htmlExtractor: add isPrivateUrl check before CSS fetch
- mediaCapture: add isPrivateUrl check before Lottie fetch
- mediaCapture: fix previewPage leak (try/finally)
- mediaCapture: skip Lottie files > 2MB for preview (CDP limit)
- contentExtractor: skip images > 4MB for Gemini captioning
- index.ts: check Content-Length before response.buffer() (OOM guard)
- snapshot.ts: register error handler before server.listen()

Capture improvements:
- Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare)
- step-6-build: sub-agent dispatch template with explicit rules:
  pass file PATHS not contents, use local fonts not Google Fonts,
  verify ../assets/ references after each beat

* fix(capture): catalog before DOM mutation, networkidle2, faster Gemini

Critical: asset cataloger now runs BEFORE extractHtml which converts img
src to data URLs. Framer sites like heykuba.com went from 2 to 78 images.

- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load wait: scroll to bottom, wait for img.complete
- CSS background-image cataloging for Framer/Webflow
- SVG naming: checks class, id, parent, inner text (not just aria-label)
- Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img)
- maxOutputTokens 300->500, descriptions sorted captioned-first
- Remove tsx fallback from step-1 (reviewer nit, published CLI has it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 12:03:16 -07:00