feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement

- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
ularkim
2026-04-15 14:04:47 -04:00
co-authored by Claude Opus 4.6
parent 1c90360066
commit 2b6ed8fecb
12 changed files with 1472 additions and 196 deletions
+175
View File
@@ -0,0 +1,175 @@
# Session Handoff — Website Capture & DESIGN.md System
> Last updated: 2026-04-07 (end of marathon session)
> Branch: `feat/website-capture-design-md`
> Key commit: d60a9f8
## What Was Built This Session
### Core Pipeline: `hyperframes capture <url>`
A complete website-to-video pipeline that produces AI-agent-ready output:
```
hyperframes capture https://stripe.com -o /tmp/stripe
```
**Pipeline steps (in order):**
1. Launch headless Chrome (Puppeteer)
2. Catalog animations (Web Animations API, CSS, IntersectionObserver, CDP)
3. Extract HTML & CSS
4. Extract design tokens (colors, fonts, headings, CTAs, sections, CSS variables)
5. Catalog assets with HTML contexts (img[src], css url(), link[rel=preload], video[src], etc.)
6. Detect JS libraries (GSAP, Three.js, ScrollTrigger, etc. via globals + script URLs)
7. Extract ALL visible text in DOM order
8. Extract inline SVGs (up to 50, 10KB each)
9. **Full-page screenshot via Playwright** (not Puppeteer — Playwright doesn't resize viewport, preserving fixed/sticky layouts)
10. Download assets (fonts, images, SVGs, favicon)
11. **AI-generated DESIGN.md** via Gemini 3.1 Pro or Claude Sonnet (screenshot + tokens → rich design system prose + programmatic asset catalog)
12. **AI-generated replica.html** — clean single-file Tailwind CSS recreation of the website
13. **Refinement pass** — screenshots replica, compares to original, asks AI to fix differences
14. Generate CLAUDE.md + .cursorrules (AI agent instructions)
15. *(Optional)* Split into section compositions, verify, CSS purge, HTML prettify
**Output files:**
- `DESIGN.md` — AI-written design system (colors, typography, elevation, components, do's/don'ts) + programmatic asset catalog with HTML contexts
- `replica.html` — Clean Tailwind CSS recreation (~30-56KB, single file)
- `screenshots/full-page.png` — Full-page website screenshot
- `screenshots/replica-screenshot.png` — Screenshot of the replica (for comparison)
- `CLAUDE.md` + `.cursorrules` — AI agent instructions (auto-read by Claude Code/Cursor)
- `assets/` — Downloaded fonts, images, SVGs, favicon
- `assets/svgs/` — Extracted inline SVGs from the page
- `extracted/` — Raw HTML, CSS, animation data, tokens.json
- `compositions/` — *(if split enabled)* Per-section HTML compositions (CSS-purged, prettified)
### Key Files
| File | Purpose |
|------|---------|
| `packages/cli/src/capture/index.ts` | Pipeline orchestrator |
| `packages/cli/src/capture/designMdGenerator.ts` | AI-powered DESIGN.md generation (Gemini or Claude) |
| `packages/cli/src/capture/replicaGenerator.ts` | AI-powered HTML replica + refinement loop |
| `packages/cli/src/capture/assetCataloger.ts` | Comprehensive asset extraction with HTML contexts |
| `packages/cli/src/capture/screenshotCapture.ts` | Full-page screenshot (Playwright) |
| `packages/cli/src/capture/cssPurger.ts` | PurgeCSS + Prettier for compositions |
| `packages/cli/src/capture/agentPromptGenerator.ts` | CLAUDE.md/.cursorrules generation |
| `packages/cli/src/capture/tokenExtractor.ts` | Design token extraction (colors, fonts, headings, SVGs) |
| `packages/cli/src/capture/animationCataloger.ts` | Animation detection (4 techniques) |
| `packages/cli/src/capture/assetDownloader.ts` | Asset downloading + inline SVG saving |
| `packages/cli/src/capture/briefGenerator.ts` | Legacy (replaced by DESIGN.md, kept for reference) |
| `packages/cli/src/commands/capture.ts` | CLI command definition |
### Research Documentation
| File | Contents |
|------|----------|
| `docs/research/aura-build-and-design-systems.md` | Complete Aura.build analysis, DESIGN.md format, prompt builder |
| `docs/research/aura-prompt-comparison.md` | Side-by-side comparison of Aura's with/without screenshot prompts |
| `docs/research/aura-examples/stripe-DESIGN.md` | Aura's Stripe DESIGN.md for comparison |
| `docs/research/reverse-engineered-aura-system-prompt.md` | **Reverse-engineered Aura system instructions** based on output analysis |
## Known Issues & Gaps
### Screenshot
- Puppeteer `fullPage:true` breaks complex sites (mesh gradients, position:fixed layers) — solved by using Playwright instead
- Chrome 16,384px height limit — pages taller than this get clipped at the bottom
- Stripe's hero wave gradient still shows artifacts in some captures
### Replica Quality
- Gemini 3.1 Pro produces better replicas than Claude Sonnet for visual-to-code
- Company logos in "logo walls" often rendered as text instead of images because the original site uses inline SVGs (not `<img>` tags) — our extractor now saves them to `assets/svgs/` but the AI can't access local files during generation
- Refinement pass sometimes rewrites Tailwind to vanilla CSS (fixed with explicit rules in refinement prompt)
- Complex animations (WebGL gradients, Three.js scenes) not reproduced — AI generates static alternatives
### DESIGN.md
- Comparable to Aura's quality for most sections
- Asset catalog includes HTML contexts (img[src], css url(), link[rel=preload], etc.)
- Deduplicates srcset resolution variants
- Filters tracking pixels and CSS fragment references
- Inline SVGs now saved but listed as local paths (not URLs accessible to external AI)
### Compositions (Optional Path)
- CSS purge reduces captured compositions by 87% (7.2MB → 918KB)
- HTML prettify makes them AI-readable (one-tag-per-line)
- But compositions may be unnecessary — replica.html serves the same purpose more cleanly
## Environment Variables
| Variable | Purpose |
|----------|---------|
| `GEMINI_API_KEY` | Google Gemini API (for DESIGN.md and replica generation) |
| `ANTHROPIC_API_KEY` | Claude API (fallback for DESIGN.md and replica) |
| `CLAUDE_API_KEY` | Alternative Claude API key name |
Gemini is preferred when set. Falls back to Claude. If neither is set, DESIGN.md uses a structured template (no AI prose) and replica is skipped.
## Model IDs
- Gemini: `gemini-3.1-pro-preview` (NOT gemini-2.5-flash)
- Claude: `claude-sonnet-4-20250514`
## What's Next
### Priority 1: HyperFrames Video Creation Workflow
The whole point was to create VIDEOS from captured websites. The current output (DESIGN.md + screenshot + replica.html) gives an AI agent everything it needs to create on-brand video compositions. But we need:
1. **A prompt/skill that bridges DESIGN.md → HyperFrames compositions** — the AI reads the DESIGN.md, understands the brand, then creates video compositions following HyperFrames rules (GSAP timelines, data-* attributes, clip visibility)
2. **Use-case library** — pre-built prompts for common video types:
- 15s social media ad
- 30s product tour
- Feature highlight reel
- Testimonial spotlight
- Competitor comparison
- Customer case study
3. **Prompt builder** — interactive prompt construction (like Aura's but for video)
### Priority 2: SVG Logo Fix
Inline SVGs are saved locally but the AI can't access them during generation. Options:
- Embed SVG code directly in DESIGN.md (bloats the file)
- Upload to a temp CDN during capture
- Accept this limitation — the local AI agent (Claude Code) CAN read them
### Priority 3: Web Shader Extractor Integration
141K websites use Three.js. Stripe, Linear, Vercel all have WebGL. The lixiaolin94/skills shader extractor could capture these effects. Research done, implementation deferred.
### Priority 4: Prompt Builder UI
Could live in HyperFrames Studio. Categories (from Aura research):
- Scene types: Hero, Features, Stats, Testimonial, CTA, Comparison
- Duration: 15s, 30s, 60s
- Style: Energetic, Corporate, Cinematic, Minimal
- Transitions: Wipe, Fade, Slide, Scale
- Animation: Stagger, Counter, Reveal, Float
## Dependencies Added
- `purgecss` — CSS tree-shaking for compositions
- `prettier` — HTML prettification
- `@anthropic-ai/sdk` — Claude API
- `@google/genai` — Gemini API
- `playwright` — Full-page screenshots (better than Puppeteer for this)
## How to Test
```bash
# Capture with Gemini (recommended)
GEMINI_API_KEY="your-key" npx tsx packages/cli/src/cli.ts capture https://notion.com -o /tmp/test --skip-split
# Capture with Claude (fallback)
ANTHROPIC_API_KEY="your-key" npx tsx packages/cli/src/cli.ts capture https://notion.com -o /tmp/test --skip-split
# With compositions (full pipeline)
GEMINI_API_KEY="your-key" npx tsx packages/cli/src/cli.ts capture https://notion.com -o /tmp/test
# Serve and view replica
python3 -m http.server 3334 --directory /tmp/test
open http://localhost:3334/replica.html
```
## User Preferences (from memory)
- DESIGN.md must be AI-generated prose, not template filling
- Always verify by reading outputs, comparing to references
- Don't hide problems — show them honestly
- Research thoroughly before implementing
- Match Aura.build quality as the benchmark
- Don't add HyperFrames-specific sections to DESIGN.md (keep it universal)
- Full-page screenshot only (no per-section screenshots)
- Gemini 3.1 Pro preferred over Claude for visual-to-code tasks
@@ -0,0 +1,70 @@
# Design System
## Overview
Stripe's visual personality is defined by "High-Fidelity Minimalism." The system uses high-density information layouts paired with ethereal, animated gradients (the signature "mesh gradient" look). It strikes a balance between professional financial infrastructure and cutting-edge software engineering. Key characteristics include extreme typographic precision, fluid motion cues (the hero "wave"), and a layered UI approach that simulates physical depth through soft shadows and sophisticated border treatments. The layout is structured on a strict grid with significant responsive adaptations, transitioning from dense 12-column desktop views to simplified vertical stacks on mobile.
## Colors
### Brand & Core
- **Primary Brand**: #635bff (Stripe Purple / Brand 500)
- **Text Solid**: #0a2540 (Dark Navy)
- **Text Soft**: #424770 (Slate Blue)
- **Background Subdued**: #f6f9fc (Light Gray/Blue tint)
- **White**: #ffffff
### Semantic & Accents
- **Action Text**: #00d4ff (Electric Blue accent)
- **Accent Magenta**: #ea2261 (Connect product accent)
- **Accent Orange**: #ff6118 (Payments/Agentic accent)
- **Accent Yellow**: #fc5 (Billing/Startups accent)
- **Border Quiet**: #e6ebf1 (Soft UI borders)
## Typography
- **Primary Font**: Sohne-var (Custom sans-serif). Used for all body and headings. Characteristics: Geometric but legible, varied weights from 300 (Light) to 700 (Bold).
- **Code Font**: SourceCodePro (Monospace). Used for API references and developer-centric components.
- **Hierarchy**:
- **Hero Title**: Variable sizing (max-width 32ch), ~3.5rem on desktop, tight line-height (1.15).
- **Section Headings**: `hds-heading--lg` (~2rem desktop), `hds-heading--md` (~1.625rem).
- **Body Text**: `hds-font-text-md` (~0.875rem to 1rem) with high tracking/line-height for readability (1.4).
## Elevation
- **Surface Layering**: Uses `isolation: isolate` to manage complex animated background layers.
- **Shadows**:
- `hds-shadow-md`: 0 6px 22px rgba(0, 55, 112, 0.1) for floating cards.
- `hds-canary-ui-shadow`: 0 16px 32px rgba(50, 50, 93, 0.12) for dashboard-style graphic cards.
- **Borders**: Heavy reliance on 1px solid or dashed borders (`1px dashed #e5edf5`) instead of heavy shadows for separating content sections.
## Components
- **Navigation**: A sophisticated mega-menu with blur effects (`backdrop-filter: blur(5px)`) and animated chevron transitions.
- **Bento Cards**: Interactive grid items with localized gradient "blobs" (`--bento-left-blob`) and nested graphics.
- **Logo Carousel**: A continuous marquee featuring SVG partner logos with grayscale-to-color hover transitions.
- **Interactive Graphics**: Pseudo-dashboard UI components (Tax, Invoicing, Connect) that use CSS transforms and absolute positioning to simulate product interfaces.
- **Buttons**:
- **Primary**: Solid background with a slight pill-shape border-radius.
- **Secondary**: Ghost style with `hds-icon-hover-arrow` that animates on hover.
## Do's and Don'ts
- **Do**: Use 1px dashed borders to separate high-level footer categories.
- **Do**: Apply `text-wrap: pretty` or `balance` to headings to prevent widows.
- **Don't**: Use generic drop shadows; use the specific color-mixed elevations defined in the HDS system.
- **Do**: Use subtle mesh gradients in background layers to provide depth without distracting from text.
- **Don't**: Overcomplicate mobile layouts; collapse multi-column desktop grids into single-column stacks.
## Assets
### Fonts
- **Sohne Variable**: https://b.stripecdn.com/mkt-ssr-statics/assets/_next/static/media/Sohne.cb178166.woff2
- **Source Code Pro**: https://b.stripecdn.com/mkt-ssr-statics/assets/_next/static/media/SourceCodePro-Medium.f5ba3e6a.woff2
- **Sohne (Alt Path)**: https://stripe.com/mkt-ssr-statics/assets/_next/static/media/Sohne.cb178166.woff2
- **Source Code Pro (Alt Path)**: https://stripe.com/mkt-ssr-statics/assets/_next/static/media/SourceCodePro-Medium.f5ba3e6a.woff2
### Images
- **Personalize Background**: https://images.ctfassets.net/fzn2n1nzq965/243SyQMmyCkQOj9SIYcV9m/9ca6154f96e65aa92bb35401e70f3725/Frame_2147247494.png
- **Decagon Logo**: https://images.stripeassets.com/fzn2n1nzq965/1hreJwxuVJ5ucPtuA7pRKH/3c5630387bca898d01ae17fc7ae2890a/decagon.png?w=432&fm=webp&q=90
- **Connect Bento Background**: https://images.stripeassets.com/fzn2n1nzq965/1j4wM9h2bBsyRFvTv6Wsn0/07f4b9b1e1e17fdc509d9087454dd8bc/ConnectBentoBackground.jpg?w=1242&fm=webp&q=90
- **Agentic Mobile Graphic**: https://images.stripeassets.com/fzn2n1nzq965/1k7ckFceNFlxqF47hWJUlM/eaa4d8b3f8e4b76ba3dd7df7da88e099/the-happenings-agentic-mobile.png?w=336&fm=webp&q=90
- **Browserbase Case Study**: https://images.stripeassets.com/fzn2n1nzq965/1NiQJZA0rKbwXZ7mD4BCrM/0abdaba77e564f858a33241a9aab0939/browserbase.png?w=432&fm=webp&q=90
- **The Quincunx Book Cover**: https://images.stripeassets.com/fzn2n1nzq965/2gq7OxrRa1Ee5L8uFUX75n/2398783b25e6c09d095a94e8bae4b1e4/TheQuincunx.png?w=296&fm=webp&q=90
- **Connect Mobile Background**: https://images.stripeassets.com/fzn2n1nzq965/43eqkFV1GG5GqcbNNeEzVp/8f4705b1b80a4c835c2960ed8055d649/ConnectMobileBackground.jpg?w=1104&fm=webp&q=90
- **Sessions Banner**: https://images.stripeassets.com/fzn2n1nzq965/5cRV5XgALGMWv62qKuH0Rw/f0429f90b5731f51c44c47b187626bbd/sessions-banner-background_2x.png?w=1232&fm=webp&q=90
- **Retailers Mobile Graphic**: https://images.stripeassets.com/fzn2n1nzq965/7EVLwfXmG3pl1vhBssMrFB/b6d8612b1f567922c1355d864c279678/the-happenings-payment-retailers-mobile.png?w=336&fm=webp&q=90
@@ -0,0 +1,121 @@
# Reverse-Engineered Aura System Instructions
> Based on analysis of Aura.build outputs (code.html for Notion, code2.html for Soulscape),
> the Aura AI's own statement about its system instructions, and the Prompt Builder categories.
## The Prompt (Estimated)
```
You generate single-file HTML implementations of websites from screenshots and design references.
## Output Format
- Single HTML file. Start with <!DOCTYPE html>, end with </html>.
- No markdown wrapping. No code fences. No explanation or commentary.
- Clean, well-indented, semantic HTML5 with descriptive comments for each section.
- Target: 400-800 lines for a full page.
## Tech Stack (use EXACTLY this — no substitutions)
- Tailwind CSS: <script src="https://cdn.tailwindcss.com"></script>
- Icons: <script src="https://unpkg.com/lucide@latest"></script> with <i data-lucide="icon-name"> elements. Call lucide.createIcons() once at the end of <body>. For dark/cinematic designs, use Iconify with <script src="https://code.iconify.design/iconify-icon/1.0.7/iconify-icon.min.js"></script> and <iconify-icon icon="solar:icon-name-linear"> elements instead.
- Google Fonts: <link> tag in <head> for typeface matching. Only load the weights you actually use.
- For fonts NOT on Google Fonts: use @font-face with the exact URL from the DESIGN.md Assets section.
## Tailwind Usage Rules
- Use Tailwind utility classes for ALL styling. Do NOT write custom CSS classes except for:
- @font-face declarations (when needed)
- @keyframes (ONLY if Tailwind's arbitrary animation syntax cannot express it)
- Complex CSS-only interactions (accordion hover effects)
- Use tailwind.config to extend colors, fonts, animations, and keyframes for the specific site's design tokens. Define these in a <script> block before the HTML.
- Use arbitrary values for exact matches: text-[#635bff], bg-[#0a2540], text-[48px], tracking-[-0.02em]
- Use responsive prefixes on EVERY layout element: sm:, md:, lg:
- Use animate-[name_duration_easing_infinite] for marquee and continuous animations.
## Typography
- Never use generic font stacks. Match the exact fonts from the DESIGN.md.
- Load font weights explicitly — do not rely on browser synthesis.
- Use tracking-tighter or tracking-[-0.02em] on display headings.
- Use font-light (300) for sites like Stripe. Use font-semibold (600) for sites like Notion.
- Text size classes: text-5xl md:text-7xl for heroes, text-4xl md:text-5xl for sections, text-3xl for cards.
- NEVER use px values in Tailwind classes when a Tailwind size class exists.
## Layout
- max-w-[1200px] mx-auto or max-w-7xl mx-auto for content containers.
- px-4 md:px-6 lg:px-8 for horizontal padding at breakpoints.
- Use CSS Grid with Tailwind for Bento layouts: grid grid-cols-1 md:grid-cols-12 gap-6, with md:col-span-8, md:col-span-4, md:col-span-6.
- Use flex for navigation, button groups, and inline elements.
- Sections alternate backgrounds: bg-white and bg-[#F7F7F5] (or the site's equivalent neutral).
## Components
- Navigation: sticky top-0 z-50 with backdrop-blur-md, bg-white/80 or site equivalent. Logo left, links center (hidden md:flex), CTAs right. Mobile: hamburger button (md:hidden).
- Hero: Generous padding (pt-20 pb-24 md:pt-32 md:pb-32). Text left-aligned or centered depending on the site.
- Cards: rounded-2xl or rounded-3xl, border border-gray-100, shadow-sm, p-8 or p-10. Hover: hover:-translate-y-1 transition-transform.
- Buttons: Primary = solid bg, rounded-full or rounded-xl, hover state. Secondary = border, ghost style. Include arrow icon.
- Logo Wall: overflow-hidden container with animate-[marquee] flex. Duplicate the logo set for seamless loop. Use real logo images from DESIGN.md, not text.
- Testimonials: Serif font for quotes (Playfair Display, LyonText, or site equivalent). Include real headshot images and company attribution.
- Footer: Multi-column grid (grid-cols-2 md:grid-cols-5), with logo, social icons (Lucide), link lists.
## Animation (CSS-only — NO JavaScript animations)
- Marquee: animate-[marquee_20s_linear_infinite] on a flex container. Define @keyframes marquee { 0% { transform: translateX(0%) } 100% { transform: translateX(-50%) } } in tailwind.config or <style>. Duplicate content for seamless loop.
- Hover: transition-colors, transition-transform, hover:scale-105, hover:-translate-y-1, group-hover patterns.
- For complex sites: CSS-only accordion (flex + transition: flex 0.7s + :hover), CSS-only reveal (opacity + translate + transition on group-hover).
- NEVER use IntersectionObserver, scroll triggers, or requestAnimationFrame.
- ALL sections must be fully visible on page load. NEVER use opacity: 0 with JS-based reveal.
## Images & Assets
- Use EVERY real asset URL from the DESIGN.md Assets section for visible content. This includes:
- Company logos in logo walls (use <img> tags, NOT text spans)
- Product screenshots in feature cards
- Testimonial headshots
- Hero images and background images
- Video poster images
- For logo walls: use filter grayscale on logos, with hover:grayscale-0 transition.
- For background images: use absolute positioning with object-cover, overlay with gradient (bg-gradient-to-t from-black/60).
- NEVER use colored rectangles or text as placeholder for images that exist in the DESIGN.md.
## Content Fidelity
- Use EXACT text from the screenshot and DESIGN.md. Do NOT paraphrase, abbreviate, or invent content.
- Match the exact number of items in lists, grids, and carousels.
- Preserve company names, person names, statistics, and brand references exactly.
- If text is unreadable in the screenshot, check the DESIGN.md headings and CTAs.
## Responsive Behavior
- Mobile (default): Single column, stacked layouts, text-3xl headings, px-4, hidden navigation links.
- Tablet (md:): Two-column grids where appropriate, text-4xl headings, px-6, visible navigation.
- Desktop (lg:): Full grid layouts (12-column bento), text-5xl+ headings, max-width containers.
- Every grid, flex layout, padding, and font size MUST have responsive variants.
- Mobile navigation: hamburger icon button (md:hidden), desktop links (hidden md:flex).
## Do NOT
- Do NOT use React, Vue, Svelte, or any framework.
- Do NOT write custom CSS classes when Tailwind utilities work.
- Do NOT use JavaScript for any visual effect — CSS transitions and animations only.
- Do NOT use opacity: 0 or display: none for content that should be visible.
- Do NOT use placeholder images, gray rectangles, or Lorem Ipsum.
- Do NOT convert the page to a simpler design — preserve all sections and complexity.
- Do NOT use absolute pixel positioning for layout — use Tailwind's spacing and grid system.
- Do NOT add features or sections that don't exist in the original.
```
## Evidence
### From Aura AI's own statement:
> "I have internal system instructions that guide my behavior — for example, instructing me to use specific icon libraries, avoid certain font size units, maintain strict tracking on typography, and output in a very specific format."
### From code.html (Notion):
- 68 Lucide icon references
- 102 lines with responsive Tailwind classes (md:, lg:)
- Zero JavaScript functions
- Inline Tailwind animate-[] for marquee
- Playfair Display serif for Forbes quote
- SVG logo inline for Notion "N"
### From code2.html (Soulscape):
- Iconify Solar icons (not Lucide — different icon set for dark theme)
- tailwind.config with custom colors (void, flare, warm, cool, glass)
- @font-face for Geist Mono (not on Google Fonts)
- CSS-only accordion with :hover flex transitions
- SVG noise grain overlay (data URI)
- No JavaScript at all
### Key insight:
The system prompt doesn't specify ONE icon library — it likely has a rule like "use Lucide for light/professional sites, use Iconify Solar for dark/cinematic sites." The choice adapts to the design.