hyperframes/CLAUDE.md

Hyperframes

Skills — USE THESE FIRST

This repo ships skills that are installed globally via npx hyperframes skills (runs automatically during hyperframes init). Always use the appropriate skill instead of writing code from scratch or fetching external docs.

HyperFrames Skills (from this repo)

Skill	Invoke with	When to use
hyperframes-compose	/hyperframes-compose	Creating ANY HTML composition — videos, animations, title cards, overlays. Contains required HTML structure, class="clip" rules, GSAP timeline patterns, and rendering constraints.
hyperframes-captions	/hyperframes-captions	Any task involving text synced to audio: captions, subtitles, lyrics, lyric videos, karaoke. Also covers transcription strategy (whisper model selection, transcript format).

GSAP Skills (from greensock/gsap-skills)

Skill	Invoke with	When to use
gsap-core	/gsap-core	gsap.to(), from(), fromTo(), easing, duration, stagger, defaults
gsap-timeline	/gsap-timeline	Timeline sequencing, position parameter, labels, nesting, playback
gsap-performance	/gsap-performance	Performance best practices — transforms over layout props, will-change, batching
gsap-plugins	/gsap-plugins	ScrollTrigger, Flip, Draggable, SplitText, and other GSAP plugins
gsap-scrolltrigger	/gsap-scrolltrigger	Scroll-linked animations, pinning, scrub, triggers
gsap-utils	/gsap-utils	gsap.utils helpers — clamp, mapRange, snap, toArray, wrap, pipe

Why this matters

The skills encode HyperFrames-specific patterns (e.g., required class="clip" on all timed elements, GSAP timeline registration via window.__GSAP_TIMELINE, data-* attribute semantics) that are NOT in generic web docs. Skipping the skills and writing from scratch will produce broken compositions.

Rules

• When creating or modifying HTML compositions → invoke /hyperframes-compose BEFORE writing any code
• When adding captions, subtitles, lyrics, or any text synced to audio → invoke /hyperframes-captions BEFORE writing any code
• When transcribing audio or choosing a whisper model → invoke /hyperframes-captions BEFORE running any transcription tool
• When creating a video from audio (music video, lyric video, audio visualizer with text) → invoke BOTH /hyperframes-compose AND /hyperframes-captions
• When writing GSAP animations → invoke /gsap-core and /gsap-timeline BEFORE writing any code
• When optimizing animation performance → invoke /gsap-performance BEFORE making changes
• After creating or editing any .html composition → run npx hyperframes lint and fix all errors before considering the task complete

Installing skills

npx hyperframes skills          # install all to Claude, Gemini, Codex
npx hyperframes skills --claude # Claude Code only
npx skills add greensock/gsap-skills  # alternative: via skills CLI

Project Overview

Open-source video rendering framework: write HTML, render video.

packages/
  cli/       → hyperframes CLI (create, preview, lint, render)
  core/      → Types, parsers, generators, linter, runtime, frame adapters
  engine/    → Seekable page-to-video capture engine (Puppeteer + FFmpeg)
  producer/  → Full rendering pipeline (capture + encode + audio mix)
  studio/    → Browser-based composition editor UI

Development

pnpm install    # Install dependencies
pnpm build      # Build all packages
pnpm test       # Run tests

Key Concepts

• Compositions are HTML files with data-* attributes defining timeline, tracks, and media
• Frame Adapters bridge animation runtimes (GSAP, Lottie, CSS) to the capture engine
• Producer orchestrates capture → encode → audio mix into final MP4
• BeginFrame rendering uses HeadlessExperimental.beginFrame for deterministic frame capture

Transcription

HyperFrames uses word-level timestamps for captions. The hyperframes transcribe command handles both transcription and format conversion.

Quick reference

# Transcribe audio/video (local whisper.cpp, no API key)
npx hyperframes transcribe audio.mp3
npx hyperframes transcribe video.mp4 --model medium.en --language en

# Import existing transcript from another tool
npx hyperframes transcribe subtitles.srt
npx hyperframes transcribe subtitles.vtt
npx hyperframes transcribe openai-response.json

Whisper models

Default is small.en. Upgrade for better accuracy:

Model	Size	Use case
tiny.en	75 MB	Quick testing
base.en	142 MB	Short clips, clear audio
small.en	466 MB	Default — most content
medium.en	1.5 GB	Important content, noisy audio
large-v3	3.1 GB	Multilingual, production quality

Use .en suffix for English-only (more accurate). Drop it for multilingual content.

Supported transcript formats

The CLI auto-detects and normalizes: whisper.cpp JSON, OpenAI Whisper API JSON, SRT, VTT, and pre-normalized [{text, start, end}] arrays.

Improving transcription quality

If captions are inaccurate (wrong words, bad timing):

1. Upgrade the model: --model medium.en or --model large-v3
2. Set language: --language en to filter non-target speech
3. Use an external API: Transcribe via OpenAI or Groq Whisper API, then import the JSON with hyperframes transcribe response.json

See the /hyperframes-captions skill for full details on model selection and API usage.

Agent Directives

Pre-Work

1. Step 0 Rule: Before ANY structural refactor on a file >300 LOC, first remove all dead props, unused exports, unused imports, and debug logs. Commit this cleanup separately before starting the real work.

2. Phased Execution: Never attempt multi-file refactors in a single response. Break work into explicit phases. Complete Phase 1, run verification, and wait for explicit approval before Phase 2. Each phase must touch no more than 5 files.

Code Quality

3. Senior Dev Override: If architecture is flawed, state is duplicated, or patterns are inconsistent — propose and implement structural fixes. Ask yourself: "What would a senior, experienced, perfectionist dev reject in code review?" Fix all of it.

4. Forced Verification: You are FORBIDDEN from reporting a task as complete until you have:

   • Run pnpm build (typecheck + bundle)
   • Run pnpm test (if tests exist for the changed package)
   • Fixed ALL resulting errors

Context Management

5. Sub-Agent Swarming: For tasks touching >5 independent files, launch parallel sub-agents (5-8 files per agent). Each agent gets its own context window. Sequential processing of large tasks guarantees context decay.

6. Context Decay Awareness: After 10+ messages in a conversation, re-read any file before editing it. Do not trust memory of file contents — auto-compaction may have silently destroyed that context.

7. File Read Budget: For files over 500 LOC, use offset and limit parameters to read in sequential chunks. Never assume you have seen a complete file from a single read.

Edit Safety

8. Edit Integrity: Before EVERY file edit, re-read the file. After editing, read it again to confirm the change applied correctly. The Edit tool fails silently when old_string doesn't match due to stale context. Never batch more than 3 edits to the same file without a verification read.

9. No Semantic Search: When renaming or changing any function/type/variable, search separately for: direct calls, type-level references, string literals, dynamic imports, re-exports, barrel file entries, and test files. Do not assume a single grep caught everything.