mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
Miguel ÁngelandClaude Opus 4.6
a5d9069f0f
Add mechanical overrides for AI coding agents: phased execution, forced verification (build + test), sub-agent swarming for large tasks, context decay awareness, edit integrity rules, and thorough rename search requirements. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
150 lines
8.6 KiB
Markdown
150 lines
8.6 KiB
Markdown
# Hyperframes
|
|
|
|
## Skills — USE THESE FIRST
|
|
|
|
This repo ships skills that are installed globally via `npx hyperframes skills` (runs automatically during `hyperframes init`). **Always use the appropriate skill instead of writing code from scratch or fetching external docs.**
|
|
|
|
### HyperFrames Skills (from this repo)
|
|
|
|
| Skill | Invoke with | When to use |
|
|
| ------------------------ | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| **hyperframes-compose** | `/hyperframes-compose` | Creating ANY HTML composition — videos, animations, title cards, overlays. Contains required HTML structure, `class="clip"` rules, GSAP timeline patterns, and rendering constraints. |
|
|
| **hyperframes-captions** | `/hyperframes-captions` | Any task involving text synced to audio: captions, subtitles, lyrics, lyric videos, karaoke. Also covers transcription strategy (whisper model selection, transcript format). |
|
|
|
|
### GSAP Skills (from [greensock/gsap-skills](https://github.com/greensock/gsap-skills))
|
|
|
|
| Skill | Invoke with | When to use |
|
|
| ---------------------- | --------------------- | -------------------------------------------------------------------------------- |
|
|
| **gsap-core** | `/gsap-core` | `gsap.to()`, `from()`, `fromTo()`, easing, duration, stagger, defaults |
|
|
| **gsap-timeline** | `/gsap-timeline` | Timeline sequencing, position parameter, labels, nesting, playback |
|
|
| **gsap-performance** | `/gsap-performance` | Performance best practices — transforms over layout props, will-change, batching |
|
|
| **gsap-plugins** | `/gsap-plugins` | ScrollTrigger, Flip, Draggable, SplitText, and other GSAP plugins |
|
|
| **gsap-scrolltrigger** | `/gsap-scrolltrigger` | Scroll-linked animations, pinning, scrub, triggers |
|
|
| **gsap-utils** | `/gsap-utils` | `gsap.utils` helpers — clamp, mapRange, snap, toArray, wrap, pipe |
|
|
|
|
### Why this matters
|
|
|
|
The skills encode HyperFrames-specific patterns (e.g., required `class="clip"` on all timed elements, GSAP timeline registration via `window.__GSAP_TIMELINE`, `data-*` attribute semantics) that are NOT in generic web docs. Skipping the skills and writing from scratch will produce broken compositions.
|
|
|
|
### Rules
|
|
|
|
- When creating or modifying HTML compositions → invoke `/hyperframes-compose` BEFORE writing any code
|
|
- When adding captions, subtitles, lyrics, or any text synced to audio → invoke `/hyperframes-captions` BEFORE writing any code
|
|
- When transcribing audio or choosing a whisper model → invoke `/hyperframes-captions` BEFORE running any transcription tool
|
|
- When creating a video from audio (music video, lyric video, audio visualizer with text) → invoke BOTH `/hyperframes-compose` AND `/hyperframes-captions`
|
|
- When writing GSAP animations → invoke `/gsap-core` and `/gsap-timeline` BEFORE writing any code
|
|
- When optimizing animation performance → invoke `/gsap-performance` BEFORE making changes
|
|
- After creating or editing any `.html` composition → run `npx hyperframes lint` and fix all errors before considering the task complete
|
|
|
|
### Installing skills
|
|
|
|
```bash
|
|
npx hyperframes skills # install all to Claude, Gemini, Codex
|
|
npx hyperframes skills --claude # Claude Code only
|
|
npx skills add greensock/gsap-skills # alternative: via skills CLI
|
|
```
|
|
|
|
## Project Overview
|
|
|
|
Open-source video rendering framework: write HTML, render video.
|
|
|
|
```
|
|
packages/
|
|
cli/ → hyperframes CLI (create, preview, lint, render)
|
|
core/ → Types, parsers, generators, linter, runtime, frame adapters
|
|
engine/ → Seekable page-to-video capture engine (Puppeteer + FFmpeg)
|
|
producer/ → Full rendering pipeline (capture + encode + audio mix)
|
|
studio/ → Browser-based composition editor UI
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
pnpm install # Install dependencies
|
|
pnpm build # Build all packages
|
|
pnpm test # Run tests
|
|
```
|
|
|
|
## Key Concepts
|
|
|
|
- **Compositions** are HTML files with `data-*` attributes defining timeline, tracks, and media
|
|
- **Frame Adapters** bridge animation runtimes (GSAP, Lottie, CSS) to the capture engine
|
|
- **Producer** orchestrates capture → encode → audio mix into final MP4
|
|
- **BeginFrame rendering** uses `HeadlessExperimental.beginFrame` for deterministic frame capture
|
|
|
|
## Transcription
|
|
|
|
HyperFrames uses word-level timestamps for captions. The `hyperframes transcribe` command handles both transcription and format conversion.
|
|
|
|
### Quick reference
|
|
|
|
```bash
|
|
# Transcribe audio/video (local whisper.cpp, no API key)
|
|
npx hyperframes transcribe audio.mp3
|
|
npx hyperframes transcribe video.mp4 --model medium.en --language en
|
|
|
|
# Import existing transcript from another tool
|
|
npx hyperframes transcribe subtitles.srt
|
|
npx hyperframes transcribe subtitles.vtt
|
|
npx hyperframes transcribe openai-response.json
|
|
```
|
|
|
|
### Whisper models
|
|
|
|
Default is `small.en`. Upgrade for better accuracy:
|
|
|
|
| Model | Size | Use case |
|
|
| ----------- | ------ | -------------------------------- |
|
|
| `tiny.en` | 75 MB | Quick testing |
|
|
| `base.en` | 142 MB | Short clips, clear audio |
|
|
| `small.en` | 466 MB | **Default** — most content |
|
|
| `medium.en` | 1.5 GB | Important content, noisy audio |
|
|
| `large-v3` | 3.1 GB | Multilingual, production quality |
|
|
|
|
Use `.en` suffix for English-only (more accurate). Drop it for multilingual content.
|
|
|
|
### Supported transcript formats
|
|
|
|
The CLI auto-detects and normalizes: whisper.cpp JSON, OpenAI Whisper API JSON, SRT, VTT, and pre-normalized `[{text, start, end}]` arrays.
|
|
|
|
### Improving transcription quality
|
|
|
|
If captions are inaccurate (wrong words, bad timing):
|
|
|
|
1. **Upgrade the model**: `--model medium.en` or `--model large-v3`
|
|
2. **Set language**: `--language en` to filter non-target speech
|
|
3. **Use an external API**: Transcribe via OpenAI or Groq Whisper API, then import the JSON with `hyperframes transcribe response.json`
|
|
|
|
See the `/hyperframes-captions` skill for full details on model selection and API usage.
|
|
|
|
## Agent Directives
|
|
|
|
### Pre-Work
|
|
|
|
1. **Step 0 Rule**: Before ANY structural refactor on a file >300 LOC, first remove all dead props, unused exports, unused imports, and debug logs. Commit this cleanup separately before starting the real work.
|
|
|
|
2. **Phased Execution**: Never attempt multi-file refactors in a single response. Break work into explicit phases. Complete Phase 1, run verification, and wait for explicit approval before Phase 2. Each phase must touch no more than 5 files.
|
|
|
|
### Code Quality
|
|
|
|
3. **Senior Dev Override**: If architecture is flawed, state is duplicated, or patterns are inconsistent — propose and implement structural fixes. Ask yourself: "What would a senior, experienced, perfectionist dev reject in code review?" Fix all of it.
|
|
|
|
4. **Forced Verification**: You are FORBIDDEN from reporting a task as complete until you have:
|
|
- Run `pnpm build` (typecheck + bundle)
|
|
- Run `pnpm test` (if tests exist for the changed package)
|
|
- Fixed ALL resulting errors
|
|
|
|
### Context Management
|
|
|
|
5. **Sub-Agent Swarming**: For tasks touching >5 independent files, launch parallel sub-agents (5-8 files per agent). Each agent gets its own context window. Sequential processing of large tasks guarantees context decay.
|
|
|
|
6. **Context Decay Awareness**: After 10+ messages in a conversation, re-read any file before editing it. Do not trust memory of file contents — auto-compaction may have silently destroyed that context.
|
|
|
|
7. **File Read Budget**: For files over 500 LOC, use offset and limit parameters to read in sequential chunks. Never assume you have seen a complete file from a single read.
|
|
|
|
### Edit Safety
|
|
|
|
8. **Edit Integrity**: Before EVERY file edit, re-read the file. After editing, read it again to confirm the change applied correctly. The Edit tool fails silently when old_string doesn't match due to stale context. Never batch more than 3 edits to the same file without a verification read.
|
|
|
|
9. **No Semantic Search**: When renaming or changing any function/type/variable, search separately for: direct calls, type-level references, string literals, dynamic imports, re-exports, barrel file entries, and test files. Do not assume a single grep caught everything.
|