ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-07 10:06:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,108 Commits 900 Branches 407 Tags
837ffdcb5d443af357a39b94bb8e25cafe5d4709
Commit Graph
9 Commits
Author SHA1 Message Date
James 4413d25b0e refactor(skills): consolidate tts/whisper guidance into hyperframes-media 
Code review found the new hyperframes-media skill was parallel
content with skills/hyperframes/references/tts.md and the "Whisper
Model Guide" section of transcript-guide.md — same voice table, same
.en-translates-non-English warning, same TTS→transcribe chain in
both places. Plus some scope creep in hyperframes-media (audio/video
HTML snippets that duplicate the canonical track docs in
hyperframes/SKILL.md:265+).

Consolidation:

- hyperframes-media is now the single source of truth for CLI
  invocation, voice selection, multilingual phonemization, whisper
  model selection, and the .en gotcha. Picked up the multilingual
  prefix decoding from the deleted tts.md.
- skills/hyperframes/references/tts.md deleted; the bullet in
  hyperframes/SKILL.md is removed (no replacement — agents land on
  hyperframes-media via its own description).
- skills/hyperframes/references/transcript-guide.md keeps only the
  caption-side concerns: input-format table, mandatory quality
  check, cleaning JS, external-API import path, and the
  "if no transcript exists" flow. The intro bash recipe and Whisper
  Model Guide section both moved to hyperframes-media. Top of the
  file now points to hyperframes-media for CLI/model details.

Other tightening in hyperframes-media:

- Dropped WHAT-narration filler and the inline <audio>/<video> HTML
  snippets — they duplicate the canonical track-attribute docs in
  hyperframes/SKILL.md.
- Added the `id` field (`w0`, `w1`, ...) to the transcript output
  shape — the actual Word interface in
  packages/cli/src/whisper/normalize.ts includes it (optional for
  backwards compat), used by caption override logic.
- Compressed the TTS → Transcribe → Captions chain section.

Net: hyperframes-media 147 → 136 lines, transcript-guide.md 152 →
106 lines, tts.md gone (-75 lines).
 2026-05-04 22:06:02 +00:00
Vance Ingalls 22f0e6a5cd feat(skills): design.md integration, shared video references, Claude Design gaps (#549) 
## What

Major skill infrastructure update: design.md support, shared video-composition references, and creative direction patterns extracted from website-to-hyperframes into the base hyperframes skill.

## Changes

### design.md Integration (lightweight)
- Step 0a reads any format design.md (YAML, prose, tables) — no format mandate
- Brand colors/fonts are strict; video layout adapts per video-composition.md
- Font warning gate: warns user if design.md names fonts without local .woff2 files
- Design picker generates spec-compliant design.md with YAML frontmatter + prose
- Picker generates contextual options from user's prompt (3-4 architectures, 5-6 palettes, 3 type pairings)

### Shared Video References (extracted from website-to-hyperframes)
- `video-composition.md` — density, scale, color presence, frame composition rules. Light canvas guidance (don't override user palette). **Always read.**
- `beat-direction.md` — per-beat planning (concept → mood → choreography verbs → transition), rhythm templates by video type
- `techniques.md` — 11 visual techniques with code patterns (SVG drawing, Canvas 2D, kinetic type, Lottie, etc.)
- `narration.md` — pacing, tone, script structure, number pronunciation, hooks
- `motion-principles.md` — gained image motion treatment + load-bearing GSAP rules

### Claude Design Transfer Brief (6 gaps applied)
1. Discovery step for exploratory requests (audience, platform, priority, variations)
2. Anti-scope-creep: "build what was asked, every element earns its place"
3. Read-source discipline: "read actual files, don't guess"
4. Rhythm planning: declare scene rhythm before implementing
5. Variations as first-class output for exploratory requests
6. Two-phase verification: fast checks block, slow checks parallel

### Prompt Expansion Updated
- Uses beat-direction format (concept → mood → verbs → depth layers)
- Rhythm declaration before scene breakdown
- References video-composition.md and beat-direction.md

### Key Design Decision
**design.md = brand truth, not video layout spec.** Background color is strict from design.md (don't switch light to dark). Video-composition rules teach how to make any palette work cinematically.

## Files Changed (16)

**New shared references:**
- `skills/hyperframes/references/video-composition.md`
- `skills/hyperframes/references/beat-direction.md`
- `skills/hyperframes/references/techniques.md`
- `skills/hyperframes/references/narration.md`

**Updated:**
- `skills/hyperframes/SKILL.md` — discovery, anti-scope-creep, rhythm, variations, two-phase verify, new references
- `skills/hyperframes/references/prompt-expansion.md` — beat-direction format
- `skills/hyperframes/references/motion-principles.md` — image treatment + GSAP rules
- `skills/hyperframes/references/design-picker.md` — contextual generation
- `skills/hyperframes/visual-styles.md` — YAML token blocks per preset
- `skills/hyperframes/house-style.md` — design.md precedence
- `skills/hyperframes/templates/design-picker.html` — spec-compliant output
- `skills/website-to-hyperframes/references/*` — now reference shared files

## Test plan

- [x] Design picker generates and serves correctly
- [x] Picker output is spec-compliant design.md
- [x] Composition built from picker design.md renders in Studio
- [x] Before/after eval: 4 topics × 2 versions showing skill guidance impact
- [x] Light canvas compositions respect user palette (don't switch to dark)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-29 17:48:54 -07:00
James Russo 4a55bc8673 feat(cli): add --lang and auto-infer phonemizer locale from voice prefix (#351) 
* feat(cli): add --lang and auto-infer phonemizer locale from voice prefix

`hyperframes tts` was calling Kokoro's `model.create(text, voice=, speed=)`
with no language argument, so Kokoro's default phonemizer (en-us) was
applied regardless of the voice selected. Picking `ef_dora` or `jf_alpha`
and feeding it Spanish or Japanese text produced English-phonemized
output.

Closes #349.

- `manager.ts`: add `SUPPORTED_LANGS`, `inferLangFromVoiceId`, and
  `isSupportedLang`. Attach a `defaultLang` field to every bundled voice
  and expand the bundled list with `ef_dora`, `ff_siwis`, `jf_alpha`,
  `zf_xiaobei` so `--list` surfaces multilingual options.
- `synthesize.ts`: accept optional `lang: SupportedLang` in
  `SynthesizeOptions`, forward it to the Python worker as `argv[7]`.
  The worker introspects `Kokoro.create`'s signature and only passes
  `lang=` when the installed kokoro-onnx version supports it. Returned
  metadata now includes `lang` and `langApplied` so callers can detect
  silent no-ops. Bump the cached script filename to `synth-v2.py` so
  existing installs pick up the new script automatically.
- `commands/tts.ts`: add `--lang, -l` with validation against
  `SUPPORTED_LANGS`. Resolution order is explicit `--lang` > inferred
  from voice prefix > `en-us`. When explicit lang disagrees with the
  voice-implied lang (legitimate for stylized accents), emit a
  dim-level hint; suppress under `--json`. When kokoro-onnx silently
  ignores the kwarg, log that too. Update `--list` with a new
  "Lang code" column and add multilingual examples.
- Tests: new `manager.test.ts` covering every supported prefix, the
  unknown-prefix fallback, case-insensitivity, `isSupportedLang`
  validation, and a regression guard that every bundled voice has a
  valid `defaultLang` matching its ID.
- Docs: `docs/packages/cli.mdx` and `skills/hyperframes/references/tts.md`
  updated with the flag, examples, the espeak-ng dependency note for
  non-English phonemization, and the voice-prefix → lang table.

Backward compatibility:
- English voices (a*/b* prefixes) continue to phonemize as en-us / en-gb
  — no change.
- Non-English voices now phonemize correctly by default (bug fix, not a
  regression).
- Older kokoro-onnx versions that don't know the `lang` kwarg keep
  working via signature introspection; the CLI logs a dim note if
  `--lang` was requested but ignored.

Verification:
- `bun --cwd packages/cli test` — 128 tests pass (incl. 17 new).
- `bunx oxlint` and `bunx oxfmt --check` clean on changed files.
- `bun run build` succeeds.
- `npx tsx packages/cli/src/cli.ts tts --help` / `--list` render cleanly;
  invalid `--lang` produces a clean error with the valid-codes list.

* refactor(cli): simplify tts --lang implementation

Post-review cleanup on #351. Net -21 lines.

- Drop `defaultLang` field + `makeVoice()` helper from VoiceInfo —
  compute via `inferLangFromVoiceId(v.id)` at read time in listVoices.
  The only reader was the --list table; caching the derived value on
  every voice added a self-consistency invariant we had to test.
- Drop redundant `lang` field from SynthesizeResult — caller already
  knows the requested lang since it passed it in; only `langApplied`
  carries information the caller can't derive.
- Use `errorBox` for --lang validation to match the house style in
  render.ts (other validation errors already use errorBox).
- Reuse existing `langList` module constant in the validation error
  instead of re-joining SUPPORTED_LANGS.
- Inline `DEFAULT_LANG` — used once in inferLangFromVoiceId.
- Trim WHAT-restating comments and the duplicate prefix-enumeration
  JSDoc on inferLangFromVoiceId (VOICE_PREFIX_LANG already carries
  per-row comments).
- Clean up orphaned `synth*.py` files in ~/.cache/hyperframes/tts
  when writing the current versioned script, so repeated upgrades
  don't leak files.
- Drop the `EN-US` case-sensitive-rejection test assertion — the CLI
  lowercases input before validation, so accepting mixed case is a
  feature, not a bug.

Tests: 16/16 in `manager.test.ts`, 127/127 full CLI suite pass.
Lint + format + typecheck clean.
 2026-04-20 10:51:00 -07:00
Vance IngallsandClaude Opus 4.6 a262ad59f3 chore(skills): remove 1,685 lines of redundant skill content (#283) 
* chore(skills): remove 1,685 lines of redundant and irrelevant skill content

- Remove 5 GSAP references irrelevant to HyperFrames (scrolltrigger,
  plugins, react, frameworks, utils) — no scroll, no frameworks, no
  interactive plugins in video compositions
- Remove shader-setup.md and shader-transitions.md — duplicated by
  @hyperframes/shader-transitions package (packages/shader-transitions/)
- Remove marker-highlight.md and examples.md — JS library docs superseded
  by css-patterns.md (deterministic, GSAP-driven, fully seekable)
- Trim CLAUDE.md to dev-only instructions — move product docs (transcription,
  TTS, player) to skills where they belong
- Deduplicate house-style.md typography/motion sections — point to
  dedicated references instead of repeating rules
- Clean up stale references to deleted files across SKILL.md and catalog.md
- Update gsap skill description to reflect HyperFrames-only scope

Skills: 5,230 → 3,714 lines (29% reduction)
CLAUDE.md: 204 → 50 lines (75% reduction)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): update broken marker-highlight.md references in captions.md

Point to css-patterns.md instead of deleted marker-highlight.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): update stale shader CSS rule to reference package API

BG_COLOR was from the old manual setup. Now it's bgColor in the
@hyperframes/shader-transitions init() config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address 6 doc gaps surfaced by eval agents

P0: Document HyperShader as IIFE global name in shader-transitions README
P1: Replace async fetch() with sync XHR in effects.md audio data loading
    (fetch violates synchronous timeline construction rule in SKILL.md)
P1: Change <div> to <span> in css-patterns.md marker highlight patterns
    (<div> inside <p> is invalid HTML, breaks layout in inline contexts)
P2: Clarify bgColor as fallback color in shader-transitions README
P2: Add data-start to Composition Clips table in SKILL.md
    (root composition element needs data-start="0", linter enforces it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(templates): update init templates to match trimmed skill scope

- Remove ScrollTrigger/plugins/React/Vue/Svelte from gsap skill description
- Replace class="clip" with accurate pattern examples in skill intro text
  (class="clip" is still in Key Rules where it belongs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): remove contradictory 5:1 contrast threshold from house-style

house-style.md said 5:1 minimum, but hyperframes validate enforces
WCAG AA (4.5:1 normal text, 3:1 large text). Now defers to validate
instead of stating a conflicting number.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-15 10:51:10 -07:00
Vance Ingalls cb3d94c2a5 feat: add @hyperframes/shader-transitions package (#251) 
## Summary

New `@hyperframes/shader-transitions` package that encapsulates WebGL shader transitions into a single `HyperShader.init()` call. Replaces ~200 lines of per-composition boilerplate that LLMs failed to wire correctly 60% of the time.

### API

```js
var tl = HyperShader.init({
  bgColor: "#0a0a1a",
  accentColor: "#6366f1",
  scenes: ["scene1", "scene2", "scene3", "scene4", "scene5"],
  transitions: [
    { time: 7.2, shader: "cross-warp-morph", duration: 0.7 },
    { time: 15.2, shader: "domain-warp", duration: 0.7 },
  ]
});
tl.from("#s1-title", { y: 50, opacity: 0, duration: 0.7 }, 0.3);
```

### What the library handles

- **13 shader programs**: domain-warp, ridged-burn, whip-pan, sdf-iris, ripple-waves, gravitational-lens, cinematic-zoom, chromatic-split, glitch, swirl-vortex, thermal-distortion, cross-warp-morph, light-leak
- **html2canvas** bundled as dependency (not CDN) — single script tag for CLI users
- **DOM-during-holds**: canvas hidden between transitions, GSAP animations play on live DOM
- **Async capture with pause/resume**: timeline pauses during capture, resumes after textures uploaded — prevents progress tween from running ahead
- **Accent color theming**: `accentColor` derives dark/mid/bright uniforms. Burns, glows, leaks match the composition palette
- **Graceful degradation**: falls back silently when WebGL unavailable

### Code quality (from 3 review agents)

- No `!` non-null assertions — all WebGL creation calls throw on failure
- Vertex shader compiled once, cached across all programs
- Uniform/attribute locations cached per program via WeakMap (not looked up every frame)
- Captured canvases freed after texture upload (8MB each)
- Single timeline creation (was creating two, discarding one)
- Shared `tickShader()` render callback (was copy-pasted)
- `.finally()` for DOM restore in capture (was duplicated in `.then`/`.catch`)
- `parseHex` validates input (was silently producing NaN on invalid hex)
- Dead `ND`/`CP` shader library exports removed

### Shader-compatible CSS rules (transitions.md)

6 rules for compositions using shader transitions:
1. No `transparent` in gradients (canvas interpolates through black)
2. No gradient backgrounds on elements < 4px
3. No CSS variables on captured elements
4. `data-no-capture` for uncapturable decoratives
5. No gradient opacity < 0.15
6. Every `.scene` must have explicit `background-color` matching `bgColor`

### Build output

- IIFE (~214KB with html2canvas bundled, ~65KB gzipped) — `window.HyperShader`
- ESM + CJS + TypeScript declarations
- tsup build following `@hyperframes/player` conventions

## Test plan
- [ ] `bun run build` succeeds (includes shader-transitions)
- [ ] `bunx oxlint packages/shader-transitions/src/` — 0 errors
- [ ] Create a composition using `HyperShader.init()` — verify transitions fire, DOM animations play, accent colors match
- [ ] Test graceful degradation: composition works without WebGL (no transitions, no crash)
- [ ] Verify pause/resume: scrub to transition boundary — no jump in progress

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:44:00 -07:00
Vance Ingalls 5de2af5bde feat(skills): improve hyperframes composition quality rules (#250) 
## Summary

Overhaul the hyperframes composition skill based on 26 eval rounds (~100 generated compositions). The goal: prevent known AI design tells and composition bugs while giving the LLM maximum creative freedom.

### Typography (`fonts.md` → `typography.md`)
- Two-tier banned font list (32 fonts): tier 1 bans training-data defaults, tier 2 bans the reflex replacements
- Font discovery script: queries Google Fonts API, 5 dynamic categories, top 5 randomized per run
- Selection philosophy: register-first thinking, cross-check assumptions

### Google Fonts on-demand (`deterministicFonts.ts`)
- Any Google Font works without pre-bundling — compiler fetches woff2 at compile time
- Cached to `~/.cache/hyperframes/fonts/<slug>/<weight>-<style>.woff2`
- Parallel woff2 fetches via `Promise.allSettled` (was sequential)
- Single `mkdirSync({ recursive: true })` per family (was `existsSync` x11)
- Skip redundant `readFileSync` when buffer is already in memory from fetch

### Layout rules (`SKILL.md`)
- Flexbox with gap for content text — prevents overlap from absolute positioning
- `position: absolute` reserved for decoratives only
- Cards/containers explicitly banned

### Background layer (`house-style.md`)
- 3-5 persistent decorative elements per scene (glows, ghost text, accent lines)
- All decoratives MUST have ambient GSAP animation — static decoratives banned
- WRONG/RIGHT code examples

### Transition rules (`SKILL.md`)
- Always use transitions, always entrance animations, exit animations banned except final scene
- WRONG/RIGHT code examples showing banned exit patterns

### Other
- Flash cut transition removed
- CLAUDE.md: `bun install` / `bun run build` / `bun run test` (was pnpm)
- house-style.md trimmed from 184 to ~80 lines
- SKILL.md trimmed from 364 to ~230 lines

## Test plan
- [ ] `bun install` succeeds, workspace links resolve
- [ ] `bun run build` succeeds
- [ ] `npx hyperframes lint` passes on existing compositions
- [ ] Generate a composition with `/hyperframes` skill — verify flexbox, background decoratives with animation, entrance-only animations, no banned fonts
- [ ] Verify Google Fonts on-demand: use a non-bundled font, run `npx hyperframes preview`

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:40:57 -07:00
Vance IngallsandClaude Opus 4.6 4c5b8e38a1 feat(skills): add typography and motion principles, fix validate $& bug (#228) 
Add two new skill reference files that address measured LLM composition failures:

- fonts.md: Typography principles — banned fonts, guardrails for violations
  (pairing two sans-serifs, defaulting to 400/700 weight), and guidance the LLM
  genuinely doesn't apply without being told (register switching, tension as
  meaning, easing direction as emotion). Includes Google Fonts API discovery
  script with 7-category multi-strategy query.

- motion-principles.md: Motion design principles — guardrails for same-ease and
  same-speed defaults, y-axis entrance monotony, and guidance for build/breathe/
  resolve scene structure, hard cuts as intentional transitions, visual
  composition rules for video-not-web density.

Both files validated against baseline evals: 3 compositions created without
guidance confirmed the LLM reaches for banned fonts (Inter, Cormorant Garamond,
Playfair Display, Roboto Condensed), uses power2.out on 45-72% of tweens,
enters 80%+ of elements from y-axis, and pairs multiple sans-serifs.

Also:
- Fix validate.ts $& replacement bug (runtime source containing $& caused
  String.prototype.replace to re-insert the matched <script src=""> tag)
- Clean up font loading guidance across skills (compiler embeds automatically)
- Update house-style.md to reference fonts.md

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 20:49:46 -07:00
Vance IngallsandClaude Opus 4.6 569513145b feat(skills): add WebGL shader transitions and restructure catalog (#213) 
* feat(skills): add WebGL shader transitions and restructure catalog

Add 14 WebGL fragment shader transitions to the transitions skill:
domain warp dissolve, ridged burn, whip pan, SDF iris, ripple waves,
gravitational lens, cinematic zoom, chromatic radial split, glitch,
swirl vortex, thermal distortion, flash through white, cross-warp
morph, and light leak (shader).

Restructure catalog.md from a 1045-line monolith into a 105-line
routing layer with 15 reference files. SKILL.md loads at 101 lines,
catalog.md loads at 105 lines — reference files loaded on demand
only for the transition type being implemented.

Key additions:
- Full WebGL setup boilerplate with media capture (images, video,
  object-fit: cover, live video re-upload during transitions)
- Hard rules for shader transitions capturing all bugs found during
  development (Y-flip, preserveDrawingBuffer, fwidth, boomerang,
  tween proxy reuse, tl.call vs onComplete)
- CSS vs Shader decision guide in SKILL.md
- Visual pattern warning against repeating geometric patterns
- Shader transitions slotted into mood/energy mapping tables
- Noise libraries: quintic C2, ridged, erosion FBM, cosine palette

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(skills): fold transitions into hyperframes skill

Move transitions from a standalone skill (4th top-level) into
hyperframes/references/transitions/, aligning with the consolidation
in #211 that reduced 15 skills to 3. Fewer standalone skills means
higher trigger reliability for multi-skill tasks.

Also removes stale text-burn-dom.html reference from css-destruction.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 14:50:13 -07:00
James RussoandClaude Opus 4.6 0a0d5d3654 refactor(skills): consolidate 15 skills into 3 (#211) 
* refactor(skills): consolidate 15 skills into 3 for better trigger reliability

Merge 9 GSAP skills (core, timeline, scrolltrigger, plugins, utils, react,
frameworks, performance, effects) and 6 HyperFrames skills (compose, captions,
tts, audio-reactive, marker-highlight, cli) into 3 consolidated skills:

- `gsap` — core API + timelines + performance in SKILL.md; scrolltrigger,
  plugins, utils, react, frameworks, effects in references/
- `hyperframes` — composition authoring rules in SKILL.md; captions, tts,
  audio-reactive, marker-highlight in references/
- `hyperframes-cli` — CLI commands (init, lint, preview, render, etc.)

Why: With 15 separate skills, agents must correctly trigger the right subset
for any task. "Create an animated video with captions" needed 6+ skills to
fire — each with ~90% trigger accuracy means ~53% chance of getting all of
them. With 3 skills, that same task needs just `hyperframes` + `gsap` (~90%
both fire). Progressive disclosure still works via references/ files loaded
on demand.

Also fixes: CLAUDE.md referenced `window.__GSAP_TIMELINE` (incorrect) —
corrected to `window.__timelines`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): add --skip-skills flag to init command

Allow skipping the AI coding skills installation prompt during
`hyperframes init` with `--skip-skills`. Useful when skills are
already installed or when the user wants to scaffold without them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address code review feedback on consolidation

Restore content lost during over-compression:

- captions: fix overflow to `visible` (not hidden — clips glow effects),
  add container pattern warning, scale headroom formula, and self-lint
  placement guidance
- audio-reactive: restore sampling frequency pattern (per-frame tl.call
  loop vs single tween) and textShadow-on-container gotcha
- effects/typewriter: restore word rotation, appending words, spacing
  with static text, and multi-line cursor handoff patterns
- effects/audio-visualizer: restore spatial mapping conventions, fetch vs
  inline loading, WebGL/DOM rendering approaches, and canvas layering
- hyperframes-cli: restore --strict-all flag in render flags table

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update build:copy and template for consolidated skill names

- build:copy: reference skills/hyperframes, skills/hyperframes-cli,
  skills/gsap instead of the old 15 skill directory names
- _shared/CLAUDE.md template: update skill table to consolidated names

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 11:21:49 -07:00