ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-07 10:06:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
592 Commits 900 Branches 407 Tags
22f0e6a5cd4f46adf363b7f0fc522c9b60b4e69f
Commit Graph
8 Commits
Author SHA1 Message Date
Vance Ingalls 22f0e6a5cd feat(skills): design.md integration, shared video references, Claude Design gaps (#549) 
## What

Major skill infrastructure update: design.md support, shared video-composition references, and creative direction patterns extracted from website-to-hyperframes into the base hyperframes skill.

## Changes

### design.md Integration (lightweight)
- Step 0a reads any format design.md (YAML, prose, tables) — no format mandate
- Brand colors/fonts are strict; video layout adapts per video-composition.md
- Font warning gate: warns user if design.md names fonts without local .woff2 files
- Design picker generates spec-compliant design.md with YAML frontmatter + prose
- Picker generates contextual options from user's prompt (3-4 architectures, 5-6 palettes, 3 type pairings)

### Shared Video References (extracted from website-to-hyperframes)
- `video-composition.md` — density, scale, color presence, frame composition rules. Light canvas guidance (don't override user palette). **Always read.**
- `beat-direction.md` — per-beat planning (concept → mood → choreography verbs → transition), rhythm templates by video type
- `techniques.md` — 11 visual techniques with code patterns (SVG drawing, Canvas 2D, kinetic type, Lottie, etc.)
- `narration.md` — pacing, tone, script structure, number pronunciation, hooks
- `motion-principles.md` — gained image motion treatment + load-bearing GSAP rules

### Claude Design Transfer Brief (6 gaps applied)
1. Discovery step for exploratory requests (audience, platform, priority, variations)
2. Anti-scope-creep: "build what was asked, every element earns its place"
3. Read-source discipline: "read actual files, don't guess"
4. Rhythm planning: declare scene rhythm before implementing
5. Variations as first-class output for exploratory requests
6. Two-phase verification: fast checks block, slow checks parallel

### Prompt Expansion Updated
- Uses beat-direction format (concept → mood → verbs → depth layers)
- Rhythm declaration before scene breakdown
- References video-composition.md and beat-direction.md

### Key Design Decision
**design.md = brand truth, not video layout spec.** Background color is strict from design.md (don't switch light to dark). Video-composition rules teach how to make any palette work cinematically.

## Files Changed (16)

**New shared references:**
- `skills/hyperframes/references/video-composition.md`
- `skills/hyperframes/references/beat-direction.md`
- `skills/hyperframes/references/techniques.md`
- `skills/hyperframes/references/narration.md`

**Updated:**
- `skills/hyperframes/SKILL.md` — discovery, anti-scope-creep, rhythm, variations, two-phase verify, new references
- `skills/hyperframes/references/prompt-expansion.md` — beat-direction format
- `skills/hyperframes/references/motion-principles.md` — image treatment + GSAP rules
- `skills/hyperframes/references/design-picker.md` — contextual generation
- `skills/hyperframes/visual-styles.md` — YAML token blocks per preset
- `skills/hyperframes/house-style.md` — design.md precedence
- `skills/hyperframes/templates/design-picker.html` — spec-compliant output
- `skills/website-to-hyperframes/references/*` — now reference shared files

## Test plan

- [x] Design picker generates and serves correctly
- [x] Picker output is spec-compliant design.md
- [x] Composition built from picker design.md renders in Studio
- [x] Before/after eval: 4 topics × 2 versions showing skill guidance impact
- [x] Light canvas compositions respect user palette (don't switch to dark)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-29 17:48:54 -07:00
James Russo 4a55bc8673 feat(cli): add --lang and auto-infer phonemizer locale from voice prefix (#351) 
* feat(cli): add --lang and auto-infer phonemizer locale from voice prefix

`hyperframes tts` was calling Kokoro's `model.create(text, voice=, speed=)`
with no language argument, so Kokoro's default phonemizer (en-us) was
applied regardless of the voice selected. Picking `ef_dora` or `jf_alpha`
and feeding it Spanish or Japanese text produced English-phonemized
output.

Closes #349.

- `manager.ts`: add `SUPPORTED_LANGS`, `inferLangFromVoiceId`, and
  `isSupportedLang`. Attach a `defaultLang` field to every bundled voice
  and expand the bundled list with `ef_dora`, `ff_siwis`, `jf_alpha`,
  `zf_xiaobei` so `--list` surfaces multilingual options.
- `synthesize.ts`: accept optional `lang: SupportedLang` in
  `SynthesizeOptions`, forward it to the Python worker as `argv[7]`.
  The worker introspects `Kokoro.create`'s signature and only passes
  `lang=` when the installed kokoro-onnx version supports it. Returned
  metadata now includes `lang` and `langApplied` so callers can detect
  silent no-ops. Bump the cached script filename to `synth-v2.py` so
  existing installs pick up the new script automatically.
- `commands/tts.ts`: add `--lang, -l` with validation against
  `SUPPORTED_LANGS`. Resolution order is explicit `--lang` > inferred
  from voice prefix > `en-us`. When explicit lang disagrees with the
  voice-implied lang (legitimate for stylized accents), emit a
  dim-level hint; suppress under `--json`. When kokoro-onnx silently
  ignores the kwarg, log that too. Update `--list` with a new
  "Lang code" column and add multilingual examples.
- Tests: new `manager.test.ts` covering every supported prefix, the
  unknown-prefix fallback, case-insensitivity, `isSupportedLang`
  validation, and a regression guard that every bundled voice has a
  valid `defaultLang` matching its ID.
- Docs: `docs/packages/cli.mdx` and `skills/hyperframes/references/tts.md`
  updated with the flag, examples, the espeak-ng dependency note for
  non-English phonemization, and the voice-prefix → lang table.

Backward compatibility:
- English voices (a*/b* prefixes) continue to phonemize as en-us / en-gb
  — no change.
- Non-English voices now phonemize correctly by default (bug fix, not a
  regression).
- Older kokoro-onnx versions that don't know the `lang` kwarg keep
  working via signature introspection; the CLI logs a dim note if
  `--lang` was requested but ignored.

Verification:
- `bun --cwd packages/cli test` — 128 tests pass (incl. 17 new).
- `bunx oxlint` and `bunx oxfmt --check` clean on changed files.
- `bun run build` succeeds.
- `npx tsx packages/cli/src/cli.ts tts --help` / `--list` render cleanly;
  invalid `--lang` produces a clean error with the valid-codes list.

* refactor(cli): simplify tts --lang implementation

Post-review cleanup on #351. Net -21 lines.

- Drop `defaultLang` field + `makeVoice()` helper from VoiceInfo —
  compute via `inferLangFromVoiceId(v.id)` at read time in listVoices.
  The only reader was the --list table; caching the derived value on
  every voice added a self-consistency invariant we had to test.
- Drop redundant `lang` field from SynthesizeResult — caller already
  knows the requested lang since it passed it in; only `langApplied`
  carries information the caller can't derive.
- Use `errorBox` for --lang validation to match the house style in
  render.ts (other validation errors already use errorBox).
- Reuse existing `langList` module constant in the validation error
  instead of re-joining SUPPORTED_LANGS.
- Inline `DEFAULT_LANG` — used once in inferLangFromVoiceId.
- Trim WHAT-restating comments and the duplicate prefix-enumeration
  JSDoc on inferLangFromVoiceId (VOICE_PREFIX_LANG already carries
  per-row comments).
- Clean up orphaned `synth*.py` files in ~/.cache/hyperframes/tts
  when writing the current versioned script, so repeated upgrades
  don't leak files.
- Drop the `EN-US` case-sensitive-rejection test assertion — the CLI
  lowercases input before validation, so accepting mixed case is a
  feature, not a bug.

Tests: 16/16 in `manager.test.ts`, 127/127 full CLI suite pass.
Lint + format + typecheck clean.
 2026-04-20 10:51:00 -07:00
Vance IngallsandClaude Opus 4.6 a262ad59f3 chore(skills): remove 1,685 lines of redundant skill content (#283) 
* chore(skills): remove 1,685 lines of redundant and irrelevant skill content

- Remove 5 GSAP references irrelevant to HyperFrames (scrolltrigger,
  plugins, react, frameworks, utils) — no scroll, no frameworks, no
  interactive plugins in video compositions
- Remove shader-setup.md and shader-transitions.md — duplicated by
  @hyperframes/shader-transitions package (packages/shader-transitions/)
- Remove marker-highlight.md and examples.md — JS library docs superseded
  by css-patterns.md (deterministic, GSAP-driven, fully seekable)
- Trim CLAUDE.md to dev-only instructions — move product docs (transcription,
  TTS, player) to skills where they belong
- Deduplicate house-style.md typography/motion sections — point to
  dedicated references instead of repeating rules
- Clean up stale references to deleted files across SKILL.md and catalog.md
- Update gsap skill description to reflect HyperFrames-only scope

Skills: 5,230 → 3,714 lines (29% reduction)
CLAUDE.md: 204 → 50 lines (75% reduction)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): update broken marker-highlight.md references in captions.md

Point to css-patterns.md instead of deleted marker-highlight.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): update stale shader CSS rule to reference package API

BG_COLOR was from the old manual setup. Now it's bgColor in the
@hyperframes/shader-transitions init() config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address 6 doc gaps surfaced by eval agents

P0: Document HyperShader as IIFE global name in shader-transitions README
P1: Replace async fetch() with sync XHR in effects.md audio data loading
    (fetch violates synchronous timeline construction rule in SKILL.md)
P1: Change <div> to <span> in css-patterns.md marker highlight patterns
    (<div> inside <p> is invalid HTML, breaks layout in inline contexts)
P2: Clarify bgColor as fallback color in shader-transitions README
P2: Add data-start to Composition Clips table in SKILL.md
    (root composition element needs data-start="0", linter enforces it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(templates): update init templates to match trimmed skill scope

- Remove ScrollTrigger/plugins/React/Vue/Svelte from gsap skill description
- Replace class="clip" with accurate pattern examples in skill intro text
  (class="clip" is still in Key Rules where it belongs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): remove contradictory 5:1 contrast threshold from house-style

house-style.md said 5:1 minimum, but hyperframes validate enforces
WCAG AA (4.5:1 normal text, 3:1 large text). Now defers to validate
instead of stating a conflicting number.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-15 10:51:10 -07:00
Vance Ingalls cb3d94c2a5 feat: add @hyperframes/shader-transitions package (#251) 
## Summary

New `@hyperframes/shader-transitions` package that encapsulates WebGL shader transitions into a single `HyperShader.init()` call. Replaces ~200 lines of per-composition boilerplate that LLMs failed to wire correctly 60% of the time.

### API

```js
var tl = HyperShader.init({
  bgColor: "#0a0a1a",
  accentColor: "#6366f1",
  scenes: ["scene1", "scene2", "scene3", "scene4", "scene5"],
  transitions: [
    { time: 7.2, shader: "cross-warp-morph", duration: 0.7 },
    { time: 15.2, shader: "domain-warp", duration: 0.7 },
  ]
});
tl.from("#s1-title", { y: 50, opacity: 0, duration: 0.7 }, 0.3);
```

### What the library handles

- **13 shader programs**: domain-warp, ridged-burn, whip-pan, sdf-iris, ripple-waves, gravitational-lens, cinematic-zoom, chromatic-split, glitch, swirl-vortex, thermal-distortion, cross-warp-morph, light-leak
- **html2canvas** bundled as dependency (not CDN) — single script tag for CLI users
- **DOM-during-holds**: canvas hidden between transitions, GSAP animations play on live DOM
- **Async capture with pause/resume**: timeline pauses during capture, resumes after textures uploaded — prevents progress tween from running ahead
- **Accent color theming**: `accentColor` derives dark/mid/bright uniforms. Burns, glows, leaks match the composition palette
- **Graceful degradation**: falls back silently when WebGL unavailable

### Code quality (from 3 review agents)

- No `!` non-null assertions — all WebGL creation calls throw on failure
- Vertex shader compiled once, cached across all programs
- Uniform/attribute locations cached per program via WeakMap (not looked up every frame)
- Captured canvases freed after texture upload (8MB each)
- Single timeline creation (was creating two, discarding one)
- Shared `tickShader()` render callback (was copy-pasted)
- `.finally()` for DOM restore in capture (was duplicated in `.then`/`.catch`)
- `parseHex` validates input (was silently producing NaN on invalid hex)
- Dead `ND`/`CP` shader library exports removed

### Shader-compatible CSS rules (transitions.md)

6 rules for compositions using shader transitions:
1. No `transparent` in gradients (canvas interpolates through black)
2. No gradient backgrounds on elements < 4px
3. No CSS variables on captured elements
4. `data-no-capture` for uncapturable decoratives
5. No gradient opacity < 0.15
6. Every `.scene` must have explicit `background-color` matching `bgColor`

### Build output

- IIFE (~214KB with html2canvas bundled, ~65KB gzipped) — `window.HyperShader`
- ESM + CJS + TypeScript declarations
- tsup build following `@hyperframes/player` conventions

## Test plan
- [ ] `bun run build` succeeds (includes shader-transitions)
- [ ] `bunx oxlint packages/shader-transitions/src/` — 0 errors
- [ ] Create a composition using `HyperShader.init()` — verify transitions fire, DOM animations play, accent colors match
- [ ] Test graceful degradation: composition works without WebGL (no transitions, no crash)
- [ ] Verify pause/resume: scrub to transition boundary — no jump in progress

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:44:00 -07:00
Vance Ingalls 5de2af5bde feat(skills): improve hyperframes composition quality rules (#250) 
## Summary

Overhaul the hyperframes composition skill based on 26 eval rounds (~100 generated compositions). The goal: prevent known AI design tells and composition bugs while giving the LLM maximum creative freedom.

### Typography (`fonts.md` → `typography.md`)
- Two-tier banned font list (32 fonts): tier 1 bans training-data defaults, tier 2 bans the reflex replacements
- Font discovery script: queries Google Fonts API, 5 dynamic categories, top 5 randomized per run
- Selection philosophy: register-first thinking, cross-check assumptions

### Google Fonts on-demand (`deterministicFonts.ts`)
- Any Google Font works without pre-bundling — compiler fetches woff2 at compile time
- Cached to `~/.cache/hyperframes/fonts/<slug>/<weight>-<style>.woff2`
- Parallel woff2 fetches via `Promise.allSettled` (was sequential)
- Single `mkdirSync({ recursive: true })` per family (was `existsSync` x11)
- Skip redundant `readFileSync` when buffer is already in memory from fetch

### Layout rules (`SKILL.md`)
- Flexbox with gap for content text — prevents overlap from absolute positioning
- `position: absolute` reserved for decoratives only
- Cards/containers explicitly banned

### Background layer (`house-style.md`)
- 3-5 persistent decorative elements per scene (glows, ghost text, accent lines)
- All decoratives MUST have ambient GSAP animation — static decoratives banned
- WRONG/RIGHT code examples

### Transition rules (`SKILL.md`)
- Always use transitions, always entrance animations, exit animations banned except final scene
- WRONG/RIGHT code examples showing banned exit patterns

### Other
- Flash cut transition removed
- CLAUDE.md: `bun install` / `bun run build` / `bun run test` (was pnpm)
- house-style.md trimmed from 184 to ~80 lines
- SKILL.md trimmed from 364 to ~230 lines

## Test plan
- [ ] `bun install` succeeds, workspace links resolve
- [ ] `bun run build` succeeds
- [ ] `npx hyperframes lint` passes on existing compositions
- [ ] Generate a composition with `/hyperframes` skill — verify flexbox, background decoratives with animation, entrance-only animations, no banned fonts
- [ ] Verify Google Fonts on-demand: use a non-bundled font, run `npx hyperframes preview`

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:40:57 -07:00
Vance IngallsandClaude Opus 4.6 4c5b8e38a1 feat(skills): add typography and motion principles, fix validate $& bug (#228) 
Add two new skill reference files that address measured LLM composition failures:

- fonts.md: Typography principles — banned fonts, guardrails for violations
  (pairing two sans-serifs, defaulting to 400/700 weight), and guidance the LLM
  genuinely doesn't apply without being told (register switching, tension as
  meaning, easing direction as emotion). Includes Google Fonts API discovery
  script with 7-category multi-strategy query.

- motion-principles.md: Motion design principles — guardrails for same-ease and
  same-speed defaults, y-axis entrance monotony, and guidance for build/breathe/
  resolve scene structure, hard cuts as intentional transitions, visual
  composition rules for video-not-web density.

Both files validated against baseline evals: 3 compositions created without
guidance confirmed the LLM reaches for banned fonts (Inter, Cormorant Garamond,
Playfair Display, Roboto Condensed), uses power2.out on 45-72% of tweens,
enters 80%+ of elements from y-axis, and pairs multiple sans-serifs.

Also:
- Fix validate.ts $& replacement bug (runtime source containing $& caused
  String.prototype.replace to re-insert the matched <script src=""> tag)
- Clean up font loading guidance across skills (compiler embeds automatically)
- Update house-style.md to reference fonts.md

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 20:49:46 -07:00
Vance IngallsandClaude Opus 4.6 569513145b feat(skills): add WebGL shader transitions and restructure catalog (#213) 
* feat(skills): add WebGL shader transitions and restructure catalog

Add 14 WebGL fragment shader transitions to the transitions skill:
domain warp dissolve, ridged burn, whip pan, SDF iris, ripple waves,
gravitational lens, cinematic zoom, chromatic radial split, glitch,
swirl vortex, thermal distortion, flash through white, cross-warp
morph, and light leak (shader).

Restructure catalog.md from a 1045-line monolith into a 105-line
routing layer with 15 reference files. SKILL.md loads at 101 lines,
catalog.md loads at 105 lines — reference files loaded on demand
only for the transition type being implemented.

Key additions:
- Full WebGL setup boilerplate with media capture (images, video,
  object-fit: cover, live video re-upload during transitions)
- Hard rules for shader transitions capturing all bugs found during
  development (Y-flip, preserveDrawingBuffer, fwidth, boomerang,
  tween proxy reuse, tl.call vs onComplete)
- CSS vs Shader decision guide in SKILL.md
- Visual pattern warning against repeating geometric patterns
- Shader transitions slotted into mood/energy mapping tables
- Noise libraries: quintic C2, ridged, erosion FBM, cosine palette

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(skills): fold transitions into hyperframes skill

Move transitions from a standalone skill (4th top-level) into
hyperframes/references/transitions/, aligning with the consolidation
in #211 that reduced 15 skills to 3. Fewer standalone skills means
higher trigger reliability for multi-skill tasks.

Also removes stale text-burn-dom.html reference from css-destruction.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 14:50:13 -07:00
James RussoandClaude Opus 4.6 0a0d5d3654 refactor(skills): consolidate 15 skills into 3 (#211) 
* refactor(skills): consolidate 15 skills into 3 for better trigger reliability

Merge 9 GSAP skills (core, timeline, scrolltrigger, plugins, utils, react,
frameworks, performance, effects) and 6 HyperFrames skills (compose, captions,
tts, audio-reactive, marker-highlight, cli) into 3 consolidated skills:

- `gsap` — core API + timelines + performance in SKILL.md; scrolltrigger,
  plugins, utils, react, frameworks, effects in references/
- `hyperframes` — composition authoring rules in SKILL.md; captions, tts,
  audio-reactive, marker-highlight in references/
- `hyperframes-cli` — CLI commands (init, lint, preview, render, etc.)

Why: With 15 separate skills, agents must correctly trigger the right subset
for any task. "Create an animated video with captions" needed 6+ skills to
fire — each with ~90% trigger accuracy means ~53% chance of getting all of
them. With 3 skills, that same task needs just `hyperframes` + `gsap` (~90%
both fire). Progressive disclosure still works via references/ files loaded
on demand.

Also fixes: CLAUDE.md referenced `window.__GSAP_TIMELINE` (incorrect) —
corrected to `window.__timelines`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): add --skip-skills flag to init command

Allow skipping the AI coding skills installation prompt during
`hyperframes init` with `--skip-skills`. Useful when skills are
already installed or when the user wants to scaffold without them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address code review feedback on consolidation

Restore content lost during over-compression:

- captions: fix overflow to `visible` (not hidden — clips glow effects),
  add container pattern warning, scale headroom formula, and self-lint
  placement guidance
- audio-reactive: restore sampling frequency pattern (per-frame tl.call
  loop vs single tween) and textShadow-on-container gotcha
- effects/typewriter: restore word rotation, appending words, spacing
  with static text, and multi-line cursor handoff patterns
- effects/audio-visualizer: restore spatial mapping conventions, fetch vs
  inline loading, WebGL/DOM rendering approaches, and canvas layering
- hyperframes-cli: restore --strict-all flag in render flags table

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update build:copy and template for consolidated skill names

- build:copy: reference skills/hyperframes, skills/hyperframes-cli,
  skills/gsap instead of the old 15 skill directory names
- _shared/CLAUDE.md template: update skill table to consolidated names

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 11:21:49 -07:00