mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-09 20:07:39 +00:00
James RussoandClaude Opus 4.6
0a0d5d3654
* refactor(skills): consolidate 15 skills into 3 for better trigger reliability Merge 9 GSAP skills (core, timeline, scrolltrigger, plugins, utils, react, frameworks, performance, effects) and 6 HyperFrames skills (compose, captions, tts, audio-reactive, marker-highlight, cli) into 3 consolidated skills: - `gsap` — core API + timelines + performance in SKILL.md; scrolltrigger, plugins, utils, react, frameworks, effects in references/ - `hyperframes` — composition authoring rules in SKILL.md; captions, tts, audio-reactive, marker-highlight in references/ - `hyperframes-cli` — CLI commands (init, lint, preview, render, etc.) Why: With 15 separate skills, agents must correctly trigger the right subset for any task. "Create an animated video with captions" needed 6+ skills to fire — each with ~90% trigger accuracy means ~53% chance of getting all of them. With 3 skills, that same task needs just `hyperframes` + `gsap` (~90% both fire). Progressive disclosure still works via references/ files loaded on demand. Also fixes: CLAUDE.md referenced `window.__GSAP_TIMELINE` (incorrect) — corrected to `window.__timelines`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): add --skip-skills flag to init command Allow skipping the AI coding skills installation prompt during `hyperframes init` with `--skip-skills`. Useful when skills are already installed or when the user wants to scaffold without them. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(skills): address code review feedback on consolidation Restore content lost during over-compression: - captions: fix overflow to `visible` (not hidden — clips glow effects), add container pattern warning, scale headroom formula, and self-lint placement guidance - audio-reactive: restore sampling frequency pattern (per-frame tl.call loop vs single tween) and textShadow-on-container gotcha - effects/typewriter: restore word rotation, appending words, spacing with static text, and multi-line cursor handoff patterns - effects/audio-visualizer: restore spatial mapping conventions, fetch vs inline loading, WebGL/DOM rendering approaches, and canvas layering - hyperframes-cli: restore --strict-all flag in render flags table Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): update build:copy and template for consolidated skill names - build:copy: reference skills/hyperframes, skills/hyperframes-cli, skills/gsap instead of the old 15 skill directory names - _shared/CLAUDE.md template: update skill table to consolidated names Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
77 lines
3.0 KiB
Markdown
77 lines
3.0 KiB
Markdown
# Audio-Reactive Animation
|
|
|
|
Drive visuals from music, voice, or sound. Any GSAP-animatable property can respond to pre-extracted audio data.
|
|
|
|
## Audio Data Format
|
|
|
|
```js
|
|
var AUDIO_DATA = {
|
|
fps: 30,
|
|
totalFrames: 900,
|
|
frames: [{ bands: [0.82, 0.45, 0.31, ...] }, ...]
|
|
};
|
|
```
|
|
|
|
- `frames[i].bands[]` — frequency band amplitudes, 0-1. Index 0 = bass, higher = treble.
|
|
- Each band normalized independently across the full track.
|
|
|
|
## Mapping Audio to Visuals
|
|
|
|
| Audio signal | Visual property | Effect |
|
|
| ---------------------- | --------------------------------- | -------------------------- |
|
|
| Bass (bands[0]) | `scale` | Pulse on beat |
|
|
| Treble (bands[12-14]) | `textShadow`, `boxShadow` | Glow intensity |
|
|
| Overall amplitude | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift |
|
|
| Mid-range (bands[4-8]) | `borderRadius`, `width` | Shape morphing |
|
|
|
|
Any GSAP-tweenable property works — `clipPath`, `filter`, SVG attributes, CSS custom properties.
|
|
|
|
## Content, Not Medium
|
|
|
|
Audio provides **timing and intensity**. The visual vocabulary comes from the narrative.
|
|
|
|
**Never add:** equalizer bars, spectrum analyzers, waveform displays, musical notes clip art, generic particle systems, rainbow color cycling, strobing white on beats, abstract pulsing orbs.
|
|
|
|
**Instead:** Let content guide the visual and audio drive its behavior. Bass makes warmth _swell_. Treble sharpens _contrast_. The visual choice comes from "what does this piece feel like?"
|
|
|
|
## Sampling Pattern
|
|
|
|
Audio reactivity requires per-frame sampling via a `for` loop with `tl.call()`, not a single tween:
|
|
|
|
```js
|
|
// ✅ Correct — sample every frame
|
|
for (var f = 0; f < AUDIO_DATA.totalFrames; f++) {
|
|
tl.call(
|
|
(function (frame) {
|
|
return function () {
|
|
draw(frame);
|
|
};
|
|
})(AUDIO_DATA.frames[f]),
|
|
[],
|
|
f / AUDIO_DATA.fps,
|
|
);
|
|
}
|
|
|
|
// ❌ Wrong — single tween, doesn't react to audio
|
|
gsap.to(".el", { scale: 1.2, duration: totalDuration });
|
|
```
|
|
|
|
Without per-frame sampling, the composition doesn't actually react to audio.
|
|
|
|
## textShadow Gotcha
|
|
|
|
`textShadow` on a parent container with semi-transparent children (e.g., inactive caption words at `rgba(255,255,255,0.3)`) renders a visible glow rectangle behind all children. Fix: apply `scale` to the container for beat pulse, but apply `textShadow` to individual active words only.
|
|
|
|
## Guidelines
|
|
|
|
- **Subtlety for text** — 3-6% scale variation, soft glow. Heavy pulsing makes text unreadable.
|
|
- **Go bigger on non-text** — backgrounds and shapes can handle 10-30% swings.
|
|
- **Match the energy** — corporate = subtle; music video = dramatic.
|
|
- **Deterministic** — pre-extracted data, no Web Audio API, no runtime analysis.
|
|
|
|
## Constraints
|
|
|
|
- All audio data must be pre-extracted (use `extract-audio-data.py` from the gsap skill's scripts/)
|
|
- No `Math.random()` or `Date.now()`
|
|
- Audio reactivity runs on the same GSAP timeline as everything else
|