mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-15 13:58:20 +00:00
refactor(skills): extract audio-reactive into standalone skill
Audio reactivity applies to any visual element in a composition, not just captions. Move it to its own skill referenced from compose-video. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
bbb6c48b7b
commit
afa5021b02
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: audio-reactive
|
||||
description: Drive any visual element in a HyperFrames composition from audio data — captions, backgrounds, shapes, overlays, anything GSAP can animate. Use when a composition should respond to music, voice, or sound.
|
||||
trigger: Use when a composition involves music, beat-synced animation, audio visualization, or any visual element that should react to sound.
|
||||
---
|
||||
|
||||
# Audio-Reactive Animation
|
||||
|
||||
When audio data is available (extracted via `extract-audio-data.py` or loaded from `audio-data.json`), any visual element in the composition can be driven by the music — captions, backgrounds, shapes, overlays, anything GSAP can animate.
|
||||
|
||||
## Audio Data Format
|
||||
|
||||
```js
|
||||
var AUDIO_DATA = {
|
||||
fps: 30, // frame rate of the analysis
|
||||
totalFrames: 900, // total analyzed frames
|
||||
frames: [
|
||||
{ bands: [0.82, 0.45, 0.31, ...] }, // per-frame frequency band amplitudes
|
||||
// ...
|
||||
]
|
||||
};
|
||||
```
|
||||
|
||||
- `frames[i].bands[]` — frequency band amplitudes, normalized 0–1. Index 0 = bass, higher indices = mids and treble.
|
||||
- `fps` — frame rate of the analysis (matches composition frame rate)
|
||||
- `totalFrames` — total number of analyzed frames
|
||||
|
||||
## Mapping Audio to Visuals
|
||||
|
||||
Map frequency bands and amplitude to any GSAP-animatable property. The creative choice is yours — these are common mappings:
|
||||
|
||||
| Audio signal | Visual property | Effect |
|
||||
| ---------------------- | --------------------------------- | -------------------------- |
|
||||
| Bass (bands[0]) | `scale` | Pulse on beat |
|
||||
| Treble (bands[12–14]) | `textShadow`, `boxShadow` | Glow intensity |
|
||||
| Overall amplitude | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift |
|
||||
| Beat onset | `scale`, `color`, `rotation` | Flash or pop on hits |
|
||||
| Mid-range (bands[4–8]) | `borderRadius`, `width`, `height` | Shape morphing |
|
||||
|
||||
These are starting points. Any property GSAP can tween is fair game — `clipPath`, `filter`, `backgroundPosition`, SVG attributes, custom CSS properties.
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Subtlety for text.** Captions and readable text should stay in the 3–6% scale variation range with soft glow. Heavy pulsing makes text unreadable.
|
||||
- **Go bigger on non-text elements.** Backgrounds, shapes, and decorative elements can handle 10–30% scale swings, full color shifts, and dramatic transforms.
|
||||
- **Match the energy.** A corporate explainer needs subtle reactivity. A music video can go hard.
|
||||
- **Deterministic.** Audio data is pre-extracted — no Web Audio API, no `AnalyserNode`, no runtime mic input. The data is static JSON, the animation is repeatable.
|
||||
|
||||
## Constraints
|
||||
|
||||
- All audio data must be pre-extracted — no runtime audio analysis
|
||||
- No `Math.random()` or `Date.now()` — deterministic rendering applies
|
||||
- Audio reactivity runs on the same GSAP timeline as everything else
|
||||
@@ -276,46 +276,13 @@ group.words.forEach(function (word, wi) {
|
||||
});
|
||||
```
|
||||
|
||||
## Audio-Reactive Captions
|
||||
|
||||
When audio data is available (extracted via `extract-audio-data.py`), tie caption properties to the music. Captions pulse, glow, and shift with the beat.
|
||||
|
||||
```js
|
||||
// Load audio data alongside the transcript
|
||||
var AUDIO_DATA = /* loaded from audio-data.json */;
|
||||
|
||||
for (var f = 0; f < AUDIO_DATA.totalFrames; f++) {
|
||||
tl.call(function (frameIdx) {
|
||||
return function () {
|
||||
var frame = AUDIO_DATA.frames[frameIdx];
|
||||
if (!frame) return;
|
||||
// Find the currently visible caption group
|
||||
var activeGroup = document.querySelector('.caption-group[data-active="true"]');
|
||||
if (!activeGroup) return;
|
||||
var words = activeGroup.querySelectorAll("span");
|
||||
|
||||
// Bass drives scale pulse on the whole group
|
||||
var bassPulse = 1 + frame.bands[0] * 0.06;
|
||||
gsap.set(activeGroup, { scale: bassPulse });
|
||||
|
||||
// Treble drives glow intensity
|
||||
var treble = Math.max(frame.bands[12] || 0, frame.bands[13] || 0, frame.bands[14] || 0);
|
||||
var glow = Math.round(treble * 15);
|
||||
gsap.set(activeGroup, { textShadow: "0 0 " + glow + "px rgba(255,255,255," + (treble * 0.6) + ")" });
|
||||
};
|
||||
}(f), [], f / AUDIO_DATA.fps);
|
||||
}
|
||||
```
|
||||
|
||||
Keep audio reactivity subtle for captions — 3-6% scale variation and soft glow. Heavy pulsing makes text unreadable. Audio reactivity works best as a background texture, not the main event.
|
||||
|
||||
## Combining Techniques
|
||||
|
||||
The best dynamic captions layer 2-3 techniques together. A few combinations that work:
|
||||
|
||||
| Combination | Energy | Best for |
|
||||
| ---------------------------------------- | ----------- | ------------------------------- |
|
||||
| Karaoke highlight + bass pulse | Medium-high | Music videos, lyric videos |
|
||||
| Karaoke highlight + audio reactivity | Medium-high | Music videos, lyric videos |
|
||||
| Staggered entrance + scatter exit | High | Hype content, trailers |
|
||||
| Clip-path reveal + fade exit | Medium | Corporate, storytelling |
|
||||
| Slam heroes + elastic others + drop exit | Very high | Product launches, announcements |
|
||||
|
||||
@@ -140,6 +140,7 @@ Video must be `muted playsinline`. Audio is always a separate `<audio>` element:
|
||||
For PiP, title cards, and slide show patterns, see [patterns.md](./patterns.md).
|
||||
For data, stats, and infographics, see [data-in-motion.md](./data-in-motion.md).
|
||||
For typewriter text and other GSAP animation effects, see the `gsap-effects` skill.
|
||||
For audio-driven animation (beat sync, glow, pulse), see the `audio-reactive` skill.
|
||||
|
||||
## Output Checklist
|
||||
|
||||
|
||||
Reference in New Issue
Block a user