From afa5021b02abaffd689a11b977cae655cb917b49 Mon Sep 17 00:00:00 2001 From: Vance Ingalls Date: Wed, 1 Apr 2026 10:28:45 -0700 Subject: [PATCH] refactor(skills): extract audio-reactive into standalone skill Audio reactivity applies to any visual element in a composition, not just captions. Move it to its own skill referenced from compose-video. Co-Authored-By: Claude Opus 4.6 (1M context) --- skills/audio-reactive/SKILL.md | 53 +++++++++++++++++++ .../dynamic-techniques.md | 35 +----------- skills/hyperframes-compose/SKILL.md | 1 + 3 files changed, 55 insertions(+), 34 deletions(-) create mode 100644 skills/audio-reactive/SKILL.md diff --git a/skills/audio-reactive/SKILL.md b/skills/audio-reactive/SKILL.md new file mode 100644 index 0000000000..bc70478abc --- /dev/null +++ b/skills/audio-reactive/SKILL.md @@ -0,0 +1,53 @@ +--- +name: audio-reactive +description: Drive any visual element in a HyperFrames composition from audio data — captions, backgrounds, shapes, overlays, anything GSAP can animate. Use when a composition should respond to music, voice, or sound. +trigger: Use when a composition involves music, beat-synced animation, audio visualization, or any visual element that should react to sound. +--- + +# Audio-Reactive Animation + +When audio data is available (extracted via `extract-audio-data.py` or loaded from `audio-data.json`), any visual element in the composition can be driven by the music — captions, backgrounds, shapes, overlays, anything GSAP can animate. + +## Audio Data Format + +```js +var AUDIO_DATA = { + fps: 30, // frame rate of the analysis + totalFrames: 900, // total analyzed frames + frames: [ + { bands: [0.82, 0.45, 0.31, ...] }, // per-frame frequency band amplitudes + // ... + ] +}; +``` + +- `frames[i].bands[]` — frequency band amplitudes, normalized 0–1. Index 0 = bass, higher indices = mids and treble. +- `fps` — frame rate of the analysis (matches composition frame rate) +- `totalFrames` — total number of analyzed frames + +## Mapping Audio to Visuals + +Map frequency bands and amplitude to any GSAP-animatable property. The creative choice is yours — these are common mappings: + +| Audio signal | Visual property | Effect | +| ---------------------- | --------------------------------- | -------------------------- | +| Bass (bands[0]) | `scale` | Pulse on beat | +| Treble (bands[12–14]) | `textShadow`, `boxShadow` | Glow intensity | +| Overall amplitude | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift | +| Beat onset | `scale`, `color`, `rotation` | Flash or pop on hits | +| Mid-range (bands[4–8]) | `borderRadius`, `width`, `height` | Shape morphing | + +These are starting points. Any property GSAP can tween is fair game — `clipPath`, `filter`, `backgroundPosition`, SVG attributes, custom CSS properties. + +## Guidelines + +- **Subtlety for text.** Captions and readable text should stay in the 3–6% scale variation range with soft glow. Heavy pulsing makes text unreadable. +- **Go bigger on non-text elements.** Backgrounds, shapes, and decorative elements can handle 10–30% scale swings, full color shifts, and dramatic transforms. +- **Match the energy.** A corporate explainer needs subtle reactivity. A music video can go hard. +- **Deterministic.** Audio data is pre-extracted — no Web Audio API, no `AnalyserNode`, no runtime mic input. The data is static JSON, the animation is repeatable. + +## Constraints + +- All audio data must be pre-extracted — no runtime audio analysis +- No `Math.random()` or `Date.now()` — deterministic rendering applies +- Audio reactivity runs on the same GSAP timeline as everything else diff --git a/skills/hyperframes-captions/dynamic-techniques.md b/skills/hyperframes-captions/dynamic-techniques.md index 5c5907f597..9c78a6cfd6 100644 --- a/skills/hyperframes-captions/dynamic-techniques.md +++ b/skills/hyperframes-captions/dynamic-techniques.md @@ -276,46 +276,13 @@ group.words.forEach(function (word, wi) { }); ``` -## Audio-Reactive Captions - -When audio data is available (extracted via `extract-audio-data.py`), tie caption properties to the music. Captions pulse, glow, and shift with the beat. - -```js -// Load audio data alongside the transcript -var AUDIO_DATA = /* loaded from audio-data.json */; - -for (var f = 0; f < AUDIO_DATA.totalFrames; f++) { - tl.call(function (frameIdx) { - return function () { - var frame = AUDIO_DATA.frames[frameIdx]; - if (!frame) return; - // Find the currently visible caption group - var activeGroup = document.querySelector('.caption-group[data-active="true"]'); - if (!activeGroup) return; - var words = activeGroup.querySelectorAll("span"); - - // Bass drives scale pulse on the whole group - var bassPulse = 1 + frame.bands[0] * 0.06; - gsap.set(activeGroup, { scale: bassPulse }); - - // Treble drives glow intensity - var treble = Math.max(frame.bands[12] || 0, frame.bands[13] || 0, frame.bands[14] || 0); - var glow = Math.round(treble * 15); - gsap.set(activeGroup, { textShadow: "0 0 " + glow + "px rgba(255,255,255," + (treble * 0.6) + ")" }); - }; - }(f), [], f / AUDIO_DATA.fps); -} -``` - -Keep audio reactivity subtle for captions — 3-6% scale variation and soft glow. Heavy pulsing makes text unreadable. Audio reactivity works best as a background texture, not the main event. - ## Combining Techniques The best dynamic captions layer 2-3 techniques together. A few combinations that work: | Combination | Energy | Best for | | ---------------------------------------- | ----------- | ------------------------------- | -| Karaoke highlight + bass pulse | Medium-high | Music videos, lyric videos | +| Karaoke highlight + audio reactivity | Medium-high | Music videos, lyric videos | | Staggered entrance + scatter exit | High | Hype content, trailers | | Clip-path reveal + fade exit | Medium | Corporate, storytelling | | Slam heroes + elastic others + drop exit | Very high | Product launches, announcements | diff --git a/skills/hyperframes-compose/SKILL.md b/skills/hyperframes-compose/SKILL.md index 60eafb3f63..ac2a3263ae 100644 --- a/skills/hyperframes-compose/SKILL.md +++ b/skills/hyperframes-compose/SKILL.md @@ -140,6 +140,7 @@ Video must be `muted playsinline`. Audio is always a separate `