diff --git a/CLAUDE.md b/CLAUDE.md index 6243bab06..96dd07605 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,40 +4,22 @@ This repo ships skills that are installed globally via `npx hyperframes skills` (runs automatically during `hyperframes init`). **Always use the appropriate skill instead of writing code from scratch or fetching external docs.** -### HyperFrames Skills (from this repo) +### Skills -| Skill | Invoke with | When to use | -| ------------------------ | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **hyperframes-compose** | `/hyperframes-compose` | Creating ANY HTML composition — videos, animations, title cards, overlays. Contains required HTML structure, `class="clip"` rules, GSAP timeline patterns, and rendering constraints. | -| **hyperframes-captions** | `/hyperframes-captions` | Any task involving text synced to audio: captions, subtitles, lyrics, lyric videos, karaoke. Also covers transcription strategy (whisper model selection, transcript format). | -| **hyperframes-tts** | `/hyperframes-tts` | Generating speech from text: narration, voiceovers, text-to-speech. Voice selection, speed control, and combining TTS output with compositions. | -| **marker-highlight** | `/marker-highlight` | Animated text highlighting — marker sweeps, hand-drawn circles, burst lines, scribble, sketchout. Use with captions for dynamic emphasis. | - -### GSAP Skills (from [greensock/gsap-skills](https://github.com/greensock/gsap-skills)) - -| Skill | Invoke with | When to use | -| ---------------------- | --------------------- | -------------------------------------------------------------------------------- | -| **gsap-core** | `/gsap-core` | `gsap.to()`, `from()`, `fromTo()`, easing, duration, stagger, defaults | -| **gsap-timeline** | `/gsap-timeline` | Timeline sequencing, position parameter, labels, nesting, playback | -| **gsap-performance** | `/gsap-performance` | Performance best practices — transforms over layout props, will-change, batching | -| **gsap-plugins** | `/gsap-plugins` | ScrollTrigger, Flip, Draggable, SplitText, and other GSAP plugins | -| **gsap-scrolltrigger** | `/gsap-scrolltrigger` | Scroll-linked animations, pinning, scrub, triggers | -| **gsap-utils** | `/gsap-utils` | `gsap.utils` helpers — clamp, mapRange, snap, toArray, wrap, pipe | +| Skill | Invoke with | When to use | +| ------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions/subtitles, TTS narration, audio-reactive animation, marker highlights. Composition authoring rules. | +| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts, doctor. Use when scaffolding, validating, previewing, or rendering. | +| **gsap** | `/gsap` | GSAP animations — tweens, timelines, easing, ScrollTrigger, plugins (Flip, Draggable, SplitText, etc.), React/Vue/Svelte, performance optimization. | ### Why this matters -The skills encode HyperFrames-specific patterns (e.g., required `class="clip"` on all timed elements, GSAP timeline registration via `window.__GSAP_TIMELINE`, `data-*` attribute semantics) that are NOT in generic web docs. Skipping the skills and writing from scratch will produce broken compositions. +The skills encode HyperFrames-specific patterns (e.g., required `class="clip"` on all timed elements, GSAP timeline registration via `window.__timelines`, `data-*` attribute semantics) that are NOT in generic web docs. Skipping the skills and writing from scratch will produce broken compositions. ### Rules -- When creating or modifying HTML compositions → invoke `/hyperframes-compose` BEFORE writing any code -- When adding captions, subtitles, lyrics, or any text synced to audio → invoke `/hyperframes-captions` BEFORE writing any code -- When transcribing audio or choosing a whisper model → invoke `/hyperframes-captions` BEFORE running any transcription tool -- When generating speech from text (narration, voiceover, TTS) → invoke `/hyperframes-tts` BEFORE running any TTS command -- When creating a video from audio (music video, lyric video, audio visualizer with text) → invoke BOTH `/hyperframes-compose` AND `/hyperframes-captions` -- When writing GSAP animations → invoke `/gsap-core` and `/gsap-timeline` BEFORE writing any code -- When optimizing animation performance → invoke `/gsap-performance` BEFORE making changes -- When adding animated text emphasis (highlight sweeps, circles, bursts, scribbles) → invoke `/marker-highlight` BEFORE writing any code +- When creating or modifying HTML compositions, captions, TTS, audio-reactive, or marker highlights → invoke `/hyperframes` BEFORE writing any code +- When writing GSAP animations (tweens, timelines, ScrollTrigger, plugins) → invoke `/gsap` BEFORE writing any code - After creating or editing any `.html` composition → run `npx hyperframes lint` and `npx hyperframes validate` in parallel, fix all errors before opening the studio or considering the task complete. `lint` checks the HTML structure statically; `validate` loads the composition in headless Chrome and catches runtime JS errors, missing assets, and failed network requests. Always validate before `npx hyperframes preview`. ### Installing skills @@ -143,7 +125,7 @@ If captions are inaccurate (wrong words, bad timing): 2. **Set language**: `--language en` to filter non-target speech 3. **Use an external API**: Transcribe via OpenAI or Groq Whisper API, then import the JSON with `hyperframes transcribe response.json` -See the `/hyperframes-captions` skill for full details on model selection and API usage. +See the `/hyperframes` skill (references/captions.md and references/transcript-guide.md) for full details on model selection and API usage. ## Text-to-Speech diff --git a/packages/cli/package.json b/packages/cli/package.json index 3ee9a0c07..4f2146f63 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -21,7 +21,7 @@ "build:fonts": "cd ../producer && tsx scripts/generate-font-data.ts", "build:studio": "cd ../studio && bun run build", "build:runtime": "tsx scripts/build-runtime.ts", - "build:copy": "mkdir -p dist/studio dist/docs dist/templates dist/skills && cp -r ../studio/dist/* dist/studio/ && cp -r src/templates/blank src/templates/_shared dist/templates/ && cp -r ../../skills/hyperframes-compose ../../skills/hyperframes-captions ../../skills/hyperframes-tts dist/skills/ && (cp src/docs/*.md dist/docs/ 2>/dev/null || true)", + "build:copy": "mkdir -p dist/studio dist/docs dist/templates dist/skills && cp -r ../studio/dist/* dist/studio/ && cp -r src/templates/blank src/templates/_shared dist/templates/ && cp -r ../../skills/hyperframes ../../skills/hyperframes-cli ../../skills/gsap dist/skills/ && (cp src/docs/*.md dist/docs/ 2>/dev/null || true)", "typecheck": "tsc --noEmit" }, "dependencies": { diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts index ccf5b0677..89002311d 100644 --- a/packages/cli/src/commands/init.ts +++ b/packages/cli/src/commands/init.ts @@ -7,6 +7,7 @@ export const examples: Example[] = [ ["Start from an existing video file", "hyperframes init my-video --video clip.mp4"], ["Start from an audio file", "hyperframes init my-video --audio track.mp3"], ["Non-interactive mode (for CI or AI agents)", "hyperframes init my-video --non-interactive"], + ["Skip AI coding skills installation", "hyperframes init my-video --skip-skills"], ]; import { existsSync, @@ -390,12 +391,17 @@ export default defineCommand({ type: "boolean", description: "Disable interactive prompts (for CI/agents)", }, + "skip-skills": { + type: "boolean", + description: "Skip AI coding skills installation", + }, }, async run({ args }) { const templateFlag = args.template; const videoFlag = args.video; const audioFlag = args.audio; const skipTranscribe = args["skip-transcribe"] === true; + const skipSkills = args["skip-skills"] === true; const nonInteractive = args["non-interactive"] === true; const modelFlag = args.model; const languageFlag = args.language; @@ -693,17 +699,19 @@ export default defineCommand({ clack.note(files.map((f) => c.accent(f)).join("\n"), c.success(`Created ${name}/`)); // Offer to install AI coding skills - const installSkills = await clack.confirm({ - message: "Install AI coding skills? (for Claude Code, Cursor, Codex, etc.)", - initialValue: true, - }); - if (clack.isCancel(installSkills)) { - clack.cancel("Setup cancelled."); - process.exit(0); - } - if (installSkills) { - const skillsCmd = await import("./skills.js").then((m) => m.default); - await runCommand(skillsCmd, { rawArgs: [] }); + if (!skipSkills) { + const installSkills = await clack.confirm({ + message: "Install AI coding skills? (for Claude Code, Cursor, Codex, etc.)", + initialValue: true, + }); + if (clack.isCancel(installSkills)) { + clack.cancel("Setup cancelled."); + process.exit(0); + } + if (installSkills) { + const skillsCmd = await import("./skills.js").then((m) => m.default); + await runCommand(skillsCmd, { rawArgs: [] }); + } } // Auto-launch studio preview diff --git a/packages/cli/src/templates/_shared/CLAUDE.md b/packages/cli/src/templates/_shared/CLAUDE.md index 0e74629f2..7a65b2955 100644 --- a/packages/cli/src/templates/_shared/CLAUDE.md +++ b/packages/cli/src/templates/_shared/CLAUDE.md @@ -4,13 +4,11 @@ **Always invoke the relevant skill before writing or modifying compositions.** Skills encode framework-specific patterns (e.g., `class="clip"`, `window.__timelines`, `data-*` attributes) that are NOT in generic web docs. Skipping them produces broken compositions. -| Skill | Command | When to use | -| ------------------------ | ----------------------- | ------------------------------------------------------------------------------------------------------ | -| **hyperframes-compose** | `/hyperframes-compose` | Creating or editing ANY HTML composition — videos, animations, title cards, overlays, sub-compositions | -| **hyperframes-captions** | `/hyperframes-captions` | Any text synced to audio: captions, subtitles, lyrics, karaoke. Also covers transcription strategy. | -| **gsap-core** | `/gsap-core` | GSAP tweens: `gsap.to()`, `from()`, `fromTo()`, easing, stagger, defaults | -| **gsap-timeline** | `/gsap-timeline` | Timeline sequencing, position parameter, labels, nesting | -| **gsap-performance** | `/gsap-performance` | Animation performance — transforms over layout props, will-change, batching | +| Skill | Command | When to use | +| ------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------- | +| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions, TTS, audio-reactive animation, marker highlights | +| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts | +| **gsap** | `/gsap` | GSAP animations — tweens, timelines, easing, ScrollTrigger, plugins, React/Vue/Svelte, performance optimization | > **Skills not available?** Ask the user to run `npx hyperframes skills` and restart their > agent session, or install manually: `npx skills add heygen-com/hyperframes` and diff --git a/skills/audio-reactive/SKILL.md b/skills/audio-reactive/SKILL.md deleted file mode 100644 index 8eaee925c..000000000 --- a/skills/audio-reactive/SKILL.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -name: audio-reactive -description: Audio-reactive animation — drive visuals from music, voice, or sound in HyperFrames. Maps frequency bands and amplitude to any GSAP-animatable property. -trigger: Use when a composition involves music, beat-synced animation, audio visualization, or any visual reacting to sound. ---- - -# Audio-Reactive Animation - -When audio data is available (extracted via `extract-audio-data.py` or loaded from `audio-data.json`), any visual element in the composition can be driven by the music — captions, backgrounds, shapes, overlays, anything GSAP can animate. - -## Audio Data Format - -```js -var AUDIO_DATA = { - fps: 30, // frame rate of the analysis - totalFrames: 900, // total analyzed frames - frames: [ - { bands: [0.82, 0.45, 0.31, ...] }, // per-frame frequency band amplitudes - // ... - ] -}; -``` - -- `frames[i].bands[]` — frequency band amplitudes, normalized 0–1. Index 0 = bass, higher indices = mids and treble. -- `fps` — frame rate of the analysis (matches composition frame rate) -- `totalFrames` — total number of analyzed frames - -## Mapping Audio to Visuals - -Map frequency bands and amplitude to any GSAP-animatable property. The creative choice is yours — these are common mappings: - -| Audio signal | Visual property | Effect | -| ---------------------- | --------------------------------- | -------------------------- | -| Bass (bands[0]) | `scale` | Pulse on beat | -| Treble (bands[12–14]) | `textShadow`, `boxShadow` | Glow intensity | -| Overall amplitude | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift | -| Beat onset | `scale`, `color`, `rotation` | Flash or pop on hits | -| Mid-range (bands[4–8]) | `borderRadius`, `width`, `height` | Shape morphing | - -These are starting points. Any property GSAP can tween is fair game — `clipPath`, `filter`, `backgroundPosition`, SVG attributes, custom CSS properties. - -## Content, Not Medium - -Audio data provides **timing and intensity** for visuals grounded in the content. It tells the animation _when_ and _how much_ — not _what to show_. The visual vocabulary comes from the narrative, theme, and emotion of the piece. A funeral dirge and a party anthem should produce completely different visuals even though their audio data has the same shape. - -**Never add these — they represent the medium, not the content:** - -- Frequency equalizer bars, spectrum analyzers, radial spectrum rings — technical readouts that just say "audio exists" -- Waveform displays, oscilloscope lines, VU meters — diagnostic tools, not creative choices -- Musical notes, vinyl records, turntable imagery — clip art signaling "music is playing" -- Generic particle systems driven by amplitude — interchangeable across any song or mood -- Background color cycling through rainbow hues on frequency — rave aesthetic regardless of content -- Strobing/flashing white on beat hits — lazy beat sync (also an accessibility problem) -- Abstract pulsing orbs, breathing geometric wireframes, concentric rings — would look identical on a lullaby or a metal track - -**Instead, let the content guide the visual and the audio drive its behavior:** - -- If the scene is warm, bass makes the warmth _swell_ (slight scale, deeper color saturation) -- If the mood is tense, treble sharpens _contrast_ or tightens _letterSpacing_ -- If text is the focus, bass gives it subtle _weight_ (shadow depth, y offset) while treble adds _shimmer_ (glow, lightness) -- The visual choice comes from asking "what does this piece feel like?" — audio data just animates the answer - -## Guidelines - -- **Subtlety for text.** Captions and readable text should stay in the 3–6% scale variation range with soft glow. Heavy pulsing makes text unreadable. -- **Go bigger on non-text elements.** Backgrounds, shapes, and decorative elements can handle 10–30% scale swings, full color shifts, and dramatic transforms. -- **Match the energy.** A corporate explainer needs subtle reactivity. A music video can go hard. -- **Deterministic.** Audio data is pre-extracted — no Web Audio API, no `AnalyserNode`, no runtime mic input. The data is static JSON, the animation is repeatable. - -## Sampling Frequency - -**One tween per element is not audio reactivity.** Reading peak bass/treble for a time range and creating a single tween at the start sets one static value — the viewer cannot perceive it as reactive. Audio reactivity requires multiple tweens over time so the visual _changes_ with the music. - -Sample at **100-200ms intervals** throughout the element's visible lifetime: - -```js -// ✓ Perceptible: sample every 150ms, create a tween at each point -for (var t = group.start; t < group.end; t += 0.15) { - var bass = getBass(t); - tl.to(el, { scale: 1 + bass * 0.06, duration: 0.075, ease: "sine.inOut" }, t); -} - -// ✗ Imperceptible: one tween from peak values -var peakBass = getPeakBass(group.start, group.end); -tl.to(el, { scale: 1 + peakBass * 0.06, duration: 0.3 }, group.start); -``` - -## textShadow on Containers - -**Never apply `textShadow` to a container that has semi-transparent children.** When a caption group has inactive words at `rgba(255,255,255,0.3)`, a `textShadow` on the parent div renders a visible glow rectangle behind all children — it looks like a gray background, not a glow effect. - -Apply `scale` to the group container for bass-reactive pulsing. Apply `textShadow` to individual active word elements only. - -## Constraints - -- All audio data must be pre-extracted — no runtime audio analysis -- No `Math.random()` or `Date.now()` — deterministic rendering applies -- Audio reactivity runs on the same GSAP timeline as everything else diff --git a/skills/gsap-effects/SKILL.md b/skills/gsap-effects/SKILL.md deleted file mode 100644 index 17b112eae..000000000 --- a/skills/gsap-effects/SKILL.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -name: gsap-effects -description: Typewriter text, audio visualizer, and drop-in animation effects for HyperFrames compositions. Use for character-by-character reveals, spectrum bars, waveforms, or audio-reactive visuals. ---- - -# GSAP Effects - -Drop-in animation patterns for HyperFrames compositions. Each effect is a self-contained reference with the HTML, CSS, and code needed to add it to a composition. - -These effects follow all HyperFrames composition rules — deterministic, no randomness, timelines registered via `window.__timelines`. - -## Available Effects - -| Effect | File | Use when | -| ---------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -| Typewriter | [typewriter.md](./typewriter.md) | Text should appear character by character, with or without a blinking cursor | -| Audio Visualizer | [audio-visualizer.md](./audio-visualizer.md) | Reactive bars, waveforms, circles, or glow that respond to audio. Includes extraction script and Canvas 2D patterns | diff --git a/skills/gsap-effects/audio-visualizer.md b/skills/gsap-effects/audio-visualizer.md deleted file mode 100644 index c9edb2592..000000000 --- a/skills/gsap-effects/audio-visualizer.md +++ /dev/null @@ -1,238 +0,0 @@ -# Audio Visualizer - -Reactive audio visualizations for HyperFrames compositions. Pre-extracts amplitude and frequency data from an audio file, then drives rendering from the GSAP timeline. - -## Why Pre-Extraction - -HyperFrames renders frame-by-frame in headless Chrome — there's no audio playing during rendering, so the Web Audio API's real-time `AnalyserNode` won't work. Instead, extract all audio data before the composition runs and bake it as a static JSON array. The composition reads the array by frame index. This is fully deterministic and seekable. - -## Step 1: Extract Audio Data - -```bash -python skills/gsap-effects/scripts/extract-audio-data.py audio.mp3 -o audio-data.json -python skills/gsap-effects/scripts/extract-audio-data.py video.mp4 --fps 30 --bands 16 -o audio-data.json -``` - -Requires ffmpeg and numpy (`pip install numpy`). - -| Flag | Default | Description | -| --------- | --------------- | -------------------------------------------------------- | -| `--fps` | 30 | Must match the composition/render FPS | -| `--bands` | 16 | Number of frequency bands (more = finer spectrum detail) | -| `-o` | audio-data.json | Output path | - -The script uses a 4096-sample FFT window (not the per-frame sample count) to ensure each frequency band maps to distinct FFT bins. Bands are logarithmically spaced from 30Hz to 16kHz — the useful range for music. Each band is normalized independently across the full track so treble activity is visible even when bass is louder in absolute terms. - -## Step 2: Understanding the Data - -```json -{ - "duration": 180.5, - "fps": 30, - "bands": 16, - "totalFrames": 5415, - "frames": [ - { "time": 0.0, "rms": 0.0, "bands": [0.0, 0.0, 0.0, ...] }, - { "time": 0.0333, "rms": 0.42, "bands": [0.8, 0.6, 0.3, ...] } - ] -} -``` - -**`rms`** (0-1) — overall loudness of this frame, normalized across the full track. 0 is silence, 1 is the loudest moment in the entire audio. Use this for anything that should respond to overall energy: scaling, pulsing, glow intensity, opacity, movement speed. - -**`bands`** (array of 0-1 values) — frequency magnitudes. Each value is normalized independently for that band across the full track, so a 0.8 in treble means "this is 80% of the loudest this treble band gets anywhere in the audio" — not that treble is as loud as bass in absolute terms. This is what makes all frequency ranges visually active. - -- Index 0 = lowest bass (~30Hz). Index `n-1` = highest treble (~16kHz). -- Low indices (0-3) react to kick drums, bass lines, sub-bass rumble. -- Mid indices (4-9) react to vocals, guitars, synths, most melodic content. -- High indices (10-15) react to hi-hats, cymbals, sibilance, brightness. - -## Loading the Data - -Embed the data in the composition so it's available when the timeline runs. - -```js -// Option A: inline (small files, under ~500KB) -const AUDIO_DATA = { - /* paste audio-data.json contents */ -}; -setupTimeline(AUDIO_DATA); - -// Option B: fetch (large files) -fetch("audio-data.json") - .then((r) => r.json()) - .then((data) => { - setupTimeline(data); - }); - -function setupTimeline(AUDIO_DATA) { - // Register tl.call() draws here — AUDIO_DATA is guaranteed to be loaded - for (let f = 0; f < AUDIO_DATA.totalFrames; f++) { - tl.call( - () => { - draw(AUDIO_DATA.frames[f]); - }, - [], - f / AUDIO_DATA.fps, - ); - } -} -``` - -With fetch, wrap all timeline setup inside the callback so `AUDIO_DATA` is available when the `for` loop reads `totalFrames`. The fetch completes before the renderer's first seek because it waits for `window.__hf` readiness. - -## Step 3: Drive Rendering from the Timeline - -Register a `tl.call()` at every frame interval. Each call reads the pre-computed data and renders. This is deterministic and seekable — scrubbing in the studio works because each frame's draw is tied to a specific timeline position. - -## Rendering Approaches - -The data is framework-agnostic. Here's how to wire it up in each approach. - -### Canvas 2D - -Best for: bars, waveforms, circles, gradients, particles. Most common choice. - -```js -const canvas = document.querySelector("#viz-canvas"); -const ctx = canvas.getContext("2d"); - -for (let f = 0; f < AUDIO_DATA.totalFrames; f++) { - tl.call( - () => { - const frame = AUDIO_DATA.frames[f]; - if (!frame) return; - ctx.clearRect(0, 0, canvas.width, canvas.height); - // read frame.rms and frame.bands, draw whatever you want - }, - [], - f / AUDIO_DATA.fps, - ); -} -``` - -### WebGL / Three.js - -HyperFrames has a Three.js adapter that patches `THREE.Clock` for deterministic time. Create your scene normally, then update uniforms or object properties from the audio data each frame. - -```js -// In your Three.js setup: -const uniforms = { uBass: { value: 0 }, uMid: { value: 0 }, uRms: { value: 0 } }; - -for (let f = 0; f < AUDIO_DATA.totalFrames; f++) { - tl.call( - () => { - const frame = AUDIO_DATA.frames[f]; - if (!frame) return; - uniforms.uBass.value = Math.max(frame.bands[0], frame.bands[1], frame.bands[2]); - uniforms.uMid.value = Math.max(frame.bands[6], frame.bands[7], frame.bands[8]); - uniforms.uRms.value = frame.rms; - }, - [], - f / AUDIO_DATA.fps, - ); -} -``` - -### DOM Elements - -For simpler visualizations (a few bars, a pulsing element), you can animate DOM elements directly. Less performant than Canvas for many elements, but fine for under ~20. - -```js -const bars = document.querySelectorAll(".bar"); -for (let f = 0; f < AUDIO_DATA.totalFrames; f++) { - tl.call( - () => { - const frame = AUDIO_DATA.frames[f]; - if (!frame) return; - bars.forEach((bar, i) => { - bar.style.height = frame.bands[i] * 100 + "%"; - }); - }, - [], - f / AUDIO_DATA.fps, - ); -} -``` - -## Spatial Mapping - -When laying out frequency data spatially, follow these conventions so visualizations read naturally: - -- **Horizontal layouts**: low frequencies (bass) on the left, high frequencies (treble) on the right. Iterate the bands array left-to-right. -- **Vertical layouts**: low frequencies at the bottom, high frequencies at the top. -- **Circular layouts**: bass starts at the top (12 o'clock) and wraps clockwise. Mirror the bands array for a full circle. - -## Motion Principles - -### Smoothing - -Raw per-frame data changes abruptly. Blend with the previous frame for fluid motion: - -```js -let prev = null; -const smoothing = 0.25; // 0 = no smoothing, higher = more lag - -function smooth(f) { - const raw = AUDIO_DATA.frames[f]; - if (!raw) return prev; - if (!prev) { - prev = { rms: raw.rms, bands: [...raw.bands] }; - return prev; - } - prev = { - rms: prev.rms * smoothing + raw.rms * (1 - smoothing), - bands: raw.bands.map((b, i) => prev.bands[i] * smoothing + b * (1 - smoothing)), - }; - return prev; -} -``` - -Lower smoothing (0.1-0.2) feels snappy and responsive — good for percussive music. Higher smoothing (0.3-0.5) feels languid and flowing — good for ambient or orchestral. - -### Value Mapping - -Audio data is 0-1 but visual properties need different ranges. Map with intention: - -- **Scale/size**: multiply by a max value. A bar's height = `bands[i] * maxHeight`. Don't let elements disappear at 0 — add a minimum: `minHeight + bands[i] * (maxHeight - minHeight)`. -- **Opacity**: low values should still be slightly visible. `0.15 + bands[i] * 0.85` keeps elements present during quiet moments. -- **Color intensity**: shift between a muted base and a vivid peak. Interpolate HSL lightness or RGB channels based on the value. -- **Position/offset**: use rms to drive drift or wobble. Small movements (5-20px) feel organic; large movements look chaotic. - -### What Makes It Feel Good - -- **Bass drives the big moves.** Scale, position shifts, and glow should react to low bands. Bass is what makes a visualization feel like it's "hitting." -- **Treble drives the detail.** Small particle movements, edge shimmer, opacity flicker. Treble adds texture without dominating. -- **RMS drives global properties.** Background brightness, overall scale, color warmth. It's the "energy level" of the whole frame. -- **Don't animate everything at once.** Pick 2-3 visual properties to tie to the audio. More than that looks noisy. -- **Quiet sections should still have life.** A completely static frame during a soft passage looks broken. Keep minimum values above zero. - -## Band Count Guide - -| Bands | Detail level | Good for | -| ----- | ------------ | ------------------------------------------- | -| 4 | Low | Simple pulsing, background glow | -| 8 | Medium | Bar visualizations, basic spectrum | -| 16 | High | Detailed EQ, circular visualizers (default) | -| 32 | Very high | Smooth curves, dense radial layouts | - -More bands = larger JSON file. 16 is a good default. - -## Layering - -Layer multiple canvases with CSS z-index for depth: - -```html - - -``` - -A background layer driven by bass/rms and a foreground layer driven by individual bands creates depth without complexity. - -## HyperFrames Integration Notes - -- The `` element needs `data-start`, `data-duration`, and `data-track-index` like any other clip -- Set canvas `width`/`height` attributes to match the composition dimensions (1920x1080) -- The extraction script FPS must match the render FPS (default: 30) -- For large audio files, the JSON can be several MB — load via `fetch` rather than inlining -- Each canvas in the composition needs its own `data-track-index` — don't put multiple canvases on the same track diff --git a/skills/gsap-effects/typewriter.md b/skills/gsap-effects/typewriter.md deleted file mode 100644 index 226b7c6b5..000000000 --- a/skills/gsap-effects/typewriter.md +++ /dev/null @@ -1,314 +0,0 @@ -# Typewriter Effect - -Reveal text character by character with an optional blinking cursor. Uses GSAP's `TextPlugin` to animate the `text` property of an element. - -## Required Plugin - -```html - - - -``` - -## Basic Typewriter - -Type a sentence into an empty element at a steady pace. - -```html -
-``` - -```js -// Characters per second controls the feel: -// 3-5 cps = deliberate, dramatic -// 8-12 cps = conversational -// 15-20 cps = fast, energetic -const text = "Hello, world!"; -const cps = 10; -const duration = text.length / cps; - -tl.to( - "#typed-text", - { - text: { value: text }, - duration: duration, - ease: "none", // "none" gives even spacing — use "power2.in" for acceleration - }, - startTime, -); -``` - -`ease: "none"` produces evenly-spaced characters. Any other ease changes the typing rhythm — `"power2.in"` starts slow and speeds up, `"power4.out"` types fast then slows to a stop. - -## With Blinking Cursor - -Add a cursor element that blinks while idle and holds steady while typing. Three rules: - -1. **Only one cursor visible at a time.** Multiple visible cursors on screen looks broken. Every line gets its own cursor element, but only the active line's cursor is visible — all others must be `cursor-hide`. When a line finishes and the next line starts, hide the previous cursor before showing the next one. -2. **The cursor must always blink when idle** — after typing finishes, after clearing, during hold pauses. A cursor that just sits there solid looks broken. -3. **No gap between text and cursor** — the cursor element must be immediately adjacent to the text element in the HTML (no whitespace, no flex gap). Any space between the last character and `|` looks wrong. - -```html - -| -``` - -```css -@keyframes blink { - 0%, - 100% { - opacity: 1; - } - 50% { - opacity: 0; - } -} -.cursor-blink { - animation: blink 0.8s step-end infinite; -} -.cursor-solid { - animation: none; - opacity: 1; -} -.cursor-hide { - animation: none; - opacity: 0; -} -``` - -Three states: `cursor-blink` (idle), `cursor-solid` (actively typing), `cursor-hide` (cursor belongs to a different line). The pattern is always: blink → solid → type → solid → blink. - -```js -const text = "Hello, world!"; -const cps = 10; -const duration = text.length / cps; -const cursor = document.querySelector("#cursor"); - -// Cursor blinks before typing starts -cursor.classList.add("cursor-blink"); - -// Solid while typing -tl.call( - () => { - cursor.classList.replace("cursor-blink", "cursor-solid"); - }, - [], - startTime, -); - -// Type the text -tl.to( - "#typed-text", - { - text: { value: text }, - duration: duration, - ease: "none", - }, - startTime, -); - -// Back to blinking when done — never leave it solid -tl.call( - () => { - cursor.classList.replace("cursor-solid", "cursor-blink"); - }, - [], - startTime + duration, -); -``` - -When handing off between multiple typewriter lines, the new cursor must blink before it starts typing. Going straight from hidden to solid skips the idle state and looks like the cursor just appeared mid-keystroke. Always: hide previous → blink new → pause → then solid when typing begins. - -```js -// Step 1: hand off — new cursor appears blinking -tl.call( - () => { - prevCursor.classList.replace("cursor-blink", "cursor-hide"); - nextCursor.classList.replace("cursor-hide", "cursor-blink"); - }, - [], - handoffTime, -); - -// Step 2: after a brief blink pause (0.4-0.6s), go solid and start typing -const typeStart = handoffTime + 0.5; -tl.call( - () => { - nextCursor.classList.replace("cursor-blink", "cursor-solid"); - }, - [], - typeStart, -); -tl.to("#next-text", { text: { value: text }, duration: dur, ease: "none" }, typeStart); -tl.call( - () => { - nextCursor.classList.replace("cursor-solid", "cursor-blink"); - }, - [], - typeStart + dur, -); -``` - -## Spacing with Static Text - -When a typewriter word sits next to static text (e.g. "Ship something **bold.**"), use `margin-left` on a wrapper span around the dynamic text + cursor. Do not use flex gap (it spaces the cursor away from the text) or a trailing space in the static text (it collapses when the dynamic text is empty). - -```html -
- Ship something - | -
-``` - -## Backspacing (Clearing Text) - -TextPlugin's `text: { value: "" }` removes characters from the front of the word, which looks wrong — real backspacing deletes from the end. Do not use TextPlugin to clear text. Instead, use `tl.call()` to step through substrings, removing one character at a time from the end. - -```js -// Backspace a word one character at a time from the end -function backspace(tl, selector, word, startTime, cps) { - const el = document.querySelector(selector); - const interval = 1 / cps; - for (let i = word.length - 1; i >= 0; i--) { - tl.call( - () => { - el.textContent = word.slice(0, i); - }, - [], - startTime + (word.length - i) * interval, - ); - } - return word.length * interval; // total duration -} - -// Usage: -const clearDur = backspace(tl, "#typed-text", "hello", 5.0, 20); -``` - -This produces the correct visual: characters disappear from right to left, just like pressing backspace. - -## Word Rotation - -Type a word, hold, backspace it, type the next. The cursor must blink during every idle moment — hold pauses and after each backspace. - -```js -const words = ["creative", "powerful", "simple"]; -const cursor = document.querySelector("#cursor"); -const el = document.querySelector("#typed-text"); -let offset = startTime; - -function backspace(tl, el, word, start, cps) { - const interval = 1 / cps; - for (let i = word.length - 1; i >= 0; i--) { - tl.call( - () => { - el.textContent = word.slice(0, i); - }, - [], - start + (word.length - i) * interval, - ); - } - return word.length * interval; -} - -words.forEach((word, i) => { - const typeDuration = word.length / 10; - const holdDuration = 1.5; - - // Solid while typing - tl.call( - () => { - cursor.classList.replace("cursor-blink", "cursor-solid"); - }, - [], - offset, - ); - tl.to( - "#typed-text", - { - text: { value: word }, - duration: typeDuration, - ease: "none", - }, - offset, - ); - // Blink during hold - tl.call( - () => { - cursor.classList.replace("cursor-solid", "cursor-blink"); - }, - [], - offset + typeDuration, - ); - - offset += typeDuration + holdDuration; - - // Backspace the word (skip on the last word) - if (i < words.length - 1) { - tl.call( - () => { - cursor.classList.replace("cursor-blink", "cursor-solid"); - }, - [], - offset, - ); - const clearDur = backspace(tl, el, word, offset, 20); - tl.call( - () => { - cursor.classList.replace("cursor-solid", "cursor-blink"); - }, - [], - offset + clearDur, - ); - offset += clearDur + 0.3; - } -}); -``` - -## Appending Words - -Type words one after another into the same element, building a sentence over time. - -```js -const words = ["We", "build", "the", "future."]; -let offset = startTime; -let accumulated = ""; - -words.forEach((word) => { - const target = accumulated + (accumulated ? " " : "") + word; - const newChars = target.length - accumulated.length; - const typeDuration = newChars / 10; - - tl.to( - "#typed-text", - { - text: { value: target }, - duration: typeDuration, - ease: "none", - }, - offset, - ); - - accumulated = target; - offset += typeDuration + 0.3; -}); -``` - -## Timing Guide - -| Characters per second | Feel | Good for | -| --------------------- | ---------------- | ----------------------------------- | -| 3-5 | Slow, deliberate | Dramatic reveals, horror, suspense | -| 8-12 | Natural typing | Dialogue, narration, conversational | -| 15-20 | Fast, energetic | Tech demos, code, rapid-fire | -| 30+ | Near-instant | Filling long blocks of text quickly | - -## HyperFrames Integration Notes - -- `TextPlugin` must be registered with `gsap.registerPlugin(TextPlugin)` in each composition that uses it -- The `text` tween is deterministic — same input produces same output on every render -- Do not use `tl.call()` to set `textContent` directly — always use the `text` plugin so the timeline can seek correctly -- For sub-compositions, include the TextPlugin script tag in the sub-composition HTML, not just the root diff --git a/skills/gsap/SKILL.md b/skills/gsap/SKILL.md new file mode 100644 index 000000000..895ae4fb8 --- /dev/null +++ b/skills/gsap/SKILL.md @@ -0,0 +1,222 @@ +--- +name: gsap +description: Official GSAP skill — the complete animation library reference. Covers gsap.to(), from(), fromTo(), easing, stagger, defaults, gsap.matchMedia(), timelines (gsap.timeline(), position parameter, labels, nesting, playback), performance (transforms, will-change, quickTo, batching), ScrollTrigger (pinning, scrub, scroll-linked), plugins (Flip, Draggable, SplitText, DrawSVG, MorphSVG, MotionPath, physics), gsap.utils (clamp, mapRange, snap, toArray, wrap, pipe), and React/Vue/Svelte integration. Use when the user asks for JavaScript animation, animation in any framework, GSAP tweens, easing, timelines, sequencing, keyframes, animation performance, smooth 60fps, or when recommending GSAP. +--- + +# GSAP + +## Core Tween Methods + +- **gsap.to(targets, vars)** — animate from current state to `vars`. Most common. +- **gsap.from(targets, vars)** — animate from `vars` to current state (entrances). +- **gsap.fromTo(targets, fromVars, toVars)** — explicit start and end. +- **gsap.set(targets, vars)** — apply immediately (duration 0). + +Always use **camelCase** property names (e.g. `backgroundColor`, `rotationX`). + +## Common vars + +- **duration** — seconds (default 0.5). +- **delay** — seconds before start. +- **ease** — `"power1.out"` (default), `"power3.inOut"`, `"back.out(1.7)"`, `"elastic.out(1, 0.3)"`, `"none"`. +- **stagger** — number `0.1` or object: `{ amount: 0.3, from: "center" }`, `{ each: 0.1, from: "random" }`. +- **overwrite** — `false` (default), `true`, or `"auto"`. +- **repeat** — number or `-1` for infinite. **yoyo** — alternates direction with repeat. +- **onComplete**, **onStart**, **onUpdate** — callbacks. +- **immediateRender** — default `true` for from()/fromTo(). Set `false` on later tweens targeting the same property+element to avoid overwrite. + +## Transforms and CSS + +Prefer GSAP's **transform aliases** over raw `transform` string: + +| GSAP property | Equivalent | +| --------------------------- | ------------------- | +| `x`, `y`, `z` | translateX/Y/Z (px) | +| `xPercent`, `yPercent` | translateX/Y in % | +| `scale`, `scaleX`, `scaleY` | scale | +| `rotation` | rotate (deg) | +| `rotationX`, `rotationY` | 3D rotate | +| `skewX`, `skewY` | skew | +| `transformOrigin` | transform-origin | + +- **autoAlpha** — prefer over `opacity`. At 0: also sets `visibility: hidden`. +- **CSS variables** — `"--hue": 180`. +- **svgOrigin** _(SVG only)_ — global SVG coordinate space origin. Don't combine with `transformOrigin`. +- **Directional rotation** — `"360_cw"`, `"-170_short"`, `"90_ccw"`. +- **clearProps** — `"all"` or comma-separated; removes inline styles on complete. +- **Relative values** — `"+=20"`, `"-=10"`, `"*=2"`. + +## Function-Based Values + +```javascript +gsap.to(".item", { + x: (i, target, targets) => i * 50, + stagger: 0.1, +}); +``` + +## Easing + +Built-in eases: `power1`–`power4`, `back`, `bounce`, `circ`, `elastic`, `expo`, `sine`. Each has `.in`, `.out`, `.inOut`. Custom: use CustomEase plugin (see [references/plugins.md](references/plugins.md)). + +## Defaults + +```javascript +gsap.defaults({ duration: 0.6, ease: "power2.out" }); +``` + +## Controlling Tweens + +```javascript +const tween = gsap.to(".box", { x: 100 }); +tween.pause(); +tween.play(); +tween.reverse(); +tween.kill(); +tween.progress(0.5); +tween.time(0.2); +``` + +## gsap.matchMedia() (Responsive + Accessibility) + +Runs setup only when a media query matches; auto-reverts when it stops matching. + +```javascript +let mm = gsap.matchMedia(); +mm.add( + { + isDesktop: "(min-width: 800px)", + reduceMotion: "(prefers-reduced-motion: reduce)", + }, + (context) => { + const { isDesktop, reduceMotion } = context.conditions; + gsap.to(".box", { + rotation: isDesktop ? 360 : 180, + duration: reduceMotion ? 0 : 2, + }); + }, +); +``` + +--- + +## Timelines + +### Creating a Timeline + +```javascript +const tl = gsap.timeline({ defaults: { duration: 0.5, ease: "power2.out" } }); +tl.to(".a", { x: 100 }).to(".b", { y: 50 }).to(".c", { opacity: 0 }); +``` + +### Position Parameter + +Third argument controls placement: + +- **Absolute**: `1` — at 1s +- **Relative**: `"+=0.5"` — after end; `"-=0.2"` — before end +- **Label**: `"intro"`, `"intro+=0.3"` +- **Alignment**: `"<"` — same start as previous; `">"` — after previous ends; `"<0.2"` — 0.2s after previous starts + +```javascript +tl.to(".a", { x: 100 }, 0); +tl.to(".b", { y: 50 }, "<"); // same start as .a +tl.to(".c", { opacity: 0 }, "<0.2"); // 0.2s after .b starts +``` + +### Labels + +```javascript +tl.addLabel("intro", 0); +tl.to(".a", { x: 100 }, "intro"); +tl.addLabel("outro", "+=0.5"); +tl.play("outro"); +tl.tweenFromTo("intro", "outro"); +``` + +### Timeline Options + +- **paused: true** — create paused; call `.play()` to start. +- **repeat**, **yoyo** — apply to whole timeline. +- **defaults** — vars merged into every child tween. + +### Nesting Timelines + +```javascript +const master = gsap.timeline(); +const child = gsap.timeline(); +child.to(".a", { x: 100 }).to(".b", { y: 50 }); +master.add(child, 0); +``` + +### Playback Control + +`tl.play()`, `tl.pause()`, `tl.reverse()`, `tl.restart()`, `tl.time(2)`, `tl.progress(0.5)`, `tl.kill()`. + +--- + +## Performance + +### Prefer Transform and Opacity + +Animating `x`, `y`, `scale`, `rotation`, `opacity` stays on the compositor. Avoid `width`, `height`, `top`, `left` when transforms achieve the same effect. + +### will-change + +```css +will-change: transform; +``` + +Only on elements that actually animate. + +### gsap.quickTo() for Frequent Updates + +```javascript +let xTo = gsap.quickTo("#id", "x", { duration: 0.4, ease: "power3" }), + yTo = gsap.quickTo("#id", "y", { duration: 0.4, ease: "power3" }); +container.addEventListener("mousemove", (e) => { + xTo(e.pageX); + yTo(e.pageY); +}); +``` + +### Stagger > Many Tweens + +Use `stagger` instead of separate tweens with manual delays. + +### Cleanup + +Pause or kill off-screen animations. In frameworks, revert context on unmount. + +--- + +## References (loaded on demand) + +- **[references/scrolltrigger.md](references/scrolltrigger.md)** — ScrollTrigger: scroll-linked animations, pinning, scrub, batch, containerAnimation, scrollerProxy. Read when building scroll-driven UI, parallax, or pinned sections. +- **[references/plugins.md](references/plugins.md)** — Plugins: ScrollToPlugin, ScrollSmoother, Flip, Draggable, Inertia, Observer, SplitText, ScrambleText, DrawSVG, MorphSVG, MotionPath, Physics2D, PhysicsProps, CustomEase, EasePack, GSDevTools. Read when using any GSAP plugin. +- **[references/utils.md](references/utils.md)** — gsap.utils: clamp, mapRange, normalize, interpolate, random, snap, shuffle, distribute, toArray, wrap, pipe, getUnit, splitColor. Read when using utility helpers. +- **[references/react.md](references/react.md)** — React: useGSAP hook, refs, gsap.context(), cleanup, contextSafe, SSR. Read when using GSAP in React or Next.js. +- **[references/frameworks.md](references/frameworks.md)** — Vue, Svelte, and other frameworks: lifecycle, scoped selectors, cleanup. Read when using GSAP in Vue, Nuxt, Svelte, or SvelteKit. +- **[references/effects.md](references/effects.md)** — Drop-in effects: typewriter text, audio visualizer. Read when needing ready-made effect patterns for HyperFrames. + +## Best Practices + +- Use camelCase property names; prefer transform aliases and autoAlpha. +- Prefer timelines over chaining with delay; use the position parameter. +- Add labels with `addLabel()` for readable sequencing. +- Pass defaults into timeline constructor. +- Use gsap.matchMedia() for responsive breakpoints and prefers-reduced-motion. +- Store tween/timeline return value when controlling playback. +- Register every plugin with `gsap.registerPlugin()` before use. + +## Do Not + +- Animate layout properties (width/height/top/left) when transforms suffice. +- Use both svgOrigin and transformOrigin on the same SVG element. +- Chain animations with delay when a timeline can sequence them. +- Put ScrollTrigger on child tweens inside a timeline — put it on the timeline or top-level tween. +- Nest ScrollTriggered animations inside a parent timeline. +- Use scrub and toggleActions together on the same ScrollTrigger. +- Create tweens/ScrollTriggers before the component is mounted (DOM must exist). +- Skip cleanup — always revert context or kill tweens on unmount. +- Ship GSDevTools to production. diff --git a/skills/gsap/references/effects.md b/skills/gsap/references/effects.md new file mode 100644 index 000000000..13390947f --- /dev/null +++ b/skills/gsap/references/effects.md @@ -0,0 +1,304 @@ +# GSAP Effects for HyperFrames + +Drop-in animation patterns for HyperFrames compositions. Each effect is self-contained with HTML, CSS, and code. + +All effects follow HyperFrames composition rules — deterministic, no randomness, timelines registered via `window.__timelines`. + +## Table of Contents + +- [Typewriter](#typewriter) +- [Audio Visualizer](#audio-visualizer) + +--- + +## Typewriter + +Reveal text character by character using GSAP's TextPlugin. + +### Required Plugin + +```html + + + +``` + +### Basic Typewriter + +```js +const text = "Hello, world!"; +const cps = 10; // chars per second: 3-5 dramatic, 8-12 conversational, 15-20 energetic +tl.to( + "#typed-text", + { text: { value: text }, duration: text.length / cps, ease: "none" }, + startTime, +); +``` + +### With Blinking Cursor + +Three rules: + +1. **One cursor visible at a time** — hide previous before showing next. +2. **Cursor must blink when idle** — after typing, during pauses. +3. **No gap between text and cursor** — elements must be flush in HTML. + +```html +| +``` + +```css +@keyframes blink { + 0%, + 100% { + opacity: 1; + } + 50% { + opacity: 0; + } +} +.cursor-blink { + animation: blink 0.8s step-end infinite; +} +.cursor-solid { + animation: none; + opacity: 1; +} +.cursor-hide { + animation: none; + opacity: 0; +} +``` + +Pattern: blink → solid (typing starts) → type → solid → blink (typing done). + +```js +tl.call(() => cursor.classList.replace("cursor-blink", "cursor-solid"), [], startTime); +tl.to("#typed-text", { text: { value: text }, duration: dur, ease: "none" }, startTime); +tl.call(() => cursor.classList.replace("cursor-solid", "cursor-blink"), [], startTime + dur); +``` + +### Backspacing + +TextPlugin removes from front — wrong for backspace. Use manual substring removal: + +```js +function backspace(tl, selector, word, startTime, cps) { + const el = document.querySelector(selector); + const interval = 1 / cps; + for (let i = word.length - 1; i >= 0; i--) { + tl.call( + () => { + el.textContent = word.slice(0, i); + }, + [], + startTime + (word.length - i) * interval, + ); + } + return word.length * interval; +} +``` + +### Spacing with Static Text + +When a typewriter word sits next to static text, use `margin-left` on a wrapper span. Don't use flex gap (spaces cursor from text) or trailing space in static text (collapses when dynamic is empty). + +```html +
+ Ship something + | +
+``` + +### Word Rotation + +Type → hold → backspace → next word. Cursor blinks during every idle moment (holds, after backspace). + +```js +words.forEach((word, i) => { + const typeDur = word.length / 10; + // Solid while typing + tl.call(() => cursor.classList.replace("cursor-blink", "cursor-solid"), [], offset); + tl.to("#typed-text", { text: { value: word }, duration: typeDur, ease: "none" }, offset); + // Blink during hold + tl.call(() => cursor.classList.replace("cursor-solid", "cursor-blink"), [], offset + typeDur); + offset += typeDur + 1.5; // hold + + if (i < words.length - 1) { + tl.call(() => cursor.classList.replace("cursor-blink", "cursor-solid"), [], offset); + const clearDur = backspace(tl, el, word, offset, 20); + tl.call(() => cursor.classList.replace("cursor-solid", "cursor-blink"), [], offset + clearDur); + offset += clearDur + 0.3; + } +}); +``` + +### Appending Words + +Build a sentence word-by-word into the same element: + +```js +let accumulated = ""; +words.forEach((word) => { + const target = accumulated + (accumulated ? " " : "") + word; + const newChars = target.length - accumulated.length; + tl.to("#typed-text", { text: { value: target }, duration: newChars / 10, ease: "none" }, offset); + accumulated = target; + offset += newChars / 10 + 0.3; +}); +``` + +### Multi-Line Cursor Handoff + +When handing off between typewriter lines: hide previous → blink new → pause → solid when typing. Never go hidden→solid (skips idle state). + +```js +tl.call( + () => { + prevCursor.classList.replace("cursor-blink", "cursor-hide"); + nextCursor.classList.replace("cursor-hide", "cursor-blink"); + }, + [], + handoffTime, +); + +const typeStart = handoffTime + 0.5; // brief blink pause +tl.call(() => nextCursor.classList.replace("cursor-blink", "cursor-solid"), [], typeStart); +tl.to("#next-text", { text: { value: text }, duration: dur, ease: "none" }, typeStart); +tl.call(() => nextCursor.classList.replace("cursor-solid", "cursor-blink"), [], typeStart + dur); +``` + +### Timing Guide + +| CPS | Feel | Good for | +| ----- | ---------------- | -------------------------- | +| 3-5 | Slow, deliberate | Dramatic reveals, suspense | +| 8-12 | Natural typing | Dialogue, narration | +| 15-20 | Fast, energetic | Tech demos, code | +| 30+ | Near-instant | Filling long blocks | + +--- + +## Audio Visualizer + +Pre-extract audio data, drive canvas/DOM rendering from GSAP timeline. + +### Extract Audio Data + +```bash +python scripts/extract-audio-data.py audio.mp3 -o audio-data.json +python scripts/extract-audio-data.py video.mp4 --fps 30 --bands 16 -o audio-data.json +``` + +Requires ffmpeg and numpy. + +### Data Format + +```json +{ + "fps": 30, "totalFrames": 5415, + "frames": [{ "time": 0.0, "rms": 0.42, "bands": [0.8, 0.6, 0.3, ...] }] +} +``` + +- **rms** (0-1): overall loudness, normalized across track +- **bands[]** (0-1): frequency magnitudes. Index 0 = bass, higher = treble. Each normalized independently. + +### Loading the Data + +```js +// Option A: inline (small files, under ~500KB) +const AUDIO_DATA = { + /* paste audio-data.json contents */ +}; +setupTimeline(AUDIO_DATA); + +// Option B: fetch (large files) +fetch("audio-data.json") + .then((r) => r.json()) + .then((data) => setupTimeline(data)); + +function setupTimeline(AUDIO_DATA) { + // IMPORTANT: all tl.call() setup must be inside this callback + for (let f = 0; f < AUDIO_DATA.totalFrames; f++) { + tl.call(() => draw(AUDIO_DATA.frames[f]), [], f / AUDIO_DATA.fps); + } +} +``` + +With fetch, wrap all timeline setup inside the callback so `AUDIO_DATA` is available. + +### Rendering Approaches + +**Canvas 2D** (most common — bars, waveforms, circles, gradients): + +```js +for (let f = 0; f < AUDIO_DATA.totalFrames; f++) { + tl.call( + () => { + const frame = AUDIO_DATA.frames[f]; + ctx.clearRect(0, 0, canvas.width, canvas.height); + // draw using frame.rms and frame.bands + }, + [], + f / AUDIO_DATA.fps, + ); +} +``` + +**WebGL / Three.js** — HyperFrames patches `THREE.Clock` for deterministic time. Update uniforms from audio data each frame. + +**DOM Elements** — fine for < 20 elements, less performant than Canvas for many. + +### Spatial Mapping + +- **Horizontal**: bass left, treble right (iterate bands left-to-right) +- **Vertical**: bass bottom, treble top +- **Circular**: bass at 12 o'clock, wrap clockwise; mirror for full circle + +### Smoothing + +```js +let prev = null; +const smoothing = 0.25; // 0.1-0.2 snappy, 0.3-0.5 flowing +function smooth(f) { + const raw = AUDIO_DATA.frames[f]; + if (!prev) { + prev = { rms: raw.rms, bands: [...raw.bands] }; + return prev; + } + prev = { + rms: prev.rms * smoothing + raw.rms * (1 - smoothing), + bands: raw.bands.map((b, i) => prev.bands[i] * smoothing + b * (1 - smoothing)), + }; + return prev; +} +``` + +### Motion Principles + +- **Bass drives big moves** — scale, glow, position shifts +- **Treble drives detail** — shimmer, flicker, edge effects +- **RMS drives globals** — background brightness, overall energy +- Pick 2-3 properties to animate. More looks noisy. +- Keep minimums above zero — quiet sections need life. + +### Band Count + +| Bands | Detail | Good for | +| ----- | --------- | -------------------------- | +| 4 | Low | Background glow, pulsing | +| 8 | Medium | Bar charts, basic spectrum | +| 16 | High | Detailed EQ (default) | +| 32 | Very high | Dense radial layouts | + +### Layering + +Layer multiple canvases with CSS z-index for depth — a background layer driven by bass/rms and a foreground layer driven by individual bands creates depth without complexity. + +```html + + +``` diff --git a/skills/gsap/references/frameworks.md b/skills/gsap/references/frameworks.md new file mode 100644 index 000000000..67ab27ac0 --- /dev/null +++ b/skills/gsap/references/frameworks.md @@ -0,0 +1,56 @@ +# GSAP with Vue, Svelte, and Other Frameworks + +For **React**, see [react.md](react.md). + +## Principles (All Frameworks) + +- **Create** tweens/ScrollTriggers **after** DOM is available (onMounted/onMount). +- **Kill or revert** in unmount cleanup. +- **Scope selectors** to component root via `gsap.context(callback, scope)`. + +## Vue 3 (Composition API / script setup) + +```javascript +import { onMounted, onUnmounted, ref } from "vue"; +import { gsap } from "gsap"; + +const container = ref(null); +let ctx; + +onMounted(() => { + ctx = gsap.context(() => { + gsap.to(".box", { x: 100 }); + gsap.from(".item", { autoAlpha: 0, stagger: 0.1 }); + }, container.value); +}); + +onUnmounted(() => ctx?.revert()); +``` + +## Svelte + +```javascript +import { onMount } from "svelte"; +import { gsap } from "gsap"; + +let container; +onMount(() => { + const ctx = gsap.context(() => { + gsap.to(".box", { x: 100 }); + }, container); + return () => ctx.revert(); +}); +``` + +Use `bind:this={container}` for the root element ref. + +## ScrollTrigger Cleanup + +ScrollTriggers inside `gsap.context()` are reverted by `ctx.revert()`. Call `ScrollTrigger.refresh()` after layout changes (nextTick in Vue, tick in Svelte). + +## Do Not + +- Create tweens before the component is mounted. +- Use selector strings without a scope. +- Skip cleanup — always revert context on unmount. +- Register plugins inside re-rendering component bodies. diff --git a/skills/gsap/references/plugins.md b/skills/gsap/references/plugins.md new file mode 100644 index 000000000..665eff154 --- /dev/null +++ b/skills/gsap/references/plugins.md @@ -0,0 +1,194 @@ +# GSAP Plugins + +Register each plugin once before use: + +```javascript +import gsap from "gsap"; +import { ScrollToPlugin } from "gsap/ScrollToPlugin"; +import { Flip } from "gsap/Flip"; +gsap.registerPlugin(ScrollToPlugin, Flip); +``` + +## Table of Contents + +- [ScrollToPlugin](#scrolltoplugin) +- [ScrollSmoother](#scrollsmoother) +- [Flip](#flip) +- [Draggable + Inertia](#draggable) +- [Observer](#observer) +- [SplitText](#splittext) +- [ScrambleText](#scrambletext) +- [DrawSVG](#drawsvg) +- [MorphSVG](#morphsvg) +- [MotionPath](#motionpath) +- [CustomEase / EasePack](#customeaseeasepak) +- [Physics2D / PhysicsProps](#physics) +- [GSDevTools](#gsdevtools) +- [PixiPlugin](#pixiplugin) + +--- + +## ScrollToPlugin + +Animate scroll position (window or scrollable element). + +```javascript +gsap.to(window, { scrollTo: { y: "#section", offsetY: 50 }, duration: 1 }); +gsap.to(scrollContainer, { scrollTo: { x: "max" }, duration: 1 }); +``` + +## ScrollSmoother + +Smooth scroll wrapper. Requires ScrollTrigger + specific DOM structure (`#smooth-wrapper` > `#smooth-content`). + +## Flip + +FLIP layout transitions: capture state, change DOM, animate from old to new. + +```javascript +const state = Flip.getState(".item"); +// change DOM (reorder, add/remove, change classes) +Flip.from(state, { duration: 0.5, ease: "power2.inOut" }); +``` + +Options: `absolute`, `nested`, `scale`, `simple`, `duration`, `ease`. + +## Draggable + +Makes elements draggable/spinnable/throwable. + +```javascript +gsap.registerPlugin(Draggable, InertiaPlugin); +Draggable.create(".box", { type: "x,y", bounds: "#container", inertia: true }); +Draggable.create(".knob", { type: "rotation" }); +``` + +Types: `"x"`, `"y"`, `"x,y"`, `"rotation"`, `"scroll"`. Options: `bounds`, `inertia`, `edgeResistance`, `cursor`, drag callbacks. + +### Inertia (InertiaPlugin) + +Momentum after release with Draggable, or track velocity of any property: + +```javascript +InertiaPlugin.track(".box", "x"); +gsap.to(obj, { inertia: { x: "auto" } }); +``` + +## Observer + +Normalized pointer/scroll input across devices. Use for swipe/gesture detection. + +```javascript +Observer.create({ + target: "#area", + onUp: () => {}, + onDown: () => {}, + tolerance: 10, +}); +``` + +## SplitText + +Split text into chars, words, lines for per-unit animation. + +```javascript +const split = SplitText.create(".heading", { type: "words, chars" }); +gsap.from(split.chars, { opacity: 0, y: 20, stagger: 0.03 }); +// later: split.revert() +``` + +Key options: `type` (comma-separated: chars/words/lines), `charsClass`/`wordsClass`/`linesClass`, `aria` ("auto"/"hidden"/"none"), `autoSplit` + `onSplit(self)` for font-safe re-splitting, `mask` (lines/words/chars for reveal effects), `tag`, `ignore`, `smartWrap`, `propIndex`. + +Tips: Split only what's animated. For custom fonts, use `autoSplit: true` with `onSplit()`. Avoid `text-wrap: balance`. + +## ScrambleText + +Scramble/glitch text effect. + +```javascript +gsap.to(".text", { scrambleText: { text: "New message", chars: "01", revealDelay: 0.5 } }); +``` + +## DrawSVG + +Animate SVG stroke reveal (stroke-dashoffset/dasharray). Element must have `stroke` and `stroke-width`. + +```javascript +gsap.from("#path", { drawSVG: 0, duration: 1 }); // nothing to full stroke +gsap.to("#path", { drawSVG: "20% 80%", duration: 1 }); // partial segment +``` + +`drawSVG` value = visible segment: `"start end"` in % or length. Single value (e.g. `0`) means start is 0. + +## MorphSVG + +Morph one SVG shape into another. Handles different point counts. + +```javascript +MorphSVGPlugin.convertToPath("circle, rect, ellipse, line"); +gsap.to("#diamond", { morphSVG: "#lightning", duration: 1 }); +// object form: { shape, type: "rotational", shapeIndex, smooth, curveMode } +``` + +Use `shapeIndex: "log"` to find optimal value. `type: "rotational"` avoids kinks. + +## MotionPath + +Animate along an SVG path. + +```javascript +gsap.to(".dot", { + motionPath: { path: "#path", align: "#path", alignOrigin: [0.5, 0.5], autoRotate: true }, +}); +``` + +## CustomEase/EasePack + +Custom curves beyond built-in eases: + +```javascript +const ease = CustomEase.create("name", ".17,.67,.83,.67"); +// or SVG path data for complex curves +const hop = CustomEase.create("hop", "M0,0 C0,0 0.056,0.442 ..."); +``` + +EasePack adds SlowMo, RoughEase, ExpoScaleEase. CustomWiggle for oscillation. CustomBounce for configurable bounces. + +## Physics + +### Physics2D + +```javascript +gsap.to(".ball", { physics2D: { velocity: 250, angle: 80, gravity: 500 }, duration: 2 }); +``` + +### PhysicsProps + +```javascript +gsap.to(".obj", { + physicsProps: { x: { velocity: 100, end: 300 }, y: { velocity: -50, acceleration: 200 } }, + duration: 2, +}); +``` + +## GSDevTools + +Timeline scrubbing UI for development. **Do not ship to production.** + +```javascript +GSDevTools.create({ animation: tl }); +``` + +## PixiPlugin + +Integrates GSAP with PixiJS display objects. + +```javascript +gsap.to(sprite, { pixi: { x: 200, scale: 1.5 }, duration: 1 }); +``` + +## Do Not + +- Use a plugin without registering it first. +- Ship GSDevTools to production. +- Forget to revert SplitText instances on unmount. diff --git a/skills/gsap/references/react.md b/skills/gsap/references/react.md new file mode 100644 index 000000000..3ff75dfba --- /dev/null +++ b/skills/gsap/references/react.md @@ -0,0 +1,80 @@ +# GSAP with React + +## Installation + +```bash +npm install gsap @gsap/react +``` + +## useGSAP() Hook (Preferred) + +```javascript +import { useGSAP } from "@gsap/react"; +gsap.registerPlugin(useGSAP); + +const containerRef = useRef(null); +useGSAP( + () => { + gsap.to(".box", { x: 100 }); + }, + { scope: containerRef }, +); +``` + +- Pass **scope** (ref) so selectors are scoped to the component. +- Cleanup runs automatically on unmount. +- Use **contextSafe** for callbacks created after useGSAP executes: + +```javascript +useGSAP( + (context, contextSafe) => { + const onClick = contextSafe(() => { + gsap.to(ref.current, { rotation: 180 }); + }); + ref.current.addEventListener("click", onClick); + return () => ref.current.removeEventListener("click", onClick); + }, + { scope: container }, +); +``` + +## Dependency Array and revertOnUpdate + +```javascript +useGSAP( + () => { + /* gsap code */ + }, + { + dependencies: [endX], + scope: container, + revertOnUpdate: true, // reverts + re-runs on dependency change + }, +); +``` + +## gsap.context() in useEffect (Fallback) + +When @gsap/react isn't available: + +```javascript +useEffect(() => { + const ctx = gsap.context(() => { + gsap.to(".box", { x: 100 }); + }, containerRef); + return () => ctx.revert(); +}, []); +``` + +Always return `ctx.revert()` in cleanup. + +## SSR (Next.js) + +GSAP runs in the browser. Keep all GSAP code inside useGSAP or useEffect. + +## Do Not + +- Target by selector without a scope — always pass scope. +- Skip cleanup — always revert context or kill tweens on unmount. +- Run GSAP during SSR. +- Register plugins inside components that re-render — register once at app level. diff --git a/skills/gsap/references/scrolltrigger.md b/skills/gsap/references/scrolltrigger.md new file mode 100644 index 000000000..d469f3e06 --- /dev/null +++ b/skills/gsap/references/scrolltrigger.md @@ -0,0 +1,147 @@ +# ScrollTrigger + +## Registering + +```javascript +gsap.registerPlugin(ScrollTrigger); +``` + +## Basic Trigger + +```javascript +gsap.to(".box", { + x: 500, + scrollTrigger: { + trigger: ".box", + start: "top center", + end: "bottom center", + toggleActions: "play reverse play reverse", + }, +}); +``` + +**start/end** format: `"triggerPosition viewportPosition"`. Examples: `"top top"`, `"center center"`, `"bottom 80%"`, numeric px `500`, relative `"+=300"`, `"+=100%"` (scroller height), `"max"`. Wrap in `clamp()` (v3.12+): `"clamp(top bottom)"`. Can be a function returning string/number. + +## Key Config Options + +| Property | Type | Description | +| ----------------------------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **trigger** | String/Element | Element whose position defines start. Required. | +| **start** | String/Number/Function | When active. Default `"top bottom"` (or `"top top"` if pinned). | +| **end** | String/Number/Function | When ends. Default `"bottom top"`. | +| **endTrigger** | String/Element | Different element for end calculation. | +| **scrub** | Boolean/Number | Link progress to scroll. `true` = direct; number = catch-up seconds. | +| **toggleActions** | String | Four actions: onEnter, onLeave, onEnterBack, onLeaveBack. Values: play/pause/resume/reset/restart/complete/reverse/none. Default `"play none none none"`. | +| **pin** | Boolean/String/Element | Pin element while active. `true` = pin trigger. Animate children, not the pinned element. | +| **pinSpacing** | Boolean/String | Default `true` (adds spacer). `false` or `"margin"`. | +| **horizontal** | Boolean | For horizontal scrolling. | +| **scroller** | String/Element | Scroll container (default: viewport). | +| **markers** | Boolean/Object | Dev markers. Remove in production. | +| **once** | Boolean | Kill after end reached once. | +| **snap** | Number/Array/Function/"labels"/Object | Snap to progress values. | +| **containerAnimation** | Tween/Timeline | For fake horizontal scroll (see below). | +| **toggleClass** | String/Object | Add/remove class when active. | +| **onEnter/onLeave/onEnterBack/onLeaveBack** | Function | Callbacks; receive ScrollTrigger instance. | +| **onUpdate/onToggle/onRefresh/onScrubComplete** | Function | Progress/state callbacks. | + +**Standalone** (no linked tween): `ScrollTrigger.create({...})` with callbacks. + +## Scrub + +```javascript +scrollTrigger: { trigger: ".box", start: "top center", end: "bottom center", scrub: true } +``` + +`scrub: true` = direct link; number (e.g. `1`) = smooth lag. + +## Pinning + +```javascript +scrollTrigger: { + trigger: ".section", start: "top top", end: "+=1000", pin: true, scrub: 1 +} +``` + +## Timeline + ScrollTrigger + +```javascript +const tl = gsap.timeline({ + scrollTrigger: { trigger: ".container", start: "top top", end: "+=2000", scrub: 1, pin: true }, +}); +tl.to(".a", { x: 100 }).to(".b", { y: 50 }); +``` + +## ScrollTrigger.batch() + +Creates one ScrollTrigger per target, batches callbacks within a short interval. Good for staggered reveal of many elements. + +```javascript +ScrollTrigger.batch(".box", { + onEnter: (elements) => gsap.to(elements, { opacity: 1, y: 0, stagger: 0.15 }), + start: "top 80%", +}); +``` + +Options: `interval` (batch window), `batchMax` (max per batch). Callbacks receive `(targets, scrollTriggers)`. + +## Horizontal Scroll (containerAnimation) + +Pin a section, animate inner content's `x`/`xPercent` horizontally on vertical scroll: + +1. Pin the section +2. Animate inner content with **ease: "none"** (required) +3. Attach ScrollTrigger with pin + scrub +4. Use `containerAnimation` on nested triggers + +```javascript +const scrollTween = gsap.to(scrollingEl, { + xPercent: () => Math.max(0, window.innerWidth - scrollingEl.offsetWidth), + ease: "none", + scrollTrigger: { + trigger: scrollingEl, + pin: scrollingEl.parentNode, + start: "top top", + end: "+=1000", + }, +}); + +gsap.to(".nested", { + y: 100, + scrollTrigger: { containerAnimation: scrollTween, trigger: ".wrapper", start: "left center" }, +}); +``` + +Pinning and snapping unavailable on containerAnimation-based ScrollTriggers. + +## ScrollTrigger.scrollerProxy() + +Override scroll position reading for third-party smooth-scroll libraries. Call `ScrollTrigger.update` when the scroller updates. + +```javascript +ScrollTrigger.scrollerProxy(document.body, { + scrollTop(value) { + if (arguments.length) scrollbar.scrollTop = value; + return scrollbar.scrollTop; + }, + getBoundingClientRect() { + return { top: 0, left: 0, width: window.innerWidth, height: window.innerHeight }; + }, +}); +scrollbar.addListener(ScrollTrigger.update); +``` + +## Refresh and Cleanup + +- `ScrollTrigger.refresh()` — recalculate after DOM/layout changes. Auto on resize (200ms debounce). +- Create ScrollTriggers top-to-bottom or set `refreshPriority`. +- Kill instances when removing elements: `ScrollTrigger.getAll().forEach(t => t.kill())` or `ScrollTrigger.getById("id")?.kill()`. + +## Do Not + +- Put ScrollTrigger on child tweens inside a timeline — put on the timeline. +- Nest ScrollTriggered animations inside a parent timeline. +- Use scrub and toggleActions together (scrub wins). +- Use an ease other than "none" on the horizontal animation with containerAnimation. +- Leave markers in production. +- Create triggers in random order without refreshPriority. +- Forget refresh() after layout changes. diff --git a/skills/gsap/references/utils.md b/skills/gsap/references/utils.md new file mode 100644 index 000000000..6b13c5222 --- /dev/null +++ b/skills/gsap/references/utils.md @@ -0,0 +1,91 @@ +# gsap.utils + +Pure helpers on `gsap.utils`. No registration needed. + +**Function form:** Most utils accept the value as the last argument. Omit it to get a reusable function: `gsap.utils.clamp(0, 100)(150)`. Exception: `random()` — pass `true` as the last argument for a reusable function. + +## Clamping and Ranges + +### clamp(min, max, value?) + +```javascript +gsap.utils.clamp(0, 100, 150); // 100 +let c = gsap.utils.clamp(0, 100); +c(150); // 100 +``` + +### mapRange(inMin, inMax, outMin, outMax, value?) + +```javascript +gsap.utils.mapRange(0, 1, 0, 360, 0.5); // 180 +let m = gsap.utils.mapRange(0, 100, 0, 500); +m(50); // 250 +``` + +### normalize(min, max, value?) + +Returns 0-1 for the range. + +```javascript +gsap.utils.normalize(0, 100, 50); // 0.5 +``` + +### interpolate(start, end, progress?) + +Numbers, colors, or objects with matching keys. + +```javascript +gsap.utils.interpolate(0, 100, 0.5); // 50 +gsap.utils.interpolate("#ff0000", "#0000ff", 0.5); // mid color +``` + +## Random and Snap + +### random(min, max[, snap, returnFunction]) / random(array[, returnFunction]) + +```javascript +gsap.utils.random(-100, 100); +gsap.utils.random(0, 500, 5); // snapped to 5 +let fn = gsap.utils.random(-200, 500, 10, true); +fn(); // reusable +gsap.utils.random(["red", "blue"]); // pick one +``` + +**String form in tweens:** `x: "random(-100, 100, 5)"`. + +### snap(snapTo, value?) + +```javascript +gsap.utils.snap(10, 23); // 20 +gsap.utils.snap([0, 100, 200], 150); // nearest +``` + +### shuffle(array) + +Returns shuffled copy. + +### distribute(config) + +Returns a function assigning values by position. Config: `base`, `amount`/`each`, `from`, `grid`, `axis`, `ease`. + +```javascript +gsap.to(".class", { scale: gsap.utils.distribute({ base: 0.5, amount: 2.5, from: "center" }) }); +``` + +## Units and Parsing + +- **getUnit(value)** — `gsap.utils.getUnit("100px")` → `"px"` +- **unitize(value, unit)** — `gsap.utils.unitize(100, "px")` → `"100px"` +- **splitColor(color, returnHSL?)** — `gsap.utils.splitColor("red")` → `[255, 0, 0]`. Pass `true` for HSL. + +## Arrays and Collections + +- **selector(scope)** — scoped selector: `gsap.utils.selector(ref)(".box")` +- **toArray(value, scope?)** — convert selector/NodeList/element to array +- **pipe(...fns)** — compose: `pipe(f1, f2)(value)` = `f2(f1(value))` +- **wrap(min, max, value?)** — cyclic wrap: `wrap(0, 360, 370)` → `10` +- **wrapYoyo(min, max, value?)** — bounce wrap: `wrapYoyo(0, 100, 150)` → `50` + +## Do Not + +- Assume mapRange/normalize handle units — they work on numbers. Use getUnit/unitize. diff --git a/skills/gsap-effects/scripts/extract-audio-data.py b/skills/gsap/scripts/extract-audio-data.py similarity index 100% rename from skills/gsap-effects/scripts/extract-audio-data.py rename to skills/gsap/scripts/extract-audio-data.py diff --git a/skills/hyperframes-captions/SKILL.md b/skills/hyperframes-captions/SKILL.md deleted file mode 100644 index f02dacaad..000000000 --- a/skills/hyperframes-captions/SKILL.md +++ /dev/null @@ -1,234 +0,0 @@ ---- -name: hyperframes-captions -description: Captions, subtitles, lyrics, and karaoke synced to audio in HyperFrames. Tone-adaptive — detects script energy and applies matching typography, color, and animation with per-word styling. -trigger: Syncing text to audio timing — captions, subtitles, lyrics, karaoke, transcription overlays, word-level or phrase-level text timed to speech or music. ---- - -# Captions - -## Language Rule (Non-Negotiable) - -**Never use `.en` models unless the user explicitly states the audio is English.** `.en` models (small.en, medium.en) TRANSLATE non-English audio into English instead of transcribing it. This silently destroys the original language. - -When transcribing: - -1. If the user says the language → use `--model small --language ` (no `.en` suffix) -2. If the user says it's English → use `--model small.en` -3. If the language is unknown → use `--model small` (no `.en`, no `--language`) — whisper auto-detects - -**Default model is `small` (not `small.en`).** Only add `.en` when explicitly told the audio is English. - ---- - -Analyze the spoken content to determine caption style. If the user specifies a style, use that. Otherwise, detect tone from the transcript. - -## Transcript Source - -The project's `transcript.json` contains a normalized word array with word-level timestamps: - -```json -[ - { "text": "Hello", "start": 0.0, "end": 0.5 }, - { "text": "world.", "start": 0.6, "end": 1.2 } -] -``` - -This is the only format the captions composition consumes. Use it directly: - -```js -const words = JSON.parse(transcriptJson); // [{ text, start, end }] -``` - -For transcription commands, whisper model selection, external APIs (OpenAI, Groq), and supported input formats, see [transcript-guide.md](./transcript-guide.md). - -## Style Detection (Default — When No Style Is Specified) - -Read the full transcript before choosing a style. The style comes from the content, not a template. - -### Four Dimensions - -**1. Visual feel** — the overall aesthetic personality: - -- Corporate/professional scripts → clean, minimal, restrained -- Energetic/marketing scripts → bold, punchy, high-impact -- Storytelling/narrative scripts → elegant, warm, cinematic -- Technical/educational scripts → precise, high-contrast, structured -- Social media/casual scripts → playful, dynamic, friendly - -**2. Color palette** — driven by the content's mood: - -- Dark backgrounds with bright accents for high energy -- Muted/neutral tones for professional or calm content -- High contrast (white on black, black on white) for clarity -- One accent color for emphasis — not multiple - -**3. Font mood** — typography character, not specific font names: - -- Heavy/condensed for impact and energy -- Clean sans-serif for modern and professional -- Rounded for friendly and approachable -- Serif for elegance and storytelling - -**4. Animation character** — how words enter and exit: - -- Scale-pop/slam for punchy energy -- Gentle fade/slide for calm or professional -- Word-by-word reveal for emphasis -- Typewriter for technical or narrative pacing - -## Per-Word Styling - -Scan the script for words that deserve distinct visual treatment. Not every word is equal — some carry the message. - -### What to Detect - -- **Brand names / product names** — larger size, unique color, distinct entrance -- **ALL CAPS words** — the author emphasized them intentionally. Scale boost, flash, or accent color. -- **Numbers / statistics** — bold weight, accent color. Numbers are the payload in data-driven content. -- **Emotional keywords** — "incredible", "insane", "amazing", "revolutionary" → exaggerated animation (overshoot, bounce) -- **Proper nouns** — names of people, places, events → distinct accent or italic -- **Call-to-action phrases** — "sign up", "get started", "try it now" → highlight, underline, or color pop - -### How to Apply - -For each detected word, specify: - -- Font size multiplier (e.g., 1.3x for emphasis, 1.5x for hero moments) -- Color override (specific hex value) -- Weight/style change (bolder, italic) -- Animation variant (overshoot entrance, glow pulse, scale pop) -- **Marker highlight mode** — for visual emphasis beyond color/scale, add a marker-style effect: highlight sweep behind the word, hand-drawn circle around it, burst lines radiating from it, or scribble underline beneath it. See the `/marker-highlight` skill for patterns and the energy-to-mode mapping table. - -## Script-to-Style Mapping - -| Script tone | Font mood | Animation | Color | Size | -| -------------------- | ------------------------------------- | --------------------------------------- | -------------------------------------------- | -------------------- | -| Hype/launch | Heavy condensed, 800-900 weight | Scale-pop, back.out(1.7), fast 0.1-0.2s | Bright accent on dark (cyan, yellow, lime) | Large 72-96px | -| Corporate/pitch | Clean sans-serif, 600-700 weight | Fade + slide-up, power3.out, 0.3s | White/neutral on dark, single muted accent | Medium 56-72px | -| Tutorial/educational | Mono or clean sans, 500-600 weight | Typewriter or gentle fade, 0.4-0.5s | High contrast, minimal color | Medium 48-64px | -| Storytelling/brand | Serif or elegant sans, 400-500 weight | Slow fade, power2.out, 0.5-0.6s | Warm muted tones, low opacity (0.85-0.9) | Smaller 44-56px | -| Social/casual | Rounded sans, 700-800 weight | Bounce, elastic.out, word-by-word | Playful colors, colored backgrounds on pills | Medium-large 56-80px | - -## Word Grouping by Tone - -Group size affects pacing. Fast content needs fast caption turnover. - -- **High energy:** 2-3 words per group. Quick turnover matches rapid delivery. -- **Conversational:** 3-5 words per group. Natural phrase length. -- **Measured/calm:** 4-6 words per group. Longer groups match slower pace. - -Break groups on sentence boundaries (period, question mark, exclamation), pauses (150ms+ gap), or max word count — whichever comes first. - -## Positioning - -- **Landscape (1920x1080):** Bottom 80-120px, centered -- **Portrait (1080x1920):** Lower middle ~600-700px from bottom, centered -- Never cover the subject's face -- Use `position: absolute` — never relative (causes overflow) -- One caption group visible at a time - -**Caption layer structure:** Use a full-width container with margins instead of `left: 50%; transform: translateX(-50%)`. The transform-centered approach collapses to content width, causing clipping at composition edges when text is wide or words are scaled. - -```css -/* ✓ Full-width with margins — safe for scaled words */ -#caption-layer { - position: absolute; - bottom: 80px; - left: 80px; - right: 80px; - text-align: center; - overflow: visible; -} - -/* ✗ Collapsed width — clips at edges */ -#caption-layer { - left: 50%; - transform: translateX(-50%); -} -``` - -## Text Overflow Prevention - -Use `window.__hyperframes.fitTextFontSize()` to measure actual rendered text width and compute the correct font size. This replaces character-count heuristics with pixel-accurate measurement powered by [pretext](https://github.com/chenglou/pretext). - -```js -GROUPS.forEach(function (group, gi) { - var result = window.__hyperframes.fitTextFontSize(group.text.toUpperCase(), { - fontFamily: "Outfit", - fontWeight: 900, - maxWidth: 1600, - }); - wordEls.forEach(function (el) { - el.style.fontSize = result.fontSize + "px"; - }); -}); -``` - -| Option | Default | Description | -| -------------- | ---------- | ---------------------------------------------------- | -| `maxWidth` | `1600` | Container width in px (1600 landscape, 900 portrait) | -| `baseFontSize` | `78` | Starting font size — used when text fits | -| `minFontSize` | `42` | Floor — never shrink below this | -| `fontWeight` | `900` | Must match the CSS font-weight | -| `fontFamily` | `"Outfit"` | Must match the CSS font-family | -| `step` | `2` | Decrement step in px per iteration | - -`fontWeight` and `fontFamily` must match the CSS applied to the text elements exactly, or measurements will be inaccurate. - -**Scale headroom for emphasis words:** If per-word styling scales words above 1.0x (e.g., `scale: 1.3` on "GOLDEN"), the scaled word occupies more width than `fitTextFontSize` measured. Reduce `maxWidth` to compensate: `maxWidth = safeWidth / maxScale`. For example, with max scale 1.3x on a 1920px composition with 80px margins: `maxWidth = 1760 / 1.3 ≈ 1350`. - -**Safety nets (still required in CSS):** - -- `max-width` on caption container (reduced from composition width to account for emphasis scale) -- `overflow: visible` — **not** `overflow: hidden`. Hidden clips scaled emphasis words and their glow effects. Rely on `fitTextFontSize` with reduced `maxWidth` instead. -- `position: absolute` on all caption elements -- Explicit `height` on caption container (e.g., `200px`) - -## Caption Exit Guarantee - -Captions that stick on screen are the most common caption bug. Every caption group **must** have a hard kill after its exit animation. - -```js -// Animate exit (soft — can fail if tweens conflict) -tl.to(groupEl, { opacity: 0, scale: 0.95, duration: 0.12, ease: "power2.in" }, group.end - 0.12); - -// Hard kill at group.end (deterministic — guarantees invisible) -tl.set(groupEl, { opacity: 0, visibility: "hidden" }, group.end); -``` - -**Why both?** The `tl.to` exit can fail to fully hide a group when karaoke word-level tweens conflict with the parent exit tween, `fromTo` entrance tweens lock values that override later tweens, or timeline scrubbing lands between the exit start and end. The `tl.set` at `group.end` is a deterministic kill — it fires at an exact time, doesn't animate, and can't be overridden. - -**Self-lint rule:** After building the timeline, verify every caption group has a hard kill: - -```js -GROUPS.forEach(function (group, gi) { - var el = document.getElementById("cg-" + gi); - if (!el) return; - tl.seek(group.end + 0.01); - var computed = window.getComputedStyle(el); - if (computed.opacity !== "0" && computed.visibility !== "hidden") { - console.warn( - "[caption-lint] group " + gi + " still visible at t=" + (group.end + 0.01).toFixed(2) + "s", - ); - } -}); -tl.seek(0); -``` - -Place this **before** `window.__timelines[id] = tl` so it runs at composition init. - -## References - -For dynamic animation techniques (karaoke, clip-path reveals, slam words, scatter exits, elastic entrances, 3D rotation, audio-reactive captions, pretext-based positioning and grouping), see [dynamic-techniques.md](./dynamic-techniques.md). - -For animated text emphasis (highlight sweeps, hand-drawn circles, burst lines, scribble underlines, sketchout effects) that pairs with per-word styling, see the `/marker-highlight` skill. - -For transcription commands, whisper models, external APIs, and troubleshooting, see [transcript-guide.md](./transcript-guide.md). - -## Constraints - -- **Deterministic.** No `Math.random()`, no `Date.now()`. -- **Sync to transcript timestamps.** Words appear when spoken. -- **One group visible at a time.** No overlapping caption groups. -- **Every caption group must have a hard `tl.set` kill at `group.end`.** Exit animations alone are not sufficient. -- **Check project root** for font files before defaulting to Google Fonts. diff --git a/skills/hyperframes-cli/SKILL.md b/skills/hyperframes-cli/SKILL.md index d68ed903c..414fd3ba9 100644 --- a/skills/hyperframes-cli/SKILL.md +++ b/skills/hyperframes-cli/SKILL.md @@ -1,31 +1,23 @@ --- name: hyperframes-cli -description: Preview, render, lint, validate, scaffold, or troubleshoot HyperFrames compositions. Also use after finishing a composition — lint and preview are the natural next steps. +description: HyperFrames CLI tool — hyperframes init, lint, preview, render, transcribe, tts, doctor, browser, info, upgrade, compositions, docs, benchmark. Use when scaffolding a project, linting or validating compositions, previewing in the studio, rendering to video, transcribing audio, generating TTS, or troubleshooting the HyperFrames environment. --- # HyperFrames CLI -The CLI turns HTML compositions into previews and rendered video. Everything runs through `npx hyperframes`. - -```bash -npx hyperframes -``` - -Requires Node.js >= 22 and FFmpeg. Run `npx hyperframes doctor` if anything fails. +Everything runs through `npx hyperframes`. Requires Node.js >= 22 and FFmpeg. ## Workflow -The natural sequence when building a composition: +1. **Scaffold** — `npx hyperframes init my-video` +2. **Write** — author HTML composition (see the `hyperframes` skill) +3. **Lint** — `npx hyperframes lint` +4. **Preview** — `npx hyperframes preview` +5. **Render** — `npx hyperframes render` -1. **Scaffold** — `npx hyperframes init my-video` (new projects only) -2. **Write** — author HTML composition (see `compose-video` skill) -3. **Lint** — `npx hyperframes lint` to catch structural errors -4. **Preview** — `npx hyperframes preview` to see it live in the studio -5. **Render** — `npx hyperframes render` to export video +Lint before preview — catches missing `data-composition-id`, overlapping tracks, unregistered timelines. -**Lint before preview.** It catches missing `data-composition-id`, overlapping tracks on the same `data-track-index`, unregistered timelines, and other structural issues that silently produce broken output. A 2-second lint saves minutes debugging a blank screen. Both `preview` and `render` auto-lint, but linting explicitly after editing gives you a chance to fix issues without waiting for the server or renderer to spin up. - -## Scaffolding New Projects +## Scaffolding ```bash npx hyperframes init my-video # interactive wizard @@ -37,83 +29,86 @@ npx hyperframes init my-video --non-interactive # skip prompts (CI/agents) Templates: `blank`, `warm-grain`, `play-mode`, `swiss-grid`, `vignelli`, `decision-tree`, `kinetic-type`, `product-promo`, `nyt-graph`. -`init` creates the right file structure, copies media, transcribes audio with Whisper, and installs AI coding skills. Use it instead of creating files by hand — the template includes boilerplate that's easy to forget. +`init` creates the right file structure, copies media, transcribes audio with Whisper, and installs AI coding skills. Use it instead of creating files by hand. ## Linting ```bash npx hyperframes lint # current directory npx hyperframes lint ./my-project # specific project -npx hyperframes lint --verbose # include info-level findings -npx hyperframes lint --json # machine-readable output for scripting +npx hyperframes lint --verbose # info-level findings +npx hyperframes lint --json # machine-readable ``` Lints `index.html` and all files in `compositions/`. Reports errors (must fix), warnings (should fix), and info (with `--verbose`). -**When to lint:** - -- After writing or editing any composition file — always -- Before rendering — `render` blocks on errors with `--strict`, but linting first is faster -- After timing changes — overlapping clips on the same track are a common mistake - -## Previewing in the Studio +## Previewing ```bash npx hyperframes preview # serve current directory -npx hyperframes preview ./my-project # specific project npx hyperframes preview --port 4567 # custom port (default 3002) ``` -Opens the studio in your browser automatically. Hot-reloads on file changes. Run from the project root (directory containing `index.html`). +Hot-reloads on file changes. Opens the studio in your browser automatically. -## Rendering to Video +## Rendering ```bash -npx hyperframes render # standard MP4 -npx hyperframes render --output final.mp4 # named output -npx hyperframes render --quality draft # fast iteration -npx hyperframes render --fps 60 --quality high -o hd.mp4 # high quality -npx hyperframes render --format webm -o overlay.webm # transparent WebM -npx hyperframes render --docker -o deterministic.mp4 # reproducible +npx hyperframes render # standard MP4 +npx hyperframes render --output final.mp4 # named output +npx hyperframes render --quality draft # fast iteration +npx hyperframes render --fps 60 --quality high # final delivery +npx hyperframes render --format webm # transparent WebM +npx hyperframes render --docker # byte-identical ``` -| Flag | Options | Default | Notes | -| -------------- | --------------------- | ---------------------------- | ------------------------------------- | -| `--output` | path | renders/name_timestamp.mp4 | Output file path | -| `--fps` | 24, 30, 60 | 30 | 60fps doubles render time | -| `--quality` | draft, standard, high | standard | Use draft while iterating | -| `--format` | mp4, webm | mp4 | WebM supports transparency | -| `--workers` | 1-8 or auto | auto (half CPU cores, max 4) | Each spawns a Chrome process | -| `--docker` | flag | off | Byte-identical output across machines | -| `--gpu` | flag | off | GPU-accelerated encoding | -| `--strict` | flag | off | Fail on lint errors | -| `--strict-all` | flag | off | Fail on errors AND warnings | +| Flag | Options | Default | Notes | +| -------------- | --------------------- | -------------------------- | --------------------------- | +| `--output` | path | renders/name_timestamp.mp4 | Output path | +| `--fps` | 24, 30, 60 | 30 | 60fps doubles render time | +| `--quality` | draft, standard, high | standard | draft for iterating | +| `--format` | mp4, webm | mp4 | WebM supports transparency | +| `--workers` | 1-8 or auto | auto | Each spawns Chrome | +| `--docker` | flag | off | Reproducible output | +| `--gpu` | flag | off | GPU-accelerated encoding | +| `--strict` | flag | off | Fail on lint errors | +| `--strict-all` | flag | off | Fail on errors AND warnings | -**Quality guidance:** +**Quality guidance:** `draft` while iterating, `standard` for review, `high` for final delivery. -- `draft` while iterating on timing and layout — fast feedback -- `standard` for review and most deliverables -- `high` only for final delivery where render time doesn't matter +## Transcription + +```bash +npx hyperframes transcribe audio.mp3 +npx hyperframes transcribe video.mp4 --model medium.en --language en +npx hyperframes transcribe subtitles.srt # import existing +npx hyperframes transcribe subtitles.vtt +npx hyperframes transcribe openai-response.json +``` + +## Text-to-Speech + +```bash +npx hyperframes tts "Text here" --voice af_nova --output narration.wav +npx hyperframes tts script.txt --voice bf_emma +npx hyperframes tts --list # show all voices +``` ## Troubleshooting ```bash -npx hyperframes doctor # check environment (Chrome, FFmpeg, Node, memory, disk) -npx hyperframes browser # manage bundled Chrome installation +npx hyperframes doctor # check environment (Chrome, FFmpeg, Node, memory) +npx hyperframes browser # manage bundled Chrome npx hyperframes info # version and environment details npx hyperframes upgrade # check for updates ``` -Run `doctor` first if rendering fails or produces unexpected results. Common issues: +Run `doctor` first if rendering fails. Common issues: missing FFmpeg, missing Chrome, low memory. -- Missing FFmpeg → `brew install ffmpeg` -- Missing Chrome → `npx hyperframes browser ensure` -- Low memory → close other apps (each render worker uses ~256MB) - -## Other Commands +## Other ```bash -npx hyperframes compositions # list compositions in current project -npx hyperframes docs # open documentation in browser +npx hyperframes compositions # list compositions in project +npx hyperframes docs # open documentation npx hyperframes benchmark . # benchmark render performance ``` diff --git a/skills/hyperframes-tts/SKILL.md b/skills/hyperframes-tts/SKILL.md deleted file mode 100644 index e8a8cda08..000000000 --- a/skills/hyperframes-tts/SKILL.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -name: hyperframes-tts -description: Generate speech audio locally using Kokoro-82M (no API key). Use when asked to create narration, voiceover, or text-to-speech audio for compositions, or when a user needs spoken audio from text. Covers voice selection, speed tuning, and integrating TTS output with compositions and captions. ---- - -# Text-to-Speech - -## Voice Selection - -Match voice to content. Default is `af_heart`. - -| Content type | Voice | Why | -| ----------------- | --------------------- | ----------------------------- | -| Product demo | `af_heart`/`af_nova` | Warm, professional | -| Tutorial / how-to | `am_adam`/`bf_emma` | Neutral, easy to follow | -| Marketing / promo | `af_sky`/`am_michael` | Energetic or authoritative | -| Documentation | `bf_emma`/`bm_george` | Clear British English, formal | -| Casual / social | `af_heart`/`af_sky` | Approachable, natural | - -Run `npx hyperframes tts --list` for all 54 voices (8 languages: EN, JP, ZH, KO, FR, DE, IT, PT). - -## Speed Tuning - -- **0.7-0.8** — Tutorial, complex content, accessibility -- **1.0** — Natural pace (default) -- **1.1-1.2** — Intros, transitions, upbeat content -- **1.5+** — Rarely appropriate; test carefully - -## Composing with TTS Audio - -Generate a voiceover and use it as the audio track: - -```bash -npx hyperframes tts "Your script here" --voice af_nova --output narration.wav -``` - -Then reference it in the composition as a standard `