mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
docs(skill): shader/audio/render/snapshot guidance from regression evidence
Adds decision guidance and conventions the 8-site regression test exposed as blind spots — agents had the capability but never reached for it. step-1-capture: - Clarify capture goes into <project-dir>/capture/ so capture artifacts stay isolated from later build files (SCRIPT/STORYBOARD/DESIGN/compositions/) - 7/8 regression tests already did this; codify as the convention step-4-storyboard: - Add "When to pick which" decision table for shader vs CSS vs hard cut transitions. Shader transitions were available but used in 0/8 tests — every test defaulted to CSS. The table frames shaders as "reveals, reaction shots, brand moments" vs CSS as "connective tissue" - Update technique count (10 → 11) step-6-build: - Mid-scene activity table gets a new row for audio-reactive logo/CTA animation (bass pulse, treble glow). Audio-reactive was used in 0/8 tests despite narration being present in all of them step-7-validate: - Snapshot section: explicit "use hyperframes snapshot, don't roll custom" with the default naming pattern spelled out. Stripe's run used custom ffmpeg naming (beat-6-cta-at-20.5s.png) instead of frame-XX-at-Ys.png - New render section: require --output renders/<project>.mp4 so final MP4s have predictable names. Without this, 7/8 tests produced wildly different filenames (preview.mp4, cal_2026-04-19_20-29-21.mp4, basecamp.mp4, etc.) techniques.md: - New technique #11: Audio-Reactive Animation. Covers the sampling pattern (per-frame tl.call, not single tween), when to use (music/dramatic VO videos), intensity ranges (3-5% for text/logos, 10-30% for backgrounds), and anti-patterns (equalizer bars, waveforms, strobing). Cross-references skills/hyperframes/references/audio-reactive.md for the full API Made-with: Cursor
This commit is contained in:
ukimsanov
@@ -2,10 +2,18 @@
|
||||
|
||||
## Run the capture
|
||||
|
||||
Create a project directory for your video, then capture the website into a `capture/` subfolder within it:
|
||||
|
||||
```bash
|
||||
npx hyperframes capture <URL> -o captures/<project-name>
|
||||
npx hyperframes capture <URL> -o <project-dir>/capture
|
||||
```
|
||||
|
||||
Example: `npx hyperframes capture https://stripe.com -o videos/stripe-launch/capture`
|
||||
|
||||
Keeping the capture artifacts (`screenshots/`, `assets/`, `extracted/`, `AGENTS.md`, `CLAUDE.md`) in a dedicated `capture/` subfolder keeps them isolated from the later build files (`SCRIPT.md`, `STORYBOARD.md`, `DESIGN.md`, `compositions/`, `index.html`, `narration.wav`, `transcript.json`, `renders/`, `snapshots/`), which all live at `<project-dir>/` root.
|
||||
|
||||
For exploratory captures that aren't becoming a video yet, `-o captures/<name>` at the repo root is fine — the isolation convention only matters when you're building a video on top of the capture.
|
||||
|
||||
No API keys required. The capture extracts design tokens, screenshots, fonts, and assets with DOM-context descriptions automatically.
|
||||
|
||||
**Optional:** Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) in a `.env` file at the repo root for richer AI-powered image descriptions via Gemini 3.1 Flash Lite vision (~$0.001/image).
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
- **DESIGN.md** — your color palette, font rules, components, Do's/Don'ts. Every creative decision must be grounded in this brand identity. If it says "white backgrounds with purple accent" — plan light scenes, not dark moody ones.
|
||||
- **`extracted/asset-descriptions.md`** — read EVERY line. This is your menu of available visuals. Each line describes what the image actually shows (e.g., "translucent ribbons in orange, pink, and purple on white background" or "a high-speed train under a dark starry sky"). Use these descriptions to decide which assets belong in which beat. Assets you don't understand from the description — view them directly before assigning.
|
||||
- **[techniques.md](techniques.md)** — 10 visual techniques (SVG path drawing, Canvas 2D art, CSS 3D, per-word typography, Lottie, video compositing, typing effect, variable fonts, MotionPath, velocity transitions). Pick 2-3 per beat and specify them in the storyboard.
|
||||
- **[techniques.md](techniques.md)** — 11 visual techniques (SVG path drawing, Canvas 2D art, CSS 3D, per-word typography, Lottie, video compositing, typing effect, variable fonts, MotionPath, velocity transitions, audio-reactive). Pick 2-3 per beat and specify them in the storyboard.
|
||||
|
||||
The storyboard is the creative north star. It tells the engineer exactly what to build for each beat — mood, camera, animations, transitions, assets, sound. Write it as if you're briefing a motion designer who's never seen the website.
|
||||
|
||||
@@ -116,6 +116,16 @@ Every element gets a verb. If you can't name the verb, the element is not yet de
|
||||
|
||||
How this beat hands off to the next. Specify the type and parameters.
|
||||
|
||||
**When to pick which:**
|
||||
|
||||
| Choose shader transition for | Choose CSS transition for | Choose hard cut for |
|
||||
| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------- |
|
||||
| Reveals, big reaction shots, product/logo unveils, energy shifts, "wow" moments | Continuous camera-motion beats where the scene feels like one move broken into cuts | Rapid-fire lists, percussive edits on the beat, comedic timing |
|
||||
| Any moment the music/VO punctuates with a downbeat or SFX hit | Beats that ease from one composition into the next with shared motion vocabulary | Sequences of 3+ quick tempo-matched switches |
|
||||
| Brand moments where the transition itself _is_ the visual | Minimal/editorial pacing | Anytime a 0.3-0.8s transition would feel too slow |
|
||||
|
||||
Rule of thumb: if the beat is the _centerpiece_ of the video, shader-transition into it. If the beat is connective tissue, CSS-transition. A brand reel of 5-7 beats usually wants 1-2 shader transitions (the hero reveal + the CTA) and the rest CSS or hard cuts — too many shader transitions flatten their impact.
|
||||
|
||||
**CSS transitions** (choose from `skills/hyperframes/references/transitions/catalog.md`):
|
||||
|
||||
- Velocity-matched upward: exit `y:-150, blur:30px, 0.33s power2.in` → entry `y:150→0, blur:30px→0, 1.0s power2.out`
|
||||
|
||||
@@ -70,12 +70,13 @@ Use `gsap.from()` — animate FROM offscreen/invisible TO the CSS position. The
|
||||
|
||||
Every visible element must have continuous motion. A still image on a still background is a JPEG with a progress bar.
|
||||
|
||||
| Element type | Mid-scene activity |
|
||||
| ---------------------- | ------------------------------------------------ |
|
||||
| Image / screenshot | Slow zoom (scale 1→1.03), slow pan, or Ken Burns |
|
||||
| Stat / number | Counter animates from 0 to target |
|
||||
| Logo grid | Subtle shimmer sweep, or gentle scale pulse |
|
||||
| Any persistent element | Subtle float (y ±4-6px, sine.inOut, yoyo) |
|
||||
| Element type | Mid-scene activity |
|
||||
| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Image / screenshot | Slow zoom (scale 1→1.03), slow pan, or Ken Burns |
|
||||
| Stat / number | Counter animates from 0 to target |
|
||||
| Logo grid | Subtle shimmer sweep, or gentle scale pulse |
|
||||
| Any persistent element | Subtle float (y ±4-6px, sine.inOut, yoyo) |
|
||||
| Logo / CTA (with music or dramatic VO) | Audio-reactive scale/glow — bass pulses the logo (3–4%), treble glows the CTA. See technique #11 in `techniques.md` for the sampling pattern |
|
||||
|
||||
### 6. Add exit / transition
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ npx hyperframes validate
|
||||
|
||||
## Visual Verification (snapshot)
|
||||
|
||||
After lint and validate pass, capture snapshot frames to SEE your own output:
|
||||
After lint and validate pass, capture snapshot frames to SEE your own output. **Always use `hyperframes snapshot`** — do not roll your own ffmpeg/headless Chrome script; the default naming (`frame-XX-at-Ys.png`) is expected by later tooling.
|
||||
|
||||
```bash
|
||||
npx hyperframes snapshot <project-dir> --at <beat-midpoints>
|
||||
@@ -32,7 +32,7 @@ Calculate the midpoint of each beat from your STORYBOARD.md timings. For a 4-bea
|
||||
npx hyperframes snapshot <project-dir> --at 2.9,10.4,18.7,23.9
|
||||
```
|
||||
|
||||
This renders one frame per beat at the moment when content is most visible. Use timestamps where the most content is on screen — usually 60-70% into each beat, after entrances finish but before exits start.
|
||||
This renders one frame per beat at the moment when content is most visible. Use timestamps where the most content is on screen — usually 60-70% into each beat, after entrances finish but before exits start. Output lands in `<project-dir>/snapshots/` with filenames like `frame-00-at-2.9s.png`.
|
||||
|
||||
**View every snapshot image carefully.** Don't glance and move on. For each frame, check:
|
||||
|
||||
@@ -70,3 +70,17 @@ npx hyperframes preview
|
||||
```
|
||||
|
||||
Open the studio in a browser. Scrub through every beat.
|
||||
|
||||
## Render
|
||||
|
||||
Once preview looks correct, render to MP4. **Always pass `--output renders/<project-name>.mp4`** so the final file has a predictable name — do not rely on the timestamped default (`<project>_YYYY-MM-DD_HH-MM-SS.mp4`), which makes files hard to reference from docs/handoffs/CI.
|
||||
|
||||
```bash
|
||||
npx hyperframes render --output renders/<project-name>.mp4
|
||||
```
|
||||
|
||||
Example: `npx hyperframes render --output renders/stripe-launch.mp4`
|
||||
|
||||
For social-media vertical output, add `--format` or `--viewport 1080x1920` (depending on CLI flag support); check `npx hyperframes render --help`.
|
||||
|
||||
If the final MP4 exceeds ~15 MB for a 30-second video, consider `--quality medium` or `--crf 23` to compress — but verify the quality trade-off on a snapshot pass before accepting.
|
||||
|
||||
@@ -338,6 +338,44 @@ The fastest point of both curves meets at the cut — the viewer perceives smoot
|
||||
|
||||
---
|
||||
|
||||
## 11. Audio-Reactive Animation
|
||||
|
||||
Drive any GSAP-tweenable property from the playing audio. Bass pulses a logo on kick drums. Treble glows a CTA on cymbals. Amplitude breathes a background during quiet phrases. The result: motion that feels locked to the track in a way pre-authored tweens never can.
|
||||
|
||||
**When to use:** Any video with music or dramatic narration — brand reels, product launches, hype edits. Skip for calm/tutorial pacing.
|
||||
|
||||
**How it works:** Pre-extract audio frequency bands into a JSON file, then sample per-frame via `tl.call()`:
|
||||
|
||||
```js
|
||||
// audio-data.json: { fps: 30, totalFrames: 900, frames: [{ bands: [0.82, 0.45, 0.31, ...] }, ...] }
|
||||
for (var f = 0; f < AUDIO_DATA.totalFrames; f++) {
|
||||
tl.call(
|
||||
(function (frame) {
|
||||
return function () {
|
||||
var bass = frame.bands[0]; // 0–1
|
||||
var treble = frame.bands[13];
|
||||
gsap.set(".logo", { scale: 1 + bass * 0.04 }); // 3–4% pulse on bass
|
||||
gsap.set(".cta", { filter: `drop-shadow(0 0 ${treble * 24}px #00C3FF)` });
|
||||
};
|
||||
})(AUDIO_DATA.frames[f]),
|
||||
[],
|
||||
f / AUDIO_DATA.fps,
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Per-frame sampling is required — a single tween will not react. Use the extract script:
|
||||
|
||||
```bash
|
||||
python3 skills/hyperframes/scripts/extract-audio-data.py narration.wav --fps 30 --bands 16 -o audio-data.json
|
||||
```
|
||||
|
||||
Keep text/logo intensity subtle (≤5% scale, ≤30% glow) — audio-reactive motion on tiny elements reads as jitter. Bigger backgrounds can push to 10–30%.
|
||||
|
||||
**Never do:** equalizer bars, spectrum analyzers, waveform displays, strobing, rainbow color cycling. The audio provides _timing and intensity_; the visual vocabulary still comes from the brand. See `skills/hyperframes/references/audio-reactive.md` for the full API and anti-patterns.
|
||||
|
||||
---
|
||||
|
||||
## When to Use What
|
||||
|
||||
| Video energy | Techniques to combine |
|
||||
|
||||
Reference in New Issue
Block a user