Commit Graph
3 Commits
Author SHA1 Message Date
Vance Ingalls cb0b17062a feat(skills): add marker-highlight skill for animated text highlighting (#190)
## Summary

- **New skill:** **`marker-highlight`** — integrates [MarkerHighlight.js](https://github.com/Robincodes-Sandbox/marker-highlight) into HyperFrames compositions. Canvas-based animated text highlighting with 5 drawing modes: marker pen, circle, burst, scribble, and sketchout.
- **Studio fix:** added missing `captionSync` to useEffect dependency array (oxlint exhaustive-deps)
- **Studio fix:** `loadOverrides` now checks `res.ok` before parsing, preventing 404 console noise on projects without captions

## Skill details

The skill documents the non-obvious GSAP integration pattern discovered during development:

1. **One highlighter per container** — the library clears ALL `.highlight` divs from the shared parent on init, so multiple instances on sibling marks conflict
2. **`data-color`** **\+** **`data-original-bgcolor`** — prevents the CSS background-color flash that occurs when the library reads and clears the mark's background
3. **Canvas pre-draw + clear + reanimate** — `animate: false` pre-draws statically, canvases are hidden, then cleared and shown with `reanimateMark()` at trigger time for clean animated reveals
4. **`onReverseComplete`** **for rewind** — hides highlight divs when the timeline seeks backward past the trigger point

## Test plan

- [ ] `npx hyperframes lint` passes on test-composition
- [ ] Studio preview shows marker highlight on "something" at 1s, circle on "love" at 2.2s
- [ ] Rewind past trigger points hides highlights
- [ ] No 404 console errors for caption-overrides.json on non-caption projects



[Screen Recording 2026-04-02 at 1.56.30 AM.mov <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.com/user-attachments/thumbnails/53b03f4e-538e-477a-b738-7a033b99a84e.mov" />](https://app.graphite.com/user-attachments/video/53b03f4e-538e-477a-b738-7a033b99a84e.mov)



🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-02 13:48:24 -07:00
Vance Ingalls 37404f23da feat(whisper+captions): language detection, audio-reactive captions, multilingual defaults (#175)
## Summary

**Whisper improvements:**
- Auto-detect language and switch from `.en` to multilingual model when needed
- Detect speech onset in WAV to strip hallucinated words before speech begins
- Merge whisper-cpp token fragments: contractions (`didn` + `'t` → `didn't`), split capitals (`C` + `aught` → `Caught`), dropped-g (`shin` + `in'` → `shinin'`)
- Interpolate zero-duration word clusters for reliable karaoke timing

**Captions skill updates (folded from #176):**
- Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits
- Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time
- Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model)
- Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility

**Multilingual defaults (folded from #186):**
- Default whisper model changed from `small.en` to `small` to prevent silent translation of non-English audio
- Added non-negotiable language rule to captions skill

## Test plan

- [ ] `pnpm test` passes (contraction merging, fragment merging, zero-duration interpolation, speech onset)
- [ ] Transcribe non-English audio — verify it transcribes in original language, not translates
- [ ] Skill files render correctly, cross-references resolve
- [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-02 00:08:41 -07:00
Vance Ingalls 159a2e7113 feat(skills): add dynamic caption techniques and split captions skill into references (#173)
## Summary

- Split the captions skill from a single 611-line file into focused references: `SKILL.md` (core rules), `transcript-guide.md` (whisper/transcription), `dynamic-techniques.md` (animation patterns)
- Add `audio-reactive` skill with "Content, Not Medium" constraint — steers away from generic visualizations (equalizer bars, spectrum analyzers, waveforms) toward content-grounded animation where audio drives *when* and *how much*, not *what to show*
- Add initial dynamic caption technique selection by energy level

## Test plan

- [ ] All skill files render correctly as markdown
- [ ] Cross-references between files use correct relative paths
- [ ] `audio-reactive/SKILL.md` contains the anti-pattern list and content-grounded examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-01 23:24:25 -07:00