- Reduce poll interval from 500ms to 200ms for faster detection
- Fall back to DOM querySelector for iframe when ref isn't set yet
- Runtime wrapper sets data-caption-wrapper for consistent detection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Runtime hook's wrapper was missing data-caption-wrapper attribute, so
the overlay created a second wrapper on top. Transforms were applied to
different wrappers causing values to compound on reload.
Also:
- Suppress auto-save during loadOverrides to prevent immediate re-write
- Only sync non-default transform values to avoid spurious overrides
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove store reset from useEffect cleanup — was wiping state on dep changes
and breaking auto-save subscription
- Move panel auto-open into useEffect instead of during render
- Add fallback 500ms poll alongside postMessage listener for initial load
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace polling with a message listener for hf-preview state/timeline
events. These fire after loadExternalCompositions completes, guaranteeing
.caption-group elements exist in the DOM. Eliminates the race condition
on initial page load.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Caption edit mode now activates automatically when .caption-group elements
are found in the iframe DOM. Works from index.html (master) without
requiring drill-down into the captions sub-composition. Finds the captions
source file via data-composition-src attribute.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Caption overlay and hover states always active on captions compositions
- Clicking a word auto-opens the property panel
- Remove Edit Captions button — edit mode activates automatically
- Reset edit mode when navigating away from captions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove typography, color, opacity, stroke, and background controls.
Keep only Position (X/Y) and Transform (Scale/Rotation) — the controls
that work reliably with the wrapper-based override system.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace single "color" with activeColor (spoken) and dimColor (inactive).
Runtime hook modifies existing GSAP color tweens by detecting dim vs active
states via opacity heuristic. Property panel shows both color pickers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
No longer needed — caption overrides are a JSON data file, not HTML patches.
The runtime hook re-applies overrides on every reload, so preview refresh
during editing is safe and ensures the preview stays in sync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace manual Save button with 800ms debounced auto-save. Writing to
caption-overrides.json is safe because preview refresh is suppressed
during caption edit mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- getWordEl skips wrapper spans and finds actual word elements inside them
- readWordBoxes handles wrapped words in position calculations
- getOrCreateWrapper detects if element IS a wrapper to prevent double-wrapping
- Remove debug logging from save
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- applyCaptionOverrides() now runs both when compositions load async AND when
already ready (standalone composition view)
- Overrides file written to project root (not compositions/) to match <base href>
- Remove debug logging
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Instead of killing transform tweens, wrap each overridden word span in an
inline-block wrapper span. Editor transforms (position, scale, rotation)
are applied to the wrapper while the inner span keeps all GSAP entrance,
exit, and karaoke animations intact.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add optional `id` field to transcript Word interface, assigned as `w{index}` during normalization
- caption-overrides.json uses `wordId` (preferred) with `wordIndex` fallback
- Runtime hook calls gsap.killTweensOf for overridden transform properties before applying
- Studio parser carries wordId through from transcript to segments
- Add typecheck to lefthook pre-commit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New caption-overrides.json format: [{wordIndex, x, y, scale, rotation, color, ...}]
- Runtime hook in packages/core applies overrides via gsap.set() after compositions load
- Studio writes/reads caption-overrides.json instead of injecting code into HTML
- Original composition source is never modified
- Remove old parseExistingOverrides and source patching code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Canvas: move (translate), scale, and rotate handles on selected words via gsap.set()
- Property panel: real-time style updates applied directly to iframe DOM
- Save: patches original source with gsap.killTweensOf + gsap.set per edited word
- Parser: reads existing editor overrides on re-entry to preserve prior edits
- Suppress preview refresh during caption edit mode to prevent reload cycle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements generateCaptionHtml(model) which serializes a CaptionModel
into a complete captions.html HyperFrames composition with template
wrapper, CSS per-group classes, TRANSCRIPT array, and GSAP timeline
registration via window.__timelines["captions"].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds `buildCaptionModel` to the caption parser — groups TranscriptWord
arrays into CaptionGroups (default 5 words each), creates CaptionSegments
with correct timing and groupIndex, and wires in DEFAULT_STYLE,
DEFAULT_CONTAINER, and DEFAULT_ANIMATION_SET. Covers 23 tests (13 new).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ConsoleMessage.type() returns "warn" not "warning"
- Type pageerror callback parameter as Error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Loads a composition in headless Chrome and reports console errors,
uncaught exceptions, and failed network requests for project assets.
Agents run this before opening the studio to catch runtime issues.
- Bundle HTML with inlined runtime (no CDN dependency)
- Minimal file server for project assets
- Captures JS errors, pageerror events, 404s on project files
- JSON output mode for programmatic consumption
- ~3s execution time, suitable as a blocking pre-preview step
Updates CLAUDE.md and compose skill to require lint + validate
before opening the studio.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Vite dev server was loading the HyperFrames runtime IIFE from
cdn.jsdelivr.net. This caused compositions using window.__hyperframes
(e.g., fitTextFontSize) to fail when the CDN script didn't load in
the iframe context.
Add a /api/runtime.js endpoint to the Vite plugin that serves the
local built runtime, matching what studioServer.ts already does for
the CLI embedded server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove installSkills(), getBundledSkillsDir(), project-level skill copying,
and the --skip-skills flag. Replace post-scaffold messaging with directions
to run `npx skills add heygen-com/hyperframes`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove custom skill installation from the CLI in favor of
vercel-labs/skills. Fixes staleness, deduplication, listing budget,
and completeness issues identified by agentic harness pattern review.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
**Whisper improvements:**
- Auto-detect language and switch from `.en` to multilingual model when needed
- Detect speech onset in WAV to strip hallucinated words before speech begins
- Merge whisper-cpp token fragments: contractions (`didn` + `'t` → `didn't`), split capitals (`C` + `aught` → `Caught`), dropped-g (`shin` + `in'` → `shinin'`)
- Interpolate zero-duration word clusters for reliable karaoke timing
**Captions skill updates (folded from #176):**
- Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits
- Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time
- Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model)
- Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility
**Multilingual defaults (folded from #186):**
- Default whisper model changed from `small.en` to `small` to prevent silent translation of non-English audio
- Added non-negotiable language rule to captions skill
## Test plan
- [ ] `pnpm test` passes (contraction merging, fragment merging, zero-duration interpolation, speech onset)
- [ ] Transcribe non-English audio — verify it transcribes in original language, not translates
- [ ] Skill files render correctly, cross-references resolve
- [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
- Split the captions skill from a single 611-line file into focused references: `SKILL.md` (core rules), `transcript-guide.md` (whisper/transcription), `dynamic-techniques.md` (animation patterns)
- Add `audio-reactive` skill with "Content, Not Medium" constraint — steers away from generic visualizations (equalizer bars, spectrum analyzers, waveforms) toward content-grounded animation where audio drives *when* and *how much*, not *what to show*
- Add initial dynamic caption technique selection by energy level
## Test plan
- [ ] All skill files render correctly as markdown
- [ ] Cross-references between files use correct relative paths
- [ ] `audio-reactive/SKILL.md` contains the anti-pattern list and content-grounded examples
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Context
`npx hyperframes render` fails to load assets and produces all-black video. Root causes:
1. CDN scripts (GSAP, Lottie) are left as `<script src="https://...">` in compiled HTML — the headless browser must fetch them over the network, which fails in Docker, CI, and firewalled environments
2. Assets referenced from outside the project directory (e.g. `../shared-assets/hero.png`) 404 because the file server only serves from `projectDir`
3. When GSAP fails to load, the timeline never registers, all `.clip` elements stay `visibility: hidden`, and every frame is black — with no diagnostic output
4. The linter reports `missing_gsap_script` when GSAP is bundled inline (no `<script src>` tag), which blocks users from working around the CDN issue
## What changed
### 1\. CDN script inlining (`htmlCompiler.ts`)
`compileForRender` now downloads all external `<script src="https://...">` tags at compile time and inlines their content into the HTML. Rendering no longer needs network access.
| Before | After |
| --- | --- |
| `<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>` stays in HTML, browser must fetch at runtime | Script downloaded during compilation, embedded as `<script>/* inlined: https://... */\n...code...</script>` |
| Docker/CI render → `net::ERR_NAME_NOT_RESOLVED` → black video | Render works fully offline |
| No feedback when CDN fails | `[Compiler] WARNING: Failed to download CDN script: ... Consider bundling it locally` |
### 2\. External asset copying (`htmlCompiler.ts`, `renderOrchestrator.ts`)
After compilation, the HTML is scanned for `src`, `href`, and CSS `url()` references that resolve outside `projectDir`. These files are copied into the compiled output directory so the file server can serve them.
| Before | After |
| --- | --- |
| `background-image: url(../shared-assets/hero.png)` → 404 (file server can't serve outside `projectDir`) | Asset detected, copied to compiled dir, path rewritten → serves correctly |
| `<img src="../shared-assets/logo.png">` → 404 | Same fix — works for all `src`/`href` attributes and CSS `url()` |
### 3\. Black video diagnostics (`renderOrchestrator.ts`)
When composition duration is 0 (which would produce a black video), the error now probes the browser for diagnostics instead of a generic message.
| Before | After |
| --- | --- |
| `Invalid composition duration: 0. Check that GSAP timelines are registered.` | `Composition duration is 0 — this would produce a black video.\n\nDiagnostics:\n - GSAP is not loaded — CDN script may have failed to download. Bundle GSAP locally...\n - Browser: [Browser:PAGEERROR] gsap is not defined` |
| Asset 404s during page load silently logged | `[Render] Asset load failure: ...` + `[WARN] Browser encountered network failures during page load` |
### 4\. Linter: recognize inline GSAP (`core/lint/rules/gsap.ts`)
The `missing_gsap_script` rule now recognizes GSAP bundled inline — matching the producer's inlining comment (`/* inlined: ...gsap... */`), GSAP library internals (`_gsScope`, `GreenSock`), and large inline scripts (>5KB) referencing gsap.
| Before | After |
| --- | --- |
| User inlines GSAP → linter errors with `missing_gsap_script` | Inline GSAP detected, no false error |
| Producer inlines CDN → linter errors on the compiled HTML | Producer's `/* inlined: ... */` comment recognized |
## Test plan
- [x] `pnpm build` passes
- [x] Core tests pass (410/410, +2 new)
- [x] **Reproduced baseline failures on** **`main`**: CDN scripts not inlined, external assets 404, no diagnostics
- [x] **Verified fixes**: CDN script inlined, external assets copied and served, diagnostics printed
- [x] Render with working CDN → `[Compiler] Inlined CDN script: ...` → render succeeds
- [x] Render with broken CDN → `[Compiler] WARNING: Failed to download CDN script` + browser errors surfaced
- [x] Render with assets outside project dir → `[Compiler] Found 1 asset(s) outside project directory` → assets served correctly
- [x] Linter with inline GSAP → no `missing_gsap_script` false positive
