Instead of killing transform tweens, wrap each overridden word span in an
inline-block wrapper span. Editor transforms (position, scale, rotation)
are applied to the wrapper while the inner span keeps all GSAP entrance,
exit, and karaoke animations intact.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add optional `id` field to transcript Word interface, assigned as `w{index}` during normalization
- caption-overrides.json uses `wordId` (preferred) with `wordIndex` fallback
- Runtime hook calls gsap.killTweensOf for overridden transform properties before applying
- Studio parser carries wordId through from transcript to segments
- Add typecheck to lefthook pre-commit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New caption-overrides.json format: [{wordIndex, x, y, scale, rotation, color, ...}]
- Runtime hook in packages/core applies overrides via gsap.set() after compositions load
- Studio writes/reads caption-overrides.json instead of injecting code into HTML
- Original composition source is never modified
- Remove old parseExistingOverrides and source patching code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Canvas: move (translate), scale, and rotate handles on selected words via gsap.set()
- Property panel: real-time style updates applied directly to iframe DOM
- Save: patches original source with gsap.killTweensOf + gsap.set per edited word
- Parser: reads existing editor overrides on re-entry to preserve prior edits
- Suppress preview refresh during caption edit mode to prevent reload cycle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements generateCaptionHtml(model) which serializes a CaptionModel
into a complete captions.html HyperFrames composition with template
wrapper, CSS per-group classes, TRANSCRIPT array, and GSAP timeline
registration via window.__timelines["captions"].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds `buildCaptionModel` to the caption parser — groups TranscriptWord
arrays into CaptionGroups (default 5 words each), creates CaptionSegments
with correct timing and groupIndex, and wires in DEFAULT_STYLE,
DEFAULT_CONTAINER, and DEFAULT_ANIMATION_SET. Covers 23 tests (13 new).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ConsoleMessage.type() returns "warn" not "warning"
- Type pageerror callback parameter as Error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Loads a composition in headless Chrome and reports console errors,
uncaught exceptions, and failed network requests for project assets.
Agents run this before opening the studio to catch runtime issues.
- Bundle HTML with inlined runtime (no CDN dependency)
- Minimal file server for project assets
- Captures JS errors, pageerror events, 404s on project files
- JSON output mode for programmatic consumption
- ~3s execution time, suitable as a blocking pre-preview step
Updates CLAUDE.md and compose skill to require lint + validate
before opening the studio.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Vite dev server was loading the HyperFrames runtime IIFE from
cdn.jsdelivr.net. This caused compositions using window.__hyperframes
(e.g., fitTextFontSize) to fail when the CDN script didn't load in
the iframe context.
Add a /api/runtime.js endpoint to the Vite plugin that serves the
local built runtime, matching what studioServer.ts already does for
the CLI embedded server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove installSkills(), getBundledSkillsDir(), project-level skill copying,
and the --skip-skills flag. Replace post-scaffold messaging with directions
to run `npx skills add heygen-com/hyperframes`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove custom skill installation from the CLI in favor of
vercel-labs/skills. Fixes staleness, deduplication, listing budget,
and completeness issues identified by agentic harness pattern review.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
**Whisper improvements:**
- Auto-detect language and switch from `.en` to multilingual model when needed
- Detect speech onset in WAV to strip hallucinated words before speech begins
- Merge whisper-cpp token fragments: contractions (`didn` + `'t` → `didn't`), split capitals (`C` + `aught` → `Caught`), dropped-g (`shin` + `in'` → `shinin'`)
- Interpolate zero-duration word clusters for reliable karaoke timing
**Captions skill updates (folded from #176):**
- Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits
- Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time
- Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model)
- Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility
**Multilingual defaults (folded from #186):**
- Default whisper model changed from `small.en` to `small` to prevent silent translation of non-English audio
- Added non-negotiable language rule to captions skill
## Test plan
- [ ] `pnpm test` passes (contraction merging, fragment merging, zero-duration interpolation, speech onset)
- [ ] Transcribe non-English audio — verify it transcribes in original language, not translates
- [ ] Skill files render correctly, cross-references resolve
- [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
- Split the captions skill from a single 611-line file into focused references: `SKILL.md` (core rules), `transcript-guide.md` (whisper/transcription), `dynamic-techniques.md` (animation patterns)
- Add `audio-reactive` skill with "Content, Not Medium" constraint — steers away from generic visualizations (equalizer bars, spectrum analyzers, waveforms) toward content-grounded animation where audio drives *when* and *how much*, not *what to show*
- Add initial dynamic caption technique selection by energy level
## Test plan
- [ ] All skill files render correctly as markdown
- [ ] Cross-references between files use correct relative paths
- [ ] `audio-reactive/SKILL.md` contains the anti-pattern list and content-grounded examples
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Context
`npx hyperframes render` fails to load assets and produces all-black video. Root causes:
1. CDN scripts (GSAP, Lottie) are left as `<script src="https://...">` in compiled HTML — the headless browser must fetch them over the network, which fails in Docker, CI, and firewalled environments
2. Assets referenced from outside the project directory (e.g. `../shared-assets/hero.png`) 404 because the file server only serves from `projectDir`
3. When GSAP fails to load, the timeline never registers, all `.clip` elements stay `visibility: hidden`, and every frame is black — with no diagnostic output
4. The linter reports `missing_gsap_script` when GSAP is bundled inline (no `<script src>` tag), which blocks users from working around the CDN issue
## What changed
### 1\. CDN script inlining (`htmlCompiler.ts`)
`compileForRender` now downloads all external `<script src="https://...">` tags at compile time and inlines their content into the HTML. Rendering no longer needs network access.
| Before | After |
| --- | --- |
| `<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>` stays in HTML, browser must fetch at runtime | Script downloaded during compilation, embedded as `<script>/* inlined: https://... */\n...code...</script>` |
| Docker/CI render → `net::ERR_NAME_NOT_RESOLVED` → black video | Render works fully offline |
| No feedback when CDN fails | `[Compiler] WARNING: Failed to download CDN script: ... Consider bundling it locally` |
### 2\. External asset copying (`htmlCompiler.ts`, `renderOrchestrator.ts`)
After compilation, the HTML is scanned for `src`, `href`, and CSS `url()` references that resolve outside `projectDir`. These files are copied into the compiled output directory so the file server can serve them.
| Before | After |
| --- | --- |
| `background-image: url(../shared-assets/hero.png)` → 404 (file server can't serve outside `projectDir`) | Asset detected, copied to compiled dir, path rewritten → serves correctly |
| `<img src="../shared-assets/logo.png">` → 404 | Same fix — works for all `src`/`href` attributes and CSS `url()` |
### 3\. Black video diagnostics (`renderOrchestrator.ts`)
When composition duration is 0 (which would produce a black video), the error now probes the browser for diagnostics instead of a generic message.
| Before | After |
| --- | --- |
| `Invalid composition duration: 0. Check that GSAP timelines are registered.` | `Composition duration is 0 — this would produce a black video.\n\nDiagnostics:\n - GSAP is not loaded — CDN script may have failed to download. Bundle GSAP locally...\n - Browser: [Browser:PAGEERROR] gsap is not defined` |
| Asset 404s during page load silently logged | `[Render] Asset load failure: ...` + `[WARN] Browser encountered network failures during page load` |
### 4\. Linter: recognize inline GSAP (`core/lint/rules/gsap.ts`)
The `missing_gsap_script` rule now recognizes GSAP bundled inline — matching the producer's inlining comment (`/* inlined: ...gsap... */`), GSAP library internals (`_gsScope`, `GreenSock`), and large inline scripts (>5KB) referencing gsap.
| Before | After |
| --- | --- |
| User inlines GSAP → linter errors with `missing_gsap_script` | Inline GSAP detected, no false error |
| Producer inlines CDN → linter errors on the compiled HTML | Producer's `/* inlined: ... */` comment recognized |
## Test plan
- [x] `pnpm build` passes
- [x] Core tests pass (410/410, +2 new)
- [x] **Reproduced baseline failures on** **`main`**: CDN scripts not inlined, external assets 404, no diagnostics
- [x] **Verified fixes**: CDN script inlined, external assets copied and served, diagnostics printed
- [x] Render with working CDN → `[Compiler] Inlined CDN script: ...` → render succeeds
- [x] Render with broken CDN → `[Compiler] WARNING: Failed to download CDN script` + browser errors surfaced
- [x] Render with assets outside project dir → `[Compiler] Found 1 asset(s) outside project directory` → assets served correctly
- [x] Linter with inline GSAP → no `missing_gsap_script` false positive
* feat(cli): improve --help with grouped commands and per-command examples
Replaces citty's flat COMMANDS list with kubectl-style grouped categories
and adds examples to every subcommand. Root help now groups 14 commands
into 5 categories (Getting Started, Project, Tooling, AI & Integrations,
Settings). Per-command --help now appends a formatted Examples section
with practical usage patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(cli): use shared colors, lazy-load help, fix description drift
- Replace hand-rolled ANSI helpers with existing ui/colors.ts (fixes
non-conformant NO_COLOR handling)
- Add cyan and gray to shared color module
- Lazy-load help.ts via dynamic import to avoid allocating help data
on non-help invocations
- Fix description drift: benchmark and transcribe descriptions now
match their command meta.description
- Unify tuple order: ROOT_EXAMPLES now uses [comment, command] to
match COMMAND_EXAMPLES
- Remove redundant comment restating type annotation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): resolve typecheck error in showUsage generic signature
The lazy-load wrapper needs to cast CommandDef<T> to CommandDef when
forwarding to the help module, since TypeScript's generic variance
makes the direct assignment incompatible.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(cli): remove kubectl references from help comments
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting
## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.
## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`
## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
## What
Move the id-less media fix into the shared timing compiler so producer can resolve durations for sub-composition videos before inlining, then carry the merged result through engine parsing, regression coverage, and the regression Docker image used in CI.
This PR now does five concrete things:
- assigns stable ids to id-less media in core `compileTimingAttrs()` so unresolved duration injection can target them
- keeps the engine-side `parseVideoElements()` support for `video[src]` plus the newer `data-duration` / natural-duration fallback from `main`
- makes producer prefer sub-composition media metadata over the later inlined-document parse when the same media id appears in both places
- makes `sub-composition-video` a runnable regression test by fixing its metadata and checking in the missing `output/compiled.html` snapshot
- removes the stale `pnpm-workspace.yaml` copy step from `Dockerfile.test`, so regression CI builds the Bun-based test image from the current workspace layout
## Why
- media without an explicit `id` could not participate in unresolved-duration resolution early enough
- producer could lose the resolved sub-composition timing by overwriting it with the later inlined parse
- the regression fixture intended to cover this case was not actually running in CI because its `meta.json` was incomplete and the required compiled snapshot was missing
- the regression image definition still expected a deleted `pnpm-workspace.yaml`, so GitHub Actions failed before the test shard could start
Putting the id-generation step in core makes the behavior reusable instead of relying on producer-only HTML patching.
## How
### Shared compiler
- core `compileTimingAttrs()` now auto-assigns stable ids to id-less `video` / `audio` tags
- those generated ids are returned in `unresolved`, so `injectDurations()` can add `data-duration` and `data-end` to the same media element later in the pipeline
- added core tests that cover auto-id assignment and duration injection for generated ids
### Producer
- when producer combines `subVideos` / `subAudios` with the media re-parsed from the final inlined HTML, it now lets the sub-composition metadata win
- this preserves the resolved/clamped timing already computed for nested media instead of overwriting it with the later parse
- `sub-composition-video` now has valid regression metadata and a checked-in `output/compiled.html` snapshot so CI actually executes it
### Engine
- resolved the merge conflict in `videoFrameExtractor` by keeping the broader `video[src]` parsing from this branch and the `data-duration` / natural-duration fallback that landed on `main`
- added a focused engine unit test for videos without ids
### CI image
- `Dockerfile.test` now copies only `package.json` and `bun.lock` at the workspace root before `bun install --frozen-lockfile`
- this matches the current monorepo layout and removes the obsolete pnpm-era dependency on `pnpm-workspace.yaml`
## Test plan
- [x] `bun run --filter @hyperframes/core test`
- [x] `bun run --filter @hyperframes/engine test`
- [x] `bun run --filter @hyperframes/producer test --update --sequential sub-composition-video`
- [x] `bun run --filter @hyperframes/producer test --sequential sub-composition-video`
- [x] Browser check with `agent-browser` against the compiled fixture page (`http://127.0.0.1:8123/compiled.html`)
- [x] Clean tracked-only Docker build of `Dockerfile.test` with the PR version of the file applied
## Notes
- Latest regression workflow is green on `main`, but before this PR the `sub-composition-video` fixture was being skipped by the harness rather than exercised end to end.
- The CI Docker fix was validated from a tracked-only export to avoid local untracked worktree artifacts affecting the result.
## What
Added progress reporting to the skills installation process by converting synchronous operations to asynchronous ones and implementing progress callbacks.
## Why
The skills installation process can take a significant amount of time, especially when cloning repositories or running npm operations. Users need feedback about what's happening during the installation to understand progress and know the system hasn't frozen.
## How
- Converted `execFileSync` calls to a new `execFileAsync` function using promises
- Made all installation functions (`runSkillsAdd`, `gitClone`, `fetchRepo`, `fallbackInstall`) asynchronous
- Added an optional `onProgress` callback parameter to `installAllSkills` that accepts progress messages
- Integrated progress reporting in the `init` command by passing spinner message updates to the progress callback
- Added progress messages for key installation steps like "Installing {source} skills..." and "Cloning skill repositories..."
- Added "giget" to the external dependencies list in the build configuration
## Test plan
- [ ] Unit tests added/updated
- [ ] Manual testing performed
- [ ] Documentation updated (if applicable)
* feat(skills): add audio visualizer effect with extraction script
Adds reactive audio visualization patterns for HyperFrames:
Script: extract-audio-data.py pre-extracts per-frame RMS amplitude and
frequency band data via ffmpeg. Uses a 4096-sample FFT window for clean
frequency resolution and per-band normalization across the full track
so treble activity is visible alongside louder bass.
Patterns: spectrum bars, mirrored waveform, pulsing circle, circular
visualizer, background glow — all Canvas 2D driven from the GSAP
timeline via tl.call() at each frame.
Includes smoothing helper, band count guide, band ordering rules
(horizontal: low-left high-right), and combining patterns section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(skills): replace prescriptive examples with data model + motion principles
Removes five hardcoded draw functions that would get copy-pasted verbatim.
Replaces with:
- Clear data model docs (what rms and bands mean, how to index)
- Rendering approach setup for Canvas 2D, WebGL/Three.js, and DOM
- Motion principles (smoothing, value mapping, what makes it feel good)
- Spatial mapping conventions (low-left/high-right, etc)
The LLM invents the visualization; the skill teaches the data contract
and motion constraints.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(skills): off-by-one in band slicing, add data loading, fix trigger
- Fix exclusive slice end: high_bin clamped to n_bins (not n_bins-1)
so the last FFT bin in each band is included
- Add data loading section to skill doc (inline and fetch patterns)
- Fix example JSON to show frame 0 at time 0.0
- Update description to trigger when audio is analyzed and music detected
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(skills): require numpy, fix bugs, clean up skill doc
Script rewrite:
- numpy is now required (pure-Python DFT was unusable for real files)
- Use np.frombuffer instead of struct.unpack (~10x less memory)
- Precompute Hann window and band edges (were recalculated every frame)
- Extract SAMPLE_RATE as module-level constant
- Clamp band bins to prevent max() on empty slice
- Validate --fps and --bands inputs
Skill doc fixes:
- Fix fetch loading example (was null ref on sync for-loop)
- Remove redundant Canvas 2D section (was duplicate of Step 3)
- Fix opening line (said "Canvas 2D" but doc covers 3 approaches)
- Fix undefined W/H in example
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Add `/api/projects/:id/upload` endpoint for multipart file uploads with automatic dedup naming
- Wire `onImportFiles` from Assets tab "Import media" button through to the upload API
- Add global drag-drop overlay — drop media files **anywhere** in the studio, not just the Assets panel
- Files that already exist get `(2)`, `(3)` suffixes instead of overwriting
- Support uploading into subdirectories via `?dir=` param — dropping on a folder imports there
- Make folders draggable in the file tree + support drop-to-root
- Add `bodyLimit` middleware for early rejection of oversized payloads
- Surface skipped/failed uploads via toast notification instead of console-only
Addresses feedback: _"Wish I could upload/drag-drop assets directly in the Studio (music, images, video) like a CapCut media panel"_
## Test plan
- [x] Open studio, drag an image/video/audio file onto any part of the UI
- [x] Verify the drop overlay appears with "Drop files to import" message
- [x] Drop the file — verify it appears in the Assets tab and file tree
- [x] Drop a file with the same name — verify it gets a `(2)` suffix
- [x] Click "Import media" button in Assets tab — verify file picker works
- [x] Import multiple files at once via drag-drop
- [x] Drop a file onto a nested folder in the file tree — verify it lands in that folder
- [x] Drag a folder in the file tree and drop it on another folder or root — verify it moves
- [x] Drop a file >500MB — verify toast notification appears
- [x] Verify drag overlay doesn't get stuck when dragging over nested UI elements
## Summary
- **Fixes**: `<img src="../icon.svg">` and similar `../` relative asset references in sub-compositions resolve to 404 after inlining into the root document
- **Unifies**: Both `hyperframes preview` (core bundler) and `hyperframes render` (producer) now use the same shared logic — no duplication
## Root cause
When inlining a sub-composition at `compositions/scene.html` into root `index.html`, a relative path like `../icon.svg` is correct from `compositions/` (it points to project root) but after inlining, `../` escapes the project directory.
## Fix
Extracts path rewriting into a shared `rewriteSubCompPaths.ts` utility in `@hyperframes/core`, used by both the bundler and the producer. **Only rewrites paths starting with** **`../`** — plain relative paths like `assets/foo.svg` are already correct from the root perspective and must not be rewritten (this was the regression cause in `overlay-montage-prod`: sub-composition asset refs like `assets/notch.svg` were incorrectly being rewritten to `compositions/assets/notch.svg`).
## Regression fix
The earlier version of this fix (now in history) rewrote ALL relative paths including `assets/foo.svg`, breaking the `overlay-montage-prod` regression test. This PR fixes that by scoping rewrites to `../`\-prefixed paths only.
## Test plan
- [x] `../icon.svg` in sub-composition renders correctly in both preview and render
- [x] `assets/foo.svg` (no `../`) in sub-composition still resolves correctly — not rewritten
- [x] `overlay-montage-prod` regression test passes
- [x] All other regression shards pass
