Instead of killing transform tweens, wrap each overridden word span in an
inline-block wrapper span. Editor transforms (position, scale, rotation)
are applied to the wrapper while the inner span keeps all GSAP entrance,
exit, and karaoke animations intact.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add optional `id` field to transcript Word interface, assigned as `w{index}` during normalization
- caption-overrides.json uses `wordId` (preferred) with `wordIndex` fallback
- Runtime hook calls gsap.killTweensOf for overridden transform properties before applying
- Studio parser carries wordId through from transcript to segments
- Add typecheck to lefthook pre-commit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New caption-overrides.json format: [{wordIndex, x, y, scale, rotation, color, ...}]
- Runtime hook in packages/core applies overrides via gsap.set() after compositions load
- Studio writes/reads caption-overrides.json instead of injecting code into HTML
- Original composition source is never modified
- Remove old parseExistingOverrides and source patching code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Canvas: move (translate), scale, and rotate handles on selected words via gsap.set()
- Property panel: real-time style updates applied directly to iframe DOM
- Save: patches original source with gsap.killTweensOf + gsap.set per edited word
- Parser: reads existing editor overrides on re-entry to preserve prior edits
- Suppress preview refresh during caption edit mode to prevent reload cycle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements generateCaptionHtml(model) which serializes a CaptionModel
into a complete captions.html HyperFrames composition with template
wrapper, CSS per-group classes, TRANSCRIPT array, and GSAP timeline
registration via window.__timelines["captions"].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds `buildCaptionModel` to the caption parser — groups TranscriptWord
arrays into CaptionGroups (default 5 words each), creates CaptionSegments
with correct timing and groupIndex, and wires in DEFAULT_STYLE,
DEFAULT_CONTAINER, and DEFAULT_ANIMATION_SET. Covers 23 tests (13 new).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ConsoleMessage.type() returns "warn" not "warning"
- Type pageerror callback parameter as Error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Loads a composition in headless Chrome and reports console errors,
uncaught exceptions, and failed network requests for project assets.
Agents run this before opening the studio to catch runtime issues.
- Bundle HTML with inlined runtime (no CDN dependency)
- Minimal file server for project assets
- Captures JS errors, pageerror events, 404s on project files
- JSON output mode for programmatic consumption
- ~3s execution time, suitable as a blocking pre-preview step
Updates CLAUDE.md and compose skill to require lint + validate
before opening the studio.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Vite dev server was loading the HyperFrames runtime IIFE from
cdn.jsdelivr.net. This caused compositions using window.__hyperframes
(e.g., fitTextFontSize) to fail when the CDN script didn't load in
the iframe context.
Add a /api/runtime.js endpoint to the Vite plugin that serves the
local built runtime, matching what studioServer.ts already does for
the CLI embedded server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
.en models (small.en, medium.en) translate non-English audio into English
instead of transcribing it. This silently destroys the original language.
- Add non-negotiable language rule to captions skill
- Update model tables to default to models without .en suffix
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds two lint rules for caption compositions:
- caption_transcript_not_inline: warns when transcript is loaded via
fetch() instead of an inline var TRANSCRIPT array
- caption_transcript_parse_error: warns when the inline TRANSCRIPT
array is not valid JSON (common with unquoted keys + apostrophes)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Documents the inline TRANSCRIPT requirement, JSON-quoted keys, and
CSS class conventions needed for the studio caption editor to detect
and edit captions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The player needs window.__timelines registered immediately to start
playback. GSAP timelines are mutable so tl.call() entries added
later in the fetch callback still fire when seeked.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Never overflow: hidden on caption containers — clips glow/scale effects
- Music requires audio-reactive captions (any genre, any energy level)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Auto-detect language and switch from .en to multilingual model when needed
- Detect speech onset in WAV to strip hallucinated words before speech begins
- Merge whisper-cpp token fragments: contractions (didn + 't → didn't),
split capitals (C + aught → Caught), dropped-g (shin + in' → shinin')
- Interpolate zero-duration word clusters for reliable karaoke timing
- Replace non-null assertions with optional chaining in tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Split the captions skill from a single 611-line file into focused references: `SKILL.md` (core rules), `transcript-guide.md` (whisper/transcription), `dynamic-techniques.md` (animation patterns)
- Add `audio-reactive` skill with "Content, Not Medium" constraint — steers away from generic visualizations (equalizer bars, spectrum analyzers, waveforms) toward content-grounded animation where audio drives *when* and *how much*, not *what to show*
- Add initial dynamic caption technique selection by energy level
## Test plan
- [ ] All skill files render correctly as markdown
- [ ] Cross-references between files use correct relative paths
- [ ] `audio-reactive/SKILL.md` contains the anti-pattern list and content-grounded examples
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Context
`npx hyperframes render` fails to load assets and produces all-black video. Root causes:
1. CDN scripts (GSAP, Lottie) are left as `<script src="https://...">` in compiled HTML — the headless browser must fetch them over the network, which fails in Docker, CI, and firewalled environments
2. Assets referenced from outside the project directory (e.g. `../shared-assets/hero.png`) 404 because the file server only serves from `projectDir`
3. When GSAP fails to load, the timeline never registers, all `.clip` elements stay `visibility: hidden`, and every frame is black — with no diagnostic output
4. The linter reports `missing_gsap_script` when GSAP is bundled inline (no `<script src>` tag), which blocks users from working around the CDN issue
## What changed
### 1\. CDN script inlining (`htmlCompiler.ts`)
`compileForRender` now downloads all external `<script src="https://...">` tags at compile time and inlines their content into the HTML. Rendering no longer needs network access.
| Before | After |
| --- | --- |
| `<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>` stays in HTML, browser must fetch at runtime | Script downloaded during compilation, embedded as `<script>/* inlined: https://... */\n...code...</script>` |
| Docker/CI render → `net::ERR_NAME_NOT_RESOLVED` → black video | Render works fully offline |
| No feedback when CDN fails | `[Compiler] WARNING: Failed to download CDN script: ... Consider bundling it locally` |
### 2\. External asset copying (`htmlCompiler.ts`, `renderOrchestrator.ts`)
After compilation, the HTML is scanned for `src`, `href`, and CSS `url()` references that resolve outside `projectDir`. These files are copied into the compiled output directory so the file server can serve them.
| Before | After |
| --- | --- |
| `background-image: url(../shared-assets/hero.png)` → 404 (file server can't serve outside `projectDir`) | Asset detected, copied to compiled dir, path rewritten → serves correctly |
| `<img src="../shared-assets/logo.png">` → 404 | Same fix — works for all `src`/`href` attributes and CSS `url()` |
### 3\. Black video diagnostics (`renderOrchestrator.ts`)
When composition duration is 0 (which would produce a black video), the error now probes the browser for diagnostics instead of a generic message.
| Before | After |
| --- | --- |
| `Invalid composition duration: 0. Check that GSAP timelines are registered.` | `Composition duration is 0 — this would produce a black video.\n\nDiagnostics:\n - GSAP is not loaded — CDN script may have failed to download. Bundle GSAP locally...\n - Browser: [Browser:PAGEERROR] gsap is not defined` |
| Asset 404s during page load silently logged | `[Render] Asset load failure: ...` + `[WARN] Browser encountered network failures during page load` |
### 4\. Linter: recognize inline GSAP (`core/lint/rules/gsap.ts`)
The `missing_gsap_script` rule now recognizes GSAP bundled inline — matching the producer's inlining comment (`/* inlined: ...gsap... */`), GSAP library internals (`_gsScope`, `GreenSock`), and large inline scripts (>5KB) referencing gsap.
| Before | After |
| --- | --- |
| User inlines GSAP → linter errors with `missing_gsap_script` | Inline GSAP detected, no false error |
| Producer inlines CDN → linter errors on the compiled HTML | Producer's `/* inlined: ... */` comment recognized |
## Test plan
- [x] `pnpm build` passes
- [x] Core tests pass (410/410, +2 new)
- [x] **Reproduced baseline failures on** **`main`**: CDN scripts not inlined, external assets 404, no diagnostics
- [x] **Verified fixes**: CDN script inlined, external assets copied and served, diagnostics printed
- [x] Render with working CDN → `[Compiler] Inlined CDN script: ...` → render succeeds
- [x] Render with broken CDN → `[Compiler] WARNING: Failed to download CDN script` + browser errors surfaced
- [x] Render with assets outside project dir → `[Compiler] Found 1 asset(s) outside project directory` → assets served correctly
- [x] Linter with inline GSAP → no `missing_gsap_script` false positive
* feat(cli): improve --help with grouped commands and per-command examples
Replaces citty's flat COMMANDS list with kubectl-style grouped categories
and adds examples to every subcommand. Root help now groups 14 commands
into 5 categories (Getting Started, Project, Tooling, AI & Integrations,
Settings). Per-command --help now appends a formatted Examples section
with practical usage patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(cli): use shared colors, lazy-load help, fix description drift
- Replace hand-rolled ANSI helpers with existing ui/colors.ts (fixes
non-conformant NO_COLOR handling)
- Add cyan and gray to shared color module
- Lazy-load help.ts via dynamic import to avoid allocating help data
on non-help invocations
- Fix description drift: benchmark and transcribe descriptions now
match their command meta.description
- Unify tuple order: ROOT_EXAMPLES now uses [comment, command] to
match COMMAND_EXAMPLES
- Remove redundant comment restating type annotation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): resolve typecheck error in showUsage generic signature
The lazy-load wrapper needs to cast CommandDef<T> to CommandDef when
forwarding to the help module, since TypeScript's generic variance
makes the direct assignment incompatible.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(cli): remove kubectl references from help comments
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting
## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.
## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`
## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
## What
Move the id-less media fix into the shared timing compiler so producer can resolve durations for sub-composition videos before inlining, then carry the merged result through engine parsing, regression coverage, and the regression Docker image used in CI.
This PR now does five concrete things:
- assigns stable ids to id-less media in core `compileTimingAttrs()` so unresolved duration injection can target them
- keeps the engine-side `parseVideoElements()` support for `video[src]` plus the newer `data-duration` / natural-duration fallback from `main`
- makes producer prefer sub-composition media metadata over the later inlined-document parse when the same media id appears in both places
- makes `sub-composition-video` a runnable regression test by fixing its metadata and checking in the missing `output/compiled.html` snapshot
- removes the stale `pnpm-workspace.yaml` copy step from `Dockerfile.test`, so regression CI builds the Bun-based test image from the current workspace layout
## Why
- media without an explicit `id` could not participate in unresolved-duration resolution early enough
- producer could lose the resolved sub-composition timing by overwriting it with the later inlined parse
- the regression fixture intended to cover this case was not actually running in CI because its `meta.json` was incomplete and the required compiled snapshot was missing
- the regression image definition still expected a deleted `pnpm-workspace.yaml`, so GitHub Actions failed before the test shard could start
Putting the id-generation step in core makes the behavior reusable instead of relying on producer-only HTML patching.
## How
### Shared compiler
- core `compileTimingAttrs()` now auto-assigns stable ids to id-less `video` / `audio` tags
- those generated ids are returned in `unresolved`, so `injectDurations()` can add `data-duration` and `data-end` to the same media element later in the pipeline
- added core tests that cover auto-id assignment and duration injection for generated ids
### Producer
- when producer combines `subVideos` / `subAudios` with the media re-parsed from the final inlined HTML, it now lets the sub-composition metadata win
- this preserves the resolved/clamped timing already computed for nested media instead of overwriting it with the later parse
- `sub-composition-video` now has valid regression metadata and a checked-in `output/compiled.html` snapshot so CI actually executes it
### Engine
- resolved the merge conflict in `videoFrameExtractor` by keeping the broader `video[src]` parsing from this branch and the `data-duration` / natural-duration fallback that landed on `main`
- added a focused engine unit test for videos without ids
### CI image
- `Dockerfile.test` now copies only `package.json` and `bun.lock` at the workspace root before `bun install --frozen-lockfile`
- this matches the current monorepo layout and removes the obsolete pnpm-era dependency on `pnpm-workspace.yaml`
## Test plan
- [x] `bun run --filter @hyperframes/core test`
- [x] `bun run --filter @hyperframes/engine test`
- [x] `bun run --filter @hyperframes/producer test --update --sequential sub-composition-video`
- [x] `bun run --filter @hyperframes/producer test --sequential sub-composition-video`
- [x] Browser check with `agent-browser` against the compiled fixture page (`http://127.0.0.1:8123/compiled.html`)
- [x] Clean tracked-only Docker build of `Dockerfile.test` with the PR version of the file applied
## Notes
- Latest regression workflow is green on `main`, but before this PR the `sub-composition-video` fixture was being skipped by the harness rather than exercised end to end.
- The CI Docker fix was validated from a tracked-only export to avoid local untracked worktree artifacts affecting the result.
## What
Added progress reporting to the skills installation process by converting synchronous operations to asynchronous ones and implementing progress callbacks.
## Why
The skills installation process can take a significant amount of time, especially when cloning repositories or running npm operations. Users need feedback about what's happening during the installation to understand progress and know the system hasn't frozen.
## How
- Converted `execFileSync` calls to a new `execFileAsync` function using promises
- Made all installation functions (`runSkillsAdd`, `gitClone`, `fetchRepo`, `fallbackInstall`) asynchronous
- Added an optional `onProgress` callback parameter to `installAllSkills` that accepts progress messages
- Integrated progress reporting in the `init` command by passing spinner message updates to the progress callback
- Added progress messages for key installation steps like "Installing {source} skills..." and "Cloning skill repositories..."
- Added "giget" to the external dependencies list in the build configuration
## Test plan
- [ ] Unit tests added/updated
- [ ] Manual testing performed
- [ ] Documentation updated (if applicable)
* feat(skills): add audio visualizer effect with extraction script
Adds reactive audio visualization patterns for HyperFrames:
Script: extract-audio-data.py pre-extracts per-frame RMS amplitude and
frequency band data via ffmpeg. Uses a 4096-sample FFT window for clean
frequency resolution and per-band normalization across the full track
so treble activity is visible alongside louder bass.
Patterns: spectrum bars, mirrored waveform, pulsing circle, circular
visualizer, background glow — all Canvas 2D driven from the GSAP
timeline via tl.call() at each frame.
Includes smoothing helper, band count guide, band ordering rules
(horizontal: low-left high-right), and combining patterns section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(skills): replace prescriptive examples with data model + motion principles
Removes five hardcoded draw functions that would get copy-pasted verbatim.
Replaces with:
- Clear data model docs (what rms and bands mean, how to index)
- Rendering approach setup for Canvas 2D, WebGL/Three.js, and DOM
- Motion principles (smoothing, value mapping, what makes it feel good)
- Spatial mapping conventions (low-left/high-right, etc)
The LLM invents the visualization; the skill teaches the data contract
and motion constraints.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(skills): off-by-one in band slicing, add data loading, fix trigger
- Fix exclusive slice end: high_bin clamped to n_bins (not n_bins-1)
so the last FFT bin in each band is included
- Add data loading section to skill doc (inline and fetch patterns)
- Fix example JSON to show frame 0 at time 0.0
- Update description to trigger when audio is analyzed and music detected
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(skills): require numpy, fix bugs, clean up skill doc
Script rewrite:
- numpy is now required (pure-Python DFT was unusable for real files)
- Use np.frombuffer instead of struct.unpack (~10x less memory)
- Precompute Hann window and band edges (were recalculated every frame)
- Extract SAMPLE_RATE as module-level constant
- Clamp band bins to prevent max() on empty slice
- Validate --fps and --bands inputs
Skill doc fixes:
- Fix fetch loading example (was null ref on sync for-loop)
- Remove redundant Canvas 2D section (was duplicate of Step 3)
- Fix opening line (said "Canvas 2D" but doc covers 3 approaches)
- Fix undefined W/H in example
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
