Commit Graph
52 Commits
Author SHA1 Message Date
Miguel Ángel f1a37400b4 fix(runtime): silent-first-play + loading overlay for preview (#293)
## Summary

Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit.

### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`)

`syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window):

- `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click."
- The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged.

`HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`.

### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`)

The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s.

### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`)

Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration.

Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR.

### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`)

Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync.

## Verification

- 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset.
- Full monorepo build green (`bun run build`), typecheck clean, lint/format clean.
- End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass:

| Scenario | Metric | Result |
| --- | --- | --- |
| Normal first play | Audio plays from click, smooth progression |  |
| Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip |  |
| Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) |  |
| Scrub mid-playback | Lands exactly at target frame |  |

## Files changed

- `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve.
- `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset.
- `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes.
- `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check.

## Follow-ups (deferred)

- Tight-threshold short-window drift correction for lip-synced dialogue.
- A perf-regression test that fails on `waiting`\-event resurgence.

## Test plan

- [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words.
- [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events).
- [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers.
- [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame.
2026-04-16 21:51:06 +02:00
Ular KimsanovandClaude Opus 4.6 87f4c77e2f feat: website capture pipeline + 7-step video production skill (#284)
* feat(cli): add website capture with AI-powered DESIGN.md generation

Adds `hyperframes capture <url>` command that extracts a complete design
system from any website, producing AI-agent-ready output:

- Full-page screenshot (lazy-load aware, nav at top)
- AI-generated DESIGN.md via Claude API (colors, typography, elevation,
  components, do's/don'ts) with programmatic asset catalog (136+ assets
  with HTML context annotations like img[src], css url(), link[rel=preload])
- CSS-purged compositions (87% size reduction via PurgeCSS)
- HTML-prettified compositions (one-tag-per-line for AI readability)
- CLAUDE.md + .cursorrules auto-generated for AI agent instructions
- Asset deduplication (srcset variants) and tracking pixel filtering

* feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement

- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update session handoff with slack research findings

- key finding: team already wants DESIGN.md integration (James, Bin, Vance)
- skills quality matters enormously - must invoke /hyperframes-compose
- eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide)
- templates at templates/ need study before finalizing skill
- session handoff updated with critical next steps

* refactor(cli): simplify capture pipeline, remove replica generator

* feat(capture): add Lottie detection and WebGL shader extraction

Captures Lottie animations via network interception and WebGL shader
source via gl.shaderSource hooking during site crawl. Updates
website-to-hyperframes skill with asset planning guidance, Lottie/shader
reading instructions, and stronger creative direction for scene planning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(capture): clean pipeline + shader-first creative workflow

Capture pipeline:
- Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps)
- Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md)
- Remove dead canvas-to-image code in htmlExtractor (post canvas removal)
- Parallelize image downloads (batches of 5 via Promise.allSettled)
- Fix pre-existing TS error (match[1] guard in font downloader)
- Default capture output to captures/<hostname>

Skill creative overhaul:
- Add shader transition selection to creative director step (Step 4)
- Add shader wiring instructions to engineer step (Step 5)
- Replace 4-line energy modifiers with visual vocabulary table
- Strip rigid scene-by-scene templates from video-recipes.md
- Strip example fill data from scene plan tables
- Add "read transition refs before planning" instruction
- Add creative ambition language ("how the hell did they make this")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add skill architecture redesign spec

Comprehensive redesign of website-to-hyperframes skill and capture
pipeline based on code review findings and Claude Code architecture
research. Key changes: remove AI auto-generation, restructure skill
into phases, embed shader boilerplate in scaffold, fix color format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add implementation plan for skill architecture redesign

13-task plan covering: capture pipeline cleanup (remove AI generation,
fix colors to HEX, add asset descriptions, shader-ready scaffold),
skill restructuring (4 phases with artifact gates), and compose skill
Visual Identity Gate upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(capture): remove AI auto-generation and SDK dependencies

* fix(capture): convert extracted colors to HEX format

* refactor(capture): remove AI key path, add asset descriptions generator

* refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions

* feat(capture): pre-wire shader transitions in index.html scaffold

* chore: remove duplicate visual-styles.md (canonical is in hyperframes/)

* refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator

* feat(skill): add Phase 1 understand reference

* feat(skill): add Phase 2 design reference with full DESIGN.md schema

* feat(skill): add Phase 3 creative direction reference

* feat(skill): add Phase 4 build reference with inline shader example

* feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md

* docs: update CLAUDE.md skill references for phase-based workflow

* fix: address code review findings

- Remove orphaned `false` argument in generateAgentPrompt call (critical:
  was shifting hasLottie, hasShaders, catalogedAssets parameters)
- Add HSL color handling in rgbToHex via temp element resolution
- Remove build artifact commit section from phase-4-build.md
- Fix __GSAP_TIMELINE reference to __timelines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(capture): regex double-escape + simplify scaffold + fix asset descriptions

- Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\()
  so browser receives valid regex patterns via page.evaluate()
- Simplify index.html scaffold: scene slots + audio + timeline + comment pointing
  to shader-setup.md reference (no broken inline shader boilerplate)
- Fix asset descriptions: use CatalogedAsset.contexts/notes instead of
  nonexistent htmlContext field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: code review — 16 bugs, 7-step skill rewrite, cleanup

Code fixes:
- snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero
  for --frames 1, port bind error handling, rAF-based render settle
- index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini
  batch/rate-limit comments, fix video preview viewport y-coordinate
- tokenExtractor.ts: remove dead seen[si] dedup code
- gsap.ts: index ALL classes for inline-style transform conflict detection

Skill architecture rewrite (4-phase → 7-step):
- Replace phase-1 through phase-4 with step-1 through step-7
- Add techniques.md (10 visual techniques with code patterns)
- Fix /hyperframes-compose → /hyperframes (skill doesn't exist)
- Fix captures/arc-browser reference → shader-setup.md (file doesn't exist)
- Fix step-7 hardcoded captures/stripe path
- Document Gemini API free/paid rate limits in step-1

Cleanup:
- CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step
- .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast)
- Delete old phase-*.md, animation-recreation.md, tts-integration.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dev artifacts, research docs, wrong lockfiles

Remove files that shouldn't ship in this PR:
- docs/research/ (aura analysis, prompt catalogs)
- docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes)
- docs/superpowers/ planning and spec docs
- pnpm-lock.yaml at root and cli (repo uses bun, not pnpm)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention

Main PR #283 removed the full skills table from CLAUDE.md and moved it
to AGENTS.md. Align with that decision: use main's slim dev-focused
format, fix pnpm→bun references, add one-line /website-to-hyperframes
pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add capture command to help groups

The capture command was registered in cli.ts but missing from
the help groups, so it wouldn't appear in `hyperframes --help`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format skill reference files (oxfmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: regenerate bun.lock after rebase

The lockfile was stale after rebasing onto main — bun install
--frozen-lockfile failed in CI because new dependencies (google/genai,
patchright, purgecss) weren't reflected in the lockfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments + improve capture quality

Review fixes (16 comments from jrusso1020 + vanceingalls):
- screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots
- screenshotCapture: dynamic screenshot count based on page height (30% overlap)
- snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard
- htmlExtractor: fix code injection via parameterized evaluate
- index.ts: video preview re-measures position after scroll, .env file loading
- capture.ts: BLOCKED.md on timeout failures
- gsap.ts: 5 inline-style lint tests added (all pass)
- Remove Playwright, patchright deps; @google/genai to optionalDependencies
- Gitignore: generic patterns instead of 20 hardcoded directories
- Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance)

Capture quality improvements (tested on 10+ websites):
- Color extraction: canvas-based oklch/lab resolver, pixel sampling via
  elementFromPoint, broad sweep for accent colors, gradient/shadow extraction
- Section detection: broadened selectors for div-based layouts, height cap
  to skip page-level wrappers, parent bg walkup for dark sites
- Font downloads: cap 6 per family / 30 total (Cal.com: 306→30)
- CTA detection: text pattern matching + nav context filtering
- Heading text: innerText with whitespace normalization
- Gemini captioning: maxOutputTokens 100→300, .env auto-loading
- .env.example updated with GEMINI_API_KEY docs
- TTS ranking: Kokoro first with Python 3.10+ note

* fix: address PR review comments + improve capture quality

Review round 2 fixes (jrusso1020 + vanceingalls):
- verify/index.ts: add path traversal guard (relative + isAbsolute)
- verify/index.ts: fix sections[i] undefined typecheck error (CI green)
- index.ts: escape Lottie JSON with \u003c to prevent </script> breakout
- step-4-storyboard: fix technique count contradiction (2-3 per beat, not
  across whole video)
- step-6-build: perspective tilt uses gsap.set() instead of CSS transform
  (avoids GSAP overwrite conflict)
- step-1-capture: reorder — command first, Gemini note after (zero-config
  is the default path, API key is optional enhancement)
- step-7-validate: add tsx fallback for snapshot command
- step-3-script: vary hook patterns, don't default to number every time
- assetDownloader: exempt SVGs from 10KB minimum filter (company logos
  like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets)

Note: adm-zip was NOT removed (reviewer #3) — it's still in
packages/cli/package.json:30. The root package.json had patchright
and purgecss removed, not adm-zip.

Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms
zero references in the entire codebase. The @anthropic-ai/sdk dependency
was removed earlier in this branch.

* refactor(capture): split index.ts (1175 to 566 lines) into modules

Mechanical extraction, zero logic changes.

New files:
- mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots
- contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions
- scaffolding.ts (135 lines): .env loading, project scaffold generation

Also fixes false-positive BLOCKED.md with structural Cloudflare detection.
Tested on 20 websites, pre/post output identical.

* chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss)

The --split feature auto-generates compositions from captured HTML — a
different approach from the /website-to-hyperframes skill workflow where
agents build compositions from scratch using the storyboard.

No skill file, no step reference, and no test session ever used --split.
Removes 923 lines of unused code + purgecss dependency.

Backed up to ~/Desktop/capture-split-backup/ for reference.

* fix(security): add ssrf protection, lottie injection fix, oom guard

- assetDownloader: add isPrivateUrl() guard blocking private IP ranges
  (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata
  endpoints, localhost, and non-HTTP schemes
- mediaCapture: fix Lottie JSON injection by loading shell HTML first
  then passing animation data via parameterized page.evaluate()
- index.ts: check Content-Length header before response.buffer() in
  Lottie network interception to avoid OOM on multi-GB responses

* fix(capture): security fixes, timeout, sub-agent dispatch instructions

Security (from miguel-heygen review):
- assetDownloader: export isPrivateUrl() SSRF guard
- htmlExtractor: add isPrivateUrl check before CSS fetch
- mediaCapture: add isPrivateUrl check before Lottie fetch
- mediaCapture: fix previewPage leak (try/finally)
- mediaCapture: skip Lottie files > 2MB for preview (CDP limit)
- contentExtractor: skip images > 4MB for Gemini captioning
- index.ts: check Content-Length before response.buffer() (OOM guard)
- snapshot.ts: register error handler before server.listen()

Capture improvements:
- Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare)
- step-6-build: sub-agent dispatch template with explicit rules:
  pass file PATHS not contents, use local fonts not Google Fonts,
  verify ../assets/ references after each beat

* fix(capture): catalog before DOM mutation, networkidle2, faster Gemini

Critical: asset cataloger now runs BEFORE extractHtml which converts img
src to data URLs. Framer sites like heykuba.com went from 2 to 78 images.

- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load wait: scroll to bottom, wait for img.complete
- CSS background-image cataloging for Framer/Webflow
- SVG naming: checks class, id, parent, inner text (not just aria-label)
- Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img)
- maxOutputTokens 300->500, descriptions sorted captioned-first
- Remove tsx fallback from step-1 (reviewer nit, published CLI has it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:03:16 -07:00
James Russo 969474e843 feat(cli): registry resolver + installer (#254)
## What

PR 3/17 of the catalog system rollout. Introduces the registry resolver/installer abstraction. No UX change — `init --template` still works identically. Stacks on #253.

**New module: `packages/cli/src/registry/`**
- `remote.ts` — fetches manifests (`registry.json`, `registry-item.json`) and item files from a GitHub-hosted registry. 24h cache on manifests; item files stream straight to `destDir`
- `resolver.ts` — `listRegistryItems`, `loadAllItems` (parallel fetch for picker UX), `resolveItem` (single-item fetch with `Available:` error)
- `installer.ts` — `assertSafeTarget` (runtime path-traversal guard) + `installItem` (parallel file download with up-front validation; all-or-nothing semantics)
- `index.ts` — barrel

**Registry content:**
- `registry/registry.json` — top-level manifest in PR 1's `RegistryManifest` shape. 8 examples
- `registry/examples/<id>/registry-item.json` — per-item manifest for each existing example, generated from legacy `templates.json` + HTML data-attribute probing
- `registry/examples/templates.json` — **deleted**, replaced by the above

**Compat layer:**
- `packages/cli/src/templates/{remote,generators}.ts` — thin shims that delegate to `../registry/`, keeping `init.ts`'s existing imports stable. `init.ts` doesn't move to the new API until PR 5 where it's part of a larger UX pass

**Tooling:**
- `scripts/generate-registry-items.ts` — idempotent one-off generator for this PR, kept in-repo for future example additions (`--only <name>` flag)

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a). Tracker entry in local `hyperframes-catalog-plan.md`.

## Why

Every future PR (`hyperframes add`, seed blocks, seed components, custom registries) otherwise has to keep piling onto the ad-hoc fetch + `cpSync` pattern in the old `fetchRemoteTemplate`. The new module is the single place that understands the registry wire format and file layout. **This is also where PR 1's schema comes alive.**

## How

### Scope-trimmed from the plan

- **No transitive dependency resolution yet.** Examples have no deps today. `resolveItem` doesn't walk `registryDependencies`; PR 5 adds that when blocks/components need it.
- **No ajv schema validation yet.** TS types + runtime path-traversal guard are the only safety nets. Full JSON-Schema validation lands when the registry starts accepting third-party content (PR 14 / custom registries).
- **init.ts refactor deferred to PR 5.** Compat shims keep this PR small and reviewable. PR 5 rewrites init alongside adding the `add` command.

### Safety

- `assertSafeTarget` rejects absolute paths, `..` segments, Windows drive letters, and any target that `path.resolve` shows to escape `destDir`. Mirrors the PR 1 schema `pattern`/`not.anyOf` on `target`, but runs at install-time so a registry that bypasses schema validation still can't write outside the project
- Up-front validation in `installItem` means a malformed item fails **before** any file is written. Atomic-ish semantics: all files land or none do

### Caching

- 24h manifest cache lives at `~/.hyperframes/cache/` per existing convention, but now keyed by `<baseUrl>__<kind>__<name>.json` so PR 14 custom registries can coexist

## Test plan

- [x] `bun run test` in `packages/cli`: **70 passed** (was 57 on #253, +13). Same 4 pre-existing failures (SRT/VTT whisper normalizer + `lintProject` clean-project test) — identical to main. No regressions
- [x] **Resolver unit tests (8):** filter by type, parallel load with fail-safe, resolve-by-name with `Available:` error message, unreachable-registry handling
- [x] **Installer unit tests (5):** accepts simple relative paths, rejects `..` segments, rejects Unix absolute paths, rejects Windows drive letters, permits `.` and dotfile-like names
- [x] **Smoke test**: `hyperframes init /tmp/x --template blank` (bundled code path, unchanged) works end-to-end
- [x] `bunx oxfmt --check` + `bunx oxlint`: clean
- [x] Pre-commit typecheck (core + studio): clean. CLI typecheck has 2 pre-existing errors (`render.ts`, `studioServer.ts` — unrelated `"mov"` format issue on main)
- [ ] **Smoke test remote fetch (`--template warm-grain`)** — verifiable only post-merge; registry paths live on `main` after this PR lands

## Breaking / migration

**No end-user-visible UX change.** `init --template <name>` still works the same way. Internally, `templates.json` is gone and the CLI now reads `registry.json` + `registry-item.json` per example.

Installed CLIs on old versions (`hyperframes@0.1.0`–`0.3.0`) already broke at PR 2 merge (see #253 rollout note). The next CLI release after this lands (`0.3.1`+) is the full fix.

## Commits

1. `generate-registry-items.ts` + generated manifests + deleted `templates.json`
2. Resolver + installer + compat shims
3. Unit tests

(All squashed into one commit on this branch; see `git log feat/registry-resolver ^refactor/registry-examples-dir`.)

## Stacks on

#253 — base branch. When #253 merges, this rebases onto `main`.

## Next in stack

PR 4 — `feat(cli)!: rename --template to --example`. Single clean cut, no alias. Tiny PR (~150 lines) that mostly updates `init.ts`'s argument schema, help text, and docs. Depends on this PR so the new flag name can be applied against the refactored code path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-13 20:41:23 -07:00
James RussoandClaude Opus 4.6 eb338ae859 feat(core): add registry schema + TS types (#252)
* feat(core): add registry schema + TS types

PR 1/17 of the catalog system rollout. Foundation for a shadcn-style
registry with three item tiers: examples (full projects), blocks
(sub-compositions), components (effect snippets).

## What

- TS types: RegistryItem (discriminated union of ExampleItem/BlockItem/
  ComponentItem), RegistryManifest, FileTarget, ItemType, FileType
- JSON Schemas: schemas/registry.json, schemas/registry-item.json
- Compile-time exhaustiveness asserts on ITEM_TYPES/FILE_TYPES so adding
  to the TS union without updating the constant stops compiling
- Drift-guard test: schema enums must equal ITEM_TYPES/FILE_TYPES by
  set-equality; exactly 2 distinct type enums in registry-item.json
- Public API via new ./registry export path plus re-exports from root;
  schemas exposed via ./schemas/registry.json export for external tooling

## Why

- Every downstream PR (resolver, installer, hyperframes add, docs
  codegen, CI previews, skill, catalog command) builds on these types
- Getting the shape right now avoids painful migrations later

## How

- Discriminated union enforces that components do not have dimensions
  or duration and examples/blocks must have them (schema mirrors via
  if/then/else on the type discriminant)
- target path pattern rejects .. segments, Unix absolute paths, and
  Windows drive letters (defense-in-depth; CLI validates at runtime in
  PR 3)
- name pattern requires alphanumeric start and end (no trailing hyphens)
- Optional metadata: version, author, license, deprecated, minCliVersion
- additionalProperties: false on nested objects (catches typos on
  critical fields) but relaxed on top-level RegistryItem (allows
  third-party custom metadata in PR 15 custom registries)

## Test plan

- [x] Unit tests: 11 new tests covering type guards, discriminant
      narrowing, schema/TS drift guards, schema \$id sanity, optional
      metadata acceptance, and compile-time checks (via @ts-expect-error)
- [x] bun run test in packages/core: 445 passed (was 434 on main,
      +11 from this PR)
- [x] bunx oxfmt and bunx oxlint: clean
- [x] bun run typecheck: clean
- [ ] Manual testing: N/A (types + schemas only)
- [ ] Documentation updated: per-item doc pages land in PR 9 (codegen
      from these manifests); guide updates in PR 10+

## Breaking / migration

None. Pure additive — new module, new export paths, no existing
surface touched.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(core): remove version field + hyperframes:demo file type

Address review feedback from Miguel:

- Remove `version` from RegistryItemBase + schema. Per shadcn model,
  the registry is versioned by git tags, not per-item. The adversarial
  review added it; the original design doc was correct.
- Remove `hyperframes:demo` from FileType union + FILE_TYPES constant
  + schema. Demo files exist on disk for the CI preview pipeline but
  are NOT installed to user projects and should not appear in
  registry-item.json files[]. Neither shadcn nor Remotion has a
  dedicated demo file type — demos are just compositions.
- Add `required: ["type"]` to the if-condition in the schema's
  allOf discriminant (Miguel's nit — makes the condition self-
  contained)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): add shader-transitions to Dockerfile.test

PR #251 added packages/shader-transitions/ to the workspace but didn't
update Dockerfile.test to COPY its package.json. This caused
`bun install --frozen-lockfile` to fail in the regression Docker build:
bun saw a lockfile referencing @hyperframes/shader-transitions but the
package.json wasn't present in the container, so it wanted to remove
the entry — triggering "lockfile had changes."

Verified: Docker build passes with `--no-cache` after this fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 20:18:37 -07:00
James RussoandClaude Opus 4.6 18de86e4bd fix(player): handle Infinity duration; add lint rules for data-duration and Math.ceil overshoot (#243)
* fix(player): handle Infinity duration from runtime gracefully

When compositions have repeating animations without data-duration, the
runtime sends durationInFrames: Infinity. The player now ignores
non-finite duration values instead of displaying "Infinity:NaN" in the
controls. formatTime also returns "0:00" for non-finite inputs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(lint): add data-duration and Math.ceil overshoot rules

- Add root_composition_missing_data_duration warning when the root
  composition element is missing data-duration, which causes the runtime
  to infer Infinity for loop-inflated timelines.
- Add gsap_repeat_ceil_overshoot warning that catches
  repeat: Math.ceil(d/c)-1 patterns which overshoot the intended
  duration. Recommends Math.floor instead.
- Fix gsap_infinite_repeat fixHint to suggest Math.floor (not Math.ceil).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(player): wait for injected runtime before declaring ready

When the player auto-injects the runtime script (because the
composition has GSAP timelines but no runtime), it would immediately
declare ready on the next probe cycle — before the runtime script
finished loading from CDN. This caused play() to send a postMessage
that nobody received, making autoplay silently fail.

Now the probe waits for the runtime bridge (__hf or __player) to
appear before proceeding to the ready state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 10:41:13 -07:00
Miguel Ángel 794b02153d fix(lint): upgrade bare composition HTML to error (#242)
## Summary

- Upgrades `root_composition_missing_html_wrapper` from **warning** to **error** — a bare `<div data-composition-id>` as `index.html` without `<!DOCTYPE html>/<html>/<body>` causes browsers to quirks-mode, the preview server to fail, and the bundler to silently skip runtime injection
- Improves the error message to explain _why_ this is bad, and includes a snippet of the offending root element
- Skips `<template>`\-wrapped compositions (already caught by the separate `standalone_composition_wrapped_in_template` rule)
- Adds 8 tests covering the exact screenshot scenario, proper HTML, sub-compositions, plain HTML, and template wrappers

## Test plan

- [x] All 441 existing tests pass (`vitest run`)
- [x] 8 new tests for `root_composition_missing_html_wrapper` and `standalone_composition_wrapped_in_template`
- [x] TypeScript build clean (`tsc --noEmit`)
- [x] oxlint + oxfmt pass
- [x] Run `npx hyperframes lint` on a bare composition `index.html` and verify it now reports an error
2026-04-11 05:00:47 +02:00
Miguel Ángel ceb54811c6 fix(runtime): preload media on init to prevent broken first-play audio (#234)
## Summary
- Audio (and video) sounds broken/choppy on first play in the studio preview, but works fine on second play
- Root cause: `<audio>` elements default to `preload="metadata"`, which only fetches enough data to determine duration — not enough for smooth playback. When `el.play()` fires, the browser hasn't buffered the audio data yet
- The runtime now eagerly sets `preload="auto"` and calls `load()` during init, ensuring media is fully buffered before the user clicks play
- `syncRuntimeMedia` now defers `play()` on unbuffered media by registering a `canplay` listener, instead of silently swallowing the failure

## Testing
Verified with agent-browser against a 26s narration composition (soulscape-film):

```
# After fix — audio element state at init:
preload: "auto"
readyState: 4 (HAVE_ENOUGH_DATA)
buffered: 26.07s (entire file)
duration: 26.07s
```

Audio is fully buffered before any play attempt, so first-play works identically to subsequent plays.

## Files changed
- `packages/core/src/runtime/init.ts` — set `preload="auto"` + `load()` in `bindMediaMetadataListeners`
- `packages/core/src/runtime/media.ts` — defer `play()` on unbuffered media via `canplay` listener
- `packages/core/src/runtime/media.test.ts` — updated test + added unbuffered media test case
2026-04-10 00:05:32 +02:00
Vance IngallsandClaude Opus 4.6 fc973ee2e8 feat(lint): add rules for missing data-start, template wrapper, and DOCTYPE
Three new lint rules that catch structural issues causing compositions
to fail silently in preview:

- root_composition_missing_data_start: Root composition needs data-start="0"
  for the runtime to begin playback
- standalone_composition_wrapped_in_template: index.html should not be
  wrapped in <template> (only sub-compositions use that)
- root_composition_missing_html_wrapper: index.html needs <!DOCTYPE html>
  and <html> wrapper for the bundler

Also adds rawSource to LintContext so rules can inspect pre-template-stripped
HTML, and isSubComposition to linter options so rules can distinguish root
from sub-composition files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:45:25 -07:00
Miguel Ángel 43e9252065 feat: add MOV (ProRes 4444) as transparent video output format (#224)
## Summary

- Adds `--format mov` to the render CLI for ProRes 4444 transparent video output
- ProRes 4444 with alpha is the industry standard for transparent video overlays, supported by CapCut, Final Cut, Premiere, DaVinci, and After Effects
- WebM VP9 alpha technically works but is ignored by all major video editors — only browsers decode it
- Adds MOV to the studio export dropdown alongside MP4 and WebM

## Transparency format comparison

| Format | Codec | Alpha | Video editors | Browsers | File size |
| --- | --- | --- | --- | --- | --- |
| **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No (won't play in browser) | Large (~5-40 MB) |
| **WebM** | VP9 | Yes | None (shows black) | Chrome, Firefox | Small (~200 KB) |
| **MP4** | H.264 | No | All | All | Small |

> **Note:** ProRes MOV files do not play in Chromium browsers — they are an intermediate/editing format, not a delivery format. Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify transparency works correctly.

## Changes

- **CLI**: Add `mov` to `--format` validation, examples, and output path logic
- **Engine**: `getEncoderPreset()` returns ProRes 4444 (`yuva444p10le`) for `mov` format; handle `.mov` in `applyFaststart` and `muxVideoWithAudio`; add `pix_fmt` to streaming encoder ProRes path
- **Producer**: Treat `mov` like `webm` for alpha capture (PNG frames, screenshot mode, `forceScreenshot`)
- **Studio**: Add MOV option to export format dropdown and render queue hook
- **Core**: Add `mov` to studio API types, render route, and mime helpers
- **Tests**: Add encoder preset tests for mov format (42 total, all passing)

## Usage

```bash
hyperframes render --format mov --output overlay.mov
```

## Test plan

- [x] `pnpm build` passes
- [x] `pnpm --filter @hyperframes/engine test` — 42 tests pass (2 new for MOV)
- [x] `oxlint` and `oxfmt` clean on all 12 changed files
- [x] End-to-end local render produces ProRes 4444 (`yuva444p12le`) with working alpha
- [x] Docker render with `--format mov` — ProRes 4444 confirmed via ffprobe
- [x] Studio dropdown shows MOV option in built JS
- [x] Transparency verified with [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video)
2026-04-08 03:11:37 +02:00
Miguel Ángel f56b4c8620 fix(core): load head styles/scripts from non-template sub-compositions (#219)
## Summary

Fixes a bug where non-template sub-compositions (full HTML documents loaded via `data-composition-src`) lost all `<head>` styles and scripts. This affected **three code paths**:

1. **Runtime** (`compositionLoader.ts`) — browser preview via iframe fetch
2. **Bundler** (`htmlBundler.ts`) — studio preview HTML bundling (**this was causing the black preview**)
3. Producer fix is in PR #220

## What it fixes

**Eval prompt #25** (iris-wipe) renders entirely black in both the studio preview and rendered video because scene backgrounds (`#EF4444` red, `#3B82F6` blue), positioning, and the GSAP CDN script were all in `<head>` and silently dropped.

### Verified

Rebuilt core, started studio preview, fetched the bundled HTML from `/api/projects/iris-wipe/preview` — confirmed `#scene1 { background: #EF4444 }` and `.scene { position: absolute }` are now present in the output.

## Root cause

All three code paths did the same thing:
```js
const contentHtml = template ? template.innerHTML : bodyEl.innerHTML;
// ^ <head> content is already lost here
```

## Test plan

- [x] All 429 core tests pass
- [x] Studio preview endpoint returns correct bundled HTML with head styles included
- [x] `pnpm --filter @hyperframes/core build` succeeds

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-07 17:17:25 +02:00
Miguel Ángel 0d33238381 fix(core): add lint rule for infinite GSAP repeat (#218)
## Summary

Adds a new lint rule `gsap_infinite_repeat` that flags `repeat: -1` in GSAP timelines as an error. This is a hard enforcement of the skill guardrail added in PR #217.

## What it fixes

The deterministic capture engine (`HeadlessExperimental.beginFrame`) seeks to exact frame times on a paused GSAP timeline. When a timeline contains `repeat: -1`, the timeline duration is infinite, which causes the capture engine to produce incorrect/blurry output.

**Eval prompt #20** (loading-spinner, scored 2.0/5) used `repeat: -1` on a dots animation cycle, producing "a highly compressed and blurry loading animation lacking visual clarity and professional polish."

## Changes

- `packages/core/src/lint/rules/gsap.ts` — new `gsap_infinite_repeat` rule (regex scan for `repeat: -1`)
- `packages/core/src/lint/rules/gsap.test.ts` — 2 new tests (detects infinite repeat, allows finite repeat)

## Test plan

- [x] `pnpm --filter @hyperframes/core test` — all 429 tests pass
- [x] Rule catches `repeat: -1` and reports as error with fix hint
- [x] Rule does not flag `repeat: 4` (finite repeats)
2026-04-07 16:53:50 +02:00
Miguel Ángel 5655dabff6 feat: allow clip animation + ship <hyperframes-player> web component (#209)
## Summary

Two independent initiatives that improve agent DX and expand HyperFrames' reach.

### Initiative 1: Fix the Clip Animation Footgun

- `gsap_animates_clip_element` lint rule now uses smart detection — only errors when GSAP animates `visibility` or `display` on a clip element
- All other properties (opacity, transform, x, y, scale, etc.) are allowed silently
- This was the #1 agent failure in QA (10/10 agents hit it on v0.2.1)

### Initiative 2: `<hyperframes-player>` Web Component

- New `@hyperframes/player` package — zero dependencies, 3.3KB gzipped
- Iframe-based web component with Shadow DOM for perfect isolation
- Video-like API: `play()`, `pause()`, `seek()`, `currentTime`, `duration`, events
- Controls overlay with play/pause, scrubber (mouse + touch), time display, auto-hide
- Full docs page at `docs/packages/player.mdx`

## Before / After

### Clip animation lint

**Before (10/10 agents hit this):**

```
✗ gsap_animates_clip_element: GSAP animation targets a clip element.
  Selector "#title" resolves to element <div id="title" class="clip">.
  The framework manages clip visibility — animate an inner wrapper instead.
  Fix: Wrap content in a child <div> and target that with GSAP.
```

**After (only errors on actual conflicts):**

```
# This passes lint — no error:
tl.from("#title", { opacity: 0, y: -50, scale: 0.8 }, 0);

# This still errors — actual conflict with runtime:
tl.to("#title", { visibility: "hidden" }, 3);
✗ gsap_animates_clip_element: GSAP animation sets visibility on a clip element.
  Fix: Remove the visibility/display tween. Use opacity for fade effects.
```

### Embeddable player

**Before:** No way to embed a composition in a web page.
**After:**

```html
<script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script>
<hyperframes-player src="./composition/index.html" controls></hyperframes-player>
```

```js
const player = document.querySelector('hyperframes-player');
player.play();
player.pause();
player.seek(2.5);
player.addEventListener('ready', (e) => console.log('Duration:', e.detail.duration));
```

## Test plan

- [x] 427 core tests pass (20 GSAP lint tests with smart detection)
- [x] 7 player tests pass (formatTime + element registration)
- [x] TypeScript compiles cleanly (core + player)
- [x] Lint: GSAP animating clip with safe props → 0 errors
- [x] Lint: GSAP animating clip with `visibility` → 1 error (correct)
- [x] Player builds to 3.3KB gzipped ESM
- [x] Lockfile updated for CI
- [x] Docs page added at `docs/packages/player.mdx`
2026-04-06 19:59:39 +02:00
Miguel Ángel baa3d813be fix: address QA report P0-P2 issues for 10/10 agent experience (#208)
## Summary

Addresses all 8 issues from the QA report to improve agent and user experience.

### P0 — Must Fix

- **Blank template broken captions**: Removed `compositions/captions.html` and its reference from the blank template. Every agent (10/10) hit 404 errors during render.
- **Inner-wrapper example**: Added a clear structural comment in the blank template showing the correct `class="clip"` + inner wrapper pattern.
- **Sub-composition introspection**: `hyperframes compositions` now reads external HTML files referenced via `data-composition-src` and shows their real duration/element count instead of `0.0s / 0 elements`.

### P1 — Fix Soon

- **Font mapping warnings**: Now lists all mapped fonts, suggests alternatives (use a mapped font, add @font-face, install locally), and links to docs.
- **Browser 404s**: Non-font "Failed to load resource" 404s now prefixed with `[non-blocking]` instead of `[Browser:ERROR]`.
- **Render concurrency**: Default workers increased from `cores/2` (max 4) to `cores*3/4` (max 6). Added `--concurrency` alias.

### P2 — Nice to Have

- **Transform conflict fix suggestion**: `gsap_css_transform_conflict` now suggests exact GSAP property replacements (e.g., `xPercent: -50, yPercent: -50`).
- **Upgrade --yes**: Now actually runs the install instead of just printing the command.

## Before / After

### Font mapping warning

**Before:**

```
[Compiler] No deterministic font mapping for: DM Sans
```

**After:**

```
[Compiler] No deterministic font mapping for: DM Sans
  Mapped fonts: arial → inter, courier → jetbrains-mono, ...
  To fix, pick one:
    1. Use a mapped font name instead (see list above)
    2. Add a @font-face block in your HTML with a local or hosted font file
    3. Install the font locally on the render machine (Docker: add to Dockerfile)
    4. Add an alias to FONT_ALIASES in deterministicFonts.ts (for contributors)
```

### Browser 404s during render

**Before:** `[Browser:ERROR] Failed to load resource: the server responded with a status of 404`
**After:** `[non-blocking] Failed to load resource: the server responded with a status of 404`

### Transform conflict lint

**Before:** `Fix: Remove the transform from CSS and use tl.fromTo...`
**After:** `Fix: Remove transform: translate(-50%, -50%) from CSS and replace with GSAP properties: xPercent: -50, yPercent: -50`

### Compositions command

**Before:**

```
overlay   0.0s   1920×1080   0 elements
```

**After:**

```
overlay   8.0s   1920×1080   2 elements ← compositions/overlay.html
```

## Test plan

- [x] All 422 core tests pass
- [x] TypeScript compiles cleanly (all 4 packages)
- [x] Full monorepo build succeeds
- [x] `hyperframes init --template blank` ships without broken captions reference
2026-04-06 18:56:39 +02:00
Vance Ingalls d8bffd41f9 feat(lint,skills): add caption/audio-reactive lint rules and skill guidance (#207)
## What

Bumped all package versions to `0.2.2-alpha.4` and added five new lint rules for caption and GSAP animation quality checks.

## Why

The new lint rules address common issues in HyperFrames compositions:
- Caption overflow clipping when emphasis words are scaled above 1.0x
- Text shadow artifacts on caption group containers with semi-transparent children
- Mismatch between fitText maxWidth and scaled word dimensions
- Imperceptible audio reactivity from single tweens instead of time-sampled animations
- Scene layer visibility conflicts when relying only on opacity tweens

## How

Added three new caption-specific lint rules in `captions.ts`:
- `caption_overflow_clips_scaled_words` - detects `overflow: hidden` on caption containers when scripts scale words above 1.0x
- `caption_textshadow_on_group_container` - flags textShadow tweens applied to group containers instead of individual words
- `caption_fittext_scale_mismatch` - calculates effective width from fitText maxWidth × max scale factor and warns when it exceeds safe bounds

Added two new GSAP lint rules in `gsap.ts`:
- `audio_reactive_single_tween_per_group` - identifies audio-reactive captions using peak values instead of time-sampled loops
- `scene_layer_missing_visibility_kill` - detects multi-scene compositions missing hard visibility kills after opacity exit tweens

Enhanced documentation with new mask reveals guide and updated existing skills with overflow handling, scene management, and audio reactivity best practices.

## Test plan

- [x] Lint rules tested against existing composition patterns
- [x] Documentation updated with new techniques and constraints
- [x] Version bumps applied consistently across all packages
2026-04-03 10:52:06 -07:00
Miguel Ángel e2c8ed5d83 refactor(core): replace cheerio with linkedom to drop deprecated whatwg-encoding (#187)
## Summary

- `cheerio` pulls `encoding-sniffer` → `whatwg-encoding@3.1.1` (deprecated), causing a warning on every `npm install -g hyperframes`
- `linkedom` was already bundled into the CLI via tsup `noExternal` and has zero deprecated transitive deps
- Rewrote `htmlBundler.ts` and `subComposition.ts` to use standard DOM APIs via `linkedom`
- Added a `parseHTMLContent` helper that wraps HTML fragments in a full document structure (required for `linkedom` to populate `document.body`)
- Removed `cheerio` from `cli` dependencies and tsup `external` list
- Replaced `cheerio` with `linkedom` in `core` `optionalDependencies`

## Test plan

- [x] All 411 tests pass (`bun run test` in `packages/core`)
- [x] Full monorepo build succeeds (`bun run build`)
- [x] TypeScript typecheck passes
2026-04-03 16:06:30 +02:00
Miguel Ángel 4bb01fd1b6 feat(lint): add rules for non-deterministic code, missing clip class, overlapping tracks, and rAF detection (#193)
## Summary

Adds 4 new lint rules found missing during comprehensive E2E testing:

- **`non_deterministic_code`** (error) — detects `Math.random()`, `Date.now()`, `new Date()`, `performance.now()`, `crypto.getRandomValues()` in scripts. Confirmed: two renders with Math.random() produced different checksums.
- **`timed_element_missing_clip_class`** (warning) — flags elements with `data-start`/`data-duration` but no `class="clip"`. Without it, elements are visible forever instead of only during their time window.
- **`overlapping_clips_same_track`** (error) — detects clips on the same `data-track-index` with overlapping time ranges.
- **`requestanimationframe_in_composition`** (warning) — warns that rAF-based animations don't sync with frame capture. Discovered when Vivus.js (rAF-based) produced incorrect output.

12 new tests, all passing.

**Part 1 of 5** in a stacked PR series fixing E2E test findings.

## Test plan

- [x] 12 new tests (3 per rule: positive, negative, edge case)
- [x] Full suite: 56 pass in rule tests
- [x] `npx tsx scripts/lint-skills.ts` — no issues
2026-04-02 14:09:50 -07:00
Vance Ingalls cb0b17062a feat(skills): add marker-highlight skill for animated text highlighting (#190)
## Summary

- **New skill:** **`marker-highlight`** — integrates [MarkerHighlight.js](https://github.com/Robincodes-Sandbox/marker-highlight) into HyperFrames compositions. Canvas-based animated text highlighting with 5 drawing modes: marker pen, circle, burst, scribble, and sketchout.
- **Studio fix:** added missing `captionSync` to useEffect dependency array (oxlint exhaustive-deps)
- **Studio fix:** `loadOverrides` now checks `res.ok` before parsing, preventing 404 console noise on projects without captions

## Skill details

The skill documents the non-obvious GSAP integration pattern discovered during development:

1. **One highlighter per container** — the library clears ALL `.highlight` divs from the shared parent on init, so multiple instances on sibling marks conflict
2. **`data-color`** **\+** **`data-original-bgcolor`** — prevents the CSS background-color flash that occurs when the library reads and clears the mark's background
3. **Canvas pre-draw + clear + reanimate** — `animate: false` pre-draws statically, canvases are hidden, then cleared and shown with `reanimateMark()` at trigger time for clean animated reveals
4. **`onReverseComplete`** **for rewind** — hides highlight divs when the timeline seeks backward past the trigger point

## Test plan

- [ ] `npx hyperframes lint` passes on test-composition
- [ ] Studio preview shows marker highlight on "something" at 1s, circle on "love" at 2.2s
- [ ] Rewind past trigger points hides highlights
- [ ] No 404 console errors for caption-overrides.json on non-caption projects



[Screen Recording 2026-04-02 at 1.56.30 AM.mov <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.com/user-attachments/thumbnails/53b03f4e-538e-477a-b738-7a033b99a84e.mov" />](https://app.graphite.com/user-attachments/video/53b03f4e-538e-477a-b738-7a033b99a84e.mov)



🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-02 13:48:24 -07:00
Vance IngallsandClaude Opus 4.6 5e2781b459 fix(studio): address caption designer PR feedback (#200)
* fix(studio): address caption designer PR feedback

Fixes from review comments on feature/caption-designer (#180):

- fix(generator): guard named colors in hexToRgba — "red", "transparent"
  no longer produce NaN rgba values
- fix(sync): log auto-save failures instead of silently swallowing them
- fix(sync): check res.ok before parsing caption-overrides response
- refactor(components): extract Section, Row, inputCls into shared.tsx
  to eliminate duplication between CaptionPropertyPanel and
  CaptionAnimationPanel
- fix(store): replace non-deterministic Date.now()+Math.random() ID with
  counter-based group IDs
- fix(store): read selectedGroupId from state param instead of get() to
  avoid stale reads in batched set() calls
- fix(overlay): remove cssScale multiplier from getBoundingClientRect
  coords — the browser already accounts for CSS transforms
- docs(parser): add comment explaining the lazy ];  regex assumption

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): address remaining caption designer feedback

Overlay: handle both per-word spans (generator output) and grouped text
nodes (existing templates). Wraps text nodes into individual spans on
demand so the overlay can target words in any caption format.

Property panel: add Typography (font, size, weight, spacing) and Color
(color, active, dim, opacity) sections alongside existing Position and
Transform controls.

Timeline: move caption timeline into a dedicated flex-shrink-0 section
below the main timeline tracks instead of inside the scrollable area.
Gives it fixed 60px height that's always visible.

Caption overrides: classify color tweens by comparing target color to
the dim baseline instead of relying on timeline position order. This
handles compositions with custom color tweens correctly.

App.tsx: remove polling interval, rely on runtime postMessage events
for caption detection. Add clarifying comment on why useEffect is
appropriate (external event subscription).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): restore cssScale in overlay coordinate conversion

getBoundingClientRect() on iframe-internal elements returns coordinates
in the iframe's native resolution (1920x1080), not the CSS-scaled
display size. The cssScale multiplier is needed to convert to parent
window coordinates. The earlier removal was incorrect — it only worked
at 1:1 scale.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): fix reversed scaling on left-side corner handles

Scale interaction used horizontal dx from start position, which goes
negative when dragging left handles outward. Now uses distance from box
center — dragging away from center increases scale regardless of which
corner handle is used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): make rotation respond to horizontal drag only

Rotation handle sits directly above the word, so atan2-based rotation
barely responds to left/right movement. Replace with linear horizontal
mapping: drag right = clockwise, drag left = counter-clockwise,
200px = 90 degrees. Vertical movement is ignored.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): remove animation tab and typography/color from property panel

Keep only Position (X, Y) and Transform (Scale, Rotation) controls.
Remove tab switcher UI since there's only one view now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix oxfmt formatting in CLAUDE.md and captions skill docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 11:38:35 -07:00
Vance Ingalls d36c1785b9 feat(captions): energy-based technique selection and mandatory quality checks (#176)
## Summary

- Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits
- Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time, no per-frame callbacks needed
- Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model)
- Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility
- Add multilingual model guidance and decision tree for model selection

## Test plan

- [ ] Skill files render correctly as markdown
- [ ] Cross-references between SKILL.md, dynamic-techniques.md, and transcript-guide.md resolve correctly
- [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-02 00:47:43 -07:00
Vance Ingalls 37404f23da feat(whisper+captions): language detection, audio-reactive captions, multilingual defaults (#175)
## Summary

**Whisper improvements:**
- Auto-detect language and switch from `.en` to multilingual model when needed
- Detect speech onset in WAV to strip hallucinated words before speech begins
- Merge whisper-cpp token fragments: contractions (`didn` + `'t` → `didn't`), split capitals (`C` + `aught` → `Caught`), dropped-g (`shin` + `in'` → `shinin'`)
- Interpolate zero-duration word clusters for reliable karaoke timing

**Captions skill updates (folded from #176):**
- Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits
- Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time
- Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model)
- Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility

**Multilingual defaults (folded from #186):**
- Default whisper model changed from `small.en` to `small` to prevent silent translation of non-English audio
- Added non-negotiable language rule to captions skill

## Test plan

- [ ] `pnpm test` passes (contraction merging, fragment merging, zero-duration interpolation, speech onset)
- [ ] Transcribe non-English audio — verify it transcribes in original language, not translates
- [ ] Skill files render correctly, cross-references resolve
- [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-02 00:08:41 -07:00
Miguel Ángel 1efb23dfae fix(producer): inline CDN scripts for offline render and detect black video (#182)
## Context

`npx hyperframes render` fails to load assets and produces all-black video. Root causes:

1. CDN scripts (GSAP, Lottie) are left as `<script src="https://...">` in compiled HTML — the headless browser must fetch them over the network, which fails in Docker, CI, and firewalled environments
2. Assets referenced from outside the project directory (e.g. `../shared-assets/hero.png`) 404 because the file server only serves from `projectDir`
3. When GSAP fails to load, the timeline never registers, all `.clip` elements stay `visibility: hidden`, and every frame is black — with no diagnostic output
4. The linter reports `missing_gsap_script` when GSAP is bundled inline (no `<script src>` tag), which blocks users from working around the CDN issue

## What changed

### 1\. CDN script inlining (`htmlCompiler.ts`)

`compileForRender` now downloads all external `<script src="https://...">` tags at compile time and inlines their content into the HTML. Rendering no longer needs network access.

| Before | After |
| --- | --- |
| `<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>` stays in HTML, browser must fetch at runtime | Script downloaded during compilation, embedded as `<script>/* inlined: https://... */\n...code...</script>` |
| Docker/CI render → `net::ERR_NAME_NOT_RESOLVED` → black video | Render works fully offline |
| No feedback when CDN fails | `[Compiler] WARNING: Failed to download CDN script: ... Consider bundling it locally` |

### 2\. External asset copying (`htmlCompiler.ts`, `renderOrchestrator.ts`)

After compilation, the HTML is scanned for `src`, `href`, and CSS `url()` references that resolve outside `projectDir`. These files are copied into the compiled output directory so the file server can serve them.

| Before | After |
| --- | --- |
| `background-image: url(../shared-assets/hero.png)` → 404 (file server can't serve outside `projectDir`) | Asset detected, copied to compiled dir, path rewritten → serves correctly |
| `<img src="../shared-assets/logo.png">` → 404 | Same fix — works for all `src`/`href` attributes and CSS `url()` |

### 3\. Black video diagnostics (`renderOrchestrator.ts`)

When composition duration is 0 (which would produce a black video), the error now probes the browser for diagnostics instead of a generic message.

| Before | After |
| --- | --- |
| `Invalid composition duration: 0. Check that GSAP timelines are registered.` | `Composition duration is 0 — this would produce a black video.\n\nDiagnostics:\n  - GSAP is not loaded — CDN script may have failed to download. Bundle GSAP locally...\n  - Browser: [Browser:PAGEERROR] gsap is not defined` |
| Asset 404s during page load silently logged | `[Render] Asset load failure: ...` + `[WARN] Browser encountered network failures during page load` |

### 4\. Linter: recognize inline GSAP (`core/lint/rules/gsap.ts`)

The `missing_gsap_script` rule now recognizes GSAP bundled inline — matching the producer's inlining comment (`/* inlined: ...gsap... */`), GSAP library internals (`_gsScope`, `GreenSock`), and large inline scripts (>5KB) referencing gsap.

| Before | After |
| --- | --- |
| User inlines GSAP → linter errors with `missing_gsap_script` | Inline GSAP detected, no false error |
| Producer inlines CDN → linter errors on the compiled HTML | Producer's `/* inlined: ... */` comment recognized |

## Test plan

- [x] `pnpm build` passes
- [x] Core tests pass (410/410, +2 new)
- [x] **Reproduced baseline failures on** **`main`**: CDN scripts not inlined, external assets 404, no diagnostics
- [x] **Verified fixes**: CDN script inlined, external assets copied and served, diagnostics printed
- [x] Render with working CDN → `[Compiler] Inlined CDN script: ...` → render succeeds
- [x] Render with broken CDN → `[Compiler] WARNING: Failed to download CDN script` + browser errors surfaced
- [x] Render with assets outside project dir → `[Compiler] Found 1 asset(s) outside project directory` → assets served correctly
- [x] Linter with inline GSAP → no `missing_gsap_script` false positive
2026-04-01 23:15:00 +02:00
Miguel Ángel 2213e9f7a4 refactor(lint): split monolithic test file into per-rule-module test files (#181)
## Summary

- Splits the 872-line `hyperframeLinter.test.ts` into focused per-domain test files matching the rule module structure from #170
- Tests still use the public API (`lintHyperframeHtml`) — no internals exposed
- Fixes misplaced tests: `gsap_css_transform_conflict` tests were inside `describe("lintScriptUrls")`, caption/adapter tests were inside `describe("template_literal_selector rule")`

## New structure

```
src/lint/
  hyperframeLinter.test.ts   — orchestrator integration + lintScriptUrls (8 tests)
  rules/
    core.test.ts             — root attrs, timeline registry, host id (6 tests)
    media.test.ts            — duplicate ids, missing id/src, preload (6 tests)
    gsap.test.ts             — clip element, transform conflict, missing script (12 tests)
    captions.test.ts         — caption exit, overflow, relative position (5 tests)
    composition.test.ts      — external script, template literal selector (5 tests)
    adapters.test.ts         — missing Lottie/Three.js scripts (8 tests)
```

## Test plan

- [x] 404 tests pass (402 before — 2 extra from previously misplaced adapter tests now correctly counted)
- [x] Lint + format clean
2026-04-01 18:30:13 +02:00
Miguel Ángel 1681350ac4 fix(preview): rewrite sub-composition asset urls in styles (#174)
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting

## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.

## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`

## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
2026-04-01 18:13:38 +02:00
Miguel Ángel 110ea12597 fix(core,engine,producer): handle id-less media in sub-composition renders (#96)
## What

Move the id-less media fix into the shared timing compiler so producer can resolve durations for sub-composition videos before inlining, then carry the merged result through engine parsing, regression coverage, and the regression Docker image used in CI.

This PR now does five concrete things:

- assigns stable ids to id-less media in core `compileTimingAttrs()` so unresolved duration injection can target them
- keeps the engine-side `parseVideoElements()` support for `video[src]` plus the newer `data-duration` / natural-duration fallback from `main`
- makes producer prefer sub-composition media metadata over the later inlined-document parse when the same media id appears in both places
- makes `sub-composition-video` a runnable regression test by fixing its metadata and checking in the missing `output/compiled.html` snapshot
- removes the stale `pnpm-workspace.yaml` copy step from `Dockerfile.test`, so regression CI builds the Bun-based test image from the current workspace layout

## Why

- media without an explicit `id` could not participate in unresolved-duration resolution early enough
- producer could lose the resolved sub-composition timing by overwriting it with the later inlined parse
- the regression fixture intended to cover this case was not actually running in CI because its `meta.json` was incomplete and the required compiled snapshot was missing
- the regression image definition still expected a deleted `pnpm-workspace.yaml`, so GitHub Actions failed before the test shard could start

Putting the id-generation step in core makes the behavior reusable instead of relying on producer-only HTML patching.

## How

### Shared compiler

- core `compileTimingAttrs()` now auto-assigns stable ids to id-less `video` / `audio` tags
- those generated ids are returned in `unresolved`, so `injectDurations()` can add `data-duration` and `data-end` to the same media element later in the pipeline
- added core tests that cover auto-id assignment and duration injection for generated ids

### Producer

- when producer combines `subVideos` / `subAudios` with the media re-parsed from the final inlined HTML, it now lets the sub-composition metadata win
- this preserves the resolved/clamped timing already computed for nested media instead of overwriting it with the later parse
- `sub-composition-video` now has valid regression metadata and a checked-in `output/compiled.html` snapshot so CI actually executes it

### Engine

- resolved the merge conflict in `videoFrameExtractor` by keeping the broader `video[src]` parsing from this branch and the `data-duration` / natural-duration fallback that landed on `main`
- added a focused engine unit test for videos without ids

### CI image

- `Dockerfile.test` now copies only `package.json` and `bun.lock` at the workspace root before `bun install --frozen-lockfile`
- this matches the current monorepo layout and removes the obsolete pnpm-era dependency on `pnpm-workspace.yaml`

## Test plan

- [x] `bun run --filter @hyperframes/core test`
- [x] `bun run --filter @hyperframes/engine test`
- [x] `bun run --filter @hyperframes/producer test --update --sequential sub-composition-video`
- [x] `bun run --filter @hyperframes/producer test --sequential sub-composition-video`
- [x] Browser check with `agent-browser` against the compiled fixture page (`http://127.0.0.1:8123/compiled.html`)
- [x] Clean tracked-only Docker build of `Dockerfile.test` with the PR version of the file applied

## Notes

- Latest regression workflow is green on `main`, but before this PR the `sub-composition-video` fixture was being skipped by the harness rather than exercised end to end.
- The CI Docker fix was validated from a tracked-only export to avoid local untracked worktree artifacts affecting the result.
2026-04-01 02:59:22 +02:00
Miguel Ángel 5421c23fff refactor(lint): break 1,314-line monolith into focused rule modules with plugin system (#170)
## Summary

- Breaks `hyperframeLinter.ts` from 1,314 lines (one massive function) into a plugin system of focused rule modules
- Introduces `LintContext` — HTML is parsed once and shared across all rules
- Adds `LintRule<TContext>` type as the formal contract for rules
- Public API unchanged: `lintHyperframeHtml`, `lintMediaUrls`, `lintScriptUrls` signatures identical

## New file structure

```
src/lint/
  utils.ts          — shared types (OpenTag, ExtractedBlock), regex constants, helpers
  context.ts        — LintContext type + buildLintContext() factory
  rules/
    core.ts         — structural rules (root attrs, timeline registry, script syntax)
    media.ts        — media element rules (duplicate ids, video pitfalls, placeholder URLs, etc.)
    gsap.ts         — GSAP rules + GSAP-specific parsing utils
    captions.ts     — caption rules
    composition.ts  — timed element, deprecated attrs, template literal selector, external scripts
    adapters.ts     — Lottie + Three.js missing-script rules (from PR #149)
  hyperframeLinter.ts — orchestrator only (~200 lines, down from 1,314)
```

## Adding a new adapter rule going forward

1. Create `src/lint/rules/my-adapter.ts` exporting `myAdapterRules: LintRule[]`
2. Import and spread into `ALL_RULES` in `hyperframeLinter.ts`

## Test plan

- [x] All 402 core tests pass unchanged
- [x] Full workspace build clean (`pnpm build`)
- [x] TypeScript strict mode clean (`pnpm tsc --noEmit`)
2026-04-01 00:50:13 +02:00
Miguel Ángel 5f3488e996 feat(studio): drag-drop assetfile/folder and asset import anywhere in the studio (#155)
## Summary

- Add `/api/projects/:id/upload` endpoint for multipart file uploads with automatic dedup naming
- Wire `onImportFiles` from Assets tab "Import media" button through to the upload API
- Add global drag-drop overlay — drop media files **anywhere** in the studio, not just the Assets panel
- Files that already exist get `(2)`, `(3)` suffixes instead of overwriting
- Support uploading into subdirectories via `?dir=` param — dropping on a folder imports there
- Make folders draggable in the file tree + support drop-to-root
- Add `bodyLimit` middleware for early rejection of oversized payloads
- Surface skipped/failed uploads via toast notification instead of console-only

Addresses feedback: _"Wish I could upload/drag-drop assets directly in the Studio (music, images, video) like a CapCut media panel"_

## Test plan

- [x] Open studio, drag an image/video/audio file onto any part of the UI
- [x] Verify the drop overlay appears with "Drop files to import" message
- [x] Drop the file — verify it appears in the Assets tab and file tree
- [x] Drop a file with the same name — verify it gets a `(2)` suffix
- [x] Click "Import media" button in Assets tab — verify file picker works
- [x] Import multiple files at once via drag-drop
- [x] Drop a file onto a nested folder in the file tree — verify it lands in that folder
- [x] Drag a folder in the file tree and drop it on another folder or root — verify it moves
- [x] Drop a file >500MB — verify toast notification appears
- [x] Verify drag overlay doesn't get stuck when dragging over nested UI elements
2026-04-01 00:21:34 +02:00
Miguel Ángel 7265d0adfd fix(producer): rewrite relative asset paths when inlining sub-compositions (#166)
## Summary

- **Fixes**: `<img src="../icon.svg">` and similar `../` relative asset references in sub-compositions resolve to 404 after inlining into the root document
- **Unifies**: Both `hyperframes preview` (core bundler) and `hyperframes render` (producer) now use the same shared logic — no duplication

## Root cause

When inlining a sub-composition at `compositions/scene.html` into root `index.html`, a relative path like `../icon.svg` is correct from `compositions/` (it points to project root) but after inlining, `../` escapes the project directory.

## Fix

Extracts path rewriting into a shared `rewriteSubCompPaths.ts` utility in `@hyperframes/core`, used by both the bundler and the producer. **Only rewrites paths starting with** **`../`** — plain relative paths like `assets/foo.svg` are already correct from the root perspective and must not be rewritten (this was the regression cause in `overlay-montage-prod`: sub-composition asset refs like `assets/notch.svg` were incorrectly being rewritten to `compositions/assets/notch.svg`).

## Regression fix

The earlier version of this fix (now in history) rewrote ALL relative paths including `assets/foo.svg`, breaking the `overlay-montage-prod` regression test. This PR fixes that by scoping rewrites to `../`\-prefixed paths only.

## Test plan

- [x] `../icon.svg` in sub-composition renders correctly in both preview and render
- [x] `assets/foo.svg` (no `../`) in sub-composition still resolves correctly — not rewritten
- [x] `overlay-montage-prod` regression test passes
- [x] All other regression shards pass
2026-04-01 00:13:55 +02:00
Miguel Ángel ecb590d444 feat(studio): full IDE-like file management (#147)
## Summary

- **API**: POST (create), DELETE (delete), PATCH (rename/move), POST duplicate endpoints with null-byte sanitization
- **FileTree**: right-click context menu with New File, New Folder, Rename, Duplicate, Delete
- **Drag-and-drop**: move files between folders with visual feedback and subtree guard
- **Inline editing**: rename/create inputs with filename validation
- **Header actions**: quick New File / New Folder buttons in the FILES header

Stacks on top of `feat/studio-code-quality`.

## Test plan

- [x] Right-click file → Rename, Delete, Duplicate all work
- [x] Right-click folder → New File, New Folder, Delete work
- [x] Drag file from one folder to another
- [x] Create file with invalid name (`../foo`, `a/b`) → rejected client-side
- [x] Delete currently-edited file → editor clears
- [x] Studio build succeeds
2026-03-31 20:03:05 +02:00
Vance IngallsandClaude Opus 4.6 9cbfec1eca feat(skills): add hyperframes-cli skill (#154)
* feat(skills): add hyperframes-cli skill for CLI workflow guidance

Adds a new skill that teaches AI agents how to use the HyperFrames CLI
(init, lint, dev, render, doctor). Previously, agents had no way to
discover the CLI — the compose-video skill only covered HTML authoring.
This led to agents searching for binaries, finding the monorepo, and
running bun run studio manually instead of using npx hyperframes dev.

Also registers the skill in init.ts so new projects get it bundled
alongside hyperframes-compose and hyperframes-captions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(cli): rename dev command to preview

The command starts a preview server — "preview" describes what users
are doing more accurately than "dev". Updates the command name, file
name, all CLI references, docs, skills, and template CLAUDE.md.

22 files updated across CLI source, docs, skills, and templates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): replace stale dev reference with preview in CLI skill

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(docs): catch remaining dev references missed in rename

- testing-local-changes.mdx: two inline command examples
- troubleshooting.mdx: anchor link #dev → #preview, "dev server" → "preview server"
- cli.mdx: "dev server" → "preview server"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 00:30:55 -07:00
ef6225da1d feat(core): add fitTextFontSize utility for pixel-accurate text measurement (#152)
Add @chenglou/pretext dependency and fitTextFontSize() utility that uses
canvas measureText to compute the largest font size that fits text within
a given width. Replaces character-count heuristics with actual font-aware
measurement.

- New fitTextFontSize() in @hyperframes/core/text, exposed on window.__hyperframes
- Generalized for all text elements (captions, titles, etc.), not just captions
- Unit tests (mocked pretext) + browser integration test (real Chromium canvas)
- Updated captions skill docs with usage, exit guarantee, and self-lint patterns

Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 20:14:13 -07:00
Miguel Ángel dac304ed9f refactor(studio): code quality — 22 findings, dead code removal, App.tsx split (#144)
## Summary

Full code quality review of the studio package, fixing 22 of 25 findings. Removes dead code, extracts modules from App.tsx, fixes accessibility and performance issues.

## Critical fixes (3)

- **`aria-valuenow`** on seek bar now updates imperatively via `liveTime.subscribe` — screen readers previously always reported position 0
- **Speed menu** closes on outside click (was permanently stuck open)
- **RenderQueue auto-scroll** moved from render phase to `useEffect` (was violating React render purity via `queueMicrotask` during render)

## Dead code removed (-331 lines)

| File | Lines | Why dead |
|---|---|---|
| `PreviewPanel.tsx` | 180 | Replaced by NLELayout + NLEPreview |
| `useCodeEditor.ts` | 80 | Exported but never imported |
| `formatTick` alias | 2 | Deprecated, unused |
| `onClipChange` prop | 5 | Declared, never used |
| `trackH` prop | 5 | Declared, never used |
| `editRange*` + updaters in store | 60 | Never read or written |

## App.tsx extraction

| Extracted to | Lines | What |
|---|---|---|
| `components/LintModal.tsx` | 130 | Lint results modal + LintFinding type |
| `components/MediaPreview.tsx` | 75 | Image/video/audio/font file previewer |
| `utils/mediaTypes.ts` | 15 | Shared regex constants (App.tsx and AssetsTab.tsx had diverged copies) |

## Performance fixes

- `useMemo` for `compositions`/`assets` derivation from `fileTree`
- `useMemo` for `buildTree(files)` in FileTree
- Debounced `handleContentChange` PUT (600ms — was firing on every keystroke)
- CompositionsTab iframe hover debounced (300ms — was mounting immediately)
- `VideoFrameThumbnail` re-extracts frame when `src` prop changes

## Not addressed (3 — low priority)

- #6: SystemIcons consolidation (large refactor across many files)
- #16-17: Overlay dismiss pattern standardization
- #18: Inline SVG → Phosphor replacement (gradual, per-PR)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-31 04:21:12 +02:00
James RussoandClaude Opus 4.6 2f99e33bbe feat(cli,core): standalone transcribe command, transcript normalization, caption lint rules (#151)
* feat(cli,core): add standalone transcribe command, transcript normalization, and caption lint rules

- Add `hyperframes transcribe` command for transcribing audio/video and importing
  existing transcripts (SRT, VTT, OpenAI Whisper API JSON, whisper.cpp JSON)
- Add transcript format normalizer (normalize.ts) with auto-detection and
  conversion to standard [{text, start, end}] word arrays
- Upgrade default whisper model from base.en to small.en for better accuracy
- Add --model and --language flags to both `transcribe` and `init` commands
- Extract shared patchCaptionHtml() to eliminate duplication between init.ts
  and transcribe.ts (init.ts reduced by ~55 lines)
- Add 3 caption lint rules: caption_exit_missing_hard_kill,
  caption_text_overflow_risk, caption_container_relative_position
- Update captions skill with model guide, format docs, music guidance,
  text overflow prevention, caption exit guarantee pattern
- Expand captions skill trigger to cover lyrics, karaoke, lyric videos

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(cli): add transcribe command and --model/--language flags to CLI docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): fix blank template lint issues

- blank/index.html: remove data-start from video (was nested in timed parent),
  add class="clip" for initial hidden state
- blank/captions.html: add max-width + overflow:hidden to prevent text clipping,
  add tl.set hard kill after exit tween to prevent stuck captions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add lint-after-edit rule to repo and project CLAUDE.md

Agents must run `npx hyperframes lint` after editing compositions.
Also expand captions skill description in project template.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format _shared/CLAUDE.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 18:58:06 -07:00
Miguel Ángel 6d54217e74 feat(core): support inline template compositions (#146)
## Summary

- Adds `loadInlineTemplateCompositions` to the runtime to handle compositions defined inline via `<template id="X-template">` paired with empty host elements that have `data-composition-id="X"` but no `data-composition-src`
- Updates the HTML bundler to inline template content into matching hosts during compilation
- Users can now define sub-compositions inline instead of requiring separate files in `compositions/`

### Before

```html
<!-- This showed nothing — the template was inert and the host was empty -->
<template id="logo-reveal-template">
  <div data-composition-id="logo-reveal" data-width="1920" data-height="1080">
    <style>...</style>
    <script>/* animation */</script>
  </div>
</template>

<div data-composition-id="logo-reveal" data-start="0" data-duration="10"
     data-width="1920" data-height="1080"></div>
```

### After

The runtime detects the matching `<template>` and injects its content into the host element — styles hoisted to `<head>`, scripts executed, dimensions copied. Works in both preview and render.

## Test plan

- [x] 9 new unit tests for `loadInlineTemplateCompositions` (basic mount, no-op cases, style/script injection, dimensions)
- [x] 3 new unit tests for bundler inline template handling
- [x] All 384 existing tests pass
2026-03-31 03:46:23 +02:00
Miguel Ángel 1aca29a414 fix(core,cli): improve lint output - JSON flag, info/warning counts, severity display (#134)
## Summary

- Respect `--json` flag on all lint exit paths so agents always get machine-readable output
- Separate `infoCount` from `warningCount` in linter results (was conflated)
- Display `info` vs `warning` severity distinctly in lint output
2026-03-31 02:45:19 +02:00
Vance IngallsandClaude Opus 4.6 a97dc75702 fix(lint): detect GSAP animations targeting clip elements (tab crash) (#114)
* fix(lint): detect GSAP animations targeting clip elements (tab crash)

The runtime manages clip visibility via inline styles. When GSAP also
writes inline styles on the same element, both systems trigger style
recalculations every frame, creating a runaway loop that crashes the
browser tab.

New rule gsap_animates_clip_element (error severity):
- Builds map of all elements with class="clip" (by id and class)
- Checks if any GSAP selector resolves to a clip element
- Nested selectors like "#overlay .title" are correctly ignored
- Merged into existing GSAP script loop (no redundant parsing)

* fix: remove non-null assertions and add missing test coverage

- Replace `!` assertions with optional chaining in lint.ts and tests
- Add shouldBlockRender tests for --strict-all without --strict
- Add clip element test for class-only detection (no id)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use optional chaining for array access in lintProject tests

TypeScript's strict mode flags array indexing as possibly undefined.
Use optional chaining and fallbacks instead of non-null assertions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 15:32:41 -07:00
Miguel Ángel 1230657ed0 fix(studio,runtime,engine,compiler): 8 bug fixes — audio, render, timeline, Lottie, thumbnails, video render (#133)
## Summary

**Original 5 bugs fixed:**

- **Bug 1 — Audio silent after seek**: Added `Accept-Ranges` / `Content-Length` + `206 Partial Content` to the static asset server for byte-range seeking.
- **Bug 2 — Download 404 after restart**: Render list endpoint now registers on-disk renders into the in-memory job map.
- **Bug 3 — Timeline stops at GSAP end**: `resolveRootTimelineFromDocument` pads the GSAP timeline to match `data-duration` when the composition declares longer.
- **Bug 4 — Render stuck at 0%**: Store `jobState` reference (not spread copy) so async progress mutations reach the SSE stream.
- **Bug 5 — Lottie missing in preview/render**: Two fixes — (a) moved Lottie adapter before GSAP so `onUpdate` wins; (b) fixed bundler silently dropping external CDN `\<script src>` tags from sub-compositions (root cause: `$content(s).html()` returns `""` for external scripts).

**3 additional bugs fixed:**

- **Bug 6 — Blank thumbnails outside monorepo**: Implemented `generateThumbnail` in the CLI adapter using Puppeteer.
- **Bug 7 — Video empty in rendered sub-compositions**: Fixed `parseVideoElements` selector from `video[id][src]` to `video[src][data-start]` + auto-assign IDs.
- **Render errors**: Failed renders now show their error message in the renders panel.

## Commits

| Commit | Description |
| --- | --- |
| `3951c6f` | fix(studio): store render job reference instead of snapshot copy |
| `f331c30` | fix(studio): make previously-completed renders downloadable after restart |
| `a5e2d04` | fix(studio): add range request support for audio/video seeking in preview |
| `f24317a` | fix(runtime): pad GSAP timeline to data-duration when composition declares longer duration |
| `7cf38ca` | fix(runtime): fix Lottie adapter conflicting with GSAP-driven animations |
| `bc99209` | fix(studio): surface render error messages in the renders panel |
| `8fc9e8b` | fix(cli): implement generateThumbnail in studio adapter |
| `90277ea` | fix(engine): render videos inside sub-compositions that lack an explicit id |
| `f5bb579` | fix(compiler): preserve external CDN scripts from sub-compositions in bundle |

## Test plan

- [x] `golden-lyric-video`: seek → audio plays from seeked position
- [x] Any project: render → progress advances past 0%, reaches 100%
- [x] Any project: complete render, restart `hyperframes dev`, Download → works
- [x] `intro-vid`: play → runs full 5s (not stopping at 3s)
- [x] `hyperframe-build-up-demo`: play → rocket Lottie visible during 0-2s  verified
- [x] Outside monorepo: Compositions sidebar shows thumbnail images (not blank)
- [x] `bug.zip` project: render → video in polaroid sub-composition appears in output
- [x] Trigger a failed render → error message shown
2026-03-30 23:58:08 +02:00
Vance Ingalls 229538c622 fix: add media rendering guardrails to prevent silent failures (#112)
## Summary

- **Lint rules** catch media elements missing `id` (renderer silently skips them), missing `src`, `preload="none"` (blocks renderer), and video nested in timed divs (freezes playback). Upgraded `video_nested_in_timed_element` from warning to error.
- **Compiler** strips `preload="none"` from media during compilation. Runs parallel, cached keyframe interval analysis via ffprobe — warns on sparse keyframes (>2s) that cause seek failures and audio/video desync. Suggested ffmpeg command preserves audio (`-c:a copy`).
- **Pre-render lint** lints `index.html` + all `compositions/*.html` sub-compositions before render via shared `lintProject()` helper. Warns by default; `--strict` blocks on errors, `--strict-all` blocks on errors + warnings.
- **Render orchestrator** logs a hint to retry with `--workers 1` when parallel capture times out on video-heavy compositions.
- **Refactor**: extracted `runFfprobe()` + `parseProbeJson()` helpers to deduplicate ~80 lines of spawn boilerplate across 3 ffprobe functions. Extracted `shouldBlockRender()` so strict flag tests exercise production code. Shared `lintProject()` used by both `lint` and `render` commands.

## Context

Discovered during a real composition build session where:
1. `<audio>` without `id` rendered silently (preview worked fine because runtime queries `[data-start]`, but renderer queries `[id][src]`)
2. `<video>` inside timed `<div>` froze on first frame
3. `preload="none"` caused 45s renderer timeout
4. YouTube clips with sparse keyframes from `yt-dlp --download-sections` caused audio/video desync
5. Parallel workers timed out on video-heavy compositions

## Test plan

- [x] Core: 365/365 tests passing (5 new lint tests)
- [x] Engine: 24/24 tests passing
- [x] CLI: 14/14 tests passing (7 lintProject + 7 shouldBlockRender)
- [x] Lint + format hooks pass
- [ ] Manual: create a composition with `<audio data-start="0" src="test.wav">` (no id) — verify `npx hyperframes lint` catches it
- [ ] Manual: run `npx hyperframes render --strict` with lint errors — verify it blocks
- [ ] Manual: run `npx hyperframes render --strict-all` with lint warnings — verify it blocks
2026-03-30 11:07:19 -07:00
Miguel Ángel bd175b64a6 refactor(core): extract shared studio API module (#113)
## Summary
Extracts all studio API routes into a shared Hono-based module at `@hyperframes/core/studio-api`.

### Architecture
- **`StudioApiAdapter` interface** — consumers inject host-specific behavior (project resolution, bundling, rendering, thumbnails)
- **Shared route modules**: projects, files, preview, lint, render, thumbnail
- **Shared helpers**: `isSafePath`, `walkDir`, `getMimeType`, `buildSubCompositionHtml`

### What this PR does
- Creates the shared module with all API routes extracted from both `vite.config.ts` and `studioServer.ts`
- Both consumers will be refactored in follow-up commits to mount this module with their own adapter

### What stays in each consumer
- **Vite**: SSR module loading, Puppeteer thumbnails, file watcher + HMR, producer HTTP proxy, multi-project scanning
- **CLI**: in-process `executeRenderJob`, local runtime serving, browser management, SPA static file serving

### Follow-up needed
- [ ] Refactor `packages/studio/vite.config.ts` to use `createStudioApi(adapter)` via `@hono/node-server`'s `getRequestListener`
- [ ] Refactor `packages/cli/src/server/studioServer.ts` to use `createStudioApi(adapter)`
- [ ] Add `./studio-api` export path to `packages/core/package.json`
- [ ] Add `hono` as peer dependency of `@hyperframes/core`

## Test plan
- [ ] Verify shared module compiles without type errors
- [ ] After consumer refactoring: all studio features work identically via both vite dev and CLI embedded servers

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-28 22:28:12 +01:00
Miguel Ángel fecd4c8485 fix(core): strip template wrapper before linting composition files (#111)
## Summary
- Composition HTML files are always wrapped in `<template id="...">` tags
- The linter was checking the raw HTML including the template wrapper, causing false positives:
  - `missing-composition-id` on files that have it inside `<template>`
  - `missing-dimensions` on files that have `data-width`/`data-height` inside `<template>`
- Fix: strip `<template>` wrapper before linting, matching how the runtime and preview server handle these files

## Test plan
- [x] Added test: `strips <template> wrapper before linting composition files`
- [x] All 345 existing tests pass
- [ ] Verify lint panel no longer shows false positives for composition files

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-28 22:08:35 +01:00
Miguel Ángel d781398813 feat(lint): add gsap_css_transform_conflict (#106)
## Changes

Added a new lint rule `gsap_css_transform_conflict` that detects when GSAP animations will silently overwrite CSS transforms.

**`gsap_css_transform_conflict` (error)** — fires when an element has `transform: translateX(-50%)` or `transform: scale()` in CSS and a GSAP `tl.to/from` tween animates `x`, `y`, `xPercent`, `yPercent`, or `scale`. GSAP silently overwrites the full CSS transform, discarding centering tricks like `translateX(-50%)`. Fix hint guides authors to the safe `fromTo` + `xPercent` pattern.

## Root cause

This bug surfaced while building compositions where title reveals were placed off-center because `tl.to("#title", { x: 0 })` stripped the `translateX(-50%)` centering from CSS.

## Test coverage

- [x] `gsap_css_transform_conflict` — `tl.to` with `x` on CSS `translateX` element → error
- [x] `gsap_css_transform_conflict` — `tl.to` with `scale` on CSS `scale()` element → error  
- [x] `gsap_css_transform_conflict` — `tl.fromTo` without CSS transform → no finding
2026-03-28 01:44:52 +01:00
Vance Ingalls f0a8644208 feat(lint): add template_literal_selector rule (#107)
Detects querySelector/querySelectorAll calls that use template literal
variables (e.g. `${compId}`) inside script tags. The HTML bundler's
cheerio/css-what parser crashes on these during compilation, causing
silent fallback to raw HTML without runtime injection.

Severity: error (breaks bundling)
Fix: replace template literal with hardcoded composition ID string
2026-03-27 17:42:28 -07:00
Miguel Ángel 5388a48adf fix(core): lint rule for timeline ID mismatches (#93)
## What

Added three new lint rules to the Hyperframe HTML linter to catch common runtime errors and invalid script references.

## Why

These lint rules prevent silent failures and runtime errors that can break Hyperframe compositions:

1. Timeline assignments without initialization guards cause silent failures when `window.__timelines` is undefined
2. Mismatched timeline IDs between `data-composition-id` attributes and `window.__timelines` keys prevent proper auto-nesting
3. Hallucinated script sources referencing non-existent `@hyperframe/` packages result in 404 errors

## How

Implemented three new lint rules with corresponding error codes:

- `timeline_registry_missing_init`: Detects timeline assignments without proper initialization guard using regex pattern matching
- `timeline_id_mismatch`: Cross-references composition IDs from HTML attributes against timeline registry keys to identify mismatches
- `hallucinated_script_src`: Checks script `src` attributes against known bad patterns for non-existent CDN packages

Each rule provides specific error messages and fix hints to guide developers toward correct implementations.

## Test plan

- [x] Unit tests added/updated
- [x] Manual testing performed
- [ ] Documentation updated (if applicable)

Added comprehensive test coverage for all three new lint rules, including both positive and negative test cases to ensure proper detection and avoid false positives.
2026-03-27 21:23:48 +01:00
Miguel Ángel 1593055482 fix(core): filter decorative elements from timeline clip collection (#92)
## What

Enhanced timeline collection to discover GSAP-animated scene elements and persistent overlays through runtime introspection.

## Why

The existing timeline collection only captured elements with explicit timing attributes (`data-start`, `data-track-index`) or media elements, missing scene elements that are animated purely through GSAP tweens and persistent overlay elements that should appear for the full composition duration.

## How

Added two new discovery mechanisms to `collectRuntimeTimelinePayload`:

1. **GSAP Timeline Introspection**: Walks the master timeline's tweens using `getChildren()` to find animated elements, calculates absolute time ranges by traversing parent timelines, and bubbles child tween ranges up to their nearest scene-level ancestors (direct children of root with IDs).
2. **Persistent Overlay Detection**: Identifies direct children of the root composition that weren't captured by DOM queries or GSAP introspection, treating them as full-duration overlay elements while filtering out non-visual elements (script, style, meta tags) and hidden elements.

Both mechanisms respect existing track assignments and create new tracks when needed to avoid conflicts.

## Test plan

- [x] Unit tests added/updated
- [x] Manual testing performed
- [ ] Documentation updated (if applicable)

Added comprehensive test coverage for:

- GSAP-animated scene element discovery via timeline introspection
- Time range bubbling from child elements to scene ancestors
- Persistent overlay inclusion as full-duration clips
- Proper filtering of non-visual elements (script/style tags)

![image.png](https://app.graphite.com/user-attachments/assets/106ea699-e243-493d-aa76-a74f303b80c3.png)

![image.png](https://app.graphite.com/user-attachments/assets/2d124d15-c933-4e12-891c-06100e6241c3.png)
2026-03-27 18:31:46 +01:00
Miguel Ángel dc0d69a21b feat(producer): support entryFile for rendering individual compositions (#42)
## Changes

- Add optional `entryFile` parameter to render API endpoints (`/v1/render` and `/v1/render-stream`)
- Enable rendering individual sub-compositions by extracting them from index.html context when the entry file is a `<template>` wrapper
- Change base64 audio/video linting from detecting "fabricated" media to prohibiting all inline base64 media
- Add manifest path resolution for bundled producer deployments

## API Changes

- `RenderConfig` — new optional `entryFile` field for specifying HTML file to render
- `server.ts` — parses `entryFile` from request body, validates file exists in project directory
- `executeRenderJob` — uses `entryFile` instead of hardcoded `"index.html"`

## Template Extraction

- `extractStandaloneEntryFromIndex` — extracts sub-composition hosts from index.html and creates standalone render context
- Handles `<template>` entry files by finding matching `data-composition-src` in index.html and isolating that host
- Resets `data-start` to 0 for standalone rendering

## Linting Updates

- Change rule #3.7 from detecting "fabricated" base64 media to prohibiting all inline base64 audio/video
- Lower detection threshold from 100+ to 20+ base64 characters
- All base64 media now triggers error severity with clearer messaging about file size bloat

## Usage

```json
POST /v1/render-stream
{ "projectDir": "/path/to/project", "entryFile": "compositions/intro.html" }
```

Omit `entryFile` for default behavior (renders `index.html`).
2026-03-27 04:28:51 +01:00
Miguel ÁngelandClaude Opus 4.6 d3c6228d89 fix(producer): preserve @import rules during CSS composition scoping
scopeCssToComposition corrupted @import url() rules because they have
no {} block. The selector regex ([^{}@]+)\{ treated the text after @
as a selector, producing invalid CSS like:

  @[data-composition-id="x"] import url('...')

This broke font loading, CSS variable resolution, and all composition
styling in rendered output.

Fix: extract @import rules before running the scoping regex, then
prepend them back unmodified.

Also adds:
- Regression test fixture (css-import-scoping)
- Common-mistakes docs: autoplay/loop, GSAP TextPlugin, sub-composition
  positioning
- Format fix for hyperframeLinter.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:51:05 -04:00
Miguel Ángel 46e9f8d248 feat(core): add loop and data-playback-rate for media elements
- data-playback-rate: per-element slow-mo/fast-forward (0.1-5x range)
  Multiplied with global transport rate. Affects timeline duration
  calculation when source duration is used as fallback.

- loop: native HTML loop attribute now works correctly in the runtime.
  Wraps media playback from mediaStart when source reaches end.
  Enables looping short clips over longer durations.

Both follow the existing data-media-start/data-volume pattern.
2026-03-24 17:27:41 -04:00
Miguel ÁngelandClaude Opus 4.6 36f3b8c87d test(regression): add editor-agent-prod regression fixture
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 15:33:13 -04:00
Vance IngallsandClaude Opus 4.6 61c5257402 fix(ci): update publish workflow to use bun install (#36)
* fix(ci): update publish workflow to use bun install

pnpm-lock.yaml was removed in the bun migration but publish.yml
still referenced it. Use bun for install/build, keep pnpm for
publish (publishConfig overrides + --provenance).

* docs: update stale pnpm references to bun across docs and scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 08:46:58 -07:00
Miguel Ángel 062f6f1c10 fix: composition issues in runtime code + producer parity 2026-03-23 23:56:42 -04:00
Vance Ingalls 20be2ea1c2 style: apply oxfmt baseline formatting across all source files (#25)
## Summary
- Run `oxfmt .` across the entire codebase to establish formatted baseline
- 299 files changed — mechanical formatting only, no logic changes
- Double quotes, semicolons, 2-space indent, trailing commas, 100 print width

Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm format:check` — all 426 files pass
- [x] `pnpm -r typecheck` — all packages pass
- [x] `pnpm build` — all packages build
- [x] All 348 tests pass
2026-03-23 17:15:14 -07:00