ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
346 Commits 856 Branches 404 Tags
ad11de698c1232b364604e8557d341e4e238455c
Commit Graph
245 Commits
Author SHA1 Message Date
James RussoClaude Opus 4.7Codex
 		ad11de698c feat(producer): auto-fallback screenshot capture for raf and iframes (#331) 
* fix(core): drive adapter seeks when composition has no GSAP timeline

renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.

Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.

GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(producer): auto-fallback screenshot capture for raf and iframes

Co-Authored-By: Codex <codex@openai.com>

* test(producer): add render compatibility regression fixtures

Co-Authored-By: Codex <codex@openai.com>

* fix(core): scrub CSS animations via WAAPI currentTime

Co-Authored-By: Codex <codex@openai.com>

* test(producer): cover css keyframe renders

Co-Authored-By: Codex <codex@openai.com>

* fix(producer): propagate virtual time into iframe documents

Co-Authored-By: Codex <codex@openai.com>

* test(producer): refresh iframe docker golden

Co-Authored-By: Codex <codex@openai.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
 2026-04-18 15:08:52 -07:00
Miguel Ángel 59aa2c9ec9 chore: release v0.4.5 2026-04-18 20:24:03 +01:00
Miguel Ángel 37370e1e7d fix(cli): set GIT_CLONE_PROTECTION_ACTIVE=0 for skills (GH #316) (#328) 
## Summary

Fixes #316 — `hyperframes skills` (and `npx skills add heygen-com/hyperframes`) fails with:

```
■  Failed to clone repository
fatal: active \`post-checkout\` hook found during \`git clone\`
└  Installation failed
```

## Root cause

Two layers stacked:

1. **Git 2.45+ refuses to execute hooks during `git clone` by default.** The opt-in is `GIT_CLONE_PROTECTION_ACTIVE=0` — the env-var name is intentionally explicit about the trade-off.
2. **Users who ran `git lfs install` globally have a post-checkout hook registered at `core.hooksPath`.** When the upstream `skills` CLI shells out to `git clone` to fetch a repo's `skills/` directory, git detects the user's LFS hook and aborts.

The check fires for **any repo**, regardless of whether the cloned repo uses LFS itself — it's protection against the user's own hooks, not the repo's content. Users who have git-lfs installed (very common) hit this for every clone the `skills` CLI does.

## The fix

`hyperframes skills` wraps `npx skills add`. The wrapper now sets `GIT_CLONE_PROTECTION_ACTIVE=0` on the spawned child's env via a single helper (`gitCloneFriendlyEnv`) with a docstring at the call site explaining exactly why. The rest of `process.env` is preserved — proxy settings, extra CA certs, locale, etc. stay untouched.

## What this fix doesn't do (deliberately)

This is the **code-path-we-own** fix. The deeper root cause is that the upstream `skills` CLI (vercel-labs/skills) should set this env var when it shells out to `git clone`. That would fix the bug for every user invoking `skills` directly — not just those who route through our wrapper. An upstream issue should be opened separately; not landing it as part of this PR.

## Users who call `npx skills add` directly

Documented in the new troubleshooting subsection: set the env var manually.

```bash
GIT_CLONE_PROTECTION_ACTIVE=0 npx skills add heygen-com/hyperframes
```

## Tests

`packages/cli/src/commands/skills.test.ts` — 2 cases:
- Every spawned child has `GIT_CLONE_PROTECTION_ACTIVE=0`
- The rest of `process.env` is preserved (not a wiped env)

Uses `vi.mock` on `node:child_process` because ESM doesn't allow live-module `vi.spyOn` on re-exported bindings.

## Docs

`docs/packages/cli.mdx` — new **Troubleshooting** subsection under the `skills` command. Explains both the automatic fix (`hyperframes skills` users are already covered) and the manual workaround (`npx skills add …` users).

## Closes

- #316
 2026-04-18 21:17:26 +02:00
Miguel Ángel e8a48a62d0 fix(producer): external assets work on Windows (GH #321) (#324) 
* fix(producer): external assets work on Windows (GH #321)

Two Unix-only assumptions in the external-asset pipeline caused every
absolute path on Windows to be rejected as "unsafe" at render time:

1. Containment checks used `child.startsWith(parent + "/")`. On Windows
   the separator is `\`, so the predicate is always false unless the
   paths are equal — every external asset tripped the safety guard in
   `renderOrchestrator.ts`. The reporter saw:

     [Render] Skipping external asset with unsafe path:
       hf-ext/D:\coder\reactGin\hyperframes\reading\assets\segment_001.wav

   Fix: use `path.relative()` through a shared helper
   `isPathInside(child, parent)` that normalises separators per-platform
   and correctly rejects siblings whose names start with the parent
   (e.g. `/foo/bar-sibling` is NOT inside `/foo/bar`).

2. The external-asset key was built as `"hf-ext/" + absPath.replace(/^\//, "")`.
   A Windows absolute path (`D:\coder\...`) became
   `"hf-ext/D:\\coder\\..."` — and because Node's `path.join` treats a
   drive-letter prefix as absolute, `join(compileDir, key)` silently
   escaped `compileDir`. Fix: `toExternalAssetKey()` strips the drive
   colon and normalises to forward slashes, producing
   `hf-ext/D/coder/...` — a pure relative path that `path.join` cannot
   promote to absolute on any OS.

Both helpers live in `packages/producer/src/utils/paths.ts` and are
exercised by 14 unit tests covering Unix paths, Windows drive-letter
paths, mixed separators, sibling-prefix confusion, and `..` traversal.

Docs: new "External assets" section in `docs/packages/producer.mdx`
describes detection, sanitised keys, and the cross-platform containment
invariant.

Closes #321.

* fix(producer): address review on #324 — UNC + integration test

Addresses the non-blocking observations from the PR #324 staff review
(https://github.com/heygen-com/hyperframes/pull/324#issuecomment):

1. UNC and extended-length Windows paths.
   `toExternalAssetKey` now handles:
   - `\\?\D:\very\long\path\clip.mp4` (extended-length)      → `hf-ext/D/very/long/path/clip.mp4`
   - `\\server\share\file.wav` (plain UNC)                    → `hf-ext/unc/server/share/file.wav`
   - `\\?\UNC\server\share\file.wav` (extended-length UNC)   → `hf-ext/unc/server/share/file.wav`
   The UNC-collapsed form keeps the server boundary so two different
   servers exposing the same share/file name cannot collide under one
   relative key. Previously both edge cases silently produced keys with
   stray `?` or `:` characters that downstream `isPathInside` rejected —
   not a security hole, but a silent drop of user assets.

2. Short-circuit on already-sanitised input.
   `toExternalAssetKey("hf-ext/…")` now returns its input unchanged
   instead of prepending `hf-ext/` a second time. Makes the helper
   genuinely idempotent, which is what the unit test claimed all along.
   Renamed the test accordingly.

3. JSDoc caller contract.
   `toExternalAssetKey` now documents that it expects canonicalised
   input (`path.resolve`'d upstream) and does not strip `..`
   components. `isPathInside` at copy time is still the defensive
   backstop — called out explicitly in the doc so future callers read
   the contract before the code.

4. End-to-end integration test.
   `renderOrchestrator.test.ts` gains two seam tests that run the full
   external-asset pipeline — build the sanitised key, populate an
   `externalAssets` map, invoke `writeCompiledArtifacts`, and assert
   both the success path (the file lands under `<compileDir>/hf-ext/…`)
   and the escape-rejection path (a malicious `hf-ext/../../etc/passwd`
   key does NOT materialise above `compileDir`). `writeCompiledArtifacts`
   is exported for the test seam with a clear JSDoc disclaimer that
   it's not part of the public API.

22 tests pass across `paths.test.ts` (17) and `renderOrchestrator.test.ts` (5).

Out of scope for this follow-up (tracked as follow-ups):
- Centralising every `startsWith("/")` absolute-path check into a
  shared helper across htmlCompiler / audioExtractor / audioMixer /
  videoFrameExtractor. Mentioned in the review; touches 5 files and
  deserves its own PR.
- Windows CI runner.
 2026-04-18 20:59:51 +02:00
Dylanwoo 64e3735100 fix(cli): doctor shows platform-correct install hints (#319) 
The "FFmpeg not found" hint was hardcoded to `sudo apt install ffmpeg`
for any non-macOS platform — Windows users would see an apt command that
doesn't exist on their system, and Red Hat / Arch users got the wrong
package manager too.

`getFFmpegInstallHint()` already exists in browser/ffmpeg.ts (and is
already used by render.ts) and handles darwin / linux / win32 correctly.
Use it here too.

Also rewrite checkFFprobe:
- it previously used `which ffprobe` which is not available on Windows
  (cmd uses `where`), so on Windows the check always reported "Not
  found" even when ffprobe was on PATH
- run `ffprobe -version` directly instead, which works cross-platform
  whenever ffprobe is resolvable on PATH, and surfaces the version
  string in the same style as the FFmpeg check
 2026-04-18 11:59:49 -07:00
James 0f4fcbeed8 chore: release v0.4.4 2026-04-18 03:37:09 +00:00
James RussoandClaude Opus 4.7 686e45dac0 fix(engine): suppress font-load 404s by checking console location URL (#313) 
Chrome's "Failed to load resource" message text does not include the failing
URL — it's only on msg.location().url. The previous filter in frameCapture.ts
only checked msg.text(), so every font 404 (e.g. Google Fonts <link> tags
in sandboxed render environments) fell through to the "[non-blocking]"
prefix instead of being suppressed.

Extract the classifier into isFontResourceError() and match against both
text and location.url, and extend the extension match to .ttf/.otf. Adds
a unit test covering the URL-in-location, URL-in-text, and non-font cases.

This is a targeted fix for the render-output noise that PR #311 attempted
to address by adding a ~120-entry SYSTEM_FONTS skip list. That approach
silently shadowed existing FONT_ALIASES (arial→inter, helvetica→inter,
courier new→jetbrains-mono, segoe ui→roboto, etc.) and changed render
output on Linux fleets that don't have those fonts installed. Fixing the
console-log filter here suppresses the noise without changing any font
resolution behavior.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-17 17:38:46 -07:00
Miguel Ángel 42d39866ff fix(studio): iPhone Safari layout + touch-drag scrubber (#308) 
Stacked on top of #307. Fixes three mobile UX bugs that made the studio unusable on iPhone Safari — discovered while testing the audio-ownership work from #307 on a physical device.

## Bugs fixed

### 1. Untappable Play button / bottom controls

`#root` was set to `100vh`. iOS Safari reports `100vh` as the **largest** viewport (toolbar hidden) and never shrinks it — so with the toolbar visible, the bottom of the layout sits under it. The Play button + timecode were fully occluded.

### 2. Scrubber not draggable by touch

The seek bar had only `onMouseDown`. Mouse events don't fire for touches on iOS Safari, so nothing responded. You could tap to jump but not drag.

### 3. Safari's horizontal swipe hijacked scrubber drags

Even when the seek bar caught `pointerdown`, `touch-action: manipulation` still let Safari consume horizontal edge-swipes for back-navigation — dragging the scrubber left was impossible.

## What changed

| File | Fix |
|---|---|
| `packages/studio/src/styles/studio.css` | `#root` → `height: 100dvh` with `100vh` fallback. Dynamic viewport height shrinks when the iOS toolbar is visible, so the bottom of `#root` lines up with the visible area. |
| `packages/studio/src/App.tsx` | Two `h-screen` containers → `h-full` so nested children fill the now-dynamic parent instead of asserting `100vh` and overflowing. |
| `packages/studio/index.html` | Added `viewport-fit=cover` so iOS exposes real `env(safe-area-inset-bottom)` values. |
| `packages/studio/src/player/components/PlayerControls.tsx` | Controls row gets `padding-bottom: calc(0.5rem + env(safe-area-inset-bottom))` so it clears the landscape home indicator. Scrubber replaced `onMouseDown` with `onPointerDown` + `setPointerCapture`, plus `touch-action: none` so Safari doesn't hijack horizontal swipes. Added `pointercancel` + window-level `pointerup` fallbacks. |

All desktop code paths are unchanged: `100dvh` falls back to `100vh`, `env(safe-area-inset-bottom)` is `0` off-iOS, Pointer Events subsume Mouse Events on desktop.

## Verified live

Via the `cloudflared` tunnel I ran during review on the factory-series-c-video project:

- iPhone Safari, portrait: Play button now fully visible and tappable. Bottom controls sit just above the URL bar.
- iPhone Safari, landscape: controls clear the home indicator.
- Finger-drag the scrubber left and right: tracks the touch smoothly, finger can leave the 6 px bar height without losing the drag.
- Desktop click-to-seek and click-drag: still work.
- Arrow-key seeking: still works.

## Stacked dependency

Base is `fix/player-audio-ownership-review` (PR #307). Once #307 merges, rebase this branch onto `main` — the changes are fully independent; the stacking is just to avoid waiting on the review for #307 before shipping pure UX wins.

## Test plan

- [x] `tsc --noEmit` on `packages/studio` — clean
- [x] `bun run --filter @hyperframes/studio build` — clean
- [x] Live repro on iPhone Safari via cloudflared tunnel: Play button tappable, scrubber drags with touch
- [ ] Android Chrome sanity pass before release (same Pointer Events code path, but worth eye-balling)
 2026-04-18 01:44:46 +02:00
Miguel Ángel c49181f1fa fix(player): address #298 review — tighter drift, dynamic proxies, ownership event (#307) 
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment.

## Significant

### 1\. Drift threshold 150 ms → 50 ms

_mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity.

**Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction.

### 2\. Dynamic sub-composition media proxies

Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track.

Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect.

### 3\. `bridgeMuted` sticky in `syncRuntimeMedia`

The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message.

`syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs.

### 4\. Reset `_audioOwner` on iframe reload

The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently).

`_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`.

## Worth addressing

### 5\. Promotion → observable event + reason

Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state.

**Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once.

### 6\. Parent proxy play() rejection → `playbackerror` event

Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back.

### 7\. Mobile verification on real hardware

Tested with a tunnel in a real iOS device.

## Test gaps (from review)

- `userMuted` stickiness (mirror of the existing `outputMuted` test).
- **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`.
- **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic.
- **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once.
- **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote).
- **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately.
- **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag.

## Minor

- One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear).

## What's good (from the review)

Kept as-is — noted for posterity:

- `muted` vs `volume` framing (orthogonal channels).
- Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing.
- Two orthogonal mute channels.
- Backwards compat (new actions / messages safely ignored by either side).

## Test results

- `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe)
- `packages/core/src/runtime/bridge.test.ts` — **15 tests pass**
- `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface)
- Typecheck green on `core` + `player`
- `tsup` build green on `core` / `player` / `cli`
- Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold.

## Test plan

- [x] Unit tests (83 total across touched files)
- [x] Typecheck clean
- [x] Build clean
- [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened
- [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block
- [x] Physical iOS / Android device verification (unchanged since #298)
 2026-04-18 00:49:57 +02:00
Miguel Ángel e4cfcd3f61 fix(cli): serialize port-availability probes (#309) (#310) 
Closes #309. Full credit to @gigadeniga for the diagnosis — the root cause + proposed fix in that issue are exactly what landed here.

## The bug

\`npx hyperframes preview\` failed deterministically on Crostini (ChromeOS Linux) with \`Ports 3002–3101 are all in use\`, even when nothing was actually listening on any of them.

## Why

\`testPortOnAllHosts\` ran four probes in parallel:

\`\`\`ts
const hosts = ["127.0.0.1", "0.0.0.0", "::1", "::"];
const results = await Promise.all(hosts.map((h) => isPortAvailableOnHost(port, h)));
\`\`\`

Each probe binds a socket and then calls \`server.close()\`. Close is async — the socket stays open until its callback fires on the next event-loop tick. While it's open, the wildcard binds (\`0.0.0.0\`, \`::\`) that include the loopback address race the still-open loopback socket and return \`EADDRINUSE\` spuriously. On Crostini this happens 100% of the time; other Linux configs hit it intermittently; macOS is less predictable. Net effect: every port in the 100-port scan range appears busy and the preview refuses to start.

Reproduces on any Linux box with the standalone snippet from the issue:

\`\`\`
127.0.0.1: OK
0.0.0.0: EADDRINUSE   ← false positive
::1: OK
::: EADDRINUSE        ← false positive
\`\`\`

## Fix

Serialize the probes. Each socket is fully closed before the next opens, eliminating the race window entirely.

\`\`\`ts
for (const host of hosts) {
  const available = await isPortAvailableOnHost(port, host);
  if (!available) return false;
}
return true;
\`\`\`

Kept the four-host check rather than collapsing to just \`0.0.0.0\` + \`::\` — the multi-host coverage is load-bearing for the devbox / SSH-forwarding case where a port is free on loopback but held on the wildcard. Sequentializing is the smaller, less-behaviourally-affecting fix.

## Regression tests

\`packages/cli/src/server/portUtils.test.ts\` — three cases binding real sockets, no mocks:

- **Returns true for a genuinely free port** — directly reproduces the Crostini bug; would fail on Linux against the parallel implementation.
- **Returns false when the port is occupied on \`0.0.0.0\`** — confirms the multi-host check still catches the devbox scenario.
- **Releases each probe socket before the next run** — two back-to-back calls for the same free port both return true, pinning the sequential contract against future refactors that might try to reparallelize for perf.

## Test plan

- [x] \`bunx vitest run packages/cli/src/server/portUtils.test.ts\` — 3/3 pass
- [x] Full CLI suite — 109/109 pass
- [x] \`tsc --noEmit\` clean

## Notes

- Independent of any version bump; ship whenever.
- Probing 4 hosts serially adds at most ~tens of milliseconds per port on the scan (binds are very fast on loopback). The worst-case cost shows up when the first port in the range is free — previously 1 parallel round-trip, now 4 sequential — and it's imperceptible (\`preview\` bind is a one-time startup cost, not a hot path).
 2026-04-18 00:44:58 +02:00
Ular Kimsanov a95539a0fd Merge pull request #299 from heygen-com/feat/capture-improvements-v2 
fix: double-audio scaffold, lint rules, docs guide, Gemini 3.1
 2026-04-17 14:49:05 -04:00
ukimsanovandClaude Opus 4.6 274db7a5ef fix: address PR #299 review — lint correctness, docs, Gemini benchmark 
- lintMultipleRootCompositions: scan filesystem for HTML files with
  data-composition-id (was filtering results array — always 1 entry)
- lintDuplicateAudioTracks: order-independent attribute extraction,
  dedup by (src,start,duration,trackIndex), Infinity fallback for
  missing data-duration (matches runtime behavior)
- 10 new tests for both lint rules
- docs: explicit skill invocation, remove gsap-skills, fix indentation
- Gemini: env override (HYPERFRAMES_GEMINI_MODEL), benchmark data in
  code comment (49 imgs: 3.1-lite ~507ms/img, 2.5-lite ~230ms/img)
- cli.mdx: version-agnostic "Gemini vision" reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-17 10:15:26 -04:00
Miguel Ángel 46afe2ea5d chore: release v0.4.3 2026-04-17 12:15:37 +02:00
ularkim de5b53c08b feat(capture): switch Gemini 2.5 Flash → 3.1 Flash Lite 
Gemini 3.1 Flash Lite Preview: 2.5x faster TTFT, 45% faster output,
slightly cheaper ($0.25/M vs $0.30/M input), near-2.5-Flash quality.
Descriptions are actually more detailed in testing.
 2026-04-16 22:58:50 -04:00
ularkim a77a6cbbf7 fix: double-audio bug + lint rules + docs guide + capture improvements 
Double-audio bug fix:
- scaffolding.ts: stop writing index.html in captures/ (root cause —
  runtime discovered scaffold + real index.html as two compositions)
- New lint rule: multiple_root_compositions — errors if >1 root HTML
- New lint rule: duplicate_audio_track — warns on overlapping audio

Capture improvements (from testing 30+ websites):
- Catalog runs BEFORE extractHtml (which mutates DOM — converts img src
  to data URLs). HeyKuba: 2 images → 78.
- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load image wait, CSS background-image cataloging
- SVG naming from class/id/parent (not just aria-label)
- Gemini batch 5→20, pause 12s→2s, maxOutputTokens 300→500
- Asset descriptions sorted: captioned first

Docs:
- New guide: guides/website-to-video.mdx (full tutorial)
- CLI docs: added capture and snapshot commands
- docs.json: website-to-video in Guides nav

C
 2026-04-16 22:58:50 -04:00
Miguel Ángel 3256551a5e fix(player): single-owner audio to prevent double voice in preview (#298) 
## Summary

Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip).

## Root cause

Two audio pipelines were playing the same source in parallel:

1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path.
2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message.

The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks.

Confirmed via agent-browser instrumentation on `factory-series-c-video`:
- 6 `volumechange` events per play cycle (mute-fight signature)
- Both copies audible at `volume=1`, offset growing 23ms → 80ms
- Every seek widened the drift further

PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback.

## Fix

Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime.

- **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash.
- **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold.

Two orthogonal mute channels replace the volume fight:

| Channel | Purpose |
|---|---|
| `set-muted` | User's mute preference (existing, unchanged) |
| `set-media-output-muted` | Internal ownership handoff (new) |

`syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it.

## Why this shape

- **Single owner, explicit transition.** No races, no tug-of-war.
- **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing.
- **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact.
- **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift.
- **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior.
- **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files.

## Files changed

- `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types.
- `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields.
- `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`.
- `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`.
- `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session.
- `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership.

## Verified end-to-end with agent-browser on `factory-series-c-video`

**Runtime ownership (default — desktop studio):**

| | Before | After |
|---|---|---|
| `PARENT.play()` calls per play cycle | 1 | **0** |
| iframe `volumechange` events | 6 | **0** |
| Audible streams | 2 (drifting) | **1 (iframe)** |

**Parent ownership (simulated autoplay block — direct message):**

| | Value |
|---|---|
| iframe audio | `muted=true`, `volume=1` (untouched) |
| parent audio | `muted=false`, `volume=1`, audible |
| Parent ↔ iframe `currentTime` offset | ~6 ms steady state |
| Offset > 150 ms | corrected by mirror sync |

**Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:**

Event timeline captured via agent-browser instrumentation:

```
t=0.0 ms   IFRAME.play() called                       ← runtime attempts playback
t=0.4 ms   IFRAME.play() REJECTED: NotAllowedError    ← simulated mobile gate
t=0.4 ms   →IFRAME bridge set-media-output-muted=true ← player promotes
t=0.6 ms   PARENT.play() called                       ← parent proxy starts
t=0.8 ms   ←IFRAME msg media-autoplay-blocked         ← runtime signal
t=1.0 ms   PARENT.play() resolved                     ← audible
t=1.3 ms   IFRAME muted=true, volume=1                ← iframe silenced via native muted
```

Steady state at t=4 s under promoted parent ownership:

| Element | currentTime | paused | volume | muted |
|---|---|---|---|---|
| Parent audio | 4.060 s | false | 1.0 | **false** (audible) |
| Iframe audio | 4.068 s | false | 1.0 | **true** (silent) |

**Offset: 8 ms**, single audible stream, orthogonal mute channel respected.

## Test plan

- [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`)
- [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow)
- [x] `bun run build` — all packages green
- [x] Fresh preview + browser repro on `factory-series-c-video`:
  - [x] Runtime ownership: single audio stream, no drift
  - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible
  - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset
  - [x] No `volumechange` thrash in either ownership mode
- [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above)
 2026-04-17 04:46:22 +02:00
Miguel Ángel d291358cbc chore: release v0.4.2 2026-04-17 01:16:46 +02:00
Miguel Ángel 5e52e27872 fix(engine): auto-fall back to screenshot mode when chrome-headless-shell drops HeadlessExperimental.beginFrame (#296) 
Closes #294.

## Summary

Recent `chrome-headless-shell` builds (observed on 147) no longer expose `HeadlessExperimental.beginFrame`. The domain's `enable`/`disable` methods are deprecated upstream and appear to have been dropped alongside `beginFrame` in these builds, so on Linux with chrome-headless-shell the engine aborts with

\`\`\`
Protocol error (HeadlessExperimental.beginFrame):
'HeadlessExperimental.beginFrame' wasn't found
\`\`\`

and — because the browser was launched with `--enable-begin-frame-control` — the compositor waits for beginFrames the engine can no longer deliver, so every subsequent screenshot also comes back blank. Today users have to discover `PRODUCER_FORCE_SCREENSHOT=true` themselves (openclaw did exactly that — see the issue body).

## Fix

One-time probe, right after the browser launches in beginframe mode:

1. Create a disposable CDP session.
2. `await client.send("HeadlessExperimental.enable")`.
3. Send one no-op `HeadlessExperimental.beginFrame` raced against a 2s timeout.
4. If anything throws / times out — missing method, protocol error, stuck call — close the browser, strip beginframe-only chrome flags, relaunch in screenshot mode, and set \`captureMode = "screenshot"\` for the returned session.

Probing `beginFrame` directly rather than `enable` alone is important because some builds keep the domain registered (so `.enable()` succeeds) while dropping the method itself — that's exactly the failure shape in #294.

Cost on happy path: one extra CDP round-trip per browser acquisition (≈ a few ms, since in beginframe-control mode the command returns as soon as the compositor acks). Cost on broken path: one extra launch, which is what the env-var escape hatch already forces manually.

The beginframe-only flag set is enumerated in-module and matched by the stripper, so adding/removing flags stays in one place with `buildChromeArgs`.

## Test plan

- [x] `bun run --filter=@hyperframes/engine test` — all 42 tests pass
- [x] `bun run --filter=@hyperframes/engine build` — typechecks
- [x] `bunx oxlint` + `bunx oxfmt --check` clean
- [x] Manual: standalone test on Linux x86_64 with chrome-headless-shell 146 — probe returns `supported=true`, no fallback (happy path)
- [x] Manual: same test with `--force-fail` simulating openclaw's missing-method condition — fallback triggers, flags stripped, relaunch succeeds, 6.8 KB PNG captured (broken path)
- [ ] Verify on openclaw / real chrome-headless-shell 147 build that the fallback triggers automatically without `PRODUCER_FORCE_SCREENSHOT`

## Notes

- `probeBeginFrameSupport` catches any failure generically; we trust that a working browser answers the no-op beginFrame in well under 2s.
- Warning is logged once per browser acquisition, not per frame.
- Browser pool interaction: pooled browsers cache the resolved `captureMode`, so subsequent acquires in the same process reuse the post-fallback mode without re-probing.
 2026-04-17 01:10:08 +02:00
Miguel Ángel 96376c9be0 chore: release v0.4.1 2026-04-17 00:25:34 +02:00
Miguel Ángel e70687b66c fix(player): resolve iframe media src against iframe baseURI (#295) 
## Summary

`_setupParentMedia` scans the iframe for `audio[data-start]` / `video[data-start]` and creates parallel media elements in the host document (so the studio can scrub audio at sub-frame precision without iframe cross-origin restrictions). It was reading the raw `src` attribute string and assigning it directly to the host-document element, which then resolved relative URLs against the **studio root** instead of the **iframe**.

Result: a composition like

\`\`\`html
<audio id="narration" data-start="0" data-duration="53" src="assets/narration.wav"></audio>
\`\`\`

played fine in rendered MP4 output but 404'd silently in the studio preview (parent audio got `src = http://localhost:PORT/assets/narration.wav` instead of `http://localhost:PORT/api/projects/<name>/preview/assets/narration.wav`).

## Fix

Resolve the src against \`iframeEl.ownerDocument.baseURI\` before passing it to \`_createParentMedia\`. Also read the raw \`src\` attribute on \`<source>\` fallbacks so both paths go through the same resolution.

Diff is 2 lines of meaningful change (9 total once you include the comment).

## Reproduction

1. Create a project with a narration at \`assets/narration.wav\`
2. Reference it in \`index.html\` with \`<audio data-start="0" data-duration="53" src="assets/narration.wav">\`
3. \`npx hyperframes preview\` → open, click play
4. Before: silent (parent audio's \`error.code === 4\` / \`MEDIA_ERR_SRC_NOT_SUPPORTED\`)
5. After: narration plays, scrubbing syncs

## Test plan

- [x] Existing 21 player tests pass (`bun run --filter=@hyperframes/player test`)
- [x] oxlint + oxfmt clean on changed file
- [x] Manual: verified in-studio playback of a narration sourced via relative URL
- [x] Reviewer: confirm render pipeline unaffected (render doesn't go through `_setupParentMedia`)

## Notes

No tests added for this path because the existing harness covers only the `audio-src` attribute codepath — `_setupParentMedia` is triggered by an internal probe interval against a live iframe, which the current fixture doesn't build. Happy to add one in a follow-up if reviewers want that coverage before merge.
 2026-04-17 00:23:07 +02:00
James 60780774bb chore: release v0.4.0 2026-04-16 19:58:39 +00:00
Miguel Ángel f1a37400b4 fix(runtime): silent-first-play + loading overlay for preview (#293) 
## Summary

Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit.

### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`)

`syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window):

- `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click."
- The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged.

`HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`.

### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`)

The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s.

### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`)

Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration.

Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR.

### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`)

Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync.

## Verification

- 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset.
- Full monorepo build green (`bun run build`), typecheck clean, lint/format clean.
- End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass:

| Scenario | Metric | Result |
| --- | --- | --- |
| Normal first play | Audio plays from click, smooth progression | ✅ |
| Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip | ✅ |
| Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) | ✅ |
| Scrub mid-playback | Lands exactly at target frame | ✅ |

## Files changed

- `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve.
- `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset.
- `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes.
- `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check.

## Follow-ups (deferred)

- Tight-threshold short-window drift correction for lip-synced dialogue.
- A perf-regression test that fails on `waiting`\-event resurgence.

## Test plan

- [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words.
- [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events).
- [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers.
- [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame.
 2026-04-16 21:51:06 +02:00
Ular KimsanovandClaude Opus 4.6 87f4c77e2f feat: website capture pipeline + 7-step video production skill (#284) 
* feat(cli): add website capture with AI-powered DESIGN.md generation

Adds `hyperframes capture <url>` command that extracts a complete design
system from any website, producing AI-agent-ready output:

- Full-page screenshot (lazy-load aware, nav at top)
- AI-generated DESIGN.md via Claude API (colors, typography, elevation,
  components, do's/don'ts) with programmatic asset catalog (136+ assets
  with HTML context annotations like img[src], css url(), link[rel=preload])
- CSS-purged compositions (87% size reduction via PurgeCSS)
- HTML-prettified compositions (one-tag-per-line for AI readability)
- CLAUDE.md + .cursorrules auto-generated for AI agent instructions
- Asset deduplication (srcset variants) and tracking pixel filtering

* feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement

- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update session handoff with slack research findings

- key finding: team already wants DESIGN.md integration (James, Bin, Vance)
- skills quality matters enormously - must invoke /hyperframes-compose
- eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide)
- templates at templates/ need study before finalizing skill
- session handoff updated with critical next steps

* refactor(cli): simplify capture pipeline, remove replica generator

* feat(capture): add Lottie detection and WebGL shader extraction

Captures Lottie animations via network interception and WebGL shader
source via gl.shaderSource hooking during site crawl. Updates
website-to-hyperframes skill with asset planning guidance, Lottie/shader
reading instructions, and stronger creative direction for scene planning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(capture): clean pipeline + shader-first creative workflow

Capture pipeline:
- Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps)
- Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md)
- Remove dead canvas-to-image code in htmlExtractor (post canvas removal)
- Parallelize image downloads (batches of 5 via Promise.allSettled)
- Fix pre-existing TS error (match[1] guard in font downloader)
- Default capture output to captures/<hostname>

Skill creative overhaul:
- Add shader transition selection to creative director step (Step 4)
- Add shader wiring instructions to engineer step (Step 5)
- Replace 4-line energy modifiers with visual vocabulary table
- Strip rigid scene-by-scene templates from video-recipes.md
- Strip example fill data from scene plan tables
- Add "read transition refs before planning" instruction
- Add creative ambition language ("how the hell did they make this")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add skill architecture redesign spec

Comprehensive redesign of website-to-hyperframes skill and capture
pipeline based on code review findings and Claude Code architecture
research. Key changes: remove AI auto-generation, restructure skill
into phases, embed shader boilerplate in scaffold, fix color format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add implementation plan for skill architecture redesign

13-task plan covering: capture pipeline cleanup (remove AI generation,
fix colors to HEX, add asset descriptions, shader-ready scaffold),
skill restructuring (4 phases with artifact gates), and compose skill
Visual Identity Gate upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(capture): remove AI auto-generation and SDK dependencies

* fix(capture): convert extracted colors to HEX format

* refactor(capture): remove AI key path, add asset descriptions generator

* refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions

* feat(capture): pre-wire shader transitions in index.html scaffold

* chore: remove duplicate visual-styles.md (canonical is in hyperframes/)

* refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator

* feat(skill): add Phase 1 understand reference

* feat(skill): add Phase 2 design reference with full DESIGN.md schema

* feat(skill): add Phase 3 creative direction reference

* feat(skill): add Phase 4 build reference with inline shader example

* feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md

* docs: update CLAUDE.md skill references for phase-based workflow

* fix: address code review findings

- Remove orphaned `false` argument in generateAgentPrompt call (critical:
  was shifting hasLottie, hasShaders, catalogedAssets parameters)
- Add HSL color handling in rgbToHex via temp element resolution
- Remove build artifact commit section from phase-4-build.md
- Fix __GSAP_TIMELINE reference to __timelines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(capture): regex double-escape + simplify scaffold + fix asset descriptions

- Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\()
  so browser receives valid regex patterns via page.evaluate()
- Simplify index.html scaffold: scene slots + audio + timeline + comment pointing
  to shader-setup.md reference (no broken inline shader boilerplate)
- Fix asset descriptions: use CatalogedAsset.contexts/notes instead of
  nonexistent htmlContext field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: code review — 16 bugs, 7-step skill rewrite, cleanup

Code fixes:
- snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero
  for --frames 1, port bind error handling, rAF-based render settle
- index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini
  batch/rate-limit comments, fix video preview viewport y-coordinate
- tokenExtractor.ts: remove dead seen[si] dedup code
- gsap.ts: index ALL classes for inline-style transform conflict detection

Skill architecture rewrite (4-phase → 7-step):
- Replace phase-1 through phase-4 with step-1 through step-7
- Add techniques.md (10 visual techniques with code patterns)
- Fix /hyperframes-compose → /hyperframes (skill doesn't exist)
- Fix captures/arc-browser reference → shader-setup.md (file doesn't exist)
- Fix step-7 hardcoded captures/stripe path
- Document Gemini API free/paid rate limits in step-1

Cleanup:
- CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step
- .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast)
- Delete old phase-*.md, animation-recreation.md, tts-integration.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dev artifacts, research docs, wrong lockfiles

Remove files that shouldn't ship in this PR:
- docs/research/ (aura analysis, prompt catalogs)
- docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes)
- docs/superpowers/ planning and spec docs
- pnpm-lock.yaml at root and cli (repo uses bun, not pnpm)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention

Main PR #283 removed the full skills table from CLAUDE.md and moved it
to AGENTS.md. Align with that decision: use main's slim dev-focused
format, fix pnpm→bun references, add one-line /website-to-hyperframes
pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add capture command to help groups

The capture command was registered in cli.ts but missing from
the help groups, so it wouldn't appear in `hyperframes --help`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format skill reference files (oxfmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: regenerate bun.lock after rebase

The lockfile was stale after rebasing onto main — bun install
--frozen-lockfile failed in CI because new dependencies (google/genai,
patchright, purgecss) weren't reflected in the lockfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments + improve capture quality

Review fixes (16 comments from jrusso1020 + vanceingalls):
- screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots
- screenshotCapture: dynamic screenshot count based on page height (30% overlap)
- snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard
- htmlExtractor: fix code injection via parameterized evaluate
- index.ts: video preview re-measures position after scroll, .env file loading
- capture.ts: BLOCKED.md on timeout failures
- gsap.ts: 5 inline-style lint tests added (all pass)
- Remove Playwright, patchright deps; @google/genai to optionalDependencies
- Gitignore: generic patterns instead of 20 hardcoded directories
- Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance)

Capture quality improvements (tested on 10+ websites):
- Color extraction: canvas-based oklch/lab resolver, pixel sampling via
  elementFromPoint, broad sweep for accent colors, gradient/shadow extraction
- Section detection: broadened selectors for div-based layouts, height cap
  to skip page-level wrappers, parent bg walkup for dark sites
- Font downloads: cap 6 per family / 30 total (Cal.com: 306→30)
- CTA detection: text pattern matching + nav context filtering
- Heading text: innerText with whitespace normalization
- Gemini captioning: maxOutputTokens 100→300, .env auto-loading
- .env.example updated with GEMINI_API_KEY docs
- TTS ranking: Kokoro first with Python 3.10+ note

* fix: address PR review comments + improve capture quality

Review round 2 fixes (jrusso1020 + vanceingalls):
- verify/index.ts: add path traversal guard (relative + isAbsolute)
- verify/index.ts: fix sections[i] undefined typecheck error (CI green)
- index.ts: escape Lottie JSON with \u003c to prevent </script> breakout
- step-4-storyboard: fix technique count contradiction (2-3 per beat, not
  across whole video)
- step-6-build: perspective tilt uses gsap.set() instead of CSS transform
  (avoids GSAP overwrite conflict)
- step-1-capture: reorder — command first, Gemini note after (zero-config
  is the default path, API key is optional enhancement)
- step-7-validate: add tsx fallback for snapshot command
- step-3-script: vary hook patterns, don't default to number every time
- assetDownloader: exempt SVGs from 10KB minimum filter (company logos
  like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets)

Note: adm-zip was NOT removed (reviewer #3) — it's still in
packages/cli/package.json:30. The root package.json had patchright
and purgecss removed, not adm-zip.

Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms
zero references in the entire codebase. The @anthropic-ai/sdk dependency
was removed earlier in this branch.

* refactor(capture): split index.ts (1175 to 566 lines) into modules

Mechanical extraction, zero logic changes.

New files:
- mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots
- contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions
- scaffolding.ts (135 lines): .env loading, project scaffold generation

Also fixes false-positive BLOCKED.md with structural Cloudflare detection.
Tested on 20 websites, pre/post output identical.

* chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss)

The --split feature auto-generates compositions from captured HTML — a
different approach from the /website-to-hyperframes skill workflow where
agents build compositions from scratch using the storyboard.

No skill file, no step reference, and no test session ever used --split.
Removes 923 lines of unused code + purgecss dependency.

Backed up to ~/Desktop/capture-split-backup/ for reference.

* fix(security): add ssrf protection, lottie injection fix, oom guard

- assetDownloader: add isPrivateUrl() guard blocking private IP ranges
  (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata
  endpoints, localhost, and non-HTTP schemes
- mediaCapture: fix Lottie JSON injection by loading shell HTML first
  then passing animation data via parameterized page.evaluate()
- index.ts: check Content-Length header before response.buffer() in
  Lottie network interception to avoid OOM on multi-GB responses

* fix(capture): security fixes, timeout, sub-agent dispatch instructions

Security (from miguel-heygen review):
- assetDownloader: export isPrivateUrl() SSRF guard
- htmlExtractor: add isPrivateUrl check before CSS fetch
- mediaCapture: add isPrivateUrl check before Lottie fetch
- mediaCapture: fix previewPage leak (try/finally)
- mediaCapture: skip Lottie files > 2MB for preview (CDP limit)
- contentExtractor: skip images > 4MB for Gemini captioning
- index.ts: check Content-Length before response.buffer() (OOM guard)
- snapshot.ts: register error handler before server.listen()

Capture improvements:
- Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare)
- step-6-build: sub-agent dispatch template with explicit rules:
  pass file PATHS not contents, use local fonts not Google Fonts,
  verify ../assets/ references after each beat

* fix(capture): catalog before DOM mutation, networkidle2, faster Gemini

Critical: asset cataloger now runs BEFORE extractHtml which converts img
src to data URLs. Framer sites like heykuba.com went from 2 to 78 images.

- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load wait: scroll to bottom, wait for img.complete
- CSS background-image cataloging for Framer/Webflow
- SVG naming: checks class, id, parent, inner text (not just aria-label)
- Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img)
- maxOutputTokens 300->500, descriptions sorted captioned-first
- Remove tsx fallback from step-1 (reviewer nit, published CLI has it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 12:03:16 -07:00
James RussoandClaude Opus 4.6 ebc12f7dc9 feat(render): add CRF/bitrate controls and improve default quality (#292) 
Raise default encoding quality to visually lossless at 1080p (CRF 18)
and expose fine-grained encoding controls for power users.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 12:02:24 -07:00
James 2718de8776 chore: release v0.3.2 2026-04-16 06:16:31 +00:00
James RussoandClaude Opus 4.6 237847e5c6 docs: add prompt cookbook + prompting guide for AI agents (#286) 
* docs: add prompt cookbook + prompting guide for AI agents

Addresses user feedback that there's no guidance on how to actually
prompt Claude Code (or other agents) once the hyperframes skills are
installed. Adds copy-pasteable example prompts in the README and
quickstart, a new prompting guide page, and a starter-prompt nudge in
the `hyperframes init` output.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(prompting): add vocabulary tables, rules, and TTS voice guide

Merges the best content from the internal prompt guide into
prompting.mdx: easing vocabulary, caption tone table, transition
energy matrix, audio-reactive frequency mapping, marker highlight
modes, TTS voice recommendations, rendering quality presets, and
framework rules (technical requirements vs best practices).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(prompting): rename page title to "Prompt Guide"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove greensock/gsap-skills dependency, fix Math.random nuance

The bundled skills/gsap/ already covers the GSAP surface needed for
HyperFrames compositions. Installing greensock/gsap-skills on top adds
a competing full-ecosystem skill that's mostly irrelevant (ScrollTrigger,
Draggable, SplitText, etc.) and can confuse agents about which GSAP
context to load.

Also adds seeded-PRNG nuance to the Math.random() rule in the prompt
guide (matching the skill's actual guidance).

Removed from: skills.ts, README, AGENTS.md, shared AGENTS.md/CLAUDE.md,
and prompting.mdx.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: require minimal reproduction link in bug report template

Adds a required "Link to reproduction" input field asking users to push
a minimal repro to a public GitHub repo (scaffolded via
`hyperframes init repro --non-interactive --example blank`).

Also consolidates the OS/Node/FFmpeg/version fields into a single
"Environment" field using `npx hyperframes info` output — fewer fields
to fill, more consistent data.

Follows the same pattern as Next.js and Gatsby issue templates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(issue-template): use hyperframes doctor for environment info

`hyperframes info` only prints project metadata (resolution, duration,
elements). `hyperframes doctor` prints the full environment: version,
Node.js, FFmpeg, Chrome, memory, disk, Docker — everything needed to
diagnose bugs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(prompting): mention validate alongside lint in anti-patterns

Per Vance's review comment — validate catches runtime errors (JS
exceptions, missing assets, contrast) that lint doesn't.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: replace libretto example URL with hyperframes repo

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-15 22:22:46 -07:00
Miguel Ángel a6ff9e2d9f fix(player): preserve iframe media attributes for runtime sync (#291) 
## Summary

- `_setupParentMedia()` (added in #266) was stripping `data-start`, `data-duration`, and `src` from audio/video elements inside the composition iframe
- The runtime's `syncRuntimeMedia` queries `audio[data-start]` to find media clips — removing these attributes made the runtime unable to find, sync, or play audio
- Result: silent audio in studio preview and any context where `__player.play()` is called directly (not through the web component)

## Fix

- Keep all iframe media attributes intact so the runtime can track time position and manage playback
- When parent-frame media `play()` succeeds (mobile use case), mute the iframe copies via `volume = 0` to prevent double audio
- On desktop and in the studio (which calls `__player.play()` directly), the runtime's own media sync handles playback normally

## Test plan

- [x] 21 player unit tests pass
- [x] Verified with John Wu's slideshow project: audio element preserves `data-start`, `data-duration`, `src` after runtime init
- [x] Verified runtime `syncRuntimeMedia` finds and plays audio (currentTime advances in sync with timeline)
- [x] Build passes (lint, format, typecheck)
 2026-04-16 06:20:28 +02:00
James 0a3ca498ea chore: release v0.3.1 2026-04-15 18:42:14 +00:00
Vance IngallsandClaude Opus 4.6 a262ad59f3 chore(skills): remove 1,685 lines of redundant skill content (#283) 
* chore(skills): remove 1,685 lines of redundant and irrelevant skill content

- Remove 5 GSAP references irrelevant to HyperFrames (scrolltrigger,
  plugins, react, frameworks, utils) — no scroll, no frameworks, no
  interactive plugins in video compositions
- Remove shader-setup.md and shader-transitions.md — duplicated by
  @hyperframes/shader-transitions package (packages/shader-transitions/)
- Remove marker-highlight.md and examples.md — JS library docs superseded
  by css-patterns.md (deterministic, GSAP-driven, fully seekable)
- Trim CLAUDE.md to dev-only instructions — move product docs (transcription,
  TTS, player) to skills where they belong
- Deduplicate house-style.md typography/motion sections — point to
  dedicated references instead of repeating rules
- Clean up stale references to deleted files across SKILL.md and catalog.md
- Update gsap skill description to reflect HyperFrames-only scope

Skills: 5,230 → 3,714 lines (29% reduction)
CLAUDE.md: 204 → 50 lines (75% reduction)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): update broken marker-highlight.md references in captions.md

Point to css-patterns.md instead of deleted marker-highlight.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): update stale shader CSS rule to reference package API

BG_COLOR was from the old manual setup. Now it's bgColor in the
@hyperframes/shader-transitions init() config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address 6 doc gaps surfaced by eval agents

P0: Document HyperShader as IIFE global name in shader-transitions README
P1: Replace async fetch() with sync XHR in effects.md audio data loading
    (fetch violates synchronous timeline construction rule in SKILL.md)
P1: Change <div> to <span> in css-patterns.md marker highlight patterns
    (<div> inside <p> is invalid HTML, breaks layout in inline contexts)
P2: Clarify bgColor as fallback color in shader-transitions README
P2: Add data-start to Composition Clips table in SKILL.md
    (root composition element needs data-start="0", linter enforces it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(templates): update init templates to match trimmed skill scope

- Remove ScrollTrigger/plugins/React/Vue/Svelte from gsap skill description
- Replace class="clip" with accurate pattern examples in skill intro text
  (class="clip" is still in Key Rules where it belongs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): remove contradictory 5:1 contrast threshold from house-style

house-style.md said 5:1 minimum, but hyperframes validate enforces
WCAG AA (4.5:1 normal text, 3:1 large text). Now defers to validate
instead of stating a conflicting number.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-15 10:51:10 -07:00
James Russo acf8223171 fix(skills): correct README skills table and move orphaned scripts into hyperframes skill (#282) 
## What

Fixes the README skills table to match actual skill names, and moves two orphaned script directories into the `hyperframes` skill where they belong.

## Why

**README**: Listed `hyperframes-compose` and `hyperframes-captions` as separate skills — these don't exist. Captions/compose are part of the `hyperframes` skill. Also listed `gsap-core, gsap-timeline, gsap-plugins, ...` but the actual skill is just `gsap`. Missing `hyperframes-cli` entirely.

**Orphaned scripts**: `skills/hyperframes-animation-map/` and `skills/hyperframes-contrast/` had scripts but no `SKILL.md` — they looked like broken skills and wouldn't be installed by `npx skills add`. They're helper scripts invoked by the main `hyperframes` skill (SKILL.md already references them in the "Quality Checks" section). Moving them under `skills/hyperframes/scripts/` makes them part of the skill they belong to.

## How

**README skills table** — corrected to match the 4 actual skills:
- `hyperframes` (was `hyperframes-compose` + `hyperframes-captions`)
- `hyperframes-cli` (was missing)
- `hyperframes-registry` (unchanged)
- `gsap` (was `gsap-core, gsap-timeline, gsap-plugins, ...`)

**Script moves:**
- `skills/hyperframes-animation-map/scripts/animation-map.mjs` → `skills/hyperframes/scripts/`
- `skills/hyperframes-contrast/scripts/contrast-report.mjs` → `skills/hyperframes/scripts/`
- Removed empty `skills/hyperframes-animation-map/` and `skills/hyperframes-contrast/`
- Updated path references in SKILL.md, both script headers, and `contrast-audit.browser.js`

## Test plan

- [ ] `grep -r "hyperframes-animation-map\|hyperframes-contrast" --include="*.md" --include="*.mjs" --include="*.js" --include="*.ts" .` returns no results
- [ ] `ls skills/` shows only `gsap`, `hyperframes`, `hyperframes-cli`, `hyperframes-registry`
- [ ] README skills table matches `skills/*/SKILL.md` names
- [x] Documentation updated (if applicable)
 2026-04-14 20:41:54 -07:00
James Russo 8551cffddd docs: add AGENTS.md for universal AI assistant configuration (#278) 
## What

Adds `AGENTS.md` at two levels:
1. **Repo-level** (`/AGENTS.md`) — for contributors working on HyperFrames itself
2. **Project-level** (`packages/cli/src/templates/_shared/AGENTS.md`) — scaffolded into user projects by `hyperframes init`

Also replaces the previous `AGENTS.md → CLAUDE.md` symlink with a standalone file.

## Why

AGENTS.md is the emerging universal standard for AI coding tool configuration, supported by Claude Code, Cursor, GitHub Copilot, Gemini CLI, and Codex (60K+ repos). Remotion already has one. For a project that positions itself as AI-native, this is a gap.

The project-level file (scaffolded by `init`) ensures every AI tool — not just Claude — gets framework context when working on a user's composition project. The symlink was replaced because symlinks are fragile on Windows and the content should differ (repo-level covers build/test, project-level covers composition rules).

## How

- **Repo-level AGENTS.md**: Build/test/lint commands, project structure, key conventions, skills install, doc links
- **Project-level AGENTS.md**: Composition-specific — skills, CLI commands, project structure, linting workflow, key rules, doc links
- **CLAUDE.md** remains for Claude-specific skill invocation syntax (slash commands)

## Test plan

- [ ] Verify AGENTS.md renders correctly on GitHub
- [ ] Verify no duplication with CLAUDE.md (AGENTS.md = universal basics, CLAUDE.md = Claude-specific slash commands)
- [ ] Manual testing performed
- [x] Documentation updated (if applicable)
 2026-04-14 19:13:26 -07:00
Miguel Ángel d3f2295d80 feat(skills): add contrast audit + animation map quality skills (#267) 
## Summary

Two new quality skills + CLI integration that give agents feedback loops they currently lack — pixel-level contrast auditing and structured animation analysis.

### What this unlocks for agents

**Agents can now catch accessibility failures that humans and LLMs consistently miss.** The contrast audit runs automatically on every `hyperframes validate` and reports WCAG AA violations as warnings. In the eval, 4 out of 5 palettes had failing contrast — every baseline composition shipped broken, every treatment composition caught and fixed it.

**Agents can now reason about animation choreography.** The animation map produces a structured JSON report with:
- Per-tween natural language summaries ("card1 slides 23px up over 0.5s, fades in, ends at (120, 200)")
- ASCII timeline showing the full choreography as a Gantt chart
- Stagger detection with actual intervals ("3 elements stagger at 120ms" — validates against brief specs)
- Dead zone detection (periods >1s with no animation — missing entrance or intentional hold?)
- Element lifecycles (first/last animation, final visibility — catches elements that enter but never exit)
- Scene snapshots at 5 timestamps (what's on screen at any moment)

### Changes

**Skills (new)**
- \`skills/hyperframes-contrast/\` — WCAG contrast audit skill + script
- \`skills/hyperframes-animation-map/\` — animation analysis skill + script

**CLI**
- \`hyperframes validate\` now runs contrast audit by default (warnings, not errors)
- \`hyperframes validate --no-contrast\` to skip
- \`hyperframes render --html-only\` compiles HTML without video encoding
- Browser-side WCAG code in \`contrast-audit.browser.js\`, inlined at build time via esbuild text loader

**Producer**
- Exported \`compileForRender\` for the \`--html-only\` flag

### Eval results

5 prompts x 2 arms = 10 compositions. Arm A = baseline skills. Arm B = +contrast +animation-map.

| Prompt | Failing color | Before | After |
|--------|--------------|--------|-------|
| Halflife | Cement on Ink | 2.98:1 | 5.33:1 |
| Meridian | Ash on Midnight | 2.08:1 | 5.44:1 |
| Typesmith | Pencil on Paper | 3.19:1 | 5.50:1 |
| Lattice | Gray-600 on Terminal | 2.59:1 | 7.50:1 |

Animation map correctly enumerated 142 tweens across 5 compositions, detected stagger groups, flagged pacing issues, and produced scene snapshots.

### Pitch video

https://itnjfahrnzqvcluhrtif.supabase.co/storage/v1/object/public/assets/uploads/8f043e1c-6882-4fa9-98fd-efb6b3583afa.mp4

### Dedicated evals

https://www.heygenverse.com/a/2cac956b-3d14-47bf-90e8-3c1f50e671f3

## Test plan

- [x] Eval: 10 compositions (5 baseline, 5 treatment), all rendered
- [x] Contrast audit caught 4/4 failing palettes, 0 missed
- [x] \`hyperframes validate\` shows contrast warnings by default (exit 0)
- [x] \`hyperframes validate --no-contrast\` skips audit
- [x] \`hyperframes validate --json\` includes contrast data
- [x] Animation map tested on 3 compositions (17, 27, 51 tweens)
- [x] Stagger detection, dead zones, snapshots, timeline all verified
- [x] \`bun run build\` passes
- [x] \`bun run lint\` passes (0 errors, 0 warnings, 0 skill lint issues)
 2026-04-15 03:09:56 +02:00
James Russo 13ab1932ad feat(cli): catalog browser command (#271) 
Adds `hyperframes catalog` for browsing the registry:

- Default: non-interactive table output (agent-friendly)
- --type block/component and --tag filters
- --json for machine-readable output
- --human-friendly for interactive picker that installs on select

Registered in cli.ts, help.ts, documented in docs/packages/cli.mdx.
 2026-04-14 16:46:42 -07:00
James Russo 4bde66f532 feat(skills): hyperframes-registry skill (#261) 
## What

New skill `hyperframes-registry` that teaches AI coding agents how to install and wire registry blocks and components into HyperFrames compositions.

### Skill structure
```
skills/hyperframes-registry/
  SKILL.md                          — triggers, overview, quick reference
  references/
    install-locations.md            — default paths, hyperframes.json config
    wiring-blocks.md                — iframe inclusion, data attributes, positioning
    wiring-components.md            — snippet merging (HTML, CSS, JS, timeline)
    discovery.md                    — manifest reading, item fields, available items table
    demo-html-pattern.md            — why components ship demo.html, structure conventions
  examples/
    add-block.md                    — worked example: data-chart block install + wiring
    add-component.md                — worked example: shimmer-sweep component install + wiring
```

## Why

Phase B of the catalog plan (PR 10). Without this skill, agents using `hyperframes add` have to guess how to wire installed items into compositions. The skill encodes the iframe/snippet patterns so agents get it right on the first attempt.

## How

- SKILL.md frontmatter triggers on: `hyperframes add`, "block", "component", `hyperframes.json`
- References cover every step: discovery, install, wiring blocks (iframe), wiring components (snippet merge), and the demo.html convention
- Two worked examples walk through complete install-to-preview workflows
- Updated CLAUDE.md skills table + trigger rules, README.md skills table, docs/packages/cli.mdx

## Test plan

- [x] `scripts/lint-skills.ts` passes (checked 4 skill files, no issues)
- [x] `oxfmt --check` passes on all markdown files
- [x] SKILL.md frontmatter has valid `name` and `description`
- [x] All reference links in SKILL.md resolve to existing files
- [x] CLAUDE.md, README.md, and docs CLI page updated with new skill
 2026-04-14 16:27:24 -07:00
James RussoandClaude Opus 4.6 b23b0751da fix(player): parent-frame media playback for mobile (#266) 
* fix(player): parent-frame media playback for mobile

Mobile browsers block media.play() inside iframes when the user
gesture happened in the parent frame — postMessage doesn't transfer
user activation (per the User Activation v2 spec).

## Problem

The player renders compositions in a sandboxed iframe. When a user
taps play in the parent frame, the player sends a postMessage to the
iframe's runtime, which calls audio.play(). On mobile, this fails
silently because the iframe has no user activation context.

## Solution

The player now extracts ALL timed media elements (audio/video with
data-start) from the iframe's DOM (same-origin access), creates
parent-frame copies, and disables the iframe originals. On play(),
parentMedia.play() runs synchronously in the gesture call stack,
satisfying mobile autoplay policy.

### Generic media handling

- Finds all `audio[data-start], video[data-start]` in the iframe
- Creates a parent-frame copy for each (Audio or Video element)
- Preserves data-start offsets for correct seek positioning
- Strips data-start from iframe elements so the runtime ignores them
- Falls back to iframe media for cross-origin iframes

### `audio-src` attribute

Convenience for the common single-narration case. When set, the
player starts preloading audio immediately — before the iframe loads.
This eliminates the loading delay that caused jittery playback.

### No active sync

Both parent media and the GSAP timeline are real-time systems. When
started simultaneously, they naturally stay within ~10ms — no drift
correction needed. Active sync with coarse granularity (50ms polling)
caused MORE jitter than it prevented via repeated audio seeks.

## CI

- Added unified `test` job replacing separate per-package test jobs
- Added root `test` script: `bun run --filter '*' test`
- New packages with test scripts are automatically included
- Added happy-dom for player DOM tests

## Tests

- 10 new tests for parent-frame media: preloading, play, pause,
  seek, muted/rate sync, cleanup, attribute changes
- All 21 player tests pass

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(shader-transitions): pass CI when no test files exist

Add --passWithNoTests to vitest run so the unified test job
doesn't fail on packages that have a test script but no test
files yet.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update tests for new id field and GSAP lint rule

- normalize.test.ts: loadTranscript now assigns id fields (w0, w1, etc.)
  to SRT/VTT results and empty string for words-json passthrough
- lintProject.test.ts: add GSAP CDN script to validHtml() fixture to
  satisfy the missing_gsap_script lint rule added in core

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add missing data-start/data-duration to validHtml fixture

The validHtml() test fixture was missing data-start and data-duration
attributes, triggering the root_composition_missing_data_start and
root_composition_missing_data_duration lint warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): fetch LFS objects for producer test job

Producer regression tests compare rendered output against reference MP4
files stored in git LFS. Without lfs: true, checkout fetches pointer
files instead of actual videos, causing "moov atom not found" errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: remove redundant test-producer job

The regression workflow already runs the same 28 producer fixtures
in a Docker container with prod-matching Chrome/fonts/ffmpeg, sharded
across 8 parallel matrix jobs with 40-min timeouts. The CI test-producer
job was a duplicate that ran on bare runners with worse determinism
and a 15-min timeout too short for all fixtures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-14 11:20:07 -07:00
James RussoandClaude Opus 4.6 9bf4956fae chore(shader-transitions): add to CI publish pipeline and README (#264) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-13 22:07:19 -07:00
James Russo 08fb1de61f feat(cli): add command + hyperframes.json (#256) 
## What

PR 5/17 of the catalog system rollout. Adds the `hyperframes add` verb for installing blocks and components from the registry into an existing project, plus the `hyperframes.json` project config that tells `add` which registry to use and where to drop files. Stacks on #255.

- **`packages/cli/src/commands/add.ts`** — new `hyperframes add <name>` command. Resolves an item, validates target paths, installs files in parallel, builds an include snippet, copies it to the clipboard. Exposes a testable `runAdd(opts)` function; the citty default wraps it with console output + exit handling
- **`packages/cli/src/utils/projectConfig.ts`** — read/write/normalize `hyperframes.json`. Tolerant to missing and partial configs
- **`packages/cli/src/utils/clipboard.ts`** — minimal cross-platform clipboard (pbcopy / clip.exe / wl-copy / xclip / xsel). Zero deps. Gracefully no-ops in headless environments
- **`packages/cli/src/commands/init.ts`** — write `hyperframes.json` during scaffold if not already present
- **`packages/cli/src/cli.ts`** + **`help.ts`** — register `add` under Getting Started (directly below `init`)

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a).

## UX

```bash
# Scaffold a project (now writes hyperframes.json too)
npx hyperframes init my-video --example blank
cd my-video

# Add a block — files land, snippet copied to clipboard
npx hyperframes add claude-code-window
#  ✓ Added claude-code-window (hyperframes:block)
#    compositions/claude-code-window.html
#
#  Include snippet:
#    <iframe src="compositions/claude-code-window.html" data-start="0" data-duration="6"></iframe>
#
#  Copied to clipboard — paste into your host composition.

# Add a component effect
npx hyperframes add shader-wipe

# Headless / CI — no clipboard, JSON output for tooling
npx hyperframes add shader-wipe --no-clipboard --json
```

Running `hyperframes add warm-grain` (an example) errors clearly pointing to `init --example`.

## Docs (bundled in this PR per the tracker principle)

- `docs/packages/cli.mdx` — new `add` subsection under Commands (flags, examples, trigger rules) + new `hyperframes.json` section describing the config file shape

## Tests

- **`packages/cli/src/commands/add.test.ts`** — 11 tests:
  - `remapTarget` / `buildSnippet` pure helpers (5 tests)
  - `runAdd` integration against a mocked `fetch` registry: block install lands files + returns snippet, component install respects `paths.components` remap, example-typed names throw `AddError` with code `example-type`, unknown names throw `AddError` with code `unknown-item` (4 tests plus 2 covering block default path and non-default path preservation)
- **`packages/cli/src/utils/projectConfig.test.ts`** — 9 tests:
  - Write/read round-trip, partial-config normalization, corrupt-file handling, absent-file fallback to defaults, custom paths preserved
- **CLI suite:** 92 passed (was 72 on #255, **+20**). Same 4 pre-existing failures unchanged

## Scope decisions

- **`init.ts` full port to new resolver deferred.** The original plan bundled a removal of the `packages/cli/src/templates/` compat shim. That's ~300 more lines and isn't required for `add` to work. The compat shim from #254 still functions; a separate cleanup PR handles it
- **No ajv runtime schema validation.** Manifests are trusted as schema-valid. Full validation lands when third-party registries arrive (PR 14/15). Path safety is still enforced by the installer's `assertSafeTarget` guard
- **Default project paths stay under `compositions/`.** Blocks → `compositions/<name>.html`; components → `compositions/components/<name>/<file>`. Users override via `hyperframes.json#paths`

## Breaking / migration

**None.** Pure additive — new command, new file types, no existing commands or flags change. `init.ts` now writes `hyperframes.json` but that's a new additional file, not a modification of existing output.

## Stacks on

#255 — base branch. When #255 merges, this rebases onto `main`.

## Next in stack

PR 6 — `feat(registry): seed block — claude-code-window`. First real registry item. Exercises the full `hyperframes add <name>` flow end-to-end against a committed item on `main`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 21:04:59 -07:00
James Russo c8acd8abd8 feat(cli)!: rename --template to --example (#255) 
## What

PR 4/17 of the catalog system rollout. **Single clean cut** — the old flag is gone, replaced by `--example`. Alias changes from `-t` to `-e`. Stacks on #254.

- Rename `--template` → `--example` (alias `-e`) on `hyperframes init`
- Accept `--template` as a recognized-but-errored flag so users get a clear rename hint instead of citty silently ignoring the flag and producing a blank project
- Update all user-visible strings that referenced "template" as a user-facing concept in the init flow (picker prompt, step comments, offline-fallback suggestion)
- New `init.test.ts` covering both the success case (`--example` scaffolds) and the error case (`--template` exits 1 with rename hint)

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a).

## ⚠️ Breaking change

`--template` is no longer accepted. Example:

```bash
# before
npx hyperframes init my-video --template warm-grain

# after
npx hyperframes init my-video --example warm-grain
```

Users who still type the old flag will see:

```
The --template flag was renamed to --example. Example:
  npx hyperframes init my-video --example warm-grain
```

and the command exits with code 1. This is **user guidance, not backwards compat** — the old flag's behavior is fully gone.

## Docs (bundled per the tracker principle)

- `docs/templates.mdx` — every `--template` reference
- `docs/quickstart.mdx` — agent-mode and video-mode examples
- `docs/packages/cli.mdx` — prose, `--help` flag table, `-e` alias
- `packages/cli/src/docs/templates.md` — CLI-embedded help topic
- `README.md` and `CONTRIBUTING.md` — not affected (no flag references)

User-facing renames of the `templates.mdx` page title, nav entry, and URL route are deferred to PR 11 (catalog discoverability UX) as planned.

## Why

1. **"examples"** matches shadcn + Remotion convention for full-project scaffolds and frees the word "template" for future parameterization work (string templating, placeholder substitution)
2. Once `hyperframes add` lands in PR 5, "template" vs "block" vs "component" would be three subtly different concepts sharing one word — renaming the old one to "example" makes the taxonomy self-explaining

## How

- **citty silently ignores unknown flags.** Naively removing `--template` would cause `hyperframes init my-video --template warm-grain` to silently fall through and scaffold a blank project. So `--template` stays declared in the args schema, but its run handler immediately errors with a rename hint and exits 1
- **Internal names unchanged** — `templateId` local variables, `getStaticTemplateDir` function, `BUNDLED_TEMPLATES` constant. They're implementation details; their rename is scheduled for PR 5 when the compat shims in `packages/cli/src/templates/` are fully removed alongside the `init` refactor

## Test plan

- [x] `bun run test` in `packages/cli`: **72 passed** (was 70 on #254, +2 new `init.test.ts` cases). Same 4 pre-existing failures unchanged
- [x] **New unit tests** in `init.test.ts`:
  - `--example blank` non-interactive: exits 0, writes `index.html` to the target dir
  - `--template blank` non-interactive: exits non-zero, stderr contains the rename hint + corrected command line, target dir is **not** created
- [x] **Manual smoke:**
  - `npx hyperframes init /tmp/x --example blank` → "Created /tmp/x/"
  - `npx hyperframes init /tmp/y --template blank` → "The --template flag was renamed to --example..." exit=1
- [x] `bunx oxfmt --check` + `bunx oxlint` on changed files: clean
- [x] Pre-commit typecheck (core + studio): clean

## Incidental fix

Resolver test regression from PR 3's simplify follow-up: `loadAllItems`' warning-path test was still spying on `console.warn` after the `onWarn` callback refactor. Now uses the callback directly.

## Stacks on

#254 — base branch. When #254 merges, this rebases onto `main`.

## Next in stack

PR 5 — `feat(cli): add command + hyperframes.json`. The big UX PR where:
- `init.ts` gets fully ported to the new registry resolver
- Compat shims in `packages/cli/src/templates/` are removed
- Users gain the `add` verb for installing blocks and components into existing projects
- `hyperframes.json` project-config file lands

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 20:44:23 -07:00
James Russo 969474e843 feat(cli): registry resolver + installer (#254) 
## What

PR 3/17 of the catalog system rollout. Introduces the registry resolver/installer abstraction. No UX change — `init --template` still works identically. Stacks on #253.

**New module: `packages/cli/src/registry/`**
- `remote.ts` — fetches manifests (`registry.json`, `registry-item.json`) and item files from a GitHub-hosted registry. 24h cache on manifests; item files stream straight to `destDir`
- `resolver.ts` — `listRegistryItems`, `loadAllItems` (parallel fetch for picker UX), `resolveItem` (single-item fetch with `Available:` error)
- `installer.ts` — `assertSafeTarget` (runtime path-traversal guard) + `installItem` (parallel file download with up-front validation; all-or-nothing semantics)
- `index.ts` — barrel

**Registry content:**
- `registry/registry.json` — top-level manifest in PR 1's `RegistryManifest` shape. 8 examples
- `registry/examples/<id>/registry-item.json` — per-item manifest for each existing example, generated from legacy `templates.json` + HTML data-attribute probing
- `registry/examples/templates.json` — **deleted**, replaced by the above

**Compat layer:**
- `packages/cli/src/templates/{remote,generators}.ts` — thin shims that delegate to `../registry/`, keeping `init.ts`'s existing imports stable. `init.ts` doesn't move to the new API until PR 5 where it's part of a larger UX pass

**Tooling:**
- `scripts/generate-registry-items.ts` — idempotent one-off generator for this PR, kept in-repo for future example additions (`--only <name>` flag)

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a). Tracker entry in local `hyperframes-catalog-plan.md`.

## Why

Every future PR (`hyperframes add`, seed blocks, seed components, custom registries) otherwise has to keep piling onto the ad-hoc fetch + `cpSync` pattern in the old `fetchRemoteTemplate`. The new module is the single place that understands the registry wire format and file layout. **This is also where PR 1's schema comes alive.**

## How

### Scope-trimmed from the plan

- **No transitive dependency resolution yet.** Examples have no deps today. `resolveItem` doesn't walk `registryDependencies`; PR 5 adds that when blocks/components need it.
- **No ajv schema validation yet.** TS types + runtime path-traversal guard are the only safety nets. Full JSON-Schema validation lands when the registry starts accepting third-party content (PR 14 / custom registries).
- **init.ts refactor deferred to PR 5.** Compat shims keep this PR small and reviewable. PR 5 rewrites init alongside adding the `add` command.

### Safety

- `assertSafeTarget` rejects absolute paths, `..` segments, Windows drive letters, and any target that `path.resolve` shows to escape `destDir`. Mirrors the PR 1 schema `pattern`/`not.anyOf` on `target`, but runs at install-time so a registry that bypasses schema validation still can't write outside the project
- Up-front validation in `installItem` means a malformed item fails **before** any file is written. Atomic-ish semantics: all files land or none do

### Caching

- 24h manifest cache lives at `~/.hyperframes/cache/` per existing convention, but now keyed by `<baseUrl>__<kind>__<name>.json` so PR 14 custom registries can coexist

## Test plan

- [x] `bun run test` in `packages/cli`: **70 passed** (was 57 on #253, +13). Same 4 pre-existing failures (SRT/VTT whisper normalizer + `lintProject` clean-project test) — identical to main. No regressions
- [x] **Resolver unit tests (8):** filter by type, parallel load with fail-safe, resolve-by-name with `Available:` error message, unreachable-registry handling
- [x] **Installer unit tests (5):** accepts simple relative paths, rejects `..` segments, rejects Unix absolute paths, rejects Windows drive letters, permits `.` and dotfile-like names
- [x] **Smoke test**: `hyperframes init /tmp/x --template blank` (bundled code path, unchanged) works end-to-end
- [x] `bunx oxfmt --check` + `bunx oxlint`: clean
- [x] Pre-commit typecheck (core + studio): clean. CLI typecheck has 2 pre-existing errors (`render.ts`, `studioServer.ts` — unrelated `"mov"` format issue on main)
- [ ] **Smoke test remote fetch (`--template warm-grain`)** — verifiable only post-merge; registry paths live on `main` after this PR lands

## Breaking / migration

**No end-user-visible UX change.** `init --template <name>` still works the same way. Internally, `templates.json` is gone and the CLI now reads `registry.json` + `registry-item.json` per example.

Installed CLIs on old versions (`hyperframes@0.1.0`–`0.3.0`) already broke at PR 2 merge (see #253 rollout note). The next CLI release after this lands (`0.3.1`+) is the full fix.

## Commits

1. `generate-registry-items.ts` + generated manifests + deleted `templates.json`
2. Resolver + installer + compat shims
3. Unit tests

(All squashed into one commit on this branch; see `git log feat/registry-resolver ^refactor/registry-examples-dir`.)

## Stacks on

#253 — base branch. When #253 merges, this rebases onto `main`.

## Next in stack

PR 4 — `feat(cli)!: rename --template to --example`. Single clean cut, no alias. Tiny PR (~150 lines) that mostly updates `init.ts`'s argument schema, help text, and docs. Depends on this PR so the new flag name can be applied against the refactored code path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 20:41:23 -07:00
James Russo 69d9f08061 refactor: migrate templates/ → registry/examples/ (#253) 
## What

PR 2/17 of the catalog system rollout. **Physical directory rename.** Stacks on #252.

- `git mv templates/ registry/examples/` — all 8 example directories (`decision-tree`, `kinetic-type`, `nyt-graph`, `play-mode`, `product-promo`, `swiss-grid`, `vignelli`, `warm-grain`) plus `templates.json`
- `packages/cli/src/templates/remote.ts` — `TEMPLATES_DIR` constant from `"templates"` → `"registry/examples"`, exported for regression testing
- `scripts/generate-template-previews.ts` — `remoteTemplatesDir` resolved to the new path
- Comment updates in `packages/cli/src/templates/generators.ts` and `packages/cli/src/commands/init.ts`
- New regression test `packages/cli/src/templates/remote.test.ts` pinning the path constants so future reverts fail a test instead of silently breaking installed CLIs

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a).

## Why

The current `templates/` directory is a flat "things that scaffold projects" bucket. The catalog model splits content into three tiers: **examples** (full projects — what today's templates are), **blocks** (sub-compositions), and **components** (effect snippets). `registry/examples/` is the canonical home for what was previously at `templates/`, and this PR makes room for `registry/blocks/` and `registry/components/` in future PRs without top-level clutter.

## How

- `git mv` preserves file history — GitHub renders these as renames, not deletions + additions.
- Remote template fetch via giget reads `TEMPLATES_DIR`, so updating that one constant is sufficient for the CLI's remote code path.
- The CLI's **internal** `packages/cli/src/templates/` directory (which holds the `blank` and `_shared` bundled assets plus `generators.ts`/`remote.ts`) is a separate concept and is **not** touched here. Renaming that module belongs to PR 3 where the abstraction changes to a registry resolver.
- `templates.json` keeps its existing shape and location (now at `registry/examples/templates.json`). **PR 3 will transform it** to the new `registry.json` shape introduced in PR 1 and generate a per-item `registry-item.json` for each example. Leaving the shape change to PR 3 keeps this PR a pure physical move.

## ⚠️ Breaking change for previously-installed CLIs (`hyperframes@0.1.0` – `0.3.0`)

**What happens:** every published CLI version has `TEMPLATES_DIR = "templates"` baked in. After this PR lands on `main`, those CLIs will 404 on:

- `raw.githubusercontent.com/heygen-com/hyperframes/main/templates/templates.json` (manifest list) — caught silently in `listRemoteTemplates`, so the template picker falls back to showing only `blank`
- `github:heygen-com/hyperframes/templates/<id>#main` (giget download) — raises "Template downloaded but missing index.html"

**Decision: accept the break.** Hyperframes is pre-1.0 OSS with a small installed base; complex mitigations (dual-path fetch, redirect stubs, manifest-at-old-path with empty array) add permanent maintenance cost for a one-time rename.

**Rollout plan:**

1. Merge #252 (PR 1 — types & schemas) first
2. Merge this PR (#253)
3. Ship a patched CLI release (`hyperframes@0.3.1`) in the same work-day. Already-pinned old CLIs break on remote examples, but upgrading restores full functionality
4. Note the break in release notes + `CHANGELOG.md` under the `0.3.1` entry

Users still on an older CLI will see the failure only if they invoke `hyperframes init` with `--template <non-blank>`; `--template blank` (bundled) continues to work offline on every version.

## Test plan

- [x] `bun run test` in `packages/cli`: **57 passed** (was 55 on main, +2 regression tests for the path constants). Same 4 pre-existing failures (SRT/VTT whisper normalizer + `lintProject` clean-project test) — unchanged from main. No regressions
- [x] **Manual smoke test**: `hyperframes init /tmp/x --template blank` works (bundled code path, unchanged)
- [x] `bunx oxfmt --check` + `bunx oxlint`: clean
- [x] `bun run typecheck` (core + studio, pre-commit hook): clean
- [ ] **Manual smoke test for remote fetch (`--template warm-grain`)** — not verifiable locally before merge. Remote fetch resolves `github:heygen-com/hyperframes/registry/examples/<id>#main`, which doesn't exist until this PR lands. Will work on `main` immediately after merge.

## Breaking / migration

- Internal repo path changes only. `--template` CLI flag continues to accept the same template names.
- See "Breaking change for previously-installed CLIs" above — decision is to ship a simultaneous CLI release rather than add a compat shim.

## Commits

1. `d691bd1` — initial rename + CLI path constant update
2. `fc0c642` — review feedback: docstring fix, regression tests, clarifying comment in `init.ts`, export constants for testing

## Stacks on

#252 — base branch. When #252 merges, this rebases onto `main`.

## Next in stack

PR 3 — `feat(cli): registry resolver + installer`. Transforms `templates.json` to the new `registry.json` shape (from PR 1's schema), generates `registry-item.json` for every existing example, introduces `packages/cli/src/registry/{resolver,installer,remote}.ts`, renames the `packages/cli/src/templates/` CLI module, and refactors `init` to call through the new abstraction.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 20:21:23 -07:00
James RussoandClaude Opus 4.6 eb338ae859 feat(core): add registry schema + TS types (#252) 
* feat(core): add registry schema + TS types

PR 1/17 of the catalog system rollout. Foundation for a shadcn-style
registry with three item tiers: examples (full projects), blocks
(sub-compositions), components (effect snippets).

## What

- TS types: RegistryItem (discriminated union of ExampleItem/BlockItem/
  ComponentItem), RegistryManifest, FileTarget, ItemType, FileType
- JSON Schemas: schemas/registry.json, schemas/registry-item.json
- Compile-time exhaustiveness asserts on ITEM_TYPES/FILE_TYPES so adding
  to the TS union without updating the constant stops compiling
- Drift-guard test: schema enums must equal ITEM_TYPES/FILE_TYPES by
  set-equality; exactly 2 distinct type enums in registry-item.json
- Public API via new ./registry export path plus re-exports from root;
  schemas exposed via ./schemas/registry.json export for external tooling

## Why

- Every downstream PR (resolver, installer, hyperframes add, docs
  codegen, CI previews, skill, catalog command) builds on these types
- Getting the shape right now avoids painful migrations later

## How

- Discriminated union enforces that components do not have dimensions
  or duration and examples/blocks must have them (schema mirrors via
  if/then/else on the type discriminant)
- target path pattern rejects .. segments, Unix absolute paths, and
  Windows drive letters (defense-in-depth; CLI validates at runtime in
  PR 3)
- name pattern requires alphanumeric start and end (no trailing hyphens)
- Optional metadata: version, author, license, deprecated, minCliVersion
- additionalProperties: false on nested objects (catches typos on
  critical fields) but relaxed on top-level RegistryItem (allows
  third-party custom metadata in PR 15 custom registries)

## Test plan

- [x] Unit tests: 11 new tests covering type guards, discriminant
      narrowing, schema/TS drift guards, schema \$id sanity, optional
      metadata acceptance, and compile-time checks (via @ts-expect-error)
- [x] bun run test in packages/core: 445 passed (was 434 on main,
      +11 from this PR)
- [x] bunx oxfmt and bunx oxlint: clean
- [x] bun run typecheck: clean
- [ ] Manual testing: N/A (types + schemas only)
- [ ] Documentation updated: per-item doc pages land in PR 9 (codegen
      from these manifests); guide updates in PR 10+

## Breaking / migration

None. Pure additive — new module, new export paths, no existing
surface touched.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(core): remove version field + hyperframes:demo file type

Address review feedback from Miguel:

- Remove `version` from RegistryItemBase + schema. Per shadcn model,
  the registry is versioned by git tags, not per-item. The adversarial
  review added it; the original design doc was correct.
- Remove `hyperframes:demo` from FileType union + FILE_TYPES constant
  + schema. Demo files exist on disk for the CI preview pipeline but
  are NOT installed to user projects and should not appear in
  registry-item.json files[]. Neither shadcn nor Remotion has a
  dedicated demo file type — demos are just compositions.
- Add `required: ["type"]` to the if-condition in the schema's
  allOf discriminant (Miguel's nit — makes the condition self-
  contained)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): add shader-transitions to Dockerfile.test

PR #251 added packages/shader-transitions/ to the workspace but didn't
update Dockerfile.test to COPY its package.json. This caused
`bun install --frozen-lockfile` to fail in the regression Docker build:
bun saw a lockfile referencing @hyperframes/shader-transitions but the
package.json wasn't present in the container, so it wanted to remove
the entry — triggering "lockfile had changes."

Verified: Docker build passes with `--no-cache` after this fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-13 20:18:37 -07:00
Vance Ingalls cb3d94c2a5 feat: add @hyperframes/shader-transitions package (#251) 
## Summary

New `@hyperframes/shader-transitions` package that encapsulates WebGL shader transitions into a single `HyperShader.init()` call. Replaces ~200 lines of per-composition boilerplate that LLMs failed to wire correctly 60% of the time.

### API

```js
var tl = HyperShader.init({
  bgColor: "#0a0a1a",
  accentColor: "#6366f1",
  scenes: ["scene1", "scene2", "scene3", "scene4", "scene5"],
  transitions: [
    { time: 7.2, shader: "cross-warp-morph", duration: 0.7 },
    { time: 15.2, shader: "domain-warp", duration: 0.7 },
  ]
});
tl.from("#s1-title", { y: 50, opacity: 0, duration: 0.7 }, 0.3);
```

### What the library handles

- **13 shader programs**: domain-warp, ridged-burn, whip-pan, sdf-iris, ripple-waves, gravitational-lens, cinematic-zoom, chromatic-split, glitch, swirl-vortex, thermal-distortion, cross-warp-morph, light-leak
- **html2canvas** bundled as dependency (not CDN) — single script tag for CLI users
- **DOM-during-holds**: canvas hidden between transitions, GSAP animations play on live DOM
- **Async capture with pause/resume**: timeline pauses during capture, resumes after textures uploaded — prevents progress tween from running ahead
- **Accent color theming**: `accentColor` derives dark/mid/bright uniforms. Burns, glows, leaks match the composition palette
- **Graceful degradation**: falls back silently when WebGL unavailable

### Code quality (from 3 review agents)

- No `!` non-null assertions — all WebGL creation calls throw on failure
- Vertex shader compiled once, cached across all programs
- Uniform/attribute locations cached per program via WeakMap (not looked up every frame)
- Captured canvases freed after texture upload (8MB each)
- Single timeline creation (was creating two, discarding one)
- Shared `tickShader()` render callback (was copy-pasted)
- `.finally()` for DOM restore in capture (was duplicated in `.then`/`.catch`)
- `parseHex` validates input (was silently producing NaN on invalid hex)
- Dead `ND`/`CP` shader library exports removed

### Shader-compatible CSS rules (transitions.md)

6 rules for compositions using shader transitions:
1. No `transparent` in gradients (canvas interpolates through black)
2. No gradient backgrounds on elements < 4px
3. No CSS variables on captured elements
4. `data-no-capture` for uncapturable decoratives
5. No gradient opacity < 0.15
6. Every `.scene` must have explicit `background-color` matching `bgColor`

### Build output

- IIFE (~214KB with html2canvas bundled, ~65KB gzipped) — `window.HyperShader`
- ESM + CJS + TypeScript declarations
- tsup build following `@hyperframes/player` conventions

## Test plan
- [ ] `bun run build` succeeds (includes shader-transitions)
- [ ] `bunx oxlint packages/shader-transitions/src/` — 0 errors
- [ ] Create a composition using `HyperShader.init()` — verify transitions fire, DOM animations play, accent colors match
- [ ] Test graceful degradation: composition works without WebGL (no transitions, no crash)
- [ ] Verify pause/resume: scrub to transition boundary — no jump in progress

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:44:00 -07:00
Vance Ingalls 5de2af5bde feat(skills): improve hyperframes composition quality rules (#250) 
## Summary

Overhaul the hyperframes composition skill based on 26 eval rounds (~100 generated compositions). The goal: prevent known AI design tells and composition bugs while giving the LLM maximum creative freedom.

### Typography (`fonts.md` → `typography.md`)
- Two-tier banned font list (32 fonts): tier 1 bans training-data defaults, tier 2 bans the reflex replacements
- Font discovery script: queries Google Fonts API, 5 dynamic categories, top 5 randomized per run
- Selection philosophy: register-first thinking, cross-check assumptions

### Google Fonts on-demand (`deterministicFonts.ts`)
- Any Google Font works without pre-bundling — compiler fetches woff2 at compile time
- Cached to `~/.cache/hyperframes/fonts/<slug>/<weight>-<style>.woff2`
- Parallel woff2 fetches via `Promise.allSettled` (was sequential)
- Single `mkdirSync({ recursive: true })` per family (was `existsSync` x11)
- Skip redundant `readFileSync` when buffer is already in memory from fetch

### Layout rules (`SKILL.md`)
- Flexbox with gap for content text — prevents overlap from absolute positioning
- `position: absolute` reserved for decoratives only
- Cards/containers explicitly banned

### Background layer (`house-style.md`)
- 3-5 persistent decorative elements per scene (glows, ghost text, accent lines)
- All decoratives MUST have ambient GSAP animation — static decoratives banned
- WRONG/RIGHT code examples

### Transition rules (`SKILL.md`)
- Always use transitions, always entrance animations, exit animations banned except final scene
- WRONG/RIGHT code examples showing banned exit patterns

### Other
- Flash cut transition removed
- CLAUDE.md: `bun install` / `bun run build` / `bun run test` (was pnpm)
- house-style.md trimmed from 184 to ~80 lines
- SKILL.md trimmed from 364 to ~230 lines

## Test plan
- [ ] `bun install` succeeds, workspace links resolve
- [ ] `bun run build` succeeds
- [ ] `npx hyperframes lint` passes on existing compositions
- [ ] Generate a composition with `/hyperframes` skill — verify flexbox, background decoratives with animation, entrance-only animations, no banned fonts
- [ ] Verify Google Fonts on-demand: use a non-bundled font, run `npx hyperframes preview`

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:40:57 -07:00
Miguel Ángel a9428e0b02 feat(studio): format info tooltip on export selector (#257) 
## Summary

- Adds a hover tooltip (?) next to the format dropdown in the render queue export bar
- Shows the selected format's details (codec, use case) plus a comparison with the other two formats
- Helps users pick between MP4 (general), MOV/ProRes 4444 (transparent video for editors), and WebM/VP9 (transparent for web)

## Test plan

- [x] Open studio, go to the render queue panel
- [x] Hover over the (?) icon next to the format dropdown — tooltip appears above
- [x] Switch format in the dropdown — tooltip content updates to show the selected format first
- [x] Move pointer away — tooltip dismisses
- [x] Verify tooltip doesn't clip or overflow the panel

<img width="420" height="263" alt="image" src="https://github.com/user-attachments/assets/f4bd8bf7-65ed-45ab-ac53-577b4985fa34" />
 2026-04-14 03:37:40 +02:00
Miguel Ángel 58ddb11bc5 chore: release v0.3.0 (#249) 
## Summary

Coordinated minor bump across all published packages. No source changes in this PR itself; it is the version stamp for everything that landed on main since v0.2.5.

## Version bumps

| Package | from | to |
|---|---|---|
| `@hyperframes/cli` | 0.2.5 | **0.3.0** |
| `@hyperframes/core` | 0.2.5 | **0.3.0** |
| `@hyperframes/engine` | 0.2.5 | **0.3.0** |
| `@hyperframes/player` | 0.2.7 | **0.3.0** |
| `@hyperframes/producer` | 0.2.5 | **0.3.0** |
| `@hyperframes/studio` | 0.2.9 | **0.3.0** |

Between 0.2.5 and this release, `player` and `studio` received several patch versions on npm as we iterated on the bundler, entry point, and SSR issues. 0.3.0 collapses that into a single coordinated minor so the ecosystem is aligned again.

## What is in v0.3.0

### `@hyperframes/player`

**Restored package entry points to the compiled `dist/` output.** 0.2.5 shipped with `"main": "./src/hyperframes-player.ts"` but the published tarball only included `dist/` via the `"files"` field. Every consumer trying to import the package failed with `Module not found: Can't resolve '@hyperframes/player'`. Entry points now point at the built JS/`.d.ts` files inside `dist/`.

**DOM-based root timeline resolution in the ready probe.** In a bundled preview, `window.__timelines` contains the master composition alongside its sub-compositions, for example:

```js
{
  main: GSAPTimeline(14s),
  intro: GSAPTimeline(1.5s),
  'scene2-4-canvas': GSAPTimeline(12.6s),
  'scene5-logo-outro': GSAPTimeline(3.2s),
}
```

The probe used to select the adapter with `keys[keys.length - 1]`. Object key ordering meant the last-registered sub-composition would win, so the `ready` event reported a sub-composition's duration (e.g. 3.2s) instead of the master's 14s. The probe now looks up the root composition id from the outermost `[data-composition-id]` element in the iframe DOM and uses its key. Falls back to the last key when no element is present, so standalone sub-composition previews keep working.

### `@hyperframes/studio`

**`useTimelinePlayer.getAdapter()` uses the same DOM-based root id lookup** as the player. Previously play, pause, seek, and duration readout were all driven by whichever sub-composition happened to register its timeline last.

**`Player.tsx` loads `@hyperframes/player` lazily.** The component used to call `import "@hyperframes/player"` at module scope, which runs the package's `customElements.define(...)` side effect during module evaluation. `HTMLElement` does not exist in the Node runtime, so any consumer page that transitively imported the studio during server rendering threw:

```
ReferenceError: HTMLElement is not defined
  at module evaluation (@hyperframes/studio/src/player/components/Player.tsx)
```

The import now runs inside the mount effect via `import(...)` so it only evaluates in the browser. Added a cancellation flag and deferred cleanup so a fast unmount before the dynamic import resolves does not leak listeners or DOM nodes.

**Captions module imports stripped of `.js` extensions.** Files under `src/captions/` imported siblings as `./types.js` and `./parser.js`. That is legal ESM TypeScript but Turbopack and several other bundlers refuse to resolve those specifiers against `.ts` files inside `node_modules`, breaking any consumer build that transitively pulled in the captions module. Captions now uses extensionless imports, matching the rest of the studio codebase.

### `@hyperframes/core`, `@hyperframes/cli`, `@hyperframes/engine`, `@hyperframes/producer`

Version bump only, no source changes since 0.2.5. Kept on the same version so the ecosystem is easier to reason about.

## Impact for consumers

If you use `@hyperframes/studio` in a Next.js app:
- The play button in a bundled preview reports the correct composition duration and drives the master timeline.
- The session page no longer 500s in dev mode when the studio barrel is imported (the SSR fix).
- Turbopack builds that transitively load the captions module no longer fail on `Cannot resolve './types.js'`.

If you use `@hyperframes/player` directly:
- Consumer bundlers can resolve the package again (dist entry points restored).
- The `ready` event duration reports the master, not a sub-composition.

## After merge

Publish each package to npm with `pnpm publish` (workspace deps auto-resolve).
 2026-04-14 01:16:06 +02:00
Miguel Ángel bf0d698858 fix(studio): SSR-safe player load, captions import cleanup (#248) 
* fix(studio): load @hyperframes/player lazily to support SSR

Player.tsx had a bare `import "@hyperframes/player"` at module scope. The
player package registers a class that extends HTMLElement as a side effect,
and HTMLElement doesn't exist in a Node server runtime. Any consumer that
imported from @hyperframes/studio during server-side rendering (e.g. the
Next.js App Router evaluating a client component for SSR) threw
`HTMLElement is not defined`.

Move the import inside the mount effect via dynamic `import(...)` so it
only runs in the browser, and wire up a cancellation flag and deferred
cleanup so a fast unmount doesn't leak listeners or DOM nodes.

* fix(studio): remove .js extensions from captions-internal imports

The captions module imported sibling files as `./types.js` and
`./parser.js`. That's legal ESM TypeScript, but Turbopack (and other
bundlers) refuse to resolve those specifiers against .ts files when the
package is consumed from node_modules — the rest of @hyperframes/studio
uses extensionless imports for that reason.

Align captions with the rest of the codebase so the package builds
without bundler-specific configuration in consumers.

* chore: release @hyperframes/player@0.2.7 and @hyperframes/studio@0.2.9

Ships the root-timeline resolution fix (#247), the SSR-safe player load,
and the captions import cleanup.
 2026-04-14 00:15:16 +02:00
Miguel Ángel f40447f2e8 fix(player,studio): resolve root timeline from DOM instead of last key (#247) 
Bundled previews register a master composition alongside its sub-compositions
in `window.__timelines`, e.g. { main, intro, scene2, scene5 }. Both the
player's probe and studio's getAdapter() were using `keys[keys.length - 1]`
to pick the adapter, which returned whichever timeline was registered last.

That made the player report the final sub-composition's duration as the
video length (e.g. 3.2s instead of the master's 14s) and play/pause/seek
targeted that sub-composition instead of the full composition.

Look up the outermost `[data-composition-id]` element in the iframe DOM
and use its id to select the right timeline. Falls back to last-key when
no element is present (standalone sub-composition previews) so drill-down
views keep working.

Also restores `main`/`import` entry points on @hyperframes/player to
point at compiled dist output (the src/ paths broke workspace consumers
that only receive the published tarball).
 2026-04-13 23:15:08 +02:00
Miguel Ángel 1dd898786c chore: release v0.2.5 (#246) 2026-04-13 17:55:40 +02:00
Miguel Ángel 1149602bc9 fix(studio): support web-component refs in useTimelinePlayer (#245) 
* fix(studio): support web-component refs in useTimelinePlayer

The studio's `useTimelinePlayer` hook returns an `iframeRef` that
consumers attach to an `<iframe>` element. When consumers wrap the
iframe in a custom element (e.g. `<hyperframes-player>`) that puts
the iframe inside its shadow DOM, every `iframeRef.current.contentWindow`
access returned `null` and `getAdapter()` silently failed — meaning
timeline seek, play, pause, and `refreshPlayer` all became no-ops.

Changes:
- Add `resolveIframe(el)` helper that returns the underlying iframe
  whether the host is the iframe itself, a custom element with a
  shadow-DOM iframe, or a wrapper with a descendant iframe.
- Export `resolveIframe` from the studio so consumers can pre-resolve
  the iframe before assigning it to `iframeRef`.
- Internal `useTimelinePlayer` keeps the strict `HTMLIFrameElement`
  ref type, so existing consumers attaching directly to an `<iframe>`
  are unaffected.

Also adds:
- JSDoc on the player's `iframeElement` getter.
- "Advanced: iframe access" docs section in `packages/player/README.md`
  and `docs/packages/player.mdx`.
- Type-safety lint rules in `.oxlintrc.json` and a "Type-safety
  conventions" section in `CONTRIBUTING.md`.

Backward compatible — App.tsx and NLELayout.tsx continue to work
unchanged.

* chore(lint): defer no-explicit-any rule; it broke existing codebase

The new rules added 37 errors across 32 existing files — mostly
legitimate `window as any` casts at browser-global and test-mock
boundaries. Enabling them without fixing all violations breaks CI.

Revert the `.oxlintrc.json` additions and soften the CONTRIBUTING.md
wording to describe the convention without claiming lint enforcement
(that enforcement will come in a follow-up PR that fixes all sites).
 2026-04-13 17:53:01 +02:00
James RussoandClaude Opus 4.6 18de86e4bd fix(player): handle Infinity duration; add lint rules for data-duration and Math.ceil overshoot (#243) 
* fix(player): handle Infinity duration from runtime gracefully

When compositions have repeating animations without data-duration, the
runtime sends durationInFrames: Infinity. The player now ignores
non-finite duration values instead of displaying "Infinity:NaN" in the
controls. formatTime also returns "0:00" for non-finite inputs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(lint): add data-duration and Math.ceil overshoot rules

- Add root_composition_missing_data_duration warning when the root
  composition element is missing data-duration, which causes the runtime
  to infer Infinity for loop-inflated timelines.
- Add gsap_repeat_ceil_overshoot warning that catches
  repeat: Math.ceil(d/c)-1 patterns which overshoot the intended
  duration. Recommends Math.floor instead.
- Fix gsap_infinite_repeat fixHint to suggest Math.floor (not Math.ceil).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(player): wait for injected runtime before declaring ready

When the player auto-injects the runtime script (because the
composition has GSAP timelines but no runtime), it would immediately
declare ready on the next probe cycle — before the runtime script
finished loading from CDN. This caused play() to send a postMessage
that nobody received, making autoplay silently fail.

Now the probe waits for the runtime bridge (__hf or __player) to
appear before proceeding to the ready state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-12 10:41:13 -07:00