mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
f7de8ebcb1939f830e923689bf2e30ebb9ffec2d
152
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
577e822fa6 | chore: release v0.4.10 | ||
|
Miguel Ángel
|
62ce3679e6 | chore: release v0.4.9 | ||
|
Miguel Ángel
|
479e6c3f81 | chore: release v0.4.8 | ||
|
Miguel Ángel
|
03c2158e0f |
ci: verify on windows-latest + fix cross-platform build bugs it surfaced (#342)
* fix(cli): make build copy cross-platform and deterministic
* fix(core): keep rewritten asset URLs POSIX on Windows
* ci(windows): add render verification workflow
* ci(windows): load canary gsap from cdn
* build: use dependency-aware workspace ordering
* Revert "build: use dependency-aware workspace ordering"
This reverts commit
|
||
|
Vance IngallsandClaude Opus 4.6
|
99a903be2f |
feat(hdr): layered HDR compositing, shader transitions, and HDR image support (#268)
* feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes - 15 GLSL→TypeScript shader transitions on rgb48le buffers - Dual-scene compositing with scene detection via window.__hf.transitions - --hdr flag gates ffprobe probing (zero overhead on SDR compositions) - Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT - Buffer.from() copy in writeFrame() fixes streaming encoder race condition - SDR rendering fixes (three stacked bugs) - Object.assign fix for window.__hf preservation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: tighten shader smoke thresholds + assert .scene contract - Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268. * fix(hdr): restore VIRTUAL_TIME_SHIM and applyRenderModeHints in renderOrchestrator Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes") accidentally removed two pieces of the deterministic rendering pipeline: 1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`, which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven animations advance only when `window.__hf.seek(t)` is called. 2. The `applyRenderModeHints` function and its post-`compileForRender` call site, which auto-forces screenshot capture mode for compositions the compiler flagged as needing it (RAF, iframes, etc.). Without (1), RAF animations advanced by wall-clock between the main-loop seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat` lost its automatic fallback to screenshot mode and the child-document motion stopped being captured. Both helpers are still produced by `htmlCompiler` and exercised by `renderOrchestrator.test.ts` — the orchestrator just stopped calling them. Restored: - Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js` - Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer` call sites (probe + main render) - Re-add `applyRenderModeHints` (matching the test expectations) and call it immediately after `compileForRender` - Persist `renderModeHints` in `summary.json` and the "Compiled composition metadata" log line Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression failures on `feat/hdr-layered-compositing`. Made-with: Cursor * test(engine): expand sampleRgb48le coverage + audit Uint16Array alignment Adds: - 8 new sampleRgb48le bilinear-interpolation tests covering boundary pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers. - uint16-alignment-audit.test.ts documenting the alignment requirement for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE. Background: ~105 hot-loop sites in shader transitions still use readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut overhead but requires guaranteed even byteOffsets — these tests document the contract before any future refactor lands. * fix(engine,producer): mask DOM layers during HDR layered compositing The HDR layered compositor blits z-ordered layers over a shared canvas. DOM layers used a full-page screenshot from `captureAlphaPng`, which captures *every* painted pixel on the page — root background, sibling-scene content, overlay UI elements that aren't part of the current layer. Those opaque pixels were then blitted over the canvas, overwriting any HDR content composited beneath in earlier layers. The previous workaround toggled `display:none` on hide ids via `hideVideoElements`/`showVideoElements`. That correctly hid native videos but did nothing about the root composition's background or about overlay elements that the layer grouping considered part of a different layer. This commit replaces the workaround with a precise CSS mask installed before each DOM screenshot: 1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and re-shows the layer's elements (and their descendants and their injected `__render_frame_*` siblings) with `visibility: visible !important`. CSS visibility is *not* multiplicative through descendants — a child with `visibility: visible` overrides an ancestor's `visibility: hidden`, so deeply nested layer content still paints even though every intermediate ancestor is hidden by the mass-hide rule. 2. Non-layer data-start ids are inline-hidden with `visibility: hidden !important`. Inline `!important` beats stylesheet `!important`, so this overrides the show rule for elements that fall under a show selector but should NOT paint — most importantly HDR videos and other-layer SDR videos that live as descendants of `#root`. 3. `removeDomLayerMask` tears the stylesheet down and clears the inline `visibility`/`opacity` properties so subsequent video frame injection gets a clean slate. Crucially the mask only sets `visibility`, never `opacity`. CSS opacity *is* multiplicative — `opacity: 0` on `#root` would zero out every descendant including layer videos, even with `visibility: visible`. We also extend `initTransparentBackground` to force the composition root (`[data-composition-id]`) transparent in addition to `html`/`body`, because compositions almost always set `#root { background: ... }` and that background paints across the whole viewport otherwise. Both compositing paths use the new helpers: - The per-layer DOM branch (`compositeToBuffer`) for normal frames. - The transition path (single DOM screenshot per scene) so transition frames also get a clean per-scene capture. Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`: per-layer pixel-add accounting, dumps of every captured DOM PNG, and a periodic raw `rgb48le` snapshot of the composite buffer. These were essential to diagnosing the root-overwrite bug and stay zero-cost in normal renders. Also stops the workDir / per-video frame-dir cleanup when `KEEP_TEMP=1` so the dumps survive past frame N. Made-with: Cursor * fix(engine): preserve GSAP-applied opacity across DOM-layer captures SDR clips inside an HDR composition were rendering at full opacity even when the user had animated their wrapper opacity (e.g. fade-in or yoyo). Two bugs in the per-layer screenshot path conspired to drop the GSAP-applied opacity on the floor: 1. removeDomLayerMask was unconditionally calling `el.style.removeProperty("opacity")` on every wrapper after each layer capture. applyDomLayerMask only ever sets `visibility`, so the only inline opacity present is the value GSAP wrote. Stripping it between layer captures means that on the next capture (at the same timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline is already at that time — the opacity is never restored, and the wrapper renders fully opaque. 2. injectVideoFramesBatch was reading the source <video>'s computed opacity via `parseFloat(computedStyle.opacity) || 1` and copying it onto the injected <img>. Because syncVideoFrameVisibility forces the <video> to `opacity: 0 !important` to hide it during capture, the computed value is always 0, which `|| 1` then silently flips to full opacity. The <img> is a sibling of the <video> inside the same wrapper, so it should inherit opacity from the wrapper directly instead of having a value hard-set on it. Fix both: drop the opacity removal in removeDomLayerMask, skip opacity when copying visual properties from <video> to <img>, and explicitly clear any stale inline opacity on the <img> so it inherits from the wrapper that GSAP is animating. Made-with: Cursor * fix(producer): correct hdrLayerStartTimes typo to hdrVideoStartTimes The diagnostic logging block in executeRenderJob's HDR layer composite path referenced an undeclared `hdrLayerStartTimes` map. The correct variable, declared and populated earlier in the same function, is `hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer masking work and broke the producer build/typecheck on CI. Made-with: Cursor * fix(engine): restore video opacity copy to injected frame img Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on the assumption that the <img> sibling would inherit GSAP's opacity from a shared wrapper. That breaks any composition where GSAP animates opacity directly on the <video> element itself: the <img> has no animated ancestor and renders at full opacity throughout any fade, even when the user's intent is partial or zero opacity. The CI `style-7-prod` and `style-8-prod` regressions caught this: the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut because the <img> inherited opacity 1 regardless of GSAP's tween. Restore the old explicit copy from `computedStyle.opacity` to the <img>'s inline opacity, with the `|| 1` fallback intentionally preserved. The fallback is load-bearing: GSAP's seek does not re-apply tweens that have already completed, so post-fade frames read opacity 0 from the stale `opacity: 0 !important` we apply to hide the native <video>. The `|| 1` recovers the tween's end-state opacity 1 for those frames, matching the final on-screen intent and the existing baseline renders. Handles both DOM shapes: - GSAP on wrapper: video's own computed opacity is 1, img set to 1, wrapper's opacity applies via stacking as before. - GSAP on <video>: video's computed opacity is the tween value, copied to img directly since they are siblings. Fixes: - style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33) - style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
James
|
f622d39962 | chore: release v0.4.7 | ||
|
James
|
38b7cb1c66 | chore: release v0.4.6 | ||
|
|
ad11de698c |
feat(producer): auto-fallback screenshot capture for raf and iframes (#331)
* fix(core): drive adapter seeks when composition has no GSAP timeline renderSeek returned early when deps.getTimeline() was null, skipping the onDeterministicSeek call that drives all frame adapters (CSS, WAAPI, Lottie, Three.js). That meant compositions using any non-GSAP animation primitive froze on their initial frame during capture. Now we still quantize the seek time and fire onDeterministicSeek even without a timeline, so each adapter gets a chance to advance. GSAP compositions are unaffected — timeline-driven seek still takes the same path it did before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(producer): auto-fallback screenshot capture for raf and iframes Co-Authored-By: Codex <codex@openai.com> * test(producer): add render compatibility regression fixtures Co-Authored-By: Codex <codex@openai.com> * fix(core): scrub CSS animations via WAAPI currentTime Co-Authored-By: Codex <codex@openai.com> * test(producer): cover css keyframe renders Co-Authored-By: Codex <codex@openai.com> * fix(producer): propagate virtual time into iframe documents Co-Authored-By: Codex <codex@openai.com> * test(producer): refresh iframe docker golden Co-Authored-By: Codex <codex@openai.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Codex <codex@openai.com> |
||
|
Miguel Ángel
|
59aa2c9ec9 | chore: release v0.4.5 | ||
|
James
|
0f4fcbeed8 | chore: release v0.4.4 | ||
|
Miguel Ángel
|
c49181f1fa |
fix(player): address #298 review — tighter drift, dynamic proxies, ownership event (#307)
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment. ## Significant ### 1\. Drift threshold 150 ms → 50 ms _mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity. **Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction. ### 2\. Dynamic sub-composition media proxies Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track. Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect. ### 3\. `bridgeMuted` sticky in `syncRuntimeMedia` The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message. `syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs. ### 4\. Reset `_audioOwner` on iframe reload The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently). `_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`. ## Worth addressing ### 5\. Promotion → observable event + reason Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state. **Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once. ### 6\. Parent proxy play() rejection → `playbackerror` event Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back. ### 7\. Mobile verification on real hardware Tested with a tunnel in a real iOS device. ## Test gaps (from review) - `userMuted` stickiness (mirror of the existing `outputMuted` test). - **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`. - **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic. - **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once. - **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote). - **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately. - **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag. ## Minor - One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear). ## What's good (from the review) Kept as-is — noted for posterity: - `muted` vs `volume` framing (orthogonal channels). - Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing. - Two orthogonal mute channels. - Backwards compat (new actions / messages safely ignored by either side). ## Test results - `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe) - `packages/core/src/runtime/bridge.test.ts` — **15 tests pass** - `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface) - Typecheck green on `core` + `player` - `tsup` build green on `core` / `player` / `cli` - Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold. ## Test plan - [x] Unit tests (83 total across touched files) - [x] Typecheck clean - [x] Build clean - [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened - [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block - [x] Physical iOS / Android device verification (unchanged since #298) |
||
|
Miguel Ángel
|
46afe2ea5d | chore: release v0.4.3 | ||
|
Miguel Ángel
|
3256551a5e |
fix(player): single-owner audio to prevent double voice in preview (#298)
## Summary Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip). ## Root cause Two audio pipelines were playing the same source in parallel: 1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path. 2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message. The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks. Confirmed via agent-browser instrumentation on `factory-series-c-video`: - 6 `volumechange` events per play cycle (mute-fight signature) - Both copies audible at `volume=1`, offset growing 23ms → 80ms - Every seek widened the drift further PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback. ## Fix Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime. - **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash. - **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold. Two orthogonal mute channels replace the volume fight: | Channel | Purpose | |---|---| | `set-muted` | User's mute preference (existing, unchanged) | | `set-media-output-muted` | Internal ownership handoff (new) | `syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it. ## Why this shape - **Single owner, explicit transition.** No races, no tug-of-war. - **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing. - **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact. - **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift. - **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior. - **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files. ## Files changed - `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types. - `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields. - `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`. - `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`. - `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session. - `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership. ## Verified end-to-end with agent-browser on `factory-series-c-video` **Runtime ownership (default — desktop studio):** | | Before | After | |---|---|---| | `PARENT.play()` calls per play cycle | 1 | **0** | | iframe `volumechange` events | 6 | **0** | | Audible streams | 2 (drifting) | **1 (iframe)** | **Parent ownership (simulated autoplay block — direct message):** | | Value | |---|---| | iframe audio | `muted=true`, `volume=1` (untouched) | | parent audio | `muted=false`, `volume=1`, audible | | Parent ↔ iframe `currentTime` offset | ~6 ms steady state | | Offset > 150 ms | corrected by mirror sync | **Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:** Event timeline captured via agent-browser instrumentation: ``` t=0.0 ms IFRAME.play() called ← runtime attempts playback t=0.4 ms IFRAME.play() REJECTED: NotAllowedError ← simulated mobile gate t=0.4 ms →IFRAME bridge set-media-output-muted=true ← player promotes t=0.6 ms PARENT.play() called ← parent proxy starts t=0.8 ms ←IFRAME msg media-autoplay-blocked ← runtime signal t=1.0 ms PARENT.play() resolved ← audible t=1.3 ms IFRAME muted=true, volume=1 ← iframe silenced via native muted ``` Steady state at t=4 s under promoted parent ownership: | Element | currentTime | paused | volume | muted | |---|---|---|---|---| | Parent audio | 4.060 s | false | 1.0 | **false** (audible) | | Iframe audio | 4.068 s | false | 1.0 | **true** (silent) | **Offset: 8 ms**, single audible stream, orthogonal mute channel respected. ## Test plan - [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`) - [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow) - [x] `bun run build` — all packages green - [x] Fresh preview + browser repro on `factory-series-c-video`: - [x] Runtime ownership: single audio stream, no drift - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset - [x] No `volumechange` thrash in either ownership mode - [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above) |
||
|
Miguel Ángel
|
d291358cbc | chore: release v0.4.2 | ||
|
Miguel Ángel
|
96376c9be0 | chore: release v0.4.1 | ||
|
James
|
60780774bb | chore: release v0.4.0 | ||
|
Miguel Ángel
|
f1a37400b4 |
fix(runtime): silent-first-play + loading overlay for preview (#293)
## Summary Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit. ### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`) `syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window): - `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click." - The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged. `HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`. ### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`) The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s. ### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`) Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration. Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR. ### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`) Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync. ## Verification - 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset. - Full monorepo build green (`bun run build`), typecheck clean, lint/format clean. - End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass: | Scenario | Metric | Result | | --- | --- | --- | | Normal first play | Audio plays from click, smooth progression | ✅ | | Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip | ✅ | | Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) | ✅ | | Scrub mid-playback | Lands exactly at target frame | ✅ | ## Files changed - `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve. - `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset. - `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes. - `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check. ## Follow-ups (deferred) - Tight-threshold short-window drift correction for lip-synced dialogue. - A perf-regression test that fails on `waiting`\-event resurgence. ## Test plan - [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words. - [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events). - [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers. - [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame. |
||
|
Ular KimsanovandClaude Opus 4.6
|
87f4c77e2f |
feat: website capture pipeline + 7-step video production skill (#284)
* feat(cli): add website capture with AI-powered DESIGN.md generation Adds `hyperframes capture <url>` command that extracts a complete design system from any website, producing AI-agent-ready output: - Full-page screenshot (lazy-load aware, nav at top) - AI-generated DESIGN.md via Claude API (colors, typography, elevation, components, do's/don'ts) with programmatic asset catalog (136+ assets with HTML context annotations like img[src], css url(), link[rel=preload]) - CSS-purged compositions (87% size reduction via PurgeCSS) - HTML-prettified compositions (one-tag-per-line for AI readability) - CLAUDE.md + .cursorrules auto-generated for AI agent instructions - Asset deduplication (srcset variants) and tracking pixel filtering * feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement - switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback - playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs) - replica refinement loop: generate, screenshot, compare, fix - extract inline svgs (50 max, 10kb each) to assets/svgs/ - extract visible text in dom order for content accuracy - detect js libraries (gsap, three.js, scrolltrigger) via globals - improved asset catalog grouping and naming - reverse-engineered aura system prompt documentation - comprehensive session handoff doc Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update session handoff with slack research findings - key finding: team already wants DESIGN.md integration (James, Bin, Vance) - skills quality matters enormously - must invoke /hyperframes-compose - eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide) - templates at templates/ need study before finalizing skill - session handoff updated with critical next steps * refactor(cli): simplify capture pipeline, remove replica generator * feat(capture): add Lottie detection and WebGL shader extraction Captures Lottie animations via network interception and WebGL shader source via gl.shaderSource hooking during site crawl. Updates website-to-hyperframes skill with asset planning guidance, Lottie/shader reading instructions, and stronger creative direction for scene planning. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(capture): clean pipeline + shader-first creative workflow Capture pipeline: - Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps) - Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md) - Remove dead canvas-to-image code in htmlExtractor (post canvas removal) - Parallelize image downloads (batches of 5 via Promise.allSettled) - Fix pre-existing TS error (match[1] guard in font downloader) - Default capture output to captures/<hostname> Skill creative overhaul: - Add shader transition selection to creative director step (Step 4) - Add shader wiring instructions to engineer step (Step 5) - Replace 4-line energy modifiers with visual vocabulary table - Strip rigid scene-by-scene templates from video-recipes.md - Strip example fill data from scene plan tables - Add "read transition refs before planning" instruction - Add creative ambition language ("how the hell did they make this") Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add skill architecture redesign spec Comprehensive redesign of website-to-hyperframes skill and capture pipeline based on code review findings and Claude Code architecture research. Key changes: remove AI auto-generation, restructure skill into phases, embed shader boilerplate in scaffold, fix color format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add implementation plan for skill architecture redesign 13-task plan covering: capture pipeline cleanup (remove AI generation, fix colors to HEX, add asset descriptions, shader-ready scaffold), skill restructuring (4 phases with artifact gates), and compose skill Visual Identity Gate upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(capture): remove AI auto-generation and SDK dependencies * fix(capture): convert extracted colors to HEX format * refactor(capture): remove AI key path, add asset descriptions generator * refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions * feat(capture): pre-wire shader transitions in index.html scaffold * chore: remove duplicate visual-styles.md (canonical is in hyperframes/) * refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator * feat(skill): add Phase 1 understand reference * feat(skill): add Phase 2 design reference with full DESIGN.md schema * feat(skill): add Phase 3 creative direction reference * feat(skill): add Phase 4 build reference with inline shader example * feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md * docs: update CLAUDE.md skill references for phase-based workflow * fix: address code review findings - Remove orphaned `false` argument in generateAgentPrompt call (critical: was shifting hasLottie, hasShaders, catalogedAssets parameters) - Add HSL color handling in rgbToHex via temp element resolution - Remove build artifact commit section from phase-4-build.md - Fix __GSAP_TIMELINE reference to __timelines Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(capture): regex double-escape + simplify scaffold + fix asset descriptions - Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\() so browser receives valid regex patterns via page.evaluate() - Simplify index.html scaffold: scene slots + audio + timeline + comment pointing to shader-setup.md reference (no broken inline shader boilerplate) - Fix asset descriptions: use CatalogedAsset.contexts/notes instead of nonexistent htmlContext field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: code review — 16 bugs, 7-step skill rewrite, cleanup Code fixes: - snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero for --frames 1, port bind error handling, rAF-based render settle - index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini batch/rate-limit comments, fix video preview viewport y-coordinate - tokenExtractor.ts: remove dead seen[si] dedup code - gsap.ts: index ALL classes for inline-style transform conflict detection Skill architecture rewrite (4-phase → 7-step): - Replace phase-1 through phase-4 with step-1 through step-7 - Add techniques.md (10 visual techniques with code patterns) - Fix /hyperframes-compose → /hyperframes (skill doesn't exist) - Fix captures/arc-browser reference → shader-setup.md (file doesn't exist) - Fix step-7 hardcoded captures/stripe path - Document Gemini API free/paid rate limits in step-1 Cleanup: - CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step - .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast) - Delete old phase-*.md, animation-recreation.md, tts-integration.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove dev artifacts, research docs, wrong lockfiles Remove files that shouldn't ship in this PR: - docs/research/ (aura analysis, prompt catalogs) - docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes) - docs/superpowers/ planning and spec docs - pnpm-lock.yaml at root and cli (repo uses bun, not pnpm) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention Main PR #283 removed the full skills table from CLAUDE.md and moved it to AGENTS.md. Align with that decision: use main's slim dev-focused format, fix pnpm→bun references, add one-line /website-to-hyperframes pointer. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add capture command to help groups The capture command was registered in cli.ts but missing from the help groups, so it wouldn't appear in `hyperframes --help`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format skill reference files (oxfmt) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: regenerate bun.lock after rebase The lockfile was stale after rebasing onto main — bun install --frozen-lockfile failed in CI because new dependencies (google/genai, patchright, purgecss) weren't reflected in the lockfile. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review comments + improve capture quality Review fixes (16 comments from jrusso1020 + vanceingalls): - screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots - screenshotCapture: dynamic screenshot count based on page height (30% overlap) - snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard - htmlExtractor: fix code injection via parameterized evaluate - index.ts: video preview re-measures position after scroll, .env file loading - capture.ts: BLOCKED.md on timeout failures - gsap.ts: 5 inline-style lint tests added (all pass) - Remove Playwright, patchright deps; @google/genai to optionalDependencies - Gitignore: generic patterns instead of 20 hardcoded directories - Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance) Capture quality improvements (tested on 10+ websites): - Color extraction: canvas-based oklch/lab resolver, pixel sampling via elementFromPoint, broad sweep for accent colors, gradient/shadow extraction - Section detection: broadened selectors for div-based layouts, height cap to skip page-level wrappers, parent bg walkup for dark sites - Font downloads: cap 6 per family / 30 total (Cal.com: 306→30) - CTA detection: text pattern matching + nav context filtering - Heading text: innerText with whitespace normalization - Gemini captioning: maxOutputTokens 100→300, .env auto-loading - .env.example updated with GEMINI_API_KEY docs - TTS ranking: Kokoro first with Python 3.10+ note * fix: address PR review comments + improve capture quality Review round 2 fixes (jrusso1020 + vanceingalls): - verify/index.ts: add path traversal guard (relative + isAbsolute) - verify/index.ts: fix sections[i] undefined typecheck error (CI green) - index.ts: escape Lottie JSON with \u003c to prevent </script> breakout - step-4-storyboard: fix technique count contradiction (2-3 per beat, not across whole video) - step-6-build: perspective tilt uses gsap.set() instead of CSS transform (avoids GSAP overwrite conflict) - step-1-capture: reorder — command first, Gemini note after (zero-config is the default path, API key is optional enhancement) - step-7-validate: add tsx fallback for snapshot command - step-3-script: vary hook patterns, don't default to number every time - assetDownloader: exempt SVGs from 10KB minimum filter (company logos like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets) Note: adm-zip was NOT removed (reviewer #3) — it's still in packages/cli/package.json:30. The root package.json had patchright and purgecss removed, not adm-zip. Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms zero references in the entire codebase. The @anthropic-ai/sdk dependency was removed earlier in this branch. * refactor(capture): split index.ts (1175 to 566 lines) into modules Mechanical extraction, zero logic changes. New files: - mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots - contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions - scaffolding.ts (135 lines): .env loading, project scaffold generation Also fixes false-positive BLOCKED.md with structural Cloudflare detection. Tested on 20 websites, pre/post output identical. * chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss) The --split feature auto-generates compositions from captured HTML — a different approach from the /website-to-hyperframes skill workflow where agents build compositions from scratch using the storyboard. No skill file, no step reference, and no test session ever used --split. Removes 923 lines of unused code + purgecss dependency. Backed up to ~/Desktop/capture-split-backup/ for reference. * fix(security): add ssrf protection, lottie injection fix, oom guard - assetDownloader: add isPrivateUrl() guard blocking private IP ranges (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata endpoints, localhost, and non-HTTP schemes - mediaCapture: fix Lottie JSON injection by loading shell HTML first then passing animation data via parameterized page.evaluate() - index.ts: check Content-Length header before response.buffer() in Lottie network interception to avoid OOM on multi-GB responses * fix(capture): security fixes, timeout, sub-agent dispatch instructions Security (from miguel-heygen review): - assetDownloader: export isPrivateUrl() SSRF guard - htmlExtractor: add isPrivateUrl check before CSS fetch - mediaCapture: add isPrivateUrl check before Lottie fetch - mediaCapture: fix previewPage leak (try/finally) - mediaCapture: skip Lottie files > 2MB for preview (CDP limit) - contentExtractor: skip images > 4MB for Gemini captioning - index.ts: check Content-Length before response.buffer() (OOM guard) - snapshot.ts: register error handler before server.listen() Capture improvements: - Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare) - step-6-build: sub-agent dispatch template with explicit rules: pass file PATHS not contents, use local fonts not Google Fonts, verify ../assets/ references after each beat * fix(capture): catalog before DOM mutation, networkidle2, faster Gemini Critical: asset cataloger now runs BEFORE extractHtml which converts img src to data URLs. Framer sites like heykuba.com went from 2 to 78 images. - networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets) - Lazy-load wait: scroll to bottom, wait for img.complete - CSS background-image cataloging for Framer/Webflow - SVG naming: checks class, id, parent, inner text (not just aria-label) - Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img) - maxOutputTokens 300->500, descriptions sorted captioned-first - Remove tsx fallback from step-1 (reviewer nit, published CLI has it) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
James
|
2718de8776 | chore: release v0.3.2 | ||
|
James
|
0a3ca498ea | chore: release v0.3.1 | ||
|
James Russo
|
08fb1de61f |
feat(cli): add command + hyperframes.json (#256)
## What PR 5/17 of the catalog system rollout. Adds the `hyperframes add` verb for installing blocks and components from the registry into an existing project, plus the `hyperframes.json` project config that tells `add` which registry to use and where to drop files. Stacks on #255. - **`packages/cli/src/commands/add.ts`** — new `hyperframes add <name>` command. Resolves an item, validates target paths, installs files in parallel, builds an include snippet, copies it to the clipboard. Exposes a testable `runAdd(opts)` function; the citty default wraps it with console output + exit handling - **`packages/cli/src/utils/projectConfig.ts`** — read/write/normalize `hyperframes.json`. Tolerant to missing and partial configs - **`packages/cli/src/utils/clipboard.ts`** — minimal cross-platform clipboard (pbcopy / clip.exe / wl-copy / xclip / xsel). Zero deps. Gracefully no-ops in headless environments - **`packages/cli/src/commands/init.ts`** — write `hyperframes.json` during scaffold if not already present - **`packages/cli/src/cli.ts`** + **`help.ts`** — register `add` under Getting Started (directly below `init`) Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a). ## UX ```bash # Scaffold a project (now writes hyperframes.json too) npx hyperframes init my-video --example blank cd my-video # Add a block — files land, snippet copied to clipboard npx hyperframes add claude-code-window # ✓ Added claude-code-window (hyperframes:block) # compositions/claude-code-window.html # # Include snippet: # <iframe src="compositions/claude-code-window.html" data-start="0" data-duration="6"></iframe> # # Copied to clipboard — paste into your host composition. # Add a component effect npx hyperframes add shader-wipe # Headless / CI — no clipboard, JSON output for tooling npx hyperframes add shader-wipe --no-clipboard --json ``` Running `hyperframes add warm-grain` (an example) errors clearly pointing to `init --example`. ## Docs (bundled in this PR per the tracker principle) - `docs/packages/cli.mdx` — new `add` subsection under Commands (flags, examples, trigger rules) + new `hyperframes.json` section describing the config file shape ## Tests - **`packages/cli/src/commands/add.test.ts`** — 11 tests: - `remapTarget` / `buildSnippet` pure helpers (5 tests) - `runAdd` integration against a mocked `fetch` registry: block install lands files + returns snippet, component install respects `paths.components` remap, example-typed names throw `AddError` with code `example-type`, unknown names throw `AddError` with code `unknown-item` (4 tests plus 2 covering block default path and non-default path preservation) - **`packages/cli/src/utils/projectConfig.test.ts`** — 9 tests: - Write/read round-trip, partial-config normalization, corrupt-file handling, absent-file fallback to defaults, custom paths preserved - **CLI suite:** 92 passed (was 72 on #255, **+20**). Same 4 pre-existing failures unchanged ## Scope decisions - **`init.ts` full port to new resolver deferred.** The original plan bundled a removal of the `packages/cli/src/templates/` compat shim. That's ~300 more lines and isn't required for `add` to work. The compat shim from #254 still functions; a separate cleanup PR handles it - **No ajv runtime schema validation.** Manifests are trusted as schema-valid. Full validation lands when third-party registries arrive (PR 14/15). Path safety is still enforced by the installer's `assertSafeTarget` guard - **Default project paths stay under `compositions/`.** Blocks → `compositions/<name>.html`; components → `compositions/components/<name>/<file>`. Users override via `hyperframes.json#paths` ## Breaking / migration **None.** Pure additive — new command, new file types, no existing commands or flags change. `init.ts` now writes `hyperframes.json` but that's a new additional file, not a modification of existing output. ## Stacks on #255 — base branch. When #255 merges, this rebases onto `main`. ## Next in stack PR 6 — `feat(registry): seed block — claude-code-window`. First real registry item. Exercises the full `hyperframes add <name>` flow end-to-end against a committed item on `main`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James Russo
|
969474e843 |
feat(cli): registry resolver + installer (#254)
## What PR 3/17 of the catalog system rollout. Introduces the registry resolver/installer abstraction. No UX change — `init --template` still works identically. Stacks on #253. **New module: `packages/cli/src/registry/`** - `remote.ts` — fetches manifests (`registry.json`, `registry-item.json`) and item files from a GitHub-hosted registry. 24h cache on manifests; item files stream straight to `destDir` - `resolver.ts` — `listRegistryItems`, `loadAllItems` (parallel fetch for picker UX), `resolveItem` (single-item fetch with `Available:` error) - `installer.ts` — `assertSafeTarget` (runtime path-traversal guard) + `installItem` (parallel file download with up-front validation; all-or-nothing semantics) - `index.ts` — barrel **Registry content:** - `registry/registry.json` — top-level manifest in PR 1's `RegistryManifest` shape. 8 examples - `registry/examples/<id>/registry-item.json` — per-item manifest for each existing example, generated from legacy `templates.json` + HTML data-attribute probing - `registry/examples/templates.json` — **deleted**, replaced by the above **Compat layer:** - `packages/cli/src/templates/{remote,generators}.ts` — thin shims that delegate to `../registry/`, keeping `init.ts`'s existing imports stable. `init.ts` doesn't move to the new API until PR 5 where it's part of a larger UX pass **Tooling:** - `scripts/generate-registry-items.ts` — idempotent one-off generator for this PR, kept in-repo for future example additions (`--only <name>` flag) Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a). Tracker entry in local `hyperframes-catalog-plan.md`. ## Why Every future PR (`hyperframes add`, seed blocks, seed components, custom registries) otherwise has to keep piling onto the ad-hoc fetch + `cpSync` pattern in the old `fetchRemoteTemplate`. The new module is the single place that understands the registry wire format and file layout. **This is also where PR 1's schema comes alive.** ## How ### Scope-trimmed from the plan - **No transitive dependency resolution yet.** Examples have no deps today. `resolveItem` doesn't walk `registryDependencies`; PR 5 adds that when blocks/components need it. - **No ajv schema validation yet.** TS types + runtime path-traversal guard are the only safety nets. Full JSON-Schema validation lands when the registry starts accepting third-party content (PR 14 / custom registries). - **init.ts refactor deferred to PR 5.** Compat shims keep this PR small and reviewable. PR 5 rewrites init alongside adding the `add` command. ### Safety - `assertSafeTarget` rejects absolute paths, `..` segments, Windows drive letters, and any target that `path.resolve` shows to escape `destDir`. Mirrors the PR 1 schema `pattern`/`not.anyOf` on `target`, but runs at install-time so a registry that bypasses schema validation still can't write outside the project - Up-front validation in `installItem` means a malformed item fails **before** any file is written. Atomic-ish semantics: all files land or none do ### Caching - 24h manifest cache lives at `~/.hyperframes/cache/` per existing convention, but now keyed by `<baseUrl>__<kind>__<name>.json` so PR 14 custom registries can coexist ## Test plan - [x] `bun run test` in `packages/cli`: **70 passed** (was 57 on #253, +13). Same 4 pre-existing failures (SRT/VTT whisper normalizer + `lintProject` clean-project test) — identical to main. No regressions - [x] **Resolver unit tests (8):** filter by type, parallel load with fail-safe, resolve-by-name with `Available:` error message, unreachable-registry handling - [x] **Installer unit tests (5):** accepts simple relative paths, rejects `..` segments, rejects Unix absolute paths, rejects Windows drive letters, permits `.` and dotfile-like names - [x] **Smoke test**: `hyperframes init /tmp/x --template blank` (bundled code path, unchanged) works end-to-end - [x] `bunx oxfmt --check` + `bunx oxlint`: clean - [x] Pre-commit typecheck (core + studio): clean. CLI typecheck has 2 pre-existing errors (`render.ts`, `studioServer.ts` — unrelated `"mov"` format issue on main) - [ ] **Smoke test remote fetch (`--template warm-grain`)** — verifiable only post-merge; registry paths live on `main` after this PR lands ## Breaking / migration **No end-user-visible UX change.** `init --template <name>` still works the same way. Internally, `templates.json` is gone and the CLI now reads `registry.json` + `registry-item.json` per example. Installed CLIs on old versions (`hyperframes@0.1.0`–`0.3.0`) already broke at PR 2 merge (see #253 rollout note). The next CLI release after this lands (`0.3.1`+) is the full fix. ## Commits 1. `generate-registry-items.ts` + generated manifests + deleted `templates.json` 2. Resolver + installer + compat shims 3. Unit tests (All squashed into one commit on this branch; see `git log feat/registry-resolver ^refactor/registry-examples-dir`.) ## Stacks on #253 — base branch. When #253 merges, this rebases onto `main`. ## Next in stack PR 4 — `feat(cli)!: rename --template to --example`. Single clean cut, no alias. Tiny PR (~150 lines) that mostly updates `init.ts`'s argument schema, help text, and docs. Depends on this PR so the new flag name can be applied against the refactored code path. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James RussoandClaude Opus 4.6
|
eb338ae859 |
feat(core): add registry schema + TS types (#252)
* feat(core): add registry schema + TS types
PR 1/17 of the catalog system rollout. Foundation for a shadcn-style
registry with three item tiers: examples (full projects), blocks
(sub-compositions), components (effect snippets).
## What
- TS types: RegistryItem (discriminated union of ExampleItem/BlockItem/
ComponentItem), RegistryManifest, FileTarget, ItemType, FileType
- JSON Schemas: schemas/registry.json, schemas/registry-item.json
- Compile-time exhaustiveness asserts on ITEM_TYPES/FILE_TYPES so adding
to the TS union without updating the constant stops compiling
- Drift-guard test: schema enums must equal ITEM_TYPES/FILE_TYPES by
set-equality; exactly 2 distinct type enums in registry-item.json
- Public API via new ./registry export path plus re-exports from root;
schemas exposed via ./schemas/registry.json export for external tooling
## Why
- Every downstream PR (resolver, installer, hyperframes add, docs
codegen, CI previews, skill, catalog command) builds on these types
- Getting the shape right now avoids painful migrations later
## How
- Discriminated union enforces that components do not have dimensions
or duration and examples/blocks must have them (schema mirrors via
if/then/else on the type discriminant)
- target path pattern rejects .. segments, Unix absolute paths, and
Windows drive letters (defense-in-depth; CLI validates at runtime in
PR 3)
- name pattern requires alphanumeric start and end (no trailing hyphens)
- Optional metadata: version, author, license, deprecated, minCliVersion
- additionalProperties: false on nested objects (catches typos on
critical fields) but relaxed on top-level RegistryItem (allows
third-party custom metadata in PR 15 custom registries)
## Test plan
- [x] Unit tests: 11 new tests covering type guards, discriminant
narrowing, schema/TS drift guards, schema \$id sanity, optional
metadata acceptance, and compile-time checks (via @ts-expect-error)
- [x] bun run test in packages/core: 445 passed (was 434 on main,
+11 from this PR)
- [x] bunx oxfmt and bunx oxlint: clean
- [x] bun run typecheck: clean
- [ ] Manual testing: N/A (types + schemas only)
- [ ] Documentation updated: per-item doc pages land in PR 9 (codegen
from these manifests); guide updates in PR 10+
## Breaking / migration
None. Pure additive — new module, new export paths, no existing
surface touched.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(core): remove version field + hyperframes:demo file type
Address review feedback from Miguel:
- Remove `version` from RegistryItemBase + schema. Per shadcn model,
the registry is versioned by git tags, not per-item. The adversarial
review added it; the original design doc was correct.
- Remove `hyperframes:demo` from FileType union + FILE_TYPES constant
+ schema. Demo files exist on disk for the CI preview pipeline but
are NOT installed to user projects and should not appear in
registry-item.json files[]. Neither shadcn nor Remotion has a
dedicated demo file type — demos are just compositions.
- Add `required: ["type"]` to the if-condition in the schema's
allOf discriminant (Miguel's nit — makes the condition self-
contained)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): add shader-transitions to Dockerfile.test
PR #251 added packages/shader-transitions/ to the workspace but didn't
update Dockerfile.test to COPY its package.json. This caused
`bun install --frozen-lockfile` to fail in the regression Docker build:
bun saw a lockfile referencing @hyperframes/shader-transitions but the
package.json wasn't present in the container, so it wanted to remove
the entry — triggering "lockfile had changes."
Verified: Docker build passes with `--no-cache` after this fix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
58ddb11bc5 |
chore: release v0.3.0 (#249)
## Summary
Coordinated minor bump across all published packages. No source changes in this PR itself; it is the version stamp for everything that landed on main since v0.2.5.
## Version bumps
| Package | from | to |
|---|---|---|
| `@hyperframes/cli` | 0.2.5 | **0.3.0** |
| `@hyperframes/core` | 0.2.5 | **0.3.0** |
| `@hyperframes/engine` | 0.2.5 | **0.3.0** |
| `@hyperframes/player` | 0.2.7 | **0.3.0** |
| `@hyperframes/producer` | 0.2.5 | **0.3.0** |
| `@hyperframes/studio` | 0.2.9 | **0.3.0** |
Between 0.2.5 and this release, `player` and `studio` received several patch versions on npm as we iterated on the bundler, entry point, and SSR issues. 0.3.0 collapses that into a single coordinated minor so the ecosystem is aligned again.
## What is in v0.3.0
### `@hyperframes/player`
**Restored package entry points to the compiled `dist/` output.** 0.2.5 shipped with `"main": "./src/hyperframes-player.ts"` but the published tarball only included `dist/` via the `"files"` field. Every consumer trying to import the package failed with `Module not found: Can't resolve '@hyperframes/player'`. Entry points now point at the built JS/`.d.ts` files inside `dist/`.
**DOM-based root timeline resolution in the ready probe.** In a bundled preview, `window.__timelines` contains the master composition alongside its sub-compositions, for example:
```js
{
main: GSAPTimeline(14s),
intro: GSAPTimeline(1.5s),
'scene2-4-canvas': GSAPTimeline(12.6s),
'scene5-logo-outro': GSAPTimeline(3.2s),
}
```
The probe used to select the adapter with `keys[keys.length - 1]`. Object key ordering meant the last-registered sub-composition would win, so the `ready` event reported a sub-composition's duration (e.g. 3.2s) instead of the master's 14s. The probe now looks up the root composition id from the outermost `[data-composition-id]` element in the iframe DOM and uses its key. Falls back to the last key when no element is present, so standalone sub-composition previews keep working.
### `@hyperframes/studio`
**`useTimelinePlayer.getAdapter()` uses the same DOM-based root id lookup** as the player. Previously play, pause, seek, and duration readout were all driven by whichever sub-composition happened to register its timeline last.
**`Player.tsx` loads `@hyperframes/player` lazily.** The component used to call `import "@hyperframes/player"` at module scope, which runs the package's `customElements.define(...)` side effect during module evaluation. `HTMLElement` does not exist in the Node runtime, so any consumer page that transitively imported the studio during server rendering threw:
```
ReferenceError: HTMLElement is not defined
at module evaluation (@hyperframes/studio/src/player/components/Player.tsx)
```
The import now runs inside the mount effect via `import(...)` so it only evaluates in the browser. Added a cancellation flag and deferred cleanup so a fast unmount before the dynamic import resolves does not leak listeners or DOM nodes.
**Captions module imports stripped of `.js` extensions.** Files under `src/captions/` imported siblings as `./types.js` and `./parser.js`. That is legal ESM TypeScript but Turbopack and several other bundlers refuse to resolve those specifiers against `.ts` files inside `node_modules`, breaking any consumer build that transitively pulled in the captions module. Captions now uses extensionless imports, matching the rest of the studio codebase.
### `@hyperframes/core`, `@hyperframes/cli`, `@hyperframes/engine`, `@hyperframes/producer`
Version bump only, no source changes since 0.2.5. Kept on the same version so the ecosystem is easier to reason about.
## Impact for consumers
If you use `@hyperframes/studio` in a Next.js app:
- The play button in a bundled preview reports the correct composition duration and drives the master timeline.
- The session page no longer 500s in dev mode when the studio barrel is imported (the SSR fix).
- Turbopack builds that transitively load the captions module no longer fail on `Cannot resolve './types.js'`.
If you use `@hyperframes/player` directly:
- Consumer bundlers can resolve the package again (dist entry points restored).
- The `ready` event duration reports the master, not a sub-composition.
## After merge
Publish each package to npm with `pnpm publish` (workspace deps auto-resolve).
|
||
|
Miguel Ángel
|
1dd898786c | chore: release v0.2.5 (#246) | ||
|
James RussoandClaude Opus 4.6
|
18de86e4bd |
fix(player): handle Infinity duration; add lint rules for data-duration and Math.ceil overshoot (#243)
* fix(player): handle Infinity duration from runtime gracefully When compositions have repeating animations without data-duration, the runtime sends durationInFrames: Infinity. The player now ignores non-finite duration values instead of displaying "Infinity:NaN" in the controls. formatTime also returns "0:00" for non-finite inputs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(lint): add data-duration and Math.ceil overshoot rules - Add root_composition_missing_data_duration warning when the root composition element is missing data-duration, which causes the runtime to infer Infinity for loop-inflated timelines. - Add gsap_repeat_ceil_overshoot warning that catches repeat: Math.ceil(d/c)-1 patterns which overshoot the intended duration. Recommends Math.floor instead. - Fix gsap_infinite_repeat fixHint to suggest Math.floor (not Math.ceil). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(player): wait for injected runtime before declaring ready When the player auto-injects the runtime script (because the composition has GSAP timelines but no runtime), it would immediately declare ready on the next probe cycle — before the runtime script finished loading from CDN. This caused play() to send a postMessage that nobody received, making autoplay silently fail. Now the probe waits for the runtime bridge (__hf or __player) to appear before proceeding to the ready state. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
794b02153d |
fix(lint): upgrade bare composition HTML to error (#242)
## Summary - Upgrades `root_composition_missing_html_wrapper` from **warning** to **error** — a bare `<div data-composition-id>` as `index.html` without `<!DOCTYPE html>/<html>/<body>` causes browsers to quirks-mode, the preview server to fail, and the bundler to silently skip runtime injection - Improves the error message to explain _why_ this is bad, and includes a snippet of the offending root element - Skips `<template>`\-wrapped compositions (already caught by the separate `standalone_composition_wrapped_in_template` rule) - Adds 8 tests covering the exact screenshot scenario, proper HTML, sub-compositions, plain HTML, and template wrappers ## Test plan - [x] All 441 existing tests pass (`vitest run`) - [x] 8 new tests for `root_composition_missing_html_wrapper` and `standalone_composition_wrapped_in_template` - [x] TypeScript build clean (`tsc --noEmit`) - [x] oxlint + oxfmt pass - [x] Run `npx hyperframes lint` on a bare composition `index.html` and verify it now reports an error |
||
|
Miguel Ángel
|
078ed7d5cd |
fix(deps): patch security vulnerabilities in pretext and vite (#237)
## Summary - Bump `@chenglou/pretext` ^0.0.3 → ^0.0.5 in `packages/core` — fixes **high-severity** algorithmic complexity DoS ([Dependabot #3](https://github.com/heygen-com/hyperframes/security/dependabot/3)) - Bump `vite` ^5.0.0 → ^6.4.2 in `packages/studio` — fixes **medium-severity** path traversal in optimized deps `.map` handling ([Dependabot #2](https://github.com/heygen-com/hyperframes/security/dependabot/2)) ## Test plan - [x] `pnpm --filter @hyperframes/core build` — passes - [x] `pnpm --filter @hyperframes/studio build` — passes (vite 6.4.2, 4631 modules, 3.85s) - [x] `@vitejs/plugin-react@^4.0.0` supports vite 6 (`peerDependencies: vite ^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0`) |
||
|
Miguel ÁngelandClaude Opus 4.6
|
dce2c6ee14 |
chore: release v0.2.4
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
ceb54811c6 |
fix(runtime): preload media on init to prevent broken first-play audio (#234)
## Summary - Audio (and video) sounds broken/choppy on first play in the studio preview, but works fine on second play - Root cause: `<audio>` elements default to `preload="metadata"`, which only fetches enough data to determine duration — not enough for smooth playback. When `el.play()` fires, the browser hasn't buffered the audio data yet - The runtime now eagerly sets `preload="auto"` and calls `load()` during init, ensuring media is fully buffered before the user clicks play - `syncRuntimeMedia` now defers `play()` on unbuffered media by registering a `canplay` listener, instead of silently swallowing the failure ## Testing Verified with agent-browser against a 26s narration composition (soulscape-film): ``` # After fix — audio element state at init: preload: "auto" readyState: 4 (HAVE_ENOUGH_DATA) buffered: 26.07s (entire file) duration: 26.07s ``` Audio is fully buffered before any play attempt, so first-play works identically to subsequent plays. ## Files changed - `packages/core/src/runtime/init.ts` — set `preload="auto"` + `load()` in `bindMediaMetadataListeners` - `packages/core/src/runtime/media.ts` — defer `play()` on unbuffered media via `canplay` listener - `packages/core/src/runtime/media.test.ts` — updated test + added unbuffered media test case |
||
|
Vance IngallsandClaude Opus 4.6
|
fc973ee2e8 |
feat(lint): add rules for missing data-start, template wrapper, and DOCTYPE
Three new lint rules that catch structural issues causing compositions to fail silently in preview: - root_composition_missing_data_start: Root composition needs data-start="0" for the runtime to begin playback - standalone_composition_wrapped_in_template: index.html should not be wrapped in <template> (only sub-compositions use that) - root_composition_missing_html_wrapper: index.html needs <!DOCTYPE html> and <html> wrapper for the bundler Also adds rawSource to LintContext so rules can inspect pre-template-stripped HTML, and isSubComposition to linter options so rules can distinguish root from sub-composition files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
James
|
b33bbfa0f9 | chore: release v0.2.3 | ||
|
Miguel Ángel
|
1c61a0b25a | chore: release v0.2.3-alpha.2 | ||
|
Miguel Ángel
|
43e9252065 |
feat: add MOV (ProRes 4444) as transparent video output format (#224)
## Summary - Adds `--format mov` to the render CLI for ProRes 4444 transparent video output - ProRes 4444 with alpha is the industry standard for transparent video overlays, supported by CapCut, Final Cut, Premiere, DaVinci, and After Effects - WebM VP9 alpha technically works but is ignored by all major video editors — only browsers decode it - Adds MOV to the studio export dropdown alongside MP4 and WebM ## Transparency format comparison | Format | Codec | Alpha | Video editors | Browsers | File size | | --- | --- | --- | --- | --- | --- | | **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No (won't play in browser) | Large (~5-40 MB) | | **WebM** | VP9 | Yes | None (shows black) | Chrome, Firefox | Small (~200 KB) | | **MP4** | H.264 | No | All | All | Small | > **Note:** ProRes MOV files do not play in Chromium browsers — they are an intermediate/editing format, not a delivery format. Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify transparency works correctly. ## Changes - **CLI**: Add `mov` to `--format` validation, examples, and output path logic - **Engine**: `getEncoderPreset()` returns ProRes 4444 (`yuva444p10le`) for `mov` format; handle `.mov` in `applyFaststart` and `muxVideoWithAudio`; add `pix_fmt` to streaming encoder ProRes path - **Producer**: Treat `mov` like `webm` for alpha capture (PNG frames, screenshot mode, `forceScreenshot`) - **Studio**: Add MOV option to export format dropdown and render queue hook - **Core**: Add `mov` to studio API types, render route, and mime helpers - **Tests**: Add encoder preset tests for mov format (42 total, all passing) ## Usage ```bash hyperframes render --format mov --output overlay.mov ``` ## Test plan - [x] `pnpm build` passes - [x] `pnpm --filter @hyperframes/engine test` — 42 tests pass (2 new for MOV) - [x] `oxlint` and `oxfmt` clean on all 12 changed files - [x] End-to-end local render produces ProRes 4444 (`yuva444p12le`) with working alpha - [x] Docker render with `--format mov` — ProRes 4444 confirmed via ffprobe - [x] Studio dropdown shows MOV option in built JS - [x] Transparency verified with [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) |
||
|
Miguel Ángel
|
f56b4c8620 |
fix(core): load head styles/scripts from non-template sub-compositions (#219)
## Summary Fixes a bug where non-template sub-compositions (full HTML documents loaded via `data-composition-src`) lost all `<head>` styles and scripts. This affected **three code paths**: 1. **Runtime** (`compositionLoader.ts`) — browser preview via iframe fetch 2. **Bundler** (`htmlBundler.ts`) — studio preview HTML bundling (**this was causing the black preview**) 3. Producer fix is in PR #220 ## What it fixes **Eval prompt #25** (iris-wipe) renders entirely black in both the studio preview and rendered video because scene backgrounds (`#EF4444` red, `#3B82F6` blue), positioning, and the GSAP CDN script were all in `<head>` and silently dropped. ### Verified Rebuilt core, started studio preview, fetched the bundled HTML from `/api/projects/iris-wipe/preview` — confirmed `#scene1 { background: #EF4444 }` and `.scene { position: absolute }` are now present in the output. ## Root cause All three code paths did the same thing: ```js const contentHtml = template ? template.innerHTML : bodyEl.innerHTML; // ^ <head> content is already lost here ``` ## Test plan - [x] All 429 core tests pass - [x] Studio preview endpoint returns correct bundled HTML with head styles included - [x] `pnpm --filter @hyperframes/core build` succeeds 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Miguel Ángel
|
0d33238381 |
fix(core): add lint rule for infinite GSAP repeat (#218)
## Summary Adds a new lint rule `gsap_infinite_repeat` that flags `repeat: -1` in GSAP timelines as an error. This is a hard enforcement of the skill guardrail added in PR #217. ## What it fixes The deterministic capture engine (`HeadlessExperimental.beginFrame`) seeks to exact frame times on a paused GSAP timeline. When a timeline contains `repeat: -1`, the timeline duration is infinite, which causes the capture engine to produce incorrect/blurry output. **Eval prompt #20** (loading-spinner, scored 2.0/5) used `repeat: -1` on a dots animation cycle, producing "a highly compressed and blurry loading animation lacking visual clarity and professional polish." ## Changes - `packages/core/src/lint/rules/gsap.ts` — new `gsap_infinite_repeat` rule (regex scan for `repeat: -1`) - `packages/core/src/lint/rules/gsap.test.ts` — 2 new tests (detects infinite repeat, allows finite repeat) ## Test plan - [x] `pnpm --filter @hyperframes/core test` — all 429 tests pass - [x] Rule catches `repeat: -1` and reports as error with fix hint - [x] Rule does not flag `repeat: 4` (finite repeats) |
||
|
Miguel Ángel
|
5655dabff6 |
feat: allow clip animation + ship <hyperframes-player> web component (#209)
## Summary Two independent initiatives that improve agent DX and expand HyperFrames' reach. ### Initiative 1: Fix the Clip Animation Footgun - `gsap_animates_clip_element` lint rule now uses smart detection — only errors when GSAP animates `visibility` or `display` on a clip element - All other properties (opacity, transform, x, y, scale, etc.) are allowed silently - This was the #1 agent failure in QA (10/10 agents hit it on v0.2.1) ### Initiative 2: `<hyperframes-player>` Web Component - New `@hyperframes/player` package — zero dependencies, 3.3KB gzipped - Iframe-based web component with Shadow DOM for perfect isolation - Video-like API: `play()`, `pause()`, `seek()`, `currentTime`, `duration`, events - Controls overlay with play/pause, scrubber (mouse + touch), time display, auto-hide - Full docs page at `docs/packages/player.mdx` ## Before / After ### Clip animation lint **Before (10/10 agents hit this):** ``` ✗ gsap_animates_clip_element: GSAP animation targets a clip element. Selector "#title" resolves to element <div id="title" class="clip">. The framework manages clip visibility — animate an inner wrapper instead. Fix: Wrap content in a child <div> and target that with GSAP. ``` **After (only errors on actual conflicts):** ``` # This passes lint — no error: tl.from("#title", { opacity: 0, y: -50, scale: 0.8 }, 0); # This still errors — actual conflict with runtime: tl.to("#title", { visibility: "hidden" }, 3); ✗ gsap_animates_clip_element: GSAP animation sets visibility on a clip element. Fix: Remove the visibility/display tween. Use opacity for fade effects. ``` ### Embeddable player **Before:** No way to embed a composition in a web page. **After:** ```html <script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script> <hyperframes-player src="./composition/index.html" controls></hyperframes-player> ``` ```js const player = document.querySelector('hyperframes-player'); player.play(); player.pause(); player.seek(2.5); player.addEventListener('ready', (e) => console.log('Duration:', e.detail.duration)); ``` ## Test plan - [x] 427 core tests pass (20 GSAP lint tests with smart detection) - [x] 7 player tests pass (formatTime + element registration) - [x] TypeScript compiles cleanly (core + player) - [x] Lint: GSAP animating clip with safe props → 0 errors - [x] Lint: GSAP animating clip with `visibility` → 1 error (correct) - [x] Player builds to 3.3KB gzipped ESM - [x] Lockfile updated for CI - [x] Docs page added at `docs/packages/player.mdx` |
||
|
Miguel Ángel
|
baa3d813be |
fix: address QA report P0-P2 issues for 10/10 agent experience (#208)
## Summary
Addresses all 8 issues from the QA report to improve agent and user experience.
### P0 — Must Fix
- **Blank template broken captions**: Removed `compositions/captions.html` and its reference from the blank template. Every agent (10/10) hit 404 errors during render.
- **Inner-wrapper example**: Added a clear structural comment in the blank template showing the correct `class="clip"` + inner wrapper pattern.
- **Sub-composition introspection**: `hyperframes compositions` now reads external HTML files referenced via `data-composition-src` and shows their real duration/element count instead of `0.0s / 0 elements`.
### P1 — Fix Soon
- **Font mapping warnings**: Now lists all mapped fonts, suggests alternatives (use a mapped font, add @font-face, install locally), and links to docs.
- **Browser 404s**: Non-font "Failed to load resource" 404s now prefixed with `[non-blocking]` instead of `[Browser:ERROR]`.
- **Render concurrency**: Default workers increased from `cores/2` (max 4) to `cores*3/4` (max 6). Added `--concurrency` alias.
### P2 — Nice to Have
- **Transform conflict fix suggestion**: `gsap_css_transform_conflict` now suggests exact GSAP property replacements (e.g., `xPercent: -50, yPercent: -50`).
- **Upgrade --yes**: Now actually runs the install instead of just printing the command.
## Before / After
### Font mapping warning
**Before:**
```
[Compiler] No deterministic font mapping for: DM Sans
```
**After:**
```
[Compiler] No deterministic font mapping for: DM Sans
Mapped fonts: arial → inter, courier → jetbrains-mono, ...
To fix, pick one:
1. Use a mapped font name instead (see list above)
2. Add a @font-face block in your HTML with a local or hosted font file
3. Install the font locally on the render machine (Docker: add to Dockerfile)
4. Add an alias to FONT_ALIASES in deterministicFonts.ts (for contributors)
```
### Browser 404s during render
**Before:** `[Browser:ERROR] Failed to load resource: the server responded with a status of 404`
**After:** `[non-blocking] Failed to load resource: the server responded with a status of 404`
### Transform conflict lint
**Before:** `Fix: Remove the transform from CSS and use tl.fromTo...`
**After:** `Fix: Remove transform: translate(-50%, -50%) from CSS and replace with GSAP properties: xPercent: -50, yPercent: -50`
### Compositions command
**Before:**
```
overlay 0.0s 1920×1080 0 elements
```
**After:**
```
overlay 8.0s 1920×1080 2 elements ← compositions/overlay.html
```
## Test plan
- [x] All 422 core tests pass
- [x] TypeScript compiles cleanly (all 4 packages)
- [x] Full monorepo build succeeds
- [x] `hyperframes init --template blank` ships without broken captions reference
|
||
|
Vance Ingalls
|
d8bffd41f9 |
feat(lint,skills): add caption/audio-reactive lint rules and skill guidance (#207)
## What Bumped all package versions to `0.2.2-alpha.4` and added five new lint rules for caption and GSAP animation quality checks. ## Why The new lint rules address common issues in HyperFrames compositions: - Caption overflow clipping when emphasis words are scaled above 1.0x - Text shadow artifacts on caption group containers with semi-transparent children - Mismatch between fitText maxWidth and scaled word dimensions - Imperceptible audio reactivity from single tweens instead of time-sampled animations - Scene layer visibility conflicts when relying only on opacity tweens ## How Added three new caption-specific lint rules in `captions.ts`: - `caption_overflow_clips_scaled_words` - detects `overflow: hidden` on caption containers when scripts scale words above 1.0x - `caption_textshadow_on_group_container` - flags textShadow tweens applied to group containers instead of individual words - `caption_fittext_scale_mismatch` - calculates effective width from fitText maxWidth × max scale factor and warns when it exceeds safe bounds Added two new GSAP lint rules in `gsap.ts`: - `audio_reactive_single_tween_per_group` - identifies audio-reactive captions using peak values instead of time-sampled loops - `scene_layer_missing_visibility_kill` - detects multi-scene compositions missing hard visibility kills after opacity exit tweens Enhanced documentation with new mask reveals guide and updated existing skills with overflow handling, scene management, and audio reactivity best practices. ## Test plan - [x] Lint rules tested against existing composition patterns - [x] Documentation updated with new techniques and constraints - [x] Version bumps applied consistently across all packages |
||
|
Miguel Ángel
|
e2c8ed5d83 |
refactor(core): replace cheerio with linkedom to drop deprecated whatwg-encoding (#187)
## Summary - `cheerio` pulls `encoding-sniffer` → `whatwg-encoding@3.1.1` (deprecated), causing a warning on every `npm install -g hyperframes` - `linkedom` was already bundled into the CLI via tsup `noExternal` and has zero deprecated transitive deps - Rewrote `htmlBundler.ts` and `subComposition.ts` to use standard DOM APIs via `linkedom` - Added a `parseHTMLContent` helper that wraps HTML fragments in a full document structure (required for `linkedom` to populate `document.body`) - Removed `cheerio` from `cli` dependencies and tsup `external` list - Replaced `cheerio` with `linkedom` in `core` `optionalDependencies` ## Test plan - [x] All 411 tests pass (`bun run test` in `packages/core`) - [x] Full monorepo build succeeds (`bun run build`) - [x] TypeScript typecheck passes |
||
|
James
|
06e3da9ad3 | chore: release v0.2.2 | ||
|
Miguel Ángel
|
4bb01fd1b6 |
feat(lint): add rules for non-deterministic code, missing clip class, overlapping tracks, and rAF detection (#193)
## Summary Adds 4 new lint rules found missing during comprehensive E2E testing: - **`non_deterministic_code`** (error) — detects `Math.random()`, `Date.now()`, `new Date()`, `performance.now()`, `crypto.getRandomValues()` in scripts. Confirmed: two renders with Math.random() produced different checksums. - **`timed_element_missing_clip_class`** (warning) — flags elements with `data-start`/`data-duration` but no `class="clip"`. Without it, elements are visible forever instead of only during their time window. - **`overlapping_clips_same_track`** (error) — detects clips on the same `data-track-index` with overlapping time ranges. - **`requestanimationframe_in_composition`** (warning) — warns that rAF-based animations don't sync with frame capture. Discovered when Vivus.js (rAF-based) produced incorrect output. 12 new tests, all passing. **Part 1 of 5** in a stacked PR series fixing E2E test findings. ## Test plan - [x] 12 new tests (3 per rule: positive, negative, edge case) - [x] Full suite: 56 pass in rule tests - [x] `npx tsx scripts/lint-skills.ts` — no issues |
||
|
Vance Ingalls
|
cb0b17062a |
feat(skills): add marker-highlight skill for animated text highlighting (#190)
## Summary - **New skill:** **`marker-highlight`** — integrates [MarkerHighlight.js](https://github.com/Robincodes-Sandbox/marker-highlight) into HyperFrames compositions. Canvas-based animated text highlighting with 5 drawing modes: marker pen, circle, burst, scribble, and sketchout. - **Studio fix:** added missing `captionSync` to useEffect dependency array (oxlint exhaustive-deps) - **Studio fix:** `loadOverrides` now checks `res.ok` before parsing, preventing 404 console noise on projects without captions ## Skill details The skill documents the non-obvious GSAP integration pattern discovered during development: 1. **One highlighter per container** — the library clears ALL `.highlight` divs from the shared parent on init, so multiple instances on sibling marks conflict 2. **`data-color`** **\+** **`data-original-bgcolor`** — prevents the CSS background-color flash that occurs when the library reads and clears the mark's background 3. **Canvas pre-draw + clear + reanimate** — `animate: false` pre-draws statically, canvases are hidden, then cleared and shown with `reanimateMark()` at trigger time for clean animated reveals 4. **`onReverseComplete`** **for rewind** — hides highlight divs when the timeline seeks backward past the trigger point ## Test plan - [ ] `npx hyperframes lint` passes on test-composition - [ ] Studio preview shows marker highlight on "something" at 1s, circle on "love" at 2.2s - [ ] Rewind past trigger points hides highlights - [ ] No 404 console errors for caption-overrides.json on non-caption projects [Screen Recording 2026-04-02 at 1.56.30 AM.mov <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.com/user-attachments/thumbnails/53b03f4e-538e-477a-b738-7a033b99a84e.mov" />](https://app.graphite.com/user-attachments/video/53b03f4e-538e-477a-b738-7a033b99a84e.mov) 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance IngallsandClaude Opus 4.6
|
5e2781b459 |
fix(studio): address caption designer PR feedback (#200)
* fix(studio): address caption designer PR feedback Fixes from review comments on feature/caption-designer (#180): - fix(generator): guard named colors in hexToRgba — "red", "transparent" no longer produce NaN rgba values - fix(sync): log auto-save failures instead of silently swallowing them - fix(sync): check res.ok before parsing caption-overrides response - refactor(components): extract Section, Row, inputCls into shared.tsx to eliminate duplication between CaptionPropertyPanel and CaptionAnimationPanel - fix(store): replace non-deterministic Date.now()+Math.random() ID with counter-based group IDs - fix(store): read selectedGroupId from state param instead of get() to avoid stale reads in batched set() calls - fix(overlay): remove cssScale multiplier from getBoundingClientRect coords — the browser already accounts for CSS transforms - docs(parser): add comment explaining the lazy ]; regex assumption Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): address remaining caption designer feedback Overlay: handle both per-word spans (generator output) and grouped text nodes (existing templates). Wraps text nodes into individual spans on demand so the overlay can target words in any caption format. Property panel: add Typography (font, size, weight, spacing) and Color (color, active, dim, opacity) sections alongside existing Position and Transform controls. Timeline: move caption timeline into a dedicated flex-shrink-0 section below the main timeline tracks instead of inside the scrollable area. Gives it fixed 60px height that's always visible. Caption overrides: classify color tweens by comparing target color to the dim baseline instead of relying on timeline position order. This handles compositions with custom color tweens correctly. App.tsx: remove polling interval, rely on runtime postMessage events for caption detection. Add clarifying comment on why useEffect is appropriate (external event subscription). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): restore cssScale in overlay coordinate conversion getBoundingClientRect() on iframe-internal elements returns coordinates in the iframe's native resolution (1920x1080), not the CSS-scaled display size. The cssScale multiplier is needed to convert to parent window coordinates. The earlier removal was incorrect — it only worked at 1:1 scale. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): fix reversed scaling on left-side corner handles Scale interaction used horizontal dx from start position, which goes negative when dragging left handles outward. Now uses distance from box center — dragging away from center increases scale regardless of which corner handle is used. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): make rotation respond to horizontal drag only Rotation handle sits directly above the word, so atan2-based rotation barely responds to left/right movement. Replace with linear horizontal mapping: drag right = clockwise, drag left = counter-clockwise, 200px = 90 degrees. Vertical movement is ignored. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): remove animation tab and typography/color from property panel Keep only Position (X, Y) and Transform (Scale, Rotation) controls. Remove tab switcher UI since there's only one view now. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix oxfmt formatting in CLAUDE.md and captions skill docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
d36c1785b9 |
feat(captions): energy-based technique selection and mandatory quality checks (#176)
## Summary - Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits - Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time, no per-frame callbacks needed - Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model) - Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility - Add multilingual model guidance and decision tree for model selection ## Test plan - [ ] Skill files render correctly as markdown - [ ] Cross-references between SKILL.md, dynamic-techniques.md, and transcript-guide.md resolve correctly - [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance Ingalls
|
37404f23da |
feat(whisper+captions): language detection, audio-reactive captions, multilingual defaults (#175)
## Summary **Whisper improvements:** - Auto-detect language and switch from `.en` to multilingual model when needed - Detect speech onset in WAV to strip hallucinated words before speech begins - Merge whisper-cpp token fragments: contractions (`didn` + `'t` → `didn't`), split capitals (`C` + `aught` → `Caught`), dropped-g (`shin` + `in'` → `shinin'`) - Interpolate zero-duration word clusters for reliable karaoke timing **Captions skill updates (folded from #176):** - Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits - Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time - Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model) - Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility **Multilingual defaults (folded from #186):** - Default whisper model changed from `small.en` to `small` to prevent silent translation of non-English audio - Added non-negotiable language rule to captions skill ## Test plan - [ ] `pnpm test` passes (contraction merging, fragment merging, zero-duration interpolation, speech onset) - [ ] Transcribe non-English audio — verify it transcribes in original language, not translates - [ ] Skill files render correctly, cross-references resolve - [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James
|
870ca76c8b | chore: release v0.2.1 | ||
|
Miguel Ángel
|
1efb23dfae |
fix(producer): inline CDN scripts for offline render and detect black video (#182)
## Context `npx hyperframes render` fails to load assets and produces all-black video. Root causes: 1. CDN scripts (GSAP, Lottie) are left as `<script src="https://...">` in compiled HTML — the headless browser must fetch them over the network, which fails in Docker, CI, and firewalled environments 2. Assets referenced from outside the project directory (e.g. `../shared-assets/hero.png`) 404 because the file server only serves from `projectDir` 3. When GSAP fails to load, the timeline never registers, all `.clip` elements stay `visibility: hidden`, and every frame is black — with no diagnostic output 4. The linter reports `missing_gsap_script` when GSAP is bundled inline (no `<script src>` tag), which blocks users from working around the CDN issue ## What changed ### 1\. CDN script inlining (`htmlCompiler.ts`) `compileForRender` now downloads all external `<script src="https://...">` tags at compile time and inlines their content into the HTML. Rendering no longer needs network access. | Before | After | | --- | --- | | `<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>` stays in HTML, browser must fetch at runtime | Script downloaded during compilation, embedded as `<script>/* inlined: https://... */\n...code...</script>` | | Docker/CI render → `net::ERR_NAME_NOT_RESOLVED` → black video | Render works fully offline | | No feedback when CDN fails | `[Compiler] WARNING: Failed to download CDN script: ... Consider bundling it locally` | ### 2\. External asset copying (`htmlCompiler.ts`, `renderOrchestrator.ts`) After compilation, the HTML is scanned for `src`, `href`, and CSS `url()` references that resolve outside `projectDir`. These files are copied into the compiled output directory so the file server can serve them. | Before | After | | --- | --- | | `background-image: url(../shared-assets/hero.png)` → 404 (file server can't serve outside `projectDir`) | Asset detected, copied to compiled dir, path rewritten → serves correctly | | `<img src="../shared-assets/logo.png">` → 404 | Same fix — works for all `src`/`href` attributes and CSS `url()` | ### 3\. Black video diagnostics (`renderOrchestrator.ts`) When composition duration is 0 (which would produce a black video), the error now probes the browser for diagnostics instead of a generic message. | Before | After | | --- | --- | | `Invalid composition duration: 0. Check that GSAP timelines are registered.` | `Composition duration is 0 — this would produce a black video.\n\nDiagnostics:\n - GSAP is not loaded — CDN script may have failed to download. Bundle GSAP locally...\n - Browser: [Browser:PAGEERROR] gsap is not defined` | | Asset 404s during page load silently logged | `[Render] Asset load failure: ...` + `[WARN] Browser encountered network failures during page load` | ### 4\. Linter: recognize inline GSAP (`core/lint/rules/gsap.ts`) The `missing_gsap_script` rule now recognizes GSAP bundled inline — matching the producer's inlining comment (`/* inlined: ...gsap... */`), GSAP library internals (`_gsScope`, `GreenSock`), and large inline scripts (>5KB) referencing gsap. | Before | After | | --- | --- | | User inlines GSAP → linter errors with `missing_gsap_script` | Inline GSAP detected, no false error | | Producer inlines CDN → linter errors on the compiled HTML | Producer's `/* inlined: ... */` comment recognized | ## Test plan - [x] `pnpm build` passes - [x] Core tests pass (410/410, +2 new) - [x] **Reproduced baseline failures on** **`main`**: CDN scripts not inlined, external assets 404, no diagnostics - [x] **Verified fixes**: CDN script inlined, external assets copied and served, diagnostics printed - [x] Render with working CDN → `[Compiler] Inlined CDN script: ...` → render succeeds - [x] Render with broken CDN → `[Compiler] WARNING: Failed to download CDN script` + browser errors surfaced - [x] Render with assets outside project dir → `[Compiler] Found 1 asset(s) outside project directory` → assets served correctly - [x] Linter with inline GSAP → no `missing_gsap_script` false positive |
||
|
Miguel Ángel
|
2213e9f7a4 |
refactor(lint): split monolithic test file into per-rule-module test files (#181)
## Summary - Splits the 872-line `hyperframeLinter.test.ts` into focused per-domain test files matching the rule module structure from #170 - Tests still use the public API (`lintHyperframeHtml`) — no internals exposed - Fixes misplaced tests: `gsap_css_transform_conflict` tests were inside `describe("lintScriptUrls")`, caption/adapter tests were inside `describe("template_literal_selector rule")` ## New structure ``` src/lint/ hyperframeLinter.test.ts — orchestrator integration + lintScriptUrls (8 tests) rules/ core.test.ts — root attrs, timeline registry, host id (6 tests) media.test.ts — duplicate ids, missing id/src, preload (6 tests) gsap.test.ts — clip element, transform conflict, missing script (12 tests) captions.test.ts — caption exit, overflow, relative position (5 tests) composition.test.ts — external script, template literal selector (5 tests) adapters.test.ts — missing Lottie/Three.js scripts (8 tests) ``` ## Test plan - [x] 404 tests pass (402 before — 2 extra from previously misplaced adapter tests now correctly counted) - [x] Lint + format clean |
||
|
Miguel Ángel
|
1681350ac4 |
fix(preview): rewrite sub-composition asset urls in styles (#174)
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting
## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.
## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`
## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
|