mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
f84cc492deac84d492baa172a8f5aaa8b9ed7953
65
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 terencecho
|
9abf65ae5e |
fix(player): correct playback rate for direct-timeline and audio-clock paths (#849)
## Summary - **Direct-timeline path** (GSAP compositions with `window.__timelines`): The player drives these via `DirectTimelineAdapter`, bypassing postMessage entirely. Rate changes sent `set-playback-rate` to the iframe but had no receiver — GSAP's `timeScale()` was never called. Fix: add optional `timeScale?` to `DirectTimelineAdapter` and call `this._directTimelineAdapter?.timeScale?.(rate)` in `attributeChangedCallback`. GSAP timelines expose `timeScale` natively, no composition changes required. - **Audio-clock path** (compositions with audio): Three bugs caused `TransportClock` to always run at 1x when an audio element or WebAudio context drove the clock: 1. `schedulePlayback` was called without the `playbackRate` arg (defaulted to 1). 2. `onSetPlaybackRate` and `player.setPlaybackRate` didn't call `webAudio.setRate()`. 3. `TransportClock.attachAudioSource` divided by `this._rate` instead of `el.playbackRate`, cancelling the rate multiplier. - Adds 2 regression tests to `clock.test.ts` covering the corrected audio-clock formula. ## Test plan - [ ] Unit tests: `bun run --cwd packages/core test` — 861/861 pass - [ ] Browser verification (Playwright headless, GSAP direct-timeline composition): - 1x speed → ratio 0.972 ✓ - 2x speed → ratio 1.965 ✓ - 0.5x speed → ratio 0.490 ✓ 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Phuong Le
|
3aa5cf3ab3 | fix(core): update nested timed element visibility on seek (#823) | ||
|
terencechoandClaude Sonnet 4.6
|
c08e8b2322 |
fix(player): drive composition ticks from widget-frame rAF via postMessage (#805)
Chromium throttles requestAnimationFrame in deeply nested cross-origin iframes. In Claude desktop (Electron), the composition iframe's own rAF loop stalls, so GSAP is never seeked and animation freezes even when TransportClock.isPlaying() is true. The correct fix is to drive ticks from the widget-frame rAF, which lives one level up and is not subject to the same throttling. When play() takes the runtime bridge path (no direct timeline adapter), the player now starts a parent-frame rAF loop that sends "tick" postMessages to the composition iframe on every frame. The runtime's control bridge handles "tick" by calling seekTimelineAndAdapters(clock.now()) if the clock is playing — identical to what transportTick does on each rAF, just driven from outside. The composition iframe's own rAF loop is unchanged and keeps running normally in standard browsers. Seeking GSAP twice per frame is idempotent, so there is no regression on claude.ai or any other non-throttled environment. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
ac671bdf5c |
feat(core): add TypeGPU/WebGPU runtime adapter (#755)
* feat(core): add TypeGPU/WebGPU runtime adapter
Adds a deterministic seek adapter for compositions that render with
TypeGPU or raw WebGPU. Follows the same push+poll pattern as the
Three.js adapter:
- Sets `window.__hfTypegpuTime` on every seek so render loops can
poll it instead of `performance.now()`.
- Dispatches a `"hf-seek"` CustomEvent on `window` so compositions
can imperatively re-render a single frame at the new seek position.
Compositions listen for the event and update their time uniform:
```js
window.addEventListener("hf-seek", (e) => render(e.detail.time));
```
Works with TypeGPU (docs.swmansion.com/TypeGPU) and raw WebGPU alike.
No assumptions are made about pipeline construction — multiple canvases
or renderers are supported by sharing the same event.
- 9 unit tests, all pass
- wired in init.ts adapter array
- `__hfTypegpuTime` declared in window.d.ts
* fix(core): deduplicate hf-seek dispatch across GPU adapters
Both three and typegpu adapters previously dispatched the same
"hf-seek" CustomEvent independently, causing any composition that
registered a listener to receive two events per seek tick — doubling
per-scrub GPU work even though the renders are idempotent.
Fix: extract a shared `dispatchSeekEvent` helper (seek-dispatch.ts)
that deduplicates by exact float equality within the same synchronous
call stack. Both adapters now call this helper instead of dispatching
directly.
Also adds:
- `resetSeekDispatchState()` export for test isolation
- `beforeEach` reset in three.test.ts and typegpu.test.ts
- New typegpu test: "duplicate seek to same time fires event only once"
- Docstring additions to typegpu.ts: render-mode determinism contract
(await device.queue.onSubmittedWorkDone()) and navigator.gpu feature
detection guidance for composition authors
* feat(core): video-texture render compat + TypeGPU skill
Adds the missing pieces for video-backed WebGPU effects in render mode:
- `video-texture-compat.ts`: monkey-patches `GPUQueue.copyExternalImageToTexture`
to detect the engine's injected `<img class="__render_frame__">` siblings and
transparently substitute them for `<video>` sources. Headless Chrome can't
supply decoded video frames to WebGPU, but the engine's pre-extracted frame
images work. Falls through to the original path in preview mode.
- `patchVideoTextureCompat()` wired in init.ts after adapter array creation.
- `skills/typegpu/SKILL.md`: full authoring guide for TypeGPU/WebGPU compositions
covering contract, timeline registration, video-backed effects, frosted blur
via downsample pass, WGSL patterns, and deterministic rendering.
* test(producer): add typegpu-adapter regression test
Self-contained WebGPU composition with:
- Procedural gradient background (no video dependency)
- Animated ring driven by hf-seek time uniform
- Pulsing center glow
- Two GSAP-driven captions testing adapter sync
Verifies the TypeGPU adapter's hf-seek → WebGPU render pipeline
produces deterministic frames. workers: 1 for consistency.
Note: output.mp4 baseline needs to be generated in CI — the local
Docker image can't launch Chrome (ARM/x86 mismatch on Mac).
|
||
|
|
38efe168e2 |
refactor(studio): contexts, PropertyPanel split, duration fix, perf (#748)
* feat(studio): add manual DOM editing inspector (#466) * fix: stabilize studio preview and runtime sync * fix: pass selector through timeline thumbnails * feat: add studio timeline editing * fix: disambiguate timeline edit targets * fix: stop timeline auto-scroll in fit mode * feat: use percentage-based timeline zoom * fix: sync timeline playhead on zoom changes * fix: reset timeline scroll when returning to fit * feat(studio): add manual DOM editing inspector * docs: update studio manual dom editing guide * feat(studio): add image asset picker for fills * feat(studio): add inline image uploads for fills * fix(studio): use real file input for image fill uploads * fix(studio): restore toast plumbing after rebase * fix(studio): explain in-app upload limitation * fix(studio): reuse asset-tab upload pattern in fills * feat(studio): refine manual design inspector * fix(studio): polish manual design inspector * fix(studio): keep color picker in viewport * fix(studio): clarify color picker selection * docs: update manual DOM editing guide * fix(studio): keep gradient color picker open * fix(studio): scope text color to text layers * fix(studio): add agent fallback for immovable layers * fix(studio): address manual editing review feedback * fix(studio): make local font selection reliable * fix(studio): improve dom picking and thumbnails * fix(studio): copy absolute paths in agent prompts * fix(studio): prevent timeline track cutoff * fix: copy Studio agent prompts in Safari * fix(studio): hold canvas movement from inspector * feat(studio): add persistent undo redo (#537) Studio manual editing and timeline editing mutate project files directly, but those edits had no reliable undo/redo path. Before releasing manual editing, users need a way to recover from visual property changes, source-editor saves, timeline moves/resizes/deletes, and timeline asset drops. The history also needs to survive a page refresh. A refresh should not erase the only way back from a bad manual edit. - Adds a persistent per-project edit-history model for file snapshots. - Stores undo/redo stacks in IndexedDB so history survives Studio refreshes. - Records source editor saves, manual DOM edits, and timeline mutations. - Adds toolbar undo/redo buttons with standard keyboard shortcuts: `Cmd/Ctrl+Z`, `Cmd/Ctrl+Shift+Z`, and `Ctrl+Y`. - Validates current file hashes before applying undo/redo so external file changes do not silently overwrite newer content. - Keeps history available in memory if IndexedDB persistence fails during a session. - Adds focused unit coverage for the pure history model, storage adapter, controller/hook behavior, and project-file save helper. Studio previously treated every editor mutation as an immediate file write. Manual DOM editing, timeline updates, and source-editor saves each had separate write paths, so there was no common transaction boundary where Studio could capture the file contents before and after an edit. Undo/redo needed to sit above those write paths as a file-level transaction system: capture changed files before saving, write the new contents, persist the history entry by project, then apply undo/redo only when the current file content still matches the expected snapshot. - `bun --filter @hyperframes/studio test src/utils/editHistory.test.ts src/utils/editHistoryStorage.test.ts src/hooks/usePersistentEditHistory.test.ts src/utils/studioFileHistory.test.ts` -> 4 files pass, 15 tests pass - `bun --filter @hyperframes/studio test` -> 26 files pass, 289 tests pass - `bun --filter @hyperframes/studio typecheck` - `bunx oxlint packages/studio/src/App.tsx packages/studio/src/icons/SystemIcons.tsx packages/studio/src/hooks/usePersistentEditHistory.ts packages/studio/src/hooks/usePersistentEditHistory.test.ts packages/studio/src/utils/editHistory.ts packages/studio/src/utils/editHistory.test.ts packages/studio/src/utils/editHistoryStorage.ts packages/studio/src/utils/editHistoryStorage.test.ts packages/studio/src/utils/studioFileHistory.ts packages/studio/src/utils/studioFileHistory.test.ts` -> 0 warnings, 0 errors - `bunx oxfmt --check packages/studio/src/App.tsx packages/studio/src/icons/SystemIcons.tsx packages/studio/src/hooks/usePersistentEditHistory.ts packages/studio/src/hooks/usePersistentEditHistory.test.ts packages/studio/src/utils/editHistory.ts packages/studio/src/utils/editHistory.test.ts packages/studio/src/utils/editHistoryStorage.ts packages/studio/src/utils/editHistoryStorage.test.ts packages/studio/src/utils/studioFileHistory.ts packages/studio/src/utils/studioFileHistory.test.ts` - `git diff --check` - `bun run --filter @hyperframes/core build:hyperframes-runtime` before commit hook, because the clean worktree needed the ignored runtime-inline artifact for typecheck - Lefthook pre-commit -> lint, format, typecheck pass - Lefthook commit-msg -> commitlint pass - Started Studio locally at `http://127.0.0.1:5190/#project/undo-redo-sample`. - Used `agent-browser` to select a preview element in the Inspector and change `#hero-card` from `left: 220px` to `left: 260px`. - Refreshed Studio and verified Undo stayed enabled. - Clicked Undo and verified the project file returned to `left: 220px`; clicked Redo and verified the inline `left: 260px` returned. - Used `agent-browser` to drag the `side-card` timeline clip, refreshed Studio, then verified Undo restored the previous timeline attributes and Redo reapplied the timeline move. - Recorded the tested undo/redo flow with `agent-browser`: `qa-artifacts/studio-undo-redo-2026-04-28/studio-undo-redo-flow.webm`. - Local screenshots and recordings are kept under `qa-artifacts/studio-undo-redo-2026-04-28/` and are intentionally not committed. - The scratch Studio project used for browser proof is local-only under `packages/studio/data/projects/undo-redo-sample/` and is intentionally not committed. - The PR intentionally excludes the earlier PRD/TDD planning notes under `docs/superpowers/`; those remain local-only per request. * fix: align Studio capture with preview (#595) Studio frame capture could fail for projects mounted outside the repo when the project id came from an encoded hash route. A project like `Notion Showcase` loaded as `#project/Notion%20Showcase`, but the capture URL encoded that already-encoded value again, producing `/api/projects/Notion%2520Showcase/...` and a 404. While validating the fix by seeking through the preview, capture also diverged from the visible player for nested compositions because the thumbnail route sought raw timelines instead of the same player seek path used by Studio preview. - Decodes project ids when reading Studio `#project/...` routes and centralizes project hash/API path construction. - Keeps API URLs encoded exactly once, including project names with spaces, literal `%`, reserved characters, and unicode. - Updates Studio thumbnail capture to prefer `window.__player.seek(t)` and only fall back to raw timeline seeking for standalone pages. - Preserves explicit `t=0` thumbnail requests instead of falling back to `0.5` seconds. - Adds preview-regression CI coverage for Studio routing, frame capture URL construction, thumbnail seeking, and core thumbnail seek parsing. Studio treated the hash route segment as the canonical project id even when the browser had already percent-encoded it. `buildFrameCaptureUrl` then encoded that string again, so a decoded project directory name and the capture API path no longer matched. The preview/capture mismatch was a separate seek-path issue: the visible Studio preview seeks through the HyperFrames player, which maps global time into nested composition time. The capture route bypassed that layer and paused all registered timelines at the same global time. The zero-second capture case came from parsing `t` with a truthiness fallback, so `parseFloat("0") || 0.5` became `0.5`. - `bun run --cwd packages/studio test -- vite.thumbnail.test.ts src/utils/projectRouting.test.ts src/utils/frameCapture.test.ts` - `bun run --cwd packages/core test -- src/studio-api/routes/thumbnail.test.ts` - `bunx oxfmt --check .github/workflows/preview-regression.yml packages/studio/vite.thumbnail.ts packages/studio/vite.thumbnail.test.ts packages/studio/vite.config.ts packages/studio/src/utils/projectRouting.ts packages/studio/src/utils/projectRouting.test.ts packages/studio/src/utils/frameCapture.ts packages/studio/src/App.tsx packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts` - `bunx oxlint .github/workflows/preview-regression.yml packages/studio/vite.thumbnail.ts packages/studio/vite.thumbnail.test.ts packages/studio/vite.config.ts packages/studio/src/utils/projectRouting.ts packages/studio/src/utils/projectRouting.test.ts packages/studio/src/utils/frameCapture.ts packages/studio/src/App.tsx packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts` - `bun run --cwd packages/studio typecheck` - `bun run --cwd packages/core build:hyperframes-runtime` - `bun run --cwd packages/core typecheck` - `git diff --check` Pre-commit also reran lint, format, and typecheck successfully for the committed files. Using `agent-browser`, I mounted `/Users/miguel07code/Downloads/Notion Showcase` into Studio's project data and opened: ```text http://127.0.0.1:5197/#project/Notion%20Showcase ``` Before the fix, Capture requested `/api/projects/Notion%2520Showcase/thumbnail/index.html?...` and Studio showed `Capture failed`. After the fix, I sought the preview to `0s`, `2s`, `10s`, and `18s`, captured each frame, and compared the visible preview crop against the capture output. The capture URLs all used `Notion%20Showcase`, not `Notion%2520Showcase`, and no failure toast appeared. Mean pixel diffs for preview vs capture were: - `0s`: `0.0` - `2s`: `0.8641` - `10s`: `0.3496` - `18s`: `0.2309` The small non-zero diffs are raster/antialias-level differences after resizing the capture to the preview crop dimensions. - Browser screenshots, comparison sheets, network logs, and the `agent-browser` recording are local-only under `qa-artifacts/capture-button/` and are not committed. - The local Notion Showcase project mount is an ignored symlink under `packages/studio/data/projects/` and is not committed. - Thumbnail cache versions were bumped so stale captures generated with the old seek behavior are not reused. * feat: persist studio manual edits via manifest * fix(studio): stabilize manual edit manifest rendering * fix(studio): allow master canvas layer selection * fix(studio): scale master edits in source coordinates * fix(studio): reapply manual edits during playback * fix(studio): keep rotation edit base stable * feat(studio): highlight hovered canvas target * fix(studio): drag hovered canvas targets immediately * fix(studio): rotate manual edits around center * fix(studio): keep rotate handle aligned while dragging * fix(studio): allow small rotation adjustments * fix(studio): match rotate handle size to resize handle * fix(studio): connect rotate handle line to selection * feat(studio): reset selected manual edits * fix(studio): route inspector geometry through manual edits * feat: add studio group repositioning * fix: preserve studio group selections * fix: seed additive studio selection groups * fix: select studio groups on pointerdown * fix: harden studio group overlay events * fix: address studio manual edit review feedback * fix: apply nested manual edits in drilled previews * fix: commit drag offsets from gesture math * fix: persist manual preview edits on refresh * fix: harden manual edit refresh apply * fix: share manual edit render runtime * chore: release v0.5.0-alpha.15 * feat(core): add studio animation preview APIs * feat(studio): add alpha editor layer inspector * chore: release v0.6.0-alpha.1 * feat(studio): enable inspector panels by default * fix(studio): keep motion panel opt-in * chore: release v0.6.0-alpha.2 * feat: auto-open timeline clip layers * feat: show composition loading in studio * feat: disable Studio timeline while composition loads * chore: ignore .claude directory * chore: release v0.6.0-alpha.3 * feat(studio): simplify inspector selection ux * fix(studio): keep notion preview playback moving * fix(studio): handle raster inspector clicks * fix(studio): stale selection, rotation control, design panel polish Fixes and improvements based on power-user testing feedback: 1. Fix stale selection after style edits — handleDomStyleCommit now calls refreshDomEditSelectionFromPreview after persisting, matching every other commit handler. Without this, the PropertyPanel showed frozen computedStyles after color/radius/shadow edits, making it look like editing "didn't work." Also adds error handling around the persist call. 2. Add rotation field to the Design panel Layout section — reads the current rotation angle from the manual edit manifest and commits via the existing handleDomRotationCommit handler. 3. Enable motion panel by default — STUDIO_MOTION_PANEL_ENABLED now defaults to true so the Motion tab is discoverable without env vars. 4. Color controls only when element has color — fill color section now only shows when the element has an explicit non-transparent background-color. Text color shows only when the element has a color style. Prevents showing color pickers on elements where color edits have no visible effect. 5. Exclude canvas from selection — added "canvas" to DOM_LAYER_IGNORED_TAGS so canvas elements are not selectable in the preview or listed in the layer panel. 6. Multi-selection feedback — shows "N elements selected" with guidance instead of the generic empty state when multiple elements are selected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): prevent browser launch timeout from crashing dev server The shared Puppeteer browser pool in getSharedBrowser() could throw a 30s TimeoutError during launch. This error propagated as an uncaught rejection and killed the vite process, even though generateThumbnail had its own try/catch — the browser launch promise rejected outside that scope. Now getSharedBrowser itself catches launch failures and returns null, so thumbnails degrade gracefully instead of crashing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): revert motion panel default to false Motion panel stays opt-in via env var per product direction. Only the Design panel is enabled by default. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): prevent read-only property crash in manual edit wrappers The seek/play/applyAfter wrapper functions in manualEdits.ts crashed with "Cannot set property X which has only a getter" when the player or timeline objects define seek/play as getter-only properties. This prevented ALL manual edits (position, rotation, size) from persisting to disk — the error thrown during applyCurrentStudioManualEditsToPreview aborted the save queue. Wrapped all three property assignments in try/catch so wrapping gracefully degrades when the target object is non-configurable. Verified: position edit (X=42px) now persists to .hyperframes/studio-manual-edits.json and survives page refresh. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: alpha preview e2e fixes — exports, init templates, EPIPE crash Three bugs found via automated e2e testing of the v0.6.0-alpha preview: 1. core: add missing package.json export specifiers for studio-api/manual-edits-render-script and studio-api/studio-motion-render-script — the alpha.3 npm publish failed because the studio build could not resolve these sub-paths. 2. cli: fix init --example creating empty projects — tsup leaves empty template directories in dist/ during the build, causing existsSync(templateDir) to return true and skip the remote fetch fallback. Now checks for index.html inside the dir instead. 3. engine: fix unhandled EPIPE crash in streaming encoder — ffmpeg stdin/stdout had no error handlers, so a write after the ffmpeg process exits throws an uncaught error that crashes the process. Verified with 8 consecutive e2e iterations (424 test runs, 0 flaky). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): thumbnail crash, feature defaults, multi-select UX, fps selector Power-user audit fixes for the alpha studio: - vite.config.ts: wrap thumbnail generation in try/catch so Puppeteer TimeoutError doesn't crash the entire vite dev server as an uncaught rejection. Close the page on error to prevent browser session leaks. - manualEditingAvailability.ts: enable motion panel and manual canvas drag editing by default (were both false, undiscoverable without knowing the env vars). - PropertyPanel.tsx: show "N elements selected" feedback when multiple elements are selected instead of the generic "Select an element" empty state. - RenderQueue.tsx + App.tsx: add FPS selector (24/30/60) to the render export bar instead of hardcoding 30fps. Pass the user's choice through to startRender. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: release v0.6.0-alpha.4 * fix(runtime): update clock duration when root timeline is late-bound Compositions with external sub-compositions (like apple-presentation with 7 slides) load child compositions via fetch(). The root GSAP timeline is only bound after all external compositions finish loading, but the TransportClock duration was only set during initial setup. When bindRootTimelineIfAvailable runs after the external compositions load, it captures the root timeline but never updates the clock. player.getDuration() continues returning 0, so the player's probe interval never fires the 'ready' event, and the Studio shows "Loading composition" indefinitely. Now bindRootTimelineIfAvailable updates clock.setDuration when the root timeline is late-bound. Guarded with try/catch for the early call site where clock is not yet initialized (temporal dead zone). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): block element selection while composition is loading Prevent users from selecting elements in the preview while the composition is still loading (showing "Loading composition" overlay). Selection and hover highlighting are suppressed until the player fires the ready event. Also reverts motion panel and manual drag editing defaults to false — these were accidentally set to true during the PR #693 merge. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: release v0.6.0-alpha.5 * chore: release v0.6.0-alpha.6 * fix(runtime): remove per-tick timeline.pause() that causes audio stutter The seekRuntimeTimeline helper added timeline.pause() before every totalTime() seek. During transport-driven playback, this runs 60 times per second, causing GSAP to cascade pause events to media elements on every frame. The result: audio plays/stops/plays/stops in a stutter pattern. The captured root timeline is already paused once in player.play() — the TransportClock drives it via totalTime(t) which keeps it paused. The extra per-tick pause() was redundant for the root timeline but actively harmful for media sync. Fix: restore the original inline seek for the captured timeline (totalTime without pause), keep seekRuntimeTimeline with pause() only for standalone child timelines where explicit pause control is needed. Also fixes rebase artifact: missing PropertyPanel props in App.tsx. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: release v0.6.0-alpha.7 * fix(studio): restore text field handlers lost in rebase Restores handleDomAddTextField and handleDomRemoveTextField that were dropped when resolving App.tsx conflicts during the main→next rebase. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: release v0.6.0-alpha.8 * fix(runtime): comprehensive audio stutter fix Three changes that together caused audio play/stop/play/stop stutter during transport-driven playback: 1. seekRuntimeTimeline called timeline.pause() before every totalTime() seek, 60x per second. GSAP cascades pause to media elements on every frame. Fix: restore original inline seek for the captured timeline (totalTime without pause). The timeline is already paused once in player.play(). seekRuntimeTimeline with pause() remains only for standalone child timelines. 2. player.play() removed the !tl guard, allowing play without a captured timeline. But getSafeTimelineDurationSeconds(null) returns 0, so the clock has no duration → immediately reaches end → stops → restarts. Fix: when no timeline provides duration, fall back to the root composition element's data-duration attribute. 3. Audio source attachment added networkState guard that could cause the clock to flicker between audio-source and monotonic timing on transient media states. Fix: keep !rawEl.error guard (prevents errored audio from freezing the clock) but drop the networkState check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(runtime): skip drift corrections on playing video elements Seeking a playing video resets the browser's decoder pipeline, causing a ~150ms freeze while it re-buffers. During that freeze the monotonic clock advances, drift grows, and strict sync fires another seek — creating a perpetual stutter loop (176 seek events / 8s observed on the apple-presentation composition). Skip strict and force drift corrections for playing video elements; only hard sync (>0.5s catastrophic drift) warrants the decoder-reset cost. Audio elements are unaffected and retain the full correction tiers. Also propagate the asset-loading overlay state to the timeline so controls are disabled during "Preparing preview assets", matching the existing behavior for the initial composition loading overlay. * chore: release v0.6.0-alpha.9 * feat(studio): consolidate keyboard shortcuts into single handler Move all window-level keyboard shortcuts from 4 separate files into one `handleAppKeyDown` listener in App.tsx: - Shift+T: toggle timeline (was App.tsx, separate useMountEffect) - Cmd/Ctrl+Z: undo (was App.tsx, separate useEffect) - Cmd/Ctrl+Shift+Z: redo (was App.tsx, separate useEffect) - Cmd/Ctrl+1: sidebar Compositions tab (was LeftSidebar.tsx) - Cmd/Ctrl+2: sidebar Assets tab (was LeftSidebar.tsx) - Delete/Backspace: remove selected element (was Timeline.tsx) LeftSidebar exposes a ref handle for tab switching. Timeline watches selectedElement becoming null to clean up popover/range UI state. History hotkey kept as named function for iframe forwarding. Playback shortcuts (Space, J/K/L, arrows) and caption nudge remain in their component hooks — tightly coupled to component state. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): sidebar tab overflow + hot-reload double-refresh 1. Sidebar tabs: use equal 1fr columns, shorter "Comps" label, truncate on overflow, tighter padding. Fixes tabs clipping outside the rounded pill at narrow sidebar widths. 2. Hot reload: set domEditSaveTimestampRef before every save-then-refresh path (source editor, timeline move/resize/delete, asset drop). The file-change watcher already checks this timestamp and suppresses echoed events — but source editor saves and timeline operations weren't setting it, causing a double refreshKey increment that could leave the player in a non-playable state. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): delete key removes preview-selected elements The consolidated keyboard handler only checked selectedElementId (timeline clips). When a user selected a child element in the preview via the inspector, selectedElementId was null because the element didn't correspond to a top-level timeline clip, so Delete/Backspace did nothing. Add handleDomEditElementDelete that removes the element referenced by the current domEditSelection via the remove-element mutation API. The Delete key handler now falls through from timeline selection to DOM edit selection. * fix(studio): remove unused deleteInFlightRef from Timeline Leftover from moving Delete handling to the consolidated keyboard handler in App.tsx. Also suppress pre-existing exhaustive-deps warning on the intentional every-render selection-change watcher. * fix(studio): forward all keyboard shortcuts to preview iframe The consolidated handleAppKeyDown was only added to the parent window. When focus was inside the preview iframe (after clicking an element), keydown events didn't reach the parent, so Delete and other shortcuts didn't fire. Replace the per-function iframe forwarding (handleTimelineToggleHotkey only) with the full app-level handler via a ref-stable wrapper. All app shortcuts (Delete, Undo/Redo, Shift+T, Cmd+1/2) now work from within the preview iframe. * fix(core): search inside <template> content when removing elements linkedom's document.querySelectorAll does not traverse <template> content. Elements in template-based compositions (like .title-word, .bullet-text) were invisible to the removal logic, so delete returned changed: false and the element survived the reload. Fall back to template.querySelectorAll when the document-level query returns no matches. Uses template.querySelectorAll directly (not template.content.querySelectorAll) because removing from the content DocumentFragment doesn't update the serialized output. * fix(studio): suppress loading overlay on hot-reload Only show the composition loading overlay on the first iframe load. Hot-reloads (source editor save, timeline edits, element delete) no longer flash the full-screen loading state. * fix(studio): reorder design panel, fix stroke height, rename Blending - Move Text section to the top of the panel (before Layout) - Remove Selection Colors section - Rename "Blending" to "Transparency" - Fix stroke Width/Style height mismatch by making SelectField use inline label layout matching MetricField * fix(studio): prevent panel scroll when wheel-adjusting metric inputs React registers onWheel passively, so preventDefault had no effect on the parent scroll container. Replace with a native wheel listener (passive: false) that blocks both default scroll and propagation. * chore: release v0.6.0-alpha.10 * chore: release v0.6.0-alpha.11 * fix(studio): clean next alpha inspector artifacts * chore: release v0.6.0-alpha.12 * fix(studio,player,core): eliminate double audio and manifest polling loop (#722) Three bugs that compound in Studio preview: 1. **Double audio on pause/resume**: syncRuntimeMedia played audio through the HTML <audio> element while WebAudioTransport simultaneously played the same source through AudioBufferSourceNode. Fixed by passing webAudio.isActive() as outputMuted so HTML elements stay muted when Web Audio owns playback. Also removed the priorMuted restore in stopAll() which raced with the next play cycle. 2. **Manifest polling loop**: applyStudioManualEditsToPreview and applyStudioMotionToPreview unconditionally fetched from disk on every call, even without forceFromDisk. The runtime posts state messages every frame via postMessage, triggering React re-renders that re-invoked these functions ~60x/second. Fixed by returning early when no disk read is requested, and using refs instead of callbacks in useEffect deps. 3. **Parent proxy double-play**: the player web component created parent-frame audio proxies even when the runtime bridge was available, causing two audio sources on autoplay-blocked promotion. Fixed by skipping proxy creation when _hasRuntimeBridge returns true, and synchronously muting iframe media on promotion to close the async race window. Also fixes pre-existing ResolutionPreset type missing square variants. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): improve font picker and text property controls (#736) - Line height and letter-spacing: convert from free-text to select with presets - Font style: remove oblique (browser falls back to italic), keep normal/italic - Font weight: detect available weights via document.fonts.check(), add labels - Font source: local fonts matching Google catalog tagged as Google - Font list: balanced per-source caps prevent any source from being cut off - Sort order: Google fonts rank before Local so curated fonts appear first Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): inspector visibility, undo/redo blinking, and preview caching Inspector picks invisible elements when an ancestor has GSAP-set opacity: 0 because CSS opacity is not inherited — getComputedStyle on the child still returns 1. Walk the ancestor chain in the picker, domEditing, and overlay visibility checks to catch this. Also: - Containers with all-invisible children are no longer selectable - Selection/hover overlay hides during playback and while loading - Undo/redo no longer double-refreshes (echo suppression for all file writes) - Undo/redo reloads iframe in-place instead of recreating the Player, preserving shader transition cache - Preview routes return ETag + Cache-Control headers; composition HTML uses project signature for conditional 304, binary assets use mtime+size - Loading overlay deferred 400ms so cached loads never flash it * fix(studio): remove timeline inspector buttons, enable manual dragging Remove the eye icon (inspector) and image icon (thumbnail toggle) from timeline clips. The timeline layer inspector feature and all supporting code is removed. Enable manual dragging in the preview by default. Add scrub-to-drag on X/Y/W/H fields in the design panel. Hide the Radius section when the element has no visible background. Fix pre-existing ResolutionPreset type for square presets. * chore: release v0.6.0-alpha.13 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): add rotation field, inline element drag, fix manifest load regression (#743) - Add rotation (R) field to geometry row (X, Y, W, H, R) in property panel. Goes through manifest via handleDomRotationCommit, resettable with Reset Edits. - Auto-promote display:inline elements to inline-block when dragged so translate works on inline spans. - Fix regression from polling fix: iframe load now passes readFromDiskFirst to load manifest from disk, so Reset Edits finds existing entries. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(studio): decompose App.tsx monolith (4297 → 567 lines) (#741) * refactor(studio): decompose App.tsx from 4297 to 567 lines Break the monolithic StudioApp component into focused modules: Hooks (12 new): - usePanelLayout: resizable/collapsible panel state - useFileManager: file tree, CRUD, uploads, derived lists - useManifestPersistence: manual edit + motion manifest save queue - useTimelineEditing: clip move/resize/delete/drop handlers - useDomEditSession: DOM selection, style/text commits, preview interaction - useAppHotkeys: keyboard shortcuts, undo/redo, iframe hotkey sync - useCaptionDetection: auto-detect caption compositions - useRenderClipContent: timeline clip thumbnail rendering - useConsoleErrorCapture: preview iframe console error capture - useFrameCapture: frame capture download flow - useLintModal: lint execution and modal state - useCompositionDimensions: stage-size message listener Components (6 new): - AskAgentModal: agent prompt modal - StudioHeader: toolbar with undo/redo, capture, inspector toggle - StudioLeftSidebar: file tree + code editor (handles collapsed state) - StudioPreviewArea: NLELayout + overlays + caption timeline - StudioRightPanel: Design/Motion/Renders tab panel - TimelineToolbar: zoom controls + timeline toggle Utilities (4 new): - studioHelpers: types, path helpers, DOM utilities - studioPreviewHelpers: preview pointer/player interaction - domEditHelpers: selection group algebra - studioFontHelpers: font injection + @font-face management Also removes dead timeline layer inspector code (eye icon, thumbnail toggle, layer panel) that was disabled behind a feature flag. * feat(studio): add Layer (z-index) field to design panel Adds a scrub-enabled "Layer" field below the W/H inputs in the Layout section. Available for all elements regardless of style editing capability since z-index is fundamental to composition stacking order. * docs: architecture spec for studio domain contexts, hook split, and file-size lint * docs: implementation plan for studio contexts, hook split, and file-size lint * refactor(studio): consolidate duplicate helpers in useDomEditSession Remove ~370 lines of helper functions that were copied into the hook instead of imported. All removed functions already exist in the canonical utility files (studioHelpers, studioFontHelpers, studioPreviewHelpers, domEditHelpers). Also removes the duplicate local type definitions for RightPanelTab, AgentModalAnchorPoint, and PreviewLocalPointer, and drops now-unused imports (googleFontStylesheetUrl, importedFontFaceCss, resolveVisualDomEditSelectionTarget, DomEditViewport). Temporarily excludes useDomEditSession.ts from the 500 LOC file-size check until Tasks 3-5 split it into focused hooks. * refactor(studio): extract useDomSelection from useDomEditSession * refactor(studio): extract useAskAgentModal from useDomEditSession * refactor(studio): extract usePreviewInteraction from useDomEditSession * refactor(studio): extract useDomEditCommits, useDomEditSession now thin orchestrator Split the 897-line useDomEditSession into focused hooks: - useDomEditCommits (439 LOC): manifest commits (path offset, box size, rotation, manual edits reset, motion), persist operations, element delete, font asset resolution - useDomEditTextCommits (329 LOC): style/text/text-field commits - useDomEditSession (339 LOC): thin orchestrator wiring selection, agent modal, preview interaction, and commit hooks All files now under 500 LOC limit. Removed the temporary lefthook filesize exclusion for useDomEditSession. * feat(studio): add 4 domain contexts (PanelLayout, FileManager, DomEdit, Studio) Create context providers that wrap hook return values for prop-drilling elimination. Each context destructures and reconstructs the value inside useMemo so exhaustive-deps is satisfied and re-renders are minimized. Not yet wired into App.tsx — that comes in a follow-up. * refactor(studio): wire domain contexts, eliminate prop drilling in 4 components Wire StudioProvider, PanelLayoutProvider, FileManagerProvider, and DomEditProvider in App.tsx. Migrate StudioHeader, StudioLeftSidebar, StudioPreviewArea, and StudioRightPanel to consume contexts instead of props. Prop counts reduced: - StudioHeader: 13 -> 6 - StudioLeftSidebar: 19 -> 4 - StudioPreviewArea: 37 -> 11 - StudioRightPanel: 39 -> 3 Net: -118 lines, 108 props removed from call sites. * chore: upgrade to React 19 Upgrade react and react-dom from 18.3 to 19.2.6 across the workspace. Add resolutions/overrides in root package.json to prevent peer dependency pins (e.g. @phosphor-icons/react) from pulling React 18. Regenerate bun.lock. This enables the React 19 context syntax (<Context value={...}>) used by the new domain contexts. * fix(studio): refresh preview after z-index change so stacking updates visually * fix(studio): remove duplicate duration override causing oscillation The timeline message handler set the duration twice: once via processTimelineMessage and once via a raw durationInFrames override. When drilled into a sub-composition, these could disagree, causing the duration to oscillate after element deletion. * fix(studio): use in-place iframe reload after clip delete, remove confirm dialogs Two changes to fix duration oscillation after deleting a timeline clip: 1. Replace setRefreshKey (full Player remount) with in-place iframe.contentWindow.location.reload() after deleting a clip. The full remount triggered a chaotic re-probing cycle with multiple duration sources (adapter, manifest, postMessage) fighting each other, causing the timeline to oscillate between durations. In-place reload preserves the Player web component and its state. 2. Remove window.confirm dialogs from both timeline clip delete and DOM element delete. Undo is available so the confirmation adds friction without value. * chore: gitignore docs/superpowers * feat(studio): add favicon * perf(studio): skip no-op state updates in timeline sync syncTimelineElements was called 60+ times per page load, each time triggering setElements/setDuration/setTimelineReady even when nothing changed. This caused massive re-render churn and memory usage. Add early-return guards to skip updates when values haven't changed. Also fixes the duration oscillation after element delete. * refactor(studio): split PropertyPanel.tsx (3126 LOC) into 8 focused modules The monolithic PropertyPanel.tsx exceeded the 500 LOC filesize limit. Split into cohesive modules by responsibility: - propertyPanelHelpers.ts (401) — pure utility functions, shared types/constants - propertyPanelPrimitives.tsx (357) — CommitField, MetricField, DetailField, SliderControl, SegmentedControl, SelectField, Section - propertyPanelColor.tsx (371) — ColorField, ColorSlider - propertyPanelFill.tsx (421) — ImageFillField, GradientField, asset path helpers - propertyPanelFont.tsx (455) — FontFamilyField + font catalog helpers - propertyPanelSections.tsx (453) — TextSection, TextFieldEditor, text controls - propertyPanelStyleSections.tsx (411) — StyleSections (stroke, effects, clip, fill) - PropertyPanel.tsx (347) — main component, LayerTree, re-exports for consumers All re-exports from PropertyPanel.tsx preserved for backwards compatibility. No behavioral changes — pure structural split. * fix(studio): use in-place iframe reload for all timeline operations Replace setRefreshKey with in-place iframe reload for move, resize, and asset drop — matching delete which was already fixed. Prevents the Player remount probe cycle that causes duration oscillation. * perf(studio): replace 5s polling loop with event-driven adapter init The Player's onIframeLoad used a setInterval polling loop (25 attempts × 200ms = 5 seconds) to detect when the runtime's __player/__timeline globals appeared. Each poll that missed triggered wasted work, and multiple duration sources fighting during the probe cycle caused oscillation bugs. Replace with event-driven initialization: 1. Fast path: try initializeAdapter() immediately (works for in-place reloads where the adapter is already present) 2. If not ready, listen for the runtime's "state"/"timeline" postMessage signals and initialize on the first one 3. Single 5s timeout as safety net (replaces 25 interval ticks) This eliminates the polling overhead, reduces setDuration/setElements calls to exactly 1 per load, and makes the Player responsive within one frame of the runtime being ready instead of up to 200ms later. * fix(studio): prevent duration oscillation after element delete Two fixes for the duration display oscillating between sub-composition and master durations after deleting an element in the preview: 1. Clear store elements before iframe reload in handleDomEditElementDelete. Without this, stale pre-delete elements remain in the store and cause mergeTimelineElementsPreservingDowngrades to alternate between REPLACE and PRESERVE modes as the element count fluctuates. 2. Add 500ms cooldown on enrichMissingCompositions after timeline messages. The "state" handler was calling enrichMissingCompositions every ~80ms, which added extra elements from GSAP timelines. These fought with the authoritative element list from "timeline" messages (~333ms), creating a feedback loop where element count oscillated and triggered alternating merge strategies with different durations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): single reloadPreview as source of truth for preview refresh Create reloadPreview() in App.tsx that encapsulates the correct behavior (in-place iframe reload with setRefreshKey fallback). Pass it as the sole refresh mechanism to hooks, removing direct setRefreshKey access from useTimelineEditing and useDomEditCommits. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(studio): decompose App.tsx from 4297 to 567 lines Break the monolithic StudioApp component into focused modules: Hooks (12 new): - usePanelLayout: resizable/collapsible panel state - useFileManager: file tree, CRUD, uploads, derived lists - useManifestPersistence: manual edit + motion manifest save queue - useTimelineEditing: clip move/resize/delete/drop handlers - useDomEditSession: DOM selection, style/text commits, preview interaction - useAppHotkeys: keyboard shortcuts, undo/redo, iframe hotkey sync - useCaptionDetection: auto-detect caption compositions - useRenderClipContent: timeline clip thumbnail rendering - useConsoleErrorCapture: preview iframe console error capture - useFrameCapture: frame capture download flow - useLintModal: lint execution and modal state - useCompositionDimensions: stage-size message listener Components (6 new): - AskAgentModal: agent prompt modal - StudioHeader: toolbar with undo/redo, capture, inspector toggle - StudioLeftSidebar: file tree + code editor (handles collapsed state) - StudioPreviewArea: NLELayout + overlays + caption timeline - StudioRightPanel: Design/Motion/Renders tab panel - TimelineToolbar: zoom controls + timeline toggle Utilities (4 new): - studioHelpers: types, path helpers, DOM utilities - studioPreviewHelpers: preview pointer/player interaction - domEditHelpers: selection group algebra - studioFontHelpers: font injection + @font-face management Also removes dead timeline layer inspector code (eye icon, thumbnail toggle, layer panel) that was disabled behind a feature flag. * docs: architecture spec for studio domain contexts, hook split, and file-size lint * docs: implementation plan for studio contexts, hook split, and file-size lint * refactor(studio): consolidate duplicate helpers in useDomEditSession Remove ~370 lines of helper functions that were copied into the hook instead of imported. All removed functions already exist in the canonical utility files (studioHelpers, studioFontHelpers, studioPreviewHelpers, domEditHelpers). Also removes the duplicate local type definitions for RightPanelTab, AgentModalAnchorPoint, and PreviewLocalPointer, and drops now-unused imports (googleFontStylesheetUrl, importedFontFaceCss, resolveVisualDomEditSelectionTarget, DomEditViewport). Temporarily excludes useDomEditSession.ts from the 500 LOC file-size check until Tasks 3-5 split it into focused hooks. * refactor(studio): extract useDomSelection from useDomEditSession * refactor(studio): extract useAskAgentModal from useDomEditSession * refactor(studio): extract usePreviewInteraction from useDomEditSession * refactor(studio): extract useDomEditCommits, useDomEditSession now thin orchestrator Split the 897-line useDomEditSession into focused hooks: - useDomEditCommits (439 LOC): manifest commits (path offset, box size, rotation, manual edits reset, motion), persist operations, element delete, font asset resolution - useDomEditTextCommits (329 LOC): style/text/text-field commits - useDomEditSession (339 LOC): thin orchestrator wiring selection, agent modal, preview interaction, and commit hooks All files now under 500 LOC limit. Removed the temporary lefthook filesize exclusion for useDomEditSession. * refactor(studio): wire domain contexts, eliminate prop drilling in 4 components Wire StudioProvider, PanelLayoutProvider, FileManagerProvider, and DomEditProvider in App.tsx. Migrate StudioHeader, StudioLeftSidebar, StudioPreviewArea, and StudioRightPanel to consume contexts instead of props. Prop counts reduced: - StudioHeader: 13 -> 6 - StudioLeftSidebar: 19 -> 4 - StudioPreviewArea: 37 -> 11 - StudioRightPanel: 39 -> 3 Net: -118 lines, 108 props removed from call sites. * fix(studio): refresh preview after z-index change so stacking updates visually * fix(studio): remove duplicate duration override causing oscillation The timeline message handler set the duration twice: once via processTimelineMessage and once via a raw durationInFrames override. When drilled into a sub-composition, these could disagree, causing the duration to oscillate after element deletion. * fix(studio): use in-place iframe reload after clip delete, remove confirm dialogs Two changes to fix duration oscillation after deleting a timeline clip: 1. Replace setRefreshKey (full Player remount) with in-place iframe.contentWindow.location.reload() after deleting a clip. The full remount triggered a chaotic re-probing cycle with multiple duration sources (adapter, manifest, postMessage) fighting each other, causing the timeline to oscillate between durations. In-place reload preserves the Player web component and its state. 2. Remove window.confirm dialogs from both timeline clip delete and DOM element delete. Undo is available so the confirmation adds friction without value. * chore: gitignore docs/superpowers * perf(studio): skip no-op state updates in timeline sync syncTimelineElements was called 60+ times per page load, each time triggering setElements/setDuration/setTimelineReady even when nothing changed. This caused massive re-render churn and memory usage. Add early-return guards to skip updates when values haven't changed. Also fixes the duration oscillation after element delete. * refactor(studio): split PropertyPanel.tsx (3126 LOC) into 8 focused modules The monolithic PropertyPanel.tsx exceeded the 500 LOC filesize limit. Split into cohesive modules by responsibility: - propertyPanelHelpers.ts (401) — pure utility functions, shared types/constants - propertyPanelPrimitives.tsx (357) — CommitField, MetricField, DetailField, SliderControl, SegmentedControl, SelectField, Section - propertyPanelColor.tsx (371) — ColorField, ColorSlider - propertyPanelFill.tsx (421) — ImageFillField, GradientField, asset path helpers - propertyPanelFont.tsx (455) — FontFamilyField + font catalog helpers - propertyPanelSections.tsx (453) — TextSection, TextFieldEditor, text controls - propertyPanelStyleSections.tsx (411) — StyleSections (stroke, effects, clip, fill) - PropertyPanel.tsx (347) — main component, LayerTree, re-exports for consumers All re-exports from PropertyPanel.tsx preserved for backwards compatibility. No behavioral changes — pure structural split. * fix(studio): use in-place iframe reload for all timeline operations Replace setRefreshKey with in-place iframe reload for move, resize, and asset drop — matching delete which was already fixed. Prevents the Player remount probe cycle that causes duration oscillation. * perf(studio): replace 5s polling loop with event-driven adapter init The Player's onIframeLoad used a setInterval polling loop (25 attempts × 200ms = 5 seconds) to detect when the runtime's __player/__timeline globals appeared. Each poll that missed triggered wasted work, and multiple duration sources fighting during the probe cycle caused oscillation bugs. Replace with event-driven initialization: 1. Fast path: try initializeAdapter() immediately (works for in-place reloads where the adapter is already present) 2. If not ready, listen for the runtime's "state"/"timeline" postMessage signals and initialize on the first one 3. Single 5s timeout as safety net (replaces 25 interval ticks) This eliminates the polling overhead, reduces setDuration/setElements calls to exactly 1 per load, and makes the Player responsive within one frame of the runtime being ready instead of up to 200ms later. * fix(studio): prevent duration oscillation after element delete Two fixes for the duration display oscillating between sub-composition and master durations after deleting an element in the preview: 1. Clear store elements before iframe reload in handleDomEditElementDelete. Without this, stale pre-delete elements remain in the store and cause mergeTimelineElementsPreservingDowngrades to alternate between REPLACE and PRESERVE modes as the element count fluctuates. 2. Add 500ms cooldown on enrichMissingCompositions after timeline messages. The "state" handler was calling enrichMissingCompositions every ~80ms, which added extra elements from GSAP timelines. These fought with the authoritative element list from "timeline" messages (~333ms), creating a feedback loop where element count oscillated and triggered alternating merge strategies with different durations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): single reloadPreview as source of truth for preview refresh Create reloadPreview() in App.tsx that encapsulates the correct behavior (in-place iframe reload with setRefreshKey fallback). Pass it as the sole refresh mechanism to hooks, removing direct setRefreshKey access from useTimelineEditing and useDomEditCommits. * fix: resolve lint errors from rebase (unused imports, duplicate declarations) * fix: prefix unused probeResult variable * fix: restore renderOrchestrator.ts from origin/next (rebase conflict artifact) * fix: resolve rebase conflicts by using main's producer and next's studio/player * fix: restore rebase-conflicted files from origin/next * fix: use 'load' instead of 'networkidle0' for Puppeteer waitUntil (type compatibility) * fix: restore webAudioTransport.ts from main (test compatibility) --------- Co-authored-by: Vance Ingalls <vance@heygen.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
e704a33c69 |
fix(core): preserve per-element preload ordering in render mode
The lazy media preloading refactor split bindMediaMetadataListeners into two loops: bind all listeners first, then preload all elements. This changed when metadata listeners fire relative to .load(), shifting timeline duration hydration and causing 3 transition frames to render at a slightly different state in the style-9-prod regression test. Move eager preload back inside the per-element binding loop so listener attachment and .load() happen in the same iteration, matching the original ordering. Lazy-mode demotion stays in a separate block after mediaPreloader.refresh() since it needs the full clip list. |
||
|
Miguel Ángel
|
f13c30e1f2 |
fix(core): skip media preloader activation in render mode
mediaPreloader.refresh() was called unconditionally, setting lazy=true for compositions with ≥6 clips even in render mode. player.seek then called preloadAroundTime() which evicted clips via src clearing, destroying buffered data needed for frame-accurate capture. Skip refresh() when __HF_EXPORT_RENDER_SEEK_CONFIG is set so isLazy() stays false and the preloader is completely inert during renders. |
||
|
Miguel Ángel
|
35eab94e69 |
fix(core): parent-frame proxy bypass, data-preload-eager opt-out, configurable threshold
- Player: _adoptIframeMedia now skips media with preload="metadata" or "none", preventing parent-frame proxies from bypassing the preloader. MutationObserver extended to watch preload attribute changes so proxies are created just-in-time when the preloader promotes a clip. - init.ts: lazy-mode demotion loop skips elements with data-preload-eager, letting power users keep specific clips eagerly buffered. - mediaPreloader: reads window.__HF_LAZY_PRELOAD_THRESHOLD as an override, falling back to the default 6. |
||
|
Miguel Ángel
|
a96d99680f |
fix(core): address staff review — diagnostics, comments, test coverage
- Add onActivation callback to MediaPreloadManager; wired to postRuntimeDiagnosticOnce in init.ts for observability - Document LAZY_THRESHOLD rationale (why 6) and MAX_PROMOTED defense-in-depth semantics - Add render-mode bypass contract test (isLazy with exactly 6 clips) - Add onActivation tests: fires once on lazy activation, skips below threshold, deduplicates across refreshes |
||
|
Miguel Ángel
|
b7438fa03d | fix(core): add missing mediaPreloader import dropped during rebase | ||
|
Miguel Ángel
|
0c6f438ae7 |
fix(core): add LRU eviction to media preloader, protect untimed media
Three root-cause fixes for the lazy media preloading feature: 1. Untimed media orphaned at preload="metadata": the else branch in bindMediaMetadataListeners demoted ALL media elements, but the mediaPreloader only manages timed clips (data-start). Untimed media (background audio, ambient loops) got stuck at metadata forever. Now only timed elements are demoted. 2. Monotonic promotion with no eviction: once promoted, clips stayed at preload="auto" forever. Scrubbing through the full timeline promoted everything, bringing back the OOM crash. Added LRU eviction with MAX_PROMOTED=5 — when clips leave the preload window, their src is cleared and load() called to release buffered data per MDN. On re-entry, the original src is restored. 3. Metadata preload without load(): setting preload="metadata" alone doesn't guarantee the metadata fetch in Chrome Lite mode or Firefox with media.preload.default=0. Now load() is called after demotion to ensure el.duration is populated for timeline computation. Also adds exact-boundary tests for LAZY_THRESHOLD=6 and eviction coverage (evict on scrub, src restoration, MAX_PROMOTED cap, load() called on eviction). |
||
|
Miguel Ángel
|
372da1cd28 |
feat(core): integrate media preloader into runtime and studio
Wire the MediaPreloadManager into init.ts: - Detect render mode via __HF_EXPORT_RENDER_SEEK_CONFIG (keeps eager preload) - Gate bindMediaMetadataListeners: lazy mode sets preload="metadata", eager mode keeps preload="auto" (unchanged for small compositions) - Advance preload window in the timeline poll tick loop - Call preloadAroundTime on seek for instant buffering at seek target Studio Player.tsx: hasUnloadedAssets now skips elements with preload!="auto" so deferred clips don't block the loading overlay. |
||
|
Miguel Ángel
|
773c4261e2 |
feat(core): lazy media preloading for heavy compositions
Compositions with many large video files (e.g., 6GB across 20 clips) crash the browser because the runtime eagerly sets preload="auto" + .load() on every media element at startup. All files buffer simultaneously, exhausting memory. Add a MediaPreloadManager that gates preloading based on playhead position: - Activates when a composition has ≥6 timed media elements - Only preloads clips within a 10-second lookahead window (or next 2 clips) - Far-away clips stay at preload="metadata" (resolves duration without downloading data) - Advances the window on each transport tick and immediately on seek - Render mode (window.__HF_EXPORT_RENDER_SEEK_CONFIG) keeps eager preload for deterministic frame capture - Small compositions (<6 clips) keep eager preload — no behavior change Studio's hasUnloadedAssets now skips elements with preload!="auto", so deferred clips don't block the loading overlay. |
||
|
James
|
117029a719 |
refactor(core): tighten WebAudio rate fix per review
- Hoist duplicated test mock helpers (createMockAudioContext / setupTransport / mockBuffer / mockEl) from the two describe blocks to module scope. - Drop redundant math-derivation comments in schedulePlayback; the dedicated rate-aware tests are the canonical proof. - Tighten setRate JSDoc. - Add no-op guard in setRate when the new rate equals the current rate, so a duplicate set-playback-rate postMessage doesn't re-anchor or walk active sources for nothing. - Add a regression test for the no-op guard, and strengthen the clamp test to schedule at rate=2 first so the clamp-to-1 assertion is non-vacuous. |
||
|
James
|
89ee1e36d7 |
fix(core): thread playback rate into WebAudio audio sources
WebAudioTransport scheduled AudioBufferSourceNodes with the implicit default playbackRate of 1, so non-1x transport rates desynced visuals from audio: GSAP timelines, the transport clock, and native <video> all sped up while WebAudio-routed <audio> clips kept playing at 1x. - schedulePlayback now accepts a rate, sets sourceNode.playbackRate, and scales the future-clip start delay by the rate (the in-progress buffer offset stays elapsed + mediaStart, which is rate-independent). - New setRate() updates active sources in place and rebases the getTime() reference frame so the audio-master clock stays continuous across mid-playback rate changes. - Runtime onSetPlaybackRate now forwards into webAudio.setRate, and player.play() schedules each clip with state.playbackRate. Fixes #713 |
||
|
Miguel ÁngelandClaude Opus 4.6
|
711ac22fd8 |
fix(runtime): add setter to delegated __player properties
The property delegation on window.__player used Object.defineProperty with only a getter, causing "Cannot set property renderSeek which has only a getter" when Studio's motion-wrapping code tried to reassign __player.renderSeek with a wrapped version. This cascaded into an infinite error loop making the timeline unusable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel ÁngelandClaude Opus 4.6
|
38522f996e |
fix(runtime): schedule future WebAudio clips with delay instead of playing immediately
Clips whose compositionStart is ahead of the current timeline position were starting immediately because sourceNode.start() always received when=0. Use the AudioContext scheduling API to defer future clips: sourceNode.start(ctx.currentTime + delay, mediaStart). Closes #674 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
b009288df7 |
feat(runtime): single-clock transport — eliminate pause/play audio drift (#671)
Replace the two-clock architecture (GSAP rAF ticker + HTMLMediaElement
pipeline reconciled by a 50ms polling loop) with a single TransportClock.
GSAP is always paused and seeked to clock.now() on each rAF tick.
Drift between visual timeline and audio is structurally impossible.
Architecture:
TransportClock.now() ──rAF──▶ timeline.seek(t) + el.currentTime
▲
AudioContext.currentTime (~21µs) ← WebAudio active
OR
audio.currentTime (~33ms) ← HTMLMediaElement fallback
OR
performance.now() (~1ms) ← no audio
Key changes:
- TransportClock class with monotonic + audio-master clock sources
- WebAudioTransport: routes audio through AudioBufferSourceNode for
sample-accurate scheduling, falls back gracefully to HTMLMediaElement
- rAF tick loop replaces 50ms setInterval poll; GSAP always paused
- Strict sync (40ms threshold, consecutive-sample gated) + forceSync
on play/pause/seek transitions for sub-frame media accuracy
- Buffer-stall: visuals freeze when audio is buffering instead of
running ahead
- Frame quantization preserved in seek/renderSeek (parity contract)
Browser-verified: 0.0ms drift after 40 pause/play cycles (was 400ms+).
Also fixes: CDN script HTML error responses in validate (pre-existing).
54 tests across clock, clock-drift, webAudioTransport, and media.
Closes #668
|
||
|
James
|
e87196456b |
feat(runtime): swallow() helper replaces empty catches in inlined runtime
After hf#641 inlined the runtime IIFE into every bundle, lint tools
inspecting bundled output (including Abhay's c2v eval) started flagging
empty `catch {}` blocks across the runtime. The source had explanatory
comments inside, but esbuild's minifier strips them — the IIFE ships
~10 visible patterns of `}catch{}` and consumers' linters fire on each.
Each empty catch is intentional best-effort error swallowing —
postMessage to a parent frame that may not exist, `media.play()` /
`pause()` that throw under autoplay restrictions, timeline `seek()` on
a disposed timeline, anime.js / lottie feature detection on hosts that
don't load those libraries, etc. The right behaviour stays "tried,
didn't work, move on", but doing it visibly improves three things:
- lint clean: helper call is a real statement; no `no-empty` warnings
survive minification
- debuggable: flip `window.__hfDebug = true` in DevTools to see every
swallow site with `console.debug` (silent in prod by default)
- observable: studio / embeddings can install
`window.__hf.onSwallowed = handler` to collect runtime swallow
events without polluting the page console
Implementation: `packages/core/src/runtime/diagnostics.ts` exports
`swallow(label, err?)`. 41 catch sites across 12 runtime files
converted via mechanical pass (auto-generated `runtime.<module>.siteN`
labels — labels can be tightened site-by-site as a follow-up; the
shape of the change is what matters here).
Verification:
- core 674/674 (incl. 6 new diagnostics tests covering silent default,
__hfDebug logging, legacy __HYPERFRAMES_DEBUG flag, handler hook,
handler-throws-doesn't-recurse, both-active)
- typecheck clean
- format / lint clean
- runtime IIFE rebuilds successfully (`bun run build:hyperframes-runtime`)
Refs Abhay's c2v eval — bundler artefacts now lint-clean with the
runtime body inlined.
|
||
|
Miguel ÁngelandClaude Opus 4.6
|
a7a6648852 |
feat(player): add volume/mute controls (#651)
* feat(player): add volume/mute controls to the player Adds a mute toggle button and volume slider to the controls bar, positioned between the time display and speed selector. The slider expands on hover for a compact default footprint. - `volume` attribute/property (0–1, clamped) with `volumechange` event - `muted` attribute now syncs to the controls UI (icon updates) - Three volume icons: high, low, muted — updates reactively - Volume forwarded to parent-frame audio proxies and iframe runtime via `set-volume` postMessage control - 9 new tests covering volume clamping, events, controls rendering, mute toggle, and iframe message forwarding Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(player): wire set-volume through runtime bridge + address review feedback Addresses the blocker from PR review: the iframe runtime now handles `set-volume` messages end-to-end (types → bridge → init → media sync). Runtime side: - Add `set-volume` to RuntimeBridgeControlAction union - Add `volume` field to RuntimeBridgeControlMessage - Handle `set-volume` in bridge.ts with [0,1] clamping - Store bridgeVolume in RuntimeState, apply to media elements - syncRuntimeMedia composes userVolume × clip author volume Player side: - Muted toggle now dispatches `volumechange` (HTML5 spec compliance) - Volume slider auto-unmutes when scrubbed above 0 while muted - Touch support on volume slider (touchstart/move/end) Tests: 5 new (3 bridge, 2 media) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(player): add ARIA keyboard controls, fix icon collision and clipVolume parity - Volume slider: role="slider", aria-label, aria-valuemin/max/now, tabindex=0, arrow key support (5% steps, auto-unmutes) - Volume=0 unmuted now shows low-volume icon instead of muted icon - Fix clipVolume divergence: init.ts uses Number.isFinite() matching media.ts semantics (preserves data-volume="0") - 3 new tests: ARIA attributes, volumechange on mute, icon collision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
a7b308b667 |
feat: cache shader transition preview frames (#634)
* feat: cache shader transition preview frames * fix: move shader transition loading to player |
||
|
Miguel ÁngelandClaude Opus 4.6
|
ea90bbe076 |
fix(runtime): clear play guard after hard seek to prevent audio desync on scrub (#639)
* fix(runtime): clear play guard after hard seek to prevent audio desync on scrub When scrubbing the timeline during playback, syncRuntimeMedia detects the offset jump and hard-seeks the media element. But the in-flight play() guard (playRequested WeakSet) from the previous play() call prevented the next sync tick from re-issuing play() — leaving the element paused at the new position for 50-150ms while the GSAP timeline continued advancing. This caused audible audio desync after every scrub. Fix: clear playRequested on the element after a hard seek so the very next sync tick can re-issue play(). Also adds a lint rule (video_audio_double_source) that catches compositions where an unmuted <video> and a separate <audio> point to the same source — a pattern that causes double playback at runtime. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(runtime): detect failed seeks past MP3 buffer and force full fetch Root cause: streaming MP3 with preload="metadata" only buffers the first ~15 seconds. Seeking past the buffered range silently fails — currentTime stays at 0 while the timeline advances, causing permanent audio desync that only a page refresh fixes. Three changes: 1. Move preload="auto" enforcement to run for ALL active elements on every sync tick (not just during play). This catches elements whose preload was overridden after init.ts set it. 2. After a hard seek, check if currentTime actually reached the target. If not (drift > 0.5s), call load() once to force the browser to fully fetch the media and build a complete seek index. 3. Clear the load-retry guard when the clip leaves its active window so re-entry can retry if needed. Reproduced on hyperframes.dev Hermes launch video: vo.mp3 buffered to 15.96s, seeking to 20s failed silently. bg-music.wav (fully buffered) was unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
c1b6efd9c5 |
feat(core,cli): variable schema validation + lint rules
Two lint rules + render-time validation built on top of the existing
data-composition-variables schema.
Lint rules (packages/core/src/lint/rules/composition.ts):
- invalid_variable_values_json — host's data-variable-values must parse as
a JSON object. Today the runtime swallows parse failures silently and
falls back to declared defaults, masking typos.
- invalid_composition_variables_declaration — root <html>'s
data-composition-variables must parse as an array of objects with
`id` (string), `type` (one of string/number/color/boolean/enum), `label`
(string), and `default`. Per-entry findings report which fields are
missing or invalid.
Both rules read attributes via a new `readJsonAttr` helper in lint/utils.ts.
The existing `readAttr` regex `["']([^"']+)["']` truncates JSON-in-attribute
values at the first internal quote (e.g. `data-variable-values='{"x":"y"}'`
captures only `{`); `readJsonAttr` alternates double-vs-single-quoted
branches with quote-specific char classes so JSON values round-trip cleanly.
A second helper `findHtmlTag` returns the actual <html> open tag (where
data-composition-variables lives) — distinct from `findRootTag` which
returns the first in-body composition element.
Render-time validation (packages/core/src/runtime/validateVariables.ts):
- validateVariables(values, declarations) returns a structured array of
issues: undeclared keys, type mismatches, enum-out-of-range values.
Pure / sync; works in any environment.
- formatVariableValidationIssue(issue) renders a one-line user-facing
string for CLI output.
- Both exported from @hyperframes/core for studio/tooling reuse.
CLI integration (packages/cli/src/commands/render.ts):
- New --strict-variables flag. Default behavior: print warnings and
continue. With --strict-variables: print warnings then exit 1.
- New `validateVariablesAgainstProject(indexPath, values)` helper:
reads the project's index.html, runs extractCompositionMetadata to
pull the declared schema, validates the CLI's --variables payload
against it. ensureDOMParser polyfill for Node-side parsing (same
pattern as compositions.ts).
Tests:
- 11 new validateVariables unit tests covering happy path, undeclared
keys, type mismatches (string/number/boolean/color/enum), enum range,
multiple-issue aggregation, and formatter output.
- 11 new composition.test.ts cases for both lint rules: parse errors,
shape errors, per-entry validation, unknown types, missing fields,
positive cases.
- 5 new render.test.ts cases for validateVariablesAgainstProject:
no-declarations, happy path, undeclared, type-mismatch, missing-file.
- All 646 core tests + 213 cli tests still green.
Docs:
- docs/packages/cli.mdx — added --strict-variables flag row.
This is PR 3 of a 4-PR stack. PR 4 ships skill/scaffold distribution.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
1da6f456b7 |
refactor(core): apply /simplify findings on sub-comp scoping PR
- compositionLoader.ts: drop the redundant inline `Window` cast; the
ambient `__hfVariablesByComp?` declaration in runtime/window.d.ts
already covers it within the same package.
- compositionScoping.test.ts: drop the `__captured: undefined as unknown`
initializers — `Record<string, unknown>` already permits the key, the
init was noise.
Reuse + efficiency reviews returned clean. The scoped getVariables's
per-call Object.assign({}, scoped) is consistent with the file's
existing scoped-utility conventions (gsap proxy returns fresh bound
functions per access) and acceptable since the idiomatic usage
destructures once at script init.
All 44 touched core tests still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
484ab54442 |
feat(core): scope getVariables() per sub-comp instance
Building on PR 1's getVariables() helper, this PR routes per-instance values into the correct sub-composition. Same composition source can now be embedded N times with different content via data-variable-values on each host element. How it works: - compositionLoader, before injecting wrapped scripts, layers the host element's data-variable-values JSON over the sub-comp's declared defaults (its own data-composition-variables) and writes the merged object to window.__hfVariablesByComp[compositionId]. Skipped when both sides are empty so the table only grows for instances that actually carry values. - compositionScoping's wrapper IIFE now takes a fourth parameter __hyperframes alongside the existing scoped document/gsap/window. The scoped __hyperframes shadows getVariables() to read from __hfVariablesByComp[__hfCompId], returning a fresh object each call so script mutations don't leak into the shared table. - Top-level scripts (not wrapped by compositionScoping) keep using the unscoped window.__hyperframes.getVariables(), which reads data-composition-variables defaults plus the CLI override (window.__hfVariables) — same path as PR 1. - readDeclaredDefaults is exported from getVariables.ts so the loader reuses the exact same defaults-extraction logic the helper uses for the top-level path. Inline templates (no separate <html> document root) get host overrides only — no declared defaults — since there's no separate <html> to read data-composition-variables from. External sub-comps fetched via data-composition-src get the full declared defaults + host overrides merge. Tests: 3 new compositionScoping tests covering scoped getVariables invocation, missing-entry fallback, and mutation isolation. 5 new compositionLoader tests covering merge order, declared-only path, empty-skip, invalid-host-JSON resilience, and per-instance scoping across two hosts sharing a source. 3 new getVariables tests covering the newly-public readDeclaredDefaults. All 622 core tests green. Docs: docs/concepts/compositions.mdx switched its sub-comp example from hand-rolled JSON.parse(host.dataset.variableValues) to the new __hyperframes.getVariables() pattern. data-attributes.mdx clarifies per-instance scoping behavior. This is PR 2 of a 4-PR stack. PR 3 adds schema validation + lint; PR 4 ships skill / scaffold updates. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
James Russo
|
03b82e6ff8 |
feat(core,cli,engine,producer): getVariables() helper + --variables render flag (PR 1/4) (#600)
## What Adds the parametrized-render primitive from [hf#592](https://github.com/heygen-com/hyperframes/issues/592) by introducing a `getVariables()` runtime helper plus a CLI `--variables` / `--variables-file` flag. Compositions declare variables once on the root `<html>` element (the existing `data-composition-variables` attribute, which already drives Studio editing UI), read them at runtime via `window.__hyperframes.getVariables()`, and CLI users override them at render time without touching the composition source. This is **PR 1 of a 4-PR stack**: 1. **PR 1 (this one)** — runtime helper + CLI flag + engine injection (top-level renders). 2. PR 2 — sub-comp per-instance scoping (carry the host's `data-variable-values` into the inlined sub-comp's `getVariables()`). 3. PR 3 — schema validation + lint rules (warn on undeclared variable IDs, optional `--strict-variables`). 4. PR 4 — skill / scaffold distribution (SKILL.md, AGENTS.md scaffolds, openai/plugins mirror). ## Why The existing `data-composition-variables` schema declares variable types and defaults but isn't readable from composition scripts and can't be overridden at render time. To produce N variations of a composition today, an agent has to fork the composition or edit the source HTML before each render. `--variables` collapses that into one render call per variation, matching Editframe's `--data` UX without copying their `getRenderData` framing — `getVariables()` is named for the codebase's existing "variables" terminology and works equally in dev preview and at render time. ## How - **Runtime helper** (`packages/core/src/runtime/getVariables.ts`): reads `data-composition-variables` from `document.documentElement`, extracts `{id: default}` defaults, merges `window.__hfVariables` (override) on top, returns `Partial<T>`. Same code path in dev preview (no override) and at render (with override). Generic parameter for typed editor ergonomics. Exposed both as a named export from `@hyperframes/core` and on `window.__hyperframes.getVariables` for vanilla compositions. - **CLI flag** (`packages/cli/src/commands/render.ts`): `--variables '<json>'` and `--variables-file <path>`. `parseVariablesArg` is split out as a pure function (returns a discriminated `{ ok: true } | { ok: false }` union) so all validation paths are unit-testable; the side-effecting `resolveVariablesArg` wraps it with `errorBox` + `process.exit`. Mutually exclusive with `--variables-file`; fail-fast on conflicts, missing file, unparseable JSON, or non-object payloads (string, number, array, null). - **Engine injection** (`packages/engine/src/services/frameCapture.ts`): added an `evaluateOnNewDocument` step right after the `__name` polyfill that sets `window.__hfVariables` to the parsed JSON before any page script runs. Skipped when payload is empty so we don't add pointless init scripts. Plumbed through `CaptureOptions.variables` and `RenderConfig.variables`. Docker mode forwards the flag to the in-container CLI via `dockerRunArgs`. - **Why a separate `__hfVariables` global** instead of writing into `__hyperframes.getVariables()` directly: the helper is an IIFE that has to be defined before composition scripts execute, but the *override* needs to land before *that*. `evaluateOnNewDocument` is the only reliable hook that runs before the runtime IIFE evaluates. Storing the raw value on `__hfVariables` and merging in the helper keeps both paths order-independent. ## Test plan - [x] Unit tests added/updated - 9 jsdom tests for `getVariables()` covering empty state, declared defaults only, override merge, override-wins, declared-only, invalid JSON, non-array payloads, non-object overrides, typed generic. - 7 tests for `parseVariablesArg` covering all validation paths. - 2 integration tests for `renderLocal` confirming `variables` reach `createRenderJob`. - 3 new `dockerRunArgs` assertions for `--variables` passthrough (set / not-set / empty-object). - All existing tests green: core 611, cli 208, engine 519. - [x] Manual testing performed - `npx tsx packages/cli/src/cli.ts render --help` shows both flags + the two new examples. - [x] Documentation updated - `docs/packages/cli.mdx` — added flags to the table and a "Parametrized renders" section with a worked example. - `docs/concepts/data-attributes.mdx` — added `data-composition-variables` row. ## Backwards compatibility Fully backwards compatible. Compositions without `data-composition-variables` work unchanged; `getVariables()` returns `{}` and the engine skips the injection step. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance Ingalls
|
8d83d4f132 |
fix: make caption overrides refresh-safe (#609)
## Summary This stacked PR makes caption overrides refresh-safe. Caption edits are still saved to `caption-overrides.json`, but override targets are now stable across preview refreshes and regenerated caption HTML. ## Architecture - **Stable word identity**: generated caption HTML preserves optional transcript word IDs in the `TRANSCRIPT` array and emits those IDs on word spans. - **Parser continuity**: the caption parser preserves existing transcript word `id` fields instead of regenerating index-only identity. - **Override loading**: Studio loads saved overrides by `wordId` first, with the existing `wordIndex` fallback kept for older overrides. - **Idempotent runtime wrapping**: transform overrides reuse an existing `data-caption-wrapper="true"` wrapper instead of nesting wrappers on every refresh. - **Animation compatibility**: overrides still wrap the word so inner word-level GSAP animation can continue to target the original span. ## User Impact Users can edit caption word position, scale, rotation, color, opacity, font size, font weight, and font family, then refresh without overrides drifting to the wrong word or accumulating nested wrappers. ## Main Files - `packages/core/src/runtime/captionOverrides.ts` - `packages/studio/src/captions/generator.ts` - `packages/studio/src/captions/parser.ts` - `packages/studio/src/captions/hooks/useCaptionSync.ts` ## Test Plan ```bash volta run --node 22.20.0 bun run --filter @hyperframes/core test -- src/runtime/captionOverrides.test.ts volta run --node 22.20.0 packages/studio/node_modules/.bin/vitest run --root packages/studio --config /dev/null src/captions/parser.test.ts src/captions/generator.test.ts volta run --node 22.20.0 bun run --filter @hyperframes/core typecheck volta run --node 22.20.0 bun run --filter @hyperframes/studio typecheck volta run --node 22.20.0 bunx oxlint <changed files> volta run --node 22.20.0 bunx oxfmt --check <changed files> git diff --check ``` |
||
|
JamesandClaude Opus 4.7
|
8c8dd6ad0c |
refactor(core,cli,engine): apply /simplify findings on getVariables PR
- core/runtime/getVariables.ts: collapse the noisy three-step type-guard
re-cast into a single `Record<string, unknown>` narrow with early-continue
guards. Same behaviour, ~6 lines shorter.
- cli/commands/render.ts: separate VariablesParseError from UI strings.
parseVariablesArg now returns a kind-discriminated error
(`conflict | read-error | parse-error | shape-error`) and the wrapper
resolveVariablesArg owns the title/message mapping via
`variablesErrorMessage`. Keeps the parser pure of presentation strings.
- cli/commands/render.test.ts: lift the `await import("./render.js")` into
a `beforeAll`, add a typed `expectErr` helper, assert on the structured
error kind instead of message-string regexes. Same coverage, less noise.
- engine/services/frameCapture.ts: replace the `as unknown as { ... }`
double-cast with a single named `WindowWithVariables` alias inside the
page closure.
All affected suites green (core getVariables 9, cli render 12, cli
dockerRunArgs 13).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
c0d75a5268 |
feat(core,cli,engine,producer): add getVariables() helper and --variables render flag
Adds the parametrized-render primitive from hf#592 by reusing the existing
data-composition-variables schema as the source of declared defaults.
- Runtime helper window.__hyperframes.getVariables() (also exported from
@hyperframes/core) reads data-composition-variables defaults from the
document root and merges window.__hfVariables (CLI override) on top.
Returns Partial<T> for typed access; supports a generic for editor
ergonomics. Same code path runs in dev preview and at render time.
- CLI render --variables '<json>' / --variables-file <path> populates the
override. Mutually exclusive; fail-fast on conflicting flags, missing
file, unparseable JSON, or non-object payloads. parseVariablesArg is
exported as a pure function so validation paths stay unit-testable.
- Engine injects window.__hfVariables via evaluateOnNewDocument before
any page script runs, so the helper sees the merged values on its
first call. Empty payloads are skipped to avoid pointless init scripts.
- Producer threads variables through RenderConfig and into the engine's
CaptureOptions; Docker mode forwards --variables to the in-container
CLI invocation via dockerRunArgs.
Composition authors declare variables once on the root <html> element:
<html data-composition-variables='[
{"id":"title","type":"string","label":"Title","default":"Hello"}
]'>
and read them in any composition script:
const { title } = window.__hyperframes.getVariables();
A render with `--variables '{"title":"Q4 Report"}'` overrides the default
without modifying the composition source. Missing keys fall through to
the declared defaults, so dev preview and CLI renders without --variables
behave identically.
This is PR 1 of a 4-PR stack. Sub-comp per-instance scoping (carrying
host data-variable-values through the inlined sub-comp's getVariables()
call) lands in PR 2; schema validation and lint in PR 3; skill / scaffold
distribution in PR 4.
Tests: 9 new unit tests for getVariables() (jsdom), 11 new CLI tests
covering parseVariablesArg validation paths and Docker passthrough,
2 new dockerRunArgs assertions for the --variables flag. All existing
tests green (core 611, cli 208, engine 519).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
15ee63c6e7 |
fix: harden CLI edge-case repros (#591)
## Problem I reproduced the selected open issue batch one by one and confirmed the reports were valid. The fixes all touch the CLI/runtime capture boundary, then the follow-up regression run exposed one over-broad runtime change in sub-composition host visibility and one CI-only baseline trap. Closes #590, #589, #588, #587, #586, and #584. ## What this fixes ### CLI/runtime edge cases - Makes the GSAP infinite-repeat lint rule ignore JavaScript comments, so literal `repeat:-1` text in comments is not flagged. - Lets the compositions CLI inspect `<template>` content, count visual-only template descendants, estimate simple GSAP durations, and suppress root `data-start` warnings in sub-composition lint mode. - Preserves runtime bootstrap scripts when body scripts are coalesced, and injects the runtime into a real `<head>` when source HTML has no head. - Keeps #589 fixed by loading and rendering template-wrapped sub-composition content, while restoring host visibility to the shorter of the authored parent clip window and the child composition live timeline. - Resolves snapshot/validate viewport size from root `data-width` / `data-height` instead of falling back to 1920x1080. - Skips fully off-frame text boxes during contrast sampling and bounds-checks ring samples so contrast output no longer emits `null:1` / `NaN:1`. - Marks muted videos as `data-has-audio="false"` in the core timing compiler, which fixes the same-src muted `<video>` + separate `<audio>` StaticGuard case. - Keeps user-authored `hf-seek` listeners reachable during capture by preventing author scripts from being merged into the runtime bootstrap path. ### Shared helper cleanup - Removes the stale producer-local timing compiler duplicate; producer compilation now consumes the core timing compiler. - Centralizes HTML document helpers in core: fragment parsing, embedded runtime stripping, head/body script injection, and early-head injection. - Centralizes the CLI layout/snapshot static HTML server. - Adds browser-safe core subpath helpers for Lottie readiness and CLI screenshot clip calculation; Studio's Vite config keeps the screenshot clip helper self-contained so clean-checkout test startup does not value-import core `.ts` source. - Replaces the engine parity-contract copy with a core re-export. - De-duplicates render-job cleanup and Studio static file-serving callbacks. ### Regression hardening - Replaces the embedded-runtime script stripping regex with a script-tag scanner that handles closing tags like `</script >`. - Escapes inline script bodies before wrapping them in `<script>` tags, so authored `</script` and `<!--` text cannot break out of the injected wrapper script. - Shares media-duration clamping between core and producer, with a 50 ms tolerance for ffprobe precision drift between local and CI media stacks. - Pins the affected style fixture SFX durations in source so style-1 and style-9 compile deterministically. - Restores the `vfr-screen-recording` video golden to the CI-stable baseline; the current CI failure showed the Linux render matches the old golden, while the locally refreshed macOS golden was the mismatch. ## Root cause The CLI paths had accumulated assumptions that held for simple direct-root landscape compositions but not for current composition patterns: DOM queries did not enter template content, snapshot/validate used a fixed viewport, runtime and author scripts shared a coalescing bucket, and timing compilation treated every video as audio-bearing unless authors manually overrode it. The style shard failures were not product regressions. Local and CI media probing disagreed on the short SFX clip duration by about 45 ms, and the compiler was clamping authored durations to the locally probed value. The shared clamp tolerance preserves explicit author/source durations for small probe precision differences while still clamping real overflows. The vfr fast-shard failure was a bad baseline refresh: CI actual frames matched the old `vfr-screen-recording` baseline at 40+ dB PSNR, but mismatched the macOS-refreshed golden at ~18-22 dB. The fix is to keep the Docker/Linux-stable video golden and only retain the deterministic compiled snapshot change. The sub-composition regression came from treating a host's authored parent window as the only visibility boundary. That made settled child overlays stay visible after their own live GSAP timeline ended. The corrected runtime behavior respects both contracts: parent clips still bound where the host can appear, and the child live timeline can end the host earlier. ## Verification ### Local checks - `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bun run --cwd packages/core test src/runtime/init.test.ts` - `bun run --cwd packages/cli test src/commands/compositions.test.ts src/utils/compositionViewport.test.ts` - `bun run build:hyperframes-runtime` - `bun run --cwd packages/producer test --keep-temp --sequential style-12-prod style-5-prod` - `bun run --cwd packages/producer test --sequential vfr-screen-recording hdr-hlg-regression style-7-prod` - `bun run --cwd packages/core test src/compiler/htmlCompiler.test.ts src/compiler/timingCompiler.test.ts src/index.test.ts` - `bunx oxfmt --check packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bunx oxlint packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bun run --cwd packages/core typecheck` - `bun run --cwd packages/producer typecheck` - `bun run --cwd packages/producer test --sequential style-1-prod style-9-prod` - `bun run --filter @hyperframes/studio test` with `packages/core/dist` temporarily hidden to simulate clean-checkout config loading - `git diff --check` ### CI artifact checks - Inspected failed run `25225854394` job `73969147096`: style-1 failed only on `click-sfx` `1.044898` vs `1` duration/end. - Inspected failed run `25225854394` job `73969147061`: style-9 failed only on SFX `1.044898`-based duration/end mismatches. - Inspected failed run `25225854394` job `73969147048`: `vfr-screen-recording` compilation/audio passed, visual failed after comparing against the macOS-refreshed golden. - Compared the first 10 uploaded CI vfr failure frames against the restored old baseline; minimum PSNR was `40.444705`, above the fixture threshold of `28`. ### Repro checks - `bun packages/cli/src/cli.ts lint /tmp/hf-590-repro` now passes without `gsap_infinite_repeat`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-587-repro --at 0.5 --timeout 1000` now writes a 1080x1920 PNG. - `bun packages/cli/src/cli.ts validate /tmp/hf-588-repro --timeout 500` no longer emits `null:1` / `NaN:1` contrast output. - `bun packages/cli/src/cli.ts validate /tmp/hf-586-repro --timeout 500 --contrast false` no longer emits the muted-video StaticGuard contract error. - `bun packages/cli/src/cli.ts compositions /tmp/hf-589-gsap-repro` now reports `foo 0.5s 1920x1080 1 element`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-589-gsap-repro --at 0.25 --timeout 2000` captures the expected template-backed red frame. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-584-repro --at 0.5,1.5 --timeout 500` captures the expected post-seek green frame. ### Browser verification - Refreshed the local side-by-side comparison page at `qa-artifacts/pr-591-video-compare/index.html`. - Served the comparison page locally and used `agent-browser` to load `style-12-prod`, play both videos quickly to the failed window, pause, and inspect the side-by-side frame. - Browser proof screenshot: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12-labeled.png`. - Browser proof recording: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12.webm`. - Earlier Studio proof artifacts remain local-only: `qa-artifacts/dedupe-refactor-preview.png`, `qa-artifacts/dedupe-refactor-preview-after-play.png`, `qa-artifacts/dedupe-refactor-preview.webm`. ## Notes - Browser proof and CI diagnostic artifacts are intentionally local-only and not committed. - Studio's Vite config intentionally keeps the thumbnail clip helper inline because Vite/Vitest config startup runs through Node's loader before package source `.ts` imports are transformed. - The committed PR diff changes `vfr-screen-recording/output/compiled.html` but no longer changes `vfr-screen-recording/output/output.mp4` relative to `main`. - I attempted a local `linux/amd64` Docker validation to mirror CI, but the local Docker build was blocked by Debian package download failures. The arm64 Docker image also cannot launch the x64 Puppeteer headless shell under OrbStack. The vfr baseline decision is therefore based on the uploaded CI artifact comparison above. - I kept this validated issue batch in one PR because the fixes overlap the same CLI/runtime capture surfaces. |
||
|
Miguel Ángel
|
39b3997c78 |
fix(studio): warn on anonymous timeline clips (#533)
## Problem Studio timeline editing still had two rough edges that made the latest alpha feel less polished when testing it like a video editor would: - Timeline clips for anonymous DOM nodes could surface internal fallback identities like `__node__index_*`, which made the timeline look broken instead of authored. - Elements without a stable `id` could still appear in the timeline and canvas editor, but authors did not get direct lint guidance that those elements are weaker targets for Studio and agent edits. ## What this fixes - Adds a non-blocking `studio_missing_editable_id` lint warning for timeline-visible elements that do not have an `id`. - Makes the warning point to the exact element and recommend stable, human-readable ids such as `hero-title` or `scene-1-card`. - Stops using synthetic node-index ids as runtime clip identity for anonymous DOM nodes. - Gives anonymous clips readable labels from authored metadata, composition ids, DOM ids, class names, asset filenames, text content, or a simple ordinal fallback. - Keeps those labels display-only in Studio and uses key-first identity for matching, dragging, resizing, and manifest merge preservation. - Covers the duplicate-label case where two anonymous clips both render as `Card` but still stay separate timeline entries. ## Root cause The runtime manifest used synthetic node-index ids as both identity and display fallback for timeline nodes that had no stable author-provided id. Studio then treated those internal values as user-facing clip names. The first pass improved the display label, but it also risked using that friendly label as internal identity. Two anonymous clips with the same label could then collapse into the same logical timeline element. The fix separates display labels from internal identity and prefers the timeline key whenever Studio needs to match an element. The linter also had correctness checks for render and runtime behavior, but it did not teach authors when a timeline-visible element would be harder for Studio and agents to patch reliably. That left missing ids as a silent authoring quality issue instead of actionable guidance. ## Verification ### Local checks - `bun run --cwd packages/core test -- src/lint/rules/core.test.ts src/runtime/timeline.test.ts` -> 41 tests pass - `bun run --cwd packages/studio test -- src/player/hooks/useTimelinePlayer.test.ts src/player/components/timelineTheme.test.ts` -> 23 tests pass - `bun run --cwd packages/core typecheck` - `bun run --cwd packages/studio typecheck` - `bun run --cwd packages/studio build` -> passes with the existing Vite chunk-size warning - `bunx oxlint $(git diff --name-only origin/main...HEAD)` -> 0 warnings, 0 errors - `bunx oxfmt --check $(git diff --name-only origin/main...HEAD)` - `git diff --check origin/main...HEAD` ### Browser verification - Created a scratch project at `/tmp/hf-pr533-conflict-verify` with two timed anonymous `.card` clips that both label as `Card`. - Started the local Studio dev server for `pr-533-conflict-verify`. - Used `agent-browser` to verify the timeline renders two separate `Card` clips instead of collapsing duplicate anonymous labels. - Used `agent-browser` to open the Studio lint modal and verify it shows human-readable missing-id warnings, not internal node-index labels. - Used `agent-browser` to click Play after the lint pass and confirm the timeline remains usable. - Recorded the tested Studio flow with `agent-browser`. ## Notes - Rebased onto current `main`; conflict resolution preserved both the newer mainline Studio shortcut/lint behavior and this PR's anonymous-clip identity split. - GitHub Actions are running on the rebased head. - Scratch verification files are intentionally not committed. - Local screenshots and recording from this rebase pass are under `.codex-artifacts/pr-533-conflict-rebase-2026-04-29/`. |
||
|
Miguel Ángel
|
f9d38a1542 |
feat(core): add anime.js runtime adapter (#569)
## Summary - Adds a `RuntimeDeterministicAdapter` for anime.js v4+ alongside existing Lottie, Three.js, WAAPI, and CSS adapters - Enables frame-accurate rendering of anime.js animations — the adapter converts seek time (seconds) to milliseconds and calls `.seek(timeMs)` on registered instances - Auto-discovers running instances via `anime.running`; compositions can also register manually via `window.__hfAnime` ## Usage in compositions ```html <script src="https://cdn.jsdelivr.net/npm/animejs@4.0.2/lib/anime.iife.min.js"></script> <script> const anim = anime({ targets: '.box', translateX: 250, rotate: '1turn', duration: 2000, autoplay: false, }); window.__hfAnime = window.__hfAnime || []; window.__hfAnime.push(anim); </script> ``` ## Files changed - `packages/core/src/runtime/adapters/animejs.ts` — adapter implementation - `packages/core/src/runtime/adapters/animejs.test.ts` — 15 unit tests - `packages/core/src/runtime/init.ts` — register adapter in runtime init - `packages/core/src/runtime/window.d.ts` — add `anime` and `__hfAnime` globals ## Test plan - [x] All 15 unit tests pass (`bun run --cwd packages/core test -- --run adapters/animejs`) - [x] Build passes (`bun run build`) - [x] Pre-commit hooks pass (lint, format, typecheck, commitlint) - [x] Manual test: render a composition using anime.js animations |
||
|
Miguel Ángel
|
78d949b844 | fix(studio): keep gsap timeline clips stable | ||
|
Miguel Ángel
|
3701bb9eeb | fix: flatten inlined composition roots | ||
|
Miguel Ángel
|
c32db3ea0a | fix: isolate studio sub-composition previews | ||
|
Miguel Ángel
|
e8c43f0889 |
fix: prevent nested composition videos from autoplaying on seek (#477)
## Problem Studio seek could still wake nested composition media even when the transport itself stayed paused. In the real repro from `apple-presentation`, scrubbing to `0:29` without pressing play lands on the `slide-translation` composition. That composition contains `Multilingual_Journey.mp4` inside the composition host. On the broken path: - the main Studio transport remained paused - the nested video advanced and stayed playing anyway - the user saw autoplay-like behavior even though the only action was a seek That was especially confusing because the seek was otherwise correct: the timeline moved to the right point, but the nested media stopped obeying the paused transport state. ## What this fixes ### Nested media now participates in runtime media sync - the runtime media cache no longer assumes only `video[data-start]` / `audio[data-start]` are relevant - nested media inside a composition host can now be included in the same timed-media sync pass even when the inner media element does not carry its own authored `data-start` ### Nested media timing is resolved in the host composition window - nested media start time is resolved against the enclosing composition host instead of falling back to scene-local `0` - nested media duration is clamped to the enclosing composition window so it stays aligned with the authored host clip timing ### Paused seeks land on the right frame and stay paused - after seeking into a nested composition, the inner media is now seeked to the correct frame relative to the host timeline - because it is now part of the managed media set, the runtime also keeps it paused when the transport is paused instead of letting it continue playing on its own ### Regression coverage - adds a runtime regression test that covers a nested composition video with no local `data-start` - the test verifies that `player.seek(29)` leaves the nested video paused while landing it at the expected `currentTime` ## Root cause The bug came from a mismatch between deterministic timeline seeking and media ownership. ### 1. The runtime only managed media with direct timing attrs `refreshRuntimeMediaCache()` only collected `video[data-start]` and `audio[data-start]`. That works for root-level timed media, but not for media embedded inside a composition host where timing is inherited from the host composition rather than duplicated onto the inner media node. ### 2. Nested composition seek could still advance inner media The runtime intentionally rearms sibling timelines during deterministic seek so nested timelines land on the right local offsets. That part is necessary and correct. But because the nested video was not part of the managed media cache, it could advance during that seek path without being brought back under the paused transport state afterward. ### 3. The runtime had no way to reconcile the two So the system had an inconsistent split: - timeline seek knew about the nested composition timeline - media sync did not know about the nested media inside it The fix closes that split by resolving nested media start/duration from the enclosing composition context and running it through the same sync logic as other managed media. ## Verification ### Local checks - `bun run --filter @hyperframes/core typecheck` - `bun run test -- src/runtime/init.test.ts src/runtime/media.test.ts src/runtime/player.test.ts` in `packages/core` - `bunx oxlint packages/core/src/runtime/media.ts packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bunx oxfmt --check packages/core/src/runtime/media.ts packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` ### Browser verification Verified against a repo-backed local Studio preview of `apple-presentation`: - opened `http://127.0.0.1:3014/#project/apple-presentation` - seeked to `0:29` without pressing play - confirmed the visible composition switched to `slide-translation` - confirmed `Multilingual_Journey.mp4` landed at a non-zero `currentTime` (`3.067` in the verified run) - confirmed the nested video stayed `paused` and its `currentTime` remained stable across a follow-up check instead of autoplaying ## Notes - the local browser proof artifacts under `qa-artifacts/autoplay-seek/` are verification-only and are not part of this PR - this PR is intentionally scoped to nested media ownership during paused seek; it does not broaden into unrelated runtime media refactors beyond bringing inherited nested media under the existing sync contract |
||
|
Miguel Ángel
|
970b446c49 |
feat(studio): drag assets from the sidebar onto the timeline (#464)
## Problem Studio still broke down in three concrete authoring flows around timeline assets: - you could import media into Assets, but not drag an already-imported asset from the Assets tab onto the timeline and persist it into source - dragging a file from outside the app onto the timeline only uploaded it into Assets instead of placing it at the dropped time/track - once a clip was on the timeline, there was no reliable keyboard delete flow for removing it safely from source While implementing direct external drops, another real bug showed up: - valid binary uploads like `raycast.mp4` from `Downloads` were being rejected as unsupported media in Studio dev because the Vite API bridge was corrupting multipart request bodies before they reached the upload route ## What this fixes ### Timeline asset placement from inside Studio - asset cards in the Assets tab are draggable - the timeline accepts asset drops even when it already has clips - dropping an asset onto the timeline inserts a new clip into the active composition source at the dropped time / track - asset paths are rewritten relative to the target composition file so drops into sub-compositions resolve correctly - the new clip is persisted immediately and the preview refreshes ### Direct external file drops onto the timeline - dropping a file from outside the app onto the timeline now uploads it and places it onto the dropped track/time in one shot - it no longer stops halfway by only adding the file into Assets - multiple dropped files are placed using the same drop start and successive tracks ### Delete key support - selected timeline clips can now be deleted with `Delete` / `Backspace` - deletion is persisted back to source, not just removed from local state - the delete path now uses a server-side DOM mutation helper with LinkeDOM for structural safety instead of client-side string surgery ### Binary upload fix for media files - the Studio Vite API bridge now forwards non-GET request bodies as raw bytes instead of decoding them as UTF-8 text - that preserves multipart uploads for binary media like MP4s - valid local videos from `Downloads` no longer get rejected as `Unsupported media skipped` just because the dev bridge corrupted the request body - upload validation now probes buffered media through a temp file path that preserves the file extension before saving into the project ## Root cause There were really two separate gaps: ### 1. Asset placement / deletion workflow gaps The timeline and asset systems already existed, but they were disconnected: - `AssetsTab` only supported copy/import flows - `Timeline` only handled raw file import, not positioned placement for existing assets - there was no utility layer for converting a dropped asset into persisted timeline HTML - there was no structurally safe deletion path for arbitrary selected timeline clips ### 2. Binary upload corruption in Studio dev The Studio Vite API bridge rebuilt non-GET request bodies like this: - read each request chunk - call `chunk.toString()` - concatenate into a string - construct the Fetch `Request` from that string body That works for text, but it corrupts multipart binary uploads. By the time the upload route wrote the received file and ran `ffprobe`, otherwise valid MP4s had already been mangled in-flight. ## Behavior - dropping on `index.html` inserts the asset into the root composition - dropping while drilled into a composition inserts into that composition file instead - drop X position maps to `data-start` - drop Y position maps to the current visible track row, with a new bottom track created if the drop lands below existing rows - images default to a short finite duration - audio/video default to their metadata duration when available, with a fallback duration if metadata cannot be read quickly - pressing `Delete` on a selected clip removes that clip from the underlying HTML source and clears selection in Studio - valid uploaded MP4s now survive the Studio dev API bridge intact instead of being rejected during upload validation ## Verification ### Local checks - `bunx oxlint packages/core/src/studio-api/helpers/sourceMutation.ts packages/core/src/studio-api/helpers/sourceMutation.test.ts packages/core/src/studio-api/helpers/mediaValidation.ts packages/core/src/studio-api/helpers/mediaValidation.test.ts packages/core/src/studio-api/routes/files.ts packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/sidebar/AssetsTab.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/Timeline.test.ts packages/studio/src/utils/timelineAssetDrop.ts packages/studio/src/utils/timelineAssetDrop.test.ts packages/studio/vite.config.ts packages/studio/vite.request-body.ts packages/studio/vite.request-body.test.ts` - `bunx oxfmt --check` on the touched files - `bun run --filter @hyperframes/core typecheck` - `bun run --filter @hyperframes/studio typecheck` - `bun test packages/core/src/studio-api/helpers/sourceMutation.test.ts packages/core/src/studio-api/helpers/mediaValidation.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/utils/timelineAssetDrop.test.ts packages/studio/vite.request-body.test.ts` ### Browser / live verification Verified against a live local Studio fixture: - dragging an existing asset from the Assets tab onto the timeline creates a persisted clip at the dropped position - dropping a file from outside the app directly onto the timeline uploads it and creates a persisted clip at the dropped position - selecting a dropped clip and pressing `Delete` removes it from both the live timeline and the saved source HTML - valid MP4 uploads like `raycast.mp4` now succeed through the live Studio upload route instead of being rejected as unsupported media ## Notes - the local `timeline-trio-verify` and `timeline-overlap-debug` projects used for verification are local-only and are not part of this PR - this PR is about asset placement, upload correctness, and deletion safety; it does not broaden into richer editing workflows beyond placing/removing clips from the timeline |
||
|
Miguel Ángel
|
6610b8ad00 |
fix: harden studio timeline editing and local renders (#463)
* fix: harden studio timeline editing and local renders * test: cover studio local render fallback * fix(studio): scale composition hover previews to stage size * test: normalize studio producer fallback paths * fix(studio): preserve move surface and retry render fallback |
||
|
Miguel Ángel
|
95bf333895 |
fix: stabilize apple master timeline and playback (#419)
## Summary - preserve authored non-root composition timing before runtime sanitization so Studio can build the correct master timeline for chained subcompositions - prefer the fresh runtime source in Studio dev so local preview does not serve a stale `/api/runtime.js` - restrict preserved authored timing inference to the Studio timeline payload instead of the general runtime resolver ## What this fixes This PR fixes the Apple presentation class of failures where the root `index.html` / `Master` view looked correct at first and then collapsed into an incorrect short timeline. Before this change: - the master transport could report a short duration like `0:12` instead of the real deck length (`2:21` in the Apple project) - composition clips bunched near the start instead of laying out sequentially across the deck - seeking into later parts of the deck would land in the wrong place or show the wrong active composition - local Studio debugging could be misleading because dev sometimes served a stale runtime bundle After this change: - the master transport reflects the authored composition-chain duration - master clips resolve linearly across the whole deck - late seeks land on the correct slide window - Studio dev uses the current runtime implementation, so local preview matches the branch you are testing ## Root cause There were two related issues: 1. Studio/master timeline inference lost authored composition timing - missing timing attrs were treated like `0` instead of `null` - non-root composition `data-duration` / `data-end` were stripped before Studio timing resolution could use them - root duration inference trusted an incomplete live timeline window instead of the authored composition chain 2. Preserved authored timing leaked into the general runtime resolver - preserving authored timing was correct for Studio timeline payload generation - but using those preserved attrs for normal runtime playback/render resolution caused visual regressions in producer CI - the follow-up fix keeps authored timing available only for Studio payload collection while normal runtime playback continues to resolve from the real live timeline/media state ## Why the later regression fix was needed The initial runtime change fixed the Apple master timeline, but it also widened timing inference in the core runtime too far. That caused Dockerized producer regressions because rendered visibility started respecting preserved authored timing where it should have relied on the live resolved runtime state. The latest commit fixes that by splitting the behavior: - Studio timeline payload: authored timing allowed - general runtime resolver: authored timing ignored by default That preserves the Apple master timeline fix without changing producer render semantics. ## Verification ### Local checks - `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/startResolver.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/startResolver.test.ts packages/core/src/runtime/timeline.test.ts packages/studio/vite.config.ts packages/cli/src/server/studioServer.ts` - `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/startResolver.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/startResolver.test.ts packages/core/src/runtime/timeline.test.ts packages/studio/vite.config.ts packages/cli/src/server/studioServer.ts` - `bun run --filter @hyperframes/core typecheck` - `bun run --filter @hyperframes/studio typecheck` - `bun run --filter @hyperframes/cli typecheck` - `cd packages/core && bun run test src/runtime/startResolver.test.ts src/runtime/timeline.test.ts` - `bun test packages/cli/src/server/studioServer.test.ts --timeout 20000` ### Browser proof Tested in Studio with `agent-browser` against the Apple presentation project. - root/master transport now shows `0:00 / 2:21` - master clip manifest resolves sequentially (`slide-1 -> slide-2 -> slide-3 ...`) - seeking to `120s` lands on a late slide instead of a collapsed early timeline state - after refreshing onto the fresh runtime source, the visible later-slide media advanced correctly in local Studio playback ### CI-equivalent regression proof on devbox The previously failing producer regressions were rerun on devbox using the same Dockerized path GitHub Actions uses: - `docker build -f Dockerfile.test -t hyperframes-producer:test .` - `docker run ... hyperframes-producer:test style-1-prod style-5-prod style-9-prod style-12-prod --sequential` Those previously failing suites all passed after the runtime split fix: - `style-1-prod` - `style-5-prod` - `style-9-prod` - `style-12-prod` ## Notes - the Apple project volume tweak stayed local-only for testing and is not part of this PR - this PR fixes the master/root timeline bug and the runtime regression it introduced; it does not add general subtimeline authoring support |
||
|
Vance Ingalls
|
f906797222 |
perf(player): coalesce _mirrorParentMediaTime writes (#396)
## Summary Coalesce writes to `el.currentTime` inside `_mirrorParentMediaTime` so a single jitter sample no longer triggers a parent-media seek. A drift correction now requires **two consecutive samples** above the threshold (~`MIRROR_DRIFT_THRESHOLD_SECONDS`) before the player writes back. One-shot alignment paths (`promoteToParentProxy`, `_onIframeMediaAdded`) opt out via `force: true` so initial alignment stays immediate. ## Why Step `P1-4` of the player perf proposal. `_mirrorParentMediaTime` is called every animation frame on parent media proxies. Even without true drift, browser internals report tiny jitter on `currentTime` reads — typically below 30 ms but occasionally crossing the threshold for a frame. Writing to `currentTime` triggers a seek, which is expensive *and* invalidates pipeline buffers, which causes the next frame's reading to jitter further. The result was unnecessary seek thrash on otherwise-aligned media. By requiring two consecutive over-threshold samples, transient jitter is filtered out while real drift (a sustained offset) still corrects within ~1 frame of latency. This eliminates the most common cause of dropped frames on the studio thumbnail grid. ## What changed - Each `_parentMedia` entry gains a `driftSamples` counter that increments while the absolute drift is above `MIRROR_DRIFT_THRESHOLD_SECONDS` and resets to 0 on the first sample below. - `_mirrorParentMediaTime(el, opts)` only writes back when `driftSamples >= 2`, except when `opts.force === true`. - `promoteToParentProxy` and `_onIframeMediaAdded` pass `force: true` so the first alignment after registration is still immediate (these are user-visible state transitions, not steady-state telemetry). ## Test plan - [x] 11 new unit/integration tests in `hyperframes-player.test.ts` covering: - Single-sample jitter does not trigger a write. - Two-sample sustained drift does trigger a write. - Trending drift correction (gradually increasing offset) is detected within 2 samples. - `force: true` override bypasses the sample requirement. - Out-of-range proxies (proxies whose source has been removed) do not panic. - Multiple proxies maintain independent counters — drift on one does not affect the other. - `_promoteToParentProxy` alignment is immediate. ## Stack Step `P1-4` of the player perf proposal. Builds on `P1-1` (shared adopted stylesheets) and `P1-2` (scoped media observer). Together these three target the studio multi-player render path — `P0-1*` perf gate scenarios will pick up the wins automatically. |
||
|
Vance Ingalls
|
f9863ab565 |
feat(core): add emitPerformanceMetric bridge for runtime telemetry (#393)
## Summary
Extend the runtime analytics bridge with a numeric performance metric channel. Hosts subscribe via the existing postMessage transport (one bridge, two channels) and aggregate per-session p50 / p95 for scrub latency, sustained fps, dropped frames, decoder count, composition load time, and media sync drift before forwarding to their observability pipeline.
This is the foundation other perf tooling sits on — the player itself emits the events; player-side aggregation and flush land in a follow-up.
## Why
Step `X-1` of the player perf proposal. Today there is no way for an embedding host to learn that scrub latency spiked, that a composition took 3 s to load, or that the media-sync loop is running 200 ms behind real time. The only signals are anecdotal user reports.
A single shared bridge keeps the runtime → host surface area minimal: hosts that already wire up the analytics channel get perf for free, and hosts that don't aren't paying for it.
## What changed
- New `emitPerformanceMetric(name, value, tags?)` helper in `@hyperframes/core` that forwards a `{ type: "performance-metric", name, value, tags }` envelope through the existing analytics postMessage transport.
- Six initial metric names defined in the proposal:
- `scrub_latency_ms` — wall-clock from `seek()` call to first paint at the new frame.
- `playback_fps` — sustained rAF cadence during play.
- `dropped_frames` — count of >25 ms gaps within a play window.
- `decoder_count` — number of concurrently-decoding video elements.
- `composition_load_ms` — navigation-start to player-ready.
- `media_sync_drift_ms` — drift between expected and actual decoder time.
- Each emit also writes a `performance.mark()` with `{ value, tags }` on `detail`, so the same numbers surface in the DevTools Performance panel's User Timing track for local debugging without instrumenting the host.
- Zero PostHog (or any other analytics SDK) dependency in `core` — the host decides where to forward the events.
## Test plan
- [x] Unit tests cover the envelope shape, the `performance.mark` mirror, and the no-op path when no host has wired up the bridge.
- [x] Manual: verified marks appear in the User Timing track when scrubbing the studio preview.
## Stack
Step `X-1` of the player perf proposal. Foundation for the perf gate (P0-1a/b/c) — the perf scenarios in this stack instrument these same channels for CI measurement.
|
||
|
Miguel Ángel
|
d740f5ce42 |
fix: nested GSAP sub-composition lint and render handling (#405)
## Summary - allow nested sub-composition files to inherit GSAP from their host without tripping `missing_gsap_script` - keep nested render seeks stable for sub-compositions without regressing producer baselines - stop producer render-hint detection from treating the compiler's own nested mount retry wrapper as user-authored `requestAnimationFrame()` usage ## Root Cause - the core linter treated template-based nested compositions like standalone root compositions, so it incorrectly required a local GSAP loader even when the host composition already provided GSAP - producer `detectRenderModeHints()` runs before CDN scripts are inlined, so nested GSAP exports were never failing because of the GSAP payload itself - the nested-only false positive came from the compiler-generated mount bootstrap that waits for the inlined sub-composition root with `requestAnimationFrame()` before running the hoisted inline script - preview and export seek paths also needed to stay split so the nested timeline re-arm behavior that stabilizes scrubbing does not collapse render baselines ## What Changed - lint: keep the nested GSAP false-positive fix and regression coverage for template sub-compositions - runtime: keep the render-seek behavior that preserves nested child offsets during export without changing preview scrubbing behavior - producer: mark compiler-owned mount bootstrap blocks and strip only those blocks before scanning inline scripts for raw `requestAnimationFrame()` - producer tests now cover both cases: compiler-generated wrappers are ignored, but real user-authored nested `requestAnimationFrame()` still opts into screenshot mode ## Validation - `bun test packages/core/src/lint/rules/gsap.test.ts` - `bun test packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxfmt packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxlint packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts` - `bun run --filter @hyperframes/producer test --sequential chat style-11-prod` - `style-11-prod` passed locally - `chat` still shows local-only visual drift on this macOS/ARM workstation, but the render metadata now reports `renderModeHints.recommendScreenshot=false`, which is the concrete acceptance condition for `#402` - Docker CI-image repro is blocked locally by OrbStack x86/arm64 loader mismatch, so final regression confirmation is deferred to GitHub Actions Closes #392 Closes #402 |
||
|
Miguel Ángel
|
d38a4f19fd | fix(core): restore nested seek scrubbing (#404) | ||
|
Miguel Ángel
|
1aea1415c4 |
fix: smooth scrubber end seeking (#386)
* fix: smooth scrubber end seeking * fix: stop timeline auto-scroll in fit mode * feat: use percentage-based timeline zoom * fix: sync timeline playhead on zoom changes * fix: reset timeline scroll when returning to fit * fix: keep timeline controls pinned |
||
|
Miguel Ángel
|
158204343d |
fix: stabilize studio preview and runtime sync (#389)
## Summary Stabilize the Studio preview/runtime path so timeline data, preview rendering, and thumbnails stay in sync. This PR includes: - preview hot-refresh without remounting the iframe - runtime duration/timeline fixes so Studio stops drifting from playback state - thumbnail and selector-based preview fixes - local Studio runtime serving and player-resolution fixes so dev/CI do not depend on prebuilt player artifacts - tests around preview identity and thumbnail/runtime behavior ## Why This PR Exists This is the foundation layer for timeline editing. Without it, the editor was prone to: - iframe remount flashes after saves - duration mismatches between preview and timeline - stale or incorrect thumbnails - CI/test failures when `@hyperframes/player` artifacts were not prebuilt ## Verification - `bun run --filter @hyperframes/studio test` - `bun run --filter @hyperframes/studio typecheck` - `bun run --filter @hyperframes/core typecheck` - `bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts` - `bunx oxfmt --check packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts` ## Stack - base of stack - followed by `feat: add studio timeline editing` - followed by `fix: smooth scrubber end seeking` |
||
|
James RussoandClaude Opus 4.7
|
e72bcfaed3 |
fix(player+core): correctly render and pause nested compositions (#359)
* fix(player): inject runtime immediately for nested compositions Compositions that use `data-composition-src` on child elements require the HyperFrames runtime to load those scenes — there is no way for the iframe to render without it. The existing probe loop delayed runtime injection behind a 5-tick attempts gate so the adapter path could try to resolve a timeline first. For nested compositions that race lost: a composition like the `product-promo` registry example registers an inline pre-runtime GSAP timeline at `window.__timelines["main"]` (covering only a partial duration, e.g. 14s of a 20s master) while the iframe document loads. The probe's adapter check finds that timeline and locks the player into a "ready" state against it — which short-circuits the attempts gate and the runtime never gets injected. The iframe ends up blank because the runtime is what would have loaded the child scenes via `data-composition-src`. This change splits the injection decision into a pure helper, `shouldInjectRuntime(state)`, and treats nested compositions as "inject immediately, skip the gate." Self-contained GSAP-only compositions retain the 5-tick grace period so the adapter path keeps first shot for them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(core): propagate play/pause to all sibling timelines Pausing or playing the master timeline only called `.pause()` / `.play()` on `state.capturedTimeline` — the single adapter-selected timeline. In a nested composition (a master with `data-composition-src` children), each scene's own timeline is registered as a sibling in `window.__timelines`, so they would keep advancing after the user clicked pause. The player UI froze at the paused time while the visual content continued to animate, eventually finishing all scene-level animations and landing on an empty end-state. Wire `window.__timelines` into the runtime player via a new `getTimelineRegistry` dep, iterate the registry on play/pause, and forward `timeScale` to siblings when play() starts so a changed playback-rate applies uniformly. Covered by 7 new unit tests in player.test.ts, including the identity- equality check (don't double-invoke the master), playbackRate propagation, a broken-sibling swallow, and a back-compat case with no registry supplied. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
ad11de698c |
feat(producer): auto-fallback screenshot capture for raf and iframes (#331)
* fix(core): drive adapter seeks when composition has no GSAP timeline renderSeek returned early when deps.getTimeline() was null, skipping the onDeterministicSeek call that drives all frame adapters (CSS, WAAPI, Lottie, Three.js). That meant compositions using any non-GSAP animation primitive froze on their initial frame during capture. Now we still quantize the seek time and fire onDeterministicSeek even without a timeline, so each adapter gets a chance to advance. GSAP compositions are unaffected — timeline-driven seek still takes the same path it did before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(producer): auto-fallback screenshot capture for raf and iframes Co-Authored-By: Codex <codex@openai.com> * test(producer): add render compatibility regression fixtures Co-Authored-By: Codex <codex@openai.com> * fix(core): scrub CSS animations via WAAPI currentTime Co-Authored-By: Codex <codex@openai.com> * test(producer): cover css keyframe renders Co-Authored-By: Codex <codex@openai.com> * fix(producer): propagate virtual time into iframe documents Co-Authored-By: Codex <codex@openai.com> * test(producer): refresh iframe docker golden Co-Authored-By: Codex <codex@openai.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Codex <codex@openai.com> |
||
|
Miguel Ángel
|
c49181f1fa |
fix(player): address #298 review — tighter drift, dynamic proxies, ownership event (#307)
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment. ## Significant ### 1\. Drift threshold 150 ms → 50 ms _mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity. **Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction. ### 2\. Dynamic sub-composition media proxies Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track. Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect. ### 3\. `bridgeMuted` sticky in `syncRuntimeMedia` The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message. `syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs. ### 4\. Reset `_audioOwner` on iframe reload The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently). `_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`. ## Worth addressing ### 5\. Promotion → observable event + reason Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state. **Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once. ### 6\. Parent proxy play() rejection → `playbackerror` event Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back. ### 7\. Mobile verification on real hardware Tested with a tunnel in a real iOS device. ## Test gaps (from review) - `userMuted` stickiness (mirror of the existing `outputMuted` test). - **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`. - **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic. - **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once. - **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote). - **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately. - **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag. ## Minor - One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear). ## What's good (from the review) Kept as-is — noted for posterity: - `muted` vs `volume` framing (orthogonal channels). - Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing. - Two orthogonal mute channels. - Backwards compat (new actions / messages safely ignored by either side). ## Test results - `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe) - `packages/core/src/runtime/bridge.test.ts` — **15 tests pass** - `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface) - Typecheck green on `core` + `player` - `tsup` build green on `core` / `player` / `cli` - Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold. ## Test plan - [x] Unit tests (83 total across touched files) - [x] Typecheck clean - [x] Build clean - [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened - [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block - [x] Physical iOS / Android device verification (unchanged since #298) |
||
|
Miguel Ángel
|
3256551a5e |
fix(player): single-owner audio to prevent double voice in preview (#298)
## Summary Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip). ## Root cause Two audio pipelines were playing the same source in parallel: 1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path. 2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message. The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks. Confirmed via agent-browser instrumentation on `factory-series-c-video`: - 6 `volumechange` events per play cycle (mute-fight signature) - Both copies audible at `volume=1`, offset growing 23ms → 80ms - Every seek widened the drift further PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback. ## Fix Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime. - **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash. - **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold. Two orthogonal mute channels replace the volume fight: | Channel | Purpose | |---|---| | `set-muted` | User's mute preference (existing, unchanged) | | `set-media-output-muted` | Internal ownership handoff (new) | `syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it. ## Why this shape - **Single owner, explicit transition.** No races, no tug-of-war. - **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing. - **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact. - **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift. - **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior. - **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files. ## Files changed - `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types. - `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields. - `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`. - `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`. - `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session. - `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership. ## Verified end-to-end with agent-browser on `factory-series-c-video` **Runtime ownership (default — desktop studio):** | | Before | After | |---|---|---| | `PARENT.play()` calls per play cycle | 1 | **0** | | iframe `volumechange` events | 6 | **0** | | Audible streams | 2 (drifting) | **1 (iframe)** | **Parent ownership (simulated autoplay block — direct message):** | | Value | |---|---| | iframe audio | `muted=true`, `volume=1` (untouched) | | parent audio | `muted=false`, `volume=1`, audible | | Parent ↔ iframe `currentTime` offset | ~6 ms steady state | | Offset > 150 ms | corrected by mirror sync | **Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:** Event timeline captured via agent-browser instrumentation: ``` t=0.0 ms IFRAME.play() called ← runtime attempts playback t=0.4 ms IFRAME.play() REJECTED: NotAllowedError ← simulated mobile gate t=0.4 ms →IFRAME bridge set-media-output-muted=true ← player promotes t=0.6 ms PARENT.play() called ← parent proxy starts t=0.8 ms ←IFRAME msg media-autoplay-blocked ← runtime signal t=1.0 ms PARENT.play() resolved ← audible t=1.3 ms IFRAME muted=true, volume=1 ← iframe silenced via native muted ``` Steady state at t=4 s under promoted parent ownership: | Element | currentTime | paused | volume | muted | |---|---|---|---|---| | Parent audio | 4.060 s | false | 1.0 | **false** (audible) | | Iframe audio | 4.068 s | false | 1.0 | **true** (silent) | **Offset: 8 ms**, single audible stream, orthogonal mute channel respected. ## Test plan - [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`) - [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow) - [x] `bun run build` — all packages green - [x] Fresh preview + browser repro on `factory-series-c-video`: - [x] Runtime ownership: single audio stream, no drift - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset - [x] No `volumechange` thrash in either ownership mode - [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above) |
||
|
Miguel Ángel
|
f1a37400b4 |
fix(runtime): silent-first-play + loading overlay for preview (#293)
## Summary Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit. ### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`) `syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window): - `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click." - The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged. `HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`. ### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`) The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s. ### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`) Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration. Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR. ### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`) Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync. ## Verification - 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset. - Full monorepo build green (`bun run build`), typecheck clean, lint/format clean. - End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass: | Scenario | Metric | Result | | --- | --- | --- | | Normal first play | Audio plays from click, smooth progression | ✅ | | Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip | ✅ | | Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) | ✅ | | Scrub mid-playback | Lands exactly at target frame | ✅ | ## Files changed - `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve. - `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset. - `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes. - `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check. ## Follow-ups (deferred) - Tight-threshold short-window drift correction for lip-synced dialogue. - A perf-regression test that fails on `waiting`\-event resurgence. ## Test plan - [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words. - [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events). - [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers. - [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame. |