mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-07 10:06:21 +00:00
03c47dbb37d7f8d046ae78fbf643d5f261e25b9d
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Angel Simon Sierra
|
9f6c20e482 |
perf(studio): grow composition duration live on extend, no preview remount
Extending a clip past the video end used to force the server-fallback path that fully remounts the preview iframe (the SDK fast path can't express the root composition's data-duration, and the runtime bakes+drops data-duration at load so it can't be patched live). On a large comp that remount is a visible hitch. Add a runtime control-bridge action set-root-duration -> clock.setDuration, so the studio can grow the transport length in place. On an extend the studio now posts it (and patches the clip's own timing live) instead of reloading; it only reloads when a GSAP source rewrite actually happened (the gsap-mutation endpoints now report a mutated flag). Non-animated extends — the common case — commit as fast as a normal edit. Verified: bridge dispatch + studio no-reload/post-message paths unit- tested; core/studio/studio-server typecheck + suites green; the built runtime artifact carries the handler; E2E confirms the extend no longer remounts the preview and still persists. |
||
|
Vance Ingalls
|
341e65aea2 | fix(slideshow): harden media controls in present decks (#1619) | ||
|
Vance IngallsandClaude Opus 4.7
|
f0c4dee705 |
fix(slideshow): present media controls (#1601)
* fix(slideshow): harden media controls in present decks
* refactor(slideshow): clear Fallow audit findings
Decompose flagged high-CRAP functions and extract production-code
duplications so the audit gate clears.
- core/runtime/bridge.ts handler — replace the 14-branch if-chain with a
CONTROL_HANDLERS dispatch table; flash-elements payload handling moves
to its own helper. Behavior preserved (all existing bridge.test.ts
cases hit the same dispatchers via the public installRuntimeControlBridge
API).
- player/slideshow/SlideshowController syncTo — split into
isValidSyncTarget / isCrossSlide / rerootStackTo helpers. The
stopSlideMedia decision and the stack re-rooting are now individually
named; the public method is a 4-line orchestrator.
- cli/commands/validate.ts run — extract emitJsonReport / emitTextReport
so the orchestrator no longer carries the dual JSON/text branches.
Cuts the cyclomatic complexity flagged by fallow after the
shouldIgnoreRequestFailure signature expansion shifted the fingerprint.
- player/hyperframes-player.ts — _setIframeMediaMuted and _stopIframeMedia
shared a `try { iframeDoc = contentDocument } catch { return }` preamble
(clone group 15). Extract _getSameOriginIframeDocument(): Document | null
and have both call sites consume it.
- studio/panels/SlideshowPanel.tsx — the notes controller's debounce-tail
and explicit flush() shared the pending-drain pattern (clone group 16).
Extract a drainPending() closure both call.
- player/hyperframes-player.test.ts — collapse the new stopMedia / muted
tests' repeated Object.defineProperty(iframe, "contentDocument", { get })
shape behind a stubIframeContentDocument helper.
No behavior changes — refactor only. Existing tests cover the affected
paths unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(validate): split run further; ignore test dup parity
Second Fallow pass surfaced two minor follow-ups after the first cut:
- packages/cli/src/commands/validate.ts run + emitTextReport still
carried minor CRAP findings (43.1 / 37.1, threshold 30). Extract
printValidationResult / formatConsoleEntry / formatTotals /
emitFailureReport so run becomes a try/catch + delegation, well
below the threshold; emitTextReport drops the inline format loops.
- .fallowrc.jsonc duplicates.ignore: add hyperframes-player.test.ts
alongside the existing SlideshowPanel.test.ts entry. Same reasoning
documented there — parallel arrange/act/assert test cases are
intentionally self-contained for readability; collapsing them under
shared fixtures would couple unrelated scenarios (same-origin vs
realm media, audio-locked permutations, seek bridge variants).
No behavior changes — refactor + config-policy parity only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
ukimsanov
|
48fb42e5f0 | refactor(runtime): trim color grading helpers | ||
|
ukimsanov
|
1ad158d2f0 | feat(runtime): apply color grading in preview and render | ||
|
xiayeandClaude Opus 4.7
|
5a0966dfeb |
fix(runtime,player): replay bridge state on iframe ready to repair race
The audio-locked attribute was correctly setting `muted = true` and posting `set-muted` to the iframe runtime, but on warm-cache reloads of claude.ai and inside the Claude desktop Electron client, the iframe finishes loading *after* the parent has already sent control messages — the iframe runtime's postMessage listener isn't installed yet, so the messages are silently dropped. Audio plays unmuted with no UI to recover. Confirmed via: - "First open" on claude.ai: cold cache, iframe slow → listener up before `set-muted` lands → audio muted ✅ - "Hard refresh" on claude.ai: warm cache, iframe fast → listener up after message arrives → message lost → audio plays ❌ - Claude desktop: Electron renderer consistently fast → race always loses → audio plays ❌ Fix: add a `{source: "hf-preview", type: "ready"}` event the runtime emits once `installRuntimeControlBridge` has registered the listener. The player listens for it and replays current bridge state (`set-muted`, `set-volume`, `set-playback-rate`). Pre-ready messages are now safe to send — they'll be replayed once the runtime can receive them. The replay is idempotent — re-asserting defaults is a no-op — so it's also safe across iframe reloads (new runtime instance emits ready again). Tests: 6 new (1 bridge: ready posted on install; 5 player: replays muted / volume / playback-rate / audio-locked-forced-mute / handles second ready / ignores ready from wrong source). Suites green: core 1387, player 137. Refs: - Investigation: heygen-com/hyperframes#1300 (UA-fallback attempt — unrelated to actual root cause) - claude.ai-web.log analysis revealed cross-origin iframe + race condition, not attribute stripping as originally hypothesized 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
terencechoandClaude Sonnet 4.6
|
c08e8b2322 |
fix(player): drive composition ticks from widget-frame rAF via postMessage (#805)
Chromium throttles requestAnimationFrame in deeply nested cross-origin iframes. In Claude desktop (Electron), the composition iframe's own rAF loop stalls, so GSAP is never seeked and animation freezes even when TransportClock.isPlaying() is true. The correct fix is to drive ticks from the widget-frame rAF, which lives one level up and is not subject to the same throttling. When play() takes the runtime bridge path (no direct timeline adapter), the player now starts a parent-frame rAF loop that sends "tick" postMessages to the composition iframe on every frame. The runtime's control bridge handles "tick" by calling seekTimelineAndAdapters(clock.now()) if the clock is playing — identical to what transportTick does on each rAF, just driven from outside. The composition iframe's own rAF loop is unchanged and keeps running normally in standard browsers. Seeking GSAP twice per frame is idempotent, so there is no regression on claude.ai or any other non-throttled environment. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
James
|
e87196456b |
feat(runtime): swallow() helper replaces empty catches in inlined runtime
After hf#641 inlined the runtime IIFE into every bundle, lint tools
inspecting bundled output (including Abhay's c2v eval) started flagging
empty `catch {}` blocks across the runtime. The source had explanatory
comments inside, but esbuild's minifier strips them — the IIFE ships
~10 visible patterns of `}catch{}` and consumers' linters fire on each.
Each empty catch is intentional best-effort error swallowing —
postMessage to a parent frame that may not exist, `media.play()` /
`pause()` that throw under autoplay restrictions, timeline `seek()` on
a disposed timeline, anime.js / lottie feature detection on hosts that
don't load those libraries, etc. The right behaviour stays "tried,
didn't work, move on", but doing it visibly improves three things:
- lint clean: helper call is a real statement; no `no-empty` warnings
survive minification
- debuggable: flip `window.__hfDebug = true` in DevTools to see every
swallow site with `console.debug` (silent in prod by default)
- observable: studio / embeddings can install
`window.__hf.onSwallowed = handler` to collect runtime swallow
events without polluting the page console
Implementation: `packages/core/src/runtime/diagnostics.ts` exports
`swallow(label, err?)`. 41 catch sites across 12 runtime files
converted via mechanical pass (auto-generated `runtime.<module>.siteN`
labels — labels can be tightened site-by-site as a follow-up; the
shape of the change is what matters here).
Verification:
- core 674/674 (incl. 6 new diagnostics tests covering silent default,
__hfDebug logging, legacy __HYPERFRAMES_DEBUG flag, handler hook,
handler-throws-doesn't-recurse, both-active)
- typecheck clean
- format / lint clean
- runtime IIFE rebuilds successfully (`bun run build:hyperframes-runtime`)
Refs Abhay's c2v eval — bundler artefacts now lint-clean with the
runtime body inlined.
|
||
|
Miguel ÁngelandClaude Opus 4.6
|
a7a6648852 |
feat(player): add volume/mute controls (#651)
* feat(player): add volume/mute controls to the player Adds a mute toggle button and volume slider to the controls bar, positioned between the time display and speed selector. The slider expands on hover for a compact default footprint. - `volume` attribute/property (0–1, clamped) with `volumechange` event - `muted` attribute now syncs to the controls UI (icon updates) - Three volume icons: high, low, muted — updates reactively - Volume forwarded to parent-frame audio proxies and iframe runtime via `set-volume` postMessage control - 9 new tests covering volume clamping, events, controls rendering, mute toggle, and iframe message forwarding Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(player): wire set-volume through runtime bridge + address review feedback Addresses the blocker from PR review: the iframe runtime now handles `set-volume` messages end-to-end (types → bridge → init → media sync). Runtime side: - Add `set-volume` to RuntimeBridgeControlAction union - Add `volume` field to RuntimeBridgeControlMessage - Handle `set-volume` in bridge.ts with [0,1] clamping - Store bridgeVolume in RuntimeState, apply to media elements - syncRuntimeMedia composes userVolume × clip author volume Player side: - Muted toggle now dispatches `volumechange` (HTML5 spec compliance) - Volume slider auto-unmutes when scrubbed above 0 while muted - Touch support on volume slider (touchstart/move/end) Tests: 5 new (3 bridge, 2 media) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(player): add ARIA keyboard controls, fix icon collision and clipVolume parity - Volume slider: role="slider", aria-label, aria-valuemin/max/now, tabindex=0, arrow key support (5% steps, auto-unmutes) - Volume=0 unmuted now shows low-volume icon instead of muted icon - Fix clipVolume divergence: init.ts uses Number.isFinite() matching media.ts semantics (preserves data-volume="0") - 3 new tests: ARIA attributes, volumechange on mute, icon collision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
3256551a5e |
fix(player): single-owner audio to prevent double voice in preview (#298)
## Summary Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip). ## Root cause Two audio pipelines were playing the same source in parallel: 1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path. 2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message. The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks. Confirmed via agent-browser instrumentation on `factory-series-c-video`: - 6 `volumechange` events per play cycle (mute-fight signature) - Both copies audible at `volume=1`, offset growing 23ms → 80ms - Every seek widened the drift further PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback. ## Fix Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime. - **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash. - **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold. Two orthogonal mute channels replace the volume fight: | Channel | Purpose | |---|---| | `set-muted` | User's mute preference (existing, unchanged) | | `set-media-output-muted` | Internal ownership handoff (new) | `syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it. ## Why this shape - **Single owner, explicit transition.** No races, no tug-of-war. - **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing. - **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact. - **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift. - **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior. - **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files. ## Files changed - `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types. - `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields. - `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`. - `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`. - `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session. - `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership. ## Verified end-to-end with agent-browser on `factory-series-c-video` **Runtime ownership (default — desktop studio):** | | Before | After | |---|---|---| | `PARENT.play()` calls per play cycle | 1 | **0** | | iframe `volumechange` events | 6 | **0** | | Audible streams | 2 (drifting) | **1 (iframe)** | **Parent ownership (simulated autoplay block — direct message):** | | Value | |---|---| | iframe audio | `muted=true`, `volume=1` (untouched) | | parent audio | `muted=false`, `volume=1`, audible | | Parent ↔ iframe `currentTime` offset | ~6 ms steady state | | Offset > 150 ms | corrected by mirror sync | **Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:** Event timeline captured via agent-browser instrumentation: ``` t=0.0 ms IFRAME.play() called ← runtime attempts playback t=0.4 ms IFRAME.play() REJECTED: NotAllowedError ← simulated mobile gate t=0.4 ms →IFRAME bridge set-media-output-muted=true ← player promotes t=0.6 ms PARENT.play() called ← parent proxy starts t=0.8 ms ←IFRAME msg media-autoplay-blocked ← runtime signal t=1.0 ms PARENT.play() resolved ← audible t=1.3 ms IFRAME muted=true, volume=1 ← iframe silenced via native muted ``` Steady state at t=4 s under promoted parent ownership: | Element | currentTime | paused | volume | muted | |---|---|---|---|---| | Parent audio | 4.060 s | false | 1.0 | **false** (audible) | | Iframe audio | 4.068 s | false | 1.0 | **true** (silent) | **Offset: 8 ms**, single audible stream, orthogonal mute channel respected. ## Test plan - [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`) - [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow) - [x] `bun run build` — all packages green - [x] Fresh preview + browser repro on `factory-series-c-video`: - [x] Runtime ownership: single audio stream, no drift - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset - [x] No `volumechange` thrash in either ownership mode - [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above) |
||
|
Vance IngallsandClaude Opus 4.6
|
9f8e5ba5a1 |
initial code (#2)
* feat: initial code port from hyperframes-internal Port all OSS-ready packages from the internal monorepo: - @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime - @hyperframes/cli — CLI for creating, previewing, and rendering compositions - @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg) - @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg) - @hyperframes/ui-player — browser-based video player component - @hyperframes/studio — composition editor (React frontend + Hono backend) Includes regression test suite with Docker-based test harness. All HeyGen-internal references, deployment infrastructure, and proprietary assets have been removed. Package names migrated from @app/* to @hyperframes/*. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scrub internal codenames and stale references from OSS port - Replace static.heygen.ai runtime URLs in test fixtures - Remove internal CDN publish script (publish-hyperframe-runtime.ts) - Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime with neutral names (studio, hyperframe-runtime, __hyperframeRuntime) - Fix stale Vault API / localhost references in docs - Remove broken deprecated_studio link Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining internal codenames and stale references - Delete stale producer README.md and PIPELINE.md (referenced nonexistent files) - Replace "Cerberus" codename with "HyperFrames" in test design reviews - Replace magic-edit postMessage identifiers with hf-preview/hf-parent - Rename debug-magic-edit-timeline.ts to debug-timeline.ts - Replace "Motion Cut" with "HyperFrames" in Timeline comments - Fix studio/CLI references to nonexistent archive package (use local data/projects/ dir, stub render proxy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |