mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-10 22:20:14 +00:00
ba1d807621aa473e501eb239085a82bd4e8b2562
59
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
e70687b66c |
fix(player): resolve iframe media src against iframe baseURI (#295)
## Summary `_setupParentMedia` scans the iframe for `audio[data-start]` / `video[data-start]` and creates parallel media elements in the host document (so the studio can scrub audio at sub-frame precision without iframe cross-origin restrictions). It was reading the raw `src` attribute string and assigning it directly to the host-document element, which then resolved relative URLs against the **studio root** instead of the **iframe**. Result: a composition like \`\`\`html <audio id="narration" data-start="0" data-duration="53" src="assets/narration.wav"></audio> \`\`\` played fine in rendered MP4 output but 404'd silently in the studio preview (parent audio got `src = http://localhost:PORT/assets/narration.wav` instead of `http://localhost:PORT/api/projects/<name>/preview/assets/narration.wav`). ## Fix Resolve the src against \`iframeEl.ownerDocument.baseURI\` before passing it to \`_createParentMedia\`. Also read the raw \`src\` attribute on \`<source>\` fallbacks so both paths go through the same resolution. Diff is 2 lines of meaningful change (9 total once you include the comment). ## Reproduction 1. Create a project with a narration at \`assets/narration.wav\` 2. Reference it in \`index.html\` with \`<audio data-start="0" data-duration="53" src="assets/narration.wav">\` 3. \`npx hyperframes preview\` → open, click play 4. Before: silent (parent audio's \`error.code === 4\` / \`MEDIA_ERR_SRC_NOT_SUPPORTED\`) 5. After: narration plays, scrubbing syncs ## Test plan - [x] Existing 21 player tests pass (`bun run --filter=@hyperframes/player test`) - [x] oxlint + oxfmt clean on changed file - [x] Manual: verified in-studio playback of a narration sourced via relative URL - [x] Reviewer: confirm render pipeline unaffected (render doesn't go through `_setupParentMedia`) ## Notes No tests added for this path because the existing harness covers only the `audio-src` attribute codepath — `_setupParentMedia` is triggered by an internal probe interval against a live iframe, which the current fixture doesn't build. Happy to add one in a follow-up if reviewers want that coverage before merge. |
||
|
Miguel Ángel
|
a6ff9e2d9f |
fix(player): preserve iframe media attributes for runtime sync (#291)
## Summary - `_setupParentMedia()` (added in #266) was stripping `data-start`, `data-duration`, and `src` from audio/video elements inside the composition iframe - The runtime's `syncRuntimeMedia` queries `audio[data-start]` to find media clips — removing these attributes made the runtime unable to find, sync, or play audio - Result: silent audio in studio preview and any context where `__player.play()` is called directly (not through the web component) ## Fix - Keep all iframe media attributes intact so the runtime can track time position and manage playback - When parent-frame media `play()` succeeds (mobile use case), mute the iframe copies via `volume = 0` to prevent double audio - On desktop and in the studio (which calls `__player.play()` directly), the runtime's own media sync handles playback normally ## Test plan - [x] 21 player unit tests pass - [x] Verified with John Wu's slideshow project: audio element preserves `data-start`, `data-duration`, `src` after runtime init - [x] Verified runtime `syncRuntimeMedia` finds and plays audio (currentTime advances in sync with timeline) - [x] Build passes (lint, format, typecheck) |
||
|
James RussoandClaude Opus 4.6
|
b23b0751da |
fix(player): parent-frame media playback for mobile (#266)
* fix(player): parent-frame media playback for mobile Mobile browsers block media.play() inside iframes when the user gesture happened in the parent frame — postMessage doesn't transfer user activation (per the User Activation v2 spec). ## Problem The player renders compositions in a sandboxed iframe. When a user taps play in the parent frame, the player sends a postMessage to the iframe's runtime, which calls audio.play(). On mobile, this fails silently because the iframe has no user activation context. ## Solution The player now extracts ALL timed media elements (audio/video with data-start) from the iframe's DOM (same-origin access), creates parent-frame copies, and disables the iframe originals. On play(), parentMedia.play() runs synchronously in the gesture call stack, satisfying mobile autoplay policy. ### Generic media handling - Finds all `audio[data-start], video[data-start]` in the iframe - Creates a parent-frame copy for each (Audio or Video element) - Preserves data-start offsets for correct seek positioning - Strips data-start from iframe elements so the runtime ignores them - Falls back to iframe media for cross-origin iframes ### `audio-src` attribute Convenience for the common single-narration case. When set, the player starts preloading audio immediately — before the iframe loads. This eliminates the loading delay that caused jittery playback. ### No active sync Both parent media and the GSAP timeline are real-time systems. When started simultaneously, they naturally stay within ~10ms — no drift correction needed. Active sync with coarse granularity (50ms polling) caused MORE jitter than it prevented via repeated audio seeks. ## CI - Added unified `test` job replacing separate per-package test jobs - Added root `test` script: `bun run --filter '*' test` - New packages with test scripts are automatically included - Added happy-dom for player DOM tests ## Tests - 10 new tests for parent-frame media: preloading, play, pause, seek, muted/rate sync, cleanup, attribute changes - All 21 player tests pass Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(shader-transitions): pass CI when no test files exist Add --passWithNoTests to vitest run so the unified test job doesn't fail on packages that have a test script but no test files yet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): update tests for new id field and GSAP lint rule - normalize.test.ts: loadTranscript now assigns id fields (w0, w1, etc.) to SRT/VTT results and empty string for words-json passthrough - lintProject.test.ts: add GSAP CDN script to validHtml() fixture to satisfy the missing_gsap_script lint rule added in core Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add missing data-start/data-duration to validHtml fixture The validHtml() test fixture was missing data-start and data-duration attributes, triggering the root_composition_missing_data_start and root_composition_missing_data_duration lint warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): fetch LFS objects for producer test job Producer regression tests compare rendered output against reference MP4 files stored in git LFS. Without lfs: true, checkout fetches pointer files instead of actual videos, causing "moov atom not found" errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: remove redundant test-producer job The regression workflow already runs the same 28 producer fixtures in a Docker container with prod-matching Chrome/fonts/ffmpeg, sharded across 8 parallel matrix jobs with 40-min timeouts. The CI test-producer job was a duplicate that ran on bare runners with worse determinism and a 15-min timeout too short for all fixtures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
f40447f2e8 |
fix(player,studio): resolve root timeline from DOM instead of last key (#247)
Bundled previews register a master composition alongside its sub-compositions
in `window.__timelines`, e.g. { main, intro, scene2, scene5 }. Both the
player's probe and studio's getAdapter() were using `keys[keys.length - 1]`
to pick the adapter, which returned whichever timeline was registered last.
That made the player report the final sub-composition's duration as the
video length (e.g. 3.2s instead of the master's 14s) and play/pause/seek
targeted that sub-composition instead of the full composition.
Look up the outermost `[data-composition-id]` element in the iframe DOM
and use its id to select the right timeline. Falls back to last-key when
no element is present (standalone sub-composition previews) so drill-down
views keep working.
Also restores `main`/`import` entry points on @hyperframes/player to
point at compiled dist output (the src/ paths broke workspace consumers
that only receive the published tarball).
|
||
|
Miguel Ángel
|
1149602bc9 |
fix(studio): support web-component refs in useTimelinePlayer (#245)
* fix(studio): support web-component refs in useTimelinePlayer The studio's `useTimelinePlayer` hook returns an `iframeRef` that consumers attach to an `<iframe>` element. When consumers wrap the iframe in a custom element (e.g. `<hyperframes-player>`) that puts the iframe inside its shadow DOM, every `iframeRef.current.contentWindow` access returned `null` and `getAdapter()` silently failed — meaning timeline seek, play, pause, and `refreshPlayer` all became no-ops. Changes: - Add `resolveIframe(el)` helper that returns the underlying iframe whether the host is the iframe itself, a custom element with a shadow-DOM iframe, or a wrapper with a descendant iframe. - Export `resolveIframe` from the studio so consumers can pre-resolve the iframe before assigning it to `iframeRef`. - Internal `useTimelinePlayer` keeps the strict `HTMLIFrameElement` ref type, so existing consumers attaching directly to an `<iframe>` are unaffected. Also adds: - JSDoc on the player's `iframeElement` getter. - "Advanced: iframe access" docs section in `packages/player/README.md` and `docs/packages/player.mdx`. - Type-safety lint rules in `.oxlintrc.json` and a "Type-safety conventions" section in `CONTRIBUTING.md`. Backward compatible — App.tsx and NLELayout.tsx continue to work unchanged. * chore(lint): defer no-explicit-any rule; it broke existing codebase The new rules added 37 errors across 32 existing files — mostly legitimate `window as any` casts at browser-global and test-mock boundaries. Enabling them without fixing all violations breaks CI. Revert the `.oxlintrc.json` additions and soften the CONTRIBUTING.md wording to describe the convention without claiming lint enforcement (that enforcement will come in a follow-up PR that fixes all sites). |
||
|
James RussoandClaude Opus 4.6
|
18de86e4bd |
fix(player): handle Infinity duration; add lint rules for data-duration and Math.ceil overshoot (#243)
* fix(player): handle Infinity duration from runtime gracefully When compositions have repeating animations without data-duration, the runtime sends durationInFrames: Infinity. The player now ignores non-finite duration values instead of displaying "Infinity:NaN" in the controls. formatTime also returns "0:00" for non-finite inputs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(lint): add data-duration and Math.ceil overshoot rules - Add root_composition_missing_data_duration warning when the root composition element is missing data-duration, which causes the runtime to infer Infinity for loop-inflated timelines. - Add gsap_repeat_ceil_overshoot warning that catches repeat: Math.ceil(d/c)-1 patterns which overshoot the intended duration. Recommends Math.floor instead. - Fix gsap_infinite_repeat fixHint to suggest Math.floor (not Math.ceil). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(player): wait for injected runtime before declaring ready When the player auto-injects the runtime script (because the composition has GSAP timelines but no runtime), it would immediately declare ready on the next probe cycle — before the runtime script finished loading from CDN. This caused play() to send a postMessage that nobody received, making autoplay silently fail. Now the probe waits for the runtime bridge (__hf or __player) to appear before proceeding to the ready state. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
0da93cea3d |
feat(player): add speed control with popup menu and CSS theming (#241)
Add playback speed control to the player controls bar: - Popup menu with logarithmic presets (0.25x-4x) - Custom presets via speed-presets attribute - Full CSS custom property theming (--hfp-accent, --hfp-controls-bg, etc.) - ratechange event dispatch - Exports: SPEED_PRESETS, formatSpeed, ControlsOptions - Fix package.json export condition ordering |
||
|
Miguel Ángel
|
3482441c9f |
feat(studio): use @hyperframes/player web component for preview (#238)
## Summary - **Replaces the studio's hand-rolled iframe + scaling in** **`Player.tsx`** with the `<hyperframes-player>` web component, eliminating duplicated ResizeObserver, dimension detection, and stage-size message handling - **Adds a public** **`iframeElement`** **getter** to the player web component so the studio's `useTimelinePlayer` can still access the inner iframe for clip manifest parsing, timeline probing, and DOM inspection - **Updates player package exports** to resolve from source for workspace consumers (matching `@hyperframes/core` pattern), while npm-published consumers still get built `dist/` files ### Why a separate player package? 1. **Zero dependencies, any framework** — 12KB vanilla web component vs 940KB React+Zustand+CodeMirror studio 2. **CDN-ready** — single `<script>` tag, no build pipeline needed 3. **Embeddable by third parties** — users embed compositions in their own sites without the studio 4. **Single source of truth** — studio now uses the player instead of duplicating its scaling/detection logic ## Test plan - [x] `pnpm --filter @hyperframes/player typecheck` passes - [x] `pnpm --filter @hyperframes/studio typecheck` passes - [x] `pnpm --filter @hyperframes/studio build` passes - [x] `pnpm --filter @hyperframes/studio test` passes (2 pre-existing failures, unrelated) - [x] E2E: Standalone player loads composition, detects 4s GSAP timeline, controls work, play/pause works - [x] E2E: Studio preview renders via `<hyperframes-player>`, `iframeElement` bridge works, playback controls sync correctly |
||
|
Miguel Ángel
|
5655dabff6 |
feat: allow clip animation + ship <hyperframes-player> web component (#209)
## Summary Two independent initiatives that improve agent DX and expand HyperFrames' reach. ### Initiative 1: Fix the Clip Animation Footgun - `gsap_animates_clip_element` lint rule now uses smart detection — only errors when GSAP animates `visibility` or `display` on a clip element - All other properties (opacity, transform, x, y, scale, etc.) are allowed silently - This was the #1 agent failure in QA (10/10 agents hit it on v0.2.1) ### Initiative 2: `<hyperframes-player>` Web Component - New `@hyperframes/player` package — zero dependencies, 3.3KB gzipped - Iframe-based web component with Shadow DOM for perfect isolation - Video-like API: `play()`, `pause()`, `seek()`, `currentTime`, `duration`, events - Controls overlay with play/pause, scrubber (mouse + touch), time display, auto-hide - Full docs page at `docs/packages/player.mdx` ## Before / After ### Clip animation lint **Before (10/10 agents hit this):** ``` ✗ gsap_animates_clip_element: GSAP animation targets a clip element. Selector "#title" resolves to element <div id="title" class="clip">. The framework manages clip visibility — animate an inner wrapper instead. Fix: Wrap content in a child <div> and target that with GSAP. ``` **After (only errors on actual conflicts):** ``` # This passes lint — no error: tl.from("#title", { opacity: 0, y: -50, scale: 0.8 }, 0); # This still errors — actual conflict with runtime: tl.to("#title", { visibility: "hidden" }, 3); ✗ gsap_animates_clip_element: GSAP animation sets visibility on a clip element. Fix: Remove the visibility/display tween. Use opacity for fade effects. ``` ### Embeddable player **Before:** No way to embed a composition in a web page. **After:** ```html <script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script> <hyperframes-player src="./composition/index.html" controls></hyperframes-player> ``` ```js const player = document.querySelector('hyperframes-player'); player.play(); player.pause(); player.seek(2.5); player.addEventListener('ready', (e) => console.log('Duration:', e.detail.duration)); ``` ## Test plan - [x] 427 core tests pass (20 GSAP lint tests with smart detection) - [x] 7 player tests pass (formatTime + element registration) - [x] TypeScript compiles cleanly (core + player) - [x] Lint: GSAP animating clip with safe props → 0 errors - [x] Lint: GSAP animating clip with `visibility` → 1 error (correct) - [x] Player builds to 3.3KB gzipped ESM - [x] Lockfile updated for CI - [x] Docs page added at `docs/packages/player.mdx` |