Files
hyperframes/packages/player/README.md
T
James RussoandClaude Opus 4.6 b23b0751da fix(player): parent-frame media playback for mobile (#266)
* fix(player): parent-frame media playback for mobile

Mobile browsers block media.play() inside iframes when the user
gesture happened in the parent frame — postMessage doesn't transfer
user activation (per the User Activation v2 spec).

## Problem

The player renders compositions in a sandboxed iframe. When a user
taps play in the parent frame, the player sends a postMessage to the
iframe's runtime, which calls audio.play(). On mobile, this fails
silently because the iframe has no user activation context.

## Solution

The player now extracts ALL timed media elements (audio/video with
data-start) from the iframe's DOM (same-origin access), creates
parent-frame copies, and disables the iframe originals. On play(),
parentMedia.play() runs synchronously in the gesture call stack,
satisfying mobile autoplay policy.

### Generic media handling

- Finds all `audio[data-start], video[data-start]` in the iframe
- Creates a parent-frame copy for each (Audio or Video element)
- Preserves data-start offsets for correct seek positioning
- Strips data-start from iframe elements so the runtime ignores them
- Falls back to iframe media for cross-origin iframes

### `audio-src` attribute

Convenience for the common single-narration case. When set, the
player starts preloading audio immediately — before the iframe loads.
This eliminates the loading delay that caused jittery playback.

### No active sync

Both parent media and the GSAP timeline are real-time systems. When
started simultaneously, they naturally stay within ~10ms — no drift
correction needed. Active sync with coarse granularity (50ms polling)
caused MORE jitter than it prevented via repeated audio seeks.

## CI

- Added unified `test` job replacing separate per-package test jobs
- Added root `test` script: `bun run --filter '*' test`
- New packages with test scripts are automatically included
- Added happy-dom for player DOM tests

## Tests

- 10 new tests for parent-frame media: preloading, play, pause,
  seek, muted/rate sync, cleanup, attribute changes
- All 21 player tests pass

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(shader-transitions): pass CI when no test files exist

Add --passWithNoTests to vitest run so the unified test job
doesn't fail on packages that have a test script but no test
files yet.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update tests for new id field and GSAP lint rule

- normalize.test.ts: loadTranscript now assigns id fields (w0, w1, etc.)
  to SRT/VTT results and empty string for words-json passthrough
- lintProject.test.ts: add GSAP CDN script to validHtml() fixture to
  satisfy the missing_gsap_script lint rule added in core

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add missing data-start/data-duration to validHtml fixture

The validHtml() test fixture was missing data-start and data-duration
attributes, triggering the root_composition_missing_data_start and
root_composition_missing_data_duration lint warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): fetch LFS objects for producer test job

Producer regression tests compare rendered output against reference MP4
files stored in git LFS. Without lfs: true, checkout fetches pointer
files instead of actual videos, causing "moov atom not found" errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: remove redundant test-producer job

The regression workflow already runs the same 28 producer fixtures
in a Docker container with prod-matching Chrome/fonts/ffmpeg, sharded
across 8 parallel matrix jobs with 40-min timeouts. The CI test-producer
job was a duplicate that ran on bare runners with worse determinism
and a 15-min timeout too short for all fixtures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:20:07 -07:00

204 lines
8.1 KiB
Markdown

# @hyperframes/player
Embeddable web component for playing HyperFrames compositions. Zero dependencies, works with any framework.
## Install
```bash
npm install @hyperframes/player
```
Or load directly via CDN:
```html
<script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script>
```
## Usage
```html
<hyperframes-player src="./my-composition/index.html" controls></hyperframes-player>
```
The player loads the composition in a sandboxed iframe, auto-detects its dimensions and duration, and scales it responsively to fit the container.
### With a framework
```typescript
import "@hyperframes/player";
// The custom element is now registered — use it in your markup
// React: <hyperframes-player src="..." controls />
// Vue: <hyperframes-player :src="url" controls />
```
### Poster image
Show a static image before playback starts:
```html
<hyperframes-player
src="./composition/index.html"
poster="./thumbnail.jpg"
controls
></hyperframes-player>
```
## Attributes
| Attribute | Type | Default | Description |
| --------------- | ------- | ------- | -------------------------------------------- |
| `src` | string | — | URL to the composition HTML file |
| `audio-src` | string | — | Audio URL for parent-frame playback (mobile) |
| `width` | number | 1920 | Composition width in pixels (aspect ratio) |
| `height` | number | 1080 | Composition height in pixels (aspect ratio) |
| `controls` | boolean | false | Show play/pause, scrubber, and time display |
| `muted` | boolean | false | Mute audio playback |
| `poster` | string | — | Image URL shown before playback starts |
| `playback-rate` | number | 1 | Speed multiplier (0.5 = half, 2 = double) |
| `autoplay` | boolean | false | Start playing when ready |
| `loop` | boolean | false | Restart when the composition ends |
### Mobile audio
Mobile browsers block `audio.play()` inside iframes when the user gesture happened in the parent frame (the [User Activation spec](https://html.spec.whatwg.org/multipage/interaction.html#tracking-user-activation) does not propagate activation across frame boundaries via `postMessage`).
The player handles this automatically for same-origin iframes (the default — `sandbox` includes `allow-same-origin`):
1. When the composition is ready, the player extracts all timed media (`audio[data-start]`, `video[data-start]`) from the iframe DOM and creates parent-frame copies.
2. The iframe originals are disabled (`src` and `data-start` removed) so the runtime doesn't try to play them.
3. When `play()` is called (from a user gesture), parent media `.play()` runs synchronously in the gesture call stack, satisfying mobile autoplay policy.
4. Both parent media and the GSAP timeline start simultaneously and free-run — no active sync needed since both are real-time systems.
No changes are required by consumers — this works out of the box.
The optional `audio-src` attribute can be used to start preloading a primary audio track before the iframe loads (useful on slow connections), but is not required for mobile playback.
## JavaScript API
```js
const player = document.querySelector("hyperframes-player");
// Playback
player.play();
player.pause();
player.seek(2.5); // jump to 2.5 seconds
// Properties
player.currentTime; // number (read/write)
player.duration; // number (read-only)
player.paused; // boolean (read-only)
player.ready; // boolean (read-only)
player.playbackRate; // number (read/write)
player.muted; // boolean (read/write)
player.loop; // boolean (read/write)
// Inner iframe access (for advanced consumers — see "Advanced: iframe access" below)
player.iframeElement; // HTMLIFrameElement (read-only)
```
## Advanced: iframe access
The composition runs inside a sandboxed `<iframe>` in the player's Shadow DOM. For most use cases you don't need direct access — the JavaScript API above is enough. But if you're building an editor, recorder, or custom timeline that needs to inspect the composition's DOM or read its `__player` / `__timelines` runtime objects, use the `iframeElement` getter:
```js
const player = document.querySelector("hyperframes-player");
const iframe = player.iframeElement;
// Now you can reach into the composition's DOM and runtime
iframe.contentDocument.querySelectorAll("[data-composition-id]");
iframe.contentWindow.__timelines;
```
This is the canonical way to bridge the player into tools like [`@hyperframes/studio`](../studio). The studio exports a `resolveIframe` helper that works with both iframe refs and web-component refs:
```ts
import { useTimelinePlayer, resolveIframe } from "@hyperframes/studio";
const { iframeRef } = useTimelinePlayer();
const player = document.createElement("hyperframes-player");
player.setAttribute("src", src);
container.appendChild(player);
// Forward the inner iframe so useTimelinePlayer can drive play/pause/seek.
iframeRef.current = resolveIframe(player);
```
### React: declarative ref pattern
If you prefer JSX over imperative element creation, attach a ref directly to the web component and resolve the iframe inside an effect:
```tsx
import "@hyperframes/player";
import type { HyperframesPlayer } from "@hyperframes/player";
import { useTimelinePlayer, resolveIframe } from "@hyperframes/studio";
function StudioPreview({ src }: { src: string }) {
const { iframeRef, onIframeLoad } = useTimelinePlayer();
const playerRef = useRef<HyperframesPlayer>(null);
useEffect(() => {
iframeRef.current = resolveIframe(playerRef.current);
});
return <hyperframes-player ref={playerRef} src={src} onLoad={onIframeLoad} />;
}
```
> **Heads up — common gotcha**
>
> If you pass the `<hyperframes-player>` element itself (not `iframeElement`) into a hook that expects an `<iframe>`, every `.contentWindow` / `.contentDocument` access returns `null` because the iframe lives inside the player's Shadow DOM. Always extract `iframeElement` first, or use `resolveIframe` from `@hyperframes/studio` which handles both iframe and web-component hosts transparently.
## Events
| Event | Detail | Fired when |
| ------------ | ----------------- | ------------------------------------------ |
| `ready` | `{ duration }` | Composition loaded and duration determined |
| `play` | — | Playback started |
| `pause` | — | Playback paused |
| `timeupdate` | `{ currentTime }` | Playback position changed (~10 fps) |
| `ended` | — | Reached the end (when not looping) |
| `error` | `{ message }` | Composition failed to load |
```js
player.addEventListener("ready", (e) => {
console.log(`Duration: ${e.detail.duration}s`);
});
player.addEventListener("ended", () => {
console.log("Done!");
});
```
## Sizing
The player fills its container and scales the composition to fit while preserving aspect ratio. Set a size on the element or its parent:
```css
hyperframes-player {
width: 100%;
max-width: 800px;
aspect-ratio: 16 / 9;
}
```
The `width` and `height` attributes define the composition's native resolution for aspect ratio calculation — they don't set the player's display size.
## How it works
The player renders compositions in a sandboxed `<iframe>` inside a Shadow DOM. It communicates with the HyperFrames runtime via `postMessage`. If the composition has GSAP timelines (`window.__timelines`) but no runtime, the player auto-injects it from CDN.
## Distribution
| Format | File | Use case |
| ------ | ------------------------------ | ------------------------------ |
| ESM | `hyperframes-player.js` | Bundlers (Vite, webpack, etc.) |
| CJS | `hyperframes-player.cjs` | Node.js / require() |
| IIFE | `hyperframes-player.global.js` | `<script>` tag, CDN |
All formats are minified with source maps. TypeScript definitions included.
## License
MIT