Files
hyperframes/packages/player
Vance Ingalls 80e7cd2844 perf(player): p0-1c live-playback parity test via SSIM (#401)
## Summary

Adds **scenario 06: live-playback parity** — the third and final tranche of the P0-1 perf-test buildout (`p0-1a` infra → `p0-1b` fps/scrub/drift → this).

The scenario plays the `gsap-heavy` fixture, freezes it mid-animation, screenshots the live frame, then synchronously seeks the same player back to that exact timestamp and screenshots the reference. The two PNGs are diffed with `ffmpeg -lavfi ssim` and the resulting average SSIM is emitted as `parity_ssim_min`. Baseline gate: **SSIM ≥ 0.95**.

This pins the player's two frame-production paths (the runtime's animation loop vs. `_trySyncSeek`) to each other visually, so any future drift between scrub and playback fails CI instead of silently shipping.

## Motivation

`<hyperframes-player>` produces frames two different ways:

1. **Live playback** — the runtime's animation loop advances the GSAP timeline frame-by-frame.
2. **Synchronous seek** (`_trySyncSeek`, landed in #397) — for same-origin embeds, the player calls into the iframe runtime's `seek()` directly and asks for a specific time.

These paths must agree. If they don't — different rounding, different sub-frame sampling, different state ordering — scrubbing a paused composition shows different pixels than a paused-during-playback frame at the same time. That's a class of bug that only surfaces visually, never in unit tests, and only at specific timestamps where many things are mid-flight.

`gsap-heavy` is a 10s composition with 60 tiles each running a staggered 4s out-and-back tween. At t=5.0s a large fraction of those tiles are mid-flight, so the rendered frame has many distinct, position-sensitive pixels — the worst-case input for any sub-frame disagreement. If the two paths produce identical pixels here, they'll produce identical pixels everywhere that matters.

## What changed

- **`packages/player/tests/perf/scenarios/06-parity.ts`** — new scenario (~340 lines). Owns capture, seek, screenshot, SSIM, artifact persistence, and aggregation.
- **`packages/player/tests/perf/index.ts`** — register `parity` as a scenario id, default-runs = 3, dispatch to `runParity`, include in the default scenario list.
- **`packages/player/tests/perf/perf-gate.ts`** — extend `PerfBaseline` with `paritySsimMin`.
- **`packages/player/tests/perf/baseline.json`** — `paritySsimMin: 0.95`.
- **`.github/workflows/player-perf.yml`** — add a `parity` shard (3 runs) to the matrix alongside `load` / `fps` / `scrub` / `drift`.

## How the scenario works

The hard part is making the two captures land on the *exact same timestamp* without trusting `postMessage` round-trips or arbitrary `setTimeout` settling.

1. **Install an iframe-side rAF watcher** before issuing `play()`. The watcher polls `__player.getTime()` every animation frame and, the first time `getTime() >= 5.0`, calls `__player.pause()` *from inside the same rAF tick*. `pause()` is synchronous (it calls `timeline.pause()`), so the timeline freezes at exactly that `getTime()` value with no postMessage round-trip. The watcher's Promise resolves with that frozen value as the canonical `T_actual` for the run.
2. **Confirm `isPlaying() === true`** via `frame.waitForFunction` before awaiting the watcher. Without this, the test can hang if `play()` hasn't kicked the timeline yet.
3. **Wait for paint** — two `requestAnimationFrame` ticks on the host page. The first flushes pending style/layout, the second guarantees a painted compositor commit. Same paint-settlement pattern as `packages/producer/src/parity-harness.ts`.
4. **Screenshot the live frame** — `page.screenshot({ type: "png" })`.
5. **Synchronously seek to `T_actual`** — call `el.seek(capturedTime)` on the host page. The player's public `seek()` calls `_trySyncSeek` which (same-origin) calls `__player.seek()` synchronously, so no postMessage await is needed. The runtime's deterministic `seek()` rebuilds frame state at exactly the requested time.
6. **Wait for paint** again, screenshot the reference frame.
7. **Diff with ffmpeg** — `ffmpeg -hide_banner -i reference.png -i actual.png -lavfi ssim -f null -`. ffmpeg writes per-channel + overall SSIM to stderr; we parse the `All:` value, clamp at 1.0 (ffmpeg occasionally reports 1.000001 on identical inputs), and treat it as the run's score.
8. **Persist artifacts** under `tests/perf/results/parity/run-N/` (`actual.png`, `reference.png`, `captured-time.txt`) so CI can upload them and so a failed run is locally reproducible. Directory is already gitignored via the existing `packages/player/tests/perf/results/` rule.

### Aggregation

`min()` across runs, **not** mean. We want the *worst observed* parity to pass the gate so a single bad run can't get masked by averaging. Both per-run scores and the aggregate are logged.

### Output metric

| name              | direction        | baseline             |
|-------------------|------------------|----------------------|
| `parity_ssim_min` | higher-is-better | `paritySsimMin: 0.95` |

With deterministic rendering enabled in the runner, identical pixels produce SSIM very close to 1.0; the 0.95 threshold leaves headroom for legitimate fixture-level noise (font hinting, GPU compositor variance) while still catching any real disagreement between the two paths.

## Test plan

- `bun run player:perf -- --scenarios=parity --runs=3` locally on `gsap-heavy` — passes with SSIM ≈ 0.999 across all 3 runs.
- Inspected `results/parity/run-1/actual.png` and `reference.png` side-by-side — visually identical.
- Inspected `captured-time.txt` to confirm `T_actual` lands just past 5.0s (within one frame).
- Sanity test: temporarily forced a 1-frame offset between live and reference capture; SSIM dropped well below 0.95 as expected, confirming the threshold catches real drift.
- CI: `parity` shard added alongside the existing `load` / `fps` / `scrub` / `drift` shards; same `measure`-mode / artifact-upload / aggregation flow.
- `bunx oxlint` and `bunx oxfmt --check` clean on the new scenario.

## Stack

This is the top of the perf stack:

1. #393 `perf/x-1-emit-performance-metric` — performance.measure() emission
2. #394 `perf/p1-1-share-player-styles-via-adopted-stylesheets` — adopted stylesheets
3. #395 `perf/p1-2-scope-media-mutation-observer` — scoped MutationObserver
4. #396 `perf/p1-4-coalesce-mirror-parent-media-time` — coalesce currentTime writes
5. #397 `perf/p3-1-sync-seek-same-origin` — synchronous seek path (the path this PR pins)
6. #398 `perf/p3-2-srcdoc-composition-switching` — srcdoc switching
7. #399 `perf/p0-1a-perf-test-infra` — server, runner, perf-gate, CI
8. #400 `perf/p0-1b-perf-tests-for-fps-scrub-drift` — fps / scrub / drift scenarios
9. **#401 `perf/p0-1c-live-playback-parity-test` ← you are here**

With this PR landed the perf harness covers all five proposal scenarios: `load`, `fps`, `scrub`, `drift`, `parity`.
2026-04-22 18:15:46 -07:00
..

@hyperframes/player

Embeddable web component for playing HyperFrames compositions. Zero dependencies, works with any framework.

Install

npm install @hyperframes/player

Or load directly via CDN:

<script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script>

Usage

<hyperframes-player src="./my-composition/index.html" controls></hyperframes-player>

The player loads the composition in a sandboxed iframe, auto-detects its dimensions and duration, and scales it responsively to fit the container.

With a framework

import "@hyperframes/player";

// The custom element is now registered — use it in your markup
// React: <hyperframes-player src="..." controls />
// Vue:   <hyperframes-player :src="url" controls />

Poster image

Show a static image before playback starts:

<hyperframes-player
  src="./composition/index.html"
  poster="./thumbnail.jpg"
  controls
></hyperframes-player>

Attributes

Attribute Type Default Description
src string URL to the composition HTML file
audio-src string Audio URL for parent-frame playback (mobile)
width number 1920 Composition width in pixels (aspect ratio)
height number 1080 Composition height in pixels (aspect ratio)
controls boolean false Show play/pause, scrubber, and time display
muted boolean false Mute audio playback
poster string Image URL shown before playback starts
playback-rate number 1 Speed multiplier (0.5 = half, 2 = double)
autoplay boolean false Start playing when ready
loop boolean false Restart when the composition ends

Mobile audio

Mobile browsers block audio.play() inside iframes when the user gesture happened in the parent frame (the User Activation spec does not propagate activation across frame boundaries via postMessage).

The player handles this automatically for same-origin iframes (the default — sandbox includes allow-same-origin):

  1. When the composition is ready, the player extracts all timed media (audio[data-start], video[data-start]) from the iframe DOM and creates parent-frame copies.
  2. The iframe originals are disabled (src and data-start removed) so the runtime doesn't try to play them.
  3. When play() is called (from a user gesture), parent media .play() runs synchronously in the gesture call stack, satisfying mobile autoplay policy.
  4. Both parent media and the GSAP timeline start simultaneously and free-run — no active sync needed since both are real-time systems.

No changes are required by consumers — this works out of the box.

The optional audio-src attribute can be used to start preloading a primary audio track before the iframe loads (useful on slow connections), but is not required for mobile playback.

JavaScript API

const player = document.querySelector("hyperframes-player");

// Playback
player.play();
player.pause();
player.seek(2.5); // jump to 2.5 seconds

// Properties
player.currentTime; // number (read/write)
player.duration; // number (read-only)
player.paused; // boolean (read-only)
player.ready; // boolean (read-only)
player.playbackRate; // number (read/write)
player.muted; // boolean (read/write)
player.loop; // boolean (read/write)

// Inner iframe access (for advanced consumers — see "Advanced: iframe access" below)
player.iframeElement; // HTMLIFrameElement (read-only)

Advanced: iframe access

The composition runs inside a sandboxed <iframe> in the player's Shadow DOM. For most use cases you don't need direct access — the JavaScript API above is enough. But if you're building an editor, recorder, or custom timeline that needs to inspect the composition's DOM or read its __player / __timelines runtime objects, use the iframeElement getter:

const player = document.querySelector("hyperframes-player");
const iframe = player.iframeElement;

// Now you can reach into the composition's DOM and runtime
iframe.contentDocument.querySelectorAll("[data-composition-id]");
iframe.contentWindow.__timelines;

This is the canonical way to bridge the player into tools like @hyperframes/studio. The studio exports a resolveIframe helper that works with both iframe refs and web-component refs:

import { useTimelinePlayer, resolveIframe } from "@hyperframes/studio";

const { iframeRef } = useTimelinePlayer();
const player = document.createElement("hyperframes-player");
player.setAttribute("src", src);
container.appendChild(player);

// Forward the inner iframe so useTimelinePlayer can drive play/pause/seek.
iframeRef.current = resolveIframe(player);

React: declarative ref pattern

If you prefer JSX over imperative element creation, attach a ref directly to the web component and resolve the iframe inside an effect:

import "@hyperframes/player";
import type { HyperframesPlayer } from "@hyperframes/player";
import { useTimelinePlayer, resolveIframe } from "@hyperframes/studio";

function StudioPreview({ src }: { src: string }) {
  const { iframeRef, onIframeLoad } = useTimelinePlayer();
  const playerRef = useRef<HyperframesPlayer>(null);

  useEffect(() => {
    iframeRef.current = resolveIframe(playerRef.current);
  });

  return <hyperframes-player ref={playerRef} src={src} onLoad={onIframeLoad} />;
}

Heads up — common gotcha

If you pass the <hyperframes-player> element itself (not iframeElement) into a hook that expects an <iframe>, every .contentWindow / .contentDocument access returns null because the iframe lives inside the player's Shadow DOM. Always extract iframeElement first, or use resolveIframe from @hyperframes/studio which handles both iframe and web-component hosts transparently.

Events

Event Detail Fired when
ready { duration } Composition loaded and duration determined
play Playback started
pause Playback paused
timeupdate { currentTime } Playback position changed (~10 fps)
ended Reached the end (when not looping)
error { message } Composition failed to load
player.addEventListener("ready", (e) => {
  console.log(`Duration: ${e.detail.duration}s`);
});

player.addEventListener("ended", () => {
  console.log("Done!");
});

Sizing

The player fills its container and scales the composition to fit while preserving aspect ratio. Set a size on the element or its parent:

hyperframes-player {
  width: 100%;
  max-width: 800px;
  aspect-ratio: 16 / 9;
}

The width and height attributes define the composition's native resolution for aspect ratio calculation — they don't set the player's display size.

How it works

The player renders compositions in a sandboxed <iframe> inside a Shadow DOM. It communicates with the HyperFrames runtime via postMessage. If the composition has GSAP timelines (window.__timelines) but no runtime, the player auto-injects it from CDN.

Distribution

Format File Use case
ESM hyperframes-player.js Bundlers (Vite, webpack, etc.)
CJS hyperframes-player.cjs Node.js / require()
IIFE hyperframes-player.global.js <script> tag, CDN

All formats are minified with source maps. TypeScript definitions included.

License

MIT