mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 16:42:27 +00:00
Vance Ingalls
6f05fabbf8
## Summary Second slice of `P0-1` from the player perf proposal: plugs the three steady-state scenarios — sustained playback FPS, scrub latency, and media-sync drift — into the perf gate that landed in #399. Adds the multi-video fixture they all share, wires three new shards into CI, and seeds one new baseline (`droppedFramesMax`). ## Why #399 stood up the harness and proved it with a single load-time scenario. By itself that's enough to catch regressions in initial composition setup, but it can't catch the things players actually fail at in production: - **FPS regressions** — a render-loop change that drops the ticker from 60 to 45 fps still loads fast. - **Scrub latency regressions** — the inline-vs-isolated split (#397) is exactly the kind of code path where a refactor can silently push everyone back to the postMessage round trip. - **Media drift** — runtime mirror logic (#396 in this stack) and per-frame scheduling tweaks can both cause video to slip out of sync with the composition clock without producing a single console error. Each of these is a target metric in the proposal with a concrete budget. This PR turns those budgets into gated CI signals and produces continuous data for them on every player/core/runtime change. ## What changed ### Fixture — `packages/player/tests/perf/fixtures/10-video-grid/` - `index.html`: 10-second composition, 1920×1080, 30 fps, with 10 simultaneously-decoding video tiles in a 5×2 grid plus a subtle GSAP scale "breath" on each tile (so the rAF/RVFC loops have real work to do without GSAP dominating the budget the decoder needs). - `sample.mp4`: small (~190 KB) clip checked in so the fixture is hermetic — no external CDN dependency, identical bytes on every run. - Same `data-composition-id="main"` host pattern as `gsap-heavy`, so the existing harness loader works without changes. ### `02-fps.ts` — sustained playback frame rate - Loads `10-video-grid`, calls `player.play()`, samples `requestAnimationFrame` callbacks inside the iframe for 5 s. - Crucial sequencing: install the rAF sampler **before** `play()`, wait for `__player.isPlaying() === true`, **then reset the sample buffer** — otherwise the postMessage round-trip ramp-up window drags the average down by 5–10 fps. - FPS = `(samples − 1) / (lastTs − firstTs in s)`; uses rAF timestamps (the same ones the compositor saw) rather than wall-clock `setTimeout`, so we're measuring real frame production. - Dropped-frame definition matches Chrome DevTools: gap > 1.5× (1000/60 ms) ≈ 25 ms = "missed at least one vsync." - Aggregation across runs: `min(fps)` and `max(droppedFrames)` — worst case wins, since the proposal asserts a floor on fps and a ceiling on drops. - Emits `playback_fps_min` (higher-is-better, baseline `fpsMin = 55`) and `playback_dropped_frames_max` (lower-is-better, baseline `droppedFramesMax = 3`). ### `04-scrub.ts` — scrub latency, inline + isolated - Loads `10-video-grid`, pauses, then issues 10 seek calls in two batches: first the synchronous **inline** path (`<hyperframes-player>`'s default same-origin `_trySyncSeek`), then the **isolated** path (forced by replacing `_trySyncSeek` with `() => false`, which makes the player fall back to the postMessage `_sendControl("seek")` bridge that cross-origin embeds and pre-#397 builds use). - Inline runs first so the isolated mode's monkey-patch can't bleed back into the inline samples. - Detection: a rAF watcher inside the iframe polls `__player.getTime()` until it's within `MATCH_TOLERANCE_S = 0.05 s` of the requested target. Tolerance exists because the postMessage bridge converts seconds → frame number → seconds, and that round-trip can introduce sub-frame quantization drift even for targets on the canonical fps grid. - Timing: `performance.timeOrigin + performance.now()` in both contexts. `timeOrigin` is consistent across same-process frames, so `t1 − t0` is a true wall-clock latency, not a host-only or iframe-only stopwatch. - Targets alternate forward/backward (`1.0, 7.0, 2.0, 8.0, 3.0, 9.0, 4.0, 6.0, 5.0, 0.5`) so no two consecutive seeks land near each other — protects the rAF watcher from matching against a stale `getTime()` value before the seek command is processed. - Aggregation: `percentile(95)` across the pooled per-seek latencies from every run. With 10 seeks × 2 modes × 3 runs we get 30 samples per mode per CI shard, enough for a stable p95. - Emits `scrub_latency_p95_inline_ms` (lower-is-better, baseline `scrubLatencyP95InlineMs = 33`) and `scrub_latency_p95_isolated_ms` (lower-is-better, baseline `scrubLatencyP95IsolatedMs = 80`). ### `05-drift.ts` — media sync drift - Loads `10-video-grid`, plays 6 s, instruments **every** `video[data-start]` element with `requestVideoFrameCallback`. Each callback records `(compositionTime, actualMediaTime)` plus a snapshot of the clip transform (`clipStart`, `clipMediaStart`, `clipPlaybackRate`). - Drift = `|actualMediaTime − ((compTime − clipStart) × clipPlaybackRate + clipMediaStart)|` — the same transform the runtime applies in `packages/core/src/runtime/media.ts`, snapshotted once at sampler install so the per-frame work is just subtract + multiply + abs. - Sustain window is 6 s (not the proposal's 10 s) because the fixture composition is exactly 10 s long and we want headroom before the end-of-timeline pause/clamp behavior. With 10 videos × ~25 fps × 6 s we still pool ~1500 samples per run — more than enough for a stable p95. - Same "reset buffer after play confirmed" gotcha as `02-fps.ts`: frames captured during the postMessage round-trip would compare a non-zero `mediaTime` against `getTime() === 0` and inflate drift by hundreds of ms. - Aggregation: `max()` and `percentile(95)` across the pooled per-frame drifts. The proposal's max-drift ceiling of 500 ms is intentional — the runtime hard-resyncs when `|currentTime − relTime| > 0.5 s`, so a regression past 500 ms means the corrective resync kicked in and the viewer saw a jump. - Emits `media_drift_max_ms` (lower-is-better, baseline `driftMaxMs = 500`) and `media_drift_p95_ms` (lower-is-better, baseline `driftP95Ms = 100`). ### Wiring - `packages/player/tests/perf/index.ts`: add `fps`, `scrub`, `drift` to `ScenarioId`, `DEFAULT_RUNS`, the default scenario list (`--scenarios` defaults to all four), and three new dispatch branches. - `packages/player/tests/perf/perf-gate.ts`: add `droppedFramesMax: number` to `PerfBaseline`. Other baseline keys for these scenarios were already seeded in #399. - `packages/player/tests/perf/baseline.json`: add `droppedFramesMax: 3`. - `.github/workflows/player-perf.yml`: three new matrix shards (`fps` / `scrub` / `drift`) at `runs: 3`. Same `paths-filter` and same artifact-upload pattern as the `load` shard, so the summary job aggregates them automatically. ## Methodology highlights These three patterns recur in all three scenarios and are worth noting because they're load-bearing for the numbers we report: 1. **Reset buffer after play-confirmed.** The `play()` API is async (postMessage), so any samples captured before `__player.isPlaying() === true` belong to ramp-up, not steady-state. Both `02-fps` and `05-drift` clear `__perfRafSamples` / `__perfDriftSamples` *after* the wait. Without this, fps drops 5–10 and drift inflates by hundreds of ms. 2. **Iframe-side timing.** All three scenarios time inside the iframe (`performance.timeOrigin + performance.now()` for scrub, rAF/RVFC timestamps for fps/drift) rather than host-side. The iframe is what the user sees; host-side timing would conflate Puppeteer's IPC overhead with real player latency. 3. **Stop sampling before pause.** Sampler is deactivated *before* `pause()` is issued, so the pause command's postMessage round-trip can't perturb the tail of the measurement window. ## Test plan - [x] Local: `bun run player:perf` runs all four scenarios end-to-end on the 10-video-grid fixture. - [x] Each scenario produces metrics matching its declared `baselineKey` so `perf-gate.ts` can find them. - [x] Typecheck, lint, format pass on the new files. - [x] Existing player unit tests untouched (no production code changes in this PR). - [ ] First CI run will confirm the new shards complete inside the workflow timeout and that the summary job picks up their `metrics.json` artifacts. ## Stack Step `P0-1b` of the player perf proposal. Builds on: - `P0-1a` (#399): the harness, runner, gate, and CI workflow this PR plugs new scenarios into. Followed by: - `P0-1c` (#401): `06-parity` — live playback frame vs. synchronously-seeked reference frame, compared via SSIM, on the existing `gsap-heavy` fixture from #399.
127 lines
3.4 KiB
HTML
127 lines
3.4 KiB
HTML
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8" />
|
|
<title>perf fixture: 10-video-grid</title>
|
|
<style>
|
|
:root {
|
|
color-scheme: dark;
|
|
}
|
|
html,
|
|
body {
|
|
margin: 0;
|
|
padding: 0;
|
|
background: #050714;
|
|
color: #e6e6f0;
|
|
font-family:
|
|
system-ui,
|
|
-apple-system,
|
|
sans-serif;
|
|
overflow: hidden;
|
|
}
|
|
#root {
|
|
position: relative;
|
|
width: 1920px;
|
|
height: 1080px;
|
|
display: grid;
|
|
grid-template-columns: repeat(5, 1fr);
|
|
grid-template-rows: repeat(2, 1fr);
|
|
gap: 8px;
|
|
padding: 8px;
|
|
box-sizing: border-box;
|
|
}
|
|
.tile {
|
|
position: relative;
|
|
background: #111827;
|
|
border-radius: 12px;
|
|
overflow: hidden;
|
|
box-shadow: 0 0 0 1px rgba(255, 255, 255, 0.05);
|
|
will-change: transform;
|
|
}
|
|
.tile video {
|
|
position: absolute;
|
|
inset: 0;
|
|
width: 100%;
|
|
height: 100%;
|
|
object-fit: cover;
|
|
}
|
|
.tile .label {
|
|
position: absolute;
|
|
top: 8px;
|
|
left: 8px;
|
|
z-index: 2;
|
|
font:
|
|
600 14px/1 system-ui,
|
|
sans-serif;
|
|
color: #fff;
|
|
background: rgba(0, 0, 0, 0.6);
|
|
padding: 4px 8px;
|
|
border-radius: 6px;
|
|
pointer-events: none;
|
|
}
|
|
</style>
|
|
<script src="/vendor/gsap.min.js"></script>
|
|
<script data-hyperframes-runtime="1" src="/vendor/hyperframe.runtime.iife.js"></script>
|
|
</head>
|
|
<body>
|
|
<div
|
|
id="root"
|
|
data-composition-id="main"
|
|
data-width="1920"
|
|
data-height="1080"
|
|
data-duration="10"
|
|
data-fps="30"
|
|
></div>
|
|
<script>
|
|
(function () {
|
|
var TILE_COUNT = 10;
|
|
var DURATION_SEC = 10;
|
|
|
|
var root = document.getElementById("root");
|
|
var tiles = [];
|
|
for (var i = 0; i < TILE_COUNT; i++) {
|
|
var tile = document.createElement("div");
|
|
tile.className = "tile";
|
|
tile.id = "tile-" + i;
|
|
|
|
var label = document.createElement("div");
|
|
label.className = "label";
|
|
label.textContent = "video " + (i + 1);
|
|
tile.appendChild(label);
|
|
|
|
var video = document.createElement("video");
|
|
video.id = "video-" + i;
|
|
video.setAttribute("data-start", "0");
|
|
video.setAttribute("data-duration", String(DURATION_SEC));
|
|
video.setAttribute("data-track-index", String(i));
|
|
video.setAttribute("src", "sample.mp4");
|
|
video.setAttribute("preload", "auto");
|
|
video.setAttribute("playsinline", "");
|
|
video.muted = true;
|
|
tile.appendChild(video);
|
|
|
|
root.appendChild(tile);
|
|
tiles.push(tile);
|
|
}
|
|
|
|
// Lightweight parent timeline so the player has a non-empty composition
|
|
// to drive. Each tile gets a subtle scale "breath" over the full
|
|
// duration — enough to keep GSAP scrubbing real properties without
|
|
// dominating the rAF budget that the video decoder needs.
|
|
var tl = gsap.timeline({ paused: true });
|
|
for (var j = 0; j < tiles.length; j++) {
|
|
tl.fromTo(
|
|
tiles[j],
|
|
{ scale: 0.985 },
|
|
{ scale: 1, duration: DURATION_SEC, ease: "sine.inOut" },
|
|
0,
|
|
);
|
|
}
|
|
|
|
window.__timelines = window.__timelines || {};
|
|
window.__timelines["main"] = tl;
|
|
})();
|
|
</script>
|
|
</body>
|
|
</html>
|