mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
Vance Ingalls
10d2725b54
## Summary First slice of `P0-1` from the player perf proposal: lays the foundation for a player perf gate so later PRs can plug in fps / scrub / drift / parity scenarios without rebuilding infrastructure. Ships one smoke scenario (`03-load`, cold + warm composition load) to prove the gate end-to-end on real numbers. ## Why There was no automated way to catch player perf regressions. Every perf concern in the existing proposal — composition load time, sustained FPS, scrub p95, mirror-clock drift, live-vs-seek parity — needs the same plumbing: a same-origin harness, a Puppeteer runner, a baseline file, a gate that emits structured results, and a CI workflow that runs the right scenarios on the right changes. Building that up-front in one reviewable PR lets every subsequent perf PR (`P0-1b`, `P0-1c`, and beyond) be a 100-line scenario file plus a baseline entry instead of re-litigating the framework. ## What changed ### Harness — `packages/player/tests/perf/server.ts` - `Bun.serve` on a free port, single same-origin host for the player IIFE bundle, hyperframe runtime, GSAP from `node_modules`, and fixture HTML. - Same-origin matters: cross-origin would force every probe through `postMessage`, hiding bugs and inflating numbers in ways production never sees. Tests should measure the path the studio editor actually takes. - Routes: - `/player.js` → built IIFE bundle (rebuilt on demand). - `/vendor/runtime.js`, `/vendor/gsap.min.js` → resolved from `node_modules` so fixtures don't need to ship copies. - `/fixtures/*` → fixture HTML. ### Runner — `packages/player/tests/perf/runner.ts` - `puppeteer-core` thin wrappers (`launchBrowser`, `loadHostPage`). - Uses the system Chrome detected by `setup-chrome` in CI rather than the bundled puppeteer revision — keeps the action smaller, lets us pin Chrome version policy at the workflow level, and matches what users actually run. ### Gate — `packages/player/tests/perf/perf-gate.ts` + `baseline.json` - Loads `baseline.json` (initial budgets: cold/warm comp load, fps, scrub p95 isolated/inline, drift max/p95) with a 10% `allowedRegressionRatio`. - Per-metric direction (`lower-is-better` / `higher-is-better`) so the same evaluator handles latency and throughput. - Returns a structured `GateReport` consumed by both the CLI (table output) and `metrics.json` (CI artifact). - Two modes: `measure` (log only — used during the rollout) and `enforce` (fail the build) — flip per-metric once we trust the signal, without touching the harness. ### CLI orchestrator — `packages/player/tests/perf/index.ts` - Parses `--mode` / `--scenarios` / `--runs` / `--fixture` in both space- and equals-separated form (so `--scenarios fps,scrub` and `--scenarios=fps,scrub` both work — matches what humans type and what GitHub Actions emits). - Runs scenarios, runs the gate, and **always** writes `results/metrics.json` with schema version, git SHA, metrics, and gate rows — so failed runs are still investigable from the artifact alone. ### Fixture + smoke scenario - `fixtures/gsap-heavy/index.html`: 200 stagger-animated tiles, no media. Heavy enough to make load time meaningful, light enough to be deterministic. - `scenarios/03-load.ts`: cold + warm composition load. Measures from navigation start to player `ready` event, reports p95 across runs. ### CI — `.github/workflows/player-perf.yml` - `paths-filter` on `player` / `core` / `runtime` — perf only runs when something that could move the needle actually changed. - Sets up bun + node + chrome, runs perf in `measure` mode on a shard matrix (so future scenarios shard naturally), uploads `metrics.json` artifacts, and a summary job aggregates shard results into a single PR comment. ### Wiring - `packages/player`: `puppeteer-core`, `gsap`, `@types/bun` devDeps; typecheck extended to cover the perf `tsconfig`; new `perf` script. - Root `package.json`: `player:perf` workspace script so `bun run player:perf` runs the whole suite locally with the same flags CI uses. - `.gitignore`: `packages/player/tests/perf/results/`. - Separate `tests/perf/tsconfig.json` so test code doesn't pollute the package `rootDir` while still being typechecked. ## Test plan - [x] Local: `bun run player:perf` passes — cold p95 ≈ 386 ms, warm p95 ≈ 375 ms, both well under the seeded baselines. - [x] Typecheck, lint, format pass on the perf workspace. - [x] Existing player unit tests (71/71) still green. - [ ] First CI run after merge will be the real signal: confirms `setup-chrome` works on hosted runners, the shard matrix wires up, and `metrics.json` artifacts upload. ## Stack Step `P0-1a` of the player perf proposal. The next two slices are content-only — they don't touch the harness: - `P0-1b` (#400): adds `02-fps`, `04-scrub`, `05-drift` scenarios on a 10-video-grid fixture. - `P0-1c` (#401): adds `06-parity` (live playback vs. synchronously-seeked reference, compared via SSIM). Wiring this gate up first means each follow-up is a self-contained scenario file + baseline row + workflow shard.
203 lines
6.6 KiB
TypeScript
203 lines
6.6 KiB
TypeScript
import { existsSync } from "node:fs";
|
|
import { dirname, join, resolve } from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
|
|
/**
|
|
* Static file server for player perf tests.
|
|
*
|
|
* Serves all bundles, vendor scripts, fixtures, and the embed host page from
|
|
* a single origin so the player iframe stays same-origin. Without same-origin
|
|
* the runtime probe in `_onIframeLoad` falls into the cross-origin catch path
|
|
* and the `ready` event fires later (or not at all) — which would be measured
|
|
* as a player-side regression instead of an environment artifact.
|
|
*
|
|
* URL routes:
|
|
* / → host.html (default fixture: gsap-heavy)
|
|
* /host.html?fixture=<name> → embed page hosting <hyperframes-player>
|
|
* /player/hyperframes-player.global.js
|
|
* /vendor/gsap.min.js
|
|
* /vendor/hyperframe.runtime.iife.js
|
|
* /fixtures/<name>/<file> → fixture HTML + assets
|
|
*/
|
|
|
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
const PLAYER_PKG = resolve(HERE, "../..");
|
|
const REPO_ROOT = resolve(PLAYER_PKG, "../..");
|
|
|
|
function firstExisting(candidates: string[]): string {
|
|
for (const p of candidates) {
|
|
if (existsSync(p)) return p;
|
|
}
|
|
return candidates[0] ?? "";
|
|
}
|
|
|
|
const PATHS = {
|
|
player: join(PLAYER_PKG, "dist/hyperframes-player.global.js"),
|
|
runtime: join(REPO_ROOT, "packages/core/dist/hyperframe.runtime.iife.js"),
|
|
// bun installs gsap into the package's node_modules in workspace mode, but
|
|
// hoists it to the repo root if multiple packages share the same version.
|
|
// Probe both locations so the server works regardless of layout.
|
|
gsap: firstExisting([
|
|
join(PLAYER_PKG, "node_modules/gsap/dist/gsap.min.js"),
|
|
join(REPO_ROOT, "node_modules/gsap/dist/gsap.min.js"),
|
|
]),
|
|
fixturesDir: join(HERE, "fixtures"),
|
|
} as const;
|
|
|
|
export type ServeOptions = {
|
|
port?: number;
|
|
/** Disables HTTP cache so every request is a "cold" fetch. Used for cold-load scenarios. */
|
|
noCache?: boolean;
|
|
};
|
|
|
|
export type RunningServer = {
|
|
port: number;
|
|
origin: string;
|
|
stop(): Promise<void>;
|
|
};
|
|
|
|
const MIME_TYPES: Record<string, string> = {
|
|
".html": "text/html; charset=utf-8",
|
|
".js": "application/javascript; charset=utf-8",
|
|
".mjs": "application/javascript; charset=utf-8",
|
|
".css": "text/css; charset=utf-8",
|
|
".json": "application/json; charset=utf-8",
|
|
".png": "image/png",
|
|
".jpg": "image/jpeg",
|
|
".jpeg": "image/jpeg",
|
|
".webp": "image/webp",
|
|
".mp4": "video/mp4",
|
|
".webm": "video/webm",
|
|
".mp3": "audio/mpeg",
|
|
};
|
|
|
|
function mimeFor(path: string): string {
|
|
const dot = path.lastIndexOf(".");
|
|
if (dot < 0) return "application/octet-stream";
|
|
return MIME_TYPES[path.slice(dot).toLowerCase()] ?? "application/octet-stream";
|
|
}
|
|
|
|
function buildHostHtml(fixtureName: string, width: number, height: number): string {
|
|
const playerSrc = "/player/hyperframes-player.global.js";
|
|
const fixtureSrc = `/fixtures/${fixtureName}/index.html`;
|
|
return `<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8" />
|
|
<title>player perf host: ${fixtureName}</title>
|
|
<style>
|
|
html, body { margin: 0; padding: 0; background: #000; }
|
|
hyperframes-player { display: block; }
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<hyperframes-player
|
|
id="player"
|
|
src="${fixtureSrc}"
|
|
width="${width}"
|
|
height="${height}"
|
|
muted
|
|
></hyperframes-player>
|
|
<script>
|
|
window.__playerReady = false;
|
|
window.__playerReadyAt = null;
|
|
window.__playerNavStart = performance.timeOrigin + performance.now();
|
|
const player = document.getElementById("player");
|
|
player.addEventListener("ready", function (event) {
|
|
window.__playerReady = true;
|
|
window.__playerReadyAt = performance.timeOrigin + performance.now();
|
|
window.__playerDuration = (event.detail && event.detail.duration) || 0;
|
|
});
|
|
player.addEventListener("error", function (event) {
|
|
window.__playerError = (event.detail && event.detail.message) || "unknown";
|
|
});
|
|
</script>
|
|
<script src="${playerSrc}"></script>
|
|
</body>
|
|
</html>`;
|
|
}
|
|
|
|
async function readBunFile(path: string): Promise<Response> {
|
|
if (!existsSync(path)) {
|
|
return new Response(`Not found: ${path}`, { status: 404 });
|
|
}
|
|
const file = Bun.file(path);
|
|
return new Response(file, {
|
|
headers: {
|
|
"Content-Type": mimeFor(path),
|
|
},
|
|
});
|
|
}
|
|
|
|
function applyCacheHeaders(res: Response, noCache: boolean): Response {
|
|
if (noCache) {
|
|
res.headers.set("Cache-Control", "no-store, no-cache, must-revalidate, max-age=0");
|
|
res.headers.set("Pragma", "no-cache");
|
|
res.headers.set("Expires", "0");
|
|
} else {
|
|
res.headers.set("Cache-Control", "public, max-age=3600");
|
|
}
|
|
return res;
|
|
}
|
|
|
|
export function startServer(options: ServeOptions = {}): RunningServer {
|
|
const noCache = options.noCache ?? false;
|
|
|
|
const server = Bun.serve({
|
|
port: options.port ?? 0,
|
|
async fetch(req) {
|
|
const url = new URL(req.url);
|
|
const path = url.pathname;
|
|
|
|
if (path === "/" || path === "/host.html") {
|
|
const fixture = url.searchParams.get("fixture") || "gsap-heavy";
|
|
const width = Number(url.searchParams.get("width") || "1920");
|
|
const height = Number(url.searchParams.get("height") || "1080");
|
|
const html = buildHostHtml(fixture, width, height);
|
|
return applyCacheHeaders(
|
|
new Response(html, { headers: { "Content-Type": "text/html; charset=utf-8" } }),
|
|
noCache,
|
|
);
|
|
}
|
|
|
|
if (path === "/player/hyperframes-player.global.js") {
|
|
return applyCacheHeaders(await readBunFile(PATHS.player), noCache);
|
|
}
|
|
|
|
if (path === "/vendor/hyperframe.runtime.iife.js") {
|
|
return applyCacheHeaders(await readBunFile(PATHS.runtime), noCache);
|
|
}
|
|
|
|
if (path === "/vendor/gsap.min.js") {
|
|
return applyCacheHeaders(await readBunFile(PATHS.gsap), noCache);
|
|
}
|
|
|
|
if (path.startsWith("/fixtures/")) {
|
|
const rel = path.replace(/^\/fixtures\//, "");
|
|
const filePath = join(PATHS.fixturesDir, rel);
|
|
if (!filePath.startsWith(PATHS.fixturesDir)) {
|
|
return new Response("Forbidden", { status: 403 });
|
|
}
|
|
return applyCacheHeaders(await readBunFile(filePath), noCache);
|
|
}
|
|
|
|
return new Response("Not found", { status: 404 });
|
|
},
|
|
});
|
|
|
|
// server.port is `number | undefined` in Bun's types (undefined only for unix-socket
|
|
// servers, which we never use). Narrow it once at startup so the rest of the perf
|
|
// harness can rely on a numeric origin.
|
|
const port = server.port;
|
|
if (port === undefined) {
|
|
throw new Error("[player-perf] Bun.serve did not assign a TCP port");
|
|
}
|
|
return {
|
|
port,
|
|
origin: `http://127.0.0.1:${port}`,
|
|
async stop() {
|
|
server.stop(true);
|
|
},
|
|
};
|
|
}
|