ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 12:54:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
425 Commits 855 Branches 404 Tags
6fd99109c991fe2e4f34d279a60af700e907df98
Commit Graph
308 Commits
Author SHA1 Message Date
Vance Ingalls 6fd99109c9 fix(producer): tighten resource lifecycle and harden file server (#371) 
## Summary

Five resource-management fixes in `renderOrchestrator.ts` and `fileServer.ts`: HDR encoder cleanup on non-abort errors, `frameDirMaxIndexCache` eviction, mid-transition abort responsiveness, pre-allocated transition buffers, and a path-traversal guard for the local file server.

## Why

`Chunk 5` of `plans/hdr-followups.md`. These are independent leaks/hangs/security issues that had each been called out in prior PR reviews and never landed.

## What changed

**5A — HDR encoder + `domSession` cleanup.** The HDR streaming encoder and `domSession` were spawned outside any outer `try/finally`, so a non-abort error between encoder spawn and the inner cleanup leaked the FFmpeg process and held the browser page open. Wrapped the entire HDR (and SDR streaming) capture path in a `try/finally` with explicit `*Closed` flags, and defensively close both in the outer `finally` if they haven't been closed already. `StreamingEncoder.close()` and `closeCaptureSession()` are both idempotent, so double-close is safe.

**5B — `frameDirMaxIndexCache` + `hdrFrameDirs` eviction.** `frameDirMaxIndexCache` is module-scoped and grew monotonically: every render added entries that were never removed. Lifted `hdrFrameDirs` to the outer scope, drop the matching cache entry in the per-video `rmSync` block, and sweep any survivors in the outer `finally`. The on-disk frames themselves were already torn down with `workDir`; this just stops the in-process Map from leaking entries across renders.

**5C — Abort signal between scene A and scene B.** During a shader transition the orchestrator captures scene A and scene B back-to-back inside a single outer frame iteration. An abort that arrived while scene A was capturing wouldn't be noticed until the next outer frame — after scene B had already been fully composited and discarded. Added `assertNotAborted()` at the top of the inner `[transBufferA, transBufferB]` loop so abort is observed before the second scene's DOM seek + screenshot.

**5D — Pre-allocated transition buffers (already addressed).** The transition buffers (`transBufferA`, `transBufferB`, `transOutput`, `normalCanvas`) are pre-allocated outside the per-frame loop. The remaining `Buffer.from` copies sit in HDR transfer conversion (Chunk 8B territory) and image preload, neither of which is the per-frame hot path.

**5E — `fileServer` path-traversal guard.** `fileServer.ts` joined `compiledDir` / `projectDir` with the request path and only checked `existsSync` + `isFile`. `path.join` normalizes `..` segments, so `GET /../etc/passwd` would resolve to `/etc/passwd` and be served straight off disk if the file existed. Added an `isPathInside(child, parent)` helper that resolves both sides and compares prefixes with the platform separator appended (so `/foo` doesn't match `/foobar`), and rejects any candidate that lands outside its intended root.

## Test plan

- [x] `bun run --filter @hyperframes/producer typecheck` passes.
- [x] `fileServer.test.ts` 13/13 pass (4 existing + 9 new `isPathInside` cases covering same-path, nested, prefix-only siblings, escaping traversal, traversal that resolves back inside, trailing-slash handling, and relative-path resolution).
- [x] Manual: kill a render mid-flight with a non-abort error; no orphaned `ffmpeg` processes (5A).
- [x] Manual: two render jobs back-to-back; cache cleared between jobs (5B).
- [x] Manual: abort during a transition frame; stops promptly, not after scene B (5C).
- [x] Manual: `GET /../../../etc/passwd` against the local file server returns 403/404 (5E).

## Stack

Chunk 5 of `plans/hdr-followups.md`.
 2026-04-22 21:45:44 -07:00
Miguel Ángel aea85af044 fix: improve studio timeline discoverability (#431) 2026-04-23 06:31:31 +02:00
Vance Ingalls 5256a93b2d feat(engine): wire options.hdr through chunkEncoder + dynamic SDR→HDR transfer (#370) 
## Summary

Three independent fixes that share a common thread: HDR config flowing correctly from `EngineConfig` down through every encoder. The headline fix: disk-based HDR encodes via `chunkEncoder` were silently producing BT.709-tagged output despite `options.hdr` being set.

## Why

`Chunk 3` of `plans/hdr-followups.md`. The streaming encoder was correct but `chunkEncoder.buildEncoderArgs` hard-coded BT.709 color tags and the `bt709` VUI block in `-x265-params`, even when callers passed an HDR `EncoderOptions`. Today this is harmless because `renderOrchestrator` routes native-HDR content to `streamingEncoder` and only feeds `chunkEncoder` sRGB Chrome screenshots — but the contract was a lie, and any future caller that wired HDR through `chunkEncoder` would silently get SDR output.

## What changed

**3A — `chunkEncoder` respects `options.hdr` (BT.2020 + mastering metadata).** When `options.hdr` is set, the libx265 software path emits `bt2020nc` plus the matching transfer (`smpte2084` for PQ, `arib-std-b67` for HLG) at the codec level *and* embeds master-display + max-cll SEI in `-x265-params` via `getHdrEncoderColorParams`. libx264 still tags BT.709 inside `-x264-params` (libx264 has no HDR support) but the codec-level color flags flip so the container describes pixels truthfully. GPU H.265 (nvenc/videotoolbox/qsv/vaapi) gets the BT.2020 tags but no `-x265-params` block, so static mastering metadata is omitted — acceptable for previews, not HDR-aware delivery.

**3B — `convertSdrToHdr` accepts a target transfer.** `videoFrameExtractor.convertSdrToHdr` was hard-coded to `transfer=arib-std-b67` (HLG) regardless of the surrounding composition's dominant transfer. `extractAllVideoFrames` now calls `analyzeCompositionHdr` first, then passes the dominant transfer (`"pq"` or `"hlg"`) into `convertSdrToHdr` so an SDR clip mixed into a PQ timeline gets converted with `smpte2084`, not `arib-std-b67`.

**3C — `EngineConfig.hdr` type matches its declared shape.** The IIFE for the `hdr` field returned `undefined` when `PRODUCER_HDR_TRANSFER` wasn't `"hlg"` or `"pq"`, but the field is typed as `{ transfer: HdrTransfer } | false`. Returning `false` matches the type and avoids a downstream `undefined` check.

## Test plan

- [x] `chunkEncoder.test.ts`: replaced the previous "HDR options ignored" assertions with 8 new specs covering BT.2020 + transfer tagging, master-display/max-cll embedding, libx264 fallback behavior, GPU H.265 + HDR (tags but no x265-params), and range conversion for both SDR and HDR CPU paths.
- [x] All 313 engine unit tests pass (5 new HDR specs).
- [x] `ffprobe` an HDR composition rendered through the chunk encoder path: shows `bt2020nc` color matrix, `smpte2084` transfer, and mastering display metadata.

## Stack

Chunk 3 of `plans/hdr-followups.md`. Independent of Chunks 1/4 (touches separate code paths).
 2026-04-22 20:36:29 -07:00
Vance Ingalls 60f4ebbf13 test(hdr-regression): tighten Window C maxFrameFailures budget after Chunk 1 fix (#369) 
## Summary

Tighten `hdr-regression` Window C `maxFrameFailures` from 30 → 5 now that Chunk 1 (opacity pipeline) has landed.

## Why

Window C (direct `<video>` opacity tween) was previously listed as a known failure with a `maxFrameFailures` budget of 30 to absorb expected drift until Chunk 1 landed. After the Chunk 1 fix, the regression test passes against the existing golden with **0 failed frames**. Tightening the budget catches any future drift in the opacity path immediately rather than letting up to 30 broken frames slip through.

## What changed

- `tests/hdr-regression/meta.json`: `maxFrameFailures` 30 → 5 (small budget remains for HEVC encoder noise).
- `tests/hdr-regression/README.md`: updated to mark Window C as fixed and note the tightened budget.

The HEVC encoder is byte-deterministic and the opacity fix doesn't perturb pixels at the PSNR ≥ 28 checkpoint threshold, so regenerating the golden produces byte-identical output. The golden is therefore unchanged. Window F (transform + border-radius) remains pending Chunk 4; its broken state is currently baked into the golden, so the suite is green and Chunk 4's regen will catch any drift.

## Test plan

- [x] `bun run test --filter hdr-regression` — passes with 0 failed frames at the new budget.

## Stack

Follow-up to Chunk 1 (opacity pipeline). Reviewable separately so the golden churn (none in this case) is decoupled from the code fix.
 2026-04-22 20:14:46 -07:00
Vance Ingalls 2d57918f64 fix(engine): stop clobbering native <video> opacity in HDR pipeline (#368) 
## Summary

Fix four interrelated bugs in the opacity pipeline. The headline fix: the HDR compositor was effectively ignoring direct-on-`<video>` opacity animation because the engine itself was clobbering inline opacity with `opacity: 0 !important` — switching to `visibility: hidden` resolves the bug at the root.

## Why

`Chunk 1` of `plans/hdr-followups.md`. This was the most user-visible bug in the entire follow-ups list: a GSAP-controlled opacity tween directly on a `<video>` element under HDR rendered at full brightness instead of fading.

## What changed

**1A — Stop clobbering native `<video>` opacity.** `screenshotService.injectVideoFramesBatch` and `syncVideoFrameVisibility` were applying `opacity: 0 !important` to native `<video>` elements to hide them under the injected `<img>`. That stomp clobbered any GSAP-controlled inline opacity, so the next seek read 0 from computed style and the comp went black. Switched to `visibility: hidden !important` only. Visibility hides the element from rendering without changing its opacity, so subsequent reads (and `queryElementStacking`) see the real GSAP value on every frame. The `parseFloat(...) || 1` recovery hack at `injectVideoFramesBatch` was specifically there to compensate for this stomp; it's now replaced with a `Number.isNaN` guard that defaults to 1 only when parsing actually fails.

**1B — `Number.isNaN` guards in `queryVideoElementBounds`.** `parseFloat(style.opacity) || 1` silently coerced a real opacity of 0 into 1. Switched to explicit `Number.isNaN` checks so opacity 0 stays 0. Same fix for `parseFloat(style.zIndex)`.

**1C — `instanceof HTMLElement` instead of cast.** `resolveRadius` cast `el as HTMLElement` to read `offsetWidth`/`Height`. SVG and other non-HTML elements would have crashed at runtime. Replaced the cast with an `instanceof HTMLElement` guard, and made the numeric fallback `Number.isNaN`-safe.

**1D — Opacity walk starts from the element itself.** The walk in `queryVideoElementBounds` started from `el.parentElement` for HDR videos to skip past the engine's forced `opacity: 0` on the element itself. Now that the engine never sets opacity, the special case is unnecessary — always walk from `el`. Kept the `isHdrEl` lookup because transform/border-radius logic further down still branches on it.

## Test plan

- [x] `bun run --filter @hyperframes/engine typecheck` clean.
- [x] `bun run --filter @hyperframes/engine test` — 308/308 passing.
- [x] `bun run --filter @hyperframes/producer typecheck` clean.
- [x] `oxlint` + `oxfmt --check` on both touched files.
- [x] `hdr-regression` Window C (the direct-opacity window) now passes against the regenerated golden — see follow-up PR in this stack which tightens the budget.

## Stack

Chunk 1 of `plans/hdr-followups.md`. Window C of the regression suite documents the bug; the next PR in the stack regenerates the golden and tightens its `maxFrameFailures` budget.
 2026-04-22 19:50:40 -07:00
Miguel Ángel 293d92af05 chore: release v0.4.15-alpha.1 2026-04-22 22:12:34 -04:00
Miguel Ángel b4e9d64e29 feat(cli): hyperframes publish — share projects via a public URL (#312) 
## Summary

This PR adds `hyperframes publish` as the OSS handoff into the persisted HyperFrames publish flow.

Instead of opening a local tunnel, the CLI now:

1. zips the local project
2. uploads it to the HeyGen publish backend
3. gets back a stable `hyperframes.dev` project URL plus claim token
4. prints a claimable URL for the user

Example output:

```bash
$ hyperframes publish

  Project    my-video
  Files      12
  Public     https://hyperframes.dev/p/hfp_123?claim_token=...

  Open the URL on hyperframes.dev to claim the project and continue editing.
```

## User Flow

The intended user flow is:

1. Run `hyperframes publish` from a local HyperFrames project.
2. The CLI uploads the project as a zip to the publish API.
3. The CLI prints a stable `hyperframes.dev` URL with the claim token attached.
4. The user opens that URL in the browser.
5. `hyperframes.dev` uses that URL to claim the published project and import it into the web app.
6. The user continues editing from a normal web session.

So the CLI is only responsible for packaging, upload, and printing the URL. The browser-side claim/import flow lives in the backend and web app stack.

## Routing

This PR does not expose a separate user-facing canary mode.

The CLI posts to the normal publish API host:
- `https://api2.heygen.com/v1/hyperframes/projects/publish`

Backend routing behavior is handled server-side. If the default path routes through canary, it does so without a dedicated CLI flag; if that path is unavailable, traffic falls back to prod behavior on the backend side.

## What Changed

| File | Role |
|---|---|
| `packages/cli/src/commands/publish.ts` | Adds the `hyperframes publish` command, confirmation prompt, lint-before-upload behavior, and user-facing output. |
| `packages/cli/src/utils/publishProject.ts` | Zips the local project, filters ignored files/directories, posts the archive to the publish API, and returns the published project metadata. |
| `packages/cli/src/utils/publishProject.test.ts` | Covers archive creation and successful upload response parsing. |
| `packages/cli/src/cli.ts` | Registers the new `publish` command. |
| `packages/cli/src/help.ts` | Adds `publish` to root help and examples. |
| `docs/packages/cli.mdx` | Documents the persisted publish flow. |

## Important Behavior

- Requires `index.html` at the project root.
- Ignores hidden files and common non-project directories like `.git`, `node_modules`, `dist`, `.next`, and `coverage`.
- Lints the project before upload and prints findings, but does not block publish on warnings.
- Does **not** keep a local process alive after upload.
- Does **not** open a public tunnel.
- Does **not** require HeyGen OAuth inside the CLI.

## Why This Shape

This keeps the OSS CLI simple and matches the current product direction:

- project persistence lives in HeyGen's backend
- the public URL comes from the persisted project row
- claiming/importing happens on `hyperframes.dev`
- the CLI should not own browser auth or long-lived sharing infrastructure

## Verification

In the earlier PR worktree, this flow was verified locally with the CLI build/test path and with real backend integration.

In this cleanup worktree, the narrow code/doc change was verified by inspection, but the repo-level commands are currently blocked here by missing local tool binaries and typings in the worktree environment:

- `bun run --filter @hyperframes/cli test` -> `vitest: command not found`
- `bun run --filter @hyperframes/cli typecheck` -> local dependency/type resolution failures outside this diff
- `bun run --filter @hyperframes/cli build` -> `tsx: command not found`

## Notes

This PR only covers the OSS CLI side of the flow.

The full end-to-end experience depends on the corresponding backend and `hyperframes.dev` changes that store published projects, return the stable URL, and support claim/import in the web app.
 2026-04-23 04:11:34 +02:00
Miguel Ángel 64779a0c16 chore: release v0.4.14 2026-04-22 22:07:21 -04:00
Miguel Ángel 95bf333895 fix: stabilize apple master timeline and playback (#419) 
## Summary
- preserve authored non-root composition timing before runtime sanitization so Studio can build the correct master timeline for chained subcompositions
- prefer the fresh runtime source in Studio dev so local preview does not serve a stale `/api/runtime.js`
- restrict preserved authored timing inference to the Studio timeline payload instead of the general runtime resolver

## What this fixes
This PR fixes the Apple presentation class of failures where the root `index.html` / `Master` view looked correct at first and then collapsed into an incorrect short timeline.

Before this change:
- the master transport could report a short duration like `0:12` instead of the real deck length (`2:21` in the Apple project)
- composition clips bunched near the start instead of laying out sequentially across the deck
- seeking into later parts of the deck would land in the wrong place or show the wrong active composition
- local Studio debugging could be misleading because dev sometimes served a stale runtime bundle

After this change:
- the master transport reflects the authored composition-chain duration
- master clips resolve linearly across the whole deck
- late seeks land on the correct slide window
- Studio dev uses the current runtime implementation, so local preview matches the branch you are testing

## Root cause
There were two related issues:

1. Studio/master timeline inference lost authored composition timing
- missing timing attrs were treated like `0` instead of `null`
- non-root composition `data-duration` / `data-end` were stripped before Studio timing resolution could use them
- root duration inference trusted an incomplete live timeline window instead of the authored composition chain

2. Preserved authored timing leaked into the general runtime resolver
- preserving authored timing was correct for Studio timeline payload generation
- but using those preserved attrs for normal runtime playback/render resolution caused visual regressions in producer CI
- the follow-up fix keeps authored timing available only for Studio payload collection while normal runtime playback continues to resolve from the real live timeline/media state

## Why the later regression fix was needed
The initial runtime change fixed the Apple master timeline, but it also widened timing inference in the core runtime too far. That caused Dockerized producer regressions because rendered visibility started respecting preserved authored timing where it should have relied on the live resolved runtime state.

The latest commit fixes that by splitting the behavior:
- Studio timeline payload: authored timing allowed
- general runtime resolver: authored timing ignored by default

That preserves the Apple master timeline fix without changing producer render semantics.

## Verification
### Local checks
- `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/startResolver.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/startResolver.test.ts packages/core/src/runtime/timeline.test.ts packages/studio/vite.config.ts packages/cli/src/server/studioServer.ts`
- `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/startResolver.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/startResolver.test.ts packages/core/src/runtime/timeline.test.ts packages/studio/vite.config.ts packages/cli/src/server/studioServer.ts`
- `bun run --filter @hyperframes/core typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/cli typecheck`
- `cd packages/core && bun run test src/runtime/startResolver.test.ts src/runtime/timeline.test.ts`
- `bun test packages/cli/src/server/studioServer.test.ts --timeout 20000`

### Browser proof
Tested in Studio with `agent-browser` against the Apple presentation project.
- root/master transport now shows `0:00 / 2:21`
- master clip manifest resolves sequentially (`slide-1 -> slide-2 -> slide-3 ...`)
- seeking to `120s` lands on a late slide instead of a collapsed early timeline state
- after refreshing onto the fresh runtime source, the visible later-slide media advanced correctly in local Studio playback

### CI-equivalent regression proof on devbox
The previously failing producer regressions were rerun on devbox using the same Dockerized path GitHub Actions uses:
- `docker build -f Dockerfile.test -t hyperframes-producer:test .`
- `docker run ... hyperframes-producer:test style-1-prod style-5-prod style-9-prod style-12-prod --sequential`

Those previously failing suites all passed after the runtime split fix:
- `style-1-prod`
- `style-5-prod`
- `style-9-prod`
- `style-12-prod`

## Notes
- the Apple project volume tweak stayed local-only for testing and is not part of this PR
- this PR fixes the master/root timeline bug and the runtime regression it introduced; it does not add general subtimeline authoring support
 2026-04-23 04:05:11 +02:00
Vance Ingalls 80e7cd2844 perf(player): p0-1c live-playback parity test via SSIM (#401) 
## Summary

Adds **scenario 06: live-playback parity** — the third and final tranche of the P0-1 perf-test buildout (`p0-1a` infra → `p0-1b` fps/scrub/drift → this).

The scenario plays the `gsap-heavy` fixture, freezes it mid-animation, screenshots the live frame, then synchronously seeks the same player back to that exact timestamp and screenshots the reference. The two PNGs are diffed with `ffmpeg -lavfi ssim` and the resulting average SSIM is emitted as `parity_ssim_min`. Baseline gate: **SSIM ≥ 0.95**.

This pins the player's two frame-production paths (the runtime's animation loop vs. `_trySyncSeek`) to each other visually, so any future drift between scrub and playback fails CI instead of silently shipping.

## Motivation

`<hyperframes-player>` produces frames two different ways:

1. **Live playback** — the runtime's animation loop advances the GSAP timeline frame-by-frame.
2. **Synchronous seek** (`_trySyncSeek`, landed in #397) — for same-origin embeds, the player calls into the iframe runtime's `seek()` directly and asks for a specific time.

These paths must agree. If they don't — different rounding, different sub-frame sampling, different state ordering — scrubbing a paused composition shows different pixels than a paused-during-playback frame at the same time. That's a class of bug that only surfaces visually, never in unit tests, and only at specific timestamps where many things are mid-flight.

`gsap-heavy` is a 10s composition with 60 tiles each running a staggered 4s out-and-back tween. At t=5.0s a large fraction of those tiles are mid-flight, so the rendered frame has many distinct, position-sensitive pixels — the worst-case input for any sub-frame disagreement. If the two paths produce identical pixels here, they'll produce identical pixels everywhere that matters.

## What changed

- **`packages/player/tests/perf/scenarios/06-parity.ts`** — new scenario (~340 lines). Owns capture, seek, screenshot, SSIM, artifact persistence, and aggregation.
- **`packages/player/tests/perf/index.ts`** — register `parity` as a scenario id, default-runs = 3, dispatch to `runParity`, include in the default scenario list.
- **`packages/player/tests/perf/perf-gate.ts`** — extend `PerfBaseline` with `paritySsimMin`.
- **`packages/player/tests/perf/baseline.json`** — `paritySsimMin: 0.95`.
- **`.github/workflows/player-perf.yml`** — add a `parity` shard (3 runs) to the matrix alongside `load` / `fps` / `scrub` / `drift`.

## How the scenario works

The hard part is making the two captures land on the *exact same timestamp* without trusting `postMessage` round-trips or arbitrary `setTimeout` settling.

1. **Install an iframe-side rAF watcher** before issuing `play()`. The watcher polls `__player.getTime()` every animation frame and, the first time `getTime() >= 5.0`, calls `__player.pause()` *from inside the same rAF tick*. `pause()` is synchronous (it calls `timeline.pause()`), so the timeline freezes at exactly that `getTime()` value with no postMessage round-trip. The watcher's Promise resolves with that frozen value as the canonical `T_actual` for the run.
2. **Confirm `isPlaying() === true`** via `frame.waitForFunction` before awaiting the watcher. Without this, the test can hang if `play()` hasn't kicked the timeline yet.
3. **Wait for paint** — two `requestAnimationFrame` ticks on the host page. The first flushes pending style/layout, the second guarantees a painted compositor commit. Same paint-settlement pattern as `packages/producer/src/parity-harness.ts`.
4. **Screenshot the live frame** — `page.screenshot({ type: "png" })`.
5. **Synchronously seek to `T_actual`** — call `el.seek(capturedTime)` on the host page. The player's public `seek()` calls `_trySyncSeek` which (same-origin) calls `__player.seek()` synchronously, so no postMessage await is needed. The runtime's deterministic `seek()` rebuilds frame state at exactly the requested time.
6. **Wait for paint** again, screenshot the reference frame.
7. **Diff with ffmpeg** — `ffmpeg -hide_banner -i reference.png -i actual.png -lavfi ssim -f null -`. ffmpeg writes per-channel + overall SSIM to stderr; we parse the `All:` value, clamp at 1.0 (ffmpeg occasionally reports 1.000001 on identical inputs), and treat it as the run's score.
8. **Persist artifacts** under `tests/perf/results/parity/run-N/` (`actual.png`, `reference.png`, `captured-time.txt`) so CI can upload them and so a failed run is locally reproducible. Directory is already gitignored via the existing `packages/player/tests/perf/results/` rule.

### Aggregation

`min()` across runs, **not** mean. We want the *worst observed* parity to pass the gate so a single bad run can't get masked by averaging. Both per-run scores and the aggregate are logged.

### Output metric

| name              | direction        | baseline             |
|-------------------|------------------|----------------------|
| `parity_ssim_min` | higher-is-better | `paritySsimMin: 0.95` |

With deterministic rendering enabled in the runner, identical pixels produce SSIM very close to 1.0; the 0.95 threshold leaves headroom for legitimate fixture-level noise (font hinting, GPU compositor variance) while still catching any real disagreement between the two paths.

## Test plan

- `bun run player:perf -- --scenarios=parity --runs=3` locally on `gsap-heavy` — passes with SSIM ≈ 0.999 across all 3 runs.
- Inspected `results/parity/run-1/actual.png` and `reference.png` side-by-side — visually identical.
- Inspected `captured-time.txt` to confirm `T_actual` lands just past 5.0s (within one frame).
- Sanity test: temporarily forced a 1-frame offset between live and reference capture; SSIM dropped well below 0.95 as expected, confirming the threshold catches real drift.
- CI: `parity` shard added alongside the existing `load` / `fps` / `scrub` / `drift` shards; same `measure`-mode / artifact-upload / aggregation flow.
- `bunx oxlint` and `bunx oxfmt --check` clean on the new scenario.

## Stack

This is the top of the perf stack:

1. #393 `perf/x-1-emit-performance-metric` — performance.measure() emission
2. #394 `perf/p1-1-share-player-styles-via-adopted-stylesheets` — adopted stylesheets
3. #395 `perf/p1-2-scope-media-mutation-observer` — scoped MutationObserver
4. #396 `perf/p1-4-coalesce-mirror-parent-media-time` — coalesce currentTime writes
5. #397 `perf/p3-1-sync-seek-same-origin` — synchronous seek path (the path this PR pins)
6. #398 `perf/p3-2-srcdoc-composition-switching` — srcdoc switching
7. #399 `perf/p0-1a-perf-test-infra` — server, runner, perf-gate, CI
8. #400 `perf/p0-1b-perf-tests-for-fps-scrub-drift` — fps / scrub / drift scenarios
9. **#401 `perf/p0-1c-live-playback-parity-test` ← you are here**

With this PR landed the perf harness covers all five proposal scenarios: `load`, `fps`, `scrub`, `drift`, `parity`.
 2026-04-22 18:15:46 -07:00
Vance Ingalls 6f05fabbf8 perf(player): p0-1b perf tests for fps, scrub latency, and media sync drift (#400) 
## Summary

Second slice of `P0-1` from the player perf proposal: plugs the three steady-state scenarios — sustained playback FPS, scrub latency, and media-sync drift — into the perf gate that landed in #399. Adds the multi-video fixture they all share, wires three new shards into CI, and seeds one new baseline (`droppedFramesMax`).

## Why

#399 stood up the harness and proved it with a single load-time scenario. By itself that's enough to catch regressions in initial composition setup, but it can't catch the things players actually fail at in production:

- **FPS regressions** — a render-loop change that drops the ticker from 60 to 45 fps still loads fast.
- **Scrub latency regressions** — the inline-vs-isolated split (#397) is exactly the kind of code path where a refactor can silently push everyone back to the postMessage round trip.
- **Media drift** — runtime mirror logic (#396 in this stack) and per-frame scheduling tweaks can both cause video to slip out of sync with the composition clock without producing a single console error.

Each of these is a target metric in the proposal with a concrete budget. This PR turns those budgets into gated CI signals and produces continuous data for them on every player/core/runtime change.

## What changed

### Fixture — `packages/player/tests/perf/fixtures/10-video-grid/`

- `index.html`: 10-second composition, 1920×1080, 30 fps, with 10 simultaneously-decoding video tiles in a 5×2 grid plus a subtle GSAP scale "breath" on each tile (so the rAF/RVFC loops have real work to do without GSAP dominating the budget the decoder needs).
- `sample.mp4`: small (~190 KB) clip checked in so the fixture is hermetic — no external CDN dependency, identical bytes on every run.
- Same `data-composition-id="main"` host pattern as `gsap-heavy`, so the existing harness loader works without changes.

### `02-fps.ts` — sustained playback frame rate

- Loads `10-video-grid`, calls `player.play()`, samples `requestAnimationFrame` callbacks inside the iframe for 5 s.
- Crucial sequencing: install the rAF sampler **before** `play()`, wait for `__player.isPlaying() === true`, **then reset the sample buffer** — otherwise the postMessage round-trip ramp-up window drags the average down by 5–10 fps.
- FPS = `(samples − 1) / (lastTs − firstTs in s)`; uses rAF timestamps (the same ones the compositor saw) rather than wall-clock `setTimeout`, so we're measuring real frame production.
- Dropped-frame definition matches Chrome DevTools: gap > 1.5× (1000/60 ms) ≈ 25 ms = "missed at least one vsync."
- Aggregation across runs: `min(fps)` and `max(droppedFrames)` — worst case wins, since the proposal asserts a floor on fps and a ceiling on drops.
- Emits `playback_fps_min` (higher-is-better, baseline `fpsMin = 55`) and `playback_dropped_frames_max` (lower-is-better, baseline `droppedFramesMax = 3`).

### `04-scrub.ts` — scrub latency, inline + isolated

- Loads `10-video-grid`, pauses, then issues 10 seek calls in two batches: first the synchronous **inline** path (`<hyperframes-player>`'s default same-origin `_trySyncSeek`), then the **isolated** path (forced by replacing `_trySyncSeek` with `() => false`, which makes the player fall back to the postMessage `_sendControl("seek")` bridge that cross-origin embeds and pre-#397 builds use).
- Inline runs first so the isolated mode's monkey-patch can't bleed back into the inline samples.
- Detection: a rAF watcher inside the iframe polls `__player.getTime()` until it's within `MATCH_TOLERANCE_S = 0.05 s` of the requested target. Tolerance exists because the postMessage bridge converts seconds → frame number → seconds, and that round-trip can introduce sub-frame quantization drift even for targets on the canonical fps grid.
- Timing: `performance.timeOrigin + performance.now()` in both contexts. `timeOrigin` is consistent across same-process frames, so `t1 − t0` is a true wall-clock latency, not a host-only or iframe-only stopwatch.
- Targets alternate forward/backward (`1.0, 7.0, 2.0, 8.0, 3.0, 9.0, 4.0, 6.0, 5.0, 0.5`) so no two consecutive seeks land near each other — protects the rAF watcher from matching against a stale `getTime()` value before the seek command is processed.
- Aggregation: `percentile(95)` across the pooled per-seek latencies from every run. With 10 seeks × 2 modes × 3 runs we get 30 samples per mode per CI shard, enough for a stable p95.
- Emits `scrub_latency_p95_inline_ms` (lower-is-better, baseline `scrubLatencyP95InlineMs = 33`) and `scrub_latency_p95_isolated_ms` (lower-is-better, baseline `scrubLatencyP95IsolatedMs = 80`).

### `05-drift.ts` — media sync drift

- Loads `10-video-grid`, plays 6 s, instruments **every** `video[data-start]` element with `requestVideoFrameCallback`. Each callback records `(compositionTime, actualMediaTime)` plus a snapshot of the clip transform (`clipStart`, `clipMediaStart`, `clipPlaybackRate`).
- Drift = `|actualMediaTime − ((compTime − clipStart) × clipPlaybackRate + clipMediaStart)|` — the same transform the runtime applies in `packages/core/src/runtime/media.ts`, snapshotted once at sampler install so the per-frame work is just subtract + multiply + abs.
- Sustain window is 6 s (not the proposal's 10 s) because the fixture composition is exactly 10 s long and we want headroom before the end-of-timeline pause/clamp behavior. With 10 videos × ~25 fps × 6 s we still pool ~1500 samples per run — more than enough for a stable p95.
- Same "reset buffer after play confirmed" gotcha as `02-fps.ts`: frames captured during the postMessage round-trip would compare a non-zero `mediaTime` against `getTime() === 0` and inflate drift by hundreds of ms.
- Aggregation: `max()` and `percentile(95)` across the pooled per-frame drifts. The proposal's max-drift ceiling of 500 ms is intentional — the runtime hard-resyncs when `|currentTime − relTime| > 0.5 s`, so a regression past 500 ms means the corrective resync kicked in and the viewer saw a jump.
- Emits `media_drift_max_ms` (lower-is-better, baseline `driftMaxMs = 500`) and `media_drift_p95_ms` (lower-is-better, baseline `driftP95Ms = 100`).

### Wiring

- `packages/player/tests/perf/index.ts`: add `fps`, `scrub`, `drift` to `ScenarioId`, `DEFAULT_RUNS`, the default scenario list (`--scenarios` defaults to all four), and three new dispatch branches.
- `packages/player/tests/perf/perf-gate.ts`: add `droppedFramesMax: number` to `PerfBaseline`. Other baseline keys for these scenarios were already seeded in #399.
- `packages/player/tests/perf/baseline.json`: add `droppedFramesMax: 3`.
- `.github/workflows/player-perf.yml`: three new matrix shards (`fps` / `scrub` / `drift`) at `runs: 3`. Same `paths-filter` and same artifact-upload pattern as the `load` shard, so the summary job aggregates them automatically.

## Methodology highlights

These three patterns recur in all three scenarios and are worth noting because they're load-bearing for the numbers we report:

1. **Reset buffer after play-confirmed.** The `play()` API is async (postMessage), so any samples captured before `__player.isPlaying() === true` belong to ramp-up, not steady-state. Both `02-fps` and `05-drift` clear `__perfRafSamples` / `__perfDriftSamples` *after* the wait. Without this, fps drops 5–10 and drift inflates by hundreds of ms.
2. **Iframe-side timing.** All three scenarios time inside the iframe (`performance.timeOrigin + performance.now()` for scrub, rAF/RVFC timestamps for fps/drift) rather than host-side. The iframe is what the user sees; host-side timing would conflate Puppeteer's IPC overhead with real player latency.
3. **Stop sampling before pause.** Sampler is deactivated *before* `pause()` is issued, so the pause command's postMessage round-trip can't perturb the tail of the measurement window.

## Test plan

- [x] Local: `bun run player:perf` runs all four scenarios end-to-end on the 10-video-grid fixture.
- [x] Each scenario produces metrics matching its declared `baselineKey` so `perf-gate.ts` can find them.
- [x] Typecheck, lint, format pass on the new files.
- [x] Existing player unit tests untouched (no production code changes in this PR).
- [ ] First CI run will confirm the new shards complete inside the workflow timeout and that the summary job picks up their `metrics.json` artifacts.

## Stack

Step `P0-1b` of the player perf proposal. Builds on:

- `P0-1a` (#399): the harness, runner, gate, and CI workflow this PR plugs new scenarios into.

Followed by:

- `P0-1c` (#401): `06-parity` — live playback frame vs. synchronously-seeked reference frame, compared via SSIM, on the existing `gsap-heavy` fixture from #399.
 2026-04-22 18:10:08 -07:00
Vance Ingalls 10d2725b54 perf(player): p0-1a perf test infra + composition-load smoke test (#399) 
## Summary

First slice of `P0-1` from the player perf proposal: lays the foundation for a player perf gate so later PRs can plug in fps / scrub / drift / parity scenarios without rebuilding infrastructure. Ships one smoke scenario (`03-load`, cold + warm composition load) to prove the gate end-to-end on real numbers.

## Why

There was no automated way to catch player perf regressions. Every perf concern in the existing proposal — composition load time, sustained FPS, scrub p95, mirror-clock drift, live-vs-seek parity — needs the same plumbing: a same-origin harness, a Puppeteer runner, a baseline file, a gate that emits structured results, and a CI workflow that runs the right scenarios on the right changes. Building that up-front in one reviewable PR lets every subsequent perf PR (`P0-1b`, `P0-1c`, and beyond) be a 100-line scenario file plus a baseline entry instead of re-litigating the framework.

## What changed

### Harness — `packages/player/tests/perf/server.ts`

- `Bun.serve` on a free port, single same-origin host for the player IIFE bundle, hyperframe runtime, GSAP from `node_modules`, and fixture HTML.
- Same-origin matters: cross-origin would force every probe through `postMessage`, hiding bugs and inflating numbers in ways production never sees. Tests should measure the path the studio editor actually takes.
- Routes:
  - `/player.js` → built IIFE bundle (rebuilt on demand).
  - `/vendor/runtime.js`, `/vendor/gsap.min.js` → resolved from `node_modules` so fixtures don't need to ship copies.
  - `/fixtures/*` → fixture HTML.

### Runner — `packages/player/tests/perf/runner.ts`

- `puppeteer-core` thin wrappers (`launchBrowser`, `loadHostPage`).
- Uses the system Chrome detected by `setup-chrome` in CI rather than the bundled puppeteer revision — keeps the action smaller, lets us pin Chrome version policy at the workflow level, and matches what users actually run.

### Gate — `packages/player/tests/perf/perf-gate.ts` + `baseline.json`

- Loads `baseline.json` (initial budgets: cold/warm comp load, fps, scrub p95 isolated/inline, drift max/p95) with a 10% `allowedRegressionRatio`.
- Per-metric direction (`lower-is-better` / `higher-is-better`) so the same evaluator handles latency and throughput.
- Returns a structured `GateReport` consumed by both the CLI (table output) and `metrics.json` (CI artifact).
- Two modes: `measure` (log only — used during the rollout) and `enforce` (fail the build) — flip per-metric once we trust the signal, without touching the harness.

### CLI orchestrator — `packages/player/tests/perf/index.ts`

- Parses `--mode` / `--scenarios` / `--runs` / `--fixture` in both space- and equals-separated form (so `--scenarios fps,scrub` and `--scenarios=fps,scrub` both work — matches what humans type and what GitHub Actions emits).
- Runs scenarios, runs the gate, and **always** writes `results/metrics.json` with schema version, git SHA, metrics, and gate rows — so failed runs are still investigable from the artifact alone.

### Fixture + smoke scenario

- `fixtures/gsap-heavy/index.html`: 200 stagger-animated tiles, no media. Heavy enough to make load time meaningful, light enough to be deterministic.
- `scenarios/03-load.ts`: cold + warm composition load. Measures from navigation start to player `ready` event, reports p95 across runs.

### CI — `.github/workflows/player-perf.yml`

- `paths-filter` on `player` / `core` / `runtime` — perf only runs when something that could move the needle actually changed.
- Sets up bun + node + chrome, runs perf in `measure` mode on a shard matrix (so future scenarios shard naturally), uploads `metrics.json` artifacts, and a summary job aggregates shard results into a single PR comment.

### Wiring

- `packages/player`: `puppeteer-core`, `gsap`, `@types/bun` devDeps; typecheck extended to cover the perf `tsconfig`; new `perf` script.
- Root `package.json`: `player:perf` workspace script so `bun run player:perf` runs the whole suite locally with the same flags CI uses.
- `.gitignore`: `packages/player/tests/perf/results/`.
- Separate `tests/perf/tsconfig.json` so test code doesn't pollute the package `rootDir` while still being typechecked.

## Test plan

- [x] Local: `bun run player:perf` passes — cold p95 ≈ 386 ms, warm p95 ≈ 375 ms, both well under the seeded baselines.
- [x] Typecheck, lint, format pass on the perf workspace.
- [x] Existing player unit tests (71/71) still green.
- [ ] First CI run after merge will be the real signal: confirms `setup-chrome` works on hosted runners, the shard matrix wires up, and `metrics.json` artifacts upload.

## Stack

Step `P0-1a` of the player perf proposal. The next two slices are content-only — they don't touch the harness:

- `P0-1b` (#400): adds `02-fps`, `04-scrub`, `05-drift` scenarios on a 10-video-grid fixture.
- `P0-1c` (#401): adds `06-parity` (live playback vs. synchronously-seeked reference, compared via SSIM).

Wiring this gate up first means each follow-up is a self-contained scenario file + baseline row + workflow shard.
 2026-04-22 18:04:05 -07:00
Vance Ingalls 150d9348bc perf(player): srcdoc composition switching for studio (#398) 
## Summary

Adds `srcdoc` support to `<hyperframes-player>` and uses it from studio's `Player.tsx` so composition switches no longer trigger an iframe navigation. Studio fetches the composition HTML on the parent and hands it to the iframe inline; the browser skips the navigation request, preconnect/handshake, and a redundant cache lookup.

## Why

Step `P3-2` of the player perf proposal. Profiling studio's project switcher showed that ~30–80 ms of every composition swap was spent in the iframe's own navigation pipeline — DNS / TCP / TLS reuse checks, request hand-off to the network process, and the second cache lookup against the same origin we just fetched from. For same-origin previews (`/api/projects/.../preview`) this is pure overhead: the parent already has the bytes (or can pull them from its own HTTP cache).

`srcdoc` lets us skip that pipeline entirely. The iframe loads from an in-memory string and the parent's `fetch` reuses any existing response from the page's HTTP cache, so the second-and-Nth composition switch in a session is essentially free at the network layer.

## What changed

### `<hyperframes-player>` (`packages/player/src/hyperframes-player.ts`)

- Added `srcdoc` to `observedAttributes` so runtime swaps actually fire `attributeChangedCallback`.
- On connect, both `srcdoc` and `src` are forwarded to the inner iframe — no manual precedence; the HTML spec already says `srcdoc` wins when both are present, so the browser handles arbitration.
- New `srcdoc` branch in `attributeChangedCallback`:
  - Resets `_ready = false` on every change so the next iframe `load` event re-runs probe/control/poster setup against the fresh document.
  - Distinguishes `setAttribute("srcdoc", "")` (deliberate empty document) from `removeAttribute("srcdoc")` (fall back to `src`) — the former propagates an empty-string srcdoc; the latter strips the attribute so a previously-set `src` can take over.

### Studio `Player.tsx` (`packages/studio/src/player/components/Player.tsx`)

- Hoisted `AbortController` and resolved `url` outside the dynamic-import `.then()` so the cleanup function can cancel an in-flight composition fetch when the user navigates away mid-load.
- After the player module loads, `fetch(url, { signal })` pulls the composition HTML on the parent.
  - Success → `player.setAttribute("srcdoc", html)`.
  - Network error / non-2xx → fall back to `player.setAttribute("src", url)`. Same code path the player has always taken, so this optimization is strictly a win — never a regression.
  - `AbortError` → bail without touching the DOM (component is unmounting).
- Attributes are set **before** `appendChild` so the iframe never loads an intermediate `about:blank`. That matters because:
  1. The first iframe `load` event must fire for the real composition; the existing handler treats `loadCountRef > 1` as a hot-reload and replays the reveal animation. An extra `about:blank` load would trigger the reveal on initial mount.
  2. `useTimelinePlayer` hangs setup off the first load — running it against an empty document is wasted work.

## Test plan

- [x] 7 new unit tests in `hyperframes-player.test.ts` covering:
  - `srcdoc` is in `observedAttributes`.
  - Initial `srcdoc` set before connect forwards to the iframe on connect.
  - Runtime `srcdoc` set after connect forwards via `attributeChangedCallback`.
  - `_ready` resets when `srcdoc` changes so `onIframeLoad` replays setup.
  - `removeAttribute("srcdoc")` strips the attribute on the iframe so `src` can take over.
  - Empty-string `srcdoc` is preserved (not treated as removal).
  - Both `src` and `srcdoc` set together: both get forwarded to the iframe and the browser arbitrates per spec.
- [x] Studio fallback path verified manually — disabling fetch falls back to the original `src` flow with no regression.

## Stack

Step `P3-2` of the player perf proposal. Builds on `P3-1` (sync seek) — both target the studio editor's interactive feel. With sync seek removing scrub latency and `srcdoc` removing composition-switch latency, the editor's two most-frequent interactions both shed their iframe-navigation overhead.
 2026-04-22 17:59:01 -07:00
Vance Ingalls ef3de5bcd3 feat(player): synchronous seek() API with same-origin detection (#397) 
## Summary

Formalizes the same-origin shortcut Studio has been using privately (`iframe.contentWindow.__player.seek` in `useTimelinePlayer.ts`) as a first-class behavior of `<hyperframes-player>`'s public `seek()` method. Same-origin seeks now land in the same task as the input event — no postMessage hop, no extra microtask, no perceived scrub lag. Cross-origin embeds fall through to the existing async bridge transparently.

## Why

Step `P3-1` of the player perf proposal. The current `seek()` always posts a message to the iframe runtime, which means a single user scrub incurs:

1. JS task: fire postMessage from parent
2. Browser task switch into iframe context
3. Microtask: handler dispatches
4. Frame: runtime calls `markExplicitSeek` and updates DOM

Same-origin embeds (Studio, preview pane, embedded compositions) can skip all four by calling the runtime's `seek` directly. Studio was already doing this manually but had to duplicate the local-state bookkeeping (`_currentTime`, `paused`, controls UI) — making it a first-class behavior of the player removes the workaround and gives every same-origin consumer the win for free.

## What changed

- New `_trySyncSeek(time)` helper attempts a synchronous call into the iframe's `window.__player.seek`. Returns `true` on success, `false` on cross-origin or pre-bootstrap.
- `seek()` calls `_trySyncSeek` first, falls through to the existing `_sendControl` postMessage path when sync isn't available.
- Detection is a `try/catch` on `contentWindow` access (real cross-origin iframes throw `SecurityError`) plus a `typeof` guard on `__player.seek`.
- Local `_currentTime`, the `paused` flag, and the controls UI update on both paths so scrubs never leave stale state.
- Runtime-side `seek` is the same wrapped function the postMessage handler calls — `installRuntimeControlBridge` routes through `player.seek`, so `markExplicitSeek()` and downstream runtime state are identical between the two paths.

## Test plan

- [x] 11 new unit tests in `hyperframes-player.test.ts` covering:
  - Same-origin sync path executes `__player.seek` synchronously and skips postMessage.
  - Cross-origin (simulated `SecurityError` on `contentWindow`) falls back to postMessage.
  - Pre-bootstrap (no `__player` installed) falls back to postMessage.
  - `__player.seek` not a function falls back to postMessage.
  - `_currentTime`, `paused`, and controls all stay in sync on both paths.
  - Errors thrown from `__player.seek` propagate without corrupting state.

## Stack

Step `P3-1` of the player perf proposal. Independent of the `P1-*` work — this is a pure latency win on the seek/scrub path. Combined with `P3-2` (srcdoc composition switching, next in the stack) it removes most of the iframe-bridge overhead from the studio scrubber.
 2026-04-22 17:50:23 -07:00
Vance Ingalls f906797222 perf(player): coalesce _mirrorParentMediaTime writes (#396) 
## Summary

Coalesce writes to `el.currentTime` inside `_mirrorParentMediaTime` so a single jitter sample no longer triggers a parent-media seek. A drift correction now requires **two consecutive samples** above the threshold (~`MIRROR_DRIFT_THRESHOLD_SECONDS`) before the player writes back. One-shot alignment paths (`promoteToParentProxy`, `_onIframeMediaAdded`) opt out via `force: true` so initial alignment stays immediate.

## Why

Step `P1-4` of the player perf proposal. `_mirrorParentMediaTime` is called every animation frame on parent media proxies. Even without true drift, browser internals report tiny jitter on `currentTime` reads — typically below 30 ms but occasionally crossing the threshold for a frame. Writing to `currentTime` triggers a seek, which is expensive *and* invalidates pipeline buffers, which causes the next frame's reading to jitter further. The result was unnecessary seek thrash on otherwise-aligned media.

By requiring two consecutive over-threshold samples, transient jitter is filtered out while real drift (a sustained offset) still corrects within ~1 frame of latency. This eliminates the most common cause of dropped frames on the studio thumbnail grid.

## What changed

- Each `_parentMedia` entry gains a `driftSamples` counter that increments while the absolute drift is above `MIRROR_DRIFT_THRESHOLD_SECONDS` and resets to 0 on the first sample below.
- `_mirrorParentMediaTime(el, opts)` only writes back when `driftSamples >= 2`, except when `opts.force === true`.
- `promoteToParentProxy` and `_onIframeMediaAdded` pass `force: true` so the first alignment after registration is still immediate (these are user-visible state transitions, not steady-state telemetry).

## Test plan

- [x] 11 new unit/integration tests in `hyperframes-player.test.ts` covering:
  - Single-sample jitter does not trigger a write.
  - Two-sample sustained drift does trigger a write.
  - Trending drift correction (gradually increasing offset) is detected within 2 samples.
  - `force: true` override bypasses the sample requirement.
  - Out-of-range proxies (proxies whose source has been removed) do not panic.
  - Multiple proxies maintain independent counters — drift on one does not affect the other.
  - `_promoteToParentProxy` alignment is immediate.

## Stack

Step `P1-4` of the player perf proposal. Builds on `P1-1` (shared adopted stylesheets) and `P1-2` (scoped media observer). Together these three target the studio multi-player render path — `P0-1*` perf gate scenarios will pick up the wins automatically.
 2026-04-22 17:44:49 -07:00
Vance Ingalls 9512744c2e refactor(shader-transitions): extract DEFAULT_DURATION and DEFAULT_EASE constants (#367) 
## Summary

Extract `DEFAULT_DURATION = 0.7` and `DEFAULT_EASE = "power2.inOut"` as shared constants in `hyper-shader.ts` and apply them at all three fallback sites (metadata write, browser/render mode, engine mode).

## Why

`Chunk 2` of `plans/hdr-followups.md`. The three fallback sites had drifted apart: the metadata path used `1s` / `"none"` while the actual rendering used `0.7s` / `"power2.inOut"`. A transition that omitted `duration`/`ease` would render at 0.7 s but tell the producer it was 1 s, throwing off the producer's compositing window planning and producing a visible ~0.3 s brightness dropout.

This is a small, high-value correctness fix that runs before the larger Chunk 1 / Chunk 4 work.

## What changed

- New module-level `DEFAULT_DURATION` and `DEFAULT_EASE` constants in `packages/shader-transitions/src/hyper-shader.ts`.
- All three fallback call sites (metadata, browser, engine) now use the constants.
- Explicit `ease: "none"` on the timeline-length anchor tweens elsewhere in the file is intentional (those are linear interpolators driving the shader's progress uniform) and is left unchanged.

## Test plan

- [x] Render a composition with a transition that omits `duration` and `ease` — no brightness dip in the last ~0.3 s of the transition.
- [x] Preview (browser mode) and render (engine mode) produce matching blending curves.
- [x] Render with explicit `duration: 1.5` still works (constants are fallbacks only).

## Stack

Chunk 2 of `plans/hdr-followups.md`. Lands ahead of Chunk 1 (opacity) per the suggested merge order.
 2026-04-22 17:02:42 -07:00
Vance Ingalls 5de5df7fbb refactor(types): tighten type safety, dedupe HfTransitionMeta, prune dead LUT export (#366) 
## Summary

Four small, mechanical type-safety cleanups across `engine`, `producer`, and `shader-transitions`. Zero behavior change — pure pre-cleanup so the rest of the stack ships against a tighter baseline.

## Why

`Chunk 6` of `plans/hdr-followups.md`. Several non-null assertions and a duplicate interface had accumulated as rebase artifacts and leftover work-in-progress; lands first because it touches files later chunks edit and removes friction during review.

## What changed

- `renderOrchestrator.ts`: replace `layers[layerIdx]!` with a `for (const [layerIdx, layer] of layers.entries())` so both index and element come from the iterator.
- `engine/types.ts`: drop the duplicate `HfTransitionMeta` interface (rebase artifact); the original definition above it is the documented one. The orphaned doc comment now precedes `HfProtocol`.
- `shader-transitions/hyper-shader.ts`: keep the local `HfTransitionMeta` declaration (the package ships as a standalone CDN bundle and must not depend on `@hyperframes/engine`), but add a sync comment pointing at the source of truth in `engine/src/types.ts`.
- `alphaBlit.ts` + `engine/index.ts`: drop `export` from `getSrgbToHdrLut` and remove its re-export. It was only ever called by the internal `blitRgba8OverRgb48le`; the public surface was dead code.

## Test plan

- [x] `bun run --filter @hyperframes/engine typecheck`
- [x] `bun run --filter @hyperframes/producer typecheck`
- [x] `bun run --filter @hyperframes/shader-transitions typecheck`
- [x] `bun run --filter @hyperframes/engine test` — 308/308 pass (no test changes; assertions removed in code only).

## Stack

Chunk 6 of `plans/hdr-followups.md`. Mechanical cleanup landed early per the suggested merge order.
 2026-04-22 16:56:17 -07:00
Vance Ingalls a6e14da45c perf(player): scope MutationObserver to composition hosts (#395) 
## Summary

Replace the body-wide `MutationObserver` in `<hyperframes-player>` with one scoped to top-level `[data-composition-id]` hosts. The wide observer fired on every body-level mutation — analytics scripts, runtime telemetry markers, dev overlays — even though only composition subtrees can introduce new timed media (`<audio data-start>`, etc.).

## Why

Step `P1-2` of the player perf proposal. The previous implementation observed `iframe.contentDocument.body` with `subtree: true` to pick up sub-composition `<audio data-start>` elements added after initial mount. That worked, but it was paying for callbacks from every unrelated DOM mutation in the iframe — most of which are just runtime instrumentation. Hot paths in the studio (timeline updates, telemetry markers) end up triggering the observer dozens of times per frame.

Scoping to composition hosts cuts the noise by ~10× in the studio without losing any of the timed-media wiring guarantees.

## What changed

- New `selectMediaObserverTargets(doc)` helper in `packages/player/src/mediaObserverScope.ts` that selects all top-level `[data-composition-id]` elements **excluding** nested ones — sub-composition hosts whose media is already covered by the parent observer's `subtree: true`.
- The player now attaches a single `MutationObserver` instance per top-level host (`subtree: true`), so callbacks still batch across hosts but skip out-of-host noise.
- Falls back to observing `body` when no composition hosts exist (e.g. blank iframe between `src` changes) — preserves prior behavior for non-composition documents and avoids breaking the bootstrap path.

## Test plan

- [x] 8 new unit tests in `mediaObserverScope.test.ts` covering empty docs, single host, multiple hosts, nested-host filtering, and the body-fallback path.
- [x] 2 new integration tests in `hyperframes-player.test.ts` spying on `MutationObserver.prototype.observe` to confirm the targets and options the player actually attaches in a real custom-element bootstrap.

## Stack

Step `P1-2` of the player perf proposal. Sits between `P1-1` (shared adopted stylesheets) and `P1-4` (coalescing parent media-time mirror writes) — together they target the studio multi-player render path. The perf gate scenarios in `P0-1*` will pick up the wins automatically.
 2026-04-22 16:46:54 -07:00
Vance Ingalls d7c1050e44 test(producer): add hdr-regression and hdr-hlg-regression test suites (#365) 
## Summary

Replace the trivial `hdr-pq` and `hdr-image-only` tests with two consolidated, time-windowed regression suites that exercise the full HDR pipeline. These goldens are the safety net for every other PR in this stack.

## Why

The pre-existing HDR tests covered only a single full-bleed video or image with a static text label — none of the features that the HDR pipeline has to handle differently from SDR (opacity animation, z-ordered multi-layer compositing, transforms, border-radius clipping, shader transitions, multiple HDR sources, object-fit modes, mixed HDR+SDR layering, HLG transfer). This PR builds the missing safety net first so every subsequent fix can be proven correct.

## What changed

- New `packages/producer/tests/hdr-regression/` (PQ, BT.2020, ~20 s, 1080p, 8 windows A–H):
  - A: static baseline (HDR video + DOM overlay)
  - B: wrapper-opacity fade
  - C: direct-on-`<video>` opacity tween (documents the Chunk 1 bug)
  - D: z-order sandwich (DOM → HDR → DOM)
  - E: two HDR videos side-by-side (pins PR #289)
  - F: rotation + scale + border-radius (documents the Chunk 4 bug)
  - G: `object-fit: contain`
  - H: shader crossfade between HDR video and HDR image
- New `packages/producer/tests/hdr-hlg-regression/` (HLG, ARIB STD-B67, ~5 s, 2 windows A–B) — exercises the separate HLG LUT/OETF code path that previously had **zero** coverage.
- New `scripts/generate-hdr-photo-pq.py` synthesizes `hdr-photo-pq.png` with a cICP chunk for BT.2020/PQ/full.
- Removed `tests/hdr-pq/` and `tests/hdr-image-only/`.
- Updated `.github/workflows/regression.yml` HDR shard to run the new pair sequentially.
- All compositions follow the documented timed-element pattern (`data-start`, `data-duration`, `class="clip"` directly on each timed leaf — no wrapper inheritance).

## Test plan

- [x] Goldens generated with `bun run test:update --sequential`.
- [x] `ffprobe` confirms HEVC/yuv420p10le/bt2020nc/smpte2084 (PQ) and arib-std-b67 (HLG).
- [x] Suite green with `maxFrameFailures` budgets that absorb the documented Chunk 1 / Chunk 4 known-fails — tightened in follow-up PRs in this stack.

## Stack

Foundational PR for the HDR follow-ups stack (Chunk 0 of `plans/hdr-followups.md`). Every subsequent PR builds on this safety net.
 2026-04-22 15:43:04 -07:00
Vance Ingalls ed62894d01 perf(player): share PLAYER_STYLES via adoptedStyleSheets (#394) 
## Summary

Replace per-instance `<style>` injection in `<hyperframes-player>` with a lazily constructed `CSSStyleSheet` adopted via `shadowRoot.adoptedStyleSheets`. One parsed stylesheet, many adopters — the studio thumbnail grid renders dozens of players concurrently and was paying for N parses of the same CSS.

## Why

Step `P1-1` of the player perf proposal. The previous implementation appended a `<style>` element to every shadow root, which means:

- N shadow roots → N copies of the same CSS string parsed into N independent style sheets.
- Each `<style>` lives in the DOM and contributes to layout/style invalidation work when its shadow root churns.
- The studio's project grid mounts ~30 players on initial load — that's 30 redundant parses of the same ~1 KB stylesheet on the critical path.

`adoptedStyleSheets` flips this: parse once at module load, hand the same `CSSStyleSheet` reference to every shadow root.

## What changed

- New `getSharedPlayerStyleSheet()` in `packages/player/src/styles.ts` — module-scoped and memoized; the sheet is built once per process and returned to every adopter.
- New `applyPlayerStyles(shadow)` is the single integration point. It **appends** (never replaces) the shared sheet so any pre-adopted sheets — host themes, scoped overrides, future caller-side injections — survive intact, and is idempotent so repeated calls don't multiply adoptions.
- SSR-safe via a `typeof CSSStyleSheet` guard. Failures (e.g. `replaceSync` throw, no constructor) are cached as `null` so we don't retry constructor failures forever.
- Defensive fallback path creates a per-instance `<style>` element when `adoptedStyleSheets` is unavailable (older runtimes, hostile environments). Behavior on those paths is unchanged from before.
- `PLAYER_STYLES`, `PLAY_ICON`, and `PAUSE_ICON` exports preserved — no public API change.

## Test plan

- [x] Unit tests in `styles.test.ts` cover sharing across instances, fallback when `CSSStyleSheet` is undefined or `replaceSync` throws, fallback when `adoptedStyleSheets` is unsupported on the shadow root, idempotency, and preservation of pre-existing adopted sheets.
- [x] Integration test in `hyperframes-player.test.ts` confirms two real `<hyperframes-player>` elements adopt the same `CSSStyleSheet` instance and inject zero `<style>` elements.
- [x] Build size delta is negligible (utility code replaces `container.appendChild` calls).

## Stack

Step `P1-1` of the player perf proposal. Followed by `P1-2` (scoping the media `MutationObserver`) and `P1-4` (coalescing parent media-time mirror writes) — all three target the studio multi-player render path.
 2026-04-22 15:40:33 -07:00
Vance Ingalls f9863ab565 feat(core): add emitPerformanceMetric bridge for runtime telemetry (#393) 
## Summary

Extend the runtime analytics bridge with a numeric performance metric channel. Hosts subscribe via the existing postMessage transport (one bridge, two channels) and aggregate per-session p50 / p95 for scrub latency, sustained fps, dropped frames, decoder count, composition load time, and media sync drift before forwarding to their observability pipeline.

This is the foundation other perf tooling sits on — the player itself emits the events; player-side aggregation and flush land in a follow-up.

## Why

Step `X-1` of the player perf proposal. Today there is no way for an embedding host to learn that scrub latency spiked, that a composition took 3 s to load, or that the media-sync loop is running 200 ms behind real time. The only signals are anecdotal user reports.

A single shared bridge keeps the runtime → host surface area minimal: hosts that already wire up the analytics channel get perf for free, and hosts that don't aren't paying for it.

## What changed

- New `emitPerformanceMetric(name, value, tags?)` helper in `@hyperframes/core` that forwards a `{ type: "performance-metric", name, value, tags }` envelope through the existing analytics postMessage transport.
- Six initial metric names defined in the proposal:
  - `scrub_latency_ms` — wall-clock from `seek()` call to first paint at the new frame.
  - `playback_fps` — sustained rAF cadence during play.
  - `dropped_frames` — count of >25 ms gaps within a play window.
  - `decoder_count` — number of concurrently-decoding video elements.
  - `composition_load_ms` — navigation-start to player-ready.
  - `media_sync_drift_ms` — drift between expected and actual decoder time.
- Each emit also writes a `performance.mark()` with `{ value, tags }` on `detail`, so the same numbers surface in the DevTools Performance panel's User Timing track for local debugging without instrumenting the host.
- Zero PostHog (or any other analytics SDK) dependency in `core` — the host decides where to forward the events.

## Test plan

- [x] Unit tests cover the envelope shape, the `performance.mark` mirror, and the no-op path when no host has wired up the bridge.
- [x] Manual: verified marks appear in the User Timing track when scrubbing the studio preview.

## Stack

Step `X-1` of the player perf proposal. Foundation for the perf gate (P0-1a/b/c) — the perf scenarios in this stack instrument these same channels for CI measurement.
 2026-04-22 15:08:14 -07:00
Miguel Ángel 5be207f034 chore: release v0.4.13 2026-04-22 17:28:11 -04:00
Miguel Ángel fc52d21c59 docs: clarify composition variable usage (#420) 
## Summary
- replace the unsupported `data-var-*` example with the current `data-variable-values` pattern
- document that variable values are carried through but still applied manually inside the nested composition
- add matching reference notes in the data-attributes, HTML schema, core package, and CLI docs

## Verification
- `npx mintlify dev --port 3100`
- browser verification with `agent-browser` on `/concepts/compositions` and `/reference/html-schema`
- proof artifacts saved locally under `tmp/issue-416-docs/`
 2026-04-22 20:41:35 +02:00
Miguel Ángel b6f50ce4c7 chore: release v0.4.13-alpha.4 2026-04-22 12:48:35 -04:00
Miguel Ángel 5a4dd8bec1 fix: gate studio timeline actions by capability (#415) 
## Summary
- gate timeline actions to clips Studio can control deterministically
- disable direct move/trim for generic GSAP-timed DOM clips
- add an in-clip `Copy to Agent` fallback for unsupported edits

## Why
Studio should only advertise timeline actions it can round-trip to source HTML with deterministic meaning.

This PR now follows that stricter rule:
- direct move/end-trim are only exposed for clips with a deterministic timeline window
- start trim is only exposed for clips with a real content-offset model
- unsupported motion clips now offer `Copy to Agent` so users still have a fast path to request source-level timing changes

In practice this means generic GSAP-authored DOM clips no longer pretend Studio can rewrite their visible timing just by patching `data-start` / `data-duration`.

## What changed
- added `hasPatchableTimelineTarget()` and `getTimelineEditCapabilities()` in `timelineEditing.ts`
- tightened deterministic-window detection so only media, images, and composition hosts keep direct move/end-trim controls
- kept wrapped media clips editable by recognizing real media metadata even when the host tag is a `div`
- updated `TimelineClip` / `Timeline` to guard interactions with the shared capability model
- added `buildTimelineElementAgentPrompt()` and a `Copy to Agent` fallback button for unsupported clips
- added focused tests for capability derivation and the agent-prompt helper

## Verification
### Automated
- `bun test packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/utils/sourcePatcher.test.ts`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/TimelineClip.tsx`
- `bunx oxfmt --check packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/TimelineClip.tsx`

### Browser
Verified in Studio with live browser automation against `http://127.0.0.1:4175/#project/timeline-edit-playground`:
- generic GSAP-timed clips (`feature-card`, `title-card`, `prompt-card`) show `Copy to Agent` and no direct move/trim affordances
- wrapped media (`media-card`) still exposes direct controls and remains draggable
- the local playground timings were realigned to match the authored GSAP positions, so preview visibility now matches the timeline windows during manual testing

Recording artifacts used during verification:
- `/tmp/timeline-capabilities-proof/capabilities-flow.webm`
- `/tmp/timeline-capabilities-proof/capabilities-agent-flow.webm`
 2026-04-22 18:44:17 +02:00
Miguel Ángel c46abf9fa2 chore: release v0.4.13-alpha.3 2026-04-22 11:14:31 -04:00
Miguel Ángel 2cf3558f8e fix(studio): only expose front trim for offsettable clips (#413) 
## Summary
- hide the leading trim handle for timeline clips that cannot offset their own content
- keep leading trim available for media clips backed by playback offset metadata or source duration
- map visual row priority like a normal timeline editor: top timeline rows render above lower rows

## Why This Is Needed
Generic GSAP/DOM timeline clips do not have a playback-offset model like media clips do.

That means a left trim affordance on those clips is misleading today:
- users reasonably expect front trim to remove the beginning of the animation
- the current model can only shorten the clip window, not start the motion halfway through

Instead of exposing a control that implies unsupported behavior, this PR keeps true front trim only on clips that can actually offset their content.

The PR also fixes the stacking convention so the timeline matches normal editor expectations:
- visually higher track row = higher render priority
- visually lower track row = lower render priority

## Current Flow By Element Type
### Generic motion / DOM clips
Examples: `section`, `div`, `aside`, GSAP-driven cards and overlays.

Current supported flow:
- drag the whole clip horizontally to change `data-start`
- right-trim to shorten the end of the clip window
- move between tracks to change `data-track-index`

Not supported yet:
- true front trim that removes the beginning of the animation itself

Behavior after this PR:
- no interactive left trim handle is shown
- right trim still works
- horizontal move still works

### Media clips
Examples: `video` / `audio` clips, or wrappers carrying `data-media-start` / `data-playback-start`.

Current supported flow:
- drag the whole clip horizontally to change `data-start`
- left trim advances clip start and playback offset together
- right trim shortens `data-duration`

Behavior after this PR:
- both left and right trim handles remain available
- left trim persists `data-start` plus `data-media-start` / `data-playback-start`
- right trim persists `data-duration`

## Z-Index Rule
This PR now follows the normal timeline-editor convention:
- top visual row on the timeline = highest `z-index`
- lower visual rows = lower `z-index`

Concretely, because Studio renders tracks in ascending numeric order from top to bottom, lower numeric track values now map to higher `z-index` values.

## Validation
### Automated
- `bun test packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/utils/sourcePatcher.test.ts`
- `bun run --filter @hyperframes/studio typecheck`

### Browser verification
Verified with `agent-browser` on `timeline-edit-playground`:
- generic motion clips no longer expose an interactive left trim handle
- media clips still expose both trim handles
- left trim on `media-card` persisted `data-start` and `data-media-start`
- right trim on `media-card` persisted `data-duration` only
- moving `title-card` from the bottom row to the top row persisted the highest `z-index` for the top-row clips
- recordings:
  - `/tmp/trim-fix-artifacts/trim-flow.webm`
  - `/tmp/trim-fix-artifacts/z-index-flow.webm`
 2026-04-22 17:11:15 +02:00
Miguel Ángel d740f5ce42 fix: nested GSAP sub-composition lint and render handling (#405) 
## Summary

- allow nested sub-composition files to inherit GSAP from their host without tripping `missing_gsap_script`
- keep nested render seeks stable for sub-compositions without regressing producer baselines
- stop producer render-hint detection from treating the compiler's own nested mount retry wrapper as user-authored `requestAnimationFrame()` usage

## Root Cause

- the core linter treated template-based nested compositions like standalone root compositions, so it incorrectly required a local GSAP loader even when the host composition already provided GSAP
- producer `detectRenderModeHints()` runs before CDN scripts are inlined, so nested GSAP exports were never failing because of the GSAP payload itself
- the nested-only false positive came from the compiler-generated mount bootstrap that waits for the inlined sub-composition root with `requestAnimationFrame()` before running the hoisted inline script
- preview and export seek paths also needed to stay split so the nested timeline re-arm behavior that stabilizes scrubbing does not collapse render baselines

## What Changed

- lint: keep the nested GSAP false-positive fix and regression coverage for template sub-compositions
- runtime: keep the render-seek behavior that preserves nested child offsets during export without changing preview scrubbing behavior
- producer: mark compiler-owned mount bootstrap blocks and strip only those blocks before scanning inline scripts for raw `requestAnimationFrame()`
- producer tests now cover both cases: compiler-generated wrappers are ignored, but real user-authored nested `requestAnimationFrame()` still opts into screenshot mode

## Validation

- `bun test packages/core/src/lint/rules/gsap.test.ts`
- `bun test packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxfmt packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxlint packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bun run --filter @hyperframes/producer test --sequential chat style-11-prod`
  - `style-11-prod` passed locally
  - `chat` still shows local-only visual drift on this macOS/ARM workstation, but the render metadata now reports `renderModeHints.recommendScreenshot=false`, which is the concrete acceptance condition for `#402`
- Docker CI-image repro is blocked locally by OrbStack x86/arm64 loader mismatch, so final regression confirmation is deferred to GitHub Actions

Closes #392
Closes #402
 2026-04-22 16:10:38 +02:00
Miguel Ángel 29b6274ebc chore: release v0.4.13-alpha.2 2026-04-22 00:27:06 -04:00
Miguel Ángel d38a4f19fd fix(core): restore nested seek scrubbing (#404) 2026-04-22 06:13:09 +02:00
Miguel Ángel e0749ab768 chore: release v0.4.13-alpha.1 2026-04-21 19:57:04 -04:00
Miguel Ángel 1aea1415c4 fix: smooth scrubber end seeking (#386) 
* fix: smooth scrubber end seeking

* fix: stop timeline auto-scroll in fit mode

* feat: use percentage-based timeline zoom

* fix: sync timeline playhead on zoom changes

* fix: reset timeline scroll when returning to fit

* fix: keep timeline controls pinned
 2026-04-22 01:51:41 +02:00
Miguel Ángel 0ba56f9187 feat: add studio timeline editing (#390) 
## Summary

Add the actual Studio timeline editing layer on top of the preview/runtime foundation.

This PR includes:

- drag-to-move clips across time and tracks
- left/right resize handles with media-aware trim persistence
- edge auto-scroll and edge track creation while dragging
- selector-based source patching for `data-start`, `data-duration`, `data-track-index`, `z-index`, and media trim attributes
- timeline UI cleanup, theming, hover/drag states, and the `Copy Prompt` action

## Why This PR Is Separate

This is the user-facing editing behavior. It depends on the preview/runtime fixes in the base PR, but it is much easier to review once that plumbing is isolated.

## Verification

- `bun run --filter @hyperframes/studio test`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/player/components/EditModal.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/TimelineClip.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/timelineTheme.ts packages/studio/src/player/components/timelineTheme.test.ts packages/studio/src/utils/sourcePatcher.ts packages/studio/src/utils/sourcePatcher.test.ts`
- `bunx oxfmt --check packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/player/components/EditModal.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/TimelineClip.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/timelineTheme.ts packages/studio/src/player/components/timelineTheme.test.ts packages/studio/src/utils/sourcePatcher.ts packages/studio/src/utils/sourcePatcher.test.ts`

## Browser Proof

- verified timeline drag / resize / trim flows in Studio with `agent-browser`
- verified preview hot-refresh behavior without iframe remount flashes

## Stack

- depends on #389
- followed by `fix: smooth scrubber end seeking`

[result.mp4 <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.com/user-attachments/thumbnails/ca71c177-5042-468d-906f-b353938f40f8.mp4" />](https://app.graphite.com/user-attachments/video/ca71c177-5042-468d-906f-b353938f40f8.mp4)
 2026-04-22 01:48:14 +02:00
Miguel Ángel 158204343d fix: stabilize studio preview and runtime sync (#389) 
## Summary
Stabilize the Studio preview/runtime path so timeline data, preview rendering, and thumbnails stay in sync.

This PR includes:
- preview hot-refresh without remounting the iframe
- runtime duration/timeline fixes so Studio stops drifting from playback state
- thumbnail and selector-based preview fixes
- local Studio runtime serving and player-resolution fixes so dev/CI do not depend on prebuilt player artifacts
- tests around preview identity and thumbnail/runtime behavior

## Why This PR Exists
This is the foundation layer for timeline editing. Without it, the editor was prone to:
- iframe remount flashes after saves
- duration mismatches between preview and timeline
- stale or incorrect thumbnails
- CI/test failures when `@hyperframes/player` artifacts were not prebuilt

## Verification
- `bun run --filter @hyperframes/studio test`
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/core typecheck`
- `bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts`
- `bunx oxfmt --check packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts`

## Stack
- base of stack
- followed by `feat: add studio timeline editing`
- followed by `fix: smooth scrubber end seeking`
 2026-04-22 01:42:48 +02:00
James bfce71f203 chore: release v0.4.12 2026-04-21 19:17:41 +00:00
James RussoandClaude Opus 4.7 ffc06827c4 fix(engine): auto-normalize VFR video inputs to CFR before frame extraction (#360) 
* fix(engine): auto-normalize VFR video inputs to CFR before frame extraction

Screen recordings (macOS ScreenCaptureKit, QuickTime, phone videos) are
commonly variable-frame-rate. When such inputs hit the extractor's
`-ss <start> -i <video> -t <dur> -vf fps=N` pipeline, the fps filter
can emit fewer frames than requested — for a 4-second 30fps segment
starting mid-file, the output was ~90 frames instead of 120.

`FrameLookupTable.getFrameAtTime` returns null for out-of-range indices,
so the compositor held the last valid frame and the user perceived the
video as freezing. This matches the bug report from an X community post
where a user said "all of them freezes" on their screen recording scenes.

The engine already detects VFR via `metadata.isVFR` in ffprobe.ts but
never acted on it — the compiler only logged a warning. This change
mirrors the existing SDR→HDR normalization pattern: when a source is
detected as VFR, re-encode only the used segment with
`-fps_mode cfr -r <fps> -preset fast -crf 18` before extraction.

Scoping the re-encode to `[mediaStart, mediaStart+duration]` means a
30-second clip cut from a 60-minute screen recording pays ~1s of
transcode cost, not 18s. Benchmarked locally:

  Baseline (current):         32-39% duplicate frames, 25% frame-count
                              shortfall on mid-file segments.
  Tier 1 (flag changes only): ~same — fps filter issue is not flag-fixable.
  Tier 2 (CFR preflight):     1.7-6% duplicate frames, correct frame
                              count in every scenario tested.

The compiler warning that previously told users to manually re-encode
is downgraded to `console.info` since the engine now handles it.

— Rames Jusso

* refactor(engine): clean up VFR normalization loop after review

- Drop the `vfrNormDirCreated` flag; `mkdirSync({recursive:true})` is
  idempotent and cheap.
- Don't re-wrap the `VFR→CFR conversion failed` prefix — `convertVfrToCfr`
  already throws a message with that label; adding it again in the catch
  produced "VFR→CFR conversion failed: VFR→CFR conversion failed (exit 1)".
- Shorten the Phase 2b header comment; the function docstring above
  `convertVfrToCfr` already explains the failure modes and rationale.
- Note which frame windows the VFR fixture's select filter drops so the
  magic numbers are scannable.

No behavior change; 311/311 engine tests still pass.

— Rames Jusso

* test(engine): add VFR regression unit tests

Adds a describe block that synthesizes a VFR fixture via ffmpeg and asserts
the extractor produces the expected frame count (no shortfall) and no long
runs of duplicate frames — the user-visible "frozen screen recording"
symptom. Covers both a mid-file segment and the full-file case.

Guarded with describe.skipIf(!HAS_FFMPEG) because the CI Test job on
ubuntu-24.04 and the Windows test-windows job don't install ffmpeg. The
producer-level regression test in packages/producer/tests/vfr-screen-recording/
runs inside Dockerfile.test (which has ffmpeg) and is the primary CI signal
for this bug; these unit tests are supplementary coverage for local and
any ffmpeg-equipped CI environment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): add vfr-screen-recording regression test

End-to-end CI regression coverage for PR #360 via the existing
regression-harness: renders a 3s composition containing a real macOS
ScreenCaptureKit clip (r_frame_rate=120, avg≈36fps) seeked to
mediaStart=1, then PSNR-compares against a committed output.mp4.

Fixture src/clip.mp4 (108 KB) is a 5-second excerpt downscaled to 480×332
with -fps_mode passthrough to preserve the VFR timestamps. Content is the
public hyperframes OSS repo root page — see NOTICE.md for provenance.

With the fix applied, all 100 PSNR checkpoints pass. With the fix reverted,
66 of 100 fail (PSNR drops from ~43 dB to ~20 dB in the duplicate-frame
windows). Tagged "regression,video,vfr" so it runs in the fast shard
of .github/workflows/regression.yml automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): regenerate vfr-screen-recording baseline in Docker

The committed golden output.mp4 was initially rendered on the host machine;
CI runs the renderer inside Dockerfile.test with a different Chrome +
ffmpeg build, producing pixel-level drift that failed PSNR at 54/100
checkpoints (~20 dB vs 41 dB in the VFR sparse-content windows). Both
renders are valid — the VFR source has inherent sampling ambiguity in
static segments, and different Chrome/ffmpeg builds make different valid
choices.

Regenerated the baseline via `bun run docker:test:update vfr-screen-recording`
so it matches the Docker environment CI actually uses. Matches the flow
the existing sub-composition-video, hdr-pq, etc. baselines were captured
with.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: document that producer test baselines must be captured in Docker

Hit this 2026-04-21 with the vfr-screen-recording regression test:
host-generated output.mp4 baseline tripped 54/100 PSNR checkpoints in CI
because Chrome + ffmpeg drift between the host and Dockerfile.test.

Document the `bun run --cwd packages/producer docker:test:update <name>`
flow so future contributors don't repeat the mistake.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 12:14:23 -07:00
Miguel Ángel b98093aa1c fix: remove hidden audio gain in renders (#362) 
## Summary

This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.

It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.

Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.

## Root Cause

The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:

- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
  - `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`

That exactly matches the issue reporter's measured scalar boost.

While investigating the workaround, I also found a second correctness bug:

- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output

Separately, the repo-level Studio typecheck could fail before any build step because:

- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck

## What Changed

1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build

## Why These Changes Are Needed

This is not a UX preference issue; it is a correctness and API contract issue.

- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.

Leaving the current behavior in place means:

- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed

## Testing

### Focused regression tests

Ran:

- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`

Result:

- `10 passed`

These tests specifically verify:

- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)

### Broader package verification

Ran:

- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`

Results:

- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker

## Known Verification Limitation

There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.

## User Impact

After this change:

- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts

Closes #361.
 2026-04-21 20:33:25 +02:00
James RussoandClaude Opus 4.7 e72bcfaed3 fix(player+core): correctly render and pause nested compositions (#359) 
* fix(player): inject runtime immediately for nested compositions

Compositions that use `data-composition-src` on child elements require
the HyperFrames runtime to load those scenes — there is no way for the
iframe to render without it. The existing probe loop delayed runtime
injection behind a 5-tick attempts gate so the adapter path could try
to resolve a timeline first.

For nested compositions that race lost: a composition like the
`product-promo` registry example registers an inline pre-runtime GSAP
timeline at `window.__timelines["main"]` (covering only a partial
duration, e.g. 14s of a 20s master) while the iframe document loads.
The probe's adapter check finds that timeline and locks the player into
a "ready" state against it — which short-circuits the attempts gate and
the runtime never gets injected. The iframe ends up blank because the
runtime is what would have loaded the child scenes via
`data-composition-src`.

This change splits the injection decision into a pure helper,
`shouldInjectRuntime(state)`, and treats nested compositions as
"inject immediately, skip the gate." Self-contained GSAP-only
compositions retain the 5-tick grace period so the adapter path keeps
first shot for them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(core): propagate play/pause to all sibling timelines

Pausing or playing the master timeline only called `.pause()` / `.play()`
on `state.capturedTimeline` — the single adapter-selected timeline. In a
nested composition (a master with `data-composition-src` children), each
scene's own timeline is registered as a sibling in `window.__timelines`,
so they would keep advancing after the user clicked pause. The player UI
froze at the paused time while the visual content continued to animate,
eventually finishing all scene-level animations and landing on an empty
end-state.

Wire `window.__timelines` into the runtime player via a new
`getTimelineRegistry` dep, iterate the registry on play/pause, and
forward `timeScale` to siblings when play() starts so a changed
playback-rate applies uniformly.

Covered by 7 new unit tests in player.test.ts, including the identity-
equality check (don't double-invoke the master), playbackRate
propagation, a broken-sibling swallow, and a back-compat case with no
registry supplied.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 10:18:25 -07:00
Miguel Ángel 5db554c7b6 feat(cli): silent auto-update on next run (#306) 
## Summary

Today users have to run `hyperframes upgrade` (or the right install command for their package manager) to get a new release — we ship fixes but they don't reach the install until the user remembers. This PR borrows the Claude Code model: detect the update on run N, install it in a detached background child, surface one line ("hyperframes auto-updated to vX.Y.Z") on run N+1. The user's current command never blocks, never prompts, never sees an install stream.

## Flow across two runs

```
Run N     → checkForUpdate() sees latest > current → spawn detached
            child running `npm install -g hyperframes@X` (or bun /
            pnpm / brew equivalent). Parent exits immediately.
(between) → detached child installs, writes completedUpdate into
            ~/.hyperframes/config.json, clears pendingUpdate.
Run N+1   → reportCompletedUpdate() prints one line and clears the
            marker. User is on the new version.
```

## Installer detection

Walks `realpathSync(process.argv[1])` against each package manager's well-known global prefix. Wrong guesses are biased toward `skip` — we'd rather miss an auto-update than clobber a Homebrew install with npm.

| Resolved entry path contains | Detected as | Install command |
|---|---|---|
| `…/Cellar/hyperframes/<v>/…` | `brew` | `brew upgrade hyperframes` |
| `…/.bun/…` | `bun` | `bun add -g hyperframes@<v>` |
| `…/pnpm/global/…` or `…/.pnpm/…` | `pnpm` | `pnpm add -g hyperframes@<v>` |
| `…/lib/node_modules/hyperframes/…` | `npm` | `npm install -g hyperframes@<v>` |
| `…/packages/cli/…` (workspace link) | `skip` | (no-op) |
| `…/_npx/…`, `…/bunx-…/…` | `skip` | (no-op) |
| Anything else | `skip` | (no-op) |

## Guardrails

- **Never auto-update across a major version.** The existing banner still nudges the user to run `hyperframes upgrade` explicitly.
- **Skip on CI, non-TTY, dev mode,** npx / bunx / workspace link, or any install layout the detector doesn't recognize.
- **`HYPERFRAMES_NO_AUTO_INSTALL=1`** disables the install without silencing the notice banner.
- **`HYPERFRAMES_NO_UPDATE_CHECK=1`** silences both (existing knob).
- **Fresh pending install (<10 min old)** prevents re-launch on every invocation.
- **Installer stdout + stderr go to `~/.hyperframes/auto-update.log`** for postmortem — the terminal stays clean.
- **Failed installs are surfaced once** with a prompt to run `hyperframes upgrade` manually.

## What changed

| File | Role |
|---|---|
| `packages/cli/src/utils/installerDetection.ts` | Classifies the running install → npm \| bun \| pnpm \| brew \| skip, with the right install command. |
| `packages/cli/src/utils/autoUpdate.ts` | `scheduleBackgroundInstall` + `reportCompletedUpdate`. Spawns a detached `node -e "..."` child that runs the install and writes the outcome back to the config, then `unref()`s so the parent exits immediately. |
| `packages/cli/src/telemetry/config.ts` | `pendingUpdate` + `completedUpdate` fields on the config schema. |
| `packages/cli/src/cli.ts` | Wires `reportCompletedUpdate()` at startup and `scheduleBackgroundInstall()` after `checkForUpdate()` resolves. |

## Verification

### Unit tests — 19 / 19 pass (full CLI suite 115 / 115)

- `installerDetection.test.ts` — 9 cases, one per layout (workspace, npx, bunx, brew, bun, pnpm, npm, unknown, unresolved).
- `autoUpdate.test.ts` — 10 scheduling-policy cases:
  - Minor/patch → schedules + writes pendingUpdate
  - Major bump → **does not** schedule
  - Dev mode → skipped
  - `CI=1` → skipped
  - `HYPERFRAMES_NO_AUTO_INSTALL=1` → skipped
  - Unknown installer → skipped
  - Already-on-latest → skipped
  - Fresh pending install → de-duplicated
  - Stale pending install (>10 min) → supersedes
  - Previous run already completed this version → skipped

Unit tests mock `spawn` and the installer — they verify the **policy**, not the real detached-child path.

### Live end-to-end smoke test (on this Mac, real processes)

To validate the parts the unit tests can't — actual detached spawn, real config writeback, banner surfacing in a fresh subsequent process — I wired a smoke script that exercises the exact same code path `autoUpdate.ts` uses, but with `echo …` as the "install command" so nothing global gets touched.

**Steps exercised:**

1. Backed up the user's real `~/.hyperframes/config.json`.
2. Wrote a `pendingUpdate` marker for version `0.4.99` (like `scheduleBackgroundInstall` does).
3. Spawned the **exact same detached `node -e "..."` child** the real scheduler produces, with the install command replaced by `echo 'faux install for 0.4.99'`.
4. The parent `unref()`d and continued; 800 ms later the parent re-read `config.json`.
5. Ran `reportCompletedUpdate()` in a **fresh subprocess** (via `bunx tsx -e ...`) to match the real "Run N+1" conditions, capturing its stderr.
6. Asserted the marker was cleared.
7. Restored the original config on exit.

**Observed output:**

```
  [setup] Backed up config to /Users/miguel/.hyperframes/config.json.smoke-backup
  [setup] Wrote pendingUpdate for v0.4.99
  [spawn] Detached child pid=49469
  [after] completedUpdate = {"version":"0.4.99","ok":true,"finishedAt":"2026-04-17T17:26:50.115Z"}
  [after] pendingUpdate   = (cleared)
  ✓ detached spawn + writeback verified
  [banner-subprocess] stderr: "hyperframes auto-updated to v0.4.99"
  ✓ banner fired in fresh process + marker cleared

  ALL CHECKS PASSED ✓
  [cleanup] Config restored
```

**What this proves:**

| Claim | Evidence |
|---|---|
| Detached spawn works (doesn't block the parent) | `[spawn] pid=49469` logged, parent continued immediately |
| Detached child is process-independent | Parent exited its own work while child ran `exec(CMD)` |
| Child writes correct config shape | `completedUpdate = { version: "0.4.99", ok: true, finishedAt: … }` |
| Child clears the pending marker | `pendingUpdate = (cleared)` |
| Banner fires only in a fresh process | Subprocess stderr = `"hyperframes auto-updated to v0.4.99"` |
| Banner message format | Matches the copy in `autoUpdate.ts:reportCompletedUpdate` exactly |
| Marker clears after banner | Second file read shows `completedUpdate` absent |

Both the original test-plan checkboxes (fresh install, `HYPERFRAMES_NO_AUTO_INSTALL=1`, `CI=1`) are covered by either the unit-test suite or this smoke test — the scheduling-policy gates are unit-tested under `CI=true`, and the real detached-spawn path is smoke-tested above.

### What's still worth doing

- **Physical installer test on a real `npm i -g` / `brew` / `bun add -g` environment** — the smoke test above replaces the install command with `echo`, so we've never actually seen npm/bun/brew run the real command. That's the one remaining unknown. Worth one manual run on the maintainer's machine before cutting v0.4.4.

## Test plan

- [x] `bunx vitest run` on `packages/cli` — 115 / 115 pass (incl. 19 new)
- [x] `tsc --noEmit` clean
- [x] `tsup` build clean
- [x] **Live e2e smoke test** exercising the real detached spawn + config writeback + fresh-process banner (output above)
- [x] CI green on this branch (Typecheck, Test, Test: runtime contract, Build, Lint, Format)
- [ ] One manual run on a physical `npm i -g hyperframes@0.4.2` install to confirm the real `npm install -g hyperframes@0.4.3` command actually runs when `autoUpdate.ts` delegates to it (the smoke test stopped short of executing `npm`)

## Notes

- Independent of any version bump — ship whenever.
- The existing `checkForUpdate` + `printUpdateNotice` still work unchanged; this PR adds a second stage that *applies* the update rather than just telling the user about it.
- `hyperframes upgrade` still exists and is still the right command for explicit upgrades (especially major-version jumps).
 2026-04-21 00:05:04 +02:00
Vance Ingalls acce9123b4 chore: release v0.4.11 2026-04-20 12:40:27 -07:00
Vance Ingalls 0d551e3614 chore: release v0.4.11-alpha.1 2026-04-20 12:38:05 -07:00
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346) 
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
 2026-04-20 12:16:24 -07:00
Miguel Ángel a539266683 fix: harden CDN script inlining with linkedom (#352) 
* fix: harden CDN script inlining

* test: add spanish empire regression fixture
 2026-04-20 21:13:59 +02:00
James Russo 4a55bc8673 feat(cli): add --lang and auto-infer phonemizer locale from voice prefix (#351) 
* feat(cli): add --lang and auto-infer phonemizer locale from voice prefix

`hyperframes tts` was calling Kokoro's `model.create(text, voice=, speed=)`
with no language argument, so Kokoro's default phonemizer (en-us) was
applied regardless of the voice selected. Picking `ef_dora` or `jf_alpha`
and feeding it Spanish or Japanese text produced English-phonemized
output.

Closes #349.

- `manager.ts`: add `SUPPORTED_LANGS`, `inferLangFromVoiceId`, and
  `isSupportedLang`. Attach a `defaultLang` field to every bundled voice
  and expand the bundled list with `ef_dora`, `ff_siwis`, `jf_alpha`,
  `zf_xiaobei` so `--list` surfaces multilingual options.
- `synthesize.ts`: accept optional `lang: SupportedLang` in
  `SynthesizeOptions`, forward it to the Python worker as `argv[7]`.
  The worker introspects `Kokoro.create`'s signature and only passes
  `lang=` when the installed kokoro-onnx version supports it. Returned
  metadata now includes `lang` and `langApplied` so callers can detect
  silent no-ops. Bump the cached script filename to `synth-v2.py` so
  existing installs pick up the new script automatically.
- `commands/tts.ts`: add `--lang, -l` with validation against
  `SUPPORTED_LANGS`. Resolution order is explicit `--lang` > inferred
  from voice prefix > `en-us`. When explicit lang disagrees with the
  voice-implied lang (legitimate for stylized accents), emit a
  dim-level hint; suppress under `--json`. When kokoro-onnx silently
  ignores the kwarg, log that too. Update `--list` with a new
  "Lang code" column and add multilingual examples.
- Tests: new `manager.test.ts` covering every supported prefix, the
  unknown-prefix fallback, case-insensitivity, `isSupportedLang`
  validation, and a regression guard that every bundled voice has a
  valid `defaultLang` matching its ID.
- Docs: `docs/packages/cli.mdx` and `skills/hyperframes/references/tts.md`
  updated with the flag, examples, the espeak-ng dependency note for
  non-English phonemization, and the voice-prefix → lang table.

Backward compatibility:
- English voices (a*/b* prefixes) continue to phonemize as en-us / en-gb
  — no change.
- Non-English voices now phonemize correctly by default (bug fix, not a
  regression).
- Older kokoro-onnx versions that don't know the `lang` kwarg keep
  working via signature introspection; the CLI logs a dim note if
  `--lang` was requested but ignored.

Verification:
- `bun --cwd packages/cli test` — 128 tests pass (incl. 17 new).
- `bunx oxlint` and `bunx oxfmt --check` clean on changed files.
- `bun run build` succeeds.
- `npx tsx packages/cli/src/cli.ts tts --help` / `--list` render cleanly;
  invalid `--lang` produces a clean error with the valid-codes list.

* refactor(cli): simplify tts --lang implementation

Post-review cleanup on #351. Net -21 lines.

- Drop `defaultLang` field + `makeVoice()` helper from VoiceInfo —
  compute via `inferLangFromVoiceId(v.id)` at read time in listVoices.
  The only reader was the --list table; caching the derived value on
  every voice added a self-consistency invariant we had to test.
- Drop redundant `lang` field from SynthesizeResult — caller already
  knows the requested lang since it passed it in; only `langApplied`
  carries information the caller can't derive.
- Use `errorBox` for --lang validation to match the house style in
  render.ts (other validation errors already use errorBox).
- Reuse existing `langList` module constant in the validation error
  instead of re-joining SUPPORTED_LANGS.
- Inline `DEFAULT_LANG` — used once in inferLangFromVoiceId.
- Trim WHAT-restating comments and the duplicate prefix-enumeration
  JSDoc on inferLangFromVoiceId (VOICE_PREFIX_LANG already carries
  per-row comments).
- Clean up orphaned `synth*.py` files in ~/.cache/hyperframes/tts
  when writing the current versioned script, so repeated upgrades
  don't leak files.
- Drop the `EN-US` case-sensitive-rejection test assertion — the CLI
  lowercases input before validation, so accepting mixed case is a
  feature, not a bug.

Tests: 16/16 in `manager.test.ts`, 127/127 full CLI suite pass.
Lint + format + typecheck clean.
 2026-04-20 10:51:00 -07:00
Miguel Ángel 577e822fa6 chore: release v0.4.10 2026-04-20 11:06:52 -04:00
ukimsanov 0730f88dc7 fix(cli): address snapshot PR review feedback 
Incorporates four review points on #348:

1. Fix typecheck error — cast the dynamic `@hyperframes/engine` import to a
   typed shape and drop the `as typeof x` cast on a `null`-unioned variable
   that TS rejected. CI `Typecheck` was failing on main because the cast
   conflicted with the declared union.

2. Clear stale injected <img> overlays — always call
   `syncVideoFrameVisibility(page, activeIds)` on every seek (including
   `active.length === 0`), so injected frames from a previous timestamp
   don't leak into later snapshots. The runtime's visibility toggles act
   on the native <video> but not on its injected sibling, which Copilot
   correctly flagged as a leakage source.

3. URL-decode the served video path before resolving to a filesystem path.
   Files whose names contain spaces or other URL-encoded characters were
   skipped because `new URL().pathname` preserves `%20` while the file
   server decodes inbound requests and the file on disk lives at the
   decoded name.

4. Mirror `packages/core/src/runtime/media.ts` media math so clips with a
   non-1 `defaultPlaybackRate` get the correct active window and relTime.
   Specifically: clamp `defaultPlaybackRate` to [0.1, 5], apply
   `(t - start) * rate + mediaStart`, and adjust the duration-fallback
   branch by `/ rate` when only source duration is known.

5. Kill FFmpeg on a 30s timeout so a pathological clip cannot wedge
   snapshot indefinitely. Matches the default in
   `@hyperframes/engine`'s `runFfmpeg`.

Re-verified against the same 4-timestamp A/B on launch-video-2:
MD5s match the ffmpeg-from-render ground truth
  12.5s → ef9684e36fea53a0db7adf7cfcaacad3  (Stripe)
  16.0s → 487494ca16344d55d7181408dc439a56  (Framer)
  20.5s → 8835c34ad2a45755a1c98a7e079427a1  (HeyGen 3D)
  32.5s → 34dc9450f2bd661c12039d7aa82a30b0  (GitHub finale)

No-video projects (basecamp-tour, linear-brand, commissioned/github)
still produce identical output to the pre-fix path. Latency unchanged.

Made-with: Cursor
 2026-04-20 10:30:29 -04:00
ukimsanov a3e7c6176f fix(cli): inject real video frames in snapshot to match render 
The snapshot command previously just called `tl.seek(t)` + `page.screenshot`
and trusted Chrome to advance `<video>`-element decoders. Chrome headless
silently ignores `video.currentTime = X` writes — the setter is accepted
but the decoder never moves. Result: every snapshot of a composition that
uses body-level `<video data-start>` elements renders the same frame
regardless of the requested timestamp (the z-topmost video's first-frame
paints through, because all clips share `position: absolute; inset: 0`
and visibility:hidden doesn't always prevent the GPU surface from
contributing to the composite).

The render pipeline has already solved this: for each body-level video it
extracts the needed frame via FFmpeg and overlays it as an <img> sibling
via `injectVideoFramesBatch` (packages/engine/src/services/screenshot
Service.ts). This commit ports that same primitive into `snapshot`:

1. Added `extractVideoFrameToBuffer(videoPath, t)` — one FFmpeg spawn per
   active video, `-ss` keyframe seek (~100-200 ms), writes a temp PNG.
2. After the existing seek + settle, enumerate `<video data-start>`
   elements that are active at the target time, resolve each one's
   `currentSrc` URL back to a filesystem path under `projectDir`, extract
   the frame, and call `injectVideoFramesBatch`.
3. Then screenshot — as before.

Non-breaking: when no body-level `<video data-start>` elements exist (every
other project in the repo — basecamp, linear, stripe, github component), the
new block short-circuits on `active.length === 0` and behaves identically
to the pre-fix path. Verified against three no-video projects: bit-identical
snapshot output, no latency regression.

Measured on macOS M2 (4 frames, cold):
  launch-video-2 (11 timed videos): 6.48s → 6.16s   (-5%)
  basecamp-tour  (no timed videos): 5.67s → 4.87s   (-14%)

Proof: Pre-fix MD5 at t=12.5, 16.0, 20.5, 32.5 — all 4 identical (wrong frame)
  Post-fix MD5 at same timestamps  — all 4 distinct, match ffmpeg-from-render
Made-with: Cursor
 2026-04-20 09:36:33 -04:00
Miguel Ángel 62ce3679e6 chore: release v0.4.9 2026-04-19 23:03:07 -04:00
Miguel Ángel 479e6c3f81 chore: release v0.4.8 2026-04-19 22:40:57 -04:00
Miguel Ángel 03c2158e0f ci: verify on windows-latest + fix cross-platform build bugs it surfaced (#342) 
* fix(cli): make build copy cross-platform and deterministic

* fix(core): keep rewritten asset URLs POSIX on Windows

* ci(windows): add render verification workflow

* ci(windows): load canary gsap from cdn

* build: use dependency-aware workspace ordering

* Revert "build: use dependency-aware workspace ordering"

This reverts commit 99bc2ffbdf.
 2026-04-20 04:35:55 +02:00