`page.evaluate` callbacks with nested `function` declarations crashed
with `ReferenceError: __name is not defined` whenever the host was
bundled by tsx/bun. esbuild's `keepNames` mode wraps every function
declaration — including ones inside the body of an evaluate callback —
with a `__name(fn, "name")` call to preserve `Function.prototype.name`.
The helper is injected into the host bundle but never serialized into
the function string Puppeteer ships to the browser, so the browser
context sees a free reference and throws.
Install a no-op identity `globalThis.__name` shim via
`page.evaluateOnNewDocument` during session init. We pass a string
literal (not a function) because esbuild does not transform string
contents — defining the shim inline would itself get wrapped with
`__name(...)` and produce a use-before-define cycle. Running it via
`evaluateOnNewDocument` guarantees the shim is in place before any
page script (including subsequent `page.evaluate` callbacks) executes.
Document the constraint at the call site in `videoFrameInjector.ts`
so future edits don't try to redefine the shim inside an evaluate
callback.
Made-with: Cursor
x265 emits HDR10 mastering display + content light level metadata as
in-band HEVC SEI messages, but FFmpeg's `mov` muxer doesn't extract
those into the container-level `mdcv` (Mastering Display Color Volume)
and `clli` (Content Light Level Info) boxes that ingest pipelines
read. Without them, YouTube, Apple AirPlay, and most HDR TVs see only
stream-level color tagging (`colr`) and treat the file as SDR BT.2020,
silently tone-mapping the output.
Add `mp4HdrBoxes.ts`, which surgically inserts `mdcv` + `clli` boxes
inside the HEVC sample entry (`hvc1`/`hev1`), bumps every parent box's
size, and rewrites every `stco`/`co64` chunk offset that points past
the insertion site so the file stays decodable. Reference: ISO/IEC
14496-15 (NAL-structured video) and ISO/IEC 23001-8 (CICP).
Wire the injection in two places:
- `streamingEncoder.ts`: post-encode for direct HDR renders, so
standalone calls into the engine emit YouTube-ready files.
- `renderOrchestrator.ts`: post-mux for the producer pipeline,
because FFmpeg's mp4 muxer rebuilds the container during
mux/faststart and drops the boxes we injected into the
intermediate video-only file.
Failures degrade to a warning — the file is still playable; only HDR
recognition on strict ingests is affected. Covered by 50+ unit tests
in `mp4HdrBoxes.test.ts` (parser fuzzing, box layout, stco/co64
rewriting, malformed-input rejection).
Made-with: Cursor
DOM overlays composited onto HDR video frames were oversaturated because
`blitRgba8OverRgb48le` mapped sRGB 8-bit values directly through an
HDR OETF (HLG or PQ) without first converting from BT.709 to BT.2020
color primaries. Treating sRGB values as if they already lived in the
much wider BT.2020 gamut pushed saturated colors well past the
designer's intent — e.g. sRGB pure blue (0,0,255) landed on BT.2020
blue, which is far more vivid than what was specified.
Replace `buildSrgbToHdrLut` with the full pipeline:
sRGB 8-bit
→ linear BT.709 (sRGB EOTF, 256-entry LUT)
→ linear BT.2020 (3×3 BT.2087-0 primary matrix)
→ HDR signal 16-bit (HLG/PQ OETF, 4096-entry LUT)
The matrix rows sum to 1.0, so neutral content (R=G=B) is invariant —
text and grayscale UI render identically. Chromatic content (icons,
accent colors, progress bars) is now color-accurate against BT.2020
HDR video. PQ scales relative to 203 nits SDR white per BT.2408 so
SDR overlays sit at conventional brightness inside the HDR frame.
Verified end-to-end with hdr-smoke (sdr-baseline, hdr-pq,
mixed-sdr-hdr): sampled red overlay pixels (#C1121F) in the rendered
PQ output match the calculated post-conversion 16-bit values
(~30837/18894/15630), versus the old buggy pipeline's
~33820/10529/13663 (visibly oversaturated).
Removes the `getSrgbToHdrLut` public export — the LUT is now an
internal pipeline stage, not a single-step conversion.
Made-with: Cursor
Adds a standalone smoke harness for the HDR encode pipeline plus a
guard against the silent-decode-failure regression in blitHdrVideoLayer.
- packages/producer/scripts/hdr-smoke.ts: end-to-end harness that
renders the hdr-pq and mixed-sdr-hdr regression fixtures, then
ffprobes both stream-level and frame-level side-data so we can
catch missing MasteringDisplay / MaxCLL SEI in CI without
hand-running ffprobe. Frame probe uses
-show_frames -read_intervals %+#1 since x265 emits HDR metadata
as in-band SEI prefix NAL units, not container-level boxes.
- renderOrchestrator.test.ts: new "logs decode errors via the
supplied logger" test pinning blitHdrVideoLayer's failure path
to log.warn (matches production behaviour — silent failures
were the original PR-314 review concern).
- renderOrchestrator.ts: doc-only — clarify that blitHdrVideoLayer
is exported so the time→frame math, last-frame freeze, border-
radius detection, and affine-vs-region branch can be unit tested
without spinning up the full producer.
- .gitignore: ignore packages/producer/tests/hdr-regression/_renders
and the generated hdr-full-demo workdir so smoke runs don't
pollute git status.
Writes to ffmpeg's stdin pipe that race with ffmpeg's exit emit EINVAL
or EPIPE as unhandled 'error' events on the Socket, crashing the
process. Add a no-op error handler on stdin — the exit handler already
captures the failure via the result object.
Surfaced during HDR regression renders where the last frame's write
coincides with ffmpeg finishing input processing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds the test cases the PR #314 review flagged as missing:
- empty image list / no <img> elements
- duration="0", negative, NaN, and Infinity rejection
- missing data-start defaults to 0
- duplicate ids are preserved (documents current contract)
Made-with: Cursor
Implements the decode-once cache claimed in the PR review reply:
HDR image layers now reuse decoded rgb48le buffers across the frames
they're visible. Cache is render-scoped (cleared per job) and keyed
by framePath::sourceTransfer::targetTransfer because convertTransfer
mutates the buffer in-place.
Only image layers receive the cache. Video layers would bloat memory
(every frame has a unique path: ~37 MB × 300 frames at 1080p ≈ 11 GB).
Images decode once and are blitted on every visible frame.
Also promotes the blitHdrVideoLayer catch block from log.debug to
log.warn — a blit failure means a missing/dropped HDR layer, which
is user-visible and shouldn't be silent at the default log level.
Made-with: Cursor
- Tolerate per-image HDR probe failures; warn on missing paths.
- Warn when HDR <img> source is animated (stills-only for now).
- Namespace HDR image frame dirs as hdr_img_<id> to avoid collisions.
- Drop image id from HDR layer set when extraction fails.
- Promote HDR frame extraction failure from info to warn.
- Rename hdrVideoStartTimes → hdrLayerStartTimes.
- Make CompiledComposition.images optional.
- Fix NaN data-duration slipping through parseImageElements.
- Remove dead setAttribute mutation in parseImageElements.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ImageElement type and parseImageElements() parser for <img> elements.
When --hdr is set, images are probed alongside videos for HDR color
space. HDR images get single-frame 16-bit PNG extraction and route
through the existing blitHdrVideoLayer path. SDR images unchanged.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on
the assumption that the <img> sibling would inherit GSAP's opacity from
a shared wrapper. That breaks any composition where GSAP animates opacity
directly on the <video> element itself: the <img> has no animated
ancestor and renders at full opacity throughout any fade, even when the
user's intent is partial or zero opacity.
The CI `style-7-prod` and `style-8-prod` regressions caught this:
the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut
because the <img> inherited opacity 1 regardless of GSAP's tween.
Restore the old explicit copy from `computedStyle.opacity` to the
<img>'s inline opacity, with the `|| 1` fallback intentionally
preserved. The fallback is load-bearing: GSAP's seek does not re-apply
tweens that have already completed, so post-fade frames read opacity 0
from the stale `opacity: 0 !important` we apply to hide the native
<video>. The `|| 1` recovers the tween's end-state opacity 1 for
those frames, matching the final on-screen intent and the existing
baseline renders.
Handles both DOM shapes:
- GSAP on wrapper: video's own computed opacity is 1, img set to 1,
wrapper's opacity applies via stacking as before.
- GSAP on <video>: video's computed opacity is the tween value, copied
to img directly since they are siblings.
Fixes:
- style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33)
- style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The diagnostic logging block in executeRenderJob's HDR layer composite
path referenced an undeclared `hdrLayerStartTimes` map. The correct
variable, declared and populated earlier in the same function, is
`hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer
masking work and broke the producer build/typecheck on CI.
Made-with: Cursor
SDR clips inside an HDR composition were rendering at full opacity even
when the user had animated their wrapper opacity (e.g. fade-in or
yoyo). Two bugs in the per-layer screenshot path conspired to drop the
GSAP-applied opacity on the floor:
1. removeDomLayerMask was unconditionally calling
`el.style.removeProperty("opacity")` on every wrapper after each
layer capture. applyDomLayerMask only ever sets `visibility`, so the
only inline opacity present is the value GSAP wrote. Stripping it
between layer captures means that on the next capture (at the same
timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline
is already at that time — the opacity is never restored, and the
wrapper renders fully opaque.
2. injectVideoFramesBatch was reading the source <video>'s computed
opacity via `parseFloat(computedStyle.opacity) || 1` and copying it
onto the injected <img>. Because syncVideoFrameVisibility forces the
<video> to `opacity: 0 !important` to hide it during capture, the
computed value is always 0, which `|| 1` then silently flips to
full opacity. The <img> is a sibling of the <video> inside the same
wrapper, so it should inherit opacity from the wrapper directly
instead of having a value hard-set on it.
Fix both: drop the opacity removal in removeDomLayerMask, skip opacity
when copying visual properties from <video> to <img>, and explicitly
clear any stale inline opacity on the <img> so it inherits from the
wrapper that GSAP is animating.
Made-with: Cursor
The HDR layered compositor blits z-ordered layers over a shared canvas. DOM
layers used a full-page screenshot from `captureAlphaPng`, which captures
*every* painted pixel on the page — root background, sibling-scene content,
overlay UI elements that aren't part of the current layer. Those opaque
pixels were then blitted over the canvas, overwriting any HDR content
composited beneath in earlier layers.
The previous workaround toggled `display:none` on hide ids via
`hideVideoElements`/`showVideoElements`. That correctly hid native videos
but did nothing about the root composition's background or about overlay
elements that the layer grouping considered part of a different layer.
This commit replaces the workaround with a precise CSS mask installed
before each DOM screenshot:
1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and
re-shows the layer's elements (and their descendants and their injected
`__render_frame_*` siblings) with `visibility: visible !important`. CSS
visibility is *not* multiplicative through descendants — a child with
`visibility: visible` overrides an ancestor's `visibility: hidden`, so
deeply nested layer content still paints even though every intermediate
ancestor is hidden by the mass-hide rule.
2. Non-layer data-start ids are inline-hidden with
`visibility: hidden !important`. Inline `!important` beats stylesheet
`!important`, so this overrides the show rule for elements that fall
under a show selector but should NOT paint — most importantly HDR
videos and other-layer SDR videos that live as descendants of `#root`.
3. `removeDomLayerMask` tears the stylesheet down and clears the inline
`visibility`/`opacity` properties so subsequent video frame injection
gets a clean slate.
Crucially the mask only sets `visibility`, never `opacity`. CSS opacity
*is* multiplicative — `opacity: 0` on `#root` would zero out every
descendant including layer videos, even with `visibility: visible`. We
also extend `initTransparentBackground` to force the composition root
(`[data-composition-id]`) transparent in addition to `html`/`body`,
because compositions almost always set `#root { background: ... }` and
that background paints across the whole viewport otherwise.
Both compositing paths use the new helpers:
- The per-layer DOM branch (`compositeToBuffer`) for normal frames.
- The transition path (single DOM screenshot per scene) so transition
frames also get a clean per-scene capture.
Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`:
per-layer pixel-add accounting, dumps of every captured DOM PNG, and a
periodic raw `rgb48le` snapshot of the composite buffer. These were
essential to diagnosing the root-overwrite bug and stay zero-cost in
normal renders. Also stops the workDir / per-video frame-dir cleanup
when `KEEP_TEMP=1` so the dumps survive past frame N.
Made-with: Cursor
Adds:
- 8 new sampleRgb48le bilinear-interpolation tests covering boundary
pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers.
- uint16-alignment-audit.test.ts documenting the alignment requirement
for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE.
Background: ~105 hot-loop sites in shader transitions still use
readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut
overhead but requires guaranteed even byteOffsets — these tests document
the contract before any future refactor lands.
Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR
rendering fixes") accidentally removed two pieces of the deterministic
rendering pipeline:
1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`,
which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven
animations advance only when `window.__hf.seek(t)` is called.
2. The `applyRenderModeHints` function and its post-`compileForRender`
call site, which auto-forces screenshot capture mode for compositions
the compiler flagged as needing it (RAF, iframes, etc.).
Without (1), RAF animations advanced by wall-clock between the main-loop
seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the
sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer
seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat`
lost its automatic fallback to screenshot mode and the child-document
motion stopped being captured.
Both helpers are still produced by `htmlCompiler` and exercised by
`renderOrchestrator.test.ts` — the orchestrator just stopped calling
them. Restored:
- Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js`
- Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer`
call sites (probe + main render)
- Re-add `applyRenderModeHints` (matching the test expectations) and
call it immediately after `compileForRender`
- Persist `renderModeHints` in `summary.json` and the
"Compiled composition metadata" log line
Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression
failures on `feat/hdr-layered-compositing`.
Made-with: Cursor
- Tighten the all-transitions smoke test thresholds: at progress=0 we now
require the center pixel R-channel > 35000 (was > 25000) and at
progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly
halfway between the test from-pixel (40000) and to-pixel (10000), so a
half-blended transition would silently pass.
- Add a runtime assertion in HyperShader.init() that every scene id
resolves to a DOM element with the .scene class. Without this, missing
ids silently no-op when textures + querySelectorAll(.scene) run later.
Addresses deferred review feedback from PR #268.
## Summary
HDR video elements with GSAP animations (position, scale, rotation, opacity) and CSS border-radius rendered without any transforms applied — the video just sat at (0,0) full-size. This PR adds affine transform support and rounded-corner masking for natively-composited HDR video.
## What it does
**Affine blit with bilinear interpolation:**
- `blitRgb48leAffine()` — Takes a 4x4 DOMMatrix and maps each destination pixel back to source coordinates via the inverse transform. Bilinear interpolation between the 4 nearest source pixels produces smooth edges under rotation and non-integer scaling. Optional opacity and border-radius parameters.
- `parseTransformMatrix()` — Parses CSS `matrix(a,b,c,d,e,f)` strings into `[a,b,c,d,e,f]` tuples.
**Accumulated viewport matrix:**
- `getViewportMatrix()` — Walks the `offsetParent` chain from element to viewport, accumulating position offsets and CSS transforms at each level. Correctly handles `transform-origin` using the CSS sandwich: `translate(origin) × M × translate(-origin)`. This is critical because GSAP animates transforms on wrapper divs, not directly on the video element.
**Effective opacity:**
- `getEffectiveOpacity()` — Multiplies opacity values walking up the ancestor chain. Uses `Number.isNaN()` (not `|| 1`) so opacity:0 isn't incorrectly treated as 1.
**Border-radius masks:**
- `roundedRectAlpha()` — Per-pixel anti-aliased rounded-rectangle mask with support for independent corner radii.
- `getEffectiveBorderRadius()` — Walks ancestors for `overflow:hidden` + border-radius. Resolves percentage values (e.g., `50%` for circles) via `offsetWidth`/`offsetHeight`.
**Layout dimensions for extraction:**
- Uses `offsetWidth`/`offsetHeight` (unaffected by CSS transforms) instead of `getBoundingClientRect()` (which returns the transformed bounding box and wobbles under rotation).
## Files changed
| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | `blitRgb48leAffine()`, `parseTransformMatrix()`, `roundedRectAlpha()`, `cornerAlpha()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `getViewportMatrix()`, `getEffectiveOpacity()`, `getEffectiveBorderRadius()`, `layoutWidth`/`layoutHeight` on `ElementStackingInfo` |
| `packages/producer/src/services/renderOrchestrator.ts` | Affine blit path, extraction at layout dimensions, border-radius parameter passing |
## How to test
Render a composition with an HDR video that has GSAP scale + rotation animation and a `border-radius: 50%` wrapper (circle mask). The video should rotate smoothly with round edges — no wobble, no sharp corners.
## Stack position
**5 of 6** — Stacked on #289 (z-ordered layers). Adds transform and masking support to the HDR blit that the layer compositor uses.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* refactor(engine): restructure frame reorder buffer with Map-keyed storage
Rewrites createFrameReorderBuffer to use a Map<number, Array<() => void>>
keyed by frame index instead of a flat Array<{frame, resolve}> scanned on
every advance. O(1) lookups in enqueue/flush, fast-paths for the matching-
cursor and overshoot cases, and a small fix: waitForAllDone now coexists
with the writer still waiting on the final frame instead of colliding on
the same waiter slot.
Also adds 5 unit tests (there were none before) covering the fast-path,
out-of-order gating, multi-waiter-per-frame semantics, waitForAllDone
normal path, and the overshoot case.
Comment tweaks on buildChromeArgs — the flag profile is the standard
headless-for-capture set (Puppeteer / Playwright / Chrome headless-shell
all converge on similar flags); rephrased for clarity.
* refactor(cli): simplify port availability probe with async/await
Rewrites isPortAvailableOnHost from a single new-Promise callback into an
async/await form with an intermediate `bindError: ErrnoException | null`
variable. Makes the bind-then-release flow explicit as two sequential
awaits, and broadens the non-EADDRINUSE errno commentary (EADDRNOTAVAIL
for disabled IPv6, EACCES for privileged ports, EAFNOSUPPORT for missing
address families — all treated as "this host doesn't apply", not "port
occupied").
No behavior change to existing callers; all four portUtils tests still
pass.
* docs: add CREDITS.md and surface website-to-hyperframes skill
- New CREDITS.md acknowledging prior art in the browser-based video
rendering space (Remotion) and the ecosystem HyperFrames builds on
(Puppeteer, FFmpeg, GSAP, Hono). Standard OSS practice.
- Adds the `website-to-hyperframes` skill to the skills tables in
README.md, docs/guides/prompting.mdx, and the project template at
packages/cli/src/templates/_shared/CLAUDE.md. The skill ships in
skills/ but was missing from every table.
- Adds `/hyperframes-registry` to the prose mention in the repo
CLAUDE.md.
* feat(hdr): add z-ordered multi-layer compositing with PQ support
Per-frame z-order analysis groups elements into DOM and HDR layers,
composited bottom-to-top. Adjacent DOM elements merge into single
screenshots. PQ (HDR10/smpte2084) support via sRGB-to-PQ LUT with
203-nit SDR reference white. queryElementStacking walks DOM for
effective z-index, groupIntoLayers splits on HDR/DOM boundaries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): address review feedback across stack
- Document groupIntoLayers tie-break (V8 stable sort → DOM order).
- Expand layerCompositor docstring: merge rationale, visibility inclusion.
- Add tests: empty input, negative z-index, stable tie-break at equal z.
- Document getEffectiveZIndex CSS stacking-context limitations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Compositions with HDR video AND DOM overlays (text, graphics, SDR video) couldn't render both correctly — either HDR data was lost (Chrome captures sRGB only) or DOM overlays were missing (FFmpeg pass-through skips Chrome). This PR adds in-memory alpha compositing that combines both.
## What it does
**Per-frame two-pass capture:**
1. **DOM pass** — Chrome screenshots the page with a transparent background (CDP alpha). HDR videos are hidden, leaving transparent holes where they go.
2. **HDR pass** — Pre-extracted native HLG/PQ frames (16-bit PNG from FFmpeg) are read from disk.
3. **Composite** — DOM pixels (sRGB RGBA8) are alpha-composited over HDR pixels (rgb48le) in Node.js memory, with sRGB→HLG/PQ conversion via a 256-entry lookup table.
**Key components:**
- `decodePng()` / `decodePngToRgb48le()` — Pure Node.js PNG decoders (no native dependencies). Support all 5 PNG filter types.
- `blitRgba8OverRgb48le()` — Alpha composite with per-pixel sRGB→HDR LUT conversion. Fast paths for alpha=0 (skip) and alpha=255 (overwrite).
- `initTransparentBackground()` + `captureAlphaPng()` — Split CDP transparent background setup (once) from per-frame screenshot capture (eliminates 2 CDP round-trips per frame).
- Single-pass FFmpeg extraction — All HDR frames extracted in one sequential FFmpeg run (avoids duplicate frames from per-frame `-ss` fast seek).
## Key design decisions
| Decision | Why |
|----------|-----|
| In-memory compositing (not FFmpeg overlay) | Eliminates ~2400 process spawns + temp files per render. Pure pixel math is 10x faster. |
| 16-bit PNG intermediate | Raw `-f rawvideo` loses color metadata, causing moiré artifacts. PNG is self-describing. |
| sRGB→HLG LUT (256 entries) | DOM content is sRGB. Without conversion, it appears orange-shifted in HLG stream. |
| Native HDR detection before extraction | `extractAllVideoFrames` converts SDR→HDR. Pre-extraction probe identifies original HDR sources so only truly-HDR videos get native extraction. |
## Files changed
| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | **NEW** — PNG decode, sRGB→HDR LUT, alpha compositing (14 tests) |
| `packages/engine/src/services/screenshotService.ts` | Transparent background CDP, `captureAlphaPng()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `hideVideoElements()` / `showVideoElements()` |
| `packages/engine/src/services/streamingEncoder.ts` | Input color space tags for rgb48le |
| `packages/producer/src/services/renderOrchestrator.ts` | Two-pass HDR capture loop, native HDR detection |
## How to test
Render a composition with an HDR video background and text overlays. Both should be visible — HDR video at full quality, text crisp with correct colors (not orange-shifted).
## Stack position
**3 of 6** — Stacked on #265 (HDR output pipeline). This is the foundation for all layered compositing that follows.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Adds the ability to render HDR video output (H.265 10-bit, BT.2020) from HyperFrames compositions. When the renderer detects HDR source video, it automatically switches to the HDR output pipeline — no flags needed.
## What it does
- **Auto-detection** — Probes each video source with `ffprobe`. If any has bt2020/PQ/HLG color metadata, the output switches to H.265 10-bit with correct color tags. SDR-only compositions are unaffected (H.264, bt709).
- **HLG pass-through** — Native HLG pixels from FFmpeg extraction are piped directly to the encoder without conversion. This avoids brightness loss from HLG→linear→PQ conversion (which requires an OOTF system gamma we can't reliably apply).
- **Encoder HDR support** — Both chunk and streaming encoders accept HDR presets: `libx265`, `yuv420p10le`, BT.2020 color primaries, `hvc1` codec tag (required for Apple playback).
- **WebGPU HDR capture (gated)** — A complete WebGPU float16 readback pipeline is implemented and tested but gated behind headed Chrome (headless doesn't expose WebGPU). Ready for future use with WebGPU canvas content.
- **HDR utilities** — `detectTransfer()` (PQ vs HLG), `getHdrEncoderColorParams()`, `analyzeCompositionHdr()`. 15 unit tests.
## Key design decisions
| Decision | Why |
|----------|-----|
| No `--hdr` flag | SDR content encoded as HDR causes orange shift in browsers. Auto-detect eliminates this. |
| HLG pass-through (not HLG→PQ) | Conversion loses brightness without OOTF. Pass-through matches source exactly. |
| `hvc1` codec tag | Apple QuickTime requires `hvc1` (not `hev1`) for HEVC playback. |
| 1-hour streaming timeout | HDR capture at ~6fps needs more time than the default 10-minute FFmpeg timeout. |
## Files changed
| File | What changed |
|------|-------------|
| `packages/engine/src/utils/hdr.ts` | **NEW** — HDR detection, transfer types, encoder params (15 tests) |
| `packages/engine/src/services/hdrCapture.ts` | **NEW** — WebGPU readback, HLG conversion, PQ encode |
| `packages/engine/src/services/streamingEncoder.ts` | HDR presets, raw rgb48le input, color tags |
| `packages/engine/src/services/chunkEncoder.ts` | HDR presets, conditional color tags |
| `packages/producer/src/services/renderOrchestrator.ts` | Auto-detection loop, HDR pass-through capture path |
## How to test
Render a composition with an HDR video source. The output should be H.265 10-bit with HDR metadata visible in `ffprobe` (bt2020, arib-std-b67 or smpte2084). Plays correctly in QuickTime and on HDR displays.
## Stack position
**2 of 6** — Stacked on #258 (SDR/HDR normalization). Provides the encoder infrastructure that phases 1-5 build on.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* fix(cli): use 'where' instead of 'which' on Windows for FFmpeg and browser detection
- findFFmpeg() now uses 'where ffmpeg' on Windows, 'which ffmpeg' on Unix
- whichBinary() now uses 'where' on Windows, 'which' on Unix
Fixes FFmpeg detection failure on Windows where 'which' command doesn't exist.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(cli): handle multi-line output from Windows 'where' command
Windows 'where' can return multiple paths (one per line) when there
are multiple matches on PATH. Take only the first non-empty line.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(cli): extend Windows 'where' fix to whisper, tts, and clipboard modules
- whisper/manager.ts: whichBinary() now uses 'where' on Windows
- tts/synthesize.ts: findPython() now uses 'where' on Windows
- utils/clipboard.ts: detectProvider() now uses 'where' on Windows
All functions handle multi-line output from 'where' command.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Preview is the delivery — scrub through it, iterate on tweaks, only render
once the user explicitly asks ("render it", "make the final", "I'm happy").
Rendering takes minutes per pass and is wasted work if the user wants any
changes after seeing it.
- step-7-validate: rename Render section to "Render (on-demand only)" and
make the trigger criteria explicit
- SKILL.md: step-7 summary updated to "Deliver the preview to the user
first — only render to MP4 on explicit request"
Made-with: Cursor
Adds decision guidance and conventions the 8-site regression test exposed as
blind spots — agents had the capability but never reached for it.
step-1-capture:
- Clarify capture goes into <project-dir>/capture/ so capture artifacts stay
isolated from later build files (SCRIPT/STORYBOARD/DESIGN/compositions/)
- 7/8 regression tests already did this; codify as the convention
step-4-storyboard:
- Add "When to pick which" decision table for shader vs CSS vs hard cut
transitions. Shader transitions were available but used in 0/8 tests —
every test defaulted to CSS. The table frames shaders as "reveals, reaction
shots, brand moments" vs CSS as "connective tissue"
- Update technique count (10 → 11)
step-6-build:
- Mid-scene activity table gets a new row for audio-reactive logo/CTA
animation (bass pulse, treble glow). Audio-reactive was used in 0/8 tests
despite narration being present in all of them
step-7-validate:
- Snapshot section: explicit "use hyperframes snapshot, don't roll custom"
with the default naming pattern spelled out. Stripe's run used custom
ffmpeg naming (beat-6-cta-at-20.5s.png) instead of frame-XX-at-Ys.png
- New render section: require --output renders/<project>.mp4 so final MP4s
have predictable names. Without this, 7/8 tests produced wildly different
filenames (preview.mp4, cal_2026-04-19_20-29-21.mp4, basecamp.mp4, etc.)
techniques.md:
- New technique #11: Audio-Reactive Animation. Covers the sampling pattern
(per-frame tl.call, not single tween), when to use (music/dramatic VO
videos), intensity ranges (3-5% for text/logos, 10-30% for backgrounds),
and anti-patterns (equalizer bars, waveforms, strobing). Cross-references
skills/hyperframes/references/audio-reactive.md for the full API
Made-with: Cursor
Capture pipeline:
- agentPromptGenerator now writes AGENTS.md + CLAUDE.md (drop legacy
.cursorrules), matching the dual-file convention already used by the
_shared templates in hyperframes init. AGENTS.md is picked up natively by
Cursor, Codex, Gemini CLI, Windsurf, Aider, and Jules; CLAUDE.md covers
Claude Code. Both files share the same content — a capture data inventory
that points agents to the website-to-hyperframes skill.
website-to-hyperframes skill refinements (derived from 8-site regression test):
- Drop slash-command phrasing throughout SKILL.md and step-6-build.md so the
skill works identically across Claude Code (slash), Cursor (auto-discover
by description), and other agents.
- Remove stale HANDOFF.md references from SKILL.md step-7 summary and
reference table — matches the intent of the prior step-7 cleanup.
- step-5-vo: specify narration.txt filename convention (pronunciation-
substituted spoken text; distinct from SCRIPT.md the creative doc).
- step-6 self-review adds three rules derived from actual lint warnings
observed across the 8 regression runs:
- Every <template> root needs data-start + data-duration (catches
root_composition_missing_data_start/duration, seen in 4/8 runs).
- Caption exits need a hard tl.set kill after tl.to(opacity:0), or
per-word karaoke tweens can leave captions stuck on screen
(caption_exit_missing_hard_kill).
- No duplicate media nodes with identical src + start + duration, or
the compiler discovers them twice (duplicate_media_discovery_risk).
Housekeeping:
- .gitignore: add cursor-tests/, basecamp-video/, projects/, videos/ —
local regression-test scratch dirs that should never be committed.
- Remove two broken symlinks from .claude/skills/ that pointed to paths
which never existed in the repo (.claude/skills/ is already gitignored).
Made-with: Cursor
Capture improvements:
- Font weights via document.fonts API + DOM sampling (variable font detection)
- Section background-image extraction (no more false #FFFFFF on hero photos)
- Detected libraries surfaced in CLAUDE.md brand summary
- Structured visible-text.txt with [tag] prefixes, cookie/nav noise filtered
- tokens.json cleaned: removed images/paragraphs/icons (duplicated elsewhere),
filtered sections to heading-only, trimmed cssVariables to design-relevant
- Removed redundant scroll pass in htmlExtractor (2-5s faster per capture)
- Font cap at 20 families, Placeholder/Fallback fonts filtered
CLAUDE.md rewrite:
- Removed prescriptive tone ("use exact strings" → "rephrase freely")
- Removed fluff sections (How to Create, DESIGN.md warning, Example Prompts,
Source Patterns)
- asset-descriptions.md promoted to first data row
- Removed assets-catalog.json from inventory
Skill fixes:
- Dead shader refs → point to packages/shader-transitions/README.md
- Google Fonts import in techniques.md → local @font-face placeholder
- Added Stripe DESIGN.md as light-brand example
- Removed HANDOFF.md generation from step-7
- Updated step-1 for new font weight + visible-text formats
* fix(core): drive adapter seeks when composition has no GSAP timeline
renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.
Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.
GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(producer): auto-fallback screenshot capture for raf and iframes
Co-Authored-By: Codex <codex@openai.com>
* test(producer): add render compatibility regression fixtures
Co-Authored-By: Codex <codex@openai.com>
* fix(core): scrub CSS animations via WAAPI currentTime
Co-Authored-By: Codex <codex@openai.com>
* test(producer): cover css keyframe renders
Co-Authored-By: Codex <codex@openai.com>
* fix(producer): propagate virtual time into iframe documents
Co-Authored-By: Codex <codex@openai.com>
* test(producer): refresh iframe docker golden
Co-Authored-By: Codex <codex@openai.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
* docs(guides): add performance guide and preview-stutter troubleshooting
Adds a dedicated Performance guide covering preview-vs-render cost model,
expensive CSS patterns (backdrop-filter, filter, shadows), image sizing,
and how to diagnose slow compositions with Chrome DevTools.
Cross-links from troubleshooting (new "Preview stutters" accordion) and
common-mistakes (new "Oversized source images" and "Heavy backdrop-filter
stacks" accordions). Wires the new page into docs.json nav.
Also fixes a pre-commit format hook edge case: oxfmt would exit 2 when
the only staged files matching the format glob were all covered by
.prettierignore (e.g. docs-only changes). Add --no-error-on-unmatched-pattern
to the lefthook oxfmt invocation so docs-only commits are not blocked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: call out preview performance limits at the entry points
The preview command, studio package, and determinism concept pages all
frame preview as visually equivalent to render — correct for fidelity,
misleading for playback smoothness. A user who reads those pages and
then hits a paint-heavy composition has no way to know why preview
stutters, short of drilling into troubleshooting.
Adds short notes at each entry point linking out to the new Performance
guide, so users hit the "preview is hardware-bound, render isn't"
explanation wherever they land first.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Fixes#316 — `hyperframes skills` (and `npx skills add heygen-com/hyperframes`) fails with:
```
■ Failed to clone repository
fatal: active \`post-checkout\` hook found during \`git clone\`
└ Installation failed
```
## Root cause
Two layers stacked:
1. **Git 2.45+ refuses to execute hooks during `git clone` by default.** The opt-in is `GIT_CLONE_PROTECTION_ACTIVE=0` — the env-var name is intentionally explicit about the trade-off.
2. **Users who ran `git lfs install` globally have a post-checkout hook registered at `core.hooksPath`.** When the upstream `skills` CLI shells out to `git clone` to fetch a repo's `skills/` directory, git detects the user's LFS hook and aborts.
The check fires for **any repo**, regardless of whether the cloned repo uses LFS itself — it's protection against the user's own hooks, not the repo's content. Users who have git-lfs installed (very common) hit this for every clone the `skills` CLI does.
## The fix
`hyperframes skills` wraps `npx skills add`. The wrapper now sets `GIT_CLONE_PROTECTION_ACTIVE=0` on the spawned child's env via a single helper (`gitCloneFriendlyEnv`) with a docstring at the call site explaining exactly why. The rest of `process.env` is preserved — proxy settings, extra CA certs, locale, etc. stay untouched.
## What this fix doesn't do (deliberately)
This is the **code-path-we-own** fix. The deeper root cause is that the upstream `skills` CLI (vercel-labs/skills) should set this env var when it shells out to `git clone`. That would fix the bug for every user invoking `skills` directly — not just those who route through our wrapper. An upstream issue should be opened separately; not landing it as part of this PR.
## Users who call `npx skills add` directly
Documented in the new troubleshooting subsection: set the env var manually.
```bash
GIT_CLONE_PROTECTION_ACTIVE=0 npx skills add heygen-com/hyperframes
```
## Tests
`packages/cli/src/commands/skills.test.ts` — 2 cases:
- Every spawned child has `GIT_CLONE_PROTECTION_ACTIVE=0`
- The rest of `process.env` is preserved (not a wiped env)
Uses `vi.mock` on `node:child_process` because ESM doesn't allow live-module `vi.spyOn` on re-exported bindings.
## Docs
`docs/packages/cli.mdx` — new **Troubleshooting** subsection under the `skills` command. Explains both the automatic fix (`hyperframes skills` users are already covered) and the manual workaround (`npx skills add …` users).
## Closes
- #316
* fix(producer): external assets work on Windows (GH #321)
Two Unix-only assumptions in the external-asset pipeline caused every
absolute path on Windows to be rejected as "unsafe" at render time:
1. Containment checks used `child.startsWith(parent + "/")`. On Windows
the separator is `\`, so the predicate is always false unless the
paths are equal — every external asset tripped the safety guard in
`renderOrchestrator.ts`. The reporter saw:
[Render] Skipping external asset with unsafe path:
hf-ext/D:\coder\reactGin\hyperframes\reading\assets\segment_001.wav
Fix: use `path.relative()` through a shared helper
`isPathInside(child, parent)` that normalises separators per-platform
and correctly rejects siblings whose names start with the parent
(e.g. `/foo/bar-sibling` is NOT inside `/foo/bar`).
2. The external-asset key was built as `"hf-ext/" + absPath.replace(/^\//, "")`.
A Windows absolute path (`D:\coder\...`) became
`"hf-ext/D:\\coder\\..."` — and because Node's `path.join` treats a
drive-letter prefix as absolute, `join(compileDir, key)` silently
escaped `compileDir`. Fix: `toExternalAssetKey()` strips the drive
colon and normalises to forward slashes, producing
`hf-ext/D/coder/...` — a pure relative path that `path.join` cannot
promote to absolute on any OS.
Both helpers live in `packages/producer/src/utils/paths.ts` and are
exercised by 14 unit tests covering Unix paths, Windows drive-letter
paths, mixed separators, sibling-prefix confusion, and `..` traversal.
Docs: new "External assets" section in `docs/packages/producer.mdx`
describes detection, sanitised keys, and the cross-platform containment
invariant.
Closes#321.
* fix(producer): address review on #324 — UNC + integration test
Addresses the non-blocking observations from the PR #324 staff review
(https://github.com/heygen-com/hyperframes/pull/324#issuecomment):
1. UNC and extended-length Windows paths.
`toExternalAssetKey` now handles:
- `\\?\D:\very\long\path\clip.mp4` (extended-length) → `hf-ext/D/very/long/path/clip.mp4`
- `\\server\share\file.wav` (plain UNC) → `hf-ext/unc/server/share/file.wav`
- `\\?\UNC\server\share\file.wav` (extended-length UNC) → `hf-ext/unc/server/share/file.wav`
The UNC-collapsed form keeps the server boundary so two different
servers exposing the same share/file name cannot collide under one
relative key. Previously both edge cases silently produced keys with
stray `?` or `:` characters that downstream `isPathInside` rejected —
not a security hole, but a silent drop of user assets.
2. Short-circuit on already-sanitised input.
`toExternalAssetKey("hf-ext/…")` now returns its input unchanged
instead of prepending `hf-ext/` a second time. Makes the helper
genuinely idempotent, which is what the unit test claimed all along.
Renamed the test accordingly.
3. JSDoc caller contract.
`toExternalAssetKey` now documents that it expects canonicalised
input (`path.resolve`'d upstream) and does not strip `..`
components. `isPathInside` at copy time is still the defensive
backstop — called out explicitly in the doc so future callers read
the contract before the code.
4. End-to-end integration test.
`renderOrchestrator.test.ts` gains two seam tests that run the full
external-asset pipeline — build the sanitised key, populate an
`externalAssets` map, invoke `writeCompiledArtifacts`, and assert
both the success path (the file lands under `<compileDir>/hf-ext/…`)
and the escape-rejection path (a malicious `hf-ext/../../etc/passwd`
key does NOT materialise above `compileDir`). `writeCompiledArtifacts`
is exported for the test seam with a clear JSDoc disclaimer that
it's not part of the public API.
22 tests pass across `paths.test.ts` (17) and `renderOrchestrator.test.ts` (5).
Out of scope for this follow-up (tracked as follow-ups):
- Centralising every `startsWith("/")` absolute-path check into a
shared helper across htmlCompiler / audioExtractor / audioMixer /
videoFrameExtractor. Mentioned in the review; touches 5 files and
deserves its own PR.
- Windows CI runner.
The "FFmpeg not found" hint was hardcoded to `sudo apt install ffmpeg`
for any non-macOS platform — Windows users would see an apt command that
doesn't exist on their system, and Red Hat / Arch users got the wrong
package manager too.
`getFFmpegInstallHint()` already exists in browser/ffmpeg.ts (and is
already used by render.ts) and handles darwin / linux / win32 correctly.
Use it here too.
Also rewrite checkFFprobe:
- it previously used `which ffprobe` which is not available on Windows
(cmd uses `where`), so on Windows the check always reported "Not
found" even when ffprobe was on PATH
- run `ffprobe -version` directly instead, which works cross-platform
whenever ffprobe is resolvable on PATH, and surfaces the version
string in the same style as the FFmpeg check
HTML files in this repo are compositions (user-facing content and
registry templates), not the framework source. The framework itself is
TypeScript. Hide HTML from Linguist's detection so the language bar on
GitHub reflects what the repo actually implements.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Chrome's "Failed to load resource" message text does not include the failing
URL — it's only on msg.location().url. The previous filter in frameCapture.ts
only checked msg.text(), so every font 404 (e.g. Google Fonts <link> tags
in sandboxed render environments) fell through to the "[non-blocking]"
prefix instead of being suppressed.
Extract the classifier into isFontResourceError() and match against both
text and location.url, and extend the extension match to .ttf/.otf. Adds
a unit test covering the URL-in-location, URL-in-text, and non-font cases.
This is a targeted fix for the render-output noise that PR #311 attempted
to address by adding a ~120-entry SYSTEM_FONTS skip list. That approach
silently shadowed existing FONT_ALIASES (arial→inter, helvetica→inter,
courier new→jetbrains-mono, segoe ui→roboto, etc.) and changed render
output on Linux fleets that don't have those fonts installed. Fixing the
console-log filter here suppresses the noise without changing any font
resolution behavior.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stacked on top of #307. Fixes three mobile UX bugs that made the studio unusable on iPhone Safari — discovered while testing the audio-ownership work from #307 on a physical device.
## Bugs fixed
### 1. Untappable Play button / bottom controls
`#root` was set to `100vh`. iOS Safari reports `100vh` as the **largest** viewport (toolbar hidden) and never shrinks it — so with the toolbar visible, the bottom of the layout sits under it. The Play button + timecode were fully occluded.
### 2. Scrubber not draggable by touch
The seek bar had only `onMouseDown`. Mouse events don't fire for touches on iOS Safari, so nothing responded. You could tap to jump but not drag.
### 3. Safari's horizontal swipe hijacked scrubber drags
Even when the seek bar caught `pointerdown`, `touch-action: manipulation` still let Safari consume horizontal edge-swipes for back-navigation — dragging the scrubber left was impossible.
## What changed
| File | Fix |
|---|---|
| `packages/studio/src/styles/studio.css` | `#root` → `height: 100dvh` with `100vh` fallback. Dynamic viewport height shrinks when the iOS toolbar is visible, so the bottom of `#root` lines up with the visible area. |
| `packages/studio/src/App.tsx` | Two `h-screen` containers → `h-full` so nested children fill the now-dynamic parent instead of asserting `100vh` and overflowing. |
| `packages/studio/index.html` | Added `viewport-fit=cover` so iOS exposes real `env(safe-area-inset-bottom)` values. |
| `packages/studio/src/player/components/PlayerControls.tsx` | Controls row gets `padding-bottom: calc(0.5rem + env(safe-area-inset-bottom))` so it clears the landscape home indicator. Scrubber replaced `onMouseDown` with `onPointerDown` + `setPointerCapture`, plus `touch-action: none` so Safari doesn't hijack horizontal swipes. Added `pointercancel` + window-level `pointerup` fallbacks. |
All desktop code paths are unchanged: `100dvh` falls back to `100vh`, `env(safe-area-inset-bottom)` is `0` off-iOS, Pointer Events subsume Mouse Events on desktop.
## Verified live
Via the `cloudflared` tunnel I ran during review on the factory-series-c-video project:
- iPhone Safari, portrait: Play button now fully visible and tappable. Bottom controls sit just above the URL bar.
- iPhone Safari, landscape: controls clear the home indicator.
- Finger-drag the scrubber left and right: tracks the touch smoothly, finger can leave the 6 px bar height without losing the drag.
- Desktop click-to-seek and click-drag: still work.
- Arrow-key seeking: still works.
## Stacked dependency
Base is `fix/player-audio-ownership-review` (PR #307). Once #307 merges, rebase this branch onto `main` — the changes are fully independent; the stacking is just to avoid waiting on the review for #307 before shipping pure UX wins.
## Test plan
- [x] `tsc --noEmit` on `packages/studio` — clean
- [x] `bun run --filter @hyperframes/studio build` — clean
- [x] Live repro on iPhone Safari via cloudflared tunnel: Play button tappable, scrubber drags with touch
- [ ] Android Chrome sanity pass before release (same Pointer Events code path, but worth eye-balling)
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment.
## Significant
### 1\. Drift threshold 150 ms → 50 ms
_mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity.
**Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction.
### 2\. Dynamic sub-composition media proxies
Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track.
Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect.
### 3\. `bridgeMuted` sticky in `syncRuntimeMedia`
The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message.
`syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs.
### 4\. Reset `_audioOwner` on iframe reload
The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently).
`_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`.
## Worth addressing
### 5\. Promotion → observable event + reason
Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state.
**Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once.
### 6\. Parent proxy play() rejection → `playbackerror` event
Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back.
### 7\. Mobile verification on real hardware
Tested with a tunnel in a real iOS device.
## Test gaps (from review)
- `userMuted` stickiness (mirror of the existing `outputMuted` test).
- **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`.
- **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic.
- **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once.
- **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote).
- **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately.
- **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag.
## Minor
- One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear).
## What's good (from the review)
Kept as-is — noted for posterity:
- `muted` vs `volume` framing (orthogonal channels).
- Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing.
- Two orthogonal mute channels.
- Backwards compat (new actions / messages safely ignored by either side).
## Test results
- `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe)
- `packages/core/src/runtime/bridge.test.ts` — **15 tests pass**
- `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface)
- Typecheck green on `core` + `player`
- `tsup` build green on `core` / `player` / `cli`
- Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold.
## Test plan
- [x] Unit tests (83 total across touched files)
- [x] Typecheck clean
- [x] Build clean
- [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened
- [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block
- [x] Physical iOS / Android device verification (unchanged since #298)
Closes#309. Full credit to @gigadeniga for the diagnosis — the root cause + proposed fix in that issue are exactly what landed here.
## The bug
\`npx hyperframes preview\` failed deterministically on Crostini (ChromeOS Linux) with \`Ports 3002–3101 are all in use\`, even when nothing was actually listening on any of them.
## Why
\`testPortOnAllHosts\` ran four probes in parallel:
\`\`\`ts
const hosts = ["127.0.0.1", "0.0.0.0", "::1", "::"];
const results = await Promise.all(hosts.map((h) => isPortAvailableOnHost(port, h)));
\`\`\`
Each probe binds a socket and then calls \`server.close()\`. Close is async — the socket stays open until its callback fires on the next event-loop tick. While it's open, the wildcard binds (\`0.0.0.0\`, \`::\`) that include the loopback address race the still-open loopback socket and return \`EADDRINUSE\` spuriously. On Crostini this happens 100% of the time; other Linux configs hit it intermittently; macOS is less predictable. Net effect: every port in the 100-port scan range appears busy and the preview refuses to start.
Reproduces on any Linux box with the standalone snippet from the issue:
\`\`\`
127.0.0.1: OK
0.0.0.0: EADDRINUSE ← false positive
::1: OK
::: EADDRINUSE ← false positive
\`\`\`
## Fix
Serialize the probes. Each socket is fully closed before the next opens, eliminating the race window entirely.
\`\`\`ts
for (const host of hosts) {
const available = await isPortAvailableOnHost(port, host);
if (!available) return false;
}
return true;
\`\`\`
Kept the four-host check rather than collapsing to just \`0.0.0.0\` + \`::\` — the multi-host coverage is load-bearing for the devbox / SSH-forwarding case where a port is free on loopback but held on the wildcard. Sequentializing is the smaller, less-behaviourally-affecting fix.
## Regression tests
\`packages/cli/src/server/portUtils.test.ts\` — three cases binding real sockets, no mocks:
- **Returns true for a genuinely free port** — directly reproduces the Crostini bug; would fail on Linux against the parallel implementation.
- **Returns false when the port is occupied on \`0.0.0.0\`** — confirms the multi-host check still catches the devbox scenario.
- **Releases each probe socket before the next run** — two back-to-back calls for the same free port both return true, pinning the sequential contract against future refactors that might try to reparallelize for perf.
## Test plan
- [x] \`bunx vitest run packages/cli/src/server/portUtils.test.ts\` — 3/3 pass
- [x] Full CLI suite — 109/109 pass
- [x] \`tsc --noEmit\` clean
## Notes
- Independent of any version bump; ship whenever.
- Probing 4 hosts serially adds at most ~tens of milliseconds per port on the scan (binds are very fast on loopback). The worst-case cost shows up when the first port in the range is free — previously 1 parallel round-trip, now 4 sequential — and it's imperceptible (\`preview\` bind is a one-time startup cost, not a hot path).
Closes#304.
## Summary
The three `/schema/*.json` URLs baked into every Hyperframes project as `\$schema` references are 404ing on the live docs site — blocking editor autocomplete and validation.
- \`https://hyperframes.heygen.com/schema/hyperframes.json\` — **404** (missing entirely)
- \`https://hyperframes.heygen.com/schema/registry.json\` — **404** (only in npm package)
- \`https://hyperframes.heygen.com/schema/registry-item.json\` — **404** (only in npm package)
Mintlify serves top-level non-MDX dirs in \`docs/\` at \`/\<dir>/*\` (confirmed by \`docs/logo/*.svg\` → \`/logo/*.svg\`). This PR drops the three schemas into \`docs/schema/\` so the URLs resolve.
## What changed
| File | Role |
|---|---|
| \`docs/schema/hyperframes.json\` | **New.** Authored from the \`ProjectConfig\` type in \`packages/cli/src/utils/projectConfig.ts\`. |
| \`docs/schema/registry.json\` | Mirror of \`packages/core/schemas/registry.json\`. |
| \`docs/schema/registry-item.json\` | Mirror of \`packages/core/schemas/registry-item.json\`. |
| \`scripts/sync-schemas.ts\` | Keeps the registry mirrors in lockstep with their authoritative copies in \`packages/core/schemas/\`. \`--check\` mode fails the Docs workflow on drift. |
| \`.github/workflows/docs.yml\` | Runs \`tsx scripts/sync-schemas.ts --check\` on every PR touching docs or core schemas. |
| \`package.json\` | \`sync-schemas\` / \`sync-schemas:check\` npm scripts. |
## Why not make \`packages/core/schemas/\` authoritative for \`hyperframes.json\` too?
\`hyperframes.json\` is CLI config, not a core type. Keeping the schema in \`docs/\` avoids an artificial dependency between \`@hyperframes/core\` and \`@hyperframes/cli\`. If the two ever need to align, we can flip the direction then.
## Verification
- \`bun run sync-schemas:check\` → \`2/2 in sync\`.
- Ajv (draft 2020-12, in-process) validation against 9 cases:
- ✓ real factory-series-c-video config
- ✓ default shape from \`hyperframes init\`
- ✓ \`\$schema\` is optional
- ✓ missing registry → rejected
- ✓ missing paths.assets → rejected
- ✓ extra top-level key → rejected
- ✓ empty registry string → rejected
- ✓ empty block path → rejected
- ✓ missing paths entirely → rejected
## Test plan
- [x] \`tsx scripts/sync-schemas.ts --check\` passes locally
- [x] Schemas parse as valid JSON and validate real/default project configs
- [x] After merge: \`curl -sI https://hyperframes.heygen.com/schema/hyperframes.json\` returns 200 once Mintlify redeploys
- [x] Same check for \`/schema/registry.json\` and \`/schema/registry-item.json\`
- [x] VS Code autocomplete and error-highlighting work on \`hyperframes.json\` without extra config
## Notes
- The Docs workflow now triggers on \`packages/core/schemas/**\` and \`scripts/sync-schemas.ts\` in addition to \`docs/**\`, so a core-schemas change that forgets to run \`sync-schemas\` will fail CI instead of silently publishing stale docs.
- No runtime / API changes to any package; ship independent of a version bump.