Adds the directory + SKILL.md frontmatter for a new skill that translates
Remotion (React) compositions to HyperFrames (HTML+GSAP). This is the
foundation PR; subsequent PRs in the stack add the eval harness, test
corpus, translation references, and finally the SKILL.md body.
The frontmatter description enumerates trigger phrases and explicit
out-of-scope cases (useState/useEffect, async metadata, @remotion/lambda)
so the skill bows out cleanly when a Remotion composition isn't a clean
translation target — those should use the runtime interop pattern from
PR #214 instead.
Validated with skill-creator's package_skill.py.
The snapshot command resolved the HyperFrames runtime IIFE via a
relative path that walked up three directories to packages/core/dist/.
This only worked in the monorepo dev layout — npm/npx installs have
a flat dist/ folder with cli.js and the runtime side by side.
Without the runtime, window.__player was never created and the
snapshot fell back to seeking every __timelines entry to the same
absolute time. Sub-composition timelines expect relative time
(offset from their data-start), so all beats rendered beat-1 content.
Fix: resolve("hyperframe.runtime.iife.js") from __dirname (the dist/
folder itself), where the build already copies the runtime IIFE.
- Link to claude.ai/design instead of claude.ai
- Remove raw.githubusercontent download links (just GitHub with ↓ button)
- Fix stale SKILL.md link text in prompting guide
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(lint): remove root_composition_missing_data_duration
Lint cannot statically observe the runtime's true Infinity-emission
condition: it requires a finite GSAP timeline duration AND a finite
media/sub-comp window AND timeline > floor + 1, none of which are
visible to the static linter. The looping shapes that drive the
condition are already covered by `gsap_infinite_repeat` and
`gsap_repeat_ceil_overshoot` (both from #243), which point at the
real authoring mistake — flagging the missing duration separately
was a noisy proxy for the same signal.
Per #490 review discussion, deprecate the static rule and let those
two GSAP rules carry the pre-render coverage. If perfect precision
on `durationInFrames = Infinity` is needed, that belongs on the
runtime/render path where `shouldEmitNonDeterministicInf` is
actually known.
Add regression tests pinning the removal: a docs-compliant root
without `data-duration` no longer warns, and the canonical
loop-inflated shape now surfaces only via `gsap_infinite_repeat`
instead of two duplicate findings.
* docs(skills): update step-6 build checklist after rule removal
Drops the `root_composition_missing_data_duration` reference now that
the rule is gone. Keeps the authoring recommendation (and explains the
runtime Infinity case) but points authors at the GSAP rules that
actually carry the lint signal: `gsap_infinite_repeat` and
`gsap_repeat_ceil_overshoot`.
`lintAudioSrcNotFound` resolves every `<audio src>` against the project
root, which is correct for index.html but wrong for sub-compositions —
their srcs are written relative to the sub-composition file (e.g.
`../assets/foo.mp3`), and the bundler rewrites them at compile time.
Carry the sub-composition path alongside each html source and run
`<audio src>` strings starting with `../` through the existing
`rewriteAssetPath` helper before checking existence on disk. Mirrors the
runtime/bundler behaviour so the lint check sees the same path the
renderer will fetch.
Original (un-rewritten) src is still surfaced in the finding message so
authors can grep for it in their HTML.
The studio shell paints a #0a0a0a body but never declares `color-scheme`,
so browsers render native UA chrome (scrollbars, form controls, focus
rings) in the light palette and the mismatch is obvious — especially
scrollbars, which appear as light tracks sitting on top of a near-black
panel.
Studio doesn't expose a theme toggle; the UI is dark-only. Declaring
`color-scheme: dark` on `:root` lines up the browser-native surfaces
with the rest of the chrome.
* fix(studio): restore flex layout inside the NLE timeline panel
Without `flex flex-col`, the toolbar and Timeline children render as
block elements and Timeline's `h-full` takes the panel's entire height.
The toolbar still occupies its own ~45px of flow, pushing the scroll
area below the viewport and clipping the scrollbar at the bottom of the
right-side panel.
* fix(studio): pin toolbar height with flex-shrink-0 and trim comment
Wrap the toolbar slot in a flex-shrink-0 div so Timeline back-pressure
can't squeeze the toolbar below its natural height. The visible bug
doesn't manifest at current sizes, but in a flex-col container both
children are flex items and the toolbar has no shrink guard of its own.
Trim the inline comment to a single line — the rationale belongs in
the commit message, not at the call site.
* fix(cli): shut down preview embedded-mode server on Ctrl+C
runEmbeddedMode awaited a promise that never resolved, relying on Node
to exit on SIGINT. Two things kept that from working in practice:
1. @hono/node-server's listening handle keeps the event loop alive after
the signal fires, so the process hangs even when SIGINT does arrive.
2. On Windows, some terminals (Git Bash / MSYS) don't deliver Ctrl+C to
the Node process as a SIGINT at all — the keystroke is eaten at the
TTY layer.
Register a SIGINT/SIGTERM handler that closes the server explicitly and
resolves the promise, with a 2s force-exit fallback. On Windows, run a
readline interface on stdin to catch Ctrl+C at the TTY and re-emit it
as SIGINT so the same handler fires.
Print "Shutting down studio..." as soon as the signal is received —
server.close() can take a second or two to drain keep-alive connections
and an unmarked pause reads as "stuck".
Exit code 0 because a user-initiated Ctrl+C isn't an error; non-zero
codes make pnpm print ELIFECYCLE right where the user just asked the
process to stop.
* fix(cli): close readline interface in shutdown to honour exit-code intent
After the SIGINT handler removes itself, the Windows readline interface
is still alive and listening. A second Ctrl+C during the 2s grace
period would re-emit SIGINT with no registered handler, triggering
Node's default exit-130 behaviour and contradicting the explicit
exit(0) we chose for clean teardown.
Hoist the readline handle out of the win32 branch so shutdown can
close it before invoking server.close(). Also pass the signal name
to process.emit("SIGINT", "SIGINT") to match Node's ProcessEvents
overload — runtime behaviour is unchanged.
* fix(engine): suppress benign AbortError spam from frame-capture pageerror
Frame capture pauses → seeks → screenshots → plays audio/video many times
per second. HTMLMediaElement.play() returns a promise that rejects with
AbortError whenever another pause() lands before it resolves — which it
does, every frame. The rejection is benign (output frames and mixed
audio are unaffected) but the frameCapture pageerror handler was logging
it to stderr, producing dozens of identical lines per render:
[Browser:PAGEERROR] AbortError: The play() request was interrupted by
a call to pause(). https://goo.gl/LdLk22
Filter out exactly this pattern before console.error — still pushed to
browserConsoleBuffer so it's available in the failure-diagnostic dump.
* fix(engine): trim play-abort filter comment and drop unnecessary regex flags
The why-it-exists explanation belongs in the commit message and PR
description, not as a 12-line comment block at the call site.
Chrome's play()/pause() AbortError message is always lowercase, so
the case-insensitive flag implies uncertainty that doesn't exist.
Replace the two /play\(\)/i and /pause\(\)/i regexes with plain
String.prototype.includes — same outcome, less ceremony.
## Summary
- preserve alpha for render-injected video frames by detecting alpha streams with ffprobe and extracting alpha video frames as PNG
- keep `<video loop>` semantics through static parsing, compiler duration resolution, browser media discovery, and render frame lookup
- fail embedded preview startup before opening a broken browser page when the Studio bundle is missing
- align snapshot frame injection with looped media timing and VP9 alpha extraction
## Why
The Studio preview and rendered MP4 could disagree for timed transparent looped videos. The Comfy funding composition exposed two separate parity bugs: render-injected frames needed alpha-preserving PNG extraction, and the compiler was clamping a looped `data-duration="4"` video down to the 3.125s source duration. After the first source cycle, render lookup treated the video as inactive, hid the native video, and produced the blank polygon/glow the user saw around the rounded `0:03` mark.
`hyperframes lint` and `hyperframes validate` did not catch this because they check syntax/load/console/accessibility, not preview-vs-render visual parity. This PR adds regression coverage for the compiler loop-duration path and frame lookup path.
## Verification
- `bun run --filter @hyperframes/core test -- src/compiler/timingCompiler.test.ts src/compiler/htmlCompiler.test.ts`
- `bun test packages/producer/src/services/htmlCompiler.test.ts`
- `bun run --filter @hyperframes/engine test -- videoFrameExtractor ffprobe`
- `bun run --filter @hyperframes/core typecheck`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/cli typecheck`
- `bun run lint`
- `bun run format:check ...` on touched files
- Comfy project: `node packages/cli/dist/cli.js validate` -> no console errors, 44 text elements pass WCAG AA
- Comfy project patched render from source: `/tmp/comfy-render-compare/fixed6-comfy.mp4`, 1920x1080, 30fps, 21.8s, 654 frames
- 3.00s-3.97s render contact sheet: `/tmp/comfy-render-compare/fixed6-window-contact.png`
- targeted fixed render capture at 3.733s: `/tmp/comfy-render-compare/probe-capture-fixed/captured/frame_000112.jpg`
- agent-browser Studio proof screenshot at 3.7s: `/tmp/comfy-render-compare/agent-browser-studio-3_7-fixed.png`
- agent-browser-driven recording of 3s seek pass: `/tmp/comfy-render-compare/agent-browser-wysiwyg-3s-fixed.webm`
Note: `bun run --filter @hyperframes/cli dev -- validate` is blocked in source mode by the existing `contrast-audit.browser.js` default-export loader issue; packaged `node packages/cli/dist/cli.js validate` passes for this project.
## Problem
Studio seek could still wake nested composition media even when the transport itself stayed paused.
In the real repro from `apple-presentation`, scrubbing to `0:29` without pressing play lands on the `slide-translation` composition. That composition contains `Multilingual_Journey.mp4` inside the composition host. On the broken path:
- the main Studio transport remained paused
- the nested video advanced and stayed playing anyway
- the user saw autoplay-like behavior even though the only action was a seek
That was especially confusing because the seek was otherwise correct: the timeline moved to the right point, but the nested media stopped obeying the paused transport state.
## What this fixes
### Nested media now participates in runtime media sync
- the runtime media cache no longer assumes only `video[data-start]` / `audio[data-start]` are relevant
- nested media inside a composition host can now be included in the same timed-media sync pass even when the inner media element does not carry its own authored `data-start`
### Nested media timing is resolved in the host composition window
- nested media start time is resolved against the enclosing composition host instead of falling back to scene-local `0`
- nested media duration is clamped to the enclosing composition window so it stays aligned with the authored host clip timing
### Paused seeks land on the right frame and stay paused
- after seeking into a nested composition, the inner media is now seeked to the correct frame relative to the host timeline
- because it is now part of the managed media set, the runtime also keeps it paused when the transport is paused instead of letting it continue playing on its own
### Regression coverage
- adds a runtime regression test that covers a nested composition video with no local `data-start`
- the test verifies that `player.seek(29)` leaves the nested video paused while landing it at the expected `currentTime`
## Root cause
The bug came from a mismatch between deterministic timeline seeking and media ownership.
### 1. The runtime only managed media with direct timing attrs
`refreshRuntimeMediaCache()` only collected `video[data-start]` and `audio[data-start]`. That works for root-level timed media, but not for media embedded inside a composition host where timing is inherited from the host composition rather than duplicated onto the inner media node.
### 2. Nested composition seek could still advance inner media
The runtime intentionally rearms sibling timelines during deterministic seek so nested timelines land on the right local offsets. That part is necessary and correct.
But because the nested video was not part of the managed media cache, it could advance during that seek path without being brought back under the paused transport state afterward.
### 3. The runtime had no way to reconcile the two
So the system had an inconsistent split:
- timeline seek knew about the nested composition timeline
- media sync did not know about the nested media inside it
The fix closes that split by resolving nested media start/duration from the enclosing composition context and running it through the same sync logic as other managed media.
## Verification
### Local checks
- `bun run --filter @hyperframes/core typecheck`
- `bun run test -- src/runtime/init.test.ts src/runtime/media.test.ts src/runtime/player.test.ts` in `packages/core`
- `bunx oxlint packages/core/src/runtime/media.ts packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts`
- `bunx oxfmt --check packages/core/src/runtime/media.ts packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts`
### Browser verification
Verified against a repo-backed local Studio preview of `apple-presentation`:
- opened `http://127.0.0.1:3014/#project/apple-presentation`
- seeked to `0:29` without pressing play
- confirmed the visible composition switched to `slide-translation`
- confirmed `Multilingual_Journey.mp4` landed at a non-zero `currentTime` (`3.067` in the verified run)
- confirmed the nested video stayed `paused` and its `currentTime` remained stable across a follow-up check instead of autoplaying
## Notes
- the local browser proof artifacts under `qa-artifacts/autoplay-seek/` are verification-only and are not part of this PR
- this PR is intentionally scoped to nested media ownership during paused seek; it does not broaden into unrelated runtime media refactors beyond bringing inherited nested media under the existing sync contract
## Problem
Template-wrapped sub-compositions could still lose correct parent timing during render in more than one place.
In the validated repros, a host sub-composition starting after the intro (and in one follow-up repro, starting at `20s` after earlier compositions) contained scene-local media inside it. On the broken paths:
- template-wrapped media could be missed during compile and scheduled at raw scene-local time
- already-correct first-pass offsets could be clobbered during `recompileWithResolutions()`
- even after those two fixes, the browser-metadata reconcile step in `executeRenderJob()` could still overwrite a compiled global `end` with a scene-local `data-end` from the inlined DOM, clipping the tail off late-start sub-composition media
## What this fixes
### Template-wrapped media discovery
- `parseVideoElements`, `parseImageElements`, and `parseAudioElements` now unwrap a single top-level `<template>` wrapper before scraping media
- the unwrap helper is DOM-based, not regex-based, so it avoids the CodeQL backtracking warning and only unwraps the exact single-wrapper shape we want
- multiple sibling templates or other top-level content are left untouched instead of being rewritten heuristically
### Offset preservation after duration resolution
- `recompileWithResolutions()` now preserves the first-pass sub-composition media arrays when the already-inlined HTML no longer contains `[data-composition-src]` hosts
- that prevents correctly offset media metadata from being overwritten by scene-local media parsed from the merged DOM
### Browser metadata reconciliation in the compiled time origin
- browser-discovered media can still report scene-local `data-start` / `data-end` from the merged DOM after inlining
- the producer now reprojects browser `end` values into the compiled element's time origin before reconciling them back into `composition.videos` / `composition.audios`
- this prevents late-start sub-composition media from getting truncated back to a scene-local end during the probe phase
### Regression coverage
- adds focused engine tests for the template unwrap helper
- adds producer regression coverage for both the initial compile path and the post-inline `recompileWithResolutions()` path
- adds producer regression coverage for late-start host compositions (`t≈20`) with scene-local media inside them
- adds producer unit coverage for the browser-end reprojection helper used by the reconcile path
## Root cause
There were three distinct renderer failures behind the bug:
### 1. Template contents were invisible to the media scrapers
`parseSubCompositions()` reads raw sub-composition HTML and applies the host offset to discovered media. But the engine media helpers were querying the parsed document directly, and linkedom follows browser semantics here: top-level `<template>` contents live in a `DocumentFragment`, so `querySelectorAll()` never saw those `<video>` / `<audio>` / `<img>` nodes.
That meant template-wrapped sub-compositions could silently produce zero discovered media during the first pass.
### 2. The duration-resolution recompile could clobber already-correct offsets
After the browser resolves composition durations, `recompileWithResolutions()` reparses the already-inlined HTML. By that point the original `[data-composition-src]` hosts are gone, so `parseSubCompositions()` legitimately returns no nested media.
The old code still rebuilt the deduped media arrays from the merged DOM, which let scene-local media parsed from the inlined HTML overwrite the correctly offset first-pass metadata.
### 3. The browser probe reconcile path mixed two timing coordinate systems
`discoverMediaFromBrowser()` reads `data-start` / `data-end` directly from the live DOM after sub-compositions are already inlined. For nested media, those attributes can still be scene-local even though the compiled metadata has already been offset into the parent host timeline.
The old reconcile path compared those values directly and overwrote `existing.end` whenever the numbers differed. For a late-start sub-composition, that could replace a correct global end like `25.5` with a scene-local end like `5.5`, cutting the clip off during render.
## Verification
### Local checks
- `bun test packages/engine/src/utils/htmlTemplate.test.ts`
- `bun test packages/producer/src/services/htmlCompiler.test.ts`
- `bunx vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run --filter @hyperframes/producer typecheck`
- `bunx oxlint packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxfmt --check packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts`
- `bun run build:producer`
### Render / browser verification
Verified against two local repros:
1. **Early offset repro**
- host starts at `2s`
- child media is scene-local `0-4s`
- compiled render summary keeps the child video/audio at `start: 2`
- browser verification via `agent-browser` confirmed the `2.2s` frame still shows the child clip active in the host timeline
2. **Late offset repro**
- earlier compositions run first, then the target host starts at `20s`
- child media starts scene-local at `1.5s` and should remain visible through `24.5s`
- compiled render summary keeps the child video/audio at `start: 21.5`, `end: 25.5`
- browser verification via `agent-browser` confirmed the `24.5s` frame still shows the late clip visible, which is the exact tail-clipping case the old reconcile path could break
## Notes
- the `/tmp/hf-pr475-repro` and `/tmp/hf-pr476-late-offset-repro` projects plus their browser-proof artifacts are verification-only and are not part of this PR
- this PR stays narrowly scoped to sub-composition media timing across compile, recompile, and browser probe reconciliation; it does not broaden into general sub-composition HTML normalization beyond the single-wrapper case
Fixes#473.
## Problem
The HyperFrames skill now tells agents to add deterministic `tl.set()` hard-kills after elements fade out at beat / scene boundaries, but the linter did not enforce that rule outside the narrow caption-specific check.
That made the rule easy for sub-agents to ignore: an element could fade to `opacity: 0` exactly as the next clip starts, with no explicit hidden-state set at the boundary. During non-linear seeking or frame capture, that leaves the final visibility state dependent on tween interpolation instead of an authored deterministic kill.
## What this fixes
This PR adds a generalized GSAP lint warning for scene-boundary exits:
- detects GSAP `to` / `fromTo` exit tweens that end at or near a clip `data-start` boundary
- treats `opacity: 0`, `autoAlpha: 0`, `visibility: "hidden"`, and `display: "none"` as hidden exit states
- requires a matching same-selector `tl.set(...)` hidden state at the same boundary
- scopes clip-boundary matching to the timeline's registered composition so sub-composition exits do not match unrelated root boundaries
- reports `gsap_exit_missing_hard_kill` with the selector, boundary time, source snippet, and a fix hint that preserves the authored hidden property when possible
- keeps valid compositions quiet when the boundary hard-kill already exists
## Why
Clip boundaries are the exact points where rendered frames are most sensitive to stale DOM state. A fade-out tween describes a transition, but it does not give the linter or the authoring model an explicit deterministic state to land on when seeking around the boundary.
The existing caption rule proved the class of bug was worth catching, but it only applied to caption-loop patterns. The issue in #473 is broader: any element inside a timed composition can exit at a scene boundary and need the same deterministic cleanup.
## Root cause
The GSAP lint rule parser already calculated tween windows and clip metadata existed in the lint context, but no rule connected those two facts:
- clip `data-start` values were not used as scene-boundary checkpoints for GSAP exits
- parsed GSAP windows tracked property names, but not enough property values to tell whether a tween ended in a hidden state
- hard-kill detection only existed as a caption-specific regex, so normal scene elements were missed
This PR extends the existing GSAP window metadata with parsed property values, then checks hidden-state exits against same-composition clip start boundaries and same-selector `tl.set` calls.
## Verification
### Local checks
- `bun run --filter @hyperframes/core test src/lint/rules/gsap.test.ts`
- `bunx oxlint packages/core/src/lint/rules/gsap.ts packages/core/src/lint/rules/gsap.test.ts`
- `bunx oxfmt --check packages/core/src/lint/rules/gsap.ts packages/core/src/lint/rules/gsap.test.ts`
- `bun run --filter @hyperframes/core typecheck`
- `bun run --filter @hyperframes/core test`
- `bun run --filter @hyperframes/core build`
### CLI verification
Verified against local fixtures where `#headline` exits at the next clip boundary without a hard kill:
- opacity fixture reports `gsap_exit_missing_hard_kill` for `#headline` at `3.00s`
- autoAlpha fixture reports the same warning and suggests `tl.set("#headline", { autoAlpha: 0 }, 3.00)`
- sub-composition regression test confirms a `sub` timeline exit no longer matches an unrelated root composition boundary
### Browser verification
Verified the Studio lint flow with `agent-browser` against the autoAlpha fixture:
- opened Studio at `http://127.0.0.1:43174/#project/issue-473-autoalpha`
- clicked the real `Lint` button
- confirmed the lint modal shows the new warning and the property-preserving `{ autoAlpha: 0 }` fix hint
- saved local proof artifacts under `qa-artifacts/issue-473/`
## Notes
- the `tmp/issue-473-*` fixtures and `qa-artifacts/issue-473` browser proof are local-only and are not part of this PR
- this intentionally stays heuristic-based: it warns near clip start boundaries instead of trying to build a full GSAP execution model
- expression-valued GSAP props and deeper regex-parser limitations remain outside this PR's scope; those are parser-hardening work, not required for the bug in #473
Per review feedback on PR #364: the 'hero vanishes' failure has two
mechanisms. Primary: second tween's immediateRender overwrites the first
at construction time. Secondary: tl.from() resets to its declared from-
state when seeked past timeline end, which the capture engine triggers.
Both are now named so the rule has precise rationale, not just a pattern
to avoid.
Ref: https://github.com/heygen-com/hyperframes/pull/364#pullrequestreview-4167523103
## Summary
Document why the `window.__name` polyfill in `frameCapture.ts` is necessary, expand the inline comment with the full per-runtime matrix, and add a regression test that surfaces transpiler behavior on the next failure.
Outcome of the Chunk 12 investigation: **keep the polyfill**.
## Why
`Chunk 12` of `plans/hdr-followups.md`. The polyfill had a vague comment and no test, so it was unclear whether it was still needed or could be deleted.
## Empirical findings
Probe in `/tmp/hf-name-probe`:
| Runtime / build | Injects `__name(fn, "name")` wrappers in `Function.prototype.toString()`? |
|-----------------|---------------------------------------------------------------------------|
| `bun` (TS loader) | No — verified for top-level and nested named functions / arrow expressions. |
| `tsx` (esbuild loader, `keepNames=true`) | **Yes** for nested named functions / arrows; observed crash mode in dev/test. |
| `tsc` (`noEmit` and emit) | No — does not inject the helper. |
| `tsup` for `@hyperframes/cli` (`noExternal: ["@hyperframes/engine"]`) | Polyfill *definition* is bundled, but `__name(...)` *call sites* are absent in `packages/cli/dist/cli.js` (grepped). |
**Root cause.** `@hyperframes/engine`'s `package.json` exports raw TypeScript (`main`/`exports` → `./src/index.ts`), so every consumer's transpiler decides whether to inject `__name`. Anything that runs through `tsx` (producer parity-harness, ad-hoc dev scripts, `bun run --filter @hyperframes/engine test` via Vitest's loader) will serialize wrapped function bodies into `page.evaluate(...)` and crash with `ReferenceError: __name is not defined`.
**Decision.** Keep the no-op `window.__name` shim. Cost is one `evaluateOnNewDocument` call. The alternative (rewriting every `page.evaluate(fn)` site to `page.addScriptTag({ content: "..." })`, like `packages/cli/src/commands/contrast-audit.browser.js` already does) is far more invasive and easy to regress.
## What changed
- Expanded the inline comment in `packages/engine/src/services/frameCapture.ts` to explain the per-runtime matrix above and point to the script-tag alternative.
- New `packages/engine/src/services/frameCapture-namePolyfill.test.ts` — a pure unit test (matches the rest of the engine package's no-browser-launch convention) that:
1. Asserts the polyfill is wired up via `evaluateOnNewDocument` and runs before the first awaited `browser.version()` call.
2. Probes the active Vitest transpiler for `__name(...)` injection so the next maintainer can see at a glance whether the upstream behavior has shifted.
## Test plan
- [x] `bun run --filter @hyperframes/engine test` → 408/408 pass (3 new tests in this file).
- [x] `bunx tsc --noEmit -p packages/engine` clean.
- [x] `bunx oxlint` and `bunx oxfmt --check` clean on edited files.
## Stack
Chunk 12 of `plans/hdr-followups.md`. Independent of all other chunks; closes out the investigation item.
## Summary
Add `HdrImageTransferCache` — a per-render-job bounded LRU keyed by `(imageId, targetTransfer)` — so static HDR image layers whose source transfer differs from the render's effective transfer (PQ↔HLG) are converted **once per job** instead of **once per composited frame**.
## Why
`Chunk 8B` of `plans/hdr-followups.md`. `blitHdrImageLayer` was running `Buffer.from` + `convertTransfer` on every composited frame, even though the converted buffer is identical for the entire job. For a multi-second comp at 30 fps this is hundreds of redundant transfer conversions on the hot path.
## What changed
- New `packages/producer/src/services/hdrImageTransferCache.ts` — bounded LRU keyed by `(imageId, targetTransfer)` that owns the converted HDR rgb48 buffer for static HDR image layers:
- Same-transfer requests return the source buffer untouched (zero copy).
- Cross-transfer requests pay one `Buffer.from` + `convertTransfer` on first miss, reuse the cached copy on every subsequent frame.
- Wired into `renderOrchestrator.ts` via `HdrCompositeContext.hdrImageTransferCache`, instantiated once per render job, and consumed by `blitHdrImageLayer` on both the main composite path and the transition path.
## Test plan
- [x] `packages/producer/src/services/hdrImageTransferCache.test.ts` — 12 tests:
- hit/miss semantics
- distinct keys per image and per target transfer
- LRU eviction + promotion
- `maxEntries=0` passthrough
- source-buffer immutability for cached entries
- invalid options
- [x] Re-ran the Chunk 8A HDR benchmark — for the `hdr-regression` fixture (which has cross-transfer image layers) the cache hits 100% after the first frame; for HDR fixtures without cross-transfer images the same-transfer passthrough is a no-op.
## Stack
Chunk 8B of `plans/hdr-followups.md`. Sits on top of Chunk 8C (logger gating) and Chunk 8A (benchmark harness) so the win is measurable.
## What
Forwards the new per-phase extraction breakdown and `tmpPeakBytes` fields from `RenderPerfSummary` (added in #444 and #446) to PostHog via the CLI's existing `render_complete` telemetry event.
## Why
The CLI already ships `render_complete` events to PostHog (`packages/cli/src/telemetry/client.ts`), but `events.ts:trackRenderComplete` only carried a subset of `RenderPerfSummary` — top-level timings, composition dims, and memory snapshots. After #444 added per-phase extraction breakdown (`videoExtractBreakdown`) and #446 added cache hit/miss counters, the data lives on `job.perfSummary` at render-complete but never reaches PostHog dashboards.
Without this, any PostHog insight built around "how often are we hitting the cache?", "what's the median HDR preflight cost?", or "where in the extract phase do compositions spend time?" has to be answered by Datadog log scraping instead.
## How
- **`packages/cli/src/telemetry/events.ts`** — extend `trackRenderComplete` props with 17 new optional fields: `tmpPeakBytes`, the six named stage timings, and the ten `videoExtractBreakdown` fields. All sent as flat properties (`extract_cache_hits`, `stage_capture_ms`, etc.) — PostHog insights query flat keys more ergonomically than nested objects.
- **`packages/cli/src/commands/render.ts`** — wire `job.perfSummary.videoExtractBreakdown` / `stages` / `tmpPeakBytes` into the `trackRenderMetrics` → `trackRenderComplete` hand-off.
- Naming: `extract_phase3_ms` deliberately disambiguates from `stage_video_extract_ms` — the former is just the parallel ffmpeg extract inside Phase 3; the latter is the full stage (resolve + probe + preflight + extract).
- All new fields are optional. The Docker-subprocess branch of `render.ts` that doesn't have a local `perfSummary` still compiles and ships events without them.
## Test plan
- [x] `bun run --cwd packages/cli test` — 161/161 pass
- [x] `bunx tsc -p packages/cli/tsconfig.json --noEmit` — no errors
- [x] `bunx oxlint` + `bunx oxfmt` — clean
- [ ] Once merged, verify PostHog receives the new properties on a real render event (run `hyperframes render` against a fixture and watch PostHog ingestion — telemetry auto-disables in CI, so this requires a local dev render with `HYPERFRAMES_NO_TELEMETRY` unset).
## Stack
Depends on #444 (adds the `videoExtractBreakdown` + `tmpPeakBytes` fields to `RenderPerfSummary`) and transitively on #445 → #446.
## Future work (not in this PR)
- The HeyGen internal producer server (`hyperframes-internal/packages/producer/src/server.ts`) logs `perfSummary` to Datadog via `log.info` but has no PostHog integration. Production renders are the bulk of the traffic — separate PR to either ship perfSummary to PostHog from the internal server, or materialize Datadog log-based metrics for per-phase timings.
## What
Adds a content-addressed cache for extracted video frames, keyed on the tuple `(path, mtime, size, mediaStart, duration, fps, format)`. Repeat renders of the same composition (studio edit → re-render, preview → final) skip the ffmpeg extraction entirely.
## Why
Video frame extraction is the dominant non-capture phase for video-heavy compositions. Studio iteration workflows extract the same frames over and over — each render burns ffmpeg time that adds no value.
Validated on `/tmp/hf-fixtures/cfr-sdr-cache`:
```
Cold (miss): extractMs=69, videoExtractMs=70, totalElapsedMs=2052
Warm (hit): extractMs=1, videoExtractMs=2, totalElapsedMs=1964
cacheHits: 0→1, cacheMisses: 1→0
```
The fixture is tiny (3s CFR SDR @ 30fps), so the wall-clock delta is small; the extraction-time delta (69→1ms, 98%) scales linearly with source length. For heavy-iteration workflows (a user rendering the same composition while tuning encoding params), extraction time goes to zero on every repeat render.
Depends on #444 (instrumentation surface) and #445 (segment-scope HDR preflight — otherwise cache keys would be unstable across renders on mixed-HDR compositions).
## How
- New `packages/engine/src/services/extractionCache.ts`:
- SHA-256 key over a stable JSON encoding of `(path, mtime_ms, size, mediaStart, duration, fps, format)`. Infinity duration is normalized to `-1` so unresolved natural-duration sources still produce stable keys.
- Truncates to 16 hex chars in the entry directory name — 64 bits of entropy is plenty at cache scale and keeps `ls` output short.
- `hfcache-v2-` schema prefix — bumping it invalidates old entries (callers own gc policy; the cache owns keys).
- `.hf-complete` dotfile sentinel. An entry dir without the sentinel is treated as a miss (covers crash-mid-extract and abandoned writes); the next render re-extracts over the partial frames with `-y`.
- `FRAME_FILENAME_PREFIX = "frame_"` shared with the extractor — future refactors only need to touch one place to rename frames.
- `EngineConfig.extractCacheDir` (env: `HYPERFRAMES_EXTRACT_CACHE_DIR`) gates the feature. Undefined disables caching — extraction runs into the render's workDir and cleanup removes it on render end, preserving the prior behaviour exactly. No default root is chosen by the engine; the caller (CLI, app, studio) owns the location policy.
- `ExtractedFrames.ownedByLookup` flag prevents `FrameLookupTable.cleanup` from rm'ing a shared cache dir at render end. Set to `true` on both hits and misses (misses own the directory they wrote into, but hand it over to the cache rather than deleting it).
- Phase 3 extractor flow:
1. Snapshot `(videoPath, mediaStart, start, end)` per resolved video BEFORE Phase 2a/2b preflight mutates them — so cache keys are stable across renders that use workDir-local normalized files (those files have fresh mtimes every render).
2. Compute key, `lookupCacheEntry`.
3. On hit: rebuild `ExtractedFrames` from the cache dir plus the Phase 2-probed `VideoMetadata` — no re-ffprobe.
4. On miss: `ensureCacheEntryDir`, extract with `extractVideoFramesRange(..., outputDirOverride)`, then `markCacheEntryComplete` (the sentinel write is the last step so a crash leaves the dir un-sentineled).
- `extractVideoFramesRange` gains an `outputDirOverride` parameter so cache-miss writes land directly in the keyed dir (no `join(outputDir, videoId)` wrapping).
## Test plan
- [x] 19 unit tests in `extractionCache.test.ts` covering key determinism, mtime/size invalidation, format/fps/mediaStart/duration invalidation, Infinity normalization, sentinel semantics, missing-file tolerance
- [x] 2 integration tests in `videoFrameExtractor.test.ts`:
- "reuses extracted frames on a warm cache hit" — asserts `cacheHits=1`, `extractMs<50ms` on second call against a CFR SDR fixture
- "invalidates the cache when fps changes" — different fps on second call forces a new miss
- [x] End-to-end validation with `HYPERFRAMES_EXTRACT_CACHE_DIR` set, two runs of the same fixture
- [x] Lint + format (oxlint + oxfmt)
- [x] Typecheck (engine + producer)
## What
Scopes the SDR→HDR preflight re-encode to the segment the composition actually uses, mirroring the existing VFR→CFR segment-scope fix.
## Why
`convertSdrToHdr` was re-encoding entire source files, so a 30-minute SDR screen recording contributing a 2-second clip in a mixed HDR/SDR composition ate multi-second preflight time that produced frames no one would ever read. Validated on a mixed 30s-SDR + 2s-HDR fixture: `hdrPreflightMs` drops **87%** (1162→148ms), `videoExtractMs` drops **82%** (1272→231ms), `tmpPeakBytes` drops **45%** (8.2MB→4.5MB).
Depends on #444 (phase-level instrumentation) for the measurement surface.
## How
- `convertSdrToHdr` gains `startTime` and `duration` parameters ahead of the upstream `targetTransfer` arg added by #370. New signature: `convertSdrToHdr(input, output, startTime, duration, targetTransfer, signal, config)`. `-ss $start -t $duration` is added to the ffmpeg args.
- Phase 2 now captures the full `VideoMetadata` per `resolvedVideos` entry (previously just `colorSpace`) so the caller can compute `segDuration` from `video.end - video.start` with a fallback to `metadata.durationSeconds - video.mediaStart` for unbounded (Infinity) clips — without firing another ffprobe.
- After a successful convert, `entry.video.mediaStart` is zeroed out via shallow-copy (doesn't mutate the caller's `VideoElement`) so downstream extraction seeks from 0 instead of the original offset. Mirrors what the VFR→CFR path already does.
## Test plan
Validation on `/tmp/hf-fixtures/hdr-sdr-mixed-scope`:
```
hdrPreflightMs: >1000 → 150 (gate: <300) ✓
videoExtractMs: 1272 → 237 (-82%)
tmpPeakBytes: 8.2MB → 4.5MB (-45%)
```
- [x] Unit test: new regression test synthesizes 10s SDR + 2s HDR fixture inline and asserts the converted file's duration matches the 2s used segment (pre-fix matched the 10s source)
- [x] Lint + format
- [x] Typecheck
- [x] Manual perf validation against synthesized fixture
## What
Adds per-phase timings and counters to `extractAllVideoFrames` and surfaces them on the producer's `RenderPerfSummary` as `videoExtractBreakdown` alongside a new `tmpPeakBytes` workDir size sample.
## Why
Phase 2 video extraction has five distinct sub-phases (resolve, HDR probe, HDR preflight, VFR probe, VFR preflight, per-video extract) and today they collapse into a single `videoExtractMs` stage timing. That makes every subsequent perf PR in this stack immeasurable — you can't tell whether a win came from cache hits, preflight scope reduction, or pure extraction speed.
This PR is foundational for PR #445 (segment-scope HDR preflight) and PR #446 (content-addressed extraction cache).
## How
- New `ExtractionPhaseBreakdown` type with `resolveMs`, `hdrProbeMs`, `hdrPreflightMs/Count`, `vfrProbeMs`, `vfrPreflightMs/Count`, `extractMs`, `cacheHits`, `cacheMisses`. Populated inline with `Date.now()` wrappers — overhead is sub-millisecond on every phase.
- Returned on `ExtractionResult.phaseBreakdown`.
- Producer extends `RenderPerfSummary` with `videoExtractBreakdown?: ExtractionPhaseBreakdown` and `tmpPeakBytes?: number`. `tmpPeakBytes` is sampled from the workDir right before cleanup via a new recursive-size helper that swallows errors (purely observational — a missing workDir must never fail the render).
No changes to the capture-lifecycle resource tracking — earlier versions of this instrumentation plumbed injector LRU stats through `RenderOrchestrator`, which conflicted hard with upstream #371 (`buildHdrCaptureOptions` refactor). Dropped that piece for a marginal observability loss.
## Test plan
Validation on `packages/producer/tests/vfr-screen-recording`:
```json
"videoExtractBreakdown": {
"resolveMs": 0, "hdrProbeMs": 0, "hdrPreflightMs": 0, "hdrPreflightCount": 0,
"vfrProbeMs": 0, "vfrPreflightMs": 166, "vfrPreflightCount": 1,
"extractMs": 97, "cacheHits": 0, "cacheMisses": 0
},
"tmpPeakBytes": 4578598
```
Total elapsed within noise of pre-PR baseline (2665 → 2673 → 3228ms across hosts).
- [x] Unit test: phase-breakdown assertion added to `videoFrameExtractor.test.ts`
- [x] Lint + format (oxlint + oxfmt)
- [x] Typecheck (engine + producer)
- [x] Manual perf validation against VFR fixture
- Validate data-width/data-height: fall back to defaults if NaN or <= 0
- Sync existing #gl-canvas dimensions on reuse (if init called twice)
- Import DEFAULT_WIDTH/DEFAULT_HEIGHT in capture.ts instead of
hardcoding 1920/1080 in parameter defaults
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add dynamic resolution to shader transitions and update Skeleton A
to use shaders on vertical compositions.
shader-transitions changes:
- webgl.ts: read dimensions from params instead of hardcoded constants
- capture.ts: accept width/height for html2canvas
- hyper-shader.ts: read data-width/data-height from composition root
skill + docs changes:
- Skeleton A now has 1 shader at hero reveal (s3→s4 midpoint)
- Removed "no shaders on vertical" limitation from docs
- Updated claude-design.mdx known limitations section
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Read data-width/data-height from the composition root instead of
hardcoding 1920x1080. Enables shader transitions on vertical
(1080x1920) and square (1080x1080) compositions.
Changes across 3 files:
- webgl.ts: WIDTH/HEIGHT constants → DEFAULT_WIDTH/DEFAULT_HEIGHT,
createContext and renderShader accept width/height params
- capture.ts: captureScene and captureIncomingScene accept
width/height params for html2canvas
- hyper-shader.ts: reads data-width/data-height from root element,
passes dimensions to all webgl and capture calls
GLSL shaders unchanged — they already use u_resolution uniform for
all coordinate math and work at any aspect ratio.
Backwards compatible: all params default to 1920x1080 when not
provided or when data-width/data-height are missing from the DOM.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
James