ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
514 Commits 856 Branches 404 Tags
fe017b48c7420be8436a54166ff0962280c098b1
Commit Graph
514 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Vance Ingalls 150d9348bc perf(player): srcdoc composition switching for studio (#398) 
## Summary

Adds `srcdoc` support to `<hyperframes-player>` and uses it from studio's `Player.tsx` so composition switches no longer trigger an iframe navigation. Studio fetches the composition HTML on the parent and hands it to the iframe inline; the browser skips the navigation request, preconnect/handshake, and a redundant cache lookup.

## Why

Step `P3-2` of the player perf proposal. Profiling studio's project switcher showed that ~30–80 ms of every composition swap was spent in the iframe's own navigation pipeline — DNS / TCP / TLS reuse checks, request hand-off to the network process, and the second cache lookup against the same origin we just fetched from. For same-origin previews (`/api/projects/.../preview`) this is pure overhead: the parent already has the bytes (or can pull them from its own HTTP cache).

`srcdoc` lets us skip that pipeline entirely. The iframe loads from an in-memory string and the parent's `fetch` reuses any existing response from the page's HTTP cache, so the second-and-Nth composition switch in a session is essentially free at the network layer.

## What changed

### `<hyperframes-player>` (`packages/player/src/hyperframes-player.ts`)

- Added `srcdoc` to `observedAttributes` so runtime swaps actually fire `attributeChangedCallback`.
- On connect, both `srcdoc` and `src` are forwarded to the inner iframe — no manual precedence; the HTML spec already says `srcdoc` wins when both are present, so the browser handles arbitration.
- New `srcdoc` branch in `attributeChangedCallback`:
  - Resets `_ready = false` on every change so the next iframe `load` event re-runs probe/control/poster setup against the fresh document.
  - Distinguishes `setAttribute("srcdoc", "")` (deliberate empty document) from `removeAttribute("srcdoc")` (fall back to `src`) — the former propagates an empty-string srcdoc; the latter strips the attribute so a previously-set `src` can take over.

### Studio `Player.tsx` (`packages/studio/src/player/components/Player.tsx`)

- Hoisted `AbortController` and resolved `url` outside the dynamic-import `.then()` so the cleanup function can cancel an in-flight composition fetch when the user navigates away mid-load.
- After the player module loads, `fetch(url, { signal })` pulls the composition HTML on the parent.
  - Success → `player.setAttribute("srcdoc", html)`.
  - Network error / non-2xx → fall back to `player.setAttribute("src", url)`. Same code path the player has always taken, so this optimization is strictly a win — never a regression.
  - `AbortError` → bail without touching the DOM (component is unmounting).
- Attributes are set **before** `appendChild` so the iframe never loads an intermediate `about:blank`. That matters because:
  1. The first iframe `load` event must fire for the real composition; the existing handler treats `loadCountRef > 1` as a hot-reload and replays the reveal animation. An extra `about:blank` load would trigger the reveal on initial mount.
  2. `useTimelinePlayer` hangs setup off the first load — running it against an empty document is wasted work.

## Test plan

- [x] 7 new unit tests in `hyperframes-player.test.ts` covering:
  - `srcdoc` is in `observedAttributes`.
  - Initial `srcdoc` set before connect forwards to the iframe on connect.
  - Runtime `srcdoc` set after connect forwards via `attributeChangedCallback`.
  - `_ready` resets when `srcdoc` changes so `onIframeLoad` replays setup.
  - `removeAttribute("srcdoc")` strips the attribute on the iframe so `src` can take over.
  - Empty-string `srcdoc` is preserved (not treated as removal).
  - Both `src` and `srcdoc` set together: both get forwarded to the iframe and the browser arbitrates per spec.
- [x] Studio fallback path verified manually — disabling fetch falls back to the original `src` flow with no regression.

## Stack

Step `P3-2` of the player perf proposal. Builds on `P3-1` (sync seek) — both target the studio editor's interactive feel. With sync seek removing scrub latency and `srcdoc` removing composition-switch latency, the editor's two most-frequent interactions both shed their iframe-navigation overhead.
 2026-04-22 17:59:01 -07:00
Vance Ingalls ef3de5bcd3 feat(player): synchronous seek() API with same-origin detection (#397) 
## Summary

Formalizes the same-origin shortcut Studio has been using privately (`iframe.contentWindow.__player.seek` in `useTimelinePlayer.ts`) as a first-class behavior of `<hyperframes-player>`'s public `seek()` method. Same-origin seeks now land in the same task as the input event — no postMessage hop, no extra microtask, no perceived scrub lag. Cross-origin embeds fall through to the existing async bridge transparently.

## Why

Step `P3-1` of the player perf proposal. The current `seek()` always posts a message to the iframe runtime, which means a single user scrub incurs:

1. JS task: fire postMessage from parent
2. Browser task switch into iframe context
3. Microtask: handler dispatches
4. Frame: runtime calls `markExplicitSeek` and updates DOM

Same-origin embeds (Studio, preview pane, embedded compositions) can skip all four by calling the runtime's `seek` directly. Studio was already doing this manually but had to duplicate the local-state bookkeeping (`_currentTime`, `paused`, controls UI) — making it a first-class behavior of the player removes the workaround and gives every same-origin consumer the win for free.

## What changed

- New `_trySyncSeek(time)` helper attempts a synchronous call into the iframe's `window.__player.seek`. Returns `true` on success, `false` on cross-origin or pre-bootstrap.
- `seek()` calls `_trySyncSeek` first, falls through to the existing `_sendControl` postMessage path when sync isn't available.
- Detection is a `try/catch` on `contentWindow` access (real cross-origin iframes throw `SecurityError`) plus a `typeof` guard on `__player.seek`.
- Local `_currentTime`, the `paused` flag, and the controls UI update on both paths so scrubs never leave stale state.
- Runtime-side `seek` is the same wrapped function the postMessage handler calls — `installRuntimeControlBridge` routes through `player.seek`, so `markExplicitSeek()` and downstream runtime state are identical between the two paths.

## Test plan

- [x] 11 new unit tests in `hyperframes-player.test.ts` covering:
  - Same-origin sync path executes `__player.seek` synchronously and skips postMessage.
  - Cross-origin (simulated `SecurityError` on `contentWindow`) falls back to postMessage.
  - Pre-bootstrap (no `__player` installed) falls back to postMessage.
  - `__player.seek` not a function falls back to postMessage.
  - `_currentTime`, `paused`, and controls all stay in sync on both paths.
  - Errors thrown from `__player.seek` propagate without corrupting state.

## Stack

Step `P3-1` of the player perf proposal. Independent of the `P1-*` work — this is a pure latency win on the seek/scrub path. Combined with `P3-2` (srcdoc composition switching, next in the stack) it removes most of the iframe-bridge overhead from the studio scrubber.
 2026-04-22 17:50:23 -07:00
Vance Ingalls f906797222 perf(player): coalesce _mirrorParentMediaTime writes (#396) 
## Summary

Coalesce writes to `el.currentTime` inside `_mirrorParentMediaTime` so a single jitter sample no longer triggers a parent-media seek. A drift correction now requires **two consecutive samples** above the threshold (~`MIRROR_DRIFT_THRESHOLD_SECONDS`) before the player writes back. One-shot alignment paths (`promoteToParentProxy`, `_onIframeMediaAdded`) opt out via `force: true` so initial alignment stays immediate.

## Why

Step `P1-4` of the player perf proposal. `_mirrorParentMediaTime` is called every animation frame on parent media proxies. Even without true drift, browser internals report tiny jitter on `currentTime` reads — typically below 30 ms but occasionally crossing the threshold for a frame. Writing to `currentTime` triggers a seek, which is expensive *and* invalidates pipeline buffers, which causes the next frame's reading to jitter further. The result was unnecessary seek thrash on otherwise-aligned media.

By requiring two consecutive over-threshold samples, transient jitter is filtered out while real drift (a sustained offset) still corrects within ~1 frame of latency. This eliminates the most common cause of dropped frames on the studio thumbnail grid.

## What changed

- Each `_parentMedia` entry gains a `driftSamples` counter that increments while the absolute drift is above `MIRROR_DRIFT_THRESHOLD_SECONDS` and resets to 0 on the first sample below.
- `_mirrorParentMediaTime(el, opts)` only writes back when `driftSamples >= 2`, except when `opts.force === true`.
- `promoteToParentProxy` and `_onIframeMediaAdded` pass `force: true` so the first alignment after registration is still immediate (these are user-visible state transitions, not steady-state telemetry).

## Test plan

- [x] 11 new unit/integration tests in `hyperframes-player.test.ts` covering:
  - Single-sample jitter does not trigger a write.
  - Two-sample sustained drift does trigger a write.
  - Trending drift correction (gradually increasing offset) is detected within 2 samples.
  - `force: true` override bypasses the sample requirement.
  - Out-of-range proxies (proxies whose source has been removed) do not panic.
  - Multiple proxies maintain independent counters — drift on one does not affect the other.
  - `_promoteToParentProxy` alignment is immediate.

## Stack

Step `P1-4` of the player perf proposal. Builds on `P1-1` (shared adopted stylesheets) and `P1-2` (scoped media observer). Together these three target the studio multi-player render path — `P0-1*` perf gate scenarios will pick up the wins automatically.
 2026-04-22 17:44:49 -07:00
Vance Ingalls 113f9eafd5 ci: subscribe to edited PR events so workflows re-fire after Graphite restacks (#429) 
## What

Brief description of the change.

## Why

Why is this change needed?

## How

How was this implemented? Any notable design decisions?

## Test plan

How was this tested?

- [ ] Unit tests added/updated
- [ ] Manual testing performed
- [ ] Documentation updated (if applicable)
 2026-04-22 17:32:08 -07:00
Vance Ingalls 9512744c2e refactor(shader-transitions): extract DEFAULT_DURATION and DEFAULT_EASE constants (#367) 
## Summary

Extract `DEFAULT_DURATION = 0.7` and `DEFAULT_EASE = "power2.inOut"` as shared constants in `hyper-shader.ts` and apply them at all three fallback sites (metadata write, browser/render mode, engine mode).

## Why

`Chunk 2` of `plans/hdr-followups.md`. The three fallback sites had drifted apart: the metadata path used `1s` / `"none"` while the actual rendering used `0.7s` / `"power2.inOut"`. A transition that omitted `duration`/`ease` would render at 0.7 s but tell the producer it was 1 s, throwing off the producer's compositing window planning and producing a visible ~0.3 s brightness dropout.

This is a small, high-value correctness fix that runs before the larger Chunk 1 / Chunk 4 work.

## What changed

- New module-level `DEFAULT_DURATION` and `DEFAULT_EASE` constants in `packages/shader-transitions/src/hyper-shader.ts`.
- All three fallback call sites (metadata, browser, engine) now use the constants.
- Explicit `ease: "none"` on the timeline-length anchor tweens elsewhere in the file is intentional (those are linear interpolators driving the shader's progress uniform) and is left unchanged.

## Test plan

- [x] Render a composition with a transition that omits `duration` and `ease` — no brightness dip in the last ~0.3 s of the transition.
- [x] Preview (browser mode) and render (engine mode) produce matching blending curves.
- [x] Render with explicit `duration: 1.5` still works (constants are fallbacks only).

## Stack

Chunk 2 of `plans/hdr-followups.md`. Lands ahead of Chunk 1 (opacity) per the suggested merge order.
 2026-04-22 17:02:42 -07:00
Vance Ingalls 5de5df7fbb refactor(types): tighten type safety, dedupe HfTransitionMeta, prune dead LUT export (#366) 
## Summary

Four small, mechanical type-safety cleanups across `engine`, `producer`, and `shader-transitions`. Zero behavior change — pure pre-cleanup so the rest of the stack ships against a tighter baseline.

## Why

`Chunk 6` of `plans/hdr-followups.md`. Several non-null assertions and a duplicate interface had accumulated as rebase artifacts and leftover work-in-progress; lands first because it touches files later chunks edit and removes friction during review.

## What changed

- `renderOrchestrator.ts`: replace `layers[layerIdx]!` with a `for (const [layerIdx, layer] of layers.entries())` so both index and element come from the iterator.
- `engine/types.ts`: drop the duplicate `HfTransitionMeta` interface (rebase artifact); the original definition above it is the documented one. The orphaned doc comment now precedes `HfProtocol`.
- `shader-transitions/hyper-shader.ts`: keep the local `HfTransitionMeta` declaration (the package ships as a standalone CDN bundle and must not depend on `@hyperframes/engine`), but add a sync comment pointing at the source of truth in `engine/src/types.ts`.
- `alphaBlit.ts` + `engine/index.ts`: drop `export` from `getSrgbToHdrLut` and remove its re-export. It was only ever called by the internal `blitRgba8OverRgb48le`; the public surface was dead code.

## Test plan

- [x] `bun run --filter @hyperframes/engine typecheck`
- [x] `bun run --filter @hyperframes/producer typecheck`
- [x] `bun run --filter @hyperframes/shader-transitions typecheck`
- [x] `bun run --filter @hyperframes/engine test` — 308/308 pass (no test changes; assertions removed in code only).

## Stack

Chunk 6 of `plans/hdr-followups.md`. Mechanical cleanup landed early per the suggested merge order.
 2026-04-22 16:56:17 -07:00
Vance Ingalls a6e14da45c perf(player): scope MutationObserver to composition hosts (#395) 
## Summary

Replace the body-wide `MutationObserver` in `<hyperframes-player>` with one scoped to top-level `[data-composition-id]` hosts. The wide observer fired on every body-level mutation — analytics scripts, runtime telemetry markers, dev overlays — even though only composition subtrees can introduce new timed media (`<audio data-start>`, etc.).

## Why

Step `P1-2` of the player perf proposal. The previous implementation observed `iframe.contentDocument.body` with `subtree: true` to pick up sub-composition `<audio data-start>` elements added after initial mount. That worked, but it was paying for callbacks from every unrelated DOM mutation in the iframe — most of which are just runtime instrumentation. Hot paths in the studio (timeline updates, telemetry markers) end up triggering the observer dozens of times per frame.

Scoping to composition hosts cuts the noise by ~10× in the studio without losing any of the timed-media wiring guarantees.

## What changed

- New `selectMediaObserverTargets(doc)` helper in `packages/player/src/mediaObserverScope.ts` that selects all top-level `[data-composition-id]` elements **excluding** nested ones — sub-composition hosts whose media is already covered by the parent observer's `subtree: true`.
- The player now attaches a single `MutationObserver` instance per top-level host (`subtree: true`), so callbacks still batch across hosts but skip out-of-host noise.
- Falls back to observing `body` when no composition hosts exist (e.g. blank iframe between `src` changes) — preserves prior behavior for non-composition documents and avoids breaking the bootstrap path.

## Test plan

- [x] 8 new unit tests in `mediaObserverScope.test.ts` covering empty docs, single host, multiple hosts, nested-host filtering, and the body-fallback path.
- [x] 2 new integration tests in `hyperframes-player.test.ts` spying on `MutationObserver.prototype.observe` to confirm the targets and options the player actually attaches in a real custom-element bootstrap.

## Stack

Step `P1-2` of the player perf proposal. Sits between `P1-1` (shared adopted stylesheets) and `P1-4` (coalescing parent media-time mirror writes) — together they target the studio multi-player render path. The perf gate scenarios in `P0-1*` will pick up the wins automatically.
 2026-04-22 16:46:54 -07:00
Vance Ingalls d7c1050e44 test(producer): add hdr-regression and hdr-hlg-regression test suites (#365) 
## Summary

Replace the trivial `hdr-pq` and `hdr-image-only` tests with two consolidated, time-windowed regression suites that exercise the full HDR pipeline. These goldens are the safety net for every other PR in this stack.

## Why

The pre-existing HDR tests covered only a single full-bleed video or image with a static text label — none of the features that the HDR pipeline has to handle differently from SDR (opacity animation, z-ordered multi-layer compositing, transforms, border-radius clipping, shader transitions, multiple HDR sources, object-fit modes, mixed HDR+SDR layering, HLG transfer). This PR builds the missing safety net first so every subsequent fix can be proven correct.

## What changed

- New `packages/producer/tests/hdr-regression/` (PQ, BT.2020, ~20 s, 1080p, 8 windows A–H):
  - A: static baseline (HDR video + DOM overlay)
  - B: wrapper-opacity fade
  - C: direct-on-`<video>` opacity tween (documents the Chunk 1 bug)
  - D: z-order sandwich (DOM → HDR → DOM)
  - E: two HDR videos side-by-side (pins PR #289)
  - F: rotation + scale + border-radius (documents the Chunk 4 bug)
  - G: `object-fit: contain`
  - H: shader crossfade between HDR video and HDR image
- New `packages/producer/tests/hdr-hlg-regression/` (HLG, ARIB STD-B67, ~5 s, 2 windows A–B) — exercises the separate HLG LUT/OETF code path that previously had **zero** coverage.
- New `scripts/generate-hdr-photo-pq.py` synthesizes `hdr-photo-pq.png` with a cICP chunk for BT.2020/PQ/full.
- Removed `tests/hdr-pq/` and `tests/hdr-image-only/`.
- Updated `.github/workflows/regression.yml` HDR shard to run the new pair sequentially.
- All compositions follow the documented timed-element pattern (`data-start`, `data-duration`, `class="clip"` directly on each timed leaf — no wrapper inheritance).

## Test plan

- [x] Goldens generated with `bun run test:update --sequential`.
- [x] `ffprobe` confirms HEVC/yuv420p10le/bt2020nc/smpte2084 (PQ) and arib-std-b67 (HLG).
- [x] Suite green with `maxFrameFailures` budgets that absorb the documented Chunk 1 / Chunk 4 known-fails — tightened in follow-up PRs in this stack.

## Stack

Foundational PR for the HDR follow-ups stack (Chunk 0 of `plans/hdr-followups.md`). Every subsequent PR builds on this safety net.
 2026-04-22 15:43:04 -07:00
Vance Ingalls ed62894d01 perf(player): share PLAYER_STYLES via adoptedStyleSheets (#394) 
## Summary

Replace per-instance `<style>` injection in `<hyperframes-player>` with a lazily constructed `CSSStyleSheet` adopted via `shadowRoot.adoptedStyleSheets`. One parsed stylesheet, many adopters — the studio thumbnail grid renders dozens of players concurrently and was paying for N parses of the same CSS.

## Why

Step `P1-1` of the player perf proposal. The previous implementation appended a `<style>` element to every shadow root, which means:

- N shadow roots → N copies of the same CSS string parsed into N independent style sheets.
- Each `<style>` lives in the DOM and contributes to layout/style invalidation work when its shadow root churns.
- The studio's project grid mounts ~30 players on initial load — that's 30 redundant parses of the same ~1 KB stylesheet on the critical path.

`adoptedStyleSheets` flips this: parse once at module load, hand the same `CSSStyleSheet` reference to every shadow root.

## What changed

- New `getSharedPlayerStyleSheet()` in `packages/player/src/styles.ts` — module-scoped and memoized; the sheet is built once per process and returned to every adopter.
- New `applyPlayerStyles(shadow)` is the single integration point. It **appends** (never replaces) the shared sheet so any pre-adopted sheets — host themes, scoped overrides, future caller-side injections — survive intact, and is idempotent so repeated calls don't multiply adoptions.
- SSR-safe via a `typeof CSSStyleSheet` guard. Failures (e.g. `replaceSync` throw, no constructor) are cached as `null` so we don't retry constructor failures forever.
- Defensive fallback path creates a per-instance `<style>` element when `adoptedStyleSheets` is unavailable (older runtimes, hostile environments). Behavior on those paths is unchanged from before.
- `PLAYER_STYLES`, `PLAY_ICON`, and `PAUSE_ICON` exports preserved — no public API change.

## Test plan

- [x] Unit tests in `styles.test.ts` cover sharing across instances, fallback when `CSSStyleSheet` is undefined or `replaceSync` throws, fallback when `adoptedStyleSheets` is unsupported on the shadow root, idempotency, and preservation of pre-existing adopted sheets.
- [x] Integration test in `hyperframes-player.test.ts` confirms two real `<hyperframes-player>` elements adopt the same `CSSStyleSheet` instance and inject zero `<style>` elements.
- [x] Build size delta is negligible (utility code replaces `container.appendChild` calls).

## Stack

Step `P1-1` of the player perf proposal. Followed by `P1-2` (scoping the media `MutationObserver`) and `P1-4` (coalescing parent media-time mirror writes) — all three target the studio multi-player render path.
 2026-04-22 15:40:33 -07:00
Vance Ingalls f9863ab565 feat(core): add emitPerformanceMetric bridge for runtime telemetry (#393) 
## Summary

Extend the runtime analytics bridge with a numeric performance metric channel. Hosts subscribe via the existing postMessage transport (one bridge, two channels) and aggregate per-session p50 / p95 for scrub latency, sustained fps, dropped frames, decoder count, composition load time, and media sync drift before forwarding to their observability pipeline.

This is the foundation other perf tooling sits on — the player itself emits the events; player-side aggregation and flush land in a follow-up.

## Why

Step `X-1` of the player perf proposal. Today there is no way for an embedding host to learn that scrub latency spiked, that a composition took 3 s to load, or that the media-sync loop is running 200 ms behind real time. The only signals are anecdotal user reports.

A single shared bridge keeps the runtime → host surface area minimal: hosts that already wire up the analytics channel get perf for free, and hosts that don't aren't paying for it.

## What changed

- New `emitPerformanceMetric(name, value, tags?)` helper in `@hyperframes/core` that forwards a `{ type: "performance-metric", name, value, tags }` envelope through the existing analytics postMessage transport.
- Six initial metric names defined in the proposal:
  - `scrub_latency_ms` — wall-clock from `seek()` call to first paint at the new frame.
  - `playback_fps` — sustained rAF cadence during play.
  - `dropped_frames` — count of >25 ms gaps within a play window.
  - `decoder_count` — number of concurrently-decoding video elements.
  - `composition_load_ms` — navigation-start to player-ready.
  - `media_sync_drift_ms` — drift between expected and actual decoder time.
- Each emit also writes a `performance.mark()` with `{ value, tags }` on `detail`, so the same numbers surface in the DevTools Performance panel's User Timing track for local debugging without instrumenting the host.
- Zero PostHog (or any other analytics SDK) dependency in `core` — the host decides where to forward the events.

## Test plan

- [x] Unit tests cover the envelope shape, the `performance.mark` mirror, and the no-op path when no host has wired up the bridge.
- [x] Manual: verified marks appear in the User Timing track when scrubbing the studio preview.

## Stack

Step `X-1` of the player perf proposal. Foundation for the perf gate (P0-1a/b/c) — the perf scenarios in this stack instrument these same channels for CI measurement.
 2026-04-22 15:08:14 -07:00
James Russo ef26798e98 ci(regression): build test Docker image once, share across shards (#427) 
* ci(regression): build test Docker image once, share across shards

Splits regression.yml into a `build-image` job + the existing
`regression-shards` matrix. The build job produces a Docker tarball via
`docker/build-push-action` with `outputs: type=docker,dest=...`, uploads
it as a GHA artifact (retention 1 day, gzip level 1), and each shard
downloads + `docker load`s it instead of rebuilding.

Measured on PR #419 regression runs before the change:
- Docker build step: ~234s per shard WITH GHA layer cache hit
- 11 shards × ~234s = ~43 min of runner time per PR just on redundant
  image builds

Cold-cache cases are much worse — happening right now on PR #419 after
release commit b6f50ce bumped every `packages/*/package.json`, invalidating
the COPY layer that feeds `bun install --frozen-lockfile`. All 10 shards
are currently 25-30+ min into a parallel rebuild, thundering-herding
the same npm packages from 10 runners.

After this change:
- 1× build (~4 min warm, ~15 min cold) + 11× (download + `docker load`)
- Expected ~15-20s overhead per shard for artifact download + load
- Net savings: ~30-40 min of runner time per PR run on warm cache,
  substantially more on cold cache

The build job doesn't checkout LFS — Dockerfile.test only COPYs source +
package manifests, never the golden baselines, so the image build never
needed LFS. Shards still need LFS for the tests/**/output/output.mp4
baselines they validate against.

* ci(regression): add explicit least-privilege permissions

Addresses CodeQL warning 'Workflow does not contain permissions'.
Defaults the workflow GITHUB_TOKEN to `contents: read` only. The
build-image job elevates to `actions: write` because
`docker/build-push-action` with `cache-from/to: type=gha` uses the
GitHub Actions cache API, which needs read+write on the actions scope.
 2026-04-22 14:36:52 -07:00
James Russo 6accf099ac docs(readme): note git-lfs requirement for full clones (#423) 
* docs(readme): note git-lfs requirement for full clones

Repo uses Git LFS for regression-test baselines (~240 MB of .mp4 files
under packages/producer/tests/**/output.mp4). Users cloning without
git-lfs installed hit a cryptic 'git-lfs: command not found' error, as
reported in #407.

Document the requirement with install instructions and the
GIT_LFS_SKIP_SMUDGE=1 escape hatch.

* docs(readme): add Windows install instructions for git-lfs

Per review from @miguel-heygen.

* chore(ci): fix oxfmt formatting on renovate.json

Drive-by to unblock CI. Landed unformatted in #422 because
Renovate's config-migration PR bypasses the lefthook pre-commit hook,
so every subsequent PR's `bun run format:check` (which scans the whole
repo) was failing on this file.
 2026-04-22 14:33:02 -07:00
Miguel Ángel 5be207f034 chore: release v0.4.13 v0.4.13 2026-04-22 17:28:11 -04:00
James Russo 2e8e579df2 ci: skip PR runs when targeting a non-main base branch (#426) 
* ci: skip PR runs when targeting a non-main base branch

Adds `branches: [main]` to the `pull_request:` trigger of each workflow
that runs on PRs (CI, regression, Windows render verification, Docs,
Catalog Previews). PRs whose base is something other than main — typical
for stacked PRs — no longer trigger these workflows.

On a 5-PR Graphite stack this turns 5× CI runs into 1× (when the tip
of the stack reaches main). When a child PR is rebased/promoted so its
base becomes main, CI fires as normal.

publish.yml and the default CodeQL setup are untouched: publish already
filters to main, and CodeQL is default-setup (org UI, not a repo YAML).

* chore(ci): fix oxfmt formatting on renovate.json

Same drive-by as #423. Renovate's config-migration PR #422 landed
unformatted (Renovate bot skips lefthook), so every PR branched from
current main fails `bun run format:check`. Whichever of #423 / #426
merges first cleans it up.
 2026-04-22 14:19:04 -07:00
James Russo 5ab97a6af9 ci(regression): add concurrency group to cancel superseded runs (#425) 
Matches the pattern already in place on ci.yml, docs.yml,
windows-render.yml, and catalog-previews.yml. The regression workflow
was the only one without it.

Without this, rapid pushes to a PR leave prior regression runs still
executing their full matrix (~10 parallel shards across styles-a..g,
fast, render-compat, hdr) even though they'll be thrown away. On a busy
day this alone can eat a double-digit share of the GitHub hosted runner
pool and stretch queues for every open PR.
 2026-04-22 13:56:18 -07:00
renovate[bot] 7800a9ffed chore(config): migrate config .github/renovate.json (#422) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2026-04-22 12:16:10 -07:00
Miguel Ángel fc52d21c59 docs: clarify composition variable usage (#420) 
## Summary
- replace the unsupported `data-var-*` example with the current `data-variable-values` pattern
- document that variable values are carried through but still applied manually inside the nested composition
- add matching reference notes in the data-attributes, HTML schema, core package, and CLI docs

## Verification
- `npx mintlify dev --port 3100`
- browser verification with `agent-browser` on `/concepts/compositions` and `/reference/html-schema`
- proof artifacts saved locally under `tmp/issue-416-docs/`
 2026-04-22 20:41:35 +02:00
James Russo 4df85579bb chore(ci): add Renovate config for grouped dependency updates (#417) 2026-04-22 11:25:37 -07:00
Miguel Ángel b6f50ce4c7 chore: release v0.4.13-alpha.4 v0.4.13-alpha.4 2026-04-22 12:48:35 -04:00
Miguel Ángel 5a4dd8bec1 fix: gate studio timeline actions by capability (#415) 
## Summary
- gate timeline actions to clips Studio can control deterministically
- disable direct move/trim for generic GSAP-timed DOM clips
- add an in-clip `Copy to Agent` fallback for unsupported edits

## Why
Studio should only advertise timeline actions it can round-trip to source HTML with deterministic meaning.

This PR now follows that stricter rule:
- direct move/end-trim are only exposed for clips with a deterministic timeline window
- start trim is only exposed for clips with a real content-offset model
- unsupported motion clips now offer `Copy to Agent` so users still have a fast path to request source-level timing changes

In practice this means generic GSAP-authored DOM clips no longer pretend Studio can rewrite their visible timing just by patching `data-start` / `data-duration`.

## What changed
- added `hasPatchableTimelineTarget()` and `getTimelineEditCapabilities()` in `timelineEditing.ts`
- tightened deterministic-window detection so only media, images, and composition hosts keep direct move/end-trim controls
- kept wrapped media clips editable by recognizing real media metadata even when the host tag is a `div`
- updated `TimelineClip` / `Timeline` to guard interactions with the shared capability model
- added `buildTimelineElementAgentPrompt()` and a `Copy to Agent` fallback button for unsupported clips
- added focused tests for capability derivation and the agent-prompt helper

## Verification
### Automated
- `bun test packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/utils/sourcePatcher.test.ts`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/TimelineClip.tsx`
- `bunx oxfmt --check packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/TimelineClip.tsx`

### Browser
Verified in Studio with live browser automation against `http://127.0.0.1:4175/#project/timeline-edit-playground`:
- generic GSAP-timed clips (`feature-card`, `title-card`, `prompt-card`) show `Copy to Agent` and no direct move/trim affordances
- wrapped media (`media-card`) still exposes direct controls and remains draggable
- the local playground timings were realigned to match the authored GSAP positions, so preview visibility now matches the timeline windows during manual testing

Recording artifacts used during verification:
- `/tmp/timeline-capabilities-proof/capabilities-flow.webm`
- `/tmp/timeline-capabilities-proof/capabilities-agent-flow.webm`
 2026-04-22 18:44:17 +02:00
Miguel Ángel c46abf9fa2 chore: release v0.4.13-alpha.3 v0.4.13-alpha.3 2026-04-22 11:14:31 -04:00
Miguel Ángel 2cf3558f8e fix(studio): only expose front trim for offsettable clips (#413) 
## Summary
- hide the leading trim handle for timeline clips that cannot offset their own content
- keep leading trim available for media clips backed by playback offset metadata or source duration
- map visual row priority like a normal timeline editor: top timeline rows render above lower rows

## Why This Is Needed
Generic GSAP/DOM timeline clips do not have a playback-offset model like media clips do.

That means a left trim affordance on those clips is misleading today:
- users reasonably expect front trim to remove the beginning of the animation
- the current model can only shorten the clip window, not start the motion halfway through

Instead of exposing a control that implies unsupported behavior, this PR keeps true front trim only on clips that can actually offset their content.

The PR also fixes the stacking convention so the timeline matches normal editor expectations:
- visually higher track row = higher render priority
- visually lower track row = lower render priority

## Current Flow By Element Type
### Generic motion / DOM clips
Examples: `section`, `div`, `aside`, GSAP-driven cards and overlays.

Current supported flow:
- drag the whole clip horizontally to change `data-start`
- right-trim to shorten the end of the clip window
- move between tracks to change `data-track-index`

Not supported yet:
- true front trim that removes the beginning of the animation itself

Behavior after this PR:
- no interactive left trim handle is shown
- right trim still works
- horizontal move still works

### Media clips
Examples: `video` / `audio` clips, or wrappers carrying `data-media-start` / `data-playback-start`.

Current supported flow:
- drag the whole clip horizontally to change `data-start`
- left trim advances clip start and playback offset together
- right trim shortens `data-duration`

Behavior after this PR:
- both left and right trim handles remain available
- left trim persists `data-start` plus `data-media-start` / `data-playback-start`
- right trim persists `data-duration`

## Z-Index Rule
This PR now follows the normal timeline-editor convention:
- top visual row on the timeline = highest `z-index`
- lower visual rows = lower `z-index`

Concretely, because Studio renders tracks in ascending numeric order from top to bottom, lower numeric track values now map to higher `z-index` values.

## Validation
### Automated
- `bun test packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/utils/sourcePatcher.test.ts`
- `bun run --filter @hyperframes/studio typecheck`

### Browser verification
Verified with `agent-browser` on `timeline-edit-playground`:
- generic motion clips no longer expose an interactive left trim handle
- media clips still expose both trim handles
- left trim on `media-card` persisted `data-start` and `data-media-start`
- right trim on `media-card` persisted `data-duration` only
- moving `title-card` from the bottom row to the top row persisted the highest `z-index` for the top-row clips
- recordings:
  - `/tmp/trim-fix-artifacts/trim-flow.webm`
  - `/tmp/trim-fix-artifacts/z-index-flow.webm`
 2026-04-22 17:11:15 +02:00
Miguel Ángel d740f5ce42 fix: nested GSAP sub-composition lint and render handling (#405) 
## Summary

- allow nested sub-composition files to inherit GSAP from their host without tripping `missing_gsap_script`
- keep nested render seeks stable for sub-compositions without regressing producer baselines
- stop producer render-hint detection from treating the compiler's own nested mount retry wrapper as user-authored `requestAnimationFrame()` usage

## Root Cause

- the core linter treated template-based nested compositions like standalone root compositions, so it incorrectly required a local GSAP loader even when the host composition already provided GSAP
- producer `detectRenderModeHints()` runs before CDN scripts are inlined, so nested GSAP exports were never failing because of the GSAP payload itself
- the nested-only false positive came from the compiler-generated mount bootstrap that waits for the inlined sub-composition root with `requestAnimationFrame()` before running the hoisted inline script
- preview and export seek paths also needed to stay split so the nested timeline re-arm behavior that stabilizes scrubbing does not collapse render baselines

## What Changed

- lint: keep the nested GSAP false-positive fix and regression coverage for template sub-compositions
- runtime: keep the render-seek behavior that preserves nested child offsets during export without changing preview scrubbing behavior
- producer: mark compiler-owned mount bootstrap blocks and strip only those blocks before scanning inline scripts for raw `requestAnimationFrame()`
- producer tests now cover both cases: compiler-generated wrappers are ignored, but real user-authored nested `requestAnimationFrame()` still opts into screenshot mode

## Validation

- `bun test packages/core/src/lint/rules/gsap.test.ts`
- `bun test packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxfmt packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxlint packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bun run --filter @hyperframes/producer test --sequential chat style-11-prod`
  - `style-11-prod` passed locally
  - `chat` still shows local-only visual drift on this macOS/ARM workstation, but the render metadata now reports `renderModeHints.recommendScreenshot=false`, which is the concrete acceptance condition for `#402`
- Docker CI-image repro is blocked locally by OrbStack x86/arm64 loader mismatch, so final regression confirmation is deferred to GitHub Actions

Closes #392
Closes #402
 2026-04-22 16:10:38 +02:00
Miguel Ángel 29b6274ebc chore: release v0.4.13-alpha.2 v0.4.13-alpha.2 2026-04-22 00:27:06 -04:00
Miguel Ángel d38a4f19fd fix(core): restore nested seek scrubbing (#404) 2026-04-22 06:13:09 +02:00
Miguel Ángel e0749ab768 chore: release v0.4.13-alpha.1 v0.4.13-alpha.1 2026-04-21 19:57:04 -04:00
Miguel Ángel 1aea1415c4 fix: smooth scrubber end seeking (#386) 
* fix: smooth scrubber end seeking

* fix: stop timeline auto-scroll in fit mode

* feat: use percentage-based timeline zoom

* fix: sync timeline playhead on zoom changes

* fix: reset timeline scroll when returning to fit

* fix: keep timeline controls pinned
 2026-04-22 01:51:41 +02:00
Miguel Ángel 0ba56f9187 feat: add studio timeline editing (#390) 
## Summary

Add the actual Studio timeline editing layer on top of the preview/runtime foundation.

This PR includes:

- drag-to-move clips across time and tracks
- left/right resize handles with media-aware trim persistence
- edge auto-scroll and edge track creation while dragging
- selector-based source patching for `data-start`, `data-duration`, `data-track-index`, `z-index`, and media trim attributes
- timeline UI cleanup, theming, hover/drag states, and the `Copy Prompt` action

## Why This PR Is Separate

This is the user-facing editing behavior. It depends on the preview/runtime fixes in the base PR, but it is much easier to review once that plumbing is isolated.

## Verification

- `bun run --filter @hyperframes/studio test`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/player/components/EditModal.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/TimelineClip.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/timelineTheme.ts packages/studio/src/player/components/timelineTheme.test.ts packages/studio/src/utils/sourcePatcher.ts packages/studio/src/utils/sourcePatcher.test.ts`
- `bunx oxfmt --check packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/player/components/EditModal.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/TimelineClip.tsx packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/timelineTheme.ts packages/studio/src/player/components/timelineTheme.test.ts packages/studio/src/utils/sourcePatcher.ts packages/studio/src/utils/sourcePatcher.test.ts`

## Browser Proof

- verified timeline drag / resize / trim flows in Studio with `agent-browser`
- verified preview hot-refresh behavior without iframe remount flashes

## Stack

- depends on #389
- followed by `fix: smooth scrubber end seeking`

[result.mp4 <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.com/user-attachments/thumbnails/ca71c177-5042-468d-906f-b353938f40f8.mp4" />](https://app.graphite.com/user-attachments/video/ca71c177-5042-468d-906f-b353938f40f8.mp4)
 2026-04-22 01:48:14 +02:00
Miguel Ángel 158204343d fix: stabilize studio preview and runtime sync (#389) 
## Summary
Stabilize the Studio preview/runtime path so timeline data, preview rendering, and thumbnails stay in sync.

This PR includes:
- preview hot-refresh without remounting the iframe
- runtime duration/timeline fixes so Studio stops drifting from playback state
- thumbnail and selector-based preview fixes
- local Studio runtime serving and player-resolution fixes so dev/CI do not depend on prebuilt player artifacts
- tests around preview identity and thumbnail/runtime behavior

## Why This PR Exists
This is the foundation layer for timeline editing. Without it, the editor was prone to:
- iframe remount flashes after saves
- duration mismatches between preview and timeline
- stale or incorrect thumbnails
- CI/test failures when `@hyperframes/player` artifacts were not prebuilt

## Verification
- `bun run --filter @hyperframes/studio test`
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/core typecheck`
- `bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts`
- `bunx oxfmt --check packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts`

## Stack
- base of stack
- followed by `feat: add studio timeline editing`
- followed by `fix: smooth scrubber end seeking`
 2026-04-22 01:42:48 +02:00
James Russo 4ce1792601 feat(codex-plugin): add OpenAI Codex plugin at repo root (#387) 2026-04-21 15:33:32 -07:00
James bfce71f203 chore: release v0.4.12 v0.4.12 2026-04-21 19:17:41 +00:00
James RussoandClaude Opus 4.7 ffc06827c4 fix(engine): auto-normalize VFR video inputs to CFR before frame extraction (#360) 
* fix(engine): auto-normalize VFR video inputs to CFR before frame extraction

Screen recordings (macOS ScreenCaptureKit, QuickTime, phone videos) are
commonly variable-frame-rate. When such inputs hit the extractor's
`-ss <start> -i <video> -t <dur> -vf fps=N` pipeline, the fps filter
can emit fewer frames than requested — for a 4-second 30fps segment
starting mid-file, the output was ~90 frames instead of 120.

`FrameLookupTable.getFrameAtTime` returns null for out-of-range indices,
so the compositor held the last valid frame and the user perceived the
video as freezing. This matches the bug report from an X community post
where a user said "all of them freezes" on their screen recording scenes.

The engine already detects VFR via `metadata.isVFR` in ffprobe.ts but
never acted on it — the compiler only logged a warning. This change
mirrors the existing SDR→HDR normalization pattern: when a source is
detected as VFR, re-encode only the used segment with
`-fps_mode cfr -r <fps> -preset fast -crf 18` before extraction.

Scoping the re-encode to `[mediaStart, mediaStart+duration]` means a
30-second clip cut from a 60-minute screen recording pays ~1s of
transcode cost, not 18s. Benchmarked locally:

  Baseline (current):         32-39% duplicate frames, 25% frame-count
                              shortfall on mid-file segments.
  Tier 1 (flag changes only): ~same — fps filter issue is not flag-fixable.
  Tier 2 (CFR preflight):     1.7-6% duplicate frames, correct frame
                              count in every scenario tested.

The compiler warning that previously told users to manually re-encode
is downgraded to `console.info` since the engine now handles it.

— Rames Jusso

* refactor(engine): clean up VFR normalization loop after review

- Drop the `vfrNormDirCreated` flag; `mkdirSync({recursive:true})` is
  idempotent and cheap.
- Don't re-wrap the `VFR→CFR conversion failed` prefix — `convertVfrToCfr`
  already throws a message with that label; adding it again in the catch
  produced "VFR→CFR conversion failed: VFR→CFR conversion failed (exit 1)".
- Shorten the Phase 2b header comment; the function docstring above
  `convertVfrToCfr` already explains the failure modes and rationale.
- Note which frame windows the VFR fixture's select filter drops so the
  magic numbers are scannable.

No behavior change; 311/311 engine tests still pass.

— Rames Jusso

* test(engine): add VFR regression unit tests

Adds a describe block that synthesizes a VFR fixture via ffmpeg and asserts
the extractor produces the expected frame count (no shortfall) and no long
runs of duplicate frames — the user-visible "frozen screen recording"
symptom. Covers both a mid-file segment and the full-file case.

Guarded with describe.skipIf(!HAS_FFMPEG) because the CI Test job on
ubuntu-24.04 and the Windows test-windows job don't install ffmpeg. The
producer-level regression test in packages/producer/tests/vfr-screen-recording/
runs inside Dockerfile.test (which has ffmpeg) and is the primary CI signal
for this bug; these unit tests are supplementary coverage for local and
any ffmpeg-equipped CI environment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): add vfr-screen-recording regression test

End-to-end CI regression coverage for PR #360 via the existing
regression-harness: renders a 3s composition containing a real macOS
ScreenCaptureKit clip (r_frame_rate=120, avg≈36fps) seeked to
mediaStart=1, then PSNR-compares against a committed output.mp4.

Fixture src/clip.mp4 (108 KB) is a 5-second excerpt downscaled to 480×332
with -fps_mode passthrough to preserve the VFR timestamps. Content is the
public hyperframes OSS repo root page — see NOTICE.md for provenance.

With the fix applied, all 100 PSNR checkpoints pass. With the fix reverted,
66 of 100 fail (PSNR drops from ~43 dB to ~20 dB in the duplicate-frame
windows). Tagged "regression,video,vfr" so it runs in the fast shard
of .github/workflows/regression.yml automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): regenerate vfr-screen-recording baseline in Docker

The committed golden output.mp4 was initially rendered on the host machine;
CI runs the renderer inside Dockerfile.test with a different Chrome +
ffmpeg build, producing pixel-level drift that failed PSNR at 54/100
checkpoints (~20 dB vs 41 dB in the VFR sparse-content windows). Both
renders are valid — the VFR source has inherent sampling ambiguity in
static segments, and different Chrome/ffmpeg builds make different valid
choices.

Regenerated the baseline via `bun run docker:test:update vfr-screen-recording`
so it matches the Docker environment CI actually uses. Matches the flow
the existing sub-composition-video, hdr-pq, etc. baselines were captured
with.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: document that producer test baselines must be captured in Docker

Hit this 2026-04-21 with the vfr-screen-recording regression test:
host-generated output.mp4 baseline tripped 54/100 PSNR checkpoints in CI
because Chrome + ffmpeg drift between the host and Dockerfile.test.

Document the `bun run --cwd packages/producer docker:test:update <name>`
flow so future contributors don't repeat the mistake.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 12:14:23 -07:00
Miguel Ángel b98093aa1c fix: remove hidden audio gain in renders (#362) 
## Summary

This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.

It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.

Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.

## Root Cause

The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:

- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
  - `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`

That exactly matches the issue reporter's measured scalar boost.

While investigating the workaround, I also found a second correctness bug:

- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output

Separately, the repo-level Studio typecheck could fail before any build step because:

- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck

## What Changed

1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build

## Why These Changes Are Needed

This is not a UX preference issue; it is a correctness and API contract issue.

- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.

Leaving the current behavior in place means:

- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed

## Testing

### Focused regression tests

Ran:

- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`

Result:

- `10 passed`

These tests specifically verify:

- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)

### Broader package verification

Ran:

- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`

Results:

- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker

## Known Verification Limitation

There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.

## User Impact

After this change:

- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts

Closes #361.
 2026-04-21 20:33:25 +02:00
James RussoandClaude Opus 4.7 e72bcfaed3 fix(player+core): correctly render and pause nested compositions (#359) 
* fix(player): inject runtime immediately for nested compositions

Compositions that use `data-composition-src` on child elements require
the HyperFrames runtime to load those scenes — there is no way for the
iframe to render without it. The existing probe loop delayed runtime
injection behind a 5-tick attempts gate so the adapter path could try
to resolve a timeline first.

For nested compositions that race lost: a composition like the
`product-promo` registry example registers an inline pre-runtime GSAP
timeline at `window.__timelines["main"]` (covering only a partial
duration, e.g. 14s of a 20s master) while the iframe document loads.
The probe's adapter check finds that timeline and locks the player into
a "ready" state against it — which short-circuits the attempts gate and
the runtime never gets injected. The iframe ends up blank because the
runtime is what would have loaded the child scenes via
`data-composition-src`.

This change splits the injection decision into a pure helper,
`shouldInjectRuntime(state)`, and treats nested compositions as
"inject immediately, skip the gate." Self-contained GSAP-only
compositions retain the 5-tick grace period so the adapter path keeps
first shot for them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(core): propagate play/pause to all sibling timelines

Pausing or playing the master timeline only called `.pause()` / `.play()`
on `state.capturedTimeline` — the single adapter-selected timeline. In a
nested composition (a master with `data-composition-src` children), each
scene's own timeline is registered as a sibling in `window.__timelines`,
so they would keep advancing after the user clicked pause. The player UI
froze at the paused time while the visual content continued to animate,
eventually finishing all scene-level animations and landing on an empty
end-state.

Wire `window.__timelines` into the runtime player via a new
`getTimelineRegistry` dep, iterate the registry on play/pause, and
forward `timeScale` to siblings when play() starts so a changed
playback-rate applies uniformly.

Covered by 7 new unit tests in player.test.ts, including the identity-
equality check (don't double-invoke the master), playbackRate
propagation, a broken-sibling swallow, and a back-compat case with no
registry supplied.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 10:18:25 -07:00
Alex Coulombe 731baebbaa docs(website-to-hyperframes): add load-bearing GSAP authoring rules 
Add five animation-authoring rules to step-6-build.md that the linter
cannot catch but that silently ship broken output. Surfaced from two
independent builds (2026-04-20) where compositions passed lint and
still rendered elements invisible / unscrubbing.

Rules added:
- No iframes for captured content (don't scrub)
- Never stack transform tweens on one element (entrance + Ken Burns
  on same img silently kills it)
- Prefer tl.fromTo() over tl.from() inside .clip scenes
  (immediateRender interacts badly with scene boundaries)
- Ambient pulses must attach to seekable tl, not standalone gsap.to()
- Generalize the caption hard-kill rule to every scene-boundary exit
 2026-04-21 13:09:54 -04:00
James Russo 733d454d11 docs: add Hyperframes vs Remotion comparison (#355) 
Adds honest Hyperframes vs Remotion comparison: README section with
paragraph + table + open-source-vs-source-available callout, plus a
full guide at docs/guides/hyperframes-vs-remotion.mdx walking through
the core React-vs-HTML decision, practical differences (including a
GSAP side-by-side), and licensing.

Closes #318
 2026-04-20 15:21:03 -07:00
Miguel Ángel 5db554c7b6 feat(cli): silent auto-update on next run (#306) 
## Summary

Today users have to run `hyperframes upgrade` (or the right install command for their package manager) to get a new release — we ship fixes but they don't reach the install until the user remembers. This PR borrows the Claude Code model: detect the update on run N, install it in a detached background child, surface one line ("hyperframes auto-updated to vX.Y.Z") on run N+1. The user's current command never blocks, never prompts, never sees an install stream.

## Flow across two runs

```
Run N     → checkForUpdate() sees latest > current → spawn detached
            child running `npm install -g hyperframes@X` (or bun /
            pnpm / brew equivalent). Parent exits immediately.
(between) → detached child installs, writes completedUpdate into
            ~/.hyperframes/config.json, clears pendingUpdate.
Run N+1   → reportCompletedUpdate() prints one line and clears the
            marker. User is on the new version.
```

## Installer detection

Walks `realpathSync(process.argv[1])` against each package manager's well-known global prefix. Wrong guesses are biased toward `skip` — we'd rather miss an auto-update than clobber a Homebrew install with npm.

| Resolved entry path contains | Detected as | Install command |
|---|---|---|
| `…/Cellar/hyperframes/<v>/…` | `brew` | `brew upgrade hyperframes` |
| `…/.bun/…` | `bun` | `bun add -g hyperframes@<v>` |
| `…/pnpm/global/…` or `…/.pnpm/…` | `pnpm` | `pnpm add -g hyperframes@<v>` |
| `…/lib/node_modules/hyperframes/…` | `npm` | `npm install -g hyperframes@<v>` |
| `…/packages/cli/…` (workspace link) | `skip` | (no-op) |
| `…/_npx/…`, `…/bunx-…/…` | `skip` | (no-op) |
| Anything else | `skip` | (no-op) |

## Guardrails

- **Never auto-update across a major version.** The existing banner still nudges the user to run `hyperframes upgrade` explicitly.
- **Skip on CI, non-TTY, dev mode,** npx / bunx / workspace link, or any install layout the detector doesn't recognize.
- **`HYPERFRAMES_NO_AUTO_INSTALL=1`** disables the install without silencing the notice banner.
- **`HYPERFRAMES_NO_UPDATE_CHECK=1`** silences both (existing knob).
- **Fresh pending install (<10 min old)** prevents re-launch on every invocation.
- **Installer stdout + stderr go to `~/.hyperframes/auto-update.log`** for postmortem — the terminal stays clean.
- **Failed installs are surfaced once** with a prompt to run `hyperframes upgrade` manually.

## What changed

| File | Role |
|---|---|
| `packages/cli/src/utils/installerDetection.ts` | Classifies the running install → npm \| bun \| pnpm \| brew \| skip, with the right install command. |
| `packages/cli/src/utils/autoUpdate.ts` | `scheduleBackgroundInstall` + `reportCompletedUpdate`. Spawns a detached `node -e "..."` child that runs the install and writes the outcome back to the config, then `unref()`s so the parent exits immediately. |
| `packages/cli/src/telemetry/config.ts` | `pendingUpdate` + `completedUpdate` fields on the config schema. |
| `packages/cli/src/cli.ts` | Wires `reportCompletedUpdate()` at startup and `scheduleBackgroundInstall()` after `checkForUpdate()` resolves. |

## Verification

### Unit tests — 19 / 19 pass (full CLI suite 115 / 115)

- `installerDetection.test.ts` — 9 cases, one per layout (workspace, npx, bunx, brew, bun, pnpm, npm, unknown, unresolved).
- `autoUpdate.test.ts` — 10 scheduling-policy cases:
  - Minor/patch → schedules + writes pendingUpdate
  - Major bump → **does not** schedule
  - Dev mode → skipped
  - `CI=1` → skipped
  - `HYPERFRAMES_NO_AUTO_INSTALL=1` → skipped
  - Unknown installer → skipped
  - Already-on-latest → skipped
  - Fresh pending install → de-duplicated
  - Stale pending install (>10 min) → supersedes
  - Previous run already completed this version → skipped

Unit tests mock `spawn` and the installer — they verify the **policy**, not the real detached-child path.

### Live end-to-end smoke test (on this Mac, real processes)

To validate the parts the unit tests can't — actual detached spawn, real config writeback, banner surfacing in a fresh subsequent process — I wired a smoke script that exercises the exact same code path `autoUpdate.ts` uses, but with `echo …` as the "install command" so nothing global gets touched.

**Steps exercised:**

1. Backed up the user's real `~/.hyperframes/config.json`.
2. Wrote a `pendingUpdate` marker for version `0.4.99` (like `scheduleBackgroundInstall` does).
3. Spawned the **exact same detached `node -e "..."` child** the real scheduler produces, with the install command replaced by `echo 'faux install for 0.4.99'`.
4. The parent `unref()`d and continued; 800 ms later the parent re-read `config.json`.
5. Ran `reportCompletedUpdate()` in a **fresh subprocess** (via `bunx tsx -e ...`) to match the real "Run N+1" conditions, capturing its stderr.
6. Asserted the marker was cleared.
7. Restored the original config on exit.

**Observed output:**

```
  [setup] Backed up config to /Users/miguel/.hyperframes/config.json.smoke-backup
  [setup] Wrote pendingUpdate for v0.4.99
  [spawn] Detached child pid=49469
  [after] completedUpdate = {"version":"0.4.99","ok":true,"finishedAt":"2026-04-17T17:26:50.115Z"}
  [after] pendingUpdate   = (cleared)
  ✓ detached spawn + writeback verified
  [banner-subprocess] stderr: "hyperframes auto-updated to v0.4.99"
  ✓ banner fired in fresh process + marker cleared

  ALL CHECKS PASSED ✓
  [cleanup] Config restored
```

**What this proves:**

| Claim | Evidence |
|---|---|
| Detached spawn works (doesn't block the parent) | `[spawn] pid=49469` logged, parent continued immediately |
| Detached child is process-independent | Parent exited its own work while child ran `exec(CMD)` |
| Child writes correct config shape | `completedUpdate = { version: "0.4.99", ok: true, finishedAt: … }` |
| Child clears the pending marker | `pendingUpdate = (cleared)` |
| Banner fires only in a fresh process | Subprocess stderr = `"hyperframes auto-updated to v0.4.99"` |
| Banner message format | Matches the copy in `autoUpdate.ts:reportCompletedUpdate` exactly |
| Marker clears after banner | Second file read shows `completedUpdate` absent |

Both the original test-plan checkboxes (fresh install, `HYPERFRAMES_NO_AUTO_INSTALL=1`, `CI=1`) are covered by either the unit-test suite or this smoke test — the scheduling-policy gates are unit-tested under `CI=true`, and the real detached-spawn path is smoke-tested above.

### What's still worth doing

- **Physical installer test on a real `npm i -g` / `brew` / `bun add -g` environment** — the smoke test above replaces the install command with `echo`, so we've never actually seen npm/bun/brew run the real command. That's the one remaining unknown. Worth one manual run on the maintainer's machine before cutting v0.4.4.

## Test plan

- [x] `bunx vitest run` on `packages/cli` — 115 / 115 pass (incl. 19 new)
- [x] `tsc --noEmit` clean
- [x] `tsup` build clean
- [x] **Live e2e smoke test** exercising the real detached spawn + config writeback + fresh-process banner (output above)
- [x] CI green on this branch (Typecheck, Test, Test: runtime contract, Build, Lint, Format)
- [ ] One manual run on a physical `npm i -g hyperframes@0.4.2` install to confirm the real `npm install -g hyperframes@0.4.3` command actually runs when `autoUpdate.ts` delegates to it (the smoke test stopped short of executing `npm`)

## Notes

- Independent of any version bump — ship whenever.
- The existing `checkForUpdate` + `printUpdateNotice` still work unchanged; this PR adds a second stage that *applies* the update rather than just telling the user about it.
- `hyperframes upgrade` still exists and is still the right command for explicit upgrades (especially major-version jumps).
 2026-04-21 00:05:04 +02:00
Vance Ingalls c4bcc52f3b ci: add workflow_dispatch trigger to publish workflow (#354) 2026-04-20 13:37:34 -07:00
Vance Ingalls acce9123b4 chore: release v0.4.11 v0.4.11 2026-04-20 12:40:27 -07:00
Vance Ingalls 0d551e3614 chore: release v0.4.11-alpha.1 v0.4.11-alpha.1 2026-04-20 12:38:05 -07:00
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346) 
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
 2026-04-20 12:16:24 -07:00
Miguel Ángel a539266683 fix: harden CDN script inlining with linkedom (#352) 
* fix: harden CDN script inlining

* test: add spanish empire regression fixture
 2026-04-20 21:13:59 +02:00
James Russo 4a55bc8673 feat(cli): add --lang and auto-infer phonemizer locale from voice prefix (#351) 
* feat(cli): add --lang and auto-infer phonemizer locale from voice prefix

`hyperframes tts` was calling Kokoro's `model.create(text, voice=, speed=)`
with no language argument, so Kokoro's default phonemizer (en-us) was
applied regardless of the voice selected. Picking `ef_dora` or `jf_alpha`
and feeding it Spanish or Japanese text produced English-phonemized
output.

Closes #349.

- `manager.ts`: add `SUPPORTED_LANGS`, `inferLangFromVoiceId`, and
  `isSupportedLang`. Attach a `defaultLang` field to every bundled voice
  and expand the bundled list with `ef_dora`, `ff_siwis`, `jf_alpha`,
  `zf_xiaobei` so `--list` surfaces multilingual options.
- `synthesize.ts`: accept optional `lang: SupportedLang` in
  `SynthesizeOptions`, forward it to the Python worker as `argv[7]`.
  The worker introspects `Kokoro.create`'s signature and only passes
  `lang=` when the installed kokoro-onnx version supports it. Returned
  metadata now includes `lang` and `langApplied` so callers can detect
  silent no-ops. Bump the cached script filename to `synth-v2.py` so
  existing installs pick up the new script automatically.
- `commands/tts.ts`: add `--lang, -l` with validation against
  `SUPPORTED_LANGS`. Resolution order is explicit `--lang` > inferred
  from voice prefix > `en-us`. When explicit lang disagrees with the
  voice-implied lang (legitimate for stylized accents), emit a
  dim-level hint; suppress under `--json`. When kokoro-onnx silently
  ignores the kwarg, log that too. Update `--list` with a new
  "Lang code" column and add multilingual examples.
- Tests: new `manager.test.ts` covering every supported prefix, the
  unknown-prefix fallback, case-insensitivity, `isSupportedLang`
  validation, and a regression guard that every bundled voice has a
  valid `defaultLang` matching its ID.
- Docs: `docs/packages/cli.mdx` and `skills/hyperframes/references/tts.md`
  updated with the flag, examples, the espeak-ng dependency note for
  non-English phonemization, and the voice-prefix → lang table.

Backward compatibility:
- English voices (a*/b* prefixes) continue to phonemize as en-us / en-gb
  — no change.
- Non-English voices now phonemize correctly by default (bug fix, not a
  regression).
- Older kokoro-onnx versions that don't know the `lang` kwarg keep
  working via signature introspection; the CLI logs a dim note if
  `--lang` was requested but ignored.

Verification:
- `bun --cwd packages/cli test` — 128 tests pass (incl. 17 new).
- `bunx oxlint` and `bunx oxfmt --check` clean on changed files.
- `bun run build` succeeds.
- `npx tsx packages/cli/src/cli.ts tts --help` / `--list` render cleanly;
  invalid `--lang` produces a clean error with the valid-codes list.

* refactor(cli): simplify tts --lang implementation

Post-review cleanup on #351. Net -21 lines.

- Drop `defaultLang` field + `makeVoice()` helper from VoiceInfo —
  compute via `inferLangFromVoiceId(v.id)` at read time in listVoices.
  The only reader was the --list table; caching the derived value on
  every voice added a self-consistency invariant we had to test.
- Drop redundant `lang` field from SynthesizeResult — caller already
  knows the requested lang since it passed it in; only `langApplied`
  carries information the caller can't derive.
- Use `errorBox` for --lang validation to match the house style in
  render.ts (other validation errors already use errorBox).
- Reuse existing `langList` module constant in the validation error
  instead of re-joining SUPPORTED_LANGS.
- Inline `DEFAULT_LANG` — used once in inferLangFromVoiceId.
- Trim WHAT-restating comments and the duplicate prefix-enumeration
  JSDoc on inferLangFromVoiceId (VOICE_PREFIX_LANG already carries
  per-row comments).
- Clean up orphaned `synth*.py` files in ~/.cache/hyperframes/tts
  when writing the current versioned script, so repeated upgrades
  don't leak files.
- Drop the `EN-US` case-sensitive-rejection test assertion — the CLI
  lowercases input before validation, so accepting mixed case is a
  feature, not a bug.

Tests: 16/16 in `manager.test.ts`, 127/127 full CLI suite pass.
Lint + format + typecheck clean.
 2026-04-20 10:51:00 -07:00
Miguel Ángel 577e822fa6 chore: release v0.4.10 v0.4.10 2026-04-20 11:06:52 -04:00
Ular Kimsanov b7a644e2c7 Merge pull request #348 from heygen-com/fix/snapshot-video-frame-injection 
fix(cli): inject real video frames in snapshot to match render
 2026-04-20 11:00:54 -04:00
ukimsanov 0730f88dc7 fix(cli): address snapshot PR review feedback 
Incorporates four review points on #348:

1. Fix typecheck error — cast the dynamic `@hyperframes/engine` import to a
   typed shape and drop the `as typeof x` cast on a `null`-unioned variable
   that TS rejected. CI `Typecheck` was failing on main because the cast
   conflicted with the declared union.

2. Clear stale injected <img> overlays — always call
   `syncVideoFrameVisibility(page, activeIds)` on every seek (including
   `active.length === 0`), so injected frames from a previous timestamp
   don't leak into later snapshots. The runtime's visibility toggles act
   on the native <video> but not on its injected sibling, which Copilot
   correctly flagged as a leakage source.

3. URL-decode the served video path before resolving to a filesystem path.
   Files whose names contain spaces or other URL-encoded characters were
   skipped because `new URL().pathname` preserves `%20` while the file
   server decodes inbound requests and the file on disk lives at the
   decoded name.

4. Mirror `packages/core/src/runtime/media.ts` media math so clips with a
   non-1 `defaultPlaybackRate` get the correct active window and relTime.
   Specifically: clamp `defaultPlaybackRate` to [0.1, 5], apply
   `(t - start) * rate + mediaStart`, and adjust the duration-fallback
   branch by `/ rate` when only source duration is known.

5. Kill FFmpeg on a 30s timeout so a pathological clip cannot wedge
   snapshot indefinitely. Matches the default in
   `@hyperframes/engine`'s `runFfmpeg`.

Re-verified against the same 4-timestamp A/B on launch-video-2:
MD5s match the ffmpeg-from-render ground truth
  12.5s → ef9684e36fea53a0db7adf7cfcaacad3  (Stripe)
  16.0s → 487494ca16344d55d7181408dc439a56  (Framer)
  20.5s → 8835c34ad2a45755a1c98a7e079427a1  (HeyGen 3D)
  32.5s → 34dc9450f2bd661c12039d7aa82a30b0  (GitHub finale)

No-video projects (basecamp-tour, linear-brand, commissioned/github)
still produce identical output to the pre-fix path. Latency unchanged.

Made-with: Cursor
 2026-04-20 10:30:29 -04:00
ukimsanov a3e7c6176f fix(cli): inject real video frames in snapshot to match render 
The snapshot command previously just called `tl.seek(t)` + `page.screenshot`
and trusted Chrome to advance `<video>`-element decoders. Chrome headless
silently ignores `video.currentTime = X` writes — the setter is accepted
but the decoder never moves. Result: every snapshot of a composition that
uses body-level `<video data-start>` elements renders the same frame
regardless of the requested timestamp (the z-topmost video's first-frame
paints through, because all clips share `position: absolute; inset: 0`
and visibility:hidden doesn't always prevent the GPU surface from
contributing to the composite).

The render pipeline has already solved this: for each body-level video it
extracts the needed frame via FFmpeg and overlays it as an <img> sibling
via `injectVideoFramesBatch` (packages/engine/src/services/screenshot
Service.ts). This commit ports that same primitive into `snapshot`:

1. Added `extractVideoFrameToBuffer(videoPath, t)` — one FFmpeg spawn per
   active video, `-ss` keyframe seek (~100-200 ms), writes a temp PNG.
2. After the existing seek + settle, enumerate `<video data-start>`
   elements that are active at the target time, resolve each one's
   `currentSrc` URL back to a filesystem path under `projectDir`, extract
   the frame, and call `injectVideoFramesBatch`.
3. Then screenshot — as before.

Non-breaking: when no body-level `<video data-start>` elements exist (every
other project in the repo — basecamp, linear, stripe, github component), the
new block short-circuits on `active.length === 0` and behaves identically
to the pre-fix path. Verified against three no-video projects: bit-identical
snapshot output, no latency regression.

Measured on macOS M2 (4 frames, cold):
  launch-video-2 (11 timed videos): 6.48s → 6.16s   (-5%)
  basecamp-tour  (no timed videos): 5.67s → 4.87s   (-14%)

Proof: Pre-fix MD5 at t=12.5, 16.0, 20.5, 32.5 — all 4 identical (wrong frame)
  Post-fix MD5 at same timestamps  — all 4 distinct, match ffmpeg-from-render
Made-with: Cursor
 2026-04-20 09:36:33 -04:00
Miguel Ángel 62ce3679e6 chore: release v0.4.9 v0.4.9 2026-04-19 23:03:07 -04:00
Ular KimsanovandClaude Opus 4.7 a78ec5ec76 fix(skill): all file paths use capture/ subfolder prefix (#345) 
* fix(skill): all file paths use capture/ subfolder prefix

Step-1 was updated to capture into <project-dir>/capture/ but all
file-read instructions across steps 1-6 still used bare paths
(extracted/tokens.json, assets/hero.png). Agents following the skill
looked for files in wrong locations, producing broken compositions
with 404'd images and fonts.

- step-1: all 12+ file-read paths prefixed with capture/
- step-2: tokens.json and asset-descriptions.md paths fixed
- step-3: visible-text.txt path fixed
- step-4: asset-descriptions.md, asset examples, directory tree expanded
  to show capture/ children (screenshots/, assets/, extracted/)
- step-5: minor cleanup
- step-6: asset-descriptions.md path fixed, composition examples already
  correct (../capture/assets/)
- techniques.md: Lottie, video, font paths all prefixed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(skill): address Copilot review — font comment path, add videos/ to tree

- techniques.md: comment says ../capture/assets/fonts/ (matches composition context)
- step-4 file tree: add videos/ under capture/assets/

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 05:01:25 +02:00
Miguel Ángel 479e6c3f81 chore: release v0.4.8 v0.4.8 2026-04-19 22:40:57 -04:00