ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-05 17:30:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
445 Commits 893 Branches 406 Tags
cc9403b6bd3de05f85ae3a43a0467a92fbb6d37c
Commit Graph
27 Commits
Author SHA1 Message Date
Miguel Ángel 95bf333895 fix: stabilize apple master timeline and playback (#419) 
## Summary
- preserve authored non-root composition timing before runtime sanitization so Studio can build the correct master timeline for chained subcompositions
- prefer the fresh runtime source in Studio dev so local preview does not serve a stale `/api/runtime.js`
- restrict preserved authored timing inference to the Studio timeline payload instead of the general runtime resolver

## What this fixes
This PR fixes the Apple presentation class of failures where the root `index.html` / `Master` view looked correct at first and then collapsed into an incorrect short timeline.

Before this change:
- the master transport could report a short duration like `0:12` instead of the real deck length (`2:21` in the Apple project)
- composition clips bunched near the start instead of laying out sequentially across the deck
- seeking into later parts of the deck would land in the wrong place or show the wrong active composition
- local Studio debugging could be misleading because dev sometimes served a stale runtime bundle

After this change:
- the master transport reflects the authored composition-chain duration
- master clips resolve linearly across the whole deck
- late seeks land on the correct slide window
- Studio dev uses the current runtime implementation, so local preview matches the branch you are testing

## Root cause
There were two related issues:

1. Studio/master timeline inference lost authored composition timing
- missing timing attrs were treated like `0` instead of `null`
- non-root composition `data-duration` / `data-end` were stripped before Studio timing resolution could use them
- root duration inference trusted an incomplete live timeline window instead of the authored composition chain

2. Preserved authored timing leaked into the general runtime resolver
- preserving authored timing was correct for Studio timeline payload generation
- but using those preserved attrs for normal runtime playback/render resolution caused visual regressions in producer CI
- the follow-up fix keeps authored timing available only for Studio payload collection while normal runtime playback continues to resolve from the real live timeline/media state

## Why the later regression fix was needed
The initial runtime change fixed the Apple master timeline, but it also widened timing inference in the core runtime too far. That caused Dockerized producer regressions because rendered visibility started respecting preserved authored timing where it should have relied on the live resolved runtime state.

The latest commit fixes that by splitting the behavior:
- Studio timeline payload: authored timing allowed
- general runtime resolver: authored timing ignored by default

That preserves the Apple master timeline fix without changing producer render semantics.

## Verification
### Local checks
- `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/startResolver.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/startResolver.test.ts packages/core/src/runtime/timeline.test.ts packages/studio/vite.config.ts packages/cli/src/server/studioServer.ts`
- `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/startResolver.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/startResolver.test.ts packages/core/src/runtime/timeline.test.ts packages/studio/vite.config.ts packages/cli/src/server/studioServer.ts`
- `bun run --filter @hyperframes/core typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/cli typecheck`
- `cd packages/core && bun run test src/runtime/startResolver.test.ts src/runtime/timeline.test.ts`
- `bun test packages/cli/src/server/studioServer.test.ts --timeout 20000`

### Browser proof
Tested in Studio with `agent-browser` against the Apple presentation project.
- root/master transport now shows `0:00 / 2:21`
- master clip manifest resolves sequentially (`slide-1 -> slide-2 -> slide-3 ...`)
- seeking to `120s` lands on a late slide instead of a collapsed early timeline state
- after refreshing onto the fresh runtime source, the visible later-slide media advanced correctly in local Studio playback

### CI-equivalent regression proof on devbox
The previously failing producer regressions were rerun on devbox using the same Dockerized path GitHub Actions uses:
- `docker build -f Dockerfile.test -t hyperframes-producer:test .`
- `docker run ... hyperframes-producer:test style-1-prod style-5-prod style-9-prod style-12-prod --sequential`

Those previously failing suites all passed after the runtime split fix:
- `style-1-prod`
- `style-5-prod`
- `style-9-prod`
- `style-12-prod`

## Notes
- the Apple project volume tweak stayed local-only for testing and is not part of this PR
- this PR fixes the master/root timeline bug and the runtime regression it introduced; it does not add general subtimeline authoring support
 2026-04-23 04:05:11 +02:00
Vance Ingalls f906797222 perf(player): coalesce _mirrorParentMediaTime writes (#396) 
## Summary

Coalesce writes to `el.currentTime` inside `_mirrorParentMediaTime` so a single jitter sample no longer triggers a parent-media seek. A drift correction now requires **two consecutive samples** above the threshold (~`MIRROR_DRIFT_THRESHOLD_SECONDS`) before the player writes back. One-shot alignment paths (`promoteToParentProxy`, `_onIframeMediaAdded`) opt out via `force: true` so initial alignment stays immediate.

## Why

Step `P1-4` of the player perf proposal. `_mirrorParentMediaTime` is called every animation frame on parent media proxies. Even without true drift, browser internals report tiny jitter on `currentTime` reads — typically below 30 ms but occasionally crossing the threshold for a frame. Writing to `currentTime` triggers a seek, which is expensive *and* invalidates pipeline buffers, which causes the next frame's reading to jitter further. The result was unnecessary seek thrash on otherwise-aligned media.

By requiring two consecutive over-threshold samples, transient jitter is filtered out while real drift (a sustained offset) still corrects within ~1 frame of latency. This eliminates the most common cause of dropped frames on the studio thumbnail grid.

## What changed

- Each `_parentMedia` entry gains a `driftSamples` counter that increments while the absolute drift is above `MIRROR_DRIFT_THRESHOLD_SECONDS` and resets to 0 on the first sample below.
- `_mirrorParentMediaTime(el, opts)` only writes back when `driftSamples >= 2`, except when `opts.force === true`.
- `promoteToParentProxy` and `_onIframeMediaAdded` pass `force: true` so the first alignment after registration is still immediate (these are user-visible state transitions, not steady-state telemetry).

## Test plan

- [x] 11 new unit/integration tests in `hyperframes-player.test.ts` covering:
  - Single-sample jitter does not trigger a write.
  - Two-sample sustained drift does trigger a write.
  - Trending drift correction (gradually increasing offset) is detected within 2 samples.
  - `force: true` override bypasses the sample requirement.
  - Out-of-range proxies (proxies whose source has been removed) do not panic.
  - Multiple proxies maintain independent counters — drift on one does not affect the other.
  - `_promoteToParentProxy` alignment is immediate.

## Stack

Step `P1-4` of the player perf proposal. Builds on `P1-1` (shared adopted stylesheets) and `P1-2` (scoped media observer). Together these three target the studio multi-player render path — `P0-1*` perf gate scenarios will pick up the wins automatically.
 2026-04-22 17:44:49 -07:00
Vance Ingalls f9863ab565 feat(core): add emitPerformanceMetric bridge for runtime telemetry (#393) 
## Summary

Extend the runtime analytics bridge with a numeric performance metric channel. Hosts subscribe via the existing postMessage transport (one bridge, two channels) and aggregate per-session p50 / p95 for scrub latency, sustained fps, dropped frames, decoder count, composition load time, and media sync drift before forwarding to their observability pipeline.

This is the foundation other perf tooling sits on — the player itself emits the events; player-side aggregation and flush land in a follow-up.

## Why

Step `X-1` of the player perf proposal. Today there is no way for an embedding host to learn that scrub latency spiked, that a composition took 3 s to load, or that the media-sync loop is running 200 ms behind real time. The only signals are anecdotal user reports.

A single shared bridge keeps the runtime → host surface area minimal: hosts that already wire up the analytics channel get perf for free, and hosts that don't aren't paying for it.

## What changed

- New `emitPerformanceMetric(name, value, tags?)` helper in `@hyperframes/core` that forwards a `{ type: "performance-metric", name, value, tags }` envelope through the existing analytics postMessage transport.
- Six initial metric names defined in the proposal:
  - `scrub_latency_ms` — wall-clock from `seek()` call to first paint at the new frame.
  - `playback_fps` — sustained rAF cadence during play.
  - `dropped_frames` — count of >25 ms gaps within a play window.
  - `decoder_count` — number of concurrently-decoding video elements.
  - `composition_load_ms` — navigation-start to player-ready.
  - `media_sync_drift_ms` — drift between expected and actual decoder time.
- Each emit also writes a `performance.mark()` with `{ value, tags }` on `detail`, so the same numbers surface in the DevTools Performance panel's User Timing track for local debugging without instrumenting the host.
- Zero PostHog (or any other analytics SDK) dependency in `core` — the host decides where to forward the events.

## Test plan

- [x] Unit tests cover the envelope shape, the `performance.mark` mirror, and the no-op path when no host has wired up the bridge.
- [x] Manual: verified marks appear in the User Timing track when scrubbing the studio preview.

## Stack

Step `X-1` of the player perf proposal. Foundation for the perf gate (P0-1a/b/c) — the perf scenarios in this stack instrument these same channels for CI measurement.
 2026-04-22 15:08:14 -07:00
Miguel Ángel d740f5ce42 fix: nested GSAP sub-composition lint and render handling (#405) 
## Summary

- allow nested sub-composition files to inherit GSAP from their host without tripping `missing_gsap_script`
- keep nested render seeks stable for sub-compositions without regressing producer baselines
- stop producer render-hint detection from treating the compiler's own nested mount retry wrapper as user-authored `requestAnimationFrame()` usage

## Root Cause

- the core linter treated template-based nested compositions like standalone root compositions, so it incorrectly required a local GSAP loader even when the host composition already provided GSAP
- producer `detectRenderModeHints()` runs before CDN scripts are inlined, so nested GSAP exports were never failing because of the GSAP payload itself
- the nested-only false positive came from the compiler-generated mount bootstrap that waits for the inlined sub-composition root with `requestAnimationFrame()` before running the hoisted inline script
- preview and export seek paths also needed to stay split so the nested timeline re-arm behavior that stabilizes scrubbing does not collapse render baselines

## What Changed

- lint: keep the nested GSAP false-positive fix and regression coverage for template sub-compositions
- runtime: keep the render-seek behavior that preserves nested child offsets during export without changing preview scrubbing behavior
- producer: mark compiler-owned mount bootstrap blocks and strip only those blocks before scanning inline scripts for raw `requestAnimationFrame()`
- producer tests now cover both cases: compiler-generated wrappers are ignored, but real user-authored nested `requestAnimationFrame()` still opts into screenshot mode

## Validation

- `bun test packages/core/src/lint/rules/gsap.test.ts`
- `bun test packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxfmt packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxlint packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bun run --filter @hyperframes/producer test --sequential chat style-11-prod`
  - `style-11-prod` passed locally
  - `chat` still shows local-only visual drift on this macOS/ARM workstation, but the render metadata now reports `renderModeHints.recommendScreenshot=false`, which is the concrete acceptance condition for `#402`
- Docker CI-image repro is blocked locally by OrbStack x86/arm64 loader mismatch, so final regression confirmation is deferred to GitHub Actions

Closes #392
Closes #402
 2026-04-22 16:10:38 +02:00
Miguel Ángel d38a4f19fd fix(core): restore nested seek scrubbing (#404) 2026-04-22 06:13:09 +02:00
Miguel Ángel 1aea1415c4 fix: smooth scrubber end seeking (#386) 
* fix: smooth scrubber end seeking

* fix: stop timeline auto-scroll in fit mode

* feat: use percentage-based timeline zoom

* fix: sync timeline playhead on zoom changes

* fix: reset timeline scroll when returning to fit

* fix: keep timeline controls pinned
 2026-04-22 01:51:41 +02:00
Miguel Ángel 158204343d fix: stabilize studio preview and runtime sync (#389) 
## Summary
Stabilize the Studio preview/runtime path so timeline data, preview rendering, and thumbnails stay in sync.

This PR includes:
- preview hot-refresh without remounting the iframe
- runtime duration/timeline fixes so Studio stops drifting from playback state
- thumbnail and selector-based preview fixes
- local Studio runtime serving and player-resolution fixes so dev/CI do not depend on prebuilt player artifacts
- tests around preview identity and thumbnail/runtime behavior

## Why This PR Exists
This is the foundation layer for timeline editing. Without it, the editor was prone to:
- iframe remount flashes after saves
- duration mismatches between preview and timeline
- stale or incorrect thumbnails
- CI/test failures when `@hyperframes/player` artifacts were not prebuilt

## Verification
- `bun run --filter @hyperframes/studio test`
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/core typecheck`
- `bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts`
- `bunx oxfmt --check packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts`

## Stack
- base of stack
- followed by `feat: add studio timeline editing`
- followed by `fix: smooth scrubber end seeking`
 2026-04-22 01:42:48 +02:00
James RussoandClaude Opus 4.7 e72bcfaed3 fix(player+core): correctly render and pause nested compositions (#359) 
* fix(player): inject runtime immediately for nested compositions

Compositions that use `data-composition-src` on child elements require
the HyperFrames runtime to load those scenes — there is no way for the
iframe to render without it. The existing probe loop delayed runtime
injection behind a 5-tick attempts gate so the adapter path could try
to resolve a timeline first.

For nested compositions that race lost: a composition like the
`product-promo` registry example registers an inline pre-runtime GSAP
timeline at `window.__timelines["main"]` (covering only a partial
duration, e.g. 14s of a 20s master) while the iframe document loads.
The probe's adapter check finds that timeline and locks the player into
a "ready" state against it — which short-circuits the attempts gate and
the runtime never gets injected. The iframe ends up blank because the
runtime is what would have loaded the child scenes via
`data-composition-src`.

This change splits the injection decision into a pure helper,
`shouldInjectRuntime(state)`, and treats nested compositions as
"inject immediately, skip the gate." Self-contained GSAP-only
compositions retain the 5-tick grace period so the adapter path keeps
first shot for them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(core): propagate play/pause to all sibling timelines

Pausing or playing the master timeline only called `.pause()` / `.play()`
on `state.capturedTimeline` — the single adapter-selected timeline. In a
nested composition (a master with `data-composition-src` children), each
scene's own timeline is registered as a sibling in `window.__timelines`,
so they would keep advancing after the user clicked pause. The player UI
froze at the paused time while the visual content continued to animate,
eventually finishing all scene-level animations and landing on an empty
end-state.

Wire `window.__timelines` into the runtime player via a new
`getTimelineRegistry` dep, iterate the registry on play/pause, and
forward `timeScale` to siblings when play() starts so a changed
playback-rate applies uniformly.

Covered by 7 new unit tests in player.test.ts, including the identity-
equality check (don't double-invoke the master), playbackRate
propagation, a broken-sibling swallow, and a back-compat case with no
registry supplied.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 10:18:25 -07:00
James RussoClaude Opus 4.7Codex
 		ad11de698c feat(producer): auto-fallback screenshot capture for raf and iframes (#331) 
* fix(core): drive adapter seeks when composition has no GSAP timeline

renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.

Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.

GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(producer): auto-fallback screenshot capture for raf and iframes

Co-Authored-By: Codex <codex@openai.com>

* test(producer): add render compatibility regression fixtures

Co-Authored-By: Codex <codex@openai.com>

* fix(core): scrub CSS animations via WAAPI currentTime

Co-Authored-By: Codex <codex@openai.com>

* test(producer): cover css keyframe renders

Co-Authored-By: Codex <codex@openai.com>

* fix(producer): propagate virtual time into iframe documents

Co-Authored-By: Codex <codex@openai.com>

* test(producer): refresh iframe docker golden

Co-Authored-By: Codex <codex@openai.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
 2026-04-18 15:08:52 -07:00
Miguel Ángel c49181f1fa fix(player): address #298 review — tighter drift, dynamic proxies, ownership event (#307) 
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment.

## Significant

### 1\. Drift threshold 150 ms → 50 ms

_mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity.

**Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction.

### 2\. Dynamic sub-composition media proxies

Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track.

Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect.

### 3\. `bridgeMuted` sticky in `syncRuntimeMedia`

The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message.

`syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs.

### 4\. Reset `_audioOwner` on iframe reload

The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently).

`_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`.

## Worth addressing

### 5\. Promotion → observable event + reason

Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state.

**Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once.

### 6\. Parent proxy play() rejection → `playbackerror` event

Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back.

### 7\. Mobile verification on real hardware

Tested with a tunnel in a real iOS device.

## Test gaps (from review)

- `userMuted` stickiness (mirror of the existing `outputMuted` test).
- **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`.
- **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic.
- **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once.
- **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote).
- **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately.
- **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag.

## Minor

- One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear).

## What's good (from the review)

Kept as-is — noted for posterity:

- `muted` vs `volume` framing (orthogonal channels).
- Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing.
- Two orthogonal mute channels.
- Backwards compat (new actions / messages safely ignored by either side).

## Test results

- `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe)
- `packages/core/src/runtime/bridge.test.ts` — **15 tests pass**
- `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface)
- Typecheck green on `core` + `player`
- `tsup` build green on `core` / `player` / `cli`
- Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold.

## Test plan

- [x] Unit tests (83 total across touched files)
- [x] Typecheck clean
- [x] Build clean
- [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened
- [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block
- [x] Physical iOS / Android device verification (unchanged since #298)
 2026-04-18 00:49:57 +02:00
Miguel Ángel 3256551a5e fix(player): single-owner audio to prevent double voice in preview (#298) 
## Summary

Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip).

## Root cause

Two audio pipelines were playing the same source in parallel:

1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path.
2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message.

The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks.

Confirmed via agent-browser instrumentation on `factory-series-c-video`:
- 6 `volumechange` events per play cycle (mute-fight signature)
- Both copies audible at `volume=1`, offset growing 23ms → 80ms
- Every seek widened the drift further

PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback.

## Fix

Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime.

- **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash.
- **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold.

Two orthogonal mute channels replace the volume fight:

| Channel | Purpose |
|---|---|
| `set-muted` | User's mute preference (existing, unchanged) |
| `set-media-output-muted` | Internal ownership handoff (new) |

`syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it.

## Why this shape

- **Single owner, explicit transition.** No races, no tug-of-war.
- **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing.
- **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact.
- **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift.
- **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior.
- **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files.

## Files changed

- `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types.
- `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields.
- `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`.
- `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`.
- `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session.
- `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership.

## Verified end-to-end with agent-browser on `factory-series-c-video`

**Runtime ownership (default — desktop studio):**

| | Before | After |
|---|---|---|
| `PARENT.play()` calls per play cycle | 1 | **0** |
| iframe `volumechange` events | 6 | **0** |
| Audible streams | 2 (drifting) | **1 (iframe)** |

**Parent ownership (simulated autoplay block — direct message):**

| | Value |
|---|---|
| iframe audio | `muted=true`, `volume=1` (untouched) |
| parent audio | `muted=false`, `volume=1`, audible |
| Parent ↔ iframe `currentTime` offset | ~6 ms steady state |
| Offset > 150 ms | corrected by mirror sync |

**Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:**

Event timeline captured via agent-browser instrumentation:

```
t=0.0 ms   IFRAME.play() called                       ← runtime attempts playback
t=0.4 ms   IFRAME.play() REJECTED: NotAllowedError    ← simulated mobile gate
t=0.4 ms   →IFRAME bridge set-media-output-muted=true ← player promotes
t=0.6 ms   PARENT.play() called                       ← parent proxy starts
t=0.8 ms   ←IFRAME msg media-autoplay-blocked         ← runtime signal
t=1.0 ms   PARENT.play() resolved                     ← audible
t=1.3 ms   IFRAME muted=true, volume=1                ← iframe silenced via native muted
```

Steady state at t=4 s under promoted parent ownership:

| Element | currentTime | paused | volume | muted |
|---|---|---|---|---|
| Parent audio | 4.060 s | false | 1.0 | **false** (audible) |
| Iframe audio | 4.068 s | false | 1.0 | **true** (silent) |

**Offset: 8 ms**, single audible stream, orthogonal mute channel respected.

## Test plan

- [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`)
- [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow)
- [x] `bun run build` — all packages green
- [x] Fresh preview + browser repro on `factory-series-c-video`:
  - [x] Runtime ownership: single audio stream, no drift
  - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible
  - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset
  - [x] No `volumechange` thrash in either ownership mode
- [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above)
 2026-04-17 04:46:22 +02:00
Miguel Ángel f1a37400b4 fix(runtime): silent-first-play + loading overlay for preview (#293) 
## Summary

Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit.

### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`)

`syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window):

- `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click."
- The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged.

`HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`.

### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`)

The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s.

### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`)

Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration.

Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR.

### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`)

Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync.

## Verification

- 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset.
- Full monorepo build green (`bun run build`), typecheck clean, lint/format clean.
- End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass:

| Scenario | Metric | Result |
| --- | --- | --- |
| Normal first play | Audio plays from click, smooth progression | ✅ |
| Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip | ✅ |
| Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) | ✅ |
| Scrub mid-playback | Lands exactly at target frame | ✅ |

## Files changed

- `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve.
- `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset.
- `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes.
- `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check.

## Follow-ups (deferred)

- Tight-threshold short-window drift correction for lip-synced dialogue.
- A perf-regression test that fails on `waiting`\-event resurgence.

## Test plan

- [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words.
- [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events).
- [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers.
- [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame.
 2026-04-16 21:51:06 +02:00
Miguel Ángel ceb54811c6 fix(runtime): preload media on init to prevent broken first-play audio (#234) 
## Summary
- Audio (and video) sounds broken/choppy on first play in the studio preview, but works fine on second play
- Root cause: `<audio>` elements default to `preload="metadata"`, which only fetches enough data to determine duration — not enough for smooth playback. When `el.play()` fires, the browser hasn't buffered the audio data yet
- The runtime now eagerly sets `preload="auto"` and calls `load()` during init, ensuring media is fully buffered before the user clicks play
- `syncRuntimeMedia` now defers `play()` on unbuffered media by registering a `canplay` listener, instead of silently swallowing the failure

## Testing
Verified with agent-browser against a 26s narration composition (soulscape-film):

```
# After fix — audio element state at init:
preload: "auto"
readyState: 4 (HAVE_ENOUGH_DATA)
buffered: 26.07s (entire file)
duration: 26.07s
```

Audio is fully buffered before any play attempt, so first-play works identically to subsequent plays.

## Files changed
- `packages/core/src/runtime/init.ts` — set `preload="auto"` + `load()` in `bindMediaMetadataListeners`
- `packages/core/src/runtime/media.ts` — defer `play()` on unbuffered media via `canplay` listener
- `packages/core/src/runtime/media.test.ts` — updated test + added unbuffered media test case
 2026-04-10 00:05:32 +02:00
Miguel Ángel f56b4c8620 fix(core): load head styles/scripts from non-template sub-compositions (#219) 
## Summary

Fixes a bug where non-template sub-compositions (full HTML documents loaded via `data-composition-src`) lost all `<head>` styles and scripts. This affected **three code paths**:

1. **Runtime** (`compositionLoader.ts`) — browser preview via iframe fetch
2. **Bundler** (`htmlBundler.ts`) — studio preview HTML bundling (**this was causing the black preview**)
3. Producer fix is in PR #220

## What it fixes

**Eval prompt #25** (iris-wipe) renders entirely black in both the studio preview and rendered video because scene backgrounds (`#EF4444` red, `#3B82F6` blue), positioning, and the GSAP CDN script were all in `<head>` and silently dropped.

### Verified

Rebuilt core, started studio preview, fetched the bundled HTML from `/api/projects/iris-wipe/preview` — confirmed `#scene1 { background: #EF4444 }` and `.scene { position: absolute }` are now present in the output.

## Root cause

All three code paths did the same thing:
```js
const contentHtml = template ? template.innerHTML : bodyEl.innerHTML;
// ^ <head> content is already lost here
```

## Test plan

- [x] All 429 core tests pass
- [x] Studio preview endpoint returns correct bundled HTML with head styles included
- [x] `pnpm --filter @hyperframes/core build` succeeds

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-07 17:17:25 +02:00
Vance Ingalls cb0b17062a feat(skills): add marker-highlight skill for animated text highlighting (#190) 
## Summary

- **New skill:** **`marker-highlight`** — integrates [MarkerHighlight.js](https://github.com/Robincodes-Sandbox/marker-highlight) into HyperFrames compositions. Canvas-based animated text highlighting with 5 drawing modes: marker pen, circle, burst, scribble, and sketchout.
- **Studio fix:** added missing `captionSync` to useEffect dependency array (oxlint exhaustive-deps)
- **Studio fix:** `loadOverrides` now checks `res.ok` before parsing, preventing 404 console noise on projects without captions

## Skill details

The skill documents the non-obvious GSAP integration pattern discovered during development:

1. **One highlighter per container** — the library clears ALL `.highlight` divs from the shared parent on init, so multiple instances on sibling marks conflict
2. **`data-color`** **\+** **`data-original-bgcolor`** — prevents the CSS background-color flash that occurs when the library reads and clears the mark's background
3. **Canvas pre-draw + clear + reanimate** — `animate: false` pre-draws statically, canvases are hidden, then cleared and shown with `reanimateMark()` at trigger time for clean animated reveals
4. **`onReverseComplete`** **for rewind** — hides highlight divs when the timeline seeks backward past the trigger point

## Test plan

- [ ] `npx hyperframes lint` passes on test-composition
- [ ] Studio preview shows marker highlight on "something" at 1s, circle on "love" at 2.2s
- [ ] Rewind past trigger points hides highlights
- [ ] No 404 console errors for caption-overrides.json on non-caption projects



[Screen Recording 2026-04-02 at 1.56.30 AM.mov <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.com/user-attachments/thumbnails/53b03f4e-538e-477a-b738-7a033b99a84e.mov" />](https://app.graphite.com/user-attachments/video/53b03f4e-538e-477a-b738-7a033b99a84e.mov)



🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-02 13:48:24 -07:00
Vance IngallsandClaude Opus 4.6 5e2781b459 fix(studio): address caption designer PR feedback (#200) 
* fix(studio): address caption designer PR feedback

Fixes from review comments on feature/caption-designer (#180):

- fix(generator): guard named colors in hexToRgba — "red", "transparent"
  no longer produce NaN rgba values
- fix(sync): log auto-save failures instead of silently swallowing them
- fix(sync): check res.ok before parsing caption-overrides response
- refactor(components): extract Section, Row, inputCls into shared.tsx
  to eliminate duplication between CaptionPropertyPanel and
  CaptionAnimationPanel
- fix(store): replace non-deterministic Date.now()+Math.random() ID with
  counter-based group IDs
- fix(store): read selectedGroupId from state param instead of get() to
  avoid stale reads in batched set() calls
- fix(overlay): remove cssScale multiplier from getBoundingClientRect
  coords — the browser already accounts for CSS transforms
- docs(parser): add comment explaining the lazy ];  regex assumption

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): address remaining caption designer feedback

Overlay: handle both per-word spans (generator output) and grouped text
nodes (existing templates). Wraps text nodes into individual spans on
demand so the overlay can target words in any caption format.

Property panel: add Typography (font, size, weight, spacing) and Color
(color, active, dim, opacity) sections alongside existing Position and
Transform controls.

Timeline: move caption timeline into a dedicated flex-shrink-0 section
below the main timeline tracks instead of inside the scrollable area.
Gives it fixed 60px height that's always visible.

Caption overrides: classify color tweens by comparing target color to
the dim baseline instead of relying on timeline position order. This
handles compositions with custom color tweens correctly.

App.tsx: remove polling interval, rely on runtime postMessage events
for caption detection. Add clarifying comment on why useEffect is
appropriate (external event subscription).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): restore cssScale in overlay coordinate conversion

getBoundingClientRect() on iframe-internal elements returns coordinates
in the iframe's native resolution (1920x1080), not the CSS-scaled
display size. The cssScale multiplier is needed to convert to parent
window coordinates. The earlier removal was incorrect — it only worked
at 1:1 scale.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): fix reversed scaling on left-side corner handles

Scale interaction used horizontal dx from start position, which goes
negative when dragging left handles outward. Now uses distance from box
center — dragging away from center increases scale regardless of which
corner handle is used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): make rotation respond to horizontal drag only

Rotation handle sits directly above the word, so atan2-based rotation
barely responds to left/right movement. Replace with linear horizontal
mapping: drag right = clockwise, drag left = counter-clockwise,
200px = 90 degrees. Vertical movement is ignored.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): remove animation tab and typography/color from property panel

Keep only Position (X, Y) and Transform (Scale, Rotation) controls.
Remove tab switcher UI since there's only one view now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix oxfmt formatting in CLAUDE.md and captions skill docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 11:38:35 -07:00
Vance Ingalls d36c1785b9 feat(captions): energy-based technique selection and mandatory quality checks (#176) 
## Summary

- Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits
- Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time, no per-frame callbacks needed
- Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model)
- Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility
- Add multilingual model guidance and decision tree for model selection

## Test plan

- [ ] Skill files render correctly as markdown
- [ ] Cross-references between SKILL.md, dynamic-techniques.md, and transcript-guide.md resolve correctly
- [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-02 00:47:43 -07:00
Vance IngallsJamesClaude Opus 4.6
 		ef6225da1d feat(core): add fitTextFontSize utility for pixel-accurate text measurement (#152) 
Add @chenglou/pretext dependency and fitTextFontSize() utility that uses
canvas measureText to compute the largest font size that fits text within
a given width. Replaces character-count heuristics with actual font-aware
measurement.

- New fitTextFontSize() in @hyperframes/core/text, exposed on window.__hyperframes
- Generalized for all text elements (captions, titles, etc.), not just captions
- Unit tests (mocked pretext) + browser integration test (real Chromium canvas)
- Updated captions skill docs with usage, exit guarantee, and self-lint patterns

Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 20:14:13 -07:00
Miguel Ángel 6d54217e74 feat(core): support inline template compositions (#146) 
## Summary

- Adds `loadInlineTemplateCompositions` to the runtime to handle compositions defined inline via `<template id="X-template">` paired with empty host elements that have `data-composition-id="X"` but no `data-composition-src`
- Updates the HTML bundler to inline template content into matching hosts during compilation
- Users can now define sub-compositions inline instead of requiring separate files in `compositions/`

### Before

```html
<!-- This showed nothing — the template was inert and the host was empty -->
<template id="logo-reveal-template">
  <div data-composition-id="logo-reveal" data-width="1920" data-height="1080">
    <style>...</style>
    <script>/* animation */</script>
  </div>
</template>

<div data-composition-id="logo-reveal" data-start="0" data-duration="10"
     data-width="1920" data-height="1080"></div>
```

### After

The runtime detects the matching `<template>` and injects its content into the host element — styles hoisted to `<head>`, scripts executed, dimensions copied. Works in both preview and render.

## Test plan

- [x] 9 new unit tests for `loadInlineTemplateCompositions` (basic mount, no-op cases, style/script injection, dimensions)
- [x] 3 new unit tests for bundler inline template handling
- [x] All 384 existing tests pass
 2026-03-31 03:46:23 +02:00
Miguel Ángel 1230657ed0 fix(studio,runtime,engine,compiler): 8 bug fixes — audio, render, timeline, Lottie, thumbnails, video render (#133) 
## Summary

**Original 5 bugs fixed:**

- **Bug 1 — Audio silent after seek**: Added `Accept-Ranges` / `Content-Length` + `206 Partial Content` to the static asset server for byte-range seeking.
- **Bug 2 — Download 404 after restart**: Render list endpoint now registers on-disk renders into the in-memory job map.
- **Bug 3 — Timeline stops at GSAP end**: `resolveRootTimelineFromDocument` pads the GSAP timeline to match `data-duration` when the composition declares longer.
- **Bug 4 — Render stuck at 0%**: Store `jobState` reference (not spread copy) so async progress mutations reach the SSE stream.
- **Bug 5 — Lottie missing in preview/render**: Two fixes — (a) moved Lottie adapter before GSAP so `onUpdate` wins; (b) fixed bundler silently dropping external CDN `\<script src>` tags from sub-compositions (root cause: `$content(s).html()` returns `""` for external scripts).

**3 additional bugs fixed:**

- **Bug 6 — Blank thumbnails outside monorepo**: Implemented `generateThumbnail` in the CLI adapter using Puppeteer.
- **Bug 7 — Video empty in rendered sub-compositions**: Fixed `parseVideoElements` selector from `video[id][src]` to `video[src][data-start]` + auto-assign IDs.
- **Render errors**: Failed renders now show their error message in the renders panel.

## Commits

| Commit | Description |
| --- | --- |
| `3951c6f` | fix(studio): store render job reference instead of snapshot copy |
| `f331c30` | fix(studio): make previously-completed renders downloadable after restart |
| `a5e2d04` | fix(studio): add range request support for audio/video seeking in preview |
| `f24317a` | fix(runtime): pad GSAP timeline to data-duration when composition declares longer duration |
| `7cf38ca` | fix(runtime): fix Lottie adapter conflicting with GSAP-driven animations |
| `bc99209` | fix(studio): surface render error messages in the renders panel |
| `8fc9e8b` | fix(cli): implement generateThumbnail in studio adapter |
| `90277ea` | fix(engine): render videos inside sub-compositions that lack an explicit id |
| `f5bb579` | fix(compiler): preserve external CDN scripts from sub-compositions in bundle |

## Test plan

- [x] `golden-lyric-video`: seek → audio plays from seeked position
- [x] Any project: render → progress advances past 0%, reaches 100%
- [x] Any project: complete render, restart `hyperframes dev`, Download → works
- [x] `intro-vid`: play → runs full 5s (not stopping at 3s)
- [x] `hyperframe-build-up-demo`: play → rocket Lottie visible during 0-2s ✅ verified
- [x] Outside monorepo: Compositions sidebar shows thumbnail images (not blank)
- [x] `bug.zip` project: render → video in polaroid sub-composition appears in output
- [x] Trigger a failed render → error message shown
 2026-03-30 23:58:08 +02:00
Miguel Ángel 1593055482 fix(core): filter decorative elements from timeline clip collection (#92) 
## What

Enhanced timeline collection to discover GSAP-animated scene elements and persistent overlays through runtime introspection.

## Why

The existing timeline collection only captured elements with explicit timing attributes (`data-start`, `data-track-index`) or media elements, missing scene elements that are animated purely through GSAP tweens and persistent overlay elements that should appear for the full composition duration.

## How

Added two new discovery mechanisms to `collectRuntimeTimelinePayload`:

1. **GSAP Timeline Introspection**: Walks the master timeline's tweens using `getChildren()` to find animated elements, calculates absolute time ranges by traversing parent timelines, and bubbles child tween ranges up to their nearest scene-level ancestors (direct children of root with IDs).
2. **Persistent Overlay Detection**: Identifies direct children of the root composition that weren't captured by DOM queries or GSAP introspection, treating them as full-duration overlay elements while filtering out non-visual elements (script, style, meta tags) and hidden elements.

Both mechanisms respect existing track assignments and create new tracks when needed to avoid conflicts.

## Test plan

- [x] Unit tests added/updated
- [x] Manual testing performed
- [ ] Documentation updated (if applicable)

Added comprehensive test coverage for:

- GSAP-animated scene element discovery via timeline introspection
- Time range bubbling from child elements to scene ancestors
- Persistent overlay inclusion as full-duration clips
- Proper filtering of non-visual elements (script/style tags)

![image.png](https://app.graphite.com/user-attachments/assets/106ea699-e243-493d-aa76-a74f303b80c3.png)

![image.png](https://app.graphite.com/user-attachments/assets/2d124d15-c933-4e12-891c-06100e6241c3.png)
 2026-03-27 18:31:46 +01:00
Miguel Ángel 46e9f8d248 feat(core): add loop and data-playback-rate for media elements 
- data-playback-rate: per-element slow-mo/fast-forward (0.1-5x range)
  Multiplied with global transport rate. Affects timeline duration
  calculation when source duration is used as fallback.

- loop: native HTML loop attribute now works correctly in the runtime.
  Wraps media playback from mediaStart when source reaches end.
  Enables looping short clips over longer durations.

Both follow the existing data-media-start/data-volume pattern.
 2026-03-24 17:27:41 -04:00
Vance IngallsandClaude Opus 4.6 61c5257402 fix(ci): update publish workflow to use bun install (#36) 
* fix(ci): update publish workflow to use bun install

pnpm-lock.yaml was removed in the bun migration but publish.yml
still referenced it. Use bun for install/build, keep pnpm for
publish (publishConfig overrides + --provenance).

* docs: update stale pnpm references to bun across docs and scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 08:46:58 -07:00
Miguel Ángel 062f6f1c10 fix: composition issues in runtime code + producer parity 2026-03-23 23:56:42 -04:00
Vance Ingalls 20be2ea1c2 style: apply oxfmt baseline formatting across all source files (#25) 
## Summary
- Run `oxfmt .` across the entire codebase to establish formatted baseline
- 299 files changed — mechanical formatting only, no logic changes
- Double quotes, semicolons, 2-space indent, trailing commas, 100 print width

Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm format:check` — all 426 files pass
- [x] `pnpm -r typecheck` — all packages pass
- [x] `pnpm build` — all packages build
- [x] All 348 tests pass
 2026-03-23 17:15:14 -07:00
Vance Ingalls 323ff8f860 fix: resolve oxlint errors across codebase (#24) 
## Summary
- Remove 5 unused `beforeEach` imports from test files
- Remove unused imports (`existsSync`, `TimelineCompositionElement`)
- Remove unused destructured variables (`options`, `width`, `height`, `goldenEl`)
- Remove dead `formatDuration` function
- Fix unused catch parameters (`catch (err)` → `catch`)
- Prefix unused `renderError` state with `_`
- Add `eslint-disable-next-line` for 2 React exhaustive-deps false positives (stable ref + zustand setter)

Part 2/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm lint` — 0 errors on 193 files
- [x] All 348 tests pass (core + engine)
 2026-03-23 16:41:41 -07:00
Vance IngallsandClaude Opus 4.6 9f8e5ba5a1 initial code (#2) 
* feat: initial code port from hyperframes-internal

Port all OSS-ready packages from the internal monorepo:
- @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime
- @hyperframes/cli — CLI for creating, previewing, and rendering compositions
- @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg)
- @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg)
- @hyperframes/ui-player — browser-based video player component
- @hyperframes/studio — composition editor (React frontend + Hono backend)

Includes regression test suite with Docker-based test harness.

All HeyGen-internal references, deployment infrastructure, and
proprietary assets have been removed. Package names migrated
from @app/* to @hyperframes/*.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrub internal codenames and stale references from OSS port

- Replace static.heygen.ai runtime URLs in test fixtures
- Remove internal CDN publish script (publish-hyperframe-runtime.ts)
- Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime
  with neutral names (studio, hyperframe-runtime, __hyperframeRuntime)
- Fix stale Vault API / localhost references in docs
- Remove broken deprecated_studio link

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove remaining internal codenames and stale references

- Delete stale producer README.md and PIPELINE.md (referenced nonexistent files)
- Replace "Cerberus" codename with "HyperFrames" in test design reviews
- Replace magic-edit postMessage identifiers with hf-preview/hf-parent
- Rename debug-magic-edit-timeline.ts to debug-timeline.ts
- Replace "Motion Cut" with "HyperFrames" in Timeline comments
- Fix studio/CLI references to nonexistent archive package
  (use local data/projects/ dir, stub render proxy)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 22:43:56 -07:00