mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
Miguel Ángel
e8c43f0889
## Problem Studio seek could still wake nested composition media even when the transport itself stayed paused. In the real repro from `apple-presentation`, scrubbing to `0:29` without pressing play lands on the `slide-translation` composition. That composition contains `Multilingual_Journey.mp4` inside the composition host. On the broken path: - the main Studio transport remained paused - the nested video advanced and stayed playing anyway - the user saw autoplay-like behavior even though the only action was a seek That was especially confusing because the seek was otherwise correct: the timeline moved to the right point, but the nested media stopped obeying the paused transport state. ## What this fixes ### Nested media now participates in runtime media sync - the runtime media cache no longer assumes only `video[data-start]` / `audio[data-start]` are relevant - nested media inside a composition host can now be included in the same timed-media sync pass even when the inner media element does not carry its own authored `data-start` ### Nested media timing is resolved in the host composition window - nested media start time is resolved against the enclosing composition host instead of falling back to scene-local `0` - nested media duration is clamped to the enclosing composition window so it stays aligned with the authored host clip timing ### Paused seeks land on the right frame and stay paused - after seeking into a nested composition, the inner media is now seeked to the correct frame relative to the host timeline - because it is now part of the managed media set, the runtime also keeps it paused when the transport is paused instead of letting it continue playing on its own ### Regression coverage - adds a runtime regression test that covers a nested composition video with no local `data-start` - the test verifies that `player.seek(29)` leaves the nested video paused while landing it at the expected `currentTime` ## Root cause The bug came from a mismatch between deterministic timeline seeking and media ownership. ### 1. The runtime only managed media with direct timing attrs `refreshRuntimeMediaCache()` only collected `video[data-start]` and `audio[data-start]`. That works for root-level timed media, but not for media embedded inside a composition host where timing is inherited from the host composition rather than duplicated onto the inner media node. ### 2. Nested composition seek could still advance inner media The runtime intentionally rearms sibling timelines during deterministic seek so nested timelines land on the right local offsets. That part is necessary and correct. But because the nested video was not part of the managed media cache, it could advance during that seek path without being brought back under the paused transport state afterward. ### 3. The runtime had no way to reconcile the two So the system had an inconsistent split: - timeline seek knew about the nested composition timeline - media sync did not know about the nested media inside it The fix closes that split by resolving nested media start/duration from the enclosing composition context and running it through the same sync logic as other managed media. ## Verification ### Local checks - `bun run --filter @hyperframes/core typecheck` - `bun run test -- src/runtime/init.test.ts src/runtime/media.test.ts src/runtime/player.test.ts` in `packages/core` - `bunx oxlint packages/core/src/runtime/media.ts packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bunx oxfmt --check packages/core/src/runtime/media.ts packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` ### Browser verification Verified against a repo-backed local Studio preview of `apple-presentation`: - opened `http://127.0.0.1:3014/#project/apple-presentation` - seeked to `0:29` without pressing play - confirmed the visible composition switched to `slide-translation` - confirmed `Multilingual_Journey.mp4` landed at a non-zero `currentTime` (`3.067` in the verified run) - confirmed the nested video stayed `paused` and its `currentTime` remained stable across a follow-up check instead of autoplaying ## Notes - the local browser proof artifacts under `qa-artifacts/autoplay-seek/` are verification-only and are not part of this PR - this PR is intentionally scoped to nested media ownership during paused seek; it does not broaden into unrelated runtime media refactors beyond bringing inherited nested media under the existing sync contract
Hyperframe Runtime Engine
This folder owns the runtime that powers preview and producer parity.
Current Direction
- Runtime source of truth is converging on
hyperframe.ts. - Build produces:
dist/hyperframe.runtime.iife.js(browser bootstrap)dist/hyperframe.runtime.mjs(tooling/tests)dist/hyperframe.manifest.json(version + sha256 + artifact map)
- FE owns iframe runtime injection.
- BE persists raw generated HTML without injecting runtime scripts.
- Producer validates pinned runtime checksum from manifest before render.
Runtime Contract (Stable Surface)
Globals:
window.__playerwindow.__playerReadywindow.__renderReadywindow.__timelineswindow.__clipManifest
postMessage:
- parent -> runtime control:
source: "hf-parent"type: "control"- actions:
play,pause,seek,set-muted,set-playback-rate,enable-pick-mode,disable-pick-mode
- runtime -> parent events:
source: "hf-preview"type: "state"andtype: "timeline"
Determinism baseline:
renderSeekis the producer-canonical seek path.- 30fps quantization and readiness gates are correctness requirements.
Build
bun run --filter @hyperframes/core build:hyperframes-runtime
Security Expectations
- Runtime bootstrap URL must be version-pinned and host-allowlisted.
- Iframe bridge payloads must be schema-validated.
- Unsafe URL schemes (
javascript:and unapproveddata:) are rejected. - Fail closed if runtime bootstrap/handshake is not healthy.
Product Editing Model
- Primary mode: prompt + element picking.
- Secondary mode: manual precision controls.
- Avoid timeline-first manual workflows as default product path.