mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 17:30:50 +00:00
Miguel Ángel
cd832f01ac
The runtime plays audio two ways — a Web Audio transport (sample-accurate) and the HTMLMediaElement as a fallback — and mutes the elements when Web Audio takes over so they don't double-play. That mute gate was global: it muted every element the moment ANY Web Audio source was active (webAudio.isActive()). A track Web Audio had not claimed yet (its larger buffer decodes slower) was muted on the fallback AND not playing on Web Audio = silent, while the other tracks played. With TTS narration + BGM + SFX, the narration (largest buffer) lost the decode race and dropped out intermittently, with every file fully loaded. Make the mute per-element: an element is muted only when its own Web Audio source is live, or the user / parent-proxy force-mute is set. A track Web Audio has not claimed stays audible on the HTMLMedia fallback until the transport takes it over — which also lets narration start immediately on cold play instead of waiting for its buffer to decode. Also in this change: - Don't permanently blacklist a transient fetch failure in the Web Audio decoder (_failedSrcs was never cleared); only blacklist genuinely undecodable bytes, so a late-arriving asset (404 then available) self-heals on the next play. - Stop re-issuing play() every tick on an errored / no-source element.
Hyperframe Runtime Engine
This folder owns the runtime that powers preview and producer parity.
Current Direction
- Runtime source of truth is converging on
hyperframe.ts. - Build produces:
dist/hyperframe.runtime.iife.js(browser bootstrap)dist/hyperframe.runtime.mjs(tooling/tests)dist/hyperframe.manifest.json(version + sha256 + artifact map)
- FE owns iframe runtime injection.
- BE persists raw generated HTML without injecting runtime scripts.
- Producer validates pinned runtime checksum from manifest before render.
Runtime Contract (Stable Surface)
Globals:
window.__playerwindow.__playerReadywindow.__renderReadywindow.__timelineswindow.__clipManifest
postMessage:
- parent -> runtime control:
source: "hf-parent"type: "control"- actions:
play,pause,seek,set-muted,set-playback-rate,enable-pick-mode,disable-pick-mode
- runtime -> parent events:
source: "hf-preview"type: "state"andtype: "timeline"type: "ready"— emitted once wheninstallRuntimeControlBridgeregisters the control-message listener. The parent uses it to replay current playback state (set-muted,set-volume,set-playback-rate) so any control message sent before the listener was installed isn't lost. Emitted again on every iframe reload because the new runtime instance starts with no state.
Determinism baseline:
renderSeekis the producer-canonical seek path.- 30fps quantization and readiness gates are correctness requirements.
Build
bun run --filter @hyperframes/core build:hyperframes-runtime
Security Expectations
- Runtime bootstrap URL must be version-pinned and host-allowlisted.
- Iframe bridge payloads must be schema-validated.
- Unsafe URL schemes (
javascript:and unapproveddata:) are rejected. - Fail closed if runtime bootstrap/handshake is not healthy.
Product Editing Model
- Primary mode: prompt + element picking.
- Secondary mode: manual precision controls.
- Avoid timeline-first manual workflows as default product path.