Files
hyperframes/packages/core/src/runtime/bridge.test.ts
T
Miguel Ángel 3256551a5e fix(player): single-owner audio to prevent double voice in preview (#298)
## Summary

Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip).

## Root cause

Two audio pipelines were playing the same source in parallel:

1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path.
2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message.

The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks.

Confirmed via agent-browser instrumentation on `factory-series-c-video`:
- 6 `volumechange` events per play cycle (mute-fight signature)
- Both copies audible at `volume=1`, offset growing 23ms → 80ms
- Every seek widened the drift further

PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback.

## Fix

Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime.

- **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash.
- **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold.

Two orthogonal mute channels replace the volume fight:

| Channel | Purpose |
|---|---|
| `set-muted` | User's mute preference (existing, unchanged) |
| `set-media-output-muted` | Internal ownership handoff (new) |

`syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it.

## Why this shape

- **Single owner, explicit transition.** No races, no tug-of-war.
- **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing.
- **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact.
- **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift.
- **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior.
- **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files.

## Files changed

- `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types.
- `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields.
- `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`.
- `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`.
- `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session.
- `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership.

## Verified end-to-end with agent-browser on `factory-series-c-video`

**Runtime ownership (default — desktop studio):**

| | Before | After |
|---|---|---|
| `PARENT.play()` calls per play cycle | 1 | **0** |
| iframe `volumechange` events | 6 | **0** |
| Audible streams | 2 (drifting) | **1 (iframe)** |

**Parent ownership (simulated autoplay block — direct message):**

| | Value |
|---|---|
| iframe audio | `muted=true`, `volume=1` (untouched) |
| parent audio | `muted=false`, `volume=1`, audible |
| Parent ↔ iframe `currentTime` offset | ~6 ms steady state |
| Offset > 150 ms | corrected by mirror sync |

**Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:**

Event timeline captured via agent-browser instrumentation:

```
t=0.0 ms   IFRAME.play() called                       ← runtime attempts playback
t=0.4 ms   IFRAME.play() REJECTED: NotAllowedError    ← simulated mobile gate
t=0.4 ms   →IFRAME bridge set-media-output-muted=true ← player promotes
t=0.6 ms   PARENT.play() called                       ← parent proxy starts
t=0.8 ms   ←IFRAME msg media-autoplay-blocked         ← runtime signal
t=1.0 ms   PARENT.play() resolved                     ← audible
t=1.3 ms   IFRAME muted=true, volume=1                ← iframe silenced via native muted
```

Steady state at t=4 s under promoted parent ownership:

| Element | currentTime | paused | volume | muted |
|---|---|---|---|---|
| Parent audio | 4.060 s | false | 1.0 | **false** (audible) |
| Iframe audio | 4.068 s | false | 1.0 | **true** (silent) |

**Offset: 8 ms**, single audible stream, orthogonal mute channel respected.

## Test plan

- [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`)
- [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow)
- [x] `bun run build` — all packages green
- [x] Fresh preview + browser repro on `factory-series-c-video`:
  - [x] Runtime ownership: single audio stream, no drift
  - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible
  - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset
  - [x] No `volumechange` thrash in either ownership mode
- [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above)
2026-04-17 04:46:22 +02:00

140 lines
4.7 KiB
TypeScript

import { describe, it, expect, vi } from "vitest";
import { installRuntimeControlBridge } from "./bridge";
function createMockDeps() {
return {
onPlay: vi.fn(),
onPause: vi.fn(),
onSeek: vi.fn(),
onSetMuted: vi.fn(),
onSetMediaOutputMuted: vi.fn(),
onSetPlaybackRate: vi.fn(),
onEnablePickMode: vi.fn(),
onDisablePickMode: vi.fn(),
};
}
function makeControlMessage(action: string, extra?: Record<string, unknown>) {
return new MessageEvent("message", {
data: { source: "hf-parent", type: "control", action, ...extra },
});
}
describe("installRuntimeControlBridge", () => {
it("dispatches play command", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("play"));
expect(deps.onPlay).toHaveBeenCalledOnce();
});
it("dispatches pause command", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("pause"));
expect(deps.onPause).toHaveBeenCalledOnce();
});
it("dispatches seek command with frame and mode", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("seek", { frame: 150, seekMode: "drag" }));
expect(deps.onSeek).toHaveBeenCalledWith(150, "drag");
});
it("seek defaults frame to 0 and seekMode to commit", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("seek"));
expect(deps.onSeek).toHaveBeenCalledWith(0, "commit");
});
it("dispatches set-muted command", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("set-muted", { muted: true }));
expect(deps.onSetMuted).toHaveBeenCalledWith(true);
});
it("dispatches set-media-output-muted command", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("set-media-output-muted", { muted: true }));
expect(deps.onSetMediaOutputMuted).toHaveBeenCalledWith(true);
handler(makeControlMessage("set-media-output-muted", { muted: false }));
expect(deps.onSetMediaOutputMuted).toHaveBeenCalledWith(false);
});
it("set-media-output-muted coerces absent flag to false", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("set-media-output-muted"));
expect(deps.onSetMediaOutputMuted).toHaveBeenCalledWith(false);
});
it("dispatches set-playback-rate command", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("set-playback-rate", { playbackRate: 2 }));
expect(deps.onSetPlaybackRate).toHaveBeenCalledWith(2);
});
it("defaults playbackRate to 1", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("set-playback-rate"));
expect(deps.onSetPlaybackRate).toHaveBeenCalledWith(1);
});
it("dispatches enable-pick-mode", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("enable-pick-mode"));
expect(deps.onEnablePickMode).toHaveBeenCalledOnce();
});
it("dispatches disable-pick-mode", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("disable-pick-mode"));
expect(deps.onDisablePickMode).toHaveBeenCalledOnce();
});
it("ignores messages from wrong source", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(
new MessageEvent("message", {
data: { source: "other", type: "control", action: "play" },
}),
);
expect(deps.onPlay).not.toHaveBeenCalled();
});
it("ignores messages with wrong type", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(
new MessageEvent("message", {
data: { source: "hf-parent", type: "state", action: "play" },
}),
);
expect(deps.onPlay).not.toHaveBeenCalled();
});
it("ignores null data", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(new MessageEvent("message", { data: null }));
expect(deps.onPlay).not.toHaveBeenCalled();
});
it("handles flash-elements command without crashing", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
expect(() =>
handler(makeControlMessage("flash-elements", { selectors: [".test"], duration: 500 })),
).not.toThrow();
});
});