fix(slideshow): present media controls (#1601)

* fix(slideshow): harden media controls in present decks

* refactor(slideshow): clear Fallow audit findings

Decompose flagged high-CRAP functions and extract production-code
duplications so the audit gate clears.

- core/runtime/bridge.ts handler — replace the 14-branch if-chain with a
  CONTROL_HANDLERS dispatch table; flash-elements payload handling moves
  to its own helper. Behavior preserved (all existing bridge.test.ts
  cases hit the same dispatchers via the public installRuntimeControlBridge
  API).

- player/slideshow/SlideshowController syncTo — split into
  isValidSyncTarget / isCrossSlide / rerootStackTo helpers. The
  stopSlideMedia decision and the stack re-rooting are now individually
  named; the public method is a 4-line orchestrator.

- cli/commands/validate.ts run — extract emitJsonReport / emitTextReport
  so the orchestrator no longer carries the dual JSON/text branches.
  Cuts the cyclomatic complexity flagged by fallow after the
  shouldIgnoreRequestFailure signature expansion shifted the fingerprint.

- player/hyperframes-player.ts — _setIframeMediaMuted and _stopIframeMedia
  shared a `try { iframeDoc = contentDocument } catch { return }` preamble
  (clone group 15). Extract _getSameOriginIframeDocument(): Document | null
  and have both call sites consume it.

- studio/panels/SlideshowPanel.tsx — the notes controller's debounce-tail
  and explicit flush() shared the pending-drain pattern (clone group 16).
  Extract a drainPending() closure both call.

- player/hyperframes-player.test.ts — collapse the new stopMedia / muted
  tests' repeated Object.defineProperty(iframe, "contentDocument", { get })
  shape behind a stubIframeContentDocument helper.

No behavior changes — refactor only. Existing tests cover the affected
paths unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(validate): split run further; ignore test dup parity

Second Fallow pass surfaced two minor follow-ups after the first cut:

- packages/cli/src/commands/validate.ts run + emitTextReport still
  carried minor CRAP findings (43.1 / 37.1, threshold 30). Extract
  printValidationResult / formatConsoleEntry / formatTotals /
  emitFailureReport so run becomes a try/catch + delegation, well
  below the threshold; emitTextReport drops the inline format loops.

- .fallowrc.jsonc duplicates.ignore: add hyperframes-player.test.ts
  alongside the existing SlideshowPanel.test.ts entry. Same reasoning
  documented there — parallel arrange/act/assert test cases are
  intentionally self-contained for readability; collapsing them under
  shared fixtures would couple unrelated scenarios (same-origin vs
  realm media, audio-locked permutations, seek bridge variants).

No behavior changes — refactor + config-policy parity only.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-06-19 17:17:39 -07:00
committed by GitHub
co-authored by Claude Opus 4.7
parent cd832f01ac
commit f0c4dee705
23 changed files with 959 additions and 371 deletions
+6
View File
@@ -247,6 +247,12 @@
// SlideshowPanel.test.ts: parallel arrange/act/assert test cases — collapsing
// them would hurt readability of what each case verifies.
"packages/studio/src/components/panels/SlideshowPanel.test.ts",
// hyperframes-player.test.ts: parallel arrange/act/assert test cases verifying
// distinct behaviors (same-origin vs realm media, audio-locked permutations,
// seek bridge variants). Each case is self-contained for readability;
// extracting the iframe / mock-audio setup helpers would over-couple
// unrelated scenarios under a shared fixture.
"packages/player/src/hyperframes-player.test.ts",
// present.ts mirrors play.ts's server startup + console-output block. The
// shared low-level pieces (resolve*/injectRuntime/listenOnFreePort) are in
// utils/compositionServer.ts; the remaining clone is per-command logging text
+16 -2
View File
@@ -174,7 +174,7 @@ function buildPresentPage(projectName: string, islandJson: string): string {
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>${projectName} — Presenter</title>
<title>${escHtml(projectName)} — Presenter</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
html, body { height: 100%; background: #0a0a0a; overflow: hidden; }
@@ -193,7 +193,7 @@ function buildPresentPage(projectName: string, islandJson: string): string {
</head>
<body>
<hyperframes-slideshow tabindex="0" sound>
<hyperframes-player src="/composition/index.html"></hyperframes-player>
<hyperframes-player interactive src="/composition/index.html"></hyperframes-player>
<script type="application/hyperframes-slideshow+json">
${islandJson}
</script>
@@ -241,10 +241,16 @@ ${islandJson}
// Mute state is owned by <hyperframes-slideshow sound>; mirror it.
var muted = false;
function setClipsMuted(nextMuted) {
Object.keys(clips).forEach(function (name) {
clips[name].muted = nextMuted;
});
}
var ss = document.querySelector("hyperframes-slideshow");
if (ss) {
ss.addEventListener("hf-sound", function (e) {
muted = e.detail && e.detail.muted === true;
setClipsMuted(muted);
});
}
@@ -295,3 +301,11 @@ ${islandJson}
</body>
</html>`;
}
function escHtml(value: string): string {
return value
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
@@ -9,12 +9,26 @@ describe("shouldIgnoreRequestFailure", () => {
expect(shouldIgnoreRequestFailure("http://127.0.0.1:3000/video.mp4", "net::ERR_ABORTED")).toBe(
true,
);
expect(
shouldIgnoreRequestFailure(
"https://www.heygenverse.com/s/50f13ccf-9002-4d80-b567-9d4c0eac30d8/raw",
"net::ERR_ABORTED",
"media",
),
).toBe(true);
});
it("keeps non-media and non-aborted failures reportable", () => {
expect(
shouldIgnoreRequestFailure("http://127.0.0.1:3000/assets/map.png", "net::ERR_ABORTED"),
).toBe(false);
expect(
shouldIgnoreRequestFailure(
"https://www.heygenverse.com/s/50f13ccf-9002-4d80-b567-9d4c0eac30d8/raw",
"net::ERR_ABORTED",
"xhr",
),
).toBe(false);
expect(
shouldIgnoreRequestFailure("http://127.0.0.1:3000/assets/sfx.wav", "net::ERR_FAILED"),
).toBe(false);
+97 -61
View File
@@ -32,8 +32,13 @@ const CONTRAST_SAMPLES = 5;
const SEEK_SETTLE_MS = 150;
const MEDIA_EXTENSIONS = /\.(aac|flac|m4a|mov|mp3|mp4|oga|ogg|wav|webm)$/i;
export function shouldIgnoreRequestFailure(url: string, errorText: string | undefined): boolean {
export function shouldIgnoreRequestFailure(
url: string,
errorText: string | undefined,
resourceType?: string,
): boolean {
if (errorText !== "net::ERR_ABORTED") return false;
if (resourceType === "media") return true;
try {
return MEDIA_EXTENSIONS.test(new URL(url).pathname);
} catch {
@@ -166,7 +171,7 @@ async function validateInBrowser(
const url = req.url();
if (url.includes("favicon") || url.startsWith("data:")) return;
const failureText = req.failure()?.errorText;
if (shouldIgnoreRequestFailure(url, failureText)) return;
if (shouldIgnoreRequestFailure(url, failureText, req.resourceType())) return;
const path = decodeURIComponent(new URL(url).pathname).replace(/^\//, "");
errors.push({
level: "error",
@@ -210,6 +215,74 @@ function printContrastFailures(failures: ContrastEntry[]) {
}
}
function emitJsonReport(
errors: ConsoleEntry[],
warnings: ConsoleEntry[],
contrast: ContrastEntry[] | undefined,
contrastFailures: ContrastEntry[],
): void {
console.log(
JSON.stringify(
withMeta({
ok: errors.length === 0,
errors,
warnings,
contrast,
contrastFailures: contrastFailures.length,
}),
null,
2,
),
);
}
function formatConsoleEntry(prefix: string, e: ConsoleEntry): string {
return ` ${prefix} ${e.text}${e.line ? c.dim(` (line ${e.line})`) : ""}`;
}
function formatTotals(
errors: ConsoleEntry[],
warnings: ConsoleEntry[],
contrastFailures: ContrastEntry[],
): string {
const parts = [`${errors.length} error(s)`, `${warnings.length} warning(s)`];
if (contrastFailures.length > 0) parts.push(`${contrastFailures.length} contrast warning(s)`);
return parts.join(", ");
}
function emitTextReport(
errors: ConsoleEntry[],
warnings: ConsoleEntry[],
contrastFailures: ContrastEntry[],
contrastPassed: ContrastEntry[],
): void {
const hasIssues = errors.length > 0 || warnings.length > 0 || contrastFailures.length > 0;
if (!hasIssues) {
const suffix =
contrastPassed.length > 0 ? ` · ${contrastPassed.length} text elements pass WCAG AA` : "";
console.log(`${c.success("◇")} No console errors${suffix}`);
return;
}
console.log();
for (const e of errors) console.log(formatConsoleEntry(c.error("✗"), e));
for (const w of warnings) console.log(formatConsoleEntry(c.warn("⚠"), w));
if (contrastFailures.length > 0) printContrastFailures(contrastFailures);
console.log();
console.log(`${c.accent("◇")} ${formatTotals(errors, warnings, contrastFailures)}`);
}
function emitFailureReport(message: string, asJson: boolean): void {
if (asJson) {
console.log(
JSON.stringify(withMeta({ ok: false, error: message, errors: [], warnings: [] }), null, 2),
);
return;
}
console.error(`${c.error("✗")} ${message}`);
}
export default defineCommand({
meta: {
name: "validate",
@@ -239,73 +312,36 @@ Examples:
const project = resolveProject(args.dir);
const timeout = parseInt(args.timeout as string, 10) || 3000;
const useContrast = args.contrast ?? true;
const asJson = Boolean(args.json);
if (!args.json) {
if (!asJson) {
console.log(`${c.accent("◆")} Validating ${c.accent(project.name)} in headless Chrome`);
}
try {
const { errors, warnings, contrast } = await validateInBrowser(project.dir, {
timeout,
contrast: useContrast,
});
const contrastFailures = (contrast ?? []).filter((e) => !e.wcagAA);
const contrastPassed = (contrast ?? []).filter((e) => e.wcagAA);
if (args.json) {
console.log(
JSON.stringify(
withMeta({
ok: errors.length === 0,
errors,
warnings,
contrast,
contrastFailures: contrastFailures.length,
}),
null,
2,
),
);
process.exit(errors.length > 0 ? 1 : 0);
}
if (errors.length === 0 && warnings.length === 0 && contrastFailures.length === 0) {
const suffix =
contrastPassed.length > 0 ? ` · ${contrastPassed.length} text elements pass WCAG AA` : "";
console.log(`${c.success("◇")} No console errors${suffix}`);
return;
}
console.log();
for (const e of errors) {
console.log(` ${c.error("✗")} ${e.text}${e.line ? c.dim(` (line ${e.line})`) : ""}`);
}
for (const w of warnings) {
console.log(` ${c.warn("⚠")} ${w.text}${w.line ? c.dim(` (line ${w.line})`) : ""}`);
}
if (contrastFailures.length > 0) printContrastFailures(contrastFailures);
console.log();
const parts = [`${errors.length} error(s)`, `${warnings.length} warning(s)`];
if (contrastFailures.length > 0) parts.push(`${contrastFailures.length} contrast warning(s)`);
console.log(`${c.accent("◇")} ${parts.join(", ")}`);
process.exit(errors.length > 0 ? 1 : 0);
const result = await validateInBrowser(project.dir, { timeout, contrast: useContrast });
const exitCode = printValidationResult(result, asJson);
process.exit(exitCode);
} catch (err: unknown) {
const message = err instanceof Error ? err.message : String(err);
if (args.json) {
console.log(
JSON.stringify(
withMeta({ ok: false, error: message, errors: [], warnings: [] }),
null,
2,
),
);
process.exit(1);
}
console.error(`${c.error("✗")} ${message}`);
emitFailureReport(message, asJson);
process.exit(1);
}
},
});
function printValidationResult(
result: { errors: ConsoleEntry[]; warnings: ConsoleEntry[]; contrast?: ContrastEntry[] },
asJson: boolean,
): number {
const { errors, warnings, contrast } = result;
const contrastFailures = (contrast ?? []).filter((e) => !e.wcagAA);
const contrastPassed = (contrast ?? []).filter((e) => e.wcagAA);
if (asJson) {
emitJsonReport(errors, warnings, contrast, contrastFailures);
} else {
emitTextReport(errors, warnings, contrastFailures, contrastPassed);
}
return errors.length > 0 ? 1 : 0;
}
+8
View File
@@ -5,6 +5,7 @@ function createMockDeps() {
return {
onPlay: vi.fn(),
onPause: vi.fn(),
onStopMedia: vi.fn(),
onSeek: vi.fn(),
onTick: vi.fn(),
onSetMuted: vi.fn(),
@@ -39,6 +40,13 @@ describe("installRuntimeControlBridge", () => {
expect(deps.onPause).toHaveBeenCalledOnce();
});
it("dispatches stop-media command", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
handler(makeControlMessage("stop-media"));
expect(deps.onStopMedia).toHaveBeenCalledOnce();
});
it("dispatches seek command with frame and mode", () => {
const deps = createMockDeps();
const handler = installRuntimeControlBridge(deps);
+37 -54
View File
@@ -5,6 +5,7 @@ import type { RuntimeBridgeControlMessage, RuntimeOutboundMessage } from "./type
type BridgeDeps = {
onPlay: () => void;
onPause: () => void;
onStopMedia: () => void;
onSeek: (frame: number, seekMode: "drag" | "commit") => void;
onTick: () => void;
onSetMuted: (muted: boolean) => void;
@@ -29,60 +30,33 @@ export function postRuntimeMessage(payload: RuntimeOutboundMessage): void {
}
}
export function installRuntimeControlBridge(deps: BridgeDeps): (event: MessageEvent) => void {
const handler = (event: MessageEvent) => {
const data = event.data as Partial<RuntimeBridgeControlMessage> | null;
if (!data || data.source !== "hf-parent" || data.type !== "control") return;
const action = data.action;
if (action === "play") {
deps.onPlay();
return;
}
if (action === "pause") {
deps.onPause();
return;
}
if (action === "seek") {
deps.onSeek(Number(data.frame ?? 0), data.seekMode ?? "commit");
return;
}
if (action === "tick") {
deps.onTick();
return;
}
if (action === "set-muted") {
deps.onSetMuted(Boolean(data.muted));
return;
}
if (action === "set-volume") {
deps.onSetVolume(Math.max(0, Math.min(1, Number(data.volume ?? 1))));
return;
}
if (action === "set-media-output-muted") {
deps.onSetMediaOutputMuted(Boolean(data.muted));
return;
}
if (action === "set-playback-rate") {
deps.onSetPlaybackRate(Number(data.playbackRate ?? 1));
return;
}
if (action === "set-color-grading") {
deps.onSetColorGrading(data.target ?? null, data.grading ?? null);
return;
}
if (action === "set-color-grading-compare") {
deps.onSetColorGradingCompare(data.target ?? null, data.compare ?? null);
return;
}
if (action === "enable-pick-mode") {
deps.onEnablePickMode();
return;
}
if (action === "disable-pick-mode") {
deps.onDisablePickMode();
return;
}
if (action === "flash-elements") {
type BridgeControlData = Partial<RuntimeBridgeControlMessage>;
type ControlHandler = (data: BridgeControlData, deps: BridgeDeps) => void;
// Per-action dispatchers. Splitting the handler into a lookup table keeps the
// top-level message listener trivial (one map lookup), and each action's logic
// becomes individually testable / inheritable for fallow's CRAP analysis.
const CONTROL_HANDLERS: Record<string, ControlHandler> = {
play: (_d, deps) => deps.onPlay(),
pause: (_d, deps) => deps.onPause(),
"stop-media": (_d, deps) => deps.onStopMedia(),
seek: (data, deps) => deps.onSeek(Number(data.frame ?? 0), data.seekMode ?? "commit"),
tick: (_d, deps) => deps.onTick(),
"set-muted": (data, deps) => deps.onSetMuted(Boolean(data.muted)),
"set-volume": (data, deps) =>
deps.onSetVolume(Math.max(0, Math.min(1, Number(data.volume ?? 1)))),
"set-media-output-muted": (data, deps) => deps.onSetMediaOutputMuted(Boolean(data.muted)),
"set-playback-rate": (data, deps) => deps.onSetPlaybackRate(Number(data.playbackRate ?? 1)),
"set-color-grading": (data, deps) =>
deps.onSetColorGrading(data.target ?? null, data.grading ?? null),
"set-color-grading-compare": (data, deps) =>
deps.onSetColorGradingCompare(data.target ?? null, data.compare ?? null),
"enable-pick-mode": (_d, deps) => deps.onEnablePickMode(),
"disable-pick-mode": (_d, deps) => deps.onDisablePickMode(),
"flash-elements": (data) => handleFlashElements(data),
};
function handleFlashElements(data: BridgeControlData): void {
// Briefly highlight elements — used by the chat-canvas bridge
// to show what changed after an agent edit
const selectors = (data as Record<string, unknown>).selectors as string[] | undefined;
@@ -91,6 +65,15 @@ export function installRuntimeControlBridge(deps: BridgeDeps): (event: MessageEv
flashElements(selectors, duration);
}
}
export function installRuntimeControlBridge(deps: BridgeDeps): (event: MessageEvent) => void {
const handler = (event: MessageEvent) => {
const data = event.data as BridgeControlData | null;
if (!data || data.source !== "hf-parent" || data.type !== "control") return;
const action = data.action;
if (typeof action !== "string") return;
const fn = CONTROL_HANDLERS[action];
if (fn) fn(data, deps);
};
window.addEventListener("message", handler);
// Announce that the bridge listener is installed so the parent can replay
+7
View File
@@ -1839,6 +1839,13 @@ export function initSandboxRuntimeModular(): void {
player.pause();
emitAnalyticsEvent("composition_paused", { time: player.getTime() });
},
onStopMedia: () => {
webAudio.stopAll();
const mediaEls = document.querySelectorAll("video, audio");
for (const el of mediaEls) {
if (el instanceof HTMLMediaElement && !el.paused) el.pause();
}
},
onSeek: (frame, _seekMode) => {
const time = Math.max(0, frame) / state.canonicalFps;
player.seek(time);
+1
View File
@@ -16,6 +16,7 @@ export type RuntimeBridgeControlAction =
| "tick"
| "set-volume"
| "set-media-output-muted"
| "stop-media"
| "flash-elements";
export type RuntimeBridgeControlMessage = {
+147 -2
View File
@@ -1,6 +1,55 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { formatTime, formatSpeed, SPEED_PRESETS } from "./controls.js";
// Install a stubbed contentDocument getter on the given iframe element. The
// new stopMedia / muted tests repeat this `Object.defineProperty(... { get })`
// shape; routing through a named helper keeps the per-test bodies focused on
// the actual assertion.
function stubIframeContentDocument(iframe: HTMLIFrameElement, doc: Document): void {
Object.defineProperty(iframe, "contentDocument", {
configurable: true,
get: () => doc,
});
}
function createForeignFrameMediaDocument(): {
doc: Document;
video: HTMLMediaElement & { pause: ReturnType<typeof vi.fn> };
audio: HTMLMediaElement & { pause: ReturnType<typeof vi.fn> };
} {
class FrameElement {
readonly tagName: string;
ownerDocument: {
defaultView: { Element: typeof FrameElement; HTMLMediaElement: typeof FrameElement };
} | null = null;
constructor(tagName: string) {
this.tagName = tagName;
}
}
class FrameMedia extends FrameElement {
muted = false;
defaultMuted = false;
pause = vi.fn();
}
const video = new FrameMedia("VIDEO");
const audio = new FrameMedia("AUDIO");
const fakeDoc = {
defaultView: { Element: FrameElement, HTMLMediaElement: FrameMedia },
querySelectorAll: () => [video, audio],
};
video.ownerDocument = fakeDoc;
audio.ownerDocument = fakeDoc;
return {
doc: fakeDoc as unknown as Document,
video: video as unknown as HTMLMediaElement & { pause: ReturnType<typeof vi.fn> },
audio: audio as unknown as HTMLMediaElement & { pause: ReturnType<typeof vi.fn> },
};
}
// ── Controls unit tests ──
describe("SPEED_PRESETS", () => {
@@ -833,8 +882,16 @@ describe("HyperframesPlayer seek() sync path", () => {
seek: (t: number) => void;
play: () => void;
pause: () => void;
stopMedia: () => void;
iframe: HTMLIFrameElement;
_currentTime: number;
_parentMedia: Array<{
el: { pause: ReturnType<typeof vi.fn>; src: string };
start: number;
duration: number;
driftSamples: number;
source?: HTMLMediaElement | null;
}>;
};
let player: PlayerInternal;
@@ -925,7 +982,8 @@ describe("HyperframesPlayer seek() sync path", () => {
player.seek(2);
expect(timeline.seek).toHaveBeenCalledTimes(1);
expect(timeline.seek).toHaveBeenCalledWith(2);
// suppressEvents=false so onUpdate fires (imperative-visibility compositions repaint).
expect(timeline.seek).toHaveBeenCalledWith(2, false);
expect(post).not.toHaveBeenCalled();
});
@@ -964,11 +1022,65 @@ describe("HyperframesPlayer seek() sync path", () => {
pause.mockClear();
player.seek(2);
expect(timeline.seek).toHaveBeenCalledWith(2);
expect(timeline.seek).toHaveBeenCalledWith(2, false);
expect(pause).toHaveBeenCalledTimes(1);
expect(post).not.toHaveBeenCalled();
});
it("stopMedia pauses slide media without stopping global audio-src proxies", () => {
const post = vi.fn();
const doc = document.implementation.createHTMLDocument("composition");
const iframeVideo = doc.createElement("video");
const iframeAudio = doc.createElement("audio");
const iframeVideoPause = vi.fn();
const iframeAudioPause = vi.fn();
Object.defineProperty(iframeVideo, "pause", { configurable: true, value: iframeVideoPause });
Object.defineProperty(iframeAudio, "pause", { configurable: true, value: iframeAudioPause });
doc.body.append(iframeVideo, iframeAudio);
stubIframeContentDocument(player.iframe, doc);
stubContentWindow({ postMessage: post });
const slideProxyPause = vi.fn();
const globalProxyPause = vi.fn();
player._parentMedia.push(
{
el: { pause: slideProxyPause, src: "https://cdn.example.com/slide.mp4" },
start: 0,
duration: 5,
driftSamples: 0,
source: iframeVideo,
},
{
el: { pause: globalProxyPause, src: "https://cdn.example.com/background.mp3" },
start: 0,
duration: Infinity,
driftSamples: 0,
source: null,
},
);
player.stopMedia();
expect(post).toHaveBeenCalledWith(expect.objectContaining({ action: "stop-media" }), "*");
expect(iframeVideoPause).toHaveBeenCalledOnce();
expect(iframeAudioPause).toHaveBeenCalledOnce();
expect(slideProxyPause).toHaveBeenCalledOnce();
expect(globalProxyPause).not.toHaveBeenCalled();
});
it("stopMedia pauses iframe-realm media elements", () => {
const post = vi.fn();
const { doc, video, audio } = createForeignFrameMediaDocument();
stubIframeContentDocument(player.iframe, doc);
stubContentWindow({ postMessage: post });
player.stopMedia();
expect(post).toHaveBeenCalledWith(expect.objectContaining({ action: "stop-media" }), "*");
expect(video.pause).toHaveBeenCalledOnce();
expect(audio.pause).toHaveBeenCalledOnce();
});
it("does not bypass an installed runtime bridge for direct __timelines playback", () => {
const timeline: TimelineStub = {
duration: vi.fn(() => 5),
@@ -1413,6 +1525,39 @@ describe("HyperframesPlayer volume and mute", () => {
expect(player.hasAttribute("muted")).toBe(false);
});
it("muted property directly mutes same-origin iframe media", () => {
document.body.appendChild(player);
const doc = document.implementation.createHTMLDocument("composition");
const video = doc.createElement("video");
const authoredMuted = doc.createElement("audio");
authoredMuted.defaultMuted = true;
doc.body.append(video, authoredMuted);
stubIframeContentDocument(player.iframeElement, doc);
player.muted = true;
expect(video.muted).toBe(true);
expect(authoredMuted.muted).toBe(true);
player.muted = false;
expect(video.muted).toBe(false);
expect(authoredMuted.muted).toBe(true);
});
it("muted property mutes iframe-realm media elements", () => {
document.body.appendChild(player);
const { doc, video, audio } = createForeignFrameMediaDocument();
audio.defaultMuted = true;
stubIframeContentDocument(player.iframeElement, doc);
player.muted = true;
expect(video.muted).toBe(true);
expect(audio.muted).toBe(true);
player.muted = false;
expect(video.muted).toBe(false);
expect(audio.muted).toBe(true);
});
it("sends set-volume control to iframe", () => {
document.body.appendChild(player);
+42 -1
View File
@@ -3,6 +3,7 @@ import { isControlsClick, setupControls, setupPoster } from "./controls-setup.js
import { adoptShadowStyles, createCompositionIframe, scaleIframeToFit } from "./iframe-dom.js";
import { DirectTimelineClock } from "./direct-timeline-clock.js";
import { ParentMediaManager } from "./parent-media.js";
import { isRealmHtmlMediaElement } from "./media-element-guards.js";
import { handleRuntimeMessage } from "./runtime-message-handler.js";
import {
SHADER_CAPTURE_SCALE_ATTR,
@@ -309,6 +310,12 @@ class HyperframesPlayer extends HTMLElement {
this.dispatchEvent(new Event("pause"));
}
stopMedia() {
this._sendControl("stop-media");
this._stopIframeMedia();
this._media.stopAdoptedMedia();
}
seek(timeInSeconds: number) {
if (!this._trySyncSeek(timeInSeconds) && !this._tryDirectTimelineSeek(timeInSeconds)) {
this._sendControl("seek", { frame: Math.round(timeInSeconds * 30) });
@@ -434,6 +441,7 @@ class HyperframesPlayer extends HTMLElement {
return;
}
this._media.updateMuted(val !== null);
this._setIframeMediaMuted(val !== null);
this._sendControl("set-muted", { muted: val !== null });
this.controlsApi?.updateMuted(val !== null);
this.dispatchEvent(new Event("volumechange"));
@@ -476,6 +484,35 @@ class HyperframesPlayer extends HTMLElement {
}
}
/**
* Returns the iframe's contentDocument if same-origin and reachable,
* otherwise null. Accessing contentDocument can throw on cross-origin
* iframes this swallows that as a clean null sentinel.
*/
private _getSameOriginIframeDocument(): Document | null {
try {
return this.iframe.contentDocument;
} catch {
return null;
}
}
private _setIframeMediaMuted(muted: boolean): void {
const iframeDoc = this._getSameOriginIframeDocument();
if (!iframeDoc) return;
for (const el of iframeDoc.querySelectorAll("video, audio")) {
if (isRealmHtmlMediaElement(el)) el.muted = muted || el.defaultMuted;
}
}
private _stopIframeMedia(): void {
const iframeDoc = this._getSameOriginIframeDocument();
if (!iframeDoc) return;
for (const el of iframeDoc.querySelectorAll("video, audio")) {
if (isRealmHtmlMediaElement(el)) el.pause();
}
}
/**
* Replay current bridge state to the iframe runtime. Triggered when the
* runtime announces `{type: "ready"}` repairs the race where the parent
@@ -531,7 +568,10 @@ class HyperframesPlayer extends HTMLElement {
// GSAP seek() preserves play state; player seek() contract lands paused.
private _tryDirectTimelineSeek(t: number): boolean {
return this._withDirectTimeline((tl) => {
tl.seek(t);
// suppressEvents=false: fire the timeline's onUpdate so compositions that
// drive scene visibility imperatively (via the root timeline's onUpdate,
// e.g. slideshow decks) repaint on a paused seek — not only while playing.
tl.seek(t, false);
tl.pause();
});
}
@@ -624,6 +664,7 @@ class HyperframesPlayer extends HTMLElement {
} catch {
/* cross-origin */
}
this._setIframeMediaMuted(this.muted);
if (this.hasAttribute("autoplay")) this.play();
}
@@ -0,0 +1,14 @@
export function isRealmElement(node: Node): node is Element {
const view = node.ownerDocument?.defaultView;
if (view && node instanceof view.Element) return true;
return node instanceof Element;
}
export function isRealmHtmlMediaElement(node: Node): node is HTMLMediaElement {
if (!isRealmElement(node)) return false;
if (node.tagName !== "AUDIO" && node.tagName !== "VIDEO") return false;
const view = node.ownerDocument?.defaultView;
if (view && node instanceof view.HTMLMediaElement) return true;
return node instanceof HTMLMediaElement;
}
+34 -21
View File
@@ -10,6 +10,7 @@
*/
import { selectMediaObserverTargets } from "./mediaObserverScope.js";
import { isRealmElement, isRealmHtmlMediaElement } from "./media-element-guards.js";
/** Minimum absolute drift before a currentTime correction is attempted. */
const MIRROR_DRIFT_THRESHOLD_SECONDS = 0.05;
@@ -172,6 +173,12 @@ export class ParentMediaManager {
for (const m of this._entries) m.el.pause();
}
stopAdoptedMedia(): void {
for (const m of this._entries) {
if (m.source) m.el.pause();
}
}
seekAll(timeInSeconds: number): void {
for (const m of this._entries) {
// Re-read live bounds so a trim/move just before a paused scrub gates and
@@ -228,8 +235,8 @@ export class ParentMediaManager {
// Synchronously mute iframe media to close the race window.
if (iframeDoc) {
for (const el of iframeDoc.querySelectorAll<HTMLMediaElement>("video, audio")) {
el.muted = true;
for (const el of iframeDoc.querySelectorAll("video, audio")) {
if (isRealmHtmlMediaElement(el)) el.muted = true;
}
}
@@ -251,10 +258,10 @@ export class ParentMediaManager {
* install a MutationObserver for media added later (sub-composition activation).
*/
setupFromIframe(iframeDoc: Document): void {
const mediaEls = iframeDoc.querySelectorAll<HTMLMediaElement>(
"audio[data-start], video[data-start]",
);
for (const iframeEl of mediaEls) this._adoptIframeMedia(iframeEl);
const mediaEls = iframeDoc.querySelectorAll("audio[data-start], video[data-start]");
for (const iframeEl of mediaEls) {
if (isRealmHtmlMediaElement(iframeEl)) this._adoptIframeMedia(iframeEl);
}
this._observeDynamicMedia(iframeDoc);
}
@@ -388,7 +395,7 @@ export class ParentMediaManager {
if (m.type === "attributes" && m.attributeName === "preload") {
const target = m.target;
if (
target instanceof HTMLMediaElement &&
isRealmHtmlMediaElement(target) &&
target.matches("audio[data-start], video[data-start]") &&
target.preload === "auto"
) {
@@ -398,28 +405,34 @@ export class ParentMediaManager {
}
for (const added of m.addedNodes) {
if (!(added instanceof Element)) continue;
if (!isRealmElement(added)) continue;
const candidates: HTMLMediaElement[] = [];
if (added.matches?.("audio[data-start], video[data-start]")) {
candidates.push(added as HTMLMediaElement);
if (
isRealmHtmlMediaElement(added) &&
added.matches("audio[data-start], video[data-start]")
) {
candidates.push(added);
}
const inside = added.querySelectorAll("audio[data-start], video[data-start]");
for (const el of inside) {
if (isRealmHtmlMediaElement(el)) candidates.push(el);
}
const inside = added.querySelectorAll?.<HTMLMediaElement>(
"audio[data-start], video[data-start]",
);
if (inside) for (const el of inside) candidates.push(el);
for (const el of candidates) this._adoptIframeMedia(el);
}
for (const removed of m.removedNodes) {
if (!(removed instanceof Element)) continue;
if (!isRealmElement(removed)) continue;
const dropped: HTMLMediaElement[] = [];
if (removed.matches?.("audio[data-start], video[data-start]")) {
dropped.push(removed as HTMLMediaElement);
if (
isRealmHtmlMediaElement(removed) &&
removed.matches("audio[data-start], video[data-start]")
) {
dropped.push(removed);
}
const inside = removed.querySelectorAll("audio[data-start], video[data-start]");
for (const el of inside) {
if (isRealmHtmlMediaElement(el)) dropped.push(el);
}
const inside = removed.querySelectorAll?.<HTMLMediaElement>(
"audio[data-start], video[data-start]",
);
if (inside) for (const el of inside) dropped.push(el);
for (const el of dropped) this._detachIframeMedia(el);
}
}
@@ -12,6 +12,7 @@ function fakePlayer() {
}),
play: vi.fn(() => {}),
pause: vi.fn(() => {}),
stopMedia: vi.fn(() => {}),
onTimeUpdate: (fn: (t: number) => void) => {
cb = fn;
return () => {
@@ -41,15 +42,14 @@ const SHOW: ResolvedSlideshow = {
};
/**
* Factory: controller on SHOW, advanced to fragmentIndex=1 via playback
* (emit 2 frag 0, next(), emit 4 frag 1). Used across Fix 8b + backToMain tests.
* Factory: controller on SHOW, advanced to fragmentIndex=1. Construction enters
* slide a at fragmentIndex 0 (its first fragment); one next() reveals fragment 1.
* Navigation is synchronous (seek-driven) no playback emit needed.
*/
function showAtFrag1() {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
p.emit(2.2); // fragmentIndex=0
c.next(); // target=4
p.emit(4.2); // fragmentIndex=1
c.next(); // fragmentIndex 0 → 1
return { p, c };
}
@@ -66,28 +66,48 @@ function showAtSlide1InDeep() {
}
describe("SlideshowController linear nav", () => {
it("enters the first slide on construction: jumps to the first hold (no auto-play)", () => {
it("enters the first slide on construction: seeks to the first fragment (no auto-play)", () => {
const p = fakePlayer();
new SlideshowController(p, SHOW);
// No auto-play: seeks (jumps) to the first hold, fragments[0]=2;
// playTo then plays a brief forward render-nudge and pauses there.
const c = new SlideshowController(p, SHOW);
// Synchronous seek-only hold: jump to fragments[0]=2, fragmentIndex 0, never play.
expect(p.seek).toHaveBeenCalledWith(2);
expect(p.play).toHaveBeenCalled();
expect(p.play).not.toHaveBeenCalled();
expect(c.position.slideIndex).toBe(0);
expect(c.position.fragmentIndex).toBe(0);
});
it("holds (pauses) at slide end when timeupdate reaches it", () => {
it("never auto-plays — a single seek both repaints and holds", () => {
const p = fakePlayer();
new SlideshowController(p, SHOW);
p.emit(2); // first fragment — handled separately; still inside slide
p.emit(5); // reached end
expect(p.pause).toHaveBeenCalled();
// Determinism: navigation is a pure seek; the player is never put into a
// playing state that could run on into the next fragment/scene.
expect(p.play).not.toHaveBeenCalled();
});
it("does not stop media on construction or same-slide fragment navigation", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
expect(p.stopMedia).not.toHaveBeenCalled();
c.next(); // slide a fragment 0 -> fragment 1, same slide
expect(p.stopMedia).not.toHaveBeenCalled();
});
it("stops media before changing to another slide", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
c.next(); // fragment 0 -> fragment 1, same slide
c.next(); // slide a -> slide b
expect(p.stopMedia).toHaveBeenCalledOnce();
expect(p.seek).toHaveBeenLastCalledWith(7.5);
});
it("next stops at the first fragment, not the next slide", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
p.emit(2.2); // play reaches fragment 0, controller pauses
expect(p.pause).toHaveBeenCalled();
// Construction already lands on fragment 0 of slide a.
expect(c.position.slideIndex).toBe(0);
expect(c.position.fragmentIndex).toBe(0);
});
@@ -95,10 +115,8 @@ describe("SlideshowController linear nav", () => {
it("next past the last fragment advances to the next slide immediately", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
c.next(); // -> fragment 1 target (2)
p.emit(2.2);
c.next(); // -> fragment 2 target (4)
p.emit(4.2);
// construction → fragment 0; next → fragment 1; next → slide b (no fragments)
c.next(); // fragmentIndex 0 → 1 (seek 4)
c.next(); // no more fragments — advance to slide b immediately
expect(c.position.slideIndex).toBe(1);
expect(p.seek).toHaveBeenLastCalledWith(7.5); // slide b midpoint
@@ -144,16 +162,13 @@ describe("SlideshowController linear nav", () => {
expect(c.position.slideIndex).toBe(0);
});
it("auto-pauses at a fragment, then next advances to the FOLLOWING fragment (not the end)", () => {
it("at a fragment, next advances to the FOLLOWING fragment (not the end)", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
p.emit(2.2); // auto-pause at fragments[0]=2
expect(c.position.fragmentIndex).toBe(0);
p.pause.mockClear(); // clear the pause from the auto-stop above
expect(c.position.fragmentIndex).toBe(0); // construction → fragment 0
c.next(); // should target fragments[1]=4, NOT slide.end=5
p.emit(4.2);
expect(p.pause).toHaveBeenCalled(); // must pause at 4, not skip to 5
expect(c.position.fragmentIndex).toBe(1);
expect(p.seek).toHaveBeenLastCalledWith(4);
});
});
@@ -191,6 +206,17 @@ describe("SlideshowController branching", () => {
expect(p.seek).toHaveBeenLastCalledWith(11.5); // slide c midpoint
});
it("stops media when entering and leaving a branch", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
c.enterBranch("deep");
expect(p.stopMedia).toHaveBeenCalledTimes(1);
c.back();
expect(p.stopMedia).toHaveBeenCalledTimes(2);
});
it("counter is scoped to the current sequence", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
@@ -218,31 +244,24 @@ describe("SlideshowController branching", () => {
});
});
describe("SlideshowController Fix 8a — fragmentIndex advances via onTime not next()", () => {
it("next() does NOT pre-increment fragmentIndex; onTime advances it when hold fires", () => {
describe("SlideshowController — fragmentIndex advances synchronously on next()", () => {
it("construction lands on fragment 0; next() reveals fragment 1 immediately", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
// fragmentIndex starts at -1 (enterSlide sets it)
expect(c.position.fragmentIndex).toBe(-1);
// Call next() — should NOT pre-increment fragmentIndex
// Seek-only model: entering a fragmented slide shows its first fragment.
expect(c.position.fragmentIndex).toBe(0);
expect(p.seek).toHaveBeenLastCalledWith(2); // fragments[0]
c.next();
expect(c.position.fragmentIndex).toBe(-1); // still -1 until onTime fires
// Simulate playback reaching the hold point (fragments[0]=2)
p.emit(2.2);
expect(c.position.fragmentIndex).toBe(0); // onTime advanced it
expect(c.position.fragmentIndex).toBe(1); // synchronous, no played tick
expect(p.seek).toHaveBeenLastCalledWith(4); // fragments[1]
});
it("next() after auto-pause targets the FOLLOWING fragment without pre-increment (regression)", () => {
it("next() targets the FOLLOWING fragment (not slide end) while fragments remain", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
p.emit(2.2); // auto-pause at fragments[0]=2; fragmentIndex=0
expect(c.position.fragmentIndex).toBe(0);
p.pause.mockClear();
c.next(); // should target fragments[1]=4 — fragmentIndex stays 0 until emit
expect(c.position.fragmentIndex).toBe(0); // NOT yet 1
p.emit(4.2);
expect(p.pause).toHaveBeenCalled();
expect(c.position.fragmentIndex).toBe(1); // onTime advanced it
c.next(); // fragment 0 → 1 (fragments[1]=4, NOT slide.end=5)
expect(c.position.fragmentIndex).toBe(1);
expect(p.seek).toHaveBeenLastCalledWith(4);
});
});
@@ -261,15 +280,14 @@ describe("SlideshowController Fix 8b — back() restores parent fragmentIndex",
expect(p.seek).toHaveBeenLastCalledWith(4); // fragments[1] = 4
});
it("back() when parent fragmentIndex=-1 seeks to slide start", () => {
it("resuming a fragmented slide at fragmentIndex -1 seeks to slide start", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
// Enter branch immediately (slide a HAS fragments; fragmentIndex is still -1,
// i.e. before the first reveal → resume to slide.start).
c.enterBranch("deep");
c.back();
// fragmentIndex -1 on a fragmented slide = before the first reveal. This state
// is reachable via syncTo (audience mirror); resume should seek to slide.start.
c.syncTo("main", 0, -1);
expect(c.position.fragmentIndex).toBe(-1);
expect(p.seek).toHaveBeenLastCalledWith(0);
expect(p.seek).toHaveBeenLastCalledWith(0); // slide a start
});
it("back() to a NO-fragment parent slide resumes at its midpoint, not frame 0", () => {
@@ -372,16 +390,17 @@ describe("SlideshowController Fix #backToMain — restores fragment position lik
expect(p.seek).toHaveBeenLastCalledWith(4);
});
it("backToMain when root fragmentIndex=-1 seeks to slide start", () => {
it("backToMain restores the root fragment the branch was entered from", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
// Enter branch immediately (root fragmentIndex is still -1)
// Construction lands on slide a fragment 0; enter a branch, then return.
c.enterBranch("deep");
c.backToMain();
expect(c.position.fragmentIndex).toBe(-1);
expect(p.seek).toHaveBeenLastCalledWith(0); // slide start
expect(c.position.slideIndex).toBe(0);
expect(c.position.fragmentIndex).toBe(0);
expect(p.seek).toHaveBeenLastCalledWith(2); // fragments[0]
});
it("backToMain with multiple nested branches restores root slide position", () => {
@@ -593,13 +612,13 @@ describe("SlideshowController next() — reveals remaining fragments at slide en
it("reveals the next fragment even when currentTime is already at slide end", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
// Simulate a static jump to slide end without having stepped through fragments.
p.currentTime = 5; // slide a end, fragmentIndex still -1
c.next();
// Should target the first fragment (2) rather than advancing to slide b.
// Static jump to slide end; pending fragments should still be revealed in order
// (the playhead position doesn't gate fragment stepping).
p.currentTime = 5; // slide a end
c.next(); // fragment 0 → 1, stays on slide a
expect(c.position.slideIndex).toBe(0);
expect(p.play).toHaveBeenCalled();
expect(p.seek).not.toHaveBeenLastCalledWith(5); // not advanced to slide b start
expect(c.position.fragmentIndex).toBe(1);
expect(p.seek).toHaveBeenLastCalledWith(4); // fragments[1], not slide b
});
});
@@ -614,12 +633,10 @@ describe("SlideshowController syncTo", () => {
expect(c.position.sequenceId).toBe("deep");
expect(c.position.slideIndex).toBe(0);
// Slide c has no fragments, so resumeSlide lands at its midpoint (restFrame) —
// the same visible-at-rest position enterSlide uses — not slide start. It then
// plays a render-nudge so the composition repaints; onTime pauses at the hold.
// the same visible-at-rest position enterSlide uses — not slide start. A single
// seek both repaints and holds (no sustained playback).
expect(p.seek).toHaveBeenLastCalledWith(11.5); // slide c midpoint (10 + 3*0.5)
expect(p.play).toHaveBeenCalled();
p.emit(50); // player passes the render-nudge hold
expect(p.pause).toHaveBeenCalled();
expect(p.play).not.toHaveBeenCalled();
});
it("syncs a main-line slide+fragment position without animating", () => {
@@ -632,6 +649,26 @@ describe("SlideshowController syncTo", () => {
expect(p.seek).toHaveBeenLastCalledWith(4); // fragments[1] = 4
});
it("does not stop media when syncing only the fragment within the same slide", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
c.syncTo("main", 0, 1);
expect(p.stopMedia).not.toHaveBeenCalled();
});
it("stops media when audience sync moves to a different slide or sequence", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
c.syncTo("main", 1, -1);
expect(p.stopMedia).toHaveBeenCalledTimes(1);
c.syncTo("deep", 0, -1);
expect(p.stopMedia).toHaveBeenCalledTimes(2);
});
it("ignores an unknown sequence target", () => {
const p = fakePlayer();
const c = new SlideshowController(p, SHOW);
@@ -4,6 +4,7 @@ export interface PlayerPort {
seek(t: number): void;
play(): void;
pause(): void;
stopMedia?(): void;
readonly currentTime: number;
onTimeUpdate(cb: (t: number) => void): () => void;
}
@@ -15,33 +16,21 @@ interface StackFrame {
}
const MAIN = "main";
const EPS = 0.001;
// Seconds to play past a restored/mirrored position so the composition repaints
// (a bare paused seek doesn't re-render some compositions; pausing on the first
// timeupdate fires before a paint).
const RENDER_NUDGE = 0.2;
export class SlideshowController {
private stack: StackFrame[] = [{ sequenceId: MAIN, slideIndex: 0, fragmentIndex: -1 }];
private holdAt: number | null = null;
// The logical hold (a fragment time / slide point). playTo() plays a short way
// PAST it (to holdAt) so the composition repaints; holdTarget is what onTime
// matches against fragments to advance fragmentIndex.
private holdTarget: number | null = null;
private changeCbs = new Set<() => void>();
private unsub: () => void;
constructor(
private player: PlayerPort,
private show: ResolvedSlideshow,
) {
this.unsub = player.onTimeUpdate((t) => this.onTime(t));
this.enterSlide(0);
}
// fallow-ignore-next-line unused-class-member
dispose(): void {
this.unsub();
// No subscriptions to tear down — navigation is seek-driven (see playTo).
}
private slidesOf(sequenceId: string): ResolvedSlide[] {
@@ -103,19 +92,29 @@ export class SlideshowController {
for (const cb of this.changeCbs) cb();
}
private stopSlideMedia(): void {
this.player.stopMedia?.();
}
private enterSlide(index: number): void {
if (index !== this.frame.slideIndex) this.stopSlideMedia();
this.frame.slideIndex = index;
this.frame.fragmentIndex = -1;
this.holdAt = null;
const slide = this.currentSlide;
if (!slide) return;
if (!slide) {
this.frame.fragmentIndex = -1;
return;
}
// Jump to the slide's first hold and stay there (no auto-progress). With
// fragments that's the first fragment; without, a settled frame INSIDE the
// slide (its midpoint) — NOT slide.end, which is the boundary where the next
// scene begins (else slide 1 would render slide 2's content).
const firstHold =
slide.fragments.length > 0 ? (slide.fragments[0] ?? slide.end) : this.restFrame(slide);
this.playTo(firstHold);
// fragments that's the first fragment (fragmentIndex 0); without, a settled
// frame INSIDE the slide (its midpoint) — NOT slide.end, which is the boundary
// where the next scene begins (else slide 1 would render slide 2's content).
if (slide.fragments.length > 0) {
this.frame.fragmentIndex = 0;
this.playTo(slide.fragments[0] ?? slide.end);
} else {
this.frame.fragmentIndex = -1;
this.playTo(this.restFrame(slide));
}
this.emitChange();
}
@@ -145,51 +144,25 @@ export class SlideshowController {
: slide.fragments.length > 0
? slide.start
: this.restFrame(slide);
this.holdAt = null;
this.playTo(seekTime);
this.emitChange();
}
private nextStop(slide: ResolvedSlide, fragmentIndex: number): number {
const next = slide.fragments[fragmentIndex + 1];
return next ?? slide.end;
}
/**
* Jump to hold time `t` and pause there NO sustained playback, so slides
* never auto-progress. Seeks just before `t` and plays a short render-nudge
* ending at `t`: a bare paused seek doesn't repaint some compositions, and
* pausing on the first timeupdate fires before a paint. onTime() pauses at `t`
* and advances fragmentIndex when `t` is a fragment boundary.
* Jump to hold time `t` and hold there a pure, synchronous seek with NO
* sustained playback, so a slide can never auto-progress.
*
* `player.seek(t)` drives the composition's GSAP timeline directly (the player
* reaches the same-origin iframe's `__timelines`), and GSAP `.seek()` renders
* that frame synchronously AND leaves the timeline paused. So one seek both
* repaints and holds deterministically, in every window including a
* backgrounded one. (The previous play-a-frame-then-pause-on-a-timeupdate
* "render nudge" left an unfocused audience window playing while it waited for
* a throttled tick to pause it that was the auto-progress / one-side-frozen
* flakiness. fragmentIndex is now set by the caller, not on a played tick.)
*/
private playTo(t: number): void {
// Seek to the EXACT target so the first repainted frame is the correct one —
// seeking BEFORE it (as a backward render-nudge) flashes a pre-target frame
// / the previous scene. Then play a short way PAST it so the composition
// actually repaints (a bare paused seek doesn't), and onTime() pauses there.
const slide = this.currentSlide;
this.holdTarget = t;
this.holdAt = slide ? Math.min(t + RENDER_NUDGE, slide.end) : t + RENDER_NUDGE;
this.player.seek(t);
this.player.play();
}
private onTime(tt: number): void {
if (this.holdAt !== null && tt >= this.holdAt - EPS) {
const target = this.holdTarget;
this.holdAt = null;
this.holdTarget = null;
// Advance fragmentIndex if the logical target is a fragment boundary.
const slide = this.currentSlide;
if (slide && target !== null) {
const fragIdx = slide.fragments.indexOf(target);
if (fragIdx !== -1) {
this.frame.fragmentIndex = fragIdx;
this.emitChange();
}
}
this.player.pause();
}
}
next(): void {
@@ -197,9 +170,10 @@ export class SlideshowController {
if (!slide) return;
const hasMoreFragments = this.frame.fragmentIndex + 1 < slide.fragments.length;
if (hasMoreFragments) {
// Reveal the next fragment. onTime() advances fragmentIndex at the hold.
const nextTarget = this.nextStop(slide, this.frame.fragmentIndex);
this.playTo(nextTarget);
// Reveal the next fragment — advance the index and seek to its hold time.
this.frame.fragmentIndex += 1;
const target = slide.fragments[this.frame.fragmentIndex] ?? slide.end;
this.playTo(target);
this.emitChange();
return;
}
@@ -233,12 +207,14 @@ export class SlideshowController {
enterBranch(sequenceId: string): void {
const seq = this.show.sequences[sequenceId];
if (!seq || seq.slides.length === 0) return;
this.stopSlideMedia();
this.stack.push({ sequenceId, slideIndex: 0, fragmentIndex: -1 });
this.enterSlide(0);
}
back(): void {
if (this.stack.length <= 1) return;
this.stopSlideMedia();
this.stack.pop();
// Restore the saved fragmentIndex from the parent frame rather than
// resetting to -1 (which enterSlide would do). This preserves the exact
@@ -248,6 +224,7 @@ export class SlideshowController {
backToMain(): void {
if (this.stack.length <= 1) return;
this.stopSlideMedia();
this.stack = [this.stack[0]];
this.resumeSlide(this.frame.slideIndex, this.frame.fragmentIndex);
}
@@ -258,18 +235,40 @@ export class SlideshowController {
* statically via resumeSlide.
*/
syncTo(sequenceId: string, slideIndex: number, fragmentIndex: number): void {
const base = this.stack[0];
if (!base) return;
if (this.frame.sequenceId !== sequenceId) {
this.stack = [base];
if (sequenceId !== MAIN) {
const seq = this.show.sequences[sequenceId];
if (!seq || seq.slides.length === 0) return;
this.stack.push({ sequenceId, slideIndex: 0, fragmentIndex: -1 });
}
}
const slides = this.slidesOf(this.frame.sequenceId);
if (slideIndex < 0 || slideIndex >= slides.length) return;
if (!this.isValidSyncTarget(sequenceId, slideIndex)) return;
if (this.isCrossSlide(sequenceId, slideIndex)) this.stopSlideMedia();
if (!this.rerootStackTo(sequenceId)) return;
this.resumeSlide(slideIndex, fragmentIndex);
}
/** True if the target sequence + slide index resolves to a real slide. */
private isValidSyncTarget(sequenceId: string, slideIndex: number): boolean {
if (!this.stack[0]) return false;
const targetSlides =
sequenceId === MAIN ? this.show.slides : (this.show.sequences[sequenceId]?.slides ?? null);
if (!targetSlides) return false;
return slideIndex >= 0 && slideIndex < targetSlides.length;
}
/** True if the sync target lands on a different slide than current. */
private isCrossSlide(sequenceId: string, slideIndex: number): boolean {
return this.frame.sequenceId !== sequenceId || this.frame.slideIndex !== slideIndex;
}
/**
* Re-root the navigation stack to `sequenceId` if we're not already there.
* Returns false only when the target branch sequence is empty (no slides),
* mirroring the early-return guard in the previous inline form.
*/
private rerootStackTo(sequenceId: string): boolean {
if (this.frame.sequenceId === sequenceId) return true;
const base = this.stack[0];
if (!base) return false;
this.stack = [base];
if (sequenceId === MAIN) return true;
const seq = this.show.sequences[sequenceId];
if (!seq || seq.slides.length === 0) return false;
this.stack.push({ sequenceId, slideIndex: 0, fragmentIndex: -1 });
return true;
}
}
@@ -330,6 +330,38 @@ describe("<hyperframes-slideshow>", () => {
el.remove();
});
it("mute button applies globally to child players and page media", () => {
const el = document.createElement("hyperframes-slideshow") as any;
el.setAttribute("sound", "");
const player = document.createElement("hyperframes-player") as any;
player.muted = false;
el.appendChild(player);
const pageVideo = document.createElement("video");
document.body.append(pageVideo, el);
el.__setControllerForTest({
next: () => {},
prev: () => {},
onChange: () => () => {},
counter: { index: 1, total: 1 },
breadcrumb: [{ id: "main", label: "Main deck" }],
currentSlide: { hotspots: [] },
nextSlide: null,
});
const muteBtn = el.querySelector("[data-hf-mute]") as HTMLElement;
muteBtn.click();
expect(player.muted).toBe(true);
expect(pageVideo.muted).toBe(true);
const muteBtnAfter = el.querySelector("[data-hf-mute]") as HTMLElement;
muteBtnAfter.click();
expect(player.muted).toBe(false);
expect(pageVideo.muted).toBe(false);
el.remove();
pageVideo.remove();
});
it("mute button glyph reflects muted state (aria-pressed)", () => {
const el = makeEl({ index: 1, total: 2 });
el.setAttribute("sound", "");
@@ -459,10 +491,12 @@ describe("handleRuntimeMessage scenes seam", () => {
// ---------------------------------------------------------------------------
describe("<hyperframes-slideshow> presenter mode", () => {
beforeEach(async () => {
localStorage.clear();
await import("./hyperframes-slideshow.js");
});
afterEach(() => {
localStorage.clear();
vi.restoreAllMocks();
});
@@ -656,6 +690,56 @@ describe("<hyperframes-slideshow> presenter mode", () => {
el.remove();
});
it("presenter notes are editable and reload from localStorage", () => {
const el = makePresenterWithSlides({
currentSlide: { sceneId: "intro", notes: "Original manifest notes" },
nextSlide: { sceneId: "features", notes: "Highlight top 3 features" },
});
const notes = el.querySelector("[data-hf-presenter-notes]");
expect(notes).toBeInstanceOf(HTMLTextAreaElement);
const textarea = notes as HTMLTextAreaElement;
expect(textarea.value).toBe("Original manifest notes");
textarea.value = "Edited speaker notes";
textarea.dispatchEvent(new Event("input", { bubbles: true }));
const storageKey = textarea.getAttribute("data-hf-presenter-notes-key");
expect(storageKey).toBeTruthy();
expect(localStorage.getItem(storageKey ?? "")).toBe("Edited speaker notes");
el.remove();
const reloaded = makePresenterWithSlides({
currentSlide: { sceneId: "intro", notes: "Original manifest notes" },
nextSlide: { sceneId: "features", notes: "Highlight top 3 features" },
});
const reloadedNotes = reloaded.querySelector("[data-hf-presenter-notes]");
expect(reloadedNotes).toBeInstanceOf(HTMLTextAreaElement);
expect((reloadedNotes as HTMLTextAreaElement).value).toBe("Edited speaker notes");
reloaded.remove();
});
it("presenter notes preserve an intentionally cleared local edit", () => {
const el = makePresenterWithSlides({
currentSlide: { sceneId: "intro", notes: "Original manifest notes" },
nextSlide: null,
});
const notes = el.querySelector("[data-hf-presenter-notes]");
expect(notes).toBeInstanceOf(HTMLTextAreaElement);
const textarea = notes as HTMLTextAreaElement;
textarea.value = "";
textarea.dispatchEvent(new Event("input", { bubbles: true }));
el.remove();
const reloaded = makePresenterWithSlides({
currentSlide: { sceneId: "intro", notes: "Original manifest notes" },
nextSlide: null,
});
const reloadedNotes = reloaded.querySelector("[data-hf-presenter-notes]");
expect(reloadedNotes).toBeInstanceOf(HTMLTextAreaElement);
expect((reloadedNotes as HTMLTextAreaElement).value).toBe("");
reloaded.remove();
});
it("presenter chrome contains next slide sceneId when nextSlide is set", () => {
const el = makePresenterWithSlides({
currentSlide: { sceneId: "intro", notes: "Intro notes" },
@@ -32,10 +32,16 @@ interface ControllerLike {
dispose?(): void;
}
interface SlideNotesTarget {
sceneId?: string;
}
type PlayerElement = HTMLElement & {
seek(t: number): void;
play(): void;
pause(): void;
stopMedia?(): void;
muted?: boolean;
readonly currentTime: number;
readonly ready: boolean;
};
@@ -48,6 +54,8 @@ function isPlayerElement(el: HTMLElement): el is PlayerElement {
);
}
const PRESENTER_NOTES_STORAGE_PREFIX = "hf-slideshow:presenter-notes:v1:";
// Injected once per document to avoid duplicating @keyframes across multiple elements.
let _keyframesInjected = false;
function injectKeyframesOnce(): void {
@@ -277,6 +285,10 @@ export class HyperframesSlideshow extends HTMLElement {
seek: (t) => playerEl.seek(t),
play: () => playerEl.play(),
pause: () => playerEl.pause(),
stopMedia: () => {
playerEl.stopMedia?.();
this.stopDocumentMedia();
},
get currentTime() {
return playerEl.currentTime;
},
@@ -551,9 +563,14 @@ export class HyperframesSlideshow extends HTMLElement {
const muteBtn = chrome.querySelector("[data-hf-mute]");
const prevBtn = chrome.querySelector("[data-hf-prev]");
const nextBtn = chrome.querySelector("[data-hf-next]");
const notesInput = chrome.querySelector("[data-hf-presenter-notes]");
if (muteBtn) muteBtn.addEventListener("click", () => this.toggleMute());
if (prevBtn) prevBtn.addEventListener("click", () => this.controller?.prev());
if (nextBtn) nextBtn.addEventListener("click", () => this.controller?.next());
if (notesInput instanceof HTMLTextAreaElement) {
const key = notesInput.getAttribute("data-hf-presenter-notes-key");
notesInput.addEventListener("input", () => this.writePresenterNotes(key, notesInput.value));
}
const fsBtn = chrome.querySelector("[data-hf-fullscreen]");
if (fsBtn) fsBtn.addEventListener("click", () => this.toggleFullscreen());
for (const btn of chrome.querySelectorAll("[data-hotspot-id]")) {
@@ -588,6 +605,7 @@ export class HyperframesSlideshow extends HTMLElement {
} else {
this.removeAttribute("data-hf-muted");
}
this.applyGlobalMute(this._muted);
this.dispatchEvent(
new CustomEvent("hf-sound", {
detail: { muted: this._muted },
@@ -599,6 +617,83 @@ export class HyperframesSlideshow extends HTMLElement {
this.render();
}
private applyGlobalMute(muted: boolean): void {
for (const player of this.querySelectorAll("hyperframes-player")) {
if (!(player instanceof HTMLElement)) continue;
const playerEl = player as Partial<PlayerElement> & HTMLElement;
if ("muted" in playerEl) {
playerEl.muted = muted;
} else if (muted) {
playerEl.setAttribute("muted", "");
} else {
playerEl.removeAttribute("muted");
}
}
const doc = this.ownerDocument;
for (const el of doc.querySelectorAll("video, audio")) {
if (el instanceof HTMLMediaElement) el.muted = muted || el.defaultMuted;
}
}
private stopDocumentMedia(): void {
const doc = this.ownerDocument;
for (const el of doc.querySelectorAll("video, audio")) {
if (el instanceof HTMLMediaElement) el.pause();
}
}
private presenterNotesDeckKey(): string {
const explicit = this.getAttribute("notes-storage-key")?.trim();
if (explicit) return explicit;
const playerSrc = this.querySelector("hyperframes-player")?.getAttribute("src") ?? "";
let resolvedPlayerSrc = playerSrc;
try {
const baseHref = typeof location !== "undefined" ? location.href : "http://localhost/";
resolvedPlayerSrc = new URL(playerSrc, baseHref).href;
} catch {
// Keep the raw src when URL construction is unavailable.
}
const locationKey =
typeof location !== "undefined" ? `${location.origin}${location.pathname}` : "";
const title = this.ownerDocument.title;
return `${locationKey}|${title}|${resolvedPlayerSrc}`;
}
private presenterNotesStorageKey(slide: SlideNotesTarget): string | null {
const pos = this.controller?.position;
if (!pos) return null;
return `${PRESENTER_NOTES_STORAGE_PREFIX}${JSON.stringify([
this.presenterNotesDeckKey(),
pos.sequenceId,
pos.slideIndex,
slide.sceneId ?? "",
])}`;
}
private readPresenterNotes(key: string | null): string | null {
if (!key) return null;
if (typeof window === "undefined") return null;
try {
return window.localStorage.getItem(key);
} catch {
return null;
}
}
private writePresenterNotes(key: string | null, notes: string): void {
if (!key) return;
if (typeof window === "undefined") return;
try {
window.localStorage.setItem(key, notes);
} catch {
// localStorage may be disabled or quota-limited; editing still works for
// the current render even when persistence is unavailable.
}
}
private renderPresenter(): void {
if (!this.controller) return;
const { counter, currentSlide, nextSlide } = this.controller;
@@ -620,9 +715,12 @@ export class HyperframesSlideshow extends HTMLElement {
// Full-overlay chrome (pointer-events:none); the notes panel and nav cluster
// are the only interactive children.
const notesStorageKey = this.presenterNotesStorageKey(currentSlide);
const notes = this.readPresenterNotes(notesStorageKey) ?? currentSlide.notes ?? "";
this.paintChrome(
buildPresenterLayout({
notes: currentSlide.notes ?? "",
notes,
notesStorageKey,
nextText: nextPanelText(nextSlide),
counterText: `${counter.index} / ${counter.total}`,
elapsedText: formatElapsed(elapsedSec),
@@ -83,6 +83,7 @@ export class SlideshowChannel {
*/
export function buildPresenterLayout(opts: {
notes: string;
notesStorageKey: string | null;
nextText: string;
counterText: string;
elapsedText: string;
@@ -90,9 +91,7 @@ export function buildPresenterLayout(opts: {
}): string {
const esc = (s: string) => s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
const escAttr = (s: string) => esc(s).replace(/"/g, "&quot;");
const notes = opts.notes
? esc(opts.notes)
: `<span style="opacity:.4">No notes for this slide</span>`;
const notes = esc(opts.notes);
// Branch entries for the current slide — the presenter clicks these to enter a
// branch (the audience follows). The component wires [data-hotspot-id] to
// enterBranch(); positioned pills don't align with the letterboxed slide, so
@@ -110,7 +109,7 @@ export function buildPresenterLayout(opts: {
: "";
return `
<div data-hf-presenter style="position:absolute;left:0;right:0;bottom:0;height:32%;display:flex;background:#11151f;color:#fff;border-top:2px solid rgba(255,255,255,0.12);box-sizing:border-box;font-family:sans-serif;pointer-events:auto;">
<div data-hf-presenter-notes style="flex:1;min-width:0;padding:24px 36px;overflow-y:auto;font-size:21px;line-height:1.55;">${notes}</div>
<textarea data-hf-presenter-notes data-hf-presenter-notes-key="${escAttr(opts.notesStorageKey ?? "")}" aria-label="Speaker notes" placeholder="No notes for this slide" spellcheck="true" style="flex:1;min-width:0;padding:24px 36px;overflow:auto;font:inherit;font-size:21px;line-height:1.55;color:#fff;background:transparent;border:0;outline:none;resize:none;white-space:pre-wrap;pointer-events:auto;">${notes}</textarea>
<div style="width:380px;flex-shrink:0;border-left:1px solid rgba(255,255,255,0.12);padding:24px 28px;display:flex;flex-direction:column;gap:10px;">
<div style="font-size:12px;text-transform:uppercase;letter-spacing:.12em;opacity:.55;">Up next</div>
<div data-hf-presenter-next style="font-size:17px;opacity:.9;line-height:1.4;">${esc(opts.nextText)}</div>
+7
View File
@@ -23,6 +23,13 @@ export const PLAYER_STYLES = /* css */ `
pointer-events: none;
}
/* Opt-in: an interactive composition (e.g. a live slideshow/app with playable
media or controls) let pointer events reach the iframe content. */
:host([interactive]) .hfp-container,
:host([interactive]) .hfp-iframe {
pointer-events: auto;
}
.hfp-poster {
position: absolute;
inset: 0;
+3 -1
View File
@@ -21,7 +21,9 @@ export interface RuntimeDurationAdapter {
export interface DirectTimelineAdapter {
duration: () => number;
time: () => number;
seek: (timeInSeconds: number) => unknown;
// suppressEvents mirrors GSAP's timeline.seek(position, suppressEvents); pass
// false to fire onUpdate (so imperative-visibility compositions repaint on seek).
seek: (timeInSeconds: number, suppressEvents?: boolean) => unknown;
play: () => unknown;
pause: () => unknown;
/** Optional: set playback rate (e.g. GSAP's timeScale). Called when the player's playbackRate changes. */
@@ -57,7 +57,6 @@ export function safeParseManifest(html: string): SlideshowManifest {
import {
toggleMainLineSlide,
reorderMainLineSlide,
reorderBranchSlide,
setSlideNotes,
addFragment,
removeFragment,
@@ -94,19 +93,24 @@ export function makeSlideshowNotesController(): NotesController {
let pending: Pending | null = null;
let timer: ReturnType<typeof setTimeout> | null = null;
// Atomically swap the pending entry out and fire its persist (if any).
// Used by both the debounce timer's tail and the explicit flush() path.
const drainPending = (): void => {
const p = pending;
if (p === null) return;
pending = null;
p.persist(p.manifest).catch((err: unknown) => {
console.error("[slideshow] notes persist failed:", err);
});
};
return {
schedule(manifest, persist, delayMs) {
if (timer !== null) clearTimeout(timer);
pending = { manifest, persist };
timer = setTimeout(() => {
timer = null;
const p = pending;
if (p !== null) {
pending = null;
p.persist(p.manifest).catch((err: unknown) => {
console.error("[slideshow] notes persist failed:", err);
});
}
drainPending();
}, delayMs);
return timer;
},
@@ -116,13 +120,7 @@ export function makeSlideshowNotesController(): NotesController {
clearTimeout(timer);
timer = null;
}
const p = pending;
if (p !== null) {
pending = null;
p.persist(p.manifest).catch((err: unknown) => {
console.error("[slideshow] notes persist failed:", err);
});
}
drainPending();
},
cancel() {
@@ -293,15 +291,6 @@ export function SlideshowPanel({ scenes, onPersist, onPersistNotes }: SlideshowP
[applyManifest],
);
const handleReorderBranchSlide = useCallback(
(sequenceId: string, sceneId: string, dir: "up" | "down") => {
applyManifest(reorderBranchSlide(manifestRef.current, sequenceId, sceneId, dir)).catch(
() => {},
);
},
[applyManifest],
);
const handleSetNotes = useCallback(
(notes: string) => {
if (!selectedSceneId) return;
@@ -459,7 +448,6 @@ export function SlideshowPanel({ scenes, onPersist, onPersistNotes }: SlideshowP
selectedSceneId={selectedSceneId}
selectedSequenceId={selectedSequenceId}
onSelectBranchSlide={handleSelectBranchSlide}
onReorderBranchSlide={handleReorderBranchSlide}
/>
)}
@@ -216,7 +216,6 @@ export interface BranchTreeProps {
selectedSceneId: string | null;
selectedSequenceId: string | null;
onSelectBranchSlide: (sequenceId: string, sceneId: string) => void;
onReorderBranchSlide: (sequenceId: string, sceneId: string, dir: "up" | "down") => void;
}
export function BranchTree({
@@ -229,7 +228,6 @@ export function BranchTree({
selectedSceneId,
selectedSequenceId,
onSelectBranchSlide,
onReorderBranchSlide,
}: BranchTreeProps) {
const [newLabel, setNewLabel] = useState("");
const inputId = useId();
@@ -280,7 +278,6 @@ export function BranchTree({
selectedSceneId={selectedSceneId}
selectedSequenceId={selectedSequenceId}
onSelectBranchSlide={onSelectBranchSlide}
onReorderBranchSlide={onReorderBranchSlide}
/>
))}
</div>
@@ -298,7 +295,6 @@ interface BranchItemProps {
selectedSceneId: string | null;
selectedSequenceId: string | null;
onSelectBranchSlide: (sequenceId: string, sceneId: string) => void;
onReorderBranchSlide: (sequenceId: string, sceneId: string, dir: "up" | "down") => void;
}
function BranchItem({
@@ -310,7 +306,6 @@ function BranchItem({
selectedSceneId,
selectedSequenceId,
onSelectBranchSlide,
onReorderBranchSlide,
}: BranchItemProps) {
const [editing, setEditing] = useState(false);
const [draft, setDraft] = useState(seq.label);
@@ -362,8 +357,7 @@ function BranchItem({
</div>
<div className="flex flex-col gap-px pl-2">
{scenes.map((scene) => {
const branchPos = seq.slides.findIndex((s) => s.sceneId === scene.id);
const assigned = branchPos !== -1;
const assigned = seq.slides.some((s) => s.sceneId === scene.id);
const isSelected = selectedSequenceId === seq.id && selectedSceneId === scene.id;
return (
<div
@@ -378,7 +372,6 @@ function BranchItem({
className="accent-studio-accent flex-shrink-0"
/>
{assigned ? (
<>
<button
type="button"
aria-pressed={isSelected}
@@ -389,28 +382,6 @@ function BranchItem({
>
{scene.label || scene.id}
</button>
<span className="text-[9px] text-neutral-600 tabular-nums flex-shrink-0">
{branchPos + 1}/{seq.slides.length}
</span>
<button
type="button"
aria-label={`Move ${scene.label || scene.id} earlier in branch ${seq.label}`}
disabled={branchPos <= 0}
className="text-[10px] text-neutral-500 hover:text-neutral-200 disabled:opacity-30 disabled:cursor-default px-0.5"
onClick={() => onReorderBranchSlide(seq.id, scene.id, "up")}
>
</button>
<button
type="button"
aria-label={`Move ${scene.label || scene.id} later in branch ${seq.label}`}
disabled={branchPos >= seq.slides.length - 1}
className="text-[10px] text-neutral-500 hover:text-neutral-200 disabled:opacity-30 disabled:cursor-default px-0.5"
onClick={() => onReorderBranchSlide(seq.id, scene.id, "down")}
>
</button>
</>
) : (
<span className="flex-1 truncate">{scene.label || scene.id}</span>
)}
+63 -8
View File
@@ -137,25 +137,50 @@ These are hard constraints, not suggestions. A slide that violates them will be
- **Bottom-up market sizing only.** Never write "$50B TAM" without showing the math. Build from unit economics up: accounts × ACV, or transactions × take-rate.
- **Font minimum 30pt equivalent.** At 1920×1080, a headline is 7296px; body copy is 48px. Never go below 40px for any text the audience must read.
## Porting source pages
When converting an existing page into a slideshow, source fidelity is part of the contract. Do not replace source-specific widgets with simplified approximations unless the user explicitly asks for a redesign.
- Port mechanical visuals from the source DOM/CSS/JS as exactly as practical: custom players, canvas visualizers, timelines, playheads, stems, expanding circles, hover states, and other interactive details should survive the conversion.
- Audit the source for atypical page movement, especially behavior driven by scroll, wheel, touch, hash state, resize, or a requestAnimationFrame loop. Treat fixed viewports with translated/scaled "world" layers, parallax, pinned panels, horizontal scrollers, scroll-scrubbed timelines, section snapping, and zoom-to-element cameras as source behavior. Scroll is often the source's transition trigger, so preserve the transition by extracting its progress stops, easing, and camera/focus states, then re-host that motion on slideshow navigation through timeline positions, fragments, or a reusable player/harness hook. Do not simulate a literal page-scroll-down transition inside the slide; the viewer should feel camera travel/zoom from one focal point to another, not see a webpage being scrolled. Keep each slide-to-slide camera move continuous: avoid intermediate route stops that reverse x/y direction or zoom unless the source visibly does that at the same boundary. A transition that darts around before landing is worse than a simpler direct focal move.
- Preserve the source's media crop semantics. Treat screenshots, tweets/social posts, product UI captures, charts, docs, code, leaderboards, and any image with readable text as content evidence, not decorative media: use the source aspect ratio (`height: auto`) or `object-fit: contain` inside a stable frame. Use `object-fit: cover` only when the source did, or for intentionally decorative/background/cinematic thumbnails. After fitting these captures into a slide, inspect all four edges for truncated text, logos, controls, or captions; a visible crop on meaningful content is a bug unless the source itself cropped it.
- If a behavior is generic to slideshows, put it in the player/controller or in a reusable skill snippet. Do not solve it with one-off deck scripts.
- Stacked scene frames must never block interaction on the active slide. Hidden frames need both visual hiding and event gating:
```css
.scene-frame {
opacity: 0;
visibility: hidden;
pointer-events: none;
}
.scene-frame.is-active {
opacity: 1;
visibility: visible;
pointer-events: auto;
}
```
If visibility is driven imperatively, set all three properties (`opacity`, `visibility`, and `pointerEvents`) in the visibility controller. `opacity: 0` alone still leaves an invisible layer that can swallow clicks.
---
## Fragments: reveal hold-points within a slide
A fragment is a time (in seconds) within a slide's `[start, end]` range where the controller pauses before the next reveal.
A fragment is an absolute composition-timeline time (seconds) within a slide's `[start, end]` range where the controller should hold a reveal state.
**How it works:**
1. Player enters the slide — seeks to `start`, then plays.
2. Controller pauses at `fragments[0]`. The first element's GSAP entrance has just landed.
3. User presses Next (or →) — plays to `fragments[1]`, pauses again.
4. After the last fragment, Next plays to `slide.end` and holds.
5. Next again advances to the next slide.
1. Player enters a fragmented slide — seeks directly to `fragments[0]` and holds there.
2. User presses Next (or →) — controller seeks to `fragments[1]` and holds.
3. After the last fragment, Next advances to the next slide.
4. A slide without fragments enters at a rest frame inside the slide, usually its midpoint, not exactly at `slide.end`.
Fragment times must fall within `[start, end]` (inclusive of both bounds). The lint rule rejects only fragments outside that range (`time < start` or `time > end`).
Fragment times are **absolute composition-timeline positions** — the same coordinate space as `data-start` — not offsets relative to the scene's start.
Each fragment is a play-to-and-hold, not a seek jump — so every element that enters between the previous hold-point and this one plays its GSAP entrance animation. Design the clip entrance animations to work as sequential reveals.
Navigation is seek-driven, not play-driven. The controller never starts playback just to move between fragments; each navigation command is a deterministic seek to the target hold time. Design fragment states so they are correct at the target timeline time.
---
@@ -362,13 +387,43 @@ Wrap the composition in `<hyperframes-slideshow>` around `<hyperframes-player>`
**Presenter mode:** the Present button calls `window.open('?mode=audience')` for a fullscreen audience window; the originating tab becomes the presenter view (current slide reduced, next-slide preview, notes, elapsed timer). Both windows sync via `BroadcastChannel('hf-slideshow')`.
Presenter notes are editable in the presenter view. Edits are stored in `localStorage` per deck and slide, layered over the manifest notes without rewriting the composition file. Do not add one-off note-editing scripts to decks; rely on the shared slideshow player behavior. If a standalone/custom wrapper truly needs to implement this outside the shared player, use the deterministic storage snippet in `skills/slideshow/references/standalone-harness.md`.
### Media cleanup on slide exit
The slideshow controller owns slide-exit media cleanup. When navigation changes slide or sequence, it calls `hyperframes-player.stopMedia()` before entering the next slide. That command:
- posts `stop-media` to the iframe runtime, which stops WebAudio and pauses native `<video>` / `<audio>` elements;
- pauses same-origin iframe media directly as a fallback; and
- pauses parent-frame proxies adopted from iframe media.
Same-slide fragment navigation does **not** stop media. Global/deck-level parent audio, such as a background track wired through `audio-src`, is not treated as slide media.
Do not add per-slide cleanup scripts for normal media players. Keep slide video/audio as normal media in the composition; use `data-has-audio="true"` only when the player should preserve audible native video audio instead of treating it as silent visual media.
When implementing direct iframe fallback cleanup, treat iframe media as cross-realm DOM. Do not test iframe nodes with the parent page's `el instanceof HTMLMediaElement`; that returns false in real browsers. Use `el.ownerDocument.defaultView.HTMLMediaElement` (or an equivalent tag/duck-type guard) before setting `muted` or calling `pause()`.
### Global nav mute
When `<hyperframes-slideshow sound>` renders the nav mute button, that button is the global mute control for the page. It must mute:
- child `<hyperframes-player>` instances, including same-origin iframe media;
- top-level page `<audio>` / `<video>` elements; and
- wrapper-owned SFX/global `Audio` objects via the `hf-sound` event.
Do not add a second mute button inside the composition. If a wrapper script creates `new Audio(...)` objects that are not attached to the DOM, it must listen for `hf-sound` and set `clip.muted = detail.muted` on each object, not merely skip future plays.
The same cross-realm rule applies here: global mute must reach iframe `<video>` / `<audio>` elements through the child frame's DOM realm. A passing unit test in a single DOM realm is not enough; verify in a browser that the actual iframe media elements report `muted: true` after clicking the nav mute button.
`hyperframes present` serves built bundles from `packages/player/dist`. After changing player or slideshow chrome behavior, run `bun run build` in `packages/player` and restart the present server before testing in a browser.
---
## Running a slideshow standalone (interim)
The **durable answer** is engine-hosted: `hyperframes preview --slideshow` / studio present mode will host the composition over the real HyperFrames engine, which drives seek-timelines, owns the gesture frame, and reads the island from the composition. That path is coming; prefer it once it ships.
Until then, standalone demos (a composition opened via the bare player bundle in a browser, without the engine) require workarounds for four gaps: the player does not drive GSAP seek-timelines, the island must be duplicated into the wrapper, audio must live in the parent frame, and animations must be self-driving. These patterns are documented in:
Until then, standalone demos (a composition opened via the bare player bundle in a browser, without the engine) require workarounds for three gaps: the composition must expose a seekable root timeline, the island must be duplicated into the wrapper, and wrapper-owned SFX/global audio should live in the parent frame. These patterns are documented in:
```
skills/slideshow/references/standalone-harness.md
@@ -4,12 +4,11 @@
These patterns are a **temporary workaround** for standalone demos. The durable solution is engine-hosted: a future `hyperframes preview --slideshow` / studio present mode will host the composition over the real HyperFrames engine, which drives seek-timelines frame-by-frame, owns the gesture frame, and reads the slideshow island directly from the composition. When that path ships, most of what follows collapses.
Until then, a standalone slideshow opened via the bare player bundle must work around four facts:
Until then, a standalone slideshow opened via the bare player bundle must work around three facts:
1. The bare `<hyperframes-player>` does **not** drive GSAP/Three seek-timelines frame-by-frame — only the engine does. Animations that wait to be seeked stay at frame 0.
1. The composition must expose a seekable `window.__timelines.root` timeline. Anything outside that seek path, such as Three.js loops or imperative entrance effects, must be self-driving.
2. `<hyperframes-slideshow>` reads the slideshow island from its **own innerHTML** (the wrapper element), not from the composition the player loads. The island must be duplicated into the wrapper.
3. The composition runs in the player's **iframe**; user keypresses and pointer events land on the **parent page**. Any gesture-gated API (Audio, AudioContext) must live in the parent — an iframe without its own user activation is permanently autoplay-blocked.
4. Autoplay, Three.js, and entrance animations must be **self-driving** because the engine is not present.
3. The composition runs in the player's **iframe**; user keypresses and pointer events land on the **parent page**. Wrapper-owned SFX/global audio should live in the parent, where the activation token is reliable. Normal slide media stays in the composition and is stopped by the slideshow player on slide exit.
Do not treat these as the blessed authoring model. When the engine-hosted path ships, compositions authored the normal way will just work.
@@ -104,6 +103,60 @@ The parent page hosts the two dist bundles, wraps the components, duplicates the
</html>
```
### Editable presenter notes
The shared `<hyperframes-slideshow>` presenter already renders speaker notes as an editable textarea and stores edits in `localStorage`. Do not add deck-specific note editors when the shared player is available.
For interim custom wrappers that cannot use the shared presenter chrome, use this deterministic storage contract exactly so notes migrate cleanly:
```js
const NOTES_STORAGE_PREFIX = "hf-slideshow:presenter-notes:v1:";
function notesDeckKey(slideshowEl) {
const explicit = slideshowEl.getAttribute("notes-storage-key");
if (explicit && explicit.trim()) return explicit.trim();
const playerSrc = slideshowEl.querySelector("hyperframes-player")?.getAttribute("src") || "";
let resolvedPlayerSrc = playerSrc;
try {
resolvedPlayerSrc = new URL(playerSrc, location.href).href;
} catch {}
return `${location.origin}${location.pathname}|${document.title}|${resolvedPlayerSrc}`;
}
function notesStorageKey(slideshowEl, position, slide) {
return `${NOTES_STORAGE_PREFIX}${JSON.stringify([
notesDeckKey(slideshowEl),
position.sequenceId,
position.slideIndex,
slide.sceneId || "",
])}`;
}
function readPresenterNotes(slideshowEl, position, slide) {
const key = notesStorageKey(slideshowEl, position, slide);
try {
const stored = localStorage.getItem(key);
return stored == null ? slide.notes || "" : stored;
} catch {
return slide.notes || "";
}
}
function wirePresenterNotes(textarea, slideshowEl, position, slide) {
const key = notesStorageKey(slideshowEl, position, slide);
textarea.value = readPresenterNotes(slideshowEl, position, slide);
textarea.addEventListener("input", function () {
try {
localStorage.setItem(key, textarea.value);
} catch {}
});
}
```
Clearing the textarea must save an empty string, not remove the local value, because a presenter may intentionally blank a manifest note for their run. Use `notes-storage-key="stable-deck-id"` on `<hyperframes-slideshow>` when a standalone demo has a stable project id; otherwise the fallback key isolates by page, title, and player `src`.
---
## 3. Playhead-driven scene visibility
@@ -112,6 +165,8 @@ Without the engine, scenes are driven by a `root` GSAP timeline that the composi
The key insight: scene backgrounds must be `transparent` (not opaque) if you want a Three.js canvas behind them; the body/html background and scene inline `background` set the visual fill.
For converted source pages, port source-specific widgets exactly where practical. Custom canvas players, waveform/timeline decorations, expanding rings, playheads, hover states, and event wiring are source material, not optional polish. Also audit for atypical page movement: scroll-scrubbed cameras, parallax, pinned sections, horizontal scrollers, section snapping, translated/scaled world layers, or zoom-to-element navigation. Scroll is often the source's transition trigger, so extract the scroll-progress stops, easing, and camera/focus states, then re-host that motion on slideshow navigation through timeline positions, fragments, or reusable harness hooks. Do not recreate the browser's literal page-scroll-down motion inside a slide; translate it into camera travel/zoom from one focus area to the next. If the same mechanical behavior appears across decks, move it into the player or this harness instead of copying a fragile one-off script.
```html
<!-- In index.html (composition) -->
@@ -125,6 +180,7 @@ The key insight: scene backgrounds must be `transparent` (not opaque) if you wan
height: 1080px;
overflow: hidden;
opacity: 0; /* hidden at rest — visibility controller shows the active one */
visibility: hidden; /* opacity:0 alone still lets invisible frames block clicks */
pointer-events: none; /* inactive scenes must not swallow events */
}
</style>
@@ -188,6 +244,7 @@ The key insight: scene backgrounds must be `transparent` (not opaque) if you wan
if (!el) continue;
var active = t >= s.start && t < s.end;
el.style.opacity = active ? "1" : "0";
el.style.visibility = active ? "visible" : "hidden";
el.style.pointerEvents = active ? "auto" : "none";
if (active && lastActiveId !== s.id) {
@@ -334,7 +391,11 @@ Omitting any scene (including branch scenes) from this manifest means the slides
## 6. Audio/SFX — built-in mute control via `<hyperframes-slideshow sound>`
Audio **must** live in the parent page, not the composition iframe. Browsers enforce user-activation for AudioContext and HTMLAudioElement.play() — an iframe without its own activation (i.e., the user never clicked inside it) is permanently autoplay-blocked. The user's keypress lands on the parent, so the parent is the only frame that can get the activation token.
Wrapper-owned SFX should live in the parent page. Browsers enforce user-activation for AudioContext and HTMLAudioElement.play() — an iframe without its own activation (i.e., the user never clicked inside it) is often autoplay-blocked. The user's keypress lands on the parent, so the parent is the reliable frame for click/transition sound effects.
Normal slide media should stay in the composition. The slideshow player now stops slide media automatically on slide/sequence changes by calling `hyperframes-player.stopMedia()`, which pauses iframe `<video>` / `<audio>`, runtime WebAudio, and parent proxies adopted from iframe media. Same-slide fragment reveals do not stop media, and global/deck-level parent audio such as `audio-src` is left alone. Do not hand-roll per-slide cleanup scripts for regular video/audio players.
Implementation detail: iframe media elements belong to the iframe's DOM realm. Fallback code in the parent page/player must not use the parent page's `el instanceof HTMLMediaElement` check for iframe nodes; in real browsers that fails and leaves videos audible. Use `el.ownerDocument.defaultView.HTMLMediaElement` or a tag/duck-type guard before setting `muted` or calling `pause()`.
### Mute toggle — built-in chrome control
@@ -348,9 +409,11 @@ The component:
- Tracks `muted` state (default `false`); exposes a `muted` getter
- Reflects to a `data-hf-muted` attribute on the host when muted
- Applies mute globally to child `<hyperframes-player>` media and top-level page `<audio>` / `<video>` elements
- Dispatches `CustomEvent("hf-sound", { detail: { muted }, bubbles: true, composed: true })` on every toggle
- Browser-checks the actual iframe media state after changes; every composition `<video>` / `<audio>` should report `muted: true` after clicking the nav mute button
The parent audio player gates on the `hf-sound` event:
Wrapper-owned `new Audio(...)` objects are not attached to the DOM, so the parent audio player must mirror the `hf-sound` event onto each clip:
```js
var muted = false;
@@ -358,6 +421,9 @@ var slideshow = document.querySelector("hyperframes-slideshow");
if (slideshow) {
slideshow.addEventListener("hf-sound", function (e) {
muted = e.detail && e.detail.muted === true;
Object.keys(clips).forEach(function (name) {
clips[name].muted = muted;
});
});
}
// In message handler:
@@ -424,7 +490,7 @@ Do NOT add a mute button inside the composition. The `#sfx-mute` coral button pa
function unlock() {
if (unlocked) return;
unlocked = true;
// Prime each clip: play muted then immediately pause/reset.
// Prime wrapper-owned SFX clips: play muted then immediately pause/reset.
// This moves the clip into the "allowed" state so later plays are instant.
Object.keys(clips).forEach(function (name) {
var el = clips[name];
@@ -587,7 +653,7 @@ if (!renderer) {
| Failure | Symptom | One-line fix |
| ----------------------------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Island not duplicated in wrapper | Slideshow chrome never renders; no slide counter, no prev/next | Copy the `<script type="application/hyperframes-slideshow+json">` block verbatim into the `<hyperframes-slideshow>` element in demo.html |
| Audio in the iframe | All SFX silent | Move Audio elements and unlock logic to demo.html; post `{type:'hf-sfx',name}` from index.html |
| Wrapper SFX in the iframe | Click/transition sounds silent | Move SFX Audio elements and unlock logic to demo.html; post `{type:'hf-sfx',name}` from index.html |
| No self-clock in composition | All scene frames stacked / wrong slide visible at load | Add the root GSAP timeline (`window.__timelines["root"]`) and the `onUpdate` visibility controller as shown in Section 3 |
| Content opacity:0 with no engine | Blank slides — `[data-anim]` elements invisible at rest | Call `updateVisibility(0)` synchronously after defining the controller so the first slide is shown immediately |
| Keydown bound to the element without focus | ArrowLeft/Right dead | Add `tabindex="0"` to `<hyperframes-slideshow>` so it can receive keyboard focus |