From 318724f5f4b7293de19f610f309dec2748942ed8 Mon Sep 17 00:00:00 2001 From: WaterrrForever Date: Fri, 6 Nov 2026 16:16:21 +0000 Subject: [PATCH] feat(studio): show the agent's storyboard in a Storyboard tab beside the preview (#7284) * feat(studio): show the agent's storyboard in a Storyboard tab beside the preview The board an agent writes per the storyboard recipe (boards/storyboard.html, or a root storyboard.html) now shows in Studio the way HyperFrames Desktop shows it: a Preview | Storyboard strip over the preview panel, only while the project has a board. Studio reads the page's class="frame" boxes into a board of stills, with acts, titles, times and words, and one frame at a time with arrow keys. A page with no frames shows as written in a script-less iframe. A board written while Studio is open brings its tab to the front, as Desktop does at the agent's stop. A rewrite of the page, a still beside it or STORYBOARD.md reloads it in place, and a dot marks a rewrite seen from the preview. "Show in preview" goes back to the film at the frame's start. Notes, drawings and the go belong to Desktop's agent, so the board's bar carries the desktop-app button. Its funnel events now say which button was pressed (placement: header or storyboard). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * refactor(studio): split the storyboard view and its frame reader into smaller parts Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * docs(skills): the agent's storyboard stop points at the Storyboard tab, not the file With the board shown beside the preview in Studio as in Desktop, the recipe's stop and style-sampling's stop 1 tell the person to look in the Storyboard tab, and to start a preview and give its link when none is running, instead of handing over boards/storyboard.html to open from disk. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * fix(studio): the storyboard listens on Studio's own file-change channel, and its notes are trimmed The board opened its own /api/events stream, which the Vite dev server behind the runtime smoke does not serve. It now subscribes through subscribeFileChanges, the one channel picker the external-change coordinator already uses (test adapter, Vite HMR, then SSE), so dev and built Studio deliver the same changes to both. The new files' comments are cut to the package's share, and the recipe's Storyboard-tab line moves to the paragraph that names the board, leaving the stop step as it was. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * fix(studio): the board hears file changes through the coordinator, not a second subscription The external-change coordinator stays the one subscriber to the file-change transport (HMR in dev, SSE when built), as externalFileChangeOwnership asserts. It announces each decoded change through fileChangeFanout, and the storyboard listens there. Mounting EditorShell alone, as its tests do, no longer opens an EventSource, and the coordinator file stays under the line cap. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * feat(studio): the storyboard's Open in desktop app asks the app to open on the board, and its card says why - The storyboard bar's button posts `doc: "storyboard"` with its page id. The preview server passes only that value on, and `openInDesktop` writes it into `.hyperframes/desktop-cta.json` beside the page id, so the app can open the project on its Storyboard tab with Build in sight (hyperframes-internal reads it). - The guide card opened from the storyboard says "Approve this storyboard in the desktop app" and "Comment on a frame, draw on it, then press Build", and its last step says the app opens on this storyboard. The header's card is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * feat(studio): approve the storyboard in Studio; comments on frames stay in the desktop app A Studio user without the desktop app had no way to approve the board from the tab, and the bar said approval was in the desktop app, so the tab read as a dead end. Studio talks to the person's own agent through their chat, so the bar's Approve copies "The storyboard is approved. Build the video from it." for them to paste to it, and says so in a toast (or spells the words out when the clipboard refuses). Commenting and drawing on frames stay in the desktop app. The storyboard recipe tells the agent those pasted words are the person's go. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * fix(studio): the storyboard's desktop card offers commenting, since approving now happens in Studio Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * fix(studio): hand-off intent before dispatch, steady film under the board, stills outside the board folder - openInDesktop writes desktop-cta.json before sending the folder, so a running app reads the storyboard ask on arrival; a plain or header open drops an earlier storyboard ask; a failed open puts the earlier record back. - PreviewStage keeps the film at one place in the tree with or without a board, so a board appearing never remounts it; the automatic switch goes through the same pause as a click, and a play while the board is up is paused. - The board reloads when a still it draws changes, including a root board's stills/ and a boards/ page's ../assets. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * chore(cli): fit the Storyboard tab in the CLI's size budget; reword a record helper's doc The CLI tarball carries Studio's build, which now holds the Storyboard tab; merged with main it measures 7,982,002 bytes packed, 7 bytes past the 1% tolerance. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy * fix(studio): hold the player while the storyboard covers the film A pause after the fact let J's reverse loop schedule its next frame, so the hidden film kept seeking. The stage now takes the player's own hold while the board is up (and only when no one else holds it), so J, Space and playback requests are refused before they start; Preview releases it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HebQXPiLcYKQfvPWB3Fgxy --------- Co-authored-by: Claude Opus 5.5 --- packages/cli/src/server/desktopRoutes.test.ts | 22 +- packages/cli/src/server/desktopRoutes.ts | 13 +- packages/cli/src/utils/desktopApp.test.ts | 78 +++++ packages/cli/src/utils/desktopApp.ts | 62 +++- packages/cli/src/utils/projectRecords.ts | 23 +- .../src/components/DesktopGuideCard.tsx | 25 +- .../studio/src/components/EditorShell.tsx | 17 +- .../OpenInDesktopButton.dom.test.tsx | 72 +++- .../src/components/OpenInDesktopButton.tsx | 45 ++- .../ApproveBoardButton.dom.test.tsx | 48 +++ .../storyboard/ApproveBoardButton.tsx | 33 ++ .../storyboard/PreviewStage.dom.test.tsx | 114 +++++++ .../PreviewStage.player.dom.test.tsx | 72 ++++ .../components/storyboard/PreviewStage.tsx | 93 +++++ .../storyboard/StoryboardBoardView.tsx | 322 ++++++++++++++++++ .../storyboard/storyboardBoard.test.ts | 183 ++++++++++ .../components/storyboard/storyboardBoard.ts | 148 ++++++++ .../storyboard/useStoryboardBoard.ts | 99 ++++++ .../studio/src/hooks/fileChangeFanout.test.ts | 20 ++ packages/studio/src/hooks/fileChangeFanout.ts | 13 + .../hooks/useExternalFileChangeCoordinator.ts | 7 +- scripts/package-size-budgets.json | 2 +- skills-manifest.json | 4 +- skills/hyperframes/references/storyboard.md | 2 +- skills/style-sampling/references/remix.md | 2 +- 25 files changed, 1453 insertions(+), 66 deletions(-) create mode 100644 packages/studio/src/components/storyboard/ApproveBoardButton.dom.test.tsx create mode 100644 packages/studio/src/components/storyboard/ApproveBoardButton.tsx create mode 100644 packages/studio/src/components/storyboard/PreviewStage.dom.test.tsx create mode 100644 packages/studio/src/components/storyboard/PreviewStage.player.dom.test.tsx create mode 100644 packages/studio/src/components/storyboard/PreviewStage.tsx create mode 100644 packages/studio/src/components/storyboard/StoryboardBoardView.tsx create mode 100644 packages/studio/src/components/storyboard/storyboardBoard.test.ts create mode 100644 packages/studio/src/components/storyboard/storyboardBoard.ts create mode 100644 packages/studio/src/components/storyboard/useStoryboardBoard.ts create mode 100644 packages/studio/src/hooks/fileChangeFanout.test.ts create mode 100644 packages/studio/src/hooks/fileChangeFanout.ts diff --git a/packages/cli/src/server/desktopRoutes.test.ts b/packages/cli/src/server/desktopRoutes.test.ts index f77d354a1f..07b8dd9e37 100644 --- a/packages/cli/src/server/desktopRoutes.test.ts +++ b/packages/cli/src/server/desktopRoutes.test.ts @@ -19,7 +19,9 @@ function server( place: { platform?: NodeJS.Platform; installed?: () => boolean } = {}, ) { const app = new Hono(); - const open = vi.fn((_dir: string, _cta?: string): DesktopOpenResult => OPENED); + const open = vi.fn( + (_dir: string, _cta?: string, _doc?: "storyboard"): DesktopOpenResult => OPENED, + ); mountDesktopRoutes(app, "/films/a", { ready, open, ...place }); const post = (headers: Record, body?: unknown) => app.request("/api/open-in-desktop", { @@ -118,7 +120,7 @@ describe("open-in-desktop routes", () => { it("once live, opens this server's project for Studio's own POST", async () => { const { open, post } = server(true); expect(await (await post(STUDIO)).json()).toEqual(OPENED); - expect(open).toHaveBeenCalledWith("/films/a", undefined); + expect(open).toHaveBeenCalledWith("/films/a", undefined, undefined); }); it("passes the press's page id on to the hand-off, and drops anything that is not one", async () => { @@ -128,9 +130,19 @@ describe("open-in-desktop routes", () => { await post(STUDIO, { cta: "../../etc/passwd" }); await post(STUDIO, { cta: 42 }); expect(open.mock.calls).toEqual([ - ["/films/a", id], - ["/films/a", undefined], - ["/films/a", undefined], + ["/films/a", id, undefined], + ["/films/a", undefined, undefined], + ["/films/a", undefined, undefined], + ]); + }); + + it("passes on only the storyboard as the page to open the project on", async () => { + const { open, post } = server(true); + await post(STUDIO, { doc: "storyboard" }); + await post(STUDIO, { doc: "../index.html" }); + expect(open.mock.calls).toEqual([ + ["/films/a", undefined, "storyboard"], + ["/films/a", undefined, undefined], ]); }); }); diff --git a/packages/cli/src/server/desktopRoutes.ts b/packages/cli/src/server/desktopRoutes.ts index d2ec41af91..bd5d12b0dc 100644 --- a/packages/cli/src/server/desktopRoutes.ts +++ b/packages/cli/src/server/desktopRoutes.ts @@ -27,7 +27,8 @@ export function mountDesktopRoutes( projectDir: string, { ready = HANDOFF_READY, - open = (dir: string, cta?: string) => openInDesktop(dir, { cta, src: "browser-editor" }), + open = (dir: string, cta?: string, doc?: "storyboard") => + openInDesktop(dir, { cta, doc, src: "browser-editor" }), platform = process.platform, installed = () => desktopInstalled({ platform }), } = {}, @@ -57,7 +58,13 @@ export function mountDesktopRoutes( downloadUrl, }); const body: unknown = await c.req.json().catch(() => null); - const cta = (body as { cta?: unknown } | null)?.cta; - return c.json(open(projectDir, typeof cta === "string" && CTA_ID.test(cta) ? cta : undefined)); + const { cta, doc } = (body ?? {}) as { cta?: unknown; doc?: unknown }; + return c.json( + open( + projectDir, + typeof cta === "string" && CTA_ID.test(cta) ? cta : undefined, + doc === "storyboard" ? doc : undefined, + ), + ); }); } diff --git a/packages/cli/src/utils/desktopApp.test.ts b/packages/cli/src/utils/desktopApp.test.ts index 10f525f9e7..0ea014e124 100644 --- a/packages/cli/src/utils/desktopApp.test.ts +++ b/packages/cli/src/utils/desktopApp.test.ts @@ -103,6 +103,84 @@ describe("openInDesktop", () => { expect(existsSync(record)).toBe(false); openInDesktop(dir, { ...LIVE, cta: "page-1", open: () => true }); expect(JSON.parse(readFileSync(record, "utf8"))).toMatchObject({ cta: "page-1" }); + expect(JSON.parse(readFileSync(record, "utf8"))).not.toHaveProperty("doc"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("tells the app to open on the storyboard when the press came from the Storyboard tab", () => { + const dir = mkdtempSync(join(tmpdir(), "hf-cta-doc-")); + const record = join(dir, ".hyperframes", "desktop-cta.json"); + try { + writeFileSync(join(dir, "index.html"), ""); + openInDesktop(dir, { ...LIVE, cta: "page-1", doc: "storyboard", open: () => true }); + expect(JSON.parse(readFileSync(record, "utf8"))).toMatchObject({ + cta: "page-1", + doc: "storyboard", + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("has the storyboard ask in place when the app is handed the folder, not after", () => { + const dir = mkdtempSync(join(tmpdir(), "hf-cta-order-")); + const record = join(dir, ".hyperframes", "desktop-cta.json"); + try { + writeFileSync(join(dir, "index.html"), ""); + let seen: unknown = null; + openInDesktop(dir, { + ...LIVE, + cta: "page-1", + doc: "storyboard", + open: () => { + seen = JSON.parse(readFileSync(record, "utf8")); + return true; + }, + }); + expect(seen).toMatchObject({ cta: "page-1", doc: "storyboard" }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("doesn't reuse an earlier storyboard ask on a later header or plain open", () => { + const dir = mkdtempSync(join(tmpdir(), "hf-cta-later-")); + const record = join(dir, ".hyperframes", "desktop-cta.json"); + const read = () => JSON.parse(readFileSync(record, "utf8")); + try { + writeFileSync(join(dir, "index.html"), ""); + openInDesktop(dir, { ...LIVE, cta: "page-1", doc: "storyboard", open: () => true }); + let seen: unknown = null; + openInDesktop(dir, { ...LIVE, cta: "page-2", open: () => ((seen = read()), true) }); + expect(seen).toMatchObject({ cta: "page-2" }); + expect(seen).not.toHaveProperty("doc"); + + openInDesktop(dir, { ...LIVE, cta: "page-1", doc: "storyboard", open: () => true }); + openInDesktop(dir, { ...LIVE, open: () => ((seen = read()), true) }); + expect(seen).toMatchObject({ cta: "page-1" }); + expect(seen).not.toHaveProperty("doc"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("puts the earlier record back when the app didn't take the folder", () => { + const dir = mkdtempSync(join(tmpdir(), "hf-cta-undo-")); + const record = join(dir, ".hyperframes", "desktop-cta.json"); + try { + writeFileSync(join(dir, "index.html"), ""); + openInDesktop(dir, { ...LIVE, cta: "page-1", open: () => true }); + const earlier = readFileSync(record, "utf8"); + openInDesktop(dir, { + ...LIVE, + cta: "page-2", + doc: "storyboard", + open: () => false, + installed: () => true, + }); + expect(readFileSync(record, "utf8")).toBe(earlier); } finally { rmSync(dir, { recursive: true, force: true }); } diff --git a/packages/cli/src/utils/desktopApp.ts b/packages/cli/src/utils/desktopApp.ts index f03a79bce8..c26d71d664 100644 --- a/packages/cli/src/utils/desktopApp.ts +++ b/packages/cli/src/utils/desktopApp.ts @@ -7,7 +7,7 @@ import { addCatchUpNote, markSeen, readSeen, seenPath } from "./appHistory.js"; import { createProjectConfig } from "./projectConfig.js"; import { isCI } from "./attendedTerminal.js"; import { isAgentDriven } from "../telemetry/agent_runtime.js"; -import { writeRecord } from "./projectRecords.js"; +import { readRecord, removeRecord, writeRecord } from "./projectRecords.js"; // The HyperFrames desktop app, then builds named in HYPERFRAMES_DESKTOP_BUNDLE_IDS / _APP_NAMES (comma-separated). const list = (name: string): string[] => @@ -180,10 +180,12 @@ export function openInDesktop( installed = () => desktopInstalled({ platform }), env = process.env, cta, + doc, src = "cli", ...where }: AppLookup & { cta?: string; + doc?: "storyboard"; src?: DownloadSource; ready?: boolean; open?: (bundleId: string, dir: string) => boolean; @@ -198,26 +200,50 @@ export function openInDesktop( reason, downloadUrl: desktopDownloadUrl(platform, src) ?? STUDIO_PAGE, }); - const opened = (app: string): DesktopOpenResult => { - if (cta) writeRecord(dir, DESKTOP_CTA_RECORD, JSON.stringify({ cta, at: Date.now() })); - return { - opened: true, - app, - handedOver: leaveHandoff(dir, agentSession(env)), - }; + const opened = (app: string): DesktopOpenResult => ({ + opened: true, + app, + handedOver: leaveHandoff(dir, agentSession(env)), + }); + const dispatch = (): DesktopOpenResult => { + if (platform === "darwin") { + const bundleId = bundleIds().find((id) => open(id, dir)); + if (!bundleId) return notOpened(installed() ? "open-failed" : "not-installed"); + return opened(bundleId === bundleIds()[0] ? DEFAULT_APP : bundleId); + } + const apps = desktopApps({ platform, env, ...where }); + if (!apps) return notOpened("unsupported-platform"); + if (!apps.length) return notOpened("not-installed"); + const app = apps.find(({ executable }) => launch(executable, dir)); + return app ? opened(app.name) : notOpened("open-failed"); }; if (!ready) return notOpened("handoff-unavailable"); markBareIndexAsProject(dir); - if (platform === "darwin") { - const bundleId = bundleIds().find((id) => open(id, dir)); - if (!bundleId) return notOpened(installed() ? "open-failed" : "not-installed"); - return opened(bundleId === bundleIds()[0] ? DEFAULT_APP : bundleId); - } - const apps = desktopApps({ platform, env, ...where }); - if (!apps) return notOpened("unsupported-platform"); - if (!apps.length) return notOpened("not-installed"); - const app = apps.find(({ executable }) => launch(executable, dir)); - return app ? opened(app.name) : notOpened("open-failed"); + const undoIntent = leaveIntent(dir, cta, doc); + const result = dispatch(); + if (!result.opened) undoIntent(); + return result; +} + +/** Written before the folder is sent, since a running app reads it on arrival; the undo puts back what was there. */ +function leaveIntent(dir: string, cta?: string, doc?: "storyboard"): () => void { + const before = readRecord(dir, DESKTOP_CTA_RECORD, 4096); + const earlier = ((): { doc?: unknown } | null => { + try { + return JSON.parse(before); + } catch { + return null; + } + })(); + if (cta || doc) + writeRecord(dir, DESKTOP_CTA_RECORD, JSON.stringify({ cta, at: Date.now(), doc })); + else if (earlier?.doc !== undefined) + writeRecord(dir, DESKTOP_CTA_RECORD, JSON.stringify({ ...earlier, doc: undefined })); + else return () => {}; + return () => { + if (!before) removeRecord(dir, DESKTOP_CTA_RECORD); + else writeRecord(dir, DESKTOP_CTA_RECORD, before); + }; } /** The `hyperframes open` line for a project, relative to where the person is. */ diff --git a/packages/cli/src/utils/projectRecords.ts b/packages/cli/src/utils/projectRecords.ts index 76201a96c3..047516c0d2 100644 --- a/packages/cli/src/utils/projectRecords.ts +++ b/packages/cli/src/utils/projectRecords.ts @@ -1,4 +1,13 @@ -import { closeSync, constants, fstatSync, lstatSync, mkdirSync, openSync, readSync } from "node:fs"; +import { + closeSync, + constants, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readSync, + unlinkSync, +} from "node:fs"; import { join } from "node:path"; import { replaceFileAtomically } from "@hyperframes/core/atomic-file"; @@ -81,3 +90,15 @@ export function writeRecord(projectDir: string, name: string, data: string): boo const dir = recordsDir(projectDir, true); return dir !== null && replaceFile(join(dir, name), data, 0o600); } + +/** Deletes a record that is a plain file; false when there is none to delete. */ +export function removeRecord(projectDir: string, name: string): boolean { + const dir = recordsDir(projectDir); + try { + if (!dir || !lstatSync(join(dir, name)).isFile()) return false; + unlinkSync(join(dir, name)); + return true; + } catch { + return false; + } +} diff --git a/packages/studio/src/components/DesktopGuideCard.tsx b/packages/studio/src/components/DesktopGuideCard.tsx index 313c889057..dea9136420 100644 --- a/packages/studio/src/components/DesktopGuideCard.tsx +++ b/packages/studio/src/components/DesktopGuideCard.tsx @@ -7,10 +7,17 @@ import { Button, cn } from "./ui"; // The desktop-app guide card under OpenInDesktopButton: Framey and three steps (download, install, open), all drawn // from one `Progress`. +export type DesktopCtaPlacement = "header" | "storyboard"; + export type DesktopRoute = { handoff: boolean; downloadUrl: string | null; - context: { os: string; app_installed: boolean; download_available: boolean }; + context: { + os: string; + app_installed: boolean; + download_available: boolean; + placement: DesktopCtaPlacement; + }; }; /** The OS for the download, and what "this …" calls the machine. */ @@ -145,10 +152,14 @@ export function labelOf(p: Progress): string { return isWaiting(p) ? "Finish installing" : "Install the desktop app"; } +const boardFirst = (p: Progress) => p.route.context.placement === "storyboard"; + function subtitleOf(p: Progress): string { if (p.failed) return "Almost there: the last step needs another try."; if (!isInstalled(p)) - return "Watch it, click a frame, tell Framey what to change. Three steps, about two minutes."; + return boardFirst(p) + ? "Comment on a frame, draw on it, then press Build. Three steps, about two minutes." + : "Watch it, click a frame, tell Framey what to change. Three steps, about two minutes."; return p.route.handoff ? "Everything is ready. One press and you're in." : "Almost there: one last step."; @@ -252,7 +263,11 @@ function OpenStep({ progress, projectDir, onOpen }: CardProps) { ); return ( -

Same project, same files, in the desktop app.

+

+ {boardFirst(progress) + ? "Same project, same files. It opens on this storyboard." + : "Same project, same files, in the desktop app."} +

+ ); +} diff --git a/packages/studio/src/components/storyboard/PreviewStage.dom.test.tsx b/packages/studio/src/components/storyboard/PreviewStage.dom.test.tsx new file mode 100644 index 0000000000..9a8037b56b --- /dev/null +++ b/packages/studio/src/components/storyboard/PreviewStage.dom.test.tsx @@ -0,0 +1,114 @@ +// @vitest-environment happy-dom +import React, { act, useEffect } from "react"; +import { createRoot, type Root } from "react-dom/client"; +import { create } from "zustand"; +import { afterEach, beforeEach, expect, it, vi } from "vitest"; +import type { StoryboardBoard } from "./useStoryboardBoard"; + +(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true; + +const pauses = vi.fn(); +const player = create<{ + isPlaying: boolean; + playLocked: boolean; + setPlayLocked: (locked: boolean) => void; + setIsPlaying: (playing: boolean) => void; + requestPlayback: (playing: boolean) => void; + requestSeek: (seconds: number) => void; +}>((set) => ({ + isPlaying: false, + playLocked: false, + setPlayLocked: (playLocked) => set({ playLocked }), + setIsPlaying: (isPlaying) => set({ isPlaying }), + requestPlayback: (playing) => { + if (!playing) pauses(); + }, + requestSeek: () => {}, +})); +vi.mock("../../player", () => ({ usePlayerStore: player })); +vi.mock("../../utils/studioTelemetry", () => ({ trackStudioEvent: vi.fn() })); +vi.mock("./StoryboardBoardView", () => ({ + StoryboardBoardView: () =>
, +})); +let board: StoryboardBoard; +vi.mock("./useStoryboardBoard", () => ({ useStoryboardBoard: () => board })); + +const { PreviewStage } = await import("./PreviewStage"); + +const noBoard: StoryboardBoard = { + path: null, + revision: 0, + read: { state: "loading" }, + appeared: false, +}; +const written: StoryboardBoard = { ...noBoard, path: "boards/storyboard.html", appeared: true }; + +let mounts = 0; +function Film() { + useEffect(() => { + mounts += 1; + }, []); + return
; +} + +let root: Root; +const render = () => + act(async () => + root.render( + + + , + ), + ); +const boardShown = () => document.querySelector('[data-testid="board-view"]') !== null; + +beforeEach(() => { + mounts = 0; + board = noBoard; + player.setState({ isPlaying: false, playLocked: false }); + const host = document.createElement("div"); + document.body.append(host); + root = createRoot(host); +}); + +afterEach(async () => { + await act(async () => root.unmount()); + document.body.innerHTML = ""; + vi.clearAllMocks(); +}); + +it("keeps the film mounted and pauses it when the agent's board first appears", async () => { + player.setState({ isPlaying: true }); + await render(); + board = written; + await render(); + expect(boardShown()).toBe(true); + expect(mounts).toBe(1); + expect(pauses).toHaveBeenCalled(); +}); + +it("keeps the film mounted when the board goes and comes back", async () => { + board = written; + await render(); + board = noBoard; + await render(); + expect(boardShown()).toBe(false); + board = written; + await render(); + expect(mounts).toBe(1); +}); + +it("holds the player while the board covers the film, and leaves another owner's hold alone", async () => { + board = written; + await render(); + expect(player.getState().playLocked).toBe(true); + board = noBoard; + await render(); + expect(player.getState().playLocked).toBe(false); + player.setState({ playLocked: true }); + board = written; + await render(); + board = noBoard; + await render(); + expect(player.getState().playLocked).toBe(true); +}); diff --git a/packages/studio/src/components/storyboard/PreviewStage.player.dom.test.tsx b/packages/studio/src/components/storyboard/PreviewStage.player.dom.test.tsx new file mode 100644 index 0000000000..55bce4c4ae --- /dev/null +++ b/packages/studio/src/components/storyboard/PreviewStage.player.dom.test.tsx @@ -0,0 +1,72 @@ +// @vitest-environment happy-dom +import React, { act } from "react"; +import { createRoot, type Root } from "react-dom/client"; +import { afterEach, expect, it, vi } from "vitest"; +import { + makeAdapterWindow, + makeFakeIframe, + renderTimelinePlayerHarness, + resetPlayerStore, +} from "../../player/hooks/timelinePlayerTestHarness"; +import { usePlayerStore } from "../../player/store/playerStore"; +import type { StoryboardBoard } from "./useStoryboardBoard"; + +(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true; + +vi.mock("../../utils/studioTelemetry", () => ({ trackStudioEvent: vi.fn() })); +vi.mock("./StoryboardBoardView", () => ({ + StoryboardBoardView: ({ onShowInPreview }: { onShowInPreview: (s: number | null) => void }) => ( + + )} +
+
+ + Approve to have your agent build it. Comments on frames are in the desktop app + + + +
+
+ ); +} + +function BoardGrid({ + frames, + still, + onOpen, +}: { + frames: BoardFrame[]; + still: (frame: BoardFrame) => string | null; + onOpen: (index: number) => void; +}) { + let index = 0; + return ( +
+ {groupByAct(frames).map((group, g) => ( +
+ {group.act && ( +

+ {group.act} +

+ )} +
+ {group.frames.map((frame) => { + const at = index++; + const src = still(frame); + return ( + + ); + })} +
+
+ ))} +
+ ); +} + +function OpenFrame({ + frame, + index, + count, + src, + onStep, + onClose, + onShowInPreview, +}: { + frame: BoardFrame; + index: number; + count: number; + src: string | null; + onStep: (to: number) => void; + onClose: () => void; + onShowInPreview: () => void; +}) { + const onKey = useCallback( + (e: KeyboardEvent) => { + const target = e.target as HTMLElement | null; + if (target?.closest("input, textarea, [contenteditable='true']")) return; + if (e.key === "ArrowLeft") onStep(index - 1); + else if (e.key === "ArrowRight") onStep(index + 1); + else if (e.key === "Escape") onClose(); + else return; + e.preventDefault(); + e.stopPropagation(); + }, + [index, onStep, onClose], + ); + useEffect(() => { + window.addEventListener("keydown", onKey, true); + return () => window.removeEventListener("keydown", onKey, true); + }, [onKey]); + + return ( +
+
+ +
+ + + {index + 1} / {count} + + +
+ +
+
+ {src ? ( + + ) : ( + Not sketched yet + )} +
+
+ + {frame.number}: {frame.title || "Untitled"} + {frame.time && ( + {frame.time} + )} + + {frame.words &&

{frame.words}

} +
+
+ ); +} + +/** The agent's page itself, its scripts off: what a board Studio cannot read into frames still looks like. */ +function PageAsWritten({ url }: { url: string }) { + return ( +