From d4e4b07e3cbc101eae5d1a59d858a113ea1cf6f1 Mon Sep 17 00:00:00 2001 From: WaterrrForever Date: Wed, 4 Nov 2026 18:41:56 +0000 Subject: [PATCH] fix(cli): a wedged render trips its 60 s watchdog, one local render at a time, and fonts the cloud can't resolve are flagged (#7168) * fix(producer): a wedged effect scan or screenshot-session open trips the 60 s stall watchdog The sequential streaming watchdog raced only the capture call. Session init (where the fast-capture effect scan runs) and opening the screenshot session for effect frames waited on CDP's protocol timeout instead, which scales with output area up to 30 minutes, so a starved page hung a render for ~25 minutes before the watchdog saw the next frame. The effect scan now gives up when a chunk outlasts its 120 s budget, opening the effect-frame session gets its own no-progress window, and session init gets a 300 s one; each trip retries on a fresh session like any other stall. Co-Authored-By: Claude Opus 5.5 * feat(cli): one local render at a time per machine; a second waits and says for whom Renders started side by side from different sessions starve each other until frames stop landing. `hyperframes render` now takes a machine-wide slot (a lock file in the HyperFrames cache directory, held by pid) after its preflight and holds it for the render or the whole batch. A second render waits and prints the pid and project it is waiting behind; `--no-wait` fails at once instead. A holder whose process is gone is taken over, and HYPERFRAMES_RENDER_LOCK=off restores side-by-side renders. `--workers` is unchanged. Co-Authored-By: Claude Opus 5.5 * fix(lint): flag fonts a cloud render cannot resolve, including the font shorthand Lint read only `font-family`, so `font: 500 16px "PingFang SC", sans-serif` passed while the renderer, which reads the shorthand too, drew the installed Mac font locally and the cloud refused it. The shorthand now counts for font_family_without_font_face (its parser moves to @hyperframes/parsers so core and lint share one), and a new font_face_local_only warning names an @font-face whose only sources are local(): it renders here and falls back in the cloud. The rule's hint and the format reference stop recommending local() for OS fonts. Co-Authored-By: Claude Opus 5.5 * fix(cli): a cloud render that refuses a font says which project files use it The cloud's "fonts that could not be resolved: PingFang SC" named no composition. For a render submitted from a local project, the failure now lists each refused family with the project files that name it, and says how to ship the font so the cloud can use it. Co-Authored-By: Claude Opus 5.5 * chore(cli): keep comment shares and exports within the gates, and split the single-render notes out of executeHeldRender Co-Authored-By: Claude Opus 5.5 * fix(parsers): read a spaced line-height slash and var() before the size in the font shorthand `font: 16px / 1.5 Inter` was read as the family `/ 1.5 Inter`, and `font: var(--weight) 16px Inter` as `16px Inter`, because a var() counted as the size. The parser now follows the shorthand grammar: optional style, variant, weight and width keywords (or var()), then a length, percentage, keyword or calc() size, then a `/ line-height` in any spacing. When no size can be told apart from a var(), it says so: lint skips that declaration instead of reporting a missing font, and the renderer keeps its best guess. Co-Authored-By: Claude Opus 5.5 * fix(cli): say the render slot is one per user account, not per machine The lock lives in the account's HyperFrames cache folder, so renders by two user accounts never queued behind each other while the messages, flag help and docs said "this machine". They now say "per user account" and that HYPERFRAMES_CACHE_DIR set to a shared folder shares the slot across accounts. Co-Authored-By: Claude Opus 5.5 * fix(cli): reclaim a render lock or takeover file left half-written A lock or `.break` file that exists but cannot be parsed (a crash mid-write where hard links are unavailable) was never reclaimed: a waiting render waited forever and --no-wait refused forever. Once such a file is older than any real write (5 s by mtime) it counts as abandoned: the lock goes through the same `.break` takeover as a dead holder's, and a takeover file is removed. Co-Authored-By: Claude Opus 5.5 * fix(producer): opening the capture session counts against the setup stall budget The sequential streaming attempt opened its own capture session (Chrome launch and new page) before the setup watchdog and outside the attempt's cleanup, so a wedged launch still waited for the protocol timeout. Opening now shares the 300 s setup window with session initialisation, a trip fails the attempt like any other sequential stall, and a session that finishes opening after its attempt gave up is closed instead of leaking a browser. Co-Authored-By: Claude Opus 5.5 * chore(parsers): keep the shorthand result type module-private Co-Authored-By: Claude Opus 5.5 * fix(cli): reclaim an unreadable render lock only when its writer is provably gone Where hard links are unavailable the lock is created by an exclusive open and then written, so a contender paused between the two leaves an unreadable lock while it is still alive. The 5 s mtime grace took such a lock over, and both processes then held the slot. Each contender now writes an owner marker under a name only it can take (pid plus random suffix, holding its process identity) before it creates the lock, keeps it while it holds the slot, and removes it otherwise. An unreadable lock or `.break` is reclaimed only when no other contender's marker belongs to a live process; markers of dead processes, or of a reused pid, are deleted. No reclaim is decided by time alone. Co-Authored-By: Claude Opus 5.5 * refactor: split acquireRenderLock and the streaming-stage session stub under the complexity gate acquireRenderLock hands contender setup and a single marked claim to their own functions, and the test stub for createCaptureSession splits its held-open cases from the fake session it builds, so neither exceeds Fallow's cognitive or CRAP thresholds. Behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 * chore(ci): raise the CLI and engine size budgets for the watchdog, lock and font-lint additions Main's CLI unpacked size was already ~0.99% over its budget; the render lock and owner markers push it past the 1% tolerance. The CLI unpacked budget takes the same value as #7169 and #7170 (25,159,107) so the PRs merge cleanly. The engine's packed size grows by about 4.7 KB for the effect-scan and session-open watchdogs; its packed budget moves to 5% above the measured 467,097, as the budget file's headroom rule sets it. Co-Authored-By: Claude Opus 5.5 * fix(parsers): a var() size glued to its line height is the shorthand's size `font: 700 var(--size)/1.1 "Inter"` read as no family at all: the token was tested whole against var(), and the slash after it hid that it was the size, so the renderer injected no face for Inter. A var() followed by `/ line-height` (glued or spaced) can only be the size, and is now read as one. Co-Authored-By: Claude Opus 5.5 * refactor(parsers): split the shorthand size search out of parseFontShorthand Keeps parseFontShorthand under Fallow's cognitive-complexity gate; behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 * fix(cli): render lock never removes another render's takeover, knows a reused pid, works in a shared folder and on Windows - A `.break` takeover file is removed only by the render that wrote it. One whose writer is gone stops the render with the file to delete, instead of being removed on an earlier reading, which could remove a live `.break` written since and let two renders hold the slot. The takeover re-reads the lock under its own `.break` right before replacing it. - The lock record keeps its holder's process start time, so a pid that now belongs to another process (an OOM-killed render in a container with a persisted cache) no longer holds the slot forever. - The `locks` folder is created writable for every account, so accounts sharing HYPERFRAMES_CACHE_DIR take turns instead of the second failing with EACCES; a folder this account cannot write in says how to fix it, and another account's file that cannot be removed is left, not fatal. - Windows: no PowerShell start-time lookup per check (pid liveness only), a file another handle holds (EPERM/EBUSY/EACCES) reads as held, a lock that vanished is retried once instead of in a tight loop, and a wait stops when its signal aborts. The tests end every acquire they start before removing their folder, and the paused contender writes through its open descriptor. Co-Authored-By: Claude Opus 5.5 * chore(ci): raise the lint packed size budget for the font-shorthand rule With main's audio-attribute rule (#7169) merged in, the font-shorthand and local()-only font findings take @hyperframes/lint's packed tarball to 181925 bytes, over the 179338 budget. The new budget is that measurement plus the usual 5% headroom. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- docs/packages/cli.mdx | 10 + docs/reference/lint-rules.mdx | 8 +- packages/cli/src/cloud/fontFailure.test.ts | 63 +++ packages/cli/src/cloud/fontFailure.ts | 88 ++++ packages/cli/src/commands/cloud/render.ts | 15 +- packages/cli/src/commands/render.ts | 11 + packages/cli/src/commands/render/execute.ts | 36 +- packages/cli/src/commands/render/plan.ts | 3 + .../cli/src/commands/render/renderSlot.ts | 47 ++ packages/cli/src/utils/projectLock.ts | 14 +- packages/cli/src/utils/renderLock.test.ts | 420 ++++++++++++++++++ packages/cli/src/utils/renderLock.ts | 154 +++++++ packages/cli/src/utils/renderLockClaim.ts | 102 +++++ packages/cli/src/utils/renderLockOwners.ts | 75 ++++ .../deterministicFonts-shorthand.test.ts | 4 + packages/core/src/fonts/deterministicFonts.ts | 27 +- .../engine/src/services/riskFrames.test.ts | 22 +- packages/engine/src/services/riskFrames.ts | 27 +- packages/lint/src/rules/fonts.test.ts | 59 ++- packages/lint/src/rules/fonts.ts | 64 ++- packages/parsers/src/composition.ts | 1 + packages/parsers/src/fontShorthand.test.ts | 38 ++ packages/parsers/src/fontShorthand.ts | 99 +++++ .../stages/captureStreamingStage.test.ts | 163 ++++++- .../render/stages/captureStreamingStage.ts | 58 ++- scripts/package-size-budgets.json | 4 +- skills-manifest.json | 2 +- skills/hyperframes/references/format.md | 2 +- 28 files changed, 1530 insertions(+), 86 deletions(-) create mode 100644 packages/cli/src/cloud/fontFailure.test.ts create mode 100644 packages/cli/src/cloud/fontFailure.ts create mode 100644 packages/cli/src/commands/render/renderSlot.ts create mode 100644 packages/cli/src/utils/renderLock.test.ts create mode 100644 packages/cli/src/utils/renderLock.ts create mode 100644 packages/cli/src/utils/renderLockClaim.ts create mode 100644 packages/cli/src/utils/renderLockOwners.ts create mode 100644 packages/parsers/src/fontShorthand.test.ts create mode 100644 packages/parsers/src/fontShorthand.ts diff --git a/docs/packages/cli.mdx b/docs/packages/cli.mdx index 58756fc0e3..3fd98a30d8 100644 --- a/docs/packages/cli.mdx +++ b/docs/packages/cli.mdx @@ -1824,6 +1824,7 @@ How it uses the machine, and when it gives up: | `--frames-cache-dir` | `/hyperframes-extract-cache-` | Where extracted source frames cache. `off`, `none`, `false`, or `0` disables it. `doctor` reports the live value. | | `--max-concurrent-renders` | `2` | Concurrent jobs on the Producer server, 1–10 | | `--best-effort` / `--no-best-effort` | on | Finish with capture-readiness warnings, or fail when media is missing or unready | +| `--wait` / `--no-wait` | on | One local render at a time per user account: wait while another runs, or fail at once. Env: `HYPERFRAMES_RENDER_LOCK=off` renders side by side. | | `--strict` / `--strict-all` | off | Fail on lint errors, or on errors and warnings | | `--strict-variables` | off | Fail rather than warn on an undeclared or mistyped variable key | | `--batch-concurrency` / `--batch-fail-fast` | `1` / off | Batch rows at once, and whether the first failure stops the rest | @@ -1831,6 +1832,15 @@ How it uses the machine, and when it gives up: | `--quiet` | off | Less output | | `--skill` | — | Record which authoring workflow started this render, in usage telemetry | +Renders started side by side starve each other of CPU and GPU until frames stop +landing, so a second local or `--docker` render from the same user account waits +for the first and says which render it is waiting for (pid and project). The +slot is a lock file in that account's HyperFrames cache folder, so renders by +different user accounts do not queue behind each other; set +`HYPERFRAMES_CACHE_DIR` to a folder they share to give them one slot. A render +whose process died frees its slot on its own. `--workers` is unaffected: it sets +the parallelism inside the one render that runs. + Low-memory detection reads **host** RAM, not cgroup or container limits, so containerised callers — including `--docker` — should set `PRODUCER_LOW_MEMORY_MODE` explicitly. diff --git a/docs/reference/lint-rules.mdx b/docs/reference/lint-rules.mdx index 4a2506b359..e1ab7d7043 100644 --- a/docs/reference/lint-rules.mdx +++ b/docs/reference/lint-rules.mdx @@ -777,7 +777,13 @@ Source: `packages/lint/src/rules/fonts.ts`. **Fix:** Declare each family with `@font-face` pointing at the captured `.woff2` files, or bundle the font under `capture/assets/fonts/`. -The declaration may use a quoted or unquoted family name. If `font-family` is the last descriptor in the block, its final semicolon is optional. +The declaration may use a quoted or unquoted family name. If `font-family` is the last descriptor in the block, its final semicolon is optional. Families named in a `font` shorthand (`font: 500 16px/1 "PingFang SC", sans-serif`) count the same as `font-family`. + +### `font_face_local_only` + +**Warning.** An `@font-face` names only `local()` sources. A local render draws the font installed on this machine; a cloud render, or another machine without it, falls back to a different font. + +**Fix:** Add a `url()` source pointing at a `.woff2` file inside the project, or use a Google Fonts family. ### `system_font_will_alias` diff --git a/packages/cli/src/cloud/fontFailure.test.ts b/packages/cli/src/cloud/fontFailure.test.ts new file mode 100644 index 0000000000..02629c2639 --- /dev/null +++ b/packages/cli/src/cloud/fontFailure.test.ts @@ -0,0 +1,63 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { describeUnresolvedFontUse, unresolvedFontFamilies } from "./fontFailure.js"; + +let project: string; + +beforeEach(() => { + project = mkdtempSync(join(tmpdir(), "hf-font-failure-")); + mkdirSync(join(project, "compositions")); + mkdirSync(join(project, "node_modules", "pkg"), { recursive: true }); + writeFileSync( + join(project, "index.html"), + '
', + ); + writeFileSync( + join(project, "compositions", "title.html"), + '', + ); + writeFileSync( + join(project, "compositions", "lower.html"), + "", + ); + writeFileSync(join(project, "node_modules", "pkg", "x.css"), 'b { font-family: "PingFang SC"; }'); +}); + +afterEach(() => rmSync(project, { recursive: true, force: true })); + +describe("unresolvedFontFamilies", () => { + it.each([ + ["fonts that could not be resolved: PingFang SC", ["PingFang SC"]], + [ + 'font localization failed (FontFetchError): unresolved font families: "PingFang SC", "Hiragino Sans" (+1 more)', + ["PingFang SC", "Hiragino Sans"], + ], + [ + "[Compiler] Unresolved fonts in fail-closed mode: PingFang SC, Geeza Pro. Distributed renders require all fonts to be resolvable.", + ["PingFang SC", "Geeza Pro"], + ], + ["Chrome crashed while capturing frame 12", []], + ])("reads %s", (message, families) => { + expect(unresolvedFontFamilies(message)).toEqual(families); + }); +}); + +describe("describeUnresolvedFontUse", () => { + it("names the composition file that uses the font the cloud refused", () => { + const said = describeUnresolvedFontUse( + "Render failed: fonts that could not be resolved: PingFang SC, Missing Face", + project, + ); + + expect(said).toContain(`PingFang SC: used in ${join("compositions", "title.html")}`); + expect(said).not.toContain("lower.html"); + expect(said).not.toContain("node_modules"); + expect(said).toContain("Missing Face: not found in the project files"); + }); + + it("adds nothing to a failure that is not about fonts", () => { + expect(describeUnresolvedFontUse("Chrome crashed", project)).toBeUndefined(); + }); +}); diff --git a/packages/cli/src/cloud/fontFailure.ts b/packages/cli/src/cloud/fontFailure.ts new file mode 100644 index 0000000000..7beaaf7fd9 --- /dev/null +++ b/packages/cli/src/cloud/fontFailure.ts @@ -0,0 +1,88 @@ +import { readdirSync, readFileSync } from "node:fs"; +import { extname, join, relative } from "node:path"; + +const UNRESOLVED_FONTS = + /(?:fonts? that could not be resolved|unresolved font famil(?:y|ies)|unresolved fonts[^:\n]*):\s*([^\n]+)/i; +const SKIPPED_DIRS = new Set(["node_modules", "renders", "dist"]); +const MAX_FILES = 2_000; + +/** The families a cloud failure says it could not resolve; empty when it is about something else. */ +export function unresolvedFontFamilies(message: string): string[] { + const listed = UNRESOLVED_FONTS.exec(message)?.[1]; + if (!listed) return []; + const families = listed + .replace(/\(\+\d+ more\)/g, "") + .replace(/\.\s.*$|\.$/, "") + .split(",") + .map((family) => + family + .trim() + .replace(/^["']|["']$/g, "") + .trim(), + ) + .filter(Boolean); + return [...new Set(families)]; +} + +function styleSources(dir: string, found: string[] = []): string[] { + let entries; + try { + entries = readdirSync(dir, { withFileTypes: true }); + } catch { + return found; + } + for (const entry of entries) { + if (found.length >= MAX_FILES) break; + if (entry.name.startsWith(".") || SKIPPED_DIRS.has(entry.name)) continue; + const path = join(dir, entry.name); + if (entry.isDirectory()) styleSources(path, found); + else if ([".html", ".htm", ".css"].includes(extname(entry.name).toLowerCase())) { + found.push(path); + } + } + return found; +} + +const escapeRegExp = (text: string) => text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + +/** Each family with the project files that name it, relative to the project. */ +function filesNamingFonts(projectDir: string, families: string[]): Map { + const named = new Map(); + const patterns = families.map( + (family) => [family, new RegExp(`(^|[^\\w-])${escapeRegExp(family)}(?![\\w-])`, "i")] as const, + ); + for (const path of styleSources(projectDir)) { + let text: string; + try { + text = readFileSync(path, "utf8"); + } catch { + continue; + } + for (const [family, pattern] of patterns) { + if (!pattern.test(text)) continue; + named.set(family, [...(named.get(family) ?? []), relative(projectDir, path)]); + } + } + return named; +} + +/** Which project files use each font a cloud render could not resolve, or undefined when none is named. */ +export function describeUnresolvedFontUse( + failureMessage: string, + projectDir: string, +): string | undefined { + const families = unresolvedFontFamilies(failureMessage); + if (families.length === 0) return undefined; + const named = filesNamingFonts(projectDir, families); + const lines = families.map((family) => { + const files = named.get(family); + return files?.length + ? `${family}: used in ${files.join(", ")}` + : `${family}: not found in the project files`; + }); + return ( + `${lines.join("\n")}\n` + + "Cloud renders cannot use fonts installed on your machine. Ship the font file with an " + + "@font-face url() source, or switch to a Google Fonts family. `hyperframes lint` flags these." + ); +} diff --git a/packages/cli/src/commands/cloud/render.ts b/packages/cli/src/commands/cloud/render.ts index e3932bad11..21348ae160 100644 --- a/packages/cli/src/commands/cloud/render.ts +++ b/packages/cli/src/commands/cloud/render.ts @@ -43,6 +43,7 @@ import { } from "@hyperframes/core"; import { c } from "../../ui/colors.js"; import { errorBox, formatBytes, formatDuration } from "../../ui/format.js"; +import { describeUnresolvedFontUse } from "../../cloud/fontFailure.js"; import { resolveDefaultFpsArg } from "../../utils/renderArgs.js"; import { resolveProject, type ProjectDir } from "../../utils/project.js"; import { readProjectConfig } from "../../utils/projectConfig.js"; @@ -348,7 +349,7 @@ export default defineCommand({ }); if (detail.status === "failed") { - handleFailedRender(detail, asJson); + handleFailedRender(detail, asJson, project.kind === "dir" ? (project.dir ?? ".") : undefined); } if (!detail.video_url) { @@ -1022,14 +1023,22 @@ function formatTickLine(detail: HyperframesRenderDetail, elapsedMs: number): str // Terminal handlers // --------------------------------------------------------------------------- -function handleFailedRender(detail: HyperframesRenderDetail, asJson: boolean): never { +function handleFailedRender( + detail: HyperframesRenderDetail, + asJson: boolean, + projectDir?: string, +): never { if (asJson) { console.log(JSON.stringify(withMeta({ render: detail }), null, 2)); failCommand(); } + const fontUse = + projectDir && detail.failure_message + ? describeUnresolvedFontUse(detail.failure_message, projectDir) + : undefined; errorBox( "Render failed", - detail.failure_message ?? "(no failure_message returned)", + [detail.failure_message ?? "(no failure_message returned)", fontUse].filter(Boolean).join("\n"), `Inspect: hyperframes cloud get ${detail.render_id}`, ); failCommand(); diff --git a/packages/cli/src/commands/render.ts b/packages/cli/src/commands/render.ts index 4dbe1f5726..9bba3e9196 100644 --- a/packages/cli/src/commands/render.ts +++ b/packages/cli/src/commands/render.ts @@ -15,6 +15,7 @@ import { seedProjectAuthoringSkill } from "../utils/projectConfig.js"; import { deliveryFlagArgs } from "../utils/deliveryFlags.js"; import type { CatalogUsage } from "../utils/catalogUsage.js"; import { presentRenderPlan } from "./render/present.js"; +import { acquireRenderSlot } from "./render/renderSlot.js"; import { executeRenderPlan, renderLintContinuationHint, runRenderLint } from "./render/execute.js"; // Test-only seams retained at the command boundary for render behavior tests. export { resolveBrowserGpuForCli, renderLintContinuationHint, runRenderLint }; @@ -319,6 +320,15 @@ export default defineCommand({ "Allow output with structured capture-readiness warnings (default). Use --no-best-effort to fail on missing or unready media.", default: true, }, + wait: { + type: "boolean", + description: + "Wait while another render of this user account runs, then start (default): renders side by " + + "side starve each other. Use --no-wait to fail at once instead. Env: HYPERFRAMES_RENDER_LOCK=off " + + "lets renders run side by side; HYPERFRAMES_CACHE_DIR set to a shared folder shares the slot " + + "across user accounts.", + default: true, + }, strict: { type: "boolean", description: "Fail render on lint errors", @@ -479,6 +489,7 @@ export default defineCommand({ renderDocker, renderLocal, checkResolution: checkRenderResolutionPreflight, + acquireRenderSlot, }, cancellation, ); diff --git a/packages/cli/src/commands/render/execute.ts b/packages/cli/src/commands/render/execute.ts index fb594e1805..a50eca3811 100644 --- a/packages/cli/src/commands/render/execute.ts +++ b/packages/cli/src/commands/render/execute.ts @@ -43,6 +43,7 @@ export interface RenderExecutionDependencies { renderDocker: RenderExecutor; renderLocal: RenderExecutor; checkResolution: ResolutionPreflight; + acquireRenderSlot?: (plan: RenderPlan, signal?: AbortSignal) => Promise<() => void>; } // Exported only through render.ts so command tests can lock the user-facing guidance. @@ -105,6 +106,30 @@ export async function executeRenderPlan( await runResolutionPreflight(plan, dependencies.checkResolution); assertRenderActive(cancellation); + const releaseSlot = await dependencies.acquireRenderSlot?.(plan, cancellation?.signal); + try { + assertRenderActive(cancellation); + await executeHeldRender(plan, browserPath, variables, dependencies, cancellation, { + batchModule, + preparedBatch, + }); + } finally { + releaseSlot?.(); + } +} + +async function executeHeldRender( + plan: RenderPlan, + browserPath: string | undefined, + variables: RenderOptions["variables"], + dependencies: RenderExecutionDependencies, + cancellation: RenderCancellationScope | undefined, + batch: { + batchModule?: typeof import("../batchRender.js"); + preparedBatch?: import("../batchRender.js").PreparedBatchRender; + }, +): Promise { + const { batchModule, preparedBatch } = batch; if (plan.batchPath && batchModule && preparedBatch) { await executeBatchRender( plan, @@ -117,6 +142,13 @@ export async function executeRenderPlan( return; } + reportSingleRenderNotes(plan, variables); + const options = renderOptionsFromPlan(plan, browserPath, variables); + const execute = plan.useDocker ? dependencies.renderDocker : dependencies.renderLocal; + await execute(plan.project.dir, plan.outputPath, options, cancellation); +} + +function reportSingleRenderNotes(plan: RenderPlan, variables: RenderOptions["variables"]): void { if (variables && Object.keys(variables).length > 0) { const issues = validateVariablesAgainstProject(plan.renderTarget, variables); reportVariableIssues(issues, { strict: plan.strictVariables, quiet: plan.effectiveQuiet }); @@ -127,10 +159,6 @@ export async function executeRenderPlan( c.dim(`ProRes ${plan.proresProfile} has no alpha channel: transparent areas render black.`), ); } - - const options = renderOptionsFromPlan(plan, browserPath, variables); - const execute = plan.useDocker ? dependencies.renderDocker : dependencies.renderLocal; - await execute(plan.project.dir, plan.outputPath, options, cancellation); } /** diff --git a/packages/cli/src/commands/render/plan.ts b/packages/cli/src/commands/render/plan.ts index 5247008e33..04402e6069 100644 --- a/packages/cli/src/commands/render/plan.ts +++ b/packages/cli/src/commands/render/plan.ts @@ -99,6 +99,7 @@ export interface RenderCommandArgs { quiet?: boolean; debug?: boolean; "best-effort"?: boolean; + wait?: boolean; strict?: boolean; "strict-all"?: boolean; "lint-verbose"?: boolean; @@ -164,6 +165,7 @@ export interface RenderPlan { /** Segmented capture: keep the segment directory after success. */ keepSegments: boolean; bestEffort: boolean; + waitForRenderSlot: boolean; batchJson: boolean; effectiveQuiet: boolean; strictAll: boolean; @@ -637,6 +639,7 @@ export function createRenderPlan(args: RenderCommandArgs, now = new Date()): Ren resumeSegments: args.resume ?? false, keepSegments: args["keep-segments"] ?? false, bestEffort: args["best-effort"] ?? true, + waitForRenderSlot: args.wait ?? true, batchJson, effectiveQuiet: quiet || batchJson, strictAll: args["strict-all"] ?? false, diff --git a/packages/cli/src/commands/render/renderSlot.ts b/packages/cli/src/commands/render/renderSlot.ts new file mode 100644 index 0000000000..c06d84e17d --- /dev/null +++ b/packages/cli/src/commands/render/renderSlot.ts @@ -0,0 +1,47 @@ +import { c } from "../../ui/colors.js"; +import { errorBox } from "../../ui/format.js"; +import { failCommand } from "../../utils/commandResult.js"; +import { + acquireRenderLock, + describeRenderLockHolder, + RenderLockBusy, + type RenderLockOptions, +} from "../../utils/renderLock.js"; +import { RenderLockStaleBreak, RenderLockUnwritable } from "../../utils/renderLockClaim.js"; +import type { RenderPlan } from "./plan.js"; + +/** One local render at a time per user account: a second waits for the slot, or fails under --no-wait. */ +export async function acquireRenderSlot( + plan: Pick, + signal?: AbortSignal, + lock: Partial = {}, +): Promise<() => void> { + try { + return await acquireRenderLock({ + project: plan.project.dir, + wait: plan.waitForRenderSlot, + signal, + onWait: (holder, path) => + console.error( + c.warn( + ` Waiting for another render to finish (${describeRenderLockHolder(holder, Date.now())}). ` + + "Renders run one at a time per user account; pass --no-wait to stop instead.", + ) + c.dim(`\n Lock: ${path}`), + ), + ...lock, + }); + } catch (error) { + if (error instanceof RenderLockStaleBreak || error instanceof RenderLockUnwritable) { + errorBox("The render slot cannot be taken", error.message); + return failCommand(1, error); + } + if (!(error instanceof RenderLockBusy)) throw error; + errorBox( + "Another render is running for this user account", + describeRenderLockHolder(error.holder, Date.now()), + "Rerun without --no-wait to start when it finishes, or set HYPERFRAMES_RENDER_LOCK=off to render side by side. " + + "Set HYPERFRAMES_CACHE_DIR to a shared folder to share the slot across user accounts.", + ); + return failCommand(1, error); + } +} diff --git a/packages/cli/src/utils/projectLock.ts b/packages/cli/src/utils/projectLock.ts index b310a3c506..7067e74f89 100644 --- a/packages/cli/src/utils/projectLock.ts +++ b/packages/cli/src/utils/projectLock.ts @@ -6,7 +6,7 @@ import { readToolContext, type ToolContext } from "./toolContext.js"; export class ProjectLockTimeout extends Error {} -interface Holder { +export interface Holder { pid: number; command: string; startedAt: number; @@ -17,15 +17,15 @@ const POLL_MS = 250; export const projectLockPath = (projectDir: string) => join(projectDir, ".hyperframes", "project.lock"); -function readHolder(path: string): Holder | null { +export function readHolder(path: string): T | null { try { - return JSON.parse(readFileSync(path, "utf8")) as Holder; + return JSON.parse(readFileSync(path, "utf8")) as T; } catch { return null; } } -function alive(pid: number): boolean { +export function alive(pid: number): boolean { try { process.kill(pid, 0); return true; @@ -103,6 +103,10 @@ export async function withProjectLock( try { return await run(); } finally { - if (sameHolder(readHolder(path), holder)) unlinkSync(path); + release(path, holder); } } + +export function release(path: string, holder: Holder): void { + if (sameHolder(readHolder(path), holder)) unlinkSync(path); +} diff --git a/packages/cli/src/utils/renderLock.test.ts b/packages/cli/src/utils/renderLock.test.ts new file mode 100644 index 0000000000..e31813d587 --- /dev/null +++ b/packages/cli/src/utils/renderLock.test.ts @@ -0,0 +1,420 @@ +import { spawnSync } from "node:child_process"; +import { + chmodSync, + closeSync, + existsSync, + mkdirSync, + mkdtempSync, + openSync, + readdirSync, + readFileSync, + rmSync, + statSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { acquireRenderLock, RenderLockBusy, type RenderLockOptions } from "./renderLock.js"; +import { type RenderLockHolder } from "./renderLockClaim.js"; +import { startToken } from "./renderLockOwners.js"; +import { RenderLockStaleBreak } from "./renderLockClaim.js"; +import type { Holder } from "./projectLock.js"; +import { acquireRenderSlot } from "../commands/render/renderSlot.js"; + +let dir: string; +let path: string; + +beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), "hf-render-lock-")); + path = join(dir, "locks", "render.lock"); +}); + +const stillWaiting: Array<{ stop: AbortController; settled: Promise }> = []; + +/** An acquire that every test ends before its folder is removed, so no loop outlives the test. */ +function startAcquire(options: Omit): Promise<() => void> { + const stop = new AbortController(); + const acquiring = acquireRenderLock({ ...options, signal: stop.signal }); + stillWaiting.push({ stop, settled: acquiring.catch(() => undefined) }); + return acquiring; +} + +afterEach(async () => { + for (const { stop } of stillWaiting) stop.abort(new Error("test over")); + await Promise.all(stillWaiting.splice(0).map(({ settled }) => settled)); + vi.useRealTimers(); + vi.restoreAllMocks(); + rmSync(dir, { recursive: true, force: true }); +}); + +const deadPid = () => spawnSync(process.execPath, ["-e", ""]).pid; + +describe("acquireRenderLock", () => { + it("refuses a second render under --no-wait, naming the render that holds the slot", async () => { + const release = await acquireRenderLock({ project: "/work/brand", wait: false, path, env: {} }); + + const second = acquireRenderLock({ project: "/work/crime", wait: false, path, env: {} }); + + await expect(second).rejects.toBeInstanceOf(RenderLockBusy); + await expect(second).rejects.toThrow(`pid ${process.pid}, project /work/brand`); + release(); + expect(existsSync(path)).toBe(false); + }); + + it("waits while another render holds the slot and starts once it is released", async () => { + vi.useFakeTimers(); + const release = await acquireRenderLock({ project: "/work/brand", wait: false, path, env: {} }); + const waits: Array = []; + let started = false; + const second = startAcquire({ + project: "/work/crime", + wait: true, + path, + env: {}, + onWait: (holder) => waits.push(holder), + }).then((releaseSecond) => { + started = true; + return releaseSecond; + }); + + await vi.advanceTimersByTimeAsync(10_000); + expect(started).toBe(false); + expect(waits.map((holder) => holder?.project)).toEqual(["/work/brand"]); + + release(); + await vi.advanceTimersByTimeAsync(1_000); + expect(started).toBe(true); + (await second)(); + }); + + it("takes over a slot whose render process is gone", async () => { + const stale: RenderLockHolder = { + pid: deadPid(), + command: "render", + startedAt: 1, + project: "/work/killed", + }; + await acquireRenderLock({ project: "/work/seed", wait: false, path, env: {} }).then((r) => r()); + writeFileSync(path, JSON.stringify(stale)); + + const release = await acquireRenderLock({ project: "/work/next", wait: false, path, env: {} }); + + release(); + }); + + describe("a takeover file (.break)", () => { + const record = (pid: number, project: string): RenderLockHolder => ({ + pid, + command: "render", + startedAt: 1, + project, + identity: startToken(pid), + }); + const seed = (breaker: string) => { + mkdirSync(join(dir, "locks"), { recursive: true }); + writeFileSync(path, JSON.stringify(record(deadPid(), "/work/killed"))); + writeFileSync(`${path}.break`, breaker); + }; + + it.each([ + ["readable", () => JSON.stringify(record(deadPid(), "/work/breaker"))], + ["half-written", () => '{"pid":'], + ])( + "left %s by a render that died is never deleted on an earlier reading; the render says what to delete", + async (_kind, content) => { + const left = content(); + seed(left); + for (const wait of [false, true]) { + await expect( + startAcquire({ project: "/work/next", wait, path, env: {} }), + ).rejects.toBeInstanceOf(RenderLockStaleBreak); + } + expect(readFileSync(`${path}.break`, "utf8")).toBe(left); + }, + ); + + it("that a live render is still writing is left to it, and that render's takeover holds alone", async () => { + vi.useFakeTimers({ now: Date.now() }); + mkdirSync(join(dir, "locks"), { recursive: true }); + writeFileSync(path, JSON.stringify(record(deadPid(), "/work/killed"))); + let resume: () => void = () => {}; + let heldBy: string[] = []; + const first = startAcquire({ + project: "/work/one", + wait: false, + path, + env: {}, + beforeReplace: () => new Promise((resolve) => (resume = resolve)), + }).then((release) => ((heldBy = [...heldBy, "one"]), release)); + await vi.waitFor(() => expect(existsSync(`${path}.break`)).toBe(true)); + const breakWhileReplacing = readFileSync(`${path}.break`, "utf8"); + + await expect( + startAcquire({ project: "/work/two", wait: false, path, env: {} }), + ).rejects.toBeInstanceOf(RenderLockBusy); + const second = startAcquire({ project: "/work/two", wait: true, path, env: {} }).then( + (release) => ((heldBy = [...heldBy, "two"]), release), + ); + await vi.advanceTimersByTimeAsync(30_000); + expect(readFileSync(`${path}.break`, "utf8")).toBe(breakWhileReplacing); + expect(heldBy).toEqual([]); + + resume(); + const releaseFirst = await first; + await vi.advanceTimersByTimeAsync(5_000); + expect(heldBy).toEqual(["one"]); + releaseFirst(); + await vi.advanceTimersByTimeAsync(1_000); + expect(heldBy).toEqual(["one", "two"]); + (await second)(); + }); + }); + + it.skipIf(process.platform === "win32")( + "takes over a lock whose pid now belongs to a different process", + async () => { + mkdirSync(join(dir, "locks"), { recursive: true }); + const reused: RenderLockHolder = { + pid: process.pid, + command: "render", + startedAt: 1, + project: "/work/oom-killed", + identity: "linux:1", + }; + writeFileSync(path, JSON.stringify(reused)); + + const release = await acquireRenderLock({ + project: "/work/next", + wait: false, + path, + env: {}, + }); + + expect(JSON.parse(readFileSync(path, "utf8")).project).toBe("/work/next"); + release(); + }, + ); + + describe.skipIf(process.platform === "win32")("a cache folder shared by accounts", () => { + it("creates its locks folder writable for every account", async () => { + const release = await acquireRenderLock({ project: "/work/a", wait: false, path, env: {} }); + expect(statSync(join(dir, "locks")).mode & 0o777).toBe(0o777); + release(); + }); + + it.skipIf(process.getuid?.() === 0)( + "says how to fix a locks folder this account cannot write in", + async () => { + mkdirSync(join(dir, "locks"), { mode: 0o755 }); + chmodSync(join(dir, "locks"), 0o555); + try { + await expect( + acquireRenderLock({ project: "/work/b", wait: false, path, env: {} }), + ).rejects.toThrow(`chmod 777 ${join(dir, "locks")}`); + } finally { + chmodSync(join(dir, "locks"), 0o755); + } + }, + ); + }); + + it("keeps the slot in this user's HyperFrames cache folder, which HYPERFRAMES_CACHE_DIR can share", async () => { + vi.stubEnv("HYPERFRAMES_CACHE_DIR", join(dir, "shared")); + try { + const release = await acquireRenderLock({ project: "/work/brand", wait: false, env: {} }); + expect(existsSync(join(dir, "shared", "locks", "render.lock"))).toBe(true); + release(); + } finally { + vi.unstubAllEnvs(); + } + }); + + describe("a lock file that cannot be read yet", () => { + /** A contender that opens the lock exclusively, then stalls before writing its holder. */ + const pausedAfterOpen = (gate: Promise) => async (file: string, holder: Holder) => { + const fd = openSync(file, "wx"); + await gate; + writeFileSync(fd, JSON.stringify(holder)); + closeSync(fd); + return true; + }; + + it("is not taken from a live contender paused between opening and writing it", async () => { + vi.useFakeTimers({ now: Date.now() }); + let resume: () => void = () => {}; + const writerA = startAcquire({ + project: "/work/a", + wait: false, + path, + env: {}, + claim: pausedAfterOpen(new Promise((resolve) => (resume = resolve))), + }); + let heldByB = false; + const writerB = startAcquire({ project: "/work/b", wait: true, path, env: {} }).then( + (release) => { + heldByB = true; + return release; + }, + ); + + await vi.advanceTimersByTimeAsync(60_000); + expect(heldByB).toBe(false); + await expect( + acquireRenderLock({ project: "/work/c", wait: false, path, env: {} }), + ).rejects.toBeInstanceOf(RenderLockBusy); + + resume(); + const releaseA = await writerA; + await vi.advanceTimersByTimeAsync(5_000); + expect(heldByB).toBe(false); + releaseA(); + await vi.advanceTimersByTimeAsync(1_000); + expect(heldByB).toBe(true); + (await writerB)(); + }); + + it("is reclaimed when the contender that opened it died before writing", async () => { + let crash: (reason: Error) => void = () => {}; + const died = new Promise((_, reject) => (crash = reject)); + const writerA = startAcquire({ + project: "/work/a", + wait: false, + path, + env: {}, + pid: deadPid(), + // A dead process holds no handle, so the opened file is closed, as the OS would close it. + claim: async (file) => { + closeSync(openSync(file, "wx")); + await died; + return true; + }, + }); + await vi.waitFor(() => expect(existsSync(path)).toBe(true)); + + const release = await acquireRenderLock({ project: "/work/b", wait: false, path, env: {} }); + + expect(readdirSync(join(dir, "locks")).filter((n) => n.includes(".owner-"))).toHaveLength(1); + release(); + expect(readdirSync(join(dir, "locks"))).toEqual([]); + crash(new Error("A is gone")); + await expect(writerA).rejects.toThrow("A is gone"); + }); + + it("is reclaimed when no contender is left at all", async () => { + mkdirSync(join(dir, "locks"), { recursive: true }); + writeFileSync(path, '{"pid":'); + + const release = await acquireRenderLock({ + project: "/work/next", + wait: false, + path, + env: {}, + }); + + release(); + expect(existsSync(path)).toBe(false); + }); + }); + + it("stops waiting, and stops marking itself a contender, when its signal aborts", async () => { + vi.useFakeTimers(); + const release = await acquireRenderLock({ project: "/work/brand", wait: false, path, env: {} }); + const stop = new AbortController(); + const waiting = acquireRenderLock({ + project: "/work/crime", + wait: true, + path, + env: {}, + signal: stop.signal, + }); + await vi.advanceTimersByTimeAsync(3_000); + + stop.abort(new Error("cancelled")); + await expect(waiting).rejects.toThrow("cancelled"); + release(); + await vi.advanceTimersByTimeAsync(5_000); + expect(readdirSync(join(dir, "locks"))).toEqual([]); + }); + + it("reads a file another handle holds (Windows EPERM/EBUSY) as held, not as a failure", async () => { + const busy = Object.assign(new Error("EBUSY: resource busy or locked"), { code: "EBUSY" }); + + await expect( + acquireRenderLock({ + project: "/work/crime", + wait: false, + path, + env: {}, + claim: () => { + throw busy; + }, + }), + ).rejects.toBeInstanceOf(RenderLockBusy); + expect(readdirSync(join(dir, "locks")).filter((n) => n.includes(".owner-"))).toEqual([]); + }); + + it("never starts PowerShell for a start-time lookup on Windows", () => { + const platform = Object.getOwnPropertyDescriptor(process, "platform")!; + Object.defineProperty(process, "platform", { ...platform, value: "win32" }); + try { + expect(startToken(process.pid)).toBeNull(); + } finally { + Object.defineProperty(process, "platform", platform); + } + }); + + it("lets renders run side by side with HYPERFRAMES_RENDER_LOCK=off", async () => { + const env = { HYPERFRAMES_RENDER_LOCK: "off" }; + await acquireRenderLock({ project: "/work/brand", wait: false, path, env }); + + await expect( + acquireRenderLock({ project: "/work/crime", wait: false, path, env }), + ).resolves.toBeTypeOf("function"); + }); +}); + +describe("acquireRenderSlot", () => { + it("fails the command under --no-wait and tells the person which render holds the slot", async () => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const release = await acquireRenderLock({ project: "/work/brand", wait: false, path, env: {} }); + + await expect( + acquireRenderSlot( + { project: { dir: "/work/crime" } as never, waitForRenderSlot: false }, + undefined, + { path, env: {} }, + ), + ).rejects.toThrow("Command failed"); + + const said = vi.mocked(console.error).mock.calls.flat().join("\n"); + expect(said).toContain("Another render is running for this user account"); + expect(said).toContain("HYPERFRAMES_CACHE_DIR to a shared folder"); + expect(said).toContain(`pid ${process.pid}, project /work/brand`); + release(); + }); + + it("says who it is waiting for while it waits", async () => { + vi.useFakeTimers(); + vi.spyOn(console, "error").mockImplementation(() => {}); + const release = await acquireRenderLock({ project: "/work/brand", wait: false, path, env: {} }); + + const stop = new AbortController(); + const slot = acquireRenderSlot( + { project: { dir: "/work/crime" } as never, waitForRenderSlot: true }, + stop.signal, + { path, env: {} }, + ); + stillWaiting.push({ stop, settled: slot.catch(() => undefined) }); + await vi.advanceTimersByTimeAsync(3_000); + + const said = vi.mocked(console.error).mock.calls.flat().join("\n"); + expect(said).toContain( + `Waiting for another render to finish (pid ${process.pid}, project /work/brand`, + ); + expect(said).toContain("one at a time per user account"); + release(); + await vi.advanceTimersByTimeAsync(1_000); + (await slot)(); + }); +}); diff --git a/packages/cli/src/utils/renderLock.ts b/packages/cli/src/utils/renderLock.ts new file mode 100644 index 0000000000..63509a4bdd --- /dev/null +++ b/packages/cli/src/utils/renderLock.ts @@ -0,0 +1,154 @@ +import { existsSync } from "node:fs"; +import { dirname } from "node:path"; +import { hyperframesCacheDir } from "@hyperframes/core/cache-dir"; +import { readHolder, release } from "./projectLock.js"; +import { + claimSlot, + ensureLockDir, + RenderLockUnwritable, + type RenderLockHolder, +} from "./renderLockClaim.js"; +import { + isHeldError, + liveOtherOwners, + removeIfPresent, + startToken, + writeOwnerMarker, +} from "./renderLockOwners.js"; + +export class RenderLockBusy extends Error { + constructor( + readonly holder: RenderLockHolder | null, + readonly path: string, + ) { + super(`Another render is running (${describeRenderLockHolder(holder, Date.now())})`); + this.name = "RenderLockBusy"; + } +} + +export interface RenderLockOptions { + project: string; + /** False: fail at once with {@link RenderLockBusy} instead of waiting for the slot. */ + wait: boolean; + signal?: AbortSignal; + /** Called once per holder this render starts waiting behind. */ + onWait?: (holder: RenderLockHolder | null, path: string) => void; + path?: string; + pollMs?: number; + env?: NodeJS.ProcessEnv; + pid?: number; + claim?: (path: string, holder: RenderLockHolder, marker: string) => boolean | Promise; + /** Runs while this render holds `.break`, right before it re-checks and replaces a stale lock. */ + beforeReplace?: () => void | Promise; +} + +const POLL_MS = 1_000; + +const renderLockPath = () => hyperframesCacheDir("locks", "render.lock"); + +/** `HYPERFRAMES_RENDER_LOCK=off` lets this account's renders run side by side. */ +function renderLockDisabled(env: NodeJS.ProcessEnv = process.env): boolean { + const value = env.HYPERFRAMES_RENDER_LOCK?.trim().toLowerCase(); + return value === "off" || value === "0" || value === "false"; +} + +export function describeRenderLockHolder(holder: RenderLockHolder | null, now: number): string { + if (!holder) return "holder unknown"; + const seconds = Math.max(0, Math.round((now - holder.startedAt) / 1000)); + const running = seconds >= 60 ? `${Math.floor(seconds / 60)}m ${seconds % 60}s` : `${seconds}s`; + return `pid ${holder.pid}, project ${holder.project}, running ${running}`; +} + +function pause(ms: number, signal: AbortSignal | undefined): Promise { + return new Promise((resolve, reject) => { + if (signal?.aborted) return reject(signal.reason); + const onAbort = () => { + clearTimeout(timer); + reject(signal?.reason); + }; + const timer = setTimeout(() => { + signal?.removeEventListener("abort", onAbort); + resolve(); + }, ms); + signal?.addEventListener("abort", onAbort, { once: true }); + }); +} + +const sameWaitTarget = (a: RenderLockHolder | null, b: RenderLockHolder | null) => + a?.pid === b?.pid && a?.startedAt === b?.startedAt; + +/** + * Takes the one render slot of this user account (its HyperFrames cache folder), waiting while a live + * process holds it; a holder whose process is gone is taken over. Returns the release. + */ +export async function acquireRenderLock(options: RenderLockOptions): Promise<() => void> { + if (renderLockDisabled(options.env)) return () => {}; + const contender = prepareContender(options); + let waitingBehind: RenderLockHolder | null | undefined; + let retriedVanished = false; + for (;;) { + options.signal?.throwIfAborted(); + const releaseSlot = await claimOnce(contender); + if (releaseSlot) return releaseSlot; + const { path } = contender; + const current = readHolder(path); + retriedVanished = !current && !existsSync(path) && !retriedVanished; + if (retriedVanished) continue; + if (!options.wait) throw new RenderLockBusy(current, path); + if (waitingBehind === undefined || !sameWaitTarget(waitingBehind, current)) { + waitingBehind = current; + options.onWait?.(current, path); + } + await pause(options.pollMs ?? POLL_MS, options.signal); + } +} + +interface Contender { + path: string; + holder: RenderLockHolder; + owner: { pid: number; identity: string | null }; + claim: NonNullable; +} + +function prepareContender(options: RenderLockOptions): Contender { + const path = options.path ?? renderLockPath(); + ensureLockDir(dirname(path)); + liveOtherOwners(path); + const pid = options.pid ?? process.pid; + const identity = startToken(pid); + return { + path, + holder: { pid, command: "render", startedAt: Date.now(), project: options.project, identity }, + owner: { pid, identity }, + claim: + options.claim ?? + ((lock, holder, marker) => claimSlot(lock, holder, marker, options.beforeReplace)), + }; +} + +const notWritable = (error: unknown) => + process.platform !== "win32" && + ["EACCES", "EPERM"].includes((error as NodeJS.ErrnoException).code ?? ""); + +async function claimOnce({ path, holder, owner, claim }: Contender): Promise<(() => void) | null> { + let marker: string; + try { + marker = writeOwnerMarker(path, owner); + } catch (error) { + throw notWritable(error) ? new RenderLockUnwritable(dirname(path), error) : error; + } + let held = false; + try { + held = await claim(path, holder, marker); + } catch (error) { + if (notWritable(error)) throw new RenderLockUnwritable(dirname(path), error); + if (!isHeldError(error)) throw error; + } finally { + if (!held) removeIfPresent(marker); + } + if (!held) return null; + return () => { + release(path, holder); + removeIfPresent(marker); + }; +} diff --git a/packages/cli/src/utils/renderLockClaim.ts b/packages/cli/src/utils/renderLockClaim.ts new file mode 100644 index 0000000000..3ce94ee5ab --- /dev/null +++ b/packages/cli/src/utils/renderLockClaim.ts @@ -0,0 +1,102 @@ +import { chmodSync, existsSync, mkdirSync, unlinkSync } from "node:fs"; +import { createFileAtomically } from "@hyperframes/core/atomic-file"; +import { readHolder, type Holder } from "./projectLock.js"; +import { liveOtherOwners, removeIfPresent, sameProcess } from "./renderLockOwners.js"; + +export interface RenderLockHolder extends Holder { + project: string; + identity?: string | null; +} + +export class RenderLockStaleBreak extends Error { + constructor(readonly breakPath: string) { + super( + `A render that was reclaiming the render slot stopped part way and left ${breakPath}. ` + + "No render is using it: delete that file, then run the render again.", + ); + this.name = "RenderLockStaleBreak"; + } +} + +export class RenderLockUnwritable extends Error { + constructor( + readonly dir: string, + cause: unknown, + ) { + super( + `This account cannot write to ${dir}, where renders take their turn. If accounts share ` + + `HYPERFRAMES_CACHE_DIR, make the folder writable for all of them (chmod 777 ${dir}); ` + + "otherwise unset HYPERFRAMES_CACHE_DIR.", + { cause }, + ); + this.name = "RenderLockUnwritable"; + } +} + +/** Accounts sharing a cache folder take turns in one `locks` folder, so each of them must write in it. */ +export function ensureLockDir(dir: string): void { + if (mkdirSync(dir, { recursive: true }) !== undefined && process.platform !== "win32") { + chmodSync(dir, 0o777); + } +} + +const holderAlive = (holder: RenderLockHolder) => sameProcess(holder.pid, holder.identity); + +const sameRecord = (a: RenderLockHolder | null, b: RenderLockHolder) => + a?.pid === b.pid && a.startedAt === b.startedAt && a.identity === b.identity; + +function createExclusive(path: string, holder: Holder): boolean { + try { + createFileAtomically(path, JSON.stringify(holder)); + return true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "EEXIST") return false; + throw error; + } +} + +const writerAbsent = (file: string, lockPath: string, marker: string) => + existsSync(file) && !readHolder(file) && liveOtherOwners(lockPath, marker) === 0; + +/** + * Replaces a stale lock under `.break`, which only its writer ever removes: a `.break` whose writer is + * gone stops the render with what to delete, since removing it on an earlier reading could remove a + * live one written since. + */ +async function takeOver( + path: string, + holder: RenderLockHolder, + observed: RenderLockHolder | null, + marker: string, + beforeReplace?: () => void | Promise, +): Promise { + const breakPath = `${path}.break`; + if (!createExclusive(breakPath, holder)) { + const breaker = readHolder(breakPath); + if (breaker ? holderAlive(breaker) : !writerAbsent(breakPath, path, marker)) return false; + throw new RenderLockStaleBreak(breakPath); + } + try { + await beforeReplace?.(); + const stillStale = observed + ? sameRecord(readHolder(path), observed) + : writerAbsent(path, path, marker); + if (stillStale) removeIfPresent(path); + return createExclusive(path, holder); + } finally { + if (sameRecord(readHolder(breakPath), holder)) unlinkSync(breakPath); + } +} + +/** Creates the lock, or replaces one whose holder is provably gone. */ +export async function claimSlot( + path: string, + holder: RenderLockHolder, + marker: string, + beforeReplace?: () => void | Promise, +): Promise { + if (createExclusive(path, holder)) return true; + const current = readHolder(path); + if (current ? holderAlive(current) : !writerAbsent(path, path, marker)) return false; + return takeOver(path, holder, current, marker, beforeReplace); +} diff --git a/packages/cli/src/utils/renderLockOwners.ts b/packages/cli/src/utils/renderLockOwners.ts new file mode 100644 index 0000000000..d7d17c1c94 --- /dev/null +++ b/packages/cli/src/utils/renderLockOwners.ts @@ -0,0 +1,75 @@ +import { randomBytes } from "node:crypto"; +import { readdirSync, readFileSync, unlinkSync, writeFileSync } from "node:fs"; +import { basename, dirname, join } from "node:path"; +import { processIdentity } from "./orphanCleanup.js"; +import { alive } from "./projectLock.js"; + +interface LockOwner { + pid: number; + identity: string | null; +} + +const markerPrefix = (lockPath: string) => `${basename(lockPath)}.owner-`; + +// Windows refuses to delete or replace a file another handle still holds: that file is in use, not abandoned. +const HELD_CODES = new Set(process.platform === "win32" ? ["EPERM", "EBUSY", "EACCES"] : ["EBUSY"]); +const UNREMOVABLE_CODES = new Set(["ENOENT", "EPERM", "EACCES", "EBUSY"]); + +const errorCode = (error: unknown) => (error as NodeJS.ErrnoException).code ?? ""; + +export function isHeldError(error: unknown): boolean { + return HELD_CODES.has(errorCode(error)); +} + +/** Null on Windows, where the lookup starts PowerShell for every call: liveness falls back to the pid. */ +export function startToken(pid: number): string | null { + return process.platform === "win32" ? null : processIdentity(pid); +} + +/** Marks a contender, under a name only it can take, before it creates the lock or its `.break`. */ +export function writeOwnerMarker(lockPath: string, owner: LockOwner): string { + const name = `${markerPrefix(lockPath)}${owner.pid}-${randomBytes(6).toString("hex")}`; + const marker = join(dirname(lockPath), name); + writeFileSync(marker, JSON.stringify(owner), { flag: "wx" }); + return marker; +} + +export function removeIfPresent(marker: string): void { + try { + unlinkSync(marker); + } catch (error) { + if (!UNREMOVABLE_CODES.has(errorCode(error))) throw error; + } +} + +function recordedIdentity(marker: string): string | null { + try { + const owner = JSON.parse(readFileSync(marker, "utf8")) as Partial; + return typeof owner.identity === "string" ? owner.identity : null; + } catch { + return null; + } +} + +/** The pid is alive and, where its start time is known on both sides, it is the same process. */ +export function sameProcess(pid: number, identity: string | null | undefined): boolean { + if (!Number.isInteger(pid) || pid <= 0 || !alive(pid)) return false; + const current = identity && startToken(pid); + return !identity || !current || current === identity; +} + +/** Counts the other contenders still alive, deleting the markers of processes that are gone. */ +export function liveOtherOwners(lockPath: string, own?: string): number { + const dir = dirname(lockPath); + const prefix = markerPrefix(lockPath); + let live = 0; + for (const name of readdirSync(dir)) { + if (!name.startsWith(prefix)) continue; + const marker = join(dir, name); + if (marker === own) continue; + if (sameProcess(Number(name.slice(prefix.length).split("-")[0]), recordedIdentity(marker))) { + live += 1; + } else removeIfPresent(marker); + } + return live; +} diff --git a/packages/core/src/fonts/deterministicFonts-shorthand.test.ts b/packages/core/src/fonts/deterministicFonts-shorthand.test.ts index 67cd6aea3b..e6419016ae 100644 --- a/packages/core/src/fonts/deterministicFonts-shorthand.test.ts +++ b/packages/core/src/fonts/deterministicFonts-shorthand.test.ts @@ -38,6 +38,10 @@ describe("a bundled family named only in the font shorthand loads", () => { ["a keyword size", "p { font: bold large Inter; }"], ["a calc() size", "p { font: calc(10px + 2vw) Inter; }"], ["a nested calc() size", 'p { font: calc(var(--size) * 1px)/1 "Inter"; }'], + ["a spaced line-height slash", "p { font: 24px / 1.5 Inter; }"], + ["a custom-property weight before the size", "p { font: var(--weight) 24px Inter; }"], + ["a custom-property size", "p { font: var(--size) Inter; }"], + ["a custom-property size with a line-height", 'p { font: 700 var(--size)/1.1 "Inter"; }'], ])("%s", async (_case, css) => { expect(await inject(css)).toMatch(/@font-face\s*\{[^}]*font-family:\s*"Inter"/i); }); diff --git a/packages/core/src/fonts/deterministicFonts.ts b/packages/core/src/fonts/deterministicFonts.ts index c3dd506f2c..3202c90f5c 100644 --- a/packages/core/src/fonts/deterministicFonts.ts +++ b/packages/core/src/fonts/deterministicFonts.ts @@ -11,6 +11,7 @@ import { import { tmpdir } from "node:os"; import { join } from "node:path"; +import { fontShorthandFamilies } from "@hyperframes/parsers/composition"; import { FONT_ALIAS_MAP, resolveAliasDisplayName } from "./aliases.js"; import { extractSfntFace } from "./sfntNames.js"; import { locateSystemFontVariants, SYSTEM_FONT_SIZE_LIMIT } from "./systemFontLocator.js"; @@ -265,32 +266,6 @@ function collectCssCustomProperties(css: string, customProperties: Map { - if (char === "(") depth += 1; - const inside = depth > 0; - if (char === ")") depth -= 1; - return inside ? "_" : char; - }); -} - -function fontShorthandFamilies(value: string): string { - for (const { 0: masked, index } of maskParens(value).matchAll(/\S+/g)) { - const end = index + masked.length; - const token = value.slice(index, end); - const size = token.split("/")[0] ?? ""; - if (FONT_SIZE_KEYWORD.test(size) || FONT_SIZE_TOKEN.test(token)) { - return value.slice(end).trim(); - } - } - return ""; -} - function* iterateCssRootFontFamilyDeclarations( root: postcss.Root, ): Generator { diff --git a/packages/engine/src/services/riskFrames.test.ts b/packages/engine/src/services/riskFrames.test.ts index 5536098528..b569bf2987 100644 --- a/packages/engine/src/services/riskFrames.test.ts +++ b/packages/engine/src/services/riskFrames.test.ts @@ -1,4 +1,4 @@ -import { afterEach, describe, expect, it } from "vitest"; +import { afterEach, describe, expect, it, vi } from "vitest"; import type { Page } from "puppeteer-core"; import { MAX_RISK_FRAME_FRACTION, @@ -120,6 +120,26 @@ describe("scanCssEffectLiveFrames", () => { expect(seeks).toEqual([240]); }); + it("gives up when a chunk never answers, not when the protocol timeout fires", async () => { + vi.useFakeTimers(); + try { + const page = { evaluate: () => new Promise(() => {}) } as unknown as Page; + const scan = scanCssEffectLiveFrames(page, 30, 5_000, 5_000, { + budgetMs: 120_000, + now: () => Date.now(), + }); + const outcome = { settled: false, live: undefined as Set | null | undefined }; + void scan.then((live) => Object.assign(outcome, { settled: true, live })); + + await vi.advanceTimersByTimeAsync(119_000); + expect(outcome.settled).toBe(false); + await vi.advanceTimersByTimeAsync(1_000); + expect(outcome).toEqual({ settled: true, live: null }); + } finally { + vi.useRealTimers(); + } + }); + it("reports the frames scanned after each chunk", async () => { const seen: Array<[number, number]> = []; diff --git a/packages/engine/src/services/riskFrames.ts b/packages/engine/src/services/riskFrames.ts index 90e089b311..4c2fc39814 100644 --- a/packages/engine/src/services/riskFrames.ts +++ b/packages/engine/src/services/riskFrames.ts @@ -61,6 +61,23 @@ async function scanChunk(page: Page, times: number[]): Promise { }, times); } +/** The chunk's frames, or null once `ms` pass first: a wedged page answers only at the protocol timeout. */ +function withinBudget(work: Promise, ms: number): Promise { + return new Promise((resolve, reject) => { + const timer = setTimeout(() => resolve(null), Math.max(0, ms)); + work.then( + (value) => { + clearTimeout(timer); + resolve(value); + }, + (error: unknown) => { + clearTimeout(timer); + reject(error); + }, + ); + }); +} + interface ScanOptions { budgetMs?: number; now?: () => number; @@ -84,7 +101,15 @@ export async function scanCssEffectLiveFrames( const to = Math.min(totalFrames, from + SCAN_CHUNK_FRAMES); const times: number[] = []; for (let f = from; f < to; f++) times.push(quantizeTimeToFrame(f / fps, fps)); - for (const i of await scanChunk(page, times)) live.add(from + i); + const shown = await withinBudget(scanChunk(page, times), budgetMs - (now() - started)); + if (!shown) { + console.error( + `[engine] fast capture: the effect scan got no answer from the page within its ` + + `${budgetMs / 1000} s budget (frames ${from}-${to - 1})`, + ); + return null; + } + for (const i of shown) live.add(from + i); onProgress?.(to, totalFrames); const projectedMs = ((now() - started) / to) * totalFrames; if (to < totalFrames && projectedMs > budgetMs) { diff --git a/packages/lint/src/rules/fonts.test.ts b/packages/lint/src/rules/fonts.test.ts index f57df3bbee..36566adc70 100644 --- a/packages/lint/src/rules/fonts.test.ts +++ b/packages/lint/src/rules/fonts.test.ts @@ -219,6 +219,35 @@ p { FONT-FAMILY: "Real Brand", sans-serif !important; } expect(findings).toHaveLength(0); }); + it("reads the family list of a font shorthand, as the renderer does", async () => { + const findings = await findByCode( + `
`, + "font_family_without_font_face", + ); + expect(findings).toHaveLength(1); + expect(findings[0]).toMatchObject({ line: 2, column: 6 }); + expect(findings[0]!.message).toContain("pingfang sc"); + expect(findings[0]!.message).not.toContain("inter"); + }); + + it.each([ + "16px / 1.5 Inter", + "var(--weight) 16px Inter", + "var(--size) 'Brand Sans'", + "'Brand Sans' 16px", + ])("does not invent a family from a valid or unreadable shorthand: font: %s", async (font) => { + const findings = await findByCode( + `
`, + "font_family_without_font_face", + ); + expect(findings).toHaveLength(0); + }); + it("flags font-family used without @font-face", async () => { const html = `
@@ -269,12 +298,12 @@ p { FONT-FAMILY: "Real Brand", sans-serif !important; } expect(findings).toHaveLength(0); }); - it("fixHint mentions the local() pattern for system fonts", async () => { + it("fixHint says an installed OS font does not reach a cloud render", async () => { const html = `
`; const findings = await findByCode(html, "font_family_without_font_face"); - expect(findings[0]!.fixHint).toContain("local("); + expect(findings[0]!.fixHint).toContain("cloud render has no copy"); }); it("does not flag generic font families", async () => { @@ -490,4 +519,30 @@ p { FONT-FAMILY: "Real Brand", sans-serif !important; } expect(findings[0]!.message).not.toContain("apple-system"); }); }); + + describe("font_face_local_only", () => { + it("warns when an @font-face only names a font installed on this machine", async () => { + const findings = await findByCode( + `
`, + "font_face_local_only", + ); + expect(findings).toHaveLength(1); + expect(findings[0]).toMatchObject({ severity: "warning", line: 2 }); + expect(findings[0]!.message).toContain("pingfang sc"); + }); + + it("stays quiet when the face also ships a font file", async () => { + const findings = await findByCode( + `
`, + "font_face_local_only", + ); + expect(findings).toHaveLength(0); + }); + }); }); diff --git a/packages/lint/src/rules/fonts.ts b/packages/lint/src/rules/fonts.ts index 1404b425af..b7161b8510 100644 --- a/packages/lint/src/rules/fonts.ts +++ b/packages/lint/src/rules/fonts.ts @@ -1,6 +1,10 @@ import postcss from "postcss"; import type { SourceLocation } from "../sourceCoordinates"; -import { FONT_ALIAS_KEYS, resolveAliasDisplayName } from "@hyperframes/parsers/composition"; +import { + FONT_ALIAS_KEYS, + parseFontShorthand, + resolveAliasDisplayName, +} from "@hyperframes/parsers/composition"; import type { LintContext, HyperframeLintFinding } from "../context"; import { isRegistrySourceFile, isRegistryInstalledFile } from "./composition"; @@ -107,14 +111,22 @@ function resolveUsedFontNames( }); } +function usedFamilyList(decl: postcss.Declaration): string { + if (isFontFaceDeclaration(decl)) return ""; + if (decl.prop.toLowerCase() !== "font") return decl.value; + const shorthand = parseFontShorthand(decl.value); + return shorthand.certain ? shorthand.families : ""; +} + +const FONT_FAMILY_PROPS = /^font(?:-family)?$/i; + function extractUsedFontFamilies(styles: Array<{ content: string }>): string[] { const used: string[] = []; const seen = new Set(); const properties = collectFontCustomProperties(styles); for (const style of styles) { - parseFontCss(style.content)?.walkDecls(/^font-family$/i, (decl) => { - if (isFontFaceDeclaration(decl)) return; - for (const name of resolveUsedFontNames(decl.value, properties)) { + parseFontCss(style.content)?.walkDecls(FONT_FAMILY_PROPS, (decl) => { + for (const name of resolveUsedFontNames(usedFamilyList(decl), properties)) { if (!GENERIC_FAMILIES.has(name) && !seen.has(name)) { seen.add(name); used.push(name); @@ -244,13 +256,42 @@ export const fontRules: Array<(ctx: LintContext) => HyperframeLintFinding[]> = [ "Text will fall back to a generic font, producing incorrect typography in the video.", fixHint: "Add @font-face { font-family: '...'; src: url('capture/assets/fonts/...woff2'); } " + - "for each font family, pointing to the captured .woff2 files. For an OS-bundled " + - "system font (e.g. Hiragino Sans, Microsoft YaHei) that has no downloadable file, " + - "use src: local('Exact Font Name') instead. The declaration alone satisfies this " + - "check without needing a font file.", + "for each font family, pointing to the captured .woff2 files, or switch to a Google Fonts " + + "family. An OS font such as PingFang SC or Hiragino Sans renders locally from the " + + "installed copy, but a cloud render has no copy and refuses it.", }); return findings; }, + + ({ styles, rawSource, options, locate }) => { + if (isRegistrySourceFile(options.filePath) || isRegistryInstalledFile(rawSource)) return []; + const findings: HyperframeLintFinding[] = []; + for (const style of styles) { + parseFontCss(style.content)?.walkAtRules(/^font-face$/i, (rule) => { + let family: string | null = null; + let sources = ""; + rule.walkDecls((decl) => { + const prop = decl.prop.toLowerCase(); + if (prop === "font-family") family = normalizeUsedFontName(decl.value); + else if (prop === "src") sources += ` ${decl.value}`; + }); + if (!family || !/\blocal\s*\(/i.test(sources) || /\burl\s*\(/i.test(sources)) return; + findings.push({ + ...locate(style, rule.source?.start?.offset), + code: "font_face_local_only", + severity: "warning", + message: + `@font-face '${family}' only names a font installed on this machine (src: local()). ` + + "A local render uses the installed copy; a cloud render or another machine has none and " + + "falls back to a different font.", + fixHint: + "Add a url() source to the font file (.woff2) inside the project, or use a Google Fonts " + + "family, so every render draws the same glyphs.", + }); + }); + } + return findings; + }, ]; /** Aggregate diagnostics get a location only when exactly one declaration supplies their evidence. */ @@ -263,10 +304,9 @@ function uniqueFontLocation( const properties = collectFontCustomProperties(styles); for (const style of styles) { try { - postcss.parse(style.content).walkDecls(/^font-family$/i, (decl) => { - if (isFontFaceDeclaration(decl)) return; - if (!resolveUsedFontNames(decl.value, properties).some((name) => families.includes(name))) - return; + postcss.parse(style.content).walkDecls(FONT_FAMILY_PROPS, (decl) => { + const named = resolveUsedFontNames(usedFamilyList(decl), properties); + if (!named.some((name) => families.includes(name))) return; locations.push(locate(style, decl.source?.start?.offset)); }); } catch { diff --git a/packages/parsers/src/composition.ts b/packages/parsers/src/composition.ts index 23da71ea16..5d18b5b8fc 100644 --- a/packages/parsers/src/composition.ts +++ b/packages/parsers/src/composition.ts @@ -9,6 +9,7 @@ export { CANONICAL_FONT_DISPLAY_NAMES, resolveAliasDisplayName, } from "./fontAliases.js"; +export { fontShorthandFamilies, parseFontShorthand } from "./fontShorthand.js"; export { decodeUrlPathVariants } from "./utils/urlPath.js"; export { parseCompositionVariables, diff --git a/packages/parsers/src/fontShorthand.test.ts b/packages/parsers/src/fontShorthand.test.ts new file mode 100644 index 0000000000..8b89ec5513 --- /dev/null +++ b/packages/parsers/src/fontShorthand.test.ts @@ -0,0 +1,38 @@ +import { describe, expect, it } from "vitest"; +import { fontShorthandFamilies, parseFontShorthand } from "./fontShorthand.js"; + +describe("parseFontShorthand", () => { + it.each([ + ["16px Inter", "Inter"], + ["16px/1.5 Inter, sans-serif", "Inter, sans-serif"], + ["16px / 1.5 Inter", "Inter"], + ["16px/ 1.5 Inter", "Inter"], + ["16px /1.5 Inter", "Inter"], + ["italic small-caps bold condensed 16px/2 'PingFang SC', serif", "'PingFang SC', serif"], + ['500 16px/1 "PingFang SC", Inter, sans-serif', '"PingFang SC", Inter, sans-serif'], + ["var(--weight) 16px Inter", "Inter"], + ["var(--weight) calc(1rem + 2px) / var(--leading) Inter", "Inter"], + ["oblique 10deg large Inter", "Inter"], + ["16px var(--family)", "var(--family)"], + ['700 var(--size)/1.1 "Inter"', '"Inter"'], + ["var(--size) / 1.2 Inter, sans-serif", "Inter, sans-serif"], + ])("reads %s as family list %s", (value, families) => { + expect(parseFontShorthand(value)).toEqual({ families, certain: true }); + }); + + it.each(["caption", "inherit", "16px", "16px / 1.5"])("finds no family in %s", (value) => { + expect(parseFontShorthand(value)).toEqual({ families: "", certain: true }); + }); + + it.each([ + ["var(--size) Inter", "Inter"], + ["var(--weight) var(--size) Inter", "Inter"], + ["Inter 16px", ""], + ])("is unsure of %s, guessing %s", (value, families) => { + expect(parseFontShorthand(value)).toEqual({ families, certain: false }); + }); + + it("gives the renderer its best guess even when unsure", () => { + expect(fontShorthandFamilies("var(--size) Inter, sans-serif")).toBe("Inter, sans-serif"); + }); +}); diff --git a/packages/parsers/src/fontShorthand.ts b/packages/parsers/src/fontShorthand.ts new file mode 100644 index 0000000000..5b73e713b1 --- /dev/null +++ b/packages/parsers/src/fontShorthand.ts @@ -0,0 +1,99 @@ +const FONT_SIZE_KEYWORD = + /^(?:xx-small|x-small|small|medium|large|x-large|xx-large|xxx-large|smaller|larger)$/i; +const FONT_SIZE_VALUE = + /^(?:\d*\.?\d+(?!(?:deg|grad|rad|turn)$)[a-z%]+|(?:calc|min|max|clamp)\(.*\))$/i; +const PRE_SIZE_KEYWORD = + /^(?:normal|italic|oblique|small-caps|bold|bolder|lighter|(?:ultra-|extra-|semi-)?(?:condensed|expanded)|[+-]?\d*\.?\d+(?:deg|grad|rad|turn)?)$/i; +const CUSTOM_PROPERTY = /^var\(.*\)$/i; +const NO_FAMILY = + /^(?:caption|icon|menu|message-box|small-caption|status-bar|inherit|initial|unset|revert|revert-layer)$/i; + +function maskGroups(value: string): string { + let depth = 0; + let quote: string | null = null; + return value.replace(/[\s\S]/g, (char) => { + if (quote) { + if (char === quote) quote = null; + return "_"; + } + if (char === '"' || char === "'") { + quote = char; + return "_"; + } + if (char === "(") depth += 1; + const inside = depth > 0; + if (char === ")") depth -= 1; + return inside ? "_" : char; + }); +} + +interface Token { + text: string; + masked: string; + start: number; +} + +function tokens(value: string): Token[] { + return [...maskGroups(value).matchAll(/\S+/g)].map(({ 0: masked, index }) => ({ + text: value.slice(index, index + masked.length), + masked, + start: index, + })); +} + +const isSize = (text: string) => FONT_SIZE_KEYWORD.test(text) || FONT_SIZE_VALUE.test(text); + +/** Index of the first family token after the size at `at`, skipping a `/ line-height` in any spacing. */ +function familyStart(list: Token[], at: number): number { + const size = list[at]!; + const slash = size.masked.indexOf("/"); + if (slash >= 0) return slash === size.masked.length - 1 ? at + 2 : at + 1; + const next = list[at + 1]?.masked; + if (next === "/") return at + 3; + return next?.startsWith("/") ? at + 2 : at + 1; +} + +const carriesLineHeight = (list: Token[], at: number) => + list[at]!.masked.includes("/") || (list[at + 1]?.masked.startsWith("/") ?? false); + +interface FontShorthandFamilies { + families: string; + /** False when the shorthand's size cannot be told from its family: `families` is a best guess. */ + certain: boolean; +} + +const headOf = ({ text, masked }: Token) => + masked.includes("/") ? text.slice(0, masked.indexOf("/")) : text; + +function findSize(list: Token[]): { at: number; unsure: boolean } { + let unsure = false; + for (let i = 0; i < list.length; i++) { + const head = headOf(list[i]!); + const isVar = CUSTOM_PROPERTY.test(head); + if (isSize(head) || (isVar && carriesLineHeight(list, i))) return { at: i, unsure }; + if (!isVar && !PRE_SIZE_KEYWORD.test(head)) unsure = true; + } + return { at: -1, unsure }; +} + +function lastLeadingVar(list: Token[]): number { + let last = -1; + for (let i = 0; i < list.length - 1; i++) if (CUSTOM_PROPERTY.test(list[i]!.text)) last = i; + return last; +} + +/** The family list of a `font` shorthand, after its optional style/weight/width, size and line height. */ +export function parseFontShorthand(value: string): FontShorthandFamilies { + const list = tokens(value); + if (list.length === 1 && NO_FAMILY.test(list[0]!.text)) return { families: "", certain: true }; + const rest = (from: number) => (list[from] ? value.slice(list[from].start).trim() : ""); + const size = findSize(list); + if (size.at >= 0) return { families: rest(familyStart(list, size.at)), certain: !size.unsure }; + const lastVar = lastLeadingVar(list); + if (lastVar < 0) return { families: "", certain: !size.unsure }; + return { families: rest(lastVar + 1), certain: false }; +} + +export function fontShorthandFamilies(value: string): string { + return parseFontShorthand(value).families; +} diff --git a/packages/producer/src/services/render/stages/captureStreamingStage.test.ts b/packages/producer/src/services/render/stages/captureStreamingStage.test.ts index 6ed213c5bc..f148f6127c 100644 --- a/packages/producer/src/services/render/stages/captureStreamingStage.test.ts +++ b/packages/producer/src/services/render/stages/captureStreamingStage.test.ts @@ -1,5 +1,5 @@ // fallow-ignore-file code-duplication -import { afterAll, describe, expect, it, mock, setSystemTime } from "bun:test"; +import { afterAll, describe, expect, it, mock, setSystemTime, vi } from "bun:test"; import { mkdtempSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -77,6 +77,11 @@ let batchesRequested: number[][] | null = null; let sessionHasBeforeCapture = false; const deFrame = (index: number) => Buffer.concat([Buffer.from(`de${index}:`), Buffer.alloc(30_000)]); +// The screenshot session for risk frames never finishes opening; the capture session never initialises. +let hangRiskSessionOpen = false; +// The capture session's own open answers only when the test settles it. +let primaryOpen: Promise | null = null; +let hangInitializeSession = false; let failPrepareCaptureSessionForReuse = false; let initializeSessionErrorMessage = "initialize failed"; const browserConsoleBuffer = ["[FrameCapture:ERROR] page.goto failed"]; @@ -149,6 +154,31 @@ function rejectWhenAborted(signal: AbortSignal | undefined): Promise { }); } +function heldOpen(screenshot: boolean): Promise | null { + if (screenshot) return hangRiskSessionOpen ? new Promise(() => {}) : null; + return primaryOpen; +} + +function fakeSession(screenshot: boolean) { + sessionsCreated++; + return { + isScreenshotFallback: screenshot, + onBeforeCapture: sessionHasBeforeCapture ? async () => {} : undefined, + deRiskFrames: screenshot ? undefined : sessionRiskFrames, + isInitialized: false, + browserConsoleBuffer, + options: { captureBeyondViewport: false }, + workerEncodeEnabled: sessionWorkerEncodeEnabled, + captureMode: captureSessionMode, + page: { + evaluate: mock(async (): Promise => []) + .mockResolvedValueOnce([]) + .mockResolvedValueOnce({}), + }, + warnings: [], + }; +} + mock.module("@hyperframes/engine", () => ({ calculateOptimalWorkers: () => 1, convertTransfer: () => {}, @@ -203,23 +233,10 @@ mock.module("@hyperframes/engine", () => ({ _options: unknown, _injector: unknown, config?: { useDrawElement?: boolean }, - ) => ({ - ...(sessionsCreated++ && {}), - isScreenshotFallback: config?.useDrawElement === false, - onBeforeCapture: sessionHasBeforeCapture ? async () => {} : undefined, - deRiskFrames: config?.useDrawElement === false ? undefined : sessionRiskFrames, - isInitialized: false, - browserConsoleBuffer, - options: { captureBeyondViewport: false }, - workerEncodeEnabled: sessionWorkerEncodeEnabled, - captureMode: captureSessionMode, - page: { - evaluate: mock(async (): Promise => []) - .mockResolvedValueOnce([]) - .mockResolvedValueOnce({}), - }, - warnings: [], - }), + ) => { + const screenshot = config?.useDrawElement === false; + return heldOpen(screenshot) ?? fakeSession(screenshot); + }, createFrameReorderBuffer: (startFrame: number) => ({ ...(reorderStarts.push(startFrame) && {}), waitForFrame: async () => {}, @@ -318,6 +335,7 @@ mock.module("@hyperframes/engine", () => ({ getCapturePerfSummary: () => ({}), getFfmpegBinary: () => "ffmpeg", initializeSession: async (session: { isInitialized: boolean }) => { + if (hangInitializeSession) return new Promise(() => {}); if (failInitializeSession) { throw new Error(initializeSessionErrorMessage); } @@ -1024,6 +1042,115 @@ describe("runCaptureStreamingStage", () => { }, ); + describe("a wedge outside the capture call", () => { + /** Runs the stage on a fake clock, a second at a time, until it settles or `maxMs` pass. */ + const settleOnFakeClock = async (run: Promise, maxMs: number) => { + const state = { done: false, error: undefined as unknown, elapsedMs: 0 }; + run.then( + () => (state.done = true), + (error: unknown) => Object.assign(state, { done: true, error }), + ); + const flush = async () => { + for (let i = 0; i < 200; i++) await Promise.resolve(); + }; + await flush(); + while (!state.done && state.elapsedMs < maxMs) { + vi.advanceTimersByTime(1_000); + state.elapsedMs += 1_000; + await flush(); + } + return state; + }; + + it("trips the watchdog when the screenshot session for an effect frame never opens", async () => { + const { runCaptureStreamingStage } = await import("./captureStreamingStage.js"); + sessionWorkerEncodeEnabled = true; + sessionHasBeforeCapture = true; + sessionRiskFrames = new Set([0]); + hangRiskSessionOpen = true; + const prev = process.env.HF_DE_STALL_MS; + delete process.env.HF_DE_STALL_MS; + delete process.env.HF_DE_PARALLEL_STALL_MS; + vi.useFakeTimers(); + try { + const cfg = { forceScreenshot: false, ffmpegStreamingTimeout: 3_600_000 }; + const state = await settleOnFakeClock( + runCaptureStreamingStage({ ...createInput(cfg), totalFrames: 10 }), + 30 * 60_000, + ); + expect(state.done).toBe(true); + expect(state.elapsedMs).toBeLessThanOrEqual(65_000); + expect((state.error as Error).message).toContain( + "no frame progress for 60000ms (stuck at frame 0/10 while opening the screenshot session", + ); + } finally { + vi.useRealTimers(); + hangRiskSessionOpen = false; + sessionWorkerEncodeEnabled = false; + sessionHasBeforeCapture = false; + sessionRiskFrames = undefined; + if (prev !== undefined) process.env.HF_DE_STALL_MS = prev; + } + }); + + it("trips the watchdog when the capture session never finishes opening, and closes it if it opens late", async () => { + const { runCaptureStreamingStage } = await import("./captureStreamingStage.js"); + let openLate: (session: unknown) => void = () => {}; + primaryOpen = new Promise((resolve) => (openLate = resolve)); + closeCaptureSession.mockClear(); + const prev = process.env.HF_DE_STALL_MS; + delete process.env.HF_DE_STALL_MS; + delete process.env.HF_DE_PARALLEL_STALL_MS; + vi.useFakeTimers(); + try { + const cfg = { forceScreenshot: false, ffmpegStreamingTimeout: 3_600_000 }; + const state = await settleOnFakeClock( + runCaptureStreamingStage({ ...createInput(cfg), totalFrames: 10 }), + 30 * 60_000, + ); + expect(state.done).toBe(true); + expect(state.elapsedMs).toBeLessThanOrEqual(305_000); + expect((state.error as Error).message).toContain( + "no frame progress for 300000ms (stuck at frame 0/10 while opening the capture session", + ); + + const late = { browserConsoleBuffer: [] }; + openLate(late); + for (let i = 0; i < 20; i++) await Promise.resolve(); + expect(closeCaptureSession).toHaveBeenCalledWith(late); + } finally { + vi.useRealTimers(); + primaryOpen = null; + if (prev !== undefined) process.env.HF_DE_STALL_MS = prev; + } + }); + + it("trips the watchdog when the capture session never finishes initialising", async () => { + const { runCaptureStreamingStage } = await import("./captureStreamingStage.js"); + hangInitializeSession = true; + const prev = process.env.HF_DE_STALL_MS; + delete process.env.HF_DE_STALL_MS; + delete process.env.HF_DE_PARALLEL_STALL_MS; + vi.useFakeTimers(); + try { + const cfg = { forceScreenshot: false, ffmpegStreamingTimeout: 3_600_000 }; + const state = await settleOnFakeClock( + runCaptureStreamingStage({ ...createInput(cfg), totalFrames: 10 }), + 30 * 60_000, + ); + expect(state.done).toBe(true); + expect(state.elapsedMs).toBeLessThanOrEqual(305_000); + expect((state.error as Error).message).toMatch( + /^\[Render\] Sequential drawElement capture stalled: no frame progress for 300000ms/, + ); + } finally { + vi.useRealTimers(); + hangInitializeSession = false; + if (prev !== undefined) process.env.HF_DE_STALL_MS = prev; + } + }); + }); + it("trips the stall watchdog on the single-worker worker-encode pipeline when capture makes no progress", async () => { hangSequentialUntilStall = true; sessionWorkerEncodeEnabled = true; diff --git a/packages/producer/src/services/render/stages/captureStreamingStage.ts b/packages/producer/src/services/render/stages/captureStreamingStage.ts index fb0882eb3c..68cda7e7e0 100644 --- a/packages/producer/src/services/render/stages/captureStreamingStage.ts +++ b/packages/producer/src/services/render/stages/captureStreamingStage.ts @@ -98,6 +98,8 @@ import type { SdrStreamingCapturePlan } from "../capturePlan.js"; * fallback. */ const DEFAULT_CAPTURE_STALL_MS = 60_000; +/** Opening a session, and the effect scan inside it, before a first frame can land. */ +const SESSION_INIT_STALL_MS = 300_000; const DE_STALL_POLL_MS = 5_000; const STALL_ADVICE = "Chrome stopped answering. If another app is keeping the GPU or CPU busy " + @@ -187,7 +189,17 @@ function captureModeLabel(mode: CaptureSession["captureMode"]): string { return "screenshot"; } -type StalledSession = Pick; +type StalledSession = { captureMode: CaptureSession["captureMode"]; captureStep?: string }; + +const OPENING_CAPTURE_SESSION: StalledSession = { + captureMode: "screenshot", + captureStep: "opening the capture session", +}; + +const OPENING_RISK_SESSION: StalledSession = { + captureMode: "screenshot", + captureStep: "opening the screenshot session for effect frames", +}; class SequentialCaptureStallError extends Error { readonly captureMode: CaptureSession["captureMode"]; @@ -492,7 +504,7 @@ function createDrainFrameGuard(args: { /** Frames drawElement cannot reproduce, and how to capture one by screenshot. */ interface RiskFrameCapture { frames: ReadonlySet; - /** Opens the screenshot session; kept out of the stall budget, a browser launch is slow. */ + /** Opens the screenshot session; it gets a no-progress window of its own, a browser launch is slow. */ open: () => Promise; capture: (frameIndex: number, time: number) => Promise; } @@ -521,7 +533,12 @@ async function runWorkerEncodePipelineLoop( const guardFrame = (item: Pending, buf: Buffer): Promise => item.trusted ? Promise.resolve(buf) : guard(session, item.idx, buf); const captureRisk = async (idx: number): Promise => { - await risk!.open(); + await raceAgainstStall( + risk!.open(), + stallTimeoutMs - quiet.idle(0), + { session: OPENING_RISK_SESSION, frameIndex: idx, totalFrames, stallTimeoutMs }, + abortSignal, + ); quiet.reset(); return raceAgainstStall( risk!.capture(idx, frameTime(idx)), @@ -1003,15 +1020,31 @@ export async function runCaptureStreamingStage( let resumes = 0; for (;;) { const videoInjector = createRenderVideoFrameInjector(); - const session = - probeSession ?? - (await createCaptureSession( + const initStallMs = Math.max(SESSION_INIT_STALL_MS, resolveCaptureStallTimeoutMs()); + const setup = createStallClock(); + const withinInitBudget = (work: Promise, stalled: StalledSession) => + raceAgainstStall( + work, + initStallMs - setup.idle(0), + { session: stalled, frameIndex: resume.from, totalFrames, stallTimeoutMs: initStallMs }, + abortSignal, + ); + const openSession = async (): Promise => { + const opening = createCaptureSession( fileServer.url, framesDir, buildCaptureOptions(), videoInjector, captureCfg, - )); + ); + try { + return await withinInitBudget(opening, OPENING_CAPTURE_SESSION); + } catch (error) { + void opening.then(closeCaptureSession, () => {}); + throw error; + } + }; + const session = probeSession ?? (await openSession()); captureBeyondViewport = session.options.captureBeyondViewport; if (probeSession) { prepareCaptureSessionForReuse(session, framesDir, videoInjector); @@ -1020,6 +1053,8 @@ export async function runCaptureStreamingStage( // The screenshot capture of today: same config with fast capture off. Its frames are not // consecutive, so static-frame reuse is off. let riskSession: CaptureSession | null = null; + // A stalled attempt stops waiting on an open still in flight; that session closes itself. + let attemptClosed = false; const openRiskSession = async (): Promise => { const opened = await createCaptureSession( fileServer.url, @@ -1033,6 +1068,10 @@ export async function runCaptureStreamingStage( enableBrowserPool: false, }, ); + if (attemptClosed) { + await closeCaptureSession(opened); + throw new Error("[Render] Capture attempt ended while its screenshot session opened"); + } riskSession = opened; await initializeSession(opened); return opened; @@ -1041,12 +1080,12 @@ export async function runCaptureStreamingStage( let resumed = false; try { if (!session.isInitialized) { - await initializeSession(session); + await withinInitBudget(initializeSession(session), session); } // Probe-initialized video comps defer verification + canvas injection // until the injector exists (attached by prepareCaptureSessionForReuse // above) — complete it now so drawElement runs VERIFIED on video comps. - await completeDeferredDrawElementInit(session); + await withinInitBudget(completeDeferredDrawElementInit(session), session); assertNotAborted(); lastBrowserConsole = session.browserConsoleBuffer; @@ -1146,6 +1185,7 @@ export async function runCaptureStreamingStage( resumes++; resumed = true; } finally { + attemptClosed = true; // Keep the latest console buffer for success and cleanup-error summaries. lastBrowserConsole = session.browserConsoleBuffer; await closeCaptureSession(session); diff --git a/scripts/package-size-budgets.json b/scripts/package-size-budgets.json index c11b4c855d..e6dfac2ae7 100644 --- a/scripts/package-size-budgets.json +++ b/scripts/package-size-budgets.json @@ -12,7 +12,7 @@ "unpacked": 4268445 }, "@hyperframes/engine": { - "packed": 462420, + "packed": 490452, "unpacked": 1620913 }, "@hyperframes/gcp-cloud-run": { @@ -24,7 +24,7 @@ "unpacked": 3645432 }, "@hyperframes/lint": { - "packed": 179338, + "packed": 191022, "unpacked": 737342 }, "@hyperframes/parsers": { diff --git a/skills-manifest.json b/skills-manifest.json index 3841587cdd..b17fc312f5 100644 --- a/skills-manifest.json +++ b/skills-manifest.json @@ -6,7 +6,7 @@ "files": 2 }, "hyperframes": { - "hash": "c7c2479d8aee134b", + "hash": "ceaef1d1899f6fb7", "files": 77 }, "hyperframes-cli": { diff --git a/skills/hyperframes/references/format.md b/skills/hyperframes/references/format.md index 1a64c00daf..b8c767991d 100644 --- a/skills/hyperframes/references/format.md +++ b/skills/hyperframes/references/format.md @@ -95,7 +95,7 @@ The detail behind the contract in `SKILL.md`. Full references: `npx hyperframes - Bundled, so they resolve by name: Inter, Montserrat, Outfit, Nunito, Oswald, League Gothic, Archivo Black, Space Mono, IBM Plex Mono, JetBrains Mono, EB Garamond, Playfair Display, Source Code Pro, Noto Sans JP, Roboto, Open Sans, Lato, Poppins. Common system fonts (Helvetica, Arial, Times, Menlo and the like) are swapped for the nearest of these. - The bundle only has Latin text at a few weights per family (Inter: 400, 700, 900). The compiler still fetches each of these families from Google Fonts for the other weights and scripts, and caches the result in `~/.cache/hyperframes/fonts`. -- Any other family needs `@font-face` in the file that uses it, or lint fails `font_family_without_font_face`: `src: url('assets/fonts/x.woff2')` for a file in the project (a capture's own fonts, or one you download there), or `local('Exact Name')` for a font installed on this machine. +- Any other family needs `@font-face` in the file that uses it, or lint fails `font_family_without_font_face`: `src: url('assets/fonts/x.woff2')` for a file in the project (a capture's own fonts, or one you download there), A `local('Exact Name')`-only face draws the font installed on this machine; a cloud render has no copy of it (lint warns `font_face_local_only`), so ship the file instead when the video may render in the cloud. ## Timing and the timeline