From 924727a0b462b63c28c68a5fdf971ba7a6b1a369 Mon Sep 17 00:00:00 2001 From: Vance Ingalls Date: Tue, 7 Jul 2026 15:48:07 -0700 Subject: [PATCH] =?UTF-8?q?feat(producer,cli):=20drawElement=20priority=20?= =?UTF-8?q?inversion=20=E2=80=94=20single-worker=20streaming=20over=20auto?= =?UTF-8?q?-parallel=20(#2026)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(producer,cli): drawElement priority inversion — single-worker streaming over auto-parallel clamp:parallel eats 50% of local renders (1,326/fortnight; DE engagement stuck at 3.8%) by routing multi-worker renders to unverified screenshot capture. Benchmarks (2026-07-08, 4 comps x W1/W2/W3/W5) show that above the ~900-frame amortization crossover, single-worker VERIFIED drawElement streaming beats screenshot-parallel at EVERY worker count (2,380f: 66s vs 109-127s; 3,600f: 33s vs 39-56s; parallel scaling flattens past W2), while below it DE's fixed init cost loses by <=2.2s. - shouldPreferSingleWorkerDrawElement (exported predicate + 7 unit tests): inverts an AUTO-resolved multi-worker render to workerCount=1 when the comp matches the benchmarked configuration — default-on DE (darwin hardware clamp upstream), no compile gate, no forced-screenshot hint, mp4 output, single-worker streaming eligible, and totalFrames >= HF_DE_SINGLE_MIN_FRAMES (default 900; 0 disables). Explicit --workers N is always honored. - Inverted renders keep the probe session and land on the worker-encode streaming drain — the ONLY path with runtime self-verification, so this moves ~40% of previously-clamped renders onto the verified fast path. Comps that later hit an init-time gate (~1.5% of local renders) render single-worker screenshot streaming; accepted trade. - Telemetry: de_worker_inversion on render_complete (orchestrator -> perfSummary.workerInversion -> CLI), plus the worker_resolution observability checkpoint now records deWorkerInversion. Validation: e2e matrix on 2,381f comp — auto->5 workers inverted to 1, DE verified 4x inf PSNR, RENDER_OK; short comp (360f) auto stays 5-worker; explicit WORKERS=3 honored; HF_DE_SINGLE_MIN_FRAMES=0 disables. Canary suite 7/7 (PSNRs identical). renderOrchestrator tests 86/86. tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 * fix(producer): review fixes — inversion routing guards, calibration skip, retry revert Max code-review round on the inversion (13 confirmed findings): - Streaming spawn-failure disk fallback now clamps default-on drawElement (deClampReason=disk_path, DE-mode probe closed) exactly like the pre-capture clamp — previously it carried useDrawElement=true onto the unverified disk path, the hole the verified-path confinement exists to close, newly reachable for every inverted render. - Predicate gained the routing knowledge it was blind to: layered/HDR and shader-transition comps (drawElement never runs there), supersampling (deviceScaleFactor>1 init gate), a probe session whose init gates already disengaged DE, and the PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true explicit parallel-DE opt-in (honored like --workers N). - Eligibility is evaluated BEFORE capture calibration and skips it when the inversion pins workers to 1 regardless of the estimate — the throwaway calibration browser + sample captures cost ~41s on the 2,381-frame benchmark comp (auto render: 111.6s -> 70.1s total). - Self-verify retry reverts the inversion: the re-render returns to the pre-inversion parallel screenshot path (disk) instead of single-worker screenshot streaming, the slowest shape for exactly the comps drawElement damages. - HF_DE_SINGLE_MIN_FRAMES="" (set-but-empty) now falls back to the 900 default instead of aliasing the 0 kill switch. - Timeout advisory uses the RESOLVED worker count — an inverted render that times out no longer prints "Retry with --workers 1" (the configuration that just failed). - Telemetry: deWorkerInversion recorded in capture observability (failed renders are attributable), emitted as literal false when not fired (queryable denominator), and the drawElement perf input shape is one exported DrawElementPerfInput type instead of three copies. - Tests: requestedWorkers undefined (the value production actually passes) + the four new predicate guards; 91/91. Validation: e2e auto render — calibration skipped (deInversionEligible), inversion fires, DE verified 4x inf, total 70.1s (was 111.6s); HF_DE_SINGLE_MIN_FRAMES=0 restores calibration + parallel; canary suite 7/7 (PSNRs identical); tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 * fix(producer,cli): review round 2 — loss-cohort telemetry, retry-plan helper, boundary tests - de_worker_inversion is now a tri-state string ("inverted" | "reverted" | "none") instead of a boolean: the self-verify retry marks the render "reverted" rather than resetting to false, so the dashboard can segment the lost-inversion cohort first-class instead of inferring it from deSelfVerifyFallback + frame-count joins (james-russo #1). - The retry rollback is extracted to resolveInversionRetryPlan (pure, exported) with unit coverage: pre-inversion worker-count restore, streaming re-resolution (multi-worker retry -> disk), "reverted" state, null when never inverted (james-russo #2). - WOULD_RESOLVE_MULTI_WORKER named constant replaces the bare sentinel 2 (james-russo #5); minFrames: -1 boundary case added (miga #3). 94/94 renderOrchestrator tests; tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 * feat(producer,cli): emit de_pre_inversion_workers for the parallel counterfactual The ramp-down decision needs "did DE beat the parallel render it displaced", not just "did DE beat single-worker screenshot". Emitting the worker count the auto-resolution chose BEFORE the inversion pinned it to 1 makes the parallel counterfactual computable per render (screenshot ms/frame from the verify samples / W x the measured parallel-efficiency curve). Set only when the inversion fired. Smoke: 2,381f auto render -> de_worker_inversion="inverted", de_pre_inversion_workers=5, mode=drawelement, verify armed 4. 99/99 tests. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- packages/cli/src/commands/render.ts | 2 + packages/cli/src/telemetry/events.ts | 4 + .../src/services/render/observability.ts | 2 + .../src/services/render/perfSummary.ts | 49 ++-- .../src/services/renderOrchestrator.test.ts | 138 ++++++++++ .../src/services/renderOrchestrator.ts | 252 +++++++++++++++++- 6 files changed, 407 insertions(+), 40 deletions(-) diff --git a/packages/cli/src/commands/render.ts b/packages/cli/src/commands/render.ts index b5afaf12b..a07e1c988 100644 --- a/packages/cli/src/commands/render.ts +++ b/packages/cli/src/commands/render.ts @@ -1678,6 +1678,8 @@ function trackRenderMetrics( deCaptureMode: perf?.drawElement?.mode, deCompileGate: perf?.drawElement?.compileGate, deClampReason: perf?.drawElement?.clampReason, + deWorkerInversion: perf?.drawElement?.workerInversion, + dePreInversionWorkers: perf?.drawElement?.preInversionWorkers, deGateReason: perf?.drawElement?.gateReason, deWorkerEncode: perf?.drawElement?.workerEncode, deVerifyArmed: perf?.drawElement?.verifyArmed, diff --git a/packages/cli/src/telemetry/events.ts b/packages/cli/src/telemetry/events.ts index bdf6e59f6..a6d017676 100644 --- a/packages/cli/src/telemetry/events.ts +++ b/packages/cli/src/telemetry/events.ts @@ -120,6 +120,8 @@ export function trackRenderComplete( deCaptureMode?: string; deCompileGate?: string; deClampReason?: string; + deWorkerInversion?: string; + dePreInversionWorkers?: number; deGateReason?: string; deWorkerEncode?: boolean; deVerifyArmed?: number; @@ -196,6 +198,8 @@ export function trackRenderComplete( de_capture_mode: props.deCaptureMode, de_compile_gate: props.deCompileGate, de_clamp_reason: props.deClampReason, + de_worker_inversion: props.deWorkerInversion, + de_pre_inversion_workers: props.dePreInversionWorkers, de_gate_reason: props.deGateReason, de_worker_encode: props.deWorkerEncode, de_verify_armed: props.deVerifyArmed, diff --git a/packages/producer/src/services/render/observability.ts b/packages/producer/src/services/render/observability.ts index f44e03b53..498b2edf0 100644 --- a/packages/producer/src/services/render/observability.ts +++ b/packages/producer/src/services/render/observability.ts @@ -42,6 +42,8 @@ export interface RenderCaptureObservability { browserGpuMode?: string; /** drawElement per-render self-verification tripped → whole render re-ran via screenshot. */ deSelfVerifyFallback?: boolean; + /** Auto-parallel inversion outcome: "inverted" (fired, held) | "reverted" (fired, self-verify retry rolled back). */ + deWorkerInversion?: "inverted" | "reverted"; protocolTimeoutMs?: number; pageNavigationTimeoutMs?: number; playerReadyTimeoutMs?: number; diff --git a/packages/producer/src/services/render/perfSummary.ts b/packages/producer/src/services/render/perfSummary.ts index 498ed9890..5dab95d24 100644 --- a/packages/producer/src/services/render/perfSummary.ts +++ b/packages/producer/src/services/render/perfSummary.ts @@ -40,23 +40,30 @@ export function pushWorkerDedupPerfs( * render-level drawElement outcome. mode/gateReason |-join distinct values * across workers (bounded cardinality); counters SUM. */ +/** Orchestrator-supplied render-level drawElement outcome (one shape, used by + * both the aggregate function and buildRenderPerfSummary's input). */ +export interface DrawElementPerfInput { + compileGate?: string; + clampReason?: string; + workerInversion?: "inverted" | "reverted"; + /** Auto-resolved worker count before the inversion pinned it to 1 (set only when the inversion fired). */ + preInversionWorkers?: number; + selfVerifyFallback: boolean; + fallbackReason?: string; + drainStats?: { + verifyChecked: number; + verifyMinDb?: number; + blankSuspects: number; + blankDeterministicAccepts: number; + blankRecaptures: number; + }; +} + // Flat field mapping — branches are ?? fallbacks, not logic. // fallow-ignore-next-line complexity function aggregateDrawElement( perfs: CapturePerfSummary[], - de: { - compileGate?: string; - clampReason?: string; - selfVerifyFallback: boolean; - fallbackReason?: string; - drainStats?: { - verifyChecked: number; - verifyMinDb?: number; - blankSuspects: number; - blankDeterministicAccepts: number; - blankRecaptures: number; - }; - }, + de: DrawElementPerfInput, ): RenderPerfSummary["drawElement"] { if (perfs.length === 0) return undefined; const modes = [...new Set(perfs.map((p) => p.captureMode).filter(Boolean))].sort(); @@ -68,6 +75,8 @@ function aggregateDrawElement( mode: modes.join("|") || "unknown", compileGate: de.compileGate, clampReason: de.clampReason, + workerInversion: de.workerInversion ?? "none", + preInversionWorkers: de.preInversionWorkers, gateReason: gateReasons.length > 0 ? gateReasons.join("|") : undefined, workerEncode: perfs.some((p) => p.deWorkerEncode), verifyArmed: perfs.reduce((sum, p) => sum + (p.deVerifyArmed ?? 0), 0), @@ -135,19 +144,7 @@ export function buildRenderPerfSummary(input: { peakHeapUsedBytes: number; /** Per-session/per-worker static-dedup perf; aggregated into `staticDedup`. */ dedupPerfs: CapturePerfSummary[]; - drawElement?: { - compileGate?: string; - clampReason?: string; - selfVerifyFallback: boolean; - fallbackReason?: string; - drainStats?: { - verifyChecked: number; - verifyMinDb?: number; - blankSuspects: number; - blankDeterministicAccepts: number; - blankRecaptures: number; - }; - }; + drawElement?: DrawElementPerfInput; }): RenderPerfSummary { return { renderId: input.job.id, diff --git a/packages/producer/src/services/renderOrchestrator.test.ts b/packages/producer/src/services/renderOrchestrator.test.ts index 0d1516bbb..1af6be66b 100644 --- a/packages/producer/src/services/renderOrchestrator.test.ts +++ b/packages/producer/src/services/renderOrchestrator.test.ts @@ -28,6 +28,8 @@ import { MAX_TRANSIENT_CAPTURE_RETRIES, resolveCaptureForceScreenshotForPageSideCompositing, shouldDiscardProbeSessionForPageSideCompositing, + resolveInversionRetryPlan, + shouldPreferSingleWorkerDrawElement, shouldUseStreamingEncode, } from "./renderOrchestrator.js"; import { ensureFrameWritten } from "./render/stages/captureHdrFrameShared.js"; @@ -1564,3 +1566,139 @@ describe("resolveDeviceScaleFactor", () => { ).toThrow(/aspect ratio/); }); }); + +describe("shouldPreferSingleWorkerDrawElement (DE priority inversion)", () => { + const eligible = { + workerCount: 5, + requestedWorkers: "auto" as const, + useDrawElement: true, + deCompileGate: undefined, + forceScreenshot: false, + outputFormat: "mp4" as const, + totalFrames: 2380, + minFrames: 900, + singleWorkerStreamingOk: true, + layeredOrEffectRoute: false, + supersampling: false, + probeDeGated: false, + experimentalParallelDeOptIn: false, + }; + + it("inverts an auto-resolved multi-worker render for an eligible long comp", () => { + expect(shouldPreferSingleWorkerDrawElement(eligible)).toBe(true); + }); + + it("honors explicitly requested workers", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, requestedWorkers: 3 })).toBe(false); + }); + + it("inverts for requestedWorkers undefined — the value production actually passes for auto", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, requestedWorkers: undefined })).toBe( + true, + ); + }); + + it("skips comps routed to layered/HDR/shader paths (drawElement never runs there)", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, layeredOrEffectRoute: true })).toBe( + false, + ); + }); + + it("skips supersampled renders (engine init-time DE gate)", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, supersampling: true })).toBe(false); + }); + + it("skips when the probe session already shows DE gated out", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, probeDeGated: true })).toBe(false); + }); + + it("honors the explicit experimental parallel-DE opt-in", () => { + expect( + shouldPreferSingleWorkerDrawElement({ ...eligible, experimentalParallelDeOptIn: true }), + ).toBe(false); + }); + + it("skips below the amortization threshold (measured crossover ~900 frames)", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, totalFrames: 360 })).toBe(false); + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, totalFrames: 900 })).toBe(true); + }); + + it("is disabled by minFrames <= 0 (HF_DE_SINGLE_MIN_FRAMES=0 kill switch)", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, minFrames: 0 })).toBe(false); + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, minFrames: -1 })).toBe(false); + }); + + it("requires drawElement to be enabled and ungated", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, useDrawElement: false })).toBe(false); + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, deCompileGate: "3d" })).toBe(false); + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, forceScreenshot: true })).toBe(false); + }); + + it("only applies to the benchmarked configuration (mp4 + streaming-eligible)", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, outputFormat: "webm" })).toBe(false); + expect( + shouldPreferSingleWorkerDrawElement({ ...eligible, singleWorkerStreamingOk: false }), + ).toBe(false); + }); + + it("is a no-op when workers already resolved to 1", () => { + expect(shouldPreferSingleWorkerDrawElement({ ...eligible, workerCount: 1 })).toBe(false); + }); +}); + +describe("resolveInversionRetryPlan (self-verify retry rollback)", () => { + const cfg = { enableStreamingEncode: true, streamingEncodeMaxDurationSeconds: 240 }; + + it("returns null when the render was never inverted", () => { + expect( + resolveInversionRetryPlan({ + deWorkerInversion: undefined, + preInversionWorkerCount: 5, + cfg, + outputFormat: "mp4", + durationSeconds: 80, + }), + ).toBe(null); + expect( + resolveInversionRetryPlan({ + deWorkerInversion: "reverted", + preInversionWorkerCount: 5, + cfg, + outputFormat: "mp4", + durationSeconds: 80, + }), + ).toBe(null); + }); + + it("restores the pre-inversion worker count and routes multi-worker retries to disk", () => { + const plan = resolveInversionRetryPlan({ + deWorkerInversion: "inverted", + preInversionWorkerCount: 5, + cfg, + outputFormat: "mp4", + durationSeconds: 80, + }); + expect(plan).toEqual({ + workerCount: 5, + // shouldUseStreamingEncode is workerCount===1-only — parallel retry + // goes through the disk path. + useStreamingEncode: false, + deWorkerInversion: "reverted", + }); + }); + + it("keeps streaming when the pre-inversion resolution was already single-worker", () => { + const plan = resolveInversionRetryPlan({ + deWorkerInversion: "inverted", + preInversionWorkerCount: 1, + cfg, + outputFormat: "mp4", + durationSeconds: 80, + }); + expect(plan).toEqual({ + workerCount: 1, + useStreamingEncode: true, + deWorkerInversion: "reverted", + }); + }); +}); diff --git a/packages/producer/src/services/renderOrchestrator.ts b/packages/producer/src/services/renderOrchestrator.ts index 0854bae35..ce3a95c9a 100644 --- a/packages/producer/src/services/renderOrchestrator.ts +++ b/packages/producer/src/services/renderOrchestrator.ts @@ -116,7 +116,10 @@ import { } from "./render/stages/extractVideosStage.js"; import { runAudioStage } from "./render/stages/audioStage.js"; import { runCaptureStage } from "./render/stages/captureStage.js"; -import { runCaptureStreamingStage } from "./render/stages/captureStreamingStage.js"; +import { + type CaptureStreamingStageResult, + runCaptureStreamingStage, +} from "./render/stages/captureStreamingStage.js"; import { runCaptureHdrStage } from "./render/stages/captureHdrStage.js"; import { runEncodeStage } from "./render/stages/encodeStage.js"; import { runAssembleStage } from "./render/stages/assembleStage.js"; @@ -373,6 +376,10 @@ export interface RenderPerfSummary { compileGate?: string; /** Producer clamp that disabled default DE: parallel | disk_path. */ clampReason?: string; + /** Auto-parallel inversion outcome: "inverted" (fired, held), "reverted" (fired, self-verify retry rolled back), "none". */ + workerInversion?: string; + /** Worker count the auto-resolution chose BEFORE the inversion pinned it to 1 — the parallel counterfactual for speedup math. Only set when the inversion fired. */ + preInversionWorkers?: number; /** Engine init-time gate: swiftshader | css_effect:* | at_risk_timeline | 3d_init_failed | supersampling | render_mode_hint. */ gateReason?: string; /** Worker-encode drain (the verified path) was active. */ @@ -950,6 +957,98 @@ export function shouldUseStreamingEncode( return workerCount === 1; } +/** + * DE priority inversion predicate: should an AUTO-resolved multi-worker render + * drop to single-worker verified drawElement streaming? + * + * Benchmarked 2026-07-08: above ~900 frames DE-single beats screenshot-parallel + * at every worker count (2,380f: 66s vs 109–127s at W2–W5); below it DE's fixed + * init cost (verify + dedup arming) loses by a small margin. Only fires for the + * exact benchmarked configuration: default-on DE, mp4, streaming-eligible, + * no compile gate, no forced screenshot, workers not explicitly requested. + */ +export function shouldPreferSingleWorkerDrawElement(args: { + workerCount: number; + /** job.config.workers — a number means the user explicitly chose. */ + requestedWorkers: number | "auto" | undefined; + useDrawElement: boolean; + deCompileGate: string | undefined; + forceScreenshot: boolean; + outputFormat: NonNullable; + totalFrames: number; + /** Amortization threshold; <=0 disables the inversion. */ + minFrames: number; + /** shouldUseStreamingEncode(cfg, format, 1, duration) at the call site. */ + singleWorkerStreamingOk: boolean; + /** + * Comp routes to the layered-composite / page-side-compositing paths + * (HDR content or shader transitions) — those force screenshots and never + * run drawElement or streaming, so an inversion would only mislabel + * telemetry and keep the probe session alive through the heaviest stage. + */ + layeredOrEffectRoute: boolean; + /** deviceScaleFactor > 1 — the engine's supersampling gate blocks DE. */ + supersampling: boolean; + /** + * The probe session already ran the engine's init-time DE gates and DE did + * NOT engage (not drawelement mode, not a deferred video comp) — inverting + * would pin a known-screenshot render to one worker. + */ + probeDeGated: boolean; + /** + * PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true is an explicit opt-in that + * deliberately allows parallel drawElement (bypassing the downstream + * clamp) — honor it like an explicit --workers request. + */ + experimentalParallelDeOptIn: boolean; +}): boolean { + return ( + args.workerCount > 1 && + typeof args.requestedWorkers !== "number" && + args.useDrawElement && + !args.deCompileGate && + !args.forceScreenshot && + args.outputFormat === "mp4" && + args.minFrames > 0 && + args.totalFrames >= args.minFrames && + args.singleWorkerStreamingOk && + !args.layeredOrEffectRoute && + !args.supersampling && + !args.probeDeGated && + !args.experimentalParallelDeOptIn + ); +} + +/** + * Plan the self-verify retry for an inverted render: the inversion bet on + * drawElement and lost, so the re-render returns to the pre-inversion parallel + * screenshot path (streaming re-resolved for that worker count — multi-worker + * routes to the disk stage). Returns null when the render was not inverted. + */ +export function resolveInversionRetryPlan(args: { + deWorkerInversion: "inverted" | "reverted" | undefined; + preInversionWorkerCount: number; + cfg: Pick; + outputFormat: NonNullable; + durationSeconds: number; +}): { + workerCount: number; + useStreamingEncode: boolean; + deWorkerInversion: "reverted"; +} | null { + if (args.deWorkerInversion !== "inverted") return null; + return { + workerCount: args.preInversionWorkerCount, + useStreamingEncode: shouldUseStreamingEncode( + args.cfg, + args.outputFormat, + args.preInversionWorkerCount, + args.durationSeconds, + ), + deWorkerInversion: "reverted", + }; +} + export function resolveCaptureForceScreenshotForPageSideCompositing(args: { forceScreenshot: boolean; usePageSideCompositing: boolean; @@ -1217,6 +1316,9 @@ export async function executeRenderJob( // whether self-verify fell back, and the drain-side counters. const deCompileGate = compileResult.deCompileGate; let deClampReason: string | undefined; + // "inverted" = fired and held; "reverted" = fired but the self-verify + // retry rolled back to the parallel path; undefined = never fired. + let deWorkerInversion: "inverted" | "reverted" | undefined; let deSelfVerifyFallback = false; let deFallbackReason: string | undefined; let deDrainStats: import("./render/stages/captureStreamingStage.js").DeDrainStats | undefined; @@ -1550,11 +1652,53 @@ export async function executeRenderJob( const htmlInCanvasDetected = compiled.renderModeHints.reasons.some( (r) => r.code === "htmlInCanvas", ); + // Only use the HDR encoder preset when there's HDR content to pass through — + // either native HDR videos OR native HDR images. For SDR-only compositions, + // auto mode stays SDR since H.265 10-bit causes browser color management + // issues (orange shift) with no quality benefit. (Computed here, ahead of + // worker resolution, because the DE inversion below must not fire for + // comps that route to the layered/HDR paths.) + const nativeHdrIds = new Set([...nativeHdrVideoIds, ...nativeHdrImageIds]); + const hasHdrContent = Boolean(effectiveHdr && nativeHdrIds.size > 0); + // DE priority inversion eligibility — evaluated BEFORE capture calibration + // because when every multi-worker resolution would be inverted to 1 anyway, + // the calibration stage (a throwaway Chrome launch + timeline-spread sample + // captures, seconds of wall clock) buys nothing and is skipped. + // Threshold override: HF_DE_SINGLE_MIN_FRAMES (0 disables the inversion; + // a set-but-empty var falls back to the default, it is NOT the kill switch). + const deSingleMinFramesRaw = process.env.HF_DE_SINGLE_MIN_FRAMES; + const deSingleMinFramesNum = + deSingleMinFramesRaw === undefined || deSingleMinFramesRaw.trim() === "" + ? 900 + : Number(deSingleMinFramesRaw); + const deSingleMinFrames = Number.isFinite(deSingleMinFramesNum) ? deSingleMinFramesNum : 900; + // "Would ANY multi-worker resolution be inverted?" — if workers resolve + // to 1 naturally the outcome is identical either way. + const WOULD_RESOLVE_MULTI_WORKER = 2; + const deInversionEligible = shouldPreferSingleWorkerDrawElement({ + workerCount: WOULD_RESOLVE_MULTI_WORKER, + requestedWorkers: job.config.workers, + useDrawElement: cfg.useDrawElement, + deCompileGate, + forceScreenshot: captureForceScreenshot, + outputFormat, + totalFrames, + minFrames: deSingleMinFrames, + singleWorkerStreamingOk: shouldUseStreamingEncode(cfg, outputFormat, 1, job.duration), + layeredOrEffectRoute: hasHdrContent || compiled.hasShaderTransitions, + supersampling: deviceScaleFactor > 1, + probeDeGated: + probeSession !== null && + probeSession.captureMode !== "drawelement" && + !probeSession.deInitDeferred, + experimentalParallelDeOptIn: process.env.PRODUCER_EXPERIMENTAL_FAST_CAPTURE === "true", + }); if ( job.config.workers === undefined && totalFrames >= 60 && !htmlInCanvasDetected && - !cfg.lowMemoryMode + !cfg.lowMemoryMode && + !deInversionEligible ) { const outcome = await observeRenderStage( observability, @@ -1593,6 +1737,7 @@ export async function executeRenderJob( totalFrames, htmlInCanvasDetected, lowMemoryMode: Boolean(cfg.lowMemoryMode), + deInversionEligible, }); } @@ -1606,8 +1751,30 @@ export async function executeRenderJob( log, captureCalibration?.estimate, ); - updateCaptureObservability({ workerCount }); - observability.checkpoint("worker_resolution", "resolved", { workerCount }); + // DE priority inversion — see shouldPreferSingleWorkerDrawElement for the + // policy and benchmark rationale (eligibility resolved above, before + // calibration). Comps that pass every static check but hit an engine + // INIT-time gate at capture (css-effects / at-risk, ~1.5% of local + // renders) render single-worker screenshot streaming — slower than + // parallel would have been, accepted for the routing win everywhere else. + // `preInversionWorkerCount` lets the self-verify retry return to the + // parallel path when the drawElement bet loses. + const preInversionWorkerCount = workerCount; + if (deInversionEligible && workerCount > 1) { + deWorkerInversion = "inverted"; + log.info( + "[Render] Fast capture: single-worker drawElement streaming preferred over " + + `${workerCount}-worker screenshot capture (${totalFrames} frames >= ` + + `${deSingleMinFrames}; verified path, measured faster at every worker count). ` + + "Set HF_DE_SINGLE_MIN_FRAMES=0 or --workers N to override.", + ); + workerCount = 1; + } + updateCaptureObservability({ workerCount, deWorkerInversion }); + observability.checkpoint("worker_resolution", "resolved", { + workerCount, + deWorkerInversion: deWorkerInversion ?? "none", + }); if (workerCount > 1 && probeSession) { lastBrowserConsole = probeSession.browserConsoleBuffer; @@ -1672,12 +1839,8 @@ export async function executeRenderJob( }; const videoExt = FORMAT_EXT[outputFormat] ?? ".mp4"; const videoOnlyPath = join(workDir, `video-only${videoExt}`); - // Only use the HDR encoder preset when there's HDR content to pass through — - // either native HDR videos OR native HDR images. For SDR-only compositions, - // auto mode stays SDR since H.265 10-bit causes browser color management - // issues (orange shift) with no quality benefit. - const nativeHdrIds = new Set([...nativeHdrVideoIds, ...nativeHdrImageIds]); - const hasHdrContent = Boolean(effectiveHdr && nativeHdrIds.size > 0); + // (nativeHdrIds / hasHdrContent are computed above, ahead of worker + // resolution, for the DE inversion eligibility check.) // Page-side compositing opt-in: when the engine is configured to run the // shader blend inside Chrome via a page-side WebGL canvas, the layered // Node-side composite path is unnecessary for SDR shader transitions. @@ -1911,9 +2074,41 @@ export async function executeRenderJob( deSelfVerifyFallback: true, }); probeSession = null; - streamingRes = await invokeStreaming(); + const inversionRetryPlan = resolveInversionRetryPlan({ + deWorkerInversion, + preInversionWorkerCount, + cfg, + outputFormat, + durationSeconds: job.duration, + }); + if (inversionRetryPlan) { + // The inversion bet on drawElement and lost — re-render on the + // pre-inversion parallel screenshot path instead of single-worker + // screenshot streaming (the slowest capture shape for this size). + // "reverted" (not cleared) so telemetry keeps the lost-inversion + // cohort distinguishable from renders that never inverted. + deWorkerInversion = inversionRetryPlan.deWorkerInversion; + workerCount = inversionRetryPlan.workerCount; + useStreamingEncode = inversionRetryPlan.useStreamingEncode; + updateCaptureObservability({ + workerCount, + useStreamingEncode, + deWorkerInversion, + }); + log.info( + `[Render] Reverting worker inversion for the retry: ${workerCount} workers, ` + + `streaming=${useStreamingEncode}.`, + ); + } + if (useStreamingEncode) { + streamingRes = await invokeStreaming(); + } else { + // Parallel retry goes through the disk path below. + streamingRes = { success: false } satisfies CaptureStreamingStageResult; + } // The first attempt's error marked the phase failed; the retry - // recovered it — don't brand the render as failed in telemetry. + // recovered it (or was rerouted to disk) — don't brand the render + // as failed in telemetry. observability.clearFailure("capture_streaming"); } const captureFrameMs = Date.now() - captureFrameStart; @@ -1935,6 +2130,29 @@ export async function executeRenderJob( perfStages.encodeMs = streamingRes.encodeMs; // Overlapped with capture } else { useStreamingEncode = false; + // The disk path has no drain-time self-verification — clamp + // default-on drawElement here exactly like the pre-capture clamp + // (verified-path confinement). Skipped when screenshots are already + // forced (nothing to clamp) or under the explicit experimental + // opt-in, mirroring the clamp above. + if ( + cfg.useDrawElement && + !captureForceScreenshot && + process.env.PRODUCER_EXPERIMENTAL_FAST_CAPTURE !== "true" + ) { + cfg.useDrawElement = false; + deClampReason = deClampReason ?? "disk_path"; + log.info( + "[Render] Fast capture: drawElement disabled for the disk fallback — " + + "streaming encoder spawn failed and the disk path has no runtime " + + "self-verification.", + ); + if (probeSession && probeSession.captureMode === "drawelement") { + lastBrowserConsole = probeSession.browserConsoleBuffer; + await closeCaptureSession(probeSession); + probeSession = null; + } + } updateCaptureObservability({ useStreamingEncode }); observability.checkpoint("capture_streaming", "spawn failed; falling back to disk"); } @@ -2091,6 +2309,8 @@ export async function executeRenderJob( drawElement: { compileGate: deCompileGate, clampReason: deClampReason, + workerInversion: deWorkerInversion, + preInversionWorkers: deWorkerInversion ? preInversionWorkerCount : undefined, selfVerifyFallback: deSelfVerifyFallback, fallbackReason: deFallbackReason, drainStats: deDrainStats, @@ -2183,10 +2403,14 @@ export async function executeRenderJob( errorMessage.includes("Waiting failed") || errorMessage.includes("timeout exceeded") || errorMessage.includes("Navigation timeout"); - const wasParallel = job.config.workers !== 1; + // Use the RESOLVED worker count (auto renders — and inverted ones — may + // have run single-worker even though job.config.workers is unset), so the + // "--workers 1" advisory never points at the configuration that just failed. + const wasParallel = + (captureObservability.workerCount ?? (job.config.workers === 1 ? 1 : 2)) > 1; if (isTimeoutError && wasParallel) { log.warn( - `Parallel capture timed out with ${job.config.workers ?? "auto"} workers. ` + + `Parallel capture timed out with ${captureObservability.workerCount ?? "auto"} workers. ` + `Video-heavy compositions often need sequential capture. Retry with --workers 1`, ); }