feat(engine,producer): verified interleaved parallel drawElement streaming (opt-in)

Step 2 of the DE engagement plan: multi-worker drawElement capture through
the streaming encoder, with the full runtime self-verification net riding
along — the confinement rule that kept the parallel clamp in place is now
satisfied on this path. Opt-in via HF_DE_PARALLEL_STREAM=true; default
routing (including the #2026 single-worker inversion) is unchanged.

Mechanism:
- distributeFramesInterleaved + WorkerTask.frameStride: worker i captures
  frames i, i+N, i+2N... — seek-based capture makes stride free and the
  ordered writer's reorder window shrinks from totalFrames/N to N (contiguous
  chunks serialize workers behind the writer).
- Depth-2 pipelined worker-encode produce in the parallel worker loop (the
  same shape as the sequential loop; frame k's in-page encode overlaps
  k+stride's produce). HF_DE_PAR_DEBUG=1 traces the first frames per worker.
- Drain guard extracted to createDrainFrameGuard (session-parameterized):
  every parallel frame gets the SAME blank-guard + PSNR self-verify as the
  sequential drain, against its owning worker's pre-injection ground truth
  (all sessions arm identical sample indices from
  CaptureOptions.compositionDurationSeconds).
- FrameReorderBuffer.abort(err): a failed worker (e.g. verification error)
  rejects all parked and future waiters — without this, peers park forever
  in waitForFrame and the pool (which awaits ALL workers before surfacing
  errors) deadlocks. Found by the verify-trip test; unit-tested.
- The typed DrawElementVerificationError is preserved past the pool's
  error-string flattening so the orchestrator's verify-retry recognizes it.
- Static-dedup stride hazard fixed: lastEncodeResult reuse now requires EVERY
  frame in (lastEncodeResultFrame, i] to be predicted-static (sequential
  capture reduces to the old has(i) check).
- Workers get separate browser PROCESSES under the flag: pages co-tenant in
  one browser starve non-active pages of BeginFrames on the paint-wait path
  (measured 86s vs 30s on a 3,245-frame rAF comp).

Validation:
- Happy path W3: verify samples pass across workers (4x inf on the 2,381f
  comp), output vs single-worker DE = 59.3dB (encode noise floor) — the
  interleave + dedup-stride produce identical pixels.
- Verify-trip (marginal comp + HF_DE_VERIFY_MIN_DB=45): fails at frame 649
  (32.2dB < 45), peers abort instead of deadlocking, whole render retries
  via parallel screenshot, RENDER_OK in 42.6s.
- Canary suite 7/7 with the flag off (default paths untouched); producer
  orchestrator tests 99/99; engine suite 909 passed (14 pre-existing main
  failures, stash-A/B verified); reorder-buffer abort unit tests.

Perf note: capture-only parallel speedup measured 1.38x (W2) / 1.52x (W3)
over single-worker DE in the spike; end-to-end numbers on this machine are
currently noisy (separate-browser init overhead + bench load) — clean
benchmarks before any default routing change. The flag stays explicit
opt-in; promoting it into the router replaces the #2026 W=1 pin for the
same cohort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-08 16:11:46 -07:00
co-authored by Claude Fable 5
parent df1d20b765
commit b3493a7b61
7 changed files with 340 additions and 73 deletions
+30 -5
View File
@@ -161,6 +161,11 @@ export interface CaptureSession {
* worker-encode path (mirrors `lastFrameBuffer` on the screenshot path). Only set
* when static-frame dedup is armed on the drawElement path. */
lastEncodeResult?: Promise<Buffer>;
/** Frame index lastEncodeResult belongs to — static-dedup reuse must verify
* every frame in (lastEncodeResultFrame, i] is predicted-static (under the
* interleaved parallel stride the "previous" produced frame is iN, and
* reusing it for frame i is only valid when the whole gap is static). */
lastEncodeResultFrame?: number;
/** Per-render self-verification ground truth (ungated-release safety net):
* K screenshot frames captured at init BEFORE the drawElement canvas is
* injected (the only window where a page screenshot shows the live DOM, not
@@ -2657,9 +2662,23 @@ export async function captureFrameToBufferPipelined(
// and skip the seek + drawElement + encode entirely. Same predicate as the serial
// path; clip-cut frames are excluded from staticFrames so they always capture.
if (session.staticFrames?.has(frameIndex) && session.lastEncodeResult) {
session.staticDedupCount = (session.staticDedupCount ?? 0) + 1;
session.capturePerf.frames += 1;
return { encodeResult: session.lastEncodeResult, captureTimeMs: Date.now() - startTime };
// Reuse is valid only when EVERY frame in (lastEncodeResultFrame, i] is
// predicted-static — then all of them (and the reused buffer) share the
// same pixels. Sequential capture reduces to has(i) (gap = {i}); the
// interleaved parallel stride makes the gap N frames wide.
const lastIdx = session.lastEncodeResultFrame ?? frameIndex - 1;
let gapStatic = true;
for (let j = lastIdx + 1; j <= frameIndex; j++) {
if (!session.staticFrames.has(j)) {
gapStatic = false;
break;
}
}
if (gapStatic) {
session.staticDedupCount = (session.staticDedupCount ?? 0) + 1;
session.capturePerf.frames += 1;
return { encodeResult: session.lastEncodeResult, captureTimeMs: Date.now() - startTime };
}
}
try {
@@ -2694,7 +2713,10 @@ export async function captureFrameToBufferPipelined(
session.capturePerf.frameMs.push(boundaryMs);
}
const boundaryResult = Promise.resolve(buffer);
if (session.staticFrames) session.lastEncodeResult = boundaryResult;
if (session.staticFrames) {
session.lastEncodeResult = boundaryResult;
session.lastEncodeResultFrame = frameIndex;
}
return { encodeResult: boundaryResult, captureTimeMs: Date.now() - startTime };
}
@@ -2721,7 +2743,10 @@ export async function captureFrameToBufferPipelined(
session.capturePerf.frameMs.push(captureTimeMs);
// Task B: retain this encode result so a following static frame can reuse it.
if (session.staticFrames) session.lastEncodeResult = encodeResult;
if (session.staticFrames) {
session.lastEncodeResult = encodeResult;
session.lastEncodeResultFrame = frameIndex;
}
return { encodeResult, captureTimeMs };
} catch (captureError) {