ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-11 06:30:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(producer,engine,cli): oom wrapping, cancellation, fallback-reason gaps

Browse Source 
Three defects found by max-effort code review of this branch:

1. The Bun OOM exact-match regex was defeated by this codebase's own
   parallel-worker error wrapping. executeParallelCapture/formatWorkerFailure
   (parallelCoordinator.ts) always wrap a worker's error as
   "Worker N: <message>", optionally suffixed and joined with other workers'
   segments, all prefixed "[Parallel] Capture failed: ". That wrapping
   defeated the exact-message check for exactly the cohort (deParallelRouter
   routed, N separate Chrome processes) the OOM-drops-to-1 fix targets — a
   real OOM there would retry at the SAME worker count instead of dropping
   to 1. Added a second pattern that recovers the signal by requiring
   "out of memory" appear as the WHOLE content of a "Worker N: ..." segment
   (bounded by end-of-string/"; "), preserving the same exact-match property
   (no bare substring match) while surviving the wrapping. Verified against
   the real wrapping logic, not a hand-typed guess at its shape.

2. shouldRetryViaPinnedFallback didn't exclude cancellation, so aborting a
   render mid-capture on the pinned router/inversion cohort would detour
   through spawning a fresh encoder/capture session before the outer catch's
   RenderCancelledError branch ended the render — delaying "stop" with a
   pointless resource spin-up/tear-down. Added an isCancellation param
   (checked first, before isVerifyError) using the same
   `err instanceof RenderCancelledError || abortSignal?.aborted` check the
   outer catch already uses.

3. deFallbackReason (this PR's new "oom"/"capture_error" values) was set
   locally but never mirrored into RenderCaptureObservability alongside
   deSelfVerifyFallback, so a render that fails AFTER a fallback attempt
   (perfSummary never built) was indistinguishable in render_error telemetry
   from one that never attempted any fallback — undercutting the "how often
   does the OOM retry fire on a render that still ultimately fails"
   question this branch exists to answer. Threaded through
   RenderCaptureObservability → RenderObservabilityTelemetryPayload →
   renderObservabilityTelemetryPayload, mirroring the existing
   deSelfVerifyFallback plumbing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-09 20:31:24 -07:00
co-authored by Claude Sonnet 5
parent b3f244a7e9
commit a355fb2f6b
9 changed files with 160 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/producer/src/services/render/observability.ts
+10
View File
@@ -42,6 +42,16 @@ export interface RenderCaptureObservability {
browserGpuMode?: string;
/** drawElement per-render self-verification tripped → whole render re-ran via screenshot. */
deSelfVerifyFallback?: boolean;
/**
* Why the capture-stage retry (self-verify OR the pinned-worker-count
* fallback) fired: "blank"/"psnr" for a real self-verify trip,
* "oom"/"capture_error" for the widened generic-failure retry. Set
* whenever a fallback is attempted, independent of whether that retry
* itself later succeeds — so a render that fails AFTER a fallback attempt
* (perfSummary never built) is still distinguishable in failure-path
* telemetry from one that never attempted any fallback.
*/
deFallbackReason?: string;
/** Auto-parallel inversion outcome: "inverted" (fired, held) | "reverted" (fired, self-verify retry rolled back). */
deWorkerInversion?: "inverted" | "reverted";
/** Worker count the resolver would have used absent the inversion; undefined if it never fired. */