Commit Graph
4 Commits
Author SHA1 Message Date
Vance IngallsandClaude Sonnet 5 ec921e143b feat(producer,cli): full telemetry visibility for DE parallel-router/inversion failures
render_error previously carried zero DE-cohort context — a hard failure while
routed (worker crash, OOM, capture timeout from the fixed 3-worker pin
overriding calibration) was indistinguishable from any other failure. The
data existed (RenderCaptureObservability is mutated live and survives into
job.errorDetails on the failure path) but was never projected into the
render_error payload, which only ever drew de_* fields from perfSummary
(success-only).

- RenderCaptureObservability now also records dePreInversionWorkers /
  dePreRouterWorkers — the worker count calibration would have picked absent
  the experiment — so a resource-pressure failure can be correlated with the
  router overriding a lower calibrated count.
- New capture-sourced de_* fields on RenderObservabilityTelemetryPayload,
  shared by trackRenderComplete and trackRenderError. Explicit
  perfSummary-sourced fields still win on render_complete (spread moved
  first in the event object) — this is purely a failure-path fallback.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-09 15:31:58 -07:00
Vance Ingalls 8ba3c33915 fix(engine,producer,cli): close review gaps in sub-timeline fail-fast
Address PR #2045 review feedback:
- Share a SubTimelineWaitOutcome type (engine) end-to-end instead of
  widening to string across CapturePerfSummary / RenderPerfSummary /
  telemetry, so the three layers can't drift.
- Dedupe scriptLoadFailures on push — a 4xx response and its trailing
  requestfailed both recorded the same URL, doubling the failed-URL
  list in the fail-fast warning.
- Thread the sub-timeline-wait outcome into render_error (not just
  render_complete): a render that fail-fasts and then fails downstream
  (pollVideosReady, extract, encode) previously dropped this signal on
  the floor. dedupPerfs is now function-scoped so the catch path can
  read it, same treatment as the existing captureAttempts array.
2026-07-08 16:09:47 -07:00
James RussoandClaude Opus 4.8 733f88cb1f feat(producer,cli): render-reliability telemetry counters for capture hardening (#1850)
Follow-up to the render-reliability batch (#1841/#1842/#1843). Threads two capture-reliability counters through the existing observability → CLI-telemetry pipeline (no new PostHog wiring) so #1842's hardening is measurable on dashboard 1783183:

- transient-retry burn (CaptureAttemptSummary.reason gains "transient-retry"; counted into RenderCaptureObservability.transientRetries on BOTH the recovered and the still-failed paths via a shared helper).
- OOM classification (memoryExhaustionDetected set when describeMemoryExhaustion classifies the failure).

Surfaced as capture_transient_retries + capture_memory_exhaustion_detected render-event props. Tests cover the attempt tagging and the payload mapping.

Further follow-up (different subsystems): encoder-frame-0-exit signal, and P1-3 pre-flight-rejection / P1-4 cli_env_check counters.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 20:57:39 -07:00
Miguel Ángel 29d6f1eac9 fix(render): add end-to-end observability (#1248) 2026-06-06 23:55:58 -04:00