ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(cli): keep the DE parallel-router trial on until a real failure, not first engagement

Browse Source 
Only consuming telemetry from one data point per install badly undersampled
the "routed" (successful) outcome — the far more common case. Changed
maybeConsumeDeParallelRouterTrial to only turn the trial off when the
router's OWN safety net actually fired (deParallelRouter === "reverted"),
not on a clean "routed" success. This runs the experiment on every eligible
render for an install indefinitely until it hits one real failure, then
stops for that install going forward — trading a slightly higher per-install
ceiling on experimental-path exposure for dramatically more successful-
routing telemetry volume across the fleet.

Also fixed a related edge case while updating this: a render that merely
"routed" (router fired, self-verify never even tripped) but then crashed
for an unrelated reason (e.g. cancellation) no longer counts as a router
failure — only "reverted" (the router's fallback path actually engaged)
does. Cancelling a render isn't evidence the router is unsafe.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-09 21:49:21 -07:00
co-authored by Claude Sonnet 5
parent 37b6a4e7e5
commit 19f90b0b92
3 changed files with 67 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/cli/src/commands/render.ts
+32 -20
View File
@@ -1612,14 +1612,17 @@ function createNoopProducerLogger(): ProducerLogger {
/**
* Enable the DE parallel-router experiment (`HF_DE_PARALLEL_ROUTER`, default
* off) for this render, one time per install, so we get real-traffic router
* telemetry (revert rate, verify-db distribution) without requiring anyone
* to manually set the env var — see `HyperframesConfig.deParallelRouterTrialFired`.
* off) for this render, on EVERY eligible render for this install, so we get
* real-traffic router telemetry (revert rate, verify-db distribution)
* without requiring anyone to manually set the env var — see
* `HyperframesConfig.deParallelRouterTrialFired`. Deliberately runs
* indefinitely (not just once) to maximize "routed" success-telemetry
* volume; see `maybeConsumeDeParallelRouterTrial` for what turns it off.
* Returns whether this call armed it (so the caller knows to check for
* consumption afterward) — false if it's already fired once, or the user
* already set the env var themselves (never override an explicit choice),
* or telemetry is disabled (no point risking the experimental path if we
* can't even record the resulting signal).
* consumption afterward) — false if it's already failed once for this
* install, or the user already set the env var themselves (never override
* an explicit choice), or telemetry is disabled (no point risking the
* experimental path if we can't even record the resulting signal).
*/
function maybeEnableDeParallelRouterTrial(quiet: boolean): boolean {
if (process.env.HF_DE_PARALLEL_ROUTER !== undefined) return false;
@@ -1629,8 +1632,9 @@ function maybeEnableDeParallelRouterTrial(quiet: boolean): boolean {
if (!quiet) {
console.log(
c.dim(
" Trying the experimental parallel drawElement capture path once for this install " +
"(opt out: HF_DE_PARALLEL_ROUTER=false)",
" Trying the experimental parallel drawElement capture path for this install " +
"(disabled automatically if it ever needs to fall back; opt out anytime: " +
"HF_DE_PARALLEL_ROUTER=false)",
),
);
}
@@ -1638,20 +1642,28 @@ function maybeEnableDeParallelRouterTrial(quiet: boolean): boolean {
}
/**
* After a trial-armed render, persist that the experiment actually engaged
* (routed or reverted — either produces telemetry) so it's never enabled
* again for this install. Checks both the success path (`perfSummary`) and
* the failure path (`errorDetails.observability.capture`, mutated in place
* before a hard failure throws) — a crash while routed still counts as a
* fired trial. No-ops if the router never actually became eligible for this
* render (e.g. too few frames): the trial stays available for a future run.
* After a trial-armed render, persist that the router's OWN bet actually
* failed — its self-verify/generic-failure safety net fired
* (`deParallelRouter === "reverted"`) — so it's never enabled again for this
* install. A clean "routed" (the render succeeded with no fallback) does
* NOT consume the trial — the whole point is to keep trying on every
* eligible render until we see one real failure signal, maximizing
* successful-routing telemetry volume rather than stopping at the first
* data point. Checks both the success path (`perfSummary`) and the failure
* path (`errorDetails.observability.capture`, mutated in place before a
* hard failure throws) — a render that still failed even after the
* fallback retry counts too. A render that crashed for an unrelated reason
* while merely "routed" (never reached "reverted" — e.g. cancellation)
* does NOT count as a router failure and does not turn the trial off.
* No-ops if the router never became eligible for this render (e.g. too few
* frames): the trial stays available for a future run either way.
*/
function maybeConsumeDeParallelRouterTrial(trialArmed: boolean, job: RenderJob): void {
if (!trialArmed) return;
const engaged =
job.perfSummary?.drawElement?.parallelRouter ??
job.errorDetails?.observability?.capture.deParallelRouter;
if (engaged === undefined) return;
const failed =
job.perfSummary?.drawElement?.parallelRouter === "reverted" ||
job.errorDetails?.observability?.capture.deParallelRouter === "reverted";
if (!failed) return;
const config = readConfig();
config.deParallelRouterTrialFired = true;
writeConfig(config);