mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-14 13:27:13 +00:00
Vance IngallsandClaude Opus 5
e9de2fa14f
Recut after pre-registering the read exposed two flaws in the first cut:
1. Attribution was wrong. de_short_band keyed on frame count + element
ceiling alone, so a webm render, a compile-gated comp, or a forced
screenshot at 400f reported "applied" while its routing was untouched —
poisoning the measurement cohort with unaffected renders and diluting
any effect toward zero. Now the predicate is evaluated twice (900 floor
vs band floor) and the band is DECISIVE only when the calls disagree:
every other eligibility condition passed and only the floor differed.
The cohort contains exactly the renders whose routing the band decides.
2. A same-release flip is unfalsifiable. composition_element_count ships
WITH the routing change, so the before-period cannot be filtered to the
same cohort as the after-period — the comparison would show a speedup
even if the change did nothing (the after-cohort excludes big comps by
construction; the before-cohort includes them). Routing is therefore
gated behind HF_DE_SHORT_BAND_ROUTE, default OFF: this release computes
and emits the full band decision on every render ("applied" is the
counterfactual "would have inverted"), a follow-up flips the default.
Identical cohort selector on both sides of the boundary, and the
skipped/oversize renders in the same frame band form a concurrent
control — a difference-in-differences that absorbs secular drift
(content mix, version-correlated populations, hardware), which a plain
before/after cannot.
Also: countElementTags now counts HTML void elements. Counting only
closers read an image gallery as a tiny comp and opened the band on
exactly the content most likely to lose it (images skew expensive to
paint). Opening tags stay uncounted — inline scripts' `a < b` would
false-positive. Counter semantics are frozen while the baseline is read:
the distribution the baseline release records must be measured by the
same counter that later gates.
Revert-rate baseline for the pre-registered read, measured over 14d
fleet-wide: the 900+ inversion runs 31,756 inverted / 1,705 reverted =
5.1%. At the benched 1.16-1.24x win and ~1.8x revert cost, expected net
for the band is ~12%. Kill criteria for the flip release: DiD <= 0,
in-band revert rate > 5.1% baseline, or band fallback rate > DE baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>