R6/R7 blockers.
An already-open Studio kept emitting server-side render telemetry after
another process disabled CLI telemetry. refreshTelemetryPosture() only ran
while serving a fresh SPA document and on /api/telemetry-identity, which
Studio has no consumer for, so the render POST and its async outcome used
the posture cached when the preview server booted. It now refreshes at the
render boundary and again immediately before the completion/error event,
so an opt-out during a long render is honoured.
The identity tests were passing vacuously: their mocks omitted
readConfigFresh and resetTelemetryPostureCache, and the resulting
missing-export error was swallowed by the refresh's own catch. Mocked
properly, plus the enabled -> external disable -> next response transition
and the suppression path at the layer that drops the event.
A full reset also did not persist its new lineage in a long-lived process:
syncInstallState returned early on a process-lifetime memo even after
~/.hyperframes was deleted, so install-state was never recreated and the
next config-only re-mint rolled a third seed instead of inheriting the
second. The memo is now revalidated against the file.
Also drops a stale reference to assertNoOverdueCanaries and stops the
workflow and docs claiming the sunset job routes anything to the owner —
it names them in the run log and notifies nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CodeQL flagged both on the new workflow: an unscoped GITHUB_TOKEN and an
unpinned third-party action. Matches the pins ci.yml already uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- A long-lived preview cached its telemetry posture in two places
(readConfig and shouldTrack). Running `telemetry disable` in another
terminal left it resolving canaries and injecting the CLI id for hours.
Both caches are now dropped together at a request boundary.
- Studio minted and shipped a telemetry id for every render regardless of
the browser profile's opt-out, and the server emitted the outcome under
CLI policy, which cannot see localStorage or DNT. The browser now sends
an explicit telemetryOptOut, distinct from an old client's omission.
- Any non-empty HYPERFRAMES_PREVIEW_HOST disabled the DNS-rebinding guard,
so even a loopback bind accepted a hostile Host. The guard now holds for
loopback binds and, on a LAN bind, admits only names this machine
answers on.
- sunsetAfter had no reader of the current date. A scheduled workflow runs
scripts/check-canary-sunset.ts weekly, so a failure lands on the
rollout's owner rather than on an unrelated PR author.
- The install-state seed memo outlived `rm -rf ~/.hyperframes`,
resurrecting a cleared cohort. Removed; it only saved a read on a
readConfig cache miss.
Docs updated for the Host rule and the 100% exclusion carve-out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both pages explained intensity's independence by saying details and
effects are applied after the u_intensity mix. The conclusion was right
but the mechanism was wrong for several families.
Verified in the shader: applyCrtWarp runs on the uv before sampling;
sampleMedia itself carries pixelate, chromaBleed and the tape family;
sampleChromaticMedia and applyDigitalGlitch shape sampleColor — all
before the mix at runtime/colorGrading.ts:1234. Only grain,
filmArtifacts, monoScreen, engraving, crosshatch, halftone, twoInkPrint,
bloom, scanlines and vignette run after it.
Pre-mix effects are already present on both sides of
mix(sampleColor.rgb, applyColorGrade(sampleColor.rgb), u_intensity), so
intensity does not scale them either. Both pages now say details and
effects sit outside the mix — some before, some after — without making
the ordering the reason.
Reported by miguel-heygen.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three issues in runFfprobe's process and stream handling.
A filePath of exactly "-" hung for 30 seconds. `--` stops option parsing,
so "-intro.mp4" is safe, but ffprobe rewrites "-" to `fd:` AFTER option
parsing and reads stdin — and stdin was an inherited pipe the parent
never writes to and never ends. The probe ran to the deadline and failed
with an empty diagnostic, because ffprobe never errored so stderr was
blank: 30010 ms and no message, against 28 ms for a normal missing-file
error. Rejected up front, and the child now gets stdio ["ignore", ...]
so no future invocation can block on stdin either.
stdout was decoded per chunk. `stdout += data.toString()` decodes each
64 KiB pipe chunk independently, so a multi-byte character straddling a
boundary became U+FFFD on both sides — verified: 200 KB of 3-byte
characters produced 15 replacements and a string 9 characters longer
than the source. -show_format output above ~64 KiB with non-ASCII tag
text returns silently mangled values, since JSON.parse still succeeds.
Now accumulated through StringDecoder.
Note on testing that one: U+FFFD is valid JSON string content, and
nothing on extractMediaMetadata's public surface exposes a tag value, so
there is no assertion that fails against the old implementation. Rather
than add a test that cannot fail, it is stated here and the bound below
is what the new test covers.
stdout was unbounded. stderr is capped by ManagedChildProcess but stdout
was not, and analyzeKeyframeIntervals emits one line per frame — an
all-intra ProRes proxy can produce an arbitrarily large string. Capped
at 8M characters, which real -show_streams JSON is nowhere near.
Tests: "-" rejected without spawning, the stdio shape, and the size
bound. Reverting the stdin guards fails 1. The first draft of the bound
checked before appending, so a single oversized chunk passed — the test
caught it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous gate was a HE-AAC DENYLIST, so every other profile still
got the 1024-sample formula. ffprobe reports codec_name "aac" for all of
them; the framing lives in the profile:
LC 1024 samples/frame <- the only one this maths fits
HE-AAC v1/v2 2048 output samples against a doubled sample_rate
LD 512
ELD 480
Main/SSR/LTP 1024 nominally, unverified here
xHE-AAC variable
LD and ELD therefore had their already-correct container duration
overwritten with a value 2x / ~2.13x too large, and an unknown or
missing profile fell through — so an unrecognised HE spelling preserved
the exact truncation the previous commit set out to close.
Now an affirmative match on LC. Skipping the refinement is harmless:
format.duration is already correct before it runs.
Tests: 11 non-LC profiles (including LD, ELD, xHE-AAC, empty and
unrecognised) assert the container duration is kept AND that the second
probe is not launched; LC still refines, with whitespace tolerated. The
pre-existing duration table asserted that an UNPROFILED "aac" stream
refines — the behaviour under review — so it now states LC explicitly
and adds an unprofiled row that must not refine.
Also strengthened the `--` separator test while it was failing: it
compared a flattened count of 3 across three spawns, which one call
emitting three terminators would satisfy. Now asserts the last two argv
entries per call.
Reverting the allowlist fails 8.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The packet-count probe is a refinement — durationSeconds is already
correct from format.duration before it runs — but it was written as if
it were load-bearing.
It could fail the whole call. No try/catch, and `-count_packets` demuxes
the entire container against runFfprobe's fixed 30s deadline, so a long
AAC file on slow or network storage timed out and extractAudioMetadata
rejected. htmlCompiler catches that under the comment "Source file has
no audio stream", returns duration 0, drops the audio element, and the
render ships silent with no warning. Now caught, keeping the container
duration.
It ignored the caller's AbortSignal. Only the first probe received it,
so aborting during the packet probe let the child run to completion and
the call resolved with full metadata after cancellation — while
audioPadTrim's comment claims the wrapper preserves cancellation. The
signal is forwarded, and an abort still propagates rather than being
swallowed as a refinement failure.
It halved HE-AAC durations. ffprobe reports codec_name "aac" for
HE-AAC v1/v2 as well — the marker is in the profile field — and with SBR
each packet carries 2048 output samples against the doubled output
sample_rate, so the 1024 assumption computed exactly half. A 10:00
podcast became 5:00 and htmlCompiler truncated the audio there. Gated on
profile, with `profile` added to FFProbeStream.
Tests: probe failure, junk output, three HE-AAC profile spellings (which
also assert the second probe is not attempted), and that plain AAC-LC is
still refined. Reverting the guards fails 5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two paths the previous guards still let through.
Rounding could recreate Infinity after the finite check. `raw * 100`
overflows for a finite-but-huge rate — "1e307", "1e307/1" — so `rounded`
became Infinity and passed the positivity check, reaching exactly the
`-r Infinity` failure the finite guard exists to prevent. The rounded
result is now checked too.
The rational operands still used parseFloat. The plain-number path
switched to Number() so trailing garbage fails the whole string, but the
numerator and denominator did not, so "60fps/1", "60/1fps" and
"30garbage/1garbage" returned valid rates while the contract says
malformed frame rates fail closed. Both operands are now parsed strictly,
and an empty operand ("/", "/1", "30/") is rejected rather than coerced.
Tests: 8 malformed inputs and 3 overflow cases in the direct table.
Reverting either fix fails 5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
parseFrameRate guarded its operands but not its result, so several
inputs produced values that are not usable frame rates — and nothing
downstream catches them, because callers use `meta.fps || 30`, which
only rescues 0 and NaN. Everything below was truthy and flowed into
buildEncoderArgs as `-r <value>` (rejected by ffmpeg mid-render) and
into frameCount arithmetic.
"1e308/1e-10", "2/1e-320" -> Infinity (finite operands, infinite quotient)
"-30/1", "30/-1", "-60" -> negative (sign never checked)
"30/1/2" -> 30 (parts.length !== 2 fell through)
"60fps" -> 60 (parseFloat stops at garbage)
Now: the quotient is checked rather than the operands, non-positive is
rejected, more than two parts is rejected, and the single-part path uses
Number() rather than parseFloat so trailing garbage fails the whole
string.
Separately, 2dp rounding collapsed any rate below 0.005 to exactly 0,
and the caller's 30fps default then re-encoded a 300-second 1/300-fps
timelapse as a ~1/30-second clip with frameCount 9000 for a 1-frame
file. Those floor to 0.01 instead.
parseFrameRate is now exported and tested directly. The previous table
drove it through extractMediaMetadata behind a spawn mock, costing a
vi.resetModules() plus a re-import of core's 238-file barrel per row
(74.9 ms vs 0.094 ms) — and 4 of its 7 rows produced identical values
against the pre-fix implementation, so it could not fail for the bugs it
existed to catch. The replacement fails 9 against that implementation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three defects in how ffprobe output and the PNG fallback are combined.
The cICP fallback was unreachable. `ffprobeColorSpace ?? stillImageMeta
?.colorSpace` discarded the PNG result whenever ffprobe returned ANY
colour field — and ffprobe emits color_space "gbr" for every PNG,
including a plain rgb24 with no colour metadata. So on the build the
parser exists for (reports gbr, does not decode cICP) an HDR PQ PNG
resolved colorTransfer "" , isHdrColorSpace() returned false, and the
still graded SDR. Now merged per field.
hasAlpha's anchor bound to one alternative. In
/(^|[^a-z])yuva|rgba|.../ the `|` is looser than concatenation, so
(^|[^a-z]) guarded `yuva` and nothing else. The list also omitted abgr,
ya8, ya16 and ayuv64, and `gray[a-z0-9]*a` matched only gray8a/gray16a —
names FFmpeg renamed to ya8/ya16 in 2013, so dead against modern builds.
A ya8 grayscale-plus-alpha PNG reported hasAlpha:false, resolveFrameFormat
picked jpg and the overlay flattened to an opaque rectangle. Replaced
with the start-anchored form studio-server already uses, extracted as
exported pixelFormatHasAlpha so the test asserts the shipped predicate
rather than a copy of the pattern.
The PNG parse ran eagerly and was discarded. It sat before the first
await, so readFileSync plus the CRC walk executed for every file before
a single ffprobe was spawned — a caller fanning out over
composition.images with Promise.all serialised entirely: 12 4K PNGs took
2649 ms against 170 ms probe-only, 2.5 s of event-loop stall that also
blocks Puppeteer IPC. On the happy path the value was then thrown away.
Now lazily memoized behind the paths that actually consult it.
Tests: 18 pix_fmt cases against the real predicate. Reverting the regex
fails 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both pages said intensity does not scale a grade and that 0.5 renders the
same as 0. That is wrong, and the error was mine: I tested intensity only
against twoInkPrint and tapeDamage, then generalised from an effects-only
result.
The shader mixes ungraded against graded at u_intensity
(runtime/colorGrading.ts:1234), so it does scale adjust, wheels, curves,
hueCurves, secondaries and the LUT. Runtime tests pin 0.25 and 0.75
reaching the uniform. What it does not scale is details and effects —
grain, filmArtifacts, monoScreen, engraving, crosshatch, halftone,
twoInkPrint and the tape/CRT families are all applied after that mix
(:1235-1262), which is exactly what I had measured.
Both pages now say intensity ramps the primary grade only, and to animate
the specific effect when the look is effect-based.
Reported by miguel-heygen.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Repoint the generated-artwork exit bridge at color-grading. The nav
inserted the new page between generated-artwork and vfx-and-liquid-glass
but the "Next" link still skipped it (miga-heygen, blocking).
- Correct the LUT custom property name: --hf-color-grading-lut-intensity,
not --hf-color-grading-lut. Real transcription error, surfaced while
checking the CSS-variable review comment against the source table in
packages/core/src/colorGrading.ts:921-931.
- Add #t=0.1 posterframe hints and a contextual italic caption to all ten
videos, matching the sibling prompting chapters.
- Reconcile British spelling to American throughout. The file path, nav
slug, page title and every cross-reference were already "color".
- De-duplicate the animatable-effect caveat list: the chapter now links to
the guide's Animating a Grade section instead of restating which effects
are verified working, so the list is maintained in one place.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
zlib.crc32 landed in Node 22.2.0, but engine and cli both declare
`"node": ">=22"` and the runtime gate is major-only, so 22.0 and 22.1
are supported. A NAMED import of a missing export throws at module
EVALUATION — ffprobe.ts would have failed to load at all on those
runtimes, before any PNG was touched, taking every probe with it.
Namespace import plus a capability check, with the previous
bit-at-a-time implementation retained as the fallback. Modern runtimes
keep the 210ms -> 1.3ms win; older ones keep working.
Raising the floor to >=22.2.0 was the alternative, but that is a
user-facing support change and does not belong in a PNG bug fix.
Tests: the same HDR PNG parses identically with the native export
absent, and a corrupt chunk still rejects on the fallback path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
R5 blockers
- Negative install-state latch was cached for the process lifetime, but
only `true` is monotonic across processes. A long-lived preview server
held a stale `false` and could re-enrol after another process tripped
the breaker. Only the positive is cached now; `false` re-reads.
- The real breaker writer used writeConfig(), which collapses
{ok:true, mirrored:false} to success, so a run that mirrored nothing
reported done with the latch only on the erasable store. It consumes
writeConfigWithResult and retries until both stores carry it.
Bucketing integrity
- Storage-restricted Studio profiles all bucketed on the literal
"anonymous": computed against the shipped hash, 100% of them were
enrolled in calibration-50 rather than 50%, and they merged into one
PostHog person. Per-session random id instead — persists nothing.
- bucketSeed had read/write authority backwards: install-state is
write-once authoritative, but readConfig took config.json's blindly, so
the stores could hold different seeds until a re-mint flipped every
cohort. Merged on read, like the latch.
- An unwritable ~/.hyperframes with no config.json re-minted per call,
re-rolling the seed on every command, and the "cohorts will not be
stable" warning was unreachable on that path.
- A corrupt PRE-MOVE state file was never deleted, so a machine reset
with `rm -rf ~/.hyperframes` reported predecessorFound/stateFileCorrupt
forever — poisoning the exact metric this work exists to produce.
Opt-out honoring
- CLI canary decisions memoized per process, so `hyperframes telemetry
disable` during a running preview server was ignored for hours while
the server kept serving pre-opt-out decisions. The memo is keyed on the
telemetry posture.
- shouldTrack() memoized, contradicting policy.ts's documented "not
memoized" contract that policy.test.ts asserts.
- The Studio override path resolved the bucket unit eagerly as an
argument, minting and PERSISTING a tracking id for an opted-out profile
— a value evaluateCanary discards unread.
- Storage reads could throw out of telemetry into a post-commit catch
block, reporting an already-committed edit as failed.
- readConfig printed an unsilenceable stderr warning on every invocation
for installs that opted out of telemetry entirely.
Host split
- isLoopbackHost rejected 0.0.0.0, so the documented
HYPERFRAMES_PREVIEW_HOST LAN mode silently lost CLI→Studio identity
stitching and split one user across two PostHog persons. Identity is
now allowed when the operator explicitly opted into LAN binding.
- Corrected the comment claiming the guard refuses spoofed Hosts: a
non-browser client sets Host freely. It is a browser DNS-rebinding
mitigation, not access control, and now says so.
Semantics and test hygiene
- percentage:100 did not mean everyone — exclude and no_unit_id sat above
the fast path, so the registry's "delete the entry at 100" step was an
unstaged flip for CI and seedless installs.
- CLI cohort adoption returned before evaluateCanary, dropping Studio's
own webdriver exclusion.
- overdueCanaries() was asserted against wall-clock time, so the whole
core suite would go red on 2026-09-15 for every unrelated PR; and `>`
against midnight made a canary overdue ON its sunset date.
- Statistical assertions ran on unseeded randomUUID() populations tight
enough to fail ~1 run in 200. Seeded.
Also: broke a config -> policy -> transport -> config import cycle by
moving POSTHOG_API_KEY to a leaf module.
Tests: 2347 CLI (bundle absent), 3153 Studio, 1450 core. Fault injection
covers the latch, seed authority, LAN identity, webdriver exclusion and
the anonymous-bucket fix. Two pre-existing tests asserted behaviour these
findings identify as wrong (shouldTrack memoization, 100%-excludes-CI)
and were rewritten with the reasoning stated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three defects in the PNG metadata fallback, all introduced when the
cICP early return became an accumulator.
Corrupt trailing chunk nulls a good result. cICP must precede IDAT, so
continuing past it only visits chunks this parser ignores — while making
whole-file integrity a precondition for returning anything. A truncated
or bad-CRC chunk after cICP in an otherwise-good HDR PNG returned null,
and extractMediaMetadata then re-throws the ffprobe error it had
swallowed instead of using the fallback it just computed: the render
dies on a host without FFmpeg, or grades SDR on a build that does not
decode cICP. Now stops once dimensions and colour are known.
A second IHDR overwrote the dimensions. PNG permits exactly one, first,
but nothing enforced that here — a trailing [IHDR 1x1] replaced a real
3840x2160 and the producer laid out a one-pixel image. Anchored to the
first. The length guard was also `>= 8` against a spec length of 13,
which accepted a truncated header and read height out of the CRC bytes.
crc32 was hand-rolled bit-at-a-time and fed a Buffer.concat per chunk.
Since the walk no longer stops early it CRC'd whole files: 210 ms on a
12 MiB PNG, 647 ms on a 35 MiB 4K one, synchronously on the event loop,
plus ~11 MB of garbage per parse from concatenating a 4-byte type tag
onto every chunk. node:zlib's crc32 is native and takes a running seed,
so type and data hash in sequence with no copy. 210.28 ms -> 1.291 ms.
Tests: 5 regressions — corrupt-after-cICP, truncation after cICP,
second IHDR, short IHDR, and that a corrupt IHDR/cICP still rejects.
Reverting the break or the anchor fails 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
P1 — the required Test lane was red, and it was my test. The
hostile-Host SPA case asserted a 200, which only holds when
packages/studio/dist is built: true on a dev box, false in CI, so it
passed locally and failed there. The Host split moved into a pure
buildStudioHeadScriptsForHost() and is asserted directly; the route test
no longer depends on build state. Verified by running the CLI suite with
the bundle moved aside — 2330 pass.
P1 — studio:* still bypassed most privacy controls. It honoured two
localStorage keys but not navigator.doNotTrack,
VITE_HYPERFRAMES_NO_TELEMETRY, Vite dev mode or API-key eligibility, and
canary enrolment honoured a different single control. New
telemetry/policy.ts is the one answer to "may this profile be measured",
consumed by both transports and by enrolment. It imports only ./config,
so no cycle with the modules that import it. Each control is asserted
individually.
P1 — LAN/remote preview lost the authoritative decisions. Withholding
the whole head script for any non-loopback Host also dropped the safe
{enabled, forced} map, sending a supported HYPERFRAMES_PREVIEW_HOST=
0.0.0.0 Studio back to re-deriving. Identity injection is now gated
separately from decision injection: identity is loopback-only, decisions
always publish.
P1 — the breaker latch was neither authoritative nor truthfully
persisted. syncInstallState swallowed its own failures so
writeConfigWithResult always reported ok, and reads took the flag only
from config.json. The latch is now merged into every effective read,
which makes install-state authoritative and closes both the failed-mirror
and stale-concurrent-writer paths; the write additionally reports
mirrored: false rather than swallowing.
P2 — public contracts. canary-rollouts.mdx said a config wipe loses the
breaker (it does not) and documented the superseded {name: boolean} map
with unconditional CLI precedence; both corrected, with the precedence
ladder written out and the override exception stated explicitly. PR body
rewritten — it still named ~/.local/state, claimed state survives
deleting ~/.hyperframes, and carried stale counts.
Tests: 2330 CLI (bundle absent), 3151 Studio, 24 core. Fault injection:
reverting each fix alone fails 5 CLI / 5 Studio.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guide covers the contract; the chapter covers the two failures the
contract cannot express — choosing a source that has something for the
treatment to remove, and separating a subject so part of the frame can
be graded while the rest is protected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ten A/B demos, each with the plain-language prompt that produced it and
the payload it compiled to. Slots into Level 4 — Substance.
The chapter teaches technique and leaves the key/bound reference to the
Colour Grading guide rather than duplicating it.
Also adds two sections to docs/guides/color-grading.mdx:
- Animating a Grade — the nine CSS custom properties, plus driving the
payload from the timeline for effects that have none. Documents that
--hf-color-grading-intensity does not scale a grade at render time,
and that payload-rewrite animation is effect-dependent: verified
working for halftone and twoInkPrint, verified not working for
crtCurvature, scanlines, chromaBleed and chromaticAberration.
- Limiting a Grade to Part of the Frame — grading qualifies by value,
never by screen position, so a region has to become its own layer.
Includes the three layer recipes and a worked face-redaction example.
Previously the support matrix said "not supported" with nowhere to go.
Renders are served from the CDN; docs/images/ is gitignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
P1 — SPA route bypassed the DNS-rebinding guard. Guarding only
/api/telemetry-identity left the catch-all as an open side door: a
rebound origin could fetch `/` and read __HF_CLI_DISTINCT_ID and
__HF_CLI_BUCKET_SEED straight out of the returned HTML. The SPA response
now applies the same isLoopbackHost() check; an untrusted Host still gets
a working Studio, just with no identity, seed or decisions injected.
Route-level regression added.
P1 — a CLI cohort roll could override Studio's own opt-out.
decideStudioCanary() adopted the injected decision before checking
isOptedOut(), so CLI-telemetry-on plus Studio-opted-out still enrolled
Studio. A bare boolean could not express the difference between a
deliberate override and an ordinary cohort roll, so the injected map now
carries provenance ({ enabled, forced }). Forced wins outright — it is
the documented escalation channel and must behave the same on both
surfaces — while a percentage roll now loses to this profile's opt-out.
Full interaction matrix tested.
P1 — the legacy studio:* path sat outside both contracts.
utils/studioTelemetry.ts shipped its own opt-out key and its own send
loop, so the documented hyperframes-studio:telemetryDisabled did not
silence it and its events carried no cohort assignment. It now honours
both keys (the legacy one stays, so nobody already opted out is quietly
re-enabled) and mixes in canaryEventProperties(), making "every
telemetry event carries the assignment" actually true.
P2 — partial salvage could drop a tripped breaker.
salvageInstallState() discarded the whole record when markerAt and
bucketSeed were both unusable, taking deParallelRouterTrialFired with it
and re-enrolling a machine whose router already failed. All three fields
are now independently salvageable.
Docs: canary-rollouts.mdx said "disabling telemetry disables the
reporting, not the enrolment" — exactly backwards since the opt-out gate
landed. Corrected; checked for other copies, none.
Tests: 13 new (4 opt-out precedence, 4 legacy-path opt-out and canary
props, 3 route-level host guard, 2 breaker salvage). Fault injection:
each of the four fixes reverted independently fails its own tests
(2 CLI + 1 Studio + 2 Studio).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second half of a review comment I had only half-addressed: the opt-out
behaviour shipped in 4f464dc4, but "should we add this into/connected to
the existing telemetry doc?" did not.
There is no standalone telemetry page — the canonical disclosure is the
`telemetry` command section in packages/cli. Links now run both ways so
neither page describes canaries as a separate system:
- packages/cli #telemetry: telemetry state also controls canary
enrolment, with every opt-out route named.
- contributing/canary-rollouts: a Note at the top, before any of the
how-to, saying canaries sit behind the telemetry switch and why (a
canary is a measured rollout; an install that reports nothing cannot be
compared, so enrolling it buys no signal).
- guides/feedback "Opting Out": disabling telemetry also ends canary
enrolment, alongside the feedback prompt and usage tracking.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six findings from review, none behaviour-critical on their own but three
of them quietly corrupt the data the rollout is judged by.
Endpoint no longer serves bucketSeed (studioServer.ts). Studio gets its
canary answers from the injected decisions map now, so nothing needed the
seed over HTTP — and an unauthenticated local endpoint is a strictly
worse place for it than a script scoped to Studio's own document. The
endpoint itself predates this PR and still serves distinctId, so it also
gains a Host guard: a remote page can rebind its hostname to 127.0.0.1
and read the response as same-origin, but the request still carries THAT
hostname, which is what makes it refusable.
predecessorFound no longer reports corruption as a fresh install. It
returned null for both "file absent" and "file unreadable", so a partial
disk write looked like a new machine — understating recoverable churn,
the one thing the field measures. Now distinguishes absent from corrupt
and emits install_state_file_corrupt alongside.
A mangled markerAt no longer discards a salvageable bucketSeed. markerAt
is only a timestamp and can be restamped; the seed cannot be recovered,
and losing it silently re-rolls the install's cohort.
The seed backfill no longer ignores its write result. An unwritable
~/.hyperframes meant a different seed every invocation with no
diagnostic, and made the field's own "backfilled once" docstring false.
Warns once per process with the underlying error.
FNV-1a's ASCII constraint is now explicit rather than incidental. It
hashes UTF-16 code units while reference FNV-1a is byte-oriented, so the
two agree only on ASCII; the registry's kebab-case assertion is what
makes non-ASCII unreachable, and both ends now say so. Not a live bug —
names are kebab-case and units are UUIDs.
de-parallel-router is pinned at 0%. The registry is data, so a ramp is a
one-line edit with no review surface, and its own description says to
ramp only alongside the circuit breaker.
Tests: 8 new (corruption vs absence, seed salvage, backfill write
failure, 17 host-guard cases, registry pin). One existing test asserted
predecessorFound: false on corruption — that was the bug, updated with a
note. Fault injection: restoring the old corrupt handling fails 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes both cross-surface findings with one mechanism. The CLI publishes
window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }); a CLI-launched
Studio takes it as authoritative over its own seed, URL override and the
registry percentage.
Studio re-deriving could not agree with the CLI in three cases:
- Telemetry off. The CLI resolves telemetry_opt_out, but Studio's
opt-out is a separate localStorage flag it cannot see, so it would
evaluate normally and could enrol on a render the CLI excluded. The
previous commit gated each surface independently; that fixed silent
enrolment per surface but NOT the disagreement between them.
- HF_CANARY_* override. Env vars never cross into the browser — Studio
reads only its URL param / sessionStorage — so a support session
forcing a canary on got the CLI forced and Studio guessing.
- No seed injected. Studio falls back to a different unit id, i.e. a
different bucket.
Shipping the decision instead of the inputs makes divergence structurally
impossible: one evaluation, two surfaces. It also exposes strictly less —
booleans about features, rather than the seed buckets derive from — which
is why it is safe to publish with telemetry off, the case it exists for.
Studio still evaluates locally when standalone, or for a canary the CLI
did not publish, and ignores a non-boolean value rather than trusting it.
Tests: 6 Studio (CLI-off wins over unset local flag, CLI-on with no URL
param, beats contradicting override, beats seed, falls back per-canary,
rejects non-boolean) and 4 CLI (decisions with telemetry off and no
identity, alongside identity when on, script-tag escaping on a hostile
canary name, throwing resolver degrades to identity only). Four existing
identity tests asserted the old "nothing when telemetry off" contract and
were updated; the registry is now mocked there so string assertions don't
move when a canary is added or ramped. Fault injection: dropping the
adoption fails 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both reviewers flagged the same gap: seed injection was gated on
telemetryShouldTrack(), but canary EVALUATION was not. An install with
DO_NOT_TRACK=1 was still bucketed and still had real code paths flipped
(e.g. HF_DE_PARALLEL_ROUTER), silently and unmeasurably.
A canary is a measured rollout — we enrol a slice precisely so it can be
compared against everyone else. An install that sends nothing can't be
compared, so enrolling it buys no signal and only changes that user's
code path, on an experimental feature, without their knowledge. That is
the wrong side of an opt-out.
Resolves to a new `telemetry_opt_out` reason BEFORE bucketing, so no
cohort is assigned at all. Distinct from `excluded` because "why is my
canary off" has a very different answer for CI than for opted-out, and
the reason never reaches telemetry by construction.
Covers every opt-out route: persisted preference, the runtime env vars
and dev/telemetry-disabled builds via policy.ts, and Studio's
hyperframes-studio:telemetryDisabled.
An explicit HF_CANARY_* / ?hf_canary_*= override still wins — a
deliberate local choice, not silent enrolment, and the documented way to
exercise a canary with telemetry off.
The CLI check mirrors shouldTrack() rather than importing it: client.ts
already imports canary.ts for canaryEventProperties, so depending on it
would be a cycle. Both read the same two inputs, so they cannot disagree.
Tests: 9 new across CLI and Studio (preference off, each runtime
override, no bucket assigned, override still honoured, flag properties
all-false). Fault injection: removing the CLI gate fails 6, removing the
Studio gate fails 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adopts the #2904 pattern for bucketSeed. The seed rides in
install-state.json, which now lives beside config.json in ~/.hyperframes
rather than in ~/.local/state/hyperframes/.
Review rejected persisting state outside the config dir to defeat a
user's reset, and that objection is sharpest for the seed: it is the one
field that would turn install-state into a persistent pseudonymous
identifier surviving `rm -rf ~/.hyperframes`.
The carryover still earns its place, just against the churn that
actually happens. config.json is rewritten on every command and every
render, and readConfig recovers from any parse/permission/IO failure by
minting a fresh identity — so a re-mint would reshuffle cohorts
mid-rollout. A no-schema file written once at mint is decoupled from
that without leaving the directory. Config re-mint: cohorts hold.
Directory deleted: cohorts go too, deliberately.
A pre-move seed is adopted by the same migration, so installs already
carrying one do not have a live cohort reshuffled under them.
Tests: seed survives a re-mint, does NOT survive deleting the config
dir, and migrates from the pre-move path. Prose in config.ts, canary.ts
and canary-rollouts.mdx corrected — it still claimed cohorts survive a
wipe. Docs gain a removal-path section.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cohort membership now survives a config wipe. Canaries bucket on a
dedicated bucketSeed (fresh random UUID, distinct from anonymousId by
design) that is mirrored write-once into the install-state file and
inherited at mint: a wipe re-rolls the telemetry id but never the canary
assignment. This removes cumulative-exposure drift for the recoverable
churn bucket entirely — the residual drift comes only from fresh
machines, containers, and genuinely new users — and keeps before/after
comparisons valid across a reinstall.
The seed is never emitted in telemetry (only the resulting true/false
assignments are), so it does not link the old id to the new one
server-side. The residual linker is the flag vector itself (k bits for k
live canaries), documented as such. An explicit reset still works by
deleting the state file, and the no-identity test now also asserts the
seed differs from the anonymousId.
Cross-surface coherence: the CLI's studio server injects the seed as
window.__HF_CLI_BUCKET_SEED (same telemetry gate and script-escaping as
the distinct id, and on the /api/telemetry-identity fallback), and the
Studio binding buckets on it when present — without this the CLI would
bucket on the seed while Studio bucketed on the distinct id, splitting
one machine across cohorts (calibration check 4 would catch exactly
this). Standalone Studio still buckets on its localStorage id: the
browser has no second storage location, so that id doubles as the seed.
Legacy configs are backfilled once (lineage seed if the state file has
one, else minted) and persisted immediately — an unpersisted seed would
re-roll cohorts every process. Safe to ship in the same release as the
first canaries: no prior release emitted canary properties, so the
bucketing-unit change is unobservable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The "what cannot be fixed" framing predated the state-file rollover: the
drift still exists, but a re-rolled install no longer re-enters a failed
path, and install_predecessor_found splits the drift into recoverable vs
unrecoverable. Window-length policy comes from the residual, not raw
turnover.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributors in the wild have no access to the project's analytics backend,
so the contributing doc must not point them at it: drop the internal
dashboard link, the backend-specific SQL blocks, and the vendor naming.
The calibration check definitions stay (public, fixed terms of the
experiment); the queries live with the dashboard tiles that run them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Telemetry carries the assignment but not the decision reason, so a manual
HF_CANARY_* toggle mid-window reads as a cohort flip. Rule it out before
treating a small-nonzero stability read as a mechanism bug; a reason
property is deliberately deferred until the check actually comes back dirty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
As written, check 4's condition (dual-surface AND value flipped anywhere)
was a strict subset of check 3's (value flipped) — if check 3 read zero,
check 4 was vacuously zero and added no independent signal. Compare the
value each surface actually reported instead, and state plainly that any
disagreement is also a check-3 flip: this check's job is attributing such
a flip to binding divergence. Matching fix applied to the live PostHog
tile (insight jQi7QdW1, dashboard 1918875).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verified against packages/*/src env reads: 57 distinct HF_*/PRODUCER_*
toggles pre-existing this branch (the raw grep said 59, but two of those
are HF_CANARY_TEST_* fixtures introduced by this branch's own tests).
The number is cited externally now, so it should match what the repo
actually has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four pre-registered checks now exist as PostHog tiles. Without the link
the doc describes queries someone has to re-type; with it the doc and the
dashboard are one artifact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Registers two canaries that gate nothing — `calibration-10` (10%) and
`calibration-50` (50%) — so the rollout mechanism can be proven against real
traffic before any real feature depends on it. Zero behavioural risk: they are
read by nothing.
They answer what the unit tests structurally cannot. The tests bucket
generated UUIDs and weight every install equally; real render volume is
heavily skewed toward a few heavy installs, and real install ids churn (~25x
more distinct ids over 30 days than in any single day on the desktop render
population).
Four checks, pre-registered in the docs so the read is not post-hoc:
1. ACCURACY — does 10% land at 10%, install-weighted AND event-weighted?
2. DRIFT — how fast does CUMULATIVE exposure climb above target as ids churn?
The instantaneous share is flat by construction; the set of installs
enrolled at some point is not.
3. STABILITY — does any install ever change cohort? Must be zero. Percentages
are held FIXED for the window precisely so a flip is unambiguously a bug;
during a real ramp a false->true flip would be correct instead.
4. CROSS-SURFACE — do the CLI and Studio bindings agree for the same install?
A CLI-launched Studio adopts the CLI id, and 16,961 installs currently
share an id across both surfaces, so this is measurable.
Plus an independence check: overlap between the two calibration canaries
should be ~p1*p2 (~5%), not ~min(p1,p2) (~10%, which would mean every canary
lands on the same unlucky cohort).
The docs also record what calibration CANNOT fix: per-install cohorts never
flip, but a person who wipes their config gets a new id and a fresh roll.
Preventing that needs stable identity across resets, and both candidates were
rejected — hardware fingerprinting correlates the cohort with hardware (fatal
for a rendering experiment, and it survives uninstall) and account identity
covers only ~3.6% of local rendering installs. The drift is therefore a
measured, accepted limit, and the point of calibrating is to size it and pick
canary window lengths accordingly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the single `canaries: "a,b"` telemetry property with PostHog's own
flag shape, one property per registered canary:
$feature/canary-de-parallel-router: "true" | "false"
PostHog treats `$feature/<key>` as a first-class flag property, so breakdowns,
funnels split by cohort and the experiment surfaces work on a canary with
nothing configured server-side. The decision still happens locally: the render
path forbids render-time network calls, behaviour must not depend on analytics
being reachable, and neither the CLI nor Studio ships posthog-js (both
hand-roll a batch POST, so there is no SDK to evaluate a real flag with).
Decide locally, analyse natively.
Two decisions worth recording:
- BOTH ARMS ARE EMITTED. A non-enrolled install reports "false" rather than
omitting the property. Absent means "this build predates the canary", which
is a different fact from "this install is control" — collapsing them makes a
ramp unreadable, because you cannot separate a control group from an old
version.
- KEYS ARE NAMESPACED with a `canary-` infix. A real PostHog flag namespace
already exists in this project, owned by the web app (`enable-chat-tab`, set
by posthog-js from `$lib=web` events). Namespacing guarantees a canary key
can never alias a real flag key and have the two fight over one property.
Values are the strings "true"/"false" to match how PostHog records boolean
flag values, so the property is directly comparable to a real flag.
98 core / 1437, 166 cli / 2194, 269 studio / 2982 green; tsc clean across all
three packages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the Studio (browser) binding so a canary can span the CLI and the editor,
and fixes a bundling mistake the studio test suite caught.
## The binding
Same public API as the CLI — `isCanaryEnabled("name")` — so a call site reads
identically whether it runs in Node or the browser. Three inputs differ:
- UNIT ID: `resolveStudioDistinctId()`, which already adopts
`window.__HF_CLI_DISTINCT_ID` when the CLI launched Studio. A CLI-launched
Studio therefore lands in the SAME cohort as the CLI: a rollout spanning
render and editor is coherent for that user instead of enrolling their
terminal but not their editor. A test pins that the id is passed through
unmodified — prefixing or re-hashing it would silently break that parity.
- OVERRIDE: no `process.env` in a page, so `?hf_canary_<name>=on` mirrored
into sessionStorage. Session scope is deliberate. A URL is the right carrier
(shareable — "support: open this link"), but persisting a URL-borne override
to localStorage would let one click silently pin a browser into a cohort
forever, long after anyone remembers why. Closing the tab is the reset;
`=reset` clears it explicitly.
- EXCLUSION: `navigator.webdriver` stands in for the CLI's `is_ci`. Automated
browsers mint a fresh localStorage id per run, so they would hop cohorts
between runs — noise in the signal, nothing learned about real users. An
override still reaches them, which is how you test a canary under Playwright.
Studio's `trackEvent` now attaches `canaries` to every event, mirroring the CLI.
## The bundling fix
Importing the `@hyperframes/core` barrel into studio browser code broke two
unrelated hook test files with an esbuild TextEncoder invariant violation. The
barrel re-exports the whole core surface (parsers, lint, studio-server), so it
drags a Node-oriented dependency graph into a browser bundle — the test
failure was the symptom, the bundle bloat was the bug.
`@hyperframes/core` now exposes `./canary` and `./canary-registry`, declared in
packages/core/package-subpaths.json (the generated source of truth for exports —
hand-editing package.json is reverted by the sync script) and marked
`environments: [browser, bun, node]`. Both the studio AND cli bindings import
the leaf modules; the CLI gets the same benefit for a different reason, since
this resolves on the startup path — the reason the producer is lazily loaded.
Verified: the two hook files pass again; 269 studio files / 2982 tests, 98
core / 1433, 166 cli / 2194 green, `bun run lint` clean including the subpath
check. Fault-injection confirms both design decisions are pinned — swapping
session for local storage fails the scope test, prefixing the unit id fails the
CLI/Studio cohort-parity test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up to the canary primitive.
Telemetry: every event now carries a `canaries` property listing the cohorts
the install is enrolled in, attached in `trackEvent` so it lands on ALL
events rather than renders only — a staged rollout is only as useful as the
ability to split any metric by cohort. Resolved after the shouldTrack guard,
so opted-out installs never pay for it, and omitted entirely (not null or "")
when the install is in no canary, since PostHog treats those as real values.
Test hardening, after validating the shipped code against 60k synthetic and
101 real fleet install ids:
- Pin FNV-1a against canonical vectors, AND assert the shipped canaryBucket
actually uses that hash. Without the second assertion the first is
tautological — it would only prove the test's own copy is correct while
canary.ts drifted to a different hash, silently reshuffling every live
cohort. Fault-injection confirms only this assertion catches a hash change;
the distribution tests stay green because a perturbed hash is still
well-distributed.
- Tighten the share test from a 0.6x-1.4x band to +/-1 percentage point.
Measured error was 0.16pp at n=60k, so the old band would have passed a
badly skewed hash.
- Add chi-square uniformity across all 100 buckets (chi2 89.0 vs 148.2
critical at p=0.001). A lumpy hash yields roughly the right total share
while overloading some buckets, so the share test alone cannot catch it.
- Assert N concurrent canaries enrol binomially rather than in lockstep:
8 canaries at 10% put ~43% of installs in none and zero in all eight,
matching binomial(8, 0.1). Correlated slices would put ~10% in all eight.
Also verified 88,443 of 88,448 fleet install ids are well-formed UUIDs; the
5 that are not fail closed, which is the intended direction.
Docs: docs/contributing/canary-rollouts.mdx, registered in docs.json (an
unregistered page is invisible in the nav).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds a reusable staged-rollout primitive so a change can ship to a stable
slice of installs instead of all-or-nothing.
The gap it fills: the repo carries ~49 HF_*/PRODUCER_* booleans and every one
is binary — a feature is either off (and therefore unexercised on real
traffic) or on for everyone (and therefore a fleet-wide bet). The
parallel-drawElement router sat in that gap for weeks: default-off produced
almost no signal, and flipping it default-on would have exposed 100% of
eligible installs at once.
Shape:
- packages/core/src/canary.ts — pure evaluator. No fs, no network, no
`process`; the caller supplies the unit id and overrides, so it imports
cleanly into the CLI, producer, engine, studio-server, the browser-side
studio bundle and the embeddable player. FNV-1a rather than node:crypto for
the same reason.
- packages/core/src/canaryRegistry.ts — every rollout in one table (name,
percentage, owner, description, sunsetAfter), so "what is rolling out, to
whom, owned by whom" is answerable without grepping 49 env vars.
- packages/cli/src/telemetry/canary.ts — supplies the three things only the
CLI knows: anonymousId, the HF_CANARY_<FEATURE> override, and is_ci.
Day-to-day API is `isCanaryEnabled("name")`.
Three properties the tests pin, because getting them wrong is subtle:
- Slices are INDEPENDENT per feature: the bucket hashes `feature:unitId`, not
the id alone. Bucketing on the id would hand every concurrent experiment to
the same unlucky cohort and make two rollouts unreadable apart.
- Ramping is INCLUSIVE: `bucket < percentage`, so widening 10 -> 25 keeps the
original cohort and before/after comparisons survive the ramp.
- It fails CLOSED: no unit id, unknown name, or CI install means not enrolled.
A canary exists to bound blast radius, so "we don't know who this is" must
never mean "enrol everyone".
Registry entries also carry a sunset date, and a test fails once one is past
due — a canary that outlives its rollout is a permanent fork of the product
with none of the review a permanent fork would get.
Ships with de-parallel-router registered at 0%: inert, and ready to ramp in a
patch release once #2840 lands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The old demo argued that a locked frame crops the story. The sharper claim is
what a cut throws away: a move carries the spatial relationship between two
shots, a cross-fade carries none.
Five stages each get their own colour and the halves diverge on how the next
colour arrives -- in place, or by travelling to it. 30-47% of pixels differ,
against 3% for every version where the WITHOUT half moved the content.
That 3% is the trap now documented in the rule: translating the scene and
translating the camera are the same transform, so sliding content past a fixed
frame is not a static comparison, it is the same shot rendered twice.
Embed is cache-busted to -v2; docs images carry immutable year-long cache, so
reusing the filename would serve returning visitors the old render.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CodeQL (incomplete multi-character sanitization, code-scanning/803) on
the script-stripping regex added in c61a24b51 — a failing check, and
correct: a single replace can reform the pattern it just removed, since
`<scr<script>ipt>` leaves a whole `<script>` behind.
The security framing does not apply — the stripped string is counted and
discarded, never rendered, inserted, or served — but the incompleteness
is real for this use: a reformed tag survives into the match pass and
perturbs the element count the routing gate reads. Suppressing a gate
over a technicality when the fix is four lines is the wrong trade.
Now loops to a fixed point. Terminates by construction: each iteration
either strictly shortens the string or changes nothing and exits.
Regression covers the reform case and an unterminated `<script>` that
must not spin; fault injection confirms the reform test fails under the
old single pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review findings on #2891. Two of them bite directly on this PR's own
purpose — making the fleet element-count distribution readable — so they
are fixed rather than noted.
countElementTags counted `</` + letter anywhere, including inside inline
JS. A compiled comp containing `const h = "</div>"` or a template literal
building `</span>` inflated the count once per occurrence. Compiled comps
embed large inline scripts, so the bias is systematic, not noise, and it
lands entirely on the ~83% of renders with no probe session — precisely
the cohort this PR exists to characterize. Script and style bodies are
now stripped before matching; losing their own closing tags costs 1-2
counts against a threshold in the thousands.
The new elementCount fell back to 0 when its page.evaluate threw,
following the tweenCount pattern beside it. For this field that pattern
is wrong: evaluate failures concentrate on the huge-DOM compositions the
field is meant to observe, and a 0 there is indistinguishable from a
legitimately empty comp, so the fleet p50/p99 would absorb both silently.
It is now undefined on failure, the INIT console line omits the token
entirely rather than emitting a zero, and the parser reports absent —
mirroring the live/static provenance split the routing resolver already
uses.
Also documented: the "every render reaches this path" claim holds only
for renders that survive to end of init, so the tail is survivor-biased
and should be read as a lower bound; and the two element-count fields now
say plainly which is which — composition_element_count gates routing,
observability_init_element_count is the observational counterpart — so
the follow-up analysis can't query the wrong one.
Nits: envInt is integer-only per its name, both live-DOM reads use
getElementsByTagName (live collection length, no NodeList materialized on
the 40k-node tail), and the attribution block notes that it runs with
routing off by design.
Fault injection confirms the new tests bite: disabling script stripping
fails 4, and the zero-vs-undefined case is pinned separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The short-comp routing gate can only read a live element count when a
probe session exists, and the first v0.7.83 data shows that is far rarer
than estimated: 17% of renders (86/503), not the ">=28%" the video-presence
proxy suggested. The other 83% fall back to a static source scan, which
is exactly blind to the shape that motivated the live count — small
markup, thousands of script-created nodes.
That leaves the fleet element-count distribution unknowable for most
renders, and the observed distribution is already surprising: p99 ~900,
max 1,420 against a 2,500 ceiling calibrated on 7k/20k/40k synthetic
nodes. Either the ceiling is close to irrelevant, or the large-DOM tail
is hiding in the 83% we cannot see. Both readings change what PR B
should do, and neither is decidable from probed renders alone (they are
a biased sample — they got a probe *because* they carry media or
unresolved compositions).
So measure it where every render already goes: capture-session init.
`collectSessionInitTelemetry` gains a querySelectorAll("*") count beside
the tween count it already collects, riding the same channel to
`observability_init_element_count`. This is observational only — capture
has begun, far too late to route on — and it deliberately does not feed
the gate. It answers the distribution question the gate cannot.
Coverage for this channel is proven rather than assumed: the tween-count
fix that shipped in v0.7.83 took the clamped-parallel bucket from 0/272
renders to 217/217, and 23.1% -> 100% overall.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Live telemetry panel, an isometric deploy pipeline wider than the frame, and
a progressive edge rollout across twelve regions. In each, the rule follows
from the content rather than being applied to it.
Rule 2's demo carries the strongest evidence in the chapter: deleting the
three stages the locked frame never shows and re-rendering leaves that half
identical on all 180 frames.
Rule 3 gains a rule-level distinction the rebuild surfaced — stagger what is
happening, not what is merely present.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the generic card/chip/disc subjects with ones where the rule is
unavoidable given the content: a payment sheet over a receipt, a split-flap
departure board, and a render-farm corridor.
Rules 5 and 6 in the grammar list were one-liners that under-said what their
demos prove. Rule 5 now carries the mass claim, the follow-through lag, and
the transforms-only caveat (a counter that overshoots renders a false value).
Rule 6 credits occlusion over blur and prefers one foreground element.
Section intro no longer endorses over-cranking; every movement in an applied
half has to finish "this moves because...".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
EaseCurveSection.tsx was 635 lines against CI's 600-line cap, which has
been failing the File size check on main for four consecutive runs and
blocks release cuts.
Moves the two self-contained presentational pieces into a sibling
EaseModeControls.tsx, following the pattern the directory already uses
(easeCurveSvg, easePresetLibrary, EaseParamFields): the mode radio group
(EaseModeToggle) and the preset grid (EasePresetGrid), plus the mode
vocabulary they own — EASE_MODES, the EaseMode type, MODE_LABELS,
DEFAULT_EASE_BY_MODE, and the DEFAULT_CURVE/Pts pair those defaults are
built from. Both components are stateless: they take the current
selection and emit a committed ease string, so nothing had to be
rewired. Only the symbols the parent still references are exported —
EASE_MODES and DEFAULT_EASE_BY_MODE became file-internal, since the
components that consume them moved too.
No behaviour change. EaseCurveSection is now 556 lines, the new file 110.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
R4 review finding, and the comment I wrote in R3 was simply wrong: the
probe session is NOT running for every render. probeStage's needsBrowser
gate launches one only for unknown duration, unresolved compositions, or
specific media cases — and hasRuntimeInsertedMedia matches only
createElement("video"|"audio"), never createElement("span"). So the exact
shape that motivated the live-DOM fix (a known-duration, media-free
caption comp building thousands of nodes in script) gets NO probe, falls
back to the static source scan, reads as ~2 elements, and could enter the
applied cohort at 40k live nodes. The R3 fix measured the right thing but
only for the population that already had a probe.
Now the count carries provenance and the band fails closed:
- resolveCompositionElementCount returns { count, source: "live" |
"static" }. Only "live" — an actual DOM measurement — may open the band.
- resolveDeShortBand gains a third decisive outcome, "unmeasured", for
the static case. It deliberately does NOT report skipped_elements: a
static undercount is not a real oversize observation, and putting it in
the control arm would contaminate the DiD just as putting it in the
treatment arm would. Neither cohort; never routes.
- composition_element_count_source ships alongside the count, so the
fleet rate of "static" sizes the population a future
conditional-probe-launch would unlock — which is the data PR B needs to
decide whether that launch cost is worth paying.
Regression coverage walks the real chain rather than a full render, using
the production functions in pipeline order: probeRequiresBrowser (newly
extracted from the inline needsBrowser expression, so the gate is
testable at all) returns false for the caption-comp shape → the resolver
reports static and a count under the ceiling → the band reports
unmeasured, not applied. Fault injection confirms it bites: removing the
one guard line fails exactly these three tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
R3 review finding: a string scan of compiled.html — however the regex is
tuned — cannot see elements a composition's own script creates at
runtime. The repo already has a production shape that hits this exactly:
style-10-prod's per-transcript-word caption generator builds one <span>
per word via document.createElement, measuring 2 source tags against
thousands of live nodes after init. That's the same unbounded-undercount
failure class as the earlier <img>/SVG counterexamples, but this one
has no static-scan fix — the elements simply don't exist as tags in the
string.
resolveCompositionElementCount() now prefers the live DOM size, queried
from the probe session that's already running for every render at this
point in the pipeline (its Chrome gets reused for capture on the common
single-worker path, so this costs one extra CDP evaluate, not a browser
launch) once that session's init sequence has completed — session.page
.evaluate(() => document.querySelectorAll("*").length) sees runtime-
generated DOM the source scan never could. countElementTags remains as
the fallback for the rare case with no initialized probe session
(evaluate throws, session absent, or not yet initialized).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both reviewers (Rames, Magi) independently flagged the same thing: by the
time the return statement executes, hadFired is always false — the guard
above already returns early for every case where hadFired was true. The
merge expression wantFired || hadFired || undefined was defensively
correct but misleading; it reads as "OR the two together" when the
function has already established only one of them can be true here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two review-blocking issues from Miguel's R2 pass (both confirmed by
running the counterexamples directly):
1. countElementTags still undercounted unboundedly. The void-element fix
covered HTML tags but SVG elements (<circle/>, <path/>, ...) are
neither closing-tag-shaped nor in the HTML void list, so
"<circle/>".repeat(40000) reported 0 — the same failure class as the
original <img> counterexample, and the exact shape of comp the
measured 1.8x regression case is made of. A 2500 ceiling cannot bound
an error with no bound of its own. Added a third alternative matching
any self-closing tag; verified it doesn't false-positive on the
adversarial minified-JS case (unspaced "<b/c>", which reads like a
tag open but never contains the literal two-char "/>" the alt requires).
2. HF_DE_SHORT_MAX_ELEMENTS=0 (the documented kill switch) still reported
deShortBand: "skipped_elements" for every in-band render instead of
undefined — attributing "comp too large" when the real cause was "band
disabled," which would have polluted the DiD control cohort with
kill-switched renders and made the post-flip read look like the
ceiling was too tight. Extracted the attribution logic into
resolveDeShortBand(), a pure function gated on bandEnabled
(deShortBandMaxElements > 0) as well as decisiveness — and made it
independently unit-testable, since the inline version could only be
exercised by a full render pipeline run.
Also from this review round: the inversion log line could report
"400 frames >= 900" for a band-routed inversion; it now names the floor
that actually decided the render. Tightened shortBand's type to match its
peer fields' unions (workerInversion, parallelRouter) instead of a bare
string. Clarified the tween-count merge docblock, which claimed workers
always agree (semantically true) while the code takes a defensive max
(in case one doesn't) — the two aren't in conflict, but the comment read
as if they were.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The routing surface the short-comp benchmarks validated is (motion x DOM
size x frames). After the baseline release, fleet telemetry carries DOM
size (composition_element_count) and frames on every render — but the
motion proxy, observability_init_tween_count, has 0% coverage on the
exact renders the band routes: parallel workers' console buffers (and so
the [FrameCapture:INIT] line the summary parses) only propagate to the
orchestrator on FAILURE. Single-worker screenshot renders report it;
the multi-worker clamp bucket never does. Verified against 7d of fleet
data: 35k screenshot renders carry tween counts, 0 of 9,600 band renders.
Fix rides the one channel parallel workers already return on success —
the per-worker CapturePerfSummary. Sessions record initTelemetry on
every init path; the perf summary now carries it; the orchestrator
max-merges across workers (same multi-session semantics the console
parser uses) and feeds it to the observability summary as a structured
fallback, console lines still refining when present.
With this, every band render carries full coordinates — (elements,
tweens, frames, path, speed) — which buys two reads: regressing wild DE
speed against element count on the existing 900+ inversions validates
the bench's 0.50ms/element slope BEFORE the routing flip, and any
post-flip misroute can be reproduced locally by feeding its telemetry
row straight into gen-crossover-comp's knobs (--movers ~ tween count,
--static ~ element count) and re-benching.
(Also drops a now-stale fallow suppression in render.ts — the test-only
reset export it guarded gained real test importers, so the issue it
suppressed no longer exists and the gate flags the leftover.)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Recut after pre-registering the read exposed two flaws in the first cut:
1. Attribution was wrong. de_short_band keyed on frame count + element
ceiling alone, so a webm render, a compile-gated comp, or a forced
screenshot at 400f reported "applied" while its routing was untouched —
poisoning the measurement cohort with unaffected renders and diluting
any effect toward zero. Now the predicate is evaluated twice (900 floor
vs band floor) and the band is DECISIVE only when the calls disagree:
every other eligibility condition passed and only the floor differed.
The cohort contains exactly the renders whose routing the band decides.
2. A same-release flip is unfalsifiable. composition_element_count ships
WITH the routing change, so the before-period cannot be filtered to the
same cohort as the after-period — the comparison would show a speedup
even if the change did nothing (the after-cohort excludes big comps by
construction; the before-cohort includes them). Routing is therefore
gated behind HF_DE_SHORT_BAND_ROUTE, default OFF: this release computes
and emits the full band decision on every render ("applied" is the
counterfactual "would have inverted"), a follow-up flips the default.
Identical cohort selector on both sides of the boundary, and the
skipped/oversize renders in the same frame band form a concurrent
control — a difference-in-differences that absorbs secular drift
(content mix, version-correlated populations, hardware), which a plain
before/after cannot.
Also: countElementTags now counts HTML void elements. Counting only
closers read an image gallery as a tiny comp and opened the band on
exactly the content most likely to lose it (images skew expensive to
paint). Opening tags stay uncounted — inline scripts' `a < b` would
false-positive. Counter semantics are frozen while the baseline is read:
the distribution the baseline release records must be measured by the
same counter that later gates.
Revert-rate baseline for the pre-registered read, measured over 14d
fleet-wide: the 900+ inversion runs 31,756 inverted / 1,705 reverted =
5.1%. At the benched 1.16-1.24x win and ~1.8x revert cost, expected net
for the band is ~12%. Kill criteria for the flip release: DiD <= 0,
in-band revert rate > 5.1% baseline, or band fallback rate > DE baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
31% of fleet renders (24h, v0.7.78+) are DE-eligible comps clamped to
parallel screenshot purely because they sit under the 900-frame inversion
floor — the median fleet render is ~250-600 frames, below every DE entry
threshold. This opens a 250-899 frame band, gated on composition size.
Measured, not assumed. A controlled sweep (fixed synthetic content,
{250,400,600,900}f, single-DE vs parallel-screenshot-W4, 3 reps, capture
mode verified per row, AC power, load-gated) showed single-DE winning
1.16-1.24x at every size — but only for content in constant motion. A
follow-up 2x2 found motion and DOM size pull in OPPOSITE directions, so
neither alone predicts the winner (ratio = ss4/de1, >1 means DE wins):
24 movers / 0 nodes -> 1.05
320 movers / 0 nodes -> 1.24
320 movers / 7000 nodes -> 1.09
24 movers / 7000 nodes -> 0.96
24 movers / 20000 nodes -> 0.71
24 movers / 40000 nodes -> 0.55
DE's wall-clock scales ~0.50ms/element against parallel screenshot's
~0.22ms — drawElement repaints the whole tree per frame while fan-out
amortizes it — so the downside is NOT bounded and a bare floor drop would
have handed a 1.8x regression to large comps. Since motion only ever helps
DE, an element ceiling calibrated at the lowest-motion case is safe at
every motion level; crossover there is ~3.9k, and the default sits at 2500.
The predicate is untouched; the call site picks the floor. Above the
ceiling, or at 900+ frames, behaviour is bit-identical to today — the
change can only add inversions in the new band, never remove one.
Instrumentation, since this ships at full exposure rather than cohorted:
`composition_element_count` on EVERY render (the fleet distribution of the
gate variable is unknown — without it we cannot tell whether 2500 opens the
band for most short comps or almost none, nor re-derive the threshold from
real content), and `de_short_band` = applied | skipped_elements, unset when
the frame count made the band irrelevant, so a fleet perf shift is
attributable to this change rather than to content mix.
Safety is unchanged and already proven on this path: per-frame PSNR
self-verify with screenshot fallback, exactly as the 900+ band has shipped
default-on. Knobs: HF_DE_SHORT_MIN_FRAMES, HF_DE_SHORT_MAX_ELEMENTS (0
disables the band).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The DE parallel-router breaker's tripped state lived in the same config
file as the install id, so the most common identity reset — deleting
~/.hyperframes — also re-enrolled the machine into an experimental path
that had already failed on it.
Mirror exactly two facts into a machine-local state file
(~/.local/state/hyperframes/install-state.json) that a config wipe does
not touch:
- markerAt: written unconditionally on every install, so the fraction of
fresh mints that find it directly measures recoverable id churn
(config wiped, machine persisted) vs unrecoverable (fresh
machine/container/new user). Emitted as install_predecessor_found on
telemetry events; absent (not false) on configs predating the field.
- deParallelRouterTrialFired: a breaker tripped by a previous install
stays tripped for the new one. Config corruption takes the same mint
path, so it survives that too.
The file deliberately holds NO identity — no anonymousId, no counters.
A wiped config still gets a fresh id unconditionally; only the safety
fact about the machine survives. Sync happens inside writeConfig so no
breaker write site can forget it; failures are swallowed (telemetry
must never break the CLI) but leave the memo unset so a later write
retries. `hyperframes telemetry` lists the state path for transparency.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
trackRenderComplete requires `gpu: boolean`; the two new opt-out test
calls omitted it, failing Typecheck in CI. The fix already existed on the
stacked branch, so only this base branch was broken.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two review findings on the floor/telemetry PR:
1. powerStateFields() is spread into the properties object at the CALL SITE,
so it ran before trackEvent's own `if (!shouldTrack()) return` guard —
telemetry-disabled installs paid two blocking `pmset` subprocess spawns
per render for an event that was then discarded. Now short-circuits on
shouldTrack() (memoized, so no cost on the tracked path). Regression test
asserts pmset is not sampled when telemetry is off; fault-injection
verified it fails without the guard.
2. The DE parallel router pinned workerCount to 3 and skipped calibration
even when verified parallel DE STREAMING — the entire reason for the pin
— could not run for that render. The common case is a composition over
streamingEncodeMaxDurationSeconds (240 s default): the duration cap
disables streaming before the router's force flag is consulted, so the
render got a hard-coded 3 workers chosen by a benchmark for a path it was
not on, instead of the calibrated count. shouldPreferParallelDrawElement
now takes parallelStreamingAvailable and withholds the bet without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three review findings on the win32 drawElement PR:
1. gpu_renderer shipped the raw UNMASKED_RENDERER_WEBGL string — unbounded,
driver-authored, GPU-model-specific, and |-joined across parallel
sessions, i.e. high cardinality by construction, against this file's own
convention of sanitizing engine-sourced strings (deGateReason is a
bucket; error messages go through redactTelemetryString). Now bucketed at
the source by classifyGpuRenderer to <backend>/<vendor>
(metal/apple, d3d11/nvidia, swiftshader/other, ...), which is the whole
analytic signal the win32 rollout needs and nothing else. The raw string
never leaves the engine.
2. gpu_renderer reached render_complete only, so a crashed render — the
cohort the field exists to attribute — carried no backend. It now rides
RenderCaptureObservability (deGpuRenderer, sourced from the live probe
session like the de_* counters), so both render_complete and
render_error carry it and a hard failure still reports its GPU backend.
On render_complete the perfSummary value still wins by spread order.
3. Restore the fallow-ignore-next-line suppression above
__resetDeParallelRouterTrialStateForTests: CLI test files are not fallow
entry points, so removing it fails the CI dead-code audit (local
pre-commit passed only because of its changed-file scope).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Vance Ingalls