fix(cli,core,studio): close 15 review findings + 2 R5 blockers

R5 blockers
- Negative install-state latch was cached for the process lifetime, but
  only `true` is monotonic across processes. A long-lived preview server
  held a stale `false` and could re-enrol after another process tripped
  the breaker. Only the positive is cached now; `false` re-reads.
- The real breaker writer used writeConfig(), which collapses
  {ok:true, mirrored:false} to success, so a run that mirrored nothing
  reported done with the latch only on the erasable store. It consumes
  writeConfigWithResult and retries until both stores carry it.

Bucketing integrity
- Storage-restricted Studio profiles all bucketed on the literal
  "anonymous": computed against the shipped hash, 100% of them were
  enrolled in calibration-50 rather than 50%, and they merged into one
  PostHog person. Per-session random id instead — persists nothing.
- bucketSeed had read/write authority backwards: install-state is
  write-once authoritative, but readConfig took config.json's blindly, so
  the stores could hold different seeds until a re-mint flipped every
  cohort. Merged on read, like the latch.
- An unwritable ~/.hyperframes with no config.json re-minted per call,
  re-rolling the seed on every command, and the "cohorts will not be
  stable" warning was unreachable on that path.
- A corrupt PRE-MOVE state file was never deleted, so a machine reset
  with `rm -rf ~/.hyperframes` reported predecessorFound/stateFileCorrupt
  forever — poisoning the exact metric this work exists to produce.

Opt-out honoring
- CLI canary decisions memoized per process, so `hyperframes telemetry
  disable` during a running preview server was ignored for hours while
  the server kept serving pre-opt-out decisions. The memo is keyed on the
  telemetry posture.
- shouldTrack() memoized, contradicting policy.ts's documented "not
  memoized" contract that policy.test.ts asserts.
- The Studio override path resolved the bucket unit eagerly as an
  argument, minting and PERSISTING a tracking id for an opted-out profile
  — a value evaluateCanary discards unread.
- Storage reads could throw out of telemetry into a post-commit catch
  block, reporting an already-committed edit as failed.
- readConfig printed an unsilenceable stderr warning on every invocation
  for installs that opted out of telemetry entirely.

Host split
- isLoopbackHost rejected 0.0.0.0, so the documented
  HYPERFRAMES_PREVIEW_HOST LAN mode silently lost CLI→Studio identity
  stitching and split one user across two PostHog persons. Identity is
  now allowed when the operator explicitly opted into LAN binding.
- Corrected the comment claiming the guard refuses spoofed Hosts: a
  non-browser client sets Host freely. It is a browser DNS-rebinding
  mitigation, not access control, and now says so.

Semantics and test hygiene
- percentage:100 did not mean everyone — exclude and no_unit_id sat above
  the fast path, so the registry's "delete the entry at 100" step was an
  unstaged flip for CI and seedless installs.
- CLI cohort adoption returned before evaluateCanary, dropping Studio's
  own webdriver exclusion.
- overdueCanaries() was asserted against wall-clock time, so the whole
  core suite would go red on 2026-09-15 for every unrelated PR; and `>`
  against midnight made a canary overdue ON its sunset date.
- Statistical assertions ran on unseeded randomUUID() populations tight
  enough to fail ~1 run in 200. Seeded.

Also: broke a config -> policy -> transport -> config import cycle by
moving POSTHOG_API_KEY to a leaf module.

Tests: 2347 CLI (bundle absent), 3153 Studio, 1450 core. Fault injection
covers the latch, seed authority, LAN identity, webdriver exclusion and
the anonymous-bucket fix. Two pre-existing tests asserted behaviour these
findings identify as wrong (shouldTrack memoization, 100%-excludes-CI)
and were rewritten with the reasoning stated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-31 01:27:37 -07:00
co-authored by Claude Opus 5
parent 5e2a9432f1
commit 3f69a2c635
24 changed files with 648 additions and 109 deletions
+77 -12
View File
@@ -1,5 +1,4 @@
import { describe, expect, it } from "vitest";
import { randomUUID } from "node:crypto";
import { canaryBucket, evaluateCanary, parseCanaryOverride, type CanaryInput } from "./canary.js";
import { CANARIES, canaryEnvVar, findCanary, overdueCanaries } from "./canaryRegistry.js";
import { CANARY_FEATURE_PREFIX, canaryFeatureKey, canaryFeatureProperties } from "./canary.js";
@@ -27,9 +26,36 @@ function rawFnv(input: string): number {
return hash >>> 0;
}
/** A realistic population: install ids are v4 UUIDs (`randomUUID()`). */
function uuids(n: number): string[] {
return Array.from({ length: n }, () => randomUUID());
/**
* A realistic population: v4-shaped UUIDs, but from a SEEDED PRNG.
*
* These ids feed statistical assertions (share within 1pp, chi-square
* uniformity) whose thresholds are tight enough to fail by chance on a
* genuinely random draw: measured at ~4 failures per 1500 runs for the share
* bound (the pct=50 case has a binomial SD of 0.354pp, so 1pp is only 2.8
* sigma) and 1 per 1000 for chi-square by its own construction. That made the
* whole @hyperframes/core suite flaky for unrelated PRs. Seeded means the
* population is fixed, so a failure is a real change in the hash — which is
* the only thing these tests are for.
*/
function uuids(n: number, seed = 0x9e3779b9): string[] {
let state = seed >>> 0;
const nextByte = (): number => {
// xorshift32 — deterministic, and uniform enough to stand in for a real
// id population. Not used for anything security-relevant.
state ^= state << 13;
state >>>= 0;
state ^= state >>> 17;
state ^= state << 5;
state >>>= 0;
return state & 0xff;
};
const hex = (count: number): string =>
Array.from({ length: count }, () => nextByte().toString(16).padStart(2, "0")).join("");
return Array.from(
{ length: n },
() => `${hex(4)}-${hex(2)}-4${hex(2).slice(1)}-a${hex(2).slice(1)}-${hex(6)}`,
);
}
describe("fnv1a32 (via canaryBucket)", () => {
@@ -87,9 +113,9 @@ describe("evaluateCanary", () => {
}
});
it("fails closed without a unit id — unknown must never mean everyone", () => {
it("fails closed without a unit id — unknown must never mean a PARTIAL cohort", () => {
for (const id of [undefined, "", " "]) {
expect(evaluateCanary(base({ unitId: id, percentage: 100 }))).toEqual({
expect(evaluateCanary(base({ unitId: id, percentage: 50 }))).toEqual({
enabled: false,
reason: "no_unit_id",
});
@@ -97,12 +123,32 @@ describe("evaluateCanary", () => {
});
it("excludes flagged units (CI) from percentage enrolment but not from an override", () => {
expect(evaluateCanary(base({ percentage: 100, exclude: true })).reason).toBe("excluded");
expect(evaluateCanary(base({ percentage: 100, exclude: true, override: true })).enabled).toBe(
expect(evaluateCanary(base({ percentage: 50, exclude: true })).reason).toBe("excluded");
expect(evaluateCanary(base({ percentage: 50, exclude: true, override: true })).enabled).toBe(
true,
);
});
// 100 is the one percentage where "we don't know who this is" and "this is
// CI" stop mattering: the registry's step 4 says to delete the entry and the
// guard at 100-and-holding, so any population still resolving false here
// would take the new path for the FIRST time at deletion — unstaged, and
// invisible on the dashboard that said it was safe.
it.each([
["no unit id", { unitId: undefined }],
["blank unit id", { unitId: " " }],
["excluded (CI)", { exclude: true }],
])("at 100%% enrols %s, so deleting the guard changes nothing", (_label, extra) => {
expect(evaluateCanary(base({ percentage: 100, ...extra }))).toEqual({
enabled: true,
reason: "in_cohort",
});
});
it("an explicit off still wins at 100%", () => {
expect(evaluateCanary(base({ percentage: 100, override: false })).enabled).toBe(false);
});
it("clamps out-of-range and fractional percentages", () => {
expect(evaluateCanary(base({ percentage: -5 })).enabled).toBe(false);
expect(evaluateCanary(base({ percentage: 999 })).enabled).toBe(true);
@@ -264,10 +310,29 @@ describe("registry", () => {
expect(findCanary("nope")).toBeUndefined();
});
it("no canary is past its sunset date", () => {
// Fails the suite when a rollout has been left half-finished. Either take
// it to 100 and delete the entry, or move the date deliberately.
expect(overdueCanaries()).toEqual([]);
// Deliberately NOT `overdueCanaries()` with the ambient date. That assertion
// reads wall-clock time, so it turns the entire @hyperframes/core suite red
// on a calendar date for every unrelated PR — a broken build nobody caused
// and whose fix is unrelated to the change under test. The registry's own
// freshness is enforced by the pinned dates below plus the sunset REPORT,
// which is advisory rather than a gate.
it("every canary carries a parseable sunset date in the future at authoring time", () => {
const authored = new Date("2026-07-31T00:00:00Z");
for (const c of CANARIES) {
const sunset = Date.parse(`${c.sunsetAfter}T00:00:00Z`);
expect(Number.isFinite(sunset), `${c.name} has an unparseable sunsetAfter`).toBe(true);
expect(sunset, `${c.name} was authored already-expired`).toBeGreaterThan(authored.getTime());
}
});
it("reports a canary as overdue only AFTER the whole sunset day has passed", () => {
const [first] = CANARIES;
if (!first) return;
const day = first.sunsetAfter;
expect(overdueCanaries(new Date(`${day}T00:00:00Z`))).not.toContain(first.name);
expect(overdueCanaries(new Date(`${day}T23:59:59Z`))).not.toContain(first.name);
const dayAfter = new Date(Date.parse(`${day}T00:00:00Z`) + 86_400_000);
expect(overdueCanaries(dayAfter)).toContain(first.name);
});
});