feat(cli): bucket canaries on a machine-lineage seed, not the telemetry id

Cohort membership now survives a config wipe. Canaries bucket on a
dedicated bucketSeed (fresh random UUID, distinct from anonymousId by
design) that is mirrored write-once into the install-state file and
inherited at mint: a wipe re-rolls the telemetry id but never the canary
assignment. This removes cumulative-exposure drift for the recoverable
churn bucket entirely — the residual drift comes only from fresh
machines, containers, and genuinely new users — and keeps before/after
comparisons valid across a reinstall.

The seed is never emitted in telemetry (only the resulting true/false
assignments are), so it does not link the old id to the new one
server-side. The residual linker is the flag vector itself (k bits for k
live canaries), documented as such. An explicit reset still works by
deleting the state file, and the no-identity test now also asserts the
seed differs from the anonymousId.

Cross-surface coherence: the CLI's studio server injects the seed as
window.__HF_CLI_BUCKET_SEED (same telemetry gate and script-escaping as
the distinct id, and on the /api/telemetry-identity fallback), and the
Studio binding buckets on it when present — without this the CLI would
bucket on the seed while Studio bucketed on the distinct id, splitting
one machine across cohorts (calibration check 4 would catch exactly
this). Standalone Studio still buckets on its localStorage id: the
browser has no second storage location, so that id doubles as the seed.

Legacy configs are backfilled once (lineage seed if the state file has
one, else minted) and persisted immediately — an unpersisted seed would
re-roll cohorts every process. Safe to ship in the same release as the
first canaries: no prior release emitted canary properties, so the
bucketing-unit change is unobservable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-30 15:16:16 -07:00
co-authored by Claude Opus 5
parent 0d98de023f
commit 8dd20ac2e8
10 changed files with 327 additions and 57 deletions
+9 -2
View File
@@ -16,7 +16,11 @@ import {
loadRuntimeSourceSignature,
} from "./runtimeSource.js";
import { VERSION as version } from "../version.js";
import { buildStudioHeadScripts, resolveCliTelemetryDistinctId } from "./telemetryIdentity.js";
import {
buildStudioHeadScripts,
resolveCliBucketSeed,
resolveCliTelemetryDistinctId,
} from "./telemetryIdentity.js";
import { emitStudioRenderComplete, emitStudioRenderError } from "./studioRenderTelemetry.js";
import { isDevMode } from "../utils/env.js";
import {
@@ -652,7 +656,10 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
// distinct id (no PII) so the browser session can join the CLI's PostHog
// person, or `{ distinctId: null }` when CLI telemetry is disabled.
app.get("/api/telemetry-identity", (c) => {
return c.json({ distinctId: resolveCliTelemetryDistinctId() });
return c.json({
distinctId: resolveCliTelemetryDistinctId(),
bucketSeed: resolveCliBucketSeed(),
});
});
app.get("/api/events", (c) => {