feat(cli,studio): telemetry opt-out is canary opt-out

Both reviewers flagged the same gap: seed injection was gated on
telemetryShouldTrack(), but canary EVALUATION was not. An install with
DO_NOT_TRACK=1 was still bucketed and still had real code paths flipped
(e.g. HF_DE_PARALLEL_ROUTER), silently and unmeasurably.

A canary is a measured rollout — we enrol a slice precisely so it can be
compared against everyone else. An install that sends nothing can't be
compared, so enrolling it buys no signal and only changes that user's
code path, on an experimental feature, without their knowledge. That is
the wrong side of an opt-out.

Resolves to a new `telemetry_opt_out` reason BEFORE bucketing, so no
cohort is assigned at all. Distinct from `excluded` because "why is my
canary off" has a very different answer for CI than for opted-out, and
the reason never reaches telemetry by construction.

Covers every opt-out route: persisted preference, the runtime env vars
and dev/telemetry-disabled builds via policy.ts, and Studio's
hyperframes-studio:telemetryDisabled.

An explicit HF_CANARY_* / ?hf_canary_*= override still wins — a
deliberate local choice, not silent enrolment, and the documented way to
exercise a canary with telemetry off.

The CLI check mirrors shouldTrack() rather than importing it: client.ts
already imports canary.ts for canaryEventProperties, so depending on it
would be a cycle. Both read the same two inputs, so they cannot disagree.

Tests: 9 new across CLI and Studio (preference off, each runtime
override, no bucket assigned, override still honoured, flag properties
all-false). Fault injection: removing the CLI gate fails 6, removing the
Studio gate fails 3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-30 16:11:34 -07:00
co-authored by Claude Opus 5
parent 9f2b892a71
commit 4f464dc424
6 changed files with 186 additions and 33 deletions
@@ -192,3 +192,36 @@ describe("telemetry", () => {
expect(canaryEventProperties()["$feature/canary-on-everywhere"]).toBe("false");
});
});
describe("telemetry opt-out is canary opt-out", () => {
// The studio opt-out lever, per telemetry/config.ts.
const OPT_OUT_KEY = "hyperframes-studio:telemetryDisabled";
it("does not enrol an opted-out browser profile", () => {
localStorage.setItem(OPT_OUT_KEY, "1");
// on-everywhere is at 100% — it would be on for everyone otherwise.
expect(resolveCanary("on-everywhere")).toEqual({
enabled: false,
reason: "telemetry_opt_out",
});
});
it("never buckets an opted-out profile — no cohort is assigned at all", () => {
localStorage.setItem(OPT_OUT_KEY, "1");
expect(resolveCanary("on-everywhere").bucket).toBeUndefined();
});
it("still honours an explicit URL override", () => {
localStorage.setItem(OPT_OUT_KEY, "1");
setSearch("?hf_canary_off_everywhere=on");
expect(resolveCanary("off-everywhere")).toEqual({ enabled: true, reason: "forced_on" });
});
it("reports every canary as false when opted out", () => {
localStorage.setItem(OPT_OUT_KEY, "1");
expect(canaryEventProperties()).toEqual({
"$feature/canary-on-everywhere": "false",
"$feature/canary-off-everywhere": "false",
});
});
});