fix(cli,studio): adopt the CLI's canary decisions in a launched Studio

Closes both cross-surface findings with one mechanism. The CLI publishes
window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }); a CLI-launched
Studio takes it as authoritative over its own seed, URL override and the
registry percentage.

Studio re-deriving could not agree with the CLI in three cases:

  - Telemetry off. The CLI resolves telemetry_opt_out, but Studio's
    opt-out is a separate localStorage flag it cannot see, so it would
    evaluate normally and could enrol on a render the CLI excluded. The
    previous commit gated each surface independently; that fixed silent
    enrolment per surface but NOT the disagreement between them.
  - HF_CANARY_* override. Env vars never cross into the browser — Studio
    reads only its URL param / sessionStorage — so a support session
    forcing a canary on got the CLI forced and Studio guessing.
  - No seed injected. Studio falls back to a different unit id, i.e. a
    different bucket.

Shipping the decision instead of the inputs makes divergence structurally
impossible: one evaluation, two surfaces. It also exposes strictly less —
booleans about features, rather than the seed buckets derive from — which
is why it is safe to publish with telemetry off, the case it exists for.
Studio still evaluates locally when standalone, or for a canary the CLI
did not publish, and ignores a non-boolean value rather than trusting it.

Tests: 6 Studio (CLI-off wins over unset local flag, CLI-on with no URL
param, beats contradicting override, beats seed, falls back per-canary,
rejects non-boolean) and 4 CLI (decisions with telemetry off and no
identity, alongside identity when on, script-tag escaping on a hostile
canary name, throwing resolver degrades to identity only). Four existing
identity tests asserted the old "nothing when telemetry off" contract and
were updated; the registry is now mocked there so string assertions don't
move when a canary is added or ramped. Fault injection: dropping the
adoption fails 4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-30 16:29:31 -07:00
co-authored by Claude Opus 5
parent 4f464dc424
commit 98b23a8850
6 changed files with 271 additions and 34 deletions
@@ -225,3 +225,53 @@ describe("telemetry opt-out is canary opt-out", () => {
});
});
});
describe("CLI-launched Studio adopts the CLI's decisions", () => {
const OPT_OUT_KEY = "hyperframes-studio:telemetryDisabled";
afterEach(() => {
delete window.__HF_CLI_CANARY_DECISIONS;
});
// The divergence this exists for: CLI telemetry off resolves every canary
// to telemetry_opt_out, but Studio's opt-out is a SEPARATE localStorage
// flag it cannot see — left to itself it would evaluate and could enrol.
it("stays off when the CLI opted out, even though Studio's own flag is unset", () => {
expect(localStorage.getItem(OPT_OUT_KEY)).toBeNull();
window.__HF_CLI_CANARY_DECISIONS = { "on-everywhere": false };
expect(resolveCanary("on-everywhere").enabled).toBe(false);
});
// HF_CANARY_* never crosses into the browser, so before this the CLI was
// forced on and Studio silently guessed from the percentage.
it("turns on when the CLI forced it on, with no URL param present", () => {
window.__HF_CLI_CANARY_DECISIONS = { "off-everywhere": true };
expect(resolveCanary("off-everywhere").enabled).toBe(true);
});
it("beats a contradicting URL override — one render must not run half-enrolled", () => {
window.__HF_CLI_CANARY_DECISIONS = { "on-everywhere": false };
setSearch("?hf_canary_on_everywhere=on");
expect(resolveCanary("on-everywhere").enabled).toBe(false);
});
it("beats the seed-derived bucket", () => {
window.__HF_CLI_BUCKET_SEED = "5f1c9d2e-0000-4000-8000-aaaaaaaaaaaa";
window.__HF_CLI_CANARY_DECISIONS = { "on-everywhere": false };
expect(resolveCanary("on-everywhere").enabled).toBe(false);
});
it("falls back to local evaluation for a canary the CLI did not publish", () => {
window.__HF_CLI_CANARY_DECISIONS = { "off-everywhere": true };
expect(resolveCanary("on-everywhere").enabled).toBe(true);
});
it("ignores a non-boolean value rather than trusting it", () => {
window.__HF_CLI_CANARY_DECISIONS = { "on-everywhere": "false" } as unknown as Record<
string,
boolean
>;
// Falls through to local evaluation: on-everywhere is at 100%.
expect(resolveCanary("on-everywhere").enabled).toBe(true);
});
});