fix(cli,studio): adopt the CLI's canary decisions in a launched Studio

Closes both cross-surface findings with one mechanism. The CLI publishes
window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }); a CLI-launched
Studio takes it as authoritative over its own seed, URL override and the
registry percentage.

Studio re-deriving could not agree with the CLI in three cases:

  - Telemetry off. The CLI resolves telemetry_opt_out, but Studio's
    opt-out is a separate localStorage flag it cannot see, so it would
    evaluate normally and could enrol on a render the CLI excluded. The
    previous commit gated each surface independently; that fixed silent
    enrolment per surface but NOT the disagreement between them.
  - HF_CANARY_* override. Env vars never cross into the browser — Studio
    reads only its URL param / sessionStorage — so a support session
    forcing a canary on got the CLI forced and Studio guessing.
  - No seed injected. Studio falls back to a different unit id, i.e. a
    different bucket.

Shipping the decision instead of the inputs makes divergence structurally
impossible: one evaluation, two surfaces. It also exposes strictly less —
booleans about features, rather than the seed buckets derive from — which
is why it is safe to publish with telemetry off, the case it exists for.
Studio still evaluates locally when standalone, or for a canary the CLI
did not publish, and ignores a non-boolean value rather than trusting it.

Tests: 6 Studio (CLI-off wins over unset local flag, CLI-on with no URL
param, beats contradicting override, beats seed, falls back per-canary,
rejects non-boolean) and 4 CLI (decisions with telemetry off and no
identity, alongside identity when on, script-tag escaping on a hostile
canary name, throwing resolver degrades to identity only). Four existing
identity tests asserted the old "nothing when telemetry off" contract and
were updated; the registry is now mocked there so string assertions don't
move when a canary is added or ramped. Fault injection: dropping the
adoption fails 4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-30 16:29:31 -07:00
co-authored by Claude Opus 5
parent 4f464dc424
commit 98b23a8850
6 changed files with 271 additions and 34 deletions
+26
View File
@@ -118,6 +118,32 @@ export function isCanaryEnabled(name: string): boolean {
return resolveCanary(name).enabled;
}
/**
* Every registered canary's resolved on/off for this process, for handing to
* a CLI-launched Studio.
*
* Studio adopts these wholesale instead of re-deriving, because re-deriving
* cannot agree in three cases:
*
* - **Telemetry off.** The CLI resolves `telemetry_opt_out`; Studio's own
* opt-out is a separate localStorage flag on a different machine-level
* switch, so it would evaluate normally and could enrol.
* - **`HF_CANARY_*` override.** Env vars do not cross into the browser at
* all — Studio only reads its URL param / sessionStorage — so a support
* session forcing a canary on got the CLI forced and Studio guessing.
* - **No seed injected.** Whenever the seed is withheld Studio falls back
* to a different unit id, i.e. a different bucket.
*
* Shipping the decision rather than the inputs makes divergence structurally
* impossible: one evaluation, two surfaces. It is also strictly less to
* expose — booleans about features, not the seed the buckets derive from.
*/
export function canaryDecisionsForStudio(): Record<string, boolean> {
const out: Record<string, boolean> = {};
for (const canary of CANARIES) out[canary.name] = resolveCanary(canary.name).enabled;
return out;
}
/**
* Canary assignments as PostHog flag properties — `$feature/canary-<name>`
* set to `"true"` / `"false"` for every registered canary. Spread onto every