Commit Graph
3 Commits
Author SHA1 Message Date
Vance IngallsandClaude Opus 5 98b23a8850 fix(cli,studio): adopt the CLI's canary decisions in a launched Studio
Closes both cross-surface findings with one mechanism. The CLI publishes
window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }); a CLI-launched
Studio takes it as authoritative over its own seed, URL override and the
registry percentage.

Studio re-deriving could not agree with the CLI in three cases:

  - Telemetry off. The CLI resolves telemetry_opt_out, but Studio's
    opt-out is a separate localStorage flag it cannot see, so it would
    evaluate normally and could enrol on a render the CLI excluded. The
    previous commit gated each surface independently; that fixed silent
    enrolment per surface but NOT the disagreement between them.
  - HF_CANARY_* override. Env vars never cross into the browser — Studio
    reads only its URL param / sessionStorage — so a support session
    forcing a canary on got the CLI forced and Studio guessing.
  - No seed injected. Studio falls back to a different unit id, i.e. a
    different bucket.

Shipping the decision instead of the inputs makes divergence structurally
impossible: one evaluation, two surfaces. It also exposes strictly less —
booleans about features, rather than the seed buckets derive from — which
is why it is safe to publish with telemetry off, the case it exists for.
Studio still evaluates locally when standalone, or for a canary the CLI
did not publish, and ignores a non-boolean value rather than trusting it.

Tests: 6 Studio (CLI-off wins over unset local flag, CLI-on with no URL
param, beats contradicting override, beats seed, falls back per-canary,
rejects non-boolean) and 4 CLI (decisions with telemetry off and no
identity, alongside identity when on, script-tag escaping on a hostile
canary name, throwing resolver degrades to identity only). Four existing
identity tests asserted the old "nothing when telemetry off" contract and
were updated; the registry is now mocked there so string assertions don't
move when a canary is added or ramped. Fault injection: dropping the
adoption fails 4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 16:29:31 -07:00
Vance IngallsandClaude Opus 5 8dd20ac2e8 feat(cli): bucket canaries on a machine-lineage seed, not the telemetry id
Cohort membership now survives a config wipe. Canaries bucket on a
dedicated bucketSeed (fresh random UUID, distinct from anonymousId by
design) that is mirrored write-once into the install-state file and
inherited at mint: a wipe re-rolls the telemetry id but never the canary
assignment. This removes cumulative-exposure drift for the recoverable
churn bucket entirely — the residual drift comes only from fresh
machines, containers, and genuinely new users — and keeps before/after
comparisons valid across a reinstall.

The seed is never emitted in telemetry (only the resulting true/false
assignments are), so it does not link the old id to the new one
server-side. The residual linker is the flag vector itself (k bits for k
live canaries), documented as such. An explicit reset still works by
deleting the state file, and the no-identity test now also asserts the
seed differs from the anonymousId.

Cross-surface coherence: the CLI's studio server injects the seed as
window.__HF_CLI_BUCKET_SEED (same telemetry gate and script-escaping as
the distinct id, and on the /api/telemetry-identity fallback), and the
Studio binding buckets on it when present — without this the CLI would
bucket on the seed while Studio bucketed on the distinct id, splitting
one machine across cohorts (calibration check 4 would catch exactly
this). Standalone Studio still buckets on its localStorage id: the
browser has no second storage location, so that id doubles as the seed.

Legacy configs are backfilled once (lineage seed if the state file has
one, else minted) and persisted immediately — an unpersisted seed would
re-roll cohorts every process. Safe to ship in the same release as the
first canaries: no prior release emitted canary properties, so the
bucketing-unit change is unobservable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:16:16 -07:00
James Russo 9c4d9e50a0 feat(telemetry): unify CLI and Studio PostHog identity (Layer 1) (#1829)
* feat(telemetry): unify CLI and Studio PostHog identity (Layer 1)

Seed the CLI's anonymous distinct_id into Studio at launch so a developer's
CLI and their Studio browser session resolve to the same PostHog person.
Also unifies Studio's two previously-independent anonymous ids into one
source of truth. Uses only the existing anonymous machine id (no new PII).

- cli: inject window.__HF_CLI_DISTINCT_ID into the served index.html <head>
  (mirrors the existing __HF_STUDIO_ENV__ injection) + add a fallback
  GET /api/telemetry-identity endpoint. Only seeds when CLI telemetry is
  enabled; empty/no-op otherwise.
- studio: new telemetry/distinctId.ts single source of truth; adopts the
  CLI-seeded id when present, else falls back to the existing per-browser
  localStorage id. Both Studio clients (studio:* and studio_*/render) now
  share this one id.

* fix(telemetry): keep Studio distinct_id resolver fail-silent on getItem

resolveStudioDistinctId read localStorage.getItem() outside a try/catch
while every other external access in the module is guarded. In a
storage-restricted context where the localStorage reference resolves but
getItem throws, the resolver threw — breaking the module's fail-silent
contract (telemetry must never break Studio). Guard the reads and treat a
throw as "no id". Also drop an unnecessary `as` cast in the test per the
repo CLAUDE.md convention (the optional global is already declared).

* refactor(telemetry): address review feedback on identity unification

- dedup safeLocalStorage/safeSessionStorage into utils/safeStorage.ts,
  used by both telemetry/config.ts and telemetry/distinctId.ts (Miga #6)
- replace redundant `??=` with `=` in the no-storage branch; cachedId is
  guaranteed null there (Miga #2)
- extract buildStudioHeadScripts() so the "identity script before env
  script" head-injection ordering is a pure, tested invariant (Miga #5)
- add tests: head-script ordering + telemetry-off passthrough, and a
  Studio memoization test proving an adopted CLI id survives a later
  window.__HF_CLI_DISTINCT_ID reassignment (Rames)
- clarify the XSS-escaping comment (both < and / escaped so no </script>
  sequence can form) (Miga #1)
2026-07-01 09:21:18 -07:00