mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-11 23:00:03 +00:00
feat(cli): bucket canaries on a machine-lineage seed, not the telemetry id
Cohort membership now survives a config wipe. Canaries bucket on a dedicated bucketSeed (fresh random UUID, distinct from anonymousId by design) that is mirrored write-once into the install-state file and inherited at mint: a wipe re-rolls the telemetry id but never the canary assignment. This removes cumulative-exposure drift for the recoverable churn bucket entirely — the residual drift comes only from fresh machines, containers, and genuinely new users — and keeps before/after comparisons valid across a reinstall. The seed is never emitted in telemetry (only the resulting true/false assignments are), so it does not link the old id to the new one server-side. The residual linker is the flag vector itself (k bits for k live canaries), documented as such. An explicit reset still works by deleting the state file, and the no-identity test now also asserts the seed differs from the anonymousId. Cross-surface coherence: the CLI's studio server injects the seed as window.__HF_CLI_BUCKET_SEED (same telemetry gate and script-escaping as the distinct id, and on the /api/telemetry-identity fallback), and the Studio binding buckets on it when present — without this the CLI would bucket on the seed while Studio bucketed on the distinct id, splitting one machine across cohorts (calibration check 4 would catch exactly this). Standalone Studio still buckets on its localStorage id: the browser has no second storage location, so that id doubles as the seed. Legacy configs are backfilled once (lineage seed if the state file has one, else minted) and persisted immediately — an unpersisted seed would re-roll cohorts every process. Safe to ship in the same release as the first canaries: no prior release emitted canary properties, so the bucketing-unit change is unobservable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
0d98de023f
commit
8dd20ac2e8
@@ -47,6 +47,7 @@ beforeEach(() => {
|
||||
sessionStorage.clear();
|
||||
setSearch("");
|
||||
delete window.__HF_CLI_DISTINCT_ID;
|
||||
delete window.__HF_CLI_BUCKET_SEED;
|
||||
Object.defineProperty(navigator, "webdriver", { value: false, configurable: true });
|
||||
__resetStudioCanaryCacheForTests();
|
||||
__resetStudioDistinctIdForTests();
|
||||
@@ -131,19 +132,35 @@ describe("automated browsers", () => {
|
||||
});
|
||||
|
||||
describe("cohort identity", () => {
|
||||
it("buckets on the Studio distinct id unmodified — so a CLI-launched Studio shares the CLI's cohort", () => {
|
||||
// distinctId.ts adopts window.__HF_CLI_DISTINCT_ID when the CLI launched
|
||||
// Studio. This asserts the binding passes that id through untouched: if it
|
||||
// prefixed or re-hashed it, the editor would land in a different cohort
|
||||
// than the terminal for the same user, and a rollout spanning both would
|
||||
// be incoherent.
|
||||
it("buckets on the CLI's bucket seed when injected — the unit that survives config wipes", () => {
|
||||
// The CLI buckets on its bucketSeed (inherited across config wipes via
|
||||
// the install-state file), so a CLI-launched Studio must bucket on the
|
||||
// SAME seed or the two surfaces would split one machine across cohorts.
|
||||
const cliId = "db0c1f4a-b95e-4c35-90c6-1a15bd76f717";
|
||||
const cliSeed = "5f1c9d2e-0000-4000-8000-aaaaaaaaaaaa";
|
||||
window.__HF_CLI_DISTINCT_ID = cliId;
|
||||
window.__HF_CLI_BUCKET_SEED = cliSeed;
|
||||
__resetStudioDistinctIdForTests();
|
||||
__resetStudioCanaryCacheForTests();
|
||||
|
||||
// Telemetry identity still adopts the DISTINCT id — the seed only buckets.
|
||||
expect(resolveStudioDistinctId()).toBe(cliId);
|
||||
const viaBinding = resolveCanary("on-everywhere").bucket;
|
||||
const bySeed = evaluateCanary({
|
||||
feature: "on-everywhere",
|
||||
unitId: cliSeed,
|
||||
percentage: 100,
|
||||
}).bucket;
|
||||
expect(viaBinding).toBe(bySeed);
|
||||
});
|
||||
|
||||
it("buckets on the Studio distinct id when no seed is injected (standalone Studio)", () => {
|
||||
const cliId = "db0c1f4a-b95e-4c35-90c6-1a15bd76f717";
|
||||
window.__HF_CLI_DISTINCT_ID = cliId;
|
||||
__resetStudioDistinctIdForTests();
|
||||
__resetStudioCanaryCacheForTests();
|
||||
|
||||
expect(resolveStudioDistinctId()).toBe(cliId);
|
||||
// 50% so the answer is id-dependent rather than trivially true.
|
||||
const viaBinding = resolveCanary("on-everywhere").bucket;
|
||||
const direct = evaluateCanary({
|
||||
feature: "on-everywhere",
|
||||
|
||||
Reference in New Issue
Block a user