mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-17 15:24:19 +00:00
docs(canary): make calibration check 4 compare per-surface values
As written, check 4's condition (dual-surface AND value flipped anywhere) was a strict subset of check 3's (value flipped) — if check 3 read zero, check 4 was vacuously zero and added no independent signal. Compare the value each surface actually reported instead, and state plainly that any disagreement is also a check-3 flip: this check's job is attributing such a flip to binding divergence. Matching fix applied to the live PostHog tile (insight jQi7QdW1, dashboard 1918875). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
6f6f01c8ea
commit
c43b4e2d95
@@ -181,17 +181,23 @@ SELECT count() AS installs_that_flipped FROM (
|
||||
|
||||
**4. Cross-surface agreement — do CLI and Studio agree for the same install?**
|
||||
A CLI-launched Studio adopts the CLI's id, so the same install must report the
|
||||
same cohort on both. Disagreement means the two bindings have diverged.
|
||||
same cohort on both. Disagreement means the two bindings have diverged. Note
|
||||
this compares the *value reported per surface* — any disagreement here is also
|
||||
a check-3 flip, so this check's job is attribution: it isolates the flips that
|
||||
are binding divergence rather than within-surface instability.
|
||||
|
||||
```sql
|
||||
SELECT count() AS installs_disagreeing FROM (
|
||||
SELECT distinct_id
|
||||
SELECT countIf(cli_val != studio_val) AS installs_disagreeing FROM (
|
||||
SELECT
|
||||
distinct_id,
|
||||
anyIf(properties['$feature/canary-calibration-10'], NOT startsWith(event, 'studio')) AS cli_val,
|
||||
anyIf(properties['$feature/canary-calibration-10'], startsWith(event, 'studio')) AS studio_val
|
||||
FROM events
|
||||
WHERE timestamp >= now() - INTERVAL 14 DAY
|
||||
AND JSONHas(properties, '$feature/canary-calibration-10')
|
||||
GROUP BY distinct_id
|
||||
HAVING uniqExact(startsWith(event, 'studio')) > 1 -- seen on both surfaces
|
||||
AND uniqExact(properties['$feature/canary-calibration-10']) > 1
|
||||
HAVING countIf(startsWith(event, 'studio')) > 0 -- seen on both surfaces
|
||||
AND countIf(NOT startsWith(event, 'studio')) > 0
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user