117 Commits

Include renames

Author	SHA1	Message	Date
Miguel Ángel	f822200fb8	feat(telemetry): measure which lint rules fire, cost, and fail to converge (#3367) ... * feat(telemetry): measure which lint rules fire, cost, and fail to converge Lint rule changes are currently argued from anecdote. This adds the three measurements needed to argue them from data. `lint_report`, once per `hyperframes lint` or `hyperframes check`: - `code_counts` / `codes` — which rules actually fire, and how often - `rule_group_ms` — milliseconds per rule-source module (core, gsap, media, ...) - `slowest_rule` / `slowest_rule_ms` — slowest single rule as `<group>#<index>` - `rule_count` — how many rules this build ran `lint_rule_streak`, once per finding that survives an edit to its file: - `edits` — how many edits the finding survived - `cleared` — whether it eventually went away The streak event is the one that matters. A lint pass costs about 5ms, so per-rule CPU is not what makes the authoring loop slow; a rule an agent cannot satisfy is, because every failed attempt costs a full edit-and-relint cycle. A single run cannot see that, so `lint_rule_streak` reconstructs it across runs: high `edits` with `cleared: false` is a rule nobody can fix, and the `cleared: true` distribution is the baseline to judge it against. An iteration is counted only when the file's content digest CHANGED and the finding is still there. Re-linting an untouched project is not an attempt, which is what stops `check` (which lints on every invocation) from inflating the numbers. Rule identity is the source module plus an index within it. Naming all 86 rules would make the timings prettier but it is a refactor this measurement does not need: the group locates the file, and the index locates the rule. Version, agent runtime, CI flag, and invocation id are already attached to every event by `trackEvent`, so lint pain can be split by CLI version and by which agent produced it without adding anything here. Privacy: only rule codes, counts, and timings are sent. Streak state lives in ~/.hyperframes/lint-streaks.json alongside config.json (so `rm -rf ~/.hyperframes` is still a full reset) and stores digests only — no file paths, no project names, no composition source. Nothing is written and nothing is emitted when telemetry is off. Entries expire after 14 days and are capped at 500 files. `EventProperties` gains string arrays and numeric maps. `codes` and `code_counts` are inherently a set and a histogram; flattening them into dynamic top-level keys would make them unqueryable. PostHog stores both natively. `trackLintRun` is the single call site shared by `lint` and `check`, and it swallows every error — telemetry must never turn a green lint red. * feat(telemetry): emit per-group rule counts so slowest_rule stays comparable Review catch on #3367: `slowest_rule` is the one positional key in either event. It is `<group>#<index>`, so adding or removing a rule renumbers every later slot in that group and the same string means different rules in two builds. #3366 does exactly that to 34 of 81 surviving slots, and `rule_count` alone says only THAT the ruleset moved, not which groups. `rule_group_counts` carries the per-group sizes alongside it, so a consumer comparing two builds can tell which groups' indices still mean the same thing without anyone having to remember which release dropped rules. `codes`, `code_counts` and `rule_group_ms` are keyed by name and were never affected. Also corrects the rule count in the RULE_GROUPS comment: 86, not ~60, as LINT_RULE_COUNT in the same file computes.	2026-08-20 18:26:14 -04:00
Miguel ÁngelandMiguel Angel Simon Sierra	1ae2067b8d	feat(catalog): put the variables panel back, on payloads (#3199) ... * feat(catalog): put the variables panel back, on payloads The panel drove its preview by loading an .html from docs/public, a type the host does not publish, so it showed an empty frame in production and was parked when the catalog was re-landed. It now mounts the same JSON payload the plain player uses and re-mounts it as values change, injecting them as window.__hfVariables into the composition head before any of its scripts run, which is where the runtime reads overrides from. Doing it in the markup rather than after load is what stops the composition initialising with the wrong values first. 172 items with variables get the panel back; the playhead carries across a change so a tweak mid-shot does not jump back to frame zero. * fix(docs): drop the unused url form and the needless escapes * fix(docs): the panel cannot reference a binding beside the export * feat(catalog): make importing an SVG the obvious move A reader arrives at this control with a shape, not with path data, and the panel asked for the coordinates first. Import is now the primary action in a drop target you can see is a drop target, and the raw path sits behind a disclosure for anyone who wants it. * feat(cli): let a fruitless catalog search report the gap An agent that searches by meaning and finds nothing worth installing knows something we do not: the name of a move the catalog is missing. There was no way to tell us, so that knowledge was lost at the end of every run. hyperframes feedback --search-miss "<query>" --wanted "<the move>" records it. It carries no rating, so it never lands in the rating metric, and it is a separate deliberate command rather than something catalog --query does on its own: plain search still sends nothing, which is what the CLI promises. --rating stops being required at the arg level, since a miss has no rating to give. The check moved into the run body, where an absent one is now handled rather than crashing on undefined. * feat(cli): carry tuned variable values into the install snippet Someone who tunes a block on its catalog page had no way to keep those values: the install command was the same one everybody gets, and the tuning stayed on the page. hyperframes add <item> --vars '<json>' now prints a mount element carrying data-variable-values, so the values land where the block is used. They ride on the host rather than being written into the installed file. That keeps the composition on disk byte-identical to the registry's, so a later reinstall can still tell an edit from an update, and it lets two mounts of the same block carry different values. * fix(catalog): serve the item's own directory so runtime paths resolve Some compositions assemble their asset URLs at run time — "compositions/components/" + texture + ".png" for the texture masks, a font the compiler pulled into _remote_media — and no scan of the markup can see a string that does not exist until a script concatenates it. Those items either rendered black or were dropped to a video that had never been uploaded. Each item that needs it now has its prepared directory published, and its payload carries a <base> pointing at it, so any relative path the composition invents resolves. caption-texture renders its masks again, and variable-font-flex has a preview at all for the first time: its MP4 and poster are both 403. Both layouts are published, because which one a composition asks for differs per item, and a directory only earns that if it is under 2 MB. The 12 MB texture sheet keeps the recorded video it already had. * fix(catalog): let the variables panel actually drive the composition Every control on the panel was inert. The values reached the composition and nothing repainted, because the payload had already been compiled: compiling inlines a mounted component and resolves its variables into the markup and CSS, so by the time a reader turns a knob there is nothing left to change. An item that declares variables now ships uncompiled, keeping the mount the runtime loads at run time, which is the only state where data-variable-values still means anything. The component travels inline as a data URI rather than a sibling file, because .html is the one type the docs host will not publish. The demo's own pinned values come off, so the reader's choices reach the mount instead of losing to the values the demo picked to show itself off. Measured on the rendered frame rather than the DOM: green rgb(98,207,144), blue rgb(6,6,199), violet rgb(177,147,230), and back to green. docs/public/catalog drops from 48 MB to 35 MB along the way, since an uncompiled payload carries far less than an inlined one. * feat(catalog): keep variable changes in the url A reader who tuned a piece lost it on reload, and had nothing to send anyone. The values now live in the query string, scoped by composition id so two links never read each other,and only the ones that differ from the defaults are written, so changing one knob gives a short URL rather than every variable spelled out. replaceState rather than pushState: dragging a slider should not leave a trail of history entries. An unreadable value is ignored rather than thrown, so a truncated or hand-edited link opens the piece at its defaults. * fix(catalog): only rewrite the url when a value actually changed * refactor(catalog): memoise the declared defaults on their content * feat(catalog): offer an install command carrying the tuned values The Install block is generated before anyone touches a knob, so it can only ever print the plain command. Someone who spent a minute tuning a piece copied it and got the defaults back. The panel now carries its own command in the Snippet tab, with --vars holding exactly the values that differ. An untouched piece still offers the same short command, so nothing gets noisier for the common case. * fix(catalog): a piece with nothing to render is a skip, not a failure caption-blend-difference is a stylesheet and a paragraph of prose — a class you add to your own captions, with no standalone scene to show. The generator treated that as a build failure, so every run ended by reporting something broken when nothing was. It now reports the shape it is and keeps its recorded video, which is the only honest preview such an item has. A genuine render failure still throws. * fix(catalog): restore variables from the url on a cold load A shared link opened at the defaults. The first render happens on the server, where there is no window to read the query string from, and React then hydrates against that markup and never revisits it — so the values only appeared once you touched a control. The URL is read again after mount, which is the first moment it exists. The value is also escaped once now rather than twice: URLSearchParams already decodes on the way out, and decoding a second time turned an SVG path full of percent-escapes into something that no longer parsed, besides doubling the length of every link. * fix(catalog): mount the preview with the values a link carried The frame was built from the declared defaults and the shared values were posted to it afterwards, which is too late for anything the composition reads once at init: a path arrived after the mark had already been drawn from the default one, so a link looked right in the panel and wrong on screen. * feat(catalog): the install command follows the values you tuned Copying the Install line gave the plain command back, because that block is generated before anyone touches a knob and had no way to know what changed. The tuned command only existed in the panel Snippet tab, which is not where anyone looks for it. The line now reads the same query string the panel writes, so the two agree without either component knowing the other exists, and a shared link carries the right command too. replaceState fires no event, so the panel announces its own writes. * fix(catalog): send a text variable to the preview once it is finished Every other control in the explorer reports a whole value on every event: a slider at any position is a position, a swatch is a colour. A text field is not. Typing v3 into a badge posted v first, so the preview remounted and rendered a composition built from half a word. The post now waits while a text field has focus and goes out when the edit is committed, with Enter or by clicking away. The field itself is unchanged and still tracks every keystroke. --------- Co-authored-by: Miguel Angel Simon Sierra <miguelangelsi07@gmail.com>	2026-08-10 22:40:01 -04:00
Miguel Ángel	68205dbbc1	feat(cli): search the catalog by meaning, on this machine (#3089) ... * feat(cli): search the catalog by meaning, in three named tiers Browsing the registry means matching names and tags, which fails whenever the author's wording differs from yours. "make the pace feel faster" finds nothing when the move is described as "velocity-driven blur". This ranks by meaning instead. Three tiers, and the command always says which one answered: words shared vocabulary, free, offline, no account on-device bge-small, free, offline, one opt-in download hosted Gemini, free for signed-in HeyGen users The tier is stated because a quietly worse answer looks exactly like a good one. --json carries it as a token alongside dropped, shown, total and top_score, so an agent reads provenance as data rather than matching English that is written to be reworded. Two consents, asked once each, and never conflated. Sending a query is a privacy question, so the prompt says the query is sent. Downloading a model is a disk and bandwidth question, so that prompt talks about size. Neither fires without a terminal: an unattended run sends nothing and downloads nothing unless a flag records that a person agreed. The catalog is derived from registry-item.json rather than from a separate document, so the set that is ranked and the set that can be installed are the same object by construction. Only the on-device vectors are committed; the hosted vectors are nine megabytes and belong on the server. top_score is reported and never acted on. A "nothing matched" threshold looked clean on long briefs and collapsed on the short queries people type: "a logo appears" scores 0.6181 and keyboard mash scores 0.6417, so any cut that catches the noise rejects the real query. The measurement is in the evals directory rather than in this branch. Not covered here. The published recall figures were measured against a separate hand-written document, not against registry text, so they should not be quoted for this catalog until re-measured. The offline tier needs a normal install: a single-file build cannot load the native ONNX runtime, which the command now reports instead of silently degrading. And the drop-detection path has never been observed firing outside its author's tests. * fix(cli): make this branch pass the repo's own gates Three things `bun run lint` and `fallow audit --base origin/main` rejected. CI runs both, so none of this branch would have gone green. Found by running them, not by reading the diff. process.exit in catalog.ts, twice: an invalid --type and a cancelled picker. check:cli-process-ownership reserves that for cli.ts, and the rule is not cosmetic — process.exit tears the process down where it stands, so anything cli.ts has queued to run on the way out is dropped. finishCommand throws a CliResultSignal that cli.ts turns into the exit code, which is what init.ts already does for a cancelled prompt. Three exports with no consumers. normalize keeps its body and loses its export; localEmbedder is the only caller. modelsDirectory goes entirely, having no caller inside its file or out. The WordPieceConfig re-export goes, and with it the import it existed to forward: the type is exported from wordpiece.ts, where its consumers already take it from. Complexity. prepareOnDeviceTier is lifted out of run(), which took run from 64 cyclomatic and CRAP 948 to 54 and 684. That block is one decision — can the offline tier run, and if not, why not — and its only product is a list of warnings, so it reads and tests as a unit, which it could not do inline. The rest is suppressed rather than refactored, each with its reason on the line above. Finishing run() means extracting its three output paths, and that is a refactor of a command this branch already changes for other reasons: a separate initiative, not something to absorb here. Every suppression says what shape the function has and why; a bare marker on a function nobody can justify is how a threshold stops meaning anything. Verified: `bun run lint` exits 0, fallow reports no issues across 27 changed files, and 2540 CLI tests pass. * feat(cli): ship the local search tiers only, drop the hosted one Search now has two tiers, both local: shared-vocabulary word matching, and the opt-in on-device model. The hosted tier, which sent the query to a HeyGen endpoint and ranked it with a hosted model, is removed. This is a scope decision, not a defect. The endpoint works and its own change is reviewed and green; it is simply not what we want to ship first. Landing local only means the feature has no backend dependency, no auth requirement, and nothing leaves the machine unless someone opts into downloading a model. Gone: registry/smartSearch.ts and its test, the --smart and --no-smart flags, the outcome plumbing through the command, the remote branch of applySearch, the remote tier, and the hosted-only JSON fields (ranking, catalog_version, top_score). Also the smartSearchEnabled consent field in telemetry config, which was the persisted storage behind the hosted consent and would otherwise have been left as dead configuration surface. Kept exactly as they were: both local tiers, the --on-device and --yes flags, the download consent prompt, and the runtime check that happens before the download rather than after it. The --json envelope still reports query, tier, tier_detail, shown, total, dropped, warnings and results, so an agent can still tell which tier answered and why. tierToken now distinguishes on-device from words. Verified: lint exits 0, fallow reports no issues, 2522 CLI tests pass, and the command was exercised directly. A query answers on the on-device tier where the model is installed and falls back to word matching where it is not, reporting that fallback in warnings rather than silently. An unknown --type still exits 1 with a readable message, and --smart is now rejected as an unknown flag. * fix(cli): count only moves this registry cannot install as dropped The dropped count was computed against the list left after the user's own --type and --tag filters, so every move the user excluded was reported as one the registry is missing. Filtering made the number go up: the same query reported 277 unfiltered and 302 with --type block. The count exists so a caller can tell "nothing matched your words" apart from "the ranker suggested things this project cannot install". Conflating it with user filtering destroys exactly that signal, and worse, genuine index skew and a self-inflicted filter printed a byte-identical line with opposite remedies -- one means refresh the shelf, the other means drop a flag, and refreshing does nothing. Now counted against the registry rather than the filtered view. The manifest is already fetched whole and narrowed in memory, so keeping the unnarrowed name set costs no extra request, and item loading still runs only on the filtered subset. Verified against ground truth rather than by eye: the vector artifact holds 411 names, the registry holds 168 installable items, and 134 of those names exist in both, so 277 are genuinely uninstallable. The count now reads 277 unfiltered, 277 under --type block, 277 under --type component and 277 under --tag, and the skew it reports is real -- the artifact predates dropping the UI primitives and still ranks moves that are no longer on the shelf. Reported by Vance Ingalls, who also noted this closes an item the status doc listed as unverified. Two earlier sweeps could not make the count fire because neither combined a filter with a query. Tests pin the three cases: a genuinely absent name counts, a filter-excluded name does not, and a fully installable ranking reports zero. * fix(cli): tell the user when meaning search cannot see the catalog The on-device index was fetched once and never revalidated: the only freshness check was two existsSync calls. A move added after that fetch was invisible to meaning search permanently, not down-ranked but absent from the candidate set. The registry manifest on the same command carries a 24h TTL, so the two halves of one feature disagreed about staleness. The dropped count reported over-coverage only, names the index has that the registry lacks. Under-coverage was never computed, so the harmless direction was instrumented and the costly one was silent. Reproduced with an index truncated to 120 of 168 moves: dropped read 0, perfect health, while 48 moves were unreachable. Counts under-coverage from the name list the artifact already carries, so no extra request. Warns only when non-zero, and names the remedy. The remedy had to be made true: --on-device could not refresh a stale index because hasLocalVectors short-circuited the fetch. That flag now refetches when the index is absent or no longer covering. Two defects the reproduction surfaced. A failed refresh reported the tier unavailable while the old vectors were still on disk and still ranking. And the fetch wrote its two files one at a time, so failing between them paired a new name list with an old matrix, a hard load error rather than stale data. It now writes both or neither, which matters more once refresh runs on staleness. top_score returns, scoped to the on-device tier and set to the score of the best result actually shown rather than the ranking head, which can describe a row the caller never received. Also: scripts/ is now typechecked. It never was, which is how a build script that crashes after the paid embedding call, and two scripts whose imports do not resolve at all, went unnoticed. 43 errors fixed, no suppressions. And the docs stop describing a --smart hosted tier that was deleted, an item that does not exist, and a registry refresh that cannot fix a stale vector index. * ci: fail when the search index stops covering the registry The catalog vector artifact is regenerated by hand. Nothing in CI, in package.json or in a hook rebuilds it, because embedding needs the 32 MB model. So adding a registry item silently makes it invisible to meaning search until someone remembers to regenerate. The failure is asymmetric, which is what makes it easy to miss. Removing an item is self-healing: the ranker still scores the dead vector, then filters the name before display, so a user is never offered something they cannot install. Adding one is not: the item is absent from the candidate set entirely, not ranked low. Comparing the two name lists needs neither the model nor a network call, so the gate runs in seconds. CI checks rather than fixes, for the same reason it cannot regenerate. Scoped to blocks and components. Examples are starter projects a user scaffolds, never something catalog ranks, and the artifact carries no vector for them, so demanding one would keep this gate permanently red and it would be ignored within a week. Verified in both directions rather than assumed: adding an unindexed item exits 1 and names it, restoring the registry exits 0. * fix(catalog): rebuild the search index from the registry build-local-vectors.ts read registry/catalog-artifact/catalog.json, a file no script in this repo writes and which is not committed, so the documented regeneration command failed on a missing path. That is why the index could drift from the registry with nothing to run to fix it. It now reads registry/blocks/* and registry/components/* through catalogFromRegistry, the existing helper that already produced the right shape but had no caller. Rebuilding reproduces the shipped 168 rows byte for byte. A lefthook catalog-index command regenerates and re-stages both artifact files whenever a staged registry-item.json changes, mirroring the skills-manifest pattern, so adding or removing an item keeps the index in sync without anyone remembering to. Verified end to end: staging a new item took the artifact 168 to 169 rows and staged it in 0.80s. * fix(cli): refuse a half-downloaded vector cache The two artifact files have to agree on how many rows there are, and until now nothing checked that before writing them. A truncated or wrong-model response landed in the cache and only failed at load, on every later search, until someone cleared it by hand. The pair is now checked first and refused as a unit, and the cache is created 0o700 with 0o600 files rather than inheriting the umask of a directory the caller may have pointed anywhere. Also lifts the capture setup the two preview generators had drifted into sharing into scripts/preview-capture.ts, and splits the vector builders batching and packing out of main. Both were findings the audit attributed to this branch. * fix(cli): keep the catalog vitest run with the tests it runs Restacking took the base package.json wholesale, which dropped the vitest dependency and the scripts/catalog run this PR adds. Both belong here rather than under it. * fix(cli): stop the declined model download from happening anyway Answering no to the on-device download offer recorded no and warned, then carried on. The guard below it is localModelConsent() !== false, which the decline had just made false, so it was skipped rather than taken: control reached recordLocalModelConsent(true), overwrote the answer with yes, and fetched the 32 MB model the user had refused. Next run it never asked again. No test could catch it. The stub pinned localModelStatus to ready, so the prompt never fired, and recordLocalModelConsent was a no-op that recorded nothing. Two tests now cover the offer, and they need three things the old stubs did not model: the run has to look like a terminal, because off one the command treats --on-device as the consent and never asks; the ONNX probe has to answer true, or an accepted offer returns at the runtime guard before it can download; and the status has to follow the recorded answer, or the second offer later in the run fires as well. Removing the return makes the decline test fail. * fix(catalog): let someone without the model still add a component The pre-commit hook rebuilds the search index, and rebuilding needs the 32 MB embedding model. An outside contributor adding a registry item does not have it, so their commit died inside the ONNX loader on an ENOENT naming a path they never set, and the CI gate then told them to run the command that had just crashed. The model is an opt-in for search, not a build dependency, so nobody is charged for it to contribute. The builder checks first and explains itself, exiting 3 for cannot as distinct from 1 for failed. The hook treats 3 as skip and lets the commit through. The gate now names both paths: regenerate if you have the model, leave it if you do not and a maintainer will. Verified both ways: with no model the builder explains and the hook exits 0; with the model it still regenerates byte-identically. * docs: say that anyone can add a registry item, and stop hand-editing a generated file Two defects, one of them the reason 64 stale entries survived in registry.json. The checklist told contributors to add their item to registry/registry.json. That file is generated from the item directories, so an entry added by hand survives until the next regeneration and then vanishes, and one left behind for a directory that no longer exists is worse: hyperframes add resolves the name and then fails on missing files. Both CONTRIBUTING.md and the agent-facing skill reference now run the generator instead. Nothing said contribution was maintainer-only, but nothing said it was not either, and two steps do need assets an outside contributor has no reason to install. Those are now named in a table with what happens if you do not have them, matching how the preview image was already handled. The search index is the new one: the model behind it is a 32 MB opt-in for search, not a build dependency. * fix(cli): harden on-device catalog search * fix(cli): refresh stale catalog vectors * test: create catalog vector temp dirs securely	2026-08-09 22:59:15 -07:00
Vance IngallsandClaude Fable 5	19defeabfe	feat(core,cli): ship the DE parallel router fleet-wide — remove the canary gate ... Deletes the `de-parallel-router` canary entry and the `isCanaryEnabled` guard in render.ts together, leaving the producer's default-ON in place. Net effect for users: the parallel drawElement router is on for everyone again. ## Why, and why not a ramp Gating at 5% was itself the regression. Measured 2026-08-08, the day after v0.7.101 shipped the canary: fleet router exposure fell from 3.13-4.25% of non-CI renders to **0.13%**, roughly 25x, because out-of-cohort installs are explicitly disarmed and #2840 deleted the everyone-armed trial in the same change. 2,537 installs lost a feature they already had. Severity is speed only, never output, and nothing is persisted to disk. PR #2840's body claimed "the canary does not make exposure smaller; it makes it chosen and revertible." That was true of the end state and false of the first step. This lands the end state. Entry and guard go together deliberately: at >=100 the evaluator short-circuits ahead of the CI/seedless exclusions, so removing only the entry would have flipped whatever still resolved false at deletion time, unstaged. ## Both stated blockers are void - **≤4-CPU / Docker coverage gap.** Docker renders never use drawElement — 0 of 4,281 across every CPU tier, software GL gates it out — and the router requires it. No percentage could ever expose Docker, so no ramp closes that gap. ≤4 CPUs yields ~42 drawElement candidates in three days. - **PRINFRA-372.** Its signature has hits on 0.4.12, 0.4.37, 0.6.52, 0.6.93, 0.6.109 and 0.6.110 — versions predating drawElement (v0.7.38) and therefore this router. It is real, still live on 0.7.101, and belongs to the screenshot/beginframe path. 11 reproduction runs across four configurations on the enriched profile (darwin/arm64 25.5.0) came back clean. ## Safety unchanged The per-install circuit breaker and the per-render self-verify are untouched; `HF_DE_PARALLEL_ROUTER=false` remains the user-facing kill switch. Post-canary data at 14 days: >8 CPUs 3.02% revert (177/5,857), 5-8 CPUs 2.40% (6/250) — consistent with the 2.75-3.16% baseline. Revert path is now a code revert rather than a registry edit. That is the trade this shape accepts in exchange for one release instead of two. ## Corrects two claims that shipped wrong `~17x jump in exposure onto <=4 CPUs / Docker` overstated the reach, and `~11% of installs already route` was an OUTCOME (the share clearing eligibility and the old 25-render cap), not an exposure setting — read as a rollout knob it inverts the arithmetic, which is how gating at 5% came to cut exposure rather than ramp it. Both are recorded in render.ts so they are not reintroduced. ## Tests Removed the core wiring assertion and the two CLI canary-gating tests, which pinned a gate that no longer exists. Added the inverse guarantee in its place: an ordinary install must come out of the breaker with the var UNSET so the producer default applies — writing "false" there is precisely what disarmed the fleet at 5%. core 1701 passing, cli 2491 passing, studio canary 29 passing. The 2 failures in play.test.ts reproduce on clean origin/main and are unrelated (#3114 area). oxlint and oxfmt clean. Note: telemetry for this rollout stops with the entry — `$feature/canary-de-parallel-router` and `canary_reason_de_parallel_router` are emitted from the registry, so the `Ramp —` tiles and the exposure-floor alert on PostHog dashboard 1918875 go blank once this ships. Watch drawElement engagement on 1807532 instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-08 14:10:25 -07:00
Miguel Ángel	0bda6b55b8	feat(cli): track which registry items add installs (#3099) ... * feat(cli): track which registry items `add` installs `cli_command` records that `add` ran and nothing about what it installed, and the registry is served from raw.githubusercontent.com, which gives no per-item counter either — so there is no way to tell which block or component people actually pull, and no way to know what is worth building more of. Emit one `registry_item_added` event per item written into a project, from `runAdd` after the install succeeds. That is the single choke point: the bulk `add <tag>` path re-enters it per item, and a failed or compatibility-refused install throws before it, so a refused install is never counted as a download. `requested` separates the item the user named from the transitive `registryDependencies` pulled in behind it; without it a popular dependency outranks everything that depends on it. Item names are public registry identifiers, never user content or project data, and the event goes through `trackEvent` — an install that opted out via `hyperframes telemetry disable`, `HYPERFRAMES_NO_TELEMETRY` or `DO_NOT_TRACK` sends nothing. * test(cli): cover `add` telemetry end to end against the built CLI The unit tests assert the emit seam and nothing past it. `shouldTrack()` short-circuits whenever `isDevMode()` is true, and that is true for any `.ts` entry, so under vitest a real event and no event are indistinguishable and the transport is never exercised at all. Drive the built CLI instead and assert on the HTTP body it actually produces: one event per installed item, the dependency reported with `requested: false`, an opted-out install sending no request at all (not merely one without this event), and a refused install counting nothing. Two fixtures, because neither case is reachable through the real registry. The registry origin is a first-class project setting, so a local one supplies the `registryDependencies` edge that no shipped catalog item declares today; and `globalThis.fetch` is wrapped to capture the batch rather than send it. The faked 200 is load-bearing: only a failed flush leaves events queued, and only a non-empty queue spawns the detached `flushSync` child that would bypass the hook and reach production analytics. Verified the check can fail — forcing `requested: true` for every item turns it red on exactly the dependency assertion.	2026-08-07 16:00:23 -07:00
James	7640adc5f2	fix(cli): refresh identity persistence classification	2026-08-07 09:31:59 -07:00
Vance IngallsandClaude Opus 5	b3990ac789	feat(studio): emit the canary reason on Studio events too ... Review caught that the same anti-pattern was still live in the Studio binding: canaryEventProperties destructured only `enabled` and dropped the reason. Its own doc comment promised 'identical shape to the CLI, so a rollout spanning both reads as one flag' — which the CLI-only fix had just made false. This matters beyond symmetry. A CLI-launched Studio adopts the CLI's decisions and shares its bucket seed, so a cohort flip can surface on either surface. Emitting attribution on only one leaves Studio-observed flips unattributable and makes the two flip counts irreconcilable — and Studio is the surface most likely to expose a shared-seed-with-diverging-id pattern, which is the open question the reason exists to answer. Also adds the no_unit_id emission test the CLI side advertised but never asserted, and a Studio pair pinning that a URL override and a cohort roll produce the same assignment with different reasons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-06 14:39:56 -07:00
Vance IngallsandClaude Opus 5	076657a639	feat(core,cli): emit the canary decision reason alongside the assignment ... The calibration contract deferred this until the stability check came back dirty. It did: the first fleet read found 304 installs (1.08%) reporting both values for a canary whose percentage never moved, and the genuinely anomalous ones could not be separated from a developer toggling HF_CANARY_*, because the assignment alone is identical in both cases. resolveCanary has always computed the reason and canaryEventProperties dropped it. Now every canary emits canary_reason_<name> beside its assignment. Deliberately outside the $feature/ namespace: PostHog treats those as flag values, and a non-boolean there would corrupt the flag's own breakdowns. Two of the six wire values are immediately useful beyond override attribution. 'excluded' identifies CI installs, which today have to be dropped by joining on is_ci — conflating them with out_of_cohort is what made the first accuracy read look like a significant failure (9.22% against a 10% target) when it was not. 'no_unit_id' surfaces the fails-closed corner. The reason is optional on the core helper so existing callers are unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-06 13:57:08 -07:00
WaterrrForever	3b65321a20	feat(cli): classify identity persistence on every telemetry event (#3065) ... * feat(cli): classify identity persistence on every telemetry event Install-grain metrics currently trust every anonymousId equally, but ephemeral/isolated-HOME workloads mint a fresh id per run — one machine produced 2,956 rotating render identities since Jul 30 (94.4% seen on a single render command), inflating acquisition and diluting per-install penetration while looking like real product usage. Every event now carries: - identity_persistence: durable (id loaded from a preexisting config — proven to survive a process boundary) | unknown (minted+persisted this run; an ephemeral HOME is indistinguishable from a genuine first run from inside one process) | process_only (persist failed). Sticky per process so a fresh install re-reading its own write cannot self-promote. - config_write_outcome: ok | ok_unmirrored | failed for the identity- establishing write; absent when the id came from disk. - invocation_id: random uuid per CLI process, so one invocation's events group even when the install identity is untrustworthy (unlike run_id, which needs an orchestrator to set HYPERFRAMES_RUN_ID). Install metrics can then count only durable identities, and a daily churn monitor can alert on the unknown share. * fix(cli): require the anonymousId to come off disk before classifying durable Review finding: materializeConfig mints a replacement anonymousId when a hand-edited/image-baked config lacks one. That replacement only reaches disk when the bucket-seed backfill happens to write; with a seed present the read path performs no write at all, so the install re-mints a fresh id every run while the unconditional durable branch stamped each of them with the one label durable-only counting is allowed to trust. durable now requires parseNonEmptyString(parsed.anonymousId): a minted replacement classifies like a fresh mint — by the backfill write outcome when that path runs (unknown/process_only), and process_only on the no-write path where the id provably dies with the process. Two tests pin both shapes.	2026-08-06 22:19:14 +08:00
Miguel Ángel	b9233525b2	feat(cli): add device authorization login (#2836) ... * feat(cli): add device authorization login * refactor(auth): simplify device authorization flow * fix(cli): harden device authorization flow * refactor(cli): simplify device auth validation tests	2026-08-04 20:36:42 -07:00
Vance IngallsandClaude Opus 5	3f8dca165d	fix(cli,core): refresh telemetry posture at the render boundary ... R6/R7 blockers. An already-open Studio kept emitting server-side render telemetry after another process disabled CLI telemetry. refreshTelemetryPosture() only ran while serving a fresh SPA document and on /api/telemetry-identity, which Studio has no consumer for, so the render POST and its async outcome used the posture cached when the preview server booted. It now refreshes at the render boundary and again immediately before the completion/error event, so an opt-out during a long render is honoured. The identity tests were passing vacuously: their mocks omitted readConfigFresh and resetTelemetryPostureCache, and the resulting missing-export error was swallowed by the refresh's own catch. Mocked properly, plus the enabled -> external disable -> next response transition and the suppression path at the layer that drops the event. A full reset also did not persist its new lineage in a long-lived process: syncInstallState returned early on a process-lifetime memo even after ~/.hyperframes was deleted, so install-state was never recreated and the next config-only re-mint rolled a third seed instead of inheriting the second. The memo is now revalidated against the file. Also drops a stale reference to assertNoOverdueCanaries and stops the workflow and docs claiming the sunset job routes anything to the owner — it names them in the run log and notifies nobody. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-01 18:11:45 -07:00
Vance IngallsandClaude Opus 5	6f0df2640b	fix(cli,studio,core): close five R5 telemetry and canary findings ... - A long-lived preview cached its telemetry posture in two places (readConfig and shouldTrack). Running `telemetry disable` in another terminal left it resolving canaries and injecting the CLI id for hours. Both caches are now dropped together at a request boundary. - Studio minted and shipped a telemetry id for every render regardless of the browser profile's opt-out, and the server emitted the outcome under CLI policy, which cannot see localStorage or DNT. The browser now sends an explicit telemetryOptOut, distinct from an old client's omission. - Any non-empty HYPERFRAMES_PREVIEW_HOST disabled the DNS-rebinding guard, so even a loopback bind accepted a hostile Host. The guard now holds for loopback binds and, on a LAN bind, admits only names this machine answers on. - sunsetAfter had no reader of the current date. A scheduled workflow runs scripts/check-canary-sunset.ts weekly, so a failure lands on the rollout's owner rather than on an unrelated PR author. - The install-state seed memo outlived `rm -rf ~/.hyperframes`, resurrecting a cleared cohort. Removed; it only saved a read on a readConfig cache miss. Docs updated for the Host rule and the 100% exclusion carve-out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-01 17:21:41 -07:00
Vance IngallsandClaude Opus 5	3f69a2c635	fix(cli,core,studio): close 15 review findings + 2 R5 blockers ... R5 blockers - Negative install-state latch was cached for the process lifetime, but only `true` is monotonic across processes. A long-lived preview server held a stale `false` and could re-enrol after another process tripped the breaker. Only the positive is cached now; `false` re-reads. - The real breaker writer used writeConfig(), which collapses {ok:true, mirrored:false} to success, so a run that mirrored nothing reported done with the latch only on the erasable store. It consumes writeConfigWithResult and retries until both stores carry it. Bucketing integrity - Storage-restricted Studio profiles all bucketed on the literal "anonymous": computed against the shipped hash, 100% of them were enrolled in calibration-50 rather than 50%, and they merged into one PostHog person. Per-session random id instead — persists nothing. - bucketSeed had read/write authority backwards: install-state is write-once authoritative, but readConfig took config.json's blindly, so the stores could hold different seeds until a re-mint flipped every cohort. Merged on read, like the latch. - An unwritable ~/.hyperframes with no config.json re-minted per call, re-rolling the seed on every command, and the "cohorts will not be stable" warning was unreachable on that path. - A corrupt PRE-MOVE state file was never deleted, so a machine reset with `rm -rf ~/.hyperframes` reported predecessorFound/stateFileCorrupt forever — poisoning the exact metric this work exists to produce. Opt-out honoring - CLI canary decisions memoized per process, so `hyperframes telemetry disable` during a running preview server was ignored for hours while the server kept serving pre-opt-out decisions. The memo is keyed on the telemetry posture. - shouldTrack() memoized, contradicting policy.ts's documented "not memoized" contract that policy.test.ts asserts. - The Studio override path resolved the bucket unit eagerly as an argument, minting and PERSISTING a tracking id for an opted-out profile — a value evaluateCanary discards unread. - Storage reads could throw out of telemetry into a post-commit catch block, reporting an already-committed edit as failed. - readConfig printed an unsilenceable stderr warning on every invocation for installs that opted out of telemetry entirely. Host split - isLoopbackHost rejected 0.0.0.0, so the documented HYPERFRAMES_PREVIEW_HOST LAN mode silently lost CLI→Studio identity stitching and split one user across two PostHog persons. Identity is now allowed when the operator explicitly opted into LAN binding. - Corrected the comment claiming the guard refuses spoofed Hosts: a non-browser client sets Host freely. It is a browser DNS-rebinding mitigation, not access control, and now says so. Semantics and test hygiene - percentage:100 did not mean everyone — exclude and no_unit_id sat above the fast path, so the registry's "delete the entry at 100" step was an unstaged flip for CI and seedless installs. - CLI cohort adoption returned before evaluateCanary, dropping Studio's own webdriver exclusion. - overdueCanaries() was asserted against wall-clock time, so the whole core suite would go red on 2026-09-15 for every unrelated PR; and `>` against midnight made a canary overdue ON its sunset date. - Statistical assertions ran on unseeded randomUUID() populations tight enough to fail ~1 run in 200. Seeded. Also: broke a config -> policy -> transport -> config import cycle by moving POSTHOG_API_KEY to a leaf module. Tests: 2347 CLI (bundle absent), 3153 Studio, 1450 core. Fault injection covers the latch, seed authority, LAN identity, webdriver exclusion and the anonymous-bucket fix. Two pre-existing tests asserted behaviour these findings identify as wrong (shouldTrack memoization, 100%-excludes-CI) and were rewritten with the reasoning stated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-31 01:27:37 -07:00
Vance IngallsandClaude Opus 5	5e2a9432f1	fix(cli,studio): close the five R4 blocking gaps ... P1 — the required Test lane was red, and it was my test. The hostile-Host SPA case asserted a 200, which only holds when packages/studio/dist is built: true on a dev box, false in CI, so it passed locally and failed there. The Host split moved into a pure buildStudioHeadScriptsForHost() and is asserted directly; the route test no longer depends on build state. Verified by running the CLI suite with the bundle moved aside — 2330 pass. P1 — studio:* still bypassed most privacy controls. It honoured two localStorage keys but not navigator.doNotTrack, VITE_HYPERFRAMES_NO_TELEMETRY, Vite dev mode or API-key eligibility, and canary enrolment honoured a different single control. New telemetry/policy.ts is the one answer to "may this profile be measured", consumed by both transports and by enrolment. It imports only ./config, so no cycle with the modules that import it. Each control is asserted individually. P1 — LAN/remote preview lost the authoritative decisions. Withholding the whole head script for any non-loopback Host also dropped the safe {enabled, forced} map, sending a supported HYPERFRAMES_PREVIEW_HOST= 0.0.0.0 Studio back to re-deriving. Identity injection is now gated separately from decision injection: identity is loopback-only, decisions always publish. P1 — the breaker latch was neither authoritative nor truthfully persisted. syncInstallState swallowed its own failures so writeConfigWithResult always reported ok, and reads took the flag only from config.json. The latch is now merged into every effective read, which makes install-state authoritative and closes both the failed-mirror and stale-concurrent-writer paths; the write additionally reports mirrored: false rather than swallowing. P2 — public contracts. canary-rollouts.mdx said a config wipe loses the breaker (it does not) and documented the superseded {name: boolean} map with unconditional CLI precedence; both corrected, with the precedence ladder written out and the override exception stated explicitly. PR body rewritten — it still named ~/.local/state, claimed state survives deleting ~/.hyperframes, and carried stale counts. Tests: 2330 CLI (bundle absent), 3151 Studio, 24 core. Fault injection: reverting each fix alone fails 5 CLI / 5 Studio. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 23:03:20 -07:00
Vance IngallsandClaude Opus 5	f81ab0162e	fix(cli,studio): close the four R3 blocking gaps ... P1 — SPA route bypassed the DNS-rebinding guard. Guarding only /api/telemetry-identity left the catch-all as an open side door: a rebound origin could fetch `/` and read __HF_CLI_DISTINCT_ID and __HF_CLI_BUCKET_SEED straight out of the returned HTML. The SPA response now applies the same isLoopbackHost() check; an untrusted Host still gets a working Studio, just with no identity, seed or decisions injected. Route-level regression added. P1 — a CLI cohort roll could override Studio's own opt-out. decideStudioCanary() adopted the injected decision before checking isOptedOut(), so CLI-telemetry-on plus Studio-opted-out still enrolled Studio. A bare boolean could not express the difference between a deliberate override and an ordinary cohort roll, so the injected map now carries provenance ({ enabled, forced }). Forced wins outright — it is the documented escalation channel and must behave the same on both surfaces — while a percentage roll now loses to this profile's opt-out. Full interaction matrix tested. P1 — the legacy studio:* path sat outside both contracts. utils/studioTelemetry.ts shipped its own opt-out key and its own send loop, so the documented hyperframes-studio:telemetryDisabled did not silence it and its events carried no cohort assignment. It now honours both keys (the legacy one stays, so nobody already opted out is quietly re-enabled) and mixes in canaryEventProperties(), making "every telemetry event carries the assignment" actually true. P2 — partial salvage could drop a tripped breaker. salvageInstallState() discarded the whole record when markerAt and bucketSeed were both unusable, taking deParallelRouterTrialFired with it and re-enrolling a machine whose router already failed. All three fields are now independently salvageable. Docs: canary-rollouts.mdx said "disabling telemetry disables the reporting, not the enrolment" — exactly backwards since the opt-out gate landed. Corrected; checked for other copies, none. Tests: 13 new (4 opt-out precedence, 4 legacy-path opt-out and canary props, 3 route-level host guard, 2 breaker salvage). Fault injection: each of the four fixes reverted independently fails its own tests (2 CLI + 1 Studio + 2 Studio). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 18:35:22 -07:00
Vance IngallsandClaude Opus 5	31361b8e5b	fix(cli,core): close the remaining canary review findings ... Six findings from review, none behaviour-critical on their own but three of them quietly corrupt the data the rollout is judged by. Endpoint no longer serves bucketSeed (studioServer.ts). Studio gets its canary answers from the injected decisions map now, so nothing needed the seed over HTTP — and an unauthenticated local endpoint is a strictly worse place for it than a script scoped to Studio's own document. The endpoint itself predates this PR and still serves distinctId, so it also gains a Host guard: a remote page can rebind its hostname to 127.0.0.1 and read the response as same-origin, but the request still carries THAT hostname, which is what makes it refusable. predecessorFound no longer reports corruption as a fresh install. It returned null for both "file absent" and "file unreadable", so a partial disk write looked like a new machine — understating recoverable churn, the one thing the field measures. Now distinguishes absent from corrupt and emits install_state_file_corrupt alongside. A mangled markerAt no longer discards a salvageable bucketSeed. markerAt is only a timestamp and can be restamped; the seed cannot be recovered, and losing it silently re-rolls the install's cohort. The seed backfill no longer ignores its write result. An unwritable ~/.hyperframes meant a different seed every invocation with no diagnostic, and made the field's own "backfilled once" docstring false. Warns once per process with the underlying error. FNV-1a's ASCII constraint is now explicit rather than incidental. It hashes UTF-16 code units while reference FNV-1a is byte-oriented, so the two agree only on ASCII; the registry's kebab-case assertion is what makes non-ASCII unreachable, and both ends now say so. Not a live bug — names are kebab-case and units are UUIDs. de-parallel-router is pinned at 0%. The registry is data, so a ramp is a one-line edit with no review surface, and its own description says to ramp only alongside the circuit breaker. Tests: 8 new (corruption vs absence, seed salvage, backfill write failure, 17 host-guard cases, registry pin). One existing test asserted predecessorFound: false on corruption — that was the bug, updated with a note. Fault injection: restoring the old corrupt handling fails 4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 16:38:11 -07:00
Vance IngallsandClaude Opus 5	98b23a8850	fix(cli,studio): adopt the CLI's canary decisions in a launched Studio ... Closes both cross-surface findings with one mechanism. The CLI publishes window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }); a CLI-launched Studio takes it as authoritative over its own seed, URL override and the registry percentage. Studio re-deriving could not agree with the CLI in three cases: - Telemetry off. The CLI resolves telemetry_opt_out, but Studio's opt-out is a separate localStorage flag it cannot see, so it would evaluate normally and could enrol on a render the CLI excluded. The previous commit gated each surface independently; that fixed silent enrolment per surface but NOT the disagreement between them. - HF_CANARY_* override. Env vars never cross into the browser — Studio reads only its URL param / sessionStorage — so a support session forcing a canary on got the CLI forced and Studio guessing. - No seed injected. Studio falls back to a different unit id, i.e. a different bucket. Shipping the decision instead of the inputs makes divergence structurally impossible: one evaluation, two surfaces. It also exposes strictly less — booleans about features, rather than the seed buckets derive from — which is why it is safe to publish with telemetry off, the case it exists for. Studio still evaluates locally when standalone, or for a canary the CLI did not publish, and ignores a non-boolean value rather than trusting it. Tests: 6 Studio (CLI-off wins over unset local flag, CLI-on with no URL param, beats contradicting override, beats seed, falls back per-canary, rejects non-boolean) and 4 CLI (decisions with telemetry off and no identity, alongside identity when on, script-tag escaping on a hostile canary name, throwing resolver degrades to identity only). Four existing identity tests asserted the old "nothing when telemetry off" contract and were updated; the registry is now mocked there so string assertions don't move when a canary is added or ramped. Fault injection: dropping the adoption fails 4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 16:29:31 -07:00
Vance IngallsandClaude Opus 5	4f464dc424	feat(cli,studio): telemetry opt-out is canary opt-out ... Both reviewers flagged the same gap: seed injection was gated on telemetryShouldTrack(), but canary EVALUATION was not. An install with DO_NOT_TRACK=1 was still bucketed and still had real code paths flipped (e.g. HF_DE_PARALLEL_ROUTER), silently and unmeasurably. A canary is a measured rollout — we enrol a slice precisely so it can be compared against everyone else. An install that sends nothing can't be compared, so enrolling it buys no signal and only changes that user's code path, on an experimental feature, without their knowledge. That is the wrong side of an opt-out. Resolves to a new `telemetry_opt_out` reason BEFORE bucketing, so no cohort is assigned at all. Distinct from `excluded` because "why is my canary off" has a very different answer for CI than for opted-out, and the reason never reaches telemetry by construction. Covers every opt-out route: persisted preference, the runtime env vars and dev/telemetry-disabled builds via policy.ts, and Studio's hyperframes-studio:telemetryDisabled. An explicit HF_CANARY_* / ?hf_canary_*= override still wins — a deliberate local choice, not silent enrolment, and the documented way to exercise a canary with telemetry off. The CLI check mirrors shouldTrack() rather than importing it: client.ts already imports canary.ts for canaryEventProperties, so depending on it would be a cycle. Both read the same two inputs, so they cannot disagree. Tests: 9 new across CLI and Studio (preference off, each runtime override, no bucket assigned, override still honoured, flag properties all-false). Fault injection: removing the CLI gate fails 6, removing the Studio gate fails 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 16:11:34 -07:00
Vance IngallsandClaude Opus 5	9f2b892a71	fix(cli): keep the canary bucket seed inside the config dir ... Adopts the #2904 pattern for bucketSeed. The seed rides in install-state.json, which now lives beside config.json in ~/.hyperframes rather than in ~/.local/state/hyperframes/. Review rejected persisting state outside the config dir to defeat a user's reset, and that objection is sharpest for the seed: it is the one field that would turn install-state into a persistent pseudonymous identifier surviving `rm -rf ~/.hyperframes`. The carryover still earns its place, just against the churn that actually happens. config.json is rewritten on every command and every render, and readConfig recovers from any parse/permission/IO failure by minting a fresh identity — so a re-mint would reshuffle cohorts mid-rollout. A no-schema file written once at mint is decoupled from that without leaving the directory. Config re-mint: cohorts hold. Directory deleted: cohorts go too, deliberately. A pre-move seed is adopted by the same migration, so installs already carrying one do not have a live cohort reshuffled under them. Tests: seed survives a re-mint, does NOT survive deleting the config dir, and migrates from the pre-move path. Prose in config.ts, canary.ts and canary-rollouts.mdx corrected — it still claimed cohorts survive a wipe. Docs gain a removal-path section. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 15:21:46 -07:00
Vance IngallsandClaude Opus 5	8dd20ac2e8	feat(cli): bucket canaries on a machine-lineage seed, not the telemetry id ... Cohort membership now survives a config wipe. Canaries bucket on a dedicated bucketSeed (fresh random UUID, distinct from anonymousId by design) that is mirrored write-once into the install-state file and inherited at mint: a wipe re-rolls the telemetry id but never the canary assignment. This removes cumulative-exposure drift for the recoverable churn bucket entirely — the residual drift comes only from fresh machines, containers, and genuinely new users — and keeps before/after comparisons valid across a reinstall. The seed is never emitted in telemetry (only the resulting true/false assignments are), so it does not link the old id to the new one server-side. The residual linker is the flag vector itself (k bits for k live canaries), documented as such. An explicit reset still works by deleting the state file, and the no-identity test now also asserts the seed differs from the anonymousId. Cross-surface coherence: the CLI's studio server injects the seed as window.__HF_CLI_BUCKET_SEED (same telemetry gate and script-escaping as the distinct id, and on the /api/telemetry-identity fallback), and the Studio binding buckets on it when present — without this the CLI would bucket on the seed while Studio bucketed on the distinct id, splitting one machine across cohorts (calibration check 4 would catch exactly this). Standalone Studio still buckets on its localStorage id: the browser has no second storage location, so that id doubles as the seed. Legacy configs are backfilled once (lineage seed if the state file has one, else minted) and persisted immediately — an unpersisted seed would re-roll cohorts every process. Safe to ship in the same release as the first canaries: no prior release emitted canary properties, so the bucketing-unit change is unobservable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 15:16:16 -07:00
Vance IngallsandClaude Opus 5	a1682e1228	feat(core): emit canary assignments as PostHog flag properties ... Replaces the single `canaries: "a,b"` telemetry property with PostHog's own flag shape, one property per registered canary: $feature/canary-de-parallel-router: "true" | "false" PostHog treats `$feature/<key>` as a first-class flag property, so breakdowns, funnels split by cohort and the experiment surfaces work on a canary with nothing configured server-side. The decision still happens locally: the render path forbids render-time network calls, behaviour must not depend on analytics being reachable, and neither the CLI nor Studio ships posthog-js (both hand-roll a batch POST, so there is no SDK to evaluate a real flag with). Decide locally, analyse natively. Two decisions worth recording: - BOTH ARMS ARE EMITTED. A non-enrolled install reports "false" rather than omitting the property. Absent means "this build predates the canary", which is a different fact from "this install is control" — collapsing them makes a ramp unreadable, because you cannot separate a control group from an old version. - KEYS ARE NAMESPACED with a `canary-` infix. A real PostHog flag namespace already exists in this project, owned by the web app (`enable-chat-tab`, set by posthog-js from `$lib=web` events). Namespacing guarantees a canary key can never alias a real flag key and have the two fight over one property. Values are the strings "true"/"false" to match how PostHog records boolean flag values, so the property is directly comparable to a real flag. 98 core / 1437, 166 cli / 2194, 269 studio / 2982 green; tsc clean across all three packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 15:14:58 -07:00
Vance IngallsandClaude Opus 5	71ee156dac	feat(studio): browser canary binding + leaf subpath imports ... Adds the Studio (browser) binding so a canary can span the CLI and the editor, and fixes a bundling mistake the studio test suite caught. ## The binding Same public API as the CLI — `isCanaryEnabled("name")` — so a call site reads identically whether it runs in Node or the browser. Three inputs differ: - UNIT ID: `resolveStudioDistinctId()`, which already adopts `window.__HF_CLI_DISTINCT_ID` when the CLI launched Studio. A CLI-launched Studio therefore lands in the SAME cohort as the CLI: a rollout spanning render and editor is coherent for that user instead of enrolling their terminal but not their editor. A test pins that the id is passed through unmodified — prefixing or re-hashing it would silently break that parity. - OVERRIDE: no `process.env` in a page, so `?hf_canary_<name>=on` mirrored into sessionStorage. Session scope is deliberate. A URL is the right carrier (shareable — "support: open this link"), but persisting a URL-borne override to localStorage would let one click silently pin a browser into a cohort forever, long after anyone remembers why. Closing the tab is the reset; `=reset` clears it explicitly. - EXCLUSION: `navigator.webdriver` stands in for the CLI's `is_ci`. Automated browsers mint a fresh localStorage id per run, so they would hop cohorts between runs — noise in the signal, nothing learned about real users. An override still reaches them, which is how you test a canary under Playwright. Studio's `trackEvent` now attaches `canaries` to every event, mirroring the CLI. ## The bundling fix Importing the `@hyperframes/core` barrel into studio browser code broke two unrelated hook test files with an esbuild TextEncoder invariant violation. The barrel re-exports the whole core surface (parsers, lint, studio-server), so it drags a Node-oriented dependency graph into a browser bundle — the test failure was the symptom, the bundle bloat was the bug. `@hyperframes/core` now exposes `./canary` and `./canary-registry`, declared in packages/core/package-subpaths.json (the generated source of truth for exports — hand-editing package.json is reverted by the sync script) and marked `environments: [browser, bun, node]`. Both the studio AND cli bindings import the leaf modules; the CLI gets the same benefit for a different reason, since this resolves on the startup path — the reason the producer is lazily loaded. Verified: the two hook files pass again; 269 studio files / 2982 tests, 98 core / 1433, 166 cli / 2194 green, `bun run lint` clean including the subpath check. Fault-injection confirms both design decisions are pinned — swapping session for local storage fails the scope test, prefixing the unit id fails the CLI/Studio cohort-parity test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 15:14:35 -07:00
Vance IngallsandClaude Opus 5	df1521a0b6	feat(cli): attach canary cohort to telemetry, harden canary tests, document ... Follow-up to the canary primitive. Telemetry: every event now carries a `canaries` property listing the cohorts the install is enrolled in, attached in `trackEvent` so it lands on ALL events rather than renders only — a staged rollout is only as useful as the ability to split any metric by cohort. Resolved after the shouldTrack guard, so opted-out installs never pay for it, and omitted entirely (not null or "") when the install is in no canary, since PostHog treats those as real values. Test hardening, after validating the shipped code against 60k synthetic and 101 real fleet install ids: - Pin FNV-1a against canonical vectors, AND assert the shipped canaryBucket actually uses that hash. Without the second assertion the first is tautological — it would only prove the test's own copy is correct while canary.ts drifted to a different hash, silently reshuffling every live cohort. Fault-injection confirms only this assertion catches a hash change; the distribution tests stay green because a perturbed hash is still well-distributed. - Tighten the share test from a 0.6x-1.4x band to +/-1 percentage point. Measured error was 0.16pp at n=60k, so the old band would have passed a badly skewed hash. - Add chi-square uniformity across all 100 buckets (chi2 89.0 vs 148.2 critical at p=0.001). A lumpy hash yields roughly the right total share while overloading some buckets, so the share test alone cannot catch it. - Assert N concurrent canaries enrol binomially rather than in lockstep: 8 canaries at 10% put ~43% of installs in none and zero in all eight, matching binomial(8, 0.1). Correlated slices would put ~10% in all eight. Also verified 88,443 of 88,448 fleet install ids are well-formed UUIDs; the 5 that are not fail closed, which is the intended direction. Docs: docs/contributing/canary-rollouts.mdx, registered in docs.json (an unregistered page is invisible in the nav). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 15:14:35 -07:00
Vance IngallsandClaude Opus 5	3aea786687	feat(core): percentage-based canary rollouts ... Adds a reusable staged-rollout primitive so a change can ship to a stable slice of installs instead of all-or-nothing. The gap it fills: the repo carries ~49 HF_*/PRODUCER_* booleans and every one is binary — a feature is either off (and therefore unexercised on real traffic) or on for everyone (and therefore a fleet-wide bet). The parallel-drawElement router sat in that gap for weeks: default-off produced almost no signal, and flipping it default-on would have exposed 100% of eligible installs at once. Shape: - packages/core/src/canary.ts — pure evaluator. No fs, no network, no `process`; the caller supplies the unit id and overrides, so it imports cleanly into the CLI, producer, engine, studio-server, the browser-side studio bundle and the embeddable player. FNV-1a rather than node:crypto for the same reason. - packages/core/src/canaryRegistry.ts — every rollout in one table (name, percentage, owner, description, sunsetAfter), so "what is rolling out, to whom, owned by whom" is answerable without grepping 49 env vars. - packages/cli/src/telemetry/canary.ts — supplies the three things only the CLI knows: anonymousId, the HF_CANARY_<FEATURE> override, and is_ci. Day-to-day API is `isCanaryEnabled("name")`. Three properties the tests pin, because getting them wrong is subtle: - Slices are INDEPENDENT per feature: the bucket hashes `feature:unitId`, not the id alone. Bucketing on the id would hand every concurrent experiment to the same unlucky cohort and make two rollouts unreadable apart. - Ramping is INCLUSIVE: `bucket < percentage`, so widening 10 -> 25 keeps the original cohort and before/after comparisons survive the ramp. - It fails CLOSED: no unit id, unknown name, or CI install means not enrolled. A canary exists to bound blast radius, so "we don't know who this is" must never mean "enrol everyone". Registry entries also carry a sunset date, and a test fails once one is past due — a canary that outlives its rollout is a permanent fork of the product with none of the review a permanent fork would get. Ships with de-parallel-router registered at 0%: inert, and ready to ramp in a patch release once #2840 lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 15:13:56 -07:00
Vance Ingalls	dae1b63d4a	fix(cli): move install-state into the config dir so deleting it is a full reset	2026-07-30 13:15:20 -07:00
Vance IngallsandClaude Opus 5	d74afc7b7d	feat(engine): measure live DOM size on every render, not just probed ones ... The short-comp routing gate can only read a live element count when a probe session exists, and the first v0.7.83 data shows that is far rarer than estimated: 17% of renders (86/503), not the ">=28%" the video-presence proxy suggested. The other 83% fall back to a static source scan, which is exactly blind to the shape that motivated the live count — small markup, thousands of script-created nodes. That leaves the fleet element-count distribution unknowable for most renders, and the observed distribution is already surprising: p99 ~900, max 1,420 against a 2,500 ceiling calibrated on 7k/20k/40k synthetic nodes. Either the ceiling is close to irrelevant, or the large-DOM tail is hiding in the 83% we cannot see. Both readings change what PR B should do, and neither is decidable from probed renders alone (they are a biased sample — they got a probe *because* they carry media or unresolved compositions). So measure it where every render already goes: capture-session init. `collectSessionInitTelemetry` gains a querySelectorAll("*") count beside the tween count it already collects, riding the same channel to `observability_init_element_count`. This is observational only — capture has begun, far too late to route on — and it deliberately does not feed the gate. It answers the distribution question the gate cannot. Coverage for this channel is proven rather than assumed: the tween-count fix that shipped in v0.7.83 took the clamped-parallel bucket from 0/272 renders to 217/217, and 23.1% -> 100% overall. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-29 22:55:12 -07:00
Vance Ingalls	941167dd3c	Merge pull request #2874 from heygen-com/feat/breaker-carryover ... feat(cli): roll circuit-breaker state over across config wipes	2026-07-29 17:40:04 -07:00
Vance IngallsandClaude Opus 5	4dbf0d90b0	fix(producer): fail closed when no live element count is available ... R4 review finding, and the comment I wrote in R3 was simply wrong: the probe session is NOT running for every render. probeStage's needsBrowser gate launches one only for unknown duration, unresolved compositions, or specific media cases — and hasRuntimeInsertedMedia matches only createElement("video"|"audio"), never createElement("span"). So the exact shape that motivated the live-DOM fix (a known-duration, media-free caption comp building thousands of nodes in script) gets NO probe, falls back to the static source scan, reads as ~2 elements, and could enter the applied cohort at 40k live nodes. The R3 fix measured the right thing but only for the population that already had a probe. Now the count carries provenance and the band fails closed: - resolveCompositionElementCount returns { count, source: "live" | "static" }. Only "live" — an actual DOM measurement — may open the band. - resolveDeShortBand gains a third decisive outcome, "unmeasured", for the static case. It deliberately does NOT report skipped_elements: a static undercount is not a real oversize observation, and putting it in the control arm would contaminate the DiD just as putting it in the treatment arm would. Neither cohort; never routes. - composition_element_count_source ships alongside the count, so the fleet rate of "static" sizes the population a future conditional-probe-launch would unlock — which is the data PR B needs to decide whether that launch cost is worth paying. Regression coverage walks the real chain rather than a full render, using the production functions in pipeline order: probeRequiresBrowser (newly extracted from the inline needsBrowser expression, so the gate is testable at all) returns false for the caption-comp shape → the resolver reports static and a count under the ceiling → the band reports unmeasured, not applied. Fault injection confirms it bites: removing the one guard line fails exactly these three tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-29 17:02:26 -07:00
Vance IngallsandClaude Opus 5	ec76985f40	fix(cli): simplify nextInstallState's dead hadFired branch (review nit) ... Both reviewers (Rames, Magi) independently flagged the same thing: by the time the return statement executes, hadFired is always false — the guard above already returns early for every case where hadFired was true. The merge expression wantFired || hadFired || undefined was defensively correct but misleading; it reads as "OR the two together" when the function has already established only one of them can be true here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-29 11:28:57 -07:00
Vance IngallsandClaude Opus 5	0749cd9ff8	feat(producer): open the DE single-worker inversion to short comps under an element ceiling ... 31% of fleet renders (24h, v0.7.78+) are DE-eligible comps clamped to parallel screenshot purely because they sit under the 900-frame inversion floor — the median fleet render is ~250-600 frames, below every DE entry threshold. This opens a 250-899 frame band, gated on composition size. Measured, not assumed. A controlled sweep (fixed synthetic content, {250,400,600,900}f, single-DE vs parallel-screenshot-W4, 3 reps, capture mode verified per row, AC power, load-gated) showed single-DE winning 1.16-1.24x at every size — but only for content in constant motion. A follow-up 2x2 found motion and DOM size pull in OPPOSITE directions, so neither alone predicts the winner (ratio = ss4/de1, >1 means DE wins): 24 movers / 0 nodes -> 1.05 320 movers / 0 nodes -> 1.24 320 movers / 7000 nodes -> 1.09 24 movers / 7000 nodes -> 0.96 24 movers / 20000 nodes -> 0.71 24 movers / 40000 nodes -> 0.55 DE's wall-clock scales ~0.50ms/element against parallel screenshot's ~0.22ms — drawElement repaints the whole tree per frame while fan-out amortizes it — so the downside is NOT bounded and a bare floor drop would have handed a 1.8x regression to large comps. Since motion only ever helps DE, an element ceiling calibrated at the lowest-motion case is safe at every motion level; crossover there is ~3.9k, and the default sits at 2500. The predicate is untouched; the call site picks the floor. Above the ceiling, or at 900+ frames, behaviour is bit-identical to today — the change can only add inversions in the new band, never remove one. Instrumentation, since this ships at full exposure rather than cohorted: `composition_element_count` on EVERY render (the fleet distribution of the gate variable is unknown — without it we cannot tell whether 2500 opens the band for most short comps or almost none, nor re-derive the threshold from real content), and `de_short_band` = applied | skipped_elements, unset when the frame count made the band irrelevant, so a fleet perf shift is attributable to this change rather than to content mix. Safety is unchanged and already proven on this path: per-frame PSNR self-verify with screenshot fallback, exactly as the 900+ band has shipped default-on. Knobs: HF_DE_SHORT_MIN_FRAMES, HF_DE_SHORT_MAX_ELEMENTS (0 disables the band). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-29 02:36:40 -07:00
Vance IngallsandClaude Opus 5	dfe92b2aab	feat(cli): roll circuit-breaker state over across config wipes ... The DE parallel-router breaker's tripped state lived in the same config file as the install id, so the most common identity reset — deleting ~/.hyperframes — also re-enrolled the machine into an experimental path that had already failed on it. Mirror exactly two facts into a machine-local state file (~/.local/state/hyperframes/install-state.json) that a config wipe does not touch: - markerAt: written unconditionally on every install, so the fraction of fresh mints that find it directly measures recoverable id churn (config wiped, machine persisted) vs unrecoverable (fresh machine/container/new user). Emitted as install_predecessor_found on telemetry events; absent (not false) on configs predating the field. - deParallelRouterTrialFired: a breaker tripped by a previous install stays tripped for the new one. Config corruption takes the same mint path, so it survives that too. The file deliberately holds NO identity — no anonymousId, no counters. A wiped config still gets a fresh id unconditionally; only the safety fact about the machine survives. Sync happens inside writeConfig so no breaker write site can forget it; failures are swallowed (telemetry must never break the CLI) but leave the memo unset so a later write retries. `hyperframes telemetry` lists the state path for transparency. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-29 00:46:41 -07:00
Miguel Ángel	880021411d	fix(cli): make telemetry opt-out durable (#2852) ... * fix(cli): make telemetry opt-out durable * fix(cli): make telemetry status trustworthy	2026-07-28 20:35:54 +02:00
Vance Ingalls	011f46bc18	Merge remote-tracking branch 'origin/main' into 07-27-feat_producer_lower_parallel-de_router_floor_to_700_frames_power-state_telemetry ... # Conflicts: # packages/cli/src/telemetry/events.ts	2026-07-28 05:01:24 -07:00
Vance IngallsandClaude Opus 5	cddc90ae37	fix(cli): add required gpu prop to power-state test calls ... trackRenderComplete requires `gpu: boolean`; the two new opt-out test calls omitted it, failing Typecheck in CI. The fix already existed on the stacked branch, so only this base branch was broken. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 01:29:58 -07:00
Vance IngallsandClaude Opus 5	b99c803898	fix(producer): guard pmset behind shouldTrack + don't pin workers without streaming (review) ... Two review findings on the floor/telemetry PR: 1. powerStateFields() is spread into the properties object at the CALL SITE, so it ran before trackEvent's own `if (!shouldTrack()) return` guard — telemetry-disabled installs paid two blocking `pmset` subprocess spawns per render for an event that was then discarded. Now short-circuits on shouldTrack() (memoized, so no cost on the tracked path). Regression test asserts pmset is not sampled when telemetry is off; fault-injection verified it fails without the guard. 2. The DE parallel router pinned workerCount to 3 and skipped calibration even when verified parallel DE STREAMING — the entire reason for the pin — could not run for that render. The common case is a composition over streamingEncodeMaxDurationSeconds (240 s default): the duration cap disables streaming before the router's force flag is consulted, so the render got a hard-coded 3 workers chosen by a benchmark for a path it was not on, instead of the calibrated count. shouldPreferParallelDrawElement now takes parallelStreamingAvailable and withholds the bet without it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 00:59:22 -07:00
Vance IngallsandClaude Opus 5	4520cd240b	fix(engine): bucket gpu_renderer + cover the failure cohort (review) ... Three review findings on the win32 drawElement PR: 1. gpu_renderer shipped the raw UNMASKED_RENDERER_WEBGL string — unbounded, driver-authored, GPU-model-specific, and |-joined across parallel sessions, i.e. high cardinality by construction, against this file's own convention of sanitizing engine-sourced strings (deGateReason is a bucket; error messages go through redactTelemetryString). Now bucketed at the source by classifyGpuRenderer to <backend>/<vendor> (metal/apple, d3d11/nvidia, swiftshader/other, ...), which is the whole analytic signal the win32 rollout needs and nothing else. The raw string never leaves the engine. 2. gpu_renderer reached render_complete only, so a crashed render — the cohort the field exists to attribute — carried no backend. It now rides RenderCaptureObservability (deGpuRenderer, sourced from the live probe session like the de_* counters), so both render_complete and render_error carry it and a hard failure still reports its GPU backend. On render_complete the perfSummary value still wins by spread order. 3. Restore the fallow-ignore-next-line suppression above __resetDeParallelRouterTrialStateForTests: CLI test files are not fallow entry points, so removing it fails the CI dead-code audit (local pre-commit passed only because of its changed-file scope). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 00:55:10 -07:00
Vance IngallsandClaude Opus 5	cb30157ebb	feat(engine): open drawElement fast capture to Windows hardware GPU ... Widen the default-on drawElement clamp from darwin-only to darwin|win32 (still requiring a non-software-GPU browser). The darwin restriction was a validation envelope, not an architectural limit — the CanvasDrawElement Chrome flag ships on every platform, and every safety layer that made the macOS default-on release (v0.7.38) survivable is platform-neutral: compile-time gates, the SwiftShader init gate, per-render worker-encode self-verification with screenshot fallback, and the blank guard. Worst case on an unvalidated D3D11 backend is the same as on Metal: verify catches a bad frame and the render re-runs on the screenshot baseline. Why now: 30-day telemetry shows ~206k non-CI hardware-GPU Windows renders (~78% of the win32 fleet, 18k installs) held on the slow screenshot path by the clamp — the second-largest perf population after macOS, carrying ~1,550 capture-hours/month in the DE-eligible >=700-frame band alone at a measured ~2x speedup opportunity. Instrumentation for the new cohort: drawElement session init now records the raw WebGL UNMASKED_RENDERER_WEBGL string (detectSwiftShader generalized to detectGpuBackend — same single evaluate, the string was previously read and discarded) and threads it session -> CapturePerfSummary -> RenderPerfSummary -> render_complete as `gpu_renderer`. drawElement damage proved compositor-backend-specific throughout the macOS rollout, so D3D11-cohort failures must cluster by ANGLE backend + GPU vendor (NVIDIA/AMD/Intel), not just `os`. The two DE clamp branches are extracted into a pure, unit-tested `resolveDefaultDrawElement` (platform + GPU mode + worker-encode + explicit opt-in), which also drops resolveConfig's cyclomatic complexity. The win32 streaming-encode compound tests collapse onto one shared helper. Linux stays excluded: that fleet is headless/Docker SwiftShader, where DE has no speedup and known rendering defects. Kill switches unchanged: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false, --experimental-fast-capture=false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 00:00:21 -07:00
Vance IngallsandClaude Opus 5	3da31e399b	feat(producer): lower parallel-DE router floor to 700 frames + power-state telemetry ... HF_DE_PARALLEL_MIN_FRAMES default 2000 -> 700, re-calibrated by a controlled crossover sweep (fixed content-per-frame, three synthetic profiles x {350..3000f} x {single,par2,par3} x 3 reps, resolved worker counts and capture modes verified per run): par3 beats single at EVERY size in every profile — +17-21% at 700f rising to +28-34% at 3000f. That includes a 24-sub-composition profile built specifically to reproduce the 'workers re-pay init' failure the original 2000 floor guarded against (92k tweens, ~2.5s pollSubCompositionTimelines per worker): workers initialize concurrently, so duplicated init costs CPU, not wall-clock, and the comp still parallelizes +19% at 700f. Below ~700f the win thins toward +10% while paying three hardware-GPU browsers, so a floor remains. par2 loses to par3 in every cell of every profile — the router's existing 3-worker pin is confirmed, not changed. Harness: plans/drawelement-fast-capture/de-crossover-bench.sh (docs repo). Also adds on_battery / low_power_mode to render_complete and render_error. The DE fleet is macOS laptops, and bench sweeps on an M4 Pro caught the SAME render flipping between ~9.6 and ~17.2 ms/frame power-management regimes with no existing telemetry signal to segment by — the router soak reading this change needs that dimension to interpret perf on the machines users actually render on. Sampled per event (volatile), pmset-based, darwin-only, null-safe on failure. Router stays default-off behind HF_DE_PARALLEL_ROUTER; this tunes what it will do when the soak clears it to flip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-27 22:51:28 -07:00
James Russo	49cfc1c80c	refactor(cli): extract telemetry delivery into transport.ts (#2344) ... ## What Extracts the reliability-critical telemetry **delivery layer** — the in-memory event queue, async `flush()`, and the exit-time detached-child `flushSync()` — out of `packages/cli/src/telemetry/client.ts` into a new `transport.ts`. `client.ts` stays the CLI-facing **policy** layer: - `shouldTrack()` opt-out checks (dev mode, `DO_NOT_TRACK`, `HYPERFRAMES_NO_TELEMETRY`, config) - `trackEvent()` system-metadata enrichment - `showTelemetryNotice()` first-run disclosure …and re-exports `flush` / `flushSync`, so `events.ts`, `index.ts`, and the `cli.ts` exit handlers keep importing from `./client.js` **unchanged**. ## Why This is the code path that had the process-exit data-loss bug fixed in #2105 — render telemetry was ~6× undercounted and geographically US-skewed because the old drain-first flush emptied the queue before delivery confirmed, and the render command's `process.exit()` teardown killed the in-flight request. Isolating the delivery mechanism into its own focused, dependency-light module (only `./config` + node builtins) keeps that subtle, reliability-critical path in one place and reduces `client.ts` to just policy. Follow-up to the render-telemetry-gap investigation. A delivery-health canary was also added to the [CLI Observability dashboard](https://us.posthog.com/project/356858/dashboard/1634055) — `render_complete ÷ successful render commands`, which should sit ~1.0 and would surface any regression of this class immediately. ## How Pure code motion — **no behavior change, public API identical**. `transport.ts` owns the queue and stamps each event's dedup `uuid` + ISO timestamp in a new `enqueue()`; `trackEvent()` enriches with system metadata then calls `enqueue()`. `buildPayload`/`flush`/`flushSync` bodies are moved verbatim. ## Test plan - [x] `vitest run src/telemetry/client.test.ts src/telemetry/events.test.ts` → **38/38 pass** (client.test.ts still validates queue-retention, uuid idempotency, and the detached-child flushSync path through the public API — unchanged) - [x] `oxlint` clean, `oxfmt --check` clean - [x] `tsc --noEmit` — no new type errors in `telemetry/` - [x] `bun run build` succeeds 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-07-27 18:32:49 -07:00
Miguel Angel Simon Sierra	597c14a887	fix(telemetry): send feedback as plain events, not PostHog surveys ... CLI and Studio feedback were emitted as `survey sent` with `$survey_*` properties, so every rating was ingested as a PostHog survey response even though no survey definition, targeting, or popover backs them. Emit `cli_render_feedback` and `studio_feedback` with plain `rating` / `comment` properties instead. Same fields, same call sites, same opt-out.	2026-07-27 16:46:51 +02:00
Vance Ingalls	c6462a0a22	fix(producer): emit heap advisory at orchestrator, lock message + telemetry props with tests	2026-07-21 20:39:06 -07:00
Vance Ingalls	12e599a6ba	fix(engine): realistic worker memory budget + sizing/feedback telemetry	2026-07-21 14:31:14 -07:00
James Russo	1b4cf12cd3	refactor(cli): add stderr diagnostics logger + oxlint guard (#2551)	2026-07-16 09:35:27 -04:00
James Russo	d92d1d4f51	fix(cli): keep diagnostics on stderr for --json commands (#2522)	2026-07-16 09:14:39 -04:00
Vance Ingalls	af923d947c	fix(producer,cli): round the fallback dB consistently, thread the verify threshold through ... Review feedback on #2411 (Rames): the crash-survival RenderCaptureObservability mirror passed deFallbackFailedDb raw/unrounded while the render_complete perfSummary path rounded to 1 decimal — the same underlying PSNR could ship two different values to PostHog depending on which event fired. Extracted the existing inline round/clamp expression (previously duplicated for verifyMinDb and fallbackFailedDb) into a shared roundDb helper, applied once at the single point deFallbackFailedDb is derived from the thrown error so both downstream consumers agree. Also threads verifyThresholdDb (captured on the error but never propagated, per the nit) through DrawElementPerfInput/RenderCaptureObservability/render.ts/ telemetry as de_fallback_threshold_db on both events — the HF_DE_VERIFY_MIN_DB value the failing dB breached, letting ops read "28.4dB failed a 32dB threshold" directly instead of cross-referencing config.	2026-07-14 23:29:49 -07:00
Vance Ingalls	a5cbb78ff9	feat(engine,producer,cli): capture failing dB/frame index on drawElement verify fallback ... de_fallback_reason only told you the fallback happened (blank/psnr/oom/ capture_error), not the failing PSNR or frame index — that data existed as text inside the thrown error's message and was discarded on the way to telemetry. DrawElementVerificationError now carries structured frameIndex/failedDb/verifyThresholdDb; the orchestrator reads them via the new getDrawElementVerificationDetails helper instead of regexing message text, and both telemetry surfaces (the render_complete perfSummary path and the crash-survival RenderCaptureObservability mirror) emit de_fallback_failed_db / de_fallback_frame_index. Needed to distinguish "32dB vs the 32dB threshold, tune it" from "12dB real corruption, investigate" during the parallel-router soak — currently that distinction is invisible.	2026-07-14 23:29:49 -07:00
Vance Ingalls	9bcd279b29	feat(cli): surface stale project pin to non-TTY agents, throttled	2026-07-14 15:28:52 -07:00
Miguel Ángel	990f5c3145	feat(feedback): adopt 0–10 recommendation scale (#2438) ... * feat(feedback): adopt 10-point recommendation scale * docs(feedback): keep OSS scale contract self-contained	2026-07-14 15:46:47 -04:00
Vance Ingalls	4ca1552e20	fix(core,cli): attribute figma REST failures to the endpoint that failed ... cli_error had no way to tell which figma REST call (images, files_nodes, variables_local, styles, ...) actually hit RATE_LIMITED/FORBIDDEN/etc, so the dashboard could see failures spike but not which call caused them. FigmaClientError now carries a low-cardinality endpoint label (never the raw fileKey/nodeId), threaded through to cli_error's endpoint property.	2026-07-13 21:41:55 +00:00
kiritowoo	ccf5f20b3b	fix(cli): stop losing render telemetry to the exit race (#2105)	2026-07-12 21:05:20 +08:00