Contributors in the wild have no access to the project's analytics backend,
so the contributing doc must not point them at it: drop the internal
dashboard link, the backend-specific SQL blocks, and the vendor naming.
The calibration check definitions stay (public, fixed terms of the
experiment); the queries live with the dashboard tiles that run them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Telemetry carries the assignment but not the decision reason, so a manual
HF_CANARY_* toggle mid-window reads as a cohort flip. Rule it out before
treating a small-nonzero stability read as a mechanism bug; a reason
property is deliberately deferred until the check actually comes back dirty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
As written, check 4's condition (dual-surface AND value flipped anywhere)
was a strict subset of check 3's (value flipped) — if check 3 read zero,
check 4 was vacuously zero and added no independent signal. Compare the
value each surface actually reported instead, and state plainly that any
disagreement is also a check-3 flip: this check's job is attributing such
a flip to binding divergence. Matching fix applied to the live PostHog
tile (insight jQi7QdW1, dashboard 1918875).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four pre-registered checks now exist as PostHog tiles. Without the link
the doc describes queries someone has to re-type; with it the doc and the
dashboard are one artifact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Registers two canaries that gate nothing — `calibration-10` (10%) and
`calibration-50` (50%) — so the rollout mechanism can be proven against real
traffic before any real feature depends on it. Zero behavioural risk: they are
read by nothing.
They answer what the unit tests structurally cannot. The tests bucket
generated UUIDs and weight every install equally; real render volume is
heavily skewed toward a few heavy installs, and real install ids churn (~25x
more distinct ids over 30 days than in any single day on the desktop render
population).
Four checks, pre-registered in the docs so the read is not post-hoc:
1. ACCURACY — does 10% land at 10%, install-weighted AND event-weighted?
2. DRIFT — how fast does CUMULATIVE exposure climb above target as ids churn?
The instantaneous share is flat by construction; the set of installs
enrolled at some point is not.
3. STABILITY — does any install ever change cohort? Must be zero. Percentages
are held FIXED for the window precisely so a flip is unambiguously a bug;
during a real ramp a false->true flip would be correct instead.
4. CROSS-SURFACE — do the CLI and Studio bindings agree for the same install?
A CLI-launched Studio adopts the CLI id, and 16,961 installs currently
share an id across both surfaces, so this is measurable.
Plus an independence check: overlap between the two calibration canaries
should be ~p1*p2 (~5%), not ~min(p1,p2) (~10%, which would mean every canary
lands on the same unlucky cohort).
The docs also record what calibration CANNOT fix: per-install cohorts never
flip, but a person who wipes their config gets a new id and a fresh roll.
Preventing that needs stable identity across resets, and both candidates were
rejected — hardware fingerprinting correlates the cohort with hardware (fatal
for a rendering experiment, and it survives uninstall) and account identity
covers only ~3.6% of local rendering installs. The drift is therefore a
measured, accepted limit, and the point of calibrating is to size it and pick
canary window lengths accordingly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the single `canaries: "a,b"` telemetry property with PostHog's own
flag shape, one property per registered canary:
$feature/canary-de-parallel-router: "true" | "false"
PostHog treats `$feature/<key>` as a first-class flag property, so breakdowns,
funnels split by cohort and the experiment surfaces work on a canary with
nothing configured server-side. The decision still happens locally: the render
path forbids render-time network calls, behaviour must not depend on analytics
being reachable, and neither the CLI nor Studio ships posthog-js (both
hand-roll a batch POST, so there is no SDK to evaluate a real flag with).
Decide locally, analyse natively.
Two decisions worth recording:
- BOTH ARMS ARE EMITTED. A non-enrolled install reports "false" rather than
omitting the property. Absent means "this build predates the canary", which
is a different fact from "this install is control" — collapsing them makes a
ramp unreadable, because you cannot separate a control group from an old
version.
- KEYS ARE NAMESPACED with a `canary-` infix. A real PostHog flag namespace
already exists in this project, owned by the web app (`enable-chat-tab`, set
by posthog-js from `$lib=web` events). Namespacing guarantees a canary key
can never alias a real flag key and have the two fight over one property.
Values are the strings "true"/"false" to match how PostHog records boolean
flag values, so the property is directly comparable to a real flag.
98 core / 1437, 166 cli / 2194, 269 studio / 2982 green; tsc clean across all
three packages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up to the canary primitive.
Telemetry: every event now carries a `canaries` property listing the cohorts
the install is enrolled in, attached in `trackEvent` so it lands on ALL
events rather than renders only — a staged rollout is only as useful as the
ability to split any metric by cohort. Resolved after the shouldTrack guard,
so opted-out installs never pay for it, and omitted entirely (not null or "")
when the install is in no canary, since PostHog treats those as real values.
Test hardening, after validating the shipped code against 60k synthetic and
101 real fleet install ids:
- Pin FNV-1a against canonical vectors, AND assert the shipped canaryBucket
actually uses that hash. Without the second assertion the first is
tautological — it would only prove the test's own copy is correct while
canary.ts drifted to a different hash, silently reshuffling every live
cohort. Fault-injection confirms only this assertion catches a hash change;
the distribution tests stay green because a perturbed hash is still
well-distributed.
- Tighten the share test from a 0.6x-1.4x band to +/-1 percentage point.
Measured error was 0.16pp at n=60k, so the old band would have passed a
badly skewed hash.
- Add chi-square uniformity across all 100 buckets (chi2 89.0 vs 148.2
critical at p=0.001). A lumpy hash yields roughly the right total share
while overloading some buckets, so the share test alone cannot catch it.
- Assert N concurrent canaries enrol binomially rather than in lockstep:
8 canaries at 10% put ~43% of installs in none and zero in all eight,
matching binomial(8, 0.1). Correlated slices would put ~10% in all eight.
Also verified 88,443 of 88,448 fleet install ids are well-formed UUIDs; the
5 that are not fail closed, which is the intended direction.
Docs: docs/contributing/canary-rollouts.mdx, registered in docs.json (an
unregistered page is invisible in the nav).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Problem: @hyperframes/studio package docs showed bun run dev without the
localhost URL contributors need after #2901 fixed contributing.mdx only.
- Fix: note that the studio dev server listens on localhost:5190 per
packages/studio/vite.config.ts server.port.
- Verification: preflight_ship.py + read vite.config.ts; bun run format:check.
- Problem: setup guide pointed contributors to localhost:3000 after bun run dev.
- Fix: use localhost:5190 to match packages/studio/vite.config.ts server.port.
- Verification: preflight_ship.py + read vite.config.ts server.port (5190).
* docs(prompting): correct workflow one-liners against skill contracts
general-video leads with its positive identity and companion mode;
faceless-explainer keys on invented visuals instead of TTS;
talking-head-recut uses the 'graphic overlays' trigger term;
motion-graphics gains its input side and overlay output;
music-to-video stops implying images are required.
* docs(prompting): make vocabulary video grids readable
Replace the 4-5 column table hack with a 3-column CSS grid,
switch demo clips to autoplay muted loops (no black poster frame,
no player chrome over tiny videos), and align cells at 16:9.
* docs(prompting): document the opening interview and run-shape questions
The guide taught prompt shapes but never prepared readers for the
conversation that follows: the intent interview, the two run-shape
questions (storyboard, automation vs companion), the just-build-it
skip, and BRIEF.md as the resumable artifact. Add that section to the
overview, a disambiguation note on the storyboards page, and free up
'companion' as a reserved term in media-and-audio.
* docs(guides): make BRIEF.md the pipeline's Step 3 artifact
Step 3 (Strategy & Messaging) listed no output while describing
exactly what BRIEF.md now captures. Name the artifact in the step
table, project tree, step body, gate, and iterating list, and fix
SCRIPT.md's step label in the tree (Step 4, not 3).
* docs(quickstart): realign the setup surface with the skills catalog
The quickstart drifted from docs/guides/skills.mdx, CLAUDE.md, and the
prompting overview — it had never been updated when those surfaces were:
- `--full-depth` on both install commands, with the reason inline. Without
it `skills add` fetches the skills.sh registry blob, which lags `main` by
hours, so a reader following the quickstart installs stale skills.
- `check` in the `/hyperframes-cli` row, and a validate step in the manual
dev loop, which went preview → render with no gate at all. The prompting
overview calls `check` "the step people skip and regret" and states both
`lint` and `check` must pass before rendering.
- `/hyperframes-keyframes` in the core-skills table (8 rows → 9).
- `/figma` in the optional-workflow list (10 → 11).
* docs(skills): close the catalog drift class and complete the music-to-video input
Follow-up on the two review nits from #2872.
`/music-to-video`'s SKILL.md names three inputs — an audio file, a video to
pull audio from, or a track generated from a mood brief. Every compressed copy
of that description carried only the first two, and the third is the one that
makes "a complete video needs zero assets" true. Fixed on all eight surfaces
that state it, so no surface is now more correct than its siblings: the
prompting overview and quickstart setup tables, docs/guides/skills.mdx, the
README catalog, root CLAUDE.md + AGENTS.md, both CLI project templates, and the
router's own routes/music-to-video.md Input line (whose Interview must-haves
already listed all three).
The drift was structural, not accidental: the sync set declared in
docs/guides/skills.mdx and in CLAUDE.md's "Skill catalog maintenance" named
four surfaces and never the two setup tables, so those two were free to rot
while the declared four stayed correct. Both declarations now name them, and
both say the set applies to a *changed contract* — a reworded description —
not only to an added or renamed skill.
skills-manifest.json regenerated for the touched route file.
* docs(claude): point the routing-surface rule at routes/, not the moved stubs
Item 3 of "Skill catalog maintenance" still sent readers to
`references/workflow-catalog.md` for a workflow's input/output/trigger
contract and `references/route-briefs.md` for its interview entry. Both are
now "moved" stubs — the contract and the interview entry live together in
`references/routes/<workflow>.md`, one read per candidate route.
Same failure class the previous commit fixed at item 1: a maintenance rule
outliving the layout it describes. Swept the tree for other pointers at the
two stubs; there are none, so this closes it rather than fixing one instance.
The old demo argued that a locked frame crops the story. The sharper claim is
what a cut throws away: a move carries the spatial relationship between two
shots, a cross-fade carries none.
Five stages each get their own colour and the halves diverge on how the next
colour arrives -- in place, or by travelling to it. 30-47% of pixels differ,
against 3% for every version where the WITHOUT half moved the content.
That 3% is the trap now documented in the rule: translating the scene and
translating the camera are the same transform, so sliding content past a fixed
frame is not a static comparison, it is the same shot rendered twice.
Embed is cache-busted to -v2; docs images carry immutable year-long cache, so
reusing the filename would serve returning visitors the old render.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Live telemetry panel, an isometric deploy pipeline wider than the frame, and
a progressive edge rollout across twelve regions. In each, the rule follows
from the content rather than being applied to it.
Rule 2's demo carries the strongest evidence in the chapter: deleting the
three stages the locked frame never shows and re-rendering leaves that half
identical on all 180 frames.
Rule 3 gains a rule-level distinction the rebuild surfaced — stagger what is
happening, not what is merely present.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the generic card/chip/disc subjects with ones where the rule is
unavoidable given the content: a payment sheet over a receipt, a split-flap
departure board, and a render-farm corridor.
Rules 5 and 6 in the grammar list were one-liners that under-said what their
demos prove. Rule 5 now carries the mass claim, the follow-through lag, and
the transforms-only caveat (a counter that overshoots renders a false value).
Rule 6 credits occlusion over blur and prefers one foreground element.
Section intro no longer endorses over-cranking; every movement in an applied
half has to finish "this moves because...".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(cli): persist authoring skill in hyperframes.json for durable render attribution
authoring_skill was stamped only on the first render through a workflow
passing --skill, so re-renders, `npm run render`, --batch, existing-project
renders, and general-video lost it — leaving 77-96% of real-human render
volume un-attributed and the skills-penetration metric misleadingly low.
Persist the owning skill in hyperframes.json: `init --skill` stamps it at
creation, `render` resolves the flag then falls back to the stored value, and
an explicit --skill seeds it (seed-once, never overwriting the creating
workflow's identity). Activate all render-producing creation workflows to
declare their skill at init.
Forward-only: does not rewrite historical telemetry.
* fix(cli): patch hyperframes.json in place when seeding the authoring skill
seedProjectAuthoringSkill is the only writer that touches an already existing
hyperframes.json — every other writeProjectConfig call site is guarded to write
only when the file is absent, which made the whole-file overwrite safe by
construction. Round-tripping the seed through normalizeConfig broke that: it
rebuilds the object from a field whitelist with no rest-spread, so any key
outside the schema was silently dropped, a media block was materialized in
projects that never had one, and key order was rewritten. hyperframes.json is
normally committed, so a render introduced a diff the user never asked for, and
any field added to the schema later would be deleted by a render on an older
CLI.
Parse the raw JSON, set authoringSkill, write it back, reusing the file's own
indentation. Unknown keys and formatting survive; the only delta is the key
being added. A corrupt config is now left untouched instead of clobbered.
Seed-once semantics are unchanged, still normalized so a hand-edited garbage
slug neither reaches telemetry nor wedges the seed.
Reported independently by both reviewers on #2762.
* fix(cli): create the docker build context with mkdtempSync
The `--docker` build context was created at a guessable path derived from
`Date.now()` in the world-writable OS temp dir. Another local user can
pre-create or symlink that path and have the build read a Dockerfile they
control. mkdtempSync gets a random suffix and 0o700 from the kernel, and it
creates the directory itself, so the separate mkdirSync goes away.
Pre-existing on main (alert #432, 2026-06-04, packages/cli/src/commands/render.ts),
surfaced against this branch only because the seed commit shifted line numbers in
the same file. Fixed here to unblock the CodeQL gate on #2762 rather than left for
a follow-up; the remaining 10 js/insecure-temporary-file alerts elsewhere in the
repo are untouched and still want their own pass.
* fix(cli): drop the check-then-use race when seeding the authoring skill
The seed tested for the config with existsSync and then wrote, which is a
check-then-use race: the file can be created or swapped between the check and
the write (CodeQL js/file-system-race).
Read once and branch on the failure reason instead. Only ENOENT creates a
config from scratch; any other read failure (permissions, I/O) now leaves an
existing file alone rather than overwriting it with a default, so this is also
strictly safer than the version it replaces.
Also replaces the `as Record<string, unknown>` assertion with an isJsonObject
type guard, per the repo's no-assertion convention.
Behaviour unchanged: all 4 seed regression tests still pass, and the
create/preserve/seed-once/corrupt-untouched paths were re-verified end to end.
