mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-15 13:58:20 +00:00
7474f38420652ea675305562917eaae49ccb477d
144
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 WaterrrForeverandClaude Opus 5.5
|
688229314f |
feat(skills)!: publish seven knowledge skills aligned with Studio and retire the workflow skills (#5651)
* refactor(skills): fold six domain skills into hyperframes and media-use guides
hyperframes-core, -animation, -keyframes, -creative and -studio move to
skills/hyperframes/{core,animation,keyframes,creative,studio}/, and
hyperframes-audio moves to skills/media-use/mix/. Each moved SKILL.md becomes
a frontmatter-free GUIDE.md so installers that scan recursively for SKILL.md
no longer see them as skills; the published list goes from 21 to 15.
Relative links, code paths and skill-name mentions are rewritten to the new
locations, the catalog surfaces, marketplace core-skills list, fallback core
set, module-copy targets and skills-manifest.json follow, and
`hyperframes skills update` now removes the retired skills (RETIRED_SKILLS)
when they are still installed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): slim the core entries and give media-use a HeyGen and a free path
The four entries every run opens are rewritten to what an agent cannot infer:
hyperframes 21.8 KB to 7.7 KB, hyperframes-cli 24.8 KB to 6.9 KB,
hyperframes-registry 9.9 KB to 4.3 KB, media-use 9.7 KB to 4.9 KB. Detail moved
whole into references (routing close calls and creator edits, CLI pin upgrades,
catalog search tiers, CLI conventions and feedback reports).
- hyperframes: state table, routing, the render contract, a terminal work loop
(check, snapshot, preview before render), the desktop-app hand-off, and a
table of the folded guides.
- media-use: one table of HeyGen versus free local tools per need; HeyGen
sign-in, credits and the CLI's video-agent, avatar, clone, lipsync, translate
and clipping commands; every free path works with no account.
- scripts/check-core-skills.mjs holds entries and descriptions to budget and
checks every taught `npx hyperframes` command and path; runs in lint.
- Core entries are exempt from the formatter: its table padding cost ~1.5 KB
per entry on every run.
- Content tests follow the text to its new home.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): plugin runs skip skills update; media-use explains HeyGen sign-in and TTS setup
The smoke eval showed a presenter request never reaching media-use and the
free voiceover failing on a missing kokoro-onnx; plugin runs also ran the
skills update the plugin rules forbid.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(skills): drop a committed Python bytecode file and ignore bytecode
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): align the entry with the Studio skills and keep the workflows' measured facts
The entry drops routing and the intent interview and carries the contract, the
editable-project rules and the work loop. Studio's format, runtimes, motion,
look and captions references come over for terminal use, frame presets move to
assets/, media-use gains Studio's mixing and local-command references, and
film.md and overlays.md keep what the workflow skills measured.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills)!: retire the workflow skills and folded guides; publish seven knowledge skills
The skill pack now publishes exactly seven skills - hyperframes,
hyperframes-cli, hyperframes-registry, media-use, figma, slideshow and
remotion-to-hyperframes - carrying knowledge, not process. The eight
workflow skills, the folded domain guides and the routing/interview
references are deleted; their measured facts already live in the
hyperframes references (format, runtimes, motion, look, captions, film,
overlays) and media-use's mixing reference.
- bake-basemap.mjs and locate.mjs move to skills/hyperframes/scripts/
(bake-basemap gains --help); extract-audio-data.py moves to
media-use/audio/scripts/ for the caption motion reference.
- skills update: the eight workflows join RETIRED_SKILLS so existing
installs prune them (by lock attribution only, since their names are
generic); naming a retired skill reports where its knowledge went and
succeeds instead of failing as unknown.
- Capture prompts point at the hyperframes skill's references/film.md;
the live-map warning names the new bake-basemap path.
- Content tests that pinned knowledge are retargeted to the kept
references; tests of deleted workflows, scripts and process go.
- Catalog surfaces, templates and docs describe the seven skills.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): resolve the local.md captions link and allow capture's output files in citations
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): regenerate the skills manifest from tracked files only
The committed media-use hash included an ignored __pycache__ bytecode file present
in the local checkout, so CI's clean checkout hashed differently.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(docs): restore generated catalog pages and docs navigation to main
Catalog PRs commit item sources only; CI regenerates the pages from the edited
registry sources. docs/docs.json is catalog navigation, so its Workflows group
rename is left for a non-catalog PR.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(cli): raise the CLI size budget for the skill references it bundles
The CLI bundles skills/hyperframes, which now carries the folded domain guides and
frame presets (155 KB to 1.08 MB in git). Budgets are the CI-measured size plus 5%:
packed 7526634, unpacked 22593213.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test: keep repo-sweep guards meaningful after the workflow skills left
The last shell ffprobe caller and most root index.html examples lived in retired
skills. The shell detector is now pinned on a literal command line, and the
structure-rule sweep needs more than 5 roots (11 remain) rather than 30.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): slim figma, slideshow and remotion to the Studio shape; restore the storyboard board
slideshow 33.5 KB to 4.9, figma 18.2 to 6.2, remotion-to-hyperframes 11.0 to
4.9, each keeping the facts only it holds (the figma motion helpers, verify gate,
storyboard decoding and usage beacons; present and hand-off; the Remotion lint,
SSIM check and API map). The board page format from the storyboard recipe comes
back as hyperframes/references/storyboard.md: the desktop app's Storyboard tab
reads boards/storyboard.html by its frame, title and time classes. The entry and
the init template now say to snapshot the moments an edit changed before
replying, which the terminal eval found skipped in 22 of 38 runs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): manifest without local bytecode; Studio's edit prompt names kept files
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): render when the person asks for the file or for no stops, not only after approval
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): keep the voiceover carve in media-use; presets stage their own fonts
Review follow-ups on #5651:
- carve.mjs and its tests move to skills/media-use/audio/scripts; the audio
docs, mixing.md and two Studio comments point there.
- code-editorial's FRAME.md says to copy its bundled WOFF2s and add the
@font-face blocks, instead of promising the retired build-frame.mjs.
- media-use states the host rule before the HeyGen sign-in steps.
- storyboard-format marks transition_in as a note the agent builds itself.
- Two stale citations (creator-editing-recipes.md, sub-compositions.md).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(skills): regenerate the manifest without local bytecode
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): the CLI render step follows step 4's rule, not approval alone
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): run the core-skills check from lint again after the main merge
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): films that tell a story start from three storylines; maps use real imagery
Opus 5.5 arm D read motion.md and applied it, but its launch film was a
feature list (pain, three feature cards, proof, logo): film.md left story
to the model. And the map case never opened film.md, so it drew Natural
Earth outlines instead of baking satellite tiles.
- references/story.md: the generic half of Studio's moment-film story and
craft (one thing, three storylines with world and device, the hook,
beats, worlds, copy, brand, tells of a generated film). The person picks
a storyline, or the agent takes its recommendation when asked for the
file or for no stops.
- The entry's work loop names the storyline step for films over ~15 s; the
references table routes stories to story.md and maps to film.md.
- film.md points at story.md and gives maps and real places a section.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): story.md follows the entry: no voiceover unless asked
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): bake-basemap starts from a globe and stops waiting out its idle timeout
The Opus 5.5 map run spent 13 of its 38 minutes on parallel bakes that
stalled and 15 more recording 300 frames, after rewriting the script because
it could only start on a flat map.
- PROJECTION=globe with a seeded starfield and atmosphere, and FROM/PAN to
turn the globe to CENTER before the dive.
- Each frame polls for loaded tiles instead of waiting for an "idle" event
that had often already fired: 30 frames went from 198-219 s to 10-13 s
locally with byte-identical frames.
- Progress prints seconds a frame and minutes left; film.md shows the globe
call and says to run one bake at a time.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): render in the foreground; wait on a detached render's log, never pgrep -f
Both Opus 5.5 explainer runs started a detached render, then waited with
`until ! pgrep -f "hyperframes render"`, which matches its own loop: each
render took 88-117 s and the wait sat for the tool's full 10 minutes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): route each kind of film; storylines carry look and sound; add review and sound
The entry routes by request (launch, explainer, music, data, vertical, 3D,
character) to a self-contained file under references/types/, and a film
over ~15 s now gets music and sounds on its visible actions unless the
person asked for silence. Each storyline names its look and its sound, so
picking one picks both; STORYBOARD.md frames carry `reads` and `sound`.
review.md is the whole-film pass: the check warnings, the review sheets,
seven questions to look for, P0/P1/P2 notes and REVIEW.md. motion.md gains
eases by role, six title entrances, a reading-time rule and nine measured
transitions; look.md a design plan before the first scene, with presets
kept for decks and named picks. story.md adds the second default that
generated design falls into, with what to do instead.
The blank template and format.md no longer teach the fade-and-slide-up
entrance on a dark canvas.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): check warns on unread lines, dead stretches, small type, empty frame 0 and silence
A coding agent cannot watch its film, so check now measures what a viewer
notices over time. A new `craft` section holds warnings only (exit 0
unless --strict), sampled on the dense 8 fps grid the overlap pass
already seeks, skipped for --range checks:
- line_unread: a reading-size line (>= 30 px tall at 1080) on screen
under 0.4 s for 1-2 words, else under 3/4 of 0.5 s + 0.3 s a word;
the film's last line needs 2 s. Short lines are re-measured between
samples to ~0.01 s before judging. Captions-track and
data-layout-ignore text are exempt.
- dead_stretch: more than 3 s where the keepsMoving liveness signature
does not change, outside the final 3 s and any data-hold element;
quiet when sweep_static fires or a whole-composition keepsMoving
assertion owns the limit.
- text_below_floor: text under 2.6% of the short side (28 px on 1080)
held 0.5 s, one finding per file listing every element.
- first_frame_empty: frame 0 paints only the background.
- film_silent: over 15 s with no unmuted audio or audible video.
Each message names the fix and the skill reference to read. keepsMoving
now also leaves the film's last 3 s (a still end card) out of its
static runs, so the two agree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): snapshot --review captures a beat sheet, seam strips and a phone sheet
snapshot --review picks the frames that show a film over time instead of
evenly spaced ones, in the same single browser session:
- review-beats.jpg: one settled frame per scene at its start + 0.3 s,
plus one per timed data-beat="name" element, labelled time and id.
Scenes are the composition's top-level timed clips and sub-composition
hosts, read like `timeline` reads rows (captions and audio excluded)
and timed by the runtime's start resolver over the live page.
- review-seam-NN.jpg: frames at -0.2, -0.1, 0, +0.1, +0.2 s around
every cut (each scene start, and any scene end no scene starts at).
- review-phone.jpg: the beat frames at 360 px wide.
--composition scopes the review to one sub-composition; with fewer than
two scenes the beats spread evenly and a note says so. snapshot also
gains --json: one envelope with every frame's path and time and, under
--review, the scenes, beats and seams behind the sheets; progress moves
to stderr.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): craft warnings stay out of --strict; a hold gets a real next action, not a caret
Craft findings (unread lines, dead stretches, small type, an empty first
frame, silence) are judgment calls a film may break on purpose, so
--strict keeps gating defects only. The check reference no longer
suggests a blinking caret to keep a still frame moving.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): review spreads beats when every scene starts together; map credits skip the type floor
A map film whose basemap and overlay both start at 0 s got one beat
frame; the even spread now keys on distinct scene starts. film.md tells
agents to mark the imagery credit data-layout-ignore and back it, since
the new type floor flagged it on the first real run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): review frames each scene at its middle too, not only 0.3 s in
On the first real explainer, 0.3 s into a scene showed a bare colour
field before its title entered. Each scene longer than about a second
and a half now adds a frame at its middle, so the sheet shows what the
scene builds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): a film is built from a numeric SPEC.md, with defaults banned by name and a look test
Studied in full: flxrnc's Motion Lab (vague vs directed prompt), 0xMovez's
course, EianLu's riso skill, agentchengfeng's style samples and the two
Opus 5.5 guides. Their common point: adjectives are where the default look
lives, so the agent now writes SPEC.md before any scene (GOAL, OUTPUT,
TIME, LOOK, MOTION, SHOTS, BANNED, QA), with a number for every adjective.
spec.md bans the defaults by name, unless the brand or reference uses
them (a dark void with small things floating, glow and particles, the
cream-serif-mono-label look, fade-and-slide-up, corner brackets), and
grows that list after the first render. Each storyline gets a 3 s test of
its hero shot before the pick; a look is written as the rules of one
medium. Composition numbers: the hero fills 40% of the frame, one hero
move per shot, the shot's information in within 2.5 s then a held poster.
motion.md adds springs by damping ratio, shorter exits, overlap, stagger
and the beat grid; review.md reviews as a harsh director and grows BANNED;
vertical.md takes the riso skill's measured safe band; explainer.md the
shapes of information.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): story before look; every object reads as itself; posters only for info cards
The first spec-driven C02 swapped the dark default for a flat Swiss
poster, and the story went with it: two black bars nobody read as mirrors,
slogans instead of cause and effect, a storyline picked for its picture.
The spec now serves the story: a storyline is picked for the story a
stranger would follow, the look test asks first whether a stranger
understands what they see, and SPEC.md gains an OBJECTS line giving each
thing the material and details that make it read without a label. The
2.5 s held-poster rule is limited to information cards; in a story film
every shot causes the next. Explainers start from a person or situation
the viewer knows. Review names every object as a stranger would and
treats one that needs explaining as a P0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): storylines are three films in words, each with a device and a form
The three storylines are pitched as text and the person picks from them;
nothing is built for a storyline before the pick, so the 3 s hero-shot
look test is gone. Ideas pair a device with a form, five or more first,
keeping the three that differ most, and the one stop states the rest in
an assumption line that names every paid item beside its free option.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: list main's new embedded-captions hf-cli test among the retired workflow files
The merge with main removed it with the rest of the retired skill, and the
deletion guard rejects any removal it has not been told about.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(core): keep the audio preset comment within core's size budget
The comment's new citation to media-use's mixing reference made the published
package 14 bytes larger than its budget, which sits at main's exact size.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): craft keeps its own 8 fps grid; comments back within the checks
Review on #5651 found that the merge moved craft onto the overlap pass's
times, which main caps at 180, so a brief line in a long film fell between
samples and line_unread missed it. Craft now builds its own grid at 8 fps up to
600 samples, as before the merge, and a 60 s film with a line shown for 0.2 s
pins it.
The Comments check passes again: REVIEW.md and STORYLINES.md join the files a
skill writes into a project, the comment lines this branch added come back
under each file's share at the merge-base, and the audio preset comment drops
a word the check reads as history.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
0e9f380c2e |
feat(cli): render telemetry reports how busy the machine's CPU was (#5646)
* feat(cli): render telemetry reports how busy the machine's CPU was render_complete and render_error now carry system-wide CPU busy share (average and busiest 2 s window), cores busy in that window, and the one-minute load average at start and end, for CLI and Studio renders. Nothing is sampled when telemetry is off. * test(cli): render tests use a private project folder instead of /tmp/project |
||
|
Miguel Ángel
|
96e2154d7c |
fix(engine): say why WSL renders fall back to software and how to get the GPU (#5560)
* fix(engine): say why WSL renders fall back to software and how to get the GPU Under WSL the GPU check often falls back to SwiftShader and the only output was the probe line, with Docker passthrough advice for --browser-gpu. Print one note that says what was checked (can this user open the render node), the exact fix, and the native Windows alternative. The CLI's WSL detector moves to the engine so both use one. Part of #5548 * fix(engine): keep WSL GPU notes true to the probe result and render mode The WSL note now prints only when auto mode really selects software, and says whether the GPU check found no GPU or could not finish. Explicit --browser-gpu keeps its unverified and Chrome-install advice and adds the WSL checks. The render-node check verifies group membership before suggesting usermod. |
||
|
Miguel Ángel
|
cf1b04bb71 |
feat(cli): accept host-managed HeyGen access tokens (#5177)
* fix(auth): accept host-managed HeyGen access tokens * fix(auth): share environment header validation * docs(auth): condense credential resolver guidance * fix(auth): report the account a browser login just signed in Identify the account from the tokens the login issued, not a fresh credential lookup that an environment token or API key outranks. * docs(auth): list the host access token in the CLI credential order * fix(cli): name a rejected host access token on publish Publish told a host-token user to log in again, which cannot help while the token outranks the login. * docs(auth): show the media credential order main resolves * fix(cli): warn at logout that a host access token still signs commands One map of environment credential variables, owned by the resolver, now feeds the logout warning and both publish messages. * docs(auth): scope the media credential order to the audio scripts * refactor(cli): read every env credential name from the resolver's map |
||
|
Miguel Ángel
|
80c2547eac |
feat(cli): send missing features and host-app friction as ratingless feedback (#5140)
* docs(skills): ask agents inside a host app to report its friction as feedback * feat(cli): send missing features and host-app friction as ratingless feedback * fix(cli): keep the rating error, refuse blank reports and document ratingless feedback * docs(skills): leave names and figures out of missing-feature reports * docs(skills): keep anything private out of missing-feature reports |
||
|
Miguel Ángel
|
9e0d05d450 |
fix(engine): auto worker count stays within the heap budget (#5149)
* fix(engine): automatic worker count never exceeds what the V8 heap can feed A 24 GB Mac exporting a 43 s portrait project with 22 videos picked 5 capture workers against a 4192 MB heap that feeds about 4, and the render died at 62% with JavaScript heap out of memory. The heap budget was only advisory; it now caps the automatic count (bound 'heap'). An explicit --workers N is untouched. * test(engine): heap-cap test restores its mocks even when an assertion fails Review findings on #5149: the os and v8 mocks were restored only if every assertion passed, and two comments still called the heap cap advisory. * docs(engine): heap budget comments say it caps auto sizing |
||
|
Miguel Ángel
|
9fbec22596 |
fix(cli): load telemetry without the core and engine barrels, and stop the history test racing (#5153)
* fix(cli): load telemetry without the core and engine barrels, and stop the history test racing * fix(cli): load history from its leaf subpath, and fail the history test fast on a crashed CLI |
||
|
Miguel Ángel
|
4178a8f95d |
fix(cli): reject extra positional arguments on every command (#4905)
* fix(cli): reject extra positional arguments on every command * fix(cli): read the command name without assuming a resolved meta * fix(cli): join free-text words for catalog and tts, reject a bad repo root * test(cli): load the listed commands through static imports * fix(cli): keep a tts file path out of the word join and read the parsed json flag * fix(cli): apply the file exception only to a tts .txt path * fix(cli): keep usageLine private and test the json fallback for undeclared json * fix(cli): keep extra arguments and the repo root value out of telemetry * test(cli): type the skills update lookup * fix(cli): read json mode from the parsed flag, reject tts words with --text-file, restore add's hint * fix(cli): check tts words with --text-file before --list and test --json=false |
||
|
Miguel Ángel
|
fa044a89e5 |
feat(cli): tag telemetry and feedback with the app that launched the CLI (#4632)
* feat(cli): tag telemetry and feedback with the app that launched the CLI * fix(cli): keep the client tag to a short slug and correct the telemetry docs * test(cli): read the delivered batch through the shared helper * refactor(cli): let the tag pattern alone bound the client tag's length * docs(cli): describe the client tag generically as the launching app's tag * style(cli): format the client tag test cases * test(cli): use a neutral example value for the client tag * chore(skills): regenerate the skills manifest for the telemetry reference |
||
|
Miguel Ángel
|
693c385508 |
fix(cli): commands exit right away even when the network or DNS is broken (#4604)
* fix(cli): commands exit without waiting on update checks or telemetry The update and skills checks run in a detached child that refreshes the caches for the next run; the parent only reads the caches. finalizeCli no longer awaits the telemetry flush: the exit handler flushSync() delivers what is queued. The events beacon, feedback and figma error paths stop awaiting flush() too. * fix(cli): retry a failed background check at most once an hour Before starting the background worker, the CLI stamps a per-check attempt time and skips a check it attempted within the hour, so a dead network no longer starts a worker on every command. The stale-pin notice writes onto a fresh read, detached spawns set windowsHide, and a test ties the worker path to its build entry. * fix(cli): a failed-check stamp dated in the future no longer silences update checks |
||
|
Miguel Ángel
|
627e9476af |
fix(cli): a failed command sends one error report, naming its real cause (#4586)
* fix(cli): a failed command sends one cli_error, naming its real cause * chore(cli): drop a comment the parameter name already says * fix(cli): a failure caught by its own command is still reported, and only once * test(cli): normalize-audio test passes the flags it requires * style(cli): format the failure tracking test * fix(cli): one function unwraps and reports a command failure, once * docs(cli): shorten the command failure reporter's comment |
||
|
Miguel Ángel
|
4984a268f7 |
fix(cli): Studio's catalog install records the item and honours paths, as add does (#4573)
* fix(cli): Studio's catalog install goes through add, so it records the item and honours paths * fix(cli): Studio's catalog lists the project's registry and install paths survive symlinks * refactor(cli): name the Studio catalog's registry option plainly * feat(cli): registry_item_added says whether add or Studio installed the item * fix(cli): dropping the same catalog block in Studio twice installs it again * chore(cli): drop a comment that restates add's source option * fix(cli): Studio's re-record finds lock entries by their resolved path * test(cli): the symlinked-folder install test checks the resize happened * fix(cli): Studio mounts the installed item's own file by the path add recorded * fix(cli): Studio's install never hands back a file the project kept for its own edit |
||
|
Miguel Ángel
|
950cccdac0 |
test: cli and studio-server tests remove the temp folders they create (#4566)
* test: cli and studio-server tests remove the temp folders they create * test(cli): each temp folder is removed where it is made, even when a test fails * test(cli): the one-liner helper relies on its project's own cleanup |
||
|
Miguel Ángel
|
16dc05e9fb |
fix(cli): stop background update and skills checks from holding a finished command open (#4556)
* fix(cli): skip background checks in unattended runs and never hold exit on a silent DNS * fix(cli): probe DNS only on background checks so foreground upgrade and skills commands still wait * fix(cli): keep update checks for agents and skip them only for CI, dev and the opt-out * refactor(cli): give the update-check opt-outs one owner shared with auto-install * docs(cli): stop the auto-install notes claiming runs without a terminal are skipped * fix(cli): run telemetry flushes one at a time so an overlapping flush never resends an event * fix(cli): let a telemetry flush run even when the one before it threw * fix(cli): cap exit wait at one telemetry attempt when an overlapping flush already failed |
||
|
Miguel Ángel
|
d290c05b09 |
fix(cli): --yes in an unattended run no longer overrides a no saved by another process (#4547)
* fix(cli): --yes in an unattended run no longer overrides a no saved by another process * fix(cli): every settings writer takes the lock, and --yes needs a real terminal to reverse a no * fix(cli): the thin-results download offer also needs a real terminal * fix(cli): one owner-checked settings lock, and an unreadable settings file never counts as a yes * fix(cli): settings lock removals go through a reaper, and a non-boolean answer counts as a no * fix(cli): a settings lock that cannot be released no longer fails a write that landed * fix(cli): a leftover settings lock is never taken over, and doctor names it * fix(cli): a background update that cannot be recorded is not launched again every run * fix(cli): every on-device install re-reads the saved answer first, and an unsaved yes installs nothing * fix(cli): the thin-results offer stops when a yes cannot be saved * fix(cli): only the consent-checked step installs the on-device runtime, and a leftover lock warns once * fix(cli): the leftover-lock warning follows the attended-terminal rule |
||
|
Miguel Ángel
|
2734ee2c02 |
feat(cli): hyperframes history lists, undoes and restores a project's recorded changes (#4449)
* feat(cli): hyperframes history lists, undoes and restores a project's recorded changes * fix(cli): an agent's history turn ends after 10 idle minutes on every path A turn left open no longer takes the person's later edits: the engine files a CLI turn's writes by the same idle rule as a preview window. Studio claims take their paths back from an agent's window; refusals, --json shapes, --limit and time parsing follow the review. * refactor(cli): the project-history opener lives in its own module The history command and other commands open a project's history the same way: through a running preview, or directly under the engine's lock. * docs(skills): the history loop lines sit in one marked trial block per skill * fix(cli): a turn marker without its last write time has ended Idle-limit tests set file times instead of racing the clock, and the restore refusal test picks a ref no entry id starts with. * docs(cli): the history entry shape comment names no other app * refactor(cli): history reuses the busy error class and its who helper * fix(cli): undo --who reverts the agent's whole last turn and nothing older When the open turn recorded nothing, undo fell back to the agent's earlier turn; when a mid-turn command split the turn, it reverted only the newest part. The turn now keeps the entries it filed, and undo --who reverts exactly those or refuses. Engine tests pin the write timing rules, and a date that rolls over (Feb 31) is refused. * fix(cli): an undo --who that meets a conflict leaves the whole turn as it was It redoes the parts it already undid, lists the newer entries as they are now, and offers --just-this and --back-to-before for the whole turn. The last turn is recorded atomically before its marker is cleared. * chore(cli): history keeps each file's comment share at or under main's * fix(cli): history keeps log order past a turn's idle limit and undoes a turn's cut parts * fix(cli): history drops unused imports and splits undo and reopen steps |
||
|
Vance IngallsandClaude Sonnet 5
|
b32b9a7dab |
feat(runtime): data-vfx-chain — a seek-deterministic per-pixel effect element (#4306)
* feat(core): data-vfx-chain defs and chain parsing (mirrors data-fx-chain)
Mirrors audioFx.ts's shape for the per-pixel VFX chain: a versioned
JSON chain attribute (data-vfx-chain), a def registry with typed
params (number/enum/bool/ref), and parse/serialize/normalize helpers.
HF_VFX holds one def, fractal-noise, with a placeholder frag; the
kernel and its consumers (packages/core/src/runtime/vfx.ts) land in
Task 1.2/1.3 of the vfx-chain-runtime plan, so vfx.ts is added to
.fallowrc.jsonc's entry list for now.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(runtime): paint data-vfx-chain canvases on every seek
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(vfx): fractal-noise kernel (Basic/fBm approximation) with CPU reference
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(vfx): self capture through layoutsubtree with the page-composite readiness protocol
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(vfx): wave-warp kernel (Sine)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(vfx): displacement-map kernel (self-referential map)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(vfx): chain order is nesting order; preceding CSS effects are captured
...and following ones apply outside. The plan's one-line message for Task 2.4
is 105 characters; commitlint caps the header at 100, so the tail moved here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(vfx): seek determinism and CPU-reference PSNR for fractal-noise
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(vfx): size the output canvas from the host's layout box, not its transformed AABB
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(telemetry): vfx node count, capture class and types on render_complete
Threads vfxNodeCount, vfxCapture and vfxTypes from a static scan of the
compiled composition's data-vfx-chain hosts (mirrors the existing
compositionElementCount/colorGradingCount scan) through
RenderPerfSummary.drawElement into the render_complete event, next to
de_capture_mode. Exports chainCapture/parseVfxChain/HfVfxCapture from
@hyperframes/core's index for the producer to consume. The PostHog
dashboard tile is not created here — left for the coordinator.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(vfx): data-vfx-chain guide — element shape, capture table, Studio flag, known constraints
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(vfx): register chain hosts that arrive with a sub-composition mount
`initVfx(document.body, fps)` ran once, synchronously, three lines after
`void loadExternalCompositions(...)` — so every `data-vfx-chain` host that
enters the DOM only when a `data-composition-src` mount resolves was never
registered. An unregistered host paints nothing and reports nothing: no
`[HyperFrames] composition script error`, the `.hf-vfx-out` canvas keeps its
300x150 default, and `paintVfx` never even arms the page-composite flag.
Measured on a real multi-file export served raw: `#main-l5-text`'s output
canvas stayed 300x150 and `window.__timelines` never gained the sub-comp.
Re-scan in the loader's `.finally`, before `maybePublishRenderReady()`, so
registration precedes readiness. `initVfx` now releases the outgoing
registry's programs, capture textures and ping-pong targets first — the
output canvas hands back the same WebGL2 context on a re-scan, so the
previous pass's GL objects would otherwise leak.
Two browser cases in `vfxDeterminism.test.ts`, both against the real runtime
bundle: a host whose `.hf-vfx-in` IS the mount (already worked — the mount
replaces the wrapper's children, not the wrapper, so the registered source
node survives), and a host that arrives INSIDE a mounted sub-composition
(failed with `armed: false` and no error; passes now).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(vfx): runtime tsconfig typecheck passes
Under noUncheckedIndexedAccess, resolveCaptureSource's null return
widened VfxEntry.src past its VfxCaptureSource | undefined type; the
function is internal-only, so its return type moves to
VfxCaptureSource | undefined instead. Indexing PingPong.framebuffers
by a non-literal index produced | undefined against
WebGLFramebuffer | null, coalesced to null. No behavior change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(vfx): structural DOM predicates instead of realm-bound instanceof
vfx.ts checked `instanceof HTMLCanvasElement`/`HTMLElement` directly, which
returns false for inner-realm nodes when the Studio previews across an
iframe realm. domRealm.ts had no canvas predicate yet, so this adds
isCanvasElement mirroring the existing isImageElement pattern, then swaps
all three vfx.ts checks to the structural predicates the lint guard requires.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(vfx): paint self-capture hosts under hyperframes snapshot, not only the engine path
`paintVfx(t, { engineMode: true })` armed the page-composite protocol and
returned, betting that every capturing chain runs under `frameCapture.ts`.
`hyperframes snapshot` (and check/compare/validate/layout, and Studio's
thumbnail capture) seek through the same `renderSeek`, never read
`__hf_page_composite_pending` and never call `__hf_page_composite_resolve` —
so a `self` chain painted nothing there, in silence, because its content
lives in a `layoutsubtree` canvas the browser never paints directly.
Engine mode now arms the preview-side capture as well and paints on whichever
completes first. A seek token stops a superseded async capture from repainting
a newer frame, and a registry-identity check stops it painting an entry
`initVfx` already released. `resolveVfxCapture` stays non-idempotent on
purpose: the engine resolves only after its own before-capture hooks have
mutated the DOM, so its capture is still the authoritative one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(vfx): an exporter-emitted .hf-vfx-out gets the box the spec fixes for it
findOrCreateOut adopted a canvas the exporter had already emitted without
applying the layout cssText, so it kept the default position:static;
display:inline and wrapped to the line AFTER the layer-wide .hf-vfx-src
canvas — painting a second, misplaced copy of the layer one full layer
height below the host. Measured on retro-wave #main-l6-text: .hf-vfx-src
rect top 315.1 h 93.0, .hf-vfx-out rect top 409.2, both style=null,
position static (ae-mcp findings Task 3.5b). Hosts with no emitted
out-canvas were never affected, which is why only this one host moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(telemetry): vfx_node_count counts hosts, not chain nodes — rename to vfx_host_count
The field increments once per `data-vfx-chain` attribute occurrence, never
per node inside a chain's JSON, so the name promised something the scan does
not measure. Renames the internal `vfxNodeCount` and the wire property
`vfx_node_count` to `vfxHostCount`/`vfx_host_count` across the orchestrator
scan, the perf summary, the CLI render path and the telemetry event. No
dashboard consumes the field yet, so the wire rename costs nothing now and
would cost a migration later.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(vfx): join the preview capture to the seek-completion barrier, not race it
`paintVfx` fired `capturePreviewThenPaint` and forgot it, so nothing the CLI
awaits waited for a `self` chain's paint. Every snapshot-family command
(`snapshot`, `check`, `compare`, `validate`, `layout`) seeks through
`seekCompositionTimeline`, which drains `window.__hfWaitForSeekCompletion`
and then allows one settle race before screenshotting — and only the
`"hf-seek"` event's `waitUntil()` could put anything in that barrier. The
screenshot therefore landed on however many hosts happened to have finished,
which varied with host count and machine speed.
Extracts the "track, untrack on settle, retain the first rejection" block
`dispatch()` built inline into an exported `registerSeekCompletion`, and has
`paintVfx` register the preview capture through it. Also awaits the per-host
paints with `Promise.all` instead of sequentially: N hosts cost about two
animation frames now, not 2N.
Measured on the pre-fix bundle with two `self` hosts, both output canvases
read `[0,0,0,0]` at the moment the barrier returned; they painted only
because the settle race that follows happened to be long enough. Both tests
added here fail on that bundle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(vfx): report WebGL context loss loudly instead of silently no-op painting
One WebGL2 context per `data-vfx-chain` host, against a browser cap that is
commonly 16 and shared with `colorGrading.ts`'s per-element contexts. A lost
context makes every later call on it do nothing, per spec — so the chain kept
"painting" a blank canvas with no error, which is exactly what the module's
loud-failure contract exists to prevent.
Adds the `webglcontextlost` listener pair's first half, following
`colorGrading.ts`'s precedent: `preventDefault()`, mark the entry, report
once through `reportVfxError`. The entry is then skipped by `paintVfx`,
`capturePreviewThenPaint` and `resolveVfxCapture` — silently, since the one
loud report already happened and repeating it per frame is spam.
Restore is left unhandled on purpose, with the reasoning in the doc comment:
`initVfx`'s existing re-scan already rebuilds every GL object, and a second
narrower rebuild path for `webglcontextrestored` would duplicate it. The
shared-context redesign the review floated is likewise out of scope here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* perf(vfx): cache uniform locations at pass-build time instead of querying every paint
`setPassUniforms` asked the driver for `u_size`, `u_t`, `u_fps` and every
param's location on every paint of every pass, and `bindPass` did the same
for `u_src` on every bind. `buildPasses` already constructs each `VfxPass`
once at init, right after the program links, and a linked program's
locations never change — so they are resolved there now and read from
`pass.locations` in the hot path.
Same values, same `null`-means-optimised-out semantics (`gl.uniform*(null,
…)` is a defined no-op, which is what the per-frame query produced too), so
the existing vfx.test.ts and the browser determinism suite cover it
unchanged — both still green, the latter against a rebuilt runtime bundle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(vfx): bound the paint wait per host so a stalled compositor can't hang the seek barrier
`paintVfx` registers `capturePreviewThenPaint` into the shared seek-completion
barrier (
|
||
|
Brun3y
|
105984cac4 | feat(cli): add IBM Bob runtime support (#3930) | ||
|
Miguel Ángel
|
aa05278b78 | feat(cli): render events carry the browser's build and install-path shape (#4180) | ||
|
Vance IngallsandClaude Opus 5
|
ddbc9a8a98 |
feat(cli): emit Chrome memory, capture path and segment fields on render events
Both events carry the live observability values, so a render_error that never built a perf summary still reports the last known Chrome memory. render_complete prefers the aggregate from RenderPerfSummary.chromeMemory, which covers every worker rather than whichever session sampled last. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
James Russo
|
2a559714ee | fix(cli): distinguish agent and execution harness telemetry (#4071) | ||
|
miga-heygenandMiguel Ángel
|
71bb88719e |
feat(telemetry): render output-shape, adapter, count, and environment properties (#4017)
* feat(telemetry): add output-shape request facts to render events output_resolution_preset, output_format, hdr_mode, video_frame_format, and gif_fps_capped join render_complete/render_error, all threaded from already-resolved RenderOptions/RenderPlan fields rather than recomputed: createRenderPlan already computes gifFpsCapped (clamping a requested --fps above 30 to 30 for --format gif) but only used it for a console warning; this threads that same decision into RenderOptions and telemetry. All five are known before the pipeline starts, so both events carry them even when a render fails before a perfSummary exists. Also wires output_format/output_resolution_preset into Studio's separate emitStudioRenderComplete/emitStudioRenderError path, since Studio's request genuinely varies those two per-request. hdr_mode, video_frame_format, and gif_fps_capped stay CLI-only: Studio exposes no HDR-mode, video-frame-format, or GIF output control to users at all, so there is no per-request value to report for those three. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * feat(telemetry): add adapters_used to render events adapters_used lists the runtime deterministic adapters (packages/core/src/runtime/adapters/) a composition exercises. Investigated the actual codebase rather than the ticket's guessed 7-adapter list and found 12 real registered adapters (gsap, css, waapi, animejs, lottie, three, typegpu, d3, leaflet, mapbox, maplibre, google-maps) — the map/data-source adapters share the exact same window.__hf<Name> registration mechanism and are equally real "adapters used" facts, so all 12 are included. Detection unions a live page.evaluate probe (piggybacked on the same probeSession resolveCompositionElementCount already uses) with a static regex fallback over the compiled HTML, since this is purely observational with no gating role (unlike compositionElementCount) — union strictly increases recall rather than needing one source of truth. Threaded through the same plumbing chain as the already-merged composition_element_tags/aroll_video_count/heygen_video_count. Studio's emitStudioRenderComplete/emitStudioRenderError get this field automatically, with no Studio-side code change, via the same shared capture-observability spread heygen_video_count already uses. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * feat(telemetry): add audio/image/sub-composition/color-grading counts audio_count, image_count, sub_composition_count, audio_group_count, color_grading_count, and has_lut join composition_element_tags in the same single scanElementTags pass — audio/image/audio-group counts read from the scan's uncapped tag map so a 50+-distinct-tag composition can't push a named count into the "other" bucket. sub_composition_count counts data-composition-src mounts directly rather than reusing collectSubCompositionSrcs, which dedupes by src and drops placeholder/remote mounts (a different, smaller fact than the mount count this field reports). has_lut decodes the HTML-entity-escaped form of data-color-grading before parsing, since linkedom's serializer re-emits it "-escaped on every compile-pipeline round-trip — without the decode this silently reported false for every LUT-graded composition. The LUT-presence check now mirrors normalizeLut's own definition of "has a LUT" exactly, closing a gap where a blank or empty lut value would have reported true. Threaded through the same plumbing chain as the already-merged per-tag/provenance/adapter fields; Studio's emitter picks these up automatically via the shared capture-observability spread. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * feat(telemetry): add browser_version_major and ffmpeg_version_major Both are already-run subprocess calls whose output was discarded: readToolVersion (checkFFmpeg's preflight) already runs `ffmpeg -version`, and chromeLaunchOutcome already runs `--version` as a Chrome launch sanity probe on every render. A shared extractMajorVersion helper pulls the first X.Y-shaped number from either banner, threaded onto EnvironmentCheckOutcome/ EnvironmentCheckResult, then RenderOptions right after runEnvironmentChecks resolves in the local render path only. Docker renders never populate these two RenderOptions fields (the containerized producer runs its own preflight the host CLI never sees), so both properties come back absent there rather than reporting the host's own toolchain versions for a render that didn't use them. Studio does not call runEnvironmentChecks or this ffmpeg/Chrome preflight at all before creating a render job, so its render_complete/ render_error events will not carry these two properties unless Studio's own startup path is separately instrumented — a genuine parity gap, not an oversight in this change. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * feat(telemetry): add authoring_skill_source and authoring_skill_invalid Both derive from resolution logic createRenderPlan already runs: authoring_skill_source names whichever step actually won attribution (an explicit --skill flag, or the skill persisted in hyperframes.json) using the same flagSkill/projectConfigSkill values already computed for authoringSkill itself, not a second lookup. authoring_skill_invalid surfaces the raw --skill value when it fails slug-shape validation (normalizeSkillSlug rejects malformed input, not unregistered skill names) — this value already exists as RenderPlan.invalidAuthoringSkill and already drives a CLI warning in present.ts; this change threads the same field into telemetry rather than adding a parallel one. Threaded through both RenderOptions builders in execute.ts (single and batch render) and all three telemetry call sites in render.ts, including the Docker path — skill attribution is a request-level fact independent of which backend executes the render. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * feat(telemetry): add root_body_mismatch and root_body_delta_px_bucket Measures the composition root's authored data-width/data-height against the scaffold's own html/body CSS size, straight from the compiled document string scanElementTags already scans — the source-level fact, not runtime DOM state. init.ts's applyCompositionSizing (the #4001 fix) only patches the live DOM after the browser loads this same HTML, so this measurement stays meaningful whether or not that fix ran for a given render. Reuses @hyperframes/parsers' canvasScaffoldPatterns regexes (the same ones packages/lint's root_dimensions_mismatch rule reads) for the CSS side. Root-tag detection here is a lighter first-data-composition-id-match heuristic, not lint's more thorough findRootTag (which also handles a leading decorative <svg> defs block) — that helper isn't part of @hyperframes/lint's public API, and expanding it is out of scope here. The delta is bucketed (0 / 1-10 / 11-50 / 51+ px) rather than reported raw, and both fields come back absent — not a false/"0" default — when the root or the CSS block can't be read at all. Threaded through the same plumbing as every other static-scan field this week, including the live-capture-observability fallback used on a crash that never reaches perfSummary. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * feat(telemetry): add hf_env_overrides (names only, capped at 20) Reports the sorted names of HF_*/HYPERFRAMES_* env vars present when createRenderPlan resolves — never their values, since several of these carry filesystem paths. Snapshotting at plan-resolution time (the very first line of the render command, before any pipeline code runs) matters: this render's own preflight later injects HYPERFRAMES_FFMPEG_PATH/HYPERFRAMES_FFPROBE_PATH into its own process.env, which would otherwise make those two always read back as "operator overrides" whether or not the operator ever set them. Reports an empty array, never an absent key, when nothing is set, matching every other capped/bucketed field added this week. Capped at 20 names, documented alongside the cap constant. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * fix(cli): stop clobbering studio render_complete fields, fix an env leak trackRenderComplete's explicit audio_count/image_count/root_body_mismatch/ adapters_used/etc. keys, sourced from perfSummary.drawElement and only ever populated on the CLI's own render.ts path, always won the object spread over the observability-capture fallback those same keys share with trackRenderError, even when the direct value was undefined. studioRenderTelemetry.ts's emitStudioRenderComplete never populates drawElement, only the capture fallback, so 24 fields silently came back missing from every studio-triggered render_complete event despite Studio having computed and sent the capture value. Each field now falls back to its captureXxx counterpart via a new directOrCapture() helper when the direct field is absent, with regression tests proving both the fallback and that the direct value still wins when both are present. Also excludes HF_SHADER_WORKER_ENTRY from hf_env_overrides: cli.ts sets this on every invocation to point the bundled shader worker pool at its own file, sharing the HF_ prefix by coincidence rather than because an operator set it. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * style: remove em and en dashes from added code comments No behavior change. Rewrites 22 comment lines this PR's own diff introduced, using a comma or a period where the dash separated two clauses. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * test(producer): add a temporary diagnostic for a CI-only rootBodyMismatch failure Not a real fix. The rootBodyMismatch test group fails deterministically in CI (confirmed on two runs of the same head) but will not reproduce locally by any method tried, including an isolated file run, the exact combined producer:test:unit command, a worker-count-constrained run, and a fresh clone with a clean install and build matching CI's own steps. This dumps the intermediate regex-match state for the simplest failing fixture to stderr so the next CI run's log says what actually differs. To be removed once that answer lands. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * test(producer): widen the rootBodyMismatch diagnostic to compare import paths Still not a real fix. The previous diagnostic ( |
||
|
miga-heygenandMiguel Ángel
|
3704863d96 |
feat(telemetry): per-tag, provenance, and failure-taxonomy properties on render events (#4004)
* feat(telemetry): add per-tag element and a-roll counts to render_complete
countElementTags's single regex scan (the ~83% of renders that get no
live probe session) is extended into scanElementTags, which returns a
per-tag breakdown and an a-roll video count alongside the same total,
all from one pass over the same script/style-stripped markup — the
total is the sum of the per-tag map by construction, so the two
properties can't drift apart. countElementTags itself is removed; it
had zero production callers after this change.
New render_complete properties: composition_element_tags ({tag: count},
raw lowercase tag names, capped at the top 50 by count with the rest
folded into an "other" key) and aroll_video_count (<video
data-aroll="true"> elements from the same scan). Both are static-scan-
only, mirroring the existing composition_element_count_source split —
the live probe-DOM path never runs this scan.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* feat(telemetry): count HeyGen-provenance video from the same tag scan
heygenVideoCount joins arollVideoCount/byTag as a third property
derived from scanElementTags's single pass: <video
data-media-source="heygen"> elements. media-use's resolve.md now
documents that the skill stamps this attribute on a mounted video only
when the ledger record's provider is "heygen.video" (the mount itself
is always the agent's own composition edit, never written by resolve
scripts) — the ledger stays the single source of truth for provenance,
the HTML attribute is a projection of it, and no other provider value
is invented beyond this one signal.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
* feat(telemetry): add a bucketable failure taxonomy to render_error
error_name (error.name, e.g. one of the ~20 typed producer error
classes) and failed_stage_code (job.currentStage/failedStage
normalized to a stable snake_case code via normalizeStageCode)
join the existing free-text error_message/failed_stage, so a fleet
query can bucket failures without parsing prose. Error-conditional by
nature — render_complete has no equivalent, since a successful render
has no error to name or stage to blame.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
|
||
|
Miguel Ángel
|
f059a6eb53 |
feat(cli): default BeginFrame, agent blank, and looks quality (#3927)
* feat(cli): render summary names the capture path, gpu mode and stage timings * fix(lint): drop gsap_exit_missing_hard_kill, its hazard does not reproduce on cold workers * fix(cli): render slow-path hint fires only when the gpu was auto-probed * test(cli): isolate agent hint suite from ambient agent env keys The suite runs inside agent sessions. Ambient env keys fill the 16-key hint cap and fail assertions that a plain shell would pass. * docs(music-to-video): drop leftover hard-kill mandate after lint rule removal Frame workers still required a tl.set hard-kill that the dropped lint rule no longer enforces. Align the self-check with montage.md. * refactor(cli): share hint-key pattern and collapse render summary args Export HINT_KEY_PATTERN for the test isolation strip. Read duration and frame count from the perf summary instead of passing them beside it. * test(cli): lock capture-mode preference and 16-key hint isolation Session mode, including drawelement, must print over observability. The hint suite now fails if ambient keys already fill the 16-key cap. * feat(cli): default BeginFrame, agent blank, and looks quality Local auto render opts out of the software-GPU screenshot clamp so BeginFrame can run. init --agent scaffolds a centered Inter blank with no prompts. --quality looks (now the default) is CRF 16; delivery is high. * fix(cli): honor screenshot env and teach agent pitfalls on the happy path Local auto still requests BeginFrame unless PRODUCER_FORCE_SCREENSHOT=true. The agent blank fills its canvas so --resolution can resize it. Skills now name the four pitfalls and tell agents to read the render summary line. * feat(cli): default init to the centered blank Bare init scaffolds the centered Inter stage. --agent is a hidden alias. * fix(cli): pack from-file and use it for video init TTY --video with no --example selects from-file. Pack copies that template. * fix(cli): wire --video onto from-file even with --example blank One resolver picks the scaffold. Spawned init --video asserts a-roll src. * fix(cli): narrow looks CRF on the quality alias Only looks carries crf. Typecheck failed on the union without a guard. * style(cli): shorten render-pipeline summary comment |
||
|
Miguel ÁngelandVance Ingalls
|
9308eadcfc |
fix(render): bound and recover capture stalls (#3700)
* test(producer): pin unpinned drawElement stall retry
* fix(engine): bound drawElement frames so a wedged renderer falls back instead of failing (#3172)
* fix(engine): bound drawElement frames so a wedged renderer falls back instead of failing
A single drawElement frame could kill an entire render. On one comp
(caption-editorial-emphasis) drawElementImage returns normally and the renderer
then stops draining its task queue: the setTimeout(…, 0) that schedules
toDataURL never fires, the capture page.evaluate never settles, and 60s later
the stage watchdog fails the whole render. Deterministic on 152.0.7977.30,
always the same frame. Root-cause detail in PRINFRA-488.
Two gaps, both closed here.
1. Nothing below the 60s stage watchdog bounded a frame. Adds a per-frame
deadline (HF_DE_FRAME_TIMEOUT_MS, default 15000, 0 disables) around the WHOLE
frame operation, not just the drawElementImage call — the stall surfaces at
whichever page round-trip comes next, and was observed at both the capture
and the seek's background-image decode. Bounding one call missed it.
Deliberately NO per-frame screenshot fallback for this error: once the
renderer stops scheduling it is wedged for every subsequent round-trip on
that page. Measured — the screenshot fallback blew the same deadline. The
frame fails fast instead, so the recovery that actually works can run.
2. That recovery was ineligible. shouldRetryViaPinnedFallback only retried a
generic capture failure when the worker count was PINNED by the inversion or
the router; a comp that engaged drawElement on the ordinary single-worker
path had no whole-render fallback at all. A renderer stall is now retryable
on any routing — the failure is a property of drawElement itself, and the
retry re-renders on a fresh page via screenshot. Reported as
de_fallback_reason "de_renderer_stall" so it is distinguishable from
capture_error in telemetry, and counted per-session as deFrameTimeouts.
Verified end to end on the repro: previously exit 1 with "stalled: no frame
progress for 60000ms"; now the deadline fires, the render retries via
screenshot, and RENDER_OK with a valid 1920x1080 / 240-frame / 8.0s MP4.
Tests cover the predicate both ways plus the cross-package error match. Engine
1,481 and producer 587 green (audioPadTrim.integration flakes only under
full-suite parallel load — passes in isolation, and on clean HEAD it is green
too, so it is not from this change).
Refs PRINFRA-488
* fix(engine,producer,cli): count drawElement frame timeouts where they happen
`session.deFrameTimeouts` was incremented inside `captureFrameCore`'s catch,
but the deadline races that function from OUTSIDE it — `withFrameDeadline` is
applied by `captureFrameToBuffer`, and `DeFrameTimeoutError` is constructed
nowhere else. So the branch was unreachable: on a wedged renderer the counter
stayed 0, the engine-side stall log never printed, and
`CapturePerfSummary.deFrameTimeouts` reported 0 on every affected render. The
end-to-end recovery worked throughout — `isDeRendererStallError` matches by
name at the producer — only the observability was dead.
Moved the increment and the log into an `onTimeout` hook that fires from the
deadline itself, deleted the unreachable branch and the now-unused
`isDeFrameTimeoutError`, and threaded the count out to
`render_completed.de_frame_timeouts` so the rate is graphable apart from the
other `capture_error` fallbacks. A stall always costs a whole-render re-run,
which is why it deserves its own series.
Also corrected the `fallbackReason` docstring, which still listed only
`psnr | blank | oom | capture_error` after this branch added
`de_renderer_stall`.
`withFrameDeadline` is exported for a two-case fake-timer test: the deadline
fires and calls `onTimeout` once, and a resolving inner promise passes its
value through with no late timer. The first case fails if the counter moves
back inside the work promise.
(cherry picked from commit
|
||
|
Miguel Ángel
|
e0007b86e0 | fix(render): allow resolution scaling for alpha output (#3634) | ||
|
Miguel Ángel
|
045b3a4fd7 |
feat(cli): report which catalog items a render actually used (#3470)
`registry_item_added` fires when a catalog block is installed and `render_complete` fires when a video is produced, but nothing joined them, so "did this video use the catalog?" had no answer. `hyperframes add` now records each installed item in `hyperframes.json` (installed files are plain composition HTML with no provenance marker, so this manifest is the only record that a file came from the registry), and `render_complete` reports both the items the project installed and the blocks the rendered composition actually reaches. An item installed and then never mounted was tried and dropped, which no add-time event can express. The scan answering "which sub-compositions does this file mount" now has one owner, `collectSubCompositionSrcs` in `@hyperframes/parsers`, shared with lint's `lintMissingOrEmptySubComposition`. It holds two invariants that were previously restated per call site and got re-derived wrongly: it is a text scan rather than a DOM query, because `<template>` content is inert and every sub-composition except the render entry is wrapped in one; and references resolve root-relative at every nesting level, matching `parseSubCompositions`. It walks tag by tag rather than running open-ended spans across the whole file, so a malformed composition cannot stall the render plan. Also: `registryItems` is declared in the config schema, which closes with `additionalProperties: false`, with an ajv-backed test pinning every key the CLI writes; counts are never truncated by the name cap, and the reported used blocks stay a subset of the reported installed ones, with `registry_items_truncated` marking a windowed list; and an unreadable manifest reports itself rather than posing as a project that never used the catalog. |
||
|
Miguel Ángel
|
f822200fb8 |
feat(telemetry): measure which lint rules fire, cost, and fail to converge (#3367)
* feat(telemetry): measure which lint rules fire, cost, and fail to converge Lint rule changes are currently argued from anecdote. This adds the three measurements needed to argue them from data. `lint_report`, once per `hyperframes lint` or `hyperframes check`: - `code_counts` / `codes` — which rules actually fire, and how often - `rule_group_ms` — milliseconds per rule-source module (core, gsap, media, ...) - `slowest_rule` / `slowest_rule_ms` — slowest single rule as `<group>#<index>` - `rule_count` — how many rules this build ran `lint_rule_streak`, once per finding that survives an edit to its file: - `edits` — how many edits the finding survived - `cleared` — whether it eventually went away The streak event is the one that matters. A lint pass costs about 5ms, so per-rule CPU is not what makes the authoring loop slow; a rule an agent cannot satisfy is, because every failed attempt costs a full edit-and-relint cycle. A single run cannot see that, so `lint_rule_streak` reconstructs it across runs: high `edits` with `cleared: false` is a rule nobody can fix, and the `cleared: true` distribution is the baseline to judge it against. An iteration is counted only when the file's content digest CHANGED and the finding is still there. Re-linting an untouched project is not an attempt, which is what stops `check` (which lints on every invocation) from inflating the numbers. Rule identity is the source module plus an index within it. Naming all 86 rules would make the timings prettier but it is a refactor this measurement does not need: the group locates the file, and the index locates the rule. Version, agent runtime, CI flag, and invocation id are already attached to every event by `trackEvent`, so lint pain can be split by CLI version and by which agent produced it without adding anything here. Privacy: only rule codes, counts, and timings are sent. Streak state lives in ~/.hyperframes/lint-streaks.json alongside config.json (so `rm -rf ~/.hyperframes` is still a full reset) and stores digests only — no file paths, no project names, no composition source. Nothing is written and nothing is emitted when telemetry is off. Entries expire after 14 days and are capped at 500 files. `EventProperties` gains string arrays and numeric maps. `codes` and `code_counts` are inherently a set and a histogram; flattening them into dynamic top-level keys would make them unqueryable. PostHog stores both natively. `trackLintRun` is the single call site shared by `lint` and `check`, and it swallows every error — telemetry must never turn a green lint red. * feat(telemetry): emit per-group rule counts so slowest_rule stays comparable Review catch on #3367: `slowest_rule` is the one positional key in either event. It is `<group>#<index>`, so adding or removing a rule renumbers every later slot in that group and the same string means different rules in two builds. #3366 does exactly that to 34 of 81 surviving slots, and `rule_count` alone says only THAT the ruleset moved, not which groups. `rule_group_counts` carries the per-group sizes alongside it, so a consumer comparing two builds can tell which groups' indices still mean the same thing without anyone having to remember which release dropped rules. `codes`, `code_counts` and `rule_group_ms` are keyed by name and were never affected. Also corrects the rule count in the RULE_GROUPS comment: 86, not ~60, as LINT_RULE_COUNT in the same file computes. |
||
|
Miguel ÁngelandMiguel Angel Simon Sierra
|
1ae2067b8d |
feat(catalog): put the variables panel back, on payloads (#3199)
* feat(catalog): put the variables panel back, on payloads The panel drove its preview by loading an .html from docs/public, a type the host does not publish, so it showed an empty frame in production and was parked when the catalog was re-landed. It now mounts the same JSON payload the plain player uses and re-mounts it as values change, injecting them as window.__hfVariables into the composition head before any of its scripts run, which is where the runtime reads overrides from. Doing it in the markup rather than after load is what stops the composition initialising with the wrong values first. 172 items with variables get the panel back; the playhead carries across a change so a tweak mid-shot does not jump back to frame zero. * fix(docs): drop the unused url form and the needless escapes * fix(docs): the panel cannot reference a binding beside the export * feat(catalog): make importing an SVG the obvious move A reader arrives at this control with a shape, not with path data, and the panel asked for the coordinates first. Import is now the primary action in a drop target you can see is a drop target, and the raw path sits behind a disclosure for anyone who wants it. * feat(cli): let a fruitless catalog search report the gap An agent that searches by meaning and finds nothing worth installing knows something we do not: the name of a move the catalog is missing. There was no way to tell us, so that knowledge was lost at the end of every run. hyperframes feedback --search-miss "<query>" --wanted "<the move>" records it. It carries no rating, so it never lands in the rating metric, and it is a separate deliberate command rather than something catalog --query does on its own: plain search still sends nothing, which is what the CLI promises. --rating stops being required at the arg level, since a miss has no rating to give. The check moved into the run body, where an absent one is now handled rather than crashing on undefined. * feat(cli): carry tuned variable values into the install snippet Someone who tunes a block on its catalog page had no way to keep those values: the install command was the same one everybody gets, and the tuning stayed on the page. hyperframes add <item> --vars '<json>' now prints a mount element carrying data-variable-values, so the values land where the block is used. They ride on the host rather than being written into the installed file. That keeps the composition on disk byte-identical to the registry's, so a later reinstall can still tell an edit from an update, and it lets two mounts of the same block carry different values. * fix(catalog): serve the item's own directory so runtime paths resolve Some compositions assemble their asset URLs at run time — "compositions/components/" + texture + ".png" for the texture masks, a font the compiler pulled into _remote_media — and no scan of the markup can see a string that does not exist until a script concatenates it. Those items either rendered black or were dropped to a video that had never been uploaded. Each item that needs it now has its prepared directory published, and its payload carries a <base> pointing at it, so any relative path the composition invents resolves. caption-texture renders its masks again, and variable-font-flex has a preview at all for the first time: its MP4 and poster are both 403. Both layouts are published, because which one a composition asks for differs per item, and a directory only earns that if it is under 2 MB. The 12 MB texture sheet keeps the recorded video it already had. * fix(catalog): let the variables panel actually drive the composition Every control on the panel was inert. The values reached the composition and nothing repainted, because the payload had already been compiled: compiling inlines a mounted component and resolves its variables into the markup and CSS, so by the time a reader turns a knob there is nothing left to change. An item that declares variables now ships uncompiled, keeping the mount the runtime loads at run time, which is the only state where data-variable-values still means anything. The component travels inline as a data URI rather than a sibling file, because .html is the one type the docs host will not publish. The demo's own pinned values come off, so the reader's choices reach the mount instead of losing to the values the demo picked to show itself off. Measured on the rendered frame rather than the DOM: green rgb(98,207,144), blue rgb(6,6,199), violet rgb(177,147,230), and back to green. docs/public/catalog drops from 48 MB to 35 MB along the way, since an uncompiled payload carries far less than an inlined one. * feat(catalog): keep variable changes in the url A reader who tuned a piece lost it on reload, and had nothing to send anyone. The values now live in the query string, scoped by composition id so two links never read each other,and only the ones that differ from the defaults are written, so changing one knob gives a short URL rather than every variable spelled out. replaceState rather than pushState: dragging a slider should not leave a trail of history entries. An unreadable value is ignored rather than thrown, so a truncated or hand-edited link opens the piece at its defaults. * fix(catalog): only rewrite the url when a value actually changed * refactor(catalog): memoise the declared defaults on their content * feat(catalog): offer an install command carrying the tuned values The Install block is generated before anyone touches a knob, so it can only ever print the plain command. Someone who spent a minute tuning a piece copied it and got the defaults back. The panel now carries its own command in the Snippet tab, with --vars holding exactly the values that differ. An untouched piece still offers the same short command, so nothing gets noisier for the common case. * fix(catalog): a piece with nothing to render is a skip, not a failure caption-blend-difference is a stylesheet and a paragraph of prose — a class you add to your own captions, with no standalone scene to show. The generator treated that as a build failure, so every run ended by reporting something broken when nothing was. It now reports the shape it is and keeps its recorded video, which is the only honest preview such an item has. A genuine render failure still throws. * fix(catalog): restore variables from the url on a cold load A shared link opened at the defaults. The first render happens on the server, where there is no window to read the query string from, and React then hydrates against that markup and never revisits it — so the values only appeared once you touched a control. The URL is read again after mount, which is the first moment it exists. The value is also escaped once now rather than twice: URLSearchParams already decodes on the way out, and decoding a second time turned an SVG path full of percent-escapes into something that no longer parsed, besides doubling the length of every link. * fix(catalog): mount the preview with the values a link carried The frame was built from the declared defaults and the shared values were posted to it afterwards, which is too late for anything the composition reads once at init: a path arrived after the mark had already been drawn from the default one, so a link looked right in the panel and wrong on screen. * feat(catalog): the install command follows the values you tuned Copying the Install line gave the plain command back, because that block is generated before anyone touches a knob and had no way to know what changed. The tuned command only existed in the panel Snippet tab, which is not where anyone looks for it. The line now reads the same query string the panel writes, so the two agree without either component knowing the other exists, and a shared link carries the right command too. replaceState fires no event, so the panel announces its own writes. * fix(catalog): send a text variable to the preview once it is finished Every other control in the explorer reports a whole value on every event: a slider at any position is a position, a swatch is a colour. A text field is not. Typing v3 into a badge posted v first, so the preview remounted and rendered a composition built from half a word. The post now waits while a text field has focus and goes out when the edit is committed, with Enter or by clicking away. The field itself is unchanged and still tracks every keystroke. --------- Co-authored-by: Miguel Angel Simon Sierra <miguelangelsi07@gmail.com> |
||
|
Miguel Ángel
|
68205dbbc1 |
feat(cli): search the catalog by meaning, on this machine (#3089)
* feat(cli): search the catalog by meaning, in three named tiers Browsing the registry means matching names and tags, which fails whenever the author's wording differs from yours. "make the pace feel faster" finds nothing when the move is described as "velocity-driven blur". This ranks by meaning instead. Three tiers, and the command always says which one answered: words shared vocabulary, free, offline, no account on-device bge-small, free, offline, one opt-in download hosted Gemini, free for signed-in HeyGen users The tier is stated because a quietly worse answer looks exactly like a good one. --json carries it as a token alongside dropped, shown, total and top_score, so an agent reads provenance as data rather than matching English that is written to be reworded. Two consents, asked once each, and never conflated. Sending a query is a privacy question, so the prompt says the query is sent. Downloading a model is a disk and bandwidth question, so that prompt talks about size. Neither fires without a terminal: an unattended run sends nothing and downloads nothing unless a flag records that a person agreed. The catalog is derived from registry-item.json rather than from a separate document, so the set that is ranked and the set that can be installed are the same object by construction. Only the on-device vectors are committed; the hosted vectors are nine megabytes and belong on the server. top_score is reported and never acted on. A "nothing matched" threshold looked clean on long briefs and collapsed on the short queries people type: "a logo appears" scores 0.6181 and keyboard mash scores 0.6417, so any cut that catches the noise rejects the real query. The measurement is in the evals directory rather than in this branch. Not covered here. The published recall figures were measured against a separate hand-written document, not against registry text, so they should not be quoted for this catalog until re-measured. The offline tier needs a normal install: a single-file build cannot load the native ONNX runtime, which the command now reports instead of silently degrading. And the drop-detection path has never been observed firing outside its author's tests. * fix(cli): make this branch pass the repo's own gates Three things `bun run lint` and `fallow audit --base origin/main` rejected. CI runs both, so none of this branch would have gone green. Found by running them, not by reading the diff. process.exit in catalog.ts, twice: an invalid --type and a cancelled picker. check:cli-process-ownership reserves that for cli.ts, and the rule is not cosmetic — process.exit tears the process down where it stands, so anything cli.ts has queued to run on the way out is dropped. finishCommand throws a CliResultSignal that cli.ts turns into the exit code, which is what init.ts already does for a cancelled prompt. Three exports with no consumers. normalize keeps its body and loses its export; localEmbedder is the only caller. modelsDirectory goes entirely, having no caller inside its file or out. The WordPieceConfig re-export goes, and with it the import it existed to forward: the type is exported from wordpiece.ts, where its consumers already take it from. Complexity. prepareOnDeviceTier is lifted out of run(), which took run from 64 cyclomatic and CRAP 948 to 54 and 684. That block is one decision — can the offline tier run, and if not, why not — and its only product is a list of warnings, so it reads and tests as a unit, which it could not do inline. The rest is suppressed rather than refactored, each with its reason on the line above. Finishing run() means extracting its three output paths, and that is a refactor of a command this branch already changes for other reasons: a separate initiative, not something to absorb here. Every suppression says what shape the function has and why; a bare marker on a function nobody can justify is how a threshold stops meaning anything. Verified: `bun run lint` exits 0, fallow reports no issues across 27 changed files, and 2540 CLI tests pass. * feat(cli): ship the local search tiers only, drop the hosted one Search now has two tiers, both local: shared-vocabulary word matching, and the opt-in on-device model. The hosted tier, which sent the query to a HeyGen endpoint and ranked it with a hosted model, is removed. This is a scope decision, not a defect. The endpoint works and its own change is reviewed and green; it is simply not what we want to ship first. Landing local only means the feature has no backend dependency, no auth requirement, and nothing leaves the machine unless someone opts into downloading a model. Gone: registry/smartSearch.ts and its test, the --smart and --no-smart flags, the outcome plumbing through the command, the remote branch of applySearch, the remote tier, and the hosted-only JSON fields (ranking, catalog_version, top_score). Also the smartSearchEnabled consent field in telemetry config, which was the persisted storage behind the hosted consent and would otherwise have been left as dead configuration surface. Kept exactly as they were: both local tiers, the --on-device and --yes flags, the download consent prompt, and the runtime check that happens before the download rather than after it. The --json envelope still reports query, tier, tier_detail, shown, total, dropped, warnings and results, so an agent can still tell which tier answered and why. tierToken now distinguishes on-device from words. Verified: lint exits 0, fallow reports no issues, 2522 CLI tests pass, and the command was exercised directly. A query answers on the on-device tier where the model is installed and falls back to word matching where it is not, reporting that fallback in warnings rather than silently. An unknown --type still exits 1 with a readable message, and --smart is now rejected as an unknown flag. * fix(cli): count only moves this registry cannot install as dropped The dropped count was computed against the list left after the user's own --type and --tag filters, so every move the user excluded was reported as one the registry is missing. Filtering made the number go up: the same query reported 277 unfiltered and 302 with --type block. The count exists so a caller can tell "nothing matched your words" apart from "the ranker suggested things this project cannot install". Conflating it with user filtering destroys exactly that signal, and worse, genuine index skew and a self-inflicted filter printed a byte-identical line with opposite remedies -- one means refresh the shelf, the other means drop a flag, and refreshing does nothing. Now counted against the registry rather than the filtered view. The manifest is already fetched whole and narrowed in memory, so keeping the unnarrowed name set costs no extra request, and item loading still runs only on the filtered subset. Verified against ground truth rather than by eye: the vector artifact holds 411 names, the registry holds 168 installable items, and 134 of those names exist in both, so 277 are genuinely uninstallable. The count now reads 277 unfiltered, 277 under --type block, 277 under --type component and 277 under --tag, and the skew it reports is real -- the artifact predates dropping the UI primitives and still ranks moves that are no longer on the shelf. Reported by Vance Ingalls, who also noted this closes an item the status doc listed as unverified. Two earlier sweeps could not make the count fire because neither combined a filter with a query. Tests pin the three cases: a genuinely absent name counts, a filter-excluded name does not, and a fully installable ranking reports zero. * fix(cli): tell the user when meaning search cannot see the catalog The on-device index was fetched once and never revalidated: the only freshness check was two existsSync calls. A move added after that fetch was invisible to meaning search permanently, not down-ranked but absent from the candidate set. The registry manifest on the same command carries a 24h TTL, so the two halves of one feature disagreed about staleness. The dropped count reported over-coverage only, names the index has that the registry lacks. Under-coverage was never computed, so the harmless direction was instrumented and the costly one was silent. Reproduced with an index truncated to 120 of 168 moves: dropped read 0, perfect health, while 48 moves were unreachable. Counts under-coverage from the name list the artifact already carries, so no extra request. Warns only when non-zero, and names the remedy. The remedy had to be made true: --on-device could not refresh a stale index because hasLocalVectors short-circuited the fetch. That flag now refetches when the index is absent or no longer covering. Two defects the reproduction surfaced. A failed refresh reported the tier unavailable while the old vectors were still on disk and still ranking. And the fetch wrote its two files one at a time, so failing between them paired a new name list with an old matrix, a hard load error rather than stale data. It now writes both or neither, which matters more once refresh runs on staleness. top_score returns, scoped to the on-device tier and set to the score of the best result actually shown rather than the ranking head, which can describe a row the caller never received. Also: scripts/ is now typechecked. It never was, which is how a build script that crashes after the paid embedding call, and two scripts whose imports do not resolve at all, went unnoticed. 43 errors fixed, no suppressions. And the docs stop describing a --smart hosted tier that was deleted, an item that does not exist, and a registry refresh that cannot fix a stale vector index. * ci: fail when the search index stops covering the registry The catalog vector artifact is regenerated by hand. Nothing in CI, in package.json or in a hook rebuilds it, because embedding needs the 32 MB model. So adding a registry item silently makes it invisible to meaning search until someone remembers to regenerate. The failure is asymmetric, which is what makes it easy to miss. Removing an item is self-healing: the ranker still scores the dead vector, then filters the name before display, so a user is never offered something they cannot install. Adding one is not: the item is absent from the candidate set entirely, not ranked low. Comparing the two name lists needs neither the model nor a network call, so the gate runs in seconds. CI checks rather than fixes, for the same reason it cannot regenerate. Scoped to blocks and components. Examples are starter projects a user scaffolds, never something catalog ranks, and the artifact carries no vector for them, so demanding one would keep this gate permanently red and it would be ignored within a week. Verified in both directions rather than assumed: adding an unindexed item exits 1 and names it, restoring the registry exits 0. * fix(catalog): rebuild the search index from the registry build-local-vectors.ts read registry/catalog-artifact/catalog.json, a file no script in this repo writes and which is not committed, so the documented regeneration command failed on a missing path. That is why the index could drift from the registry with nothing to run to fix it. It now reads registry/blocks/* and registry/components/* through catalogFromRegistry, the existing helper that already produced the right shape but had no caller. Rebuilding reproduces the shipped 168 rows byte for byte. A lefthook catalog-index command regenerates and re-stages both artifact files whenever a staged registry-item.json changes, mirroring the skills-manifest pattern, so adding or removing an item keeps the index in sync without anyone remembering to. Verified end to end: staging a new item took the artifact 168 to 169 rows and staged it in 0.80s. * fix(cli): refuse a half-downloaded vector cache The two artifact files have to agree on how many rows there are, and until now nothing checked that before writing them. A truncated or wrong-model response landed in the cache and only failed at load, on every later search, until someone cleared it by hand. The pair is now checked first and refused as a unit, and the cache is created 0o700 with 0o600 files rather than inheriting the umask of a directory the caller may have pointed anywhere. Also lifts the capture setup the two preview generators had drifted into sharing into scripts/preview-capture.ts, and splits the vector builders batching and packing out of main. Both were findings the audit attributed to this branch. * fix(cli): keep the catalog vitest run with the tests it runs Restacking took the base package.json wholesale, which dropped the vitest dependency and the scripts/catalog run this PR adds. Both belong here rather than under it. * fix(cli): stop the declined model download from happening anyway Answering no to the on-device download offer recorded no and warned, then carried on. The guard below it is localModelConsent() !== false, which the decline had just made false, so it was skipped rather than taken: control reached recordLocalModelConsent(true), overwrote the answer with yes, and fetched the 32 MB model the user had refused. Next run it never asked again. No test could catch it. The stub pinned localModelStatus to ready, so the prompt never fired, and recordLocalModelConsent was a no-op that recorded nothing. Two tests now cover the offer, and they need three things the old stubs did not model: the run has to look like a terminal, because off one the command treats --on-device as the consent and never asks; the ONNX probe has to answer true, or an accepted offer returns at the runtime guard before it can download; and the status has to follow the recorded answer, or the second offer later in the run fires as well. Removing the return makes the decline test fail. * fix(catalog): let someone without the model still add a component The pre-commit hook rebuilds the search index, and rebuilding needs the 32 MB embedding model. An outside contributor adding a registry item does not have it, so their commit died inside the ONNX loader on an ENOENT naming a path they never set, and the CI gate then told them to run the command that had just crashed. The model is an opt-in for search, not a build dependency, so nobody is charged for it to contribute. The builder checks first and explains itself, exiting 3 for cannot as distinct from 1 for failed. The hook treats 3 as skip and lets the commit through. The gate now names both paths: regenerate if you have the model, leave it if you do not and a maintainer will. Verified both ways: with no model the builder explains and the hook exits 0; with the model it still regenerates byte-identically. * docs: say that anyone can add a registry item, and stop hand-editing a generated file Two defects, one of them the reason 64 stale entries survived in registry.json. The checklist told contributors to add their item to registry/registry.json. That file is generated from the item directories, so an entry added by hand survives until the next regeneration and then vanishes, and one left behind for a directory that no longer exists is worse: hyperframes add resolves the name and then fails on missing files. Both CONTRIBUTING.md and the agent-facing skill reference now run the generator instead. Nothing said contribution was maintainer-only, but nothing said it was not either, and two steps do need assets an outside contributor has no reason to install. Those are now named in a table with what happens if you do not have them, matching how the preview image was already handled. The search index is the new one: the model behind it is a 32 MB opt-in for search, not a build dependency. * fix(cli): harden on-device catalog search * fix(cli): refresh stale catalog vectors * test: create catalog vector temp dirs securely |
||
|
Vance IngallsandClaude Fable 5
|
19defeabfe |
feat(core,cli): ship the DE parallel router fleet-wide — remove the canary gate
Deletes the `de-parallel-router` canary entry and the `isCanaryEnabled` guard in render.ts together, leaving the producer's default-ON in place. Net effect for users: the parallel drawElement router is on for everyone again. ## Why, and why not a ramp Gating at 5% was itself the regression. Measured 2026-08-08, the day after v0.7.101 shipped the canary: fleet router exposure fell from 3.13-4.25% of non-CI renders to **0.13%**, roughly 25x, because out-of-cohort installs are explicitly disarmed and #2840 deleted the everyone-armed trial in the same change. 2,537 installs lost a feature they already had. Severity is speed only, never output, and nothing is persisted to disk. PR #2840's body claimed "the canary does not make exposure smaller; it makes it chosen and revertible." That was true of the end state and false of the first step. This lands the end state. Entry and guard go together deliberately: at >=100 the evaluator short-circuits ahead of the CI/seedless exclusions, so removing only the entry would have flipped whatever still resolved false at deletion time, unstaged. ## Both stated blockers are void - **≤4-CPU / Docker coverage gap.** Docker renders never use drawElement — 0 of 4,281 across every CPU tier, software GL gates it out — and the router requires it. No percentage could ever expose Docker, so no ramp closes that gap. ≤4 CPUs yields ~42 drawElement candidates in three days. - **PRINFRA-372.** Its signature has hits on 0.4.12, 0.4.37, 0.6.52, 0.6.93, 0.6.109 and 0.6.110 — versions predating drawElement (v0.7.38) and therefore this router. It is real, still live on 0.7.101, and belongs to the screenshot/beginframe path. 11 reproduction runs across four configurations on the enriched profile (darwin/arm64 25.5.0) came back clean. ## Safety unchanged The per-install circuit breaker and the per-render self-verify are untouched; `HF_DE_PARALLEL_ROUTER=false` remains the user-facing kill switch. Post-canary data at 14 days: >8 CPUs 3.02% revert (177/5,857), 5-8 CPUs 2.40% (6/250) — consistent with the 2.75-3.16% baseline. Revert path is now a code revert rather than a registry edit. That is the trade this shape accepts in exchange for one release instead of two. ## Corrects two claims that shipped wrong `~17x jump in exposure onto <=4 CPUs / Docker` overstated the reach, and `~11% of installs already route` was an OUTCOME (the share clearing eligibility and the old 25-render cap), not an exposure setting — read as a rollout knob it inverts the arithmetic, which is how gating at 5% came to cut exposure rather than ramp it. Both are recorded in render.ts so they are not reintroduced. ## Tests Removed the core wiring assertion and the two CLI canary-gating tests, which pinned a gate that no longer exists. Added the inverse guarantee in its place: an ordinary install must come out of the breaker with the var UNSET so the producer default applies — writing "false" there is precisely what disarmed the fleet at 5%. core 1701 passing, cli 2491 passing, studio canary 29 passing. The 2 failures in play.test.ts reproduce on clean origin/main and are unrelated (#3114 area). oxlint and oxfmt clean. Note: telemetry for this rollout stops with the entry — `$feature/canary-de-parallel-router` and `canary_reason_de_parallel_router` are emitted from the registry, so the `Ramp —` tiles and the exposure-floor alert on PostHog dashboard 1918875 go blank once this ships. Watch drawElement engagement on 1807532 instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
0bda6b55b8 |
feat(cli): track which registry items add installs (#3099)
* feat(cli): track which registry items `add` installs `cli_command` records that `add` ran and nothing about what it installed, and the registry is served from raw.githubusercontent.com, which gives no per-item counter either — so there is no way to tell which block or component people actually pull, and no way to know what is worth building more of. Emit one `registry_item_added` event per item written into a project, from `runAdd` after the install succeeds. That is the single choke point: the bulk `add <tag>` path re-enters it per item, and a failed or compatibility-refused install throws before it, so a refused install is never counted as a download. `requested` separates the item the user named from the transitive `registryDependencies` pulled in behind it; without it a popular dependency outranks everything that depends on it. Item names are public registry identifiers, never user content or project data, and the event goes through `trackEvent` — an install that opted out via `hyperframes telemetry disable`, `HYPERFRAMES_NO_TELEMETRY` or `DO_NOT_TRACK` sends nothing. * test(cli): cover `add` telemetry end to end against the built CLI The unit tests assert the emit seam and nothing past it. `shouldTrack()` short-circuits whenever `isDevMode()` is true, and that is true for any `.ts` entry, so under vitest a real event and no event are indistinguishable and the transport is never exercised at all. Drive the built CLI instead and assert on the HTTP body it actually produces: one event per installed item, the dependency reported with `requested: false`, an opted-out install sending no request at all (not merely one without this event), and a refused install counting nothing. Two fixtures, because neither case is reachable through the real registry. The registry origin is a first-class project setting, so a local one supplies the `registryDependencies` edge that no shipped catalog item declares today; and `globalThis.fetch` is wrapped to capture the batch rather than send it. The faked 200 is load-bearing: only a failed flush leaves events queued, and only a non-empty queue spawns the detached `flushSync` child that would bypass the hook and reach production analytics. Verified the check can fail — forcing `requested: true` for every item turns it red on exactly the dependency assertion. |
||
|
James
|
7640adc5f2 | fix(cli): refresh identity persistence classification | ||
|
Vance IngallsandClaude Opus 5
|
b3990ac789 |
feat(studio): emit the canary reason on Studio events too
Review caught that the same anti-pattern was still live in the Studio binding: canaryEventProperties destructured only `enabled` and dropped the reason. Its own doc comment promised 'identical shape to the CLI, so a rollout spanning both reads as one flag' — which the CLI-only fix had just made false. This matters beyond symmetry. A CLI-launched Studio adopts the CLI's decisions and shares its bucket seed, so a cohort flip can surface on either surface. Emitting attribution on only one leaves Studio-observed flips unattributable and makes the two flip counts irreconcilable — and Studio is the surface most likely to expose a shared-seed-with-diverging-id pattern, which is the open question the reason exists to answer. Also adds the no_unit_id emission test the CLI side advertised but never asserted, and a Studio pair pinning that a URL override and a cohort roll produce the same assignment with different reasons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
076657a639 |
feat(core,cli): emit the canary decision reason alongside the assignment
The calibration contract deferred this until the stability check came back dirty. It did: the first fleet read found 304 installs (1.08%) reporting both values for a canary whose percentage never moved, and the genuinely anomalous ones could not be separated from a developer toggling HF_CANARY_*, because the assignment alone is identical in both cases. resolveCanary has always computed the reason and canaryEventProperties dropped it. Now every canary emits canary_reason_<name> beside its assignment. Deliberately outside the $feature/ namespace: PostHog treats those as flag values, and a non-boolean there would corrupt the flag's own breakdowns. Two of the six wire values are immediately useful beyond override attribution. 'excluded' identifies CI installs, which today have to be dropped by joining on is_ci — conflating them with out_of_cohort is what made the first accuracy read look like a significant failure (9.22% against a 10% target) when it was not. 'no_unit_id' surfaces the fails-closed corner. The reason is optional on the core helper so existing callers are unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
WaterrrForever
|
3b65321a20 |
feat(cli): classify identity persistence on every telemetry event (#3065)
* feat(cli): classify identity persistence on every telemetry event Install-grain metrics currently trust every anonymousId equally, but ephemeral/isolated-HOME workloads mint a fresh id per run — one machine produced 2,956 rotating render identities since Jul 30 (94.4% seen on a single render command), inflating acquisition and diluting per-install penetration while looking like real product usage. Every event now carries: - identity_persistence: durable (id loaded from a preexisting config — proven to survive a process boundary) | unknown (minted+persisted this run; an ephemeral HOME is indistinguishable from a genuine first run from inside one process) | process_only (persist failed). Sticky per process so a fresh install re-reading its own write cannot self-promote. - config_write_outcome: ok | ok_unmirrored | failed for the identity- establishing write; absent when the id came from disk. - invocation_id: random uuid per CLI process, so one invocation's events group even when the install identity is untrustworthy (unlike run_id, which needs an orchestrator to set HYPERFRAMES_RUN_ID). Install metrics can then count only durable identities, and a daily churn monitor can alert on the unknown share. * fix(cli): require the anonymousId to come off disk before classifying durable Review finding: materializeConfig mints a replacement anonymousId when a hand-edited/image-baked config lacks one. That replacement only reaches disk when the bucket-seed backfill happens to write; with a seed present the read path performs no write at all, so the install re-mints a fresh id every run while the unconditional durable branch stamped each of them with the one label durable-only counting is allowed to trust. durable now requires parseNonEmptyString(parsed.anonymousId): a minted replacement classifies like a fresh mint — by the backfill write outcome when that path runs (unknown/process_only), and process_only on the no-write path where the id provably dies with the process. Two tests pin both shapes. |
||
|
Miguel Ángel
|
b9233525b2 |
feat(cli): add device authorization login (#2836)
* feat(cli): add device authorization login * refactor(auth): simplify device authorization flow * fix(cli): harden device authorization flow * refactor(cli): simplify device auth validation tests |
||
|
Vance IngallsandClaude Opus 5
|
3f8dca165d |
fix(cli,core): refresh telemetry posture at the render boundary
R6/R7 blockers. An already-open Studio kept emitting server-side render telemetry after another process disabled CLI telemetry. refreshTelemetryPosture() only ran while serving a fresh SPA document and on /api/telemetry-identity, which Studio has no consumer for, so the render POST and its async outcome used the posture cached when the preview server booted. It now refreshes at the render boundary and again immediately before the completion/error event, so an opt-out during a long render is honoured. The identity tests were passing vacuously: their mocks omitted readConfigFresh and resetTelemetryPostureCache, and the resulting missing-export error was swallowed by the refresh's own catch. Mocked properly, plus the enabled -> external disable -> next response transition and the suppression path at the layer that drops the event. A full reset also did not persist its new lineage in a long-lived process: syncInstallState returned early on a process-lifetime memo even after ~/.hyperframes was deleted, so install-state was never recreated and the next config-only re-mint rolled a third seed instead of inheriting the second. The memo is now revalidated against the file. Also drops a stale reference to assertNoOverdueCanaries and stops the workflow and docs claiming the sunset job routes anything to the owner — it names them in the run log and notifies nobody. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
6f0df2640b |
fix(cli,studio,core): close five R5 telemetry and canary findings
- A long-lived preview cached its telemetry posture in two places (readConfig and shouldTrack). Running `telemetry disable` in another terminal left it resolving canaries and injecting the CLI id for hours. Both caches are now dropped together at a request boundary. - Studio minted and shipped a telemetry id for every render regardless of the browser profile's opt-out, and the server emitted the outcome under CLI policy, which cannot see localStorage or DNT. The browser now sends an explicit telemetryOptOut, distinct from an old client's omission. - Any non-empty HYPERFRAMES_PREVIEW_HOST disabled the DNS-rebinding guard, so even a loopback bind accepted a hostile Host. The guard now holds for loopback binds and, on a LAN bind, admits only names this machine answers on. - sunsetAfter had no reader of the current date. A scheduled workflow runs scripts/check-canary-sunset.ts weekly, so a failure lands on the rollout's owner rather than on an unrelated PR author. - The install-state seed memo outlived `rm -rf ~/.hyperframes`, resurrecting a cleared cohort. Removed; it only saved a read on a readConfig cache miss. Docs updated for the Host rule and the 100% exclusion carve-out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
3f69a2c635 |
fix(cli,core,studio): close 15 review findings + 2 R5 blockers
R5 blockers
- Negative install-state latch was cached for the process lifetime, but
only `true` is monotonic across processes. A long-lived preview server
held a stale `false` and could re-enrol after another process tripped
the breaker. Only the positive is cached now; `false` re-reads.
- The real breaker writer used writeConfig(), which collapses
{ok:true, mirrored:false} to success, so a run that mirrored nothing
reported done with the latch only on the erasable store. It consumes
writeConfigWithResult and retries until both stores carry it.
Bucketing integrity
- Storage-restricted Studio profiles all bucketed on the literal
"anonymous": computed against the shipped hash, 100% of them were
enrolled in calibration-50 rather than 50%, and they merged into one
PostHog person. Per-session random id instead — persists nothing.
- bucketSeed had read/write authority backwards: install-state is
write-once authoritative, but readConfig took config.json's blindly, so
the stores could hold different seeds until a re-mint flipped every
cohort. Merged on read, like the latch.
- An unwritable ~/.hyperframes with no config.json re-minted per call,
re-rolling the seed on every command, and the "cohorts will not be
stable" warning was unreachable on that path.
- A corrupt PRE-MOVE state file was never deleted, so a machine reset
with `rm -rf ~/.hyperframes` reported predecessorFound/stateFileCorrupt
forever — poisoning the exact metric this work exists to produce.
Opt-out honoring
- CLI canary decisions memoized per process, so `hyperframes telemetry
disable` during a running preview server was ignored for hours while
the server kept serving pre-opt-out decisions. The memo is keyed on the
telemetry posture.
- shouldTrack() memoized, contradicting policy.ts's documented "not
memoized" contract that policy.test.ts asserts.
- The Studio override path resolved the bucket unit eagerly as an
argument, minting and PERSISTING a tracking id for an opted-out profile
— a value evaluateCanary discards unread.
- Storage reads could throw out of telemetry into a post-commit catch
block, reporting an already-committed edit as failed.
- readConfig printed an unsilenceable stderr warning on every invocation
for installs that opted out of telemetry entirely.
Host split
- isLoopbackHost rejected 0.0.0.0, so the documented
HYPERFRAMES_PREVIEW_HOST LAN mode silently lost CLI→Studio identity
stitching and split one user across two PostHog persons. Identity is
now allowed when the operator explicitly opted into LAN binding.
- Corrected the comment claiming the guard refuses spoofed Hosts: a
non-browser client sets Host freely. It is a browser DNS-rebinding
mitigation, not access control, and now says so.
Semantics and test hygiene
- percentage:100 did not mean everyone — exclude and no_unit_id sat above
the fast path, so the registry's "delete the entry at 100" step was an
unstaged flip for CI and seedless installs.
- CLI cohort adoption returned before evaluateCanary, dropping Studio's
own webdriver exclusion.
- overdueCanaries() was asserted against wall-clock time, so the whole
core suite would go red on 2026-09-15 for every unrelated PR; and `>`
against midnight made a canary overdue ON its sunset date.
- Statistical assertions ran on unseeded randomUUID() populations tight
enough to fail ~1 run in 200. Seeded.
Also: broke a config -> policy -> transport -> config import cycle by
moving POSTHOG_API_KEY to a leaf module.
Tests: 2347 CLI (bundle absent), 3153 Studio, 1450 core. Fault injection
covers the latch, seed authority, LAN identity, webdriver exclusion and
the anonymous-bucket fix. Two pre-existing tests asserted behaviour these
findings identify as wrong (shouldTrack memoization, 100%-excludes-CI)
and were rewritten with the reasoning stated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
5e2a9432f1 |
fix(cli,studio): close the five R4 blocking gaps
P1 — the required Test lane was red, and it was my test. The
hostile-Host SPA case asserted a 200, which only holds when
packages/studio/dist is built: true on a dev box, false in CI, so it
passed locally and failed there. The Host split moved into a pure
buildStudioHeadScriptsForHost() and is asserted directly; the route test
no longer depends on build state. Verified by running the CLI suite with
the bundle moved aside — 2330 pass.
P1 — studio:* still bypassed most privacy controls. It honoured two
localStorage keys but not navigator.doNotTrack,
VITE_HYPERFRAMES_NO_TELEMETRY, Vite dev mode or API-key eligibility, and
canary enrolment honoured a different single control. New
telemetry/policy.ts is the one answer to "may this profile be measured",
consumed by both transports and by enrolment. It imports only ./config,
so no cycle with the modules that import it. Each control is asserted
individually.
P1 — LAN/remote preview lost the authoritative decisions. Withholding
the whole head script for any non-loopback Host also dropped the safe
{enabled, forced} map, sending a supported HYPERFRAMES_PREVIEW_HOST=
0.0.0.0 Studio back to re-deriving. Identity injection is now gated
separately from decision injection: identity is loopback-only, decisions
always publish.
P1 — the breaker latch was neither authoritative nor truthfully
persisted. syncInstallState swallowed its own failures so
writeConfigWithResult always reported ok, and reads took the flag only
from config.json. The latch is now merged into every effective read,
which makes install-state authoritative and closes both the failed-mirror
and stale-concurrent-writer paths; the write additionally reports
mirrored: false rather than swallowing.
P2 — public contracts. canary-rollouts.mdx said a config wipe loses the
breaker (it does not) and documented the superseded {name: boolean} map
with unconditional CLI precedence; both corrected, with the precedence
ladder written out and the override exception stated explicitly. PR body
rewritten — it still named ~/.local/state, claimed state survives
deleting ~/.hyperframes, and carried stale counts.
Tests: 2330 CLI (bundle absent), 3151 Studio, 24 core. Fault injection:
reverting each fix alone fails 5 CLI / 5 Studio.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
f81ab0162e |
fix(cli,studio): close the four R3 blocking gaps
P1 — SPA route bypassed the DNS-rebinding guard. Guarding only
/api/telemetry-identity left the catch-all as an open side door: a
rebound origin could fetch `/` and read __HF_CLI_DISTINCT_ID and
__HF_CLI_BUCKET_SEED straight out of the returned HTML. The SPA response
now applies the same isLoopbackHost() check; an untrusted Host still gets
a working Studio, just with no identity, seed or decisions injected.
Route-level regression added.
P1 — a CLI cohort roll could override Studio's own opt-out.
decideStudioCanary() adopted the injected decision before checking
isOptedOut(), so CLI-telemetry-on plus Studio-opted-out still enrolled
Studio. A bare boolean could not express the difference between a
deliberate override and an ordinary cohort roll, so the injected map now
carries provenance ({ enabled, forced }). Forced wins outright — it is
the documented escalation channel and must behave the same on both
surfaces — while a percentage roll now loses to this profile's opt-out.
Full interaction matrix tested.
P1 — the legacy studio:* path sat outside both contracts.
utils/studioTelemetry.ts shipped its own opt-out key and its own send
loop, so the documented hyperframes-studio:telemetryDisabled did not
silence it and its events carried no cohort assignment. It now honours
both keys (the legacy one stays, so nobody already opted out is quietly
re-enabled) and mixes in canaryEventProperties(), making "every
telemetry event carries the assignment" actually true.
P2 — partial salvage could drop a tripped breaker.
salvageInstallState() discarded the whole record when markerAt and
bucketSeed were both unusable, taking deParallelRouterTrialFired with it
and re-enrolling a machine whose router already failed. All three fields
are now independently salvageable.
Docs: canary-rollouts.mdx said "disabling telemetry disables the
reporting, not the enrolment" — exactly backwards since the opt-out gate
landed. Corrected; checked for other copies, none.
Tests: 13 new (4 opt-out precedence, 4 legacy-path opt-out and canary
props, 3 route-level host guard, 2 breaker salvage). Fault injection:
each of the four fixes reverted independently fails its own tests
(2 CLI + 1 Studio + 2 Studio).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
31361b8e5b |
fix(cli,core): close the remaining canary review findings
Six findings from review, none behaviour-critical on their own but three of them quietly corrupt the data the rollout is judged by. Endpoint no longer serves bucketSeed (studioServer.ts). Studio gets its canary answers from the injected decisions map now, so nothing needed the seed over HTTP — and an unauthenticated local endpoint is a strictly worse place for it than a script scoped to Studio's own document. The endpoint itself predates this PR and still serves distinctId, so it also gains a Host guard: a remote page can rebind its hostname to 127.0.0.1 and read the response as same-origin, but the request still carries THAT hostname, which is what makes it refusable. predecessorFound no longer reports corruption as a fresh install. It returned null for both "file absent" and "file unreadable", so a partial disk write looked like a new machine — understating recoverable churn, the one thing the field measures. Now distinguishes absent from corrupt and emits install_state_file_corrupt alongside. A mangled markerAt no longer discards a salvageable bucketSeed. markerAt is only a timestamp and can be restamped; the seed cannot be recovered, and losing it silently re-rolls the install's cohort. The seed backfill no longer ignores its write result. An unwritable ~/.hyperframes meant a different seed every invocation with no diagnostic, and made the field's own "backfilled once" docstring false. Warns once per process with the underlying error. FNV-1a's ASCII constraint is now explicit rather than incidental. It hashes UTF-16 code units while reference FNV-1a is byte-oriented, so the two agree only on ASCII; the registry's kebab-case assertion is what makes non-ASCII unreachable, and both ends now say so. Not a live bug — names are kebab-case and units are UUIDs. de-parallel-router is pinned at 0%. The registry is data, so a ramp is a one-line edit with no review surface, and its own description says to ramp only alongside the circuit breaker. Tests: 8 new (corruption vs absence, seed salvage, backfill write failure, 17 host-guard cases, registry pin). One existing test asserted predecessorFound: false on corruption — that was the bug, updated with a note. Fault injection: restoring the old corrupt handling fails 4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
98b23a8850 |
fix(cli,studio): adopt the CLI's canary decisions in a launched Studio
Closes both cross-surface findings with one mechanism. The CLI publishes
window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }); a CLI-launched
Studio takes it as authoritative over its own seed, URL override and the
registry percentage.
Studio re-deriving could not agree with the CLI in three cases:
- Telemetry off. The CLI resolves telemetry_opt_out, but Studio's
opt-out is a separate localStorage flag it cannot see, so it would
evaluate normally and could enrol on a render the CLI excluded. The
previous commit gated each surface independently; that fixed silent
enrolment per surface but NOT the disagreement between them.
- HF_CANARY_* override. Env vars never cross into the browser — Studio
reads only its URL param / sessionStorage — so a support session
forcing a canary on got the CLI forced and Studio guessing.
- No seed injected. Studio falls back to a different unit id, i.e. a
different bucket.
Shipping the decision instead of the inputs makes divergence structurally
impossible: one evaluation, two surfaces. It also exposes strictly less —
booleans about features, rather than the seed buckets derive from — which
is why it is safe to publish with telemetry off, the case it exists for.
Studio still evaluates locally when standalone, or for a canary the CLI
did not publish, and ignores a non-boolean value rather than trusting it.
Tests: 6 Studio (CLI-off wins over unset local flag, CLI-on with no URL
param, beats contradicting override, beats seed, falls back per-canary,
rejects non-boolean) and 4 CLI (decisions with telemetry off and no
identity, alongside identity when on, script-tag escaping on a hostile
canary name, throwing resolver degrades to identity only). Four existing
identity tests asserted the old "nothing when telemetry off" contract and
were updated; the registry is now mocked there so string assertions don't
move when a canary is added or ramped. Fault injection: dropping the
adoption fails 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
4f464dc424 |
feat(cli,studio): telemetry opt-out is canary opt-out
Both reviewers flagged the same gap: seed injection was gated on telemetryShouldTrack(), but canary EVALUATION was not. An install with DO_NOT_TRACK=1 was still bucketed and still had real code paths flipped (e.g. HF_DE_PARALLEL_ROUTER), silently and unmeasurably. A canary is a measured rollout — we enrol a slice precisely so it can be compared against everyone else. An install that sends nothing can't be compared, so enrolling it buys no signal and only changes that user's code path, on an experimental feature, without their knowledge. That is the wrong side of an opt-out. Resolves to a new `telemetry_opt_out` reason BEFORE bucketing, so no cohort is assigned at all. Distinct from `excluded` because "why is my canary off" has a very different answer for CI than for opted-out, and the reason never reaches telemetry by construction. Covers every opt-out route: persisted preference, the runtime env vars and dev/telemetry-disabled builds via policy.ts, and Studio's hyperframes-studio:telemetryDisabled. An explicit HF_CANARY_* / ?hf_canary_*= override still wins — a deliberate local choice, not silent enrolment, and the documented way to exercise a canary with telemetry off. The CLI check mirrors shouldTrack() rather than importing it: client.ts already imports canary.ts for canaryEventProperties, so depending on it would be a cycle. Both read the same two inputs, so they cannot disagree. Tests: 9 new across CLI and Studio (preference off, each runtime override, no bucket assigned, override still honoured, flag properties all-false). Fault injection: removing the CLI gate fails 6, removing the Studio gate fails 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
9f2b892a71 |
fix(cli): keep the canary bucket seed inside the config dir
Adopts the #2904 pattern for bucketSeed. The seed rides in install-state.json, which now lives beside config.json in ~/.hyperframes rather than in ~/.local/state/hyperframes/. Review rejected persisting state outside the config dir to defeat a user's reset, and that objection is sharpest for the seed: it is the one field that would turn install-state into a persistent pseudonymous identifier surviving `rm -rf ~/.hyperframes`. The carryover still earns its place, just against the churn that actually happens. config.json is rewritten on every command and every render, and readConfig recovers from any parse/permission/IO failure by minting a fresh identity — so a re-mint would reshuffle cohorts mid-rollout. A no-schema file written once at mint is decoupled from that without leaving the directory. Config re-mint: cohorts hold. Directory deleted: cohorts go too, deliberately. A pre-move seed is adopted by the same migration, so installs already carrying one do not have a live cohort reshuffled under them. Tests: seed survives a re-mint, does NOT survive deleting the config dir, and migrates from the pre-move path. Prose in config.ts, canary.ts and canary-rollouts.mdx corrected — it still claimed cohorts survive a wipe. Docs gain a removal-path section. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
8dd20ac2e8 |
feat(cli): bucket canaries on a machine-lineage seed, not the telemetry id
Cohort membership now survives a config wipe. Canaries bucket on a dedicated bucketSeed (fresh random UUID, distinct from anonymousId by design) that is mirrored write-once into the install-state file and inherited at mint: a wipe re-rolls the telemetry id but never the canary assignment. This removes cumulative-exposure drift for the recoverable churn bucket entirely — the residual drift comes only from fresh machines, containers, and genuinely new users — and keeps before/after comparisons valid across a reinstall. The seed is never emitted in telemetry (only the resulting true/false assignments are), so it does not link the old id to the new one server-side. The residual linker is the flag vector itself (k bits for k live canaries), documented as such. An explicit reset still works by deleting the state file, and the no-identity test now also asserts the seed differs from the anonymousId. Cross-surface coherence: the CLI's studio server injects the seed as window.__HF_CLI_BUCKET_SEED (same telemetry gate and script-escaping as the distinct id, and on the /api/telemetry-identity fallback), and the Studio binding buckets on it when present — without this the CLI would bucket on the seed while Studio bucketed on the distinct id, splitting one machine across cohorts (calibration check 4 would catch exactly this). Standalone Studio still buckets on its localStorage id: the browser has no second storage location, so that id doubles as the seed. Legacy configs are backfilled once (lineage seed if the state file has one, else minted) and persisted immediately — an unpersisted seed would re-roll cohorts every process. Safe to ship in the same release as the first canaries: no prior release emitted canary properties, so the bucketing-unit change is unobservable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
a1682e1228 |
feat(core): emit canary assignments as PostHog flag properties
Replaces the single `canaries: "a,b"` telemetry property with PostHog's own
flag shape, one property per registered canary:
$feature/canary-de-parallel-router: "true" | "false"
PostHog treats `$feature/<key>` as a first-class flag property, so breakdowns,
funnels split by cohort and the experiment surfaces work on a canary with
nothing configured server-side. The decision still happens locally: the render
path forbids render-time network calls, behaviour must not depend on analytics
being reachable, and neither the CLI nor Studio ships posthog-js (both
hand-roll a batch POST, so there is no SDK to evaluate a real flag with).
Decide locally, analyse natively.
Two decisions worth recording:
- BOTH ARMS ARE EMITTED. A non-enrolled install reports "false" rather than
omitting the property. Absent means "this build predates the canary", which
is a different fact from "this install is control" — collapsing them makes a
ramp unreadable, because you cannot separate a control group from an old
version.
- KEYS ARE NAMESPACED with a `canary-` infix. A real PostHog flag namespace
already exists in this project, owned by the web app (`enable-chat-tab`, set
by posthog-js from `$lib=web` events). Namespacing guarantees a canary key
can never alias a real flag key and have the two fight over one property.
Values are the strings "true"/"false" to match how PostHog records boolean
flag values, so the property is directly comparable to a real flag.
98 core / 1437, 166 cli / 2194, 269 studio / 2982 green; tsc clean across all
three packages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
71ee156dac |
feat(studio): browser canary binding + leaf subpath imports
Adds the Studio (browser) binding so a canary can span the CLI and the editor,
and fixes a bundling mistake the studio test suite caught.
## The binding
Same public API as the CLI — `isCanaryEnabled("name")` — so a call site reads
identically whether it runs in Node or the browser. Three inputs differ:
- UNIT ID: `resolveStudioDistinctId()`, which already adopts
`window.__HF_CLI_DISTINCT_ID` when the CLI launched Studio. A CLI-launched
Studio therefore lands in the SAME cohort as the CLI: a rollout spanning
render and editor is coherent for that user instead of enrolling their
terminal but not their editor. A test pins that the id is passed through
unmodified — prefixing or re-hashing it would silently break that parity.
- OVERRIDE: no `process.env` in a page, so `?hf_canary_<name>=on` mirrored
into sessionStorage. Session scope is deliberate. A URL is the right carrier
(shareable — "support: open this link"), but persisting a URL-borne override
to localStorage would let one click silently pin a browser into a cohort
forever, long after anyone remembers why. Closing the tab is the reset;
`=reset` clears it explicitly.
- EXCLUSION: `navigator.webdriver` stands in for the CLI's `is_ci`. Automated
browsers mint a fresh localStorage id per run, so they would hop cohorts
between runs — noise in the signal, nothing learned about real users. An
override still reaches them, which is how you test a canary under Playwright.
Studio's `trackEvent` now attaches `canaries` to every event, mirroring the CLI.
## The bundling fix
Importing the `@hyperframes/core` barrel into studio browser code broke two
unrelated hook test files with an esbuild TextEncoder invariant violation. The
barrel re-exports the whole core surface (parsers, lint, studio-server), so it
drags a Node-oriented dependency graph into a browser bundle — the test
failure was the symptom, the bundle bloat was the bug.
`@hyperframes/core` now exposes `./canary` and `./canary-registry`, declared in
packages/core/package-subpaths.json (the generated source of truth for exports —
hand-editing package.json is reverted by the sync script) and marked
`environments: [browser, bun, node]`. Both the studio AND cli bindings import
the leaf modules; the CLI gets the same benefit for a different reason, since
this resolves on the startup path — the reason the producer is lazily loaded.
Verified: the two hook files pass again; 269 studio files / 2982 tests, 98
core / 1433, 166 cli / 2194 green, `bun run lint` clean including the subpath
check. Fault-injection confirms both design decisions are pinned — swapping
session for local storage fails the scope test, prefixing the unit id fails the
CLI/Studio cohort-parity test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
df1521a0b6 |
feat(cli): attach canary cohort to telemetry, harden canary tests, document
Follow-up to the canary primitive. Telemetry: every event now carries a `canaries` property listing the cohorts the install is enrolled in, attached in `trackEvent` so it lands on ALL events rather than renders only — a staged rollout is only as useful as the ability to split any metric by cohort. Resolved after the shouldTrack guard, so opted-out installs never pay for it, and omitted entirely (not null or "") when the install is in no canary, since PostHog treats those as real values. Test hardening, after validating the shipped code against 60k synthetic and 101 real fleet install ids: - Pin FNV-1a against canonical vectors, AND assert the shipped canaryBucket actually uses that hash. Without the second assertion the first is tautological — it would only prove the test's own copy is correct while canary.ts drifted to a different hash, silently reshuffling every live cohort. Fault-injection confirms only this assertion catches a hash change; the distribution tests stay green because a perturbed hash is still well-distributed. - Tighten the share test from a 0.6x-1.4x band to +/-1 percentage point. Measured error was 0.16pp at n=60k, so the old band would have passed a badly skewed hash. - Add chi-square uniformity across all 100 buckets (chi2 89.0 vs 148.2 critical at p=0.001). A lumpy hash yields roughly the right total share while overloading some buckets, so the share test alone cannot catch it. - Assert N concurrent canaries enrol binomially rather than in lockstep: 8 canaries at 10% put ~43% of installs in none and zero in all eight, matching binomial(8, 0.1). Correlated slices would put ~10% in all eight. Also verified 88,443 of 88,448 fleet install ids are well-formed UUIDs; the 5 that are not fail closed, which is the intended direction. Docs: docs/contributing/canary-rollouts.mdx, registered in docs.json (an unregistered page is invisible in the nav). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |