140 Commits
Author SHA1 Message Date
Vance Ingalls 859ac622c2 fix(player): report and fail closed on runtime delivery errors (#3472)
* feat(player): report runtime data application

* fix(player): fail closed on runtime data delivery

* fix(player): close runtime delivery and sandbox gaps

* fix(player): satisfy runtime contract and CodeQL

* refactor(player): simplify runtime tag scanner

* style(player): apply repository formatter

* fix(player): type runtime tag boundaries

* fix(core): mint guest-local runtime-data ids in a separate space from host ids
2026-08-30 13:00:14 -07:00
Miguel Ángel 7e96e60fe2 ci: bound the ffmpeg apt fetch so a stalled mirror costs a retry, not the job (#3356)
* ci: bound the ffmpeg apt fetch so a stalled mirror costs a retry, not the job

Hosted runners intermittently stall on an apt mirror, and an unbounded
apt-get inherits the whole job budget. The producer integration lane normally
finishes in ~11 minutes against a 20 minute cap; on a stalled fetch it ran to
the cap and failed. Same step, same shape, reproduces on main's tip — it is not
specific to any one PR.

The cost is not one red check. On the run that prompted this, four went red off
that single step: the two jobs that install ffmpeg, plus a Test gate and a
preview-regression gate that both fail closed when their dependency does not
succeed. So a mirror stall reads as a producer defect and a preview defect.

Each attempt is now bounded and retried three times, and the five workflows
that installed ffmpeg share one action instead of five copies of the command.
Deliberately still apt: caching the binary would strip it from the shared
libraries it links against, and switching to a static build would change the
ffmpeg under the producer's output comparisons. Neither belongs in a fix for a
network stall.

* ci: drop the stray version echo left in the player-perf ffmpeg step

Converting the step to the shared action left the trailing `ffmpeg -version`
line behind, and YAML folded it into the `uses:` value — so the runner looked
for an action at a path with the command appended and failed all four perf
shards.

It parsed cleanly, which is why validating with a YAML load did not catch it:
`uses: ./path\n  ffmpeg -version` is a legal folded scalar. The check that
does catch it asserts every local `uses:` resolves to a directory containing
an action file, which is now what I ran. The action prints the version itself.

* ci: bound the ffmpeg fetch at the connection, not with a wall-clock kill

The first version wrapped apt in `timeout` and retried. A passing run showed
why that is the wrong shape: the mirror is slow rather than hung — the install
spent ~15 minutes pulling packages from azure.archive.ubuntu.com and finished
successfully. Killing it at 300s discarded a download that was making progress
and started over, so the retry turned a slow mirror into a slower one, and the
worst case of three attempts exceeded the job's own 20 minute cap.

Bound the connection instead. Acquire::Retries re-fetches the one package whose
connection stalled while keeping everything already downloaded, and
Acquire::http::Timeout caps how long any single connection may sit idle. That
addresses the stall the original report described without punishing the slow
case that is far more common.
2026-08-19 18:39:05 -04:00
Vance Ingalls de4062a933 fix: create temp dirs with mkdtemp, not a name built from Date.now() (#3241)
* fix: create temp dirs with mkdtemp, not a name built from Date.now()

Closes nine open `js/insecure-temporary-file` alerts — the technically
correct ones. An audit of all 29 open alerts for that rule split them
three ways:

- 19 false positives: the write lands inside a directory the caller
  already made with `mkdtempSync`, and CodeQL's dataflow reaches
  `tmpdir()` without seeing the mkdtemp in between.
- 1 mitigated: `fontCompression.ts` writes with `flag: "wx"` and only
  takes the tmpdir branch inside Lambda, where /tmp is single-tenant.
- 9 real, and these are them. A name built from `Date.now()` under the
  shared temp dir, followed by `mkdirSync`, is guessable to the
  millisecond AND leaves a window between choosing the name and creating
  it, so on a shared machine another user can pre-create or symlink the
  path first.

`mkdtempSync` closes both halves: it picks the random suffix and creates
the directory 0700 in one syscall. Same shape, one line shorter, and the
alerts go away rather than being dismissed.

Six sites in `normalize.test.ts` (its `mkdirSync` import goes with them),
one in `generate-catalog-previews.ts` — that single construction accounted
for three alerts, since the other two were writes into the directory it
made.

No shared helper. `mkdtempSync` is already the stdlib primitive for
exactly this, and the two callers live in different packages, so a wrapper
would need a home in core to serve one CLI test and one build script —
more indirection than the line it saves.

Deliberately not touching the other 20: excluding the rule repo-wide would
hide this class of bug from future code, which is the reason these are
fixed rather than silenced.

* fix: track the wav temp dir for cleanup and finish the mkdtemp sweep

The wav helper pushed the file path into `dirs`, so `afterEach` removed
`tone.wav` and left the directory it had just made — four per suite run.
Push the directory and derive the file path from it. Measured: the old code
leaks 4 directories per run, the new code leaks 0.

Three sites still built a predictable name and then created it. CodeQL never
flagged them — its dataflow reaches the template preview writes through a
`readdir` walk and does not connect them back to the `tmpdir()` root — so the
alert list was narrower than the pattern, and closing only the alerts would
turn the rule green while the shape survived where nothing would re-flag it.
`generate-template-previews.ts` is the near-twin of the file this change
started from, and the other two are producer dev entry points. All three use
the path only through the variable, so the random suffix changes nothing.

Catalog previews now call the existing `createCatalogPreviewTempDir` instead
of repeating its body. That test was in no runner, so it pinned uniqueness and
mode 0700 on a function nothing called; adding it to `test:scripts` alongside
a real caller makes it load-bearing. The rationale for the primitive moves to
the helper, which is now the only place it lives.

* ci: re-run catalog previews when the temp-dir module changes

Routing the renderer through `createCatalogPreviewTempDir` made that module
part of its runtime path, and the workflow already states the rule for the
sibling case: a module the renderer imports has to appear in the trigger, or a
change to it alone never re-runs the job that exercises it. Add it to the
`paths:` filter and to the renderer canary, so a PR touching only the temp-dir
allocation still renders both shape canaries.

Verified against this branch's own range: the previous argument list does not
report the file, so a helper-only PR was invisible to both checks.
2026-08-14 11:20:37 -07:00
Miguel Ángel 9734578e60 feat(registry): bring back the video-primitive moves (#3169)
Restores the 208 catalog items reverted after their previews 404'd in
production, this time on the payload mechanism rather than the .html files
that caused the outage.

The generator no longer writes a preview document to docs/public. That writer,
and the machinery under it, existed only to produce files the docs host
discards, so it is gone rather than bypassed. Items now embed the composition
itself via a payload, which is what the previous change already does for the
items that were already in the catalog.

The variables explorer is parked, not restored: it drove its preview through
the same unpublished .html path, so it would have shown an empty frame. Items
that declare variables get the live player plus the static variables table, and
reconnecting the explorer to payloads is a follow-up.
2026-08-10 18:46:03 -04:00
Miguel Ángel c86d4013f5 Revert "feat(registry): the video-primitive moves, documented and customisable (#3090)" (#3162)
This reverts commit 3b53bfd2f7.
2026-08-10 14:47:47 -04:00
Miguel Ángel 3b53bfd2f7 feat(registry): the video-primitive moves, documented and customisable (#3090)
* feat(registry): add the video-primitive moves, and rebuild the catalog around them

Adds the motion primitives: 277 new components and the blocks that go with
them, plus the ui-primitives, themes and generators they are produced by. The
registry index goes from 176 items to 454, and the search catalog is rebuilt so
the set that is ranked is the set that can be installed.

Additive on purpose. An earlier pass of this port used rsync --delete, which
removed 101 files that exist on main because the incoming set is not a superset
of the current one: beat-freeze-cut and camcorder-hud among them. Whether the
re-port replaces those or sits alongside them is a product decision and not one
a sync flag should make, so nothing is removed here. If any of them are meant
to go, that belongs in its own commit where it can be seen.

The generator is ported too. Main's version only scans examples, so running it
without this change silently rewrote the index down to nine items. It also
rewrites example manifests from templates.json and will overwrite hand-edits;
those were reverted here after each run, and the diff is worth reading rather
than trusting.

Not covered. The 445 moves are not individually reviewed in this commit; the
machinery that ranks and installs them landed separately so it could be read on
its own. The internal evaluation corpus is deliberately absent: it is 1,400
files of briefs, gold labels and verdicts, and this repository is public.

* docs(catalog): publish the primitive and component pages

Adds the Mintlify pages for the moves this PR ships: 163 component pages, 13
primitive pages, and the navigation that lists them. Without these the moves
land installable and undocumented, which is the worse half of a catalog.

Three things left out deliberately.

The 78 MB of docs/public. Nothing references it: every page loads its preview
from static.heygen.ai, so those bytes would be weight in a public repo with no
reader. Checked rather than assumed, by grepping the pages for the path.

Pages for the thirteen moves that were specified and never built. They had
documentation but no registry item, so a reader would have followed a page to a
`hyperframes add` that fails. Their nav entries are pruned with them, and every
one of the 309 remaining catalog and primitive nav entries was verified to point
at a page that exists.

Spike and scratch files that sit alongside the real docs on the source branch:
qa-gallery.html, experiment pages, bundled player javascript. They are working
artifacts, not documentation.

Not covered: the pages are generated output and have not been read individually.
The nav is verified to resolve, and the previews load from a CDN this commit
does not control, so a broken image would show up in review rather than here.

* docs: list the primitive and component pages in the site navigation

The pages this PR adds were unreachable: nothing in docs.json pointed at them.
This appends a Motion primitives group to the existing Catalog tab and a
Primitives tab, both built from main's navigation rather than replacing it.

Copying the source branch's docs.json wholesale was the first attempt and was
wrong. That file describes a different site, tabs Documentation / Catalog /
Primitives / Packages / SDK / Reference against main's Guides / Studio /
Catalog / Developers, and it references pages only that branch has, so the
preview server reported six dead links.

Verified by running the preview and resolving every entry: 484 page refs, 0
dead, no warnings. Group-relative refs are why a flat existence check is the
wrong validator here: cursor resolves through catalog/components and mcp
through guides, so checking docs/<ref>.mdx flat pruned 22 entries that were
fine.

* fix(registry): restore what the port took from main's components

Two regressions this branch introduced into items main already ships. Both were
found by the repo's own gates in packages/cli, not by reading the diff, and
neither is visible to the no-deletions check: no file was deleted, the contents
of files were changed.

The four liquid-glass blocks stopped installing their library. main lists
lib/liquid-glass.iife.js as a second file on each; the port wrote the older
manifest over main's and dropped that entry. The file is still in git and still
on disk, it simply stopped being something `hyperframes add` writes, so the
installed composition's <script src="lib/liquid-glass.iife.js"> would have
resolved to nothing. Every one of the 294 registry-item.json files this branch
touches was then audited against main: these four lost a file entry, and no
item lost a top-level key.

Fourteen caption components gained an empty <video>. The port added
`<video id="wp-video" ...></video>` — no src, no <source> — to each component
and its demo. It renders nothing and the registry linter rejects it as
media_missing_src. Removed rather than given a placeholder, because main's
version of each of these composes over whatever the host composition provides,
so the element only ever added a broken node; a made-up src would ship a
reference to footage that does not exist.

The removal is deliberately surgical. Four of the fourteen also carry a
substantial rewrite from the port, and only the media element and the rule that
styled it are touched, so a blunt revert cannot take the rewrite with it.

Verified: 2540 CLI tests pass, `bun run lint` exits 0. Before this, three tests
failed.

* docs(catalog): play the real composition, and show what can be changed

Four changes to generate-catalog-pages.ts, so all 445 pages stay consistent
rather than 445 files being edited by hand.

The preview plays the composition instead of pointing at a video. Every new
page pointed at static.heygen.ai/<name>.mp4 and every one of those answered
403, so the reader got a black box where the whole point of the page is to
show them the thing. The objects were never uploaded and rendering 445 of them
would have to happen again on every change. The player is already the thing
being documented, so the page embeds it: the item's directory is copied under
docs/public and an iframe loads it through a small wrapper. 444 of 445 pages
play; the remaining one is a texture item that uses its own preview panel.

The iframe is not decoration. Compositions set styles on `body`, so dropping
the element straight into the MDX would put a composition's global CSS in the
same document as the documentation around it.

Three things this got wrong first, all found by opening the page rather than
reading the output:

  - The wrapper loaded itself. `../<dir>/<name>.html` from inside preview/<dir>/
    resolves back into preview/<dir>/. The player embedded the player and the
    frame went black with a second set of controls shrinking into the corner.
  - Copying only demo.html was not enough. Most demos are a mount shell whose
    child carries data-composition-src="./<name>.html", so the sibling has to
    come with it. Every URL answered 200 and the frame was still empty.
  - `autoplay` and `loop` are not player attributes. Writing them did nothing
    and every preview sat paused on frame 0 — which is blank for any
    composition that animates in. The wrapper drives play() and loops on
    `ended` instead.

The Variables table. generateParams reads `params`; every item ported from the
video-primitives work declares `variables`, a richer schema with a type, a
default and a range. 112 items carry one and not a single page showed it, so
the most useful thing on the page was the one thing missing.

Nav groups. `if (entry.type === "component") return "Effects"` was the
catch-all, so Effects held 267 of 445 pages: an alphabetical wall. Rules keyed
on tags that already exist in the manifests split it; the largest group is now
73.

An install command with a visible copy button. A plain code fence renders one
on hover only, and it was absent from the accessibility tree entirely. This is
the one line every reader comes to take. navigator.clipboard is unavailable on
insecure origins, which is exactly the local preview these pages are written
against, so the fallback path is load-bearing and is what was exercised in
testing.

Verified: 888 preview URLs fetched, 0 failures. Regenerating three times in a
row produces no change, after a first attempt where "Variables" was added to
GENERATED_HEADINGS with a capital V — the set is compared lowercased, so each
run carried the previous section forward and appended a new one.

Not covered: the 445 pages were not read individually. Coverage here is that
every preview resolves and that a page from each of the block and component
paths was opened and watched.

* docs(catalog): put the code on the page

A reviewer with no stake in the work compared these pages against shadcn/ui's
component pages and motion.dev, and returned one gap: the pages carry almost no
code, so they are pointers to a file the reader does not have yet. Its sharpest
example was the Variables table — names, defaults and accepted values, headed
"set the ones you want to change on the element", on a page that never shows an
element or the syntax for setting anything on one.

Two additions, in the generator so all 445 pages get them.

A snippet under the Variables table: the real mount element with
data-variable-values filled in from the item's own defaults, so it is
copy-and-run correct before it is edited. That is the syntax the demos actually
use, not an illustration written for the page.

The item's source, in a collapsed Accordion. These files run 99 to 463 lines,
so inlining them raw would bury everything else; collapsed, the code is on the
page and one click away. Accordion is already what these docs use for this.

A second reviewer, fresh, confirmed the change landed: it called the table and
snippet actionable rather than filler and said the collapsed source earns its
place.

Also here: the preview retries play() until the clock moves. `ready` can flip
before the runtime the player injects for a mounted sub-composition has finished
wiring up, and a play() landing in that window silently does nothing.

Both reviewers additionally reported every preview frozen at 0:00 and called it
fatal. It is not. The player's clock runs on requestAnimationFrame
(direct-timeline-clock.ts), browsers suspend rAF in a hidden tab, and the
reviewing tab was hidden: document.visibilityState read "hidden" while the
player reported ready and not paused, and a one-second rAF loop never completed
a single tick. Seeking the same composition by hand renders it correctly at any
offset. So the retry stops after ~15s instead of spinning forever, and the
comment says why an automated check of a background tab will always read 0.

Not covered: the reviewers' other standing finding, that only some items carry
variables at all, so the pages do not have one shape. 125 of the 206 items
tagged as a primitive declare none, and giving them variables means authoring
them into each composition, not editing metadata.

Verified: lint exits 0, the no-deletions gate passes, nav resolves 598 refs with
0 dead, and regenerating three times running changes nothing.

* feat(registry): give 55 primitives variables that actually do something

The catalog pages listed variables for 112 of 454 items and nothing for the
rest, so most pages could show a reader what a piece looks like but not what
they could change about it. This adds them to 55 more, taking the count to 167.

These are not metadata. A variable is only real if the composition reads it, so
each one is declared on the root, validated in the composition's own script, and
wired to something visible: travel distance, blur radius, direction, density,
accent family, tone, label text. Declaring a knob the code ignores would put a
table in the published docs that lies about the piece, which is worse than
having no table.

Every one falls back to its declared default when the incoming value is missing
or unrecognised, so a bad override degrades to the shipped look rather than to a
broken frame. With no overrides at all, each item renders exactly as it did
before: that was checked per item against `git show HEAD:` in a real browser,
comparing computed styles rather than eyeballing.

Four things this ran into that are worth writing down.

An apostrophe anywhere in a description terminates the single-quoted
data-composition-variables attribute and breaks the HTML parse. Every
declaration in the registry now parses; that is checked, not assumed.

Where a timeline drives GSAP's own y/scale/filter, GSAP writes inline styles
that beat any CSS custom property, so those knobs cannot be won from the
composition. Most of these items keep their motion in a user-owned "Timeline
integration" comment rather than in code, so no timing variables were declared
for them at all. A direction knob on a wipe can still be wired honestly, by
remapping clip-path inset sides through multipliers whose defaults reproduce the
original exactly.

Colour tokens that only reach a :focus-visible outline, or an element sitting at
opacity 0 at rest, render identically in a video. Those were skipped rather than
shipped as knobs that appear to do nothing.

CSS shorthand defaults need care: `border: var(--x, 0 solid transparent)` moves
computed border-color off currentColor even at zero width. Defaults were chosen
to reproduce the original computed style, not merely to look equivalent.

Not covered: 72 primitives still have no variables, and the UI-primitive demos
that scripts/sync-ui-primitives.ts mirrors are now stale for the converted
items. Nothing runs that script in CI today.

Verified: every declaration parses and deep-equals its manifest array, every
declared id is read by the composition, no demo.html changed, and lint exits 0.

* feat(registry): variables for 16 more primitives, and stop the snippet clipping

Takes the count from 167 to 183 of 454. Same contract as the last batch: each
variable is declared on the root, validated in the composition's own script,
and wired to something visible, because a knob the code ignores would put a
table in the published docs that lies about the piece.

The snippet under each Variables table was clipping. Its data-variable-values
payload is one long line and the code block cut it off mid-value, with no wrap
and no scrollbar, so the one line on the page that exists to be copied could not
be read. The fence now carries `wrap`. Worth noting how that survived: the
generated markdown was correct and every mechanical check passed. It only failed
in a browser, which is where it was eventually seen.

Two techniques this round that are worth keeping.

Where an accent has a themed token family, the knob sets a new custom property
consumed by that one surface, with a fallback to the existing token, rather than
overriding the shared accent. Default therefore sets nothing, so an externally
themed accent is not clobbered, and the non-default options still follow the
theme in both light and dark.

Where GSAP owns the property outright and no CSS multiply can win — number-wheel
animates `y` inline — the knob is wired at build time instead: extra revolutions
lengthen the digit strip and move the target, so travel changes while the resting
frame stays identical. That is a real answer rather than a skipped knob.

Motion knobs that multiply a timeline-driven custom property collapse to identity
at rest, so every one of them was verified with that property pinned to a
mid-flight value rather than at t=0, where all options look the same by
construction.

Not covered: 56 primitives still have no variables.

Verified: every declaration parses and deep-equals its manifest array, every
declared id is read by the composition, no demo.html changed, lint exits 0, and
the wrap fix was confirmed on the rendered page rather than in the markdown.

* feat(registry): variables for 21 more primitives

Takes the count from 183 to 204 of 454. Same contract: declared on the root,
validated in the composition script, wired to something visible, defaults
reproducing the pre-edit render exactly.

Three kinds of knob were turned down this round rather than faked, and the
reasons are worth keeping.

A knob that contradicts its own motion. The sheet panel could be moved to the
left, but the recipe drives GSAP x from the right, so the panel would slide in
from the wrong side while the control claimed otherwise.

A knob that needs two defaults. A separator length means width horizontally and
height vertically, so one token would be wrong half the time.

An option that is not an option. Two components were given a green accent that
probed byte-identical to their default, because the theme accent already is that
token. A row in the docs table that does nothing is worse than a missing row, so
it was replaced with one that differs.

Accent knobs set a new property with a fallback to the shared token rather than
overriding it, verified by rendering with an external accent in place and
confirming the default still yields to it. Motion knobs multiply a
timeline-driven property so they never fight the inline styles GSAP writes;
because those collapse to identity at rest, each was checked twice, once at rest
against HEAD and once with the driven property pinned mid-flight.

Verified: every declaration parses and deep-equals its manifest array, every
declared id is read, defaults match HEAD on computed styles and on a pixel hash
of the rendered element, no demo.html changed, lint exits 0, and the
no-deletions gate passes.

* feat(registry): variables for 4 more primitives, and make manifests agree with their compositions

Takes the count to 207 of 454.

Four items carried a different description for their exit variable in
registry-item.json than in their own data-composition-variables. The catalog page
renders the manifest, so the published table described the knob one way while the
composition header described it another. The composition wins: it is the file
that implements the variable and the declaration is what the runtime reads.

The skill docs no longer describe a hosted tier, since the CLI now ships the two
local tiers only, and skills-manifest.json is regenerated to match.

* feat(registry): variables for 4 more primitives

Takes the count to 211 of 454.

Two knobs are worth calling out because they touch things the timeline also
touches. skeleton-block slide multiplies the driven row offset, so it is
identity at rest and only bends the middle of the move. slider value sets the
resting fill together with the readout text, aria-valuenow and aria-valuetext,
so all three agree; a composition that tweens the fill takes over from there and
owns the readout, which the comment header states plainly rather than hiding. A
multiplier was rejected there because a 0 to 1 tween would push the fill past
the end of the track.

Knobs on elements that sit at opacity 0 at rest were kept only where the shipped
recipe reveals them, and verified with the reveal forced on as well as at rest.
Skipped: an aria-label string knob that never renders, and an accent token
declared in one item CSS that nothing consumes.

* refactor(registry): drop the UI primitives, this is a video catalog

Removes 66 items tagged ui-primitive: accordions, buttons, inputs, dialogs, a
calendar. They are a shadcn-style interface component set that happens to be
expressible as HTML. None of them animate anything, so in a catalog whose job is
to offer moves for video they widen the surface without making it more useful,
and each one is a page a reader has to skip past to reach something that moves.

Every one is new on this branch. None exists on main, so nothing main ships is
being taken away; that was confirmed against origin/main before deleting rather
than assumed, and the no-deletions gate still passes.

Removed with them: registry/ui-primitives, the Operator Black token and contract
files only these items consumed, and the tooling that maintained them
(sync-ui-primitives.ts and scripts/lib/ui-primitives). No other registry item
declares a dependency on any of the 66, so nothing else loses a piece. The now
empty UI Primitives navigation rule goes too.

Generated output is pruned with the sources. The page generator writes files but
never removes ones whose source has gone, so a stale page would have survived and
404d its own preview. Verified: 0 orphan pages, 0 orphan previews, and the
navigation resolves 532 references with none dead.

This does discard variables authored for 54 of them earlier on this branch. That
work is in the history if these ever come back, and it is the right trade: they
should not have been in a video catalog to begin with.

The catalog is now 388 items. Lint exits 0 and 2522 CLI tests pass.

* feat(registry): every motion and transition primitive is now customisable

The last 16 primitives get variables, so none is left without them. 173 of 388
items now declare variables; the rest are blocks and showcases, which are whole
scenes rather than parameterised moves.

Same contract throughout: declared on the root, validated in the composition
script, wired to something visible, and falling back to the declared default on
missing or unrecognised input. With no overrides every item renders exactly as
it did before, verified per item against the pre-change render in a real browser
at rest and at pinned mid-flight states, comparing computed styles and rects and
in most cases a screenshot hash.

This round refused several knobs rather than shipping ones that only look real.

A tilt-card depth knob was written, measured, and thrown away: the card sets
overflow hidden, which forces transform-style flat, so the authored translateZ
is already inert and every option probed identical. It ships a glow knob
instead, which drives inset and visibly changes at rest and under the drift.

slot-machine-roll has no free travel knob because the roll is exactly one row
height and any multiplier lands the reel off-register; size scales row height
and roll distance together, which is the only honest version. soft-blur-in
offers up and down but not left and right, because the shipped tween resets y
and not x, so a horizontal offset would never animate away.

Two pre-existing bugs surfaced while checking honestly, both left alone as out
of scope but worth recording. zoom-through-transition and tracking-in each tween
a custom property that is never set, so CSS reads it as zero and the move starts
from zero rather than from its authored value. The depth and tracking knobs are
scoped around that and their headers say so, rather than pretending the tween is
what it appears to be.

Verified: 388 items, 0 primitives without variables, 0 items where the manifest
and the composition disagree, 0 declared-but-unread variables, nav resolves 532
references with none dead, no demo.html changed, lint exits 0, and the
no-deletions gate passes.

* feat(registry): raise the catalog quality bar, and add eight primitives

Cuts 37 components, adds 8, and writes down the standard both decisions were
made against.

The 37 removals are all new on this branch and absent from main, so nothing
shipped is withdrawn. Each was audited with two pieces of evidence: source
identity after name normalisation, and a composition-level contact sheet
showing the members animate identically.

The largest group was 13 files byte-identical apart from an h3 and one
sentence. Nothing marqueed, panned, zoomed, deployed or dragged. An honesty
tiebreak decided survivors: frosted-glass-wipe has no backdrop-filter,
spring-scale-in has no spring, masked-slide-reveal has no mask,
short-slide-right travels up, and three-particle-ribbon differs from
three-orbiting-cards by one number while having neither particles nor a
ribbon.

Two independent audits agreed 10 out of 10 on a shared calibration sample,
in both directions, including three items a first pass wrongly condemned.

The rubric is the durable part. Fatal criteria are separated from fixable
ones, because no-timeline alone hits 97 items including some of the best;
promoting it would have cut 97 and left a worse catalog. It also records the
harness rules that make a verdict reproducible: render from the composition
rather than the demo, since demos carry content the installed item does not,
and mount sub-compositions rather than inlining them, since inlining renders
black frames indistinguishable from a dead item.

The eight additions target measured gaps. Camera language ranked first
because PSNR across 30 reference demos showed the most impressive
environments barely move: they are sets, not shots.

camera-shake carries nine lens-accurate profiles, amplitude scaled by focal
length so a wide lens shakes differently from a telephoto. rack-focus splats
each light through the aperture shape, so a defocused point becomes an image
of the iris, with flux conserved so highlights survive defocus.
camera-dolly-zoom solves focal length from distance, holding subject size to
0.000 percent drift while the background grows 53 percent. Plus
oscilloscope-trace with history-free phosphor persistence, bar-chart-race,
split-flap-board, spiral-galaxy and vfx-anamorphic-flare.

Each is verified by rendered frames and a seek-equals-playback check rather
than by check passing, which is not a visual gate.

* fix(registry): let the split-flap board finish flipping on screen

The board declared 8s but every flap had settled by 3.5s, so more than half the composition was a still frame and the catalog preview opened on it.

* fix(registry): keep the thread-message-stack payload parseable

A line wrap had put literal newlines inside the JSON string literals of the blocks data-hf-primitive-data payload, so JSON.parse threw in the browser and the composition never ran. The preview script hid it: it rewrote the payload in the temporary copy it captured from, so the catalog picture looked right while every installed copy stayed broken. That repair pass is gone and the payload is fixed where it ships.

Its two tests could not have caught this. Both were written against vitest in a directory the repo runs with node:test, so neither was in test:scripts and neither had ever run. They are converted and registered, along with a new one that JSON.parses every payload in the registry, and that one was checked against the re-wrapped shape before being kept.

* fix(registry): close the apostrophe that truncated a variables declaration

chromatic-aberration-wipe described its accent as "the incoming scene's gradient" inside a single-quoted data-composition-variables attribute, so the attribute ended mid-JSON and the tag never closed. The formatter refused to parse the file, which is how it surfaced, but the runtime would have read a truncated declaration.

Also formats the 159 registry and docs files the branch had left unformatted, regenerates the skills manifest, and drops docs/primitives: those 13 pages import /snippets/PrimitivePlayer.jsx and read docs/public/primitives/, neither of which is on this branch, so mint failed the build on them. Nothing links to them and they ship whole on feat-video-primitives.

CI ran test:scripts before building core, so the preview test added here failed on a missing dist rather than on anything it checks. It now runs after the builds.

* fix(registry): make the review findings real fixes

Ten items declared variables on their composition root but had no variables key in the manifest, so their generated pages shipped no explorer at all. Their manifests now mirror the root. Two more disagreed only in description text, and the root was the truthful side: both compositions paint an inset ring, not the slabs or colour pair the manifest described.

The caption <video> removal left 24 CSS rules addressing elements that no longer exist. Removed, excluding the four ids that were already orphaned on main.

thread-message-stack could not stay fixed: oxfmt reflows a divs contents and lands a newline inside a JSON string literal, so the payload broke again on the next format. A script tag is not an option because the runtime strips every script out of the mounted clone. The reader normalizes HTML whitespace instead, which is what makes it survive any reflow, and the guard test now asserts that contract rather than the byte layout.

downloadFile had lost its 30s timeout, DownloadOptions, and the mid-pipeline error plumbing in a rewrite that was only meant to fix redirects. Five callers were left with no stall guard. Restored, redirect handling kept.

warnUnknownEnumValues re-did the parse readDeclaredDefaults had already done. Both now share one readDeclarations, and the rest splits into compositionLabel, declaredOptions and unknownEnumValue. 1909 core tests unchanged.

Deletes build-qa-gallery, theme-gate and generate-primitive-pages: nothing invokes them, two read a coverage map four directories above the repo root, and the pages the third generates are no longer on this branch. Wires check-no-main-deletions, which is the opposite case, real and tested and never run.

* fix(registry): stop shipping a stale copy of the catalog-search work

This branch carried re-authored copies of the CLI search commits rather than the ones on their own PR, so merging it would have rolled back six later fixes: the vector cache that refuses a half download, the 0o700/0o600 modes, the rebuilt-from-registry index generator, the coverage gate and its CI job, and the scripts typecheck. Those files now come from that branch.

registry.json still listed 64 items whose directories the UI-primitive removal deleted, so hyperframes add would resolve a name and then fail on missing files. Regenerated from disk: 358 searchable items, 358 vectors, gate green.

Also drops an internal provenance block from thread-message-stack, along with the type and the two JSON schemas that existed only to describe it. It published an artifact id, a version id and a heygenverse:// URI, none of which mean anything to someone installing a block, and a public registry is the wrong place for them.

Typechecking scripts/ for the first time surfaced 45 errors in this branch. Fixed rather than suppressed: the geometry test reads positions through one accessor that names a missing index instead of letting NaN reach a tolerance compare, and the null-returning shape helper is asserted at its call sites, except in the test whose subject is the null.

* refactor(scripts): split the page builder into its numbered sections

generateItemMdx had grown to cyclomatic 26 across 196 lines while its own comments already named the seams. previewSection, usageSection and footerSection now own one each, taking it to 13. Regenerating all 358 pages afterwards produces a byte-identical tree, which is the check that matters for a generator.

* fix(cli): repair what the cross-branch file take broke

Taking files wholesale from the catalog-search branch reverted the downloadFile timeout restored one commit earlier, so five callers were back to no stall guard at HEAD. Restored on both branches this time, since that branch never had it either.

It also took that branch test:scripts line without the vitest it depends on, so the script exited 127 and the CI Test job would have failed on a missing binary rather than a test. vitest is a root devDependency now, and the run is scoped to scripts/catalog/ with the slash: without it the prefix also matched catalog-preview-temp.test.ts, a node:test file with no vitest suite in it. Both branches had that one.

Four registry items and their docs copies carried absolute paths from a working directory. A public registry is the wrong place for them and history is permanent, so the sentences now name the source without the path.

Skill docs came from before the code they describe: the catalog command reports unindexed and applies installability after ranking, and both SKILL.md files now say so.

Also drops a double type assertion and ten dead ?? NaN coalesces from the geometry test, the second of which reintroduced exactly the NaN-into-a-tolerance-compare that the checked accessor exists to prevent.

* fix(ci): resolve core from source and take only item directories

The scripts typecheck failed on generate-registry-items importing @hyperframes/core by package name. It resolves on a machine with a warm node_modules, which is why it passed locally, and not in CI. Every other script in the directory already imports core from source and says why in a comment.

The preview job derived its item list with a sed that needs a trailing slash, so registry/components/CATALOG.md never matched, survived as a full path, and was handed to the renderer as an item name. The grep now requires a directory component. Simulated against this PR: 219 items, none of them a path.

* refactor(registry): load gsap from the cdn like every other item

store-badge-lockup vendored gsap 3.14.2 as a 4,200 line minified file and installed it into the users project, while 540 other items load that exact version from jsDelivr. Repointed, the copy deleted and the manifest entry with it, so hyperframes add store-badge-lockup no longer writes a second copy of gsap into someone elses compositions directory. Re-rendered and re-generated: the preview still draws.

* feat(registry): swap in the detailed device models

Replaces the iPhone and MacBook models in the three device blocks. The old assets were untitled meshes with no keyboard on the laptop; these name every part and model the keycaps, speaker grilles, antenna bands and camera plateau.

Not a drop-in. The compositions found the screen by side effects, the material that happened to carry an emissiveMap for the phone and a mesh literally called matte for the laptop, and neither exists now. They select front-glass and display instead.

Both panels ship UVs authored for a tiling material, the laptop runs u -6.3 to 6.3, so one screen image clamped and smeared across the panel. Planar UVs are derived from each panel bounds at load.

The old phone display sat at the model minimum Z and the timeline spins assume a screen facing -Z. These face +Z, so the model is aligned by reading which of its own parts is front rather than re-timing the animation.

Removes the hand-drawn Apple logo from two blocks: the replacement ships apple-logo meshes, and the drawn one used coordinates read off the old lid, so it floated beside the device.

The preview copy only took top level files, so models/, lib/ and assets/ never reached docs/public and 38 items rendered there without their assets. That is the source of the non-blocking 404s in the preview job. It recurses now, which also brings vendored bundles across, so the generated tree is out of the lint scope.

The html-in-canvas notice is a Danger callout: without the flag the preview is a black rectangle, which is a prerequisite rather than a caveat.

* chore(registry): rebase onto the merged catalog search

This branch carried its own copy of the catalog-search work so that merging it in either order could not regress the other. That copy is now the older one: main has the consent fix, the contributor path for someone without the embedding model, the restored download test and the corrected gate message. Every file main owns is taken from main, and the three duplicated CLI commits are dropped rather than replayed.

Regenerated afterwards, because the registry it describes has changed: registry.json, the vector index, the catalog pages and the skills manifest.

* fix(scripts): stop the rebase reverting the preview pipeline

Resolving the rebase in favour of this branch took three files whose newer versions had already merged, so the branch quietly reverted them.

generate-catalog-previews.ts lost encodeForWeb, which exists because publishing masters directly put 25 Mbps files on the docs CDN and one 20-second preview was 60 MB. It also lost the ffmpeg transcode, so a jpeg capture was being written straight to a .png path while the comment above still said it transcoded, and it lost openOpaqueCapture, re-creating the second copy of a capture setup that was extracted precisely to stop there being two. This PR renders previews for over 200 items, so all three shipped at scale.

scripts/tsconfig.json regained exclusions that hole the gate, and generate-template-previews.ts went back to importing the producer by package name, which is the CI failure that import was changed to fix.

All three are taken from main. Also drops an alignScreenToMinusZ copied into the laptop block, which has no front and back to compare and never called it, and makes the preview copy lstat so a symlinked directory cannot send it outside the repo.

* fix(registry): clear the five items this PR added that the linter rejects

The registry linter is not wired into CI, so five items this PR adds were shipping with real render defects nobody would have seen fail.

caption-camera-follow and grade-split-reveal styled their root by its own class. Sub-composition CSS is scoped to [data-composition-id=...] <selector>, so a selector whose leftmost part is the root class becomes a descendant selector and stops matching the root: the scene renders unstyled at render time while looking correct in every static check and in preview. Both now key off the attribute the scoper already adds.

logo-brand-close tweened letterSpacing, which the browser snaps to integer device pixels, so the ease-out tail stutters under seek-by-frame capture. It is a scaleX now.

terminal-simulator named SFMono-Regular, which the renderer cannot resolve, so the text silently fell back.

oversized-cursor was a false positive: the rule scans raw source for head tags and a literal one written inside a JS comment paired with the real closing tag. Confirmed against a render, nothing leaks into frame, so the comment says head element rather than the tag.

Also stops generate-registry-items.ts dropping catalogArtifact.revision. build-local-vectors.ts stamps it so the CLI and the coverage gate can tell whether the published vectors still describe this registry; regenerating the item list erased it, and the gate then failed until someone rebuilt the index.

41 items still fail the linter, every one of them pre-existing on main.
2026-08-10 14:26:55 -04:00
Miguel Ángel 68205dbbc1 feat(cli): search the catalog by meaning, on this machine (#3089)
* feat(cli): search the catalog by meaning, in three named tiers

Browsing the registry means matching names and tags, which fails whenever the
author's wording differs from yours. "make the pace feel faster" finds nothing
when the move is described as "velocity-driven blur". This ranks by meaning
instead.

Three tiers, and the command always says which one answered:

  words       shared vocabulary, free, offline, no account
  on-device   bge-small, free, offline, one opt-in download
  hosted      Gemini, free for signed-in HeyGen users

The tier is stated because a quietly worse answer looks exactly like a good
one. --json carries it as a token alongside dropped, shown, total and
top_score, so an agent reads provenance as data rather than matching English
that is written to be reworded.

Two consents, asked once each, and never conflated. Sending a query is a
privacy question, so the prompt says the query is sent. Downloading a model is
a disk and bandwidth question, so that prompt talks about size. Neither fires
without a terminal: an unattended run sends nothing and downloads nothing
unless a flag records that a person agreed.

The catalog is derived from registry-item.json rather than from a separate
document, so the set that is ranked and the set that can be installed are the
same object by construction. Only the on-device vectors are committed; the
hosted vectors are nine megabytes and belong on the server.

top_score is reported and never acted on. A "nothing matched" threshold looked
clean on long briefs and collapsed on the short queries people type: "a logo
appears" scores 0.6181 and keyboard mash scores 0.6417, so any cut that catches
the noise rejects the real query. The measurement is in the evals directory
rather than in this branch.

Not covered here. The published recall figures were measured against a separate
hand-written document, not against registry text, so they should not be quoted
for this catalog until re-measured. The offline tier needs a normal install: a
single-file build cannot load the native ONNX runtime, which the command now
reports instead of silently degrading. And the drop-detection path has never
been observed firing outside its author's tests.

* fix(cli): make this branch pass the repo's own gates

Three things `bun run lint` and `fallow audit --base origin/main` rejected.
CI runs both, so none of this branch would have gone green. Found by running
them, not by reading the diff.

process.exit in catalog.ts, twice: an invalid --type and a cancelled picker.
check:cli-process-ownership reserves that for cli.ts, and the rule is not
cosmetic — process.exit tears the process down where it stands, so anything
cli.ts has queued to run on the way out is dropped. finishCommand throws a
CliResultSignal that cli.ts turns into the exit code, which is what init.ts
already does for a cancelled prompt.

Three exports with no consumers. normalize keeps its body and loses its export;
localEmbedder is the only caller. modelsDirectory goes entirely, having no
caller inside its file or out. The WordPieceConfig re-export goes, and with it
the import it existed to forward: the type is exported from wordpiece.ts, where
its consumers already take it from.

Complexity. prepareOnDeviceTier is lifted out of run(), which took run from 64
cyclomatic and CRAP 948 to 54 and 684. That block is one decision — can the
offline tier run, and if not, why not — and its only product is a list of
warnings, so it reads and tests as a unit, which it could not do inline.

The rest is suppressed rather than refactored, each with its reason on the line
above. Finishing run() means extracting its three output paths, and that is a
refactor of a command this branch already changes for other reasons: a separate
initiative, not something to absorb here. Every suppression says what shape the
function has and why; a bare marker on a function nobody can justify is how a
threshold stops meaning anything.

Verified: `bun run lint` exits 0, fallow reports no issues across 27 changed
files, and 2540 CLI tests pass.

* feat(cli): ship the local search tiers only, drop the hosted one

Search now has two tiers, both local: shared-vocabulary word matching, and the
opt-in on-device model. The hosted tier, which sent the query to a HeyGen
endpoint and ranked it with a hosted model, is removed.

This is a scope decision, not a defect. The endpoint works and its own change is
reviewed and green; it is simply not what we want to ship first. Landing local
only means the feature has no backend dependency, no auth requirement, and
nothing leaves the machine unless someone opts into downloading a model.

Gone: registry/smartSearch.ts and its test, the --smart and --no-smart flags,
the outcome plumbing through the command, the remote branch of applySearch, the
remote tier, and the hosted-only JSON fields (ranking, catalog_version,
top_score). Also the smartSearchEnabled consent field in telemetry config, which
was the persisted storage behind the hosted consent and would otherwise have
been left as dead configuration surface.

Kept exactly as they were: both local tiers, the --on-device and --yes flags,
the download consent prompt, and the runtime check that happens before the
download rather than after it. The --json envelope still reports query, tier,
tier_detail, shown, total, dropped, warnings and results, so an agent can still
tell which tier answered and why. tierToken now distinguishes on-device from
words.

Verified: lint exits 0, fallow reports no issues, 2522 CLI tests pass, and the
command was exercised directly. A query answers on the on-device tier where the
model is installed and falls back to word matching where it is not, reporting
that fallback in warnings rather than silently. An unknown --type still exits 1
with a readable message, and --smart is now rejected as an unknown flag.

* fix(cli): count only moves this registry cannot install as dropped

The dropped count was computed against the list left after the user's own
--type and --tag filters, so every move the user excluded was reported as one
the registry is missing. Filtering made the number go up: the same query
reported 277 unfiltered and 302 with --type block.

The count exists so a caller can tell "nothing matched your words" apart from
"the ranker suggested things this project cannot install". Conflating it with
user filtering destroys exactly that signal, and worse, genuine index skew and a
self-inflicted filter printed a byte-identical line with opposite remedies --
one means refresh the shelf, the other means drop a flag, and refreshing does
nothing.

Now counted against the registry rather than the filtered view. The manifest is
already fetched whole and narrowed in memory, so keeping the unnarrowed name set
costs no extra request, and item loading still runs only on the filtered subset.

Verified against ground truth rather than by eye: the vector artifact holds 411
names, the registry holds 168 installable items, and 134 of those names exist in
both, so 277 are genuinely uninstallable. The count now reads 277 unfiltered,
277 under --type block, 277 under --type component and 277 under --tag, and the
skew it reports is real -- the artifact predates dropping the UI primitives and
still ranks moves that are no longer on the shelf.

Reported by Vance Ingalls, who also noted this closes an item the status doc
listed as unverified. Two earlier sweeps could not make the count fire because
neither combined a filter with a query.

Tests pin the three cases: a genuinely absent name counts, a filter-excluded
name does not, and a fully installable ranking reports zero.

* fix(cli): tell the user when meaning search cannot see the catalog

The on-device index was fetched once and never revalidated: the only
freshness check was two existsSync calls. A move added after that fetch was
invisible to meaning search permanently, not down-ranked but absent from the
candidate set. The registry manifest on the same command carries a 24h TTL,
so the two halves of one feature disagreed about staleness.

The dropped count reported over-coverage only, names the index has that the
registry lacks. Under-coverage was never computed, so the harmless direction
was instrumented and the costly one was silent. Reproduced with an index
truncated to 120 of 168 moves: dropped read 0, perfect health, while 48
moves were unreachable.

Counts under-coverage from the name list the artifact already carries, so no
extra request. Warns only when non-zero, and names the remedy.

The remedy had to be made true: --on-device could not refresh a stale index
because hasLocalVectors short-circuited the fetch. That flag now refetches
when the index is absent or no longer covering.

Two defects the reproduction surfaced. A failed refresh reported the tier
unavailable while the old vectors were still on disk and still ranking. And
the fetch wrote its two files one at a time, so failing between them paired
a new name list with an old matrix, a hard load error rather than stale
data. It now writes both or neither, which matters more once refresh runs on
staleness.

top_score returns, scoped to the on-device tier and set to the score of the
best result actually shown rather than the ranking head, which can describe
a row the caller never received.

Also: scripts/ is now typechecked. It never was, which is how a build script
that crashes after the paid embedding call, and two scripts whose imports do
not resolve at all, went unnoticed. 43 errors fixed, no suppressions.

And the docs stop describing a --smart hosted tier that was deleted, an
item that does not exist, and a registry refresh that cannot fix a stale
vector index.

* ci: fail when the search index stops covering the registry

The catalog vector artifact is regenerated by hand. Nothing in CI, in
package.json or in a hook rebuilds it, because embedding needs the 32 MB
model. So adding a registry item silently makes it invisible to meaning
search until someone remembers to regenerate.

The failure is asymmetric, which is what makes it easy to miss. Removing an
item is self-healing: the ranker still scores the dead vector, then filters
the name before display, so a user is never offered something they cannot
install. Adding one is not: the item is absent from the candidate set
entirely, not ranked low.

Comparing the two name lists needs neither the model nor a network call, so
the gate runs in seconds. CI checks rather than fixes, for the same reason
it cannot regenerate.

Scoped to blocks and components. Examples are starter projects a user
scaffolds, never something catalog ranks, and the artifact carries no vector
for them, so demanding one would keep this gate permanently red and it would
be ignored within a week.

Verified in both directions rather than assumed: adding an unindexed item
exits 1 and names it, restoring the registry exits 0.

* fix(catalog): rebuild the search index from the registry

build-local-vectors.ts read registry/catalog-artifact/catalog.json, a file no script in this repo writes and which is not committed, so the documented regeneration command failed on a missing path. That is why the index could drift from the registry with nothing to run to fix it.

It now reads registry/blocks/* and registry/components/* through catalogFromRegistry, the existing helper that already produced the right shape but had no caller. Rebuilding reproduces the shipped 168 rows byte for byte.

A lefthook catalog-index command regenerates and re-stages both artifact files whenever a staged registry-item.json changes, mirroring the skills-manifest pattern, so adding or removing an item keeps the index in sync without anyone remembering to. Verified end to end: staging a new item took the artifact 168 to 169 rows and staged it in 0.80s.

* fix(cli): refuse a half-downloaded vector cache

The two artifact files have to agree on how many rows there are, and until now nothing checked that before writing them. A truncated or wrong-model response landed in the cache and only failed at load, on every later search, until someone cleared it by hand. The pair is now checked first and refused as a unit, and the cache is created 0o700 with 0o600 files rather than inheriting the umask of a directory the caller may have pointed anywhere.

Also lifts the capture setup the two preview generators had drifted into sharing into scripts/preview-capture.ts, and splits the vector builders batching and packing out of main. Both were findings the audit attributed to this branch.

* fix(cli): keep the catalog vitest run with the tests it runs

Restacking took the base package.json wholesale, which dropped the vitest dependency and the scripts/catalog run this PR adds. Both belong here rather than under it.

* fix(cli): stop the declined model download from happening anyway

Answering no to the on-device download offer recorded no and warned, then carried on. The guard below it is localModelConsent() !== false, which the decline had just made false, so it was skipped rather than taken: control reached recordLocalModelConsent(true), overwrote the answer with yes, and fetched the 32 MB model the user had refused. Next run it never asked again.

No test could catch it. The stub pinned localModelStatus to ready, so the prompt never fired, and recordLocalModelConsent was a no-op that recorded nothing.

Two tests now cover the offer, and they need three things the old stubs did not model: the run has to look like a terminal, because off one the command treats --on-device as the consent and never asks; the ONNX probe has to answer true, or an accepted offer returns at the runtime guard before it can download; and the status has to follow the recorded answer, or the second offer later in the run fires as well. Removing the return makes the decline test fail.

* fix(catalog): let someone without the model still add a component

The pre-commit hook rebuilds the search index, and rebuilding needs the 32 MB embedding model. An outside contributor adding a registry item does not have it, so their commit died inside the ONNX loader on an ENOENT naming a path they never set, and the CI gate then told them to run the command that had just crashed.

The model is an opt-in for search, not a build dependency, so nobody is charged for it to contribute. The builder checks first and explains itself, exiting 3 for cannot as distinct from 1 for failed. The hook treats 3 as skip and lets the commit through. The gate now names both paths: regenerate if you have the model, leave it if you do not and a maintainer will.

Verified both ways: with no model the builder explains and the hook exits 0; with the model it still regenerates byte-identically.

* docs: say that anyone can add a registry item, and stop hand-editing a generated file

Two defects, one of them the reason 64 stale entries survived in registry.json.

The checklist told contributors to add their item to registry/registry.json. That file is generated from the item directories, so an entry added by hand survives until the next regeneration and then vanishes, and one left behind for a directory that no longer exists is worse: hyperframes add resolves the name and then fails on missing files. Both CONTRIBUTING.md and the agent-facing skill reference now run the generator instead.

Nothing said contribution was maintainer-only, but nothing said it was not either, and two steps do need assets an outside contributor has no reason to install. Those are now named in a table with what happens if you do not have them, matching how the preview image was already handled. The search index is the new one: the model behind it is a 32 MB opt-in for search, not a build dependency.

* fix(cli): harden on-device catalog search

* fix(cli): refresh stale catalog vectors

* test: create catalog vector temp dirs securely
2026-08-09 22:59:15 -07:00
Miguel Ángel f28bc80a1d feat(scripts): fail a branch that deletes files main still ships (#3150)
* feat(scripts): fail a branch that deletes files main still ships

Written after a scare that turned out to be a measurement error, and the error is the reason it exists. Comparing tip to tip on a branch a month behind reports every file main has added since the merge base as a deletion: 1,284 of them, an entire skills tree among them, none of it real. A merge keeps mains side and a pull request shows the three-dot diff, which reported zero.

So the gate uses the three-dot form and reports renames separately, because in a name-only diff a rename is indistinguishable from a deletion and treating them alike would either mask real loss or block every legitimate move.

* ci: enforce the no-deletions guard
2026-08-09 22:24:42 -07:00
Miguel Ángel 79dff20516 feat(scripts): typecheck the scripts directory (#3149)
* feat(scripts): typecheck the scripts directory

scripts/ was the one TypeScript surface nothing typechecked. Adding a project for it surfaced real errors rather than style: a preview generator passing string | undefined where a string was required, a readdir result indexed without a bound, and two non-null assertions standing in for a filter that could have narrowed the type instead.

The two preview generators had also drifted into sharing a capture setup, down to the comment explaining why the capture is opaque. That lifts into scripts/preview-capture.ts, so the reason is written once and both callers own the handles they have to close.

@hyperframes/core and @hyperframes/producer become dev dependencies because the scripts import them; without that the project resolves on a machine with a warm node_modules and fails in CI.

* fix(scripts): use source imports consistently
2026-08-09 22:07:57 -07:00
Miguel Ángel 0bda6b55b8 feat(cli): track which registry items add installs (#3099)
* feat(cli): track which registry items `add` installs

`cli_command` records that `add` ran and nothing about what it installed, and
the registry is served from raw.githubusercontent.com, which gives no per-item
counter either — so there is no way to tell which block or component people
actually pull, and no way to know what is worth building more of.

Emit one `registry_item_added` event per item written into a project, from
`runAdd` after the install succeeds. That is the single choke point: the bulk
`add <tag>` path re-enters it per item, and a failed or compatibility-refused
install throws before it, so a refused install is never counted as a download.

`requested` separates the item the user named from the transitive
`registryDependencies` pulled in behind it; without it a popular dependency
outranks everything that depends on it.

Item names are public registry identifiers, never user content or project data,
and the event goes through `trackEvent` — an install that opted out via
`hyperframes telemetry disable`, `HYPERFRAMES_NO_TELEMETRY` or `DO_NOT_TRACK`
sends nothing.

* test(cli): cover `add` telemetry end to end against the built CLI

The unit tests assert the emit seam and nothing past it. `shouldTrack()`
short-circuits whenever `isDevMode()` is true, and that is true for any `.ts`
entry, so under vitest a real event and no event are indistinguishable and the
transport is never exercised at all.

Drive the built CLI instead and assert on the HTTP body it actually produces:
one event per installed item, the dependency reported with `requested: false`,
an opted-out install sending no request at all (not merely one without this
event), and a refused install counting nothing.

Two fixtures, because neither case is reachable through the real registry. The
registry origin is a first-class project setting, so a local one supplies the
`registryDependencies` edge that no shipped catalog item declares today; and
`globalThis.fetch` is wrapped to capture the batch rather than send it. The
faked 200 is load-bearing: only a failed flush leaves events queued, and only a
non-empty queue spawns the detached `flushSync` child that would bypass the
hook and reach production analytics.

Verified the check can fail — forcing `requested: true` for every item turns it
red on exactly the dependency assertion.
2026-08-07 16:00:23 -07:00
Miguel Ángel 218eff7d36 fix(scripts): render template-only blocks in catalog previews (#3098)
* fix(scripts): render template-only blocks in catalog previews

The catalog preview renderer treated any file containing `__timelines` as a
standalone composition and rendered it as index.html directly. The 12 VS Code
snippet blocks register their timeline inside a `<template>`, which stays
inert until a host mounts it, so every one of them failed with "Composition
has zero duration" and no preview could be produced from the registry at all.
Six of the previews on the docs CDN were hand-made from a project still
mounting Monokai, so Dark+, High Contrast, High Contrast Light, Solarized
Light, Visual Studio Dark and Visual Studio Light all showed Monokai's video.

Detect standalone-ness on the document with template content stripped, mount
the mirrored install-layout copy so a block's own `../assets/*` references
resolve, and capture posters opaque: `format: "png"` is the engine's
transparent mode and forces `background-image: none` on every composition
root, which erased the desktop backdrop these blocks paint.

Publishing gets the missing half too: preview URLs are stable and the objects
are uploaded `immutable` with a one-year max-age, so a re-upload alone never
reaches a reader.

* fix(scripts): install ffmpeg in the preview job and fix the sibling renderer

The canary this PR added caught its own regression: the poster transcode
shells out to ffmpeg, which ubuntu-latest does not ship and this job never
needed, so both canaries failed with `spawnSync ffmpeg ENOENT`. Install it
the way every other render job does. `encodeForWeb` has always shelled out to
the same binary; the job only got away with it because `--skip-video` skipped
that path.

generate-template-previews.ts captures posters through the same transparent
`format: "png"` mode, so any template painting its own backdrop loses it
exactly as the code snippets did. Fixing one renderer and leaving its sibling
on the broken call would just move the bug.

Also fold the three separate parses of registry-item.json into one read: they
had drifted into three different failure behaviours for the same file.
2026-08-07 15:17:30 -07:00
Miguel Ángel ebdd1893c4 fix(studio): reconcile external edits before reload (#2993)
* fix(studio): reconcile external edits before reload

* fix(ci): retry transient workspace installs

Make external reload retry behavior honest and isolate reload listeners.

Remove the dead SDK timestamp parameter.
2026-08-04 23:13:19 +00:00
ukimsanov 4ef1511b19 ci: re-run catalog previews when the containment module changes
Rames' non-blocking note on #2975. The paths filter listed the renderer but not
scripts/registry-target-paths.mjs, which it imports — so a future change to the
path-traversal defence alone would never re-run the only job that exercises it.

That is the same shape as the bug the module exists to prevent: the check is
present, the thing that would catch a regression in it is not wired to run.
2026-08-04 03:06:32 -07:00
ukimsanov 8b98b41eed fix(docs): keep the changelog and weekly archive reachable
Rames' review on #2978. Two pages left the sidebar without a redirect and
without being deleted, so they survived only as direct URLs: `docs/changelog.mdx`
and `docs/weekly-updates.mdx`.

Not deliberate, and the stack says so — #2979 upgrades `weekly-updates.mdx`,
importing DocsVideo and converting four raw <video> tags. You do not invest in
a page you meant to retire, and it carries `rss: true`, so it is a subscribable
feed. `product-updates.mdx`, which this stack adds to both the nav and the
footer, links to `/changelog` three times and `/weekly-updates` once. One of
those is advice to read the release archive before upgrading a production
workflow.

Both are back in the Explore group next to Product updates, which is where a
reader looking for "what changed" would go.

Worth naming why the verification missed it: the checker walks navigation → file,
which is why it correctly reported zero dangling entries. The file → navigation
direction — a page that exists, is not in the sidebar, and has no redirect — was
never checked, and that is exactly where these two sat.

`--check-redirects` on the existing `mint broken-links` step closes the adjacent
gap: it resolves every redirect destination, so a future restructure cannot
leave a redirect pointing at a page it removed. It does not catch the orphan
case above.

Also retargets `/guides/pipeline`. It pointed at `/concepts`, which explains how
a project is put together; the retired page was a seven-step process. `/workflows`
is the closer intent. The old step 3, "Strategy & Messaging", has no successor
anywhere in the docs — worth deciding deliberately rather than routing around.
2026-08-04 00:39:31 -07:00
Vance Ingalls 71fd96bbf1 Merge pull request #2854 from heygen-com/feat/canary-rollouts
feat(core): percentage-based canary rollouts + calibration experiment
2026-08-03 22:35:37 -07:00
James Russo 9792c32950 fix(producer): reject asset media type mismatches (#2937)
* fix(producer): reject asset media type mismatches

* fix(engine): document read-only AVIF probe

* fix(engine): bound read-only AVIF brand probe

* fix(producer): make media preflight lifecycle-safe

* fix(producer): reconcile runtime media before preflight

* fix(engine): avoid writable file-open detection

* fix(producer): close runtime media preflight gaps
2026-08-03 18:16:41 -07:00
James Russo d6191965cf fix: pin release publishing to merge commit (#2959) 2026-08-02 21:32:08 -07:00
Vance IngallsandClaude Opus 5 3f8dca165d fix(cli,core): refresh telemetry posture at the render boundary
R6/R7 blockers.

An already-open Studio kept emitting server-side render telemetry after
another process disabled CLI telemetry. refreshTelemetryPosture() only ran
while serving a fresh SPA document and on /api/telemetry-identity, which
Studio has no consumer for, so the render POST and its async outcome used
the posture cached when the preview server booted. It now refreshes at the
render boundary and again immediately before the completion/error event,
so an opt-out during a long render is honoured.

The identity tests were passing vacuously: their mocks omitted
readConfigFresh and resetTelemetryPostureCache, and the resulting
missing-export error was swallowed by the refresh's own catch. Mocked
properly, plus the enabled -> external disable -> next response transition
and the suppression path at the layer that drops the event.

A full reset also did not persist its new lineage in a long-lived process:
syncInstallState returned early on a process-lifetime memo even after
~/.hyperframes was deleted, so install-state was never recreated and the
next config-only re-mint rolled a third seed instead of inheriting the
second. The memo is now revalidated against the file.

Also drops a stale reference to assertNoOverdueCanaries and stops the
workflow and docs claiming the sunset job routes anything to the owner —
it names them in the run log and notifies nobody.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 18:11:45 -07:00
Vance IngallsandClaude Opus 5 cad6b394f4 ci(canary): pin actions and scope the sunset workflow token
CodeQL flagged both on the new workflow: an unscoped GITHUB_TOKEN and an
unpinned third-party action. Matches the pins ci.yml already uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 17:54:55 -07:00
Vance IngallsandClaude Opus 5 6f0df2640b fix(cli,studio,core): close five R5 telemetry and canary findings
- A long-lived preview cached its telemetry posture in two places
  (readConfig and shouldTrack). Running `telemetry disable` in another
  terminal left it resolving canaries and injecting the CLI id for hours.
  Both caches are now dropped together at a request boundary.
- Studio minted and shipped a telemetry id for every render regardless of
  the browser profile's opt-out, and the server emitted the outcome under
  CLI policy, which cannot see localStorage or DNT. The browser now sends
  an explicit telemetryOptOut, distinct from an old client's omission.
- Any non-empty HYPERFRAMES_PREVIEW_HOST disabled the DNS-rebinding guard,
  so even a loopback bind accepted a hostile Host. The guard now holds for
  loopback binds and, on a LAN bind, admits only names this machine
  answers on.
- sunsetAfter had no reader of the current date. A scheduled workflow runs
  scripts/check-canary-sunset.ts weekly, so a failure lands on the
  rollout's owner rather than on an unrelated PR author.
- The install-state seed memo outlived `rm -rf ~/.hyperframes`,
  resurrecting a cleared cohort. Removed; it only saved a read on a
  readConfig cache miss.

Docs updated for the Host rule and the 100% exclusion carve-out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 17:21:41 -07:00
Miguel Ángel c6925e471a feat(studio): enable timeline virtualization by default (#2926)
* feat(studio): enable timeline virtualization by default

* fix(ci): measure timeline performance in production React
2026-07-31 19:43:47 +02:00
Miguel Ángel 723d3381c4 fix(studio): keep dense keyframes readable (#2925)
* perf(studio): define timeline viewport budgets and fixtures

* test(studio): gate timeline viewport performance in Chromium

* refactor(studio): isolate clip drag lifecycle

* refactor(studio): extract timeline render contracts

* perf(studio): centralize timeline viewport geometry

* perf(studio): follow playhead across virtualized rows

* perf(studio): add timeline clip-window index primitive

* perf(studio): virtualize timeline clip windows

* perf(studio): stop timeline scroll work when row virtualization is off

The row virtualization stack made the timeline publish a viewport snapshot
on every scroll frame and swap `renderClipContent` across every mounted clip
at gesture start and settle. Both are windowing concessions, and neither was
gated on the flag, so the build users actually run paid for them while
mounting all 1,000 clips anyway. Measured on a 3,000-clip project: median
scroll step 16.6ms to 76.9ms, p95 17.9ms to 189.4ms, 40 long tasks to 247.

Gate both on the row virtualization flag. The scroll path now stops at the
door when the flag is off, so `isScrolling` stays false and resize-driven
and programmatic syncs still publish through the immediate path.

The flag moves into its own module: the scroll-viewport hook needs to read
it, and the virtualization hook already imports the viewport snapshot type
back, which would have closed an import cycle.

Also release the perf fixture lease from the fixture rather than from the
test-hook effect. Loading a fixture writes player state, which changed that
effect's dependency identities and tore it down on the next frame, so the
lease was revoked moments after it was taken and live iframe discovery
overwrote the fixture before the gate could measure it.

The e2e gate gains a flag-off arm (`test:timeline-default`, 1,000 elements)
next to the existing flag-on one. It refuses the 50,000-element combination,
verifies from the mounted DOM that the server under test matches the
requested flag, and skips the DOM-size budgets for the unvirtualized build
rather than relaxing them, so a skipped budget never reads as a passed one.

Verified against a live Studio dev server on the fixture project:

  flag off, before: interactionP95 303.1ms, longest task 194ms, 0/5 runs pass
  flag off, after:  interactionP95  33.6ms, longest task   0ms, 5/5 runs pass
  flag on,  after:  interactionP95  33.2ms, 4/5 runs pass, exit 0

The flag-on arm's fourth run reproducibly reports a 55-58ms long task
against a 50ms budget. That is the residual tail of the window swap itself,
tracked separately and not addressed here.

* ci(studio): run the timeline viewport gate on studio changes

The gate has existed since the row virtualization stack landed but nothing
under `.github/` referenced it, so it only ever ran when someone ran it by
hand. That is how the flag-off scroll regression reached eight merged-ready
PRs without anything noticing.

Adds a `studio-timeline-viewport` job that boots two Studio dev servers, one
per flag state, and runs both arms of the gate against them. Two servers are
needed because row virtualization is read from `import.meta.env` at module
load, so one process cannot serve both builds.

Scoped to a new `studio` paths filter rather than the broad `code` one: the
gate only says anything about `packages/studio`, `packages/core` and
`packages/studio-server`.

Adds a `ci` tier. It applies the constrained budgets without any emulation,
because a hosted runner is already slower and noisier than the machine the
strict numbers were recorded on, while the existing `low-resource` tier would
throttle it a further 4x and measure the throttle rather than the build.

The fixture composition is tracked under `tests/e2e/fixtures` but Studio
resolves projects from the gitignored `data/projects`, so the job copies it
into place instead of a project directory being committed.

Both arms run in about 7 seconds each locally, so the job cost is almost
entirely dependency install and the workspace build it shares with
`studio-load-smoke`.

* fix(ci): preserve both timeline gate evidence arms

* ci(studio): report timeline gate arm statuses

* ci(studio): require timeline gate evidence artifacts

* fix(studio): keep dense keyframes readable

* fix(ci): resolve timeline stack audit findings
2026-07-31 18:05:22 +02:00
James Russo 3a0590925c perf(ci): run the two heaviest fixtures in distributed mode (#2825)
* perf(ci): run the two heaviest fixtures in distributed mode

* test(ci): pin distributed-mode fixtures to harness support
2026-07-26 22:42:12 -07:00
James Russo f67012eb9f ci(regression): compute the shard matrix from recorded fixture timings (#2815)
* ci(regression): compute the shard matrix from recorded fixture timings

* ci(regression): refresh shard timings from a green post-PSNR run

* fix(ci): close two silent-skip holes in the shard schedule contract

* ci(regression): schedule the new static-volume-future-set fixture

* test(producer): regenerate static-volume-future-set golden in the pinned container
2026-07-26 19:16:39 -07:00
James 2a284a8e3a fix(gcp): enforce effective BeginFrame capture 2026-07-27 00:02:37 +00:00
James RussoandMiao Yang 696cbdbbd0 chore(skills): package Codex plugin upload (#2668)
* chore(skills): package Codex plugin upload

* chore(skills): harden Codex plugin content

* fix(skills): satisfy plugin quality gates

* fix(skills): address plugin packaging review

* fix(plugin): simplify asset validation

* fix(skills): correct embedded-captions catalog count to 35 after nightcity removal

The nightcity theme removal left SKILL.md claiming 36 identities in
four places, including the frontmatter description the router reads.
The catalog now has 35 entries (10 classic + 25 themed).

---------

Co-authored-by: Miao Yang <miao.yang@heygen.com>
2026-07-22 00:41:40 +08:00
Miguel Ángel 6b4df032fe ci: rerun regression checks after PR base edits (#2659) 2026-07-20 14:47:15 +02:00
James RussoandJake Moran e96ebd74de feat(skills): add changelog-video skill for repo-native CC + Codex discovery (#2552)
Packages Jake Moran's changelog-video pipeline (v1, validated end-to-end
by Home on the Jun 23-29 range) as a repo-native skill set that Claude
Code (.claude/skills/) and Codex CLI (.agents/skills/) auto-discover the
moment the repo is opened. No install step; run the skill against a
changelog markdown for a given git range and it produces a lint-clean,
seam-gate-green 1080x1080 MP4 (~45-60s, Annie VO, mock-UI visualizations,
caption rail) end-to-end.

Six skills added byte-identical in both mirror dirs:
- changelog-video (pipeline entry point)
- motion-doctrine (carries seam-stamp.mjs + seam-gate.mjs)
- cut-the-curve, captions-overlay, seam-craft, oversized-cursor

Layout:
- .claude/skills/  - Claude Code project-local auto-discover
- .agents/skills/  - Codex CLI project-local auto-discover (verified via
                     Magi's clean-home Codex 0.144.3 repro; NOT .codex/skills/)

Fonts, animated background (12 MB), house BGM (5 MB), lexicon, and
align-captions ship inside the skill dirs. .gitattributes routes only
.claude/skills/**/*.{mp4,mp3} + .agents/skills/**/*.{mp4,mp3} through
LFS — narrowly scoped so unrelated Player, Studio, registry, and
marketplace media stay put. HeyGen CLI auth is the one credential the
skill needs; Node >= 22, ffmpeg, and headless Chrome are documented
alongside in both READMEs.

.gitignore: rewrites .claude/ and .agents/ blocks to keep agent-installed
skill hygiene while re-including the six repo-native skill dirs plus
README.md.

CI:
- Extends changes.skills filter to match .claude/skills/**,
  .agents/skills/**, scripts/lint-skills.ts, and scripts/check-skill-mirror.mjs.
- New 'Skills: project-native lint + mirror' job runs the extended
  lint-skills.ts (schema-driven; required { name, description } + optional
  { license, allowed-tools, metadata }, name pattern check, description
  length check) plus a new check-skill-mirror.mjs byte-integrity script
  (24 mirrored files must match; README.md deliberately per-CLI).
- Wired into 'bun run lint' locally.

Frontmatter validator:
- Rejects unsupported top-level keys (catches category:-style drift).
- Requires name + description.
- Validates name pattern (^[a-z][a-z0-9-]{0,63}$) and description shape
  (non-empty, <=1024 chars).
- Missing frontmatter block itself is a first-class error.

Also strips unsupported top-level 'category:' frontmatter from Jake's
motion-doctrine and cut-the-curve SKILL.mds (both mirrors), rewrites the
TTS invocation from ~/.claude/skills/media-use/... to the tracked
skills/hyperframes-media/scripts/heygen-tts.mjs, swaps npx hyperframes@latest
for the repo-local CLI in the gate step, and fixes a lint issue in Jake's
seam-gate.mjs (ternary-for-side-effect -> if/else).

Validated end-to-end by Home on Jun 23-29 (MP4 posted in C0ACCNHLG3U
thread 1784181166.041319). Independently reviewed R1/R2/R3 by Magi.

Co-authored-by: Jake Moran <jake@heygen.com>
2026-07-16 17:29:19 -04:00
James 83db364f81 test(repo): enforce workspace contracts 2026-07-11 21:57:45 -07:00
James 0af07a07c5 fix(core): enforce strict runtime safety 2026-07-11 21:31:30 -07:00
James 9e7b11998c test(producer): gate source tests by execution lane 2026-07-11 10:38:02 -07:00
James 585aa9f6b2 chore(repo): forbid tracked generated artifacts 2026-07-11 01:20:22 -07:00
Vance Ingalls e04f6dda37 feat(engine,cli): drawElement fast-capture config + CLI flag (#1916)
## drawElement fast-capture — config + CLI flag (stack 1/6)

Foundation layer for the drawElement fast-capture feature: the config surface and CLI/Docker plumbing that the rest of the stack builds on.

### What this adds
- **`packages/engine/src/config.ts`** — new config fields for fast capture: `useDrawElement` / `enableDrawElementWorkerEncode` (macOS-GPU `drawElementImage` capture + worker-offloaded JPEG encode), resolved from env in `resolveConfig` (env `HF_DE_WORKER_ENCODE`). Wired alongside main's existing `staticFrameDedup` (unified downstream in 4/6).
- **`packages/cli/src/commands/render.ts`** — `--experimental-fast-capture` flag → sets `experimentalFastCapture`; `--debug` passthrough.
- **`packages/cli/src/utils/dockerRunArgs.ts`** — pass the fast-capture env through to the container.
- **`.github/workflows/fast-video-validation.yml`** — CI job validating fast-capture renders.
- `.oxlintrc.json` / `.fallowrc.jsonc` — ignore-pattern housekeeping for the new paths.

### Notes
- Config-only + entrypoint; no capture behavior yet (that's 2/6–4/6).
- Tests: `config.test.ts`, `dockerRunArgs.test.ts` added.

---
**Stack (drawElement fast-capture, rebased onto current `main`, supersedes #1295 + #1444):**
1. **#1916 config + CLI** ← you are here
2. #1917 drawElementImage capture service
3. #1918 3D projection + compositor-effect risk gate
4. #1919 frame-capture core (routing, worker-encode, static-dedup unification)
5. #1920 producer render stages + remote bg-image localizer
6. #1921 lint rule + player media sync

⚠️ Intermediate PRs (1–5) are split by package boundary for review and **do not each compile independently** (cross-file deps); the complete feature is green at the stack tip (#1921) — tsc-clean on engine + producer, 231 tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-06 15:57:30 -07:00
WaterrrForever 8f0bfef757 ci: auto-publish changed skills to ClawHub on push to main (#1835)
* ci: sync changed skills to ClawHub on push to main

Add a GitHub Actions workflow that runs `clawhub sync` whenever skills/**
changes on main, publishing only changed skills to ClawHub
(https://clawhub.ai/heygen-com) under the heygen-com publisher and
auto-bumping the patch version. Unchanged skills are a no-op, so it is
safe to run on every push. Requires the CLAWHUB_TOKEN repo secret.

* ci: use Node 22 to match the CI fleet's LTS

Address review on #1835 (Miga): the rest of the CI fleet (ci.yml,
windows-render, player-perf, preview-regression, docs, catalog-previews)
pins setup-node to Node 22 LTS. Node 24 is current, not LTS, and could
introduce subtle differences. Align this workflow to 22.
2026-07-02 15:15:48 +08:00
Miguel Ángel e73076e93c fix(core): publish runtime inline artifact (#1787) 2026-06-29 14:43:25 -07:00
WaterrrForeverandClaude Opus 4.8 bf630bfe1e fix(cli): always check GitHub skills on init while skills.sh syncs (#1768)
* fix(cli): always check GitHub skills on init while skills.sh syncs

The "don't pass --skip-skills" guidance lives in SKILL.md, which ships
through the laggy skills.sh registry and can't be relied on to reach the
agent — so an agent that improvises `--skip-skills` silently dodges the
GitHub skills freshness pull. Put the guarantee in the CLI instead (the
one channel that updates promptly via `npx hyperframes@latest`):

- Neuter the `--skip-skills` FLAG so it no longer skips the check; gate
  skipping on the HYPERFRAMES_SKIP_SKILLS=1 env var instead (the
  agent/user CLI path never sets it). Print a one-line notice when the
  ignored flag is passed.
- Wire the env escape hatch into the init test helper (one place) and the
  CI smoke-test / windows-canary steps so they stay offline and fast.
- Update the skill docs that previously told agents `--skip-skills` opts
  out.

Temporary measure while skills.sh catches up — revert init.ts's
`skipSkills` to `args["skip-skills"] === true` once it does (noted inline).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): build @hyperframes/lint before core in Test and Studio jobs

The lint extraction (#1756) made @hyperframes/lint a runtime dependency of
core — core's compiled compiler/staticGuard.js imports it via the package's
"node" export condition (./dist/index.js). But the Test and Studio-load-smoke
jobs pre-build only @hyperframes/{parsers,studio-server} before packages/core,
so loading core's dist at test / dev-server time fails with:

  ERR_MODULE_NOT_FOUND: Cannot find module .../@hyperframes/lint/dist/index.js
  imported from .../packages/core/dist/compiler/staticGuard.js

Build the canonical pre-core set @hyperframes/{parsers,lint,studio-server}
(the glob the root build script uses) in both jobs so it can't drift again.
The SDK job is left as-is — it builds parsers+core only and passes.

Reproduced locally: removing packages/lint/dist reproduces the exact
ERR_MODULE_NOT_FOUND; building lint resolves it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): address PR #1768 review — stale comment + harden offline init

- Update the stale interactive-path comment that still said "Opt out with
  --skip-skills"; the flag is neutered, opt-out is HYPERFRAMES_SKIP_SKILLS=1.
- Wrap installAllSkills in ensureSkillsCurrent with try/catch. installAllSkills
  is already non-strict (swallows its own failures), but since --skip-skills no
  longer escapes this path, every init — including offline ones that fall through
  to "install anyway" — runs it. The guard guarantees a skills-install failure
  only warns and proceeds, never breaks init.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:57:27 +08:00
Miguel Ángel 7a4853dfe6 refactor: extract @hyperframes/studio-server from core (#1757)
* refactor: extract @hyperframes/studio-server package from core

Moves all studio-api routes, helpers, and Hono server wiring from
packages/core/src/studio-api/ into a new standalone packages/studio-server
package (@hyperframes/studio-server).

Core keeps thin re-export stubs at @hyperframes/core/studio-api and the
subpath helpers (screenshot-clip, draft-markers, etc.) for backward
compatibility. Consumer imports (cli studioServer, vite adapter/config,
producer htmlCompiler, studio manualEditsTypes) are updated to import from
@hyperframes/studio-server directly.

Also exports rewriteInlineStyleAssetUrls from @hyperframes/core root (was
in compiler/rewriteSubCompPaths.ts but not re-exported), required by
@hyperframes/studio-server/helpers/subComposition.

Removes postcss-selector-parser from @hyperframes/core dependencies (moved
to @hyperframes/studio-server which owns the routes that used it).

Depends on @hyperframes/parsers (PR #1755).

* fix(ci): add parsers+studio-server to Dockerfile and build before preview tests

* fix(ci): build @hyperframes/studio-server before Test and studio load smoke

Studio's vite.config.ts imports @hyperframes/studio-server, which resolves
via its "node" export condition to built dist. The Test and studio-load-smoke
jobs only built parsers + core, so esbuild's config load failed to resolve the
package entry. Build studio-server too.

* fix(studio): repoint sdkCutoverParity test import to studio-server

sourceMutation moved from core's studio-api to @hyperframes/studio-server;
the test still imported the deleted core path. This was masked while studio's
vite.config failed to load (couldn't resolve studio-server); now that the
config loads, the test runs and the stale import surfaced.
2026-06-27 01:24:01 -04:00
Miguel Ángel 98d0bdd73c refactor: extract @hyperframes/lint from core (#1756)
* refactor: extract @hyperframes/lint package from core

Moves all lint rules, hyperframeLinter, lintProject, and related types
from packages/core/src/lint/ into a new standalone packages/lint package.

Core keeps a thin re-export stub at @hyperframes/core/lint for backward
compatibility. Consumer imports (cli lint command, producer hyperframeLint)
are updated to import from @hyperframes/lint directly.

Depends on @hyperframes/parsers (PR #1755).

* fix: restore postcss-selector-parser in core (sourceMutation.ts still uses it)

* fix(ci): add parsers+lint to Dockerfile and build before preview tests

* chore: update bun.lock after restoring postcss-selector-parser dep

* test(cli): update lintProject test for string-dir signature from @hyperframes/lint

* refactor(core): single-source the lint engine in @hyperframes/lint

Delete core's byte-identical copy of the lint rule engine and re-point
staticGuard at @hyperframes/lint, so the render-time render-gate and the
studio preview share one rule engine instead of two copies that could
silently diverge. Back-compat preserved via the @hyperframes/core/lint stub.

Addresses review feedback on the dual-copy footgun.
2026-06-27 01:16:40 -04:00
Miguel Ángel cdf9c817e1 refactor: extract @hyperframes/parsers from core (#1755)
## Summary

Extracts the GSAP parser/writer suite, HTML parser, hf-ids, spring-ease, and the shared composition data types out of `@hyperframes/core/src/parsers/` into a new, independently-publishable **`@hyperframes/parsers`** package.

This is the foundation of the [#1749](https://github.com/heygen-com/hyperframes/issues/1749) effort: make HyperFrames' parsing/linting/validation usable as plain libraries in a Node app, without shelling out to the CLI. Parsers is the standalone base every other extracted package builds on.

**Part 1 of 3** — splits #1754 into independently-reviewable pieces. Parts 2 (lint) and 3 (studio-server) stack on this branch.

## What moves

| | |
|---|---|
| Source moved out of core | **~9,900 LOC** (`src/parsers/` → `packages/parsers/src/`) |
| Total lines removed from core (incl. tests + goldens) | ~19,600 |
| Files relocated | 39 |
| Tests carried over | **660 passing** (5 skipped, 3 todo) |

The big movers: `gsapParser` / `gsapParserAcorn` (the recast + acorn dual parsers), `gsapWriterAcorn`, `gsapSerialize`, `gsapUnroll`, `htmlParser`, `hfIds`, `springEase`, `stableIds`, plus the `__goldens__` corpus.

## Bundle footprint of the new package

| Artifact | Size |
|---|---|
| `dist/` (unpacked) | 1.7 MB |
| npm tarball (packed) | 409 KB |
| `dist/index.js` | 90 KB (**~21 KB gzipped**) |
| Heaviest entries | `gsapWriterAcorn.js` 93 KB · `gsapParser.js` 91 KB |

Most of the weight is the GSAP AST machinery (recast/babel/acorn). It's tree-shakeable via subpath entries (`@hyperframes/parsers/hf-ids`, `/gsap-constants`, etc.) so a consumer that only needs `hf-ids` (2 KB) doesn't pull the parsers.

## How `@hyperframes/core` changes

The interesting part: **core sheds its entire AST toolchain.**

| core `dependencies` | before | after |
|---|---|---|
| count | 9 | 6 |
| removed | — | `@babel/parser`, `acorn`, `acorn-walk`, `magic-string`, `recast` |
| added | — | `@hyperframes/parsers`, `linkedom` |

Before this PR, importing `@hyperframes/core` at all dragged in babel + recast + acorn just to construct types. Now those live behind `@hyperframes/parsers`, and a consumer that only wants core's runtime/compiler types never resolves the parser stack. Core keeps thin `@deprecated` re-export stubs at the old subpaths (`@hyperframes/core/gsap-parser`, `/gsap-constants`, …) so nothing downstream breaks.

## Design notes

- **`"bun"` export condition before `"node"`** in every package export. Bun resolves the TypeScript source directly (no pre-built `dist/`), while Node/tsx/Docker contexts fall through to `"node"` → `dist/`. This keeps the dev loop zero-build while published artifacts stay Node-consumable.
- `@hyperframes/parsers` is **standalone** — zero `@hyperframes/*` dependencies — so it can be the base of the stack.

## Test plan

- [x] `bun run --filter @hyperframes/parsers test` — 660 tests pass
- [x] `bun run --filter @hyperframes/sdk test` — 382 tests pass
- [x] `bun run build` — full monorepo build succeeds
- [x] Fallow audit passes on CI
2026-06-27 00:46:26 -04:00
WaterrrForeverandClaude Opus 4.8 d70ee134cc feat(cli): add skills version check, update, and freshness manifest (#1738)
* feat(cli): add skills version check, update, and freshness manifest

Give the HyperFrames skill bundle a content fingerprint so agents and
users can tell whether installed skills are the latest version, on any
platform that can run the CLI.

- skills-manifest.json (repo root): per-skill sha256 over the whole skill
  directory; minimal {source, skills}, no version/timestamp so it is fully
  deterministic. Generated by scripts/gen-skills-manifest.ts.
- `hyperframes skills check` [--json]: compares installed skills to the
  manifest; exits non-zero when something is outdated (agent/CI gate).
- `hyperframes skills update`: thin wrapper over `npx skills update`.
- Passive nudge on render/lint/validate when skills are stale (24h cache,
  same opt-out as the CLI self-update notice).
- "latest" resolved via `git ls-remote` + SHA-pinned raw URL to dodge
  GitHub raw-CDN lag, falling back to the main branch URL.
- CI job + lefthook hook keep skills-manifest.json in sync with skills/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): add execFile to child_process mock in skills test

skills.test.ts mocks node:child_process but only declared execFileSync
and spawn. Loading skills.js transitively loads skillsManifest.ts, which
runs promisify(execFile) at module load, so vitest threw on the missing
execFile named export. Add a bare stub — these tests never invoke it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): init installs all skills; skills update pulls the full set

Make `hyperframes init` the single place skills are pulled in full, and
make "update" mean "get everything" rather than "refresh what's there".

- init now always installs/refreshes ALL skills (incl. ones not yet
  present) instead of prompting "Install AI coding skills?" — opt out
  with `init --skip-skills`. Both the interactive and non-interactive
  paths pass `--all --yes` so the complete set is fetched.
- `hyperframes skills update` switches from `npx skills update` (which
  only refreshes already-installed skills) to `skills add --all`, so it
  installs missing skills too — the same install step init runs.
- SKILL.md documents init-installs-all and the new update semantics.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): skills check treats missing skills as needing an update

The full skill set is now the goal (init and `skills update` both pull
all, including ones not installed), so a partial install is no longer
"a choice" — it's something to fix.

- diffSkills: updateAvailable is now true when anything is outdated OR
  missing (local-only still doesn't count). So `skills check` exits
  non-zero — and renders "Update:" instead of "up to date" — whenever a
  skill is missing, not just when one is stale.
- The passive render/lint/validate nudge follows suit: it now counts
  missing alongside outdated ("N skills out of date or missing"),
  tracked via a new skillsMissingCount cache field.
- SKILL.md documents the stricter check.

Note: platforms that intentionally vendor only a subset of skills (e.g.
a Codex snapshot) will now see check report non-zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): install/update skills straight from the GitHub repo

`skills add owner/repo` can resolve through the skills.sh registry, which
lags behind the repo — so `update` could install a stale version while
`check` (which resolves latest directly from GitHub) keeps reporting
"outdated", an endless loop.

Switch the install source to the full GitHub URL
(https://github.com/heygen-com/hyperframes), which makes `skills add`
git-clone the repo directly at latest main, bypassing the registry. This
covers `hyperframes skills`, `hyperframes skills update`, and `init`'s
skill install — all of which go through SOURCES. Now install/update and
check agree on what "latest" means.

The init "install skills" hint now points at `npx hyperframes skills
update` so the manual path uses the same GitHub-direct fetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): init checks skills against GitHub, installs only when stale

`hyperframes init` now runs the skills version check first and only
(re)installs when something is outdated or missing — instead of
unconditionally re-pulling every time. Re-running init on an
already-current project is now a no-op ("skills are already up to date").

- New ensureSkillsCurrent() helper, shared by both the interactive and
  non-interactive init paths (no duplicated install logic).
- The check resolves "latest" straight from GitHub (same source the
  install uses); best-effort — if it can't reach GitHub it installs anyway.
- SKILL.md updated to describe the check-then-install behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(cli): address skills manifest review feedback

From the PR review (points 1, 2, 4, 5):

1. Remove the `local-only` skill status. checkSkills only ever hashes
   manifest-listed skills, so a local-only status could never appear in
   the end-to-end output — and making it appear would wrongly flag
   unrelated skills (the `.../skills` dir is shared across sources).
   diffSkills now reports only on manifest skills; skills on disk that
   aren't in the manifest are ignored.
2. Drop the redundant per-directory sort in listFilesSorted — the single
   final out.sort() is what guarantees a deterministic hash (verified:
   manifest unchanged).
4. resolveLatestManifest local-path detection now uses path.isAbsolute,
   so Windows absolute paths (C:\...) are treated as local instead of
   falling through to a remote fetch.
5. fetchManifest validates the response shape (asSkillsManifest) instead
   of a blind `as` cast, so a CDN error page served as 200 fails with a
   clear error rather than a cryptic crash later in diffSkills.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): strict skills update + auto-discover any agent host

Address PR review (Magi blocker + James/Rames robustness):

- Blocker (Magi): `skills update` is the documented recovery path for
  `skills check || skills update`, but it delegated to installAllSkills()
  which swallowed missing-npx and failed `skills add` as "skipped",
  exiting 0 even when nothing changed. Add a strict mode that throws on
  failure; update sets a non-zero exit (init stays best-effort). New tests
  simulate a non-zero `skills add` (exit 1) and the success path.

- Robustness (James/Rames #2): the upstream `skills` CLI installs into
  ~72 agent conventions; a hard-coded list (4, or even 11) can't track
  that. Replace defaultSkillRoots with discoverSkillRoots — it scans cwd +
  $HOME for any `<host>/skills/<manifest-skill>/SKILL.md` (plus the XDG
  `.config/<host>/skills`), so detection is structural and future-proof,
  no closed list. agentFromDir infers the host from the path.

- Tests (Rames #3): temp-fixture detection tests for every convention ×
  {project, global}, scope priority, claude-code preference, the
  no-install case, the --dir override, and an unknown/new host (proving
  the no-closed-list property).

- Docs (Rames #4/#5): SKILL.md notes init's best-effort GitHub round-trip;
  findRepoManifest climbs 16 levels (was 8) for deep monorepos.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): resolve CodeQL file-system race + de-flake Windows npx test

Two CI fixes:

- CodeQL (high, js/file-system-race) at gen-skills-manifest.ts: the
  existsSync(outPath) precheck followed by writeFileSync(outPath) is a
  check-then-write race. Read the committed manifest directly in a
  try/catch instead (missing/unreadable ⇒ "no committed manifest"), so
  there's no precheck to race against. Behavior is unchanged.

- Windows Tests: npxCommand.test.ts's real `npx --version` smoke test
  cold-starts slower than vitest's 5s default on Windows runners and
  timed out. Give the test 60s headroom (and a 30s exec timeout). Kept
  as a real execution check — mocking would reduce it to a tautology.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): repair garbled npx smoke-test timeout comment

The explanatory comment for the 60s timeout was scrambled across the
callback/timeout arguments, failing oxfmt --check (and thus preflight,
which in turn skipped preview-parity and failed the regression gate).
Move it above the it() call so it no longer sits between call arguments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 22:52:42 +08:00
James Russo 0c05025c04 ci(skills): run skills/**/*.test.mjs in CI (#1724)
skills/**/*.test.mjs files (e.g. skills/media-use/scripts/resolve.test.mjs
and skills/media-use/scripts/lib/manifest.test.mjs) are bare `node --test`
files with only `node:` built-in imports. They aren't part of any workspace
package, and the existing `Test` job's path filter (the `code` filter in
the `changes` job) excludes `skills/**`, so even on PRs that touch only
skills/ those tests never run.

This matters for regression guards. The shell-injection probe test added in
HF#1723 feeds probe() a filename containing `clip"; touch INJECTED; echo
".mp4` and asserts no marker file is created. The test passes locally but
under the current job graph it would never run in CI on a follow-up skills/
change that re-introduces the bug.

Closing the gap with a dedicated `Test: skills` job rather than relaxing
the `code` filter. The existing `Test` job's steps run `bun run test:scripts`
(hardcoded file list) and `bun run --filter '*' test` (workspace packages
only), neither of which would actually execute skills tests even if the
filter let `skills/**` through. The dedicated job needs no `bun install`,
just node 22, since the tests only import from `node:` and relative paths.

The discovery step shells out to `find` and fails loudly when zero test
files match, so a future rename or layout change can't silently turn this
into a no-op pass.

Spotted by Via in HF#1723 review thread, confirmed by James as a separate
follow-up rather than a blocker for HF#1723.

--
Jerrai (https://claude.com/claude-code)
2026-06-25 12:50:43 -07:00
Miguel Ángel 041f2fa196 fix(media-use): kill shell command injection in probe/heygen-search/eval
Swap execSync(<shell-string>) → execFileSync(file, [argv]) in probe.mjs, heygen-search.mjs, and eval.mjs so hostile filenames / queries / manifest metadata can't inject shell. Adds probe.test.mjs regression guard and a CI Test (skills) job so it actually runs. Closes the media-use High/Critical scanner alert.
2026-06-25 15:16:52 -04:00
Miguel Ángel 60d3eeb1f7 test(producer): add stream duration parity check to regression harness (#1652)
Probes the rendered output for video and audio stream durations after
render and fails the test if they differ by more than 0.5s. Catches
mux-level truncation regressions like the ffmpeg -shortest bug (#1648)
where one stream gets silently cut short.

Runs on all non-png-sequence fixtures with audio — no new meta.json
field needed since this is a universal invariant, not a per-fixture
threshold.
2026-06-22 18:32:59 -04:00
Miguel Ángel 25b717dd9c fix(cli): resolve npx shims on Windows (#1626) 2026-06-21 17:50:22 -04:00
Vance IngallsandClaude Opus 4.8 7607a714a6 ci(publish): publish @hyperframes/sdk to npm (#1587)
The SDK is version-bumped by scripts/set-version.ts (it's in the PACKAGES list)
but was never added to the publish_pkg list in publish.yml — so
@hyperframes/sdk@0.6.112 sits on the version line yet is absent from npm (404),
while core/player/engine/etc. all shipped at 0.6.112. Add the missing publish
call so the SDK ships with every release.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 22:55:22 -07:00
Miguel Ángel a241f2591e fix(studio): break all 7 circular dependency cycles and fix rules-of-hooks violation (#1422) 2026-06-13 19:23:05 -04:00
Vance IngallsandClaude Sonnet 4.6 a0ee97210b fix(sdk,core): css tokenizer, override-set replay, setattribute safety, persist errors (#1350)
* fix(sdk,core): css tokenizer, override-set replay, setattribute safety, persist errors

* test(sdk,ci): smoke test + explicit sdk-tests CI gate

Smoke test covers the full public surface:
  openComposition → setStyle/setText/dispatch(moveElement) → serialize
  applyPatches + ORIGIN_APPLY_PATCHES tagging
  batch() coalescing + transactional rollback on throw
  undo/redo round-trip
  persist adapter write + persist:error surfacing
  T3 embedded mode: override-set apply on open + getOverrides round-trip

Adds sdk-tests CI job so SDK coverage is explicitly named and required —
prevents a repeat of the demo-next vitest-never-ran incident.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sdk): export adapter types, awaitable flush(), never-coalesce mode

- Export PersistAdapter, PreviewAdapter, PersistVersionEntry from package
  root — callers can now write typed fakes without reaching into internals
- Add flush(): Promise<void> to Composition interface + CompositionImpl —
  app-close handlers can await a clean drain of the persist queue
- coalesceMs <= 0 disables coalescing entirely in createHistory — enables
  deterministic test scenarios without per-entry timestamp manipulation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(sdk): p2 edge cases — setText no-text-node, override-remove non-existent, flush in smoke

- setText on element with no prior text node (firstTextIdx=-1 path)
- applyOverrideSet null removal on non-existent prop is a no-op (no throw)
- smoke persist test uses comp.flush() instead of setTimeout
- can() JSDoc clarifies Phase 3b false-return is intentional feature-detection

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci: trigger regression suite

* fix(ci): add packages/sdk/package.json to Dockerfile.test workspace copy

bun install --frozen-lockfile fails in the regression Docker build because
the lockfile references the sdk workspace member but its package.json was
not copied into the image before the install step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 14:07:49 -07:00
Miguel Ángel c12987e301 fix(producer): revert Proxy-based wrapTimeline to plain-object approach (#1284)
* fix(producer): revert Proxy-based wrapTimeline to plain-object approach

The `new Proxy` wrapper for GSAP timelines introduced in #1279 causes
Chrome headless to hang indefinitely during page.goto — DOMContentLoaded
never fires. The plain-object approach (explicit method allowlist) loads
in <800ms on the same composition.

The Proxy's generic get/set traps interact badly with Chrome's internal
object inspection (Symbol checks, thenable probing, DevTools serialization)
during HTML parsing, creating a permanent navigation hang. The
maybePublishRenderReady listener fix from #1279 is preserved — only the
wrapTimeline implementation is reverted.

Compositions using GSAP methods outside the allowlist (eventCallback,
labels, repeat, etc.) will see those calls silently dropped rather than
forwarded. This is the same behavior as v0.6.81 and earlier. A safer
forwarding approach can be explored separately without blocking renders.

* fix(producer): address review — stale meta.json descriptions + silently-dropped methods doc

- three-boundary: description referenced Proxy fix but the test uses
  onUpdate in to() vars (allowlist path), not eventCallback
- three-boundary-deferred: same — pins Bug 2's deferred-race, not Bug 1
- Add inline doc comment listing silently-dropped GSAP methods and the
  onUpdate workaround

* ci: add page.goto timing canary to CLI smoke test

Parse page.goto completion times from the render log and fail if the
slowest navigation exceeds 5s. Catches wrapTimeline regressions that
block DOMContentLoaded before the 60s timeout fires.

Refs: #1285

* fix(producer): forward all GSAP methods via dynamic enumeration at wrap time

Instead of silently dropping methods outside a static allowlist, enumerate
the real timeline's prototype chain at wrap time and generate plain-object
forwarding stubs for every method not already covered.

This achieves the same coverage as the `new Proxy` approach from #1279
without the Chrome headless navigation hang — no Proxy trap surfaces are
exposed to Chrome internals. Methods prefixed with `_` (GSAP private) are
skipped. All forwarded methods flush pending batch operations before
delegating, matching the existing allowlist behavior.

Closes #1285

* fix(producer): make proxy non-thenable + harden CI canary

- Skip `then` in forwardRemainingMethods — GSAP timelines are thenable
  (tl.then resolves on completion), and forwarding it makes the proxy
  thenable too: Promise.resolve(proxy) or await proxy hangs forever for
  paused timelines
- Add unit test: Promise.resolve(proxy) resolves immediately, real
  then() is never called
- CI canary: exit 1 (not 0) when no page.goto timing is found in logs,
  so a log-format change loudly breaks CI instead of silently disabling
  the canary
2026-06-08 21:10:43 -04:00
Vance IngallsandClaude Sonnet 4.6 25420bf4cf ci: skip ffmpeg-static CDN download on ubuntu; retry Windows FFmpeg install (#1275)
* ci: skip ffmpeg-static CDN download on ubuntu; retry Windows FFmpeg install

ubuntu-24.04 runners ship /usr/bin/ffmpeg. Set FFMPEG_BIN so ffmpeg-static's
postinstall script skips its GitHub-release binary download, preventing bun
install failures when that CDN is unavailable.

For Windows: increase BtbN/FFmpeg-Builds download max-attempts 3→8 with
longer backoff (30×attempt s) and set FFMPEG_BIN after install so bun install
also skips ffmpeg-static's download in both render and test jobs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci: fix FFMPEG_BIN approach — use writable copy via composite action

/usr/bin/ffmpeg is not writable by the runner user. When bun runs
ffmpeg-static's postinstall script in a context where process.exit(0) is
intercepted, the skip-if-exists check has no effect and the download proceeds
to the destination path. Pointing FFMPEG_BIN at a system path (/usr/bin/ffmpeg)
therefore causes EACCES even when the CDN returns 200.

Replace the top-level env var with a prepare-ffmpeg-bin composite action that
copies the system ffmpeg to $RUNNER_TEMP (writable). Call it before every
bun install step in the CI workflow. Whether the postinstall script skips or
overwrites, the write target is now writable and the job succeeds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci: skip apt fallback in prepare-ffmpeg-bin; use stub when ffmpeg absent

ubuntu-24.04 GHA runners do not have ffmpeg pre-installed. The apt-get
fallback triggered the install of ffmpeg and its dependencies, but the
Azure apt mirror returned 404 for libcaca0, aborting the composite action.

ffmpeg-static's postinstall only needs a regular file to exist at FFMPEG_BIN
in order to reach the statSync check and call process.exit(0) — it does not
need a real executable. Write a minimal shell stub when 'which ffmpeg' returns
empty. Jobs that require an actual ffmpeg binary (cli-smoke-required) install
it via apt before calling this action, so 'which ffmpeg' returns the real path
and the copy branch runs instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 10:22:31 -07:00
James RussoandClaude Opus 4.8 4da567df22 feat(gcp-cloud-run): Google Cloud Run + Workflows distributed render adapter (#1253)
* feat(gcp-cloud-run): add Google Cloud Run + Workflows distributed render adapter

Adds @hyperframes/gcp-cloud-run, the GCP counterpart to @hyperframes/aws-lambda
(issue #932). The OSS distributed primitives (plan, renderChunk x N, assemble)
are unchanged; this package is the storage/compute/orchestration glue.

Package: Cloud Run handler (one image, three actions), runs under bun; GCS
transport; in-image chrome-headless-shell resolver; client SDK
(renderToCloudRun, getRenderProgress, deploySite, computeRenderCost); Dockerfile;
Cloud Workflows definition; Terraform module; CLI cloudrun
deploy|sites|render|render-batch|progress|destroy with --output-resolution and
--strict-variables; 62 unit tests + docs + live smoke script.

Shared extraction (removes ~640 lines of adapter duplication): move the
cloud-agnostic config validator + content-hash into producer/distributed; both
adapters import them. Validated end-to-end on GCP at 37.4 dB PSNR vs baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): resolve @hyperframes/gcp-cloud-run in the CLI build + root build

The CLI bundle (esbuild) couldn't resolve `@hyperframes/gcp-cloud-run/sdk`,
failing Build/Typecheck/CLI-smoke (and the perf/windows/regression jobs that
build first). Mirror the aws-lambda handling: mark the gcp adapter + its /sdk
subpath external in tsup.config.ts with a source alias, and add gcp-cloud-run
to the root `build` filter so its dist exists for publish + runtime.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): copy gcp-cloud-run manifest in Dockerfile.test for frozen install

The regression test image runs `bun install --frozen-lockfile` after copying
each workspace package.json individually. The CLI now depends on
@hyperframes/gcp-cloud-run (workspace:*), so the frozen install fails to
resolve it unless its manifest is present. Add the COPY line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): add machine-sizing flags to `cloudrun deploy`

Closes the parity gap with `lambda deploy` (which exposes --memory etc.).
`cloudrun deploy` now threads --cpu, --memory, --max-instances, and --timeout
into the Terraform apply; omitted flags keep the module defaults
(4 vCPU / 16Gi / 100 instances / 3600s). For finer control, apply the module
directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gcp-cloud-run): address PR review (security, waste, limits, alerts)

- server.ts: bucket-allowlist guard no longer fails open silently. Unset env
  logs a one-time WARNING; "*" is an explicit opt-out; otherwise it enforces.
- server.ts: stop double-shipping audio.aac. It already rides in the plan
  tarball every consumer downloads, so drop the redundant standalone upload
  (plan) + re-download/overwrite (assemble); assemble reads it from the untar,
  falling back to a supplied AudioGcsUri for compat.
- server.ts: chunk extension via path.extname() instead of slice(lastIndexOf).
- workflow.yaml: clamp parallel concurrency_limit to math.min(chunkCount, 20)
  — Cloud Workflows hard-caps concurrent iterations at 20.
- Dockerfile: pin bun (bun-v1.3.9) so an interop change can't silently break
  the image rebuild.
- terraform: add min_instances var (default 0); add a workflow-failure alert
  (finished_execution_count status=FAILED) alongside the request-count one.
- costAccounting: document that displayCost excludes GCS storage/egress.

Verified against the actual APIs: @google-cloud/workflows@4.4.0
ICreateExecutionRequest has no executionId (so the idempotency-token suggestion
isn't available in this client); Workflows concurrency cap is 20; failure
metric is workflows.googleapis.com/finished_execution_count (status label).
174 adapter tests pass, fallow/oxlint/oxfmt/terraform clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gcp-cloud-run): address round-2 review — error code + CFR forwarding

- workflow.yaml: rename the zero-chunk failure code PLAN_TOO_LARGE →
  PLAN_PRODUCED_ZERO_CHUNKS. The old code implied a size-ceiling breach (the
  opposite cause), misleading anyone triaging the alert.
- workflow.yaml: forward Config.cfr to the assemble step
  (`Cfr: ${("cfr" in config) and config.cfr}`). It was read by the handler
  but never sent, so exact-CFR was silently off for every Cloud Run render.
  Uses the same `in`-operator guard already proven in the retryable predicate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(release): include gcp-cloud-run in set-version PACKAGES list

set-version.ts (driven by release:prepare) bumps an explicit package list to
the shared version on each release. gcp-cloud-run was wired into the build +
publish.yml but missing here, so a release would leave it at a stale version
and publish.yml would push the wrong version. Add it so the new package
version-bumps + publishes in lockstep with the others.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 14:43:38 -07:00