Commit Graph
10 Commits
Author SHA1 Message Date
Miguel Ángel 4d9a4e4e86 fix(cli): keep OpenRouter as a vision provider for capture (#7189)
* fix(cli): keep OpenRouter as a vision provider for capture

Reverts #7188.

* fix(cli): shorten the restored thinking budget comment
2026-11-04 22:43:24 +00:00
Miguel Ángel 2ac011422e refactor(cli): remove OpenRouter as a vision provider for capture (#7188)
* refactor(cli): remove OpenRouter as a vision provider for capture

* refactor(cli): tidy comment and formatting after OpenRouter removal

* refactor(cli): shorten the thinking-budget comment

* chore(skills): refresh the skills manifest for the film reference
2026-11-04 21:37:59 +00:00
Miguel Ángel 5583ae4293 perf(cli): commands run without esbuild, puppeteer-core or sharp until they need them (#5930)
* perf(cli): commands run without esbuild, puppeteer-core or sharp until they need them

A host that bundles the CLI can now leave these packages out: each command
loads them only where it uses them, and sharp installs once on first use when
it is missing. The build check fails on any static import of them again.

* fix(cli): use a host's sharp from NODE_PATH and let hosts turn first-use installs off

* fix(cli): capture fails when sharp is missing, and one install serves every caller

A missing sharp now stops the asset download up front instead of counting every
image as unavailable. Concurrent first uses share one install, a failed install
is not retried in the same run, and preview thumbnails report why. Hosts can turn
off installs for a list of packages, e.g. HYPERFRAMES_NO_OPTIONAL_INSTALL=sharp.

* fix(cli): snapshot --describe still works without sharp, resizing only when it is installed

* style(cli): format the sharp loader and its tests

* build(cli): sharp is an optional dependency, so a host can ship the CLI without it

* build(studio-server): keep sharp out of the bundle now that it is optional
2026-10-20 09:12:47 +00:00
Miguel Ángel e8fdfd0d65 fix(cli): install onnxruntime-node and google-genai on first use (#4156)
* fix(cli): install onnxruntime-node and google-genai on first use

Both carry deprecated transitive dependencies (boolean, node-domexception)
that showed up as npm warnings on every npx hyperframes install. They move
to devDependencies (types and the Intel-binding check still need them at
build time; the bundler externalizes them) and load through a new
on-demand loader that installs into the CLI's own cache dir the first
time a feature needs them, with one stderr line, no prompt, and a
manual-install command named in the error if the network is unavailable.

hyperframes doctor lists each as installed (with version) or
not-installed-yet.

* fix(cli): sweep stale install staging and bound network retries

* fix(cli): optional-package install only sweeps staging dirs whose process is dead

Staging dirs now carry a per-call suffix after the pid, and the sweep skips any dir
whose pid is alive, so concurrent first runs no longer delete each other's install.

The install error suggests the network only for network failures, and the manifest
path has one owner. install() takes an injectable npm runner so tests drive the real code.

* fix(cli): optional-package sweep also clears old-format staging dirs and tests a live foreign dir

The concurrent-callers test shared one pid, so it never exercised the sweep rule; a live foreign
staging dir test now does. Old-format dirs of a dead pid are swept, and a failed cleanup no
longer aborts a viable install.

* fix(cli): optional-package sweep names a stale dir it could not remove

A locked leftover is still skipped so the install proceeds, but the warning makes it visible.

* docs(cli): shorten the install docblock to the comment limit

Comment-only change.
2026-09-19 05:27:35 -04:00
WaterrrForever 8eea3c913f fix(capture): let vision captioning authenticate the way a server can (#3561)
* fix(capture): let vision captioning authenticate the way a server can

Three defects in one phase, all of which end with a capture that reports
"Captioned N/N images" and then "0 images captioned with Gemini" — a
successful-looking run that hands the agent nothing to see by.

1. Credential. The captioner only accepted an API key. A server deployment
   holds a service account, not a key, and a rejected key is indistinguishable
   from an unset one here: every request returns empty text and no error. Vertex
   is now a first-class provider, ranked above the bare key and below an explicit
   OPENROUTER_API_KEY opt-in, configured by HYPERFRAMES_VERTEX_SERVICE_ACCOUNT +
   HYPERFRAMES_VERTEX_PROJECT_ID (region via HYPERFRAMES_VERTEX_LOCATION). It
   carries its own model default because the Gemini API's flash-lite preview id
   is not resolvable on Vertex.

2. Empty captions. Thinking tokens are drawn from maxOutputTokens, so a model
   left free to think can spend the whole budget and return no text — a
   successful request with no caption. thinkingBudget is pinned to 0; a one-line
   factual caption needs no reasoning.

3. Native abort. Rasterizing a batch of SVGs concurrently drove up to SVG_BATCH
   simultaneous librsvg renders through libvips and corrupted the heap:
   `free(): unaligned chunk detected in tcache 2` (SIGABRT) during this phase,
   twice in fourteen days, losing the whole capture each time. A native abort
   cannot be caught, so the concurrency is removed rather than handled —
   rasterization is serialized and libvips' worker pool is bounded, while the
   vision requests, which are the slow leg, stay parallel. Throughput barely
   moves: 225 captions across three real captures, 0 failures, 13-25s each.

* test(capture): pin the rasterization loop to one render at a time

The serialization fix shipped without a regression test on the grounds that native
heap corruption is not unit-testable. The corruption is not, but the property that
prevents it is: `sharp` is mocked to record how many renders are in flight, and a
six-SVG batch must never reach two. A deliberately slow caption stub makes
overlapping renders the faster path, so a future refactor that "optimises" the loop
back to `Promise.all` fails here instead of aborting in production.

Also covered: `sharp.concurrency(1)` is applied — serializing the loop while leaving
libvips' pool at the host core count still fans one render across every core — and an
unrasterizable SVG is skipped without breaking serialization for its siblings.

Verified as a real guard: reverting only contentExtractor.ts to origin/main fails 7 of
the 22 cases in this file.


* fix(capture): tell the truth in the asset-descriptions header when Vertex captioned

The provider gate in `contentExtractor` accepts Vertex when a project and a
service account are both set -- which is the configuration a server
deployment actually has. The header written next to the captions still
tested only for an API key, so a capture whose captions Vertex had just
generated was labelled "GEMINI_API_KEY not set -- descriptions below are
catalog-derived".

That header is not cosmetic: it travels into the context the template
editor reads, telling it to distrust captions that are real.

Mirror the same two variables here, and name every provider in the fallback
text instead of only the API key.

* fix(capture): hand libvips' worker pool back after the renders

`sharp.concurrency(1)` is process-global and was set once, for the whole life of
the process. The bound is right for the rasterize loop -- a native abort in
libvips cannot be caught, so the renders must not overlap -- but its scope was
every later sharp caller in the process, none of which asked for captioning, all
of them pinned to one thread from then on.

Now the host's value is read first and restored in a `finally` around the
rasterize loop, so a skipped SVG cannot cost the process its threads either. The
vision requests below are network work and gain nothing from a pinned pool.

The mock had to grow the getter half of sharp's API -- `concurrency()` with no
argument reports the current value -- since save-and-restore is untestable
without it. Verified as a real guard: dropping only the restore fails both new
cases.

Raised by Rames Jusso in review of #3561 and concurred by Magi.
2026-09-01 23:34:59 +08:00
Miguel Ángel 9ae0007261 fix(cli): preserve capture failure diagnostics 2026-07-31 20:30:07 +00:00
Miguel Ángel dfc60797a2 fix(cli): report bounded vision failures 2026-07-31 20:30:07 +00:00
Miguel Ángel 49091e6142 fix(cli): propagate live capture budget 2026-07-31 20:30:07 +00:00
Miguel Ángel 45aead84c9 test(cli): cover bounded capture stages 2026-07-31 20:30:07 +00:00
James RussoandClaude Opus 4.8 646ffff927 feat(cli): support OpenRouter as an alternative vision provider for capture captioning (#1478)
* feat(cli): support OpenRouter as an alternative vision provider for capture captioning

`hyperframes capture` could only enrich asset descriptions with Gemini vision,
which requires a Google API key. Add OpenRouter as an alternative so users
without Google access can caption via any vision-capable model through one
unified key.

Provider is selected by which key is present: OPENROUTER_API_KEY → OpenRouter
(OpenAI-style /chat/completions with an image_url data URI), else
GEMINI_API_KEY/GOOGLE_API_KEY → Gemini (unchanged), else DOM-only as before.
OpenRouter wins if both are set. Default model is google/gemini-3.1-flash-lite
(the OpenRouter analog of the Gemini path's existing 3.1-flash-lite tier),
overridable via HYPERFRAMES_OPENROUTER_MODEL.

Both vision call sites — the image loop and the rasterized-SVG loop — route
through a single `captionOne` dispatcher, so the new provider works for SVGs too
(the original PR #840 only patched the image loop, which would have left
OpenRouter-only users with crashing SVG captioning). The OpenRouter path checks
res.ok and surfaces the status/body on failure.

Reimplements #840 (which was unmergeable: saved with a UTF-8 BOM + CRLF so
GitHub rendered it as a binary diff, used `any`, reused the Gemini model env
var, and had a hallucinated default model id).

- Adds unit tests for the OpenRouter path (happy path, graceful degradation on
  non-OK status, no-key skip).
- Documents OPENROUTER_API_KEY in the website-to-video guide and the CLI capture
  reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(cli): fix typecheck in OpenRouter caption test — capture request without `as`

The test cast `fetchMock.mock.calls[0]` to a tuple (TS2352: `[] | undefined`
doesn't overlap `[string, RequestInit]`), which failed the Typecheck CI job.
Capture the url/init inside the typed mock and assert via `new Headers()` +
`typeof` narrowing instead — no `as` assertions (which the repo bans anyway).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 20:14:45 -07:00