mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-18 07:50:13 +00:00
refactor(cli): remove OpenRouter as a vision provider for capture (#7188)
* refactor(cli): remove OpenRouter as a vision provider for capture * refactor(cli): tidy comment and formatting after OpenRouter removal * refactor(cli): shorten the thinking-budget comment * chore(skills): refresh the skills manifest for the film reference
This commit is contained in:
@@ -75,7 +75,7 @@ vision key is available.
|
||||
| **Voice (TTS)** | HeyGen → ElevenLabs → Kokoro | HeyGen credential (above) · then `ELEVENLABS_API_KEY` | Kokoro: `pip install kokoro-onnx soundfile` |
|
||||
| **Music (BGM)** | HeyGen library → Lyria → MusicGen | HeyGen credential (above) · then `GEMINI_API_KEY` → `GOOGLE_API_KEY` | MusicGen: `pip install transformers torch soundfile numpy` |
|
||||
| **Sound effects** | HeyGen library → bundled library | HeyGen credential (above) | bundled — no deps |
|
||||
| **Capture descriptions** | OpenRouter → Gemini | `OPENROUTER_API_KEY` → `GEMINI_API_KEY` | None; optional for [website capture](/guides/product-launch-video) |
|
||||
| **Capture descriptions** | Gemini | `GEMINI_API_KEY` | None; optional for [website capture](/guides/product-launch-video) |
|
||||
|
||||
Run `npx hyperframes doctor` to check which local dependencies are installed.
|
||||
The media workflows run `hyperframes auth status` before generation and tell
|
||||
@@ -154,7 +154,6 @@ to a shared space with `--space`.
|
||||
| `GEMINI_API_KEY` / `GOOGLE_API_KEY` | Explicit Gemini TTS selection and Lyria music generation; capture descriptions use `GEMINI_API_KEY`. |
|
||||
| `GOOGLE_APPLICATION_CREDENTIALS` / `GCS_CREDS` | Gemini TTS service-account JSON file path / injected JSON. Used when neither API key is set. |
|
||||
| `GOOGLE_CLOUD_PROJECT` / `GCLOUD_PROJECT_ID` | Gemini OAuth quota project; first set value wins, then the service account project. |
|
||||
| `OPENROUTER_API_KEY` | Capture descriptions; takes priority over Gemini for that step. |
|
||||
|
||||
## Related topics
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ platform limits.
|
||||
The Cloudflare template is a TanStack Start application on Workers. A
|
||||
`RenderContainer` Durable Object runs the renderer and streams the result to
|
||||
R2. The template also includes optional prompt-based generation with a user's
|
||||
own OpenRouter credential.
|
||||
own model-provider credential.
|
||||
|
||||
[Deploy the Cloudflare template](https://deploy.workers.cloudflare.com/?url=https://github.com/heygen-com/hyperframes-cloudflare-template)
|
||||
|
||||
|
||||
@@ -387,9 +387,7 @@ to those local files. Distinct URLs such as `demo.woff2?v=1` and
|
||||
URL unchanged.
|
||||
|
||||
For AI image descriptions, set `GEMINI_API_KEY` in a `.env` file
|
||||
(~$0.001/image), or `OPENROUTER_API_KEY` to route any vision model through
|
||||
[OpenRouter](https://openrouter.ai) — it wins if both are set, and
|
||||
`HYPERFRAMES_OPENROUTER_MODEL` overrides the model.
|
||||
(~$0.001/image).
|
||||
|
||||
#### Local HTML files
|
||||
|
||||
@@ -2515,7 +2513,7 @@ hyperframes auth logout --yes # no prompt
|
||||
| `HYPERFRAMES_OAUTH_CLIENT_ID` | Override the selected public OAuth client. |
|
||||
|
||||
For the keys other capabilities use — ElevenLabs and Gemini for voice and music
|
||||
fallback, OpenRouter and Gemini for capture — and how the skills prioritize
|
||||
fallback, Gemini for capture — and how the skills prioritize
|
||||
them, see [Authentication and API keys](/guides/authentication).
|
||||
|
||||
## hyperframes cloud
|
||||
|
||||
@@ -4,6 +4,17 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { captionImagesWithGemini } from "./contentExtractor.js";
|
||||
|
||||
const { generateContentMock } = vi.hoisted(() => ({ generateContentMock: vi.fn() }));
|
||||
|
||||
vi.mock("../utils/optionalPackages.js", () => ({
|
||||
loadSharp: async () => (await import("sharp")).default,
|
||||
loadOptionalPackage: vi.fn(async () => ({
|
||||
GoogleGenAI: class {
|
||||
models = { generateContent: generateContentMock };
|
||||
},
|
||||
})),
|
||||
}));
|
||||
|
||||
const hooks = vi.hoisted(() => ({
|
||||
afterStat: () => {},
|
||||
beforeRead: () => {},
|
||||
@@ -41,10 +52,6 @@ describe("caption image file reads", () => {
|
||||
let dir: string;
|
||||
let file: string;
|
||||
let sentBody: string;
|
||||
const fetchMock = vi.fn(async (_url: string, init?: RequestInit) => {
|
||||
sentBody = typeof init?.body === "string" ? init.body : "";
|
||||
return new Response(JSON.stringify({ choices: [{ message: { content: "A hero." } }] }));
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
dir = fs.mkdtempSync(join(tmpdir(), "hf-caption-read-"));
|
||||
@@ -56,15 +63,19 @@ describe("caption image file reads", () => {
|
||||
hooks.descriptor = -1;
|
||||
hooks.closed = [];
|
||||
sentBody = "";
|
||||
fetchMock.mockClear();
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "test-key");
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
generateContentMock.mockReset();
|
||||
generateContentMock.mockImplementation(async (request: unknown) => {
|
||||
sentBody = JSON.stringify(request);
|
||||
return { text: "A hero." };
|
||||
});
|
||||
vi.stubEnv("GEMINI_API_KEY", "test-key");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
hooks.afterStat = () => {};
|
||||
hooks.beforeRead = () => {};
|
||||
vi.unstubAllGlobals();
|
||||
vi.unstubAllEnvs();
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
@@ -92,7 +103,7 @@ describe("caption image file reads", () => {
|
||||
hooks.beforeRead = read;
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, []);
|
||||
expect(captions).toEqual(size === 4_000_000 ? { "hero.png": "A hero." } : {});
|
||||
expect(fetchMock).toHaveBeenCalledTimes(size === 4_000_000 ? 1 : 0);
|
||||
expect(generateContentMock).toHaveBeenCalledTimes(size === 4_000_000 ? 1 : 0);
|
||||
expect(read).toHaveBeenCalledTimes(size === 4_000_000 ? 1 : 0);
|
||||
expectClosed();
|
||||
});
|
||||
@@ -105,8 +116,8 @@ describe("caption image file reads", () => {
|
||||
else hooks.beforeRead = fail;
|
||||
const warnings: string[] = [];
|
||||
expect(await captionImagesWithGemini(dir, () => {}, warnings)).toEqual({});
|
||||
expect(warnings).toEqual(["OpenRouter vision failed for 1 asset(s); captions omitted."]);
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
expect(warnings).toEqual(["Gemini vision failed for 1 asset(s); captions omitted."]);
|
||||
expect(generateContentMock).not.toHaveBeenCalled();
|
||||
expectClosed();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -73,15 +73,10 @@ vi.mock("../utils/optionalPackages.js", () => ({
|
||||
})),
|
||||
}));
|
||||
|
||||
// These tests exercise the OpenRouter provider path only — it makes a plain
|
||||
// `fetch` call we can stub, with no native (`sharp`) or `@google/genai`
|
||||
// dependency. OpenRouter wins over Gemini when OPENROUTER_API_KEY is set, so we
|
||||
// don't need to clear the Gemini keys for the OpenRouter cases.
|
||||
|
||||
function makeProjectWithImages(files = ["hero.png"]): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "hf-caption-"));
|
||||
mkdirSync(join(dir, "assets"), { recursive: true });
|
||||
// Contents are irrelevant to the OpenRouter path (it just base64-encodes the
|
||||
// Contents are irrelevant to the provider path (it just base64-encodes the
|
||||
// bytes); only the .png extension matters for the image filter.
|
||||
for (const file of files) {
|
||||
writeFileSync(join(dir, "assets", file), Buffer.from([0x89, 0x50, 0x4e, 0x47]));
|
||||
@@ -89,184 +84,55 @@ function makeProjectWithImages(files = ["hero.png"]): string {
|
||||
return dir;
|
||||
}
|
||||
|
||||
describe("captionImagesWithGemini — OpenRouter provider", () => {
|
||||
describe("captionImagesWithGemini — request and outcome handling", () => {
|
||||
const dirs: string[] = [];
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
generateContentMock.mockReset();
|
||||
vi.unstubAllEnvs();
|
||||
for (const d of dirs) rmSync(d, { recursive: true, force: true });
|
||||
dirs.length = 0;
|
||||
});
|
||||
|
||||
it("captions via OpenRouter when OPENROUTER_API_KEY is set", async () => {
|
||||
function geminiOnly(): void {
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-test-key");
|
||||
vi.stubEnv("GOOGLE_API_KEY", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
}
|
||||
|
||||
it("captions the image through Gemini with the configured model", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-test-key");
|
||||
vi.stubEnv("HYPERFRAMES_OPENROUTER_MODEL", "google/gemini-3.1-flash-lite");
|
||||
|
||||
// Capture the request inside the mock, where the args are well-typed —
|
||||
// avoids casting `mock.calls` (and the repo's ban on `as` assertions).
|
||||
let capturedUrl: string | undefined;
|
||||
let capturedInit: RequestInit | undefined;
|
||||
const fetchMock = vi.fn(async (url: string, init?: RequestInit) => {
|
||||
capturedUrl = url;
|
||||
capturedInit = init;
|
||||
return new Response(
|
||||
JSON.stringify({ choices: [{ message: { content: "A dark hero with blue accents." } }] }),
|
||||
{ status: 200, headers: { "content-type": "application/json" } },
|
||||
);
|
||||
geminiOnly();
|
||||
vi.stubEnv("HYPERFRAMES_GEMINI_MODEL", "gemini-test-model");
|
||||
let request:
|
||||
| { model: string; contents: Array<{ parts: Array<{ inlineData?: unknown }> }> }
|
||||
| undefined;
|
||||
generateContentMock.mockImplementation(async (value: typeof request) => {
|
||||
request = value;
|
||||
return { text: "A dark hero with blue accents." };
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const warnings: string[] = [];
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, warnings);
|
||||
|
||||
expect(captions).toEqual({ "hero.png": "A dark hero with blue accents." });
|
||||
expect(warnings).toEqual([]);
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
expect(capturedUrl).toBe("https://openrouter.ai/api/v1/chat/completions");
|
||||
expect(new Headers(capturedInit?.headers).get("authorization")).toBe("Bearer or-test-key");
|
||||
const body = JSON.parse(typeof capturedInit?.body === "string" ? capturedInit.body : "{}");
|
||||
expect(body.model).toBe("google/gemini-3.1-flash-lite");
|
||||
const image = body.messages[0].content.find((p: { type: string }) => p.type === "image_url");
|
||||
expect(image?.image_url?.url).toMatch(/^data:image\/png;base64,/);
|
||||
});
|
||||
|
||||
it("degrades gracefully (no throw, no captions) when OpenRouter returns a non-OK status", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-bad-key");
|
||||
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(
|
||||
async () =>
|
||||
new Response("provider detail containing or-bad-key", {
|
||||
status: 401,
|
||||
statusText: "Unauthorized",
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
const warnings: string[] = [];
|
||||
let outcome: VisionCaptionOutcome | undefined;
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, warnings, {
|
||||
onOutcome: (value) => {
|
||||
outcome = value;
|
||||
},
|
||||
});
|
||||
|
||||
expect(captions).toEqual({});
|
||||
expect(warnings).toEqual(["OpenRouter vision failed for 1 asset(s); captions omitted."]);
|
||||
expect(warnings.join(" ")).not.toContain("or-bad-key");
|
||||
expect(outcome).toEqual({
|
||||
timedOutRequests: 0,
|
||||
failedRequests: 1,
|
||||
budgetExhausted: false,
|
||||
});
|
||||
if (!outcome) throw new Error("Expected vision caption outcome");
|
||||
expect(resolveVisionPhaseCompletion(outcome, 10_000)).toEqual({
|
||||
status: "degraded",
|
||||
reason: "provider-error",
|
||||
expect(generateContentMock).toHaveBeenCalledTimes(1);
|
||||
expect(request?.model).toBe("gemini-test-model");
|
||||
expect(request?.contents[0]?.parts[0]?.inlineData).toEqual({
|
||||
mimeType: "image/png",
|
||||
data: Buffer.from([0x89, 0x50, 0x4e, 0x47]).toString("base64"),
|
||||
});
|
||||
});
|
||||
|
||||
it("terminates captioning when the vision provider never responds", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-hanging-key");
|
||||
vi.stubEnv("HYPERFRAMES_VISION_TIMEOUT_MS", "20");
|
||||
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn((_url: string, init?: RequestInit) => {
|
||||
return new Promise<Response>((_resolve, reject) => {
|
||||
init?.signal?.addEventListener("abort", () => reject(init.signal?.reason), {
|
||||
once: true,
|
||||
});
|
||||
});
|
||||
}),
|
||||
);
|
||||
|
||||
const warnings: string[] = [];
|
||||
const result = await Promise.race([
|
||||
captionImagesWithGemini(dir, () => {}, warnings),
|
||||
new Promise<"hung">((resolve) => setTimeout(() => resolve("hung"), 250)),
|
||||
]);
|
||||
|
||||
expect(result).toEqual({});
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ label: "success JSON", status: 200, partialBody: '{"choices":[' },
|
||||
{ label: "failure text", status: 500, partialBody: "upstream failed: " },
|
||||
])(
|
||||
"terminates captioning when OpenRouter $label body never ends",
|
||||
async ({ status, partialBody }) => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-stalled-body-key");
|
||||
vi.stubEnv("HYPERFRAMES_VISION_TIMEOUT_MS", "20");
|
||||
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async () => {
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(new TextEncoder().encode(partialBody));
|
||||
},
|
||||
});
|
||||
return new Response(body, {
|
||||
status,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}),
|
||||
);
|
||||
|
||||
const warnings: string[] = [];
|
||||
let outcome: VisionCaptionOutcome | undefined;
|
||||
const result = await Promise.race([
|
||||
captionImagesWithGemini(dir, () => {}, warnings, {
|
||||
onOutcome: (value) => {
|
||||
outcome = value;
|
||||
},
|
||||
}),
|
||||
new Promise<"hung">((resolve) => setTimeout(() => resolve("hung"), 250)),
|
||||
]);
|
||||
|
||||
expect(result).toEqual({});
|
||||
expect(warnings).toEqual(["OpenRouter vision timed out for 1 asset(s); captions omitted."]);
|
||||
expect(outcome).toEqual({
|
||||
timedOutRequests: 1,
|
||||
failedRequests: 0,
|
||||
budgetExhausted: false,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps successful captions while reporting a failed sibling request", async () => {
|
||||
const dir = makeProjectWithImages(["failed.png", "success.png"]);
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-mixed-failure-key");
|
||||
|
||||
let requestCount = 0;
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async () => {
|
||||
requestCount++;
|
||||
if (requestCount === 1) {
|
||||
return new Response("upstream exploded with or-mixed-failure-key", {
|
||||
status: 500,
|
||||
statusText: "Internal Server Error",
|
||||
});
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({ choices: [{ message: { content: "Successful image." } }] }),
|
||||
{ status: 200, headers: { "content-type": "application/json" } },
|
||||
);
|
||||
}),
|
||||
);
|
||||
geminiOnly();
|
||||
generateContentMock
|
||||
.mockRejectedValueOnce(new Error("upstream exploded with gemini-test-key"))
|
||||
.mockResolvedValue({ text: "Successful image." });
|
||||
|
||||
const warnings: string[] = [];
|
||||
let outcome: VisionCaptionOutcome | undefined;
|
||||
@@ -277,8 +143,8 @@ describe("captionImagesWithGemini — OpenRouter provider", () => {
|
||||
});
|
||||
|
||||
expect(captions).toEqual({ "success.png": "Successful image." });
|
||||
expect(warnings).toEqual(["OpenRouter vision failed for 1 asset(s); captions omitted."]);
|
||||
expect(warnings.join(" ")).not.toContain("or-mixed-failure-key");
|
||||
expect(warnings).toEqual(["Gemini vision failed for 1 asset(s); captions omitted."]);
|
||||
expect(warnings.join(" ")).not.toContain("gemini-test-key");
|
||||
expect(outcome).toEqual({
|
||||
timedOutRequests: 0,
|
||||
failedRequests: 1,
|
||||
@@ -289,35 +155,15 @@ describe("captionImagesWithGemini — OpenRouter provider", () => {
|
||||
it("keeps a successful sibling caption when another request times out", async () => {
|
||||
const dir = makeProjectWithImages(["hang.png", "success.png"]);
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-mixed-key");
|
||||
geminiOnly();
|
||||
vi.stubEnv("HYPERFRAMES_VISION_TIMEOUT_MS", "20");
|
||||
// Both fixtures contain the same bytes, so use call order: the first request hangs.
|
||||
generateContentMock
|
||||
.mockImplementationOnce(() => new Promise(() => {}))
|
||||
.mockResolvedValue({ text: "Successful image." });
|
||||
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn((_url: string, init?: RequestInit) => {
|
||||
const body = JSON.parse(typeof init?.body === "string" ? init.body : "{}");
|
||||
const imageUrl = body.messages?.[0]?.content?.[1]?.image_url?.url;
|
||||
if (typeof imageUrl === "string" && imageUrl.includes("iVBORw==")) {
|
||||
// Both fixtures contain the same bytes, so use call order: the first
|
||||
// request hangs while the second succeeds independently.
|
||||
const fetchMock = vi.mocked(fetch);
|
||||
if (fetchMock.mock.calls.length === 1) return new Promise<Response>(() => {});
|
||||
}
|
||||
return Promise.resolve(
|
||||
new Response(
|
||||
JSON.stringify({ choices: [{ message: { content: "Successful image." } }] }),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
},
|
||||
),
|
||||
);
|
||||
}),
|
||||
);
|
||||
|
||||
const warnings: string[] = [];
|
||||
let outcome: VisionCaptionOutcome | undefined;
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, warnings, {
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, [], {
|
||||
onOutcome: (value) => {
|
||||
outcome = value;
|
||||
},
|
||||
@@ -339,35 +185,31 @@ describe("captionImagesWithGemini — OpenRouter provider", () => {
|
||||
it("does not call a configured provider when vision is skipped", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-unused-key");
|
||||
const fetchMock = vi.fn();
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
geminiOnly();
|
||||
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, [], { skipVision: true });
|
||||
|
||||
expect(captions).toEqual({});
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
expect(generateContentMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not start a vision batch after the capture budget is exhausted", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-budget-key");
|
||||
const fetchMock = vi.fn();
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
geminiOnly();
|
||||
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, [], {
|
||||
remainingMs: () => 0,
|
||||
});
|
||||
|
||||
expect(captions).toEqual({});
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
expect(generateContentMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("classifies non-provider pipeline failures without leaking the local path", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "hf-caption-no-assets-"));
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-unused-key");
|
||||
geminiOnly();
|
||||
|
||||
const warnings: string[] = [];
|
||||
let outcome: VisionCaptionOutcome | undefined;
|
||||
@@ -378,13 +220,13 @@ describe("captionImagesWithGemini — OpenRouter provider", () => {
|
||||
});
|
||||
|
||||
expect(captions).toEqual({});
|
||||
expect(warnings).toEqual(["OpenRouter captioning failed internally; captions omitted."]);
|
||||
expect(warnings).toEqual(["Gemini captioning failed internally; captions omitted."]);
|
||||
expect(warnings.join(" ")).not.toContain(dir);
|
||||
expect(outcome).toEqual({
|
||||
timedOutRequests: 0,
|
||||
failedRequests: 0,
|
||||
budgetExhausted: false,
|
||||
internalError: true,
|
||||
budgetExhausted: false,
|
||||
});
|
||||
if (!outcome) throw new Error("Expected vision caption outcome");
|
||||
expect(resolveVisionPhaseCompletion(outcome, 10_000)).toEqual({
|
||||
@@ -396,18 +238,15 @@ describe("captionImagesWithGemini — OpenRouter provider", () => {
|
||||
it("skips captioning entirely when no provider key is present", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "");
|
||||
vi.stubEnv("GEMINI_API_KEY", "");
|
||||
vi.stubEnv("GOOGLE_API_KEY", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
|
||||
const fetchMock = vi.fn();
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const warnings: string[] = [];
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, warnings);
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, []);
|
||||
|
||||
expect(captions).toEqual({});
|
||||
expect(fetchMock).not.toHaveBeenCalled();
|
||||
expect(generateContentMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -424,7 +263,6 @@ describe("captionImagesWithGemini — Gemini provider", () => {
|
||||
it("terminates captioning when Gemini never settles even if the SDK ignores abort", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "");
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-hanging-key");
|
||||
vi.stubEnv("HYPERFRAMES_VISION_TIMEOUT_MS", "20");
|
||||
generateContentMock.mockImplementation(() => new Promise(() => {}));
|
||||
@@ -440,7 +278,6 @@ describe("captionImagesWithGemini — Gemini provider", () => {
|
||||
it("reports a rejected Gemini request without leaking its error detail", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "");
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-secret-key");
|
||||
generateContentMock.mockRejectedValue(
|
||||
new Error("provider echoed gemini-secret-key in its response"),
|
||||
@@ -494,7 +331,6 @@ describe("captionImagesWithGemini — Vertex AI provider", () => {
|
||||
});
|
||||
|
||||
function vertexEnv(): void {
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "prefab-kit-000000");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", SERVICE_ACCOUNT);
|
||||
}
|
||||
@@ -585,7 +421,6 @@ describe("captionImagesWithGemini — Vertex AI provider", () => {
|
||||
it("skips captioning when the service account is unparseable, without echoing it", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "");
|
||||
vi.stubEnv("GEMINI_API_KEY", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "prefab-kit-000000");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "{not-json super-secret-material");
|
||||
@@ -610,28 +445,9 @@ describe("captionImagesWithGemini — Vertex AI provider", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("stays on OpenRouter when the user has opted into it explicitly", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vertexEnv();
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-key");
|
||||
const fetchMock = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({ choices: [{ message: { content: "A caption." } }] }),
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
await captionImagesWithGemini(dir, () => {}, []);
|
||||
|
||||
expect(fetchMock).toHaveBeenCalled();
|
||||
expect(clientOptions).toHaveLength(0);
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it("needs both halves of the credential before it will use Vertex", async () => {
|
||||
const dir = makeProjectWithImages();
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "");
|
||||
vi.stubEnv("GEMINI_API_KEY", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "prefab-kit-000000");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
@@ -683,22 +499,15 @@ describe("captionImagesWithGemini — SVG rasterization", () => {
|
||||
// makes "one at a time" the assertion, not "errors are reported".
|
||||
const dir = makeProjectWithSvgs(6);
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-key");
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async () => {
|
||||
// Slower than a render, so overlapping renders would be the easy way to go faster —
|
||||
// the point is that the loop does not take it.
|
||||
await new Promise((resolve) => setTimeout(resolve, 15));
|
||||
return new Response(
|
||||
JSON.stringify({ choices: [{ message: { content: "A dark glyph." } }] }),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
},
|
||||
);
|
||||
}),
|
||||
);
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-key");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
generateContentMock.mockImplementation(async () => {
|
||||
// Slower than a render, so overlapping renders would be the easy way to go faster —
|
||||
// the point is that the loop does not take it.
|
||||
await new Promise((resolve) => setTimeout(resolve, 15));
|
||||
return { text: "A dark glyph." };
|
||||
});
|
||||
|
||||
const captions = await captionImagesWithGemini(dir, () => {}, []);
|
||||
|
||||
@@ -714,17 +523,10 @@ describe("captionImagesWithGemini — SVG rasterization", () => {
|
||||
// which asked for captioning, so the bound has to be given back when the renders are done.
|
||||
const dir = makeProjectWithSvgs(1);
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-key");
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(
|
||||
async () =>
|
||||
new Response(JSON.stringify({ choices: [{ message: { content: "A glyph." } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
),
|
||||
);
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-key");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
generateContentMock.mockResolvedValue({ text: "A glyph." });
|
||||
|
||||
await captionImagesWithGemini(dir, () => {}, []);
|
||||
|
||||
@@ -737,17 +539,10 @@ describe("captionImagesWithGemini — SVG rasterization", () => {
|
||||
// its threads.
|
||||
const dir = makeProjectWithSvgs(2);
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-key");
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(
|
||||
async () =>
|
||||
new Response(JSON.stringify({ choices: [{ message: { content: "A glyph." } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
),
|
||||
);
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-key");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
generateContentMock.mockResolvedValue({ text: "A glyph." });
|
||||
// Unreadable, which is how an exotic SVG behaves through sharp.
|
||||
rmSync(join(dir, "assets", "svgs", "logo-0.svg"));
|
||||
mkdirSync(join(dir, "assets", "svgs", "logo-0.svg"));
|
||||
@@ -760,17 +555,10 @@ describe("captionImagesWithGemini — SVG rasterization", () => {
|
||||
it("keeps captioning the rest when one SVG cannot be rasterized", async () => {
|
||||
const dir = makeProjectWithSvgs(3);
|
||||
dirs.push(dir);
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "or-key");
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(
|
||||
async () =>
|
||||
new Response(JSON.stringify({ choices: [{ message: { content: "A glyph." } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
),
|
||||
);
|
||||
vi.stubEnv("GEMINI_API_KEY", "gemini-key");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_PROJECT_ID", "");
|
||||
vi.stubEnv("HYPERFRAMES_VERTEX_SERVICE_ACCOUNT", "");
|
||||
generateContentMock.mockResolvedValue({ text: "A glyph." });
|
||||
// One file is unreadable, which is how an exotic SVG behaves through sharp.
|
||||
rmSync(join(dir, "assets", "svgs", "logo-1.svg"));
|
||||
mkdirSync(join(dir, "assets", "svgs", "logo-1.svg"));
|
||||
|
||||
@@ -245,9 +245,8 @@ export async function extractVisibleText(page: Page): Promise<string> {
|
||||
/**
|
||||
* Caption downloaded images using a vision model.
|
||||
*
|
||||
* Provider is chosen by which API key is present: OPENROUTER_API_KEY → OpenRouter
|
||||
* (any vision model via its OpenAI-style API), else GEMINI_API_KEY/GOOGLE_API_KEY
|
||||
* → Google Gemini, else no captioning. OpenRouter wins if both are set.
|
||||
* Provider is chosen by which credential is present: Vertex service account, else
|
||||
* GEMINI_API_KEY/GOOGLE_API_KEY → Google Gemini, else no captioning.
|
||||
*
|
||||
* Batches requests to stay under free-tier rate limits.
|
||||
* Returns a map of filename -> caption string.
|
||||
@@ -277,7 +276,6 @@ export async function captionImagesWithGemini(
|
||||
reportOutcome();
|
||||
return geminiCaptions;
|
||||
}
|
||||
const openRouterKey = process.env.OPENROUTER_API_KEY;
|
||||
const geminiKey = process.env.GEMINI_API_KEY || process.env.GOOGLE_API_KEY;
|
||||
// Vertex authenticates with a service account and a project instead of an API key. Server
|
||||
// deployments have those; they often do not have a working Gemini API key, and a rejected
|
||||
@@ -286,28 +284,20 @@ export async function captionImagesWithGemini(
|
||||
const vertexProject = process.env.HYPERFRAMES_VERTEX_PROJECT_ID;
|
||||
const vertexServiceAccount = process.env.HYPERFRAMES_VERTEX_SERVICE_ACCOUNT;
|
||||
const useVertex = Boolean(vertexProject && vertexServiceAccount);
|
||||
if (!openRouterKey && !useVertex && !geminiKey) {
|
||||
if (!useVertex && !geminiKey) {
|
||||
reportOutcome();
|
||||
return geminiCaptions;
|
||||
}
|
||||
|
||||
// OpenRouter takes priority — it's the explicit opt-in for users without Google access.
|
||||
// Vertex outranks the bare API key because it is the credential a deployment actually
|
||||
// holds. All three satisfy the same single-image → one-line-caption contract
|
||||
// holds. Both satisfy the same single-image → one-line-caption contract
|
||||
// (`captionOne`), so the batching and SVG-rasterization loops stay provider-agnostic.
|
||||
const provider: "openrouter" | "vertex" | "gemini" = openRouterKey
|
||||
? "openrouter"
|
||||
: useVertex
|
||||
? "vertex"
|
||||
: "gemini";
|
||||
const providerName = { openrouter: "OpenRouter", vertex: "Vertex AI", gemini: "Gemini" }[
|
||||
provider
|
||||
];
|
||||
// Override per provider via HYPERFRAMES_OPENROUTER_MODEL / HYPERFRAMES_VERTEX_MODEL /
|
||||
// HYPERFRAMES_GEMINI_MODEL. Vertex publishes a different model set than the Gemini API —
|
||||
// the API's flash-lite preview id is not resolvable there — so it carries its own default.
|
||||
const provider: "vertex" | "gemini" = useVertex ? "vertex" : "gemini";
|
||||
const providerName = { vertex: "Vertex AI", gemini: "Gemini" }[provider];
|
||||
// Override per provider via HYPERFRAMES_VERTEX_MODEL / HYPERFRAMES_GEMINI_MODEL. Vertex
|
||||
// publishes a different model set than the Gemini API (the API's flash-lite preview id is
|
||||
// not resolvable there), so it carries its own default.
|
||||
const model = {
|
||||
openrouter: process.env.HYPERFRAMES_OPENROUTER_MODEL || "google/gemini-3.1-flash-lite",
|
||||
vertex: process.env.HYPERFRAMES_VERTEX_MODEL || "gemini-2.5-flash",
|
||||
gemini: process.env.HYPERFRAMES_GEMINI_MODEL || "gemini-3.1-flash-lite-preview",
|
||||
}[provider];
|
||||
@@ -326,114 +316,72 @@ export async function captionImagesWithGemini(
|
||||
}) => Promise<string>;
|
||||
|
||||
let captionOne: CaptionOne;
|
||||
if (provider === "openrouter") {
|
||||
captionOne = async ({ mimeType, base64, prompt, maxTokens, timeoutMs }) => {
|
||||
return runBoundedVisionRequest(async (signal) => {
|
||||
const res = await fetch("https://openrouter.ai/api/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${openRouterKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
signal,
|
||||
body: JSON.stringify({
|
||||
model,
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: prompt },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: `data:${mimeType};base64,${base64}` },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
max_tokens: maxTokens,
|
||||
}),
|
||||
});
|
||||
if (!res.ok) {
|
||||
await res.text();
|
||||
throw new Error(`OpenRouter request failed with HTTP ${res.status}`);
|
||||
}
|
||||
const data = (await res.json()) as {
|
||||
choices?: Array<{ message?: { content?: string } }>;
|
||||
};
|
||||
return data.choices?.[0]?.message?.content?.trim() || "";
|
||||
}, timeoutMs);
|
||||
};
|
||||
} else {
|
||||
let GoogleGenAI: OptionalPackageModules["@google/genai"]["GoogleGenAI"];
|
||||
let GoogleGenAI: OptionalPackageModules["@google/genai"]["GoogleGenAI"];
|
||||
try {
|
||||
({ GoogleGenAI } = await loadOptionalPackage("@google/genai", "vision captioning"));
|
||||
} catch (err) {
|
||||
warnings.push(`Skipped vision captioning: ${(err as Error).message}`);
|
||||
internalError = true;
|
||||
reportOutcome();
|
||||
return geminiCaptions;
|
||||
}
|
||||
let ai: InstanceType<typeof GoogleGenAI>;
|
||||
if (provider === "vertex") {
|
||||
// Re-narrow for TS; `useVertex` already guaranteed both are set.
|
||||
if (!vertexProject || !vertexServiceAccount) return geminiCaptions;
|
||||
let credentials: Record<string, unknown>;
|
||||
try {
|
||||
({ GoogleGenAI } = await loadOptionalPackage("@google/genai", "vision captioning"));
|
||||
} catch (err) {
|
||||
warnings.push(`Skipped vision captioning: ${(err as Error).message}`);
|
||||
credentials = JSON.parse(vertexServiceAccount) as Record<string, unknown>;
|
||||
} catch {
|
||||
warnings.push(
|
||||
"HYPERFRAMES_VERTEX_SERVICE_ACCOUNT is not valid JSON; skipped vision captioning.",
|
||||
);
|
||||
internalError = true;
|
||||
reportOutcome();
|
||||
return geminiCaptions;
|
||||
}
|
||||
let ai: InstanceType<typeof GoogleGenAI>;
|
||||
if (provider === "vertex") {
|
||||
// Re-narrow for TS; `useVertex` already guaranteed both are set.
|
||||
if (!vertexProject || !vertexServiceAccount) return geminiCaptions;
|
||||
let credentials: Record<string, unknown>;
|
||||
try {
|
||||
credentials = JSON.parse(vertexServiceAccount) as Record<string, unknown>;
|
||||
} catch {
|
||||
warnings.push(
|
||||
"HYPERFRAMES_VERTEX_SERVICE_ACCOUNT is not valid JSON; skipped vision captioning.",
|
||||
);
|
||||
internalError = true;
|
||||
reportOutcome();
|
||||
return geminiCaptions;
|
||||
}
|
||||
ai = new GoogleGenAI({
|
||||
vertexai: true,
|
||||
project: vertexProject,
|
||||
location: process.env.HYPERFRAMES_VERTEX_LOCATION || "us-central1",
|
||||
googleAuthOptions: { credentials },
|
||||
});
|
||||
} else {
|
||||
// Unreachable when geminiKey is unset (guarded above); re-narrow for TS.
|
||||
if (!geminiKey) return geminiCaptions;
|
||||
ai = new GoogleGenAI({ apiKey: geminiKey });
|
||||
}
|
||||
captionOne = async ({ mimeType, base64, prompt, maxTokens, timeoutMs }) => {
|
||||
const response = await runBoundedVisionRequest(
|
||||
(signal) =>
|
||||
ai.models.generateContent({
|
||||
model,
|
||||
contents: [
|
||||
{
|
||||
role: "user",
|
||||
parts: [{ inlineData: { mimeType, data: base64 } }, { text: prompt }],
|
||||
},
|
||||
],
|
||||
config: {
|
||||
maxOutputTokens: maxTokens,
|
||||
// A one-line factual caption needs no reasoning, and leaving thinking on is
|
||||
// not merely wasteful: thinking tokens are drawn from maxOutputTokens, so the
|
||||
// model can spend the whole budget and return empty text. That surfaces as a
|
||||
// successful request with no caption — the capture then logs "Captioned N/N
|
||||
// images" followed by "0 images captioned", which is what production shows.
|
||||
thinkingConfig: { thinkingBudget: 0 },
|
||||
abortSignal: signal,
|
||||
httpOptions: { timeout: timeoutMs },
|
||||
},
|
||||
}),
|
||||
timeoutMs,
|
||||
);
|
||||
return response.text?.trim() || "";
|
||||
};
|
||||
ai = new GoogleGenAI({
|
||||
vertexai: true,
|
||||
project: vertexProject,
|
||||
location: process.env.HYPERFRAMES_VERTEX_LOCATION || "us-central1",
|
||||
googleAuthOptions: { credentials },
|
||||
});
|
||||
} else {
|
||||
// Unreachable when geminiKey is unset (guarded above); re-narrow for TS.
|
||||
if (!geminiKey) return geminiCaptions;
|
||||
ai = new GoogleGenAI({ apiKey: geminiKey });
|
||||
}
|
||||
captionOne = async ({ mimeType, base64, prompt, maxTokens, timeoutMs }) => {
|
||||
const response = await runBoundedVisionRequest(
|
||||
(signal) =>
|
||||
ai.models.generateContent({
|
||||
model,
|
||||
contents: [
|
||||
{
|
||||
role: "user",
|
||||
parts: [{ inlineData: { mimeType, data: base64 } }, { text: prompt }],
|
||||
},
|
||||
],
|
||||
config: {
|
||||
maxOutputTokens: maxTokens,
|
||||
// Thinking tokens come out of maxOutputTokens, so a thinking model can spend the
|
||||
// whole budget and return empty text: a successful request with no caption.
|
||||
thinkingConfig: { thinkingBudget: 0 },
|
||||
abortSignal: signal,
|
||||
httpOptions: { timeout: timeoutMs },
|
||||
},
|
||||
}),
|
||||
timeoutMs,
|
||||
);
|
||||
return response.text?.trim() || "";
|
||||
};
|
||||
|
||||
const imageFiles = readdirSync(join(outputDir, "assets")).filter((f: string) =>
|
||||
/\.(png|jpg|jpeg|webp|gif)$/i.test(f),
|
||||
);
|
||||
|
||||
// Caption in parallel batches. Gemini free tier is ~5 RPM (slow but $0),
|
||||
// paid/OpenRouter ~2000 RPM. We batch 20 with a 2s inter-batch pause and rely
|
||||
// paid ~2000 RPM. We batch 20 with a 2s inter-batch pause and rely
|
||||
// on Promise.allSettled so one rate-limited image does not fail its siblings;
|
||||
// rejected requests are aggregated into a sanitized warning below.
|
||||
const BATCH_SIZE = 20;
|
||||
|
||||
@@ -34,8 +34,7 @@ import type { VisionCaptionOutcome } from "./contentExtractor.js";
|
||||
function hasVisionCredentials(skipVision: boolean): boolean {
|
||||
return Boolean(
|
||||
!skipVision &&
|
||||
(process.env.OPENROUTER_API_KEY ||
|
||||
process.env.GEMINI_API_KEY ||
|
||||
(process.env.GEMINI_API_KEY ||
|
||||
process.env.GOOGLE_API_KEY ||
|
||||
(process.env.HYPERFRAMES_VERTEX_PROJECT_ID &&
|
||||
process.env.HYPERFRAMES_VERTEX_SERVICE_ACCOUNT)),
|
||||
|
||||
@@ -206,13 +206,11 @@ describe("loadEnvFile", () => {
|
||||
|
||||
it("loads supported vision provider settings without replacing inherited credentials", () => {
|
||||
const settings = {
|
||||
OPENROUTER_API_KEY: "router-key",
|
||||
GOOGLE_API_KEY: "google-key",
|
||||
HYPERFRAMES_VERTEX_PROJECT_ID: "example-project",
|
||||
HYPERFRAMES_VERTEX_SERVICE_ACCOUNT: '{"client_email":"test@example.com"}',
|
||||
HYPERFRAMES_VERTEX_LOCATION: "us-central1",
|
||||
HYPERFRAMES_VERTEX_MODEL: "vertex-model",
|
||||
HYPERFRAMES_OPENROUTER_MODEL: "router-model",
|
||||
HYPERFRAMES_GEMINI_MODEL: "gemini-model",
|
||||
HYPERFRAMES_VISION_TIMEOUT_MS: "2000",
|
||||
};
|
||||
|
||||
@@ -13,11 +13,8 @@ describe("applyDotEnv", () => {
|
||||
|
||||
it("reads a comment after an empty value as no value", () => {
|
||||
const env: NodeJS.ProcessEnv = {};
|
||||
applyDotEnv(
|
||||
"GEMINI_API_KEY= # paste your key here\nOPENROUTER_API_KEY=\t# later\nHASH=#abc",
|
||||
env,
|
||||
);
|
||||
expect(env).toEqual({ GEMINI_API_KEY: "", OPENROUTER_API_KEY: "", HASH: "#abc" });
|
||||
applyDotEnv("GEMINI_API_KEY= # paste your key here\nGOOGLE_API_KEY=\t# later\nHASH=#abc", env);
|
||||
expect(env).toEqual({ GEMINI_API_KEY: "", GOOGLE_API_KEY: "", HASH: "#abc" });
|
||||
});
|
||||
|
||||
it("never takes the feedback email from a project file", () => {
|
||||
@@ -79,7 +76,7 @@ describe("applyDotEnv quoted multiline values", () => {
|
||||
|
||||
it("a commented-out example with an unclosed quote does not swallow the real keys below it", () => {
|
||||
const env: NodeJS.ProcessEnv = {};
|
||||
applyDotEnv('# OPENROUTER_API_KEY="sk-old\nGEMINI_API_KEY=real1\nGOOGLE_API_KEY=real2', env);
|
||||
applyDotEnv('# FIGMA_TOKEN="sk-old\nGEMINI_API_KEY=real1\nGOOGLE_API_KEY=real2', env);
|
||||
expect(env).toEqual({ GEMINI_API_KEY: "real1", GOOGLE_API_KEY: "real2" });
|
||||
});
|
||||
|
||||
@@ -133,7 +130,7 @@ describe("loadProjectDotEnv", () => {
|
||||
});
|
||||
|
||||
it("says nothing when every key is one the CLI reads, or there is no .env", () => {
|
||||
const dir = projectWith("OPENROUTER_API_KEY=k\n");
|
||||
const dir = projectWith("GOOGLE_API_KEY=k\n");
|
||||
try {
|
||||
const warn = vi.fn();
|
||||
loadProjectDotEnv(dir, {}, warn);
|
||||
|
||||
@@ -7,14 +7,12 @@ export const PROJECT_ENV_KEYS = [
|
||||
"HYPERFRAMES_API_KEY",
|
||||
"ELEVENLABS_API_KEY",
|
||||
"FIGMA_TOKEN",
|
||||
"OPENROUTER_API_KEY",
|
||||
"GEMINI_API_KEY",
|
||||
"GOOGLE_API_KEY",
|
||||
"HYPERFRAMES_VERTEX_PROJECT_ID",
|
||||
"HYPERFRAMES_VERTEX_SERVICE_ACCOUNT",
|
||||
"HYPERFRAMES_VERTEX_LOCATION",
|
||||
"HYPERFRAMES_VERTEX_MODEL",
|
||||
"HYPERFRAMES_OPENROUTER_MODEL",
|
||||
"HYPERFRAMES_GEMINI_MODEL",
|
||||
"HYPERFRAMES_VISION_TIMEOUT_MS",
|
||||
// Telemetry opt-outs only turn sending off, and a project .env could already set them.
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"files": 2
|
||||
},
|
||||
"hyperframes": {
|
||||
"hash": "ceaef1d1899f6fb7",
|
||||
"hash": "8e36e506a04c1bac",
|
||||
"files": 77
|
||||
},
|
||||
"hyperframes-cli": {
|
||||
|
||||
@@ -13,7 +13,7 @@ What is specific to HyperFrames, or was measured on shipped films, when one vide
|
||||
|
||||
- `npx hyperframes capture <url> --json` writes `screenshots/full-page.png`: a 1x plate of the whole page, taken after the scroll pass with sticky and fixed elements made static, pixel-exact for a 1920-wide viewport travelling down it. It is skipped above 16384 px tall (use `scroll-*.png`); pushing in past 1:1 needs its own 2x capture of that region.
|
||||
- To show the site as it is, use its screenshots. Rebuilding the page in HTML drifts from the real layout; rebuild only the part that moves, or overlay captured assets at measured positions.
|
||||
- `--capture-budget` (default 120000 ms) bounds work after navigation; `--timeout` bounds navigation only. Videos are not downloaded: `capture --video <dir> --list`, then `--index N`. `asset-descriptions.md` has vision captions only when `OPENROUTER_API_KEY`, `GEMINI_API_KEY` or `GOOGLE_API_KEY` is set.
|
||||
- `--capture-budget` (default 120000 ms) bounds work after navigation; `--timeout` bounds navigation only. Videos are not downloaded: `capture --video <dir> --list`, then `--index N`. `asset-descriptions.md` has vision captions only when `GEMINI_API_KEY` or `GOOGLE_API_KEY` is set.
|
||||
- "very little text content" with an empty asset catalog means the capture is unusable even when it says `ok: true`. A `logo-*.svg` name is a guess from page structure, not proof.
|
||||
- Brand tokens: link blue `#0000EE` and status reds and greens are never the accent; the accent is the colour that recurs across the most roles. Icon fonts (Font Awesome, icomoon, webflow-icons) are never the display or body face; a captured monospace goes to the mono role. Use only weights the brand font ships, or 600/700 render as fake bold. Every preset has a light ground: for a dark brand, swap ground and ink.
|
||||
- Capture keeps each `@font-face` rule, `unicode-range` subsets included, in `extracted/page.html`; copying the font files alone loses which characters each covers. Name copied fonts by family, weight and style, or an upright and an italic "Regular" collide.
|
||||
|
||||
Reference in New Issue
Block a user