mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
feat(cli): make cloud archives size-aware
This commit is contained in:
James
@@ -70,6 +70,7 @@
|
||||
"fontkit": "^2.0.4",
|
||||
"giget": "^3.2.0",
|
||||
"hono": "^4.0.0",
|
||||
"ignore": "^5.3.2",
|
||||
"onnxruntime-node": "1.23.2",
|
||||
"open": "^10.0.0",
|
||||
"postcss": "^8.5.8",
|
||||
|
||||
+27
-5
@@ -71,9 +71,9 @@ The credential is **shared with the [`heygen` CLI](https://github.com/heygen-com
|
||||
```
|
||||
Your machine HeyGen cloud
|
||||
┌─────────────────────────┐ ┌─────────────────────────────────┐
|
||||
│ zip project │ ──POST──▶│ /v3/assets │
|
||||
│ (excludes .git, │ upload │ → asset_id │
|
||||
│ node_modules, dist…) │ │ │
|
||||
│ zip project │ ──PUT───▶│ direct-to-S3 asset upload │
|
||||
│ (.hyperframesignore + │ upload │ → asset_id │
|
||||
│ generated outputs) │ │ │
|
||||
│ │ ──POST──▶│ /v3/hyperframes/renders │
|
||||
│ │ submit │ → render_id (queued) │
|
||||
│ │ │ Chromium + FFmpeg render │
|
||||
@@ -84,12 +84,33 @@ The credential is **shared with the [`heygen` CLI](https://github.com/heygen-com
|
||||
|
||||
1. **Resolve the project** — a local directory (default `.`), or skip the upload with `--asset-id` / `--url`.
|
||||
2. **Auto-detect the aspect ratio** from the entry HTML's `data-width`/`data-height` so you rarely set it by hand.
|
||||
3. **Zip** the project (same ignore set as `hyperframes publish`).
|
||||
4. **Upload** the zip to `POST /v3/assets`, yielding an `asset_id`.
|
||||
3. **Zip** the project (same ignore set as `hyperframes publish`, including `.hyperframesignore`).
|
||||
4. **Upload** the zip through the direct-to-S3 asset flow, yielding an `asset_id`.
|
||||
5. **Submit** the render to `POST /v3/hyperframes/renders`.
|
||||
6. **Poll** `GET /v3/hyperframes/renders/{id}` until it completes or fails (skip with `--no-wait`).
|
||||
7. **Download** the signed video URL to disk.
|
||||
|
||||
## Control archive size
|
||||
|
||||
Hosted cloud project uploads are limited to 200 MB. HyperFrames automatically excludes root-level `renders/` and `snapshots/` plus development-only paths such as `.git`, `node_modules`, `dist`, `.next`, `coverage`, and dotfiles.
|
||||
|
||||
Use a gitignore-style `.hyperframesignore` at the project root for additional generated or intermediate files that are not needed at render time:
|
||||
|
||||
```gitignore
|
||||
/snapshots2/
|
||||
/exports/
|
||||
/assets/source-master.mp4
|
||||
```
|
||||
|
||||
Inspect the exact archive without authenticating, uploading, or starting a render:
|
||||
|
||||
```bash
|
||||
hyperframes cloud render . --dry-run
|
||||
hyperframes cloud render . --dry-run --json
|
||||
```
|
||||
|
||||
The dry run reports compressed size, file count, and the ten largest included files. Rules also apply to `hyperframes publish`. Avoid broad patterns such as `assets/`: dynamically selected media may not appear as an obvious static HTML reference.
|
||||
|
||||
## Render options
|
||||
|
||||
The most-used flags — see the [CLI reference](/packages/cli#hyperframes-cloud) for the full list.
|
||||
@@ -103,6 +124,7 @@ The most-used flags — see the [CLI reference](/packages/cli#hyperframes-cloud)
|
||||
| `--aspect-ratio` | auto | `16:9`, `9:16`, or `1:1`. Auto-detected from a local project's `data-width`/`data-height`; for `--asset-id`/`--url` it defaults to `16:9` unless set. |
|
||||
| `--composition` / `-c` | `index.html` | Entry HTML file inside the zip. |
|
||||
| `--output` / `-o` | `renders/<render_id>.<ext>` | Local destination for the download. |
|
||||
| `--dry-run` | off | Build and inspect a local project zip without authenticating, uploading, or rendering. |
|
||||
|
||||
```bash
|
||||
# Pick a composition and an output path.
|
||||
|
||||
@@ -1063,7 +1063,9 @@ hyperframes cloud list # browse recent renders
|
||||
|
||||
#### `cloud render [<projectDir>]`
|
||||
|
||||
End-to-end render: zips the project (excluding `.git`, `node_modules`, `dist`, `.next`, `coverage`, dotfiles), uploads it via `POST /v3/assets`, submits `POST /v3/hyperframes/renders`, polls `GET /v3/hyperframes/renders/{id}` until the render completes or fails, and streams the resulting video to disk.
|
||||
End-to-end render: zips the project (excluding root `renders`/`snapshots`, `.git`, `node_modules`, `dist`, `.next`, `coverage`, dotfiles, and project `.hyperframesignore` rules), uploads it through the direct-to-S3 asset flow, submits `POST /v3/hyperframes/renders`, polls `GET /v3/hyperframes/renders/{id}` until the render completes or fails, and streams the resulting video to disk.
|
||||
|
||||
Use `hyperframes cloud render --dry-run` to inspect the compressed size and largest included files without authenticating, uploading, or starting a render. Project-specific `.hyperframesignore` rules use gitignore syntax and also apply to `hyperframes publish`; keep rules narrow because dynamically selected assets may not have obvious static references.
|
||||
|
||||
Render parameters mirror the local `hyperframes render` UX where they overlap:
|
||||
|
||||
@@ -1080,6 +1082,7 @@ Render parameters mirror the local `hyperframes render` UX where they overlap:
|
||||
| `--strict-variables` | off | Fail when variables are undeclared or have the wrong type. |
|
||||
| `--title` | — | Free-text label echoed back in detail responses. |
|
||||
| `--output` / `-o` | `renders/<render_id>.<ext>` | Local destination for the downloaded video. |
|
||||
| `--dry-run` | off | Build and inspect a local project zip without authenticating, uploading, or rendering. |
|
||||
|
||||
Lifecycle / control flags:
|
||||
|
||||
|
||||
@@ -39,6 +39,7 @@
|
||||
"fontkit": "^2.0.4",
|
||||
"giget": "^3.2.0",
|
||||
"hono": "^4.0.0",
|
||||
"ignore": "^5.3.2",
|
||||
"onnxruntime-node": "1.23.2",
|
||||
"open": "^10.0.0",
|
||||
"postcss": "^8.5.8",
|
||||
|
||||
@@ -50,6 +50,20 @@ describe("cloud/errors reportApiError", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("points oversized projects to dry-run and .hyperframesignore", () => {
|
||||
const err = new HyperframesApiError({
|
||||
status: 413,
|
||||
message: "project too large",
|
||||
code: "hyperframes_project_too_large",
|
||||
});
|
||||
expect(() => reportApiError("Upload failed", err)).toThrow("process.exit called");
|
||||
expect(errorBox).toHaveBeenCalledWith(
|
||||
"Upload failed (HTTP 413)",
|
||||
"project too large",
|
||||
"The zip exceeded the 200 MB limit. Run `hyperframes cloud render --dry-run`, then add only verified-unneeded paths to `.hyperframesignore` or pre-host required large media.",
|
||||
);
|
||||
});
|
||||
|
||||
it("prefers the code-specific hint over a caller suggestion", () => {
|
||||
const err = new HyperframesApiError({
|
||||
status: 400,
|
||||
|
||||
@@ -19,7 +19,7 @@ import { HyperframesApiError } from "./_gen/client.js";
|
||||
*/
|
||||
const ERROR_CODE_HINTS: Record<string, string> = {
|
||||
hyperframes_project_too_large:
|
||||
"The zip exceeded the 200 MB limit. Trim large media (or pre-host them and reference by URL), then try again.",
|
||||
"The zip exceeded the 200 MB limit. Run `hyperframes cloud render --dry-run`, then add only verified-unneeded paths to `.hyperframesignore` or pre-host required large media.",
|
||||
hyperframes_render_not_found:
|
||||
"The render_id no longer exists — either soft-deleted or never created.",
|
||||
invalid_parameter:
|
||||
|
||||
@@ -18,6 +18,7 @@ import { c } from "../ui/colors.js";
|
||||
export const examples: Example[] = [
|
||||
["Render the current directory in the cloud", "hyperframes cloud render"],
|
||||
["Render a specific project", "hyperframes cloud render ./my-video"],
|
||||
["Inspect the project zip without uploading", "hyperframes cloud render --dry-run"],
|
||||
[
|
||||
"Render at 60fps + high quality, save to a path",
|
||||
"hyperframes cloud render ./my-video --fps 60 --quality high -o ./out.mp4",
|
||||
|
||||
@@ -1,14 +1,19 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { mkdtempSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { join, resolve } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import {
|
||||
resolveAspectRatioForSubmit,
|
||||
validateDryRunSource,
|
||||
validateResolutionFormatCombo,
|
||||
type ProjectInputSource,
|
||||
} from "./render.js";
|
||||
|
||||
const cliEntry = resolve(fileURLToPath(import.meta.url), "..", "..", "..", "cli.ts");
|
||||
|
||||
// errorBox writes to console; silence it so test output stays clean.
|
||||
beforeEach(() => {
|
||||
vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
@@ -51,6 +56,76 @@ describe("validateResolutionFormatCombo", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("validateDryRunSource", () => {
|
||||
it("accepts a local directory and rejects already-uploaded sources", () => {
|
||||
const exit = trapExit();
|
||||
expect(() => validateDryRunSource({ kind: "dir", dir: "." }, true)).not.toThrow();
|
||||
expect(() => validateDryRunSource({ kind: "asset_id", assetId: "asst_123" }, true)).toThrow(
|
||||
"process.exit:1",
|
||||
);
|
||||
expect(() =>
|
||||
validateDryRunSource({ kind: "url", url: "https://example.com/project.zip" }, true),
|
||||
).toThrow("process.exit:1");
|
||||
expect(exit).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("cloud render --dry-run", () => {
|
||||
it("reports the real archive without contacting the cloud", () => {
|
||||
const dir = writeComposition(1920, 1080);
|
||||
try {
|
||||
writeFileSync(
|
||||
join(dir, "index.html"),
|
||||
`<!doctype html><html data-composition-variables='[{"id":"title","type":"string","label":"Title","default":"x"}]'><body><div data-composition-id="main" data-width="1920" data-height="1080"></div></body></html>`,
|
||||
"utf-8",
|
||||
);
|
||||
writeFileSync(join(dir, "asset.bin"), "archive-input", "utf-8");
|
||||
const result = spawnSync(
|
||||
"bun",
|
||||
[
|
||||
"run",
|
||||
cliEntry,
|
||||
"cloud",
|
||||
"render",
|
||||
dir,
|
||||
"--dry-run",
|
||||
"--json",
|
||||
"--variables",
|
||||
'{"extra":1}',
|
||||
],
|
||||
{
|
||||
encoding: "utf-8",
|
||||
timeout: 30_000,
|
||||
env: {
|
||||
...process.env,
|
||||
CI: "1",
|
||||
HEYGEN_API_URL: "http://127.0.0.1:1",
|
||||
HYPERFRAMES_NO_UPDATE_CHECK: "1",
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
expect(result.status).toBe(0);
|
||||
const payload = JSON.parse(result.stdout) as {
|
||||
archive: {
|
||||
file_count: number;
|
||||
size_bytes: number;
|
||||
upload_limit_bytes: number;
|
||||
exceeds_upload_limit: boolean;
|
||||
largest_files: Array<{ path: string }>;
|
||||
};
|
||||
};
|
||||
expect(payload.archive.file_count).toBe(2);
|
||||
expect(payload.archive.size_bytes).toBeGreaterThan(0);
|
||||
expect(payload.archive.upload_limit_bytes).toBe(200 * 1024 * 1024);
|
||||
expect(payload.archive.exceeds_upload_limit).toBe(false);
|
||||
expect(payload.archive.largest_files.map((file) => file.path)).toContain("asset.bin");
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveAspectRatioForSubmit — non-local sources", () => {
|
||||
it("trusts an explicit flag for asset_id / url", () => {
|
||||
trapExit();
|
||||
|
||||
@@ -35,9 +35,13 @@ import {
|
||||
} from "../../cloud/detectAspectRatio.js";
|
||||
import { c } from "../../ui/colors.js";
|
||||
import { errorBox, formatBytes, formatDuration } from "../../ui/format.js";
|
||||
import { resolveProject } from "../../utils/project.js";
|
||||
import { resolveProject, type ProjectDir } from "../../utils/project.js";
|
||||
import { normalizeErrorMessage } from "../../utils/errorMessage.js";
|
||||
import { createPublishArchive } from "../../utils/publishProject.js";
|
||||
import {
|
||||
buildPublishFileMap,
|
||||
type PublishArchiveResult,
|
||||
zipPublishFileMap,
|
||||
} from "../../utils/publishProject.js";
|
||||
import {
|
||||
reportVariableIssues,
|
||||
resolveVariablesArg,
|
||||
@@ -63,18 +67,23 @@ import type {
|
||||
HyperframesCloudClient,
|
||||
HyperframesRenderDetail,
|
||||
} from "../../cloud/index.js";
|
||||
import { isAbsolute, resolve as resolvePath } from "node:path";
|
||||
import { isAbsolute, relative, resolve as resolvePath } from "node:path";
|
||||
import { existsSync } from "node:fs";
|
||||
|
||||
const VALID_QUALITY = ["draft", "standard", "high"] as const;
|
||||
const VALID_FORMAT = ["mp4", "webm", "mov"] as const;
|
||||
const VALID_RESOLUTION = ["1080p", "4k"] as const;
|
||||
const VALID_ASPECT_RATIO = ["16:9", "9:16", "1:1"] as const;
|
||||
// Mirrors the binary-byte max_bytes contract used by direct asset uploads.
|
||||
const DIRECT_UPLOAD_LIMIT_BYTES = 200 * 1024 * 1024;
|
||||
const ARCHIVE_DIAGNOSTIC_THRESHOLD_BYTES = 150 * 1024 * 1024;
|
||||
const LARGEST_FILE_COUNT = 10;
|
||||
|
||||
const FORMAT_EXT: Record<string, string> = { mp4: ".mp4", webm: ".webm", mov: ".mov" };
|
||||
|
||||
export const examples: Example[] = [
|
||||
["Render the current directory in the cloud", "hyperframes cloud render"],
|
||||
["Inspect archive size without uploading", "hyperframes cloud render --dry-run"],
|
||||
[
|
||||
"Pick a specific composition + output path",
|
||||
"hyperframes cloud render . --composition compositions/intro.html -o ./renders/intro.mp4",
|
||||
@@ -178,6 +187,11 @@ export default defineCommand({
|
||||
description: "Emit machine-readable JSON instead of human-friendly progress",
|
||||
default: false,
|
||||
},
|
||||
"dry-run": {
|
||||
type: "boolean",
|
||||
description: "Build and inspect the project zip without authenticating or uploading",
|
||||
default: false,
|
||||
},
|
||||
"idempotency-key": {
|
||||
type: "string",
|
||||
description: "Optional Idempotency-Key for safe retries (1-255 chars from [A-Za-z0-9_:.-])",
|
||||
@@ -207,6 +221,7 @@ export default defineCommand({
|
||||
assetId: args["asset-id"],
|
||||
url: args.url,
|
||||
});
|
||||
validateDryRunSource(project, args["dry-run"] ?? false);
|
||||
|
||||
// 4k supersampling runs through the alpha-incompatible screenshot path;
|
||||
// reject the combination client-side instead of failing mid-render.
|
||||
@@ -226,6 +241,12 @@ export default defineCommand({
|
||||
asJson,
|
||||
);
|
||||
|
||||
if (args["dry-run"]) {
|
||||
if (project.kind !== "dir") throw new Error("Dry-run project must be a local directory");
|
||||
reportDryRun(prepareLocalArchive(project, args.composition, asJson), asJson);
|
||||
return;
|
||||
}
|
||||
|
||||
const variables = resolveVariablesAndValidateIfLocal(
|
||||
args.variables,
|
||||
args["variables-file"],
|
||||
@@ -234,8 +255,16 @@ export default defineCommand({
|
||||
);
|
||||
|
||||
const client = await createCloudClient();
|
||||
const preparedArchive =
|
||||
project.kind === "dir" ? prepareLocalArchive(project, args.composition, asJson) : undefined;
|
||||
|
||||
const upload = await maybeUploadProject(client, project, asJson, args["idempotency-key"]);
|
||||
const upload = await maybeUploadProject(
|
||||
client,
|
||||
project,
|
||||
preparedArchive,
|
||||
asJson,
|
||||
args["idempotency-key"],
|
||||
);
|
||||
const submitted = await submitRender(client, {
|
||||
projectInput: upload.projectInput,
|
||||
fps,
|
||||
@@ -334,6 +363,15 @@ function validateIdempotencyKey(key: string | undefined): void {
|
||||
}
|
||||
}
|
||||
|
||||
export function validateDryRunSource(source: ProjectInputSource, dryRun: boolean): void {
|
||||
if (!dryRun || source.kind === "dir") return;
|
||||
errorBox(
|
||||
"Invalid --dry-run input",
|
||||
"--dry-run inspects a local project directory and cannot be combined with --asset-id or --url.",
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Project resolution (dir | asset-id | url) — exactly one source
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -523,10 +561,107 @@ interface UploadResult {
|
||||
projectInput: CreateHyperframesRenderRequest["project"];
|
||||
}
|
||||
|
||||
interface ArchiveFileSummary {
|
||||
path: string;
|
||||
size_bytes: number;
|
||||
}
|
||||
|
||||
interface PreparedLocalArchive {
|
||||
project: ProjectDir;
|
||||
archive: PublishArchiveResult;
|
||||
largestFiles: ArchiveFileSummary[];
|
||||
}
|
||||
|
||||
function summarizeLargestFiles(fileMap: Map<string, Buffer>): ArchiveFileSummary[] {
|
||||
return [...fileMap.entries()]
|
||||
.map(([path, content]) => ({ path, size_bytes: content.byteLength }))
|
||||
.sort((a, b) => b.size_bytes - a.size_bytes || a.path.localeCompare(b.path))
|
||||
.slice(0, LARGEST_FILE_COUNT);
|
||||
}
|
||||
|
||||
function prepareLocalArchive(
|
||||
source: ProjectInputSource,
|
||||
composition: string | undefined,
|
||||
asJson: boolean,
|
||||
): PreparedLocalArchive {
|
||||
const project = resolveProject(source.dir);
|
||||
if (!asJson) {
|
||||
console.log("");
|
||||
console.log(`${c.accent("◆")} Zipping ${c.accent(project.name)}`);
|
||||
}
|
||||
|
||||
try {
|
||||
const fileMap = buildPublishFileMap(project.dir);
|
||||
const entryPath = relative(
|
||||
project.dir,
|
||||
resolvePath(project.dir, composition ?? "index.html"),
|
||||
).replaceAll("\\", "/");
|
||||
if (!fileMap.has(entryPath)) {
|
||||
throw new Error(
|
||||
`Composition "${composition ?? "index.html"}" is excluded from the archive. Check .hyperframesignore.`,
|
||||
);
|
||||
}
|
||||
const archive = zipPublishFileMap(fileMap);
|
||||
if (!asJson) {
|
||||
console.log(
|
||||
c.dim(` ${archive.fileCount} files · ${formatBytes(archive.buffer.byteLength)}`),
|
||||
);
|
||||
}
|
||||
const prepared = { project, archive, largestFiles: summarizeLargestFiles(fileMap) };
|
||||
if (!asJson && archive.buffer.byteLength >= ARCHIVE_DIAGNOSTIC_THRESHOLD_BYTES) {
|
||||
reportLargestFiles(prepared);
|
||||
}
|
||||
return prepared;
|
||||
} catch (err) {
|
||||
const msg = normalizeErrorMessage(err);
|
||||
errorBox("Zip failed", msg, "Check the project and .hyperframesignore for missing files.");
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function reportLargestFiles(prepared: PreparedLocalArchive): void {
|
||||
console.log(c.dim(" Largest included files:"));
|
||||
for (const file of prepared.largestFiles) {
|
||||
console.log(c.dim(` ${formatBytes(file.size_bytes).padStart(9)} ${file.path}`));
|
||||
}
|
||||
console.log(c.dim(" Exclude verified-unneeded files with .hyperframesignore."));
|
||||
}
|
||||
|
||||
function reportDryRun(prepared: PreparedLocalArchive, asJson: boolean): void {
|
||||
const sizeBytes = prepared.archive.buffer.byteLength;
|
||||
if (asJson) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
withMeta({
|
||||
archive: {
|
||||
project: prepared.project.name,
|
||||
file_count: prepared.archive.fileCount,
|
||||
size_bytes: sizeBytes,
|
||||
upload_limit_bytes: DIRECT_UPLOAD_LIMIT_BYTES,
|
||||
exceeds_upload_limit: sizeBytes > DIRECT_UPLOAD_LIMIT_BYTES,
|
||||
largest_files: prepared.largestFiles,
|
||||
},
|
||||
}),
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
if (sizeBytes < ARCHIVE_DIAGNOSTIC_THRESHOLD_BYTES) reportLargestFiles(prepared);
|
||||
console.log("");
|
||||
console.log(`${c.success("✓")} Dry run complete — nothing uploaded`);
|
||||
if (sizeBytes > DIRECT_UPLOAD_LIMIT_BYTES) {
|
||||
console.log(c.warn(` Archive exceeds the 200 MB direct-upload limit.`));
|
||||
}
|
||||
}
|
||||
|
||||
// fallow-ignore-next-line complexity
|
||||
async function maybeUploadProject(
|
||||
client: HyperframesCloudClient,
|
||||
source: ProjectInputSource,
|
||||
preparedArchive: PreparedLocalArchive | undefined,
|
||||
asJson: boolean,
|
||||
idempotencyKey: string | undefined,
|
||||
): Promise<UploadResult> {
|
||||
@@ -537,22 +672,8 @@ async function maybeUploadProject(
|
||||
return { projectInput: { type: "url", url: source.url! } };
|
||||
}
|
||||
|
||||
const project = resolveProject(source.dir);
|
||||
if (!asJson) {
|
||||
console.log("");
|
||||
console.log(`${c.accent("◆")} Zipping ${c.accent(project.name)}`);
|
||||
}
|
||||
let archive;
|
||||
try {
|
||||
archive = createPublishArchive(project.dir);
|
||||
} catch (err) {
|
||||
const msg = normalizeErrorMessage(err);
|
||||
errorBox("Zip failed", msg, "Check the project for missing files or unreadable permissions.");
|
||||
process.exit(1);
|
||||
}
|
||||
if (!asJson) {
|
||||
console.log(c.dim(` ${archive.fileCount} files · ${formatBytes(archive.buffer.byteLength)}`));
|
||||
}
|
||||
if (!preparedArchive) throw new Error("Local project archive was not prepared");
|
||||
const { project, archive } = preparedArchive;
|
||||
|
||||
if (!asJson) {
|
||||
console.log("");
|
||||
|
||||
@@ -49,6 +49,17 @@ describe("hyperframes-core contract docs", () => {
|
||||
expect(renderReference).toContain("timeline:");
|
||||
expect(renderReference).toContain("buildCompositionCensus");
|
||||
});
|
||||
|
||||
it("teaches safe cloud archive size remediation", () => {
|
||||
const skill = read("skills", "hyperframes-cli", "SKILL.md");
|
||||
const cloudReference = read("skills", "hyperframes-cli", "references", "cloud.md");
|
||||
|
||||
expect(skill).toContain("cloud render --dry-run --json");
|
||||
expect(skill).toContain("Never ignore an asset merely because it is large");
|
||||
expect(cloudReference).toContain(".hyperframesignore");
|
||||
expect(cloudReference).toContain("Never ignore all of `assets/`");
|
||||
expect(cloudReference).toContain("dynamically computed asset path");
|
||||
});
|
||||
});
|
||||
|
||||
describe("media-use TTS documentation", () => {
|
||||
|
||||
@@ -208,6 +208,84 @@ describe("createPublishArchive", () => {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("skips root render outputs but keeps same-named nested source directories", () => {
|
||||
const dir = makeProjectDir();
|
||||
try {
|
||||
writeFileSync(join(dir, "index.html"), "<html></html>", "utf-8");
|
||||
mkdirSync(join(dir, "renders"));
|
||||
writeFileSync(join(dir, "renders", "old.mp4"), "render-output", "utf-8");
|
||||
mkdirSync(join(dir, "snapshots"));
|
||||
writeFileSync(join(dir, "snapshots", "frame.png"), "snapshot-output", "utf-8");
|
||||
mkdirSync(join(dir, "assets", "renders"), { recursive: true });
|
||||
writeFileSync(join(dir, "assets", "renders", "source.mp4"), "source", "utf-8");
|
||||
|
||||
const zip = new AdmZip(createPublishArchive(dir).buffer);
|
||||
const entries = zip.getEntries().map((entry) => entry.entryName);
|
||||
|
||||
expect(entries).toContain("assets/renders/source.mp4");
|
||||
expect(entries).not.toContain("renders/old.mp4");
|
||||
expect(entries).not.toContain("snapshots/frame.png");
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("applies gitignore-style rules from .hyperframesignore", () => {
|
||||
const dir = makeProjectDir();
|
||||
try {
|
||||
writeFileSync(join(dir, "index.html"), "<html></html>", "utf-8");
|
||||
mkdirSync(join(dir, "exports"));
|
||||
writeFileSync(join(dir, "exports", "draft.mp4"), "draft", "utf-8");
|
||||
mkdirSync(join(dir, "assets"));
|
||||
writeFileSync(join(dir, "assets", "discard.psd"), "discard", "utf-8");
|
||||
writeFileSync(join(dir, "assets", "keep.psd"), "keep", "utf-8");
|
||||
writeFileSync(
|
||||
join(dir, ".hyperframesignore"),
|
||||
["# Generated source files", "/exports/", "*.psd", "!assets/keep.psd", ""].join("\n"),
|
||||
"utf-8",
|
||||
);
|
||||
|
||||
const zip = new AdmZip(createPublishArchive(dir).buffer);
|
||||
const entries = zip.getEntries().map((entry) => entry.entryName);
|
||||
|
||||
expect(entries).toContain("assets/keep.psd");
|
||||
expect(entries).not.toContain("exports/draft.mp4");
|
||||
expect(entries).not.toContain("assets/discard.psd");
|
||||
expect(entries).not.toContain(".hyperframesignore");
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("allows .hyperframesignore to re-include a default output directory", () => {
|
||||
const dir = makeProjectDir();
|
||||
try {
|
||||
writeFileSync(join(dir, "index.html"), "<html></html>", "utf-8");
|
||||
mkdirSync(join(dir, "snapshots"));
|
||||
writeFileSync(join(dir, "snapshots", "reference.png"), "reference", "utf-8");
|
||||
writeFileSync(join(dir, ".hyperframesignore"), "!/snapshots/\n", "utf-8");
|
||||
|
||||
const zip = new AdmZip(createPublishArchive(dir).buffer);
|
||||
expect(zip.getEntries().map((entry) => entry.entryName)).toContain("snapshots/reference.png");
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("fails clearly when .hyperframesignore excludes index.html", () => {
|
||||
const dir = makeProjectDir();
|
||||
try {
|
||||
writeFileSync(join(dir, "index.html"), "<html></html>", "utf-8");
|
||||
writeFileSync(join(dir, ".hyperframesignore"), "/index.html\n", "utf-8");
|
||||
|
||||
expect(() => createPublishArchive(dir)).toThrow(
|
||||
"Project archive must include index.html at the root. Check that .hyperframesignore does not exclude it.",
|
||||
);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("createPublishArchive (U6 cloud-render regression guard)", () => {
|
||||
|
||||
@@ -2,6 +2,7 @@ import { basename, dirname, join, posix, relative, resolve } from "node:path";
|
||||
import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";
|
||||
import { parseHTML } from "linkedom";
|
||||
import AdmZip from "adm-zip";
|
||||
import ignore, { type Ignore } from "ignore";
|
||||
import { CSS_URL_RE, isNonRelativeUrl, isPathInside } from "@hyperframes/core";
|
||||
import { buildAuthHeaders } from "../auth/client.js";
|
||||
import { tryResolveCredential } from "../auth/index.js";
|
||||
@@ -9,6 +10,8 @@ import { writeProjectLink } from "./projectLink.js";
|
||||
|
||||
const IGNORED_DIRS = new Set([".git", "node_modules", "dist", ".next", "coverage"]);
|
||||
const IGNORED_FILES = new Set([".DS_Store", "Thumbs.db"]);
|
||||
const HYPERFRAMES_IGNORE_FILE = ".hyperframesignore";
|
||||
const DEFAULT_PROJECT_IGNORE = ["/renders/", "/snapshots/"];
|
||||
const PUBLISH_CONTENT_TYPE = "application/zip";
|
||||
const PUBLISH_METADATA_TIMEOUT_MS = 30_000;
|
||||
const PUBLISH_UPLOAD_MIN_TIMEOUT_MS = 120_000;
|
||||
@@ -174,7 +177,21 @@ function shouldIgnoreSegment(segment: string): boolean {
|
||||
return segment.startsWith(".") || IGNORED_DIRS.has(segment) || IGNORED_FILES.has(segment);
|
||||
}
|
||||
|
||||
function collectProjectFiles(rootDir: string, currentDir: string, paths: string[]): void {
|
||||
function createProjectIgnore(rootDir: string): Ignore {
|
||||
const matcher = ignore().add(DEFAULT_PROJECT_IGNORE);
|
||||
const ignorePath = join(rootDir, HYPERFRAMES_IGNORE_FILE);
|
||||
if (existsSync(ignorePath)) {
|
||||
matcher.add(readFileSync(ignorePath, "utf-8"));
|
||||
}
|
||||
return matcher;
|
||||
}
|
||||
|
||||
function collectProjectFiles(
|
||||
rootDir: string,
|
||||
currentDir: string,
|
||||
paths: string[],
|
||||
matcher: Ignore,
|
||||
): void {
|
||||
for (const entry of readdirSync(currentDir, { withFileTypes: true })) {
|
||||
if (shouldIgnoreSegment(entry.name)) continue;
|
||||
const absolutePath = join(currentDir, entry.name);
|
||||
@@ -182,11 +199,13 @@ function collectProjectFiles(rootDir: string, currentDir: string, paths: string[
|
||||
if (!relativePath) continue;
|
||||
|
||||
if (entry.isDirectory()) {
|
||||
collectProjectFiles(rootDir, absolutePath, paths);
|
||||
if (matcher.ignores(`${relativePath}/`)) continue;
|
||||
collectProjectFiles(rootDir, absolutePath, paths, matcher);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!statSync(absolutePath).isFile()) continue;
|
||||
if (matcher.ignores(relativePath)) continue;
|
||||
paths.push(relativePath);
|
||||
}
|
||||
}
|
||||
@@ -388,9 +407,11 @@ export function localizeExternalAssets(
|
||||
export function buildPublishFileMap(projectDir: string): Map<string, Buffer> {
|
||||
const absProjectDir = resolve(projectDir);
|
||||
const filePaths: string[] = [];
|
||||
collectProjectFiles(absProjectDir, absProjectDir, filePaths);
|
||||
collectProjectFiles(absProjectDir, absProjectDir, filePaths, createProjectIgnore(absProjectDir));
|
||||
if (!filePaths.includes("index.html")) {
|
||||
throw new Error("Project must include an index.html file at the root before publish.");
|
||||
throw new Error(
|
||||
"Project archive must include index.html at the root. Check that .hyperframesignore does not exclude it.",
|
||||
);
|
||||
}
|
||||
|
||||
const fileContents = new Map<string, Buffer>();
|
||||
@@ -418,10 +439,10 @@ export function zipPublishFileMap(fileContents: Map<string, Buffer>): PublishArc
|
||||
|
||||
/**
|
||||
* Thin composition of `buildPublishFileMap` + `zipPublishFileMap` — signature
|
||||
* and behavior UNCHANGED from before the U6 split. `cloud render`
|
||||
* (`commands/cloud/render.ts`, `maybeUploadProject`) calls this directly and
|
||||
* must stay byte-identical (never see baked proxies); only `publish.ts` calls
|
||||
* the two halves separately with a baking transform in between.
|
||||
* and behavior UNCHANGED from before the U6 split. `cloud render` composes the
|
||||
* same two functions without an intermediate transform and must stay
|
||||
* byte-identical (never see baked proxies); only `publish.ts` inserts a baking
|
||||
* transform between them.
|
||||
*/
|
||||
export function createPublishArchive(projectDir: string): PublishArchiveResult {
|
||||
return zipPublishFileMap(buildPublishFileMap(projectDir));
|
||||
|
||||
@@ -17,8 +17,8 @@
|
||||
* original untouched. On-disk project files are never modified — only the
|
||||
* in-memory archive file map passed in by `publish.ts` (built via
|
||||
* `buildPublishFileMap`, baked here, then zipped via `zipPublishFileMap`).
|
||||
* `cloud render` never calls this: it uses `createPublishArchive` directly,
|
||||
* which has no baking hook (R2 in the plan).
|
||||
* `cloud render` never calls this: it builds and zips the file map without an
|
||||
* intermediate baking transform (R2 in the plan).
|
||||
*
|
||||
* Alpha-bearing sources bake as VP9/WebM so transparency survives. A failed
|
||||
* hostile transcode aborts publish with a
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
"files": 102
|
||||
},
|
||||
"hyperframes-cli": {
|
||||
"hash": "8bee5964bf77cc24",
|
||||
"hash": "10526c9dca9ae054",
|
||||
"files": 11
|
||||
},
|
||||
"hyperframes-core": {
|
||||
|
||||
@@ -72,6 +72,7 @@ Treat tiny unstyled content, canvas-sized icons, missing hero elements, or timel
|
||||
- Use one `HYPERFRAMES_RUN_ID` for all commands in the same verification loop.
|
||||
- Use `--strict`, `--strict-all`, and `--strict-variables` when the corresponding warnings, variables, or CI conditions must gate the render.
|
||||
- JSON paths redact the home directory as `$HOME`; do not try to reverse the redaction.
|
||||
- When a hosted cloud project approaches or exceeds the 200 MB upload limit, use `cloud render --dry-run --json` and follow the `.hyperframesignore` investigation in `references/cloud.md`. Never ignore an asset merely because it is large.
|
||||
- Never render merely because checks pass. Pause at the final preview and wait for approval.
|
||||
|
||||
## Studio-directed edits
|
||||
|
||||
@@ -38,12 +38,46 @@ Credential resolution order (first match wins): `HEYGEN_API_KEY`, then `HYPERFRA
|
||||
|
||||
1. **Resolve the project**: a local directory (default `.`), or skip the upload with `--asset-id` / `--url`.
|
||||
2. **Auto-detect aspect ratio** from the entry HTML's `data-width`/`data-height`.
|
||||
3. **Zip** the project (same ignore set as `hyperframes publish`, so it excludes `.git`, `node_modules`, `dist`, and so on).
|
||||
4. **Upload** the zip to `POST /v3/assets`, yielding an `asset_id`.
|
||||
3. **Zip** the project (same ignore set as `hyperframes publish`, including `.hyperframesignore`).
|
||||
4. **Upload** the zip through the direct-to-S3 asset flow, yielding an `asset_id`.
|
||||
5. **Submit** the render to `POST /v3/hyperframes/renders`, yielding a `render_id`.
|
||||
6. **Poll** `GET /v3/hyperframes/renders/{id}` until it completes or fails (skip with `--no-wait`).
|
||||
7. **Download** the signed video URL to disk.
|
||||
|
||||
## Archive size and `.hyperframesignore`
|
||||
|
||||
The direct-upload limit is 200 MB. HyperFrames automatically excludes root-level `renders/` and `snapshots/`, along with its existing development exclusions such as `.git`, `node_modules`, `dist`, `.next`, `coverage`, and dotfiles. Add project-specific gitignore-style rules to `<project>/.hyperframesignore` when other generated or intermediate assets are not required at render time. The same rules affect `hyperframes publish`.
|
||||
|
||||
Inspect the exact archive without authenticating, uploading, spending credits, or starting a render:
|
||||
|
||||
```bash
|
||||
npx hyperframes cloud render <project> --dry-run --json
|
||||
```
|
||||
|
||||
The result reports compressed `size_bytes`, `file_count`, the 200 MB limit, and the ten largest included files.
|
||||
|
||||
When a cloud upload reports a size-limit error, agents must use this workflow:
|
||||
|
||||
1. Run the dry-run command and inspect the largest included files and directories.
|
||||
2. Classify obvious generated outputs first: old renders, extra snapshot/contact-sheet directories, caches, exported previews, and source media used only to produce final assets.
|
||||
3. Before excluding anything else, search `src`, `href`, `url()`, `data-composition-src`, JavaScript strings, manifests, and variable-driven paths across every HTML, CSS, and JavaScript entry.
|
||||
4. Preserve existing `.hyperframesignore` comments and rules. Add the narrowest verified-unneeded root-relative paths; prefer an exact directory or file over a broad wildcard.
|
||||
5. Never ignore `index.html`, the selected composition, mounted sub-compositions, fonts, images, audio, video, scripts, or manifests merely because they are large. Never ignore all of `assets/`.
|
||||
6. Rerun dry-run until the archive is below the limit, then run `npx hyperframes check`. Remember that `check` sees the source directory, so it cannot prove a dynamically computed asset path remains in the filtered archive; the reference audit is still required.
|
||||
|
||||
Example:
|
||||
|
||||
```gitignore
|
||||
# Additional generated verification passes
|
||||
/snapshots2/
|
||||
/snapshots3/
|
||||
|
||||
# Master used only to produce the final background clips
|
||||
/assets/bg-pattern.mp4
|
||||
```
|
||||
|
||||
Rules support comments, globs, and negation. A later rule can override a default, for example `!/snapshots/` when that directory intentionally contains render inputs.
|
||||
|
||||
## Render options
|
||||
|
||||
| Flag | Default | Meaning |
|
||||
@@ -55,6 +89,7 @@ Credential resolution order (first match wins): `HEYGEN_API_KEY`, then `HYPERFRA
|
||||
| `--aspect-ratio` | auto | `16:9`, `9:16`, or `1:1`. Auto from a local project's `data-width`/`data-height`; defaults to `16:9` for `--asset-id`/`--url`. |
|
||||
| `--composition` / `-c` | `index.html` | Entry HTML file inside the zip. |
|
||||
| `--output` / `-o` | `renders/<render_id>.<ext>` | Local download destination. |
|
||||
| `--dry-run` | off | Build and inspect a local project zip without authenticating, uploading, or rendering. |
|
||||
|
||||
```bash
|
||||
npx hyperframes cloud render . \
|
||||
|
||||
Reference in New Issue
Block a user