feat(cli): add capture-video for on-demand video fetch from manifest

The capture pipeline writes capture/extracted/video-manifest.json
listing every <video> element on the source page (URL, dimensions,
heading, caption, preview PNG) but deliberately doesn't download the
mp4s — sites with 15+ feature videos would balloon capture size by
hundreds of MB.

User direction on this tradeoff: keep the capture lean, let the AI
agent decide per-beat whether the actual mp4 is worth fetching, and
provide a single CLI command to fetch one when the answer is yes.
That preserves the determinism (the manifest is the contract) and
the choice (agent picks what's relevant).

The motivating case is heygen.com's "Orb" — a 3D-rendered hero
animation that's at index 0 of the heygen manifest (768×768, heading
"Turn your ideas into videos in minutes"). On the last real-AI-test
run, the agent KNEW about it ("the genuine article was in the capture")
but composed a CSS-3D cube approximation instead because there was no
one-step way to fetch the actual mp4. This command closes that gap.

New command surface:

  hyperframes capture-video <project-dir> --list
      List manifest entries with index, dimensions, heading, URL.

  hyperframes capture-video <project-dir> --index 0
      Download the manifest entry at the given index.

  hyperframes capture-video <project-dir> --url <URL>
      Download a specific entry by exact URL match.

Behavior:
- Reads <project-dir>/capture/extracted/video-manifest.json.
- Resolves the target entry by --index or --url (clear error message
  on out-of-range / missing).
- Downloads to <project-dir>/capture/assets/videos/<filename> — same
  layout as the rest of the captured-assets tree, so compositions
  reference videos with the same `capture/assets/...` path pattern
  they already use for SVGs and rasters.
- Decodes URL-encoded characters and sanitizes the filename so
  manifest entries like `Frame-2147227325%20(1).mp4` land as
  `Frame-2147227325_1_.mp4` on disk.
- Idempotent: if the file already exists, warns and skips (delete +
  re-run to refetch).
- On success, prints a ready-to-paste <video> embed snippet with
  data-start / data-duration / data-track-index. Aspect-aware: square
  videos (e.g. the Orb) get a 5s default duration suggestion vs 4s
  for landscape feature videos.

Files changed:
- packages/cli/src/commands/capture-video.ts — new command (~170 LOC).
- packages/cli/src/cli.ts — registered `capture-video` subcommand.
- packages/cli/src/help.ts — added to the "Getting Started" group so
  it appears in `hyperframes --help`.
- skills/website-to-hyperframes/references/beat-builder-guide.md —
  updated the "Captured videos" landmine (from commit e6aa7fed) to
  point at the CLI command instead of the raw curl pattern. The CLI
  command becomes the documented path; curl is no longer mentioned
  (one path is better than two).

Smoke-tested against the existing /videos/heygen-com capture:
`capture-video --list` correctly lists all 9 manifest entries with
the Orb at index 0 (768×768, "Turn your ideas into videos in
minutes"). Did NOT actually download in this commit — that's the
user's runtime choice. Command + help output verified.

Closes the asset-discoverability bundle for this turn:
  e6aa7fed - doc landmine for asset-descriptions.md search
  02f95f95 - broadened isLogo via structural DOM signals
  953004b9 - LOGO tags in asset-descriptions.md (Gemini & no-Gemini)
  this    - on-demand video fetch for the Orb-class assets
This commit is contained in:
ukimsanov
2026-06-12 01:12:43 -07:00
parent 953004b958
commit 570dedbde2
4 changed files with 190 additions and 8 deletions
+1
View File
@@ -145,6 +145,7 @@ const subCommands = {
validate: () => import("./commands/validate.js").then((m) => m.default),
snapshot: () => import("./commands/snapshot.js").then((m) => m.default),
capture: () => import("./commands/capture.js").then((m) => m.default),
"capture-video": () => import("./commands/capture-video.js").then((m) => m.default),
lambda: () => import("./commands/lambda.js").then((m) => m.default),
cloudrun: () => import("./commands/cloudrun.js").then((m) => m.default),
cloud: () => import("./commands/cloud.js").then((m) => m.default),
+176
View File
@@ -0,0 +1,176 @@
import { defineCommand } from "citty";
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { resolve, join, basename } from "node:path";
import { c } from "../ui/colors.js";
import type { Example } from "./_examples.js";
export const examples: Example[] = [
["Download the hero video (index 0) from a captured project's manifest", "capture-video ./my-project --index 0"],
["Download a specific video by exact URL", "capture-video ./my-project --url https://cdn.example.com/hero.mp4"],
["List entries in the manifest without downloading", "capture-video ./my-project --list"],
];
/**
* The capture pipeline writes capture/extracted/video-manifest.json listing
* every <video> element on the source page (URL, dimensions, heading, caption,
* preview PNG) but deliberately does NOT download the mp4s — sites with
* dozens of feature videos would balloon the capture size to hundreds of MB.
*
* This command lets agents pull just the ONE video a beat needs (e.g.
* heygen.com's "Orb" hero animation at index 0) on demand. The downloaded
* file lands at capture/assets/videos/<filename-from-manifest> so beat
* compositions can reference it as `capture/assets/videos/<filename>` —
* same pattern as the captured SVGs and rasters.
*/
async function fetchToBuffer(url: string): Promise<Buffer> {
const r = await fetch(url, { redirect: "follow" });
if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText} for ${url}`);
const ab = await r.arrayBuffer();
return Buffer.from(ab);
}
function safeFilename(name: string): string {
// Manifest filenames sometimes carry URL-encoded chars (e.g.
// "Frame-2147227325%20(1).mp4"). Decode and replace anything that
// could be hostile on disk while keeping the extension intact.
let decoded = name;
try {
decoded = decodeURIComponent(name);
} catch {
/* keep raw */
}
return decoded.replace(/[^A-Za-z0-9._-]+/g, "_");
}
export default defineCommand({
meta: {
name: "capture-video",
description:
"Download a video referenced in capture/extracted/video-manifest.json (on-demand; the capture pipeline only writes the manifest + preview PNGs)",
},
args: {
project: {
type: "positional",
description: "Path to the captured project directory",
required: true,
},
index: {
type: "string",
description: "Manifest entry index to download (0-based)",
},
url: {
type: "string",
description: "Exact video URL to download (must match a manifest entry)",
},
list: {
type: "boolean",
description: "List manifest entries (index, dimensions, heading) and exit",
},
},
async run({ args }) {
const projectDir = resolve(String(args.project));
const manifestPath = join(projectDir, "capture", "extracted", "video-manifest.json");
if (!existsSync(manifestPath)) {
console.error(
`${c.error("✗")} no video-manifest.json at ${manifestPath}\n` +
` Was this directory produced by \`hyperframes capture\`?`,
);
process.exitCode = 1;
return;
}
type Entry = {
index: number;
url: string;
filename: string;
width: number;
height: number;
heading: string;
caption: string;
ariaLabel: string;
preview: string;
};
let manifest: Entry[];
try {
manifest = JSON.parse(readFileSync(manifestPath, "utf-8"));
} catch (e) {
console.error(`${c.error("✗")} video-manifest.json is malformed: ${(e as Error).message}`);
process.exitCode = 1;
return;
}
if (args.list) {
if (manifest.length === 0) {
console.log(c.dim("(manifest is empty — no <video> elements on the captured page)"));
return;
}
console.log(`${manifest.length} video entr${manifest.length === 1 ? "y" : "ies"} in ${manifestPath}:`);
for (const e of manifest) {
console.log(
` ${c.bold(`[${e.index}]`)} ${e.filename} — ${e.width}×${e.height}` +
(e.heading ? `\n heading: "${e.heading}"` : "") +
`\n url: ${e.url}`,
);
}
return;
}
let entry: Entry;
if (args.index != null) {
const i = Number(args.index);
if (!Number.isInteger(i) || i < 0 || i >= manifest.length) {
console.error(
`${c.error("✗")} --index ${args.index} out of range (manifest has ${manifest.length} entries; valid: 0..${manifest.length - 1})`,
);
process.exitCode = 1;
return;
}
entry = manifest[i]!;
} else if (args.url != null) {
const found = manifest.find((e) => e.url === args.url);
if (!found) {
console.error(
`${c.error("✗")} no manifest entry with url=${args.url}\n` +
` Run with --list to see what's available.`,
);
process.exitCode = 1;
return;
}
entry = found;
} else {
console.error(
`${c.error("✗")} specify --index <N> or --url <URL> (or --list to see what's in the manifest)`,
);
process.exitCode = 1;
return;
}
const outDir = join(projectDir, "capture", "assets", "videos");
mkdirSync(outDir, { recursive: true });
const fname = safeFilename(entry.filename || basename(entry.url));
const outPath = join(outDir, fname);
const relPath = `capture/assets/videos/${fname}`;
if (existsSync(outPath)) {
console.log(`${c.warn("⚠")} already downloaded: ${relPath} (skipping)`);
console.log(` Delete the file and re-run to refetch.`);
return;
}
console.log(`${c.accent("▸")} downloading [${entry.index}] ${entry.filename} (${entry.width}×${entry.height})`);
console.log(` from: ${entry.url}`);
try {
const buf = await fetchToBuffer(entry.url);
writeFileSync(outPath, buf);
const sizeKb = Math.round(buf.length / 1024);
const sizeStr = sizeKb > 1024 ? `${(sizeKb / 1024).toFixed(1)}MB` : `${sizeKb}KB`;
console.log(`${c.success("◇")} wrote ${relPath} (${sizeStr})`);
console.log(
` Reference it from a beat composition as:\n` +
` <video src="${relPath}" data-start="0" data-duration="${entry.width === entry.height ? 5 : 4}" data-track-index="0" autoplay muted loop></video>`,
);
} catch (e) {
console.error(`${c.error("✗")} download failed: ${(e as Error).message}`);
process.exitCode = 1;
}
},
});
+1
View File
@@ -22,6 +22,7 @@ const GROUPS: Group[] = [
["init", "Scaffold a new composition project"],
["add", "Install a block or component from the registry"],
["capture", "Capture a website for video production"],
["capture-video", "Download a video referenced in capture's video-manifest.json (on-demand)"],
["catalog", "Browse and install blocks and components"],
["preview", "Start the studio for previewing compositions"],
["publish", "Upload a project and get a stable public URL"],
@@ -76,20 +76,24 @@ If `asset-descriptions.md` has no Gemini Vision descriptions (the heading says "
### Captured videos — they're in the manifest, not on disk
The capture pipeline writes `capture/extracted/video-manifest.json` listing every `<video>` element on the source page (URL, dimensions, heading, caption, preview PNG). It does **not** download the actual mp4s — that would balloon the capture size on sites with many videos.
The capture pipeline writes `capture/extracted/video-manifest.json` listing every `<video>` element on the source page (URL, dimensions, heading, caption, preview PNG). It does **not** download the actual mp4s — that would balloon the capture size on sites with many videos (some marketing pages have 15+ feature videos).
When a beat genuinely needs the hero video (e.g. heygen.com's "Orb" — a 3D-rendered animation that's hard to approximate in CSS), fetch just that one:
When a beat genuinely needs the hero video (e.g. heygen.com's "Orb" — a 3D-rendered animation that's hard to approximate in CSS), fetch just that one with the CLI:
```bash
# Find the entry in capture/extracted/video-manifest.json — note its `url`.
# Then download it to assets/videos/:
mkdir -p assets/videos
curl -sL "<url-from-manifest>" -o assets/videos/<filename-from-manifest>
# List what's in the manifest:
npx tsx packages/cli/src/cli.ts capture-video . --list
# Download by index (the hero is usually index 0):
npx tsx packages/cli/src/cli.ts capture-video . --index 0
# Or by exact URL (when you've already inspected video-manifest.json):
npx tsx packages/cli/src/cli.ts capture-video . --url https://cdn.example.com/hero.mp4
```
Embed as a `<video>` element with `data-start` / `data-duration` like any other clip. See the `/hyperframes` skill's video-composition reference for the contract.
The file lands at `capture/assets/videos/<filename-from-manifest>` (matching the rest of the captured-assets layout). The command prints a ready-to-paste `<video>` snippet on success. Embed it in your beat composition with `data-start` / `data-duration` like any other clip — see the `/hyperframes` skill's video-composition reference for the contract.
Don't bulk-download every video in the manifest — most aren't relevant to your beat. Pick the one the beat actually needs.
**Don't bulk-download every video in the manifest** — most aren't relevant to your beat. Pick the ONE the beat actually needs, by looking at the manifest's `heading` and `caption` fields + the preview PNG at `capture/assets/videos/previews/video-N-preview.png`.
## Step 2: Build the composition