mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 07:19:52 +00:00
feat(media-use): resolve engine + BGM/SFX/image/icon providers (#1683)
* feat(media-use): core infrastructure — manifest, cache, adopt, probe Foundation for media-use — the media resolution layer for HyperFrames. - manifest.mjs: JSONL read/write/find for .media/manifest.jsonl - index-gen.mjs: regenerate agent-readable index.md from manifest - cache.mjs: content-addressed global cache at ~/.media/ (SHA-256, sentinel) - freeze.mjs: download URL or copy local file to .media/ - probe.mjs: extract duration/dimensions via ffprobe - adopt.mjs: scan assets/ directory, register existing files with metadata - 19 passing tests (manifest round-trip, cache, promote, index generation) * fix(media-use): oxfmt formatting + cap freeze download size Format adopt/cache/probe/manifest.test (CI oxfmt --check gate). Cap freezeUrl downloads at 256MB so a hostile/runaway URL can't fill the disk (addresses CodeQL #670: network data written to file). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(media-use): resolve engine + all providers + brand from frame.md - resolve.mjs: cheapest-first cascade - BGM/SFX via heygen --headers, Image/Icon via heygen asset search - Brand tokens from frame.md / design.md (local, no API) - SKILL.md: full agent docs + hyperframes.dev/design redirect - Router skill + workflow skill references * fix(media-use): oxfmt formatting for resolve + providers Format brand/heygen-search/providers/sfx providers + resolve.mjs (CI oxfmt --check gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(media-use): align providers with the real heygen CLI surface (v0.1.6) Verified live against the official Go `heygen` CLI v0.1.6 with a valid key: - Caller attribution: pass `--headers 'X-HeyGen-Client-Source: media-use'` (the allowlisted flag the CLI added for media-use in v0.1.6). The old `--x-source media-use` was never a real flag and broke every call. - Command is `asset search` (the `list` leaf was dropped in v0.1.6), not `asset search list`. - `--min-score` is sent server-side: honored by `audio sounds list`, but the `asset search` backend rejects it and returns no score field, so only the audio providers pass it (image/icon don't). - Drop hardcoded `ext` so resolve.mjs derives it from the URL: catalog icons are .png (not .svg), some BGM is .wav (not .mp3). Also: surface CLI/auth failures on stderr instead of swallowing them as 'no results', carry icon width/height through, and document the heygen CLI install + >= v0.1.6 requirement. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Miguel Ángel
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
73f6d3e5be
commit
b2dc353725
@@ -3,6 +3,8 @@ name: faceless-explainer
|
||||
description: "turn arbitrary text — an article, notes, a topic, a brief — into a faceless explainer video, up to ~3 min (sweet spot 30-90s), where every visual is invented (typography, abstract graphics, diagrams, data-viz) rather than captured. There is no URL, no website capture, and no real assets. Use this skill for topic explainers, concept breakdowns, how-tos, listicles, and narrative explainers. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a GitHub PR (use /pr-to-video), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first."
|
||||
---
|
||||
|
||||
> **media-use**: Before sourcing audio/images, call `/media-use` to resolve BGM/SFX/images from the HeyGen catalog. Run `--adopt` first to register existing assets. See `/media-use` skill.
|
||||
|
||||
# Faceless Explainer to HyperFrames
|
||||
|
||||
Use this skill to turn a body of text into an explainer video: pick a design system, plan a teaching story, and build it frame by frame in HyperFrames. **Faceless** means every visual is invented downstream — there is no capture step and no real asset inventory.
|
||||
|
||||
@@ -11,6 +11,8 @@ description: >
|
||||
metadata: { "tags": "orchestrator, general-video, fallback, freeform, composition-authoring" }
|
||||
---
|
||||
|
||||
> **media-use**: Before sourcing audio/images, call `/media-use` to resolve BGM/SFX/images from the HeyGen catalog. Run `--adopt` first to register existing assets. See `/media-use` skill.
|
||||
|
||||
# general-video — general video workflow
|
||||
|
||||
> **Confirm the route before you build.** This is the **fallback** for custom composition authoring. If the input clearly fits a specialized workflow, prefer it: marketed product → `/product-launch-video`; general site → `/website-to-video`; topic explainer → `/faceless-explainer`; GitHub PR → `/pr-to-video`; existing footage → `/embedded-captions` · `/graphic-overlays`; short unnarrated motion graphic → `/motion-graphics`; Remotion port → `/remotion-to-hyperframes`. **Out of scope**: live / at-render-time data, NLE-style editing of a finished video, or producing footage HyperFrames can't capture. Unsure? **Read `/hyperframes` first.**
|
||||
|
||||
@@ -31,6 +31,7 @@ Atomic capabilities you load **on demand** — not full video workflows. For "ma
|
||||
| **Animate** — atomic motion, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU) | `/hyperframes-animation` |
|
||||
| **Creative direction** — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive | `/hyperframes-creative` |
|
||||
| **Media** — TTS voiceover, background music, transcription, background removal, captions | `/hyperframes-media` |
|
||||
| **Media resolve** — find + freeze BGM, SFX, images, icons from HeyGen catalog into `.media/` with manifest tracking | `/media-use` |
|
||||
| **CLI dev loop** — init, lint, validate, inspect, preview, render, publish, doctor | `/hyperframes-cli` |
|
||||
| **Install registry blocks / components** (`hyperframes add`) | `/hyperframes-registry` |
|
||||
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
---
|
||||
name: media-use
|
||||
description: Agent Media OS — resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) handles the full cascade: project cache, global cache, HeyGen catalog search, freeze, register. Keeps search noise on disk, hands the agent a path. Use when a composition needs background music, sound effects, images, or icons.
|
||||
---
|
||||
|
||||
# media-use
|
||||
|
||||
Resolve media needs into frozen local files. One verb, four types, zero context noise.
|
||||
|
||||
## When to use
|
||||
|
||||
Call `resolve` whenever a composition needs media — background music, sound effects, images, or icons. media-use searches the HeyGen catalog, downloads the best match, freezes it locally, and registers it in a manifest. The agent gets back one line; all search noise stays on disk.
|
||||
|
||||
## Resolve
|
||||
|
||||
```bash
|
||||
node <SKILL_DIR>/scripts/resolve.mjs --type <type> --intent "<description>" --project <dir>
|
||||
```
|
||||
|
||||
Returns one line: `resolved <id> → <path> (<type>, <metadata>)`
|
||||
|
||||
### Types
|
||||
|
||||
| Type | What it finds | Provider |
|
||||
| ------- | ------------------- | ---------------------------------------- |
|
||||
| `bgm` | Background music | HeyGen audio catalog (10k+ tracks) |
|
||||
| `sfx` | Sound effects | Bundled 19-file library + HeyGen catalog |
|
||||
| `image` | Photos, backgrounds | HeyGen asset search (75k+ vectors) |
|
||||
| `icon` | Icons, logos | HeyGen asset search (type=icon) |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Background music
|
||||
node <SKILL_DIR>/scripts/resolve.mjs --type bgm --intent "upbeat tech launch" --project .
|
||||
# → resolved bgm_001 → .media/audio/bgm/bgm_001.mp3 (bgm, 25s)
|
||||
|
||||
# Sound effect
|
||||
node <SKILL_DIR>/scripts/resolve.mjs --type sfx --intent "whoosh" --project .
|
||||
# → resolved sfx_001 → .media/audio/sfx/sfx_001.mp3 (sfx, 0.57s)
|
||||
|
||||
# Image
|
||||
node <SKILL_DIR>/scripts/resolve.mjs --type image --intent "gradient tech background" --project .
|
||||
# → resolved image_001 → .media/images/image_001.jpg (image)
|
||||
|
||||
# Icon
|
||||
node <SKILL_DIR>/scripts/resolve.mjs --type icon --intent "rocket" --project .
|
||||
# → resolved icon_001 → .media/images/icon_001.png (icon, transparent)
|
||||
```
|
||||
|
||||
### Flags
|
||||
|
||||
| Flag | Description |
|
||||
| --------------- | ------------------------------------------ |
|
||||
| `--type, -t` | Media type: bgm, sfx, image, icon |
|
||||
| `--intent, -i` | What you need (natural language) |
|
||||
| `--entity, -e` | Entity name for cache matching (optional) |
|
||||
| `--project, -p` | Project directory (default: .) |
|
||||
| `--adopt` | Bulk-import existing assets/ into manifest |
|
||||
| `--json` | Output JSON instead of one-line result |
|
||||
|
||||
## How it works
|
||||
|
||||
1. Check project `.media/manifest.jsonl` for exact-prompt match
|
||||
2. Scan existing `assets/` directory for unregistered files matching the need
|
||||
3. Check global cache `~/.media/` for reusable asset
|
||||
4. Search via provider (HeyGen audio catalog, HeyGen asset search)
|
||||
5. Freeze file to `.media/<type>/`, register in manifest, regenerate `index.md`
|
||||
|
||||
The agent gets back **one line**. Candidates, scores, provenance stay on disk.
|
||||
|
||||
## Adopt existing projects
|
||||
|
||||
Most HyperFrames projects already have assets in `assets/`. media-use adopts them:
|
||||
|
||||
```bash
|
||||
node <SKILL_DIR>/scripts/resolve.mjs --adopt --project .
|
||||
# → adopted 9 assets from assets/
|
||||
# bgm_001 → assets/bgm/mango-fizz.mp3 (bgm, 146.6s)
|
||||
# image_001 → assets/images/avatar.jpg (image, 400×400)
|
||||
```
|
||||
|
||||
`ffprobe` extracts real duration and dimensions. During resolve, unregistered files in `assets/` matching the intent are adopted on the fly.
|
||||
|
||||
## Reading the inventory
|
||||
|
||||
After resolve or adopt, read `.media/index.md` for the full inventory:
|
||||
|
||||
```
|
||||
# .media · 4 assets
|
||||
|
||||
id type dur dims path description
|
||||
bgm_001 bgm 25s — .media/audio/bgm/bgm_001.mp3 upbeat tech launch
|
||||
sfx_001 sfx 0.6s — .media/audio/sfx/sfx_001.mp3 whoosh
|
||||
image_001 image — 1920×1080 .media/images/image_001.jpg gradient tech background
|
||||
icon_001 icon — 200×200 .media/images/icon_001.png rocket
|
||||
```
|
||||
|
||||
## Cross-project reuse
|
||||
|
||||
Assets are cached automatically on resolve. Subsequent resolves for the same prompt hit the global cache at `~/.media/` — no re-download, no provider call. Promote an asset explicitly with `organize --promote <id>` to make it reusable across all projects.
|
||||
|
||||
## Files
|
||||
|
||||
- `.media/manifest.jsonl` — machine SSOT, one JSON record per line
|
||||
- `.media/index.md` — agent-readable table (id, type, dur, dims, path, description)
|
||||
- `~/.media/` — global cross-project reuse cache (content-addressed, SHA-256)
|
||||
|
||||
## CLI tools used
|
||||
|
||||
| Tool | Purpose | Required? |
|
||||
| --------- | ------------------------------------------ | ------------- |
|
||||
| `ffprobe` | Probe duration, dimensions, codec on adopt | Yes |
|
||||
| `heygen` | Audio catalog, asset search | For providers |
|
||||
|
||||
Install the `heygen` CLI (single static binary, no runtime) and authenticate:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://static.heygen.ai/cli/install.sh | bash # installs latest to ~/.local/bin
|
||||
heygen update # if already installed: needs >= v0.1.6
|
||||
export HEYGEN_API_KEY=<your-key> # or: heygen auth login --key <key>
|
||||
```
|
||||
|
||||
Requires **heygen >= v0.1.6** — the providers tag requests with the allowlisted `--headers 'X-HeyGen-Client-Source: media-use'` flag, added in v0.1.6. `asset search` is a pre-launch command hidden from `heygen --help`, but it runs. Without a `heygen` on PATH (or a valid key) the providers print a one-line diagnostic to stderr and resolve falls through to "no provider could resolve".
|
||||
@@ -0,0 +1,20 @@
|
||||
import { heygenSearch } from "./heygen-search.mjs";
|
||||
|
||||
export const bgmProvider = {
|
||||
async search(intent) {
|
||||
const results = heygenSearch("audio sounds list", intent, { type: "music" });
|
||||
if (!results) return null;
|
||||
const best = results[0];
|
||||
return {
|
||||
url: best.audio_url,
|
||||
source: "search",
|
||||
// ext derived from audio_url by resolve.mjs — catalog tracks are .mp3 or .wav
|
||||
metadata: {
|
||||
description: best.description || intent,
|
||||
duration: best.duration || null,
|
||||
provider: "heygen.audio.sounds",
|
||||
provenance: { track_id: best.id, score: best.score, query: intent },
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,59 @@
|
||||
import { readFileSync, existsSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
function findDesignSpec(projectDir) {
|
||||
for (const name of ["frame.md", "design.md", "DESIGN.md"]) {
|
||||
const p = join(projectDir, name);
|
||||
if (existsSync(p)) return { path: p, name };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseFrontmatter(content) {
|
||||
const match = content.match(/^---\n([\s\S]*?)\n---/);
|
||||
if (!match) return null;
|
||||
const yaml = match[1];
|
||||
const tokens = {};
|
||||
for (const line of yaml.split("\n")) {
|
||||
const m = line.match(/^\s*(\w[\w-]*):\s*(.+)/);
|
||||
if (m) tokens[m[1]] = m[2].trim().replace(/^["']|["']$/g, "");
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function extractColors(tokens) {
|
||||
const colors = [];
|
||||
for (const [k, v] of Object.entries(tokens)) {
|
||||
if (typeof v === "string" && /^#[0-9a-fA-F]{3,8}$/.test(v)) {
|
||||
colors.push({ name: k, hex: v });
|
||||
}
|
||||
}
|
||||
return colors;
|
||||
}
|
||||
|
||||
export const brandProvider = {
|
||||
async search(intent, { projectDir } = {}) {
|
||||
if (!projectDir) return null;
|
||||
const spec = findDesignSpec(projectDir);
|
||||
if (!spec) return null;
|
||||
const content = readFileSync(spec.path, "utf8");
|
||||
const tokens = parseFrontmatter(content);
|
||||
if (!tokens) return null;
|
||||
const colors = extractColors(tokens);
|
||||
return {
|
||||
localPath: spec.path,
|
||||
source: "local",
|
||||
ext: ".md",
|
||||
metadata: {
|
||||
description: "Brand tokens from " + spec.name,
|
||||
provider: "design_spec",
|
||||
provenance: {
|
||||
file: spec.name,
|
||||
colors,
|
||||
font: tokens.font || tokens.typography || null,
|
||||
logo: tokens.logo || null,
|
||||
},
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,45 @@
|
||||
import { execSync } from "node:child_process";
|
||||
|
||||
export function heygenSearch(subcommand, query, { type, limit = 5, minScore } = {}) {
|
||||
const q = query.replace(/'/g, "'\\''");
|
||||
// Tag the caller via the CLI's allowlisted attribution header (heygen >= v0.1.6).
|
||||
const parts = [
|
||||
`heygen --headers 'X-HeyGen-Client-Source: media-use' ${subcommand} --query '${q}'`,
|
||||
];
|
||||
if (type) parts.push(`--type ${type}`);
|
||||
parts.push(`--limit ${limit}`);
|
||||
// Server-side score floor. Honored by `audio sounds list`; the `asset search`
|
||||
// backend rejects it, so only audio providers pass minScore (see image-provider).
|
||||
if (minScore != null) parts.push(`--min-score ${minScore}`);
|
||||
|
||||
let out;
|
||||
try {
|
||||
out = execSync(parts.join(" "), {
|
||||
encoding: "utf8",
|
||||
timeout: 15000,
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
} catch (err) {
|
||||
// Don't swallow a broken command / auth failure as "no results" — that turns
|
||||
// a typo or expired key into a silent dead end. Surface it, then give up.
|
||||
const detail = err.stderr?.toString().trim() || err.stdout?.toString().trim() || err.message;
|
||||
console.error(`media-use: \`heygen ${subcommand}\` failed: ${detail}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(out);
|
||||
} catch {
|
||||
console.error(`media-use: \`heygen ${subcommand}\` returned non-JSON output`);
|
||||
return null;
|
||||
}
|
||||
if (parsed?.error) {
|
||||
const e = parsed.error;
|
||||
console.error(`media-use: \`heygen ${subcommand}\` error: ${e.message ?? JSON.stringify(e)}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
const data = parsed?.data;
|
||||
return Array.isArray(data) && data.length > 0 ? data : null;
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
import { heygenSearch } from "./heygen-search.mjs";
|
||||
|
||||
export const imageProvider = {
|
||||
async search(intent) {
|
||||
const results = heygenSearch("asset search", intent, { type: "image" });
|
||||
if (!results) return null;
|
||||
const best = results[0];
|
||||
return {
|
||||
url: best.url,
|
||||
source: "search",
|
||||
// ext derived from the asset URL by resolve.mjs (.jpg/.png/.webp)
|
||||
metadata: {
|
||||
description: intent,
|
||||
width: best.width || null,
|
||||
height: best.height || null,
|
||||
transparent: best.is_transparent || false,
|
||||
provider: "heygen.asset.search",
|
||||
provenance: { asset_id: best.id, score: best.score },
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
export const iconProvider = {
|
||||
async search(intent) {
|
||||
// No minScore: the `asset search` backend rejects --min-score and returns no score field.
|
||||
const results = heygenSearch("asset search", intent, { type: "icon" });
|
||||
if (!results) return null;
|
||||
const best = results[0];
|
||||
return {
|
||||
url: best.url,
|
||||
source: "search",
|
||||
// ext derived from the asset URL by resolve.mjs — catalog icons are .png, not .svg
|
||||
metadata: {
|
||||
description: intent,
|
||||
width: best.width || null,
|
||||
height: best.height || null,
|
||||
transparent: best.is_transparent ?? true,
|
||||
provider: "heygen.asset.search",
|
||||
provenance: { asset_id: best.id, score: best.score, type: "icon" },
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,29 @@
|
||||
import { sfxProvider } from "./sfx-provider.mjs";
|
||||
import { imageProvider, iconProvider } from "./image-provider.mjs";
|
||||
import { bgmProvider } from "./bgm-provider.mjs";
|
||||
import { brandProvider } from "./brand-provider.mjs";
|
||||
|
||||
const STUB = {
|
||||
async search() {
|
||||
return null;
|
||||
},
|
||||
};
|
||||
|
||||
const registry = {
|
||||
bgm: { ...bgmProvider, type: "bgm" },
|
||||
sfx: { ...sfxProvider, type: "sfx" },
|
||||
voice: { ...STUB, type: "voice" },
|
||||
image: { ...imageProvider, type: "image" },
|
||||
icon: { ...iconProvider, type: "icon" },
|
||||
brand: { ...brandProvider, type: "brand" },
|
||||
};
|
||||
|
||||
export function getProvider(type) {
|
||||
const p = registry[type];
|
||||
if (!p) throw new Error(`unknown media type: ${type}`);
|
||||
return p;
|
||||
}
|
||||
|
||||
export function listTypes() {
|
||||
return Object.keys(registry);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import { heygenSearch } from "./heygen-search.mjs";
|
||||
|
||||
export const sfxProvider = {
|
||||
async search(intent) {
|
||||
const results = heygenSearch("audio sounds list", intent, {
|
||||
type: "sound_effects",
|
||||
minScore: 0.4,
|
||||
});
|
||||
if (!results) return null;
|
||||
const best = results[0];
|
||||
return {
|
||||
url: best.audio_url,
|
||||
source: "search",
|
||||
// ext derived from audio_url by resolve.mjs — catalog SFX are .mp3 or .wav
|
||||
metadata: {
|
||||
description: best.description || best.name || intent,
|
||||
duration: best.duration || null,
|
||||
provider: "heygen.audio.sounds",
|
||||
provenance: { track_id: best.id, score: best.score, query: intent },
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,247 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { existsSync } from "node:fs";
|
||||
import { resolve, join, extname } from "node:path";
|
||||
import { parseArgs } from "node:util";
|
||||
import { appendRecord, findByPrompt, findByEntity, nextId, typeSubdir } from "./lib/manifest.mjs";
|
||||
import { regenerateIndex } from "./lib/index-gen.mjs";
|
||||
import { cacheGet, cacheGetByEntity, importFromCache } from "./lib/cache.mjs";
|
||||
import { getProvider, listTypes } from "./lib/providers.mjs";
|
||||
import { freezeUrl, freezeLocalFile } from "./lib/freeze.mjs";
|
||||
import { findExistingAsset } from "./lib/adopt.mjs";
|
||||
|
||||
const { values: args } = parseArgs({
|
||||
options: {
|
||||
type: { type: "string", short: "t" },
|
||||
intent: { type: "string", short: "i" },
|
||||
entity: { type: "string", short: "e" },
|
||||
project: { type: "string", short: "p", default: "." },
|
||||
adopt: { type: "boolean", default: false },
|
||||
json: { type: "boolean", default: false },
|
||||
help: { type: "boolean", short: "h", default: false },
|
||||
},
|
||||
strict: true,
|
||||
});
|
||||
|
||||
if (args.help) {
|
||||
console.log(`media-use resolve — turn a media need into a frozen local file
|
||||
|
||||
Usage:
|
||||
node resolve.mjs --type <type> --intent "<description>" [--project <dir>]
|
||||
|
||||
Types: ${listTypes().join(", ")}
|
||||
|
||||
Options:
|
||||
--type, -t Media type (required)
|
||||
--intent, -i What you need (required)
|
||||
--entity, -e Entity name for cache matching (optional)
|
||||
--project, -p Project directory (default: .)
|
||||
--adopt Adopt all existing assets/ files into the manifest
|
||||
--json Output JSON instead of one-line result
|
||||
--help, -h Show this help`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (args.adopt) {
|
||||
const { adoptExistingAssets } = await import("./lib/adopt.mjs");
|
||||
const projectDir = resolve(args.project);
|
||||
const adopted = adoptExistingAssets(projectDir);
|
||||
if (args.json) {
|
||||
console.log(JSON.stringify({ ok: true, adopted: adopted.length, assets: adopted }));
|
||||
} else if (adopted.length === 0) {
|
||||
console.log("no new assets to adopt (assets/ empty or already registered)");
|
||||
} else {
|
||||
console.log(`adopted ${adopted.length} asset${adopted.length === 1 ? "" : "s"} from assets/`);
|
||||
for (const r of adopted) console.log(` ${r.id} → ${r.path} (${r.type})`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (!args.type || !args.intent) {
|
||||
console.error("error: --type and --intent are required");
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const projectDir = resolve(args.project);
|
||||
const type = args.type;
|
||||
const intent = args.intent;
|
||||
const entity = args.entity || null;
|
||||
|
||||
async function run() {
|
||||
// 1. project manifest — exact-prompt match
|
||||
const projectHit = findByPrompt(projectDir, intent, type);
|
||||
if (projectHit && existsSync(join(projectDir, projectHit.path))) {
|
||||
return result(projectHit, "cached");
|
||||
}
|
||||
|
||||
// 1b. entity match in project
|
||||
if (entity) {
|
||||
const entityHit = findByEntity(projectDir, entity);
|
||||
if (entityHit && entityHit.type === type && existsSync(join(projectDir, entityHit.path))) {
|
||||
return result(entityHit, "cached");
|
||||
}
|
||||
}
|
||||
|
||||
// 1c. scan existing assets/ directory for unregistered matches
|
||||
const existingAsset = findExistingAsset(projectDir, intent, type);
|
||||
if (existingAsset) {
|
||||
const id = nextId(projectDir, type);
|
||||
const record = {
|
||||
id,
|
||||
type: existingAsset.type,
|
||||
path: existingAsset.relativePath,
|
||||
source: "existing",
|
||||
description: existingAsset.name.replace(/[-_]/g, " "),
|
||||
provenance: { provider: "local", adopted: true, prompt: intent },
|
||||
};
|
||||
appendRecord(projectDir, record);
|
||||
regenerateIndex(projectDir);
|
||||
return result(record, "existing");
|
||||
}
|
||||
|
||||
// 2. global cache — exact-prompt or entity match
|
||||
const cacheHit = cacheGet(intent, type);
|
||||
if (cacheHit) {
|
||||
const id = nextId(projectDir, type);
|
||||
const ext = extname(cacheHit.cached_path);
|
||||
const localPath = `.media/${typeSubdir(type)}/${id}${ext}`;
|
||||
const imported = importFromCache(cacheHit, projectDir, id, localPath);
|
||||
if (imported) {
|
||||
appendRecord(projectDir, imported);
|
||||
regenerateIndex(projectDir);
|
||||
return result(imported, "reused");
|
||||
}
|
||||
}
|
||||
|
||||
if (entity) {
|
||||
const entityCacheHit = cacheGetByEntity(entity);
|
||||
if (entityCacheHit && entityCacheHit.type === type) {
|
||||
const id = nextId(projectDir, type);
|
||||
const ext = extname(entityCacheHit.cached_path);
|
||||
const localPath = `.media/${typeSubdir(type)}/${id}${ext}`;
|
||||
const imported = importFromCache(entityCacheHit, projectDir, id, localPath);
|
||||
if (imported) {
|
||||
appendRecord(projectDir, imported);
|
||||
regenerateIndex(projectDir);
|
||||
return result(imported, "reused");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. provider search
|
||||
const provider = getProvider(type);
|
||||
let searchResult = null;
|
||||
try {
|
||||
searchResult = await provider.search(intent, { entity, projectDir });
|
||||
} catch {
|
||||
// search failed, try generate
|
||||
}
|
||||
|
||||
// 4. generate fallback
|
||||
if (!searchResult && provider.generate) {
|
||||
try {
|
||||
searchResult = await provider.generate(intent, { entity, projectDir });
|
||||
} catch {
|
||||
// generate failed too
|
||||
}
|
||||
}
|
||||
|
||||
if (!searchResult) {
|
||||
if (args.json) {
|
||||
console.log(
|
||||
JSON.stringify({ ok: false, error: `no provider could resolve ${type}: "${intent}"` }),
|
||||
);
|
||||
} else {
|
||||
console.error(`error: no provider could resolve ${type}: "${intent}"`);
|
||||
}
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// 5. freeze + register
|
||||
const id = nextId(projectDir, type);
|
||||
const ext = searchResult.ext || extFromUrl(searchResult.url || "") || defaultExt(type);
|
||||
const localPath = `.media/${typeSubdir(type)}/${id}${ext}`;
|
||||
const fullPath = join(projectDir, localPath);
|
||||
|
||||
if (searchResult.localPath) {
|
||||
freezeLocalFile(searchResult.localPath, fullPath);
|
||||
} else if (searchResult.url) {
|
||||
await freezeUrl(searchResult.url, fullPath);
|
||||
} else {
|
||||
console.error("error: provider returned no url or localPath");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const record = {
|
||||
id,
|
||||
type,
|
||||
path: localPath,
|
||||
source: searchResult.source || "search",
|
||||
description: searchResult.metadata?.description || intent,
|
||||
...(searchResult.metadata?.duration != null && { duration: searchResult.metadata.duration }),
|
||||
...(searchResult.metadata?.width != null && { width: searchResult.metadata.width }),
|
||||
...(searchResult.metadata?.height != null && { height: searchResult.metadata.height }),
|
||||
...(searchResult.metadata?.transparent != null && {
|
||||
transparent: searchResult.metadata.transparent,
|
||||
}),
|
||||
...(entity && { entity }),
|
||||
provenance: {
|
||||
provider: searchResult.metadata?.provider || "unknown",
|
||||
prompt: intent,
|
||||
...searchResult.metadata?.provenance,
|
||||
},
|
||||
};
|
||||
|
||||
appendRecord(projectDir, record);
|
||||
regenerateIndex(projectDir);
|
||||
return result(record, searchResult.source || "search");
|
||||
}
|
||||
|
||||
function result(record, source) {
|
||||
if (args.json) {
|
||||
console.log(JSON.stringify({ ok: true, ...record, _source: source }));
|
||||
} else {
|
||||
const meta = formatMeta(record, source);
|
||||
console.log(`resolved ${record.id} → ${record.path} (${meta})`);
|
||||
}
|
||||
}
|
||||
|
||||
function formatMeta(record, source) {
|
||||
const parts = [record.type];
|
||||
if (record.duration != null) parts.push(`${record.duration}s`);
|
||||
if (record.width && record.height) parts.push(`${record.width}×${record.height}`);
|
||||
if (record.transparent) parts.push("transparent");
|
||||
if (source === "reused") parts.push("reused");
|
||||
if (source === "generated") parts.push("generated");
|
||||
return parts.join(", ");
|
||||
}
|
||||
|
||||
function extFromUrl(url) {
|
||||
try {
|
||||
return extname(new URL(url).pathname) || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const DEFAULT_EXT = {
|
||||
bgm: ".wav",
|
||||
sfx: ".mp3",
|
||||
voice: ".wav",
|
||||
image: ".jpg",
|
||||
icon: ".svg",
|
||||
brand: ".png",
|
||||
};
|
||||
|
||||
function defaultExt(type) {
|
||||
return DEFAULT_EXT[type] || ".bin";
|
||||
}
|
||||
|
||||
run().catch((err) => {
|
||||
if (args.json) {
|
||||
console.log(JSON.stringify({ ok: false, error: err.message }));
|
||||
} else {
|
||||
console.error(`error: ${err.message}`);
|
||||
}
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -3,6 +3,8 @@ name: pr-to-video
|
||||
description: "turn a GitHub pull request (a PR URL like github.com/<owner>/<repo>/pull/<N>, an <owner>/<repo>#<N> ref, or 'this PR' in a checked-out repo) into a code-change explainer video, up to ~3 min (sweet spot 30-90s) — changelog, feature reveal, fix, or refactor walkthrough, rendered from the diff / commits / files. The input is a CODE CHANGE read via the gh CLI; there is no website capture. Use this skill for a GitHub PR. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a topic explainer with no PR (use /faceless-explainer), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first."
|
||||
---
|
||||
|
||||
> **media-use**: Before sourcing audio/images, call `/media-use` to resolve BGM/SFX/images from the HeyGen catalog. Run `--adopt` first to register existing assets. See `/media-use` skill.
|
||||
|
||||
# PR to HyperFrames
|
||||
|
||||
Use this skill to ingest a GitHub pull request, understand the change, plan a code-change explainer, and build it frame by frame in HyperFrames. The input is a **code change** (read via `gh`), not a website — there is **no capture step and no real assets** beyond the contributors' avatars.
|
||||
|
||||
@@ -3,6 +3,8 @@ name: product-launch-video
|
||||
description: "turn a product or marketing URL, pasted script, or brief into a product launch video, including SaaS promos, feature reveals, app launches, company promos, and product marketing videos. Use this skill when the user wants to market, launch, promote, or reveal a product. Do not use it for general non-launch website tours, non-product topic explainers, GitHub pull requests, captioning existing footage, or short unnarrated motion graphics. If the intent is unclear, route through /hyperframes first."
|
||||
---
|
||||
|
||||
> **media-use**: Before sourcing audio/images, call `/media-use` to resolve BGM/SFX/images from the HeyGen catalog. Run `--adopt` first to register existing assets. See `/media-use` skill.
|
||||
|
||||
# Product Launch to HyperFrames
|
||||
|
||||
Use this skill to capture a product, understand its brand, plan a launch video, and build it frame by frame in HyperFrames.
|
||||
|
||||
@@ -3,6 +3,8 @@ name: website-to-video
|
||||
description: "Capture a general website/URL and turn it into a HyperFrames video (site tour, showcase, or social clip from the site's own visuals). Uses headless Chrome screenshots + brand assets. Use when intent is general — portfolio/blog/landing-page showcase or social clip from the site. NOT for: product/SaaS launch or promo (→ /product-launch-video, even from a URL); topic explainer with no site (→ /faceless-explainer); GitHub PR (→ /pr-to-video); adding captions to existing video (→ /embedded-captions); short unnarrated page-highlight motion graphic (→ /motion-graphics). Unclear launch-vs-general-site? Ask one question or start at /hyperframes."
|
||||
---
|
||||
|
||||
> **media-use**: Before sourcing audio/images, call `/media-use` to resolve BGM/SFX/images from the HeyGen catalog. Run `--adopt` first to register existing assets. See `/media-use` skill.
|
||||
|
||||
# Website to HyperFrames
|
||||
|
||||
Capture a website, then produce a professional video from it.
|
||||
|
||||
Reference in New Issue
Block a user