ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-11-12 20:47:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: retire hyperframes-media skill, fold into media-use (U22)

Browse Source 
Delete the hyperframes-media skill (engine relocated in U20). Sweep all 32
referencing files → media-use: engine paths → media-use/audio/scripts, reference
links → media-use/audio/references, skill-name/routing mentions → /media-use.
Catalog dedup: drop the duplicate audio row from README/CLAUDE/skills.mdx (the
single media-use row now covers resolve + audio), router + hyperframes-core
domain list point to media-use, skill count 19→18. Manifest regenerated (18,
hyperframes-media gone). Grep gate clean (only plan docs reference it as history).
dist/ artifacts regenerate from source on build.
This commit is contained in:
Miguel Angel Simon Sierra
2026-07-03 19:22:33 -07:00
parent 8fad6acc9b
commit ea8979a18d
34 changed files with 71 additions and 176 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+4 -5
View File
@@ -4,11 +4,11 @@ Open-source video rendering framework: write HTML, render video.
## Skills
This repo ships 21 AI agent skills via [vercel-labs/skills](https://github.com/vercel-labs/skills). Install them before writing compositions — they encode framework-specific patterns that generic docs don't cover.
This repo ships 20 AI agent skills via [vercel-labs/skills](https://github.com/vercel-labs/skills). Install them before writing compositions — they encode framework-specific patterns that generic docs don't cover.
```bash
npx skills add heygen-com/hyperframes # interactive picker
npx skills add heygen-com/hyperframes --all # install all 21 (skips picker)
npx skills add heygen-com/hyperframes --all # install all 20 (skips picker)
npx skills add heygen-com/hyperframes --skill <name> # just one (bare name, no leading slash)
```
@@ -36,8 +36,7 @@ Atomic capabilities the creation workflows compose against — pull one when you
- `/hyperframes-animation` — all animation knowledge: atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP default, plus Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU).
- `/hyperframes-keyframes` — seek-safe keyframe authoring across runtimes: GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, text trails, cursor demos, 3D depth; plus `hyperframes keyframes` diagnostics for surfacing and verifying rendered motion.
- `/hyperframes-creative` — non-animation creative direction: `frame.md` / `design.md` handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns.
- `/hyperframes-media` — audio + media: TTS voiceover, background music, sound effects, Whisper transcription, background removal, caption authoring (one shared `scripts/audio.mjs` engine, multi-provider).
- `/media-use` — resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) over the HeyGen catalog with manifest tracking; keeps search noise on disk.
- `/media-use` — the media OS: resolve any media need (BGM, SFX, image, icon, voice) into a frozen local file + ledger record; generate via TTS / music when the catalog misses; transcribe, remove backgrounds, and reuse assets across projects. One shared `scripts/audio.mjs` engine + manifest tracking; keeps search noise on disk.
- `/hyperframes-cli` — CLI dev loop: `init`, `add`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, `lambda` (AWS Lambda cloud rendering).
- `/hyperframes-registry` — install and wire registry blocks and components into compositions via `hyperframes add`. Covers authoring a new block or component to contribute upstream.
- `/figma` — import Figma assets, tokens, components, and storyboard sections → animatics (REST/CLI) plus Motion animations and shaders (MCP) into a composition.
@@ -49,7 +48,7 @@ When adding a new skill, or substantially renaming / repurposing an existing one
1. The skill list above (CLAUDE.md) AND the `## Skills` section in `README.md` AND `docs/guides/skills.mdx` (rendered at [hyperframes.heygen.com/guides/skills](https://hyperframes.heygen.com/guides/skills)). Out-of-date entries silently kill discovery.
2. If the skill changes the routing surface for "make a video" requests, also update the capability map and intent router in `skills/hyperframes/SKILL.md` — that's the canonical router agents read first.
3. Mirror the Router / Creation workflows / Domain skills grouping across all three surfaces so a skill always lives in the same column.
4. Skill count appears in the README and CLAUDE.md intro lines ("21 AI agent skills…") — update on add/remove. The `docs/guides/skills.mdx` page deliberately omits a count to avoid drift; keep it count-free.
4. Skill count appears in the README and CLAUDE.md intro lines ("20 AI agent skills…") — update on add/remove. The `docs/guides/skills.mdx` page deliberately omits a count to avoid drift; keep it count-free.
The skill's own `SKILL.md` frontmatter `description:` is the source of truth for the one-line "use when" blurb; copy from there into the catalog rather than paraphrasing.
README.md
+2 -3
View File
@@ -49,7 +49,7 @@ The skills teach agents the HyperFrames production loop: plan the video, write v
## Skills
HyperFrames ships 21 skills agents load on demand. Read `/hyperframes` first — it's the router and capability map; it picks a workflow for any "make me a video" request and points to the domain skills below.
HyperFrames ships 20 skills agents load on demand. Read `/hyperframes` first — it's the router and capability map; it picks a workflow for any "make me a video" request and points to the domain skills below.
Run `npx skills add heygen-com/hyperframes` for the interactive picker, `npx skills add heygen-com/hyperframes --all` to install all 21 at once (skips the picker), or `npx skills add heygen-com/hyperframes --skill <name>` for just one (bare name, no leading `/`).
@@ -85,8 +85,7 @@ Atomic capabilities the creation workflows compose against — pull one when you
| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). |
| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. |
| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. |
| `/hyperframes-media` | Audio + media — TTS voiceover, background music, sound effects, Whisper transcription, background removal, caption authoring (one shared audio engine). |
| `/media-use` | Resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) over the HeyGen catalog with manifest tracking. |
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, voice) into a frozen local file + ledger record, generate via TTS/music when the catalog misses, transcribe, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). |
| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. |
| `/figma` | Import Figma assets, tokens, components, and storyboard sections → animatics (REST/CLI) plus Motion animations and shaders (MCP) into a composition. |
docs/guides/antigravity.mdx
+1 -1
View File
@@ -42,7 +42,7 @@ Antigravity uses **semantic matching** — when you type a prompt, the agent com
| "Create a 10-second product intro with captions" | `hyperframes`, `hyperframes-core` |
| "Add a GSAP scale-pop to the title" | `hyperframes-animation` |
| "Use Tailwind for styling" | `hyperframes-core` |
| "Transcribe this audio and add captions" | `hyperframes-media` |
| "Transcribe this audio and add captions" | `media-use` |
| "Add a shimmer sweep transition" | `hyperframes-registry` |
The skill is only injected into the agent's context window when it matches — this keeps the context clean and focused.
docs/guides/pipeline.mdx
+1 -1
View File
@@ -142,7 +142,7 @@ npx hyperframes transcribe narration.wav
| `narration.txt` | The exact spoken text with pronunciation substitutions applied (`API` → `A P I`, `$2T` → `two trillion`). Distinct from `SCRIPT.md` so you can regenerate the audio later with a different voice without redoing the substitutions. |
| `transcript.json`| `[{ text, start, end }]` for every word. Every later step reads this for timing. |
Hyperframes ships multiple TTS adapters (Kokoro, ElevenLabs, HeyGen); see [`/hyperframes-media`](/guides/prompting) for the skill that picks one. After generating audio, update `STORYBOARD.md` with the real beat boundaries from `transcript.json`.
Hyperframes ships multiple TTS adapters (Kokoro, ElevenLabs, HeyGen); see [`/media-use`](/guides/prompting) for the skill that picks one. After generating audio, update `STORYBOARD.md` with the real beat boundaries from `transcript.json`.
**Gate:** `narration.wav`, `narration.txt`, and `transcript.json` exist. `STORYBOARD.md` beat timings reference real timestamps, not estimates.
docs/guides/prompting.mdx
+1 -1
View File
@@ -24,7 +24,7 @@ The installer shows a picker. Select the **core skills** below — every project
| `/hyperframes-animation`| All animation — motion rules, scene blueprints, transitions, and the runtime adapters (GSAP, Lottie, Three.js, Anime.js, CSS, WAAPI, TypeGPU) |
| `/hyperframes-creative` | Creative direction — design spec, palettes, typography, narration, beats |
| `/hyperframes-cli` | Dev-loop CLI — `init`, `lint`, `inspect`, `preview`, `render`, `doctor` |
| `/hyperframes-media` | Asset preprocessing — `tts`, `transcribe`, `remove-background` |
| `/media-use` | Asset preprocessing — `tts`, `transcribe`, `remove-background` |
| `/hyperframes-registry` | Block and component installation via `hyperframes add` |
| `/general-video` | The general authoring workflow — fallback for any video that doesn't match a specific workflow below |
docs/guides/skills.mdx
+1 -2
View File
@@ -80,8 +80,7 @@ Atomic capabilities the creation workflows compose against — pull one when you
| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). |
| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. |
| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. |
| `/hyperframes-media` | Audio + media — TTS voiceover, background music, sound effects, Whisper transcription, background removal, caption authoring (one shared audio engine). |
| `/media-use` | Resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) over the HeyGen catalog with manifest tracking. |
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, voice) into a frozen local file + ledger record, generate via TTS/music when the catalog misses, transcribe, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). |
| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. |
| `/figma` | Import Figma assets, tokens, components, and storyboard sections → animatics (REST/CLI) plus Motion animations and shaders (MCP) into a composition. |
docs/packages/cli.mdx
+1 -1
View File
@@ -930,7 +930,7 @@ Word-level transcripts (whisper output) are grouped into readable caption cues o
| `--codex` | Install to Codex CLI (`~/.codex/skills/`) |
| `--cursor` | Install to Cursor (`.cursor/skills/` in current project) |
Skills are fetched from GitHub: the `/hyperframes` entry skill (routes "make me a video" to a workflow), the composition contract and Tailwind v4 browser-runtime guidance (`/hyperframes-core`), all animation including the GSAP / Anime.js / CSS / Lottie / Three.js / WAAPI / TypeGPU runtime adapters (`/hyperframes-animation`), creative direction (`/hyperframes-creative`), media preprocessing (`/hyperframes-media`), registry block/component wiring (`/hyperframes-registry`), and the video workflows. The `init` command also offers to install skills automatically after scaffolding a project.
Skills are fetched from GitHub: the `/hyperframes` entry skill (routes "make me a video" to a workflow), the composition contract and Tailwind v4 browser-runtime guidance (`/hyperframes-core`), all animation including the GSAP / Anime.js / CSS / Lottie / Three.js / WAAPI / TypeGPU runtime adapters (`/hyperframes-animation`), creative direction (`/hyperframes-creative`), media preprocessing (`/media-use`), registry block/component wiring (`/hyperframes-registry`), and the video workflows. The `init` command also offers to install skills automatically after scaffolding a project.
#### Troubleshooting: `fatal: active post-checkout hook found during git clone`
docs/quickstart.mdx
+1 -1
View File
@@ -22,7 +22,7 @@ The installer shows a picker. Select the **core set** every project needs — th
| `/hyperframes-animation` | All animation + the GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU runtime adapters |
| `/hyperframes-creative` | Design direction — palettes, typography, narration, beat planning |
| `/hyperframes-cli` | Dev-loop CLI — init, lint, preview, render, doctor |
| `/hyperframes-media` | Asset preprocessing — TTS, transcription, background removal |
| `/media-use` | Asset preprocessing — TTS, transcription, background removal |
| `/hyperframes-registry` | Install catalog blocks and components |
| `/general-video` | The general authoring workflow — the fallback for any video that doesn't match a specific workflow |
packages/cli/src/audio/providers.ts
+1 -1
View File
@@ -1,7 +1,7 @@
/**
* Which voice / music engine a workflow will actually use, and whether
* its local dependencies are present. Mirrors the resolution order the
* hyperframes-media skill scripts use, so `auth status` and `doctor`
* media-use skill scripts use, so `auth status` and `doctor`
* report the same engine the render pipeline would pick:
*
* voice: HeyGen Starfish → ElevenLabs (key + `elevenlabs`) → Kokoro (local)
packages/cli/src/commands/auth/status-guidance.ts
+1 -1
View File
@@ -63,7 +63,7 @@ function offlineEngineLines(engines?: OfflineEngineLine[]): string[] {
* so it's left to the docs — not dangled here as a command a fresh machine
* can't run. Names the local fallback so "no key" never reads as a failure,
* and never steers users toward a per-repo `.env`. Mirrors the
* hyperframes-media skill's Preflight section.
* media-use skill's Preflight section.
*/
export function buildUnconfiguredLines(
ctx: UnconfiguredContext,
packages/cli/src/templates/_shared/AGENTS.md
+1 -1
View File
@@ -17,7 +17,7 @@
**Porting an existing composition?** `/remotion-to-hyperframes` translates a Remotion (React) composition into HyperFrames HTML — a source migration, separate from the creation workflows above.
The domain skills (`/hyperframes-core`, `/hyperframes-animation`, `/hyperframes-creative`, `/hyperframes-cli`, `/hyperframes-media`, `/hyperframes-registry`) and the full capability map live inside `/hyperframes` — it is the single source of truth for which skill handles which intent.
The domain skills (`/hyperframes-core`, `/hyperframes-animation`, `/hyperframes-creative`, `/hyperframes-cli`, `/media-use`, `/hyperframes-registry`) and the full capability map live inside `/hyperframes` — it is the single source of truth for which skill handles which intent.
> **Tailwind v4 projects** (`hyperframes init --tailwind`): see `/hyperframes-core` → `references/tailwind.md`.
packages/cli/src/templates/_shared/CLAUDE.md
+1 -1
View File
@@ -17,7 +17,7 @@
**Porting an existing composition?** `/remotion-to-hyperframes` translates a Remotion (React) composition into HyperFrames HTML — a source migration, separate from the creation workflows above.
The domain skills (`/hyperframes-core`, `/hyperframes-animation`, `/hyperframes-creative`, `/hyperframes-cli`, `/hyperframes-media`, `/hyperframes-registry`) and the full capability map live inside `/hyperframes` — it is the single source of truth for which skill handles which intent.
The domain skills (`/hyperframes-core`, `/hyperframes-animation`, `/hyperframes-creative`, `/hyperframes-cli`, `/media-use`, `/hyperframes-registry`) and the full capability map live inside `/hyperframes` — it is the single source of truth for which skill handles which intent.
> **Tailwind v4 projects** (`hyperframes init --tailwind`): see `/hyperframes-core` → `references/tailwind.md`.
packages/studio/src/components/sidebar/BlocksTab.tsx
+1 -1
View File
@@ -234,7 +234,7 @@ function buildAgentPrompt(
captions: [
`Using /hyperframes, add the "${title}" caption style (registry: ${name}) to my composition.`,
`${description}`,
`Transcribe the audio with /hyperframes-media, then wire the transcript into this caption component. Match the font colors and animation timing to my composition's design tokens. Place it as an overlay above the main content with the highest z-index.`,
`Transcribe the audio with /media-use, then wire the transcript into this caption component. Match the font colors and animation timing to my composition's design tokens. Place it as an overlay above the main content with the highest z-index.`,
].join("\n\n"),
vfx: [
`Using /hyperframes, add the "${title}" VFX (registry: ${name}) as a full-screen overlay on my composition.`,
scripts/test-skills-fresh.sh
+1 -1
View File
@@ -237,7 +237,7 @@ ROUTER="hyperframes"
WORKFLOWS=(product-launch-video website-to-video faceless-explainer embedded-captions \
talking-head-recut pr-to-video motion-graphics general-video \
remotion-to-hyperframes slideshow)
DOMAIN=(hyperframes-core hyperframes-creative hyperframes-animation hyperframes-cli hyperframes-media hyperframes-registry)
DOMAIN=(hyperframes-core hyperframes-creative hyperframes-animation hyperframes-cli media-use hyperframes-registry)
MISSING=()
check_skill() { if [[ -d "$SKILLS_DIR/$1" ]]; then ok "$SKILLS_DIR/$1/"; else MISSING+=("$1"); fi; }
skills-manifest.json
+12 -16
View File
@@ -2,41 +2,37 @@
"source": "heygen-com/hyperframes",
"skills": {
"embedded-captions": {
"hash": "d5561ac019e72ed5",
"hash": "6fe56c988b610648",
"files": 144
},
"faceless-explainer": {
"hash": "95cdb23e0b20d4e1",
"hash": "5b3fdda7fbff9df8",
"files": 17
},
"general-video": {
"hash": "a30225e30ec7b06c",
"hash": "5eb796e7acb96f75",
"files": 1
},
"hyperframes": {
"hash": "00ad363cda72268d",
"hash": "f618b60bcd92bd37",
"files": 1
},
"hyperframes-animation": {
"hash": "ced7bd16b3cdb376",
"hash": "962b70629c7e2ad6",
"files": 108
},
"hyperframes-cli": {
"hash": "ed24d781c2ae462b",
"hash": "1b9189c1c7548b3c",
"files": 7
},
"hyperframes-core": {
"hash": "6243b6f09c94cce1",
"hash": "546064ec52c58fe0",
"files": 13
},
"hyperframes-creative": {
"hash": "bb248c1bc5cc28b5",
"files": 67
},
"hyperframes-media": {
"hash": "a891c231a816dd62",
"files": 1
},
"hyperframes-registry": {
"hash": "e3b389526834109d",
"files": 10
@@ -46,19 +42,19 @@
"files": 75
},
"motion-graphics": {
"hash": "5a256811f730f36e",
"hash": "cce43d66c288a0ec",
"files": 23
},
"music-to-video": {
"hash": "abea9101f4322954",
"hash": "0f8fa763d1e3fec9",
"files": 132
},
"pr-to-video": {
"hash": "d39061fb81afdf8c",
"hash": "98d9118467598f2d",
"files": 21
},
"product-launch-video": {
"hash": "e97a4a584b94e2ba",
"hash": "4f7026988de251bc",
"files": 18
},
"remotion-to-hyperframes": {
@@ -74,7 +70,7 @@
"files": 27
},
"website-to-video": {
"hash": "ac39931f9a7da749",
"hash": "c846c1db95ba5a41",
"files": 32
}
}
skills/embedded-captions/modes/standard/_anatomy.md
+1 -1
View File
@@ -226,7 +226,7 @@ The gallery used a `setInterval` loop; HyperFrames needs the same beats as **abs
## Pairs with HF skills
- `hyperframes-media` — `remove-background` (the matte) + `transcribe` (word timings).
- `media-use` — `remove-background` (the matte) + `transcribe` (word timings).
- `hyperframes-captions` — transcript consumption, grouping, positioning, exit guarantees, `fitTextFontSize`.
- `hyperframes-animation/rules/asr-keyword-glow.md` — the verbatim active-word envelope.
- `hyperframes-gsap` — single paused timeline, transform aliases, ease palette.
skills/embedded-captions/references/rail.md
+1 -1
View File
@@ -10,7 +10,7 @@ with only the climax(es) promoted to embed. Rail is not a fallback — it's the
> **Implementation note.** A dedicated rail renderer is the next build step. The rail is a
> plain `fg` caption track and maps cleanly onto hyperframes' native caption pipeline
> (`hyperframes-media` captions) — prefer reusing that over hand-rolling. Until wired, render
> (`media-use` captions) — prefer reusing that over hand-rolling. Until wired, render
> the rail as a simple `data-caption-layer="fg"` composition (no matte overlay for these caps).
## Position & safe area
skills/embedded-captions/scripts/transcribe.cjs
+1 -1
View File
@@ -110,7 +110,7 @@ function main() {
console.error("usage: transcribe.cjs <project-dir> [model] [language]");
process.exit(1);
}
// Default = multilingual `small`, NOT `small.en`. Per hyperframes-media: ".en models
// Default = multilingual `small`, NOT `small.en`. Per media-use: ".en models
// mistranslate non-English and mis-handle accented speech; default to small (auto-detects
// language)." We hardcoded small.en before — it hallucinated a wrong transcript on an
// accented speaker. Pass `small.en` only for known-clean-English; tough accents → a larger model.
skills/faceless-explainer/SKILL.md
+3 -3
View File
@@ -25,7 +25,7 @@ Initialize only if `hyperframes.json` is missing. Name `<project>` from the topi
`npx hyperframes init "videos/<project>" --non-interactive --example=blank` — `init` checks the installed skills against the latest on GitHub and updates the global set if any are out of date.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../hyperframes-media` → Preflight for the canonical guidance.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../media-use` → Preflight for the canonical guidance.
**Gate:** `hyperframes.json` exists, and angle, length, aspect ratio, and language are locked; sign-in status was shown (signed in, or continuing offline).
@@ -88,7 +88,7 @@ Start audio after Step 3 approval. Run it in the background, then continue to St
`node <SKILL_DIR>/scripts/audio.mjs --script ./SCRIPT.md --storyboard ./STORYBOARD.md --hyperframes . --out ./audio_meta.json &`
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and the same `~/.heygen` credential as TTS. For provider details, read `../hyperframes-media/references/tts.md`.
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and the same `~/.heygen` credential as TTS. For provider details, read `../media-use/audio/references/tts.md`.
If there is no narration and no `SCRIPT.md`, skip voice generation. BGM may still run if the storyboard has a music mood.
@@ -200,7 +200,7 @@ The reusable, domain-agnostic shot shapes live in `../hyperframes-animation/blue
| `[../hyperframes-animation/blueprints-index.md](../hyperframes-animation/blueprints-index.md)` | Step 3: role→blueprint menu. Step 4: pick the shot shape. |
| `[../hyperframes-core/references/storyboard-format.md](../hyperframes-core/references/storyboard-format.md)` | Step 3: write `STORYBOARD.md`. |
| `[../hyperframes-core/references/script-format.md](../hyperframes-core/references/script-format.md)` | Step 3: write `SCRIPT.md`. |
| `[../hyperframes-media/references/tts.md](../hyperframes-media/references/tts.md)` | Step 3.1: choose or understand TTS providers and voices. |
| `[../media-use/audio/references/tts.md](../media-use/audio/references/tts.md)` | Step 3.1: choose or understand TTS providers and voices. |
| `[references/visual-design.md](references/visual-design.md)` | Step 4: write the frame's shot sequence (+ Layout vocabulary). |
| `[references/motion-language.md](references/motion-language.md)` | Step 4: the motion vocabulary + the motion doctrine. |
| `[references/cut-catalog.md](references/cut-catalog.md)` | Step 4-5: the cut catalog (worker builds within-frame seams). |
skills/general-video/SKILL.md
+16 -16
View File
@@ -96,24 +96,24 @@ Never use `position: absolute; top: Npx` on a content container — it overflows
## Build — delegate to the domain skills
This maps the skill's full surface (see the `description`) to its references — non-exhaustive; when an intent isn't listed, route through `hyperframes-creative` (look/concept), `hyperframes-animation` (motion), `hyperframes-core` (contract), `hyperframes-media` (audio/captions). **The first row is ADDITIVE — read it AND your intent row, not one or the other.**
This maps the skill's full surface (see the `description`) to its references — non-exhaustive; when an intent isn't listed, route through `hyperframes-creative` (look/concept), `hyperframes-animation` (motion), `hyperframes-core` (contract), `media-use` (audio/captions). **The first row is ADDITIVE — read it AND your intent row, not one or the other.**
| Building… | Read first (in order) |
| --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **ALWAYS — every non-trivial piece, on top of your intent row below** | `hyperframes-creative/references/house-style.md` + `references/video-composition.md` (also gated in Step 1 / HARD-GATE; the "produced, not generated" foreground detailing) |
| **Kinetic typography / text-forward** | `hyperframes-animation/techniques.md` (kinetic type) + `adapters/gsap-easing-and-stagger.md` + `rules/kinetic-beat-slam.md` |
| **Title card / lower-third / overlay / PiP / text-behind-subject** | `hyperframes-creative/references/composition-patterns.md` + (for the centered/sized frame) `hyperframes-core` → "Root must be sized" |
| **Logo / brand-mark reveal** | `hyperframes-animation/rules/svg-path-draw.md` (draw-on) + `rules/3d-text-depth-layers.md` + `rules/scale-swap-transition.md` |
| **Data / stats / numbers** | `hyperframes-animation/rules/counting-dynamic-scale.md` + `rules/stat-bars-and-fills.md` + `hyperframes-creative/references/data-in-motion.md` |
| **Product / app / UI demo** | `hyperframes-animation/rules/3d-page-scroll.md` + `rules/cursor-click-ripple.md` + `rules/press-release-spring.md` |
| **Audio-reactive / music-driven** | `hyperframes-creative/references/audio-reactive.md` (pre-extract bands; map to motion) |
| **Narrated / voiceover / music / SFX / captions** | `hyperframes-media` → the shared audio engine `scripts/audio.mjs` (one call = TTS + BGM + SFX → `audio_meta.json`); caption authoring + asset placement via `hyperframes-core`. See **Audio** below. |
| **Multi-scene / transitions** | `hyperframes-animation/transitions/overview.md` **then** `transitions/catalog.md` (you are not done after the overview — the GSAP recipe is in the catalog) |
| **Modular / sub-compositions** | `hyperframes-core/references/composition-patterns.md` + `references/sub-compositions.md` |
| Building… | Read first (in order) |
| --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **ALWAYS — every non-trivial piece, on top of your intent row below** | `hyperframes-creative/references/house-style.md` + `references/video-composition.md` (also gated in Step 1 / HARD-GATE; the "produced, not generated" foreground detailing) |
| **Kinetic typography / text-forward** | `hyperframes-animation/techniques.md` (kinetic type) + `adapters/gsap-easing-and-stagger.md` + `rules/kinetic-beat-slam.md` |
| **Title card / lower-third / overlay / PiP / text-behind-subject** | `hyperframes-creative/references/composition-patterns.md` + (for the centered/sized frame) `hyperframes-core` → "Root must be sized" |
| **Logo / brand-mark reveal** | `hyperframes-animation/rules/svg-path-draw.md` (draw-on) + `rules/3d-text-depth-layers.md` + `rules/scale-swap-transition.md` |
| **Data / stats / numbers** | `hyperframes-animation/rules/counting-dynamic-scale.md` + `rules/stat-bars-and-fills.md` + `hyperframes-creative/references/data-in-motion.md` |
| **Product / app / UI demo** | `hyperframes-animation/rules/3d-page-scroll.md` + `rules/cursor-click-ripple.md` + `rules/press-release-spring.md` |
| **Audio-reactive / music-driven** | `hyperframes-creative/references/audio-reactive.md` (pre-extract bands; map to motion) |
| **Narrated / voiceover / music / SFX / captions** | `media-use` → the shared audio engine `scripts/audio.mjs` (one call = TTS + BGM + SFX → `audio_meta.json`); caption authoring + asset placement via `hyperframes-core`. See **Audio** below. |
| **Multi-scene / transitions** | `hyperframes-animation/transitions/overview.md` **then** `transitions/catalog.md` (you are not done after the overview — the GSAP recipe is in the catalog) |
| **Modular / sub-compositions** | `hyperframes-core/references/composition-patterns.md` + `references/sub-compositions.md` |
### Audio: one engine (TTS · BGM · SFX)
Only when the piece calls for it (per "build exactly what was asked" — no ambient music on a title card). Don't hand-roll TTS or vendor a copy: write a neutral `audio_request.json` and call the shared engine in `hyperframes-media`. It auto-degrades on one switch — HeyGen credential present → HeyGen TTS + music/SFX **retrieval**; absent → ElevenLabs/Kokoro TTS, Lyria/MusicGen BGM **generation**, and the bundled SFX library. Full flag list + request/meta schema: the header comment of `hyperframes-media/scripts/audio.mjs`.
Only when the piece calls for it (per "build exactly what was asked" — no ambient music on a title card). Don't hand-roll TTS or vendor a copy: write a neutral `audio_request.json` and call the shared engine in `media-use`. It auto-degrades on one switch — HeyGen credential present → HeyGen TTS + music/SFX **retrieval**; absent → ElevenLabs/Kokoro TTS, Lyria/MusicGen BGM **generation**, and the bundled SFX library. Full flag list + request/meta schema: the header comment of `media-use/audio/scripts/audio.mjs`.
```jsonc
// audio_request.json — one line per narrated segment; `id` is yours (joins audio_meta back)
@@ -127,11 +127,11 @@ Only when the piece calls for it (per "build exactly what was asked" — no ambi
```
```bash
# <MEDIA_DIR> = the installed hyperframes-media skill dir (sibling of this skill)
# <MEDIA_DIR> = the installed media-use skill dir (sibling of this skill)
node <MEDIA_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json
```
Then read `audio_meta.json`: mount each `voices[].path` + (`bgm.path`, `sfx[]`) as `<audio>` tracks and use `voices[].words` for captions, all per `hyperframes-core` (audio tracks + caption authoring). If BGM took the generate path (`bgm_pending: true`), run `hyperframes-media/scripts/wait-bgm.mjs` before final render.
Then read `audio_meta.json`: mount each `voices[].path` + (`bgm.path`, `sfx[]`) as `<audio>` tracks and use `voices[].words` for captions, all per `hyperframes-core` (audio tracks + caption authoring). If BGM took the generate path (`bgm_pending: true`), run `media-use/audio/scripts/wait-bgm.mjs` before final render.
## Output checklist → `hyperframes-cli`
skills/hyperframes-animation/rules/asr-keyword-glow.md
+2 -2
View File
@@ -280,7 +280,7 @@ Ease family — discrete choice:
## Pairs with HF skills
- `/hyperframes-animation` — single driver, multi-element envelope
- `/hyperframes-media` — `hyperframes transcribe` outputs real ASR data
- `/hyperframes-media` — pair with caption rendering
- `/media-use` — `hyperframes transcribe` outputs real ASR data
- `/media-use` — pair with caption rendering
- `/hyperframes-core` — composition wiring
- `/hyperframes-cli` — `hyperframes lint`
skills/hyperframes-cli/SKILL.md
+1 -1
View File
@@ -56,7 +56,7 @@ Cross-cutting rules that hold for every command:
- **Tailwind projects** (`init --tailwind`) → use `hyperframes-core` (Tailwind reference) before editing classes or theme tokens.
- **Registry blocks/components** (`hyperframes add`, `hyperframes catalog`) → use `hyperframes-registry` for install paths, sub-composition wiring, and snippet merging.
- **Asset preprocessing** (`tts`, `transcribe`, `remove-background`) → use `hyperframes-media` for voice selection, Whisper model rules, captions, and TTS-to-captions chain.
- **Asset preprocessing** (`tts`, `transcribe`, `remove-background`) → use `media-use` for voice selection, Whisper model rules, captions, and TTS-to-captions chain.
- **Parametrized renders** (`--variables`) → declared via `data-composition-variables` on `<html>`; see `hyperframes-core` for the full schema.
## Lambda (Cloud Rendering)
skills/hyperframes-cli/references/init-and-scaffold.md
+1 -1
View File
@@ -27,7 +27,7 @@ Other useful flags:
When using `--tailwind`, invoke the `hyperframes-core` (Tailwind reference) skill before editing classes or theme tokens. The scaffold uses Tailwind v4 browser runtime patterns, not Studio's Tailwind v3 setup.
When `--audio` or `--video` is supplied, `init` transcribes the file with Whisper. For voice/model selection see the `hyperframes-media` skill.
When `--audio` or `--video` is supplied, `init` transcribes the file with Whisper. For voice/model selection see the `media-use` skill.
## capture
skills/hyperframes-cli/references/upgrade-info-misc.md
+1 -1
View File
@@ -72,4 +72,4 @@ npx hyperframes remove-background
These produce assets (narration audio, word-level transcripts, transparent video) that get dropped into a composition. Each may download its own model on first run.
For voice selection, Whisper model rules, output format choice, and the TTS → transcript → captions chain, invoke the `hyperframes-media` skill. This skill stays focused on the dev loop.
For voice selection, Whisper model rules, output format choice, and the TTS → transcript → captions chain, invoke the `media-use` skill. This skill stays focused on the dev loop.
skills/hyperframes-core/SKILL.md
+1 -1
View File
@@ -7,7 +7,7 @@ description: The HyperFrames composition contract — build one renderable proje
HyperFrames renders video from HTML. A composition is an HTML file whose DOM declares timing with `data-*` attributes, whose animation runtime is seekable, and whose media playback is owned by the framework.
This skill is the **technical contract** — how to build one hyperframes project. The body below is the build guide; per-topic detail lives in `references/` (index next), read on demand. Other concerns live in the sibling domain skills — `hyperframes-animation`, `hyperframes-creative`, `hyperframes-media`, `hyperframes-cli`, `hyperframes-registry`. The capability map in `/hyperframes` says what each one covers.
This skill is the **technical contract** — how to build one hyperframes project. The body below is the build guide; per-topic detail lives in `references/` (index next), read on demand. Other concerns live in the sibling domain skills — `hyperframes-animation`, `hyperframes-creative`, `media-use`, `hyperframes-cli`, `hyperframes-registry`. The capability map in `/hyperframes` says what each one covers.
## References
skills/hyperframes-core/references/script-format.md
+2 -2
View File
@@ -2,7 +2,7 @@
The **locked narration** for a project: the final spoken lines + voice + delivery. It is an _optional_ plan-layer file — a video with no narration (bgm-only, silent overlay) has none. The storyboard's per-frame `voiceover` is the lighter, editable _guide_; `SCRIPT.md` is the _commit_. (Storyboard format → `references/storyboard-format.md`.)
This file defines the SCRIPT.md **shape** only. Synthesizing the spoken lines into audio is a capability owned by `hyperframes-media` → `references/tts.md`.
This file defines the SCRIPT.md **shape** only. Synthesizing the spoken lines into audio is a capability owned by `media-use` → `references/tts.md`.
Free-form markdown — there is no strict parser; the Studio renders it read-only beside the Storyboard board, and the TTS step extracts the indented spoken lines.
@@ -46,4 +46,4 @@ A header block, then one section per spoken line.
## To TTS
Feed each line's spoken text to `npx hyperframes tts` (pin `--voice` / `--provider` from the header; capture word timestamps for captions). Real per-word timing replaces the `**Time:**` guides. CLI contract → `hyperframes-media/references/tts.md`.
Feed each line's spoken text to `npx hyperframes tts` (pin `--voice` / `--provider` from the header; capture word timestamps for captions). Real per-word timing replaces the `**Time:**` guides. CLI contract → `media-use/audio/references/tts.md`.
skills/hyperframes-media/SKILL.md
-97
View File
@@ -1,97 +0,0 @@
---
name: hyperframes-media
description: Audio and media assets for HyperFrames compositions, produced by one shared audio engine (`scripts/audio.mjs`) — multi-provider TTS (HeyGen / ElevenLabs / Kokoro local), background music + sound effects (HeyGen audio-library retrieval by default, with local Lyria / MusicGen BGM generation and a bundled SFX library as the no-credential fallback), Whisper transcription, background removal, and caption authoring. Use for voiceover / TTS, BGM, SFX / sound effects, transcription, captions / subtitles / lyrics / karaoke / per-word styling, voice + provider selection, and music-mood prompting.
---
# HyperFrames Media
Create the audio and media assets a composition needs — voiceover (TTS), background music + sound effects, transcription, captions, background removal — then consume and animate that data in HTML. For placing assets into compositions, see `hyperframes-core`.
## The audio engine — one source for TTS · BGM · SFX
Workflows do NOT hand-roll audio or vendor a copy. There is one engine — **`scripts/audio.mjs`** — that takes a neutral `audio_request.json` and writes `audio_meta.json` (plus assets under `assets/voice|bgm|sfx`):
```bash
# <MEDIA_DIR> = this skill's directory
node <MEDIA_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json
```
All three capabilities degrade on **ONE switch** — whether a HeyGen credential is present (resolved from `$HEYGEN_API_KEY` / `$HYPERFRAMES_API_KEY` / `~/.heygen`, **not** the CLI):
| Capability | HeyGen credential present | absent |
| ---------- | -------------------------------------------------- | ---------------------------------------------------- |
| TTS | HeyGen Starfish REST (native word timestamps) | → ElevenLabs → Kokoro (chain `transcribe` for words) |
| BGM | HeyGen music **retrieval** | Lyria → MusicGen local **generation** (detached) |
| SFX | HeyGen sound-effects **retrieval** (min_score 0.4) | bundled 21-file library (`assets/sfx/`) |
- **Request** (`audio_request.json`): `{ provider?, lang?, speed?, lines: [{ id, text, sfx?: [names] }], bgm: { mode?, query?, prompt? } }`. `id` joins each line back to the caller's model (a frame number, a scene id, …). `bgm.mode` = `retrieve | generate | none`; omit for auto (retrieve when credentialed, else generate). An **explicit** `retrieve` is strict — it skips rather than starting a detached generate (for callers with no `wait-bgm` step).
- **Output** (`audio_meta.json`, id-keyed): `{ tts_provider, voice_id, bgm, bgm_pending, …, voices: [{ id, path, duration_s, words }], sfx: [{ id, name, file, source, offset_s, duration_s, volume }], total_duration_s }`.
- `--only tts,bgm,sfx` runs a subset and **merges** into an existing `--out` (e.g. TTS+BGM early, SFX once cues exist).
- BGM generate is spawned **detached** (`bgm_pending: true`) — run `scripts/wait-bgm.mjs` before assembling.
- `scripts/heygen-tts.mjs` is a single-shot CLI over the same code (one text → wav + words) for when you just need HeyGen TTS without a request file.
Full flag list + the `audio_meta.json` schema live in the header of `scripts/audio.mjs`. The references below cover the provider details and edge cases behind each capability.
## Preflight — show sign-in status before any audio
**Always run this before generating voice or BGM — inside a full workflow _or_ a one-off "generate me a BGM/voiceover" request.** No HeyGen credential is **not** a reason to silently fall back to local engines: first recommend signing in and let the user decide. Run the shared preflight and **relay its output verbatim** — don't improvise your own "missing key" prompt, and don't offer to write keys into a per-repo `.env`:
```bash
npx hyperframes auth status
```
- **Signed in** → it prints the account; proceed.
- **Not signed in** (`exit 1` is expected here — "not signed in" is a normal state, not a failure) → it prints registration-first guidance. Recommend signing in: `npx hyperframes auth login` is browser OAuth — it **signs in and creates an account** (always available through this repo's CLI). To use an existing HeyGen API key (from app.heygen.com/settings/api), run `npx hyperframes auth login --api-key` — it saves to the shared `~/.heygen` (no per-repo `.env`). The output also lists the local engines voice/BGM will fall back to and a `pip` hint when deps are missing. **Relay this output as-is — don't paraphrase it into your own wording.** Then **STOP and wait** for the user to choose — sign in, or say "go" / "local" to continue offline — **before generating anything.** This is a real decision point, not a passing note: don't fold it into another question, and don't proceed past it on your own. (Exception: in autonomous / non-interactive mode, note the status and continue offline.)
- `npx hyperframes auth status --json` returns `{ configured, recommended_action, offline_engines }` for deterministic branching.
- **If the CLI can't run** (not on PATH and `npx` can't fetch it) → still **recommend signing in** (`npx hyperframes auth login`) and **STOP for the user's choice** — don't treat "no credential" as a silent green light for local generation.
Credential resolution, full key priority, and the local-dependency list are in `references/requirements.md`.
## Provider chains (the detail behind the engine)
**TTS** — first available provider wins (the engine, or `npx hyperframes tts "..."`):
| Order | Provider | Detected when | Word timestamps |
| ----- | ----------------------------- | -------------------------------------------- | ---------------------------------------------------------------- |
| 1 | HeyGen (Starfish) | `$HEYGEN_API_KEY` / `hyperframes auth login` | **Yes, native** — pass `--words narration.words.json` to capture |
| 2 | ElevenLabs | `$ELEVENLABS_API_KEY` set | No — chain `transcribe` after |
| 3 | Kokoro-82M (local, 54 voices) | always (no key required) | No — chain `transcribe` after |
> The published `hyperframes tts` CLI is often the local-only build (its `--help` says "Kokoro-82M", no `--provider`/`--words`) and silently falls back to Kokoro even with `$HEYGEN_API_KEY` set. That is why the engine's HeyGen path is the self-contained `scripts/heygen-tts.mjs` (REST), NOT the CLI; the CLI is used only for the Kokoro path. See `references/tts.md`.
**BGM & SFX** — by default **retrieved** from the HeyGen audio library (`/v3/audio/sounds`), same credential as HeyGen TTS, with the no-credential fallback from the switch above:
| Asset | HeyGen `type` | Lands in | Fallback (no credential) |
| ----- | ------------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------- |
| BGM | `music` | `assets/bgm/track.mp3` (retrieve) · `track.wav` (generate) | Lyria / MusicGen generation |
| SFX | `sound_effects` (min_score 0.4) | `assets/sfx/<slug>.mp3` | bundled 21-file library (`assets/sfx/*` + `manifest.json`) |
See `references/bgm.md` and `references/sfx.md`.
## Routing
| Task | Read |
| ------------------------------------------------------------------- | -------------------------------------------- |
| The audio engine — request/meta schema, `--only`, the switch | `scripts/audio.mjs` (header comment) |
| `npx hyperframes tts` / `heygen-tts.mjs` — providers, voices, words | `references/tts.md` |
| BGM — HeyGen retrieval + local Lyria / MusicGen generation | `references/bgm.md` |
| SFX — HeyGen retrieval (min_score 0.4) + bundled local library | `references/sfx.md` |
| `npx hyperframes transcribe` — Whisper, model rules, output shape | `references/transcribe.md` |
| `npx hyperframes remove-background` — transparent cutouts | `references/remove-background.md` |
| TTS → transcription → captions (no recorded voiceover) | `references/tts-to-captions.md` |
| Caption authoring — style detection, layout, word grouping, exit | `references/captions/authoring.md` |
| Transcript handling — input formats, quality gates, cleanup, APIs | `references/captions/transcript-handling.md` |
| Caption motion — karaoke, marker effects, audio-reactive | `references/captions/motion.md` |
| Model caches, system dependencies, troubleshooting | `references/requirements.md` |
## Non-negotiable rules
- **One engine, no vendored copies.** Produce audio via `scripts/audio.mjs` (or `heygen-tts.mjs` for one-shot HeyGen TTS). Don't re-implement TTS/BGM/SFX inside a workflow — write an `audio_request.json` adapter and call the engine.
- **"HeyGen available" = a resolvable credential, not the CLI.** The whole switch keys off `heygenCredential()`; the published `hyperframes tts` may be Kokoro-only, and there is no `hyperframes bgm` / `hyperframes sfx` command at all.
- **Voice IDs are provider-specific.** `am_michael` is Kokoro-only; HeyGen UUIDs don't work on Kokoro. If you pass `--voice`, also pin `--provider` to avoid silent provider drift when the user's env changes.
- **Always pass `--model` to `transcribe`.** The CLI default `small.en` silently translates non-English audio. See `references/transcribe.md` → "Language Rule".
- **HeyGen returns word timestamps; ElevenLabs / Kokoro do not.** The engine chains `transcribe` automatically for the latter two; standalone, pass `--words` to HeyGen or run `transcribe` against the audio file.
- **Captions consume the flat word-array format** with `{ id, text, start, end }`. See `references/transcribe.md` → "Output Shape".
- **`remove-background --background-output` is hole-cut, not inpainted.** For "scene without the person", a different tool is needed. See `references/remove-background.md` → "When NOT the right tool".
- **BGM/SFX default to HeyGen retrieval; the no-credential fallback is generation (BGM) or the bundled library (SFX).** `/audio/sounds` ranks by a text query — name effects concretely (`glass shatter`, not `dramatic sound`); a no-match **skips**, never blocks the render. SFX sit at volume ~0.35 under voice + BGM. See `references/sfx.md` / `references/bgm.md`.
- **Treat workflow caption HTML as generated output.** For preset-backed videos, the reusable skin source lives at `.hyperframes/caption-skin.html` and the workflow script writes `compositions/captions.html`; do not edit generated `compositions/captions.html` to fix the skin. Rebuild via the workflow's `captions.mjs`, or use that workflow's explicit overrides mechanism when present.
skills/hyperframes/SKILL.md
+1 -2
View File
@@ -31,8 +31,7 @@ Atomic capabilities you load **on demand** — not full video workflows. For "ma
| **Animate** — atomic motion, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU) | `/hyperframes-animation` |
| **Author seek-safe keyframes** — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth, plus `hyperframes keyframes` diagnostics | `/hyperframes-keyframes` |
| **Creative direction** — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive | `/hyperframes-creative` |
| **Media** — TTS voiceover, background music, transcription, background removal, captions | `/hyperframes-media` |
| **Media resolve** — find + freeze BGM, SFX, images, icons from HeyGen catalog into `.media/` with manifest tracking | `/media-use` |
| **Media** — resolve/generate BGM, SFX, image, icon, voice; TTS voiceover, transcription, background removal, captions; cross-project reuse | `/media-use` |
| **CLI dev loop** — init, lint, validate, inspect, preview, render, publish, doctor | `/hyperframes-cli` |
| **Install registry blocks / components** (`hyperframes add`) | `/hyperframes-registry` |
| **Import Figma content** — assets, tokens, components, storyboards→animatics (REST/CLI); Motion, shaders (MCP) | `/figma` |
skills/motion-graphics/phases/source/guide.md
+1 -1
View File
@@ -1,6 +1,6 @@
# source phase — asset sourcing (asset-first)
Runs **only when `shot-plan.json.asset_needs` is non-empty** (the form categories never reach here). Sources each needed asset → a **frozen project-local path** + a ledger (`assets/index.md`). Uses `/hyperframes-media` (capture / asset prep) plus, **when an external asset-search skill such as media-use is installed**, its `resolve` step. If no such search capability is available, it **degrades to asset-free** (see below).
Runs **only when `shot-plan.json.asset_needs` is non-empty** (the form categories never reach here). Sources each needed asset → a **frozen project-local path** + a ledger (`assets/index.md`). Uses `/media-use` (capture / asset prep) plus, **when an external asset-search skill such as media-use is installed**, its `resolve` step. If no such search capability is available, it **degrades to asset-free** (see below).
## Per asset_need
skills/music-to-video/SKILL.md
+1 -1
View File
@@ -24,7 +24,7 @@ Workflow: Step 0 setup → `hyperframes.json` + `assets/bgm.mp3`; Step 1 analyze
Goal: Establish the music source, create the HyperFrames project, and note any user-supplied media.
The **music is the spine** — establish one track before anything else. This skill is tuned for **fast, high-energy BGM**: a strong beat grid drives the cuts (calm tracks work, but pace by phrase rather than beat). If the user gave you audio — a music file, or a video to pull the audio from — use it. If not, generate one: choose the mood from the user's description (e.g. "driving synthwave", "trap beat", "upbeat corporate") and produce a track via `/hyperframes-media` (`references/bgm.md` — HeyGen retrieval when credentialed, else local Lyria / MusicGen; ElevenLabs or another generator also works). Before generating, run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it)** — it shows whether BGM comes from HeyGen or local MusicGen and, if not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or continue offline with local MusicGen — before generating the track**; don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `/hyperframes-media` → Preflight for the canonical guidance. Either way the track lands at `assets/bgm.mp3`. Stage any user-supplied images or videos so frames can weave them in on the beat grid; otherwise typography carries the whole video.
The **music is the spine** — establish one track before anything else. This skill is tuned for **fast, high-energy BGM**: a strong beat grid drives the cuts (calm tracks work, but pace by phrase rather than beat). If the user gave you audio — a music file, or a video to pull the audio from — use it. If not, generate one: choose the mood from the user's description (e.g. "driving synthwave", "trap beat", "upbeat corporate") and produce a track via `/media-use` (`references/bgm.md` — HeyGen retrieval when credentialed, else local Lyria / MusicGen; ElevenLabs or another generator also works). Before generating, run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it)** — it shows whether BGM comes from HeyGen or local MusicGen and, if not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or continue offline with local MusicGen — before generating the track**; don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `/media-use` → Preflight for the canonical guidance. Either way the track lands at `assets/bgm.mp3`. Stage any user-supplied images or videos so frames can weave them in on the beat grid; otherwise typography carries the whole video.
Initialize only if `hyperframes.json` is missing. Name `<project>` from the brief in kebab-case, such as `midnight-drive-loop` — never a timestamp. `init` checks the installed skills against the latest on GitHub and updates the global set if any are out of date.
skills/pr-to-video/SKILL.md
+3 -3
View File
@@ -44,7 +44,7 @@ Initialize only if `hyperframes.json` is missing. Name `<project>` from the PR i
`npx hyperframes init "videos/<project>" --non-interactive --example=blank` — `init` checks the installed skills against the latest on GitHub and updates the global set if any are out of date.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../hyperframes-media` → Preflight for the canonical guidance.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../media-use` → Preflight for the canonical guidance.
**Gate:** `hyperframes.json` exists; the PR ref is captured; angle, length, aspect ratio, and language are locked; sign-in status was shown (signed in, or continuing offline).
@@ -117,7 +117,7 @@ Start audio after Step 3 approval. Run it in the background, then continue to St
`node <SKILL_DIR>/scripts/audio.mjs --script ./SCRIPT.md --storyboard ./STORYBOARD.md --hyperframes . --out ./audio_meta.json &`
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and the same `~/.heygen` credential as TTS. For provider details, read `../hyperframes-media/references/tts.md`.
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and the same `~/.heygen` credential as TTS. For provider details, read `../media-use/audio/references/tts.md`.
If there is no narration and no `SCRIPT.md`, skip voice generation. BGM may still run if the storyboard has a music mood.
@@ -229,7 +229,7 @@ The reusable, domain-agnostic shot shapes live in `../hyperframes-animation/blue
| `[../hyperframes-animation/blueprints-index.md](../hyperframes-animation/blueprints-index.md)` | Step 3: role→blueprint menu. Step 4: pick the shot shape. |
| `[../hyperframes-core/references/storyboard-format.md](../hyperframes-core/references/storyboard-format.md)` | Step 3: write `STORYBOARD.md`. |
| `[../hyperframes-core/references/script-format.md](../hyperframes-core/references/script-format.md)` | Step 3: write `SCRIPT.md`. |
| `[../hyperframes-media/references/tts.md](../hyperframes-media/references/tts.md)` | Step 3.1: choose or understand TTS providers. |
| `[../media-use/audio/references/tts.md](../media-use/audio/references/tts.md)` | Step 3.1: choose or understand TTS providers. |
| `[references/visual-design.md](references/visual-design.md)` | Step 4: write the frame's shot sequence (+ Layout vocabulary). |
| `[references/code-vocabulary.md](references/code-vocabulary.md)` | Step 4 + 5: pick + fill the `code-*` block for a code beat. |
| `[references/motion-language.md](references/motion-language.md)` | Step 4: the motion vocabulary + the motion doctrine. |
skills/product-launch-video/SKILL.md
+3 -3
View File
@@ -25,7 +25,7 @@ Initialize only if `hyperframes.json` is missing. Name `<project>` from the bran
`npx hyperframes init "videos/<project>" --non-interactive --example=blank` — `init` checks the installed skills against the latest on GitHub and updates the global set if any are out of date.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../hyperframes-media` → Preflight for the canonical guidance.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../media-use` → Preflight for the canonical guidance.
**Gate:** `hyperframes.json` exists, and angle, length, aspect ratio, and language are locked; sign-in status was shown (signed in, or continuing offline).
@@ -87,7 +87,7 @@ Start audio after Step 3 approval. Run it in the background, then continue to St
`node <SKILL_DIR>/scripts/audio.mjs --script ./SCRIPT.md --storyboard ./STORYBOARD.md --hyperframes . --out ./audio_meta.json &`
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and uses the same `~/.heygen` credential as TTS. For provider details, read `../hyperframes-media/references/tts.md`.
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and uses the same `~/.heygen` credential as TTS. For provider details, read `../media-use/audio/references/tts.md`.
If there is no narration and no `SCRIPT.md`, skip voice generation. BGM may still run if the storyboard has a music mood.
@@ -199,7 +199,7 @@ The reusable, product-agnostic shot shapes live in `../hyperframes-animation/blu
| `[../hyperframes-animation/blueprints-index.md](../hyperframes-animation/blueprints-index.md)` | Step 3: role→blueprint menu. Step 4: pick the shot shape. |
| `[../hyperframes-core/references/storyboard-format.md](../hyperframes-core/references/storyboard-format.md)` | Step 3: write `STORYBOARD.md`. |
| `[../hyperframes-core/references/script-format.md](../hyperframes-core/references/script-format.md)` | Step 3: write `SCRIPT.md`. |
| `[../hyperframes-media/references/tts.md](../hyperframes-media/references/tts.md)` | Step 3.1: choose or understand TTS providers and voices. |
| `[../media-use/audio/references/tts.md](../media-use/audio/references/tts.md)` | Step 3.1: choose or understand TTS providers and voices. |
| `[references/visual-design.md](references/visual-design.md)` | Step 4: write the frame's shot sequence (+ Layout vocabulary). |
| `[references/motion-language.md](references/motion-language.md)` | Step 4: the motion vocabulary + the motion doctrine. |
| `[references/cut-catalog.md](references/cut-catalog.md)` | Step 4-5: the cut catalog (worker builds within-frame seams). |
skills/website-to-video/SKILL.md
+1 -1
View File
@@ -36,7 +36,7 @@ If you find yourself reasoning "auto mode says bias toward action, so I'll skip
Capture the site, then read the extracted data to understand the **brand and product** — what it does, who it's for, what voice it speaks in, what mood it lives in. The captured assets are a brand toolkit for later, not the building blocks the video is made from.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../hyperframes-media` → Preflight for the canonical guidance.
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../media-use` → Preflight for the canonical guidance.
**Gate:** Site summary printed — strategy-first (what the product does, who it's for, brand voice) before the asset / color / font inventory; sign-in status was shown (signed in, or continuing offline).
skills/website-to-video/references/capabilities.md
+1 -1
View File
@@ -667,7 +667,7 @@ Control bridge actions: play, pause, seek, set-muted, set-playback-rate, enable-
| ------------------------- | ----------------------------------------------------------------------- |
| hyperframes | Core framework skill (composition authoring) |
| hyperframes-cli | All CLI commands as an agent skill |
| hyperframes-media | Media workflows (TTS, transcribe, captions integration) |
| media-use | Media workflows (TTS, transcribe, captions integration) |
| hyperframes-registry | Installing blocks/components |
| hyperframes-contrast | WCAG audit (scripts/contrast-report.mjs) |
| hyperframes-animation-map | Per-tween bbox + flags report |