James RussoandMiao Yang
696cbdbbd0
* chore(skills): package Codex plugin upload * chore(skills): harden Codex plugin content * fix(skills): satisfy plugin quality gates * fix(skills): address plugin packaging review * fix(plugin): simplify asset validation * fix(skills): correct embedded-captions catalog count to 35 after nightcity removal The nightcity theme removal left SKILL.md claiming 36 identities in four places, including the frontmatter description the router reads. The catalog now has 35 entries (10 classic + 25 themed). --------- Co-authored-by: Miao Yang <miao.yang@heygen.com>
7.1 KiB
Setup and providers — install, auth, RAM ladders, forcing a provider
Setup — install heygen first (free-usage path)
Install the HeyGen CLI through its verified release instructions, then run:
heygen update # free usage needs the OAuth-capable CLI (v0.3.0+)
heygen auth login --oauth # OAuth = free subscription credits; --api-key bills API credits
This unlocks the FREE path for bgm/sfx/image/icon catalog search, TTS (voice), and avatar videos. Sign in with --oauth — the free allowance rides on the OAuth session (an API key bills API credits instead). media-use requires heygen >= v0.3.0 uniformly (the OAuth free-usage path needs it), so --doctor nudges older CLIs to update even for API-key-only use. Before resolving anything, verify setup with:
node <SKILL_DIR>/scripts/resolve.mjs --doctor
Providers
media-use holds no keys; every external tool owns its auth. Generation is
centered on the HeyGen CLI free-usage path. Install and authenticate heygen
before resolving bgm/sfx/image/icon/voice/avatar-video. Local tools are opt-in
alternatives where they exist: mflux for image, Kokoro for voice, Parakeet for
transcription, and LTX for local video generation. resolve spec-checks
AVAILABLE RAM for those local ladders (describeModelLadder); the agent can
see the ladder and override.
| Type | Provider / path |
|---|---|
| bgm/sfx | heygen catalog free-usage path |
| image | heygen search free-usage path; optional local mflux; codex image_gen upsell |
| voice | heygen tts free-usage path; optional local Kokoro (free, on-device) |
| icon | heygen asset search free-usage path |
| logo | svgl, then simple-icons, then GitHub org avatar, then domain favicon (all free) |
| grade/lut | local core-preset map, params/CDN look index, deterministic buildCube fallback |
| video | heygen avatar video free-usage path (sign-in nudge on auth failure); optional local LTX (videogen ladder). Image-to-video / photo-avatar / dub stay manual heygen recipes |
Local Kokoro (voice), mflux (image), and LTX (video) run on-device (free,
private, offline once cached). The codex CLI remains the ChatGPT-sub image
upsell. Cost rule (X4): the agent confirms before an agent-initiated paid call;
a user-requested one just runs — heygen.video is flagged paid (metered free
allowance) so an agent-initiated resolve --type video confirms first.
To force a specific generator (e.g. a user says "make this image with codex"),
pass --provider codex: it pins resolution to that provider and skips the
free-usage default. See references/operations.md for the RAM ladders and
provider recipes.
--local-only skips every network provider, including the free HeyGen ones,
leaving the project + global cache and any installed local provider. For
HeyGen-only types, that means no fresh resolve.
CLI tools used (what to run, and how to enable each)
resolve auto-cascades; each provider shells one CLI. HeyGen is the
free-usage path for bgm/sfx/image/icon catalog search, TTS (voice), and avatar
video, so those capabilities need heygen installed and authenticated. Local
tools are OPT-IN alternatives where they exist; install one to unlock its free,
private, on-device path instead of or ahead of HeyGen for that type. Only
ffmpeg/ffprobe are strictly required for the tool to run at all.
| Tool | Serves | Install |
|---|---|---|
ffmpeg/ffprobe |
adopt probing, smart-grade signalstats, cut, duck bake, loudnorm | system package (brew install ffmpeg) |
heygen |
catalog (bgm/sfx/image/icon) + TTS (voice) + avatar video — the free-usage path | install through verified HeyGen release instructions, then heygen auth login --oauth (needs >= v0.3.0) |
mflux-generate |
local image gen (FLUX), best-for-RAM | uv venv ~/.venvs/mflux && VIRTUAL_ENV=~/.venvs/mflux uv pip install mflux==0.9.6 |
codex |
image gen upsell (ChatGPT sub) | Codex CLI, logged in via ChatGPT (owns its own auth) |
parakeet-mlx |
local transcription (default ASR, best) | uv venv ~/.venvs/parakeet && VIRTUAL_ENV=~/.venvs/parakeet uv pip install parakeet-mlx |
ltx-2-mlx |
local video gen | git clone https://github.com/dgrauet/ltx-2-mlx && cd ltx-2-mlx && uv sync --all-extras |
npx hyperframes |
Kokoro TTS (voice), whisper.cpp (transcribe fallback), remove-background | via the hyperframes CLI; whisper.cpp is built on first use (Homebrew on macOS, else git+cmake), models download from HuggingFace |
The RAM-graded local-model shortlist + exact per-tier install/invoke lives in
scripts/lib/local-models.mjs (the agent can read describeModelLadder(cap, specs)
to see which model fits this machine). Without a tool on PATH, its provider
prints a one-line diagnostic to stderr and resolve falls through where another
provider exists (e.g. no mflux -> codex image upsell; no parakeet-mlx -> whisper.cpp).
heygen asset search is a pre-launch command hidden from heygen --help, but it
runs; providers tag requests with the allowlisted X-HeyGen-Client-Source header
(v0.3.0+).