mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-12 07:09:59 +00:00
feat(media-use): add video generation (HeyGen avatar-video + local LTX fallback) (#2614)
* fix(media-use): tag HeyGen TTS generation with attribution header
Centralizes the X-HeyGen-Client-Source header into HEYGEN_CLIENT_SOURCE_ARGV
in heygen-cli.mjs and reuses it in heygen-search.mjs (dropping the duplicated
inline literal) so voice-provider's `voice speech create` call carries it too.
The generation call was previously untagged, making media-use TTS usage
invisible in HeyGen's billing/analytics warehouse; the read-only `voice list`
discovery call intentionally stays untagged.
* feat(media-use): add local LTX video generate provider
* feat(media-use): add HeyGen avatar-video generate provider
* feat(media-use): register video as a real provider type
* docs(media-use): document the wired video type and full HeyGen tagging coverage
resolve --type video is now the default path (HeyGen avatar video first,
local LTX fallback, sign-in nudge on auth failure) instead of a manual
recipe; correct the claim that only search requests are tagged now that
TTS and avatar-video generation carry the attribution header too.
* fix(media-use): wire --avatar-id/--voice-id CLI flags and close video-provider auth/cache gaps
- resolve.mjs never implemented the --avatar-id/--voice-id override that
operations.md documented, so following the docs crashed with
ERR_PARSE_ARGS_UNKNOWN_OPTION; wire the flags through to ctx.
- defaultAvatarId/defaultStarfishVoiceId cached a failed discovery lookup
as a permanent null, disabling heygen.video after one transient miss;
cache only a truthy id, matching the same fix in voice-provider.mjs's
defaultVoiceId.
- the avatar-video onboarding nudge only fired on a video-create failure,
never when avatar/voice discovery itself was unauthenticated (the
common unauthenticated case) -- propagate the discovery failure reason
so onboarding fires either way.
- dedupe the CLI-shelling JSON helper (heygen-cli.mjs's new runHeygenJson)
and the local-model argv-template builder (local-models.mjs's new
buildArgv) instead of leaving byte-identical copies in each provider.
* fix(media-use): address avatar-video PR review feedback
- heygenVideoGenerate short-circuits after the first discovery-call
failure instead of always attempting both avatar list and voice list,
so an unauthenticated caller gets one onboarding message and one
provider-error telemetry ping instead of a double-fire.
- runHeygenJson logs a diagnostic when a CLI call succeeds but returns
unparseable JSON, instead of silently returning null.
- dedupe the "avatar video is free" onboarding string into one constant
(was duplicated across three call sites).
* fix(media-use): match review-requested naming and message conventions
- export AVATAR_VIDEO_SIGNIN_MESSAGE from heygen-video-provider.mjs so
the test imports the canonical string instead of redeclaring it.
- runHeygenJson's non-JSON diagnostic now matches heygen-search.mjs's
existing wording ("returned non-JSON output").
This commit is contained in:
Miguel Ángel
@@ -46,8 +46,8 @@
|
|||||||
"files": 10
|
"files": 10
|
||||||
},
|
},
|
||||||
"media-use": {
|
"media-use": {
|
||||||
"hash": "b036210167a3ca39",
|
"hash": "f15fce08a493d70e",
|
||||||
"files": 133
|
"files": 139
|
||||||
},
|
},
|
||||||
"motion-graphics": {
|
"motion-graphics": {
|
||||||
"hash": "cc93e3d220a0ebf5",
|
"hash": "cc93e3d220a0ebf5",
|
||||||
|
|||||||
+31
-26
@@ -25,20 +25,20 @@ node <SKILL_DIR>/scripts/resolve.mjs --doctor
|
|||||||
|
|
||||||
HyperFrames owns media _playback_; media-use owns everything else. Each row is enforced by `scripts/lib/coverage.test.mjs` so the claim can't rot.
|
HyperFrames owns media _playback_; media-use owns everything else. Each row is enforced by `scripts/lib/coverage.test.mjs` so the claim can't rot.
|
||||||
|
|
||||||
| HyperFrames gap | media-use owns it via |
|
| HyperFrames gap | media-use owns it via |
|
||||||
| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| Audio-only, no image/icon | `resolve --type image\|icon` (heygen asset search) |
|
| Audio-only, no image/icon | `resolve --type image\|icon` (heygen asset search) |
|
||||||
| No third-party brand logos | `resolve --type logo` (svgl → simple-icons → GitHub org avatar → domain favicon) |
|
| No third-party brand logos | `resolve --type logo` (svgl → simple-icons → GitHub org avatar → domain favicon) |
|
||||||
| No voice / audio generation | `resolve --type voice` (HeyGen TTS free-usage path; optional local Kokoro) + the audio engine (`audio/scripts/audio.mjs`) |
|
| No voice / audio generation | `resolve --type voice` (HeyGen TTS free-usage path; optional local Kokoro) + the audio engine (`audio/scripts/audio.mjs`) |
|
||||||
| Scattered/duplicated audio engine | one consolidated engine under `audio/` (hyperframes-media retired) |
|
| Scattered/duplicated audio engine | one consolidated engine under `audio/` (hyperframes-media retired) |
|
||||||
| No agent media-ops (cut/reframe/transform) | `references/operations.md` + `resolve --from` to register outputs |
|
| No agent media-ops (cut/reframe/transform) | `references/operations.md` + `resolve --from` to register outputs |
|
||||||
| No transcript-driven cutting | `scripts/transcript-cut.mjs` compiles word-timestamp edits into cut lists |
|
| No transcript-driven cutting | `scripts/transcript-cut.mjs` compiles word-timestamp edits into cut lists |
|
||||||
| No auto-duck / publish loudness | `scripts/audio-duck.mjs` + `references/operations.md` loudnorm/sidechain recipes |
|
| No auto-duck / publish loudness | `scripts/audio-duck.mjs` + `references/operations.md` loudnorm/sidechain recipes |
|
||||||
| No cross-project memory | global content-addressed cache + auto-promote (`~/.media`) |
|
| No cross-project memory | global content-addressed cache + auto-promote (`~/.media`) |
|
||||||
| No color-grade authoring | `resolve --type grade` emits a paste-ready `data-color-grading` block; `resolve --type lut` freezes validated `.cube` files |
|
| No color-grade authoring | `resolve --type grade` emits a paste-ready `data-color-grading` block; `resolve --type lut` freezes validated `.cube` files |
|
||||||
| No image generation | RAM-graded local mflux (FLUX) via `scripts/lib/mflux-provider.mjs`, codex `image_gen` upsell (`scripts/lib/codex-provider.mjs`) |
|
| No image generation | RAM-graded local mflux (FLUX) via `scripts/lib/mflux-provider.mjs`, codex `image_gen` upsell (`scripts/lib/codex-provider.mjs`) |
|
||||||
| No video generation | HeyGen avatar video + image-to-video (animate any still into a talking clip), photo-avatar, dub/translate — `heygen` CLI free-usage path (`references/operations.md`); optional spec-gated local LTX (`videogen` in `scripts/lib/local-models.mjs`) |
|
| No video generation | `resolve --type video` — HeyGen avatar video first (free-usage path, sign-in nudge on auth failure), local LTX fallback (`videogen` in `scripts/lib/local-models.mjs`); image-to-video, photo-avatar, dub/translate remain manual `heygen` CLI recipes (`references/operations.md`) |
|
||||||
| Weak local-model defaults | HeyGen free-usage path via the `heygen` CLI; local open-source tools only as opt-in alternatives (`scripts/lib/local-run.mjs`) |
|
| Weak local-model defaults | HeyGen free-usage path via the `heygen` CLI; local open-source tools only as opt-in alternatives (`scripts/lib/local-run.mjs`) |
|
||||||
|
|
||||||
## When to use
|
## When to use
|
||||||
|
|
||||||
@@ -249,20 +249,21 @@ transcription, and LTX for local video generation. `resolve` spec-checks
|
|||||||
AVAILABLE RAM for those local ladders (`describeModelLadder`); the agent can
|
AVAILABLE RAM for those local ladders (`describeModelLadder`); the agent can
|
||||||
see the ladder and override.
|
see the ladder and override.
|
||||||
|
|
||||||
| Type | Provider / path |
|
| Type | Provider / path |
|
||||||
| --------- | ----------------------------------------------------------------------------------------------------------- |
|
| --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| bgm/sfx | heygen catalog free-usage path |
|
| bgm/sfx | heygen catalog free-usage path |
|
||||||
| image | heygen search free-usage path; optional local mflux; codex `image_gen` upsell |
|
| image | heygen search free-usage path; optional local mflux; codex `image_gen` upsell |
|
||||||
| voice | heygen tts free-usage path; optional local **Kokoro** (free, on-device) |
|
| voice | heygen tts free-usage path; optional local **Kokoro** (free, on-device) |
|
||||||
| icon | heygen asset search free-usage path |
|
| icon | heygen asset search free-usage path |
|
||||||
| logo | svgl, then simple-icons, then GitHub org avatar, then domain favicon (all free) |
|
| logo | svgl, then simple-icons, then GitHub org avatar, then domain favicon (all free) |
|
||||||
| grade/lut | local core-preset map, params/CDN look index, deterministic `buildCube` fallback |
|
| grade/lut | local core-preset map, params/CDN look index, deterministic `buildCube` fallback |
|
||||||
| video | heygen avatar / image-to-video / photo-avatar / dub free-usage path; optional local LTX (`videogen` ladder) |
|
| video | heygen avatar video free-usage path (sign-in nudge on auth failure); optional local LTX (`videogen` ladder). Image-to-video / photo-avatar / dub stay manual `heygen` recipes |
|
||||||
|
|
||||||
Local Kokoro (voice), mflux (image), and LTX (video) run on-device (free,
|
Local Kokoro (voice), mflux (image), and LTX (video) run on-device (free,
|
||||||
private, offline once cached). The `codex` CLI remains the ChatGPT-sub image
|
private, offline once cached). The `codex` CLI remains the ChatGPT-sub image
|
||||||
upsell. Cost rule (X4): the agent confirms before an agent-initiated paid call;
|
upsell. Cost rule (X4): the agent confirms before an agent-initiated paid call;
|
||||||
a user-requested one just runs.
|
a user-requested one just runs — `heygen.video` is flagged paid (metered free
|
||||||
|
allowance) so an agent-initiated `resolve --type video` confirms first.
|
||||||
|
|
||||||
To force a specific generator (e.g. a user says "make this image with codex"),
|
To force a specific generator (e.g. a user says "make this image with codex"),
|
||||||
pass `--provider codex`: it pins resolution to that provider and skips the
|
pass `--provider codex`: it pins resolution to that provider and skips the
|
||||||
@@ -425,8 +426,12 @@ prints a one-line diagnostic to stderr and resolve falls through where another
|
|||||||
provider exists (e.g. no `mflux` -> codex image upsell; no `parakeet-mlx` -> whisper.cpp).
|
provider exists (e.g. no `mflux` -> codex image upsell; no `parakeet-mlx` -> whisper.cpp).
|
||||||
|
|
||||||
`heygen asset search` is a pre-launch command hidden from `heygen --help`, but it
|
`heygen asset search` is a pre-launch command hidden from `heygen --help`, but it
|
||||||
runs; providers tag requests with the allowlisted `X-HeyGen-Client-Source` header
|
runs. Every media-use call that shells `heygen` — catalog search AND every
|
||||||
(v0.3.0+).
|
generating call (TTS, avatar video) — tags requests with the allowlisted
|
||||||
|
`X-HeyGen-Client-Source: media-use` header (v0.3.0+), sourced from one shared
|
||||||
|
constant (`HEYGEN_CLIENT_SOURCE_ARGV` in `scripts/lib/heygen-cli.mjs`) so a
|
||||||
|
future call site can't silently ship untagged. Read-only discovery calls
|
||||||
|
(`voice list`, `avatar list`) are intentionally left untagged.
|
||||||
|
|
||||||
## Telemetry
|
## Telemetry
|
||||||
|
|
||||||
|
|||||||
@@ -197,44 +197,64 @@ GENERATES. Two paths, best-for-the-machine picked automatically:
|
|||||||
|
|
||||||
`--local-only` keeps mflux (once cached) and skips codex (network).
|
`--local-only` keeps mflux (once cached) and skips codex (network).
|
||||||
|
|
||||||
## Generate: video (local first, HeyGen avatar upsell)
|
## Generate: video (`resolve --type video`, HeyGen avatar first)
|
||||||
|
|
||||||
Operate-on-video ships now; GENERATING video is local-first with a HeyGen
|
`resolve --type video "<intent>"` is the default path. It generates a
|
||||||
avatar upsell (decision X3).
|
script-driven HeyGen avatar video first (the free-usage allowance — OAuth
|
||||||
|
sessions ride the web-plan free avatar-video quota where eligible, API keys
|
||||||
|
follow normal API billing), falling back to local generative LTX only when
|
||||||
|
HeyGen is unavailable, uncredentialed, or `--local-only` is passed. The two
|
||||||
|
are non-substitutable outputs (a real presenter vs. a generic generative
|
||||||
|
clip), so treat the fallback as "HeyGen wasn't reachable," not "upgrade the
|
||||||
|
quality":
|
||||||
|
|
||||||
- **Local (default): LTX 2.3 on MLX** via `dgrauet/ltx-2-mlx`, the `videogen`
|
- **HeyGen avatar video (default, free for new API users):**
|
||||||
ladder in `local-models.mjs`. Generative clips (t2v / i2v), spec-gated to RAM.
|
`heygenVideoGenerate` (`scripts/lib/heygen-video-provider.mjs`) shells the
|
||||||
Verified on 24GB: 512x320 x 33f with audio.
|
`heygen` CLI — never the raw API — auto-picking a public avatar and a
|
||||||
- **HeyGen avatar upsell (better, script-driven): the `heygen` CLI**, NOT the
|
starfish voice (override with `--avatar-id`/`--voice-id`, threaded through
|
||||||
raw API. For a talking-head / avatar video, `heygen video create` (avatar
|
as `ctx.avatarId`/`ctx.voiceId`). If the CLI reports `not_authenticated`,
|
||||||
engine IV by default) beats a generative clip when you want a real presenter.
|
the provider prints an onboarding recommendation (avatar video is free for
|
||||||
Browser OAuth uses the web-plan/free avatar-video allowance where eligible;
|
new API users — sign in) to stderr and falls through to LTX instead of
|
||||||
API keys follow the normal API billing path. Always pass
|
hard-failing.
|
||||||
`--headers "X-HeyGen-Client-Source: media-use"` on any generating `heygen`
|
- **Local fallback: LTX 2.3 on MLX** via `dgrauet/ltx-2-mlx`, the `videogen`
|
||||||
command (`video create`, `avatar create`, `video-translate`) — it's the
|
ladder in `local-models.mjs` (`ltx-video-provider.mjs`). Generative clips
|
||||||
allowlisted attribution header (persistent flag, works on every subcommand) that
|
(t2v), spec-gated to RAM. Verified on 24GB: 512x320 x 33f with audio.
|
||||||
tags the usage as media-use in billing/resource meta so it shows up in the API
|
|
||||||
dashboards. Read-only discovery (`avatar list`, `voice list`) doesn't need it.
|
|
||||||
|
|
||||||
```bash
|
Every generating `heygen` call from media-use — TTS, avatar video, and
|
||||||
# discover an avatar + a starfish voice, then create + wait
|
catalog search — sends the allowlisted `X-HeyGen-Client-Source: media-use`
|
||||||
heygen avatar list --ownership public --limit 5
|
header (persistent flag, works on every subcommand) via the shared
|
||||||
heygen voice list --engine starfish --limit 5
|
`HEYGEN_CLIENT_SOURCE_ARGV` constant (`scripts/lib/heygen-cli.mjs`), so usage
|
||||||
heygen video create --headers "X-HeyGen-Client-Source: media-use" --wait -d '{
|
tags correctly in billing/resource meta and shows up in the API dashboards.
|
||||||
"type": "avatar",
|
Read-only discovery (`avatar list`, `voice list`) doesn't need it.
|
||||||
"avatar_id": "<avatar-id>",
|
|
||||||
"script": "Your narration here.",
|
|
||||||
"voice_id": "<voice-id>"
|
|
||||||
}'
|
|
||||||
```
|
|
||||||
|
|
||||||
Avatar videos are deterministic + script-driven (lip-sync from a script or a
|
For structured bodies `resolve --type video` doesn't expose yet (a specific
|
||||||
pre-recorded `audio_url`), distinct from the generative LTX clips. After it
|
`avatar_id`/`voice_id` combination beyond the ctx overrides, or a
|
||||||
renders, `resolve --from <downloaded.mp4> --type video` to ledger it.
|
pre-recorded `audio_url` instead of a script), the raw `heygen video create`
|
||||||
|
recipe below remains the escape hatch:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# discover an avatar + a starfish voice, then create + wait
|
||||||
|
heygen avatar list --ownership public --limit 5
|
||||||
|
heygen voice list --engine starfish --limit 5
|
||||||
|
heygen video create --headers "X-HeyGen-Client-Source: media-use" --wait -d '{
|
||||||
|
"type": "avatar",
|
||||||
|
"avatar_id": "<avatar-id>",
|
||||||
|
"script": "Your narration here.",
|
||||||
|
"voice_id": "<voice-id>"
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Avatar videos are deterministic + script-driven (lip-sync from a script or a
|
||||||
|
pre-recorded `audio_url`), distinct from the generative LTX clips. After a
|
||||||
|
manual recipe renders, `resolve --from <downloaded.mp4> --type video` to
|
||||||
|
ledger it (not needed when generating via `resolve --type video` directly —
|
||||||
|
that already ledgers the result).
|
||||||
|
|
||||||
### Image-to-video (animate any still into a talking clip)
|
### Image-to-video (animate any still into a talking clip)
|
||||||
|
|
||||||
`heygen video create` takes the raw `POST /v3/videos` body, so switching `type`
|
Not wired into `resolve --type video` (deferred — the `avatar` type covers
|
||||||
|
the default script-driven case). `heygen video create` takes the raw
|
||||||
|
`POST /v3/videos` body, so switching `type`
|
||||||
from `avatar` to `image` animates **any image of a person** into a lip-synced
|
from `avatar` to `image` animates **any image of a person** into a lip-synced
|
||||||
talking video, with no avatar/photo-avatar creation step first. Point `image` at a
|
talking video, with no avatar/photo-avatar creation step first. Point `image` at a
|
||||||
public URL or an uploaded `asset_id`, and drive speech with a `script`+`voice_id`
|
public URL or an uploaded `asset_id`, and drive speech with a `script`+`voice_id`
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { execFileSync } from "node:child_process";
|
||||||
import { track } from "./telemetry.mjs";
|
import { track } from "./telemetry.mjs";
|
||||||
|
|
||||||
// v0.3.0 is the first CLI that can use an OAuth session; v0.1.x/0.2.x reject it
|
// v0.3.0 is the first CLI that can use an OAuth session; v0.1.x/0.2.x reject it
|
||||||
@@ -10,6 +11,7 @@ export const HEYGEN_INSTALL_COMMAND =
|
|||||||
"curl -fsSL https://static.heygen.ai/cli/install.sh | bash && heygen auth login --oauth";
|
"curl -fsSL https://static.heygen.ai/cli/install.sh | bash && heygen auth login --oauth";
|
||||||
export const HEYGEN_AUTH_COMMAND = "heygen auth login --oauth";
|
export const HEYGEN_AUTH_COMMAND = "heygen auth login --oauth";
|
||||||
export const HEYGEN_UPDATE_COMMAND = "heygen update";
|
export const HEYGEN_UPDATE_COMMAND = "heygen update";
|
||||||
|
export const HEYGEN_CLIENT_SOURCE_ARGV = ["--headers", "X-HeyGen-Client-Source: media-use"];
|
||||||
|
|
||||||
export const HEYGEN_NOT_FOUND_MESSAGE = `media-use: heygen CLI not found — it's the free path for bgm/image/voice/avatar-video. Install: ${HEYGEN_INSTALL_COMMAND}`;
|
export const HEYGEN_NOT_FOUND_MESSAGE = `media-use: heygen CLI not found — it's the free path for bgm/image/voice/avatar-video. Install: ${HEYGEN_INSTALL_COMMAND}`;
|
||||||
export const HEYGEN_NOT_AUTHENTICATED_MESSAGE = `media-use: heygen CLI not authenticated (free usage) — run: ${HEYGEN_AUTH_COMMAND}`;
|
export const HEYGEN_NOT_AUTHENTICATED_MESSAGE = `media-use: heygen CLI not authenticated (free usage) — run: ${HEYGEN_AUTH_COMMAND}`;
|
||||||
@@ -132,6 +134,31 @@ export async function flushHeygenFailureTracking() {
|
|||||||
await Promise.all(pendingFailureTracking);
|
await Promise.all(pendingFailureTracking);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Shared discovery/generation helper for the CLI-shelling providers (voice,
|
||||||
|
// avatar-video): run a heygen JSON subcommand, report+classify on failure,
|
||||||
|
// and hand the caller the classified reason via onError (used by callers that
|
||||||
|
// need to distinguish e.g. not_authenticated from other failures).
|
||||||
|
export function runHeygenJson(bin, argv, label, onError) {
|
||||||
|
let out;
|
||||||
|
try {
|
||||||
|
out = execFileSync(bin, argv, {
|
||||||
|
encoding: "utf8",
|
||||||
|
timeout: 120000,
|
||||||
|
stdio: ["pipe", "pipe", "pipe"],
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
reportHeygenFailure(err, `${bin} ${label}`);
|
||||||
|
onError?.(classifyHeygenErrorCode(err));
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return JSON.parse(out);
|
||||||
|
} catch {
|
||||||
|
console.error(`media-use: \`${bin} ${label}\` returned non-JSON output`);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export function firstSemver(text) {
|
export function firstSemver(text) {
|
||||||
const match = String(text || "").match(/\bv?(\d+)\.(\d+)\.(\d+)\b/);
|
const match = String(text || "").match(/\bv?(\d+)\.(\d+)\.(\d+)\b/);
|
||||||
return match ? `${match[1]}.${match[2]}.${match[3]}` : null;
|
return match ? `${match[1]}.${match[2]}.${match[3]}` : null;
|
||||||
|
|||||||
@@ -1,18 +1,12 @@
|
|||||||
import { execFileSync } from "node:child_process";
|
import { execFileSync } from "node:child_process";
|
||||||
import { reportHeygenFailure } from "./heygen-cli.mjs";
|
import { HEYGEN_CLIENT_SOURCE_ARGV, reportHeygenFailure } from "./heygen-cli.mjs";
|
||||||
|
|
||||||
export function heygenSearch(subcommand, query, { type, limit = 5, minScore } = {}) {
|
export function heygenSearch(subcommand, query, { type, limit = 5, minScore } = {}) {
|
||||||
// execFileSync with an argv array (no shell), so query/type/etc. are passed as
|
// execFileSync with an argv array (no shell), so query/type/etc. are passed as
|
||||||
// literal arguments — no quoting tricks, no command injection. subcommand is a
|
// literal arguments — no quoting tricks, no command injection. subcommand is a
|
||||||
// hardcoded multi-word string (e.g. "audio sounds list"), split into tokens.
|
// hardcoded multi-word string (e.g. "audio sounds list"), split into tokens.
|
||||||
// Tag the caller via the CLI's allowlisted attribution header (heygen >= v0.3.0).
|
// Tag the caller via the CLI's allowlisted attribution header (heygen >= v0.3.0).
|
||||||
const args = [
|
const args = [...HEYGEN_CLIENT_SOURCE_ARGV, ...subcommand.split(" "), "--query", query];
|
||||||
"--headers",
|
|
||||||
"X-HeyGen-Client-Source: media-use",
|
|
||||||
...subcommand.split(" "),
|
|
||||||
"--query",
|
|
||||||
query,
|
|
||||||
];
|
|
||||||
if (type) args.push("--type", type);
|
if (type) args.push("--type", type);
|
||||||
args.push("--limit", String(limit));
|
args.push("--limit", String(limit));
|
||||||
// Server-side score floor. Honored by `audio sounds list`; the `asset search`
|
// Server-side score floor. Honored by `audio sounds list`; the `asset search`
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
import { strict as assert } from "node:assert";
|
||||||
|
import { chmodSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { test } from "node:test";
|
||||||
|
import { HEYGEN_CLIENT_SOURCE_HEADERS } from "../../audio/scripts/lib/heygen.mjs";
|
||||||
|
import { HEYGEN_CLIENT_SOURCE_ARGV } from "./heygen-cli.mjs";
|
||||||
|
import { heygenSearch } from "./heygen-search.mjs";
|
||||||
|
|
||||||
|
test("tags HeyGen searches with the shared media-use client source", () => {
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "media-use-heygen-search-"));
|
||||||
|
const capturePath = join(dir, "argv.log");
|
||||||
|
const heygenPath = join(dir, "heygen");
|
||||||
|
const previousPath = process.env.PATH;
|
||||||
|
const previousCapturePath = process.env.HEYGEN_CAPTURE_PATH;
|
||||||
|
|
||||||
|
writeFileSync(
|
||||||
|
heygenPath,
|
||||||
|
`#!/bin/sh
|
||||||
|
printf '%s\\n' "$*" >> "$HEYGEN_CAPTURE_PATH"
|
||||||
|
printf '%s\\n' '{"data":[{"id":"x"}]}'
|
||||||
|
`,
|
||||||
|
);
|
||||||
|
chmodSync(heygenPath, 0o755);
|
||||||
|
process.env.PATH = `${dir}:${previousPath ?? ""}`;
|
||||||
|
process.env.HEYGEN_CAPTURE_PATH = capturePath;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = heygenSearch("audio sounds list", "ocean", { limit: 1 });
|
||||||
|
const argv = readFileSync(capturePath, "utf8").trim();
|
||||||
|
|
||||||
|
assert.deepEqual(result, [{ id: "x" }]);
|
||||||
|
assert.match(argv, /X-HeyGen-Client-Source: media-use/);
|
||||||
|
} finally {
|
||||||
|
if (previousPath === undefined) delete process.env.PATH;
|
||||||
|
else process.env.PATH = previousPath;
|
||||||
|
if (previousCapturePath === undefined) delete process.env.HEYGEN_CAPTURE_PATH;
|
||||||
|
else process.env.HEYGEN_CAPTURE_PATH = previousCapturePath;
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("keeps CLI and REST media-use client source headers in lockstep", () => {
|
||||||
|
const [entry] = Object.entries(HEYGEN_CLIENT_SOURCE_HEADERS);
|
||||||
|
assert.ok(entry);
|
||||||
|
const [key, value] = entry;
|
||||||
|
|
||||||
|
assert.equal(HEYGEN_CLIENT_SOURCE_ARGV[1], `${key}: ${value}`);
|
||||||
|
});
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
import { execFileSync } from "node:child_process";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { freezeUrl } from "./freeze.mjs";
|
||||||
|
import {
|
||||||
|
classifyHeygenErrorCode,
|
||||||
|
HEYGEN_AUTH_COMMAND,
|
||||||
|
HEYGEN_CLIENT_SOURCE_ARGV,
|
||||||
|
reportHeygenFailure,
|
||||||
|
runHeygenJson,
|
||||||
|
} from "./heygen-cli.mjs";
|
||||||
|
|
||||||
|
export const AVATAR_VIDEO_SIGNIN_MESSAGE = `media-use: avatar video is free for new API users — sign in: ${HEYGEN_AUTH_COMMAND}`;
|
||||||
|
|
||||||
|
// Cache only a truthy id -- a transient discovery failure must not poison the
|
||||||
|
// cache with `null` and permanently disable heygen.video for the rest of the
|
||||||
|
// process. `onError` lets the caller distinguish not_authenticated (and nudge
|
||||||
|
// onboarding) from any other discovery failure.
|
||||||
|
let cachedAvatarId;
|
||||||
|
function defaultAvatarId(onError) {
|
||||||
|
if (cachedAvatarId) return cachedAvatarId;
|
||||||
|
const j = runHeygenJson(
|
||||||
|
"heygen",
|
||||||
|
["avatar", "list", "--ownership", "public", "--limit", "1"],
|
||||||
|
"avatar list",
|
||||||
|
onError,
|
||||||
|
);
|
||||||
|
cachedAvatarId = j?.data?.[0]?.avatar_id || null;
|
||||||
|
return cachedAvatarId;
|
||||||
|
}
|
||||||
|
|
||||||
|
let cachedStarfishVoiceId;
|
||||||
|
function defaultStarfishVoiceId(onError) {
|
||||||
|
if (cachedStarfishVoiceId) return cachedStarfishVoiceId;
|
||||||
|
const j = runHeygenJson(
|
||||||
|
"heygen",
|
||||||
|
["voice", "list", "--engine", "starfish", "--limit", "1"],
|
||||||
|
"voice list",
|
||||||
|
onError,
|
||||||
|
);
|
||||||
|
cachedStarfishVoiceId = j?.data?.[0]?.voice_id || null;
|
||||||
|
return cachedStarfishVoiceId;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function heygenVideoGenerate(intent, ctx) {
|
||||||
|
let discoveryFailureReason = null;
|
||||||
|
const captureReason = (reason) => {
|
||||||
|
discoveryFailureReason ??= reason;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Short-circuit: once one discovery call fails, the result is null either
|
||||||
|
// way, so don't attempt the second -- that would double-fire the onboarding
|
||||||
|
// message and the provider-error telemetry ping for what's really one failure.
|
||||||
|
const avatarId = ctx?.avatarId || defaultAvatarId(captureReason);
|
||||||
|
if (!avatarId) {
|
||||||
|
if (discoveryFailureReason === "not_authenticated") console.error(AVATAR_VIDEO_SIGNIN_MESSAGE);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const voiceId = ctx?.voiceId || defaultStarfishVoiceId(captureReason);
|
||||||
|
if (!voiceId) {
|
||||||
|
if (discoveryFailureReason === "not_authenticated") console.error(AVATAR_VIDEO_SIGNIN_MESSAGE);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
let out;
|
||||||
|
try {
|
||||||
|
out = execFileSync(
|
||||||
|
"heygen",
|
||||||
|
[
|
||||||
|
...HEYGEN_CLIENT_SOURCE_ARGV,
|
||||||
|
"video",
|
||||||
|
"create",
|
||||||
|
"--wait",
|
||||||
|
"-d",
|
||||||
|
JSON.stringify({
|
||||||
|
type: "avatar",
|
||||||
|
avatar_id: avatarId,
|
||||||
|
script: intent,
|
||||||
|
voice_id: voiceId,
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
{
|
||||||
|
encoding: "utf8",
|
||||||
|
timeout: 300000,
|
||||||
|
stdio: ["pipe", "pipe", "pipe"],
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} catch (err) {
|
||||||
|
if (classifyHeygenErrorCode(err) === "not_authenticated") {
|
||||||
|
console.error(AVATAR_VIDEO_SIGNIN_MESSAGE);
|
||||||
|
}
|
||||||
|
reportHeygenFailure(err, "heygen video create");
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
let parsed;
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(out);
|
||||||
|
} catch {
|
||||||
|
console.error("media-use: `heygen video create` returned invalid JSON");
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const videoUrl = parsed?.data?.video_url;
|
||||||
|
if (typeof videoUrl !== "string" || !videoUrl) {
|
||||||
|
console.error("media-use: `heygen video create` returned no video URL");
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const tmpPath = join(tmpdir(), `media-use-heygen-video-${process.pid}-${Date.now()}.mp4`);
|
||||||
|
try {
|
||||||
|
await freezeUrl(videoUrl, tmpPath);
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`media-use: heygen video download failed: ${err.message}`);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
localPath: tmpPath,
|
||||||
|
ext: ".mp4",
|
||||||
|
source: "generated",
|
||||||
|
metadata: {
|
||||||
|
description: intent,
|
||||||
|
provider: "heygen.video",
|
||||||
|
provenance: { prompt: intent },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,344 @@
|
|||||||
|
import { strict as assert } from "node:assert";
|
||||||
|
import { chmodSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||||
|
import http from "node:http";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { test } from "node:test";
|
||||||
|
import { HEYGEN_NOT_AUTHENTICATED_MESSAGE } from "./heygen-cli.mjs";
|
||||||
|
import { AVATAR_VIDEO_SIGNIN_MESSAGE } from "./heygen-video-provider.mjs";
|
||||||
|
|
||||||
|
const VIDEO_FIXTURE = Buffer.from("tiny heygen video fixture");
|
||||||
|
let importCount = 0;
|
||||||
|
|
||||||
|
async function freshGenerate() {
|
||||||
|
importCount += 1;
|
||||||
|
const module = await import(`./heygen-video-provider.mjs?test=${importCount}`);
|
||||||
|
return module.heygenVideoGenerate;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function listenVideoServer() {
|
||||||
|
const server = http.createServer((req, res) => {
|
||||||
|
if (req.url !== "/video.mp4") {
|
||||||
|
res.writeHead(404).end();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
res.writeHead(200, {
|
||||||
|
"content-length": VIDEO_FIXTURE.length,
|
||||||
|
"content-type": "video/mp4",
|
||||||
|
});
|
||||||
|
res.end(VIDEO_FIXTURE);
|
||||||
|
});
|
||||||
|
await new Promise((resolve) => server.listen(0, resolve));
|
||||||
|
const address = server.address();
|
||||||
|
assert.ok(address && typeof address !== "string");
|
||||||
|
return {
|
||||||
|
server,
|
||||||
|
url: `http://127.0.0.1:${address.port}/video.mp4`,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function listenFailingVideoServer() {
|
||||||
|
const server = http.createServer((req, res) => {
|
||||||
|
res.writeHead(500).end();
|
||||||
|
});
|
||||||
|
await new Promise((resolve) => server.listen(0, resolve));
|
||||||
|
const address = server.address();
|
||||||
|
assert.ok(address && typeof address !== "string");
|
||||||
|
return {
|
||||||
|
server,
|
||||||
|
url: `http://127.0.0.1:${address.port}/video.mp4`,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function closeServer(server) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
server.close((err) => (err ? reject(err) : resolve()));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function withFakeHeygen(options, run) {
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "media-use-heygen-video-provider-"));
|
||||||
|
const capturePath = join(dir, "argv.log");
|
||||||
|
const heygenPath = join(dir, "heygen");
|
||||||
|
const previousEnv = {
|
||||||
|
PATH: process.env.PATH,
|
||||||
|
HEYGEN_CAPTURE_PATH: process.env.HEYGEN_CAPTURE_PATH,
|
||||||
|
HEYGEN_VIDEO_MODE: process.env.HEYGEN_VIDEO_MODE,
|
||||||
|
HEYGEN_VIDEO_RESPONSE: process.env.HEYGEN_VIDEO_RESPONSE,
|
||||||
|
HEYGEN_DISCOVERY_MODE: process.env.HEYGEN_DISCOVERY_MODE,
|
||||||
|
HYPERFRAMES_NO_TELEMETRY: process.env.HYPERFRAMES_NO_TELEMETRY,
|
||||||
|
};
|
||||||
|
|
||||||
|
writeFileSync(
|
||||||
|
heygenPath,
|
||||||
|
`#!/bin/sh
|
||||||
|
printf '%s\\n' "$*" >> "$HEYGEN_CAPTURE_PATH"
|
||||||
|
case "$*" in
|
||||||
|
*"avatar list"*)
|
||||||
|
case "$HEYGEN_DISCOVERY_MODE" in
|
||||||
|
auth) printf '%s\\n' 'HTTP 401 Unauthorized' >&2; exit 1 ;;
|
||||||
|
*) printf '%s\\n' '{"data":[{"avatar_id":"avatar-public-1"}]}' ;;
|
||||||
|
esac
|
||||||
|
;;
|
||||||
|
*"voice list"*)
|
||||||
|
case "$HEYGEN_DISCOVERY_MODE" in
|
||||||
|
auth) printf '%s\\n' 'HTTP 401 Unauthorized' >&2; exit 1 ;;
|
||||||
|
*) printf '%s\\n' '{"data":[{"voice_id":"voice-starfish-1"}]}' ;;
|
||||||
|
esac
|
||||||
|
;;
|
||||||
|
*"video create"*)
|
||||||
|
case "$HEYGEN_VIDEO_MODE" in
|
||||||
|
auth) printf '%s\\n' 'HTTP 401 Unauthorized' >&2; exit 1 ;;
|
||||||
|
other) printf '%s\\n' 'provider unavailable' >&2; exit 1 ;;
|
||||||
|
*) printf '%s\\n' "$HEYGEN_VIDEO_RESPONSE" ;;
|
||||||
|
esac
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
`,
|
||||||
|
);
|
||||||
|
chmodSync(heygenPath, 0o755);
|
||||||
|
process.env.PATH = `${dir}:${previousEnv.PATH ?? ""}`;
|
||||||
|
process.env.HEYGEN_CAPTURE_PATH = capturePath;
|
||||||
|
process.env.HEYGEN_VIDEO_MODE = options.mode ?? "success";
|
||||||
|
process.env.HEYGEN_VIDEO_RESPONSE = options.response ?? "";
|
||||||
|
process.env.HEYGEN_DISCOVERY_MODE = options.discoveryMode ?? "";
|
||||||
|
process.env.HYPERFRAMES_NO_TELEMETRY = "1";
|
||||||
|
|
||||||
|
try {
|
||||||
|
return await run({
|
||||||
|
invocations: () => readFileSync(capturePath, "utf8").trim().split("\n"),
|
||||||
|
});
|
||||||
|
} finally {
|
||||||
|
for (const [key, value] of Object.entries(previousEnv)) {
|
||||||
|
if (value === undefined) delete process.env[key];
|
||||||
|
else process.env[key] = value;
|
||||||
|
}
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function bodyFromInvocation(invocation) {
|
||||||
|
const marker = " -d ";
|
||||||
|
const start = invocation.indexOf(marker);
|
||||||
|
assert.notEqual(start, -1);
|
||||||
|
return JSON.parse(invocation.slice(start + marker.length));
|
||||||
|
}
|
||||||
|
|
||||||
|
test("downloads a generated avatar video and returns the generated MP4 result", async () => {
|
||||||
|
const { server, url } = await listenVideoServer();
|
||||||
|
let localPath;
|
||||||
|
try {
|
||||||
|
await withFakeHeygen(
|
||||||
|
{ response: JSON.stringify({ data: { video_url: url } }) },
|
||||||
|
async ({ invocations }) => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const intent = "Welcome to the HyperFrames launch";
|
||||||
|
const result = await heygenVideoGenerate(intent, {});
|
||||||
|
localPath = result?.localPath;
|
||||||
|
const calls = invocations();
|
||||||
|
const create = calls.find((call) => call.includes("video create"));
|
||||||
|
|
||||||
|
assert.ok(create);
|
||||||
|
assert.match(create, /--headers X-HeyGen-Client-Source: media-use/);
|
||||||
|
assert.deepEqual(bodyFromInvocation(create), {
|
||||||
|
type: "avatar",
|
||||||
|
avatar_id: "avatar-public-1",
|
||||||
|
script: intent,
|
||||||
|
voice_id: "voice-starfish-1",
|
||||||
|
});
|
||||||
|
assert.ok(result);
|
||||||
|
assert.equal(join(tmpdir(), result.localPath.slice(tmpdir().length + 1)), result.localPath);
|
||||||
|
assert.match(result.localPath, /media-use-heygen-video-\d+-\d+\.mp4$/);
|
||||||
|
assert.deepEqual(result, {
|
||||||
|
localPath: result.localPath,
|
||||||
|
ext: ".mp4",
|
||||||
|
source: "generated",
|
||||||
|
metadata: {
|
||||||
|
description: intent,
|
||||||
|
provider: "heygen.video",
|
||||||
|
provenance: { prompt: intent },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
assert.deepEqual(readFileSync(result.localPath), VIDEO_FIXTURE);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
if (localPath) rmSync(localPath, { force: true });
|
||||||
|
await closeServer(server);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("tags video creation but not avatar or voice discovery", async () => {
|
||||||
|
const { server, url } = await listenVideoServer();
|
||||||
|
let localPath;
|
||||||
|
try {
|
||||||
|
await withFakeHeygen(
|
||||||
|
{ response: JSON.stringify({ data: { video_url: url } }) },
|
||||||
|
async ({ invocations }) => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Header regression guard", {});
|
||||||
|
localPath = result?.localPath;
|
||||||
|
const calls = invocations();
|
||||||
|
|
||||||
|
assert.equal(calls.length, 3);
|
||||||
|
assert.match(calls[0], /^avatar list --ownership public --limit 1$/);
|
||||||
|
assert.doesNotMatch(calls[0], /X-HeyGen-Client-Source/);
|
||||||
|
assert.match(calls[1], /^voice list --engine starfish --limit 1$/);
|
||||||
|
assert.doesNotMatch(calls[1], /X-HeyGen-Client-Source/);
|
||||||
|
assert.match(calls[2], /video create/);
|
||||||
|
assert.match(calls[2], /X-HeyGen-Client-Source: media-use/);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
if (localPath) rmSync(localPath, { force: true });
|
||||||
|
await closeServer(server);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("uses explicit avatar and voice overrides without discovery", async () => {
|
||||||
|
const { server, url } = await listenVideoServer();
|
||||||
|
let localPath;
|
||||||
|
try {
|
||||||
|
await withFakeHeygen(
|
||||||
|
{ response: JSON.stringify({ data: { video_url: url } }) },
|
||||||
|
async ({ invocations }) => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Use my presenter", {
|
||||||
|
avatarId: "avatar-override",
|
||||||
|
voiceId: "voice-override",
|
||||||
|
});
|
||||||
|
localPath = result?.localPath;
|
||||||
|
const calls = invocations();
|
||||||
|
|
||||||
|
assert.equal(calls.length, 1);
|
||||||
|
assert.match(calls[0], /video create/);
|
||||||
|
assert.deepEqual(bodyFromInvocation(calls[0]), {
|
||||||
|
type: "avatar",
|
||||||
|
avatar_id: "avatar-override",
|
||||||
|
script: "Use my presenter",
|
||||||
|
voice_id: "voice-override",
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
if (localPath) rmSync(localPath, { force: true });
|
||||||
|
await closeServer(server);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("caches discovered avatar and voice IDs for the process", async () => {
|
||||||
|
const { server, url } = await listenVideoServer();
|
||||||
|
const localPaths = new Set();
|
||||||
|
try {
|
||||||
|
await withFakeHeygen(
|
||||||
|
{ response: JSON.stringify({ data: { video_url: url } }) },
|
||||||
|
async ({ invocations }) => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
for (const intent of ["First avatar video", "Second avatar video"]) {
|
||||||
|
const result = await heygenVideoGenerate(intent, {});
|
||||||
|
if (result) localPaths.add(result.localPath);
|
||||||
|
}
|
||||||
|
const calls = invocations();
|
||||||
|
|
||||||
|
assert.equal(calls.filter((call) => call.startsWith("avatar list ")).length, 1);
|
||||||
|
assert.equal(calls.filter((call) => call.startsWith("voice list ")).length, 1);
|
||||||
|
assert.equal(calls.filter((call) => call.includes("video create")).length, 2);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
for (const localPath of localPaths) rmSync(localPath, { force: true });
|
||||||
|
await closeServer(server);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("prints auth onboarding and reports an unauthenticated create failure", async (t) => {
|
||||||
|
const errors = [];
|
||||||
|
t.mock.method(console, "error", (message) => errors.push(message));
|
||||||
|
|
||||||
|
await withFakeHeygen({ mode: "auth" }, async () => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Sign-in failure", {
|
||||||
|
avatarId: "avatar-override",
|
||||||
|
voiceId: "voice-override",
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.ok(errors.includes(AVATAR_VIDEO_SIGNIN_MESSAGE));
|
||||||
|
assert.ok(errors.includes(HEYGEN_NOT_AUTHENTICATED_MESSAGE));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("reports other create failures without auth onboarding", async (t) => {
|
||||||
|
const errors = [];
|
||||||
|
t.mock.method(console, "error", (message) => errors.push(message));
|
||||||
|
|
||||||
|
await withFakeHeygen({ mode: "other" }, async () => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Provider failure", {
|
||||||
|
avatarId: "avatar-override",
|
||||||
|
voiceId: "voice-override",
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.ok(!errors.includes(AVATAR_VIDEO_SIGNIN_MESSAGE));
|
||||||
|
assert.ok(errors.includes("media-use: `heygen video create` failed: provider unavailable"));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("falls through on non-JSON and error responses", async (t) => {
|
||||||
|
t.mock.method(console, "error", () => {});
|
||||||
|
|
||||||
|
for (const response of ["not JSON", '{"error":{"message":"render failed"}}']) {
|
||||||
|
await withFakeHeygen({ response }, async () => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Unusable response", {
|
||||||
|
avatarId: "avatar-override",
|
||||||
|
voiceId: "voice-override",
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("onboards and returns null when avatar/voice discovery itself is unauthenticated", async (t) => {
|
||||||
|
const errors = [];
|
||||||
|
t.mock.method(console, "error", (message) => errors.push(message));
|
||||||
|
|
||||||
|
// Both avatar list AND voice list would fail unauthenticated (discoveryMode
|
||||||
|
// "auth" applies to both in the fake CLI) -- the short-circuit after the
|
||||||
|
// first failure must mean only one is ever attempted, so the onboarding
|
||||||
|
// message and the provider-error telemetry ping each fire exactly once
|
||||||
|
// instead of double-firing for what's really one auth failure.
|
||||||
|
await withFakeHeygen({ discoveryMode: "auth" }, async ({ invocations }) => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Discovery auth failure", {});
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.equal(
|
||||||
|
errors.filter((message) => message === AVATAR_VIDEO_SIGNIN_MESSAGE).length,
|
||||||
|
1,
|
||||||
|
"onboarding message must fire exactly once, not once per failed discovery call",
|
||||||
|
);
|
||||||
|
const calls = invocations();
|
||||||
|
assert.equal(calls.length, 1, "must short-circuit after the first discovery failure");
|
||||||
|
assert.match(calls[0], /^avatar list /);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("download failure after a successful create returns null and logs a diagnostic", async () => {
|
||||||
|
const { server, url } = await listenFailingVideoServer();
|
||||||
|
try {
|
||||||
|
await withFakeHeygen({ response: JSON.stringify({ data: { video_url: url } }) }, async () => {
|
||||||
|
const heygenVideoGenerate = await freshGenerate();
|
||||||
|
const result = await heygenVideoGenerate("Download failure", {
|
||||||
|
avatarId: "avatar-override",
|
||||||
|
voiceId: "voice-override",
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
});
|
||||||
|
} finally {
|
||||||
|
await closeServer(server);
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -205,6 +205,17 @@ export function listModels(capability) {
|
|||||||
return tableFor(capability).slice();
|
return tableFor(capability).slice();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Tokenize an `invoke` template on whitespace first, then substitute each
|
||||||
|
// token, so a `{prompt}`/`{model_path}` value with spaces stays a single argv
|
||||||
|
// entry. Shared by every local-model provider (mflux, LTX) that builds argv
|
||||||
|
// from a MODELS[...].invoke template.
|
||||||
|
export function buildArgv(template, vars) {
|
||||||
|
return template
|
||||||
|
.trim()
|
||||||
|
.split(/\s+/)
|
||||||
|
.map((tok) => tok.replace(/\{(\w+)\}/g, (_, k) => (k in vars ? String(vars[k]) : `{${k}}`)));
|
||||||
|
}
|
||||||
|
|
||||||
/** Does this machine meet a model's needs? Apple Silicon unified memory counts as VRAM. */
|
/** Does this machine meet a model's needs? Apple Silicon unified memory counts as VRAM. */
|
||||||
export function meetsSpecs(model, specs) {
|
export function meetsSpecs(model, specs) {
|
||||||
const n = model.needs || {};
|
const n = model.needs || {};
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
import { execFileSync } from "node:child_process";
|
||||||
|
import { existsSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { probeSpecs } from "./specs.mjs";
|
||||||
|
import { buildArgv, selectModel } from "./local-models.mjs";
|
||||||
|
|
||||||
|
export async function ltxVideoGenerate(
|
||||||
|
intent,
|
||||||
|
ctx,
|
||||||
|
execFn = execFileSync,
|
||||||
|
pathExists = existsSync,
|
||||||
|
) {
|
||||||
|
const specs = ctx?.specs || probeSpecs();
|
||||||
|
const sel = selectModel("videogen", specs, { preferTier: ctx?.preferTier });
|
||||||
|
if (sel.recommend) {
|
||||||
|
console.error(
|
||||||
|
`media-use: local video gen not enabled (${sel.reason}). Enable a fitting free on-device LTX model to use this provider.`,
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const { model } = sel;
|
||||||
|
const bin = model.invoke.trim().split(/\s+/)[0];
|
||||||
|
try {
|
||||||
|
execFn("which", [bin], { stdio: ["ignore", "ignore", "ignore"] });
|
||||||
|
} catch {
|
||||||
|
console.error(
|
||||||
|
`media-use: local video gen not enabled (\`${bin}\` not on PATH). Install for free on-device LTX: ${model.install}`,
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const outPath = join(tmpdir(), `media-use-ltx-${process.pid}-${Date.now()}.mp4`);
|
||||||
|
const width = ctx?.width || 512;
|
||||||
|
const height = ctx?.height || 320;
|
||||||
|
const frames = ctx?.frames || 33;
|
||||||
|
const argv = buildArgv(model.invoke, {
|
||||||
|
prompt: intent,
|
||||||
|
w: width,
|
||||||
|
h: height,
|
||||||
|
frames,
|
||||||
|
out: outPath,
|
||||||
|
});
|
||||||
|
argv.shift();
|
||||||
|
|
||||||
|
try {
|
||||||
|
execFn(bin, argv, {
|
||||||
|
encoding: "utf8",
|
||||||
|
timeout: 1_800_000,
|
||||||
|
stdio: ["ignore", "pipe", "pipe"],
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
console.error(
|
||||||
|
`media-use: local video gen (${model.id}) failed: ${err.stderr?.toString().trim().slice(-200) || err.message}`,
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (!pathExists(outPath)) return null;
|
||||||
|
return {
|
||||||
|
localPath: outPath,
|
||||||
|
ext: ".mp4",
|
||||||
|
source: "generated",
|
||||||
|
metadata: {
|
||||||
|
description: intent,
|
||||||
|
provider: "ltx.local",
|
||||||
|
provenance: { prompt: intent },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
import { test } from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { dirname } from "node:path";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { ltxVideoGenerate } from "./ltx-video-provider.mjs";
|
||||||
|
|
||||||
|
const fittingSpecs = { availableRamMB: 20000, gpu: { present: true } };
|
||||||
|
|
||||||
|
test("no fitting local model: falls through without checking for a binary", async (t) => {
|
||||||
|
t.mock.method(console, "error", () => {});
|
||||||
|
const calls = [];
|
||||||
|
|
||||||
|
const result = await ltxVideoGenerate(
|
||||||
|
"a calm ocean wave at sunset",
|
||||||
|
{ specs: { availableRamMB: 100, gpu: { present: true } } },
|
||||||
|
(...call) => calls.push(call),
|
||||||
|
() => true,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.deepEqual(calls, []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("binary missing from PATH: prints the model install hint and falls through", async (t) => {
|
||||||
|
const errors = [];
|
||||||
|
t.mock.method(console, "error", (message) => errors.push(message));
|
||||||
|
const calls = [];
|
||||||
|
const fakeExec = (...call) => {
|
||||||
|
calls.push(call);
|
||||||
|
throw new Error("not found");
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = await ltxVideoGenerate(
|
||||||
|
"a calm ocean wave at sunset",
|
||||||
|
{ specs: fittingSpecs },
|
||||||
|
fakeExec,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.equal(calls.length, 1);
|
||||||
|
assert.deepEqual(calls[0].slice(0, 2), ["which", ["ltx-2-mlx"]]);
|
||||||
|
assert.equal(errors.length, 1);
|
||||||
|
assert.match(errors[0], /git clone https:\/\/github\.com\/dgrauet\/ltx-2-mlx/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("generate argv substitutes a spaced prompt after tokenizing and uses verified defaults", async () => {
|
||||||
|
const calls = [];
|
||||||
|
const checkedPaths = [];
|
||||||
|
const fakeExec = (...call) => calls.push(call);
|
||||||
|
const pathExists = (path) => {
|
||||||
|
checkedPaths.push(path);
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
const intent = "a calm ocean wave at sunset";
|
||||||
|
|
||||||
|
const result = await ltxVideoGenerate(intent, { specs: fittingSpecs }, fakeExec, pathExists);
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.equal(calls.length, 2);
|
||||||
|
const [bin, argv, opts] = calls[1];
|
||||||
|
assert.equal(bin, "ltx-2-mlx");
|
||||||
|
assert.equal(opts.timeout, 1_800_000);
|
||||||
|
|
||||||
|
const expectedPairs = [
|
||||||
|
["--prompt", intent],
|
||||||
|
["--width", "512"],
|
||||||
|
["--height", "320"],
|
||||||
|
["--frames", "33"],
|
||||||
|
["--output", checkedPaths[0]],
|
||||||
|
];
|
||||||
|
let previousIndex = -1;
|
||||||
|
for (const [flag, value] of expectedPairs) {
|
||||||
|
const index = argv.indexOf(flag);
|
||||||
|
assert.ok(index > previousIndex, `${flag} should follow the previous required option`);
|
||||||
|
assert.equal(argv[index + 1], value);
|
||||||
|
previousIndex = index;
|
||||||
|
}
|
||||||
|
assert.equal(argv.filter((arg) => arg === intent).length, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("successful generation returns the generated MP4 result", async () => {
|
||||||
|
const calls = [];
|
||||||
|
const fakeExec = (...call) => calls.push(call);
|
||||||
|
const intent = "a calm ocean wave at sunset";
|
||||||
|
|
||||||
|
const result = await ltxVideoGenerate(intent, { specs: fittingSpecs }, fakeExec, () => true);
|
||||||
|
|
||||||
|
assert.ok(result);
|
||||||
|
assert.equal(calls.length, 2);
|
||||||
|
assert.equal(dirname(result.localPath), tmpdir());
|
||||||
|
assert.match(result.localPath, /media-use-ltx-\d+-\d+\.mp4$/);
|
||||||
|
assert.deepEqual(result, {
|
||||||
|
localPath: result.localPath,
|
||||||
|
ext: ".mp4",
|
||||||
|
source: "generated",
|
||||||
|
metadata: {
|
||||||
|
description: intent,
|
||||||
|
provider: "ltx.local",
|
||||||
|
provenance: { prompt: intent },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("generate failure returns null instead of throwing", async (t) => {
|
||||||
|
t.mock.method(console, "error", () => {});
|
||||||
|
let calls = 0;
|
||||||
|
const fakeExec = () => {
|
||||||
|
calls += 1;
|
||||||
|
if (calls === 2) {
|
||||||
|
const error = new Error("generation failed");
|
||||||
|
error.stderr = "LTX failed";
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = await ltxVideoGenerate(
|
||||||
|
"storm clouds",
|
||||||
|
{ specs: fittingSpecs },
|
||||||
|
fakeExec,
|
||||||
|
() => true,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
assert.equal(calls, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("missing generated output returns null", async () => {
|
||||||
|
const result = await ltxVideoGenerate(
|
||||||
|
"storm clouds",
|
||||||
|
{ specs: fittingSpecs },
|
||||||
|
() => {},
|
||||||
|
() => false,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert.equal(result, null);
|
||||||
|
});
|
||||||
@@ -3,7 +3,7 @@ import { existsSync } from "node:fs";
|
|||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { probeSpecs } from "./specs.mjs";
|
import { probeSpecs } from "./specs.mjs";
|
||||||
import { selectModel } from "./local-models.mjs";
|
import { buildArgv, selectModel } from "./local-models.mjs";
|
||||||
|
|
||||||
// Local image generation via mflux (FLUX-on-MLX), the Mac-native runner.
|
// Local image generation via mflux (FLUX-on-MLX), the Mac-native runner.
|
||||||
// Spec-gated: selectModel("imagegen", specs) returns the best FLUX-class model
|
// Spec-gated: selectModel("imagegen", specs) returns the best FLUX-class model
|
||||||
@@ -15,15 +15,6 @@ import { selectModel } from "./local-models.mjs";
|
|||||||
// non-gated community 4-bit re-uploads; the repo is resolved to a local snapshot
|
// non-gated community 4-bit re-uploads; the repo is resolved to a local snapshot
|
||||||
// (hf download, idempotent) because a bare repo id breaks mlx unflatten.
|
// (hf download, idempotent) because a bare repo id breaks mlx unflatten.
|
||||||
|
|
||||||
// Tokenize the invoke template on whitespace FIRST, then substitute each token,
|
|
||||||
// so a {prompt}/{model_path} value with spaces stays a single argv entry.
|
|
||||||
function buildArgv(template, vars) {
|
|
||||||
return template
|
|
||||||
.trim()
|
|
||||||
.split(/\s+/)
|
|
||||||
.map((tok) => tok.replace(/\{(\w+)\}/g, (_, k) => (k in vars ? String(vars[k]) : `{${k}}`)));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Resolve an HF repo to its local snapshot dir. `hf download` is idempotent and
|
// Resolve an HF repo to its local snapshot dir. `hf download` is idempotent and
|
||||||
// prints the snapshot path as its last line.
|
// prints the snapshot path as its last line.
|
||||||
function resolveSnapshot(repo) {
|
function resolveSnapshot(repo) {
|
||||||
|
|||||||
@@ -32,6 +32,8 @@ import {
|
|||||||
faviconSearch,
|
faviconSearch,
|
||||||
} from "./logo-provider.mjs";
|
} from "./logo-provider.mjs";
|
||||||
import { heygenTtsGenerate } from "./voice-provider.mjs";
|
import { heygenTtsGenerate } from "./voice-provider.mjs";
|
||||||
|
import { heygenVideoGenerate } from "./heygen-video-provider.mjs";
|
||||||
|
import { ltxVideoGenerate } from "./ltx-video-provider.mjs";
|
||||||
import { localTtsGenerate } from "./tts-local-provider.mjs";
|
import { localTtsGenerate } from "./tts-local-provider.mjs";
|
||||||
import { codexImageGenerate } from "./codex-provider.mjs";
|
import { codexImageGenerate } from "./codex-provider.mjs";
|
||||||
import { mfluxImageGenerate } from "./mflux-provider.mjs";
|
import { mfluxImageGenerate } from "./mflux-provider.mjs";
|
||||||
@@ -83,6 +85,12 @@ const REGISTRY = {
|
|||||||
P("heygen.tts", { generate: heygenTtsGenerate }),
|
P("heygen.tts", { generate: heygenTtsGenerate }),
|
||||||
A("kokoro.local", { generate: localTtsGenerate }),
|
A("kokoro.local", { generate: localTtsGenerate }),
|
||||||
],
|
],
|
||||||
|
video: [
|
||||||
|
// HeyGen avatar video first when credentialed; --local-only skips it and
|
||||||
|
// keeps LTX as the local fallback.
|
||||||
|
P("heygen.video", { generate: heygenVideoGenerate }),
|
||||||
|
A("ltx.local", { generate: ltxVideoGenerate }),
|
||||||
|
],
|
||||||
brand: [
|
brand: [
|
||||||
// Local design spec, not heygen — reads frame.md / design.md tokens.
|
// Local design spec, not heygen — reads frame.md / design.md tokens.
|
||||||
A("design_spec", { search: brandProvider.search }),
|
A("design_spec", { search: brandProvider.search }),
|
||||||
|
|||||||
@@ -1,12 +1,31 @@
|
|||||||
import { strict as assert } from "node:assert";
|
import { strict as assert } from "node:assert";
|
||||||
import { test } from "node:test";
|
import { test } from "node:test";
|
||||||
import { getProviders, getProvider, listTypes, runProviders, runCapability } from "./registry.mjs";
|
import {
|
||||||
|
getProviders,
|
||||||
|
getProvider,
|
||||||
|
listTypes,
|
||||||
|
providerMatches,
|
||||||
|
providerNamesFor,
|
||||||
|
runProviders,
|
||||||
|
runCapability,
|
||||||
|
} from "./registry.mjs";
|
||||||
|
|
||||||
// --- registry shape -------------------------------------------------------
|
// --- registry shape -------------------------------------------------------
|
||||||
|
|
||||||
test("listTypes exposes the v2 media types", () => {
|
test("listTypes exposes the v2 media types", () => {
|
||||||
const types = listTypes();
|
const types = listTypes();
|
||||||
for (const t of ["bgm", "sfx", "image", "icon", "logo", "voice", "brand", "grade", "lut"]) {
|
for (const t of [
|
||||||
|
"bgm",
|
||||||
|
"sfx",
|
||||||
|
"image",
|
||||||
|
"icon",
|
||||||
|
"logo",
|
||||||
|
"voice",
|
||||||
|
"video",
|
||||||
|
"brand",
|
||||||
|
"grade",
|
||||||
|
"lut",
|
||||||
|
]) {
|
||||||
assert.ok(types.includes(t), `missing type: ${t}`);
|
assert.ok(types.includes(t), `missing type: ${t}`);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -19,9 +38,9 @@ test("heygen provider is first for every type it serves", () => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
test("sanctioned providers only: heygen, local mflux/kokoro, codex, design spec, logo tiers", () => {
|
test("sanctioned providers only: heygen, local mflux/kokoro/ltx, codex, design spec, logo tiers", () => {
|
||||||
const allowed =
|
const allowed =
|
||||||
/^heygen|^bundled\.sfx$|^mflux\.local$|^kokoro\.local$|^codex\.image_gen$|^design_spec$|^svgl$|^simple-icons$|^github\.avatar$|^favicon\.ddg$|^color_grade\.local$|^cube_lut\.local$/;
|
/^heygen|^bundled\.sfx$|^mflux\.local$|^kokoro\.local$|^ltx\.local$|^codex\.image_gen$|^design_spec$|^svgl$|^simple-icons$|^github\.avatar$|^favicon\.ddg$|^color_grade\.local$|^cube_lut\.local$/;
|
||||||
for (const t of listTypes()) {
|
for (const t of listTypes()) {
|
||||||
for (const p of getProviders(t)) {
|
for (const p of getProviders(t)) {
|
||||||
assert.ok(allowed.test(p.name), `${t} lists unsanctioned provider: ${p.name}`);
|
assert.ok(allowed.test(p.name), `${t} lists unsanctioned provider: ${p.name}`);
|
||||||
@@ -52,6 +71,17 @@ test("voice cascade: HeyGen TTS first, Kokoro remains the local fallback", () =>
|
|||||||
assert.ok(!ps[1].paid, "local Kokoro is free");
|
assert.ok(!ps[1].paid, "local Kokoro is free");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("video cascade: HeyGen first, LTX local fallback, generate-only", async () => {
|
||||||
|
assert.deepEqual(providerNamesFor("video"), ["heygen.video", "ltx.local"]);
|
||||||
|
assert.equal(providerMatches("video", "ltx.local"), true);
|
||||||
|
|
||||||
|
const ps = getProviders("video");
|
||||||
|
assert.ok(ps[0].network, "HeyGen video is network (skipped under --local-only)");
|
||||||
|
assert.ok(ps[0].paid, "HeyGen video may bill after the OAuth free allowance");
|
||||||
|
assert.ok(!ps[1].network, "local LTX is kept under --local-only");
|
||||||
|
assert.equal(await runCapability("video", "search", "x", {}), null);
|
||||||
|
});
|
||||||
|
|
||||||
test("sfx cascade: HeyGen catalog first, bundled library remains the local fallback", () => {
|
test("sfx cascade: HeyGen catalog first, bundled library remains the local fallback", () => {
|
||||||
const ps = getProviders("sfx");
|
const ps = getProviders("sfx");
|
||||||
assert.equal(ps[0].name, "heygen.audio.sounds");
|
assert.equal(ps[0].name, "heygen.audio.sounds");
|
||||||
|
|||||||
@@ -1,29 +1,9 @@
|
|||||||
import { execFileSync } from "node:child_process";
|
import { HEYGEN_CLIENT_SOURCE_ARGV, runHeygenJson } from "./heygen-cli.mjs";
|
||||||
import { reportHeygenFailure } from "./heygen-cli.mjs";
|
|
||||||
|
|
||||||
// Voice / TTS generation via the HeyGen CLI — the only external CLI media-use
|
// Voice / TTS generation via the HeyGen CLI — the only external CLI media-use
|
||||||
// shells (CLI-only invariant: media-use holds no keys; the CLI owns auth).
|
// shells (CLI-only invariant: media-use holds no keys; the CLI owns auth).
|
||||||
// Flags verified against `heygen voice speech create --help` (v0.3.0).
|
// Flags verified against `heygen voice speech create --help` (v0.3.0).
|
||||||
|
|
||||||
function runJson(bin, argv, label) {
|
|
||||||
let out;
|
|
||||||
try {
|
|
||||||
out = execFileSync(bin, argv, {
|
|
||||||
encoding: "utf8",
|
|
||||||
timeout: 120000,
|
|
||||||
stdio: ["pipe", "pipe", "pipe"],
|
|
||||||
});
|
|
||||||
} catch (err) {
|
|
||||||
reportHeygenFailure(err, `${bin} ${label}`);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
return JSON.parse(out);
|
|
||||||
} catch {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function result(url, duration, provider, intent) {
|
function result(url, duration, provider, intent) {
|
||||||
if (!url) return null;
|
if (!url) return null;
|
||||||
return {
|
return {
|
||||||
@@ -41,11 +21,13 @@ function result(url, duration, provider, intent) {
|
|||||||
// HeyGen TTS requires a starfish-engine voice. Default to the first one the
|
// HeyGen TTS requires a starfish-engine voice. Default to the first one the
|
||||||
// catalog returns (deterministic order); pass ctx.voiceId to override.
|
// catalog returns (deterministic order); pass ctx.voiceId to override.
|
||||||
// ponytail: listed once per process; the resolved asset is frozen + cached after
|
// ponytail: listed once per process; the resolved asset is frozen + cached after
|
||||||
// first use, so the network list only happens on a cache miss.
|
// first use, so the network list only happens on a cache miss. Cache only a
|
||||||
|
// truthy id -- a transient list failure must not poison the cache with `null`
|
||||||
|
// and permanently disable TTS for the rest of the process.
|
||||||
let cachedVoiceId;
|
let cachedVoiceId;
|
||||||
function defaultVoiceId() {
|
function defaultVoiceId() {
|
||||||
if (cachedVoiceId !== undefined) return cachedVoiceId;
|
if (cachedVoiceId) return cachedVoiceId;
|
||||||
const j = runJson(
|
const j = runHeygenJson(
|
||||||
"heygen",
|
"heygen",
|
||||||
["voice", "list", "--engine", "starfish", "--limit", "1"],
|
["voice", "list", "--engine", "starfish", "--limit", "1"],
|
||||||
"voice list",
|
"voice list",
|
||||||
@@ -57,9 +39,18 @@ function defaultVoiceId() {
|
|||||||
export async function heygenTtsGenerate(intent, ctx) {
|
export async function heygenTtsGenerate(intent, ctx) {
|
||||||
const voiceId = ctx?.voiceId || defaultVoiceId();
|
const voiceId = ctx?.voiceId || defaultVoiceId();
|
||||||
if (!voiceId) return null;
|
if (!voiceId) return null;
|
||||||
const p = runJson(
|
const p = runHeygenJson(
|
||||||
"heygen",
|
"heygen",
|
||||||
["voice", "speech", "create", "--text", intent, "--voice-id", voiceId],
|
[
|
||||||
|
...HEYGEN_CLIENT_SOURCE_ARGV,
|
||||||
|
"voice",
|
||||||
|
"speech",
|
||||||
|
"create",
|
||||||
|
"--text",
|
||||||
|
intent,
|
||||||
|
"--voice-id",
|
||||||
|
voiceId,
|
||||||
|
],
|
||||||
"tts",
|
"tts",
|
||||||
);
|
);
|
||||||
return result(p?.data?.audio_url, p?.data?.duration, "heygen.tts", intent);
|
return result(p?.data?.audio_url, p?.data?.duration, "heygen.tts", intent);
|
||||||
|
|||||||
@@ -0,0 +1,46 @@
|
|||||||
|
import { strict as assert } from "node:assert";
|
||||||
|
import { chmodSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { test } from "node:test";
|
||||||
|
import { heygenTtsGenerate } from "./voice-provider.mjs";
|
||||||
|
|
||||||
|
test("tags TTS generation but not voice discovery with the media-use client source", async () => {
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "media-use-voice-provider-"));
|
||||||
|
const capturePath = join(dir, "argv.log");
|
||||||
|
const heygenPath = join(dir, "heygen");
|
||||||
|
const previousPath = process.env.PATH;
|
||||||
|
const previousCapturePath = process.env.HEYGEN_CAPTURE_PATH;
|
||||||
|
|
||||||
|
writeFileSync(
|
||||||
|
heygenPath,
|
||||||
|
`#!/bin/sh
|
||||||
|
printf '%s\\n' "$*" >> "$HEYGEN_CAPTURE_PATH"
|
||||||
|
case "$*" in
|
||||||
|
*"voice list"*) printf '%s\\n' '{"data":[{"voice_id":"voice-123"}]}' ;;
|
||||||
|
*"voice speech create"*) printf '%s\\n' '{"data":{"audio_url":"https://example.com/voice.mp3","duration":1.5}}' ;;
|
||||||
|
esac
|
||||||
|
`,
|
||||||
|
);
|
||||||
|
chmodSync(heygenPath, 0o755);
|
||||||
|
process.env.PATH = `${dir}:${previousPath ?? ""}`;
|
||||||
|
process.env.HEYGEN_CAPTURE_PATH = capturePath;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await heygenTtsGenerate("Hello from media-use", {});
|
||||||
|
const invocations = readFileSync(capturePath, "utf8").trim().split("\n");
|
||||||
|
|
||||||
|
assert.equal(invocations.length, 2);
|
||||||
|
assert.match(invocations[0], /^voice list /);
|
||||||
|
assert.doesNotMatch(invocations[0], /X-HeyGen-Client-Source/);
|
||||||
|
assert.match(invocations[1], /voice speech create/);
|
||||||
|
assert.match(invocations[1], /X-HeyGen-Client-Source: media-use/);
|
||||||
|
assert.equal(result?.url, "https://example.com/voice.mp3");
|
||||||
|
} finally {
|
||||||
|
if (previousPath === undefined) delete process.env.PATH;
|
||||||
|
else process.env.PATH = previousPath;
|
||||||
|
if (previousCapturePath === undefined) delete process.env.HEYGEN_CAPTURE_PATH;
|
||||||
|
else process.env.HEYGEN_CAPTURE_PATH = previousCapturePath;
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -37,7 +37,19 @@ import {
|
|||||||
} from "./lib/heygen-cli.mjs";
|
} from "./lib/heygen-cli.mjs";
|
||||||
import { BundledSfxAssetsError, inspectBundledSfxAssets } from "./lib/bundled-sfx-provider.mjs";
|
import { BundledSfxAssetsError, inspectBundledSfxAssets } from "./lib/bundled-sfx-provider.mjs";
|
||||||
|
|
||||||
const INGEST_TYPES = [...listTypes(), "video"];
|
const INGEST_TYPES = listTypes();
|
||||||
|
const DEFAULT_EXT = {
|
||||||
|
bgm: ".wav",
|
||||||
|
sfx: ".mp3",
|
||||||
|
voice: ".wav",
|
||||||
|
image: ".jpg",
|
||||||
|
icon: ".svg",
|
||||||
|
logo: ".svg",
|
||||||
|
brand: ".png",
|
||||||
|
video: ".mp4",
|
||||||
|
grade: ".cube",
|
||||||
|
lut: ".cube",
|
||||||
|
};
|
||||||
|
|
||||||
// resolve shells `fetch`/`freezeUrl` and modern ESM; 18 is the floor where those
|
// resolve shells `fetch`/`freezeUrl` and modern ESM; 18 is the floor where those
|
||||||
// exist without flags. Named so the --doctor node check verifies something real
|
// exist without flags. Named so the --doctor node check verifies something real
|
||||||
@@ -62,6 +74,8 @@ const { values: args } = parseArgs({
|
|||||||
for: { type: "string" },
|
for: { type: "string" },
|
||||||
"local-only": { type: "boolean", default: false },
|
"local-only": { type: "boolean", default: false },
|
||||||
provider: { type: "string" },
|
provider: { type: "string" },
|
||||||
|
"avatar-id": { type: "string" },
|
||||||
|
"voice-id": { type: "string" },
|
||||||
json: { type: "boolean", default: false },
|
json: { type: "boolean", default: false },
|
||||||
help: { type: "boolean", short: "h", default: false },
|
help: { type: "boolean", short: "h", default: false },
|
||||||
},
|
},
|
||||||
@@ -96,6 +110,8 @@ Options:
|
|||||||
suggestions (grade only)
|
suggestions (grade only)
|
||||||
--local-only Offline: skip every network provider
|
--local-only Offline: skip every network provider
|
||||||
--provider Force one generator (e.g. codex, mflux, kokoro, heygen)
|
--provider Force one generator (e.g. codex, mflux, kokoro, heygen)
|
||||||
|
--avatar-id Override the default avatar for heygen.video generation
|
||||||
|
--voice-id Override the default voice for voice/heygen.video generation
|
||||||
--json Output JSON instead of one-line result
|
--json Output JSON instead of one-line result
|
||||||
--help, -h Show this help`);
|
--help, -h Show this help`);
|
||||||
process.exit(0);
|
process.exit(0);
|
||||||
@@ -342,7 +358,14 @@ async function run() {
|
|||||||
// Offline guard: --local-only skips every remote provider (HeyGen catalog),
|
// Offline guard: --local-only skips every remote provider (HeyGen catalog),
|
||||||
// leaving the project + global cache and any local provider.
|
// leaving the project + global cache and any local provider.
|
||||||
const localOnly = args["local-only"];
|
const localOnly = args["local-only"];
|
||||||
const ctx = { entity, projectDir, localOnly, provider: args.provider };
|
const ctx = {
|
||||||
|
entity,
|
||||||
|
projectDir,
|
||||||
|
localOnly,
|
||||||
|
provider: args.provider,
|
||||||
|
avatarId: args["avatar-id"],
|
||||||
|
voiceId: args["voice-id"],
|
||||||
|
};
|
||||||
|
|
||||||
// Adherence nudge (offline, no auto-reuse): the exact-cache floor missed and
|
// Adherence nudge (offline, no auto-reuse): the exact-cache floor missed and
|
||||||
// we're about to fetch/generate. If lexically-similar assets already exist,
|
// we're about to fetch/generate. If lexically-similar assets already exist,
|
||||||
@@ -1187,19 +1210,6 @@ function extFromUrl(url) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const DEFAULT_EXT = {
|
|
||||||
bgm: ".wav",
|
|
||||||
sfx: ".mp3",
|
|
||||||
voice: ".wav",
|
|
||||||
image: ".jpg",
|
|
||||||
icon: ".svg",
|
|
||||||
logo: ".svg",
|
|
||||||
brand: ".png",
|
|
||||||
video: ".mp4",
|
|
||||||
grade: ".cube",
|
|
||||||
lut: ".cube",
|
|
||||||
};
|
|
||||||
|
|
||||||
function defaultExt(type) {
|
function defaultExt(type) {
|
||||||
return DEFAULT_EXT[type] || ".bin";
|
return DEFAULT_EXT[type] || ".bin";
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -492,6 +492,46 @@ test("--from registers a derived video as documented", () => {
|
|||||||
cleanup();
|
cleanup();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("--from type error lists video exactly once", () => {
|
||||||
|
const result = spawnResolve(["--from", "missing.mp4"]);
|
||||||
|
assert.equal(result.status, 2);
|
||||||
|
assert.match(result.stderr, /--from requires --type \(one of:/);
|
||||||
|
assert.equal(result.stderr.match(/\bvideo\b/g)?.length, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("--from uses .mp4 as the default video extension", () => {
|
||||||
|
setup();
|
||||||
|
const source = join(tmp, "extensionless-video");
|
||||||
|
writeFileSync(source, "video bytes");
|
||||||
|
|
||||||
|
const out = runResolve(["--from", source, "--type", "video", "--project", tmp, "--json"]);
|
||||||
|
const parsed = JSON.parse(out.trim());
|
||||||
|
assert.match(parsed.path, /^\.media\/video\/video_001\.mp4$/);
|
||||||
|
cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("--avatar-id/--voice-id parse as real CLI flags (regression guard: docs promise them, parseArgs must not reject them)", () => {
|
||||||
|
setup();
|
||||||
|
const result = spawnResolve(
|
||||||
|
[
|
||||||
|
"--type",
|
||||||
|
"video",
|
||||||
|
"--intent",
|
||||||
|
"regression guard",
|
||||||
|
"--local-only",
|
||||||
|
"--avatar-id",
|
||||||
|
"avatar-override",
|
||||||
|
"--voice-id",
|
||||||
|
"voice-override",
|
||||||
|
"--project",
|
||||||
|
tmp,
|
||||||
|
],
|
||||||
|
{ stdio: "pipe" },
|
||||||
|
);
|
||||||
|
assert.doesNotMatch(result.stderr || "", /ERR_PARSE_ARGS_UNKNOWN_OPTION/);
|
||||||
|
cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
test("unknown type error lists grade and lut", () => {
|
test("unknown type error lists grade and lut", () => {
|
||||||
try {
|
try {
|
||||||
runResolve(["--type", "bogus", "--intent", "x"], { stdio: "pipe" });
|
runResolve(["--type", "bogus", "--intent", "x"], { stdio: "pipe" });
|
||||||
|
|||||||
Reference in New Issue
Block a user