Commit Graph
28 Commits
Author SHA1 Message Date
DanteandJames ea7a4e433f fix(media): read ElevenLabs input as UTF-8 (#5469)
* fix(media): read ElevenLabs input as UTF-8

* fix(media): regenerate skill manifest after UTF-8 correction

---------

Co-authored-by: James <james.russo@heygen.com>
2026-10-12 22:56:24 +01:00
路从今夜白 73d89b81c7 fix(media-use): clean up temporary TTS text directories (#5477)
* fix(media-use): clean up temporary TTS text files

* chore(skills): refresh manifest for TTS cleanup
2026-10-12 21:08:50 +00:00
Miguel Ángel f46033d048 fix(skills): skill scripts run through a symlinked skills dir and agents keep long renders alive (#5396)
* fix(skills): skill scripts run when the skills dir is a symlink

* test(skills): plugin launcher fixture ships the shared main-module helper like an install

* fix(skills): the CLI's cube validator runs through a symlink too, and the guard check is stricter

* docs(skills): long agent-started renders run detached in their own process group

* docs(cli): detached renders start in their own process group, on macOS too

* docs(cli): the detached render recipe never waits on terminal input
2026-10-10 04:54:41 +00:00
Wenbo ZhuandClaude Opus 5.5 21d14b25b3 feat(media-use): reach HeyGen through a host app's gateway (HEYGEN_API_BASE) (#5181)
* feat(media-use): reach HeyGen through a host app's gateway (HEYGEN_API_BASE)

- The audio engine sends every HeyGen call to $HEYGEN_API_BASE when set, as the heygen CLI does; plain HTTP needs
  $HEYGEN_ALLOW_HTTP=1, the CLI's own rule. A host gateway (that base with its own key) wins over a host OAuth token.
- The skill says: inside a host that gives HeyGen access, no CLI install or sign-in, prefer the host's own HeyGen
  tools, relay a refused call's message as written and do not switch providers on your own.

HyperFrames Desktop sets these variables when a HeyGen API key is saved in its Settings, so media-use's TTS, catalog
search and avatar calls are paid by that key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(media-use): format the edited tables and regenerate the skills manifest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): host access applies only when HEYGEN_API_BASE is set; HeyGen credentials stay on HeyGen hosts

- The "use the host's own tools first / no CLI sign-in" guidance is now conditional on HEYGEN_API_BASE alone, so an
  agent outside such a host (Claude Code with a HeyGen MCP connector, say) keeps today's media-use journey. The
  compressed copies in README, CLAUDE.md, the prompting overview, general-video and the capability menu are back to
  main's wording.
- A base outside heygen.com gets no stored or host OAuth credential, only a key set for it (review on #5181).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): a project's .env cannot set the HeyGen base or allow HTTP

loadEnvFromDir copied HEYGEN_API_BASE from a project file into process.env, and the host gateway path then sent the
person's shell HEYGEN_API_KEY to that host. A host app sets HEYGEN_API_BASE and HEYGEN_ALLOW_HTTP in the environment
it spawns, never in a project file, so the .env loader skips both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:30:56 +00:00
Miguel Ángel 4fcad1ed74 fix(media-use): use a host app's music tools; fail loudly without the heygen CLI (#5154)
* fix(media-use): use a host app's music tools; fail loudly without the heygen CLI

* fix(media-use): name the heygen CLI only for music and sound effects; keep the sfx test offline

* fix(skills): workflow skills use a host app's own music tools first

* fix(skills): say how a host app's music track reaches the narrated workflows

* fix(skills): set a host app's music track after the last audio pass

* fix(skills): a host-music film with no narration still fetches its sound effects

* fix(media-use): keep the stub-failure test off the machine's heygen CLI; sync the skills manifest

* fix(skills): audio passes keep a host app's music and sounds; silent needs no sound cues

* fix(skills): host audio keeps string frames, stops on a broken audio_meta.json, survives a re-run

* fix(skills): report a host audio entry with no path and a broken sidecar instead of crashing

* fix(skills): keep host audio written while an audio pass runs

* test(cli): serve the telemetry capture while the resolve child waits on it

* test(cli): fail the telemetry capture when the resolve child exits non-zero

* style(skills): wrap the host audio comment to the 120-column limit
2026-10-07 07:16:51 +00:00
Miguel Ángel 2497402b27 feat(media-use): clone, list and delete HeyGen voices, and accept a host OAuth token (#4992)
* feat(media-use): clone, list and delete HeyGen voices, and accept a host OAuth token

heygen-voice.mjs clones a voice from an mp3 or wav file, lists cloned voices by name prefix, and deletes one, printing HeyGen's own error message on any refusal. HEYGEN_ACCESS_TOKEN is now the first credential source and is sent as an OAuth Bearer token.

* fix(media-use): heygen-tts prints HeyGen's own refusal instead of a generic line

* chore(media-use): regenerate the skills manifest for the voice scripts

* fix(media-use): voice list stops on a repeated page, delete accepts an empty reply, plain refusals

* test(media-use): a voice list that pages forever fails at once instead of hanging

* docs(media-use): keep heygen-voice's header to three lines, usage lives in tts.md
2026-10-04 15:07:30 +00:00
Miguel Ángel 981bcf4553 fix(media-use): say to fix an unreadable credentials path, not to log in again (#4998)
* fix(media-use): say to fix an unreadable credentials path, not to log in again

* fix(media-use): provider=heygen gives the same fix hint as the auth headers

* test(media-use): restore env in the pickProvider test with a plain if
2026-10-04 10:25:47 +00:00
Miguel Ángel 32d2bc6fe0 fix(media-use): a credentials folder means no HeyGen credential, not a crash (#4995)
* fix(media-use): a credentials folder means no HeyGen credential, not a crash

* fix(media-use): keep heygenCredential never-throws for any unreadable credentials path
2026-10-04 08:04:07 +00:00
Miguel Ángel 5f3191bdc5 fix(media-use): skip a .env folder when loading HeyGen credentials (#4988)
* fix(media-use): skip a .env folder when loading HeyGen credentials

heygen-tts.mjs and audio.mjs crashed with EISDIR when a parent folder held
a .env directory instead of a file.

* fix(media-use): still load a .env that is a pipe or a locked folder's file

* fix(media-use): read a .env without checking it first, so it cannot change in between
2026-10-04 07:29:46 +00:00
miga-heygenandMiguel Ángel 835e0c16ec fix(media-use): forward TTS speed to Kokoro (#4930)
* fix(media-use): forward TTS speed to Kokoro

synthesizeOne built the `npx hyperframes tts` argv without --speed, so a
request speed (audio_request.json `speed` or audio.mjs --speed) was ignored
on the Kokoro path and every line rendered at 1.0. Forward it when it is not
the default.

Kokoro now enforces the CLI's speed range, so audio.mjs rejects a speed
outside (0, 3] up front instead of dropping every line with an opaque
subprocess exit.

Fixes #4926

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* fix(media-use): reject zero and non-numeric TTS speed

`Number(x) || 1.0` turned 0 and non-numeric speeds into 1 before the range
check, so they passed silently. The engine now parses the raw request or
--speed value once, with no fallback, and the faceless-explainer,
pr-to-video and product-launch-video adapters forward the raw flag instead
of applying their own copy of the fallback.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

---------

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-03 03:35:20 +00:00
Poetry Of CodeandMiguel Angel Simon Sierra f1b79d8e7b fix(media-use): honour HYPERFRAMES_PYTHON in the audio engine (#4614) (#4723)
* fix(media-use): honour HYPERFRAMES_PYTHON in the audio engine (#4614)

resolvePythonCommand() only probed python3/python on PATH, so on PEP 668
systems set up with a venv via HYPERFRAMES_PYTHON (the documented setup),
doctor reported MusicGen installed while BGM was silently skipped.

Check HYPERFRAMES_PYTHON first, as the CLI's findPython() does, and fall
through to the PATH probe when the override doesn't run.

* fix(media-use): require a Python 3 version from the interpreter probe

Address review on #4723:
- The probe accepted any executable that exits 0, so a Python 2
  HYPERFRAMES_PYTHON (or python on PATH) was taken while the CLI's
  validatePythonOverride() rejects it. Require "Python 3" in the
  --version output, reading stderr too since Python 2 prints there.
- Regenerate skills-manifest.json for the media-use change.

* chore(skills): regenerate manifest without local __pycache__

* chore(media-use): trim python resolver comments to the comment limit

* style(media-use): format the python probe test

---------

Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
2026-10-02 05:51:30 -07:00
Miguel Ángel 2195db5e1d fix(media-use): never replace a voice, music or sound file the person put in the project (#4665)
* fix(media-use): record agent-made voice, music and sound effects in the project's media manifest

* fix(media-use): record in place only when --source says the agent made the file

* test(media-use): a rewritten agent file keeps one manifest record

* fix(media-use): leave a person's own sound effect unlabelled and mark locally made music generated

* refactor(media-use): record agent audio in-process through the shared manifest library

* style(media-use): format the in-process recording and refresh the skills manifest

* chore(skills): refresh the skills manifest for the media-use changes

* fix(media-use): report a broken ffprobe setting plainly and document the project sfx source

* style(skills): format the pipeline manifest assertion

* chore(skills): refresh the skills manifest

* fix(media-use): never replace a voice, music or sound file the person put in the project

* style(media-use): format the keep-person-files change

* chore(skills): refresh the skills manifest

* fix(media-use): a regenerated file gets a fresh record, and lookups skip the replaced one

* test(media-use): a replaced record no longer answers a prompt lookup in the CLI

* chore(skills): refresh the skills manifest

* chore(skills): refresh the skills manifest from a clean tree

* fix(media-use): never give a recorded file the id of a download in flight

* chore(skills): refresh the media-use manifest hash

* fix(media-use): say when audio lands on a new name, and never let two lines share one

* style(media-use): format the keep-files change

* chore(skills): refresh the media-use manifest hash

* fix(media-use): keep two sound effects off one file when a name is taken

* refactor(media-use): drop a set the one-at-a-time downloads never needed

* chore(skills): refresh the media-use manifest hash

* fix(media-use): keep two sound effects off one recorded file on a rerun

* style(media-use): format the rerun test

* chore(skills): refresh the media-use manifest hash

* fix(media-use): list only a file's newest record in candidates and the media index

* revert(media-use): leave promote as it was, nothing calls it

* test(media-use): give the index test's two assets their own files

* chore(skills): refresh the media-use manifest hash

* chore(skills): regenerate the manifest after merging the base branch

* fix(core): a figma import lists each media file once, like media-use
2026-09-29 01:45:34 +00:00
Miguel Ángel e25bc3a806 fix(media-use): record agent-made voice, music and sound effects in the media manifest (#4648)
* fix(media-use): record agent-made voice, music and sound effects in the project's media manifest

* fix(media-use): record in place only when --source says the agent made the file

* test(media-use): a rewritten agent file keeps one manifest record

* fix(media-use): leave a person's own sound effect unlabelled and mark locally made music generated

* refactor(media-use): record agent audio in-process through the shared manifest library

* style(media-use): format the in-process recording and refresh the skills manifest

* chore(skills): refresh the skills manifest for the media-use changes

* fix(media-use): report a broken ffprobe setting plainly and document the project sfx source

* style(skills): format the pipeline manifest assertion

* chore(skills): refresh the skills manifest

* fix(media-use): a regenerated file gets a fresh record, and lookups skip the replaced one

* test(media-use): a replaced record no longer answers a prompt lookup in the CLI

* chore(skills): refresh the skills manifest

* chore(skills): refresh the skills manifest from a clean tree

* fix(media-use): never give a recorded file the id of a download in flight

* chore(skills): refresh the media-use manifest hash

* fix(media-use): list only a file's newest record in candidates and the media index

* revert(media-use): leave promote as it was, nothing calls it

* test(media-use): give the index test's two assets their own files

* chore(skills): refresh the media-use manifest hash

* fix(core): a figma import lists each media file once, like media-use
2026-09-29 00:07:16 +00:00
James Russo 01601d1105 feat: add Gemini TTS models and service-account auth to media skills (#4377)
* feat: add opt-in Gemini 3.8 TTS to media skills

* feat: support Gemini service accounts and older TTS models

* fix(audio): isolate Gemini auth and narrow OAuth scope

* fix(audio): pin Gemini Python helpers to UTF-8
2026-09-24 02:21:23 +01:00
Miguel Ángel 29fc95395d fix(skills): keep script imports inside installed skill directories (#4356)
* fix(skills): keep script imports inside installed skill directories

* refactor(skills): simplify standalone module checks

* fix(skills): carry the brand-logo lookup into the media-use copy of media-fetch
2026-09-23 19:09:09 -04:00
James Russo a7e2e3853a fix(media): validate every download redirect target (#3841)
* fix(media): validate every download redirect target

* fix(media): apply redirect policy to logo HEAD probes
2026-09-10 06:16:25 -04:00
James Russo a7bf0d23d9 fix(skills): retain audio metadata file identity during generation (#3778)
* fix(skills): retain audio metadata file identity during generation

* test(skills): isolate audio metadata CLI credentials
2026-09-08 17:35:12 -04:00
James Russo a92dc6865e fix(skills): read BGM logs without pathname prechecks (#3769) 2026-09-08 16:40:20 -04:00
Miguel Ángel f8a1e2d315 fix(skills): pin UTF-8 in Python scripts instead of the platform code page (#3298)
Windows sizes Python's stdio and text-mode file IO to the ANSI code page
(cp1252), not UTF-8. Every skill Python script relied on that default:

  * analyze-beatgrid.py --print writes the glyphs cp1252 has no slot for
    (delta, arrow), so the brief died with UnicodeEncodeError on every Windows
    run — the reported crash;
  * its audiomap write_text() pairs ensure_ascii=False with the default file
    encoding, so a non-ASCII payload is unwritable there too;
  * lint_source.py read_text() raises UnicodeDecodeError before any rule runs
    when a Remotion source carries an em dash or a curly quote;
  * gen-stroke-path.py reads an SVG font whose glyph keys ARE literal
    characters, so a mis-decoded key stops matching the requested text.

Stdio is reconfigured to UTF-8 at import and every text-mode IO call names its
encoding. `errors` is carried across the reconfigure: it resets to "strict",
and CPython gives stderr "backslashreplace" on purpose so the diagnostic path
can never itself raise.

extract-audio-data.py also decoded ffmpeg's stderr strictly while reporting a
failure, which would bury the very error being reported on a Windows ffmpeg.

skills/python-encoding.test.mjs guards the class: it fails if any skill Python
script drops the stdio block or omits encoding= on a text-mode IO call. The
mode is read as a whole comma-delimited argument of mode characters only, so a
payload key like {"bpm": 120} cannot spell the check away.

Verified with a cp1252 stdio stream installed before module load, matching how
Windows starts the interpreter: pre-fix UnicodeEncodeError, post-fix both
glyphs present in the UTF-8 bytes. Not run on real Windows hardware.
2026-08-17 21:44:08 -04:00
Vance IngallsandClaude Opus 5 255cf92915 fix(skills,producer): terminate ffprobe options in shipped skill scripts
The contract test only walked packages/*/src and only .ts, so it could not see
the shipped agent tools under skills/**, which are .mjs/.cjs. 19 call sites
there and in package tests were still missing `--` immediately before the
input while the suite reported the bug class closed — a dash-prefixed filename
is parsed as an option and fails the same way.

Sweeps packages/, skills/ and scripts/ now, including .mjs/.cjs and test
files (dither.test.mjs was one of the broken sites). Excludes only the
contract test itself, which documents the contract with example argvs
including a deliberately misordered one.

Two guards were fixed while widening: the terminator must never be inserted
after `-i`, which consumes the next token (a blind pass hit an ffmpeg input
and a base64 -i), and comment prose describing a spawn is not a spawn.

Also routes every audioPadTrim probe failure through one sanitizer at the
boundary. runFfprobeJson scrubbed its own stderr, but
defaultProbeVideoFrameInfo threw `no video stream in ${videoPath}` raw into
the public PadTrimAudioResult.error, and an injected probe can throw anything.
The redaction unit tests all passed with the caller wiring deleted; the new
public-path regressions fail without it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 02:23:35 -07:00
Miguel Ángel dd938c7a16 feat(media-use): tag HeyGen calls with X-HeyGen-Client-Source (#2365)
Send `X-HeyGen-Client-Source: media-use` on every media-use HeyGen API
request (both auth types, via heygenAuthHeaders + the heygenJSON transport),
so backend billing meta can isolate media-use consumption from other free
TTS and avatar-video usage. Unconditional of auth type — a paying user's
media-use call is still media-use — unlike the OAuth-only cli-source header
that gates the free allowance.
2026-07-13 20:03:31 -04:00
Miguel Ángel eba9ff9ae1 fix(media): resolve npx without npm_execpath on Windows (#2236)
* fix(media): resolve npx without npm_execpath on Windows

* test(media): preserve npx resolution diagnostics

* chore: refresh skills manifest

* chore: refresh skills manifest after rebase
2026-07-11 18:36:13 -04:00
Miguel Ángel 9c98c1e82a fix(hyperframes-media): surface the real reason a TTS line failed [P2] (#1999)
* fix(hyperframes-media): surface the real reason a TTS line failed

synthesizeHeygen() swallowed every failure into a bare { ok:false }: a thrown
HTTP error (e.g. 402 plan_upgrade_required from heygenJSON) was caught and
discarded, a missing audio_url / failed audio fetch / failed transcode all
returned nothing. audio.mjs then logged 'TTS failed — omitted' for every line
with zero detail, so the actual cause took a hand-rolled repro to find.

Each failure path now returns an { error } string (the caught message, the HTTP
status, or the specific stage that failed), and audio.mjs appends it to the
anomaly. The subprocess providers (elevenlabs/kokoro) get the same treatment via
a shared synthResult() helper. synthesizeHeygen takes an injectable deps arg so
the failure paths are unit-tested (thrown 402, non-ok fetch, missing audio_url).

* fix(media-use): report wav transcode failures accurately

* chore: regenerate skills manifest
2026-07-11 18:32:08 -04:00
Miguel Ángel 521f2c9ba7 feat(media-use): usage telemetry for HeyGen conversion (#2130) 2026-07-10 20:07:03 -04:00
Vance IngallsandClaude Fable 5 c992a136bf fix(skills): pipeline fixes from prompt-guide validation (BGM, caption accent, voice, PR version)
Behavior fixes surfaced by the prompt-guide validation campaign (Tier 1+2
of the upstream bug list; Tier 3 tracked in #2107). Split out from the
doc-only updates, which follow in a separate PR.

- BGM level: default bed volume under narration was 0.8 linear (~-2 dB,
  ~16 dB too hot vs voice). Now 0.12 (~-18 dB) via shared
  bgmDefaultVolume() in media-use bgm.mjs + assemble-index fallbacks in
  faceless-explainer / pr-to-video / product-launch-video. Explicit
  volume still wins; silent-film 0.9 and music-to-video unchanged.
  Adds bgm.test.mjs (3 cases); bgm.md reference updated to match.
- Caption accent: semanticColors() ranked accents purely by chroma, so a
  preserved status red (#dc2626) outranked the brand accent and captions
  highlighted in error-red. Status-keyed colors now excluded via shared
  STATUS_ROLE_KEY regex consumed by both tokens.mjs and build-frame.mjs
  (all three skill copies kept in sync).
- Voice threading: workflow SKILL.md Step 3.1 blocks now instruct
  choosing the narration voice from the user's ask and passing
  --voice <id>; previously "a male voice" was silently ignored and the
  default (Marcia/am_michael) always won.
- fetch-pr shipping version: MERGED PRs get best-effort shipped_version
  + version_source in pr.json (first release published at/after merge,
  else default-branch package.json marked unreleased); ingest surfaces
  it as a 'Shipped in:' brief line; story-design.md forbids inventing
  versions when absent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 22:12:52 -07:00
Miguel ÁngelandClaude Opus 4.8 3b93f516b4 feat(media-use): use CLI free HeyGen usage (#2027)
* feat(media-use): use CLI free HeyGen usage

* fix(media-use): address #2027 R1 nits — gate cli-source header to OAuth, export origin constant

- X-HeyGen-Source is now sent only on OAuth (Bearer) requests, not API-key ones —
  the backend ignores it for API-key traffic (normal billing), so it was dead
  metadata there. buildAuthHeaders + heygenAuthHeaders + tests updated.
- Export HEYGEN_CLI_ORIGIN_HEADER ("X-HeyGen-Client-Origin") for future cli:<origin>
  consumers.
- Document the deliberate paid/X4 confirm-before-call decision on heygen.tts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* refactor(cli): drop unused origin-header export, dedup auth-client tests

Fallow flagged 5 findings on this PR:
- major: HEYGEN_CLI_ORIGIN_HEADER was exported but never emitted or
  imported — speculative dead code ("future consumers"). Remove it; a
  real consumer can add the constant when one exists.
- 4x minor duplication in client.test.ts: fold the repeated
  `.rejects.toSatisfy(auth-code)` assertion into expectAuthCode(), and the
  repeated try/catch scrubbed-message assertion into expectRejectionMessage().

No behavior change; auth/client tests still 17/17.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 18:28:26 -04:00
Miguel Ángel 42a209545b fix(media-use): create the output dir before ElevenLabs TTS writes
The ElevenLabs provider spawns a Python helper that writes straight to wavAbs
via a bare open(), which (unlike heygen/kokoro) never creates the parent dir —
so on a fresh project the save throws ENOENT and the line is silently dropped
as 'TTS failed - omitted'. mkdir -p the dir first, guarded so a mkdir failure
(EACCES/EROFS) returns { ok:false } like the rest of the branch rather than
throwing. (Migrated from #1960, whose skills/hyperframes-media path was retired
into skills/media-use; the bug moved with it.)
2026-07-07 17:16:35 -04:00
Miguel Angel Simon Sierra 5fe957363d feat(media-use): v2 media OS core (resolve cascade, providers, local generation, telemetry) + retire hyperframes-media 2026-07-06 23:41:05 -04:00