Commit Graph
97 Commits
Author SHA1 Message Date
Märten Tropp 63f32fefa9 docs(skills): fix frame-worker contract, lint codes and stale references (#5515)
* docs(skills): fix frame-worker contract, lint codes and stale references (#5467)

* docs(skills): use placeholder paths so comment citations resolve (#5467)
2026-10-13 04:12:09 +00:00
1d2aa5240d fix(media-use): transcript-cut video renders without repeated frames or a keyframe warning (#5463)
* fix(media-use): transcript-cut video renders without repeated frames or a keyframe warning

transcript-cut wrote each kept range to a temporary .mkv, and ffmpeg
writes Matroska timestamps in whole milliseconds. A 30 fps frame that
belongs at 2/30 s ended up at 0.067, the final copy kept it there, and
the renderer showed the previous frame wherever a timestamp was late.

The temporary files are now .mov, which holds the frame times exactly
and still carries PCM audio, so the audio is encoded once, as before.

The cut also gets a keyframe every second. x264 on its own left them
several seconds apart, and the renderer warned about the footage this
script had just written.

Fixes #5417

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): transcript-cut ranges start on their first frame with any ffmpeg

ffmpeg 7 and later start a kept range that begins between two source
frames one frame late, and .mov rounds that lead-in to the millisecond,
which put the frames off the grid again. ffmpeg 6 did not do this, so
the tests passed locally and failed in CI.

Each temporary file is now written at a constant frame rate, so the
first frame of a range is at zero whichever ffmpeg runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: James <james.russo@heygen.com>
2026-10-13 01:23:05 +01:00
b71c9f4e1b fix(media-use): keep frame precision on transcript-cut seek boundaries (#5426)
* fix(media-use): keep frame precision on transcript-cut seek boundaries

Fixes #5416.

A kept range whose boundary sits on a frame got rounded to the millisecond
twice on its way to ffmpeg, and at 29.97/30/60 fps that is enough to land just
past the frame: frame 386 at 30 fps is 12.866666 s, and -ss 12.867 starts that
segment on frame 387 instead. ffmpeg's accurate seek drops the frame, so the cut
came out a frame short from that join on and the picture drifted behind the
sound.

cutlist.mjs rounded the emitted range bounds to 3 decimals, so fmt() in
transcript-cut.mjs never saw more precision than that in the first place; fixing
only fmt() leaves the bug in place. Both now keep frame precision -- 6 decimals
in the cut list, full precision for the -ss/-to actually handed to ffmpeg --
while the 3-decimal fmt() is kept for the times that are only printed.

Measured at 30 fps: a cut of 0-1,1.0666666-3 produced 87 frames before and 88
after (frames 0-29 plus frames 32-89), with no timestamp gap at the join. 25 fps
is unaffected, as the issue notes.

* test(media-use): end ffprobe options with -- in transcript-cut test

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 01:08:20 +01:00
1577060dc1 fix(media-use): honor telemetryEnabled: false from shared config in optedOut (#5490)
* fix(media-use): honor telemetryEnabled: false from shared config in optedOut

The CLI's 'hyperframes telemetry disable' writes telemetryEnabled: false to ~/.hyperframes/config.json, but the media-use scripts' optedOut() only checked env vars, so transcribe.mjs, transcript-cut.mjs and audio-duck.mjs kept sending PostHog events for users who had opted out through the CLI.

optedOut() now also returns true when the shared config has telemetryEnabled === false, reusing the readSharedConfig() helper already in the same file. Applied byte-identically to both copies of telemetry.mjs (skills/media-use/scripts/lib and packages/cli/src/media-use/lib, per the copy-parity rule), with tests in both test files.

Closes #5465

* test(media-use): sandbox HOME in the telemetry opt-out test and regenerate the skills manifest

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 23:56:46 +01:00
DanteandJames ea7a4e433f fix(media): read ElevenLabs input as UTF-8 (#5469)
* fix(media): read ElevenLabs input as UTF-8

* fix(media): regenerate skill manifest after UTF-8 correction

---------

Co-authored-by: James <james.russo@heygen.com>
2026-10-12 22:56:24 +01:00
路从今夜白 73d89b81c7 fix(media-use): clean up temporary TTS text directories (#5477)
* fix(media-use): clean up temporary TTS text files

* chore(skills): refresh manifest for TTS cleanup
2026-10-12 21:08:50 +00:00
Miguel Ángel 44e5a47e09 fix(cli): media-treatment applies one treatment to every matched image or video (#5443)
* fix(cli): media-treatment applies one treatment to every matched image or video

* fix(cli): media-treatment reports its action from every target and documents the per-element merge

* fix(cli): media-treatment patches each match by its stable id so one write cannot redirect another

* fix(cli): media-treatment sets every match in one parse instead of re-finding elements

* fix(cli): media-treatment refuses media that is not an HTML element, as before

* chore(skills): refresh the media-use manifest hash for the treatment reference edit
2026-10-11 03:45:37 +00:00
Miguel Ángel f46033d048 fix(skills): skill scripts run through a symlinked skills dir and agents keep long renders alive (#5396)
* fix(skills): skill scripts run when the skills dir is a symlink

* test(skills): plugin launcher fixture ships the shared main-module helper like an install

* fix(skills): the CLI's cube validator runs through a symlink too, and the guard check is stricter

* docs(skills): long agent-started renders run detached in their own process group

* docs(cli): detached renders start in their own process group, on macOS too

* docs(cli): the detached render recipe never waits on terminal input
2026-10-10 04:54:41 +00:00
Abdullah M. Nazif 320de3973b fix(cli): preserve voice placement when ducking empty transcripts (#5271) 2026-10-08 20:09:19 +00:00
Miguel Ángel 8dae5ef89b fix(core): make grading writes previewable and honor computed properties (#5157)
* fix(core): write only authored hue curves, so a single hue curve passes lint

* fix(cli): apply over a grading saved with empty hue curves instead of refusing it

* fix(core): read grading properties from computed style, and stop advising an inline start

* fix(cli): preserve validation during legacy hue repair

* fix(cli): preview grading writes without apply
2026-10-08 15:21:47 +00:00
Wenbo ZhuandClaude Opus 5.5 21d14b25b3 feat(media-use): reach HeyGen through a host app's gateway (HEYGEN_API_BASE) (#5181)
* feat(media-use): reach HeyGen through a host app's gateway (HEYGEN_API_BASE)

- The audio engine sends every HeyGen call to $HEYGEN_API_BASE when set, as the heygen CLI does; plain HTTP needs
  $HEYGEN_ALLOW_HTTP=1, the CLI's own rule. A host gateway (that base with its own key) wins over a host OAuth token.
- The skill says: inside a host that gives HeyGen access, no CLI install or sign-in, prefer the host's own HeyGen
  tools, relay a refused call's message as written and do not switch providers on your own.

HyperFrames Desktop sets these variables when a HeyGen API key is saved in its Settings, so media-use's TTS, catalog
search and avatar calls are paid by that key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(media-use): format the edited tables and regenerate the skills manifest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): host access applies only when HEYGEN_API_BASE is set; HeyGen credentials stay on HeyGen hosts

- The "use the host's own tools first / no CLI sign-in" guidance is now conditional on HEYGEN_API_BASE alone, so an
  agent outside such a host (Claude Code with a HeyGen MCP connector, say) keeps today's media-use journey. The
  compressed copies in README, CLAUDE.md, the prompting overview, general-video and the capability menu are back to
  main's wording.
- A base outside heygen.com gets no stored or host OAuth credential, only a key set for it (review on #5181).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): a project's .env cannot set the HeyGen base or allow HTTP

loadEnvFromDir copied HEYGEN_API_BASE from a project file into process.env, and the host gateway path then sent the
person's shell HEYGEN_API_KEY to that host. A host app sets HEYGEN_API_BASE and HEYGEN_ALLOW_HTTP in the environment
it spawns, never in a project file, so the .env loader skips both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:30:56 +00:00
Miguel Ángel 4fcad1ed74 fix(media-use): use a host app's music tools; fail loudly without the heygen CLI (#5154)
* fix(media-use): use a host app's music tools; fail loudly without the heygen CLI

* fix(media-use): name the heygen CLI only for music and sound effects; keep the sfx test offline

* fix(skills): workflow skills use a host app's own music tools first

* fix(skills): say how a host app's music track reaches the narrated workflows

* fix(skills): set a host app's music track after the last audio pass

* fix(skills): a host-music film with no narration still fetches its sound effects

* fix(media-use): keep the stub-failure test off the machine's heygen CLI; sync the skills manifest

* fix(skills): audio passes keep a host app's music and sounds; silent needs no sound cues

* fix(skills): host audio keeps string frames, stops on a broken audio_meta.json, survives a re-run

* fix(skills): report a host audio entry with no path and a broken sidecar instead of crashing

* fix(skills): keep host audio written while an audio pass runs

* test(cli): serve the telemetry capture while the resolve child waits on it

* test(cli): fail the telemetry capture when the resolve child exits non-zero

* style(skills): wrap the host audio comment to the 120-column limit
2026-10-07 07:16:51 +00:00
Miguel Ángel 2497402b27 feat(media-use): clone, list and delete HeyGen voices, and accept a host OAuth token (#4992)
* feat(media-use): clone, list and delete HeyGen voices, and accept a host OAuth token

heygen-voice.mjs clones a voice from an mp3 or wav file, lists cloned voices by name prefix, and deletes one, printing HeyGen's own error message on any refusal. HEYGEN_ACCESS_TOKEN is now the first credential source and is sent as an OAuth Bearer token.

* fix(media-use): heygen-tts prints HeyGen's own refusal instead of a generic line

* chore(media-use): regenerate the skills manifest for the voice scripts

* fix(media-use): voice list stops on a repeated page, delete accepts an empty reply, plain refusals

* test(media-use): a voice list that pages forever fails at once instead of hanging

* docs(media-use): keep heygen-voice's header to three lines, usage lives in tts.md
2026-10-04 15:07:30 +00:00
Miguel Ángel 981bcf4553 fix(media-use): say to fix an unreadable credentials path, not to log in again (#4998)
* fix(media-use): say to fix an unreadable credentials path, not to log in again

* fix(media-use): provider=heygen gives the same fix hint as the auth headers

* test(media-use): restore env in the pickProvider test with a plain if
2026-10-04 10:25:47 +00:00
Miguel Ángel 18a922f433 fix(media-use): a slow download no longer leaves an empty asset that undo brings back (#4989)
* fix(media-use): a slow download no longer leaves an empty asset that undo brings back

* docs(media-use): tell the person an OAuth sign-in makes voice and avatar video free

* docs(media-use): the OAuth free usage line names the monthly allowance

* docs(media-use): the OAuth line says a free allowance
2026-10-04 08:40:46 +00:00
Miguel Ángel 32d2bc6fe0 fix(media-use): a credentials folder means no HeyGen credential, not a crash (#4995)
* fix(media-use): a credentials folder means no HeyGen credential, not a crash

* fix(media-use): keep heygenCredential never-throws for any unreadable credentials path
2026-10-04 08:04:07 +00:00
Miguel Ángel 5f3191bdc5 fix(media-use): skip a .env folder when loading HeyGen credentials (#4988)
* fix(media-use): skip a .env folder when loading HeyGen credentials

heygen-tts.mjs and audio.mjs crashed with EISDIR when a parent folder held
a .env directory instead of a file.

* fix(media-use): still load a .env that is a pipe or a locked folder's file

* fix(media-use): read a .env without checking it first, so it cannot change in between
2026-10-04 07:29:46 +00:00
miga-heygenandMiguel Ángel 835e0c16ec fix(media-use): forward TTS speed to Kokoro (#4930)
* fix(media-use): forward TTS speed to Kokoro

synthesizeOne built the `npx hyperframes tts` argv without --speed, so a
request speed (audio_request.json `speed` or audio.mjs --speed) was ignored
on the Kokoro path and every line rendered at 1.0. Forward it when it is not
the default.

Kokoro now enforces the CLI's speed range, so audio.mjs rejects a speed
outside (0, 3] up front instead of dropping every line with an opaque
subprocess exit.

Fixes #4926

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* fix(media-use): reject zero and non-numeric TTS speed

`Number(x) || 1.0` turned 0 and non-numeric speeds into 1 before the range
check, so they passed silently. The engine now parses the raw request or
--speed value once, with no fallback, and the faceless-explainer,
pr-to-video and product-launch-video adapters forward the raw flag instead
of applying their own copy of the fallback.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

---------

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-03 03:35:20 +00:00
miga-heygenandMiguel Ángel 94650480e2 docs(skills): fix unresolved cross-references in skill docs (#3951)
* docs(skills): fix unresolved cross-references in skill docs

Skill reference docs that live one directory below a skill root
(`references/`, `sub-agents/`) pointed at sibling skills with `../`,
which resolves inside the referencing skill instead of `skills/`.
Rewrite those to `../../<skill>/...` so every explicitly relative
path resolves from the file that contains it.

- faceless-explainer, pr-to-video, product-launch-video: story-design,
  visual-design, motion-language, cut-catalog, sub-agents/frame-worker
  now point at ../../hyperframes, ../../hyperframes-animation and
  ../../hyperframes-creative.
- music-to-video/references/planning.md: same fix for
  hyperframes-creative design-spec and frame-presets.
- media-use/audio/references/{bgm,tts}.md: `../SKILL.md` ->
  `../../SKILL.md` (the media-use skill root).
- hyperframes/references/frame-worker-core.md: the examples pointer now
  reads ../../hyperframes-animation/examples/; the cut-catalog pointer
  is written as skill-root shorthand (`references/cut-catalog.md`),
  matching the other paths in that file, since the target is the
  consuming workflow's own copy and has no single file-relative path.
- visual-design.md tables re-padded by oxfmt after the cell widened.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* docs(skills): fix the two skill doc cross-references added since the first pass

* chore(skills): regenerate the skills manifest after the rebase

---------

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-10-02 07:29:25 -07:00
Poetry Of CodeandMiguel Angel Simon Sierra f1b79d8e7b fix(media-use): honour HYPERFRAMES_PYTHON in the audio engine (#4614) (#4723)
* fix(media-use): honour HYPERFRAMES_PYTHON in the audio engine (#4614)

resolvePythonCommand() only probed python3/python on PATH, so on PEP 668
systems set up with a venv via HYPERFRAMES_PYTHON (the documented setup),
doctor reported MusicGen installed while BGM was silently skipped.

Check HYPERFRAMES_PYTHON first, as the CLI's findPython() does, and fall
through to the PATH probe when the override doesn't run.

* fix(media-use): require a Python 3 version from the interpreter probe

Address review on #4723:
- The probe accepted any executable that exits 0, so a Python 2
  HYPERFRAMES_PYTHON (or python on PATH) was taken while the CLI's
  validatePythonOverride() rejects it. Require "Python 3" in the
  --version output, reading stderr too since Python 2 prints there.
- Regenerate skills-manifest.json for the media-use change.

* chore(skills): regenerate manifest without local __pycache__

* chore(media-use): trim python resolver comments to the comment limit

* style(media-use): format the python probe test

---------

Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
2026-10-02 05:51:30 -07:00
Miguel Ángel 2195db5e1d fix(media-use): never replace a voice, music or sound file the person put in the project (#4665)
* fix(media-use): record agent-made voice, music and sound effects in the project's media manifest

* fix(media-use): record in place only when --source says the agent made the file

* test(media-use): a rewritten agent file keeps one manifest record

* fix(media-use): leave a person's own sound effect unlabelled and mark locally made music generated

* refactor(media-use): record agent audio in-process through the shared manifest library

* style(media-use): format the in-process recording and refresh the skills manifest

* chore(skills): refresh the skills manifest for the media-use changes

* fix(media-use): report a broken ffprobe setting plainly and document the project sfx source

* style(skills): format the pipeline manifest assertion

* chore(skills): refresh the skills manifest

* fix(media-use): never replace a voice, music or sound file the person put in the project

* style(media-use): format the keep-person-files change

* chore(skills): refresh the skills manifest

* fix(media-use): a regenerated file gets a fresh record, and lookups skip the replaced one

* test(media-use): a replaced record no longer answers a prompt lookup in the CLI

* chore(skills): refresh the skills manifest

* chore(skills): refresh the skills manifest from a clean tree

* fix(media-use): never give a recorded file the id of a download in flight

* chore(skills): refresh the media-use manifest hash

* fix(media-use): say when audio lands on a new name, and never let two lines share one

* style(media-use): format the keep-files change

* chore(skills): refresh the media-use manifest hash

* fix(media-use): keep two sound effects off one file when a name is taken

* refactor(media-use): drop a set the one-at-a-time downloads never needed

* chore(skills): refresh the media-use manifest hash

* fix(media-use): keep two sound effects off one recorded file on a rerun

* style(media-use): format the rerun test

* chore(skills): refresh the media-use manifest hash

* fix(media-use): list only a file's newest record in candidates and the media index

* revert(media-use): leave promote as it was, nothing calls it

* test(media-use): give the index test's two assets their own files

* chore(skills): refresh the media-use manifest hash

* chore(skills): regenerate the manifest after merging the base branch

* fix(core): a figma import lists each media file once, like media-use
2026-09-29 01:45:34 +00:00
Miguel Ángel e25bc3a806 fix(media-use): record agent-made voice, music and sound effects in the media manifest (#4648)
* fix(media-use): record agent-made voice, music and sound effects in the project's media manifest

* fix(media-use): record in place only when --source says the agent made the file

* test(media-use): a rewritten agent file keeps one manifest record

* fix(media-use): leave a person's own sound effect unlabelled and mark locally made music generated

* refactor(media-use): record agent audio in-process through the shared manifest library

* style(media-use): format the in-process recording and refresh the skills manifest

* chore(skills): refresh the skills manifest for the media-use changes

* fix(media-use): report a broken ffprobe setting plainly and document the project sfx source

* style(skills): format the pipeline manifest assertion

* chore(skills): refresh the skills manifest

* fix(media-use): a regenerated file gets a fresh record, and lookups skip the replaced one

* test(media-use): a replaced record no longer answers a prompt lookup in the CLI

* chore(skills): refresh the skills manifest

* chore(skills): refresh the skills manifest from a clean tree

* fix(media-use): never give a recorded file the id of a download in flight

* chore(skills): refresh the media-use manifest hash

* fix(media-use): list only a file's newest record in candidates and the media index

* revert(media-use): leave promote as it was, nothing calls it

* test(media-use): give the index test's two assets their own files

* chore(skills): refresh the media-use manifest hash

* fix(core): a figma import lists each media file once, like media-use
2026-09-29 00:07:16 +00:00
Miguel Ángel 5cbef29d3a feat(cli): transcribe with Parakeet on macOS, Linux and Windows after an opt-in install (#4583)
* feat(cli): download Parakeet with models install parakeet and transcribe with it

`hyperframes models install parakeet` installs the sherpa-onnx runtime from
npm and the Parakeet TDT 0.6B v3 int8 model from Hugging Face, pinned to a
revision. Every file is checked against its pinned size and sha256 under a
temp name before it is renamed into place. Re-running it is a no-op. On
Linux with glibc older than 2.32 it refuses up front.

`transcribe --engine auto|parakeet` now prefers that engine when it is
installed, then an installed parakeet-mlx runner, then whisper. Audio is cut
into about 60 s windows at the quietest 100 ms near each cut, and decoding
runs in a child process. If the child cannot load the runtime or crashes,
transcribe falls back with one stderr line saying why. The JSON output and
transcript.json shape are unchanged.

* fix(skills): report the real word count and engine from transcribe fallbacks

The media-use transcribe script read `words` from the CLI's transcript,
which is a flat array, so its whisper path always reported an undefined
word count. The embedded-captions script labelled every CLI transcript
whisper.cpp even when `--engine auto` ran Parakeet; it now uses the engine
the CLI reports.

* fix(cli): pin the sherpa-onnx native package to the runtime's exact version

* test(cli): type the sherpa install test doubles

* fix(cli): transcribe audio with no speech as 0 words instead of failing

Parakeet writes an empty word list for a music bed, a silent intro or a
very short clip, and loading it failed with "Unrecognized JSON transcript
format". An empty list is now read as a transcript with no words, as
whisper's empty output already is. A non-empty list still has to look like
words.

* fix(cli): re-decode a Parakeet window that skipped the start of its speech

At some window lengths the model silently drops the first clause: on a
looped speech clip, 4 of 167 lengths between 55 and 60 s, and a 61.91 s
window, lost their first 9 words. A window with a token gap of at least 1 s
that is as loud as speech is decoded again with 0.5 s of leading silence,
and the result with more tokens wins. Clean windows are unchanged.

Decoding with 1.5 s of overlap before each cut was measured too and was
worse on 573 s of audio (2.62% WER against 0.98%), so it is not used.

Empty audio no longer makes a zero-length window, which aborted the
decoder.

* fix(cli): stop Parakeet on Ctrl-C and name the repair when its install breaks

- Ctrl-C during a Parakeet transcribe now stops with exit 130 instead of
  starting whisper, and a SIGTERM to the CLI also stops its decode child.
- Ctrl-C during models install parakeet aborts the download and the npm
  install, keeps no partial file, and exits 130. A staging dir left by a
  killed install is swept by the next run.
- When Parakeet fails under --engine auto, the warning leads with the
  Parakeet error and names hyperframes models install parakeet. If the
  fallback fails too, the Parakeet error is reported rather than
  whisper_unavailable, so --optional no longer hides a broken install.
- An explicit --engine parakeet no longer falls back: it fails with the
  Parakeet error and the repair command.
- The install refuses up front on systems with no native package
  (naming the six supported ones) and on musl, instead of a raw npm error.
- 24-bit and float WAV go through ffmpeg, since sherpa-onnx reads only
  16-bit PCM. The decode timeout scales with the audio length.

* docs(cli): point Parakeet setup at models install parakeet

The transcribe docs and the media-use references named parakeet-mlx as the
only way to get Parakeet. They now name hyperframes models install
parakeet, and the CLI reference gets a models section.

* test(cli): cover the npm cancel signal and decode timeout, and fix a doubled period

The Parakeet failure line printed "failed.. To repair" when the decoder's
own message ended in a period. Tests now also pin that Ctrl-C reaches the
npm install and that the decode timeout scales with the audio length.

* fix(cli): report unreadable audio as an input error, not a Parakeet failure

Prepare the WAV once before the Parakeet fallback and hand the same file to whisper,
so ffmpeg runs once. Ctrl-C while ffmpeg runs now ends as a cancel (exit 130)
instead of starting whisper.

* fix(cli): recover a clause Parakeet drops after a pause by decoding the gap alone

Each loud gap without tokens is decoded on its own with 0.5 s either side, and only
its new tokens are spliced in. The padded whole-window retry stays as the fallback.
A window with no tokens at all is no longer decoded twice.

* perf(cli): size the decode timeout from the WAV header instead of reading the file

* docs(cli): say a broken Parakeet install still fails transcribe --optional

* fix(cli): treat Ctrl-C during audio preparation as a cancel and drop the partial WAV

ffmpeg traps SIGINT and exits 255 rather than dying by the signal, so that status now
counts as a cancel unless the timeout fired. A failed or stopped ffmpeg removes the
temp WAV it was writing. The decode timeout now sizes a prepared WAV from its file
size, so a large header can no longer hide its length.

* fix(cli): splice a re-decoded gap by whole words so its neighbours are never doubled

The gap decode splits edge words into other tokens than the window decode did. Match
neighbours as whole words within 0.3 s of each gap edge, add only whole words that start
inside the gap, and decode the gap behind 0.5 s of silence like the padded retry.

* fix(cli): count a cancel only when ffmpeg says it received a signal

ffmpeg 7 and later also exit 255 when a file is not permitted (EPERM), so exit 255
alone read a permission error as a cancel. ffmpeg's stderr is now captured, a cancel
needs its received-signal line, and any other failure reports ffmpeg's last line.
The output path is passed to the shared runner explicitly.

* fix(cli): keep a spliced gap word that has no leading space

The filter dropped such a word; it is now kept and given its word boundary.

* refactor(cli): trim three long comments in the Parakeet decoder and installer

* fix(cli): repair a Parakeet runtime missing its native binary, and pick Parakeet by language

The runtime dir is per platform and arch. Installed now needs the runtime
manifest and the native sherpa-onnx.node; a half-installed dir is removed
before npm runs, since install() keeps any dir holding the runtime manifest.
The decode worker flattens a multi-line loader error onto its prefixed line
and stops between windows once orphaned. transcribe opens its cancellation
scope before audio prep, so Ctrl-C under --json cleans up. auto uses
Parakeet only for its 25 languages and only where the host can run it;
--engine parakeet with another language fails.

* fix(cli): sweep reused-pid staging, name failed model downloads, keep ffmpeg's cause

sweepStaleStaging also removes a staging dir older than 6 hours whose pid
is alive, since containers reuse pids. A failed model download names the
file, the host and a retry. runFfmpeg keeps ffmpeg's stderr on its error,
so transcribe shows "Output file does not contain any stream" for a video
without audio, and reads a cancel only from ffmpeg's own signal line.

* fix(skills): make media-use --engine whisper run whisper and report the engine that ran

The whisper path now passes --engine whisper when asked, or --engine auto,
and reads the engine from the CLI's --json line, so a CLI Parakeet run is
no longer labelled whisper. Docs state the language rule, the real repair
behaviour and the platforms Parakeet runs on.

* test(cli): pin that a long Parakeet decoder error is kept whole up to a bound

* fix(cli): report a broken Parakeet runtime with the repair instead of skipping it

* refactor(cli): name the staging age limit instead of commenting it

* fix(skills): let media-use --engine parakeet use the CLI Parakeet when parakeet-mlx is absent

* fix(cli): repair a Parakeet runtime that fails to load, and read Ctrl-C after whisper

models install parakeet now loads the runtime in a child process, the way
the decode worker does, and reinstalls it when that fails, so a missing
shared library, a corrupt binary or missing runtime JS is repaired. It
reports changed: false when the runtime loads. transcribe keeps its cheap
manifest check. After a failure, transcribe lets the event loop turn once
before reading the Ctrl-C scope, since a signal that landed during the
synchronous whisper fallback is only handled then.

* fix(cli): read Ctrl-C from the whisper child, and re-check the Parakeet runtime after a repair

A whisper fallback killed by SIGINT or SIGTERM is a cancel (exit 130), the
same rule decode and runFfmpeg use; the loop-turn wait is gone. The
runtime probe treats a child stopped by Ctrl-C as a cancel and keeps the
runtime; a reinstall is probed again and fails with the loader's error if
it still does not load. media-use keeps the CLI's last lines, stdout
included, when hyperframes transcribe fails.

* fix(cli): give the user-stop rule one owner, SIGHUP included, and end a cancelled install at 130

stoppedByCancelSignal in renderCancellation.ts uses the scope's own signal
list (SIGINT, SIGTERM, SIGHUP) and is false whenever Node stopped the child
(a code, killed, or a spawn error). decode, runFfmpeg, the runtime probe and
the whisper fallback all route through it, so a probe killed by SIGHUP keeps
the runtime. isWhisperTimeoutError now keys on ETIMEDOUT only, which Node
sets on execFileSync's own timeout, so a SIGTERM reaches the cancel rule.
A cancelled models install sets exit 130 and returns instead of failing, and
the probe says what went wrong (timed out, crashed, no output).

* refactor(cli): let the stop-rule helper name carry its meaning
2026-09-27 23:21:16 +00:00
James Russo 0a5e3c7fc4 feat(plugins): ship portable agent bundles and installation guides (#4608)
* feat(plugins): ship portable agent bundles and installation guides

* fix(plugins): align new manifests with current release

* test(plugins): split caption fixture setup and cleanup

* test(plugins): pass render fixture path as an argument
2026-09-27 21:58:51 +00:00
James Russo ff6e2103d7 fix(skills): keep plugin workflows on their installed release (#4607)
* fix(skills): keep plugin workflows on their installed release

* docs(skills): clarify plugin manifest discovery

* fix(plugins): suppress stale notices and isolate plugin context
2026-09-27 21:05:28 +00:00
Miguel Ángel e3ef033a31 docs: credit the background-removal model as Apache-2.0, matching upstream U-2-Net (#4561) 2026-09-27 03:06:06 +00:00
Miguel Ángel d290c05b09 fix(cli): --yes in an unattended run no longer overrides a no saved by another process (#4547)
* fix(cli): --yes in an unattended run no longer overrides a no saved by another process

* fix(cli): every settings writer takes the lock, and --yes needs a real terminal to reverse a no

* fix(cli): the thin-results download offer also needs a real terminal

* fix(cli): one owner-checked settings lock, and an unreadable settings file never counts as a yes

* fix(cli): settings lock removals go through a reaper, and a non-boolean answer counts as a no

* fix(cli): a settings lock that cannot be released no longer fails a write that landed

* fix(cli): a leftover settings lock is never taken over, and doctor names it

* fix(cli): a background update that cannot be recorded is not launched again every run

* fix(cli): every on-device install re-reads the saved answer first, and an unsaved yes installs nothing

* fix(cli): the thin-results offer stops when a yes cannot be saved

* fix(cli): only the consent-checked step installs the on-device runtime, and a leftover lock warns once

* fix(cli): the leftover-lock warning follows the attended-terminal rule
2026-09-27 01:39:17 +00:00
Miguel Ángel c238abfe6f fix(media-use): tests never write into the person's media library (#4459)
* fix(media-use): tests never write into the person's media library

* fix(media-use): guard the media home only in this repo's test runs

* test(media-use): probe the media home from a plain child process

* fix(media-use): keep miss logging best-effort when home is unresolvable

* test(media-use): miss logging survives an unresolvable media home

* docs(media-use): describe the media library without naming an app

* chore(media-use): refresh the skills manifest hash
2026-09-26 14:13:56 +00:00
James Russo 01601d1105 feat: add Gemini TTS models and service-account auth to media skills (#4377)
* feat: add opt-in Gemini 3.8 TTS to media skills

* feat: support Gemini service accounts and older TTS models

* fix(audio): isolate Gemini auth and narrow OAuth scope

* fix(audio): pin Gemini Python helpers to UTF-8
2026-09-24 02:21:23 +01:00
Miguel Ángel 29fc95395d fix(skills): keep script imports inside installed skill directories (#4356)
* fix(skills): keep script imports inside installed skill directories

* refactor(skills): simplify standalone module checks

* fix(skills): carry the brand-logo lookup into the media-use copy of media-fetch
2026-09-23 19:09:09 -04:00
d49e7b95e0 feat(media-use): resolve brand logos from theSVG (#4291)
* feat(media-use): resolve brand logos from theSVG

Replace the svgl and simple-icons tiers of the `logo` cascade with a
single theSVG (thesvg.org) tier. theSVG ships 7,400+ official brand
SVGs with color, mono, light/dark and wordmark variants.

The tier fetches one manifest pinned to a glincker/thesvg commit on
jsDelivr, memoizes it per process, and matches entities locally by
normalized slug, title, or alias. That drops the hand-kept svgl alias
and simple-icons slug maps. GitHub org avatar and favicon stay as the
fallbacks for brands theSVG lacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): sanitize fetched SVGs and gate theSVG by license

freezeUrl/freezeLocalFile wrote every fetched SVG to disk unsanitized;
the theSVG tier is now the sole SVG-returning logo provider, so an
inline script tag or on-event handler would land verbatim in a user
project. Route .svg destinations through the existing Figma sanitizer
(new @hyperframes/core/sanitize-svg subpath, synced via
package-subpaths).

thesvgMatch never checked theSVG's own license field: entries theSVG
itself marks unresolved (Proprietary, Unknown, Custom, TODO, or an
explicit no-redistribution-license string) could still resolve and
freeze into a project. Gate matches to the ten SPDX ids theSVG's
submission form accepts; a rejected entry falls through to a lower
ranked theSVG match, then to the github/favicon tiers.

Also caps the theSVG manifest fetch at 20MB, streamed, matching
freeze.mjs's existing cap discipline. It had a timeout but no bound.

* chore(skills): regenerate skills-manifest for media-use/motion-graphics

The original submission edited skills/media-use and
skills/motion-graphics docs without regenerating skills-manifest.json;
its own CI never ran (fork approval gate), so the drift went
unnoticed until this pass's first CI run caught it.

* fix(media-use): drop the sanitizer's workspace import, add parity test

freeze.mjs importing @hyperframes/core/sanitize-svg broke the
test-skills CI lane: packages/cli/src/media-use tests run with node
builtins only, zero install, by design (media-use is vendored
standalone into other projects, e.g. packages/desktop). Port the
sanitizer into media-use/lib/svg-sanitize.mjs instead, and drop the
now-unused core export.

Parity between the port and packages/core/src/figma/sanitizeSvg.ts is
enforced by a new test (scripts/check-svg-sanitize-parity.test.mjs,
run in the test:scripts lane, which does have an install) over a
hostile-SVG corpus, so the two cannot drift silently.

* fix(media-use): close three sanitizer bypasses, add fetch timeout

Independent review of the maintainer pass found the SVG sanitizer
(packages/core/src/figma/sanitizeSvg.ts, reused by media-use's own
port) was itself bypassable: an unquoted href/xlink:href survived,
inline style="..." attributes were never checked, and a href bound
through a renamed XML namespace prefix (xmlns:x=".../1999/xlink",
x:href=...) slipped past the literal "xlink:href" match.

Rewrote sanitizeSvg as a tag/attribute tokenizer instead of a set of
literal-spelling regexes: parse each tag's attributes in any quoting
form, resolve the local name after stripping any namespace prefix,
then drop by local name (any on* handler, any *href whose value is
not a #fragment or data:image/ embed, any style attribute containing
url(, @import, expression( or javascript:). Ported the fix into
media-use's svg-sanitize.mjs identically. Legitimate url(#id)
references on non-style attributes (clip-path, fill) are untouched.

Also adds a 10s AbortSignal.timeout to freezeUrl's fetch, matching
every other fetchMedia call in this package - it had a size cap but
no deadline.

New cases for all three bypasses in both sanitizeSvg.test.ts and the
parity corpus, each reproducibly failing against the prior head.

* fix(media-use): replace regex SVG sanitizer with a DOM allowlist

Two rounds of regex/tokenizer patches to sanitizeSvg found new bypasses
in the same shape each time (unquoted href, style= url(), aliased
xlink prefix; then presentation-attribute url(), unquoted '=' handling,
SMIL animate/set href rewrite, self-closing-slash tag corruption,
data:image/ navigation via <a>). Denylisting attribute names against a
hand-rolled tokenizer was the wrong approach regardless of how careful
the regex.

Rewrote sanitizeSvg (packages/core/src/figma/sanitizeSvg.ts) as a real
DOM walk (linkedom, already a dependency of core and cli) with an
allowlist: known-safe elements and attributes only, everything else
dropped by construction rather than pattern-matched as dangerous.
href/xlink:href (any namespace prefix) kept only for a same-document
fragment, or data:image/ off non-<a> elements. Any attribute value or
<style> text containing url() kept only when every url() argument is a
fragment; @import/expression(/javascript: dropped from style content.
Serializing from a real DOM means output is always well-formed, so the
self-closing-slash corruption class of bug is gone by construction.

media-use's own copy (packages/cli/src/media-use/lib/svg-sanitize.mjs)
can't import linkedom directly: its CI test lane runs bare `node --test`
with no install step, so even an external dependency fails to resolve,
not just a workspace import. It's now a generated file instead of a
hand-copy: scripts/generate-media-use-svg-sanitize.mjs bundles the core
source (esbuild, minified, zero remaining imports) and
scripts/check-media-use-svg-sanitize-generated.test.mjs fails the build
on drift. Verified the generated file runs with plain node in an empty
directory with no node_modules on the path.

Every payload from both independent-review reports is now a
table-driven test in sanitizeSvg.test.ts (46 cases) and the same corpus
in check-svg-sanitize-parity.test.mjs (39 cases) asserts core and the
generated media-use copy agree and each independently rejects every
non-local url(), javascript:, data: href on <a>, and on* handler.
Mutation-proved against the real pre-fix blob, not a synthetic case.

Sampled 56 real theSVG icons across all 7 collections to check the
allowlist doesn't over-strip legitimate content: found and fixed three
missing attributes (fx/fy, enable-background, a blanket data-*
allowance) and one real behavioral bug - <style> elements were being
dropped outright, silently unstyling any Illustrator-exported logo that
colors paths via CSS classes. Fixed by keeping <style> on the allowlist
and content-checking its text the same way as an inline style=
attribute, with a regression test using that exact shape.

Full suite after the fix: sanitizeSvg.test.ts 46/46, parity+drift tests
39/39, media-use suite 298/298, tsc/oxlint/oxfmt clean.

* fix(lint): ignore generated svg-sanitize.mjs bundle in oxlint

Minified esbuild output (packages/cli/src/media-use/lib/svg-sanitize.mjs,
generated by scripts/generate-media-use-svg-sanitize.mjs) trips oxlint's
correctness rules 504 times and fails `bun run lint`. Same shape as the
existing registry/blocks/**/*.iife.js exclusions: a generated, minified,
non-hand-edited bundle, not hand-written source.

* fix(format): ignore generated svg-sanitize.mjs bundle in oxfmt

Same generated esbuild bundle that needed an oxlint exclusion in the
previous commit also fails  (minified, no formatting to
check against), cascading Preflight into Player perf, regression,
Windows render verification, and preview-regression on CI. Same
established pattern as the existing registry/blocks/**/*.iife.js and
skills/**/*.min.js entries in .prettierignore.

* refactor(media-use): reduce complexity flagged by fallow audit

isSafeAttribute (10 cyclomatic) and the parity test's aggregate-marker
check (11 cyclomatic, 26 cognitive, CRAP 132) both exceeded fallow's
per-function thresholds after the rebase pulled in a stricter baseline.
Extracted isXmlnsAttribute/isKnownAttribute as named predicates and
survivingUnsafeMarkers as a named per-output helper; no behavior change,
same guard order, same failure strings. Regenerated svg-sanitize.mjs.

* refactor(media-use): make the parity test's marker check table-driven

Fallow gives .mjs scripts no coverage credit, so its CRAP gate needs each
function at cyclomatic 4 or less. The extracted survivingUnsafeMarkers
still scored 9 (CRAP 90). The four regex markers are now a table, and the
data:image-on-<a> and non-local url() checks are small named helpers.
Same checks, same order, same failure strings.

* fix(media-use): close CSS-escape and image-set bypasses in sanitizeSvg

The url()/@import checks scanned raw text for the literal "url(" and
"@import". A browser decodes CSS escapes first, so fill="\75 rl(...)",
url\28...\29 and @\69mport all loaded an external resource, and
image-set("https://...") loads one with no url() at all. Reproduced in
headless Chrome: 8 of 8 payloads fired a request through the old code.

CSS can only load a resource through a function or @import, so CSS text
now passes only if it has no @import, every function is on a compute-only
allowlist (color, transform, filter and math functions), and every url()
is a same-document fragment, all checked on escape-decoded, lowercased
text. Same fix reaches media-use through the regenerated bundle.

Checked against all 13,384 files of the pinned theSVG corpus: 13,383
sanitize byte-identically; one drops an unreferenced id="XRP-(XRP)".
The parity oracle now fails on any surviving attacker host, so it no
longer shares the sanitizer's parsing blind spot.

* refactor(media-use): share one capped body reader for freeze and manifest

fetchJson copied freezeUrl's declared-length check and streaming size cap
line for line. Both now call readCappedBody in media-fetch.mjs, which both
already import; it has its own tests, since freeze's cap had none. The
freeze cap errors now read "freeze failed for <url>: ..." (no caller or
test matches the old wording). Also fixes freeze.mjs's comment, which
still called the sanitizer a lexical pass.

* style(media-use): format the parity test corpus

* fix(media-use): make the sanitizer's CSS scans linear on hostile input

The url() check used /url\(\s*(['"]?)([^)]*)\1\s*\)/ and the function
check /([-\w]*)\(/: on attacker padding these backtracked cubically and
quadratically (an unclosed url( plus 4,000 spaces took 20s; an 80,000
character word took 7.7s), and freeze runs the sanitizer synchronously on
bodies up to 256MB. Both are now single-pass indexOf scans (a million
characters in under 15ms), with the same verdict on ordinary input. A url()
value holding "(" is rejected, so a nested url( can't hide inside one.
A size test holds both payloads under 2s.

* test(media-use): size payloads that fail the old scans fast instead of hanging

* fix(media-use): stop the freeze timeout from aborting long downloads

The 10 s AbortSignal.timeout on freezeUrl also aborted the response body,
so any stock video, generated video or LUT that took longer than 10 s to
download failed. The timer now covers only the wait for response headers
and is cleared once they arrive; the 256 MB cap still bounds the body.

* refactor(media-use): trim logo-provider comments to the repo limit

The tier list and the thesvgMatch doc block ran past the four-line comment
cap that the merge check enforces. Same facts, shorter blocks.

* refactor(media-use): name the header wait and restore the variants note

Renames the freeze timeout controller to headerWait so it no longer reads
as HTTP headers, puts the theSVG variants fact dropped by the comment trim
next to the line that picks the default mark, and keeps thesvgMatch on the
file's JSDoc style.

---------

Co-authored-by: Imad Phelps <imad.phelps@heygen.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
2026-09-23 16:56:16 -04:00
dd97478175 fix(media-use): remove clicks and ~25ms gaps at every transcript-cut splice (#4179)
* fix(media-use): remove clicks and ~25ms gaps at every transcript-cut splice

transcript-cut extracts each kept range to its own final-codec file, then
concat-copies them. That produces two audible artifacts at every cut:

1. No boundary fades. Concat splices raw segment edges together, so the
   waveform steps discontinuously and each cut clicks.

2. Encoder priming silence. Each segment is encoded to AAC independently,
   so the encoder pads every one with priming samples. Stream-copy concat
   bakes that padding in as a silent gap at each splice. Measured 24.4 ms
   and 24.1 ms at the two splices of a three-range cut. It accumulates:
   the nth splice lands roughly 25n ms late, so splice positions drift
   away from the sum of the preceding segment durations.

The second survives the first -- adding fades alone leaves the gap.

Fix: intermediates carry PCM (.mkv for video, .wav for audio-only) instead
of the final codec, so audio is encoded exactly once, at concat. Boundary
fades of 30 ms are applied per segment, scaled down on segments shorter
than 4x the ramp and skipped on degenerate ones.

Measured on the same three-range cut, widest run of digital silence at
each splice:

  before  24.4 ms / 24.1 ms
  after    0.1 ms /  0.1 ms

--copy is unchanged: stream copy cannot filter, so it keeps producing
final-codec segments and a copy concat. Audio-only output is byte-identical
in duration before and after. scripts/lib test suite: 34/35, the one
failure (lut-preset-provider) pre-exists on main and is unrelated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(skills): regenerate manifest for media-use

* fix(media-use): honor the fadeIn/fadeOut flags fadeFilterFor was passed

The caller already computed {fadeIn, fadeOut} per segment, but
fadeFilterFor still took one parameter and always emitted both
afade ramps -- so a single kept segment (no splice at all) and every
segment's true start/end got faded regardless. Extracted the pure
function into lib/transcriptCutFade.mjs, made it honor both flags,
and added its first unit test: first/middle/last/single-segment
filter strings, no ffmpeg needed, fails against the previous version.

Also installs ffmpeg in the Test: skills CI job (same action the
Producer job uses), so transcript-cut.test.mjs's real-tone RED/GREEN
proof runs in CI instead of skipping silently for lack of the binary.

* fix(media-use): fix the packed-CLI import and lint on transcript-cut.mjs

build-copy.mjs replaces dist/skills/media-use/scripts/lib with a copy
of packages/cli/src/media-use/lib, so transcriptCutFade.mjs needed a
byte-identical twin there (and in MEDIA_USE_COPY_NAMES) or the packed
CLI's transcript-cut.mjs imported a file that no longer existed.
Moved its test alongside, matching every other lib helper's tests.

Also drops three redundant Boolean(args.copy) wrappers (a ternary
condition already coerces) that oxlint's no-extra-boolean-cast
flagged.

* style(media-use): reflow the shortened ext ternary to oxfmt's shape

Dropping the Boolean() wrapper shortened the line enough that oxfmt
0.41 now collapses the ternary onto one line; format:check was failing
on the stale multi-line version.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
2026-09-22 20:28:50 -04:00
Miguel Ángel 3091dcb787 feat(cli): route media search through media-use (#4230)
* feat(cli): move media-use engine into the CLI

* fix(cli): replace media-use compatibility asset

* fix(cli): replace moved media-use tree

* fix(cli): copy moved media-use files cleanly

* fix(cli): copy media-use engine files individually

* fix(cli): keep audio helper compatibility import

* fix(cli): clear copied skill destinations

* test(cli): preserve moved media-use test paths

* fix(cli): preserve media-use script compatibility imports

* fix(cli): preserve media-use telemetry shim

* fix(cli): resolve packaged media-use LUT assets

* fix(cli): resolve packaged bundled SFX assets

* chore(skills): regenerate media-use manifest

* fix(cli): preserve media-use parity coverage

* fix(cli): cover relocated media-use tests

* chore(skills): regenerate media-use manifest

* fix(cli): format bundled media provider

* fix(ci): run relocated media-use tests with node

* fix(ci): preserve fallow script exemptions

* fix(ci): scope media-use fallow exemption

* refactor(cli): trim media-use essay comments

* feat(cli): index media-use assets for catalog search

* feat(catalog): commit media-use search vectors

* chore(catalog): track media vector metadata

* fix(catalog): validate media vector row order

* fix(catalog): validate media artifact freshness

* fix(catalog): cover media metadata and credits

* fix(cli): reject malformed media rows

* fix(catalog): compare complete media rows

* feat(cli): route media search through media-use

* refactor(cli): trim relocated media-use comments

* fix(cli): make media search runtime-safe

* fix(cli): preserve sfx fallback diagnostics

* fix(cli): probe sfx fallback provider explicitly

* chore(skills): regenerate manifest

* refactor(cli): keep typed media ranker source

* fix(cli): align media search checks with local-first flow

* fix(cli): scope vector audit exceptions

* fix(cli): ship resolver registry beside skill

* fix(cli): keep media-use skill scripts standalone

* test(cli): cover local vector validation

* test(cli): correct media vector fixtures

* fix(cli): prefer exact media keys

* fix(cli): stem exact media keys

* test(cli): skip media lint for snippets without media

* fix(cli): align skills test discovery

Keep the CI skills job's bash discovery as the single source of truth so its empty-list guard covers the exact files Node executes. Make hasMediaVectors private because it has no external callers and needs no Fallow suppression.

* fix(cli): close media-use review gaps

Prioritize literal ids before stemmed ids and cover the collision alongside the plural regression. Add command seam tests for engine discovery, actionable failure, and argv passthrough. Add a deterministic parity check for standalone media-use copies, with documented allowlists for the standalone media-fetch shim and self-contained npx resolver. Keep the bash-discovered skills test list as the CI source of truth.

* fix(cli): lower parity check complexity

Split missing-copy and unallowlisted-drift checks so the parity guard keeps its behavior without a threshold CRAP score.

* fix(cli): close media-use review seams

Run the parity test in the skills job so skills-only edits exercise the same check. Export the verb mapping and inject command dispatch for direct coverage. Keep missing-copy diagnostics separate from drift reads.
2026-09-21 18:35:47 -04:00
Miguel Ángel 2db126d4c8 docs(skills): one owner doc for everyday edits and a hyperframes-studio skill (#4125)
* docs(skills): one owner doc for everyday edits and a hyperframes-studio skill

Agents got contradictory advice on volume fades and were blocked from splits
and copies by the Ask-agent guardrails.

- creator-editing-recipes.md is the single owner: adds align-to-event,
  copy-a-group and swap-media recipes and the one volume rule (use the lane)
- variables-and-media, data-attributes (skill and CLI copy), the gsap adapter
  and tracks-and-clips now defer to it
- Ask-agent guardrails allow the neighbour edits a split, retime, copy or swap
  requires and point at the owner doc
- new hyperframes-studio skill: sub-compositions for every scene, one caption
  track, one element kind per track, safe zones
- drift test lints every html example in the owner doc with the real linter
  and fails on any doc teaching a volume tween

* refactor(skills): drop unused tween allowlist from recipes drift test

* docs(skills): name the track kind attribute in the studio skill

* docs(skills): stop claiming lint rules that do not exist yet and fix track numbering

Review fixes:

- studio skill: drop the severity claim and the preview-overlay claim,
  group tracks by data-track-kind instead of numeric ranges that clashed
  with the recipes
- recipes: audio data-start is root time only in the root composition;
  scene-local inside a sub-composition file
- guardrails: timeline edits are exempt from the element-only and timing
  lines; the pointer names the skill reference, not a repo path
- montage doc defers to the volume lane; skill count is 21 everywhere
- drift test also scans the montage doc

* fix(skills): the ducking generator emits a volume lane and captions defer to the studio safe zones

- audio-duck.mjs prints a data-automation lane (clip-local times from the
  target data-start) instead of a timeline volume tween; --json adds the lane
- operations.md shows the generated lane; the drift test scans it
- embedded-captions safe-zone lines point at the studio skill, which owns
  the values

* chore(skills): regenerate the skills manifest after rebase

* docs(skills): remotion translation docs point volume ramps at the automation lane

* docs(skills): studio safe zones use the action-safe 90 and title-safe 80 boxes

* docs(skills): caption docs defer to the studio safe boxes

* chore(skills): regenerate the skills manifest after rebase

* fix(skills): studio skill names the safe margins; guardrail test matches the timeline-edit wording
2026-09-18 23:19:40 -04:00
miga-heygenandMiguel Ángel 3704863d96 feat(telemetry): per-tag, provenance, and failure-taxonomy properties on render events (#4004)
* feat(telemetry): add per-tag element and a-roll counts to render_complete

countElementTags's single regex scan (the ~83% of renders that get no
live probe session) is extended into scanElementTags, which returns a
per-tag breakdown and an a-roll video count alongside the same total,
all from one pass over the same script/style-stripped markup — the
total is the sum of the per-tag map by construction, so the two
properties can't drift apart. countElementTags itself is removed; it
had zero production callers after this change.

New render_complete properties: composition_element_tags ({tag: count},
raw lowercase tag names, capped at the top 50 by count with the rest
folded into an "other" key) and aroll_video_count (<video
data-aroll="true"> elements from the same scan). Both are static-scan-
only, mirroring the existing composition_element_count_source split —
the live probe-DOM path never runs this scan.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* feat(telemetry): count HeyGen-provenance video from the same tag scan

heygenVideoCount joins arollVideoCount/byTag as a third property
derived from scanElementTags's single pass: <video
data-media-source="heygen"> elements. media-use's resolve.md now
documents that the skill stamps this attribute on a mounted video only
when the ledger record's provider is "heygen.video" (the mount itself
is always the agent's own composition edit, never written by resolve
scripts) — the ledger stays the single source of truth for provenance,
the HTML attribute is a projection of it, and no other provider value
is invented beyond this one signal.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

* feat(telemetry): add a bucketable failure taxonomy to render_error

error_name (error.name, e.g. one of the ~20 typed producer error
classes) and failed_stage_code (job.currentStage/failedStage
normalized to a stable snake_case code via normalizeStageCode)
join the existing free-text error_message/failed_stage, so a fleet
query can bucket failures without parsing prose. Error-conditional by
nature — render_complete has no equivalent, since a successful render
has no error to name or stage to blame.

Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>

---------

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-09-16 12:37:57 -04:00
Miguel Ángel 95bea1631f refactor(skills): keep hyperframes-core as the HTML contract (#3928)
* refactor(skills): keep hyperframes-core as the HTML contract

Move brief, storyboard, review, production, dispatch, and
frame-worker docs to hyperframes. Slim remaining core references.

* fix(skills): restore camera recipes and root sizing

Restore Zoom, Ken Burns, crop, and clip-path recipes. #root is 100 percent.

* fix(skills): stamp size on the composition root

Runtime sizes the composition root, not html/body. Overlap pin requires is valid.
2026-09-13 21:20:58 -04:00
James Russo a7e2e3853a fix(media): validate every download redirect target (#3841)
* fix(media): validate every download redirect target

* fix(media): apply redirect policy to logo HEAD probes
2026-09-10 06:16:25 -04:00
Miguel Ángel 0f8eb892e3 fix(skills): search the component catalog before hand-building a look (#3829)
* fix(motion-graphics): make the catalog search fire before hand-authoring

The workflow's only reuse instruction pointed at catalog-map.md, a
hand-maintained snapshot of ~60 registry items, and no file in the skill
ever named `hyperframes catalog --query`. An agent asked mid-build for
CRT scanlines and a glitch effect had no instruction to search, so it
hand-authored both while caption-glitch-rgb ("RGB chromatic aberration
with CRT scanline overlay") ranks first for that query on either tier.

The search reads the hosted registry and needs nothing installed, from
any directory with no project, so "the components were not installed"
was never the cause. Say that where the reader is, since the wrong
diagnosis is the intuitive one.

Director Part 2 and the Builder now run the search before naming a
block, and catalog-map.md is labelled a partial snapshot whose misses
prove nothing. Pinned by a content test in coreSkillContent.test.ts.

* fix(skills): search the component catalog before hand-building a look

Authoring workflows never told the agent to search the component library,
so agents rebuilt effects the registry already shipped. A user reported
building an effect from scratch that the registry already contained; the
search that would have found it needs nothing installed, which is why the
usual self-diagnosis ("I forgot to install the components") is wrong.

All ten workflow skills carried zero mentions of `hyperframes catalog`.
The instruction lived only in hyperframes-cli and hyperframes-registry,
both loaded on demand, and the registry skill's own trigger named the
command rather than the symptom - circular, because an agent that never
thought to search could not reach the doc telling it to search.

- Eight workflows now run the search at the point they decide what to
  build, before authoring. The two that compile through a closed
  authoring vocabulary (embedded-captions, talking-head-recut) document
  why they deliberately do not.
- hyperframes-registry triggers on the symptom (a named look, effect,
  treatment or transition) instead of the command name; the router table
  and the catalog surfaces carry the same framing.
- Fixes hand-maintained lists that had drifted: bar-chart-race was listed
  as a hand-author gap in two files while shipping in the registry;
  stat-motion was named as an installable block and is not one; the
  caption-* family count was one high; the registry discovery tables
  claimed to be the block list while covering 97 of 180.
- bun run lint:skills now fails when a doc marked as a registry snapshot
  names an item the registry does not have.

* refactor(scripts): reuse native recursive readdir and the shared registry type

Simplify pass on the new registry-snapshot check, behaviour identical:
- collectMarkdownFiles uses readdirSync({ recursive: true }) instead of
  hand-rolled recursion, matching scripts/generate-template-previews.ts.
- registryItemNames types registry.json with the exported RegistryManifest
  instead of an ad hoc inline shape, matching scripts/catalog/build-local-vectors.ts.
  The runtime guard stays: a cast describes the file, it does not validate it.
- One report() helper replaces the duplicated print-and-count block in both
  lint passes.

* refactor(scripts): name the registry check's blind spots and stop self-arming

Applies the review findings on the new check, behaviour identical except
where noted:
- lintRegistryItemRefs returns null for an unmarked file instead of an
  empty array, so "not a snapshot" and "a clean snapshot" have one owner
  and the marker is matched once rather than twice.
- Marker detection ignores fenced blocks, so a doc that documents the
  marker syntax in an example no longer arms the check on itself. The id
  scan still reads full content, so fenced examples stay covered.
- The header comment and two tests now pin both known false negatives:
  identifiers outside backticks, and single-word item names. Measured on
  the six marked files, dropping the hyphen requirement would monitor 3
  more items and force 46 allow= entries for ordinary prose words, so the
  requirement stays and the gap is stated instead of silent.

* chore(skills): regenerate skills manifest after catalog-search edits
2026-09-10 03:21:33 +00:00
James Russo efe2404cbd fix(skills): stage media downloads in private directories (#3787) 2026-09-08 19:29:26 -04:00
James Russo a7bf0d23d9 fix(skills): retain audio metadata file identity during generation (#3778)
* fix(skills): retain audio metadata file identity during generation

* test(skills): isolate audio metadata CLI credentials
2026-09-08 17:35:12 -04:00
James Russo a92dc6865e fix(skills): read BGM logs without pathname prechecks (#3769) 2026-09-08 16:40:20 -04:00
James Russo eec04340b3 fix(core): avoid grade stats regex backtracking (#3706) 2026-09-05 12:28:59 -04:00
James RussoandClaude Opus 5 8392e84a18 fix(media-use): repoint the dead videogen tier, demote past unusable local models (#3509)
* fix(media-use): repoint the dead videogen tier, demote past unusable models

`LOCAL_MODELS.videogen`'s `large` tier named `dgrauet/ltx-2.3-mlx-bf16`, which
returns HTTP 401 and cannot be downloaded at all. It was not a dormant entry:
`rankedByPreference` sorts by descending `needs.ramMB` when no `rank` is set,
so the largest fitting tier is tried FIRST by design. Any machine clearing
32 GB *available* RAM selected the dead entry, `ltxVideoGenerate` caught the
failure and returned a bare `null`, and since `ltx.local` is last in
`["heygen.video", "ltx.local"]` and network providers are skipped under
`--local-only` (`registry.mjs:206`), local video generation failed outright
instead of falling back to the tier that works.

It survived review because the table landed with "live verification on a 24GB
M-series Mac" - and a 24 GB machine cannot select a 32 GB tier, so that entry
was unreachable on the only machine that validated it. The unit fixtures
inherit the same ceiling (`fittingSpecs` is 20000MB), so every existing test
exercised the medium tier alone.

Two changes:

1. Repoint to `dgrauet/ltx-2.3-mlx-q8` (reachable) and correct `sizeMB` from
   45000 to 28800. Measured against the HF API: the q8 repo totals 87.5 GB,
   and the registry's own targeted `--include` subset is 28.76 GB. That
   matches the sibling q4 entry's convention (`sizeMB: 20000` vs a measured
   19.48 GB subset), so 45000 was wrong under either reading. `--low-ram` is
   added because the entry's own note calls it required at this tier's 32 GB
   floor, and the invoke omitted it.

2. A repoint alone is one bad URL from a repeat, so add the missing recovery.
   `selectModelLadder` returns every fitting model best-first;
   `selectModel`'s pick is now defined as that list's head. All three sites
   that previously selected exactly one model and failed terminally walk the
   ladder instead, demoting past a tier that cannot run here - gated weights,
   runner off PATH, an OOM at a tier that nominally fits:

   - `ltx-video-provider.mjs` (videogen, the reported failure)
   - `mflux-provider.mjs` (imagegen - same shape, and its 32 GB/64 GB tiers
     are equally unverifiable on a 24 GB machine)
   - `local-run.mjs` (tts/asr/upscale - `fish-speech` missing should still
     get you Kokoro)

   Every demotion is logged rather than silent, so a quietly smaller model is
   never mistaken for the tier the machine nominally qualified for.

Also fixes the `install` string both videogen entries share: it ended at
`uv sync --all-extras`, which leaves the entry point in `.venv/bin`, so the
"`ltx-2-mlx` not on PATH" hint named a command that following the instruction
would not put on PATH.

The q8 tier is NOT live-verified - no 32 GB+ Apple Silicon machine was
available - and its notes say so. Shipping it unverified is safe precisely
because of change 2: a wrong tier now costs one failed attempt, not the whole
local path.

- Rames Jusso

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(media-use): report the real videogen download size, disclose it, discard failed partials

Addresses review feedback on #3509 (CHANGES_REQUESTED at 90df164a), plus the
follow-on ask to tell the user what a download costs before they accept it.

1. `sizeMB` described a targeted `--include` subset that no run ever gets.
   Both videogen invokes pass a repo id to `--model`, and upstream
   `resolve_model_dir()` (`ltx_pipelines_mlx/utils/_orchestration.py:35-40`)
   calls `snapshot_download(repo)` with no `allow_patterns`, so the full repo
   lands regardless of what was pre-fetched. Corrected to measured repo
   totals: q8 87500 (87,511,991,375 B) and q4 59700 (59,686,429,583 B). q4 was
   wrong the same way at 20000, so both are fixed together rather than leaving
   one convention on each side.

   My earlier claim that 28800 "matches the sibling q4 entry's convention" was
   wrong in the way that matters: the convention itself described a subset the
   runner does not honor. The file's own comment already said "blind
   snapshot-downloads the lot (60 GB q4, 88 GB q8)" three lines above the
   fields that contradicted it, and the original report measured it too ("the
   q4 cache ended at 56 GB and q8 at 82 GB"), which reconciles exactly once
   read as GiB: 59.69 GB = 55.6 GiB, 87.51 GB = 81.5 GiB. So the download is
   the complete repo both times, not a partial fetch.

   Removed the `--include` recipe rather than repairing it: it is ineffective
   (the runner refetches at generate time) and insufficient (`--two-stage` is
   "dev model + CFG at half-res, upscale, distilled LoRA refine" per upstream's
   own help text, so it needs transformer-dev AND transformer-distilled AND
   spatial_upscaler_x2; `--distilled` needs an upscaler too). The q4 tier
   verified on a 24 GB Mac only worked BECAUSE the download is unfiltered.

2. Nothing told the user what they were agreeing to before a tool started
   pulling tens of GB. `describeDownload()` in `specs.mjs` names the size and
   the directory the weights land in, and checks free space with `statfs`
   against that directory rather than cwd, since the weights do not land in
   cwd. A tier that will not fit is still offered, with a plain statement that
   it will not fit: hiding it would make a machine that could free up space
   look like it has no large tier. Unknown free space reports as unknown, not
   as zero. Wired into both providers' install hints, the `runLocalModel`
   install payload (now carrying `sizeMB`), and `describeModelLadder`.

3. Each retry attempt mints its own timestamped temp path, so a partial
   artifact from a failed tier was orphaned rather than overwritten, and a
   lower tier then succeeding hid it. Both providers discard the partial before
   demoting, guarded so a file that cannot be removed never masks the generate
   failure it came from. Video is the material case: a partial mp4 is large.

   `local-run.mjs` is deliberately unchanged here. Its `out` is caller-provided
   and identical across attempts, so a partial is overwritten rather than
   orphaned, and unlinking a path the caller named would be a footgun. The rule
   the two providers follow is: clean up what you allocate.

Tests: 553/553 across `skills/**/*.test.mjs` (+15). New coverage pins the
cleanup (failed tier's partial removed, returned artifact survives, one discard
per attempt on the all-fail path, an unremovable partial still surfaces the
real failure) and the disclosure (cache-dir precedence, statfs walk-up to the
deepest existing ancestor, unknown-vs-zero, and the will-not-fit wording).
Every new guard mutation-tested: removing any one of them turns tests red.

- Rames Jusso

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 16:15:51 -07:00
Miguel Ángel f8a1e2d315 fix(skills): pin UTF-8 in Python scripts instead of the platform code page (#3298)
Windows sizes Python's stdio and text-mode file IO to the ANSI code page
(cp1252), not UTF-8. Every skill Python script relied on that default:

  * analyze-beatgrid.py --print writes the glyphs cp1252 has no slot for
    (delta, arrow), so the brief died with UnicodeEncodeError on every Windows
    run — the reported crash;
  * its audiomap write_text() pairs ensure_ascii=False with the default file
    encoding, so a non-ASCII payload is unwritable there too;
  * lint_source.py read_text() raises UnicodeDecodeError before any rule runs
    when a Remotion source carries an em dash or a curly quote;
  * gen-stroke-path.py reads an SVG font whose glyph keys ARE literal
    characters, so a mis-decoded key stops matching the requested text.

Stdio is reconfigured to UTF-8 at import and every text-mode IO call names its
encoding. `errors` is carried across the reconfigure: it resets to "strict",
and CPython gives stderr "backslashreplace" on purpose so the diagnostic path
can never itself raise.

extract-audio-data.py also decoded ffmpeg's stderr strictly while reporting a
failure, which would bury the very error being reported on a Windows ffmpeg.

skills/python-encoding.test.mjs guards the class: it fails if any skill Python
script drops the stdio block or omits encoding= on a text-mode IO call. The
mode is read as a whole comma-delimited argument of mode characters only, so a
payload key like {"bpm": 120} cannot spell the check away.

Verified with a cp1252 stdio stream installed before module load, matching how
Windows starts the interpreter: pre-fix UnicodeEncodeError, post-fix both
glyphs present in the UTF-8 bytes. Not run on real Windows hardware.
2026-08-17 21:44:08 -04:00
Miguel Ángel 6de29f5bea feat(media-use): derive provider cost tier from the registry (#3155)
The provider registry already declares whether a provider is local, free over
the network, or paid over the network via its A/N/P constructors, but nothing
downstream could read that, so anything needing to know whether resolving
through a provider can spend the user's credit had to re-derive it by
string-matching provider names.

Expose it as providerTierFor(name) over the same table and carry the derived
value on the resolve event alongside the provider it came from. Sparse: absent
when the record carries no provider or the name is not declared. A name declared
under two media types must carry one tier; the index throws at import rather
than silently picking one.

Covered by unit tests on the lookup and by end-to-end cases that spawn the real
CLI and read the value off the payload a local server receives, one per tier.
2026-08-10 13:35:25 -04:00
ukimsanov 08fadcef41 style: format the house-narrator note in tts.md, resync manifest
Preflight failed on skills/media-use/audio/references/tts.md — the section I added
was not oxfmt-clean. Formatted, and regenerated skills-manifest.json since the
media-use hash changed.

The other files oxfmt flags (package.json files, several skill .md files,
studio/parsers sources) are not touched by this branch and fail on main too — a
pre-existing whole-repo format debt, not introduced here.
2026-08-04 16:33:57 -07:00
ukimsanov 618f73c266 Merge remote-tracking branch 'origin/main' into docs/pages-show-not-tell
# Conflicts:
#	skills-manifest.json
2026-08-04 16:16:58 -07:00
ukimsanov bb7b0c899f docs: write down the house narrator, and stop the videos sounding like two products
Every user-journey film on the docs site is narrated by ElevenLabs River
(SAz9YHcvj6GT2YYXdXww) at 145-155 wpm with music about -31 LUFS under it. That
was recorded in one launch project's notes and nowhere an agent would look.

So when I briefed six new docs videos I asked for "a music bed plus SFX" and said
nothing about voice. Two fell back to local Kokoro (am_michael, bm_george) and one
used an unspecified ElevenLabs take. Three films, three narrators, none of them
the one the rest of the site uses. Being re-voiced now.

The rule is in skills/media-use/audio/references/tts.md, next to the provider
table an agent already reads before generating a voiceover, including the reason:
falling back to a local voice because a key was not to hand produces a film that
sounds wrong beside the others. If ElevenLabs cannot be reached, say so and stop
rather than substituting.

Also on this branch: the superseded Huly film is gone from the product-launch
page, and three pages that ended up with two hero videos stacked now lead with
one. Where the older clip still showed something different — a finished motion
graphic, the same edits done in Studio — it moved below under its own heading
instead of being deleted.
2026-08-04 13:16:24 -07:00
Vance IngallsandClaude Opus 5 255cf92915 fix(skills,producer): terminate ffprobe options in shipped skill scripts
The contract test only walked packages/*/src and only .ts, so it could not see
the shipped agent tools under skills/**, which are .mjs/.cjs. 19 call sites
there and in package tests were still missing `--` immediately before the
input while the suite reported the bug class closed — a dash-prefixed filename
is parsed as an option and fails the same way.

Sweeps packages/, skills/ and scripts/ now, including .mjs/.cjs and test
files (dither.test.mjs was one of the broken sites). Excludes only the
contract test itself, which documents the contract with example argvs
including a deliberately misordered one.

Two guards were fixed while widening: the terminator must never be inserted
after `-i`, which consumes the next token (a blind pass hit an ffmpeg input
and a base64 -i), and comment prose describing a spawn is not a spawn.

Also routes every audioPadTrim probe failure through one sanitizer at the
boundary. runFfprobeJson scrubbed its own stderr, but
defaultProbeVideoFrameInfo threw `no video stream in ${videoPath}` raw into
the public PadTrimAudioResult.error, and an injected probe can throw anything.
The redaction unit tests all passed with the caller wiring deleted; the new
public-path regressions fail without it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 02:23:35 -07:00
Peter YangandJames 30900c3465 docs(audio): avoid weak music openings in short launch videos (#2882)
* docs(audio): check music energy against final cut

* chore(skills): regenerate skills manifest

---------

Co-authored-by: James <james.russo@heygen.com>
2026-07-29 15:26:43 -07:00