Commit Graph
17 Commits
Author SHA1 Message Date
Miguel Ángel 97df93fb07 fix(cli): check hears a WAV whose header misstates its byte rate, as render plays it (#7618)
* fix(cli): check hears a WAV whose header misstates its byte rate, as render plays it

A PCM WAV whose header byte rate is wrong (some speech generators write 2x) sent the silence decode's input seek past the file's end, so every clip past the middle was reported audio_clip_silent. Render already cuts such a file by decoded time (#7541); the byte-rate rule now lives in parsers and the CLI decoder seeks after decoding starts for those files.

* test(cli): keep the misstated byte-rate case with the other single-clip checks

* style(cli): format the WAV test helper

* style(parsers): keep the byte-rate rule's comment within the line limit
2026-11-12 19:30:02 +00:00
Michael WangandClaude Opus 5.5 84c7d02a19 fix(cli): keep project folders from choosing the programs and config the CLI runs (#7511)
* fix(cli): resolve helper programs from PATH, never the working directory

On Windows a bare command name is looked up in the working directory before
PATH. The CLI now sets NoDefaultCurrentDirectoryInExePath at startup, which
Node's spawn and cmd.exe both honor and child processes inherit, so tools
such as powershell.exe, taskkill and npx.cmd always come from PATH.

The shared ffmpeg/ffprobe resolver no longer scans the working directory on
Windows and skips relative PATH entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): keep Python helpers from importing files in the working directory

`python -c` and `python -m` put the working directory first on sys.path, so
a file in the project named like a probed module ran instead of the
installed package. Inline helper code now drops that entry before any
import, and `-m` helpers (pip) run through runpy after the same step. Paths
and arguments are unchanged.

On Windows, findPython also skips a `where` match in the working directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): run the skills installer and nested CLI calls independent of the project

The global skills install ran `npx skills@<pin>` in the current folder, so
npm could take the package or registry from that project's node_modules or
.npmrc. It now runs from the home folder; the install is global either way.

When the CLI starts the media-use engine it hands over its own entry file.
The local voice fallback and the audio helpers then run that same CLI with
the same Node instead of resolving `npx hyperframes` from the project.
Without a handed-over entry they behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): load only provider keys from a project .env

The audio scripts copied every key of a project .env into their environment
except two. They now take the same kind of short allowlist the CLI uses:
HeyGen, ElevenLabs and Gemini API keys plus a few harmless settings. Program
overrides, runtime options, credential files and the credential store path
come only from the person's own environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): build whisper.cpp only from a pinned, verified release

The source-build fallback cloned the upstream default branch. It now clones
the v1.9.5 release tag and refuses to build unless the checkout is the
reviewed commit for that tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): keep registry caches apart and refuse config install targets

Registry cache file names now include a digest of the exact registry URL,
and each entry records the URL it came from, so one registry's cached
manifest is never read for another. Entries from older versions are
refetched once.

Registry installs refuse targets inside .git or any hidden directory, hidden
configuration files, and package.json, hyperframes.json and
hyperframes.lock.json. An example installed by `init` may still bring its
own package.json and .gitignore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(capture): keep page text to one line in agent files and validate Vertex keys

Capture writes AGENTS.md, CLAUDE.md and .cursorrules from the page's title,
font names and colours. Page text is now flattened to one short line and
colours must be hex values. Existing agent files that capture did not write
are kept, with a warning.

HYPERFRAMES_VERTEX_SERVICE_ACCOUNT must now be a plain service-account key
with a Google token endpoint; other credential types skip captioning with a
warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(cli): find the digest-named registry cache and trim a new file's comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): match registry targets as Windows names them; skills install without HOME

Windows drops trailing dots and spaces from a path segment and reads name:stream as
name, so package.json. or .git./hooks reached the files the target rules refuse.
The global skills install now falls back to the temp folder when HOME does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): turn off the Windows working-directory lookup in every program entry

The warm-check server and client are their own tsup entries and copied the
environment before cli.js set NoDefaultCurrentDirectoryInExePath, so the server
spawned a bare powershell.exe that Windows could take from the project folder.
Every entry that hides child consoles now also loads windowsExeSearch first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): refuse Windows short-name registry targets

A segment like CLAUDE~1 or PACKAG~1.JSO opens the long name on NTFS, so it
reached the files the target rules refuse. The trailing-dot and stream-name
normalization is removed: publication already rejects those segments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 20:01:19 +00:00
Miguel Ángel c149730819 fix(cli): no macOS developer-tools prompt from git, xcrun or cc checks on a Mac without the tools (#7257)
* fix(cli): no macOS developer-tools prompt from git, xcrun or cc checks on a Mac without the tools

* fix(cli): treat Xcode.app as installed tools and only take executables from PATH

* test(cli): give skills tests a developer folder so darwin cases run on any host

* fix(cli): read macOS tool paths the same way on every host
2026-11-09 17:01:38 +00:00
JerraiandXuanru Li acd5c30e60 feat(cli): read audio stems through your HeyGen account (#6993)
* feat(cli): audio --read organises the reading by stem

Measures where each stem is heard, leaves the drums' recurring hits to the
music while it plays, names the stem of each remaining effect, writes a
voice-free mix next to the stems, and describes each audible stem and the
whole recording with Gemini.

* feat(cli): audio --read reads stems as vocals, drums, bass and melodic

Piano, guitar and other are mixed into melodic.wav next to the stems; levels,
spans, descriptions, effects and pieces are read per group, and a group heard
for under two seconds or 5% of the recording is left out of the reading.

* feat(cli): audio --read describes the six stems and writes a BGM prompt from them

Stems are read one by one again; a stem heard under two seconds or 5% of the
recording is still left out. Each stem's description gives the times of its
changes and names the effects it hears. The overall and the BGM prompt are
written from those descriptions and the measured spans, not by listening to
the mix again; the BGM prompt orders the arrangement without seconds.

* fix(cli): audio --read writes no BGM prompt for a recording without music

Foley and effects in a rhythm are not music; the reading says so instead of a
prompt that would make a remix generate a track the reference never had.

* fix(cli): audio --read prompts name no example sounds

* fix(cli): describe audio through HeyGen and keep effects measured

* docs(cli): document measured effect events without sound kinds

* fix(cli): clear audio description review checks

Use portable stem path assertions and simplify response validation.
Keep auth-derived URLs out of cache hashes and trim redundant comments.

— Jerrai

---------

Co-authored-by: Xuanru Li <157947275+xuanruli@users.noreply.github.com>
2026-11-02 18:42:05 +00:00
Miguel Ángel de6ec3811e fix: cache paths follow HYPERFRAMES_CACHE_DIR (#6716)
* fix: cache paths follow HYPERFRAMES_CACHE_DIR

Every place HyperFrames keeps downloaded or built files now takes its
folder from one helper, hyperframesCacheDir() in @hyperframes/core. The
default stays ~/.cache/hyperframes; HYPERFRAMES_CACHE_DIR moves it.

* fix: register the cache-dir subpath for bun and node, and build inspector from core source

* test: the cache-dir override test uses a folder that is absolute on every platform

* fix: the opencv cache follows HYPERFRAMES_CACHE_DIR

* fix(cli): npm installs run in their staging dir, so no cache path reaches a command line

* fix(cli): whisper.cpp is cloned inside its build dir, so no cache path is a git argument

* test(cli): the whisper clone test finds no system whisper on Windows either
2026-11-01 22:44:11 +00:00
Jerrai 4fa5f54519 fix(cli): audio windows read beats off the whole clip; beats snap to the signal and mark silence (#6692)
* fix(cli): audio windows read beats off the whole clip; beats snap to the signal and mark silence

A film query that covers part of a clip measures beats, downbeats, bpm and sections on the
whole clip and cuts them to the window, as #6653 did for kind, so a window under 6 s still
has its beats.

One step after every provider ladder moves a provider's beats and downbeats onto the
signal's own beats within a quarter beat (the snapping api did alone) and marks each with
`silent`, true where the beat falls in silence. Beats are marked, never dropped; the JSON
change is the additive `silent` array.

— Jerrai

* fix(cli): beats with no pulse are unheard; a window with none answers no steady beat

Review fixes for #6692:
- The per-beat flag is now `unheard` (renamed from `silent`, not released): true in silence
  or where no onset falls within 50 ms of the beat, so hiss, room tone or a held pad no
  longer pass as beats.
- A window keeps the whole clip's beats only when at least two of its beats are heard;
  otherwise it answers "no steady beat" as before, with no downbeats and no bpm.
- A window's bpm is the tempo of its heard beats, never the whole clip's.
- Comment lines trimmed so analyze.ts, run.ts and api.ts are at or under main's share.

— Jerrai

* fix(cli): a beat is unheard only where two beats in a row have no onset

Scored on the ground-truth set, flagging every onset-less beat also flagged heard beats the
onset detector misses: the drop downbeat after a snare roll, single beats under narration,
quiet beats late in a fade. A pulse gap now needs two onset-less beats in a row; hiss, room
tone and held pads span many beats and stay flagged, and silence is flagged as before.

— Jerrai

* fix(cli): a window with no pulse on the clip's grid reads its own beats, as before

Genesis found a window over a section far under the clip's loudest part answered "no steady
beat" where main, reading the window alone, found every beat: the clip's grid and onsets
skip a section that quiet (from about 40 dB under, for clicks). When a window keeps fewer than
two heard beats from the clip's grid, its beats, downbeats and bpm now come from the window
alone, as they did on main. Hiss still answers "no steady beat" on its own.

The hiss test's noise() repeats every 10,466 samples (0.22 s at 48 kHz), which the window's
own reading took for a pulse; that test now uses a xorshift generator.

— Jerrai

* fix(audio): unheard comes from level-free rhythm evidence, so quiet music stays heard

A beat was flagged by an absolute -50 dBFS silence and by onsets under 10% of the clip's
loudest hit, so music 30 dB or more under the loudest part read as unheard. Now:
- core localBeatEvidence reads the cached standardised onset curve over the 6 s around each
  beat: the tracker's steady-beat confidence there (clarity at the local beat lag times the
  beat's own lift) and the beat's lift over the span's mean.
- A beat is unheard in digital silence (-80 dBFS), or when its span keeps no steady beat
  (confidence under 0.15) and its own onset does not stand out (lift under 1.5). Lift alone
  keeps beats under narration heard, where syllables lower the span's clarity.
- A window keeps the clip's beats when at least two, and at least half, of them are heard.

— Jerrai
2026-10-29 01:52:48 +00:00
Xuanru LiandJerrai 1df31d8717 fix(audio): a sound is voice only with a voice-range pitch, judged on the whole clip (#6653)
* fix(audio): a sound is voice only with a voice-range pitch, judged on the whole clip

Two ways the kind guess called a non-voice sound voice:

- A short hit, click or whoosh can have a speech-shaped envelope. classify() now also
  measures the share of 40 ms frames with a 70-500 Hz fundamental (YIN, moved from the
  CLI into core) and calls a sound voice only when that share reaches 0.15. Speech
  measured 0.22-0.43; impacts, clicks and whooshes 0.
- A range query judged a clip's kind on the slice it covered, so 1.7 s of a music bed
  read as voice. A film clip's kind is now judged on all of the clip that plays.

Even-paced speech (a 64 s TTS take) moves less at syllable rate and was called music;
with no beat and a voiced share of 0.25 or more it is now voice.

* chore(audio): keep each touched file's comment share at or under its base

The Comments check failed on run.ts, targets.ts and audioAnalysis.ts because the
PR added comment lines to files whose share may only go down. Drops the added
doc comments; no code change.

— Jerrai

* fix(audio): preserve clip evidence and stereo pitch during analysis

* fix(audio): bound and cache whole-clip kind analysis

* fix(audio): retain native-rate kind evidence when available

---------

Co-authored-by: Jerrai <jerrai@heygen.com>
2026-10-28 12:48:08 +00:00
Miguel Ángel cfef528d3a fix: phone videos with an undecodable first audio track check, preview and render (#6551)
* fix: phone videos with an undecodable first audio track check, preview and render

* fix(engine): a rate-lane clip bakes its ramp from the decodable audio track

* test: phone-clip tests end ffprobe options before the input
2026-10-27 13:09:02 +00:00
Akshar Patel ec866ed1c5 fix(cli): keep audio timings when trimming long recordings (#6375) 2026-10-26 14:52:41 +00:00
Miguel Ángel 66f55ffb9b fix(cli): audio describe builds its Apple helper with a module cache an agent's sandbox can write (#6102)
* fix(cli): audio describe builds its Apple helper with a module cache an agent's sandbox can write

* chore(cli): shorten the module cache comment
2026-10-25 23:05:01 +00:00
Xuanru Li 2c1d9d50d2 feat(cli): read a whole recording as voice, effect and music lines (#6056)
* feat(cli): read a whole recording as voice, effect and music lines

`hyperframes audio <file> --read` turns an episode or reference video into
time-stamped lines an agent can read or grep: sentences with rate, pitch and
ending; pauses; sound effects grouped by identical sound, with band, attack,
tail and level over the music; and the music bed split at its cut-outs into
pieces that start, resume, return or are new.

Effects and music come from stems separated by the service at
HYPERFRAMES_AUDIO_STEMS_URL (vocals, drums, bass, guitar, piano, other).
Without stems, effects are read from the mix outside speech; with neither
stems nor a transcript, voice and effects are not read, and each case is
reported in a warning. Results are cached per file content.

* refactor(cli): name the service error parse instead of commenting it

* fix(cli): call HeyGen's audio services with the CLI's HeyGen credential

Stems for `audio --read` and the `api` analysis provider only ran when
HYPERFRAMES_AUDIO_STEMS_URL / HYPERFRAMES_AUDIO_API_URL and
HYPERFRAMES_AUDIO_API_KEY were set, so someone with HEYGEN_API_KEY or a saved
`hyperframes auth login --api-key` still got "not set" and had to know the
internal route.

Both now default to /v3/audio/stems and /v3/audio/analysis on the HeyGen API
(HEYGEN_API_URL when set) with the credential the other HeyGen commands use.
The URL variables still point at another service, which then needs its own
HYPERFRAMES_AUDIO_API_KEY, so the HeyGen credential never goes to a host it was
not issued for.

* fix(cli): keep audio on the machine unless --quality best is asked for

`hyperframes audio` defaulted to --quality best, so anyone with a HeyGen
credential uploaded their audio, and could be billed, for an ordinary beats or
--read run. tts, transcribe and remove-background never leave the machine
unasked.

The default is now local for both measuring and --read. best is unchanged and
still uploads to /v3/audio/analysis and /v3/audio/stems. When a capability goes
unanswered because a service was left out, or --read reads without stems, the
warning says --quality best would add it.

* fix(cli): bound stem transfers and preserve recording timestamps

* fix(cli): an empty audio service URL keeps the service off

HyperFrames Desktop blanks HYPERFRAMES_AUDIO_API_URL and its key while
the analysis route is behind a flag, but gives runs HEYGEN_API_KEY and
HEYGEN_API_URL for its gateway. Falling back to that credential would let a
run that passes --quality best reach the route anyway. An empty URL now turns
the service off, whatever credential is present.

* fix(ci): capture package name in release-age exclusion

(cherry picked from commit f9c6ba3573)

* fix(cli): resend the stems request with its key until the service answers

The stems service answers once the stems are made, which can outlast the
connection, and answers 409 request_in_progress to a resend while the first
request runs. A failed job's reason is failure_message.

* fix(cli): retry incomplete stem separations

* test: validate audio mock route handlers before dispatch

* test: trace premature Windows CLI runner exits

* test: remove temporary Windows runner diagnostics
2026-10-22 13:22:38 +00:00
Miguel Ángel ab2e939558 fix(cli): audio and timeline find media the way preview and render load it (#5846)
audio and timeline joined a clip's src to its composition's folder by hand,
so a project-root asset used from a nested scene, or a percent-encoded file
name, was "not found" although preview, check and render load it.
resolveAuthoredAssetFile now owns that lookup (the sub-composition rewrite,
then the decoded project-root resolve), and audio, timeline and
media-treatment all use it.
2026-10-18 15:01:14 +00:00
Miguel Ángel eab6ca15b1 fix(cli): timeline lists clips nested in a timed element and lint flags ones that never show (#5732)
* fix(cli): timeline lists clips nested in a timed element and lint flags ones that never show

* docs(lint): wrap the nested-clip entries

* docs(skills): format the nested-clip note and refresh the skills manifest

* fix(cli): keep timeline edits from colliding with a clip's own wrapper

* fix(cli): cut clips nested in a wrapper at their sub-composition's in-point

* refactor(lint): split the nested-clip visibility checks to keep the audit green

* style(cli): restore the timeline test's formatting
2026-10-17 02:27:17 +00:00
Xuanru Li 34776f3def fix(cli): analyze finalized audio assets after direct upload (#5647) 2026-10-15 09:23:10 +00:00
Xuanru Li 124492e427 feat(cli): the api provider speaks the HeyGen audio analysis contract (#5641)
* feat(cli): the api provider speaks the HeyGen audio analysis contract

The api rung now sends its key as x-api-key, a new Idempotency-Key on each
request, and 16 kHz mono 16-bit WAV: the channels are averaged and resampled
with a windowed sinc whose cutoff sits under 8 kHz, so nothing folds back in.
It reads the answer from a { "data": ... } envelope, or bare. Times stay in
seconds, and the answered times still snap onto beats measured on the signal
as decoded. The response type gains time_resolution_seconds.

* fix(cli): reject redirects from audio analysis requests

* fix(cli): preserve audio when stereo downmix cancels
2026-10-15 06:54:29 +00:00
Xuanru Li 8e340d220e feat(cli): audio answers beats, downbeats, bpm, sections and describe through a provider ladder (#5632)
* feat(cli): audio answers beats, downbeats, bpm, sections and describe through a provider ladder

Beats and downbeats try api, apple-mu, beat-this, then signal; sections try api, apple-mu,
then signal; describe tries api, then apple-sound. bpm is read off whichever provider placed
the beats. Each provider says whether it can run here and why not, and the ladder warns
when it passes one over.

The signal rung is a TypeScript tracker: log-band spectral flux, autocorrelation tempo with
a log-normal prior, Ellis dynamic-programming beats with a windowed local tempo, a low-band
bar phase, and Foote novelty sections snapped to beats. It reports no steady beat when the
onsets do not repeat.

The api rung posts the audio to HYPERFRAMES_AUDIO_API_URL with HYPERFRAMES_AUDIO_API_KEY and
moves each answered time onto the nearest signal beat within a quarter beat, reporting what
moved and what did not. beat-this runs Beat This! from a configured Python and local weights
and never downloads. apple-sound runs macOS's sound classifier through a Swift helper.
apple-mu reports that Music Understanding needs macOS 27.

words are no longer accepted: they come from hyperframes transcribe.

* docs(cli): audio's beats, downbeats, bpm, sections, describe and their provider ladder

* fix(cli): remove redundant audio context comment

* fix(core): keep flat audio novelty in one section

* fix(cli): honor BPM provider pins before analysis
2026-10-15 05:16:55 +00:00
Xuanru Li bb49858b8a feat(cli): hyperframes audio measures where sounds land, in film time (#5602)
* feat(cli): hyperframes audio measures where sounds land, in film time

Adds @hyperframes/core/audio-analysis: an energy envelope, spectral-flux
and energy onsets, the moments a sound lands (onset, steepest rise, peak,
tail end), BS.1770 loudness with true peak, silence spans and a first
guess at sfx/voice/music, all pure functions of PCM. Each capability runs
through a provider ladder so model or service providers can join later.
The radix-2 FFT moves out of audioCarve so both share it.

`hyperframes audio` decodes with FFmpeg and analyzes files, or the film's
own audio and video clips by element, second, range or boundary, reading
only the part of each file that plays. Timeline rows now carry
`mediaStart`, moved by an in-point host's cut, so file time maps to film
time.

* feat(cli): audio placements carry the element's own data-start, local to its composition

* feat(cli): audio placements carry the element's own data-start, local to its composition

* feat(cli): audio places a file with --place-at, drops --boundary

--start becomes --place-at and --media-start becomes --skip-file-seconds,
which only goes with --place-at, refuses to skip the whole file, and warns
when the skipped part held the file's loudest moment. --boundary is gone:
--at and --range cover the sounds a cut touches.

* fix(cli): pass a sound file to ffprobe as a file, on every platform

ffprobe treated a name starting with a dash as an option because the argument
list never ended the flags. The path check compared a Windows drive path to a
POSIX string, and a new timeline comment raised that file's comment share.

* fix(cli): hear a reversed stereo channel, and warn when its peak was skipped

Averaging the channels cancelled a reversed one, so a real hit was reported
as silence. The skip warning also compared against the decode margin, so a
peak just before the kept window produced no warning.
2026-10-15 02:32:10 +00:00