* feat(cli): audio answers beats, downbeats, bpm, sections and describe through a provider ladder
Beats and downbeats try api, apple-mu, beat-this, then signal; sections try api, apple-mu,
then signal; describe tries api, then apple-sound. bpm is read off whichever provider placed
the beats. Each provider says whether it can run here and why not, and the ladder warns
when it passes one over.
The signal rung is a TypeScript tracker: log-band spectral flux, autocorrelation tempo with
a log-normal prior, Ellis dynamic-programming beats with a windowed local tempo, a low-band
bar phase, and Foote novelty sections snapped to beats. It reports no steady beat when the
onsets do not repeat.
The api rung posts the audio to HYPERFRAMES_AUDIO_API_URL with HYPERFRAMES_AUDIO_API_KEY and
moves each answered time onto the nearest signal beat within a quarter beat, reporting what
moved and what did not. beat-this runs Beat This! from a configured Python and local weights
and never downloads. apple-sound runs macOS's sound classifier through a Swift helper.
apple-mu reports that Music Understanding needs macOS 27.
words are no longer accepted: they come from hyperframes transcribe.
* docs(cli): audio's beats, downbeats, bpm, sections, describe and their provider ladder
* fix(cli): remove redundant audio context comment
* fix(core): keep flat audio novelty in one section
* fix(cli): honor BPM provider pins before analysis
* fix(cli): retire old snapshot run folders
* chore: satisfy snapshot comment limit
* fix(history): read and release snapshot leases through verified handles
* test(history): run lease races in Node and check released markers
* fix(studio): verify snapshot leases from open descriptors
* feat(cli): hyperframes audio measures where sounds land, in film time
Adds @hyperframes/core/audio-analysis: an energy envelope, spectral-flux
and energy onsets, the moments a sound lands (onset, steepest rise, peak,
tail end), BS.1770 loudness with true peak, silence spans and a first
guess at sfx/voice/music, all pure functions of PCM. Each capability runs
through a provider ladder so model or service providers can join later.
The radix-2 FFT moves out of audioCarve so both share it.
`hyperframes audio` decodes with FFmpeg and analyzes files, or the film's
own audio and video clips by element, second, range or boundary, reading
only the part of each file that plays. Timeline rows now carry
`mediaStart`, moved by an in-point host's cut, so file time maps to film
time.
* feat(cli): audio placements carry the element's own data-start, local to its composition
* feat(cli): audio placements carry the element's own data-start, local to its composition
* feat(cli): audio places a file with --place-at, drops --boundary
--start becomes --place-at and --media-start becomes --skip-file-seconds,
which only goes with --place-at, refuses to skip the whole file, and warns
when the skipped part held the file's loudest moment. --boundary is gone:
--at and --range cover the sounds a cut touches.
* fix(cli): pass a sound file to ffprobe as a file, on every platform
ffprobe treated a name starting with a dash as an option because the argument
list never ended the flags. The path check compared a Windows drive path to a
POSIX string, and a new timeline comment raised that file's comment share.
* fix(cli): hear a reversed stereo channel, and warn when its peak was skipped
Averaging the channels cancelled a reversed one, so a real hit was reported
as silence. The skip warning also compared against the decode margin, so a
peak just before the kept window produced no warning.
* fix(cli): report spend usage for Claude plans without rate windows
Spend-metered Claude plans (for example Enterprise) return null five_hour
and seven_day windows from the OAuth usage endpoint and put their allowance
in spend / extra_usage instead. The parser rejected every such response as
invalid_usage_response, so `hyperframes usage` always reported unknown.
- Read spend.percent (falling back to extra_usage.utilization) into a new
`spend` window on known results; over-limit spend caps at fully used.
- Report `no_usage_windows` instead of `invalid_usage_response` when a
well-formed response simply has no window.
- Document the `spend` field in the hyperframes-cli skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): satisfy usage budget comment ratchet
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* chore: release v0.8.159
* chore: v0.8.159 notes list the first-frame fix
* chore: v0.8.159 notes list the sandboxed GPU check fix
* chore: v0.8.159 notes list the hidden-scene fix and the offline font test
* feat(cli): snapshot --keep redoes one board still and leaves the others
* fix(skills): the board recipe waits for a released snapshot --keep
* fix(cli): snapshot --keep refuses two times with one name and replaces linked stills
init's transcodeToMp4 piped ffmpeg's stdout and stderr and never read them.
ffmpeg's progress log filled the pipe buffer and ffmpeg blocked in write(),
so init hung with its spinner running. Both streams are now ignored.
render --docker --quiet piped the container's stdout without reading it; it
is now ignored. stderr is still read and kept as a bounded tail.
* feat(cli): init --keep-original copies a clip as is, without transcoding
`hyperframes init --keep-original` copies the --video file under its own
name and wires it into the project the way a web-safe clip is wired today:
no transcode and no prompt, interactive or not. Default behaviour is unchanged.
Theora leaves init's web-safe codecs and joins the preview proxy's
browser-hostile codecs, so init converts it and the preview proxy serves an
H.264 copy instead of a blank picture.
* fix(studio-server): keep the codec table's comment share at its base level
* fix(inspector): report nested hosts' authored window, one location per source line
The runtime strips data-duration/data-end from nested composition hosts after
boot and keeps them as data-hf-authored-*. Timeline inspection built its
resolver without authored attrs, so a host with data-duration="4" reported
its GSAP timeline length (3.72s) instead.
Timeline text and events[].sources listed the same file:line once per mapped
snippet (creation, caller, dependency), so a one-line script printed one
location up to nine times.
* refactor(inspector): carry the dedupe reason in the helper's name
* fix(inspector): end continuation on a page that fits no snippet; fill budget with candidates
* fix(inspector): timeline groups by component, names attr values, eases, film end
* fix(studio-server): stamp hf ids into a file without reformatting it
* fix(inspector): find an element's tag before its served hf id reaches the source
* fix(inspector): live element map shows authored classes and text, not the playhead's
* fix(inspector): collapse script-built text so the element map ignores the playhead
Class names stay in source order, a runtime-stripped id is shown as id="…"
rather than a selector, and a grouped row names which children it covers.
* fix(inspector): keep paging, stamping, and the live map still on the reported edges
A snippet larger than the page ended the walk, so later snippets could not
be reached. Raw text ended at a longer tag name, which could stamp an id
inside a script. An empty or mixed source element showed live text, and an
unknown duration read as a film that ends at 0s.
* fix(inspector): mint ids for an empty hf id and keep an unquoted class
An empty data-hf-id was treated as already assigned, so stamping left it
blank while the older stamper minted one. An unquoted class was read as no
class, so the live map dropped it.
* fix(inspector): narrow markup before reading a child tag
skipMarkup returns a boolean, so the build still sees Markup where a tag
is required. Narrow on the kind first so the inspector typechecks.
* fix: quiet check false positives and name the ancestor of an id-less target
Endpoint contact, disjoint fromTo pairs, and a negative z-index under a
stacking-context parent were warnings with bad advice. Layout and contrast
blamed a dot grid, a circular avatar's panel, and a visibility override.
A faint texture veil is no longer the click target.
* fix(lint): drop a history sentence and six gsap comments
The comments check treats "load-bearing" as history, and gsap.ts added six
comment lines. The names already carry what those lines said.
* fix(parsers): ignore an empty id written inside another attribute
The empty-id scan matched the first data-hf-id="" in the tag, including one
inside a quoted title. It now skips quoted text, so the real attribute is minted.
* fix(lint): keep real covers and a negative z-index under a card
A gradient scale-down treated every fill like a radial dot, so a panel
that starts transparent stopped counting as a cover. A stacking context
on any ancestor cleared a negative z-index, and a faint wrapper made
images inside it unpickable.
* fix: keep an emoji from shifting a stamped id
The quote mask replaced one emoji with one space, so the id landed before
the closing quote. A radial cover that turns opaque after a clear stop
stays fully painted.
* fix(lint): score a letter-only dot as a dot, not a color name
namesAColor saw #fff and rgba as words, so those dot grids counted as a
full cover. Hex and rgb colors are stripped before that word check.
* chore(docker): pin FFmpeg 8.0.1 in the Docker images and host CI
Install one pinned static BtbN build (n8.0.1-66, month-end release) in
Dockerfile.test, the CLI render image, the Cloud Run image and the k8s
example, and bump the host CI install-ffmpeg-linux action from 7.1.3 to
the same build. Regression baselines and the shipped images now use the
same FFmpeg version instead of Debian's apt ffmpeg 5.1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(producer): regenerate six regression baselines on FFmpeg 8.0.1
The Docker test image now renders with FFmpeg 8.0.1 instead of Debian's
5.1. Six fixtures fail against the 5.1 baselines; everything else passes
unchanged, including vfr-screen-recording.
- style-2-prod, style-10-prod: FFmpeg 8 honors the MP3 gapless header,
so the probed SFX duration is 1.0s instead of 1.044898s. Regenerate
output.mp4 and compiled.html.
- vignelli-stacking: one checkpoint (8.06s) now shows source frame 201
of the 25fps a-roll, which is the right frame for that time; the old
baseline showed frame 202.
- missing-host-comp-id, variables-prod, speed-ramp: AAC encoder output
differs enough to drop the RMS-envelope correlation on flat signals
below 0.9. Sample correlation against the source stays at 0.9999.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): read the ffmpeg version from n-prefixed release builds
Release-branch ffmpeg builds, including the BtbN build now pinned in CI,
print "ffmpeg version n8.0.1-66-...". extractMajorVersion skipped the
"n8" token and returned 0 (and 1 for n7.1.3), so the CLI telemetry
smoke check failed. parseToolVersion fell back to the whole banner line.
Accept an optional "n" before the version number in both.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): keep the preflight comment share flat
Fold the n-prefix note into the existing doc line instead of adding new
comment lines, which the comment ratchet rejects.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(docker): pick the FFmpeg build per arch and fail on a bad binary
Dockerfile.test, the k8s example and the Cloud Run image always extracted the
amd64 archive on multiarch bases, and the piped version check passed even when
the binary could not run. All four images now pick linux64 or linuxarm64 by
TARGETARCH, fail on any other arch, and grep both ffmpeg and ffprobe for the
pinned version.
style-2-prod and style-10-prod compiled.html go back to the previous golden
with only the SFX data-duration/data-end changed, instead of a full
recompile that also picked up unrelated compiler output.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(engine): say why WSL renders fall back to software and how to get the GPU
Under WSL the GPU check often falls back to SwiftShader and the only output
was the probe line, with Docker passthrough advice for --browser-gpu. Print
one note that says what was checked (can this user open the render node),
the exact fix, and the native Windows alternative. The CLI's WSL detector
moves to the engine so both use one.
Part of #5548
* fix(engine): keep WSL GPU notes true to the probe result and render mode
The WSL note now prints only when auto mode really selects software, and
says whether the GPU check found no GPU or could not finish. Explicit
--browser-gpu keeps its unverified and Chrome-install advice and adds the
WSL checks. The render-node check verifies group membership before
suggesting usermod.
* feat(studio-server): accept a crf override on the render route
The Studio render route only forwarded draft/standard/high, so a Studio or
desktop export could not ask for a specific CRF the way `hyperframes render
--crf` can. Film-grain finishing passes need CRF 12 to keep the grain.
The route now takes an optional integer `crf` (0-51 for mp4, 0-63 for webm),
rejects an out-of-range value with 400 instead of dropping it, ignores it for
mov, and both Studio adapters pass it to the render job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(studio-server): keep the crf change free of new comment lines
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(studio-server): accept videoBitrate, videoFrameFormat and hdrMode on the render route
Same options `hyperframes render` already takes (--video-bitrate,
--video-frame-format, --sdr/--hdr), so Studio callers are not limited to
the quality presets. crf and videoBitrate are an either/or, matching the CLI;
an invalid value is a 400. Parsing moves to helpers/encoderOptions.ts.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore: release v0.8.154
* ci: give the studio test job fifteen minutes
The queue run installs and builds inside the same ten minute cap.
It was still inside the studio tests when that cap cancelled the job.
* fix(cli): bound checks of non-finishing compositions
* fix(cli): account for cold startup in deadline witness
* fix(cli): wait for profile owners before test cleanup
* test(cli): separate portable deadline and Linux process witnesses
* test(cli): preserve per-launch browser profile ownership
* fix(cli): start check deadline after browser readiness
* chore: release v0.8.152
* docs(changelog): correct the v0.8.152 release notes
The summary overstated snapshot retention, history turns, and video state.
It also left out the new artifact paths and nullable keyframe times.
* feat(cli): save check, snapshot and render artifacts per revision
* fix(cli): keep clean --snapshots inside the project
A symlinked .hyperframes or artifacts folder was followed, so the command
deleted snapshot and check files outside the project.
* test(cli): keep check, snapshot, and render successful when recording fails
A held history or a corrupt output registry warns and leaves the command result alone.
* fix(cli): drop snapshot records when a replaced run is not recorded
A reused --output rewrites the files even when history is busy.
Those records are removed so the old revision does not keep the new pixels.
The preview checkpoint stops waiting after five seconds.
A bad PNG header no longer fails the snapshot.
* fix(studio-server): keep the video-state comment share from rising
The forget helper's doc comment pushed the file above its base share.
* fix(cli): post the preview checkpoint and keep cleanup inside artifacts
A preview checkpoint with no body was sent as GET, so Studio never ran it.
Cleanup now requires the real path to stay under the artifacts folder.
A failed follow-up drop does not warn twice.
The studio-server unpacked budget covers the record-dropping helper.
* docs(cli): shorten the snapshot skill notes
The command examples already show range and composition.
The prose only keeps the output path and the flags that are easy to misuse.
* chore: leave package headroom for artifact cleanup
* test(cli): compare artifact paths portably
* style(cli): format portable artifact assertion
* fix(keyframes): report unresolved GSAP timing instead of guessing rest poses
* fix(keyframes): drop later times when a delay cannot be resolved
Boolean(null) treated an unreadable delay as no delay,
so the next cursor-relative tween kept a guessed start.
* fix(keyframes): keep a staggered rest pose in Studio
The parser stops are what Studio holds, so a staggered from keeps its rest
pose there. The CLI reports that pose as unresolved, and an unread position
no longer seeds the next tween at zero.
onnxruntime's 1DS telemetry, on by default in its macOS wheels, can abort the
Python process at exit when an upload response races teardown. execFileSync
then throws for a line whose WAV was already written, and the audio skill drops
it as non-fatal.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(cli): guard history turns with --exclusive, --who and --if-open
* fix(cli): make exclusive history turns atomic
Re-read the open turn only after acquiring history ownership so
concurrent exclusive begins cannot both claim the same project.
Cover the race with a deterministic two-process regression test.
* fix(cli): refuse a second exclusive turn while a preview is open
The preview's begin window now rejects another agent's live turn, and an
idle turn no longer blocks exclusive begin. The two-process test waits on
pids collected from claim drafts instead of seeing both files at once.
* feat(cli): HF_TOOL_CONTEXT artifact events and project lock for agent harnesses
* fix(cli): serialize stale project lock takeover
Use a short-lived break lock so only one waiter can remove a dead
holder's lock. Cover the contention path with six real processes.
* fix(cli): hold snapshot lock through finalization
Keep the project lock while contact sheets, artifact events, and frame
descriptions read captured files so another snapshot cannot replace them.
* fix(cli): recover a project lock after the breaker crashes
A killed takeover left project.lock.break in place, so later commands
waited out the timeout even though both holders were dead. Reclaim that
file only when its recorded process is dead, and compare artifact paths
after the platform resolves them.
* fix(cli): accept a killed breaker process with no signal name
Windows ends the process without reporting SIGKILL, so the regression
was failing before it checked that the leftover breaker can be reclaimed.
* fix(cli): leave a killed project-lock breaker in place
Reclaiming a dead breaker unlinks a name another process may just have
created, so two commands can hold the lock. The timeout names that file
and its pid and says to delete it to continue.
* feat(cli): scope check to a time range or a set of components
* fix(cli): keep ranged check gates truthful
Refuse ranged caption and frame gates when none of their seek times
falls inside the requested range. Make ranged motion reporting,
overlap timestamps, filtering tests, and scope documentation match.
A GSAP tween that outlives a later tween on the same target and property by a
moment takes the property back for that tail and leaves its own end value on
screen. A banner word whose drift ended 20 ms after its exit stayed in the
middle of the frame for the rest of the scene, in playback and in renders.
hyperframes check now reads every tween off the live timelines and warns on
that shape (tail up to 0.25 s), with the fix: end the earlier tween first, or
add overwrite: "auto" to the later one. Longer tails, like a slow drift that
resumes after a short move, are left alone.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(inspector): query timelines and inspect the live preview
* fix(inspector): align live timeline contracts
Keep realm-safe target detection through the restack and align
floating-point interval boundaries. Update the live group fixture and
make the private runtime entry build itself before use.
* fix(inspector): declare generated build inputs
* fix(engine): encode and tag ProRes mov output as BT.709
* test(engine): read the ProRes colour tags from the first frame
* test(engine): end the ProRes tag probe's options with --
* fix(media-use): keep frame precision on transcript-cut seek boundaries
Fixes#5416.
A kept range whose boundary sits on a frame got rounded to the millisecond
twice on its way to ffmpeg, and at 29.97/30/60 fps that is enough to land just
past the frame: frame 386 at 30 fps is 12.866666 s, and -ss 12.867 starts that
segment on frame 387 instead. ffmpeg's accurate seek drops the frame, so the cut
came out a frame short from that join on and the picture drifted behind the
sound.
cutlist.mjs rounded the emitted range bounds to 3 decimals, so fmt() in
transcript-cut.mjs never saw more precision than that in the first place; fixing
only fmt() leaves the bug in place. Both now keep frame precision -- 6 decimals
in the cut list, full precision for the -ss/-to actually handed to ffmpeg --
while the 3-decimal fmt() is kept for the times that are only printed.
Measured at 30 fps: a cut of 0-1,1.0666666-3 produced 87 frames before and 88
after (frames 0-29 plus frames 32-89), with no timestamp gap at the join. 25 fps
is unaffected, as the issue notes.
* test(media-use): end ffprobe options with -- in transcript-cut test
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(media-use): honor telemetryEnabled: false from shared config in optedOut
The CLI's 'hyperframes telemetry disable' writes telemetryEnabled: false to ~/.hyperframes/config.json, but the media-use scripts' optedOut() only checked env vars, so transcribe.mjs, transcript-cut.mjs and audio-duck.mjs kept sending PostHog events for users who had opted out through the CLI.
optedOut() now also returns true when the shared config has telemetryEnabled === false, reusing the readSharedConfig() helper already in the same file. Applied byte-identically to both copies of telemetry.mjs (skills/media-use/scripts/lib and packages/cli/src/media-use/lib, per the copy-parity rule), with tests in both test files.
Closes#5465
* test(media-use): sandbox HOME in the telemetry opt-out test and regenerate the skills manifest
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): add hyperframes/api entry with transcribe and removeBackground
HyperFrames Desktop runs the CLI as a child process, relaunching Electron
with ELECTRON_RUN_AS_NODE=1. A programmatic entry lets it import plain
functions and run them in a utilityProcess instead, so the RunAsNode fuse
can be turned off.
packages/cli/src/api.ts exports transcribe() and removeBackground(): typed
options in, typed result out, progress through callbacks, failures thrown
as TranscribeError / RemoveBackgroundError with a code and the original
error as cause. Neither prints, reads argv or exits.
The transcribe and remove-background commands now keep only flag
handling, printing and exit codes, and call these functions. Their
output, JSON and exit codes are unchanged.
The package gains an exports map with "./api" (built dist plus d.ts) and
"./package.json", generated from package-subpaths.json like the sibling
packages.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): trim api.ts comments under the package's comment share
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): trim trailing dots without a quadratic regex
CodeQL flags the trailing-dots regex on the Parakeet error now that it sits
behind a library entry. Same result, linear time.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): make the transcribe API's signal stop every stage
Review follow-ups on the api entry:
- parakeet-mlx now runs as an async child that the caller's signal can
kill. Its failures keep the synchronous run's message, status and stderr.
- A pre-aborted signal throws before any setup. Whisper checks the signal
before each setup stage, and the model download gets it.
- onEngine reports the model whisper runs (small for Spanish, not small.en).
The resolver moved to whisper/modelForLanguage.ts so both share it.
- api/transcribe.ts imports everything at module scope. The command test
hoists its whisper mock, since that module now loads during imports.
- whisper's internal TranscribeOptions/TranscribeResult types are renamed
WhisperOptions/RunnerResult, so the API's public names are unique.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): let an AbortSignal stop whisper's install and audio prep
Review follow-ups on the api entry:
- whisper's runtime install (brew, git clone, cmake) and its ffmpeg audio
prep run through a new stoppableExecFile when an API caller passes a
signal. The child runs off the event loop, an abort ends it and every
process it started, and the call rejects with a "cancelled"
TranscribeError once they are gone. A failure throws the same error
execFileSync throws (message, code, errno, status, signal, stdout,
stderr), compared field by field in stoppableExecFile.test.ts.
parakeet-mlx uses the same helper.
- Without a signal the helper is execFileSync, and the CLI passes none, so
the CLI output stays byte for byte the same as hyperframes 0.8.146.
- TranscribeOptions.signal now says what it covers: a tool lookup that is
already running (capped at 10 seconds) finishes first.
- New regressions in api.test.ts abort during a stand-in install step and
during a stand-in ffmpeg convert. Both end in under 10 seconds with no
process left, and both fail with the old synchronous calls.
- Tests load every module at the top of the file: namespace spies replace
the importOriginal partial mocks, and node:fs and node:child_process are
spied through their default exports and synced with
syncBuiltinESMExports. remove-background --info imports its manager
statically. The scanner in api.test.ts now reads every file this PR
touches and catches importOriginal and vi.importActual.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): media-treatment applies one treatment to every matched image or video
* fix(cli): media-treatment reports its action from every target and documents the per-element merge
* fix(cli): media-treatment patches each match by its stable id so one write cannot redirect another
* fix(cli): media-treatment sets every match in one parse instead of re-finding elements
* fix(cli): media-treatment refuses media that is not an HTML element, as before
* chore(skills): refresh the media-use manifest hash for the treatment reference edit
* fix(cli): the browser download unpacks on machines without an unzip command
* fix(cli): a corrupt browser download on the yauzl path is retried and named
* revert(cli): drop the cause-chain reader; @puppeteer/browsers never passes the cause through
Xuanru Li