Commit Graph
5 Commits
Author SHA1 Message Date
1d2aa5240d fix(media-use): transcript-cut video renders without repeated frames or a keyframe warning (#5463)
* fix(media-use): transcript-cut video renders without repeated frames or a keyframe warning

transcript-cut wrote each kept range to a temporary .mkv, and ffmpeg
writes Matroska timestamps in whole milliseconds. A 30 fps frame that
belongs at 2/30 s ended up at 0.067, the final copy kept it there, and
the renderer showed the previous frame wherever a timestamp was late.

The temporary files are now .mov, which holds the frame times exactly
and still carries PCM audio, so the audio is encoded once, as before.

The cut also gets a keyframe every second. x264 on its own left them
several seconds apart, and the renderer warned about the footage this
script had just written.

Fixes #5417

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media-use): transcript-cut ranges start on their first frame with any ffmpeg

ffmpeg 7 and later start a kept range that begins between two source
frames one frame late, and .mov rounds that lead-in to the millisecond,
which put the frames off the grid again. ffmpeg 6 did not do this, so
the tests passed locally and failed in CI.

Each temporary file is now written at a constant frame rate, so the
first frame of a range is at zero whichever ffmpeg runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: James <james.russo@heygen.com>
2026-10-13 01:23:05 +01:00
b71c9f4e1b fix(media-use): keep frame precision on transcript-cut seek boundaries (#5426)
* fix(media-use): keep frame precision on transcript-cut seek boundaries

Fixes #5416.

A kept range whose boundary sits on a frame got rounded to the millisecond
twice on its way to ffmpeg, and at 29.97/30/60 fps that is enough to land just
past the frame: frame 386 at 30 fps is 12.866666 s, and -ss 12.867 starts that
segment on frame 387 instead. ffmpeg's accurate seek drops the frame, so the cut
came out a frame short from that join on and the picture drifted behind the
sound.

cutlist.mjs rounded the emitted range bounds to 3 decimals, so fmt() in
transcript-cut.mjs never saw more precision than that in the first place; fixing
only fmt() leaves the bug in place. Both now keep frame precision -- 6 decimals
in the cut list, full precision for the -ss/-to actually handed to ffmpeg --
while the 3-decimal fmt() is kept for the times that are only printed.

Measured at 30 fps: a cut of 0-1,1.0666666-3 produced 87 frames before and 88
after (frames 0-29 plus frames 32-89), with no timestamp gap at the join. 25 fps
is unaffected, as the issue notes.

* test(media-use): end ffprobe options with -- in transcript-cut test

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: James <james.russo@heygen.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 01:08:20 +01:00
dd97478175 fix(media-use): remove clicks and ~25ms gaps at every transcript-cut splice (#4179)
* fix(media-use): remove clicks and ~25ms gaps at every transcript-cut splice

transcript-cut extracts each kept range to its own final-codec file, then
concat-copies them. That produces two audible artifacts at every cut:

1. No boundary fades. Concat splices raw segment edges together, so the
   waveform steps discontinuously and each cut clicks.

2. Encoder priming silence. Each segment is encoded to AAC independently,
   so the encoder pads every one with priming samples. Stream-copy concat
   bakes that padding in as a silent gap at each splice. Measured 24.4 ms
   and 24.1 ms at the two splices of a three-range cut. It accumulates:
   the nth splice lands roughly 25n ms late, so splice positions drift
   away from the sum of the preceding segment durations.

The second survives the first -- adding fades alone leaves the gap.

Fix: intermediates carry PCM (.mkv for video, .wav for audio-only) instead
of the final codec, so audio is encoded exactly once, at concat. Boundary
fades of 30 ms are applied per segment, scaled down on segments shorter
than 4x the ramp and skipped on degenerate ones.

Measured on the same three-range cut, widest run of digital silence at
each splice:

  before  24.4 ms / 24.1 ms
  after    0.1 ms /  0.1 ms

--copy is unchanged: stream copy cannot filter, so it keeps producing
final-codec segments and a copy concat. Audio-only output is byte-identical
in duration before and after. scripts/lib test suite: 34/35, the one
failure (lut-preset-provider) pre-exists on main and is unrelated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(skills): regenerate manifest for media-use

* fix(media-use): honor the fadeIn/fadeOut flags fadeFilterFor was passed

The caller already computed {fadeIn, fadeOut} per segment, but
fadeFilterFor still took one parameter and always emitted both
afade ramps -- so a single kept segment (no splice at all) and every
segment's true start/end got faded regardless. Extracted the pure
function into lib/transcriptCutFade.mjs, made it honor both flags,
and added its first unit test: first/middle/last/single-segment
filter strings, no ffmpeg needed, fails against the previous version.

Also installs ffmpeg in the Test: skills CI job (same action the
Producer job uses), so transcript-cut.test.mjs's real-tone RED/GREEN
proof runs in CI instead of skipping silently for lack of the binary.

* fix(media-use): fix the packed-CLI import and lint on transcript-cut.mjs

build-copy.mjs replaces dist/skills/media-use/scripts/lib with a copy
of packages/cli/src/media-use/lib, so transcriptCutFade.mjs needed a
byte-identical twin there (and in MEDIA_USE_COPY_NAMES) or the packed
CLI's transcript-cut.mjs imported a file that no longer existed.
Moved its test alongside, matching every other lib helper's tests.

Also drops three redundant Boolean(args.copy) wrappers (a ternary
condition already coerces) that oxlint's no-extra-boolean-cast
flagged.

* style(media-use): reflow the shortened ext ternary to oxfmt's shape

Dropping the Boolean() wrapper shortened the line enough that oxfmt
0.41 now collapses the ternary onto one line; format:check was failing
on the stale multi-line version.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
2026-09-22 20:28:50 -04:00
Vance IngallsandClaude Opus 5 255cf92915 fix(skills,producer): terminate ffprobe options in shipped skill scripts
The contract test only walked packages/*/src and only .ts, so it could not see
the shipped agent tools under skills/**, which are .mjs/.cjs. 19 call sites
there and in package tests were still missing `--` immediately before the
input while the suite reported the bug class closed — a dash-prefixed filename
is parsed as an option and fails the same way.

Sweeps packages/, skills/ and scripts/ now, including .mjs/.cjs and test
files (dither.test.mjs was one of the broken sites). Excludes only the
contract test itself, which documents the contract with example argvs
including a deliberately misordered one.

Two guards were fixed while widening: the terminator must never be inserted
after `-i`, which consumes the next token (a blind pass hit an ffmpeg input
and a base64 -i), and comment prose describing a spawn is not a spawn.

Also routes every audioPadTrim probe failure through one sanitizer at the
boundary. runFfprobeJson scrubbed its own stderr, but
defaultProbeVideoFrameInfo threw `no video stream in ${videoPath}` raw into
the public PadTrimAudioResult.error, and an injected probe can throw anything.
The redaction unit tests all passed with the caller wiring deleted; the new
public-path regressions fail without it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 02:23:35 -07:00
Miguel Angel Simon Sierra ac0153cb30 feat(media-use): transcription (parakeet), transcript-cut, audio-duck editing tools 2026-07-06 23:41:35 -04:00