ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-05 00:56:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(engine): wait for first frame decode + drop B-frames so renders play in every player

Browse Source 
Three related render robustness fixes:

1. frameCapture.ts: bump videos-ready check from `readyState >= 1`
   (HAVE_METADATA — only dimensions known) to `>= 2` (HAVE_CURRENT_DATA —
   first frame is rasterized). Without this, when two `<video>` elements
   with different codecs (h264 mp4 + VP9 webm) decode at different rates,
   the faster one passes readiness while the slower one still hasn't
   painted, producing a black "first frame" for the slower clip.

2. chunkEncoder.ts (libx264 path) + streamingEncoder.ts: disable B-frames
   for h264 (`-bf 0`). Standard libx264 with B-frames produces negative
   DTS at stream start (the first B-frame's decode order is "before" the
   first I-frame's presentation time). VS Code preview, several browser
   <video> implementations, and some HW decoders freeze on the first
   frame and only audio plays. -bf 0 makes PTS == DTS at every frame,
   eliminating the issue at the source. Quality cost is ~5–10% larger
   files at the same CRF — worthwhile for "the file plays everywhere".

3. chunkEncoder.ts (encoder + mux paths): add `-avoid_negative_ts make_zero`
   as belt-and-suspenders against negative DTS sneaking back in via
   `-c:v copy` mux passes when audio/video PTS bases differ.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
James
2026-05-04 20:27:56 -07:00
co-authored by Claude Opus 4.7
parent 688052d368
commit 0e541673e0
3 changed files with 41 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/engine/src/services/chunkEncoder.ts
+17
View File
@@ -145,6 +145,18 @@ export function buildEncoderArgs(
if (bitrate) args.push("-b:v", bitrate);
else args.push("-crf", String(quality));
// Disable B-frames. Standard h264 with B-frames produces negative DTS
// at the start of the stream (the first B-frame's decode order is
// "before" the first I-frame's presentation time). VS Code's video
// preview, several browser <video> pipelines, and some HW decoders
// freeze on the first frame when DTS is negative, so audio plays alone.
// -bf 0 makes PTS == DTS at every frame, eliminating the issue at the
// source. Quality cost is ~5–10% larger files at the same CRF — a
// worthwhile trade for "the file plays everywhere".
if (codec === "h264") {
args.push("-bf", "0");
}
// Encoder-specific params: anti-banding + color space tagging.
// aq-mode=3 redistributes bits to dark flat areas (gradients).
// For HDR x265 paths we additionally embed BT.2020 + transfer + HDR static
@@ -239,6 +251,8 @@ export function buildEncoderArgs(
args.push("-pix_fmt", pixelFormat);
}
args.push("-avoid_negative_ts", "make_zero");
args.push("-y", outputPath);
return args;
}
@@ -510,6 +524,9 @@ export async function muxVideoWithAudio(
} else {
args.push("-c:a", "aac", "-b:a", "192k", "-movflags", "+faststart");
}
// PTS bases can diverge during mux and reintroduce negative DTS. See
// buildEncoderArgs for the full reasoning on why that breaks playback.
args.push("-avoid_negative_ts", "make_zero");
args.push("-shortest", "-y", outputPath);
const processTimeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;