From 7d1d8ead609979974adac3c5fae3cf1b8a81cf03 Mon Sep 17 00:00:00 2001 From: James Date: Tue, 5 May 2026 04:18:04 +0000 Subject: [PATCH] fix(producer): sample PSNR checkpoints from common duration of rendered+snapshot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four regression tests (font-variant-numeric, many-cuts, missing-host-comp-id, variables-prod) failed on this PR with `Unable to parse PSNR output at s`. Root cause: the harness derived all 100 checkpoints from the *rendered* video's container duration, then asked ffmpeg's PSNR filter to compare the same frame index from both videos. The encoder changes earlier in this PR add `-avoid_negative_ts make_zero` to the mux step. With AAC audio that shifts the first audio sample to t=0 instead of the encoder-delay offset, extending reported container duration by ~20ms without changing video frame count. For the four failing tests, the i=99 checkpoint then landed on a frame index that exists in the rendered video but not in the snapshot baseline (e.g. round(2.98998 * 24) = 72 in a 72-frame baseline). ffmpeg's PSNR filter ran on zero matched frames and emitted no `average:` line, so the parser threw. Fix: probe both videos and use min(rendered, snapshot) duration when spreading checkpoints. This is the correct semantics for symmetric PSNR comparison anyway — both videos must have a frame at every sampled time. The change is local to the harness; no encoder behavior changes, no baselines regenerated. Other regression tests with audio (chat, sub-composition-video, vignelli-stacking) passed because their checkpoint-99 frame index landed inside the baseline's frame range with several frames of slack. The four failing tests had round-number durations where a 20ms drift was enough to push the last checkpoint past `nb_frames - 1`. --- packages/producer/src/regression-harness.ts | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/packages/producer/src/regression-harness.ts b/packages/producer/src/regression-harness.ts index 27cb2098f..a518fba18 100644 --- a/packages/producer/src/regression-harness.ts +++ b/packages/producer/src/regression-harness.ts @@ -650,7 +650,14 @@ async function runTestSuite( // Visual comparison (100 frames, 1 per 1% of video duration) logPretty("Comparing visual quality (100 checkpoints)...", "🔍"); const videoMetadata = await extractMediaMetadata(renderedOutputPath); - const videoDuration = videoMetadata.durationSeconds; + const snapshotMetadata = await extractMediaMetadata(snapshotVideoPath); + // Sample at the common duration. Container duration can drift between + // rendered and snapshot when encoder/mux flags change (e.g. -avoid_negative_ts + // can shift the first audio sample, extending reported duration without + // changing video frame count). Using the rendered duration alone makes the + // last checkpoint land on a frame index that may not exist in the snapshot, + // which causes ffmpeg's PSNR filter to emit no `average:` line. + const videoDuration = Math.min(videoMetadata.durationSeconds, snapshotMetadata.durationSeconds); const visualCheckpoints: Array<{ time: number; psnr: number; passed: boolean }> = []; for (let i = 0; i < 100; i++) {