feat(producer): renderStretch to re-time short compositions across longer scenes (#2676)

Linear: VA-1859

## Problem

For a `fit_to_scene` B-roll where the composition's intrinsic timeline (e.g. `data-duration=1.0s` → 30 frames) is shorter than the scene it fills (e.g. 4.8s narration), the producer renders only the intrinsic 30 frames and the downstream compositor frame-holds/PTS-stretches that fixed clip to the scene length. Spreading 30 unique frames over 4.8s starves motion to ~6 effective fps → a visibly choppy result. Root cause: the producer welds one `composition.duration` to both the frame count and the 1:1 seek mapping, with no notion of a target output length.

## Fix

Add optional `renderStretch: number` (default `1.0` = no-op), `renderStretch = intrinsic / target`:

- **Frame count** comes from the target: `outputDuration = intrinsic / renderStretch`, `totalFrames = outputDuration × fps` (`probeStage.ts`). `composition.duration` stays intrinsic (drives video/audio windows).
- **Per-frame seek** is scaled: `time = (frameIndex / fps) × renderStretch`, so the N output frames map across `[0, intrinsic]` — a fresh frame per output frame.

All seek sites go through a single shared `outputFrameToTimelineSeconds(frameIndex, fps, renderStretch)` helper (`core.types.ts`), consumed by every capture path so none can silently diverge:
- parallel (`parallelCoordinator.ts`), `sdr_streaming` (`captureStreamingStage.ts` ×3), `sdr_disk` (`captureStage.ts`), HDR loops.
- DrawElement + static self-verify (`frameCapture.ts`) — ground-truth seek uses the same mapping, so PSNR compares like-for-like (no spurious verification failure on stretched comps).
- Distributed path: `renderStretch` threaded through `DistributedRenderConfig` → chunk workers, and **folded into the plan hash only when `!= 1`** so a pre-stretch cached plan is never reused.

With `renderStretch = 1` (or omitted → `?? 1`): every seek is `×1.0` (IEEE-754 identity), frame counts unchanged, and the plan hash is byte-identical — a provable no-op. `player.ts` absolute-seek is untouched.

## Verify

- typecheck (core + engine + producer): pass. lint/format/fallow/commitlint: pass. `planHash` + `renderRequest` unit suites: pass.
- Adversarial self-review found + fixed three capture-path gaps (streaming, self-verify, distributed) before this revision.
- **Not yet runtime-verified** on a real render — needs a fit_to_scene render at `renderStretch < 1` confirming N distinct frames over the target length (draft until then).

Paired with experiment-framework#42766, which computes and forwards `renderStretch = hf intrinsic / scene duration`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
This commit is contained in:
Xuanru Li
2026-07-21 13:30:44 -07:00
committed by GitHub
parent 696cbdbbd0
commit e786b78b33
20 changed files with 159 additions and 15 deletions
+12
View File
@@ -49,6 +49,8 @@ export interface RenderRequestOptions {
variables?: Record<string, unknown>;
outputResolution?: CanvasResolution;
outputResolutionAspectAgnostic?: boolean;
/** intrinsic/target timeline re-time; 1 = no-op. Audio is silence-padded to output length, not time-stretched. */
renderStretch?: number;
engineConfig: EngineConfig;
distributed?: DistributedRenderOptions;
}
@@ -105,6 +107,13 @@ function assertOptionalInteger(options: Record<string, unknown>, field: string,
}
}
function assertOptionalPositiveNumber(options: Record<string, unknown>, field: string): void {
const value = options[field];
if (value !== undefined && (typeof value !== "number" || !Number.isFinite(value) || value <= 0)) {
throw new Error(`Render request ${field} must be a positive number`);
}
}
function assertOptionalString(options: Record<string, unknown>, field: string): void {
if (options[field] !== undefined && typeof options[field] !== "string") {
throw new Error(`Render request ${field} must be a string`);
@@ -160,6 +169,7 @@ function assertRequestOptionScalars(options: Record<string, unknown>): void {
assertOptionalInteger(options, "gifLoop");
assertOptionalInteger(options, "workers", 1);
assertOptionalInteger(options, "crf");
assertOptionalPositiveNumber(options, "renderStretch");
for (const field of ["useGpu", "debug", "outputResolutionAspectAgnostic"] as const) {
assertOptionalBoolean(options, field);
}
@@ -284,6 +294,7 @@ export function distributedConfigFromRequest(
videoFrameFormat: options.videoFrameFormat,
outputResolution: options.outputResolution,
outputResolutionAspectAgnostic: options.outputResolutionAspectAgnostic,
renderStretch: options.renderStretch,
chunkSize: distributed.chunkSize,
maxParallelChunks: distributed.maxParallelChunks,
targetChunkFrames: distributed.targetChunkFrames,
@@ -339,6 +350,7 @@ export function renderRequestFromDistributedConfig(input: {
...optionalProperty("videoFrameFormat", config.videoFrameFormat),
...optionalProperty("outputResolution", config.outputResolution),
...optionalProperty("outputResolutionAspectAgnostic", config.outputResolutionAspectAgnostic),
...optionalProperty("renderStretch", config.renderStretch),
...optionalProperty("hdrMode", config.hdrMode),
...optionalProperty("strictness", config.strictness),
...optionalProperty("entryFile", config.entryFile),