test(producer): add webm-vp9 distributed regression fixture (#952)

* feat(producer): enable webm in distributed mode via concat-copy

PR 8.2 of the WebM distributed-rendering plan (v1.5 backlog #1; see
DISTRIBUTED-RENDERING-PLAN.md §7.2). Wires libvpx-vp9 webm through the
distributed pipeline now that PR 8.1 proved concat-copy works.

Architectural decision: Path A (concat-copy) — based on PR 8.1's smoke
test result (9/9 tests pass for both yuv420p and yuva420p VP9 streams).
The simpler architecture wins; no re-encode in assemble, no encode-
parallelism loss.

Changes:

- plan.ts:
  - DistributedRenderConfig.format and PlanResult.format now include
    "webm" — type-level acceptance matches the runtime gate.
  - rejectUnsupportedDistributedFormat() no longer trips on webm. HDR
    mp4 remains the only refused configuration.
  - resolveEncoderTriple() returns libvpx-vp9-software + yuva420p +
    preset="good" for format="webm". yuva420p preserves alpha — the
    format's main reason for existing for web delivery.
  - codec= remains rejected for non-mp4 formats (mov is always ProRes
    4444; webm is always libvpx-vp9). The error message lists all four
    distributed-supported formats.
  - FormatNotSupportedInDistributedError docstring updated to reflect
    the new reality (only HDR is unsupported).

- freezePlan.ts: LockedRenderConfig.encoder gains "libvpx-vp9-software".
  Mirrors libx265-software / prores-software / png-sequence in shape;
  the chunk worker reads this discriminant to decide encode args.

- renderChunk.ts: drops the now-incorrect cast that excluded webm from
  buildSyntheticRenderJob's format input; tightens the preset-format
  cast to include webm.

- assemble.ts: docstring + comment updates. The mp4/mov concat-copy
  path is format-agnostic — webm uses the exact same code (applyFaststart
  is a no-op for webm via the existing chunkEncoder.ts gate;
  muxVideoWithAudio already routes webm to libopus audio).

- planFormatBanlist.test.ts: webm-rejection tests removed; replaced with
  "accepts webm" tests + a HDR+webm combo test that verifies HDR is the
  trip regardless of format.

- plan.test.ts: new describe block pins the webm wiring contract:
  format="webm" produces an encoder=libvpx-vp9-software /
  pixelFormat=yuva420p planDir with closedGop=true and gopSize=chunkSize.

- webm-concat-copy.test.ts (smoke): extended with a yuva420p variant
  that proves the alpha pixel format the distributed pipeline actually
  emits also round-trips through concat-copy. 9/9 tests pass locally.

§8 format support matrix in DISTRIBUTED-RENDERING-PLAN.md is intentionally
left unchanged at this PR — it flips to ✓ in PR 8.4 once the end-to-end
fixture (PR 8.3) is green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(producer): include webm in plan-time needsAlpha + strengthen alpha smoke

PR review feedback from Miguel and Vai on #951 caught a real bug:
`plan.ts`'s `needsAlpha` disjunction excluded `"webm"`, so the plan
stage froze `forceScreenshot: false` into the `LockedRenderConfig`
even though distributed webm uses `yuva420p`. Every chunk worker
captured opaque RGB via BeginFrame (which doesn't preserve alpha on
Linux headless-shell), and libvpx-vp9 encoded uniformly-opaque alpha
that the encoder then dropped — producing un-keyable webm.

Two changes:

1. **plan.ts**: include `"webm"` in `needsAlpha`. Matches the
   in-process renderer's logic at `renderOrchestrator.ts:1469`
   (`const needsAlpha = isWebm || isMov || isPngSequence`); the two
   sites must stay in sync since the distributed pipeline's PSNR
   regression compares against the in-process baseline.

2. **Smoke test (yuva420p describe)**: source frames now use a real
   alpha gradient (`geq=a='X*255/W'` on top of `testsrc2`) instead of
   `testsrc2 + format=rgba` which was uniformly opaque. The decode-
   pix_fmt assertion is dropped (ffprobe reports `yuv420p` for
   VP9-with-alpha because the alpha lives in a Matroska
   `BlockAdditional` sidecar) and replaced with two stronger checks:
   - `TAG:ALPHA_MODE=1` is present on the stream — proves the
     encoder was actually configured for alpha
   - alpha plane variance after `-c:v libvpx-vp9 -i ... -pix_fmt rgba
     -vf extractplanes=a,signalstats` — proves the alpha sub-stream
     round-trips through concat-copy with spatially-varying content,
     not uniform/dropped alpha
   - decode-test gate is now exit-code-only (was `exitCode || stderr`
     which would flake on chatty ffmpeg `-v error` builds emitting
     non-fatal DTS/container notes)

These checks would have caught the `needsAlpha` bug before review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(aws-lambda): widen narrow format types to include webm

CI on PR #951 was failing at typecheck/build because the producer's
`DistributedRenderConfig.format` widened to include webm in this PR
but the aws-lambda package's narrow `"mp4" | "mov" | "png-sequence"`
type literals in `events.ts`, `handler.ts`, and `validateConfig.ts`
hadn't kept up. `renderToLambda.ts:87` passed `config.format` (now
including webm) into a parameter typed against the narrow union,
producing TS2345.

This widening originally landed in PR #952 (test fixture PR) but
needs to be atomic with the producer's widening here to keep each
PR independently typecheck-clean.

Also refactor `formatExtension` from a switch dispatch to a
`Record<DistributedFormat, string>` lookup. Adding the webm case
tipped the switch's CRAP to the 30.0 fallow threshold; the lookup
table drops cyclomatic from 5 to 1 with the same compile-time
exhaustiveness guarantee (TS errors on missing entries when
`DistributedFormat` adds a new format). The runtime
`_exhaustive: never` throw was only protecting against a string
slipping past TS; `validateConfig.ts`'s `ALLOWED_FORMATS` already
gates untrusted input at the SDK boundary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): add webm-vp9 distributed regression fixture

PR 8.3 of the WebM distributed-rendering plan (v1.5 backlog #1; see
DISTRIBUTED-RENDERING-PLAN.md §7.2). End-to-end regression coverage for
the webm distributed path PRs 8.1 and 8.2 wired up.

Adds packages/producer/tests/distributed/webm-vp9/ matching the
mp4-h264-sdr fixture pattern: a 2-second composition (60 frames @ 30fps)
with text, a crossfade across the frame-30 chunk seam, and a continuous
icon rotation — exercises chunk-boundary continuity for both display
contents and VP9 closed-GOP alpha encoding. `chunkSize: 15` produces 4
chunks so 3 seams are tested, and the crossfade straddles the middle
seam to surface alpha-plane discontinuities introduced by alt-ref drift.

Baseline regenerated inside Dockerfile.test via
`bun run --cwd packages/producer docker:test:update webm-vp9`. Runs in:

  - in-process mode: byte-identical match against baseline ✓
  - distributed-simulated mode: PSNR 56.88-63.49 dB across 100
    checkpoints, well above the 30 dB threshold ✓

Wiring updates required to let webm flow through the harness:

- regression-harness-distributed.ts:
  - checkDistributedSupport() no longer rejects webm. HDR mp4 + NTSC
    fps + non-{24,30,60} fps remain rejected.
  - RunDistributedSimulatedInput.format widened to include webm.
  - Docstring + comments updated.

- regression-harness-distributed.test.ts: webm-rejection test replaced
  with "accepts format=webm" test.

- regression-harness.ts: the now-incorrect format cast at the
  distributed-input call site is dropped; comment about why webm was
  excluded is replaced with "webm is now distributed-supported".

- regression-harness-lambda-local-types.ts: RunLambdaLocalInput.format
  widened to include webm so lambda-local mode can also exercise webm
  fixtures end-to-end.

- aws-lambda webm support (Path A through the Lambda handler):
  - formatExtension.ts: DistributedFormat gains "webm" → ".webm" case.
  - events.ts: RenderChunkEvent / AssembleEvent / PlanLambdaResult
    Format widened to include webm.
  - sdk/validateConfig.ts: ALLOWED_FORMATS gains "webm".
  - handler.ts: downloadChunkObjects format param widened.

The Lambda handler delegates to the producer's assemble() primitive
which PR 8.2 already taught to handle webm (concat-copy + applyFaststart
no-op + muxVideoWithAudio with libopus); no Lambda-side rendering
changes are needed beyond the type/validation surfaces above.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(aws-lambda): drop stale webm rejection from validateConfig docblock

PR #952 review nit (Miguel): the validateConfig.ts file-header comment
still claimed the SDK rejects webm, but the runtime check no longer
does (ALLOWED_FORMATS now includes 'webm'). Update the docblock to
reflect that only force-hdr remains an SDK-side rejection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(regression): add webm-vp9 to shard-3 + refactor formatExtension

Three follow-ups bundled together (Vai's review feedback on PR #952
plus the fallow audit finding that surfaced when the webm case was
added):

1. **Wire webm-vp9 into CI regression.** The fixture was added in this
   PR but never appeared in any `.github/workflows/regression.yml`
   shard's args allowlist, so the regression harness's positional-args
   gate skipped it in CI. Append `webm-vp9` to shard-3 (which already
   carries `mp4-h264-sdr` + `webm-transparency`) so the fixture runs.

2. **Fix stale "four hard gates" prose in checkDistributedSupport
   docstring.** Earlier in the stack I removed the webm bullet but
   didn't update the count. Two gates remain (fps + hdr).

3. **Refactor `formatExtension` from switch to lookup table.** Adding
   the webm case made the switch dispatch's CRAP score hit 30.0
   (cyclomatic = 5, plus the function's small body). Replaced with a
   `Record<DistributedFormat, string>` lookup, which:
   - drops cyclomatic from 5 → 1,
   - keeps exhaustiveness enforcement at compile time (TS errors if
     a new format gets added to `DistributedFormat` without a
     matching key in the Record literal),
   - drops the runtime `_exhaustive: never` throw, which was only
     guarding against an arbitrary string slipping past TS — a
     caller-side concern, not this function's job.

   The function now reads as a table lookup, which matches what it
   actually does, and the fallow audit now reports zero new
   complexity findings (down from 1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
James Russo
2026-05-19 03:13:30 -04:00
committed by GitHub
co-authored by Claude Opus 4.7
parent c336508d4e
commit 6d2569c6bb
10 changed files with 335 additions and 33 deletions
@@ -12,8 +12,9 @@
*
* The check is deliberately narrow — it covers the *shape* errors any
* caller could have surfaced with `tsc` if they passed a literal, plus
* the documented `webm`/`force-hdr` rejections from §5.3 of the
* distributed-rendering plan. Anything deeper (font availability, plan
* the `force-hdr` rejection (HDR mp4 isn't supported in distributed
* mode). webm was previously rejected here too; v0.7+ supports it via
* closed-GOP concat-copy. Anything deeper (font availability, plan
* size cap, GPU mode at runtime) needs the actual planner.
*/
@@ -70,12 +70,9 @@ describe("checkDistributedSupport()", () => {
}
});
it("rejects format=webm", () => {
it("accepts format=webm (distributed-supported via closed-GOP concat-copy)", () => {
const result = checkDistributedSupport({ fps: { num: 30, den: 1 }, format: "webm" });
expect(result.supported).toBe(false);
if (!result.supported) {
expect(result.reason).toMatch(/webm/);
}
expect(result.supported).toBe(true);
});
it("rejects hdr=true", () => {
@@ -25,9 +25,9 @@
* capture jitter, so the harness can't use it as a per-test gate.
*
* Not every fixture can run in distributed-simulated mode. Distributed mode
* refuses webm, HDR mp4, NTSC framerates, and non-{24,30,60} fps at plan
* time. Fixtures that don't meet the constraints are skipped — the harness
* logs the reason and the fixture is treated as "passed (skipped)" in
* refuses HDR mp4, NTSC framerates, and non-{24,30,60} fps at plan time.
* Fixtures that don't meet the constraints are skipped — the harness logs
* the reason and the fixture is treated as "passed (skipped)" in
* distributed-simulated mode.
*/
@@ -67,15 +67,13 @@ export type DistributedSupportResult = { supported: true } | { supported: false;
/**
* Decide whether a fixture's `renderConfig` is one the distributed pipeline
* can actually run. The four hard gates:
* can actually run. Two hard gates:
*
* - fps must be `{ num: 24|30|60, den: 1 }`. `DistributedRenderConfig.fps`
* accepts only the three integer values, and rationals like
* `{ num: 30000, den: 1001 }` (NTSC) trip the type system at the call
* site. We surface this gate in code rather than only in TS so the
* harness can skip the fixture cleanly instead of throwing.
* - format must not be `webm`. `plan()` refuses webm with
* `FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED`.
* - hdr must not be `true`. Distributed mode is SDR-only at v1.
*
* Callers that want the structured reason can read it off the returned
@@ -99,13 +97,6 @@ export function checkDistributedSupport(renderConfig: {
reason: `fps ${fpsNum} not in {24, 30, 60} (DistributedRenderConfig.fps is a closed set)`,
};
}
const format = renderConfig.format ?? "mp4";
if (format === "webm") {
return {
supported: false,
reason: "format=webm refused in distributed mode (VP9+matroska concat-copy is unstable)",
};
}
if (renderConfig.hdr === true) {
return {
supported: false,
@@ -129,7 +120,7 @@ export interface RunDistributedSimulatedInput {
renderedOutputPath: string;
/** From the fixture's renderConfig — must pass `checkDistributedSupport`. */
fps: 24 | 30 | 60;
format: "mp4" | "mov" | "png-sequence";
format: "mp4" | "mov" | "png-sequence" | "webm";
/**
* Codec for `format: "mp4"`. Defaults to `"h264"`; pass `"h265"` to
* exercise the libx265 closed-GOP path. Ignored for non-mp4 formats —
@@ -26,7 +26,7 @@ export interface RunLambdaLocalInput {
*/
width: number;
height: number;
format: "mp4" | "mov" | "png-sequence";
format: "mp4" | "mov" | "png-sequence" | "webm";
codec?: "h264" | "h265";
chunkSize?: number;
maxParallelChunks?: number;
+7 -10
View File
@@ -94,8 +94,8 @@ type TestMetadata = {
* single video file — the harness branches its comparison logic
* accordingly (per-frame byte equality instead of PSNR). `"mov"` and
* `"webm"` are encoded video containers that share the PSNR path with
* `"mp4"`. `"webm"` is rejected by the distributed pipeline at plan
* time; the in-process renderer accepts it.
* `"mp4"`. Distributed mode supports all four — webm goes through
* libvpx-vp9 with closed-GOP concat-copy.
*/
format?: "mp4" | "webm" | "mov" | "png-sequence";
/**
@@ -163,7 +163,7 @@ type TestResult = {
passed: boolean;
/**
* Set when `--mode=distributed-simulated` skips a fixture that the
* distributed pipeline can't run (webm, HDR, NTSC fps, fps∉{24,30,60}).
* distributed pipeline can't run (HDR, NTSC fps, fps∉{24,30,60}).
* `passed` is `true` for skipped fixtures — skipping is a clean outcome,
* not a failure — but the summary distinguishes them.
*/
@@ -939,19 +939,16 @@ async function runTestSuite(
result.skipped = { reason: support.reason };
return result;
}
// `checkDistributedSupport` already narrowed fps to {24,30,60} and
// rejected webm; the cast surfaces that guarantee to TS.
// `checkDistributedSupport` already narrowed fps to {24,30,60}; the
// cast surfaces that guarantee to TS. webm is now distributed-
// supported via closed-GOP concat-copy, so the format passes through.
const fpsNum = suite.meta.renderConfig.fps.num as 24 | 30 | 60;
const distributedInput = {
projectDir: tempSrcDir,
tempRoot,
renderedOutputPath,
fps: fpsNum,
// `runDistributedSimulatedRender` / `runLambdaLocalRender`'s
// `format` parameter accepts the distributed-supported set;
// the harness type allows `"webm"` too but
// `checkDistributedSupport` rejected that above. Narrow.
format: outputFormat as "mp4" | "mov" | "png-sequence",
format: outputFormat,
codec: suite.meta.renderConfig.codec,
chunkSize: suite.meta.renderConfig.chunkSize,
maxParallelChunks: suite.meta.renderConfig.maxParallelChunks,
@@ -0,0 +1,17 @@
{
"name": "Distributed: webm VP9",
"description": "60-frame composition (2s @ 30fps) with text and a small rotating SVG icon, rendered to webm (VP9 + yuva420p). renderConfig.format=webm routes the distributed pipeline through libvpx-vp9 with closed-GOP keyint params (-g N -keyint_min N -auto-alt-ref 0 -cpu-used 2) so per-chunk VP9 output can be losslessly stitched with `ffmpeg -f concat -c copy`. The in-process baseline renders to webm too (codec=vp9, pixelFormat=yuva420p), so the harness's PSNR comparison measures 'libvpx-vp9 chunked + concat' against 'libvpx-vp9 single-pass'. Closed-GOP forces more keyframes than open-GOP, which inflates per-chunk bitrate at constant CRF; PSNR threshold is set at 30 dB to absorb the resulting cross-mode drift without masking gross regressions.",
"tags": ["distributed", "webm", "vp9", "sdr"],
"minPsnr": 30,
"maxFrameFailures": 0,
"minAudioCorrelation": 0.9,
"maxAudioLagWindows": 120,
"renderConfig": {
"fps": 30,
"format": "webm",
"chunkSize": 15
}
}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b98efde6993524eaec419b2a7d3c37f33dcc84602bbd89cfcb1eea41cc125506
size 78674
@@ -0,0 +1,131 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta content="width=device-width, initial-scale=1.0" name="viewport" />
<title>webm VP9 distributed fixture</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.2/gsap.min.js"></script>
<style>
@import url("https://fonts.googleapis.com/css2?family=Space+Mono:wght@400;700&display=swap");
body,
html {
margin: 0;
padding: 0;
width: 640px;
height: 360px;
/* Transparent background — webm + yuva420p output preserves alpha,
* which is the format's reason for existing. The renderer's
* `initTransparentBackground` helper enforces this in production.
*/
background: transparent;
overflow: hidden;
font-family: "Space Mono", monospace;
}
#main-comp {
position: relative;
width: 640px;
height: 360px;
}
.stage {
position: absolute;
inset: 0;
}
.label {
position: absolute;
top: 22%;
left: 50%;
transform: translateX(-50%);
font-size: 18px;
letter-spacing: 4px;
color: #94a3b8;
text-transform: uppercase;
}
.title {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
font-family: "Space Mono", monospace;
font-size: 56px;
font-weight: 700;
color: #6366f1;
white-space: nowrap;
}
.icon {
position: absolute;
bottom: 18%;
left: 50%;
transform: translate(-50%, 0);
width: 48px;
height: 48px;
}
</style>
</head>
<body>
<div
id="main-comp"
data-composition-id="main-comp"
data-width="640"
data-height="360"
data-start="0"
data-duration="2"
>
<div class="stage" id="stage-a">
<div class="label">CHUNK</div>
<div class="title" id="title-a">PHASE&nbsp;ONE</div>
</div>
<div class="stage" id="stage-b" style="opacity: 0">
<div class="label">CHUNK</div>
<div class="title" id="title-b">PHASE&nbsp;TWO</div>
</div>
<!-- Small inline SVG icon; no external image fetch required. -->
<svg class="icon" viewBox="0 0 48 48" xmlns="http://www.w3.org/2000/svg" id="icon">
<circle cx="24" cy="24" r="18" fill="none" stroke="#6366f1" stroke-width="4" />
<circle cx="24" cy="24" r="6" fill="#6366f1" />
</svg>
<!--
No audio element on purpose. Opus frame quantization at 20ms grain
pads a 2-second silent track past 2.0s of container time, which
extends the muxed webm's duration past nb_frames/fps and trips the
harness PSNR sampler at the very last checkpoint. The chunk-
boundary contracts this fixture pins are video-only; omitting
audio keeps container duration == 2.0s exactly. Other webm-with-
audio fixtures cover the mux path separately when added.
-->
</div>
<script>
// Build a single timeline pinned to the composition's wall-clock so
// every frame is fully determined by the seek position. Chunk
// boundaries at frames {15, 30, 45} sit inside the crossfade window
// (frames 27-33 = 0.9s-1.1s) and the icon rotation — both of which
// therefore exercise per-chunk state continuity AND VP9 closed-GOP
// alpha encoding (`-auto-alt-ref 0` so chunk seams remain
// independently decodable).
const tl = gsap.timeline({ paused: true });
window.__timelines = window.__timelines || {};
window.__timelines["main-comp"] = tl;
const stageA = document.getElementById("stage-a");
const stageB = document.getElementById("stage-b");
const icon = document.getElementById("icon");
// Crossfade A → B straddling the frame-30 chunk seam. The opacity
// animation goes through the alpha plane in yuva420p output, which
// is exactly what we want to test under chunked-concat conditions.
tl.to(stageA, { opacity: 0, duration: 0.2, ease: "none" }, 0.9);
tl.to(stageB, { opacity: 1, duration: 0.2, ease: "none" }, 0.9);
// Continuous icon rotation — the absolute angle at any time must
// match across chunk boundaries, so a state-keeping regression in
// the engine's virtual clock would show up as a rotation
// discontinuity at frame 15, 30, or 45.
tl.to(icon, { rotation: 360, duration: 2, ease: "none" }, 0);
</script>
</body>
</html>