mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-11 14:50:02 +00:00
feat(producer): enable webm in distributed mode via concat-copy (#951)
* feat(producer): enable webm in distributed mode via concat-copy PR 8.2 of the WebM distributed-rendering plan (v1.5 backlog #1; see DISTRIBUTED-RENDERING-PLAN.md §7.2). Wires libvpx-vp9 webm through the distributed pipeline now that PR 8.1 proved concat-copy works. Architectural decision: Path A (concat-copy) — based on PR 8.1's smoke test result (9/9 tests pass for both yuv420p and yuva420p VP9 streams). The simpler architecture wins; no re-encode in assemble, no encode- parallelism loss. Changes: - plan.ts: - DistributedRenderConfig.format and PlanResult.format now include "webm" — type-level acceptance matches the runtime gate. - rejectUnsupportedDistributedFormat() no longer trips on webm. HDR mp4 remains the only refused configuration. - resolveEncoderTriple() returns libvpx-vp9-software + yuva420p + preset="good" for format="webm". yuva420p preserves alpha — the format's main reason for existing for web delivery. - codec= remains rejected for non-mp4 formats (mov is always ProRes 4444; webm is always libvpx-vp9). The error message lists all four distributed-supported formats. - FormatNotSupportedInDistributedError docstring updated to reflect the new reality (only HDR is unsupported). - freezePlan.ts: LockedRenderConfig.encoder gains "libvpx-vp9-software". Mirrors libx265-software / prores-software / png-sequence in shape; the chunk worker reads this discriminant to decide encode args. - renderChunk.ts: drops the now-incorrect cast that excluded webm from buildSyntheticRenderJob's format input; tightens the preset-format cast to include webm. - assemble.ts: docstring + comment updates. The mp4/mov concat-copy path is format-agnostic — webm uses the exact same code (applyFaststart is a no-op for webm via the existing chunkEncoder.ts gate; muxVideoWithAudio already routes webm to libopus audio). - planFormatBanlist.test.ts: webm-rejection tests removed; replaced with "accepts webm" tests + a HDR+webm combo test that verifies HDR is the trip regardless of format. - plan.test.ts: new describe block pins the webm wiring contract: format="webm" produces an encoder=libvpx-vp9-software / pixelFormat=yuva420p planDir with closedGop=true and gopSize=chunkSize. - webm-concat-copy.test.ts (smoke): extended with a yuva420p variant that proves the alpha pixel format the distributed pipeline actually emits also round-trips through concat-copy. 9/9 tests pass locally. §8 format support matrix in DISTRIBUTED-RENDERING-PLAN.md is intentionally left unchanged at this PR — it flips to ✓ in PR 8.4 once the end-to-end fixture (PR 8.3) is green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(producer): include webm in plan-time needsAlpha + strengthen alpha smoke PR review feedback from Miguel and Vai on #951 caught a real bug: `plan.ts`'s `needsAlpha` disjunction excluded `"webm"`, so the plan stage froze `forceScreenshot: false` into the `LockedRenderConfig` even though distributed webm uses `yuva420p`. Every chunk worker captured opaque RGB via BeginFrame (which doesn't preserve alpha on Linux headless-shell), and libvpx-vp9 encoded uniformly-opaque alpha that the encoder then dropped — producing un-keyable webm. Two changes: 1. **plan.ts**: include `"webm"` in `needsAlpha`. Matches the in-process renderer's logic at `renderOrchestrator.ts:1469` (`const needsAlpha = isWebm || isMov || isPngSequence`); the two sites must stay in sync since the distributed pipeline's PSNR regression compares against the in-process baseline. 2. **Smoke test (yuva420p describe)**: source frames now use a real alpha gradient (`geq=a='X*255/W'` on top of `testsrc2`) instead of `testsrc2 + format=rgba` which was uniformly opaque. The decode- pix_fmt assertion is dropped (ffprobe reports `yuv420p` for VP9-with-alpha because the alpha lives in a Matroska `BlockAdditional` sidecar) and replaced with two stronger checks: - `TAG:ALPHA_MODE=1` is present on the stream — proves the encoder was actually configured for alpha - alpha plane variance after `-c:v libvpx-vp9 -i ... -pix_fmt rgba -vf extractplanes=a,signalstats` — proves the alpha sub-stream round-trips through concat-copy with spatially-varying content, not uniform/dropped alpha - decode-test gate is now exit-code-only (was `exitCode || stderr` which would flake on chatty ffmpeg `-v error` builds emitting non-fatal DTS/container notes) These checks would have caught the `needsAlpha` bug before review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(aws-lambda): widen narrow format types to include webm CI on PR #951 was failing at typecheck/build because the producer's `DistributedRenderConfig.format` widened to include webm in this PR but the aws-lambda package's narrow `"mp4" | "mov" | "png-sequence"` type literals in `events.ts`, `handler.ts`, and `validateConfig.ts` hadn't kept up. `renderToLambda.ts:87` passed `config.format` (now including webm) into a parameter typed against the narrow union, producing TS2345. This widening originally landed in PR #952 (test fixture PR) but needs to be atomic with the producer's widening here to keep each PR independently typecheck-clean. Also refactor `formatExtension` from a switch dispatch to a `Record<DistributedFormat, string>` lookup. Adding the webm case tipped the switch's CRAP to the 30.0 fallow threshold; the lookup table drops cyclomatic from 5 to 1 with the same compile-time exhaustiveness guarantee (TS errors on missing entries when `DistributedFormat` adds a new format). The runtime `_exhaustive: never` throw was only protecting against a string slipping past TS; `validateConfig.ts`'s `ALLOWED_FORMATS` already gates untrusted input at the SDK boundary. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
James Russo
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
07de7e61ed
commit
21f5066832
@@ -304,3 +304,233 @@ describe("webm VP9 concat-copy smoke", () => {
|
||||
expect(nbFrames).toBe(TOTAL_FRAMES);
|
||||
});
|
||||
});
|
||||
|
||||
describe("webm VP9 concat-copy smoke (yuva420p alpha)", () => {
|
||||
// The wired-up distributed webm path uses yuva420p, not yuv420p — that
|
||||
// matches the in-process renderer's webm pixel format (alpha video, the
|
||||
// format's main reason for existing). yuva420p VP9 streams have a few
|
||||
// extra concat-copy hazards that yuv420p doesn't (the alpha sub-stream
|
||||
// is muxed via `-metadata:s:v:0 alpha_mode=1` and concat-copy must
|
||||
// preserve that metadata across chunks).
|
||||
//
|
||||
// This block re-runs the same three verifications on yuva420p output to
|
||||
// pin the contract for what the distributed pipeline actually emits.
|
||||
let alphaRoot: string;
|
||||
let alphaFramesDir: string;
|
||||
let alphaChunkDir: string;
|
||||
let alphaConcatListPath: string;
|
||||
let alphaOutputPath: string;
|
||||
|
||||
beforeAll(() => {
|
||||
alphaRoot = mkdtempSync(join(tmpdir(), "hf-webm-concat-smoke-alpha-"));
|
||||
alphaFramesDir = join(alphaRoot, "frames");
|
||||
alphaChunkDir = join(alphaRoot, "chunks");
|
||||
mkdirSync(alphaFramesDir, { recursive: true });
|
||||
mkdirSync(alphaChunkDir, { recursive: true });
|
||||
alphaConcatListPath = join(alphaRoot, "concat-list.txt");
|
||||
alphaOutputPath = join(alphaRoot, "output.webm");
|
||||
|
||||
// For alpha frames, generate RGBA PNGs with spatially-varying alpha
|
||||
// so the encoder can't drop the alpha plane as uniform/redundant.
|
||||
// `testsrc2 + format=rgba` (the prior shape) produced uniformly-
|
||||
// opaque alpha and the libvpx-vp9 encoder silently downgraded the
|
||||
// output to yuv420p — masking any bug in the alpha pipeline. Here
|
||||
// `geq=a='X*255/W'` writes a horizontal alpha gradient on top of
|
||||
// the testsrc2 RGB so the alpha track has real per-pixel content.
|
||||
const frameGen = runFfmpegSync([
|
||||
"-hide_banner",
|
||||
"-y",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
`testsrc2=s=${WIDTH}x${HEIGHT}:r=${FPS}:d=${TOTAL_FRAMES / FPS}`,
|
||||
"-vf",
|
||||
"format=rgba,geq=r='r(X,Y)':g='g(X,Y)':b='b(X,Y)':a='X*255/W'",
|
||||
"-frames:v",
|
||||
String(TOTAL_FRAMES),
|
||||
join(alphaFramesDir, "frame_%04d.png"),
|
||||
]);
|
||||
if (frameGen.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`[alpha smoke setup] frame generation failed (exit ${frameGen.exitCode}): ` +
|
||||
frameGen.stderr.slice(-400),
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
rmSync(alphaRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("encodes 4 yuva420p VP9 chunks with closed-GOP args", () => {
|
||||
for (let chunkIdx = 0; chunkIdx < CHUNK_COUNT; chunkIdx++) {
|
||||
const startNumber = chunkIdx * CHUNK_SIZE + 1;
|
||||
const chunkPath = join(alphaChunkDir, `chunk_${String(chunkIdx).padStart(4, "0")}.webm`);
|
||||
const inputArgs = [
|
||||
"-framerate",
|
||||
String(FPS),
|
||||
"-start_number",
|
||||
String(startNumber),
|
||||
"-i",
|
||||
join(alphaFramesDir, "frame_%04d.png"),
|
||||
"-frames:v",
|
||||
String(CHUNK_SIZE),
|
||||
];
|
||||
const args = buildEncoderArgs(
|
||||
{
|
||||
fps: { num: FPS, den: 1 },
|
||||
width: WIDTH,
|
||||
height: HEIGHT,
|
||||
codec: "vp9",
|
||||
preset: "good",
|
||||
quality: 32,
|
||||
// yuva420p is what the distributed pipeline actually emits for
|
||||
// webm; the alpha branch in chunkEncoder.ts adds the
|
||||
// `-metadata:s:v:0 alpha_mode=1` tag we want to verify
|
||||
// round-trips through concat-copy.
|
||||
pixelFormat: "yuva420p",
|
||||
lockGopForChunkConcat: true,
|
||||
gopSize: CHUNK_SIZE,
|
||||
},
|
||||
inputArgs,
|
||||
chunkPath,
|
||||
);
|
||||
const result = runFfmpegSync(["-hide_banner", "-loglevel", "error", ...args]);
|
||||
if (result.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`[alpha smoke chunk ${chunkIdx}] yuva420p VP9 encode failed (exit ${result.exitCode}):\n` +
|
||||
`args: ${JSON.stringify(args)}\n` +
|
||||
`stderr: ${result.stderr.slice(-1000)}`,
|
||||
);
|
||||
}
|
||||
expect(existsSync(chunkPath)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("concat-copies the 4 yuva420p chunks into a single alpha WebM", () => {
|
||||
const lines: string[] = [];
|
||||
for (let chunkIdx = 0; chunkIdx < CHUNK_COUNT; chunkIdx++) {
|
||||
const chunkPath = join(alphaChunkDir, `chunk_${String(chunkIdx).padStart(4, "0")}.webm`);
|
||||
lines.push(`file '${chunkPath.replace(/'/g, "'\\''")}'`);
|
||||
}
|
||||
writeFileSync(alphaConcatListPath, `${lines.join("\n")}\n`, "utf-8");
|
||||
|
||||
const result = runFfmpegSync([
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-f",
|
||||
"concat",
|
||||
"-safe",
|
||||
"0",
|
||||
"-i",
|
||||
alphaConcatListPath,
|
||||
"-c",
|
||||
"copy",
|
||||
"-y",
|
||||
alphaOutputPath,
|
||||
]);
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`[alpha smoke concat-copy] failed (exit ${result.exitCode}). ` +
|
||||
`yuva420p webm concat-copy is broken — PR 8.2 must take Path B. ` +
|
||||
`Failure fingerprint: ${result.stderr.slice(-1000)}`,
|
||||
);
|
||||
}
|
||||
expect(existsSync(alphaOutputPath)).toBe(true);
|
||||
expect(statSync(alphaOutputPath).size).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("decodes alpha-track WebM cleanly without seam errors", () => {
|
||||
const decodeResult = runFfmpegSync([
|
||||
"-hide_banner",
|
||||
"-v",
|
||||
"error",
|
||||
"-i",
|
||||
alphaOutputPath,
|
||||
"-f",
|
||||
"null",
|
||||
"-",
|
||||
]);
|
||||
// Gate only on exit code — `-v error` ffmpeg builds can emit
|
||||
// non-fatal stderr (DTS warnings, container-quirk notes) and we
|
||||
// don't want the test to flake on chatty stderr in a future
|
||||
// libavformat upgrade. Surface stderr in the failure message for
|
||||
// forensic context.
|
||||
if (decodeResult.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`[alpha smoke decode-test] failed (exit ${decodeResult.exitCode}). ` +
|
||||
`Failure fingerprint: ${decodeResult.stderr.slice(-1000) || "(no stderr)"}`,
|
||||
);
|
||||
}
|
||||
|
||||
const probeResult = runFfprobeSync([
|
||||
"-v",
|
||||
"error",
|
||||
"-select_streams",
|
||||
"v:0",
|
||||
"-show_streams",
|
||||
alphaOutputPath,
|
||||
]);
|
||||
expect(probeResult.exitCode).toBe(0);
|
||||
expect(probeResult.stdout).toMatch(/codec_name=vp9/);
|
||||
// libvpx-vp9 stores the alpha plane as a Matroska `BlockAdditional`
|
||||
// sidecar, NOT in the main stream's `pix_fmt` — so `ffprobe` always
|
||||
// reports `pix_fmt=yuv420p` for VP9-with-alpha. The right signal that
|
||||
// alpha encoding was enabled is the stream-level `TAG:ALPHA_MODE=1`
|
||||
// tag the encoder writes when `-metadata:s:v:0 alpha_mode=1` is set
|
||||
// on a yuva420p input.
|
||||
expect(probeResult.stdout).toMatch(/ALPHA_MODE=1/);
|
||||
});
|
||||
|
||||
it("alpha plane round-trips through concat-copy with spatially-varying content", () => {
|
||||
// Decode the concat-copied WebM via the libvpx-vp9 decoder forced to
|
||||
// RGBA, then extract the alpha plane and check it has real spatial
|
||||
// variance — catches the failure mode where the encoder accepted
|
||||
// yuva420p input but dropped the alpha sub-stream silently
|
||||
// (uniform alpha would mask any plan-time bug like the `needsAlpha`
|
||||
// hole that hid this PR's bug before review caught it). The
|
||||
// gradient source produces YMIN ≈ 0 / YMAX ≈ 255 on the alpha
|
||||
// plane; uniform alpha would give YMIN == YMAX. Spread > 100 is a
|
||||
// generous floor that catches the bad case cleanly.
|
||||
//
|
||||
// `-c:v libvpx-vp9` before `-i` is the load-bearing piece: ffmpeg's
|
||||
// default VP9 decoder path strips the BlockAdditional alpha track
|
||||
// when decoding to non-rgba pixel formats; forcing the libvpx-vp9
|
||||
// decoder + `-pix_fmt rgba` is how we get the alpha plane back.
|
||||
const statsResult = runFfmpegSync([
|
||||
"-hide_banner",
|
||||
"-v",
|
||||
"error",
|
||||
"-c:v",
|
||||
"libvpx-vp9",
|
||||
"-i",
|
||||
alphaOutputPath,
|
||||
"-pix_fmt",
|
||||
"rgba",
|
||||
"-vf",
|
||||
"extractplanes=a,signalstats,metadata=mode=print:file=-",
|
||||
"-f",
|
||||
"null",
|
||||
"-",
|
||||
]);
|
||||
if (statsResult.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`[alpha smoke signalstats] failed (exit ${statsResult.exitCode}): ` +
|
||||
`${statsResult.stderr.slice(-500)}`,
|
||||
);
|
||||
}
|
||||
const yminMatch = statsResult.stdout.match(/lavfi\.signalstats\.YMIN=(\d+)/);
|
||||
const ymaxMatch = statsResult.stdout.match(/lavfi\.signalstats\.YMAX=(\d+)/);
|
||||
if (!yminMatch || !ymaxMatch) {
|
||||
throw new Error(
|
||||
`[alpha smoke signalstats] could not parse YMIN/YMAX from output: ` +
|
||||
`${statsResult.stdout.slice(0, 500)}`,
|
||||
);
|
||||
}
|
||||
const ymin = Number.parseInt(yminMatch[1], 10);
|
||||
const ymax = Number.parseInt(ymaxMatch[1], 10);
|
||||
expect(ymax - ymin).toBeGreaterThan(100);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user