Merge pull request #81 from heygen-com/feat/webm-transparency

feat(render): WebM output with VP9 alpha transparency
This commit is contained in:
James Russo
2026-03-26 22:07:57 -07:00
committed by GitHub
17 changed files with 428 additions and 96 deletions
+22 -1
View File
@@ -233,7 +233,7 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
<Tab title="Build">
### `render`
Render a composition to MP4:
Render a composition to MP4 or WebM:
```bash
# Local mode (fast iteration)
@@ -242,6 +242,9 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
# Docker mode (deterministic output)
npx hyperframes render --docker --output output.mp4
# WebM with transparency (for overlays, captions, lower thirds)
npx hyperframes render --format webm --output overlay.webm
# With options
npx hyperframes render --output output.mp4 --fps 60 --quality high
```
@@ -249,6 +252,7 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
| Flag | Values | Default | Description |
|------|--------|---------|-------------|
| `--output` | path | `renders/<name>.mp4` | Output file path |
| `--format` | mp4, webm | mp4 | Output format (WebM renders with transparency) |
| `--fps` | 24, 30, 60 | 30 | Frames per second |
| `--quality` | draft, standard, high | standard | Encoding quality preset |
| `--workers` | 1-8 | 4 | Parallel render workers |
@@ -256,6 +260,23 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
| `--docker` | — | off | Use Docker for [deterministic rendering](/concepts/determinism) |
| `--quiet` | — | off | Suppress verbose output |
#### WebM with Transparency
Use `--format webm` to render compositions with a transparent background. This produces VP9 video with alpha channel in a WebM container — the standard format for overlayable video.
```bash
# Render a caption overlay with transparent background
npx hyperframes render --format webm --output captions.webm
# Overlay on another video with FFmpeg
ffmpeg -c:v libvpx-vp9 -i captions.webm -i background.mp4 \
-filter_complex "[1:v][0:v]overlay=0:0" -y composited.mp4
```
<Tip>
For transparency to work, your composition's HTML should use `background: transparent` on the root elements. WebM renders use PNG frame capture (instead of JPEG) to preserve the alpha channel.
</Tip>
See [Rendering](/guides/rendering) for all options and modes.
### `benchmark`
+30 -11
View File
@@ -22,7 +22,7 @@ npm install @hyperframes/engine
- Implement a custom encoding backend (not FFmpeg)
**Use a different package if you want to:**
- Render an HTML composition to a finished MP4 — use the [producer](/packages/producer) or [CLI](/packages/cli)
- Render an HTML composition to a finished MP4 or WebM — use the [producer](/packages/producer) or [CLI](/packages/cli)
- Preview compositions in the browser — use the [CLI](/packages/cli) or [studio](/packages/studio)
- Lint or parse composition HTML — use [core](/packages/core)
@@ -142,7 +142,7 @@ await releaseBrowser(browser);
### Encoding
The engine includes FFmpeg encoding utilities:
The engine includes FFmpeg encoding utilities with support for MP4 (h264) and WebM (VP9 with alpha):
```typescript
import {
@@ -150,23 +150,42 @@ import {
muxVideoWithAudio,
applyFaststart,
detectGpuEncoder,
getEncoderPreset,
ENCODER_PRESETS,
} from '@hyperframes/engine';
// Get format-aware encoder settings
const mp4Preset = getEncoderPreset('standard', 'mp4');
// { codec: "h264", pixelFormat: "yuv420p", preset: "medium", quality: 23 }
const webmPreset = getEncoderPreset('standard', 'webm');
// { codec: "vp9", pixelFormat: "yuva420p", preset: "good", quality: 23 }
// Encode captured frames to video
await encodeFramesFromDir(framesDir, 'frame_%06d.png', outputPath, {
fps: 30,
...webmPreset,
});
// Mix video with audio (uses Opus for WebM, AAC for MP4)
await muxVideoWithAudio(videoPath, audioPath, outputPath);
// Apply MP4 faststart for streaming (no-op for WebM)
await applyFaststart(inputPath, outputPath);
// Detect GPU encoding support
const gpu = await detectGpuEncoder();
// gpu: "nvenc" | "videotoolbox" | "vaapi" | null
// Encode captured frames to video
await encodeFramesFromDir({ framesDir, outputPath, fps: 30, quality: 'standard' });
// Mix video with audio tracks
await muxVideoWithAudio({ videoPath, audioTracks, outputPath });
// Apply MP4 faststart for streaming
await applyFaststart(outputPath);
```
#### WebM with VP9 Alpha
When encoding for transparency, use `format: "webm"` with `getEncoderPreset()`. This configures:
- **VP9 codec** (`libvpx-vp9`) with alpha-capable `yuva420p` pixel format
- **`-auto-alt-ref 0`** and **`alpha_mode=1`** metadata for proper alpha encoding
- **`-row-mt 1`** for multi-threaded VP9 encoding
- **Opus audio** in the mux step (instead of AAC for MP4)
### Streaming Encoder
For memory-efficient encoding without writing frames to disk:
+25 -4
View File
@@ -3,7 +3,7 @@ title: "@hyperframes/producer"
description: "Full HTML-to-video rendering pipeline with encoding, audio mixing, and Docker support."
---
The producer package combines the [engine's](/packages/engine) frame capture with FFmpeg encoding to deliver a complete HTML-to-MP4 rendering pipeline. It handles runtime injection, readiness gates, audio mixing, and optional Docker-based deterministic rendering.
The producer package combines the [engine's](/packages/engine) frame capture with FFmpeg encoding to deliver a complete HTML-to-video rendering pipeline. It supports MP4 (h264) and WebM (VP9 with alpha transparency), and handles runtime injection, readiness gates, audio mixing, and optional Docker-based deterministic rendering.
```bash
npm install @hyperframes/producer
@@ -12,7 +12,7 @@ npm install @hyperframes/producer
## When to Use
**Use `@hyperframes/producer` when you need to:**
- Render compositions to MP4 programmatically from Node.js (e.g., in a backend service or CI pipeline)
- Render compositions to MP4 or WebM programmatically from Node.js (e.g., in a backend service or CI pipeline)
- Build a custom rendering service with fine-grained control over the pipeline
- Run visual regression tests against golden baselines
- Benchmark render performance across different configurations
@@ -44,8 +44,8 @@ The producer orchestrates the full render pipeline:
<Step title="Capture frames via the engine">
Uses the [engine's](/packages/engine) BeginFrame pipeline to capture each frame as a pixel buffer.
</Step>
<Step title="Encode to MP4 via FFmpeg">
Pipes frame buffers into FFmpeg with the selected quality preset and encoding settings.
<Step title="Encode to MP4 or WebM via FFmpeg">
Pipes frame buffers into FFmpeg with the selected quality preset. MP4 uses h264; WebM uses VP9 with alpha transparency support.
</Step>
<Step title="Mix audio tracks">
Extracts audio from video clips and audio elements, applies `data-volume` and `data-media-start` offsets, and mixes them into the final MP4.
@@ -77,12 +77,33 @@ import type { RenderConfig } from '@hyperframes/producer';
const config: RenderConfig = {
fps: 30, // 24, 30, or 60
quality: 'standard', // 'draft', 'standard', or 'high'
format: 'mp4', // 'mp4' or 'webm' (WebM renders with transparency)
workers: 4, // Parallel render workers (1-8)
useGpu: false, // GPU-accelerated encoding
debug: false, // Debug logging
};
```
#### WebM with Transparency
Set `format: 'webm'` to render with a transparent background using VP9 alpha:
```typescript
const job = createRenderJob({
fps: 30,
quality: 'standard',
format: 'webm',
});
await executeRenderJob(job, './my-overlay', './overlay.webm');
```
When `format: 'webm'`:
- Frames are captured as PNG (preserves alpha channel)
- Chrome's page background is set to transparent via CDP
- FFmpeg encodes with VP9 + `yuva420p` pixel format
- Audio is encoded as Opus (instead of AAC for MP4)
### Progress Callbacks
```typescript
+40 -3
View File
@@ -10,14 +10,29 @@ import { trackRenderComplete, trackRenderError } from "../telemetry/events.js";
const VALID_FPS = new Set([24, 30, 60]);
const VALID_QUALITY = new Set(["draft", "standard", "high"]);
const VALID_FORMAT = new Set(["mp4", "webm"]);
export default defineCommand({
meta: { name: "render", description: "Render a composition to MP4" },
meta: {
name: "render",
description: `Render a composition to MP4 or WebM
Examples:
hyperframes render --output output.mp4
hyperframes render --format webm --output overlay.webm # transparent WebM
hyperframes render --fps 60 --quality high --output hd.mp4
hyperframes render --docker --output deterministic.mp4`,
},
args: {
dir: { type: "positional", description: "Project directory", required: false },
output: { type: "string", description: "Output path (default: renders/<name>.mp4)" },
fps: { type: "string", description: "Frame rate: 24, 30, 60", default: "30" },
quality: { type: "string", description: "Quality: draft, standard, high", default: "standard" },
format: {
type: "string",
description: "Output format: mp4, webm (WebM renders with transparency)",
default: "mp4",
},
workers: { type: "string", description: "Parallel workers 1-8" },
docker: { type: "boolean", description: "Use Docker for deterministic render", default: false },
gpu: { type: "boolean", description: "Use GPU encoding", default: false },
@@ -43,6 +58,14 @@ export default defineCommand({
}
const quality = qualityRaw as "draft" | "standard" | "high";
// ── Validate format ─────────────────────────────────────────────────
const formatRaw = args.format ?? "mp4";
if (!VALID_FORMAT.has(formatRaw)) {
errorBox("Invalid format", `Got "${formatRaw}". Must be mp4 or webm.`);
process.exit(1);
}
const format = formatRaw as "mp4" | "webm";
// ── Validate workers ──────────────────────────────────────────────────
let workers: number | undefined;
if (args.workers != null) {
@@ -56,7 +79,10 @@ export default defineCommand({
// ── Resolve output path ───────────────────────────────────────────────
const rendersDir = resolve("renders");
const outputPath = args.output ? resolve(args.output) : join(rendersDir, `${project.name}.mp4`);
const ext = format === "webm" ? ".webm" : ".mp4";
const outputPath = args.output
? resolve(args.output)
: join(rendersDir, `${project.name}${ext}`);
// Ensure output directory exists
const outputDir = dirname(outputPath);
@@ -129,11 +155,19 @@ export default defineCommand({
// ── Render ────────────────────────────────────────────────────────────
if (useDocker) {
await renderDocker(project.dir, outputPath, { fps, quality, workers, gpu: useGpu, quiet });
await renderDocker(project.dir, outputPath, {
fps,
quality,
format,
workers,
gpu: useGpu,
quiet,
});
} else {
await renderLocal(project.dir, outputPath, {
fps,
quality,
format,
workers,
gpu: useGpu,
quiet,
@@ -146,6 +180,7 @@ export default defineCommand({
interface RenderOptions {
fps: 24 | 30 | 60;
quality: "draft" | "standard" | "high";
format: "mp4" | "webm";
workers?: number;
gpu: boolean;
quiet: boolean;
@@ -164,6 +199,7 @@ async function renderDocker(
const job = producer.createRenderJob({
fps: options.fps,
quality: options.quality,
format: options.format,
workers: options.workers,
useGpu: options.gpu,
});
@@ -206,6 +242,7 @@ async function renderLocal(
const job = producer.createRenderJob({
fps: options.fps,
quality: options.quality,
format: options.format,
workers: options.workers,
useGpu: options.gpu,
});
+1
View File
@@ -88,6 +88,7 @@ export {
applyFaststart,
detectGpuEncoder,
ENCODER_PRESETS,
getEncoderPreset,
type GpuEncoder,
} from "./services/chunkEncoder.js";
export type { EncoderOptions, EncodeResult, MuxResult } from "./services/chunkEncoder.types.js";
@@ -108,11 +108,7 @@ export async function acquireBrowser(
} else {
// Screenshot mode with renderSeek: works on all platforms.
captureMode = "screenshot";
if (headlessShell && !isLinux) {
executablePath = headlessShell;
} else {
executablePath = undefined; // Puppeteer uses its bundled Chromium
}
executablePath = headlessShell ?? undefined;
}
const ppt = await getPuppeteer();
@@ -1,5 +1,5 @@
import { describe, it, expect } from "vitest";
import { ENCODER_PRESETS } from "./chunkEncoder.js";
import { ENCODER_PRESETS, getEncoderPreset } from "./chunkEncoder.js";
describe("ENCODER_PRESETS", () => {
it("has draft, standard, and high presets", () => {
@@ -25,3 +25,40 @@ describe("ENCODER_PRESETS", () => {
expect(ENCODER_PRESETS.standard.quality).toBeLessThan(ENCODER_PRESETS.draft.quality);
});
});
describe("getEncoderPreset", () => {
it("returns h264 with yuv420p for mp4 format", () => {
const preset = getEncoderPreset("standard", "mp4");
expect(preset.codec).toBe("h264");
expect(preset.pixelFormat).toBe("yuv420p");
});
it("returns vp9 with yuva420p for webm format", () => {
const preset = getEncoderPreset("standard", "webm");
expect(preset.codec).toBe("vp9");
expect(preset.pixelFormat).toBe("yuva420p");
});
it("maps draft ultrafast to vp9 realtime deadline", () => {
const preset = getEncoderPreset("draft", "webm");
expect(preset.preset).toBe("realtime");
expect(preset.codec).toBe("vp9");
});
it("maps standard/high to vp9 good deadline", () => {
expect(getEncoderPreset("standard", "webm").preset).toBe("good");
expect(getEncoderPreset("high", "webm").preset).toBe("good");
});
it("preserves quality values across formats", () => {
for (const q of ["draft", "standard", "high"] as const) {
expect(getEncoderPreset(q, "webm").quality).toBe(ENCODER_PRESETS[q].quality);
}
});
it("defaults to mp4 when format is omitted", () => {
const preset = getEncoderPreset("standard");
expect(preset.codec).toBe("h264");
expect(preset.pixelFormat).toBe("yuv420p");
});
});
+42 -19
View File
@@ -6,7 +6,7 @@
*/
import { spawn } from "child_process";
import { existsSync, mkdirSync, readdirSync, statSync, writeFileSync } from "fs";
import { copyFileSync, existsSync, mkdirSync, readdirSync, statSync, writeFileSync } from "fs";
import { join, dirname } from "path";
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
import { type GpuEncoder, getCachedGpuEncoder, getGpuEncoderName } from "../utils/gpuEncoder.js";
@@ -21,6 +21,26 @@ export const ENCODER_PRESETS = {
high: { preset: "slow", quality: 18, codec: "h264" as const },
};
/**
* Get encoder preset for a given quality and output format.
* WebM uses VP9 with alpha-capable pixel format; MP4 uses h264.
*/
export function getEncoderPreset(
quality: "draft" | "standard" | "high",
format: "mp4" | "webm" = "mp4",
): { preset: string; quality: number; codec: "h264" | "vp9"; pixelFormat: string } {
const base = ENCODER_PRESETS[quality];
if (format === "webm") {
return {
preset: base.preset === "ultrafast" ? "realtime" : "good",
quality: base.quality,
codec: "vp9",
pixelFormat: "yuva420p",
};
}
return { ...base, pixelFormat: "yuv420p" };
}
// Re-export GPU utilities so existing consumers that import from chunkEncoder still work.
export { detectGpuEncoder, type GpuEncoder } from "../utils/gpuEncoder.js";
@@ -83,6 +103,11 @@ function buildEncoderArgs(
} else if (codec === "vp9") {
args.push("-c:v", "libvpx-vp9", "-b:v", bitrate || "0", "-crf", String(quality));
args.push("-deadline", preset === "ultrafast" ? "realtime" : "good");
args.push("-row-mt", "1");
if (pixelFormat === "yuva420p") {
args.push("-auto-alt-ref", "0");
args.push("-metadata:s:v:0", "alpha_mode=1");
}
} else if (codec === "prores") {
args.push("-c:v", "prores_ks", "-profile:v", preset, "-vendor", "apl0");
return [...args, "-y", outputPath];
@@ -243,7 +268,8 @@ export async function encodeFramesChunkedConcat(
}
const startNumber = i * chunkSize;
const framesInChunk = Math.min(chunkSize, files.length - startNumber);
const chunkPath = join(chunkDir, `chunk_${String(i).padStart(4, "0")}.mp4`);
const ext = outputPath.endsWith(".webm") ? ".webm" : ".mp4";
const chunkPath = join(chunkDir, `chunk_${String(i).padStart(4, "0")}${ext}`);
const inputPath = join(framesDir, framePattern);
const inputArgs = [
"-framerate",
@@ -347,23 +373,15 @@ export async function muxVideoWithAudio(
const outputDir = dirname(outputPath);
if (!existsSync(outputDir)) mkdirSync(outputDir, { recursive: true });
const args = [
"-i",
videoPath,
"-i",
audioPath,
"-c:v",
"copy",
"-c:a",
"aac",
"-b:a",
"192k",
"-shortest",
"-movflags",
"+faststart",
"-y",
outputPath,
];
const isWebm = outputPath.endsWith(".webm");
const args = ["-i", videoPath, "-i", audioPath, "-c:v", "copy"];
if (isWebm) {
args.push("-c:a", "libopus", "-b:a", "128k");
} else {
args.push("-c:a", "aac", "-b:a", "192k", "-movflags", "+faststart");
}
args.push("-shortest", "-y", outputPath);
const processTimeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;
const result = await runFfmpeg(args, { signal, timeout: processTimeout });
@@ -394,6 +412,11 @@ export async function applyFaststart(
signal?: AbortSignal,
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout">>,
): Promise<MuxResult> {
// faststart is MP4-only (moves moov atom to file start for streaming)
if (outputPath.endsWith(".webm")) {
if (inputPath !== outputPath) copyFileSync(inputPath, outputPath);
return { success: true, outputPath, durationMs: 0 };
}
const args = ["-i", inputPath, "-c", "copy", "-movflags", "+faststart", "-y", outputPath];
const processTimeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;
@@ -107,6 +107,16 @@ export async function createCaptureSession(
};
await page.setViewport(viewport);
// For PNG capture (used by WebM/transparency), make the page background transparent
// so Chrome's screenshot captures alpha channel data. Must use the same CDP session
// that the screenshot service uses (getCdpSession caches per page).
if (options.format === "png") {
const cdp = await getCdpSession(page);
await cdp.send("Emulation.setDefaultBackgroundColorOverride", {
color: { r: 0, g: 0, b: 0, a: 0 },
});
}
return {
browser,
page,
@@ -104,6 +104,8 @@ export async function pageScreenshotCapture(page: Page, options: CaptureOptions)
const result = await client.send("Page.captureScreenshot", {
format,
quality: format === "jpeg" ? (options.quality ?? 80) : undefined,
fromSurface: true,
captureBeyondViewport: false,
optimizeForSpeed: true,
});
return Buffer.from(result.data, "base64");
@@ -172,6 +172,11 @@ function buildStreamingArgs(
} else if (codec === "vp9") {
args.push("-c:v", "libvpx-vp9", "-b:v", bitrate || "0", "-crf", String(quality));
args.push("-deadline", preset === "ultrafast" ? "realtime" : "good");
args.push("-row-mt", "1");
if (pixelFormat === "yuva420p") {
args.push("-auto-alt-ref", "0");
args.push("-metadata:s:v:0", "alpha_mode=1");
}
} else if (codec === "prores") {
args.push("-c:v", "prores_ks", "-profile:v", preset, "-vendor", "apl0");
return [...args, "-y", outputPath];
+43 -23
View File
@@ -31,6 +31,7 @@ type TestMetadata = {
maxAudioLagWindows: number;
renderConfig: {
fps: 24 | 30 | 60;
format?: "mp4" | "webm"; // Optional: defaults to "mp4"
workers?: number; // Optional: auto-calculates if omitted
};
};
@@ -145,6 +146,9 @@ function validateMetadata(meta: unknown): TestMetadata {
if (![24, 30, 60].includes(rc.fps as number)) {
throw new Error("meta.json: 'renderConfig.fps' must be 24, 30, or 60");
}
if (rc.format !== undefined && rc.format !== "mp4" && rc.format !== "webm") {
throw new Error("meta.json: 'renderConfig.format' must be 'mp4' or 'webm' (or omit for mp4)");
}
if (rc.workers !== undefined) {
if (typeof rc.workers !== "number" || rc.workers < 1) {
throw new Error("meta.json: 'renderConfig.workers' must be >= 1 (or omit to auto-calculate)");
@@ -306,28 +310,37 @@ function psnrAtCheckpoint(
}
function extractMonoPcm16(videoPath: string): Int16Array {
const { stdout } = runFfmpeg(
[
"-hide_banner",
"-loglevel",
"error",
"-i",
videoPath,
"-vn",
"-ac",
"1",
"-ar",
"16000",
"-f",
"s16le",
"-",
],
`Audio extraction (${videoPath})`,
);
if (stdout.byteLength < 2) {
try {
const { stdout } = runFfmpeg(
[
"-hide_banner",
"-loglevel",
"error",
"-i",
videoPath,
"-vn",
"-ac",
"1",
"-ar",
"16000",
"-f",
"s16le",
"-",
],
`Audio extraction (${videoPath})`,
);
if (stdout.byteLength < 2) {
return new Int16Array(0);
}
return new Int16Array(stdout.buffer, stdout.byteOffset, Math.floor(stdout.byteLength / 2));
} catch (err) {
// No audio stream (e.g., WebM without audio) — log but don't fail
const msg = err instanceof Error ? err.message : String(err);
if (!msg.includes("does not contain any stream")) {
logPretty(`Audio extraction warning: ${msg.slice(0, 200)}`, "⚠️");
}
return new Int16Array(0);
}
return new Int16Array(stdout.buffer, stdout.byteOffset, Math.floor(stdout.byteLength / 2));
}
function buildRmsEnvelope(samples: Int16Array, windowSize = 2048, hopSize = 1024): number[] {
@@ -535,12 +548,14 @@ async function runTestSuite(
mkdirSync(tempRoot, { recursive: true });
const tempDownloadDir = join(tempRoot, "downloads");
const renderedOutputPath = join(tempRoot, "output.mp4");
const outputFormat = suite.meta.renderConfig.format ?? "mp4";
const videoExt = outputFormat === "webm" ? ".webm" : ".mp4";
const renderedOutputPath = join(tempRoot, `output${videoExt}`);
// Snapshot files stored in test's output/ directory
const snapshotDir = join(suite.dir, "output");
const snapshotCompiledPath = join(snapshotDir, "compiled.html");
const snapshotVideoPath = join(snapshotDir, "output.mp4");
const snapshotVideoPath = join(snapshotDir, `output${videoExt}`);
console.log(JSON.stringify({ event: "test_start", suite: suite.id, name: suite.meta.name }));
logPretty(`Running test: ${suite.meta.name}`, "🧪");
@@ -626,6 +641,7 @@ async function runTestSuite(
const job = createRenderJob({
fps: suite.meta.renderConfig.fps,
quality: "high", // Always use max quality for tests
format: outputFormat,
workers: suite.meta.renderConfig.workers,
useGpu: false,
debug: false,
@@ -643,7 +659,11 @@ async function runTestSuite(
}
copyFileSync(renderedOutputPath, snapshotVideoPath);
console.log(
JSON.stringify({ event: "snapshot_updated", suite: suite.id, file: "output/output.mp4" }),
JSON.stringify({
event: "snapshot_updated",
suite: suite.id,
file: `output/output${videoExt}`,
}),
);
result.visual = { passed: true, failedFrames: 0, checkpoints: [] };
result.audio = { passed: true, correlation: 1, lagWindows: 0 };
@@ -44,7 +44,7 @@ import {
encodeFramesChunkedConcat,
muxVideoWithAudio,
applyFaststart,
ENCODER_PRESETS,
getEncoderPreset,
processCompositionAudio,
type AudioElement,
calculateOptimalWorkers,
@@ -99,6 +99,8 @@ export type RenderStatus =
export interface RenderConfig {
fps: 24 | 30 | 60;
quality: "draft" | "standard" | "high";
/** Output container format. WebM uses VP9+alpha for transparency. */
format?: "mp4" | "webm";
workers?: number;
useGpu?: boolean;
debug?: boolean;
@@ -354,7 +356,13 @@ export async function executeRenderJob(
let restoreLogger: (() => void) | null = null;
const perfStages: Record<string, number> = {};
const perfOutputPath = join(workDir, "perf-summary.json");
const cfg = job.config.producerConfig ?? resolveConfig();
const cfg = { ...(job.config.producerConfig ?? resolveConfig()) };
const outputFormat = (job.config.format ?? "mp4") as "mp4" | "webm";
const isWebm = outputFormat === "webm";
// WebM/transparency requires screenshot mode — beginFrame doesn't support alpha channel
if (isWebm) {
cfg.forceScreenshot = true;
}
const enableChunkedEncode = cfg.enableChunkedEncode;
const chunkedEncodeSize = cfg.chunkSizeFrames;
const enableStreamingEncode = cfg.enableStreamingEncode;
@@ -458,8 +466,8 @@ export async function executeRenderJob(
width,
height,
fps: job.config.fps,
format: "jpeg",
quality: 80,
format: isWebm ? "png" : "jpeg",
quality: isWebm ? undefined : 80,
};
probeSession = await createCaptureSession(
fileServer.url,
@@ -702,14 +710,15 @@ export async function executeRenderJob(
width,
height,
fps: job.config.fps,
format: "jpeg",
quality: job.config.quality === "draft" ? 80 : 95,
format: isWebm ? "png" : "jpeg",
quality: isWebm ? undefined : job.config.quality === "draft" ? 80 : 95,
};
const workerCount = calculateOptimalWorkers(job.totalFrames!, job.config.workers, cfg);
const videoOnlyPath = join(workDir, "video-only.mp4");
const preset = ENCODER_PRESETS[job.config.quality];
const videoExt = isWebm ? ".webm" : ".mp4";
const videoOnlyPath = join(workDir, `video-only${videoExt}`);
const preset = getEncoderPreset(job.config.quality, outputFormat);
job.framesRendered = 0;
@@ -727,6 +736,7 @@ export async function executeRenderJob(
codec: preset.codec,
preset: preset.preset,
quality: preset.quality,
pixelFormat: preset.pixelFormat,
useGpu: job.config.useGpu,
imageFormat: captureOptions.format || "jpeg",
},
@@ -940,36 +950,32 @@ export async function executeRenderJob(
const stage5Start = Date.now();
updateJobStatus(job, "encoding", "Encoding video", 75, onProgress);
const frameExt = isWebm ? "png" : "jpg";
const framePattern = `frame_%06d.${frameExt}`;
const encoderOpts = {
fps: job.config.fps,
width,
height,
codec: preset.codec,
preset: preset.preset,
quality: preset.quality,
pixelFormat: preset.pixelFormat,
useGpu: job.config.useGpu,
};
const encodeResult = enableChunkedEncode
? await encodeFramesChunkedConcat(
framesDir,
"frame_%06d.jpg",
framePattern,
videoOnlyPath,
{
fps: job.config.fps,
width,
height,
codec: preset.codec,
preset: preset.preset,
quality: preset.quality,
useGpu: job.config.useGpu,
},
encoderOpts,
chunkedEncodeSize,
abortSignal,
)
: await encodeFramesFromDir(
framesDir,
"frame_%06d.jpg",
framePattern,
videoOnlyPath,
{
fps: job.config.fps,
width,
height,
codec: preset.codec,
preset: preset.preset,
quality: preset.quality,
useGpu: job.config.useGpu,
},
encoderOpts,
abortSignal,
);
assertNotAborted();
@@ -1060,7 +1066,7 @@ export async function executeRenderJob(
if (job.config.debug) {
// Copy output MP4 into debug dir for easy access
if (existsSync(outputPath)) {
const debugOutput = join(workDir, "output.mp4");
const debugOutput = join(workDir, isWebm ? "output.webm" : "output.mp4");
copyFileSync(outputPath, debugOutput);
}
} else {
@@ -0,0 +1,14 @@
{
"name": "webm-transparency",
"description": "WebM VP9 render with transparent background (alpha channel). Validates the full WebM pipeline: PNG capture, VP9 encoding with yuva420p, and alpha_mode=1 metadata.",
"tags": ["webm", "transparency", "fast"],
"minPsnr": 30,
"maxFrameFailures": 2,
"minAudioCorrelation": 0.0,
"maxAudioLagWindows": 120,
"renderConfig": {
"fps": 30,
"format": "webm",
"workers": 1
}
}
@@ -0,0 +1,49 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<style>
* { margin: 0; padding: 0; }
html, body { background: transparent; overflow: hidden; }
</style>
</head>
<body>
<div id="root" data-composition-id="main" data-start="0" data-duration="2" data-width="640" data-height="360" style="width: 640px; height: 360px; background: transparent; position: relative">
<!-- Semi-transparent red box that animates across the frame -->
<div id="box" data-start="0" data-duration="2" data-track-index="0" style="
width: 120px;
height: 120px;
background: rgba(255, 0, 0, 0.7);
border-radius: 16px;
position: absolute;
top: 120px;
left: 50px;
"></div>
<!-- White text with transparent background -->
<div id="label" data-start="0.5" data-duration="1.5" data-track-index="1" style="
font-family: sans-serif;
font-size: 32px;
font-weight: bold;
color: white;
text-shadow: 0 2px 8px rgba(0,0,0,0.5);
position: absolute;
top: 280px;
left: 50px;
opacity: 0;
">Overlay Test</div>
<script src="https://cdn.jsdelivr.net/npm/gsap@3/dist/gsap.min.js"></script>
</div>
<script>window.__timelines = window.__timelines || {};
const tl = gsap.timeline({ paused: true });
// Animate box sliding right
tl.to("#box", { left: 470, duration: 2, ease: "power2.inOut" }, 0);
// Fade in label
tl.to("#label", { opacity: 1, duration: 0.5, ease: "power2.out" }, 0.5);
window.__timelines["main"] = tl;</script></body>
</html>
@@ -0,0 +1,71 @@
<!doctype html>
<html>
<head>
<meta charset="UTF-8" />
<style>
* { margin: 0; padding: 0; }
html, body { background: transparent; overflow: hidden; }
</style>
</head>
<body>
<div
id="root"
data-composition-id="main"
data-start="0"
data-duration="2"
data-width="640"
data-height="360"
style="width: 640px; height: 360px; background: transparent; position: relative"
>
<!-- Semi-transparent red box that animates across the frame -->
<div
id="box"
data-start="0"
data-duration="2"
data-track-index="0"
style="
width: 120px;
height: 120px;
background: rgba(255, 0, 0, 0.7);
border-radius: 16px;
position: absolute;
top: 120px;
left: 50px;
"
></div>
<!-- White text with transparent background -->
<div
id="label"
data-start="0.5"
data-duration="1.5"
data-track-index="1"
style="
font-family: sans-serif;
font-size: 32px;
font-weight: bold;
color: white;
text-shadow: 0 2px 8px rgba(0,0,0,0.5);
position: absolute;
top: 280px;
left: 50px;
opacity: 0;
"
>Overlay Test</div>
<script src="https://cdn.jsdelivr.net/npm/gsap@3/dist/gsap.min.js"></script>
<script>
window.__timelines = window.__timelines || {};
const tl = gsap.timeline({ paused: true });
// Animate box sliding right
tl.to("#box", { left: 470, duration: 2, ease: "power2.inOut" }, 0);
// Fade in label
tl.to("#label", { opacity: 1, duration: 0.5, ease: "power2.out" }, 0.5);
window.__timelines["main"] = tl;
</script>
</div>
</body>
</html>