test(producer): add variables-prod regression test for the variables stack

End-to-end Docker regression test that exercises the full variables
chain: meta.json renderConfig.variables → harness → createRenderJob →
RenderConfig → CaptureOptions → engine evaluateOnNewDocument →
window.__hfVariables → getVariables() → DOM text → rendered pixels.

Fixture (packages/producer/tests/variables-prod/):
- src/index.html: composition with three declared variables (title,
  subtitle, bgColor) read via window.__hyperframes.getVariables() and
  rendered as positioned text on a colored background. No animation —
  keeps the regression frame-stable so it isolates "did the variables
  flow through?" from motion concerns.
- meta.json: tags ["variables", "composition"] (runs in the existing
  fast shard's tag filter), renderConfig.variables provides override
  values the baseline reflects ("Override Title", "Override subtitle",
  #0a3d62). Defaults would produce a visibly different frame, so a
  failing baseline that reflects defaults means the variables didn't
  propagate.
- output/output.mp4: Docker-generated baseline per the project's
  CLAUDE.md golden-baseline rule.

Harness change (packages/producer/src/regression-harness.ts):
- TestMetadata.renderConfig gains an optional variables field,
  validated as a JSON object in the meta.json validator.
- The createRenderJob call site forwards renderConfig.variables to
  RenderConfig.variables, which the engine already consumes via
  evaluateOnNewDocument (PR #600).

Verified: docker:test variables-prod passes 100/100 visual checkpoints
and audio correlation 1.000 against the committed baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
James
2026-05-04 20:33:43 +00:00
committed by James Russo
co-authored by Claude Opus 4.7
parent 62c2589839
commit 58b4234809
6 changed files with 268 additions and 0 deletions
@@ -36,6 +36,14 @@ type TestMetadata = {
workers?: number; // Optional: auto-calculates if omitted
/** Force HDR in the harness; omitted/false preserves historical SDR-only test behavior. */
hdr?: boolean;
/**
* Render-time variable overrides, equivalent to `hyperframes render
* --variables '<json>'`. Injected as `window.__hfVariables` before any
* page script runs so the runtime helper `getVariables()` returns the
* merged result of declared defaults (`data-composition-variables`)
* and these overrides. Omit when the test doesn't exercise variables.
*/
variables?: Record<string, unknown>;
};
};
@@ -160,6 +168,12 @@ function validateMetadata(meta: unknown): TestMetadata {
if (rc.hdr !== undefined && typeof rc.hdr !== "boolean") {
throw new Error("meta.json: 'renderConfig.hdr' must be a boolean (or omit for false)");
}
if (
rc.variables !== undefined &&
(rc.variables === null || typeof rc.variables !== "object" || Array.isArray(rc.variables))
) {
throw new Error("meta.json: 'renderConfig.variables' must be a JSON object (or omitted)");
}
return m as TestMetadata;
}
@@ -601,6 +615,7 @@ async function runTestSuite(
useGpu: false,
debug: false,
hdrMode: suite.meta.renderConfig.hdr ? "force-hdr" : "force-sdr",
variables: suite.meta.renderConfig.variables,
});
await executeRenderJob(job, tempSrcDir, renderedOutputPath);
@@ -0,0 +1,17 @@
{
"name": "Variables Prod",
"description": "End-to-end regression for the variables system. Composition declares title/subtitle/bgColor in data-composition-variables and reads them via window.__hyperframes.getVariables(). renderConfig.variables provides overrides that flow CLI → producer → engine evaluateOnNewDocument → window.__hfVariables → helper merge → DOM text → rendered pixels. If any link in that chain breaks, the rendered text/background diverges from the baseline and PSNR fails.",
"tags": ["variables", "composition"],
"minPsnr": 30,
"maxFrameFailures": 0,
"minAudioCorrelation": 0.9,
"maxAudioLagWindows": 120,
"renderConfig": {
"fps": 24,
"variables": {
"title": "Override Title",
"subtitle": "Override subtitle",
"bgColor": "#0a3d62"
}
}
}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:be9a5689e27272b191fa566843e45f88f728b36f677a3db49f89e9193627a667
size 117111
@@ -0,0 +1,96 @@
<!doctype html>
<html
lang="en"
data-composition-variables='[
{"id":"title","type":"string","label":"Title","default":"Default Title"},
{"id":"subtitle","type":"string","label":"Subtitle","default":"Default subtitle"},
{"id":"bgColor","type":"color","label":"Background","default":"#0b0b0d"}
]'
>
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=1920, height=1080" />
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
html,
body {
margin: 0;
width: 1920px;
height: 1080px;
overflow: hidden;
}
body {
background: #0b0b0d;
font-family: -apple-system, "Helvetica Neue", Arial, sans-serif;
}
[data-composition-id="main"] {
width: 100%;
height: 100%;
position: relative;
}
.title {
position: absolute;
top: 380px;
left: 0;
right: 0;
font-size: 140px;
font-weight: 700;
color: #ffffff;
text-align: center;
line-height: 1;
}
.subtitle {
position: absolute;
top: 600px;
left: 0;
right: 0;
font-size: 56px;
color: #9aa0a6;
text-align: center;
line-height: 1;
}
</style>
</head>
<body>
<div
id="root"
data-composition-id="main"
data-start="0"
data-duration="3"
data-width="1920"
data-height="1080"
>
<audio
id="silence-track"
src="silence.wav"
data-start="0"
data-duration="3"
data-track-index="0"
></audio>
<h1 id="title-el" class="title clip" data-start="0" data-duration="3"></h1>
<p id="subtitle-el" class="subtitle clip" data-start="0" data-duration="3"></p>
</div>
<script>
// Render-time values: declared defaults from data-composition-variables
// merged with overrides from meta.json's renderConfig.variables (which
// arrive via window.__hfVariables, injected by the engine).
const vars = window.__hyperframes.getVariables();
document.body.style.background = vars.bgColor;
document.getElementById("title-el").textContent = vars.title;
document.getElementById("subtitle-el").textContent = vars.subtitle;
window.__timelines = window.__timelines || {};
const tl = gsap.timeline({ paused: true });
// Static placement — no animation. Keeps the regression frame-stable
// and isolates "did the variables flow through?" from motion concerns.
tl.to({}, { duration: 2 });
window.__timelines["main"] = tl;
</script>
</body>
</html>