Files
James 70e0b8bf87 feat(skills): remotion-to-hyperframes eval harness (2/7)
Adds the deterministic eval primitives the skill calls into:

  scripts/render_diff.sh    SSIM diff between two MP4s, JSON summary, configurable threshold
  scripts/frame_strip.sh    side-by-side comparison strip for visual debugging
  scripts/lint_source.py    pre-translation lint over Remotion source — blocks/warnings/infos

The harness is decoupled from the render pipeline: it accepts paths to
already-rendered MP4s. The skill orchestrator (PR 7) drives both renders
and feeds the outputs in. This keeps the harness usable in CI, in
sandboxes, and on any machine that has ffmpeg without needing the full
Remotion + HyperFrames toolchain.

Lint catches the patterns from the skill's out-of-scope list:
- useState / useReducer (state-machine driven animation)
- useEffect with deps (side effects)
- async calculateMetadata (Promise-returning composition metadata)
- @remotion/lambda imports
- third-party React UI libraries (MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI)
- delayRender / useCallback / useMemo (warnings)
- staticFile / interpolateColors (info — translatable but flagged)

Smoke test (scripts/tests/smoke.sh) exercises all three scripts against
synthetic inputs: identical ffmpeg testsrc videos pass at threshold 0.99,
different ffmpeg testsrc videos fail at 0.99, frame_strip produces a
strip.png, lint produces 0 blockers on a clean fixture and >=3 blockers
on a fixture that uses useState + useEffect + MUI + async metadata.

Validated locally: smoke.sh exits 0.
2026-04-27 23:54:09 +00:00

51 lines
1.8 KiB
TypeScript

import React, { useState, useEffect, useLayoutEffect } from "react";
import { useCurrentFrame, AbsoluteFill, delayRender, continueRender } from "remotion";
import { Button } from "@mui/material";
// Custom hook in `export const useFoo = ...` form — earlier custom-hook
// regex anchored to `^\s*(?:function|const|let)` and missed the `export`
// prefix. This covers the regression.
export const useFadeMixed = (n: number) => {
const f = useCurrentFrame();
return f / n;
};
export const BadComposition: React.FC = () => {
const frame = useCurrentFrame();
const [data, setData] = useState<string | null>(null);
const [handle] = useState(() => delayRender());
// Multi-line useEffect body with commas inside (fillRect args) — regression
// coverage for r2hf/use-effect-deps. An earlier regex `[^,]+` would stop at
// the first comma inside the body and miss the deps array entirely.
useEffect(() => {
fetch("/api/data")
.then((r) => r.json())
.then((d) => {
const ctx = document.createElement("canvas").getContext("2d");
ctx?.fillRect(0, 0, 100, 100);
setData(d.text);
continueRender(handle);
});
}, [handle]);
// Expression-bodied useEffect — the form `useEffect(() => fetch(...), [deps])`
// has no closing `}`, which an earlier regex anchored on. This and the
// useLayoutEffect below cover the false-negative cases Miguel surfaced.
useEffect(() => fetch("/api/heartbeat"), [frame]);
useLayoutEffect(() => (document.title = `frame ${frame}`), [frame]);
return (
<AbsoluteFill>
<Button>{data ?? "loading"}</Button>
<span>{frame}</span>
</AbsoluteFill>
);
};
export const calculateMetadata = async () => {
const res = await fetch("/api/duration");
const { duration } = await res.json();
return { durationInFrames: duration };
};