mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-10 22:20:14 +00:00
James
9ff46d79b7
Adds the first two test fixtures the skill is graded against. Each fixture
ships:
- remotion-src/ full Remotion project (package.json, src/, remotion.config.ts, tsconfig.json)
- hf-src/ hand-translated HyperFrames composition (index.html)
- expected.json tier metadata + SSIM threshold + translation notes + measured validation
- README.md human walk-through of the translation choices
- setup.sh (T2 only) generates binary assets (PNG, WAV) via ffmpeg
T1 — title-card-fade
- 3 s @ 30 fps, 1280x720
- Single AbsoluteFill, single useCurrentFrame interpolate
with multi-segment input [0,15,75,90] -> [0,1,1,0]
- Validated mean SSIM 0.974, threshold 0.95
(~0.025 gap from font-fallback divergence between Remotion's bundled
Chromium and HF's chrome-headless-shell)
T2 — title-image-outro
- 6 s @ 30 fps, 1280x720, three Sequences (TitleScene, ImageScene, OutroScene)
- Exercises spring, interpolate, Audio, Img, staticFile
- Spring -> GSAP back.out(1.4) translation
- Validated mean SSIM 0.985, threshold 0.95
(translation came out cleaner than predicted; spring->back.out drift was
smaller than the ~0.05 budget I'd expected)
- setup.sh generates a 200x200 blue PNG and a 6 s silent WAV via ffmpeg
so binaries stay out of the repo
Calibration done end-to-end: rendered Remotion baseline + HF translation,
ran scripts/render_diff.sh, set thresholds ~0.02 below measured p05.
Critical Remotion config: setVideoImageFormat("png") + setColorSpace("bt709").
The default JPEG output writes yuvj420p (full-range) which costs ~0.05 SSIM
vs HF's yuv420p (limited-range). Both fixtures' remotion.config.ts encode
this so render_diff.sh measures translation fidelity, not encoder differences.
Both fixtures lint clean (0 blockers via scripts/lint_source.py).
T2 staticFile() references correctly flagged as info-level findings.
The fixtures are not yet wired into CI — that comes with PR 7's orchestrator.
For now, render and eval are documented in each README and run by hand.
Tier 2 — title-image-outro
What it tests
Three-scene composition. Each scene exercises a different Remotion idiom:
- Scene 1 (0–2 s) — TitleScene with
spring({damping:12, stiffness:100, mass:1})driving atransform: scale()on text. Tests the lossyspring → GSAP easetranslation. - Scene 2 (2–4 s) — ImageScene that fades in a
staticFile-loaded image and linearly scales it from 0.8 → 1.0. Tests asset paths + linearinterpolate. - Scene 3 (4–6 s) — OutroScene with a 1-s linear fade-in. Sanity check after the harder scenes.
A silent 6-second WAV plays throughout at volume={0.5}. Tests <Audio> translation.
If a translation passes T2, the skill correctly handles <Sequence> boundaries,
<Audio> / <Img> / staticFile, and the Remotion spring → GSAP ease heuristic.
Translation walk-through
| Remotion | HyperFrames |
|---|---|
<Sequence from={0} durationInFrames={60}> |
<div data-start="0" data-duration="2" data-track-index="0"> |
spring({frame, fps, config: {damping:12, stiffness:100, mass:1}}) |
gsap.to(target, { scale: 1, duration: 0.7, ease: "back.out(1.4)" }) |
<Audio src={staticFile("music.wav")} volume={0.5} /> |
<audio src="assets/music.wav" data-start="0" data-duration="6" data-volume="0.5" data-track-index="1"> |
<Img src={staticFile("square.png")} /> |
<img src="assets/square.png"> (with setup.sh copying into both trees) |
interpolate(frame, [0, 15], [0, 1]) at 30 fps |
gsap.to(target, { opacity: 1, duration: 0.5, ease: "none" }) |
The scene crossfading is a HyperFrames idiom, not a Remotion one: at scene boundaries
we gsap.set(scene, { opacity: 0 }) so the previous scene disappears at the
right time. Remotion does this implicitly by virtue of <Sequence>'s durationInFrames.
How to render and evaluate
# 1. Generate the binary assets (PNG + WAV) via ffmpeg
./setup.sh
# 2. Render Remotion baseline
cd remotion-src && npm install && npm run render
# 3. Render HyperFrames translation
cd ../hf-src && npx hyperframes render --output ../hf.mp4
# 4. Compare
../../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diff
Why threshold 0.95?
Same threshold as T1 (expected.json codifies it for the orchestrator). Spring → back.out(1.4)
came in cleaner than predicted during calibration — the validated mean is 0.985 against the
0.95 gate. If the translation breaks anything else (spring overshoot wrong, stagger off,
asset path drift), mean SSIM will fall well below 0.95 — that's the failure signal.