mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-11 14:50:02 +00:00
James
9ff46d79b7
Adds the first two test fixtures the skill is graded against. Each fixture
ships:
- remotion-src/ full Remotion project (package.json, src/, remotion.config.ts, tsconfig.json)
- hf-src/ hand-translated HyperFrames composition (index.html)
- expected.json tier metadata + SSIM threshold + translation notes + measured validation
- README.md human walk-through of the translation choices
- setup.sh (T2 only) generates binary assets (PNG, WAV) via ffmpeg
T1 — title-card-fade
- 3 s @ 30 fps, 1280x720
- Single AbsoluteFill, single useCurrentFrame interpolate
with multi-segment input [0,15,75,90] -> [0,1,1,0]
- Validated mean SSIM 0.974, threshold 0.95
(~0.025 gap from font-fallback divergence between Remotion's bundled
Chromium and HF's chrome-headless-shell)
T2 — title-image-outro
- 6 s @ 30 fps, 1280x720, three Sequences (TitleScene, ImageScene, OutroScene)
- Exercises spring, interpolate, Audio, Img, staticFile
- Spring -> GSAP back.out(1.4) translation
- Validated mean SSIM 0.985, threshold 0.95
(translation came out cleaner than predicted; spring->back.out drift was
smaller than the ~0.05 budget I'd expected)
- setup.sh generates a 200x200 blue PNG and a 6 s silent WAV via ffmpeg
so binaries stay out of the repo
Calibration done end-to-end: rendered Remotion baseline + HF translation,
ran scripts/render_diff.sh, set thresholds ~0.02 below measured p05.
Critical Remotion config: setVideoImageFormat("png") + setColorSpace("bt709").
The default JPEG output writes yuvj420p (full-range) which costs ~0.05 SSIM
vs HF's yuv420p (limited-range). Both fixtures' remotion.config.ts encode
this so render_diff.sh measures translation fidelity, not encoder differences.
Both fixtures lint clean (0 blockers via scripts/lint_source.py).
T2 staticFile() references correctly flagged as info-level findings.
The fixtures are not yet wired into CI — that comes with PR 7's orchestrator.
For now, render and eval are documented in each README and run by hand.
27 lines
1.7 KiB
JSON
27 lines
1.7 KiB
JSON
{
|
|
"tier": 1,
|
|
"name": "title-card-fade",
|
|
"composition_id": "TitleCard",
|
|
"description": "Solid black background, single 'HELLO' element fades in 0-0.5s, holds 0.5-2.5s, fades out 2.5-3.0s. Tests the most basic Remotion → HyperFrames translation: a single AbsoluteFill, a single useCurrentFrame-driven interpolate, no audio, no media, no custom React components.",
|
|
"duration_seconds": 3,
|
|
"fps": 30,
|
|
"width": 1280,
|
|
"height": 720,
|
|
"ssim_threshold": 0.95,
|
|
"validation": {
|
|
"measured_mean_ssim": 0.974,
|
|
"measured_min_ssim": 0.972,
|
|
"measured_p05_ssim": 0.972,
|
|
"measured_p95_ssim": 0.983,
|
|
"measured_at": "2026-04-27",
|
|
"measured_against": "remotion@4.0 (PNG output, BT.709) vs hyperframes@0.4.15-alpha.1"
|
|
},
|
|
"translation_notes": [
|
|
"Remotion: AbsoluteFill → HF: position:absolute;inset:0 div",
|
|
"Remotion: interpolate(frame, [0,15,75,90], [0,1,1,0]) at fps=30 → HF: paused GSAP timeline with three keyframed tweens at 0s, 0.5s, 2.5s with ease:'none' (linear matches Remotion's default linear interpolation)",
|
|
"No fonts loaded; both renderers use system Helvetica/Arial fallback. The Linux fallback diverges between Remotion's bundled Chromium and HyperFrames' chrome-headless-shell — same fontWeight:800 renders perceptibly bolder in HF. This costs ~0.025 mean SSIM and is the dominant non-translation noise floor.",
|
|
"Remotion config must use setVideoImageFormat('png') + setColorSpace('bt709'); the JPEG default writes yuvj420p (full-range) which costs ~0.05 SSIM vs HF's yuv420p (limited-range)."
|
|
],
|
|
"rationale": "Threshold 0.95 sits ~0.02 below measured p05. A real translation regression (wrong easing, wrong durations) drops mean SSIM by 0.05+. Encoder/font drift between CI runs is bounded at ~0.01."
|
|
}
|