mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-11 06:30:03 +00:00
James
9ff46d79b7
Adds the first two test fixtures the skill is graded against. Each fixture
ships:
- remotion-src/ full Remotion project (package.json, src/, remotion.config.ts, tsconfig.json)
- hf-src/ hand-translated HyperFrames composition (index.html)
- expected.json tier metadata + SSIM threshold + translation notes + measured validation
- README.md human walk-through of the translation choices
- setup.sh (T2 only) generates binary assets (PNG, WAV) via ffmpeg
T1 — title-card-fade
- 3 s @ 30 fps, 1280x720
- Single AbsoluteFill, single useCurrentFrame interpolate
with multi-segment input [0,15,75,90] -> [0,1,1,0]
- Validated mean SSIM 0.974, threshold 0.95
(~0.025 gap from font-fallback divergence between Remotion's bundled
Chromium and HF's chrome-headless-shell)
T2 — title-image-outro
- 6 s @ 30 fps, 1280x720, three Sequences (TitleScene, ImageScene, OutroScene)
- Exercises spring, interpolate, Audio, Img, staticFile
- Spring -> GSAP back.out(1.4) translation
- Validated mean SSIM 0.985, threshold 0.95
(translation came out cleaner than predicted; spring->back.out drift was
smaller than the ~0.05 budget I'd expected)
- setup.sh generates a 200x200 blue PNG and a 6 s silent WAV via ffmpeg
so binaries stay out of the repo
Calibration done end-to-end: rendered Remotion baseline + HF translation,
ran scripts/render_diff.sh, set thresholds ~0.02 below measured p05.
Critical Remotion config: setVideoImageFormat("png") + setColorSpace("bt709").
The default JPEG output writes yuvj420p (full-range) which costs ~0.05 SSIM
vs HF's yuv420p (limited-range). Both fixtures' remotion.config.ts encode
this so render_diff.sh measures translation fidelity, not encoder differences.
Both fixtures lint clean (0 blockers via scripts/lint_source.py).
T2 staticFile() references correctly flagged as info-level findings.
The fixtures are not yet wired into CI — that comes with PR 7's orchestrator.
For now, render and eval are documented in each README and run by hand.
35 lines
1.2 KiB
Bash
Executable File
35 lines
1.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# setup.sh — generate the binary assets this fixture needs.
|
|
#
|
|
# Both Remotion and HyperFrames variants need a 200x200 blue PNG and a
|
|
# 6-second silent WAV. Generating them via ffmpeg keeps binaries out of
|
|
# the repo while still letting the fixture render reproducibly.
|
|
#
|
|
# Run from the fixture root: ./setup.sh
|
|
|
|
set -euo pipefail
|
|
|
|
THIS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
|
|
if ! command -v ffmpeg >/dev/null 2>&1; then
|
|
echo "error: ffmpeg not on PATH" >&2
|
|
exit 2
|
|
fi
|
|
|
|
mkdir -p "$THIS_DIR/remotion-src/public" "$THIS_DIR/hf-src/assets"
|
|
|
|
# 200x200 solid blue PNG, ~200 bytes.
|
|
ffmpeg -y -hide_banner -loglevel error \
|
|
-f lavfi -i "color=color=#3066be:size=200x200" -frames:v 1 \
|
|
"$THIS_DIR/remotion-src/public/square.png"
|
|
cp "$THIS_DIR/remotion-src/public/square.png" "$THIS_DIR/hf-src/assets/square.png"
|
|
|
|
# 6-second silent WAV at 8 kHz mono. ~96 KB if checked in, but it is generated.
|
|
ffmpeg -y -hide_banner -loglevel error \
|
|
-f lavfi -i "anullsrc=cl=mono:r=8000" -t 6 -acodec pcm_s16le \
|
|
"$THIS_DIR/remotion-src/public/music.wav"
|
|
cp "$THIS_DIR/remotion-src/public/music.wav" "$THIS_DIR/hf-src/assets/music.wav"
|
|
|
|
echo "generated:"
|
|
ls -la "$THIS_DIR/remotion-src/public/" "$THIS_DIR/hf-src/assets/"
|