mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 07:19:52 +00:00
feat(core,figma-skill): mechanical motion translation + objective fidelity gate
Two guarantees so figma-motion imports can't drift from the design again: - motionContextToDocs(): raw get_motion_context response -> MotionDoc[], in code. Parses the motion.dev snippets (the reliable encoding; the CSS snippets stretch durations and can disagree), strips loop-wrap tail keyframes (sub-ms segments at the window end are the loop reset, not authored motion), preserves bezier eases verbatim. Fixture test uses the verbatim response from a real Motion timeline whose translation was frame-validated against Figma's own export_video render. - skills/figma/scripts/verify-motion.mjs: mandatory post-render gate. Compares motion-energy deltas between the render and the export_video ground truth so static import fidelity cancels out and the score isolates choreography. Calibrated on a faithful translation (min 20.3dB) vs a diverging one (min 5.0dB); threshold 15dB. The skill's Motion step now routes through both: no hand transcription, no unverified completion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
44653e186a
commit
a2243f7586
@@ -88,8 +88,8 @@ Node tree → editable HTML at exact figma geometry, packaged as a registry item
|
||||
No REST equivalent exists. You drive the MCP tools, then hand output to the pure helpers in `@hyperframes/core/figma`:
|
||||
|
||||
1. `get_motion_context(fileKey, nodeId)` — use `recursive:true` on the parent frame (one call for the whole scene, not one per element). Save the raw JSON next to the project (`.media/figma-cache/`) so retranslation is free.
|
||||
2. Normalize into a `MotionDoc`: per animated property a `MotionTrack` { property (motion.dev name), values, times (0..1), ease[] (named or `[x1,y1,x2,y2]` bezier), duration, repeat }. Selector = the element's stable id (`#<id>` from Phase-3 output or the authored scene). **Translate values VERBATIM — never paraphrase, simplify, or invent keyframes.** Decoding rules (field-tested): the response carries two encodings — the motion.dev snippet is the timeline-cohort window (all tracks share the cohort duration), the CSS snippet may stretch per-track durations; use ONE encoding consistently (prefer motion.dev, cohort-windowed). Keyframes at times ≈0.9999→1 are **loop-wrap markers** (the instant reset at the loop boundary), NOT authored motion — drop them and realize the wrap via the tween's `repeat` restart; inventing a visible return/fade-out where the source has a wrap snap is the known failure mode.
|
||||
2b. **Validate against ground truth before calling it done**: `export_video` on the cohort's `rootNodeId` gives Figma's own render of the timeline. Extract a frame grid from both videos at the same interval (e.g. `fps=5` contact sheets) and compare — element positions, fade states, and rotation phase must match frame-for-frame. A translation that hasn't been compared to the export is unverified.
|
||||
2. Normalize into `MotionDoc`s with `motionContextToDocs(rawResponse, { selectorFor, repeat })` from `@hyperframes/core/figma` — **never transcribe keyframe numbers by hand**. The helper encodes the field-tested decoding rules mechanically: it parses the motion.dev snippets (the reliable encoding — the CSS snippets stretch durations and can disagree; they are ignored), strips loop-wrap tail keyframes (sub-millisecond segments at times ≈0.9999→1 are the loop's instant reset, not authored motion — the wrap is realized by `repeat` restart), and preserves bezier eases verbatim. `selectorFor` must return the ids from the Phase-3 component import — don't derive selectors from node names.
|
||||
2b. **Validate against ground truth before calling it done — mandatory**: `export_video` on the cohort's `rootNodeId` gives Figma's own render of the timeline. Run `node skills/figma/scripts/verify-motion.mjs --reference <export.mp4> --render <render.mp4> --crop WxH+X+Y` — it compares motion-energy deltas (static import fidelity cancels out) and fails below 15dB min motion-PSNR (calibrated: faithful ≈ 20+, diverging ≈ 5). Measure `--crop` from the render's actual card edges, don't guess. FAIL means re-check the translation, not the threshold.
|
||||
3. `motionToGsap(doc)` → `emitTimelineScript(spec)` → inject as a `<script>` after the GSAP + CustomEase CDN tags. Paused, finite, registered on `window.__timelines` with a literal key.
|
||||
4. Untranslatable track (shader-driven, unsupported prop, complex masks) → bake: `export_video` → freeze MP4 → embed as `<video class="clip">`. Exception: shader-driven tracks — figma's export path flattens shaders to the base color (see Shaders below), so a bake there silently loses the shader; ask the user for a native figma export instead. Always say which path you used and why. Named eases outside the mapped set fall back to linear — the mapping table lives in `motionEase.ts`; flag the fallback to the user when it fires.
|
||||
5. Run `npx hyperframes lint && npx hyperframes validate` before calling it done.
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Objective fidelity gate for figma-motion imports (skill step 2b).
|
||||
*
|
||||
* Compares the HyperFrames render against Figma's own `export_video` output
|
||||
* using MOTION-ENERGY deltas: for each sample window [t, t+interval], the
|
||||
* frame difference ref(t+i)-ref(t) is compared (PSNR) against
|
||||
* render(t+i)-render(t). Static import divergence (fonts, rasterized edges,
|
||||
* subpixel geometry — the hybrid-fidelity ceiling) cancels out of both
|
||||
* deltas, so the score isolates choreography: trajectories, timing, easing.
|
||||
*
|
||||
* Calibration (SDS "Unlocked" card, 2026-07): a faithful translation scored
|
||||
* min 20.3dB / mean 27.7dB; a diverging one (invented retract keyframes,
|
||||
* wrong durations) scored min 5.0dB / mean 23.1dB. Default threshold 15dB
|
||||
* sits between with margin on both sides.
|
||||
*
|
||||
* node verify-motion.mjs --reference figma-export.mp4 --render out.mp4 \
|
||||
* [--crop WxH+X+Y] [--interval 0.2] [--min-motion-psnr 15]
|
||||
*
|
||||
* --crop selects the card region inside the (usually larger) composition
|
||||
* frame. Measure it from the render (the card's left/top edge + scaled
|
||||
* size), don't guess: a wrong crop reads as motion divergence.
|
||||
*/
|
||||
import { execFileSync, execSync } from "node:child_process";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
function arg(name, fallback) {
|
||||
const i = process.argv.indexOf(`--${name}`);
|
||||
return i > -1 ? process.argv[i + 1] : fallback;
|
||||
}
|
||||
const reference = arg("reference");
|
||||
const render = arg("render");
|
||||
if (!reference || !render) {
|
||||
console.error(
|
||||
"usage: verify-motion.mjs --reference ref.mp4 --render out.mp4 [--crop WxH+X+Y] [--interval 0.2] [--min-motion-psnr 15]",
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
const crop = arg("crop", null);
|
||||
const interval = Number(arg("interval", "0.2"));
|
||||
const minMotion = Number(arg("min-motion-psnr", "15"));
|
||||
|
||||
const ffprobe = (file) =>
|
||||
Number(
|
||||
execFileSync("ffprobe", ["-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", file])
|
||||
.toString()
|
||||
.trim(),
|
||||
);
|
||||
const refDur = ffprobe(reference);
|
||||
const renderDur = ffprobe(render);
|
||||
const end = Math.min(refDur, renderDur) - interval - 0.01;
|
||||
|
||||
const dims = execFileSync("ffprobe", ["-v", "error", "-select_streams", "v", "-show_entries", "stream=width,height", "-of", "csv=p=0", reference])
|
||||
.toString().trim().split(",").map(Number);
|
||||
const [rw, rh] = dims;
|
||||
|
||||
let cropFilter = "";
|
||||
if (crop) {
|
||||
const m = crop.match(/^(\d+)x(\d+)\+(\d+)\+(\d+)$/);
|
||||
if (!m) { console.error("bad --crop, expected WxH+X+Y"); process.exit(2); }
|
||||
cropFilter = `crop=${m[1]}:${m[2]}:${m[3]}:${m[4]},`;
|
||||
}
|
||||
|
||||
const dir = mkdtempSync(join(tmpdir(), "verify-motion-"));
|
||||
const frame = (src, t, vf, dst) => {
|
||||
const args = ["-y", "-v", "error", "-ss", String(t), "-i", src, "-frames:v", "1"];
|
||||
if (vf) args.push("-vf", vf);
|
||||
execFileSync("ffmpeg", args.concat(dst));
|
||||
};
|
||||
const diff = (a, b, dst) =>
|
||||
execFileSync("ffmpeg", ["-y", "-v", "error", "-i", a, "-i", b, "-filter_complex", "blend=all_mode=difference", dst]);
|
||||
const psnr = (a, b) => {
|
||||
const err = execSync(`ffmpeg -i ${JSON.stringify(a)} -i ${JSON.stringify(b)} -lavfi psnr -f null - 2>&1`).toString();
|
||||
const m = err.match(/average:([\d.]+|inf)/);
|
||||
return m ? (m[1] === "inf" ? 99 : Number(m[1])) : NaN;
|
||||
};
|
||||
|
||||
const renderVf = `${cropFilter}scale=${rw}:${rh}`;
|
||||
const results = [];
|
||||
for (let t = 0; t <= end; t = Math.round((t + interval) * 1000) / 1000) {
|
||||
const t1 = Math.round((t + interval) * 1000) / 1000;
|
||||
frame(reference, t, null, join(dir, "ra.png"));
|
||||
frame(reference, t1, null, join(dir, "rb.png"));
|
||||
frame(render, t, renderVf, join(dir, "oa.png"));
|
||||
frame(render, t1, renderVf, join(dir, "ob.png"));
|
||||
diff(join(dir, "ra.png"), join(dir, "rb.png"), join(dir, "rd.png"));
|
||||
diff(join(dir, "oa.png"), join(dir, "ob.png"), join(dir, "od.png"));
|
||||
results.push({ t, motion: psnr(join(dir, "rd.png"), join(dir, "od.png")), abs: psnr(join(dir, "rb.png"), join(dir, "ob.png")) });
|
||||
}
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
|
||||
const min = Math.min(...results.map((r) => r.motion));
|
||||
const mean = results.reduce((s, r) => s + r.motion, 0) / results.length;
|
||||
for (const r of results)
|
||||
console.log(
|
||||
`window ${r.t.toFixed(2)}s→${(r.t + interval).toFixed(2)}s motion-psnr=${r.motion.toFixed(2)}dB (abs=${r.abs.toFixed(1)}dB)${r.motion < minMotion ? " <-- BELOW THRESHOLD" : ""}`,
|
||||
);
|
||||
console.log(`\nwindows=${results.length} min-motion=${min.toFixed(2)}dB mean-motion=${mean.toFixed(2)}dB threshold=${minMotion}dB`);
|
||||
if (min < minMotion) {
|
||||
console.log("VERDICT: FAIL — choreography diverges from the Figma export (check timings, invented keyframes, durations)");
|
||||
process.exit(1);
|
||||
}
|
||||
console.log("VERDICT: PASS — motion matches the Figma export within the static-fidelity ceiling");
|
||||
Reference in New Issue
Block a user