From b7769b23647f3acd2bf72038587a6e9295d8ec6c Mon Sep 17 00:00:00 2001 From: James Date: Mon, 27 Apr 2026 19:01:53 +0000 Subject: [PATCH] feat(skills): remotion-to-hyperframes SKILL.md + orchestrator (7/7) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The leaf PR. Replaces the placeholder SKILL.md from PR 1 with the real 5-step workflow that loads the per-topic references on demand (skill-creator's progressive-disclosure pattern), and adds a top-level orchestrator that runs every tier and reports a pass/fail summary. SKILL.md changes: - Frontmatter unchanged from PR 1 (already covers the trigger phrases and out-of-scope cases) - Body rewritten as a 5-step workflow: 1. Lint (load escape-hatch.md if blockers) 2. Plan (load api-map.md, then per-topic references on demand) 3. Generate (HF index.html with paused GSAP timeline) 4. Validate (render_diff.sh against per-tier threshold) 5. Document gaps (TRANSLATION_NOTES.md if needed) - Includes a "Source contains -> Load reference" table so the agent only loads the references the source actually needs - Documents the validated baseline numbers (T1 0.974, T2 0.985, T3 0.953, T4 8/8) so reviewers can reproduce - Calls out the critical Remotion encoder config (PNG + BT.709) that avoids the ~0.05 SSIM hit from yuvj420p vs yuv420p Orchestrator (assets/test-corpus/run.sh): - Iterates tier-1-* through tier-4-* directories - T1-T3: setup -> lint -> npm install (lazy) -> render Remotion -> render HF -> SSIM diff at the fixture's expected threshold -> generate strip on failure - T4: validate.sh (lint-only) - Emits run-report.json with per-tier pass/fail and aggregate counts - Accepts a single-tier argument for fast iteration: ./run.sh tier-1-title-card Validated end-to-end on a clean checkout: ▶ tier-1-title-card → mean SSIM 0.9739 (≥ 0.95) ✓ ▶ tier-2-multi-scene → mean SSIM 0.985292 (≥ 0.95) ✓ ▶ tier-3-data-driven → mean SSIM 0.952941 (≥ 0.9) ✓ ▶ tier-4-escape-hatch → 8/8 cases ✓ passed 4/4, failed 0, skipped 0 Closes the 7-PR stack: scaffold, eval harness, 4 tiers of corpus, references, and now the SKILL.md body that ties everything together. --- skills/remotion-to-hyperframes/SKILL.md | 91 ++++++- .../assets/test-corpus/.gitignore | 1 + .../assets/test-corpus/run.sh | 249 ++++++++++++++++++ 3 files changed, 330 insertions(+), 11 deletions(-) create mode 100644 skills/remotion-to-hyperframes/assets/test-corpus/.gitignore create mode 100755 skills/remotion-to-hyperframes/assets/test-corpus/run.sh diff --git a/skills/remotion-to-hyperframes/SKILL.md b/skills/remotion-to-hyperframes/SKILL.md index 7d054d6bb..9f43af4fc 100644 --- a/skills/remotion-to-hyperframes/SKILL.md +++ b/skills/remotion-to-hyperframes/SKILL.md @@ -7,26 +7,95 @@ description: Translate a Remotion (React-based) video composition into a HyperFr ## Overview -Translate Remotion (React-based) video compositions into HyperFrames (HTML + GSAP) compositions. Most Remotion idioms have direct HyperFrames equivalents — the translation is mechanical for ~80% of typical compositions. This skill encodes the mapping and guards against the lossy 20%. +Translate Remotion (React-based) video compositions into HyperFrames (HTML + GSAP) compositions. Most Remotion idioms have direct HyperFrames equivalents — the translation is mechanical for ~80% of typical compositions. This skill encodes the mapping and guards against the lossy 20% by refusing to translate patterns that don't fit HF's seek-driven model and recommending the runtime interop pattern from [PR #214](https://github.com/heygen-com/hyperframes/pull/214) instead. + +The skill ships with a **tiered test corpus** (T1–T4, 4 fixtures total) that grades translations against measured SSIM thresholds. Don't translate without running the eval — a translation that "looks right" but renders 0.05 SSIM lower than the validated baseline is silently wrong. ## Workflow -1. **Lint the source.** Run the source-lint script against the Remotion project to surface any patterns that can't translate cleanly (React state hooks, async metadata, third-party React components). If the source uses any blocker pattern, recommend the runtime interop escape hatch (PR #214 pattern) instead of attempting a translation. +### Step 1: Lint the source -2. **Scaffold the translation.** Generate a HyperFrames HTML skeleton from the Remotion source — `Composition` props become `data-*` attributes on the root `#stage` div, `` wrappers become elements with `data-start` / `data-duration` / `data-track-index`, `` becomes `
`. Leave timing-sensitive and easing-sensitive sections marked for refinement. +Run [`scripts/lint_source.py`](scripts/lint_source.py) over the Remotion source directory. The lint detects patterns that can't translate cleanly: -3. **Refine timing and easing.** Convert each `useCurrentFrame`-driven `interpolate` / `spring` call into an equivalent paused GSAP tween on the composition timeline. This is the part where translation correctness matters most — easing curves and stagger timing are what readers notice. +- **Blockers** (refuse + recommend interop): `useState`, `useReducer`, `useEffect`/`useLayoutEffect` with non-empty deps, async `calculateMetadata`, third-party React UI libraries (MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI). +- **Warnings** (translate after dropping the construct): `@remotion/lambda` config, `delayRender`, `useCallback`, `useMemo`, custom hooks. +- **Info** (translate with note): `staticFile`, `interpolateColors`. -4. **Validate by frame-diff.** Render both the original Remotion composition and the translated HyperFrames composition, then compute per-frame SSIM. Threshold-based pass/fail tells the user which scenes are visually correct and which need another pass. +If any blocker fires, **stop**. Read [`references/escape-hatch.md`](references/escape-hatch.md) and surface the recommendation message. Warnings don't stop translation — drop the offending construct in step 3 and note the gap in `TRANSLATION_NOTES.md`. `@remotion/lambda` config is the canonical warning case: the skill drops the import + `renderMediaOnLambda(...)` calls but translates the rest of the composition. -5. **Document the gaps.** Any Remotion features that didn't translate (custom React subcomponents requiring manual rewrite, library transitions without a HyperFrames equivalent, etc.) get listed in a `TRANSLATION_NOTES.md` next to the output so the user can finish them or decide to use the runtime interop instead. +### Step 2: Plan the translation + +Read [`references/api-map.md`](references/api-map.md) — the index of every Remotion API and its HF equivalent or per-topic reference. Identify which topic references you'll need based on what the source uses: + +| Source contains | Load reference | +| ------------------------------------------------------------------------- | --------------------------------------------- | +| `Composition`, `defaultProps`, `schema`, `calculateMetadata` | [`parameters.md`](references/parameters.md) | +| `Sequence`, `Series`, `Loop`, `AbsoluteFill`, `Freeze` | [`sequencing.md`](references/sequencing.md) | +| `useCurrentFrame`, `interpolate`, `spring`, `Easing`, `interpolateColors` | [`timing.md`](references/timing.md) | +| `Audio`, `Video`, `Img`, `IFrame`, `staticFile`, `delayRender` | [`media.md`](references/media.md) | +| `TransitionSeries`, `@remotion/transitions` | [`transitions.md`](references/transitions.md) | +| `@remotion/lottie` | [`lottie.md`](references/lottie.md) | +| `@remotion/google-fonts/`, `Font.loadFont`, `@font-face` | [`fonts.md`](references/fonts.md) | + +Don't load all of them — load only what the specific source needs. + +### Step 3: Generate the HF composition + +Emit `index.html` with: + +- Root `
` carrying the composition's `data-composition-id`, `data-start="0"`, `data-duration` (in seconds), `data-fps`, `data-width`, `data-height`, plus one `data-*` per scalar prop. +- A flat list of scene divs with `data-start` / `data-duration` / `data-track-index`. +- Inline `