mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
Miguel ÁngelandClaude Opus 4.8
57b3c78987
* feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare CLI Add color grading to media-use as first-class resolve types plus a faithful comparison command. All local, offline, deterministic — no model, no GPU. - resolve -t grade / -t lut: produce a data-color-grading block (or a frozen .cube). Look cascade: core preset (no file) -> bundled .cube library -> parametric buildCube. Emitted .cube is Rec.709 and validated against core's colorLuts constraints (LUT_3D_SIZE <= 64) before it is frozen. - smart grade (grade --for <media>): ffmpeg signalstats -> adjust suggestion (exposure / contrast / white balance), surfaced with the measured evidence on stderr as a starting point; never auto-applied. - hyperframes grade-compare: renders N candidate grades onto a reference frame through the real runtime shader into one labeled comparison PNG, so an agent picks a look without opening Studio. Prepends an "original" baseline cell by default (--no-baseline to omit). Shares the headless-capture pipeline with snapshot via capture/captureCompositionFrame. - media-use SKILL: proactive "media opportunity pass" guidance (grounded signal -> offer, ask once, surface don't mutate). Verified: media-use 116/116, grade-compare 7/7, snapshot 9/9, lint + format clean, full build green, comparison renders end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * test(cli): narrow grade-compare baseline assertion off unknown-typed grading Assert the whole cell via toEqual instead of reaching into .grading.preset / .grading.lut on the unknown-typed field, keeping the test typecheck-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): agent-authored LUTs via --params + validate --from cube; never-read-.cube guardrail - resolve -t lut / -t grade --params '<json>': build a parametric .cube from explicit params (bypassing the intent cascade), validate, and freeze in one step. --intent becomes the optional description. Lets an agent commit a look it computed itself. - --from <file.cube> now validates the ingested LUT for lut/grade types and rejects an invalid/oversized cube (no partial write) — the escape hatch for a LUT the agent generated with its own code. - SKILL.md: hard rule to never read a .cube body into context (~size^3 lines, zero legible signal) — inspect via grade-compare (see it) or cube-validate (ok/size), read the manifest description for meaning; plus both authoring paths and the parametric-vs-film-stock ceiling note. Verified: media-use 116/116, lint + format clean; smokes — --params builds a valid frozen cube, grade --params returns a lut block, bad JSON and an oversized --from cube are both rejected with no stray file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): grade-compare validates referenced LUTs, warns on no-op cells, caps candidates Bug-bash follow-ups — grade-compare silently accepted bad input: - Validate LUT *content*, not just existence: each referenced .cube is parsed with core's parseCubeLut (now exported from @hyperframes/core) and rejected with a per-cell error ("LUT for \"<label>\" is not a valid .cube: ..."). A file that exists but isn't a valid cube no longer renders a silent no-op cell. - Warn on inactive cells: a grading that normalizes to inactive (e.g. a malformed {lut:12345}) emits a stderr warning naming the cell; the auto-prepended "original" baseline is intentionally inactive and stays silent. stdout remains valid JSON. - Cap candidates at 16 (excluding baseline): over-cap input renders the first N and reports {truncated:true, total:M} on stdout + a stderr note — no silent drop, no unbounded giant sheet. Verified: grade-compare 10/10; non-cube LUT → clear error; {lut:12345} → warning + ok; 20 cells → cells=17 truncated total=20; valid runs unchanged. Lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(cli): general `hyperframes compare` visual-variant primitive Generalize grade-compare's "render N variants → one labeled sheet → the agent looks and picks" loop into a standalone command that works on ANY variation (font, layout, motion, grade, whole compositions) — the tool never needs to know what differs. - `hyperframes compare <path...> [--at <sec>] [--labels a,b,c] [--out] [--cols] [--json]`: renders each agent-authored composition variant through the real runtime (captureCompositionFrame) and stitches one labeled comparison sheet + JSON ({ok, sheet, rendered, variants, truncated?/total?}). 2+ paths required; caps at 16 with loud truncation. It presents, it does not judge — choosing is the caller's job. - Factored the shared "render a labeled set → contact sheet" path so compare, grade-compare, and snapshot all sit on it (no duplication). grade-compare is now the first color-specific specialization of this primitive. - New pathArgs util + contactSheet test; hyperframes-cli SKILL documents compare as the agent's "see your own renders and choose" primitive. Verified: 26/26 across compare + grade-compare + snapshot + contactSheet (no regressions); compare renders 3 variants into one visibly-distinct labeled sheet; 2+-path error path clean; lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): green the skills CI — skip ffmpeg tests when absent, oxfmt markdown The "Test: skills" CI job runs bare `node --test` with no ffmpeg on PATH (by design — skills tests are meant to be node-builtin-only). The grade-analyzer + smart-grade tests shell to ffmpeg and were failing there with ENOENT. Guard them to skip when ffmpeg isn't on PATH; they still run locally / where it is. Also oxfmt README.md + hyperframes/media-use SKILL.md (the whole-repo `oxfmt --check .` Format job caught markdown left unformatted by the rebase conflict resolution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): skip core-conformance test when tsx is unavailable The "Test: skills" CI job installs no deps, so the normalizeHfColorGrading conformance test (which imports core's TS via `node --import tsx`) failed there. Guard it to skip when tsx can't resolve; runs locally / in the deps-installed Test job. Completes the skills-CI greening (the ffmpeg guards handled the rest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): escape grade-compare src double-quotes (CodeQL XSS) + Windows-safe compare test - grade-compare built `<img src="...">` (double-quoted) with the single-quote escaper, leaving `"` unescaped — a `"` in the frame path could break out (CodeQL: incomplete HTML attribute sanitization). Use escapeXml for src. - compare label test hard-coded POSIX paths that can't match on Windows; assert the derived labels (the subject); path resolution is covered elsewhere. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * refactor(media-use): generate LUT library from params (drop committed .cube files) The 3 bundled .cube files were 733 lines each (2,199 total) and were themselves buildCube output — pure repo bloat. Replace with compact per-look params in luts/index.json, generated on resolve; add an optional `url` for future scanned LUTs to be CDN-hosted + downloaded on demand (freezeUrl) instead of committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): serve library LUTs from CDN on-demand (static.heygen.ai/luts), params fallback Looks now carry a CDN `url` (hosted at s3://heygen-public/luts → static.heygen.ai/luts/<id>.cube); resolve downloads + validates + freezes on demand, like bgm/image. `params` stays as the deterministic offline fallback (--local-only, or if the download fails), so resolution is never blocked on the network. Provider prefers url, falls back to params. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(media-use): address #2041 review — atomic LUT writes, compare telemetry, follow-ups - Atomic .cube writes: library provider (url + params) and the parametric generator now write to a .tmp path, validate, then rename, so a crash can never orphan an invalid .cube at the final path (was validate-after-write). - track("media_use_resolve") now emits provenance.via (url/params-fallback/params). - grade-compare + compare: --timeout flag (was hardcoded 5000) and a media_use_compare event (cells, truncated, total, render_ready_timed_out); openSettledCompositionPage now surfaces the render-ready timeout. - compare staging skips node_modules/.git; --for gets an upfront existence check. - Rec.709 luma comment; HYPERFRAMES_ANALYZE_TIMEOUT_MS override; measured note uses basename; LUT s3 hosting moved from index.json into luts/README.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
126 lines
12 KiB
Plaintext
126 lines
12 KiB
Plaintext
---
|
|
title: Skills catalog
|
|
description: "Every HyperFrames skill agents load on demand — what each one is for, and how to install them."
|
|
---
|
|
|
|
HyperFrames ships AI agent skills via [vercel-labs/skills](https://github.com/vercel-labs/skills). They teach your agent the framework-specific patterns — `data-*` attributes, GSAP timeline registration, adapter registries, runtime-owned media playback — that generic web docs don't cover. **Install them before writing compositions** and your agent produces valid HyperFrames HTML on the first try.
|
|
|
|
The skills split into three groups:
|
|
|
|
- **Router** — the entry skill that picks a workflow for any "make me a…" request — usually a video, but also a navigable deck (`/slideshow`) or a composition port (`/remotion-to-hyperframes`).
|
|
- **Creation workflows** — one per input shape (URL, PR, music track, captions, etc.). Each owns its task end-to-end: project setup, gated steps, and the final deliverable. Read by the router; you can also invoke directly if you already know which one fits.
|
|
- **Domain skills** — atomic capabilities (animation, media, CLI, registry) the workflows compose against; they never own the end-to-end task. Load one when you need that specific layer.
|
|
|
|
## Install
|
|
|
|
<Steps>
|
|
<Step title="Pick what to install (interactive picker)">
|
|
```bash
|
|
npx skills add heygen-com/hyperframes --full-depth
|
|
```
|
|
|
|
Opens a picker so you can choose which skills to add. Keep `--full-depth`: it installs the current `main`. Without it, `skills add` fetches the skills.sh registry blob, which lags `main` by hours, so you may get an older copy of a skill. Works with [Claude Code](https://claude.ai/claude-code), [Cursor](https://cursor.sh), [Gemini CLI](https://github.com/google-gemini/gemini-cli), [Codex CLI](https://github.com/openai/codex), [GitHub Copilot CLI](/guides/copilot-cli), and [Google Antigravity](/guides/antigravity).
|
|
</Step>
|
|
<Step title="Or install everything at once (skip the picker)">
|
|
```bash
|
|
npx skills add heygen-com/hyperframes --all --full-depth
|
|
```
|
|
|
|
Writes every skill to your project in one shot. Recommended when you want the full set without selecting from the picker.
|
|
</Step>
|
|
<Step title="Or install one skill at a time">
|
|
```bash
|
|
npx skills add heygen-com/hyperframes --skill <name> --full-depth
|
|
```
|
|
|
|
Pass the bare skill name (no leading `/`) — e.g. `--skill hyperframes-animation`. Useful when you want a single capability without the full set.
|
|
</Step>
|
|
<Step title="Read /hyperframes first">
|
|
Once installed, invoke `/hyperframes` in your agent. It's the capability map for everything below and routes "make me a…" intent — a video, a deck, or a composition port — to the right creation workflow based on your input.
|
|
</Step>
|
|
</Steps>
|
|
|
|
<Tip>
|
|
**Don't see a slash command after install?** Open a new agent session, or run `/skills` (Copilot CLI) / restart your agent's skill loader. Most agents pick up new skills on the next prompt.
|
|
</Tip>
|
|
|
|
## Keeping skills current
|
|
|
|
After the first install, skills stay lean and current on their own — nothing re-pulls the full set behind your back:
|
|
|
|
- **Core set** — the `/hyperframes` router, the `hyperframes-*` domain skills, and `/media-use`. `npx hyperframes init` (which every creation workflow runs when scaffolding) refreshes the core set plus anything else you already have installed. It never _expands_ the install — workflow skills you haven't used are not pulled — and re-running `init` on an up-to-date machine is a no-op. Offline (or rate-limited) it degrades gracefully and never hard-fails.
|
|
- **Workflow skills** (and `/figma`) — install **on demand**, when their workflow is first triggered. The `/hyperframes` router runs `npx hyperframes skills update <workflow>` before entering a workflow, so the one it routes to is guaranteed present.
|
|
|
|
Check or refresh manually anytime:
|
|
|
|
```bash
|
|
npx hyperframes skills check # non-zero exit if installed skills are stale or the core set is incomplete
|
|
npx hyperframes skills update # refresh the core set + everything installed (never expands)
|
|
npx hyperframes skills update <name> # also install a specific workflow / domain skill on demand
|
|
npx hyperframes skills # install the full set explicitly
|
|
```
|
|
|
|
`skills check` reports workflow skills you haven't installed as _available on demand_, not as a failure.
|
|
|
|
## Router
|
|
|
|
The single entry point. Read `/hyperframes` before invoking anything else — it knows what's available and which workflow fits your request.
|
|
|
|
| Skill | Use when |
|
|
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `/hyperframes` | **Read first** for any request to make / create / edit / animate / render a video, animation, or motion graphic. Capability map for the domain skills and intent router for the creation workflows below. |
|
|
|
|
## Creation workflows
|
|
|
|
One workflow per input shape. The router (`/hyperframes`) picks one of these for you — but you can invoke them directly when you already know which fits.
|
|
|
|
| Skill | Use when |
|
|
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `/product-launch-video` | Marketing / launching / promoting a **product** — from its URL, a brief, or a script (even if the site is only named). Up to ~3 min (sweet spot 30-90s). |
|
|
| `/website-to-video` | Turning a **general website** into a video — site tour, portfolio / landing-page showcase, social clip from the site's own visuals. |
|
|
| `/faceless-explainer` | **Explaining a topic / concept** from arbitrary text — no product, no URL, no website capture; every visual is LLM-invented (typography / abstract / diagram / data-viz). |
|
|
| `/pr-to-video` | A **GitHub pull request** (PR URL, `owner/repo#N` ref, or "this PR") → changelog / feature-reveal / fix / refactor explainer, read via the `gh` CLI. |
|
|
| `/embedded-captions` | Adding **captions / subtitles** to an existing talking-head video (footage untouched) — verbatim rail, embedded climax behind the subject, or pure-cinematic embed. |
|
|
| `/talking-head-recut` | Packaging an existing talking-head / interview / podcast video with **designed graphic overlays** — lower-thirds, data callouts, kinetic titles, pull-quotes, side panels, PiP. |
|
|
| `/motion-graphics` | A short, **unnarrated, design-led motion graphic** (~under 10s) — kinetic type, stat / chart hit, logo sting, lower-third, animated tweet / headline. MP4 or transparent overlay. |
|
|
| `/music-to-video` | A **music track** (audio file, or video to pull audio from) → a **beat-synced** video — lyric, slideshow, or kinetic promo; music drives pacing. |
|
|
| `/slideshow` | A **presentation / pitch deck / interactive deck** — discrete slides, fragment reveals, branching, hotspot navigation, presenter mode. Output is a navigable deck, not a rendered video. |
|
|
| `/general-video` | **Anything else** — longer or multi-scene pieces, brand / sizzle reel, title card, static loop, freeform composition. Input- and length-agnostic fallback. |
|
|
| `/remotion-to-hyperframes` | **Porting an existing Remotion** (React) composition's source to HyperFrames HTML. One-way migration, not creation. |
|
|
|
|
## Domain skills (loaded on demand)
|
|
|
|
Atomic capabilities the creation workflows compose against — pull one when you need that specific layer.
|
|
|
|
| Skill | Covers |
|
|
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. |
|
|
| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). |
|
|
| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. |
|
|
| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. |
|
|
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, logo, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
|
|
| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress / destroy / policies`). |
|
|
| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. |
|
|
| `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. |
|
|
|
|
## Source of truth
|
|
|
|
The one-line "use when" for each skill comes from its own `SKILL.md` frontmatter `description:` field in the [hyperframes repo](https://github.com/heygen-com/hyperframes/tree/main/skills). The same catalog lives in the [README's `## Skills` section](https://github.com/heygen-com/hyperframes#skills) and the repo `CLAUDE.md`; all three surfaces are kept in sync when a skill is added or renamed.
|
|
|
|
## Next steps
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Quickstart" icon="rocket" href="/quickstart">
|
|
Install skills, scaffold a project, and render your first video.
|
|
</Card>
|
|
<Card title="Claude Design" icon="palette" href="/guides/claude-design">
|
|
Produce a valid first draft in Claude Design, then refine in any AI coding agent.
|
|
</Card>
|
|
<Card title="GitHub Copilot CLI" icon="terminal" href="/guides/copilot-cli">
|
|
Install and invoke HyperFrames skills from Copilot CLI in your terminal.
|
|
</Card>
|
|
<Card title="Google Antigravity" icon="atom" href="/guides/antigravity">
|
|
Use HyperFrames skills in Google Antigravity.
|
|
</Card>
|
|
</CardGroup>
|