mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-12 15:20:13 +00:00
feat(skills): add changelog-video skill for repo-native CC + Codex discovery (#2552)
Packages Jake Moran's changelog-video pipeline (v1, validated end-to-end
by Home on the Jun 23-29 range) as a repo-native skill set that Claude
Code (.claude/skills/) and Codex CLI (.agents/skills/) auto-discover the
moment the repo is opened. No install step; run the skill against a
changelog markdown for a given git range and it produces a lint-clean,
seam-gate-green 1080x1080 MP4 (~45-60s, Annie VO, mock-UI visualizations,
caption rail) end-to-end.
Six skills added byte-identical in both mirror dirs:
- changelog-video (pipeline entry point)
- motion-doctrine (carries seam-stamp.mjs + seam-gate.mjs)
- cut-the-curve, captions-overlay, seam-craft, oversized-cursor
Layout:
- .claude/skills/ - Claude Code project-local auto-discover
- .agents/skills/ - Codex CLI project-local auto-discover (verified via
Magi's clean-home Codex 0.144.3 repro; NOT .codex/skills/)
Fonts, animated background (12 MB), house BGM (5 MB), lexicon, and
align-captions ship inside the skill dirs. .gitattributes routes only
.claude/skills/**/*.{mp4,mp3} + .agents/skills/**/*.{mp4,mp3} through
LFS — narrowly scoped so unrelated Player, Studio, registry, and
marketplace media stay put. HeyGen CLI auth is the one credential the
skill needs; Node >= 22, ffmpeg, and headless Chrome are documented
alongside in both READMEs.
.gitignore: rewrites .claude/ and .agents/ blocks to keep agent-installed
skill hygiene while re-including the six repo-native skill dirs plus
README.md.
CI:
- Extends changes.skills filter to match .claude/skills/**,
.agents/skills/**, scripts/lint-skills.ts, and scripts/check-skill-mirror.mjs.
- New 'Skills: project-native lint + mirror' job runs the extended
lint-skills.ts (schema-driven; required { name, description } + optional
{ license, allowed-tools, metadata }, name pattern check, description
length check) plus a new check-skill-mirror.mjs byte-integrity script
(24 mirrored files must match; README.md deliberately per-CLI).
- Wired into 'bun run lint' locally.
Frontmatter validator:
- Rejects unsupported top-level keys (catches category:-style drift).
- Requires name + description.
- Validates name pattern (^[a-z][a-z0-9-]{0,63}$) and description shape
(non-empty, <=1024 chars).
- Missing frontmatter block itself is a first-class error.
Also strips unsupported top-level 'category:' frontmatter from Jake's
motion-doctrine and cut-the-curve SKILL.mds (both mirrors), rewrites the
TTS invocation from ~/.claude/skills/media-use/... to the tracked
skills/hyperframes-media/scripts/heygen-tts.mjs, swaps npx hyperframes@latest
for the repo-local CLI in the gate step, and fixes a lint issue in Jake's
seam-gate.mjs (ternary-for-side-effect -> if/else).
Validated end-to-end by Home on Jun 23-29 (MP4 posted in C0ACCNHLG3U
thread 1784181166.041319). Independently reviewed R1/R2/R3 by Magi.
Co-authored-by: Jake Moran <jake@heygen.com>
This commit is contained in:
James Russo
co-authored by
Jake Moran
Jake Moran
parent
172f2757f0
commit
e96ebd74de
@@ -0,0 +1,115 @@
|
||||
# Build Spec — square 1080 changelog master
|
||||
|
||||
Single-doc `index.html`: scenes are absolutely-positioned `.slide` clips on
|
||||
track 1; the master timeline `tl` (MUST be named `tl` — seam-stamp emits
|
||||
`tl.*`) is paused, padded to total duration, registered as
|
||||
`window.__timelines["main"]`. See `examples/master-skeleton.html` for the
|
||||
verbatim scaffold.
|
||||
|
||||
## Brand tokens (HeyGen for Developers)
|
||||
|
||||
```css
|
||||
@font-face {
|
||||
font-family: "ABC Solar Display";
|
||||
font-weight: 700;
|
||||
src: url(assets/fonts/ABCSolarDisplay-Bold.woff2) format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "TT Norms Pro";
|
||||
font-weight: 400;
|
||||
src: url(assets/fonts/TT_Norms_Pro_Normal.woff2) format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "TT Norms Pro";
|
||||
font-weight: 500;
|
||||
src: url(assets/fonts/TT_Norms_Pro_Medium.woff2) format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "TT Norms Pro";
|
||||
font-weight: 700;
|
||||
src: url(assets/fonts/TT_Norms_Pro_Bold.woff2) format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "TT Norms Mono";
|
||||
font-weight: 400;
|
||||
src: url(assets/fonts/tt_norms_pro_mono_regular-webfont.woff2) format("woff2");
|
||||
}
|
||||
/* ink #f5f6f4 · ink2 rgba(245,246,244,.72) · dim rgba(245,246,244,.66) —
|
||||
NOT .45: the contrast gate fails small text under 4.5:1 over the glass.
|
||||
green #5ef17c (RATIONED: one moment per scene) · bg #0a0c0b ·
|
||||
chip-bg rgba(10,12,11,.78) · glass-line rgba(190,255,205,.32) */
|
||||
```
|
||||
|
||||
Copy fonts from `<SKILL_DIR>/assets/fonts/` into the project's
|
||||
`assets/fonts/` (local @font-face with format('woff2') embeds at render).
|
||||
Glass card: chip-bg fill, 1px glass-line border, radius 22,
|
||||
`box-shadow: 0 24px 60px rgba(0,0,0,.5)`. NO backdrop-filter. Chips: mono
|
||||
18-22px, radius 12, 1px rgba(255,255,255,.14) border on rgba(255,255,255,.05).
|
||||
Display type: ABC Solar Display 700. Body: TT Norms Pro. Everything
|
||||
code/UI-label: TT Norms Mono. Safe margins x/y ∈ [76, 1004].
|
||||
|
||||
## Animated background (the house pattern)
|
||||
|
||||
Encode the bundled source to the film's exact duration — the render
|
||||
compiler shortens a video's slot to the media length, so the encode must be
|
||||
≥ total:
|
||||
|
||||
```bash
|
||||
ffmpeg -y -stream_loop 15 -i <SKILL_DIR>/assets/bg-pattern.mp4 -t <TOTAL> \
|
||||
-vf "scale=1080:1080,fps=30,eq=saturation=0.72,drawbox=c=black@0.5:t=fill" \
|
||||
-an -c:v libx264 -crf 20 -pix_fmt yuv420p assets/bg-pattern-<TOTAL>s.mp4
|
||||
```
|
||||
|
||||
Keep the darkening crush — the raw pattern is far too loud. Mount as
|
||||
`<video id="bg-video" class="clip" muted>` on track 0 + a static
|
||||
`rgba(8,10,9,.25)` scrim div. `<video>` needs the id (silent/black
|
||||
otherwise) and must stay flat 2D (no 3D ancestors).
|
||||
|
||||
## Scene anatomy
|
||||
|
||||
- Chrome (untimed, z 6): kicker chip top-left `HYPERFRAMES WEEKLY · <RANGE>`,
|
||||
progress dots top-right (one per theme; `tl.set` backgroundColor at each
|
||||
cut — active #f5f6f4, done .45; never tl.call for state).
|
||||
- Title (≤2s): mono kicker date, ABC Solar h1 ~104px, green rule sweep.
|
||||
- Theme scene: sec-chip `0N · THEME NAME` (top 128) + ABC Solar headline
|
||||
~54px (top 186) + the mock (from the visualization registry) filling
|
||||
y ∈ [288, 944].
|
||||
- Outro (≤3.5s): kicker FULL DIGEST, "See what shipped." ~96px, green rule,
|
||||
mono URL chip, tag line. Fade all + chrome ~0.5s before end.
|
||||
- Caption rail per `script-voice.md` (top: 1002).
|
||||
|
||||
## Seams + internal life (doctrine mechanics)
|
||||
|
||||
- `ledger.json`: every ordinary seam `cut-the-curve LEFT` (x, dir −1), exit
|
||||
and entry selectors = the slide wrappers. Outro entry `travel: 8`.
|
||||
- `seam-stamp.mjs --ledger ledger.json --write index.html` owns ALL wrapper
|
||||
entries/exits — author none yourself. Title (film open) authors its own
|
||||
entry only.
|
||||
- Slides: CSS `opacity: 0` base; `data-start` = exactly the cut time.
|
||||
- Each scene's shell (chip, headline, mock chrome, initial state) is
|
||||
COMPOSED at local t=0 — the wrapper flies it in. Internal reveals start
|
||||
≥0.4s after the cut and end ≥0.45s before the next cut (stamped exits
|
||||
begin at cut −0.34s).
|
||||
- Every internal beat lands on a VO word from `vo-words.json`. Name each
|
||||
scene's sustained-motion route in the plan (sequenced UI life for mocks,
|
||||
staged reveals for checklists). One green moment per scene.
|
||||
- Init states via `gsap.set(...)` at build time; animation via sequential
|
||||
`tl.to(...)` only (no plain-object keyframes, no repeat:-1, no tweening
|
||||
left/top — set base position in CSS, tween x/y). Counters: object tween
|
||||
with an `onUpdate` in the tween config (cache DOM refs; never
|
||||
`tl.eventCallback`).
|
||||
|
||||
## Lint/check gotchas (all hit before, all pre-solved)
|
||||
|
||||
- Mock containers with intentional stacking: `data-layout-allow-overlap` on
|
||||
the slide root; elements a playhead/line crosses:
|
||||
`data-layout-allow-occlusion`.
|
||||
- Dim text: `rgba(245,246,244,.66)` minimum (contrast gate).
|
||||
- Audio: every `<audio>` carries an `id`. BGM: the house track ships at
|
||||
`<SKILL_DIR>/assets/bgm.mp3` (159s instrumental) — copy it to the project
|
||||
as `bgm.mp3` and mount on its own track:
|
||||
`<audio id="bgm" src="bgm.mp3" data-start="0" data-duration="<TOTAL>" data-track-index="2" data-volume="0.14" data-media-start="0">`
|
||||
(0.14 sits under the VO; use a user-supplied track only when given one).
|
||||
- Preview server caches the bundle — RESTART after edits, then verify on the
|
||||
raw comp page (`/api/projects/<id>/preview/comp/index.html`) via
|
||||
`window.__player.seek(t)`.
|
||||
@@ -0,0 +1,66 @@
|
||||
{
|
||||
"_readme": "display → spoken. Plain-text phonetics for HeyGen TTS (no SSML). Grow this file; never guess a missing term — ask, add, then listen to the generated line. Slash commands: caption /name, speak 'slash <name>'.",
|
||||
"/figma": "slash Figma",
|
||||
"/hyperframes": "slash hyper-frames",
|
||||
"JSON": "jay-sawn",
|
||||
"GSAP": "jee-sap",
|
||||
"CLI": "C L I",
|
||||
"CDP": "C D P",
|
||||
"API": "A P I",
|
||||
"SDK": "S D K",
|
||||
"UI": "U I",
|
||||
"UX": "U X",
|
||||
"CSS": "C S S",
|
||||
"HTML": "H T M L",
|
||||
"SVG": "S V G",
|
||||
"URL": "U R L",
|
||||
"TTS": "T T S",
|
||||
"ASR": "A S R",
|
||||
"MCP": "M C P",
|
||||
"LRU": "L R U",
|
||||
"GC": "G C",
|
||||
"VFR": "V F R",
|
||||
"CFR": "C F R",
|
||||
"SDR": "S D R",
|
||||
"HDR": "H D R",
|
||||
"GPU": "G P U",
|
||||
"CPU": "C P U",
|
||||
"PNG": "P N G",
|
||||
"MP4": "em pee four",
|
||||
"MP3": "em pee three",
|
||||
"WebM": "web em",
|
||||
"WebGL": "web G L",
|
||||
"wav": "wave",
|
||||
"ffmpeg": "ff-mpeg",
|
||||
"ffprobe": "ff-probe",
|
||||
"OAuth": "oh-auth",
|
||||
"SSML": "S S M L",
|
||||
"OOM": "out of memory",
|
||||
"e2e": "end to end",
|
||||
"DX": "developer experience",
|
||||
"QA": "Q A",
|
||||
"npm": "N P M",
|
||||
"npx": "N P X",
|
||||
"regen": "ree-jen",
|
||||
"repo": "repo",
|
||||
"HeyGen": "hey-jen",
|
||||
"heygen.com": "hey-jen dot com",
|
||||
"hyperframes.heygen.com": "hyperframes dot hey-jen dot com",
|
||||
"MusicGen": "music-jen",
|
||||
"Kokoro": "koh-koh-roh",
|
||||
"Figma": "Figma",
|
||||
"drawElement": "draw element",
|
||||
"DrawElement": "draw element",
|
||||
"moveElement": "move element",
|
||||
"storyboard": "storyboard",
|
||||
"keyframes": "key-frames",
|
||||
"sub-compositions": "sub compositions",
|
||||
"WSL": "W S L",
|
||||
"Linux": "linnucks",
|
||||
"symlink": "sim-link",
|
||||
"lint": "lint",
|
||||
"toast": "toast",
|
||||
"vite": "veet",
|
||||
"Chromium": "chromium",
|
||||
"headless": "headless"
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
# Script + Voice: the two-layer contract
|
||||
|
||||
The script is the single source of truth for BOTH the VO and the captions,
|
||||
written as token lines. The VO reads the `spoken` layer; captions render the
|
||||
`display` layer. This is a hard quality gate: a caption showing "jay-sawn" or
|
||||
a VO saying "juh-son" ("JSON" read literally) are both build failures.
|
||||
|
||||
## Register (how it should sound)
|
||||
|
||||
- Conversational, not release-notes. "The big one this week —" beats
|
||||
"Theme 1:". Contractions welcome. Second person allowed ("your clips").
|
||||
- Informational, never salesy; no superlatives the changelog doesn't earn.
|
||||
- One breath per beat: sentences ≤ ~14 words; let punctuation pace the read.
|
||||
- Numbers with meaning stay ("fifteen releases"); commit hashes, PR numbers,
|
||||
and version micro-detail are never spoken.
|
||||
- Open with the week + the marquee, close with the digest pointer
|
||||
("See everything at hyperframes dot heygen dot com").
|
||||
- **Teach the simple command.** When a feature has a one-line invocation (a
|
||||
slash command, a CLI one-liner), the script says it verbatim ("start your
|
||||
prompt with /figma…") and the mock shows it being typed — the command is
|
||||
the visible CAUSE of the result. Slash commands speak as "slash <name>",
|
||||
caption as `/name`.
|
||||
|
||||
## Token-line format (`script-tokens.json`)
|
||||
|
||||
```json
|
||||
{
|
||||
"lines": [
|
||||
{
|
||||
"id": "l1",
|
||||
"tokens": ["This", "week", "at", "HyperFrames,", { "display": "JSON", "spoken": "jay-sawn" }]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- A bare string = display and spoken identical.
|
||||
- An object = the layers diverge. `display` keeps standard spelling AND the
|
||||
punctuation captions should show; `spoken` is what the TTS reads.
|
||||
- One line = one caption phrase (≤ ~40 chars of display text). Line grouping
|
||||
is an authoring decision made here, not downstream.
|
||||
- Build `vo-spoken.txt` by joining every token's spoken form with spaces,
|
||||
lines joined into sentences/paragraphs as punctuated.
|
||||
|
||||
## Phonetics rules (ElevenLabs-style best practices, plain-text)
|
||||
|
||||
HeyGen TTS takes plain text (no SSML), so pronunciation is controlled by
|
||||
spelling, hyphens, and spacing:
|
||||
|
||||
1. **Initialisms** (each letter said): space or hyphen the letters —
|
||||
`CLI → "C L I"`, `CDP → "C D P"`, `API → "A P I"`.
|
||||
2. **Acronyms said as words**: respell phonetically —
|
||||
`JSON → "jay-sawn"`, `GSAP → "jee-sap"`.
|
||||
3. **Mixed / pronounceable compounds**: hyphenated LOWERCASE phonetics, one
|
||||
fluid run — `ffmpeg → "ff-mpeg"` (ear-tested; the TTS reads "ff" as a
|
||||
fluid "eff-eff"), `WebM → "web em"`, `OAuth → "oh-auth"`. Never spaced
|
||||
capitals here: the TTS reads spaced caps as isolated letter names with
|
||||
hard stops ("F F em-peg" comes out "eff… eff… em-peg"). Reserve spaced
|
||||
capitals for TRUE initialisms (CLI, API) where a deliberate
|
||||
letter-by-letter read is the goal. When candidates are close, generate
|
||||
A/B takes of the real sentence and let the user pick by ear.
|
||||
4. **Versions/numbers**: expand — `v0.7.36 → "version zero point seven
|
||||
point thirty-six"` (usually: don't speak versions at all),
|
||||
`1080×1080 → "ten-eighty by ten-eighty"`.
|
||||
5. **URLs**: `hyperframes.heygen.com → "hyperframes dot hey-jen dot com"`.
|
||||
6. **Filenames/extensions**: `.mp4 → "dot em pee four"` — or rephrase so the
|
||||
extension isn't spoken.
|
||||
7. **Emphasis/pauses**: commas and em-dashes, never caps. Ellipses are
|
||||
unreliable in TTS — use an em-dash.
|
||||
|
||||
The shared vocabulary lives in `references/lexicon.json`
|
||||
(`display → spoken`). Consult it for EVERY technical term; if a term is
|
||||
missing, ask the user for the pronunciation and add the entry — never guess,
|
||||
never ship unheard. New entries: listen to that line in the generated VO
|
||||
before accepting.
|
||||
|
||||
## Alignment (spoken timestamps → display captions)
|
||||
|
||||
`heygen-tts.mjs --words` returns word timestamps of the SPOKEN text.
|
||||
`scripts/align-captions.mjs` walks the spoken stream against the token lines
|
||||
(one display token may cover several spoken words — "C L I" is three) and
|
||||
emits `captions.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"lines": [
|
||||
{
|
||||
"end": 3.1,
|
||||
"w": [
|
||||
["This", 0.22],
|
||||
["week", 0.4],
|
||||
["JSON", 1.1]
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Each display word carries the start time of its FIRST spoken word; a line's
|
||||
`end` = the next line's start (last line: last word end + 0.6). The aligner
|
||||
warns `MISMATCH` when the heard word doesn't fuzzy-match the expected spoken
|
||||
form — every warning must be resolved (fix the lexicon spelling or the
|
||||
transcript) before the captions are trusted.
|
||||
|
||||
## Caption rail (rendering)
|
||||
|
||||
Per `captions-overlay`: a quiet OVERLAY, never a reserved band. One line,
|
||||
bottom-center (top: 1002px on 1080-square), TT Norms Pro 500 25px,
|
||||
ink .92, soft dark text-shadow, words fading in (0.12s) on their timestamps,
|
||||
phrase swaps as sets. Keep critical small text out of the bottom ~80px
|
||||
center span; everything else may run under the rail.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Visualization Registry
|
||||
|
||||
The routing table for "show, don't tell". Classes, strongest first:
|
||||
|
||||
1. **ui-recreate** — the change lives in a surface we can faithfully mock.
|
||||
2. **ui-analog** — no exact surface, but an honest UI metaphor exists
|
||||
(panel, meter, pipeline) whose behavior IS the change.
|
||||
3. **terminal** — the change is a CLI command/flag; type it, show the result.
|
||||
4. **checklist** — non-visual (fix lists, dependency bumps). Last resort.
|
||||
|
||||
Never invent UI that implies a screen that doesn't exist — an analog must
|
||||
depict the _behavior_ (speed, batching, caching), not a fake product page.
|
||||
|
||||
## Known surfaces (ui-recreate)
|
||||
|
||||
| Surface | Mock anatomy | Proven choreography |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| **Studio editor / timeline** | glass app frame: titlebar (traffic dots, mono app name, Export pill), preview strip w/ gradient art, ruler + ticks, lanes, playhead, mono-labeled clips | clips drop/stack into lanes; drag→edge/playhead snap w/ green snap-line flash; marquee → group move → group resize; playhead scrub drives preview art (hue-rotate) |
|
||||
| **Inspector / design panel** | panel: MONO section headers (INSPECTOR / VARIABLES), key-value rows, hairline dividers, dashed empty slot | binding pill (`{{ var }}`) flies into a property slot; value swap via masked slide (old up-out, new up-in); selection box draws on canvas |
|
||||
| **Canvas + element** | mini stage card, dashed selection box, live text element | text updates same-frame with panel edits (green underline pulse = the live-preview moment) |
|
||||
| **Variant renders** | small cards: display-font title + mono filename | cascade out diagonally, stagger ≤0.15s |
|
||||
| **Storyboard view** | row of scene thumbnails w/ mono scene labels | thumbnails file in as a waterfall; one gets dragged to reorder |
|
||||
| **Terminal / CLI** | glass strip, mono 19-20px, `$ ` prompt dim | chars type (stagger .02), result line lands after a 0.3-0.5s beat |
|
||||
| **Render panel** | RENDER header, big tabular-nums frame counter, progress bar (green fill = the moment), status chips | bar + counter run with `power2.in` (slow→fast reads as "faster"); chips land on their VO words |
|
||||
|
||||
## Proven analogs (ui-analog)
|
||||
|
||||
| Change type | Analog | Behavior |
|
||||
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Color grading / LUT | slider rows (label + track + knob) beside footage art | each knob move re-filters the art SAME frame (causal) |
|
||||
| Render/extraction speed | render panel counter + bar | ease tells the story; add a before/after time chip if the claim is numeric |
|
||||
| Batching (frames, requests) | row of small ticks | brackets draw around groups; ticks nudge into clusters |
|
||||
| Caching | two identical request rows | first row runs a full bar; second short-circuits instantly to ✓ with a `cache` chip |
|
||||
| Import/translation pipelines (e.g. Figma→HF) | source artifact card morphs/docks into a HF comp card — a pure DOM/CSS mock of the source (NO Figma API, no tokens, nothing fetched) | staged: source card → arrow/flight of extracted chips (tokens, components, motion) → assembled comp; chips are the carriers |
|
||||
| One-pass / dedupe | N parallel item rows collapse onto one shared lane | rows glide onto one bar; count chip decrements |
|
||||
| Concurrency caps / locks | queue of chips entering a gate | first k pass, rest hold; gate chip shows the cap |
|
||||
| Error surfacing (toast, reason) | the surface's corner grows a toast card | action fails subtly → toast slides in, mono reason text |
|
||||
|
||||
## Checklist scene (last resort)
|
||||
|
||||
Glass card, ≤6 mono rows, green ✓ ticks landing on each item's VO word
|
||||
(back.out(1.5), 0.3s) + row brightness pulse. Items beyond 6: cut, they live
|
||||
in the digest link.
|
||||
|
||||
## Adding a surface
|
||||
|
||||
When a new UI area ships, add a row here (anatomy + choreography) the first
|
||||
time it's mocked, so the next changelog reuses it instead of re-deriving it.
|
||||
Reference in New Issue
Block a user