mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-13 15:49:53 +00:00
kiritowooandkiritowoo
5242dde2dc
* feat(telemetry): attribute renders to the authoring workflow skill Add an optional `--skill` flag to `hyperframes render` and tag the `render_complete` / `render_error` events with `authoring_skill`, so render usage can be broken down per authoring workflow. The value is slug-gated (a malformed value is ignored) and the existing anonymous / opt-out telemetry pipeline is otherwise unchanged. Each end-user workflow that renders now passes `--skill=<name>` on its render command: embedded-captions, faceless-explainer, graphic-overlays, motion-graphics, music-to-video, pr-to-video, product-launch-video, remotion-to-hyperframes, website-to-video. Not instrumented, by design: general-video renders freeform with no canonical render command to attach to, and slideshow produces an interactive deck rather than a rendered video. Both can follow up if per-skill numbers are wanted. * fix(telemetry): address review — shared slug util, equals-form flag, invalid-value warning - Extract the SKILL_SLUG regex + a normalizeSkillSlug() helper into telemetry/skill.ts, shared by the `events` and `render` commands (the regex was duplicated). `render` adopts normalizeSkillSlug (so it now trims the value, matching `events`); `events` references the shared SKILL_SLUG. + unit test. - `render` warns on a non-empty but invalid --skill value (e.g. a camelCase typo) so attribution isn't silently lost — stderr only, never fails the render. - embedded-captions render script: `--skill embedded-captions` -> `--skill=embedded-captions`. On an older CLI that does not declare --skill, the space form leaks the value as a positional and clobbers the project dir (resolveProject fails); the equals form is parsed as a self-delimiting flag and safely ignored. Verified via Node parseArgs(strict:false). Addresses review feedback on the PR (shared util + .trim drift, version-skew safety, invalid-value visibility). --------- Co-authored-by: kiritowoo <295860553+kiritowoo@users.noreply.github.com>
213 lines
18 KiB
Markdown
213 lines
18 KiB
Markdown
---
|
||
name: pr-to-video
|
||
description: "turn a GitHub pull request (a PR URL like github.com/<owner>/<repo>/pull/<N>, an <owner>/<repo>#<N> ref, or 'this PR' in a checked-out repo) into a code-change explainer video, up to ~3 min (sweet spot 30-90s) — changelog, feature reveal, fix, or refactor walkthrough, rendered from the diff / commits / files. The input is a CODE CHANGE read via the gh CLI; there is no website capture. Use this skill for a GitHub PR. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a topic explainer with no PR (use /faceless-explainer), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first."
|
||
---
|
||
|
||
# PR to HyperFrames
|
||
|
||
Use this skill to ingest a GitHub pull request, understand the change, plan a code-change explainer, and build it frame by frame in HyperFrames. The input is a **code change** (read via `gh`), not a website — there is **no capture step and no real assets** beyond the contributors' avatars.
|
||
|
||
> **Confirm the route before Step 0.** You are the orchestrator. Run each step, verify its gate, and only then continue. This skill is for a **GitHub pull request** (a code change). Route other intents elsewhere: a product launch/promo → `/product-launch-video`; a general website tour → `/website-to-video`; a topic explainer with no PR → `/faceless-explainer`; captions on existing footage → `/embedded-captions`; a short unnarrated motion graphic → `/motion-graphics`; a whole-repo or multi-PR release walkthrough → `/general-video`. **Out of scope:** live / at-render-time data — PR facts are read once at author time and baked in. If the user says only "make a video" or the route is uncertain, read `/hyperframes` first.
|
||
|
||
You are the orchestrator. Work in `videos/<project>/`. Run steps in order and pass each gate before continuing. User-gated steps are Step 0, Step 3, and Step 6. Do every step yourself except Step 5, where you dispatch one sub-agent per frame. Do not put design or motion rules here; those live in the frame-worker sub-agent, `hyperframes-creative`, and `hyperframes-animation`.
|
||
|
||
Workflow: Step 0 setup → `hyperframes.json`; Step 1 ingest → `capture/extracted/` + `assets/<login>.png`; Step 2 design system → `frame.md`; Step 3 storyboard/script → `STORYBOARD.md` and `SCRIPT.md`; Step 3.1 audio → `audio_meta.json`; Step 4 visual design → enriched `STORYBOARD.md`; Step 5 frames → `compositions/frames/NN-*.html` and `index.html`; Step 6 final render → `renders/video.mp4`.
|
||
|
||
---
|
||
|
||
## Step 0: Setup and Brief
|
||
|
||
Goal: Lock the PR reference and the core video brief, and create the HyperFrames project if needed.
|
||
|
||
Get the **PR reference** (a full URL, an `<owner>/<repo>#<N>` ref, or "this PR" in a checked-out repo) and, in one message, confirm the brief — lead with a recommended default for each and pre-fill anything `/hyperframes` already set: **angle** (changelog / feature-reveal / fix-explainer / refactor-walkthrough — default: infer from the PR), **audience** (default: developers), **length** (default ~60-90s), **aspect** (default 16:9), **language**. The style is always **claude**. Proceed only after the user replies; a "go" accepts the defaults.
|
||
|
||
Initialize only if `hyperframes.json` is missing. Name `<project>` from the PR in kebab-case, such as `acme-sdk-pr-1842`; never use the workspace name or a timestamp.
|
||
|
||
`npx hyperframes init "videos/<project>" --non-interactive --skip-skills --example=blank`
|
||
|
||
**Gate:** `hyperframes.json` exists; the PR ref is captured; angle, length, aspect ratio, and language are locked.
|
||
|
||
---
|
||
|
||
## Step 1: Ingest the PR (no capture)
|
||
|
||
Goal: Fetch the PR's facts and fold them into the project as the source of information. There is **no website capture**. `fetch-pr.mjs` runs `gh` deterministically — completing the files list via paginated `gh api` so a large PR doesn't truncate at ~100 files, and writing only `capture/pr.json` + `capture/diff.patch` (no scratch dir). Then `ingest.mjs` folds that into the synthetic capture package offline.
|
||
|
||
```bash
|
||
PR="<url | owner/repo#N | N>"
|
||
|
||
# Fetch the PR deterministically: runs gh, completes the files list via paginated
|
||
# gh api (so a big PR doesn't truncate at ~100 files), writes only capture/pr.json +
|
||
# capture/diff.patch — no scratch dir. gh auth / not-found / private errors exit 1 here.
|
||
(cd "videos/<project>" && node <SKILL_DIR>/scripts/fetch-pr.mjs --pr "$PR" --out-dir ./capture)
|
||
|
||
# Offline transform → capture/extracted/{tokens.json (colors:[] → claude palette),
|
||
# visible-text.txt (the brief), people.json (contributors, bot-filtered, avatarFile=assets/<login>.png)}.
|
||
(cd "videos/<project>" && node <SKILL_DIR>/scripts/ingest.mjs \
|
||
--pr-json ./capture/pr.json --diff ./capture/diff.patch --out-dir ./capture/extracted)
|
||
|
||
# The people front's one network step — download each contributor's GitHub avatar to
|
||
# assets/<login>.png for an optional credits close. Best-effort; always exits 0.
|
||
(cd "videos/<project>" && node <SKILL_DIR>/scripts/fetch-people-avatars.mjs \
|
||
--people ./capture/extracted/people.json)
|
||
```
|
||
|
||
If `fetch-pr.mjs` exits 1 (gh auth / not found / private), report its stderr and stop — **do not fabricate PR contents**. If `ingest.mjs` exits 1, read its stderr (usually a malformed `pr.json`), fix, and rerun (deterministic). `fetch-people-avatars.mjs` always exits 0; missing avatars just mean no credits close to author.
|
||
|
||
**Gate:** `capture/pr.json`, `capture/diff.patch`, `capture/extracted/tokens.json`, `capture/extracted/visible-text.txt`, and `capture/extracted/people.json` exist; you can state the PR's change in one clear sentence. `assets/<login>.png` is best-effort — its absence is not a failure.
|
||
|
||
---
|
||
|
||
## Step 2: Design System
|
||
|
||
Goal: Adopt the claude frame preset; a script turns it into this video's `frame.md` + caption skin.
|
||
|
||
The style is fixed — **claude** (warm editorial; a navy code surface built for diffs). Run:
|
||
|
||
```bash
|
||
node <SKILL_DIR>/scripts/build-frame.mjs --preset claude --hyperframes .
|
||
```
|
||
|
||
The script copies the claude preset's `FRAME.md` → `frame.md`, remixes it onto any brand tokens in `capture/extracted/tokens.json` (a PR has none → `colors:[]`/`fonts:[]` keeps claude's own palette, a complete design), copies the preset's `caption-skin.html`, and self-validates (exits 1 on a broken mapping). Proceed as soon as it exits 0 — no hand-editing.
|
||
|
||
**Gate:** `build-frame.mjs` exited 0 — `frame.md` exists from the claude preset, and `caption-skin.html` is at the project root.
|
||
|
||
---
|
||
|
||
## Step 3: Storyboard and Script
|
||
|
||
Goal: Turn the PR into an approved frame-by-frame explanation plan.
|
||
|
||
Read `references/story-design.md`, `../hyperframes-core/references/storyboard-format.md`, and `../hyperframes-core/references/script-format.md`. Use them to write `STORYBOARD.md` and, when narration is needed, `SCRIPT.md`.
|
||
|
||
Use `story-design.md` for the PR archetype (changelog / feature-reveal / fix-explainer / refactor-walkthrough), the PR-native frame types, hook, persuasion, beats, the per-frame word budget, and the optional credits close. The sequence comes from **narrative design, not the diff's file order** — explain the change, don't read the diff aloud. Feature 2–4 real diff hunks (from `capture/diff.patch`), each a small legible snippet; name the `code-*` block each wants in the frame's `scene`. Frames carry no `asset_candidates` except an optional `credits` close (2–6 `assets/<login>.png` avatars). Use the exact required fields from the storyboard and script references.
|
||
|
||
After drafting, show a frame-by-frame summary. In that same message ask the user (a) to approve or request changes, and (b) whether they want a live preview of the storyboard scaffold (`npx hyperframes preview`) — open it only on a yes. Iterate until approved; carry the preview choice to Step 6.
|
||
|
||
**Gate:** `STORYBOARD.md` exists, every frame has the required narrative fields, `SCRIPT.md` exists when narration is needed, and the user approved the plan.
|
||
|
||
---
|
||
|
||
## Step 3.1: Audio
|
||
|
||
Goal: Generate narration, word timings, music, and audio metadata from the approved script.
|
||
|
||
Start audio after Step 3 approval. Run it in the background, then continue to Step 4.
|
||
|
||
`node <SKILL_DIR>/scripts/audio.mjs --script ./SCRIPT.md --storyboard ./STORYBOARD.md --hyperframes . --out ./audio_meta.json &`
|
||
|
||
The audio script handles narration, word timings, BGM lookup from HeyGen's music library, and timing metadata. BGM mood comes from the storyboard's `music:` field. This uses the HeyGen Audio API for retrieval, not generation, and the same `~/.heygen` credential as TTS. For provider details, read `../hyperframes-media/references/tts.md`.
|
||
|
||
If there is no narration and no `SCRIPT.md`, skip voice generation. BGM may still run if the storyboard has a music mood.
|
||
|
||
**Gate:** audio job has started, or the project is marked silent.
|
||
|
||
---
|
||
|
||
## Step 4: Frame Visual Design
|
||
|
||
Goal: Add the visual direction, layout intent, and motion choices to each storyboard frame.
|
||
|
||
Edit `STORYBOARD.md` in place. Do not create another storyboard. Use `frame.md` as the source of truth for color, type, layout feel, and style.
|
||
|
||
Read `references/visual-design.md`, `references/composition.md`, `references/motion-language.md`, `references/code-vocabulary.md`, and `../hyperframes-animation/`. Use `visual-design.md` for required frame fields and the required `## Video direction` block, and for how a code beat names a `code-*` block as its `focal`. Use `code-vocabulary.md` to pick the right block per beat (diff = `code-diff`, refactor = `code-morph`, new code = `code-typing`, …). Use `composition.md` for layout/hierarchy/focal points and `motion-language.md` + `../hyperframes-animation/` for valid effect and blueprint IDs. Do not invent effect names or block/blueprint IDs.
|
||
|
||
For every frame, add required visual and motion fields, including `effects` and `focal` and/or `roles`. For a code beat, name the `code-*` block as the `focal` and let `effects` choreograph the surrounding claude Code Surface (not the code animation, which the block owns). Add one video-wide `## Video direction` block.
|
||
|
||
Do not change story, script, `transition_in`, `asset_candidates`, or the PR source. Do not write HTML in this step. There is **no asset-staging step** — the only real assets are the credits avatars, already in `assets/`.
|
||
|
||
**Gate:** every frame has `effects` plus `focal` and/or `roles`; code frames name a `code-*` block; `## Video direction` exists.
|
||
|
||
---
|
||
|
||
## Step 5: Build Frames
|
||
|
||
Goal: Build every storyboard frame as an HTML composition and assemble the playable video.
|
||
|
||
Wait for Step 3.1 audio to finish if audio was started. Then sync durations and fetch SFX; skip both if silent.
|
||
|
||
`node <SKILL_DIR>/scripts/audio.mjs sync-durations --audio-meta ./audio_meta.json --storyboard ./STORYBOARD.md`
|
||
|
||
`node <SKILL_DIR>/scripts/audio.mjs fetch-sfx --storyboard ./STORYBOARD.md --hyperframes .`
|
||
|
||
Duration sync is mechanical: real voice duration wins; silent frames keep estimates; never hand-edit synced durations.
|
||
|
||
**Pre-install the registry blocks** named across `STORYBOARD.md` once, before dispatch, so parallel workers don't race on the registry:
|
||
|
||
`for b in <each registry block named in the storyboard>; do npx hyperframes add "$b"; done`
|
||
|
||
Before dispatch, read `sub-agents/frame-worker.md` and `../hyperframes-core/references/subagent-dispatch.md`. Dispatch one sub-agent per frame, in parallel if possible; otherwise run workers in waves. Each worker gets exactly one frame. Each worker's context must include `PROJECT_DIR`, `frame_id`, canvas size, caption status and keep-out band if captions are enabled, `ANIM_DIR` (absolute path to `../hyperframes-animation/`), and the absolute path to `references/code-vocabulary.md`. Each worker reads `frame.md`, its own `## Frame N` block, the recipe body for each cited effect/blueprint ID, and — for a code beat — `code-vocabulary.md` for the named block's inputs. Each worker writes only `compositions/frames/NN-*.html`; workers never edit `STORYBOARD.md`.
|
||
|
||
As each worker returns, mark that frame `animated` in `STORYBOARD.md`.
|
||
|
||
After audio timings exist, build captions in the background and assemble the index:
|
||
|
||
`node <SKILL_DIR>/scripts/captions.mjs build --storyboard ./STORYBOARD.md --audio-meta ./audio_meta.json --hyperframes . --out ./caption_groups.json &`
|
||
|
||
`node <SKILL_DIR>/scripts/assemble-index.mjs --storyboard ./STORYBOARD.md --hyperframes .`
|
||
|
||
`captions.mjs` uses the project's `caption-skin.html` (claude's, copied in Step 2), injecting brand tokens from `frame.md`; `captions: skipped (<reason>)` is valid. `assemble-index.mjs` stages the credits avatars from `assets/` as an idempotent backstop.
|
||
|
||
**Gate:** every frame is marked `animated`, `index.html` exists, and captions are built or explicitly skipped.
|
||
|
||
---
|
||
|
||
## Step 6: Finalize
|
||
|
||
Goal: Verify the assembled video, get user approval, and render the final MP4.
|
||
|
||
Inject transitions, run checks, pause for review, then render.
|
||
|
||
`node <SKILL_DIR>/scripts/transitions.mjs inject --storyboard ./STORYBOARD.md --hyperframes .`
|
||
|
||
`node <SKILL_DIR>/scripts/transitions.mjs verify --storyboard ./STORYBOARD.md --index ./index.html`
|
||
|
||
`npx hyperframes lint`
|
||
|
||
`npx hyperframes validate`
|
||
|
||
`npx hyperframes inspect`
|
||
|
||
`npx hyperframes snapshot --at <frame-midpoints>`
|
||
|
||
If a command fails, surface stderr and stop. Do not pile on recovery commands. If a gate names a frame, fix `compositions/frames/NN-*.html` with the cheapest safe fix: edit the frame HTML for a local issue; re-dispatch the frame worker only when the whole shot must be rebuilt.
|
||
|
||
**Known false-positive — do not chase it.** `inspect` may report a handful of `text_box_overflow` errors of ~1–4px on the **caption** highlight words (selector `#caption-word-*` / `.caption-line`). The caption pill uses a deliberately snug `line-height` (set once in `scripts/captions.mjs`) and has **no `overflow:hidden`**, so a heavy display glyph's ink spills a few px into the pill's own padding — nothing is actually clipped. Treat these as expected and proceed. Do **not** inflate the caption `line-height` (it balloons the pill, which is worse) and do **not** re-dispatch a frame for them. Only act on a `text_box_overflow` when it names a **frame** element (`#el-NN-*`), not a caption word.
|
||
|
||
After checks pass, pause for user review. The video is assembled, viewable, and editable in Studio. Manage preview only once across Step 3 and Step 6: open it if the user asked earlier, offer it if they declined earlier, do not ask again if they are already reviewing in Studio.
|
||
|
||
Preview: `npx hyperframes preview`
|
||
|
||
Render only after user approval:
|
||
|
||
`npx hyperframes render --skill=pr-to-video --quality high --output renders/video.mp4`
|
||
|
||
Do not rerun `lint`, `validate`, `inspect`, or `snapshot` after rendering unless the user asks.
|
||
|
||
**Gate:** `lint`, `validate`, and `inspect` passed before render; user approved at the review pause; `renders/video.mp4` exists. Final reply states the MP4 path and final duration.
|
||
|
||
---
|
||
|
||
## Quick Reference
|
||
|
||
**Formats:** landscape `1920x1080` by default; portrait `1080x1920`; square `1080x1080`. Set the format once in the storyboard frontmatter.
|
||
|
||
**PR deltas vs a captured-asset workflow:** no Step 1 capture (the `gh` CLI ingests the PR into a synthetic `capture/extracted/` package — `tokens.json` + `visible-text.txt` + `people.json`); the only real assets are the contributors' `assets/<login>.png` avatars (an optional credits close); no `asset-descriptions.md`, no asset-staging step. Code beats are rendered by the `code-*` registry blocks on claude's navy Code Surface; the style is always **claude**.
|
||
|
||
**Background scripts:** the workflow ships these under `scripts/`: `fetch-pr` (PR → `capture/pr.json` + `diff.patch` via `gh`; large-PR-safe, no scratch), `ingest` (→ synthetic capture package; offline), and `fetch-people-avatars` (contributor avatars → `assets/`); plus the shared engine — `build-frame` (adopt + brand-remix a preset into `frame.md` + caption skin), `audio` (TTS, BGM, SFX, duration sync), `captions`, `transitions` (inject + verify), and `assemble-index`. Everything else is the `hyperframes` CLI. Code blocks install via `npx hyperframes add <name>`.
|
||
|
||
| Read | When |
|
||
| ---------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
|
||
| `[references/story-design.md](references/story-design.md)` | Step 3: plan the PR explanation. |
|
||
| `[../hyperframes-core/references/storyboard-format.md](../hyperframes-core/references/storyboard-format.md)` | Step 3: write `STORYBOARD.md`. |
|
||
| `[../hyperframes-core/references/script-format.md](../hyperframes-core/references/script-format.md)` | Step 3: write `SCRIPT.md`. |
|
||
| `[../hyperframes-media/references/tts.md](../hyperframes-media/references/tts.md)` | Step 3.1: choose or understand TTS providers. |
|
||
| `[references/visual-design.md](references/visual-design.md)` | Step 4: enrich the storyboard visually. |
|
||
| `[references/code-vocabulary.md](references/code-vocabulary.md)` | Step 4 + 5: pick + fill the `code-*` block for a code beat. |
|
||
| `[references/composition.md](references/composition.md)` | Step 4: judge composition. |
|
||
| `[references/motion-language.md](references/motion-language.md)` | Step 4: judge motion language. |
|
||
| `[../hyperframes-animation/](../hyperframes-animation/)` | Step 4: cite effect and blueprint IDs. |
|
||
| `[sub-agents/frame-worker.md](sub-agents/frame-worker.md)` | Step 5: dispatch per-frame workers. |
|
||
| `[../hyperframes-core/references/subagent-dispatch.md](../hyperframes-core/references/subagent-dispatch.md)` | Step 5: dispatch sub-agents safely. |
|
||
| `[../hyperframes-creative/frame-presets/claude/FRAME.md](../hyperframes-creative/frame-presets/claude/FRAME.md)` | Step 2: the claude preset (fixed style). |
|