ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-14 00:00:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(skills): mode-first briefs, value-first storyboards, and destination defaults across creation workflows (#2058)

Browse Source 
* feat(skills): add brief contract — interaction modes + shared intake fields across workflows

New hyperframes-core/references/brief-contract.md, the shared intake
contract every creation workflow now runs its brief against:

- §1 interaction mode: collaborative (default) vs autonomous, ongoing
  vs one-time signals, mode set once and carried forward, and a gate
  taxonomy (preference / checkpoint / quality / routing) — autonomous
  skips waiting, never verification
- §2 field registry: destination→aspect derivation (feed → 1:1,
  Shorts/TikTok → 9:16, else 16:9), message, angle, length, audience,
  language, narration — each workflow binds fields as ask or state
- §3 question rules: one round with one question per asked field
  (native question UI mandatory when available, recommended option
  first with a receipt), never drop a question as inferable, and a
  mode legend advertised in the intro text instead of asked

Wired into the surfaces:

- hyperframes router: detect mode at entry, derive aspect from
  destination instead of stating 16:9
- product-launch-video / pr-to-video / faceless-explainer: ask/state
  binding tables at Step 0; Step 3/6 checkpoint-gate branches
  (autonomous posts a heads-up with a preview hint before render)
- website-to-video: local mode definition now defers to the contract
- music-to-video, general-video, embedded-captions, talking-head-recut,
  slideshow, motion-graphics: mode semantics wired per gate type
- storyboard-format: new optional 'mode' frontmatter key
- pr-to-video: length tier is a ceiling, not a floor — a one-headline
  PR recommends inside the 30–90s sweet spot regardless of diff size

* feat(skills): story spine + mode-first brief across creation workflows

Story — the reverse-iceberg feedback:

- New hyperframes-creative/references/story-spine.md, three rules for
  the narrated workflows: the hook speaks the viewer's outcome
  language, the value claim lands by beat 2 (implementation is the
  footnote of the story, not the spine), and the storyboard is
  presented as a proposal — 'This video tells [audience] that
  [message]' plus a per-frame why: drawn from narrativeRole
- pr-to-video: feature-reveal reordered promise-first (impact leads,
  diff/mechanism follow as evidence); hooks ban file/function names;
  fix-explainer, refactor-walkthrough, changelog unchanged
- product-launch-video / faceless-explainer hook rules aligned to the
  spine; website-to-video's beat summary gains the echo line + why:;
  general-video points at the spine from its plan step

Brief — hardened after live-test drift:

- Mode is now the first question (Collaborative recommended vs
  Autonomous), its own round, skipped when the request carries a
  signal; autonomous asks nothing further until one final
  preview-or-render question before render
- Step 0 rewritten as a literal two-round question script in each
  shot-sequence workflow (website-to-video's editorial register,
  channel-agnostic); brief-contract.md §3 reduced to invariants so the
  procedure lives in exactly one place

* feat(skills): split type minimums by viewing context

typography.md: full-screen viewing keeps body 20px / headline 60px;
in-feed destinations (X / LinkedIn / Instagram — brief-contract's
destination field) scale to body >=32px, headline >=90px, data labels
>=24px. First-pass values, to be calibrated against real renders.

* feat(skills): storyboard proposal as a table + credits close by default

- story-spine § 3: the proposal presents frames as a markdown table
  (frame · beat · on screen · why) instead of dense paragraphs; the
  three shot-sequence workflows and website-to-video's beat summary
  reference the same shape
- pr-to-video: the credits close is now the default ending — every PR
  video ends on a contributors frame (committers by commit count, 1-6
  avatars), with no taste judgment; the only skip is when no avatar
  was fetched, and the user can cut the frame in the proposal

* fix(skills): address review nits on the brief/story contracts

- embedded-captions: the identity procedure now states both sides of the
  preference gate inline (user picks; autonomous picks with a stated why)
- website-to-video step-2-brief: note that its mode section is the
  workflow's application of brief-contract.md, not a second definition
- brief-contract: resuming a project reads mode from STORYBOARD.md
  frontmatter — a recorded mode counts as set, closing the write-only gap

* docs(skills): add a non-code receipts example to the brief contract

Review nit (jrusso1020, #2058): the receipts example in § 3 was
PR-video-shaped only. A destination-shaped example joins it so the rule
reads as workflow-neutral.
This commit is contained in:
WaterrrForever
2026-07-09 01:58:51 +08:00
committed by GitHub
parent 81884a7495
commit 17b852784b
23 changed files with 341 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/pr-to-video/SKILL.md
+47 -29
View File
@@ -13,7 +13,7 @@ Use this skill to ingest a GitHub pull request, understand the change, plan a co
> **Confirm the route before Step 0.** You are the orchestrator. Run each step, verify its gate, and only then continue. This skill is for a **GitHub pull request** (a code change). Route other intents elsewhere: a product launch/promo → `/product-launch-video`; a general website tour → `/website-to-video`; a topic explainer with no PR → `/faceless-explainer`; captions on existing footage → `/embedded-captions`; a short unnarrated motion graphic → `/motion-graphics`; a whole-repo or multi-PR release walkthrough → `/general-video`. **Out of scope:** live / at-render-time data — PR facts are read once at author time and baked in. If the user says only "make a video" or the route is uncertain, read `/hyperframes` first.
You are the orchestrator. Work in `videos/<project>/`. Run steps in order and pass each gate before continuing. User-gated steps are Step 0, Step 3, and Step 6. Do every step yourself except Step 5, where you dispatch one sub-agent per frame. Do not put design or motion rules here; those live in the frame-worker sub-agent, this skill's local `../hyperframes-animation/rules/` + `../hyperframes-animation/blueprints/`, and `hyperframes-creative`.
You are the orchestrator. Work in `videos/<project>/`. Run steps in order and pass each gate before continuing. User-gated steps are Step 0, Step 3, and Step 6. Read `../hyperframes-core/references/brief-contract.md` before Step 0 — it defines the two modes, the gate types, and the brief fields; the mode governs the Step 0/3/6 gates. Do every step yourself except Step 5, where you dispatch one sub-agent per frame. Do not put design or motion rules here; those live in the frame-worker sub-agent, this skill's local `../hyperframes-animation/rules/` + `../hyperframes-animation/blueprints/`, and `hyperframes-creative`.
Workflow: Step 0 setup → `hyperframes.json`; Step 1 ingest → `capture/extracted/` + `assets/<login>.png`; Step 2 design system → `frame.md`; Step 3 storyboard/script → `STORYBOARD.md` and `SCRIPT.md`; Step 3.1 audio → `audio_meta.json`; Step 4 visual design → enriched `STORYBOARD.md`; Step 5 frames → `compositions/frames/NN-*.html` and `index.html`; Step 6 final render → `renders/video.mp4`.
@@ -23,7 +23,23 @@ Workflow: Step 0 setup → `hyperframes.json`; Step 1 ingest → `capture/extrac
Goal: Lock the PR reference and the core video brief, and create the HyperFrames project if needed.
Get the **PR reference** (a full URL, an `<owner>/<repo>#<N>` ref, or "this PR" in a checked-out repo) and, in one message, confirm the brief — lead with a recommended default for each and pre-fill anything `/hyperframes` already set: **angle** (changelog / feature-reveal / fix-explainer / refactor-walkthrough — default: infer from the PR), **audience** (default: developers), **length** (default: **scale to the PR's change size** — see below), **aspect** (default 16:9), **language**. The style is always **claude**. Proceed only after the user replies; a "go" accepts the defaults.
Get the **PR reference** (a full URL, an `<owner>/<repo>#<N>` ref, or "this PR" in a checked-out repo), then confirm the brief in two rounds — through the question UI when the environment has one, conversationally otherwise. The intro text states **message** (the ONE thing the video must say about this change, in one sentence), **language**, and the style (always **claude**). Skip a question only when the user's request already answered it.
**Round 1 — mode.** One question, asked first. Skip it when the request already carried a signal ("surprise me" / "just build it"):
- **Collaborative (recommended)** — confirm the key choices together before building.
- **Autonomous** — every decision is made for the user, each stated with its reason; the only remaining question is preview-before-render.
Autonomous → ask nothing more. State the locked brief (all fields + receipts) as a heads-up and proceed straight through; the preview question waits at Step 6.
**Round 2 — the brief (collaborative).** One round, these four questions, recommended option first with its receipt:
- **Angle — what story does this PR tell?** changelog / feature-reveal / fix-explainer / refactor-walkthrough; recommend the one the PR itself suggests, with its basis.
- **Audience — who is it for?** developers (default) · mixed technical · non-technical stakeholders.
- **Length — how long?** From the size table below; the tier is a ceiling, and a one-headline story recommends inside 30–90s ("+A/−D across F files" is the receipt).
- **Destination — where will it play?** YouTube / embed → 16:9 (default for a code explainer) · X / LinkedIn / Instagram feed → 1:1 · Shorts / TikTok → 9:16.
A "go" accepts all recommended defaults.
**Recommend the length from the PR's change size**, not a fixed guess. Before confirming the brief, peek at the PR once — a read-only call that also grounds the angle (Step 1 still does the full deterministic fetch):
@@ -40,7 +56,7 @@ Pick the tier from `additions + deletions` (nudged up by `changedFiles`) and lea
| substantial (~200–600 lines) | ~70–110s |
| large (≳ 600 lines, or 25+ files) | ~110–180s |
State the basis in one phrase when you propose it (e.g. "~40s — small change, +44/−13 across 12 files"). A huge PR doesn't mean a long video — if the story is one headline change, keep it tight and say so.
State the basis in one phrase when you propose it (e.g. "~40s — small change, +44/−13 across 12 files"). A huge PR doesn't mean a long video — the tier is a **ceiling** on how much story the diff can support, never a floor to fill. When the story is **one headline change**, recommend inside the 30–90s sweet spot regardless of the size tier, and say so (the tier's range can still appear as a non-recommended option for a fuller walkthrough).
Initialize only if `hyperframes.json` is missing. Name `<project>` from the PR in kebab-case, such as `acme-sdk-pr-1842`; never use the workspace name or a timestamp.
@@ -48,7 +64,7 @@ Initialize only if `hyperframes.json` is missing. Name `<project>` from the PR i
**Show sign-in status before the brief** — run `npx hyperframes auth status` and **relay its output verbatim (don't paraphrase or rewrite it).** It reports whether voice/BGM will use HeyGen or local engines and, when not signed in, how to sign in. **If not signed in, STOP and wait for the user to choose — sign in, or say "go"/"offline" to continue with local engines — before asking the brief or anything else.** Treat it as a real decision point, not a passing note; don't fold the choice into the brief question, and don't write keys into a per-repo `.env`. (In autonomous mode, note the status and continue offline.) See `../media-use` → Preflight for the canonical guidance.
**Gate:** `hyperframes.json` exists; the PR ref is captured; angle, length, aspect ratio, and language are locked; sign-in status was shown (signed in, or continuing offline).
**Gate:** `hyperframes.json` exists; the PR ref is captured; angle, audience, length, destination → aspect, message, and language are locked; sign-in status was shown (signed in, or continuing offline).
---
@@ -70,7 +86,7 @@ PR="<url | owner/repo#N | N>"
--pr-json ./capture/pr.json --diff ./capture/diff.patch --out-dir ./capture/extracted)
# The people front's one network step — download each contributor's GitHub avatar to
# assets/<login>.png for an optional credits close. Best-effort; always exits 0.
# assets/<login>.png for the credits close. Best-effort; always exits 0.
(cd "videos/<project>" && node <SKILL_DIR>/scripts/fetch-people-avatars.mjs \
--people ./capture/extracted/people.json)
```
@@ -101,13 +117,13 @@ The script copies the claude preset's `FRAME.md` → `frame.md`, remixes it onto
Goal: Turn the PR into an approved frame-by-frame explanation plan.
Read `references/story-design.md`, `../hyperframes-animation/blueprints-index.md`, `../hyperframes-core/references/storyboard-format.md`, and `../hyperframes-core/references/script-format.md`. Use them to write `STORYBOARD.md` and, when narration is needed, `SCRIPT.md`.
Read `../hyperframes-creative/references/story-spine.md` (hook language, value-before-evidence, storyboard-as-proposal), `references/story-design.md`, `../hyperframes-animation/blueprints-index.md`, `../hyperframes-core/references/storyboard-format.md`, and `../hyperframes-core/references/script-format.md`. Use them to write `STORYBOARD.md` and, when narration is needed, `SCRIPT.md`.
Use `story-design.md` for the PR archetype (changelog / feature-reveal / fix-explainer / refactor-walkthrough), the PR-native frame types, hook, persuasion, beats, the per-frame word budget, and the optional credits close. The sequence comes from **narrative design, not the diff's file order** — explain the change, don't read the diff aloud. As a **soft guide**, consult the role→blueprint menu in `../hyperframes-animation/blueprints-index.md`: for each beat, write the voiceover in the shape its candidate blueprint implies and tag that candidate `blueprint:` id when one fits (story truth still decides which beats exist — never force a beat to fit a shape). Feature 2–4 real diff hunks (from `capture/diff.patch`), each a small legible snippet; name the `code-*` block each wants in the frame's `scene`. Frames carry no `asset_candidates` except an optional `credits` close (2–6 `assets/<login>.png` avatars). Use the exact required fields from the storyboard and script references.
Use `story-design.md` for the PR archetype (changelog / feature-reveal / fix-explainer / refactor-walkthrough), the PR-native frame types, hook, persuasion, beats, the per-frame word budget, and the credits close. The sequence comes from **narrative design, not the diff's file order** — explain the change, don't read the diff aloud. As a **soft guide**, consult the role→blueprint menu in `../hyperframes-animation/blueprints-index.md`: for each beat, write the voiceover in the shape its candidate blueprint implies and tag that candidate `blueprint:` id when one fits (story truth still decides which beats exist — never force a beat to fit a shape). Feature 2–4 real diff hunks (from `capture/diff.patch`), each a small legible snippet; name the `code-*` block each wants in the frame's `scene`. Frames carry no `asset_candidates` except the `credits` close (1–6 `assets/<login>.png` avatars). Use the exact required fields from the storyboard and script references.
After drafting, show a frame-by-frame summary. In that same message ask the user (a) to approve or request changes, and (b) whether they want a live preview of the storyboard scaffold (`npx hyperframes preview`) — open it only on a yes. Iterate until approved; carry the preview choice to Step 6.
After drafting, present the plan as a proposal per story-spine § 3: open by echoing **"This video tells [audience] that [message]"**, then the frame table — one row per frame: frame · beat (type, duration) · on screen · why (its `narrativeRole`, traced to the message). In that same message ask the user (a) to approve or request changes, and (b) whether they want a live preview of the storyboard scaffold (`npx hyperframes preview`) — open it only on a yes. Iterate until approved; carry the preview choice to Step 6. This is a **checkpoint gate** (brief contract § 1): in autonomous mode, post the same summary as a heads-up and proceed — the preview question is asked once, at Step 6.
**Gate:** `STORYBOARD.md` exists, every frame has the required narrative fields, `SCRIPT.md` exists when narration is needed, and the user approved the plan.
**Gate:** `STORYBOARD.md` exists, every frame has the required narrative fields, `SCRIPT.md` exists when narration is needed, and the user approved the plan (autonomous: the summary was posted as a heads-up).
---
@@ -201,42 +217,44 @@ If a command fails, surface stderr and stop — don't pile on recovery commands.
**Known false-positive — do not chase it.** `inspect` may report a handful of `text_box_overflow` errors of ~1–4px on the **caption** highlight words (selector `#caption-word-*` / `.caption-line`). The caption pill uses a deliberately snug `line-height` (set once in `scripts/captions.mjs`) and has **no `overflow:hidden`**, so a heavy display glyph's ink spills a few px into the pill's own padding — nothing is actually clipped. Treat these as expected and proceed. Do **not** inflate the caption `line-height` (it balloons the pill, which is worse). Only act on a `text_box_overflow` when it names a **frame** element (`#el-NN-*`), not a caption word.
After checks pass, pause for user review. The video is assembled, viewable, and editable in Studio. Manage preview only once across Step 3 and Step 6: open it if the user asked earlier, offer it if they declined earlier, do not ask again if they are already reviewing in Studio.
After checks pass, pause for user review. The video is assembled, viewable, and editable in Studio. Manage preview only once across Step 3 and Step 6: open it if the user asked earlier, offer it if they declined earlier, do not ask again if they are already reviewing in Studio. In autonomous mode this is the one question the mode keeps: ask "preview first, or render?" — open the preview on yes, render on no — then deliver the MP4 with the contact sheet and the frame ids so revisions can target a single frame.
Preview: `npx hyperframes preview`
Render only after user approval:
Render only after user approval (autonomous mode: after the preview-or-render question):
`npx hyperframes render --skill=pr-to-video --quality high --output renders/video.mp4`
Do not rerun `lint`, `validate`, `inspect`, or `snapshot` after rendering unless the user asks.
**Gate:** `lint`, `validate`, and `inspect` passed before render; user approved at the review pause; `renders/video.mp4` exists. Final reply states the MP4 path and final duration.
**Gate:** `lint`, `validate`, and `inspect` passed before render; user approved at the review pause (autonomous: checks passed and the delivery includes the contact sheet); `renders/video.mp4` exists. Final reply states the MP4 path and final duration.
---
## Quick Reference
**Formats:** landscape `1920x1080` by default; portrait `1080x1920`; square `1080x1080`. Set the format once in the storyboard frontmatter.
**Formats:** landscape `1920x1080`; portrait `1080x1920`; square `1080x1080` — derived from the destination (brief contract § 2). Set the format once in the storyboard frontmatter.
**PR deltas vs a captured-asset workflow:** no Step 1 capture (the `gh` CLI ingests the PR into a synthetic `capture/extracted/` package — `tokens.json` + `visible-text.txt` + `people.json`); the only real assets are the contributors' `assets/<login>.png` avatars (an optional credits close); no `asset-descriptions.md`, no asset-staging step. Code beats are rendered by the `code-*` registry blocks on claude's navy Code Surface; the style is always **claude**.
**PR deltas vs a captured-asset workflow:** no Step 1 capture (the `gh` CLI ingests the PR into a synthetic `capture/extracted/` package — `tokens.json` + `visible-text.txt` + `people.json`); the only real assets are the contributors' `assets/<login>.png` avatars (the credits close); no `asset-descriptions.md`, no asset-staging step. Code beats are rendered by the `code-*` registry blocks on claude's navy Code Surface; the style is always **claude**.
**Background scripts:** the workflow ships these under `scripts/`: `fetch-pr` (PR → `capture/pr.json` + `diff.patch` via `gh`; large-PR-safe, no scratch), `ingest` (→ synthetic capture package; offline), and `fetch-people-avatars` (contributor avatars → `assets/`); plus the shared engine — `build-frame` (adopt + brand-remix a preset into `frame.md` + caption skin), `audio` (TTS, BGM, SFX, duration sync), `captions`, `transitions` (inject + verify), and `assemble-index`. Everything else is the `hyperframes` CLI. Code blocks install via `npx hyperframes add <name>`.
The reusable, domain-agnostic shot shapes live in `../hyperframes-animation/blueprints/` (indexed by `../hyperframes-animation/blueprints-index.md`); the `code-*` registry blocks are the code-beat vocabulary (`references/code-vocabulary.md`).
| Read | When |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| `[references/story-design.md](references/story-design.md)` | Step 3: plan the PR explanation. |
| `[../hyperframes-animation/blueprints-index.md](../hyperframes-animation/blueprints-index.md)` | Step 3: role→blueprint menu. Step 4: pick the shot shape. |
| `[../hyperframes-core/references/storyboard-format.md](../hyperframes-core/references/storyboard-format.md)` | Step 3: write `STORYBOARD.md`. |
| `[../hyperframes-core/references/script-format.md](../hyperframes-core/references/script-format.md)` | Step 3: write `SCRIPT.md`. |
| `[../media-use/audio/references/tts.md](../media-use/audio/references/tts.md)` | Step 3.1: choose or understand TTS providers. |
| `[references/visual-design.md](references/visual-design.md)` | Step 4: write the frame's shot sequence (+ Layout vocabulary). |
| `[references/code-vocabulary.md](references/code-vocabulary.md)` | Step 4 + 5: pick + fill the `code-*` block for a code beat. |
| `[references/motion-language.md](references/motion-language.md)` | Step 4: the motion vocabulary + the motion doctrine. |
| `[references/cut-catalog.md](references/cut-catalog.md)` | Step 4-5: the cut catalog (worker builds within-frame seams). |
| `[../hyperframes-animation/rules-index.md](../hyperframes-animation/rules-index.md)` + `[../hyperframes-animation/rules/](../hyperframes-animation/rules/)` | Step 5: local rule recipe bodies for the cited motions. |
| `[sub-agents/frame-worker.md](sub-agents/frame-worker.md)` | Step 5: dispatch per-frame workers. |
| `[../hyperframes-core/references/subagent-dispatch.md](../hyperframes-core/references/subagent-dispatch.md)` | Step 5: dispatch sub-agents safely. |
| `[../hyperframes-creative/frame-presets/claude/FRAME.md](../hyperframes-creative/frame-presets/claude/FRAME.md)` | Step 2: the claude preset (fixed style). |
| Read | When |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| `[../hyperframes-core/references/brief-contract.md](../hyperframes-core/references/brief-contract.md)` | Step 0: the interaction mode, brief fields, and how to ask. |
| `[../hyperframes-creative/references/story-spine.md](../hyperframes-creative/references/story-spine.md)` | Step 3: story doctrine — hook language, value-before-evidence, proposal shape. |
| `[references/story-design.md](references/story-design.md)` | Step 3: plan the PR explanation. |
| `[../hyperframes-animation/blueprints-index.md](../hyperframes-animation/blueprints-index.md)` | Step 3: role→blueprint menu. Step 4: pick the shot shape. |
| `[../hyperframes-core/references/storyboard-format.md](../hyperframes-core/references/storyboard-format.md)` | Step 3: write `STORYBOARD.md`. |
| `[../hyperframes-core/references/script-format.md](../hyperframes-core/references/script-format.md)` | Step 3: write `SCRIPT.md`. |
| `[../media-use/audio/references/tts.md](../media-use/audio/references/tts.md)` | Step 3.1: choose or understand TTS providers. |
| `[references/visual-design.md](references/visual-design.md)` | Step 4: write the frame's shot sequence (+ Layout vocabulary). |
| `[references/code-vocabulary.md](references/code-vocabulary.md)` | Step 4 + 5: pick + fill the `code-*` block for a code beat. |
| `[references/motion-language.md](references/motion-language.md)` | Step 4: the motion vocabulary + the motion doctrine. |
| `[references/cut-catalog.md](references/cut-catalog.md)` | Step 4-5: the cut catalog (worker builds within-frame seams). |
| `[../hyperframes-animation/rules-index.md](../hyperframes-animation/rules-index.md)` + `[../hyperframes-animation/rules/](../hyperframes-animation/rules/)` | Step 5: local rule recipe bodies for the cited motions. |
| `[sub-agents/frame-worker.md](sub-agents/frame-worker.md)` | Step 5: dispatch per-frame workers. |
| `[../hyperframes-core/references/subagent-dispatch.md](../hyperframes-core/references/subagent-dispatch.md)` | Step 5: dispatch sub-agents safely. |
| `[../hyperframes-creative/frame-presets/claude/FRAME.md](../hyperframes-creative/frame-presets/claude/FRAME.md)` | Step 2: the claude preset (fixed style). |