ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-05 00:56:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(skills): make captions non-optional in changelog-video (#2729)

Browse Source 
* docs(skills): make captions non-optional in changelog-video

The Jul 13-20 build shipped without captions because the pipeline had two soft
failure modes:

- Empty vo-words.json from the TTS provider was silently accepted, so
  align-captions.mjs had nothing to align. captions.json was never produced.
- Step 5 tolerated the master-skeleton's LINES array staying empty. The build
  lint-checked and validated clean, but the caption rail rendered no text.

This adds three hard gates on top of the existing scaffold:

- Step 4 adds a whisper forced-alignment fallback so a missing TTS timestamp
  payload no longer breaks the caption pipeline. Whisper only supplies timings;
  captions still use the DISPLAY layer from script-tokens.json.
- Step 5 flags an empty LINES array as a shipped bug and shows the exact
  IIFE-input shape it expects.
- Step 6 adds gate 5: sample 3-4 frames across the VO window and confirm visible
  caption text on each. If any spoken window renders no caption, the build is
  red.

Anti-patterns table gets two rows covering the empty-LINES and
missing-word-timings failure modes. .claude and .agents mirrors stay
byte-identical.

Follow-up to #2669 (pre-build gate); paired with the v5 rebuild that
retrofitted captions onto the shipped Jul 13-20 video.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(skills): bump changelog caption size to 32px for phone-screen legibility

25px reads too small at 1080² when the video is viewed on a phone without
zoom. Bumped to 32px (font-size 25→32, top 1002→990, height 40→52, alpha
.92→.94) so captions carry across small-screen playback. Change is CSS-only
on the master-skeleton; existing per-scene chrome and layouts are untouched.

Retrofitted the Jul 13-20 v6 build with the same values (identical CSS
block) so the shipped video and the future default match. .claude and
.agents mirrors stay byte-identical.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(skills): sync caption position refs to top: 990 across SKILL.md + build-spec

Miga caught: the master-skeleton bump to top: 990 / 32px / height: 52 was
not reflected in the SKILL.md prose or build-spec.md. All three files now
name the same values so the doc, gate, and code stay consistent. Also
clarifies build-spec.md's caption-rail line to point at SKILL.md step 5 as
the source of the LINES-populated requirement.

.claude and .agents mirrors byte-identical.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(skills): sync script-voice.md caption rail to top: 990 / 32px

Third and final stale reference to top: 1002 / 25px — script-voice.md was
still stating the pre-bump values. Now matches master-skeleton.html,
build-spec.md, and SKILL.md. Also bumps the 'keep critical small text out'
guidance from ~80px to ~100px to reflect the slightly taller caption rail.

Full-directory sweep confirms zero remaining occurrences of 1002 or 25px
across .claude/ and .agents/ skill mirrors.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
James Russo
2026-07-21 20:38:46 -04:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 2e97e5b1ef
commit 807078c7cd
8 changed files with 128 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/changelog-video/SKILL.md
+57 -13
View File
@@ -39,7 +39,7 @@ ffmpeg -y -stream_loop 15 -i <SKILL_DIR>/assets/bg-pattern.mp4 -t <TOTAL> \
cp <SKILL_DIR>/examples/master-skeleton.html project/index.html
```
Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, caption rail at `top: 1002`) that every scene inherits from the scaffold.
Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, 32px caption rail at `top: 990`) that every scene inherits from the scaffold.
Only THEN begin steps 1-6 below. Steps 1-4 (parse, route, script, VO) plan what goes into the scaffold; step 5 fills placeholders (`<RANGE>`, `<TOTAL>`, `<CUT_N>`, `<DUR_N>`, scene bodies) inside the already-copied `project/index.html` — you do NOT rewrite the scaffold's chrome, fonts, palette, or layout shell.
@@ -99,6 +99,26 @@ The aligner prints `MISMATCH` warnings — resolve every one before building
is the clock**: all beat times come from `vo-words.json`; a VO regen re-opens
every seam.
**Word-timings are a hard gate.** Before moving on to step 5, verify
`vo-words.json` is non-empty and has a `words: [...]` array with `start`/`end`
per word. If it's empty (0 bytes) or missing the array — a known failure mode
when the TTS provider returns audio but no timestamp payload — DO NOT proceed
without them. Fallback: forced-align the produced audio against the display
script using local whisper:
```bash
uvx --from openai-whisper whisper voiceover.mp3 \
--model base.en --language en --word_timestamps True \
--output_format json --output_dir .
# then run align-captions.mjs with --words voiceover.json (same shape)
```
Whisper mishears TTS renderings ("gee-sap" → "gsap", "heyjen" → "hey Jen",
etc.) — captions still use the DISPLAY spelling from `script-tokens.json`;
whisper only supplies the timestamps. `align-captions.mjs` handles the join.
This fallback is the difference between a captioned build and a silently
uncaptioned one.
### 5 · Build
Follow `references/build-spec.md` exactly: brand tokens + fonts (bundled in
@@ -107,6 +127,22 @@ chrome, caption rail, one rationed green moment per scene. Then the doctrine
order: `ledger.json` (all ordinary seams cut-the-curve LEFT) → seam-stamp →
internal beats on VO words → seam-gate verify.
**Captions are non-optional.** The master-skeleton ships a caption-rail IIFE
that reads a `LINES` array — leaving that array empty is a shipped bug, not a
style choice. Populate it from `captions.json` before proceeding to step 6:
```javascript
// paste in place of "const LINES = /* … */ []" in the caption-rail IIFE:
const LINES = /* contents of captions.json */ [
{ id: 0, end: 2.74, w: [["This", 0.0], ["week,", 0.30], …] },
…
];
```
If `align-captions.mjs` was skipped or `LINES` is `[]`, the frame check in
step 6 will fail — do not paper over it by removing `#cap-line` from the
scaffold.
### 6 · Gates (all green before presenting)
1. `bun run --cwd packages/cli hyperframes check` (or the installed
@@ -120,6 +156,12 @@ internal beats on VO words → seam-gate verify.
4. Do NOT render unless the user asks. After a requested render, verify
frames from the MP4 (`ffmpeg -ss <t> … -frames:v 1`): captions present,
background video not black, no tiny/frozen frames.
5. **Caption presence gate — hard fail.** Sample 3-4 frames spread across
the VO's spoken window (e.g. `t=3`, `t=15`, `t=30`, `t=42` for a 48s VO)
and confirm the caption rail at `top: 990` renders visible text on each.
If any frame in a spoken interval is missing captions, the build ships
uncaptioned — treat it as a red gate and re-check step 5's `LINES`
population. This is exactly what went wrong on the Jul 13-20 v4 build.
## Project layout
@@ -136,15 +178,17 @@ projects/active/weekly-changelog-<range>/
## Anti-patterns
| Don't | Instead |
| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| Bullet-point slides for UI changes | Mock the surface acting out the change |
| Fake UI for un-representable items | Honest checklist scene |
| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard |
| Phonetic spellings in captions | Captions always render the display layer |
| Guessing an unknown term's pronunciation | Ask, then grow the lexicon |
| Speaking every changelog item | ≤3 per theme; the digest link carries the rest |
| Green accents everywhere | One green moment per scene (#5ef17c) |
| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always |
| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand |
| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise |
| Don't | Instead |
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Bullet-point slides for UI changes | Mock the surface acting out the change |
| Fake UI for un-representable items | Honest checklist scene |
| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard |
| Phonetic spellings in captions | Captions always render the display layer |
| Guessing an unknown term's pronunciation | Ask, then grow the lexicon |
| Speaking every changelog item | ≤3 per theme; the digest link carries the rest |
| Green accents everywhere | One green moment per scene (#5ef17c) |
| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always |
| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand |
| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise |
| Shipping with the `LINES` array empty in the scaffold | Step 4 must produce a populated `captions.json`; step 5 must paste it into the IIFE; step 6 gate 5 must confirm captions on rendered frames. An empty `LINES` = uncaptioned ship = re-do the run |
| No `vo-words.json` → skip captions and ship anyway | Fall back to whisper forced alignment on the produced audio; captions are non-optional |