mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 16:42:27 +00:00
docs(skills): make captions non-optional in changelog-video (#2729)
* docs(skills): make captions non-optional in changelog-video The Jul 13-20 build shipped without captions because the pipeline had two soft failure modes: - Empty vo-words.json from the TTS provider was silently accepted, so align-captions.mjs had nothing to align. captions.json was never produced. - Step 5 tolerated the master-skeleton's LINES array staying empty. The build lint-checked and validated clean, but the caption rail rendered no text. This adds three hard gates on top of the existing scaffold: - Step 4 adds a whisper forced-alignment fallback so a missing TTS timestamp payload no longer breaks the caption pipeline. Whisper only supplies timings; captions still use the DISPLAY layer from script-tokens.json. - Step 5 flags an empty LINES array as a shipped bug and shows the exact IIFE-input shape it expects. - Step 6 adds gate 5: sample 3-4 frames across the VO window and confirm visible caption text on each. If any spoken window renders no caption, the build is red. Anti-patterns table gets two rows covering the empty-LINES and missing-word-timings failure modes. .claude and .agents mirrors stay byte-identical. Follow-up to #2669 (pre-build gate); paired with the v5 rebuild that retrofitted captions onto the shipped Jul 13-20 video. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(skills): bump changelog caption size to 32px for phone-screen legibility 25px reads too small at 1080² when the video is viewed on a phone without zoom. Bumped to 32px (font-size 25→32, top 1002→990, height 40→52, alpha .92→.94) so captions carry across small-screen playback. Change is CSS-only on the master-skeleton; existing per-scene chrome and layouts are untouched. Retrofitted the Jul 13-20 v6 build with the same values (identical CSS block) so the shipped video and the future default match. .claude and .agents mirrors stay byte-identical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(skills): sync caption position refs to top: 990 across SKILL.md + build-spec Miga caught: the master-skeleton bump to top: 990 / 32px / height: 52 was not reflected in the SKILL.md prose or build-spec.md. All three files now name the same values so the doc, gate, and code stay consistent. Also clarifies build-spec.md's caption-rail line to point at SKILL.md step 5 as the source of the LINES-populated requirement. .claude and .agents mirrors byte-identical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(skills): sync script-voice.md caption rail to top: 990 / 32px Third and final stale reference to top: 1002 / 25px — script-voice.md was still stating the pre-bump values. Now matches master-skeleton.html, build-spec.md, and SKILL.md. Also bumps the 'keep critical small text out' guidance from ~80px to ~100px to reflect the slightly taller caption rail. Full-directory sweep confirms zero remaining occurrences of 1002 or 25px across .claude/ and .agents/ skill mirrors. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
James Russo
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
2e97e5b1ef
commit
807078c7cd
@@ -39,7 +39,7 @@ ffmpeg -y -stream_loop 15 -i <SKILL_DIR>/assets/bg-pattern.mp4 -t <TOTAL> \
|
||||
cp <SKILL_DIR>/examples/master-skeleton.html project/index.html
|
||||
```
|
||||
|
||||
Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, caption rail at `top: 1002`) that every scene inherits from the scaffold.
|
||||
Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, 32px caption rail at `top: 990`) that every scene inherits from the scaffold.
|
||||
|
||||
Only THEN begin steps 1-6 below. Steps 1-4 (parse, route, script, VO) plan what goes into the scaffold; step 5 fills placeholders (`<RANGE>`, `<TOTAL>`, `<CUT_N>`, `<DUR_N>`, scene bodies) inside the already-copied `project/index.html` — you do NOT rewrite the scaffold's chrome, fonts, palette, or layout shell.
|
||||
|
||||
@@ -99,6 +99,26 @@ The aligner prints `MISMATCH` warnings — resolve every one before building
|
||||
is the clock**: all beat times come from `vo-words.json`; a VO regen re-opens
|
||||
every seam.
|
||||
|
||||
**Word-timings are a hard gate.** Before moving on to step 5, verify
|
||||
`vo-words.json` is non-empty and has a `words: [...]` array with `start`/`end`
|
||||
per word. If it's empty (0 bytes) or missing the array — a known failure mode
|
||||
when the TTS provider returns audio but no timestamp payload — DO NOT proceed
|
||||
without them. Fallback: forced-align the produced audio against the display
|
||||
script using local whisper:
|
||||
|
||||
```bash
|
||||
uvx --from openai-whisper whisper voiceover.mp3 \
|
||||
--model base.en --language en --word_timestamps True \
|
||||
--output_format json --output_dir .
|
||||
# then run align-captions.mjs with --words voiceover.json (same shape)
|
||||
```
|
||||
|
||||
Whisper mishears TTS renderings ("gee-sap" → "gsap", "heyjen" → "hey Jen",
|
||||
etc.) — captions still use the DISPLAY spelling from `script-tokens.json`;
|
||||
whisper only supplies the timestamps. `align-captions.mjs` handles the join.
|
||||
This fallback is the difference between a captioned build and a silently
|
||||
uncaptioned one.
|
||||
|
||||
### 5 · Build
|
||||
|
||||
Follow `references/build-spec.md` exactly: brand tokens + fonts (bundled in
|
||||
@@ -107,6 +127,22 @@ chrome, caption rail, one rationed green moment per scene. Then the doctrine
|
||||
order: `ledger.json` (all ordinary seams cut-the-curve LEFT) → seam-stamp →
|
||||
internal beats on VO words → seam-gate verify.
|
||||
|
||||
**Captions are non-optional.** The master-skeleton ships a caption-rail IIFE
|
||||
that reads a `LINES` array — leaving that array empty is a shipped bug, not a
|
||||
style choice. Populate it from `captions.json` before proceeding to step 6:
|
||||
|
||||
```javascript
|
||||
// paste in place of "const LINES = /* … */ []" in the caption-rail IIFE:
|
||||
const LINES = /* contents of captions.json */ [
|
||||
{ id: 0, end: 2.74, w: [["This", 0.0], ["week,", 0.30], …] },
|
||||
…
|
||||
];
|
||||
```
|
||||
|
||||
If `align-captions.mjs` was skipped or `LINES` is `[]`, the frame check in
|
||||
step 6 will fail — do not paper over it by removing `#cap-line` from the
|
||||
scaffold.
|
||||
|
||||
### 6 · Gates (all green before presenting)
|
||||
|
||||
1. `bun run --cwd packages/cli hyperframes check` (or the installed
|
||||
@@ -120,6 +156,12 @@ internal beats on VO words → seam-gate verify.
|
||||
4. Do NOT render unless the user asks. After a requested render, verify
|
||||
frames from the MP4 (`ffmpeg -ss <t> … -frames:v 1`): captions present,
|
||||
background video not black, no tiny/frozen frames.
|
||||
5. **Caption presence gate — hard fail.** Sample 3-4 frames spread across
|
||||
the VO's spoken window (e.g. `t=3`, `t=15`, `t=30`, `t=42` for a 48s VO)
|
||||
and confirm the caption rail at `top: 990` renders visible text on each.
|
||||
If any frame in a spoken interval is missing captions, the build ships
|
||||
uncaptioned — treat it as a red gate and re-check step 5's `LINES`
|
||||
population. This is exactly what went wrong on the Jul 13-20 v4 build.
|
||||
|
||||
## Project layout
|
||||
|
||||
@@ -136,15 +178,17 @@ projects/active/weekly-changelog-<range>/
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
| Don't | Instead |
|
||||
| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Bullet-point slides for UI changes | Mock the surface acting out the change |
|
||||
| Fake UI for un-representable items | Honest checklist scene |
|
||||
| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard |
|
||||
| Phonetic spellings in captions | Captions always render the display layer |
|
||||
| Guessing an unknown term's pronunciation | Ask, then grow the lexicon |
|
||||
| Speaking every changelog item | ≤3 per theme; the digest link carries the rest |
|
||||
| Green accents everywhere | One green moment per scene (#5ef17c) |
|
||||
| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always |
|
||||
| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand |
|
||||
| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise |
|
||||
| Don't | Instead |
|
||||
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Bullet-point slides for UI changes | Mock the surface acting out the change |
|
||||
| Fake UI for un-representable items | Honest checklist scene |
|
||||
| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard |
|
||||
| Phonetic spellings in captions | Captions always render the display layer |
|
||||
| Guessing an unknown term's pronunciation | Ask, then grow the lexicon |
|
||||
| Speaking every changelog item | ≤3 per theme; the digest link carries the rest |
|
||||
| Green accents everywhere | One green moment per scene (#5ef17c) |
|
||||
| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always |
|
||||
| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand |
|
||||
| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise |
|
||||
| Shipping with the `LINES` array empty in the scaffold | Step 4 must produce a populated `captions.json`; step 5 must paste it into the IIFE; step 6 gate 5 must confirm captions on rendered frames. An empty `LINES` = uncaptioned ship = re-do the run |
|
||||
| No `vo-words.json` → skip captions and ship anyway | Fall back to whisper forced alignment on the produced audio; captions are non-optional |
|
||||
|
||||
@@ -33,9 +33,9 @@
|
||||
.glass { background: rgba(10,12,11,.78); border: 1px solid rgba(190,255,205,.32);
|
||||
border-radius: 22px; box-shadow: 0 24px 60px rgba(0,0,0,.5); }
|
||||
/* caption rail — overlay on top of the film, never a reserved band */
|
||||
#cap-line { position: absolute; left: 0; right: 0; top: 1002px; height: 40px; text-align: center;
|
||||
z-index: 7; font-family: 'TT Norms Pro', sans-serif; font-weight: 500; font-size: 25px;
|
||||
letter-spacing: .01em; color: rgba(245,246,244,.92);
|
||||
#cap-line { position: absolute; left: 0; right: 0; top: 990px; height: 52px; text-align: center;
|
||||
z-index: 7; font-family: 'TT Norms Pro', sans-serif; font-weight: 500; font-size: 32px;
|
||||
letter-spacing: .01em; color: rgba(245,246,244,.94);
|
||||
text-shadow: 0 2px 14px rgba(0,0,0,.85), 0 0 3px rgba(0,0,0,.6);
|
||||
white-space: nowrap; pointer-events: none; }
|
||||
.cap-phrase { position: absolute; left: 0; right: 0; }
|
||||
|
||||
@@ -76,7 +76,7 @@ otherwise) and must stay flat 2D (no 3D ancestors).
|
||||
y ∈ [288, 944].
|
||||
- Outro (≤3.5s): kicker FULL DIGEST, "See what shipped." ~96px, green rule,
|
||||
mono URL chip, tag line. Fade all + chrome ~0.5s before end.
|
||||
- Caption rail per `script-voice.md` (top: 1002).
|
||||
- Caption rail per `script-voice.md` (top: 990, font-size: 32px, height: 52). Mandatory — populate the master-skeleton's `LINES` array from `captions.json` before render; see SKILL.md step 5.
|
||||
|
||||
## Seams + internal life (doctrine mechanics)
|
||||
|
||||
|
||||
@@ -105,7 +105,7 @@ transcript) before the captions are trusted.
|
||||
## Caption rail (rendering)
|
||||
|
||||
Per `captions-overlay`: a quiet OVERLAY, never a reserved band. One line,
|
||||
bottom-center (top: 1002px on 1080-square), TT Norms Pro 500 25px,
|
||||
ink .92, soft dark text-shadow, words fading in (0.12s) on their timestamps,
|
||||
phrase swaps as sets. Keep critical small text out of the bottom ~80px
|
||||
bottom-center (top: 990px, height: 52px on 1080-square), TT Norms Pro 500 32px,
|
||||
ink .94, soft dark text-shadow, words fading in (0.12s) on their timestamps,
|
||||
phrase swaps as sets. Keep critical small text out of the bottom ~100px
|
||||
center span; everything else may run under the rail.
|
||||
|
||||
Reference in New Issue
Block a user