From 807078c7cde9d5c8403588722d1cd9397c513a0d Mon Sep 17 00:00:00 2001 From: James Russo Date: Tue, 21 Jul 2026 20:38:46 -0400 Subject: [PATCH] docs(skills): make captions non-optional in changelog-video (#2729) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs(skills): make captions non-optional in changelog-video The Jul 13-20 build shipped without captions because the pipeline had two soft failure modes: - Empty vo-words.json from the TTS provider was silently accepted, so align-captions.mjs had nothing to align. captions.json was never produced. - Step 5 tolerated the master-skeleton's LINES array staying empty. The build lint-checked and validated clean, but the caption rail rendered no text. This adds three hard gates on top of the existing scaffold: - Step 4 adds a whisper forced-alignment fallback so a missing TTS timestamp payload no longer breaks the caption pipeline. Whisper only supplies timings; captions still use the DISPLAY layer from script-tokens.json. - Step 5 flags an empty LINES array as a shipped bug and shows the exact IIFE-input shape it expects. - Step 6 adds gate 5: sample 3-4 frames across the VO window and confirm visible caption text on each. If any spoken window renders no caption, the build is red. Anti-patterns table gets two rows covering the empty-LINES and missing-word-timings failure modes. .claude and .agents mirrors stay byte-identical. Follow-up to #2669 (pre-build gate); paired with the v5 rebuild that retrofitted captions onto the shipped Jul 13-20 video. Co-Authored-By: Claude Opus 4.7 * docs(skills): bump changelog caption size to 32px for phone-screen legibility 25px reads too small at 1080² when the video is viewed on a phone without zoom. Bumped to 32px (font-size 25→32, top 1002→990, height 40→52, alpha .92→.94) so captions carry across small-screen playback. Change is CSS-only on the master-skeleton; existing per-scene chrome and layouts are untouched. Retrofitted the Jul 13-20 v6 build with the same values (identical CSS block) so the shipped video and the future default match. .claude and .agents mirrors stay byte-identical. Co-Authored-By: Claude Opus 4.7 * docs(skills): sync caption position refs to top: 990 across SKILL.md + build-spec Miga caught: the master-skeleton bump to top: 990 / 32px / height: 52 was not reflected in the SKILL.md prose or build-spec.md. All three files now name the same values so the doc, gate, and code stay consistent. Also clarifies build-spec.md's caption-rail line to point at SKILL.md step 5 as the source of the LINES-populated requirement. .claude and .agents mirrors byte-identical. Co-Authored-By: Claude Opus 4.7 * docs(skills): sync script-voice.md caption rail to top: 990 / 32px Third and final stale reference to top: 1002 / 25px — script-voice.md was still stating the pre-bump values. Now matches master-skeleton.html, build-spec.md, and SKILL.md. Also bumps the 'keep critical small text out' guidance from ~80px to ~100px to reflect the slightly taller caption rail. Full-directory sweep confirms zero remaining occurrences of 1002 or 25px across .claude/ and .agents/ skill mirrors. Co-Authored-By: Claude Opus 4.7 --------- Co-authored-by: Claude Opus 4.7 --- .agents/skills/changelog-video/SKILL.md | 70 +++++++++++++++---- .../examples/master-skeleton.html | 6 +- .../changelog-video/references/build-spec.md | 2 +- .../references/script-voice.md | 6 +- .claude/skills/changelog-video/SKILL.md | 70 +++++++++++++++---- .../examples/master-skeleton.html | 6 +- .../changelog-video/references/build-spec.md | 2 +- .../references/script-voice.md | 6 +- 8 files changed, 128 insertions(+), 40 deletions(-) diff --git a/.agents/skills/changelog-video/SKILL.md b/.agents/skills/changelog-video/SKILL.md index 46e6ff2e0..0bfd4f291 100644 --- a/.agents/skills/changelog-video/SKILL.md +++ b/.agents/skills/changelog-video/SKILL.md @@ -39,7 +39,7 @@ ffmpeg -y -stream_loop 15 -i /assets/bg-pattern.mp4 -t \ cp /examples/master-skeleton.html project/index.html ``` -Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, caption rail at `top: 1002`) that every scene inherits from the scaffold. +Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, 32px caption rail at `top: 990`) that every scene inherits from the scaffold. Only THEN begin steps 1-6 below. Steps 1-4 (parse, route, script, VO) plan what goes into the scaffold; step 5 fills placeholders (``, ``, ``, ``, scene bodies) inside the already-copied `project/index.html` — you do NOT rewrite the scaffold's chrome, fonts, palette, or layout shell. @@ -99,6 +99,26 @@ The aligner prints `MISMATCH` warnings — resolve every one before building is the clock**: all beat times come from `vo-words.json`; a VO regen re-opens every seam. +**Word-timings are a hard gate.** Before moving on to step 5, verify +`vo-words.json` is non-empty and has a `words: [...]` array with `start`/`end` +per word. If it's empty (0 bytes) or missing the array — a known failure mode +when the TTS provider returns audio but no timestamp payload — DO NOT proceed +without them. Fallback: forced-align the produced audio against the display +script using local whisper: + +```bash +uvx --from openai-whisper whisper voiceover.mp3 \ + --model base.en --language en --word_timestamps True \ + --output_format json --output_dir . +# then run align-captions.mjs with --words voiceover.json (same shape) +``` + +Whisper mishears TTS renderings ("gee-sap" → "gsap", "heyjen" → "hey Jen", +etc.) — captions still use the DISPLAY spelling from `script-tokens.json`; +whisper only supplies the timestamps. `align-captions.mjs` handles the join. +This fallback is the difference between a captioned build and a silently +uncaptioned one. + ### 5 · Build Follow `references/build-spec.md` exactly: brand tokens + fonts (bundled in @@ -107,6 +127,22 @@ chrome, caption rail, one rationed green moment per scene. Then the doctrine order: `ledger.json` (all ordinary seams cut-the-curve LEFT) → seam-stamp → internal beats on VO words → seam-gate verify. +**Captions are non-optional.** The master-skeleton ships a caption-rail IIFE +that reads a `LINES` array — leaving that array empty is a shipped bug, not a +style choice. Populate it from `captions.json` before proceeding to step 6: + +```javascript +// paste in place of "const LINES = /* … */ []" in the caption-rail IIFE: +const LINES = /* contents of captions.json */ [ + { id: 0, end: 2.74, w: [["This", 0.0], ["week,", 0.30], …] }, + … +]; +``` + +If `align-captions.mjs` was skipped or `LINES` is `[]`, the frame check in +step 6 will fail — do not paper over it by removing `#cap-line` from the +scaffold. + ### 6 · Gates (all green before presenting) 1. `bun run --cwd packages/cli hyperframes check` (or the installed @@ -120,6 +156,12 @@ internal beats on VO words → seam-gate verify. 4. Do NOT render unless the user asks. After a requested render, verify frames from the MP4 (`ffmpeg -ss … -frames:v 1`): captions present, background video not black, no tiny/frozen frames. +5. **Caption presence gate — hard fail.** Sample 3-4 frames spread across + the VO's spoken window (e.g. `t=3`, `t=15`, `t=30`, `t=42` for a 48s VO) + and confirm the caption rail at `top: 990` renders visible text on each. + If any frame in a spoken interval is missing captions, the build ships + uncaptioned — treat it as a red gate and re-check step 5's `LINES` + population. This is exactly what went wrong on the Jul 13-20 v4 build. ## Project layout @@ -136,15 +178,17 @@ projects/active/weekly-changelog-/ ## Anti-patterns -| Don't | Instead | -| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -| Bullet-point slides for UI changes | Mock the surface acting out the change | -| Fake UI for un-representable items | Honest checklist scene | -| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard | -| Phonetic spellings in captions | Captions always render the display layer | -| Guessing an unknown term's pronunciation | Ask, then grow the lexicon | -| Speaking every changelog item | ≤3 per theme; the digest link carries the rest | -| Green accents everywhere | One green moment per scene (#5ef17c) | -| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always | -| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand | -| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise | +| Don't | Instead | +| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Bullet-point slides for UI changes | Mock the surface acting out the change | +| Fake UI for un-representable items | Honest checklist scene | +| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard | +| Phonetic spellings in captions | Captions always render the display layer | +| Guessing an unknown term's pronunciation | Ask, then grow the lexicon | +| Speaking every changelog item | ≤3 per theme; the digest link carries the rest | +| Green accents everywhere | One green moment per scene (#5ef17c) | +| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always | +| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand | +| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise | +| Shipping with the `LINES` array empty in the scaffold | Step 4 must produce a populated `captions.json`; step 5 must paste it into the IIFE; step 6 gate 5 must confirm captions on rendered frames. An empty `LINES` = uncaptioned ship = re-do the run | +| No `vo-words.json` → skip captions and ship anyway | Fall back to whisper forced alignment on the produced audio; captions are non-optional | diff --git a/.agents/skills/changelog-video/examples/master-skeleton.html b/.agents/skills/changelog-video/examples/master-skeleton.html index dc141233e..35a6f81c2 100644 --- a/.agents/skills/changelog-video/examples/master-skeleton.html +++ b/.agents/skills/changelog-video/examples/master-skeleton.html @@ -33,9 +33,9 @@ .glass { background: rgba(10,12,11,.78); border: 1px solid rgba(190,255,205,.32); border-radius: 22px; box-shadow: 0 24px 60px rgba(0,0,0,.5); } /* caption rail — overlay on top of the film, never a reserved band */ - #cap-line { position: absolute; left: 0; right: 0; top: 1002px; height: 40px; text-align: center; - z-index: 7; font-family: 'TT Norms Pro', sans-serif; font-weight: 500; font-size: 25px; - letter-spacing: .01em; color: rgba(245,246,244,.92); + #cap-line { position: absolute; left: 0; right: 0; top: 990px; height: 52px; text-align: center; + z-index: 7; font-family: 'TT Norms Pro', sans-serif; font-weight: 500; font-size: 32px; + letter-spacing: .01em; color: rgba(245,246,244,.94); text-shadow: 0 2px 14px rgba(0,0,0,.85), 0 0 3px rgba(0,0,0,.6); white-space: nowrap; pointer-events: none; } .cap-phrase { position: absolute; left: 0; right: 0; } diff --git a/.agents/skills/changelog-video/references/build-spec.md b/.agents/skills/changelog-video/references/build-spec.md index beea84313..d376e7bbd 100644 --- a/.agents/skills/changelog-video/references/build-spec.md +++ b/.agents/skills/changelog-video/references/build-spec.md @@ -76,7 +76,7 @@ otherwise) and must stay flat 2D (no 3D ancestors). y ∈ [288, 944]. - Outro (≤3.5s): kicker FULL DIGEST, "See what shipped." ~96px, green rule, mono URL chip, tag line. Fade all + chrome ~0.5s before end. -- Caption rail per `script-voice.md` (top: 1002). +- Caption rail per `script-voice.md` (top: 990, font-size: 32px, height: 52). Mandatory — populate the master-skeleton's `LINES` array from `captions.json` before render; see SKILL.md step 5. ## Seams + internal life (doctrine mechanics) diff --git a/.agents/skills/changelog-video/references/script-voice.md b/.agents/skills/changelog-video/references/script-voice.md index ba2bdee1f..b39a550f0 100644 --- a/.agents/skills/changelog-video/references/script-voice.md +++ b/.agents/skills/changelog-video/references/script-voice.md @@ -105,7 +105,7 @@ transcript) before the captions are trusted. ## Caption rail (rendering) Per `captions-overlay`: a quiet OVERLAY, never a reserved band. One line, -bottom-center (top: 1002px on 1080-square), TT Norms Pro 500 25px, -ink .92, soft dark text-shadow, words fading in (0.12s) on their timestamps, -phrase swaps as sets. Keep critical small text out of the bottom ~80px +bottom-center (top: 990px, height: 52px on 1080-square), TT Norms Pro 500 32px, +ink .94, soft dark text-shadow, words fading in (0.12s) on their timestamps, +phrase swaps as sets. Keep critical small text out of the bottom ~100px center span; everything else may run under the rail. diff --git a/.claude/skills/changelog-video/SKILL.md b/.claude/skills/changelog-video/SKILL.md index 46e6ff2e0..0bfd4f291 100644 --- a/.claude/skills/changelog-video/SKILL.md +++ b/.claude/skills/changelog-video/SKILL.md @@ -39,7 +39,7 @@ ffmpeg -y -stream_loop 15 -i /assets/bg-pattern.mp4 -t \ cp /examples/master-skeleton.html project/index.html ``` -Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, caption rail at `top: 1002`) that every scene inherits from the scaffold. +Then **read `references/build-spec.md` end-to-end** (not skimmed) — it defines the brand tokens (TT Norms Pro + ABC Solar Display + TT Norms Mono, cream `#f5f6f4`, rationed green `#5ef17c`, glass cards with green-tinted borders, kicker/sec-chip pill shape, 32px caption rail at `top: 990`) that every scene inherits from the scaffold. Only THEN begin steps 1-6 below. Steps 1-4 (parse, route, script, VO) plan what goes into the scaffold; step 5 fills placeholders (``, ``, ``, ``, scene bodies) inside the already-copied `project/index.html` — you do NOT rewrite the scaffold's chrome, fonts, palette, or layout shell. @@ -99,6 +99,26 @@ The aligner prints `MISMATCH` warnings — resolve every one before building is the clock**: all beat times come from `vo-words.json`; a VO regen re-opens every seam. +**Word-timings are a hard gate.** Before moving on to step 5, verify +`vo-words.json` is non-empty and has a `words: [...]` array with `start`/`end` +per word. If it's empty (0 bytes) or missing the array — a known failure mode +when the TTS provider returns audio but no timestamp payload — DO NOT proceed +without them. Fallback: forced-align the produced audio against the display +script using local whisper: + +```bash +uvx --from openai-whisper whisper voiceover.mp3 \ + --model base.en --language en --word_timestamps True \ + --output_format json --output_dir . +# then run align-captions.mjs with --words voiceover.json (same shape) +``` + +Whisper mishears TTS renderings ("gee-sap" → "gsap", "heyjen" → "hey Jen", +etc.) — captions still use the DISPLAY spelling from `script-tokens.json`; +whisper only supplies the timestamps. `align-captions.mjs` handles the join. +This fallback is the difference between a captioned build and a silently +uncaptioned one. + ### 5 · Build Follow `references/build-spec.md` exactly: brand tokens + fonts (bundled in @@ -107,6 +127,22 @@ chrome, caption rail, one rationed green moment per scene. Then the doctrine order: `ledger.json` (all ordinary seams cut-the-curve LEFT) → seam-stamp → internal beats on VO words → seam-gate verify. +**Captions are non-optional.** The master-skeleton ships a caption-rail IIFE +that reads a `LINES` array — leaving that array empty is a shipped bug, not a +style choice. Populate it from `captions.json` before proceeding to step 6: + +```javascript +// paste in place of "const LINES = /* … */ []" in the caption-rail IIFE: +const LINES = /* contents of captions.json */ [ + { id: 0, end: 2.74, w: [["This", 0.0], ["week,", 0.30], …] }, + … +]; +``` + +If `align-captions.mjs` was skipped or `LINES` is `[]`, the frame check in +step 6 will fail — do not paper over it by removing `#cap-line` from the +scaffold. + ### 6 · Gates (all green before presenting) 1. `bun run --cwd packages/cli hyperframes check` (or the installed @@ -120,6 +156,12 @@ internal beats on VO words → seam-gate verify. 4. Do NOT render unless the user asks. After a requested render, verify frames from the MP4 (`ffmpeg -ss … -frames:v 1`): captions present, background video not black, no tiny/frozen frames. +5. **Caption presence gate — hard fail.** Sample 3-4 frames spread across + the VO's spoken window (e.g. `t=3`, `t=15`, `t=30`, `t=42` for a 48s VO) + and confirm the caption rail at `top: 990` renders visible text on each. + If any frame in a spoken interval is missing captions, the build ships + uncaptioned — treat it as a red gate and re-check step 5's `LINES` + population. This is exactly what went wrong on the Jul 13-20 v4 build. ## Project layout @@ -136,15 +178,17 @@ projects/active/weekly-changelog-/ ## Anti-patterns -| Don't | Instead | -| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -| Bullet-point slides for UI changes | Mock the surface acting out the change | -| Fake UI for un-representable items | Honest checklist scene | -| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard | -| Phonetic spellings in captions | Captions always render the display layer | -| Guessing an unknown term's pronunciation | Ask, then grow the lexicon | -| Speaking every changelog item | ≤3 per theme; the digest link carries the rest | -| Green accents everywhere | One green moment per scene (#5ef17c) | -| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always | -| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand | -| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise | +| Don't | Instead | +| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Bullet-point slides for UI changes | Mock the surface acting out the change | +| Fake UI for un-representable items | Honest checklist scene | +| Plain "JSON"/"CLI" in the TTS text | Lexicon spoken forms; display stays standard | +| Phonetic spellings in captions | Captions always render the display layer | +| Guessing an unknown term's pronunciation | Ask, then grow the lexicon | +| Speaking every changelog item | ≤3 per theme; the digest link carries the rest | +| Green accents everywhere | One green moment per scene (#5ef17c) | +| Starting from a prior video's index.html | Step 0 — copy `examples/master-skeleton.html` from this skill into `project/index.html`, always | +| Hand-crafted `@font-face` / WebGL shader / custom BGM | Step 0 — copy this skill's `assets/` verbatim; the skill's assets ARE the brand | +| Delivered without CloudFront invalidation | Run `aws cloudfront create-invalidation` on distribution `E2BSLVSZ7FG3U0` for the exact path after any S3 replace — CDN caches the old file otherwise | +| Shipping with the `LINES` array empty in the scaffold | Step 4 must produce a populated `captions.json`; step 5 must paste it into the IIFE; step 6 gate 5 must confirm captions on rendered frames. An empty `LINES` = uncaptioned ship = re-do the run | +| No `vo-words.json` → skip captions and ship anyway | Fall back to whisper forced alignment on the produced audio; captions are non-optional | diff --git a/.claude/skills/changelog-video/examples/master-skeleton.html b/.claude/skills/changelog-video/examples/master-skeleton.html index dc141233e..35a6f81c2 100644 --- a/.claude/skills/changelog-video/examples/master-skeleton.html +++ b/.claude/skills/changelog-video/examples/master-skeleton.html @@ -33,9 +33,9 @@ .glass { background: rgba(10,12,11,.78); border: 1px solid rgba(190,255,205,.32); border-radius: 22px; box-shadow: 0 24px 60px rgba(0,0,0,.5); } /* caption rail — overlay on top of the film, never a reserved band */ - #cap-line { position: absolute; left: 0; right: 0; top: 1002px; height: 40px; text-align: center; - z-index: 7; font-family: 'TT Norms Pro', sans-serif; font-weight: 500; font-size: 25px; - letter-spacing: .01em; color: rgba(245,246,244,.92); + #cap-line { position: absolute; left: 0; right: 0; top: 990px; height: 52px; text-align: center; + z-index: 7; font-family: 'TT Norms Pro', sans-serif; font-weight: 500; font-size: 32px; + letter-spacing: .01em; color: rgba(245,246,244,.94); text-shadow: 0 2px 14px rgba(0,0,0,.85), 0 0 3px rgba(0,0,0,.6); white-space: nowrap; pointer-events: none; } .cap-phrase { position: absolute; left: 0; right: 0; } diff --git a/.claude/skills/changelog-video/references/build-spec.md b/.claude/skills/changelog-video/references/build-spec.md index beea84313..d376e7bbd 100644 --- a/.claude/skills/changelog-video/references/build-spec.md +++ b/.claude/skills/changelog-video/references/build-spec.md @@ -76,7 +76,7 @@ otherwise) and must stay flat 2D (no 3D ancestors). y ∈ [288, 944]. - Outro (≤3.5s): kicker FULL DIGEST, "See what shipped." ~96px, green rule, mono URL chip, tag line. Fade all + chrome ~0.5s before end. -- Caption rail per `script-voice.md` (top: 1002). +- Caption rail per `script-voice.md` (top: 990, font-size: 32px, height: 52). Mandatory — populate the master-skeleton's `LINES` array from `captions.json` before render; see SKILL.md step 5. ## Seams + internal life (doctrine mechanics) diff --git a/.claude/skills/changelog-video/references/script-voice.md b/.claude/skills/changelog-video/references/script-voice.md index ba2bdee1f..b39a550f0 100644 --- a/.claude/skills/changelog-video/references/script-voice.md +++ b/.claude/skills/changelog-video/references/script-voice.md @@ -105,7 +105,7 @@ transcript) before the captions are trusted. ## Caption rail (rendering) Per `captions-overlay`: a quiet OVERLAY, never a reserved band. One line, -bottom-center (top: 1002px on 1080-square), TT Norms Pro 500 25px, -ink .92, soft dark text-shadow, words fading in (0.12s) on their timestamps, -phrase swaps as sets. Keep critical small text out of the bottom ~80px +bottom-center (top: 990px, height: 52px on 1080-square), TT Norms Pro 500 32px, +ink .94, soft dark text-shadow, words fading in (0.12s) on their timestamps, +phrase swaps as sets. Keep critical small text out of the bottom ~100px center span; everything else may run under the rail.