mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-19 19:36:04 +00:00
step-3-storyboard.md (line 436) and step-4-vo.md (line 116) instruct the agent to write `**Narration start:** <n>s [optional parens]` in STORYBOARD.md Global Direction when narration begins after t=0 (e.g. 0.8s after a hero intro settles, or to align with a music intro). w2h-prep was ignoring that line in two places: 1. The voice object emitted to group_spec.json hardcoded `start_s: 0`, so assemble-index always wrote the narration <audio> with data-start="0" — meaning the audio played from t=0 regardless of what the storyboard said. 2. splitTranscriptPerScene partitioned transcript words by comparing AUDIO-TIME timestamps (start=0 = first audio sample) against COMPOSITION-TIME scene windows (start_s = cumulative beat duration from t=0). The two axes only coincide when narration starts at 0. With any offset > 0, words land in the wrong scenes or get dropped when they fall outside the partition windows. The lived consequence on agents using a music-intro pattern: captions silently desynced by exactly the narration offset, and the narration audio played at t=0 instead of t=offset (overlapping the music intro). Fix: * parseNarrationStart(projectDir) parses the storyboard line defensively: handles `0.8s`, `0.8`, `2`, parenthetical commentary after the value. Defaults to 0 on absent file / missing line / malformed value / negative. * splitTranscriptPerScene gains a voiceStartS param (default 0 -> backward-compatible). Word timestamps are shifted by +voiceStartS BEFORE partition, converting audio-time -> composition-time so the partition window matches. * main() calls parseNarrationStart once, passes it to the splitter, and uses it for both voice.start_s and voice.duration_s (totalDuration - narrationStart, floored at 0). Verified the parser against 5 cases (decimal+s+parens, missing file, missing line, integer, malformed) and the splitter against a 4-word / 2-scene fixture with offset=0.8 (correct comp-time alignment) and offset=0 (bit-identical to prior behavior). Downstream impact: - assemble-index already reads voice.start_s / voice.duration_s and emits them as data-start / data-duration on the narration <audio> — no change needed there. - captions.mjs reads scene-local word timestamps and adds back scene start_s — also no change needed, because the splitter now emits scene-local times relative to the shifted comp-time word stream.