mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 16:42:27 +00:00
ukimsanov
592301248e
**The poster guard I added twice was unworkable and I never ran it.** It called existsSync on docs/images/catalog/<name>.png. That directory is gitignored — previews are generated locally, uploaded to the CDN and never committed — so the check is false on every clean checkout and in CI. It would have stripped the poster from all 168 pages, not the 13 with a missing file. It also referenced REPO_ROOT, which does not exist in that file, so the script crashed on the first item. I described this guard in two commit messages without once executing the generator. The poster is now gone entirely, which is the smaller and more honest fix. These previews are autoPlay muted loop, so the poster is visible for a few hundred milliseconds; 13 of the 168 files do not exist and the browser fetches the poster before the video. Removing the attribute kills 13 x 403 and 168 needless image requests, and there is nothing to keep in sync. Also applied a complexity pass to the delivery encode: - hasAudio() deleted. 17 lines and a spawnSync per item to choose between "-c:a aac" and "-an". ffmpeg ignores -c:a when the input has no audio stream; checked, exit 0, output carries no audio track. - The 40-line spawn + Promise wrapper is execFileSync. Everything around it in that script is already synchronous. - The duplicated poster lookup is gone with the poster itself. Net 76 lines lighter. Generator runs clean, emits 168 pages, carry-forward intact.
130 lines
5.0 KiB
Plaintext
130 lines
5.0 KiB
Plaintext
---
|
|
title: "Use voice, music, sound, and captions"
|
|
sidebarTitle: "Voice, sound, and captions"
|
|
description: "Build an understandable audio mix, transcribe real speech, and turn it into readable captions."
|
|
---
|
|
|
|
import { DocsVideo } from "/snippets/docs-video.jsx";
|
|
|
|
Give each audio layer one job. Add another layer only when it helps the viewer
|
|
understand or feel the video.
|
|
|
|
| Layer | Main job |
|
|
| ------------- | --------------------------------------------------- |
|
|
| Voiceover | Carry the explanation or story |
|
|
| Source audio | Preserve speech or sound from existing footage |
|
|
| Music | Shape pace, tone, and structure |
|
|
| Sound effects | Reinforce a specific action or transition |
|
|
| Captions | Make spoken words readable and searchable on screen |
|
|
|
|
|
|
<DocsVideo
|
|
title="Narration, music ducking under it, and effects landing on the beat"
|
|
src="https://static.heygen.ai/hyperframes-oss/docs/images/showcase/voice-audio-demo-v1.mp4"
|
|
poster="https://static.heygen.ai/hyperframes-oss/docs/images/showcase/voice-audio-demo-v1.jpg"
|
|
/>
|
|
|
|
Turn the sound on for this one — the audio is the subject. You can watch the
|
|
music duck as narration arrives and see each effect land against the waveform.
|
|
|
|
## Create a voiceover
|
|
|
|
Give the agent the approved wording and useful voice direction:
|
|
|
|
```text
|
|
Generate the approved SCRIPT.md as a warm, direct voiceover.
|
|
Natural pace, no announcer energy. Keep the product-name pronunciation exact.
|
|
```
|
|
|
|
The project can keep the script, audio, and word-level transcript separately.
|
|
That lets you regenerate the voice without discarding the timing and caption
|
|
work around it.
|
|
|
|
Listen before building the final edit. Fix wording, pronunciation, or delivery
|
|
in the script or voice direction rather than cutting around a bad read.
|
|
|
|
## Transcribe recorded speech
|
|
|
|
Transcription creates timed words from an audio or video file. Start with the
|
|
automatic engine: it uses Parakeet when installed and falls back to Whisper.
|
|
|
|
```bash
|
|
npx hyperframes transcribe interview.mp4
|
|
```
|
|
|
|
Pass the spoken language when you know it. This filters non-target speech and
|
|
lets the Whisper fallback choose a multilingual model when needed.
|
|
|
|
```bash
|
|
npx hyperframes transcribe interview.mp4 --language es
|
|
```
|
|
|
|
For difficult English audio, force the larger English Whisper model. For an
|
|
unknown language, use multilingual `large-v3` and let Whisper detect it.
|
|
|
|
```bash
|
|
npx hyperframes transcribe interview.mp4 --engine whisper --model medium.en
|
|
npx hyperframes transcribe interview.mp4 --engine whisper --model large-v3
|
|
```
|
|
|
|
Larger models take longer and do not remove the need to read the result.
|
|
|
|
The command can also import an existing `.srt`, `.vtt`, or supported transcript
|
|
JSON file:
|
|
|
|
```bash
|
|
npx hyperframes transcribe subtitles.srt
|
|
```
|
|
|
|
## Turn the transcript into captions
|
|
|
|
1. Read the complete transcript.
|
|
2. Correct names, product terms, numbers, punctuation, and obvious recognition
|
|
errors.
|
|
3. Group words into short phrases that break on meaning.
|
|
4. Open [Captions in Studio](/studio/captions) to inspect the rhythm and adjust
|
|
placement, scale, or rotation. Make durable wording, timing, style, and
|
|
animation changes in the caption source or ask the agent to update it.
|
|
5. Watch once with sound for synchronization and once muted for readability.
|
|
|
|
Captions should follow what was said. They should not cover a face, product
|
|
control, or another important visual. Use emphasis on the few words that carry
|
|
the point instead of making every word compete.
|
|
|
|
For an existing talking-head clip, use [Captions or
|
|
recuts](/guides/captions-and-recuts) to choose between plain captions, designed
|
|
graphic overlays, and an actual change to the spoken edit.
|
|
|
|
## Mix for understanding
|
|
|
|
- Keep voice clear above music.
|
|
- Duck music under important speech instead of reducing the whole track
|
|
equally.
|
|
- Use sound effects for meaningful events, not every movement.
|
|
- Avoid cutting words, breaths, or reverb tails accidentally at scene
|
|
boundaries.
|
|
- Let a deliberate silence remain silent.
|
|
- Listen on ordinary headphones and laptop speakers.
|
|
|
|
When narration drives the project, time visual changes to its real transcript
|
|
rather than estimated scene lengths. When music drives the project, use the
|
|
[music-video workflow](/guides/music-to-video) and its analyzed structure.
|
|
|
|
## Review the complete sound pass
|
|
|
|
- The correct voice and source recordings are used.
|
|
- Names, claims, and captions are accurate.
|
|
- Speech stays clear through the loudest musical section.
|
|
- Effects arrive with the action they support.
|
|
- The opening does not surprise the listener with excessive volume.
|
|
- Music and ambience finish intentionally instead of stopping at the file edge.
|
|
|
|
Render a review file and listen without watching once. Audio problems are often
|
|
easier to notice when the visuals are not competing for attention.
|
|
|
|
## Related topics
|
|
|
|
- [Add captions or repackage talking-head footage](/guides/captions-and-recuts)
|
|
- [Edit captions in Studio](/studio/captions)
|
|
- [Finish, render, and share](/guides/export-and-share)
|