mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
Vance Ingalls
0eca7b1e0a
* docs(audio): document the audio effects system The effects feature had no documentation at all — the only mentions anywhere in docs/ were changelog entries. This adds the three pages the color-grading feature already has, one per audience, and closes a gap in the existing audio guide. - prompting/audio-effects — a new Level 5 chapter on asking for a mix in symptoms rather than in filters, with the voiceover carve as the headline, the level-before-depth check, groups, and the three requests that have no honest answer (de-essing, noise removal, tone matching). - studio/audio-effects — the rack, presets by symptom, the carve module, groups, the deliberately asymmetric mute/solo, automation lanes, and troubleshooting. - reference/audio-effects — the contract: all four attributes, every effect and parameter range, why some parameters cannot be automated, the 19 presets, the five jobs and five one-knob profiles, carve semantics, the group model, the render bus, preview/render parity, and the three lint rules. Also points the existing "duck music under important speech" advice in guides/voice-and-audio at the carve, which does that properly and was never named there. Facts verified against source rather than the shipped skill prose: the panel section is "Audio FX" (the neighbouring "Effects" section is visual effects, and an earlier draft of this page named the wrong one), MAX_AUTOMATION_POINTS is 512, fromPreset carries a preset id rather than a boolean, the leveller targets the track's own 80th percentile, group mute reaches the render while solo never leaves Studio, and the CLI carve currently writes clip ids. No screenshots in this pass, and no placeholders for them either — the pages carry concrete markup and tables instead. Screenshots of the rack, the carve module, and a group row would each earn their place later. mint validate and mint broken-links both pass. * docs(audio): give audio its own Studio group, split by task Review feedback: the audio page did not belong in Studio / Edit. That group is already one task per page — canvas, timeline, animation, captions — and a single page covering the rack, the carve, groups, and automation lanes was four tasks bundled together and dropped in beside them. Studio now has an Audio group holding four task pages: - Effects and presets — the rack, presets by symptom, adding single effects in a working order, the one-knob controls, Even Out Levels - Voiceover carve — its own page, because it is the feature people come for - Groups, mute, and solo — including why mute reaches the export and solo cannot - Automation lanes — drawing envelopes, the shape menu, and which parameters cannot move at all Not a new top-level tab: the tabs here are audience-scoped (Guides, Studio, Catalog, Developers), so a feature tab would be the only one of its kind and would strand the prompting chapter out of its Level 5 sequence and the reference page out of Developers. Repointed the deep link in guides/voice-and-audio at the carve's own page. Lane interactions verified in source before documenting: the right-click menu offers Ramp up, Ramp down, Swell, Dip, and Simplify (which needs three points). Deliberately not documented: "clicking a lane label reveals it in the rack", which is not on main. mint validate and mint broken-links both pass. * docs(audio): fix the four review blockers All four verified in source before fixing; the review was right on every count. **The registry was incomplete.** `pitchshift` ships at `audioFx.ts:509-535` (`semitones` −12–12, `mix` 0–1, worklet-backed so neither automatable) and was missing entirely — sixteen effects, not fifteen. It also joins the worklet list, so five effects expose no automatable parameters rather than four. **Three presets were missing.** `chipmunk`, `giant`, and `monster` ship at `audioFxPresets.ts:336-357`, all built on `pitchshift`. Twenty-two presets, and Character holds ten. Fixed in both the reference table and the Studio list. **The copyable markup contradicted the warning above it.** The page said `carve.mjs` only finds double-quoted attributes and then gave three single-quoted examples — copying the chain example would make a later carve miss the existing chain and overwrite it. All three are now double-quoted with `"`, each followed by its unescaped reading so it stays legible. **The attribute table over-claimed.** `data-audio-group` is a plain id, not JSON, and is ignored on `<video>`; the other three also live on `<hf-audio-group>` for a group. The table now carries shape and valid host per attribute. **The automation contract was wrong on two axes.** A clip lane's `t` is clip-relative but a GROUP lane's is composition time, because a group has no `data-start` (`webAudioTransport.ts:337-342`, `audioMixer.ts:1311-1344`) — both pages now split the two clocks. And `volume` is not 0–1: the ceiling is `MAX_AUDIO_GAIN`, +12 dB or about 3.981 (`audioGain.ts:8-9`), so a boosting lane is valid and documented. **Current-main drift.** #3416 is merged, so the CLI now records the voices' shared group when it is safe and falls back to clip ids when that group contains the bed or a music/SFX member. Documented, including why neither refusal shows up on the run that writes it, and rebased onto main. mint validate and mint broken-links both pass. * docs(audio): name the real add-menu family, and finish propagating pitchshift Second review round. All three findings were my own incomplete propagation — I corrected the reference for `pitchshift` last round and left the reader-facing pages behind it. **The add-menu family is `Time`, not `Space`.** `propertyPanelFxAddMenu.tsx:22-28` labels the four groups Filters / Dynamics / Non-linear / Time, and the time group holds pitchshift, delay, chorus, phaser, and reverb. The Studio page sent readers looking for a group that does not exist. It is now a table naming the family and its contents, and the reference's "Time — space and width" heading is retitled, since that description stopped covering the family the moment pitch shift joined it. Also from the same file: the menu offers the named jobs in place of a bare `peaking`, because picking `peaking` is picking a machine and leaving the real decision — which range — for afterwards. Worth saying on the task page. **Pitch shift was missing from both no-automation lists** that a reader actually follows — `studio/audio-automation` and, unflagged but the same defect, the prompting chapter. Five worklet effects in all four places now. Called out explicitly on the Studio page, because a rising pitch is exactly the thing someone reaches for a lane to do, and the lane will not report that it cannot. **Narrowed the group-metadata sentence.** "The other three are JSON, and on a group they live on `<hf-audio-group>`" swept in `data-fx-carve`, contradicting the table directly above it. Only `data-fx-chain` and `data-automation` are group metadata. mint validate and mint broken-links both pass.
178 lines
7.0 KiB
Plaintext
178 lines
7.0 KiB
Plaintext
---
|
||
title: Audio effects and mixing
|
||
description: "Ask for a mix in symptoms rather than in filters — make music step out of the way of narration, clean a voice, and know which requests have no honest answer."
|
||
---
|
||
|
||
The last chapter gave your video a voice. This one makes the voice and everything
|
||
around it *listenable*.
|
||
|
||
Audio work is where prompts go wrong in a specific way: it is tempting to name
|
||
the machine. "Add a high-pass at 80 Hz with a Q of 0.7" is a real instruction, and
|
||
it is worse than the request that produced it, because it commits you to a fix
|
||
before anyone has established the problem. The mix is stored as effects on each
|
||
track — filters, dynamics, character, space, and envelopes that move any of them
|
||
over time — and the agent reaches into that toolbox for you. Your job is to
|
||
describe the symptom accurately.
|
||
|
||
## Say what it sounds like, not what to add
|
||
|
||
Every effect in the rack exists to answer a complaint. Name the complaint.
|
||
|
||
```text
|
||
The narration sounds muffled, like it's behind cardboard. Fix it.
|
||
```
|
||
|
||
```text
|
||
The voice sounds amateur — clean it up but don't make it sound processed.
|
||
```
|
||
|
||
```text
|
||
There's a hum under the interview audio the whole way through.
|
||
```
|
||
|
||
Each of those lands on a specific, small change: a cut at 250 Hz, the
|
||
`voice-clean` chain, a high-pass under the voice. You do not have to know which —
|
||
and if you name the wrong mechanism, you get the wrong mechanism applied
|
||
confidently.
|
||
|
||
This matters more than it does for picture work, because **you can hear a mix and
|
||
the agent cannot**. A prompt describing a sound is evidence. A prompt naming a
|
||
filter is a guess wearing evidence's clothes.
|
||
|
||
## The one request worth learning by name
|
||
|
||
Music under narration is the single most common audio problem, and the reflex fix
|
||
is wrong.
|
||
|
||
Ducking the whole music track works and costs the music all of its presence for
|
||
the entire voiceover — it goes limp for as long as anyone is talking. The voice
|
||
does not need the whole spectrum, though. It needs the few bands it actually
|
||
occupies. Taking only those out of the music is called a **carve**, and it keeps
|
||
the low end and the top, so the bed is still music while the voice stays
|
||
intelligible.
|
||
|
||
Ask for it in those terms:
|
||
|
||
```text
|
||
Put the music under the narration properly — carve it so the voice stays clear
|
||
without the track going limp.
|
||
```
|
||
|
||
```text
|
||
The music is fighting the voiceover. Make room for the voice in the music
|
||
rather than just turning the music down.
|
||
```
|
||
|
||
Two things are worth knowing so you can judge the result:
|
||
|
||
- **It follows the speech.** Silence leaves the music alone; a loud passage pushes
|
||
it to full depth. It is not a fixed dip held through every pause.
|
||
- **It lives on the music**, and names the voices it makes room for. If someone
|
||
tells you they carved the voice track, that is a bug rather than a taste
|
||
decision.
|
||
|
||
If it comes back and the music sounds *hollow* rather than simply quieter, say so
|
||
in exactly that word — it means the depth is too high, and it is the one failure
|
||
mode with an obvious sound:
|
||
|
||
```text
|
||
Too far — the music sounds notched now. Back it off.
|
||
```
|
||
|
||
## Ask for levels before asking for more depth
|
||
|
||
A carve cannot fix music that is simply louder than the voice. When narration is
|
||
buried, the useful instruction is about **level**, not about spectrum:
|
||
|
||
```text
|
||
Check the actual loudness of the voiceover against the music before touching
|
||
the carve — the voice may just be quieter than the bed.
|
||
```
|
||
|
||
Machine-generated speech commonly arrives far below a mastered music track. That
|
||
gap is arithmetic, not taste, and no amount of carving closes it. Asking for the
|
||
measurement first turns one round of guessing into a number.
|
||
|
||
## Group the things that belong together
|
||
|
||
Five narration clips want one set of effects, one fader, and one mute — not five
|
||
copies that drift apart as you edit. Say so:
|
||
|
||
```text
|
||
Group all the narration clips as one voiceover group, then carve the music
|
||
against that group.
|
||
```
|
||
|
||
That phrasing is worth the extra clause. A carve pointed at individual clips has
|
||
to list every one of them, and it stays right only until you add another — the
|
||
sixth clip plays outside the carve's awareness and the music silently fails to
|
||
duck under it. A carve pointed at a *group* picks up whatever is in the group at
|
||
the time it runs.
|
||
|
||
Use groups for sound effects too, when there are more than a couple. One place to
|
||
turn all of them down is worth more than precise individual levels you will never
|
||
revisit.
|
||
|
||
## Ask for movement when a static setting will not do
|
||
|
||
Anything that should change over the video's length is an envelope, and you ask
|
||
for it by describing the shape in time:
|
||
|
||
```text
|
||
Fade the music out over the last three seconds.
|
||
```
|
||
|
||
```text
|
||
Bring the ambience up while the wide shot is on screen, then pull it back
|
||
under the interview.
|
||
```
|
||
|
||
One caveat worth carrying: a few effects cannot move over time at all — the
|
||
compressor, limiter, gate, bitcrush, and pitch shift are configured whole rather
|
||
than knob-by-knob. If you ask for a compressor that tightens as a scene builds, the
|
||
honest answer is a level stage moving in front of it instead. An agent that
|
||
quietly writes an envelope on one of those has written something inert, so if a
|
||
requested change does nothing audible, that is the first thing to suspect.
|
||
|
||
## Three requests with no honest answer
|
||
|
||
Naming a gap is more useful than accepting the nearest preset and calling it the
|
||
thing.
|
||
|
||
**De-essing.** Sharp `s` sounds need a detector faster than anything in the
|
||
toolbox. The nearest fix is a narrow cut in the 5–9 kHz range, swept to find
|
||
where that particular voice spits. It is always on, so it costs a little air on
|
||
every word. That trade is usually worth it — but ask for it knowing it is a
|
||
trade:
|
||
|
||
```text
|
||
The s sounds are spitting. I know there's no real de-esser — put a narrow cut
|
||
where this voice actually sibilates and tell me what it cost.
|
||
```
|
||
|
||
**Noise removal.** A gate closes the gaps between phrases; the hiss *underneath*
|
||
the words is untouched by anything available. A source with audible hiss needs a
|
||
better source, and an agent telling you so is being accurate rather than lazy.
|
||
|
||
**Matching one voice to another.** There is no match-curve tool. Two takes can be
|
||
brought closer by hand with an EQ, which is predictable in a way a derived curve
|
||
would not be — but it is hand work, and worth asking for as such.
|
||
|
||
## Describe the whole mix once, at the end
|
||
|
||
Individual fixes accumulate into something nobody has judged as a whole. One
|
||
closing instruction is worth more than three more adjustments:
|
||
|
||
```text
|
||
Render an audio-only pass and check the mix end to end: voice clear through
|
||
the loudest musical moment, effects where the action is, nothing surprising
|
||
at the open, and the music finishing on purpose rather than at the file edge.
|
||
```
|
||
|
||
Audio problems are easier to notice when the picture is not competing for
|
||
attention. Listening once without watching catches things twenty prompt rounds
|
||
will not.
|
||
|
||
*Next: [Design systems and brand](/prompting/design-systems) — pointing the agent
|
||
at a source of brand truth instead of describing a vibe.*
|