diff --git a/skills/hyperframes/references/sound-effects.md b/skills/hyperframes/references/sound-effects.md index 42b5f6a778..35f7d790cd 100644 --- a/skills/hyperframes/references/sound-effects.md +++ b/skills/hyperframes/references/sound-effects.md @@ -70,7 +70,7 @@ All times in the analysis are **clip-relative** (0 = the clip's own start), _not 1. **Trim dead air first.** A catalog clip can carry silence padding — a "5s riser" may be 2s silence + 5s rise + 1s silence — and if you place the raw file it fires late and wastes its tail. Set `data-media-start = onset` and `data-duration = tail − onset` so only the real content plays. (`sfx add`/`inspect` print this as a ready `trim` line.) After trimming, the audible sound begins **exactly at `data-start`**. 2. **Anchor to the beat.** A **hit** (click, impact, whoosh) aligns by its attack — now at the start — so `data-start = beat`. A **build** (riser) aligns by its peak, which sits `peak − onset` into the trimmed clip, so `data-start = beat − (peak − onset)`. Example: a riser with onset 2.0s, peak 6.0s landing on a beat at 10.0s → `data-media-start="2.0"`, `data-start="6.0"` (= 10.0 − 4.0); fire the drop at 10.0s. They overlap — that's the chain. -3. **Level** — the catalog is loudness-normalized, so the volume hierarchy is your baseline; you usually don't touch it. Use measured LUFS only to _nudge_: if two layered clips clash, drop `data-volume` on the one with the higher (louder, less-negative) LUFS. +3. **Level** — the catalog is loudness-normalized at the file level, so the default is to omit `data-volume` and let the clip play natural. Only nudge when measured LUFS shows a real mismatch — if two layered clips clash, drop `data-volume` on the one with the higher (louder, less-negative) LUFS. 4. **Shape** — the sparkline spans the whole clip; block `i` sits at `t ≈ (i + 0.5) / length × duration`. Read it for _where_ the energy builds, drops, or breaks — then place your trigger there. (There's no need to _cut_ the audio into a new file — `data-media-start` + `data-duration` trim at render time, losslessly and reversibly. Only mid-clip gaps, which a single window can't remove, would call for a different clip.) @@ -103,7 +103,10 @@ hyperframes sfx inspect Search hits return `name`, `description`, `duration`, and `score`. Use the **duration to tell the mode apart** — a 0.5s result is a hit, a 10s result is a bed. Always `add` (download) and embed the **local file**; never embed the search result's `audio_url` (it's a presigned URL that expires before render): ```html - + + ``` ### Access (free, but needs a key) @@ -129,23 +132,25 @@ SFX are plain audio clips — no special engine handling. The mixer reads exactl - **`data-start`** — when the clip fires (seconds): the trigger frame (or the scene start, for a bed). - **`data-duration`** — how long it plays; use the catalog duration. (`data-end` also works; duration is end minus start.) -- **`data-volume`** — 0 to 1. This is the **only** knob that sets level under voiceover. There is **no pan** — volume and timing are the only audio controls. -- **`data-media-start`** — trim into the source file (skip the first N seconds), for clips with a slow attack. +- **`data-media-start`** — trim into the source file (skip the first N seconds), for clips with a slow attack. Set from `sfx add`'s `onsetSec` so the audible attack lines up with `data-start`. +- **`data-volume`** — 0 to 1. **Optional.** Omit to play at natural catalog level (= 1.0). Use only as a per-clip nudge, not as a default duck. -`data-track-index` does **not** affect audio mixing — every clip mixes together regardless of its index. It only organizes the visual timeline. So `data-volume`, not the track index, is what keeps SFX under the VO. +`data-track-index` does **not** affect audio mixing — every clip mixes together regardless of its index. It only organizes the visual timeline. -## Volume hierarchy +## Volume — default is natural, no auto-duck -Layer levels so nothing fights the voice: +**SFX play at their natural catalog-normalized level.** The catalog is loudness-normalized at the file level, so embedding a clip without `data-volume` lets it ring at the level the catalog committed to. Multiple SFX can overlap, and SFX can stack on top of voiceover — the engine never auto-ducks, by design. If the natural mix sounds wrong, fix it by trimming dead air, shifting timing, or swapping clips, not by reflexively scaling volume down. -| Layer | data-volume | -| -------------------- | ----------- | -| Narration / VO | 1.0 | -| Background music | 0.4 – 0.6 | -| Sound effects (hits) | 0.2 – 0.35 | -| Ambient bed | 0.08 – 0.2 | +The columns below are **reference nudge ranges** for the rare per-clip override (a single clip measurably louder than its neighbors, a bed designed to be felt-not-heard). They are NOT defaults the agent applies on every clip. -A bed sits at the very bottom — it should be _felt_, not heard. A whoosh at 1.0 over narration is jarring; at 0.3 it punctuates without masking the words. When two SFX should match in level, compare their measured LUFS (from `sfx add`/`inspect`) rather than assuming equal `data-volume` sounds equal. +| Layer | nudge range (only if needed) | +| ------------------------------ | ---------------------------- | +| Narration / VO | 1.0 (always) | +| Background music under VO | 0.4 – 0.6 | +| Sound effects (hits) — nudge | 0.7 – 1.0 (default = natural)| +| Ambient bed — felt-not-heard | 0.08 – 0.2 | + +When two SFX should match in perceived level, compare their measured LUFS (from `sfx add` / `sfx inspect`) — that's more reliable than guessing-and-tweaking `data-volume`. ## Timing wisdom @@ -172,9 +177,9 @@ A bed sits at the very bottom — it should be _felt_, not heard. A whoosh at 1. **An atmospheric scene** — a tense product teaser over a dark background: -- Lay **one bed** (a low tension drone, ~10s) under the whole scene at `data-volume="0.12"`. No beat needed — it sets mood. -- Punctuate only the real beats on top: a riser into the product reveal, one impact on the reveal frame. -- Result: a continuous floor + two punctuation sounds. +- Lay **one bed** (a low tension drone, ~10s) under the whole scene. A "felt, not heard" bed is one of the few cases where setting `data-volume="0.12"` is right — it's the exception that proves the natural-level rule. +- Punctuate only the real beats on top — at natural volume: a riser into the product reveal, one impact on the reveal frame. Don't reflexively scale them down. +- Result: a continuous floor + two punctuation sounds, ringing as the catalog committed to. ## Constraints diff --git a/skills/website-to-hyperframes/references/step-3-storyboard.md b/skills/website-to-hyperframes/references/step-3-storyboard.md index 151046d4a6..4c5a6db2f6 100644 --- a/skills/website-to-hyperframes/references/step-3-storyboard.md +++ b/skills/website-to-hyperframes/references/step-3-storyboard.md @@ -287,16 +287,16 @@ When planning beats, decide which ones deserve an HTML-in-Canvas treatment vs. a SFX come from HeyGen's global catalog via the `hyperframes sfx` CLI — **not** a bundled file set. The full model (when to use them, the five families, the volume hierarchy, the trim/anchor recipe) lives in [`../../hyperframes/references/sound-effects.md`](../../hyperframes/references/sound-effects.md); read it. This step only decides _which moments get sound and what kind_ — Step 5 searches the catalog and wires the clips. -**Browse first, then assign by function — not by filename.** Run `hyperframes sfx list` to see what families exist (impacts, whooshes, risers, memes, stingers, ambiences, …). For each beat that earns a sound, specify it by _what it does + how it feels_ (the catalog is searched by meaning), the moment, and a volume: +**Browse first, then assign by function — not by filename.** Run `hyperframes sfx list` to see what families exist (impacts, whooshes, risers, memes, stingers, ambiences, …). For each beat that earns a sound, specify it by _what it does + how it feels_ (the catalog is searched by meaning) and the moment: -- a punchy transition whoosh at `0.2s`, volume `0.3` — on the hero image snapping in -- a soft success chime at `3.8s`, volume `0.5` — on the logo appearing +- a punchy transition whoosh at `0.2s` — on the hero image snapping in +- a soft success chime at `3.8s` — on the logo appearing **Less is more.** Most beats need zero SFX; one per beat is typical (see the global doc's "count beats, not animations"). Never place SFX on shader transitions — they're already an audio-visual event. -**Placement, trimming, and chains are all in the global doc — don't restate per-file timing here.** Step 5 follows it: `sfx add` measures each clip (peak, onset/tail, loudness), then trims dead air and anchors hits/risers precisely. (This replaces the old "peak at start/end, never trim" rules — catalog clips aren't a fixed hand-tuned set, so trimming to the _measured_ onset/tail is how you avoid late or cut-off sounds.) +**Placement, trimming, and chains are all in the global doc — don't restate per-file timing here.** Step 5 follows it: `sfx add` measures each clip (peak, onset/tail, loudness), then trims dead air via `data-media-start` (= `onsetSec`) and anchors hits/risers precisely against `data-start`. The trim is what aligns the audible attack to the beat frame — that timing precision matters even more now that volume defaults to natural. -**Volume under narration:** HyperFrames has no auto-ducking — keep SFX at 0.2–0.3 under VO. Note the intended volume per entry so Step 5 wires it. +**Volume — default natural.** Don't pre-prescribe per-clip volumes here. SFX play at the catalog's loudness-normalized level (no `data-volume` emitted). The engine has no auto-ducking — that's by design. Multiple SFX may overlap, and SFX may stack on top of VO; if a specific clip measurably overpowers its neighbors, Step 5 can nudge that one clip's `data-volume`, but it's the exception, not the default. The "felt, not heard" ambient bed is the one common case for an explicit lower volume (~0.1). **Access:** the catalog is free but needs a HeyGen API key. If it isn't set when Step 5 runs `sfx`, ask the user for one (free at https://app.heygen.com/developers/api) or build without SFX — never silently drop them. diff --git a/skills/website-to-hyperframes/scripts/assemble-index.mjs b/skills/website-to-hyperframes/scripts/assemble-index.mjs index e506e65f98..912658fb56 100644 --- a/skills/website-to-hyperframes/scripts/assemble-index.mjs +++ b/skills/website-to-hyperframes/scripts/assemble-index.mjs @@ -354,9 +354,14 @@ sfx.forEach((cue, i) => { ); return; } - const vol = cue.volume != null ? cue.volume : 0.35; + // SFX default: play at natural catalog-normalized level. The catalog is + // loudness-normalized at the file level, so emitting no `data-volume` + // attribute (= 1.0) lets clips ring as-recorded. Agents can still set + // `cue.volume` explicitly to nudge a specific clip — that override is + // honored. Even when multiple SFX overlap or stack with VO, the resulting + // mix is the catalog's natural sum (no auto-duck, by design). if (sfxEmitted === 0) body.push(` `); - body.push( + const lines = [ ` { ` data-start="${cue.t}"`, ` data-duration="${cue.duration}"`, ` data-track-index="${20 + i}"`, - ` data-volume="${vol}"`, - ` >`, - ); + ]; + // `cue.media_start` lets a clip skip dead air at the head (set from the + // `sfx add` analysis's `onsetSec`). Aligning audible-onset to `data-start` + // is the timing-alignment knob the volume change preserves. + if (cue.media_start != null) lines.push(` data-media-start="${cue.media_start}"`); + if (cue.volume != null) lines.push(` data-volume="${cue.volume}"`); + lines.push(` >`); + body.push(...lines); sfxEmitted++; });