mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 09:16:55 +00:00
* refactor(skills): fold six domain skills into hyperframes and media-use guides
hyperframes-core, -animation, -keyframes, -creative and -studio move to
skills/hyperframes/{core,animation,keyframes,creative,studio}/, and
hyperframes-audio moves to skills/media-use/mix/. Each moved SKILL.md becomes
a frontmatter-free GUIDE.md so installers that scan recursively for SKILL.md
no longer see them as skills; the published list goes from 21 to 15.
Relative links, code paths and skill-name mentions are rewritten to the new
locations, the catalog surfaces, marketplace core-skills list, fallback core
set, module-copy targets and skills-manifest.json follow, and
`hyperframes skills update` now removes the retired skills (RETIRED_SKILLS)
when they are still installed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): slim the core entries and give media-use a HeyGen and a free path
The four entries every run opens are rewritten to what an agent cannot infer:
hyperframes 21.8 KB to 7.7 KB, hyperframes-cli 24.8 KB to 6.9 KB,
hyperframes-registry 9.9 KB to 4.3 KB, media-use 9.7 KB to 4.9 KB. Detail moved
whole into references (routing close calls and creator edits, CLI pin upgrades,
catalog search tiers, CLI conventions and feedback reports).
- hyperframes: state table, routing, the render contract, a terminal work loop
(check, snapshot, preview before render), the desktop-app hand-off, and a
table of the folded guides.
- media-use: one table of HeyGen versus free local tools per need; HeyGen
sign-in, credits and the CLI's video-agent, avatar, clone, lipsync, translate
and clipping commands; every free path works with no account.
- scripts/check-core-skills.mjs holds entries and descriptions to budget and
checks every taught `npx hyperframes` command and path; runs in lint.
- Core entries are exempt from the formatter: its table padding cost ~1.5 KB
per entry on every run.
- Content tests follow the text to its new home.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): plugin runs skip skills update; media-use explains HeyGen sign-in and TTS setup
The smoke eval showed a presenter request never reaching media-use and the
free voiceover failing on a missing kokoro-onnx; plugin runs also ran the
skills update the plugin rules forbid.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(skills): drop a committed Python bytecode file and ignore bytecode
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): align the entry with the Studio skills and keep the workflows' measured facts
The entry drops routing and the intent interview and carries the contract, the
editable-project rules and the work loop. Studio's format, runtimes, motion,
look and captions references come over for terminal use, frame presets move to
assets/, media-use gains Studio's mixing and local-command references, and
film.md and overlays.md keep what the workflow skills measured.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills)!: retire the workflow skills and folded guides; publish seven knowledge skills
The skill pack now publishes exactly seven skills - hyperframes,
hyperframes-cli, hyperframes-registry, media-use, figma, slideshow and
remotion-to-hyperframes - carrying knowledge, not process. The eight
workflow skills, the folded domain guides and the routing/interview
references are deleted; their measured facts already live in the
hyperframes references (format, runtimes, motion, look, captions, film,
overlays) and media-use's mixing reference.
- bake-basemap.mjs and locate.mjs move to skills/hyperframes/scripts/
(bake-basemap gains --help); extract-audio-data.py moves to
media-use/audio/scripts/ for the caption motion reference.
- skills update: the eight workflows join RETIRED_SKILLS so existing
installs prune them (by lock attribution only, since their names are
generic); naming a retired skill reports where its knowledge went and
succeeds instead of failing as unknown.
- Capture prompts point at the hyperframes skill's references/film.md;
the live-map warning names the new bake-basemap path.
- Content tests that pinned knowledge are retargeted to the kept
references; tests of deleted workflows, scripts and process go.
- Catalog surfaces, templates and docs describe the seven skills.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): resolve the local.md captions link and allow capture's output files in citations
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): regenerate the skills manifest from tracked files only
The committed media-use hash included an ignored __pycache__ bytecode file present
in the local checkout, so CI's clean checkout hashed differently.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(docs): restore generated catalog pages and docs navigation to main
Catalog PRs commit item sources only; CI regenerates the pages from the edited
registry sources. docs/docs.json is catalog navigation, so its Workflows group
rename is left for a non-catalog PR.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(cli): raise the CLI size budget for the skill references it bundles
The CLI bundles skills/hyperframes, which now carries the folded domain guides and
frame presets (155 KB to 1.08 MB in git). Budgets are the CI-measured size plus 5%:
packed 7526634, unpacked 22593213.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test: keep repo-sweep guards meaningful after the workflow skills left
The last shell ffprobe caller and most root index.html examples lived in retired
skills. The shell detector is now pinned on a literal command line, and the
structure-rule sweep needs more than 5 roots (11 remain) rather than 30.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): slim figma, slideshow and remotion to the Studio shape; restore the storyboard board
slideshow 33.5 KB to 4.9, figma 18.2 to 6.2, remotion-to-hyperframes 11.0 to
4.9, each keeping the facts only it holds (the figma motion helpers, verify gate,
storyboard decoding and usage beacons; present and hand-off; the Remotion lint,
SSIM check and API map). The board page format from the storyboard recipe comes
back as hyperframes/references/storyboard.md: the desktop app's Storyboard tab
reads boards/storyboard.html by its frame, title and time classes. The entry and
the init template now say to snapshot the moments an edit changed before
replying, which the terminal eval found skipped in 22 of 38 runs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): manifest without local bytecode; Studio's edit prompt names kept files
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): render when the person asks for the file or for no stops, not only after approval
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): keep the voiceover carve in media-use; presets stage their own fonts
Review follow-ups on #5651:
- carve.mjs and its tests move to skills/media-use/audio/scripts; the audio
docs, mixing.md and two Studio comments point there.
- code-editorial's FRAME.md says to copy its bundled WOFF2s and add the
@font-face blocks, instead of promising the retired build-frame.mjs.
- media-use states the host rule before the HeyGen sign-in steps.
- storyboard-format marks transition_in as a note the agent builds itself.
- Two stale citations (creator-editing-recipes.md, sub-compositions.md).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(skills): regenerate the manifest without local bytecode
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): the CLI render step follows step 4's rule, not approval alone
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): run the core-skills check from lint again after the main merge
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): films that tell a story start from three storylines; maps use real imagery
Opus 5.5 arm D read motion.md and applied it, but its launch film was a
feature list (pain, three feature cards, proof, logo): film.md left story
to the model. And the map case never opened film.md, so it drew Natural
Earth outlines instead of baking satellite tiles.
- references/story.md: the generic half of Studio's moment-film story and
craft (one thing, three storylines with world and device, the hook,
beats, worlds, copy, brand, tells of a generated film). The person picks
a storyline, or the agent takes its recommendation when asked for the
file or for no stops.
- The entry's work loop names the storyline step for films over ~15 s; the
references table routes stories to story.md and maps to film.md.
- film.md points at story.md and gives maps and real places a section.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): story.md follows the entry: no voiceover unless asked
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): bake-basemap starts from a globe and stops waiting out its idle timeout
The Opus 5.5 map run spent 13 of its 38 minutes on parallel bakes that
stalled and 15 more recording 300 frames, after rewriting the script because
it could only start on a flat map.
- PROJECTION=globe with a seeded starfield and atmosphere, and FROM/PAN to
turn the globe to CENTER before the dive.
- Each frame polls for loaded tiles instead of waiting for an "idle" event
that had often already fired: 30 frames went from 198-219 s to 10-13 s
locally with byte-identical frames.
- Progress prints seconds a frame and minutes left; film.md shows the globe
call and says to run one bake at a time.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): render in the foreground; wait on a detached render's log, never pgrep -f
Both Opus 5.5 explainer runs started a detached render, then waited with
`until ! pgrep -f "hyperframes render"`, which matches its own loop: each
render took 88-117 s and the wait sat for the tool's full 10 minutes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): route each kind of film; storylines carry look and sound; add review and sound
The entry routes by request (launch, explainer, music, data, vertical, 3D,
character) to a self-contained file under references/types/, and a film
over ~15 s now gets music and sounds on its visible actions unless the
person asked for silence. Each storyline names its look and its sound, so
picking one picks both; STORYBOARD.md frames carry `reads` and `sound`.
review.md is the whole-film pass: the check warnings, the review sheets,
seven questions to look for, P0/P1/P2 notes and REVIEW.md. motion.md gains
eases by role, six title entrances, a reading-time rule and nine measured
transitions; look.md a design plan before the first scene, with presets
kept for decks and named picks. story.md adds the second default that
generated design falls into, with what to do instead.
The blank template and format.md no longer teach the fade-and-slide-up
entrance on a dark canvas.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): check warns on unread lines, dead stretches, small type, empty frame 0 and silence
A coding agent cannot watch its film, so check now measures what a viewer
notices over time. A new `craft` section holds warnings only (exit 0
unless --strict), sampled on the dense 8 fps grid the overlap pass
already seeks, skipped for --range checks:
- line_unread: a reading-size line (>= 30 px tall at 1080) on screen
under 0.4 s for 1-2 words, else under 3/4 of 0.5 s + 0.3 s a word;
the film's last line needs 2 s. Short lines are re-measured between
samples to ~0.01 s before judging. Captions-track and
data-layout-ignore text are exempt.
- dead_stretch: more than 3 s where the keepsMoving liveness signature
does not change, outside the final 3 s and any data-hold element;
quiet when sweep_static fires or a whole-composition keepsMoving
assertion owns the limit.
- text_below_floor: text under 2.6% of the short side (28 px on 1080)
held 0.5 s, one finding per file listing every element.
- first_frame_empty: frame 0 paints only the background.
- film_silent: over 15 s with no unmuted audio or audible video.
Each message names the fix and the skill reference to read. keepsMoving
now also leaves the film's last 3 s (a still end card) out of its
static runs, so the two agree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): snapshot --review captures a beat sheet, seam strips and a phone sheet
snapshot --review picks the frames that show a film over time instead of
evenly spaced ones, in the same single browser session:
- review-beats.jpg: one settled frame per scene at its start + 0.3 s,
plus one per timed data-beat="name" element, labelled time and id.
Scenes are the composition's top-level timed clips and sub-composition
hosts, read like `timeline` reads rows (captions and audio excluded)
and timed by the runtime's start resolver over the live page.
- review-seam-NN.jpg: frames at -0.2, -0.1, 0, +0.1, +0.2 s around
every cut (each scene start, and any scene end no scene starts at).
- review-phone.jpg: the beat frames at 360 px wide.
--composition scopes the review to one sub-composition; with fewer than
two scenes the beats spread evenly and a note says so. snapshot also
gains --json: one envelope with every frame's path and time and, under
--review, the scenes, beats and seams behind the sheets; progress moves
to stderr.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): craft warnings stay out of --strict; a hold gets a real next action, not a caret
Craft findings (unread lines, dead stretches, small type, an empty first
frame, silence) are judgment calls a film may break on purpose, so
--strict keeps gating defects only. The check reference no longer
suggests a blinking caret to keep a still frame moving.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): review spreads beats when every scene starts together; map credits skip the type floor
A map film whose basemap and overlay both start at 0 s got one beat
frame; the even spread now keys on distinct scene starts. film.md tells
agents to mark the imagery credit data-layout-ignore and back it, since
the new type floor flagged it on the first real run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): review frames each scene at its middle too, not only 0.3 s in
On the first real explainer, 0.3 s into a scene showed a bare colour
field before its title entered. Each scene longer than about a second
and a half now adds a frame at its middle, so the sheet shows what the
scene builds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(skills): a film is built from a numeric SPEC.md, with defaults banned by name and a look test
Studied in full: flxrnc's Motion Lab (vague vs directed prompt), 0xMovez's
course, EianLu's riso skill, agentchengfeng's style samples and the two
Opus 5.5 guides. Their common point: adjectives are where the default look
lives, so the agent now writes SPEC.md before any scene (GOAL, OUTPUT,
TIME, LOOK, MOTION, SHOTS, BANNED, QA), with a number for every adjective.
spec.md bans the defaults by name, unless the brand or reference uses
them (a dark void with small things floating, glow and particles, the
cream-serif-mono-label look, fade-and-slide-up, corner brackets), and
grows that list after the first render. Each storyline gets a 3 s test of
its hero shot before the pick; a look is written as the rules of one
medium. Composition numbers: the hero fills 40% of the frame, one hero
move per shot, the shot's information in within 2.5 s then a held poster.
motion.md adds springs by damping ratio, shorter exits, overlap, stagger
and the beat grid; review.md reviews as a harsh director and grows BANNED;
vertical.md takes the riso skill's measured safe band; explainer.md the
shapes of information.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): story before look; every object reads as itself; posters only for info cards
The first spec-driven C02 swapped the dark default for a flat Swiss
poster, and the story went with it: two black bars nobody read as mirrors,
slogans instead of cause and effect, a storyline picked for its picture.
The spec now serves the story: a storyline is picked for the story a
stranger would follow, the look test asks first whether a stranger
understands what they see, and SPEC.md gains an OBJECTS line giving each
thing the material and details that make it read without a label. The
2.5 s held-poster rule is limited to information cards; in a story film
every shot causes the next. Explainers start from a person or situation
the viewer knows. Review names every object as a stranger would and
treats one that needs explaining as a P0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(skills): storylines are three films in words, each with a device and a form
The three storylines are pitched as text and the person picks from them;
nothing is built for a storyline before the pick, so the 3 s hero-shot
look test is gone. Ideas pair a device with a form, five or more first,
keeping the three that differ most, and the one stop states the rest in
an assumption line that names every paid item beside its free option.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: list main's new embedded-captions hf-cli test among the retired workflow files
The merge with main removed it with the rest of the retired skill, and the
deletion guard rejects any removal it has not been told about.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(core): keep the audio preset comment within core's size budget
The comment's new citation to media-use's mixing reference made the published
package 14 bytes larger than its budget, which sits at main's exact size.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): craft keeps its own 8 fps grid; comments back within the checks
Review on #5651 found that the merge moved craft onto the overlap pass's
times, which main caps at 180, so a brief line in a long film fell between
samples and line_unread missed it. Craft now builds its own grid at 8 fps up to
600 samples, as before the merge, and a 60 s film with a line shown for 0.2 s
pins it.
The Comments check passes again: REVIEW.md and STORYLINES.md join the files a
skill writes into a project, the comment lines this branch added come back
under each file's share at the merge-base, and the audio preset comment drops
a word the check reads as history.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
532 lines
27 KiB
Plaintext
532 lines
27 KiB
Plaintext
---
|
||
title: "Audio effects implementation"
|
||
sidebarTitle: "Audio effects"
|
||
description: "The four audio attributes, every effect and parameter, automation targets, the voiceover carve, audio groups, and how preview and render stay identical."
|
||
---
|
||
|
||
Use this reference when Studio controls are not enough: you need to write a mix
|
||
into HTML, know a parameter's exact range, automate a knob, or understand why
|
||
something behaves differently in the render than in preview. For everyday use
|
||
start with [Mix audio and apply effects](/studio/audio-effects), and for what to
|
||
ask an agent for, [Audio effects and mixing](/prompting/audio-effects).
|
||
|
||
A composition carries its whole mix in the HTML. There is no session file and
|
||
nothing to load beside the markup.
|
||
|
||
## The four attributes
|
||
|
||
| Attribute | Holds | Shape | Goes on | Read at playback |
|
||
| --- | --- | --- | --- | --- |
|
||
| `data-fx-chain` | The effects, in signal order | JSON | `<audio>`, `<video>`, `<hf-audio-group>` | Yes |
|
||
| `data-automation` | Envelopes on volume or an effect parameter | JSON | `<audio>`, `<video>`, `<hf-audio-group>` | Yes |
|
||
| `data-fx-carve` | A voiceover carve's own settings | JSON | `<audio>`, `<video>` | No — see [below](#data-fx-carve) |
|
||
| `data-audio-group` | The id of the group this clip belongs to | A plain id, not JSON | `<audio>`, audible `<video>` | Yes |
|
||
|
||
`data-audio-group` is the odd one: a bare string, and **ignored on a muted
|
||
`<video>`** (`muted`, or `data-has-audio="false"`). The other three are JSON. Of those, only `data-fx-chain` and `data-automation`
|
||
can also sit on an `<hf-audio-group>`, where they apply to the group's whole bus;
|
||
`data-fx-carve` belongs to a clip.
|
||
|
||
**Write the JSON attributes double-quoted, escaping the JSON's own quotes as
|
||
`"` and `&` as `&`** — as every example on this page does. The browser
|
||
reads them through `getAttribute` and does not mind either way, but
|
||
`carve.mjs` finds them with a `name="..."` regex, so a single-quoted
|
||
attribute is invisible to it: a carve reports no existing chain and overwrites
|
||
work it could not see.
|
||
|
||
Nothing static validates a chain. Preview plays an unreadable chain **dry** so
|
||
the composition stays workable; the render **fails the whole mix** rather than
|
||
shipping a dry track that sounds plausible and is wrong.
|
||
|
||
### `data-fx-chain`
|
||
|
||
```html
|
||
<audio
|
||
id="narration"
|
||
src="assets/vo.wav"
|
||
data-start="0"
|
||
data-duration="12"
|
||
data-fx-chain="{"version":1,"nodes":[
|
||
{"type":"highpass","id":"n1","label":"Remove Rumble",
|
||
"params":{"frequency":120,"q":0.707,"poles":"2"}},
|
||
{"type":"peaking","id":"n2","label":"Reduce Mud",
|
||
"params":{"frequency":250,"gain":-3,"q":1.2}},
|
||
{"type":"limiter","id":"n3","enabled":false,
|
||
"params":{"limit":-1,"attack":5,"release":50,"level_out":0}}
|
||
]}"
|
||
></audio>
|
||
```
|
||
|
||
Unescaped, that chain reads: a high-pass labelled Remove Rumble, a peaking cut
|
||
labelled Reduce Mud, and a bypassed limiter.
|
||
|
||
- **Order is signal order.** Each node processes what the one before produced.
|
||
- `type` is an effect id from [the registry](#effect-registry). `params` are in
|
||
the units a person thinks in — dB, ms, Hz. Out-of-range values are clamped on
|
||
read, so a chain that parses is always safe to realise.
|
||
- `id` is a stable handle. Automation addresses nodes by id, never by position,
|
||
so reordering a chain cannot re-point a lane at a different effect. A node
|
||
with no id loads fine but cannot be automated. Studio hands out the first free
|
||
`n1`, `n2`, …
|
||
- `label` replaces the effect's own name in the rack. Write one whenever a node
|
||
is doing a named job — two `peaking` nodes otherwise show the same row twice
|
||
and you cannot tell the mud cut from the clarity lift.
|
||
- `enabled: false` is bypass: the node stays in the chain, out of the signal
|
||
path. Absent means enabled.
|
||
- `fromCarve: true` marks a node the carve analysis generated. `fromPreset`
|
||
carries the id of the preset that wrote the node, which is how re-applying a
|
||
preset replaces its own nodes in place. **Do not set `fromCarve` by hand** —
|
||
the next carve deletes exactly those nodes.
|
||
|
||
### `data-automation`
|
||
|
||
```html
|
||
data-automation="{"version":1,"lanes":[
|
||
{"target":"volume","points":[{"t":0,"v":1},{"t":2.5,"v":0.4}]},
|
||
{"target":"fx.n2.gain","points":[{"t":0,"v":0},{"t":1,"v":-6,"curve":0.4}]}
|
||
]}"
|
||
```
|
||
|
||
That is two lanes: the track's volume dropping to 0.4 by 2.5 s, and node `n2`'s
|
||
gain moving to −6 dB by 1 s along a bent segment.
|
||
|
||
| Field | Meaning |
|
||
| --- | --- |
|
||
| `target` | `volume` for the track's own level, or `fx.<nodeId>.<param>` |
|
||
| `t` | Seconds — on **which clock depends on where the lane lives**, see below |
|
||
| `v` | The parameter's own unit — dB for a gain, Hz for a frequency, a linear multiplier for volume |
|
||
| `curve` | −1 to 1, bends the segment *leaving* a point; positive holds low then rises late |
|
||
| `viaX` / `viaY` | An interior point the segment passes through (progress 0–1, value travelled 0–1); supersedes `curve` when both are present |
|
||
|
||
### Two clocks
|
||
|
||
| Lane on | `t: 0` means |
|
||
| --- | --- |
|
||
| A clip (`<audio>` / `<video>`) | The start of **that clip**. A bed with `data-start="8"` has `t: 0` at composition time 8 |
|
||
| A group (`<hf-audio-group>`) | **Composition time.** A group has no `data-start`, so its clock starts at zero |
|
||
|
||
Getting this backwards is the most expensive mistake on the page: a group lane
|
||
written in clip-relative time lands wherever the first member happens to start.
|
||
Preview and render agree on both clocks.
|
||
|
||
### Volume is not capped at 1
|
||
|
||
`v` on a `volume` lane is a linear gain multiplier with a ceiling of **+12 dB**,
|
||
which is about `3.981` — the same ceiling `data-volume` uses. A lane that boosts
|
||
above unity is valid and will play; `1` is unity, not the maximum.
|
||
|
||
A lane holds its first value backwards to the start of its clip and its last
|
||
value forward to the end. **A bed that begins before the voice therefore needs an
|
||
explicit "no cut" point at `t: 0`, or it starts out already ducked.** Maximum 512
|
||
points per lane. A lane whose node is gone is pruned on read rather than
|
||
erroring — so a mistyped `nodeId` costs you the envelope silently. Read ids back
|
||
out of the chain rather than assuming what was minted.
|
||
|
||
A `volume` lane and a GSAP tween on `volume` conflict: the lane wins and the
|
||
tween is ignored. The linter reports that as `audio_volume_double_automation`.
|
||
|
||
### `data-fx-carve`
|
||
|
||
```html
|
||
data-fx-carve="{"enabled":true,"sources":["voiceover"],"strength":0.25}"
|
||
```
|
||
|
||
Unescaped: `{"enabled":true,"sources":["voiceover"],"strength":0.25}`.
|
||
|
||
- `sources` names **what this bed makes room for** — element ids, or a group id,
|
||
which expands to its current members on every analysis.
|
||
- `strength` is 0–1 and derives the whole mechanism.
|
||
- `enabled: false` keeps the settings and stops the carve. It exists because a
|
||
bed with exactly one candidate voice is carved by default; with "off" as an
|
||
absent attribute, switching it off would read as never-configured and the
|
||
default would put it back.
|
||
|
||
This attribute is **not read at playback** — the chain and lanes it produced are
|
||
what play. It exists so strength can be changed on an existing carve instead of
|
||
being guessed back out of the filters.
|
||
|
||
Older projects may carry six mechanism numbers (`maxCutDb`, `bands`, `q`,
|
||
`intelligibilityBias`, `duckDb`, `headroomDb`) instead of `strength`. They still
|
||
load: the depth maps back onto a strength and the rest is re-derived. A stored
|
||
carve with no `enabled` reads as on, a single `source` reads as a one-voice
|
||
`sources` list, and a stored `dynamic` is ignored — every carve follows the
|
||
speech now.
|
||
|
||
### `data-audio-group`
|
||
|
||
```html
|
||
<style>hf-audio-group { display: none !important; }</style>
|
||
|
||
<hf-audio-group id="voiceover" data-label="Voiceover" data-volume="1"></hf-audio-group>
|
||
|
||
<audio id="vo-intro" data-audio-group="voiceover" src="assets/vo1.wav" …></audio>
|
||
<audio id="vo-middle" data-audio-group="voiceover" src="assets/vo2.wav" …></audio>
|
||
```
|
||
|
||
Membership is held by the **member**, not by the group nesting its members, so a
|
||
deleted clip simply disappears from the group on the next resolve and nothing
|
||
dangles.
|
||
|
||
- `<hf-audio-group>` is optional metadata: `data-label`, `data-volume`,
|
||
`data-hidden`, and its own `data-fx-chain` / `data-automation`. A group with
|
||
members but no element still resolves, using its id as the label.
|
||
- **Audio and audible video.** A `<video>` with sound (`data-has-audio="true"`,
|
||
no `muted`) joins its group in preview and render; on a muted video
|
||
`data-audio-group` is ignored. Groups do not nest.
|
||
- `data-audio-group=""` is no group at all.
|
||
- The element must be inert. The runtime injects
|
||
`hf-audio-group{display:none!important}` so an unknown custom element cannot
|
||
take a flex gap or shift `:nth-child` — but author the rule yourself so a bare
|
||
preview never lays it out.
|
||
- Group ids and element ids share one namespace.
|
||
|
||
### The master bus: `data-fx-chain` on the composition root
|
||
|
||
```html
|
||
<div id="root" data-composition-id="main" data-width="1920" data-height="1080"
|
||
data-fx-chain="{"version":1,"nodes":[
|
||
{"type":"truepeak","id":"m1","label":"Delivery ceiling",
|
||
"params":{"ceiling":-1.5,"lookahead":3,"release":80}}
|
||
]}"
|
||
>
|
||
```
|
||
|
||
The master bus is the `data-fx-chain` on the top-level composition root. It runs
|
||
once over the sum of every clip and every group, after each group's own chain and
|
||
fader, so a limiter here sees the peak of the whole mix. A clip can belong to
|
||
only one group, so this is the way to put one effect across everything.
|
||
|
||
- Only the **top-level** composition root is read as the master bus. On a clip or
|
||
a group `data-fx-chain` keeps its own meaning, and on a sub-composition's root
|
||
it is not applied while a parent composition contains it. A sub-composition file
|
||
previewed or rendered on its own is the top-level root, so its chain is the
|
||
master bus there.
|
||
- There is no master automation: `data-automation` on the root is not read as
|
||
the master's.
|
||
- The render sums the tracks to a float WAV, runs the chain, and only then
|
||
encodes AAC, so the chain reads unclipped samples. Preview inserts the chain
|
||
between the clips' sum and the monitor.
|
||
- With a master chain on the root, preview plays every audible `<video>` through
|
||
that bus too, as the render does. Without one, a video at volume 1 with no
|
||
chain, group or automation of its own stays on native output.
|
||
- An unreadable master chain **fails the whole mix**: the render stops with an
|
||
`audio_processing_failed` error under every strictness setting, and exports
|
||
nothing. Preview plays dry with no message, and no lint rule checks a chain's
|
||
contents, so one typo on the root blocks every export of that composition.
|
||
Check a master chain in preview before rendering.
|
||
|
||
## Effect registry
|
||
|
||
Sixteen effects. Values outside a range are clamped on read. **AUTO** marks a
|
||
parameter an automation lane can drive; anything unmarked cannot move over time.
|
||
`HF_AUDIO_FX` in `@hyperframes/core/audio-fx` is the source of truth.
|
||
|
||
### Filter — which frequencies a track may occupy
|
||
|
||
| Effect | Parameters |
|
||
| --- | --- |
|
||
| `highpass` | `frequency` 20–20000 Hz (300, log) **AUTO** · `q` 0.1–20 (0.707, log) **AUTO** · `poles` `1`\|`2` (2) |
|
||
| `lowpass` | `frequency` 100–20000 Hz (8000, log) **AUTO** · `q` 0.1–20 (0.707, log) **AUTO** · `poles` `1`\|`2` (2) |
|
||
| `peaking` | `frequency` 20–20000 Hz (1000, log) **AUTO** · `gain` −40–40 dB (0) **AUTO** · `q` 0.1–20 (1, log) **AUTO** |
|
||
| `lowshelf` | `frequency` 20–2000 Hz (200, log) **AUTO** · `gain` −40–40 dB (0) **AUTO** |
|
||
| `highshelf` | `frequency` 500–20000 Hz (4000, log) **AUTO** · `gain` −40–40 dB (0) **AUTO** |
|
||
|
||
`q` is bandwidth — higher is narrower. `poles` is the slope: `2` is the usual
|
||
biquad (12 dB/oct), `1` is gentler (6 dB/oct). Shelving filters have no `q`; the
|
||
Web Audio spec leaves it unused for them.
|
||
|
||
### Dynamics — how level behaves over time
|
||
|
||
| Effect | Parameters |
|
||
| --- | --- |
|
||
| `gain` | `gain` −60–12 dB (0) **AUTO** |
|
||
| `compressor` | `threshold` −60–0 dB (−24) · `ratio` 1–20 (4) · `attack` 0.01–2000 ms (20, log) · `release` 0.01–9000 ms (250, log) · `knee` 1–8 (2.83) · `makeup` 0–36 dB (0) · `mix` 0–1 (1) |
|
||
| `limiter` | `limit` −24–0 dB (−1) · `attack` 0.1–80 ms (5) · `release` 1–8000 ms (50, log) · `level_out` −24–24 dB (0) |
|
||
| `truepeak` | `ceiling` −24–0 dBTP (−1) · `lookahead` 0.5–10 ms (3) · `release` 10–2000 ms (80, log) |
|
||
| `gate` | `threshold` −80–0 dB (−35) · `range` −80–0 dB (−24) · `ratio` 1–20 (10) · `attack` 0.01–9000 ms (1, log) · `release` 0.01–9000 ms (100, log) · `knee` 1–8 (2.83) |
|
||
|
||
Cuts on `gain` reach −60 dB, boosts stop at +12: it is a level stage for making
|
||
room, and a chain that could add 40 dB would clip long before that was useful.
|
||
`knee` of 1 is a hard corner. `mix` below 1 blends the dry signal back in
|
||
(parallel compression). `range` is how far down the gate pulls when closed.
|
||
|
||
`limiter` is an envelope follower: it reacts to the level of the signal, has no
|
||
lookahead, and can let a sample peak or an inter-sample peak through. `truepeak`
|
||
is the one to use for a delivery ceiling. It estimates the true peak on a 4x
|
||
oversampled signal, reads `lookahead` ms ahead so the gain is already down when
|
||
the peak arrives, and holds that estimate at `ceiling`. Channels share one
|
||
gain. It delays its output by `lookahead` plus about 0.3 ms (3.3 to 10.3 ms across
|
||
the range): the offline render trims that delay, live preview plays it, so the
|
||
sound is that much behind the picture. A seek clears the limiter's delay line and
|
||
held gain on a group or master bus. Changing `lookahead` rebuilds the node.
|
||
|
||
The ceiling is held against that 4x estimate, not against the waveform itself. A
|
||
4x interpolator cannot see the peaks of content close to Nyquist, so the true peak
|
||
can end above `ceiling`. Measured against an ideal 16x interpolation, tones, pink
|
||
noise and a dense mix ended at most 0.6 dB over. Full-band white noise driven 8 dB
|
||
or more into the limiter ended 1.2 to 1.7 dB over (median 1.4 dB, 28 runs of 10
|
||
to 30 s). A meter that also reads at 4x, such as ffmpeg's `ebur128=peak=true`,
|
||
reports that same noise about 0.1 dB over, so it will not show the miss. Under a
|
||
hard delivery limit set `ceiling` at least 2 dB below it: −3 dBTP for a −1 dBTP
|
||
limit.
|
||
|
||
### Nonlinear — changes the waveform's shape
|
||
|
||
| Effect | Parameters |
|
||
| --- | --- |
|
||
| `saturate` | `type` `tanh`\|`atan`\|`cubic`\|`exp`\|`alg`\|`quintic`\|`sin`\|`erf`\|`hard` (tanh) · `threshold` −40–0 dB (−6) · `output` −24–24 dB (0) **AUTO** · `oversample` 1–8× (4) |
|
||
| `bitcrush` | `bits` 1–32 (8) · `samples` 1–250× (1) · `mix` 0–1 (1) |
|
||
|
||
`tanh` is the gentlest curve, `hard` is outright clipping. Higher `oversample`
|
||
costs more CPU and keeps aliasing down. `samples` repeats each sample N times — a
|
||
crude downsample, which is where the lo-fi character comes from.
|
||
|
||
### Time — where a track sits, and how it moves
|
||
|
||
| Effect | Parameters |
|
||
| --- | --- |
|
||
| `delay` | `time` 1–5000 ms (250, log) **AUTO** · `feedback` 0.01–0.95 (0.35) **AUTO** · `mix` 0–1 (0.4) **AUTO** |
|
||
| `reverb` | `size` 0.05–1 (0.7) · `damping` 0–1 (0.5) · `wet` 0–1 (0.35) **AUTO** · `dry` 0–1 (0.7) **AUTO** |
|
||
| `chorus` | `delay` 1–100 ms (7) **AUTO** · `depth` 0–10 ms (2) **AUTO** · `speed` 0.01–10 Hz (1) **AUTO** · `mix` 0–1 (0.5) **AUTO** |
|
||
| `phaser` | `in_gain` 0–1 (0.4) **AUTO** · `out_gain` 0–2 (0.74) **AUTO** · `delay` 0.1–5 ms (3) · `decay` 0–0.99 (0.4) · `speed` 0.1–2 Hz (0.5) **AUTO** · `type` `0`\|`1` (0) |
|
||
| `pitchshift` | `semitones` −12–12 st (0, whole steps) · `mix` 0–1 (1) |
|
||
|
||
`pitchshift` moves pitch without changing playback speed, which is what
|
||
separates it from a playback-rate change. Neither of its parameters can be
|
||
automated — it is a worklet.
|
||
|
||
Reverb convolves a *generated* impulse and both preview and render generate the
|
||
same one, so a room is reproducible without shipping an impulse file. Higher
|
||
`damping` rolls the top off the tail faster. `feedback` is bounded below 1
|
||
because at 1 it never decays.
|
||
|
||
## Why some parameters cannot be automated
|
||
|
||
Automation is handed to the audio thread once, as native `AudioParam` ramps and
|
||
curves — that is what keeps it sample-accurate and identical between preview and
|
||
render. A parameter can therefore only be automated if an `AudioParam` backs it.
|
||
Three kinds do not:
|
||
|
||
| Kind | Effects | Consequence |
|
||
| --- | --- | --- |
|
||
| Worklet processor options | `compressor`, `limiter`, `truepeak`, `gate`, `bitcrush`, `pitchshift` | **No parameter is automatable at all** |
|
||
| A WaveShaper curve | `saturate`'s `type`, `threshold`, `oversample` | Only its `output` stage is a real param |
|
||
| A convolution impulse | `reverb`'s `size`, `damping` | `wet` / `dry` are gain stages and automate fine |
|
||
|
||
**A lane on a non-automatable parameter is silently inert.** To make one of those
|
||
behave differently over time, automate a `gain` stage around it instead: a lane
|
||
on a `gain` before a compressor changes how hard the compressor is driven, which
|
||
is most of what automating its threshold would have done.
|
||
|
||
## Presets, jobs, and one-knob profiles
|
||
|
||
Every one of these is a shortcut to a chain you could have built by hand. Open
|
||
any of them and you find ordinary effects with their parameters showing.
|
||
|
||
### Presets
|
||
|
||
Twenty-two, in four families. Applying one **appends**; re-applying one already
|
||
present replaces its own nodes in place, because position is signal order.
|
||
|
||
| Family | Ids |
|
||
| --- | --- |
|
||
| Voice | `voice-clean`, `voice-broadcast`, `voice-warm` |
|
||
| Repair | `rumble-cut`, `room-gate`, `boom-tame`, `harsh-tame` |
|
||
| Character | `telephone`, `radio-am`, `megaphone`, `lofi-tape`, `pa-system`, `intercom`, `doofus-worble`, `chipmunk`, `giant`, `monster` |
|
||
| Space | `room-tight`, `room-natural`, `hall`, `slap-echo`, `dub-throw` |
|
||
|
||
`chipmunk`, `giant`, and `monster` are built on `pitchshift`, so they change the
|
||
speaker rather than the channel.
|
||
|
||
`voice-clean` is Remove Rumble → Reduce Mud → Even Out Loudness → Add Clarity →
|
||
Peak Ceiling. `voice-broadcast` is denser and more forward; `voice-warm` adds
|
||
body rather than cutting it.
|
||
|
||
A preset's nodes are wrapped in a wet/dry blend, so `presetAmount` (0–1) fades
|
||
the whole thing, and `fx.preset.<id>` is an automation target that ramps it over
|
||
time. That is the only way to automate a preset as a unit — its nodes share no
|
||
common parameter.
|
||
|
||
### Jobs
|
||
|
||
Five named `peaking` filters with the frequency already chosen. Picking the job
|
||
is picking the range, which is what makes a single "how much" knob honest.
|
||
|
||
| Job | Symptom | Sets |
|
||
| --- | --- | --- |
|
||
| Tame Boominess | Too much chest — it booms | 200 Hz, −4 dB, Q 1.4 |
|
||
| Reduce Mud | Muffled, like it is behind cardboard | 250 Hz, −3 dB, Q 1.2 |
|
||
| Reduce Boxiness | Sounds like a small room, or a box | 400 Hz, −3 dB, Q 1.4 |
|
||
| Add Clarity | Words are hard to make out | 3 kHz, +2.5 dB, Q 1 |
|
||
| Soften Harshness | Harsh and tiring to listen to | 3.2 kHz, −3 dB, Q 1.6 |
|
||
|
||
Writing one by hand, carry the name in `label` — the parameters alone are not
|
||
the job.
|
||
|
||
**Every job also ships inside a preset at identical settings**, which is where
|
||
the five came from. So check what a preset already contains before adding a job
|
||
on top: `voice-clean` plus a Reduce Mud job is −6 dB at 250 Hz where −3 was
|
||
meant.
|
||
|
||
### One-knob profiles
|
||
|
||
Five effects have no single parameter that can honestly be their face — a
|
||
compressor's threshold means nothing without its ratio. They get a derived
|
||
control instead, 0–1, setting several parameters together.
|
||
|
||
| Effect | Knob | 0 → 1 | Sets |
|
||
| --- | --- | --- | --- |
|
||
| `compressor` | Evenness | Barely touched → very even, quite squashed | threshold, ratio, attack, release, makeup |
|
||
| `gate` | Tightness | Only true silence → cuts quiet words too | threshold, range, release |
|
||
| `saturate` | Warmth | Just a sheen → openly distorted | threshold, output |
|
||
| `reverb` | Space | A small tight room → a big open hall | size, wet, dry |
|
||
| `bitcrush` | Crush | Slightly gritty → destroyed | bits, samples, mix |
|
||
|
||
Evenness, Warmth, and Space are **level-matched**: the make-up gain, output trim,
|
||
and dry leg move with the drive, so turning the knob up does not also turn the
|
||
track up. Tightness and Crush are not, because neither has a trim to move.
|
||
|
||
The chain stores the mechanism values, not the knob position — the knob is read
|
||
back by inverting the curve, so hand-editing a parameter under a profile is
|
||
allowed and simply moves the knob.
|
||
|
||
## Voiceover carve
|
||
|
||
A carve is a **relationship, not an effect**. The settings live on the *bed* —
|
||
the track that gets processed — and name the voices to listen to, exactly as a
|
||
sidechain compressor does. Never put a carve on a voice track.
|
||
|
||
`strength` derives six numbers that move together in any real mix: how deep to
|
||
cut, how many bands, how wide, how far to favour intelligibility over raw voice
|
||
energy, how far the level may drop, and how far under the voice to aim.
|
||
|
||
| Strength | Result |
|
||
| --- | --- |
|
||
| `0` | Spectral only — one band, no level match at all |
|
||
| `0.25` (default) | A 6 dB dip in three bands with 6 dB of level room |
|
||
| `0.5` | The dip reaches 10 dB — where a carve starts being heard as an effect |
|
||
| Above `0.5` | Deliberate territory for a loud bed under a quiet voice |
|
||
|
||
Every value becomes an envelope of the speech's own level: silence leaves the bed
|
||
alone, a loud passage pushes the carve to full depth. There is no static mode.
|
||
|
||
Inside a carved bed the signal runs through the dips first, then the level match,
|
||
then anything you built yourself — which is why a limiter you add still acts as
|
||
the last ceiling.
|
||
|
||
Voices are summed onto the bed's clock before anything is measured, so one
|
||
analysis covers all of them. Voices that never play while the bed does are left
|
||
out; they cannot mask it.
|
||
|
||
**Name a group, not a run of clip ids.** A list of ids has to be exhaustively
|
||
right and stays right only until the next edit — a fourth narration clip added
|
||
later plays outside the carve's awareness and the bed silently fails to duck
|
||
under it. A `sources` list naming two or more plain clip ids is reported as
|
||
`audio_carve_ungrouped_sources`.
|
||
|
||
There are two arrangements where naming the group is **worse** than naming ids,
|
||
and both come from the group form resolving later and wider than the analysis
|
||
that wrote it:
|
||
|
||
| Arrangement | What would happen |
|
||
| --- | --- |
|
||
| The bed is itself a member of the voice group | The bed is handed to itself as a voice and carved against its own content |
|
||
| The group also holds a member classified music or SFX | That member enters the sidechain on a later analysis, ducking the bed under a whoosh |
|
||
|
||
Neither shows up on the run that writes it — the analysis sums the voices it
|
||
detected, so the first pass is correct however wrong the stored attribute is. It
|
||
surfaces on the next re-analysis, against a file the previous run declared good.
|
||
Keep the bed in its own group.
|
||
|
||
What a carve writes is an ordinary chain of `peaking` filters plus a `gain`
|
||
stage, tagged `fromCarve`, and one automation lane per carved parameter. That
|
||
tagging is the whole trick: a re-run replaces the previous carve and leaves every
|
||
effect and lane you built by hand exactly where it was.
|
||
|
||
### From the command line
|
||
|
||
The script ships with the `media-use` skill; `<SKILL_DIR>` is that skill's folder.
|
||
|
||
```bash
|
||
node <SKILL_DIR>/audio/scripts/carve.mjs --comp index.html
|
||
```
|
||
|
||
That is the whole command — it finds the voice and the bed itself and prints what
|
||
it decided:
|
||
|
||
```text
|
||
bed music-bed (name looks like music)
|
||
voice narration (only track left)
|
||
carve strength 0.25
|
||
bands 400Hz -6dB q1.4, 1000Hz -3dB q1.4, 1600Hz -3.17dB q1.4
|
||
level 216-point envelope, floor -6 dB
|
||
```
|
||
|
||
Name tracks with `--bed` / `--voice` (repeatable) when the automatic choice is
|
||
wrong, `--strength` to push it, `--dry-run` to see the report and write nothing.
|
||
It refuses when it cannot tell which track is the bed rather than carving the
|
||
wrong one.
|
||
|
||
**What it records in `sources`.** When every voice it analysed shares exactly one
|
||
group, and that group is safe to name, it writes the group id. It falls back to
|
||
the individual clip ids — and says so — when the group contains the bed or a
|
||
member classified music or SFX, for the reasons in the table above. A group
|
||
member classified voice or unknown that this run did not analyse is *not* a
|
||
reason to fall back: picking up a clip that starts playing later is the whole
|
||
point of naming the group.
|
||
|
||
Track choice is by name first, using the same classifier Studio's picker uses, so
|
||
the two cannot disagree: an id or filename that looks like music (`music`, `bgm`,
|
||
`bed`, `score`…) is the bed, and everything else playing over it that is not
|
||
SFX-shaped is a voice. Needs `ffmpeg` on `PATH` and `@hyperframes/core`
|
||
installed in the project.
|
||
|
||
## Groups, mute, and solo
|
||
|
||
A group sums its members through one bus, and the bus carries the group's own
|
||
volume, FX chain, and automation. In the render each group is sub-mixed into a
|
||
single track at full composition length before entering the main mix, so a
|
||
group-level effect hears the sum rather than each member separately.
|
||
|
||
**Mute and solo are not symmetric, on purpose.**
|
||
|
||
| Control | Where it lives | Reaches the export |
|
||
| --- | --- | --- |
|
||
| Mute | `data-hidden` on the group element or the clip | **Yes** — muted members are dropped from the rendered mix |
|
||
| Solo ("hear only this") | Studio session state | **No** — it never reaches the engine |
|
||
|
||
Solo is a listening tool. An element is audible while any solo is active only if
|
||
it is itself soloed or its own group is soloed; a group bus is never itself
|
||
attenuated by solo, so a soloed member's path out stays open. A group that is not
|
||
soloed while one of its members is shows as half-lit — the display-only signal
|
||
that some of what is under it still plays.
|
||
|
||
## Preview and render
|
||
|
||
Both read the same two attributes through the same builders, which is why
|
||
preview predicts the render:
|
||
|
||
| | Preview | Render |
|
||
| --- | --- | --- |
|
||
| Context | Live `AudioContext` | `OfflineAudioContext` in the headless browser |
|
||
| Unreadable chain | Plays **dry**, so the composition stays workable | **Fails the whole mix** |
|
||
| Volume lane | Scheduled on the graph | Baked into the PCM by the mixer |
|
||
| Editing an attribute mid-playback | Picked up by a `MutationObserver` | n/a |
|
||
|
||
Effects with a tail — `reverb`, `delay` — make the rendered track **longer** than
|
||
its source, and the mix is told how much. A bed with reverb therefore no longer
|
||
ends exactly at its `data-duration`. That is expected.
|
||
|
||
## What the linter checks
|
||
|
||
Almost no static gate covers a mix. Three rules exist:
|
||
|
||
| Rule | Reports |
|
||
| --- | --- |
|
||
| `audio_carve_ungrouped_sources` | A carve naming two or more plain clip ids instead of a group |
|
||
| `audio_volume_double_automation` | A `volume` lane on a track that also has a GSAP tween on `volume` — the lane wins, the tween is ignored |
|
||
| `audio_volume_tween_overrides_gain` | An authored `data-volume` on a track whose `volume` is tweened — the tween's values are absolute and replace that gain rather than scaling it |
|
||
|
||
Nothing validates the chain or the effect lanes at all. What enforces those is
|
||
the render. Beyond that, a mix is verified by rendering and listening.
|