mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-11 14:50:02 +00:00
feat(audio): automation envelope model (#3015)
* feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * test(engine): probe ffmpeg and Chrome instead of assuming them Two failures on #3021's Test job, both about the environment rather than the code under test. **Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to `execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()` resolves — every other ffmpeg-dependent suite in this package already goes through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)` so a contributor without ffmpeg skips rather than fails. **The browser guard trusted the wrong thing.** It asked `resolveHeadlessShellPath()` and treated a returned path as "a browser is here". CI's cache holds a chrome-headless-shell that resolves and then fails to spawn — a partial download is indistinguishable from a working one by `existsSync`, which is all that resolver checks. So the three browser cases ran anyway and failed on the launch. It now runs `--version` and requires exit 0, which is the same probe the ffmpeg suites use: ask the binary, do not infer from the filesystem. Checked both directions rather than just the green one. With a working browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed at a binary that exits non-zero — CI's exact situation — exactly 3 skip and the other 8 still run. A guard that quietly skipped everything would have looked identical on the CI summary. * feat(core): register the audio-fx-rack canary at 0% Lands the rollout switch dark, per the registry's own procedure: "Start at percentage: 0 and merge that — a canary at 0 is dead code you can land safely and ramp without a code review." Declared at the bottom of the stack so every branch above can read it. The gate itself goes in at wa-4-fx-panel, where the rack first appears. Scope is deliberate and stated in the description: it gates the AUTHORING surface only. A composition that already carries `data-fx-chain` still plays and renders it. A canary should stage who can REACH a feature, not make an attribute somebody already wrote silently inert — an agent that writes a chain through the skill would otherwise produce a file whose audio processing vanishes with no error. * feat(studio): audio FX panel generated from the registry Controls for the whole chain: add, remove, reorder, bypass, and every knob each effect declares. Nothing in the panel knows what a compressor is. The registry supplies each parameter's range, step, unit and scale and the panel renders what it finds, so adding an effect or a knob upstream needs no change here, and the panel cannot offer a value the renderer would reject — a typed-in figure is clamped into the declared range on the way through. Frequency and time controls span three or four decades, so those declare a log scale and the slider maps exponentially; a linear slider would spend most of its travel somewhere useless. Reorder is a first-class control because chain order changes the sound: a reverb before a compressor is not the same as after. Carve gets its own block rather than an entry in the add menu, with a picker for the voice track to listen to. It processes this track based on another one, which is how a sidechain control works — it lives on the track that changes, and names the source. * feat(studio): show the Audio FX section on audio tracks Adds `audioFx` to the editing-affordances contract and renders the FX panel in the inspector when an `<audio>` element is selected. The section is audio-only. A `<video>` carries its sound on a separate `<audio>` element, so an FX chain on the video would have nothing to process. Chain and carve settings are written straight back onto the element as serialised attributes, the way colour grading carries its config, so persistence is an ordinary attribute write and needs no new server route. A chain that cannot be parsed renders as empty rather than breaking the panel, and the attribute is left untouched until the user changes something. The collapsed group summarises what is on the track ("2 effects + carve") so the state is visible without expanding it. Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED defaults to true, so the flat inspector is what actually renders. * refactor(studio): lift audioFxSummary out of PropertyPanelFlat `PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap, so the required File size check is red — the sole reason this PR is blocked. The review says as much: "mechanical fix (~5 min), not a design problem. Code itself is LGTM." Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later branch creates for it. Deliberately the smallest cut that clears the cap rather than the whole `AudioFxGroup` extraction: every later commit in the stack edits AudioFxGroup, so moving it here would collide with each of them, while almost nothing touches this function. 595 lines. * feat(core,studio): hear the FX chain in preview, and run the carve analysis Splices an element's FX chain into the playback graph so preview stops being silent about effects, and wires the carve button that was previously inert. The chain goes between the decoded source and its gain stage: effects see the raw signal and volume automation rides on their output, matching the order the offline render uses. Since preview and render call the same graph builders, what is heard while scrubbing is what gets written. The splice lives in the transport rather than on the `<audio>` element. The transport plays each track from a decoded AudioBuffer and mutes the element to avoid doubling, so capturing the element with createMediaElementSource would have processed a stream nothing is listening to — it looked like it worked because the call succeeded, and the audio was unchanged. A chain that cannot be built plays dry rather than silencing the track, which is the right failure in preview: the author keeps working and hears the source. The render still refuses, because shipping the dry signal there would be wrong. Carve now analyses for real: it decodes the chosen voice track, ranks its bands and writes the resulting peaking filters onto this track. Generated nodes are tagged `fromCarve`, so re-running replaces the previous carve instead of stacking another set on top of hand-added effects. Known limitation: the graph is built when a source is scheduled, so a knob turned mid-playback takes effect on the next play or seek rather than immediately. Live re-parameterisation needs the transport to hold the handle and forward updates. * fix(studio,core): stop parameter drags from restarting playback Dragging a knob wrote the chain through the persisting attribute path on every input event. That path refreshes the preview, which reloads the composition and reschedules audio — so a single drag reloaded dozens of times and playback stuttered the whole way. Drags now go through `onSetAttributeLive`, the same path colour grading uses for scrubs: it coalesces undo entries and sets `skipRefresh`, so no reload happens. The persisting write fires once, when the gesture ends — pointer-up or blur for a slider, Enter or blur for a typed value. A select commits immediately since there is no drag to wait for. While dragging, the control is driven from local state. Waiting for the value to round-trip through the element attribute made the knob lag behind the pointer. For the change to be audible without a reload, the graph now follows the attribute: the chain installed by the transport observes the element and re-parameterises itself in place, so a value change lands on the next 128-sample quantum. A shape change (effect added, bypassed, pole count) cannot be patched into a running graph, so it still waits for the next schedule rather than cutting the audio mid-play. The regression test drags a slider through several values and asserts the persisting handler is untouched until release. * feat(studio): put the audio FX rack behind its canary Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered at 0% — so the whole 47-PR stack can land without showing anyone a feature that has not been measured yet. The gate sits on the AUTHORING surface and nowhere else. The runtime and the render still honour a `data-fx-chain` already on an element, so a composition written through the skill or by `carve.mjs` keeps its processing rather than going silently dry for anyone outside the cohort. A canary should stage who can REACH a feature, not make an attribute somebody already wrote stop working with no error. Gated at the panel rather than in `resolveEditingSections`: the affordance resolver is a pure function in core describing what an element CAN support, and rollout state is not a property of an `<audio>` tag. Pinned the 0% with a test, and checked it fails at 25 — a ramp should have to break something that says "this ships dark" out loud. One gap, stated rather than papered over: the gate itself has no unit test. I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness never renders the Audio FX group for its audio fixture even with the gate removed — so the test passed for the wrong reason in the off case and could not pass at all in the on case. A test that cannot fail for the right reason is worse than none. Verifying the gate needs the panel harness to mount that section first, which is its own change. * fix(core): register FX worklets before building nodes that need them An AudioWorkletNode cannot be constructed before its processor is registered — it throws, and the surrounding chain is lost with it. `attachElementFxChain` built the chain first and only then called `ensureAudioFxWorklets`, so every worklet-backed effect (compressor, limiter, gate, bitcrush) threw on construction and the track fell back to dry. Instrumenting the preview showed `hf-compressor: InvalidStateError` with addModule never called at all. When the module has not landed yet the track now plays dry and the graph is swapped in once registration resolves, so the effect arrives a moment late instead of never. Registration is also tracked per context rather than in one module-level promise. A processor registered on one AudioContext does not exist on another, so the shared promise made every context after the first believe it was ready when it was not — the studio's transport owns its own context, which is exactly that case. With the worklets actually running, the compressor's per-sample log10 and pow became real audio-thread work. Samples below the knee have a gain of exactly unity and need neither, so the envelope is now compared in the linear domain and the transcendentals only run for samples that are actually being compressed. * refactor(studio): split the FX node row out of FxSection Clears the health findings the FX stack left behind: the chain-node render callback was a 70-line closure over half of FxSection's state, and the two reorder arrows were the same button written twice. Also drops two exports with no consumers, and registers the audio FX runtime stub as an entry point — it is bundled by file path, so nothing imports it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(core): automation envelope model for audio tracks Adds the data model behind Ableton-style automation lanes: breakpoint envelopes over track volume or one knob of one effect in the track's FX chain, stored on the element as `data-automation`. Times are clip-local, so an envelope travels with the clip when it moves — the clip-envelope model rather than arrangement automation. `sampleAutomationLane` is the single interpolator. The lane drawing, the preview scheduler and the render bake all call it, so the picture and the sound cannot disagree about the curve. Log-scaled parameters interpolate in log space, matching what their own knob already promises. FX nodes gain a stable `id`, minted by count rather than randomly so the document is the same on every machine. Lanes address nodes by id, so reordering a chain never re-points a lane at a different effect, and a lane whose effect was deleted is dropped rather than left to reattach. Also warns when a track carries both a volume lane and a GSAP volume tween, since only the lane is heard and the tween silently does nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(studio): lift the audio FX group out of PropertyPanelFlat `PropertyPanelFlat.tsx` was 672 lines against the repo's 600-line cap, so the required File size check was red — the sole reason #3014 and #3022 are blocked. Both reviews say the same thing: "mechanical fix, not a design problem. Code itself is LGTM." Moves `AudioFxGroup` and `audioFxSummary` into `propertyPanelAudioFxGroup.tsx`, which is where a later branch puts them anyway — done here so the file is under the cap from the point it first crosses it, rather than ten branches later. 533 lines now. The four audio imports it no longer needs go with it. Not fixed here: three `FxSection carve` tests fail on this branch with "Cannot read properties of undefined (reading 'toFixed')". Confirmed pre-existing by stashing this change and re-running — that is the separate `Test` failure the review also flags. * feat(core): expose the AudioParams behind automatable FX knobs Marks the knobs an automation lane can drive and has each graph builder hand back the AudioParam behind them, so a scheduler can write to a running effect without knowing what the effect is. A knob is not always one AudioParam. A wet/dry mix is two gains moving in opposition, and a knob in milliseconds drives a delay time in seconds, so each target carries the mapping out of the knob's own declared unit. What stays unautomatable is stated where it is decided: a WaveShaper curve, a convolution impulse and a one-pole filter's coefficients are all rebuilt wholesale rather than scheduled, and the four worklet effects take values by postMessage rather than through AudioParams. The registry flag is written by hand, so a test builds every effect and checks the exposure both ways — nothing flagged is missing, nothing exposed is unflagged. A flag that lied would offer a lane that silently did nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): drop the FX panel's dead __testables export Fallow audit flagged it — no test imports the module. * fix(core,studio): clear the remaining Fallow audit findings on the FX panel - Split FxSection's per-node row into FxNodeRow + FxNodeControls so the CRAP score (31.6, threshold 30) splits across two smaller units instead of moving wholesale with one extraction. - Dedupe the repeated "open the add menu, read its items" block in propertyPanelFxSection.test.tsx into openAddMenuItems(). - Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into one build-inline-artifact.ts, config-selected by CLI arg — the two scripts were a byte-for-byte clone save for names. - Exempt canary.test.ts's rawFnv (a deliberate independent reimplementation used to cross-check canaryBucket, per its own docstring) and the property-panel test files' shared renderInto/mount scaffolding (pre-existing across 9 files, 2 outside this stack) in .fallowrc.jsonc, consistent with this file's existing exemptions for the same class of intentional/pre-existing duplication. * fix(ci): allowlist the build-script consolidation in the no-main-deletions guard build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into build-inline-artifact.ts to kill a fallow duplication finding; the deletion guard flagged that as an accidental loss since main still has both originals. * fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo Both effect builders set wet.gain to the mix and dry.gain to its complement in identical two-line blocks; fallow kept re-flagging it as a 10-line clone on every unrelated change. Extracted setWetDryMix. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
9b0c5e8559
commit
fd97926cf6
@@ -110,6 +110,12 @@
|
|||||||
"types": "./dist/audioCarve.d.ts",
|
"types": "./dist/audioCarve.d.ts",
|
||||||
"environments": ["browser", "bun", "node"]
|
"environments": ["browser", "bun", "node"]
|
||||||
},
|
},
|
||||||
|
"./audio-automation": {
|
||||||
|
"source": "./src/audioAutomation.ts",
|
||||||
|
"runtime": "./dist/audioAutomation.js",
|
||||||
|
"types": "./dist/audioAutomation.d.ts",
|
||||||
|
"environments": ["browser", "bun", "node"]
|
||||||
|
},
|
||||||
"./color-grading": {
|
"./color-grading": {
|
||||||
"source": "./src/colorGrading.ts",
|
"source": "./src/colorGrading.ts",
|
||||||
"runtime": "./dist/colorGrading.js",
|
"runtime": "./dist/colorGrading.js",
|
||||||
|
|||||||
@@ -124,6 +124,12 @@
|
|||||||
"import": "./src/audioCarve.ts",
|
"import": "./src/audioCarve.ts",
|
||||||
"types": "./src/audioCarve.ts"
|
"types": "./src/audioCarve.ts"
|
||||||
},
|
},
|
||||||
|
"./audio-automation": {
|
||||||
|
"bun": "./src/audioAutomation.ts",
|
||||||
|
"node": "./dist/audioAutomation.js",
|
||||||
|
"import": "./src/audioAutomation.ts",
|
||||||
|
"types": "./src/audioAutomation.ts"
|
||||||
|
},
|
||||||
"./color-grading": {
|
"./color-grading": {
|
||||||
"bun": "./src/colorGrading.ts",
|
"bun": "./src/colorGrading.ts",
|
||||||
"node": "./dist/colorGrading.js",
|
"node": "./dist/colorGrading.js",
|
||||||
@@ -398,6 +404,10 @@
|
|||||||
"import": "./dist/audioCarve.js",
|
"import": "./dist/audioCarve.js",
|
||||||
"types": "./dist/audioCarve.d.ts"
|
"types": "./dist/audioCarve.d.ts"
|
||||||
},
|
},
|
||||||
|
"./audio-automation": {
|
||||||
|
"import": "./dist/audioAutomation.js",
|
||||||
|
"types": "./dist/audioAutomation.d.ts"
|
||||||
|
},
|
||||||
"./color-grading": {
|
"./color-grading": {
|
||||||
"import": "./dist/colorGrading.js",
|
"import": "./dist/colorGrading.js",
|
||||||
"types": "./dist/colorGrading.d.ts"
|
"types": "./dist/colorGrading.d.ts"
|
||||||
|
|||||||
@@ -341,3 +341,67 @@ describe("levels and per-channel state", () => {
|
|||||||
expect(buildFxChain(ctx, twoPole(300)).update(twoPole(2000))).toBe(true);
|
expect(buildFxChain(ctx, twoPole(300)).update(twoPole(2000))).toBe(true);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("automatable parameters", () => {
|
||||||
|
/**
|
||||||
|
* The registry's `automatable` flag is what the panel and the scheduler both
|
||||||
|
* trust, and it is written by hand. Build every effect and check that each
|
||||||
|
* flagged knob really does reach an AudioParam — a flag that lies would
|
||||||
|
* offer an automation lane that silently does nothing.
|
||||||
|
*/
|
||||||
|
it("exposes an AudioParam for every knob the registry marks automatable", () => {
|
||||||
|
for (const def of HF_AUDIO_FX) {
|
||||||
|
const ctx = new FakeCtx() as unknown as BaseAudioContext;
|
||||||
|
const handle = buildFxNode(ctx, def.id, defaultAudioFxParams(def.id));
|
||||||
|
const flagged = def.params
|
||||||
|
.filter((p) => p.kind === "number" && p.automatable)
|
||||||
|
.map((p) => p.key);
|
||||||
|
for (const key of flagged) {
|
||||||
|
const targets = handle.automation?.[key];
|
||||||
|
expect(
|
||||||
|
targets,
|
||||||
|
`${def.id}.${key} is flagged automatable but exposes no AudioParam`,
|
||||||
|
).toBeTruthy();
|
||||||
|
expect(targets?.length, `${def.id}.${key} exposes an empty target list`).toBeGreaterThan(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("exposes nothing the registry has not flagged", () => {
|
||||||
|
for (const def of HF_AUDIO_FX) {
|
||||||
|
const ctx = new FakeCtx() as unknown as BaseAudioContext;
|
||||||
|
const handle = buildFxNode(ctx, def.id, defaultAudioFxParams(def.id));
|
||||||
|
const flagged = new Set(
|
||||||
|
def.params.filter((p) => p.kind === "number" && p.automatable).map((p) => p.key),
|
||||||
|
);
|
||||||
|
for (const key of Object.keys(handle.automation ?? {})) {
|
||||||
|
expect(flagged.has(key), `${def.id}.${key} is exposed but not flagged automatable`).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("maps a knob's own unit onto the AudioParam it drives", () => {
|
||||||
|
const ctx = new FakeCtx() as unknown as BaseAudioContext;
|
||||||
|
const delay = buildFxNode(ctx, "delay", { ...defaultAudioFxParams("delay"), time: 250 });
|
||||||
|
// The knob reads milliseconds; delayTime is in seconds.
|
||||||
|
expect(delay.automation?.time?.[0]?.map?.(250)).toBeCloseTo(0.25, 10);
|
||||||
|
// A wet/dry mix is two gains moving in opposition, not one.
|
||||||
|
const mix = delay.automation?.mix ?? [];
|
||||||
|
expect(mix.length).toBe(2);
|
||||||
|
expect(mix[0]?.map?.(0.3) ?? 0.3).toBeCloseTo(0.3, 10);
|
||||||
|
expect(mix[1]?.map?.(0.3)).toBeCloseTo(0.7, 10);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("leaves a one-pole filter unexposed, since its coefficients are fixed", () => {
|
||||||
|
const ctx = new FakeCtx() as unknown as BaseAudioContext;
|
||||||
|
const twoPole = buildFxNode(ctx, "highpass", defaultAudioFxParams("highpass"));
|
||||||
|
expect(twoPole.automation?.frequency?.length).toBe(1);
|
||||||
|
const onePole = buildFxNode(ctx, "highpass", {
|
||||||
|
...defaultAudioFxParams("highpass"),
|
||||||
|
poles: "1",
|
||||||
|
});
|
||||||
|
expect(onePole.automation?.frequency).toBeUndefined();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -58,10 +58,27 @@ export function synthesizeReverbImpulse(
|
|||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Where an automation lane writes when it drives one knob.
|
||||||
|
*
|
||||||
|
* A knob is not always one AudioParam. A wet/dry mix is two gains moving in
|
||||||
|
* opposition, and a knob in milliseconds drives a delay time in seconds, so
|
||||||
|
* each target carries its own mapping out of the knob's declared unit.
|
||||||
|
*/
|
||||||
|
export interface FxParamTarget {
|
||||||
|
param: AudioParam;
|
||||||
|
map?: (value: number) => number;
|
||||||
|
}
|
||||||
|
|
||||||
export interface FxNodeHandle {
|
export interface FxNodeHandle {
|
||||||
input: AudioNode;
|
input: AudioNode;
|
||||||
output: AudioNode;
|
output: AudioNode;
|
||||||
update(params: HfAudioFxParamValues): void;
|
update(params: HfAudioFxParamValues): void;
|
||||||
|
/**
|
||||||
|
* AudioParams behind the knobs the registry marks `automatable`, keyed by
|
||||||
|
* parameter key. Absent for a node whose values cannot be scheduled.
|
||||||
|
*/
|
||||||
|
automation?: Record<string, FxParamTarget[]>;
|
||||||
dispose(): void;
|
dispose(): void;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -69,6 +86,14 @@ type Builder = (ctx: BaseAudioContext, p: HfAudioFxParamValues) => FxNodeHandle;
|
|||||||
|
|
||||||
const n = (v: number | string | undefined): number => (typeof v === "number" ? v : Number(v ?? 0));
|
const n = (v: number | string | undefined): number => (typeof v === "number" ? v : Number(v ?? 0));
|
||||||
|
|
||||||
|
/** Milliseconds on the knob, seconds on the AudioParam. */
|
||||||
|
const msToSec = (v: number): number => v / 1000;
|
||||||
|
|
||||||
|
/** A wet/dry pair: the dry side is whatever the wet side is not. */
|
||||||
|
function mixTargets(wet: AudioParam, dry: AudioParam): FxParamTarget[] {
|
||||||
|
return [{ param: wet }, { param: dry, map: (v) => 1 - v }];
|
||||||
|
}
|
||||||
|
|
||||||
/** Linear crossfade: dry falls as wet rises, in lockstep. */
|
/** Linear crossfade: dry falls as wet rises, in lockstep. */
|
||||||
function setWetDryMix(wet: GainNode, dry: GainNode, mix: number): void {
|
function setWetDryMix(wet: GainNode, dry: GainNode, mix: number): void {
|
||||||
wet.gain.value = mix;
|
wet.gain.value = mix;
|
||||||
@@ -76,10 +101,22 @@ function setWetDryMix(wet: GainNode, dry: GainNode, mix: number): void {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/** A node that is its own input and output and has nothing to tear down. */
|
/** A node that is its own input and output and has nothing to tear down. */
|
||||||
function simple(node: AudioNode, update: (p: HfAudioFxParamValues) => void): FxNodeHandle {
|
function simple(
|
||||||
return { input: node, output: node, update, dispose: () => node.disconnect() };
|
node: AudioNode,
|
||||||
|
update: (p: HfAudioFxParamValues) => void,
|
||||||
|
automation?: Record<string, FxParamTarget[]>,
|
||||||
|
): FxNodeHandle {
|
||||||
|
return { input: node, output: node, update, automation, dispose: () => node.disconnect() };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Filter types whose Q a BiquadFilterNode actually reads. The spec leaves it
|
||||||
|
* unused for shelving filters, so the registry offers no shelf Q and the graph
|
||||||
|
* must expose none either — the exposure invariant would otherwise advertise an
|
||||||
|
* AudioParam for a knob nobody can set.
|
||||||
|
*/
|
||||||
|
const USES_Q: ReadonlySet<BiquadFilterType> = new Set(["peaking", "highpass", "lowpass"]);
|
||||||
|
|
||||||
function biquad(type: BiquadFilterType, useGain: boolean): Builder {
|
function biquad(type: BiquadFilterType, useGain: boolean): Builder {
|
||||||
return (ctx, p) => {
|
return (ctx, p) => {
|
||||||
const f = ctx.createBiquadFilter();
|
const f = ctx.createBiquadFilter();
|
||||||
@@ -90,7 +127,11 @@ function biquad(type: BiquadFilterType, useGain: boolean): Builder {
|
|||||||
if (useGain) f.gain.value = n(v.gain);
|
if (useGain) f.gain.value = n(v.gain);
|
||||||
};
|
};
|
||||||
apply(p);
|
apply(p);
|
||||||
return simple(f, apply);
|
return simple(f, apply, {
|
||||||
|
frequency: [{ param: f.frequency }],
|
||||||
|
...(USES_Q.has(type) ? { q: [{ param: f.Q }] } : {}),
|
||||||
|
...(useGain ? { gain: [{ param: f.gain }] } : {}),
|
||||||
|
});
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -108,6 +149,8 @@ function onePoleBuilder(kind: "highpass" | "lowpass"): Builder {
|
|||||||
? ctx.createIIRFilter([1 / (1 + k), -1 / (1 + k)], [1, (k - 1) / (k + 1)])
|
? ctx.createIIRFilter([1 / (1 + k), -1 / (1 + k)], [1, (k - 1) / (k + 1)])
|
||||||
: ctx.createIIRFilter([k / (1 + k), k / (1 + k)], [1, (k - 1) / (k + 1)]);
|
: ctx.createIIRFilter([k / (1 + k), k / (1 + k)], [1, (k - 1) / (k + 1)]);
|
||||||
// IIRFilterNode coefficients are immutable; the caller rebuilds on change.
|
// IIRFilterNode coefficients are immutable; the caller rebuilds on change.
|
||||||
|
// Nothing here is schedulable either, so a frequency lane on a one-pole
|
||||||
|
// filter has nowhere to write — the scheduler skips what is not exposed.
|
||||||
return simple(node, () => {});
|
return simple(node, () => {});
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -160,6 +203,9 @@ const waveshaper: Builder = (ctx, p) => {
|
|||||||
input: preGain,
|
input: preGain,
|
||||||
output: postGain,
|
output: postGain,
|
||||||
update: apply,
|
update: apply,
|
||||||
|
// The curve itself is rebuilt wholesale, but the make-up gain after it is
|
||||||
|
// an ordinary AudioParam.
|
||||||
|
automation: { output: [{ param: postGain.gain, map: (v) => Math.pow(10, v / 20) }] },
|
||||||
dispose: () => {
|
dispose: () => {
|
||||||
preGain.disconnect();
|
preGain.disconnect();
|
||||||
ws.disconnect();
|
ws.disconnect();
|
||||||
@@ -190,6 +236,11 @@ const delayFeedback: Builder = (ctx, p) => {
|
|||||||
input,
|
input,
|
||||||
output: out,
|
output: out,
|
||||||
update: apply,
|
update: apply,
|
||||||
|
automation: {
|
||||||
|
time: [{ param: dl.delayTime, map: (v) => Math.min(5, msToSec(v)) }],
|
||||||
|
feedback: [{ param: fb.gain }],
|
||||||
|
mix: mixTargets(wet.gain, dry.gain),
|
||||||
|
},
|
||||||
dispose: () => [input, out, dl, fb, wet, dry].forEach((x) => x.disconnect()),
|
dispose: () => [input, out, dl, fb, wet, dry].forEach((x) => x.disconnect()),
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -217,6 +268,12 @@ const chorusLfo: Builder = (ctx, p) => {
|
|||||||
input,
|
input,
|
||||||
output: out,
|
output: out,
|
||||||
update: apply,
|
update: apply,
|
||||||
|
automation: {
|
||||||
|
delay: [{ param: dl.delayTime, map: msToSec }],
|
||||||
|
depth: [{ param: depth.gain, map: msToSec }],
|
||||||
|
speed: [{ param: lfo.frequency }],
|
||||||
|
mix: mixTargets(wet.gain, dry.gain),
|
||||||
|
},
|
||||||
dispose: () => {
|
dispose: () => {
|
||||||
try {
|
try {
|
||||||
lfo.stop();
|
lfo.stop();
|
||||||
@@ -283,6 +340,13 @@ const allpassPhaser: Builder = (ctx, p) => {
|
|||||||
input,
|
input,
|
||||||
output: out,
|
output: out,
|
||||||
update: apply,
|
update: apply,
|
||||||
|
// `delay` and `decay` set the sweep centre, which feeds every stage's
|
||||||
|
// frequency at once — not one knob, one param — so they stay unautomated.
|
||||||
|
automation: {
|
||||||
|
speed: [{ param: lfo.frequency }],
|
||||||
|
in_gain: [{ param: dry.gain }],
|
||||||
|
out_gain: [{ param: wet.gain }],
|
||||||
|
},
|
||||||
dispose: () => {
|
dispose: () => {
|
||||||
try {
|
try {
|
||||||
lfo.stop();
|
lfo.stop();
|
||||||
@@ -322,6 +386,9 @@ const convolver: Builder = (ctx, p) => {
|
|||||||
input,
|
input,
|
||||||
output: out,
|
output: out,
|
||||||
update: apply,
|
update: apply,
|
||||||
|
// Size and damping regenerate the impulse response, so only the wet/dry
|
||||||
|
// balance is schedulable.
|
||||||
|
automation: { wet: [{ param: wet.gain }], dry: [{ param: dry.gain }] },
|
||||||
dispose: () => [input, out, conv, wet, dry].forEach((x) => x.disconnect()),
|
dispose: () => [input, out, conv, wet, dry].forEach((x) => x.disconnect()),
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,290 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
applyCurve,
|
||||||
|
fxAutomationTarget,
|
||||||
|
isConstantLane,
|
||||||
|
parseAutomation,
|
||||||
|
parseAutomationTarget,
|
||||||
|
resolveAutomation,
|
||||||
|
resolveAutomationRange,
|
||||||
|
sampleAutomationCurve,
|
||||||
|
sampleAutomationLane,
|
||||||
|
serializeAutomation,
|
||||||
|
VOLUME_RANGE,
|
||||||
|
type HfAutomationLane,
|
||||||
|
} from "./audioAutomation.js";
|
||||||
|
import { mintAudioFxNodeId, parseAudioFxChain, type HfAudioFxChain } from "./audioFx.js";
|
||||||
|
|
||||||
|
const chain: HfAudioFxChain = {
|
||||||
|
version: 1,
|
||||||
|
nodes: [
|
||||||
|
{ type: "peaking", id: "n1", enabled: true, params: { frequency: 1000, gain: 0, Q: 1 } },
|
||||||
|
{ type: "highpass", id: "n2", enabled: true, params: {} },
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const lane = (points: HfAutomationLane["points"], target = "volume"): HfAutomationLane => ({
|
||||||
|
target,
|
||||||
|
points,
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("targets", () => {
|
||||||
|
it("reads volume and fx targets, and rejects anything else", () => {
|
||||||
|
expect(parseAutomationTarget("volume")).toEqual({ kind: "volume" });
|
||||||
|
expect(parseAutomationTarget("fx.n1.frequency")).toEqual({
|
||||||
|
kind: "fx",
|
||||||
|
nodeId: "n1",
|
||||||
|
param: "frequency",
|
||||||
|
});
|
||||||
|
expect(parseAutomationTarget("fx.n1")).toBeNull();
|
||||||
|
expect(parseAutomationTarget("gain")).toBeNull();
|
||||||
|
expect(parseAutomationTarget("")).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("resolves a range from the registry, not from the lane", () => {
|
||||||
|
const r = resolveAutomationRange(fxAutomationTarget("n1", "frequency"), chain);
|
||||||
|
expect(r).not.toBeNull();
|
||||||
|
expect(r?.scale).toBe("log");
|
||||||
|
expect(r?.unit).toBe("Hz");
|
||||||
|
expect(r?.min).toBeGreaterThan(0);
|
||||||
|
expect(resolveAutomationRange("volume", chain)).toEqual(VOLUME_RANGE);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("has no range for a missing node, a missing param, or an enum param", () => {
|
||||||
|
expect(resolveAutomationRange("fx.nope.frequency", chain)).toBeNull();
|
||||||
|
expect(resolveAutomationRange("fx.n1.nonsense", chain)).toBeNull();
|
||||||
|
// `poles` is an enum: there is no envelope between one and two poles.
|
||||||
|
expect(resolveAutomationRange("fx.n2.poles", chain)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("normalisation", () => {
|
||||||
|
it("sorts points and collapses duplicate times, keeping the later value", () => {
|
||||||
|
const parsed = parseAutomation(
|
||||||
|
JSON.stringify({
|
||||||
|
version: 1,
|
||||||
|
lanes: [
|
||||||
|
{
|
||||||
|
target: "volume",
|
||||||
|
points: [
|
||||||
|
{ t: 2, v: 0.2 },
|
||||||
|
{ t: 0, v: 1 },
|
||||||
|
{ t: 2, v: 0.9 },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(parsed.lanes[0]!.points).toEqual([
|
||||||
|
{ t: 0, v: 1 },
|
||||||
|
{ t: 2, v: 0.9 },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops non-finite points rather than letting NaN reach an AudioParam", () => {
|
||||||
|
const parsed = parseAutomation(
|
||||||
|
JSON.stringify({
|
||||||
|
version: 1,
|
||||||
|
lanes: [
|
||||||
|
{
|
||||||
|
target: "volume",
|
||||||
|
points: [
|
||||||
|
{ t: 0, v: 0.5 },
|
||||||
|
{ t: 1, v: null },
|
||||||
|
{ t: "x", v: 1 },
|
||||||
|
{ t: 2, v: 0.25 },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(parsed.lanes[0]!.points).toEqual([
|
||||||
|
{ t: 0, v: 0.5 },
|
||||||
|
{ t: 2, v: 0.25 },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("clamps volume into 0..1 at parse time", () => {
|
||||||
|
const parsed = parseAutomation(
|
||||||
|
JSON.stringify({
|
||||||
|
version: 1,
|
||||||
|
lanes: [
|
||||||
|
{
|
||||||
|
target: "volume",
|
||||||
|
points: [
|
||||||
|
{ t: 0, v: 4 },
|
||||||
|
{ t: 1, v: -2 },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(parsed.lanes[0]!.points.map((p) => p.v)).toEqual([1, 0]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses malformed input instead of silently losing an envelope", () => {
|
||||||
|
expect(() => parseAutomation("{")).toThrow(/not valid JSON/);
|
||||||
|
expect(() => parseAutomation(JSON.stringify({ version: 9, lanes: [] }))).toThrow(
|
||||||
|
/Unsupported automation version/,
|
||||||
|
);
|
||||||
|
expect(() => parseAutomation(JSON.stringify({ version: 1 }))).toThrow(/lanes/);
|
||||||
|
expect(() =>
|
||||||
|
parseAutomation(JSON.stringify({ version: 1, lanes: [{ target: "nope", points: [] }] })),
|
||||||
|
).toThrow(/unreadable target/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("round-trips through the attribute", () => {
|
||||||
|
const source = {
|
||||||
|
version: 1,
|
||||||
|
lanes: [
|
||||||
|
lane([
|
||||||
|
{ t: 0, v: 0.8 },
|
||||||
|
{ t: 3, v: 0.2, curve: 0.5 },
|
||||||
|
]),
|
||||||
|
lane(
|
||||||
|
[
|
||||||
|
{ t: 0, v: 200 },
|
||||||
|
{ t: 4, v: 8000 },
|
||||||
|
],
|
||||||
|
"fx.n1.frequency",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
};
|
||||||
|
expect(parseAutomation(serializeAutomation(source))).toEqual(source);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("resolveAutomation", () => {
|
||||||
|
it("drops lanes whose effect was deleted and clamps the rest to the registry", () => {
|
||||||
|
const resolved = resolveAutomation(
|
||||||
|
{
|
||||||
|
version: 1,
|
||||||
|
lanes: [
|
||||||
|
lane([{ t: 0, v: 1_000_000 }], "fx.n1.frequency"),
|
||||||
|
lane([{ t: 0, v: 0.5 }], "fx.gone.frequency"),
|
||||||
|
lane([{ t: 0, v: 0.5 }]),
|
||||||
|
],
|
||||||
|
},
|
||||||
|
chain,
|
||||||
|
);
|
||||||
|
expect(resolved.lanes.map((l) => l.target)).toEqual(["fx.n1.frequency", "volume"]);
|
||||||
|
const range = resolveAutomationRange("fx.n1.frequency", chain);
|
||||||
|
expect(resolved.lanes[0]!.points[0]!.v).toBe(range?.max);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops every fx lane when the track has no chain at all", () => {
|
||||||
|
const resolved = resolveAutomation(
|
||||||
|
{ version: 1, lanes: [lane([{ t: 0, v: 1 }], "fx.n1.frequency"), lane([{ t: 0, v: 1 }])] },
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
expect(resolved.lanes.map((l) => l.target)).toEqual(["volume"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("sampling", () => {
|
||||||
|
const ramp = lane([
|
||||||
|
{ t: 1, v: 0 },
|
||||||
|
{ t: 3, v: 1 },
|
||||||
|
]);
|
||||||
|
|
||||||
|
it("holds the end values outside the points", () => {
|
||||||
|
expect(sampleAutomationLane(ramp, 0)).toBe(0);
|
||||||
|
expect(sampleAutomationLane(ramp, 1)).toBe(0);
|
||||||
|
expect(sampleAutomationLane(ramp, 3)).toBe(1);
|
||||||
|
expect(sampleAutomationLane(ramp, 99)).toBe(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("interpolates linearly between them", () => {
|
||||||
|
expect(sampleAutomationLane(ramp, 2)).toBeCloseTo(0.5, 10);
|
||||||
|
expect(sampleAutomationLane(ramp, 1.5)).toBeCloseTo(0.25, 10);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("interpolates a log-scaled parameter in log space", () => {
|
||||||
|
const sweep = lane(
|
||||||
|
[
|
||||||
|
{ t: 0, v: 200 },
|
||||||
|
{ t: 4, v: 8000 },
|
||||||
|
],
|
||||||
|
"fx.n1.frequency",
|
||||||
|
);
|
||||||
|
// Halfway through the sweep is the geometric mean, not the arithmetic one:
|
||||||
|
// an even-sounding sweep, which is what a log knob already promises.
|
||||||
|
expect(sampleAutomationLane(sweep, 2, "log")).toBeCloseTo(Math.sqrt(200 * 8000), 6);
|
||||||
|
expect(sampleAutomationLane(sweep, 2, "linear")).toBeCloseTo(4100, 6);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("bends a segment with curve, staying pinned at both ends", () => {
|
||||||
|
const bent = lane([
|
||||||
|
{ t: 0, v: 0, curve: 1 },
|
||||||
|
{ t: 1, v: 1 },
|
||||||
|
]);
|
||||||
|
expect(sampleAutomationLane(bent, 0)).toBe(0);
|
||||||
|
expect(sampleAutomationLane(bent, 1)).toBe(1);
|
||||||
|
// Positive curve holds low and rises late.
|
||||||
|
expect(sampleAutomationLane(bent, 0.5)).toBeLessThan(0.5);
|
||||||
|
const eased = lane([
|
||||||
|
{ t: 0, v: 0, curve: -1 },
|
||||||
|
{ t: 1, v: 1 },
|
||||||
|
]);
|
||||||
|
expect(sampleAutomationLane(eased, 0.5)).toBeGreaterThan(0.5);
|
||||||
|
expect(applyCurve(0.5, 0)).toBe(0.5);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("walks a dense lane by bisection, not by scanning", () => {
|
||||||
|
const points = Array.from({ length: 200 }, (_, i) => ({ t: i, v: i % 2 }));
|
||||||
|
const dense = lane(points);
|
||||||
|
expect(sampleAutomationLane(dense, 100)).toBe(0);
|
||||||
|
expect(sampleAutomationLane(dense, 101)).toBe(1);
|
||||||
|
expect(sampleAutomationLane(dense, 100.5)).toBeCloseTo(0.5, 10);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("caps a pathological lane so the scheduler cannot be hung", () => {
|
||||||
|
const parsed = parseAutomation(
|
||||||
|
JSON.stringify({
|
||||||
|
version: 1,
|
||||||
|
lanes: [
|
||||||
|
{ target: "volume", points: Array.from({ length: 5000 }, (_, i) => ({ t: i, v: 0.5 })) },
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(parsed.lanes[0]!.points.length).toBe(512);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("samples a curve at both endpoints", () => {
|
||||||
|
const curve = sampleAutomationCurve(ramp, 1, 3, 5);
|
||||||
|
expect(curve.length).toBe(5);
|
||||||
|
expect(curve[0]).toBe(0);
|
||||||
|
expect(curve[4]).toBe(1);
|
||||||
|
expect(curve[2]).toBeCloseTo(0.5, 6);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("spots a lane not worth scheduling", () => {
|
||||||
|
expect(isConstantLane(lane([{ t: 0, v: 0.5 }]))).toBe(true);
|
||||||
|
expect(
|
||||||
|
isConstantLane(
|
||||||
|
lane([
|
||||||
|
{ t: 0, v: 0.5 },
|
||||||
|
{ t: 2, v: 0.5 },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
).toBe(true);
|
||||||
|
expect(isConstantLane(ramp)).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("chain node ids", () => {
|
||||||
|
it("mints the first free id and survives a round trip", () => {
|
||||||
|
expect(mintAudioFxNodeId({ version: 1, nodes: [] })).toBe("n1");
|
||||||
|
expect(mintAudioFxNodeId(chain)).toBe("n3");
|
||||||
|
const gap: HfAudioFxChain = { version: 1, nodes: [{ type: "peaking", id: "n2" }] };
|
||||||
|
expect(mintAudioFxNodeId(gap)).toBe("n1");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps ids through parse so lanes stay pointed at the same effect", () => {
|
||||||
|
const json = JSON.stringify({
|
||||||
|
version: 1,
|
||||||
|
nodes: [{ type: "peaking", id: "n7", params: {} }],
|
||||||
|
});
|
||||||
|
expect(parseAudioFxChain(json).nodes[0]!.id).toBe("n7");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,368 @@
|
|||||||
|
/**
|
||||||
|
* Automation envelopes for audio tracks.
|
||||||
|
*
|
||||||
|
* A lane is a list of breakpoints over one parameter — track volume, or one
|
||||||
|
* knob of one effect in the track's FX chain. Ableton's clip-envelope model:
|
||||||
|
* times are clip-local, so an envelope travels with the clip when it moves.
|
||||||
|
*
|
||||||
|
* Nothing here touches Web Audio. This module owns the format and the
|
||||||
|
* interpolation, and the same `sampleAutomationLane` is used to draw the lane
|
||||||
|
* in the timeline, to schedule it in preview, and to bake it at render — one
|
||||||
|
* curve, three consumers, or the picture and the sound disagree.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { getAudioFxDef, type HfAudioFxChain, type HfAudioFxNumberParam } from "./audioFx.js";
|
||||||
|
|
||||||
|
export const HF_AUDIO_AUTOMATION_ATTR = "data-automation";
|
||||||
|
|
||||||
|
/** Automation files are versioned; a reader must refuse a version it doesn't know. */
|
||||||
|
export const HF_AUDIO_AUTOMATION_VERSION = 1;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A pathological document should not be able to hang the scheduler, which
|
||||||
|
* expands every segment into scheduled ramps. Well past any hand-drawn
|
||||||
|
* envelope; a lane is dense at 100 points.
|
||||||
|
*/
|
||||||
|
export const MAX_AUTOMATION_POINTS = 512;
|
||||||
|
|
||||||
|
export interface HfAutomationPoint {
|
||||||
|
/** Seconds from the start of the clip, not the composition. */
|
||||||
|
t: number;
|
||||||
|
/** Value in the parameter's own unit — dB for a threshold, Hz for a cutoff. */
|
||||||
|
v: number;
|
||||||
|
/**
|
||||||
|
* Curvature of the segment *leaving* this point, -1..1. Absent or 0 is a
|
||||||
|
* straight line; positive holds low then rises late, negative rises early.
|
||||||
|
*/
|
||||||
|
curve?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface HfAutomationLane {
|
||||||
|
/** `volume`, or `fx.<nodeId>.<paramKey>`. */
|
||||||
|
target: string;
|
||||||
|
points: HfAutomationPoint[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface HfAutomation {
|
||||||
|
version: number;
|
||||||
|
lanes: HfAutomationLane[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export class AudioAutomationError extends Error {
|
||||||
|
constructor(message: string) {
|
||||||
|
super(message);
|
||||||
|
this.name = "AudioAutomationError";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export const VOLUME_TARGET = "volume";
|
||||||
|
|
||||||
|
export type HfAutomationTarget = { kind: "volume" } | { kind: "fx"; nodeId: string; param: string };
|
||||||
|
|
||||||
|
/** Split a target string. Returns null for anything unrecognised. */
|
||||||
|
export function parseAutomationTarget(target: string): HfAutomationTarget | null {
|
||||||
|
if (target === VOLUME_TARGET) return { kind: "volume" };
|
||||||
|
const parts = target.split(".");
|
||||||
|
if (parts.length !== 3 || parts[0] !== "fx") return null;
|
||||||
|
const [, nodeId, param] = parts;
|
||||||
|
if (!nodeId || !param) return null;
|
||||||
|
return { kind: "fx", nodeId, param };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fxAutomationTarget(nodeId: string, param: string): string {
|
||||||
|
return `fx.${nodeId}.${param}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The value range a lane is drawn and clamped against.
|
||||||
|
*
|
||||||
|
* Volume is linear 0..1, matching `data-volume` and the existing volume
|
||||||
|
* envelope machinery — no dB conversion enters the volume path. Everything
|
||||||
|
* else is read from the effect registry, so a lane can never offer a value the
|
||||||
|
* renderer would reject, and the log-scaled knobs sweep the way a DAW's do.
|
||||||
|
*/
|
||||||
|
export interface AutomationRange {
|
||||||
|
min: number;
|
||||||
|
max: number;
|
||||||
|
step: number;
|
||||||
|
unit: string;
|
||||||
|
label: string;
|
||||||
|
scale: "linear" | "log";
|
||||||
|
/** Where an empty lane draws its flat line, and what a new point starts at. */
|
||||||
|
default: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const VOLUME_RANGE: AutomationRange = {
|
||||||
|
min: 0,
|
||||||
|
max: 1,
|
||||||
|
step: 0.01,
|
||||||
|
unit: "",
|
||||||
|
label: "Volume",
|
||||||
|
scale: "linear",
|
||||||
|
default: 1,
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolve a lane's target against a chain. Returns null when the target names
|
||||||
|
* a node or parameter that is not there — the effect was deleted, or the
|
||||||
|
* parameter is an enum, which has no envelope between its values.
|
||||||
|
*/
|
||||||
|
export function resolveAutomationRange(
|
||||||
|
target: string,
|
||||||
|
chain: HfAudioFxChain | undefined,
|
||||||
|
): AutomationRange | null {
|
||||||
|
const parsed = parseAutomationTarget(target);
|
||||||
|
if (!parsed) return null;
|
||||||
|
if (parsed.kind === "volume") return VOLUME_RANGE;
|
||||||
|
const node = chain?.nodes.find((n) => n.id === parsed.nodeId);
|
||||||
|
if (!node) return null;
|
||||||
|
const def = getAudioFxDef(node.type);
|
||||||
|
const param = def?.params.find((p) => p.key === parsed.param);
|
||||||
|
if (!param || param.kind !== "number") return null;
|
||||||
|
const p = param as HfAudioFxNumberParam;
|
||||||
|
return {
|
||||||
|
min: p.min,
|
||||||
|
max: p.max,
|
||||||
|
step: p.step,
|
||||||
|
unit: p.unit,
|
||||||
|
label: `${def?.label ?? node.type} · ${p.label}`,
|
||||||
|
scale: p.scale === "log" && p.min > 0 ? "log" : "linear",
|
||||||
|
default: p.default,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Coerce a JSON field to a number, treating anything that is not already one —
|
||||||
|
* or a string that plainly reads as one — as absent.
|
||||||
|
*
|
||||||
|
* Not `Number(raw)`: that turns `null`, `""` and `[]` into 0, which for an
|
||||||
|
* envelope means a silent drop to zero rather than a point the reader rejects.
|
||||||
|
*/
|
||||||
|
function numberOrNull(raw: unknown): number | null {
|
||||||
|
if (typeof raw === "number") return Number.isFinite(raw) ? raw : null;
|
||||||
|
if (typeof raw === "string" && raw.trim() !== "") {
|
||||||
|
const n = Number(raw);
|
||||||
|
return Number.isFinite(n) ? n : null;
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function clampCurve(v: unknown): number {
|
||||||
|
const n = numberOrNull(v);
|
||||||
|
if (n === null || n === 0) return 0;
|
||||||
|
return Math.min(1, Math.max(-1, n));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clean one point, or reject it.
|
||||||
|
*
|
||||||
|
* A missing or non-finite value reaching an AudioParam silences the node for
|
||||||
|
* the rest of the render, so such a point is dropped rather than coerced.
|
||||||
|
*/
|
||||||
|
function cleanPoint(
|
||||||
|
p: HfAutomationPoint | undefined,
|
||||||
|
range: AutomationRange | null,
|
||||||
|
): HfAutomationPoint | null {
|
||||||
|
const t = numberOrNull(p?.t);
|
||||||
|
const v = numberOrNull(p?.v);
|
||||||
|
if (t === null || v === null) return null;
|
||||||
|
const clamped = range ? Math.min(range.max, Math.max(range.min, v)) : v;
|
||||||
|
const curve = clampCurve(p?.curve);
|
||||||
|
return { t: Math.max(0, t), v: clamped, ...(curve ? { curve } : {}) };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Order points, drop unusable ones, and collapse duplicate times.
|
||||||
|
*
|
||||||
|
* Later wins on a tie so that dragging a point onto another replaces it rather
|
||||||
|
* than leaving an invisible one underneath.
|
||||||
|
*/
|
||||||
|
function normalizePoints(
|
||||||
|
points: readonly HfAutomationPoint[],
|
||||||
|
range: AutomationRange | null,
|
||||||
|
): HfAutomationPoint[] {
|
||||||
|
const clean = points
|
||||||
|
.map((p) => cleanPoint(p, range))
|
||||||
|
.filter((p): p is HfAutomationPoint => p !== null)
|
||||||
|
.sort((a, b) => a.t - b.t);
|
||||||
|
const out: HfAutomationPoint[] = [];
|
||||||
|
for (const p of clean) {
|
||||||
|
if (out.length > 0 && out[out.length - 1]!.t === p.t) out[out.length - 1] = p;
|
||||||
|
else out.push(p);
|
||||||
|
}
|
||||||
|
return out.slice(0, MAX_AUTOMATION_POINTS);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Structural normalisation, with no knowledge of the chain: sort and clean the
|
||||||
|
* points of every lane and drop lanes that carry none. Range clamping and
|
||||||
|
* orphan removal need the chain and happen in `resolveAutomation`.
|
||||||
|
*/
|
||||||
|
export function normalizeAutomation(automation: HfAutomation): HfAutomation {
|
||||||
|
const lanes: HfAutomationLane[] = [];
|
||||||
|
for (const lane of automation.lanes) {
|
||||||
|
if (!parseAutomationTarget(lane.target)) continue;
|
||||||
|
const range = lane.target === VOLUME_TARGET ? VOLUME_RANGE : null;
|
||||||
|
const points = normalizePoints(lane.points ?? [], range);
|
||||||
|
if (points.length > 0) lanes.push({ target: lane.target, points });
|
||||||
|
}
|
||||||
|
return { version: HF_AUDIO_AUTOMATION_VERSION, lanes };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Bind automation to a chain: clamp each lane into its parameter's declared
|
||||||
|
* range and drop lanes whose target no longer exists.
|
||||||
|
*
|
||||||
|
* Dropping is deliberate. An envelope on a deleted effect has nothing to
|
||||||
|
* drive, and keeping it would silently reattach if an unrelated effect later
|
||||||
|
* took the same node id.
|
||||||
|
*/
|
||||||
|
export function resolveAutomation(
|
||||||
|
automation: HfAutomation,
|
||||||
|
chain: HfAudioFxChain | undefined,
|
||||||
|
): HfAutomation {
|
||||||
|
const lanes: HfAutomationLane[] = [];
|
||||||
|
for (const lane of automation.lanes) {
|
||||||
|
const range = resolveAutomationRange(lane.target, chain);
|
||||||
|
if (!range) continue;
|
||||||
|
const points = normalizePoints(lane.points, range);
|
||||||
|
if (points.length > 0) lanes.push({ target: lane.target, points });
|
||||||
|
}
|
||||||
|
return { version: HF_AUDIO_AUTOMATION_VERSION, lanes };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse an automation attribute.
|
||||||
|
*
|
||||||
|
* Malformed input throws rather than being skipped, matching the chain reader:
|
||||||
|
* a track that quietly loses its envelope renders differently from the project
|
||||||
|
* the author saved, which is worse than refusing.
|
||||||
|
*/
|
||||||
|
export function parseAutomation(json: string): HfAutomation {
|
||||||
|
let raw: unknown;
|
||||||
|
try {
|
||||||
|
raw = JSON.parse(json);
|
||||||
|
} catch (err) {
|
||||||
|
throw new AudioAutomationError(`Automation is not valid JSON: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
if (typeof raw !== "object" || raw === null) {
|
||||||
|
throw new AudioAutomationError("Automation must be a JSON object.");
|
||||||
|
}
|
||||||
|
const obj = raw as { version?: unknown; lanes?: unknown };
|
||||||
|
if (obj.version !== HF_AUDIO_AUTOMATION_VERSION) {
|
||||||
|
throw new AudioAutomationError(`Unsupported automation version: ${String(obj.version)}`);
|
||||||
|
}
|
||||||
|
if (!Array.isArray(obj.lanes)) {
|
||||||
|
throw new AudioAutomationError("Automation is missing a `lanes` array.");
|
||||||
|
}
|
||||||
|
const lanes: HfAutomationLane[] = obj.lanes.map((l, i) => {
|
||||||
|
if (typeof l !== "object" || l === null) {
|
||||||
|
throw new AudioAutomationError(`Lane ${i} is not an object.`);
|
||||||
|
}
|
||||||
|
const lane = l as { target?: unknown; points?: unknown };
|
||||||
|
if (typeof lane.target !== "string" || !parseAutomationTarget(lane.target)) {
|
||||||
|
throw new AudioAutomationError(`Lane ${i} has an unreadable target: ${String(lane.target)}`);
|
||||||
|
}
|
||||||
|
if (!Array.isArray(lane.points)) {
|
||||||
|
throw new AudioAutomationError(`Lane ${i} is missing a \`points\` array.`);
|
||||||
|
}
|
||||||
|
return { target: lane.target, points: lane.points as HfAutomationPoint[] };
|
||||||
|
});
|
||||||
|
return normalizeAutomation({ version: HF_AUDIO_AUTOMATION_VERSION, lanes });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Serialise for the `data-automation` attribute. */
|
||||||
|
export function serializeAutomation(automation: HfAutomation): string {
|
||||||
|
return JSON.stringify({
|
||||||
|
version: HF_AUDIO_AUTOMATION_VERSION,
|
||||||
|
lanes: automation.lanes.map((lane) => ({
|
||||||
|
target: lane.target,
|
||||||
|
points: lane.points.map((p) => ({
|
||||||
|
t: p.t,
|
||||||
|
v: p.v,
|
||||||
|
...(p.curve ? { curve: p.curve } : {}),
|
||||||
|
})),
|
||||||
|
})),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shape the 0..1 progress across a segment.
|
||||||
|
*
|
||||||
|
* `curve` is an exponent in disguise: 0 is linear, and the ends reach a
|
||||||
|
* quarter-power and a fourth-power bend, which is about the range a DAW's
|
||||||
|
* envelope handle covers before the segment stops reading as a curve.
|
||||||
|
*/
|
||||||
|
export function applyCurve(x: number, curve: number | undefined): number {
|
||||||
|
if (!curve) return x;
|
||||||
|
return Math.pow(x, Math.pow(2, 2 * curve));
|
||||||
|
}
|
||||||
|
|
||||||
|
function lerpValue(a: number, b: number, x: number, scale: "linear" | "log"): number {
|
||||||
|
// A frequency sweep interpolated linearly spends almost all its time in the
|
||||||
|
// top octave. Log-scaled parameters interpolate in log space so a 200 Hz to
|
||||||
|
// 8 kHz move sounds like an even sweep, which is what the knob's own scale
|
||||||
|
// already promises.
|
||||||
|
if (scale === "log" && a > 0 && b > 0) {
|
||||||
|
return Math.exp(Math.log(a) + (Math.log(b) - Math.log(a)) * x);
|
||||||
|
}
|
||||||
|
return a + (b - a) * x;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Value of a lane at a clip-local time.
|
||||||
|
*
|
||||||
|
* Outside the points, the envelope holds — the first value before it starts
|
||||||
|
* and the last value after it ends, so a lane never snaps to zero at the edges.
|
||||||
|
*/
|
||||||
|
export function sampleAutomationLane(
|
||||||
|
lane: HfAutomationLane,
|
||||||
|
t: number,
|
||||||
|
scale: "linear" | "log" = "linear",
|
||||||
|
): number {
|
||||||
|
const pts = lane.points;
|
||||||
|
if (pts.length === 0) return 0;
|
||||||
|
const first = pts[0]!;
|
||||||
|
if (t <= first.t) return first.v;
|
||||||
|
const last = pts[pts.length - 1]!;
|
||||||
|
if (t >= last.t) return last.v;
|
||||||
|
|
||||||
|
let lo = 0;
|
||||||
|
let hi = pts.length - 1;
|
||||||
|
while (hi - lo > 1) {
|
||||||
|
const mid = (lo + hi) >> 1;
|
||||||
|
if (pts[mid]!.t <= t) lo = mid;
|
||||||
|
else hi = mid;
|
||||||
|
}
|
||||||
|
const a = pts[lo]!;
|
||||||
|
const b = pts[hi]!;
|
||||||
|
const span = b.t - a.t;
|
||||||
|
if (span <= 0) return b.v;
|
||||||
|
return lerpValue(a.v, b.v, applyCurve((t - a.t) / span, a.curve), scale);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** True when the lane is a single value, i.e. worth setting once and not scheduling. */
|
||||||
|
export function isConstantLane(lane: HfAutomationLane): boolean {
|
||||||
|
return lane.points.length <= 1 || lane.points.every((p) => p.v === lane.points[0]!.v);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sample a lane onto evenly spaced times, for consumers that want a plain
|
||||||
|
* curve rather than a breakpoint list: `setValueCurveAtTime`, the render-side
|
||||||
|
* volume envelope, and the lane's own drawing code.
|
||||||
|
*/
|
||||||
|
export function sampleAutomationCurve(
|
||||||
|
lane: HfAutomationLane,
|
||||||
|
from: number,
|
||||||
|
to: number,
|
||||||
|
count: number,
|
||||||
|
scale: "linear" | "log" = "linear",
|
||||||
|
): Float32Array {
|
||||||
|
const n = Math.max(2, Math.floor(count));
|
||||||
|
const out = new Float32Array(n);
|
||||||
|
const span = to - from;
|
||||||
|
for (let i = 0; i < n; i += 1) {
|
||||||
|
out[i] = sampleAutomationLane(lane, from + (span * i) / (n - 1), scale);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
@@ -33,6 +33,15 @@ export interface HfAudioFxNumberParam {
|
|||||||
default: number;
|
default: number;
|
||||||
/** Frequency-style controls need a log knob to be usable. */
|
/** Frequency-style controls need a log knob to be usable. */
|
||||||
scale?: "linear" | "log";
|
scale?: "linear" | "log";
|
||||||
|
/**
|
||||||
|
* The knob is backed by an AudioParam, so an automation lane can drive it.
|
||||||
|
*
|
||||||
|
* Not every knob can be: a WaveShaper curve, a convolution impulse and a
|
||||||
|
* worklet's `processorOptions` are all set wholesale rather than scheduled.
|
||||||
|
* A graph builder must expose an AudioParam for every parameter flagged here
|
||||||
|
* — `audioFxGraph.test.ts` builds each effect and checks it.
|
||||||
|
*/
|
||||||
|
automatable?: boolean;
|
||||||
/** One line explaining what turning this does, shown on the control. */
|
/** One line explaining what turning this does, shown on the control. */
|
||||||
hint?: string;
|
hint?: string;
|
||||||
}
|
}
|
||||||
@@ -81,6 +90,7 @@ const freq = (
|
|||||||
step: 1,
|
step: 1,
|
||||||
default: def,
|
default: def,
|
||||||
scale: "log",
|
scale: "log",
|
||||||
|
automatable: true,
|
||||||
});
|
});
|
||||||
|
|
||||||
const qParam = (def = 0.707, hint = "Bandwidth — higher is narrower."): HfAudioFxNumberParam => ({
|
const qParam = (def = 0.707, hint = "Bandwidth — higher is narrower."): HfAudioFxNumberParam => ({
|
||||||
@@ -93,6 +103,7 @@ const qParam = (def = 0.707, hint = "Bandwidth — higher is narrower."): HfAudi
|
|||||||
step: 0.01,
|
step: 0.01,
|
||||||
default: def,
|
default: def,
|
||||||
scale: "log",
|
scale: "log",
|
||||||
|
automatable: true,
|
||||||
hint,
|
hint,
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -105,6 +116,7 @@ const gainDb = (min = -40, max = 40, def = 0): HfAudioFxNumberParam => ({
|
|||||||
max,
|
max,
|
||||||
step: 0.1,
|
step: 0.1,
|
||||||
default: def,
|
default: def,
|
||||||
|
automatable: true,
|
||||||
});
|
});
|
||||||
|
|
||||||
const poles: HfAudioFxEnumParam = {
|
const poles: HfAudioFxEnumParam = {
|
||||||
@@ -411,6 +423,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "output",
|
key: "output",
|
||||||
|
automatable: true,
|
||||||
label: "Output",
|
label: "Output",
|
||||||
unit: "dB",
|
unit: "dB",
|
||||||
min: -24,
|
min: -24,
|
||||||
@@ -482,6 +495,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "time",
|
key: "time",
|
||||||
|
automatable: true,
|
||||||
label: "Time",
|
label: "Time",
|
||||||
unit: "ms",
|
unit: "ms",
|
||||||
min: 1,
|
min: 1,
|
||||||
@@ -495,6 +509,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "feedback",
|
key: "feedback",
|
||||||
|
automatable: true,
|
||||||
label: "Feedback",
|
label: "Feedback",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0.01,
|
min: 0.01,
|
||||||
@@ -505,6 +520,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "mix",
|
key: "mix",
|
||||||
|
automatable: true,
|
||||||
label: "Mix",
|
label: "Mix",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -524,6 +540,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "delay",
|
key: "delay",
|
||||||
|
automatable: true,
|
||||||
label: "Delay",
|
label: "Delay",
|
||||||
unit: "ms",
|
unit: "ms",
|
||||||
min: 1,
|
min: 1,
|
||||||
@@ -534,6 +551,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "depth",
|
key: "depth",
|
||||||
|
automatable: true,
|
||||||
label: "Depth",
|
label: "Depth",
|
||||||
unit: "ms",
|
unit: "ms",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -544,6 +562,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "speed",
|
key: "speed",
|
||||||
|
automatable: true,
|
||||||
label: "Rate",
|
label: "Rate",
|
||||||
unit: "Hz",
|
unit: "Hz",
|
||||||
min: 0.01,
|
min: 0.01,
|
||||||
@@ -554,6 +573,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "mix",
|
key: "mix",
|
||||||
|
automatable: true,
|
||||||
label: "Mix",
|
label: "Mix",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -573,6 +593,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "in_gain",
|
key: "in_gain",
|
||||||
|
automatable: true,
|
||||||
label: "Input",
|
label: "Input",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -583,6 +604,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "out_gain",
|
key: "out_gain",
|
||||||
|
automatable: true,
|
||||||
label: "Output",
|
label: "Output",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -615,6 +637,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "speed",
|
key: "speed",
|
||||||
|
automatable: true,
|
||||||
label: "Rate",
|
label: "Rate",
|
||||||
unit: "Hz",
|
unit: "Hz",
|
||||||
min: 0.1,
|
min: 0.1,
|
||||||
@@ -666,6 +689,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "wet",
|
key: "wet",
|
||||||
|
automatable: true,
|
||||||
label: "Wet",
|
label: "Wet",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -676,6 +700,7 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
|||||||
{
|
{
|
||||||
kind: "number",
|
kind: "number",
|
||||||
key: "dry",
|
key: "dry",
|
||||||
|
automatable: true,
|
||||||
label: "Dry",
|
label: "Dry",
|
||||||
unit: "",
|
unit: "",
|
||||||
min: 0,
|
min: 0,
|
||||||
@@ -735,6 +760,14 @@ export function normalizeAudioFxParams(
|
|||||||
export interface HfAudioFxNode {
|
export interface HfAudioFxNode {
|
||||||
/** Effect id from HF_AUDIO_FX. */
|
/** Effect id from HF_AUDIO_FX. */
|
||||||
type: string;
|
type: string;
|
||||||
|
/**
|
||||||
|
* Stable handle for this node within its chain, minted when the node is
|
||||||
|
* added. Automation lanes address nodes by id (`fx.<id>.<param>`) so that
|
||||||
|
* reordering the chain never re-points a lane at a different effect. Older
|
||||||
|
* chains have no ids; they load fine and simply cannot be automated until
|
||||||
|
* the panel touches them.
|
||||||
|
*/
|
||||||
|
id?: string;
|
||||||
/** Set on nodes the carve analysis generated, so re-running replaces them
|
/** Set on nodes the carve analysis generated, so re-running replaces them
|
||||||
* instead of stacking another set on top of hand-added effects. */
|
* instead of stacking another set on top of hand-added effects. */
|
||||||
fromCarve?: boolean;
|
fromCarve?: boolean;
|
||||||
@@ -781,12 +814,19 @@ export function parseAudioFxChain(json: string): HfAudioFxChain {
|
|||||||
if (typeof n !== "object" || n === null) {
|
if (typeof n !== "object" || n === null) {
|
||||||
throw new AudioFxChainError(`Node ${i} is not an object.`);
|
throw new AudioFxChainError(`Node ${i} is not an object.`);
|
||||||
}
|
}
|
||||||
const node = n as { type?: unknown; enabled?: unknown; params?: unknown; fromCarve?: unknown };
|
const node = n as {
|
||||||
|
type?: unknown;
|
||||||
|
id?: unknown;
|
||||||
|
enabled?: unknown;
|
||||||
|
params?: unknown;
|
||||||
|
fromCarve?: unknown;
|
||||||
|
};
|
||||||
if (typeof node.type !== "string" || !BY_ID.has(node.type)) {
|
if (typeof node.type !== "string" || !BY_ID.has(node.type)) {
|
||||||
throw new AudioFxChainError(`Node ${i} has unknown effect type: ${String(node.type)}`);
|
throw new AudioFxChainError(`Node ${i} has unknown effect type: ${String(node.type)}`);
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
type: node.type,
|
type: node.type,
|
||||||
|
...(typeof node.id === "string" && node.id ? { id: node.id } : {}),
|
||||||
...(node.fromCarve === true ? { fromCarve: true as const } : {}),
|
...(node.fromCarve === true ? { fromCarve: true as const } : {}),
|
||||||
enabled: node.enabled !== false,
|
enabled: node.enabled !== false,
|
||||||
params: normalizeAudioFxParams(
|
params: normalizeAudioFxParams(
|
||||||
@@ -809,9 +849,23 @@ export function serializeAudioFxChain(chain: HfAudioFxChain): string {
|
|||||||
version: HF_AUDIO_FX_CHAIN_VERSION,
|
version: HF_AUDIO_FX_CHAIN_VERSION,
|
||||||
nodes: chain.nodes.map((node) => ({
|
nodes: chain.nodes.map((node) => ({
|
||||||
type: node.type,
|
type: node.type,
|
||||||
|
...(node.id ? { id: node.id } : {}),
|
||||||
...(node.fromCarve === true ? { fromCarve: true } : {}),
|
...(node.fromCarve === true ? { fromCarve: true } : {}),
|
||||||
...(node.enabled === false ? { enabled: false } : {}),
|
...(node.enabled === false ? { enabled: false } : {}),
|
||||||
params: normalizeAudioFxParams(node.type, node.params),
|
params: normalizeAudioFxParams(node.type, node.params),
|
||||||
})),
|
})),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Next free node id for a chain, as `n1`, `n2`, … — counted rather than random
|
||||||
|
* so that adding an effect produces the same document on every machine, which
|
||||||
|
* compositions require.
|
||||||
|
*/
|
||||||
|
export function mintAudioFxNodeId(chain: HfAudioFxChain): string {
|
||||||
|
const taken = new Set(chain.nodes.map((n) => n.id).filter(Boolean));
|
||||||
|
for (let i = 1; ; i += 1) {
|
||||||
|
const id = `n${i}`;
|
||||||
|
if (!taken.has(id)) return id;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -428,3 +428,42 @@ describe("media_variable_src_no_fallback", () => {
|
|||||||
expect(result.findings.some((f) => f.code === "media_missing_src")).toBe(true);
|
expect(result.findings.some((f) => f.code === "media_missing_src")).toBe(true);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("audio_volume_double_automation", () => {
|
||||||
|
const withScript = (audioAttrs: string, script: string) => `<!DOCTYPE html><html><body>
|
||||||
|
<div id="root" data-composition-id="main" data-start="0" data-width="1920" data-height="1080" data-duration="10">
|
||||||
|
<audio id="bgm" src="a.wav" data-start="0" data-duration="10" ${audioAttrs}></audio>
|
||||||
|
</div>
|
||||||
|
<script>${script}</script>
|
||||||
|
</body></html>`;
|
||||||
|
|
||||||
|
const LANE = `data-automation='{"version":1,"lanes":[{"target":"volume","points":[{"t":0,"v":1}]}]}'`;
|
||||||
|
|
||||||
|
it("warns when a lane and a GSAP volume tween both shape the same track", async () => {
|
||||||
|
const res = await lintHyperframeHtml(
|
||||||
|
withScript(LANE, `tl.to("#bgm", { volume: 0, duration: 1 });`),
|
||||||
|
);
|
||||||
|
const finding = res.findings.find((f) => f.code === "audio_volume_double_automation");
|
||||||
|
expect(finding?.severity).toBe("warning");
|
||||||
|
expect(finding?.elementId).toBe("bgm");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("stays quiet for a lane alone, a tween alone, or a tween on another track", async () => {
|
||||||
|
const laneOnly = await lintHyperframeHtml(withScript(LANE, `tl.to("#bgm", { x: 10 });`));
|
||||||
|
const tweenOnly = await lintHyperframeHtml(withScript("", `tl.to("#bgm", { volume: 0 });`));
|
||||||
|
const otherTrack = await lintHyperframeHtml(withScript(LANE, `tl.to("#vo", { volume: 0 });`));
|
||||||
|
for (const res of [laneOnly, tweenOnly, otherTrack]) {
|
||||||
|
expect(res.findings.some((f) => f.code === "audio_volume_double_automation")).toBe(false);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores a lane that automates something other than volume", async () => {
|
||||||
|
const res = await lintHyperframeHtml(
|
||||||
|
withScript(
|
||||||
|
`data-automation='{"version":1,"lanes":[{"target":"fx.n1.frequency","points":[{"t":0,"v":200}]}]}'`,
|
||||||
|
`tl.to("#bgm", { volume: 0 });`,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
expect(res.findings.some((f) => f.code === "audio_volume_double_automation")).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
import type { LintContext, HyperframeLintFinding } from "../context";
|
import type { LintContext, HyperframeLintFinding } from "../context";
|
||||||
import { readAttr, readDecodedAttr, truncateSnippet, isMediaTag } from "../utils";
|
import { readAttr, readDecodedAttr, stripJsComments, truncateSnippet, isMediaTag } from "../utils";
|
||||||
import { validateColorGradingContract } from "@hyperframes/parsers/color-grading-contract";
|
import { validateColorGradingContract } from "@hyperframes/parsers/color-grading-contract";
|
||||||
|
|
||||||
function escapeRegExp(value: string): string {
|
function escapeRegExp(value: string): string {
|
||||||
@@ -597,4 +597,45 @@ export const mediaRules: Array<(ctx: LintContext) => HyperframeLintFinding[]> =
|
|||||||
|
|
||||||
// imperative_media_control
|
// imperative_media_control
|
||||||
findImperativeMediaControlFindings,
|
findImperativeMediaControlFindings,
|
||||||
|
|
||||||
|
// audio_volume_double_automation
|
||||||
|
findVolumeDoubleAutomationFindings,
|
||||||
];
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A track can have its volume shaped by an automation lane or by a GSAP tween,
|
||||||
|
* and only the lane is heard: the runtime reads `data-automation` first and
|
||||||
|
* never falls through to the probed tween. Both present means one of them is
|
||||||
|
* silently doing nothing, which is invisible in the file and in preview.
|
||||||
|
*/
|
||||||
|
function findVolumeDoubleAutomationFindings(ctx: LintContext): HyperframeLintFinding[] {
|
||||||
|
const automated = ctx.tags
|
||||||
|
.filter((tag) => isMediaTag(tag.name))
|
||||||
|
.map((tag) => ({ tag, automation: readDecodedAttr(tag.raw, "data-automation") }))
|
||||||
|
.filter((entry) => entry.automation && /"target"\s*:\s*"volume"/.test(entry.automation))
|
||||||
|
.map((entry) => ({ ...entry, id: readAttr(entry.tag.raw, "id") }))
|
||||||
|
.filter((entry): entry is typeof entry & { id: string } => Boolean(entry.id));
|
||||||
|
if (automated.length === 0) return [];
|
||||||
|
|
||||||
|
const script = ctx.scripts.map((block) => stripJsComments(block.content)).join("\n");
|
||||||
|
const findings: HyperframeLintFinding[] = [];
|
||||||
|
for (const { tag, id } of automated) {
|
||||||
|
// ponytail: a tween is recognised by a `volume` key appearing shortly after
|
||||||
|
// the element's own selector, rather than by parsing the timeline. It reads
|
||||||
|
// the same call the runtime's own probe would pick up, and the rule only
|
||||||
|
// warns, so a miss costs nothing.
|
||||||
|
const escaped = escapeRegExp(id);
|
||||||
|
const tweened = new RegExp(`#${escaped}(?![\\w-])[^;]{0,200}?\\bvolume\\s*:`, "s").test(script);
|
||||||
|
if (!tweened) continue;
|
||||||
|
findings.push({
|
||||||
|
code: "audio_volume_double_automation",
|
||||||
|
severity: "warning",
|
||||||
|
message: `#${id} has both a volume automation lane and a GSAP tween on \`volume\`. The lane wins — the tween is ignored in preview and in the render.`,
|
||||||
|
elementId: id,
|
||||||
|
fixHint:
|
||||||
|
"Keep one of them: delete the volume lane to go back to tweening, or drop the tween and shape the level in the automation lane.",
|
||||||
|
snippet: truncateSnippet(tag.raw),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return findings;
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user