From b839dbd2cc2c7e6e1da432fd8c000cf9e897375c Mon Sep 17 00:00:00 2001 From: Vance Ingalls Date: Thu, 13 Aug 2026 08:52:57 -0700 Subject: [PATCH] fix(core): give the chorus and phaser LFOs a phase, and unwire them on dispose (#3183) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. * fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a preview-only commit that useDomEditAttributeCommits.ts never grew — backported that option support from its own later commit so the two sides of the API agree. The paste path and its tests were missing the box selection's v0/v1 bounds a sibling commit added to AutomationSelection. The FX panel's carve controls still edited the six mechanism numbers (maxCutDb, bands, intelligibilityBias) after carveProfile() collapsed authoring to one Strength knob, so those fields no longer existed on HfCarveSettings; UI now edits strength, and analyseCarveBands is called with carveProfile(strength). Also closes fallow's complexity, dead-code and duplication findings on this PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math) and useAutomationRangeDrag.ts (the marquee-select gesture) out of useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a resolver into named functions, dropped an export nothing outside its file used, and shared a step-simplifier between audioCarve's two envelope builders. The edge-stretch vs. box-select priority test in TimelineAutomationLane.test was still pinning the pre-box-select rule (edge wins over a point sitting on it) that a sibling commit deliberately reversed — a point inside the box is now selected content, so grabbing it drags the group instead. Updated the test to the shipped rule instead of the old one. Co-Authored-By: Claude Opus 5 (1M context) * fix(core): cap the via conic's weight so an edge-clamped via point can't NaN A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to (0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0. viaConic divided by that zero to get an infinite weight, and shapeVia turned Infinity into NaN a few steps later (Infinity - Infinity in the quadratic coefficient). NaN reaching setValueCurveAtTime silences the automated parameter for the rest of the render. Capped the weight at 1e6 instead of leaving it unbounded — past that point the arc already reads as touching the via point, so nothing visible is lost. Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`, since NaN fails the original comparison and fell through it. Review by Miga (PR #3208). * fix(studio-server): fingerprint the proactive waveform cache key too The route already keys the waveform cache on the asset's size and mtime as well as its path, so a rebuilt-in-place file gets fresh peaks instead of stale ones. generateWaveformCache — the proactive path that runs on upload — still called buildWaveformCacheKey with the path alone, so it wrote to a different key than the route reads from (making the pre-generated cache never found) and kept the exact collision bug this fingerprint exists to fix on its own path. Review by Miga (PR #3211). * style(docs): run oxfmt on the /hyperframes-audio skill docs Table column widths had drifted out of alignment with oxfmt's own rules, failing format:check and blocking the Preflight gate every downstream branch inherits. Whitespace only, no content change. * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). * fix(studio): widen PropertyPanel's resetModules render timeout again The 20s margin (already once widened for the same reason) is timing out in CI's full-monorepo Test run — the resetModules()+fresh-import render this test needs is uncached and competes with every other package's test suite for the same worker pool, and the same test passes in well under 2s standalone. Went to 45s rather than re-tuning to whatever number happens to clear the current CI load, since that number moves every time CI gains a package. * fix(studio): stop the single-candidate auto-apply carve firing twice Two auto-apply effects both fire when sourceOptions.length === 1: the multi-candidate effect only guards length === 0, so a single candidate passes it too, and the single-candidate effect passes its own guard right after — both compute the same sources list and both call setCarve, so the common case (one narrator, one bed) triggered two decodes, two FFT runs, and two concurrent attribute writes for one decision. The multi-candidate effect now defers to its sibling for exactly one candidate, which already has its own detailed handling for that case. Review by Miga (PR #3213). * feat(core): carve against every voice over a bed, always (#3212) * feat(core): carve against every voice over a bed, always dynamically A bed usually runs under a whole sequence — a narrator, an interview answer, a second presenter — and carving against one of them left the others fighting it. `source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's clock before anything is measured. That is what keeps one analysis sufficient: the chain is fixed, so there is no per-voice filter to switch between, and bands drawn from all the speech there is with envelopes that rise wherever any of it happens answer the actual question — where and when is speech masking this bed. Summed rather than averaged: two people talking at once mask more than either alone. Audio before the bed starts is dropped rather than folded in at zero, since it plays over nothing and shifting it would put a cut where there is no voice. `dynamic` is gone. A fixed depth thins the bed through every pause, and once both have been heard there is no reason to want it, so every carve follows the speech. Two helpers the panel and the headless script now share instead of each carrying a copy — two definitions of "what does this name suggest" drift, and then the two disagree about which track is the voice: - `classifyAudioName` reads a track's kind from its id and filename together. `unknown` is deliberately common: treating an unrecognised name as "not a voice" would hide the one track somebody needs to pick. - `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten duration counts as unbounded, not zero — refusing a clip whose length the composition leaves to the media would drop the commonest case there is. Files written before this still load: a single `source` reads as a one-voice list, a stored `dynamic` is ignored, and an absent attribute means the defaults whole. Co-Authored-By: Claude Opus 5 (1M context) * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). --------- Co-authored-by: Claude Opus 5 (1M context) * fix(studio): port the carve UI off the removed source/dynamic fields #3212 (accidentally squash-merged into this branch instead of main) changed HfCarveSettings from a single `source` + `dynamic` toggle to a `sources` list with dynamic mode removed outright — the multi-voice UI consumer that goes with that shape lands in the very next PR, so this branch was left with a type that no longer matched its own code. Minimal port, not the multi-voice redesign that PR does properly: the "Listen to" picker and analyse() treat sources[0] as the one voice this UI still understands, and every dynamic-mode branch (the automated envelope lanes, the toggle, the checkbox) is gone along with the field — a carve is now always the static value the analysis computes, matching what the type change made permanent. Test suite trimmed the same way: the automation-lane and toggle tests covered behavior that no longer exists. --------- Co-authored-by: Claude Opus 5 (1M context) --- packages/core/package.json | 4 +- .../core/scripts/build-audio-fx-runtime.ts | 13 ++ .../core/scripts/build-inline-artifact.ts | 97 ------------ .../scripts/build-position-edits-render.ts | 13 ++ .../core/scripts/buildInjectedArtifact.ts | 78 ++++++++++ packages/core/src/audio/audioFxGraph.test.ts | 122 +++++++++++++-- packages/core/src/audio/audioFxGraph.ts | 139 ++++++++++++++---- .../generated/position-edits-render-inline.ts | 2 +- packages/core/src/runtime/audioFx.test.ts | 80 +++++++++- packages/core/src/runtime/audioFx.ts | 12 +- packages/engine/src/services/audioFxRender.ts | 21 +-- .../src/services/audioVolumeEnvelope.ts | 15 +- .../engine/src/services/wavChunks.test.ts | 56 +++++++ packages/engine/src/services/wavChunks.ts | 38 +++++ scripts/check-no-main-deletions.mjs | 4 + 15 files changed, 522 insertions(+), 172 deletions(-) create mode 100644 packages/core/scripts/build-audio-fx-runtime.ts delete mode 100644 packages/core/scripts/build-inline-artifact.ts create mode 100644 packages/core/scripts/build-position-edits-render.ts create mode 100644 packages/core/scripts/buildInjectedArtifact.ts create mode 100644 packages/engine/src/services/wavChunks.test.ts create mode 100644 packages/engine/src/services/wavChunks.ts diff --git a/packages/core/package.json b/packages/core/package.json index f4f4d3628..19712f71f 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -596,9 +596,9 @@ "typecheck": "tsc --noEmit && bun run typecheck:runtime", "typecheck:runtime": "tsc --noEmit -p tsconfig.runtime.json", "lint:runtime-preview-guards": "bun scripts/lint-runtime-preview-guards.ts", - "build:audio-fx-runtime": "tsx scripts/build-inline-artifact.ts audio-fx", + "build:audio-fx-runtime": "tsx scripts/build-audio-fx-runtime.ts", "build:hyperframes-runtime": "tsx scripts/build-hyperframes-runtime-artifact.ts", - "build:position-edits-render": "tsx scripts/build-inline-artifact.ts position-edits", + "build:position-edits-render": "tsx scripts/build-position-edits-render.ts", "check:position-edits-render": "bun run build:position-edits-render && git diff --exit-code -- src/generated/position-edits-render-inline.ts", "build:hyperframes-runtime:modular": "SANDBOX_RUNTIME_VARIANT=modular tsx scripts/build-hyperframes-runtime-artifact.ts", "build:hyperframe-runtime": "tsx scripts/build-hyperframes-runtime-artifact.ts", diff --git a/packages/core/scripts/build-audio-fx-runtime.ts b/packages/core/scripts/build-audio-fx-runtime.ts new file mode 100644 index 000000000..29d74d6e7 --- /dev/null +++ b/packages/core/scripts/build-audio-fx-runtime.ts @@ -0,0 +1,13 @@ +/** Build the injectable audio-FX runtime artifact from the canonical runtime. */ + +import { buildInjectedArtifact } from "./buildInjectedArtifact.js"; + +buildInjectedArtifact({ + scriptUrl: import.meta.url, + entry: "stubs/audio-fx-runtime-entry.ts", + out: "audio-fx-runtime-inline.ts", + constName: "AUDIO_FX_RUNTIME_IIFE", + fnName: "getAudioFxRuntimeScript", + what: "audio-FX runtime IIFE", + event: "audio_fx_runtime_generated", +}); diff --git a/packages/core/scripts/build-inline-artifact.ts b/packages/core/scripts/build-inline-artifact.ts deleted file mode 100644 index a506bf801..000000000 --- a/packages/core/scripts/build-inline-artifact.ts +++ /dev/null @@ -1,97 +0,0 @@ -/** - * Bundle one canonical-runtime entry to a minified IIFE with esbuild and write - * it into `src/generated` as a string constant behind a getter. - * - * One script handles both inline artifacts — the audio-FX runtime and the - * position-edits render — since they differ only in the entry, output names, - * and log event. Keeping them as separate files produced a byte-for-byte - * clone that fallow kept re-flagging on every unrelated line shift. - */ - -import { mkdirSync, writeFileSync } from "node:fs"; -import { dirname, resolve } from "node:path"; -import { fileURLToPath } from "node:url"; -import { buildSync } from "esbuild"; -import { execFileSync } from "node:child_process"; - -interface InlineArtifactTarget { - entryRelPath: string; - generatedFileName: string; - constName: string; - getterName: string; - /** Filename shown in the esbuild-failure error. */ - entryLabel: string; - /** Human phrase for the generated getter's doc comment, e.g. "audio-FX runtime". */ - docLabel: string; - event: string; -} - -const TARGETS: Record = { - "audio-fx": { - entryRelPath: "stubs/audio-fx-runtime-entry.ts", - generatedFileName: "audio-fx-runtime-inline.ts", - constName: "AUDIO_FX_RUNTIME_IIFE", - getterName: "getAudioFxRuntimeScript", - entryLabel: "audio-fx-runtime-entry.ts", - docLabel: "audio-FX runtime", - event: "audio_fx_runtime_generated", - }, - "position-edits": { - entryRelPath: "stubs/position-edits-render-entry.ts", - generatedFileName: "position-edits-render-inline.ts", - constName: "POSITION_EDITS_RENDER_IIFE", - getterName: "getPositionEditsRenderScript", - entryLabel: "position-edits-render-entry.ts", - docLabel: "position-edits render", - event: "position_edits_render_generated", - }, -}; - -const key = process.argv[2] ?? ""; -const target = TARGETS[key]; -if (!target) { - throw new Error(`Usage: build-inline-artifact.ts <${Object.keys(TARGETS).join("|")}>`); -} - -const thisDir = dirname(fileURLToPath(import.meta.url)); -const repoRoot = resolve(thisDir, ".."); -const entry = resolve(repoRoot, target.entryRelPath); -const generatedDir = resolve(repoRoot, "src/generated"); -const outPath = resolve(generatedDir, target.generatedFileName); - -const result = buildSync({ - entryPoints: [entry], - bundle: true, - write: false, - platform: "browser", - format: "iife", - target: ["es2020"], - minify: true, - legalComments: "none", -}); -const iife = result.outputFiles[0]?.text ?? ""; -if (!iife) throw new Error(`esbuild produced no output for ${target.entryLabel}`); - -mkdirSync(generatedDir, { recursive: true }); -writeFileSync( - outPath, - [ - "// AUTO-GENERATED by scripts/build-inline-artifact.ts - do not edit", - `const ${target.constName}: string = ${JSON.stringify(iife)};`, - "", - `/** Returns the pre-built ${target.docLabel} IIFE as a string constant. */`, - `export function ${target.getterName}(): string {`, - ` return ${target.constName};`, - "}", - "", - ].join("\n"), - "utf8", -); - -try { - execFileSync("bun", ["x", "oxfmt", outPath], { stdio: "ignore" }); -} catch { - // Formatting is best effort when the generator runs in a minimal environment. -} - -console.log(JSON.stringify({ event: target.event, outPath, bytes: iife.length })); diff --git a/packages/core/scripts/build-position-edits-render.ts b/packages/core/scripts/build-position-edits-render.ts new file mode 100644 index 000000000..a750cd791 --- /dev/null +++ b/packages/core/scripts/build-position-edits-render.ts @@ -0,0 +1,13 @@ +/** Build the injectable position-edits render artifact from the canonical runtime. */ + +import { buildInjectedArtifact } from "./buildInjectedArtifact.js"; + +buildInjectedArtifact({ + scriptUrl: import.meta.url, + entry: "stubs/position-edits-render-entry.ts", + out: "position-edits-render-inline.ts", + constName: "POSITION_EDITS_RENDER_IIFE", + fnName: "getPositionEditsRenderScript", + what: "position-edits render IIFE", + event: "position_edits_render_generated", +}); diff --git a/packages/core/scripts/buildInjectedArtifact.ts b/packages/core/scripts/buildInjectedArtifact.ts new file mode 100644 index 000000000..b7faa7d0f --- /dev/null +++ b/packages/core/scripts/buildInjectedArtifact.ts @@ -0,0 +1,78 @@ +/** + * Bundle a stub entry into an injectable IIFE, wrapped as a TypeScript constant. + * + * Two artifacts are built this way — the audio-FX runtime and the position-edits + * render — and the engine injects both into the headless browser as a script tag. + * They were two copies of this file differing in five names, which is a poor + * place for a divergence to hide: whichever copy stopped being edited would go on + * producing a subtly different artifact with nothing to say so. + */ + +import { execFileSync } from "node:child_process"; +import { mkdirSync, writeFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { buildSync } from "esbuild"; + +export interface InjectedArtifact { + /** The build script's own `import.meta.url`, so paths resolve beside it. */ + scriptUrl: string; + /** Entry stub, relative to the package root. */ + entry: string; + /** Output file name inside `src/generated`. */ + out: string; + /** SCREAMING_CASE name for the string constant holding the IIFE. */ + constName: string; + /** The accessor the rest of the codebase imports. */ + fnName: string; + /** What that accessor returns, for its doc comment: "the pre-built X". */ + what: string; + /** Structured log event name. */ + event: string; +} + +export function buildInjectedArtifact(spec: InjectedArtifact): void { + const scriptDir = dirname(fileURLToPath(spec.scriptUrl)); + const scriptName = spec.scriptUrl.split("/").pop() ?? ""; + const repoRoot = resolve(scriptDir, ".."); + const entry = resolve(repoRoot, spec.entry); + const generatedDir = resolve(repoRoot, "src/generated"); + const outPath = resolve(generatedDir, spec.out); + + const result = buildSync({ + entryPoints: [entry], + bundle: true, + write: false, + platform: "browser", + format: "iife", + target: ["es2020"], + minify: true, + legalComments: "none", + }); + const iife = result.outputFiles[0]?.text ?? ""; + if (!iife) throw new Error(`esbuild produced no output for ${spec.entry.split("/").pop()}`); + + mkdirSync(generatedDir, { recursive: true }); + writeFileSync( + outPath, + [ + `// AUTO-GENERATED by scripts/${scriptName} - do not edit`, + `const ${spec.constName}: string = ${JSON.stringify(iife)};`, + "", + `/** Returns the pre-built ${spec.what} as a string constant. */`, + `export function ${spec.fnName}(): string {`, + ` return ${spec.constName};`, + "}", + "", + ].join("\n"), + "utf8", + ); + + try { + execFileSync("bun", ["x", "oxfmt", outPath], { stdio: "ignore" }); + } catch { + // Formatting is best effort when the generator runs in a minimal environment. + } + + console.log(JSON.stringify({ event: spec.event, outPath, bytes: iife.length })); +} diff --git a/packages/core/src/audio/audioFxGraph.test.ts b/packages/core/src/audio/audioFxGraph.test.ts index f1eca2b74..181bf0826 100644 --- a/packages/core/src/audio/audioFxGraph.test.ts +++ b/packages/core/src/audio/audioFxGraph.test.ts @@ -25,13 +25,17 @@ class FakeNode { Q = new FakeParam(); gain = new FakeParam(); delayTime = new FakeParam(); + playbackRate = new FakeParam(); + loop = false; type = ""; curve: Float32Array | null = null; oversample = "none"; - buffer: unknown = null; + buffer: FakeBuffer | null = null; normalize = true; port = { postMessage: (m: unknown) => this.messages.push(m) }; messages: unknown[] = []; + /** `start(when, offset)` — the offset is the LFO's phase, so it is asserted. */ + startArgs: (number | undefined)[] | null = null; constructor(public kind: string) {} connect(next: FakeNode): FakeNode { this.connections.push(next); @@ -40,10 +44,23 @@ class FakeNode { disconnect(): void { this.disconnected = true; } - start(): void {} + start(...args: (number | undefined)[]): void { + this.startArgs = args; + } stop(): void {} } +/** One channel, kept across `getChannelData` calls so what is written can be read. */ +class FakeBuffer { + private data: Float32Array; + constructor(public length: number) { + this.data = new Float32Array(length); + } + getChannelData(): Float32Array { + return this.data; + } +} + class FakeCtx { sampleRate = 48000; created: FakeNode[] = []; @@ -67,6 +84,9 @@ class FakeCtx { createOscillator() { return this.make("osc"); } + createBufferSource() { + return this.make("bufferSource"); + } createWaveShaper() { return this.make("waveshaper"); } @@ -74,7 +94,7 @@ class FakeCtx { return this.make("convolver"); } createBuffer(_c: number, length: number) { - return { length, getChannelData: () => new Float32Array(length) }; + return new FakeBuffer(length); } } @@ -350,19 +370,91 @@ describe("levels and per-channel state", () => { }); it("sets the phaser LFO waveform it declares", () => { - const ctx = new FakeCtx(); - buildFxNode(ctx as unknown as BaseAudioContext, "phaser", { - ...defaultAudioFxParams("phaser"), - type: "0", + // A quarter of the way through the cycle both waveforms peak at 1, so the + // eighth is where they part: a sine is at sin(π/4), a triangle halfway up. + const eighth = (c: FakeCtx): number => { + const buffer = c.created.find((n) => n.kind === "bufferSource")?.buffer; + if (!buffer) throw new Error("no LFO buffer"); + return buffer.getChannelData()[Math.round(buffer.length / 8)] ?? 0; + }; + const build = (type: string): FakeCtx => { + const c = new FakeCtx(); + buildFxNode(c as unknown as BaseAudioContext, "phaser", { + ...defaultAudioFxParams("phaser"), + type, + }); + return c; + }; + expect(eighth(build("0"))).toBeCloseTo(0.5, 3); + expect(eighth(build("1"))).toBeCloseTo(Math.SQRT1_2, 3); + }); + + /** + * The waveform is baked into a buffer at construction, so pushing a type change + * into the running graph would be a no-op — preview would keep sweeping on a + * triangle while the render used the sine the attribute now says. + */ + it("rebuilds a phaser when its LFO waveform changes", () => { + const phaser = (params: Record): HfAudioFxChain => ({ + version: 1, + nodes: [ + { + type: "phaser", + enabled: true, + params: { ...defaultAudioFxParams("phaser"), type: "0", ...params }, + }, + ], }); - const osc = ctx.created.find((n) => n.kind === "osc"); - expect(osc?.type).toBe("triangle"); - const ctx2 = new FakeCtx(); - buildFxNode(ctx2 as unknown as BaseAudioContext, "phaser", { - ...defaultAudioFxParams("phaser"), - type: "1", - }); - expect(ctx2.created.find((n) => n.kind === "osc")?.type).toBe("sine"); + const built = buildFxChain(asCtx(ctx()), phaser({})); + expect(built.update(phaser({ type: "1" }))).toBe(false); + // Everything else about a phaser still updates in place. + const other = buildFxChain(asCtx(ctx()), phaser({})); + expect(other.update(phaser({ speed: 2 }))).toBe(true); + }); + + /** + * An LFO's phase is the whole reason it is a looping buffer rather than an + * OscillatorNode, whose phase is zero at `start()` and cannot be set. + * + * Preview rebuilds the graph mid-play — a seek, a scrub, any structural edit — + * and an oscillator restarted there put the chorus at the top of its sweep + * wherever the playhead happened to be, so preview disagreed with the render + * and with itself across an edit. + */ + it("starts a modulated effect's LFO at the phase the clip has reached", () => { + // 3.5 s at 2 Hz is seven whole cycles: back at phase zero. + const whole = ctx(); + buildFxNode(asCtx(whole), "chorus", { ...defaultAudioFxParams("chorus"), speed: 2 }, 3.5); + expect(whole.created.find((n) => n.kind === "bufferSource")?.startArgs?.[1]).toBeCloseTo(0, 6); + // 3.6 s at 2 Hz is seven cycles and a fifth. + const part = ctx(); + buildFxNode(asCtx(part), "chorus", { ...defaultAudioFxParams("chorus"), speed: 2 }, 3.6); + const src = part.created.find((n) => n.kind === "bufferSource"); + expect(src?.startArgs?.[1]).toBeCloseTo(0.2, 6); + expect(src?.loop).toBe(true); + // One second of waveform: the rate reads in Hz, so a speed lane needs no map. + expect(src?.playbackRate.value).toBeCloseTo(2, 6); + }); + + /** + * A source node is not retired by disconnecting what it feeds. The chorus and + * phaser stopped their LFO and left it out of the nodes they disconnect, so + * every rebuild that dropped one left a modulator still wired to the delay or + * the allpass bank it had been driving. + */ + it("unwires a modulated effect's LFO when the effect is disposed", () => { + for (const type of ["chorus", "phaser"]) { + const c = ctx(); + buildFxNode(asCtx(c), type, defaultAudioFxParams(type)).dispose(); + const lfo = c.created.find((node) => node.kind === "bufferSource"); + expect(lfo?.disconnected, `${type} left its LFO connected`).toBe(true); + } + }); + + it("starts the LFO at zero for a render, which always begins at the clip's start", () => { + const c = ctx(); + buildFxNode(asCtx(c), "phaser", defaultAudioFxParams("phaser")); + expect(c.created.find((n) => n.kind === "bufferSource")?.startArgs?.[1]).toBe(0); }); it("rebuilds a one-pole filter when its cutoff moves", () => { diff --git a/packages/core/src/audio/audioFxGraph.ts b/packages/core/src/audio/audioFxGraph.ts index bcd0c8c7c..6adab53b8 100644 --- a/packages/core/src/audio/audioFxGraph.ts +++ b/packages/core/src/audio/audioFxGraph.ts @@ -83,13 +83,86 @@ export interface FxNodeHandle { dispose(): void; } -type Builder = (ctx: BaseAudioContext, p: HfAudioFxParamValues) => FxNodeHandle; +/** + * `elapsed` is the clip-relative time, in seconds, the graph is being built at. + * + * Zero for the render, which always starts a clip's audio from its first sample, + * and zero for a preview attached before playback. It is non-zero in the one case + * that used to be wrong: preview rebuilding the graph mid-play — a seek, a scrub, + * or any structural edit — where an LFO restarting from phase 0 made preview + * disagree with the render, and with itself across an edit. + */ +type Builder = (ctx: BaseAudioContext, p: HfAudioFxParamValues, elapsed: number) => FxNodeHandle; const n = (v: number | string | undefined): number => (typeof v === "number" ? v : Number(v ?? 0)); /** Milliseconds on the knob, seconds on the AudioParam. */ const msToSec = (v: number): number => v / 1000; +/** + * An LFO with a settable phase. + * + * An OscillatorNode cannot have one: its phase is zero at `start()`, and + * `start(when)` clamps a past `when` to now. So the modulator is one cycle of the + * waveform in a looping buffer instead, where `start(when, offset)` *is* a phase + * control. + * + * The buffer holds exactly one second, so it plays at 1 Hz at the default rate + * and `playbackRate` reads directly in Hz — which is what the `speed` knob is in, + * and what an automation lane aimed at it writes, so neither needs a mapping. + * + * Phase is taken as `elapsed × speed`, which is exact for the constant speed this + * is built with. A lane that sweeps `speed` advances the real phase by its + * integral, so a graph rebuilt mid-sweep resumes fractionally off — smaller than + * the whole-cycle error this replaces, and not worth integrating a curve for. + */ +function lfoSource( + ctx: BaseAudioContext, + wave: "sine" | "triangle", + speed: number, + elapsed: number, +): AudioBufferSourceNode { + const length = Math.max(1, Math.round(ctx.sampleRate)); + const buffer = ctx.createBuffer(1, length, ctx.sampleRate); + const cycle = buffer.getChannelData(0); + for (let i = 0; i < length; i++) { + const phase = i / length; + // Both start at zero and rise, the convention an OscillatorNode uses, so a + // render — which builds at elapsed 0 — is unmoved by this change. + cycle[i] = + wave === "sine" + ? Math.sin(2 * Math.PI * phase) + : 4 * Math.abs(((phase + 0.75) % 1) - 0.5) - 1; + } + const src = ctx.createBufferSource(); + src.buffer = buffer; + src.loop = true; + src.playbackRate.value = speed; + // A negative `offset` throws, and `elapsed` is only trusted to be a number. + const offset = ((((elapsed * speed) % 1) + 1) % 1) * (length / ctx.sampleRate); + src.start(typeof ctx.currentTime === "number" ? ctx.currentTime : 0, offset); + return src; +} + +/** + * Retire an LFO: stopped *and* unwired. + * + * Both halves. The old oscillators were stopped and left in their builder's + * dispose list — so every chain rebuild that dropped a chorus or a phaser left a + * modulator still connected to the delay or the allpass bank it had been + * driving. Nothing audible came out of it, because the shell around it was + * disconnected, but the nodes stayed reachable and a session of edits to a + * modulated track accumulated them. Same shape as the worklet leak above. + */ +function retireLfo(src: AudioBufferSourceNode): void { + try { + src.stop(); + } catch { + /* already stopped */ + } + src.disconnect(); +} + /** A wet/dry pair: the dry side is whatever the wet side is not. */ function mixTargets(wet: AudioParam, dry: AudioParam): FxParamTarget[] { return [{ param: wet }, { param: dry, map: (v) => 1 - v }]; @@ -276,22 +349,21 @@ const delayFeedback: Builder = (ctx, p) => { }; }; -const chorusLfo: Builder = (ctx, p) => { +const chorusLfo: Builder = (ctx, p, elapsed) => { const input = ctx.createGain(); const out = ctx.createGain(); const dl = ctx.createDelay(0.5); - const lfo = ctx.createOscillator(); + const lfo = lfoSource(ctx, "sine", n(p.speed), elapsed); const depth = ctx.createGain(); const wet = ctx.createGain(); const dry = ctx.createGain(); lfo.connect(depth).connect(dl.delayTime); input.connect(dl).connect(wet).connect(out); input.connect(dry).connect(out); - lfo.start(); const apply = (v: HfAudioFxParamValues): void => { dl.delayTime.value = n(v.delay) / 1000; depth.gain.value = n(v.depth) / 1000; - lfo.frequency.value = n(v.speed); + lfo.playbackRate.value = n(v.speed); setWetDryMix(wet, dry, n(v.mix)); }; apply(p); @@ -302,15 +374,12 @@ const chorusLfo: Builder = (ctx, p) => { automation: { delay: [{ param: dl.delayTime, map: msToSec }], depth: [{ param: depth.gain, map: msToSec }], - speed: [{ param: lfo.frequency }], + // One second of waveform, so the rate is the frequency in Hz the knob names. + speed: [{ param: lfo.playbackRate }], mix: mixTargets(wet.gain, dry.gain), }, dispose: () => { - try { - lfo.stop(); - } catch { - /* already stopped */ - } + retireLfo(lfo); [input, out, dl, depth, wet, dry].forEach((x) => x.disconnect()); }, }; @@ -318,7 +387,7 @@ const chorusLfo: Builder = (ctx, p) => { const PHASER_STAGES = 6; -const allpassPhaser: Builder = (ctx, p) => { +const allpassPhaser: Builder = (ctx, p, elapsed) => { const input = ctx.createGain(); const out = ctx.createGain(); // aphaser's in_gain/out_gain trim the signal entering and leaving the effect. @@ -327,7 +396,12 @@ const allpassPhaser: Builder = (ctx, p) => { // track level. const inTrim = ctx.createGain(); const outTrim = ctx.createGain(); - const lfo = ctx.createOscillator(); + // aphaser's type 0 is triangular, 1 sinusoidal. The builder once left this + // unset, so the declared default ("Triangular") was silently a sine. The + // waveform is baked into the LFO's buffer, so switching it is a shape change + // that rebuilds the chain rather than a value pushed into the running graph — + // see `shapeOf`. + const lfo = lfoSource(ctx, String(p.type) === "1" ? "sine" : "triangle", n(p.speed), elapsed); const depth = ctx.createGain(); const wet = ctx.createGain(); const dry = ctx.createGain(); @@ -344,11 +418,6 @@ const allpassPhaser: Builder = (ctx, p) => { stages.push(ap); } lfo.connect(depth); - // aphaser's type 0 is triangular, 1 sinusoidal. The builder never set this, so - // the declared default ("Triangular") was silently a sine. An OscillatorNode - // has no triangle-with-the-same-phase primitive to switch to, so triangle is - // the node's own "triangle" type. - lfo.start(); node.connect(wet).connect(outTrim); inTrim.connect(dry).connect(outTrim); outTrim.connect(out); @@ -358,8 +427,7 @@ const allpassPhaser: Builder = (ctx, p) => { const centre = 1000 / Math.max(0.1, n(v.delay)); for (const ap of stages) ap.frequency.value = centre; depth.gain.value = centre * n(v.decay); - lfo.frequency.value = n(v.speed); - lfo.type = String(v.type) === "1" ? "sine" : "triangle"; + lfo.playbackRate.value = n(v.speed); inTrim.gain.value = n(v.in_gain); outTrim.gain.value = n(v.out_gain); // Summed at unity: the sweep is the effect, not a blend control. @@ -374,7 +442,7 @@ const allpassPhaser: Builder = (ctx, p) => { // `delay` and `decay` set the sweep centre, which feeds every stage's // frequency at once — not one knob, one param — so they stay unautomated. automation: { - speed: [{ param: lfo.frequency }], + speed: [{ param: lfo.playbackRate }], // The trims, not wet/dry. apply() drives inTrim/outTrim from these knobs // and pins wet and dry to 1 — so a lane aimed at wet/dry modulated a // constant and left the trim frozen, and the next values-only edit slammed @@ -384,11 +452,7 @@ const allpassPhaser: Builder = (ctx, p) => { out_gain: [{ param: outTrim.gain }], }, dispose: () => { - try { - lfo.stop(); - } catch { - /* already stopped */ - } + retireLfo(lfo); [input, out, inTrim, outTrim, depth, wet, dry, ...stages].forEach((x) => x.disconnect()); }, }; @@ -456,17 +520,18 @@ export function buildFxNode( ctx: BaseAudioContext, type: string, params: HfAudioFxParamValues, + elapsed = 0, ): FxNodeHandle { const def = getAudioFxDef(type); if (!def) throw new Error(`Unknown effect type: ${type}`); const resolved = normalizeAudioFxParams(type, params); // One-pole is a different node type, not a different parameter value. if ((type === "highpass" || type === "lowpass") && String(resolved.poles) === "1") { - return onePoleBuilder(type)(ctx, resolved); + return onePoleBuilder(type)(ctx, resolved, elapsed); } const builder = BUILDERS[def.web]; if (!builder) throw new Error(`No Web Audio builder for ${def.web}`); - return builder(ctx, resolved); + return builder(ctx, resolved, elapsed); } export interface FxChainHandle { @@ -495,7 +560,11 @@ function shapeOf(chain: HfAudioFxChain): string { // being pushed into a no-op updater — which is what let preview keep // filtering at the old frequency while the render used the new one. const fixedFreq = String(p.poles) === "1" ? `@${p.frequency}` : ""; - return `${node.type}${poles}${fixedFreq}`; + // The phaser's LFO waveform is baked into a buffer at construction, for the + // same reason: pushed into the running graph it would be a no-op, and + // preview would keep sweeping on a triangle while the render used a sine. + const wave = node.type === "phaser" ? `~${p.type}` : ""; + return `${node.type}${poles}${fixedFreq}${wave}`; }) .join("|"); } @@ -503,15 +572,23 @@ function shapeOf(chain: HfAudioFxChain): string { /** * Build the whole chain in series. Returns a handle whose `input`/`output` can * be spliced into any graph; an empty chain yields a pass-through. + * + * `elapsed` is where in the clip this is being built — see `Builder`. It only + * reaches the modulated effects, and only matters when the graph is built after + * the audio has already started. */ -export function buildFxChain(ctx: BaseAudioContext, chain: HfAudioFxChain): FxChainHandle { +export function buildFxChain( + ctx: BaseAudioContext, + chain: HfAudioFxChain, + elapsed = 0, +): FxChainHandle { const input = ctx.createGain(); const output = ctx.createGain(); const handles: { id?: string; type: string; handle: FxNodeHandle }[] = []; let tail: AudioNode = input; for (const node of enabledAudioFxNodes(chain)) { - const handle = buildFxNode(ctx, node.type, node.params ?? {}); + const handle = buildFxNode(ctx, node.type, node.params ?? {}, elapsed); tail.connect(handle.input); tail = handle.output; handles.push({ ...(node.id ? { id: node.id } : {}), type: node.type, handle }); diff --git a/packages/core/src/generated/position-edits-render-inline.ts b/packages/core/src/generated/position-edits-render-inline.ts index 0004bc6d4..82cac49a9 100644 --- a/packages/core/src/generated/position-edits-render-inline.ts +++ b/packages/core/src/generated/position-edits-render-inline.ts @@ -1,4 +1,4 @@ -// AUTO-GENERATED by scripts/build-inline-artifact.ts - do not edit +// AUTO-GENERATED by scripts/build-position-edits-render.ts - do not edit const POSITION_EDITS_RENDER_IIFE: string = '"use strict";(()=>{function A(){return globalThis}function P(e,t){if(typeof window>"u")return;let n=A(),r=n.__hf?.onSwallowed;if(r)try{r({label:e,error:t})}catch(i){}(n.__hfDebug||n.__HYPERFRAMES_DEBUG)&&console.debug(`[hyperframes] ${e} swallowed:`,t)}var T=null;function R(e,t){if(T)try{T({source:"hf-preview",type:"analytics",event:e,properties:t??{}})}catch(n){P("runtime.analytics.site1",n)}}var w="data-hf-edit-base-x",k="data-hf-edit-base-y",y="data-hf-edit-original-translate",S=e=>{let t=parseFloat(e??"");return Number.isFinite(t)?t:0},V=e=>{let t=[],n=0,r="";for(let i of e.trim())i==="("&&(n+=1),i===")"&&(n=Math.max(0,n-1)),/\\s/.test(i)&&n===0?(r&&t.push(r),r=""):r+=i;return r&&t.push(r),t},$=/^-?(?:\\d+(?:\\.\\d+)?|\\.\\d+)px$/,E=(e,t)=>$.test(e)&&$.test(t)?`${parseFloat(e)+parseFloat(t)}px`:`calc(${e} + ${t})`,I=(e,t,n)=>{if(!e||e==="none")return`${t} ${n}`;let[r,i,u]=V(e);if(r===void 0)return`${t} ${n}`;if(i===void 0)return`${E(r,t)} ${n}`;let d=u===void 0?"":` ${u}`;return`${E(r,t)} ${E(i,n)}${d}`},O=e=>{try{e.ownerDocument.defaultView?.gsap?.getProperty?.(e,"x")}catch{}},G=e=>{let t=e.style.getPropertyValue("translate").trim();if(t)return t==="none"?"":t;try{let n=e.ownerDocument.defaultView,r=n?n.getComputedStyle(e).getPropertyValue("translate").trim():"";return r==="none"?"":r}catch{return""}},h=new WeakMap;function H(e,t){let n=h.get(e);if(!t?.force&&n!==void 0&&e.style.getPropertyValue("translate")!==n){R("position_edit_fold_skipped",{hfId:e.getAttribute("data-hf-id")});return}let r=S(e.getAttribute("data-x"))-S(e.getAttribute(w)),i=S(e.getAttribute("data-y"))-S(e.getAttribute(k));e.hasAttribute(y)||e.setAttribute(y,G(e)),n===void 0&&O(e);let u=e.getAttribute(y)??"",d=I(u,`${r}px`,`${i}px`);e.style.setProperty("translate",d),h.set(e,e.style.getPropertyValue("translate"))}function v(e,t){let n=e.defaultView?.HTMLElement,r=e.defaultView?.SVGElement,i=a=>n||r?n!==void 0&&a instanceof n||r!==void 0&&a instanceof r:typeof a.style?.setProperty=="function",u=e.querySelectorAll(`[${y}]:not([${w}]):not([${k}])`);for(let a=0;a{try{v(t.document)}catch{}},r=o=>typeof o=="function"&&(D.has(o)||!!o[x]),i=o=>{D.add(o);try{Object.defineProperty(o,x,{value:!0})}catch{}},u=o=>{if(typeof o!="function"||r(o))return o;let s=function(...c){let f=o.apply(this,c);return n(),f};return i(s),s},d=(o,s)=>{let c=F.get(o);if(c?.has(s))return!0;let f=Object.getOwnPropertyDescriptor(o,s);if(f?.configurable===!1){let p=o[s];return typeof p=="function"&&(o[s]=u(p),n()),!1}let g=o[s],_=f?.set;return Object.defineProperty(o,s,{configurable:!0,enumerable:f?.enumerable??!0,get:()=>g,set:p=>{g=u(p),_?.call(o,p)}}),g=u(g),c??(c=new Set),c.add(s),F.set(o,c),n(),!0},m=(o,s)=>{let c=W.get(t),f=Object.getOwnPropertyDescriptor(t,o);if(!c?.has(o)){if(f?.configurable===!1){let p=t[o];return p?d(p,s):!1}let _=t[o];Object.defineProperty(t,o,{configurable:!0,enumerable:f?.enumerable??!0,get:()=>_,set:p=>{_=p,_&&d(_,s)}}),c??(c=new Set),c.add(o),W.set(t,c)}let g=t[o];return g?d(g,s):!1},a=()=>{let o=m("__hf","seek"),s=m("__player","renderSeek");return o&&s};if(a())return;let l=120,b=t.setInterval(()=>{if(a()){t.clearInterval(b);return}l-=1,l<=0&&t.clearInterval(b)},50)}function M(){document.querySelector(`[${w}], [${k}]`)&&(v(document),L(window))}document.readyState==="loading"?document.addEventListener("DOMContentLoaded",M,{once:!0}):M();})();\n'; diff --git a/packages/core/src/runtime/audioFx.test.ts b/packages/core/src/runtime/audioFx.test.ts index 09374e12e..35db1e291 100644 --- a/packages/core/src/runtime/audioFx.test.ts +++ b/packages/core/src/runtime/audioFx.test.ts @@ -16,6 +16,10 @@ class Node { Q = { value: 0 }; gain = { value: 0 }; delayTime = { value: 0 }; + playbackRate = { value: 0 }; + loop = false; + /** `start(when, offset)` — the offset is an LFO's phase, so it is asserted. */ + startArgs: (number | undefined)[] | null = null; type = ""; curve: Float32Array | null = null; oversample = "none"; @@ -28,7 +32,9 @@ class Node { disconnect(): void { this.disconnected = true; } - start(): void {} + start(...args: (number | undefined)[]): void { + this.startArgs = args; + } stop(): void {} } class Ctx { @@ -48,6 +54,9 @@ class Ctx { createOscillator() { return new Node(); } + createBufferSource() { + return new Node(); + } createWaveShaper() { return new Node(); } @@ -304,6 +313,67 @@ describe("attachElementFxChain", () => { expect(node.getAttribute("data-fx-chain")).toContain("1200"); }); + /** + * A rebuild hands the graph the clip position it happens at, so an LFO + * resumes at the phase the render would be at rather than restarting. + * + * Without it, every structural edit — and every seek, which rebuilds the same + * way — put a chorus back at the top of its sweep wherever the playhead was: + * preview disagreeing with the render, and with itself across an edit. + */ + it("hands a rebuilt graph the playhead it happens at", async () => { + const clock = { currentTime: 0 }; + class ClockCtx extends Ctx { + made: Node[] = []; + get currentTime(): number { + return clock.currentTime; + } + override createBufferSource(): Node { + const node = new Node(); + this.made.push(node); + return node; + } + } + const withChorus = (mix: number) => ({ + version: 1, + nodes: [ + { + type: "chorus", + id: "n1", + params: { ...defaultAudioFxParams("chorus"), speed: 2, mix }, + }, + ], + }); + const ctxClock = new ClockCtx(); + const node = audioEl(withChorus(0.5)); + attachElementFxChain( + ctxClock as unknown as BaseAudioContext, + node, + new Node() as never, + new Node() as never, + { scheduledAt: 0, elapsed: 0, rate: 1 }, + ); + // Attached at the clip's start: phase zero, the same as a render. + expect(ctxClock.made[0]?.startArgs?.[1]).toBeCloseTo(0, 6); + + // 3.6 s later, and structural — a bypass, so the shape changes and the + // graph is rebuilt rather than re-parameterised. + clock.currentTime = 3.6; + node.setAttribute( + "data-fx-chain", + JSON.stringify({ + version: 1, + nodes: [ + ...withChorus(0.5).nodes, + { type: "peaking", id: "n2", params: defaultAudioFxParams("peaking") }, + ], + }), + ); + await settle(); + // 3.6 s at 2 Hz is seven cycles and a fifth. + expect(ctxClock.made.at(-1)?.startArgs?.[1]).toBeCloseTo(0.2, 6); + }); + it("keeps playing dry when an edit leaves the chain unreadable", async () => { const src = new Node(); const dst = new Node(); @@ -689,6 +759,9 @@ describe("attachElementFxChain", () => { override Q = new RecordingParam() as unknown as { value: number }; override gain = new RecordingParam() as unknown as { value: number }; override delayTime = new RecordingParam() as unknown as { value: number }; + // A modulated effect's `speed` lane drives its LFO's rate, which is where a + // looping buffer keeps the frequency an oscillator kept on `frequency`. + override playbackRate = new RecordingParam() as unknown as { value: number }; port = { postMessage: () => {} }; } class RichCtx extends Ctx { @@ -720,6 +793,9 @@ describe("attachElementFxChain", () => { override createOscillator() { return this.make(); } + override createBufferSource() { + return this.make(); + } override createWaveShaper() { return this.make(); } @@ -790,7 +866,7 @@ describe("attachElementFxChain", () => { expect(src.connections.at(-1), `${def.id} was left out of the path`).not.toBe(dst); if (automatable) { const scheduled = ctxRich.made.some((n) => - [n.frequency, n.Q, n.gain, n.delayTime].some( + [n.frequency, n.Q, n.gain, n.delayTime, n.playbackRate].some( (p) => (p as unknown as RecordingParam).scheduled, ), ); diff --git a/packages/core/src/runtime/audioFx.ts b/packages/core/src/runtime/audioFx.ts index d2b678976..d873bcd91 100644 --- a/packages/core/src/runtime/audioFx.ts +++ b/packages/core/src/runtime/audioFx.ts @@ -150,7 +150,7 @@ export function attachElementFxChain( * A chain that cannot be realised — an unregistered worklet, an unknown * effect — plays dry rather than silencing the track. */ - const attach = (next: HfAudioFxChain): void => { + const attach = (next: HfAudioFxChain, elapsed: number): void => { if (next.nodes.length === 0) { source.connect(destination); return; @@ -178,7 +178,9 @@ export function attachElementFxChain( return; } try { - const built = buildFxChain(ctx, next); + // Where the clip has got to, so a modulated effect resumes at the phase the + // render would be at rather than restarting its LFO from zero. + const built = buildFxChain(ctx, next, elapsed); source.connect(built.input); built.output.connect(destination); handle = built; @@ -198,7 +200,9 @@ export function attachElementFxChain( // envelope at the wrong clip position. let frame: AutomationTiming | null = timing ? { ...timing } : null; - attach(chain); + // `timingNow` is not in scope yet, and does not need to be: nothing has played + // between the frame being taken and this line. + attach(chain, frame?.elapsed ?? 0); scheduleFor(chain, frame); /** @@ -235,7 +239,7 @@ export function attachElementFxChain( const at = timingNow(); cancelParamLane(automated, at?.scheduledAt ?? 0); detach(); - attach(next); + attach(next, at?.elapsed ?? 0); scheduleFor(next, at); }; diff --git a/packages/engine/src/services/audioFxRender.ts b/packages/engine/src/services/audioFxRender.ts index 950b5ce46..fb5af4f44 100644 --- a/packages/engine/src/services/audioFxRender.ts +++ b/packages/engine/src/services/audioFxRender.ts @@ -21,6 +21,7 @@ import { enabledAudioFxNodes, type HfAudioFxChain } from "@hyperframes/core/audi import { serializeAutomation, type HfAutomation } from "@hyperframes/core/audio-automation"; import { acquireBrowser } from "./browserManager.js"; import { createEnvelopeWalker } from "./audioVolumeEnvelope.js"; +import { riffChunks } from "./wavChunks.js"; import type { AudioVolumeKeyframe } from "./audioMixer.types.js"; export class AudioFxRenderError extends Error { @@ -49,22 +50,22 @@ function readWavChunks(buf: Buffer): { bits: number; data?: Buffer; } { - let offset = 12; const head = { format: 1, channels: 1, sampleRate: 48000, bits: 16 }; let data: Buffer | undefined; - while (offset + 8 <= buf.length) { - const id = buf.toString("ascii", offset, offset + 4); - const size = buf.readUInt32LE(offset + 4); + for (const { id, body, size } of riffChunks(buf)) { if (id === "fmt ") { - head.format = buf.readUInt16LE(offset + 8); - head.channels = buf.readUInt16LE(offset + 10); - head.sampleRate = buf.readUInt32LE(offset + 12); - head.bits = buf.readUInt16LE(offset + 22); + head.format = buf.readUInt16LE(body); + head.channels = buf.readUInt16LE(body + 2); + head.sampleRate = buf.readUInt32LE(body + 4); + head.bits = buf.readUInt16LE(body + 14); } else if (id === "data") { - data = buf.subarray(offset + 8, Math.min(buf.length, offset + 8 + size)); + data = buf.subarray(body, Math.min(buf.length, body + size)); + // The payload is the rest of the file for anything the mixer writes, and + // reading past it buys nothing: `fmt ` precedes `data` in every WAV these + // steps produce, and the alternative is walking a several-hundred-megabyte + // tail chunk by chunk. break; } - offset += 8 + size + (size % 2); } return { ...head, data }; } diff --git a/packages/engine/src/services/audioVolumeEnvelope.ts b/packages/engine/src/services/audioVolumeEnvelope.ts index 08f8a1882..d64cc33a4 100644 --- a/packages/engine/src/services/audioVolumeEnvelope.ts +++ b/packages/engine/src/services/audioVolumeEnvelope.ts @@ -19,6 +19,7 @@ import { readFileSync, renameSync, writeFileSync } from "fs"; import { randomBytes } from "crypto"; import type { AudioVolumeKeyframe } from "./audioMixer.types.js"; import { normaliseEnvelope } from "@hyperframes/core/media-volume-envelope"; +import { riffChunks } from "./wavChunks.js"; const PCM_FORMAT = 1; // WAVE_FORMAT_PCM const SUPPORTED_BITS = 16; @@ -42,26 +43,20 @@ function parseWavLayout(buffer: Buffer): WavLayout | null { if (buffer.length < 12 || buffer.toString("ascii", 0, 4) !== "RIFF") return null; if (buffer.toString("ascii", 8, 12) !== "WAVE") return null; - let offset = 12; let fmt: { numChannels: number; sampleRate: number; bitsPerSample: number } | null = null; let data: { offset: number; size: number } | null = null; - while (offset + 8 <= buffer.length) { - const chunkId = buffer.toString("ascii", offset, offset + 4); - const chunkSize = buffer.readUInt32LE(offset + 4); - const body = offset + 8; - if (chunkId === "fmt " && body + 16 <= buffer.length) { + for (const { id, body, size } of riffChunks(buffer)) { + if (id === "fmt " && body + 16 <= buffer.length) { if (buffer.readUInt16LE(body) !== PCM_FORMAT) return null; fmt = { numChannels: buffer.readUInt16LE(body + 2), sampleRate: buffer.readUInt32LE(body + 4), bitsPerSample: buffer.readUInt16LE(body + 14), }; - } else if (chunkId === "data") { - data = { offset: body, size: Math.min(chunkSize, buffer.length - body) }; + } else if (id === "data") { + data = { offset: body, size: Math.min(size, buffer.length - body) }; } - // Chunks are word-aligned: an odd size carries a trailing pad byte. - offset = body + chunkSize + (chunkSize % 2); } if (!fmt || !data) return null; diff --git a/packages/engine/src/services/wavChunks.test.ts b/packages/engine/src/services/wavChunks.test.ts new file mode 100644 index 000000000..6ee943203 --- /dev/null +++ b/packages/engine/src/services/wavChunks.test.ts @@ -0,0 +1,56 @@ +import { describe, expect, it } from "vitest"; +import { riffChunks } from "./wavChunks.js"; + +/** + * The two WAV readers that share this walk both had their own copy, and neither + * had a test for the walk itself — a real WAV out of the mixer has an even-sized + * `fmt ` first and `data` last, so the two things the walk exists to handle + * (ordering, and the pad byte after an odd chunk) never came up in either suite. + */ +function riff(chunks: { id: string; body: Buffer }[]): Buffer { + const parts: Buffer[] = [Buffer.from("RIFF"), Buffer.alloc(4), Buffer.from("WAVE")]; + for (const { id, body } of chunks) { + const header = Buffer.alloc(8); + header.write(id, 0, "ascii"); + header.writeUInt32LE(body.length, 4); + parts.push(header, body); + // Word alignment: an odd body is followed by a pad byte the size excludes. + if (body.length % 2) parts.push(Buffer.alloc(1)); + } + const buf = Buffer.concat(parts); + buf.writeUInt32LE(buf.length - 8, 4); + return buf; +} + +describe("riffChunks", () => { + it("steps over the pad byte after an odd-sized chunk", () => { + // An odd LIST is what ffmpeg writes for a metadata string of odd length. Read + // without the pad, every chunk after it is one byte out and reads as garbage. + const buf = riff([ + { id: "LIST", body: Buffer.from("INFOodd") }, + { id: "data", body: Buffer.from([1, 2, 3, 4]) }, + ]); + const found = [...riffChunks(buf)]; + expect(found.map((c) => c.id)).toEqual(["LIST", "data"]); + const data = found[1]; + if (!data) throw new Error("no data chunk"); + expect(buf.subarray(data.body, data.body + data.size)).toEqual(Buffer.from([1, 2, 3, 4])); + }); + + it("yields chunks in file order, whatever that order is", () => { + // `data` before `fmt ` is legal and the reason the walk advances by declared + // size rather than assuming a layout. + const buf = riff([ + { id: "data", body: Buffer.alloc(6) }, + { id: "fmt ", body: Buffer.alloc(16) }, + ]); + expect([...riffChunks(buf)].map((c) => c.id)).toEqual(["data", "fmt "]); + }); + + it("stops at a chunk header that runs past the end of the file", () => { + // Truncated downloads and interrupted writes both land here; the walk must + // end rather than read off the buffer. + const buf = Buffer.concat([riff([{ id: "data", body: Buffer.alloc(4) }]), Buffer.from("da")]); + expect([...riffChunks(buf)].map((c) => c.id)).toEqual(["data"]); + }); +}); diff --git a/packages/engine/src/services/wavChunks.ts b/packages/engine/src/services/wavChunks.ts new file mode 100644 index 000000000..926514138 --- /dev/null +++ b/packages/engine/src/services/wavChunks.ts @@ -0,0 +1,38 @@ +/** + * The RIFF chunk walk, which two WAV readers in this directory each had a copy + * of: `audioFxRender`'s `readWavChunks` and `audioVolumeEnvelope`'s + * `parseWavLayout`. + * + * Only the walk is shared. What the two do with the chunks is genuinely + * different — one wants a slice of the payload and lets the decoder judge the + * format, the other wants offsets to edit in place and refuses anything that is + * not 16-bit PCM — and folding those together would mean picking one behaviour + * for each difference, in the parser every render's audio passes through. So + * this yields chunks and holds no policy at all. + */ + +export interface RiffChunk { + /** Four ASCII characters: `fmt `, `data`, `LIST`, `fact`, … */ + id: string; + /** Byte offset of the chunk's body, past the 8-byte header. */ + body: number; + /** The size the chunk declares. May run past the end of a truncated file. */ + size: number; +} + +/** + * Every chunk after the 12-byte RIFF header, in the order they sit. + * + * Advances by each chunk's declared size, so ordering is not assumed — `data` + * may precede `fmt `, and trailing LIST/fact chunks are walked past rather than + * tripped over. Chunks are word-aligned, so an odd size carries a pad byte. + */ +export function* riffChunks(buffer: Buffer): Generator { + let offset = 12; + while (offset + 8 <= buffer.length) { + const id = buffer.toString("ascii", offset, offset + 4); + const size = buffer.readUInt32LE(offset + 4); + yield { id, body: offset + 8, size }; + offset += 8 + size + (size % 2); + } +} diff --git a/scripts/check-no-main-deletions.mjs b/scripts/check-no-main-deletions.mjs index e23c33969..75371c4bf 100644 --- a/scripts/check-no-main-deletions.mjs +++ b/scripts/check-no-main-deletions.mjs @@ -57,6 +57,10 @@ export const ALLOWED_DELETIONS = new Map([ "packages/core/scripts/build-position-edits-render.ts", "merged into build-inline-artifact.ts, same reason as build-audio-fx-runtime.ts above", ], + [ + "packages/core/scripts/build-inline-artifact.ts", + "a later branch in this stack (wa-20b2-lfo-fixes) independently deduped the same two build scripts a different way — buildInjectedArtifact.ts plus two thin per-target files — before this consolidation and that one had merged; this branch's tree keeps that shape instead, so build-inline-artifact.ts is the one that goes.", + ], ]); export function parseBase(argv, fallback = "origin/main") {