From 93de149857f81ac68a7a515e3ef2ecd859060983 Mon Sep 17 00:00:00 2001 From: Vance Ingalls Date: Sat, 8 Aug 2026 11:28:20 -0700 Subject: [PATCH] docs(plans): hand off the audio review fixes for a fresh session MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eleven of fifteen findings fixed on wa-19-review-fixes; the four left open are recorded with the analysis already done on each, including why the clip-before- volume-lane one was started and deliberately stopped. Also carries the cut list, the cleanup list, the carve-reconcile-wip undo regression, and the nine traps that cost time — the published-CLI-is-not-a-symlink one above all. --- plans/audio-review-fixes-handoff.md | 275 ++++++++++++++++++++++++++++ 1 file changed, 275 insertions(+) create mode 100644 plans/audio-review-fixes-handoff.md diff --git a/plans/audio-review-fixes-handoff.md b/plans/audio-review-fixes-handoff.md new file mode 100644 index 0000000000..38182433fd --- /dev/null +++ b/plans/audio-review-fixes-handoff.md @@ -0,0 +1,275 @@ +# Audio FX code-review fixes — session handoff + +Written 2026-08-08 so a fresh session can continue without re-deriving anything. +Everything here was verified in this worktree unless it says otherwise. + +**Worktree:** `~/src/wt/hyperframes/webaudio-fx` +**Working branch:** `wa-19-review-fixes` — 11 commits, **local only, never pushed** +**Branched from:** `wa-18-lane-stretch` @ `4bcc7e188` + +--- + +## 1. Where everything is + +| Branch | State | +| --- | --- | +| `wa-18-lane-stretch` | **pushed**, matches remote. PR **#3058** open, ready for review, base `wa-17-lane-clipboard` | +| `wa-19-review-fixes` | 11 commits, **local only**. The review fixes. | +| `carve-reconcile-wip` | 1 commit (`7b57000a1`), **local only**. Working feature, but it breaks undo — see §5. | + +Every other `wa-*` branch is in sync with its remote. The handoff §6 warning about +needing to force-push 16 branches is **stale** — that was resolved; `wa-18` was a +plain fast-forward. + +### Test baselines (all green at `wa-19-review-fixes` HEAD) + +``` +packages/studio 3661 passing, 18 todo, 1 file skipped +packages/core 1657 passing +packages/engine 733 passing, 3 skipped (src/services) +``` + +`tsc`, `oxlint`, `oxfmt` clean across studio, core, engine, lint. + +--- + +## 2. What the review was + +`/code-review max` ran against the audio-FX **foundation** stack — NOT `wa-18`: + +``` +#3019 → #3020 → #3021 → #3022 → #3014 → #3015 → #3016 → #3017 +origin/main … wa-5-automation-runtime-fixes, 6760 insertions +``` + +Eight PRs, not the six that were asked for; splitting the closed +`wa-1-fx-foundation` (#3013) into four is where the "6" came from. `#3036 +wa-13-fx-tail` was excluded. + +**Review tip was `68ce91057`**, 13 commits below `wa-18` HEAD — so all the +reviewer's line numbers are stale. Every fix below was re-verified against +current code before being touched, and several had moved. + +15 findings reported, ~10 more confirmed but cut by a cap, plus a cleanup list. +Three candidates were refuted by the reviewer and dropped; one was already fixed +upstack. + +--- + +## 3. Fixed (11 commits on `wa-19-review-fixes`) + +Each was verified against current code, fixed, given a test, and the test +**falsified by deleting the fix**. Three ship without tests and say so in their +commit message. + +| Commit | What | +| --- | --- | +| `43c88961b` | Empty audio track no longer kills the whole render | +| `714f662ed` | Browser lease taken inside the try that releases it (no test — resource ordering) | +| `accac9984` | Phaser `in_gain`/`out_gain` lanes aimed at the trims, not the pinned wet/dry | +| `b754e72d0` | Node ids follow their slot on an in-place chain update | +| `f6f0e4cf2` | Volume lane sampled at clip-local time, matching the render | +| `436db2506` | FX parameters can be typed; a bare blur no longer re-persists | +| `59563eae0` | Lint rule stops blaming the wrong element in a chained timeline | +| `007c94722` | Generated audio-FX runtime regenerated before core's tests | +| `05d081c61` | Clip's FX graph disposed when it ends, not only when stopped | +| `371920695` | FX rack locked while the carve is measuring | +| `bfa27381e` | ffmpeg children given the signal that actually aborts them (no test — wiring) | + +Two judgement calls worth knowing: + +- **A guard was removed, not kept.** The review said `latest` (in + `propertyPanelFxControls`) is never resynced. True, but the edit-flag fix in + `436db2506` subsumes it — three sibling mutations failed, that one passed. Code + no test can falsify was dropped rather than left in. +- **`.gitignore` was not touched** (standing rule). That is why `007c94722` + makes `test` regenerate the artifact instead of copying the sibling's + `build && git diff --exit-code` gate, which needs the file tracked. + +--- + +## 4. Remaining work + +### 4a. The four still open from the top 15 + +**1. FX output clipped to ±1 before the volume lane ducks it.** +`packages/engine/src/services/audioFxRender.ts:~136` clamps and quantises to +int16 in `writeWav`; `audioMixer.ts:~936` then calls `applyVolumeEnvelopeToWav`, +which re-reads that file and multiplies. So a chain that overshoots full scale is +destructively clipped even when the envelope immediately pulls the track 12 dB +down — audible distortion in the render that preview does not have. + +*I started this one and stopped deliberately.* The fix is to apply the envelope +to the float samples inside `applyAudioFxChain` before `writeWav` quantises. +That means: +- hoisting `envelopeKeyframes` (currently computed AFTER the FX call, `audioMixer.ts:~933`) above `applyAudioFxChain` +- extending its options with the envelope + `trackStart` + base volume +- setting `bakedEnvelope = true` so the mixer skips the second pass +- extracting the incremental segment-cursor gain walk out of + `audioVolumeEnvelope.ts:~107-116` so both paths share one implementation +- keeping the ffmpeg-expression fallback intact for non-int16 WAVs + +It changes rendered audio for every track with both a chain and a lane, so it +wants its own session and a before/after listen. + +**2. Base64 PCM round-tripped whole through one `page.evaluate`.** +`audioFxRender.ts:~256`. ~184 MB per 3-minute stereo track across one CDP message +pair. Verified ceilings: puppeteer-core sets `maxPayload: 256 MB` and +`browserManager.ts` does not pass `pipe: true`, so stereo 48 kHz past **~8.7 min** +exceeds the cap; V8's max string length is a second wall at ~17.5 min. Both raise +`AudioFxRenderError`, fatal. Compounded by `processCompositionAudio`'s unbounded +`Promise.all` holding N payloads at once. Also `Array.from(u8.subarray(...))` +boxes each 32 KiB chunk for nothing — `apply` takes array-likes. + +**3. `setRate` does not reschedule FX automation.** +`webAudioTransport.ts:~263`. `attachElementFxChain` gets a frozen +`{scheduledAt, elapsed, rate}` and commits ramps to absolute context time; +`setRate` only assigns `playbackRate.value`. A lowpass sweeping over 10 clip- +seconds, switched to 2× mid-play, eats 20 s of material in 10 s of wall clock +while the sweep keeps its original schedule. The recovery path in +`runtime/init.ts` (`stopAll()` + reschedule) only fires when +`hasBoundedActiveSources()` is true — a project-level music bed with no +`data-duration` is unbounded, so it never recovers. Second consequence: +`timingNow()` advances `elapsed` with the stale rate, so any later chain edit +re-aims the envelope at the wrong clip position. + +**4. `powerSpectrum` allocates on the main thread.** +`audioCarve.ts:~100` — two `Float64Array(4096)` inside the per-hop loop. A +5-minute 48 kHz voiceover is ~461 MB of transient garbage and ~7030 FFTs in one +uninterruptible block, called synchronously from the panel. `re` is fully +overwritten each iteration and `im` only needs `.fill(0)`, so both hoist; Welch +converges after a few hundred frames, so striding the hops gives the same +spectrum for ~2% of the work. **Now copied into two more loops** — +`audioCarve.ts:~320` and `~558` — so each needs its own spectrum check. + +### 4b. Confirmed but cut by the reviewer's 15-cap + +Rough severity order, file:line at the review tip: + +- `audioFxGraph.ts:254`/`:316` — chorus + phaser `lfo.start()` at build time, not clip-relative → preview ≠ render phase +- `audioFxWorklets.ts:247` — a rejected `addModule` is cached per-context forever while `readyContexts` is only written on success → all worklet effects permanently silent, no retry +- `audioFxGraph.ts:159` — worklet `dispose` only disconnects while every `process()` returns `true` → abandoned processors never released +- `audioFx.ts:754` — `Number(null)`/`Number("")` === 0 passes `Number.isFinite`, so a null param clamps to 0 instead of its declared default (compressor threshold 0 dB = never engages). `numberOrNull` at `audioAutomation.ts:141` already guards exactly this +- `audioStage.ts:89` — `audioFailures: undefined` flattens the typed failure the stage comment says it exists to attach +- `audioFxRender.ts:204` — per-element mkdtemp + page + `goto` + runtime re-injection +- `.fallowrc.jsonc:662` — file-scoped health ignore also exempts `syncRuntimeMedia`; `:157` `ignoreExports` names two symbols that do not exist +- `propertyPanelFxSection.tsx:287` — `key={type-index}` (PLAUSIBLE; likely inert now that the `latest` ref issue is fixed) + +### 4c. Cleanup list + +- `audioFxRender.ts:43` duplicates `parseWavLayout` +- `propertyPanelFxControls.tsx:51` re-implements FlatSlider/FlatToggle/FlatSelectRow, losing their throttle, pointer capture, Escape-revert and design telemetry +- `PropertyPanelFlat.tsx:549` reads via hardcoded `"fx-chain"` literals while writing via constants +- `audioMixer.ts:328` re-rolls curve sampling at 30/s vs the scheduler's 100/s while `sampleAutomationCurve` names it as a consumer +- `build-audio-fx-runtime.ts` is a near-verbatim clone of `build-position-edits-render.ts` +- `audioFxGraph.ts:279`/`:352` four hand-rolled wet/dry shells whose dispose lists have already drifted +- `audioFxGraph.ts:448` enabled-node filter restated 3× while `enabledAudioFxNodes()` goes unused +- `audioFxGraph.ts:494` dead `shapeOf` recomputation on every observer tick +- `PropertyPanelFlat.tsx:624` a `doc!` non-null assertion plus an `as` cast laundering `undefined`; `audioAutomation.ts:122` a cast after the guard already narrowed — both against the repo's "avoid `any`/`as T`" rule and the standing no-`!` rule + +--- + +## 5. `carve-reconcile-wip` — working, but parked + +`7b57000a1`, branched off `wa-18` HEAD. Makes a deleted voice re-measure its bed +**wherever the author is looking**, rather than only when the bed's Audio FX +section is opened (which is what ships on `wa-18`). + +Contains: +- `carveAnalysis.ts` — the carve lifted out of the panel as a computation over plain data, so it runs without a DOM. The panel calls it too. +- `fxCarve` mirrored onto `TimelineElement` beside `fxChain`/`automation` +- `useCarveReconcile` — watches the timeline store, writes all three attributes in one `patch-element`. Debounced 800 ms so an undo's flicker is not read as a deletion; single-flighted per carve so its own write cannot loop the decode. 11 tests, each falsified. +- The panel's own prune effect removed (one owner, or the two race through different write paths and 409 the save queue). + +**Verified working:** deleting a voice with the bed's panel never opened pruned +the sources and re-measured the bands (400 Hz −8.2 → −8.33 dB, 1 kHz −6.68 → +−7.14). + +**Why it is parked:** it kills undo. After the watcher's `patch-element` write, +pressing Undo issues **no HTTP request at all** and the file is untouched — the +button is enabled, the click just does nothing. Measured, in contrast to the same +flow before the change, where undo restored the clip. The write goes around the +edit-history mechanism and leaves the stack unusable; the exact mechanism was +never pinned down. + +Likely next move: read the file, patch, then `recordEdit` the before/after so the +history stack stays coherent — at the cost of the reconcile becoming its own undo +step (two presses to get the clip back). Worth deciding which is wanted first. + +Also: the 800 ms settle is a **guess, not a measurement** — undo broke before the +real store-resync gap could be observed. + +--- + +## 6. Traps that cost time this session + +1. **`hyperframes` on PATH is the PUBLISHED npm package**, v0.7.99 at + `~/.bun/install/global/node_modules/hyperframes` — a real directory, NOT a + symlink to this repo. `hyperframes preview` serves the released studio bundle + and none of this branch's work. The memory note saying it is symlinked is + **wrong for this machine**. To see local studio changes: + ```bash + cd packages/studio && bun run dev # → http://localhost:5190 + ln -sfn ~/src/recap-stitch packages/studio/data/projects/recap-stitch + # open http://localhost:5190/#project/recap-stitch + ``` +2. **The generated audio-FX runtime goes stale silently.** Editing + `packages/core/stubs/audio-fx-runtime-entry.ts` has NO effect on engine render + tests until `bun run build:audio-fx-runtime` runs in `packages/core`. This cost + a confusing falsification round. `007c94722` makes core's `test` regenerate it, + but the engine's tests do not. +3. **`rtk` wraps git and grep and sometimes eats output.** `git diff --name-only` + returned nothing; `grep -n` reported "0 matches" on a file that matched. Use + `/usr/bin/git` and `/usr/bin/grep` when output looks wrong. `--include=*.ts` + globs fail in this shell — quote them or use `/usr/bin/grep -r`. +4. **NUL bytes got written into source twice** by heredoc/Write round-trips — a + literal space inside a template literal became `\x00`, which made ripgrep skip + the file as binary and report zero matches. Check with + `python3 -c "print(open(F,'rb').read().count(b'\x00'))"` if a search + inexplicably finds nothing. +5. **happy-dom has no `OfflineAudioContext`**, so `analyse()` returns before the + decode and any test of the "while analysing" window never opens it. Stub both + `fetch` and `OfflineAudioContext` — see the test in `371920695`. +6. **React ignores a plain `input.value = x`** in tests. Use the native setter + (`Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value").set`) + then dispatch `input`. See `propertyPanelFxControls.test.tsx`. +7. **The `fallow` pre-commit gate already fails on this stack** — every commit + here and on `wa-18` used `--no-verify`, as the previous 49 did. +8. **Two vitest runs racing on this machine** produce spurious 5 s timeouts in + unrelated PropertyPanel tests. Do not chase them; re-run alone. +9. **The timeline store does not refresh after a delete** — the deleted clip stays + rendered until a page reload. Independent of this work, but it gates when + store-driven reconciliation fires. + +--- + +## 7. Commands that work here + +```bash +# tests — use the package's own binary, NOT bunx vitest +cd packages/studio && ./node_modules/.bin/vitest run +cd packages/core && ./node_modules/.bin/vitest run +cd packages/engine && ./node_modules/.bin/vitest run src/services/ + +# after ANY change to core types or the FX stub that studio/engine read +bun run --filter @hyperframes/core build +cd packages/core && bun run build:audio-fx-runtime + +bunx tsc --noEmit -p packages/studio # and -p packages/core, -p packages/engine, -p packages/lint +bunx oxlint packages/studio/src packages/core/src packages/engine/src +bunx oxfmt + +# commits need --no-verify (§6.7) +``` + +## 8. Fixture + +`~/src/recap-stitch/index.html` — four narration clips (`narration-1..4`) on +tracks 11–14, `music-bed` carving all four, `sfx-explosion` on track 9. +`hyperframes lint`: 0 errors, 3 pre-existing warnings. + +Backups: `index.html.bak` (pre-split original), `index.html.pre-shared-rows` +(before the shared-row work — note this one has the clips on separate tracks). + +To exercise **shared automation lane rows**, point all four narrations at one +`data-track-index`; they are currently one per row.