Files
hyperframes/plans/webaudio-stack-handoff.md
Vance IngallsandClaude Sonnet 5 d6c4774ef4 feat(studio): instrument the audio FX rack, including work an agent did (#3229)
* fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened

useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a
preview-only commit that useDomEditAttributeCommits.ts never grew — backported
that option support from its own later commit so the two sides of the API
agree. The paste path and its tests were missing the box selection's v0/v1
bounds a sibling commit added to AutomationSelection. The FX panel's carve
controls still edited the six mechanism numbers (maxCutDb, bands,
intelligibilityBias) after carveProfile() collapsed authoring to one Strength
knob, so those fields no longer existed on HfCarveSettings; UI now edits
strength, and analyseCarveBands is called with carveProfile(strength).

Also closes fallow's complexity, dead-code and duplication findings on this
PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math)
and useAutomationRangeDrag.ts (the marquee-select gesture) out of
useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a
resolver into named functions, dropped an export nothing outside its file
used, and shared a step-simplifier between audioCarve's two envelope
builders.

The edge-stretch vs. box-select priority test in TimelineAutomationLane.test
was still pinning the pre-box-select rule (edge wins over a point sitting on
it) that a sibling commit deliberately reversed — a point inside the box is
now selected content, so grabbing it drags the group instead. Updated the
test to the shipped rule instead of the old one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(core): cap the via conic's weight so an edge-clamped via point can't NaN

A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to
(0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0.
viaConic divided by that zero to get an infinite weight, and shapeVia turned
Infinity into NaN a few steps later (Infinity - Infinity in the quadratic
coefficient). NaN reaching setValueCurveAtTime silences the automated
parameter for the rest of the render.

Capped the weight at 1e6 instead of leaving it unbounded — past that point
the arc already reads as touching the via point, so nothing visible is lost.
Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`,
since NaN fails the original comparison and fell through it.

Review by Miga (PR #3208).

* fix(studio-server): fingerprint the proactive waveform cache key too

The route already keys the waveform cache on the asset's size and mtime as
well as its path, so a rebuilt-in-place file gets fresh peaks instead of
stale ones. generateWaveformCache — the proactive path that runs on upload —
still called buildWaveformCacheKey with the path alone, so it wrote to a
different key than the route reads from (making the pre-generated cache
never found) and kept the exact collision bug this fingerprint exists to fix
on its own path.

Review by Miga (PR #3211).

* style(docs): run oxfmt on the /hyperframes-audio skill docs

Table column widths had drifted out of alignment with oxfmt's own rules,
failing format:check and blocking the Preflight gate every downstream
branch inherits. Whitespace only, no content change.

* fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration

\b treats `_` as a word character, so \bbed\b never matched bed_01,
music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap —
an underscore-separated bed classified as "unknown" and could end up offered
as its own carve source. Replaced the short hints with a boundary that
actually excludes letters and digits on both sides.

clipsOverlap computed end = start + duration without guarding sign, so a
negative duration put end before start — an interval that does not describe
anything, and one specific case showed it silently dropping a real overlap
(a shorter, earlier broken end rejected a clip that genuinely contained the
point). Duration clamps to zero instead: a clip cannot un-play time, and a
zero-length clip at its start is the sane reading of "duration nobody wrote
down as positive."

Review by Miga (PR #3212).

* fix(studio): widen PropertyPanel's resetModules render timeout again

The 20s margin (already once widened for the same reason) is timing out in
CI's full-monorepo Test run — the resetModules()+fresh-import render this
test needs is uncached and competes with every other package's test suite
for the same worker pool, and the same test passes in well under 2s
standalone. Went to 45s rather than re-tuning to whatever number happens to
clear the current CI load, since that number moves every time CI gains a
package.

* fix(studio): stop the single-candidate auto-apply carve firing twice

Two auto-apply effects both fire when sourceOptions.length === 1: the
multi-candidate effect only guards length === 0, so a single candidate
passes it too, and the single-candidate effect passes its own guard right
after — both compute the same sources list and both call setCarve, so the
common case (one narrator, one bed) triggered two decodes, two FFT runs, and
two concurrent attribute writes for one decision.

The multi-candidate effect now defers to its sibling for exactly one
candidate, which already has its own detailed handling for that case.

Review by Miga (PR #3213).

* feat(core): carve against every voice over a bed, always (#3212)

* feat(core): carve against every voice over a bed, always dynamically

A bed usually runs under a whole sequence — a narrator, an interview answer, a
second presenter — and carving against one of them left the others fighting it.
`source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's
clock before anything is measured. That is what keeps one analysis sufficient: the
chain is fixed, so there is no per-voice filter to switch between, and bands drawn
from all the speech there is with envelopes that rise wherever any of it happens
answer the actual question — where and when is speech masking this bed.

Summed rather than averaged: two people talking at once mask more than either
alone. Audio before the bed starts is dropped rather than folded in at zero, since
it plays over nothing and shifting it would put a cut where there is no voice.

`dynamic` is gone. A fixed depth thins the bed through every pause, and once both
have been heard there is no reason to want it, so every carve follows the speech.

Two helpers the panel and the headless script now share instead of each carrying a
copy — two definitions of "what does this name suggest" drift, and then the two
disagree about which track is the voice:

- `classifyAudioName` reads a track's kind from its id and filename together.
  `unknown` is deliberately common: treating an unrecognised name as "not a voice"
  would hide the one track somebody needs to pick.
- `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten
  duration counts as unbounded, not zero — refusing a clip whose length the
  composition leaves to the media would drop the commonest case there is.

Files written before this still load: a single `source` reads as a one-voice list,
a stored `dynamic` is ignored, and an absent attribute means the defaults whole.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration

\b treats `_` as a word character, so \bbed\b never matched bed_01,
music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap —
an underscore-separated bed classified as "unknown" and could end up offered
as its own carve source. Replaced the short hints with a boundary that
actually excludes letters and digits on both sides.

clipsOverlap computed end = start + duration without guarding sign, so a
negative duration put end before start — an interval that does not describe
anything, and one specific case showed it silently dropping a real overlap
(a shorter, earlier broken end rejected a clip that genuinely contained the
point). Duration clamps to zero instead: a clip cannot un-play time, and a
zero-length clip at its start is the sane reading of "duration nobody wrote
down as positive."

Review by Miga (PR #3212).

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(studio): port the carve UI off the removed source/dynamic fields

#3212 (accidentally squash-merged into this branch instead of main) changed
HfCarveSettings from a single `source` + `dynamic` toggle to a `sources`
list with dynamic mode removed outright — the multi-voice UI consumer that
goes with that shape lands in the very next PR, so this branch was left with
a type that no longer matched its own code.

Minimal port, not the multi-voice redesign that PR does properly: the
"Listen to" picker and analyse() treat sources[0] as the one voice this UI
still understands, and every dynamic-mode branch (the automated envelope
lanes, the toggle, the checkbox) is gone along with the field — a carve is
now always the static value the analysis computes, matching what the type
change made permanent. Test suite trimmed the same way: the automation-lane
and toggle tests covered behavior that no longer exists.

* refactor(studio): break up the FX rack's largest functions and files

Fallow flagged 9 complexity findings and 2 file-size violations after the
telemetry stack landed. Extracts FxPresetRun, FxAddMenu, FxRackChain,
FxNodeOpenBody, FxNodeParams, and useFxAudition/useFxCarve/useFxLevelling/
useFxChainObserved out of propertyPanelFxSection.tsx and
propertyPanelAudioFxGroup.tsx, splits propertyPanelFxNodeRow.tsx's open-face
rendering into its own component, and dedupes a clone in studioTelemetry.ts.
Pure structural move — no behavior change; full test suite still green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 15:05:51 -07:00

10 KiB
Raw Permalink Blame History

Web Audio FX stack — session handoff

Written 2026-08-12. Worktree ~/src/wt/hyperframes/webaudio-fx, branch wa-20d7-fx-telemetry (the stack tip). Everything below is pushed; local and origin are in sync across all 47 branches.


1. What this is

47 open PRs, one continuous chain from main to wa-20d7-fx-telemetry, grouped as GitHub stack #3237. ~32k added lines, of which 15,374 are production code — the rest is tests (14,277, a 0.93:1 ratio) and docs.

The feature: audio effects for HyperFrames compositions. A registry of effects, one Web Audio graph shared by preview and offline render, a studio rack panel, 19 presets / 5 named jobs / 5 one-knob profiles, voiceover carve, a levelling script, automation lanes, and telemetry.

Scope observation worth acting on: automation lanes are 9,175 lines — 28% of everything, wa-9 through wa-17. That is a general timeline-envelope editor that audio FX is merely the first consumer of. It reviews as a separate feature and arguably should have been one.


2. Current state

Open wa-* PRs 47
Drafts 0
Approved 1 (#3210)
Changes requested 1 (#3209 — fixed, needs re-review)
Awaiting a verdict 45
Branches over the 600-line cap 0
Studio tests at the tip 3,748 pass
Chain continuous, bases verified unchanged

Tip commits (newest first):

631f6d9e3 style: run oxfmt over the markdown this stack added
ec86035b3 docs(skills): correct the claim that a pause spectrum reveals a filter
2a1708c43 docs(skills): fix two things a retest showed the diagnosis guidance got wrong
dc0933279 docs(skills): teach the audio skill to diagnose a file nobody described
e0b36f46b feat(studio): instrument the audio FX rack, including work an agent did
eca192716 fix(core): stop the rack telling a music bed it will thin the voice out
1cc4781e0 feat(studio): title the carve, hide voice presets off voice tracks, fix panel contrast

3. Traps — read before running anything

Rebuild core after any rebase or branch switch. packages/core/src/generated/ is gitignored and derived from core sources. Stale, it fails ~157 studio tests with Cannot read properties of undefined. This looked exactly like a real regression twice this session and was not.

cd packages/core && bun run build

Mid-stack commits do not individually typecheck. Several branches have real tsc errors that only resolve further up. Verified byte-identical to the original history — do not "fix" them. Check the branch tip, not each commit.

bun run format:check covers markdown. Running oxfmt on .ts/.tsx only is how 51 Preflight failures happened. Format the whole repo before committing docs.

gh stack link tries to re-base the bottom PR onto main. It attempted this twice and GitHub's validation blocked it both times. Always snapshot every base before and diff after:

gh pr list --state open --limit 400 --json number,baseRefName > /tmp/before.json
# ... operation ...
# diff number->baseRefName; expect zero changes

while read drops a final line without a trailing newline. Silently skipped #3019 when marking drafts ready. Verify counts after batch loops.

Cascading a fix down-stack: git rebase --update-refs --onto <fixed-branch> <old-sha> <tip> rewrites all 47 refs in one pass. Git 2.50 supports it. Back up first: git for-each-ref --format='%(refname:short) %(objectname)' 'refs/heads/wa-*' > /tmp/backup.txt.


4. Open work, highest value first

4.1 #3209 needs re-review (blocker was fixed)

wa-18c-box-select deleted #3207's edge-stretch feature — 248-line hook, its test, retimeRange, and the lane wiring. All four reviewer claims verified true. Restored and folded into the consolidated hook.

One arbitration call needs your ruling. #3207's rule: a selection's edge outranks a point sitting on it (because every range op leaves a breakpoint on the edge it created, so point-first broke the second stretch). #3209 changed the selection to a box, where that contradicts its own test — which presses at t=0/v=1, simultaneously the t0 edge and a selected point.

I inverted it: selected content wins, the edge stretches everywhere it is not also selected content. Defensible under a box, but it is a product decision between two deliberate designs. Confirm or reverse.

4.2 CI failures not yet investigated

regression, preview-regression, player-perf across many PRs. main is green, so these are ours. Needs per-PR log analysis; may be flaky. Not touched.

4.3 Two files over the 600-line cap at the tip

propertyPanelFxSection.tsx (1017) and propertyPanelAudioFxGroup.tsx (1003). Pre-existing, grown by the telemetry PR. CI's path filter skipped the check on #3229, so nothing catches it. Real violation hiding behind a filter.

4.4 Three pre-existing test failures

FxSection carve "toFixed" on wa-2-fx-preview. Confirmed pre-existing by stashing. Separate from anything done this session.

4.5 Review throughput

45 of 47 have no verdict. 13 were drafts until this session. The stack merges bottom-up, so #3019 → #3020 → … is the order. Landing the bottom few unblocks everything.


5. Telemetry and dashboard

PostHog dashboard 1986431 "Audio FX rack — usage", project 356858. 12 HogQL tiles, every query verified to execute. Tiles stay empty until a build carrying studio:audio_fx_* ships — that is expected, not broken.

Two things to know:

  • agent_runtime is on EVERY studio:* event, not just audio. The CLI detects the driving agent from its own env (12 vendors, cli/src/telemetry/agent_runtime.ts) and publishes it as window.__HF_CLI_AGENT_RUNTIME; studio reads it via telemetry/agentRuntime.ts. Encoded as the string "none", never omitted — this project has already produced one wrong conclusion from comparing a populated sentinel against an absence.
  • Agent-applied effects are only visible via audio_fx_chain_observed. An agent edits the composition HTML or runs carve.mjs, so no panel event fires. That event carries authored_outside (no panel edits behind the change). carve.mjs is deliberately NOT instrumented — its output is already identifiable by the fromCarve tag.

You cannot verify telemetry locally. browserTelemetryAllowed() is false under Vite dev, and the CLI's isDevMode() is true whenever it runs from .ts source. Both guards exist to stop developers polluting production. Do not defeat them; first real data arrives from a released build.

Read the vault page posthog-cli-telemetry-query-traps before writing any HogQL — it carries the is_ci denominator and clock-boundary traps.


6. The /hyperframes-audio skill

Extended this session with references/presets.md (the catalogue) and references/diagnosis.md (how to diagnose a file you cannot hear).

Evaluated, not assumed. Blind runs on damaged audio, agent holding the skill, no labels. 2 of 4 correct — and three rounds of doc improvements did not move that number, which is the finding.

The structural result:

Additive defects are solvable. Filter defects are not, from the file alone.

Measured, gap spectrum vs the clean take: rumble (noise added) shows +44.7 dB in the pause — unmissable. Boomy (+7 dB @ 200 Hz), sibilant (+10 dB @ 7 kHz) and dull (9 dB shelf) show nothing. A filter multiplies; applied to a take whose gaps sit at the quantisation floor it leaves them there.

So the pause answers "was something added?" and cannot answer "was something filtered?". The doc now says never to rule out EQ on a null pause result — one run did exactly that and shipped a high-pass for an inaudible 72 dBFS rumble on a file whose real problem was no top end.

Implication: the fix is a better reference, not better prose. An agent that applies effects knows the before state; one handed someone else's finished audio is in the genuinely under-determined case.

Test bench

packages/studio/data/projects/fx-test-bench — one clean 7 s narration damaged 11 ways, one per shipped fix. Lint and browser gate pass. GROUND-TRUTH.md is the answer key and is marked keep-away-from-anything-being-evaluated. Untracked scratch; will not land in a PR.


7. Corrections made this session — do not re-derive

  • "44 unreviewed" was wrong. A commented review leaves reviewDecision as REVIEW_REQUIRED. 30 of those had been reviewed. Count reviews, not verdicts.
  • "47/47 have descriptions" was wrong. I tested body length; the unfilled PR template passes it. 13 PRs — the whole lower half — carried the template. All 13 now have real bodies written from their own commits, and 6 had placeholder titles ("wa 1 fx registry") which were rewritten.
  • PR #3058 is closed, superseded by the #3207#3215 split. wa-18-lane-stretch and wa-20d-rack-design are dead local branches backing zero PRs. Do not push them — wa-20d-rack-design would re-add ~2,153 lines of stripped handoff docs.
  • Case B's sibilance diagnosis was right for unverifiable reasons. It claimed to measure the room-tone gaps; the gap shows nothing at 7 kHz. Right answer, reasoning I could not reproduce.

8. Conventions

  • bun, not pnpm/npm. oxlint / oxfmt, not eslint/prettier/biome.
  • Commits need --no-verify (pre-commit hooks are slow and sometimes fight mid-rebase state).
  • Do not push or update PRs unless asked.
  • Signed commits are required; filter-branch strips signatures and the push is rejected with GH013.
  • Composition changes: npx hyperframes lint then npx hyperframes check.