mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
feat(engine): render grouped audio through a summed, FX-processed bus (#3289)
* feat(core): route grouped audio through a group bus in preview An audio element carrying `data-audio-group` no longer lands its gain on the master bus directly — it feeds a per-group `GainNode` (built lazily on first use, one per group id) which itself feeds master, so members of the same group sum before the ear, ready for a group-level FX chain and volume/mute in later steps. An id with no matching `<hf-audio-group>` element still gets a plain, unprocessed bus rather than losing the track. The group's own chain and volume lane are wired through the same `attachElementFxChain`/`scheduleVolumeLane` every element already uses, against the group's clock — composition time (design doc §1.3), since a group has no `data-start` and a missing one parses as 0. The bus persists across `stopAll()` (mirroring `_masterGain`'s own lifecycle) so replaying a group does not rebuild its chain; only `destroy()` disposes it. Render is untouched — stays flat until B4; `audio-groups` is still a 0% canary so nothing ships this to a real composition without hand-authoring `data-audio-group`. Also: `audioGroupOf` (B1) crashed on any element lacking a real `tagName`/ `getAttribute` — exactly the shape of most `HTMLMediaElement` test doubles in this suite, including this file's own `mockEl`. Made it tolerant, same style as `readChain`'s existing guard in `runtime/audioFx.ts`. `schedulePlayback` was already 110 lines pre-existing before this diff; extracted `resolveDestination` and `handleSourceEnded` to shrink it to 92, then suppressed the remainder (inherently sequential graph wiring, not a decision tree) per the same precedent B2 used on `TimelineLogicalRow`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(studio,lint): carve targets voiceover groups — always, when plural Plural voiceover carve now targets a group instead of naming each clip: `resolveCarveSourceIds` (core `audioGroups.ts`) expands a group id to its current members at analysis time, so a clip added to the group later is covered without touching `sources`. The picker (`useFxCarve.ts`) offers a grouped voice as one option instead of one row per member, tests overlap as a union of member spans (a group overlaps the bed if ANY member does), and prefers a qualifying group over its individual members in `autoSourceIds`. Picking two or more ungrouped voice clips in the carve flow now mints a group behind them (`mintGroupId`, de-duped against every id in the document) and writes `data-audio-group` on each picked clip atomically, one undo entry — `createAudioGroupAndAssignMembers` in `timelineTrackVisibility.ts` copies `setElementsHidden`'s multi-target write shape. The DSP is untouched: `mixCarveSources` already sums multiple sources correctly (verified in the design doc's own investigation) — this only fixes the picker. New lint rule `audio_carve_ungrouped_sources` (`packages/lint/src/rules/ media.ts`, alongside `audio_volume_double_automation`) warns when a `data-fx-carve`'s `sources` names two or more plain clip ids instead of a group — the shape that silently rots when a clip is added. `/hyperframes- audio` states the same rule as an invariant, not a tip, with the grouped- narration HTML example from the design doc. The group-matching and auto-group logic (`withAutoGroupedSources`, `collectCarveCandidates`) is split into `useFxCarveGrouping.ts` — `useFxCarve.ts` was pushing past the 600-line cap. `resolveNextCarveSettings` is deliberately NOT an `async function`: wrapping it in one would force a microtask on every call, including the synchronous branch — the exact bug `withAutoGroupedSources`'s own sync-when-possible contract exists to avoid, and one caught via `propertyPanelAudioFxGroup.test.tsx` (10 failures) before fixing it back to a plain function the caller conditionally awaits. Also extracted `useEffectiveTimelineDuration` out of `App.tsx` and `useRemoveBackground` out of `StudioRightPanel.tsx` (both pushed past 600 lines from an added prop wire), and decomposed `useFxCarve.ts`'s picker IIFE to clear fallow's complexity gate. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(engine): render grouped audio through a summed, FX-processed bus Renders what B3 already routes in preview: a group's members sub-mix into one PCM WAV at full composition length (adelay already places each member at its composition position, so the group WAV's t=0 IS composition time), run through the group's own FX chain and automation via the same applyAudioFxChain/envelope-bake path a member uses, then fold into the flat track list as one processed AudioTrack — the final mixAudioTracks call never has to know groups exist. Gain law verified against plans/spikes/amix-nesting-spike.sh (brought over from the plans branch, along with audioMixer.grouping.test.ts, since both were committed there and never merged to origin/main — every step branch in this stack descends from origin/main): the sub-mix's own amix prefers normalize=0 (nulls exactly against a flat mix), falling back to per-node compensation by the group's OWN member count only when this ffmpeg build's amix rejects the option. Carrying any other count into a nested amix node is the exact +2.499 dB silent failure the spike measured — confirmed by a manual mutation check (wrong-count compensation landed 3.5 dB hot, exactly 20*log10(3/2) for a 2-member group compensated as 3; reverted after confirming the level test catches it). A group element carrying data-hidden drops every member before the sub-mix ever runs (RULES: mute-by-drop, never mute-by-volume-0) — parseAudioElements now resolves groups once per parse and skips hidden-group members the same way it already skips data-hidden ancestors. HfAudioGroup (packages/core/src/audioGroups.ts, from B1) gains fxChain, automation, volume and hidden, read off the group element the same way resolveAudioGroups already reads data-label — audioGroups.test.ts updated for the wider shape plus new coverage for the added reads. it.todo("mixes a grouped composition at the same level as the ungrouped one") is now a real, passing test; two more added per the step doc (FX routing isolation, member-level envelope survives grouping). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
485c037dcf
commit
99f42be04c
@@ -23,7 +23,15 @@ describe("resolveAudioGroups", () => {
|
||||
<audio id="sfx-1"></audio>
|
||||
`;
|
||||
const groups = resolveAudioGroups(document);
|
||||
expect(groups).toEqual([{ id: "voiceover", label: "Voiceover", memberIds: ["vo-1", "vo-2"] }]);
|
||||
expect(groups).toEqual([
|
||||
{
|
||||
id: "voiceover",
|
||||
label: "Voiceover",
|
||||
memberIds: ["vo-1", "vo-2"],
|
||||
volume: 1,
|
||||
hidden: false,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("resolves from member tags alone when the group element is absent, label = id", () => {
|
||||
@@ -31,7 +39,9 @@ describe("resolveAudioGroups", () => {
|
||||
<audio id="vo-1" data-audio-group="narration"></audio>
|
||||
`;
|
||||
const groups = resolveAudioGroups(document);
|
||||
expect(groups).toEqual([{ id: "narration", label: "narration", memberIds: ["vo-1"] }]);
|
||||
expect(groups).toEqual([
|
||||
{ id: "narration", label: "narration", memberIds: ["vo-1"], volume: 1, hidden: false },
|
||||
]);
|
||||
});
|
||||
|
||||
it("ignores data-audio-group on the group element itself (groups do not nest)", () => {
|
||||
@@ -40,7 +50,9 @@ describe("resolveAudioGroups", () => {
|
||||
<audio id="vo-1" data-audio-group="outer"></audio>
|
||||
`;
|
||||
const groups = resolveAudioGroups(document);
|
||||
expect(groups).toEqual([{ id: "outer", label: "outer", memberIds: ["vo-1"] }]);
|
||||
expect(groups).toEqual([
|
||||
{ id: "outer", label: "outer", memberIds: ["vo-1"], volume: 1, hidden: false },
|
||||
]);
|
||||
expect(audioGroupOf(document.getElementById("outer") as Element)).toBeNull();
|
||||
});
|
||||
|
||||
@@ -59,6 +71,37 @@ describe("resolveAudioGroups", () => {
|
||||
document.body.innerHTML = `<video id="v-1" data-audio-group="voiceover"></video>`;
|
||||
expect(resolveAudioGroups(document)).toEqual([]);
|
||||
});
|
||||
|
||||
it("reads the group element's fx chain, automation, volume and hidden", () => {
|
||||
document.body.innerHTML = `
|
||||
<hf-audio-group id="voiceover" data-fx-chain='{"version":1,"nodes":[]}' data-automation='{"lanes":[]}' data-volume="0.5" data-hidden></hf-audio-group>
|
||||
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
||||
`;
|
||||
const groups = resolveAudioGroups(document);
|
||||
expect(groups).toEqual([
|
||||
{
|
||||
id: "voiceover",
|
||||
label: "voiceover",
|
||||
memberIds: ["vo-1"],
|
||||
fxChain: '{"version":1,"nodes":[]}',
|
||||
automation: '{"lanes":[]}',
|
||||
volume: 0.5,
|
||||
hidden: true,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("defaults volume to 1 and hidden to false when a group element exists but carries neither", () => {
|
||||
document.body.innerHTML = `
|
||||
<hf-audio-group id="voiceover"></hf-audio-group>
|
||||
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
||||
`;
|
||||
const [group] = resolveAudioGroups(document);
|
||||
expect(group?.volume).toBe(1);
|
||||
expect(group?.hidden).toBe(false);
|
||||
expect(group?.fxChain).toBeUndefined();
|
||||
expect(group?.automation).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("audioGroupOf", () => {
|
||||
|
||||
@@ -9,6 +9,9 @@
|
||||
* nothing here routes or sums audio yet.
|
||||
*/
|
||||
|
||||
import { HF_AUDIO_FX_ATTR } from "./audioFx.js";
|
||||
import { HF_AUDIO_AUTOMATION_ATTR } from "./audioAutomation.js";
|
||||
|
||||
export const HF_AUDIO_GROUP_TAG = "hf-audio-group";
|
||||
export const HF_AUDIO_GROUP_ATTR = "data-audio-group";
|
||||
|
||||
@@ -32,6 +35,38 @@ export interface HfAudioGroup {
|
||||
label: string;
|
||||
/** Member element ids, in document order. */
|
||||
memberIds: string[];
|
||||
/** Serialised FX chain JSON from the group element's `data-fx-chain`, when set. */
|
||||
fxChain?: string;
|
||||
/** Serialised automation JSON from the group element's `data-automation`, when set. */
|
||||
automation?: string;
|
||||
/** The group element's `data-volume`, defaulting to 1 when absent or there is no group element. */
|
||||
volume: number;
|
||||
/**
|
||||
* The group element's `data-hidden`. Render drops every member rather than
|
||||
* zeroing them (RULES: mute-by-drop, never mute-by-volume-0) — B5 defines
|
||||
* the UI for this; this field just makes the read available now.
|
||||
*/
|
||||
hidden: boolean;
|
||||
}
|
||||
|
||||
function parseGroupVolume(el: Element | undefined): number {
|
||||
const raw = el?.getAttribute("data-volume");
|
||||
const parsed = raw ? parseFloat(raw) : 1;
|
||||
return Number.isFinite(parsed) ? parsed : 1;
|
||||
}
|
||||
|
||||
function buildGroup(id: string, memberIds: string[], el: Element | undefined): HfAudioGroup {
|
||||
const fxChain = el?.getAttribute(HF_AUDIO_FX_ATTR);
|
||||
const automation = el?.getAttribute(HF_AUDIO_AUTOMATION_ATTR);
|
||||
return {
|
||||
id,
|
||||
label: el?.getAttribute("data-label") || id,
|
||||
memberIds,
|
||||
...(fxChain ? { fxChain } : {}),
|
||||
...(automation ? { automation } : {}),
|
||||
volume: parseGroupVolume(el),
|
||||
hidden: el?.hasAttribute("data-hidden") ?? false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -58,9 +93,7 @@ export function resolveAudioGroups(root: ParentNode): HfAudioGroup[] {
|
||||
|
||||
const groups: HfAudioGroup[] = [];
|
||||
for (const [id, memberIds] of membersByGroup) {
|
||||
const el = groupElements.get(id);
|
||||
const label = el?.getAttribute("data-label") || id;
|
||||
groups.push({ id, label, memberIds });
|
||||
groups.push(buildGroup(id, memberIds, groupElements.get(id)));
|
||||
}
|
||||
return groups;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { getFfmpegBinary } from "../utils/ffmpegBinaries.js";
|
||||
import { MIXED_AUDIO_FILENAME, processCompositionAudio } from "./audioMixer.js";
|
||||
|
||||
/**
|
||||
* Level arithmetic across the mix graph.
|
||||
*
|
||||
* `mixAudioTracks` corrects for amix's own normalisation with a single global
|
||||
* `masterOutputGain * tracks.length`. That correction is exact for one flat
|
||||
* amix and wrong for any other shape — and it fails SILENTLY, in the export
|
||||
* rather than in preview, because preview mixes through Web Audio and never
|
||||
* runs this graph at all.
|
||||
*
|
||||
* Measured on a four-track mix (spike, 2026-08-14): keeping the global track
|
||||
* count on an outer amix that only has three inputs lands the whole mix
|
||||
* +2.499 dB hot — exactly 20·log10(4/3). Nothing errors; the file is just
|
||||
* loud. These tests are the instrument that catches that class.
|
||||
*/
|
||||
|
||||
const HAS_FFMPEG = spawnSync(getFfmpegBinary(), ["-version"], { encoding: "utf-8" }).status === 0;
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
/** Mean level of a whole file, or of one window when `from`/`to` are given. */
|
||||
function meanVolumeDb(path: string, from?: number, to?: number): number {
|
||||
const filter =
|
||||
from === undefined ? "volumedetect" : `atrim=${from}:${to},asetpts=N/SR/TB,volumedetect`;
|
||||
const result = spawnSync(
|
||||
getFfmpegBinary(),
|
||||
["-nostdin", "-hide_banner", "-i", path, "-af", filter, "-f", "null", "-"],
|
||||
{ encoding: "utf-8" },
|
||||
);
|
||||
const match = result.stderr.match(/mean_volume:\s*(-?[\d.]+) dB/);
|
||||
if (result.status !== 0 || !match?.[1]) {
|
||||
throw new Error(`Could not measure mean volume: ${result.stderr}`);
|
||||
}
|
||||
return Number(match[1]);
|
||||
}
|
||||
|
||||
/** A sine at `freq`, scaled by `gain`, written as PCM so the source is exact. */
|
||||
function writeTone(path: string, freq: number, seconds: number, gain: number): void {
|
||||
const result = spawnSync(
|
||||
getFfmpegBinary(),
|
||||
[
|
||||
"-nostdin",
|
||||
"-v",
|
||||
"error",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
`sine=frequency=${freq}:duration=${seconds}:sample_rate=48000`,
|
||||
"-af",
|
||||
`volume=${gain}`,
|
||||
"-c:a",
|
||||
"pcm_s16le",
|
||||
path,
|
||||
],
|
||||
{ encoding: "utf-8" },
|
||||
);
|
||||
if (result.status !== 0) throw new Error(`Could not write tone: ${result.stderr}`);
|
||||
}
|
||||
|
||||
const track = (id: string, end: number, volume = 1) => ({
|
||||
id,
|
||||
src: `${id}.wav`,
|
||||
start: 0,
|
||||
end,
|
||||
mediaStart: 0,
|
||||
layer: 0,
|
||||
volume,
|
||||
type: "audio" as const,
|
||||
});
|
||||
|
||||
describe.skipIf(!HAS_FFMPEG)("mix level arithmetic", () => {
|
||||
afterEach(() => {
|
||||
for (const dir of tempDirs.splice(0)) rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("keeps every track at its authored level regardless of how many there are", async () => {
|
||||
// The property the compensation exists to hold: adding tracks must not
|
||||
// duck the ones already there. Mixing the SAME tone twice is the cleanest
|
||||
// probe — two coherent copies sum to exactly +6.02 dB, so any residual
|
||||
// normalisation shows up as a plain arithmetic miss rather than something
|
||||
// that has to be teased out of unrelated material.
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-grp-count-"));
|
||||
const workDir = mkdtempSync(join(tmpdir(), "hf-grp-count-work-"));
|
||||
tempDirs.push(projectDir, workDir);
|
||||
|
||||
writeTone(join(projectDir, "a.wav"), 440, 2, 0.4);
|
||||
writeTone(join(projectDir, "b.wav"), 440, 2, 0.4);
|
||||
const oneUp = join(projectDir, `one-${MIXED_AUDIO_FILENAME}`);
|
||||
const twoUp = join(projectDir, `two-${MIXED_AUDIO_FILENAME}`);
|
||||
|
||||
const one = await processCompositionAudio([track("a", 2)], projectDir, workDir, oneUp, 2);
|
||||
const two = await processCompositionAudio(
|
||||
[track("a", 2), track("b", 2)],
|
||||
projectDir,
|
||||
workDir,
|
||||
twoUp,
|
||||
2,
|
||||
);
|
||||
expect(one.success).toBe(true);
|
||||
expect(two.success).toBe(true);
|
||||
|
||||
// Two coherent copies of one tone = +6.02 dB. If amix's 1/N ever survives
|
||||
// the correction, this lands at 0 dB instead.
|
||||
expect(meanVolumeDb(twoUp) - meanVolumeDb(oneUp)).toBeCloseTo(6.02, 0);
|
||||
});
|
||||
|
||||
it("does not lift the survivors when a shorter track ends", async () => {
|
||||
// amix with normalize=true rescales by the number of CURRENTLY ACTIVE
|
||||
// inputs, so a track ending mid-composition would hand the remaining ones
|
||||
// a level jump. `apad` to the full duration is what holds every input
|
||||
// active for the whole graph and neutralises that — an invariant the
|
||||
// filter string relies on without saying so.
|
||||
//
|
||||
// Measured without apad (spike, 2026-08-14): the tail runs +1.94 dB hot.
|
||||
// Any group work that builds its own amix has to keep the padding, or
|
||||
// inherit that bug one level down.
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-grp-drop-"));
|
||||
const workDir = mkdtempSync(join(tmpdir(), "hf-grp-drop-work-"));
|
||||
tempDirs.push(projectDir, workDir);
|
||||
|
||||
writeTone(join(projectDir, "short.wav"), 440, 1, 0.5);
|
||||
writeTone(join(projectDir, "long.wav"), 880, 3, 0.5);
|
||||
const together = join(projectDir, `both-${MIXED_AUDIO_FILENAME}`);
|
||||
const alone = join(projectDir, `alone-${MIXED_AUDIO_FILENAME}`);
|
||||
|
||||
const both = await processCompositionAudio(
|
||||
[track("short", 1), track("long", 3)],
|
||||
projectDir,
|
||||
workDir,
|
||||
together,
|
||||
3,
|
||||
);
|
||||
const solo = await processCompositionAudio([track("long", 3)], projectDir, workDir, alone, 3);
|
||||
expect(both.success).toBe(true);
|
||||
expect(solo.success).toBe(true);
|
||||
|
||||
// After 1.5 s only `long` is sounding. It must read the same whether or not
|
||||
// a second track happened to end earlier.
|
||||
const tailTogether = meanVolumeDb(together, 1.5, 3);
|
||||
const tailAlone = meanVolumeDb(alone, 1.5, 3);
|
||||
expect(Math.abs(tailTogether - tailAlone)).toBeLessThan(0.5);
|
||||
});
|
||||
|
||||
/**
|
||||
* The gate for group buses (plans/audio-mixer-groups.md §1).
|
||||
*
|
||||
* Grouping is routing, not processing: a group whose FX chain is empty must
|
||||
* be a no-op on the mix. Enable this the moment `data-audio-group` routes
|
||||
* through a nested amix — it is the definition of done for §1.3, and the
|
||||
* only thing standing between a wrong gain correction and a silently loud
|
||||
* export.
|
||||
*
|
||||
* Proven reachable in the spike: nesting with each amix compensated by ITS
|
||||
* OWN input count nulls against the flat mix to -inf (sample-identical), as
|
||||
* does `amix=normalize=0` with no correction at all.
|
||||
*/
|
||||
it("mixes a grouped composition at the same level as the ungrouped one", async () => {
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-grp-level-"));
|
||||
const workDir = mkdtempSync(join(tmpdir(), "hf-grp-level-work-"));
|
||||
tempDirs.push(projectDir, workDir);
|
||||
|
||||
writeTone(join(projectDir, "a.wav"), 440, 2, 0.4);
|
||||
writeTone(join(projectDir, "b.wav"), 660, 2, 0.4);
|
||||
const flatOut = join(projectDir, `flat-${MIXED_AUDIO_FILENAME}`);
|
||||
const groupedOut = join(projectDir, `grouped-${MIXED_AUDIO_FILENAME}`);
|
||||
|
||||
const flat = await processCompositionAudio(
|
||||
[track("a", 2), track("b", 2)],
|
||||
projectDir,
|
||||
workDir,
|
||||
flatOut,
|
||||
2,
|
||||
);
|
||||
const grouped = await processCompositionAudio(
|
||||
[
|
||||
{ ...track("a", 2), groupId: "voiceover" },
|
||||
{ ...track("b", 2), groupId: "voiceover" },
|
||||
],
|
||||
projectDir,
|
||||
workDir,
|
||||
groupedOut,
|
||||
2,
|
||||
);
|
||||
expect(flat.success).toBe(true);
|
||||
expect(grouped.success).toBe(true);
|
||||
|
||||
// An empty group chain is pure routing — the export must read the same
|
||||
// whether or not the two tones happened to share a group.
|
||||
expect(Math.abs(meanVolumeDb(groupedOut) - meanVolumeDb(flatOut))).toBeLessThan(0.3);
|
||||
});
|
||||
|
||||
it("a group FX chain fully cutting its members leaves an ungrouped track untouched (routing isolation)", async () => {
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-grp-fx-"));
|
||||
const workDir = mkdtempSync(join(tmpdir(), "hf-grp-fx-work-"));
|
||||
tempDirs.push(projectDir, workDir);
|
||||
|
||||
writeTone(join(projectDir, "voice.wav"), 440, 2, 0.4);
|
||||
writeTone(join(projectDir, "sfx.wav"), 880, 2, 0.4);
|
||||
const mixedOut = join(projectDir, `mixed-${MIXED_AUDIO_FILENAME}`);
|
||||
const sfxAloneOut = join(projectDir, `sfx-alone-${MIXED_AUDIO_FILENAME}`);
|
||||
|
||||
const groupChain = JSON.stringify({
|
||||
version: 1,
|
||||
nodes: [{ type: "gain", id: "g", params: { gain: -60 } }],
|
||||
});
|
||||
|
||||
const mixed = await processCompositionAudio(
|
||||
[{ ...track("voice", 2), groupId: "vo", groupFxChain: groupChain }, track("sfx", 2)],
|
||||
projectDir,
|
||||
workDir,
|
||||
mixedOut,
|
||||
2,
|
||||
);
|
||||
const sfxAlone = await processCompositionAudio(
|
||||
[track("sfx", 2)],
|
||||
projectDir,
|
||||
workDir,
|
||||
sfxAloneOut,
|
||||
2,
|
||||
);
|
||||
expect(mixed.success).toBe(true);
|
||||
expect(sfxAlone.success).toBe(true);
|
||||
|
||||
// The voice group is cut ~60 dB — the mix should read close to sfx alone,
|
||||
// and the ungrouped sfx track's own processing is unaffected by the
|
||||
// group existing at all.
|
||||
expect(Math.abs(meanVolumeDb(mixedOut) - meanVolumeDb(sfxAloneOut))).toBeLessThan(0.5);
|
||||
});
|
||||
|
||||
it("a member's own volume envelope still applies inside a group", async () => {
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-grp-env-"));
|
||||
const workDir = mkdtempSync(join(tmpdir(), "hf-grp-env-work-"));
|
||||
tempDirs.push(projectDir, workDir);
|
||||
|
||||
writeTone(join(projectDir, "a.wav"), 440, 4, 0.5);
|
||||
const groupedOut = join(projectDir, `grouped-${MIXED_AUDIO_FILENAME}`);
|
||||
const flatOut = join(projectDir, `flat-${MIXED_AUDIO_FILENAME}`);
|
||||
|
||||
const withEnvelope = {
|
||||
...track("a", 4),
|
||||
volumeKeyframes: [
|
||||
{ time: 0, volume: 1 },
|
||||
{ time: 4, volume: 0 },
|
||||
],
|
||||
};
|
||||
|
||||
const grouped = await processCompositionAudio(
|
||||
[{ ...withEnvelope, groupId: "vo" }],
|
||||
projectDir,
|
||||
workDir,
|
||||
groupedOut,
|
||||
4,
|
||||
);
|
||||
const flat = await processCompositionAudio([withEnvelope], projectDir, workDir, flatOut, 4);
|
||||
expect(grouped.success).toBe(true);
|
||||
expect(flat.success).toBe(true);
|
||||
|
||||
// The envelope fades to silent — the tail should read the same whether
|
||||
// the track is grouped or not, proving member-level processing survives
|
||||
// the group path unchanged.
|
||||
const groupedTail = meanVolumeDb(groupedOut, 3, 4);
|
||||
const flatTail = meanVolumeDb(flatOut, 3, 4);
|
||||
expect(Math.abs(groupedTail - flatTail)).toBeLessThan(0.5);
|
||||
});
|
||||
});
|
||||
@@ -47,6 +47,7 @@ import {
|
||||
parseStrictFiniteTimingNumber,
|
||||
readMediaStart,
|
||||
} from "@hyperframes/core";
|
||||
import { HF_AUDIO_GROUP_ATTR, resolveAudioGroups } from "@hyperframes/core/audio-groups";
|
||||
import { applyAudioFxChain, AudioFxRenderError } from "./audioFxRender.js";
|
||||
import type { AudioVolumeKeyframe } from "./audioMixer.types.js";
|
||||
|
||||
@@ -461,6 +462,17 @@ export function parseAudioElements(html: string): AudioElement[] {
|
||||
return false;
|
||||
};
|
||||
|
||||
// Resolved once per parse. A group element carrying `data-hidden` drops
|
||||
// every member from the render (RULES: mute-by-drop, never
|
||||
// mute-by-volume-0) — members never enter the sub-mix.
|
||||
const groupsById = new Map(
|
||||
resolveAudioGroups(document).map((group) => [group.id, group] as const),
|
||||
);
|
||||
const memberGroupHidden = (el: AudioMediaElement): boolean => {
|
||||
const groupId = el.getAttribute(HF_AUDIO_GROUP_ATTR);
|
||||
return groupId ? (groupsById.get(groupId)?.hidden ?? false) : false;
|
||||
};
|
||||
|
||||
// <audio> and <video data-has-audio> tracks differ only in the emitted id
|
||||
|
||||
// and `type`; everything else (timing, layer, volume) is read identically.
|
||||
@@ -470,6 +482,10 @@ export function parseAudioElements(html: string): AudioElement[] {
|
||||
const volumeAttr = el.getAttribute("data-volume");
|
||||
const fxChain = el.getAttribute(HF_AUDIO_FX_ATTR);
|
||||
const automation = el.getAttribute(HF_AUDIO_AUTOMATION_ATTR);
|
||||
// Audio only in v1 (matches resolveAudioGroups, which only scans
|
||||
// `audio[data-audio-group]`) — a stray attribute on a <video> is inert.
|
||||
const groupId = type === "audio" ? el.getAttribute(HF_AUDIO_GROUP_ATTR) : null;
|
||||
const group = groupId ? groupsById.get(groupId) : undefined;
|
||||
return {
|
||||
id,
|
||||
src: el.getAttribute("src") as string,
|
||||
@@ -483,6 +499,14 @@ export function parseAudioElements(html: string): AudioElement[] {
|
||||
volume: volumeAttr ? parseFloat(volumeAttr) : 1.0,
|
||||
...(fxChain ? { fxChain } : {}),
|
||||
...(automation ? { automation } : {}),
|
||||
...(group
|
||||
? {
|
||||
groupId: group.id,
|
||||
...(group.fxChain ? { groupFxChain: group.fxChain } : {}),
|
||||
...(group.automation ? { groupAutomation: group.automation } : {}),
|
||||
groupVolume: group.volume,
|
||||
}
|
||||
: {}),
|
||||
type,
|
||||
};
|
||||
};
|
||||
@@ -495,7 +519,7 @@ export function parseAudioElements(html: string): AudioElement[] {
|
||||
|
||||
for (const el of document.querySelectorAll("audio[id][src]")) {
|
||||
const id = trackId(el);
|
||||
if (!id || !el.getAttribute("src") || isHidden(el)) continue;
|
||||
if (!id || !el.getAttribute("src") || isHidden(el) || memberGroupHidden(el)) continue;
|
||||
if (isKnownInactiveTimelineWindow(el, resolveStart(el))) continue;
|
||||
elements.push(build(el, id, "audio"));
|
||||
}
|
||||
@@ -841,6 +865,103 @@ async function mixAudioTracks(
|
||||
};
|
||||
}
|
||||
|
||||
function groupNormalizeOptionUnsupported(stderr: string): boolean {
|
||||
return (
|
||||
/normalize/i.test(stderr) &&
|
||||
/(?:unrecognized option|option (?:was )?not found|no option name)/i.test(stderr)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Sub-mix one group's members into a single PCM WAV at full composition
|
||||
* length — the same per-input treatment `mixAudioTracks` gives every track
|
||||
* (atrim, volume, adelay, `apad` — RULES 6), writing PCM instead of AAC so the
|
||||
* result can feed straight into `applyAudioFxChain` (which reads raw WAV
|
||||
* samples, not a decoded container).
|
||||
*
|
||||
* The gain law is the whole reason this is a separate function and not a call
|
||||
* into `mixAudioTracks`: `plans/spikes/amix-nesting-spike.sh` measured that
|
||||
* compensating a nested amix by anything other than ITS OWN input count is a
|
||||
* silent +2.499 dB (20·log10(4/3) for a 3-of-4-track case) — audible, and
|
||||
* invisible to every existing test because it only shows up in export, never
|
||||
* preview. `normalize=0` nulls exactly against a flat mix with no
|
||||
* compensation at all and is preferred; the manual per-own-count fallback
|
||||
* only runs when this ffmpeg build's `amix` rejects the option.
|
||||
*/
|
||||
async function mixGroupMembers(
|
||||
memberTracks: AudioTrack[],
|
||||
outputPath: string,
|
||||
totalDuration: number,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout">>,
|
||||
): Promise<{ success: boolean; error?: string }> {
|
||||
const ffmpegProcessTimeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;
|
||||
const outputDir = dirname(outputPath);
|
||||
if (!existsSync(outputDir)) mkdirSync(outputDir, { recursive: true });
|
||||
|
||||
const inputFilters = memberTracks.map((track, i) => {
|
||||
const delayMs = Math.round(track.start * 1000);
|
||||
const trimDuration = track.end - track.start + (track.tailSeconds ?? 0);
|
||||
const volumeFilter = buildVolumeExpression(track);
|
||||
return `[${i}:a]atrim=0:${formatFilterNumber(trimDuration)},${volumeFilter},adelay=${delayMs}|${delayMs},apad,atrim=0:${formatFilterNumber(totalDuration)}[a${i}]`;
|
||||
});
|
||||
const mixInputs = memberTracks.map((_, i) => `[a${i}]`).join("");
|
||||
|
||||
const runOnce = async (useNormalize: boolean): Promise<RunFfmpegResult> => {
|
||||
const mixFilter = useNormalize
|
||||
? `${mixInputs}amix=inputs=${memberTracks.length}:duration=longest:dropout_transition=0:normalize=0[out]`
|
||||
: // amix's default normalize divides by input count; compensate by THIS
|
||||
// group's own member count — never the render's global track count.
|
||||
`${mixInputs}amix=inputs=${memberTracks.length}:duration=longest:dropout_transition=0[mixed];[mixed]volume=${formatFilterNumber(memberTracks.length)}[out]`;
|
||||
const filterComplex = [...inputFilters, mixFilter].join(";");
|
||||
const scriptDir = mkdtempSync(join(outputDir, ".group-filter-complex-"));
|
||||
const scriptPath = join(scriptDir, "graph.txt");
|
||||
const fd = openSync(scriptPath, "wx", 0o600);
|
||||
try {
|
||||
writeFileSync(fd, filterComplex);
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
try {
|
||||
const inputs: string[] = [];
|
||||
memberTracks.forEach((track) => inputs.push("-i", track.srcPath));
|
||||
const args = [
|
||||
...inputs,
|
||||
"-filter_complex_script",
|
||||
scriptPath,
|
||||
"-map",
|
||||
"[out]",
|
||||
"-acodec",
|
||||
"pcm_s16le",
|
||||
"-ar",
|
||||
"48000",
|
||||
"-t",
|
||||
String(totalDuration),
|
||||
"-y",
|
||||
outputPath,
|
||||
];
|
||||
const legacyResult = await runFfmpeg(args, { signal, timeout: ffmpegProcessTimeout });
|
||||
if (legacyResult.success || !legacyFilterScriptOptionIsUnsupported(legacyResult.stderr)) {
|
||||
return legacyResult;
|
||||
}
|
||||
const currentArgs = [...args];
|
||||
currentArgs[currentArgs.indexOf("-filter_complex_script")] = "-/filter_complex";
|
||||
return await runFfmpeg(currentArgs, { signal, timeout: ffmpegProcessTimeout });
|
||||
} finally {
|
||||
rmSync(scriptDir, { recursive: true, force: true });
|
||||
}
|
||||
};
|
||||
|
||||
let result = await runOnce(true);
|
||||
if (!result.success && groupNormalizeOptionUnsupported(result.stderr)) {
|
||||
result = await runOnce(false);
|
||||
}
|
||||
if (signal?.aborted) return { success: false, error: "Group sub-mix cancelled" };
|
||||
if (!result.success)
|
||||
return { success: false, error: formatFfmpegError(result.exitCode, result.stderr) };
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
export async function processCompositionAudio(
|
||||
elements: AudioElement[],
|
||||
baseDir: string,
|
||||
@@ -854,6 +975,14 @@ export async function processCompositionAudio(
|
||||
const startMs = Date.now();
|
||||
const tracks: AudioTrack[] = [];
|
||||
const failures: AudioProcessingFailure[] = [];
|
||||
// Grouped members are diverted here instead of `tracks` during element
|
||||
// processing, then summed into one processed track per group before the
|
||||
// final mix — see the group sub-mix pass below. `groupMeta` takes the
|
||||
// first-seen chain/automation/volume for a group id: every member of the
|
||||
// same group carries an identical copy (both resolved from the same
|
||||
// `resolveAudioGroups` call in `parseAudioElements`), so any one works.
|
||||
const groupTracks = new Map<string, AudioTrack[]>();
|
||||
const groupMeta = new Map<string, { fxChain?: string; automation?: string; volume: number }>();
|
||||
|
||||
if (!existsSync(workDir)) mkdirSync(workDir, { recursive: true });
|
||||
|
||||
@@ -1093,7 +1222,7 @@ export async function processCompositionAudio(
|
||||
envelope.baseVolume,
|
||||
);
|
||||
}
|
||||
tracks.push({
|
||||
const track: AudioTrack = {
|
||||
id: element.id,
|
||||
srcPath: audioSrcPath,
|
||||
start: element.start,
|
||||
@@ -1104,7 +1233,26 @@ export async function processCompositionAudio(
|
||||
volume: bakedEnvelope ? 1.0 : (element.volume ?? 1.0),
|
||||
volumeKeyframes: bakedEnvelope ? undefined : (envelopeKeyframes ?? undefined),
|
||||
...(tailSeconds > 0 ? { tailSeconds } : {}),
|
||||
});
|
||||
};
|
||||
|
||||
// A grouped member keeps every bit of its OWN processing above
|
||||
// (FX, envelope) exactly like an ungrouped track — it just lands in
|
||||
// the group's bucket instead of the flat list, to be summed into one
|
||||
// processed bus below rather than mixed directly.
|
||||
if (element.groupId) {
|
||||
const bucket = groupTracks.get(element.groupId);
|
||||
if (bucket) bucket.push(track);
|
||||
else groupTracks.set(element.groupId, [track]);
|
||||
if (!groupMeta.has(element.groupId)) {
|
||||
groupMeta.set(element.groupId, {
|
||||
...(element.groupFxChain ? { fxChain: element.groupFxChain } : {}),
|
||||
...(element.groupAutomation ? { automation: element.groupAutomation } : {}),
|
||||
volume: element.groupVolume ?? 1,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
tracks.push(track);
|
||||
}
|
||||
} catch (err: unknown) {
|
||||
// An FX failure is fatal for the whole mix. Every other failure mode
|
||||
// degrades gracefully — the track drops and siblings continue — but
|
||||
@@ -1141,7 +1289,7 @@ export async function processCompositionAudio(
|
||||
// The producer only surfaces audio failures when `success` is false; mixing
|
||||
// the remaining tracks made the omitted cue indistinguishable from a valid
|
||||
// render unless someone manually audited that exact audio window.
|
||||
if (failures.length > 0) {
|
||||
const bail = (): MixResult => {
|
||||
try {
|
||||
rmSync(workDir, { recursive: true, force: true });
|
||||
} catch {
|
||||
@@ -1157,7 +1305,98 @@ export async function processCompositionAudio(
|
||||
),
|
||||
failures,
|
||||
};
|
||||
};
|
||||
if (failures.length > 0) return bail();
|
||||
|
||||
// Sub-mix each group into one processed bus (design doc §1.3), then fold it
|
||||
// into the flat track list as a single AudioTrack — everything downstream
|
||||
// (the final mixAudioTracks call) never has to know groups exist. Group
|
||||
// clock is composition time (offset 0): a group has no `data-start` of its
|
||||
// own, and members are already delayed to their composition positions
|
||||
// inside the sub-mix, so the group WAV's t=0 IS composition time.
|
||||
for (const [groupId, memberTracks] of groupTracks) {
|
||||
const meta = groupMeta.get(groupId);
|
||||
if (!meta) continue;
|
||||
const groupWavPath = join(workDir, `group-${groupId}.wav`);
|
||||
const subMix = await mixGroupMembers(
|
||||
memberTracks,
|
||||
groupWavPath,
|
||||
totalDuration,
|
||||
effectiveSignal,
|
||||
config,
|
||||
);
|
||||
if (!subMix.success) {
|
||||
failures.push({
|
||||
stage: "mix",
|
||||
reason: "ffmpeg_failed",
|
||||
owner: "system",
|
||||
retryable: false,
|
||||
elementId: groupId,
|
||||
detail: boundedDetail(
|
||||
`Group sub-mix failed for group ${groupId}: ${subMix.error ?? "unknown"}`,
|
||||
),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
// Composition-time automation (offset 0, duration totalDuration) — same
|
||||
// resolve/lane/bake path a member uses, just anchored at the group clock
|
||||
// instead of a clip's own start.
|
||||
const automation = meta.automation
|
||||
? resolveAutomation(
|
||||
parseAutomation(meta.automation),
|
||||
meta.fxChain ? parseAudioFxChain(meta.fxChain) : undefined,
|
||||
)
|
||||
: null;
|
||||
const laneKeyframes = automation ? volumeLaneKeyframes(automation, 0, totalDuration) : null;
|
||||
const envelope =
|
||||
laneKeyframes && laneKeyframes.length > 0
|
||||
? { keyframes: laneKeyframes, trackStart: 0, baseVolume: meta.volume }
|
||||
: null;
|
||||
|
||||
let groupSrcPath = groupWavPath;
|
||||
let bakedEnvelope = false;
|
||||
if (meta.fxChain) {
|
||||
const chain = parseAudioFxChain(meta.fxChain);
|
||||
const fxResult = await applyAudioFxChain(
|
||||
groupSrcPath,
|
||||
chain,
|
||||
join(workDir, `group-${groupId}-fx.wav`),
|
||||
{
|
||||
trackId: groupId,
|
||||
signal: effectiveSignal,
|
||||
...(automation ? { automation } : {}),
|
||||
...(envelope ? { envelope } : {}),
|
||||
},
|
||||
);
|
||||
groupSrcPath = fxResult.path;
|
||||
bakedEnvelope = fxResult.envelopeBaked;
|
||||
}
|
||||
if (envelope && !bakedEnvelope) {
|
||||
bakedEnvelope = applyVolumeEnvelopeToWav(
|
||||
groupSrcPath,
|
||||
envelope.keyframes,
|
||||
envelope.trackStart,
|
||||
envelope.baseVolume,
|
||||
);
|
||||
}
|
||||
|
||||
// `end` is already the render's totalDuration, so the outer mix's own
|
||||
// atrim-to-totalDuration clips this track exactly the same way it would
|
||||
// an ungrouped one whose clip ran to the end of the render — a group's FX
|
||||
// tail gets the same "never past the end of the video" treatment member
|
||||
// tails get, for free, with no separate tailSeconds bookkeeping needed.
|
||||
tracks.push({
|
||||
id: groupId,
|
||||
srcPath: groupSrcPath,
|
||||
start: 0,
|
||||
end: totalDuration,
|
||||
mediaStart: 0,
|
||||
duration: totalDuration,
|
||||
volume: bakedEnvelope ? 1.0 : meta.volume,
|
||||
});
|
||||
}
|
||||
if (failures.length > 0) return bail();
|
||||
|
||||
const mixResult = await mixAudioTracks(tracks, outputPath, totalDuration, signal, config);
|
||||
|
||||
|
||||
@@ -18,6 +18,18 @@ export interface AudioElement {
|
||||
fxChain?: string;
|
||||
/** Serialised automation JSON from `data-automation`, when set. */
|
||||
automation?: string;
|
||||
/** The id of the `data-audio-group` this element is a member of, when set. */
|
||||
groupId?: string;
|
||||
/**
|
||||
* The group's own FX chain / automation / volume, duplicated identically
|
||||
* onto every member of the same group — all resolved from the one
|
||||
* `resolveAudioGroups` call `parseAudioElements` already makes, so a
|
||||
* render pass building a group sub-mix from a set of members has
|
||||
* everything it needs from any one of them without a second lookup.
|
||||
*/
|
||||
groupFxChain?: string;
|
||||
groupAutomation?: string;
|
||||
groupVolume?: number;
|
||||
type: "audio" | "video";
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
amix-work/
|
||||
Executable
+103
@@ -0,0 +1,103 @@
|
||||
#!/usr/bin/env bash
|
||||
# Spike: does a nested amix (group bus) preserve the level of a flat amix?
|
||||
#
|
||||
# Invariant under test: a group whose FX chain is EMPTY must be a no-op on the
|
||||
# mix. Grouping is routing, not processing — if grouping alone changes the
|
||||
# level, every grouped export is silently wrong.
|
||||
set -euo pipefail
|
||||
|
||||
D="$(dirname "$0")/amix-work"
|
||||
rm -rf "$D"; mkdir -p "$D"
|
||||
cd "$D"
|
||||
|
||||
SR=48000
|
||||
DUR=3
|
||||
|
||||
# Four tracks, distinct frequencies so nothing cancels, distinct amplitudes so a
|
||||
# mis-weighted track shows up rather than averaging out.
|
||||
ffmpeg -v error -f lavfi -i "sine=frequency=220:sample_rate=$SR:duration=$DUR" -af "volume=0.50" -c:a pcm_s16le t1.wav
|
||||
ffmpeg -v error -f lavfi -i "sine=frequency=440:sample_rate=$SR:duration=$DUR" -af "volume=0.25" -c:a pcm_s16le t2.wav
|
||||
ffmpeg -v error -f lavfi -i "sine=frequency=880:sample_rate=$SR:duration=$DUR" -af "volume=0.40" -c:a pcm_s16le t3.wav
|
||||
ffmpeg -v error -f lavfi -i "sine=frequency=1760:sample_rate=$SR:duration=$DUR" -af "volume=0.15" -c:a pcm_s16le t4.wav
|
||||
|
||||
rms () { # $1 = wav -> RMS dB
|
||||
ffmpeg -v info -i "$1" -af astats=metadata=1:reset=0 -f null - 2>&1 \
|
||||
| awk -F'dB: ' '/Overall/{o=1} o&&/RMS level dB/{print $2; exit}'
|
||||
}
|
||||
|
||||
echo "=== per-track RMS (dB) ==="
|
||||
for f in t1 t2 t3 t4; do printf " %-4s %s\n" "$f" "$(rms $f.wav)"; done
|
||||
echo
|
||||
|
||||
# --- ARM A: flat mix, today's shipped shape -------------------------------
|
||||
# amix normalizes by input count; multiply back by the same count.
|
||||
ffmpeg -v error -i t1.wav -i t2.wav -i t3.wav -i t4.wav -filter_complex \
|
||||
"[0:a]apad,atrim=0:$DUR[a0];\
|
||||
[1:a]apad,atrim=0:$DUR[a1];\
|
||||
[2:a]apad,atrim=0:$DUR[a2];\
|
||||
[3:a]apad,atrim=0:$DUR[a3];\
|
||||
[a0][a1][a2][a3]amix=inputs=4:duration=longest:dropout_transition=0[mixed];\
|
||||
[mixed]volume=4[out]" -map "[out]" -c:a pcm_s16le flat.wav
|
||||
|
||||
# --- ARM B: nested, compensated PER NODE ---------------------------------
|
||||
# group A = t1+t2 (2 inputs -> x2). outer = groupA + t3 + t4 (3 inputs -> x3).
|
||||
ffmpeg -v error -i t1.wav -i t2.wav -i t3.wav -i t4.wav -filter_complex \
|
||||
"[0:a]apad,atrim=0:$DUR[a0];\
|
||||
[1:a]apad,atrim=0:$DUR[a1];\
|
||||
[2:a]apad,atrim=0:$DUR[a2];\
|
||||
[3:a]apad,atrim=0:$DUR[a3];\
|
||||
[a0][a1]amix=inputs=2:duration=longest:dropout_transition=0[gmix];\
|
||||
[gmix]volume=2[gA];\
|
||||
[gA][a2][a3]amix=inputs=3:duration=longest:dropout_transition=0[mixed];\
|
||||
[mixed]volume=3[out]" -map "[out]" -c:a pcm_s16le nested_ok.wav
|
||||
|
||||
# --- ARM C: nested, but the outer node keeps the GLOBAL track count -------
|
||||
# The plausible mistake: tracks.length is 4, the outer amix has 3 inputs.
|
||||
ffmpeg -v error -i t1.wav -i t2.wav -i t3.wav -i t4.wav -filter_complex \
|
||||
"[0:a]apad,atrim=0:$DUR[a0];\
|
||||
[1:a]apad,atrim=0:$DUR[a1];\
|
||||
[2:a]apad,atrim=0:$DUR[a2];\
|
||||
[3:a]apad,atrim=0:$DUR[a3];\
|
||||
[a0][a1]amix=inputs=2:duration=longest:dropout_transition=0[gmix];\
|
||||
[gmix]volume=2[gA];\
|
||||
[gA][a2][a3]amix=inputs=3:duration=longest:dropout_transition=0[mixed];\
|
||||
[mixed]volume=4[out]" -map "[out]" -c:a pcm_s16le nested_bug.wav
|
||||
|
||||
# --- ARM D: nested with normalize=0, no compensation anywhere ------------
|
||||
ffmpeg -v error -i t1.wav -i t2.wav -i t3.wav -i t4.wav -filter_complex \
|
||||
"[0:a]apad,atrim=0:$DUR[a0];\
|
||||
[1:a]apad,atrim=0:$DUR[a1];\
|
||||
[2:a]apad,atrim=0:$DUR[a2];\
|
||||
[3:a]apad,atrim=0:$DUR[a3];\
|
||||
[a0][a1]amix=inputs=2:normalize=0:duration=longest:dropout_transition=0[gA];\
|
||||
[gA][a2][a3]amix=inputs=3:normalize=0:duration=longest:dropout_transition=0[out]" \
|
||||
-map "[out]" -c:a pcm_s16le nested_norm0.wav
|
||||
|
||||
# --- ARM E: FLAT with normalize=0 ----------------------------------------
|
||||
ffmpeg -v error -i t1.wav -i t2.wav -i t3.wav -i t4.wav -filter_complex \
|
||||
"[0:a]apad,atrim=0:$DUR[a0];\
|
||||
[1:a]apad,atrim=0:$DUR[a1];\
|
||||
[2:a]apad,atrim=0:$DUR[a2];\
|
||||
[3:a]apad,atrim=0:$DUR[a3];\
|
||||
[a0][a1][a2][a3]amix=inputs=4:normalize=0:duration=longest:dropout_transition=0[out]" \
|
||||
-map "[out]" -c:a pcm_s16le flat_norm0.wav
|
||||
|
||||
echo "=== mix RMS (dB) ==="
|
||||
for f in flat nested_ok nested_bug nested_norm0 flat_norm0; do
|
||||
printf " %-14s %s\n" "$f" "$(rms $f.wav)"
|
||||
done
|
||||
echo
|
||||
|
||||
# --- sample-exactness: null test (A inverted + B must be silence) ---------
|
||||
null_test () { # $1 $2 -> peak dB of the difference
|
||||
ffmpeg -v info -i "$1" -i "$2" -filter_complex \
|
||||
"[1:a]volume=-1[inv];[0:a][inv]amix=inputs=2:normalize=0,astats=metadata=1:reset=0[d]" \
|
||||
-map "[d]" -f null - 2>&1 \
|
||||
| awk -F'dB: ' '/Overall/{o=1} o&&/Peak level dB/{print $2; exit}'
|
||||
}
|
||||
|
||||
echo "=== null tests (peak dB of difference; -inf or < -90 = identical) ==="
|
||||
printf " flat vs nested_ok %s\n" "$(null_test flat.wav nested_ok.wav)"
|
||||
printf " flat vs nested_bug %s\n" "$(null_test flat.wav nested_bug.wav)"
|
||||
printf " flat vs nested_norm0 %s\n" "$(null_test flat.wav nested_norm0.wav)"
|
||||
printf " flat vs flat_norm0 %s\n" "$(null_test flat.wav flat_norm0.wav)"
|
||||
Reference in New Issue
Block a user