mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 09:16:55 +00:00
feat(audio): composition master bus (data-fx-chain on the root) (#5420)
* feat(audio): composition master bus (data-fx-chain on the root) A clip can belong to only one audio group, so there was no way to put one effect, such as a limiter, across the whole mix. The data-fx-chain on the top-level composition root now runs once over the sum of every clip and every group, after each group's own chain and fader. No new attribute: a sub-composition root's data-fx-chain was never applied to anything and still is not. Render: when a master chain exists, mixAudioTracks writes a float WAV instead of AAC, applyAudioFxChain runs the chain over it, and ffmpeg encodes AAC last, so the chain reads unclipped samples. An unreadable chain is a mix failure, as for a group. Preview: the transport splices the chain between the clips' sum and the monitor, re-anchored per play generation and watched for edits. Only the top-level root is read. No master automation: the root's data-automation is not read as the bus's. No lint rule: data-fx-chain is valid on clips, groups and roots, and the one place it does nothing (a sub-composition root) has always been silent. — Jerrai * fix(core): preview routes every audible video through the master bus A root data-fx-chain is attached only inside the Web Audio graph, but a plainly audible <video> joined that graph only for its own chain, group, automation or a volume over 1. So preview limited the music and left the talking head dry, and a composition whose only audio is video got no master chain at all, while export sums every audible element into the master. A master chain on the top-level composition root is now a reason to join, the same as a video's own chain. Elements under a root with no chain keep native output. Sub-composition roots are still ignored. — Jerrai * fix(audio): master bus review round 2 (temp dir, tests, docs, Studio meter) - audioMixer: a fatal FX error in the master pass or a group bus left the work directory (the full float sum, about 1.4 GB an hour) on disk. The final mix now runs in try/finally and the group rethrow removes it first. - Tests: the "once over the sum" case now uses a limiter then a gain, so a per-clip chain (-10.4) and a doubled one (-18) both fail it; a new case proves the sum stays float until the chain reads it; the fader case uses a group volume of 0.5 so the fader's position is exercised; the work-dir removal is covered for both paths. - Docs: "Three rules exist" (the master-bus lint rule is gone), a bad master chain stops the render with audio_processing_failed, a sub-composition file opened on its own has its root chain as the master bus, and a seek clears a group or master limiter. - Studio: TimelineElement.masterChain carries the root's data-fx-chain, so the program meter shows for a video-only composition whose only audio routing is the master chain, as preview now plays it. - Comment blocks over four lines cut; webAudioTransport.setRate split so its CRAP score is under the Fallow threshold. — Jerrai * fix(audio): master chain reaches tracks the render adds; honest plain-limiter copy - Master chain: it was carried on each parsed AudioElement, so a track the producer adds itself (appendAutoDetectedVideoAudio, the browser-discovered audio in probeStage) never had it, and a composition whose only audio was such a track exported with no master chain, no ceiling and no warning. It is now a composition property: parseMasterFxChain(html) at compile, carried on CompiledComposition and CompositionMetadata, passed to processCompositionAudio by runAudioStage (in-process and distributed plan). AudioElement.masterFxChain is removed so there is one source. Each forward (compile stage, local orchestrator, distributed plan) has a test that fails when it is dropped. - Plain limiter: it follows the level with no lookahead and Level after is applied past the ceiling, so it cannot promise "nothing gets past" or "never clips". Studio copy (description, panel text, knob label, summary) and the hyperframes-audio skill now say a fast or inter-sample peak can pass it and point to the true-peak limiter for a delivery limit. A copy test guards it. — Jerrai * fix(producer): keep master bus comments within the ratchet; run the plan test with the ffmpeg tests Drop three one-line field comments that raised their files' comment share, and list plan.masterBus.test.ts with the integration tests because plan() reads the ffmpeg version. — Jerrai * chore(ci): repin the producer test-classification source in the reachability map The reachability map pins the hash of test-classification.mjs, which now lists one more integration test. The map's producer lane command already covers every test under src, in both the unit and integration matrix lanes. — Jerrai
This commit is contained in:
@@ -180,6 +180,41 @@ dangles.
|
||||
preview never lays it out.
|
||||
- Group ids and element ids share one namespace.
|
||||
|
||||
### The master bus: `data-fx-chain` on the composition root
|
||||
|
||||
```html
|
||||
<div id="root" data-composition-id="main" data-width="1920" data-height="1080"
|
||||
data-fx-chain="{"version":1,"nodes":[
|
||||
{"type":"truepeak","id":"m1","label":"Delivery ceiling",
|
||||
"params":{"ceiling":-1.5,"lookahead":3,"release":80}}
|
||||
]}"
|
||||
>
|
||||
```
|
||||
|
||||
The master bus is the `data-fx-chain` on the top-level composition root. It runs
|
||||
once over the sum of every clip and every group, after each group's own chain and
|
||||
fader, so a limiter here sees the peak of the whole mix. A clip can belong to
|
||||
only one group, so this is the way to put one effect across everything.
|
||||
|
||||
- Only the **top-level** composition root is read as the master bus. On a clip or
|
||||
a group `data-fx-chain` keeps its own meaning, and on a sub-composition's root
|
||||
it is not applied while a parent composition contains it. A sub-composition file
|
||||
previewed or rendered on its own is the top-level root, so its chain is the
|
||||
master bus there.
|
||||
- There is no master automation: `data-automation` on the root is not read as
|
||||
the master's.
|
||||
- The render sums the tracks to a float WAV, runs the chain, and only then
|
||||
encodes AAC, so the chain reads unclipped samples. Preview inserts the chain
|
||||
between the clips' sum and the monitor.
|
||||
- With a master chain on the root, preview plays every audible `<video>` through
|
||||
that bus too, as the render does. Without one, a video at volume 1 with no
|
||||
chain, group or automation of its own stays on native output.
|
||||
- An unreadable master chain **fails the whole mix**: the render stops with an
|
||||
`audio_processing_failed` error under every strictness setting, and exports
|
||||
nothing. Preview plays dry with no message, and no lint rule checks a chain's
|
||||
contents, so one typo on the root blocks every export of that composition.
|
||||
Check a master chain in preview before rendering.
|
||||
|
||||
## Effect registry
|
||||
|
||||
Sixteen effects. Values outside a range are clamped on read. **AUTO** marks a
|
||||
@@ -223,7 +258,7 @@ the peak arrives, and holds that estimate at `ceiling`. Channels share one
|
||||
gain. It delays its output by `lookahead` plus about 0.3 ms (3.3 to 10.3 ms across
|
||||
the range): the offline render trims that delay, live preview plays it, so the
|
||||
sound is that much behind the picture. A seek clears the limiter's delay line and
|
||||
held gain on a group bus. Changing `lookahead` rebuilds the node.
|
||||
held gain on a group or master bus. Changing `lookahead` rebuilds the node.
|
||||
|
||||
The ceiling is held against that 4x estimate, not against the waveform itself. A
|
||||
4x interpolator cannot see the peaks of content close to Nyquist, so the true peak
|
||||
|
||||
@@ -50,14 +50,19 @@ describe("AUDIBLE_MEDIA_SELECTOR", () => {
|
||||
describe("audibleVideoNeedsWebAudio", () => {
|
||||
it("keeps a unity, unprocessed video on native output", () => {
|
||||
expect(audibleVideoNeedsWebAudio({})).toBe(false);
|
||||
expect(audibleVideoNeedsWebAudio({ volume: 1, masterChain: null })).toBe(false);
|
||||
expect(audibleVideoNeedsWebAudio({ masterChain: "" })).toBe(false);
|
||||
expect(audibleVideoNeedsWebAudio({ volume: 1, fxChain: "", automation: null })).toBe(false);
|
||||
expect(audibleVideoNeedsWebAudio({ volume: Number.NaN })).toBe(false);
|
||||
});
|
||||
|
||||
it.each([{ volume: 1.5 }, { fxChain: "[]" }, { automation: "{}" }, { audioGroup: "music" }])(
|
||||
"routes a video carrying %o",
|
||||
(fields) => {
|
||||
expect(audibleVideoNeedsWebAudio(fields)).toBe(true);
|
||||
},
|
||||
);
|
||||
it.each([
|
||||
{ volume: 1.5 },
|
||||
{ fxChain: "[]" },
|
||||
{ automation: "{}" },
|
||||
{ audioGroup: "music" },
|
||||
{ volume: 1, masterChain: '{"version":1,"nodes":[]}' },
|
||||
])("routes a video carrying %o", (fields) => {
|
||||
expect(audibleVideoNeedsWebAudio(fields)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -18,8 +18,10 @@ export function audibleVideoNeedsWebAudio(fields: {
|
||||
fxChain?: string | null;
|
||||
automation?: string | null;
|
||||
audioGroup?: string | null;
|
||||
masterChain?: string | null;
|
||||
}): boolean {
|
||||
return (
|
||||
(fields.volume ?? 1) > 1 || Boolean(fields.fxChain || fields.automation || fields.audioGroup)
|
||||
(fields.volume ?? 1) > 1 ||
|
||||
Boolean(fields.fxChain || fields.automation || fields.audioGroup || fields.masterChain)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -288,7 +288,8 @@ export const HF_AUDIO_FX: readonly HfAudioFxDef[] = [
|
||||
id: "limiter",
|
||||
label: "Limiter",
|
||||
group: "dynamics",
|
||||
description: "Hard ceiling: nothing gets past the limit.",
|
||||
description:
|
||||
"Peak limiter that follows the signal's level. It has no lookahead, so a fast peak or an inter-sample peak can pass the ceiling; use the true-peak limiter for a delivery limit.",
|
||||
params: [
|
||||
{
|
||||
kind: "number",
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { defaultAudioFxParams, HF_AUDIO_FX } from "./audioFx.js";
|
||||
import { defaultAudioFxParams, getAudioFxDef, HF_AUDIO_FX } from "./audioFx.js";
|
||||
import { HF_AUDIO_FX_PRESETS } from "./audioFxPresets.js";
|
||||
import { audioBandAt, BANDS, EFFECT_COPY, PRESET_PROBLEM, SUMMARY } from "./audioFxCopy.js";
|
||||
|
||||
@@ -197,3 +197,25 @@ describe("the true-peak copy does not promise a hard ceiling", () => {
|
||||
expect(hint).toMatch(/headroom/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the plain limiter copy does not promise a hard ceiling", () => {
|
||||
// The limiter follows the signal's level with no lookahead, and Level after is applied past
|
||||
// the ceiling, so a fast or inter-sample peak can end above it.
|
||||
it("says nothing about never clipping or nothing getting past", () => {
|
||||
const copy = EFFECT_COPY.limiter;
|
||||
const summary = SUMMARY.limiter;
|
||||
const description = getAudioFxDef("limiter")?.description;
|
||||
if (!copy || !summary || !description) throw new Error("limiter copy is missing");
|
||||
const shown = [
|
||||
description,
|
||||
copy.does,
|
||||
copy.reachFor,
|
||||
copy.params.limit?.label,
|
||||
summary({}),
|
||||
].join("\n");
|
||||
expect(shown).not.toMatch(
|
||||
/never|nothing|guarantee|hard ceiling|no louder|must not|cannot exceed/i,
|
||||
);
|
||||
expect(shown).toMatch(/can (still )?(get )?pass|get past/i);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -152,13 +152,13 @@ export const EFFECT_COPY: Record<string, EffectCopy> = {
|
||||
},
|
||||
limiter: {
|
||||
title: "Peak Ceiling",
|
||||
does: "Nothing gets louder than this, ever. A safety net at the end of the chain.",
|
||||
reachFor: "You want to be sure it never clips or spikes.",
|
||||
does: "Pulls the loudest moments down toward this ceiling. It reacts to what it hears, so a fast peak can still get past, and Level after is applied on top of it.",
|
||||
reachFor: "You want peaks held down and can accept a small overshoot.",
|
||||
primary: "limit",
|
||||
primaryEnds: { low: "A lot of headroom", high: "Right up to the edge" },
|
||||
params: {
|
||||
limit: {
|
||||
label: "Never exceed",
|
||||
label: "Ceiling",
|
||||
ends: { low: "A lot of headroom", high: "Right up to the edge" },
|
||||
},
|
||||
attack: { label: "How fast it catches" },
|
||||
@@ -411,7 +411,7 @@ export const SUMMARY: Record<string, (p: P) => string> = {
|
||||
: `${n(p.gain) > 0 ? "More" : "Less"} sparkle above ${hz(p.frequency)}`,
|
||||
compressor: (p) =>
|
||||
`Evening out, ${strength(Math.min(1, (n(p.ratio, 3) - 1) / 7), ["gentle", "moderate", "firm"])}`,
|
||||
limiter: (p) => `Nothing louder than ${n(p.limit, -1)} dB`,
|
||||
limiter: (p) => `Peaks pulled down toward ${n(p.limit, -1)} dB`,
|
||||
truepeak: (p) => `Peaks pulled down toward ${n(p.ceiling, -1)} dBTP`,
|
||||
gate: (p) => `Closing gaps quieter than ${n(p.threshold, -45)} dB`,
|
||||
saturate: (p) =>
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
ensureAudioGroupInertStyle,
|
||||
HF_AUDIO_GROUP_ATTR,
|
||||
isMemberGroupHidden,
|
||||
readMasterFxChain,
|
||||
resolveAudioGroups,
|
||||
resolveCarveSourceIds,
|
||||
resolveGroupElement,
|
||||
@@ -305,3 +306,27 @@ describe("ensureAudioGroupInertStyle", () => {
|
||||
expect(document.querySelectorAll("#__hf-audio-group-inert")).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("readMasterFxChain", () => {
|
||||
const CHAIN = `{"version":1,"nodes":[]}`;
|
||||
|
||||
it("reads the chain off the top-level composition root", () => {
|
||||
document.body.innerHTML = `<div data-composition-id="main" data-fx-chain="${CHAIN}"></div>`;
|
||||
expect(readMasterFxChain(document)).toBe('{"version":1,"nodes":[]}');
|
||||
});
|
||||
|
||||
it("is undefined when the root carries none", () => {
|
||||
document.body.innerHTML = `<div data-composition-id="main"><audio id="a"></audio></div>`;
|
||||
expect(readMasterFxChain(document)).toBeUndefined();
|
||||
});
|
||||
|
||||
it("ignores a chain on a nested composition root", () => {
|
||||
document.body.innerHTML = `<div data-composition-id="main"><div data-composition-id="inner" data-fx-chain="${CHAIN}"></div></div>`;
|
||||
expect(readMasterFxChain(document)).toBeUndefined();
|
||||
});
|
||||
|
||||
it("ignores a chain on a clip", () => {
|
||||
document.body.innerHTML = `<div data-composition-id="main"><audio id="a" data-fx-chain="${CHAIN}"></audio></div>`;
|
||||
expect(readMasterFxChain(document)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -128,6 +128,22 @@ export function isMemberGroupHidden(
|
||||
return resolveGroupElement(doc, groupId)?.hasAttribute("data-hidden") ?? false;
|
||||
}
|
||||
|
||||
/**
|
||||
* The top-level composition root, the only element the master bus is read from;
|
||||
* a sub-composition's root is skipped, so repeating one cannot stack chains.
|
||||
*/
|
||||
export function resolveMasterBusElement(root: ParentNode): Element | null {
|
||||
for (const el of root.querySelectorAll("[data-composition-id]")) {
|
||||
if (!el.parentElement?.closest("[data-composition-id]")) return el;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** The master bus chain: the top-level root's `data-fx-chain`, if it carries one. */
|
||||
export function readMasterFxChain(root: ParentNode): string | undefined {
|
||||
return resolveMasterBusElement(root)?.getAttribute(HF_AUDIO_FX_ATTR) || undefined;
|
||||
}
|
||||
|
||||
export function resolveAudioGroups(root: ParentNode): HfAudioGroup[] {
|
||||
const membersByGroup = new Map<string, string[]>();
|
||||
const candidates = root.querySelectorAll(
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// @vitest-environment happy-dom
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { attachElementFxChain } from "./audioFx.js";
|
||||
import { attachElementFxChain, type FxAttributes } from "./audioFx.js";
|
||||
import { defaultAudioFxParams, HF_AUDIO_FX, HF_AUDIO_FX_ATTR } from "../audioFx.js";
|
||||
|
||||
/**
|
||||
@@ -435,7 +435,7 @@ describe("attachElementFxChain", () => {
|
||||
],
|
||||
});
|
||||
|
||||
const build = () => {
|
||||
const build = (attrs?: FxAttributes) => {
|
||||
const clock = { currentTime: 0 };
|
||||
const made: { frequency: TimedParam }[] = [];
|
||||
class TimedNode extends Node {
|
||||
@@ -461,6 +461,7 @@ describe("attachElementFxChain", () => {
|
||||
new Node() as never,
|
||||
new Node() as never,
|
||||
{ scheduledAt: 0, elapsed: 0, rate: 1 },
|
||||
attrs,
|
||||
);
|
||||
return { clock, node, handle, param: () => made[0]?.frequency as unknown as TimedParam };
|
||||
};
|
||||
@@ -495,6 +496,14 @@ describe("attachElementFxChain", () => {
|
||||
expect(param().last()).toEqual({ time: 4, duration: 1 });
|
||||
});
|
||||
|
||||
it("books no lane for a chain told its element carries none (the master bus)", () => {
|
||||
// The root's `data-automation` belongs to the composition, not to the bus.
|
||||
const { param } = build({ automation: null });
|
||||
expect(param().last()).toBeUndefined();
|
||||
const { param: ownParam } = build();
|
||||
expect(ownParam().last()).toEqual({ time: 0, duration: 8 });
|
||||
});
|
||||
|
||||
it("ignores a rate that is not a rate", () => {
|
||||
const { clock, handle, param } = build();
|
||||
clock.currentTime = 2;
|
||||
|
||||
@@ -38,10 +38,9 @@ const NO_AUTOMATION: HfAutomation = { version: 1, lanes: [] };
|
||||
function readAutomation(
|
||||
el: { getAttribute?(name: string): string | null },
|
||||
chain: HfAudioFxChain,
|
||||
attr: string | null = HF_AUDIO_AUTOMATION_ATTR,
|
||||
): HfAutomation {
|
||||
const raw =
|
||||
(typeof el.getAttribute === "function" ? el.getAttribute(HF_AUDIO_AUTOMATION_ATTR) : null) ??
|
||||
"";
|
||||
const raw = (attr && typeof el.getAttribute === "function" ? el.getAttribute(attr) : null) ?? "";
|
||||
if (!raw) return NO_AUTOMATION;
|
||||
try {
|
||||
return resolveAutomation(parseAutomation(raw), chain);
|
||||
@@ -119,12 +118,18 @@ export interface ElementFxHandle {
|
||||
setRate(rate: number): void;
|
||||
}
|
||||
|
||||
/** The attribute a chain's lanes are read from; the master bus has none. */
|
||||
export interface FxAttributes {
|
||||
automation: string | null;
|
||||
}
|
||||
|
||||
export function attachElementFxChain(
|
||||
ctx: BaseAudioContext,
|
||||
el: { getAttribute?(name: string): string | null },
|
||||
source: AudioNode,
|
||||
destination: AudioNode,
|
||||
timing?: AutomationTiming,
|
||||
attrs: FxAttributes = { automation: HF_AUDIO_AUTOMATION_ATTR },
|
||||
): ElementFxHandle | null {
|
||||
const { chain } = readChain(el);
|
||||
|
||||
@@ -202,7 +207,13 @@ export function attachElementFxChain(
|
||||
const scheduleFor = (next: HfAudioFxChain, at: AutomationTiming | null): void => {
|
||||
automated =
|
||||
at && handle
|
||||
? scheduleChainAutomation(readAutomation(el, next), next, handle.nodes, at, handle.presets)
|
||||
? scheduleChainAutomation(
|
||||
readAutomation(el, next, attrs.automation),
|
||||
next,
|
||||
handle.nodes,
|
||||
at,
|
||||
handle.presets,
|
||||
)
|
||||
: [];
|
||||
};
|
||||
|
||||
@@ -282,7 +293,7 @@ export function attachElementFxChain(
|
||||
});
|
||||
observer.observe(target, {
|
||||
attributes: true,
|
||||
attributeFilter: [HF_AUDIO_FX_ATTR, HF_AUDIO_AUTOMATION_ATTR],
|
||||
attributeFilter: attrs.automation ? [HF_AUDIO_FX_ATTR, attrs.automation] : [HF_AUDIO_FX_ATTR],
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -5883,6 +5883,69 @@ describe("initSandboxRuntimeModular", () => {
|
||||
expect(ctx.mediaElementSources).toBe(0);
|
||||
});
|
||||
|
||||
describe("under a composition root with a master chain", () => {
|
||||
const MASTER_CHAIN = '{"version":1,"nodes":[]}';
|
||||
const setRootChain = () =>
|
||||
document.querySelector("[data-root]")?.setAttribute("data-fx-chain", MASTER_CHAIN);
|
||||
|
||||
it("joins the graph for a plain audible video at unity, so the master bus limits it", async () => {
|
||||
const video = mountMedia("video", { "data-has-audio": "true", "data-volume": "1" });
|
||||
setRootChain();
|
||||
const captureSpy = spyCapture();
|
||||
|
||||
await startPlayback();
|
||||
|
||||
expect(captureSpy).toHaveBeenCalledTimes(1);
|
||||
expect(captureSpy.mock.calls[0]?.[0]).toBe(video);
|
||||
expect(captureSpy.mock.calls[0]?.[4]).toBe(1);
|
||||
});
|
||||
|
||||
it("acquires a media element source for it", async () => {
|
||||
mountMedia("video", { "data-has-audio": "true" });
|
||||
setRootChain();
|
||||
|
||||
await startPlayback();
|
||||
await Promise.resolve();
|
||||
|
||||
expect(ctx.mediaElementSources).toBe(1);
|
||||
});
|
||||
|
||||
it("joins a video whose only audio is the composition, and every audio clip beside it", async () => {
|
||||
const video = mountMedia("video", { "data-has-audio": "true" });
|
||||
const audio = mountMedia("audio");
|
||||
setRootChain();
|
||||
const captureSpy = spyCapture();
|
||||
|
||||
await startPlayback();
|
||||
|
||||
expect(captureSpy.mock.calls.map((call) => call[0])).toEqual([video, audio]);
|
||||
});
|
||||
|
||||
it("still keeps a muted or data-has-audio=false video out of the graph", async () => {
|
||||
mountMedia("video", { "data-has-audio": "true", muted: "" });
|
||||
mountMedia("video", { "data-has-audio": "false" });
|
||||
setRootChain();
|
||||
const captureSpy = spyCapture();
|
||||
|
||||
await startPlayback();
|
||||
|
||||
expect(captureSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not count a chain on a sub-composition root", async () => {
|
||||
const video = mountMedia("video", { "data-has-audio": "true" });
|
||||
const inner = document.createElement("div");
|
||||
inner.setAttribute("data-composition-id", "inner");
|
||||
inner.setAttribute("data-fx-chain", MASTER_CHAIN);
|
||||
video.before(inner);
|
||||
const captureSpy = spyCapture();
|
||||
|
||||
await startPlayback();
|
||||
|
||||
expect(captureSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
["data-fx-chain", "[]"],
|
||||
["data-automation", "[]"],
|
||||
|
||||
@@ -114,6 +114,7 @@ import {
|
||||
ensureAudioGroupInertStyle,
|
||||
HF_AUDIO_GROUP_TAG,
|
||||
isMemberGroupHidden,
|
||||
readMasterFxChain,
|
||||
} from "../audioGroups";
|
||||
import { clampNativeMediaVolume } from "../audioGain";
|
||||
import { quantizeTimeToFrame } from "../inline-scripts/parityContract";
|
||||
@@ -383,7 +384,7 @@ function pageAnimationsForOnePass(): () => Animation[] {
|
||||
return () => (list ??= document.getAnimations());
|
||||
}
|
||||
|
||||
// A `<video>` joins only for a gain `el.volume` cannot express: capture is a one-way door.
|
||||
// A `<video>` joins only for a gain `el.volume` cannot express or a root master chain: capture is a one-way door.
|
||||
const joinsWebAudio = (el: Element): el is HTMLMediaElement =>
|
||||
isAudioElement(el) ||
|
||||
(isVideoElement(el) &&
|
||||
@@ -393,6 +394,7 @@ const joinsWebAudio = (el: Element): el is HTMLMediaElement =>
|
||||
fxChain: el.getAttribute("data-fx-chain"),
|
||||
automation: el.getAttribute("data-automation"),
|
||||
audioGroup: el.getAttribute("data-audio-group"),
|
||||
masterChain: readMasterFxChain(el.ownerDocument),
|
||||
}));
|
||||
const WEB_AUDIO_MEDIA = "audio[data-start], video[data-start]";
|
||||
const webAudioMediaIn = (root: ParentNode): HTMLMediaElement[] =>
|
||||
|
||||
@@ -1113,6 +1113,92 @@ describe("WebAudioTransport", () => {
|
||||
expect(fader.gain.value).toBeCloseTo(0.5, 6);
|
||||
});
|
||||
|
||||
describe("master bus", () => {
|
||||
const GAIN_CHAIN = JSON.stringify({
|
||||
version: 1,
|
||||
nodes: [{ type: "gain", id: "g", params: { gain: -6 } }],
|
||||
});
|
||||
|
||||
function withBusOut(setup: ReturnType<typeof setupGroupTransport>) {
|
||||
const busOut = { gain: { value: 1 }, connect: vi.fn(), disconnect: vi.fn() };
|
||||
(setup.transport as unknown as { _busOut: unknown })._busOut = busOut;
|
||||
return busOut;
|
||||
}
|
||||
|
||||
it("splices the root's chain between the sum and the monitor", async () => {
|
||||
document.body.innerHTML = `<div id="root" data-composition-id="main"></div>`;
|
||||
document.getElementById("root")!.setAttribute("data-fx-chain", GAIN_CHAIN);
|
||||
const setup = setupGroupTransport();
|
||||
const busOut = withBusOut(setup);
|
||||
|
||||
await scheduleGrouped(setup.transport, setup.gen, "a");
|
||||
|
||||
const { masterGain } = setup.mock;
|
||||
expect(masterGain.disconnect).toHaveBeenCalledWith(busOut);
|
||||
// Wired into the chain's input, not straight to the bus output.
|
||||
expect(masterGain.connect).toHaveBeenCalledTimes(1);
|
||||
expect(masterGain.connect).not.toHaveBeenCalledWith(busOut);
|
||||
});
|
||||
|
||||
it("feeds grouped members through the same master bus", async () => {
|
||||
document.body.innerHTML = `<div id="root" data-composition-id="main" data-fx-chain='${GAIN_CHAIN}'></div>`;
|
||||
const setup = setupGroupTransport();
|
||||
const busOut = withBusOut(setup);
|
||||
|
||||
await scheduleGrouped(setup.transport, setup.gen, "a", "vo");
|
||||
|
||||
expect(setup.mock.masterGain.disconnect).toHaveBeenCalledWith(busOut);
|
||||
expect(setup.mock.masterGain.connect).not.toHaveBeenCalledWith(busOut);
|
||||
});
|
||||
|
||||
it("limits a plain video at unity played from its media element", async () => {
|
||||
document.body.innerHTML = `<div id="root" data-composition-id="main" data-fx-chain='${GAIN_CHAIN}'><video id="talk" data-has-audio="true"></video></div>`;
|
||||
const setup = setupGroupTransport();
|
||||
const busOut = withBusOut(setup);
|
||||
const video = document.getElementById("talk") as unknown as HTMLMediaElement;
|
||||
|
||||
await setup.transport.scheduleMediaElementPlayback(video, 0, 0, 0, 1, setup.gen, 1);
|
||||
|
||||
expect(setup.mock.gainNodes[0]!.connect).toHaveBeenCalledWith(setup.mock.masterGain);
|
||||
expect(setup.mock.masterGain.disconnect).toHaveBeenCalledWith(busOut);
|
||||
expect(setup.mock.masterGain.connect).not.toHaveBeenCalledWith(busOut);
|
||||
});
|
||||
|
||||
it("leaves the sum wired straight through when the root has no chain", async () => {
|
||||
document.body.innerHTML = `<div id="root" data-composition-id="main"></div>`;
|
||||
const setup = setupGroupTransport();
|
||||
const busOut = withBusOut(setup);
|
||||
|
||||
await scheduleGrouped(setup.transport, setup.gen, "a");
|
||||
|
||||
expect(setup.mock.masterGain.connect).toHaveBeenLastCalledWith(busOut);
|
||||
});
|
||||
|
||||
it("ignores a chain on a sub-composition's root", async () => {
|
||||
document.body.innerHTML = `<div id="root" data-composition-id="main"><div data-composition-id="inner" data-fx-chain='${GAIN_CHAIN}'></div></div>`;
|
||||
const setup = setupGroupTransport();
|
||||
const busOut = withBusOut(setup);
|
||||
|
||||
await scheduleGrouped(setup.transport, setup.gen, "a");
|
||||
|
||||
expect(setup.mock.masterGain.connect).toHaveBeenLastCalledWith(busOut);
|
||||
});
|
||||
|
||||
it("is built once for the session, not once per member", async () => {
|
||||
document.body.innerHTML = `<div id="root" data-composition-id="main" data-fx-chain='${GAIN_CHAIN}'></div>`;
|
||||
const setup = setupGroupTransport();
|
||||
withBusOut(setup);
|
||||
|
||||
await scheduleGrouped(setup.transport, setup.gen, "a");
|
||||
const nodesAfterFirst = setup.mock.gainNodes.length;
|
||||
await scheduleGrouped(setup.transport, setup.gen, "b");
|
||||
|
||||
// The second member adds its own element gain and nothing else.
|
||||
expect(setup.mock.gainNodes.length).toBe(nodesAfterFirst + 1);
|
||||
expect(setup.mock.masterGain.disconnect).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("group mute (B5)", () => {
|
||||
it("a group created with data-hidden already set starts muted (mute gain at 0)", async () => {
|
||||
document.body.innerHTML = `<hf-audio-group id="vo" data-hidden></hf-audio-group>`;
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import { attachElementFxChain, readElementAutomation, type ElementFxHandle } from "./audioFx.js";
|
||||
import {
|
||||
attachElementFxChain,
|
||||
readElementAutomation,
|
||||
type ElementFxHandle,
|
||||
type FxAttributes,
|
||||
} from "./audioFx.js";
|
||||
import {
|
||||
clearParamLane,
|
||||
scheduleParamLane,
|
||||
@@ -6,7 +11,12 @@ import {
|
||||
type AutomationTiming,
|
||||
} from "../audio/audioFxAutomation.js";
|
||||
import { VOLUME_RANGE } from "../audioAutomation.js";
|
||||
import { audioGroupOf, readAudioGroupVolume, resolveGroupElement } from "../audioGroups.js";
|
||||
import {
|
||||
audioGroupOf,
|
||||
readAudioGroupVolume,
|
||||
resolveGroupElement,
|
||||
resolveMasterBusElement,
|
||||
} from "../audioGroups.js";
|
||||
import { swallow } from "./diagnostics";
|
||||
import { createLevelTap, type LevelTap, type StereoLevel } from "./levelTap.js";
|
||||
import { clampAudioGain } from "../audioGain.js";
|
||||
@@ -14,6 +24,8 @@ import { getDebugSurface } from "./globals.js";
|
||||
import { readElementPlaybackRate } from "./media.js";
|
||||
import { classifyWebAudioMediaRoute, reportWebAudioMediaRoute } from "./webAudioRoute.js";
|
||||
|
||||
const MASTER_FX_ATTRS: FxAttributes = { automation: null };
|
||||
|
||||
function normalizeRate(rate: number): number {
|
||||
if (!Number.isFinite(rate) || rate <= 0) return 1;
|
||||
return rate;
|
||||
@@ -139,6 +151,10 @@ export class WebAudioTransport {
|
||||
private _mediaElementSources = new WeakMap<HTMLMediaElement, MediaElementAudioSourceNode>();
|
||||
private _activeSources: ScheduledSource[] = [];
|
||||
private _masterGain: GainNode | null = null;
|
||||
/** Where the master bus chain lands; the program meter taps here. */
|
||||
private _busOut: GainNode | null = null;
|
||||
/** The chain on the composition root, built on first schedule and kept for the session. */
|
||||
private _master: { fx: ElementFxHandle | null; generation: number } | null = null;
|
||||
/** Preview volume and mute. Downstream of `_masterGain`; meters tap the program. */
|
||||
private _monitorGain: GainNode | null = null;
|
||||
private _masterVolume = 1;
|
||||
@@ -190,8 +206,10 @@ export class WebAudioTransport {
|
||||
this._ctx = new AudioContext();
|
||||
this._ctx.onstatechange = () => this.rest();
|
||||
this._masterGain = this._ctx.createGain();
|
||||
this._busOut = this._ctx.createGain();
|
||||
this._monitorGain = this._ctx.createGain();
|
||||
this._masterGain.connect(this._monitorGain);
|
||||
this._masterGain.connect(this._busOut);
|
||||
this._busOut.connect(this._monitorGain);
|
||||
this._monitorGain.connect(this._ctx.destination);
|
||||
this.applyMasterGain();
|
||||
if (this._metering) this.attachMasterTap();
|
||||
@@ -573,7 +591,7 @@ export class WebAudioTransport {
|
||||
|
||||
private attachMasterTap(): void {
|
||||
if (this._ctx && this._masterGain && !this._masterTap) {
|
||||
this._masterTap = createLevelTap(this._ctx, this._masterGain);
|
||||
this._masterTap = createLevelTap(this._ctx, this._busOut ?? this._masterGain);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -601,6 +619,36 @@ export class WebAudioTransport {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The master bus chain, between the sum of every clip and group and the monitor.
|
||||
* A seek re-anchors it once per play generation; no root yet is retried next schedule.
|
||||
*/
|
||||
private ensureMasterBus(doc: Document, timing: AutomationTiming): void {
|
||||
if (!this._ctx || !this._masterGain || !this._busOut) return;
|
||||
if (this._master) {
|
||||
if (this._master.generation === this._playGeneration) return;
|
||||
try {
|
||||
this._master.fx?.reanchor(timing);
|
||||
this._master.generation = this._playGeneration;
|
||||
} catch (err) {
|
||||
swallow("webAudioTransport.masterReanchor", err);
|
||||
}
|
||||
return;
|
||||
}
|
||||
const root = resolveMasterBusElement(doc);
|
||||
if (!root) return;
|
||||
this._masterGain.disconnect(this._busOut);
|
||||
const fx = attachElementFxChain(
|
||||
this._ctx,
|
||||
root,
|
||||
this._masterGain,
|
||||
this._busOut,
|
||||
timing,
|
||||
MASTER_FX_ATTRS,
|
||||
);
|
||||
this._master = { fx, generation: this._playGeneration };
|
||||
}
|
||||
|
||||
/** Master, unless `el` belongs to a group — then that group's bus (built on
|
||||
* first use, per `groupInput`). */
|
||||
private resolveDestination(
|
||||
@@ -610,9 +658,10 @@ export class WebAudioTransport {
|
||||
safeRate: number,
|
||||
): GainNode | null {
|
||||
if (!this._masterGain) return null;
|
||||
const groupTiming: AutomationTiming = { scheduledAt, elapsed: compositionTime, rate: safeRate };
|
||||
this.ensureMasterBus(el.ownerDocument, groupTiming);
|
||||
const groupId = audioGroupOf(el);
|
||||
if (!groupId) return this._masterGain;
|
||||
const groupTiming: AutomationTiming = { scheduledAt, elapsed: compositionTime, rate: safeRate };
|
||||
return this.groupInput(groupId, el.ownerDocument, groupTiming) ?? this._masterGain;
|
||||
}
|
||||
|
||||
@@ -785,17 +834,22 @@ export class WebAudioTransport {
|
||||
swallow("webAudioTransport.setRate", err);
|
||||
}
|
||||
}
|
||||
// Group buses are not in `_activeSources` — they outlive it — so their FX
|
||||
// automation needs re-aiming here too, or a rate change leaves a group's
|
||||
// envelopes running the old plan over audio at the new speed.
|
||||
for (const group of this._groups.values()) {
|
||||
this.setBusRate(safeRate);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Group buses and the master bus are not in `_activeSources` — they outlive it — so
|
||||
// their FX automation needs re-aiming too, or a rate change leaves their envelopes
|
||||
// running the old plan over audio at the new speed.
|
||||
private setBusRate(rate: number): void {
|
||||
const buses = [...this._groups.values(), ...(this._master ? [this._master] : [])];
|
||||
for (const bus of buses) {
|
||||
try {
|
||||
group.fx?.setRate(safeRate);
|
||||
bus.fx?.setRate(rate);
|
||||
} catch (err) {
|
||||
swallow("webAudioTransport.setRate.group", err);
|
||||
swallow("webAudioTransport.setRate.bus", err);
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// A bounded source's wall-clock duration was baked into start()'s duration
|
||||
@@ -887,6 +941,8 @@ export class WebAudioTransport {
|
||||
this.stopMetering();
|
||||
for (const group of this._groups.values()) group.dispose();
|
||||
this._groups.clear();
|
||||
this._master?.fx?.dispose();
|
||||
this._master = null;
|
||||
this._bufferCache.clear();
|
||||
this._failedSrcs.clear();
|
||||
this._mediaElementSources = new WeakMap();
|
||||
@@ -903,6 +959,7 @@ export class WebAudioTransport {
|
||||
}
|
||||
this._ctx = null;
|
||||
this._masterGain = null;
|
||||
this._busOut = null;
|
||||
this._monitorGain = null;
|
||||
this._masterVolume = 1;
|
||||
this._masterMuted = false;
|
||||
|
||||
@@ -256,6 +256,7 @@ export { createVideoFrameInjector } from "./services/videoFrameInjector.js";
|
||||
export {
|
||||
MIXED_AUDIO_FILENAME,
|
||||
parseAudioElements,
|
||||
parseMasterFxChain,
|
||||
processCompositionAudio,
|
||||
} from "./services/audioMixer.js";
|
||||
export { cloneCaptureWarning, cloneCaptureWarnings } from "./services/captureWarning.js";
|
||||
|
||||
@@ -6,6 +6,12 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { applyVolumeEnvelopeToWav } from "./audioVolumeEnvelope.js";
|
||||
import { defaultAudioFxParams, type HfAudioFxChain } from "@hyperframes/core/audio-fx";
|
||||
import { applyAudioFxChain, AudioFxRenderError, readWav, writeWav } from "./audioFxRender.js";
|
||||
import {
|
||||
MIXED_AUDIO_FILENAME,
|
||||
parseAudioElements,
|
||||
parseMasterFxChain,
|
||||
processCompositionAudio,
|
||||
} from "./audioMixer.js";
|
||||
import { resolveHeadlessShellPath } from "./browserManager.js";
|
||||
import { getFfmpegBinary } from "../utils/ffmpegBinaries.js";
|
||||
|
||||
@@ -530,3 +536,197 @@ describe("an empty track", () => {
|
||||
expect(existsSync(output)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
/** Highest sample level of a file in dBFS, as ffmpeg's volumedetect reports it. */
|
||||
function maxVolumeDb(path: string): number {
|
||||
const result = spawnSync(
|
||||
getFfmpegBinary(),
|
||||
["-nostdin", "-hide_banner", "-i", path, "-af", "volumedetect", "-f", "null", "-"],
|
||||
{ encoding: "utf-8" },
|
||||
);
|
||||
const match = result.stderr.match(/max_volume:\s*(-?[\d.]+) dB/);
|
||||
if (!match?.[1]) throw new Error(`Could not measure ${path}: ${result.stderr}`);
|
||||
return Number(match[1]);
|
||||
}
|
||||
|
||||
const chainJson = (...nodes: Record<string, unknown>[]): string =>
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
nodes: nodes.map((node, i) => ({ id: `n${i + 1}`, enabled: true, ...node })),
|
||||
});
|
||||
|
||||
describe("master bus: reading it", () => {
|
||||
const html = (root: string, inner = ""): string =>
|
||||
`<div ${root}>${inner}<audio id="a" src="a.wav" data-start="0" data-duration="1"></audio></div>`;
|
||||
const CHAIN = '{"version":1,"nodes":[]}';
|
||||
const attr = `data-fx-chain='${CHAIN}'`;
|
||||
|
||||
it("reads the root's chain as the master, not as a clip's own", () => {
|
||||
const doc = html(`data-composition-id="main" ${attr}`);
|
||||
expect(parseMasterFxChain(doc)).toBe(CHAIN);
|
||||
const [a] = parseAudioElements(doc);
|
||||
expect(a?.fxChain).toBeUndefined();
|
||||
expect(a?.groupFxChain).toBeUndefined();
|
||||
});
|
||||
|
||||
it("has none when the root carries none", () => {
|
||||
expect(parseMasterFxChain(html(`data-composition-id="main"`))).toBeUndefined();
|
||||
});
|
||||
|
||||
it("leaves a sub-composition root's chain as it was: not the master, not its clips' chain", () => {
|
||||
const inner = `<div data-composition-id="inner" ${attr}><audio id="b" src="b.wav" data-start="0" data-duration="1"></audio></div>`;
|
||||
const doc = html(`data-composition-id="main"`, inner);
|
||||
expect(parseMasterFxChain(doc)).toBeUndefined();
|
||||
const [a, b] = parseAudioElements(doc);
|
||||
for (const el of [a, b]) {
|
||||
expect(el?.fxChain).toBeUndefined();
|
||||
expect(el?.groupFxChain).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it("takes the master from the top-level root when a sub-composition root carries one too", () => {
|
||||
const MASTER = '{"version":1,"nodes":[{"type":"gain","id":"m","params":{"gain":-3}}]}';
|
||||
const inner = `<div data-composition-id="inner" ${attr}></div>`;
|
||||
expect(
|
||||
parseMasterFxChain(html(`data-composition-id="main" data-fx-chain='${MASTER}'`, inner)),
|
||||
).toBe(MASTER);
|
||||
});
|
||||
});
|
||||
|
||||
describe.skipIf(!HAS_FFMPEG)("master bus: reporting an unreadable chain", () => {
|
||||
it("fails the mix with a failure instead of throwing", async () => {
|
||||
writeWav(join(dir, "a.wav"), new Float32Array(SR).fill(0.1), SR);
|
||||
const result = await processCompositionAudio(
|
||||
[
|
||||
{
|
||||
id: "a",
|
||||
src: "a.wav",
|
||||
start: 0,
|
||||
end: 1,
|
||||
mediaStart: 0,
|
||||
layer: 0,
|
||||
volume: 1,
|
||||
type: "audio",
|
||||
},
|
||||
],
|
||||
dir,
|
||||
join(dir, "work"),
|
||||
join(dir, `bad-${MIXED_AUDIO_FILENAME}`),
|
||||
1,
|
||||
undefined,
|
||||
{ masterFxChain: "{not json" },
|
||||
);
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.failures?.[0]?.elementId).toBe("master");
|
||||
});
|
||||
});
|
||||
|
||||
describe.skipIf(!HAS_BROWSER || !HAS_FFMPEG)("master bus: rendering it", () => {
|
||||
const element = (id: string, extra: Record<string, unknown> = {}) => ({
|
||||
id,
|
||||
src: `${id}.wav`,
|
||||
start: 0,
|
||||
end: 1,
|
||||
mediaStart: 0,
|
||||
layer: 0,
|
||||
volume: 1,
|
||||
type: "audio" as const,
|
||||
...extra,
|
||||
});
|
||||
const steady = (peak: number, freq: number): Float32Array =>
|
||||
Float32Array.from({ length: SR }, (_, i) => peak * Math.sin((2 * Math.PI * freq * i) / SR));
|
||||
|
||||
it("limits the sum of everything that feeds it", async () => {
|
||||
// Two tones at -1 dBFS sum to about +5 dBFS: no clip's own chain can see that.
|
||||
writeWav(join(dir, "a.wav"), steady(0.9, 300), SR, 1, true);
|
||||
writeWav(join(dir, "b.wav"), steady(0.9, 300), SR, 1, true);
|
||||
const master = chainJson({
|
||||
type: "truepeak",
|
||||
params: { ceiling: -6, lookahead: 3, release: 80 },
|
||||
});
|
||||
const out = join(dir, `limited-${MIXED_AUDIO_FILENAME}`);
|
||||
const flat = join(dir, `flat-${MIXED_AUDIO_FILENAME}`);
|
||||
const mix = (path: string, masterFxChain?: string) =>
|
||||
processCompositionAudio(
|
||||
[element("a"), element("b")],
|
||||
dir,
|
||||
join(dir, "work"),
|
||||
path,
|
||||
1,
|
||||
undefined,
|
||||
{ masterFxChain },
|
||||
);
|
||||
expect((await mix(flat)).success).toBe(true);
|
||||
expect((await mix(out, master)).success).toBe(true);
|
||||
// The unprocessed sum has nowhere to go but full scale.
|
||||
expect(maxVolumeDb(flat)).toBeGreaterThan(-1);
|
||||
expect(maxVolumeDb(out)).toBeLessThanOrEqual(-5.5);
|
||||
}, 180_000);
|
||||
|
||||
/** Mix `elements` for one second and read the file's highest level in dBFS. */
|
||||
const mixedPeakDb = async (
|
||||
name: string,
|
||||
elements: Parameters<typeof processCompositionAudio>[0],
|
||||
masterFxChain?: string,
|
||||
): Promise<number> => {
|
||||
const out = join(dir, `${name}-${MIXED_AUDIO_FILENAME}`);
|
||||
const result = await processCompositionAudio(
|
||||
elements,
|
||||
dir,
|
||||
join(dir, "work"),
|
||||
out,
|
||||
1,
|
||||
undefined,
|
||||
{ masterFxChain },
|
||||
);
|
||||
expect(result.success).toBe(true);
|
||||
return maxVolumeDb(out);
|
||||
};
|
||||
const writeTone = (id: string, peak: number): void =>
|
||||
writeWav(join(dir, `${id}.wav`), steady(peak, 300), SR, 1, true);
|
||||
|
||||
it("applies the root's data-fx-chain once over the sum, from authored HTML", async () => {
|
||||
// Two in-phase 0.3 tones sum to -4.4 dBFS. A -6 dBTP limiter then a -6 dB gain
|
||||
// land that at -12. Per clip, neither clip reaches the ceiling and the result is
|
||||
// -10.4; applied twice it is -18. A linear chain alone could not tell these apart.
|
||||
writeTone("a", 0.3);
|
||||
writeTone("b", 0.3);
|
||||
const master = chainJson(
|
||||
{ type: "truepeak", params: { ceiling: -6, lookahead: 3, release: 80 } },
|
||||
{ type: "gain", params: { gain: -6 } },
|
||||
);
|
||||
const clip = (id: string) =>
|
||||
`<audio id="${id}" src="${id}.wav" data-start="0" data-end="1" data-duration="1"></audio>`;
|
||||
const html = `<div data-composition-id="main" data-fx-chain='${master}'>${clip("a")}${clip("b")}</div>`;
|
||||
const elements = parseAudioElements(html);
|
||||
expect(elements).toHaveLength(2);
|
||||
expect(await mixedPeakDb("once", elements, parseMasterFxChain(html))).toBeCloseTo(-12, 0);
|
||||
}, 180_000);
|
||||
|
||||
it("keeps the sum in float until the chain has read it", async () => {
|
||||
// The sum is +5 dBFS. A -6 dB master gain lands it at -1 if the sum is float; a sum
|
||||
// clipped to 16 bits at full scale first would land at -6.
|
||||
writeTone("a", 0.9);
|
||||
writeTone("b", 0.9);
|
||||
const masterFxChain = chainJson({ type: "gain", params: { gain: -6 } });
|
||||
const peak = await mixedPeakDb("float", [element("a"), element("b")], masterFxChain);
|
||||
expect(peak).toBeGreaterThan(-2);
|
||||
}, 180_000);
|
||||
|
||||
it("runs after each group's own chain and fader", async () => {
|
||||
// The group limits its 0.5 tone to -20 dBFS, then its fader takes 6 dB off: -26. A
|
||||
// +12 dB master gain after both lands at -14. A fader ahead of the limiter would
|
||||
// leave -20 and end at -8; a master ahead of the group would be limited away at -26.
|
||||
writeTone("g", 0.5);
|
||||
const group = {
|
||||
groupId: "bus",
|
||||
groupFxChain: chainJson({
|
||||
type: "truepeak",
|
||||
params: { ceiling: -20, lookahead: 3, release: 80 },
|
||||
}),
|
||||
groupVolume: 0.5,
|
||||
};
|
||||
const masterFxChain = chainJson({ type: "gain", params: { gain: 12 } });
|
||||
expect(await mixedPeakDb("ordered", [element("g", group)], masterFxChain)).toBeCloseTo(-14, 0);
|
||||
}, 180_000);
|
||||
});
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// fallow-ignore-file code-duplication
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
@@ -66,6 +66,7 @@ vi.mock("../utils/ffprobe.js", async (importOriginal) => {
|
||||
});
|
||||
|
||||
import { RATE_RANGE } from "@hyperframes/core/audio-automation";
|
||||
import { AudioFxRenderError } from "./audioFxRender.js";
|
||||
import { parseAudioElements, processCompositionAudio } from "./audioMixer.js";
|
||||
|
||||
describe("parseAudioElements strict literal timing", () => {
|
||||
@@ -105,6 +106,10 @@ describe("parseAudioElements — <source> children", () => {
|
||||
});
|
||||
describe("processCompositionAudio", () => {
|
||||
const tempDirs: string[] = [];
|
||||
const gainChain = JSON.stringify({
|
||||
version: 1,
|
||||
nodes: [{ type: "gain", id: "g", params: { gain: -3 } }],
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
@@ -580,6 +585,42 @@ describe("processCompositionAudio", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
["the master pass", {}, { masterFxChain: gainChain }],
|
||||
["a group bus", { groupId: "bus", groupFxChain: gainChain }, {}],
|
||||
])("removes its work directory when %s fails fatally", async (_label, extra, config) => {
|
||||
const baseDir = mkdtempSync(join(tmpdir(), "hf-audio-base-"));
|
||||
const workDir = mkdtempSync(join(tmpdir(), "hf-audio-work-"));
|
||||
tempDirs.push(baseDir, workDir);
|
||||
writeFileSync(join(baseDir, "bed.wav"), "stub");
|
||||
applyAudioFxChainMock.mockRejectedValueOnce(new AudioFxRenderError("browser would not start"));
|
||||
|
||||
await expect(
|
||||
processCompositionAudio(
|
||||
[
|
||||
{
|
||||
id: "bed",
|
||||
src: "bed.wav",
|
||||
start: 0,
|
||||
end: 2,
|
||||
mediaStart: 0,
|
||||
layer: 0,
|
||||
volume: 1,
|
||||
type: "audio",
|
||||
...extra,
|
||||
},
|
||||
],
|
||||
baseDir,
|
||||
workDir,
|
||||
join(baseDir, "out.m4a"),
|
||||
2,
|
||||
undefined,
|
||||
config,
|
||||
),
|
||||
).rejects.toThrow(AudioFxRenderError);
|
||||
expect(existsSync(workDir)).toBe(false);
|
||||
});
|
||||
|
||||
it("lets an FX tail run past the clip, still bounded by the composition", async () => {
|
||||
// A reverb is still decaying when the clip's own audio stops. Trimming at
|
||||
// the clip boundary is what cut every tail short in the render.
|
||||
|
||||
@@ -27,7 +27,11 @@ import type {
|
||||
MixResult,
|
||||
} from "./audioMixer.types.js";
|
||||
import { applyVolumeEnvelopeToWav } from "./audioVolumeEnvelope.js";
|
||||
import { HF_AUDIO_FX_ATTR, parseAudioFxChain } from "@hyperframes/core/audio-fx";
|
||||
import {
|
||||
HF_AUDIO_FX_ATTR,
|
||||
parseAudioFxChain,
|
||||
type HfAudioFxChain,
|
||||
} from "@hyperframes/core/audio-fx";
|
||||
import {
|
||||
HF_AUDIO_AUTOMATION_ATTR,
|
||||
parseAutomation,
|
||||
@@ -49,7 +53,7 @@ import {
|
||||
timeAtSourceTime,
|
||||
type RateSpec,
|
||||
} from "@hyperframes/core";
|
||||
import { resolveAudioGroups } from "@hyperframes/core/audio-groups";
|
||||
import { readMasterFxChain, resolveAudioGroups } from "@hyperframes/core/audio-groups";
|
||||
import { applyAudioFxChain, AudioFxRenderError } from "./audioFxRender.js";
|
||||
import type { AudioVolumeKeyframe } from "./audioMixer.types.js";
|
||||
|
||||
@@ -518,6 +522,15 @@ export function volumeLaneKeyframes(
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* The top-level root's `data-fx-chain`. It belongs to the composition, not to a clip,
|
||||
* so `processCompositionAudio` takes it beside the tracks: a track the render adds
|
||||
* itself has no parsed element to carry it.
|
||||
*/
|
||||
export function parseMasterFxChain(html: string): string | undefined {
|
||||
return readMasterFxChain(parseHTML(unwrapTemplate(html)).document);
|
||||
}
|
||||
|
||||
export function parseAudioElements(html: string): AudioElement[] {
|
||||
const elements: AudioElement[] = [];
|
||||
const { document } = parseHTML(unwrapTemplate(html));
|
||||
@@ -744,6 +757,7 @@ async function mixAudioTracks(
|
||||
totalDuration: number,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout" | "audioGain">>,
|
||||
floatWav = false,
|
||||
): Promise<MixResult> {
|
||||
const ffmpegProcessTimeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;
|
||||
const masterOutputGain = config?.audioGain ?? DEFAULT_CONFIG.audioGain;
|
||||
@@ -807,10 +821,7 @@ async function mixAudioTracks(
|
||||
scriptPath,
|
||||
"-map",
|
||||
"[out]",
|
||||
"-acodec",
|
||||
"aac",
|
||||
"-b:a",
|
||||
"192k",
|
||||
...(floatWav ? ["-acodec", "pcm_f32le", "-ar", "48000"] : ["-acodec", "aac", "-b:a", "192k"]),
|
||||
"-t",
|
||||
String(totalDuration),
|
||||
"-y",
|
||||
@@ -1027,6 +1038,56 @@ async function mixGroupMembers(
|
||||
return { success: true, degradedAutomation };
|
||||
}
|
||||
|
||||
/**
|
||||
* Sum the tracks to a float WAV, run the master chain over the sum, then AAC-encode.
|
||||
* Groups are already folded into `tracks`, and the sum stays float until the encoder.
|
||||
*/
|
||||
async function mixThroughMasterBus(
|
||||
tracks: AudioTrack[],
|
||||
masterChain: HfAudioFxChain,
|
||||
outputPath: string,
|
||||
workDir: string,
|
||||
totalDuration: number,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout" | "audioGain">>,
|
||||
): Promise<MixResult> {
|
||||
const sumPath = join(workDir, "master-sum.wav");
|
||||
const sum = await mixAudioTracks(tracks, sumPath, totalDuration, signal, config, true);
|
||||
if (!sum.success) return { ...sum, outputPath };
|
||||
|
||||
const fx = await applyAudioFxChain(sumPath, masterChain, join(workDir, "master-fx.wav"), {
|
||||
trackId: "master",
|
||||
signal,
|
||||
});
|
||||
const encoded = await runFfmpeg(
|
||||
[
|
||||
"-i",
|
||||
fx.path,
|
||||
"-t",
|
||||
String(totalDuration),
|
||||
"-acodec",
|
||||
"aac",
|
||||
"-b:a",
|
||||
"192k",
|
||||
"-y",
|
||||
outputPath,
|
||||
],
|
||||
{ signal, timeout: config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout },
|
||||
);
|
||||
if (!encoded.success) {
|
||||
const failure = ffmpegFailure("mix", encoded);
|
||||
return {
|
||||
success: false,
|
||||
outputPath,
|
||||
durationMs: sum.durationMs + encoded.durationMs,
|
||||
tracksProcessed: 0,
|
||||
error: failure.detail,
|
||||
failures: [failure],
|
||||
};
|
||||
}
|
||||
return { ...sum, outputPath, durationMs: sum.durationMs + encoded.durationMs };
|
||||
}
|
||||
|
||||
export async function processCompositionAudio(
|
||||
elements: AudioElement[],
|
||||
baseDir: string,
|
||||
@@ -1034,7 +1095,10 @@ export async function processCompositionAudio(
|
||||
outputPath: string,
|
||||
totalDuration: number,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout" | "audioGain">>,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout" | "audioGain">> & {
|
||||
/** Serialised master bus chain, from `parseMasterFxChain`. */
|
||||
masterFxChain?: string;
|
||||
},
|
||||
compiledDir?: string,
|
||||
): Promise<MixResult> {
|
||||
const startMs = Date.now();
|
||||
@@ -1356,12 +1420,15 @@ export async function processCompositionAudio(
|
||||
// The producer only surfaces audio failures when `success` is false; mixing
|
||||
// the remaining tracks made the omitted cue indistinguishable from a valid
|
||||
// render unless someone manually audited that exact audio window.
|
||||
const bail = (): MixResult => {
|
||||
const removeWorkDir = (): void => {
|
||||
try {
|
||||
rmSync(workDir, { recursive: true, force: true });
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
};
|
||||
const bail = (): MixResult => {
|
||||
removeWorkDir();
|
||||
return {
|
||||
success: false,
|
||||
outputPath,
|
||||
@@ -1375,6 +1442,28 @@ export async function processCompositionAudio(
|
||||
};
|
||||
if (failures.length > 0) return bail();
|
||||
|
||||
// Parsed before any group is mixed, so a malformed master chain fails fast and
|
||||
// as a failure, the way an unreadable group chain does.
|
||||
let masterChain: HfAudioFxChain | undefined;
|
||||
const masterFxJson = config?.masterFxChain;
|
||||
if (masterFxJson) {
|
||||
try {
|
||||
masterChain = parseAudioFxChain(masterFxJson);
|
||||
} catch (err: unknown) {
|
||||
failures.push({
|
||||
stage: "mix",
|
||||
reason: "internal",
|
||||
owner: "user",
|
||||
retryable: false,
|
||||
elementId: "master",
|
||||
detail: boundedDetail(
|
||||
`Master bus chain is unreadable: ${err instanceof Error ? err.message : String(err)}`,
|
||||
),
|
||||
});
|
||||
return bail();
|
||||
}
|
||||
}
|
||||
|
||||
// Sub-mix each group into one processed bus (design doc §1.3), then fold it
|
||||
// into the flat track list as a single AudioTrack — everything downstream
|
||||
// (the final mixAudioTracks call) never has to know groups exist. Group
|
||||
@@ -1479,7 +1568,10 @@ export async function processCompositionAudio(
|
||||
...(bakedEnvelope || !laneKeyframes?.length ? {} : { volumeKeyframes: laneKeyframes }),
|
||||
});
|
||||
} catch (err: unknown) {
|
||||
if (err instanceof AudioFxRenderError) throw err;
|
||||
if (err instanceof AudioFxRenderError) {
|
||||
removeWorkDir();
|
||||
throw err;
|
||||
}
|
||||
failures.push({
|
||||
stage: "mix",
|
||||
reason: "internal",
|
||||
@@ -1496,12 +1588,22 @@ export async function processCompositionAudio(
|
||||
}
|
||||
if (failures.length > 0) return bail();
|
||||
|
||||
const mixResult = await mixAudioTracks(tracks, outputPath, totalDuration, signal, config);
|
||||
|
||||
let mixResult: MixResult;
|
||||
try {
|
||||
rmSync(workDir, { recursive: true, force: true });
|
||||
} catch {
|
||||
/* ignore */
|
||||
mixResult =
|
||||
masterChain && tracks.length > 0
|
||||
? await mixThroughMasterBus(
|
||||
tracks,
|
||||
masterChain,
|
||||
outputPath,
|
||||
workDir,
|
||||
totalDuration,
|
||||
signal,
|
||||
config,
|
||||
)
|
||||
: await mixAudioTracks(tracks, outputPath, totalDuration, signal, config);
|
||||
} finally {
|
||||
removeWorkDir();
|
||||
}
|
||||
|
||||
// A group whose sub-mix had to drop member automation reports it the same
|
||||
|
||||
@@ -18,6 +18,7 @@ const INTEGRATION_TEST_FILES = new Set([
|
||||
"src/services/distributed/assemble.test.ts",
|
||||
"src/services/distributed/chunkBoundary.test.ts",
|
||||
"src/services/distributed/crossWorkerIdempotency.test.ts",
|
||||
"src/services/distributed/plan.masterBus.test.ts",
|
||||
"src/services/distributed/plan.test.ts",
|
||||
"src/services/distributed/planSizeCap.test.ts",
|
||||
"src/services/distributed/renderChunk.test.ts",
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
import { afterEach, beforeEach, expect, it, vi } from "vitest";
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { plan } from "./plan.js";
|
||||
|
||||
const MASTER_CHAIN = '{"version":1,"nodes":[]}';
|
||||
const seen = vi.hoisted((): { audioInput?: { masterFxChain?: string } } => ({}));
|
||||
|
||||
vi.mock("../render/stages/audioStage.js", () => ({
|
||||
runAudioStage: async (input: { masterFxChain?: string; workDir: string }) => {
|
||||
seen.audioInput = input;
|
||||
return {
|
||||
audioOutputPath: `${input.workDir}/audio.m4a`,
|
||||
hasAudio: false,
|
||||
audioProcessMs: 0,
|
||||
};
|
||||
},
|
||||
}));
|
||||
|
||||
let runRoot: string;
|
||||
|
||||
beforeEach(() => {
|
||||
runRoot = mkdtempSync(join(tmpdir(), "hf-plan-master-bus-"));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(runRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("plan() gives the audio stage the composition's master chain", async () => {
|
||||
const projectDir = join(runRoot, "project");
|
||||
mkdirSync(projectDir, { recursive: true });
|
||||
writeFileSync(
|
||||
join(projectDir, "index.html"),
|
||||
`<!doctype html>
|
||||
<html><head><meta charset="utf-8"></head><body>
|
||||
<div data-composition-id="root" data-width="320" data-height="240" data-duration="1" data-fx-chain='${MASTER_CHAIN}'>
|
||||
<p>master bus plan fixture</p>
|
||||
</div>
|
||||
</body></html>`,
|
||||
"utf-8",
|
||||
);
|
||||
const planDir = join(runRoot, "plan");
|
||||
mkdirSync(planDir, { recursive: true });
|
||||
|
||||
await plan(projectDir, { fps: 30, width: 320, height: 240, format: "mp4" }, planDir);
|
||||
|
||||
expect(seen.audioInput?.masterFxChain).toBe(MASTER_CHAIN);
|
||||
});
|
||||
@@ -1130,6 +1130,7 @@ export async function buildLocalExecutionPlan(
|
||||
ffmpegProcessTimeout: cfg.ffmpegProcessTimeout,
|
||||
audioGain: cfg.audioGain,
|
||||
audios: composition.audios,
|
||||
masterFxChain: composition.masterFxChain,
|
||||
abortSignal,
|
||||
assertNotAborted,
|
||||
});
|
||||
|
||||
@@ -744,6 +744,25 @@ describe("detectRenderModeHints", () => {
|
||||
expect(document.querySelector("#root")?.getAttribute("src")).toBe("root.png");
|
||||
});
|
||||
|
||||
it("reads the root's data-fx-chain as the composition's master chain, apart from any clip", async () => {
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-master-chain-"));
|
||||
const chain = '{"version":1,"nodes":[]}';
|
||||
writeFileSync(
|
||||
join(projectDir, "index.html"),
|
||||
`<!doctype html><html><body>
|
||||
<div data-composition-id="main" data-width="100" data-height="100" data-duration="1" data-fx-chain='${chain}'>
|
||||
<video id="talk" src="talk.mp4" data-start="0" data-duration="1"></video>
|
||||
</div>
|
||||
</body></html>`,
|
||||
);
|
||||
|
||||
const compiled = await compileForRender(projectDir, join(projectDir, "index.html"), projectDir);
|
||||
|
||||
expect(compiled.masterFxChain).toBe(chain);
|
||||
expect(compiled.audios.map((audio) => audio.id)).toEqual(["talk-audio"]);
|
||||
expect(compiled.audios[0]).not.toHaveProperty("fxChain");
|
||||
});
|
||||
|
||||
// Shared fixture builder for the assertSubCompositionsUsable / EmptyCompositionError
|
||||
// pre-flight tests below. `subCompFiles` is a map of compositions/-relative
|
||||
// filename to raw file content (empty string / malformed text / valid HTML).
|
||||
|
||||
@@ -79,6 +79,7 @@ import {
|
||||
analyzeKeyframeIntervals,
|
||||
assertMediaPayload,
|
||||
NotMediaPayloadError,
|
||||
parseMasterFxChain,
|
||||
probeMediaProfile,
|
||||
} from "@hyperframes/engine";
|
||||
import {
|
||||
@@ -110,6 +111,7 @@ export interface CompiledComposition {
|
||||
videos: VideoElement[];
|
||||
audios: AudioElement[];
|
||||
images: ImageElement[];
|
||||
masterFxChain?: string;
|
||||
unresolvedCompositions: UnresolvedElement[];
|
||||
/** Assets that resolve outside projectDir. Keys are the path used in HTML, values are absolute filesystem paths. */
|
||||
externalAssets: Map<string, string>;
|
||||
@@ -2146,6 +2148,7 @@ export async function compileForRender(
|
||||
// two bare `<video>`s, both auto-numbered `hf-video-0` — collapsed into one
|
||||
// entry and injected frames onto whichever element came first. See #3340.
|
||||
const { videos, audios, images } = collectRenderMedia(html);
|
||||
const masterFxChain = parseMasterFxChain(html);
|
||||
|
||||
// Advisory video checks (sparse keyframes, VFR). Fire-and-forget — these spawn
|
||||
// ffprobe subprocesses and should not block compilation since they only produce warnings.
|
||||
@@ -2198,6 +2201,7 @@ export async function compileForRender(
|
||||
videos,
|
||||
audios,
|
||||
images,
|
||||
...(masterFxChain ? { masterFxChain } : {}),
|
||||
unresolvedCompositions,
|
||||
externalAssets,
|
||||
width,
|
||||
|
||||
@@ -47,6 +47,7 @@ export interface CompositionMetadata {
|
||||
videos: VideoElement[];
|
||||
audios: AudioElement[];
|
||||
images: ImageElement[];
|
||||
masterFxChain?: string;
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import type { AudioElement } from "@hyperframes/engine";
|
||||
import { parseMasterFxChain, type AudioElement, type VideoElement } from "@hyperframes/engine";
|
||||
|
||||
const { processCompositionAudioMock } = vi.hoisted(() => ({
|
||||
processCompositionAudioMock: vi.fn(),
|
||||
@@ -14,6 +14,7 @@ vi.mock("@hyperframes/engine", async (importOriginal) => {
|
||||
});
|
||||
|
||||
import { runAudioStage } from "./audioStage.js";
|
||||
import { appendAutoDetectedVideoAudio } from "./extractVideosStage.js";
|
||||
import { EncoderInterruptedError } from "../encoderInterruption.js";
|
||||
|
||||
// Regression: hasAudio flipping to false used to be indistinguishable from
|
||||
@@ -87,6 +88,55 @@ describe("runAudioStage", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("hands the master chain to the mixer when the only track is the one the render added for a plain video", async () => {
|
||||
// A plain <video> has no audio entry at compile time; the render adds one after probing
|
||||
// its file. That track is not parsed from HTML, so the chain cannot ride on it.
|
||||
const chain = '{"version":1,"nodes":[]}';
|
||||
const html = `<div data-composition-id="main" data-fx-chain='${chain}'><video id="talk" src="talk.mp4" data-start="0" data-duration="5"></video></div>`;
|
||||
const video: VideoElement = {
|
||||
id: "talk",
|
||||
src: "talk.mp4",
|
||||
start: 0,
|
||||
end: 5,
|
||||
mediaStart: 0,
|
||||
loop: false,
|
||||
hasAudio: true,
|
||||
};
|
||||
const composition = {
|
||||
videos: [video],
|
||||
audios: [] as AudioElement[],
|
||||
masterFxChain: parseMasterFxChain(html),
|
||||
};
|
||||
appendAutoDetectedVideoAudio(composition, [
|
||||
{
|
||||
videoId: "talk",
|
||||
metadata: { hasAudio: true },
|
||||
} as Parameters<typeof appendAutoDetectedVideoAudio>[1][number],
|
||||
]);
|
||||
expect(composition.audios.map((audio) => audio.id)).toEqual(["talk-audio"]);
|
||||
processCompositionAudioMock.mockResolvedValue({
|
||||
success: true,
|
||||
outputPath: "audio.m4a",
|
||||
durationMs: 1,
|
||||
tracksProcessed: 1,
|
||||
});
|
||||
|
||||
await runAudioStage(
|
||||
makeInput({ audios: composition.audios, masterFxChain: composition.masterFxChain }),
|
||||
);
|
||||
|
||||
expect(processCompositionAudioMock).toHaveBeenCalledWith(
|
||||
composition.audios,
|
||||
expect.any(String),
|
||||
expect.any(String),
|
||||
expect.any(String),
|
||||
5,
|
||||
undefined,
|
||||
{ ffmpegProcessTimeout: 3_600_000, audioGain: 1, masterFxChain: chain },
|
||||
expect.any(String),
|
||||
);
|
||||
});
|
||||
|
||||
it("falls back to a generic message when the mixer fails without an error string", async () => {
|
||||
processCompositionAudioMock.mockResolvedValue({
|
||||
success: false,
|
||||
|
||||
@@ -40,6 +40,7 @@ export interface AudioStageInput {
|
||||
audioGain: number;
|
||||
/** Read-only view of `composition.audios`. */
|
||||
audios: CompositionMetadata["audios"];
|
||||
masterFxChain?: string;
|
||||
abortSignal: AbortSignal | undefined;
|
||||
assertNotAborted: () => void;
|
||||
/** Where a per-track failure's detail goes. Optional so tests need not pass one. */
|
||||
@@ -72,6 +73,7 @@ export async function runAudioStage(input: AudioStageInput): Promise<AudioStageR
|
||||
ffmpegProcessTimeout,
|
||||
audioGain,
|
||||
audios,
|
||||
masterFxChain,
|
||||
abortSignal,
|
||||
assertNotAborted,
|
||||
log,
|
||||
@@ -99,7 +101,7 @@ export async function runAudioStage(input: AudioStageInput): Promise<AudioStageR
|
||||
audioOutputPath,
|
||||
duration,
|
||||
abortSignal,
|
||||
{ ffmpegProcessTimeout, audioGain },
|
||||
{ ffmpegProcessTimeout, audioGain, masterFxChain },
|
||||
compiledDir,
|
||||
);
|
||||
} catch (err) {
|
||||
|
||||
@@ -242,6 +242,40 @@ describe("runCompileStage — forceScreenshot snapshot", () => {
|
||||
});
|
||||
});
|
||||
|
||||
const MASTER_CHAIN = '{"version":1,"nodes":[]}';
|
||||
const MASTER_BUS_HTML = `<!doctype html>
|
||||
<html>
|
||||
<head><meta charset="utf-8"></head>
|
||||
<body>
|
||||
<div data-composition-id="root" data-width="1920" data-height="1080" data-duration="1" data-fx-chain='${MASTER_CHAIN}'>
|
||||
<p>master bus composition</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>`;
|
||||
|
||||
describe("runCompileStage — master bus", () => {
|
||||
it("hands the root's chain to the composition metadata the audio stage reads", async () => {
|
||||
const fixture = setupFixture(MASTER_BUS_HTML);
|
||||
try {
|
||||
const result = await runCompileStage({
|
||||
projectDir: join(fixture.workDir, "project"),
|
||||
workDir: fixture.workDir,
|
||||
htmlPath: fixture.htmlPath,
|
||||
entryFile: "index.html",
|
||||
job: createJob(),
|
||||
cfg: createCfg(),
|
||||
needsAlpha: false,
|
||||
log: noopLog,
|
||||
assertNotAborted: () => {},
|
||||
});
|
||||
expect(result.compiled.masterFxChain).toBe(MASTER_CHAIN);
|
||||
expect(result.composition.masterFxChain).toBe(MASTER_CHAIN);
|
||||
} finally {
|
||||
fixture.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("runCompileStage — font fetch cancellation", () => {
|
||||
let fixture: CompileFixture | null = null;
|
||||
|
||||
|
||||
@@ -310,6 +310,7 @@ export async function runCompileStage(input: CompileStageInput): Promise<Compile
|
||||
videos: compiled.videos,
|
||||
audios: compiled.audios,
|
||||
images: compiled.images,
|
||||
masterFxChain: compiled.masterFxChain,
|
||||
width: compiled.width,
|
||||
height: compiled.height,
|
||||
};
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createRenderJob, executeRenderJob } from "./renderOrchestrator.js";
|
||||
|
||||
const MASTER_CHAIN = '{"version":1,"nodes":[]}';
|
||||
const seen = vi.hoisted((): { audioInput?: { masterFxChain?: string } } => ({}));
|
||||
|
||||
vi.mock("@hyperframes/engine", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@hyperframes/engine")>()),
|
||||
assertConfiguredFfmpegBinariesExist: () => {},
|
||||
}));
|
||||
vi.mock("./render/stages/compileStage.js", () => ({
|
||||
runCompileStage: async () => ({
|
||||
compiled: { html: "<div></div>" },
|
||||
composition: {
|
||||
width: 320,
|
||||
height: 180,
|
||||
duration: 1,
|
||||
videos: [],
|
||||
images: [],
|
||||
audios: [],
|
||||
masterFxChain: '{"version":1,"nodes":[]}',
|
||||
},
|
||||
deviceScaleFactor: 1,
|
||||
outputWidth: 320,
|
||||
outputHeight: 180,
|
||||
compileOnlyMs: 0,
|
||||
forceScreenshot: true,
|
||||
}),
|
||||
}));
|
||||
vi.mock("./render/stages/probeStage.js", () => ({
|
||||
runProbeStage: async ({ compiled }: { compiled: object }) => ({
|
||||
compiled,
|
||||
fileServer: null,
|
||||
probeSession: null,
|
||||
lastBrowserConsole: [],
|
||||
duration: 1,
|
||||
totalFrames: 30,
|
||||
browserProbeMs: 0,
|
||||
}),
|
||||
}));
|
||||
vi.mock("./render/stages/extractVideosStage.js", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("./render/stages/extractVideosStage.js")>()),
|
||||
runExtractVideosStage: async () => ({
|
||||
extractionResult: null,
|
||||
frameLookup: null,
|
||||
videoReadinessSkipIds: new Set<string>(),
|
||||
videoMetadataHints: new Map(),
|
||||
nativeHdrVideoIds: new Set<string>(),
|
||||
videoTransfers: new Map(),
|
||||
nativeHdrImageIds: new Set<string>(),
|
||||
imageTransfers: new Map(),
|
||||
hdrImageSrcPaths: new Map(),
|
||||
imageColorSpaces: [],
|
||||
videoExtractMs: 0,
|
||||
failureToEnforce: undefined,
|
||||
}),
|
||||
}));
|
||||
vi.mock("./render/stages/audioStage.js", () => ({
|
||||
runAudioStage: async (input: { masterFxChain?: string }) => {
|
||||
seen.audioInput = input;
|
||||
throw new Error("audio stage reached");
|
||||
},
|
||||
}));
|
||||
|
||||
const dirs: string[] = [];
|
||||
afterEach(() => {
|
||||
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("the local renderer gives the audio stage the composition's master chain", async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), "hf-master-bus-orchestrator-"));
|
||||
dirs.push(root);
|
||||
writeFileSync(join(root, "index.html"), "<div></div>");
|
||||
const job = createRenderJob({
|
||||
fps: 30,
|
||||
quality: "standard",
|
||||
hdrMode: "force-sdr",
|
||||
logger: { info() {}, warn() {}, error() {}, debug() {} },
|
||||
});
|
||||
await expect(executeRenderJob(job, root, join(root, "output.mp4"))).rejects.toThrow(
|
||||
"audio stage reached",
|
||||
);
|
||||
expect(seen.audioInput?.masterFxChain).toBe(MASTER_CHAIN);
|
||||
});
|
||||
@@ -3356,6 +3356,7 @@ async function executeRenderPipeline(input: {
|
||||
ffmpegProcessTimeout: cfg.ffmpegProcessTimeout,
|
||||
audioGain: cfg.audioGain,
|
||||
audios: composition.audios,
|
||||
masterFxChain: composition.masterFxChain,
|
||||
abortSignal: executionSignal,
|
||||
assertNotAborted,
|
||||
log,
|
||||
|
||||
@@ -632,6 +632,61 @@ describe("audio FX attributes on parsed elements", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("the master chain on parsed elements", () => {
|
||||
const CHAIN = '{"version":1,"nodes":[{"type":"truepeak","id":"m1","params":{}}]}';
|
||||
const parseVideo = (rootAttrs: string, inner = ""): TimelineElement | undefined => {
|
||||
const doc = new DOMParser().parseFromString(
|
||||
`<div data-composition-id="main" data-start="0" data-duration="10" ${rootAttrs}>
|
||||
<video id="talk" data-start="0" data-duration="10" data-has-audio="true"></video>${inner}
|
||||
</div>`,
|
||||
"text/html",
|
||||
);
|
||||
return parseTimelineFromDOM(doc, 10).find((e) => e.domId === "talk");
|
||||
};
|
||||
|
||||
it("copies the root's data-fx-chain onto an element that has none of its own", () => {
|
||||
const talk = parseVideo(`data-fx-chain='${CHAIN}'`);
|
||||
expect(talk?.masterChain).toBe(CHAIN);
|
||||
expect(talk?.fxChain).toBeUndefined();
|
||||
});
|
||||
|
||||
it("leaves it unset when the root carries none", () => {
|
||||
expect(parseVideo("")?.masterChain).toBeUndefined();
|
||||
});
|
||||
|
||||
it("ignores a sub-composition root's chain", () => {
|
||||
const inner = `<div data-composition-id="inner" data-fx-chain='${CHAIN}'></div>`;
|
||||
expect(parseVideo("", inner)?.masterChain).toBeUndefined();
|
||||
});
|
||||
|
||||
it("carries it on the manifest path too", () => {
|
||||
const doc = makeDoc(`
|
||||
<div data-composition-id="main" data-fx-chain='${CHAIN}'>
|
||||
<video id="talk" data-has-audio="true"></video>
|
||||
</div>
|
||||
`);
|
||||
const element = createTimelineElementFromManifestClip({
|
||||
clip: {
|
||||
id: "talk",
|
||||
label: "Talk",
|
||||
kind: "video",
|
||||
tagName: "video",
|
||||
start: 0,
|
||||
duration: 10,
|
||||
track: 0,
|
||||
compositionId: null,
|
||||
parentCompositionId: "main",
|
||||
compositionSrc: null,
|
||||
assetUrl: null,
|
||||
},
|
||||
fallbackIndex: 0,
|
||||
doc,
|
||||
hostEl: doc.getElementById("talk"),
|
||||
});
|
||||
expect(element.masterChain).toBe(CHAIN);
|
||||
});
|
||||
});
|
||||
|
||||
// A composition clip trimmed by 30 px collapsed to its first nested video's 1.77 s: that length capped the trim.
|
||||
describe("a composition clip's source length", () => {
|
||||
const scene = `
|
||||
|
||||
@@ -12,6 +12,7 @@ import type { TimelineElement } from "../store/playerStore";
|
||||
import type { ClipManifestClip, IframeWindow, TimelineLike } from "./playbackTypes";
|
||||
import { resolveCssStackingContextId } from "@hyperframes/core/runtime/stacking-context";
|
||||
import { readClipTiming } from "@hyperframes/core/composition-contract";
|
||||
import { readMasterFxChain } from "@hyperframes/core/audio-groups";
|
||||
import { linkScopeOf } from "@hyperframes/core/media-link";
|
||||
import { createRuntimeStartTimeResolver } from "@hyperframes/core/runtime/start-resolver";
|
||||
import { groupInfoFor } from "./timelineGroupInfo";
|
||||
@@ -167,6 +168,8 @@ export function createTimelineElementFromManifestClip(params: {
|
||||
if (info.fxChain) entry.audioGroupFxChain = info.fxChain;
|
||||
if (info.automation) entry.audioGroupAutomation = info.automation;
|
||||
}
|
||||
const masterChain = readMasterFxChain(doc ?? hostEl.ownerDocument);
|
||||
if (masterChain) entry.masterChain = masterChain;
|
||||
const fxChain = hostEl.getAttribute("data-fx-chain");
|
||||
if (fxChain) entry.fxChain = fxChain;
|
||||
const automation = hostEl.getAttribute("data-automation");
|
||||
@@ -330,6 +333,8 @@ export function parseTimelineFromDOM(
|
||||
// Read from the element, like the manifest path does: without these an audio
|
||||
// clip parsed straight from the DOM reserved no automation height and drew no
|
||||
// lanes, while the property panel still showed its chain.
|
||||
const domMasterChain = readMasterFxChain(doc);
|
||||
if (domMasterChain) entry.masterChain = domMasterChain;
|
||||
const domFxChain = el.getAttribute("data-fx-chain");
|
||||
if (domFxChain) entry.fxChain = domFxChain;
|
||||
const domAutomation = el.getAttribute("data-automation");
|
||||
|
||||
@@ -62,9 +62,10 @@ export interface TimelineElement {
|
||||
/** Clip-edge fades from `data-fade-in` / `data-fade-out`, seconds; absent means none. */
|
||||
fadeIn?: number;
|
||||
fadeOut?: number;
|
||||
/** Verbatim `data-fx-chain` / `data-automation`; see automationLaneData. */
|
||||
/** Verbatim `data-fx-chain` / `data-automation` (`masterChain`: the root's); see automationLaneData. */
|
||||
fxChain?: string;
|
||||
automation?: string;
|
||||
masterChain?: string;
|
||||
/** Path from data-composition-src — identifies sub-composition elements */
|
||||
compositionSrc?: string;
|
||||
/** Set by data-timeline-locked on the host element — disables move and trim in Studio. */
|
||||
|
||||
@@ -69,6 +69,15 @@ describe("hasProjectAudio", () => {
|
||||
expect(hasProjectAudio([clip({ tag: "video", hasAudio: true, ...fields })])).toBe(true);
|
||||
});
|
||||
|
||||
it("counts an audible video when the composition root carries a master chain", () => {
|
||||
const masterChain = '{"version":1,"nodes":[]}';
|
||||
expect(hasProjectAudio([clip({ tag: "video", hasAudio: true, masterChain })])).toBe(true);
|
||||
expect(
|
||||
hasProjectAudio([clip({ tag: "video", muted: true, hasAudio: true, masterChain })]),
|
||||
).toBe(false);
|
||||
expect(hasProjectAudio([clip({ tag: "video", masterChain })])).toBe(false);
|
||||
});
|
||||
|
||||
it("ignores a muted video even when it carries processing", () => {
|
||||
expect(hasProjectAudio([clip({ tag: "video", hasAudio: true, muted: true, volume: 2 })])).toBe(
|
||||
false,
|
||||
|
||||
@@ -61,7 +61,7 @@
|
||||
],
|
||||
"sources": {
|
||||
"packages/producer/scripts/run-test-lane.mjs": "638885e14480d03290803389325a655f3749442167d784e07bb3af3404598af7",
|
||||
"packages/producer/scripts/test-classification.mjs": "5cf0848da08f0fd2c40521d1556cfdb00b5a072cbd0ea0bc54945990abb709e4",
|
||||
"packages/producer/scripts/test-classification.mjs": "cdfa4035d0e2c0b1fea4d4c5b4697e0287d48c4474ab105f5e3d1ce839cca500",
|
||||
".github/workflows/ci.yml#job#producer-source-tests": "45da72aa6cf08cdfd5bcf5311e89f12af4842f9eebc6735df19da8c02f4d4864",
|
||||
"packages/producer/package.json#script#test:unit": "ae6beda8319d166774be958ea5593cd27dbf7f6c9bce1d787a3fe28dfc3477b1",
|
||||
"packages/producer/package.json#script#test:integration": "008589f37d026877595a13501cbb66abd5cbe50e21fdcac6904f45466c3f475d",
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
"files": 122
|
||||
},
|
||||
"hyperframes-audio": {
|
||||
"hash": "5dce20c2722a7af2",
|
||||
"hash": "780c9d4d8d23ab40",
|
||||
"files": 8
|
||||
},
|
||||
"hyperframes-cli": {
|
||||
|
||||
@@ -197,9 +197,9 @@ for rumble; a low-pass darkens or muffles deliberately.
|
||||
|
||||
**Dynamics** (`gain`, `compressor`, `limiter`, `truepeak`, `gate`) decide how a track's level
|
||||
behaves over time. Compression narrows the distance between loud and quiet so the
|
||||
quiet parts can come up. A limiter is a ceiling — it does not shape anything, it
|
||||
guarantees nothing gets past. `limiter` follows the level, so for a delivery
|
||||
ceiling use `truepeak`, which looks ahead and measures the inter-sample peak. It
|
||||
quiet parts can come up. A limiter pulls peaks down toward a ceiling and does not
|
||||
shape anything else. `limiter` follows the level and has no lookahead, so a fast
|
||||
or inter-sample peak can pass its ceiling; for a delivery ceiling use `truepeak`, which looks ahead and measures the inter-sample peak. It
|
||||
holds a 4x estimate at the ceiling, not the waveform itself, so full-band noise can
|
||||
end up to 1.7 dB over: under a hard delivery limit set the ceiling at least 2 dB
|
||||
below it. A gate removes what is below a threshold, which is
|
||||
@@ -344,6 +344,13 @@ of, so only the filters survive — and a bus and its members are one signal pat
|
||||
so the bed then runs through the bus's filters AND its own. The
|
||||
`audio_group_carve_attr` lint rule catches it.
|
||||
|
||||
**One effect across the whole mix: the master bus.** A clip names exactly one
|
||||
group, so a limiter or EQ over everything cannot be a group. Put it in
|
||||
`data-fx-chain` on the top-level composition root: it runs once over the sum of all clips and groups, after each
|
||||
group's chain and fader. A delivery ceiling is the usual use — `truepeak` last in
|
||||
the master chain. Only the top-level root is read as the master bus; a
|
||||
sub-composition root's `data-fx-chain` is not applied inside a parent composition. No master automation.
|
||||
|
||||
**One clip is not a bus.** A group exists to give several tracks one chain, one
|
||||
fader and one clock. Wrapping a single clip in a bus buys nothing the clip's own
|
||||
`data-fx-chain` does not already do, and it doubles the places a later edit has
|
||||
|
||||
@@ -110,6 +110,15 @@ cannot move: worklet processor options, a WaveShaper curve and a convolution
|
||||
impulse are all set wholesale. `fx-registry.md` marks each parameter; the
|
||||
worklet effects (`compressor`, `limiter`, `truepeak`, `gate`, `bitcrush`) have none at all.
|
||||
|
||||
## `data-fx-chain` on the root — the master bus
|
||||
|
||||
The `data-fx-chain` JSON on the top-level composition root (the element with
|
||||
`data-composition-id` that no other composition contains) is the master bus. It
|
||||
runs once over the sum of every clip and group, after each group's own chain and
|
||||
fader. On a clip or a group `data-fx-chain` keeps its own meaning, and on a
|
||||
sub-composition root inside a parent composition it is not applied. The master bus has no
|
||||
automation lanes.
|
||||
|
||||
## `data-fx-carve` — the carve's settings
|
||||
|
||||
```json
|
||||
|
||||
Reference in New Issue
Block a user