mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-19 19:36:04 +00:00
* feat(audio): composition master bus (data-fx-chain on the root) A clip can belong to only one audio group, so there was no way to put one effect, such as a limiter, across the whole mix. The data-fx-chain on the top-level composition root now runs once over the sum of every clip and every group, after each group's own chain and fader. No new attribute: a sub-composition root's data-fx-chain was never applied to anything and still is not. Render: when a master chain exists, mixAudioTracks writes a float WAV instead of AAC, applyAudioFxChain runs the chain over it, and ffmpeg encodes AAC last, so the chain reads unclipped samples. An unreadable chain is a mix failure, as for a group. Preview: the transport splices the chain between the clips' sum and the monitor, re-anchored per play generation and watched for edits. Only the top-level root is read. No master automation: the root's data-automation is not read as the bus's. No lint rule: data-fx-chain is valid on clips, groups and roots, and the one place it does nothing (a sub-composition root) has always been silent. — Jerrai * fix(core): preview routes every audible video through the master bus A root data-fx-chain is attached only inside the Web Audio graph, but a plainly audible <video> joined that graph only for its own chain, group, automation or a volume over 1. So preview limited the music and left the talking head dry, and a composition whose only audio is video got no master chain at all, while export sums every audible element into the master. A master chain on the top-level composition root is now a reason to join, the same as a video's own chain. Elements under a root with no chain keep native output. Sub-composition roots are still ignored. — Jerrai * fix(audio): master bus review round 2 (temp dir, tests, docs, Studio meter) - audioMixer: a fatal FX error in the master pass or a group bus left the work directory (the full float sum, about 1.4 GB an hour) on disk. The final mix now runs in try/finally and the group rethrow removes it first. - Tests: the "once over the sum" case now uses a limiter then a gain, so a per-clip chain (-10.4) and a doubled one (-18) both fail it; a new case proves the sum stays float until the chain reads it; the fader case uses a group volume of 0.5 so the fader's position is exercised; the work-dir removal is covered for both paths. - Docs: "Three rules exist" (the master-bus lint rule is gone), a bad master chain stops the render with audio_processing_failed, a sub-composition file opened on its own has its root chain as the master bus, and a seek clears a group or master limiter. - Studio: TimelineElement.masterChain carries the root's data-fx-chain, so the program meter shows for a video-only composition whose only audio routing is the master chain, as preview now plays it. - Comment blocks over four lines cut; webAudioTransport.setRate split so its CRAP score is under the Fallow threshold. — Jerrai * fix(audio): master chain reaches tracks the render adds; honest plain-limiter copy - Master chain: it was carried on each parsed AudioElement, so a track the producer adds itself (appendAutoDetectedVideoAudio, the browser-discovered audio in probeStage) never had it, and a composition whose only audio was such a track exported with no master chain, no ceiling and no warning. It is now a composition property: parseMasterFxChain(html) at compile, carried on CompiledComposition and CompositionMetadata, passed to processCompositionAudio by runAudioStage (in-process and distributed plan). AudioElement.masterFxChain is removed so there is one source. Each forward (compile stage, local orchestrator, distributed plan) has a test that fails when it is dropped. - Plain limiter: it follows the level with no lookahead and Level after is applied past the ceiling, so it cannot promise "nothing gets past" or "never clips". Studio copy (description, panel text, knob label, summary) and the hyperframes-audio skill now say a fast or inter-sample peak can pass it and point to the true-peak limiter for a delivery limit. A copy test guards it. — Jerrai * fix(producer): keep master bus comments within the ratchet; run the plan test with the ffmpeg tests Drop three one-line field comments that raised their files' comment share, and list plan.masterBus.test.ts with the integration tests because plan() reads the ffmpeg version. — Jerrai * chore(ci): repin the producer test-classification source in the reachability map The reachability map pins the hash of test-classification.mjs, which now lists one more integration test. The map's producer lane command already covers every test under src, in both the unit and integration matrix lanes. — Jerrai
333 lines
13 KiB
TypeScript
333 lines
13 KiB
TypeScript
import { beforeEach, describe, expect, it } from "vitest";
|
|
import {
|
|
audioGroupOf,
|
|
ensureAudioGroupInertStyle,
|
|
HF_AUDIO_GROUP_ATTR,
|
|
isMemberGroupHidden,
|
|
readMasterFxChain,
|
|
resolveAudioGroups,
|
|
resolveCarveSourceIds,
|
|
resolveGroupElement,
|
|
} from "./audioGroups.js";
|
|
import { AUDIO_GROUP_RENDER_ID_ATTR, MEDIA_RENDER_ID_ATTR } from "./compiler/mediaRenderIds.js";
|
|
|
|
beforeEach(() => {
|
|
document.body.innerHTML = "";
|
|
});
|
|
|
|
describe("resolveAudioGroups", () => {
|
|
it("returns one group of two members plus ignores an ungrouped track", () => {
|
|
document.body.innerHTML = `
|
|
<hf-audio-group id="voiceover" data-label="Voiceover"></hf-audio-group>
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
|
<audio id="sfx-1"></audio>
|
|
`;
|
|
const groups = resolveAudioGroups(document);
|
|
expect(groups).toEqual([
|
|
{
|
|
id: "voiceover",
|
|
label: "Voiceover",
|
|
memberIds: ["vo-1", "vo-2"],
|
|
volume: 1,
|
|
hidden: false,
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("resolves from member tags alone when the group element is absent, label = id", () => {
|
|
document.body.innerHTML = `
|
|
<audio id="vo-1" data-audio-group="narration"></audio>
|
|
`;
|
|
const groups = resolveAudioGroups(document);
|
|
expect(groups).toEqual([
|
|
{ id: "narration", label: "narration", memberIds: ["vo-1"], volume: 1, hidden: false },
|
|
]);
|
|
});
|
|
|
|
it("ignores data-audio-group on the group element itself (groups do not nest)", () => {
|
|
document.body.innerHTML = `
|
|
<hf-audio-group id="outer" data-audio-group="outer"></hf-audio-group>
|
|
<audio id="vo-1" data-audio-group="outer"></audio>
|
|
`;
|
|
const groups = resolveAudioGroups(document);
|
|
expect(groups).toEqual([
|
|
{ id: "outer", label: "outer", memberIds: ["vo-1"], volume: 1, hidden: false },
|
|
]);
|
|
expect(audioGroupOf(document.getElementById("outer") as Element)).toBeNull();
|
|
});
|
|
|
|
it("drops a member removed from the DOM on re-resolve — nothing dangles", () => {
|
|
document.body.innerHTML = `
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
|
`;
|
|
expect(resolveAudioGroups(document)[0].memberIds).toEqual(["vo-1", "vo-2"]);
|
|
|
|
document.getElementById("vo-2")?.remove();
|
|
expect(resolveAudioGroups(document)[0].memberIds).toEqual(["vo-1"]);
|
|
});
|
|
|
|
it("groups an audible video and ignores a muted or silent one", () => {
|
|
document.body.innerHTML = `
|
|
<video id="v-1" data-has-audio="true" data-audio-group="voiceover"></video>
|
|
<video id="v-2" muted data-audio-group="voiceover"></video>
|
|
<video id="v-3" data-has-audio="false" data-audio-group="voiceover"></video>`;
|
|
expect(resolveAudioGroups(document).map((g) => g.memberIds)).toEqual([["v-1"]]);
|
|
});
|
|
|
|
it("reads the group element's fx chain, automation, volume and hidden", () => {
|
|
document.body.innerHTML = `
|
|
<hf-audio-group id="voiceover" data-fx-chain='{"version":1,"nodes":[]}' data-automation='{"lanes":[]}' data-volume="0.5" data-hidden></hf-audio-group>
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
`;
|
|
const groups = resolveAudioGroups(document);
|
|
expect(groups).toEqual([
|
|
{
|
|
id: "voiceover",
|
|
label: "voiceover",
|
|
memberIds: ["vo-1"],
|
|
fxChain: '{"version":1,"nodes":[]}',
|
|
automation: '{"lanes":[]}',
|
|
volume: 0.5,
|
|
hidden: true,
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("defaults volume to 1 and hidden to false when a group element exists but carries neither", () => {
|
|
document.body.innerHTML = `
|
|
<hf-audio-group id="voiceover"></hf-audio-group>
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
`;
|
|
const [group] = resolveAudioGroups(document);
|
|
expect(group?.volume).toBe(1);
|
|
expect(group?.hidden).toBe(false);
|
|
expect(group?.fxChain).toBeUndefined();
|
|
expect(group?.automation).toBeUndefined();
|
|
});
|
|
});
|
|
|
|
describe("audioGroupOf", () => {
|
|
it("reads the member's group id", () => {
|
|
document.body.innerHTML = `<audio id="vo-1" data-audio-group="voiceover"></audio>`;
|
|
expect(audioGroupOf(document.getElementById("vo-1") as Element)).toBe("voiceover");
|
|
});
|
|
|
|
it("returns null when the attribute is absent", () => {
|
|
document.body.innerHTML = `<audio id="vo-1"></audio>`;
|
|
expect(audioGroupOf(document.getElementById("vo-1") as Element)).toBeNull();
|
|
});
|
|
|
|
it("reads an audible video's membership and ignores a muted video's", () => {
|
|
document.body.innerHTML = `
|
|
<video id="v-1" data-has-audio="true" data-audio-group="voiceover"></video>
|
|
<video id="v-2" muted data-audio-group="voiceover"></video>`;
|
|
expect(audioGroupOf(document.getElementById("v-1") as Element)).toBe("voiceover");
|
|
expect(audioGroupOf(document.getElementById("v-2") as Element)).toBeNull();
|
|
});
|
|
|
|
it("normalizes an empty membership attribute to null", () => {
|
|
document.body.innerHTML = `<audio id="vo-1" data-audio-group=""></audio>`;
|
|
expect(audioGroupOf(document.getElementById("vo-1") as Element)).toBeNull();
|
|
expect(resolveAudioGroups(document)).toEqual([]);
|
|
});
|
|
|
|
// Groups do not nest, and the group element is not a member of itself.
|
|
it("returns null for the group element even when it carries the attribute", () => {
|
|
document.body.innerHTML = `<hf-audio-group id="bus" data-audio-group="other"></hf-audio-group>`;
|
|
expect(audioGroupOf(document.getElementById("bus") as Element)).toBeNull();
|
|
});
|
|
});
|
|
|
|
describe("resolveCarveSourceIds", () => {
|
|
it("expands a group id to its current members", () => {
|
|
document.body.innerHTML = `
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
|
`;
|
|
expect(resolveCarveSourceIds(document, ["voiceover"])).toEqual(["vo-1", "vo-2"]);
|
|
});
|
|
|
|
it("picks up a member added to the group after the carve was set (analysis-time, not frozen)", () => {
|
|
document.body.innerHTML = `
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
|
`;
|
|
expect(resolveCarveSourceIds(document, ["voiceover"])).toEqual(["vo-1", "vo-2"]);
|
|
document.body.insertAdjacentHTML(
|
|
"beforeend",
|
|
`<audio id="vo-3" data-audio-group="voiceover"></audio>`,
|
|
);
|
|
expect(resolveCarveSourceIds(document, ["voiceover"])).toEqual(["vo-1", "vo-2", "vo-3"]);
|
|
});
|
|
|
|
it("passes through a plain clip id that still exists", () => {
|
|
document.body.innerHTML = `<audio id="vo-1"></audio>`;
|
|
expect(resolveCarveSourceIds(document, ["vo-1"])).toEqual(["vo-1"]);
|
|
});
|
|
|
|
it("drops an id that resolves to nothing — a deleted clip, an empty or vanished group", () => {
|
|
document.body.innerHTML = `<audio id="vo-1"></audio>`;
|
|
expect(resolveCarveSourceIds(document, ["vo-1", "deleted", "no-such-group"])).toEqual(["vo-1"]);
|
|
});
|
|
|
|
it("dedupes and preserves first-seen order across a mix of group and plain ids", () => {
|
|
document.body.innerHTML = `
|
|
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
|
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
|
`;
|
|
expect(resolveCarveSourceIds(document, ["voiceover", "vo-1"])).toEqual(["vo-1", "vo-2"]);
|
|
});
|
|
});
|
|
|
|
describe(HF_AUDIO_GROUP_ATTR, () => {
|
|
it("is the attribute name membership is keyed on", () => {
|
|
expect(HF_AUDIO_GROUP_ATTR).toBe("data-audio-group");
|
|
});
|
|
});
|
|
|
|
describe("resolveGroupElement", () => {
|
|
const doc = (html: string): Document => {
|
|
const d = document.implementation.createHTMLDocument("t");
|
|
d.body.innerHTML = html;
|
|
return d;
|
|
};
|
|
|
|
it("returns the bus for a real <hf-audio-group>", () => {
|
|
const d = doc(`<hf-audio-group id="vo" data-volume="0.5"></hf-audio-group>`);
|
|
expect(resolveGroupElement(d, "vo")?.tagName.toLowerCase()).toBe("hf-audio-group");
|
|
});
|
|
|
|
// The trap: a bare getElementById read a member's OWN fader and chain as the
|
|
// bus's, applying both a second time on the sub-mix.
|
|
it("refuses an <audio> sharing the group id", () => {
|
|
const d = doc(`<audio id="vo" data-audio-group="vo" data-volume="0.5"></audio>`);
|
|
expect(resolveGroupElement(d, "vo")).toBeNull();
|
|
});
|
|
|
|
it("refuses an unrelated element sharing the group id", () => {
|
|
const d = doc(`<div id="bg" data-hidden></div>`);
|
|
expect(resolveGroupElement(d, "bg")).toBeNull();
|
|
});
|
|
|
|
it("matches a stamped id containing selector syntax without throwing", () => {
|
|
const d = doc("");
|
|
const bus = d.createElement("hf-audio-group");
|
|
bus.setAttribute(MEDIA_RENDER_ID_ATTR, 'vo"\\instance');
|
|
d.body.append(bus);
|
|
|
|
expect(resolveGroupElement(d, 'vo"\\instance')).toBe(bus);
|
|
});
|
|
});
|
|
|
|
describe("isMemberGroupHidden", () => {
|
|
const doc = (html: string): Document => {
|
|
const d = document.implementation.createHTMLDocument("t");
|
|
d.body.innerHTML = html;
|
|
return d;
|
|
};
|
|
|
|
// Membership is on the MEMBER, so the bus is never an ancestor and
|
|
// `closest("[data-hidden]")` cannot see it.
|
|
it("sees a muted bus that does not nest its member", () => {
|
|
const d = doc(
|
|
`<hf-audio-group id="vo" data-hidden></hf-audio-group>
|
|
<div><audio id="vo-1" data-audio-group="vo"></audio></div>`,
|
|
);
|
|
const member = d.getElementById("vo-1");
|
|
expect(member?.closest("[data-hidden]")).toBeNull();
|
|
expect(isMemberGroupHidden(d, member)).toBe(true);
|
|
});
|
|
|
|
it("is false for an unmuted bus and for a member with no group", () => {
|
|
const d = doc(
|
|
`<hf-audio-group id="vo"></hf-audio-group>
|
|
<audio id="vo-1" data-audio-group="vo"></audio>
|
|
<audio id="lone"></audio>`,
|
|
);
|
|
expect(isMemberGroupHidden(d, d.getElementById("vo-1"))).toBe(false);
|
|
expect(isMemberGroupHidden(d, d.getElementById("lone"))).toBe(false);
|
|
});
|
|
|
|
it("uses the stamped bus instance when repeated compositions disagree on mute", () => {
|
|
const d = doc(`
|
|
<hf-audio-group id="bed" ${MEDIA_RENDER_ID_ATTR}="bed"></hf-audio-group>
|
|
<audio id="m1" data-audio-group="bed" ${AUDIO_GROUP_RENDER_ID_ATTR}="bed"></audio>
|
|
<hf-audio-group id="bed" ${MEDIA_RENDER_ID_ATTR}="bed__hf2" data-hidden></hf-audio-group>
|
|
<audio id="m2" data-audio-group="bed" ${AUDIO_GROUP_RENDER_ID_ATTR}="bed__hf2"></audio>
|
|
`);
|
|
|
|
expect(isMemberGroupHidden(d, d.getElementById("m1"))).toBe(false);
|
|
expect(isMemberGroupHidden(d, d.getElementById("m2"))).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe("resolveCarveSourceIds — empty group", () => {
|
|
it("drops an empty group's own bus id instead of returning it as a clip", () => {
|
|
const d = document.implementation.createHTMLDocument("t");
|
|
d.body.innerHTML = `<hf-audio-group id="voiceover"></hf-audio-group>`;
|
|
// No members, so the group resolves to nothing; its element must not pass
|
|
// the existence check as if it were a clip the analysis could read.
|
|
expect(resolveCarveSourceIds(d, ["voiceover"])).toEqual([]);
|
|
});
|
|
});
|
|
|
|
describe("ensureAudioGroupInertStyle", () => {
|
|
it("takes the group element out of layout", () => {
|
|
document.body.innerHTML = `<hf-audio-group id="voiceover"></hf-audio-group>`;
|
|
const el = document.getElementById("voiceover") as HTMLElement;
|
|
ensureAudioGroupInertStyle(document);
|
|
expect(getComputedStyle(el).display).toBe("none");
|
|
});
|
|
|
|
// An unknown custom element is an ordinary inline box, so in a flex or grid
|
|
// root it takes a slot: a gap, a justify-content share, and every
|
|
// :nth-child after it shifts. An author rule must not be able to put it
|
|
// back — and an id selector outranks this rule's type selector no matter
|
|
// which stylesheet came last, so `!important` is the only thing holding the
|
|
// contract. Dropping it makes this case fail.
|
|
it("beats an author rule that outranks it on specificity", () => {
|
|
document.head.insertAdjacentHTML(
|
|
"beforeend",
|
|
`<style id="author">#voiceover{display:flex}</style>`,
|
|
);
|
|
document.body.innerHTML = `<hf-audio-group id="voiceover"></hf-audio-group>`;
|
|
ensureAudioGroupInertStyle(document);
|
|
expect(getComputedStyle(document.getElementById("voiceover") as HTMLElement).display).toBe(
|
|
"none",
|
|
);
|
|
document.getElementById("author")?.remove();
|
|
});
|
|
|
|
it("injects once, however many times it is called", () => {
|
|
ensureAudioGroupInertStyle(document);
|
|
ensureAudioGroupInertStyle(document);
|
|
expect(document.querySelectorAll("#__hf-audio-group-inert")).toHaveLength(1);
|
|
});
|
|
});
|
|
|
|
describe("readMasterFxChain", () => {
|
|
const CHAIN = `{"version":1,"nodes":[]}`;
|
|
|
|
it("reads the chain off the top-level composition root", () => {
|
|
document.body.innerHTML = `<div data-composition-id="main" data-fx-chain="${CHAIN}"></div>`;
|
|
expect(readMasterFxChain(document)).toBe('{"version":1,"nodes":[]}');
|
|
});
|
|
|
|
it("is undefined when the root carries none", () => {
|
|
document.body.innerHTML = `<div data-composition-id="main"><audio id="a"></audio></div>`;
|
|
expect(readMasterFxChain(document)).toBeUndefined();
|
|
});
|
|
|
|
it("ignores a chain on a nested composition root", () => {
|
|
document.body.innerHTML = `<div data-composition-id="main"><div data-composition-id="inner" data-fx-chain="${CHAIN}"></div></div>`;
|
|
expect(readMasterFxChain(document)).toBeUndefined();
|
|
});
|
|
|
|
it("ignores a chain on a clip", () => {
|
|
document.body.innerHTML = `<div data-composition-id="main"><audio id="a" data-fx-chain="${CHAIN}"></audio></div>`;
|
|
expect(readMasterFxChain(document)).toBeUndefined();
|
|
});
|
|
});
|