mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
feat(cli): add normalize-audio to match one clip's loudness to another (#3306)
* feat(cli): add normalize-audio to match one clip's loudness to another Measures two authored `<audio>` clips with FFmpeg's integrated EBU R128 loudness and writes the target's matching `data-volume`, leaving the reference untouched. The measurement is bounded to the window the composition actually plays. `data-end` bounds a clip's timeline window just as `data-duration` does, and `-ss`/`-t` belong before `-i`: after it they bound the OUTPUT, and with `-f null` there is none, so ebur128 keeps integrating past the clip. On a fixture whose played window is -61.8 LUFS inside a file that measures -27.9 whole, either mistake reports a loudness the composition never plays and "corrects" an already-matched clip by tens of dB. Two EBU R128 passes run between reading the composition and writing it, each bounded only by a two-minute timeout, and the skill docs tell agents to keep Studio open meanwhile — so the attribute patch is re-applied to a fresh read and written through a temp file and a rename. Under `--json` the failures are documents too: an agent doing `JSON.parse(stdout)` on a bare error line throws. A pair needing more than the +12 dB ceiling has a source-file problem rather than a mixer one — mixer gain raises the noise floor with the signal — so the refusal names the remedy. * fix(cli): validate --tolerance before paying for the measurement Each EBU R128 pass is bounded at 120s and normalize-audio runs two, so parsing the argument afterwards made a typo'd --tolerance cost both of them before failing on something that was wrong from the start. Not pinned by a test: the ordering is internal to the command and neither it nor the parser is exported, so covering it would mean restructuring for a spy rather than asserting the behaviour. * docs(cli): restore the blank line between the preview and normalize-audio sections Lost when I resolved the rebase conflict against the background-preview docs by hand instead of letting the formatter near it. oxfmt --check failed on the one file, which fails Preflight — and because preview-parity needs Preflight it skipped, and the preview-regression gate fails closed on a skip, so a missing newline read as a preview defect. The quieter half: the same needs chain meant the required Test context was never created at that head. Not failing — absent, so there was no test signal at all on the PR.
This commit is contained in:
Miguel Ángel
+27
-1
@@ -23,7 +23,7 @@ npm install -g hyperframes
|
||||
| You want to | Use |
|
||||
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Start a project | [`init`](#init), [`add`](#add), [`catalog`](#catalog) |
|
||||
| Bring in source material | [`capture`](#capture), [`transcribe`](#transcribe), [`tts`](#tts), [`remove-background`](#remove-background), [`media-treatment`](#media-treatment), [`beats`](#beats) |
|
||||
| Bring in source material | [`capture`](#capture), [`transcribe`](#transcribe), [`tts`](#tts), [`remove-background`](#remove-background), [`media-treatment`](#media-treatment), [`beats`](#beats), [`normalize-audio`](#normalize-audio) |
|
||||
| Look at it, or share it | [`preview`](#preview), [`present`](#present-and-play), [`play`](#present-and-play), [`publish`](#publish) |
|
||||
| Find problems | [`lint`](#lint), [`check`](#check), [`snapshot`](#snapshot), [`keyframes`](#keyframes), [`compare`](#compare-and-grade-compare), [`grade-compare`](#compare-and-grade-compare) |
|
||||
| Make a file | [`render`](#render), [`benchmark`](#benchmark) |
|
||||
@@ -428,6 +428,32 @@ Needs a local Chrome, the same one `render` uses. Run
|
||||
within a frame or two — a different headless-Chrome audio sample rate can shift
|
||||
a beat slightly.
|
||||
|
||||
## Look at it
|
||||
|
||||
### `normalize-audio`
|
||||
|
||||
Match one authored audio clip to a reference using the exact local media bytes
|
||||
and FFmpeg's integrated EBU R128 loudness measurement.
|
||||
|
||||
```bash
|
||||
# Measure only (the default)
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio
|
||||
|
||||
# Persist the computed gain on #user-audio
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio --write
|
||||
|
||||
# Agent-readable result
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio --json
|
||||
```
|
||||
|
||||
`--reference` and `--target` are `<audio>` element ids, with or without `#`.
|
||||
The reference stays unchanged; the command accounts for both clips' current
|
||||
`data-volume` and writes the target's absolute matched gain. It measures
|
||||
`data-media-start` and `data-duration`, rejects remote or out-of-project sources,
|
||||
and refuses a result that exceeds Studio's +12 dB ceiling or would clip. Dry-run
|
||||
is deliberate: inspect the measurement before passing `--write`. Use
|
||||
`--tolerance <LU>` to change the default 0.5 LU no-op threshold.
|
||||
|
||||
### `preview`
|
||||
|
||||
Start a live preview server with hot reload.
|
||||
|
||||
@@ -47,6 +47,19 @@ previews with `--status`, `--stop`, `--list`, and `--kill-all`. Add `--json` to
|
||||
managed lifecycle commands for machine-readable output. `--foreground --json`
|
||||
prints the ready-session envelope once, then remains attached until stopped.
|
||||
|
||||
### `normalize-audio`
|
||||
|
||||
Measure two local authored audio clips with integrated LUFS and match the target
|
||||
to the unchanged reference. The command is a dry run unless `--write` is passed:
|
||||
|
||||
```bash
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio --write
|
||||
```
|
||||
|
||||
It updates only the target element's `data-volume` and refuses unsafe boosts
|
||||
that exceed Studio's +12 dB ceiling or would clip.
|
||||
|
||||
### `render`
|
||||
|
||||
Render a composition to MP4. Run from the project directory; the positional
|
||||
|
||||
@@ -134,6 +134,7 @@ const commandLoaders = {
|
||||
lint: () => import("./commands/lint.js").then((m) => m.default),
|
||||
check: () => import("./commands/check.js").then((m) => m.default),
|
||||
beats: () => import("./commands/beats.js").then((m) => m.default),
|
||||
"normalize-audio": () => import("./commands/normalize-audio.js").then((m) => m.default),
|
||||
inspect: () => import("./commands/inspect.js").then((m) => m.default),
|
||||
keyframes: () => import("./commands/keyframes.js").then((m) => m.default),
|
||||
layout: () => import("./commands/layout.js").then((m) => m.default),
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
import { resolve } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
audioNormalizationPlan,
|
||||
audioTags,
|
||||
loudnessMeasureArgs,
|
||||
parseEbur128Summary,
|
||||
resolveLocalAudioPath,
|
||||
updateAudioVolume,
|
||||
} from "./normalize-audio.js";
|
||||
|
||||
const PROJECT = "/tmp/example-project";
|
||||
|
||||
describe("audioTags", () => {
|
||||
it("reads quoted attributes without treating a quoted > as the end of the tag", () => {
|
||||
const html = `<audio id='reference' title="a > b" src="assets/ref.mp4" data-volume="1"></audio>`;
|
||||
expect(audioTags(html)).toEqual([
|
||||
expect.objectContaining({ id: "reference", src: "assets/ref.mp4", volume: 1 }),
|
||||
]);
|
||||
});
|
||||
|
||||
it("rejects duplicate ids instead of normalizing an arbitrary element", () => {
|
||||
const html = `<audio id="voice" src="a.wav"></audio><audio id="voice" src="b.wav"></audio>`;
|
||||
expect(() => audioTags(html)).toThrow(/duplicate audio id "voice"/i);
|
||||
});
|
||||
|
||||
it("ignores audio-like text in comments, scripts, and styles", () => {
|
||||
const html = `
|
||||
<!-- <audio id="comment" src="comment.wav"></audio> -->
|
||||
<script>const example = '<audio id="script" src="script.wav"></audio>';</script>
|
||||
<style>.demo::after { content: '<audio id="style" src="style.wav">'; }</style>
|
||||
<audio id="real" src="real.wav"></audio>
|
||||
`;
|
||||
expect(audioTags(html).map((tag) => tag.id)).toEqual(["real"]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[`<audio id="bad" src="a.wav" data-volume="loud"></audio>`, /data-volume/i],
|
||||
[`<audio id="bad" src="a.wav" data-media-start="-1"></audio>`, /data-media-start/i],
|
||||
[`<audio id="bad" src="a.wav" data-duration="0"></audio>`, /data-duration/i],
|
||||
])("rejects an invalid authored number", (html, expected) => {
|
||||
expect(() => audioTags(html)).toThrow(expected);
|
||||
});
|
||||
|
||||
it("measures only the window a data-end trim actually plays", () => {
|
||||
// `data-end` is a first-class trim the parser, the runtime and the render
|
||||
// mixer all honour. Measuring the whole source for one of these clips reads
|
||||
// a loudness the composition never plays, and `--write` then "corrects" an
|
||||
// already-matched clip by tens of dB.
|
||||
const html = `<audio id="vo" src="vo.wav" data-start="4" data-end="8"></audio>`;
|
||||
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: 4 }));
|
||||
});
|
||||
|
||||
it("prefers an explicit data-duration over data-end", () => {
|
||||
const html = `<audio id="vo" src="vo.wav" data-start="4" data-end="8" data-duration="3"></audio>`;
|
||||
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: 3 }));
|
||||
});
|
||||
|
||||
it("leaves an untrimmed clip unbounded", () => {
|
||||
const html = `<audio id="vo" src="vo.wav" data-start="4"></audio>`;
|
||||
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: null }));
|
||||
});
|
||||
|
||||
it("ignores a data-end that does not outlast its start", () => {
|
||||
const html = `<audio id="vo" src="vo.wav" data-start="8" data-end="4"></audio>`;
|
||||
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: null }));
|
||||
});
|
||||
});
|
||||
|
||||
describe("loudnessMeasureArgs", () => {
|
||||
it("bounds the INPUT, so the loudness filter never sees past the clip", () => {
|
||||
// `-t` after `-i` bounds the output, and `-f null` has no output worth
|
||||
// bounding: ffmpeg keeps feeding the graph and ebur128 integrates audio the
|
||||
// clip never plays. Measured on a fixture whose played window is -61.8 LUFS,
|
||||
// output-side `-t` reported -33.8.
|
||||
const args = loudnessMeasureArgs("clip.wav", { mediaStart: 2, duration: 4 });
|
||||
expect(args.indexOf("-ss")).toBeLessThan(args.indexOf("-i"));
|
||||
expect(args.indexOf("-t")).toBeLessThan(args.indexOf("-i"));
|
||||
expect(args.slice(args.indexOf("-ss"), args.indexOf("-i"))).toEqual(["-ss", "2", "-t", "4"]);
|
||||
});
|
||||
|
||||
it("leaves an untrimmed clip unbounded", () => {
|
||||
const args = loudnessMeasureArgs("clip.wav", { mediaStart: 0, duration: null });
|
||||
expect(args).not.toContain("-ss");
|
||||
expect(args).not.toContain("-t");
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveLocalAudioPath", () => {
|
||||
it("resolves a local source and strips query and fragment suffixes", () => {
|
||||
expect(resolveLocalAudioPath(PROJECT, "assets/voice%20one.wav?v=2#clip")).toBe(
|
||||
resolve(PROJECT, "assets/voice one.wav"),
|
||||
);
|
||||
});
|
||||
|
||||
it.each(["https://cdn.example.com/a.wav", "/etc/passwd", "../outside.wav", "data:x"])(
|
||||
"rejects a non-project source: %s",
|
||||
(src) => expect(() => resolveLocalAudioPath(PROJECT, src)).toThrow(/local project file/i),
|
||||
);
|
||||
});
|
||||
|
||||
describe("parseEbur128Summary", () => {
|
||||
it("takes the final integrated loudness and true-peak summary", () => {
|
||||
const stderr = `
|
||||
[Parsed_ebur128_0] t: 1.0 I: -18.2 LUFS
|
||||
[Parsed_ebur128_0] Summary:
|
||||
Integrated loudness:
|
||||
I: -15.5 LUFS
|
||||
Threshold: -25.5 LUFS
|
||||
True peak:
|
||||
Peak: -3.2 dBFS
|
||||
`;
|
||||
expect(parseEbur128Summary(stderr)).toEqual({ integratedLufs: -15.5, truePeakDbfs: -3.2 });
|
||||
});
|
||||
|
||||
it("fails when FFmpeg did not produce a usable summary", () => {
|
||||
expect(() => parseEbur128Summary("no audio stream")).toThrow(/integrated loudness/i);
|
||||
});
|
||||
});
|
||||
|
||||
describe("audioNormalizationPlan", () => {
|
||||
it("preserves the reference and attenuates the louder target", () => {
|
||||
const plan = audioNormalizationPlan(
|
||||
{ id: "target-audio", volume: 1, integratedLufs: -15.5, truePeakDbfs: -3.2 },
|
||||
{ id: "user-audio", volume: 1, integratedLufs: -11.7, truePeakDbfs: -0.2 },
|
||||
);
|
||||
|
||||
expect(plan.gainDb).toBeCloseTo(-3.8, 6);
|
||||
expect(plan.volume).toBeCloseTo(0.645654, 5);
|
||||
expect(plan.projectedLufs).toBeCloseTo(-15.5, 6);
|
||||
expect(plan.projectedTruePeakDbfs).toBeCloseTo(-4, 6);
|
||||
});
|
||||
|
||||
it("includes the reference's authored gain in the target", () => {
|
||||
const plan = audioNormalizationPlan(
|
||||
{ id: "reference", volume: 2, integratedLufs: -20, truePeakDbfs: -8 },
|
||||
{ id: "target", volume: 0.5, integratedLufs: -18, truePeakDbfs: -6 },
|
||||
);
|
||||
|
||||
expect(plan.referenceLufs).toBeCloseTo(-13.9794, 4);
|
||||
expect(plan.volume).toBeCloseTo(1.588656, 6);
|
||||
expect(plan.projectedLufs).toBeCloseTo(plan.referenceLufs, 6);
|
||||
});
|
||||
|
||||
it("refuses a gain beyond Studio's +12 dB ceiling", () => {
|
||||
expect(() =>
|
||||
audioNormalizationPlan(
|
||||
{ id: "reference", volume: 1, integratedLufs: -5, truePeakDbfs: -1 },
|
||||
{ id: "target", volume: 1, integratedLufs: -30, truePeakDbfs: -30 },
|
||||
),
|
||||
).toThrow(/\+12 dB/i);
|
||||
});
|
||||
|
||||
it("refuses a boost that would clip", () => {
|
||||
expect(() =>
|
||||
audioNormalizationPlan(
|
||||
{ id: "reference", volume: 1, integratedLufs: -10, truePeakDbfs: -1 },
|
||||
{ id: "target", volume: 1, integratedLufs: -15, truePeakDbfs: -2 },
|
||||
),
|
||||
).toThrow(/clip/i);
|
||||
});
|
||||
});
|
||||
|
||||
describe("updateAudioVolume", () => {
|
||||
it("updates only the selected audio element and preserves surrounding source", () => {
|
||||
const html = `<!doctype html>\n<audio id="ref" src="a.wav" data-volume="1"></audio>\n<audio data-volume='1' id='target' src='b.wav'></audio>\n`;
|
||||
expect(updateAudioVolume(html, "target", 0.645654)).toBe(
|
||||
`<!doctype html>\n<audio id="ref" src="a.wav" data-volume="1"></audio>\n<audio data-volume='0.645654' id='target' src='b.wav'></audio>\n`,
|
||||
);
|
||||
});
|
||||
|
||||
it("adds data-volume when it is absent", () => {
|
||||
expect(updateAudioVolume(`<audio id="target" src="b.wav" />`, "target", 2)).toBe(
|
||||
`<audio id="target" src="b.wav" data-volume="2" />`,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,601 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { existsSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
||||
import { isAbsolute, relative, resolve } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { formatAudioGain, MAX_AUDIO_GAIN_DB } from "@hyperframes/core/audio-gain";
|
||||
import { findFfBinary } from "@hyperframes/parsers/ff-binaries";
|
||||
import { defineCommand } from "citty";
|
||||
import { c } from "../ui/colors.js";
|
||||
import { resolveProject } from "../utils/project.js";
|
||||
import { failCommand } from "../utils/commandResult.js";
|
||||
import type { Example } from "./_examples.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const FFMPEG_TIMEOUT_MS = 120_000;
|
||||
const DEFAULT_TOLERANCE_LU = 0.5;
|
||||
|
||||
interface AttributeSpan {
|
||||
value: string;
|
||||
valueStart: number;
|
||||
valueEnd: number;
|
||||
}
|
||||
|
||||
interface TagRange {
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
interface ParsedStartTag extends TagRange {
|
||||
closing: boolean;
|
||||
name: string;
|
||||
}
|
||||
|
||||
interface ScanResult {
|
||||
cursor: number;
|
||||
audioRange: TagRange | null;
|
||||
}
|
||||
|
||||
export interface AudioTag {
|
||||
id: string;
|
||||
src: string;
|
||||
volume: number;
|
||||
mediaStart: number;
|
||||
duration: number | null;
|
||||
start: number;
|
||||
end: number;
|
||||
volumeAttribute: AttributeSpan | null;
|
||||
}
|
||||
|
||||
export interface LoudnessMeasurement {
|
||||
integratedLufs: number;
|
||||
truePeakDbfs: number;
|
||||
}
|
||||
|
||||
export interface NormalizationTrack extends LoudnessMeasurement {
|
||||
id: string;
|
||||
volume: number;
|
||||
}
|
||||
|
||||
export interface AudioNormalizationPlan {
|
||||
referenceId: string;
|
||||
targetId: string;
|
||||
referenceLufs: number;
|
||||
targetLufs: number;
|
||||
previousVolume: number;
|
||||
volume: number;
|
||||
gainDb: number;
|
||||
changeDb: number;
|
||||
projectedLufs: number;
|
||||
projectedTruePeakDbfs: number;
|
||||
}
|
||||
|
||||
function authoredNumber(
|
||||
raw: string | undefined,
|
||||
fallback: number,
|
||||
attribute: string,
|
||||
id: string,
|
||||
strictlyPositive = false,
|
||||
): number {
|
||||
if (raw === undefined) return fallback;
|
||||
const value = Number(raw);
|
||||
if (!Number.isFinite(value) || (strictlyPositive ? value <= 0 : value < 0)) {
|
||||
throw new Error(`Audio #${id} has an invalid ${attribute}: ${raw}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function attributeFromMatch(
|
||||
match: RegExpMatchArray,
|
||||
absoluteOffset: number,
|
||||
): [string, AttributeSpan] | null {
|
||||
if (match.index === undefined) return null;
|
||||
const name = match[1]?.toLowerCase();
|
||||
const value = matchedAttributeValue(match);
|
||||
if (!name || value === undefined) return null;
|
||||
|
||||
const whole = match[0];
|
||||
const localValueStart = attributeValueStart(whole);
|
||||
const valueStart = absoluteOffset + match.index + localValueStart;
|
||||
return [name, { value, valueStart, valueEnd: valueStart + value.length }];
|
||||
}
|
||||
|
||||
function matchedAttributeValue(match: RegExpMatchArray): string | undefined {
|
||||
return match[2] ?? match[3] ?? match[4];
|
||||
}
|
||||
|
||||
function attributeValueStart(attribute: string): number {
|
||||
let cursor = attribute.indexOf("=") + 1;
|
||||
while (/\s/.test(attribute[cursor] ?? "")) cursor += 1;
|
||||
const quote = attribute[cursor];
|
||||
return quote === '"' || quote === "'" ? cursor + 1 : cursor;
|
||||
}
|
||||
|
||||
function attributesInTag(html: string, from: number, to: number): Map<string, AttributeSpan> {
|
||||
const tag = html.slice(from, to);
|
||||
const nameEnd = tag.search(/\s|\/?\s*>/);
|
||||
const attributes = new Map<string, AttributeSpan>();
|
||||
if (nameEnd < 0) return attributes;
|
||||
|
||||
const pattern = /([^\s=/>]+)\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'=<>`]+))/g;
|
||||
for (const match of tag.slice(nameEnd).matchAll(pattern)) {
|
||||
const attribute = attributeFromMatch(match, from + nameEnd);
|
||||
if (attribute) attributes.set(...attribute);
|
||||
}
|
||||
return attributes;
|
||||
}
|
||||
|
||||
function startTagEnd(html: string, start: number): number {
|
||||
let quote = "";
|
||||
let end = start + 1;
|
||||
for (; end < html.length; end += 1) {
|
||||
const char = html[end];
|
||||
if (quote) {
|
||||
if (char === quote) quote = "";
|
||||
} else if (char === '"' || char === "'") {
|
||||
quote = char;
|
||||
} else if (char === ">") {
|
||||
return end + 1;
|
||||
}
|
||||
}
|
||||
throw new Error("Unterminated HTML start tag");
|
||||
}
|
||||
|
||||
function commentEnd(html: string, start: number): number {
|
||||
const end = html.indexOf("-->", start + 4);
|
||||
if (end < 0) throw new Error("Unterminated HTML comment");
|
||||
return end + 3;
|
||||
}
|
||||
|
||||
function parsedStartTag(html: string, start: number): ParsedStartTag | null {
|
||||
const match = html.slice(start).match(/^<\s*(\/?)\s*([a-z][a-z\d:-]*)\b/i);
|
||||
if (!match) return null;
|
||||
return {
|
||||
start,
|
||||
end: startTagEnd(html, start),
|
||||
closing: match[1] === "/",
|
||||
name: match[2]?.toLowerCase() ?? "",
|
||||
};
|
||||
}
|
||||
|
||||
function rawTextElementEnd(html: string, lower: string, tag: ParsedStartTag): number | null {
|
||||
if (tag.closing || (tag.name !== "script" && tag.name !== "style")) return null;
|
||||
const closeStart = lower.indexOf(`</${tag.name}`, tag.end);
|
||||
if (closeStart < 0) throw new Error(`Unterminated <${tag.name}> element`);
|
||||
return startTagEnd(html, closeStart);
|
||||
}
|
||||
|
||||
function scanMarkup(html: string, lower: string, start: number): ScanResult {
|
||||
if (html.startsWith("<!--", start)) {
|
||||
return { cursor: commentEnd(html, start), audioRange: null };
|
||||
}
|
||||
const tag = parsedStartTag(html, start);
|
||||
if (!tag) {
|
||||
const marker = html[start + 1];
|
||||
const cursor = marker === "!" || marker === "?" ? startTagEnd(html, start) : start + 1;
|
||||
return { cursor, audioRange: null };
|
||||
}
|
||||
const rawTextEnd = rawTextElementEnd(html, lower, tag);
|
||||
if (rawTextEnd !== null) return { cursor: rawTextEnd, audioRange: null };
|
||||
const audioRange = !tag.closing && tag.name === "audio" ? { start, end: tag.end } : null;
|
||||
return { cursor: tag.end, audioRange };
|
||||
}
|
||||
|
||||
function audioTagRanges(html: string): TagRange[] {
|
||||
const lower = html.toLowerCase();
|
||||
const ranges: TagRange[] = [];
|
||||
let cursor = 0;
|
||||
while (cursor < html.length) {
|
||||
const start = html.indexOf("<", cursor);
|
||||
if (start < 0) break;
|
||||
const scanned = scanMarkup(html, lower, start);
|
||||
if (scanned.audioRange) ranges.push(scanned.audioRange);
|
||||
cursor = scanned.cursor;
|
||||
}
|
||||
return ranges;
|
||||
}
|
||||
|
||||
function trimmedAttribute(attributes: Map<string, AttributeSpan>, name: string): string {
|
||||
return attributes.get(name)?.value.trim() ?? "";
|
||||
}
|
||||
|
||||
/**
|
||||
* How much of the source the composition actually plays.
|
||||
*
|
||||
* `data-duration` is the explicit trim; absent, `data-end` still bounds the
|
||||
* clip's timeline window, and the parser, the runtime and the render mixer all
|
||||
* honour it. Measuring the whole file for a `data-end`-trimmed clip reads a
|
||||
* loudness the composition never plays — and `--write` then "corrects" an
|
||||
* already-matched clip by tens of dB.
|
||||
*/
|
||||
function authoredDuration(attributes: Map<string, AttributeSpan>, id: string): number | null {
|
||||
const raw = attributes.get("data-duration")?.value;
|
||||
if (raw !== undefined) return authoredNumber(raw, 0, "data-duration", id, true);
|
||||
|
||||
const rawEnd = attributes.get("data-end")?.value;
|
||||
if (rawEnd === undefined) return null;
|
||||
const end = authoredNumber(rawEnd, 0, "data-end", id, true);
|
||||
const start = authoredNumber(attributes.get("data-start")?.value, 0, "data-start", id);
|
||||
return end > start ? end - start : null;
|
||||
}
|
||||
|
||||
function requiredAudioIdentity(attributes: Map<string, AttributeSpan>) {
|
||||
const id = trimmedAttribute(attributes, "id");
|
||||
const src = trimmedAttribute(attributes, "src");
|
||||
if (!id) throw new Error("Every normalized <audio> element needs an id");
|
||||
if (!src) throw new Error(`Audio #${id} has no src`);
|
||||
return { id, src };
|
||||
}
|
||||
|
||||
function audioTagFromRange(html: string, range: TagRange): AudioTag {
|
||||
const attributes = attributesInTag(html, range.start, range.end);
|
||||
const { id, src } = requiredAudioIdentity(attributes);
|
||||
const volumeAttribute = attributes.get("data-volume") ?? null;
|
||||
return {
|
||||
id,
|
||||
src,
|
||||
volume: authoredNumber(volumeAttribute?.value, 1, "data-volume", id),
|
||||
mediaStart: authoredNumber(
|
||||
attributes.get("data-media-start")?.value,
|
||||
0,
|
||||
"data-media-start",
|
||||
id,
|
||||
),
|
||||
duration: authoredDuration(attributes, id),
|
||||
start: range.start,
|
||||
end: range.end,
|
||||
volumeAttribute,
|
||||
};
|
||||
}
|
||||
|
||||
/** Read the authored audio clips without reserializing the composition. */
|
||||
export function audioTags(html: string): AudioTag[] {
|
||||
const seen = new Set<string>();
|
||||
const tags: AudioTag[] = [];
|
||||
for (const range of audioTagRanges(html)) {
|
||||
const tag = audioTagFromRange(html, range);
|
||||
if (seen.has(tag.id)) throw new Error(`Duplicate audio id "${tag.id}"`);
|
||||
seen.add(tag.id);
|
||||
tags.push(tag);
|
||||
}
|
||||
return tags;
|
||||
}
|
||||
|
||||
/** Resolve only a file contained by the project. Remote and traversal sources are not measured. */
|
||||
export function resolveLocalAudioPath(projectDir: string, src: string): string {
|
||||
const withoutSuffix = src.split(/[?#]/, 1)[0] ?? "";
|
||||
let decoded = "";
|
||||
try {
|
||||
decoded = decodeURIComponent(withoutSuffix);
|
||||
} catch {
|
||||
throw new Error(`Audio source must be a local project file: ${src}`);
|
||||
}
|
||||
if (!decoded || isAbsolute(decoded) || /^(?:[a-z][a-z\d+.-]*:|\/\/)/i.test(decoded)) {
|
||||
throw new Error(`Audio source must be a local project file: ${src}`);
|
||||
}
|
||||
const root = resolve(projectDir);
|
||||
const file = resolve(root, decoded);
|
||||
const rel = relative(root, file);
|
||||
if (!rel || rel === ".." || rel.startsWith(`..${process.platform === "win32" ? "\\" : "/"}`)) {
|
||||
throw new Error(`Audio source must be a local project file: ${src}`);
|
||||
}
|
||||
return file;
|
||||
}
|
||||
|
||||
/** Parse FFmpeg's final EBU R128 summary, not its per-window log lines. */
|
||||
export function parseEbur128Summary(stderr: string): LoudnessMeasurement {
|
||||
const summaryAt = stderr.lastIndexOf("Summary:");
|
||||
const summary = summaryAt >= 0 ? stderr.slice(summaryAt) : "";
|
||||
const integrated = summary.match(/\bI:\s*(-?\d+(?:\.\d+)?)\s+LUFS\b/);
|
||||
const peak = summary.match(/\bPeak:\s*(-?\d+(?:\.\d+)?)\s+dBFS\b/);
|
||||
const integratedLufs = Number(integrated?.[1]);
|
||||
const truePeakDbfs = Number(peak?.[1]);
|
||||
if (!Number.isFinite(integratedLufs)) {
|
||||
throw new Error("FFmpeg did not report integrated loudness for this audio stream");
|
||||
}
|
||||
if (!Number.isFinite(truePeakDbfs)) {
|
||||
throw new Error("FFmpeg did not report true peak for this audio stream");
|
||||
}
|
||||
return { integratedLufs, truePeakDbfs };
|
||||
}
|
||||
|
||||
function gainDb(volume: number): number {
|
||||
return volume > 0 ? 20 * Math.log10(volume) : Number.NEGATIVE_INFINITY;
|
||||
}
|
||||
|
||||
/** Calculate the target's absolute authored gain while keeping the reference unchanged. */
|
||||
export function audioNormalizationPlan(
|
||||
referenceTrack: NormalizationTrack,
|
||||
targetTrack: NormalizationTrack,
|
||||
): AudioNormalizationPlan {
|
||||
if (!(referenceTrack.volume > 0) || !(targetTrack.volume > 0)) {
|
||||
throw new Error("Muted audio cannot be used for loudness matching");
|
||||
}
|
||||
const referenceLufs = referenceTrack.integratedLufs + gainDb(referenceTrack.volume);
|
||||
const targetLufs = targetTrack.integratedLufs + gainDb(targetTrack.volume);
|
||||
const wantedGainDb = referenceLufs - targetTrack.integratedLufs;
|
||||
if (wantedGainDb > MAX_AUDIO_GAIN_DB + 1e-9) {
|
||||
throw new Error(
|
||||
`Matching #${targetTrack.id} needs ${wantedGainDb.toFixed(1)} dB, beyond the +${MAX_AUDIO_GAIN_DB} dB authoring ceiling. ` +
|
||||
`A gap this large belongs in the source file, not the mixer — mixer gain raises the noise floor with the signal. ` +
|
||||
`Normalize ${targetTrack.id}'s asset offline (e.g. ffmpeg loudnorm), or lower #${referenceTrack.id} instead.`,
|
||||
);
|
||||
}
|
||||
const volume = 10 ** (wantedGainDb / 20);
|
||||
const projectedTruePeakDbfs = targetTrack.truePeakDbfs + wantedGainDb;
|
||||
if (projectedTruePeakDbfs > 0) {
|
||||
throw new Error(
|
||||
`Matching #${targetTrack.id} would clip at +${projectedTruePeakDbfs.toFixed(1)} dBFS; limit or preprocess the source first`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
referenceId: referenceTrack.id,
|
||||
targetId: targetTrack.id,
|
||||
referenceLufs,
|
||||
targetLufs,
|
||||
previousVolume: targetTrack.volume,
|
||||
volume,
|
||||
gainDb: wantedGainDb,
|
||||
changeDb: wantedGainDb - gainDb(targetTrack.volume),
|
||||
projectedLufs: targetTrack.integratedLufs + wantedGainDb,
|
||||
projectedTruePeakDbfs,
|
||||
};
|
||||
}
|
||||
|
||||
/** Patch one authored attribute in place so scripts/styles/comments remain byte-stable. */
|
||||
export function updateAudioVolume(html: string, id: string, volume: number): string {
|
||||
const tag = audioTags(html).find((candidate) => candidate.id === id);
|
||||
if (!tag) throw new Error(`Audio #${id} was not found`);
|
||||
const value = formatAudioGain(volume);
|
||||
if (tag.volumeAttribute) {
|
||||
return (
|
||||
html.slice(0, tag.volumeAttribute.valueStart) +
|
||||
value +
|
||||
html.slice(tag.volumeAttribute.valueEnd)
|
||||
);
|
||||
}
|
||||
|
||||
let insertion = tag.end - 1;
|
||||
let tail = insertion - 1;
|
||||
while (tail >= tag.start && /\s/.test(html[tail] ?? "")) tail -= 1;
|
||||
if (html[tail] === "/") {
|
||||
insertion = tail;
|
||||
while (insertion > tag.start && /\s/.test(html[insertion - 1] ?? "")) insertion -= 1;
|
||||
} else {
|
||||
while (insertion > tag.start && /\s/.test(html[insertion - 1] ?? "")) insertion -= 1;
|
||||
}
|
||||
return html.slice(0, insertion) + ` data-volume="${value}"` + html.slice(insertion);
|
||||
}
|
||||
|
||||
/**
|
||||
* FFmpeg arguments that measure exactly the window the composition plays.
|
||||
*
|
||||
* `-ss` / `-t` go BEFORE `-i`, as INPUT options. After `-i` they bound the
|
||||
* output, and with `-f null` there is no real output to bound: ffmpeg keeps
|
||||
* feeding the filter graph past the limit, so ebur128 integrates audio the clip
|
||||
* never plays. Measured on a 12 s file whose first 4 s — the played window — is
|
||||
* -61.8 LUFS: output-side `-t 4` reported -33.8 LUFS, input-side reports -61.8,
|
||||
* the same as physically cutting the file first.
|
||||
*/
|
||||
export function loudnessMeasureArgs(
|
||||
file: string,
|
||||
tag: Pick<AudioTag, "mediaStart" | "duration">,
|
||||
): string[] {
|
||||
const args = ["-hide_banner", "-nostats"];
|
||||
if (tag.mediaStart > 0) args.push("-ss", String(tag.mediaStart));
|
||||
if (tag.duration !== null && tag.duration > 0) args.push("-t", String(tag.duration));
|
||||
args.push("-i", file, "-map", "0:a:0", "-vn", "-af", "ebur128=peak=true", "-f", "null", "-");
|
||||
return args;
|
||||
}
|
||||
|
||||
async function measureAudio(
|
||||
ffmpegPath: string,
|
||||
file: string,
|
||||
tag: Pick<AudioTag, "mediaStart" | "duration">,
|
||||
): Promise<LoudnessMeasurement> {
|
||||
const result = await execFileAsync(ffmpegPath, loudnessMeasureArgs(file, tag), {
|
||||
encoding: "utf8",
|
||||
maxBuffer: 16 * 1024 * 1024,
|
||||
timeout: FFMPEG_TIMEOUT_MS,
|
||||
});
|
||||
return parseEbur128Summary(result.stderr);
|
||||
}
|
||||
|
||||
/**
|
||||
* Under `--json` every outcome has to be one parseable document, including the
|
||||
* failures — this command exists to be driven by an agent, and an agent doing
|
||||
* `JSON.parse(stdout)` on a bare error line just throws.
|
||||
*/
|
||||
function fail(message: string, json: boolean): never {
|
||||
if (json) console.log(JSON.stringify({ ok: false, error: message }, null, 2));
|
||||
else console.error(c.error(message));
|
||||
failCommand();
|
||||
}
|
||||
|
||||
function byId(tags: readonly AudioTag[], rawId: string, role: string): AudioTag {
|
||||
const id = rawId.trim().replace(/^#/, "");
|
||||
const tag = tags.find((candidate) => candidate.id === id);
|
||||
if (!tag) throw new Error(`${role} audio #${id} was not found`);
|
||||
return tag;
|
||||
}
|
||||
|
||||
interface NormalizeAudioOptions {
|
||||
dir?: string;
|
||||
reference: string;
|
||||
target: string;
|
||||
tolerance: string;
|
||||
write: boolean;
|
||||
}
|
||||
|
||||
interface NormalizeAudioResult extends AudioNormalizationPlan {
|
||||
ok: true;
|
||||
wrote: boolean;
|
||||
withinTolerance: boolean;
|
||||
tolerance: number;
|
||||
indexPath: string;
|
||||
}
|
||||
|
||||
function selectedTags(html: string, referenceId: string, targetId: string) {
|
||||
const tags = audioTags(html);
|
||||
const referenceTag = byId(tags, referenceId, "Reference");
|
||||
const targetTag = byId(tags, targetId, "Target");
|
||||
if (referenceTag.id === targetTag.id) {
|
||||
throw new Error("Reference and target must be different audio elements");
|
||||
}
|
||||
return { referenceTag, targetTag };
|
||||
}
|
||||
|
||||
function requiredFfmpeg(): string {
|
||||
const ffmpegPath = findFfBinary("ffmpeg", { configuredMustExist: true });
|
||||
if (!ffmpegPath) throw new Error("FFmpeg is required to measure integrated loudness");
|
||||
return ffmpegPath;
|
||||
}
|
||||
|
||||
function existingAudioFile(projectDir: string, tag: AudioTag): string {
|
||||
const file = resolveLocalAudioPath(projectDir, tag.src);
|
||||
if (!existsSync(file)) {
|
||||
throw new Error(`Audio #${tag.id} file was not found: ${relative(projectDir, file)}`);
|
||||
}
|
||||
return file;
|
||||
}
|
||||
|
||||
async function measuredPlan(
|
||||
projectDir: string,
|
||||
referenceTag: AudioTag,
|
||||
targetTag: AudioTag,
|
||||
): Promise<AudioNormalizationPlan> {
|
||||
const ffmpegPath = requiredFfmpeg();
|
||||
const referenceFile = existingAudioFile(projectDir, referenceTag);
|
||||
const targetFile = existingAudioFile(projectDir, targetTag);
|
||||
const [referenceMeasurement, targetMeasurement] = await Promise.all([
|
||||
measureAudio(ffmpegPath, referenceFile, referenceTag),
|
||||
measureAudio(ffmpegPath, targetFile, targetTag),
|
||||
]);
|
||||
return audioNormalizationPlan(
|
||||
{ id: referenceTag.id, volume: referenceTag.volume, ...referenceMeasurement },
|
||||
{ id: targetTag.id, volume: targetTag.volume, ...targetMeasurement },
|
||||
);
|
||||
}
|
||||
|
||||
/** Never leave a half-written composition behind if the process dies mid-write. */
|
||||
function writeAtomically(path: string, contents: string): void {
|
||||
const temporary = `${path}.hf-normalize-${process.pid}.tmp`;
|
||||
writeFileSync(temporary, contents);
|
||||
renameSync(temporary, path);
|
||||
}
|
||||
|
||||
function parsedTolerance(raw: string): number {
|
||||
const tolerance = Number(raw);
|
||||
if (!Number.isFinite(tolerance) || tolerance < 0) {
|
||||
throw new Error("--tolerance must be a non-negative number");
|
||||
}
|
||||
return tolerance;
|
||||
}
|
||||
|
||||
async function normalizeAudio(options: NormalizeAudioOptions): Promise<NormalizeAudioResult> {
|
||||
const project = resolveProject(options.dir);
|
||||
const html = readFileSync(project.indexPath, "utf8");
|
||||
const { referenceTag, targetTag } = selectedTags(html, options.reference, options.target);
|
||||
// Before the measurement, not after: each EBU R128 pass is bounded at 120s, so
|
||||
// validating this afterwards made a typo'd --tolerance cost two of them before
|
||||
// failing on an argument that was wrong the whole time.
|
||||
const tolerance = parsedTolerance(options.tolerance);
|
||||
const plan = await measuredPlan(project.dir, referenceTag, targetTag);
|
||||
const withinTolerance = Math.abs(plan.referenceLufs - plan.targetLufs) <= tolerance;
|
||||
const wrote = options.write && !withinTolerance;
|
||||
// Re-read: two EBU R128 passes ran since the snapshot above, each bounded
|
||||
// only by FFMPEG_TIMEOUT_MS, and the skill tells agents to keep Studio open
|
||||
// on the project while normalizing. Serializing the stale snapshot would
|
||||
// silently revert every edit made during that window. The write is one
|
||||
// attribute patch, so re-applying it to the current file is the same edit.
|
||||
if (wrote) {
|
||||
writeAtomically(
|
||||
project.indexPath,
|
||||
updateAudioVolume(readFileSync(project.indexPath, "utf8"), targetTag.id, plan.volume),
|
||||
);
|
||||
}
|
||||
return { ok: true, wrote, withinTolerance, tolerance, indexPath: project.indexPath, ...plan };
|
||||
}
|
||||
|
||||
function printHumanResult(result: NormalizeAudioResult): void {
|
||||
console.log(
|
||||
`${c.bold(`#${result.referenceId}`)} ${result.referenceLufs.toFixed(1)} LUFS → ` +
|
||||
`${c.bold(`#${result.targetId}`)} ${result.targetLufs.toFixed(1)} LUFS`,
|
||||
);
|
||||
if (result.withinTolerance) {
|
||||
console.log(
|
||||
c.success(`Already matched within ${result.tolerance.toFixed(1)} LU; no change needed.`),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Set #${result.targetId} data-volume ${result.previousVolume.toFixed(3)} → ${result.volume.toFixed(6)} ` +
|
||||
`(${result.changeDb >= 0 ? "+" : ""}${result.changeDb.toFixed(1)} dB).`,
|
||||
);
|
||||
console.log(
|
||||
result.wrote
|
||||
? c.success(`Updated ${relative(process.cwd(), result.indexPath) || "index.html"}.`)
|
||||
: c.dim("Dry run only. Pass --write to persist this gain."),
|
||||
);
|
||||
}
|
||||
|
||||
export const examples: Example[] = [
|
||||
[
|
||||
"Measure two authored clips and preview the matching gain",
|
||||
"hyperframes normalize-audio --reference target-audio --target user-audio",
|
||||
],
|
||||
[
|
||||
"Persist the measured gain into index.html",
|
||||
"hyperframes normalize-audio --reference target-audio --target user-audio --write",
|
||||
],
|
||||
];
|
||||
|
||||
export default defineCommand({
|
||||
meta: {
|
||||
name: "normalize-audio",
|
||||
description: "Match one local audio clip to another using integrated LUFS",
|
||||
},
|
||||
args: {
|
||||
dir: { type: "positional", description: "Project directory", required: false },
|
||||
reference: {
|
||||
type: "string",
|
||||
description: "Audio element id whose effective loudness should be preserved",
|
||||
required: true,
|
||||
},
|
||||
target: {
|
||||
type: "string",
|
||||
description: "Audio element id whose data-volume should be matched",
|
||||
required: true,
|
||||
},
|
||||
write: {
|
||||
type: "boolean",
|
||||
description: "Persist the measured target gain into index.html",
|
||||
default: false,
|
||||
},
|
||||
tolerance: {
|
||||
type: "string",
|
||||
description: `Skip writes within this many LU (default ${DEFAULT_TOLERANCE_LU})`,
|
||||
default: String(DEFAULT_TOLERANCE_LU),
|
||||
},
|
||||
json: { type: "boolean", description: "Print machine-readable output", default: false },
|
||||
},
|
||||
async run({ args }) {
|
||||
try {
|
||||
const result = await normalizeAudio({
|
||||
dir: args.dir,
|
||||
reference: args.reference,
|
||||
target: args.target,
|
||||
tolerance: args.tolerance,
|
||||
write: args.write,
|
||||
});
|
||||
if (args.json) {
|
||||
const { indexPath: _, ...jsonResult } = result;
|
||||
console.log(JSON.stringify(jsonResult, null, 2));
|
||||
return;
|
||||
}
|
||||
printHumanResult(result);
|
||||
} catch (error) {
|
||||
fail(error instanceof Error ? error.message : String(error), Boolean(args.json));
|
||||
}
|
||||
},
|
||||
});
|
||||
@@ -46,6 +46,10 @@ const GROUPS: Group[] = [
|
||||
"media-treatment",
|
||||
"Discover, apply, or clear deterministic media treatments on one media element",
|
||||
],
|
||||
[
|
||||
"normalize-audio",
|
||||
"Match one authored audio clip's loudness to another using integrated LUFS",
|
||||
],
|
||||
[
|
||||
"grade-compare",
|
||||
"Render candidate color grades onto a reference frame as one labeled comparison PNG",
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
"files": 121
|
||||
},
|
||||
"hyperframes-audio": {
|
||||
"hash": "c96ccac4b1127b8c",
|
||||
"hash": "6bdb36e1571586fe",
|
||||
"files": 6
|
||||
},
|
||||
"hyperframes-cli": {
|
||||
|
||||
@@ -135,6 +135,41 @@ the ambiguity instead.
|
||||
|
||||
## Recipes
|
||||
|
||||
### Compare loudness from the bytes the listener actually hears
|
||||
|
||||
Do not call two clips equally loud because their Studio faders, waveform peaks,
|
||||
or cached asset metadata match. Those are controls and proxies, not a loudness
|
||||
measurement. Resolve the exact URLs used by preview/render, download or inspect
|
||||
those exact served bytes, and measure each decoded stream with FFmpeg's
|
||||
`ebur128` filter. Compare the integrated LUFS values.
|
||||
|
||||
For a target loudness, the required move is:
|
||||
|
||||
```text
|
||||
gain_db = target_lufs - measured_lufs
|
||||
linear_gain = 10 ** (gain_db / 20)
|
||||
```
|
||||
|
||||
When both clips are local authored `<audio>` elements with stable ids, use the
|
||||
CLI instead of transcribing that arithmetic by hand:
|
||||
|
||||
```bash
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio
|
||||
npx hyperframes normalize-audio --reference target-audio --target user-audio --write
|
||||
```
|
||||
|
||||
The first command is a dry run. The second writes only the target's
|
||||
`data-volume`, after accounting for both existing gains and refusing a boost
|
||||
that would clip or exceed Studio's ceiling. Always choose the reference from the
|
||||
author's stated intent; the command does not guess which clip should define the
|
||||
mix.
|
||||
|
||||
Studio's clip-gain fader uses `0 dB` / linear gain `1` at its physical midpoint
|
||||
and provides up to `+12 dB` on the upper half. After changing gain, measure the
|
||||
served preview/render bytes again. If a listener still hears a mismatch, trust
|
||||
the report and first verify the asset URL and bytes are current; do not explain
|
||||
it away with matching peaks or a stale proxy measurement.
|
||||
|
||||
All verified with ffmpeg 8.1.1. `-hide_banner` keeps the output readable;
|
||||
`volumedetect` prints to stderr, so do not silence it with `-v error`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user