ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-10 22:20:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(audio): raise the authoring gain ceiling and carry it through the probes (#3333)

Browse Source 
* fix(audio): raise the authoring gain ceiling and carry it through the probes

Builds on #3328, which made the preview graph apply author gain and user volume
exactly once each. That ownership is now correct but everything is still clamped
to 1.0, so a clip authored above unity cannot be heard or rendered.

`HTMLMediaElement.volume` is spec-clamped to [0,1], so both timeline probes lost
a clip's authored gain the moment it also carried a fade: the probe seeded the
element at the clamped value and every sample read back at or below 0 dB, and
the mixer prefers probed keyframes over the static volume. Both probes now
shadow the accessor for their own duration and forward the clamped value to the
native setter, so the authored gain survives while nothing outside the probe
ever sees an illegal volume.

Measured on one 6 s composition, first 4 s: unity -32.8 LUFS, boosted-with-fade
-32.8 before and -27.0 after — +5.8 dB, exactly the gain the clip was authored
at.

One ceiling, defined once in `audioGain.ts` and reachable from both sides: the
render mixer imports it, and the page-serialized probe takes it as a parameter
rather than re-literalling it. User volume stays spec-clamped — it is a fader,
not a gain.

Also holds the percent volume slider above unity in both property panels. That
control tops out at 100%, so one touch would cap a boosted clip and drop up to
12 dB that now genuinely renders; the dB fader that can represent these levels
replaces it in the next PR.

* fix(audio): carry a static above-unity gain onto the preview gain node

Review follow-up.

`setElementVolume` receives the clip's author gain and clamped it to [0,1],
so a static `data-volume` above unity was capped on the WebAudio preview path
while the render honoured it — the exact preview/render divergence this
ceiling exists to close. Automation lanes hid it: they schedule ramps onto the
param directly and never pass through here. The master volume beside it stays
spec-clamped, because a user fader is not a gain.

Verified by mutation: restoring the [0,1] clamp reds the new case.

Also scope the leveller's rationale to this rung — `VOLUME_RANGE` still stops
at unity until the dB fader lands, so "both now span the same range" was
premature — and say why the GSAP-tracking fallback is unity-capped: it reads
back through `el.volume`, which the spec pins to [0,1], so it cannot observe an
above-unity value however wide the clamp gets.

* fix(audio): restore the live test files this branch overwrote, and uncap preview

Review blocker: three files were wholesale copies from the abandoned #3304
branch laid over a two-day-newer base, so they silently reverted work that had
landed in between. CI could not see it — deleted tests do not fail.

- `audioMixer.test.ts` was byte-identical to #3304's head: 1186 lines against a
  base of 1353. Gone with it were the `data-playback-start` fallthrough cases
  from #3322 — merged 54 minutes before this branch's own merge base — and all
  retiming coverage (`playbackRate` 7 to 0, `atempo` 5 to 0), the strict
  literal-timing table, and the zero-window cases.
- `mediaVolumeEnvelope.test.ts` dropped the trailing-garbage duration case and
  the plateau-retention case.
- `packages/core/package.json` rolled the package version back 0.8.3 to 0.7.109.

All three are restored from `main` with only this PR's additions re-applied on
top, and the subpath export is regenerated by the repo's own script rather than
hand-edited.

Also closes the preview/render split the same review raised. Two clamps had to
go, not one: `setElementVolume` capped the author gain at the transport, and
the first-tick branch in `syncRuntimeMedia` trusted `el.volume` — which is
spec-bound to [0,1] and so cannot represent a boost, opening a boosted clip at
0 dB for one tick before the steady-state branch took over. Both pinned by
tests, both verified by mutation.
This commit is contained in:
Miguel Ángel
2026-08-18 20:07:42 -04:00
committed by GitHub
parent 74149e249a
commit e282ff15cc
18 changed files with 552 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/producer/src/services/htmlCompiler.ts
+73 -32
View File
@@ -27,6 +27,7 @@ import {
type ResolvedDuration,
type UnresolvedElement,
} from "@hyperframes/core";
import { MAX_AUDIO_GAIN } from "@hyperframes/core/audio-gain";
import {
assignBundledRuntimeCompositionIds,
type BundledHostCompositionIdentity,
@@ -2256,8 +2257,44 @@ export async function discoverAudioVolumeAutomationFromTimeline(
return { id, start, end };
});
return page.evaluate(
({ clips, duration, step }) => {
({ clips, duration, step, maxGain }) => {
const results: { id: string; keyframes: { time: number; volume: number }[] }[] = [];
const clampGain = (value: number) =>
Number.isFinite(value) ? Math.max(0, Math.min(maxGain, value)) : 1;
// `HTMLMediaElement.volume` is spec-clamped to [0,1], so a clip authored
// above unity — or a GSAP tween seeded from one — reads back as 0 dB and
// the whole authored boost is lost from the mix. Shadow the accessor for
// the probe so the authored value survives; the native setter still gets
// the clamped value. Mirrors `withUnclampedVolume` in
// packages/core/src/audioGain.ts, which the preview probe uses; this copy
// exists only because the probe body is serialized into the page.
//
// Guarded because this body is also executed directly by tests that stand
// in for a Page, where there is no DOM and no HTMLMediaElement.
const volumeDescriptor =
typeof HTMLMediaElement === "undefined"
? undefined
: Object.getOwnPropertyDescriptor(HTMLMediaElement.prototype, "volume");
const nativeVolumeGet = volumeDescriptor?.get;
const nativeVolumeSet = volumeDescriptor?.set;
const withUnclampedVolume = <T>(el: HTMLMediaElement, probe: () => T): T => {
if (!nativeVolumeGet || !nativeVolumeSet) return probe();
let authored = Number(nativeVolumeGet.call(el));
Object.defineProperty(el, "volume", {
configurable: true,
get: () => authored,
set: (value: number) => {
authored = Number(value);
nativeVolumeSet.call(el, Math.max(0, Math.min(1, authored)));
},
});
try {
return probe();
} finally {
delete (el as unknown as Record<"volume", unknown>).volume;
nativeVolumeSet.call(el, Math.max(0, Math.min(1, authored)));
}
};
const timelines = (window as unknown as { __timelines?: Record<string, unknown> })
.__timelines;
if (!timelines) return results;
@@ -2290,42 +2327,46 @@ export async function discoverAudioVolumeAutomationFromTimeline(
const sampleStart = Math.max(0, start);
const sampleEnd = Math.min(duration, end);
const initialVolumeAttr = Number.parseFloat(el.dataset.volume ?? "");
if (Number.isFinite(initialVolumeAttr)) {
el.volume = Math.max(0, Math.min(1, initialVolumeAttr));
}
const keyframes: { time: number; volume: number }[] = [];
let previousSample: { time: number; volume: number } | undefined;
for (let t = sampleStart; t <= sampleEnd + 0.000001; t = Math.min(sampleEnd, t + step)) {
seekTl(t);
const rawVolume = Number(el.volume);
if (!Number.isFinite(rawVolume)) {
if (t === sampleEnd) break;
continue;
const keyframes = withUnclampedVolume(el, () => {
if (Number.isFinite(initialVolumeAttr)) {
el.volume = clampGain(initialVolumeAttr);
}
const volume = Math.max(0, Math.min(1, rawVolume));
const sample = {
time: Number(t.toFixed(6)),
volume: Number(volume.toFixed(6)),
};
const last = keyframes.at(-1);
if (!last || Math.abs(last.volume - volume) > 0.0001) {
// Retain the preceding real sample when compression omitted a flat
// run. Continuous ramps already have that sample as their last
// keyframe, so their interpolation remains unchanged.
if (last && previousSample && previousSample.time > last.time) {
keyframes.push(previousSample);
const keyframes: { time: number; volume: number }[] = [];
let previousSample: { time: number; volume: number } | undefined;
for (let t = sampleStart; t <= sampleEnd + 0.000001; t = Math.min(sampleEnd, t + step)) {
seekTl(t);
const rawVolume = Number(el.volume);
if (!Number.isFinite(rawVolume)) {
if (t === sampleEnd) break;
continue;
}
keyframes.push(sample);
} else if (t === sampleEnd && sample.time > last.time) {
keyframes.push(sample);
const volume = clampGain(rawVolume);
const sample = {
time: Number(t.toFixed(6)),
volume: Number(volume.toFixed(6)),
};
const last = keyframes.at(-1);
if (!last || Math.abs(last.volume - volume) > 0.0001) {
// Retain the preceding real sample when compression omitted a flat
// run. Continuous ramps already have that sample as their last
// keyframe, so their interpolation remains unchanged.
if (last && previousSample && previousSample.time > last.time) {
keyframes.push(previousSample);
}
keyframes.push(sample);
} else if (t === sampleEnd && sample.time > last.time) {
keyframes.push(sample);
}
previousSample = sample;
if (t === sampleEnd) break;
}
previousSample = sample;
if (t === sampleEnd) break;
}
return keyframes;
});
const staticAttr = Number.parseFloat(el.dataset.volume ?? "");
const staticVolume = Number.isFinite(staticAttr) ? Math.max(0, Math.min(1, staticAttr)) : 1;
const staticVolume = Number.isFinite(staticAttr) ? clampGain(staticAttr) : 1;
const hasAutomation = keyframes.some(
(keyframe) => Math.abs(keyframe.volume - staticVolume) > 0.0001,
);
@@ -2337,7 +2378,7 @@ export async function discoverAudioVolumeAutomationFromTimeline(
seekTl(0);
return results;
},
{ clips, duration: compositionDuration, step: sampleStep },
{ clips, duration: compositionDuration, step: sampleStep, maxGain: MAX_AUDIO_GAIN },
);
}