mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 10:14:30 +00:00
## Summary
This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.
It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.
Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.
## Root Cause
The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:
- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
- `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`
That exactly matches the issue reporter's measured scalar boost.
While investigating the workaround, I also found a second correctness bug:
- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output
Separately, the repo-level Studio typecheck could fail before any build step because:
- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck
## What Changed
1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build
## Why These Changes Are Needed
This is not a UX preference issue; it is a correctness and API contract issue.
- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.
Leaving the current behavior in place means:
- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed
## Testing
### Focused regression tests
Ran:
- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`
Result:
- `10 passed`
These tests specifically verify:
- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)
### Broader package verification
Ran:
- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`
Results:
- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker
## Known Verification Limitation
There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.
## User Impact
After this change:
- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts
Closes #361.
68 lines
1.7 KiB
TypeScript
68 lines
1.7 KiB
TypeScript
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
|
import { join } from "node:path";
|
|
import { tmpdir } from "node:os";
|
|
|
|
const { runFfmpegMock } = vi.hoisted(() => ({
|
|
runFfmpegMock: vi.fn(async () => ({
|
|
success: true,
|
|
durationMs: 1,
|
|
stderr: "",
|
|
exitCode: 0,
|
|
})),
|
|
}));
|
|
|
|
vi.mock("../utils/runFfmpeg.js", () => ({
|
|
runFfmpeg: runFfmpegMock,
|
|
}));
|
|
|
|
import { processCompositionAudio } from "./audioMixer.js";
|
|
|
|
describe("processCompositionAudio", () => {
|
|
const tempDirs: string[] = [];
|
|
|
|
afterEach(() => {
|
|
runFfmpegMock.mockClear();
|
|
for (const dir of tempDirs.splice(0)) {
|
|
rmSync(dir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("preserves muted tracks and uses unity master gain by default", async () => {
|
|
const baseDir = mkdtempSync(join(tmpdir(), "hf-audio-base-"));
|
|
const workDir = mkdtempSync(join(tmpdir(), "hf-audio-work-"));
|
|
tempDirs.push(baseDir, workDir);
|
|
|
|
writeFileSync(join(baseDir, "voice.wav"), "stub");
|
|
|
|
const result = await processCompositionAudio(
|
|
[
|
|
{
|
|
id: "voice",
|
|
src: "voice.wav",
|
|
start: 0,
|
|
end: 2,
|
|
mediaStart: 0,
|
|
layer: 0,
|
|
volume: 0,
|
|
type: "audio",
|
|
},
|
|
],
|
|
baseDir,
|
|
workDir,
|
|
join(baseDir, "out.m4a"),
|
|
2,
|
|
);
|
|
|
|
expect(result.success).toBe(true);
|
|
expect(runFfmpegMock).toHaveBeenCalledTimes(2);
|
|
|
|
const mixArgs = runFfmpegMock.mock.calls[1]?.[0];
|
|
const filterIndex = mixArgs.indexOf("-filter_complex");
|
|
const filter = mixArgs[filterIndex + 1];
|
|
|
|
expect(filter).toContain("volume=0");
|
|
expect(filter).toContain("[mixed]volume=1[out]");
|
|
});
|
|
});
|