ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-10 22:20:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(producer): audio drops + blank images on FFmpeg 4.x/CORS-restricted origins (#1140)

Browse Source 
* fix(engine): remove amix normalize=0 to fix audio on FFmpeg 4.x/6.x

amix's normalize=0 option is absent from many FFmpeg builds (e.g.
FFmpeg 4.2 on Ubuntu 20.04). When the option is not recognized, FFmpeg
fails the entire filter graph initialization, processCompositionAudio
returns success:false, and the assembled video has no audio stream.

Replace normalize=0 + weights='1...' with the amix default behavior
(normalize=true, divides by track count) and multiply the master output
gain by the track count to restore the original per-track volumes.
The net volume is identical across all FFmpeg versions.

Fixes #1136-adjacent: reported as 'audio doesn't play' in rendered MP4.

* fix(producer): strip img crossorigin + fix audioExtractor normalize=0

Two follow-up fixes:

1. htmlCompiler: strip crossorigin attribute from <img> elements during
   compilation. External images (e.g. S3) with crossorigin='anonymous'
   force CORS-mode requests against the renderer's localhost file server,
   which S3 rejects → images render blank. Matches the existing video
   strip at line 261.

2. audioExtractor: same amix normalize=0 bug as audioMixer.ts. The
   audioExtractor path is used for <video data-has-audio='true'> mixing
   in the CLI's local render pipeline; on FFmpeg 4.x it would also drop
   audio silently. Fix: remove normalize=0, compensate with volume=N.

* test(engine,producer): pin amix normalize contract + img crossorigin strip

- audioMixer.test.ts: assert filter has no normalize=/weights=; add
  3-track test confirming compensatedGain = masterGain × N = 3
- htmlCompiler.test.ts: parallel tests for img and video crossorigin
  strip (covers both elements, not just video)
This commit is contained in:
Miguel Ángel
2026-05-31 09:55:49 -04:00
committed by GitHub
parent f8abff2e1c
commit fc608ad7fc
5 changed files with 121 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/engine/src/services/audioMixer.ts
+5 -3
View File
@@ -398,9 +398,11 @@ async function mixAudioTracks(
});
const mixInputs = tracks.map((_, i) => `[a${i}]`).join("");
const weights = tracks.map(() => "1").join(" ");
const mixFilter = `${mixInputs}amix=inputs=${tracks.length}:duration=longest:dropout_transition=0:normalize=0:weights='${weights}'[mixed]`;
const postMixGainFilter = `[mixed]volume=${masterOutputGain}[out]`;
const mixFilter = `${mixInputs}amix=inputs=${tracks.length}:duration=longest:dropout_transition=0[mixed]`;
// amix divides output by inputs count (default normalize=true). Multiply master
// gain by track count so per-track volumes authored in data-volume are preserved.
const compensatedGain = masterOutputGain * tracks.length;
const postMixGainFilter = `[mixed]volume=${formatFilterNumber(compensatedGain)}[out]`;
const fullFilter = [...filterParts, mixFilter, postMixGainFilter].join(";");
return [