Compositions are now self-contained: the compiler captures font files
and embeds them as woff2 data URIs, eliminating silent render-time
fallback when the render environment lacks the author's fonts.
Resolution order (each tier falls through to the next):
1. Existing @font-face → use as-is
2. Bundled alias (38 cross-platform mappings) → embed data URI
3. Google Fonts → fetch, cache, embed
4. Local system font → locate on OS, compress to woff2, embed
5. Local @font-face paths → read file, compress, inline as data URI
6. External CDN stylesheets → fetch CSS, extract @font-face, inline
7. Alias map fallback → closest bundled equivalent
8. Actionable error with guidance
Key changes:
- System font locator (macOS/Windows/Linux) with path-bounding and
symlink defense (realpathSync + O_NOFOLLOW)
- woff2 compression via wawoff2 (WASM, cross-platform)
- Multi-weight/style variant capture with length-sorted token matching
- External stylesheet inlining with SSRF defense (assertPublicHttpsUrl,
HTTPS-only, private-host blocking, 2MB cap, 4-concurrent limit)
- Studio auto-import via GET /fonts/file API + renderAliasFor() derived
from shared FONT_ALIAS_MAP (no more hand-curated drift)
- failClosedFontFetch throws on unresolved fonts in distributed renders
- Single source of truth: @hyperframes/core/fonts/aliases
- system_font_will_alias lint rule (escalates to warning for distributed)
- Default to Inter + JetBrains Mono in templates and CSS reset
* fix(producer): restore 30s calibration timeout ceiling to prevent render hang
The v0.6.74 change (Math.min → Math.max in createCaptureCalibrationConfig)
raised the calibration protocol timeout from 30s to the default 300s. When
a CDP call stalls during session init — page.goto, pollHfReady, or any
page.evaluate — the 300s timeout makes the render appear to hang
indefinitely at "Initializing calibration session...".
Restore Math.min so calibration stays capped at 30s: if Chrome is stuck,
fail fast and let the fallback path recover. Also add phase-level timing
logs to initializeSession so the next report pinpoints which step stalls.
Closes#1231
* fix(producer): add render pipeline observability for faster triage
Log the resolved environment at pipeline start (platform, arch, node
version, all timeout values, GPU mode), the calibration config showing
the actual timeout being used vs the parent, Chrome version and capture
mode at browser launch, and a structured failure summary on error with
stage timings and console errors. These four log categories give agents
and users enough context to file actionable issues without needing to
reproduce the problem.
* fix(engine): add missing pollVideosReady phase log in screenshot path
The BeginFrame path logged this phase but the screenshot path didn't,
creating an instrumentation gap when diagnosing hangs on macOS where
screenshot mode is always used.
## What
Adds an auto-detected **low-memory safe render profile**. On hosts at or below 8 GB total RAM, the render pipeline collapses to its cheapest shape instead of running multiple concurrent Chrome instances.
When `lowMemoryMode` is active and the user hasn't passed `--workers`, the orchestrator:
- **skips auto-worker calibration** — no throwaway second Chrome just to time 5 frames;
- **pins to a single worker** — so the probe Chrome is reused for capture, never N concurrent;
- **prefers screenshot capture over BeginFrame** — avoids the BeginFrame protocol-timeout → relaunch churn on slow hardware;
- logs a one-line explanation of what it did and how to override.
Builds on #1221 (merged), which fixed the calibration timeout cap, the `<= 8192` boundary, and added the CLI timeout flags.
## Why
Reported in #1218 / #1219: renders on 8 GB laptops sit at low progress for minutes or stall. Root cause (per the triage thread) is architectural — the default pipeline launches up to 4 Chrome instances sequentially/overlapping (probe, calibration, capture, screenshot-fallback), each ~256 MB+, on machines with ~3 GB free. The concurrent browsers drive memory pressure that makes every CDP call slow and spikes V8 GC pauses.
#1221 made the timeouts and memory flags *apply correctly*; this PR removes the expensive shape entirely on the machines that can't afford it, rather than tuning it. "Smarter by default."
## How
- **`packages/engine/src/services/systemMemory.ts`** (new): one shared `isLowMemorySystem()` / `getSystemTotalMb()`, de-duplicating the `totalmem()` reads previously copied in `config.ts` and `browserManager.ts`. Threshold is inclusive (`<= 8192 MB`) — real "8 GB" hardware reports ~7600–8192 MB after firmware/iGPU reservations, so a strict `<` would skip the optimisation on the very hardware that needs it.
- **`config.ts`**: new `lowMemoryMode` field on `EngineConfig`, resolved tri-state — explicit override → `PRODUCER_LOW_MEMORY_MODE` (on/off) → auto-detect from total RAM.
- **`renderOrchestrator.ts`**: gate calibration off, pin workers to 1, force screenshot capture, and emit a safe-mode log line when `lowMemoryMode` is set and `--workers` is absent.
- **`render.ts`**: `--low-memory-mode` / `--no-low-memory-mode` override (sets the env var the producer's `resolveConfig` reads) + docs table entry.
Fully overridable: an explicit `--workers N` restores calibration-free parallelism; `--no-low-memory-mode` / `PRODUCER_LOW_MEMORY_MODE=false` restores the full default shape.
### Deliberately deferred (separate PRs)
- **Reuse the probe session for calibration**: only executes on the tier *above* 8 GB (safe-mode skips calibration on the target boxes). A correct BeginFrame-mode reuse would lose calibration's fast-fail-to-screenshot timeout — real risk on a path the reported scenario never hits. Better scoped on its own.
- **Retuning `calculateOptimalWorkers`'s `totalmem*0.5/256` memory model**: hot path for *all* renders incl. servers/Lambda, outside this PR's local-laptop scope.
## Test plan
- [x] Unit tests added/updated — `systemMemory.test.ts` (8192 boundary cases), `config.test.ts` (tri-state env resolution + explicit-override precedence). Engine suite passes (25 relevant tests).
- [x] `tsc` clean across engine/producer/cli; `oxlint` + `oxfmt` clean; removed an unused export so the `fallow --fail-on-issues` dead-code gate stays green.
- [x] Documentation updated — `docs/packages/cli.mdx` render-flags table.
- [ ] Manual testing on a real ≤ 8 GB host — not yet run; behaviour is unit-covered and the safe path (1 worker + screenshot) is already a supported render shape.
Note: one pre-existing producer test (`rejects a maliciously crafted key…`) fails identically on `main` — environment-specific path test, unrelated to this change.
Closes#1219
## Problem
On 8GB RAM machines, renders time out at 5% with `Runtime.callFunctionOn timed out` during the duration probe. User-set timeout env vars (`PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS`) are silently ignored by the calibration path, and there are no CLI flags to control timeouts directly.
## Root causes
1. **Calibration timeout cap overrides user settings** — `createCaptureCalibrationConfig` used `Math.min(cfg.protocolTimeout, 30_000)`, meaning even if the user set 300s, calibration still capped at 30s. On slow hardware this causes unnecessary timeouts.
2. **8GB systems get no low-memory treatment** — `getLowMemoryFlags()`, `getGpuMemBudgetMb()`, `memoryAdaptiveCacheLimit()`, and `memoryAdaptiveCacheBytesMb()` all used `< 8192` as the threshold. Systems reporting exactly 8192 MB (common for 8GB machines) fell through to the "plenty of memory" path, getting no Chrome heap reduction or cache limits.
3. **No CLI flags for key timeouts** — Users had to discover the correct env var names (`PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS`, `PRODUCER_PLAYER_READY_TIMEOUT_MS`) by reading source. The non-existent `PUPPETEER_PROTOCOL_TIMEOUT` and `--browser-timeout` were common guesses that did nothing.
## Changes
- `captureCost.ts`: `Math.min` → `Math.max` so the 30s calibration default is a floor, not a ceiling. User-set higher timeouts are now respected.
- `browserManager.ts`: `>= 8192` → `> 8192` in `getLowMemoryFlags()` and `<= 8192` in `getGpuMemBudgetMb()` so 8GB systems get reduced Chrome heap and GPU memory budget.
- `config.ts`: `< 8192` → `<= 8192` in `memoryAdaptiveCacheLimit()` and `memoryAdaptiveCacheBytesMb()` so 8GB systems get reduced frame cache limits.
- `render.ts`: Added `--protocol-timeout <ms>` and `--player-ready-timeout <ms>` CLI flags, wired through `resolveConfig` overrides.
- Updated calibration tests to match the new floor-not-ceiling behavior.
- Added fallow suppressions for pre-existing unused exports in `captureCost.ts`.
## Test plan
- [x] Engine config tests pass (`vitest run src/config.test.ts`)
- [x] Browser manager tests pass (`vitest run src/services/browserManager.test.ts`)
- [x] Calibration safeguard tests pass (4/4 in `renderOrchestrator.test.ts`)
- [x] TypeScript compiles cleanly for engine and cli packages
- [ ] CI pipeline
* feat(core): GSAP keyframe parsing, mutations, and API routes
* feat(core): spring physics solver + runtime fixes + spring ease editor
* feat(core): spring physics solver + runtime fixes + spring ease editor
Revert totalTime nudge that caused black first frames in from() tweens.
Keep stale CSS offset cleanup. Regenerate baselines for offset cleanup.
* ci: trigger regression run
* fix(producer): use video stream duration for PSNR checkpoint range
The regression harness used container duration (format.duration) to
compute PSNR checkpoints. Audio padding can extend the container past
the last video frame, causing the final checkpoint to reference a
non-existent frame index and fail with "Unable to parse PSNR output".
Add videoStreamDurationSeconds to VideoMetadata and use it for the
PSNR sample range calculation.
* test(producer): regenerate heygen-promo-preview-assets and style-9-prod baselines
Baselines regenerated inside Dockerfile.test on the devbox to match
the current runtime init.ts changes. Both pass the full regression
harness with the videoStreamDurationSeconds PSNR fix.
* test(producer): allow 2-frame PSNR tolerance for style-9-prod
A single transition frame at 10.742s renders with marginal PSNR
(26.6 dB vs 30 threshold) on CI runners but passes on the devbox
Docker image. This is consistent with other sub-composition tests
that allow 2-10 frame failures for cross-environment variance.
* fix(cli): reject directory --composition and add --browser-timeout (#1199)
Two unrelated symptoms from issue #1199, fixed together:
1. `--composition .` (or any directory path) used to slip past the
existsSync check in render.ts and explode downstream as
`EISDIR: illegal operation on a directory, read` when the producer
readFileSync'd the entry. The CLI now treats `.` / `""` as "omit
the flag" (falls back to index.html) and rejects other directory
paths with an actionable error pointing at the .html shape.
2. The 60s Puppeteer page.goto timeout in frameCapture.ts was hard-
coded, so heavy compositions (many videos / fonts / asset requests)
could not complete `domcontentloaded` in time. Add a configurable
`pageNavigationTimeout` to EngineConfig (default 60_000, env
fallback PRODUCER_PAGE_NAVIGATION_TIMEOUT_MS) and expose it as
`--browser-timeout <seconds>` on `hyperframes render`. The flag
threads through both renderLocal (via resolveConfig) and the
docker bridge (via buildDockerRunArgs).
Tests:
- render.test.ts: forwards/omits pageNavigationTimeout into resolveConfig
- dockerRunArgs.test.ts: forwards/omits --browser-timeout (seconds)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): address PR #1200 review — extract validators, tighten bounds
Addresses Vai's blockers and Miguel's nits on PR #1200:
- Vai blocker 1 (fallow CRAP) + blocker 3 (no argv tests):
Extract --browser-timeout and --composition validators into pure
helpers in utils/renderArgs.ts with a structured-result discriminant.
Drops ~45 lines of inline validation from run(), reducing its CRAP
score 1290→978 and cyclomatic 75→65. 19 new unit tests cover the
parse branches (sub-ms, overflow, NaN, Infinity, empty, negative,
".", "./", whitespace, directory, missing, ../escape, sibling-prefix).
- Vai blocker 2 (sub-ms → timeout:0 = "no timeout"): reject inputs
that round to <1 ms. Puppeteer treats page.goto({timeout:0}) as
wait-forever, so --browser-timeout 0.0004 silently flipped the
semantics. Now rejected with an explicit "rounds to 0 ms" error.
- Vai important 5 (1e10 accepted → setTimeout overflow): cap at
86_400s (24h). Above Node's TIMEOUT_MAX ≈ 2^31-1 ms setTimeout
fires immediately, the opposite of "long timeout."
- Vai important 4 (related timeouts unmentioned): CLI help and docs
now flag PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS and the 45s
playerReadyTimeout as the other knobs heavy compositions may need.
- Vai nit 7 (s/ms unit mismatch): help text and docs row both call
out the SECONDS-vs-MILLISECONDS difference between flag and env.
- Vai nit 8 / Miguel nit (composition flag discoverability): the
--composition description now says "Pass `.` (or omit the flag)
to render the project's index.html."
- Miguel nit (dead branch): the entryFile === "" unreachable branch
is gone. New helper uses `if (!trimmed || trimmed === ".")`.
Also adds a trailing-separator guard on the project-containment check
(sibling-prefix bypass: /proj-evil/x.html no longer slips past
startsWith('/proj')) — flagged by the code review.
The three remaining fallow complexity findings on render.ts (run,
renderDocker, trackRenderMetrics) are inherited from main; this PR
reduces run() but does not refactor it. Suppressed with
fallow-ignore-next-line markers and inline rationale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): diverge --browser-timeout error messages per Vai nit 5
The `not-a-number` and `not-positive` branches in browserTimeoutErrorMessage
shared the generic "Must be a positive number of seconds" message even
though the discriminant carried distinct kinds. Diverge them so users see
the specific failure mode:
--browser-timeout abc → "Got \"abc\", which is not a number."
--browser-timeout -5 → "Got \"-5\" seconds, which is not positive."
The shared hint ("pass a positive number of seconds, e.g. 180") is
preserved on both branches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(producer): localize remote <img> sources + await image readiness
Producer's frame-capture has `pollVideosReady` (waits readyState >= 2 for
every <video>) but no equivalent for <img>. Combined with htmlCompiler's
`collectExternalAssets` explicitly skipping http(s) URLs (line 805-806),
agent-pipeline-generated compositions (astral / daphne / hyperion
multi-v2 outputs with raw S3 <img src>) reach Chrome with a network
dependency that races the readiness gate AND can be evicted mid-render.
Either path produces blank-frame flicker.
Reproduction (02_kobe agent output, 42s render @ 30fps): scene_02's
remote S3 background-image painted from t=7.0s, vanished at t=10.5s
(frame size 139KB vs 700-940KB neighbors), back at t=11.0s. GSAP
timeline said opacity:1 throughout — Chrome simply didn't have the
pixels.
Two-layer fix:
1. **Producer** — `localizeRemoteImageSources` in `htmlCompiler.ts`
mirrors the existing `localizeRemoteMediaSources` (video/audio) +
`localizeRemoteFontFaces` pattern, reusing `downloadAndRewriteUrls`
and the `_remote_media/` subdir. Wired into `compileForRender`
between the media and font localize steps. Once the file is local,
Chrome's image cache is bounded by disk reads, not S3 latency.
2. **Engine** — `pollImagesReady` + `decodeAllImages` helpers in
`frameCapture.ts` parallel to `pollVideosReady`. Waits for every
`<img>` (skipping data: URIs) to have `complete && naturalWidth > 0`,
then forces GPU upload via `img.decode()`. Called from both the
classic-xvfb path and the BeginFrame path after their respective
video readiness checks. Defense-in-depth — Layer 1 closes the
symptom for current+future agent-pipeline outputs; Layer 2 protects
any future code path that leaves a remote URL in place.
Tests: 7 new cases in `htmlCompiler.test.ts` covering happy-path
rewrite, 404 fallback, dedup of duplicate URLs, non-HTTP and data:
URI passthrough, both quote styles, and the agent-pipeline shape where
`src` is not the first attribute. All pass alongside the existing 56
htmlCompiler tests.
* fix(producer): scope remote-img regex to real src; correct stale comments
Review follow-ups on the remote-<img> localization fix:
- Tighten REMOTE_IMG_TAG_RE with a (?<![\w-]) lookbehind so it matches a
real `src` attribute only. The previous `\bsrc` also matched `data-src`
(and `data-*-src`) lazy-loader placeholders, which would download/rewrite
a URL the render never paints. Added a regression test; `srcset` stays
excluded by the `\s*=` requirement.
- Fix comments that claimed frameCapture has "no pollImagesReady analog" —
this PR adds exactly that, so the docstrings were self-contradictory.
Reframed localization as the primary fix and pollImagesReady as the
defense-in-depth layer, and documented the <img src>-only scope
(srcset / <picture> / SVG <image> / CSS background-image are follow-ups).
Verified locally end-to-end on the 02_kobe repro: all 4 remote S3 <img>
URLs localize to _remote_media/, the render completes, and the frame at
t~10.5s that was a 139KB blank in the broken render now paints the trophy
background in every native-fps frame. htmlCompiler.test.ts 64 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(engine): pollImagesReady broken-image escape + skip decode on in-flight
Addresses two real bugs Magi caught in review on hf#1197:
1. pollImagesReady would spin the full pageReadyTimeout (45s default)
for any <img> that settled with an error — Chrome marks 404 / decode
failure / CORS rejection with (complete=true, naturalWidth=0), and
the previous predicate `complete && naturalWidth > 0` returned false
for those, so the poll ran to timeout. This is the HTMLImageElement
equivalent of pollVideosReady's `ve.error` early-exit. Add a
`complete && naturalWidth === 0` branch that treats settled-with-
error as done — waiting won't make it load. Particularly relevant
because localizeRemoteImageSources falls back to the original URL on
download failure; that failed URL is now hit by a 45s stall instead
of the broken-image marker rendering immediately.
2. decodeAllImages called img.decode() on every image, including those
still in flight after pollImagesReady timed out. Per the WHATWG spec,
decode() on a loading image awaits the fetch — never resolving
until the network completes or puppeteer's evaluate timeout fires
and throws an uncaught error that aborts the render. Pre-filter to
only call decode() on images that successfully loaded.
Test coverage: new frameCapture-pollImagesReady.test.ts with 8 cases
covering empty docs, all-loaded, broken (complete + naturalWidth=0),
data: URI, empty src, in-flight → resolves, in-flight → timeout, and
the mixed batch. The broken-image test explicitly asserts elapsed <
500ms on a 1000ms timeout — guards against the regression Magi flagged.
* docs(engine): clarify decodeAllImages prevents init race, not eviction
Vai correctly noted that decode() forces initial GPU upload but does not
prevent Chrome from evicting decoded pixels mid-render. The producer-side
localizeRemoteImageSources is what bounds the eviction risk (local
file-server paging vs S3 re-fetch). Comment updated to reflect that split
of responsibilities.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(engine): fast-fail on zero duration instead of 45s timeout
When a composition's runtime finishes initializing but reports zero
duration (no GSAP timeline and no data-duration attribute), the engine
previously polled for the full 45-second timeout before failing.
Now, after 10 seconds of polling, the engine checks whether the runtime
has finished (window.__renderReady === true) with a working seek
function but zero duration. If so, it fails immediately with a
diagnostic message explaining what's wrong and how to fix it.
This also improves the generic timeout error message to include runtime
state (whether __player exists, __hf.seek, GSAP timelines, declared
duration) so users can self-diagnose.
PostHog data: 555-1,234 occurrences/day, each wasting 45s of user time.
* fix(engine): throttle diagnostic polls and tighten zero-duration fast-fail
Two nit fixes in pollHfReady:
1. Throttle evaluateHfDiagnostic calls to once per ~1000ms after the 10s
mark. Previously called on every 100ms loop tick, generating ~350 CDP
round-trips per failed render. One check per second is sufficient to
detect a permanently-zero composition.
2. Change fast-fail condition from 'duration === 0' to
'!hasTimeline && declaredDuration <= 0'. A composition with a GSAP
timeline but no data-duration attribute should not be fast-failed —
GSAP sets duration synchronously before __renderReady via __timelines,
so a non-empty __timelines is a reliable signal that duration will
eventually be non-zero. Only compositions with NEITHER a GSAP timeline
NOR a declared duration are permanently zero.
GSAP's volume tween on an <audio> element causes Chrome to construct an
AudioContext. In headless Chrome, the autoplay policy blocks AudioContext
startup with "The AudioContext was not allowed to start" — the frame-capture
loop then waits for it indefinitely and deadlocks before the BeginFrame
fallback can recover. The render hangs at "Starting frame capture" with
0 output frames and times out.
Adding --autoplay-policy=no-user-gesture-required lets the AudioContext start
without a user gesture, which is safe in the headless rendering context where
no real user interaction is possible anyway.
Applied to both the main Chrome launch (browserManager) and the HDR capture
path (hdrCapture).
Fixes#1176. Reported by Abhai (Infinity agent, external).
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(producer): localize remote @font-face src URLs before render
Remote font URLs in @font-face blocks fail with a CORS rejection when
the renderer fetches them from http://localhost:PORT (S3 does not echo
the local origin in Access-Control-Allow-Origin). Chrome falls back to
the next font in the stack (e.g. Arial), producing wrong typography.
localizeRemoteFontFaces() scans <style> blocks, extracts HTTP url()
references inside @font-face rules, downloads them in parallel into
_remote_media/, and rewrites the CSS url() references to local paths —
the same pattern as localizeRemoteMediaSources() for <video>/<audio>.
Background url() references outside @font-face blocks are intentionally
left untouched to avoid downloading arbitrary images.
The shared download+rewrite logic is extracted into downloadAndRewriteUrls()
to eliminate duplication between the two localize functions.
Reported via the Beasty Style caption template (Komika Axis .ttf from S3
falling back to Arial on every cloud render).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(engine): add SSRF guard to downloadToTemp (blocks private/IMDS addresses)
Customer-supplied compositions can author @font-face src URLs (and <video>/
<audio> src attrs via the existing localize path) that point to private
infrastructure. Without a guard, the producer's downloadToTemp would fetch
http://169.254.169.254/... (AWS IMDS), RFC1918, loopback, etc., save the
response to _remote_media/, and expose it via the local file server.
assertPublicHttpsUrl() rejects:
- Non-HTTPS (http://) — all composition fetches must use HTTPS
- 169.254.x (AWS link-local / IMDS)
- 127.x / localhost / 0.x (loopback / unspecified)
- 10.x, 172.16–172.31, 192.168.x (RFC1918)
- [::1], [fc...], [fd...] (IPv6 loopback + unique-local)
The guard fires before the cache check so a blocked URL never gets into
the in-flight map. Applies to both the font-face localize path (PR #1155)
and the existing video/audio localize path (PR #1146) since both call
downloadToTemp.
Note: DNS-rebinding bypasses are not closed by this check (hostname
comparison only, no DNS resolution). Acceptable risk for current threat
model; server-side DNS validation can be layered on later.
12 unit tests covering all blocked ranges + the allowed edge cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(engine): fix TypeScript strict-mode error in urlDownloader SSRF guard
m[1] from RegExp.match() is typed string | undefined; parseInt requires string.
Use nullish coalescing to satisfy tsc without changing runtime behavior —
the regex guarantees m[1] is always defined when the match succeeds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(engine): use vitest import in urlDownloader test
bun:test is not available in CI — the engine package runs tests via vitest.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
processCompositionAudio prepares all tracks in parallel (Promise.all),
so for N tracks the mix call lands at index N, not index 1. The 3-track
test was reading calls[1] (the second prepare call) instead of calls[3]
(the mix call), causing indexOf("-filter_complex") to return -1 and the
subsequent assertions to read the wrong args.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(engine): remove amix normalize=0 to fix audio on FFmpeg 4.x/6.x
amix's normalize=0 option is absent from many FFmpeg builds (e.g.
FFmpeg 4.2 on Ubuntu 20.04). When the option is not recognized, FFmpeg
fails the entire filter graph initialization, processCompositionAudio
returns success:false, and the assembled video has no audio stream.
Replace normalize=0 + weights='1...' with the amix default behavior
(normalize=true, divides by track count) and multiply the master output
gain by the track count to restore the original per-track volumes.
The net volume is identical across all FFmpeg versions.
Fixes #1136-adjacent: reported as 'audio doesn't play' in rendered MP4.
* fix(producer): strip img crossorigin + fix audioExtractor normalize=0
Two follow-up fixes:
1. htmlCompiler: strip crossorigin attribute from <img> elements during
compilation. External images (e.g. S3) with crossorigin='anonymous'
force CORS-mode requests against the renderer's localhost file server,
which S3 rejects → images render blank. Matches the existing video
strip at line 261.
2. audioExtractor: same amix normalize=0 bug as audioMixer.ts. The
audioExtractor path is used for <video data-has-audio='true'> mixing
in the CLI's local render pipeline; on FFmpeg 4.x it would also drop
audio silently. Fix: remove normalize=0, compensate with volume=N.
* test(engine,producer): pin amix normalize contract + img crossorigin strip
- audioMixer.test.ts: assert filter has no normalize=/weights=; add
3-track test confirming compensatedGain = masterGain × N = 3
- htmlCompiler.test.ts: parallel tests for img and video crossorigin
strip (covers both elements, not just video)
Two perf fixes caught in #1118 review:
1. Cache guard: probeAndCacheVolumeKeyframes now short-circuits when
the element is already in volumeKeyframeCache. Without the guard
every bindMediaMetadataListeners call (every 30 RAF ticks) re-probed
all bound elements — N elements × full-composition timeline seeks at
60 Hz regardless of whether keyframes were already known.
bindRootTimelineIfAvailable still clears the cache on a new timeline
capture so keyframes stay fresh when the composition is rebound.
2. PCM cursor: audioVolumeEnvelope.ts had the incremental segment
cursor (O(N+M) overall) before #1118 extracted the interpolation into
interpolateVolumeGain. The shared function restarts from segment=0 on
each call — fine for the preview path (one call per RAF tick) but
O(N×M) for the PCM path (one call per sample: 48 kHz × duration).
Napkin math: a 10-min render went from ~30M to ~460M ops. Restored
the inline incremental scan in the engine bake loop; engine now only
imports normaliseEnvelope from core.
Preview audio with GSAP volume fades (e.g. data-volume="0" with a
gsap.to("#bgm", {volume:0.25, ...})) played ~1s then silenced. Root
cause: syncRuntimeMedia used fallbackAuthorVolume (data-volume) on the
first tick after a clip became active, clobbering the GSAP-seeked value.
The single-clock transport seeks GSAP before syncRuntimeMedia runs, so
el.volume already holds the animated value — we just need to trust it.
Fix — three layers, matching the renderer's approach (PR #1117):
1. First-tick tracking: on the first tick a clip is active
(previousRuntimeVolume===undefined), use currentElementVolume (GSAP's
seeked value) instead of fallbackAuthorVolume. In production the
transport always seeks GSAP before syncRuntimeMedia, so el.volume is
already at the correct animated position.
2. Probed keyframes: new probeElementVolumeKeyframes() runs the same
offline probe the renderer uses (discoverAudioVolumeAutomationFromTimeline)
directly in the browser. init.ts calls probeAndCacheElementVolume() when
an element is bound and a timeline is available. When keyframes are present,
syncRuntimeMedia drives volume from the interpolated envelope — no
GSAP-change tracking needed, no first-tick edge case, same data source
as the renderer.
3. Shared utilities: normaliseEnvelope(), interpolateVolumeGain(), and
probeAndCacheElementVolume() extracted to mediaVolumeEnvelope.ts and
exported from @hyperframes/core/media-volume-envelope. The engine's
audioVolumeEnvelope.ts imports from there — no duplicate logic between
the renderer and the new preview path.
Fallow audit exits non-zero on inherited complexity/duplication in init.ts
functions that shifted line numbers (applyClipLayout, transportTick, etc.),
unchanged by this PR — same known false-positive pattern noted in #1117.
Lint, format, typecheck, and unit tests all pass.
53 core/media tests pass (3 updated to pre-set el.volume to match the
runtime's bindMediaMetadataListeners — corrects a missing setup step).
audioVolumeEnvelope tests (6) still pass.
Miguel Ángel