Three root-cause fixes for the lazy media preloading feature:
1. Untimed media orphaned at preload="metadata": the else branch in
bindMediaMetadataListeners demoted ALL media elements, but the
mediaPreloader only manages timed clips (data-start). Untimed media
(background audio, ambient loops) got stuck at metadata forever.
Now only timed elements are demoted.
2. Monotonic promotion with no eviction: once promoted, clips stayed
at preload="auto" forever. Scrubbing through the full timeline
promoted everything, bringing back the OOM crash. Added LRU eviction
with MAX_PROMOTED=5 — when clips leave the preload window, their src
is cleared and load() called to release buffered data per MDN. On
re-entry, the original src is restored.
3. Metadata preload without load(): setting preload="metadata" alone
doesn't guarantee the metadata fetch in Chrome Lite mode or Firefox
with media.preload.default=0. Now load() is called after demotion
to ensure el.duration is populated for timeline computation.
Also adds exact-boundary tests for LAZY_THRESHOLD=6 and eviction
coverage (evict on scrub, src restoration, MAX_PROMOTED cap, load()
called on eviction).
Wire the MediaPreloadManager into init.ts:
- Detect render mode via __HF_EXPORT_RENDER_SEEK_CONFIG (keeps eager preload)
- Gate bindMediaMetadataListeners: lazy mode sets preload="metadata",
eager mode keeps preload="auto" (unchanged for small compositions)
- Advance preload window in the timeline poll tick loop
- Call preloadAroundTime on seek for instant buffering at seek target
Studio Player.tsx: hasUnloadedAssets now skips elements with
preload!="auto" so deferred clips don't block the loading overlay.
Compositions with many large video files (e.g., 6GB across 20 clips) crash
the browser because the runtime eagerly sets preload="auto" + .load() on
every media element at startup. All files buffer simultaneously, exhausting
memory.
Add a MediaPreloadManager that gates preloading based on playhead position:
- Activates when a composition has ≥6 timed media elements
- Only preloads clips within a 10-second lookahead window (or next 2 clips)
- Far-away clips stay at preload="metadata" (resolves duration without
downloading data)
- Advances the window on each transport tick and immediately on seek
- Render mode (window.__HF_EXPORT_RENDER_SEEK_CONFIG) keeps eager preload
for deterministic frame capture
- Small compositions (<6 clips) keep eager preload — no behavior change
Studio's hasUnloadedAssets now skips elements with preload!="auto", so
deferred clips don't block the loading overlay.
Portrait compositions (1080x1920) rendered pillarboxed inside a fixed 80x45px
landscape container, wasting space with black bars on both sides.
The thumbnail container now derives its dimensions from the composition's
stage size: landscape gets 80px wide, portrait gets 45px tall. The preview
scale calculation uses matching card dimensions so the iframe fills the
container without letterboxing.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three small follow-ups on the shared.ts extraction, addressing review
feedback on #720:
- Hoist `BROWSER_MEDIA_EPSILON` from `probeStage.ts` into `shared.ts`
so any future stage that reconciles browser media (chunked rendering
re-probe, for instance) doesn't have to redeclare it.
- Migrate `renderOrchestrator.test.ts` to import the five moved symbols
(`applyRenderModeHints`, `projectBrowserEndToCompositionTimeline`,
`resolveDeviceScaleFactor`, `writeCompiledArtifacts`,
`CompositionMetadata`) directly from `./render/shared.js`. This is
the clean end state — the back-compat re-export through
`renderOrchestrator.ts` was a stepping-stone.
- Drop the back-compat re-export block from `renderOrchestrator.ts`.
No remaining importers go through it (verified via grep across
`packages/`). The five symbols now have exactly one path:
`./render/shared.js`.
No behavior change. Renderer smoke-tested inside `Dockerfile.test`
against `font-variant-numeric`, `many-cuts`, and `variables-prod` —
audio correlations 1.000 / 0.994 / 0.975, matching every prior PR in
the stack.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Breaks the runtime circular import between `renderOrchestrator.ts` and
the stage files under `services/render/stages/`. Before this change the
stages imported runtime helpers (`writeCompiledArtifacts`,
`applyRenderModeHints`, `resolveDeviceScaleFactor`,
`projectBrowserEndToCompositionTimeline`) and types (`CompositionMetadata`)
back from `renderOrchestrator.ts`, which itself imports the stage
functions. The cycle resolved at build time because both modules
finished initializing before any stage was invoked, but it was fragile
and would keep growing as more stages were extracted.
This PR:
- Adds `packages/producer/src/services/render/shared.ts` and moves the
four functions plus the `CompositionMetadata` interface into it.
- Has `renderOrchestrator.ts` re-export everything from `shared.ts`, so
external callers (the existing `renderOrchestrator.test.ts`, any code
importing `applyRenderModeHints` etc. from the orchestrator) keep
working with no churn on their side.
- Updates `compileStage.ts` and `probeStage.ts` to import the runtime
helpers from `../shared.js`. The only remaining import from
`renderOrchestrator.ts` in the stages is `import type { RenderJob }`,
which is erased at runtime and creates no cycle.
- Removes the imports the orchestrator no longer needs after losing
the four function definitions (`CANVAS_DIMENSIONS`, `VideoElement`,
`AudioElement`, `ImageElement`).
No behavior change. Renderer smoke-tested inside `Dockerfile.test`
against `font-variant-numeric`, `many-cuts`, and `variables-prod` —
all PSNR / audio-correlation baselines match PR 1.3 exactly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The probe stage previously assigned `job.duration` and `job.totalFrames`
inside its body AND the sequencer re-asserted them after the call to
restore TS narrowing. Two writers for the same field is a maintenance
hazard — a future refactor could drop one and create a silent skew.
Move ownership: the stage computes `duration` and `totalFrames` and
returns them; the sequencer is the sole writer onto the `RenderJob`.
This also aligns with the eventual chunk-worker model where a chunk
running in a separate process cannot mutate the orchestrator's `job`.
No observable behavior change. `job.duration` / `job.totalFrames` end
up with the same values; the zero-duration `throw` still happens
inside the stage (now using the local `duration` constant) before any
sequencer-side assignment. Verified by:
- `bun run --filter @hyperframes/producer typecheck` clean
- `bun test packages/producer/src/services/` 175 pass / 1 pre-existing
unrelated failure on `main`
Review feedback addressed: vanceingalls on #719.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment-only cleanup. Removes "PR 1.x", "Phase 1 PR", and "Phase 3 PR 3.1"
references from JSDoc blocks in `compileStage.ts`, `probeStage.ts`,
`planHash.ts`, and `freezePlan.ts`. Track / PR identifiers rot quickly and
belong in PR descriptions, not in source. Design-doc section citations
(DISTRIBUTED-RENDERING-PLAN.md §X.Y) are kept — those reference a stable
external artifact.
Also tightens the `probeStage.ts` `browserProbeMs` doc string to say
"near-zero when `needsBrowser` was false" instead of "0" — the Date.now()
delta around the function body is sub-ms but not literally zero.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the browser probe / duration discovery / recompile / media
reconciliation block out of `executeRenderJob` into
`services/render/stages/probeStage.ts`. No behavior change. The sequencer
calls `runProbeStage` at the same code point with identical inputs and
outputs.
The probe stage owns the `FileServerHandle` and the `CaptureSession` it
creates and returns them to the sequencer. The sequencer still tracks
them in its `let fileServer` / `let probeSession` bindings and closes
them in its `finally` block — the resource lifetime is unchanged.
`recompileWithResolutions` lives inside this stage because it depends on
browser-resolved durations even though §2.1 of the distributed plan
lists recompile as a sibling phase.
Preserved invariants:
- `composition` is mutated in place (videos / audios / duration) so
downstream stages see the reconciled view through the same reference.
- `job.duration` and `job.totalFrames` end up with the same values at
the same code points. The result type carries `duration: number`
alongside `totalFrames: number`, and the sequencer re-asserts the
assignments after the call so TypeScript's control-flow narrowing
works for the rest of `executeRenderJob`.
- `perfStages.browserProbeMs` and `perfStages.compileMs` are written at
the same code points with the same values.
- The "Composition duration is 0" diagnostic builds the same hint string
from the same console-buffer regex and `__timelines` probe.
- The post-probe "failed network requests" warning fires with the same
regex, the same first-10/first-5 slicing, and the same `console.warn`
prefix.
Renderer smoke-tested inside `Dockerfile.test` against `font-variant-numeric`,
`many-cuts`, and `variables-prod` — all PSNR / audio correlation baselines
match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a `TODO(distributed-render):` comment near the `applyRenderModeHints`
call documenting that this caller-owned-object mutation needs to move
into the result type before `freezePlan` wires up. The mutation pattern
works in-process but won't survive across processes / replays from a
frozen plan — the value belongs in `LockedRenderConfig`, not on a
mutated `EngineConfig`.
No behavior change. Comment-only.
Review feedback addressed: vanceingalls on #718.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the pure compile sub-stage (`compileForRender` + `applyRenderModeHints`
+ `writeCompiledArtifacts` + `CompositionMetadata` build + DPR resolution)
out of `executeRenderJob` into `services/render/stages/compileStage.ts`.
No behavior change. The sequencer calls `runCompileStage` at the same code
point with identical inputs and outputs. The following invariants are
preserved verbatim:
- `cfg.forceScreenshot` is still mutated by `applyRenderModeHints`.
- `perfStages.compileOnlyMs` is set to the same wall-clock interval (around
the `compileForRender` call only).
- The "Compiled composition metadata" log line is emitted after artifact
writes with the same payload shape.
- The "Supersampling composition via deviceScaleFactor" log line is emitted
only when `deviceScaleFactor > 1`.
- `stage1Start`, `updateJobStatus(..., "Compiling composition", 5, ...)`,
and `perfStages.compileMs` (set at the end of probe) remain at their
current code points in the sequencer.
The probe sub-stage (`if (needsBrowser)`) is unchanged — it is extracted
separately in PR 1.3. `recompileWithResolutions` lives inside the probe
block because it depends on browser-resolved durations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add `PLAN_HASH_SCHEMA_PREFIX = "hyperframes-plan-hash-v1\x00"` mixed
into every digest. Future framing changes must bump the trailing
integer; this makes cross-version mismatches visible at the wire
format instead of producing silent collisions. Impossible to backfill
later, easy to bake in now.
- Hoist the `0x00` field delimiter to module scope (`FIELD_DELIMITER`).
- Document the UTF-8 encoding contract for all string-typed input
fields in the file-level JSDoc, so external verifiers know the
encoding without reading the implementation.
- Add a known-digest test for one fixed reference input
(`995b4105...`). If the framing changes silently this test fails,
forcing the developer to also bump the schema prefix.
- Add an explicit `canonicalJsonStringify(undefined) → TypeError` test
to pin the contract.
No callers yet, so no behavior change in any code path.
Review feedback addressed: vanceingalls on #717.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seed PR for the Phase 1 staged refactor of `executeRenderJob`. Creates the
new `packages/producer/src/services/render/stages/` directory and adds two
files that subsequent stage-extraction PRs build on:
- `planHash.ts`: a content-addressed sha256 helper plus a canonical-JSON
serializer, with unit tests covering determinism, asset-order
independence, sensitivity to each hashed component, and a delimiter-
framing test against path/sha boundary collisions.
- `freezePlan.ts`: signature-only skeleton (throws "not implemented") for
the eventual plan-freeze step. No callers; the body lands later when the
distributed-render primitives compose the Phase 1 stages.
Zero behavior change. No code in `executeRenderJob` is touched and no
existing exports move. The new files are not yet referenced anywhere
outside the `stages/` directory.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four existing presets only cover 16:9 (landscape) and 9:16 (portrait)
aspect ratios. A 1080×1080 square comp had nowhere to land at any scale:
"Auto" rendered at the comp's authored 1080×1080, and picking 1080p or 4K
mapped to a landscape/portrait preset whose aspect ratio mismatched, which
the producer's resolveDeviceScaleFactor validator rejects with
"does not match the aspect ratio of the composition".
Add `square` (1080×1080) and `square-4k` (2160×2160) to CANVAS_DIMENSIONS
in core. The existing `keyof typeof CANVAS_DIMENSIONS` derivation
extends the `CanvasResolution` union and `VALID_CANVAS_RESOLUTIONS` array
automatically, so the producer's validator, the render API route, and
the CLI `--resolution` flag pick the new presets up without further
changes.
- core: extend CANVAS_DIMENSIONS, RESOLUTION_ALIASES, and the
htmlParser to recognize `data-resolution="square|square-4k"` and to
infer square from equal width/height (vs. the prior "square defaults
to portrait" tie-breaker).
- studio: extend the local ResolutionPreset / CANVAS_DIMENSIONS mirrors;
collapse isPortraitComp into a 3-way `compAspect` helper so
resolveResolution returns the square preset for square comps.
- cli: update --resolution help text on `init` and `render` to mention
the new presets.
- tests: add square cases to renderOrchestrator's resolveDeviceScaleFactor
suite (returns 1 for square→square, 2 for square→square-4k, rejects
landscape preset on square comp), update the htmlParser test that
previously pinned the "square→portrait" tiebreaker.
Cleanup from the /simplify pass on PR #715.
- App.tsx: subscribe to the runtime's `stage-size` message (which
carries authoritative width/height post-applyCompositionSizing)
instead of re-parsing data-width/data-height from the iframe DOM.
Drops the cross-origin try/catch, querySelector, and parseInt logic,
and fires once per comp load instead of on every state/timeline tick.
- App.tsx: import CompositionDimensions from RenderQueue instead of
inlining the shape.
- RenderQueue.tsx: replace scaleLabel() with a SCALE_LABEL record,
inline the one-call formatDims helper, and trim the type comment to
the WHY.
Two bugs in getSharedBrowser() could take down the entire Vite dev
server:
1. Unhandled rejection from puppeteer.launch() — the timeout error
surfaces through puppeteer's internal RxJS chain, and any uncaught
path crashes the Node process. The thumbnail route's try/catch
doesn't always intercept it.
2. _browserLaunchPromise was never reset on failure, so subsequent
thumbnail requests reused a stale rejected promise instead of
retrying.
Wrap the IIFE in try/catch, return null on any failure (the thumbnail
route already handles a null adapter result with a 500), and reset
_browserLaunchPromise in a finally block so a transient launch failure
doesn't poison the singleton. Also drop the launch timeout from
puppeteer's 30s default to 10s so a wedged handshake fails fast instead
of stalling every pending thumbnail.
Verified locally: the dev server now logs
"[Studio] puppeteer launch failed — thumbnails disabled: ..." and
keeps serving the studio UI after a thumbnail request fails.
Orientation is a property of the composition, not a user choice — the
backend's portrait/landscape presets are tied to the comp's authored
aspect ratio. Letting users pick "1080p portrait" for a landscape
composition just produces a wrong-aspect render.
The dropdown now exposes three scale choices (Auto / 1080p / 4K) and
maps to the correct portrait/landscape preset based on the active
composition's data-width / data-height. Native <select title> tooltips
are unreliable across browsers, so the resolved dimensions render
inline in each option label (e.g. "1080p · 1920×1080") — always
visible, no hover needed.
App.tsx tracks the active comp's dimensions by listening for the
existing hf-preview state/timeline postMessages (same source the
caption-detection logic uses) and passes them to RenderQueue. The
useRenderQueue / backend contract is unchanged: RenderQueue still emits
"landscape" | "portrait" | "landscape-4k" | "portrait-4k" | "auto".
- Hoist duplicated test mock helpers (createMockAudioContext / setupTransport /
mockBuffer / mockEl) from the two describe blocks to module scope.
- Drop redundant math-derivation comments in schedulePlayback; the dedicated
rate-aware tests are the canonical proof.
- Tighten setRate JSDoc.
- Add no-op guard in setRate when the new rate equals the current rate, so a
duplicate set-playback-rate postMessage doesn't re-anchor or walk active
sources for nothing.
- Add a regression test for the no-op guard, and strengthen the clamp test
to schedule at rate=2 first so the clamp-to-1 assertion is non-vacuous.
WebAudioTransport scheduled AudioBufferSourceNodes with the implicit
default playbackRate of 1, so non-1x transport rates desynced visuals
from audio: GSAP timelines, the transport clock, and native <video>
all sped up while WebAudio-routed <audio> clips kept playing at 1x.
- schedulePlayback now accepts a rate, sets sourceNode.playbackRate,
and scales the future-clip start delay by the rate (the in-progress
buffer offset stays elapsed + mediaStart, which is rate-independent).
- New setRate() updates active sources in place and rebases the
getTime() reference frame so the audio-master clock stays continuous
across mid-playback rate changes.
- Runtime onSetPlaybackRate now forwards into webAudio.setRate, and
player.play() schedules each clip with state.playbackRate.
Fixes#713
The property delegation on window.__player used Object.defineProperty
with only a getter, causing "Cannot set property renderSeek which has
only a getter" when Studio's motion-wrapping code tried to reassign
__player.renderSeek with a wrapped version. This cascaded into an
infinite error loop making the timeline unusable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The postMessage state path set `_currentTime` without clamping, while the
direct timeline path already used `Math.min(currentTime, _duration)`. A
final-frame state message with a frame count slightly past the end would
set `_currentTime > _duration`, causing the progress bar (position:
absolute, no overflow guard) to bleed out of the scrubber track and
visually cover the volume button, and the time display to show values
like "0:05 / 0:04".
- Clamp `_currentTime` in `_onMessage` to match the direct timeline path
- Clamp defensively in `updateTime` so the display layer never overflows
- Add `overflow: hidden` + `min-width: 0` to `.hfp-scrubber` as a CSS
safety net; remove now-redundant `border-radius` from `.hfp-progress`
(parent `overflow: hidden` handles clipping to the rounded shape)
- Apply the same `overflow: hidden` fix to `.hfp-volume-slider` for
consistency; remove redundant `border-radius` from `.hfp-volume-fill`
- Add regression test covering the postMessage over-duration case
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the rigid `--fps 24|30|60` whitelist with a numeric range and
adds support for ffmpeg-style fractional framerates so NTSC stays exact
end-to-end.
- `--fps 30` keeps working (integer fps)
- `--fps 30000/1001` now means exact NTSC 29.97 (not the lossy decimal)
- `--fps 24000/1001`, `--fps 60000/1001`, `--fps 25/50/120/240` all work
- Decimals like `--fps 29.97` are rejected with a friendly error pointing
the user at the rational form, since `29.97` and `30000/1001` round
to different framerates inside ffmpeg
Carries an `Fps = { num: number; den: number }` rational end-to-end:
RenderConfig, EncoderOptions, StreamingEncoderOptions, CaptureOptions,
DockerRenderOptions, Studio API request body, regression-harness
meta.json. The `-r` and `-framerate` ffmpeg args emit the rational form
verbatim (`30000/1001`) so no decimal round-trip happens at the encoder
boundary. Frame-interval math uses `1000 * den / num` ms (33.366… for
NTSC, 33.333… for integer 30).
Helpers live in @hyperframes/core:
- `parseFps(input: string | number): FpsParseResult` — discriminated
parser used by both the CLI and the Studio API route
- `fpsToFfmpegArg(fps: Fps): string` — emits "30" or "30000/1001"
- `fpsToNumber(fps: Fps): number` — for arithmetic (telemetry, frame
count, frame-index → time)
Studio API wire format accepts polymorphic `fps: number | string`:
- number → integer fps (`30`)
- string → rational (`"30000/1001"`)
Decimals are rejected; matches the same rule as the CLI.
Existing meta.json fixtures with integer `"fps": 30` continue to load
unchanged — the regression-harness validator now normalizes both number
and string inputs through `parseFps`.
Clips whose compositionStart is ahead of the current timeline position
were starting immediately because sourceNode.start() always received
when=0. Use the AudioContext scheduling API to defer future clips:
sourceNode.start(ctx.currentTime + delay, mediaStart).
Closes#674
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the two-clock architecture (GSAP rAF ticker + HTMLMediaElement
pipeline reconciled by a 50ms polling loop) with a single TransportClock.
GSAP is always paused and seeked to clock.now() on each rAF tick.
Drift between visual timeline and audio is structurally impossible.
Architecture:
TransportClock.now() ──rAF──▶ timeline.seek(t) + el.currentTime
▲
AudioContext.currentTime (~21µs) ← WebAudio active
OR
audio.currentTime (~33ms) ← HTMLMediaElement fallback
OR
performance.now() (~1ms) ← no audio
Key changes:
- TransportClock class with monotonic + audio-master clock sources
- WebAudioTransport: routes audio through AudioBufferSourceNode for
sample-accurate scheduling, falls back gracefully to HTMLMediaElement
- rAF tick loop replaces 50ms setInterval poll; GSAP always paused
- Strict sync (40ms threshold, consecutive-sample gated) + forceSync
on play/pause/seek transitions for sub-frame media accuracy
- Buffer-stall: visuals freeze when audio is buffering instead of
running ahead
- Frame quantization preserved in seek/renderSeek (parity contract)
Browser-verified: 0.0ms drift after 40 pause/play cycles (was 400ms+).
Also fixes: CDN script HTML error responses in validate (pre-existing).
54 tests across clock, clock-drift, webAudioTransport, and media.
Closes#668
Miguel Ángel