mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-07 01:56:04 +00:00
c8a3e5fc4dbd57dee80e8a7d530b01bf7a55aa32
119
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 James
|
abc102e3d6 |
refactor(producer): finish thinning executeRenderJob
Move file-level helpers and inline blocks out of renderOrchestrator.ts into focused render/* modules. executeRenderJob shrinks from ~897 to ~675 lines; renderOrchestrator.ts from 2725 to ~2104. New under packages/producer/src/services/render/: - hdrPerf.ts: HdrPerfCollector + helpers - captureCost.ts: capture-cost + calibration helpers, plus a new runCaptureCalibration helper that owns the BeginFrame->screenshot fallback - hdrMode.ts: resolveEffectiveHdrMode - perfSummary.ts: buildRenderPerfSummary - cleanup.ts: safeCleanup, cleanupRenderResources, buildRenderErrorDetails shared.ts adds createCompiledFrameSrcResolver, materializeExtractedFramesForCompiledDir, createMemorySampler. Moved symbols are re-exported from renderOrchestrator.ts for backwards compatibility; tests update to import from the new paths. No behavior change: producer smoke set is PSNR-identical to main inside Dockerfile.test. lefthook.yml: belt-and-suspenders fix so the filesize hook actually skips .test.ts / .generated.ts files. The hook-level exclude regex does not filter the staged_files expansion inside the shell loop, so the loop now does its own check. |
||
|
James
|
e221cb8d5c |
refactor(producer): snapshot cfg.forceScreenshot at compile time, stop mutating mid-pipeline
Resolve the compileStage TODO from PR #720. cfg.forceScreenshot is now computed exactly once inside compileStage (after applyRenderModeHints) and returned on CompileStageResult.forceScreenshot. The sequencer stores it on a local captureForceScreenshot; downstream capture stages take the value as an explicit parameter and derive their own engine config rather than reading cfg.forceScreenshot. Mid-pipeline mutations removed: - renderOrchestrator.ts: the pre-compile alpha-output mutation moved into compileStage so the resolution is one operation in one place. - captureHdrStage.ts: stopped mutating caller-owned cfg; the layered composite path now uses a local hdrCfg derived from cfg plus forceScreenshot=true. The stage throws if called with forceScreenshot=false to make the contract explicit. - BeginFrame auto-worker calibration fallback: still flips capture mode on a timeout, but flips the local boolean instead of cfg. The screenshot-mode retry uses a derived cfg view. captureStage / captureStreamingStage add a forceScreenshot input and derive captureCfg (identity-equal to cfg when the values already agree, so no extra allocation on the common path). lefthook.yml: grandfather renderOrchestrator.ts and captureHdrStage.ts in the new 500-line filesize hook (#748). Both pre-date the hook and are actively being shrunk in the producer stages stack. Unblocks Phase 3 chunked rendering: LockedRenderConfig.forceScreenshot in the distributed plan is computed here and survives across processes without depending on shared mutable state. |
||
|
JamesandClaude Opus 4.7
|
d351843ab8 |
refactor(producer): move updateJobStatus to render/shared.ts
First of several focused PRs that flatten the runtime cycle between the capture stages and `renderOrchestrator.ts` (documented as a known follow-up across PRs 1.6 / 1.7 / 1.8 / 1.9). `updateJobStatus` was the most-imported orchestrator helper: 5 of the 6 capture / encode / assemble stages reach back into the orchestrator for it. Moving it to `render/shared.ts` (where the other small cross-cutting utilities already live) breaks the runtime cycle for five stages in one move: - captureStage - captureStreamingStage - captureHdrStage - encodeStage - assembleStage Each of those stages now imports `updateJobStatus` from `../shared.js` at runtime, and the only thing they pull from `renderOrchestrator.js` is type-only (`RenderJob`, `ProgressCallback`, etc.) — type imports are erased at runtime, so no cycle. The orchestrator's own internal call sites (`updateJobStatus(...)` for the inline progress updates and the `complete` / `failed` / `cancelled` transitions) are unchanged in body; they now import the function from the same shared module. Follow-up PRs will move: - `executeDiskCaptureWithAdaptiveRetry` + capture-retry helpers (breaks the captureStage cycle entirely) - The six HDR helpers + `resolveCompositeTransfer` (breaks captureHdrStage) - `collectVideoMetadataHints`, `collectVideoReadinessSkipIds`, `materializeExtractedFramesForCompiledDir` (breaks extractVideosStage) No behavior change. Verified inside `Dockerfile.test`: font-variant-numeric, many-cuts, gsap-letters-render-compat, hdr-regression — 4/4 PASS with identical audio correlations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
513775e659 |
refactor(producer): /simplify cleanup pass across stage modules
Comment + interface cleanup driven by the /simplify review. No code change beyond removing dead fields. - audioStage: drop unused `job: RenderJob` from `AudioStageInput` (the stage destructures it but never references the value). - encodeStage: drop unused `fps` + `useGpu` from `EncodeStageInput`; read both from `job.config.*` inside the stage (matches the pattern used by captureStage and captureStreamingStage). - captureStreamingStage: drop the unused `captureDurationMs` field from `CaptureStreamingStageResult` (sequencer never reads it — it uses its own `Date.now() - stage4Start` for `perfStages.captureMs`). Also drops the now-dead `streamStart` local. - captureStreamingStage: rewrite the "Known follow-up" header comment to drop the "PR 1.3.5" reference per `feedback_no_internal_track_names_in_source`. - captureHdrStage: drop the "Lifted verbatim from `executeRenderJob`" refactor-narration sentence in the header doc (the "Hard constraints preserved verbatim" list below it is real long-term documentation and stays). - Sequencer call sites updated to drop the now-removed fields. Verified inside `Dockerfile.test`: 4/4 fixtures pass with PSNR / audio correlations unchanged (font-variant-numeric, many-cuts, gsap-letters, hdr-regression). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
46954b7f57 |
refactor(producer): document executeRenderJob as a thin sequencer
Final polish PR of the Phase 1 stack. Comment-only — zero code change. After PRs #725, #726, #730, #731, #733, #734, the `executeRenderJob` function now composes eight stage modules instead of inlining the pipeline. Updates the file-level JSDoc to point at each stage module and explains the orchestrator's residual responsibilities: shared resource lifetime, perf counters, error diagnostics, and the `try/finally` cleanup. Adds JSDoc on `executeRenderJob` itself summarising what it returns and when it throws. The function body is unchanged. The line count dropped from ~2,200 (pre-Phase-1) to ~880; the remainder is in-sequencer setup that doesn't naturally compose into a stage (calibration, worker resolution, HDR auto-detection, preset selection, final perf-summary assembly) plus the orchestrator's `try/finally` resource ownership. Verified inside `Dockerfile.test`: font-variant-numeric (1.000), many-cuts (0.994), variables-prod (0.975), hdr-regression (1.000) — 4/4 PASS with audio correlations identical to every prior PR in the Phase 1 stack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
434539e99a |
refactor(producer): extract encodeStage and assembleStage
Move the final two stages of `executeRenderJob` into their own files:
- `services/render/stages/encodeStage.ts` (Stage 5): handles both the
png-sequence path (rename + copy + audio sidecar) and the encoded path
(`encodeFramesFromDir` or `encodeFramesChunkedConcat`).
- `services/render/stages/assembleStage.ts` (Stage 6): runs
`muxVideoWithAudio` when `hasAudio`, otherwise `applyFaststart`.
Skipped for png-sequence (sequencer gates the call).
Both stages are mechanical extractions of small, self-contained blocks.
The sequencer's call sites preserve the same conditions and the same
`perfStages.encodeMs` / `perfStages.assembleMs` assignments.
Hard constraints preserved verbatim:
- The `updateJobStatus` payloads ("Writing PNG sequence" / "Encoding
video" at 75%; "Assembling final video" at 90%) fire from inside the
stages at the same code points.
- The png-sequence "no PNGs were captured" error throws verbatim.
- The png-sequence audio sidecar is only written when
`hasAudio && existsSync(audioOutputPath)`.
- `enableChunkedEncode` selects `encodeFramesChunkedConcat` vs.
`encodeFramesFromDir` with the same args.
- The mux + faststart error messages (`Audio muxing failed: ...`,
`Faststart failed: ...`) throw verbatim on `success: false`.
Removes the now-orphaned imports from the orchestrator:
`encodeFramesFromDir`, `encodeFramesChunkedConcat`, `muxVideoWithAudio`,
`applyFaststart`.
Verified inside `Dockerfile.test`:
- font-variant-numeric (1.000), many-cuts (0.994),
sub-composition-video (0.947), gsap-letters-render-compat (1.000),
hdr-regression (1.000) — 5/5 PASS, audio correlations identical to
prior PRs in the stack. Exercises encoded mp4 + HDR (encode + assemble
both run) and the streaming-fusion path (encode skipped by sequencer).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
5e4641fb77 |
refactor(producer): extract captureHdrStage (HDR / shader-transition path)
Move the Z-ordered HDR / shader-transition layered composite branch (`if (useLayeredComposite)`) out of `executeRenderJob` into `services/render/stages/captureHdrStage.ts`. The largest extraction by LOC (~745 lines of body lifted verbatim) and the riskiest by cleanup invariants. Body is lifted byte-for-byte — only the surrounding scope changes. Cleanup invariants preserved verbatim (design doc §11 flagged these explicitly): - `hdrEncoderClosed` / `domSessionClosed` flags gate the defensive-close paths so they don't run twice when the success path already closed. - `hdrVideoFrameSources` is drained + cleared in the outer `finally` regardless of how the body exited. - `cfg.forceScreenshot = true` is set unconditionally inside the layered path because `captureAlphaPng` hangs under `--enable-begin-frame-control`. Other invariants preserved: - `hdrPerf` is created at the top of the stage and returned; the sequencer's `finalizeHdrPerf` consumes it for the perf summary. - The `Layered compositing frame N/M` `updateJobStatus` payload fires at the same per-frame point with `25 + frameProgress * 55`. - `composition` and `compiled` are read-only in the stage. - `hdrDiagnostics` is mutated in place (counters incremented at the same code points). - `nativeHdrIds` is recomputed inside the stage from `nativeHdrVideoIds` + `nativeHdrImageIds` (the sequencer's computation is unchanged; the stage just doesn't need it passed in). To support the extraction, the following symbols are newly exported from `renderOrchestrator.ts`: - Helper functions: `createHdrPerfCollector`, `addHdrTiming`, `closeHdrVideoFrameSource`, `blitHdrVideoLayer`, `blitHdrImageLayer`, `compositeHdrFrame`. - Types: `HdrPerfCollector`, `HdrPerfTimingKey`, `HdrVideoFrameSource`, `HdrImageBuffer`, `HdrCompositeContext`, `HdrTransitionMeta`, `TransitionRange`. These are internal helpers — the stage is currently the only consumer, and the cycle (orchestrator imports `runCaptureHdrStage`; stage imports helpers back) is safe at runtime. A future PR will consolidate the helpers into a shared module (same follow-up planned for the capture helpers in PRs 1.6 and 1.7). Removes the now-orphaned imports from the orchestrator: `openSync`, `fpsToFfmpegArg`, `spawnStreamingEncoder`, `StreamingEncoder` type, `runFfmpeg`, `initTransparentBackground`, `decodePngToRgb48le`, `queryElementStacking`, `TRANSITIONS`, `crossfade`, `resampleRgb48leObjectFit`, `normalizeObjectFit`, `TransitionFn` type, `createHdrImageTransferCache`. Verified inside `Dockerfile.test`: - **HDR fixtures (3/3 PASS)**: hdr-regression, hdr-hlg-regression, vignelli-stacking — audio correlations 1.000 / 1.000 / 0.982. - **Non-HDR fixtures (4/4 PASS)**: font-variant-numeric, many-cuts, sub-composition-video, gsap-letters-render-compat — audio correlations 1.000 / 0.994 / 0.947 / 1.000. - 7/7 fixtures total pass with PSNR / audio correlations matching every prior PR in the stack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
ba90b411ed |
refactor(producer): extract captureStreamingStage (single-machine fusion)
Move the streaming encode fusion path (`useStreamingEncode === true` with
successful encoder spawn) out of `executeRenderJob` into
`services/render/stages/captureStreamingStage.ts`. The stage owns:
- `spawnStreamingEncoder` invocation, including the abort-rethrow vs.
graceful-fallback handling.
- Parallel + sequential capture-to-stdin loops (Stage 4 absorbs Stage 5
for streaming renders).
- The streaming encoder's `close()` + result check.
- Defensive cleanup of the streaming encoder in the stage's own
`try/finally`.
The stage returns either `{ success: true, ... }` (sequencer skips the
disk path AND inline Stage 5) or `{ success: false }` (sequencer falls
back to the disk path). The sequencer's `useStreamingEncode` flag is
no longer flipped imperatively — the result type makes the branch
selection explicit.
Hard constraints preserved verbatim:
- `probeSession` is closed at the same code points (parallel: after
capture; sequential: in session finally). The local binding nulls
via the returned result.
- `lastBrowserConsole` is set to the buffer of whichever session was
active last (probe close path or sequential session finally).
- `job.framesRendered` is updated per-frame; `Streaming frame N/M
[(K workers)]` `updateJobStatus` payloads fire at the same 30-frame
and completion checkpoints (parallel) or every frame (sequential),
with the same percentage math `25 + frameProgress * 55`.
- `Streaming encode failed: <err>` still throws on the encoder's
`success: false` close result.
- The defensive `try/finally` close-on-throw is preserved, now inside
the stage instead of the orchestrator.
- `perfStages.captureMs` is still set by the sequencer from
`stage4Start`; the stage also returns `encodeMs` for the encoder's
overlapped duration (assigned to `perfStages.encodeMs`).
Removes the orphaned `createFrameReorderBuffer` and
`prepareCaptureSessionForReuse` imports from the orchestrator after
the streaming code moved.
Verified inside `Dockerfile.test`:
- 5/5 fixtures PASS (font-variant-numeric, many-cuts, variables-prod,
sub-composition-video, gsap-letters-render-compat).
- `gsap-letters-render-compat` (single-worker render, 4s duration)
exercises the new streaming stage end-to-end —
`streaming-encode gate enabled=true` confirmed in the log.
- The other 4 fixtures exercise the disk path (workerCount > 1).
Known follow-up: same runtime import cycle situation as captureStage —
the stage imports `updateJobStatus` and types from
`renderOrchestrator.ts`, which imports the stage back. Safe (deferred
to runtime); a future PR will flatten this once all 8 stages are
extracted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
d39df2de6c |
refactor(producer): extract captureStage (SDR disk path)
Move the SDR / DOM-only-HDR disk-capture body out of `executeRenderJob` into `services/render/stages/captureStage.ts`. Covers both branches of the disk path: parallel capture via `executeDiskCaptureWithAdaptiveRetry` (`workerCount > 1`) and sequential per-process capture (`workerCount === 1`, reusing `probeSession` when available). The HDR layered branch (`useLayeredComposite === true`) and the streaming encode fusion path (`useStreamingEncode === true` with successful encoder spawn) stay inline in the sequencer — they will be extracted by the next two PRs in the stack. Hard constraints preserved verbatim: - `probeSession` is closed (and the sequencer's `let probeSession` nulled via the returned result) at the same points. - `captureAttempts` is mutated in place — the parallel retry loop still pushes each attempt onto the array the sequencer owns. - `workerCount` reassignment from adaptive retry survives via the returned result. - `lastBrowserConsole` is set to the buffer of whichever session was active last (probe close path or sequential capture finally). - `job.framesRendered` is updated at the same per-frame / per-progress points; `Capturing frame N/M [(K workers)]` `updateJobStatus` payloads fire at the same 30-frame and completion checkpoints. - `perfStages.captureMs` is still computed by the sequencer from the outer `stage4Start` so its window covers both the in-sequencer setup (fileServer init, calibration, worker resolution, preset selection) AND the capture call. Two small new exports on `renderOrchestrator.ts`: - `executeDiskCaptureWithAdaptiveRetry` — was a private helper; the stage calls it directly. - `updateJobStatus` — was a private helper; the stage uses it for the per-frame progress callbacks so the `completedAt` branch matches. These re-introduce a small runtime cycle between the stage and the orchestrator (orchestrator imports `runCaptureStage`; stage imports helpers back). The cycle is safe (both modules finish loading before any stage function is invoked at runtime) and will be flattened in a follow-up PR that consolidates capture helpers into a shared module. Removes the now-orphaned `captureFrame` import from the orchestrator. Verified inside `Dockerfile.test`: - `font-variant-numeric`: audio correlation 1.000 - `many-cuts`: 0 failed frames, audio correlation 0.994 - `variables-prod`: PSNR ~69 dB, audio correlation 0.975 - `sub-composition-video`: PSNR ~43-52 dB, audio correlation 0.947 (exercises video extraction + capture end-to-end) - `gsap-letters-render-compat`: PSNR ~53-55 dB, audio correlation 1.000 (exercises the parallel capture path; 5/5 PASS overall) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
aec29fe599 |
refactor(producer): extract audioStage from executeRenderJob
Move the audio mixing sub-stage of `executeRenderJob` into `services/render/stages/audioStage.ts`. Trivial wrapper around `processCompositionAudio` with the same skip-when-empty path. No behavior change: - `audioOutputPath` is still `join(workDir, "audio.aac")` regardless of whether the composition has audio. - `hasAudio` still reflects `audioResult.success` (false when no audio elements or when the mixer returns success: false). - `perfStages.audioProcessMs` is set at the same end-of-stage point whether or not the mixer ran. - The "Processing audio tracks" progress callback fires at 20% at the same code point. Removes the now-unused `processCompositionAudio` import from the orchestrator (oxlint flagged it). Verified inside `Dockerfile.test` against `font-variant-numeric`, `many-cuts`, `variables-prod` — 3/3 pass with audio correlations 1.000 / 0.994 / 0.975 (identical to prior PRs in the stack). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
8951ed939c |
refactor(producer): extract extractVideosStage + add materializeSymlinks param
Move the video frame extraction sub-stage out of `executeRenderJob` into `services/render/stages/extractVideosStage.ts`. The stage covers HDR color-space pre-detection for videos and images, the `extractAllVideoFrames` call, frame-lookup-table construction, video readiness skip-id collection, video metadata hints, and the auto-detect of audio tracks from video files. Hard constraints preserved verbatim: - `composition.audios` is still mutated in place to add audio entries auto-discovered from video files via ffprobe. - `perfStages.videoExtractMs` is set at the same end-of-stage point. - `materializeExtractedFramesForCompiledDir` is still called once when `extractionResult.extracted` is non-empty. - `force-sdr` mode still skips ALL ffprobe overhead. New for distributed mode (`materializeSymlinks: boolean`, default false): - Plumbs through to `materializeExtractedFramesForCompiledDir` via a new option of the same name. When `true`, the helper invokes `cpSync(recursive)` instead of `symlinkSync` so the staged frames are real files inside `compiledDir`. Symlinks don't survive S3 / GCS round-trips, so distributed `plan()` will pass `true` once it lands. Default `false` preserves the in-process renderer's symlink behavior. - New unit test covers the copy path and asserts `symlinkSync` is NOT invoked; the existing symlink test was updated with the parallel guard that `cpSync` is NOT invoked. Removes the imports the orchestrator no longer needs after the extraction: `extractAllVideoFrames`, `resolveProjectRelativeSrc`, `createFrameLookupTable`, `FrameLookupTable`, `detectTransfer`, `isHdrColorSpace`, `extractMediaMetadata`, `VideoColorSpace` (oxlint flagged each). Verified: - `bunx oxlint` + `bunx oxfmt --check` clean - `bun run --filter @hyperframes/producer typecheck` + `build` clean - `bun test packages/producer/src/services/` — 176 pass, 1 pre-existing unrelated failure - `docker run hyperframes-producer:test` against `font-variant-numeric`, `many-cuts`, `variables-prod`, `sub-composition-video` — 4/4 PASS with correlations 1.000 / 0.994 / 0.947 / 0.975 (sub-composition-video is the one that exercises video frame extraction end-to-end) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
89d83fbbee |
refactor(producer): migrate test imports to render/shared.ts, drop re-export
Three small follow-ups on the shared.ts extraction, addressing review feedback on #720: - Hoist `BROWSER_MEDIA_EPSILON` from `probeStage.ts` into `shared.ts` so any future stage that reconciles browser media (chunked rendering re-probe, for instance) doesn't have to redeclare it. - Migrate `renderOrchestrator.test.ts` to import the five moved symbols (`applyRenderModeHints`, `projectBrowserEndToCompositionTimeline`, `resolveDeviceScaleFactor`, `writeCompiledArtifacts`, `CompositionMetadata`) directly from `./render/shared.js`. This is the clean end state — the back-compat re-export through `renderOrchestrator.ts` was a stepping-stone. - Drop the back-compat re-export block from `renderOrchestrator.ts`. No remaining importers go through it (verified via grep across `packages/`). The five symbols now have exactly one path: `./render/shared.js`. No behavior change. Renderer smoke-tested inside `Dockerfile.test` against `font-variant-numeric`, `many-cuts`, and `variables-prod` — audio correlations 1.000 / 0.994 / 0.975, matching every prior PR in the stack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
0a54078d25 |
refactor(producer): move shared render helpers to render/shared.ts
Breaks the runtime circular import between `renderOrchestrator.ts` and
the stage files under `services/render/stages/`. Before this change the
stages imported runtime helpers (`writeCompiledArtifacts`,
`applyRenderModeHints`, `resolveDeviceScaleFactor`,
`projectBrowserEndToCompositionTimeline`) and types (`CompositionMetadata`)
back from `renderOrchestrator.ts`, which itself imports the stage
functions. The cycle resolved at build time because both modules
finished initializing before any stage was invoked, but it was fragile
and would keep growing as more stages were extracted.
This PR:
- Adds `packages/producer/src/services/render/shared.ts` and moves the
four functions plus the `CompositionMetadata` interface into it.
- Has `renderOrchestrator.ts` re-export everything from `shared.ts`, so
external callers (the existing `renderOrchestrator.test.ts`, any code
importing `applyRenderModeHints` etc. from the orchestrator) keep
working with no churn on their side.
- Updates `compileStage.ts` and `probeStage.ts` to import the runtime
helpers from `../shared.js`. The only remaining import from
`renderOrchestrator.ts` in the stages is `import type { RenderJob }`,
which is erased at runtime and creates no cycle.
- Removes the imports the orchestrator no longer needs after losing
the four function definitions (`CANVAS_DIMENSIONS`, `VideoElement`,
`AudioElement`, `ImageElement`).
No behavior change. Renderer smoke-tested inside `Dockerfile.test`
against `font-variant-numeric`, `many-cuts`, and `variables-prod` —
all PSNR / audio-correlation baselines match PR 1.3 exactly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
20242515ec |
refactor(producer): make sequencer the sole writer of job.duration/totalFrames
The probe stage previously assigned `job.duration` and `job.totalFrames` inside its body AND the sequencer re-asserted them after the call to restore TS narrowing. Two writers for the same field is a maintenance hazard — a future refactor could drop one and create a silent skew. Move ownership: the stage computes `duration` and `totalFrames` and returns them; the sequencer is the sole writer onto the `RenderJob`. This also aligns with the eventual chunk-worker model where a chunk running in a separate process cannot mutate the orchestrator's `job`. No observable behavior change. `job.duration` / `job.totalFrames` end up with the same values; the zero-duration `throw` still happens inside the stage (now using the local `duration` constant) before any sequencer-side assignment. Verified by: - `bun run --filter @hyperframes/producer typecheck` clean - `bun test packages/producer/src/services/` 175 pass / 1 pre-existing unrelated failure on `main` Review feedback addressed: vanceingalls on #719. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
56ac52384f |
refactor(producer): drop internal PR/phase identifiers from stages doc
Comment-only cleanup. Removes "PR 1.x", "Phase 1 PR", and "Phase 3 PR 3.1" references from JSDoc blocks in `compileStage.ts`, `probeStage.ts`, `planHash.ts`, and `freezePlan.ts`. Track / PR identifiers rot quickly and belong in PR descriptions, not in source. Design-doc section citations (DISTRIBUTED-RENDERING-PLAN.md §X.Y) are kept — those reference a stable external artifact. Also tightens the `probeStage.ts` `browserProbeMs` doc string to say "near-zero when `needsBrowser` was false" instead of "0" — the Date.now() delta around the function body is sub-ms but not literally zero. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
90f14703b2 |
refactor(producer): extract probeStage from executeRenderJob
Move the browser probe / duration discovery / recompile / media reconciliation block out of `executeRenderJob` into `services/render/stages/probeStage.ts`. No behavior change. The sequencer calls `runProbeStage` at the same code point with identical inputs and outputs. The probe stage owns the `FileServerHandle` and the `CaptureSession` it creates and returns them to the sequencer. The sequencer still tracks them in its `let fileServer` / `let probeSession` bindings and closes them in its `finally` block — the resource lifetime is unchanged. `recompileWithResolutions` lives inside this stage because it depends on browser-resolved durations even though §2.1 of the distributed plan lists recompile as a sibling phase. Preserved invariants: - `composition` is mutated in place (videos / audios / duration) so downstream stages see the reconciled view through the same reference. - `job.duration` and `job.totalFrames` end up with the same values at the same code points. The result type carries `duration: number` alongside `totalFrames: number`, and the sequencer re-asserts the assignments after the call so TypeScript's control-flow narrowing works for the rest of `executeRenderJob`. - `perfStages.browserProbeMs` and `perfStages.compileMs` are written at the same code points with the same values. - The "Composition duration is 0" diagnostic builds the same hint string from the same console-buffer regex and `__timelines` probe. - The post-probe "failed network requests" warning fires with the same regex, the same first-10/first-5 slicing, and the same `console.warn` prefix. Renderer smoke-tested inside `Dockerfile.test` against `font-variant-numeric`, `many-cuts`, and `variables-prod` — all PSNR / audio correlation baselines match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
d168397758 |
refactor(producer): flag cfg.forceScreenshot mutation as distributed-render TODO
Add a `TODO(distributed-render):` comment near the `applyRenderModeHints` call documenting that this caller-owned-object mutation needs to move into the result type before `freezePlan` wires up. The mutation pattern works in-process but won't survive across processes / replays from a frozen plan — the value belongs in `LockedRenderConfig`, not on a mutated `EngineConfig`. No behavior change. Comment-only. Review feedback addressed: vanceingalls on #718. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
4f648c8122 |
refactor(producer): extract compileStage from executeRenderJob
Move the pure compile sub-stage (`compileForRender` + `applyRenderModeHints` + `writeCompiledArtifacts` + `CompositionMetadata` build + DPR resolution) out of `executeRenderJob` into `services/render/stages/compileStage.ts`. No behavior change. The sequencer calls `runCompileStage` at the same code point with identical inputs and outputs. The following invariants are preserved verbatim: - `cfg.forceScreenshot` is still mutated by `applyRenderModeHints`. - `perfStages.compileOnlyMs` is set to the same wall-clock interval (around the `compileForRender` call only). - The "Compiled composition metadata" log line is emitted after artifact writes with the same payload shape. - The "Supersampling composition via deviceScaleFactor" log line is emitted only when `deviceScaleFactor > 1`. - `stage1Start`, `updateJobStatus(..., "Compiling composition", 5, ...)`, and `perfStages.compileMs` (set at the end of probe) remain at their current code points in the sequencer. The probe sub-stage (`if (needsBrowser)`) is unchanged — it is extracted separately in PR 1.3. `recompileWithResolutions` lives inside the probe block because it depends on browser-resolved durations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
426bd983c9 |
refactor(producer): address review feedback on planHash
- Add `PLAN_HASH_SCHEMA_PREFIX = "hyperframes-plan-hash-v1\x00"` mixed into every digest. Future framing changes must bump the trailing integer; this makes cross-version mismatches visible at the wire format instead of producing silent collisions. Impossible to backfill later, easy to bake in now. - Hoist the `0x00` field delimiter to module scope (`FIELD_DELIMITER`). - Document the UTF-8 encoding contract for all string-typed input fields in the file-level JSDoc, so external verifiers know the encoding without reading the implementation. - Add a known-digest test for one fixed reference input (`995b4105...`). If the framing changes silently this test fails, forcing the developer to also bump the schema prefix. - Add an explicit `canonicalJsonStringify(undefined) → TypeError` test to pin the contract. No callers yet, so no behavior change in any code path. Review feedback addressed: vanceingalls on #717. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
4e1926be74 |
refactor(producer): scaffold services/render/stages/ + planHash utility
Seed PR for the Phase 1 staged refactor of `executeRenderJob`. Creates the new `packages/producer/src/services/render/stages/` directory and adds two files that subsequent stage-extraction PRs build on: - `planHash.ts`: a content-addressed sha256 helper plus a canonical-JSON serializer, with unit tests covering determinism, asset-order independence, sensitivity to each hashed component, and a delimiter- framing test against path/sha boundary collisions. - `freezePlan.ts`: signature-only skeleton (throws "not implemented") for the eventual plan-freeze step. No callers; the body lands later when the distributed-render primitives compose the Phase 1 stages. Zero behavior change. No code in `executeRenderJob` is touched and no existing exports move. The new files are not yet referenced anywhere outside the `stages/` directory. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
James
|
aa715ca8a0 |
feat(core,studio,cli): add square + square-4k canvas resolutions
The four existing presets only cover 16:9 (landscape) and 9:16 (portrait) aspect ratios. A 1080×1080 square comp had nowhere to land at any scale: "Auto" rendered at the comp's authored 1080×1080, and picking 1080p or 4K mapped to a landscape/portrait preset whose aspect ratio mismatched, which the producer's resolveDeviceScaleFactor validator rejects with "does not match the aspect ratio of the composition". Add `square` (1080×1080) and `square-4k` (2160×2160) to CANVAS_DIMENSIONS in core. The existing `keyof typeof CANVAS_DIMENSIONS` derivation extends the `CanvasResolution` union and `VALID_CANVAS_RESOLUTIONS` array automatically, so the producer's validator, the render API route, and the CLI `--resolution` flag pick the new presets up without further changes. - core: extend CANVAS_DIMENSIONS, RESOLUTION_ALIASES, and the htmlParser to recognize `data-resolution="square|square-4k"` and to infer square from equal width/height (vs. the prior "square defaults to portrait" tie-breaker). - studio: extend the local ResolutionPreset / CANVAS_DIMENSIONS mirrors; collapse isPortraitComp into a 3-way `compAspect` helper so resolveResolution returns the square preset for square comps. - cli: update --resolution help text on `init` and `render` to mention the new presets. - tests: add square cases to renderOrchestrator's resolveDeviceScaleFactor suite (returns 1 for square→square, 2 for square→square-4k, rejects landscape preset on square comp), update the htmlParser test that previously pinned the "square→portrait" tiebreaker. |
||
|
Theodor Kleynhans
|
5dcc89c930 |
feat(cli): accept ffmpeg-style rational fps (NTSC, PAL, slow-mo)
Replaces the rigid `--fps 24|30|60` whitelist with a numeric range and
adds support for ffmpeg-style fractional framerates so NTSC stays exact
end-to-end.
- `--fps 30` keeps working (integer fps)
- `--fps 30000/1001` now means exact NTSC 29.97 (not the lossy decimal)
- `--fps 24000/1001`, `--fps 60000/1001`, `--fps 25/50/120/240` all work
- Decimals like `--fps 29.97` are rejected with a friendly error pointing
the user at the rational form, since `29.97` and `30000/1001` round
to different framerates inside ffmpeg
Carries an `Fps = { num: number; den: number }` rational end-to-end:
RenderConfig, EncoderOptions, StreamingEncoderOptions, CaptureOptions,
DockerRenderOptions, Studio API request body, regression-harness
meta.json. The `-r` and `-framerate` ffmpeg args emit the rational form
verbatim (`30000/1001`) so no decimal round-trip happens at the encoder
boundary. Frame-interval math uses `1000 * den / num` ms (33.366… for
NTSC, 33.333… for integer 30).
Helpers live in @hyperframes/core:
- `parseFps(input: string | number): FpsParseResult` — discriminated
parser used by both the CLI and the Studio API route
- `fpsToFfmpegArg(fps: Fps): string` — emits "30" or "30000/1001"
- `fpsToNumber(fps: Fps): number` — for arithmetic (telemetry, frame
count, frame-index → time)
Studio API wire format accepts polymorphic `fps: number | string`:
- number → integer fps (`30`)
- string → rational (`"30000/1001"`)
Decimals are rejected; matches the same rule as the CLI.
Existing meta.json fixtures with integer `"fps": 30` continue to load
unchanged — the regression-harness validator now normalizes both number
and string inputs through `parseFps`.
|
||
|
James
|
453bd6e313 | fix(engine,producer): reject alpha+supersample, unit-test clip plumbing | ||
|
James
|
2b2281d490 | refactor: dedupe resolution presets and clean up 4k stack | ||
|
James
|
1545763ea3 | fix(cli,producer): cross-multiply aspect check, CLI HDR guard, honest banner | ||
|
James
|
e07aeba213 | feat(cli): add --resolution flag to hyperframes render for one-line 4k | ||
|
James
|
8203005488 | fix(engine): byte-budget the frame data uri cache to bound memory at 4k | ||
|
Vance Ingalls
|
b0fb664873 |
fix: render shader transitions for SDR compositions (#640)
* feat: cache shader transition preview frames * fix: move shader transition loading to player * fix: render shader transitions for sdr compositions |
||
|
James
|
7d1d8ead60 |
fix(producer): sample PSNR checkpoints from common duration of rendered+snapshot
Four regression tests (font-variant-numeric, many-cuts, missing-host-comp-id, variables-prod) failed on this PR with `Unable to parse PSNR output at <last checkpoint>s`. Root cause: the harness derived all 100 checkpoints from the *rendered* video's container duration, then asked ffmpeg's PSNR filter to compare the same frame index from both videos. The encoder changes earlier in this PR add `-avoid_negative_ts make_zero` to the mux step. With AAC audio that shifts the first audio sample to t=0 instead of the encoder-delay offset, extending reported container duration by ~20ms without changing video frame count. For the four failing tests, the i=99 checkpoint then landed on a frame index that exists in the rendered video but not in the snapshot baseline (e.g. round(2.98998 * 24) = 72 in a 72-frame baseline). ffmpeg's PSNR filter ran on zero matched frames and emitted no `average:` line, so the parser threw. Fix: probe both videos and use min(rendered, snapshot) duration when spreading checkpoints. This is the correct semantics for symmetric PSNR comparison anyway — both videos must have a frame at every sampled time. The change is local to the harness; no encoder behavior changes, no baselines regenerated. Other regression tests with audio (chat, sub-composition-video, vignelli-stacking) passed because their checkpoint-99 frame index landed inside the baseline's frame range with several frames of slack. The four failing tests had round-number durations where a 20ms drift was enough to push the last checkpoint past `nb_frames - 1`. |
||
|
JamesandClaude Opus 4.7
|
f4ecf96918 |
fix(engine,cli,producer): address PR #627 review feedback
- engine/chunkEncoder, engine/streamingEncoder: extend `-bf 0` to GPU h264
paths (nvenc, qsv, vaapi) and `-b_strategy 0` for qsv so GPU-encoded
outputs avoid negative-DTS freezes too — not just SW libx264.
- engine/videoFrameExtractor: detect mid-path traversal (e.g.
`assets/../../foo.mp4`) by normalizing first and re-anchoring at the
project root. Adds a regression test.
- engine/videoFrameExtractor: dedupe stderr "src not resolvable" warnings
by `video.src` so a comp with N broken sources logs once, not N times.
- engine/videoFrameExtractor.test: drop dynamic `require("node:fs")`,
use ES `import { writeFileSync } from "node:fs"`.
- engine/ffprobe: extract `readTagCI` helper for case-insensitive ffprobe
tag reads (will recur for other libavformat-versioned sidecar tags).
- cli/background-removal/pipeline: collapse Quality / QUALITIES /
QUALITY_CRF / DEFAULT_QUALITY / isQuality surface using
`Quality = keyof typeof QUALITY_CRF`.
- producer/renderOrchestrator: replace `v.src.startsWith("/")` with
`isAbsolute(v.src)` in the HDR probe path so Windows absolute paths
(`C:\...`) aren't treated as relative — matches the audioMixer guard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
2f96d5c7ab |
fix(engine,producer): URL-clamp sub-comp src paths and warn on silent extraction misses
A <video src='../assets/foo.mp4'> inside a sub-composition silently dropped from extraction; the rendered output froze on the first decoded frame for the entire clip, with no error in stdout. Root cause: browser URL resolver clamps '..' at origin root (studio preview loads fine), but path.join(projectDir, '../assets/foo.mp4') normalizes to parent-of-project/assets/foo.mp4, which usually doesn't exist. existsSync returns false, extraction is skipped, no frame lookup is built, the per-frame injector has nothing to swap, and the <video> element's first decoded frame paints every screenshot. - Adds resolveProjectRelativeSrc in videoFrameExtractor that mirrors browser clamping (literal join first, then leading '..' stripped). - Surfaces a loud stderr warning when the resolver misses. - Mirrors fix in audioMixer.ts (same bug for <audio src='../'>) and renderOrchestrator HDR probe loop. - +6 regression tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
58b4234809 |
test(producer): add variables-prod regression test for the variables stack
End-to-end Docker regression test that exercises the full variables
chain: meta.json renderConfig.variables → harness → createRenderJob →
RenderConfig → CaptureOptions → engine evaluateOnNewDocument →
window.__hfVariables → getVariables() → DOM text → rendered pixels.
Fixture (packages/producer/tests/variables-prod/):
- src/index.html: composition with three declared variables (title,
subtitle, bgColor) read via window.__hyperframes.getVariables() and
rendered as positioned text on a colored background. No animation —
keeps the regression frame-stable so it isolates "did the variables
flow through?" from motion concerns.
- meta.json: tags ["variables", "composition"] (runs in the existing
fast shard's tag filter), renderConfig.variables provides override
values the baseline reflects ("Override Title", "Override subtitle",
#0a3d62). Defaults would produce a visibly different frame, so a
failing baseline that reflects defaults means the variables didn't
propagate.
- output/output.mp4: Docker-generated baseline per the project's
CLAUDE.md golden-baseline rule.
Harness change (packages/producer/src/regression-harness.ts):
- TestMetadata.renderConfig gains an optional variables field,
validated as a JSON object in the meta.json validator.
- The createRenderJob call site forwards renderConfig.variables to
RenderConfig.variables, which the engine already consumes via
evaluateOnNewDocument (PR #600).
Verified: docker:test variables-prod passes 100/100 visual checkpoints
and audio correlation 1.000 against the committed baseline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
c0d75a5268 |
feat(core,cli,engine,producer): add getVariables() helper and --variables render flag
Adds the parametrized-render primitive from hf#592 by reusing the existing
data-composition-variables schema as the source of declared defaults.
- Runtime helper window.__hyperframes.getVariables() (also exported from
@hyperframes/core) reads data-composition-variables defaults from the
document root and merges window.__hfVariables (CLI override) on top.
Returns Partial<T> for typed access; supports a generic for editor
ergonomics. Same code path runs in dev preview and at render time.
- CLI render --variables '<json>' / --variables-file <path> populates the
override. Mutually exclusive; fail-fast on conflicting flags, missing
file, unparseable JSON, or non-object payloads. parseVariablesArg is
exported as a pure function so validation paths stay unit-testable.
- Engine injects window.__hfVariables via evaluateOnNewDocument before
any page script runs, so the helper sees the merged values on its
first call. Empty payloads are skipped to avoid pointless init scripts.
- Producer threads variables through RenderConfig and into the engine's
CaptureOptions; Docker mode forwards --variables to the in-container
CLI invocation via dockerRunArgs.
Composition authors declare variables once on the root <html> element:
<html data-composition-variables='[
{"id":"title","type":"string","label":"Title","default":"Hello"}
]'>
and read them in any composition script:
const { title } = window.__hyperframes.getVariables();
A render with `--variables '{"title":"Q4 Report"}'` overrides the default
without modifying the composition source. Missing keys fall through to
the declared defaults, so dev preview and CLI renders without --variables
behave identically.
This is PR 1 of a 4-PR stack. Sub-comp per-instance scoping (carrying
host data-variable-values through the inlined sub-comp's getVariables()
call) lands in PR 2; schema validation and lint in PR 3; skill / scaffold
distribution in PR 4.
Tests: 9 new unit tests for getVariables() (jsdom), 11 new CLI tests
covering parseVariablesArg validation paths and Docker passthrough,
2 new dockerRunArgs assertions for the --variables flag. All existing
tests green (core 611, cli 208, engine 519).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
2a897d351c |
fix: address PR #596 review issues (#597)
## Summary Fixes three issues identified in the [post-merge review](https://github.com/heygen-com/hyperframes/pull/596#pullrequestreview-4214283515) of PR #596: - **P1 (cache bypass):** When `extractCacheDir` is set, extracted frames live outside `compiledDir`, so `createCompiledFrameSrcResolver` rejects them and every frame falls back to base64 data URIs. Fix: symlink cached frame directories into `compiledDir/__hyperframes_video_frames/` after extraction and remap `framePaths` so the served-frame fast path works. - **P2 (pooled browser stale state):** `closeCaptureSession` force-killed the Chrome process on timeout via raw `SIGKILL` without clearing `pooledBrowser` / `pooledBrowserRefCount`, leaving other sessions with a dead browser reference. Fix: add `forceReleaseBrowser()` in `browserManager` that atomically clears pool state before killing the process. - **P3 (reserved chars in URLs):** `createCompiledFrameSrcResolver` encodes path segments with `encodeURIComponent`, but the file server used `c.req.path` (which only applies `decodeURI`) to look up files on disk. Video IDs containing `#`, `?`, or `%` produced 404s. Fix: apply `decodeURIComponent` per path segment in the file server's catch-all route. ## Test plan - [x] `createCompiledFrameSrcResolver` tests: symlinked cache paths resolve to served URLs; cache-external paths return null; reserved characters encode correctly - [x] `forceReleaseBrowser` tests: kills process + disconnects; tolerates already-killed process - [x] `createFileServer` test: `video%231/frame.jpg` serves file from `video#1/frame.jpg` on disk - [x] Typecheck: engine + producer pass - [x] Lint + format: 0 warnings, 0 errors |
||
|
Miguel Ángel
|
4750a981dd | fix: speed up video frame injection renders (#596) | ||
|
Miguel Ángel
|
15ee63c6e7 |
fix: harden CLI edge-case repros (#591)
## Problem I reproduced the selected open issue batch one by one and confirmed the reports were valid. The fixes all touch the CLI/runtime capture boundary, then the follow-up regression run exposed one over-broad runtime change in sub-composition host visibility and one CI-only baseline trap. Closes #590, #589, #588, #587, #586, and #584. ## What this fixes ### CLI/runtime edge cases - Makes the GSAP infinite-repeat lint rule ignore JavaScript comments, so literal `repeat:-1` text in comments is not flagged. - Lets the compositions CLI inspect `<template>` content, count visual-only template descendants, estimate simple GSAP durations, and suppress root `data-start` warnings in sub-composition lint mode. - Preserves runtime bootstrap scripts when body scripts are coalesced, and injects the runtime into a real `<head>` when source HTML has no head. - Keeps #589 fixed by loading and rendering template-wrapped sub-composition content, while restoring host visibility to the shorter of the authored parent clip window and the child composition live timeline. - Resolves snapshot/validate viewport size from root `data-width` / `data-height` instead of falling back to 1920x1080. - Skips fully off-frame text boxes during contrast sampling and bounds-checks ring samples so contrast output no longer emits `null:1` / `NaN:1`. - Marks muted videos as `data-has-audio="false"` in the core timing compiler, which fixes the same-src muted `<video>` + separate `<audio>` StaticGuard case. - Keeps user-authored `hf-seek` listeners reachable during capture by preventing author scripts from being merged into the runtime bootstrap path. ### Shared helper cleanup - Removes the stale producer-local timing compiler duplicate; producer compilation now consumes the core timing compiler. - Centralizes HTML document helpers in core: fragment parsing, embedded runtime stripping, head/body script injection, and early-head injection. - Centralizes the CLI layout/snapshot static HTML server. - Adds browser-safe core subpath helpers for Lottie readiness and CLI screenshot clip calculation; Studio's Vite config keeps the screenshot clip helper self-contained so clean-checkout test startup does not value-import core `.ts` source. - Replaces the engine parity-contract copy with a core re-export. - De-duplicates render-job cleanup and Studio static file-serving callbacks. ### Regression hardening - Replaces the embedded-runtime script stripping regex with a script-tag scanner that handles closing tags like `</script >`. - Escapes inline script bodies before wrapping them in `<script>` tags, so authored `</script` and `<!--` text cannot break out of the injected wrapper script. - Shares media-duration clamping between core and producer, with a 50 ms tolerance for ffprobe precision drift between local and CI media stacks. - Pins the affected style fixture SFX durations in source so style-1 and style-9 compile deterministically. - Restores the `vfr-screen-recording` video golden to the CI-stable baseline; the current CI failure showed the Linux render matches the old golden, while the locally refreshed macOS golden was the mismatch. ## Root cause The CLI paths had accumulated assumptions that held for simple direct-root landscape compositions but not for current composition patterns: DOM queries did not enter template content, snapshot/validate used a fixed viewport, runtime and author scripts shared a coalescing bucket, and timing compilation treated every video as audio-bearing unless authors manually overrode it. The style shard failures were not product regressions. Local and CI media probing disagreed on the short SFX clip duration by about 45 ms, and the compiler was clamping authored durations to the locally probed value. The shared clamp tolerance preserves explicit author/source durations for small probe precision differences while still clamping real overflows. The vfr fast-shard failure was a bad baseline refresh: CI actual frames matched the old `vfr-screen-recording` baseline at 40+ dB PSNR, but mismatched the macOS-refreshed golden at ~18-22 dB. The fix is to keep the Docker/Linux-stable video golden and only retain the deterministic compiled snapshot change. The sub-composition regression came from treating a host's authored parent window as the only visibility boundary. That made settled child overlays stay visible after their own live GSAP timeline ended. The corrected runtime behavior respects both contracts: parent clips still bound where the host can appear, and the child live timeline can end the host earlier. ## Verification ### Local checks - `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bun run --cwd packages/core test src/runtime/init.test.ts` - `bun run --cwd packages/cli test src/commands/compositions.test.ts src/utils/compositionViewport.test.ts` - `bun run build:hyperframes-runtime` - `bun run --cwd packages/producer test --keep-temp --sequential style-12-prod style-5-prod` - `bun run --cwd packages/producer test --sequential vfr-screen-recording hdr-hlg-regression style-7-prod` - `bun run --cwd packages/core test src/compiler/htmlCompiler.test.ts src/compiler/timingCompiler.test.ts src/index.test.ts` - `bunx oxfmt --check packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bunx oxlint packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bun run --cwd packages/core typecheck` - `bun run --cwd packages/producer typecheck` - `bun run --cwd packages/producer test --sequential style-1-prod style-9-prod` - `bun run --filter @hyperframes/studio test` with `packages/core/dist` temporarily hidden to simulate clean-checkout config loading - `git diff --check` ### CI artifact checks - Inspected failed run `25225854394` job `73969147096`: style-1 failed only on `click-sfx` `1.044898` vs `1` duration/end. - Inspected failed run `25225854394` job `73969147061`: style-9 failed only on SFX `1.044898`-based duration/end mismatches. - Inspected failed run `25225854394` job `73969147048`: `vfr-screen-recording` compilation/audio passed, visual failed after comparing against the macOS-refreshed golden. - Compared the first 10 uploaded CI vfr failure frames against the restored old baseline; minimum PSNR was `40.444705`, above the fixture threshold of `28`. ### Repro checks - `bun packages/cli/src/cli.ts lint /tmp/hf-590-repro` now passes without `gsap_infinite_repeat`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-587-repro --at 0.5 --timeout 1000` now writes a 1080x1920 PNG. - `bun packages/cli/src/cli.ts validate /tmp/hf-588-repro --timeout 500` no longer emits `null:1` / `NaN:1` contrast output. - `bun packages/cli/src/cli.ts validate /tmp/hf-586-repro --timeout 500 --contrast false` no longer emits the muted-video StaticGuard contract error. - `bun packages/cli/src/cli.ts compositions /tmp/hf-589-gsap-repro` now reports `foo 0.5s 1920x1080 1 element`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-589-gsap-repro --at 0.25 --timeout 2000` captures the expected template-backed red frame. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-584-repro --at 0.5,1.5 --timeout 500` captures the expected post-seek green frame. ### Browser verification - Refreshed the local side-by-side comparison page at `qa-artifacts/pr-591-video-compare/index.html`. - Served the comparison page locally and used `agent-browser` to load `style-12-prod`, play both videos quickly to the failed window, pause, and inspect the side-by-side frame. - Browser proof screenshot: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12-labeled.png`. - Browser proof recording: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12.webm`. - Earlier Studio proof artifacts remain local-only: `qa-artifacts/dedupe-refactor-preview.png`, `qa-artifacts/dedupe-refactor-preview-after-play.png`, `qa-artifacts/dedupe-refactor-preview.webm`. ## Notes - Browser proof and CI diagnostic artifacts are intentionally local-only and not committed. - Studio's Vite config intentionally keeps the thumbnail clip helper inline because Vite/Vitest config startup runs through Node's loader before package source `.ts` imports are transformed. - The committed PR diff changes `vfr-screen-recording/output/compiled.html` but no longer changes `vfr-screen-recording/output/output.mp4` relative to `main`. - I attempted a local `linux/amd64` Docker validation to mirror CI, but the local Docker build was blocked by Debian package download failures. The arm64 Docker image also cannot launch the x64 Puppeteer headless shell under OrbStack. The vfr baseline decision is therefore based on the uploaded CI artifact comparison above. - I kept this validated issue batch in one PR because the fixes overlap the same CLI/runtime capture surfaces. |
||
|
Miguel Ángel
|
dde26cf62d | feat: default streaming encode for sequential renders (#579) | ||
|
Miguel Ángel
|
6a59ef6106 |
fix: skip metadata waits for injected video frames (#575)
## Problem Closes #574. On Windows with cached headless-shell Chrome, a composition that reuses the same video file in three timeline clips can fail before frame capture starts: ```html <video id="video1" src="1.mp4" data-start="0" muted data-duration="4" data-track-index="0" data-media-start="0"></video> <video id="video2" src="1.mp4" data-start="4" muted data-duration="4" data-track-index="0" data-media-start="4"></video> <video id="video3" src="1.mp4" data-start="8" muted data-duration="4" data-track-index="0" data-media-start="8"></video> ``` The reported render reaches video frame extraction, then dies at frame-capture initialization with: ```text [FrameCapture] video metadata not ready after 45000ms. Video elements must load metadata before capture starts. ``` The important detail is that by this stage HyperFrames has already extracted video pixels through FFmpeg. Native Chromium video metadata is only being waited on for DOM layout stability, not because Chromium is the source of rendered pixels. ## Root Cause The render pipeline has two separate media responsibilities: - FFmpeg extracts video frames and audio from declared media. - Chromium owns DOM layout and capture, while injected FFmpeg frames supply the video pixels before each captured frame. Before this PR, every capture session still waited for every DOM `<video>` to reach `readyState >= 1` unless the element was a native HDR exception. That made native browser media metadata a hard render prerequisite even when the browser would not decode or provide the final video pixels. That is why the issue fails at `25% Starting frame capture`: FFmpeg extraction has already succeeded, but capture initialization blocks on repeated native `<video src="1.mp4">` metadata loading in cached Windows headless-shell Chrome. There was a second constraint: the readiness wait also prevents first-frame layout bugs. If a skipped `<video>` has no native metadata, Chromium can use the default `300x150` intrinsic video size, which breaks layouts such as `width: 100%; height: auto` before the first injected frame. The fix therefore must not simply skip all video readiness waits; it must provide dimensions for any skipped videos. ## What This Fixes - Treats videos with successfully extracted FFmpeg frames and usable dimensions as out-of-band rendered video sources. - Skips native browser metadata readiness waits for those extracted videos because Chromium is not responsible for their pixels. - Passes FFmpeg-probed dimensions into capture as `videoMetadataHints`. - Applies those hints before the readiness wait in both screenshot and BeginFrame initialization paths. - Sets missing `width` / `height` attributes and an explicit `aspect-ratio` only when the element does not already provide one, preserving author styles where present. - Keeps native HDR video IDs in the skip list, preserving the existing HEVC/HDR behavior where Chrome may not decode the source but FFmpeg/native HDR compositing can still render it. - Uses one `buildCaptureOptions()` helper so calibration, HDR DOM capture, streaming capture, parallel capture, and sequential capture receive the same skip IDs and metadata hints. - Adds tests for the skip-list and metadata-hint contract. - Adds a Windows CI regression that reproduces the issue shape after the canary render warms the cached-browser path. ## Reviewer Map Primary files: - `packages/producer/src/services/renderOrchestrator.ts` - `collectVideoReadinessSkipIds()` includes native HDR IDs plus extracted videos that have finite positive FFmpeg dimensions. - `collectVideoMetadataHints()` converts extracted FFmpeg metadata into capture hints. - `buildCaptureOptions()` threads `skipReadinessVideoIds` and `videoMetadataHints` into every capture path. - `packages/engine/src/services/frameCapture.ts` - `applyVideoMetadataHints()` runs in the page before video readiness polling. - Both screenshot and BeginFrame initialization call it before checking non-skipped videos for `readyState >= 1`. - `packages/engine/src/types.ts` - Adds `CaptureVideoMetadataHint` and documents that readiness skips should be paired with metadata hints when layout may depend on intrinsic dimensions. - `packages/producer/src/services/renderOrchestrator.test.ts` - Covers that extracted videos with dimensions are skipped, invalid dimensions are not, native HDR IDs are preserved, and hints are stable/sorted. - `.github/workflows/windows-render.yml` - Adds the issue #574 Windows regression with the exact three-clip markup and a generated deterministic `1.mp4`. ## Why This Is Safe The skip is intentionally gated: - A standard video is skipped only after `extractAllVideoFrames()` succeeded for that video and returned usable dimensions. - Videos with invalid dimensions are not skipped, so the old browser readiness guard still applies. - DOM videos are still present for layout and element bounds; only the native metadata wait is skipped for sources whose pixels come from FFmpeg injection. - Metadata hints are applied conservatively: existing `width`, `height`, and explicit `aspect-ratio` are not overwritten. - Non-extracted videos, images, fonts, page readiness, and `window.__hf` readiness keep the existing waits. - The fix is not limited to the sequential path from the issue; it is threaded through calibration, HDR DOM capture, streaming encode, parallel capture, and sequential capture. A first local revision skipped readiness too broadly and caused `overlay-montage-prod` first-frame layout shrinkage. The current version fixes that by pairing skips with FFmpeg metadata hints; `overlay-montage-prod` now passes and is listed in verification below. ## Verification ### Root-Cause Reproduction Before Fix The reporter did not attach the actual `1.mp4`, so the regression uses the exact issue markup and a deterministic generated 12s H.264 file named `1.mp4`. I reproduced the failure in GitHub Actions by running this branch's new Windows workflow against unpatched `main`: ```bash gh workflow run windows-render.yml --repo heygen-com/hyperframes --ref fix/reused-video-metadata -f ref=main ``` That means the workflow contains the new issue #574 regression, but the code under test is `main` without this fix. Baseline failure: - Run: https://github.com/heygen-com/hyperframes/actions/runs/25174603730 - Failed job: https://github.com/heygen-com/hyperframes/actions/runs/25174603730/job/73803179086 - Checkout proof: `ref: main`, `origin/main`, commit `8662598a3ac64018a2999d189ffb369e6d46b53a`. - Failure proof: `Browser: cache`, `staticDuration:12`, `videoCount:3`, then `25% Starting frame capture` -> `[FrameCapture] video metadata not ready after 15000ms`. This is the same failure class as the issue, on Windows, in cached-browser mode, before the fix. ### Fixed Windows Regression The same regression passes on this PR branch: - Run: https://github.com/heygen-com/hyperframes/actions/runs/25175048215 - Passing job: https://github.com/heygen-com/hyperframes/actions/runs/25175048215/job/73804781017 - Checkout proof: PR merge contains `79d6b41c9f2ba137cbfb9301678e0815b16c4f5a` merged into `8662598a3ac64018a2999d189ffb369e6d46b53a`. - Passing proof: `Browser: cache`, `staticDuration:12`, `videoCount:3`, `25% Starting frame capture`, captures `360/360` frames, renders `issue-574.mp4`, and `ffprobe` verifies `1920x1080 @ 30/1, 12s`. ### Local Checks - `bun run build:hyperframes-runtime` - `bunx vitest run packages/producer/src/services/renderOrchestrator.test.ts` - `bun run --filter @hyperframes/producer typecheck` - `bun run --filter @hyperframes/engine typecheck` - `bunx oxlint packages/engine/src/services/frameCapture.ts packages/engine/src/types.ts packages/engine/src/index.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts` - `bunx oxfmt --check .github/workflows/windows-render.yml packages/engine/src/services/frameCapture.ts packages/engine/src/types.ts packages/engine/src/index.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts` - `git diff --check` - Lefthook pre-commit: lint, format, typecheck where applicable - Lefthook commit-msg: commitlint ### Local Render Checks - Created `/tmp/hf-issue-574-repro` with the issue shape: three clips using the same `1.mp4`, `data-media-start=0/4/8`, 12s total. - `PRODUCER_PLAYER_READY_TIMEOUT_MS=5000 bun packages/cli/src/cli.ts render /tmp/hf-issue-574-repro --workers 1 --quality draft --fps 30 --output /tmp/hf-issue-574-h264-fixed-v2.mp4` -> completed. - Created `/tmp/hf-issue-574-prores` with the same three-clip shape using one FFmpeg-readable ProRes `.mov`, which exercises the browser-metadata failure class because Chromium should not be needed to decode the source. - `PRODUCER_PLAYER_READY_TIMEOUT_MS=3000 bun packages/cli/src/cli.ts render /tmp/hf-issue-574-prores --workers 1 --quality draft --fps 30 --output /tmp/hf-issue-574-prores-fixed-v2.mp4` -> completed. - `bun run --filter @hyperframes/producer test --sequential --keep-temp overlay-montage-prod` -> passed; this guards against skipped metadata shrinking `height:auto` video layout before the first injected frame. - `ffmpeg -v error -i /tmp/hf-issue-574-prores-fixed-v2.mp4 -f null -` - `ffmpeg -v error -i /tmp/hf-issue-574-h264-fixed-v2.mp4 -f null -` - `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-issue-574-h264-fixed-v2.mp4` -> H.264, 320x180, 30fps, 12.0s. ### Current PR Checks - Windows render verification: pass on https://github.com/heygen-com/hyperframes/actions/runs/25175048215. - Windows tests: pass on https://github.com/heygen-com/hyperframes/actions/runs/25175048215. - Main CI build/lint/typecheck/test/smoke jobs: pass on https://github.com/heygen-com/hyperframes/actions/runs/25175048175. - Regression shards observed passing include HDR, render-compat, styles A-G, and `overlay-montage-prod`. At the time this body was updated, the `fast` regression shard was still in progress in run https://github.com/heygen-com/hyperframes/actions/runs/25174515546. ### Browser Verification - Used `agent-browser` to open `file:///tmp/hf-issue-574-h264-fixed-v2.mp4` and verify the rendered output displays in Chromium. - Screenshot: `.debug/issue-574/h264-output-page.png` - Agent-browser recording: `.debug/issue-574/h264-output-playback.webm` ## Notes / Caveats - The reporter's exact `1.mp4` was not attached to #574. The committed Windows regression uses a generated deterministic H.264 file with the same filename and exact markup from the issue. - The exact H.264 issue shape did not reproduce the timeout on this macOS/system-Chrome machine before the fix; it rendered successfully locally. The GitHub Actions baseline above reproduces it on Windows/cache without the fix. - The Windows fixture intentionally runs after the existing canary render so the browser path is `Browser: cache`, matching the reporter's environment. - The generated fixture emits sparse-keyframe warnings. Those warnings are expected and are not the failure being fixed; the baseline failure occurs before any frame capture because native browser video metadata never becomes ready. - Browser proof artifacts are local-only under `.debug/issue-574/` and intentionally not committed. |
||
|
Miguel Ángel
|
395fb9c084 |
feat: add browser GPU render mode (#571)
## Problem HyperFrames already had `--gpu`, but that flag only controlled FFmpeg hardware encoding. The browser capture path still forced Chrome/WebGL through SwiftShader software GL via `--use-angle=swiftshader`, so WebGL-heavy local renders could leave the biggest bottleneck on the CPU path. That made the existing flag naming easy to misread: `--gpu` sounded like it accelerated the whole render, but it did not change the browser frame-capture backend. ## What this fixes - Enables host browser GPU acceleration automatically for local CLI renders. - Adds `--no-browser-gpu` as the local opt-out for software Chrome/WebGL capture. - Keeps `--browser-gpu` as an explicit local browser-GPU request. - Adds `browserGpuMode: "software" | "hardware"` to engine config, with `PRODUCER_BROWSER_GPU_MODE` env support for lower-level producer users. - Keeps Docker browser capture on the deterministic software path. - Maps hardware browser GPU mode to platform-native Chrome backends: - macOS: Metal-backed ANGLE - Windows: D3D11-backed ANGLE - Linux: EGL - Blocks explicit `--browser-gpu --docker` with a clear error because Docker browser GPU passthrough is not cross-platform. - Clarifies docs so `--gpu` means FFmpeg encoder GPU and browser GPU means Chrome/WebGL capture GPU. - Keeps encoder backend selection auto-detected from FFmpeg capabilities: - NVIDIA: NVENC - macOS: VideoToolbox - Linux: VAAPI - Intel: QSV ## Why two flags There are two separate GPU surfaces in the render pipeline: 1. Browser GPU controls Chrome frame capture. - Affects WebGL, canvas, CSS rendering, compositing, and screenshot capture inside the browser. - This is enabled automatically for local CLI renders. - Use `--no-browser-gpu` when you want the software browser baseline. 2. `--gpu` controls FFmpeg video encoding. - Affects the final encode step after frames have already been captured. - The concrete encoder is auto-detected from the host FFmpeg build and hardware. - It can be faster for some machines/codecs, but it is not equivalent to browser rendering acceleration. The controls stay independent because users may want: - `hyperframes render` for the fast local default with browser GPU capture. - `hyperframes render --no-browser-gpu` for the software-browser local baseline. - `hyperframes render --gpu` for browser GPU capture plus hardware FFmpeg encoding. - `hyperframes render --no-browser-gpu --gpu` for software browser capture plus hardware FFmpeg encoding. - `hyperframes render --docker` for deterministic browser capture. ## Why `--gpu` does not imply browser GPU Keeping `--gpu` scoped to FFmpeg encoding avoids a semantic break and keeps the risk profile explicit: - `--gpu` already means encoder acceleration. Expanding it to also change Chrome capture would silently alter behavior for users who only wanted hardware encoding. - Browser GPU and encoder GPU have different portability. Encoder GPU can work in Docker when the host exposes the right devices; browser GPU passthrough is not cross-platform, so this PR intentionally blocks explicit `--browser-gpu --docker`. - The Apple presentation benchmark shows why the controls should stay separate: browser GPU capture was the useful improvement, while macOS VideoToolbox via `--gpu` was slower and produced larger output for this `standard` H.264 run. If HyperFrames later wants a single umbrella acceleration control, it should be explicit, for example `--acceleration browser|encoder|all` or `--gpu=browser|encoder|all`, rather than changing the meaning of the existing boolean `--gpu`. ## Root cause `buildChromeArgs()` always injected `--use-gl=angle --use-angle=swiftshader`. `disableGpu` only appended `--disable-gpu`; it did not provide a hardware-GPU mode. That made the public `--gpu` flag look broader than it was, because render capture stayed software-backed even when encoder GPU was requested. ## Verification ### Local checks - `bun install` - `bun run build:hyperframes-runtime` - `bun run --filter @hyperframes/engine test src/config.test.ts src/services/browserManager.test.ts` - `bun run --filter @hyperframes/cli test src/utils/dockerRunArgs.test.ts src/commands/render.test.ts` - `bun run --filter @hyperframes/cli typecheck` - `bun run --filter @hyperframes/engine typecheck` - `bun run --filter @hyperframes/producer typecheck` - `cd packages/producer && bunx vitest run src/services/renderOrchestrator.test.ts` - `bunx oxlint packages/cli/src/commands/render.ts packages/cli/src/commands/render.test.ts packages/cli/src/utils/dockerRunArgs.ts packages/cli/src/utils/dockerRunArgs.test.ts packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/browserManager.ts packages/engine/src/services/browserManager.test.ts packages/producer/src/services/renderOrchestrator.test.ts` - `bunx oxfmt --check ...` on changed source/docs files - `git diff --check` - `bun packages/cli/src/cli.ts render --help | rg -n "browser-gpu|no-browser-gpu|GPU"` - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --output /tmp/hf-auto-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict` - Render plan prints `GPU: browser GPU (auto)`. - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --no-browser-gpu --output /tmp/hf-software-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict` - Render plan does not print browser GPU. - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --docker --browser-gpu --output /tmp/should-not-render.mp4` - Exits 1 with `Browser GPU is local-only`. - `buildDockerRunArgs()` regression coverage asserts Docker container args include `--no-browser-gpu`, preventing nested container renders from re-enabling browser GPU through the local CLI default. - `resolveBrowserGpuForCli()` regression coverage asserts `PRODUCER_BROWSER_GPU_MODE=software` opts out when no CLI browser-GPU flag is supplied, while explicit `--browser-gpu` / `--no-browser-gpu` still win. - `ffmpeg -v error -i /tmp/hf-auto-browser-gpu-smoke.mp4 -f null -` - `ffmpeg -v error -i /tmp/hf-software-browser-gpu-smoke.mp4 -f null -` - `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-browser-gpu-smoke.mp4` -> H.264, 1920x1080, 24fps, 5.0s ### Apple presentation benchmark Rendered `/Users/miguel07code/Downloads/apple-presentation.zip` as supplied after extracting to `/tmp/hf-apple-profile/apple-presentation`. Fixed settings: - 1920x1080 - 30fps - `standard` quality - 4240 frames - 141.32s duration - 8-worker cap; render auto-calibration used 6 capture workers - macOS host detected FFmpeg GPU encoder: `videotoolbox` | Mode | Equivalent flags after this PR | Wall time | vs software-browser baseline | Speed | Capture | Encode | Output | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | Software browser + CPU encode | `--no-browser-gpu` | 120.77s | baseline | 1.17x | 97.87s | 10.04s | 8.38MB | | Browser GPU + CPU encode | default local render | 70.10s | 42.0% faster | 2.02x | 50.72s | 9.91s | 8.39MB | | Software browser + encoder GPU | `--no-browser-gpu --gpu` | 133.16s | 10.3% slower | 1.06x | 103.58s | 18.31s | 25.43MB | | Browser GPU + encoder GPU | `--gpu` | 74.12s | 38.6% faster | 1.91x | 46.69s | 17.93s | 25.45MB | Result: browser GPU capture is the meaningful improvement for this WebGL/browser-capture-heavy presentation. VideoToolbox encoding was slower and produced larger files for this current `standard` H.264 path, so `--gpu` should stay separate and opt-in. Why `--gpu` plus browser GPU was slower than browser GPU alone: the combined run captured about 4.0s faster than browser GPU alone, but VideoToolbox encoding was about 8.0s slower than CPU x264 encoding, so the encode loss outweighed the capture gain. ### VideoToolbox flag check I also isolated the encode stage against the already-captured Apple frames to check whether macOS GPU encoding only needed special flags. `ffmpeg -h encoder=h264_videotoolbox` does not expose a CRF/CQ-style quality option like x264. It exposes bitrate-oriented and VideoToolbox-specific options such as `-b:v`, `-realtime`, `-profile`, `-coder`, `-prio_speed`, `-power_efficient`, and `-allow_sw`. That means our current `-q:v` mapping is not equivalent to x264 CRF and can produce very different bitrate/size behavior. Measured full-frame encode variants on this host: | VideoToolbox variant | Encode wall time | Output size | Bitrate | | --- | ---: | ---: | ---: | | Current `-q:v 64 -allow_sw 1` | 18.76s | 25.31MB | 1.43 Mbps | | Current without `-allow_sw 1` | 18.21s | 25.31MB | 1.43 Mbps | | `-b:v 500k -maxrate 750k -bufsize 1000k -profile high -coder cabac -realtime 1 -prio_speed 1 -power_efficient 0` | 20.58s | 7.42MB | 0.42 Mbps | | Same with `-b:v 1500k` | 20.84s | 16.70MB | 0.95 Mbps | | `-b:v 500k -profile baseline -coder cavlc -realtime 1 -prio_speed 1 -power_efficient 0` | 18.11s | 8.94MB | 0.51 Mbps | Conclusion: VideoToolbox can be made size/bitrate-predictable with explicit `--video-bitrate`, but the tested speed-oriented flags did not make it faster than CPU x264 wall time for this render. That reinforces keeping `--gpu` encoder acceleration explicit and separate from browser GPU capture. Artifacts from the local benchmark: - `/tmp/hf-apple-profile/results/cpu.mp4` - `/tmp/hf-apple-profile/results/browser-gpu.mp4` - `/tmp/hf-apple-profile/results/encoder-gpu.mp4` - `/tmp/hf-apple-profile/results/full-gpu.mp4` - `/tmp/hf-apple-profile/results/summary.json` All four benchmark MP4s completed `ffprobe` and full `ffmpeg -f null` decode checks. ### Pixel comparison Compared decoded MP4 output between software-browser and browser-GPU renders: - Apple presentation: - 4240 frames compared - 636 exact matching decoded frame hashes - 3604 different decoded frame hashes - Average PSNR: 57.79 dB - `css-spinner-render-compat` clean fixture: - 120 frames compared - 0 exact matching decoded frame hashes - Average PSNR: 61.57 dB Interpretation: browser GPU output is not strict hash/pixel-identical to the software-browser path after lossy H.264 encode, but the measured deltas are visually tiny. Above 50 dB PSNR is typically visually indistinguishable for normal video review. Use `--no-browser-gpu` or Docker when strict cross-run/cross-machine reproducibility matters more than local speed. ### Browser verification - Started HyperFrames Studio preview for `packages/producer/tests/css-spinner-render-compat/src`. - Used `agent-browser` to open `http://localhost:5191#project/src` and verify the composition loaded in Studio. - Screenshots: - `/tmp/hf-gpu-browser-proof/preview-loaded.png` - `/tmp/hf-gpu-browser-proof/preview-playing.png` - `/tmp/hf-gpu-browser-proof/preview-frame-60.png` - Agent-browser recordings: - `/tmp/hf-gpu-browser-proof/preview-playback.webm` - `/tmp/hf-gpu-browser-proof/preview-seek.webm` ## Notes - Browser GPU is enabled automatically for local CLI renders and disabled in Docker. - `--no-browser-gpu` is the opt-out for software Chrome/WebGL capture. - `--gpu` remains encoder-only and opt-in. - The Apple presentation zip has existing lint errors around unmanaged nested videos and imperative media `play()` calls. The benchmark still compares the same supplied source across modes, but it should not be treated as a clean deterministic-composition fixture. |
||
|
Miguel Ángel
|
bcd7230557 |
fix: fall back to screenshot mode when any CDP call times out during calibration (#567)
## Summary - **Root cause**: `shouldFallbackToScreenshotAfterCalibrationError` only matched `HeadlessExperimental.beginFrame` errors. When a composition with many heavy videos (e.g. 7 videos with sparse keyframes) caused Chrome to be unresponsive in BeginFrame mode during calibration, a `Runtime.callFunctionOn timed out` or `Runtime.evaluate timed out` error was treated as an opaque failure — not a BeginFrame-mode signal. The render kept BeginFrame mode, spawned 3 workers with `captureCostMultiplier=8`, and all 3 workers also timed out initialising their sessions (0 frames captured, render fails). - **Fix**: Add `Runtime.callFunctionOn timed out` and `Runtime.evaluate timed out` to the screenshot-fallback pattern. Any CDP call timing out during the short-timeout calibration probe now routes the render into single-worker screenshot mode — the safe fallback already used for explicit BeginFrame timeouts. - **Result**: Compositions that overwhelm BeginFrame mode (reported in #566: 7 videos, 8 audios, 330-second render) now fall back cleanly and complete instead of failing with 0 frames. ## Test plan - [x] New unit test: `falls back to screenshot mode after Runtime.callFunctionOn timeout during calibration` — asserts both `Runtime.callFunctionOn timed out` and `Runtime.evaluate timed out` return `true` - [x] All existing `capture calibration safeguards` unit tests still pass - [x] Pre-commit hooks (lint, format, typecheck) pass Fixes #566 |
||
|
Miguel Ángel
|
8b234be20c |
fix(producer): handle no-audio stream gracefully in resolveMediaDuration
When an <audio> element referenced a file with no audio stream (e.g. a
silent screen-recording used as an audio src, or a video-only clip),
extractAudioMetadata threw "[FFmpeg] No audio stream found". The error
propagated uncaught through Promise.all in compileHtmlFile and crashed
the entire render.
Apply the same graceful-skip pattern already used for missing files and
failed downloads: catch the probe error and return { duration: 0 } so
the element is excluded from the composition without aborting the render.
Confirmed via 7 production HyperframeRenderWorkflow failures all sharing
the same TemporalMagicEditActivity.RENDER_PREVIEW stack trace.
|
||
|
Vance Ingalls
|
ad44c3133a | perf(hdr): reduce layered composite overhead (#538) | ||
|
Miguel Ángel
|
54cc0f1cba | test: cover sub-composition audio extraction | ||
|
Vance IngallsandClaude Opus 4.6
|
8f97edb2b9 |
fix(hdr): filter zero-opacity elements and support overflow:hidden clip rects (#522)
* fix(hdr): filter zero-opacity elements and support overflow:hidden clip rects in HDR compositor
Two bugs in the HDR render pipeline:
1. Child data-start elements inside a parent with opacity:0 were still
composited as independent layers, painting over content in later scenes.
Fix: filter elements with effective opacity 0 before groupIntoLayers().
2. CSS overflow:hidden on ancestor elements was ignored for HDR video layers,
causing videos inside clipped containers (e.g. split-screen halves) to
render full-frame. Fix: add clipRect to ElementStackingInfo, compute it
from ancestor overflow:hidden in queryElementStacking(), and crop the
source buffer to clip bounds before blitting in blitHdrVideoLayer().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): move opacity filter into blit loop to preserve hide-list correctness
The previous approach filtered zero-opacity elements before groupIntoLayers(),
which broke the DOM screenshot hide-list — invisible video elements' <img>
replacements weren't properly hidden from sibling layer screenshots, causing
the vignelli-stacking regression.
Fix: keep all elements in groupIntoLayers() for correct hide-list generation.
Skip zero-opacity HDR elements only during the actual blit step with an early
`continue` in the compositing loop.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): route identity-matrix HDR elements through region blit for clip rect support
parseTransformMatrix returns a valid matrix even for untransformed HDR
elements (Chrome reports matrix(1,0,0,1,0,0)). This made the affine blit
path always run, bypassing the region blit path which is the only one that
applies clip rects from overflow:hidden ancestors.
Fix: detect identity matrices and route them through the region path so
the cropRgb48le clip logic is reachable for split-screen layouts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): handle translation-only matrices for clip rect support
The previous isIdentity check only caught matrix(1,0,0,1,0,0). Elements
with layout translation (e.g. right-half split at left:960px reporting
matrix(1,0,0,1,960,0)) still routed through the affine path where clip
rects are not applied.
Fix: check for translation-only matrices (scale=1, rotation=0, any tx/ty)
and route those through the region blit path. el.x/el.y from
getBoundingClientRect already include the translation, so the region path
handles positioning correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(render): auto-detect HDR from media probes, add --sdr flag
Replace the --hdr opt-in model with automatic detection. When no flags
are passed, the renderer probes all video/image sources and enables HDR
output if any HDR color space is detected. Existing --hdr flag becomes
a force override. New --sdr flag forces SDR output.
Behavior matrix:
(no flags) + HDR content → HDR output
(no flags) + SDR content → SDR output
--hdr → force HDR (defaults to HLG if no HDR sources)
--sdr → force SDR (skips probing)
--hdr --sdr → error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat(render): auto-detect HDR from media probes, add --sdr flag"
This reverts commit
|
||
|
Vance IngallsandClaude Opus 4.6
|
8e5593b6ba |
feat(render): auto-detect HDR from media probes, add --sdr flag (#526)
* feat(render): auto-detect HDR from media probes, add --sdr flag Replace the --hdr opt-in model with automatic detection. When no flags are passed, the renderer probes all video/image sources and enables HDR output if any HDR color space is detected. Existing --hdr flag becomes a force override. New --sdr flag forces SDR output. Behavior matrix: (no flags) + HDR content → HDR output (no flags) + SDR content → SDR output --hdr → force HDR (defaults to HLG if no HDR sources) --sdr → force SDR (skips probing) --hdr --sdr → error Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: align HDR auto-detect docs and tests --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
36b3fc8cd9 | fix: budget workers for expensive captures | ||
|
Miguel Ángel
|
31afb4c1b3 |
Merge pull request #518 from heygen-com/fix/studio-preview-runtime-visibility
fix: isolate Studio sub-composition previews |
||
|
Miguel Ángel
|
613f97793e | test: add preview regression gate | ||
|
Miguel Ángel
|
6ea2e54fcf | fix: preserve anonymous inlined composition roots | ||
|
Miguel Ángel
|
d3b2900d6b | fix: preserve inferred composition ids when inlining |