mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 07:19:52 +00:00
67901349eb03685dd3366fd43f32d05a0f433d38
97
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 James
|
453bd6e313 | fix(engine,producer): reject alpha+supersample, unit-test clip plumbing | ||
|
James
|
2b2281d490 | refactor: dedupe resolution presets and clean up 4k stack | ||
|
James
|
1545763ea3 | fix(cli,producer): cross-multiply aspect check, CLI HDR guard, honest banner | ||
|
James
|
e07aeba213 | feat(cli): add --resolution flag to hyperframes render for one-line 4k | ||
|
James
|
8203005488 | fix(engine): byte-budget the frame data uri cache to bound memory at 4k | ||
|
Vance Ingalls
|
b0fb664873 |
fix: render shader transitions for SDR compositions (#640)
* feat: cache shader transition preview frames * fix: move shader transition loading to player * fix: render shader transitions for sdr compositions |
||
|
James
|
7d1d8ead60 |
fix(producer): sample PSNR checkpoints from common duration of rendered+snapshot
Four regression tests (font-variant-numeric, many-cuts, missing-host-comp-id, variables-prod) failed on this PR with `Unable to parse PSNR output at <last checkpoint>s`. Root cause: the harness derived all 100 checkpoints from the *rendered* video's container duration, then asked ffmpeg's PSNR filter to compare the same frame index from both videos. The encoder changes earlier in this PR add `-avoid_negative_ts make_zero` to the mux step. With AAC audio that shifts the first audio sample to t=0 instead of the encoder-delay offset, extending reported container duration by ~20ms without changing video frame count. For the four failing tests, the i=99 checkpoint then landed on a frame index that exists in the rendered video but not in the snapshot baseline (e.g. round(2.98998 * 24) = 72 in a 72-frame baseline). ffmpeg's PSNR filter ran on zero matched frames and emitted no `average:` line, so the parser threw. Fix: probe both videos and use min(rendered, snapshot) duration when spreading checkpoints. This is the correct semantics for symmetric PSNR comparison anyway — both videos must have a frame at every sampled time. The change is local to the harness; no encoder behavior changes, no baselines regenerated. Other regression tests with audio (chat, sub-composition-video, vignelli-stacking) passed because their checkpoint-99 frame index landed inside the baseline's frame range with several frames of slack. The four failing tests had round-number durations where a 20ms drift was enough to push the last checkpoint past `nb_frames - 1`. |
||
|
JamesandClaude Opus 4.7
|
f4ecf96918 |
fix(engine,cli,producer): address PR #627 review feedback
- engine/chunkEncoder, engine/streamingEncoder: extend `-bf 0` to GPU h264
paths (nvenc, qsv, vaapi) and `-b_strategy 0` for qsv so GPU-encoded
outputs avoid negative-DTS freezes too — not just SW libx264.
- engine/videoFrameExtractor: detect mid-path traversal (e.g.
`assets/../../foo.mp4`) by normalizing first and re-anchoring at the
project root. Adds a regression test.
- engine/videoFrameExtractor: dedupe stderr "src not resolvable" warnings
by `video.src` so a comp with N broken sources logs once, not N times.
- engine/videoFrameExtractor.test: drop dynamic `require("node:fs")`,
use ES `import { writeFileSync } from "node:fs"`.
- engine/ffprobe: extract `readTagCI` helper for case-insensitive ffprobe
tag reads (will recur for other libavformat-versioned sidecar tags).
- cli/background-removal/pipeline: collapse Quality / QUALITIES /
QUALITY_CRF / DEFAULT_QUALITY / isQuality surface using
`Quality = keyof typeof QUALITY_CRF`.
- producer/renderOrchestrator: replace `v.src.startsWith("/")` with
`isAbsolute(v.src)` in the HDR probe path so Windows absolute paths
(`C:\...`) aren't treated as relative — matches the audioMixer guard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
2f96d5c7ab |
fix(engine,producer): URL-clamp sub-comp src paths and warn on silent extraction misses
A <video src='../assets/foo.mp4'> inside a sub-composition silently dropped from extraction; the rendered output froze on the first decoded frame for the entire clip, with no error in stdout. Root cause: browser URL resolver clamps '..' at origin root (studio preview loads fine), but path.join(projectDir, '../assets/foo.mp4') normalizes to parent-of-project/assets/foo.mp4, which usually doesn't exist. existsSync returns false, extraction is skipped, no frame lookup is built, the per-frame injector has nothing to swap, and the <video> element's first decoded frame paints every screenshot. - Adds resolveProjectRelativeSrc in videoFrameExtractor that mirrors browser clamping (literal join first, then leading '..' stripped). - Surfaces a loud stderr warning when the resolver misses. - Mirrors fix in audioMixer.ts (same bug for <audio src='../'>) and renderOrchestrator HDR probe loop. - +6 regression tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
58b4234809 |
test(producer): add variables-prod regression test for the variables stack
End-to-end Docker regression test that exercises the full variables
chain: meta.json renderConfig.variables → harness → createRenderJob →
RenderConfig → CaptureOptions → engine evaluateOnNewDocument →
window.__hfVariables → getVariables() → DOM text → rendered pixels.
Fixture (packages/producer/tests/variables-prod/):
- src/index.html: composition with three declared variables (title,
subtitle, bgColor) read via window.__hyperframes.getVariables() and
rendered as positioned text on a colored background. No animation —
keeps the regression frame-stable so it isolates "did the variables
flow through?" from motion concerns.
- meta.json: tags ["variables", "composition"] (runs in the existing
fast shard's tag filter), renderConfig.variables provides override
values the baseline reflects ("Override Title", "Override subtitle",
#0a3d62). Defaults would produce a visibly different frame, so a
failing baseline that reflects defaults means the variables didn't
propagate.
- output/output.mp4: Docker-generated baseline per the project's
CLAUDE.md golden-baseline rule.
Harness change (packages/producer/src/regression-harness.ts):
- TestMetadata.renderConfig gains an optional variables field,
validated as a JSON object in the meta.json validator.
- The createRenderJob call site forwards renderConfig.variables to
RenderConfig.variables, which the engine already consumes via
evaluateOnNewDocument (PR #600).
Verified: docker:test variables-prod passes 100/100 visual checkpoints
and audio correlation 1.000 against the committed baseline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
c0d75a5268 |
feat(core,cli,engine,producer): add getVariables() helper and --variables render flag
Adds the parametrized-render primitive from hf#592 by reusing the existing
data-composition-variables schema as the source of declared defaults.
- Runtime helper window.__hyperframes.getVariables() (also exported from
@hyperframes/core) reads data-composition-variables defaults from the
document root and merges window.__hfVariables (CLI override) on top.
Returns Partial<T> for typed access; supports a generic for editor
ergonomics. Same code path runs in dev preview and at render time.
- CLI render --variables '<json>' / --variables-file <path> populates the
override. Mutually exclusive; fail-fast on conflicting flags, missing
file, unparseable JSON, or non-object payloads. parseVariablesArg is
exported as a pure function so validation paths stay unit-testable.
- Engine injects window.__hfVariables via evaluateOnNewDocument before
any page script runs, so the helper sees the merged values on its
first call. Empty payloads are skipped to avoid pointless init scripts.
- Producer threads variables through RenderConfig and into the engine's
CaptureOptions; Docker mode forwards --variables to the in-container
CLI invocation via dockerRunArgs.
Composition authors declare variables once on the root <html> element:
<html data-composition-variables='[
{"id":"title","type":"string","label":"Title","default":"Hello"}
]'>
and read them in any composition script:
const { title } = window.__hyperframes.getVariables();
A render with `--variables '{"title":"Q4 Report"}'` overrides the default
without modifying the composition source. Missing keys fall through to
the declared defaults, so dev preview and CLI renders without --variables
behave identically.
This is PR 1 of a 4-PR stack. Sub-comp per-instance scoping (carrying
host data-variable-values through the inlined sub-comp's getVariables()
call) lands in PR 2; schema validation and lint in PR 3; skill / scaffold
distribution in PR 4.
Tests: 9 new unit tests for getVariables() (jsdom), 11 new CLI tests
covering parseVariablesArg validation paths and Docker passthrough,
2 new dockerRunArgs assertions for the --variables flag. All existing
tests green (core 611, cli 208, engine 519).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
2a897d351c |
fix: address PR #596 review issues (#597)
## Summary Fixes three issues identified in the [post-merge review](https://github.com/heygen-com/hyperframes/pull/596#pullrequestreview-4214283515) of PR #596: - **P1 (cache bypass):** When `extractCacheDir` is set, extracted frames live outside `compiledDir`, so `createCompiledFrameSrcResolver` rejects them and every frame falls back to base64 data URIs. Fix: symlink cached frame directories into `compiledDir/__hyperframes_video_frames/` after extraction and remap `framePaths` so the served-frame fast path works. - **P2 (pooled browser stale state):** `closeCaptureSession` force-killed the Chrome process on timeout via raw `SIGKILL` without clearing `pooledBrowser` / `pooledBrowserRefCount`, leaving other sessions with a dead browser reference. Fix: add `forceReleaseBrowser()` in `browserManager` that atomically clears pool state before killing the process. - **P3 (reserved chars in URLs):** `createCompiledFrameSrcResolver` encodes path segments with `encodeURIComponent`, but the file server used `c.req.path` (which only applies `decodeURI`) to look up files on disk. Video IDs containing `#`, `?`, or `%` produced 404s. Fix: apply `decodeURIComponent` per path segment in the file server's catch-all route. ## Test plan - [x] `createCompiledFrameSrcResolver` tests: symlinked cache paths resolve to served URLs; cache-external paths return null; reserved characters encode correctly - [x] `forceReleaseBrowser` tests: kills process + disconnects; tolerates already-killed process - [x] `createFileServer` test: `video%231/frame.jpg` serves file from `video#1/frame.jpg` on disk - [x] Typecheck: engine + producer pass - [x] Lint + format: 0 warnings, 0 errors |
||
|
Miguel Ángel
|
4750a981dd | fix: speed up video frame injection renders (#596) | ||
|
Miguel Ángel
|
15ee63c6e7 |
fix: harden CLI edge-case repros (#591)
## Problem I reproduced the selected open issue batch one by one and confirmed the reports were valid. The fixes all touch the CLI/runtime capture boundary, then the follow-up regression run exposed one over-broad runtime change in sub-composition host visibility and one CI-only baseline trap. Closes #590, #589, #588, #587, #586, and #584. ## What this fixes ### CLI/runtime edge cases - Makes the GSAP infinite-repeat lint rule ignore JavaScript comments, so literal `repeat:-1` text in comments is not flagged. - Lets the compositions CLI inspect `<template>` content, count visual-only template descendants, estimate simple GSAP durations, and suppress root `data-start` warnings in sub-composition lint mode. - Preserves runtime bootstrap scripts when body scripts are coalesced, and injects the runtime into a real `<head>` when source HTML has no head. - Keeps #589 fixed by loading and rendering template-wrapped sub-composition content, while restoring host visibility to the shorter of the authored parent clip window and the child composition live timeline. - Resolves snapshot/validate viewport size from root `data-width` / `data-height` instead of falling back to 1920x1080. - Skips fully off-frame text boxes during contrast sampling and bounds-checks ring samples so contrast output no longer emits `null:1` / `NaN:1`. - Marks muted videos as `data-has-audio="false"` in the core timing compiler, which fixes the same-src muted `<video>` + separate `<audio>` StaticGuard case. - Keeps user-authored `hf-seek` listeners reachable during capture by preventing author scripts from being merged into the runtime bootstrap path. ### Shared helper cleanup - Removes the stale producer-local timing compiler duplicate; producer compilation now consumes the core timing compiler. - Centralizes HTML document helpers in core: fragment parsing, embedded runtime stripping, head/body script injection, and early-head injection. - Centralizes the CLI layout/snapshot static HTML server. - Adds browser-safe core subpath helpers for Lottie readiness and CLI screenshot clip calculation; Studio's Vite config keeps the screenshot clip helper self-contained so clean-checkout test startup does not value-import core `.ts` source. - Replaces the engine parity-contract copy with a core re-export. - De-duplicates render-job cleanup and Studio static file-serving callbacks. ### Regression hardening - Replaces the embedded-runtime script stripping regex with a script-tag scanner that handles closing tags like `</script >`. - Escapes inline script bodies before wrapping them in `<script>` tags, so authored `</script` and `<!--` text cannot break out of the injected wrapper script. - Shares media-duration clamping between core and producer, with a 50 ms tolerance for ffprobe precision drift between local and CI media stacks. - Pins the affected style fixture SFX durations in source so style-1 and style-9 compile deterministically. - Restores the `vfr-screen-recording` video golden to the CI-stable baseline; the current CI failure showed the Linux render matches the old golden, while the locally refreshed macOS golden was the mismatch. ## Root cause The CLI paths had accumulated assumptions that held for simple direct-root landscape compositions but not for current composition patterns: DOM queries did not enter template content, snapshot/validate used a fixed viewport, runtime and author scripts shared a coalescing bucket, and timing compilation treated every video as audio-bearing unless authors manually overrode it. The style shard failures were not product regressions. Local and CI media probing disagreed on the short SFX clip duration by about 45 ms, and the compiler was clamping authored durations to the locally probed value. The shared clamp tolerance preserves explicit author/source durations for small probe precision differences while still clamping real overflows. The vfr fast-shard failure was a bad baseline refresh: CI actual frames matched the old `vfr-screen-recording` baseline at 40+ dB PSNR, but mismatched the macOS-refreshed golden at ~18-22 dB. The fix is to keep the Docker/Linux-stable video golden and only retain the deterministic compiled snapshot change. The sub-composition regression came from treating a host's authored parent window as the only visibility boundary. That made settled child overlays stay visible after their own live GSAP timeline ended. The corrected runtime behavior respects both contracts: parent clips still bound where the host can appear, and the child live timeline can end the host earlier. ## Verification ### Local checks - `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bun run --cwd packages/core test src/runtime/init.test.ts` - `bun run --cwd packages/cli test src/commands/compositions.test.ts src/utils/compositionViewport.test.ts` - `bun run build:hyperframes-runtime` - `bun run --cwd packages/producer test --keep-temp --sequential style-12-prod style-5-prod` - `bun run --cwd packages/producer test --sequential vfr-screen-recording hdr-hlg-regression style-7-prod` - `bun run --cwd packages/core test src/compiler/htmlCompiler.test.ts src/compiler/timingCompiler.test.ts src/index.test.ts` - `bunx oxfmt --check packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bunx oxlint packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bun run --cwd packages/core typecheck` - `bun run --cwd packages/producer typecheck` - `bun run --cwd packages/producer test --sequential style-1-prod style-9-prod` - `bun run --filter @hyperframes/studio test` with `packages/core/dist` temporarily hidden to simulate clean-checkout config loading - `git diff --check` ### CI artifact checks - Inspected failed run `25225854394` job `73969147096`: style-1 failed only on `click-sfx` `1.044898` vs `1` duration/end. - Inspected failed run `25225854394` job `73969147061`: style-9 failed only on SFX `1.044898`-based duration/end mismatches. - Inspected failed run `25225854394` job `73969147048`: `vfr-screen-recording` compilation/audio passed, visual failed after comparing against the macOS-refreshed golden. - Compared the first 10 uploaded CI vfr failure frames against the restored old baseline; minimum PSNR was `40.444705`, above the fixture threshold of `28`. ### Repro checks - `bun packages/cli/src/cli.ts lint /tmp/hf-590-repro` now passes without `gsap_infinite_repeat`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-587-repro --at 0.5 --timeout 1000` now writes a 1080x1920 PNG. - `bun packages/cli/src/cli.ts validate /tmp/hf-588-repro --timeout 500` no longer emits `null:1` / `NaN:1` contrast output. - `bun packages/cli/src/cli.ts validate /tmp/hf-586-repro --timeout 500 --contrast false` no longer emits the muted-video StaticGuard contract error. - `bun packages/cli/src/cli.ts compositions /tmp/hf-589-gsap-repro` now reports `foo 0.5s 1920x1080 1 element`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-589-gsap-repro --at 0.25 --timeout 2000` captures the expected template-backed red frame. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-584-repro --at 0.5,1.5 --timeout 500` captures the expected post-seek green frame. ### Browser verification - Refreshed the local side-by-side comparison page at `qa-artifacts/pr-591-video-compare/index.html`. - Served the comparison page locally and used `agent-browser` to load `style-12-prod`, play both videos quickly to the failed window, pause, and inspect the side-by-side frame. - Browser proof screenshot: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12-labeled.png`. - Browser proof recording: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12.webm`. - Earlier Studio proof artifacts remain local-only: `qa-artifacts/dedupe-refactor-preview.png`, `qa-artifacts/dedupe-refactor-preview-after-play.png`, `qa-artifacts/dedupe-refactor-preview.webm`. ## Notes - Browser proof and CI diagnostic artifacts are intentionally local-only and not committed. - Studio's Vite config intentionally keeps the thumbnail clip helper inline because Vite/Vitest config startup runs through Node's loader before package source `.ts` imports are transformed. - The committed PR diff changes `vfr-screen-recording/output/compiled.html` but no longer changes `vfr-screen-recording/output/output.mp4` relative to `main`. - I attempted a local `linux/amd64` Docker validation to mirror CI, but the local Docker build was blocked by Debian package download failures. The arm64 Docker image also cannot launch the x64 Puppeteer headless shell under OrbStack. The vfr baseline decision is therefore based on the uploaded CI artifact comparison above. - I kept this validated issue batch in one PR because the fixes overlap the same CLI/runtime capture surfaces. |
||
|
Miguel Ángel
|
dde26cf62d | feat: default streaming encode for sequential renders (#579) | ||
|
Miguel Ángel
|
6a59ef6106 |
fix: skip metadata waits for injected video frames (#575)
## Problem Closes #574. On Windows with cached headless-shell Chrome, a composition that reuses the same video file in three timeline clips can fail before frame capture starts: ```html <video id="video1" src="1.mp4" data-start="0" muted data-duration="4" data-track-index="0" data-media-start="0"></video> <video id="video2" src="1.mp4" data-start="4" muted data-duration="4" data-track-index="0" data-media-start="4"></video> <video id="video3" src="1.mp4" data-start="8" muted data-duration="4" data-track-index="0" data-media-start="8"></video> ``` The reported render reaches video frame extraction, then dies at frame-capture initialization with: ```text [FrameCapture] video metadata not ready after 45000ms. Video elements must load metadata before capture starts. ``` The important detail is that by this stage HyperFrames has already extracted video pixels through FFmpeg. Native Chromium video metadata is only being waited on for DOM layout stability, not because Chromium is the source of rendered pixels. ## Root Cause The render pipeline has two separate media responsibilities: - FFmpeg extracts video frames and audio from declared media. - Chromium owns DOM layout and capture, while injected FFmpeg frames supply the video pixels before each captured frame. Before this PR, every capture session still waited for every DOM `<video>` to reach `readyState >= 1` unless the element was a native HDR exception. That made native browser media metadata a hard render prerequisite even when the browser would not decode or provide the final video pixels. That is why the issue fails at `25% Starting frame capture`: FFmpeg extraction has already succeeded, but capture initialization blocks on repeated native `<video src="1.mp4">` metadata loading in cached Windows headless-shell Chrome. There was a second constraint: the readiness wait also prevents first-frame layout bugs. If a skipped `<video>` has no native metadata, Chromium can use the default `300x150` intrinsic video size, which breaks layouts such as `width: 100%; height: auto` before the first injected frame. The fix therefore must not simply skip all video readiness waits; it must provide dimensions for any skipped videos. ## What This Fixes - Treats videos with successfully extracted FFmpeg frames and usable dimensions as out-of-band rendered video sources. - Skips native browser metadata readiness waits for those extracted videos because Chromium is not responsible for their pixels. - Passes FFmpeg-probed dimensions into capture as `videoMetadataHints`. - Applies those hints before the readiness wait in both screenshot and BeginFrame initialization paths. - Sets missing `width` / `height` attributes and an explicit `aspect-ratio` only when the element does not already provide one, preserving author styles where present. - Keeps native HDR video IDs in the skip list, preserving the existing HEVC/HDR behavior where Chrome may not decode the source but FFmpeg/native HDR compositing can still render it. - Uses one `buildCaptureOptions()` helper so calibration, HDR DOM capture, streaming capture, parallel capture, and sequential capture receive the same skip IDs and metadata hints. - Adds tests for the skip-list and metadata-hint contract. - Adds a Windows CI regression that reproduces the issue shape after the canary render warms the cached-browser path. ## Reviewer Map Primary files: - `packages/producer/src/services/renderOrchestrator.ts` - `collectVideoReadinessSkipIds()` includes native HDR IDs plus extracted videos that have finite positive FFmpeg dimensions. - `collectVideoMetadataHints()` converts extracted FFmpeg metadata into capture hints. - `buildCaptureOptions()` threads `skipReadinessVideoIds` and `videoMetadataHints` into every capture path. - `packages/engine/src/services/frameCapture.ts` - `applyVideoMetadataHints()` runs in the page before video readiness polling. - Both screenshot and BeginFrame initialization call it before checking non-skipped videos for `readyState >= 1`. - `packages/engine/src/types.ts` - Adds `CaptureVideoMetadataHint` and documents that readiness skips should be paired with metadata hints when layout may depend on intrinsic dimensions. - `packages/producer/src/services/renderOrchestrator.test.ts` - Covers that extracted videos with dimensions are skipped, invalid dimensions are not, native HDR IDs are preserved, and hints are stable/sorted. - `.github/workflows/windows-render.yml` - Adds the issue #574 Windows regression with the exact three-clip markup and a generated deterministic `1.mp4`. ## Why This Is Safe The skip is intentionally gated: - A standard video is skipped only after `extractAllVideoFrames()` succeeded for that video and returned usable dimensions. - Videos with invalid dimensions are not skipped, so the old browser readiness guard still applies. - DOM videos are still present for layout and element bounds; only the native metadata wait is skipped for sources whose pixels come from FFmpeg injection. - Metadata hints are applied conservatively: existing `width`, `height`, and explicit `aspect-ratio` are not overwritten. - Non-extracted videos, images, fonts, page readiness, and `window.__hf` readiness keep the existing waits. - The fix is not limited to the sequential path from the issue; it is threaded through calibration, HDR DOM capture, streaming encode, parallel capture, and sequential capture. A first local revision skipped readiness too broadly and caused `overlay-montage-prod` first-frame layout shrinkage. The current version fixes that by pairing skips with FFmpeg metadata hints; `overlay-montage-prod` now passes and is listed in verification below. ## Verification ### Root-Cause Reproduction Before Fix The reporter did not attach the actual `1.mp4`, so the regression uses the exact issue markup and a deterministic generated 12s H.264 file named `1.mp4`. I reproduced the failure in GitHub Actions by running this branch's new Windows workflow against unpatched `main`: ```bash gh workflow run windows-render.yml --repo heygen-com/hyperframes --ref fix/reused-video-metadata -f ref=main ``` That means the workflow contains the new issue #574 regression, but the code under test is `main` without this fix. Baseline failure: - Run: https://github.com/heygen-com/hyperframes/actions/runs/25174603730 - Failed job: https://github.com/heygen-com/hyperframes/actions/runs/25174603730/job/73803179086 - Checkout proof: `ref: main`, `origin/main`, commit `8662598a3ac64018a2999d189ffb369e6d46b53a`. - Failure proof: `Browser: cache`, `staticDuration:12`, `videoCount:3`, then `25% Starting frame capture` -> `[FrameCapture] video metadata not ready after 15000ms`. This is the same failure class as the issue, on Windows, in cached-browser mode, before the fix. ### Fixed Windows Regression The same regression passes on this PR branch: - Run: https://github.com/heygen-com/hyperframes/actions/runs/25175048215 - Passing job: https://github.com/heygen-com/hyperframes/actions/runs/25175048215/job/73804781017 - Checkout proof: PR merge contains `79d6b41c9f2ba137cbfb9301678e0815b16c4f5a` merged into `8662598a3ac64018a2999d189ffb369e6d46b53a`. - Passing proof: `Browser: cache`, `staticDuration:12`, `videoCount:3`, `25% Starting frame capture`, captures `360/360` frames, renders `issue-574.mp4`, and `ffprobe` verifies `1920x1080 @ 30/1, 12s`. ### Local Checks - `bun run build:hyperframes-runtime` - `bunx vitest run packages/producer/src/services/renderOrchestrator.test.ts` - `bun run --filter @hyperframes/producer typecheck` - `bun run --filter @hyperframes/engine typecheck` - `bunx oxlint packages/engine/src/services/frameCapture.ts packages/engine/src/types.ts packages/engine/src/index.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts` - `bunx oxfmt --check .github/workflows/windows-render.yml packages/engine/src/services/frameCapture.ts packages/engine/src/types.ts packages/engine/src/index.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts` - `git diff --check` - Lefthook pre-commit: lint, format, typecheck where applicable - Lefthook commit-msg: commitlint ### Local Render Checks - Created `/tmp/hf-issue-574-repro` with the issue shape: three clips using the same `1.mp4`, `data-media-start=0/4/8`, 12s total. - `PRODUCER_PLAYER_READY_TIMEOUT_MS=5000 bun packages/cli/src/cli.ts render /tmp/hf-issue-574-repro --workers 1 --quality draft --fps 30 --output /tmp/hf-issue-574-h264-fixed-v2.mp4` -> completed. - Created `/tmp/hf-issue-574-prores` with the same three-clip shape using one FFmpeg-readable ProRes `.mov`, which exercises the browser-metadata failure class because Chromium should not be needed to decode the source. - `PRODUCER_PLAYER_READY_TIMEOUT_MS=3000 bun packages/cli/src/cli.ts render /tmp/hf-issue-574-prores --workers 1 --quality draft --fps 30 --output /tmp/hf-issue-574-prores-fixed-v2.mp4` -> completed. - `bun run --filter @hyperframes/producer test --sequential --keep-temp overlay-montage-prod` -> passed; this guards against skipped metadata shrinking `height:auto` video layout before the first injected frame. - `ffmpeg -v error -i /tmp/hf-issue-574-prores-fixed-v2.mp4 -f null -` - `ffmpeg -v error -i /tmp/hf-issue-574-h264-fixed-v2.mp4 -f null -` - `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-issue-574-h264-fixed-v2.mp4` -> H.264, 320x180, 30fps, 12.0s. ### Current PR Checks - Windows render verification: pass on https://github.com/heygen-com/hyperframes/actions/runs/25175048215. - Windows tests: pass on https://github.com/heygen-com/hyperframes/actions/runs/25175048215. - Main CI build/lint/typecheck/test/smoke jobs: pass on https://github.com/heygen-com/hyperframes/actions/runs/25175048175. - Regression shards observed passing include HDR, render-compat, styles A-G, and `overlay-montage-prod`. At the time this body was updated, the `fast` regression shard was still in progress in run https://github.com/heygen-com/hyperframes/actions/runs/25174515546. ### Browser Verification - Used `agent-browser` to open `file:///tmp/hf-issue-574-h264-fixed-v2.mp4` and verify the rendered output displays in Chromium. - Screenshot: `.debug/issue-574/h264-output-page.png` - Agent-browser recording: `.debug/issue-574/h264-output-playback.webm` ## Notes / Caveats - The reporter's exact `1.mp4` was not attached to #574. The committed Windows regression uses a generated deterministic H.264 file with the same filename and exact markup from the issue. - The exact H.264 issue shape did not reproduce the timeout on this macOS/system-Chrome machine before the fix; it rendered successfully locally. The GitHub Actions baseline above reproduces it on Windows/cache without the fix. - The Windows fixture intentionally runs after the existing canary render so the browser path is `Browser: cache`, matching the reporter's environment. - The generated fixture emits sparse-keyframe warnings. Those warnings are expected and are not the failure being fixed; the baseline failure occurs before any frame capture because native browser video metadata never becomes ready. - Browser proof artifacts are local-only under `.debug/issue-574/` and intentionally not committed. |
||
|
Miguel Ángel
|
395fb9c084 |
feat: add browser GPU render mode (#571)
## Problem HyperFrames already had `--gpu`, but that flag only controlled FFmpeg hardware encoding. The browser capture path still forced Chrome/WebGL through SwiftShader software GL via `--use-angle=swiftshader`, so WebGL-heavy local renders could leave the biggest bottleneck on the CPU path. That made the existing flag naming easy to misread: `--gpu` sounded like it accelerated the whole render, but it did not change the browser frame-capture backend. ## What this fixes - Enables host browser GPU acceleration automatically for local CLI renders. - Adds `--no-browser-gpu` as the local opt-out for software Chrome/WebGL capture. - Keeps `--browser-gpu` as an explicit local browser-GPU request. - Adds `browserGpuMode: "software" | "hardware"` to engine config, with `PRODUCER_BROWSER_GPU_MODE` env support for lower-level producer users. - Keeps Docker browser capture on the deterministic software path. - Maps hardware browser GPU mode to platform-native Chrome backends: - macOS: Metal-backed ANGLE - Windows: D3D11-backed ANGLE - Linux: EGL - Blocks explicit `--browser-gpu --docker` with a clear error because Docker browser GPU passthrough is not cross-platform. - Clarifies docs so `--gpu` means FFmpeg encoder GPU and browser GPU means Chrome/WebGL capture GPU. - Keeps encoder backend selection auto-detected from FFmpeg capabilities: - NVIDIA: NVENC - macOS: VideoToolbox - Linux: VAAPI - Intel: QSV ## Why two flags There are two separate GPU surfaces in the render pipeline: 1. Browser GPU controls Chrome frame capture. - Affects WebGL, canvas, CSS rendering, compositing, and screenshot capture inside the browser. - This is enabled automatically for local CLI renders. - Use `--no-browser-gpu` when you want the software browser baseline. 2. `--gpu` controls FFmpeg video encoding. - Affects the final encode step after frames have already been captured. - The concrete encoder is auto-detected from the host FFmpeg build and hardware. - It can be faster for some machines/codecs, but it is not equivalent to browser rendering acceleration. The controls stay independent because users may want: - `hyperframes render` for the fast local default with browser GPU capture. - `hyperframes render --no-browser-gpu` for the software-browser local baseline. - `hyperframes render --gpu` for browser GPU capture plus hardware FFmpeg encoding. - `hyperframes render --no-browser-gpu --gpu` for software browser capture plus hardware FFmpeg encoding. - `hyperframes render --docker` for deterministic browser capture. ## Why `--gpu` does not imply browser GPU Keeping `--gpu` scoped to FFmpeg encoding avoids a semantic break and keeps the risk profile explicit: - `--gpu` already means encoder acceleration. Expanding it to also change Chrome capture would silently alter behavior for users who only wanted hardware encoding. - Browser GPU and encoder GPU have different portability. Encoder GPU can work in Docker when the host exposes the right devices; browser GPU passthrough is not cross-platform, so this PR intentionally blocks explicit `--browser-gpu --docker`. - The Apple presentation benchmark shows why the controls should stay separate: browser GPU capture was the useful improvement, while macOS VideoToolbox via `--gpu` was slower and produced larger output for this `standard` H.264 run. If HyperFrames later wants a single umbrella acceleration control, it should be explicit, for example `--acceleration browser|encoder|all` or `--gpu=browser|encoder|all`, rather than changing the meaning of the existing boolean `--gpu`. ## Root cause `buildChromeArgs()` always injected `--use-gl=angle --use-angle=swiftshader`. `disableGpu` only appended `--disable-gpu`; it did not provide a hardware-GPU mode. That made the public `--gpu` flag look broader than it was, because render capture stayed software-backed even when encoder GPU was requested. ## Verification ### Local checks - `bun install` - `bun run build:hyperframes-runtime` - `bun run --filter @hyperframes/engine test src/config.test.ts src/services/browserManager.test.ts` - `bun run --filter @hyperframes/cli test src/utils/dockerRunArgs.test.ts src/commands/render.test.ts` - `bun run --filter @hyperframes/cli typecheck` - `bun run --filter @hyperframes/engine typecheck` - `bun run --filter @hyperframes/producer typecheck` - `cd packages/producer && bunx vitest run src/services/renderOrchestrator.test.ts` - `bunx oxlint packages/cli/src/commands/render.ts packages/cli/src/commands/render.test.ts packages/cli/src/utils/dockerRunArgs.ts packages/cli/src/utils/dockerRunArgs.test.ts packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/browserManager.ts packages/engine/src/services/browserManager.test.ts packages/producer/src/services/renderOrchestrator.test.ts` - `bunx oxfmt --check ...` on changed source/docs files - `git diff --check` - `bun packages/cli/src/cli.ts render --help | rg -n "browser-gpu|no-browser-gpu|GPU"` - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --output /tmp/hf-auto-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict` - Render plan prints `GPU: browser GPU (auto)`. - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --no-browser-gpu --output /tmp/hf-software-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict` - Render plan does not print browser GPU. - `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --docker --browser-gpu --output /tmp/should-not-render.mp4` - Exits 1 with `Browser GPU is local-only`. - `buildDockerRunArgs()` regression coverage asserts Docker container args include `--no-browser-gpu`, preventing nested container renders from re-enabling browser GPU through the local CLI default. - `resolveBrowserGpuForCli()` regression coverage asserts `PRODUCER_BROWSER_GPU_MODE=software` opts out when no CLI browser-GPU flag is supplied, while explicit `--browser-gpu` / `--no-browser-gpu` still win. - `ffmpeg -v error -i /tmp/hf-auto-browser-gpu-smoke.mp4 -f null -` - `ffmpeg -v error -i /tmp/hf-software-browser-gpu-smoke.mp4 -f null -` - `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-browser-gpu-smoke.mp4` -> H.264, 1920x1080, 24fps, 5.0s ### Apple presentation benchmark Rendered `/Users/miguel07code/Downloads/apple-presentation.zip` as supplied after extracting to `/tmp/hf-apple-profile/apple-presentation`. Fixed settings: - 1920x1080 - 30fps - `standard` quality - 4240 frames - 141.32s duration - 8-worker cap; render auto-calibration used 6 capture workers - macOS host detected FFmpeg GPU encoder: `videotoolbox` | Mode | Equivalent flags after this PR | Wall time | vs software-browser baseline | Speed | Capture | Encode | Output | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | Software browser + CPU encode | `--no-browser-gpu` | 120.77s | baseline | 1.17x | 97.87s | 10.04s | 8.38MB | | Browser GPU + CPU encode | default local render | 70.10s | 42.0% faster | 2.02x | 50.72s | 9.91s | 8.39MB | | Software browser + encoder GPU | `--no-browser-gpu --gpu` | 133.16s | 10.3% slower | 1.06x | 103.58s | 18.31s | 25.43MB | | Browser GPU + encoder GPU | `--gpu` | 74.12s | 38.6% faster | 1.91x | 46.69s | 17.93s | 25.45MB | Result: browser GPU capture is the meaningful improvement for this WebGL/browser-capture-heavy presentation. VideoToolbox encoding was slower and produced larger files for this current `standard` H.264 path, so `--gpu` should stay separate and opt-in. Why `--gpu` plus browser GPU was slower than browser GPU alone: the combined run captured about 4.0s faster than browser GPU alone, but VideoToolbox encoding was about 8.0s slower than CPU x264 encoding, so the encode loss outweighed the capture gain. ### VideoToolbox flag check I also isolated the encode stage against the already-captured Apple frames to check whether macOS GPU encoding only needed special flags. `ffmpeg -h encoder=h264_videotoolbox` does not expose a CRF/CQ-style quality option like x264. It exposes bitrate-oriented and VideoToolbox-specific options such as `-b:v`, `-realtime`, `-profile`, `-coder`, `-prio_speed`, `-power_efficient`, and `-allow_sw`. That means our current `-q:v` mapping is not equivalent to x264 CRF and can produce very different bitrate/size behavior. Measured full-frame encode variants on this host: | VideoToolbox variant | Encode wall time | Output size | Bitrate | | --- | ---: | ---: | ---: | | Current `-q:v 64 -allow_sw 1` | 18.76s | 25.31MB | 1.43 Mbps | | Current without `-allow_sw 1` | 18.21s | 25.31MB | 1.43 Mbps | | `-b:v 500k -maxrate 750k -bufsize 1000k -profile high -coder cabac -realtime 1 -prio_speed 1 -power_efficient 0` | 20.58s | 7.42MB | 0.42 Mbps | | Same with `-b:v 1500k` | 20.84s | 16.70MB | 0.95 Mbps | | `-b:v 500k -profile baseline -coder cavlc -realtime 1 -prio_speed 1 -power_efficient 0` | 18.11s | 8.94MB | 0.51 Mbps | Conclusion: VideoToolbox can be made size/bitrate-predictable with explicit `--video-bitrate`, but the tested speed-oriented flags did not make it faster than CPU x264 wall time for this render. That reinforces keeping `--gpu` encoder acceleration explicit and separate from browser GPU capture. Artifacts from the local benchmark: - `/tmp/hf-apple-profile/results/cpu.mp4` - `/tmp/hf-apple-profile/results/browser-gpu.mp4` - `/tmp/hf-apple-profile/results/encoder-gpu.mp4` - `/tmp/hf-apple-profile/results/full-gpu.mp4` - `/tmp/hf-apple-profile/results/summary.json` All four benchmark MP4s completed `ffprobe` and full `ffmpeg -f null` decode checks. ### Pixel comparison Compared decoded MP4 output between software-browser and browser-GPU renders: - Apple presentation: - 4240 frames compared - 636 exact matching decoded frame hashes - 3604 different decoded frame hashes - Average PSNR: 57.79 dB - `css-spinner-render-compat` clean fixture: - 120 frames compared - 0 exact matching decoded frame hashes - Average PSNR: 61.57 dB Interpretation: browser GPU output is not strict hash/pixel-identical to the software-browser path after lossy H.264 encode, but the measured deltas are visually tiny. Above 50 dB PSNR is typically visually indistinguishable for normal video review. Use `--no-browser-gpu` or Docker when strict cross-run/cross-machine reproducibility matters more than local speed. ### Browser verification - Started HyperFrames Studio preview for `packages/producer/tests/css-spinner-render-compat/src`. - Used `agent-browser` to open `http://localhost:5191#project/src` and verify the composition loaded in Studio. - Screenshots: - `/tmp/hf-gpu-browser-proof/preview-loaded.png` - `/tmp/hf-gpu-browser-proof/preview-playing.png` - `/tmp/hf-gpu-browser-proof/preview-frame-60.png` - Agent-browser recordings: - `/tmp/hf-gpu-browser-proof/preview-playback.webm` - `/tmp/hf-gpu-browser-proof/preview-seek.webm` ## Notes - Browser GPU is enabled automatically for local CLI renders and disabled in Docker. - `--no-browser-gpu` is the opt-out for software Chrome/WebGL capture. - `--gpu` remains encoder-only and opt-in. - The Apple presentation zip has existing lint errors around unmanaged nested videos and imperative media `play()` calls. The benchmark still compares the same supplied source across modes, but it should not be treated as a clean deterministic-composition fixture. |
||
|
Miguel Ángel
|
bcd7230557 |
fix: fall back to screenshot mode when any CDP call times out during calibration (#567)
## Summary - **Root cause**: `shouldFallbackToScreenshotAfterCalibrationError` only matched `HeadlessExperimental.beginFrame` errors. When a composition with many heavy videos (e.g. 7 videos with sparse keyframes) caused Chrome to be unresponsive in BeginFrame mode during calibration, a `Runtime.callFunctionOn timed out` or `Runtime.evaluate timed out` error was treated as an opaque failure — not a BeginFrame-mode signal. The render kept BeginFrame mode, spawned 3 workers with `captureCostMultiplier=8`, and all 3 workers also timed out initialising their sessions (0 frames captured, render fails). - **Fix**: Add `Runtime.callFunctionOn timed out` and `Runtime.evaluate timed out` to the screenshot-fallback pattern. Any CDP call timing out during the short-timeout calibration probe now routes the render into single-worker screenshot mode — the safe fallback already used for explicit BeginFrame timeouts. - **Result**: Compositions that overwhelm BeginFrame mode (reported in #566: 7 videos, 8 audios, 330-second render) now fall back cleanly and complete instead of failing with 0 frames. ## Test plan - [x] New unit test: `falls back to screenshot mode after Runtime.callFunctionOn timeout during calibration` — asserts both `Runtime.callFunctionOn timed out` and `Runtime.evaluate timed out` return `true` - [x] All existing `capture calibration safeguards` unit tests still pass - [x] Pre-commit hooks (lint, format, typecheck) pass Fixes #566 |
||
|
Miguel Ángel
|
8b234be20c |
fix(producer): handle no-audio stream gracefully in resolveMediaDuration
When an <audio> element referenced a file with no audio stream (e.g. a
silent screen-recording used as an audio src, or a video-only clip),
extractAudioMetadata threw "[FFmpeg] No audio stream found". The error
propagated uncaught through Promise.all in compileHtmlFile and crashed
the entire render.
Apply the same graceful-skip pattern already used for missing files and
failed downloads: catch the probe error and return { duration: 0 } so
the element is excluded from the composition without aborting the render.
Confirmed via 7 production HyperframeRenderWorkflow failures all sharing
the same TemporalMagicEditActivity.RENDER_PREVIEW stack trace.
|
||
|
Vance Ingalls
|
ad44c3133a | perf(hdr): reduce layered composite overhead (#538) | ||
|
Miguel Ángel
|
54cc0f1cba | test: cover sub-composition audio extraction | ||
|
Vance IngallsandClaude Opus 4.6
|
8f97edb2b9 |
fix(hdr): filter zero-opacity elements and support overflow:hidden clip rects (#522)
* fix(hdr): filter zero-opacity elements and support overflow:hidden clip rects in HDR compositor
Two bugs in the HDR render pipeline:
1. Child data-start elements inside a parent with opacity:0 were still
composited as independent layers, painting over content in later scenes.
Fix: filter elements with effective opacity 0 before groupIntoLayers().
2. CSS overflow:hidden on ancestor elements was ignored for HDR video layers,
causing videos inside clipped containers (e.g. split-screen halves) to
render full-frame. Fix: add clipRect to ElementStackingInfo, compute it
from ancestor overflow:hidden in queryElementStacking(), and crop the
source buffer to clip bounds before blitting in blitHdrVideoLayer().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): move opacity filter into blit loop to preserve hide-list correctness
The previous approach filtered zero-opacity elements before groupIntoLayers(),
which broke the DOM screenshot hide-list — invisible video elements' <img>
replacements weren't properly hidden from sibling layer screenshots, causing
the vignelli-stacking regression.
Fix: keep all elements in groupIntoLayers() for correct hide-list generation.
Skip zero-opacity HDR elements only during the actual blit step with an early
`continue` in the compositing loop.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): route identity-matrix HDR elements through region blit for clip rect support
parseTransformMatrix returns a valid matrix even for untransformed HDR
elements (Chrome reports matrix(1,0,0,1,0,0)). This made the affine blit
path always run, bypassing the region blit path which is the only one that
applies clip rects from overflow:hidden ancestors.
Fix: detect identity matrices and route them through the region path so
the cropRgb48le clip logic is reachable for split-screen layouts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): handle translation-only matrices for clip rect support
The previous isIdentity check only caught matrix(1,0,0,1,0,0). Elements
with layout translation (e.g. right-half split at left:960px reporting
matrix(1,0,0,1,960,0)) still routed through the affine path where clip
rects are not applied.
Fix: check for translation-only matrices (scale=1, rotation=0, any tx/ty)
and route those through the region blit path. el.x/el.y from
getBoundingClientRect already include the translation, so the region path
handles positioning correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(render): auto-detect HDR from media probes, add --sdr flag
Replace the --hdr opt-in model with automatic detection. When no flags
are passed, the renderer probes all video/image sources and enables HDR
output if any HDR color space is detected. Existing --hdr flag becomes
a force override. New --sdr flag forces SDR output.
Behavior matrix:
(no flags) + HDR content → HDR output
(no flags) + SDR content → SDR output
--hdr → force HDR (defaults to HLG if no HDR sources)
--sdr → force SDR (skips probing)
--hdr --sdr → error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat(render): auto-detect HDR from media probes, add --sdr flag"
This reverts commit
|
||
|
Vance IngallsandClaude Opus 4.6
|
8e5593b6ba |
feat(render): auto-detect HDR from media probes, add --sdr flag (#526)
* feat(render): auto-detect HDR from media probes, add --sdr flag Replace the --hdr opt-in model with automatic detection. When no flags are passed, the renderer probes all video/image sources and enables HDR output if any HDR color space is detected. Existing --hdr flag becomes a force override. New --sdr flag forces SDR output. Behavior matrix: (no flags) + HDR content → HDR output (no flags) + SDR content → SDR output --hdr → force HDR (defaults to HLG if no HDR sources) --sdr → force SDR (skips probing) --hdr --sdr → error Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: align HDR auto-detect docs and tests --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
36b3fc8cd9 | fix: budget workers for expensive captures | ||
|
Miguel Ángel
|
31afb4c1b3 |
Merge pull request #518 from heygen-com/fix/studio-preview-runtime-visibility
fix: isolate Studio sub-composition previews |
||
|
Miguel Ángel
|
613f97793e | test: add preview regression gate | ||
|
Miguel Ángel
|
6ea2e54fcf | fix: preserve anonymous inlined composition roots | ||
|
Miguel Ángel
|
d3b2900d6b | fix: preserve inferred composition ids when inlining | ||
|
Miguel Ángel
|
3701bb9eeb | fix: flatten inlined composition roots | ||
|
Miguel Ángel
|
c32db3ea0a | fix: isolate studio sub-composition previews | ||
|
Youssef Toufik
|
fe017b48c7 |
feat(producer): true alpha output for webm, mov, and png-sequence
Extends RenderConfig.format with "png-sequence" and patches two correctness
gaps so the existing "webm" / "mov" values actually preserve the alpha
channel end-to-end.
Engine fixes:
- screenshotService.pageScreenshotCapture: drop optimizeForSpeed for PNG
captures. The fast path uses an alpha-unaware codec that crushes real
alpha values; kept for opaque jpeg captures where it is harmless.
- frameCapture: replace the inline setDefaultBackgroundColorOverride
block (which fired pre-navigation and was reset by page.goto) with a
proper initTransparentBackground() call inside initializeSession,
after the window.__hf readiness poll. This also injects the
html/body/[data-composition-id]{background:transparent !important}
stylesheet so compositions with custom body / #root backgrounds do not
defeat the override. Wired into both screenshot-mode and beginframe-mode
branches.
Producer:
- RenderConfig.format extended to "mp4" | "webm" | "mov" | "png-sequence"
with full JSDoc.
- Streaming encode is bypassed for png-sequence (frames go straight to
disk). FORMAT_EXT extended.
- New Stage-5 png-sequence branch: mkdir outputPath, copy captured PNGs as
frame_NNNNNN.png, copy audio.aac sidecar when audio is present.
- Stage-6 mux/faststart and the debug copy are wrapped in !isPngSequence.
- README.md: new "Transparent Video Output" section.
Tests:
- New fixture tests/transparency-regression/ tagged "transparency".
- New tsx script src/transparency-test.ts asserts pixel-level alpha for
webm + png-sequence outputs. Wired as "test:transparency".
- Default "test" / "test:update" scripts pass --exclude-tags transparency
so the golden-MP4 harness ignores the new fixture.
Verified locally on macOS arm64: typecheck clean across engine + producer,
producer renderOrchestrator vitest 10/10, transparency-test passes for
both webm and png-sequence with end-to-end pixel assertions.
|
||
|
Miguel Ángel
|
2f60dd3727 | fix(producer): serve symlinked render assets (#486) | ||
|
Miguel Ángel
|
31e8144304 |
fix: render parity for transparent looped videos (#478)
## Summary - preserve alpha for render-injected video frames by detecting alpha streams with ffprobe and extracting alpha video frames as PNG - keep `<video loop>` semantics through static parsing, compiler duration resolution, browser media discovery, and render frame lookup - fail embedded preview startup before opening a broken browser page when the Studio bundle is missing - align snapshot frame injection with looped media timing and VP9 alpha extraction ## Why The Studio preview and rendered MP4 could disagree for timed transparent looped videos. The Comfy funding composition exposed two separate parity bugs: render-injected frames needed alpha-preserving PNG extraction, and the compiler was clamping a looped `data-duration="4"` video down to the 3.125s source duration. After the first source cycle, render lookup treated the video as inactive, hid the native video, and produced the blank polygon/glow the user saw around the rounded `0:03` mark. `hyperframes lint` and `hyperframes validate` did not catch this because they check syntax/load/console/accessibility, not preview-vs-render visual parity. This PR adds regression coverage for the compiler loop-duration path and frame lookup path. ## Verification - `bun run --filter @hyperframes/core test -- src/compiler/timingCompiler.test.ts src/compiler/htmlCompiler.test.ts` - `bun test packages/producer/src/services/htmlCompiler.test.ts` - `bun run --filter @hyperframes/engine test -- videoFrameExtractor ffprobe` - `bun run --filter @hyperframes/core typecheck` - `bun run --filter @hyperframes/engine typecheck` - `bun run --filter @hyperframes/producer typecheck` - `bun run --filter @hyperframes/cli typecheck` - `bun run lint` - `bun run format:check ...` on touched files - Comfy project: `node packages/cli/dist/cli.js validate` -> no console errors, 44 text elements pass WCAG AA - Comfy project patched render from source: `/tmp/comfy-render-compare/fixed6-comfy.mp4`, 1920x1080, 30fps, 21.8s, 654 frames - 3.00s-3.97s render contact sheet: `/tmp/comfy-render-compare/fixed6-window-contact.png` - targeted fixed render capture at 3.733s: `/tmp/comfy-render-compare/probe-capture-fixed/captured/frame_000112.jpg` - agent-browser Studio proof screenshot at 3.7s: `/tmp/comfy-render-compare/agent-browser-studio-3_7-fixed.png` - agent-browser-driven recording of 3s seek pass: `/tmp/comfy-render-compare/agent-browser-wysiwyg-3s-fixed.webm` Note: `bun run --filter @hyperframes/cli dev -- validate` is blocked in source mode by the existing `contrast-audit.browser.js` default-export loader issue; packaged `node packages/cli/dist/cli.js validate` passes for this project. |
||
|
Miguel Ángel
|
267ffd3fca |
fix(engine,producer): preserve template-wrapped sub-composition media offsets (#476)
## Problem Template-wrapped sub-compositions could still lose correct parent timing during render in more than one place. In the validated repros, a host sub-composition starting after the intro (and in one follow-up repro, starting at `20s` after earlier compositions) contained scene-local media inside it. On the broken paths: - template-wrapped media could be missed during compile and scheduled at raw scene-local time - already-correct first-pass offsets could be clobbered during `recompileWithResolutions()` - even after those two fixes, the browser-metadata reconcile step in `executeRenderJob()` could still overwrite a compiled global `end` with a scene-local `data-end` from the inlined DOM, clipping the tail off late-start sub-composition media ## What this fixes ### Template-wrapped media discovery - `parseVideoElements`, `parseImageElements`, and `parseAudioElements` now unwrap a single top-level `<template>` wrapper before scraping media - the unwrap helper is DOM-based, not regex-based, so it avoids the CodeQL backtracking warning and only unwraps the exact single-wrapper shape we want - multiple sibling templates or other top-level content are left untouched instead of being rewritten heuristically ### Offset preservation after duration resolution - `recompileWithResolutions()` now preserves the first-pass sub-composition media arrays when the already-inlined HTML no longer contains `[data-composition-src]` hosts - that prevents correctly offset media metadata from being overwritten by scene-local media parsed from the merged DOM ### Browser metadata reconciliation in the compiled time origin - browser-discovered media can still report scene-local `data-start` / `data-end` from the merged DOM after inlining - the producer now reprojects browser `end` values into the compiled element's time origin before reconciling them back into `composition.videos` / `composition.audios` - this prevents late-start sub-composition media from getting truncated back to a scene-local end during the probe phase ### Regression coverage - adds focused engine tests for the template unwrap helper - adds producer regression coverage for both the initial compile path and the post-inline `recompileWithResolutions()` path - adds producer regression coverage for late-start host compositions (`t≈20`) with scene-local media inside them - adds producer unit coverage for the browser-end reprojection helper used by the reconcile path ## Root cause There were three distinct renderer failures behind the bug: ### 1. Template contents were invisible to the media scrapers `parseSubCompositions()` reads raw sub-composition HTML and applies the host offset to discovered media. But the engine media helpers were querying the parsed document directly, and linkedom follows browser semantics here: top-level `<template>` contents live in a `DocumentFragment`, so `querySelectorAll()` never saw those `<video>` / `<audio>` / `<img>` nodes. That meant template-wrapped sub-compositions could silently produce zero discovered media during the first pass. ### 2. The duration-resolution recompile could clobber already-correct offsets After the browser resolves composition durations, `recompileWithResolutions()` reparses the already-inlined HTML. By that point the original `[data-composition-src]` hosts are gone, so `parseSubCompositions()` legitimately returns no nested media. The old code still rebuilt the deduped media arrays from the merged DOM, which let scene-local media parsed from the inlined HTML overwrite the correctly offset first-pass metadata. ### 3. The browser probe reconcile path mixed two timing coordinate systems `discoverMediaFromBrowser()` reads `data-start` / `data-end` directly from the live DOM after sub-compositions are already inlined. For nested media, those attributes can still be scene-local even though the compiled metadata has already been offset into the parent host timeline. The old reconcile path compared those values directly and overwrote `existing.end` whenever the numbers differed. For a late-start sub-composition, that could replace a correct global end like `25.5` with a scene-local end like `5.5`, cutting the clip off during render. ## Verification ### Local checks - `bun test packages/engine/src/utils/htmlTemplate.test.ts` - `bun test packages/producer/src/services/htmlCompiler.test.ts` - `bunx vitest run packages/producer/src/services/renderOrchestrator.test.ts` - `bun run --filter @hyperframes/engine test` - `bun run --filter @hyperframes/engine typecheck` - `bun run --filter @hyperframes/producer typecheck` - `bunx oxlint packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxfmt --check packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts` - `bun run build:producer` ### Render / browser verification Verified against two local repros: 1. **Early offset repro** - host starts at `2s` - child media is scene-local `0-4s` - compiled render summary keeps the child video/audio at `start: 2` - browser verification via `agent-browser` confirmed the `2.2s` frame still shows the child clip active in the host timeline 2. **Late offset repro** - earlier compositions run first, then the target host starts at `20s` - child media starts scene-local at `1.5s` and should remain visible through `24.5s` - compiled render summary keeps the child video/audio at `start: 21.5`, `end: 25.5` - browser verification via `agent-browser` confirmed the `24.5s` frame still shows the late clip visible, which is the exact tail-clipping case the old reconcile path could break ## Notes - the `/tmp/hf-pr475-repro` and `/tmp/hf-pr476-late-offset-repro` projects plus their browser-proof artifacts are verification-only and are not part of this PR - this PR stays narrowly scoped to sub-composition media timing across compile, recompile, and browser probe reconciliation; it does not broaden into general sub-composition HTML normalization beyond the single-wrapper case |
||
|
Vance Ingalls
|
bc27f9cee3 |
perf(producer): cache transfer-converted hdr image buffers per render job (#384)
## Summary Add `HdrImageTransferCache` — a per-render-job bounded LRU keyed by `(imageId, targetTransfer)` — so static HDR image layers whose source transfer differs from the render's effective transfer (PQ↔HLG) are converted **once per job** instead of **once per composited frame**. ## Why `Chunk 8B` of `plans/hdr-followups.md`. `blitHdrImageLayer` was running `Buffer.from` + `convertTransfer` on every composited frame, even though the converted buffer is identical for the entire job. For a multi-second comp at 30 fps this is hundreds of redundant transfer conversions on the hot path. ## What changed - New `packages/producer/src/services/hdrImageTransferCache.ts` — bounded LRU keyed by `(imageId, targetTransfer)` that owns the converted HDR rgb48 buffer for static HDR image layers: - Same-transfer requests return the source buffer untouched (zero copy). - Cross-transfer requests pay one `Buffer.from` + `convertTransfer` on first miss, reuse the cached copy on every subsequent frame. - Wired into `renderOrchestrator.ts` via `HdrCompositeContext.hdrImageTransferCache`, instantiated once per render job, and consumed by `blitHdrImageLayer` on both the main composite path and the transition path. ## Test plan - [x] `packages/producer/src/services/hdrImageTransferCache.test.ts` — 12 tests: - hit/miss semantics - distinct keys per image and per target transfer - LRU eviction + promotion - `maxEntries=0` passthrough - source-buffer immutability for cached entries - invalid options - [x] Re-ran the Chunk 8A HDR benchmark — for the `hdr-regression` fixture (which has cross-transfer image layers) the cache hits 100% after the first frame; for HDR fixtures without cross-transfer images the same-transfer passthrough is a no-op. ## Stack Chunk 8B of `plans/hdr-followups.md`. Sits on top of Chunk 8C (logger gating) and Chunk 8A (benchmark harness) so the win is measurable. |
||
|
James Russo
|
57cdf7d80e |
perf(engine): content-addressed extraction cache for video frames (#446)
## What Adds a content-addressed cache for extracted video frames, keyed on the tuple `(path, mtime, size, mediaStart, duration, fps, format)`. Repeat renders of the same composition (studio edit → re-render, preview → final) skip the ffmpeg extraction entirely. ## Why Video frame extraction is the dominant non-capture phase for video-heavy compositions. Studio iteration workflows extract the same frames over and over — each render burns ffmpeg time that adds no value. Validated on `/tmp/hf-fixtures/cfr-sdr-cache`: ``` Cold (miss): extractMs=69, videoExtractMs=70, totalElapsedMs=2052 Warm (hit): extractMs=1, videoExtractMs=2, totalElapsedMs=1964 cacheHits: 0→1, cacheMisses: 1→0 ``` The fixture is tiny (3s CFR SDR @ 30fps), so the wall-clock delta is small; the extraction-time delta (69→1ms, 98%) scales linearly with source length. For heavy-iteration workflows (a user rendering the same composition while tuning encoding params), extraction time goes to zero on every repeat render. Depends on #444 (instrumentation surface) and #445 (segment-scope HDR preflight — otherwise cache keys would be unstable across renders on mixed-HDR compositions). ## How - New `packages/engine/src/services/extractionCache.ts`: - SHA-256 key over a stable JSON encoding of `(path, mtime_ms, size, mediaStart, duration, fps, format)`. Infinity duration is normalized to `-1` so unresolved natural-duration sources still produce stable keys. - Truncates to 16 hex chars in the entry directory name — 64 bits of entropy is plenty at cache scale and keeps `ls` output short. - `hfcache-v2-` schema prefix — bumping it invalidates old entries (callers own gc policy; the cache owns keys). - `.hf-complete` dotfile sentinel. An entry dir without the sentinel is treated as a miss (covers crash-mid-extract and abandoned writes); the next render re-extracts over the partial frames with `-y`. - `FRAME_FILENAME_PREFIX = "frame_"` shared with the extractor — future refactors only need to touch one place to rename frames. - `EngineConfig.extractCacheDir` (env: `HYPERFRAMES_EXTRACT_CACHE_DIR`) gates the feature. Undefined disables caching — extraction runs into the render's workDir and cleanup removes it on render end, preserving the prior behaviour exactly. No default root is chosen by the engine; the caller (CLI, app, studio) owns the location policy. - `ExtractedFrames.ownedByLookup` flag prevents `FrameLookupTable.cleanup` from rm'ing a shared cache dir at render end. Set to `true` on both hits and misses (misses own the directory they wrote into, but hand it over to the cache rather than deleting it). - Phase 3 extractor flow: 1. Snapshot `(videoPath, mediaStart, start, end)` per resolved video BEFORE Phase 2a/2b preflight mutates them — so cache keys are stable across renders that use workDir-local normalized files (those files have fresh mtimes every render). 2. Compute key, `lookupCacheEntry`. 3. On hit: rebuild `ExtractedFrames` from the cache dir plus the Phase 2-probed `VideoMetadata` — no re-ffprobe. 4. On miss: `ensureCacheEntryDir`, extract with `extractVideoFramesRange(..., outputDirOverride)`, then `markCacheEntryComplete` (the sentinel write is the last step so a crash leaves the dir un-sentineled). - `extractVideoFramesRange` gains an `outputDirOverride` parameter so cache-miss writes land directly in the keyed dir (no `join(outputDir, videoId)` wrapping). ## Test plan - [x] 19 unit tests in `extractionCache.test.ts` covering key determinism, mtime/size invalidation, format/fps/mediaStart/duration invalidation, Infinity normalization, sentinel semantics, missing-file tolerance - [x] 2 integration tests in `videoFrameExtractor.test.ts`: - "reuses extracted frames on a warm cache hit" — asserts `cacheHits=1`, `extractMs<50ms` on second call against a CFR SDR fixture - "invalidates the cache when fps changes" — different fps on second call forces a new miss - [x] End-to-end validation with `HYPERFRAMES_EXTRACT_CACHE_DIR` set, two runs of the same fixture - [x] Lint + format (oxlint + oxfmt) - [x] Typecheck (engine + producer) |
||
|
James Russo
|
31354d52da |
perf(engine): extraction-phase instrumentation (#444)
## What Adds per-phase timings and counters to `extractAllVideoFrames` and surfaces them on the producer's `RenderPerfSummary` as `videoExtractBreakdown` alongside a new `tmpPeakBytes` workDir size sample. ## Why Phase 2 video extraction has five distinct sub-phases (resolve, HDR probe, HDR preflight, VFR probe, VFR preflight, per-video extract) and today they collapse into a single `videoExtractMs` stage timing. That makes every subsequent perf PR in this stack immeasurable — you can't tell whether a win came from cache hits, preflight scope reduction, or pure extraction speed. This PR is foundational for PR #445 (segment-scope HDR preflight) and PR #446 (content-addressed extraction cache). ## How - New `ExtractionPhaseBreakdown` type with `resolveMs`, `hdrProbeMs`, `hdrPreflightMs/Count`, `vfrProbeMs`, `vfrPreflightMs/Count`, `extractMs`, `cacheHits`, `cacheMisses`. Populated inline with `Date.now()` wrappers — overhead is sub-millisecond on every phase. - Returned on `ExtractionResult.phaseBreakdown`. - Producer extends `RenderPerfSummary` with `videoExtractBreakdown?: ExtractionPhaseBreakdown` and `tmpPeakBytes?: number`. `tmpPeakBytes` is sampled from the workDir right before cleanup via a new recursive-size helper that swallows errors (purely observational — a missing workDir must never fail the render). No changes to the capture-lifecycle resource tracking — earlier versions of this instrumentation plumbed injector LRU stats through `RenderOrchestrator`, which conflicted hard with upstream #371 (`buildHdrCaptureOptions` refactor). Dropped that piece for a marginal observability loss. ## Test plan Validation on `packages/producer/tests/vfr-screen-recording`: ```json "videoExtractBreakdown": { "resolveMs": 0, "hdrProbeMs": 0, "hdrPreflightMs": 0, "hdrPreflightCount": 0, "vfrProbeMs": 0, "vfrPreflightMs": 166, "vfrPreflightCount": 1, "extractMs": 97, "cacheHits": 0, "cacheMisses": 0 }, "tmpPeakBytes": 4578598 ``` Total elapsed within noise of pre-PR baseline (2665 → 2673 → 3228ms across hosts). - [x] Unit test: phase-breakdown assertion added to `videoFrameExtractor.test.ts` - [x] Lint + format (oxlint + oxfmt) - [x] Typecheck (engine + producer) - [x] Manual perf validation against VFR fixture |
||
|
Vance Ingalls
|
21063c66d9 |
perf(producer): gate per-frame debug meta via optional isLevelEnabled (#383)
## Summary
Add an optional `isLevelEnabled(level)` method to `ProducerLogger` and use it to short-circuit per-frame HDR composite metadata construction in `renderOrchestrator` when the log level is above debug.
Closes Chunks 8C and 8D from `plans/hdr-followups.md`.
## Why
`Chunk 8C` of `plans/hdr-followups.md`. The per-frame HDR composite snapshot (every 30 frames) was building an `Array.find` + `toFixed` + struct allocation unconditionally and handing it to a debug logger that immediately discarded it at `level="info"`. On long renders, this is allocation pressure and CPU time wasted on log meta nobody reads.
`Chunk 8D` was investigated in the same pass and found to already be guarded — see below.
## What changed
- New optional `isLevelEnabled(level: ProducerLogLevel): boolean` on `ProducerLogger`.
- `createConsoleLogger` implements it.
- `renderOrchestrator.ts` per-frame HDR composite snapshot is now gated on `i % 30 === 0 && (log.isLevelEnabled?.("debug") ?? true)` — production runs at `level="info"` skip the meta-object construction entirely; custom loggers without the new method keep their existing behavior thanks to the `?? true` fallback.
- New `packages/producer/src/logger.test.ts` (17 tests) covering level filtering, meta formatting, the `isLevelEnabled` path, a hot-loop call-site simulation that asserts zero builder invocations at info level, and the `?? true` fallback for loggers that omit the method.
- `docs/packages/producer.mdx` gains a new "Logging" section documenting `ProducerLogger`, `createConsoleLogger`, `defaultLogger`, and the `isLevelEnabled` gating pattern.
**8D resolution (no code change).** `countNonZeroAlpha` / `countNonZeroRgb48` calls live behind `shouldLog = debugDumpEnabled && debugFrameIndex >= 0`, where `debugDumpEnabled` is itself driven by `KEEP_TEMP=1`. The pixel iteration is fully skipped on production runs already, so 8D needed no fix — verified during the 8C work.
## Test plan
- [x] `bun test` in producer — 17/17 logger tests pass; existing service tests unchanged.
- [x] Hot-loop call-site simulation asserts the meta builder is invoked **zero times** at `level="info"`.
- [x] `?? true` fallback preserves prior behavior for custom logger implementations that don't define the method.
- [x] Re-ran the HDR benchmark from Chunk 8A — no regression on wall-clock, peak heap unchanged at info level.
## Stack
Chunks 8C + 8D of `plans/hdr-followups.md`. Sits on top of the benchmark harness PR (Chunk 8A) so the optimization is measurable.
|
||
|
Vance Ingalls
|
3da8c2e969 |
perf(producer): hdr benchmark harness — --tags filter, peak heap/RSS tracking, bench:hdr script (#382)
## Summary Make the existing benchmark harness genuinely useful for HDR perf work: positive `--tags` filter, peak heap/RSS sampling, a `bench:hdr` script, and a perf README documenting the captured April-2026 baseline. Lands first in the Chunk 8 sub-stack so subsequent perf PRs can be measured against a known starting point. ## Why `Chunk 8A` of `plans/hdr-followups.md`. Wall-clock timing alone can't catch slow memory regressions like an unbounded image cache — peak RSS does. And the existing harness only had `--exclude-tags`, so HDR runs had to wait for unrelated SDR fixtures. ## What changed **1. Positive `--tags` filter** in `benchmark.ts`. Adds `--tags hdr` so HDR runs don't have to wait for unrelated fixtures. Filters compose: a fixture must match `--tags` (if provided) AND must not match `--exclude-tags`. **2. Peak heap + RSS tracking** in `executeRenderJob`. A 250 ms periodic `process.memoryUsage()` sampler runs alongside every render and reports `peakRssMb` / `peakHeapUsedMb` in `RenderPerfSummary`. Sampler is `unref`'d and always cleared in `finally` so it never keeps the event loop alive or leaks across jobs. Both fields are optional on the interface for back-compat with serialized older summaries. **3. `bench:hdr` convenience script** plus a perf README at `tests/perf/README.md` documenting the harness, the new flags, and the captured April-2026 HDR baseline (PQ regression: 34.5 s / 272 MiB RSS, HLG regression: 11.5 s / 227 MiB RSS, both 1080p / 1 worker / 1 run). The benchmark output table is widened and gains `PeakRSS` / `PeakHeap` columns. A new `avgOrNull` helper preserves `null` in the JSON when no run reported memory (avoids silently coercing missing data to 0 in older snapshots). No behavior change for non-benchmark renders — the sampler runs in every `executeRenderJob` but its overhead is a single `process.memoryUsage()` call every 250 ms, well below noise. ## Test plan - [x] `bunx tsc --noEmit -p packages/producer` — clean. - [x] `bunx oxlint` / `bunx oxfmt --check` on changed files — clean. - [x] `bun test src/services/` — 60/60 pass (frameDirCache, orchestrator, etc.). - [x] `bunx tsx src/benchmark.ts --tags hdr --runs 1` — both HDR fixtures render successfully, summary table prints `PeakRSS`/`PeakHeap` columns, per-run output shows new memory line. - [x] `bunx tsx src/benchmark.ts --tags nonexistent` — exits 1 with a helpful message naming the active filters. ## Stack Chunk 8A of `plans/hdr-followups.md`. First PR in the Chunk 8 perf sub-stack; subsequent PRs (image cache, logger gating) measured against this baseline. |
||
|
Vance Ingalls
|
cc9403b6bd |
test(producer): extract frameDirMaxIndexCache to its own module and pin cross-job isolation (#381)
## Summary Extract the `frameDirMaxIndexCache` from a private module-scoped Map inside `renderOrchestrator.ts` into its own `frameDirCache.ts` module, then add a 11-test bun:test suite that pins the cross-job isolation contract added in Chunk 5B. ## Why `Chunk 9E` of `plans/hdr-followups.md`. The cache lived as a private Map inside `renderOrchestrator.ts`, which made the cross-job isolation contract from Chunk 5B impossible to unit-test directly. Extracting it both makes the contract testable and reduces orchestrator complexity slightly. ## What changed - New `packages/producer/src/services/frameDirCache.ts` exposes `getMaxFrameIndex` / `clearMaxFrameIndex` / `getMaxFrameIndexCacheSize` (plus a test-only `__resetMaxFrameIndexCacheForTests` helper). Behavior is unchanged: callers still get the same module-scoped sharing inside a job, and `renderOrchestrator`'s outer `finally` still clears every entry it registered so the cache cannot grow monotonically across renders. - `renderOrchestrator.ts`: imports the new helpers, drops the unused `readdirSync` import, updates inline comments, and replaces two `frameDirMaxIndexCache.delete` sites with `clearMaxFrameIndex`. - New `frameDirCache.test.ts` (bun:test, 11 tests) covering: - Reading the max index from a populated directory. - Ignoring filenames that don't match `frame_NNNN.png` (wrong ext, wrong prefix, wrong case, double extension, empty index group, same-named subdirectory). - Empty- and missing-directory paths returning `0` and being cached. - Intra-job invariant: subsequent readdir mutations not observed once cached. - `clearMaxFrameIndex` forcing a re-read; returns `false` for paths that were never cached. - Per-directory isolation when multiple directories are registered. - The cross-job contract from Chunk 5B: cache empty between well-behaved jobs, doesn't grow monotonically across 20 simulated renders with 3 HDR videos each (steady-state cache size stays at 3), and a buggy job that forgets to clear leaks exactly its own entries rather than affecting unrelated jobs. ## Test plan - [x] `frameDirCache.test.ts` 11/11 pass. - [x] Existing producer tests unchanged. - [x] Behavior preserved: same module-scoped sharing inside a job, same outer-`finally` eviction. ## Stack Chunk 9E of `plans/hdr-followups.md`. Test-driven extraction; complements Chunk 5B. |
||
|
Vance Ingalls
|
a3d7cc1c95 |
refactor(producer): extract HDR compositing helpers and rename media metadata (#373)
## Summary
Four behavior-preserving refactors that reduce complexity in `renderOrchestrator.ts` and clarify the engine ffprobe utility surface. Lands after the correctness fixes (Chunks 1–5) so the refactored code is already correct.
## Why
`Chunk 7` of `plans/hdr-followups.md`. The HDR composite block had grown a ~200 LOC inline closure with 14 captured deps, a repeated capture-options spread, a `extractVideoMetadata` name that now also handles still images, and per-frame re-creation of debug helpers.
## What changed
**7A — Hoist `compositeToBuffer` into a module-scoped helper.** Extract the inline HDR closure into a top-level `compositeHdrFrame()` that takes an `HdrCompositeContext` struct. Construct the context once at the top of the HDR render block and pass it through. Removes a deeply-nested closure from the middle of the orchestrator.
**7B — `buildHdrCaptureOptions()` helper.** Factor the repeated `{ ...captureOptions, skipReadinessVideoIds: ... }` spread into a named helper at the call site.
**7C — Rename `extractVideoMetadata` → `extractMediaMetadata`.** Reflects that the helper handles still images (PNG/JPEG/WebP) in addition to video. Update all callers in engine + producer (`videoFrameExtractor`, `htmlCompiler`, regression-harness, producer ffprobe re-export, tests). Re-export the old name as a deprecated alias from `@hyperframes/engine` for backward compatibility, plus the producer re-export shim.
**7D — Hoist debug counters to module scope.** `countNonZeroAlpha` and `countNonZeroRgb48` are now module-scoped so they aren't re-created per frame and so the closure has fewer captures.
Also touches the `hdr-regression` and `hdr-hlg-regression` README + `meta.json` files reviewed during this refactor.
## Test plan
- [x] `bunx tsc --noEmit -p packages/producer && bunx tsc --noEmit -p packages/engine` clean.
- [x] Engine tests: 313 pass, 0 fail (1218 expect calls).
- [x] `bunx oxlint` + `bunx oxfmt --check` clean on 8 changed source files.
- [x] Diff is structural only — no behavioral changes.
## Stack
Chunk 7 of `plans/hdr-followups.md`. Lands after the correctness fixes (Chunks 1–5) per the suggested merge order.
|
||
|
Vance Ingalls
|
53e1aeaadc |
fix(producer): wire --crf and --video-bitrate CLI overrides into encoders (#372)
## Summary Re-wire the `--crf` and `--video-bitrate` CLI flags through the three encoder spawn sites in `renderOrchestrator.ts`. They were defined and parsed in the CLI but silently dropped before reaching ffmpeg. ## Why `Chunk 10` of `plans/hdr-followups.md`. PR #292 originally wired these through with a `baseEncoderOpts` object using `effectiveQuality`/`effectiveBitrate`; PR #268 rewrote the encode paths and reverted to `preset.quality` only, accidentally dropping the override. This is a user-facing regression — `hyperframes render --crf 18` was being silently ignored. ## What changed - At the three encoder spawn sites (HDR streaming, SDR streaming, disk-based encode), `quality` defaults to `preset.quality` but is overridden by `job.config.crf` when set, and `bitrate` is set from `job.config.videoBitrate`. Mutual exclusivity is enforced upstream in the CLI, so we don't re-check it here. - Fix the contradictory note in `docs/packages/cli.mdx` that claimed CRF/bitrate were now driven only by `--quality`. The flags table now lists `--crf` and `--video-bitrate` consistent with `docs/guides/rendering.mdx`. ## Test plan - [x] `hyperframes render --crf 18 ...` now respects the CRF override (verified via ffprobe of the encoded output). - [x] `hyperframes render --hdr ...` still works (no behavior change at the default path). - [x] `hyperframes render --help` shows all flags consistent with the docs. ## Stack Chunk 10 of `plans/hdr-followups.md`. Independent of all other chunks. |
||
|
Vance Ingalls
|
6fd99109c9 |
fix(producer): tighten resource lifecycle and harden file server (#371)
## Summary Five resource-management fixes in `renderOrchestrator.ts` and `fileServer.ts`: HDR encoder cleanup on non-abort errors, `frameDirMaxIndexCache` eviction, mid-transition abort responsiveness, pre-allocated transition buffers, and a path-traversal guard for the local file server. ## Why `Chunk 5` of `plans/hdr-followups.md`. These are independent leaks/hangs/security issues that had each been called out in prior PR reviews and never landed. ## What changed **5A — HDR encoder + `domSession` cleanup.** The HDR streaming encoder and `domSession` were spawned outside any outer `try/finally`, so a non-abort error between encoder spawn and the inner cleanup leaked the FFmpeg process and held the browser page open. Wrapped the entire HDR (and SDR streaming) capture path in a `try/finally` with explicit `*Closed` flags, and defensively close both in the outer `finally` if they haven't been closed already. `StreamingEncoder.close()` and `closeCaptureSession()` are both idempotent, so double-close is safe. **5B — `frameDirMaxIndexCache` + `hdrFrameDirs` eviction.** `frameDirMaxIndexCache` is module-scoped and grew monotonically: every render added entries that were never removed. Lifted `hdrFrameDirs` to the outer scope, drop the matching cache entry in the per-video `rmSync` block, and sweep any survivors in the outer `finally`. The on-disk frames themselves were already torn down with `workDir`; this just stops the in-process Map from leaking entries across renders. **5C — Abort signal between scene A and scene B.** During a shader transition the orchestrator captures scene A and scene B back-to-back inside a single outer frame iteration. An abort that arrived while scene A was capturing wouldn't be noticed until the next outer frame — after scene B had already been fully composited and discarded. Added `assertNotAborted()` at the top of the inner `[transBufferA, transBufferB]` loop so abort is observed before the second scene's DOM seek + screenshot. **5D — Pre-allocated transition buffers (already addressed).** The transition buffers (`transBufferA`, `transBufferB`, `transOutput`, `normalCanvas`) are pre-allocated outside the per-frame loop. The remaining `Buffer.from` copies sit in HDR transfer conversion (Chunk 8B territory) and image preload, neither of which is the per-frame hot path. **5E — `fileServer` path-traversal guard.** `fileServer.ts` joined `compiledDir` / `projectDir` with the request path and only checked `existsSync` + `isFile`. `path.join` normalizes `..` segments, so `GET /../etc/passwd` would resolve to `/etc/passwd` and be served straight off disk if the file existed. Added an `isPathInside(child, parent)` helper that resolves both sides and compares prefixes with the platform separator appended (so `/foo` doesn't match `/foobar`), and rejects any candidate that lands outside its intended root. ## Test plan - [x] `bun run --filter @hyperframes/producer typecheck` passes. - [x] `fileServer.test.ts` 13/13 pass (4 existing + 9 new `isPathInside` cases covering same-path, nested, prefix-only siblings, escaping traversal, traversal that resolves back inside, trailing-slash handling, and relative-path resolution). - [x] Manual: kill a render mid-flight with a non-abort error; no orphaned `ffmpeg` processes (5A). - [x] Manual: two render jobs back-to-back; cache cleared between jobs (5B). - [x] Manual: abort during a transition frame; stops promptly, not after scene B (5C). - [x] Manual: `GET /../../../etc/passwd` against the local file server returns 403/404 (5E). ## Stack Chunk 5 of `plans/hdr-followups.md`. |
||
|
Vance Ingalls
|
5de5df7fbb |
refactor(types): tighten type safety, dedupe HfTransitionMeta, prune dead LUT export (#366)
## Summary Four small, mechanical type-safety cleanups across `engine`, `producer`, and `shader-transitions`. Zero behavior change — pure pre-cleanup so the rest of the stack ships against a tighter baseline. ## Why `Chunk 6` of `plans/hdr-followups.md`. Several non-null assertions and a duplicate interface had accumulated as rebase artifacts and leftover work-in-progress; lands first because it touches files later chunks edit and removes friction during review. ## What changed - `renderOrchestrator.ts`: replace `layers[layerIdx]!` with a `for (const [layerIdx, layer] of layers.entries())` so both index and element come from the iterator. - `engine/types.ts`: drop the duplicate `HfTransitionMeta` interface (rebase artifact); the original definition above it is the documented one. The orphaned doc comment now precedes `HfProtocol`. - `shader-transitions/hyper-shader.ts`: keep the local `HfTransitionMeta` declaration (the package ships as a standalone CDN bundle and must not depend on `@hyperframes/engine`), but add a sync comment pointing at the source of truth in `engine/src/types.ts`. - `alphaBlit.ts` + `engine/index.ts`: drop `export` from `getSrgbToHdrLut` and remove its re-export. It was only ever called by the internal `blitRgba8OverRgb48le`; the public surface was dead code. ## Test plan - [x] `bun run --filter @hyperframes/engine typecheck` - [x] `bun run --filter @hyperframes/producer typecheck` - [x] `bun run --filter @hyperframes/shader-transitions typecheck` - [x] `bun run --filter @hyperframes/engine test` — 308/308 pass (no test changes; assertions removed in code only). ## Stack Chunk 6 of `plans/hdr-followups.md`. Mechanical cleanup landed early per the suggested merge order. |
||
|
Miguel Ángel
|
d740f5ce42 |
fix: nested GSAP sub-composition lint and render handling (#405)
## Summary - allow nested sub-composition files to inherit GSAP from their host without tripping `missing_gsap_script` - keep nested render seeks stable for sub-compositions without regressing producer baselines - stop producer render-hint detection from treating the compiler's own nested mount retry wrapper as user-authored `requestAnimationFrame()` usage ## Root Cause - the core linter treated template-based nested compositions like standalone root compositions, so it incorrectly required a local GSAP loader even when the host composition already provided GSAP - producer `detectRenderModeHints()` runs before CDN scripts are inlined, so nested GSAP exports were never failing because of the GSAP payload itself - the nested-only false positive came from the compiler-generated mount bootstrap that waits for the inlined sub-composition root with `requestAnimationFrame()` before running the hoisted inline script - preview and export seek paths also needed to stay split so the nested timeline re-arm behavior that stabilizes scrubbing does not collapse render baselines ## What Changed - lint: keep the nested GSAP false-positive fix and regression coverage for template sub-compositions - runtime: keep the render-seek behavior that preserves nested child offsets during export without changing preview scrubbing behavior - producer: mark compiler-owned mount bootstrap blocks and strip only those blocks before scanning inline scripts for raw `requestAnimationFrame()` - producer tests now cover both cases: compiler-generated wrappers are ignored, but real user-authored nested `requestAnimationFrame()` still opts into screenshot mode ## Validation - `bun test packages/core/src/lint/rules/gsap.test.ts` - `bun test packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxfmt packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxlint packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts` - `bun run --filter @hyperframes/producer test --sequential chat style-11-prod` - `style-11-prod` passed locally - `chat` still shows local-only visual drift on this macOS/ARM workstation, but the render metadata now reports `renderModeHints.recommendScreenshot=false`, which is the concrete acceptance condition for `#402` - Docker CI-image repro is blocked locally by OrbStack x86/arm64 loader mismatch, so final regression confirmation is deferred to GitHub Actions Closes #392 Closes #402 |
||
|
James RussoandClaude Opus 4.7
|
ffc06827c4 |
fix(engine): auto-normalize VFR video inputs to CFR before frame extraction (#360)
* fix(engine): auto-normalize VFR video inputs to CFR before frame extraction
Screen recordings (macOS ScreenCaptureKit, QuickTime, phone videos) are
commonly variable-frame-rate. When such inputs hit the extractor's
`-ss <start> -i <video> -t <dur> -vf fps=N` pipeline, the fps filter
can emit fewer frames than requested — for a 4-second 30fps segment
starting mid-file, the output was ~90 frames instead of 120.
`FrameLookupTable.getFrameAtTime` returns null for out-of-range indices,
so the compositor held the last valid frame and the user perceived the
video as freezing. This matches the bug report from an X community post
where a user said "all of them freezes" on their screen recording scenes.
The engine already detects VFR via `metadata.isVFR` in ffprobe.ts but
never acted on it — the compiler only logged a warning. This change
mirrors the existing SDR→HDR normalization pattern: when a source is
detected as VFR, re-encode only the used segment with
`-fps_mode cfr -r <fps> -preset fast -crf 18` before extraction.
Scoping the re-encode to `[mediaStart, mediaStart+duration]` means a
30-second clip cut from a 60-minute screen recording pays ~1s of
transcode cost, not 18s. Benchmarked locally:
Baseline (current): 32-39% duplicate frames, 25% frame-count
shortfall on mid-file segments.
Tier 1 (flag changes only): ~same — fps filter issue is not flag-fixable.
Tier 2 (CFR preflight): 1.7-6% duplicate frames, correct frame
count in every scenario tested.
The compiler warning that previously told users to manually re-encode
is downgraded to `console.info` since the engine now handles it.
— Rames Jusso
* refactor(engine): clean up VFR normalization loop after review
- Drop the `vfrNormDirCreated` flag; `mkdirSync({recursive:true})` is
idempotent and cheap.
- Don't re-wrap the `VFR→CFR conversion failed` prefix — `convertVfrToCfr`
already throws a message with that label; adding it again in the catch
produced "VFR→CFR conversion failed: VFR→CFR conversion failed (exit 1)".
- Shorten the Phase 2b header comment; the function docstring above
`convertVfrToCfr` already explains the failure modes and rationale.
- Note which frame windows the VFR fixture's select filter drops so the
magic numbers are scannable.
No behavior change; 311/311 engine tests still pass.
— Rames Jusso
* test(engine): add VFR regression unit tests
Adds a describe block that synthesizes a VFR fixture via ffmpeg and asserts
the extractor produces the expected frame count (no shortfall) and no long
runs of duplicate frames — the user-visible "frozen screen recording"
symptom. Covers both a mid-file segment and the full-file case.
Guarded with describe.skipIf(!HAS_FFMPEG) because the CI Test job on
ubuntu-24.04 and the Windows test-windows job don't install ffmpeg. The
producer-level regression test in packages/producer/tests/vfr-screen-recording/
runs inside Dockerfile.test (which has ffmpeg) and is the primary CI signal
for this bug; these unit tests are supplementary coverage for local and
any ffmpeg-equipped CI environment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(producer): add vfr-screen-recording regression test
End-to-end CI regression coverage for PR #360 via the existing
regression-harness: renders a 3s composition containing a real macOS
ScreenCaptureKit clip (r_frame_rate=120, avg≈36fps) seeked to
mediaStart=1, then PSNR-compares against a committed output.mp4.
Fixture src/clip.mp4 (108 KB) is a 5-second excerpt downscaled to 480×332
with -fps_mode passthrough to preserve the VFR timestamps. Content is the
public hyperframes OSS repo root page — see NOTICE.md for provenance.
With the fix applied, all 100 PSNR checkpoints pass. With the fix reverted,
66 of 100 fail (PSNR drops from ~43 dB to ~20 dB in the duplicate-frame
windows). Tagged "regression,video,vfr" so it runs in the fast shard
of .github/workflows/regression.yml automatically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(producer): regenerate vfr-screen-recording baseline in Docker
The committed golden output.mp4 was initially rendered on the host machine;
CI runs the renderer inside Dockerfile.test with a different Chrome +
ffmpeg build, producing pixel-level drift that failed PSNR at 54/100
checkpoints (~20 dB vs 41 dB in the VFR sparse-content windows). Both
renders are valid — the VFR source has inherent sampling ambiguity in
static segments, and different Chrome/ffmpeg builds make different valid
choices.
Regenerated the baseline via `bun run docker:test:update vfr-screen-recording`
so it matches the Docker environment CI actually uses. Matches the flow
the existing sub-composition-video, hdr-pq, etc. baselines were captured
with.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: document that producer test baselines must be captured in Docker
Hit this 2026-04-21 with the vfr-screen-recording regression test:
host-generated output.mp4 baseline tripped 54/100 PSNR checkpoints in CI
because Chrome + ffmpeg drift between the host and Dockerfile.test.
Document the `bun run --cwd packages/producer docker:test:update <name>`
flow so future contributors don't repeat the mistake.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
b98093aa1c |
fix: remove hidden audio gain in renders (#362)
## Summary
This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.
It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.
Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.
## Root Cause
The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:
- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
- `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`
That exactly matches the issue reporter's measured scalar boost.
While investigating the workaround, I also found a second correctness bug:
- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output
Separately, the repo-level Studio typecheck could fail before any build step because:
- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck
## What Changed
1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build
## Why These Changes Are Needed
This is not a UX preference issue; it is a correctness and API contract issue.
- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.
Leaving the current behavior in place means:
- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed
## Testing
### Focused regression tests
Ran:
- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`
Result:
- `10 passed`
These tests specifically verify:
- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)
### Broader package verification
Ran:
- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`
Results:
- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker
## Known Verification Limitation
There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.
## User Impact
After this change:
- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts
Closes #361.
|
||
|
Vance Ingalls
|
00af29c169 |
fix(cli): forward --hdr through Docker render + HDR docs (#346)
## Summary This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI. The branch now does four things: - forwards `--hdr` through the Docker render path in the CLI - adds and expands HDR documentation across the docs site - adds first-class HDR still-image support to the engine/producer pipeline - adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags ## What changed ### CLI and docs - `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI - added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs - documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes ### Engine and producer HDR image support - added `ImageElement` support to the engine composition model and parsing path - threaded image elements through producer compilation and orchestration - probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source - included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order - integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays - forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic - skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows ### HDR metadata robustness - added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs - this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ ### Regression coverage and fixture cleanup - added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end - added `hdr-pq`, a focused HDR PQ regression fixture for the video path - updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only` - removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI - added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests ## Why The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking. The practical issue this closes is: - local host runs could pass while CI failed `hdr-image-only` - the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering - root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment - parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments ## Test plan ### Local targeted checks ```bash bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts ``` ### Producer regression runs on host ```bash bun run --cwd packages/core build:hyperframes-runtime:modular bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only ``` Observed result: - `fast` shard: 7 passed, 0 failed - `hdr` shard: 2 passed, 0 failed ### CI-equivalent Docker verification ```bash docker build -f Dockerfile.test -t hyperframes-producer:test . docker run --rm \ --security-opt seccomp=unconfined \ --shm-size=4g \ -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \ hyperframes-producer:test \ --sequential hdr-pq hdr-image-only ``` Observed result: - `hdr-image-only`: passed - `hdr-pq`: passed - shard summary: 2 passed, 0 failed ### Specific regression fixed Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with: - missing `"[Render] HDR source detected — output: PQ ..."` log line - full-frame visual mismatch across all 100 checkpoints - PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes. |
||
|
Miguel Ángel
|
a539266683 |
fix: harden CDN script inlining with linkedom (#352)
* fix: harden CDN script inlining * test: add spanish empire regression fixture |
||
|
Vance IngallsandClaude Opus 4.6
|
99a903be2f |
feat(hdr): layered HDR compositing, shader transitions, and HDR image support (#268)
* feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes - 15 GLSL→TypeScript shader transitions on rgb48le buffers - Dual-scene compositing with scene detection via window.__hf.transitions - --hdr flag gates ffprobe probing (zero overhead on SDR compositions) - Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT - Buffer.from() copy in writeFrame() fixes streaming encoder race condition - SDR rendering fixes (three stacked bugs) - Object.assign fix for window.__hf preservation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: tighten shader smoke thresholds + assert .scene contract - Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268. * fix(hdr): restore VIRTUAL_TIME_SHIM and applyRenderModeHints in renderOrchestrator Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes") accidentally removed two pieces of the deterministic rendering pipeline: 1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`, which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven animations advance only when `window.__hf.seek(t)` is called. 2. The `applyRenderModeHints` function and its post-`compileForRender` call site, which auto-forces screenshot capture mode for compositions the compiler flagged as needing it (RAF, iframes, etc.). Without (1), RAF animations advanced by wall-clock between the main-loop seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat` lost its automatic fallback to screenshot mode and the child-document motion stopped being captured. Both helpers are still produced by `htmlCompiler` and exercised by `renderOrchestrator.test.ts` — the orchestrator just stopped calling them. Restored: - Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js` - Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer` call sites (probe + main render) - Re-add `applyRenderModeHints` (matching the test expectations) and call it immediately after `compileForRender` - Persist `renderModeHints` in `summary.json` and the "Compiled composition metadata" log line Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression failures on `feat/hdr-layered-compositing`. Made-with: Cursor * test(engine): expand sampleRgb48le coverage + audit Uint16Array alignment Adds: - 8 new sampleRgb48le bilinear-interpolation tests covering boundary pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers. - uint16-alignment-audit.test.ts documenting the alignment requirement for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE. Background: ~105 hot-loop sites in shader transitions still use readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut overhead but requires guaranteed even byteOffsets — these tests document the contract before any future refactor lands. * fix(engine,producer): mask DOM layers during HDR layered compositing The HDR layered compositor blits z-ordered layers over a shared canvas. DOM layers used a full-page screenshot from `captureAlphaPng`, which captures *every* painted pixel on the page — root background, sibling-scene content, overlay UI elements that aren't part of the current layer. Those opaque pixels were then blitted over the canvas, overwriting any HDR content composited beneath in earlier layers. The previous workaround toggled `display:none` on hide ids via `hideVideoElements`/`showVideoElements`. That correctly hid native videos but did nothing about the root composition's background or about overlay elements that the layer grouping considered part of a different layer. This commit replaces the workaround with a precise CSS mask installed before each DOM screenshot: 1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and re-shows the layer's elements (and their descendants and their injected `__render_frame_*` siblings) with `visibility: visible !important`. CSS visibility is *not* multiplicative through descendants — a child with `visibility: visible` overrides an ancestor's `visibility: hidden`, so deeply nested layer content still paints even though every intermediate ancestor is hidden by the mass-hide rule. 2. Non-layer data-start ids are inline-hidden with `visibility: hidden !important`. Inline `!important` beats stylesheet `!important`, so this overrides the show rule for elements that fall under a show selector but should NOT paint — most importantly HDR videos and other-layer SDR videos that live as descendants of `#root`. 3. `removeDomLayerMask` tears the stylesheet down and clears the inline `visibility`/`opacity` properties so subsequent video frame injection gets a clean slate. Crucially the mask only sets `visibility`, never `opacity`. CSS opacity *is* multiplicative — `opacity: 0` on `#root` would zero out every descendant including layer videos, even with `visibility: visible`. We also extend `initTransparentBackground` to force the composition root (`[data-composition-id]`) transparent in addition to `html`/`body`, because compositions almost always set `#root { background: ... }` and that background paints across the whole viewport otherwise. Both compositing paths use the new helpers: - The per-layer DOM branch (`compositeToBuffer`) for normal frames. - The transition path (single DOM screenshot per scene) so transition frames also get a clean per-scene capture. Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`: per-layer pixel-add accounting, dumps of every captured DOM PNG, and a periodic raw `rgb48le` snapshot of the composite buffer. These were essential to diagnosing the root-overwrite bug and stay zero-cost in normal renders. Also stops the workDir / per-video frame-dir cleanup when `KEEP_TEMP=1` so the dumps survive past frame N. Made-with: Cursor * fix(engine): preserve GSAP-applied opacity across DOM-layer captures SDR clips inside an HDR composition were rendering at full opacity even when the user had animated their wrapper opacity (e.g. fade-in or yoyo). Two bugs in the per-layer screenshot path conspired to drop the GSAP-applied opacity on the floor: 1. removeDomLayerMask was unconditionally calling `el.style.removeProperty("opacity")` on every wrapper after each layer capture. applyDomLayerMask only ever sets `visibility`, so the only inline opacity present is the value GSAP wrote. Stripping it between layer captures means that on the next capture (at the same timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline is already at that time — the opacity is never restored, and the wrapper renders fully opaque. 2. injectVideoFramesBatch was reading the source <video>'s computed opacity via `parseFloat(computedStyle.opacity) || 1` and copying it onto the injected <img>. Because syncVideoFrameVisibility forces the <video> to `opacity: 0 !important` to hide it during capture, the computed value is always 0, which `|| 1` then silently flips to full opacity. The <img> is a sibling of the <video> inside the same wrapper, so it should inherit opacity from the wrapper directly instead of having a value hard-set on it. Fix both: drop the opacity removal in removeDomLayerMask, skip opacity when copying visual properties from <video> to <img>, and explicitly clear any stale inline opacity on the <img> so it inherits from the wrapper that GSAP is animating. Made-with: Cursor * fix(producer): correct hdrLayerStartTimes typo to hdrVideoStartTimes The diagnostic logging block in executeRenderJob's HDR layer composite path referenced an undeclared `hdrLayerStartTimes` map. The correct variable, declared and populated earlier in the same function, is `hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer masking work and broke the producer build/typecheck on CI. Made-with: Cursor * fix(engine): restore video opacity copy to injected frame img Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on the assumption that the <img> sibling would inherit GSAP's opacity from a shared wrapper. That breaks any composition where GSAP animates opacity directly on the <video> element itself: the <img> has no animated ancestor and renders at full opacity throughout any fade, even when the user's intent is partial or zero opacity. The CI `style-7-prod` and `style-8-prod` regressions caught this: the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut because the <img> inherited opacity 1 regardless of GSAP's tween. Restore the old explicit copy from `computedStyle.opacity` to the <img>'s inline opacity, with the `|| 1` fallback intentionally preserved. The fallback is load-bearing: GSAP's seek does not re-apply tweens that have already completed, so post-fade frames read opacity 0 from the stale `opacity: 0 !important` we apply to hide the native <video>. The `|| 1` recovers the tween's end-state opacity 1 for those frames, matching the final on-screen intent and the existing baseline renders. Handles both DOM shapes: - GSAP on wrapper: video's own computed opacity is 1, img set to 1, wrapper's opacity applies via stacking as before. - GSAP on <video>: video's computed opacity is the tween value, copied to img directly since they are siblings. Fixes: - style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33) - style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |