ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
714 Commits 856 Branches 404 Tags
588639fc1aa5c3529bb2be17e6eefb5b4c9a22a2
Commit Graph
38 Commits
Author SHA1 Message Date
JamesandClaude Opus 4.7 c9d5fe61ff refactor(producer): apply /simplify findings on variables-prod 
Align the no-op timeline duration with the root's data-duration. The
fixture's root has data-duration="3" but the placeholder timeline tween
was still { duration: 2 } — leftover from when I bumped the duration
from 2s to 3s to dodge the PSNR-checkpoint-at-1.99s parse edge case.
The tween is a no-op (no targets, no visible effect) so rendered pixels
don't change; baseline still passes Docker regression at 100/100
checkpoints.

Reuse + efficiency reviews otherwise clean. Two findings deferred:
silence.wav duplication is real but only 2 fixtures share it today —
worth extracting to tests/_shared/ when the third fixture lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 20:34:20 +00:00
JamesandClaude Opus 4.7 58b4234809 test(producer): add variables-prod regression test for the variables stack 
End-to-end Docker regression test that exercises the full variables
chain: meta.json renderConfig.variables → harness → createRenderJob →
RenderConfig → CaptureOptions → engine evaluateOnNewDocument →
window.__hfVariables → getVariables() → DOM text → rendered pixels.

Fixture (packages/producer/tests/variables-prod/):
- src/index.html: composition with three declared variables (title,
  subtitle, bgColor) read via window.__hyperframes.getVariables() and
  rendered as positioned text on a colored background. No animation —
  keeps the regression frame-stable so it isolates "did the variables
  flow through?" from motion concerns.
- meta.json: tags ["variables", "composition"] (runs in the existing
  fast shard's tag filter), renderConfig.variables provides override
  values the baseline reflects ("Override Title", "Override subtitle",
  #0a3d62). Defaults would produce a visibly different frame, so a
  failing baseline that reflects defaults means the variables didn't
  propagate.
- output/output.mp4: Docker-generated baseline per the project's
  CLAUDE.md golden-baseline rule.

Harness change (packages/producer/src/regression-harness.ts):
- TestMetadata.renderConfig gains an optional variables field,
  validated as a JSON object in the meta.json validator.
- The createRenderJob call site forwards renderConfig.variables to
  RenderConfig.variables, which the engine already consumes via
  evaluateOnNewDocument (PR #600).

Verified: docker:test variables-prod passes 100/100 visual checkpoints
and audio correlation 1.000 against the committed baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 20:33:43 +00:00
Miguel Ángel 15ee63c6e7 fix: harden CLI edge-case repros (#591) 
## Problem

I reproduced the selected open issue batch one by one and confirmed the reports were valid. The fixes all touch the CLI/runtime capture boundary, then the follow-up regression run exposed one over-broad runtime change in sub-composition host visibility and one CI-only baseline trap.

Closes #590, #589, #588, #587, #586, and #584.

## What this fixes

### CLI/runtime edge cases

- Makes the GSAP infinite-repeat lint rule ignore JavaScript comments, so literal `repeat:-1` text in comments is not flagged.
- Lets the compositions CLI inspect `<template>` content, count visual-only template descendants, estimate simple GSAP durations, and suppress root `data-start` warnings in sub-composition lint mode.
- Preserves runtime bootstrap scripts when body scripts are coalesced, and injects the runtime into a real `<head>` when source HTML has no head.
- Keeps #589 fixed by loading and rendering template-wrapped sub-composition content, while restoring host visibility to the shorter of the authored parent clip window and the child composition live timeline.
- Resolves snapshot/validate viewport size from root `data-width` / `data-height` instead of falling back to 1920x1080.
- Skips fully off-frame text boxes during contrast sampling and bounds-checks ring samples so contrast output no longer emits `null:1` / `NaN:1`.
- Marks muted videos as `data-has-audio="false"` in the core timing compiler, which fixes the same-src muted `<video>` + separate `<audio>` StaticGuard case.
- Keeps user-authored `hf-seek` listeners reachable during capture by preventing author scripts from being merged into the runtime bootstrap path.

### Shared helper cleanup

- Removes the stale producer-local timing compiler duplicate; producer compilation now consumes the core timing compiler.
- Centralizes HTML document helpers in core: fragment parsing, embedded runtime stripping, head/body script injection, and early-head injection.
- Centralizes the CLI layout/snapshot static HTML server.
- Adds browser-safe core subpath helpers for Lottie readiness and CLI screenshot clip calculation; Studio's Vite config keeps the screenshot clip helper self-contained so clean-checkout test startup does not value-import core `.ts` source.
- Replaces the engine parity-contract copy with a core re-export.
- De-duplicates render-job cleanup and Studio static file-serving callbacks.

### Regression hardening

- Replaces the embedded-runtime script stripping regex with a script-tag scanner that handles closing tags like `</script >`.
- Escapes inline script bodies before wrapping them in `<script>` tags, so authored `</script` and `<!--` text cannot break out of the injected wrapper script.
- Shares media-duration clamping between core and producer, with a 50 ms tolerance for ffprobe precision drift between local and CI media stacks.
- Pins the affected style fixture SFX durations in source so style-1 and style-9 compile deterministically.
- Restores the `vfr-screen-recording` video golden to the CI-stable baseline; the current CI failure showed the Linux render matches the old golden, while the locally refreshed macOS golden was the mismatch.

## Root cause

The CLI paths had accumulated assumptions that held for simple direct-root landscape compositions but not for current composition patterns: DOM queries did not enter template content, snapshot/validate used a fixed viewport, runtime and author scripts shared a coalescing bucket, and timing compilation treated every video as audio-bearing unless authors manually overrode it.

The style shard failures were not product regressions. Local and CI media probing disagreed on the short SFX clip duration by about 45 ms, and the compiler was clamping authored durations to the locally probed value. The shared clamp tolerance preserves explicit author/source durations for small probe precision differences while still clamping real overflows.

The vfr fast-shard failure was a bad baseline refresh: CI actual frames matched the old `vfr-screen-recording` baseline at 40+ dB PSNR, but mismatched the macOS-refreshed golden at ~18-22 dB. The fix is to keep the Docker/Linux-stable video golden and only retain the deterministic compiled snapshot change.

The sub-composition regression came from treating a host's authored parent window as the only visibility boundary. That made settled child overlays stay visible after their own live GSAP timeline ended. The corrected runtime behavior respects both contracts: parent clips still bound where the host can appear, and the child live timeline can end the host earlier.

## Verification

### Local checks

- `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts`
- `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts`
- `bun run --cwd packages/core test src/runtime/init.test.ts`
- `bun run --cwd packages/cli test src/commands/compositions.test.ts src/utils/compositionViewport.test.ts`
- `bun run build:hyperframes-runtime`
- `bun run --cwd packages/producer test --keep-temp --sequential style-12-prod style-5-prod`
- `bun run --cwd packages/producer test --sequential vfr-screen-recording hdr-hlg-regression style-7-prod`
- `bun run --cwd packages/core test src/compiler/htmlCompiler.test.ts src/compiler/timingCompiler.test.ts src/index.test.ts`
- `bunx oxfmt --check packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts`
- `bunx oxlint packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts`
- `bun run --cwd packages/core typecheck`
- `bun run --cwd packages/producer typecheck`
- `bun run --cwd packages/producer test --sequential style-1-prod style-9-prod`
- `bun run --filter @hyperframes/studio test` with `packages/core/dist` temporarily hidden to simulate clean-checkout config loading
- `git diff --check`

### CI artifact checks

- Inspected failed run `25225854394` job `73969147096`: style-1 failed only on `click-sfx` `1.044898` vs `1` duration/end.
- Inspected failed run `25225854394` job `73969147061`: style-9 failed only on SFX `1.044898`-based duration/end mismatches.
- Inspected failed run `25225854394` job `73969147048`: `vfr-screen-recording` compilation/audio passed, visual failed after comparing against the macOS-refreshed golden.
- Compared the first 10 uploaded CI vfr failure frames against the restored old baseline; minimum PSNR was `40.444705`, above the fixture threshold of `28`.

### Repro checks

- `bun packages/cli/src/cli.ts lint /tmp/hf-590-repro` now passes without `gsap_infinite_repeat`.
- `bun packages/cli/src/cli.ts snapshot /tmp/hf-587-repro --at 0.5 --timeout 1000` now writes a 1080x1920 PNG.
- `bun packages/cli/src/cli.ts validate /tmp/hf-588-repro --timeout 500` no longer emits `null:1` / `NaN:1` contrast output.
- `bun packages/cli/src/cli.ts validate /tmp/hf-586-repro --timeout 500 --contrast false` no longer emits the muted-video StaticGuard contract error.
- `bun packages/cli/src/cli.ts compositions /tmp/hf-589-gsap-repro` now reports `foo 0.5s 1920x1080 1 element`.
- `bun packages/cli/src/cli.ts snapshot /tmp/hf-589-gsap-repro --at 0.25 --timeout 2000` captures the expected template-backed red frame.
- `bun packages/cli/src/cli.ts snapshot /tmp/hf-584-repro --at 0.5,1.5 --timeout 500` captures the expected post-seek green frame.

### Browser verification

- Refreshed the local side-by-side comparison page at `qa-artifacts/pr-591-video-compare/index.html`.
- Served the comparison page locally and used `agent-browser` to load `style-12-prod`, play both videos quickly to the failed window, pause, and inspect the side-by-side frame.
- Browser proof screenshot: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12-labeled.png`.
- Browser proof recording: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12.webm`.
- Earlier Studio proof artifacts remain local-only: `qa-artifacts/dedupe-refactor-preview.png`, `qa-artifacts/dedupe-refactor-preview-after-play.png`, `qa-artifacts/dedupe-refactor-preview.webm`.

## Notes

- Browser proof and CI diagnostic artifacts are intentionally local-only and not committed.
- Studio's Vite config intentionally keeps the thumbnail clip helper inline because Vite/Vitest config startup runs through Node's loader before package source `.ts` imports are transformed.
- The committed PR diff changes `vfr-screen-recording/output/compiled.html` but no longer changes `vfr-screen-recording/output/output.mp4` relative to `main`.
- I attempted a local `linux/amd64` Docker validation to mirror CI, but the local Docker build was blocked by Debian package download failures. The arm64 Docker image also cannot launch the x64 Puppeteer headless shell under OrbStack. The vfr baseline decision is therefore based on the uploaded CI artifact comparison above.
- I kept this validated issue batch in one PR because the fixes overlap the same CLI/runtime capture surfaces.
 2026-05-02 00:00:08 +02:00
Miguel Ángel f9d38a1542 feat(core): add anime.js runtime adapter (#569) 
## Summary
- Adds a `RuntimeDeterministicAdapter` for anime.js v4+ alongside existing Lottie, Three.js, WAAPI, and CSS adapters
- Enables frame-accurate rendering of anime.js animations — the adapter converts seek time (seconds) to milliseconds and calls `.seek(timeMs)` on registered instances
- Auto-discovers running instances via `anime.running`; compositions can also register manually via `window.__hfAnime`

## Usage in compositions

```html
<script src="https://cdn.jsdelivr.net/npm/animejs@4.0.2/lib/anime.iife.min.js"></script>
<script>
  const anim = anime({
    targets: '.box',
    translateX: 250,
    rotate: '1turn',
    duration: 2000,
    autoplay: false,
  });
  window.__hfAnime = window.__hfAnime || [];
  window.__hfAnime.push(anim);
</script>
```

## Files changed
- `packages/core/src/runtime/adapters/animejs.ts` — adapter implementation
- `packages/core/src/runtime/adapters/animejs.test.ts` — 15 unit tests
- `packages/core/src/runtime/init.ts` — register adapter in runtime init
- `packages/core/src/runtime/window.d.ts` — add `anime` and `__hfAnime` globals

## Test plan
- [x] All 15 unit tests pass (`bun run --cwd packages/core test -- --run adapters/animejs`)
- [x] Build passes (`bun run build`)
- [x] Pre-commit hooks pass (lint, format, typecheck, commitlint)
- [x] Manual test: render a composition using anime.js animations
 2026-04-29 21:30:17 +02:00
Miguel Ángel 31afb4c1b3 Merge pull request #518 from heygen-com/fix/studio-preview-runtime-visibility 
fix: isolate Studio sub-composition previews
 2026-04-28 01:51:09 +02:00
Miguel Ángel 613f97793e test: add preview regression gate 2026-04-27 18:16:09 -04:00
Miguel Ángel 6ea2e54fcf fix: preserve anonymous inlined composition roots 2026-04-27 18:16:01 -04:00
Youssef Toufik 9e05322531 test(transparency): paint body background in fixture so it actually fails on the base SHA 
Per Miguel's review: the previous fixture had no body / root background, so it
passed against both the buggy and fixed code. The fix this PR makes (the
initTransparentBackground stylesheet injection in initializeSession) only
matters when a composition paints over the CDP default-background-color
override — exactly what we tell users not to do, but exactly what a
regression test must do.

Reproduced locally:

- base SHA (2935be6): pixel (10,10) decodes as rgba [16,16,16,255]
  (opaque #111 body bg leaks through the pre-navigation override that
  Chrome resets on goto)
- this head: pixel (10,10) decodes as rgba [0,0,0,0]
  (initTransparentBackground injects [data-composition-id]{background:transparent !important}
  AFTER navigation, force-overriding the body bg)

The pixel-level assertions in transparency-test.ts are unchanged — they
already require alpha=0 at (10,10). With the body bg painted, that
assertion now fails on any code path that doesn't actually preserve alpha
end-to-end.
 2026-04-27 21:05:56 +01:00
Youssef Toufik fe017b48c7 feat(producer): true alpha output for webm, mov, and png-sequence 
Extends RenderConfig.format with "png-sequence" and patches two correctness
gaps so the existing "webm" / "mov" values actually preserve the alpha
channel end-to-end.

Engine fixes:
- screenshotService.pageScreenshotCapture: drop optimizeForSpeed for PNG
  captures. The fast path uses an alpha-unaware codec that crushes real
  alpha values; kept for opaque jpeg captures where it is harmless.
- frameCapture: replace the inline setDefaultBackgroundColorOverride
  block (which fired pre-navigation and was reset by page.goto) with a
  proper initTransparentBackground() call inside initializeSession,
  after the window.__hf readiness poll. This also injects the
  html/body/[data-composition-id]{background:transparent !important}
  stylesheet so compositions with custom body / #root backgrounds do not
  defeat the override. Wired into both screenshot-mode and beginframe-mode
  branches.

Producer:
- RenderConfig.format extended to "mp4" | "webm" | "mov" | "png-sequence"
  with full JSDoc.
- Streaming encode is bypassed for png-sequence (frames go straight to
  disk). FORMAT_EXT extended.
- New Stage-5 png-sequence branch: mkdir outputPath, copy captured PNGs as
  frame_NNNNNN.png, copy audio.aac sidecar when audio is present.
- Stage-6 mux/faststart and the debug copy are wrapped in !isPngSequence.
- README.md: new "Transparent Video Output" section.

Tests:
- New fixture tests/transparency-regression/ tagged "transparency".
- New tsx script src/transparency-test.ts asserts pixel-level alpha for
  webm + png-sequence outputs. Wired as "test:transparency".
- Default "test" / "test:update" scripts pass --exclude-tags transparency
  so the golden-MP4 harness ignores the new fixture.

Verified locally on macOS arm64: typecheck clean across engine + producer,
producer renderOrchestrator vitest 10/10, transparency-test passes for
both webm and png-sequence with end-to-end pixel assertions.
 2026-04-27 20:33:45 +01:00
Miguel Ángel 2f60dd3727 fix(producer): serve symlinked render assets (#486) 2026-04-25 06:36:27 +02:00
Vance Ingalls 3da8c2e969 perf(producer): hdr benchmark harness — --tags filter, peak heap/RSS tracking, bench:hdr script (#382) 
## Summary

Make the existing benchmark harness genuinely useful for HDR perf work: positive `--tags` filter, peak heap/RSS sampling, a `bench:hdr` script, and a perf README documenting the captured April-2026 baseline. Lands first in the Chunk 8 sub-stack so subsequent perf PRs can be measured against a known starting point.

## Why

`Chunk 8A` of `plans/hdr-followups.md`. Wall-clock timing alone can't catch slow memory regressions like an unbounded image cache — peak RSS does. And the existing harness only had `--exclude-tags`, so HDR runs had to wait for unrelated SDR fixtures.

## What changed

**1. Positive `--tags` filter** in `benchmark.ts`. Adds `--tags hdr` so HDR runs don't have to wait for unrelated fixtures. Filters compose: a fixture must match `--tags` (if provided) AND must not match `--exclude-tags`.

**2. Peak heap + RSS tracking** in `executeRenderJob`. A 250 ms periodic `process.memoryUsage()` sampler runs alongside every render and reports `peakRssMb` / `peakHeapUsedMb` in `RenderPerfSummary`. Sampler is `unref`'d and always cleared in `finally` so it never keeps the event loop alive or leaks across jobs. Both fields are optional on the interface for back-compat with serialized older summaries.

**3. `bench:hdr` convenience script** plus a perf README at `tests/perf/README.md` documenting the harness, the new flags, and the captured April-2026 HDR baseline (PQ regression: 34.5 s / 272 MiB RSS, HLG regression: 11.5 s / 227 MiB RSS, both 1080p / 1 worker / 1 run).

The benchmark output table is widened and gains `PeakRSS` / `PeakHeap` columns. A new `avgOrNull` helper preserves `null` in the JSON when no run reported memory (avoids silently coercing missing data to 0 in older snapshots).

No behavior change for non-benchmark renders — the sampler runs in every `executeRenderJob` but its overhead is a single `process.memoryUsage()` call every 250 ms, well below noise.

## Test plan

- [x] `bunx tsc --noEmit -p packages/producer` — clean.
- [x] `bunx oxlint` / `bunx oxfmt --check` on changed files — clean.
- [x] `bun test src/services/` — 60/60 pass (frameDirCache, orchestrator, etc.).
- [x] `bunx tsx src/benchmark.ts --tags hdr --runs 1` — both HDR fixtures render successfully, summary table prints `PeakRSS`/`PeakHeap` columns, per-run output shows new memory line.
- [x] `bunx tsx src/benchmark.ts --tags nonexistent` — exits 1 with a helpful message naming the active filters.

## Stack

Chunk 8A of `plans/hdr-followups.md`. First PR in the Chunk 8 perf sub-stack; subsequent PRs (image cache, logger gating) measured against this baseline.
 2026-04-23 14:49:49 -07:00
Vance Ingalls 3089c8ee3a build(lfs): track tests/*/src/*.png via Git LFS (#376) 
## Summary

Track `tests/*/src/*.png` via Git LFS to mirror the existing policy for golden videos and `.mp4` fixtures.

## Why

`Chunk 11C` of `plans/hdr-followups.md`. Without this rule, regression suites that grow PNG fixtures over time would bloat the working-tree history and slow shallow clones.

## What changed

- `.gitattributes`: add `tests/*/src/*.png` to the LFS-tracked patterns.
- Migrates the six existing PNG fixtures (1.6 MB combined: `hdr-photo-pq.png` plus `heygen-promo-preview-assets/` screenshots) onto LFS in the same commit so the rule applies retroactively.

## Test plan

- [x] `git lfs ls-files` includes the HDR PNG fixtures after commit.
- [x] Working tree size for these files goes from 1.6 MB to 6 × ~130 B LFS pointers.

## Stack

Chunk 11C of `plans/hdr-followups.md`. Independent of all code changes.
 2026-04-23 09:31:48 -07:00
Vance Ingalls 8ffd007716 test(hdr-regression): tighten Window F maxFrameFailures budget after Chunk 4 fix (#375) 
## Summary

Tighten `hdr-regression` Window F `maxFrameFailures` from 5 → 0 now that Chunk 4 (matrix3d support + scene initial-state) has landed.

## Why

Window F (transform + scale + border-radius on the video itself) was the remaining known-fail in the `hdr-regression` suite, baked into the golden so the suite stayed green while Chunk 4 was outstanding.

After Chunk 4 fixed `parseTransformMatrix` (matrix3d support) and the shader-transitions initial-state, re-running the suite shows **0 failed frames** against the existing golden — the encoder is byte-deterministic, and Window F's GSAP rotation/scale happens to emit 2D `matrix()` rather than `matrix3d()`, so the same golden is still correct after the fix. Tightening the budget catches any drift in the layered HDR compositor immediately.

## What changed

- `tests/hdr-regression/meta.json`: `maxFrameFailures` 5 → 0 (matches `hdr-hlg-regression`).
- `tests/hdr-regression/README.md`: Window F row + Fix history section updated to reflect the new state.

## Test plan

- [x] `bun run test --filter hdr-regression` — passes with 0 failed frames at the new budget.

## Stack

Follow-up to Chunk 4 (transform & clipping). Reviewable separately so the budget tightening is decoupled from the code fix.
 2026-04-23 01:45:19 -07:00
Vance Ingalls 2e1a1d91a2 fix(engine,shader): handle matrix3d transforms and hide non-first scenes (#374) 
## Summary

Two correctness fixes in the HDR transform & clipping pipeline: `parseTransformMatrix` now handles `matrix3d(...)` (GSAP's default `force3D: true`), and shader-transitions sets every non-first scene to `opacity: 0` at `t=0` so the engine doesn't over-composite at the start.

## Why

`Chunk 4` of `plans/hdr-followups.md`. Transform extraction and border-radius computation existed but were dead — an HDR video with `rotation: 45` rendered un-rotated, and 3-scene compositions ghosted at `t=0` because every scene defaulted to CSS `opacity: 1` and contributed to the first frame.

## What changed

**Matrix3d support in `parseTransformMatrix`.** `DOMMatrix.toString()` emits `matrix3d` whenever any ancestor in the chain has used a 3D transform — most importantly GSAP's default `force3D: true`, which converts `translate(...)` into `translate3d(..., 0)`. Without this, every GSAP-driven transform was silently dropped during HDR compositing because `videoFrameInjector.getViewportMatrix()` would return `matrix3d(...)` and the blit path would parse it as `null` and fall back to identity. The 16-value column-major form is converted to its 2D affine projection (indices 0, 1, 4, 5, 12, 13 → m11, m12, m21, m22, m41, m42); Z, perspective, and out-of-plane rotation components are dropped.

**Initial-state opacity in `initEngineMode`.** The browser preview branch uses a GL canvas overlay during transitions, so scene opacity at `t=0` doesn't matter visually. The engine branch reads scene opacity directly via `queryElementStacking()` to decide which layers to composite. Without an explicit initial-state tween, every scene defaulted to CSS `opacity: 1` and contributed to the very first frame, causing ghosting/overlap until the first transition fired. `tl.set()` at position 0 anchors the initial state in the timeline graph so reverse seeks from inside a later transition restore it correctly.

These two fixes together make `el.transform` and `el.borderRadius` (already wired in Chunk 7A's `compositeHdrFrame`) actually flow through the GSAP-animated case, and keep the engine's per-frame compositing aligned with what the user sees in browser preview.

## Test plan

- [x] 6 new `alphaBlit.test.ts` cases (identity matrix3d, translate3d, scale + translate3d, rotateZ, malformed arg count, non-finite values).
- [x] Existing `hdr-regression` Window H already CSS-sets `#scene-b { opacity: 0 }` as a fallback; the new `tl.set` is redundant for that case but harmless and removes the need for compositions to remember the CSS workaround.
- [x] Manual: rotated HDR video (`rotation: 45`) appears rotated; `border-radius: 50%` clips to circle; 3-scene composition has no overlap at `t=0`.

## Stack

Chunk 4 of `plans/hdr-followups.md`. Window F of the regression suite documents the bug; the next PR in the stack tightens the `maxFrameFailures` budget to 0.
 2026-04-22 23:55:40 -07:00
Vance Ingalls a3d7cc1c95 refactor(producer): extract HDR compositing helpers and rename media metadata (#373) 
## Summary

Four behavior-preserving refactors that reduce complexity in `renderOrchestrator.ts` and clarify the engine ffprobe utility surface. Lands after the correctness fixes (Chunks 1–5) so the refactored code is already correct.

## Why

`Chunk 7` of `plans/hdr-followups.md`. The HDR composite block had grown a ~200 LOC inline closure with 14 captured deps, a repeated capture-options spread, a `extractVideoMetadata` name that now also handles still images, and per-frame re-creation of debug helpers.

## What changed

**7A — Hoist `compositeToBuffer` into a module-scoped helper.** Extract the inline HDR closure into a top-level `compositeHdrFrame()` that takes an `HdrCompositeContext` struct. Construct the context once at the top of the HDR render block and pass it through. Removes a deeply-nested closure from the middle of the orchestrator.

**7B — `buildHdrCaptureOptions()` helper.** Factor the repeated `{ ...captureOptions, skipReadinessVideoIds: ... }` spread into a named helper at the call site.

**7C — Rename `extractVideoMetadata` → `extractMediaMetadata`.** Reflects that the helper handles still images (PNG/JPEG/WebP) in addition to video. Update all callers in engine + producer (`videoFrameExtractor`, `htmlCompiler`, regression-harness, producer ffprobe re-export, tests). Re-export the old name as a deprecated alias from `@hyperframes/engine` for backward compatibility, plus the producer re-export shim.

**7D — Hoist debug counters to module scope.** `countNonZeroAlpha` and `countNonZeroRgb48` are now module-scoped so they aren't re-created per frame and so the closure has fewer captures.

Also touches the `hdr-regression` and `hdr-hlg-regression` README + `meta.json` files reviewed during this refactor.

## Test plan

- [x] `bunx tsc --noEmit -p packages/producer && bunx tsc --noEmit -p packages/engine` clean.
- [x] Engine tests: 313 pass, 0 fail (1218 expect calls).
- [x] `bunx oxlint` + `bunx oxfmt --check` clean on 8 changed source files.
- [x] Diff is structural only — no behavioral changes.

## Stack

Chunk 7 of `plans/hdr-followups.md`. Lands after the correctness fixes (Chunks 1–5) per the suggested merge order.
 2026-04-22 22:41:30 -07:00
Vance Ingalls 5256a93b2d feat(engine): wire options.hdr through chunkEncoder + dynamic SDR→HDR transfer (#370) 
## Summary

Three independent fixes that share a common thread: HDR config flowing correctly from `EngineConfig` down through every encoder. The headline fix: disk-based HDR encodes via `chunkEncoder` were silently producing BT.709-tagged output despite `options.hdr` being set.

## Why

`Chunk 3` of `plans/hdr-followups.md`. The streaming encoder was correct but `chunkEncoder.buildEncoderArgs` hard-coded BT.709 color tags and the `bt709` VUI block in `-x265-params`, even when callers passed an HDR `EncoderOptions`. Today this is harmless because `renderOrchestrator` routes native-HDR content to `streamingEncoder` and only feeds `chunkEncoder` sRGB Chrome screenshots — but the contract was a lie, and any future caller that wired HDR through `chunkEncoder` would silently get SDR output.

## What changed

**3A — `chunkEncoder` respects `options.hdr` (BT.2020 + mastering metadata).** When `options.hdr` is set, the libx265 software path emits `bt2020nc` plus the matching transfer (`smpte2084` for PQ, `arib-std-b67` for HLG) at the codec level *and* embeds master-display + max-cll SEI in `-x265-params` via `getHdrEncoderColorParams`. libx264 still tags BT.709 inside `-x264-params` (libx264 has no HDR support) but the codec-level color flags flip so the container describes pixels truthfully. GPU H.265 (nvenc/videotoolbox/qsv/vaapi) gets the BT.2020 tags but no `-x265-params` block, so static mastering metadata is omitted — acceptable for previews, not HDR-aware delivery.

**3B — `convertSdrToHdr` accepts a target transfer.** `videoFrameExtractor.convertSdrToHdr` was hard-coded to `transfer=arib-std-b67` (HLG) regardless of the surrounding composition's dominant transfer. `extractAllVideoFrames` now calls `analyzeCompositionHdr` first, then passes the dominant transfer (`"pq"` or `"hlg"`) into `convertSdrToHdr` so an SDR clip mixed into a PQ timeline gets converted with `smpte2084`, not `arib-std-b67`.

**3C — `EngineConfig.hdr` type matches its declared shape.** The IIFE for the `hdr` field returned `undefined` when `PRODUCER_HDR_TRANSFER` wasn't `"hlg"` or `"pq"`, but the field is typed as `{ transfer: HdrTransfer } | false`. Returning `false` matches the type and avoids a downstream `undefined` check.

## Test plan

- [x] `chunkEncoder.test.ts`: replaced the previous "HDR options ignored" assertions with 8 new specs covering BT.2020 + transfer tagging, master-display/max-cll embedding, libx264 fallback behavior, GPU H.265 + HDR (tags but no x265-params), and range conversion for both SDR and HDR CPU paths.
- [x] All 313 engine unit tests pass (5 new HDR specs).
- [x] `ffprobe` an HDR composition rendered through the chunk encoder path: shows `bt2020nc` color matrix, `smpte2084` transfer, and mastering display metadata.

## Stack

Chunk 3 of `plans/hdr-followups.md`. Independent of Chunks 1/4 (touches separate code paths).
 2026-04-22 20:36:29 -07:00
Vance Ingalls 60f4ebbf13 test(hdr-regression): tighten Window C maxFrameFailures budget after Chunk 1 fix (#369) 
## Summary

Tighten `hdr-regression` Window C `maxFrameFailures` from 30 → 5 now that Chunk 1 (opacity pipeline) has landed.

## Why

Window C (direct `<video>` opacity tween) was previously listed as a known failure with a `maxFrameFailures` budget of 30 to absorb expected drift until Chunk 1 landed. After the Chunk 1 fix, the regression test passes against the existing golden with **0 failed frames**. Tightening the budget catches any future drift in the opacity path immediately rather than letting up to 30 broken frames slip through.

## What changed

- `tests/hdr-regression/meta.json`: `maxFrameFailures` 30 → 5 (small budget remains for HEVC encoder noise).
- `tests/hdr-regression/README.md`: updated to mark Window C as fixed and note the tightened budget.

The HEVC encoder is byte-deterministic and the opacity fix doesn't perturb pixels at the PSNR ≥ 28 checkpoint threshold, so regenerating the golden produces byte-identical output. The golden is therefore unchanged. Window F (transform + border-radius) remains pending Chunk 4; its broken state is currently baked into the golden, so the suite is green and Chunk 4's regen will catch any drift.

## Test plan

- [x] `bun run test --filter hdr-regression` — passes with 0 failed frames at the new budget.

## Stack

Follow-up to Chunk 1 (opacity pipeline). Reviewable separately so the golden churn (none in this case) is decoupled from the code fix.
 2026-04-22 20:14:46 -07:00
Vance Ingalls 2d57918f64 fix(engine): stop clobbering native <video> opacity in HDR pipeline (#368) 
## Summary

Fix four interrelated bugs in the opacity pipeline. The headline fix: the HDR compositor was effectively ignoring direct-on-`<video>` opacity animation because the engine itself was clobbering inline opacity with `opacity: 0 !important` — switching to `visibility: hidden` resolves the bug at the root.

## Why

`Chunk 1` of `plans/hdr-followups.md`. This was the most user-visible bug in the entire follow-ups list: a GSAP-controlled opacity tween directly on a `<video>` element under HDR rendered at full brightness instead of fading.

## What changed

**1A — Stop clobbering native `<video>` opacity.** `screenshotService.injectVideoFramesBatch` and `syncVideoFrameVisibility` were applying `opacity: 0 !important` to native `<video>` elements to hide them under the injected `<img>`. That stomp clobbered any GSAP-controlled inline opacity, so the next seek read 0 from computed style and the comp went black. Switched to `visibility: hidden !important` only. Visibility hides the element from rendering without changing its opacity, so subsequent reads (and `queryElementStacking`) see the real GSAP value on every frame. The `parseFloat(...) || 1` recovery hack at `injectVideoFramesBatch` was specifically there to compensate for this stomp; it's now replaced with a `Number.isNaN` guard that defaults to 1 only when parsing actually fails.

**1B — `Number.isNaN` guards in `queryVideoElementBounds`.** `parseFloat(style.opacity) || 1` silently coerced a real opacity of 0 into 1. Switched to explicit `Number.isNaN` checks so opacity 0 stays 0. Same fix for `parseFloat(style.zIndex)`.

**1C — `instanceof HTMLElement` instead of cast.** `resolveRadius` cast `el as HTMLElement` to read `offsetWidth`/`Height`. SVG and other non-HTML elements would have crashed at runtime. Replaced the cast with an `instanceof HTMLElement` guard, and made the numeric fallback `Number.isNaN`-safe.

**1D — Opacity walk starts from the element itself.** The walk in `queryVideoElementBounds` started from `el.parentElement` for HDR videos to skip past the engine's forced `opacity: 0` on the element itself. Now that the engine never sets opacity, the special case is unnecessary — always walk from `el`. Kept the `isHdrEl` lookup because transform/border-radius logic further down still branches on it.

## Test plan

- [x] `bun run --filter @hyperframes/engine typecheck` clean.
- [x] `bun run --filter @hyperframes/engine test` — 308/308 passing.
- [x] `bun run --filter @hyperframes/producer typecheck` clean.
- [x] `oxlint` + `oxfmt --check` on both touched files.
- [x] `hdr-regression` Window C (the direct-opacity window) now passes against the regenerated golden — see follow-up PR in this stack which tightens the budget.

## Stack

Chunk 1 of `plans/hdr-followups.md`. Window C of the regression suite documents the bug; the next PR in the stack regenerates the golden and tightens its `maxFrameFailures` budget.
 2026-04-22 19:50:40 -07:00
Vance Ingalls d7c1050e44 test(producer): add hdr-regression and hdr-hlg-regression test suites (#365) 
## Summary

Replace the trivial `hdr-pq` and `hdr-image-only` tests with two consolidated, time-windowed regression suites that exercise the full HDR pipeline. These goldens are the safety net for every other PR in this stack.

## Why

The pre-existing HDR tests covered only a single full-bleed video or image with a static text label — none of the features that the HDR pipeline has to handle differently from SDR (opacity animation, z-ordered multi-layer compositing, transforms, border-radius clipping, shader transitions, multiple HDR sources, object-fit modes, mixed HDR+SDR layering, HLG transfer). This PR builds the missing safety net first so every subsequent fix can be proven correct.

## What changed

- New `packages/producer/tests/hdr-regression/` (PQ, BT.2020, ~20 s, 1080p, 8 windows A–H):
  - A: static baseline (HDR video + DOM overlay)
  - B: wrapper-opacity fade
  - C: direct-on-`<video>` opacity tween (documents the Chunk 1 bug)
  - D: z-order sandwich (DOM → HDR → DOM)
  - E: two HDR videos side-by-side (pins PR #289)
  - F: rotation + scale + border-radius (documents the Chunk 4 bug)
  - G: `object-fit: contain`
  - H: shader crossfade between HDR video and HDR image
- New `packages/producer/tests/hdr-hlg-regression/` (HLG, ARIB STD-B67, ~5 s, 2 windows A–B) — exercises the separate HLG LUT/OETF code path that previously had **zero** coverage.
- New `scripts/generate-hdr-photo-pq.py` synthesizes `hdr-photo-pq.png` with a cICP chunk for BT.2020/PQ/full.
- Removed `tests/hdr-pq/` and `tests/hdr-image-only/`.
- Updated `.github/workflows/regression.yml` HDR shard to run the new pair sequentially.
- All compositions follow the documented timed-element pattern (`data-start`, `data-duration`, `class="clip"` directly on each timed leaf — no wrapper inheritance).

## Test plan

- [x] Goldens generated with `bun run test:update --sequential`.
- [x] `ffprobe` confirms HEVC/yuv420p10le/bt2020nc/smpte2084 (PQ) and arib-std-b67 (HLG).
- [x] Suite green with `maxFrameFailures` budgets that absorb the documented Chunk 1 / Chunk 4 known-fails — tightened in follow-up PRs in this stack.

## Stack

Foundational PR for the HDR follow-ups stack (Chunk 0 of `plans/hdr-followups.md`). Every subsequent PR builds on this safety net.
 2026-04-22 15:43:04 -07:00
James RussoandClaude Opus 4.7 ffc06827c4 fix(engine): auto-normalize VFR video inputs to CFR before frame extraction (#360) 
* fix(engine): auto-normalize VFR video inputs to CFR before frame extraction

Screen recordings (macOS ScreenCaptureKit, QuickTime, phone videos) are
commonly variable-frame-rate. When such inputs hit the extractor's
`-ss <start> -i <video> -t <dur> -vf fps=N` pipeline, the fps filter
can emit fewer frames than requested — for a 4-second 30fps segment
starting mid-file, the output was ~90 frames instead of 120.

`FrameLookupTable.getFrameAtTime` returns null for out-of-range indices,
so the compositor held the last valid frame and the user perceived the
video as freezing. This matches the bug report from an X community post
where a user said "all of them freezes" on their screen recording scenes.

The engine already detects VFR via `metadata.isVFR` in ffprobe.ts but
never acted on it — the compiler only logged a warning. This change
mirrors the existing SDR→HDR normalization pattern: when a source is
detected as VFR, re-encode only the used segment with
`-fps_mode cfr -r <fps> -preset fast -crf 18` before extraction.

Scoping the re-encode to `[mediaStart, mediaStart+duration]` means a
30-second clip cut from a 60-minute screen recording pays ~1s of
transcode cost, not 18s. Benchmarked locally:

  Baseline (current):         32-39% duplicate frames, 25% frame-count
                              shortfall on mid-file segments.
  Tier 1 (flag changes only): ~same — fps filter issue is not flag-fixable.
  Tier 2 (CFR preflight):     1.7-6% duplicate frames, correct frame
                              count in every scenario tested.

The compiler warning that previously told users to manually re-encode
is downgraded to `console.info` since the engine now handles it.

— Rames Jusso

* refactor(engine): clean up VFR normalization loop after review

- Drop the `vfrNormDirCreated` flag; `mkdirSync({recursive:true})` is
  idempotent and cheap.
- Don't re-wrap the `VFR→CFR conversion failed` prefix — `convertVfrToCfr`
  already throws a message with that label; adding it again in the catch
  produced "VFR→CFR conversion failed: VFR→CFR conversion failed (exit 1)".
- Shorten the Phase 2b header comment; the function docstring above
  `convertVfrToCfr` already explains the failure modes and rationale.
- Note which frame windows the VFR fixture's select filter drops so the
  magic numbers are scannable.

No behavior change; 311/311 engine tests still pass.

— Rames Jusso

* test(engine): add VFR regression unit tests

Adds a describe block that synthesizes a VFR fixture via ffmpeg and asserts
the extractor produces the expected frame count (no shortfall) and no long
runs of duplicate frames — the user-visible "frozen screen recording"
symptom. Covers both a mid-file segment and the full-file case.

Guarded with describe.skipIf(!HAS_FFMPEG) because the CI Test job on
ubuntu-24.04 and the Windows test-windows job don't install ffmpeg. The
producer-level regression test in packages/producer/tests/vfr-screen-recording/
runs inside Dockerfile.test (which has ffmpeg) and is the primary CI signal
for this bug; these unit tests are supplementary coverage for local and
any ffmpeg-equipped CI environment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): add vfr-screen-recording regression test

End-to-end CI regression coverage for PR #360 via the existing
regression-harness: renders a 3s composition containing a real macOS
ScreenCaptureKit clip (r_frame_rate=120, avg≈36fps) seeked to
mediaStart=1, then PSNR-compares against a committed output.mp4.

Fixture src/clip.mp4 (108 KB) is a 5-second excerpt downscaled to 480×332
with -fps_mode passthrough to preserve the VFR timestamps. Content is the
public hyperframes OSS repo root page — see NOTICE.md for provenance.

With the fix applied, all 100 PSNR checkpoints pass. With the fix reverted,
66 of 100 fail (PSNR drops from ~43 dB to ~20 dB in the duplicate-frame
windows). Tagged "regression,video,vfr" so it runs in the fast shard
of .github/workflows/regression.yml automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(producer): regenerate vfr-screen-recording baseline in Docker

The committed golden output.mp4 was initially rendered on the host machine;
CI runs the renderer inside Dockerfile.test with a different Chrome +
ffmpeg build, producing pixel-level drift that failed PSNR at 54/100
checkpoints (~20 dB vs 41 dB in the VFR sparse-content windows). Both
renders are valid — the VFR source has inherent sampling ambiguity in
static segments, and different Chrome/ffmpeg builds make different valid
choices.

Regenerated the baseline via `bun run docker:test:update vfr-screen-recording`
so it matches the Docker environment CI actually uses. Matches the flow
the existing sub-composition-video, hdr-pq, etc. baselines were captured
with.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: document that producer test baselines must be captured in Docker

Hit this 2026-04-21 with the vfr-screen-recording regression test:
host-generated output.mp4 baseline tripped 54/100 PSNR checkpoints in CI
because Chrome + ffmpeg drift between the host and Dockerfile.test.

Document the `bun run --cwd packages/producer docker:test:update <name>`
flow so future contributors don't repeat the mistake.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 12:14:23 -07:00
Miguel Ángel b98093aa1c fix: remove hidden audio gain in renders (#362) 
## Summary

This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.

It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.

Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.

## Root Cause

The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:

- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
  - `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`

That exactly matches the issue reporter's measured scalar boost.

While investigating the workaround, I also found a second correctness bug:

- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output

Separately, the repo-level Studio typecheck could fail before any build step because:

- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck

## What Changed

1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build

## Why These Changes Are Needed

This is not a UX preference issue; it is a correctness and API contract issue.

- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.

Leaving the current behavior in place means:

- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed

## Testing

### Focused regression tests

Ran:

- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`

Result:

- `10 passed`

These tests specifically verify:

- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)

### Broader package verification

Ran:

- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`

Results:

- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker

## Known Verification Limitation

There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.

## User Impact

After this change:

- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts

Closes #361.
 2026-04-21 20:33:25 +02:00
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346) 
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
 2026-04-20 12:16:24 -07:00
Miguel Ángel a539266683 fix: harden CDN script inlining with linkedom (#352) 
* fix: harden CDN script inlining

* test: add spanish empire regression fixture
 2026-04-20 21:13:59 +02:00
Vance Ingalls a21a62b574 feat(engine): add HDR two-pass compositing — DOM layer + native HLG video (#288) 
## Summary

Compositions with HDR video AND DOM overlays (text, graphics, SDR video) couldn't render both correctly — either HDR data was lost (Chrome captures sRGB only) or DOM overlays were missing (FFmpeg pass-through skips Chrome). This PR adds in-memory alpha compositing that combines both.

## What it does

**Per-frame two-pass capture:**
1. **DOM pass** — Chrome screenshots the page with a transparent background (CDP alpha). HDR videos are hidden, leaving transparent holes where they go.
2. **HDR pass** — Pre-extracted native HLG/PQ frames (16-bit PNG from FFmpeg) are read from disk.
3. **Composite** — DOM pixels (sRGB RGBA8) are alpha-composited over HDR pixels (rgb48le) in Node.js memory, with sRGB→HLG/PQ conversion via a 256-entry lookup table.

**Key components:**
- `decodePng()` / `decodePngToRgb48le()` — Pure Node.js PNG decoders (no native dependencies). Support all 5 PNG filter types.
- `blitRgba8OverRgb48le()` — Alpha composite with per-pixel sRGB→HDR LUT conversion. Fast paths for alpha=0 (skip) and alpha=255 (overwrite).
- `initTransparentBackground()` + `captureAlphaPng()` — Split CDP transparent background setup (once) from per-frame screenshot capture (eliminates 2 CDP round-trips per frame).
- Single-pass FFmpeg extraction — All HDR frames extracted in one sequential FFmpeg run (avoids duplicate frames from per-frame `-ss` fast seek).

## Key design decisions

| Decision | Why |
|----------|-----|
| In-memory compositing (not FFmpeg overlay) | Eliminates ~2400 process spawns + temp files per render. Pure pixel math is 10x faster. |
| 16-bit PNG intermediate | Raw `-f rawvideo` loses color metadata, causing moiré artifacts. PNG is self-describing. |
| sRGB→HLG LUT (256 entries) | DOM content is sRGB. Without conversion, it appears orange-shifted in HLG stream. |
| Native HDR detection before extraction | `extractAllVideoFrames` converts SDR→HDR. Pre-extraction probe identifies original HDR sources so only truly-HDR videos get native extraction. |

## Files changed

| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | **NEW** — PNG decode, sRGB→HDR LUT, alpha compositing (14 tests) |
| `packages/engine/src/services/screenshotService.ts` | Transparent background CDP, `captureAlphaPng()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `hideVideoElements()` / `showVideoElements()` |
| `packages/engine/src/services/streamingEncoder.ts` | Input color space tags for rgb48le |
| `packages/producer/src/services/renderOrchestrator.ts` | Two-pass HDR capture loop, native HDR detection |

## How to test

Render a composition with an HDR video background and text overlays. Both should be visible — HDR video at full quality, text crisp with correct colors (not orange-shifted).

## Stack position

**3 of 6** — Stacked on #265 (HDR output pipeline). This is the foundation for all layered compositing that follows.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-19 16:26:54 -07:00
James RussoClaude Opus 4.7Codex
 		ad11de698c feat(producer): auto-fallback screenshot capture for raf and iframes (#331) 
* fix(core): drive adapter seeks when composition has no GSAP timeline

renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.

Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.

GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(producer): auto-fallback screenshot capture for raf and iframes

Co-Authored-By: Codex <codex@openai.com>

* test(producer): add render compatibility regression fixtures

Co-Authored-By: Codex <codex@openai.com>

* fix(core): scrub CSS animations via WAAPI currentTime

Co-Authored-By: Codex <codex@openai.com>

* test(producer): cover css keyframe renders

Co-Authored-By: Codex <codex@openai.com>

* fix(producer): propagate virtual time into iframe documents

Co-Authored-By: Codex <codex@openai.com>

* test(producer): refresh iframe docker golden

Co-Authored-By: Codex <codex@openai.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
 2026-04-18 15:08:52 -07:00
Miguel Ángel 1681350ac4 fix(preview): rewrite sub-composition asset urls in styles (#174) 
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting

## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.

## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`

## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
 2026-04-01 18:13:38 +02:00
Miguel Ángel 110ea12597 fix(core,engine,producer): handle id-less media in sub-composition renders (#96) 
## What

Move the id-less media fix into the shared timing compiler so producer can resolve durations for sub-composition videos before inlining, then carry the merged result through engine parsing, regression coverage, and the regression Docker image used in CI.

This PR now does five concrete things:

- assigns stable ids to id-less media in core `compileTimingAttrs()` so unresolved duration injection can target them
- keeps the engine-side `parseVideoElements()` support for `video[src]` plus the newer `data-duration` / natural-duration fallback from `main`
- makes producer prefer sub-composition media metadata over the later inlined-document parse when the same media id appears in both places
- makes `sub-composition-video` a runnable regression test by fixing its metadata and checking in the missing `output/compiled.html` snapshot
- removes the stale `pnpm-workspace.yaml` copy step from `Dockerfile.test`, so regression CI builds the Bun-based test image from the current workspace layout

## Why

- media without an explicit `id` could not participate in unresolved-duration resolution early enough
- producer could lose the resolved sub-composition timing by overwriting it with the later inlined parse
- the regression fixture intended to cover this case was not actually running in CI because its `meta.json` was incomplete and the required compiled snapshot was missing
- the regression image definition still expected a deleted `pnpm-workspace.yaml`, so GitHub Actions failed before the test shard could start

Putting the id-generation step in core makes the behavior reusable instead of relying on producer-only HTML patching.

## How

### Shared compiler

- core `compileTimingAttrs()` now auto-assigns stable ids to id-less `video` / `audio` tags
- those generated ids are returned in `unresolved`, so `injectDurations()` can add `data-duration` and `data-end` to the same media element later in the pipeline
- added core tests that cover auto-id assignment and duration injection for generated ids

### Producer

- when producer combines `subVideos` / `subAudios` with the media re-parsed from the final inlined HTML, it now lets the sub-composition metadata win
- this preserves the resolved/clamped timing already computed for nested media instead of overwriting it with the later parse
- `sub-composition-video` now has valid regression metadata and a checked-in `output/compiled.html` snapshot so CI actually executes it

### Engine

- resolved the merge conflict in `videoFrameExtractor` by keeping the broader `video[src]` parsing from this branch and the `data-duration` / natural-duration fallback that landed on `main`
- added a focused engine unit test for videos without ids

### CI image

- `Dockerfile.test` now copies only `package.json` and `bun.lock` at the workspace root before `bun install --frozen-lockfile`
- this matches the current monorepo layout and removes the obsolete pnpm-era dependency on `pnpm-workspace.yaml`

## Test plan

- [x] `bun run --filter @hyperframes/core test`
- [x] `bun run --filter @hyperframes/engine test`
- [x] `bun run --filter @hyperframes/producer test --update --sequential sub-composition-video`
- [x] `bun run --filter @hyperframes/producer test --sequential sub-composition-video`
- [x] Browser check with `agent-browser` against the compiled fixture page (`http://127.0.0.1:8123/compiled.html`)
- [x] Clean tracked-only Docker build of `Dockerfile.test` with the PR version of the file applied

## Notes

- Latest regression workflow is green on `main`, but before this PR the `sub-composition-video` fixture was being skipped by the harness rather than exercised end to end.
- The CI Docker fix was validated from a tracked-only export to avoid local untracked worktree artifacts affecting the result.
 2026-04-01 02:59:22 +02:00
Miguel Ángel 1230657ed0 fix(studio,runtime,engine,compiler): 8 bug fixes — audio, render, timeline, Lottie, thumbnails, video render (#133) 
## Summary

**Original 5 bugs fixed:**

- **Bug 1 — Audio silent after seek**: Added `Accept-Ranges` / `Content-Length` + `206 Partial Content` to the static asset server for byte-range seeking.
- **Bug 2 — Download 404 after restart**: Render list endpoint now registers on-disk renders into the in-memory job map.
- **Bug 3 — Timeline stops at GSAP end**: `resolveRootTimelineFromDocument` pads the GSAP timeline to match `data-duration` when the composition declares longer.
- **Bug 4 — Render stuck at 0%**: Store `jobState` reference (not spread copy) so async progress mutations reach the SSE stream.
- **Bug 5 — Lottie missing in preview/render**: Two fixes — (a) moved Lottie adapter before GSAP so `onUpdate` wins; (b) fixed bundler silently dropping external CDN `\<script src>` tags from sub-compositions (root cause: `$content(s).html()` returns `""` for external scripts).

**3 additional bugs fixed:**

- **Bug 6 — Blank thumbnails outside monorepo**: Implemented `generateThumbnail` in the CLI adapter using Puppeteer.
- **Bug 7 — Video empty in rendered sub-compositions**: Fixed `parseVideoElements` selector from `video[id][src]` to `video[src][data-start]` + auto-assign IDs.
- **Render errors**: Failed renders now show their error message in the renders panel.

## Commits

| Commit | Description |
| --- | --- |
| `3951c6f` | fix(studio): store render job reference instead of snapshot copy |
| `f331c30` | fix(studio): make previously-completed renders downloadable after restart |
| `a5e2d04` | fix(studio): add range request support for audio/video seeking in preview |
| `f24317a` | fix(runtime): pad GSAP timeline to data-duration when composition declares longer duration |
| `7cf38ca` | fix(runtime): fix Lottie adapter conflicting with GSAP-driven animations |
| `bc99209` | fix(studio): surface render error messages in the renders panel |
| `8fc9e8b` | fix(cli): implement generateThumbnail in studio adapter |
| `90277ea` | fix(engine): render videos inside sub-compositions that lack an explicit id |
| `f5bb579` | fix(compiler): preserve external CDN scripts from sub-compositions in bundle |

## Test plan

- [x] `golden-lyric-video`: seek → audio plays from seeked position
- [x] Any project: render → progress advances past 0%, reaches 100%
- [x] Any project: complete render, restart `hyperframes dev`, Download → works
- [x] `intro-vid`: play → runs full 5s (not stopping at 3s)
- [x] `hyperframe-build-up-demo`: play → rocket Lottie visible during 0-2s ✅ verified
- [x] Outside monorepo: Compositions sidebar shows thumbnail images (not blank)
- [x] `bug.zip` project: render → video in polaroid sub-composition appears in output
- [x] Trigger a failed render → error message shown
 2026-03-30 23:58:08 +02:00
JamesandClaude Opus 4.6 8d47665149 fix(render): update WebM golden baseline with working alpha 
The VP9 alpha encoding works correctly — verified by overlaying on a
green background with explicit VP9 decoder. FFmpeg's default decoder
doesn't expose VP9 alpha through ffprobe, but browsers and VP9-aware
decoders read it correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 02:42:16 +00:00
JamesandClaude Opus 4.6 efc7b82755 fix(render): fix WebM transparency — use screenshot mode + proper CDP setup 
- Force screenshot capture mode for WebM (beginFrame doesn't support
  alpha channel in chrome-headless-shell)
- Set Emulation.setDefaultBackgroundColorOverride once during session
  creation (matching experiment-framework's approach)
- Fix acquireBrowser to pass executable path in screenshot mode on
  Linux (was setting undefined, causing puppeteer-core to fail)
- Add Page.captureScreenshot params: fromSurface, captureBeyondViewport,
  optimizeForSpeed (matching experiment-framework)
- Fix regression harness extractMonoPcm16 for videos without audio
- Add getEncoderPreset unit tests
- Add webm-transparency regression test with golden baseline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 02:35:03 +00:00
JamesandClaude Opus 4.6 a146c9e527 test(render): add WebM regression test and getEncoderPreset unit tests 
- New regression test `webm-transparency`: minimal transparent composition
  rendered to WebM, validates VP9 codec and visual quality at 100
  checkpoints against golden baseline
- Extend regression harness: `renderConfig.format` field ("mp4" | "webm"),
  format-aware output paths and snapshot filenames
- Fix `extractMonoPcm16` to gracefully handle videos without audio
  streams (WebM without audio was throwing instead of returning empty)
- Unit tests for `getEncoderPreset()`: VP9/yuva420p for WebM,
  h264/yuv420p for MP4, preset mapping, quality preservation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 02:19:43 +00:00
Miguel ÁngelandClaude Opus 4.6 d3c6228d89 fix(producer): preserve @import rules during CSS composition scoping 
scopeCssToComposition corrupted @import url() rules because they have
no {} block. The selector regex ([^{}@]+)\{ treated the text after @
as a selector, producing invalid CSS like:

  @[data-composition-id="x"] import url('...')

This broke font loading, CSS variable resolution, and all composition
styling in rendered output.

Fix: extract @import rules before running the scoping regex, then
prepend them back unmodified.

Also adds:
- Regression test fixture (css-import-scoping)
- Common-mistakes docs: autoplay/loop, GSAP TextPlugin, sub-composition
  positioning
- Format fix for hyperframeLinter.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 18:51:05 -04:00
Miguel ÁngelandClaude Opus 4.6 36f3b8c87d test(regression): add editor-agent-prod regression fixture 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 15:33:13 -04:00
Vance IngallsandClaude Opus 4.6 61c5257402 fix(ci): update publish workflow to use bun install (#36) 
* fix(ci): update publish workflow to use bun install

pnpm-lock.yaml was removed in the bun migration but publish.yml
still referenced it. Use bun for install/build, keep pnpm for
publish (publishConfig overrides + --provenance).

* docs: update stale pnpm references to bun across docs and scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 08:46:58 -07:00
Vance Ingalls 20be2ea1c2 style: apply oxfmt baseline formatting across all source files (#25) 
## Summary
- Run `oxfmt .` across the entire codebase to establish formatted baseline
- 299 files changed — mechanical formatting only, no logic changes
- Double quotes, semicolons, 2-space indent, trailing commas, 100 print width

Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm format:check` — all 426 files pass
- [x] `pnpm -r typecheck` — all packages pass
- [x] `pnpm build` — all packages build
- [x] All 348 tests pass
 2026-03-23 17:15:14 -07:00
Vance IngallsandClaude Opus 4.6 b4023ae249 fix: update parity fixture to use local runtime path 
The fixture HTML had an empty src after the static.heygen.ai URL was
scrubbed. Point to the local hyperframe.runtime.iife.js which will be
copied into the fixtures dir by CI before the test runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 01:23:52 -07:00
Vance IngallsandClaude Opus 4.6 6ad12df1b6 fix: commit regression test golden baselines 
- Update .gitignore to allow packages/producer/tests/*/output/
- Commit compiled.html snapshots and output.mp4 golden baselines
  (MP4s tracked via Git LFS)

These were excluded by the blanket output/ gitignore rule but are
needed for regression tests to pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 00:56:52 -07:00
Vance IngallsandClaude Opus 4.6 9f8e5ba5a1 initial code (#2) 
* feat: initial code port from hyperframes-internal

Port all OSS-ready packages from the internal monorepo:
- @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime
- @hyperframes/cli — CLI for creating, previewing, and rendering compositions
- @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg)
- @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg)
- @hyperframes/ui-player — browser-based video player component
- @hyperframes/studio — composition editor (React frontend + Hono backend)

Includes regression test suite with Docker-based test harness.

All HeyGen-internal references, deployment infrastructure, and
proprietary assets have been removed. Package names migrated
from @app/* to @hyperframes/*.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrub internal codenames and stale references from OSS port

- Replace static.heygen.ai runtime URLs in test fixtures
- Remove internal CDN publish script (publish-hyperframe-runtime.ts)
- Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime
  with neutral names (studio, hyperframe-runtime, __hyperframeRuntime)
- Fix stale Vault API / localhost references in docs
- Remove broken deprecated_studio link

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove remaining internal codenames and stale references

- Delete stale producer README.md and PIPELINE.md (referenced nonexistent files)
- Replace "Cerberus" codename with "HyperFrames" in test design reviews
- Replace magic-edit postMessage identifiers with hf-preview/hf-parent
- Rename debug-magic-edit-timeline.ts to debug-timeline.ts
- Replace "Motion Cut" with "HyperFrames" in Timeline comments
- Fix studio/CLI references to nonexistent archive package
  (use local data/projects/ dir, stub render proxy)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 22:43:56 -07:00