## Summary
Compositions with HDR video AND DOM overlays (text, graphics, SDR video) couldn't render both correctly — either HDR data was lost (Chrome captures sRGB only) or DOM overlays were missing (FFmpeg pass-through skips Chrome). This PR adds in-memory alpha compositing that combines both.
## What it does
**Per-frame two-pass capture:**
1. **DOM pass** — Chrome screenshots the page with a transparent background (CDP alpha). HDR videos are hidden, leaving transparent holes where they go.
2. **HDR pass** — Pre-extracted native HLG/PQ frames (16-bit PNG from FFmpeg) are read from disk.
3. **Composite** — DOM pixels (sRGB RGBA8) are alpha-composited over HDR pixels (rgb48le) in Node.js memory, with sRGB→HLG/PQ conversion via a 256-entry lookup table.
**Key components:**
- `decodePng()` / `decodePngToRgb48le()` — Pure Node.js PNG decoders (no native dependencies). Support all 5 PNG filter types.
- `blitRgba8OverRgb48le()` — Alpha composite with per-pixel sRGB→HDR LUT conversion. Fast paths for alpha=0 (skip) and alpha=255 (overwrite).
- `initTransparentBackground()` + `captureAlphaPng()` — Split CDP transparent background setup (once) from per-frame screenshot capture (eliminates 2 CDP round-trips per frame).
- Single-pass FFmpeg extraction — All HDR frames extracted in one sequential FFmpeg run (avoids duplicate frames from per-frame `-ss` fast seek).
## Key design decisions
| Decision | Why |
|----------|-----|
| In-memory compositing (not FFmpeg overlay) | Eliminates ~2400 process spawns + temp files per render. Pure pixel math is 10x faster. |
| 16-bit PNG intermediate | Raw `-f rawvideo` loses color metadata, causing moiré artifacts. PNG is self-describing. |
| sRGB→HLG LUT (256 entries) | DOM content is sRGB. Without conversion, it appears orange-shifted in HLG stream. |
| Native HDR detection before extraction | `extractAllVideoFrames` converts SDR→HDR. Pre-extraction probe identifies original HDR sources so only truly-HDR videos get native extraction. |
## Files changed
| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | **NEW** — PNG decode, sRGB→HDR LUT, alpha compositing (14 tests) |
| `packages/engine/src/services/screenshotService.ts` | Transparent background CDP, `captureAlphaPng()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `hideVideoElements()` / `showVideoElements()` |
| `packages/engine/src/services/streamingEncoder.ts` | Input color space tags for rgb48le |
| `packages/producer/src/services/renderOrchestrator.ts` | Two-pass HDR capture loop, native HDR detection |
## How to test
Render a composition with an HDR video background and text overlays. Both should be visible — HDR video at full quality, text crisp with correct colors (not orange-shifted).
## Stack position
**3 of 6** — Stacked on #265 (HDR output pipeline). This is the foundation for all layered compositing that follows.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* fix(core): drive adapter seeks when composition has no GSAP timeline
renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.
Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.
GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(producer): auto-fallback screenshot capture for raf and iframes
Co-Authored-By: Codex <codex@openai.com>
* test(producer): add render compatibility regression fixtures
Co-Authored-By: Codex <codex@openai.com>
* fix(core): scrub CSS animations via WAAPI currentTime
Co-Authored-By: Codex <codex@openai.com>
* test(producer): cover css keyframe renders
Co-Authored-By: Codex <codex@openai.com>
* fix(producer): propagate virtual time into iframe documents
Co-Authored-By: Codex <codex@openai.com>
* test(producer): refresh iframe docker golden
Co-Authored-By: Codex <codex@openai.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting
## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.
## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`
## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
## What
Move the id-less media fix into the shared timing compiler so producer can resolve durations for sub-composition videos before inlining, then carry the merged result through engine parsing, regression coverage, and the regression Docker image used in CI.
This PR now does five concrete things:
- assigns stable ids to id-less media in core `compileTimingAttrs()` so unresolved duration injection can target them
- keeps the engine-side `parseVideoElements()` support for `video[src]` plus the newer `data-duration` / natural-duration fallback from `main`
- makes producer prefer sub-composition media metadata over the later inlined-document parse when the same media id appears in both places
- makes `sub-composition-video` a runnable regression test by fixing its metadata and checking in the missing `output/compiled.html` snapshot
- removes the stale `pnpm-workspace.yaml` copy step from `Dockerfile.test`, so regression CI builds the Bun-based test image from the current workspace layout
## Why
- media without an explicit `id` could not participate in unresolved-duration resolution early enough
- producer could lose the resolved sub-composition timing by overwriting it with the later inlined parse
- the regression fixture intended to cover this case was not actually running in CI because its `meta.json` was incomplete and the required compiled snapshot was missing
- the regression image definition still expected a deleted `pnpm-workspace.yaml`, so GitHub Actions failed before the test shard could start
Putting the id-generation step in core makes the behavior reusable instead of relying on producer-only HTML patching.
## How
### Shared compiler
- core `compileTimingAttrs()` now auto-assigns stable ids to id-less `video` / `audio` tags
- those generated ids are returned in `unresolved`, so `injectDurations()` can add `data-duration` and `data-end` to the same media element later in the pipeline
- added core tests that cover auto-id assignment and duration injection for generated ids
### Producer
- when producer combines `subVideos` / `subAudios` with the media re-parsed from the final inlined HTML, it now lets the sub-composition metadata win
- this preserves the resolved/clamped timing already computed for nested media instead of overwriting it with the later parse
- `sub-composition-video` now has valid regression metadata and a checked-in `output/compiled.html` snapshot so CI actually executes it
### Engine
- resolved the merge conflict in `videoFrameExtractor` by keeping the broader `video[src]` parsing from this branch and the `data-duration` / natural-duration fallback that landed on `main`
- added a focused engine unit test for videos without ids
### CI image
- `Dockerfile.test` now copies only `package.json` and `bun.lock` at the workspace root before `bun install --frozen-lockfile`
- this matches the current monorepo layout and removes the obsolete pnpm-era dependency on `pnpm-workspace.yaml`
## Test plan
- [x] `bun run --filter @hyperframes/core test`
- [x] `bun run --filter @hyperframes/engine test`
- [x] `bun run --filter @hyperframes/producer test --update --sequential sub-composition-video`
- [x] `bun run --filter @hyperframes/producer test --sequential sub-composition-video`
- [x] Browser check with `agent-browser` against the compiled fixture page (`http://127.0.0.1:8123/compiled.html`)
- [x] Clean tracked-only Docker build of `Dockerfile.test` with the PR version of the file applied
## Notes
- Latest regression workflow is green on `main`, but before this PR the `sub-composition-video` fixture was being skipped by the harness rather than exercised end to end.
- The CI Docker fix was validated from a tracked-only export to avoid local untracked worktree artifacts affecting the result.
The VP9 alpha encoding works correctly — verified by overlaying on a
green background with explicit VP9 decoder. FFmpeg's default decoder
doesn't expose VP9 alpha through ffprobe, but browsers and VP9-aware
decoders read it correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Force screenshot capture mode for WebM (beginFrame doesn't support
alpha channel in chrome-headless-shell)
- Set Emulation.setDefaultBackgroundColorOverride once during session
creation (matching experiment-framework's approach)
- Fix acquireBrowser to pass executable path in screenshot mode on
Linux (was setting undefined, causing puppeteer-core to fail)
- Add Page.captureScreenshot params: fromSurface, captureBeyondViewport,
optimizeForSpeed (matching experiment-framework)
- Fix regression harness extractMonoPcm16 for videos without audio
- Add getEncoderPreset unit tests
- Add webm-transparency regression test with golden baseline
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New regression test `webm-transparency`: minimal transparent composition
rendered to WebM, validates VP9 codec and visual quality at 100
checkpoints against golden baseline
- Extend regression harness: `renderConfig.format` field ("mp4" | "webm"),
format-aware output paths and snapshot filenames
- Fix `extractMonoPcm16` to gracefully handle videos without audio
streams (WebM without audio was throwing instead of returning empty)
- Unit tests for `getEncoderPreset()`: VP9/yuva420p for WebM,
h264/yuv420p for MP4, preset mapping, quality preservation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
scopeCssToComposition corrupted @import url() rules because they have
no {} block. The selector regex ([^{}@]+)\{ treated the text after @
as a selector, producing invalid CSS like:
@[data-composition-id="x"] import url('...')
This broke font loading, CSS variable resolution, and all composition
styling in rendered output.
Fix: extract @import rules before running the scoping regex, then
prepend them back unmodified.
Also adds:
- Regression test fixture (css-import-scoping)
- Common-mistakes docs: autoplay/loop, GSAP TextPlugin, sub-composition
positioning
- Format fix for hyperframeLinter.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): update publish workflow to use bun install
pnpm-lock.yaml was removed in the bun migration but publish.yml
still referenced it. Use bun for install/build, keep pnpm for
publish (publishConfig overrides + --provenance).
* docs: update stale pnpm references to bun across docs and scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The fixture HTML had an empty src after the static.heygen.ai URL was
scrubbed. Point to the local hyperframe.runtime.iife.js which will be
copied into the fixtures dir by CI before the test runs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update .gitignore to allow packages/producer/tests/*/output/
- Commit compiled.html snapshots and output.mp4 golden baselines
(MP4s tracked via Git LFS)
These were excluded by the blanket output/ gitignore rule but are
needed for regression tests to pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: initial code port from hyperframes-internal
Port all OSS-ready packages from the internal monorepo:
- @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime
- @hyperframes/cli — CLI for creating, previewing, and rendering compositions
- @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg)
- @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg)
- @hyperframes/ui-player — browser-based video player component
- @hyperframes/studio — composition editor (React frontend + Hono backend)
Includes regression test suite with Docker-based test harness.
All HeyGen-internal references, deployment infrastructure, and
proprietary assets have been removed. Package names migrated
from @app/* to @hyperframes/*.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: scrub internal codenames and stale references from OSS port
- Replace static.heygen.ai runtime URLs in test fixtures
- Remove internal CDN publish script (publish-hyperframe-runtime.ts)
- Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime
with neutral names (studio, hyperframe-runtime, __hyperframeRuntime)
- Fix stale Vault API / localhost references in docs
- Remove broken deprecated_studio link
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove remaining internal codenames and stale references
- Delete stale producer README.md and PIPELINE.md (referenced nonexistent files)
- Replace "Cerberus" codename with "HyperFrames" in test design reviews
- Replace magic-edit postMessage identifiers with hf-preview/hf-parent
- Rename debug-magic-edit-timeline.ts to debug-timeline.ts
- Replace "Motion Cut" with "HyperFrames" in Timeline comments
- Fix studio/CLI references to nonexistent archive package
(use local data/projects/ dir, stub render proxy)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Vance Ingalls