mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 07:19:52 +00:00
0ca1a8a9d1e36bd2cf202bfda58dc5a2df283f96
257
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vance Ingalls
|
a3d7cc1c95 |
refactor(producer): extract HDR compositing helpers and rename media metadata (#373)
## Summary
Four behavior-preserving refactors that reduce complexity in `renderOrchestrator.ts` and clarify the engine ffprobe utility surface. Lands after the correctness fixes (Chunks 1–5) so the refactored code is already correct.
## Why
`Chunk 7` of `plans/hdr-followups.md`. The HDR composite block had grown a ~200 LOC inline closure with 14 captured deps, a repeated capture-options spread, a `extractVideoMetadata` name that now also handles still images, and per-frame re-creation of debug helpers.
## What changed
**7A — Hoist `compositeToBuffer` into a module-scoped helper.** Extract the inline HDR closure into a top-level `compositeHdrFrame()` that takes an `HdrCompositeContext` struct. Construct the context once at the top of the HDR render block and pass it through. Removes a deeply-nested closure from the middle of the orchestrator.
**7B — `buildHdrCaptureOptions()` helper.** Factor the repeated `{ ...captureOptions, skipReadinessVideoIds: ... }` spread into a named helper at the call site.
**7C — Rename `extractVideoMetadata` → `extractMediaMetadata`.** Reflects that the helper handles still images (PNG/JPEG/WebP) in addition to video. Update all callers in engine + producer (`videoFrameExtractor`, `htmlCompiler`, regression-harness, producer ffprobe re-export, tests). Re-export the old name as a deprecated alias from `@hyperframes/engine` for backward compatibility, plus the producer re-export shim.
**7D — Hoist debug counters to module scope.** `countNonZeroAlpha` and `countNonZeroRgb48` are now module-scoped so they aren't re-created per frame and so the closure has fewer captures.
Also touches the `hdr-regression` and `hdr-hlg-regression` README + `meta.json` files reviewed during this refactor.
## Test plan
- [x] `bunx tsc --noEmit -p packages/producer && bunx tsc --noEmit -p packages/engine` clean.
- [x] Engine tests: 313 pass, 0 fail (1218 expect calls).
- [x] `bunx oxlint` + `bunx oxfmt --check` clean on 8 changed source files.
- [x] Diff is structural only — no behavioral changes.
## Stack
Chunk 7 of `plans/hdr-followups.md`. Lands after the correctness fixes (Chunks 1–5) per the suggested merge order.
|
||
|
Vance Ingalls
|
53e1aeaadc |
fix(producer): wire --crf and --video-bitrate CLI overrides into encoders (#372)
## Summary Re-wire the `--crf` and `--video-bitrate` CLI flags through the three encoder spawn sites in `renderOrchestrator.ts`. They were defined and parsed in the CLI but silently dropped before reaching ffmpeg. ## Why `Chunk 10` of `plans/hdr-followups.md`. PR #292 originally wired these through with a `baseEncoderOpts` object using `effectiveQuality`/`effectiveBitrate`; PR #268 rewrote the encode paths and reverted to `preset.quality` only, accidentally dropping the override. This is a user-facing regression — `hyperframes render --crf 18` was being silently ignored. ## What changed - At the three encoder spawn sites (HDR streaming, SDR streaming, disk-based encode), `quality` defaults to `preset.quality` but is overridden by `job.config.crf` when set, and `bitrate` is set from `job.config.videoBitrate`. Mutual exclusivity is enforced upstream in the CLI, so we don't re-check it here. - Fix the contradictory note in `docs/packages/cli.mdx` that claimed CRF/bitrate were now driven only by `--quality`. The flags table now lists `--crf` and `--video-bitrate` consistent with `docs/guides/rendering.mdx`. ## Test plan - [x] `hyperframes render --crf 18 ...` now respects the CRF override (verified via ffprobe of the encoded output). - [x] `hyperframes render --hdr ...` still works (no behavior change at the default path). - [x] `hyperframes render --help` shows all flags consistent with the docs. ## Stack Chunk 10 of `plans/hdr-followups.md`. Independent of all other chunks. |
||
|
Vance Ingalls
|
6fd99109c9 |
fix(producer): tighten resource lifecycle and harden file server (#371)
## Summary Five resource-management fixes in `renderOrchestrator.ts` and `fileServer.ts`: HDR encoder cleanup on non-abort errors, `frameDirMaxIndexCache` eviction, mid-transition abort responsiveness, pre-allocated transition buffers, and a path-traversal guard for the local file server. ## Why `Chunk 5` of `plans/hdr-followups.md`. These are independent leaks/hangs/security issues that had each been called out in prior PR reviews and never landed. ## What changed **5A — HDR encoder + `domSession` cleanup.** The HDR streaming encoder and `domSession` were spawned outside any outer `try/finally`, so a non-abort error between encoder spawn and the inner cleanup leaked the FFmpeg process and held the browser page open. Wrapped the entire HDR (and SDR streaming) capture path in a `try/finally` with explicit `*Closed` flags, and defensively close both in the outer `finally` if they haven't been closed already. `StreamingEncoder.close()` and `closeCaptureSession()` are both idempotent, so double-close is safe. **5B — `frameDirMaxIndexCache` + `hdrFrameDirs` eviction.** `frameDirMaxIndexCache` is module-scoped and grew monotonically: every render added entries that were never removed. Lifted `hdrFrameDirs` to the outer scope, drop the matching cache entry in the per-video `rmSync` block, and sweep any survivors in the outer `finally`. The on-disk frames themselves were already torn down with `workDir`; this just stops the in-process Map from leaking entries across renders. **5C — Abort signal between scene A and scene B.** During a shader transition the orchestrator captures scene A and scene B back-to-back inside a single outer frame iteration. An abort that arrived while scene A was capturing wouldn't be noticed until the next outer frame — after scene B had already been fully composited and discarded. Added `assertNotAborted()` at the top of the inner `[transBufferA, transBufferB]` loop so abort is observed before the second scene's DOM seek + screenshot. **5D — Pre-allocated transition buffers (already addressed).** The transition buffers (`transBufferA`, `transBufferB`, `transOutput`, `normalCanvas`) are pre-allocated outside the per-frame loop. The remaining `Buffer.from` copies sit in HDR transfer conversion (Chunk 8B territory) and image preload, neither of which is the per-frame hot path. **5E — `fileServer` path-traversal guard.** `fileServer.ts` joined `compiledDir` / `projectDir` with the request path and only checked `existsSync` + `isFile`. `path.join` normalizes `..` segments, so `GET /../etc/passwd` would resolve to `/etc/passwd` and be served straight off disk if the file existed. Added an `isPathInside(child, parent)` helper that resolves both sides and compares prefixes with the platform separator appended (so `/foo` doesn't match `/foobar`), and rejects any candidate that lands outside its intended root. ## Test plan - [x] `bun run --filter @hyperframes/producer typecheck` passes. - [x] `fileServer.test.ts` 13/13 pass (4 existing + 9 new `isPathInside` cases covering same-path, nested, prefix-only siblings, escaping traversal, traversal that resolves back inside, trailing-slash handling, and relative-path resolution). - [x] Manual: kill a render mid-flight with a non-abort error; no orphaned `ffmpeg` processes (5A). - [x] Manual: two render jobs back-to-back; cache cleared between jobs (5B). - [x] Manual: abort during a transition frame; stops promptly, not after scene B (5C). - [x] Manual: `GET /../../../etc/passwd` against the local file server returns 403/404 (5E). ## Stack Chunk 5 of `plans/hdr-followups.md`. |
||
|
Vance Ingalls
|
5de5df7fbb |
refactor(types): tighten type safety, dedupe HfTransitionMeta, prune dead LUT export (#366)
## Summary Four small, mechanical type-safety cleanups across `engine`, `producer`, and `shader-transitions`. Zero behavior change — pure pre-cleanup so the rest of the stack ships against a tighter baseline. ## Why `Chunk 6` of `plans/hdr-followups.md`. Several non-null assertions and a duplicate interface had accumulated as rebase artifacts and leftover work-in-progress; lands first because it touches files later chunks edit and removes friction during review. ## What changed - `renderOrchestrator.ts`: replace `layers[layerIdx]!` with a `for (const [layerIdx, layer] of layers.entries())` so both index and element come from the iterator. - `engine/types.ts`: drop the duplicate `HfTransitionMeta` interface (rebase artifact); the original definition above it is the documented one. The orphaned doc comment now precedes `HfProtocol`. - `shader-transitions/hyper-shader.ts`: keep the local `HfTransitionMeta` declaration (the package ships as a standalone CDN bundle and must not depend on `@hyperframes/engine`), but add a sync comment pointing at the source of truth in `engine/src/types.ts`. - `alphaBlit.ts` + `engine/index.ts`: drop `export` from `getSrgbToHdrLut` and remove its re-export. It was only ever called by the internal `blitRgba8OverRgb48le`; the public surface was dead code. ## Test plan - [x] `bun run --filter @hyperframes/engine typecheck` - [x] `bun run --filter @hyperframes/producer typecheck` - [x] `bun run --filter @hyperframes/shader-transitions typecheck` - [x] `bun run --filter @hyperframes/engine test` — 308/308 pass (no test changes; assertions removed in code only). ## Stack Chunk 6 of `plans/hdr-followups.md`. Mechanical cleanup landed early per the suggested merge order. |
||
|
Miguel Ángel
|
d740f5ce42 |
fix: nested GSAP sub-composition lint and render handling (#405)
## Summary - allow nested sub-composition files to inherit GSAP from their host without tripping `missing_gsap_script` - keep nested render seeks stable for sub-compositions without regressing producer baselines - stop producer render-hint detection from treating the compiler's own nested mount retry wrapper as user-authored `requestAnimationFrame()` usage ## Root Cause - the core linter treated template-based nested compositions like standalone root compositions, so it incorrectly required a local GSAP loader even when the host composition already provided GSAP - producer `detectRenderModeHints()` runs before CDN scripts are inlined, so nested GSAP exports were never failing because of the GSAP payload itself - the nested-only false positive came from the compiler-generated mount bootstrap that waits for the inlined sub-composition root with `requestAnimationFrame()` before running the hoisted inline script - preview and export seek paths also needed to stay split so the nested timeline re-arm behavior that stabilizes scrubbing does not collapse render baselines ## What Changed - lint: keep the nested GSAP false-positive fix and regression coverage for template sub-compositions - runtime: keep the render-seek behavior that preserves nested child offsets during export without changing preview scrubbing behavior - producer: mark compiler-owned mount bootstrap blocks and strip only those blocks before scanning inline scripts for raw `requestAnimationFrame()` - producer tests now cover both cases: compiler-generated wrappers are ignored, but real user-authored nested `requestAnimationFrame()` still opts into screenshot mode ## Validation - `bun test packages/core/src/lint/rules/gsap.test.ts` - `bun test packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxfmt packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxlint packages/producer/src/services/htmlCompiler.ts packages/producer/src/services/htmlCompiler.test.ts` - `bun run --filter @hyperframes/producer test --sequential chat style-11-prod` - `style-11-prod` passed locally - `chat` still shows local-only visual drift on this macOS/ARM workstation, but the render metadata now reports `renderModeHints.recommendScreenshot=false`, which is the concrete acceptance condition for `#402` - Docker CI-image repro is blocked locally by OrbStack x86/arm64 loader mismatch, so final regression confirmation is deferred to GitHub Actions Closes #392 Closes #402 |
||
|
James RussoandClaude Opus 4.7
|
ffc06827c4 |
fix(engine): auto-normalize VFR video inputs to CFR before frame extraction (#360)
* fix(engine): auto-normalize VFR video inputs to CFR before frame extraction
Screen recordings (macOS ScreenCaptureKit, QuickTime, phone videos) are
commonly variable-frame-rate. When such inputs hit the extractor's
`-ss <start> -i <video> -t <dur> -vf fps=N` pipeline, the fps filter
can emit fewer frames than requested — for a 4-second 30fps segment
starting mid-file, the output was ~90 frames instead of 120.
`FrameLookupTable.getFrameAtTime` returns null for out-of-range indices,
so the compositor held the last valid frame and the user perceived the
video as freezing. This matches the bug report from an X community post
where a user said "all of them freezes" on their screen recording scenes.
The engine already detects VFR via `metadata.isVFR` in ffprobe.ts but
never acted on it — the compiler only logged a warning. This change
mirrors the existing SDR→HDR normalization pattern: when a source is
detected as VFR, re-encode only the used segment with
`-fps_mode cfr -r <fps> -preset fast -crf 18` before extraction.
Scoping the re-encode to `[mediaStart, mediaStart+duration]` means a
30-second clip cut from a 60-minute screen recording pays ~1s of
transcode cost, not 18s. Benchmarked locally:
Baseline (current): 32-39% duplicate frames, 25% frame-count
shortfall on mid-file segments.
Tier 1 (flag changes only): ~same — fps filter issue is not flag-fixable.
Tier 2 (CFR preflight): 1.7-6% duplicate frames, correct frame
count in every scenario tested.
The compiler warning that previously told users to manually re-encode
is downgraded to `console.info` since the engine now handles it.
— Rames Jusso
* refactor(engine): clean up VFR normalization loop after review
- Drop the `vfrNormDirCreated` flag; `mkdirSync({recursive:true})` is
idempotent and cheap.
- Don't re-wrap the `VFR→CFR conversion failed` prefix — `convertVfrToCfr`
already throws a message with that label; adding it again in the catch
produced "VFR→CFR conversion failed: VFR→CFR conversion failed (exit 1)".
- Shorten the Phase 2b header comment; the function docstring above
`convertVfrToCfr` already explains the failure modes and rationale.
- Note which frame windows the VFR fixture's select filter drops so the
magic numbers are scannable.
No behavior change; 311/311 engine tests still pass.
— Rames Jusso
* test(engine): add VFR regression unit tests
Adds a describe block that synthesizes a VFR fixture via ffmpeg and asserts
the extractor produces the expected frame count (no shortfall) and no long
runs of duplicate frames — the user-visible "frozen screen recording"
symptom. Covers both a mid-file segment and the full-file case.
Guarded with describe.skipIf(!HAS_FFMPEG) because the CI Test job on
ubuntu-24.04 and the Windows test-windows job don't install ffmpeg. The
producer-level regression test in packages/producer/tests/vfr-screen-recording/
runs inside Dockerfile.test (which has ffmpeg) and is the primary CI signal
for this bug; these unit tests are supplementary coverage for local and
any ffmpeg-equipped CI environment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(producer): add vfr-screen-recording regression test
End-to-end CI regression coverage for PR #360 via the existing
regression-harness: renders a 3s composition containing a real macOS
ScreenCaptureKit clip (r_frame_rate=120, avg≈36fps) seeked to
mediaStart=1, then PSNR-compares against a committed output.mp4.
Fixture src/clip.mp4 (108 KB) is a 5-second excerpt downscaled to 480×332
with -fps_mode passthrough to preserve the VFR timestamps. Content is the
public hyperframes OSS repo root page — see NOTICE.md for provenance.
With the fix applied, all 100 PSNR checkpoints pass. With the fix reverted,
66 of 100 fail (PSNR drops from ~43 dB to ~20 dB in the duplicate-frame
windows). Tagged "regression,video,vfr" so it runs in the fast shard
of .github/workflows/regression.yml automatically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(producer): regenerate vfr-screen-recording baseline in Docker
The committed golden output.mp4 was initially rendered on the host machine;
CI runs the renderer inside Dockerfile.test with a different Chrome +
ffmpeg build, producing pixel-level drift that failed PSNR at 54/100
checkpoints (~20 dB vs 41 dB in the VFR sparse-content windows). Both
renders are valid — the VFR source has inherent sampling ambiguity in
static segments, and different Chrome/ffmpeg builds make different valid
choices.
Regenerated the baseline via `bun run docker:test:update vfr-screen-recording`
so it matches the Docker environment CI actually uses. Matches the flow
the existing sub-composition-video, hdr-pq, etc. baselines were captured
with.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: document that producer test baselines must be captured in Docker
Hit this 2026-04-21 with the vfr-screen-recording regression test:
host-generated output.mp4 baseline tripped 54/100 PSNR checkpoints in CI
because Chrome + ffmpeg drift between the host and Dockerfile.test.
Document the `bun run --cwd packages/producer docker:test:update <name>`
flow so future contributors don't repeat the mistake.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
b98093aa1c |
fix: remove hidden audio gain in renders (#362)
## Summary
This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.
It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.
Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.
## Root Cause
The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:
- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
- `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`
That exactly matches the issue reporter's measured scalar boost.
While investigating the workaround, I also found a second correctness bug:
- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output
Separately, the repo-level Studio typecheck could fail before any build step because:
- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck
## What Changed
1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build
## Why These Changes Are Needed
This is not a UX preference issue; it is a correctness and API contract issue.
- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.
Leaving the current behavior in place means:
- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed
## Testing
### Focused regression tests
Ran:
- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`
Result:
- `10 passed`
These tests specifically verify:
- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)
### Broader package verification
Ran:
- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`
Results:
- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker
## Known Verification Limitation
There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.
## User Impact
After this change:
- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts
Closes #361.
|
||
|
Vance Ingalls
|
00af29c169 |
fix(cli): forward --hdr through Docker render + HDR docs (#346)
## Summary This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI. The branch now does four things: - forwards `--hdr` through the Docker render path in the CLI - adds and expands HDR documentation across the docs site - adds first-class HDR still-image support to the engine/producer pipeline - adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags ## What changed ### CLI and docs - `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI - added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs - documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes ### Engine and producer HDR image support - added `ImageElement` support to the engine composition model and parsing path - threaded image elements through producer compilation and orchestration - probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source - included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order - integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays - forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic - skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows ### HDR metadata robustness - added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs - this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ ### Regression coverage and fixture cleanup - added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end - added `hdr-pq`, a focused HDR PQ regression fixture for the video path - updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only` - removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI - added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests ## Why The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking. The practical issue this closes is: - local host runs could pass while CI failed `hdr-image-only` - the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering - root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment - parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments ## Test plan ### Local targeted checks ```bash bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts ``` ### Producer regression runs on host ```bash bun run --cwd packages/core build:hyperframes-runtime:modular bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only ``` Observed result: - `fast` shard: 7 passed, 0 failed - `hdr` shard: 2 passed, 0 failed ### CI-equivalent Docker verification ```bash docker build -f Dockerfile.test -t hyperframes-producer:test . docker run --rm \ --security-opt seccomp=unconfined \ --shm-size=4g \ -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \ hyperframes-producer:test \ --sequential hdr-pq hdr-image-only ``` Observed result: - `hdr-image-only`: passed - `hdr-pq`: passed - shard summary: 2 passed, 0 failed ### Specific regression fixed Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with: - missing `"[Render] HDR source detected — output: PQ ..."` log line - full-frame visual mismatch across all 100 checkpoints - PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes. |
||
|
Miguel Ángel
|
a539266683 |
fix: harden CDN script inlining with linkedom (#352)
* fix: harden CDN script inlining * test: add spanish empire regression fixture |
||
|
Vance IngallsandClaude Opus 4.6
|
99a903be2f |
feat(hdr): layered HDR compositing, shader transitions, and HDR image support (#268)
* feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes - 15 GLSL→TypeScript shader transitions on rgb48le buffers - Dual-scene compositing with scene detection via window.__hf.transitions - --hdr flag gates ffprobe probing (zero overhead on SDR compositions) - Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT - Buffer.from() copy in writeFrame() fixes streaming encoder race condition - SDR rendering fixes (three stacked bugs) - Object.assign fix for window.__hf preservation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: tighten shader smoke thresholds + assert .scene contract - Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268. * fix(hdr): restore VIRTUAL_TIME_SHIM and applyRenderModeHints in renderOrchestrator Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes") accidentally removed two pieces of the deterministic rendering pipeline: 1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`, which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven animations advance only when `window.__hf.seek(t)` is called. 2. The `applyRenderModeHints` function and its post-`compileForRender` call site, which auto-forces screenshot capture mode for compositions the compiler flagged as needing it (RAF, iframes, etc.). Without (1), RAF animations advanced by wall-clock between the main-loop seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat` lost its automatic fallback to screenshot mode and the child-document motion stopped being captured. Both helpers are still produced by `htmlCompiler` and exercised by `renderOrchestrator.test.ts` — the orchestrator just stopped calling them. Restored: - Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js` - Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer` call sites (probe + main render) - Re-add `applyRenderModeHints` (matching the test expectations) and call it immediately after `compileForRender` - Persist `renderModeHints` in `summary.json` and the "Compiled composition metadata" log line Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression failures on `feat/hdr-layered-compositing`. Made-with: Cursor * test(engine): expand sampleRgb48le coverage + audit Uint16Array alignment Adds: - 8 new sampleRgb48le bilinear-interpolation tests covering boundary pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers. - uint16-alignment-audit.test.ts documenting the alignment requirement for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE. Background: ~105 hot-loop sites in shader transitions still use readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut overhead but requires guaranteed even byteOffsets — these tests document the contract before any future refactor lands. * fix(engine,producer): mask DOM layers during HDR layered compositing The HDR layered compositor blits z-ordered layers over a shared canvas. DOM layers used a full-page screenshot from `captureAlphaPng`, which captures *every* painted pixel on the page — root background, sibling-scene content, overlay UI elements that aren't part of the current layer. Those opaque pixels were then blitted over the canvas, overwriting any HDR content composited beneath in earlier layers. The previous workaround toggled `display:none` on hide ids via `hideVideoElements`/`showVideoElements`. That correctly hid native videos but did nothing about the root composition's background or about overlay elements that the layer grouping considered part of a different layer. This commit replaces the workaround with a precise CSS mask installed before each DOM screenshot: 1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and re-shows the layer's elements (and their descendants and their injected `__render_frame_*` siblings) with `visibility: visible !important`. CSS visibility is *not* multiplicative through descendants — a child with `visibility: visible` overrides an ancestor's `visibility: hidden`, so deeply nested layer content still paints even though every intermediate ancestor is hidden by the mass-hide rule. 2. Non-layer data-start ids are inline-hidden with `visibility: hidden !important`. Inline `!important` beats stylesheet `!important`, so this overrides the show rule for elements that fall under a show selector but should NOT paint — most importantly HDR videos and other-layer SDR videos that live as descendants of `#root`. 3. `removeDomLayerMask` tears the stylesheet down and clears the inline `visibility`/`opacity` properties so subsequent video frame injection gets a clean slate. Crucially the mask only sets `visibility`, never `opacity`. CSS opacity *is* multiplicative — `opacity: 0` on `#root` would zero out every descendant including layer videos, even with `visibility: visible`. We also extend `initTransparentBackground` to force the composition root (`[data-composition-id]`) transparent in addition to `html`/`body`, because compositions almost always set `#root { background: ... }` and that background paints across the whole viewport otherwise. Both compositing paths use the new helpers: - The per-layer DOM branch (`compositeToBuffer`) for normal frames. - The transition path (single DOM screenshot per scene) so transition frames also get a clean per-scene capture. Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`: per-layer pixel-add accounting, dumps of every captured DOM PNG, and a periodic raw `rgb48le` snapshot of the composite buffer. These were essential to diagnosing the root-overwrite bug and stay zero-cost in normal renders. Also stops the workDir / per-video frame-dir cleanup when `KEEP_TEMP=1` so the dumps survive past frame N. Made-with: Cursor * fix(engine): preserve GSAP-applied opacity across DOM-layer captures SDR clips inside an HDR composition were rendering at full opacity even when the user had animated their wrapper opacity (e.g. fade-in or yoyo). Two bugs in the per-layer screenshot path conspired to drop the GSAP-applied opacity on the floor: 1. removeDomLayerMask was unconditionally calling `el.style.removeProperty("opacity")` on every wrapper after each layer capture. applyDomLayerMask only ever sets `visibility`, so the only inline opacity present is the value GSAP wrote. Stripping it between layer captures means that on the next capture (at the same timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline is already at that time — the opacity is never restored, and the wrapper renders fully opaque. 2. injectVideoFramesBatch was reading the source <video>'s computed opacity via `parseFloat(computedStyle.opacity) || 1` and copying it onto the injected <img>. Because syncVideoFrameVisibility forces the <video> to `opacity: 0 !important` to hide it during capture, the computed value is always 0, which `|| 1` then silently flips to full opacity. The <img> is a sibling of the <video> inside the same wrapper, so it should inherit opacity from the wrapper directly instead of having a value hard-set on it. Fix both: drop the opacity removal in removeDomLayerMask, skip opacity when copying visual properties from <video> to <img>, and explicitly clear any stale inline opacity on the <img> so it inherits from the wrapper that GSAP is animating. Made-with: Cursor * fix(producer): correct hdrLayerStartTimes typo to hdrVideoStartTimes The diagnostic logging block in executeRenderJob's HDR layer composite path referenced an undeclared `hdrLayerStartTimes` map. The correct variable, declared and populated earlier in the same function, is `hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer masking work and broke the producer build/typecheck on CI. Made-with: Cursor * fix(engine): restore video opacity copy to injected frame img Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on the assumption that the <img> sibling would inherit GSAP's opacity from a shared wrapper. That breaks any composition where GSAP animates opacity directly on the <video> element itself: the <img> has no animated ancestor and renders at full opacity throughout any fade, even when the user's intent is partial or zero opacity. The CI `style-7-prod` and `style-8-prod` regressions caught this: the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut because the <img> inherited opacity 1 regardless of GSAP's tween. Restore the old explicit copy from `computedStyle.opacity` to the <img>'s inline opacity, with the `|| 1` fallback intentionally preserved. The fallback is load-bearing: GSAP's seek does not re-apply tweens that have already completed, so post-fade frames read opacity 0 from the stale `opacity: 0 !important` we apply to hide the native <video>. The `|| 1` recovers the tween's end-state opacity 1 for those frames, matching the final on-screen intent and the existing baseline renders. Handles both DOM shapes: - GSAP on wrapper: video's own computed opacity is 1, img set to 1, wrapper's opacity applies via stacking as before. - GSAP on <video>: video's computed opacity is the tween value, copied to img directly since they are siblings. Fixes: - style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33) - style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
8548a17771 |
feat(hdr): GSAP transforms and border-radius masks on HDR video (#290)
## Summary HDR video elements with GSAP animations (position, scale, rotation, opacity) and CSS border-radius rendered without any transforms applied — the video just sat at (0,0) full-size. This PR adds affine transform support and rounded-corner masking for natively-composited HDR video. ## What it does **Affine blit with bilinear interpolation:** - `blitRgb48leAffine()` — Takes a 4x4 DOMMatrix and maps each destination pixel back to source coordinates via the inverse transform. Bilinear interpolation between the 4 nearest source pixels produces smooth edges under rotation and non-integer scaling. Optional opacity and border-radius parameters. - `parseTransformMatrix()` — Parses CSS `matrix(a,b,c,d,e,f)` strings into `[a,b,c,d,e,f]` tuples. **Accumulated viewport matrix:** - `getViewportMatrix()` — Walks the `offsetParent` chain from element to viewport, accumulating position offsets and CSS transforms at each level. Correctly handles `transform-origin` using the CSS sandwich: `translate(origin) × M × translate(-origin)`. This is critical because GSAP animates transforms on wrapper divs, not directly on the video element. **Effective opacity:** - `getEffectiveOpacity()` — Multiplies opacity values walking up the ancestor chain. Uses `Number.isNaN()` (not `|| 1`) so opacity:0 isn't incorrectly treated as 1. **Border-radius masks:** - `roundedRectAlpha()` — Per-pixel anti-aliased rounded-rectangle mask with support for independent corner radii. - `getEffectiveBorderRadius()` — Walks ancestors for `overflow:hidden` + border-radius. Resolves percentage values (e.g., `50%` for circles) via `offsetWidth`/`offsetHeight`. **Layout dimensions for extraction:** - Uses `offsetWidth`/`offsetHeight` (unaffected by CSS transforms) instead of `getBoundingClientRect()` (which returns the transformed bounding box and wobbles under rotation). ## Files changed | File | What changed | |------|-------------| | `packages/engine/src/utils/alphaBlit.ts` | `blitRgb48leAffine()`, `parseTransformMatrix()`, `roundedRectAlpha()`, `cornerAlpha()` | | `packages/engine/src/services/videoFrameInjector.ts` | `getViewportMatrix()`, `getEffectiveOpacity()`, `getEffectiveBorderRadius()`, `layoutWidth`/`layoutHeight` on `ElementStackingInfo` | | `packages/producer/src/services/renderOrchestrator.ts` | Affine blit path, extraction at layout dimensions, border-radius parameter passing | ## How to test Render a composition with an HDR video that has GSAP scale + rotation animation and a `border-radius: 50%` wrapper (circle mask). The video should rotate smoothly with round edges — no wobble, no sharp corners. ## Stack position **5 of 6** — Stacked on #289 (z-ordered layers). Adds transform and masking support to the HDR blit that the layer compositor uses. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance IngallsandClaude Opus 4.6
|
0cc79a35b0 |
feat(hdr): z-ordered multi-layer compositing with PQ support (#289)
* feat(hdr): add z-ordered multi-layer compositing with PQ support Per-frame z-order analysis groups elements into DOM and HDR layers, composited bottom-to-top. Adjacent DOM elements merge into single screenshots. PQ (HDR10/smpte2084) support via sRGB-to-PQ LUT with 203-nit SDR reference white. queryElementStacking walks DOM for effective z-index, groupIntoLayers splits on HDR/DOM boundaries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(hdr): address review feedback across stack - Document groupIntoLayers tie-break (V8 stable sort → DOM order). - Expand layerCompositor docstring: merge rationale, visibility inclusion. - Add tests: empty input, negative z-index, stable tie-break at equal z. - Document getEffectiveZIndex CSS stacking-context limitations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
a21a62b574 |
feat(engine): add HDR two-pass compositing — DOM layer + native HLG video (#288)
## Summary Compositions with HDR video AND DOM overlays (text, graphics, SDR video) couldn't render both correctly — either HDR data was lost (Chrome captures sRGB only) or DOM overlays were missing (FFmpeg pass-through skips Chrome). This PR adds in-memory alpha compositing that combines both. ## What it does **Per-frame two-pass capture:** 1. **DOM pass** — Chrome screenshots the page with a transparent background (CDP alpha). HDR videos are hidden, leaving transparent holes where they go. 2. **HDR pass** — Pre-extracted native HLG/PQ frames (16-bit PNG from FFmpeg) are read from disk. 3. **Composite** — DOM pixels (sRGB RGBA8) are alpha-composited over HDR pixels (rgb48le) in Node.js memory, with sRGB→HLG/PQ conversion via a 256-entry lookup table. **Key components:** - `decodePng()` / `decodePngToRgb48le()` — Pure Node.js PNG decoders (no native dependencies). Support all 5 PNG filter types. - `blitRgba8OverRgb48le()` — Alpha composite with per-pixel sRGB→HDR LUT conversion. Fast paths for alpha=0 (skip) and alpha=255 (overwrite). - `initTransparentBackground()` + `captureAlphaPng()` — Split CDP transparent background setup (once) from per-frame screenshot capture (eliminates 2 CDP round-trips per frame). - Single-pass FFmpeg extraction — All HDR frames extracted in one sequential FFmpeg run (avoids duplicate frames from per-frame `-ss` fast seek). ## Key design decisions | Decision | Why | |----------|-----| | In-memory compositing (not FFmpeg overlay) | Eliminates ~2400 process spawns + temp files per render. Pure pixel math is 10x faster. | | 16-bit PNG intermediate | Raw `-f rawvideo` loses color metadata, causing moiré artifacts. PNG is self-describing. | | sRGB→HLG LUT (256 entries) | DOM content is sRGB. Without conversion, it appears orange-shifted in HLG stream. | | Native HDR detection before extraction | `extractAllVideoFrames` converts SDR→HDR. Pre-extraction probe identifies original HDR sources so only truly-HDR videos get native extraction. | ## Files changed | File | What changed | |------|-------------| | `packages/engine/src/utils/alphaBlit.ts` | **NEW** — PNG decode, sRGB→HDR LUT, alpha compositing (14 tests) | | `packages/engine/src/services/screenshotService.ts` | Transparent background CDP, `captureAlphaPng()` | | `packages/engine/src/services/videoFrameInjector.ts` | `hideVideoElements()` / `showVideoElements()` | | `packages/engine/src/services/streamingEncoder.ts` | Input color space tags for rgb48le | | `packages/producer/src/services/renderOrchestrator.ts` | Two-pass HDR capture loop, native HDR detection | ## How to test Render a composition with an HDR video background and text overlays. Both should be visible — HDR video at full quality, text crisp with correct colors (not orange-shifted). ## Stack position **3 of 6** — Stacked on #265 (HDR output pipeline). This is the foundation for all layered compositing that follows. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance Ingalls
|
5a3fde19d4 |
feat(engine): add HDR video output pipeline (#265)
## Summary Adds the ability to render HDR video output (H.265 10-bit, BT.2020) from HyperFrames compositions. When the renderer detects HDR source video, it automatically switches to the HDR output pipeline — no flags needed. ## What it does - **Auto-detection** — Probes each video source with `ffprobe`. If any has bt2020/PQ/HLG color metadata, the output switches to H.265 10-bit with correct color tags. SDR-only compositions are unaffected (H.264, bt709). - **HLG pass-through** — Native HLG pixels from FFmpeg extraction are piped directly to the encoder without conversion. This avoids brightness loss from HLG→linear→PQ conversion (which requires an OOTF system gamma we can't reliably apply). - **Encoder HDR support** — Both chunk and streaming encoders accept HDR presets: `libx265`, `yuv420p10le`, BT.2020 color primaries, `hvc1` codec tag (required for Apple playback). - **WebGPU HDR capture (gated)** — A complete WebGPU float16 readback pipeline is implemented and tested but gated behind headed Chrome (headless doesn't expose WebGPU). Ready for future use with WebGPU canvas content. - **HDR utilities** — `detectTransfer()` (PQ vs HLG), `getHdrEncoderColorParams()`, `analyzeCompositionHdr()`. 15 unit tests. ## Key design decisions | Decision | Why | |----------|-----| | No `--hdr` flag | SDR content encoded as HDR causes orange shift in browsers. Auto-detect eliminates this. | | HLG pass-through (not HLG→PQ) | Conversion loses brightness without OOTF. Pass-through matches source exactly. | | `hvc1` codec tag | Apple QuickTime requires `hvc1` (not `hev1`) for HEVC playback. | | 1-hour streaming timeout | HDR capture at ~6fps needs more time than the default 10-minute FFmpeg timeout. | ## Files changed | File | What changed | |------|-------------| | `packages/engine/src/utils/hdr.ts` | **NEW** — HDR detection, transfer types, encoder params (15 tests) | | `packages/engine/src/services/hdrCapture.ts` | **NEW** — WebGPU readback, HLG conversion, PQ encode | | `packages/engine/src/services/streamingEncoder.ts` | HDR presets, raw rgb48le input, color tags | | `packages/engine/src/services/chunkEncoder.ts` | HDR presets, conditional color tags | | `packages/producer/src/services/renderOrchestrator.ts` | Auto-detection loop, HDR pass-through capture path | ## How to test Render a composition with an HDR video source. The output should be H.265 10-bit with HDR metadata visible in `ffprobe` (bt2020, arib-std-b67 or smpte2084). Plays correctly in QuickTime and on HDR displays. ## Stack position **2 of 6** — Stacked on #258 (SDR/HDR normalization). Provides the encoder infrastructure that phases 1-5 build on. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
|
ad11de698c |
feat(producer): auto-fallback screenshot capture for raf and iframes (#331)
* fix(core): drive adapter seeks when composition has no GSAP timeline renderSeek returned early when deps.getTimeline() was null, skipping the onDeterministicSeek call that drives all frame adapters (CSS, WAAPI, Lottie, Three.js). That meant compositions using any non-GSAP animation primitive froze on their initial frame during capture. Now we still quantize the seek time and fire onDeterministicSeek even without a timeline, so each adapter gets a chance to advance. GSAP compositions are unaffected — timeline-driven seek still takes the same path it did before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(producer): auto-fallback screenshot capture for raf and iframes Co-Authored-By: Codex <codex@openai.com> * test(producer): add render compatibility regression fixtures Co-Authored-By: Codex <codex@openai.com> * fix(core): scrub CSS animations via WAAPI currentTime Co-Authored-By: Codex <codex@openai.com> * test(producer): cover css keyframe renders Co-Authored-By: Codex <codex@openai.com> * fix(producer): propagate virtual time into iframe documents Co-Authored-By: Codex <codex@openai.com> * test(producer): refresh iframe docker golden Co-Authored-By: Codex <codex@openai.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Codex <codex@openai.com> |
||
|
Miguel Ángel
|
e8a48a62d0 |
fix(producer): external assets work on Windows (GH #321) (#324)
* fix(producer): external assets work on Windows (GH #321) Two Unix-only assumptions in the external-asset pipeline caused every absolute path on Windows to be rejected as "unsafe" at render time: 1. Containment checks used `child.startsWith(parent + "/")`. On Windows the separator is `\`, so the predicate is always false unless the paths are equal — every external asset tripped the safety guard in `renderOrchestrator.ts`. The reporter saw: [Render] Skipping external asset with unsafe path: hf-ext/D:\coder\reactGin\hyperframes\reading\assets\segment_001.wav Fix: use `path.relative()` through a shared helper `isPathInside(child, parent)` that normalises separators per-platform and correctly rejects siblings whose names start with the parent (e.g. `/foo/bar-sibling` is NOT inside `/foo/bar`). 2. The external-asset key was built as `"hf-ext/" + absPath.replace(/^\//, "")`. A Windows absolute path (`D:\coder\...`) became `"hf-ext/D:\\coder\\..."` — and because Node's `path.join` treats a drive-letter prefix as absolute, `join(compileDir, key)` silently escaped `compileDir`. Fix: `toExternalAssetKey()` strips the drive colon and normalises to forward slashes, producing `hf-ext/D/coder/...` — a pure relative path that `path.join` cannot promote to absolute on any OS. Both helpers live in `packages/producer/src/utils/paths.ts` and are exercised by 14 unit tests covering Unix paths, Windows drive-letter paths, mixed separators, sibling-prefix confusion, and `..` traversal. Docs: new "External assets" section in `docs/packages/producer.mdx` describes detection, sanitised keys, and the cross-platform containment invariant. Closes #321. * fix(producer): address review on #324 — UNC + integration test Addresses the non-blocking observations from the PR #324 staff review (https://github.com/heygen-com/hyperframes/pull/324#issuecomment): 1. UNC and extended-length Windows paths. `toExternalAssetKey` now handles: - `\\?\D:\very\long\path\clip.mp4` (extended-length) → `hf-ext/D/very/long/path/clip.mp4` - `\\server\share\file.wav` (plain UNC) → `hf-ext/unc/server/share/file.wav` - `\\?\UNC\server\share\file.wav` (extended-length UNC) → `hf-ext/unc/server/share/file.wav` The UNC-collapsed form keeps the server boundary so two different servers exposing the same share/file name cannot collide under one relative key. Previously both edge cases silently produced keys with stray `?` or `:` characters that downstream `isPathInside` rejected — not a security hole, but a silent drop of user assets. 2. Short-circuit on already-sanitised input. `toExternalAssetKey("hf-ext/…")` now returns its input unchanged instead of prepending `hf-ext/` a second time. Makes the helper genuinely idempotent, which is what the unit test claimed all along. Renamed the test accordingly. 3. JSDoc caller contract. `toExternalAssetKey` now documents that it expects canonicalised input (`path.resolve`'d upstream) and does not strip `..` components. `isPathInside` at copy time is still the defensive backstop — called out explicitly in the doc so future callers read the contract before the code. 4. End-to-end integration test. `renderOrchestrator.test.ts` gains two seam tests that run the full external-asset pipeline — build the sanitised key, populate an `externalAssets` map, invoke `writeCompiledArtifacts`, and assert both the success path (the file lands under `<compileDir>/hf-ext/…`) and the escape-rejection path (a malicious `hf-ext/../../etc/passwd` key does NOT materialise above `compileDir`). `writeCompiledArtifacts` is exported for the test seam with a clear JSDoc disclaimer that it's not part of the public API. 22 tests pass across `paths.test.ts` (17) and `renderOrchestrator.test.ts` (5). Out of scope for this follow-up (tracked as follow-ups): - Centralising every `startsWith("/")` absolute-path check into a shared helper across htmlCompiler / audioExtractor / audioMixer / videoFrameExtractor. Mentioned in the review; touches 5 files and deserves its own PR. - Windows CI runner. |
||
|
James RussoandClaude Opus 4.6
|
ebc12f7dc9 |
feat(render): add CRF/bitrate controls and improve default quality (#292)
Raise default encoding quality to visually lossless at 1080p (CRF 18) and expose fine-grained encoding controls for power users. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
5de2af5bde |
feat(skills): improve hyperframes composition quality rules (#250)
## Summary
Overhaul the hyperframes composition skill based on 26 eval rounds (~100 generated compositions). The goal: prevent known AI design tells and composition bugs while giving the LLM maximum creative freedom.
### Typography (`fonts.md` → `typography.md`)
- Two-tier banned font list (32 fonts): tier 1 bans training-data defaults, tier 2 bans the reflex replacements
- Font discovery script: queries Google Fonts API, 5 dynamic categories, top 5 randomized per run
- Selection philosophy: register-first thinking, cross-check assumptions
### Google Fonts on-demand (`deterministicFonts.ts`)
- Any Google Font works without pre-bundling — compiler fetches woff2 at compile time
- Cached to `~/.cache/hyperframes/fonts/<slug>/<weight>-<style>.woff2`
- Parallel woff2 fetches via `Promise.allSettled` (was sequential)
- Single `mkdirSync({ recursive: true })` per family (was `existsSync` x11)
- Skip redundant `readFileSync` when buffer is already in memory from fetch
### Layout rules (`SKILL.md`)
- Flexbox with gap for content text — prevents overlap from absolute positioning
- `position: absolute` reserved for decoratives only
- Cards/containers explicitly banned
### Background layer (`house-style.md`)
- 3-5 persistent decorative elements per scene (glows, ghost text, accent lines)
- All decoratives MUST have ambient GSAP animation — static decoratives banned
- WRONG/RIGHT code examples
### Transition rules (`SKILL.md`)
- Always use transitions, always entrance animations, exit animations banned except final scene
- WRONG/RIGHT code examples showing banned exit patterns
### Other
- Flash cut transition removed
- CLAUDE.md: `bun install` / `bun run build` / `bun run test` (was pnpm)
- house-style.md trimmed from 184 to ~80 lines
- SKILL.md trimmed from 364 to ~230 lines
## Test plan
- [ ] `bun install` succeeds, workspace links resolve
- [ ] `bun run build` succeeds
- [ ] `npx hyperframes lint` passes on existing compositions
- [ ] Generate a composition with `/hyperframes` skill — verify flexbox, background decoratives with animation, entrance-only animations, no banned fonts
- [ ] Verify Google Fonts on-demand: use a non-bundled font, run `npx hyperframes preview`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
James RussoandClaude Opus 4.6
|
9115d7364c |
feat(producer): add request-level render concurrency semaphore (#232)
Add a FIFO semaphore to limit concurrent renders in the producer server, preventing Chrome CPU contention that causes beginFrame failures. - New Semaphore utility class (packages/producer/src/utils/semaphore.ts) - Both blocking render and SSE renderStream handlers acquire/release the semaphore - SSE stream sends a "queued" event when request must wait - New GET /render/queue endpoint exposes active/queued render counts - Configurable via HandlerOptions.maxConcurrentRenders or PRODUCER_MAX_CONCURRENT_RENDERS env var (default: 2) - New --max-concurrent-renders CLI flag (1-10) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
0cf03016b2 |
fix(engine): resolve external asset paths from compiled dir (#231)
## Summary
- Parent-relative paths (e.g. `src="../file.wav"`) silently drop media from rendered MP4
- The compiler rewrites external paths to `hf-ext/` and copies files to the compiled directory, but both the audio mixer and video frame extractor only resolved against `projectDir` — never finding them
- Now checks `compiledDir` first (matching the file server's resolution order), then falls back to `projectDir`
- Fixes both `<audio>` and `<video>` elements with external paths
## Real-world context
Reported in Slack by Abhai — a TTS comparison video using `<audio src="../tts-voxcpm2.wav">` (audio file in parent directory, composition in subdirectory) rendered successfully but the output MP4 had no audio stream. The render completed without any error, silently dropping the audio.
## Testing
### Environment
- Linux (Ubuntu 20.04), ffmpeg 4.2
- Test composition: `subdir/index.html` with `<audio id="bg-audio" src="../test-audio.wav">`, WAV file at parent directory
### Before fix (main)
```
[AUDIO-DEBUG] element.src=hf-ext/tmp/hf-test-231/test-audio.wav
baseDir=/tmp/hf-test-231/subdir
[AUDIO-DEBUG] resolved srcPath=/tmp/hf-test-231/subdir/hf-ext/tmp/hf-test-231/test-audio.wav
exists=false
```
- Audio mixer tries `join(projectDir, "hf-ext/...")` → file doesn't exist at that path
- Output: **9.8 KB, video stream only** (confirmed via ffprobe)
- No error logged — audio silently dropped
### After fix (this branch)
```
[AUDIO-DEBUG] element.src=hf-ext/tmp/hf-test-231/test-audio.wav
baseDir=/tmp/hf-test-231/subdir
compiledDir=/tmp/.../compiled
[AUDIO-DEBUG] fromCompiled=/tmp/.../compiled/hf-ext/tmp/hf-test-231/test-audio.wav
exists=true
[AUDIO-DEBUG] resolved srcPath=/tmp/.../compiled/hf-ext/tmp/hf-test-231/test-audio.wav
exists=true
[AUDIO-RESULT] success=true, hasAudio=true
```
- Audio mixer checks `join(compiledDir, "hf-ext/...")` first → file found
- Output: **44.6 KB, video + audio streams** (confirmed via ffprobe)
### ffprobe comparison
| Branch | File size | Streams |
|--------|-----------|---------|
| `main` | 9.8 KB | `video (h264)` only |
| `fix` | 44.6 KB | `video (h264)` + `audio (aac)` |
### Path resolution flow
1. Compiler sees `<audio src="../test-audio.wav">`
2. Compiler resolves to absolute path, maps it to `hf-ext/tmp/.../test-audio.wav`
3. Compiler copies file to `compiled/hf-ext/tmp/.../test-audio.wav`
4. Audio mixer gets `element.src = "hf-ext/tmp/.../test-audio.wav"`
5. **main**: tries `join(projectDir, src)` → not found → silent drop
6. **fix**: tries `join(compiledDir, src)` first → found → audio mixed in
### Repro
```bash
mkdir -p /tmp/test/subdir
ffmpeg -f lavfi -i "sine=frequency=440:duration=2" /tmp/test/test-audio.wav -y
# Create subdir/index.html with <audio src="../test-audio.wav" ...>
cd /tmp/test/subdir && npx hyperframes render
ffprobe -v error -show_streams output.mp4 # video only on main, video+audio on fix
```
|
||
|
Miguel Ángel
|
43e9252065 |
feat: add MOV (ProRes 4444) as transparent video output format (#224)
## Summary - Adds `--format mov` to the render CLI for ProRes 4444 transparent video output - ProRes 4444 with alpha is the industry standard for transparent video overlays, supported by CapCut, Final Cut, Premiere, DaVinci, and After Effects - WebM VP9 alpha technically works but is ignored by all major video editors — only browsers decode it - Adds MOV to the studio export dropdown alongside MP4 and WebM ## Transparency format comparison | Format | Codec | Alpha | Video editors | Browsers | File size | | --- | --- | --- | --- | --- | --- | | **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No (won't play in browser) | Large (~5-40 MB) | | **WebM** | VP9 | Yes | None (shows black) | Chrome, Firefox | Small (~200 KB) | | **MP4** | H.264 | No | All | All | Small | > **Note:** ProRes MOV files do not play in Chromium browsers — they are an intermediate/editing format, not a delivery format. Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify transparency works correctly. ## Changes - **CLI**: Add `mov` to `--format` validation, examples, and output path logic - **Engine**: `getEncoderPreset()` returns ProRes 4444 (`yuva444p10le`) for `mov` format; handle `.mov` in `applyFaststart` and `muxVideoWithAudio`; add `pix_fmt` to streaming encoder ProRes path - **Producer**: Treat `mov` like `webm` for alpha capture (PNG frames, screenshot mode, `forceScreenshot`) - **Studio**: Add MOV option to export format dropdown and render queue hook - **Core**: Add `mov` to studio API types, render route, and mime helpers - **Tests**: Add encoder preset tests for mov format (42 total, all passing) ## Usage ```bash hyperframes render --format mov --output overlay.mov ``` ## Test plan - [x] `pnpm build` passes - [x] `pnpm --filter @hyperframes/engine test` — 42 tests pass (2 new for MOV) - [x] `oxlint` and `oxfmt` clean on all 12 changed files - [x] End-to-end local render produces ProRes 4444 (`yuva444p12le`) with working alpha - [x] Docker render with `--format mov` — ProRes 4444 confirmed via ffprobe - [x] Studio dropdown shows MOV option in built JS - [x] Transparency verified with [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) |
||
|
Miguel Ángel
|
ef2f646c90 |
fix(producer): extract head assets from non-template sub-comps + fix postcss ESM (#220)
## Summary Two fixes in the producer: 1. **Head styles/scripts extraction**: mirrors the runtime fix from PR #219. The producer's `inlineSubCompositions()` parsed only `bodyEl.innerHTML` from non-template sub-compositions, discarding all `<head>` content. 2. **Externalize postcss**: postcss is a CJS module with `require("path")` — bundling it into ESM output caused "Dynamic require of path is not supported" at runtime, breaking `npx tsx cli render` and `npx tsx cli preview` from the local dev build. ## Verified Re-rendered the iris-wipe composition (eval prompt #25, previously scored 1.0/5 — entirely black): | Frame | Before fix | After fix | | --- | --- | --- | | 0\.5s | Black | Red background + "HELLO" text | | File size | 16\.9 KB (all black) | 64\.8 KB (actual content) | Scene 1 now renders correctly. Scene 2's clip-path animation has a separate GSAP issue (the lint already warns about it via `scene_layer_missing_visibility_kill`). ## Test plan - [x] `pnpm --filter @hyperframes/producer build` succeeds - [x] `node --input-type=module -e "import './dist/index.js'"` loads without error - [x] Re-render iris-wipe produces visible content (64.8 KB vs 16.9 KB) - [x] Frame extraction confirms red "HELLO" scene renders correctly |
||
|
Miguel Ángel
|
baa3d813be |
fix: address QA report P0-P2 issues for 10/10 agent experience (#208)
## Summary
Addresses all 8 issues from the QA report to improve agent and user experience.
### P0 — Must Fix
- **Blank template broken captions**: Removed `compositions/captions.html` and its reference from the blank template. Every agent (10/10) hit 404 errors during render.
- **Inner-wrapper example**: Added a clear structural comment in the blank template showing the correct `class="clip"` + inner wrapper pattern.
- **Sub-composition introspection**: `hyperframes compositions` now reads external HTML files referenced via `data-composition-src` and shows their real duration/element count instead of `0.0s / 0 elements`.
### P1 — Fix Soon
- **Font mapping warnings**: Now lists all mapped fonts, suggests alternatives (use a mapped font, add @font-face, install locally), and links to docs.
- **Browser 404s**: Non-font "Failed to load resource" 404s now prefixed with `[non-blocking]` instead of `[Browser:ERROR]`.
- **Render concurrency**: Default workers increased from `cores/2` (max 4) to `cores*3/4` (max 6). Added `--concurrency` alias.
### P2 — Nice to Have
- **Transform conflict fix suggestion**: `gsap_css_transform_conflict` now suggests exact GSAP property replacements (e.g., `xPercent: -50, yPercent: -50`).
- **Upgrade --yes**: Now actually runs the install instead of just printing the command.
## Before / After
### Font mapping warning
**Before:**
```
[Compiler] No deterministic font mapping for: DM Sans
```
**After:**
```
[Compiler] No deterministic font mapping for: DM Sans
Mapped fonts: arial → inter, courier → jetbrains-mono, ...
To fix, pick one:
1. Use a mapped font name instead (see list above)
2. Add a @font-face block in your HTML with a local or hosted font file
3. Install the font locally on the render machine (Docker: add to Dockerfile)
4. Add an alias to FONT_ALIASES in deterministicFonts.ts (for contributors)
```
### Browser 404s during render
**Before:** `[Browser:ERROR] Failed to load resource: the server responded with a status of 404`
**After:** `[non-blocking] Failed to load resource: the server responded with a status of 404`
### Transform conflict lint
**Before:** `Fix: Remove the transform from CSS and use tl.fromTo...`
**After:** `Fix: Remove transform: translate(-50%, -50%) from CSS and replace with GSAP properties: xPercent: -50, yPercent: -50`
### Compositions command
**Before:**
```
overlay 0.0s 1920×1080 0 elements
```
**After:**
```
overlay 8.0s 1920×1080 2 elements ← compositions/overlay.html
```
## Test plan
- [x] All 422 core tests pass
- [x] TypeScript compiles cleanly (all 4 packages)
- [x] Full monorepo build succeeds
- [x] `hyperframes init --template blank` ships without broken captions reference
|
||
|
Miguel ÁngelandClaude Opus 4.6
|
7294803fbc |
feat(fonts): add Playfair Display, Noto Sans JP, Roboto, and 4 more to deterministic font database (#196)
* fix(engine): suppress font-loading 404 noise in render console output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): downgrade resource 404s to buffer-only instead of suppressing Address review feedback: instead of silently dropping "Failed to load resource" errors (which could hide real asset failures), keep them in browserConsoleBuffer for diagnostics but don't print to stdout. Real asset 404s are still caught by the file server's own logging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): narrow 404 filter to font CDN domains and woff2 files only Address review: filter was too broad and could suppress real asset failures. Now only suppresses 404s matching fonts.googleapis, fonts.gstatic, or .woff2 file extensions. Missing images, scripts, and videos will still surface as [Browser:ERROR] in render output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(fonts): add Playfair Display, Noto Sans JP, Roboto, and 4 more to deterministic font database Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
1efb23dfae |
fix(producer): inline CDN scripts for offline render and detect black video (#182)
## Context `npx hyperframes render` fails to load assets and produces all-black video. Root causes: 1. CDN scripts (GSAP, Lottie) are left as `<script src="https://...">` in compiled HTML — the headless browser must fetch them over the network, which fails in Docker, CI, and firewalled environments 2. Assets referenced from outside the project directory (e.g. `../shared-assets/hero.png`) 404 because the file server only serves from `projectDir` 3. When GSAP fails to load, the timeline never registers, all `.clip` elements stay `visibility: hidden`, and every frame is black — with no diagnostic output 4. The linter reports `missing_gsap_script` when GSAP is bundled inline (no `<script src>` tag), which blocks users from working around the CDN issue ## What changed ### 1\. CDN script inlining (`htmlCompiler.ts`) `compileForRender` now downloads all external `<script src="https://...">` tags at compile time and inlines their content into the HTML. Rendering no longer needs network access. | Before | After | | --- | --- | | `<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>` stays in HTML, browser must fetch at runtime | Script downloaded during compilation, embedded as `<script>/* inlined: https://... */\n...code...</script>` | | Docker/CI render → `net::ERR_NAME_NOT_RESOLVED` → black video | Render works fully offline | | No feedback when CDN fails | `[Compiler] WARNING: Failed to download CDN script: ... Consider bundling it locally` | ### 2\. External asset copying (`htmlCompiler.ts`, `renderOrchestrator.ts`) After compilation, the HTML is scanned for `src`, `href`, and CSS `url()` references that resolve outside `projectDir`. These files are copied into the compiled output directory so the file server can serve them. | Before | After | | --- | --- | | `background-image: url(../shared-assets/hero.png)` → 404 (file server can't serve outside `projectDir`) | Asset detected, copied to compiled dir, path rewritten → serves correctly | | `<img src="../shared-assets/logo.png">` → 404 | Same fix — works for all `src`/`href` attributes and CSS `url()` | ### 3\. Black video diagnostics (`renderOrchestrator.ts`) When composition duration is 0 (which would produce a black video), the error now probes the browser for diagnostics instead of a generic message. | Before | After | | --- | --- | | `Invalid composition duration: 0. Check that GSAP timelines are registered.` | `Composition duration is 0 — this would produce a black video.\n\nDiagnostics:\n - GSAP is not loaded — CDN script may have failed to download. Bundle GSAP locally...\n - Browser: [Browser:PAGEERROR] gsap is not defined` | | Asset 404s during page load silently logged | `[Render] Asset load failure: ...` + `[WARN] Browser encountered network failures during page load` | ### 4\. Linter: recognize inline GSAP (`core/lint/rules/gsap.ts`) The `missing_gsap_script` rule now recognizes GSAP bundled inline — matching the producer's inlining comment (`/* inlined: ...gsap... */`), GSAP library internals (`_gsScope`, `GreenSock`), and large inline scripts (>5KB) referencing gsap. | Before | After | | --- | --- | | User inlines GSAP → linter errors with `missing_gsap_script` | Inline GSAP detected, no false error | | Producer inlines CDN → linter errors on the compiled HTML | Producer's `/* inlined: ... */` comment recognized | ## Test plan - [x] `pnpm build` passes - [x] Core tests pass (410/410, +2 new) - [x] **Reproduced baseline failures on** **`main`**: CDN scripts not inlined, external assets 404, no diagnostics - [x] **Verified fixes**: CDN script inlined, external assets copied and served, diagnostics printed - [x] Render with working CDN → `[Compiler] Inlined CDN script: ...` → render succeeds - [x] Render with broken CDN → `[Compiler] WARNING: Failed to download CDN script` + browser errors surfaced - [x] Render with assets outside project dir → `[Compiler] Found 1 asset(s) outside project directory` → assets served correctly - [x] Linter with inline GSAP → no `missing_gsap_script` false positive |
||
|
Miguel Ángel
|
1681350ac4 |
fix(preview): rewrite sub-composition asset urls in styles (#174)
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting
## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.
## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`
## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
|
||
|
Miguel Ángel
|
110ea12597 |
fix(core,engine,producer): handle id-less media in sub-composition renders (#96)
## What Move the id-less media fix into the shared timing compiler so producer can resolve durations for sub-composition videos before inlining, then carry the merged result through engine parsing, regression coverage, and the regression Docker image used in CI. This PR now does five concrete things: - assigns stable ids to id-less media in core `compileTimingAttrs()` so unresolved duration injection can target them - keeps the engine-side `parseVideoElements()` support for `video[src]` plus the newer `data-duration` / natural-duration fallback from `main` - makes producer prefer sub-composition media metadata over the later inlined-document parse when the same media id appears in both places - makes `sub-composition-video` a runnable regression test by fixing its metadata and checking in the missing `output/compiled.html` snapshot - removes the stale `pnpm-workspace.yaml` copy step from `Dockerfile.test`, so regression CI builds the Bun-based test image from the current workspace layout ## Why - media without an explicit `id` could not participate in unresolved-duration resolution early enough - producer could lose the resolved sub-composition timing by overwriting it with the later inlined parse - the regression fixture intended to cover this case was not actually running in CI because its `meta.json` was incomplete and the required compiled snapshot was missing - the regression image definition still expected a deleted `pnpm-workspace.yaml`, so GitHub Actions failed before the test shard could start Putting the id-generation step in core makes the behavior reusable instead of relying on producer-only HTML patching. ## How ### Shared compiler - core `compileTimingAttrs()` now auto-assigns stable ids to id-less `video` / `audio` tags - those generated ids are returned in `unresolved`, so `injectDurations()` can add `data-duration` and `data-end` to the same media element later in the pipeline - added core tests that cover auto-id assignment and duration injection for generated ids ### Producer - when producer combines `subVideos` / `subAudios` with the media re-parsed from the final inlined HTML, it now lets the sub-composition metadata win - this preserves the resolved/clamped timing already computed for nested media instead of overwriting it with the later parse - `sub-composition-video` now has valid regression metadata and a checked-in `output/compiled.html` snapshot so CI actually executes it ### Engine - resolved the merge conflict in `videoFrameExtractor` by keeping the broader `video[src]` parsing from this branch and the `data-duration` / natural-duration fallback that landed on `main` - added a focused engine unit test for videos without ids ### CI image - `Dockerfile.test` now copies only `package.json` and `bun.lock` at the workspace root before `bun install --frozen-lockfile` - this matches the current monorepo layout and removes the obsolete pnpm-era dependency on `pnpm-workspace.yaml` ## Test plan - [x] `bun run --filter @hyperframes/core test` - [x] `bun run --filter @hyperframes/engine test` - [x] `bun run --filter @hyperframes/producer test --update --sequential sub-composition-video` - [x] `bun run --filter @hyperframes/producer test --sequential sub-composition-video` - [x] Browser check with `agent-browser` against the compiled fixture page (`http://127.0.0.1:8123/compiled.html`) - [x] Clean tracked-only Docker build of `Dockerfile.test` with the PR version of the file applied ## Notes - Latest regression workflow is green on `main`, but before this PR the `sub-composition-video` fixture was being skipped by the harness rather than exercised end to end. - The CI Docker fix was validated from a tracked-only export to avoid local untracked worktree artifacts affecting the result. |
||
|
Miguel Ángel
|
7265d0adfd |
fix(producer): rewrite relative asset paths when inlining sub-compositions (#166)
## Summary - **Fixes**: `<img src="../icon.svg">` and similar `../` relative asset references in sub-compositions resolve to 404 after inlining into the root document - **Unifies**: Both `hyperframes preview` (core bundler) and `hyperframes render` (producer) now use the same shared logic — no duplication ## Root cause When inlining a sub-composition at `compositions/scene.html` into root `index.html`, a relative path like `../icon.svg` is correct from `compositions/` (it points to project root) but after inlining, `../` escapes the project directory. ## Fix Extracts path rewriting into a shared `rewriteSubCompPaths.ts` utility in `@hyperframes/core`, used by both the bundler and the producer. **Only rewrites paths starting with** **`../`** — plain relative paths like `assets/foo.svg` are already correct from the root perspective and must not be rewritten (this was the regression cause in `overlay-montage-prod`: sub-composition asset refs like `assets/notch.svg` were incorrectly being rewritten to `compositions/assets/notch.svg`). ## Regression fix The earlier version of this fix (now in history) rewrote ALL relative paths including `assets/foo.svg`, breaking the `overlay-montage-prod` regression test. This PR fixes that by scoping rewrites to `../`\-prefixed paths only. ## Test plan - [x] `../icon.svg` in sub-composition renders correctly in both preview and render - [x] `assets/foo.svg` (no `../`) in sub-composition still resolves correctly — not rewritten - [x] `overlay-montage-prod` regression test passes - [x] All other regression shards pass |
||
|
Miguel Ángel
|
b43b6fb1dc |
fix(producer): hoist external CDN scripts from sub-compositions (#164)
## Summary
- **Fixes**: External `<script src="...">` tags in sub-compositions (e.g. GSAP TextPlugin, ScrollTrigger) were silently discarded during `hyperframes render`, causing `ReferenceError` at runtime
- **Root cause**: The producer's `inlineSubCompositions` skipped external scripts with `if (src) continue` but then removed them via the blanket `querySelectorAll("style, script").forEach(remove)` — they were never hoisted to the parent document
- **Fix**: Mirror the core bundler's (`htmlBundler.ts`) approach — collect external script `src` URLs, deduplicate against existing scripts in the parent, and inject as `<script>` tags before inline composition scripts
## How to reproduce
1. Create a sub-composition that uses GSAP TextPlugin:
```html
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/TextPlugin.min.js"></script>
<script>
gsap.registerPlugin(TextPlugin);
tl.to("#text", { text: { value: "Hello" }, duration: 1 });
</script>
```
1. Run `hyperframes render`
2. Browser console shows: `[Compiler] Composition script failed scene-hook ReferenceError: TextPlugin is not defined`
## Test plan
- [x] Verify `hyperframes render` on a project using TextPlugin in a sub-composition no longer errors
- [x] Verify external scripts are deduped (same CDN URL in 2 sub-compositions → only 1 `<script>` tag)
- [x] Verify external scripts already in `index.html` are not duplicated
- [x] Verify script order: external CDN scripts load before inline composition scripts
|
||
|
Miguel Ángel
|
5bb74cc0fc |
feat(cli): simplify init flow (#163)
## Summary - **Streamline init from 7 steps to 3**: name → template → auto-opens studio preview - **Remove** interactive "Got a video or audio file?" prompt (`--video`/`--audio` flags still work) - **Remove** "What do you want to do?" post-scaffold menu (auto-launches preview) - **Fast startup**: `--version` exits in ~10ms (lazy-load telemetry + update checker) - **Brand teal**: CLI accent color is now `#3CE6AC` instead of generic cyan ### Before ``` name → overwrite? → media file? → [drag file] → transcribe? → pick template (9) → what next? ``` ### After ``` name → pick template → auto-opens studio ``` ## Test plan - [x] `hyperframes init my-project` — should prompt name + template, then auto-open preview - [x] `hyperframes --version` — should print version instantly (~10ms) - [x] `hyperframes init --video path.mp4 my-project` — video flag still works - [x] `hyperframes init --template blank my-project` — template flag still works - [x] Verify teal accent color in terminal output |
||
|
Miguel Ángel
|
1230657ed0 |
fix(studio,runtime,engine,compiler): 8 bug fixes — audio, render, timeline, Lottie, thumbnails, video render (#133)
## Summary
**Original 5 bugs fixed:**
- **Bug 1 — Audio silent after seek**: Added `Accept-Ranges` / `Content-Length` + `206 Partial Content` to the static asset server for byte-range seeking.
- **Bug 2 — Download 404 after restart**: Render list endpoint now registers on-disk renders into the in-memory job map.
- **Bug 3 — Timeline stops at GSAP end**: `resolveRootTimelineFromDocument` pads the GSAP timeline to match `data-duration` when the composition declares longer.
- **Bug 4 — Render stuck at 0%**: Store `jobState` reference (not spread copy) so async progress mutations reach the SSE stream.
- **Bug 5 — Lottie missing in preview/render**: Two fixes — (a) moved Lottie adapter before GSAP so `onUpdate` wins; (b) fixed bundler silently dropping external CDN `\<script src>` tags from sub-compositions (root cause: `$content(s).html()` returns `""` for external scripts).
**3 additional bugs fixed:**
- **Bug 6 — Blank thumbnails outside monorepo**: Implemented `generateThumbnail` in the CLI adapter using Puppeteer.
- **Bug 7 — Video empty in rendered sub-compositions**: Fixed `parseVideoElements` selector from `video[id][src]` to `video[src][data-start]` + auto-assign IDs.
- **Render errors**: Failed renders now show their error message in the renders panel.
## Commits
| Commit | Description |
| --- | --- |
| `3951c6f` | fix(studio): store render job reference instead of snapshot copy |
| `f331c30` | fix(studio): make previously-completed renders downloadable after restart |
| `a5e2d04` | fix(studio): add range request support for audio/video seeking in preview |
| `f24317a` | fix(runtime): pad GSAP timeline to data-duration when composition declares longer duration |
| `7cf38ca` | fix(runtime): fix Lottie adapter conflicting with GSAP-driven animations |
| `bc99209` | fix(studio): surface render error messages in the renders panel |
| `8fc9e8b` | fix(cli): implement generateThumbnail in studio adapter |
| `90277ea` | fix(engine): render videos inside sub-compositions that lack an explicit id |
| `f5bb579` | fix(compiler): preserve external CDN scripts from sub-compositions in bundle |
## Test plan
- [x] `golden-lyric-video`: seek → audio plays from seeked position
- [x] Any project: render → progress advances past 0%, reaches 100%
- [x] Any project: complete render, restart `hyperframes dev`, Download → works
- [x] `intro-vid`: play → runs full 5s (not stopping at 3s)
- [x] `hyperframe-build-up-demo`: play → rocket Lottie visible during 0-2s ✅ verified
- [x] Outside monorepo: Compositions sidebar shows thumbnail images (not blank)
- [x] `bug.zip` project: render → video in polaroid sub-composition appears in output
- [x] Trigger a failed render → error message shown
|
||
|
Vance Ingalls
|
229538c622 |
fix: add media rendering guardrails to prevent silent failures (#112)
## Summary - **Lint rules** catch media elements missing `id` (renderer silently skips them), missing `src`, `preload="none"` (blocks renderer), and video nested in timed divs (freezes playback). Upgraded `video_nested_in_timed_element` from warning to error. - **Compiler** strips `preload="none"` from media during compilation. Runs parallel, cached keyframe interval analysis via ffprobe — warns on sparse keyframes (>2s) that cause seek failures and audio/video desync. Suggested ffmpeg command preserves audio (`-c:a copy`). - **Pre-render lint** lints `index.html` + all `compositions/*.html` sub-compositions before render via shared `lintProject()` helper. Warns by default; `--strict` blocks on errors, `--strict-all` blocks on errors + warnings. - **Render orchestrator** logs a hint to retry with `--workers 1` when parallel capture times out on video-heavy compositions. - **Refactor**: extracted `runFfprobe()` + `parseProbeJson()` helpers to deduplicate ~80 lines of spawn boilerplate across 3 ffprobe functions. Extracted `shouldBlockRender()` so strict flag tests exercise production code. Shared `lintProject()` used by both `lint` and `render` commands. ## Context Discovered during a real composition build session where: 1. `<audio>` without `id` rendered silently (preview worked fine because runtime queries `[data-start]`, but renderer queries `[id][src]`) 2. `<video>` inside timed `<div>` froze on first frame 3. `preload="none"` caused 45s renderer timeout 4. YouTube clips with sparse keyframes from `yt-dlp --download-sections` caused audio/video desync 5. Parallel workers timed out on video-heavy compositions ## Test plan - [x] Core: 365/365 tests passing (5 new lint tests) - [x] Engine: 24/24 tests passing - [x] CLI: 14/14 tests passing (7 lintProject + 7 shouldBlockRender) - [x] Lint + format hooks pass - [ ] Manual: create a composition with `<audio data-start="0" src="test.wav">` (no id) — verify `npx hyperframes lint` catches it - [ ] Manual: run `npx hyperframes render --strict` with lint errors — verify it blocks - [ ] Manual: run `npx hyperframes render --strict-all` with lint warnings — verify it blocks |
||
|
Miguel Ángel
|
95d7dc0623 |
fix(cli): align render output naming and add WebM support to studioServer (#109)
## Summary - CLI render: use timestamped filenames (`project_date_time.ext`) matching the studio's naming convention, preventing overwrites of previous renders - studioServer: read `fps`/`quality`/`format` from POST body instead of hardcoding `fps:30`/`quality:standard`/`mp4` - studioServer: use timestamped job IDs matching the studio pattern - studioServer: fix download endpoint to serve correct content-type for WebM ## Test plan - [x] `hyperframes render --format webm` outputs timestamped WebM file - [x] `hyperframes render` outputs timestamped MP4 (no overwrite) - [x] Studio embedded server (`hyperframes dev`) renders with correct format when selected in UI - [x] Download endpoint serves correct MIME type for WebM renders |
||
|
James Russo
|
e0ee983e19 |
Merge pull request #81 from heygen-com/feat/webm-transparency
feat(render): WebM output with VP9 alpha transparency |
||
|
JamesandClaude Opus 4.6
|
83d11ac595 |
refactor: simplify review fixes for WebM PR
- Use static import for copyFileSync (was unnecessary dynamic import) - Shallow-copy config before mutating forceScreenshot (prevents caller-provided config from being permanently modified) - Consolidate isWebm/isWebmRender/outputFormat into single early declaration in renderOrchestrator - Fix debug output extension for WebM (was hardcoded .mp4) - Log unexpected audio extraction errors instead of silently swallowing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
dc0d69a21b |
feat(producer): support entryFile for rendering individual compositions (#42)
## Changes
- Add optional `entryFile` parameter to render API endpoints (`/v1/render` and `/v1/render-stream`)
- Enable rendering individual sub-compositions by extracting them from index.html context when the entry file is a `<template>` wrapper
- Change base64 audio/video linting from detecting "fabricated" media to prohibiting all inline base64 media
- Add manifest path resolution for bundled producer deployments
## API Changes
- `RenderConfig` — new optional `entryFile` field for specifying HTML file to render
- `server.ts` — parses `entryFile` from request body, validates file exists in project directory
- `executeRenderJob` — uses `entryFile` instead of hardcoded `"index.html"`
## Template Extraction
- `extractStandaloneEntryFromIndex` — extracts sub-composition hosts from index.html and creates standalone render context
- Handles `<template>` entry files by finding matching `data-composition-src` in index.html and isolating that host
- Resets `data-start` to 0 for standalone rendering
## Linting Updates
- Change rule #3.7 from detecting "fabricated" base64 media to prohibiting all inline base64 audio/video
- Lower detection threshold from 100+ to 20+ base64 characters
- All base64 media now triggers error severity with clearer messaging about file size bloat
## Usage
```json
POST /v1/render-stream
{ "projectDir": "/path/to/project", "entryFile": "compositions/intro.html" }
```
Omit `entryFile` for default behavior (renders `index.html`).
|
||
|
JamesandClaude Opus 4.6
|
efc7b82755 |
fix(render): fix WebM transparency — use screenshot mode + proper CDP setup
- Force screenshot capture mode for WebM (beginFrame doesn't support alpha channel in chrome-headless-shell) - Set Emulation.setDefaultBackgroundColorOverride once during session creation (matching experiment-framework's approach) - Fix acquireBrowser to pass executable path in screenshot mode on Linux (was setting undefined, causing puppeteer-core to fail) - Add Page.captureScreenshot params: fromSurface, captureBeyondViewport, optimizeForSpeed (matching experiment-framework) - Fix regression harness extractMonoPcm16 for videos without audio - Add getEncoderPreset unit tests - Add webm-transparency regression test with golden baseline Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.6
|
a146c9e527 |
test(render): add WebM regression test and getEncoderPreset unit tests
- New regression test `webm-transparency`: minimal transparent composition
rendered to WebM, validates VP9 codec and visual quality at 100
checkpoints against golden baseline
- Extend regression harness: `renderConfig.format` field ("mp4" | "webm"),
format-aware output paths and snapshot filenames
- Fix `extractMonoPcm16` to gracefully handle videos without audio
streams (WebM without audio was throwing instead of returning empty)
- Unit tests for `getEncoderPreset()`: VP9/yuva420p for WebM,
h264/yuv420p for MP4, preset mapping, quality preservation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.6
|
15f6b86ac0 |
feat(render): add WebM output with VP9 alpha transparency
Support rendering compositions with transparent backgrounds via `--format webm`. VP9+alpha is the standard format for overlayable video (captions, lower thirds, overlays). Changes by layer: - CLI: `--format mp4|webm` flag on render command - Producer: threads format through RenderConfig, switches to PNG capture and VP9 encoding when webm - Engine: getEncoderPreset() returns VP9 config with yuva420p; transparent page background via CDP when capturing PNG; mux uses Opus audio for WebM; VP9 flags from production: -row-mt 1, -auto-alt-ref 0, alpha_mode=1 metadata - Frame capture: Emulation.setDefaultBackgroundColorOverride a=0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
9262d01109 |
fix(producer): fall back to data-duration when GSAP timeline is empty (#78)
## What
When a composition has an empty GSAP timeline (no animations), `window.__hf.duration` was always 0, causing `hyperframes render` to time out after 45 seconds waiting for `duration > 0`.
## Why
The HF bridge script reads duration from `window.__player.getDuration()`, which returns the GSAP timeline duration. An empty timeline has duration 0. The render engine waits for `window.__hf.duration > 0` to confirm the runtime is ready — so compositions with no animations would always deadlock.
This hits the `--template blank` scaffold immediately: it generates an empty GSAP timeline and relies solely on `data-duration="10"` for composition timing.
## How
One-line change to the bridge script: when `getDuration()` returns 0, fall back to reading `data-duration` from the root `[data-composition-id]` element. This is the same value the static compiler already extracted — so we get a correct duration without any extra browser round-trips.
```js
get duration() {
var d = p.getDuration();
return d > 0 ? d : getDeclaredDuration(); // reads data-duration from root element
}
```
## Test plan
- [x] `hyperframes init my-video --template blank && hyperframes render my-video` completes successfully
- [x] Output: `output.mp4` — 10s, 1920×1080, 30fps ✓
- [x] Compositions with actual GSAP animations unaffected (fallback only triggers when timeline duration is 0)
- [x] Build passes, lint/format clean
|
||
|
Miguel Ángel
|
e554710646 |
fix(producer): propagate data-start to inner composition element during compilation (#73)
## Summary - copy data-start from the host sub-composition node to the inlined inner composition root - preserve the correct runtime offset lookup for nested compositions after producer compilation - avoid nested GSAP timelines snapping to their end state when the host starts later than t=0 |
||
|
JamesandClaude Opus 4.6
|
e746debb1a |
fix(ci): generate font data in Docker build instead of committing
The generated font data is a pure function of the generator script + @fontsource package versions — no reason to store 566KB of base64 blobs in git. Generate it during the Docker test image build instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.6
|
879a1a1ae5 |
fix(ci): commit generated font data for regression tests
The regression CI runs the producer source directly via tsx (not the bundled CLI), so it needs fontData.generated.ts to exist at import time. Remove from .gitignore and commit the generated file. Mark as linguist-generated in .gitattributes so GitHub collapses it in PR diffs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.6
|
7e059e15f9 |
fix(producer): embed font data at build time instead of runtime require.resolve
The CLI bundle uses tsup to inline @hyperframes/producer, but the
deterministicFonts module used require.resolve('@fontsource/*/package.json')
at runtime to find woff2 files on disk. When installed via npx, these
@fontsource packages don't exist, causing "Cannot find module" errors.
Replace runtime filesystem lookups with a build-time generator that reads
all @fontsource woff2 files and produces a TypeScript module with base64
data URIs. The generator runs before both producer and CLI builds, making
the bundle fully self-contained with zero @fontsource runtime dependencies.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Ángel
|
251ed23252 |
fix(producer): add margin reset when wrapping HTML fragments (#55)
## Summary
- When `index.html` is a bare fragment (just a `<div>` without `<!DOCTYPE>`), `ensureFullDocument()` wraps it in a minimal HTML document
- The wrapper was missing a CSS reset, so Chrome applied its default `body { margin: 8px }` — creating visible white lines at the top and left edges of rendered video
- Added `* { margin:0; padding:0; box-sizing:border-box }` and `body { overflow:hidden; background:#000 }` to the fragment wrapper
## Test plan
- [x] Render a composition that starts with a bare `<div>` (no `<!DOCTYPE html>`)
- [x] Verify no white lines appear at the edges of the rendered MP4
|
||
|
James Russo
|
5d8d112ee8 |
Merge pull request #54 from heygen-com/fix/manifest-path-resolution
fix(producer): resolve manifest from sibling dist/ directory |
||
|
JamesandClaude Opus 4.6
|
45a7a8c986 |
test(producer): add manifest resolution tests
Verifies: - env var override takes priority - sibling path is first candidate (catches the npm package bug) - monorepo-relative fallback works in dev - path construction invariant holds Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.6
|
fbc128ba11 |
fix(producer): resolve manifest from sibling dist/ directory
The build script already copies hyperframe.manifest.json into the producer's dist/ directory, but the runtime loader never checked there. This caused `window.__hf not ready` timeouts when rendering via the published npm package outside the monorepo, since the only candidate paths were monorepo-relative. Add a sibling-directory lookup (same dir as the bundled module) as the first candidate, so the manifest is found whether running from source, a bundled dist/, or an npm install. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel ÁngelandClaude Opus 4.6
|
d3c6228d89 |
fix(producer): preserve @import rules during CSS composition scoping
scopeCssToComposition corrupted @import url() rules because they have
no {} block. The selector regex ([^{}@]+)\{ treated the text after @
as a selector, producing invalid CSS like:
@[data-composition-id="x"] import url('...')
This broke font loading, CSS variable resolution, and all composition
styling in rendered output.
Fix: extract @import rules before running the scoping regex, then
prepend them back unmodified.
Also adds:
- Regression test fixture (css-import-scoping)
- Common-mistakes docs: autoplay/loop, GSAP TextPlugin, sub-composition
positioning
- Format fix for hyperframeLinter.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 4.6
|
61c5257402 |
fix(ci): update publish workflow to use bun install (#36)
* fix(ci): update publish workflow to use bun install pnpm-lock.yaml was removed in the bun migration but publish.yml still referenced it. Use bun for install/build, keep pnpm for publish (publishConfig overrides + --provenance). * docs: update stale pnpm references to bun across docs and scripts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |