ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-05 00:56:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,040 Commits 878 Branches 404 Tags
9983f37c13dc7d8fbd35da21e88229e3500cee9a
Commit Graph
16 Commits
Author SHA1 Message Date
Miguel Ángel 7133c396eb fix(cli): expose render debug mode 
Adds the render --debug CLI flag, forwards it through Docker/local render paths, and captures full producer debug artifacts from pipeline start.
 2026-06-23 13:04:48 -04:00
Miguel Ángel 8408a44745 fix(engine): tune VP9 cpu-used across render paths (#1614) 
* fix(engine): tune VP9 cpu-used across render paths

* fix: address VP9 review feedback
 2026-06-20 16:36:59 -04:00
James RussoClaude Opus 4.8Xuelong Mu
 		36b24acf20 feat: add video frame format render option (#1481) 
* feat: add video frame format render option

* refactor: single source of truth for video-frame-format allow-list

Addresses PR review (Via) on #1481: the ["auto","jpg","png"] set was
declared three times — render.ts (VIDEO_FRAME_FORMATS), server.ts
(inline includes), and renderConfigValidation.ts
(ALLOWED_VIDEO_FRAME_FORMATS) — three boundaries to update when a new
extraction format lands.

Hoist the constant + a reusable `isVideoFrameFormat` type guard into
@hyperframes/engine (where VideoFrameFormat is defined) and route all
three call sites through them. Behavior unchanged; also drops two
`as RenderConfig[...]` casts in favor of the guard (narrowing over
assertion, per repo TS conventions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Xuelong Mu <xuelongmu@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-15 22:05:20 -07:00
Matt Van HornandMatt Van Horn e6b8d66c2d feat(cli,producer): add gif output format with two-pass palette encode (#1333) 
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
 2026-06-10 21:17:55 -04:00
James RussoandClaude Opus 4.7 6affe2d212 fix(cli): reject directory --composition and add --browser-timeout (#1199) (#1200) 
* fix(cli): reject directory --composition and add --browser-timeout (#1199)

Two unrelated symptoms from issue #1199, fixed together:

1. `--composition .` (or any directory path) used to slip past the
   existsSync check in render.ts and explode downstream as
   `EISDIR: illegal operation on a directory, read` when the producer
   readFileSync'd the entry. The CLI now treats `.` / `""` as "omit
   the flag" (falls back to index.html) and rejects other directory
   paths with an actionable error pointing at the .html shape.

2. The 60s Puppeteer page.goto timeout in frameCapture.ts was hard-
   coded, so heavy compositions (many videos / fonts / asset requests)
   could not complete `domcontentloaded` in time. Add a configurable
   `pageNavigationTimeout` to EngineConfig (default 60_000, env
   fallback PRODUCER_PAGE_NAVIGATION_TIMEOUT_MS) and expose it as
   `--browser-timeout <seconds>` on `hyperframes render`. The flag
   threads through both renderLocal (via resolveConfig) and the
   docker bridge (via buildDockerRunArgs).

Tests:
- render.test.ts: forwards/omits pageNavigationTimeout into resolveConfig
- dockerRunArgs.test.ts: forwards/omits --browser-timeout (seconds)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): address PR #1200 review — extract validators, tighten bounds

Addresses Vai's blockers and Miguel's nits on PR #1200:

- Vai blocker 1 (fallow CRAP) + blocker 3 (no argv tests):
  Extract --browser-timeout and --composition validators into pure
  helpers in utils/renderArgs.ts with a structured-result discriminant.
  Drops ~45 lines of inline validation from run(), reducing its CRAP
  score 1290→978 and cyclomatic 75→65. 19 new unit tests cover the
  parse branches (sub-ms, overflow, NaN, Infinity, empty, negative,
  ".", "./", whitespace, directory, missing, ../escape, sibling-prefix).

- Vai blocker 2 (sub-ms → timeout:0 = "no timeout"): reject inputs
  that round to <1 ms. Puppeteer treats page.goto({timeout:0}) as
  wait-forever, so --browser-timeout 0.0004 silently flipped the
  semantics. Now rejected with an explicit "rounds to 0 ms" error.

- Vai important 5 (1e10 accepted → setTimeout overflow): cap at
  86_400s (24h). Above Node's TIMEOUT_MAX ≈ 2^31-1 ms setTimeout
  fires immediately, the opposite of "long timeout."

- Vai important 4 (related timeouts unmentioned): CLI help and docs
  now flag PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS and the 45s
  playerReadyTimeout as the other knobs heavy compositions may need.

- Vai nit 7 (s/ms unit mismatch): help text and docs row both call
  out the SECONDS-vs-MILLISECONDS difference between flag and env.

- Vai nit 8 / Miguel nit (composition flag discoverability): the
  --composition description now says "Pass `.` (or omit the flag)
  to render the project's index.html."

- Miguel nit (dead branch): the entryFile === "" unreachable branch
  is gone. New helper uses `if (!trimmed || trimmed === ".")`.

Also adds a trailing-separator guard on the project-containment check
(sibling-prefix bypass: /proj-evil/x.html no longer slips past
startsWith('/proj')) — flagged by the code review.

The three remaining fallow complexity findings on render.ts (run,
renderDocker, trackRenderMetrics) are inherited from main; this PR
reduces run() but does not refactor it. Suppressed with
fallow-ignore-next-line markers and inline rationale.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): diverge --browser-timeout error messages per Vai nit 5

The `not-a-number` and `not-positive` branches in browserTimeoutErrorMessage
shared the generic "Must be a positive number of seconds" message even
though the discriminant carried distinct kinds. Diverge them so users see
the specific failure mode:

  --browser-timeout abc   →  "Got \"abc\", which is not a number."
  --browser-timeout -5    →  "Got \"-5\" seconds, which is not positive."

The shared hint ("pass a positive number of seconds, e.g. 180") is
preserved on both branches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-04 16:47:10 -04:00
James Russo 2be41937a9 fix(cli): support arm64 hosts for --docker render (#1196) 
* fix(cli): support arm64 hosts for `--docker` render

The Docker render path pinned `--platform linux/amd64` for both build
and run, which on Apple Silicon / Graviton forced qemu emulation of
chrome-headless-shell. The emulated chrome process either SEGV'd or
hung on page navigation, producing the failures reported in #1193 /
#1194 / #1195.

Derive the platform from `process.arch` instead. On arm64 hosts:

- The image builds natively (no qemu).
- The Dockerfile skips the chrome-headless-shell install because
  Chrome for Testing only publishes a `linux64` build (verified
  against the known-good-versions manifest).
- The wrapper script leaves `PRODUCER_HEADLESS_SHELL_PATH` unset
  when no headless-shell binary is present, so the engine falls
  back to the system chromium that the Dockerfile already
  installs from apt and points at via `PUPPETEER_EXECUTABLE_PATH`.

`TARGETARCH` is forwarded as an explicit `--build-arg` instead of
relying on BuildKit's automatic platform args — the legacy
builder (and some BuildKit configs, including colima on macOS)
leaves it unset, which would silently bypass the arch conditional
in the Dockerfile.

Image tags are now suffixed with `-arm64` on arm64 hosts so amd64
and arm64 images of the same hyperframes version can coexist in
the local cache.

The arm64 path renders correctly but loses byte-for-byte parity
with amd64 (system chromium uses screenshot capture, not
HeadlessExperimental.beginFrame). The CLI prints a one-line
warning so users comparing against amd64 baselines know.

Verified on macOS 26.5 / M4 Max:

- Before: `qemu: unknown option 'type=gpu-process'` followed by a
  chrome-headless-shell SIGSEGV after ~4 minutes.
- After: 300/300 frames captured in ~18s of render time (1m18s
  wallclock including a one-time image build), MP4 produced.

Closes #1193
Closes #1194
Closes #1195

* fix(cli): address review feedback on docker arm64 fix

Follow-up to 61880cdc. Addresses one substantive review comment from
@vanceingalls and three self-review gaps.

1. Restore loud build failure on amd64 when chrome-headless-shell is
   missing (per @vanceingalls). The original Dockerfile used an `&&`
   chain that crashed the build if `find` returned empty; the new
   `if/else` wrapper silently fell through to system chromium even on
   amd64, which would mask golden-baseline regressions from a future
   @puppeteer/browsers cache layout change. The else branch now checks
   `TARGETARCH = amd64` and exits 1 with an actionable error, while
   arm64 still falls through to the system-chromium wrapper cleanly.

2. Add `HYPERFRAMES_DOCKER_PLATFORM` env override. The fix derives
   platform from `process.arch`, which silently picks the wrong arch
   in three real-world cases: x64 Node under Rosetta on Apple Silicon
   (re-triggers issue #1193), parity-regen for amd64 golden baselines
   on an arm64 host, and DOCKER_HOST pointing at a remote daemon with
   a different arch. Empty/whitespace override is a no-op (falls back
   to arch detection) so `export FOO=""` doesn't pin platform to "".

3. Fail fast when `--gpu` is requested on arm64. Docker Desktop on
   Apple Silicon doesn't implement `--gpus` passthrough; the previous
   code would crash at `docker run` with an opaque device-driver
   error. We now short-circuit with errorBox pointing at the env
   override as the workaround.

4. Close the test gap on the default-arch resolution. Every previous
   test passed `arch` explicitly; a refactor that dropped the
   `= process.arch` default would pass all tests but break every arm64
   host at runtime. Added one assertion that calls
   `resolveDockerPlatform()` with no args, plus coverage for the env
   override.

The new arm64 platform-checking logic is extracted into
`resolveDockerHostPlatform()` so `renderDocker` itself stays focused
on the build/run wiring (and below the fallow complexity gate).

Test plan:
- `bunx vitest run packages/cli/src/utils/dockerRunArgs.test.ts` — 31 passed (was 27).
- `bunx vitest run packages/cli` — 647 passed (was 643).
- E2E on macOS 26.5 / M4 Max: deleted the cached arm64 image, ran
  `--docker --quality draft --workers 1` against the blank scaffold —
  300/300 frames in 1m1s wallclock, MP4 produced.
 2026-06-04 03:13:44 -04:00
Vance IngallsClaude Opus 4.6Vai
 		f84cc492de perf(engine): faster shader transitions via page-side WebGL compositing (#832) 
* fix(cli): prefer puppeteer cache + numeric version sort (staff review)

Two correctness fixes from PR #821 self-review:

1. Cache priority order. Previous order was hyperframes-managed cache →
   puppeteer cache. HF cache is pinned to CHROME_VERSION (131-era) which
   lags 17+ releases behind upstream; if a user separately installed a
   newer chrome-headless-shell via @puppeteer/browsers install, the CLI
   would silently hand engine the older HF-cache binary while engine's
   own resolveHeadlessShellPath would have picked the newer one. Flip
   the priority so puppeteer cache wins, matching engine semantics.

2. Numeric (not lexicographic) version sort. `readdirSync.sort().reverse()`
   over names like `linux-148.0.7778.97` and `linux-99.0.6533.123` would
   return `linux-99...` first because character '9' outranks '1'. Parse
   each name into integer segments and compare them numerically.

Tests: add both-caches-populated and linux-148-beats-linux-99 cases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(engine): page-side compositing for shader transitions (opt-in spike)

Add an opt-in `--page-side-compositing` flag (CLI) backed by a new engine
config field `enablePageSideCompositing` and env var `HF_PAGE_SIDE_COMPOSITING`.
When set, SDR shader-transition compositions skip the Node-side layered blend
(the hf#677 chain) and instead run the shader inside Chrome via a page-side
WebGL canvas; the engine then captures ONE opaque RGB frame per output frame
via the existing streaming capture path.

This is the strongest non-beginFrame perf lever for Mac users, who cannot
take the beginFrame `~5×` path (Chromium structural limit, crbug.com/40656275).
Stacks on top of the hf#677 1.95× baseline.

Default OFF — existing fixture pins (byte-exact MP4 output) are preserved.
Opt-in path is intentionally PSNR-pinned, not byte-equal (WebGL is f32; Node
is f64). HDR content forces the existing layered path regardless.

Implementation:
- engine: new `EngineConfig.enablePageSideCompositing` (default false).
- producer/fileServer: new `HF_PAGE_SIDE_COMPOSITING_STUB` early-page script
  injected into the served HTML head when the flag is on.
- producer/renderOrchestrator: when the flag + no HDR + no png-sequence,
  route SDR transitions through the streaming path instead of the layered
  HDR stage.
- shader-transitions: new `engineModePageComposite.ts` installs a fullscreen
  WebGL compositor overlay and wraps `window.__hf.seek` so each seek inside
  a transition window captures both scenes via the Chromium
  `drawElementImage` API to GL textures, runs the fragment shader, and
  displays the composited result on the overlay canvas. The engine takes
  one screenshot per frame and sees the composited overlay.
- cli: new `--page-side-compositing` flag sets `HF_PAGE_SIDE_COMPOSITING=true`
  before producer load.
- scripts/page-side-compositing-smoke: bundled-CLI smoke that renders a
  representative fixture with and without the flag, validates the canary
  strings are in the shipped bundles, and writes a wall-time pair.

Determinism trade documented in the engine config doc-comment. The smoke
script enforces the bundled-CLI validation discipline from prior perf work
(see internal feedback note `validate_bundled_cli_not_dev_path`).

Runtime requirement: Chromium's `CanvasDrawElement` feature (already
enabled by the engine's `--enable-features=CanvasDrawElement` launch flag).
When the runtime feature is unavailable, the page-side installer logs a
warning and falls back to opacity-flip mode — the engine still takes the
streaming path; the transition window degrades to a hard scene swap. Vance
will validate on Mac Chrome where the feature is supported.

Co-Authored-By: Vai <vai@heygen.com>

* fix(shader-transitions): use html2canvas for page-side compositor capture

The original drawElementImage approach fails in engine render mode because
the virtual-time shim prevents Chromium from generating paint records for
cloned elements. drawElementImage requires a cached paint record from the
browser's compositor — clones created at capture time never receive one
because (a) shimmed rAFs deadlock inside the seek wrapper, (b) original
rAFs don't produce real paints under virtual-time control, and
(c) layoutsubtree canvases don't apply CSS stylesheet rules to children.

Switch scene capture to html2canvas (foreignObjectRendering: false), the
same JS-based renderer already used by the preview-mode fallback path in
capture.ts. html2canvas reads computed styles and renders via its own
canvas drawing pipeline with no dependency on the browser paint cycle.

Also fixes:
- Engine seek must return the result so Puppeteer awaits async seek
  promises (frameCapture.ts).
- GSAP opacity cache: compositor must restore scene opacity before seek,
  not after — GSAP caches inline values and skips re-writes.
- Support check gates on WebGL availability, not drawElementImage.

Perf: 15-scene shader-perf fixture (28s, 14 transitions, 30fps)
  Baseline (Node-side layered): 137s
  Page-side (html2canvas+WebGL): 33s → 4.1× speedup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(shader-transitions): simplify review fixes for page-side compositor

- Use uploadTexture (zeroes canvas backing store after upload) to prevent
  ~2.2GB transient memory pressure across 280 html2canvas calls per render
- Add ignoreElements + stabilizeTransformedBoxShadows to html2canvas call,
  matching the preview-path capture.ts behavior
- Parallelize from/to scene captures with Promise.all
- Wrap post-capture render in try/finally so opacity is always restored
- Fix WebGL context leak in isPageSideCompositingSupported probe
- Remove dead ResolvedTransition.index field
- Export stabilizeTransformedBoxShadows from capture.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(producer): unify page-side compositing gating and Docker forwarding

Addresses three issues from staff review:

1. ignoreElements filter stripped all in-scene canvases (Chart.js, D3,
   p5.js) — narrowed to data-no-capture only since the compositor canvas
   is a body sibling never in the scene subtree.

2. Docker mode silently dropped --page-side-compositing — thread
   pageSideCompositing through DockerRenderOptions/buildDockerRunArgs
   with regression tests.

3. Fragmented gating across 4 independent sites could disagree:
   - Stub injection gated only on cfg flag (leaked into HDR/alpha)
   - Probe-created fileServer never got the stub
   - needsAlpha (WebM/MOV) not excluded from the gate
   - WebGL-unavailable fallback claimed layered path would run but
     orchestrator had already disabled it

   Fix: compute stub injection at the same site as the layered-bypass
   decision (after hasHdrContent is known), using addPreHeadScript on
   the already-running fileServer. Single predicate now gates both
   decisions, including !needsAlpha.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(engine): two-phase drawElementImage capture for page-side compositing

Replace html2canvas with native drawElementImage for scene capture in
the page-side compositor. drawElementImage reads from the browser's own
paint cache, giving pixel-identical output to the preview path.

The blocker was that cloned elements inside layoutsubtree canvases have
no cached paint record under virtual time — the compositor only paints
when explicitly triggered. Fix: split the seek+composite into two phases
with an engine-forced paint between them.

Phase 1 (seek wrapper, page-side):
  - GSAP seek positions the timeline
  - Clone FROM/TO scenes into visible layoutsubtree staging canvases
  - Set window.__hf_page_composite_pending flag

Engine paint force (frameCapture.ts):
  - Detect pending flag after seek returns
  - Fire micro Page.captureScreenshot (1x1 clip) via CDP to force the
    browser compositor to paint all visible elements including staging
    canvas children

Phase 2 (page.evaluate, page-side):
  - drawElementImage reads the now-valid paint records
  - Upload textures to WebGL, run shader, show GL overlay

Key insight: staging canvases must be visible (not opacity:0) for the
browser to paint their children. They sit at z-index:-9998, behind
the main DOM and covered by the GL overlay during transitions.

Perf: 15-scene fixture (28s, 14 transitions, 30fps):
  Baseline (Node-side layered): 137s
  html2canvas + WebGL:           33s (3.7×)
  drawElementImage + WebGL:      21s (6.6×)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(engine): optimize two-phase compositor hot path

- uploadTextureSource instead of uploadTexture: eliminates ~2.3GB of
  canvas buffer alloc/dealloc churn (persistent staging canvases don't
  need the one-shot zeroing behavior)
- Fold hasPending check into seek page.evaluate: eliminates one CDP
  round-trip per frame (~700 unnecessary IPC calls on non-transition
  frames)
- Fix renderShader error handling: on failure, leave source scenes
  visible as fallback instead of hiding both scenes + GL overlay
  (which produced black frames)
- Move mutable state declarations above resolveComposite to prevent
  TDZ risk on refactor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): staff review — staging cleanup, pending flag, beginFrame guard

- Clear staging canvas children when leaving transition window (prevents
  visible clone bleed-through on transparent compositions)
- Clear __hf_page_composite_pending on all resolveComposite exit paths
- Guard micro-screenshot paint force against beginFrame mode (CDP
  Page.captureScreenshot conflicts with beginFrame compositor control)
- Update CLI flag description: document video/canvas limitation, remove
  stale PSNR claim

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): default-on page-side compositing for SDR shader transitions

Page-side compositing is now enabled by default for SDR shader-transition
renders without video content. The 6.6× speedup applies automatically —
no flag needed.

Auto-disables when:
- HDR content detected
- Alpha output (WebM/MOV/PNG-sequence)
- Composition contains <video> elements (cloneNode loses playback state)
- beginFrame capture mode (Linux headless)

Use --no-page-side-compositing to force the Node-side layered path.

Changes:
- Engine config: enablePageSideCompositing defaults to true
- CLI: flag default flipped to true; --no-page-side-compositing disables
- Orchestrator: added composition.videos.length === 0 gate
- Docker: forwards --no-page-side-compositing when explicitly disabled
- Config tests updated for new default

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): support video elements on page-side compositing fast path

Three-phase capture protocol lets shader transitions render video scenes
without falling back to the slow Node-side layered pipeline:

1. Seek → compositor records transition metadata, sets pending flag
2. onBeforeCapture → video frame injector updates <img> replacements
3. prepare → cloneNode picks up current video frames, img.decode() awaits
4. micro-screenshot → forces browser to paint cloned elements
5. resolve → drawElementImage reads paint records, shader composites

Key changes:
- Remove `composition.videos.length === 0` gate from orchestrator
- Split compositor resolve into prepare (clone) + resolve (shader)
- Move onBeforeCapture before compositor prepare in frameCapture.ts
- Await img.decode() on cloned data-URI images to prevent stale frames
- Stop manipulating scene opacity in compositor (GL canvas overlay suffices)
- Add gsap.set declaration for shader-transitions ambient types
- Add video_missing_timing_attrs lint rule for <video> without id/data-start/data-end

Performance: compositions with video now render at 7.5s (6 workers) instead
of 2m38s on the layered path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(core): auto-inject data-start on video/audio so frame extraction works without explicit attrs

The timing compiler now injects data-start="0" on <video> and <audio>
elements that lack it. This makes discoverMediaFromBrowser() find the
element (it queries video[data-start]), so the frame extraction pipeline
activates automatically. Videos "just work" without requiring authors to
add data-start, data-end, or id attributes.

Also removes the video_missing_timing_attrs lint rule — the compiler
handles the missing attributes automatically, so the lint rule would
only false-positive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(core): add data-hf-auto-start sentinel on auto-injected video timing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(producer): add discoverVideoVisibilityFromTimeline for runtime video discovery

Seeks the GSAP timeline in Puppeteer to discover when each video's parent
scene is visible (opacity > 0). Uses coarse sampling at 100ms steps followed
by binary search refinement to frame-level precision (1/60s). Only processes
videos with the data-hf-auto-start sentinel so author-specified timing is
never overridden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(producer): integrate runtime video visibility discovery into probe stage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(producer): trigger browser probe for auto-start videos, remove debug logging

The probe stage was skipping browser launch when composition duration was
already known, which meant discoverVideoVisibilityFromTimeline never ran.
Now needsBrowser also checks for data-hf-auto-start sentinel in compiled HTML.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): use mkdtempSync for smoke test work directory

Replaces hardcoded /tmp/hf-page-side-smoke with a unique temp directory
via mkdtempSync to resolve CodeQL "insecure temporary file" alert.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format smoke test script

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Vai <vai@heygen.com>
 2026-05-15 16:23:12 -07:00
Theodor Kleynhans 5dcc89c930 feat(cli): accept ffmpeg-style rational fps (NTSC, PAL, slow-mo) 
Replaces the rigid `--fps 24|30|60` whitelist with a numeric range and
adds support for ffmpeg-style fractional framerates so NTSC stays exact
end-to-end.

- `--fps 30` keeps working (integer fps)
- `--fps 30000/1001` now means exact NTSC 29.97 (not the lossy decimal)
- `--fps 24000/1001`, `--fps 60000/1001`, `--fps 25/50/120/240` all work
- Decimals like `--fps 29.97` are rejected with a friendly error pointing
  the user at the rational form, since `29.97` and `30000/1001` round
  to different framerates inside ffmpeg

Carries an `Fps = { num: number; den: number }` rational end-to-end:
RenderConfig, EncoderOptions, StreamingEncoderOptions, CaptureOptions,
DockerRenderOptions, Studio API request body, regression-harness
meta.json. The `-r` and `-framerate` ffmpeg args emit the rational form
verbatim (`30000/1001`) so no decimal round-trip happens at the encoder
boundary. Frame-interval math uses `1000 * den / num` ms (33.366… for
NTSC, 33.333… for integer 30).

Helpers live in @hyperframes/core:
- `parseFps(input: string | number): FpsParseResult` — discriminated
  parser used by both the CLI and the Studio API route
- `fpsToFfmpegArg(fps: Fps): string` — emits "30" or "30000/1001"
- `fpsToNumber(fps: Fps): number` — for arithmetic (telemetry, frame
  count, frame-index → time)

Studio API wire format accepts polymorphic `fps: number | string`:
- number → integer fps (`30`)
- string → rational (`"30000/1001"`)
Decimals are rejected; matches the same rule as the CLI.

Existing meta.json fixtures with integer `"fps": 30` continue to load
unchanged — the regression-harness validator now normalizes both number
and string inputs through `parseFps`.
 2026-05-09 00:09:15 +02:00
James e07aeba213 feat(cli): add --resolution flag to hyperframes render for one-line 4k 2026-05-07 16:58:25 +00:00
Theodor Kleynhans 4e28658173 feat(cli): expose png-sequence format 
The producer already supports `format: "png-sequence"` end-to-end (see
RenderConfig in renderOrchestrator.ts), but the CLI's VALID_FORMAT
validator rejects it before the flag reaches the producer. Surface it
the same way `mov` and `webm` are surfaced.

Behaviour:
- `--format png-sequence` accepted alongside mp4/webm/mov.
- Auto-output path uses no extension (FORMAT_EXT["png-sequence"] = "")
  since the producer treats outputPath as a directory of frame_NNNNNN.png.
- `printRenderComplete` sums the contained file sizes when outputPath
  is a directory, instead of reporting the platform-dependent inode
  size.
- DockerRenderOptions.format type extended; existing buildDockerRunArgs
  is unchanged because it forwards the string verbatim.

Tests:
- renderLocal forwards `format: "png-sequence"` to createRenderJob.
- buildDockerRunArgs propagates `--format png-sequence` to the
  container.

Docs:
- Rendering guide: format flag table, format comparison table, new
  "PNG sequence (no encoding)" section, "How it works" extended.
- CLI package docs: format flag table updated.
 2026-05-07 01:56:06 +02:00
Miguel Ángel 0e0a0e40d0 feat(cli): add --composition flag to render specific compositions (#631) 
* feat(cli): add --composition flag to render specific compositions

Expose the existing entryFile config in the producer through
a new --composition / -c CLI flag. This lets users render
individual composition files without restructuring their project:

  hyperframes render -c compositions/intro.html -o intro.mp4

The flag validates the file exists before starting the render,
threads through both local and Docker render paths, and is
documented in the CLI help, examples, and docs.

* fix(cli): address PR review — path traversal guard, forward tests, tripwire

- Add path-containment check mirroring hyperframeLint.ts: reject
  --composition paths that escape the project directory
- Normalize leading ./ from composition paths for clean render plan output
- Improve error message: suggest .html file path instead of compositions command
- Add description note about <template> sub-composition constraint
- Add render.test.ts: entryFile forwarded to createRenderJob (forward + omit)
- Update dockerRunArgs tripwire test with entryFile coverage
 2026-05-07 01:17:39 +02:00
JamesandClaude Opus 4.7 c0d75a5268 feat(core,cli,engine,producer): add getVariables() helper and --variables render flag 
Adds the parametrized-render primitive from hf#592 by reusing the existing
data-composition-variables schema as the source of declared defaults.

- Runtime helper window.__hyperframes.getVariables() (also exported from
  @hyperframes/core) reads data-composition-variables defaults from the
  document root and merges window.__hfVariables (CLI override) on top.
  Returns Partial<T> for typed access; supports a generic for editor
  ergonomics. Same code path runs in dev preview and at render time.
- CLI render --variables '<json>' / --variables-file <path> populates the
  override. Mutually exclusive; fail-fast on conflicting flags, missing
  file, unparseable JSON, or non-object payloads. parseVariablesArg is
  exported as a pure function so validation paths stay unit-testable.
- Engine injects window.__hfVariables via evaluateOnNewDocument before
  any page script runs, so the helper sees the merged values on its
  first call. Empty payloads are skipped to avoid pointless init scripts.
- Producer threads variables through RenderConfig and into the engine's
  CaptureOptions; Docker mode forwards --variables to the in-container
  CLI invocation via dockerRunArgs.

Composition authors declare variables once on the root <html> element:

  <html data-composition-variables='[
    {"id":"title","type":"string","label":"Title","default":"Hello"}
  ]'>

and read them in any composition script:

  const { title } = window.__hyperframes.getVariables();

A render with `--variables '{"title":"Q4 Report"}'` overrides the default
without modifying the composition source. Missing keys fall through to
the declared defaults, so dev preview and CLI renders without --variables
behave identically.

This is PR 1 of a 4-PR stack. Sub-comp per-instance scoping (carrying
host data-variable-values through the inlined sub-comp's getVariables()
call) lands in PR 2; schema validation and lint in PR 3; skill / scaffold
distribution in PR 4.

Tests: 9 new unit tests for getVariables() (jsdom), 11 new CLI tests
covering parseVariablesArg validation paths and Docker passthrough,
2 new dockerRunArgs assertions for the --variables flag. All existing
tests green (core 611, cli 208, engine 519).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 00:18:13 +00:00
Miguel Ángel 395fb9c084 feat: add browser GPU render mode (#571) 
## Problem

HyperFrames already had `--gpu`, but that flag only controlled FFmpeg hardware encoding. The browser capture path still forced Chrome/WebGL through SwiftShader software GL via `--use-angle=swiftshader`, so WebGL-heavy local renders could leave the biggest bottleneck on the CPU path.

That made the existing flag naming easy to misread: `--gpu` sounded like it accelerated the whole render, but it did not change the browser frame-capture backend.

## What this fixes

- Enables host browser GPU acceleration automatically for local CLI renders.
- Adds `--no-browser-gpu` as the local opt-out for software Chrome/WebGL capture.
- Keeps `--browser-gpu` as an explicit local browser-GPU request.
- Adds `browserGpuMode: "software" | "hardware"` to engine config, with `PRODUCER_BROWSER_GPU_MODE` env support for lower-level producer users.
- Keeps Docker browser capture on the deterministic software path.
- Maps hardware browser GPU mode to platform-native Chrome backends:
  - macOS: Metal-backed ANGLE
  - Windows: D3D11-backed ANGLE
  - Linux: EGL
- Blocks explicit `--browser-gpu --docker` with a clear error because Docker browser GPU passthrough is not cross-platform.
- Clarifies docs so `--gpu` means FFmpeg encoder GPU and browser GPU means Chrome/WebGL capture GPU.
- Keeps encoder backend selection auto-detected from FFmpeg capabilities:
  - NVIDIA: NVENC
  - macOS: VideoToolbox
  - Linux: VAAPI
  - Intel: QSV

## Why two flags

There are two separate GPU surfaces in the render pipeline:

1. Browser GPU controls Chrome frame capture.
   - Affects WebGL, canvas, CSS rendering, compositing, and screenshot capture inside the browser.
   - This is enabled automatically for local CLI renders.
   - Use `--no-browser-gpu` when you want the software browser baseline.

2. `--gpu` controls FFmpeg video encoding.
   - Affects the final encode step after frames have already been captured.
   - The concrete encoder is auto-detected from the host FFmpeg build and hardware.
   - It can be faster for some machines/codecs, but it is not equivalent to browser rendering acceleration.

The controls stay independent because users may want:

- `hyperframes render` for the fast local default with browser GPU capture.
- `hyperframes render --no-browser-gpu` for the software-browser local baseline.
- `hyperframes render --gpu` for browser GPU capture plus hardware FFmpeg encoding.
- `hyperframes render --no-browser-gpu --gpu` for software browser capture plus hardware FFmpeg encoding.
- `hyperframes render --docker` for deterministic browser capture.

## Why `--gpu` does not imply browser GPU

Keeping `--gpu` scoped to FFmpeg encoding avoids a semantic break and keeps the risk profile explicit:

- `--gpu` already means encoder acceleration. Expanding it to also change Chrome capture would silently alter behavior for users who only wanted hardware encoding.
- Browser GPU and encoder GPU have different portability. Encoder GPU can work in Docker when the host exposes the right devices; browser GPU passthrough is not cross-platform, so this PR intentionally blocks explicit `--browser-gpu --docker`.
- The Apple presentation benchmark shows why the controls should stay separate: browser GPU capture was the useful improvement, while macOS VideoToolbox via `--gpu` was slower and produced larger output for this `standard` H.264 run.

If HyperFrames later wants a single umbrella acceleration control, it should be explicit, for example `--acceleration browser|encoder|all` or `--gpu=browser|encoder|all`, rather than changing the meaning of the existing boolean `--gpu`.

## Root cause

`buildChromeArgs()` always injected `--use-gl=angle --use-angle=swiftshader`. `disableGpu` only appended `--disable-gpu`; it did not provide a hardware-GPU mode. That made the public `--gpu` flag look broader than it was, because render capture stayed software-backed even when encoder GPU was requested.

## Verification

### Local checks

- `bun install`
- `bun run build:hyperframes-runtime`
- `bun run --filter @hyperframes/engine test src/config.test.ts src/services/browserManager.test.ts`
- `bun run --filter @hyperframes/cli test src/utils/dockerRunArgs.test.ts src/commands/render.test.ts`
- `bun run --filter @hyperframes/cli typecheck`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run --filter @hyperframes/producer typecheck`
- `cd packages/producer && bunx vitest run src/services/renderOrchestrator.test.ts`
- `bunx oxlint packages/cli/src/commands/render.ts packages/cli/src/commands/render.test.ts packages/cli/src/utils/dockerRunArgs.ts packages/cli/src/utils/dockerRunArgs.test.ts packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/browserManager.ts packages/engine/src/services/browserManager.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt --check ...` on changed source/docs files
- `git diff --check`
- `bun packages/cli/src/cli.ts render --help | rg -n "browser-gpu|no-browser-gpu|GPU"`
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --output /tmp/hf-auto-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict`
  - Render plan prints `GPU: browser GPU (auto)`.
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --no-browser-gpu --output /tmp/hf-software-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict`
  - Render plan does not print browser GPU.
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --docker --browser-gpu --output /tmp/should-not-render.mp4`
  - Exits 1 with `Browser GPU is local-only`.
- `buildDockerRunArgs()` regression coverage asserts Docker container args include `--no-browser-gpu`, preventing nested container renders from re-enabling browser GPU through the local CLI default.
- `resolveBrowserGpuForCli()` regression coverage asserts `PRODUCER_BROWSER_GPU_MODE=software` opts out when no CLI browser-GPU flag is supplied, while explicit `--browser-gpu` / `--no-browser-gpu` still win.
- `ffmpeg -v error -i /tmp/hf-auto-browser-gpu-smoke.mp4 -f null -`
- `ffmpeg -v error -i /tmp/hf-software-browser-gpu-smoke.mp4 -f null -`
- `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-browser-gpu-smoke.mp4` -> H.264, 1920x1080, 24fps, 5.0s

### Apple presentation benchmark

Rendered `/Users/miguel07code/Downloads/apple-presentation.zip` as supplied after extracting to `/tmp/hf-apple-profile/apple-presentation`.

Fixed settings:

- 1920x1080
- 30fps
- `standard` quality
- 4240 frames
- 141.32s duration
- 8-worker cap; render auto-calibration used 6 capture workers
- macOS host detected FFmpeg GPU encoder: `videotoolbox`

| Mode | Equivalent flags after this PR | Wall time | vs software-browser baseline | Speed | Capture | Encode | Output |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Software browser + CPU encode | `--no-browser-gpu` | 120.77s | baseline | 1.17x | 97.87s | 10.04s | 8.38MB |
| Browser GPU + CPU encode | default local render | 70.10s | 42.0% faster | 2.02x | 50.72s | 9.91s | 8.39MB |
| Software browser + encoder GPU | `--no-browser-gpu --gpu` | 133.16s | 10.3% slower | 1.06x | 103.58s | 18.31s | 25.43MB |
| Browser GPU + encoder GPU | `--gpu` | 74.12s | 38.6% faster | 1.91x | 46.69s | 17.93s | 25.45MB |

Result: browser GPU capture is the meaningful improvement for this WebGL/browser-capture-heavy presentation. VideoToolbox encoding was slower and produced larger files for this current `standard` H.264 path, so `--gpu` should stay separate and opt-in.

Why `--gpu` plus browser GPU was slower than browser GPU alone: the combined run captured about 4.0s faster than browser GPU alone, but VideoToolbox encoding was about 8.0s slower than CPU x264 encoding, so the encode loss outweighed the capture gain.

### VideoToolbox flag check

I also isolated the encode stage against the already-captured Apple frames to check whether macOS GPU encoding only needed special flags.

`ffmpeg -h encoder=h264_videotoolbox` does not expose a CRF/CQ-style quality option like x264. It exposes bitrate-oriented and VideoToolbox-specific options such as `-b:v`, `-realtime`, `-profile`, `-coder`, `-prio_speed`, `-power_efficient`, and `-allow_sw`. That means our current `-q:v` mapping is not equivalent to x264 CRF and can produce very different bitrate/size behavior.

Measured full-frame encode variants on this host:

| VideoToolbox variant | Encode wall time | Output size | Bitrate |
| --- | ---: | ---: | ---: |
| Current `-q:v 64 -allow_sw 1` | 18.76s | 25.31MB | 1.43 Mbps |
| Current without `-allow_sw 1` | 18.21s | 25.31MB | 1.43 Mbps |
| `-b:v 500k -maxrate 750k -bufsize 1000k -profile high -coder cabac -realtime 1 -prio_speed 1 -power_efficient 0` | 20.58s | 7.42MB | 0.42 Mbps |
| Same with `-b:v 1500k` | 20.84s | 16.70MB | 0.95 Mbps |
| `-b:v 500k -profile baseline -coder cavlc -realtime 1 -prio_speed 1 -power_efficient 0` | 18.11s | 8.94MB | 0.51 Mbps |

Conclusion: VideoToolbox can be made size/bitrate-predictable with explicit `--video-bitrate`, but the tested speed-oriented flags did not make it faster than CPU x264 wall time for this render. That reinforces keeping `--gpu` encoder acceleration explicit and separate from browser GPU capture.

Artifacts from the local benchmark:

- `/tmp/hf-apple-profile/results/cpu.mp4`
- `/tmp/hf-apple-profile/results/browser-gpu.mp4`
- `/tmp/hf-apple-profile/results/encoder-gpu.mp4`
- `/tmp/hf-apple-profile/results/full-gpu.mp4`
- `/tmp/hf-apple-profile/results/summary.json`

All four benchmark MP4s completed `ffprobe` and full `ffmpeg -f null` decode checks.

### Pixel comparison

Compared decoded MP4 output between software-browser and browser-GPU renders:

- Apple presentation:
  - 4240 frames compared
  - 636 exact matching decoded frame hashes
  - 3604 different decoded frame hashes
  - Average PSNR: 57.79 dB
- `css-spinner-render-compat` clean fixture:
  - 120 frames compared
  - 0 exact matching decoded frame hashes
  - Average PSNR: 61.57 dB

Interpretation: browser GPU output is not strict hash/pixel-identical to the software-browser path after lossy H.264 encode, but the measured deltas are visually tiny. Above 50 dB PSNR is typically visually indistinguishable for normal video review. Use `--no-browser-gpu` or Docker when strict cross-run/cross-machine reproducibility matters more than local speed.

### Browser verification

- Started HyperFrames Studio preview for `packages/producer/tests/css-spinner-render-compat/src`.
- Used `agent-browser` to open `http://localhost:5191#project/src` and verify the composition loaded in Studio.
- Screenshots:
  - `/tmp/hf-gpu-browser-proof/preview-loaded.png`
  - `/tmp/hf-gpu-browser-proof/preview-playing.png`
  - `/tmp/hf-gpu-browser-proof/preview-frame-60.png`
- Agent-browser recordings:
  - `/tmp/hf-gpu-browser-proof/preview-playback.webm`
  - `/tmp/hf-gpu-browser-proof/preview-seek.webm`

## Notes

- Browser GPU is enabled automatically for local CLI renders and disabled in Docker.
- `--no-browser-gpu` is the opt-out for software Chrome/WebGL capture.
- `--gpu` remains encoder-only and opt-in.
- The Apple presentation zip has existing lint errors around unmanaged nested videos and imperative media `play()` calls. The benchmark still compares the same supplied source across modes, but it should not be treated as a clean deterministic-composition fixture.
 2026-04-30 06:46:14 +02:00
Vance IngallsandClaude Opus 4.6 8e5593b6ba feat(render): auto-detect HDR from media probes, add --sdr flag (#526) 
* feat(render): auto-detect HDR from media probes, add --sdr flag

Replace the --hdr opt-in model with automatic detection. When no flags
are passed, the renderer probes all video/image sources and enables HDR
output if any HDR color space is detected. Existing --hdr flag becomes
a force override. New --sdr flag forces SDR output.

Behavior matrix:
  (no flags) + HDR content → HDR output
  (no flags) + SDR content → SDR output
  --hdr → force HDR (defaults to HLG if no HDR sources)
  --sdr → force SDR (skips probing)
  --hdr --sdr → error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: align HDR auto-detect docs and tests

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-28 01:39:51 -07:00
Miguel Ángel 36b3fc8cd9 fix: budget workers for expensive captures 2026-04-27 22:53:27 -04:00
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346) 
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
 2026-04-20 12:16:24 -07:00