ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-04 07:19:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,234 Commits 870 Branches 404 Tags
229e88eac58725223eb83087135fcc5d56407b62
Commit Graph
24 Commits
Author SHA1 Message Date
ukimsanov a85a507cdc fix(shader-transitions): page-side render dropped shader transitions and final-scene content 
Page-side compositing (default on) silently dropped HyperShader.init shader
transitions in the engine render. The compositor clones the from/to scenes to
feed drawElementImage, but cloneNode copies the GSAP opacity-fade, and Chrome
won't paint hidden elements, so drawElementImage throws "No cached paint record"
and the shader degrades to a hard cut. Force the clones visible before capture,
as the html2canvas path already does via forceSceneVisibleInClone.

Also fixes the final scene's content dropping in the last beat: the core clip
runtime hides it shortly before the composition ends, and page-side screenshots
the live page (the layered path survives via forceVisible per-scene capture).
Un-hide the settled scene on non-transition frames.

Page-side only; retains the ~6.5x page-side speedup.
 2026-06-20 21:19:55 -07:00
Miguel Ángel ae717331ce fix(studio): soft-reload GSAP property edits, preserve shader cache (#1129) 
* fix(studio): soft-reload GSAP property edits without iframe reload

GSAP property value edits (opacity, x, scale, etc.) now update the live
timeline inside the preview iframe without triggering a full iframe
reload. This preserves the WebGL context and shader transition cache,
eliminating the loading overlay that appeared on every property edit.

Implementation:
- New gsapSoftReload.ts: kills the old GSAP timeline, re-executes the
  updated script, calls __hfForceTimelineRebind(), and re-seeks to the
  current time. Falls back to full reload on failure.
- useGsapScriptCommits: passes softReload: true for property value edits
  via the existing (previously unused) softReload flag on commitMutation.
- hyper-shader.ts: exposes __hfSuppressSceneMutations on the window so
  the soft-reload can suppress the MutationObserver during re-execution.
- hyper-shader.ts: getDocumentScriptSignature now excludes pure GSAP
  animation scripts from the cache key hash, so full reloads (undo,
  external changes) don't invalidate transition caches when only
  animation values changed.

* fix(studio): wrap soft-reload script in IIFE to avoid const redeclaration

The new script ran in the same global scope as the old one, causing
Identifier tl has already been declared errors from const/let
re-declarations. Wrapping in an IIFE creates a new lexical scope.
Also remove the old script element before inserting the new one.

* fix(studio): return scriptText from mutation API, drop client-side HTML parsing

The mutation API already has the extracted GSAP script text (newScript)
after rewriting. Return it as scriptText in the response so
applySoftReload receives the script directly instead of parsing HTML
client-side. This avoids DOMParser compatibility issues across test
environments and is more reliable than regex-based extraction.

* fix(studio): align soft-reload script heuristic with server-side parser

The client's findGsapScriptElement only matched gsap.timeline and
__timelines. The server's extractGsapScriptBlock also matches .to( and
.set(. Aligned the client heuristic to prevent silent fallback to full
reload for compositions that use tl.to() without gsap.timeline in the
same script.

* fix(studio): address hf#1129 review — multi-script guard, scope docs

- Return false (fallback to full reload) when multiple GSAP scripts
  exist in the document, since it's ambiguous which one to replace
- Add docstring scoping the optimization to root-document scripts
  (template-wrapped sub-compositions fall back to full reload)
- Add code comment explaining the IIFE scope constraint
- Add test for the multi-script guard

* fix(studio): align cache key filter with soft-reload script heuristic

isGsapAnimationOnlyScript now also matches .to( and .set( patterns,
matching findGsapScriptElement. Scripts using only tl.to() without
gsap.timeline were excluded from soft-reload but still busted the
shader cache on full-reload paths (undo, external changes).
 2026-05-29 23:34:57 -04:00
ukimsanovandCursor f9d22df3c9 fix(shader-transitions): real opacity crossfade for CSS transitions in engine mode 
Address Copilot round-3 review: the previous engine-mode timeline used
`tl.set(toId, opacity:1, T)` + `tl.set(fromId, opacity:0, T+dur)` for
every transition. That keeps BOTH scenes at opacity:1 throughout the
transition window. The Node-side layered compositor handles this fine —
it captures each scene separately, masks opacity per layer, and runs the
blend itself — but the page-side compositing path (one opaque RGB
screenshot per frame, opt-in via EngineConfig.enablePageSideCompositing)
relies on the page to produce a correct frame. With `shader === undefined`
the page-side compositor skips the entry, so the screenshot would show
both scenes stacked at 100% opacity (visible ghosting) instead of a blend.

Fix: schedule an actual opacity-crossfade tween in `initEngineMode`
when `t.shader === undefined`. Shader transitions keep the existing
opacity-flip pattern because the Node-side compositor needs both scenes
fully visible to capture them. The crossfade is harmless in the layered
Node path because `applyDomLayerMask` overrides per-scene opacity during
each capture anyway.

Also corrects docstrings in engineModePageComposite.ts and at the
installPageSideCompositor call site that previously claimed the GSAP
timeline "handles the blend" — it now actually does.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-19 18:47:47 -07:00
ukimsanovandCursor 351c7bfcc4 fix(shader-transitions): address Copilot round-2 review 
Three follow-up fixes from the Copilot review on commit 8cad2173:

1. Use strict `t.shader === undefined` instead of `!t.shader` (Copilot c4)
   in both the WebGL program compile loop and the page-side compositor.
   An empty-string `shader: ""` from a vanilla-JS caller (the IIFE bundle
   is hand-loaded via <script> tags in user HTML) should reach the shader
   registry and surface a loud "unknown shader" error, not silently
   degrade to a crossfade.

2. Graceful degradation when shader compile fails (Copilot c5). The
   previous `continue` dropped the transition from `cachedTransitions`,
   which also dropped its scene-visibility timeline entries and broke
   scene progression. Now: log a warning and downgrade to the CSS
   crossfade fallback (prog=null, fallback=true) so the opacity timeline
   still runs and the composition keeps playing.

3. Preserve index-to-scene-pair correlation when calling the page-side
   compositor (Copilot c6). The earlier filter `transitions.filter(t =>
   !!t.shader)` shifted indices, so a shader transition at original index
   2 (sitting between CSS crossfades) would be paired with scenes[1] and
   scenes[2] inside `installPageSideCompositor` instead of the correct
   scenes[2] and scenes[3]. The compositor now accepts the full array,
   makes `PageCompositeTransitionConfig.shader` optional, and skips
   CSS-only entries internally while keeping `transitions[i]` aligned
   with `scenes[i]`/`scenes[i+1]`.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-19 18:31:58 -07:00
ukimsanovandCursor 8cad2173dc fix(shader-transitions,producer): harden CSS-only transition lifecycle and unblock CI 
Three follow-on fixes after the optional-shader change rebased onto current
main (PR #832 introduced page-side compositing and the producer's hf#732
layered pipeline since this PR was opened).

shader-transitions/hyper-shader.ts
- Treat `cache.prog === null` as the canonical immutable marker for
  CSS-only transitions via a new `isCssOnlyTransition()` helper.
- `disposeCachedTransition()` now restores the always-ready CSS fallback
  state for prog=null caches instead of zeroing `fallback`/`ready` — the
  previous behaviour, combined with `markScenesDirty()` re-running the
  prewarm/capture pipeline, could put a CSS-only cache through the WebGL
  path and reach `renderShader(state.prog!)` with a null prog (Copilot
  review on lines 1168 + 1319).
- `markScenesDirty()` skips CSS-only caches; they have no shader to
  recompile and no texture pyramid to recapture.
- `ensureTransitionCachesReady()` filters CSS-only caches out of the
  prewarm work list.
- `tickShader()` now routes on `cache.fallback || cache.prog === null`
  and threads a narrowed non-null `prog` local into `renderShader()`,
  removing the unsound `state.prog!` non-null assertion.
- `initEngineMode()` filters CSS-only transitions before passing them to
  `installPageSideCompositor()`, which expects `shader: ShaderName`
  (required). Page-side compositing is shader-only; CSS crossfades stay
  on the GSAP opacity timeline.

producer/render/stages/captureHdrHybridLoop.ts
producer/render/stages/captureHdrSequentialLoop.ts
- Guard `activeTransition.shader` against undefined: when omitted, route
  the Node-side blend through `crossfade` (the engine's canonical
  opacity blend, equivalent to `applyFallbackTransition()` on the page).
- The hybrid path also bypasses the worker pool when `shaderName` is
  absent and runs `crossfade` inline.

This addresses the Copilot review comments and unblocks the 5 failing CI
jobs (Build, Typecheck, CLI smoke, Windows tests, Windows render) which
all rooted in 4 TS errors at these exact sites.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-19 18:17:16 -07:00
ukimsanov a78e49c181 feat(shader-transitions): make shader optional to support CSS crossfade mixing 
Allow omitting the shader field in TransitionConfig to get a smooth CSS
opacity crossfade instead of a WebGL effect. HyperShader manages all scene
visibility regardless of transition type, so shader and CSS crossfade
transitions can now be mixed freely in the same composition.

When shader is omitted:
- No WebGL program is compiled or cached for that transition
- The existing applyFallbackTransition() path handles the crossfade
- No texture prewarming needed — transition is marked ready immediately

Tested: verified with a 3-scene composition (sdf-iris + CSS crossfade)
rendered to MP4. Both transition types render correctly.

engine/src/types.ts: HfTransitionMeta.shader is now optional to match
 2026-05-19 16:11:24 -07:00
James Russo 2729ee5087 refactor: delete orphan declarations flagged by fallow (#949) 
* ci: run fallow audit in lefthook pre-commit

Mirrors the same `fallow audit --base ... --fail-on-issues` check that
runs in CI, but locally against HEAD so issues surface at commit time
instead of after the push round-trip.

Scoped to `packages/**` source files via the glob — non-code edits
(README, docs, top-level configs) skip the hook entirely.

Measured locally: ~5s in parallel with the existing lint/format/typecheck
checks. Doesn't extend wall-clock time because typecheck (~11s) is the
long pole, and lefthook runs commands in parallel.

The default `--gate new-only` means inherited findings don't block the
commit — same gate behavior as CI, so local pre-commit and PR audit
agree.

* refactor: delete orphan declarations flagged by fallow

After fallow's auto-fix de-exports unused symbols, oxlint surfaces them
as no-unused-vars. This PR deletes those orphan declarations outright.

Biggest cleanup: studio/src/icons/SystemIcons.tsx shrinks from 132 to 57
lines — 33 unused icon wrappers and their phosphor-icon imports deleted.

Other deletions across 14 more files covering paired getter/setters,
helper functions, dead env constants, internal components with no
callers, and cascading unused imports.

Cascade-causing files held back for follow-up PRs: renderOrchestrator
barrel of captureCost re-exports, telemetry/portUtils/remote barrels,
Button.tsx + ui/index.ts (would orphan whole file), studioMotion
type re-exports.

Test plan: typecheck clean across 8 packages, oxlint + oxfmt clean,
fallow audit exit 0 (remaining findings inherited), cli + studio
vitest suites pass.
 2026-05-18 21:11:03 -07:00
James 7e0a447325 refactor: drop unused exports detected by fallow auto-fix 
Run `fallow fix --auto-fixable` to remove `export` keywords from symbols
fallow's reachability analysis identifies as unused. Keeps only the cases
where the symbol is still referenced internally in its own file (so
removing `export` doesn't surface a new oxlint `no-unused-vars` error).

Result: fallow dead-code findings drop from 276 → 208 (68 fewer unused
exports), with no behavior change — each symbol is still defined and used
exactly the same way within its file.

Reverted ~20 files where fallow's auto-fix would have created cascading
"declared but never used" lint errors — those are cases where the symbol
isn't used at all, and properly cleaning them up means deleting the
declaration, not just dropping `export`. Better to land that as a
separate, narrower PR rather than mixing it into a mechanical de-export.

Also reverted four false positives where fallow missed real consumers:
- `captureCost.ts` (renderOrchestrator has two separate import blocks
  from the same module; fallow only saw the first)
- `propertyPanelHelpers.ts`, `domEditingLayers.ts` (real internal uses
  fallow's reachability missed)
- `render.ts` (functions imported via `await import()` dynamic import,
  which fallow's static analysis doesn't follow)

Test plan: bun run --filter '*' typecheck (clean), oxlint + oxfmt clean,
cli/core/studio/engine vitest suites pass (335 + 917 + 576 + 605 tests).
 2026-05-19 00:51:56 +00:00
Vance IngallsClaude Opus 4.6Vai
 		f84cc492de perf(engine): faster shader transitions via page-side WebGL compositing (#832) 
* fix(cli): prefer puppeteer cache + numeric version sort (staff review)

Two correctness fixes from PR #821 self-review:

1. Cache priority order. Previous order was hyperframes-managed cache →
   puppeteer cache. HF cache is pinned to CHROME_VERSION (131-era) which
   lags 17+ releases behind upstream; if a user separately installed a
   newer chrome-headless-shell via @puppeteer/browsers install, the CLI
   would silently hand engine the older HF-cache binary while engine's
   own resolveHeadlessShellPath would have picked the newer one. Flip
   the priority so puppeteer cache wins, matching engine semantics.

2. Numeric (not lexicographic) version sort. `readdirSync.sort().reverse()`
   over names like `linux-148.0.7778.97` and `linux-99.0.6533.123` would
   return `linux-99...` first because character '9' outranks '1'. Parse
   each name into integer segments and compare them numerically.

Tests: add both-caches-populated and linux-148-beats-linux-99 cases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(engine): page-side compositing for shader transitions (opt-in spike)

Add an opt-in `--page-side-compositing` flag (CLI) backed by a new engine
config field `enablePageSideCompositing` and env var `HF_PAGE_SIDE_COMPOSITING`.
When set, SDR shader-transition compositions skip the Node-side layered blend
(the hf#677 chain) and instead run the shader inside Chrome via a page-side
WebGL canvas; the engine then captures ONE opaque RGB frame per output frame
via the existing streaming capture path.

This is the strongest non-beginFrame perf lever for Mac users, who cannot
take the beginFrame `~5×` path (Chromium structural limit, crbug.com/40656275).
Stacks on top of the hf#677 1.95× baseline.

Default OFF — existing fixture pins (byte-exact MP4 output) are preserved.
Opt-in path is intentionally PSNR-pinned, not byte-equal (WebGL is f32; Node
is f64). HDR content forces the existing layered path regardless.

Implementation:
- engine: new `EngineConfig.enablePageSideCompositing` (default false).
- producer/fileServer: new `HF_PAGE_SIDE_COMPOSITING_STUB` early-page script
  injected into the served HTML head when the flag is on.
- producer/renderOrchestrator: when the flag + no HDR + no png-sequence,
  route SDR transitions through the streaming path instead of the layered
  HDR stage.
- shader-transitions: new `engineModePageComposite.ts` installs a fullscreen
  WebGL compositor overlay and wraps `window.__hf.seek` so each seek inside
  a transition window captures both scenes via the Chromium
  `drawElementImage` API to GL textures, runs the fragment shader, and
  displays the composited result on the overlay canvas. The engine takes
  one screenshot per frame and sees the composited overlay.
- cli: new `--page-side-compositing` flag sets `HF_PAGE_SIDE_COMPOSITING=true`
  before producer load.
- scripts/page-side-compositing-smoke: bundled-CLI smoke that renders a
  representative fixture with and without the flag, validates the canary
  strings are in the shipped bundles, and writes a wall-time pair.

Determinism trade documented in the engine config doc-comment. The smoke
script enforces the bundled-CLI validation discipline from prior perf work
(see internal feedback note `validate_bundled_cli_not_dev_path`).

Runtime requirement: Chromium's `CanvasDrawElement` feature (already
enabled by the engine's `--enable-features=CanvasDrawElement` launch flag).
When the runtime feature is unavailable, the page-side installer logs a
warning and falls back to opacity-flip mode — the engine still takes the
streaming path; the transition window degrades to a hard scene swap. Vance
will validate on Mac Chrome where the feature is supported.

Co-Authored-By: Vai <vai@heygen.com>

* fix(shader-transitions): use html2canvas for page-side compositor capture

The original drawElementImage approach fails in engine render mode because
the virtual-time shim prevents Chromium from generating paint records for
cloned elements. drawElementImage requires a cached paint record from the
browser's compositor — clones created at capture time never receive one
because (a) shimmed rAFs deadlock inside the seek wrapper, (b) original
rAFs don't produce real paints under virtual-time control, and
(c) layoutsubtree canvases don't apply CSS stylesheet rules to children.

Switch scene capture to html2canvas (foreignObjectRendering: false), the
same JS-based renderer already used by the preview-mode fallback path in
capture.ts. html2canvas reads computed styles and renders via its own
canvas drawing pipeline with no dependency on the browser paint cycle.

Also fixes:
- Engine seek must return the result so Puppeteer awaits async seek
  promises (frameCapture.ts).
- GSAP opacity cache: compositor must restore scene opacity before seek,
  not after — GSAP caches inline values and skips re-writes.
- Support check gates on WebGL availability, not drawElementImage.

Perf: 15-scene shader-perf fixture (28s, 14 transitions, 30fps)
  Baseline (Node-side layered): 137s
  Page-side (html2canvas+WebGL): 33s → 4.1× speedup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(shader-transitions): simplify review fixes for page-side compositor

- Use uploadTexture (zeroes canvas backing store after upload) to prevent
  ~2.2GB transient memory pressure across 280 html2canvas calls per render
- Add ignoreElements + stabilizeTransformedBoxShadows to html2canvas call,
  matching the preview-path capture.ts behavior
- Parallelize from/to scene captures with Promise.all
- Wrap post-capture render in try/finally so opacity is always restored
- Fix WebGL context leak in isPageSideCompositingSupported probe
- Remove dead ResolvedTransition.index field
- Export stabilizeTransformedBoxShadows from capture.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(producer): unify page-side compositing gating and Docker forwarding

Addresses three issues from staff review:

1. ignoreElements filter stripped all in-scene canvases (Chart.js, D3,
   p5.js) — narrowed to data-no-capture only since the compositor canvas
   is a body sibling never in the scene subtree.

2. Docker mode silently dropped --page-side-compositing — thread
   pageSideCompositing through DockerRenderOptions/buildDockerRunArgs
   with regression tests.

3. Fragmented gating across 4 independent sites could disagree:
   - Stub injection gated only on cfg flag (leaked into HDR/alpha)
   - Probe-created fileServer never got the stub
   - needsAlpha (WebM/MOV) not excluded from the gate
   - WebGL-unavailable fallback claimed layered path would run but
     orchestrator had already disabled it

   Fix: compute stub injection at the same site as the layered-bypass
   decision (after hasHdrContent is known), using addPreHeadScript on
   the already-running fileServer. Single predicate now gates both
   decisions, including !needsAlpha.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(engine): two-phase drawElementImage capture for page-side compositing

Replace html2canvas with native drawElementImage for scene capture in
the page-side compositor. drawElementImage reads from the browser's own
paint cache, giving pixel-identical output to the preview path.

The blocker was that cloned elements inside layoutsubtree canvases have
no cached paint record under virtual time — the compositor only paints
when explicitly triggered. Fix: split the seek+composite into two phases
with an engine-forced paint between them.

Phase 1 (seek wrapper, page-side):
  - GSAP seek positions the timeline
  - Clone FROM/TO scenes into visible layoutsubtree staging canvases
  - Set window.__hf_page_composite_pending flag

Engine paint force (frameCapture.ts):
  - Detect pending flag after seek returns
  - Fire micro Page.captureScreenshot (1x1 clip) via CDP to force the
    browser compositor to paint all visible elements including staging
    canvas children

Phase 2 (page.evaluate, page-side):
  - drawElementImage reads the now-valid paint records
  - Upload textures to WebGL, run shader, show GL overlay

Key insight: staging canvases must be visible (not opacity:0) for the
browser to paint their children. They sit at z-index:-9998, behind
the main DOM and covered by the GL overlay during transitions.

Perf: 15-scene fixture (28s, 14 transitions, 30fps):
  Baseline (Node-side layered): 137s
  html2canvas + WebGL:           33s (3.7×)
  drawElementImage + WebGL:      21s (6.6×)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(engine): optimize two-phase compositor hot path

- uploadTextureSource instead of uploadTexture: eliminates ~2.3GB of
  canvas buffer alloc/dealloc churn (persistent staging canvases don't
  need the one-shot zeroing behavior)
- Fold hasPending check into seek page.evaluate: eliminates one CDP
  round-trip per frame (~700 unnecessary IPC calls on non-transition
  frames)
- Fix renderShader error handling: on failure, leave source scenes
  visible as fallback instead of hiding both scenes + GL overlay
  (which produced black frames)
- Move mutable state declarations above resolveComposite to prevent
  TDZ risk on refactor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): staff review — staging cleanup, pending flag, beginFrame guard

- Clear staging canvas children when leaving transition window (prevents
  visible clone bleed-through on transparent compositions)
- Clear __hf_page_composite_pending on all resolveComposite exit paths
- Guard micro-screenshot paint force against beginFrame mode (CDP
  Page.captureScreenshot conflicts with beginFrame compositor control)
- Update CLI flag description: document video/canvas limitation, remove
  stale PSNR claim

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): default-on page-side compositing for SDR shader transitions

Page-side compositing is now enabled by default for SDR shader-transition
renders without video content. The 6.6× speedup applies automatically —
no flag needed.

Auto-disables when:
- HDR content detected
- Alpha output (WebM/MOV/PNG-sequence)
- Composition contains <video> elements (cloneNode loses playback state)
- beginFrame capture mode (Linux headless)

Use --no-page-side-compositing to force the Node-side layered path.

Changes:
- Engine config: enablePageSideCompositing defaults to true
- CLI: flag default flipped to true; --no-page-side-compositing disables
- Orchestrator: added composition.videos.length === 0 gate
- Docker: forwards --no-page-side-compositing when explicitly disabled
- Config tests updated for new default

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): support video elements on page-side compositing fast path

Three-phase capture protocol lets shader transitions render video scenes
without falling back to the slow Node-side layered pipeline:

1. Seek → compositor records transition metadata, sets pending flag
2. onBeforeCapture → video frame injector updates <img> replacements
3. prepare → cloneNode picks up current video frames, img.decode() awaits
4. micro-screenshot → forces browser to paint cloned elements
5. resolve → drawElementImage reads paint records, shader composites

Key changes:
- Remove `composition.videos.length === 0` gate from orchestrator
- Split compositor resolve into prepare (clone) + resolve (shader)
- Move onBeforeCapture before compositor prepare in frameCapture.ts
- Await img.decode() on cloned data-URI images to prevent stale frames
- Stop manipulating scene opacity in compositor (GL canvas overlay suffices)
- Add gsap.set declaration for shader-transitions ambient types
- Add video_missing_timing_attrs lint rule for <video> without id/data-start/data-end

Performance: compositions with video now render at 7.5s (6 workers) instead
of 2m38s on the layered path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(core): auto-inject data-start on video/audio so frame extraction works without explicit attrs

The timing compiler now injects data-start="0" on <video> and <audio>
elements that lack it. This makes discoverMediaFromBrowser() find the
element (it queries video[data-start]), so the frame extraction pipeline
activates automatically. Videos "just work" without requiring authors to
add data-start, data-end, or id attributes.

Also removes the video_missing_timing_attrs lint rule — the compiler
handles the missing attributes automatically, so the lint rule would
only false-positive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(core): add data-hf-auto-start sentinel on auto-injected video timing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(producer): add discoverVideoVisibilityFromTimeline for runtime video discovery

Seeks the GSAP timeline in Puppeteer to discover when each video's parent
scene is visible (opacity > 0). Uses coarse sampling at 100ms steps followed
by binary search refinement to frame-level precision (1/60s). Only processes
videos with the data-hf-auto-start sentinel so author-specified timing is
never overridden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(producer): integrate runtime video visibility discovery into probe stage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(producer): trigger browser probe for auto-start videos, remove debug logging

The probe stage was skipping browser launch when composition duration was
already known, which meant discoverVideoVisibilityFromTimeline never ran.
Now needsBrowser also checks for data-hf-auto-start sentinel in compiled HTML.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): use mkdtempSync for smoke test work directory

Replaces hardcoded /tmp/hf-page-side-smoke with a unique temp directory
via mkdtempSync to resolve CodeQL "insecure temporary file" alert.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format smoke test script

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Vai <vai@heygen.com>
 2026-05-15 16:23:12 -07:00
Miguel Ángel 0f3b207b23 fix: format JSON files, remove unused function, fix capture test mocks 2026-05-06 08:22:55 -07:00
Miguel Ángel 6ef52972fd fix: resolve merge conflict in shader-transitions capture.ts 
Merges main's refactored capture (CaptureSceneOptions, forceVisible,
stabilizeTransformedBoxShadows, foreignObjectRendering fallback) with
our HTML-in-Canvas drawElementImage capture path. The native capture
tries first and falls back to html2canvas on failure.
 2026-05-06 07:49:31 -07:00
Vance Ingalls a7b308b667 feat: cache shader transition preview frames (#634) 
* feat: cache shader transition preview frames

* fix: move shader transition loading to player
 2026-05-06 01:25:09 -07:00
Miguel Ángel 6d2bfe7aaa feat(engine): enable CanvasDrawElement in renderer Chrome args 
Cherry-picked from feat/html-in-canvas-launch (PR #611):

- Enable --enable-features=CanvasDrawElement in Chrome browser args
  so HTML-in-canvas compositions render correctly
- Add native drawElementImage() capture path for shader transitions
  with existing fallback preserved
- Reuse renderer Chrome args in hyperframes validate for consistent
  WebGL/CanvasDrawElement environment
- Add capture.test.ts for the new shader transition capture path
 2026-05-06 00:40:24 -07:00
ukimsanovandClaude Opus 4.6 a8f0752bf6 fix(shader-transitions): address review — NaN guard, canvas sync, shared defaults 
- Validate data-width/data-height: fall back to defaults if NaN or <= 0
- Sync existing #gl-canvas dimensions on reuse (if init called twice)
- Import DEFAULT_WIDTH/DEFAULT_HEIGHT in capture.ts instead of
  hardcoding 1920/1080 in parameter defaults

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 21:41:42 -04:00
ukimsanovandClaude Opus 4.6 6b84971f6b feat(shader-transitions): support any aspect ratio (vertical, square) 
Read data-width/data-height from the composition root instead of
hardcoding 1920x1080. Enables shader transitions on vertical
(1080x1920) and square (1080x1080) compositions.

Changes across 3 files:
- webgl.ts: WIDTH/HEIGHT constants → DEFAULT_WIDTH/DEFAULT_HEIGHT,
  createContext and renderShader accept width/height params
- capture.ts: captureScene and captureIncomingScene accept
  width/height params for html2canvas
- hyper-shader.ts: reads data-width/data-height from root element,
  passes dimensions to all webgl and capture calls

GLSL shaders unchanged — they already use u_resolution uniform for
all coordinate math and work at any aspect ratio.

Backwards compatible: all params default to 1920x1080 when not
provided or when data-width/data-height are missing from the DOM.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 21:19:49 -04:00
ukimsanovandClaude Opus 4.6 18136d4eb5 fix(shader-transitions): add inWindow guard to .catch fallback handler 
Address Vance's review on #456: the .catch handler was missing the
same tl.time() window check that .then has. Late-rejecting captures
(Safari + SVG-filter compositions) could fire gsap.to/fromTo on
scenes the playhead already left, causing flash-to-black mid-scene.

Wraps the CSS crossfade fallback in the same inWindow guard so
stale catch handlers are no-ops, matching the .then behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 20:17:13 -04:00
ukimsanov 41b4017273 docs(shader-transitions): clarify allowTaint caveat in capture comment 
Address Copilot review comment on #456: the old `allowTaint` doc comment
said the resulting canvas is "still usable as a WebGL texture via
gl.texImage2D (no pixel read-back required)", which is wrong. A tainted
canvas CANNOT be uploaded to WebGL — the spec requires SecurityError on
non-origin-clean sources with no opt-out. That's exactly what we observe
in Safari + SVG-filter compositions, and what hyper-shader.ts's catch
handler now handles via CSS crossfade.

Update the comment to correctly describe the flag's effect: it only
moves the failure point from html2canvas to texImage2D; the end-user
UX is the same (smooth CSS fade in either case). The flag remains
defensively correct for the non-taint branches where it genuinely
helps (cross-origin images with `Access-Control-Allow-Origin`).

No code change — comment only.

Made-with: Cursor
 2026-04-23 16:20:32 -04:00
ukimsanov b3b458fda4 fix(shader-transitions): harden capture against visibility, scrub, Safari taint 
Three interrelated fixes for the live-player path in
@hyperframes/shader-transitions (Studio preview, <hyperframes-player>
embeds, Claude Design in-pane iframe). Zero changes to engine mode —
initEngineMode is byte-identical; producer render pipeline and CLI
hyperframes render produce byte-identical output.

1. captureIncomingScene now forces visibility:visible during capture.
   The HF runtime sets visibility:hidden on [data-start] elements outside
   their playback window. With centered shader timing (transition.time =
   boundary - duration/2), html2canvas captures the incoming scene while
   it's still hidden → blank texture → visible blink mid-transition.
   Fix saves, overrides, captures, and restores visibility only for the
   capture window. Empirically validated via a direct html2canvas probe:
   captures of visibility:hidden elements return blank; with override,
   they return real content.

2. post-capture.dom guards on tl.time() window before mutating DOM.
   On scrub across multiple shader transitions, tl.call() fires several
   transitions' callbacks in rapid succession; each launches async
   html2canvas; each .then() unconditionally set all .scene opacities to
   0, enabled shader canvas, and pointed state at that transition. The
   last to resolve won — often for a transition the playhead had left.
   Result: scenes stuck opacity:0 mid-scene; blank screen until the next
   transition's end.call ran. Fix: check tl.time() is still inside
   [T, T+dur] before applying state; otherwise skip.

3. .catch fallback does CSS crossfade instead of hard cut.
   When capture fails (Safari canvas taint from SVG data URLs, CORS
   errors, extreme DOM complexity) the old catch snapped all scenes to
   opacity:0 then set incoming to opacity:1 — jarring instant jump. Fix
   uses gsap.to/fromTo on opacity over the intended transition duration;
   smooth 0.5s fade is strictly better UX. Hard cut preserved as
   last-resort if elements are missing.

Also adds defensive useCORS: true and allowTaint: true to the
html2canvas call. No behavior change in Chrome (capture normally
succeeds); adds resilience for cross-origin images with CORS headers
and SVG-tainted canvases respectively.

Known limitations (out of scope, follow-up tracked):

- Safari + cross-origin iframe: html2canvas is 10-12x slower than Chrome
  due to WebKit's DocumentCloner.cloneNode perf (html2canvas#3108),
  causing perceptible per-transition freezes (1.5-2s each) in Claude
  Design's in-pane preview. Needs pre-capture architecture (cache
  incoming-scene textures at init) to eliminate per-transition cost.
- SVG filter data URLs fundamentally taint html2canvas output in Safari;
  WebGL's texImage2D has no framework opt-out (WebGL spec). Addressed
  at the composition level via the Claude Design skill's anti-pattern 4
  in a parallel PR.

Made-with: Cursor
 2026-04-23 15:15:58 -04:00
Vance Ingalls 2e1a1d91a2 fix(engine,shader): handle matrix3d transforms and hide non-first scenes (#374) 
## Summary

Two correctness fixes in the HDR transform & clipping pipeline: `parseTransformMatrix` now handles `matrix3d(...)` (GSAP's default `force3D: true`), and shader-transitions sets every non-first scene to `opacity: 0` at `t=0` so the engine doesn't over-composite at the start.

## Why

`Chunk 4` of `plans/hdr-followups.md`. Transform extraction and border-radius computation existed but were dead — an HDR video with `rotation: 45` rendered un-rotated, and 3-scene compositions ghosted at `t=0` because every scene defaulted to CSS `opacity: 1` and contributed to the first frame.

## What changed

**Matrix3d support in `parseTransformMatrix`.** `DOMMatrix.toString()` emits `matrix3d` whenever any ancestor in the chain has used a 3D transform — most importantly GSAP's default `force3D: true`, which converts `translate(...)` into `translate3d(..., 0)`. Without this, every GSAP-driven transform was silently dropped during HDR compositing because `videoFrameInjector.getViewportMatrix()` would return `matrix3d(...)` and the blit path would parse it as `null` and fall back to identity. The 16-value column-major form is converted to its 2D affine projection (indices 0, 1, 4, 5, 12, 13 → m11, m12, m21, m22, m41, m42); Z, perspective, and out-of-plane rotation components are dropped.

**Initial-state opacity in `initEngineMode`.** The browser preview branch uses a GL canvas overlay during transitions, so scene opacity at `t=0` doesn't matter visually. The engine branch reads scene opacity directly via `queryElementStacking()` to decide which layers to composite. Without an explicit initial-state tween, every scene defaulted to CSS `opacity: 1` and contributed to the very first frame, causing ghosting/overlap until the first transition fired. `tl.set()` at position 0 anchors the initial state in the timeline graph so reverse seeks from inside a later transition restore it correctly.

These two fixes together make `el.transform` and `el.borderRadius` (already wired in Chunk 7A's `compositeHdrFrame`) actually flow through the GSAP-animated case, and keep the engine's per-frame compositing aligned with what the user sees in browser preview.

## Test plan

- [x] 6 new `alphaBlit.test.ts` cases (identity matrix3d, translate3d, scale + translate3d, rotateZ, malformed arg count, non-finite values).
- [x] Existing `hdr-regression` Window H already CSS-sets `#scene-b { opacity: 0 }` as a fallback; the new `tl.set` is redundant for that case but harmless and removes the need for compositions to remember the CSS workaround.
- [x] Manual: rotated HDR video (`rotation: 45`) appears rotated; `border-radius: 50%` clips to circle; 3-scene composition has no overlap at `t=0`.

## Stack

Chunk 4 of `plans/hdr-followups.md`. Window F of the regression suite documents the bug; the next PR in the stack tightens the `maxFrameFailures` budget to 0.
 2026-04-22 23:55:40 -07:00
Vance Ingalls 9512744c2e refactor(shader-transitions): extract DEFAULT_DURATION and DEFAULT_EASE constants (#367) 
## Summary

Extract `DEFAULT_DURATION = 0.7` and `DEFAULT_EASE = "power2.inOut"` as shared constants in `hyper-shader.ts` and apply them at all three fallback sites (metadata write, browser/render mode, engine mode).

## Why

`Chunk 2` of `plans/hdr-followups.md`. The three fallback sites had drifted apart: the metadata path used `1s` / `"none"` while the actual rendering used `0.7s` / `"power2.inOut"`. A transition that omitted `duration`/`ease` would render at 0.7 s but tell the producer it was 1 s, throwing off the producer's compositing window planning and producing a visible ~0.3 s brightness dropout.

This is a small, high-value correctness fix that runs before the larger Chunk 1 / Chunk 4 work.

## What changed

- New module-level `DEFAULT_DURATION` and `DEFAULT_EASE` constants in `packages/shader-transitions/src/hyper-shader.ts`.
- All three fallback call sites (metadata, browser, engine) now use the constants.
- Explicit `ease: "none"` on the timeline-length anchor tweens elsewhere in the file is intentional (those are linear interpolators driving the shader's progress uniform) and is left unchanged.

## Test plan

- [x] Render a composition with a transition that omits `duration` and `ease` — no brightness dip in the last ~0.3 s of the transition.
- [x] Preview (browser mode) and render (engine mode) produce matching blending curves.
- [x] Render with explicit `duration: 1.5` still works (constants are fallbacks only).

## Stack

Chunk 2 of `plans/hdr-followups.md`. Lands ahead of Chunk 1 (opacity) per the suggested merge order.
 2026-04-22 17:02:42 -07:00
Vance Ingalls 5de5df7fbb refactor(types): tighten type safety, dedupe HfTransitionMeta, prune dead LUT export (#366) 
## Summary

Four small, mechanical type-safety cleanups across `engine`, `producer`, and `shader-transitions`. Zero behavior change — pure pre-cleanup so the rest of the stack ships against a tighter baseline.

## Why

`Chunk 6` of `plans/hdr-followups.md`. Several non-null assertions and a duplicate interface had accumulated as rebase artifacts and leftover work-in-progress; lands first because it touches files later chunks edit and removes friction during review.

## What changed

- `renderOrchestrator.ts`: replace `layers[layerIdx]!` with a `for (const [layerIdx, layer] of layers.entries())` so both index and element come from the iterator.
- `engine/types.ts`: drop the duplicate `HfTransitionMeta` interface (rebase artifact); the original definition above it is the documented one. The orphaned doc comment now precedes `HfProtocol`.
- `shader-transitions/hyper-shader.ts`: keep the local `HfTransitionMeta` declaration (the package ships as a standalone CDN bundle and must not depend on `@hyperframes/engine`), but add a sync comment pointing at the source of truth in `engine/src/types.ts`.
- `alphaBlit.ts` + `engine/index.ts`: drop `export` from `getSrgbToHdrLut` and remove its re-export. It was only ever called by the internal `blitRgba8OverRgb48le`; the public surface was dead code.

## Test plan

- [x] `bun run --filter @hyperframes/engine typecheck`
- [x] `bun run --filter @hyperframes/producer typecheck`
- [x] `bun run --filter @hyperframes/shader-transitions typecheck`
- [x] `bun run --filter @hyperframes/engine test` — 308/308 pass (no test changes; assertions removed in code only).

## Stack

Chunk 6 of `plans/hdr-followups.md`. Mechanical cleanup landed early per the suggested merge order.
 2026-04-22 16:56:17 -07:00
Vance IngallsandClaude Opus 4.6 99a903be2f feat(hdr): layered HDR compositing, shader transitions, and HDR image support (#268) 
* feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes

- 15 GLSL→TypeScript shader transitions on rgb48le buffers
- Dual-scene compositing with scene detection via window.__hf.transitions
- --hdr flag gates ffprobe probing (zero overhead on SDR compositions)
- Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT
- Buffer.from() copy in writeFrame() fixes streaming encoder race condition
- SDR rendering fixes (three stacked bugs)
- Object.assign fix for window.__hf preservation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: tighten shader smoke thresholds + assert .scene contract

- Tighten the all-transitions smoke test thresholds: at progress=0 we now
  require the center pixel R-channel > 35000 (was > 25000) and at
  progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly
  halfway between the test from-pixel (40000) and to-pixel (10000), so a
  half-blended transition would silently pass.
- Add a runtime assertion in HyperShader.init() that every scene id
  resolves to a DOM element with the .scene class. Without this, missing
  ids silently no-op when textures + querySelectorAll(.scene) run later.

Addresses deferred review feedback from PR #268.

* fix(hdr): restore VIRTUAL_TIME_SHIM and applyRenderModeHints in renderOrchestrator

Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR
rendering fixes") accidentally removed two pieces of the deterministic
rendering pipeline:

1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`,
   which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven
   animations advance only when `window.__hf.seek(t)` is called.
2. The `applyRenderModeHints` function and its post-`compileForRender`
   call site, which auto-forces screenshot capture mode for compositions
   the compiler flagged as needing it (RAF, iframes, etc.).

Without (1), RAF animations advanced by wall-clock between the main-loop
seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the
sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer
seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat`
lost its automatic fallback to screenshot mode and the child-document
motion stopped being captured.

Both helpers are still produced by `htmlCompiler` and exercised by
`renderOrchestrator.test.ts` — the orchestrator just stopped calling
them. Restored:

- Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js`
- Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer`
  call sites (probe + main render)
- Re-add `applyRenderModeHints` (matching the test expectations) and
  call it immediately after `compileForRender`
- Persist `renderModeHints` in `summary.json` and the
  "Compiled composition metadata" log line

Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression
failures on `feat/hdr-layered-compositing`.

Made-with: Cursor

* test(engine): expand sampleRgb48le coverage + audit Uint16Array alignment

Adds:
- 8 new sampleRgb48le bilinear-interpolation tests covering boundary
  pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers.
- uint16-alignment-audit.test.ts documenting the alignment requirement
  for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE.

Background: ~105 hot-loop sites in shader transitions still use
readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut
overhead but requires guaranteed even byteOffsets — these tests document
the contract before any future refactor lands.

* fix(engine,producer): mask DOM layers during HDR layered compositing

The HDR layered compositor blits z-ordered layers over a shared canvas. DOM
layers used a full-page screenshot from `captureAlphaPng`, which captures
*every* painted pixel on the page — root background, sibling-scene content,
overlay UI elements that aren't part of the current layer. Those opaque
pixels were then blitted over the canvas, overwriting any HDR content
composited beneath in earlier layers.

The previous workaround toggled `display:none` on hide ids via
`hideVideoElements`/`showVideoElements`. That correctly hid native videos
but did nothing about the root composition's background or about overlay
elements that the layer grouping considered part of a different layer.

This commit replaces the workaround with a precise CSS mask installed
before each DOM screenshot:

1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and
   re-shows the layer's elements (and their descendants and their injected
   `__render_frame_*` siblings) with `visibility: visible !important`. CSS
   visibility is *not* multiplicative through descendants — a child with
   `visibility: visible` overrides an ancestor's `visibility: hidden`, so
   deeply nested layer content still paints even though every intermediate
   ancestor is hidden by the mass-hide rule.
2. Non-layer data-start ids are inline-hidden with
   `visibility: hidden !important`. Inline `!important` beats stylesheet
   `!important`, so this overrides the show rule for elements that fall
   under a show selector but should NOT paint — most importantly HDR
   videos and other-layer SDR videos that live as descendants of `#root`.
3. `removeDomLayerMask` tears the stylesheet down and clears the inline
   `visibility`/`opacity` properties so subsequent video frame injection
   gets a clean slate.

Crucially the mask only sets `visibility`, never `opacity`. CSS opacity
*is* multiplicative — `opacity: 0` on `#root` would zero out every
descendant including layer videos, even with `visibility: visible`. We
also extend `initTransparentBackground` to force the composition root
(`[data-composition-id]`) transparent in addition to `html`/`body`,
because compositions almost always set `#root { background: ... }` and
that background paints across the whole viewport otherwise.

Both compositing paths use the new helpers:
- The per-layer DOM branch (`compositeToBuffer`) for normal frames.
- The transition path (single DOM screenshot per scene) so transition
  frames also get a clean per-scene capture.

Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`:
per-layer pixel-add accounting, dumps of every captured DOM PNG, and a
periodic raw `rgb48le` snapshot of the composite buffer. These were
essential to diagnosing the root-overwrite bug and stay zero-cost in
normal renders. Also stops the workDir / per-video frame-dir cleanup
when `KEEP_TEMP=1` so the dumps survive past frame N.

Made-with: Cursor

* fix(engine): preserve GSAP-applied opacity across DOM-layer captures

SDR clips inside an HDR composition were rendering at full opacity even
when the user had animated their wrapper opacity (e.g. fade-in or
yoyo). Two bugs in the per-layer screenshot path conspired to drop the
GSAP-applied opacity on the floor:

1. removeDomLayerMask was unconditionally calling
   `el.style.removeProperty("opacity")` on every wrapper after each
   layer capture. applyDomLayerMask only ever sets `visibility`, so the
   only inline opacity present is the value GSAP wrote. Stripping it
   between layer captures means that on the next capture (at the same
   timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline
   is already at that time — the opacity is never restored, and the
   wrapper renders fully opaque.

2. injectVideoFramesBatch was reading the source <video>'s computed
   opacity via `parseFloat(computedStyle.opacity) || 1` and copying it
   onto the injected <img>. Because syncVideoFrameVisibility forces the
   <video> to `opacity: 0 !important` to hide it during capture, the
   computed value is always 0, which `|| 1` then silently flips to
   full opacity. The <img> is a sibling of the <video> inside the same
   wrapper, so it should inherit opacity from the wrapper directly
   instead of having a value hard-set on it.

Fix both: drop the opacity removal in removeDomLayerMask, skip opacity
when copying visual properties from <video> to <img>, and explicitly
clear any stale inline opacity on the <img> so it inherits from the
wrapper that GSAP is animating.

Made-with: Cursor

* fix(producer): correct hdrLayerStartTimes typo to hdrVideoStartTimes

The diagnostic logging block in executeRenderJob's HDR layer composite
path referenced an undeclared `hdrLayerStartTimes` map. The correct
variable, declared and populated earlier in the same function, is
`hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer
masking work and broke the producer build/typecheck on CI.

Made-with: Cursor

* fix(engine): restore video opacity copy to injected frame img

Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on
the assumption that the <img> sibling would inherit GSAP's opacity from
a shared wrapper. That breaks any composition where GSAP animates opacity
directly on the <video> element itself: the <img> has no animated
ancestor and renders at full opacity throughout any fade, even when the
user's intent is partial or zero opacity.

The CI `style-7-prod` and `style-8-prod` regressions caught this:
the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut
because the <img> inherited opacity 1 regardless of GSAP's tween.

Restore the old explicit copy from `computedStyle.opacity` to the
<img>'s inline opacity, with the `|| 1` fallback intentionally
preserved. The fallback is load-bearing: GSAP's seek does not re-apply
tweens that have already completed, so post-fade frames read opacity 0
from the stale `opacity: 0 !important` we apply to hide the native
<video>. The `|| 1` recovers the tween's end-state opacity 1 for
those frames, matching the final on-screen intent and the existing
baseline renders.

Handles both DOM shapes:
- GSAP on wrapper: video's own computed opacity is 1, img set to 1,
  wrapper's opacity applies via stacking as before.
- GSAP on <video>: video's computed opacity is the tween value, copied
  to img directly since they are siblings.

Fixes:
- style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33)
- style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-19 19:00:58 -07:00
Vance Ingalls 8548a17771 feat(hdr): GSAP transforms and border-radius masks on HDR video (#290) 
## Summary

HDR video elements with GSAP animations (position, scale, rotation, opacity) and CSS border-radius rendered without any transforms applied — the video just sat at (0,0) full-size. This PR adds affine transform support and rounded-corner masking for natively-composited HDR video.

## What it does

**Affine blit with bilinear interpolation:**
- `blitRgb48leAffine()` — Takes a 4x4 DOMMatrix and maps each destination pixel back to source coordinates via the inverse transform. Bilinear interpolation between the 4 nearest source pixels produces smooth edges under rotation and non-integer scaling. Optional opacity and border-radius parameters.
- `parseTransformMatrix()` — Parses CSS `matrix(a,b,c,d,e,f)` strings into `[a,b,c,d,e,f]` tuples.

**Accumulated viewport matrix:**
- `getViewportMatrix()` — Walks the `offsetParent` chain from element to viewport, accumulating position offsets and CSS transforms at each level. Correctly handles `transform-origin` using the CSS sandwich: `translate(origin) × M × translate(-origin)`. This is critical because GSAP animates transforms on wrapper divs, not directly on the video element.

**Effective opacity:**
- `getEffectiveOpacity()` — Multiplies opacity values walking up the ancestor chain. Uses `Number.isNaN()` (not `|| 1`) so opacity:0 isn't incorrectly treated as 1.

**Border-radius masks:**
- `roundedRectAlpha()` — Per-pixel anti-aliased rounded-rectangle mask with support for independent corner radii.
- `getEffectiveBorderRadius()` — Walks ancestors for `overflow:hidden` + border-radius. Resolves percentage values (e.g., `50%` for circles) via `offsetWidth`/`offsetHeight`.

**Layout dimensions for extraction:**
- Uses `offsetWidth`/`offsetHeight` (unaffected by CSS transforms) instead of `getBoundingClientRect()` (which returns the transformed bounding box and wobbles under rotation).

## Files changed

| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | `blitRgb48leAffine()`, `parseTransformMatrix()`, `roundedRectAlpha()`, `cornerAlpha()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `getViewportMatrix()`, `getEffectiveOpacity()`, `getEffectiveBorderRadius()`, `layoutWidth`/`layoutHeight` on `ElementStackingInfo` |
| `packages/producer/src/services/renderOrchestrator.ts` | Affine blit path, extraction at layout dimensions, border-radius parameter passing |

## How to test

Render a composition with an HDR video that has GSAP scale + rotation animation and a `border-radius: 50%` wrapper (circle mask). The video should rotate smoothly with round edges — no wobble, no sharp corners.

## Stack position

**5 of 6** — Stacked on #289 (z-ordered layers). Adds transform and masking support to the HDR blit that the layer compositor uses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-19 17:03:29 -07:00
Vance Ingalls cb3d94c2a5 feat: add @hyperframes/shader-transitions package (#251) 
## Summary

New `@hyperframes/shader-transitions` package that encapsulates WebGL shader transitions into a single `HyperShader.init()` call. Replaces ~200 lines of per-composition boilerplate that LLMs failed to wire correctly 60% of the time.

### API

```js
var tl = HyperShader.init({
  bgColor: "#0a0a1a",
  accentColor: "#6366f1",
  scenes: ["scene1", "scene2", "scene3", "scene4", "scene5"],
  transitions: [
    { time: 7.2, shader: "cross-warp-morph", duration: 0.7 },
    { time: 15.2, shader: "domain-warp", duration: 0.7 },
  ]
});
tl.from("#s1-title", { y: 50, opacity: 0, duration: 0.7 }, 0.3);
```

### What the library handles

- **13 shader programs**: domain-warp, ridged-burn, whip-pan, sdf-iris, ripple-waves, gravitational-lens, cinematic-zoom, chromatic-split, glitch, swirl-vortex, thermal-distortion, cross-warp-morph, light-leak
- **html2canvas** bundled as dependency (not CDN) — single script tag for CLI users
- **DOM-during-holds**: canvas hidden between transitions, GSAP animations play on live DOM
- **Async capture with pause/resume**: timeline pauses during capture, resumes after textures uploaded — prevents progress tween from running ahead
- **Accent color theming**: `accentColor` derives dark/mid/bright uniforms. Burns, glows, leaks match the composition palette
- **Graceful degradation**: falls back silently when WebGL unavailable

### Code quality (from 3 review agents)

- No `!` non-null assertions — all WebGL creation calls throw on failure
- Vertex shader compiled once, cached across all programs
- Uniform/attribute locations cached per program via WeakMap (not looked up every frame)
- Captured canvases freed after texture upload (8MB each)
- Single timeline creation (was creating two, discarding one)
- Shared `tickShader()` render callback (was copy-pasted)
- `.finally()` for DOM restore in capture (was duplicated in `.then`/`.catch`)
- `parseHex` validates input (was silently producing NaN on invalid hex)
- Dead `ND`/`CP` shader library exports removed

### Shader-compatible CSS rules (transitions.md)

6 rules for compositions using shader transitions:
1. No `transparent` in gradients (canvas interpolates through black)
2. No gradient backgrounds on elements < 4px
3. No CSS variables on captured elements
4. `data-no-capture` for uncapturable decoratives
5. No gradient opacity < 0.15
6. Every `.scene` must have explicit `background-color` matching `bgColor`

### Build output

- IIFE (~214KB with html2canvas bundled, ~65KB gzipped) — `window.HyperShader`
- ESM + CJS + TypeScript declarations
- tsup build following `@hyperframes/player` conventions

## Test plan
- [ ] `bun run build` succeeds (includes shader-transitions)
- [ ] `bunx oxlint packages/shader-transitions/src/` — 0 errors
- [ ] Create a composition using `HyperShader.init()` — verify transitions fire, DOM animations play, accent colors match
- [ ] Test graceful degradation: composition works without WebGL (no transitions, no crash)
- [ ] Verify pause/resume: scrub to transition boundary — no jump in progress

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 18:44:00 -07:00