mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 00:56:23 +00:00
28f948aa7dc52a41eb1b58636130c43759a2fdf0
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
2729ee5087 |
refactor: delete orphan declarations flagged by fallow (#949)
* ci: run fallow audit in lefthook pre-commit Mirrors the same `fallow audit --base ... --fail-on-issues` check that runs in CI, but locally against HEAD so issues surface at commit time instead of after the push round-trip. Scoped to `packages/**` source files via the glob — non-code edits (README, docs, top-level configs) skip the hook entirely. Measured locally: ~5s in parallel with the existing lint/format/typecheck checks. Doesn't extend wall-clock time because typecheck (~11s) is the long pole, and lefthook runs commands in parallel. The default `--gate new-only` means inherited findings don't block the commit — same gate behavior as CI, so local pre-commit and PR audit agree. * refactor: delete orphan declarations flagged by fallow After fallow's auto-fix de-exports unused symbols, oxlint surfaces them as no-unused-vars. This PR deletes those orphan declarations outright. Biggest cleanup: studio/src/icons/SystemIcons.tsx shrinks from 132 to 57 lines — 33 unused icon wrappers and their phosphor-icon imports deleted. Other deletions across 14 more files covering paired getter/setters, helper functions, dead env constants, internal components with no callers, and cascading unused imports. Cascade-causing files held back for follow-up PRs: renderOrchestrator barrel of captureCost re-exports, telemetry/portUtils/remote barrels, Button.tsx + ui/index.ts (would orphan whole file), studioMotion type re-exports. Test plan: typecheck clean across 8 packages, oxlint + oxfmt clean, fallow audit exit 0 (remaining findings inherited), cli + studio vitest suites pass. |
||
|
|
f84cc492de |
perf(engine): faster shader transitions via page-side WebGL compositing (#832)
* fix(cli): prefer puppeteer cache + numeric version sort (staff review) Two correctness fixes from PR #821 self-review: 1. Cache priority order. Previous order was hyperframes-managed cache → puppeteer cache. HF cache is pinned to CHROME_VERSION (131-era) which lags 17+ releases behind upstream; if a user separately installed a newer chrome-headless-shell via @puppeteer/browsers install, the CLI would silently hand engine the older HF-cache binary while engine's own resolveHeadlessShellPath would have picked the newer one. Flip the priority so puppeteer cache wins, matching engine semantics. 2. Numeric (not lexicographic) version sort. `readdirSync.sort().reverse()` over names like `linux-148.0.7778.97` and `linux-99.0.6533.123` would return `linux-99...` first because character '9' outranks '1'. Parse each name into integer segments and compare them numerically. Tests: add both-caches-populated and linux-148-beats-linux-99 cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(engine): page-side compositing for shader transitions (opt-in spike) Add an opt-in `--page-side-compositing` flag (CLI) backed by a new engine config field `enablePageSideCompositing` and env var `HF_PAGE_SIDE_COMPOSITING`. When set, SDR shader-transition compositions skip the Node-side layered blend (the hf#677 chain) and instead run the shader inside Chrome via a page-side WebGL canvas; the engine then captures ONE opaque RGB frame per output frame via the existing streaming capture path. This is the strongest non-beginFrame perf lever for Mac users, who cannot take the beginFrame `~5×` path (Chromium structural limit, crbug.com/40656275). Stacks on top of the hf#677 1.95× baseline. Default OFF — existing fixture pins (byte-exact MP4 output) are preserved. Opt-in path is intentionally PSNR-pinned, not byte-equal (WebGL is f32; Node is f64). HDR content forces the existing layered path regardless. Implementation: - engine: new `EngineConfig.enablePageSideCompositing` (default false). - producer/fileServer: new `HF_PAGE_SIDE_COMPOSITING_STUB` early-page script injected into the served HTML head when the flag is on. - producer/renderOrchestrator: when the flag + no HDR + no png-sequence, route SDR transitions through the streaming path instead of the layered HDR stage. - shader-transitions: new `engineModePageComposite.ts` installs a fullscreen WebGL compositor overlay and wraps `window.__hf.seek` so each seek inside a transition window captures both scenes via the Chromium `drawElementImage` API to GL textures, runs the fragment shader, and displays the composited result on the overlay canvas. The engine takes one screenshot per frame and sees the composited overlay. - cli: new `--page-side-compositing` flag sets `HF_PAGE_SIDE_COMPOSITING=true` before producer load. - scripts/page-side-compositing-smoke: bundled-CLI smoke that renders a representative fixture with and without the flag, validates the canary strings are in the shipped bundles, and writes a wall-time pair. Determinism trade documented in the engine config doc-comment. The smoke script enforces the bundled-CLI validation discipline from prior perf work (see internal feedback note `validate_bundled_cli_not_dev_path`). Runtime requirement: Chromium's `CanvasDrawElement` feature (already enabled by the engine's `--enable-features=CanvasDrawElement` launch flag). When the runtime feature is unavailable, the page-side installer logs a warning and falls back to opacity-flip mode — the engine still takes the streaming path; the transition window degrades to a hard scene swap. Vance will validate on Mac Chrome where the feature is supported. Co-Authored-By: Vai <vai@heygen.com> * fix(shader-transitions): use html2canvas for page-side compositor capture The original drawElementImage approach fails in engine render mode because the virtual-time shim prevents Chromium from generating paint records for cloned elements. drawElementImage requires a cached paint record from the browser's compositor — clones created at capture time never receive one because (a) shimmed rAFs deadlock inside the seek wrapper, (b) original rAFs don't produce real paints under virtual-time control, and (c) layoutsubtree canvases don't apply CSS stylesheet rules to children. Switch scene capture to html2canvas (foreignObjectRendering: false), the same JS-based renderer already used by the preview-mode fallback path in capture.ts. html2canvas reads computed styles and renders via its own canvas drawing pipeline with no dependency on the browser paint cycle. Also fixes: - Engine seek must return the result so Puppeteer awaits async seek promises (frameCapture.ts). - GSAP opacity cache: compositor must restore scene opacity before seek, not after — GSAP caches inline values and skips re-writes. - Support check gates on WebGL availability, not drawElementImage. Perf: 15-scene shader-perf fixture (28s, 14 transitions, 30fps) Baseline (Node-side layered): 137s Page-side (html2canvas+WebGL): 33s → 4.1× speedup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(shader-transitions): simplify review fixes for page-side compositor - Use uploadTexture (zeroes canvas backing store after upload) to prevent ~2.2GB transient memory pressure across 280 html2canvas calls per render - Add ignoreElements + stabilizeTransformedBoxShadows to html2canvas call, matching the preview-path capture.ts behavior - Parallelize from/to scene captures with Promise.all - Wrap post-capture render in try/finally so opacity is always restored - Fix WebGL context leak in isPageSideCompositingSupported probe - Remove dead ResolvedTransition.index field - Export stabilizeTransformedBoxShadows from capture.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(producer): unify page-side compositing gating and Docker forwarding Addresses three issues from staff review: 1. ignoreElements filter stripped all in-scene canvases (Chart.js, D3, p5.js) — narrowed to data-no-capture only since the compositor canvas is a body sibling never in the scene subtree. 2. Docker mode silently dropped --page-side-compositing — thread pageSideCompositing through DockerRenderOptions/buildDockerRunArgs with regression tests. 3. Fragmented gating across 4 independent sites could disagree: - Stub injection gated only on cfg flag (leaked into HDR/alpha) - Probe-created fileServer never got the stub - needsAlpha (WebM/MOV) not excluded from the gate - WebGL-unavailable fallback claimed layered path would run but orchestrator had already disabled it Fix: compute stub injection at the same site as the layered-bypass decision (after hasHdrContent is known), using addPreHeadScript on the already-running fileServer. Single predicate now gates both decisions, including !needsAlpha. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf(engine): two-phase drawElementImage capture for page-side compositing Replace html2canvas with native drawElementImage for scene capture in the page-side compositor. drawElementImage reads from the browser's own paint cache, giving pixel-identical output to the preview path. The blocker was that cloned elements inside layoutsubtree canvases have no cached paint record under virtual time — the compositor only paints when explicitly triggered. Fix: split the seek+composite into two phases with an engine-forced paint between them. Phase 1 (seek wrapper, page-side): - GSAP seek positions the timeline - Clone FROM/TO scenes into visible layoutsubtree staging canvases - Set window.__hf_page_composite_pending flag Engine paint force (frameCapture.ts): - Detect pending flag after seek returns - Fire micro Page.captureScreenshot (1x1 clip) via CDP to force the browser compositor to paint all visible elements including staging canvas children Phase 2 (page.evaluate, page-side): - drawElementImage reads the now-valid paint records - Upload textures to WebGL, run shader, show GL overlay Key insight: staging canvases must be visible (not opacity:0) for the browser to paint their children. They sit at z-index:-9998, behind the main DOM and covered by the GL overlay during transitions. Perf: 15-scene fixture (28s, 14 transitions, 30fps): Baseline (Node-side layered): 137s html2canvas + WebGL: 33s (3.7×) drawElementImage + WebGL: 21s (6.6×) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf(engine): optimize two-phase compositor hot path - uploadTextureSource instead of uploadTexture: eliminates ~2.3GB of canvas buffer alloc/dealloc churn (persistent staging canvases don't need the one-shot zeroing behavior) - Fold hasPending check into seek page.evaluate: eliminates one CDP round-trip per frame (~700 unnecessary IPC calls on non-transition frames) - Fix renderShader error handling: on failure, leave source scenes visible as fallback instead of hiding both scenes + GL overlay (which produced black frames) - Move mutable state declarations above resolveComposite to prevent TDZ risk on refactor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): staff review — staging cleanup, pending flag, beginFrame guard - Clear staging canvas children when leaving transition window (prevents visible clone bleed-through on transparent compositions) - Clear __hf_page_composite_pending on all resolveComposite exit paths - Guard micro-screenshot paint force against beginFrame mode (CDP Page.captureScreenshot conflicts with beginFrame compositor control) - Update CLI flag description: document video/canvas limitation, remove stale PSNR claim Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): default-on page-side compositing for SDR shader transitions Page-side compositing is now enabled by default for SDR shader-transition renders without video content. The 6.6× speedup applies automatically — no flag needed. Auto-disables when: - HDR content detected - Alpha output (WebM/MOV/PNG-sequence) - Composition contains <video> elements (cloneNode loses playback state) - beginFrame capture mode (Linux headless) Use --no-page-side-compositing to force the Node-side layered path. Changes: - Engine config: enablePageSideCompositing defaults to true - CLI: flag default flipped to true; --no-page-side-compositing disables - Orchestrator: added composition.videos.length === 0 gate - Docker: forwards --no-page-side-compositing when explicitly disabled - Config tests updated for new default Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): support video elements on page-side compositing fast path Three-phase capture protocol lets shader transitions render video scenes without falling back to the slow Node-side layered pipeline: 1. Seek → compositor records transition metadata, sets pending flag 2. onBeforeCapture → video frame injector updates <img> replacements 3. prepare → cloneNode picks up current video frames, img.decode() awaits 4. micro-screenshot → forces browser to paint cloned elements 5. resolve → drawElementImage reads paint records, shader composites Key changes: - Remove `composition.videos.length === 0` gate from orchestrator - Split compositor resolve into prepare (clone) + resolve (shader) - Move onBeforeCapture before compositor prepare in frameCapture.ts - Await img.decode() on cloned data-URI images to prevent stale frames - Stop manipulating scene opacity in compositor (GL canvas overlay suffices) - Add gsap.set declaration for shader-transitions ambient types - Add video_missing_timing_attrs lint rule for <video> without id/data-start/data-end Performance: compositions with video now render at 7.5s (6 workers) instead of 2m38s on the layered path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(core): auto-inject data-start on video/audio so frame extraction works without explicit attrs The timing compiler now injects data-start="0" on <video> and <audio> elements that lack it. This makes discoverMediaFromBrowser() find the element (it queries video[data-start]), so the frame extraction pipeline activates automatically. Videos "just work" without requiring authors to add data-start, data-end, or id attributes. Also removes the video_missing_timing_attrs lint rule — the compiler handles the missing attributes automatically, so the lint rule would only false-positive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(core): add data-hf-auto-start sentinel on auto-injected video timing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(producer): add discoverVideoVisibilityFromTimeline for runtime video discovery Seeks the GSAP timeline in Puppeteer to discover when each video's parent scene is visible (opacity > 0). Uses coarse sampling at 100ms steps followed by binary search refinement to frame-level precision (1/60s). Only processes videos with the data-hf-auto-start sentinel so author-specified timing is never overridden. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(producer): integrate runtime video visibility discovery into probe stage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(producer): trigger browser probe for auto-start videos, remove debug logging The probe stage was skipping browser launch when composition duration was already known, which meant discoverVideoVisibilityFromTimeline never ran. Now needsBrowser also checks for data-hf-auto-start sentinel in compiled HTML. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): use mkdtempSync for smoke test work directory Replaces hardcoded /tmp/hf-page-side-smoke with a unique temp directory via mkdtempSync to resolve CodeQL "insecure temporary file" alert. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format smoke test script Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Vai <vai@heygen.com> |
||
|
|
c1ba528a1f |
fix(cli): scan puppeteer cache for chrome-headless-shell; warn on system-chrome fallback (#821)
## What Two correctness fixes to `packages/cli/src/browser/manager.ts` so the CLI picks the right Chrome binary for any perf path that depends on `chrome-headless-shell`: 1. **Also scan the puppeteer-managed cache.** `findFromCache` now reads from both `~/.cache/hyperframes/chrome` (the CLI's own managed cache) and `~/.cache/puppeteer/chrome-headless-shell/<version>/<platform-dir>/chrome-headless-shell` (the path layout that the engine's `resolveHeadlessShellPath` already reads from). When `chrome-headless-shell` is present in either cache, it now wins over system Chrome. 2. **Warn when falling through to a non-`chrome-headless-shell` system binary on Linux.** A single one-time `console.warn` explains the perf consequence and points the user at `npx @puppeteer/browsers install chrome-headless-shell`. Linux-scoped because the BeginFrame perf path is Linux-only. ## Why Discovered in a recent spike on the BeginFrame perf path. On a clean install, the CLI's hyperframes-managed cache (`~/.cache/hyperframes/chrome`) is empty, so `findFromCache` returns `undefined`. The CLI then falls through to `findFromSystem()` and picks `/usr/bin/google-chrome`, exporting it to the engine via `PRODUCER_HEADLESS_SHELL_PATH` in `render.ts`. The engine receives that path, sees it's already set, and skips its own correct `~/.cache/puppeteer/chrome-headless-shell` scan. Regular Chrome (147+) has dropped `HeadlessExperimental.enable`. The engine's BeginFrame probe correctly catches this and silently falls back to screenshot mode — but the operator sees no signal, so any user who installed `chrome-headless-shell` via `npx @puppeteer/browsers install` (the standard puppeteer flow) silently loses the perf path. This is a "two codepaths know about 'the chrome we ship' but look in different places" bug. The fix collapses them. ## How - `findFromCache` now consults both caches. Hyperframes-managed cache wins when both contain a binary (preserves existing behavior). - New `findFromPuppeteerCache` mirrors `resolveHeadlessShellPath` from `packages/engine/src/services/browserManager.ts` — same path layout, same newest-first version sort. A comment in both files notes they need to move together if puppeteer ever changes the on-disk layout. - `warnSystemFallbackOnce` is gated on `process.platform === "linux"` and on the binary name (`basename` of the path being `chrome-headless-shell`/`.exe`). One-shot latch so a long-running `hyperframes studio` process isn't spammed. Exported test-reset helper `_resetSystemFallbackWarnForTests` for the unit tests. No public-API changes. `findBrowser`, `ensureBrowser`, `clearBrowser`, `setBrowserPath` all keep the same signatures. ## Test plan - [x] Unit tests added (`packages/cli/src/browser/manager.test.ts`, 7 tests): - cache hit on hyperframes dir - cache hit on puppeteer dir (the new path) - newest-version preference when multiple versions are cached - system fallback + Linux warning emitted - no warning when the resolved path is itself `chrome-headless-shell` (e.g. `HYPERFRAMES_BROWSER_PATH` override) - no warning on macOS (Linux-only perf path) - one-time warning idempotency across repeated `findBrowser()` calls - [x] `bun run --filter @hyperframes/cli typecheck` — passes - [x] `bun run --filter @hyperframes/cli test` — 305/305 passing - [x] `bun run --filter @hyperframes/cli build` — passes - [ ] Manual smoke on a Linux host with chrome-headless-shell in the puppeteer cache (skipped — sandbox already has both binaries and the unit tests cover the resolution logic deterministically; reviewers welcome to verify) — Vai |
||
|
|
91bdffffe6 |
fix(ci): scope LOC check to studio, split useTimelinePlayer + hyperframes-player under 500 LOC (#750)
* refactor: split useTimelinePlayer.ts and hyperframes-player.ts into focused modules (<500 LOC each) * fix(ci): scope 500 LOC check to packages/studio, add allowlist for grandfathered files * feat(cli): Linux ARM64 support — auto-install Chromium on DGX Spark / GB10 / Jetson Chrome Headless Shell has no Linux ARM64 binary. On arm64 Linux: - Detects the platform automatically - Tries to auto-install system Chromium via apt-get (works on Ubuntu/Debian ARM) - Falls back to clear manual instructions with exact commands - 'hyperframes browser ensure' guides through the setup interactively - After setup, all render commands work without any flags * fix(ci): disable Windows Defender real-time monitoring to prevent EPERM builds Path exclusions are insufficient — Defender re-scans new files created during bun install before the exclusion takes effect. Disable real-time monitoring for the entire job duration instead (standard CI practice). * refactor(studio): split all files >500 LOC + extract useToast, delete allowlist All 11 large files split into focused modules under 500 LOC. App.tsx extracted toast logic into useToast hook (493 LOC now). .filesize-allowlist deleted — no longer needed. * fix: remove unused imports from split files, extract useToast from App.tsx App.tsx: 504 → 493 lines (toast logic extracted to useToast hook) timelineDOM.ts: remove unused imports from re-export pattern MotionPanel.tsx: remove unused clampStudioCustomEasePoints import studioMotionOps.ts: remove unused StudioGsapMotionDirection import * fix(ci): use Set-MpPreference to fully disable Windows Defender (both jobs) * fix(producer): use node --experimental-strip-types instead of tsx for build:fonts Eliminates the tsx binary dependency that Windows Defender locks during bun install, causing EPERM errors. Node 22.6+ strips TypeScript types natively with no external binary. * chore: remove .filesize-allowlist — App.tsx is now 493 lines (<500) * fix(ci): disable Windows Defender before checkout to prevent all EPERM races * fix(producer): skip build:fonts if fontData.generated.ts already exists The generated file is tracked in git, so CI doesn't need to regenerate it. This avoids @fontsource/inter node_modules access on Windows which triggers EPERM from Defender scanning during bun install. |
||
|
|
d1f992570a |
fix(cli): use 'where' instead of 'which' on Windows for FFmpeg and br… (#336)
* fix(cli): use 'where' instead of 'which' on Windows for FFmpeg and browser detection - findFFmpeg() now uses 'where ffmpeg' on Windows, 'which ffmpeg' on Unix - whichBinary() now uses 'where' on Windows, 'which' on Unix Fixes FFmpeg detection failure on Windows where 'which' command doesn't exist. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(cli): handle multi-line output from Windows 'where' command Windows 'where' can return multiple paths (one per line) when there are multiple matches on PATH. Take only the first non-empty line. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(cli): extend Windows 'where' fix to whisper, tts, and clipboard modules - whisper/manager.ts: whichBinary() now uses 'where' on Windows - tts/synthesize.ts: findPython() now uses 'where' on Windows - utils/clipboard.ts: detectProvider() now uses 'where' on Windows All functions handle multi-line output from 'where' command. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
|
20be2ea1c2 |
style: apply oxfmt baseline formatting across all source files (#25)
## Summary - Run `oxfmt .` across the entire codebase to establish formatted baseline - 299 files changed — mechanical formatting only, no logic changes - Double quotes, semicolons, 2-space indent, trailing commas, 100 print width Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits) ## Test plan - [x] `pnpm format:check` — all 426 files pass - [x] `pnpm -r typecheck` — all packages pass - [x] `pnpm build` — all packages build - [x] All 348 tests pass |
||
|
|
9f8e5ba5a1 |
initial code (#2)
* feat: initial code port from hyperframes-internal Port all OSS-ready packages from the internal monorepo: - @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime - @hyperframes/cli — CLI for creating, previewing, and rendering compositions - @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg) - @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg) - @hyperframes/ui-player — browser-based video player component - @hyperframes/studio — composition editor (React frontend + Hono backend) Includes regression test suite with Docker-based test harness. All HeyGen-internal references, deployment infrastructure, and proprietary assets have been removed. Package names migrated from @app/* to @hyperframes/*. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scrub internal codenames and stale references from OSS port - Replace static.heygen.ai runtime URLs in test fixtures - Remove internal CDN publish script (publish-hyperframe-runtime.ts) - Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime with neutral names (studio, hyperframe-runtime, __hyperframeRuntime) - Fix stale Vault API / localhost references in docs - Remove broken deprecated_studio link Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining internal codenames and stale references - Delete stale producer README.md and PIPELINE.md (referenced nonexistent files) - Replace "Cerberus" codename with "HyperFrames" in test design reviews - Replace magic-edit postMessage identifiers with hf-preview/hf-parent - Rename debug-magic-edit-timeline.ts to debug-timeline.ts - Replace "Motion Cut" with "HyperFrames" in Timeline comments - Fix studio/CLI references to nonexistent archive package (use local data/projects/ dir, stub render proxy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |