ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-04 07:19:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
614 Commits 870 Branches 404 Tags
2a897d351c5a3753a6c5bbd379a00bd5132852e7
Commit Graph
7 Commits
Author SHA1 Message Date
Miguel Ángel 2a897d351c fix: address PR #596 review issues (#597) 
## Summary

Fixes three issues identified in the [post-merge review](https://github.com/heygen-com/hyperframes/pull/596#pullrequestreview-4214283515) of PR #596:

- **P1 (cache bypass):** When `extractCacheDir` is set, extracted frames live outside `compiledDir`, so `createCompiledFrameSrcResolver` rejects them and every frame falls back to base64 data URIs. Fix: symlink cached frame directories into `compiledDir/__hyperframes_video_frames/` after extraction and remap `framePaths` so the served-frame fast path works.

- **P2 (pooled browser stale state):** `closeCaptureSession` force-killed the Chrome process on timeout via raw `SIGKILL` without clearing `pooledBrowser` / `pooledBrowserRefCount`, leaving other sessions with a dead browser reference. Fix: add `forceReleaseBrowser()` in `browserManager` that atomically clears pool state before killing the process.

- **P3 (reserved chars in URLs):** `createCompiledFrameSrcResolver` encodes path segments with `encodeURIComponent`, but the file server used `c.req.path` (which only applies `decodeURI`) to look up files on disk. Video IDs containing `#`, `?`, or `%` produced 404s. Fix: apply `decodeURIComponent` per path segment in the file server's catch-all route.

## Test plan

- [x] `createCompiledFrameSrcResolver` tests: symlinked cache paths resolve to served URLs; cache-external paths return null; reserved characters encode correctly
- [x] `forceReleaseBrowser` tests: kills process + disconnects; tolerates already-killed process
- [x] `createFileServer` test: `video%231/frame.jpg` serves file from `video#1/frame.jpg` on disk
- [x] Typecheck: engine + producer pass
- [x] Lint + format: 0 warnings, 0 errors
 2026-05-02 06:30:12 +02:00
Vance Ingalls a3d7cc1c95 refactor(producer): extract HDR compositing helpers and rename media metadata (#373) 
## Summary

Four behavior-preserving refactors that reduce complexity in `renderOrchestrator.ts` and clarify the engine ffprobe utility surface. Lands after the correctness fixes (Chunks 1–5) so the refactored code is already correct.

## Why

`Chunk 7` of `plans/hdr-followups.md`. The HDR composite block had grown a ~200 LOC inline closure with 14 captured deps, a repeated capture-options spread, a `extractVideoMetadata` name that now also handles still images, and per-frame re-creation of debug helpers.

## What changed

**7A — Hoist `compositeToBuffer` into a module-scoped helper.** Extract the inline HDR closure into a top-level `compositeHdrFrame()` that takes an `HdrCompositeContext` struct. Construct the context once at the top of the HDR render block and pass it through. Removes a deeply-nested closure from the middle of the orchestrator.

**7B — `buildHdrCaptureOptions()` helper.** Factor the repeated `{ ...captureOptions, skipReadinessVideoIds: ... }` spread into a named helper at the call site.

**7C — Rename `extractVideoMetadata` → `extractMediaMetadata`.** Reflects that the helper handles still images (PNG/JPEG/WebP) in addition to video. Update all callers in engine + producer (`videoFrameExtractor`, `htmlCompiler`, regression-harness, producer ffprobe re-export, tests). Re-export the old name as a deprecated alias from `@hyperframes/engine` for backward compatibility, plus the producer re-export shim.

**7D — Hoist debug counters to module scope.** `countNonZeroAlpha` and `countNonZeroRgb48` are now module-scoped so they aren't re-created per frame and so the closure has fewer captures.

Also touches the `hdr-regression` and `hdr-hlg-regression` README + `meta.json` files reviewed during this refactor.

## Test plan

- [x] `bunx tsc --noEmit -p packages/producer && bunx tsc --noEmit -p packages/engine` clean.
- [x] Engine tests: 313 pass, 0 fail (1218 expect calls).
- [x] `bunx oxlint` + `bunx oxfmt --check` clean on 8 changed source files.
- [x] Diff is structural only — no behavioral changes.

## Stack

Chunk 7 of `plans/hdr-followups.md`. Lands after the correctness fixes (Chunks 1–5) per the suggested merge order.
 2026-04-22 22:41:30 -07:00
Miguel Ángel b98093aa1c fix: remove hidden audio gain in renders (#362) 
## Summary

This fixes a render-time audio correctness bug where Hyperframes applied a hidden post-mix gain to every rendered output, boosting audio by about +2.6 dB and causing clipping on normally leveled sources.

It also fixes a related mute bug where `data-volume="0"` was treated as falsy and silently converted back to full volume during audio track preparation.

Additionally, this PR fixes the Studio workspace typecheck path for `@hyperframes/player`, so local pre-commit/typecheck flows no longer depend on the Player package having been built first.

## Root Cause

The issue report measured a near-constant gain increase and suspected a hidden normalization step. After tracing the engine audio path, the root cause turned out to be explicit code, not FFmpeg behavior:

- `packages/engine/src/config.ts` defaulted `audioGain` to `1.35`
- `packages/engine/src/services/audioMixer.ts` always appended a post-mix FFmpeg filter:
  - `[mixed]volume=${masterOutputGain}[out]`
- with the default config, that meant every render got multiplied by `1.35`

That exactly matches the issue reporter's measured scalar boost.

While investigating the workaround, I also found a second correctness bug:

- `processCompositionAudio()` used `element.volume || 1.0`
- that coerced `0` to `1.0`
- so `data-volume="0"` did not actually mute the track in rendered output

Separately, the repo-level Studio typecheck could fail before any build step because:

- `packages/studio/src/player/components/Player.tsx` imports `@hyperframes/player`
- `packages/player/package.json` points TypeScript at built `dist/*` outputs
- in a fresh workspace, those built outputs may not exist yet
- Studio therefore failed type resolution for `@hyperframes/player` during pre-commit/typecheck

## What Changed

1. Set the engine default `audioGain` back to unity (`1`)
2. Preserve explicit zero volumes by changing `element.volume || 1.0` to `element.volume ?? 1.0`
3. Added regression coverage for both behaviors
4. Updated the producer-side config fixture to reflect the corrected default
5. Added a Studio tsconfig path mapping for `@hyperframes/player` to the local workspace source and widened `rootDir` so workspace typecheck succeeds without requiring a prior Player build

## Why These Changes Are Needed

This is not a UX preference issue; it is a correctness and API contract issue.

- The docs describe `data-volume` as a direct 0-1 control.
- Rendered output should preserve source levels unless the author explicitly changes them.
- Hidden global gain makes output non-deterministic from the author's perspective.
- `data-volume="0"` must mean silence, not full-volume playback.
- Local workspace typecheck should not require unrelated package build artifacts to exist first.

Leaving the current behavior in place means:

- voice recordings near normal peak levels can clip during render
- authors need undocumented manual compensation (`0.75`-ish scaling) to get unity output
- mute semantics in docs and code diverge
- local pre-commit/typecheck can fail for reasons unrelated to the actual diff being committed

## Testing

### Focused regression tests

Ran:

- `packages/engine/node_modules/.bin/vitest run packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.test.ts`

Result:

- `10 passed`

These tests specifically verify:

- default resolved `audioGain` is `1`
- a track with `volume: 0` stays `volume=0` in the FFmpeg filter graph
- the post-mix output filter stays at unity gain (`[mixed]volume=1[out]`)

### Broader package verification

Ran:

- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine build`
- `packages/engine/node_modules/.bin/vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/producer typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/audioMixer.ts packages/engine/src/services/audioMixer.test.ts packages/producer/src/services/renderOrchestrator.test.ts packages/studio/tsconfig.json`
- `bunx lefthook run pre-commit`

Results:

- full engine test suite passed (`309 passed`)
- engine build passed
- touched producer test file passed (`7 passed`)
- producer typecheck passed
- studio typecheck passed
- oxlint passed with `0 warnings, 0 errors`
- formatting passed
- pre-commit hook no longer hits the prior `@hyperframes/player` module-resolution blocker

## Known Verification Limitation

There is no meaningful browser UI flow for this bug: the defect is in the engine/CLI audio render pipeline rather than an interactive browser surface. Because of that, verification was done at the renderer and test level rather than through an agent-browser flow.

## User Impact

After this change:

- rendered audio matches source level by default
- authors no longer need to compensate for a hidden +2.6 dB boost
- `data-volume="0"` correctly mutes rendered audio
- the documented volume contract matches engine behavior again
- local workspace typecheck no longer depends on prebuilt `@hyperframes/player` artifacts

Closes #361.
 2026-04-21 20:33:25 +02:00
Miguel Ángel e8a48a62d0 fix(producer): external assets work on Windows (GH #321) (#324) 
* fix(producer): external assets work on Windows (GH #321)

Two Unix-only assumptions in the external-asset pipeline caused every
absolute path on Windows to be rejected as "unsafe" at render time:

1. Containment checks used `child.startsWith(parent + "/")`. On Windows
   the separator is `\`, so the predicate is always false unless the
   paths are equal — every external asset tripped the safety guard in
   `renderOrchestrator.ts`. The reporter saw:

     [Render] Skipping external asset with unsafe path:
       hf-ext/D:\coder\reactGin\hyperframes\reading\assets\segment_001.wav

   Fix: use `path.relative()` through a shared helper
   `isPathInside(child, parent)` that normalises separators per-platform
   and correctly rejects siblings whose names start with the parent
   (e.g. `/foo/bar-sibling` is NOT inside `/foo/bar`).

2. The external-asset key was built as `"hf-ext/" + absPath.replace(/^\//, "")`.
   A Windows absolute path (`D:\coder\...`) became
   `"hf-ext/D:\\coder\\..."` — and because Node's `path.join` treats a
   drive-letter prefix as absolute, `join(compileDir, key)` silently
   escaped `compileDir`. Fix: `toExternalAssetKey()` strips the drive
   colon and normalises to forward slashes, producing
   `hf-ext/D/coder/...` — a pure relative path that `path.join` cannot
   promote to absolute on any OS.

Both helpers live in `packages/producer/src/utils/paths.ts` and are
exercised by 14 unit tests covering Unix paths, Windows drive-letter
paths, mixed separators, sibling-prefix confusion, and `..` traversal.

Docs: new "External assets" section in `docs/packages/producer.mdx`
describes detection, sanitised keys, and the cross-platform containment
invariant.

Closes #321.

* fix(producer): address review on #324 — UNC + integration test

Addresses the non-blocking observations from the PR #324 staff review
(https://github.com/heygen-com/hyperframes/pull/324#issuecomment):

1. UNC and extended-length Windows paths.
   `toExternalAssetKey` now handles:
   - `\\?\D:\very\long\path\clip.mp4` (extended-length)      → `hf-ext/D/very/long/path/clip.mp4`
   - `\\server\share\file.wav` (plain UNC)                    → `hf-ext/unc/server/share/file.wav`
   - `\\?\UNC\server\share\file.wav` (extended-length UNC)   → `hf-ext/unc/server/share/file.wav`
   The UNC-collapsed form keeps the server boundary so two different
   servers exposing the same share/file name cannot collide under one
   relative key. Previously both edge cases silently produced keys with
   stray `?` or `:` characters that downstream `isPathInside` rejected —
   not a security hole, but a silent drop of user assets.

2. Short-circuit on already-sanitised input.
   `toExternalAssetKey("hf-ext/…")` now returns its input unchanged
   instead of prepending `hf-ext/` a second time. Makes the helper
   genuinely idempotent, which is what the unit test claimed all along.
   Renamed the test accordingly.

3. JSDoc caller contract.
   `toExternalAssetKey` now documents that it expects canonicalised
   input (`path.resolve`'d upstream) and does not strip `..`
   components. `isPathInside` at copy time is still the defensive
   backstop — called out explicitly in the doc so future callers read
   the contract before the code.

4. End-to-end integration test.
   `renderOrchestrator.test.ts` gains two seam tests that run the full
   external-asset pipeline — build the sanitised key, populate an
   `externalAssets` map, invoke `writeCompiledArtifacts`, and assert
   both the success path (the file lands under `<compileDir>/hf-ext/…`)
   and the escape-rejection path (a malicious `hf-ext/../../etc/passwd`
   key does NOT materialise above `compileDir`). `writeCompiledArtifacts`
   is exported for the test seam with a clear JSDoc disclaimer that
   it's not part of the public API.

22 tests pass across `paths.test.ts` (17) and `renderOrchestrator.test.ts` (5).

Out of scope for this follow-up (tracked as follow-ups):
- Centralising every `startsWith("/")` absolute-path check into a
  shared helper across htmlCompiler / audioExtractor / audioMixer /
  videoFrameExtractor. Mentioned in the review; touches 5 files and
  deserves its own PR.
- Windows CI runner.
 2026-04-18 20:59:51 +02:00
James RussoandClaude Opus 4.6 9115d7364c feat(producer): add request-level render concurrency semaphore (#232) 
Add a FIFO semaphore to limit concurrent renders in the producer server,
preventing Chrome CPU contention that causes beginFrame failures.

- New Semaphore utility class (packages/producer/src/utils/semaphore.ts)
- Both blocking render and SSE renderStream handlers acquire/release the semaphore
- SSE stream sends a "queued" event when request must wait
- New GET /render/queue endpoint exposes active/queued render counts
- Configurable via HandlerOptions.maxConcurrentRenders or PRODUCER_MAX_CONCURRENT_RENDERS env var (default: 2)
- New --max-concurrent-renders CLI flag (1-10)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 11:25:32 -07:00
Vance Ingalls 20be2ea1c2 style: apply oxfmt baseline formatting across all source files (#25) 
## Summary
- Run `oxfmt .` across the entire codebase to establish formatted baseline
- 299 files changed — mechanical formatting only, no logic changes
- Double quotes, semicolons, 2-space indent, trailing commas, 100 print width

Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm format:check` — all 426 files pass
- [x] `pnpm -r typecheck` — all packages pass
- [x] `pnpm build` — all packages build
- [x] All 348 tests pass
 2026-03-23 17:15:14 -07:00
Vance IngallsandClaude Opus 4.6 9f8e5ba5a1 initial code (#2) 
* feat: initial code port from hyperframes-internal

Port all OSS-ready packages from the internal monorepo:
- @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime
- @hyperframes/cli — CLI for creating, previewing, and rendering compositions
- @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg)
- @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg)
- @hyperframes/ui-player — browser-based video player component
- @hyperframes/studio — composition editor (React frontend + Hono backend)

Includes regression test suite with Docker-based test harness.

All HeyGen-internal references, deployment infrastructure, and
proprietary assets have been removed. Package names migrated
from @app/* to @hyperframes/*.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrub internal codenames and stale references from OSS port

- Replace static.heygen.ai runtime URLs in test fixtures
- Remove internal CDN publish script (publish-hyperframe-runtime.ts)
- Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime
  with neutral names (studio, hyperframe-runtime, __hyperframeRuntime)
- Fix stale Vault API / localhost references in docs
- Remove broken deprecated_studio link

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove remaining internal codenames and stale references

- Delete stale producer README.md and PIPELINE.md (referenced nonexistent files)
- Replace "Cerberus" codename with "HyperFrames" in test design reviews
- Replace magic-edit postMessage identifiers with hf-preview/hf-parent
- Rename debug-magic-edit-timeline.ts to debug-timeline.ts
- Replace "Motion Cut" with "HyperFrames" in Timeline comments
- Fix studio/CLI references to nonexistent archive package
  (use local data/projects/ dir, stub render proxy)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 22:43:56 -07:00