Commit Graph
27 Commits
Author SHA1 Message Date
Miguel Ángel f705402c6d fix(engine,studio): persist auto-assigned video IDs into compiled HTML; shared VideoFrameThumbnail
Render fix: parseVideoElements assigned IDs in JSDOM memory but the
compiled HTML written to disk had no id on id-less video elements, so
injectVideoFramesBatch's getElementById returned null. Now does a second
DOM pass to inject IDs into the HTML string before returning it.

Studio thumbnails: extracted VideoFrameThumbnail into a shared ui/
component. AssetsTab now uses canvas frame extraction (seeks to 10%
duration) instead of <video preload=metadata> which shows a black t=0
frame. RenderQueueItem imports from the same shared component.
2026-03-30 22:24:43 +02:00
Vance Ingalls 229538c622 fix: add media rendering guardrails to prevent silent failures (#112)
## Summary

- **Lint rules** catch media elements missing `id` (renderer silently skips them), missing `src`, `preload="none"` (blocks renderer), and video nested in timed divs (freezes playback). Upgraded `video_nested_in_timed_element` from warning to error.
- **Compiler** strips `preload="none"` from media during compilation. Runs parallel, cached keyframe interval analysis via ffprobe — warns on sparse keyframes (>2s) that cause seek failures and audio/video desync. Suggested ffmpeg command preserves audio (`-c:a copy`).
- **Pre-render lint** lints `index.html` + all `compositions/*.html` sub-compositions before render via shared `lintProject()` helper. Warns by default; `--strict` blocks on errors, `--strict-all` blocks on errors + warnings.
- **Render orchestrator** logs a hint to retry with `--workers 1` when parallel capture times out on video-heavy compositions.
- **Refactor**: extracted `runFfprobe()` + `parseProbeJson()` helpers to deduplicate ~80 lines of spawn boilerplate across 3 ffprobe functions. Extracted `shouldBlockRender()` so strict flag tests exercise production code. Shared `lintProject()` used by both `lint` and `render` commands.

## Context

Discovered during a real composition build session where:
1. `<audio>` without `id` rendered silently (preview worked fine because runtime queries `[data-start]`, but renderer queries `[id][src]`)
2. `<video>` inside timed `<div>` froze on first frame
3. `preload="none"` caused 45s renderer timeout
4. YouTube clips with sparse keyframes from `yt-dlp --download-sections` caused audio/video desync
5. Parallel workers timed out on video-heavy compositions

## Test plan

- [x] Core: 365/365 tests passing (5 new lint tests)
- [x] Engine: 24/24 tests passing
- [x] CLI: 14/14 tests passing (7 lintProject + 7 shouldBlockRender)
- [x] Lint + format hooks pass
- [ ] Manual: create a composition with `<audio data-start="0" src="test.wav">` (no id) — verify `npx hyperframes lint` catches it
- [ ] Manual: run `npx hyperframes render --strict` with lint errors — verify it blocks
- [ ] Manual: run `npx hyperframes render --strict-all` with lint warnings — verify it blocks
2026-03-30 11:07:19 -07:00
Miguel Ángel 95d7dc0623 fix(cli): align render output naming and add WebM support to studioServer (#109)
## Summary

- CLI render: use timestamped filenames (`project_date_time.ext`) matching the studio's naming convention, preventing overwrites of previous renders
- studioServer: read `fps`/`quality`/`format` from POST body instead of hardcoding `fps:30`/`quality:standard`/`mp4`
- studioServer: use timestamped job IDs matching the studio pattern
- studioServer: fix download endpoint to serve correct content-type for WebM

## Test plan

- [x] `hyperframes render --format webm` outputs timestamped WebM file
- [x] `hyperframes render` outputs timestamped MP4 (no overwrite)
- [x] Studio embedded server (`hyperframes dev`) renders with correct format when selected in UI
- [x] Download endpoint serves correct MIME type for WebM renders
2026-03-28 21:49:10 +01:00
James Russo e0ee983e19 Merge pull request #81 from heygen-com/feat/webm-transparency
feat(render): WebM output with VP9 alpha transparency
2026-03-26 22:07:57 -07:00
JamesandClaude Opus 4.6 83d11ac595 refactor: simplify review fixes for WebM PR
- Use static import for copyFileSync (was unnecessary dynamic import)
- Shallow-copy config before mutating forceScreenshot (prevents
  caller-provided config from being permanently modified)
- Consolidate isWebm/isWebmRender/outputFormat into single early
  declaration in renderOrchestrator
- Fix debug output extension for WebM (was hardcoded .mp4)
- Log unexpected audio extraction errors instead of silently swallowing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 03:40:35 +00:00
Miguel Ángel dc0d69a21b feat(producer): support entryFile for rendering individual compositions (#42)
## Changes

- Add optional `entryFile` parameter to render API endpoints (`/v1/render` and `/v1/render-stream`)
- Enable rendering individual sub-compositions by extracting them from index.html context when the entry file is a `<template>` wrapper
- Change base64 audio/video linting from detecting "fabricated" media to prohibiting all inline base64 media
- Add manifest path resolution for bundled producer deployments

## API Changes

- `RenderConfig` — new optional `entryFile` field for specifying HTML file to render
- `server.ts` — parses `entryFile` from request body, validates file exists in project directory
- `executeRenderJob` — uses `entryFile` instead of hardcoded `"index.html"`

## Template Extraction

- `extractStandaloneEntryFromIndex` — extracts sub-composition hosts from index.html and creates standalone render context
- Handles `<template>` entry files by finding matching `data-composition-src` in index.html and isolating that host
- Resets `data-start` to 0 for standalone rendering

## Linting Updates

- Change rule #3.7 from detecting "fabricated" base64 media to prohibiting all inline base64 audio/video
- Lower detection threshold from 100+ to 20+ base64 characters
- All base64 media now triggers error severity with clearer messaging about file size bloat

## Usage

```json
POST /v1/render-stream
{ "projectDir": "/path/to/project", "entryFile": "compositions/intro.html" }
```

Omit `entryFile` for default behavior (renders `index.html`).
2026-03-27 04:28:51 +01:00
JamesandClaude Opus 4.6 efc7b82755 fix(render): fix WebM transparency — use screenshot mode + proper CDP setup
- Force screenshot capture mode for WebM (beginFrame doesn't support
  alpha channel in chrome-headless-shell)
- Set Emulation.setDefaultBackgroundColorOverride once during session
  creation (matching experiment-framework's approach)
- Fix acquireBrowser to pass executable path in screenshot mode on
  Linux (was setting undefined, causing puppeteer-core to fail)
- Add Page.captureScreenshot params: fromSurface, captureBeyondViewport,
  optimizeForSpeed (matching experiment-framework)
- Fix regression harness extractMonoPcm16 for videos without audio
- Add getEncoderPreset unit tests
- Add webm-transparency regression test with golden baseline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 02:35:03 +00:00
JamesandClaude Opus 4.6 a146c9e527 test(render): add WebM regression test and getEncoderPreset unit tests
- New regression test `webm-transparency`: minimal transparent composition
  rendered to WebM, validates VP9 codec and visual quality at 100
  checkpoints against golden baseline
- Extend regression harness: `renderConfig.format` field ("mp4" | "webm"),
  format-aware output paths and snapshot filenames
- Fix `extractMonoPcm16` to gracefully handle videos without audio
  streams (WebM without audio was throwing instead of returning empty)
- Unit tests for `getEncoderPreset()`: VP9/yuva420p for WebM,
  h264/yuv420p for MP4, preset mapping, quality preservation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 02:19:43 +00:00
JamesandClaude Opus 4.6 15f6b86ac0 feat(render): add WebM output with VP9 alpha transparency
Support rendering compositions with transparent backgrounds via
`--format webm`. VP9+alpha is the standard format for overlayable
video (captions, lower thirds, overlays).

Changes by layer:
- CLI: `--format mp4|webm` flag on render command
- Producer: threads format through RenderConfig, switches to PNG
  capture and VP9 encoding when webm
- Engine: getEncoderPreset() returns VP9 config with yuva420p;
  transparent page background via CDP when capturing PNG;
  mux uses Opus audio for WebM; VP9 flags from production:
  -row-mt 1, -auto-alt-ref 0, alpha_mode=1 metadata
- Frame capture: Emulation.setDefaultBackgroundColorOverride a=0

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 02:12:39 +00:00
Miguel Ángel 9262d01109 fix(producer): fall back to data-duration when GSAP timeline is empty (#78)
## What

When a composition has an empty GSAP timeline (no animations), `window.__hf.duration` was always 0, causing `hyperframes render` to time out after 45 seconds waiting for `duration > 0`.

## Why

The HF bridge script reads duration from `window.__player.getDuration()`, which returns the GSAP timeline duration. An empty timeline has duration 0. The render engine waits for `window.__hf.duration > 0` to confirm the runtime is ready — so compositions with no animations would always deadlock.

This hits the `--template blank` scaffold immediately: it generates an empty GSAP timeline and relies solely on `data-duration="10"` for composition timing.

## How

One-line change to the bridge script: when `getDuration()` returns 0, fall back to reading `data-duration` from the root `[data-composition-id]` element. This is the same value the static compiler already extracted — so we get a correct duration without any extra browser round-trips.

```js
get duration() {
  var d = p.getDuration();
  return d > 0 ? d : getDeclaredDuration(); // reads data-duration from root element
}
```

## Test plan

- [x] `hyperframes init my-video --template blank && hyperframes render my-video` completes successfully
- [x] Output: `output.mp4` — 10s, 1920×1080, 30fps ✓  
- [x] Compositions with actual GSAP animations unaffected (fallback only triggers when timeline duration is 0)
- [x] Build passes, lint/format clean
2026-03-27 02:56:52 +01:00
Miguel Ángel e554710646 fix(producer): propagate data-start to inner composition element during compilation (#73)
## Summary
- copy data-start from the host sub-composition node to the inlined inner composition root
- preserve the correct runtime offset lookup for nested compositions after producer compilation
- avoid nested GSAP timelines snapping to their end state when the host starts later than t=0
2026-03-27 00:20:16 +01:00
JamesandClaude Opus 4.6 e746debb1a fix(ci): generate font data in Docker build instead of committing
The generated font data is a pure function of the generator script +
@fontsource package versions — no reason to store 566KB of base64
blobs in git. Generate it during the Docker test image build instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 21:34:25 +00:00
JamesandClaude Opus 4.6 879a1a1ae5 fix(ci): commit generated font data for regression tests
The regression CI runs the producer source directly via tsx (not the
bundled CLI), so it needs fontData.generated.ts to exist at import
time. Remove from .gitignore and commit the generated file.

Mark as linguist-generated in .gitattributes so GitHub collapses it
in PR diffs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 21:22:32 +00:00
JamesandClaude Opus 4.6 7e059e15f9 fix(producer): embed font data at build time instead of runtime require.resolve
The CLI bundle uses tsup to inline @hyperframes/producer, but the
deterministicFonts module used require.resolve('@fontsource/*/package.json')
at runtime to find woff2 files on disk. When installed via npx, these
@fontsource packages don't exist, causing "Cannot find module" errors.

Replace runtime filesystem lookups with a build-time generator that reads
all @fontsource woff2 files and produces a TypeScript module with base64
data URIs. The generator runs before both producer and CLI builds, making
the bundle fully self-contained with zero @fontsource runtime dependencies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 20:59:11 +00:00
Miguel Ángel 251ed23252 fix(producer): add margin reset when wrapping HTML fragments (#55)
## Summary

- When `index.html` is a bare fragment (just a `<div>` without `<!DOCTYPE>`), `ensureFullDocument()` wraps it in a minimal HTML document
- The wrapper was missing a CSS reset, so Chrome applied its default `body { margin: 8px }` — creating visible white lines at the top and left edges of rendered video
- Added `* { margin:0; padding:0; box-sizing:border-box }` and `body { overflow:hidden; background:#000 }` to the fragment wrapper

## Test plan

- [x] Render a composition that starts with a bare `<div>` (no `<!DOCTYPE html>`)
- [x] Verify no white lines appear at the edges of the rendered MP4
2026-03-26 04:48:47 +01:00
James Russo 5d8d112ee8 Merge pull request #54 from heygen-com/fix/manifest-path-resolution
fix(producer): resolve manifest from sibling dist/ directory
2026-03-25 20:31:05 -07:00
JamesandClaude Opus 4.6 45a7a8c986 test(producer): add manifest resolution tests
Verifies:
- env var override takes priority
- sibling path is first candidate (catches the npm package bug)
- monorepo-relative fallback works in dev
- path construction invariant holds

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 02:15:17 +00:00
JamesandClaude Opus 4.6 fbc128ba11 fix(producer): resolve manifest from sibling dist/ directory
The build script already copies hyperframe.manifest.json into the
producer's dist/ directory, but the runtime loader never checked there.
This caused `window.__hf not ready` timeouts when rendering via the
published npm package outside the monorepo, since the only candidate
paths were monorepo-relative.

Add a sibling-directory lookup (same dir as the bundled module) as the
first candidate, so the manifest is found whether running from source,
a bundled dist/, or an npm install.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 02:11:53 +00:00
Miguel ÁngelandClaude Opus 4.6 d3c6228d89 fix(producer): preserve @import rules during CSS composition scoping
scopeCssToComposition corrupted @import url() rules because they have
no {} block. The selector regex ([^{}@]+)\{ treated the text after @
as a selector, producing invalid CSS like:

  @[data-composition-id="x"] import url('...')

This broke font loading, CSS variable resolution, and all composition
styling in rendered output.

Fix: extract @import rules before running the scoping regex, then
prepend them back unmodified.

Also adds:
- Regression test fixture (css-import-scoping)
- Common-mistakes docs: autoplay/loop, GSAP TextPlugin, sub-composition
  positioning
- Format fix for hyperframeLinter.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:51:05 -04:00
Vance IngallsandClaude Opus 4.6 61c5257402 fix(ci): update publish workflow to use bun install (#36)
* fix(ci): update publish workflow to use bun install

pnpm-lock.yaml was removed in the bun migration but publish.yml
still referenced it. Use bun for install/build, keep pnpm for
publish (publishConfig overrides + --provenance).

* docs: update stale pnpm references to bun across docs and scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 08:46:58 -07:00
Miguel Ángel 062f6f1c10 fix: composition issues in runtime code + producer parity 2026-03-23 23:56:42 -04:00
Vance Ingalls 20be2ea1c2 style: apply oxfmt baseline formatting across all source files (#25)
## Summary
- Run `oxfmt .` across the entire codebase to establish formatted baseline
- 299 files changed — mechanical formatting only, no logic changes
- Double quotes, semicolons, 2-space indent, trailing commas, 100 print width

Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm format:check` — all 426 files pass
- [x] `pnpm -r typecheck` — all packages pass
- [x] `pnpm build` — all packages build
- [x] All 348 tests pass
2026-03-23 17:15:14 -07:00
Vance Ingalls 323ff8f860 fix: resolve oxlint errors across codebase (#24)
## Summary
- Remove 5 unused `beforeEach` imports from test files
- Remove unused imports (`existsSync`, `TimelineCompositionElement`)
- Remove unused destructured variables (`options`, `width`, `height`, `goldenEl`)
- Remove dead `formatDuration` function
- Fix unused catch parameters (`catch (err)` → `catch`)
- Prefix unused `renderError` state with `_`
- Add `eslint-disable-next-line` for 2 React exhaustive-deps false positives (stable ref + zustand setter)

Part 2/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm lint` — 0 errors on 193 files
- [x] All 348 tests pass (core + engine)
2026-03-23 16:41:41 -07:00
Vance IngallsandClaude Opus 4.6 00909c8258 fix: revert parity harness to puppeteer-core with channel: chrome
The puppeteer import caused protocol timeout issues. Revert to
puppeteer-core but add channel: "chrome" to auto-discover the
Chrome binary installed by 'puppeteer browsers install'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 01:31:33 -07:00
Vance IngallsandClaude Opus 4.6 8866fc5791 fix: use puppeteer instead of puppeteer-core in parity harness
puppeteer-core requires an explicit executablePath or channel.
puppeteer auto-discovers Chrome installed by 'puppeteer browsers install'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 01:20:11 -07:00
Vance IngallsandClaude Opus 4.6 fbfb6c6156 fix: resolve TypeScript strict null errors in producer
- server.ts: default token param to empty string to satisfy string type
- renderOrchestrator.ts: capture fileServer in local const before closure
  to preserve TypeScript null narrowing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 00:50:19 -07:00
Vance IngallsandClaude Opus 4.6 9f8e5ba5a1 initial code (#2)
* feat: initial code port from hyperframes-internal

Port all OSS-ready packages from the internal monorepo:
- @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime
- @hyperframes/cli — CLI for creating, previewing, and rendering compositions
- @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg)
- @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg)
- @hyperframes/ui-player — browser-based video player component
- @hyperframes/studio — composition editor (React frontend + Hono backend)

Includes regression test suite with Docker-based test harness.

All HeyGen-internal references, deployment infrastructure, and
proprietary assets have been removed. Package names migrated
from @app/* to @hyperframes/*.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrub internal codenames and stale references from OSS port

- Replace static.heygen.ai runtime URLs in test fixtures
- Remove internal CDN publish script (publish-hyperframe-runtime.ts)
- Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime
  with neutral names (studio, hyperframe-runtime, __hyperframeRuntime)
- Fix stale Vault API / localhost references in docs
- Remove broken deprecated_studio link

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove remaining internal codenames and stale references

- Delete stale producer README.md and PIPELINE.md (referenced nonexistent files)
- Replace "Cerberus" codename with "HyperFrames" in test design reviews
- Replace magic-edit postMessage identifiers with hf-preview/hf-parent
- Rename debug-magic-edit-timeline.ts to debug-timeline.ts
- Replace "Motion Cut" with "HyperFrames" in Timeline comments
- Fix studio/CLI references to nonexistent archive package
  (use local data/projects/ dir, stub render proxy)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 22:43:56 -07:00