mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
v0.4.10
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
78de791392 |
fix(studio): render in-process, remove producer server dependency (#235)
## Summary - The Vite dev server proxied studio renders to a separate producer server (port 9847) that needed to be started manually - When the producer wasn't running, renders silently failed — red dot, no error message, no way to know what went wrong - Replaced the proxy with direct in-process rendering via `@hyperframes/producer` — same code path as the CLI and embedded preview mode - Removed ~70 lines of SSE proxy streaming code, replaced with the same ~20-line in-process pattern used everywhere else ## DX improvement **Before:** `pnpm dev` + `npx tsx packages/producer/src/public-server.ts` (two terminals, easy to forget) **After:** `pnpm dev` (renders work immediately) ## Testing Verified manually: open studio via `pnpm dev`, navigate to a project, click Export — renders complete with live progress updates, no separate server needed. |
||
|
Miguel Ángel
|
43e9252065 |
feat: add MOV (ProRes 4444) as transparent video output format (#224)
## Summary - Adds `--format mov` to the render CLI for ProRes 4444 transparent video output - ProRes 4444 with alpha is the industry standard for transparent video overlays, supported by CapCut, Final Cut, Premiere, DaVinci, and After Effects - WebM VP9 alpha technically works but is ignored by all major video editors — only browsers decode it - Adds MOV to the studio export dropdown alongside MP4 and WebM ## Transparency format comparison | Format | Codec | Alpha | Video editors | Browsers | File size | | --- | --- | --- | --- | --- | --- | | **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No (won't play in browser) | Large (~5-40 MB) | | **WebM** | VP9 | Yes | None (shows black) | Chrome, Firefox | Small (~200 KB) | | **MP4** | H.264 | No | All | All | Small | > **Note:** ProRes MOV files do not play in Chromium browsers — they are an intermediate/editing format, not a delivery format. Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify transparency works correctly. ## Changes - **CLI**: Add `mov` to `--format` validation, examples, and output path logic - **Engine**: `getEncoderPreset()` returns ProRes 4444 (`yuva444p10le`) for `mov` format; handle `.mov` in `applyFaststart` and `muxVideoWithAudio`; add `pix_fmt` to streaming encoder ProRes path - **Producer**: Treat `mov` like `webm` for alpha capture (PNG frames, screenshot mode, `forceScreenshot`) - **Studio**: Add MOV option to export format dropdown and render queue hook - **Core**: Add `mov` to studio API types, render route, and mime helpers - **Tests**: Add encoder preset tests for mov format (42 total, all passing) ## Usage ```bash hyperframes render --format mov --output overlay.mov ``` ## Test plan - [x] `pnpm build` passes - [x] `pnpm --filter @hyperframes/engine test` — 42 tests pass (2 new for MOV) - [x] `oxlint` and `oxfmt` clean on all 12 changed files - [x] End-to-end local render produces ProRes 4444 (`yuva444p12le`) with working alpha - [x] Docker render with `--format mov` — ProRes 4444 confirmed via ffprobe - [x] Studio dropdown shows MOV option in built JS - [x] Transparency verified with [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) |
||
|
Vance Ingalls
|
d36c1785b9 |
feat(captions): energy-based technique selection and mandatory quality checks (#176)
## Summary - Rewrite script-to-style mapping as an energy detection table (high → low) with mandatory animation requirements: karaoke baseline, 2+ highlight techniques, kinetic exits - Replace `tl.call()` per-frame audio-reactive pattern with group-level GSAP tweens — read peak bass/treble for each group's time range and modulate entrance intensity at build time, no per-frame callbacks needed - Add transcript quality check with automatic retry rules (>20% music tokens = retry with larger model) - Add caption word structure lint rule (`.caption-group` + `<span>`) for studio editor compatibility - Add multilingual model guidance and decision tree for model selection ## Test plan - [ ] Skill files render correctly as markdown - [ ] Cross-references between SKILL.md, dynamic-techniques.md, and transcript-guide.md resolve correctly - [ ] `dynamic-techniques.md` audio-reactive section uses `tl.to()`/`tl.set()` only, no `tl.call()` loops 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Miguel Ángel
|
b11edbf800 |
refactor(studio): wire vite.config.ts to shared studio API module (#115)
## Summary - Replaces ~850 lines of inline route handlers in `vite.config.ts` with the shared `createStudioApi(adapter)` module - Implements `StudioApiAdapter` for the Vite dev server context (SSR-loaded bundler/linter, producer HTTP proxy, Puppeteer thumbnails) - Bridges Hono `fetch()` to Vite's Connect middleware with streaming support for SSE Now **both** consumers (CLI + studio) use the same shared API module, ensuring feature parity. ## Test plan - [x] `pnpm --filter @hyperframes/studio dev` starts correctly - [x] Home page shows project grid with thumbnails - [x] Preview plays with correct fonts/animations - [x] Sub-composition drill-down works - [x] Lint modal shows findings - [x] File read/write works in code editor - [x] Render queue works (requires producer server) 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Miguel Ángel
|
95d7dc0623 |
fix(cli): align render output naming and add WebM support to studioServer (#109)
## Summary - CLI render: use timestamped filenames (`project_date_time.ext`) matching the studio's naming convention, preventing overwrites of previous renders - studioServer: read `fps`/`quality`/`format` from POST body instead of hardcoding `fps:30`/`quality:standard`/`mp4` - studioServer: use timestamped job IDs matching the studio pattern - studioServer: fix download endpoint to serve correct content-type for WebM ## Test plan - [x] `hyperframes render --format webm` outputs timestamped WebM file - [x] `hyperframes render` outputs timestamped MP4 (no overwrite) - [x] Studio embedded server (`hyperframes dev`) renders with correct format when selected in UI - [x] Download endpoint serves correct MIME type for WebM renders |
||
|
Miguel Ángel
|
40bd159103 |
feat(studio): render queue, layout restructure, home page, hover preview (#95)
## Summary - Add render queue panel with progress tracking, download, and delete actions - Restructure App layout: home page with project picker, session-based routing - Add ExpandOnHover component for preview-on-hover interactions (uses motion/react) - CompositionsTab now supports hover preview with expanded iframe view - Vite config: guard setInterval cleanup to dev-only (fixes CI build timeout) - Add favicon and update studio package deps ## Test plan - [x] Render queue shows progress, completes, and allows download - [x] Home page lists projects and navigates to session view - [x] ExpandOnHover shows expanded preview on mouse hover with spring animation - [x] `vite build` exits cleanly (no hanging process from setInterval) 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Miguel Ángel
|
8c1ae77697 |
refactor(studio): update layout, config, and remove agent activity tracking (#64)
## Summary - **NLELayout**: Add toolbar slot, composition breadcrumb navigation, improved responsive layout - **Vite config**: Add full project API (preview, thumbnail, render, file CRUD) for standalone dev mode - Remove AgentActivityTrack component (replaced by timeline clips) - Add HTML editor utilities for composition source editing - Guard setInterval cleanup to dev-only to prevent `vite build` from hanging in CI 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance IngallsandClaude Opus 4.6
|
f4367d5726 |
feat(cli): add whisper transcription and template improvements (#53)
* feat(cli): add whisper transcription to init flow New modules: - whisper/manager.ts: download/cache whisper.cpp binary + model (~/.cache/hyperframes/whisper/) - whisper/transcribe.ts: extract audio, run whisper, save transcript.json Init flow changes: - "Got a video or audio file?" now accepts audio-only files (mp3, wav, m4a) - "Generate captions from audio?" prompt after file selection - Transcription produces transcript.json in project root - Graceful fallback if whisper/ffmpeg unavailable Supports: macOS ARM64/x86, Linux x86_64. Downloads whisper.cpp v1.7.3 from GitHub releases and ggml-base.en model from Hugging Face. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): use brew/system whisper instead of downloading binaries whisper.cpp doesn't ship pre-built macOS/Linux CLI binaries. Use brew install whisper-cpp on macOS (auto-installs if brew available), system PATH lookup otherwise. Model still downloaded from Hugging Face. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): simplify whisper install — detect or instruct, don't build Remove build-from-source complexity. If whisper-cpp is found on PATH, use it. If not, show install instructions instead of blocking: "To generate captions, install whisper-cpp: brew install whisper-cpp" The transcription prompt only appears when whisper is available. When it's not, the user sees the install command and can re-run init. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): auto-install whisper via brew or build from source ensureWhisper() now tries 4 strategies in order: 1. System PATH (whisper-cli or whisper already installed) 2. Homebrew (macOS: brew install whisper-cpp) 3. Build from source (git clone + cmake, ~30-60s) 4. Show install instructions as last resort Init flow always asks "Generate captions?" — whisper is installed automatically in the background if needed. No user intervention required on macOS with Xcode CLI tools or any system with git+cmake. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add window.__timelines guard to all templates The studio bundler doesn't always initialize window.__timelines before template scripts run, causing "Cannot set properties of undefined" errors. Add defensive guard to every template. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): patch template captions with actual transcript data After scaffolding, if transcript.json exists, replace the hardcoded word array in the template's captions composition with the real transcript data. The template's caption animation and styling are preserved — only the word data changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): show install notice when whisper needs to be installed When whisper-cpp isn't found, show an info message before the spinner: "whisper-cpp not found — installing automatically..." Then the spinner shows "Installing whisper-cpp (this may take a moment)..." Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add muted and playsinline to all template video elements The framework requires video elements to have muted and playsinline attributes. All four templates were missing these, causing video to not play in the studio preview. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): flat asset structure + separate audio tracks in templates Assets: video, images, fonts all go at project root (not assets/ or fonts/ subdirectories). The studio preview can't resolve relative paths from subdirectories due to the /preview URL suffix. Audio: added <audio> elements alongside muted <video> in all 4 templates so the video's audio plays back. The framework requires muted video + separate audio element. Removed assets/ and fonts/ directory creation from scaffoldProject. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): inject base tag for asset resolution in preview The preview iframe serves bundled HTML from /api/projects/:id/preview but relative asset paths (video.mp4, font.woff2) resolve to the wrong URL without a <base> tag. Now injects <base href="/api/projects/:id/preview/"> so relative paths route through the static asset handler. Also adds proper MIME types for video, audio, image, and font files served from the preview asset route. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): serve HyperFrames runtime in dev mode The preview runtime script had an empty src — the framework never loaded, so video playback and clip lifecycle didn't work. Now auto-detects packages/cli/dist/hyperframe-runtime.js and serves it at /api/runtime.js. No env var needed in dev mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): filter whisper special tokens from transcript Use --output-json instead of --output-json-full to avoid special tokens like [_TT_485] and [BLANK_AUDIO]. Also filter remaining bracket tokens when building the word array for captions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): use --output-json-full for word-level timestamps --output-json only produces segment-level timing (no tokens). --output-json-full is required for word-level timestamps that the captions template needs. Special tokens are filtered out by the patchTranscript function. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): patch template durations to match uploaded video Templates now use __VIDEO_DURATION__ placeholder that gets replaced with the actual probed video duration. All data-duration values on the root composition, video, audio, and caption clips are updated. Without a video, defaults to 10 seconds. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): merge punctuation tokens with preceding word Whisper outputs punctuation (. , ! ?) as separate tokens. These appeared as standalone words in captions, sometimes in the wrong group. Now merged with the preceding word during transcript normalization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): match both TRANSCRIPT and script variable names in templates Three templates use `const TRANSCRIPT = [...]` while warm-grain uses `const script = [...]`. The patchTranscript function now matches both. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): security and template fixes - Replace shell injection risk (execSync rm) with unlinkSync in transcribe.ts - Add GIT_TERMINAL_PROMPT=0 to whisper buildFromSource git clone - Fix hardcoded data-duration="18" in warm-grain captions template - Add data-start="0" to root compositions in swiss-grid, vignelli, warm-grain - Add data-start="0" to warm-grain grain-overlay composition - Deduplicate hasFFmpeg: remove from init.ts, import from whisper/manager.ts - Add my-video/ and packages/studio/data/ to .gitignore Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): format warm-grain captions and fix TS nullability errors - Format warm-grain/compositions/captions.html - Add optional chaining on token.offsets (may be undefined) - Use intermediate variable for lastWord to satisfy TS strict checks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): add blank template option, smart defaults for video vs audio - Blank template: minimal scaffolding (root composition, video, audio, GSAP timeline) with __VIDEO_SRC__ and __VIDEO_DURATION__ placeholders - Template defaults: video uploads default to "blank" (user brings their own content), audio-only defaults to "warm-grain" (motion graphics template since there's no video to show) - Audio-only projects now tracked with isAudioOnly flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): address whisper review feedback - Clean stale builds: if BUILD_DIR exists but no binary, nuke and retry - Build failures clean up BUILD_DIR so next attempt starts fresh - patchTranscript regex scoped within <script> blocks to prevent matching across block boundaries - Removed hardcoded model size hint (~148MB) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add missing rmSync import to whisper manager Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove test project and lock file Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): address review items 7-12 — execFileSync, build diagnostics, WAV verification - manager.ts: replace all execSync with execFileSync to prevent command injection - manager.ts: capture cmake stderr and include in build failure error message - transcribe.ts: verify WAV is 16kHz mono via ffprobe before passing to whisper - init.ts: replace fragile JSON formatting with JSON.stringify(words, null, 2) - init.ts: fix default duration from "10" to "5" matching DEFAULT_META - init.ts: add probeAudioDuration() and --audio/--skip-transcribe flags - init.ts: extract finalizeProject() to reduce code path duplication - init.ts: wire transcription into non-interactive path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
20be2ea1c2 |
style: apply oxfmt baseline formatting across all source files (#25)
## Summary - Run `oxfmt .` across the entire codebase to establish formatted baseline - 299 files changed — mechanical formatting only, no logic changes - Double quotes, semicolons, 2-space indent, trailing commas, 100 print width Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits) ## Test plan - [x] `pnpm format:check` — all 426 files pass - [x] `pnpm -r typecheck` — all packages pass - [x] `pnpm build` — all packages build - [x] All 348 tests pass |
||
|
Vance Ingalls
|
323ff8f860 |
fix: resolve oxlint errors across codebase (#24)
## Summary - Remove 5 unused `beforeEach` imports from test files - Remove unused imports (`existsSync`, `TimelineCompositionElement`) - Remove unused destructured variables (`options`, `width`, `height`, `goldenEl`) - Remove dead `formatDuration` function - Fix unused catch parameters (`catch (err)` → `catch`) - Prefix unused `renderError` state with `_` - Add `eslint-disable-next-line` for 2 React exhaustive-deps false positives (stable ref + zustand setter) Part 2/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits) ## Test plan - [x] `pnpm lint` — 0 errors on 193 files - [x] All 348 tests pass (core + engine) |
||
|
Vance IngallsandClaude Opus 4.6
|
9f8e5ba5a1 |
initial code (#2)
* feat: initial code port from hyperframes-internal Port all OSS-ready packages from the internal monorepo: - @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime - @hyperframes/cli — CLI for creating, previewing, and rendering compositions - @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg) - @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg) - @hyperframes/ui-player — browser-based video player component - @hyperframes/studio — composition editor (React frontend + Hono backend) Includes regression test suite with Docker-based test harness. All HeyGen-internal references, deployment infrastructure, and proprietary assets have been removed. Package names migrated from @app/* to @hyperframes/*. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scrub internal codenames and stale references from OSS port - Replace static.heygen.ai runtime URLs in test fixtures - Remove internal CDN publish script (publish-hyperframe-runtime.ts) - Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime with neutral names (studio, hyperframe-runtime, __hyperframeRuntime) - Fix stale Vault API / localhost references in docs - Remove broken deprecated_studio link Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining internal codenames and stale references - Delete stale producer README.md and PIPELINE.md (referenced nonexistent files) - Replace "Cerberus" codename with "HyperFrames" in test design reviews - Replace magic-edit postMessage identifiers with hf-preview/hf-parent - Rename debug-magic-edit-timeline.ts to debug-timeline.ts - Replace "Motion Cut" with "HyperFrames" in Timeline comments - Fix studio/CLI references to nonexistent archive package (use local data/projects/ dir, stub render proxy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |