mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
9588a6dc06660c0eede215e21e85a421a83e5723
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 alexcraviotto
|
894f6e38f1 | feat(studio): server-side waveform generation with caching | ||
|
Miguel Ángel
|
970b446c49 |
feat(studio): drag assets from the sidebar onto the timeline (#464)
## Problem Studio still broke down in three concrete authoring flows around timeline assets: - you could import media into Assets, but not drag an already-imported asset from the Assets tab onto the timeline and persist it into source - dragging a file from outside the app onto the timeline only uploaded it into Assets instead of placing it at the dropped time/track - once a clip was on the timeline, there was no reliable keyboard delete flow for removing it safely from source While implementing direct external drops, another real bug showed up: - valid binary uploads like `raycast.mp4` from `Downloads` were being rejected as unsupported media in Studio dev because the Vite API bridge was corrupting multipart request bodies before they reached the upload route ## What this fixes ### Timeline asset placement from inside Studio - asset cards in the Assets tab are draggable - the timeline accepts asset drops even when it already has clips - dropping an asset onto the timeline inserts a new clip into the active composition source at the dropped time / track - asset paths are rewritten relative to the target composition file so drops into sub-compositions resolve correctly - the new clip is persisted immediately and the preview refreshes ### Direct external file drops onto the timeline - dropping a file from outside the app onto the timeline now uploads it and places it onto the dropped track/time in one shot - it no longer stops halfway by only adding the file into Assets - multiple dropped files are placed using the same drop start and successive tracks ### Delete key support - selected timeline clips can now be deleted with `Delete` / `Backspace` - deletion is persisted back to source, not just removed from local state - the delete path now uses a server-side DOM mutation helper with LinkeDOM for structural safety instead of client-side string surgery ### Binary upload fix for media files - the Studio Vite API bridge now forwards non-GET request bodies as raw bytes instead of decoding them as UTF-8 text - that preserves multipart uploads for binary media like MP4s - valid local videos from `Downloads` no longer get rejected as `Unsupported media skipped` just because the dev bridge corrupted the request body - upload validation now probes buffered media through a temp file path that preserves the file extension before saving into the project ## Root cause There were really two separate gaps: ### 1. Asset placement / deletion workflow gaps The timeline and asset systems already existed, but they were disconnected: - `AssetsTab` only supported copy/import flows - `Timeline` only handled raw file import, not positioned placement for existing assets - there was no utility layer for converting a dropped asset into persisted timeline HTML - there was no structurally safe deletion path for arbitrary selected timeline clips ### 2. Binary upload corruption in Studio dev The Studio Vite API bridge rebuilt non-GET request bodies like this: - read each request chunk - call `chunk.toString()` - concatenate into a string - construct the Fetch `Request` from that string body That works for text, but it corrupts multipart binary uploads. By the time the upload route wrote the received file and ran `ffprobe`, otherwise valid MP4s had already been mangled in-flight. ## Behavior - dropping on `index.html` inserts the asset into the root composition - dropping while drilled into a composition inserts into that composition file instead - drop X position maps to `data-start` - drop Y position maps to the current visible track row, with a new bottom track created if the drop lands below existing rows - images default to a short finite duration - audio/video default to their metadata duration when available, with a fallback duration if metadata cannot be read quickly - pressing `Delete` on a selected clip removes that clip from the underlying HTML source and clears selection in Studio - valid uploaded MP4s now survive the Studio dev API bridge intact instead of being rejected during upload validation ## Verification ### Local checks - `bunx oxlint packages/core/src/studio-api/helpers/sourceMutation.ts packages/core/src/studio-api/helpers/sourceMutation.test.ts packages/core/src/studio-api/helpers/mediaValidation.ts packages/core/src/studio-api/helpers/mediaValidation.test.ts packages/core/src/studio-api/routes/files.ts packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/sidebar/AssetsTab.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/Timeline.test.ts packages/studio/src/utils/timelineAssetDrop.ts packages/studio/src/utils/timelineAssetDrop.test.ts packages/studio/vite.config.ts packages/studio/vite.request-body.ts packages/studio/vite.request-body.test.ts` - `bunx oxfmt --check` on the touched files - `bun run --filter @hyperframes/core typecheck` - `bun run --filter @hyperframes/studio typecheck` - `bun test packages/core/src/studio-api/helpers/sourceMutation.test.ts packages/core/src/studio-api/helpers/mediaValidation.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/utils/timelineAssetDrop.test.ts packages/studio/vite.request-body.test.ts` ### Browser / live verification Verified against a live local Studio fixture: - dragging an existing asset from the Assets tab onto the timeline creates a persisted clip at the dropped position - dropping a file from outside the app directly onto the timeline uploads it and creates a persisted clip at the dropped position - selecting a dropped clip and pressing `Delete` removes it from both the live timeline and the saved source HTML - valid MP4 uploads like `raycast.mp4` now succeed through the live Studio upload route instead of being rejected as unsupported media ## Notes - the local `timeline-trio-verify` and `timeline-overlap-debug` projects used for verification are local-only and are not part of this PR - this PR is about asset placement, upload correctness, and deletion safety; it does not broaden into richer editing workflows beyond placing/removing clips from the timeline |
||
|
Miguel Ángel
|
158204343d |
fix: stabilize studio preview and runtime sync (#389)
## Summary Stabilize the Studio preview/runtime path so timeline data, preview rendering, and thumbnails stay in sync. This PR includes: - preview hot-refresh without remounting the iframe - runtime duration/timeline fixes so Studio stops drifting from playback state - thumbnail and selector-based preview fixes - local Studio runtime serving and player-resolution fixes so dev/CI do not depend on prebuilt player artifacts - tests around preview identity and thumbnail/runtime behavior ## Why This PR Exists This is the foundation layer for timeline editing. Without it, the editor was prone to: - iframe remount flashes after saves - duration mismatches between preview and timeline - stale or incorrect thumbnails - CI/test failures when `@hyperframes/player` artifacts were not prebuilt ## Verification - `bun run --filter @hyperframes/studio test` - `bun run --filter @hyperframes/studio typecheck` - `bun run --filter @hyperframes/core typecheck` - `bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts` - `bunx oxfmt --check packages/cli/src/server/studioServer.ts packages/core/src/runtime/timeline.ts packages/core/src/runtime/timeline.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/types.ts packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/nle/NLEPreview.tsx packages/studio/src/components/nle/NLEPreview.test.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/Player.tsx packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/store/playerStore.ts packages/studio/vite.config.ts` ## Stack - base of stack - followed by `feat: add studio timeline editing` - followed by `fix: smooth scrubber end seeking` |
||
|
Miguel Ángel
|
43e9252065 |
feat: add MOV (ProRes 4444) as transparent video output format (#224)
## Summary - Adds `--format mov` to the render CLI for ProRes 4444 transparent video output - ProRes 4444 with alpha is the industry standard for transparent video overlays, supported by CapCut, Final Cut, Premiere, DaVinci, and After Effects - WebM VP9 alpha technically works but is ignored by all major video editors — only browsers decode it - Adds MOV to the studio export dropdown alongside MP4 and WebM ## Transparency format comparison | Format | Codec | Alpha | Video editors | Browsers | File size | | --- | --- | --- | --- | --- | --- | | **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No (won't play in browser) | Large (~5-40 MB) | | **WebM** | VP9 | Yes | None (shows black) | Chrome, Firefox | Small (~200 KB) | | **MP4** | H.264 | No | All | All | Small | > **Note:** ProRes MOV files do not play in Chromium browsers — they are an intermediate/editing format, not a delivery format. Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify transparency works correctly. ## Changes - **CLI**: Add `mov` to `--format` validation, examples, and output path logic - **Engine**: `getEncoderPreset()` returns ProRes 4444 (`yuva444p10le`) for `mov` format; handle `.mov` in `applyFaststart` and `muxVideoWithAudio`; add `pix_fmt` to streaming encoder ProRes path - **Producer**: Treat `mov` like `webm` for alpha capture (PNG frames, screenshot mode, `forceScreenshot`) - **Studio**: Add MOV option to export format dropdown and render queue hook - **Core**: Add `mov` to studio API types, render route, and mime helpers - **Tests**: Add encoder preset tests for mov format (42 total, all passing) ## Usage ```bash hyperframes render --format mov --output overlay.mov ``` ## Test plan - [x] `pnpm build` passes - [x] `pnpm --filter @hyperframes/engine test` — 42 tests pass (2 new for MOV) - [x] `oxlint` and `oxfmt` clean on all 12 changed files - [x] End-to-end local render produces ProRes 4444 (`yuva444p12le`) with working alpha - [x] Docker render with `--format mov` — ProRes 4444 confirmed via ffprobe - [x] Studio dropdown shows MOV option in built JS - [x] Transparency verified with [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) |
||
|
Miguel Ángel
|
e2c8ed5d83 |
refactor(core): replace cheerio with linkedom to drop deprecated whatwg-encoding (#187)
## Summary - `cheerio` pulls `encoding-sniffer` → `whatwg-encoding@3.1.1` (deprecated), causing a warning on every `npm install -g hyperframes` - `linkedom` was already bundled into the CLI via tsup `noExternal` and has zero deprecated transitive deps - Rewrote `htmlBundler.ts` and `subComposition.ts` to use standard DOM APIs via `linkedom` - Added a `parseHTMLContent` helper that wraps HTML fragments in a full document structure (required for `linkedom` to populate `document.body`) - Removed `cheerio` from `cli` dependencies and tsup `external` list - Replaced `cheerio` with `linkedom` in `core` `optionalDependencies` ## Test plan - [x] All 411 tests pass (`bun run test` in `packages/core`) - [x] Full monorepo build succeeds (`bun run build`) - [x] TypeScript typecheck passes |
||
|
Miguel Ángel
|
1681350ac4 |
fix(preview): rewrite sub-composition asset urls in styles (#174)
## Summary
- rewrite CSS `url(...)` asset paths from sub-compositions before styles are hoisted into bundled/master preview output
- rewrite standalone sub-composition preview HTML so `src`/`href` paths keep resolving correctly under the preview root `<base>`
- add regression tests for bundled CSS asset rewriting and standalone sub-composition preview rewriting
## Root cause
Standalone composition previews reused the project `<head>` with a preview-root `<base href="/api/projects/:id/preview/">`, but the sub-composition body still contained `../...` asset references. Those escaped the preview route and 404ed. Separately, bundled preview already rewrote `<img src="../...">` paths but left hoisted CSS asset references like `@font-face src: url("../font.woff2")` untouched.
## Validation
- `pnpm --filter @hyperframes/core exec vitest run src/compiler/htmlBundler.test.ts src/studio-api/helpers/subComposition.test.ts`
- `pnpm --filter @hyperframes/core exec tsc --noEmit`
- `pnpm --filter @hyperframes/producer exec tsc --noEmit` *(blocked by pre-existing `packages/producer/src/services/deterministicFonts.ts` importing missing generated file `./fontData.generated.js` in the worktree install)*
- browser verification with `agent-browser` against the local preview server using `/Users/miguel07code/dev/test-hyperframes/heygen-promo`
## Browser proof
Verified the fixed preview routes in-browser after seeking to visible frames:
- standalone composition preview
- bundled master preview
|
||
|
Miguel Ángel
|
5f3488e996 |
feat(studio): drag-drop assetfile/folder and asset import anywhere in the studio (#155)
## Summary - Add `/api/projects/:id/upload` endpoint for multipart file uploads with automatic dedup naming - Wire `onImportFiles` from Assets tab "Import media" button through to the upload API - Add global drag-drop overlay — drop media files **anywhere** in the studio, not just the Assets panel - Files that already exist get `(2)`, `(3)` suffixes instead of overwriting - Support uploading into subdirectories via `?dir=` param — dropping on a folder imports there - Make folders draggable in the file tree + support drop-to-root - Add `bodyLimit` middleware for early rejection of oversized payloads - Surface skipped/failed uploads via toast notification instead of console-only Addresses feedback: _"Wish I could upload/drag-drop assets directly in the Studio (music, images, video) like a CapCut media panel"_ ## Test plan - [x] Open studio, drag an image/video/audio file onto any part of the UI - [x] Verify the drop overlay appears with "Drop files to import" message - [x] Drop the file — verify it appears in the Assets tab and file tree - [x] Drop a file with the same name — verify it gets a `(2)` suffix - [x] Click "Import media" button in Assets tab — verify file picker works - [x] Import multiple files at once via drag-drop - [x] Drop a file onto a nested folder in the file tree — verify it lands in that folder - [x] Drag a folder in the file tree and drop it on another folder or root — verify it moves - [x] Drop a file >500MB — verify toast notification appears - [x] Verify drag overlay doesn't get stuck when dragging over nested UI elements |
||
|
Miguel Ángel
|
ecb590d444 |
feat(studio): full IDE-like file management (#147)
## Summary - **API**: POST (create), DELETE (delete), PATCH (rename/move), POST duplicate endpoints with null-byte sanitization - **FileTree**: right-click context menu with New File, New Folder, Rename, Duplicate, Delete - **Drag-and-drop**: move files between folders with visual feedback and subtree guard - **Inline editing**: rename/create inputs with filename validation - **Header actions**: quick New File / New Folder buttons in the FILES header Stacks on top of `feat/studio-code-quality`. ## Test plan - [x] Right-click file → Rename, Delete, Duplicate all work - [x] Right-click folder → New File, New Folder, Delete work - [x] Drag file from one folder to another - [x] Create file with invalid name (`../foo`, `a/b`) → rejected client-side - [x] Delete currently-edited file → editor clears - [x] Studio build succeeds |
||
|
Miguel Ángel
|
dac304ed9f |
refactor(studio): code quality — 22 findings, dead code removal, App.tsx split (#144)
## Summary Full code quality review of the studio package, fixing 22 of 25 findings. Removes dead code, extracts modules from App.tsx, fixes accessibility and performance issues. ## Critical fixes (3) - **`aria-valuenow`** on seek bar now updates imperatively via `liveTime.subscribe` — screen readers previously always reported position 0 - **Speed menu** closes on outside click (was permanently stuck open) - **RenderQueue auto-scroll** moved from render phase to `useEffect` (was violating React render purity via `queueMicrotask` during render) ## Dead code removed (-331 lines) | File | Lines | Why dead | |---|---|---| | `PreviewPanel.tsx` | 180 | Replaced by NLELayout + NLEPreview | | `useCodeEditor.ts` | 80 | Exported but never imported | | `formatTick` alias | 2 | Deprecated, unused | | `onClipChange` prop | 5 | Declared, never used | | `trackH` prop | 5 | Declared, never used | | `editRange*` + updaters in store | 60 | Never read or written | ## App.tsx extraction | Extracted to | Lines | What | |---|---|---| | `components/LintModal.tsx` | 130 | Lint results modal + LintFinding type | | `components/MediaPreview.tsx` | 75 | Image/video/audio/font file previewer | | `utils/mediaTypes.ts` | 15 | Shared regex constants (App.tsx and AssetsTab.tsx had diverged copies) | ## Performance fixes - `useMemo` for `compositions`/`assets` derivation from `fileTree` - `useMemo` for `buildTree(files)` in FileTree - Debounced `handleContentChange` PUT (600ms — was firing on every keystroke) - CompositionsTab iframe hover debounced (300ms — was mounting immediately) - `VideoFrameThumbnail` re-extracts frame when `src` prop changes ## Not addressed (3 — low priority) - #6: SystemIcons consolidation (large refactor across many files) - #16-17: Overlay dismiss pattern standardization - #18: Inline SVG → Phosphor replacement (gradual, per-PR) 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Miguel Ángel
|
1230657ed0 |
fix(studio,runtime,engine,compiler): 8 bug fixes — audio, render, timeline, Lottie, thumbnails, video render (#133)
## Summary
**Original 5 bugs fixed:**
- **Bug 1 — Audio silent after seek**: Added `Accept-Ranges` / `Content-Length` + `206 Partial Content` to the static asset server for byte-range seeking.
- **Bug 2 — Download 404 after restart**: Render list endpoint now registers on-disk renders into the in-memory job map.
- **Bug 3 — Timeline stops at GSAP end**: `resolveRootTimelineFromDocument` pads the GSAP timeline to match `data-duration` when the composition declares longer.
- **Bug 4 — Render stuck at 0%**: Store `jobState` reference (not spread copy) so async progress mutations reach the SSE stream.
- **Bug 5 — Lottie missing in preview/render**: Two fixes — (a) moved Lottie adapter before GSAP so `onUpdate` wins; (b) fixed bundler silently dropping external CDN `\<script src>` tags from sub-compositions (root cause: `$content(s).html()` returns `""` for external scripts).
**3 additional bugs fixed:**
- **Bug 6 — Blank thumbnails outside monorepo**: Implemented `generateThumbnail` in the CLI adapter using Puppeteer.
- **Bug 7 — Video empty in rendered sub-compositions**: Fixed `parseVideoElements` selector from `video[id][src]` to `video[src][data-start]` + auto-assign IDs.
- **Render errors**: Failed renders now show their error message in the renders panel.
## Commits
| Commit | Description |
| --- | --- |
| `3951c6f` | fix(studio): store render job reference instead of snapshot copy |
| `f331c30` | fix(studio): make previously-completed renders downloadable after restart |
| `a5e2d04` | fix(studio): add range request support for audio/video seeking in preview |
| `f24317a` | fix(runtime): pad GSAP timeline to data-duration when composition declares longer duration |
| `7cf38ca` | fix(runtime): fix Lottie adapter conflicting with GSAP-driven animations |
| `bc99209` | fix(studio): surface render error messages in the renders panel |
| `8fc9e8b` | fix(cli): implement generateThumbnail in studio adapter |
| `90277ea` | fix(engine): render videos inside sub-compositions that lack an explicit id |
| `f5bb579` | fix(compiler): preserve external CDN scripts from sub-compositions in bundle |
## Test plan
- [x] `golden-lyric-video`: seek → audio plays from seeked position
- [x] Any project: render → progress advances past 0%, reaches 100%
- [x] Any project: complete render, restart `hyperframes dev`, Download → works
- [x] `intro-vid`: play → runs full 5s (not stopping at 3s)
- [x] `hyperframe-build-up-demo`: play → rocket Lottie visible during 0-2s ✅ verified
- [x] Outside monorepo: Compositions sidebar shows thumbnail images (not blank)
- [x] `bug.zip` project: render → video in polaroid sub-composition appears in output
- [x] Trigger a failed render → error message shown
|
||
|
Miguel Ángel
|
bd175b64a6 |
refactor(core): extract shared studio API module (#113)
## Summary Extracts all studio API routes into a shared Hono-based module at `@hyperframes/core/studio-api`. ### Architecture - **`StudioApiAdapter` interface** — consumers inject host-specific behavior (project resolution, bundling, rendering, thumbnails) - **Shared route modules**: projects, files, preview, lint, render, thumbnail - **Shared helpers**: `isSafePath`, `walkDir`, `getMimeType`, `buildSubCompositionHtml` ### What this PR does - Creates the shared module with all API routes extracted from both `vite.config.ts` and `studioServer.ts` - Both consumers will be refactored in follow-up commits to mount this module with their own adapter ### What stays in each consumer - **Vite**: SSR module loading, Puppeteer thumbnails, file watcher + HMR, producer HTTP proxy, multi-project scanning - **CLI**: in-process `executeRenderJob`, local runtime serving, browser management, SPA static file serving ### Follow-up needed - [ ] Refactor `packages/studio/vite.config.ts` to use `createStudioApi(adapter)` via `@hono/node-server`'s `getRequestListener` - [ ] Refactor `packages/cli/src/server/studioServer.ts` to use `createStudioApi(adapter)` - [ ] Add `./studio-api` export path to `packages/core/package.json` - [ ] Add `hono` as peer dependency of `@hyperframes/core` ## Test plan - [ ] Verify shared module compiles without type errors - [ ] After consumer refactoring: all studio features work identically via both vite dev and CLI embedded servers 🤖 Generated with [Claude Code](https://claude.com/claude-code) |