## Problem
Closes#555. Studio users could inspect the preview, but there was no first-class way to capture the current rendered frame as an image.
## What this fixes
- Adds a `Capture` action to the Studio header toolbar so it does not cover the video preview.
- Downloads the current composition frame as a PNG using the current player time.
- Extends the existing thumbnail route and Studio/CLI thumbnail generators with an explicit PNG format path while preserving JPEG thumbnails for existing previews.
- Adds URL/filename utility coverage plus thumbnail route coverage for PNG requests.
## Root cause
Studio already had frame thumbnail generation, but the API path was JPEG-oriented and the editor UI only used it for previews. There was no current-frame capture affordance wired to the player state.
## Verification
### Local
- `bun run --filter @hyperframes/core test src/studio-api/routes/thumbnail.test.ts`
- `bun run --filter @hyperframes/studio test src/utils/frameCapture.test.ts src/player/components/PlayerControls.test.ts`
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/core typecheck`
- `bun run --filter @hyperframes/cli typecheck`
- `bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/types.ts packages/studio/src/App.tsx packages/studio/src/icons/SystemIcons.tsx packages/studio/vite.config.ts packages/studio/src/utils/frameCapture.ts packages/studio/src/utils/frameCapture.test.ts`
- `bunx oxfmt --check packages/cli/src/server/studioServer.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/types.ts packages/studio/src/App.tsx packages/studio/src/icons/SystemIcons.tsx packages/studio/vite.config.ts packages/studio/src/utils/frameCapture.ts packages/studio/src/utils/frameCapture.test.ts`
- `git diff --check`
### Browser
<img width="1027" height="910" alt="image" src="https://github.com/user-attachments/assets/71973af4-0279-4074-9
<img width="1026" height="902" alt="Screenshot 2026-04-29 at 16 17 22" src="https://github.com/user-attachments/assets/a32e1c19-b793-40b9-82f8-de8bbb11f123" />
060-130839a2d419" />
## Summary
- **Root cause**: `shouldFallbackToScreenshotAfterCalibrationError` only matched `HeadlessExperimental.beginFrame` errors. When a composition with many heavy videos (e.g. 7 videos with sparse keyframes) caused Chrome to be unresponsive in BeginFrame mode during calibration, a `Runtime.callFunctionOn timed out` or `Runtime.evaluate timed out` error was treated as an opaque failure — not a BeginFrame-mode signal. The render kept BeginFrame mode, spawned 3 workers with `captureCostMultiplier=8`, and all 3 workers also timed out initialising their sessions (0 frames captured, render fails).
- **Fix**: Add `Runtime.callFunctionOn timed out` and `Runtime.evaluate timed out` to the screenshot-fallback pattern. Any CDP call timing out during the short-timeout calibration probe now routes the render into single-worker screenshot mode — the safe fallback already used for explicit BeginFrame timeouts.
- **Result**: Compositions that overwhelm BeginFrame mode (reported in #566: 7 videos, 8 audios, 330-second render) now fall back cleanly and complete instead of failing with 0 frames.
## Test plan
- [x] New unit test: `falls back to screenshot mode after Runtime.callFunctionOn timeout during calibration` — asserts both `Runtime.callFunctionOn timed out` and `Runtime.evaluate timed out` return `true`
- [x] All existing `capture calibration safeguards` unit tests still pass
- [x] Pre-commit hooks (lint, format, typecheck) pass
Fixes#566
When an <audio> element referenced a file with no audio stream (e.g. a
silent screen-recording used as an audio src, or a video-only clip),
extractAudioMetadata threw "[FFmpeg] No audio stream found". The error
propagated uncaught through Promise.all in compileHtmlFile and crashed
the entire render.
Apply the same graceful-skip pattern already used for missing files and
failed downloads: catch the probe error and return { duration: 0 } so
the element is excluded from the composition without aborting the render.
Confirmed via 7 production HyperframeRenderWorkflow failures all sharing
the same TemporalMagicEditActivity.RENDER_PREVIEW stack trace.
## Problem
HyperFrames Studio made frame-accurate playback review slower than expected for editor-style workflows. Issue #527 called out missing loop playback, frame display/jump controls, preview-focused Space handling, frame stepping, and NLE-style J/K/L shuttle controls.
## What this fixes
- Adds a persistent Studio loop toggle and makes the playback loop restart when enabled.
- Adds a time/frame display toggle plus a jump-to-frame input in the player controls.
- Adds frame math helpers and frame-step behavior at the Studio preview frame rate.
- Expands keyboard handling so preview-focused Space toggles playback, ArrowLeft/ArrowRight step frames, Shift+Arrow steps 10 frames, and J/K/L shuttle controls work from the preview/timeline surface while ignoring form/button/slider targets.
- Adds J/K/L shuttle behavior: J plays backward, K pauses, L plays forward, repeated J/L ramps 1x -> 2x -> 4x, and K-held J/L frame-steps.
- Makes the preview wrapper focusable so keyboard playback shortcuts work after focusing the preview area.
## Root cause
The Studio playback layer only exposed mouse scrubbing, basic play/pause, a seconds-based readout, and slider-local arrow-key nudges. The global Space shortcut was also gated to `document.body`, so it stopped working once the actual preview/editor surface had focus. Studio needed a single playback-control layer above the runtime adapter that could translate editor keyboard intent into deterministic seek/play/pause operations.
## Verification
### Local checks
- `bun install`
- `bun run --filter @hyperframes/core build:hyperframes-runtime`
- `bunx oxfmt --check packages/studio/src/player/lib/time.ts packages/studio/src/player/lib/time.test.ts packages/studio/src/player/store/playerStore.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/components/PlayerControls.tsx packages/studio/src/player/components/PlayerControls.test.ts packages/studio/src/components/nle/NLEPreview.tsx`
- `bunx oxlint packages/studio/src/player/lib/time.ts packages/studio/src/player/lib/time.test.ts packages/studio/src/player/store/playerStore.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/components/PlayerControls.tsx packages/studio/src/player/components/PlayerControls.test.ts packages/studio/src/components/nle/NLEPreview.tsx`
- `bun run --filter @hyperframes/studio test -- src/player/lib/time.test.ts src/player/store/playerStore.test.ts src/player/components/PlayerControls.test.ts src/player/hooks/useTimelinePlayer.test.ts` -> 4 files passed, 52 tests passed
- `bun run --filter @hyperframes/studio typecheck`
- `bun run --filter @hyperframes/studio build`
- `git diff --check`
- Lefthook during commit -> lint, format, typecheck, commitlint pass
### Browser verification
- Created a temp project at `/tmp/hf-studio-nle-controls` with an animated 10s GSAP timeline.
- Started local Studio preview via `bun run --filter @hyperframes/cli dev -- preview /tmp/hf-studio-nle-controls` at `http://localhost:5194`.
- Used `agent-browser` to verify:
- loop toggle changes to active state
- frame display shows `current / total` frames
- jump-to-frame input moves the seek position to frame 45 / frame 150
- focused preview accepts Space play/pause
- ArrowRight advances one frame from preview focus
- J plays backward from frame 150 to a lower frame, then K stops
- agent-browser-driven recording of the tested flow completed
## Notes
- Local proof artifacts are intentionally not committed:
- `qa-artifacts/studio-nle-controls/frame-controls.png`
- `qa-artifacts/studio-nle-controls/playback-controls.webm`
- Closes#527.
Use loadRuntimeSource() from runtimeSource.ts instead of a single
hardcoded path. Tries: build from source (dev), inlined constant
(production), pre-built artifact (fallback). Fixes snapshot in dev
mode where __dirname is src/commands/ not dist/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(hdr): filter zero-opacity elements and support overflow:hidden clip rects in HDR compositor
Two bugs in the HDR render pipeline:
1. Child data-start elements inside a parent with opacity:0 were still
composited as independent layers, painting over content in later scenes.
Fix: filter elements with effective opacity 0 before groupIntoLayers().
2. CSS overflow:hidden on ancestor elements was ignored for HDR video layers,
causing videos inside clipped containers (e.g. split-screen halves) to
render full-frame. Fix: add clipRect to ElementStackingInfo, compute it
from ancestor overflow:hidden in queryElementStacking(), and crop the
source buffer to clip bounds before blitting in blitHdrVideoLayer().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): move opacity filter into blit loop to preserve hide-list correctness
The previous approach filtered zero-opacity elements before groupIntoLayers(),
which broke the DOM screenshot hide-list — invisible video elements' <img>
replacements weren't properly hidden from sibling layer screenshots, causing
the vignelli-stacking regression.
Fix: keep all elements in groupIntoLayers() for correct hide-list generation.
Skip zero-opacity HDR elements only during the actual blit step with an early
`continue` in the compositing loop.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): route identity-matrix HDR elements through region blit for clip rect support
parseTransformMatrix returns a valid matrix even for untransformed HDR
elements (Chrome reports matrix(1,0,0,1,0,0)). This made the affine blit
path always run, bypassing the region blit path which is the only one that
applies clip rects from overflow:hidden ancestors.
Fix: detect identity matrices and route them through the region path so
the cropRgb48le clip logic is reachable for split-screen layouts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(hdr): handle translation-only matrices for clip rect support
The previous isIdentity check only caught matrix(1,0,0,1,0,0). Elements
with layout translation (e.g. right-half split at left:960px reporting
matrix(1,0,0,1,960,0)) still routed through the affine path where clip
rects are not applied.
Fix: check for translation-only matrices (scale=1, rotation=0, any tx/ty)
and route those through the region blit path. el.x/el.y from
getBoundingClientRect already include the translation, so the region path
handles positioning correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(render): auto-detect HDR from media probes, add --sdr flag
Replace the --hdr opt-in model with automatic detection. When no flags
are passed, the renderer probes all video/image sources and enables HDR
output if any HDR color space is detected. Existing --hdr flag becomes
a force override. New --sdr flag forces SDR output.
Behavior matrix:
(no flags) + HDR content → HDR output
(no flags) + SDR content → SDR output
--hdr → force HDR (defaults to HLG if no HDR sources)
--sdr → force SDR (skips probing)
--hdr --sdr → error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat(render): auto-detect HDR from media probes, add --sdr flag"
This reverts commit 69fb52196f.
* chore(hdr): simplify review fixes — remove redundant guard, add image clip warning
- Remove redundant viewportMatrix.length >= 6 check (parseTransformMatrix
always returns 6-element array or null)
- Add clip rect warning log to blitHdrImageLayer for parity with video path
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(render): auto-detect HDR from media probes, add --sdr flag
Replace the --hdr opt-in model with automatic detection. When no flags
are passed, the renderer probes all video/image sources and enables HDR
output if any HDR color space is detected. Existing --hdr flag becomes
a force override. New --sdr flag forces SDR output.
Behavior matrix:
(no flags) + HDR content → HDR output
(no flags) + SDR content → SDR output
--hdr → force HDR (defaults to HLG if no HDR sources)
--sdr → force SDR (skips probing)
--hdr --sdr → error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: align HDR auto-detect docs and tests
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Per Miguel's review: the previous fixture had no body / root background, so it
passed against both the buggy and fixed code. The fix this PR makes (the
initTransparentBackground stylesheet injection in initializeSession) only
matters when a composition paints over the CDP default-background-color
override — exactly what we tell users not to do, but exactly what a
regression test must do.
Reproduced locally:
- base SHA (2935be6): pixel (10,10) decodes as rgba [16,16,16,255]
(opaque #111 body bg leaks through the pre-navigation override that
Chrome resets on goto)
- this head: pixel (10,10) decodes as rgba [0,0,0,0]
(initTransparentBackground injects [data-composition-id]{background:transparent !important}
AFTER navigation, force-overriding the body bg)
The pixel-level assertions in transparency-test.ts are unchanged — they
already require alpha=0 at (10,10). With the body bg painted, that
assertion now fails on any code path that doesn't actually preserve alpha
end-to-end.
Extends RenderConfig.format with "png-sequence" and patches two correctness
gaps so the existing "webm" / "mov" values actually preserve the alpha
channel end-to-end.
Engine fixes:
- screenshotService.pageScreenshotCapture: drop optimizeForSpeed for PNG
captures. The fast path uses an alpha-unaware codec that crushes real
alpha values; kept for opaque jpeg captures where it is harmless.
- frameCapture: replace the inline setDefaultBackgroundColorOverride
block (which fired pre-navigation and was reset by page.goto) with a
proper initTransparentBackground() call inside initializeSession,
after the window.__hf readiness poll. This also injects the
html/body/[data-composition-id]{background:transparent !important}
stylesheet so compositions with custom body / #root backgrounds do not
defeat the override. Wired into both screenshot-mode and beginframe-mode
branches.
Producer:
- RenderConfig.format extended to "mp4" | "webm" | "mov" | "png-sequence"
with full JSDoc.
- Streaming encode is bypassed for png-sequence (frames go straight to
disk). FORMAT_EXT extended.
- New Stage-5 png-sequence branch: mkdir outputPath, copy captured PNGs as
frame_NNNNNN.png, copy audio.aac sidecar when audio is present.
- Stage-6 mux/faststart and the debug copy are wrapped in !isPngSequence.
- README.md: new "Transparent Video Output" section.
Tests:
- New fixture tests/transparency-regression/ tagged "transparency".
- New tsx script src/transparency-test.ts asserts pixel-level alpha for
webm + png-sequence outputs. Wired as "test:transparency".
- Default "test" / "test:update" scripts pass --exclude-tags transparency
so the golden-MP4 harness ignores the new fixture.
Verified locally on macOS arm64: typecheck clean across engine + producer,
producer renderOrchestrator vitest 10/10, transparency-test passes for
both webm and png-sequence with end-to-end pixel assertions.
The snapshot command resolved the HyperFrames runtime IIFE via a
relative path that walked up three directories to packages/core/dist/.
This only worked in the monorepo dev layout — npm/npx installs have
a flat dist/ folder with cli.js and the runtime side by side.
Without the runtime, window.__player was never created and the
snapshot fell back to seeking every __timelines entry to the same
absolute time. Sub-composition timelines expect relative time
(offset from their data-start), so all beats rendered beat-1 content.
Fix: resolve("hyperframe.runtime.iife.js") from __dirname (the dist/
folder itself), where the build already copies the runtime IIFE.
Miguel Ángel