mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 17:30:50 +00:00
c609850b419e56289a4a257c5bc97142091f90ba
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
c609850b41 |
fix(engine): real back-pressure in StreamingEncoder.writeFrame (#1372)
writeFrame returned the stdin.write boolean synchronously; when FFmpeg encoded slower than workers captured, Node's writable buffer grew without bound (multi-worker worst case ~80GB over a 1h render) until the kernel OOM-killed the process. writeFrame is now async: a buffered write awaits the drain event before resolving, so back-pressure propagates through the frame reorder buffer to the capture loops and in-flight frames stay bounded. Inactivity-timer semantics are preserved: no reset before drain, so a hung FFmpeg still trips SIGTERM. The drain wait races one-shot drain/close listeners (aborted in a finally) rather than chaining onto the shared exit promise — V8 retains reaction-list entries on unsettled promises, so per-frame .then chains would accumulate ~108K closures over a 1h back-pressured render. An exit-status re-check after listener attachment closes the close-before-attach hang window. All five writeFrame call sites (streaming stage and HDR loops) check the result via a shared ensureFrameWritten guard and stop the render with a frame-indexed error when the encoder is gone instead of discarding the boolean. The MULTI_WORKER_MAX_DURATION_SECONDS cap can be relaxed in a follow-up now that buffering is bounded. Fixes #1353 |
||
|
Miguel Ángel
|
0e052e42d2 | fix(engine): support AMD AMF GPU encoding | ||
|
terencecho
|
efc16a945f |
fix(engine): treat ffmpegStreamingTimeout as per-frame inactivity, not total render time (#901)
## Summary - Convert `streamingEncoder.ts`'s safety timer from a total-render hard cap to a per-frame inactivity timeout - Reset the timer only on `accepted === true` writes — buffered writes don't count as consumer progress - Update the `ffmpegStreamingTimeout` config doc to reflect the new semantics ## The bug The timer was set once at spawn and fired SIGTERM unconditionally at `ffmpegStreamingTimeout` ms — turning a "FFmpeg is hung" guard into a hard cap on total render duration. Slow-but-progressing captures (CI runner under load, large compositions, slower compositor paths after [#838](https://github.com/heygen-com/hyperframes/pull/838)'s always-clip change) regularly exceeded the 600s default and were killed mid-encode. The symptom surfaced as: ``` Streaming encode failed: FFmpeg exited with code 255 video:NNNkB audio:0kB ... [libx264 @ ...] frame I:3 Avg QP:12.91 size: 73263 [libx264 @ ...] frame P:431 Avg QP:14.72 size: 31633 ... [libx264 @ ...] kb/s:7661.05 Exiting normally, received signal 15. ``` libx264 had encoded most frames cleanly; SIGTERM arrived during the encode, libx264 printed its end-of-encode stats, and Node observed a non-zero exit. The `audio:0kB` in stderr is incidental — `streamingEncoder` is video-only; audio is muxed later in `assembleStage`. Downstream reproduction: `style-13-prod` fails deterministically in `heygen-com/hyperframes-internal` CI after bumping `@hyperframes/producer` from 0.6.7 → 0.6.10. Bisects to #838 widening the SDR capture path at dpr=1 — same composition shape, slower per-frame, total render now crosses 600s. ## The fix Convert the timer to a heartbeat: each `writeFrame` that goes through to the kernel pipe (i.e. `stdin.write` returns `true`) resets it. Only true hangs (no successful frame write for the timeout window) trip SIGTERM now; "slow but progressing" renders are unbounded. Crucially, the heartbeat does **not** reset on `accepted === false`. A `false` return means Node had to buffer the write because FFmpeg hasn't drained the pipe yet — that's not proof of consumer progress, just proof we produced. Without this distinction, a hung FFmpeg with a live Chrome would queue frames into Node's writable buffer indefinitely (no backpressure path back to the capture loop) and grow until OOM. In steady state with a slow-but-alive FFmpeg, writes alternate between `true` and `false` as the buffer drains and refills; the `true`s are enough to keep the heartbeat ticking. Renames are intentionally avoided — `ffmpegStreamingTimeout` keeps its name and `600_000` default; only the semantics changed. The config doc spells out the new behavior so downstream consumers know what 600s now means. ## Test plan - [x] **Slow-but-progressing capture** (`accepted=true`): 9× `writeFrame` at 900ms intervals (under the 1000ms threshold) — encoder stays alive through 8.1s. Stall past the threshold — SIGTERM fires. - [x] **Stalled FFmpeg with live producer** (`accepted=false`): override `stdin.write` to return false; pump 9× `writeFrame` at 900ms intervals. SIGTERM still fires inside the 1000ms window — buffered writes don't keep the heartbeat alive. - [x] Existing 33 tests in `streamingEncoder.test.ts` still pass - [x] Lint (`oxlint`) + format (`oxfmt --check`) clean - [ ] CI regression suite 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Theodor Kleynhans
|
5dcc89c930 |
feat(cli): accept ffmpeg-style rational fps (NTSC, PAL, slow-mo)
Replaces the rigid `--fps 24|30|60` whitelist with a numeric range and
adds support for ffmpeg-style fractional framerates so NTSC stays exact
end-to-end.
- `--fps 30` keeps working (integer fps)
- `--fps 30000/1001` now means exact NTSC 29.97 (not the lossy decimal)
- `--fps 24000/1001`, `--fps 60000/1001`, `--fps 25/50/120/240` all work
- Decimals like `--fps 29.97` are rejected with a friendly error pointing
the user at the rational form, since `29.97` and `30000/1001` round
to different framerates inside ffmpeg
Carries an `Fps = { num: number; den: number }` rational end-to-end:
RenderConfig, EncoderOptions, StreamingEncoderOptions, CaptureOptions,
DockerRenderOptions, Studio API request body, regression-harness
meta.json. The `-r` and `-framerate` ffmpeg args emit the rational form
verbatim (`30000/1001`) so no decimal round-trip happens at the encoder
boundary. Frame-interval math uses `1000 * den / num` ms (33.366… for
NTSC, 33.333… for integer 30).
Helpers live in @hyperframes/core:
- `parseFps(input: string | number): FpsParseResult` — discriminated
parser used by both the CLI and the Studio API route
- `fpsToFfmpegArg(fps: Fps): string` — emits "30" or "30000/1001"
- `fpsToNumber(fps: Fps): number` — for arithmetic (telemetry, frame
count, frame-index → time)
Studio API wire format accepts polymorphic `fps: number | string`:
- number → integer fps (`30`)
- string → rational (`"30000/1001"`)
Decimals are rejected; matches the same rule as the CLI.
Existing meta.json fixtures with integer `"fps": 30` continue to load
unchanged — the regression-harness validator now normalizes both number
and string inputs through `parseFps`.
|
||
|
Vance Ingalls
|
2b25023e10 |
test(engine): cover spawnStreamingEncoder lifecycle and cleanup paths (#380)
## Summary Add unit tests that mock `child_process.spawn` to drive an in-memory "ffmpeg" through the success/failure paths used by the producer's HDR encoder and by Chunk 5A's defensive `close()` in `renderOrchestrator`. ## Why `Chunk 9D` of `plans/hdr-followups.md`. The contracts the orchestrator's try/finally cleanup (Chunk 5A) and the abort path rely on were entirely uncovered. A regression in `spawnStreamingEncoder`'s lifecycle handling would surface as a leaked ffmpeg process or a hung render, both of which are hard to diagnose after the fact. ## What changed `packages/engine/src/services/streamingEncoder.test.ts`: 7 new specs covering: - Successful exit after explicit `close()`. - Non-zero exit before `close()` returns a failure result (no throw). - `ENOENT` on spawn returns a failure result (no throw). - Abort signal triggers `SIGTERM` and a `"cancelled"` result. - `close()` is idempotent and never throws on a second call. - `writeFrame` returns `false` after the encoder has exited. - `close()` detaches the abort listener so post-close aborts don't re-kill ffmpeg. These contracts are what the `renderOrchestrator` try/finally cleanup added in Chunk 5A relies on, and what the ffprobe-unavailable test (Chunk 9B) hinted at for the encoder side. ## Test plan - [x] All new specs pass. - [x] No production code changes — pure regression coverage of existing lifecycle behavior. ## Stack Chunk 9D of `plans/hdr-followups.md`. Test-only change, complements Chunk 5A. |
||
|
roiizchakandroi32
|
c11a332ef8 |
test(engine): cover h265 NVENC in preset-mapping regression tests (#443)
hevc_nvenc uses the same p1..p7 preset vocabulary as h264_nvenc, so the mapping in `mapPresetForGpuEncoder` applies to both codecs. The initial regression suite only covered `codec: "h264"`, which left a gap: a future refactor that split the H.264 and H.265 NVENC paths could silently regress one codec without any test catching it. Add three-case loops (ultrafast → p1, medium → p4, veryslow → p7) under `codec: "h265"` to both `buildEncoderArgs` and `buildStreamingArgs` test blocks. Each case also asserts that `-c:v hevc_nvenc` is selected so the test fails loudly if the codec plumbing is broken, not just the preset translation. Follow-up to #442 per review comment from @jrusso1020. Co-authored-by: roi32 <75878108+roi32@users.noreply.github.com> |
||
|
roiizchakandroi32
|
3b8de7a5eb |
fix(engine): accept libx264 preset names with NVENC and QSV (#442)
NVENC rejects the libx264 preset vocabulary (ultrafast / medium / slow /
...) with AVERROR(EINVAL) ("Error applying encoder options: Invalid
argument"), which surfaces as a bare `FFmpeg exited with code -22` from
spawn(). Because ENCODER_PRESETS passes these names straight through to
h264_nvenc / hevc_nvenc, every `--gpu` render using the `draft` tier
failed; `standard` (medium) and `high` (slow) only worked coincidentally
on ffmpeg builds that happened to accept those aliases. QSV has the same
problem on a narrower set (ultrafast / superfast / placebo).
Add `mapPresetForGpuEncoder` in utils/gpuEncoder.ts that translates the
libx264 vocabulary to each encoder's native names:
- nvenc: libx264 -> p1..p7 (already-native pN values pass through);
unknown values fall back to p4 (medium)
- qsv: ultrafast / superfast -> veryfast; placebo -> veryslow;
everything else passes through
- videotoolbox / vaapi / null: unchanged
Both buildEncoderArgs (chunkEncoder.ts) and buildStreamingArgs
(streamingEncoder.ts) now route through the helper before pushing
`-preset` to the ffmpeg arg vector.
To make the next encoder-options failure diagnosable without re-running
ffmpeg by hand, \`formatFfmpegError\` in utils/runFfmpeg.ts now appends
the last 15 non-empty stderr lines to the error string. The four call
sites that previously swallowed stderr (encodeFramesFromDir,
muxVideoWithAudio, applyFaststart, and the streaming encoder exit
handler) have been updated.
Tested end-to-end on an RTX 4080 with ffmpeg 8.1 NVENC across
\`--quality draft|standard|high\` plus \`--video-bitrate\` and \`--crf\`
overrides; the 6 renders were visually equivalent to the CPU baseline.
Co-authored-by: roi32 <75878108+roi32@users.noreply.github.com>
|
||
|
James Russo
|
7b0c7e73b2 |
refactor: frame reorder buffer + port probe cleanup; add CREDITS.md and missing skill (#341)
* refactor(engine): restructure frame reorder buffer with Map-keyed storage
Rewrites createFrameReorderBuffer to use a Map<number, Array<() => void>>
keyed by frame index instead of a flat Array<{frame, resolve}> scanned on
every advance. O(1) lookups in enqueue/flush, fast-paths for the matching-
cursor and overshoot cases, and a small fix: waitForAllDone now coexists
with the writer still waiting on the final frame instead of colliding on
the same waiter slot.
Also adds 5 unit tests (there were none before) covering the fast-path,
out-of-order gating, multi-waiter-per-frame semantics, waitForAllDone
normal path, and the overshoot case.
Comment tweaks on buildChromeArgs — the flag profile is the standard
headless-for-capture set (Puppeteer / Playwright / Chrome headless-shell
all converge on similar flags); rephrased for clarity.
* refactor(cli): simplify port availability probe with async/await
Rewrites isPortAvailableOnHost from a single new-Promise callback into an
async/await form with an intermediate `bindError: ErrnoException | null`
variable. Makes the bind-then-release flow explicit as two sequential
awaits, and broadens the non-EADDRINUSE errno commentary (EADDRNOTAVAIL
for disabled IPv6, EACCES for privileged ports, EAFNOSUPPORT for missing
address families — all treated as "this host doesn't apply", not "port
occupied").
No behavior change to existing callers; all four portUtils tests still
pass.
* docs: add CREDITS.md and surface website-to-hyperframes skill
- New CREDITS.md acknowledging prior art in the browser-based video
rendering space (Remotion) and the ecosystem HyperFrames builds on
(Puppeteer, FFmpeg, GSAP, Hono). Standard OSS practice.
- Adds the `website-to-hyperframes` skill to the skills tables in
README.md, docs/guides/prompting.mdx, and the project template at
packages/cli/src/templates/_shared/CLAUDE.md. The skill ships in
skills/ but was missing from every table.
- Adds `/hyperframes-registry` to the prose mention in the repo
CLAUDE.md.
|
||
|
Vance Ingalls
|
5a3fde19d4 |
feat(engine): add HDR video output pipeline (#265)
## Summary Adds the ability to render HDR video output (H.265 10-bit, BT.2020) from HyperFrames compositions. When the renderer detects HDR source video, it automatically switches to the HDR output pipeline — no flags needed. ## What it does - **Auto-detection** — Probes each video source with `ffprobe`. If any has bt2020/PQ/HLG color metadata, the output switches to H.265 10-bit with correct color tags. SDR-only compositions are unaffected (H.264, bt709). - **HLG pass-through** — Native HLG pixels from FFmpeg extraction are piped directly to the encoder without conversion. This avoids brightness loss from HLG→linear→PQ conversion (which requires an OOTF system gamma we can't reliably apply). - **Encoder HDR support** — Both chunk and streaming encoders accept HDR presets: `libx265`, `yuv420p10le`, BT.2020 color primaries, `hvc1` codec tag (required for Apple playback). - **WebGPU HDR capture (gated)** — A complete WebGPU float16 readback pipeline is implemented and tested but gated behind headed Chrome (headless doesn't expose WebGPU). Ready for future use with WebGPU canvas content. - **HDR utilities** — `detectTransfer()` (PQ vs HLG), `getHdrEncoderColorParams()`, `analyzeCompositionHdr()`. 15 unit tests. ## Key design decisions | Decision | Why | |----------|-----| | No `--hdr` flag | SDR content encoded as HDR causes orange shift in browsers. Auto-detect eliminates this. | | HLG pass-through (not HLG→PQ) | Conversion loses brightness without OOTF. Pass-through matches source exactly. | | `hvc1` codec tag | Apple QuickTime requires `hvc1` (not `hev1`) for HEVC playback. | | 1-hour streaming timeout | HDR capture at ~6fps needs more time than the default 10-minute FFmpeg timeout. | ## Files changed | File | What changed | |------|-------------| | `packages/engine/src/utils/hdr.ts` | **NEW** — HDR detection, transfer types, encoder params (15 tests) | | `packages/engine/src/services/hdrCapture.ts` | **NEW** — WebGPU readback, HLG conversion, PQ encode | | `packages/engine/src/services/streamingEncoder.ts` | HDR presets, raw rgb48le input, color tags | | `packages/engine/src/services/chunkEncoder.ts` | HDR presets, conditional color tags | | `packages/producer/src/services/renderOrchestrator.ts` | Auto-detection loop, HDR pass-through capture path | ## How to test Render a composition with an HDR video source. The output should be H.265 10-bit with HDR metadata visible in `ffprobe` (bt2020, arib-std-b67 or smpte2084). Plays correctly in QuickTime and on HDR displays. ## Stack position **2 of 6** — Stacked on #258 (SDR/HDR normalization). Provides the encoder infrastructure that phases 1-5 build on. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |