mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 17:30:50 +00:00
fdb8f33fc099f0c7ba5ef3dfb033def561234e92
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
5917c0382d |
fix(producer): pass resolved engine config through every encode path (#1371)
encodeFramesFromDir was called with 5 of its 6 args, dropping the config param — the encode timeout always fell back to the hardcoded 600s default and FFMPEG_ENCODE_TIMEOUT_MS was silently ignored, so any encode over 600s wall time was deterministically SIGTERM-killed. Resolve the engine config once in the encode stage and pass it to the non-chunked, chunked, and GIF paths. The chunked per-chunk encodes previously had no timeout at all; they now honor the same config value. Review follow-ups: the chunked path's final concat spawn gains the same config-driven timeout (it previously had none); every encode-timeout kill now appends 'FFmpeg killed after exceeding ffmpegEncodeTimeout (N ms)' to the failure instead of surfacing a bare exit-255; and the orchestrator threads its already-resolved config into the encode stage via an optional EncodeStageInput.engineConfig field (direct callers and distributed chunks keep the producerConfig ?? resolveConfig() fallback). The encode-timeout tests run against a mocked child_process spawn with fake timers, removing the real-ffmpeg dependency that made the previous default-timeout test environment-fragile in CI. Fixes #1348 |
||
|
Miguel Ángel
|
0e052e42d2 | fix(engine): support AMD AMF GPU encoding | ||
|
James RussoandClaude Opus 4.7
|
5d264e146c |
docs(lambda): document webm support + simplify-review fixes (#953)
* docs(lambda): document webm support in distributed mode PR 8.4 of the WebM distributed-rendering plan (v1.5 backlog #1; see DISTRIBUTED-RENDERING-PLAN.md §7.2). User-facing docs catch up with the shipped capability. Updates docs/deploy/migrating-to-hyperframes-lambda.mdx: - "Output format" row in the migration table now lists `webm` alongside mp4 / mov / png-sequence with a note that webm uses libvpx-vp9 + closed-GOP concat-copy. HDR mp4 remains the only refused format. - "No webm distributed" caveat replaced with "webm uses closed-GOP VP9" explainer covering the encoder args (`-g <chunkSize>`, `-keyint_min <chunkSize>`, `-auto-alt-ref 0`, `-cpu-used 2`), why alt-ref disable is load-bearing, and that the output preserves alpha via yuva420p with Opus audio. - Migration checklist no longer asks adopters to filter out webm compositions; only HDR-dependent renders need to stay on the previous framework. aws-lambda.mdx doesn't currently call out webm as unsupported (only HDR in the v1 surface list), so it gets no copy edits beyond the migration guide. The internal planning doc (DISTRIBUTED-RENDERING-PLAN.md §7.2, §8, §12 — kept outside the repo) gets matching updates: format support matrix flipped ✓, v1.5 backlog #1 marked shipped, HDR promoted to the new top item, and the rev-12 → rev-13 status line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: address simplify-review findings on webm stack Folds in cleanups identified by a multi-agent code-review pass over the 4-PR webm-distributed stack: - plan.ts: `resolveEncoderTriple()` webm case now calls `getEncoderPreset(quality, "webm")` for its preset string instead of hardcoding "good". The hardcode was wrong for `quality: "draft"` (`getEncoderPreset` returns "realtime" for that tier) — would have silently overridden the draft → realtime mapping for distributed webm renders. - chunkEncoder.ts: trim the new VP9 closed-GOP comment block from ~18 lines of WHY narration down to the 6 lines that actually explain why (alt-ref + cpu-used drift). Match the alpha branch's idempotent-push comment to the same standard. - chunkEncoder.test.ts: drop the duplicate WHY comment that restated the implementation comment in plain words. - webm-concat-copy.test.ts: rewrite the file-header docstring to describe the contract being tested instead of the PR-8.1-gating history; strip "PR 8.2 / Path A / Path B" references from error messages (they belong in PR bodies, not in test output). Consolidate the yuva420p alpha smoke into a single `it()` block (was a full 4-test describe with duplicated setup) — the yuv420p block already covers the probe/decode/frame-count contract; the alpha smoke only needs to prove the alpha args don't break concat-copy. - plan.test.ts: drop the "PR 8.1 proved the contract" comment. - webm-vp9 fixture: drop the aspirational "Other webm-with-audio fixtures cover the mux path separately when added" sentence (no other fixtures exist). Regenerated the baseline via `docker:test:update webm-vp9` to reflect the updated comment. - migrating-to-hyperframes-lambda.mdx: add a paragraph about distributed webm's perf cost — ~10-25% larger files at constant CRF due to forced keyframes, and slower per-chunk encode due to `-cpu-used 2` being more conservative than the libvpx default. All unit tests + the webm-vp9 distributed-simulated regression still pass after these changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): accept --format=webm in `hyperframes lambda render` The CLI's `lambda render` subcommand's FORMATS allowlist and the `RenderArgs.format` type still narrowed to `mp4 | mov | png-sequence`, so even though the producer + aws-lambda packages now support webm end-to-end, the CLI surface rejected it with `--format must be mp4|mov| png-sequence`. Add webm to both spots and update the --help description. Surfaced during real-AWS deploy prep — the local lambda-local / distributed-simulated tests didn't go through the CLI so the gap went unnoticed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(producer): font cache writes to /tmp on Lambda (read-only \$HOME) The deterministic Google Fonts cache was rooted at `\$HOME/.cache/hyperframes/fonts`, which fails on AWS Lambda — the runtime's `\$HOME` resolves to a `/home/sbx_*` directory tree that's read-only. `mkdirSync(..., { recursive: true })` can't create that path and the plan stage trips with `ENOENT: no such file or directory, mkdir '/home/sbx_user1051/.cache/hyperframes/fonts/space-mono'` on every Lambda render that pulls a Google Font (i.e. every distributed fixture using `@import url("https://fonts.googleapis.com/...")`). Detect Lambda via `\$AWS_LAMBDA_FUNCTION_NAME` and route the cache to `tmpdir()/hyperframes/fonts` in that case. Lambda's `/tmp` survives across invocations on a warm container, so cache hit rate is the same as non-Lambda runs. Also honor an explicit `\$HYPERFRAMES_FONT_CACHE_DIR` override for adopters who want a different location regardless of the runtime. Surfaced while verifying webm distributed end-to-end on real AWS — the same bug affects mp4 fixtures using Google Fonts; webm just happened to be the one I tried first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract DistributedFormat type + trim font-cache resolver Second simplify-review pass on the webm stack flagged two cleanups: 1. **`DistributedFormat` type duplicated 10 times.** Every file in the distributed pipeline carried its own copy of `"mp4" | "mov" | "png-sequence" | "webm"` — adding a new format meant a 10-place edit with no compile-time guarantee they stayed in sync. Extract a single source of truth in `packages/producer/src/services/distributed/shared.ts`, re-export from `@hyperframes/producer/distributed` and `@hyperframes/aws-lambda/sdk`, and have all callers pull from there. The aws-lambda `ALLOWED_FORMATS` runtime tuple and the CLI's `FORMATS` tuple now both use `satisfies readonly DistributedFormat[]` so the compiler enforces the runtime allowlist stays in sync with the type. 2. **`deterministicFonts.ts` font-cache resolver was over-commented.** Trim the 7-line block to 4 lines (drop the aspirational "and other read-only-FS execution environments" — only Lambda is detected — and the warm-container `/tmp` persistence narration — anyone reading already knows Lambda /tmp semantics). Collapse the two-step `if (explicit && explicit.length > 0)` into a single nullish-coalesce expression now that the empty-string defensive check is gone (`process.env.X` is `string | undefined`, no third shape to guard against). Out-of-scope skips (called out by the agents, deferred): - In-process `RenderConfig.format` and the in-process CLI's `render.ts` format union still carry their own inline copies. The union happens to coincide today but they're separate concerns — leaving them alone limits this PR's blast radius. - `fontCacheDir(slug)` / `resolveFontCacheRoot()` naming asymmetry flagged as taste; skipping. - Pre-existing redundant `existsSync` before `mkdirSync({ recursive: true })` in `fontCacheDir` — out of scope. All tests + typecheck still pass. Lambda render still works end-to-end (no functional changes). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(lambda): drop plan-doc reference from migration checklist PR review feedback: source/docs should not mention the distributed-rendering planning doc. Tighten the migration checklist sentence to describe the webm path directly rather than referencing the doc's version label. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(producer): split resolveEncoderTriple into mp4 + non-mp4 helpers CI Fallow audit on PR #953 flagged `resolveEncoderTriple` at CRAP 31.6 — the function interleaved (a) mp4 codec validation + dispatch, (b) the non-mp4 codec-rejection throw, and (c) per-format dispatch. Splitting into `resolveMp4EncoderTriple` + `resolveNonMp4EncoderTriple` drops the top-level function's cyclomatic complexity below the threshold while preserving every error message and code path. Behavior unchanged. Also extracts an `EncoderTriple` type alias so the three functions share the return shape declaratively rather than repeating it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
James Russo
|
07de7e61ed |
feat(engine): closed-GOP VP9 encoder args + concat-copy smoke test (#950)
## Description PR 1 of 4 in the WebM (VP9) distributed-rendering series. A gating experiment that proves closed-GOP libvpx-vp9 chunks survive `ffmpeg -f concat -c copy` losslessly, so the rest of the stack can ship Path A (concat-copy) rather than the slower re-encode-in-assemble fallback. Two changes: 1. **Closed-GOP VP9 encoder args.** `buildEncoderArgs` now lays `-g <chunkSize>`, `-keyint_min <chunkSize>`, `-auto-alt-ref 0`, and `-cpu-used 2` on libvpx-vp9 when `lockGopForChunkConcat=true`. Mirrors the existing libx264/libx265 branches. The alt-ref disable is load-bearing — libvpx-vp9's default non-displayable alt-ref frames can reach across chunk seams and break concat-copy. `-cpu-used 2` pins the speed/quality tradeoff so chunks encoded on workers with different libvpx-vp9 defaults produce visually consistent output across seams. Default (`lockGopForChunkConcat` unset) preserves the existing in-process VP9 path unchanged. 2. **Concat-copy smoke test** at `packages/producer/tests/distributed/_smoke/webm-concat-copy.test.ts`. Generates 60 PNGs via lavfi `testsrc2`, encodes them as 4 VP9 chunks of 15 frames using `buildEncoderArgs` with `lockGopForChunkConcat=true`, concat-copies via `ffmpeg -f concat -c copy`, then runs three independent verifications: `ffprobe -show_streams`, `ffmpeg -f null -` decode test, and `ffprobe -count_frames`. Each verification surfaces its failure fingerprint in the error message. Smoke test passes 6/6 locally → Path A works; the rest of the stack takes it. Also exports `buildEncoderArgs` from `@hyperframes/engine` so adapters / tests can construct args without re-implementing the contract. ## Testing - [x] `bunx vitest run --root packages/engine src/services/chunkEncoder.test.ts` — 62/62 pass (new VP9 closed-GOP tests included) - [x] `bun test packages/producer/tests/distributed/_smoke/webm-concat-copy.test.ts` — passes - [x] `bunx oxlint` + `bunx oxfmt --check` on all changed files — clean - [x] `bunx tsc --noEmit -p packages/engine/tsconfig.json` — clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
JamesandClaude Opus 4.7
|
2d6372ac2a |
feat(engine): add lockGopForChunkConcat option to buildEncoderArgs
Part of Phase 2 of the distributed rendering plan (determinism hardening). See DISTRIBUTED-RENDERING-PLAN.md §7.1 and §17.2 (gating table). Adds two optional fields to EncoderOptions: lockGopForChunkConcat?: boolean // default false gopSize?: number // required when lockGopForChunkConcat=true When the flag is true on the SW libx264 / libx265 paths, buildEncoderArgs emits closed-GOP / forced-keyframe args so the resulting chunk file can be losslessly concatenated (`ffmpeg -f concat -c copy`) with sibling chunks: -g <gopSize> -keyint_min <gopSize> -sc_threshold 0 -force_key_frames "expr:eq(mod(n,<gopSize>),0)" -x264-params "...:scenecut=0:open-gop=0:repeat-headers=1" -x265-params "keyint=<gopSize>:min-keyint=<gopSize>:scenecut=0:open-gop=0:repeat-headers=1" -bf 0 (added for h265 too when locked) GPU encoders, vp9, and prores ignore the flag (their concat-copy story is separate — see plan §7.2 / §8). In-process behavior is unchanged: the default (false) path emits no new args. New unit tests pin both branches in packages/engine/src/services/ chunkEncoder.test.ts. This is part of a stack of 10 PRs; this is PR 1 of 10. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Theodor Kleynhans
|
5dcc89c930 |
feat(cli): accept ffmpeg-style rational fps (NTSC, PAL, slow-mo)
Replaces the rigid `--fps 24|30|60` whitelist with a numeric range and
adds support for ffmpeg-style fractional framerates so NTSC stays exact
end-to-end.
- `--fps 30` keeps working (integer fps)
- `--fps 30000/1001` now means exact NTSC 29.97 (not the lossy decimal)
- `--fps 24000/1001`, `--fps 60000/1001`, `--fps 25/50/120/240` all work
- Decimals like `--fps 29.97` are rejected with a friendly error pointing
the user at the rational form, since `29.97` and `30000/1001` round
to different framerates inside ffmpeg
Carries an `Fps = { num: number; den: number }` rational end-to-end:
RenderConfig, EncoderOptions, StreamingEncoderOptions, CaptureOptions,
DockerRenderOptions, Studio API request body, regression-harness
meta.json. The `-r` and `-framerate` ffmpeg args emit the rational form
verbatim (`30000/1001`) so no decimal round-trip happens at the encoder
boundary. Frame-interval math uses `1000 * den / num` ms (33.366… for
NTSC, 33.333… for integer 30).
Helpers live in @hyperframes/core:
- `parseFps(input: string | number): FpsParseResult` — discriminated
parser used by both the CLI and the Studio API route
- `fpsToFfmpegArg(fps: Fps): string` — emits "30" or "30000/1001"
- `fpsToNumber(fps: Fps): number` — for arithmetic (telemetry, frame
count, frame-index → time)
Studio API wire format accepts polymorphic `fps: number | string`:
- number → integer fps (`30`)
- string → rational (`"30000/1001"`)
Decimals are rejected; matches the same rule as the CLI.
Existing meta.json fixtures with integer `"fps": 30` continue to load
unchanged — the regression-harness validator now normalizes both number
and string inputs through `parseFps`.
|
||
|
Miguel Ángel
|
34db66ef0a |
fix(cli): prevent esbuild runtime error in global/npx installs (#452)
* fix(cli): resolve runtime fallback for globally-installed hyperframes When hyperframes is installed globally via npm, the `loadRuntimeSourceFallback()` path that dynamically imports @hyperframes/core and runs esbuild fails because @hyperframes/core is inlined into cli.js and import.meta.url resolves to the wrong location for the entry.ts source file. Add a disk-based fallback that searches for the pre-built IIFE runtime artifact in multiple locations: - Alongside the bundled CLI (dist/hyperframe-runtime.js, dist/hyperframe.runtime.iife.js) - Walking up from __dirname through node_modules The esbuild path is tried first to preserve live-rebuild behavior in dev, with the pre-built artifact search as a safety net for the bundled context. Also adds the IIFE artifact name variant to resolveRuntimePath() in the studio server so it checks both naming conventions. * fix(cli): gate esbuild fallback on source availability The previous fix still triggered esbuild's stderr output before the catch could suppress it. Now check whether the runtime entry.ts source file actually exists before attempting the on-the-fly build, avoiding the noisy error in global installs entirely. * fix(cli): remove noisy console.warn from runtime fallback The caller already handles a null return — no need to warn about something the user can't act on. If both paths fail, the /api/runtime.js route returns a 404 which the studio handles gracefully. * style(engine): fix oxfmt trailing blank line in chunkEncoder test * fix(cli): guard against null/undefined from loadHyperframeRuntimeSource Fall through to the pre-built artifact if the function returns a falsy value without throwing. * refactor(cli): consolidate runtime source resolution into single module Replace the scattered path-probing logic with a single loadRuntimeSource() that encodes the full priority chain: esbuild from source (dev only, gated on entry.ts existence) → pre-built artifact alongside cli.js → core/dist artifact → node_modules walk. Rename loadRuntimeSourceFallback → loadRuntimeSource since it's now the primary resolution function, not a fallback. |
||
|
Vance Ingalls
|
4101cb721a |
test(shader-transitions): add midpoint (p=0.5) regression invariants for all shaders (#378)
## Summary Add four midpoint (`p=0.5`) regression invariants applied via a `describe` loop over `ALL_SHADERS`, so every existing and future shader transition automatically gets coverage at the most viewer-visible point in the animation. ## Why `Chunk 9G` of `plans/hdr-followups.md`. Existing smoke tests cover only the endpoints (`p=0 ≈ from`, `p=1 ≈ to`), which miss a class of regressions that surface specifically at the midpoint and let shaders silently rot in CI: - A shader becomes a no-op (returns input as-is) - A shader prematurely completes (returns target at midpoint) - A shader doesn't write to the output buffer at all - A shader loses determinism (`Math.random` / `Date.now` / leaked state) ## What changed `packages/engine/src/utils/shaderTransitions.test.ts`: a single `describe` loop over `ALL_SHADERS` that asserts at `p=0.5`: 1. `output ≠ from` — catches no-ops 2. `output ≠ to` — catches premature completion 3. `output` is non-zero — catches blank output 4. `output` is deterministic — catches accidental non-determinism Uses two distinct uniform input colors (40000/30000/20000 vs 10000/10000/10000) so equality checks have distinct byte patterns to compare against. Even shaders that warp UVs (which would be no-ops on uniform input alone) produce `mix16(from, to, 0.5)` at every pixel, distinct from both inputs. ## Test plan - [x] 60 new tests (4 invariants × 15 shaders), all passing. - [x] Any new transition added to the registry automatically picks up the same coverage. ## Stack Chunk 9G of `plans/hdr-followups.md`. Test-only change, independent of all other chunks. |
||
|
Vance Ingalls
|
bb9e6bdf05 |
test(engine): lock down sRGB→BT.2020 LUT with byte-exact reference values (#377)
## Summary Add a 12-row reference table covering the full sRGB range with byte-exact 16-bit HLG and PQ signal values, plus three guard tests, locking down the `buildSrgbToHdrLut()` math. ## Why `Chunk 9F` of `plans/hdr-followups.md`. The matrix-free fast path through `blitRgba8OverRgb48le` runs every DOM pixel through `buildSrgbToHdrLut()` (sRGB EOTF → linear → HDR OETF → 16-bit). Any drift in the EOTF/OETF math — constant changes, branch swaps, rounding-mode regressions — would silently corrupt every text / UI / overlay pixel composited onto an HDR frame. Existing tests covered structural invariants (transparent passthrough, opaque overwrite, alpha blending, channel symmetry, HLG ≠ PQ) but no byte-exact reference values, so a uniform scale or constant tweak could pass everything. ## What changed - 12-row reference table in `alphaBlit.test.ts` covering black, shadow, mid-grays, highlight, near-white, and white with exact 16-bit HLG and PQ signal values. - Three guard tests: - **Asymmetric R/G/B (HLG):** each channel hits the LUT independently. - **Asymmetric R/G/B (PQ):** same, on the PQ path. - **BT.2408 SDR-white invariant:** PQ caps sRGB 255 at 38055 (~203 nits), well below HLG's 65535. This is the load-bearing detail that makes PQ headroom work — locking the exact value prevents a future "fix" that would re-scale PQ to peak-at-SDR-white and clip every real HDR pixel. Reference values mirror `buildSrgbToHdrLut()` exactly and were verified against the existing HLG mid-gray comment in the file. ## Test plan - [x] All new tests pass against the current LUT. - [x] Existing `alphaBlit.test.ts` invariants unchanged. ## Stack Chunk 9F of `plans/hdr-followups.md`. Test-only change, independent of all other chunks. |
||
|
Vance Ingalls
|
3089c8ee3a |
build(lfs): track tests/*/src/*.png via Git LFS (#376)
## Summary Track `tests/*/src/*.png` via Git LFS to mirror the existing policy for golden videos and `.mp4` fixtures. ## Why `Chunk 11C` of `plans/hdr-followups.md`. Without this rule, regression suites that grow PNG fixtures over time would bloat the working-tree history and slow shallow clones. ## What changed - `.gitattributes`: add `tests/*/src/*.png` to the LFS-tracked patterns. - Migrates the six existing PNG fixtures (1.6 MB combined: `hdr-photo-pq.png` plus `heygen-promo-preview-assets/` screenshots) onto LFS in the same commit so the rule applies retroactively. ## Test plan - [x] `git lfs ls-files` includes the HDR PNG fixtures after commit. - [x] Working tree size for these files goes from 1.6 MB to 6 × ~130 B LFS pointers. ## Stack Chunk 11C of `plans/hdr-followups.md`. Independent of all code changes. |
||
|
roiizchakandroi32
|
c11a332ef8 |
test(engine): cover h265 NVENC in preset-mapping regression tests (#443)
hevc_nvenc uses the same p1..p7 preset vocabulary as h264_nvenc, so the mapping in `mapPresetForGpuEncoder` applies to both codecs. The initial regression suite only covered `codec: "h264"`, which left a gap: a future refactor that split the H.264 and H.265 NVENC paths could silently regress one codec without any test catching it. Add three-case loops (ultrafast → p1, medium → p4, veryslow → p7) under `codec: "h265"` to both `buildEncoderArgs` and `buildStreamingArgs` test blocks. Each case also asserts that `-c:v hevc_nvenc` is selected so the test fails loudly if the codec plumbing is broken, not just the preset translation. Follow-up to #442 per review comment from @jrusso1020. Co-authored-by: roi32 <75878108+roi32@users.noreply.github.com> |
||
|
roiizchakandroi32
|
3b8de7a5eb |
fix(engine): accept libx264 preset names with NVENC and QSV (#442)
NVENC rejects the libx264 preset vocabulary (ultrafast / medium / slow /
...) with AVERROR(EINVAL) ("Error applying encoder options: Invalid
argument"), which surfaces as a bare `FFmpeg exited with code -22` from
spawn(). Because ENCODER_PRESETS passes these names straight through to
h264_nvenc / hevc_nvenc, every `--gpu` render using the `draft` tier
failed; `standard` (medium) and `high` (slow) only worked coincidentally
on ffmpeg builds that happened to accept those aliases. QSV has the same
problem on a narrower set (ultrafast / superfast / placebo).
Add `mapPresetForGpuEncoder` in utils/gpuEncoder.ts that translates the
libx264 vocabulary to each encoder's native names:
- nvenc: libx264 -> p1..p7 (already-native pN values pass through);
unknown values fall back to p4 (medium)
- qsv: ultrafast / superfast -> veryfast; placebo -> veryslow;
everything else passes through
- videotoolbox / vaapi / null: unchanged
Both buildEncoderArgs (chunkEncoder.ts) and buildStreamingArgs
(streamingEncoder.ts) now route through the helper before pushing
`-preset` to the ffmpeg arg vector.
To make the next encoder-options failure diagnosable without re-running
ffmpeg by hand, \`formatFfmpegError\` in utils/runFfmpeg.ts now appends
the last 15 non-empty stderr lines to the error string. The four call
sites that previously swallowed stderr (encodeFramesFromDir,
muxVideoWithAudio, applyFaststart, and the streaming encoder exit
handler) have been updated.
Tested end-to-end on an RTX 4080 with ffmpeg 8.1 NVENC across
\`--quality draft|standard|high\` plus \`--video-bitrate\` and \`--crf\`
overrides; the 6 renders were visually equivalent to the CPU baseline.
Co-authored-by: roi32 <75878108+roi32@users.noreply.github.com>
|
||
|
Vance Ingalls
|
5256a93b2d |
feat(engine): wire options.hdr through chunkEncoder + dynamic SDR→HDR transfer (#370)
## Summary
Three independent fixes that share a common thread: HDR config flowing correctly from `EngineConfig` down through every encoder. The headline fix: disk-based HDR encodes via `chunkEncoder` were silently producing BT.709-tagged output despite `options.hdr` being set.
## Why
`Chunk 3` of `plans/hdr-followups.md`. The streaming encoder was correct but `chunkEncoder.buildEncoderArgs` hard-coded BT.709 color tags and the `bt709` VUI block in `-x265-params`, even when callers passed an HDR `EncoderOptions`. Today this is harmless because `renderOrchestrator` routes native-HDR content to `streamingEncoder` and only feeds `chunkEncoder` sRGB Chrome screenshots — but the contract was a lie, and any future caller that wired HDR through `chunkEncoder` would silently get SDR output.
## What changed
**3A — `chunkEncoder` respects `options.hdr` (BT.2020 + mastering metadata).** When `options.hdr` is set, the libx265 software path emits `bt2020nc` plus the matching transfer (`smpte2084` for PQ, `arib-std-b67` for HLG) at the codec level *and* embeds master-display + max-cll SEI in `-x265-params` via `getHdrEncoderColorParams`. libx264 still tags BT.709 inside `-x264-params` (libx264 has no HDR support) but the codec-level color flags flip so the container describes pixels truthfully. GPU H.265 (nvenc/videotoolbox/qsv/vaapi) gets the BT.2020 tags but no `-x265-params` block, so static mastering metadata is omitted — acceptable for previews, not HDR-aware delivery.
**3B — `convertSdrToHdr` accepts a target transfer.** `videoFrameExtractor.convertSdrToHdr` was hard-coded to `transfer=arib-std-b67` (HLG) regardless of the surrounding composition's dominant transfer. `extractAllVideoFrames` now calls `analyzeCompositionHdr` first, then passes the dominant transfer (`"pq"` or `"hlg"`) into `convertSdrToHdr` so an SDR clip mixed into a PQ timeline gets converted with `smpte2084`, not `arib-std-b67`.
**3C — `EngineConfig.hdr` type matches its declared shape.** The IIFE for the `hdr` field returned `undefined` when `PRODUCER_HDR_TRANSFER` wasn't `"hlg"` or `"pq"`, but the field is typed as `{ transfer: HdrTransfer } | false`. Returning `false` matches the type and avoids a downstream `undefined` check.
## Test plan
- [x] `chunkEncoder.test.ts`: replaced the previous "HDR options ignored" assertions with 8 new specs covering BT.2020 + transfer tagging, master-display/max-cll embedding, libx264 fallback behavior, GPU H.265 + HDR (tags but no x265-params), and range conversion for both SDR and HDR CPU paths.
- [x] All 313 engine unit tests pass (5 new HDR specs).
- [x] `ffprobe` an HDR composition rendered through the chunk encoder path: shows `bt2020nc` color matrix, `smpte2084` transfer, and mastering display metadata.
## Stack
Chunk 3 of `plans/hdr-followups.md`. Independent of Chunks 1/4 (touches separate code paths).
|
||
|
Vance Ingalls
|
5a3fde19d4 |
feat(engine): add HDR video output pipeline (#265)
## Summary Adds the ability to render HDR video output (H.265 10-bit, BT.2020) from HyperFrames compositions. When the renderer detects HDR source video, it automatically switches to the HDR output pipeline — no flags needed. ## What it does - **Auto-detection** — Probes each video source with `ffprobe`. If any has bt2020/PQ/HLG color metadata, the output switches to H.265 10-bit with correct color tags. SDR-only compositions are unaffected (H.264, bt709). - **HLG pass-through** — Native HLG pixels from FFmpeg extraction are piped directly to the encoder without conversion. This avoids brightness loss from HLG→linear→PQ conversion (which requires an OOTF system gamma we can't reliably apply). - **Encoder HDR support** — Both chunk and streaming encoders accept HDR presets: `libx265`, `yuv420p10le`, BT.2020 color primaries, `hvc1` codec tag (required for Apple playback). - **WebGPU HDR capture (gated)** — A complete WebGPU float16 readback pipeline is implemented and tested but gated behind headed Chrome (headless doesn't expose WebGPU). Ready for future use with WebGPU canvas content. - **HDR utilities** — `detectTransfer()` (PQ vs HLG), `getHdrEncoderColorParams()`, `analyzeCompositionHdr()`. 15 unit tests. ## Key design decisions | Decision | Why | |----------|-----| | No `--hdr` flag | SDR content encoded as HDR causes orange shift in browsers. Auto-detect eliminates this. | | HLG pass-through (not HLG→PQ) | Conversion loses brightness without OOTF. Pass-through matches source exactly. | | `hvc1` codec tag | Apple QuickTime requires `hvc1` (not `hev1`) for HEVC playback. | | 1-hour streaming timeout | HDR capture at ~6fps needs more time than the default 10-minute FFmpeg timeout. | ## Files changed | File | What changed | |------|-------------| | `packages/engine/src/utils/hdr.ts` | **NEW** — HDR detection, transfer types, encoder params (15 tests) | | `packages/engine/src/services/hdrCapture.ts` | **NEW** — WebGPU readback, HLG conversion, PQ encode | | `packages/engine/src/services/streamingEncoder.ts` | HDR presets, raw rgb48le input, color tags | | `packages/engine/src/services/chunkEncoder.ts` | HDR presets, conditional color tags | | `packages/producer/src/services/renderOrchestrator.ts` | Auto-detection loop, HDR pass-through capture path | ## How to test Render a composition with an HDR video source. The output should be H.265 10-bit with HDR metadata visible in `ffprobe` (bt2020, arib-std-b67 or smpte2084). Plays correctly in QuickTime and on HDR displays. ## Stack position **2 of 6** — Stacked on #258 (SDR/HDR normalization). Provides the encoder infrastructure that phases 1-5 build on. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Miguel Ángel
|
43e9252065 |
feat: add MOV (ProRes 4444) as transparent video output format (#224)
## Summary - Adds `--format mov` to the render CLI for ProRes 4444 transparent video output - ProRes 4444 with alpha is the industry standard for transparent video overlays, supported by CapCut, Final Cut, Premiere, DaVinci, and After Effects - WebM VP9 alpha technically works but is ignored by all major video editors — only browsers decode it - Adds MOV to the studio export dropdown alongside MP4 and WebM ## Transparency format comparison | Format | Codec | Alpha | Video editors | Browsers | File size | | --- | --- | --- | --- | --- | --- | | **MOV** | ProRes 4444 | Yes | CapCut, Final Cut, Premiere, DaVinci, After Effects | No (won't play in browser) | Large (~5-40 MB) | | **WebM** | VP9 | Yes | None (shows black) | Chrome, Firefox | Small (~200 KB) | | **MP4** | H.264 | No | All | All | Small | > **Note:** ProRes MOV files do not play in Chromium browsers — they are an intermediate/editing format, not a delivery format. Use [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) to verify transparency works correctly. ## Changes - **CLI**: Add `mov` to `--format` validation, examples, and output path logic - **Engine**: `getEncoderPreset()` returns ProRes 4444 (`yuva444p10le`) for `mov` format; handle `.mov` in `applyFaststart` and `muxVideoWithAudio`; add `pix_fmt` to streaming encoder ProRes path - **Producer**: Treat `mov` like `webm` for alpha capture (PNG frames, screenshot mode, `forceScreenshot`) - **Studio**: Add MOV option to export format dropdown and render queue hook - **Core**: Add `mov` to studio API types, render route, and mime helpers - **Tests**: Add encoder preset tests for mov format (42 total, all passing) ## Usage ```bash hyperframes render --format mov --output overlay.mov ``` ## Test plan - [x] `pnpm build` passes - [x] `pnpm --filter @hyperframes/engine test` — 42 tests pass (2 new for MOV) - [x] `oxlint` and `oxfmt` clean on all 12 changed files - [x] End-to-end local render produces ProRes 4444 (`yuva444p12le`) with working alpha - [x] Docker render with `--format mov` — ProRes 4444 confirmed via ffprobe - [x] Studio dropdown shows MOV option in built JS - [x] Transparency verified with [rotato.app/tools/transparent-video](https://rotato.app/tools/transparent-video) |
||
|
James Russo
|
b491679c71 |
fix(engine): add bt709 color space + range conversion to encoding (#223)
## Description Adds proper BT.709 color space metadata and full→limited range conversion to H.264/H.265 encoding. Chrome captures frames in full-range sRGB (BT.709 primaries), but without explicit color tagging, players guess the wrong color space and range — causing color shifts across iOS/Android/desktop and crushed dark values that compound the gradient banding issue fixed in #222. **What changed:** | Setting | Before | After | |---------|--------|-------| | `color_space` | `bt470bg` (guessed) | `bt709` (explicit) | | `color_primaries` | `unknown` | `bt709` | | `color_transfer` | `unknown` | `bt709` | | `color_range` | `pc` (full, wrong for H.264) | `tv` (limited, correct) | | `time_base` | `1/15360` (varies by platform) | `1/90000` (fixed) | **Approach:** - BT.709 VUI params embedded via x264-params/x265-params (`colorprim=bt709:transfer=bt709:colormatrix=bt709`) — ensures the bitstream itself carries color info - FFmpeg-level metadata flags (`-colorspace:v bt709`, etc.) — belt-and-suspenders - `scale=in_range=pc:out_range=tv` filter converts Chrome's full-range output to TV/limited range - VAAPI path chains the range filter with existing `format=nv12,hwupload` - `-video_track_timescale 90000` for consistent cross-platform A/V timing (same as Remotion) - VP9 and ProRes encoding unaffected ## Testing - Verified via ffprobe: all 5 color metadata fields now correct - Directly tested FFmpeg args produce expected output - 40 engine tests pass (8 new: color metadata h264/h265, range filter CPU, VAAPI filter chain, GPU skip, VP9 skip, timescale) - Builds cleanly, lint + format pass |
||
|
James RussoandClaude Opus 4.6
|
7bd8939143 |
fix(engine): add anti-banding x264/x265 params for dark gradients (#222)
Add aq-mode=3 (auto-variance adaptive quantization) to CPU H.264/H.265 encoding. This redistributes bits from bright/textured areas to dark flat areas where color banding is most visible in 8-bit yuv420p output. - standard/high presets: aq-mode=3 + aq-strength=0.8 + deblock=1,1 - draft (ultrafast): aq-mode=3 only (deblock too slow for ultrafast) - GPU and VP9 encoders unaffected (have their own AQ implementations) Adds 6 regression tests verifying the params are emitted correctly. Fixes color banding on dark gradients (eval issue #3, prompts 3,5,10,14). Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.6
|
a146c9e527 |
test(render): add WebM regression test and getEncoderPreset unit tests
- New regression test `webm-transparency`: minimal transparent composition
rendered to WebM, validates VP9 codec and visual quality at 100
checkpoints against golden baseline
- Extend regression harness: `renderConfig.format` field ("mp4" | "webm"),
format-aware output paths and snapshot filenames
- Fix `extractMonoPcm16` to gracefully handle videos without audio
streams (WebM without audio was throwing instead of returning empty)
- Unit tests for `getEncoderPreset()`: VP9/yuva420p for WebM,
h264/yuv420p for MP4, preset mapping, quality preservation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 4.6
|
9f8e5ba5a1 |
initial code (#2)
* feat: initial code port from hyperframes-internal Port all OSS-ready packages from the internal monorepo: - @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime - @hyperframes/cli — CLI for creating, previewing, and rendering compositions - @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg) - @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg) - @hyperframes/ui-player — browser-based video player component - @hyperframes/studio — composition editor (React frontend + Hono backend) Includes regression test suite with Docker-based test harness. All HeyGen-internal references, deployment infrastructure, and proprietary assets have been removed. Package names migrated from @app/* to @hyperframes/*. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scrub internal codenames and stale references from OSS port - Replace static.heygen.ai runtime URLs in test fixtures - Remove internal CDN publish script (publish-hyperframe-runtime.ts) - Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime with neutral names (studio, hyperframe-runtime, __hyperframeRuntime) - Fix stale Vault API / localhost references in docs - Remove broken deprecated_studio link Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining internal codenames and stale references - Delete stale producer README.md and PIPELINE.md (referenced nonexistent files) - Replace "Cerberus" codename with "HyperFrames" in test design reviews - Replace magic-edit postMessage identifiers with hf-preview/hf-parent - Rename debug-magic-edit-timeline.ts to debug-timeline.ts - Replace "Motion Cut" with "HyperFrames" in Timeline comments - Fix studio/CLI references to nonexistent archive package (use local data/projects/ dir, stub render proxy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |