mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-16 06:19:21 +00:00
d130faeb2776ba577beced1ebf7f54055528a465
450
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c9b3d9c962 |
fix(engine): SDR renders keep footage and page colours as in the source (#4564)
* fix(engine): declare SDR extracted frames as sRGB so Chrome does not darken footage * fix(engine): encode SDR mp4 frames with the BT.709 matrix they are tagged with Chrome hands the SDR encoder BT.601 full-range JPEG captures (or RGB PNGs). The encoders only converted the range, so swscale kept the BT.601 matrix under the BT.709 tag (ffmpeg 5.1 to 7.0: CSS #C83C28 delivered as #D44824) or converted YUV to YUV through a low-precision step (ffmpeg 8+: #101010 delivered as #0B0D0D). Decode to RGB with the capture matrix, then encode with BT.709. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * fix(engine): tag SDR frames plain sRGB so every ffmpeg writes the sRGB chunk ffmpeg writes a PNG sRGB chunk only for BT.709 primaries and writes cICP only from 6.1. On ffmpeg 5.1 (Debian bookworm) a BT.601-tagged source therefore got cHRM plus gAMA 1/2.2, which Chrome decodes as a pure 2.2 power: 0x10 shadows rendered as 0x07. Declaring BT.709 primaries too gives the sRGB chunk on every version, so Chrome shows the frame code values for every SDR source. The test now reads the PNG chunks instead of ffprobe, which reports no transfer for an sRGB chunk on ffmpeg 5.1. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * fix(engine): extract SDR jpg frames as the BT.601 full range JPEG readers assume The default frame format for opaque SDR sources is JPEG. ffmpeg converted the BT.709 source to full-range YUV without changing the matrix, and every JPEG reader (Chrome included) decodes BT.601, so footage shifted: 0xC83C28 showed as 0xBB2D29. The old mp4 encode made the inverse mistake and hid it; with the encode fixed, it would reach the output. Decode through RGB with the source matrix, then encode BT.601 full range, keeping 4:2:0. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * fix(engine): round the SDR mp4 colour conversion accurately swscale's default rounding moved saturated colours up to 3 codes (pure red 254 delivered as 251 on ffmpeg 5.1 to 7.0); accurate_rnd brings that to 1. The chunk encoder now decides the HDR-or-SDR filter once, and the 10-bit GPU pad test sets hdr, which its name always claimed. * style(engine): keep the colour filter reasons in names and tests, not source comments The comment ratchet caps a file's comment share: the new filter module and the extractor carry their intent in constant names, and the encode test states why the conversion goes through RGB. * fix(engine): keep colour edges sharp through the SDR colour conversions Going through RGB upsampled chroma with swscale's default filter, so ffmpeg 4.4 to 7.x smeared colour across edges by about 2 px (red 254 next to blue came out 8 inside the blue). Nearest-neighbour resampling maps the half-resolution chroma back one to one, keeps the flat-colour results, and makes accurate_rnd redundant. The pixel tests now also check a red to blue edge. * test(producer): regenerate goldens whose colours the old mp4 matrix bent These four fixtures compare at 40 to 45 dB, and their goldens were rendered with the BT.601-under-BT.709 encode, so the corrected colours score 34 to 37 dB. Regenerated in the Dockerfile.test image. Each new golden is closer to the colour its page declares, e.g. #00d4aa was 00bcab and is now 00d3aa, and #087443 was 006643 and is now 087241. * fix(engine): keep VFR frame timing when extraction applies colour filters ffmpeg before 6.1 ignores frame durations in CFR resampling once any -vf is set, so a VFR window ending on a still lost frames. Filtered VFR extraction now resamples in one process exactly as before and filters in a second. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * fix(engine): encode raw SDR layered frames in BT.709 Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * test(engine): keep the new colour tests inside the format, comment and argv checks * fix(engine): keep HDR and unusual pixel formats out of the two-process VFR path The raw handoff dropped HDR10 light levels and misread semi-planar formats, and decoder options rejected some ffprobe colour names. Only SDR sources in formats nut carries unchanged now take it, their colour tags come back through setparams, and a process that cannot start releases the other. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * fix(engine): read untagged footage with the matrix Chrome plays it with Chrome decodes an untagged matrix as BT.709 from 720 coded lines up and as BT.601 below; extraction assumed BT.601 at every size, so HD footage without colour tags shifted hue against Studio's own preview. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * fix(engine): untagged VP9 and AV1 footage keeps BT.601, as Chrome plays it Chrome applies the 720-line rule only to codecs it decodes through ffmpeg (H.264, VP8); untagged VP9, VP9 alpha and AV1 play as BT.601 at every size. Also widens the raw-pipe pixel formats to all verified ones, and reports the pipeline process that failed first by settle order, not by per-process clocks. Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> * test(producer): regenerate goldens for corrected page colours and untagged SD footage * docs(engine): say which untagged codecs the matrix rule was measured on --------- Co-authored-by: Kenneth Shortrede <95665480+sontiac@users.noreply.github.com> |
||
|
|
3def3514d1 |
fix(engine): export draws a video in exactly the frames it is shown in (#4565)
* fix(engine): export draws a video in exactly the frames export visibility shows it * test(producer): guard that chunk frame planning snaps clip windows like export * refactor(engine): set the render fps once per frame table and name its clip window * chore(engine): trim frame table comments |
||
|
|
984ad28c7c |
fix(core): clips that meet within float rounding count as meeting in edits and playback (#4535)
* fix(core): clips that meet within float rounding count as meeting in edits and playback * fix(engine): hand a shared clip boundary to the incoming clip in export, as preview does |
||
|
|
fba943ea3a | fix(cli): render --quiet stops printing producer info and trace lines (#4478) | ||
|
|
f421851bdd |
fix(producer): give each parallel segmented-capture worker its own browser (#4475)
* fix(producer): give each parallel segmented-capture worker its own browser Segmented capture with two or more workers failed at the first frame with "Invalid JPEG input at frame 0: empty frame". Every worker opened its page in one pooled Chrome, and beginFrame capture does not survive a shared browser: a worker's first beginFrame returned no screenshot. Fresh worker sessions now follow shouldDisableBrowserPoolForParallelWorker, the rule the parallel disk path already uses. The default session factory was also built once per worker, so with a probe session every worker reused that same page. It is now built once per run, so only the first worker takes the probe. Workers are now created inside the try, so when a later worker fails to launch, the finally still closes the browsers opened before it. * fix(producer): close a segmented worker session whose initialization fails The default segmented session factory opened a fresh browser and then initialized it with nothing owning it on failure, so a page-load or init timeout leaked that Chrome. It now closes the session and rethrows, as the streaming stage does. |
||
|
|
9d4836902b |
fix(engine): name the data-no-timeline opt-out in the timeline-timeout warning (#4410)
* fix(engine): name the data-no-timeline opt-out in the timeline-timeout warning When the sub-composition timeline wait times out, two things are emitted: the stderr line inside pollSubCompositionTimelines, which names the pending composition ids and the data-no-timeline opt-out, and the structured warning on the session, which said only "Sub-composition timelines did not become ready within 45000ms". The structured one is what a programmatic caller reads: render re-prints it in the completed_with_warnings summary and returns it in the result's `warnings` array, so a CI consumer or an SDK caller sees the symptom with no remedy attached. It also reads as a fault to diagnose, when the commonest cause is a composition that legitimately has no GSAP timeline -- CSS animations or rAF -- and is paying the full playerReadyTimeout on every render because nobody told the author the wait is theirs to switch off. The message now says the wait can be intentional, names data-no-timeline as the opt-out, and keeps the async-setup case as the other branch. The pending composition ids were already computed for the stderr line and thrown away; an optional onPending callback carries them to the session so the structured warning and its details can name them too, matching the script-failure branch, which already names its failed sources. The stderr text is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(engine): normalise the pending-id result instead of casting it The enumerate step cast `page.evaluate`'s result with `as string[]`, which is a claim the call site cannot enforce: `evaluate` is loosely typed, and anything that returns a non-array turned the cast into a TypeError thrown on the very path that exists to REPORT a problem. Normalise the result instead. `pendingCompositionIds` was also never declared on `CaptureWarning.details`, whose type is closed, so the engine build failed and every downstream package failed with it. Adds coverage for the poll's own `onPending` handoff, which the existing tests in this PR could not see: they exercised `recordSubTimelineWarning` (the message text) and never `pollSubCompositionTimelines` (where the throw was). The non-array case is the one that discriminates this fix from the tempting alternative of leaving the cast and adjusting the stubs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
250c9bd509 |
fix(engine): resolve the version by walking to our own package root (#4409)
readEngineVersion resolved ../../package.json relative to import.meta.url. That path is correct from packages/engine/src/utils/ in the repo and overshoots the package root from <pkg>/dist/ in the published CLI, which bundles the engine. The require threw MODULE_NOT_FOUND, a bare catch swallowed it, and every render stamped the 0.0.0-dev fallback. No single relative literal is correct in both layouts, so resolve by walking up to our own package root instead of by a fixed depth. The name check is the fix, not a refinement. A bare "first package.json found while walking up" resolves to whatever sits above in a hoisted install and returns a confident wrong version - the same class of defect as the sentinel, and harder to notice because the answer looks plausible. Only a package.json whose own name is hyperframes or @hyperframes/* may answer. A test pins this: removing the name check fails that case and only that case. Also replaces the sentinel. 0.0.0-dev parses as a version, compares against one, and sorts below every release, so a total resolution failure was indistinguishable from a dev build - which is why it shipped unnoticed. The fallback is now "unresolved", deliberately not semver-shaped, and every failure path warns with the reason. A failed lookup still never breaks a render. The old comment claimed the engine ships as raw TS and exports ./package.json. The published package has no exports field and bundles the engine, so that premise does not hold where the code runs; it is replaced with the two layouts and why they conflict. Refs #4365 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
cbaddec51a |
fix(engine): download tests stop racing real deadlines, and failures name their cause (#4419)
* test(engine): hold the download attempt deadline so slow runners cannot trip it urlDownloader tests passed a real 1s deadline that a slow Windows attempt outlasted. The deadline timer is now held; the one test that needs it fires it by hand. * fix(engine): name the underlying error when a download fails, without its URL * test(engine): fire the stalled-body deadlines by hand instead of racing a real 20 ms timer |
||
|
|
7d1d9ddf70 |
Merge pull request #4368 from heygen-com/fix/ffprobe-nb-frames-duration-crosscheck
fix(engine): cross-check nb_frames when video stream duration is absent |
||
|
|
b021cb77be |
fix(engine): reconcile metadata duration after extraction delivers fewer frames (#4371)
When the probed video stream duration is inflated (container duration fallback includes a longer audio track), FFmpeg delivers far fewer frames than expected — typically exactly 50% for stock clips whose audio is 2x the video length. The coverage gate then aborts the render because it sees 50% coverage against the inflated expectation. After extraction, if FFmpeg delivered <=60% of the predicted frames, correct `metadata.videoStreamDurationSeconds` to the actual video extent. This reconciliation feeds through to the coverage check via the same `metadata` reference, so `expectedFramesForVideo` sees the true duration without any interface changes. This is a corrective complement to the preventive nb_frames cross-check in extractMediaMetadata — it catches cases where nb_frames is unavailable (MKV, live streams, fragmented MP4). Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
ee4c00db6a |
fix(engine): cross-check nb_frames when video stream duration is absent
When ffprobe omits the video stream's own `duration` field, `extractMediaMetadata` falls back to the container duration (`format.duration`), which includes the longest stream in the container. Stock and looping clips commonly have an audio track 2x the video length, inflating `videoStreamDurationSeconds` and causing the downstream coverage gate to expect twice the frames FFmpeg can actually produce — the "50% cluster" failure pattern. Cross-check with `nb_frames / fps` when the stream duration is absent. If the frame-derived duration is >10% shorter than the container, use it instead. This corrects the inflation at probe time without requiring a second ffprobe pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
30f548f62b |
fix: a stalled remote asset download no longer hangs a Cloud Run render (#4343)
* fix(gcp-cloud-run): run the Cloud Run image on Bun 1.4.2 so remote downloads cannot hang * fix(engine): make the download deadline abort the body pipeline |
||
|
|
29378b6c7f |
fix(engine): classify a single-keyframe video as problematic, not healthy (#3998)
* fix(engine): classify a single-keyframe video as problematic, not healthy analyzeKeyframeIntervals short-circuited to isProblematic:false whenever it found fewer than 2 keyframes, but exactly one keyframe is the worst case for seek performance, not a healthy result: every seek past frame 0 must decode the entire file. The single-keyframe branch treats the rest of the video stream's own duration (not a container duration, which a longer audio track can inflate) as one uninterrupted GOP. ffprobe's frame timestamps are absolute presentation timestamps rather than relative to stream start, so a re-muxed or trimmed clip with a nonzero start time is normalized the same way extractFinalVideoFrameTimestamp already does for the same quirk. The cache key now includes both values, matching that sibling function's convention, since a stale cache entry from a different duration/start pair would otherwise win silently. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * refactor(engine): share stream-window normalization between keyframe check and final-frame lookup extractFinalVideoFrameTimestamp and analyzeKeyframeIntervals each independently computed the same stream-start fallback, finite guard, and cache-key format from the same metadata shape. Extracted resolveStreamWindow so the two can't drift out of sync with each other. Also: match ffprobe's real trailing-comma csv output in one keyframe mock, add a test at the exact 2s threshold boundary, and comment the noUncheckedIndexedAccess-only fallback at the single-keyframe branch. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * style(engine): trim long comments to one-to-three lines --------- Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com> |
||
|
|
1b8f8a43d4 |
fix(cli): stopping the preview server no longer leaves a Chrome running (#4183)
* fix(cli): stop the preview server's browser when the server exits Cancel in-flight renders and thumbnail launches before draining the browser pool on shutdown, instead of only closing whatever browser was already registered. A render whose Chrome died from the shutdown signal itself was being misclassified as a transient failure and retried with a fresh, untracked browser that outlived the process. Reject new render and thumbnail requests once shutdown has begun, and await an in-flight thumbnail launch before closing it. * fix(cli): close preview browsers before a hung render, keep SIGINT armed shutdown() awaited renders before closing browsers, so a render slower than preview.ts 3s exit watchdog left Chrome running when it fired. Close the thumbnail browser and drain the pool concurrently with, not after, the render wait, and bound the wait under that watchdog. A second Ctrl+C/SIGTERM during shutdown removed the one-shot signal handlers, so it hit the OS default and killed the process before cleanup ran. Use persistent handlers guarded by the existing shuttingDown flag instead. Also: getThumbnailBrowser could still hand a live lease to a request that lands after shuttingDown flips true; trim a comment over budget; replace a fixed-sleep test race with a drain-signal barrier. * fix(engine): make browser pool shutdown terminal, not just draining drain() resets its drainPromise to null once it settles, so acquire() only waits for an in-flight drain -- a render still unwinding after shutdown could relaunch Chrome the instant that drain resolved (probeStage.ts:449-465 has exactly this gap between an abort check and a later acquireBrowser call). No non-shutdown caller reuses the pool after draining it (checked every drainBrowserPool()/drain() call site), but added a separate terminal close() rather than changing drain()'s own semantics, so a future reuse caller stays safe by default. BrowserLeasePool.close() sets a permanent closed flag before draining, and acquire() checks it both before and after its one await point, so a request already mid-await when close() lands still sees it once that await resolves. studioServer's shutdown() now calls the new closeBrowserPool() instead of drainBrowserPool(). Also bounds drain()'s own wait: a close() that hangs past 1s now gets escalated to a force-close instead of blocking the caller indefinitely, keeping total shutdown time under preview.ts's 3s exit watchdog alongside the existing render-wait bound. * fix(engine): trim closeBrowserPool JSDoc to house comment length |
||
|
|
cc19b8c812 |
fix(server): contain static file requests within their served roots (#4217)
* fix(server): contain static file requests within their served roots
Static asset handlers joined the raw request path onto their root
directory without checking where it landed. Hono decodes percent
escapes while routing, so an encoded backslash (%5c) arrives as a real
separator on Windows and dot segments arrive already collapsed —
"/assets/..%5c..%5csecret" and "/%2e%2e%2fsecret" could read files
above the studio bundle root (embedded Studio server) and above
compiledDir/projectDir (engine render file server). Browsers never
request those shapes, but anything that can reach the bound port can:
raw local clients, LAN peers when the preview binds 0.0.0.0, and a
shared-machine process hitting the render server.
Both handlers now resolve the request path first and refuse anything
that collapses outside its root, using the same lexical containment
the preview asset route already applies — bundle/project assets behind
symlinked directories keep being served, only dot-segment escapes are
rejected. Route-level pathSafety tests pin the behavior in both
packages, including the Windows backslash shape and the
symlinked-asset cases from the descriptor-pinning regressions.
Validated: new containment suites fail on the prior handlers (200 +
marker file content served from outside the root) and pass with the
guards; cli server (140), engine services (1024+, matching the main
baseline incl. pre-existing browserManager failures), typecheck,
oxlint, oxfmt and the full workspace build all clean.
* fix: close fallow audit findings, prove containment cross-platform
Fallow audit (health/complexity): createFileServer's request handler
exceeded the CRAP-score threshold once the new containment checks were
added. Extracted resolveContainedPaths, a small pure function that
resolves and validates both roots, cutting the handler's own branching
back down.
Fallow audit (duplication): the new pathSafety test files copied their
temp-root/studioDir setup and teardown from the existing staticAssets
test wholesale. Extracted the shared parts into
studioServerTestFixture.ts (makeStudioServerRoot, writeStudioIndexHtml,
cleanupStudioServerRoot); only the per-file vi.mock("node:path", ...)
call stays duplicated, since Vitest requires it declared statically in
each test file and it cannot be moved behind a shared function -- that
one block keeps a narrow, targeted ignore comment instead of a
file-wide one.
Security claim: isWithinProjectRoot's %5c-decoded-backslash escape is
real, but only on Windows -- POSIX never treats a literal backslash as
a separator, and Hono leaves %2f undecoded in c.req.path, so none of
the new tests' encoded-traversal payloads produce an actual escape on
Linux (verified empirically: they pass identically with the fix
reverted). isWithinProjectRoot now takes an optional path-module
override (defaulting to the host's native node:path, so every existing
caller is unchanged) so its win32 and posix branches can be unit
tested directly from one OS, the same pattern already used by
producer/fileServer.ts's isPathInside. New tests cover a backslash
traversal, a mixed backslash/forward-slash traversal, and a
drive-letter path escaping the root under path.win32, plus the
existing posix case -- confirmed each one fails when the check is
removed. The HTTP-level tests stay as-is: real regression coverage for
them comes from the windows-latest CI job, which runs both new
pathSafety suites unfiltered.
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-authored-by: orphan4183 <kingenmaww@gmail.com>
|
||
|
|
c27bdda7d9 | fix(engine): read extensible PCM and float WAV headers (#4267) | ||
|
|
839ae426dd |
feat(studio): typed audio values, clip fades, audible video drops (#4250)
* feat(studio): typed audio values, clip fades, audible video drops Audio rows in the media inspector (Volume, Speed, Media start) take a typed value: click the readout, type "-6 dB", "1.5x" or "2.5s", Enter. Speed presets are hidden on <audio> clips. Clip-edge fades: data-fade-in / data-fade-out in seconds, honoured by the preview runtime (per-tick gain), the render mixer (afade after volume) and Studio (Fade in / Fade out rows, and drag dots at the top corners of audio clips and audible videos). Linear on both sides so preview and render agree; fades that outrun the clip scale to meet inside it. Dropped videos with an audio stream now land with data-has-audio="true" instead of muted, so the mixer keeps their sound. The media metadata endpoint reports hasAudio from the ffprobe call it already makes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(audio): one owner for fades, dB, and afade-after-volume Illegal fade values become no-fade at fadeGain; typed inspector input still returns null so the readout stays red. Master and group mixes share one filter chain with volume then afade. Fade-out stays anchored to the trimmed clip end. * fix(studio): share audio helpers and bound typed media values * fix(studio): reserve sibling fade duration when editing --------- Co-authored-by: Jake Moran <jakemoran2011@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com> |
||
|
|
0cde884330 |
fix(cli): macOS 12 users get Chrome 150 automatically and can render (#4185)
* fix(cli): pick a macOS 12 capable Chrome on macOS 12 instead of the pinned build * test(engine): stop building subprocess eval scripts from strings |
||
|
|
3f3f3b7f10 |
fix(engine): audio source-not-found names the file and shares one src resolver with lint (#4172)
* fix(engine): audio source-not-found names the file and shares one src resolver with lint * fix(engine): audio source-not-found prints the resolved path with forward slashes on Windows * fix(parsers): the shared src resolver ignores a media fragment like the browser |
||
|
|
bdecae066a |
feat(engine): render speed ramps with the same source-time mapping as the preview (#4143)
* feat(engine): render speed ramps with the same source-time mapping as the preview The frame extractor, frame lookup table, coverage check, compile-time duration and audio mixer read a clip's rate lane through sourceTimeAt, so a rendered frame at t shows the source frame the preview shows. Audio bakes a ramp as consecutive source slices stretched by their mean rate, keeping pitch. The compile-time attribute reader now handles single-quoted and entity-encoded JSON attributes. Adds a speed-ramp regression fixture. * test(producer): regenerate the speed-ramp golden audio with the current engine The stored baseline predates the per-slice audio fix, so its audio drifted 1 to 3 percent in pitch from the expected source-time track and the CI render correlated at 0.61. Rendered in the CI test image, the new baseline tracks the expected pitch to 0.1 percent median, and the compare now reports visual 0 failed frames and audio correlation 1.000. * refactor(core): drop data-preserve-pitch, pitch is preserved by construction The engine always time-stretches audio with atempo, so an opt-out read only by the preview made preview and render disagree. Remove the attribute and its reader so both paths keep pitch. |
||
|
|
7f452ad276 |
feat(core): speed ramps as a rate lane in data-automation (#4129)
* feat(core): speed ramps as a rate lane in data-automation A clip can carry a rate lane in data-automation. sourceTimeAt integrates it to source time and is the one mapping the runtime uses for media seeking, playback rate, natural duration, child timelines and the audio-master clock. Rate is clamped to 0.1-10 in one place, pitch is preserved unless data-preserve-pitch is false, and ramped clips play through the media element. Lint warns on a rate lane outside video and audio. * fix(core): format the speed-ramp files and expect the unified rate clamp in the engine tests * test(engine): read the playback-rate clamp from core instead of a literal |
||
|
|
0a5c235173 |
feat(cli): hyperframes timeline prints a project's tracks and clips (#4138)
* feat(cli): hyperframes timeline prints a project's tracks and clips * docs(cli): list the timeline command * feat(cli): timeline command reports unreadable lanes; add skill docs * fix(cli): timeline stays inside the project, shares one start cache, keeps small volumes * fix(cli): timeline resolves symlinks before the project containment check * fix(cli): timeline accepts project folders whose name starts with two dots * refactor(cli): timeline and compositions share the engine's start resolver * docs(skills): point agents at the timeline command where they decide how to read a project * fix(cli): timeline prints absolute main-timeline times and the owning file for nested clips Nested rows carry absStart/absEnd on the main timeline and their owning file, alongside existing local start/end. Text prints absolute time first, local time in parentheses. Adds jq/node one-liners for common queries, tested against a fixture. * fix(cli): timeline places nested media at the runtime start and tests its query one-liners A nested media clip with a negative literal data-start was reported later than it plays, because the raw start was clamped before the host offset was added. The skill's jq and node one-liners now run against the fixture in the test suite. * fix(cli): timeline clamps a negative media start to 0 when its host starts at 0 The runtime skips the host-offset formula for a host at t=0 and clamps the start, so the timeline command now does the same. * refactor(core): one media-start rule shared by the runtime and the timeline command The runtime and the CLI each carried a copy of the rule for when a nested media clip's start takes the host offset, and the copies had drifted. Both now call resolveMediaStartSeconds. * test(cli): timeline fixture for a media start written as an expression parseNumeric and the runtime's parseStrictFiniteTimingNumber are the same function, so the two callers cannot disagree on what counts as a literal; this pins the expression case. |
||
|
|
607775bb3a | fix(engine): make urlDownloader retry tests independent of wall-clock time (#4155) | ||
|
|
439bdbd774 |
fix(engine): pin the pool's worker-failure contract and survive a throwing hook
The streaming stage's peer-death test replays the contract "hook first, with the original CaptureFailure, then the peers abort and the pool rejects with its own summary" inside its mock, and nothing on the engine side pinned it. The gate now lives in createPoolFailureHandler, tested without a browser: the hook sees the original failure before the peer signal flips, a hook that throws can no longer skip the abort (the pool still reports the classified failure, not the hook's error), failures after the first are ignored, and a cancellation leaves the peers running. Mutation-checked: removing the try/finally, swapping the order, or dropping the first-failure guard each fails one test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
5273554516 |
fix(engine): a worker death is pool-fatal on the streaming path
On the disk path a transient worker death is deliberately non-fatal: the orchestrator's adaptive retry re-captures that worker's missing frames. The streaming path has no per-worker retry and the dead worker's frames are gone, yet it inherited the same gating — so its peers parked in the ordered writer waiting for a frame that would never come, and sixty seconds later the no-progress watchdog relabelled the death as a stall. That is the `stuck at k/N` field signature. isPoolFatalWorkerFailure splits the two: every non-cancelled failure ends a streaming pool, and a new onWorkerFailure hook delivers the ORIGINAL failure to the caller before peers are aborted, so the ordered writer can be released with it rather than with a stall. Mutation-checked in both directions, including that `cancelled` never propagates (the pool is already aborting). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
95cbf56634 |
fix(engine): classify a broken encoder pipe as io, not authoring
EPIPE (darwin/linux), EOF (win32's UV_EOF for a closed named pipe) and ECONNRESET all mean the process reading our stdin has gone. Bucketed as authoring, telemetry blamed the composition for a host event. Matched on the errno code and on the message, since the code is lost when a worker pool flattens the error to a string. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
69024e17d1 |
fix(engine): keep ffmpeg's exit reason when it dies under a parked write
The field "write EPIPE" / "write EOF" render errors are this path: writeFrame waits on once(stdin, "drain") for back-pressure to clear, ffmpeg dies first, and Node rejects the drain wait with the stream's own error. That rejection escaped writeFrame as the render error, so ffmpeg's exit code and stderr — captured, but read only from the `false`-return path — were discarded. With N workers feeding one encoder the writer is parked most of the time, which is why the parallel cohort reads EPIPE where a single-worker render reads "Streaming encoder exited before frame N: FFmpeg exited with code …". Wait for the exit to settle, then report it as the close race does. Both halves are mutation-checked: rethrowing again fails the test, and returning "exit" without awaiting settlement fails the synchronous getExitStatus read. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
46d852cc93 |
refactor(engine): extract concatVideoFiles from the chunked encoder
Segmented capture needs the same concat-demuxer stream copy the chunked encoder already does. Behaviour preserved, including the external_interruption classification the caller maps to a retryable failure: the helper reports it rather than swallowing it. The concat list keeps living beside its inputs, because concurrent encodes share an output directory but not a chunk directory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
2cf8024a7c |
fix(engine): read Chrome process info from the browser target, not the page
SystemInfo.getProcessInfo is served only by the browser target; a page-target session rejects with "is only supported on the browser target". The sampler swallows probe failures by design, so this shipped as a permanently empty reading: a 10 s debug render reported chromeMemory null with capturePath already correct. Against the browser target the same render reports browser 105 MB / renderer 359 MB over 3 samples. readChromePids is extracted so the target choice is pinned by a test whose fake rejects a page session the way Chrome does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
428922162d |
feat(engine): sample Chrome process memory per capture session
Start a 2 s unref'd sampler when a session finishes initializing, stop it in closeCaptureSession, and report peaks on CapturePerfSummary. Pids come from puppeteer (browser) and CDP SystemInfo.getProcessInfo (renderer, GPU), so the long-render "Target closed" class can be correlated with Chrome memory instead of guessed at. Disable with HF_CHROME_MEMORY_SAMPLER=false; cadence HF_CHROME_MEMORY_SAMPLE_MS. The start hook is finalizeSessionInit rather than the tail of initializeSession: that helper is already the single owner of "session is ready" across both of initializeSession's exits, so the early screenshot return cannot silently skip sampling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
a35168eddd |
fix(engine): sampleOnce() joins an in-flight sample instead of skipping it
With a boolean reentrancy flag, the Task 3 close sequence (`stop(); await sampleOnce()`) returned immediately whenever an interval tick was still awaiting CDP/ps, so getCapturePerfSummary read stats from up to one interval earlier and onSample fired once more after the session had closed. Reproduced with a 350 ms getPids on a 100 ms interval: close resolved with samples=0 and the callback landed afterwards. The guard now keeps the in-flight promise and returns it to any caller, so an interval tick still never stacks a second round-trip (the existing reentrancy test holds) while a close-time call resolves only after the running tick has merged. Deviates from the plan's reference body, which used `let inFlight = false`; the ChromeMemorySampler interface and all names are unchanged, only the sampleOnce() contract is stronger. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
7cf57fa42e |
test(engine): pin the unref'd sampler interval via a cast-free timer seam
Three independent reviewers found that deleting timer.unref() in createChromeMemorySampler left the suite green: the plan's Global Constraint (fire-and-forget on an unref'd interval) had no failing test. The new test captures the real NodeJS.Timeout through the setIntervalFn seam and asserts hasRef() is false after start(); it goes red when the unref() call is removed. The seam could not be used without a cast as typed in the plan (`typeof setInterval`): with the DOM lib in scope that type also carries the `(handler, timeout) => number` overload, which a capturing wrapper cannot satisfy. setIntervalFn / clearIntervalFn keep their names but are now typed to the two-argument shape the sampler actually calls; the global timers are assignable to those shapes, so the defaults are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
53403a5896 |
feat(engine): chrome memory sampler with live onMemorySample capture option
Pure service for Phase -1 of the long-form render plan: merges per-pid RSS rows into browser/renderer peaks, last-sample total and GPU presence on an unref'd interval with a reentrancy guard, so a CDP or ps failure can never reach the capture loop. CaptureOptions.onMemorySample is the live route the producer will wire so a mid-render crash still reports memory state. Nothing instantiates the sampler yet (Task 3); no behaviour change. Beyond the plan's tests, pins the double-start, empty-sample and catch guards directly, since the interval-based error test stays green when the catch is removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
1862c34399 |
test(engine): fix readonly array type in the pid-filter test
`readonly string[][]` parses as `ReadonlyArray<string[]>`, so `calls.push` was a TS2339 in 5214552cd. It slipped through because the engine tsconfig excludes `src/**/*.test.ts` and vitest strips types; caught by running tsc on the test file explicitly. `Array<readonly string[]>` is what the neighbouring "one ps call" test already uses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
3cee3b94f7 |
fix(engine): read /proc/<pid>/status for RSS on linux instead of shelling out to ps
Every render image (packages/cli/src/docker/Dockerfile.render, packages/gcp-cloud-run/Dockerfile, examples/k8s-jobs/Dockerfile.example, Dockerfile.test) is node:22-bookworm-slim with no procps, so `ps` is ENOENT in containers and sampleProcessRss would return [] on every tick: the whole cloud fleet would report chrome_memory_samples: 0, indistinguishable from "no Chrome pids". Linux now reads VmRSS from /proc/<pid>/status with a per-pid try/catch, so one renderer that exited between discovery and sampling omits only itself (with `ps` a wholly-dead batch fails the call). No child process per tick on the primary server platform. Same precedent as the CLI's orphanCleanup.ts / processTree.ts. `ps` stays for darwin and other POSIX; `tasklist` for win32. The reader is an optional fourth parameter with a default, so Task 2/3 callers are unaffected. Scope note: spec section 2.5 records the Target-closed population as docker=false throughout, so this closes a render_complete completeness gap on cloud, not a coverage gap on the crash class this phase targets. Verification: 15/15 vitest; four linux-branch mutants (per-pid catch rethrows, regex loses /m, kb <= 0 weakened, path /status -> /stat) each fail at least one test, restored green. Real-process smoke check on darwin: sampleProcessRss([process.pid, <exited pid>]) returns one sample for the live pid (BSD ps exits 0 with partial output when a valid-range pid is gone). The linux path is unit-tested against a /proc/<pid>/status fixture only; no Linux host or VM was available in this session to execute it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
645cbdce4e |
test(engine): pin the pid filter, dedup and tasklist memory guard in sampleProcessRss
Review of 76d8e2682 found three surviving mutants: removing the
`Number.isInteger(p) && p > 0` filter, dropping the `new Set` dedup, and
weakening `kb <= 0` to `kb < 0` all passed the plan's 8 tests. The filter is
the guard that matters most: Task 2 feeds `browser.process()?.pid`, and a
single NaN or undefined reaching `ps -p` makes ps exit non-zero, which the
never-rejects catch turns into `[]` for every pid in the sample.
Mutation check: each of the three mutants now fails exactly one test
("drops invalid pids and duplicates before calling ps" for the filter and
dedup, "returns [] when the memory column is not a positive number" for the
tasklist guard); restored source is 10/10 green.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
|
bc70156f00 |
feat(engine): zero-dependency per-pid RSS sampler (ps / tasklist)
First piece of the long-form render Phase -1 telemetry: the Chrome memory sampler needs resident set size per pid without adding pidusage or systeminformation. `ps -o pid=,rss=` covers POSIX in one call for every pid; `tasklist /FO CSV` covers win32 one pid at a time. The function never rejects and omits pids that exited between discovery and sampling, so a failed sample can never reach the capture loop. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
2b2b999e62 |
Merge pull request #4109 from heygen-com/lfr/02-phase0-duration-cap
feat(producer): stream long single-worker renders by default; duration cap behind a flag (Phase 0) |
||
|
|
9981d9afb7 |
fix(capture): never silently render a webgpu composition blank (#4091)
* fix(capture): never silently render a webgpu composition blank A composition declaring data-requires-webgpu only got the Chrome WebGPU flag when hardware GPU auto-detection succeeded. On a software-GPU host it launched without the flag and rendered blank with no error. compositionRequiresWebGpu (static declaration check) and assertWebGpuAdapterAvailable (live per-page adapter probe) now live once in packages/engine's browserManager, the shared choke point every launch site already goes through. All six launch sites (frame capture, the four CLI diagnostic commands, the studio thumbnail server, and the render pipeline's capture-options builder) route through those two functions; the old CLI-only static-heuristic guard is deleted, not kept alongside the new one. A software host that cannot obtain any WebGPU adapter now fails loudly naming the requirement instead of producing a blank frame. * fix(capture): thread requiresWebGpu through the distributed chunk-render path The distributed renderChunk() path built its CaptureOptions without requiresWebGpu, so it fell through to frameCapture's own fetch-based detection, whose failure path silently treats "requires webgpu" as "doesn't" if that fetch ever fails. Every other launch site (frame capture, all four CLI diagnostic commands, the studio thumbnail server, and the in-process render pipeline) already reads the value from HTML already in memory or on disk instead of relying on that fallback. Reads the compiled composition's index.html the same way probeStage already does and derives requiresWebGpu from it before building CaptureOptions. Adds a regression test that pins the exact path renderChunk() reads at render time, since the composition doesn't actually exercise WebGPU so an end-to-end render can't distinguish the two cases on this host. * test(producer): observe the real CaptureOptions renderChunk() builds The previous regression test called plan() and compositionRequiresWebGpu() directly, bypassing renderChunk() entirely, so it could not detect a regression to the wiring it was meant to protect (confirmed: it stayed green with the fix reverted). Exports the existing createCaptureSession dependency seam so a test can substitute a spy that delegates to the real implementation while capturing the CaptureOptions object renderChunk() actually constructs. Verified non-vacuous by reverting the fix and confirming the test fails. * fix(gcp-cloud-run): resolve puppeteer-core from a real file in the webgpu probe The image build's WebGPU adapter probe imported puppeteer-core from an inline `bun -e` eval string. The eval's synthetic /app/[eval] location can't resolve the workspace-hoisted dependency, so every build of this stage failed closed with "Cannot find package 'puppeteer-core'" before the probe itself ever ran. Moves the probe into a real script file next to probe-beginframe.ts, which already imports puppeteer-core dynamically the same way and resolves fine from its real workspace location. Verified locally against a cached chrome-headless-shell binary. * fix(engine): bound the webgpu adapter probe so a stuck driver can't hang the caller requestAdapter() has no native timeout. A broken software GPU driver that never settles it would hang assertWebGpuAdapterAvailable indefinitely, including inside the GCP image build's probe RUN step, producing an opaque build timeout instead of a clean probe failure. Races the in-page request against a bounded timeout so a stuck adapter reads as unavailable and fails the same way a genuinely absent one does. * test(cli): mock the engine's webgpu exports in the browser-fallback test * test(producer): run the typegpu adapter fixture only on hosts with a hardware WebGPU adapter The fixture declares data-requires-webgpu, so a software-only host now refuses it instead of silently rendering the shader blank. The old golden had no shader in it. Tag it webgpu, skip that tag in the CI image and the test scripts, and re-record the golden from a Metal render with the shader visible. * fix(engine): bound the composition fetch that decides the webgpu flag createCaptureSession fetched the served page with no timeout before launching the browser, so a hung server stalled a render forever and hung a fake-timer test. Give it a five second limit, and let that test say it has no WebGPU requirement the way the producer does. * fix(capture): refuse a software fallback WebGPU adapter instead of rendering on it swiftshader reports a WebGPU adapter but cannot render canvas WebGPU, so a data-requires-webgpu composition on a software-only host crashed on its first frame. Refuse fallback adapters with the same clear WebGPU-unavailable error. The GCP image probe now serves its page over http (file:// hides navigator.gpu) and reports a software-only host instead of failing the image build. * fix(engine): parse the composition root tag without a backtracking regex * fix(engine): keep quoted attribute values whole when reading the composition root tag |
||
|
|
e36a53d6b6 |
fix(core): hold render-ready on declared compute, not just adapter loaders (#4065)
* feat(core): hold render-ready on declared compute and a build-ready hold, not just duration Adds the compute and paint-and-idle inputs to the composition readiness gate, plus window.__hf.buildReady: a promise registry a piece registers when it has setup work no adapter can see (mesh building, shader compiles). The runtime holds render-ready on it and the player shows its loading state instead of a blank frame while it resolves. * refactor(core): dedupe the paint-and-idle readiness test setup Simplify pass on 23810da1b: fallow flagged the two paintAndIdleReadinessInput tests as a 12-line clone; extracted the shared doc/tracking setup. * fix(core): stop the readiness settled-tracker hanging on 2+ pending promises Promise.all allocates a new promise every call, so comparing ITS identity across repeat polls (isAdapterReadinessSettled, isBuildReadinessSettled) never observes settled once 2+ promises are combined: every poll rebuilds a new combined promise, re-arms, and republishes forever without the caller ever seeing true. Reproduced with two already-resolved window.__hf.buildReady entries, which is the exact multi-piece usage the new authoring doc describes. Fixes both trackers at once with a shared createSettledTracker that compares the source promise array by element instead of the Promise.all wrapper. * fix(core): cancel readiness inputs once settleCompositionReadiness settles computeReadinessInput's poll timer and paintAndIdleReadinessInput's rAF chain kept running after the shared timeout won the race, since nothing told the losing inputs to stop. One AbortSignal now threads through all three inputs and is aborted the instant the race settles. * docs(core): fix a stale sync-callback claim in settleCompositionReadiness paintAndIdleReadinessInput always has a frame to wait on for any real document, so the sync fast path this docstring described is no longer reachable with the default inputs except for a viewless document. Found by the simplify pass on this branch. * fix(core): clear a stale buildReady entry on runtime teardown A never-resolved window.__hf.buildReady promise from a torn-down composition survived into the next init in the same window (window.__hf is never reset), permanently blocking render-ready for whatever loaded next -- the exact black-screen class this branch fixes, reproduced through its own new mechanism. Found by the independent review. Also fixes a stale hyperframes-player.ts docstring (media/images/fonts had not been updated to compute + paint-and-idle) and documents the buildReady key-collision hazard in three.md. * fix(core): skip the compute readiness wait when no runtime ever ran computeReadinessInput polled window.__renderReady for the full shared timeout on any same-origin document with no HyperFrames runtime injected (a probe-only composition using native video/duration, or a blank test iframe) — nothing was ever going to set that flag. Checking window.__hf first (the same signal composition-probe.ts already treats as "has a runtime") lets it return null instead of waiting. * fix(player): debounce the loading overlay and diagnose compute timeouts paintAndIdleReadinessInput always has a frame to wait on for an attached iframe, so settleCompositionReadiness's synchronous-settle path is unreachable in practice now — the loading overlay showed on every single Play, even ones with nothing genuinely pending. Debounce the overlay itself instead: a wait that resolves fast never shows it. Also names compute (window.__renderReady) in the stuck-assets timeout diagnostic, which previously only re-scanned media, images and fonts. * fix(engine): name a stuck buildReady key in the zero-duration diagnostic A build promise a composition registers on window.__hf.buildReady that never resolves left the export path's zero-duration timeout naming nothing useful — hasHf/hasSeek/duration all looked fine, since the runtime was genuinely present and just waiting on that one promise. Probes buildReady's keys for pending ones and names them in the hint. * fix(engine): stop the buildReady diagnostic from crashing on a rejection A microtask race against a bare resolved promise rejected the whole diagnostic expression on any rejected buildReady entry (the runtime's own createSettledTracker rejects deliberately) and mis-sorted a settled non-native thenable as pending, since its extra adoption tick loses the race. Both measured directly, not inferred. Settle each key onto its own record past one macrotask instead: no throw, and pending/resolved/rejected are distinguished correctly. A rejected key gets its own hint, separate from a merely pending one. * fix(player): report a backgrounded document instead of overclaiming a paint bound MAX_PAINT_WAIT_MS only bounds a document that keeps producing frames; one that stops entirely (backgrounded tab suspends rAF, or fewer than 3 frames ever fire) still rides the full shared 8s timeout, same as before this input existed. Corrected the comments that claimed otherwise. That gap is exactly why the stuck-assets diagnostic left paint-and-idle unreported: there was no accurate way to say it might be the cause. Added documentHidden (doc.hidden) to the diagnostic instead, since a backgrounded tab is the reachable, directly observable case. * refactor(engine): simplify the buildReady diagnostic rewrite Skip the macrotask wait entirely when a composition registers no buildReady keys — the common case, since it's opt-in. Nothing to settle means nothing to wait on. Collapse buildZeroDurationDiagnostic's inline parameter type into the existing HfDiagnostic interface instead of maintaining two copies of the same shape. * docs(engine): note the setTimeout(0)-thenable gap the review found An independent adversarial review, executing hostile inputs rather than tracing, found the macrotask wait can still misreport a thenable that resolves via its own internal setTimeout(0), since the diagnostic's own timer is registered before the thenable's. Bounded and self-correcting (only feeds the message text, and the diagnostic re-runs every ~1s), so documenting the boundary rather than chasing an unbounded fix. |
||
|
|
856ac34c53 |
test(engine): time the bun --eval spawn in resolveHeadlessShellPath tests (#4078)
These tests spawn a bun process to check platform-specific chrome binary resolution, and have hit the default test timeout on Windows CI runners before. The log only ever shows the bare timeout, with no way to tell whether the spawn itself was slow or something else in the test body stalled. Wrapping each spawn in console.time/ console.timeEnd costs nothing when the test is fast and names the slow step the next time it isn't. No behavior change. |
||
|
|
172cac433d |
docs(engine,producer): describe the opt-in duration cap where the gate's callers read it
The `parallelStreamingAvailable` doc comment still presented the 240 s cap as unconditional; since streamingEncodeDurationCapEnabled landed the cap is an operator opt-in, so with stock config long compositions are eligible for the DE parallel router and HF_CAPTURE_PARALLEL_STREAM too. The config comment now also states that PRODUCER_STREAMING_ENCODE_MAX_DURATION_SECONDS=0 no longer disables streaming on its own and names PRODUCER_ENABLE_STREAMING_ENCODE=false as the supported off switch. Comment-only; no behaviour change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
68a605c826 |
feat(engine): add streamingEncodeDurationCapEnabled config flag, default off
The 240 s streaming cap guarded a total-render ffmpeg timeout that became an
inactivity timeout in
|
||
|
|
2ab7b86868 |
feat(engine): adaptive per-frame sample count for motion blur (#4031)
* feat(engine): adaptive per-frame sample count for motion blur Measured against the accuracy bar set in #4010 (72.4dB mean PSNR vs the 960fps shutter integral), the fixed default of 16 samples misses it for hard-edge content while paying nothing extra for static/slow content is still important. This picks a sample count per frame from measured motion, floor 16 ceiling 64 (issue #4029). Two probe captures at the true shutter-window edges (never a sample that enters the average) measure motion magnitude, mapped through a monotone step table to a sample count. Frames confirmed to move only non-spatial properties (opacity, colour) skip the probe and use the floor directly, since a fade's shutter integral is a smooth blend that does not need many samples. MotionBlurPlan's window (from shutter angle/phase alone) is now independent of how many samples fill it, which is what lets the count vary per frame without moving the window; motionBlurWindowIsStatic is bounded by the widest sample inset any chosen K could produce so the static-dedup safety guarantee holds for every K in [16, 64]. * refactor(engine): simplify the adaptive motion-blur sample selection Two findings from an independent code-reuse/quality/efficiency review of the previous commit: - armStaticDedup ran the GSAP-timeline walk (computeStaticFrameSet) twice on the same page when adaptive motion blur and static dedup both apply, once for each purpose. Shares the one result instead. - The non-spatial floor path read the sample count from adaptiveSampleCount(0), which only worked because the step table happens to be sorted with the first bucket covering zero. Reads the now-exported DEFAULT_SAMPLES_PER_FRAME directly instead. * fix(engine): read raw options for the adaptive motion-blur gate Independent review caught it: every armStaticDedup call site runs before finalizeSessionInit resolves session.motionBlur, so the adaptive-classification gate's `session.motionBlur && session.motionBlur.fixedSamplesPerFrame === null` was always false at runtime. The non-spatial probe-skip described in the PR never engaged outside tests that hand-set motionBlurNonSpatialFrames directly, bypassing real init order. Reads session.options.motionBlur (set unconditionally at session creation) instead, which carries the same "adaptive requested" information the resolved plan would have. Adds a source-order lock test plus direct coverage of computeStaticFrameSet's nonSpatialOnlyFrameSet, mutation-proved against the exact regression. * fix(engine): add xPercent/yPercent to the spatial-property list Round 2 of the independent review: the spatial-property list deciding whether a frame's motion needs the adaptive probe omitted GSAP's percentage-based position properties (xPercent/yPercent) and two 3D transform properties (perspective/transformPerspective) that two other lists already in this repo (packages/parsers/src/gsapConstants.ts, packages/studio/.../gsapAnimatesProperty.ts) both carry. A composition positioned via xPercent/yPercent would have been silently floored to K=16 with no probe and no log line, the same failure shape as the round-1 bug through a data gap instead of a dead gate. Moves the list to an exported SPATIAL_TWEEN_PROPERTIES array in motionBlur.ts, passed into computeStaticFrameSet's page.evaluate as data rather than duplicated inline, so it stays a single list and is directly testable. Adds tests that execute the real page-side closure (the existing tests all mocked the whole result and never exercised the matching code), mutation-proved against the exact gap found. * fix(engine): add rotate to the spatial-property list Round 3 of the independent review, comparing SPATIAL_TWEEN_PROPERTIES side by side against the two sibling lists rather than trusting the prior commit's summary: GSAP's individual "rotate" transform property (distinct from "rotation", carried in packages/studio/.../gsapAnimatesProperty.ts's GSAP_TRANSFORM_PROPS) was still missing, same failure shape as the two prior rounds. The same pass also raised transformOrigin, which is deliberately NOT added: gsapConstants.ts's own classifyTweenPropertyGroup excludes it as a pivot-point modifier with no independent visual effect, and a test now locks that reasoning in rather than leaving it to be "fixed" again by a future pass that does not read that far. |
||
|
|
9117b36471 |
fix(engine): skip motion-blur dedup when the shutter window moves (#4025)
The static-frame dedup reuses the previous frame's buffer when a frame is byte-identical to its predecessor. With motion blur on that test is not enough: the shutter window reads content either side of the frame instant, so a still frame next to a moving one reused a buffer that carries none of that motion. At 30 fps and 16 samples frame 11's window runs 10.75 to 11.25 at a 180 degree shutter and 10.06 to 11.94 at 720, so both reach past the frame instant and the default is affected too. motionBlurWindowIsStatic now owns whether a blurred frame may reuse its predecessor's buffer. It requires every frame the window reads from to be static, and covers the previous frame's window as well, because reuse claims the two blurred outputs are equal. The range is derived from the plan's own sample tick offsets, so it cannot drift from where the samples land. The guard's contract comes from #4013 by Dante-dan, which carried it before we did. One existing test fixture is widened: it asserted reuse with only the frame itself marked static, which the guard now correctly rejects. Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com> |
||
|
|
b64a7a3a67 |
feat(producer): make sub-frame motion blur reachable from a render (#4020)
* feat(producer): make sub-frame motion blur reachable from a render Second deliverable for issue #4010. The engine gained the accumulation path, but nothing set CaptureOptions.motionBlur, so the feature could not be switched on from a real render. RenderConfig now carries it and the producer threads it into the capture options. Doing that surfaced the defect that would have made it fail silently. "Does this render capture PNG frames" was spelled four times, every one of them re-derived from needsAlpha, which describes the OUTPUT container rather than the capture: probeStage picked the probe capture format renderOrchestrator picked the render capture format encodeStage built the encoder's frame_%06d input pattern, twice captureStage built the disk worker's frame extension Motion blur is a second, independent reason to capture PNG: the sub-frame samples are averaged pixel by pixel, so JPEG samples would be averaged after a lossy quantization. Add that reason to only one of those four and the encoder goes looking for frame_%06d.jpg against files written as .png, and the render finds no frames at all rather than reporting an error. So the rule now has one owner, resolveCaptureImageFormat, which names both reasons. Everyone downstream reads the resolved capture format instead of re-deriving it: the encoder and the disk worker share frameFileExtension with writeCapturedFrame, which is what actually names the files. needsAlpha keeps only the question it really answers, whether the output preserves alpha. captureStage no longer needs needsAlpha at all, which is the tell that the re-derivation was its only reason to know. No JPEG encoder is involved. The PNG capture path is already load-bearing, since every transparent-output render uses it and the streaming encoder already selects png as its input codec. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * fix(engine): resolve the motion-blur plan on every initialization exit A real render produced no blur at all. The plan was correct, the producer wiring was correct, and the sub-frame sample times were genuinely distinct, but K=16 and K=64 rendered identically to no blur because neither ever ran. initializeSession has two exits: screenshot mode finishes its own block and returns, every other capture mode falls through to the end. The plan was resolved at the fall-through exit only, and screenshot is the one capture mode motion blur supports, so on exactly the path the feature runs on session.motionBlur was never assigned. captureFrameCore then read it as undefined and took the single-capture branch: one capture per output frame, no accumulation, no error. finalizeSessionInit is now the single owner of what finishing initialization means. It sets the plan and isInitialized together and is called at both exits, so a third exit cannot mark a session ready while forgetting the plan. Why the existing tests passed, which is the part worth keeping: the accumulation test set session.motionBlur directly on a stub session. It exercised the mechanism while bypassing the only code that decides whether the mechanism runs at all. A test that proves something works and never asks whether it is reachable is not end to end, however much behaviour it drives. Two tests. A structural guard that session.isInitialized = true appears exactly once and that the same place resolves the plan, which fails if the defect is reintroduced at either exit. And a sample-distinctness assertion that hashes the K buffers captured within one output frame and requires K distinct ones, which covers the different failure of every sub-frame seek landing on one instant. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> --------- Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com> |
||
|
|
462f9db73d |
feat(producer): add hls output format (#4014)
* feat(engine): add packageHls for HLS VOD packaging Adds `packageHls` next to `muxVideoWithAudio`: stream-copies an already-encoded H.264 video and optional AAC sidecar into an HLS VOD directory (`master.m3u8` + split video/audio renditions, MPEG-TS segments) with `-c copy`, so no re-encode happens at package time. Depends on the locked GOP from the previous commit. `-hls_time` can only cut at existing keyframes, so `gopSize = segmentSeconds x fps` is what makes every segment exactly `segmentSeconds` long and IDR-aligned. A test packages an unlocked encode to pin that: the same call yields one 4s segment instead of four 1s ones. Deliberately omits the mp4 mux's provenance tags, which MPEG-TS drops and whose `-movflags` is invalid for `-f hls`, and `-avoid_negative_ts`, which would discard the AAC priming and shift audio against video (see issue 3487). A real-ffmpeg test asserts the first presentable audio sample lands on the first video frame. No caller yet; `format: "hls"` in the producer follows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(engine): zero HLS mux base and reject % in packageHls outputDir - Pass -muxdelay 0 -muxpreload 0 so the first segment starts at PTS 0 instead of the mpegts muxer's 1.4 s base; A/V alignment through the AAC priming packet is unchanged - Throw on an outputDir containing %, which ffmpeg expands as a printf conversion in -hls_segment_filename and fails with a misleading "No such file or directory" - Assert alignment on relative spacing rather than on how one ffmpeg build clamps the priming packet's negative PTS - Encode the test fixture with the full software GOP lock (lockedGopCodecParams), not just appendLockedGopArgs - Trim comments to the non-obvious rationale Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(engine): satisfy the ffprobe argv contract in packageHls test `producer/src/utils/ffprobeArgvContract.test.ts` sweeps every package at the source level and requires `--` to be the argv entry immediately before an ffprobe input path. The `probe()` helper here passed `join(outputDir, HLS_VIDEO_PLAYLIST)` in that slot, which the contract's comma-splitting parser reads as two entries, so the producer suite was red on this branch while the engine suite stayed green. `probe()` now takes the input path as a parameter and appends the terminator itself. The sibling ffmpeg `run()` helper switches from a spread-in-array-literal to `.concat`, because a verbosity-flag-then-spread literal is indistinguishable from the ffprobe wrapper shape the contract demands a terminator on. The matcher reads raw source including comments. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(engine): drop the audio-only variant from the HLS master playlist With `a:0,agroup:aud` in `-var_stream_map`, ffmpeg lists the audio rendition twice in `master.m3u8`: as the `#EXT-X-MEDIA:TYPE=AUDIO` group the video variant references, and again as a standalone `#EXT-X-STREAM-INF` variant with no `RESOLUTION`. That is valid HLS, but a player choosing by bandwidth can select it and play sound with no picture, and the VOD consumer asked for a single rendition. `stripAudioOnlyVariants` removes any variant tag without a `RESOLUTION` attribute together with its URI line; `packageHls` applies it to the master after a successful audio+video run. The rendition group and the audio playlist are untouched, so `audio.m3u8` is still referenced from the master. Pinned by a unit test on the playlist text and by the real-ffmpeg test, which now asserts exactly one `#EXT-X-STREAM-INF`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(producer): extract hdrMode warning and decision-log helpers `resolveEffectiveHdrMode` was already over the fallow complexity gate on `main` (21 cyclomatic / 25 cognitive / CRAP 462) and only surfaces now because the HLS format work needs to touch its downgrade warning. Pull the downgrade message into `formatDowngradeWarning` and the "which mode and why" info line into `logHdrDecision`, which puts the function under the threshold. No behavior change: every log string is byte-identical and `hdrMode.test.ts` asserts them unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(producer): add hls output format Adds `format: "hls"` to `@hyperframes/producer`. An HLS render is the same opaque H.264 + AAC encode as `mp4`, stream-copied at assemble time (no re-encode) into a directory: `master.m3u8`, `video.m3u8` + `video_%05d.ts`, and `audio.m3u8` + `audio_%05d.ts` when the composition has audio. The load-bearing part is that `-c copy -f hls` can only cut a segment at an existing keyframe, so the encoder's GOP is locked to `hlsSegmentSeconds x fps` frames. `resolveHlsEncoderGopLock` returns that pair as one value spread into both encoder inputs, so the streaming and disk/chunked paths cannot disagree — a render reaching only one of them still succeeds, it just emits variable-length segments silently. - `renderFormat.ts`: `hls` is opaque (`outputNeedsAlpha` is now `!== "mp4" && !== "hls"` — missing this flips HLS to RGBA screenshot capture with nothing failing), supports page-side compositing, and `outputUsesH264Pipeline` replaces the three `outputFormat === "mp4"` router gates. - `hlsConfig.ts`: `hlsSegmentSeconds` (default 4, positive integer), the GOP derivation, and the fail-fast gates. `force-hdr` is rejected (v1 is SDR-only; HDR10 needs HEVC in fMP4), and so is `useGpu` — the GOP lock is software-only, so a GPU HLS render segments correctly by accident on videotoolbox and wrong on nvenc. - `assembleStage.ts`: `hls` branches to `packageHls` instead of mux/faststart, after the same `padOrTrimAudioToVideoFrameCount` the mp4 mux uses. - `artifactTransaction.ts`: a directory artifact can now carry a `probeTarget` + `requiredFiles`, so the #3395 truncation gate still runs for HLS against `video.m3u8`. `expectedFrames` is inert there (ffprobe reports no frame count through the hls demuxer); duration carries it. - `renderRequest.ts`: accepts `hls` + `hlsSegmentSeconds`; distributed conversion rejects it alongside `gif`. `DistributedFormat` is unchanged. `RENDER_REQUEST_VERSION` stays at 1: the new field is optional, the parser ignores unknown option keys, and an older worker already rejects `format: "hls"` with "Render request format is invalid". - `hdrMode.ts`: the non-mp4 SDR downgrade warning no longer claims "HDR + alpha" for `hls`; the helper extraction landed in the preceding refactor commit. Verified: `hlsRender.integration.test.ts` renders a real 10 s composition and asserts 3 segments, the first two exactly 4.000 s, every segment starting on an IDR frame, and PTS 0. Removing the GOP wiring collapses it to 2 segments, so the test is load-bearing. Both producer lanes (unit and integration) run on every PR in `ci.yml`, so that wiring check is on the PR gate. CLI flags and docs are deliberately left to the next PR in the series. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(engine): close the HLS master playlist rewrite race CodeQL flagged two high-severity alerts on the `stripAudioOnlyVariants` rewrite: a check-then-use race (`existsSync` then a separate path-based read and write) and insecure creation of a file at a predictable path in the OS temp dir. Both came from resolving `master.m3u8` three times. Open it once with `r+` and owner-only mode — which neither creates the playlist nor widens its permissions, ffmpeg owns that — then read, truncate and write through that one descriptor. Absence is still tolerated for the argument-level tests that stub ffmpeg, now via ENOENT rather than a prior existence check. No behavior change: `hlsRender.integration.test.ts` still renders a real composition with audio and asserts the stripped master playlist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
0a1c6f5a45 |
feat(engine): sub-frame multi-sample motion blur, capture-level accumulation (#4019)
Deliverable one of issue #4010: average K sub-frame captures into each output frame so rotation, scale and opacity blur the way After Effects does, which the translation-only catalog component (#4008) cannot do. Opt in per render with CaptureOptions.motionBlur; absent means the capture path is unchanged. The blocker was quantizeTimeToFrame flooring every seek onto the output frame grid, which collapsed all K samples onto one instant and made the averaged frame byte-identical to the unblurred one. quantizeSeekTime now owns that decision at all three quantize sites (runtime player, its no-timeline fallback, and the init transport) and refines the grid by an integer subdivision count, which can only contain the output grid, never move a frame-time instant off it. The finer grid rounds rather than floors: a sample time round-trips through the page bridge as a float, and past a few million ticks that error exceeds the floor epsilon. Accumulation sits inside captureFrameCore AFTER the static-dedup early return, so a static frame still costs one reused buffer and only moving frames pay K. It is not a before-capture hook, which would have disabled dedup outright. Callback invariant: one eventful seek per output frame at the frame time, then K suppressed sample seeks, then a suppressed restore to the frame time. The restore matters because a suppressed seek still moves the playhead, so without it the next frame's eventful seek would swallow the callbacks inside the shutter window. Scope of this deliverable: screenshot capture mode and PNG output. beginframe, drawelement and JPEG encoding are rejected with a named error rather than silently rendering unblurred. Video content is rejected for the same reason it is a documented limitation: injected frames are extracted per output frame and cannot follow a sub-frame seek. Blend defaults to sRGB, matching AE's non-linearised 8-bpc working space and the catalog component. Linear light is an explicit opt-in. Samples are folded into a running average as they are captured rather than held until the end, so peak memory is one accumulator plus one decoded sample instead of all K: at 1080p that is a constant 41 MB rather than 141 MB at 16 samples and 540 MB at the 64 maximum, on every frame of the render. PNG chunk CRC32 now has one owner in utils/crc32.ts, shared by the new encoder and ffprobe's metadata reader. The encoder previously imported zlib's crc32 by name, which throws at module evaluation on Node 22.0 and 22.1; ffprobe already carried the guarded version and the comment explaining why. The sRGB transfer function is likewise exported once and shared. Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com> |
||
|
|
b4f1929cf0 |
feat(engine): lock GOP on every encoder path (#3973)
## What this does Video encoders normally choose their own keyframe positions. A keyframe is a frame that can be decoded on its own, and you can only cut a video cleanly at one. The engine already has a flag, `lockGopForChunkConcat`, that forces a keyframe every `gopSize` frames so chunk files can be joined with `ffmpeg -c copy`. Today that logic is copy-pasted in two places inside `buildEncoderArgs` and does not exist at all in the streaming encoder. This PR: 1. **Extracts the locked-GOP logic into three small shared helpers** in `chunkEncoder.ts`: `resolveLockedGopSize` (validation), `appendLockedGopArgs` (the `-g` / `-keyint_min` / `-sc_threshold` / `-force_key_frames` args), and `lockedGopCodecParams` (the x264/x265 param fragment). `buildEncoderArgs` now calls them instead of inlining the same block twice. 2. **Adds the same flag to the streaming encoder** (`StreamingEncoderOptions.lockGopForChunkConcat` + `gopSize`), which is the path most renders under four minutes actually use. It emits the identical args as the disk encoder, including `-bf 0` for h265 under a lock so B-frame reordering cannot reintroduce negative DTS at a boundary. ## What does not change - Any render that does not set the flag produces a byte-identical ffmpeg arg list. The flag defaults to `false` everywhere. - Nothing in the tree sets the flag yet except the distributed chunk worker, which already used the disk encoder. No user-facing behavior changes. - GPU encoders, VP9 and ProRes streaming encodes continue to ignore the flag, exactly as `EncoderOptions` documents. An earlier revision of this PR made GPU encoders honor it; that was dropped because the path is unreachable until HLS lands and was verified only on videotoolbox. ## Why Groundwork for `format: "hls"` in `@hyperframes/producer`. HLS packaging stream-copies the encoded video into fixed-length segments with `-f hls -c copy`, and the segmenter can only cut on keyframes the encoder already produced. Locking the GOP at encode time is what makes every segment land exactly on a 4-second boundary without re-encoding. The follow-up PR adds the format itself and should reject `--gpu` for `hls` in v1. ## Testing - `chunkEncoder.test.ts`: the three helpers tested directly; existing locked-GOP and GPU no-op tests unchanged. - `streamingEncoder.test.ts`: new `lockGopForChunkConcat` block covering default off, libx264, libx265, and invalid `gopSize`. - Engine typecheck, oxlint, oxfmt clean. Full engine suite matches main's baseline (the 4 `ffprobe.test.ts` HDR fixture failures are pre-existing on `main`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
|
ef48485b91 |
fix(engine): tone-map HDR frames for forced SDR (#3981)
Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com> Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com> |