Commit Graph
4470 Commits
Author SHA1 Message Date
Vance IngallsandClaude Opus 5 4e4cfe7ee5 feat(producer): capture segments with N parallel workers off a shared queue
Each worker owns a session (recycled and retried per 2c) and pulls whole
segments in index order. Segments finish out of order, so the concat list is
built from the plan rather than from completion order — ordering never
reaches the encoders.

Three guards, each mutation-checked: every worker settles before any session
is closed (closing one out from under a live worker orphans its ffmpeg and
races the CDP connection); a multi-worker render never takes the
single-encoder fallback, because other workers are already encoding work it
would discard; and the queue is shared, so no frame is captured twice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:07 -07:00
Vance IngallsandClaude Fable 5.1 763ea8f97f fix(producer): call closeCaptureSession through its live binding in segmented defaults
The default deps object captured the engine's closeCaptureSession at module
load, so a test that swaps the engine module after this file is evaluated
never saw the close. The Phase 2a test that checks the probe is closed after
a first-encoder spawn failure failed from this layer upward for that reason.
Calling through the import binding restores the check; production behaviour
is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:53:06 -07:00
Vance IngallsandClaude Opus 5 79a4b02d78 test(producer): record the Phase 2c gate result
Recycling the browser on every segment produced a byte-identical file to
the single-session render, which is what makes the target-loss retry safe:
a segment re-captured on a fresh session is indistinguishable from one that
never failed. Five launches cost ~5 s, and each recycle reported a renderer
RSS peak of ~1.4 GB being returned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:06 -07:00
Vance IngallsandClaude Opus 5 c87b4d1b47 feat(producer): wire the segment browser-recycle cadence and retry telemetry
HF_SEGMENT_BROWSER_RECYCLE defaults to 3 segments, which keeps a session
under the 10k-frame band where field crash rates pass 4%. segmentRetries
lands on live capture observability, so a render that retried still reports
it if it later dies.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:06 -07:00
Vance IngallsandClaude Opus 5 7d214e96ec feat(producer): recycle the browser between segments and retry target loss once
A segment boundary is the only safe point to hand capture a fresh browser,
which is what makes the long-render failure class survivable: the session
restarts every N segments, and a Chrome target loss mid-segment re-captures
that segment once on a new session instead of failing the render.

Retry is deliberately narrow. Encoder and authoring errors reproduce on a
fresh browser, so retrying them doubles the time to the same failure — all
three guards (retry exists, retry is classified, cadence boundary) are
mutation-checked. The partial segment file is deleted before the retry so a
later resume cannot see it, and onSegmentComplete still fires only after a
successful close, so the manifest never records a partial segment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:05 -07:00
Vance IngallsandClaude Opus 5 61223223d4 feat(producer): target-loss classifier and browser recycle cadence
Only the Chrome target-loss class is worth retrying a segment for: encoder
and authoring failures reproduce on a fresh browser, so a retry just doubles
the time to the same failure. Both mutation-checked, including that a
non-Error is never classified as retryable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:05 -07:00
Vance IngallsandClaude Fable 5.1 e7c4eb52b9 fix(producer,cli): hash every byte-affecting segment input; pin the resume-flag plumbing
Review findings on the Phase 2b PR. Three inputs that change segment bytes
were missing from computeSegmentPlanHash, so --resume could splice
mismatched segments: the hardware encoder (h264_nvenc and libx264 both
report codec "h264"), the device-scaled capture size (the hash used the
composition's CSS size), and the motion-blur options (sample count changes
every averaged frame). All three are hashed now, each with a test that
fails if it is dropped. The hash and the encoder read the image format
from one binding.

The plan -> options -> request -> config chain that once dropped --resume
and --keep-segments had no test. renderOptionsFromPlan is now a pure,
exported hop with a test, renderConfigFromRequest has one for the same
flags, and the batch path, which was still dropping both flags, carries
them too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:53:04 -07:00
Vance IngallsandClaude Opus 5 ae6aedf1ca fix(cli): pass the segmented resume flags from the plan into render options
execute.ts is where the plan becomes RenderOptions, and it dropped both new
flags, so --keep-segments deleted the segment directory anyway and --resume
never resumed. Nothing failed: the render just silently re-captured
everything. Caught by the Phase 2b gate, which is the only check that spans
plan -> options -> request -> config; the gate doc now says so.

Gate result recorded: killed at 2 of 6 segments, resume skipped both,
finished in 3m32 against 5m13, and the output was byte-identical to the
uninterrupted render.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:04 -07:00
Vance IngallsandClaude Opus 5 a560f498cd feat(render): --resume and --keep-segments for segmented capture
Segments and their manifest live under renders/.hf-segments/<planHash>. A
resumed run re-validates every recorded segment before skipping it; a run
without --resume deletes the directory first, because a previous run's
segments must never be spliced into this output. The directory survives a
failure on purpose — that is the resume point — and is removed after a
successful render unless --keep-segments.

The frame-count check derives from probed duration x fps: the engine has no
exact-count reader and -count_frames decodes the whole file, while the only
thing this must catch is a segment missing frames, which shortens duration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:04 -07:00
Vance IngallsandClaude Opus 5 8c14169efd feat(producer): segmented stage skips validated segments and reports completions
segmentOutputPath now takes the segment directory, so segments can live in
a stable resume directory instead of the disposable work dir. Skipped
segments are still concatenated in order, and the fallback-on-spawn-failure
rule moves from "segment 0" to "the first segment this run still has to
capture" — on a resume that is not index 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:03 -07:00
Vance IngallsandClaude Opus 5 5daaae8498 feat(producer): segment manifest with content-addressed plan hash
Resume reuses encoded bytes, so a wrong reuse corrupts the output silently
instead of failing. Three guards, each mutation-checked: the plan hash
covers every input that determines segment bytes and is separator-joined so
adjacent fields cannot alias; a reused segment must still exist at its
recorded size; and its probed frame count must equal the slice length.

Type guards use `in` + Reflect.get rather than the plan's
`as Record<string, unknown>`, which the repo's no-cast rule forbids.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:03 -07:00
Vance IngallsandClaude Fable 5.1 cb28942110 fix(producer): drop the closed probe reference when segmented capture falls back to streaming
Review finding on the Phase 2a PR: when the first segment's encoder failed
to spawn, the stage's finally had already closed the session it opened,
including a reused probe, but the orchestrator kept its probeSession
reference and handed the closed session to the streaming branch, which
would fail a stage later with a confusing "page closed". The fallback
branch now nulls the reference; the stage test pins that the probe is
closed on that path. Also covers the negative HF_SEGMENT_FRAMES clamp the
review noted was untested.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:53:02 -07:00
Vance IngallsandClaude Opus 5 7000e32942 test(producer): record the Phase 2a gate result and its real pass criteria
The spec's "PSNR >= 45 dB vs the single-encoder output" reads as a failure
on a healthy render: the measured average is 44.4 dB because segmented
capture forces a closed GOP with scene-cut detection off, and two CRF
encodes of identical frames differ regardless. The boundary frames are the
HIGHEST-PSNR frames in the series (48-62 dB forced IDRs), which is what the
gate should actually assert.

Also records the cost this buys nothing back on: the segmented stage runs
the plain capture loop rather than the depth-2 worker-encode pipeline, so
the same fixture takes 5m16 against streaming's 2m43.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:02 -07:00
Vance IngallsandClaude Opus 5 4b54b5d26f feat(producer): route long renders to segmented capture behind HF_SEGMENTED_CAPTURE
Opt-in, single-worker, mp4/mov only, and not for HDR or shader-transition
routes — those run their own compositor and webm's VP9 concat-copy is
fragile across ffmpeg versions.

A first-segment encoder spawn failure replans onto the plain streaming
path and falls through to the branch below, so the opt-in cannot make a
render fail where the default would have succeeded. A failure after that
propagates: per-segment retry is Phase 2c.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:01 -07:00
Vance IngallsandClaude Opus 5 8391abd672 feat(producer): sequential segmented capture stage
One closed-GOP encoder per segment writing segments/segment_NNNNN.mp4,
then a concat-copy into the video-only file the assemble stage muxes. The
GOP is each segment's own frame count, so every concat boundary is an IDR;
the tail segment is shorter and says so.

A spawn failure on the first segment returns success:false so the caller
can still replan onto the single-encoder streaming path. A later one
throws: earlier segments are already encoded, and silently discarding them
would drop frames from the output.

Reuses the sequential stall watchdog from the streaming stage rather than
running unguarded; both helpers are exported for that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:01 -07:00
Vance IngallsandClaude Opus 5 d7398517bd feat(producer): sdr_segmented capture plan kind
Inert until the orchestrator passes useSegmentedCapture. replanAfterFailure
handles the kind for every failure, including capture_failure, which would
otherwise reach the "cannot apply" throw guarding the non-streaming kinds:
a drawElement failure keeps segmenting on the screenshot engine, anything
else drops to a plain streaming render.

CapturePlanTarget deliberately does not gain the kind — routing fallbacks
are streaming/disk only, and adding it there would make
`useStreamingEncode: fallback.kind === "sdr_streaming"` silently false for
a segmented target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:00 -07:00
Vance IngallsandClaude Opus 5 46d852cc93 refactor(engine): extract concatVideoFiles from the chunked encoder
Segmented capture needs the same concat-demuxer stream copy the chunked
encoder already does. Behaviour preserved, including the
external_interruption classification the caller maps to a retryable
failure: the helper reports it rather than swallowing it. The concat list
keeps living beside its inputs, because concurrent encodes share an output
directory but not a chunk directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:00 -07:00
Vance IngallsandClaude Opus 5 9c3cff4960 feat(producer): segment planner for long-form capture
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:59 -07:00
Vance IngallsandClaude Fable 5.1 5f373894d0 fix(producer): disk preflight names only causes that reach it; refresh the router field's docstring
Review follow-ups on the Phase 0 and Phase 1a PRs:

- The disk-headroom message blamed HDR and shader-transition output for a
  throw those renders can never reach (they run hdr_layered, a separate
  loop). It now lists png-sequence/gif, multi-worker screenshot capture,
  and the three ways streaming is switched off, including the duration cap
  and its threshold.
- RenderCaptureObservability.captureParallelStream still described the
  pre-split semantics ("the kill switch was on"). It now says BeginFrame
  routes by default, screenshot only with the env, that eligible_off sizes
  the held-back screenshot cohort, and that the label can disagree with the
  platform-only captureMode on Linux with system Chrome or DPR > 1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:59 -07:00
Vance IngallsandClaude Opus 5 bc2a8b5e0d docs(cli): document the parallel-stream router and its capture-mode split
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:58 -07:00
Vance IngallsandClaude Opus 5 1ae359ca00 feat(producer): stream multi-worker BeginFrame renders by default
Splits the non-DE parallel-stream router default by the capture mode the
render will actually use. BeginFrame (Linux headless-shell) defaults on:
its opt-in cohort fails at 0.05% over 11.5k renders. Screenshot capture
(macOS/Windows) stays behind HF_CAPTURE_PARALLEL_STREAM=true, because its
opt-in cohort fails at 5.5% against a ~1.1% baseline in two classes this
path owns — the parallel stall watchdog hard-fails with no retry, and a
dying encoder surfaces as a bare write EPIPE. Phase 1b flips it once those
are fixed.

resolveParallelCaptureMode mirrors the engine's preMode rather than reusing
resolveObservedCaptureMode, which is platform-only and would have
default-enabled Linux-with-system-Chrome and Linux-at-DPR>1, whose real
capture path is the screenshot one. The routed telemetry value now comes
from the same helper, so the cohort cannot disagree with the predicate that
routed it, and eligible-but-held-back renders keep emitting eligible_off.

The preflight message named --workers 1 as the fix for every disk-path
render; with the duration cap enabled a --workers 1 render lands there too.
It now names each override that can cause it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:58 -07:00
Vance IngallsandClaude Opus 5 ddbc9a8a98 feat(cli): emit Chrome memory, capture path and segment fields on render events
Both events carry the live observability values, so a render_error that
never built a perf summary still reports the last known Chrome memory.
render_complete prefers the aggregate from RenderPerfSummary.chromeMemory,
which covers every worker rather than whichever session sampled last.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:25 -07:00
Vance IngallsandClaude Opus 5 2cf8024a7c fix(engine): read Chrome process info from the browser target, not the page
SystemInfo.getProcessInfo is served only by the browser target; a
page-target session rejects with "is only supported on the browser
target". The sampler swallows probe failures by design, so this shipped as
a permanently empty reading: a 10 s debug render reported chromeMemory
null with capturePath already correct. Against the browser target the same
render reports browser 105 MB / renderer 359 MB over 3 samples.

readChromePids is extracted so the target choice is pinned by a test whose
fake rejects a page session the way Chrome does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:25 -07:00
Vance IngallsandClaude Opus 5 119dafc005 feat(producer): carry Chrome memory and capture path on observability and perf summary
Live route through updateCaptureObservability survives a crash, which is
the case this telemetry exists for: the failure path never builds a
RenderPerfSummary. The aggregate route through CapturePerfSummary (max of
per-session peaks, sum of samples) feeds render_complete.

capturePath is recorded inside syncCapturePlan rather than beside the
streaming-encode gate log, so a render that replans onto disk reports the
path it actually captured on; capturePathForPlanKind is exhaustive over
CapturePlan["kind"] so a new plan kind cannot silently inherit the wrong
label.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:25 -07:00
Vance IngallsandClaude Opus 5 428922162d feat(engine): sample Chrome process memory per capture session
Start a 2 s unref'd sampler when a session finishes initializing, stop it
in closeCaptureSession, and report peaks on CapturePerfSummary. Pids come
from puppeteer (browser) and CDP SystemInfo.getProcessInfo (renderer, GPU),
so the long-render "Target closed" class can be correlated with Chrome
memory instead of guessed at. Disable with HF_CHROME_MEMORY_SAMPLER=false;
cadence HF_CHROME_MEMORY_SAMPLE_MS.

The start hook is finalizeSessionInit rather than the tail of
initializeSession: that helper is already the single owner of "session is
ready" across both of initializeSession's exits, so the early screenshot
return cannot silently skip sampling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:03 -07:00
Vance IngallsandClaude Fable 5.1 a35168eddd fix(engine): sampleOnce() joins an in-flight sample instead of skipping it
With a boolean reentrancy flag, the Task 3 close sequence
(`stop(); await sampleOnce()`) returned immediately whenever an interval
tick was still awaiting CDP/ps, so getCapturePerfSummary read stats from up
to one interval earlier and onSample fired once more after the session had
closed. Reproduced with a 350 ms getPids on a 100 ms interval: close
resolved with samples=0 and the callback landed afterwards.

The guard now keeps the in-flight promise and returns it to any caller, so
an interval tick still never stacks a second round-trip (the existing
reentrancy test holds) while a close-time call resolves only after the
running tick has merged. Deviates from the plan's reference body, which
used `let inFlight = false`; the ChromeMemorySampler interface and all
names are unchanged, only the sampleOnce() contract is stronger.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:02 -07:00
Vance IngallsandClaude Fable 5.1 7cf57fa42e test(engine): pin the unref'd sampler interval via a cast-free timer seam
Three independent reviewers found that deleting timer.unref() in
createChromeMemorySampler left the suite green: the plan's Global Constraint
(fire-and-forget on an unref'd interval) had no failing test. The new test
captures the real NodeJS.Timeout through the setIntervalFn seam and asserts
hasRef() is false after start(); it goes red when the unref() call is
removed.

The seam could not be used without a cast as typed in the plan
(`typeof setInterval`): with the DOM lib in scope that type also carries the
`(handler, timeout) => number` overload, which a capturing wrapper cannot
satisfy. setIntervalFn / clearIntervalFn keep their names but are now typed
to the two-argument shape the sampler actually calls; the global timers are
assignable to those shapes, so the defaults are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:02 -07:00
Vance IngallsandClaude Fable 5.1 53403a5896 feat(engine): chrome memory sampler with live onMemorySample capture option
Pure service for Phase -1 of the long-form render plan: merges per-pid RSS
rows into browser/renderer peaks, last-sample total and GPU presence on an
unref'd interval with a reentrancy guard, so a CDP or ps failure can never
reach the capture loop. CaptureOptions.onMemorySample is the live route the
producer will wire so a mid-render crash still reports memory state. Nothing
instantiates the sampler yet (Task 3); no behaviour change.

Beyond the plan's tests, pins the double-start, empty-sample and catch
guards directly, since the interval-based error test stays green when the
catch is removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:02 -07:00
Vance IngallsandClaude Fable 5.1 1862c34399 test(engine): fix readonly array type in the pid-filter test
`readonly string[][]` parses as `ReadonlyArray<string[]>`, so `calls.push`
was a TS2339 in 5214552cd. It slipped through because the engine tsconfig
excludes `src/**/*.test.ts` and vitest strips types; caught by running tsc
on the test file explicitly. `Array<readonly string[]>` is what the
neighbouring "one ps call" test already uses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 3cee3b94f7 fix(engine): read /proc/<pid>/status for RSS on linux instead of shelling out to ps
Every render image (packages/cli/src/docker/Dockerfile.render,
packages/gcp-cloud-run/Dockerfile, examples/k8s-jobs/Dockerfile.example,
Dockerfile.test) is node:22-bookworm-slim with no procps, so `ps` is ENOENT
in containers and sampleProcessRss would return [] on every tick: the whole
cloud fleet would report chrome_memory_samples: 0, indistinguishable from
"no Chrome pids". Linux now reads VmRSS from /proc/<pid>/status with a
per-pid try/catch, so one renderer that exited between discovery and
sampling omits only itself (with `ps` a wholly-dead batch fails the call).
No child process per tick on the primary server platform. Same precedent as
the CLI's orphanCleanup.ts / processTree.ts. `ps` stays for darwin and
other POSIX; `tasklist` for win32. The reader is an optional fourth
parameter with a default, so Task 2/3 callers are unaffected.

Scope note: spec section 2.5 records the Target-closed population as
docker=false throughout, so this closes a render_complete completeness gap
on cloud, not a coverage gap on the crash class this phase targets.

Verification: 15/15 vitest; four linux-branch mutants (per-pid catch
rethrows, regex loses /m, kb <= 0 weakened, path /status -> /stat) each
fail at least one test, restored green. Real-process smoke check on darwin:
sampleProcessRss([process.pid, <exited pid>]) returns one sample for the
live pid (BSD ps exits 0 with partial output when a valid-range pid is
gone). The linux path is unit-tested against a /proc/<pid>/status fixture
only; no Linux host or VM was available in this session to execute it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 645cbdce4e test(engine): pin the pid filter, dedup and tasklist memory guard in sampleProcessRss
Review of 76d8e2682 found three surviving mutants: removing the
`Number.isInteger(p) && p > 0` filter, dropping the `new Set` dedup, and
weakening `kb <= 0` to `kb < 0` all passed the plan's 8 tests. The filter is
the guard that matters most: Task 2 feeds `browser.process()?.pid`, and a
single NaN or undefined reaching `ps -p` makes ps exit non-zero, which the
never-rejects catch turns into `[]` for every pid in the sample.

Mutation check: each of the three mutants now fails exactly one test
("drops invalid pids and duplicates before calling ps" for the filter and
dedup, "returns [] when the memory column is not a positive number" for the
tasklist guard); restored source is 10/10 green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 bc70156f00 feat(engine): zero-dependency per-pid RSS sampler (ps / tasklist)
First piece of the long-form render Phase -1 telemetry: the Chrome memory
sampler needs resident set size per pid without adding pidusage or
systeminformation. `ps -o pid=,rss=` covers POSIX in one call for every pid;
`tasklist /FO CSV` covers win32 one pid at a time. The function never rejects
and omits pids that exited between discovery and sampling, so a failed sample
can never reach the capture loop.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:00 -07:00
Vance Ingalls 2b2b999e62 Merge pull request #4109 from heygen-com/lfr/02-phase0-duration-cap
feat(producer): stream long single-worker renders by default; duration cap behind a flag (Phase 0)
2026-09-18 20:50:36 -07:00
Miguel Ángel a25da94b77 refactor(studio): remove the storyboard view and its Storyboard/Preview switch (#4140)
* refactor(studio): remove the storyboard view and its Storyboard/Preview switch

* refactor(studio): fold the thumbnail output branches left after the storyboard mode went

* docs(studio): drop leftover storyboard references after the view was removed

* ci: allow the storyboard view deletions in the no-deletions guard

* docs(studio): wrap an over-long comment line in the caption overlay helper

* style(studio): reindent App.tsx after the view-mode wrapper was removed
2026-09-18 23:47:12 -04:00
Miguel Ángel 2db126d4c8 docs(skills): one owner doc for everyday edits and a hyperframes-studio skill (#4125)
* docs(skills): one owner doc for everyday edits and a hyperframes-studio skill

Agents got contradictory advice on volume fades and were blocked from splits
and copies by the Ask-agent guardrails.

- creator-editing-recipes.md is the single owner: adds align-to-event,
  copy-a-group and swap-media recipes and the one volume rule (use the lane)
- variables-and-media, data-attributes (skill and CLI copy), the gsap adapter
  and tracks-and-clips now defer to it
- Ask-agent guardrails allow the neighbour edits a split, retime, copy or swap
  requires and point at the owner doc
- new hyperframes-studio skill: sub-compositions for every scene, one caption
  track, one element kind per track, safe zones
- drift test lints every html example in the owner doc with the real linter
  and fails on any doc teaching a volume tween

* refactor(skills): drop unused tween allowlist from recipes drift test

* docs(skills): name the track kind attribute in the studio skill

* docs(skills): stop claiming lint rules that do not exist yet and fix track numbering

Review fixes:

- studio skill: drop the severity claim and the preview-overlay claim,
  group tracks by data-track-kind instead of numeric ranges that clashed
  with the recipes
- recipes: audio data-start is root time only in the root composition;
  scene-local inside a sub-composition file
- guardrails: timeline edits are exempt from the element-only and timing
  lines; the pointer names the skill reference, not a repo path
- montage doc defers to the volume lane; skill count is 21 everywhere
- drift test also scans the montage doc

* fix(skills): the ducking generator emits a volume lane and captions defer to the studio safe zones

- audio-duck.mjs prints a data-automation lane (clip-local times from the
  target data-start) instead of a timeline volume tween; --json adds the lane
- operations.md shows the generated lane; the drift test scans it
- embedded-captions safe-zone lines point at the studio skill, which owns
  the values

* chore(skills): regenerate the skills manifest after rebase

* docs(skills): remotion translation docs point volume ramps at the automation lane

* docs(skills): studio safe zones use the action-safe 90 and title-safe 80 boxes

* docs(skills): caption docs defer to the studio safe boxes

* chore(skills): regenerate the skills manifest after rebase

* fix(skills): studio skill names the safe margins; guardrail test matches the timeline-edit wording
2026-09-18 23:19:40 -04:00
Vance Ingalls 34644a1993 Merge pull request #4148 from heygen-com/release/v0.8.50
chore: release v0.8.50
v0.8.50
2026-09-18 19:56:18 -07:00
Miguel Ángel b44f1f85dc feat(lint): flag nested structure, missing timing and caption tracks on the root composition (#4146)
* feat(lint): flag nested structure, missing timing and caption tracks on the root composition

One rule set for the root composition: nested layout inside a top-level
element, a timed element or media without data-duration, captions not marked
data-track-kind=captions, and captions spread over more than one track.
Findings are errors when the host is Studio and warnings elsewhere.
Sub-composition files are left free to hold layout.

* test(lint): keep the structure rules silent on the shipped catalog and skills

Runs the rules over every registry and skills html file and asserts no
findings, and treats a bare media element with no timing as a static layer.

* fix(lint): run the structure rules as Studio in both adapters and skip non-root files

The Vite Studio adapter now lints as the studio host, and HTML the project
lint did not treat as a root (components, snippets) is linted as a
sub-composition. Style, script and template content no longer count as nested
layout, sub-composition hosts need no duration, and a caption row without a
lane is not a second track.

* fix(lint): count legacy data-end as a length and pin the catalog test to real roots
2026-09-18 22:49:15 -04:00
Vance Ingalls 42f4afa367 chore: release v0.8.50 2026-09-18 19:41:06 -07:00
Vance Ingalls ea07f0e93b Merge pull request #4139 from heygen-com/sdk-cutover-decline-reason
fix(studio): a composition with a <canvas> no longer kills the SDK session
2026-09-18 19:39:22 -07:00
Miguel Ángel 7797c5ead4 feat(studio): the timeline shows one row per top-level element and no child rows (#4132)
* feat(studio): the timeline shows one row per top-level element and no child rows

* test(studio): cover the top-level id collection the timeline rows come from

* chore(studio): shorten the row-elements comment

* fix(studio): keep implicit layers as rows and say why a nested element cannot be hidden

* refactor(studio): drop implicit layers; rows are exactly what topLevelElements returns

* test(studio): untimed root wrappers are not timeline rows

* fix(studio): an empty timeline message clears the rows; drop the unused implicit contract

* fix(studio): an empty clip manifest reaches the timeline and clears the rows

* fix(studio): an empty clip manifest does not shrink the transport to its 1s floor

* chore(ci): allow deleting the child-row expansion hook and its test
2026-09-18 21:27:53 -04:00
Miguel Ángel 63d0ea4d28 fix(studio): show where a dragged file will land instead of a drop-to-add box (#4142)
Dragging a file over Studio covered the screen with a "Drop to add at the
playhead" box, but a drop lands at the pointer time on the target row. The
box is gone; the timeline now draws a landing preview at the exact start and
track the drop will use (a new row below the last lane when that is the
target), taken from the same placement function the drop commits.
2026-09-18 21:27:39 -04:00
Miguel Ángel ae0e8fcae1 fix(skills): teach and ship the remotion translation as sub-compositions (#4141)
* fix(skills): teach and ship the remotion translation as sub-compositions

The converter reference fixtures now emit one sub-composition per scene
under a mounting root, and the api map, sequencing guide and tier-2 README
teach that shape instead of the flat data-start div. SSIM per tier is
unchanged (t1 0.974415, t2 0.985993, t3 0.953275 to 0.953281).

* docs(skills): make every remotion translation doc teach the sub-composition shape

SKILL.md step 3, the sequencing guide (nested, series, crossfade), the api map
asset rows, and the parameters guide now match the shipped fixtures.

* docs(skills): say where scene cuts versus cross-scene fades live
2026-09-18 21:01:59 -04:00
Vance IngallsandClaude Opus 5 db78b4fe68 fix(studio): keep a literal relative require out of the canvas-stub docs
verify-packed-manifests scans packed JavaScript for relative import
specifiers with a set of regexes that do not strip comments, so the
`require('./canvas-shim.cjs')` inside the stub's doc comment read as a real
import of a file that is not in the studio tarball, and the Build job failed.

Describes linkedom's try/catch guard in prose instead. The same literal was in
the test file's header comment; CI only reported the first match, so both are
fixed here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 18:00:28 -07:00
Miguel Ángel 4bbcf650ae fix(player): perf harness waits for assets-ready before play instead of a fixed 5s (#4137) 2026-09-18 20:58:30 -04:00
Miguel Ángel f2976536de fix(skills): mount music-to-video motion primitives as sub-compositions (#4136)
* fix(skills): mount music-to-video motion primitives as sub-compositions

Each primitive's root now only mounts scene.html, so the root is built from
a sub-composition and the structure lint has nothing to flag. Renders match
the originals (31 of 37 identical, the rest above 54 dB PSNR).
Registry examples mark their caption layer with data-track-kind=captions.

* fix(skills): keep primitive variables and page styles where the runtime reads them

Composition variables move onto the sub-composition element, the dead
#scene rule is dropped, and the frame-worker doc names where page styles
live.

* fix(skills): shorten a carried-over comment in the kinetic-letter-in scene
2026-09-18 20:57:54 -04:00
Vance IngallsandClaude Opus 5 d5e1de9201 fix(studio): a composition with a <canvas> no longer kills the SDK session
linkedom's HTMLCanvasElement constructor calls createCanvas(300, 150) from the
Node-only `canvas` package, behind this guard:

    try { module.exports = require('canvas') }
    catch { module.exports = require('./canvas-shim.cjs') }

The guard only works with a runtime require. Vite resolves the specifier
statically, so the bundled form is `try { exports = <module> } catch {...}` — a
plain assignment that cannot throw. The catch never runs, linkedom's own shim is
bundled but unreachable, and createCanvas lands undefined. Verified in the built
artifact: the canvas module namespace carried only a `default` export.

Parsing any <canvas> tag therefore threw inside openComposition, so Studio lost
its SDK session for the entire composition. Every edit then fell back to the
server path, and the resolver shadow did not run either, which is why this was
invisible before v0.8.47 added studio:sdk_session_unavailable. In the 72h after
the flip it accounted for 1065 of 1544 session-open failures, across a large
share of 197 users. Canvas is common in compositions: Three.js, shaders,
particle effects, charts.

Aliases `canvas` to a local browser stub that mirrors linkedom's shim
(measurement-only; nothing in the SDK rasterizes through it). Reimplemented
locally rather than pointing at linkedom's commonjs/canvas-shim.cjs, which its
exports map does not expose.

Confirmed against a rebuild: the namespace now carries createCanvas, so the
call returns a stub instead of throwing.

Note the unit tests here pass with or without the fix — vitest resolves the
require at runtime, so only the bundled build was ever broken. The regression
guard therefore pins the alias itself, since nothing else in the suite can
observe the bundler.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:38:39 -07:00
Miguel Ángel 898d27869e feat(studio): preview ruler and safe-margin toggles (#4127)
* feat(studio): preview ruler and safe-margin toggles

* test(studio): cover guides placement in the preview pane and the caption band box

* feat(studio): safe margins are two nested 90 and 80 percent boxes with edge ticks

* fix(studio): safe box ticks centred on the line, no halo, tighter skill wording test
2026-09-18 19:47:27 -04:00
Miguel Ángel 14e5f57670 fix(core): script coalescing no longer moves an inline script past a src script (#4133)
* fix(core): script coalescing no longer moves an inline script past a src script

* test(core): pin how the script coalescers group inline runs

* test(core): simplify the pinned-script grouping test
2026-09-18 19:35:08 -04:00
Miguel Ángel c80ae95084 fix(studio): hovering or selecting a timeline clip no longer swaps its filmstrip (#4130) 2026-09-18 19:27:45 -04:00
Miguel Ángel 8dd697ab8f chore(ci): run check-docs-catalog in the Lint job (#4102)
* feat(catalog): browse the catalog as a live grid, not a static index

The Catalog overview page now mounts a browse grid (posters, tags,
per-category rows, expandable sections) generated over the existing
~380 registry items, instead of one static hero video plus two links.
docs/snippets/catalog-gallery.jsx and docs/catalog-gallery.css are
ported from a reference implementation of this same UX; the CSS keeps
its original comments as-is (NO_ESSAY_COMMENTS_OK: vendored file).

scripts/sync-docs-catalog.mjs and build-docs-gallery-nav.mjs are new,
generating the gallery's data and the Catalog tab's nav directly from
this repo's registry/*/registry-item.json and docs.json, so no sparse
clone or snapshot of another project is needed. scripts/check-docs-catalog.mjs
proves every catalog page from before this change still exists, on
disk and in the sidebar, with an identical page list.

No catalog page or snippet is removed; the previous overview content
(catalog-overview-player.jsx) is unused on this page but stays in the
repo.

* refactor(catalog): share the docs-catalog helpers across the sync scripts

Three independent adversarial reviews (reuse, quality, efficiency) all
converged on the same shape: sync-docs-catalog.mjs, build-docs-gallery-nav.mjs
and check-docs-catalog.mjs each hand-wrote the same docs-root resolution,
the same catalog-gallery-data.mdx parse, and the same Catalog-tab lookup.
Extracted into scripts/docs-catalog-shared.mjs.

sync-docs-catalog.mjs also ran build-docs-gallery-nav.mjs as a dynamic
import purely for its side effect, forcing a same-process re-read and
re-parse of the data file it had just written. build-docs-gallery-nav.mjs
now exports buildNav(docs), called directly with the path already
resolved; its CLI entry is guarded for standalone use.

slug() dropped the /^-+|-+$/ trim in favor of the character-scan trim
packages/core/src/tokenSlug.ts already uses for the same reason (CodeQL
flags that regex as polynomial ReDoS, js/polynomial-redos); not imported
directly since tokenSlug.ts is compiled TS with a different domain
(CSS variable naming) and no existing scripts/*.mjs imports packages/core
source at plain node runtime.

Fixed one bug found while re-verifying after the refactor: the recursive
nav walk wasn't idempotent against its own generated output — re-running
sync-docs-catalog.mjs a second time saw the "Catalog" wrapper buildNav()
adds and misattributed every flat, single-section group under it, 9
groups collapsing to 7. walk() now treats "Catalog" as transparent, the
same way it already treated "Overview". Verified: three consecutive runs
from a fresh origin/main checkout now produce identical 9 groups / 400
items each time, and check-docs-catalog.mjs still passes.

* chore(catalog): scrub an internal decision code from a code comment

* fix(catalog): stop losing a section's label when its nav is flattened

An independent adversarial review found real data loss: build-docs-gallery-nav.mjs
collapsed a group with exactly one section into a flat page list (a clean sidebar
for the common case), but sync-docs-catalog.mjs's walk() derives an item's section
from nav depth, so a second sync run over that flattened nav could no longer
recover the section name and silently fell back to the group's own label. Data &
charts / Data was the one real casualty in this catalog; Carousels and Blocks
happened to already have a section named the same as their group, so the loss
was invisible there.

Fix: only flatten when the section label equals the group label, so nothing
non-recoverable is thrown away. Verified with a deliberate mutation (reverted the
condition, reproduced the exact bug from a clean origin/main checkout, confirmed
the new check-docs-catalog.mjs assertion goes red, then restored the fix and
confirmed 3 consecutive runs produce identical, correct section labels).

check-docs-catalog.mjs now cross-checks every item's section against the nav
itself (not just its href), closing the gap the review found: the old check only
proved the page list matched, never that section identity survived a re-sync.

Also deletes docs/snippets/catalog-overview-player.jsx: unreferenced anywhere in
the repo now that catalog/index.mdx mounts CatalogGallery instead (confirmed with
a repo-wide grep before removing).

* chore(catalog): allowlist the deliberate catalog-overview-player.jsx deletion

* feat(catalog): live-hover preview, fuller sidebar, landing order

Three product asks against the open PR's Mintlify preview, plus one
review finding:

- Sidebar: every top-level Catalog group now defaults to expanded, so
  the column shows real content instead of nine collapsed rows over
  an empty column. Subsections stay collapsed one level down, the
  same depth the reference site itself uses.
- Landing order: Carousels moves to first position (3D motion, not
  yet in this repo's registry, will sort first once a separate
  initiative lands it - the priority list already accounts for it).
  Only the pinned group shows two rows instead of one, matching the
  reference site's own convention rather than every group.
- Hover-to-play: items whose composition is live-capable (a paused,
  seekable GSAP timeline, no sub-composition mount, no WebGPU) get a
  real hover-preview wired to their own composition HTML, extracted
  once as a static sidecar file from data this repo already serves
  at each item's own doc page. Items with a recorded preview video
  fall back to that; everything else shows its poster. Verified no
  WebGL/three.js item is ever marked live (238 live, 30 video, 132
  poster-only of 400).
- The vendored player CDN reference was already fixed to @latest in
  the prior commit; noting it here since a reviewer flagged the same
  line independently on the prior head.

docs/public/catalog/gallery-live/**/*.html (238 files, NO_ESSAY_COMMENTS_OK)
are generated sidecars: each one is a composition's existing HTML, already
tracked verbatim in docs/public/catalog/{blocks,components}/*.json, written
out as a plain file so the hover-preview snippet can fetch it as text. Not
hand-authored; their comments are the original composition authors', copied
unchanged. Regenerated by scripts/sync-docs-catalog.mjs on every run.

* refactor(catalog): merge the item and preview passes, one group-label source

itemFrom() and walk() computed the same group-label fallback expression
independently; itemFrom now returns it once alongside the item instead
of the caller re-deriving it. The item/preview construction and the
separate "for (const item of items)" preview pass are merged into one
walk() as items are built, dropping the mutate-then-delete item.live
scratch field. Byte-identical generated output, confirmed by diff.

* chore(catalog): scrub the private upstream repo name from code comments

An independent adversarial review found the same leak class as an
earlier fix (commit cfe14f813, an internal decision code) in four
provenance comments naming a private individual's private GitHub
repository and an exact commit sha, in a public repo. Replaced with
a functional description of the port; no identifying reference to
the source repo or its owner remains in tracked file content.

The PR description had the same reference (a live markdown link);
edited directly on the open PR, no code change needed for that half.

* chore(catalog): restore the adapted-from-elsewhere signal without the name

* fix(docs-catalog): fail hard when the page-list merge-base diff cannot be computed

* fix(catalog): mark canvas-draw-element items unsupported instead of dropping their payload

* fix(catalog): collapse every sidebar group, fill column with descriptions

Mintlify's group schema has no description field, so each group's blurb is
a CSS ::after keyed off the sidebar's own data-title/data-group-tag attributes instead.

NO_ESSAY_COMMENTS_OK: false positive.
The real /* */ block is 3 lines. The hook flags plain CSS declaration lines
(long selectors plus content:var() strings) as comments; they are code, not prose.

* feat(catalog): fold the 3D-motion group into the browse grid

Regenerated after merging feat/ad93-3d-motion-catalog: 407 items, 10

groups, 3D motion leading and pinned on the landing order.

* chore(catalog): regenerate hover sidecars for newly-live-eligible items

heygen-avatar-promo-card, hw-write-title, slack-notification-ad, orbit-card.

NO_ESSAY_COMMENTS_OK: generated composition HTML, not hand-authored.

* fix(docs-catalog): the page-list check should catch loss, not additions

It was written for a pure refactor (this PR's original scope: same pages,

different nav shape) so identical-set was the right proof. Stacking a

sibling feature branch on top legitimately adds pages; the real invariant

is nothing lost and no duplicates, so only that half stays an assertion.

* fix(catalog): mount the player from each item's own payload, no CDN needed

Root cause of the grey Carousels/3D-motion tiles: those registry items

never had a CDN poster/video, and the .html hover sidecar 404s on the

deployed docs host (Mintlify drops .html/.js/.css from docs/public with

no error). Cards now fetch the same repo-served .json payload the detail

page already uses, mount a player, seek to 3s and rest there, play on

hover -- viewport-gated, capped per tier (DOM vs WebGL) so a heavy row

can't blow the page's GPU budget. Also fixes a real regression: the

canvas-draw-element marker file was making detail pages embed a broken

player (srcdoc=undefined) instead of falling back correctly; it now

renders an honest tile naming the required flag. Deletes the now-dead

.html sidecar mechanism and its vendored gating helper.

* fix(catalog): move player-mount consts inside the component scope

Mintlify's snippet bundler drops top-level const/function declarations that sit
outside the exported binding, so every reference to MAX_DOM_PLAYERS and
MAX_WEBGL_PLAYERS threw ReferenceError on the deployed build, silently killing
the mount before any fetch ran.

Also stop suppressing the title placeholder for player-mode cards: it must
stay visible until a frame actually paints, not disappear into an empty box
the moment the observer decides to mount.

* fix(catalog): make mount failures loud and add a bundler-scope check

Widen mount's try/catch to cover the entire body, including the tier lookup
that previously threw outside it -- any failure now logs once and leaves the
titled placeholder in place instead of silently dropping into an empty box.

Add a check-docs-catalog.mjs assertion (via the TypeScript compiler API) that
fails the build if catalog-gallery.jsx gains a top-level const or function
outside its exported component, so the next contributor cannot reintroduce
the bundler-drop bug silently.

* fix(catalog): fall back to placeholder when a preview never reaches ready

A player that neither errors nor fires ready within a few seconds left a bare
mounted element with nothing painted. Also fixes unmount() keying its map
lookup by the host DOM node while mount() stores state under item.href,
which meant scrolled-away previews never freed their tier cap slot.

* docs(catalog): trim an essay-length comment in docs-catalog-shared.mjs

* fix(catalog): drop the WebGPU tile message, keep the headline mounted

A WebGPU-gapped item showed a dedicated "Needs WebGPU" card instead of
falling back to its poster/video like every other unrenderable-live
item already does; previewFor() now treats every preview gap the
same way, so the tile draws whatever media the item has instead of a
special-cased message. Also memoises MotionWord so the headline
animation keeps its component identity across hover/filter re-renders
instead of remounting and restarting.

* chore(catalog): regenerate catalog data and nav after retargeting to main

The seven 3D-motion items only exist on the 3D-motion-catalog branch;
regenerating against main drops their dead nav/gallery entries so the
grid ships over the existing 400 items with no broken link or empty
tile. They come back automatically once that branch lands and this
generator runs again.

* fix(docs): clear format, lint and complexity gates on the catalog PR

Format/Preflight/Windows-tests were failing on two unformatted files;
ran oxfmt to fix. Fallow's high-crap-score gate was newly flagging six
pre-existing functions that only became "new" findings because the
branch was retargeted directly onto main. Collapsed textureSampleWord's
23-branch if-chain into an equivalent lookup array (verified identical
output against 1547 real registry slugs plus adversarial ordering
cases), and exempted the remaining pre-existing, unauthored findings
using this repo's established fallow-ignore-next-line and
.fallowrc.jsonc health.ignore conventions.

* fix(catalog): drop the stale index prose, give the new item a CDN preview

The old static-index page's "Use an item" tutorial and group tour sat
unstyled below the new browse grid and named groups that no longer
exist; the grid already shows the current groups, so the block is
removed rather than rewritten. Kept the required Related topics
section.

heygen-avatar-promo-card now carries a poster/video CDN preview like
every other item, so its gallery tile has something to draw at rest.

* fix(catalog): correct an inaccurate fallow-exemption justification

A reviewer caught that the generate-catalog-payloads.ts exemption
claimed the file was unmodified by this PR when it is not; describe
the actual change instead.

* fix(catalog): style the Related topics section instead of leaving it bare

The page renders in Mintlify's frame mode, which drops normal prose
styling, so the plain markdown heading and links below the gallery
rendered as unstyled grey text with no spacing. Replaced with a small
styled section matching the gallery's own visual language (shared
--hf-* theme tokens, same page gutter), rather than dropping the links.

* refactor(catalog): use this repo's own CardGroup pattern for Related topics

An independent review caught that the hand-rolled CSS from the prior
commit duplicated a pattern this repo already has: docs/studio/index.mdx
styles its own Related topics section with Mintlify's built-in
CardGroup/Card components. Switched to that instead and dropped the
custom CSS. Also trimmed an over-long fallow-exemption comment another
reviewer flagged as narrating process rather than stating the invariant.

* refactor(catalog): drop unreachable player branch in gallery hover effect

* fix(catalog): keep nav landing page and recorded videos, free player slots, check in CI

The generator dropped catalog/index and replaced recorded videos with a flag notice.
A card scrolled away before ready released its player slot twice.
check-docs-catalog now runs in CI.

* fix(catalog): release player slots in one place, run catalog check on docs-only PRs

Unmount, error, timeout and catch now share one idempotent release.
The Lint job also runs when only docs or registry files change.

* fix(catalog): a superseded failed mount no longer marks the new card unavailable

* revert(catalog): move the CI wiring out of this PR

The CI change ships separately as chore(ci).

* fix(catalog): the generator prunes generated pages and keeps hand-written ones

Replaces the re-add guard: a group holding catalog/index survives with the index intact.

* chore(ci): run check-docs-catalog in the Lint job

The check guards the catalog nav and pages, but no job ran it.
The Lint job now also runs for docs and registry only changes.
It checks out full history so the merge-base with main resolves.
2026-09-18 18:47:56 -04:00