Commit Graph
5020 Commits
Author SHA1 Message Date
Vance IngallsandClaude Opus 5 7000e32942 test(producer): record the Phase 2a gate result and its real pass criteria
The spec's "PSNR >= 45 dB vs the single-encoder output" reads as a failure
on a healthy render: the measured average is 44.4 dB because segmented
capture forces a closed GOP with scene-cut detection off, and two CRF
encodes of identical frames differ regardless. The boundary frames are the
HIGHEST-PSNR frames in the series (48-62 dB forced IDRs), which is what the
gate should actually assert.

Also records the cost this buys nothing back on: the segmented stage runs
the plain capture loop rather than the depth-2 worker-encode pipeline, so
the same fixture takes 5m16 against streaming's 2m43.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:02 -07:00
Vance IngallsandClaude Opus 5 4b54b5d26f feat(producer): route long renders to segmented capture behind HF_SEGMENTED_CAPTURE
Opt-in, single-worker, mp4/mov only, and not for HDR or shader-transition
routes — those run their own compositor and webm's VP9 concat-copy is
fragile across ffmpeg versions.

A first-segment encoder spawn failure replans onto the plain streaming
path and falls through to the branch below, so the opt-in cannot make a
render fail where the default would have succeeded. A failure after that
propagates: per-segment retry is Phase 2c.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:01 -07:00
Vance IngallsandClaude Opus 5 8391abd672 feat(producer): sequential segmented capture stage
One closed-GOP encoder per segment writing segments/segment_NNNNN.mp4,
then a concat-copy into the video-only file the assemble stage muxes. The
GOP is each segment's own frame count, so every concat boundary is an IDR;
the tail segment is shorter and says so.

A spawn failure on the first segment returns success:false so the caller
can still replan onto the single-encoder streaming path. A later one
throws: earlier segments are already encoded, and silently discarding them
would drop frames from the output.

Reuses the sequential stall watchdog from the streaming stage rather than
running unguarded; both helpers are exported for that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:01 -07:00
Vance IngallsandClaude Opus 5 d7398517bd feat(producer): sdr_segmented capture plan kind
Inert until the orchestrator passes useSegmentedCapture. replanAfterFailure
handles the kind for every failure, including capture_failure, which would
otherwise reach the "cannot apply" throw guarding the non-streaming kinds:
a drawElement failure keeps segmenting on the screenshot engine, anything
else drops to a plain streaming render.

CapturePlanTarget deliberately does not gain the kind — routing fallbacks
are streaming/disk only, and adding it there would make
`useStreamingEncode: fallback.kind === "sdr_streaming"` silently false for
a segmented target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:00 -07:00
Vance IngallsandClaude Opus 5 46d852cc93 refactor(engine): extract concatVideoFiles from the chunked encoder
Segmented capture needs the same concat-demuxer stream copy the chunked
encoder already does. Behaviour preserved, including the
external_interruption classification the caller maps to a retryable
failure: the helper reports it rather than swallowing it. The concat list
keeps living beside its inputs, because concurrent encodes share an output
directory but not a chunk directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:53:00 -07:00
Vance IngallsandClaude Fable 5.1 5f373894d0 fix(producer): disk preflight names only causes that reach it; refresh the router field's docstring
Review follow-ups on the Phase 0 and Phase 1a PRs:

- The disk-headroom message blamed HDR and shader-transition output for a
  throw those renders can never reach (they run hdr_layered, a separate
  loop). It now lists png-sequence/gif, multi-worker screenshot capture,
  and the three ways streaming is switched off, including the duration cap
  and its threshold.
- RenderCaptureObservability.captureParallelStream still described the
  pre-split semantics ("the kill switch was on"). It now says BeginFrame
  routes by default, screenshot only with the env, that eligible_off sizes
  the held-back screenshot cohort, and that the label can disagree with the
  platform-only captureMode on Linux with system Chrome or DPR > 1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:59 -07:00
Vance IngallsandClaude Opus 5 9c3cff4960 feat(producer): segment planner for long-form capture
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:59 -07:00
Vance IngallsandClaude Opus 5 1ae359ca00 feat(producer): stream multi-worker BeginFrame renders by default
Splits the non-DE parallel-stream router default by the capture mode the
render will actually use. BeginFrame (Linux headless-shell) defaults on:
its opt-in cohort fails at 0.05% over 11.5k renders. Screenshot capture
(macOS/Windows) stays behind HF_CAPTURE_PARALLEL_STREAM=true, because its
opt-in cohort fails at 5.5% against a ~1.1% baseline in two classes this
path owns — the parallel stall watchdog hard-fails with no retry, and a
dying encoder surfaces as a bare write EPIPE. Phase 1b flips it once those
are fixed.

resolveParallelCaptureMode mirrors the engine's preMode rather than reusing
resolveObservedCaptureMode, which is platform-only and would have
default-enabled Linux-with-system-Chrome and Linux-at-DPR>1, whose real
capture path is the screenshot one. The routed telemetry value now comes
from the same helper, so the cohort cannot disagree with the predicate that
routed it, and eligible-but-held-back renders keep emitting eligible_off.

The preflight message named --workers 1 as the fix for every disk-path
render; with the duration cap enabled a --workers 1 render lands there too.
It now names each override that can cause it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:58 -07:00
Vance IngallsandClaude Opus 5 ddbc9a8a98 feat(cli): emit Chrome memory, capture path and segment fields on render events
Both events carry the live observability values, so a render_error that
never built a perf summary still reports the last known Chrome memory.
render_complete prefers the aggregate from RenderPerfSummary.chromeMemory,
which covers every worker rather than whichever session sampled last.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:25 -07:00
Vance IngallsandClaude Opus 5 2cf8024a7c fix(engine): read Chrome process info from the browser target, not the page
SystemInfo.getProcessInfo is served only by the browser target; a
page-target session rejects with "is only supported on the browser
target". The sampler swallows probe failures by design, so this shipped as
a permanently empty reading: a 10 s debug render reported chromeMemory
null with capturePath already correct. Against the browser target the same
render reports browser 105 MB / renderer 359 MB over 3 samples.

readChromePids is extracted so the target choice is pinned by a test whose
fake rejects a page session the way Chrome does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:25 -07:00
Vance IngallsandClaude Opus 5 119dafc005 feat(producer): carry Chrome memory and capture path on observability and perf summary
Live route through updateCaptureObservability survives a crash, which is
the case this telemetry exists for: the failure path never builds a
RenderPerfSummary. The aggregate route through CapturePerfSummary (max of
per-session peaks, sum of samples) feeds render_complete.

capturePath is recorded inside syncCapturePlan rather than beside the
streaming-encode gate log, so a render that replans onto disk reports the
path it actually captured on; capturePathForPlanKind is exhaustive over
CapturePlan["kind"] so a new plan kind cannot silently inherit the wrong
label.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:25 -07:00
Vance IngallsandClaude Opus 5 428922162d feat(engine): sample Chrome process memory per capture session
Start a 2 s unref'd sampler when a session finishes initializing, stop it
in closeCaptureSession, and report peaks on CapturePerfSummary. Pids come
from puppeteer (browser) and CDP SystemInfo.getProcessInfo (renderer, GPU),
so the long-render "Target closed" class can be correlated with Chrome
memory instead of guessed at. Disable with HF_CHROME_MEMORY_SAMPLER=false;
cadence HF_CHROME_MEMORY_SAMPLE_MS.

The start hook is finalizeSessionInit rather than the tail of
initializeSession: that helper is already the single owner of "session is
ready" across both of initializeSession's exits, so the early screenshot
return cannot silently skip sampling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 20:52:03 -07:00
Vance IngallsandClaude Fable 5.1 a35168eddd fix(engine): sampleOnce() joins an in-flight sample instead of skipping it
With a boolean reentrancy flag, the Task 3 close sequence
(`stop(); await sampleOnce()`) returned immediately whenever an interval
tick was still awaiting CDP/ps, so getCapturePerfSummary read stats from up
to one interval earlier and onSample fired once more after the session had
closed. Reproduced with a 350 ms getPids on a 100 ms interval: close
resolved with samples=0 and the callback landed afterwards.

The guard now keeps the in-flight promise and returns it to any caller, so
an interval tick still never stacks a second round-trip (the existing
reentrancy test holds) while a close-time call resolves only after the
running tick has merged. Deviates from the plan's reference body, which
used `let inFlight = false`; the ChromeMemorySampler interface and all
names are unchanged, only the sampleOnce() contract is stronger.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:02 -07:00
Vance IngallsandClaude Fable 5.1 7cf57fa42e test(engine): pin the unref'd sampler interval via a cast-free timer seam
Three independent reviewers found that deleting timer.unref() in
createChromeMemorySampler left the suite green: the plan's Global Constraint
(fire-and-forget on an unref'd interval) had no failing test. The new test
captures the real NodeJS.Timeout through the setIntervalFn seam and asserts
hasRef() is false after start(); it goes red when the unref() call is
removed.

The seam could not be used without a cast as typed in the plan
(`typeof setInterval`): with the DOM lib in scope that type also carries the
`(handler, timeout) => number` overload, which a capturing wrapper cannot
satisfy. setIntervalFn / clearIntervalFn keep their names but are now typed
to the two-argument shape the sampler actually calls; the global timers are
assignable to those shapes, so the defaults are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:02 -07:00
Vance IngallsandClaude Fable 5.1 53403a5896 feat(engine): chrome memory sampler with live onMemorySample capture option
Pure service for Phase -1 of the long-form render plan: merges per-pid RSS
rows into browser/renderer peaks, last-sample total and GPU presence on an
unref'd interval with a reentrancy guard, so a CDP or ps failure can never
reach the capture loop. CaptureOptions.onMemorySample is the live route the
producer will wire so a mid-render crash still reports memory state. Nothing
instantiates the sampler yet (Task 3); no behaviour change.

Beyond the plan's tests, pins the double-start, empty-sample and catch
guards directly, since the interval-based error test stays green when the
catch is removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:02 -07:00
Vance IngallsandClaude Fable 5.1 1862c34399 test(engine): fix readonly array type in the pid-filter test
`readonly string[][]` parses as `ReadonlyArray<string[]>`, so `calls.push`
was a TS2339 in 5214552cd. It slipped through because the engine tsconfig
excludes `src/**/*.test.ts` and vitest strips types; caught by running tsc
on the test file explicitly. `Array<readonly string[]>` is what the
neighbouring "one ps call" test already uses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 3cee3b94f7 fix(engine): read /proc/<pid>/status for RSS on linux instead of shelling out to ps
Every render image (packages/cli/src/docker/Dockerfile.render,
packages/gcp-cloud-run/Dockerfile, examples/k8s-jobs/Dockerfile.example,
Dockerfile.test) is node:22-bookworm-slim with no procps, so `ps` is ENOENT
in containers and sampleProcessRss would return [] on every tick: the whole
cloud fleet would report chrome_memory_samples: 0, indistinguishable from
"no Chrome pids". Linux now reads VmRSS from /proc/<pid>/status with a
per-pid try/catch, so one renderer that exited between discovery and
sampling omits only itself (with `ps` a wholly-dead batch fails the call).
No child process per tick on the primary server platform. Same precedent as
the CLI's orphanCleanup.ts / processTree.ts. `ps` stays for darwin and
other POSIX; `tasklist` for win32. The reader is an optional fourth
parameter with a default, so Task 2/3 callers are unaffected.

Scope note: spec section 2.5 records the Target-closed population as
docker=false throughout, so this closes a render_complete completeness gap
on cloud, not a coverage gap on the crash class this phase targets.

Verification: 15/15 vitest; four linux-branch mutants (per-pid catch
rethrows, regex loses /m, kb <= 0 weakened, path /status -> /stat) each
fail at least one test, restored green. Real-process smoke check on darwin:
sampleProcessRss([process.pid, <exited pid>]) returns one sample for the
live pid (BSD ps exits 0 with partial output when a valid-range pid is
gone). The linux path is unit-tested against a /proc/<pid>/status fixture
only; no Linux host or VM was available in this session to execute it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 645cbdce4e test(engine): pin the pid filter, dedup and tasklist memory guard in sampleProcessRss
Review of 76d8e2682 found three surviving mutants: removing the
`Number.isInteger(p) && p > 0` filter, dropping the `new Set` dedup, and
weakening `kb <= 0` to `kb < 0` all passed the plan's 8 tests. The filter is
the guard that matters most: Task 2 feeds `browser.process()?.pid`, and a
single NaN or undefined reaching `ps -p` makes ps exit non-zero, which the
never-rejects catch turns into `[]` for every pid in the sample.

Mutation check: each of the three mutants now fails exactly one test
("drops invalid pids and duplicates before calling ps" for the filter and
dedup, "returns [] when the memory column is not a positive number" for the
tasklist guard); restored source is 10/10 green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:01 -07:00
Vance IngallsandClaude Fable 5.1 bc70156f00 feat(engine): zero-dependency per-pid RSS sampler (ps / tasklist)
First piece of the long-form render Phase -1 telemetry: the Chrome memory
sampler needs resident set size per pid without adding pidusage or
systeminformation. `ps -o pid=,rss=` covers POSIX in one call for every pid;
`tasklist /FO CSV` covers win32 one pid at a time. The function never rejects
and omits pids that exited between discovery and sampling, so a failed sample
can never reach the capture loop.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 20:52:00 -07:00
Vance Ingalls 2b2b999e62 Merge pull request #4109 from heygen-com/lfr/02-phase0-duration-cap
feat(producer): stream long single-worker renders by default; duration cap behind a flag (Phase 0)
2026-09-18 20:50:36 -07:00
Miguel Ángel a25da94b77 refactor(studio): remove the storyboard view and its Storyboard/Preview switch (#4140)
* refactor(studio): remove the storyboard view and its Storyboard/Preview switch

* refactor(studio): fold the thumbnail output branches left after the storyboard mode went

* docs(studio): drop leftover storyboard references after the view was removed

* ci: allow the storyboard view deletions in the no-deletions guard

* docs(studio): wrap an over-long comment line in the caption overlay helper

* style(studio): reindent App.tsx after the view-mode wrapper was removed
2026-09-18 23:47:12 -04:00
Miguel Ángel 2db126d4c8 docs(skills): one owner doc for everyday edits and a hyperframes-studio skill (#4125)
* docs(skills): one owner doc for everyday edits and a hyperframes-studio skill

Agents got contradictory advice on volume fades and were blocked from splits
and copies by the Ask-agent guardrails.

- creator-editing-recipes.md is the single owner: adds align-to-event,
  copy-a-group and swap-media recipes and the one volume rule (use the lane)
- variables-and-media, data-attributes (skill and CLI copy), the gsap adapter
  and tracks-and-clips now defer to it
- Ask-agent guardrails allow the neighbour edits a split, retime, copy or swap
  requires and point at the owner doc
- new hyperframes-studio skill: sub-compositions for every scene, one caption
  track, one element kind per track, safe zones
- drift test lints every html example in the owner doc with the real linter
  and fails on any doc teaching a volume tween

* refactor(skills): drop unused tween allowlist from recipes drift test

* docs(skills): name the track kind attribute in the studio skill

* docs(skills): stop claiming lint rules that do not exist yet and fix track numbering

Review fixes:

- studio skill: drop the severity claim and the preview-overlay claim,
  group tracks by data-track-kind instead of numeric ranges that clashed
  with the recipes
- recipes: audio data-start is root time only in the root composition;
  scene-local inside a sub-composition file
- guardrails: timeline edits are exempt from the element-only and timing
  lines; the pointer names the skill reference, not a repo path
- montage doc defers to the volume lane; skill count is 21 everywhere
- drift test also scans the montage doc

* fix(skills): the ducking generator emits a volume lane and captions defer to the studio safe zones

- audio-duck.mjs prints a data-automation lane (clip-local times from the
  target data-start) instead of a timeline volume tween; --json adds the lane
- operations.md shows the generated lane; the drift test scans it
- embedded-captions safe-zone lines point at the studio skill, which owns
  the values

* chore(skills): regenerate the skills manifest after rebase

* docs(skills): remotion translation docs point volume ramps at the automation lane

* docs(skills): studio safe zones use the action-safe 90 and title-safe 80 boxes

* docs(skills): caption docs defer to the studio safe boxes

* chore(skills): regenerate the skills manifest after rebase

* fix(skills): studio skill names the safe margins; guardrail test matches the timeline-edit wording
2026-09-18 23:19:40 -04:00
Vance Ingalls 34644a1993 Merge pull request #4148 from heygen-com/release/v0.8.50
chore: release v0.8.50
2026-09-18 19:56:18 -07:00
Miguel Ángel b44f1f85dc feat(lint): flag nested structure, missing timing and caption tracks on the root composition (#4146)
* feat(lint): flag nested structure, missing timing and caption tracks on the root composition

One rule set for the root composition: nested layout inside a top-level
element, a timed element or media without data-duration, captions not marked
data-track-kind=captions, and captions spread over more than one track.
Findings are errors when the host is Studio and warnings elsewhere.
Sub-composition files are left free to hold layout.

* test(lint): keep the structure rules silent on the shipped catalog and skills

Runs the rules over every registry and skills html file and asserts no
findings, and treats a bare media element with no timing as a static layer.

* fix(lint): run the structure rules as Studio in both adapters and skip non-root files

The Vite Studio adapter now lints as the studio host, and HTML the project
lint did not treat as a root (components, snippets) is linted as a
sub-composition. Style, script and template content no longer count as nested
layout, sub-composition hosts need no duration, and a caption row without a
lane is not a second track.

* fix(lint): count legacy data-end as a length and pin the catalog test to real roots
2026-09-18 22:49:15 -04:00
Vance Ingalls 42f4afa367 chore: release v0.8.50 2026-09-18 19:41:06 -07:00
Vance Ingalls ea07f0e93b Merge pull request #4139 from heygen-com/sdk-cutover-decline-reason
fix(studio): a composition with a <canvas> no longer kills the SDK session
2026-09-18 19:39:22 -07:00
Miguel Ángel 7797c5ead4 feat(studio): the timeline shows one row per top-level element and no child rows (#4132)
* feat(studio): the timeline shows one row per top-level element and no child rows

* test(studio): cover the top-level id collection the timeline rows come from

* chore(studio): shorten the row-elements comment

* fix(studio): keep implicit layers as rows and say why a nested element cannot be hidden

* refactor(studio): drop implicit layers; rows are exactly what topLevelElements returns

* test(studio): untimed root wrappers are not timeline rows

* fix(studio): an empty timeline message clears the rows; drop the unused implicit contract

* fix(studio): an empty clip manifest reaches the timeline and clears the rows

* fix(studio): an empty clip manifest does not shrink the transport to its 1s floor

* chore(ci): allow deleting the child-row expansion hook and its test
2026-09-18 21:27:53 -04:00
Miguel Ángel 63d0ea4d28 fix(studio): show where a dragged file will land instead of a drop-to-add box (#4142)
Dragging a file over Studio covered the screen with a "Drop to add at the
playhead" box, but a drop lands at the pointer time on the target row. The
box is gone; the timeline now draws a landing preview at the exact start and
track the drop will use (a new row below the last lane when that is the
target), taken from the same placement function the drop commits.
2026-09-18 21:27:39 -04:00
Vance IngallsandClaude Opus 5 db78b4fe68 fix(studio): keep a literal relative require out of the canvas-stub docs
verify-packed-manifests scans packed JavaScript for relative import
specifiers with a set of regexes that do not strip comments, so the
`require('./canvas-shim.cjs')` inside the stub's doc comment read as a real
import of a file that is not in the studio tarball, and the Build job failed.

Describes linkedom's try/catch guard in prose instead. The same literal was in
the test file's header comment; CI only reported the first match, so both are
fixed here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 18:00:28 -07:00
Miguel Ángel 4bbcf650ae fix(player): perf harness waits for assets-ready before play instead of a fixed 5s (#4137) 2026-09-18 20:58:30 -04:00
Vance IngallsandClaude Opus 5 d5e1de9201 fix(studio): a composition with a <canvas> no longer kills the SDK session
linkedom's HTMLCanvasElement constructor calls createCanvas(300, 150) from the
Node-only `canvas` package, behind this guard:

    try { module.exports = require('canvas') }
    catch { module.exports = require('./canvas-shim.cjs') }

The guard only works with a runtime require. Vite resolves the specifier
statically, so the bundled form is `try { exports = <module> } catch {...}` — a
plain assignment that cannot throw. The catch never runs, linkedom's own shim is
bundled but unreachable, and createCanvas lands undefined. Verified in the built
artifact: the canvas module namespace carried only a `default` export.

Parsing any <canvas> tag therefore threw inside openComposition, so Studio lost
its SDK session for the entire composition. Every edit then fell back to the
server path, and the resolver shadow did not run either, which is why this was
invisible before v0.8.47 added studio:sdk_session_unavailable. In the 72h after
the flip it accounted for 1065 of 1544 session-open failures, across a large
share of 197 users. Canvas is common in compositions: Three.js, shaders,
particle effects, charts.

Aliases `canvas` to a local browser stub that mirrors linkedom's shim
(measurement-only; nothing in the SDK rasterizes through it). Reimplemented
locally rather than pointing at linkedom's commonjs/canvas-shim.cjs, which its
exports map does not expose.

Confirmed against a rebuild: the namespace now carries createCanvas, so the
call returns a stub instead of throwing.

Note the unit tests here pass with or without the fix — vitest resolves the
require at runtime, so only the bundled build was ever broken. The regression
guard therefore pins the alias itself, since nothing else in the suite can
observe the bundler.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:38:39 -07:00
Miguel Ángel 898d27869e feat(studio): preview ruler and safe-margin toggles (#4127)
* feat(studio): preview ruler and safe-margin toggles

* test(studio): cover guides placement in the preview pane and the caption band box

* feat(studio): safe margins are two nested 90 and 80 percent boxes with edge ticks

* fix(studio): safe box ticks centred on the line, no halo, tighter skill wording test
2026-09-18 19:47:27 -04:00
Miguel Ángel 14e5f57670 fix(core): script coalescing no longer moves an inline script past a src script (#4133)
* fix(core): script coalescing no longer moves an inline script past a src script

* test(core): pin how the script coalescers group inline runs

* test(core): simplify the pinned-script grouping test
2026-09-18 19:35:08 -04:00
Miguel Ángel c80ae95084 fix(studio): hovering or selecting a timeline clip no longer swaps its filmstrip (#4130) 2026-09-18 19:27:45 -04:00
Miguel Ángel a192701adb feat(studio): add per-project UI preference scoping (#4093)
* feat(studio): scope UI preferences per project

Panel widths, timeline zoom, and other UI preferences shared one
localStorage entry across every project. Key by the active project id
(from the existing hash route), falling back once to the pre-scoping
entry so an existing project's first load doesn't look reset.

* docs(studio): trim a comment past the repo's 4-line cap

* fix(studio): make per-project preference scoping opt-in

Auto-deriving the project id from the URL by default would have
silently flipped every existing caller into per-project mode, most of
which read once at mount and never again across a live project
switch, and some of which (playback rate, mute, snap) are documented
as deliberately project-agnostic in playerStore's reset. Require an
explicit projectId instead so today's 9 call sites are unaffected.
2026-09-18 18:47:44 -04:00
Miguel Ángel 6995a4c2f7 feat(studio): the agent sees every timeline clip, audio included (#4123) 2026-09-18 18:47:32 -04:00
Miguel Ángel 02c415dd8b feat(parsers): share top-level timeline element and track kind definitions (#4124)
* feat(parsers): share top-level timeline element and track kind definitions

One pure definition of which elements own a timeline row (sub-composition
hosts and timed elements; untimed wrappers are descended) and of a row's
track kind, so the linter and Studio's timeline agree.

* fix(parsers): align top-level rows with the runtime clip selector

Rows now follow the runtime's clip selector (data-start, data-track-index,
media tags), match tags case-insensitively for DOM adapters, and skip noscript.

* fix(parsers): treat a track-index layer without a duration as a wrapper

Matches the runtime, which skips such elements as clips, so their timed
children stay visible to the lint.
2026-09-18 18:47:18 -04:00
Miguel Ángel 0b6d1039ed feat(studio): copy, paste and duplicate clip groups on the timeline (#4064)
* feat(studio): copy, paste and duplicate clip groups on the timeline

Cmd+C on a multi-clip selection now copies all of them, not just the
first. Cmd+V pastes the group at the playhead keeping each clip's offset
from the earliest one and its original track when free. Cmd+D duplicates
the selection in place. Context menu gains Copy/Paste/Duplicate, and a
failed copy now toasts why instead of silently returning false.

Root cause was a DOM lookup that couldn't address a composition-instance
clip's root; group copy reuses the composition-aware resolver every other
timeline editor already uses. Also fixes a duplicated data-hf-id on
pasted/duplicated clips found during live testing, and reorders the
post-paste selection to land on the new clips instead of stale ones.

* fix(studio): scope the data-hf-id strip to tag brackets, not text content

A clip's visible text can contain the literal string data-hf-id="...";
the previous whole-string regex stripped it as if it were the attribute,
corrupting the copied text. Reproduced and confirmed the fix with a test.

* fix(studio): extend root duration and coalesce cut on group paste/duplicate

Paste and duplicate could silently truncate a group from export: they
inserted clips past the composition's data-duration without growing it,
so content existed in the file but never played or rendered. Both now
route through the same grow-only ratchet blockInstaller already uses.

Cmd+X on a multi-clip selection now deletes the whole selection in one
call (the existing batched delete already used by the Delete hotkey),
so one Cmd+Z restores all of them instead of needing one per clip.

The data-hf-id strip and id-capture on paste now go through DOMParser
and a tighter regex respectively, closing two escape hatches: a > inside
an earlier attribute value, and a data-id attribute preceding the real id.

* refactor(studio): share the id-attribute regex and hoist the clip DOMParser

Simplify pass on the previous commit: the id-capture lookbehind was a
second copy of the same pattern deduplicateIds already uses, now a
single exported ID_ATTR_RE both call. The per-clip DOMParser instance
in pasteTimelineClips is now created once per call, not once per clip.
Adds a test pinning stripHfIds's no-parseable-root fallback.

* fix(studio): close the stripHfIds hoist gap

stripHfIds silently returned html unstripped when DOMParser found no
body-level element (a tag the parser hoists into <head>, e.g. title or
meta). Closes it with a regex fallback and a test reproducing the exact
hoisted-tag case, not just the no-tag-at-all case from the prior commit.

* refactor(studio): dedupe clip menu item classes into one helper

The new Copy/Paste/Duplicate rows in ClipContextMenu each got their own
static enabled/disabled className string instead of the parameterized
itemClass(enabled) pattern the sibling TrackGapContextMenu already uses
for the same job. Consolidated to one function; rendered classes are
unchanged for every state.

* refactor(studio): reuse timeRangesOverlap for clip paste placement

tracksOverlap re-derived the exact half-open interval check
timelineCollision.ts already exports and uses elsewhere on the timeline.
2026-09-18 18:25:12 -04:00
Miguel Ángel 93921dccc8 fix(cli): stop the workspace build check test from depending on wall-clock mtimes (#4098) 2026-09-18 18:24:57 -04:00
Miguel Ángel 9981d9afb7 fix(capture): never silently render a webgpu composition blank (#4091)
* fix(capture): never silently render a webgpu composition blank

A composition declaring data-requires-webgpu only got the Chrome WebGPU
flag when hardware GPU auto-detection succeeded. On a software-GPU host
it launched without the flag and rendered blank with no error.

compositionRequiresWebGpu (static declaration check) and
assertWebGpuAdapterAvailable (live per-page adapter probe) now live once
in packages/engine's browserManager, the shared choke point every launch
site already goes through. All six launch sites (frame capture, the four
CLI diagnostic commands, the studio thumbnail server, and the render
pipeline's capture-options builder) route through those two functions;
the old CLI-only static-heuristic guard is deleted, not kept alongside
the new one. A software host that cannot obtain any WebGPU adapter now
fails loudly naming the requirement instead of producing a blank frame.

* fix(capture): thread requiresWebGpu through the distributed chunk-render path

The distributed renderChunk() path built its CaptureOptions without
requiresWebGpu, so it fell through to frameCapture's own fetch-based
detection, whose failure path silently treats "requires webgpu" as
"doesn't" if that fetch ever fails. Every other launch site (frame
capture, all four CLI diagnostic commands, the studio thumbnail server,
and the in-process render pipeline) already reads the value from HTML
already in memory or on disk instead of relying on that fallback.

Reads the compiled composition's index.html the same way probeStage
already does and derives requiresWebGpu from it before building
CaptureOptions. Adds a regression test that pins the exact path
renderChunk() reads at render time, since the composition doesn't
actually exercise WebGPU so an end-to-end render can't distinguish the
two cases on this host.

* test(producer): observe the real CaptureOptions renderChunk() builds

The previous regression test called plan() and compositionRequiresWebGpu()
directly, bypassing renderChunk() entirely, so it could not detect a
regression to the wiring it was meant to protect (confirmed: it stayed
green with the fix reverted).

Exports the existing createCaptureSession dependency seam so a test can
substitute a spy that delegates to the real implementation while
capturing the CaptureOptions object renderChunk() actually constructs.
Verified non-vacuous by reverting the fix and confirming the test fails.

* fix(gcp-cloud-run): resolve puppeteer-core from a real file in the webgpu probe

The image build's WebGPU adapter probe imported puppeteer-core from an
inline `bun -e` eval string. The eval's synthetic /app/[eval] location
can't resolve the workspace-hoisted dependency, so every build of this
stage failed closed with "Cannot find package 'puppeteer-core'" before
the probe itself ever ran.

Moves the probe into a real script file next to probe-beginframe.ts,
which already imports puppeteer-core dynamically the same way and
resolves fine from its real workspace location. Verified locally against
a cached chrome-headless-shell binary.

* fix(engine): bound the webgpu adapter probe so a stuck driver can't hang the caller

requestAdapter() has no native timeout. A broken software GPU driver
that never settles it would hang assertWebGpuAdapterAvailable
indefinitely, including inside the GCP image build's probe RUN step,
producing an opaque build timeout instead of a clean probe failure.

Races the in-page request against a bounded timeout so a stuck adapter
reads as unavailable and fails the same way a genuinely absent one does.

* test(cli): mock the engine's webgpu exports in the browser-fallback test

* test(producer): run the typegpu adapter fixture only on hosts with a hardware WebGPU adapter

The fixture declares data-requires-webgpu, so a software-only host now refuses it
instead of silently rendering the shader blank. The old golden had no shader in it.
Tag it webgpu, skip that tag in the CI image and the test scripts, and re-record the
golden from a Metal render with the shader visible.

* fix(engine): bound the composition fetch that decides the webgpu flag

createCaptureSession fetched the served page with no timeout before launching the
browser, so a hung server stalled a render forever and hung a fake-timer test. Give it a
five second limit, and let that test say it has no WebGPU requirement the way the
producer does.

* fix(capture): refuse a software fallback WebGPU adapter instead of rendering on it

swiftshader reports a WebGPU adapter but cannot render canvas WebGPU, so a
data-requires-webgpu composition on a software-only host crashed on its first frame.
Refuse fallback adapters with the same clear WebGPU-unavailable error. The GCP image
probe now serves its page over http (file:// hides navigator.gpu) and reports a
software-only host instead of failing the image build.

* fix(engine): parse the composition root tag without a backtracking regex

* fix(engine): keep quoted attribute values whole when reading the composition root tag
2026-09-18 18:06:23 -04:00
Miguel Ángel cb9ca4f0af chore: release v0.8.49 (#4122) 2026-09-18 17:44:18 -04:00
Miguel Ángel a7282dcd0e fix(player): wait for a cross-origin composition's assets before playing (#4103)
* fix(player): wait for a cross-origin composition's assets before playing

The player's readiness gate scans the iframe document, which is null when the
composition is embedded from another origin, so it released playback at once
and the first frames could play with images still loading.

The runtime now runs the same readiness scan inside the iframe and posts
assets-ready; its timeline message reports whether assets have settled so an
older runtime, which never posts it, is not waited on. Also declares the
animation-frame abort handler before a host whose requestAnimationFrame calls
back synchronously can reach it.

* fix(player): keep a ready cross-origin player ready across a late iframe load

The iframe load event reset readiness for any document the player cannot
read. The runtime posts its timeline before load and a paused runtime never
posts it again, so a load arriving after the assets settled left the player
not ready. A ready cross-origin player already holds this document's
handshake, because every navigation clears readiness first.

* fix(core): stop RuntimeReadinessWindow widening Window.__hf so the runtime typecheck passes

* fix(core): a torn-down runtime no longer posts assets-ready
2026-09-18 17:27:16 -04:00
Vance Ingalls f92944a2e7 Merge pull request #4100 from heygen-com/sdk-cutover-decline-reason
feat(studio): split ineligible_operation into its four real causes
2026-09-18 13:34:39 -07:00
Miguel Ángel 707f9a44c3 feat(studio): the first clip on an empty main track starts at 0 (#4099)
* feat(studio): a clip dropped onto an empty main track lands at 0

AD96 PR 1: the existing-clip-drag path (timelineCollision.ts) forces
start=0 for a clip landing on the main track for the first time, since
a track with one clip has no leading gap by convention. The file-drop
path is deferred to a follow-up once #4067 clears its file.

* refactor(studio): share the AD96 main-track drag test setup

Fallow flagged the two new tests as a 22-line clone; factor the shared
DraggedClipState builder into one helper.

* fix(studio): don't retime the rest of a multi-selection on the main-track snap

Adversarial review caught it: forcing the grabbed clip's previewStart to 0
fed straight into resolveMultiSelection's group delta, silently dragging
every other selected clip's time on a vertical-only gesture. Skip the
magnetic snap for a multi-selection drag; generalize
resolveMainTrackDropStart to also serve the upcoming file-drop path
(originTrack/isAudio instead of a full dragged element).

* feat(studio): dropping a new asset or file onto an empty main track lands at 0

AD96 PR 1, file-drop half (unblocked now that #4067 merged): the same
magnetic snap the existing-clip-drag path got now also covers a
catalog-asset drop and an OS file drop, applied once at the single DOM
drop dispatcher (timelineDragDrop.ts) so a multi-file batch still
sequences correctly after the snapped first placement. Composition and
block drops are left untouched — out of scope, per the original scout.

* refactor(studio): share the drag-over-then-drop test act() block

Fallow flagged the repeated act(() => { dragOver; drop }) pattern as
two clone groups; factor it into one dropAt helper.

* fix(studio): a mixed audio+video file drop always snaps, regardless of order

Adversarial review caught it: classifying a multi-file drop by only
the first file's kind meant [song.mp3, clip.mp4] and [clip.mp4,
song.mp3] snapped differently, purely from Finder selection order.
Classify the whole batch as audio only when EVERY file is audio, and
add the missing all-audio and mixed-order test coverage.

* refactor(studio): tighten main-track snap comment and dedupe file-drop array

* fix(studio): snap a top-gutter track-insert that lands on literal track 0

Adversarial review caught it: dragging a clip into the top insert
gutter creates a fractional lane that's never literal track 0 during
preview, but normalizeToZones can renumber it to literal 0 on commit
when the topmost visual lane was non-zero (main track genuinely
empty). That persisted start was never routed through
resolveMainTrackDropStart. Apply the same snap at commit time, using
the real post-normalize track, guarded against multi-selection the
same way the drag-preview path already is.

* fix(studio): compute the track-insert snap once so the z-sync sees it too

The previous fix snapped only the persisted edit after the fact, so the
stacking sync still read the unsnapped start. Resolve the snap inside
buildTrackInsertEdits from the final renumbered track and feed the one
value to both the edit and the z-sync candidate.

* refactor(studio): drop internal ticket codes from main-track snap comments and test names

* refactor(studio): simplify main-track snap

* fix(studio): snap the first main-track clip on its final lane and append new drops

One landing rule now runs in both the drag preview and the commit, after the
top-gutter renumber and the expanded-child-to-host conversion, so the ghost
matches the commit. New file and asset drops onto a filled main track land
after its last clip. The unused excludeKey check is gone.

* refactor(studio): keep media-panel drops out of this change

* refactor(studio): simplify main-track landing

* test(studio): pin the main-track snap on a plain lane change at commit level
2026-09-18 16:07:29 -04:00
Miguel Ángel a7ec66b124 fix(producer): cache the google fonts css lookup per process (#4097)
* fix(producer): cache the google fonts css lookup per process

Studio compiles every composition in a project separately for its
sidebar thumbnails and for gsap-animation extraction. Each compile
called injectDeterministicFontFaces, which re-fetched the same font
family's CSS metadata from fonts.googleapis.com over the network every
single time -- only the woff2 binary files were cached to disk, not
the CSS lookup itself. A project with a dozen small compositions and
three declared font families fired dozens of redundant network round
trips serially, holding the editor's asset-readiness overlay well past
its own 8-second timeout.

Adds an in-process cache for the CSS lookup keyed by the exact request
URL, self-evicting on failure so a later call still retries. Measured
on a real project: gsap-animation extraction across 12 compositions
dropped from ~16.4s to ~2.9s wall time, and per-composition thumbnail
generation dropped from 9-11s each to 10-14ms each once the cache is
warm.

* fix(producer): isolate the google fonts css cache from any one caller's abort

An adversarial review of the process-wide CSS-lookup cache found two real
bugs before it shipped. First, the cached promise carried whichever caller
reached it first's AbortSignal, so one render's cancellation could fail a
completely unrelated concurrent render sharing the same font family. Second,
the cache has no reset between test files running in one process, so two
sibling test suites asserting per-call fetch counts against the same
Google Fonts URL silently saw each other's cached result.

Strips the caller-specific AbortSignal from the shared fetch and races each
caller's own signal against it independently, so a caller can still bail out
on its own cancellation without affecting anyone else awaiting the same
lookup. Clears the cache in the two sibling test files' own beforeEach.

* fix(producer): key the google fonts css cache by fail-closed mode and drop transient statuses
2026-09-18 13:47:24 -04:00
Vance IngallsandClaude Opus 5 61db453133 feat(studio): split ineligible_operation into its four real causes
The first 24h of v0.8.47 telemetry showed 30 declines against 25 successes on
the dom family, and 26 of those 30 were `ineligible_operation` -- a single
opaque string standing in for four unrelated causes. That made the number
unactionable: a structural edit the SDK has no vocabulary for (expected, and
permanent until the op mapping grows) looked exactly like a reserved-attribute
or unsafe-attribute decline (narrow, and possibly worth fixing).

`sdkCutoverIneligibleReason` now names which check failed: unsupported_op_type,
child_scoped_op, reserved_attribute, unsafe_html_attribute, no_ops, or
target_unaddressable. `shouldUseSdkCutover` is derived from it rather than
duplicating the conjunction, so the predicate and its explanation cannot drift
-- a test pins that agreement across every case.

Also separates the two conditions that are not properties of the op batch at
all: with the dom family disabled, sdkCutoverPersist reported
`ineligible_operation`, which read as "the SDK cannot express this edit" when
the truth was "this family is switched off". It now reports feature_disabled
and session_unavailable distinctly, matching the other four chokepoints.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 10:37:30 -07:00
Miguel Ángel 15460ec541 fix(core): resolve already-errored media immediately in the asset-readiness gate (#4096)
The shared composition-readiness gate only listened for future canplay/error
events on pending media, so an asset whose error already fired before the
gate started listening was never seen -- a DOM error event doesn't refire.
That left the gate burning its full 8s timeout on every load. Measured on a
production composition with 85 duplicate small audio elements pointing at
missing files: 8.68s to 0.83s with this fix, no timeout warning.
2026-09-18 13:19:46 -04:00
Miguel Ángel cc36905c29 fix(studio): resolve a multi-clip selection's network probes in parallel (#4088) 2026-09-18 12:08:06 -04:00
Miguel Ángel 42f947f4c3 fix(studio): select and reveal a timeline clip after a drop (#4067)
* fix(studio): select and reveal a timeline clip after it's dropped

Dropping an asset, block, or composition onto the timeline placed
the clip but never selected it, so the user had to go find it. Wire
the same select+reveal pair AssetCard/AudioRow already use into all
three drop paths via a shared selectAndRevealTimelineElement helper.

Root-caused a shared bug along the way: useTimelineSelectionPreviewSync
compared the resolved selection count against only the ids already
present in timelineElements, not against every id actually wanted. A
freshly-selected id not yet in timelineElements (a just-dropped,
pasted, or duplicated clip) made that comparison pass on zero
resolved out of zero expected, falling through into a loud selection
clear instead of the intended quiet retry. Fixed the comparison and
covered drop, paste, and duplicate-of-a-group.

* refactor(studio): drop the dead fallback in the new store-key helper

Simplify-pass finding: both new selectAndRevealTimelineElement call
sites always pass a DOM id they just minted, so the `?? fallback`
after deriveTimelineStoreKey (which only returns undefined for a
domId-less, selector-less caller) could never run, and the pattern
was duplicated at both call sites. Added a non-optional variant for
callers that already hold a guaranteed DOM id, and dropped the
now-redundant "old code" narration from a test comment.

* fix(studio): cap the selection-sync retry so a dead member cannot stall the group

The bail-and-retry added for the drop/paste/duplicate race had no
upper bound: a group member that can never resolve (deleted out of
band, a stale id) made the whole group's selection retry forever
instead of settling. Cap it at a few attempts, then apply whatever
did resolve, same as the pre-fix graceful degradation, only bounded
now instead of unconditional on every sync.

* fix(studio): select and reveal a block dropped onto the preview canvas

handlePreviewBlockDrop (a real drag-and-drop onto the preview pane, wired
from usePreviewBlockDrop) was the fourth caller of addBlockToProject's
hostKey but never selected the new element, unlike the other three drop
paths this change wires up.

* fix(studio): close a stale retry-budget gap and dedupe the store-key helpers

Post-rebase simplify pass and independent review against the new head:
deriveTimelineStoreKeyForDomId now calls deriveTimelineStoreKey instead of
reimplementing its body, the composition-insert path uses the same helper
instead of hand-building the sourceFile#domId format, and the selection-sync
retry counter resets on deselect so reselecting an already-exhausted,
permanently-unresolvable id gets a fresh grace window instead of degrading
on the first attempt. Adds coverage for the asset-drop selection wiring and
the deselect/reselect retry-budget reset.

* style(studio): shorten a retry-budget docstring past the comment-line cap

* fix(studio): quiet the just-dropped-clip toast and reveal the full clip

The selection-not-found toast fired on the routine first miss while a
freshly dropped or pasted clip was still resolving; it now waits until
the retry budget is exhausted. The timeline focus coordinator likewise
cleared a reveal request on its first miss before the dropped element
had landed in logicalRows, cutting the clip off at the viewport edge;
it now tolerates a bounded number of retries like its selection-sync
sibling, so AssetCard and AudioRow reveals benefit too.

* refactor(studio): share one retry-budget tracker between the two hooks

The selection-sync and timeline-focus retry counters were the same
shape, incremented and reset independently. Extracted recordRetryAttempt
so both hooks call one tracker instead of maintaining their own copy.

* fix(studio): stop the post-edit scroll restore from undoing a fresh reveal

A dropped clip's reveal could still land cut off at the viewport edge
even with the retry fix: Timeline's "restore scroll after an edit
re-derives the elements" effect and the focus coordinator's reveal both
react to the same element-list change, but the restore effect captured
lastScrollLeftRef before the reveal had a chance to update it, so its
deferred frame stomped the reveal's scroll position back to the stale
pre-drop value. The reveal now keeps lastScrollLeftRef in sync with its
own write, and the restore effect reads the ref inside its frame instead
of capturing it early, so whichever effect runs last within the commit
wins with the freshest value. Confirmed against the original repro (a
Playwright drop scripted through the real dev server) that scrollLeft
converges to the fully-revealed value and stays there.

* docs(studio): note the load-bearing hook order behind the scroll-race fix

Flagged by an independent review of the prior commit: nothing in the
suite would catch a future refactor that reorders these two hook calls
and reintroduces the stale-scroll race.

* fix(studio): bound an unresolved focus request by wall time, not just retries

Answers two review questions on the scroll-race fix. (1) The retry
budget only advances when this effect re-runs, which needs some other
dependency to change; a request whose target never appears and which
nothing else ever re-renders sat unresolved forever, staying eligible
to resolve against an unrelated element that later reused its id. A
setTimeout backstop now clears it after 4s regardless of re-renders,
independent of the retry-count path. New test proves the leak and the
fix (fails without the timeout, passes with it).

(2) The "hook order is load-bearing" comment in Timeline.tsx no longer
holds once the geometry effect's ref read is deferred into its own
frame: a new test co-renders both hooks in both registration orders
and both converge, so the order claim is false for the current code.
Deleted the comment; the same test fails in geometry-first order when
run against the pre-deferral code, confirming order mattered before
and does not now.

* refactor(studio): drop a nonce check the store already performs

Simplify pass on the wall-time backstop commit. The timeout callback
guarded clearTimelineFocus behind a nonce match, but the store action
already no-ops on a mismatched nonce (a zustand same-reference return
short-circuits it) — the guard duplicated a check the setter owns, and
was inconsistent with the sibling budget-exhausted branch two lines up
that already calls it unguarded. Confirmed no other findings: the
two-mechanism clearing (retry count plus wall time) is genuine, not
redundant, and the timer has no leak path.
2026-09-18 11:13:48 -04:00
heygengenesis[bot]andmiguel.sierra 27303658c1 fix(core): scope selector arrays within compositions (#4035)
Co-authored-by: heygengenesis[bot] <262951085+heygengenesis[bot]@users.noreply.github.com>
Co-authored-by: miguel.sierra <229591595+miguel-heygen@users.noreply.github.com>
2026-09-18 11:12:52 -04:00