Commit Graph
2231 Commits
Author SHA1 Message Date
Ular Kimsanov a1abd57a60 Merge pull request #2010 from heygen-com/fix/studio-selection-outline-only
fix(studio): remove selection overlay fill
2026-07-06 21:21:57 -07:00
Miguel Ángel 68a8547f47 fix: align README skills table padding so oxfmt --check passes (#2014)
The media-use v2 row is the widest cell in the domain-skills table; oxfmt
re-pads every other row's trailing pipe to match. The merge added the row
without re-aligning, so oxfmt --check flagged README on CI (linux). Content
is unchanged; only trailing whitespace in 10 table rows.
2026-07-07 00:16:50 -04:00
ukimsanov 8f022d06ec fix(studio): remove selection overlay fill 2026-07-06 21:02:36 -07:00
Miguel Angel Simon Sierra ac0153cb30 feat(media-use): transcription (parakeet), transcript-cut, audio-duck editing tools 2026-07-06 23:41:35 -04:00
Miguel Angel Simon Sierra 5fe957363d feat(media-use): v2 media OS core (resolve cascade, providers, local generation, telemetry) + retire hyperframes-media 2026-07-06 23:41:05 -04:00
Miguel Angel Simon Sierra 676f461f6a feat(studio): non-destructive crop + cross-project asset view 2026-07-06 23:40:32 -04:00
Miguel Angel Simon Sierra 8c3590a90e feat(cli): parakeet ASR engine for transcribe (--engine) + HYPERFRAMES_PYTHON override 2026-07-06 23:34:32 -04:00
Miguel Ángel a2a80d5a5c fix(producer): mix muted browser media as silent for preview-render parity (#1969) 2026-07-06 23:33:50 -04:00
Vance Ingalls e9c37b5fdb Merge branch 'main' of github.com:heygen-com/hyperframes 2026-07-06 17:48:21 -07:00
Vance Ingalls d8d3a93b0d chore: release v0.7.38 v0.7.38 2026-07-06 17:47:34 -07:00
Miguel Ángel 9fe06f30da feat(cli): forward feedback submissions to the backend feedback endpoint (#2003)
* feat(cli): forward feedback submissions to backend endpoint

* fix(cli): truncate feedback fields to backend caps + ack before forwarding

Addresses PR review (via):
- Truncate comment (2000) / cli_version (100) / env (500) to the backend DTO
  caps before POSTing, so a pasted stack trace is forwarded truncated instead
  of rejected with a 422 the best-effort path swallows silently.
- Print "Thanks for the feedback!" before the best-effort forward so the ack
  isn't blocked behind the (bounded) network call.

* fix(cli): type feedback fetch mock
2026-07-06 20:44:42 -04:00
Vance IngallsandClaude Fable 5 1005703441 feat(engine,producer,cli): drawElement release telemetry on render_complete (#2002)
Default-on drawElement ships with a runtime self-verification net (#1998);
this makes its in-the-wild behavior observable. Every render_complete event
now answers: which capture mode actually ran, why drawElement disengaged
when it did (compile gate / producer clamp / engine init gate), whether the
self-verify net fired and why, and how much margin verification had.

Follows the static-dedup telemetry pattern: engine session fields →
CapturePerfSummary → RenderPerfSummary.drawElement → snake_case props on
render_complete.

New event props: de_capture_mode, de_compile_gate, de_clamp_reason,
de_gate_reason, de_worker_encode, de_verify_armed, de_verify_checked,
de_verify_min_db (margin above the 32dB threshold — drift here is the
early-warning signal before fallbacks spike), de_verify_init_ms,
de_self_verify_fallback, de_fallback_reason, de_blank_suspects,
de_blank_deterministic_accepts, de_blank_recaptures, de_boundary_frames,
de_ncpr_fallbacks.

Validated end-to-end on live renders: drawelement path reports mode/verify
counters/minDb/init cost; a blur-gated comp reports mode=screenshot +
gate_reason=css_effect:filter; a forced verification failure reports
self_verify_fallback=true + fallback_reason=psnr.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 17:39:46 -07:00
Xuanru LiandClaude Opus 4.8 906c8d04f8 fix(cli): localize remote assets before validate so it matches render (#2001)
validate served the composition over a loopback origin and let headless
Chrome fetch remote <img crossorigin>/@font-face assets cross-origin, while
the render pipeline downloads them to disk first. Buckets whose CORS
allowlist omits the loopback origin then failed the CORS-mode request with a
false net::ERR_FAILED that never occurs in the real render, pushing authors
(and agent pipelines) to delete crossorigin — which disables WebGL
color-grading/shaders for that asset.

Reuse producer's localizeRemote{Media,Image,FontFace}Sources in validate,
downloading into a temp dir served as an extra static-server asset root
(project dir untouched, cleaned up after). validate now matches render.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:20:18 -07:00
Vance Ingalls 89e36da13d fix(studio): shell UX — data-loss guards, error surfacing, dialog contracts, toasts (#1964) 2026-07-06 16:56:06 -07:00
Vance IngallsandClaude Fable 5 ccc1308839 docs(figma): storyboard blurb reworded + frames-are-app-states escalation (#2004)
Field feedback from a raw-API agent build (join-the-world-flow): the
catalog blurb's word 'animatics' encodes the PNG-slideshow architecture
the skill body explicitly forbids — an agent routing by the blurb
concludes the shipped behavior is frames-as-pictures. Reworded to
'reconstructed motion (frames read as states, not slides)' across all
catalog surfaces (skill frontmatter, CLAUDE.md, README, skills.mdx,
hyperframes router, figma guide).

Also codifies the stronger doctrine that build demonstrated as
storyboard rule 10: when every frame is the same product UI in
successive states, rebuild the app as live DOM (Phase-3 for stateful
parts, real pixels for static chrome — code what changes state, freeze
what doesn't) and perform frame deltas as interactions instead of
tweens. Spec §5.1 records the escalation + field origin.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:55:47 -07:00
Vance Ingalls 241f9d683e feat(studio,studio-server,cli): render cancel end-to-end + renders/nle/storyboard UX (#1963) 2026-07-06 16:43:48 -07:00
Vance IngallsandClaude Fable 5 ec06f4bf89 feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net (#1998)
* feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net

Flip useDrawElement + worker-encode defaults on (HF_DE_BATCH default 4),
clamped in resolveConfig to hosts where drawElement can engage (macOS +
hardware-GPU browser) so page-side shader compositing is untouched
everywhere else; explicit env opt-in keeps attempt-and-gate semantics.

Safety net makes default-on safe: the compile/init gates catch predictable
incompatibility; this catches the intermittent residue no static analysis
can see (stale paints, dropped background images, transient blank frames).

- engine: captureDeVerificationFrames — K=4 (HF_DE_VERIFY) ground-truth
  screenshots at init, after gates + armStaticDedup, BEFORE canvas
  injection (post-injection screenshots show the canvas bitmap, not the
  DOM). Runs the video-injection hook per sample; double-captures so
  rAF-driven text counters settle (a single immediate screenshot captures
  stale text and false-positives). Skips png, <10 frames, implausible
  __hf.duration (infinite-repeat GSAP sentinel).
- producer: guardFrame on both worker-encode drains — rolling-median blank
  guard with retry-once at drain (byte-identical retry ⇒ deterministic dark
  frame, accepted; retry save/restores the static-dedup anchor) + ffmpeg
  PSNR self-verify vs ground truth (HF_DE_VERIFY_MIN_DB, default 32dB;
  natural agreement ≥45dB, damage ≤25dB). Breach dumps the frame pair to
  tmpdir and throws DrawElementVerificationError.
- orchestrator: one-shot retry — on verification error the whole render
  re-runs with forceScreenshot (slower, never wrong); telemetry flag
  deSelfVerifyFallback.
- tooling: de-canary-suite.sh (7-comp release gate with expected verdicts),
  de-gatecheck.sh (init-only corpus routing classifier), we-render.mjs.

Validated: canary suite 7/7; 611-comp routing sample 54% drawelement /
37.5% gated / 8.3% comp-defect; 12/12 risk-band renders clean on bare
defaults (48/48 verify samples); engine suite 888 passed; caught two real
intermittent damage classes in the wild (background-image drop, root-props
offset) that previously shipped silently.

Kill switches: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false,
HF_DE_WORKER_ENCODE=false, HF_DE_BATCH=0, HF_DE_VERIFY=0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(engine,producer): harden the drawElement self-verification net (max code-review findings)

15 confirmed findings from the adversarial review of the default-on flip;
the load-bearing five:

- Ground-truth capture no longer scrubs GSAP state: seek(0) + forced frame
  FIRST (lazy .from()/overlap tweens record start values on first seek —
  mid-timeline scrubs corrupted them for the whole render, and since DE
  frames and truth shared the corruption, PSNR passed on damaged output),
  then ascending even-spread fractions, page left at frame 0.
- Default-on drawElement is confined to the verified path: resolveConfig
  requires worker-encode (the drain that runs the net), the orchestrator
  disengages the default when the render takes the disk path or parallel
  capture (no drain verification there), and closes a drawElement-initialized
  probe session rather than letting the unverified path reuse it. Explicit
  PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true keeps old attempt-and-gate behavior.
- Blank-frame retry can no longer splice wrong-frame pixels: recapture goes
  through recaptureDrawElementFrameForVerify — no static-dedup shortcut
  (lastEncodeResult runs ahead of the drain) and no "No cached paint record"
  screenshot fallback (post-injection that captures the canvas = the LAST
  drawn frame); any recapture failure falls back the whole render.
- Verify indices derive from the producer-resolved duration
  (CaptureOptions.compositionDurationSeconds) instead of raw __hf.duration,
  so samples always land inside the drained range.
- The platform clamp accepts "auto" GPU mode — the stock CLI resolves auto,
  and the literal-"hardware" clamp made default-on a no-op for the primary
  audience (masked in validation by explicitly-set env).

Also: NaN-safe env parses (HF_DE_VERIFY / HF_DE_VERIFY_MIN_DB / HF_DE_BATCH);
video comps skip verification when the session has no frame injector (probe
sessions — black-video truth false-positived); psnr infrastructure failures
skip the sample instead of failing the render; boundary-saturated sample
indices are skipped; shader-transition comps prefer page-side compositing
over default drawElement and compile-gated comps get page-side compositing
restored; observability.clearFailure un-brands the recovered first streaming
attempt; canary suite exempts known-marginal "any" comps from the cross-path
PSNR gate; dead we-render options removed; clamp tests pin their env.

Validated: canary suite 7/7; auto-GPU bare render engages the full stack;
disk-path and worker-encode-off renders disengage default drawElement;
malformed HF_DE_VERIFY_MIN_DB still verifies at the default threshold;
engine suite 890 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(engine,producer): review fixes — PSC intent, verify-threshold clamp, fail-closed canaries

Addresses miguel-heygen's review on #1998:

- Page-side compositing restore preserves explicit caller intent (blocker):
  resolveConfig now records pageSideCompositingAutoDisabled only when IT
  turned page-side compositing off because drawElement was on; the
  compile-time drawElement gates restore page-side compositing only when
  that flag is set. An explicit enablePageSideCompositing:false from the
  programmatic API or HF_PAGE_SIDE_COMPOSITING=false stays off. Pinned by
  two config tests.
- HF_DE_VERIFY_MIN_DB clamped to [10, 60] with a warning on out-of-range
  values: below ~10dB the check passes severe damage; above ~60dB natural
  encoder differences force a screenshot fallback on every verified render.
- de-canary-suite.sh + de-gatecheck.sh run under set -euo pipefail with
  explicit `|| true` on expected-nonzero commands (render exits handled by
  the suite's own checks, grep no-match, kill/pkill/wait races) and a hard
  FAIL when the PSNR compare produces no value — release canaries fail
  closed. Full suite re-run green (7/7) under the new flags.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:29:30 -07:00
Vance IngallsandClaude Opus 4.8 d5ecb013d7 perf(engine,producer): batch N drawElement frames per CDP round-trip (HF_DE_BATCH) (#1928)
Amortizes per-frame CDP protocol overhead (~3.5-9ms/frame) by looping
seek -> paint-wait -> drawElementImage -> createImageBitmap in ONE
page.evaluate for runs of consecutive frames; bitmaps still post to the
encode worker per frame. Validated on 19 stratified DE comps: median
1.20x on top of worker-encode (to 1.56x), zero damaged frames, edge
comps (static-dedup-heavy, clip-cut) bit-identical; mid-batch failure
re-captures via the per-frame path (screenshot-fallback semantics
preserved). Off by default; opt in with HF_DE_BATCH=4.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:27:31 -07:00
Vance IngallsandClaude Fable 5 992a9b6607 feat(lint,player): fast-capture lint rule + player media sync (#1921)
* feat(engine): drawElementImage capture service

* feat(engine): 3D projection + compositor-effect risk gate

* fix(engine): gate filter drop-shadow wherever blur gates (review)

detectCssEffectRisk documented drop-shadow as a ~29dB damage case but only
detected blur( in its three scan paths — a drop-shadow comp stayed on the
fast path despite the gate's own correctness contract. Detect drop-shadow(
in computed styles, stylesheet rules, and GSAP tween vars, pinned by a
focused test that runs the real page-side closure against a DOM shim
(computed / stylesheet / tween coverage + blur regression + effect-free
null).

Addresses miguel-heygen's blocker on #1918.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(engine): frame-capture core — fast-capture routing, worker-encode, dedup extension

# Conflicts:
#	packages/engine/src/services/screenshotService.ts

* fix(engine): document HF_FORCE_DRAWELEMENT as diagnostic-only; make armStaticDedup idempotent (review)

Addresses miguel-heygen's blockers on #1919:

- HF_FORCE_DRAWELEMENT promoted from a stale "SCRATCH/Uncommitted" comment to
  a documented diagnostic flag: it exists for upstream-Chromium repro work
  (gate-vs-API isolation, crbug 521861819 149-vs-151) and R&D on gated effect
  classes; renders under it may be damaged BY DESIGN since it bypasses gates
  whose thresholds encode measured damage. Never production; the safety-net
  blank guard also stands down under it so diagnostic frames arrive unmodified.
- armStaticDedup is now idempotent: the drawElement init path arms dedup
  before canvas injection, then initializeSession called it again — the
  second run overwrote the armed state with skipReason="capture_mode"
  (captureMode is "drawelement" by then), producing contradictory telemetry
  (armed frames + a skip reason), and re-ran the verification seeks on the
  fallback path. It now no-ops once staticFrames or a skip decision exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(producer): fast-capture render stages + remote bg-image localizer

* feat(lint,player): fast-capture lint rule + player media sync

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:22:35 -07:00
Vance IngallsandClaude Fable 5 1d0dbcd3b2 feat(producer): fast-capture render stages + remote bg-image localizer (#1920)
* feat(engine): drawElementImage capture service

* feat(engine): 3D projection + compositor-effect risk gate

* fix(engine): gate filter drop-shadow wherever blur gates (review)

detectCssEffectRisk documented drop-shadow as a ~29dB damage case but only
detected blur( in its three scan paths — a drop-shadow comp stayed on the
fast path despite the gate's own correctness contract. Detect drop-shadow(
in computed styles, stylesheet rules, and GSAP tween vars, pinned by a
focused test that runs the real page-side closure against a DOM shim
(computed / stylesheet / tween coverage + blur regression + effect-free
null).

Addresses miguel-heygen's blocker on #1918.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(engine): frame-capture core — fast-capture routing, worker-encode, dedup extension

# Conflicts:
#	packages/engine/src/services/screenshotService.ts

* fix(engine): document HF_FORCE_DRAWELEMENT as diagnostic-only; make armStaticDedup idempotent (review)

Addresses miguel-heygen's blockers on #1919:

- HF_FORCE_DRAWELEMENT promoted from a stale "SCRATCH/Uncommitted" comment to
  a documented diagnostic flag: it exists for upstream-Chromium repro work
  (gate-vs-API isolation, crbug 521861819 149-vs-151) and R&D on gated effect
  classes; renders under it may be damaged BY DESIGN since it bypasses gates
  whose thresholds encode measured damage. Never production; the safety-net
  blank guard also stands down under it so diagnostic frames arrive unmodified.
- armStaticDedup is now idempotent: the drawElement init path arms dedup
  before canvas injection, then initializeSession called it again — the
  second run overwrote the armed state with skipReason="capture_mode"
  (captureMode is "drawelement" by then), producing contradictory telemetry
  (armed frames + a skip reason), and re-ran the verification seeks on the
  fallback path. It now no-ops once staticFrames or a skip decision exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(producer): fast-capture render stages + remote bg-image localizer

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:21:39 -07:00
Vance IngallsandClaude Fable 5 0e58344dca feat(engine): frame-capture core — fast-capture routing, worker-encode, dedup extension (#1919)
* feat(engine): drawElementImage capture service

* chore(ci): ignore drawElementService exports pending upstack consumers

Fallow's per-PR audit diffs against the merge base, so the bottom of the
fast-capture stack (#1917) sees drawElementService's exports as unused —
their consumers (frameCapture) land in #1919, two PRs upstack. ignoreExports
entry documents this and can be dropped once #1919 merges.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(engine): 3D projection + compositor-effect risk gate

* fix(engine): gate filter drop-shadow wherever blur gates (review)

detectCssEffectRisk documented drop-shadow as a ~29dB damage case but only
detected blur( in its three scan paths — a drop-shadow comp stayed on the
fast path despite the gate's own correctness contract. Detect drop-shadow(
in computed styles, stylesheet rules, and GSAP tween vars, pinned by a
focused test that runs the real page-side closure against a DOM shim
(computed / stylesheet / tween coverage + blur regression + effect-free
null).

Addresses miguel-heygen's blocker on #1918.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(engine): frame-capture core — fast-capture routing, worker-encode, dedup extension

# Conflicts:
#	packages/engine/src/services/screenshotService.ts

* fix(engine): document HF_FORCE_DRAWELEMENT as diagnostic-only; make armStaticDedup idempotent (review)

Addresses miguel-heygen's blockers on #1919:

- HF_FORCE_DRAWELEMENT promoted from a stale "SCRATCH/Uncommitted" comment to
  a documented diagnostic flag: it exists for upstream-Chromium repro work
  (gate-vs-API isolation, crbug 521861819 149-vs-151) and R&D on gated effect
  classes; renders under it may be damaged BY DESIGN since it bypasses gates
  whose thresholds encode measured damage. Never production; the safety-net
  blank guard also stands down under it so diagnostic frames arrive unmodified.
- armStaticDedup is now idempotent: the drawElement init path arms dedup
  before canvas injection, then initializeSession called it again — the
  second run overwrote the armed state with skipReason="capture_mode"
  (captureMode is "drawelement" by then), producing contradictory telemetry
  (armed frames + a skip reason), and re-ran the verification seeks on the
  fallback path. It now no-ops once staticFrames or a skip decision exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:21:12 -07:00
Vance IngallsandClaude Fable 5 4749fe5716 feat(engine): 3D projection + compositor-effect risk gate (#1918)
* feat(engine): drawElementImage capture service

* chore(ci): ignore drawElementService exports pending upstack consumers

Fallow's per-PR audit diffs against the merge base, so the bottom of the
fast-capture stack (#1917) sees drawElementService's exports as unused —
their consumers (frameCapture) land in #1919, two PRs upstack. ignoreExports
entry documents this and can be dropped once #1919 merges.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(engine): 3D projection + compositor-effect risk gate

* fix(engine): gate filter drop-shadow wherever blur gates (review)

detectCssEffectRisk documented drop-shadow as a ~29dB damage case but only
detected blur( in its three scan paths — a drop-shadow comp stayed on the
fast path despite the gate's own correctness contract. Detect drop-shadow(
in computed styles, stylesheet rules, and GSAP tween vars, pinned by a
focused test that runs the real page-side closure against a DOM shim
(computed / stylesheet / tween coverage + blur regression + effect-free
null).

Addresses miguel-heygen's blocker on #1918.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:19:44 -07:00
Vance IngallsandClaude Fable 5 795934adba feat(engine): drawElementImage capture service (#1917)
* feat(engine): drawElementImage capture service

* chore(ci): ignore drawElementService exports pending upstack consumers

Fallow's per-PR audit diffs against the merge base, so the bottom of the
fast-capture stack (#1917) sees drawElementService's exports as unused —
their consumers (frameCapture) land in #1919, two PRs upstack. ignoreExports
entry documents this and can be dropped once #1919 merges.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:19:17 -07:00
Vance Ingalls cd1adcb581 feat(studio): harden ui primitives — focus rings, keyboard tooltips, dialog hook (#1962)
## Summary

Base of the studio UX-review stack (148 findings audited across the studio; 13 critical). This PR hardens the shared `components/ui` primitives that every later PR in the stack builds on.

## Changes

- **Button / IconButton**: visible `focus-visible` outline (studio accent); `disabled:pointer-events-none` removed (replaced with `disabled:cursor-not-allowed`, hover/active gated behind `enabled:`) so disabled buttons can host explain-why tooltips.
- **Tooltip**: keyboard support (`onFocus`/`onBlur` triggers), `role="tooltip"`, Escape-to-hide, viewport flip (top↔bottom) + horizontal clamping. API unchanged — all ~28 call sites unaffected.
- **HyperframesLoader**: `role="status"` on the loader; determinate track is a real `role="progressbar"` with `aria-valuenow/min/max` (was `aria-hidden`).
- **VideoFrameThumbnail**: error event resolves to a static fallback-label tile instead of an infinite shimmer; `motion-reduce` guard.
- **NEW `useDialogBehavior`**: shared modal contract — document-level Escape, Tab focus trap, focus-first-on-open, focus-restore-on-close, `canClose()` veto for dirty-draft guards. Adopted by every modal later in the stack.
- **NEW `SearchInput`**: shared search primitive with required `aria-label`, panel-input token style (kills the two-divergent-search-styles inconsistency in the sidebar).
- **studio.css**: `hf-toast-in/out` + `hf-backdrop-in` keyframes with `prefers-reduced-motion` guards (the previous `animate-in fade-in` classes were dead — no tailwindcss-animate plugin exists).

## Verification

- oxlint 0 errors, oxfmt clean, `tsc --noEmit` clean at stack top
- Full studio suite at stack top: 1189 tests pass

## Stack

PR 1/7 of the studio UX-review fixes. Merges bottom-up; the stack top is fully green (tsc + 1189 tests). Some shared-file edits span PRs, so intermediate branches may not typecheck in isolation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-06 16:07:44 -07:00
Vance Ingalls e04f6dda37 feat(engine,cli): drawElement fast-capture config + CLI flag (#1916)
## drawElement fast-capture — config + CLI flag (stack 1/6)

Foundation layer for the drawElement fast-capture feature: the config surface and CLI/Docker plumbing that the rest of the stack builds on.

### What this adds
- **`packages/engine/src/config.ts`** — new config fields for fast capture: `useDrawElement` / `enableDrawElementWorkerEncode` (macOS-GPU `drawElementImage` capture + worker-offloaded JPEG encode), resolved from env in `resolveConfig` (env `HF_DE_WORKER_ENCODE`). Wired alongside main's existing `staticFrameDedup` (unified downstream in 4/6).
- **`packages/cli/src/commands/render.ts`** — `--experimental-fast-capture` flag → sets `experimentalFastCapture`; `--debug` passthrough.
- **`packages/cli/src/utils/dockerRunArgs.ts`** — pass the fast-capture env through to the container.
- **`.github/workflows/fast-video-validation.yml`** — CI job validating fast-capture renders.
- `.oxlintrc.json` / `.fallowrc.jsonc` — ignore-pattern housekeeping for the new paths.

### Notes
- Config-only + entrypoint; no capture behavior yet (that's 2/6–4/6).
- Tests: `config.test.ts`, `dockerRunArgs.test.ts` added.

---
**Stack (drawElement fast-capture, rebased onto current `main`, supersedes #1295 + #1444):**
1. **#1916 config + CLI** ← you are here
2. #1917 drawElementImage capture service
3. #1918 3D projection + compositor-effect risk gate
4. #1919 frame-capture core (routing, worker-encode, static-dedup unification)
5. #1920 producer render stages + remote bg-image localizer
6. #1921 lint rule + player media sync

⚠️ Intermediate PRs (1–5) are split by package boundary for review and **do not each compile independently** (cross-file deps); the complete feature is green at the stack tip (#1921) — tsc-clean on engine + producer, 231 tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-06 15:57:30 -07:00
Ular Kimsanov 972bedf062 Merge pull request #1988 from heygen-com/feat/color-grading-docs
docs: document Studio Color Grading
2026-07-06 15:40:46 -07:00
ukimsanov b38c054baf docs: document Studio Color Grading 2026-07-06 15:29:37 -07:00
Ular Kimsanov c1aa35d069 Merge pull request #1987 from heygen-com/feat/studio-color-grading-inspector
feat(studio): add color grading inspector
2026-07-06 15:24:50 -07:00
Miguel Ángel 1a3330a972 fix(compiler): don't scan media tags inside comments/scripts; DOM-check auto-start (#1938) (#1940)
The timing compiler scanned raw HTML with tag regexes that weren't
comment-aware, so a comment or script merely mentioning `<video>`/`<audio>`
was rewritten as a real element — injecting id/data-start/data-hf-auto-start
into the comment text. That phantom attribute then tripped the probe stage's
substring check (`html.includes("data-hf-auto-start")`), launching an
unnecessary browser probe on every render with an unexplained empty reasons list.

- Mask comments, <script>, and <style> regions before the tag scan, then
  restore them verbatim (compileTimingAttrs, extractResolvedMedia).
- Replace the probe's substring match with a DOM query
  (video[data-hf-auto-start]) and add "auto-start video(s)" to the reasons list.
2026-07-06 18:19:35 -04:00
Miguel Ángel afa7f292fb feat(lint): warn when <html dir> can blank render output (#1893)
* feat(lint): flag dir="rtl" on <html> as a confirmed silent render failure

Two independent reports diagnosed the same exact bug: dir="rtl" (or any
non-ltr value) on <html> renders correctly in preview/snapshot but
produces a fully blank/black video from render, with no other
lint/validate/inspect check catching it - output file size (far smaller
than expected) was the only tell for both reporters. Both independently
confirmed the same fix: drop dir from <html>, keep lang, and scope
direction: rtl to individual text-containing elements via CSS instead.

Could not empirically verify the render pipeline's own root cause in this
session (headless Chrome screenshot capture is unreliable in this
sandboxed environment - even a baseline, non-RTL capture timed out), so
this ships the safe, already-confirmed advisory rather than guessing at a
runtime fix. Both reporters explicitly asked for exactly this: "deserves
a lint rule or render-time warning."

* fix(lint): only flag valid non-ltr html dir values
2026-07-06 18:16:40 -04:00
ukimsanov 04f96da169 fix(studio): share safe media path cleanup 2026-07-06 14:35:41 -07:00
ukimsanov e0090efbf6 fix(studio): avoid regex in media asset path cleanup 2026-07-06 14:15:30 -07:00
Miguel Ángel 769dc702c1 feat(cli): emit sign-in lifecycle telemetry (#2000)
* feat(cli): emit sign-in lifecycle telemetry

The CLI tracks command and render lifecycles but emits nothing for
`auth login`, so sign-in outcomes are invisible on the observability
dashboards — a completed sign-in, an abandoned browser flow, and a
rejected key all look identical (absent). This leaves a blind spot in
the same funnel the render events already cover.

Add three events mirroring the existing `trackX` pattern:
  - auth_login_started    (method: oauth | api_key)
  - auth_login_completed  (method)
  - auth_login_failed     (method, reason)

`reason` is a fixed low-cardinality enum (flow_error / no_credential /
rejected / invalid_input). No token, key, identity, email, or free text
is ever attached — consistent with the existing anonymous telemetry and
the `telemetry disable` opt-out. Wired into both the OAuth and
--api-key paths in `auth login`, with unit coverage for the new events.

* fix(cli): close sign-in telemetry funnel dropout gaps

Follow-up so `started` reconciles to `completed + failed` on the common
abandonment paths, which the first cut missed:

- Interactive prompt cancel (Ctrl-C) now surfaces as a throw that the
  single catch in the api-key path records as `aborted`, instead of a
  bare exit with no event.
- A stdin read that times out in non-TTY `--api-key` mode now records
  `aborted` before the error propagates, rather than exiting silently.
- OAuth split: a timed-out browser callback (user closed the tab) is
  tagged `flow_timeout`, separated from real `flow_error` (IdP/network),
  since the walk-away timeout is the dominant non-error dropout.

Also pre-plumb an optional `distinctId` on the three trackers, mirroring
trackRenderComplete/trackRenderError. Unused today; it lets a later
identity-level attribution be a one-line callsite change rather than a
signature sweep. Coverage added for the new reasons and forwarding.
2026-07-06 16:59:47 -04:00
ukimsanov 30a944de34 fix(studio): split color grading inspector files 2026-07-06 13:56:37 -07:00
ukimsanov 139cf568b4 feat(studio): add color grading inspector 2026-07-06 13:40:20 -07:00
Ular Kimsanov a4989dbe75 Merge pull request #1986 from heygen-com/feat/studio-media-processing-routes
feat(studio-server): add media processing routes
2026-07-06 13:39:01 -07:00
ukimsanov 413ee07da5 fix(studio-server): share background removal job runner 2026-07-06 13:27:45 -07:00
ukimsanov a3bf7eb995 feat(studio-server): add media processing routes 2026-07-06 13:25:04 -07:00
Ular Kimsanov 2757d32092 Merge pull request #1985 from heygen-com/feat/studio-color-grading-runtime
feat(runtime): render media color grading shaders
2026-07-06 13:18:06 -07:00
ukimsanov 644ae384a5 fix(runtime): satisfy color grading audit 2026-07-06 12:49:34 -07:00
ukimsanov 870964b0cf feat(runtime): render media color grading shaders 2026-07-06 12:44:43 -07:00
Ular Kimsanov e01ec16194 Merge pull request #1984 from heygen-com/feat/studio-color-grading-core
feat(core): add media color grading contract
2026-07-06 11:30:31 -07:00
ukimsanov 7c14438496 feat(core): add media color grading contract 2026-07-06 02:36:17 -07:00
Vance IngallsandClaude Fable 5 3a717fa719 fix(parsers,sdk,studio-server,studio): unify hf-id space across preview, disk, and SDK session (#1981)
* fix(parsers,sdk,studio-server,studio): unify hf-id space across preview, disk, and SDK session

Root-causes the setTiming element_not_found resolver-shadow divergence class:
timeline edits carry hf-ids read from the live preview DOM, but the preview
minted ids AFTER rewriting attributes (and never persisted them for sub-comps),
while the SDK session mints from the raw file — content-keyed minting then
yields different ids for the same element. Template-based comps were worse:
the SDK excluded the whole <template> subtree, so the session had zero
elements and every edit diverged.

- parsers: ensureHfIds now descends into <template> subtrees (linkedom's
  querySelectorAll does not), minting and pinning inner ids
- sdk: buildRoots/buildElement treat <template> as a transparent container,
  and resolution (resolveScoped, animation-id map) searches template subtrees
  via querySelectorAllDeep — template comps now model, resolve, and edit
- studio-server: the sub-comp preview route persists hf-ids to the raw file
  BEFORE the rewrite pipeline (mirrors the main route), pinning one id space
  across served DOM, disk, and SDK session
- studio: resolver-shadow skips structurally-empty sessions (no event, no
  attempt) and tags fail-open emissions with sourceReadFailed so read errors
  are distinguishable from unwired readers in telemetry

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(parsers,sdk,studio-server,studio): scope template descent, guard persist route

Addresses the 10 verified findings from the PR #1981 review:

- Restrict template transparency to COMPOSITION templates
  (<template data-composition-id>) everywhere — ensureHfIds, SDK
  buildChildren, querySelectorAllDeep. A plain <template> (runtime
  clone-source) keeps its old fully-excluded behavior: stamping its
  interior would duplicate one persisted id across every runtime clone,
  and modeling it would show phantom timeline clips.
- Guard the sub-comp persist: only .html files (the wildcard route can
  serve any project path — stamping an SVG corrupted it on disk),
  try/catch the read (file-removed race becomes 404, not 500), salt the
  etag (v2) so pre-fix cached clients don't 304 past the id pin, and
  thread the stamped content into buildSubCompositionHtml so served ids
  match the mint even when the disk write is skipped.
- Rewrite querySelectorAllDeep as a document-order DOM walk — appending
  template matches after top-level matches made duplicate-id tiebreaks
  disagree with the preview's unwrapped DOM (wrong-element edits).
- Recurse sourceMutation.querySelectorAllWithTemplates so server-side
  ops resolve ids at any template depth, matching SDK resolution.
- Replace the empty-session silent skip with ONE tagged session_empty
  event per session — silence would blind the tripwire to exactly the
  modeling-gap class that exposed the template bug. Attempts stay
  uncounted (an unmodelable comp can't cut over).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(studio-server): close TOCTOU in sub-comp hf-id persist (CodeQL js/file-system-race)

Replace the route-level stat/read/persist sequence with stampFileHfIds:
validation (fstat), read, mint, and write-back all go through ONE open
file descriptor (O_NOFOLLOW where supported), so the path cannot be
swapped between validation and write. Falls back to read-only stamping
when the file isn't writable — content-keyed minting means the SDK
derives the same ids from the same bytes even without the disk write.

Addresses miguel-heygen's blocking review on PR #1981.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(studio-server): linear-time template-attr match (CodeQL js/polynomial-redos)

promoteTemplateCompositionId's single-pattern regex backtracked
polynomially on crafted input. Two-step match: grab each <template>
open tag linearly, then find data-composition-id within that short
tag text. Same semantics (first template carrying the attr wins).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 00:16:07 -07:00
Vance IngallsandClaude Fable 5 e9076324e7 feat(cli): figma import telemetry — subcommand labels, typed error codes, figma_import event (#1979)
Closes the observability gaps on the figma integration:
- withFigmaErrors takes a command label (figma:asset|tokens|component) and
  reports the failure inline before its process.exit — the top-level
  trackCommandFailures wrapper never sees self-exiting commands, so typed
  codes (NO_TOKEN, BAD_TOKEN, FORBIDDEN, RATE_LIMITED) were invisible.
  FigmaClientError codes surface as the error name for dashboarding the
  first-run funnel (NO_TOKEN -> later success = onboarding conversion).
- new figma_import event per import: phase, duration, reused (dedup
  effectiveness), tokens variables-vs-styles mode + entry count
  (Enterprise gating rate), unresolved-binding + rasterized-node counts
  (fidelity degradation). No fileKeys, node ids, names, or descriptions.
- /figma skill fires the events beacon (figma-motion / figma-shaders /
  figma-storyboard) for the MCP phases that never touch the CLI.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 23:27:58 -07:00
James RussoandClaude Opus 4.8 232d591479 feat(cli/telemetry): surface unrecognized agents in the agent_runtime=null bucket (#1978)
* feat(cli/telemetry): surface unrecognized agents in the agent_runtime=null bucket

agent_runtime is a closed allowlist: an agent we have no rule for collapses
to null with no trace of what it was, so ~18% of CLI users are unattributable
and new agents stay invisible until reverse-engineered by hand.

Add detectAgentHints(), a self-populating residual signal computed only for
the null bucket (gated off classified events):
- agent_hint: value of AGENT / AI_AGENT (the emerging self-identification
  convention; Crush and Goose set AGENT=<name>) — names agents the allowlist
  misses.
- term_program: raw TERM_PROGRAM (editor name) — catches the IDE-terminal
  class the same way the cursor/windsurf rules do.
- agent_env_hints: sorted, comma-joined "agent-ish" env-var KEY names present
  but matched by no vendor rule — a fingerprint that clusters by agent.

Privacy stays consistent with the existing "never read secret-shaped values"
stance: agent_env_hints emits key names only; the three value-reads are vars
whose sole purpose is non-secret identification, each passed through a strict
short-slug allowlist so anything long/spaced/secret-shaped is dropped.

Breaking down agent_hint / agent_env_hints filtered to agent_runtime IS NULL
AND is_tty=false gives a ranked leaderboard of new agents to promote into
VENDOR_RULES.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli/telemetry): guard agent_hint/term_program against short credential-shaped values

Review feedback (Magi, #1978): the short-slug allowlist in sanitizeHint() still
accepted short credential-shaped values (AGENT=sk-ant-api03,
AGENT=AKIAIOSFODNN7EXAMPLE, AGENT=github_pat_abc), so the "never emit a secret"
claim wasn't actually enforced — only overlong values were dropped.

Add a credential-shape guard on top of the slug allowlist:
- known token/credential prefixes (sk-, ghp_, github_pat_, akia, ya29, ...)
- any unbroken alphanumeric run >= 16 chars (key bodies, hex, base64-ish),
  while agent names segment on _/-/. and keep each run short.

Replace the single overlong-value test with the short credential shapes from the
review (parametrized) plus a positive case (gemini_managed_agent) proving real
multi-segment names still pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 22:31:13 -07:00
James 9de41ce316 chore: release v0.7.37 v0.7.37 2026-07-06 05:59:36 +01:00
James RussoandClaude Opus 4.8 2b1109db4c test(lambda): behavioral guard for handler CJS banner shims (#1977)
Miguel's fix (f999b40d) added the require/__filename/__dirname shims to the
Lambda handler bundle after #1932 crashed every render at import with
"__dirname is not defined in ES module scope" (wawoff2's emscripten build
reads __dirname at module scope; it's inlined via producer -> fontCompression).

The accompanying test only grepped build-zip.ts for the banner literals, so it
passes even if the shim is renamed, reordered into a broken form, or if a new
inlined CJS dep needs a global the banner doesn't provide.

Replace it with a behavioral test: extract the banner to _handlerBanner.ts
(build-zip.ts self-executes on import, so it can't be imported directly),
bundle a fixture that touches __dirname/__filename/require with the real
banner, and import the output under real Node -- not the bun test runtime,
which defines __dirname in ESM and would mask a missing shim. The import
faithfully reproduces Lambda's Node ESM environment and fails with the exact
#1932 error when any shim is dropped.

Handler bundle output is unchanged (identical banner string).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 21:51:08 -07:00
Ular Kimsanov 94996d8da9 Merge pull request #1976 from heygen-com/docs-audit-fixes
docs: fix 53 inaccuracies across all documentation
2026-07-05 21:34:05 -07:00
ukimsanov 52d586dd31 docs: fix 53 inaccuracies across all documentation
Exhaustive audit of every MDX file in docs/ against skill references
and package source code. Every API signature, default value, flag,
and technical claim verified against ground truth.
2026-07-05 21:26:41 -07:00