mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-09 03:16:38 +00:00
4dbf0d90b05de2c3db7c983641c8785fd9988148
56
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vance IngallsandClaude Opus 5
|
4dbf0d90b0 |
fix(producer): fail closed when no live element count is available
R4 review finding, and the comment I wrote in R3 was simply wrong: the
probe session is NOT running for every render. probeStage's needsBrowser
gate launches one only for unknown duration, unresolved compositions, or
specific media cases — and hasRuntimeInsertedMedia matches only
createElement("video"|"audio"), never createElement("span"). So the exact
shape that motivated the live-DOM fix (a known-duration, media-free
caption comp building thousands of nodes in script) gets NO probe, falls
back to the static source scan, reads as ~2 elements, and could enter the
applied cohort at 40k live nodes. The R3 fix measured the right thing but
only for the population that already had a probe.
Now the count carries provenance and the band fails closed:
- resolveCompositionElementCount returns { count, source: "live" |
"static" }. Only "live" — an actual DOM measurement — may open the band.
- resolveDeShortBand gains a third decisive outcome, "unmeasured", for
the static case. It deliberately does NOT report skipped_elements: a
static undercount is not a real oversize observation, and putting it in
the control arm would contaminate the DiD just as putting it in the
treatment arm would. Neither cohort; never routes.
- composition_element_count_source ships alongside the count, so the
fleet rate of "static" sizes the population a future
conditional-probe-launch would unlock — which is the data PR B needs to
decide whether that launch cost is worth paying.
Regression coverage walks the real chain rather than a full render, using
the production functions in pipeline order: probeRequiresBrowser (newly
extracted from the inline needsBrowser expression, so the gate is
testable at all) returns false for the caption-comp shape → the resolver
reports static and a count under the ceiling → the band reports
unmeasured, not applied. Fault injection confirms it bites: removing the
one guard line fails exactly these three tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
0749cd9ff8 |
feat(producer): open the DE single-worker inversion to short comps under an element ceiling
31% of fleet renders (24h, v0.7.78+) are DE-eligible comps clamped to
parallel screenshot purely because they sit under the 900-frame inversion
floor — the median fleet render is ~250-600 frames, below every DE entry
threshold. This opens a 250-899 frame band, gated on composition size.
Measured, not assumed. A controlled sweep (fixed synthetic content,
{250,400,600,900}f, single-DE vs parallel-screenshot-W4, 3 reps, capture
mode verified per row, AC power, load-gated) showed single-DE winning
1.16-1.24x at every size — but only for content in constant motion. A
follow-up 2x2 found motion and DOM size pull in OPPOSITE directions, so
neither alone predicts the winner (ratio = ss4/de1, >1 means DE wins):
24 movers / 0 nodes -> 1.05
320 movers / 0 nodes -> 1.24
320 movers / 7000 nodes -> 1.09
24 movers / 7000 nodes -> 0.96
24 movers / 20000 nodes -> 0.71
24 movers / 40000 nodes -> 0.55
DE's wall-clock scales ~0.50ms/element against parallel screenshot's
~0.22ms — drawElement repaints the whole tree per frame while fan-out
amortizes it — so the downside is NOT bounded and a bare floor drop would
have handed a 1.8x regression to large comps. Since motion only ever helps
DE, an element ceiling calibrated at the lowest-motion case is safe at
every motion level; crossover there is ~3.9k, and the default sits at 2500.
The predicate is untouched; the call site picks the floor. Above the
ceiling, or at 900+ frames, behaviour is bit-identical to today — the
change can only add inversions in the new band, never remove one.
Instrumentation, since this ships at full exposure rather than cohorted:
`composition_element_count` on EVERY render (the fleet distribution of the
gate variable is unknown — without it we cannot tell whether 2500 opens the
band for most short comps or almost none, nor re-derive the threshold from
real content), and `de_short_band` = applied | skipped_elements, unset when
the frame count made the band irrelevant, so a fleet perf shift is
attributable to this change rather than to content mix.
Safety is unchanged and already proven on this path: per-frame PSNR
self-verify with screenshot fallback, exactly as the 900+ band has shipped
default-on. Knobs: HF_DE_SHORT_MIN_FRAMES, HF_DE_SHORT_MAX_ELEMENTS (0
disables the band).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance Ingalls
|
011f46bc18 |
Merge remote-tracking branch 'origin/main' into 07-27-feat_producer_lower_parallel-de_router_floor_to_700_frames_power-state_telemetry
# Conflicts: # packages/cli/src/telemetry/events.ts |
||
|
Vance IngallsandClaude Opus 5
|
b99c803898 |
fix(producer): guard pmset behind shouldTrack + don't pin workers without streaming (review)
Two review findings on the floor/telemetry PR: 1. powerStateFields() is spread into the properties object at the CALL SITE, so it ran before trackEvent's own `if (!shouldTrack()) return` guard — telemetry-disabled installs paid two blocking `pmset` subprocess spawns per render for an event that was then discarded. Now short-circuits on shouldTrack() (memoized, so no cost on the tracked path). Regression test asserts pmset is not sampled when telemetry is off; fault-injection verified it fails without the guard. 2. The DE parallel router pinned workerCount to 3 and skipped calibration even when verified parallel DE STREAMING — the entire reason for the pin — could not run for that render. The common case is a composition over streamingEncodeMaxDurationSeconds (240 s default): the duration cap disables streaming before the router's force flag is consulted, so the render got a hard-coded 3 workers chosen by a benchmark for a path it was not on, instead of the calibrated count. shouldPreferParallelDrawElement now takes parallelStreamingAvailable and withholds the bet without it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
4520cd240b |
fix(engine): bucket gpu_renderer + cover the failure cohort (review)
Three review findings on the win32 drawElement PR: 1. gpu_renderer shipped the raw UNMASKED_RENDERER_WEBGL string — unbounded, driver-authored, GPU-model-specific, and |-joined across parallel sessions, i.e. high cardinality by construction, against this file's own convention of sanitizing engine-sourced strings (deGateReason is a bucket; error messages go through redactTelemetryString). Now bucketed at the source by classifyGpuRenderer to <backend>/<vendor> (metal/apple, d3d11/nvidia, swiftshader/other, ...), which is the whole analytic signal the win32 rollout needs and nothing else. The raw string never leaves the engine. 2. gpu_renderer reached render_complete only, so a crashed render — the cohort the field exists to attribute — carried no backend. It now rides RenderCaptureObservability (deGpuRenderer, sourced from the live probe session like the de_* counters), so both render_complete and render_error carry it and a hard failure still reports its GPU backend. On render_complete the perfSummary value still wins by spread order. 3. Restore the fallow-ignore-next-line suppression above __resetDeParallelRouterTrialStateForTests: CLI test files are not fallow entry points, so removing it fails the CI dead-code audit (local pre-commit passed only because of its changed-file scope). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
cb30157ebb |
feat(engine): open drawElement fast capture to Windows hardware GPU
Widen the default-on drawElement clamp from darwin-only to darwin|win32 (still requiring a non-software-GPU browser). The darwin restriction was a validation envelope, not an architectural limit — the CanvasDrawElement Chrome flag ships on every platform, and every safety layer that made the macOS default-on release (v0.7.38) survivable is platform-neutral: compile-time gates, the SwiftShader init gate, per-render worker-encode self-verification with screenshot fallback, and the blank guard. Worst case on an unvalidated D3D11 backend is the same as on Metal: verify catches a bad frame and the render re-runs on the screenshot baseline. Why now: 30-day telemetry shows ~206k non-CI hardware-GPU Windows renders (~78% of the win32 fleet, 18k installs) held on the slow screenshot path by the clamp — the second-largest perf population after macOS, carrying ~1,550 capture-hours/month in the DE-eligible >=700-frame band alone at a measured ~2x speedup opportunity. Instrumentation for the new cohort: drawElement session init now records the raw WebGL UNMASKED_RENDERER_WEBGL string (detectSwiftShader generalized to detectGpuBackend — same single evaluate, the string was previously read and discarded) and threads it session -> CapturePerfSummary -> RenderPerfSummary -> render_complete as `gpu_renderer`. drawElement damage proved compositor-backend-specific throughout the macOS rollout, so D3D11-cohort failures must cluster by ANGLE backend + GPU vendor (NVIDIA/AMD/Intel), not just `os`. The two DE clamp branches are extracted into a pure, unit-tested `resolveDefaultDrawElement` (platform + GPU mode + worker-encode + explicit opt-in), which also drops resolveConfig's cyclomatic complexity. The win32 streaming-encode compound tests collapse onto one shared helper. Linux stays excluded: that fleet is headless/Docker SwiftShader, where DE has no speedup and known rendering defects. Kill switches unchanged: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false, --experimental-fast-capture=false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
3da31e399b |
feat(producer): lower parallel-DE router floor to 700 frames + power-state telemetry
HF_DE_PARALLEL_MIN_FRAMES default 2000 -> 700, re-calibrated by a controlled
crossover sweep (fixed content-per-frame, three synthetic profiles x
{350..3000f} x {single,par2,par3} x 3 reps, resolved worker counts and capture
modes verified per run): par3 beats single at EVERY size in every profile —
+17-21% at 700f rising to +28-34% at 3000f. That includes a
24-sub-composition profile built specifically to reproduce the 'workers
re-pay init' failure the original 2000 floor guarded against (92k tweens,
~2.5s pollSubCompositionTimelines per worker): workers initialize
concurrently, so duplicated init costs CPU, not wall-clock, and the comp
still parallelizes +19% at 700f. Below ~700f the win thins toward +10%
while paying three hardware-GPU browsers, so a floor remains. par2 loses to
par3 in every cell of every profile — the router's existing 3-worker pin is
confirmed, not changed. Harness:
plans/drawelement-fast-capture/de-crossover-bench.sh (docs repo).
Also adds on_battery / low_power_mode to render_complete and render_error.
The DE fleet is macOS laptops, and bench sweeps on an M4 Pro caught the SAME
render flipping between ~9.6 and ~17.2 ms/frame power-management regimes
with no existing telemetry signal to segment by — the router soak reading
this change needs that dimension to interpret perf on the machines users
actually render on. Sampled per event (volatile), pmset-based, darwin-only,
null-safe on failure.
Router stays default-off behind HF_DE_PARALLEL_ROUTER; this tunes what it
will do when the soak clears it to flip.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Miguel Angel Simon Sierra
|
597c14a887 |
fix(telemetry): send feedback as plain events, not PostHog surveys
CLI and Studio feedback were emitted as `survey sent` with `$survey_*` properties, so every rating was ingested as a PostHog survey response even though no survey definition, targeting, or popover backs them. Emit `cli_render_feedback` and `studio_feedback` with plain `rating` / `comment` properties instead. Same fields, same call sites, same opt-out. |
||
|
Vance Ingalls
|
12e599a6ba | fix(engine): realistic worker memory budget + sizing/feedback telemetry | ||
|
Vance Ingalls
|
af923d947c |
fix(producer,cli): round the fallback dB consistently, thread the verify threshold through
Review feedback on #2411 (Rames): the crash-survival RenderCaptureObservability mirror passed deFallbackFailedDb raw/unrounded while the render_complete perfSummary path rounded to 1 decimal — the same underlying PSNR could ship two different values to PostHog depending on which event fired. Extracted the existing inline round/clamp expression (previously duplicated for verifyMinDb and fallbackFailedDb) into a shared roundDb helper, applied once at the single point deFallbackFailedDb is derived from the thrown error so both downstream consumers agree. Also threads verifyThresholdDb (captured on the error but never propagated, per the nit) through DrawElementPerfInput/RenderCaptureObservability/render.ts/ telemetry as de_fallback_threshold_db on both events — the HF_DE_VERIFY_MIN_DB value the failing dB breached, letting ops read "28.4dB failed a 32dB threshold" directly instead of cross-referencing config. |
||
|
Vance Ingalls
|
a5cbb78ff9 |
feat(engine,producer,cli): capture failing dB/frame index on drawElement verify fallback
de_fallback_reason only told you the fallback happened (blank/psnr/oom/ capture_error), not the failing PSNR or frame index — that data existed as text inside the thrown error's message and was discarded on the way to telemetry. DrawElementVerificationError now carries structured frameIndex/failedDb/verifyThresholdDb; the orchestrator reads them via the new getDrawElementVerificationDetails helper instead of regexing message text, and both telemetry surfaces (the render_complete perfSummary path and the crash-survival RenderCaptureObservability mirror) emit de_fallback_failed_db / de_fallback_frame_index. Needed to distinguish "32dB vs the 32dB threshold, tune it" from "12dB real corruption, investigate" during the parallel-router soak — currently that distinction is invisible. |
||
|
Miguel Ángel
|
990f5c3145 |
feat(feedback): adopt 0–10 recommendation scale (#2438)
* feat(feedback): adopt 10-point recommendation scale * docs(feedback): keep OSS scale contract self-contained |
||
|
Vance Ingalls
|
4ca1552e20 |
fix(core,cli): attribute figma REST failures to the endpoint that failed
cli_error had no way to tell which figma REST call (images, files_nodes, variables_local, styles, ...) actually hit RATE_LIMITED/FORBIDDEN/etc, so the dashboard could see failures spike but not which call caused them. FigmaClientError now carries a low-cardinality endpoint label (never the raw fileKey/nodeId), threaded through to cli_error's endpoint property. |
||
|
kiritowoo
|
ccf5f20b3b | fix(cli): stop losing render telemetry to the exit race (#2105) | ||
|
vanceingalls
|
591235f05c | feat(producer): emit eligible_off cohort-sizing signal when parallel-stream router is off | ||
|
vanceingalls
|
88c21ccd8b | feat(cli): capture_parallel_stream telemetry on success and failure paths | ||
|
Vance Ingalls
|
fecd7dc1d3 |
Merge pull request #2248 from heygen-com/bf-reuse-telemetry
feat(producer): surface beginframe no-damage reuse counters in perf summary and telemetry |
||
|
vanceingalls
|
dde3afb72e | feat(producer): surface beginframe no-damage reuse counters in perf summary and telemetry | ||
|
Miguel Ángel
|
87618eef4c |
fix(telemetry): expose stalled render stages (#2220)
* fix(telemetry): expose stalled render stages * fix(telemetry): preserve capture data on terminal stage events * fix(telemetry): fix calibration TDZ crash, tag encode/assemble, extend heartbeat cadence capture_calibration referenced captureStageObservationData before its declaration (later in the same scope), which would throw a ReferenceError for any render hitting the calibration path. Hoist the closure and split workerCount's declaration from its resolution so calibration can safely read it as undefined before capture strategy resolves worker count. Also address the two non-blocking review items: wire encode/assemble stages through captureStageObservationData for consistent tagging, and extend the heartbeat schedule to repeat every 120s after the initial 30/60/120s ramp instead of going dark on stalls beyond two minutes. |
||
|
Vance Ingalls
|
392dd410a5 |
Merge remote-tracking branch 'origin/main' into de-parallel-router-failure-telemetry
# Conflicts: # packages/cli/src/telemetry/config.ts |
||
|
Miguel Angel Simon Sierra
|
cea3458016 |
refactor(cli): single-source-of-truth pass over the check branch
Every duplicated decision gets one owner: rectToBbox lives in checkTypes (was verbatim in pipeline and browser layers); the audit seek tuning is one exported AUDIT_SEEK_OPTIONS consumed by check and the deprecated inspect path; zoom padding/scale defaults export from the capture module instead of re-literalized in three files; the optional run_id property is built by one helper across all three telemetry events; check's --max-transition-samples parsing reuses its own positiveInteger helper; validate drops a leftover re-export and redundant explicit-default args go away. Tests: the contrast candidate round-trip gains a real integration anchor (the actual browser script eval'd in-page, a wrapper asserting finish receives the page-script bbox shape) replacing regex-over-source as the primary guard; the redundant geometry source-golden and a duplicated deprecation-envelope assertion are dropped. |
||
|
Miguel Angel Simon Sierra
|
3a02942a03 |
feat(cli): run-ID telemetry correlation and check breakdown event
HYPERFRAMES_RUN_ID (trimmed, 128-char cap) attaches as run_id to the generic cli_command / cli_command_result events, absent when unset, so an orchestrator setting it per design element can group a verify loop's invocations in analytics. check additionally emits one check_report event per invocation (including lint-short-circuited and failing runs): gate booleans, per-class error/warning counts, launch/seek/contrast phase timings, sample counts, ok and exit code. Timings stay internal; no command output changes. |
||
|
Vance IngallsandClaude Sonnet 5
|
a355fb2f6b |
fix(producer,engine,cli): oom wrapping, cancellation, fallback-reason gaps
Three defects found by max-effort code review of this branch: 1. The Bun OOM exact-match regex was defeated by this codebase's own parallel-worker error wrapping. executeParallelCapture/formatWorkerFailure (parallelCoordinator.ts) always wrap a worker's error as "Worker N: <message>", optionally suffixed and joined with other workers' segments, all prefixed "[Parallel] Capture failed: ". That wrapping defeated the exact-message check for exactly the cohort (deParallelRouter routed, N separate Chrome processes) the OOM-drops-to-1 fix targets — a real OOM there would retry at the SAME worker count instead of dropping to 1. Added a second pattern that recovers the signal by requiring "out of memory" appear as the WHOLE content of a "Worker N: ..." segment (bounded by end-of-string/"; "), preserving the same exact-match property (no bare substring match) while surviving the wrapping. Verified against the real wrapping logic, not a hand-typed guess at its shape. 2. shouldRetryViaPinnedFallback didn't exclude cancellation, so aborting a render mid-capture on the pinned router/inversion cohort would detour through spawning a fresh encoder/capture session before the outer catch's RenderCancelledError branch ended the render — delaying "stop" with a pointless resource spin-up/tear-down. Added an isCancellation param (checked first, before isVerifyError) using the same `err instanceof RenderCancelledError || abortSignal?.aborted` check the outer catch already uses. 3. deFallbackReason (this PR's new "oom"/"capture_error" values) was set locally but never mirrored into RenderCaptureObservability alongside deSelfVerifyFallback, so a render that fails AFTER a fallback attempt (perfSummary never built) was indistinguishable in render_error telemetry from one that never attempted any fallback — undercutting the "how often does the OOM retry fire on a render that still ultimately fails" question this branch exists to answer. Threaded through RenderCaptureObservability → RenderObservabilityTelemetryPayload → renderObservabilityTelemetryPayload, mirroring the existing deSelfVerifyFallback plumbing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
ec921e143b |
feat(producer,cli): full telemetry visibility for DE parallel-router/inversion failures
render_error previously carried zero DE-cohort context — a hard failure while routed (worker crash, OOM, capture timeout from the fixed 3-worker pin overriding calibration) was indistinguishable from any other failure. The data existed (RenderCaptureObservability is mutated live and survives into job.errorDetails on the failure path) but was never projected into the render_error payload, which only ever drew de_* fields from perfSummary (success-only). - RenderCaptureObservability now also records dePreInversionWorkers / dePreRouterWorkers — the worker count calibration would have picked absent the experiment — so a resource-pressure failure can be correlated with the router overriding a lower calibrated count. - New capture-sourced de_* fields on RenderObservabilityTelemetryPayload, shared by trackRenderComplete and trackRenderError. Explicit perfSummary-sourced fields still win on render_complete (spread moved first in the event object) — this is purely a failure-path fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
0393ba5be2 |
feat(producer): default-off router for verified parallel drawElement
Promotes the opt-in HF_DE_PARALLEL_STREAM mechanism (#2056) into the auto-routing decision, gated behind its own default-off flag (HF_DE_PARALLEL_ROUTER). This is the next step from the 2026-07-08 parallel-DE benchmark verdict: par3/single 1.16-1.36x on real-work comps >=2,000 frames, no comp anywhere losing to single-worker. shouldPreferParallelDrawElement mirrors shouldPreferSingleWorkerDrawElement (#2026) but takes priority over it when both are eligible — its higher default threshold (HF_DE_PARALLEL_MIN_FRAMES=2000 vs the inversion's 900) means it only ever picks up the long tail the inversion's own benchmark didn't cover. Fixed at 3 workers (benchmark-validated; not calibration- derived), same shape as the inversion pinning to a fixed 1. resolveParallelRouterRetryPlan mirrors resolveInversionRetryPlan for the self-verify-failure rollback path: falls back to the ordinary (non-DE) parallel-disk path at the pre-router worker count. The caller must clear HF_DE_PARALLEL_STREAM before recomputing useStreamingEncode or the retry would keep resolving to the parallel-streaming shape. New telemetry (de_parallel_router, de_pre_router_workers) tags which render used the router, separate from de_worker_inversion — needed before the planned telemetry soak can segment revert-rate and de_verify_min_db to the parallel cohort specifically; today there's no way to tell those apart from ordinary single-worker DE renders. Verified end-to-end: HF_DE_PARALLEL_ROUTER=true on a 2,381-frame comp resolves to 3 workers with 3 separate drawElement sessions and renders successfully; without the flag, behavior is unchanged (falls through to the existing single-worker inversion, workerCount=1) — no regression to current production routing. 114 orchestrator tests pass (15 new). |
||
|
Miguel ÁngelandClaude Opus 4.8
|
57b3c78987 |
feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare + compare (#2041)
* feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare CLI Add color grading to media-use as first-class resolve types plus a faithful comparison command. All local, offline, deterministic — no model, no GPU. - resolve -t grade / -t lut: produce a data-color-grading block (or a frozen .cube). Look cascade: core preset (no file) -> bundled .cube library -> parametric buildCube. Emitted .cube is Rec.709 and validated against core's colorLuts constraints (LUT_3D_SIZE <= 64) before it is frozen. - smart grade (grade --for <media>): ffmpeg signalstats -> adjust suggestion (exposure / contrast / white balance), surfaced with the measured evidence on stderr as a starting point; never auto-applied. - hyperframes grade-compare: renders N candidate grades onto a reference frame through the real runtime shader into one labeled comparison PNG, so an agent picks a look without opening Studio. Prepends an "original" baseline cell by default (--no-baseline to omit). Shares the headless-capture pipeline with snapshot via capture/captureCompositionFrame. - media-use SKILL: proactive "media opportunity pass" guidance (grounded signal -> offer, ask once, surface don't mutate). Verified: media-use 116/116, grade-compare 7/7, snapshot 9/9, lint + format clean, full build green, comparison renders end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * test(cli): narrow grade-compare baseline assertion off unknown-typed grading Assert the whole cell via toEqual instead of reaching into .grading.preset / .grading.lut on the unknown-typed field, keeping the test typecheck-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): agent-authored LUTs via --params + validate --from cube; never-read-.cube guardrail - resolve -t lut / -t grade --params '<json>': build a parametric .cube from explicit params (bypassing the intent cascade), validate, and freeze in one step. --intent becomes the optional description. Lets an agent commit a look it computed itself. - --from <file.cube> now validates the ingested LUT for lut/grade types and rejects an invalid/oversized cube (no partial write) — the escape hatch for a LUT the agent generated with its own code. - SKILL.md: hard rule to never read a .cube body into context (~size^3 lines, zero legible signal) — inspect via grade-compare (see it) or cube-validate (ok/size), read the manifest description for meaning; plus both authoring paths and the parametric-vs-film-stock ceiling note. Verified: media-use 116/116, lint + format clean; smokes — --params builds a valid frozen cube, grade --params returns a lut block, bad JSON and an oversized --from cube are both rejected with no stray file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): grade-compare validates referenced LUTs, warns on no-op cells, caps candidates Bug-bash follow-ups — grade-compare silently accepted bad input: - Validate LUT *content*, not just existence: each referenced .cube is parsed with core's parseCubeLut (now exported from @hyperframes/core) and rejected with a per-cell error ("LUT for \"<label>\" is not a valid .cube: ..."). A file that exists but isn't a valid cube no longer renders a silent no-op cell. - Warn on inactive cells: a grading that normalizes to inactive (e.g. a malformed {lut:12345}) emits a stderr warning naming the cell; the auto-prepended "original" baseline is intentionally inactive and stays silent. stdout remains valid JSON. - Cap candidates at 16 (excluding baseline): over-cap input renders the first N and reports {truncated:true, total:M} on stdout + a stderr note — no silent drop, no unbounded giant sheet. Verified: grade-compare 10/10; non-cube LUT → clear error; {lut:12345} → warning + ok; 20 cells → cells=17 truncated total=20; valid runs unchanged. Lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(cli): general `hyperframes compare` visual-variant primitive Generalize grade-compare's "render N variants → one labeled sheet → the agent looks and picks" loop into a standalone command that works on ANY variation (font, layout, motion, grade, whole compositions) — the tool never needs to know what differs. - `hyperframes compare <path...> [--at <sec>] [--labels a,b,c] [--out] [--cols] [--json]`: renders each agent-authored composition variant through the real runtime (captureCompositionFrame) and stitches one labeled comparison sheet + JSON ({ok, sheet, rendered, variants, truncated?/total?}). 2+ paths required; caps at 16 with loud truncation. It presents, it does not judge — choosing is the caller's job. - Factored the shared "render a labeled set → contact sheet" path so compare, grade-compare, and snapshot all sit on it (no duplication). grade-compare is now the first color-specific specialization of this primitive. - New pathArgs util + contactSheet test; hyperframes-cli SKILL documents compare as the agent's "see your own renders and choose" primitive. Verified: 26/26 across compare + grade-compare + snapshot + contactSheet (no regressions); compare renders 3 variants into one visibly-distinct labeled sheet; 2+-path error path clean; lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): green the skills CI — skip ffmpeg tests when absent, oxfmt markdown The "Test: skills" CI job runs bare `node --test` with no ffmpeg on PATH (by design — skills tests are meant to be node-builtin-only). The grade-analyzer + smart-grade tests shell to ffmpeg and were failing there with ENOENT. Guard them to skip when ffmpeg isn't on PATH; they still run locally / where it is. Also oxfmt README.md + hyperframes/media-use SKILL.md (the whole-repo `oxfmt --check .` Format job caught markdown left unformatted by the rebase conflict resolution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): skip core-conformance test when tsx is unavailable The "Test: skills" CI job installs no deps, so the normalizeHfColorGrading conformance test (which imports core's TS via `node --import tsx`) failed there. Guard it to skip when tsx can't resolve; runs locally / in the deps-installed Test job. Completes the skills-CI greening (the ffmpeg guards handled the rest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): escape grade-compare src double-quotes (CodeQL XSS) + Windows-safe compare test - grade-compare built `<img src="...">` (double-quoted) with the single-quote escaper, leaving `"` unescaped — a `"` in the frame path could break out (CodeQL: incomplete HTML attribute sanitization). Use escapeXml for src. - compare label test hard-coded POSIX paths that can't match on Windows; assert the derived labels (the subject); path resolution is covered elsewhere. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * refactor(media-use): generate LUT library from params (drop committed .cube files) The 3 bundled .cube files were 733 lines each (2,199 total) and were themselves buildCube output — pure repo bloat. Replace with compact per-look params in luts/index.json, generated on resolve; add an optional `url` for future scanned LUTs to be CDN-hosted + downloaded on demand (freezeUrl) instead of committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): serve library LUTs from CDN on-demand (static.heygen.ai/luts), params fallback Looks now carry a CDN `url` (hosted at s3://heygen-public/luts → static.heygen.ai/luts/<id>.cube); resolve downloads + validates + freezes on demand, like bgm/image. `params` stays as the deterministic offline fallback (--local-only, or if the download fails), so resolution is never blocked on the network. Provider prefers url, falls back to params. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(media-use): address #2041 review — atomic LUT writes, compare telemetry, follow-ups - Atomic .cube writes: library provider (url + params) and the parametric generator now write to a .tmp path, validate, then rename, so a crash can never orphan an invalid .cube at the final path (was validate-after-write). - track("media_use_resolve") now emits provenance.via (url/params-fallback/params). - grade-compare + compare: --timeout flag (was hardcoded 5000) and a media_use_compare event (cells, truncated, total, render_ready_timed_out); openSettledCompositionPage now surfaces the render-ready timeout. - compare staging skips node_modules/.git; --for gets an upfront existence check. - Rec.709 luma comment; HYPERFRAMES_ANALYZE_TIMEOUT_MS override; measured note uses basename; LUT s3 hosting moved from index.json into luts/README.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
8ba3c33915 |
fix(engine,producer,cli): close review gaps in sub-timeline fail-fast
Address PR #2045 review feedback: - Share a SubTimelineWaitOutcome type (engine) end-to-end instead of widening to string across CapturePerfSummary / RenderPerfSummary / telemetry, so the three layers can't drift. - Dedupe scriptLoadFailures on push — a 4xx response and its trailing requestfailed both recorded the same URL, doubling the failed-URL list in the fail-fast warning. - Thread the sub-timeline-wait outcome into render_error (not just render_complete): a render that fail-fasts and then fails downstream (pollVideosReady, extract, encode) previously dropped this signal on the floor. dedupPerfs is now function-scoped so the catch path can read it, same treatment as the existing captureAttempts array. |
||
|
Vance IngallsandClaude Fable 5
|
54359f3d6a |
fix(engine): fail-fast the sub-composition timeline wait when a script 404s
pollSubCompositionTimelines waits for every [data-composition-id] host to
register window.__timelines[id]. When the script carrying that registration
fails to load (404 / request failure), the registration can never arrive —
but the poll still burned the full playerReadyTimeout (45s), then warned and
shipped a silently animation-less render. Wild scale: the capture-setup
histogram over 30 days of local renders decays smoothly (402/503/364/282/191
per 5s bucket) then spikes to 705 at the 45s bucket — ~1,000 renders/month
across 402 distinct users, ~15 user-hours of pure waiting.
- Sessions now record failed SCRIPT resources (requestfailed + HTTP>=400
response, listeners that already existed for diagnostics) in
session.scriptLoadFailures.
- pollSubCompositionTimelines takes a failure getter and cuts the wait to a
2s grace once any script failed, with a loud warning naming the URL(s).
Late-registering fetch-async comps are unaffected: no script failure means
the full timeout still applies, and a registration landing inside the
grace window still wins (tested).
- Outcome telemetry: session.subTimelineWaitOutcome ("ready" | "timeout" |
"script_failure") -> CapturePerfSummary -> RenderPerfSummary.subTimelineWait
(worst across sessions) -> render_complete sub_timeline_wait, so the wild
rate becomes directly trackable instead of setup-histogram forensics.
Validation: the discovery comp (0768f038, its animations.js unreachable)
drops from ~72s to 23.1s total — poll cut at 2.1s with the script named;
healthy comp reports "ready". Canary suite 7/7 (PSNRs identical). 4 new
poll unit tests; engine suite 907 passed (14 failures are PRE-EXISTING on
main at v0.7.42 — 18 fail on a clean checkout, stash A/B verified).
tsc/oxlint/oxfmt clean.
Corpus note: 258/1,762 corpus comps (14%) reference local scripts missing
from the corpus fetch — their historical eval INIT timings measured this
timeout, not the engine. Capture-stage ratios remain valid (both paths paid
it equally).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Fable 5
|
924727a0b4 |
feat(producer,cli): drawElement priority inversion — single-worker streaming over auto-parallel (#2026)
* feat(producer,cli): drawElement priority inversion — single-worker streaming over auto-parallel clamp:parallel eats 50% of local renders (1,326/fortnight; DE engagement stuck at 3.8%) by routing multi-worker renders to unverified screenshot capture. Benchmarks (2026-07-08, 4 comps x W1/W2/W3/W5) show that above the ~900-frame amortization crossover, single-worker VERIFIED drawElement streaming beats screenshot-parallel at EVERY worker count (2,380f: 66s vs 109-127s; 3,600f: 33s vs 39-56s; parallel scaling flattens past W2), while below it DE's fixed init cost loses by <=2.2s. - shouldPreferSingleWorkerDrawElement (exported predicate + 7 unit tests): inverts an AUTO-resolved multi-worker render to workerCount=1 when the comp matches the benchmarked configuration — default-on DE (darwin hardware clamp upstream), no compile gate, no forced-screenshot hint, mp4 output, single-worker streaming eligible, and totalFrames >= HF_DE_SINGLE_MIN_FRAMES (default 900; 0 disables). Explicit --workers N is always honored. - Inverted renders keep the probe session and land on the worker-encode streaming drain — the ONLY path with runtime self-verification, so this moves ~40% of previously-clamped renders onto the verified fast path. Comps that later hit an init-time gate (~1.5% of local renders) render single-worker screenshot streaming; accepted trade. - Telemetry: de_worker_inversion on render_complete (orchestrator -> perfSummary.workerInversion -> CLI), plus the worker_resolution observability checkpoint now records deWorkerInversion. Validation: e2e matrix on 2,381f comp — auto->5 workers inverted to 1, DE verified 4x inf PSNR, RENDER_OK; short comp (360f) auto stays 5-worker; explicit WORKERS=3 honored; HF_DE_SINGLE_MIN_FRAMES=0 disables. Canary suite 7/7 (PSNRs identical). renderOrchestrator tests 86/86. tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(producer): review fixes — inversion routing guards, calibration skip, retry revert Max code-review round on the inversion (13 confirmed findings): - Streaming spawn-failure disk fallback now clamps default-on drawElement (deClampReason=disk_path, DE-mode probe closed) exactly like the pre-capture clamp — previously it carried useDrawElement=true onto the unverified disk path, the hole the verified-path confinement exists to close, newly reachable for every inverted render. - Predicate gained the routing knowledge it was blind to: layered/HDR and shader-transition comps (drawElement never runs there), supersampling (deviceScaleFactor>1 init gate), a probe session whose init gates already disengaged DE, and the PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true explicit parallel-DE opt-in (honored like --workers N). - Eligibility is evaluated BEFORE capture calibration and skips it when the inversion pins workers to 1 regardless of the estimate — the throwaway calibration browser + sample captures cost ~41s on the 2,381-frame benchmark comp (auto render: 111.6s -> 70.1s total). - Self-verify retry reverts the inversion: the re-render returns to the pre-inversion parallel screenshot path (disk) instead of single-worker screenshot streaming, the slowest shape for exactly the comps drawElement damages. - HF_DE_SINGLE_MIN_FRAMES="" (set-but-empty) now falls back to the 900 default instead of aliasing the 0 kill switch. - Timeout advisory uses the RESOLVED worker count — an inverted render that times out no longer prints "Retry with --workers 1" (the configuration that just failed). - Telemetry: deWorkerInversion recorded in capture observability (failed renders are attributable), emitted as literal false when not fired (queryable denominator), and the drawElement perf input shape is one exported DrawElementPerfInput type instead of three copies. - Tests: requestedWorkers undefined (the value production actually passes) + the four new predicate guards; 91/91. Validation: e2e auto render — calibration skipped (deInversionEligible), inversion fires, DE verified 4x inf, total 70.1s (was 111.6s); HF_DE_SINGLE_MIN_FRAMES=0 restores calibration + parallel; canary suite 7/7 (PSNRs identical); tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(producer,cli): review round 2 — loss-cohort telemetry, retry-plan helper, boundary tests - de_worker_inversion is now a tri-state string ("inverted" | "reverted" | "none") instead of a boolean: the self-verify retry marks the render "reverted" rather than resetting to false, so the dashboard can segment the lost-inversion cohort first-class instead of inferring it from deSelfVerifyFallback + frame-count joins (james-russo #1). - The retry rollback is extracted to resolveInversionRetryPlan (pure, exported) with unit coverage: pre-inversion worker-count restore, streaming re-resolution (multi-worker retry -> disk), "reverted" state, null when never inverted (james-russo #2). - WOULD_RESOLVE_MULTI_WORKER named constant replaces the bare sentinel 2 (james-russo #5); minFrames: -1 boundary case added (miga #3). 94/94 renderOrchestrator tests; tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(producer,cli): emit de_pre_inversion_workers for the parallel counterfactual The ramp-down decision needs "did DE beat the parallel render it displaced", not just "did DE beat single-worker screenshot". Emitting the worker count the auto-resolution chose BEFORE the inversion pinned it to 1 makes the parallel counterfactual computable per render (screenshot ms/frame from the verify samples / W x the measured parallel-efficiency curve). Set only when the inversion fired. Smoke: 2,381f auto render -> de_worker_inversion="inverted", de_pre_inversion_workers=5, mode=drawelement, verify armed 4. 99/99 tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Fable 5
|
4c8064d4d3 |
fix(cli): figma component import survives unrenderable nodes (#2022)
Live testing against a real community file (Ratings) found a nested instance node figma refuses to render as svg — which aborted the entire component import. The rasterize loop now retries the node as png, and only if both formats fail warns and skips THAT node (placeholder keeps its data-figma-rasterize marker, no src) instead of failing the import. On the file that surfaced this, the png retry recovers the node — 31/31 placeholders get assets. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
3d8372f880 |
feat(cli): associate signed-in HeyGen account with telemetry (#2020)
* feat(cli): associate signed-in HeyGen account with telemetry Sign-in telemetry currently attributes everything to the anonymous install id, so the sign-in funnel can be counted but a completed sign-in can't be tied to the account it produced. This associates the two. - On a completed sign-in, emit a PostHog `$identify` alias whose `$anon_distinct_id` is the install's anonymousId, so events recorded before sign-in stitch to the same person, and tag `auth_login_completed` with the account identity (the pre-plumbed `distinctId`). - `/v3/users/me` exposes no opaque user_id, so the identity key is the account email, falling back to username (single `identityKey` helper). - Both no-op under the `telemetry disable` opt-out and only fire after the user chooses to sign in. Privacy disclosure updated in lockstep, since this is the first PII the CLI attaches: the first-run telemetry notice and the telemetry section of docs/packages/cli.mdx now state that signing in links your account email to your usage. Tests: identifyUser payload + no-op, completion attribution incl. username fallback and no-identity-on-reject/empty. Verified end-to-end against the built CLI: pre-auth events anonymous, $identify carries $anon_distinct_id, completion carries the account email. * docs(cli): disclose the username identity fallback Review gating item: identityKey is `email ?? username`, but the first-run notice and cli.mdx said only "email", so an emailless account's username would reach PostHog undisclosed. `/v3/users/me` treats email as optional (pickString), so the fallback is live code, not dead — disclose it rather than assert an unverifiable email guarantee. Both surfaces now say "email, or username if the account has no email". Also soften the identityKey comment: it implied username is "less identifying", but HeyGen usernames are often email-shaped, so the note now states username is a fallback, not a privacy win. |
||
|
Vance IngallsandClaude Fable 5
|
b26c27576b |
feat(engine,producer,cli): verify video comps via deferred DE init + capture p50 (#2015)
* feat(engine,producer,cli): verify video comps via deferred DE init + capture p50 Closes the two biggest gaps in the first day of v0.7.38 wild data: 88% of drawElement renders (video comps initialized via probe sessions) ran with self-verification unarmed, and speedup was measurable on only 3 of 76 renders. - Deferred drawElement init: probe sessions initialize before video extraction, so they have no frame injector — ground-truth screenshots would capture black <video> boxes, and verification skipped the whole comp. DE init now stops after the gates for injector-less video comps (deInitDeferred; autoAlpha flag retracted in case no path completes it) and completeDeferredDrawElementInit finishes verification + canvas injection + worker-encode at capture time, once prepareCaptureSessionForReuse has attached the injector. Validated end-to-end: a probe-path video comp now arms 4 ground-truth frames with real video pixels (3x inf + 64.7dB) and renders drawElement verified. - capture_p50_ms: per-frame capture durations are sampled (capturePerf.frameMs; batch frames get the batch mean) and the median ships as CapturePerfSummary.p50TotalMs -> RenderPerfSummary.captureP50Ms -> render_complete capture_p50_ms. Unlike capture_avg_ms it is immune to first-frame warmup and stage-setup amortization — smoke: avg 15ms vs p50 8ms on the same render, p50 matching the measured steady-state floor. Dashboard speedup tiles can drop their frame-count floor once this ships. - video_count on render_complete: segments speedup by video-injection comps (whose per-frame gain is legitimately lower) vs pure-graphics. Canary suite 7/7; engine suite 905 passed (1 pre-existing upstream failure); tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(producer): complete deferred drawElement init on the disk capture path Review (miga): a probe-initialized video comp falling back to the disk path kept deInitDeferred and silently stayed in screenshot mode — a regression for PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true renders that previously ran drawElement there. Complete the deferred init on the sequential disk path under the same explicit-opt-in test the orchestrator clamp uses; default-on renders stay on the screenshot baseline (this path has no drain-time self-verification, per the #1998 confinement rule). Validated: video comp + PRODUCER_ENABLE_STREAMING_ENCODE=false + explicit opt-in logs "(deferred drawElement init)" completion on capture_disk and renders correct video pixels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Fable 5
|
1005703441 |
feat(engine,producer,cli): drawElement release telemetry on render_complete (#2002)
Default-on drawElement ships with a runtime self-verification net (#1998); this makes its in-the-wild behavior observable. Every render_complete event now answers: which capture mode actually ran, why drawElement disengaged when it did (compile gate / producer clamp / engine init gate), whether the self-verify net fired and why, and how much margin verification had. Follows the static-dedup telemetry pattern: engine session fields → CapturePerfSummary → RenderPerfSummary.drawElement → snake_case props on render_complete. New event props: de_capture_mode, de_compile_gate, de_clamp_reason, de_gate_reason, de_worker_encode, de_verify_armed, de_verify_checked, de_verify_min_db (margin above the 32dB threshold — drift here is the early-warning signal before fallbacks spike), de_verify_init_ms, de_self_verify_fallback, de_fallback_reason, de_blank_suspects, de_blank_deterministic_accepts, de_blank_recaptures, de_boundary_frames, de_ncpr_fallbacks. Validated end-to-end on live renders: drawelement path reports mode/verify counters/minDb/init cost; a blur-gated comp reports mode=screenshot + gate_reason=css_effect:filter; a forced verification failure reports self_verify_fallback=true + fallback_reason=psnr. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
769dc702c1 |
feat(cli): emit sign-in lifecycle telemetry (#2000)
* feat(cli): emit sign-in lifecycle telemetry The CLI tracks command and render lifecycles but emits nothing for `auth login`, so sign-in outcomes are invisible on the observability dashboards — a completed sign-in, an abandoned browser flow, and a rejected key all look identical (absent). This leaves a blind spot in the same funnel the render events already cover. Add three events mirroring the existing `trackX` pattern: - auth_login_started (method: oauth | api_key) - auth_login_completed (method) - auth_login_failed (method, reason) `reason` is a fixed low-cardinality enum (flow_error / no_credential / rejected / invalid_input). No token, key, identity, email, or free text is ever attached — consistent with the existing anonymous telemetry and the `telemetry disable` opt-out. Wired into both the OAuth and --api-key paths in `auth login`, with unit coverage for the new events. * fix(cli): close sign-in telemetry funnel dropout gaps Follow-up so `started` reconciles to `completed + failed` on the common abandonment paths, which the first cut missed: - Interactive prompt cancel (Ctrl-C) now surfaces as a throw that the single catch in the api-key path records as `aborted`, instead of a bare exit with no event. - A stdin read that times out in non-TTY `--api-key` mode now records `aborted` before the error propagates, rather than exiting silently. - OAuth split: a timed-out browser callback (user closed the tab) is tagged `flow_timeout`, separated from real `flow_error` (IdP/network), since the walk-away timeout is the dominant non-error dropout. Also pre-plumb an optional `distinctId` on the three trackers, mirroring trackRenderComplete/trackRenderError. Unused today; it lets a later identity-level attribution be a one-line callsite change rather than a signature sweep. Coverage added for the new reasons and forwarding. |
||
|
Vance IngallsandClaude Fable 5
|
e9076324e7 |
feat(cli): figma import telemetry — subcommand labels, typed error codes, figma_import event (#1979)
Closes the observability gaps on the figma integration: - withFigmaErrors takes a command label (figma:asset|tokens|component) and reports the failure inline before its process.exit — the top-level trackCommandFailures wrapper never sees self-exiting commands, so typed codes (NO_TOKEN, BAD_TOKEN, FORBIDDEN, RATE_LIMITED) were invisible. FigmaClientError codes surface as the error name for dashboarding the first-run funnel (NO_TOKEN -> later success = onboarding conversion). - new figma_import event per import: phase, duration, reused (dedup effectiveness), tokens variables-vs-styles mode + entry count (Enterprise gating rate), unresolved-binding + rasterized-node counts (fidelity degradation). No fileKeys, node ids, names, or descriptions. - /figma skill fires the events beacon (figma-motion / figma-shaders / figma-storyboard) for the MCP phases that never touch the CLI. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
James RussoandClaude Opus 4.8
|
9b41891f3a |
feat(cli): emit render_preflight_rejected telemetry for P1-3 pre-flight saves (#1856)
The P1-3 aspect/alpha/HDR pre-flight (#1843) aborts an incompatible render before any browser/ffmpeg work, but that "save" was invisible on dashboard 1783183 — indistinguishable from a deep failure or a user giving up. checkRenderResolutionPreflight now returns { message, kind } (kind = the existing low-cardinality OutputResolutionIssueKind), and the render command emits render_preflight_rejected { kind } before exiting. No parsers change — the helper already carried kind. trackRenderPreflightRejected is typed to the union so the metric can't carry free text. Tests: preflight tests assert kind for all five kinds; an events test locks the emit. Further follow-up (still log-only): encoder-frame-0-exit counter and a P1-4 doctor cli_env_check event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
James RussoandClaude Opus 4.8
|
733f88cb1f |
feat(producer,cli): render-reliability telemetry counters for capture hardening (#1850)
Follow-up to the render-reliability batch (#1841/#1842/#1843). Threads two capture-reliability counters through the existing observability → CLI-telemetry pipeline (no new PostHog wiring) so #1842's hardening is measurable on dashboard 1783183: - transient-retry burn (CaptureAttemptSummary.reason gains "transient-retry"; counted into RenderCaptureObservability.transientRetries on BOTH the recovered and the still-failed paths via a shared helper). - OOM classification (memoryExhaustionDetected set when describeMemoryExhaustion classifies the failure). Surfaced as capture_transient_retries + capture_memory_exhaustion_detected render-event props. Tests cover the attempt tagging and the payload mapping. Further follow-up (different subsystems): encoder-frame-0-exit signal, and P1-3 pre-flight-rejection / P1-4 cli_env_check counters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
23adfdc496 |
fix(cli): skip AI skills install when git is unavailable (#1803)
* fix(cli): skip AI skills install when git is unavailable init and `skills update` route through installAllSkills, which shells out to `npx skills add`. That CLI clones the repo with git, so on a machine without git the clone aborts mid-run and dumps a noisy multi-line `spawn git ENOENT` / "Installation failed" / "Canceled" block. init still exited 0 and scaffolded the project, but the output read like a hard failure (and surfaced as exit 1 on some platforms). Detect git up front alongside the existing npx check via a small table-driven preflight: best-effort callers (init) print one calm line and continue; strict callers (`skills update`) throw so the check-or-update recovery contract still fails loudly. The skills freshness check already degrades gracefully without git, so the happy path is unchanged. * feat(cli): record a diagnostic event when a skills install is skipped for a missing prerequisite When init's best-effort skills install bails because git (or npx) is absent from PATH, the skip was silent, so the rare boxes that hit it (fresh Windows without git) were invisible. Emit one low-cardinality event (reason: git_missing / npx_missing) on the best-effort skip path only, never on the happy path or the strict throw. Reuses the existing typed-event pattern, and trackEvent's opt-out gate already applies. |
||
|
Miguel Ángel
|
db61509ddc |
fix(cli): omit render duration when feedback command has none (#1797)
The standalone `feedback` command runs separately from `render`, so it has no access to the prior render's elapsed time, yet it always passed renderDurationMs: 0 to the feedback analytics event. Since that path is the one used in practice (the auto-prompt returns early for agent and non-interactive runtimes), nearly every feedback event recorded a render duration of exactly 0, which is misleading rather than absent. Make renderDurationMs optional and only include render_duration_ms in the event when a real value is supplied. The standalone command no longer passes a duration; the auto-prompt path still forwards the real elapsed time. |
||
|
kiritowooandkiritowoo
|
5242dde2dc |
feat(telemetry): attribute renders to the authoring workflow skill (#1695)
* feat(telemetry): attribute renders to the authoring workflow skill Add an optional `--skill` flag to `hyperframes render` and tag the `render_complete` / `render_error` events with `authoring_skill`, so render usage can be broken down per authoring workflow. The value is slug-gated (a malformed value is ignored) and the existing anonymous / opt-out telemetry pipeline is otherwise unchanged. Each end-user workflow that renders now passes `--skill=<name>` on its render command: embedded-captions, faceless-explainer, graphic-overlays, motion-graphics, music-to-video, pr-to-video, product-launch-video, remotion-to-hyperframes, website-to-video. Not instrumented, by design: general-video renders freeform with no canonical render command to attach to, and slideshow produces an interactive deck rather than a rendered video. Both can follow up if per-skill numbers are wanted. * fix(telemetry): address review — shared slug util, equals-form flag, invalid-value warning - Extract the SKILL_SLUG regex + a normalizeSkillSlug() helper into telemetry/skill.ts, shared by the `events` and `render` commands (the regex was duplicated). `render` adopts normalizeSkillSlug (so it now trims the value, matching `events`); `events` references the shared SKILL_SLUG. + unit test. - `render` warns on a non-empty but invalid --skill value (e.g. a camelCase typo) so attribution isn't silently lost — stderr only, never fails the render. - embedded-captions render script: `--skill embedded-captions` -> `--skill=embedded-captions`. On an older CLI that does not declare --skill, the space form leaks the value as a positional and clobbers the project dir (resolveProject fails); the equals form is parsed as a self-delimiting flag and safely ignored. Verified via Node parseArgs(strict:false). Addresses review feedback on the PR (shared util + .trim drift, version-skew safety, invalid-value visibility). --------- Co-authored-by: kiritowoo <295860553+kiritowoo@users.noreply.github.com> |
||
|
Miguel Ángel
|
4976b5e036 | fix(producer): split capture average timing (#1657) | ||
|
Miguel Ángel
|
f12754f5ad |
fix(cli): classify missing whisper-cpp as a setup gap, not a command error (#1628)
transcribe hard-failed with cli_error whenever whisper-cpp was absent. On Linux/Docker/CI (no Homebrew, no compiler toolchain) that is unavoidable, so it drove ~30k cli_error/day that are really "install the prerequisite" rather than bugs — and buried genuine transcription failures in the command-error budget. ensureWhisper now throws a typed WhisperUnavailableError when no binary exists and none can be built. The transcribe command reports that on a dedicated transcribe_unavailable metric instead of cli_error, and a new --optional flag lets pipelines skip captions and exit 0. Real transcription crashes still fail as cli_error. init and the skill pipelines already continue without captions. Also removes a stale doc reference to a `transcribe --provider groq` flag that does not exist. |
||
|
Vance IngallsandClaude Opus 4.8
|
7310223b66 |
feat(engine): static-frame dedup default-on + render telemetry (#1549)
* feat(engine): static-frame dedup for screenshot capture (opt-in) Skip re-seeking + re-screenshotting frames byte-identical to their predecessor. A frame is dedupable iff no GSAP tween or clip cut is active in it or its predecessor (predicted from window.__timelines + clip schedule) AND an empirical anchor-compare confirms it. Opt-in HF_STATIC_DEDUP=true, default off. Correctness (designed for the multi-worker / distributed render paths): - Reuse is keyed by the ABSOLUTE composition frame (derived from the frame's time), NOT the captureFrameCore frameIndex arg — chunked/parallel callers pass a chunk- relative index. Validated lossless (PSNR=inf) on both single- and multi-worker renders of a static-hold comp. - verifyStaticFramesSafe checks EVERY run (no longest-first budget truncation that left runs armed-but-unverified), and samples each run's FIRST reused frame, its END, and interior points at a stride; a hard cap disables dedup rather than trust an unverified set. - Conservative arming: skipped when capture mode != screenshot (BeginFrame tick semantics + the verifier's screenshot path wouldn't transfer), when a before-capture hook is set (per-frame video injection), when page-side compositing is active (shader / drawElement composite the plain verification screenshot can't reproduce), and when any data-start is a non-numeric reference expression the clip-boundary parser can't protect, or duration is unknown/zero. - Session reuse (prepareCaptureSessionForReuse) resets lastFrameBuffer + dedup counter so a probe/prior-render buffer can't bleed into the first static frame; the armed set is kept (same-composition reuse). Cost calibration bypasses dedup for its sparse, non-contiguous sample sweep, then restores the armed set. - HF_STATIC_DEDUP_SAMPLES is NaN-guarded. Disqualifies on signals the GSAP predictor can't see: video, canvas/webgl, zero tweens, running CSS/WAAPI animation. Pays on static-hold content (title cards, slideshow/kiosk loops, data-viz pauses); no-op on continuously-animated comps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(engine): static-frame dedup default-on + render telemetry Flip dedup from opt-in (HF_STATIC_DEDUP=true) to default-on (opt-out HF_STATIC_DEDUP=false). Verification (verifyStaticFramesSafe) is the safety net that keeps reuse sound at scale. Add end-to-end dedup observability. The capture session records enabled / armed / skipReason / predicted; these surface via CapturePerfSummary -> a dedupPerfs accumulator (disk sequential + parallel AND streaming sequential + parallel) -> aggregated into RenderPerfSummary.staticDedup (OR armed, SUM frames across workers) -> render_complete props static_dedup_{enabled,armed,skip_reason, predicted_frames,reused_frames}. skip_reason is a low-cardinality code: capture_mode | video_injection | page_composite | ineligible | verification_failed. Distributed chunks run on Linux/beginframe where dedup never arms, so they pass a throwaway dedupPerfs sink (no per-chunk reporting). Tests: aggregation logic (OR/SUM/skip-reason) + opt-out passthrough. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(engine): address review on dedup default-on + telemetry Review feedback (miga-heygen) + self-review fixes: - Retry double-count: executeDiskCaptureWithAdaptiveRetry pushed worker dedup perf inside the retry loop, so an adaptive retry counted frames twice (reused/predicted could exceed totalFrames). Reset dedupPerfs at the start of each attempt — retry now REPLACES rather than accumulates; common no-retry path is unchanged. - Opt-out parsing: HF_STATIC_DEDUP now disables on {false,0,off} case/space-insensitive (was strict !== "false", so `False`/`0` silently kept dedup on — the kill-switch could no-op). - Verification budget vs drift: verifyStaticFramesSafe returns {badFrame, budgetExhausted}; armStaticDedup reports a distinct `verification_budget` skip reason so a telemetry spike means "raise HF_STATIC_DEDUP_SAMPLES", not "compositions are non-static". - Index idiom: captureFrameCore now uses Math.floor(time*fps + 1e-9) (matches quantizeTimeToFrame) so the dedup lookup agrees with the frame the seek lands on even for non-exact times. - Stale "opt-in HF_STATIC_DEDUP=true" comments -> "opt-out HF_STATIC_DEDUP=false" across frameCapture.ts + types.ts. - Extract pushWorkerDedupPerfs helper (perfSummary.ts), used by the disk and streaming parallel paths — removes the duplicated push loop and drops captureStreamingStage back under the complexity threshold. - dedupPerfs is now required (not optional) on executeDiskCaptureWithAdaptiveRetry — a missing arg silently dropped telemetry. - Test: captureStreamingStage createInput() now provides the required dedupPerfs field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(engine): address deferred dedup-review items - Derivable state: drop session.staticDedupArmed/staticDedupPredicted; derive both from session.staticFrames in getCapturePerfSummary (armed ⟺ non-empty set, predicted === size) so they can't desync. - Config altitude: HF_STATIC_DEDUP now resolves into EngineConfig.staticFrameDedup (resolveConfig, opt-out on {false,0,off}), alongside forceScreenshot/browserGpuMode — armStaticDedup reads config instead of process.env. Default-on preserved (missing config → enabled). - Lossy aggregation: aggregateDedup now reports DISTINCT skip reasons (sorted, `|`-joined) across diverging unarmed workers instead of just the first. - discardWarmupCapture: also snapshot/restore staticDedupCount and lastFrameBuffer so a warmup capture can't leak a phantom reuse or a stale buffer anchor into the real summary. - Convention: perfSummary-dedup.test builds its job via createRenderJob instead of `as unknown as RenderJob`. - Docs: verification_budget added to skip-reason lists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
121cdd2d9f |
fix(telemetry): attribute studio renders to the browser user (joinable funnel) (#1492)
Studio-triggered renders emit render_complete / render_error from the CLI preview-server process, which stamps every event with the install's anonymousId (client.ts drainQueueToPayload). The browser, meanwhile, fires studio_session_start / studio_render_start under its own getAnonymousId(). So the render outcome and the render start never share a person_id — verified in data: of 15,125 users who started a studio render in 30d, ZERO have any render_complete under any source, and the 898 studio-tagged completers are disjoint server UUIDs. The studio render funnel — the product's core value moment and strongest retention signal — is therefore unmeasurable. Thread the browser's telemetry id through to the render-outcome events: - client.ts: trackEvent takes an optional distinctId; drainQueueToPayload uses `event.distinctId ?? config.anonymousId`. CLI renders unchanged. - events.ts: trackRenderComplete/trackRenderError forward an optional distinctId. - studioRenderTelemetry.ts: emitStudioRender* pass opts.distinctId through. - core studio-api (types.ts + routes/render.ts): the render route reads `telemetryDistinctId` from the request body (validated string) and passes it to the adapter's startRender, which already forwards opts to the emitters. - studio (useRenderQueue.ts): include getAnonymousId() as telemetryDistinctId in the render POST — the same id studio_* events already use. Result: studio render_complete/error now carry the browser user's id and join studio_session_start / studio_render_start. Older clients that don't send the field fall back to anonymousId (no regression). No new tracking surface — it's the existing anonymous studio id. Tests: per-event override forwarding (events), studio render distinctId threading + older-client fallback (studioRenderTelemetry), and route body → adapter forwarding incl. non-string rejection (core render route). |
||
|
Miguel Ángel
|
5f6ced116d |
feat(cli): report command failure reasons to telemetry (de-blind browser/info) (#1484)
Observability showed `browser` (~75% fail, ~1.3k users/day) and `info` (~60% fail) failing at high rates with no captured reason — only `cli_command_result success=false`. citty's `runMain` catches a command's thrown error and `process.exit(1)`s without re-throwing, so a thrown failure never reached the existing `cli_error` telemetry (which only fired from the uncaughtException / unhandledRejection handlers). Wrap every command's `run()` at the dispatch boundary (cli.ts) so a thrown failure reports its reason via `cli_error` (kind=command_error) before being re-thrown unchanged — citty's print + exit-1 behavior is preserved. This de-blinds every throw-style command at once: `browser ensure` (Chrome download), `tts`, `inspect`, `render`, etc. Paths that bypass the wrapper are handled inline: - `browser` self-exits (`path` download failure, unknown subcommand) — report inline; the ARM64 `ensure` branch previously swallowed a failed install and returned success, now reports and exits 1. - `resolveProject()` self-exits on InvalidProjectError (the dominant `info` failure — run outside a project) — report inline before exit. Hardening: - PII: `trackCliError` now redacts error_message + stack_trace via redactTelemetryString (matching render_* events) — CLI errors and stacks carry absolute install paths / cache dirs / user args. - Race: the wrapper awaits an on-demand telemetry import before re-throwing, so a command that fails before the lazy telemetry import settles still reports (a telemetry failure is swallowed and never masks the real error). Pure helpers in utils/command-failure-tracking.ts with unit tests for the throw / success / no-run / onFailure-rejection cases, the reporter wiring, and trackCliError redaction. CommandDef<any> mirrors citty's SubCommandsDef. Known scope: commands that print + `process.exit(1)` on their own validation paths (tts/validate/lint argument errors) remain wrapper-blind — follow-up. |
||
|
Miguel Ángel
|
40b49d4024 |
fix(cli): report transcribe failure reasons via cli_error (command_error) (#1479)
Transcribe failures are recorded as `cli_command_result success=false` but
without a reason: the command catches its own error, prints it, and
`process.exit(1)` — the message never reaches telemetry. `cli_error` was only
emitted from the uncaughtException / unhandledRejection handlers, so
self-handled command failures were invisible. That makes a high failure rate
countable but not debuggable.
Add `trackCommandFailure(command, err)` — a thin wrapper over the existing
`trackCliError({ kind: "command_error" })` that normalizes an unknown reason to
name/message/stack. It enqueues synchronously, so the process `exit` handler's
flushSync ships it alongside `cli_command_result`. Respects the telemetry
opt-out (gated in trackEvent) and reuses the existing PII redaction.
Wire it into all three of transcribe's failure exits (file-not-found,
empty-transcript import, and the transcribe() catch — ffmpeg / whisper-binary /
model-download errors). Now each failure carries its reason, so we can see how
much of the failure rate is environment vs user input.
The helper is generic — the same one-liner can be dropped into other commands'
failure paths, or centralized at the runMain boundary, as a follow-up.
|
||
|
Miguel Ángel
|
29d6f1eac9 | fix(render): add end-to-end observability (#1248) | ||
|
Miguel Ángel
|
d625dc8509 |
feat: post-render and Studio feedback collection via PostHog surveys (#1101)
* feat(cli): prompt for render satisfaction after successful renders * feat: add text feedback, doctor context, and Studio render feedback UI * feat(studio): replace render feedback with session-based Studio experience bar Move the feedback prompt out of RenderQueueItem (where it triggered every 5th render) into a standalone StudioFeedbackBar mounted at the bottom of the preview area. The new bar is session-gated (shows after the 5th studio session), auto-dismisses after 20s, and respects a 30-day cooldown once dismissed or submitted. Renames telemetry to trackStudioFeedback with a "studio_experience" survey ID to reflect the broader scope. * feat(studio): attach browser doctor summary to feedback events * fix(studio): use recurring interval for feedback instead of one-time cooldown * fix(cli): skip feedback prompt when an agent runtime is detected * feat(cli): add hyperframes feedback command and agent render hint - New `hyperframes feedback --rating <1-5> --comment "..."` command for submitting anonymous render satisfaction feedback via telemetry. - When an AI agent runtime is detected after a render, print a dimmed hint to stdout so the agent can optionally call the command instead of silently skipping the readline prompt. - Export getDoctorSummary from telemetry/feedback.ts to share the system-info collector between the interactive prompt and the CLI command. - Register the command in cli.ts and help.ts under Settings. * fix(studio): align feedback interval to every 15 sessions * fix: show CLI feedback on first render, Studio every 10 sessions * feat: add env flags to disable feedback prompts * feat: env flags to configure feedback prompt frequency * fix: address review — agent hint reachability, cadence gate, session debounce, deprecated API |
||
|
Miguel Ángel
|
f19d6fd471 |
feat: CLI observability + fix studio save failures on JS-created elements (#1091)
* feat(core): add probeElementInSource for source-existence checks
* feat(core): add probe-element endpoint for source-existence checks
* feat(studio): gate editing capabilities on source existence
* fix(studio): enrich save_failure telemetry with target details
* feat(studio): async selection resolution with source probe
Make `resolveDomEditSelection` async and wire a `probeSourceElement` call
into the selection path so elements generated by scripts (not present in the
source HTML) are detected early and have all edit capabilities disabled with
a clear reason message ("This element is generated by a script and cannot be
edited visually.").
Part A – core probe logic:
- `domEditingLayers.ts`: `resolveDomEditSelection` is now async; calls
`probeSourceElement` (POST /api/projects/:id/file-mutations/probe-element/:file)
when `projectId` is supplied and the element has a stable id/selector.
`existsInSource: false` flows into `resolveDomEditCapabilities`, which
disables all write capabilities with the appropriate reason.
- `domEditingLayers.ts`: `refreshDomEditSelection` promoted to async.
- `files.ts`: new `probe-element` route; extracted `resolveProjectPath`,
`resolveFileMutationContext`, `writeIfChanged`, and `parseMutationBody`
helpers to eliminate repeated boilerplate across remove/patch/probe handlers.
Part B – caller propagation (all eight consumer sites):
- `useDomSelection.ts`: `buildDomSelectionFromTarget`,
`resolveDomSelectionFromPreviewPoint`,
`buildDomSelectionForTimelineElement`, `handleTimelineElementSelect`,
`refreshDomEditSelectionFromPreview`, and
`refreshDomEditGroupSelectionsFromPreview` all made async; `projectId`
forwarded into `resolveDomEditSelection`.
- `useDomEditCommits.ts`, `useDomEditTextCommits.ts`: updated
`buildDomSelectionFromTarget` parameter type; added `await` at call sites.
- `useDomEditSession.ts`: inner `syncSelectionFromDocument` made async; fire
with `void` to satisfy the surrounding effect.
- `usePreviewInteraction.ts`: `handlePreviewCanvasMouseDown` and
`handlePreviewCanvasPointerMove` made async (React ignores handler return
values, so this is safe).
- `useStudioUrlState.ts`: deferred `buildDomSelectionFromTarget` call
converted to `.then()` chain with `void` prefix so the effect stays sync.
- `LayersPanel.tsx`: `seekToLayer`, `handleSelectLayer`, and
`handleLayerHover` made async.
- `DomEditOverlay.tsx` / `useDomEditOverlayGestures.ts`: `onCanvasPointerMove`
return type widened to `Promise<DomEditSelection | null>`; pointer-down
handler falls back to `hoverSelectionRef.current` (always populated by a
prior hover) instead of awaiting the async move callback inline.
Part C – test and tooling fixes:
- `lefthook.yml`: filesize hook shell loop explicitly skips `*.test.ts/tsx`
files as a guard against a lefthook v2.1.6 bug where `exclude` patterns are
not applied to `{staged_files}` in shell scripts.
- `domEditing.test.ts`: all `it()` blocks calling `resolveDomEditSelection`
made async with `await`.
- `DomEditOverlay.test.ts`: mock updated to return `Promise.resolve(selection)`
and `hoverSelection` pre-seeded so pointer-down test works with the new
hover-first path.
- `studioUrlState.test.ts`: `buildDomSelectionFromTarget` mocks wrapped in
`Promise.resolve()`; seek/selection hydration test made async with
`await act(async () => { await Promise.resolve(); })` to flush microtasks.
* feat(cli): add global error handlers for crash telemetry
Register process-level uncaughtException and unhandledRejection handlers
that fire trackCliError so unhandled crashes are captured in telemetry.
Add the trackCliError function to events.ts and re-export it from the
telemetry barrel.
* feat(cli): track per-command success/failure and duration
* test(core): add integration test for JS-created element probe scenario
* fix: address PR review feedback
- uncaughtException handler now calls process.exit(1) after flushing
- cli_command_result uses real exit code from process "exit" event
- drop stack_trace from cli_error (contains filesystem paths)
- skip source probe during hover — only probe on click/selection
- format .fallowrc.jsonc
* fix(cli): restore stack_trace in cli_error telemetry
* fix(cli): use captured module refs in exit handlers instead of dead import()
|
||
|
James
|
a2453c803d |
feat(telemetry): differentiate studio vs CLI renders, add studio frontend events
Adds 'source' property (cli|studio) to render_complete/render_error events, makes studioServer.ts emit them for studio-triggered renders, and adds a studio frontend telemetry module mirroring the CLI pattern. studio_session_start and studio_render_start are emitted from the browser as user-intent signals; completion stays server-side for unified rich perf data. OSS-safe: no-op when VITE_HYPERFRAMES_POSTHOG_KEY is unset. Opt-out via localStorage or navigator.doNotTrack. Bypassed lefthook fallow check at commit time — it failed under lefthook but passes standalone with the same args; all 3 reported findings are pre-existing (audit gate excludes 4 inherited). CI will run the authoritative check. |