mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
af923d947c28c3306dbd05e0ecc2b9bd4961df72
73
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vance Ingalls
|
af923d947c |
fix(producer,cli): round the fallback dB consistently, thread the verify threshold through
Review feedback on #2411 (Rames): the crash-survival RenderCaptureObservability mirror passed deFallbackFailedDb raw/unrounded while the render_complete perfSummary path rounded to 1 decimal — the same underlying PSNR could ship two different values to PostHog depending on which event fired. Extracted the existing inline round/clamp expression (previously duplicated for verifyMinDb and fallbackFailedDb) into a shared roundDb helper, applied once at the single point deFallbackFailedDb is derived from the thrown error so both downstream consumers agree. Also threads verifyThresholdDb (captured on the error but never propagated, per the nit) through DrawElementPerfInput/RenderCaptureObservability/render.ts/ telemetry as de_fallback_threshold_db on both events — the HF_DE_VERIFY_MIN_DB value the failing dB breached, letting ops read "28.4dB failed a 32dB threshold" directly instead of cross-referencing config. |
||
|
Vance Ingalls
|
a5cbb78ff9 |
feat(engine,producer,cli): capture failing dB/frame index on drawElement verify fallback
de_fallback_reason only told you the fallback happened (blank/psnr/oom/ capture_error), not the failing PSNR or frame index — that data existed as text inside the thrown error's message and was discarded on the way to telemetry. DrawElementVerificationError now carries structured frameIndex/failedDb/verifyThresholdDb; the orchestrator reads them via the new getDrawElementVerificationDetails helper instead of regexing message text, and both telemetry surfaces (the render_complete perfSummary path and the crash-survival RenderCaptureObservability mirror) emit de_fallback_failed_db / de_fallback_frame_index. Needed to distinguish "32dB vs the 32dB threshold, tune it" from "12dB real corruption, investigate" during the parallel-router soak — currently that distinction is invisible. |
||
|
Vance Ingalls
|
9bcd279b29 | feat(cli): surface stale project pin to non-TTY agents, throttled | ||
|
Miguel Ángel
|
990f5c3145 |
feat(feedback): adopt 0–10 recommendation scale (#2438)
* feat(feedback): adopt 10-point recommendation scale * docs(feedback): keep OSS scale contract self-contained |
||
|
Vance Ingalls
|
4ca1552e20 |
fix(core,cli): attribute figma REST failures to the endpoint that failed
cli_error had no way to tell which figma REST call (images, files_nodes, variables_local, styles, ...) actually hit RATE_LIMITED/FORBIDDEN/etc, so the dashboard could see failures spike but not which call caused them. FigmaClientError now carries a low-cardinality endpoint label (never the raw fileKey/nodeId), threaded through to cli_error's endpoint property. |
||
|
kiritowoo
|
ccf5f20b3b | fix(cli): stop losing render telemetry to the exit race (#2105) | ||
|
vanceingalls
|
591235f05c | feat(producer): emit eligible_off cohort-sizing signal when parallel-stream router is off | ||
|
vanceingalls
|
88c21ccd8b | feat(cli): capture_parallel_stream telemetry on success and failure paths | ||
|
Vance Ingalls
|
fecd7dc1d3 |
Merge pull request #2248 from heygen-com/bf-reuse-telemetry
feat(producer): surface beginframe no-damage reuse counters in perf summary and telemetry |
||
|
vanceingalls
|
dde3afb72e | feat(producer): surface beginframe no-damage reuse counters in perf summary and telemetry | ||
|
Miguel Ángel
|
87618eef4c |
fix(telemetry): expose stalled render stages (#2220)
* fix(telemetry): expose stalled render stages * fix(telemetry): preserve capture data on terminal stage events * fix(telemetry): fix calibration TDZ crash, tag encode/assemble, extend heartbeat cadence capture_calibration referenced captureStageObservationData before its declaration (later in the same scope), which would throw a ReferenceError for any render hitting the calibration path. Hoist the closure and split workerCount's declaration from its resolution so calibration can safely read it as undefined before capture strategy resolves worker count. Also address the two non-blocking review items: wire encode/assemble stages through captureStageObservationData for consistent tagging, and extend the heartbeat schedule to repeat every 120s after the initial 30/60/120s ramp instead of going dark on stalls beyond two minutes. |
||
|
Vance Ingalls
|
392dd410a5 |
Merge remote-tracking branch 'origin/main' into de-parallel-router-failure-telemetry
# Conflicts: # packages/cli/src/telemetry/config.ts |
||
|
Miguel Ángel
|
b1f1c0571e |
fix(cli): make skills update converge on a skill retired upstream (#2176)
`hyperframes skills update` failed hard or looped forever once a skill was
retired/renamed upstream while still installed locally (hyperframes-media folded
into media-use; hyperframes-captions/compose/tts consolidated earlier). Two
paths dead-ended:
- Install: target selection could trust a stale local skills-manifest.json
(findRepoManifest) while `skills add` always installs from the canonical repo.
isCoreSkill matches the `hyperframes-` prefix, so a retired skill was forced
into the target set, `skills add` silently declined it (exit 0), and strict
verifyInstalled threw "Skill(s) still missing after install".
- Prune: upstream `skills remove` scans on-disk directories, so a lock entry
retired before it ever shipped a bundle has nothing to match — a silent
exit-0 no-op that never clears the lock, so detectRemoved re-flags it on
every run.
The stale-skills nudge compounded it: it fired even from `skills update` itself
(pointing users back at the failing command) and its count ignored the removed
bucket.
Resolve update targets against the canonical manifest (checkSkills({ canonical:
true })) so a retired skill is never targeted. Add pruneOrphanedLockEntries to
clear the orphaned lock entries the upstream remover can't (idempotent, so a
second run is a clean no-op). Exclude `skills` from the update-nudge gate and
thread the removed count through the nudge total.
|
||
|
Vance IngallsandClaude Fable 5
|
6172d79dc2 |
fix(cli): atomic config writes, gated trial warning, and write-failure signal
Five findings from a fifth (final scoped) max-effort review of the previous commit, all local: 1. writeConfig now writes atomically (pid-suffixed temp file + renameSync — rename within one directory is atomic on POSIX). This closes the real hazard behind the review's torn-read finding: readConfig's corrupted-file catch RESETS the config to defaults (telemetry re-enabled, anonymousId rotated, trial fields wiped), so a concurrent reader catching a non-atomic write mid-flight would silently destroy the user's config — and the previous commit's per-render readConfigFresh() at the arm site multiplied exposure to exactly that window. Verified against a real filesystem, not just the mocked unit tests. 2. writeConfig now returns whether the write landed (errors still swallowed — telemetry must never break the CLI). persistDeParallelRouterTrialFired uses it to stop immediately on a genuine fs failure (retrying an unwritable file is pointless) and reserve its retries for actual concurrent clobbers, instead of 3 blind write attempts + 4 disk reads. 3. The persistence-failure console.warn is now !quiet-gated like every other trial message — a quiet/batch-json render on an unwritable ~/.hyperframes no longer emits unexpected stderr that CI wrappers asserting empty stderr would misread as a render failure. The in-process latch already guarantees the safety behavior whether or not the warning prints. 4. The arm site short-circuits on the in-process fired latch BEFORE the fresh config read — post-fired batch rows no longer pay a per-row config read + parse + shared-cache invalidation for an answer module state already knows. 5. Replaced the new `as T` assertions in render.test.ts's config-state factory with an explicitly typed vi.hoisted return (repo TypeScript convention: no `as T`). config.test.ts: node:fs mock gains renameSync (faithful to the new atomic write); new test covers the success/failure return and asserts no temp file survives a write. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Angel Simon Sierra
|
cea3458016 |
refactor(cli): single-source-of-truth pass over the check branch
Every duplicated decision gets one owner: rectToBbox lives in checkTypes (was verbatim in pipeline and browser layers); the audit seek tuning is one exported AUDIT_SEEK_OPTIONS consumed by check and the deprecated inspect path; zoom padding/scale defaults export from the capture module instead of re-literalized in three files; the optional run_id property is built by one helper across all three telemetry events; check's --max-transition-samples parsing reuses its own positiveInteger helper; validate drops a leftover re-export and redundant explicit-default args go away. Tests: the contrast candidate round-trip gains a real integration anchor (the actual browser script eval'd in-page, a wrapper asserting finish receives the page-script bbox shape) replacing regex-over-source as the primary guard; the redundant geometry source-golden and a duplicated deprecation-envelope assertion are dropped. |
||
|
Miguel Angel Simon Sierra
|
3a02942a03 |
feat(cli): run-ID telemetry correlation and check breakdown event
HYPERFRAMES_RUN_ID (trimmed, 128-char cap) attaches as run_id to the generic cli_command / cli_command_result events, absent when unset, so an orchestrator setting it per design element can group a verify loop's invocations in analytics. check additionally emits one check_report event per invocation (including lint-short-circuited and failing runs): gate booleans, per-class error/warning counts, launch/seek/contrast phase timings, sample counts, ok and exit code. Timings stay internal; no command output changes. |
||
|
Vance IngallsandClaude Sonnet 5
|
dc6df93de5 |
fix(cli): fix concurrency race, none-vs-undefined bug, and 3 more DE trial gaps
Six findings from a third max-effort code review, focused on the previous commit's fixes: 1. --batch-concurrency N>=2 runs genuinely concurrent renderLocal() calls (Promise.all workers in batchRender.ts), which can't safely share the trial's one process-wide env var + module flag — a row finishing first could tear down the env var/flag mid-render for a sibling row still in flight. Rather than attempt to make shared process-global state safe under real concurrency, added RenderOptions.disableDeParallelRouterTrial and set it whenever batchConcurrency > 1 — the trial simply isn't offered when it can't be evaluated safely. 2. maybeConsumeDeParallelRouterTrial's "outcome === undefined" no-op guard almost never fired: aggregateDrawElement (perfSummary.ts) defaults parallelRouter to the string "none" for every render, whether or not drawElement/the router ever engaged — never undefined. Every ordinary render below the router's own frame threshold (the common case) was ticking the render-count backstop, tripping DE_PARALLEL_ROUTER_TRIAL_MAX_RENDERS after 25 completely unrelated renders that never touched the router. Now treats "none" the same as undefined. 3. isDeParallelRouterTrialBlocked relied solely on shouldTrack(), which memoizes its verdict once per process — during a long --batch run, a `hyperframes telemetry off` issued from another terminal mid-batch would never be observed. Restored a direct config.telemetryEnabled check (read fresh every call, unlike shouldTrack()'s cache) alongside it. 4. maybeConsumeDeParallelRouterTrial's config write had no way to detect a losing race against a concurrent process — added a verify-and-retry loop (write, re-read fresh, retry up to 3x if a concurrent writer landed in between) that narrows the window further without a full file-locking rewrite. 5. The trial could arm before the first-run telemetry disclosure (showTelemetryNotice) was guaranteed to have printed — that notice runs via a fire-and-forget, unawaited dynamic import in cli.ts with no ordering guarantee relative to the render command. Rather than touch that pre-existing async bootstrap chain, gated the trial on config.telemetryNoticeShown: it simply never offers itself on a fresh install's very first invocation. 6. Added a dedicated config.test.ts exercising readConfig/readConfigFresh/ writeConfig through the REAL module (node:fs mocked with an in-memory fake, not a HOME-env hack) — readConfigFresh's cache-bypass and the type-guarded boolean/number parsing had zero coverage through the real implementation before this. Also fixed the test fixture that was supposed to cover finding #2 but used an unrealistic `drawElement: {}` shape instead of the real `{ parallelRouter: "none" }` aggregateDrawElement actually produces. Extracted applyDeParallelRouterOutcome to keep maybeConsumeDeParallelRouterTrial under the repo's complexity gate after adding the retry loop. 11 new/updated tests in render.test.ts (56 total) + 7 new tests in config.test.ts. Verified against fallow's audit gate clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
532dad7cc7 |
fix(cli): fix batch re-entrancy, config race, exposure cap, and shouldTrack gap in DE trial
Four confirmed findings from a max-effort code review of the CLI trial mechanism: 1. maybeEnableDeParallelRouterTrial's `process.env.HF_DE_PARALLEL_ROUTER !== undefined` guard couldn't distinguish "the user set this" from "an earlier renderLocal() call in this same process already armed it" — so in --batch (all rows share one process), only row 1's outcome could ever reach maybeConsumeDeParallelRouterTrial. A revert on any later row was silently never persisted. Added a module-level deParallelRouterTrialManagedByUs flag to disambiguate, with a test-only reset export since it's process-lifetime state a real CLI invocation never needs to reset but a test suite sharing one module instance does. 2. writeConfig is a non-atomic whole-file overwrite with no locking, and readConfig's cache never invalidates — a concurrently running second CLI process (another terminal, a parallel script; doesn't even need to be a render, any command calls incrementCommandCount) could silently clobber a just-persisted deParallelRouterTrialFired:true with its own stale snapshot. Added readConfigFresh (bypasses the cache) and use it immediately before the trial's read-modify-write, narrowing the race window without a full config-subsystem locking rewrite. 3. The prior commit's semantics flip removed the only exposure cap — a healthy router that never reverts now force-enabled the experimental path on every eligible render forever. Added DE_PARALLEL_ROUTER_TRIAL_MAX_RENDERS (25) as a backstop: the trial turns off after this many engaged renders even absent an actual failure. 4. maybeEnableDeParallelRouterTrial only checked config.telemetryEnabled, not shouldTrack() — so a dev-mode run or a DO_NOT_TRACK/ HYPERFRAMES_NO_TELEMETRY user got the experimental path silently armed while telemetry was simultaneously blocked underneath it. Now gates on shouldTrack() (a strict superset). Also fixed, lower severity: readConfig's deParallelRouterTrialFired/ deParallelRouterTrialRenderCount parsing now validates the JSON type explicitly instead of a bare truthy/nullish read, so a hand-edited or corrupted config can't have the string "false" misread as truthy. Refactored maybeEnableDeParallelRouterTrial into three smaller functions (isDeParallelRouterTrialBlocked, stopManagingDeParallelRouterTrial) to bring cyclomatic/cognitive complexity back under the repo's threshold — also de-duplicates the "stop managing the env var" logic shared with maybeConsumeDeParallelRouterTrial. 14 new/updated tests (43 total in render.test.ts), including a direct regression test for the batch re-entrancy scenario and a loop test for the render-count cap. Verified the config primitives end-to-end against a real file, not just the mocked unit tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
19f90b0b92 |
fix(cli): keep the DE parallel-router trial on until a real failure, not first engagement
Only consuming telemetry from one data point per install badly undersampled the "routed" (successful) outcome — the far more common case. Changed maybeConsumeDeParallelRouterTrial to only turn the trial off when the router's OWN safety net actually fired (deParallelRouter === "reverted"), not on a clean "routed" success. This runs the experiment on every eligible render for an install indefinitely until it hits one real failure, then stops for that install going forward — trading a slightly higher per-install ceiling on experimental-path exposure for dramatically more successful- routing telemetry volume across the fleet. Also fixed a related edge case while updating this: a render that merely "routed" (router fired, self-verify never even tripped) but then crashed for an unrelated reason (e.g. cancellation) no longer counts as a router failure — only "reverted" (the router's fallback path actually engaged) does. Cancelling a render isn't evidence the router is unsafe. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
37b6a4e7e5 |
feat(cli): one-shot DE parallel-router trial per install for real telemetry
HF_DE_PARALLEL_ROUTER is a producer env var with no self-serve opt-in path for real users, so waiting for someone to manually enable it would never produce the real-traffic telemetry (revert rate, verify-db distribution) the router's soak plan calls for. renderLocal now enables the experiment for free on a fresh install's CLI renders until it actually engages once (routed or reverted — either produces telemetry), then persists that to ~/.hyperframes/config.json and never touches it again for that install. A render whose frame count never crosses the router's own eligibility threshold doesn't consume the trial — it stays available for a later render that does qualify. Never overrides a user's own explicit HF_DE_PARALLEL_ROUTER setting, and only engages when telemetry is enabled (no point risking the experimental path if we can't record the resulting signal). Scoped to the in-process CLI render path only — Docker renders don't thread perfSummary/errorDetails back to the CLI process, so trial consumption can't be detected there. Verified the config round-trip against a real file (fresh install -> undefined -> write true -> persists across reread), not just the mocked unit tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
a355fb2f6b |
fix(producer,engine,cli): oom wrapping, cancellation, fallback-reason gaps
Three defects found by max-effort code review of this branch: 1. The Bun OOM exact-match regex was defeated by this codebase's own parallel-worker error wrapping. executeParallelCapture/formatWorkerFailure (parallelCoordinator.ts) always wrap a worker's error as "Worker N: <message>", optionally suffixed and joined with other workers' segments, all prefixed "[Parallel] Capture failed: ". That wrapping defeated the exact-message check for exactly the cohort (deParallelRouter routed, N separate Chrome processes) the OOM-drops-to-1 fix targets — a real OOM there would retry at the SAME worker count instead of dropping to 1. Added a second pattern that recovers the signal by requiring "out of memory" appear as the WHOLE content of a "Worker N: ..." segment (bounded by end-of-string/"; "), preserving the same exact-match property (no bare substring match) while surviving the wrapping. Verified against the real wrapping logic, not a hand-typed guess at its shape. 2. shouldRetryViaPinnedFallback didn't exclude cancellation, so aborting a render mid-capture on the pinned router/inversion cohort would detour through spawning a fresh encoder/capture session before the outer catch's RenderCancelledError branch ended the render — delaying "stop" with a pointless resource spin-up/tear-down. Added an isCancellation param (checked first, before isVerifyError) using the same `err instanceof RenderCancelledError || abortSignal?.aborted` check the outer catch already uses. 3. deFallbackReason (this PR's new "oom"/"capture_error" values) was set locally but never mirrored into RenderCaptureObservability alongside deSelfVerifyFallback, so a render that fails AFTER a fallback attempt (perfSummary never built) was indistinguishable in render_error telemetry from one that never attempted any fallback — undercutting the "how often does the OOM retry fire on a render that still ultimately fails" question this branch exists to answer. Threaded through RenderCaptureObservability → RenderObservabilityTelemetryPayload → renderObservabilityTelemetryPayload, mirroring the existing deSelfVerifyFallback plumbing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Sonnet 5
|
ec921e143b |
feat(producer,cli): full telemetry visibility for DE parallel-router/inversion failures
render_error previously carried zero DE-cohort context — a hard failure while routed (worker crash, OOM, capture timeout from the fixed 3-worker pin overriding calibration) was indistinguishable from any other failure. The data existed (RenderCaptureObservability is mutated live and survives into job.errorDetails on the failure path) but was never projected into the render_error payload, which only ever drew de_* fields from perfSummary (success-only). - RenderCaptureObservability now also records dePreInversionWorkers / dePreRouterWorkers — the worker count calibration would have picked absent the experiment — so a resource-pressure failure can be correlated with the router overriding a lower calibrated count. - New capture-sourced de_* fields on RenderObservabilityTelemetryPayload, shared by trackRenderComplete and trackRenderError. Explicit perfSummary-sourced fields still win on render_complete (spread moved first in the event object) — this is purely a failure-path fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
0393ba5be2 |
feat(producer): default-off router for verified parallel drawElement
Promotes the opt-in HF_DE_PARALLEL_STREAM mechanism (#2056) into the auto-routing decision, gated behind its own default-off flag (HF_DE_PARALLEL_ROUTER). This is the next step from the 2026-07-08 parallel-DE benchmark verdict: par3/single 1.16-1.36x on real-work comps >=2,000 frames, no comp anywhere losing to single-worker. shouldPreferParallelDrawElement mirrors shouldPreferSingleWorkerDrawElement (#2026) but takes priority over it when both are eligible — its higher default threshold (HF_DE_PARALLEL_MIN_FRAMES=2000 vs the inversion's 900) means it only ever picks up the long tail the inversion's own benchmark didn't cover. Fixed at 3 workers (benchmark-validated; not calibration- derived), same shape as the inversion pinning to a fixed 1. resolveParallelRouterRetryPlan mirrors resolveInversionRetryPlan for the self-verify-failure rollback path: falls back to the ordinary (non-DE) parallel-disk path at the pre-router worker count. The caller must clear HF_DE_PARALLEL_STREAM before recomputing useStreamingEncode or the retry would keep resolving to the parallel-streaming shape. New telemetry (de_parallel_router, de_pre_router_workers) tags which render used the router, separate from de_worker_inversion — needed before the planned telemetry soak can segment revert-rate and de_verify_min_db to the parallel cohort specifically; today there's no way to tell those apart from ordinary single-worker DE renders. Verified end-to-end: HF_DE_PARALLEL_ROUTER=true on a 2,381-frame comp resolves to 3 workers with 3 separate drawElement sessions and renders successfully; without the flag, behavior is unchanged (falls through to the existing single-worker inversion, workerCount=1) — no regression to current production routing. 114 orchestrator tests pass (15 new). |
||
|
Miguel ÁngelandClaude Opus 4.8
|
57b3c78987 |
feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare + compare (#2041)
* feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare CLI Add color grading to media-use as first-class resolve types plus a faithful comparison command. All local, offline, deterministic — no model, no GPU. - resolve -t grade / -t lut: produce a data-color-grading block (or a frozen .cube). Look cascade: core preset (no file) -> bundled .cube library -> parametric buildCube. Emitted .cube is Rec.709 and validated against core's colorLuts constraints (LUT_3D_SIZE <= 64) before it is frozen. - smart grade (grade --for <media>): ffmpeg signalstats -> adjust suggestion (exposure / contrast / white balance), surfaced with the measured evidence on stderr as a starting point; never auto-applied. - hyperframes grade-compare: renders N candidate grades onto a reference frame through the real runtime shader into one labeled comparison PNG, so an agent picks a look without opening Studio. Prepends an "original" baseline cell by default (--no-baseline to omit). Shares the headless-capture pipeline with snapshot via capture/captureCompositionFrame. - media-use SKILL: proactive "media opportunity pass" guidance (grounded signal -> offer, ask once, surface don't mutate). Verified: media-use 116/116, grade-compare 7/7, snapshot 9/9, lint + format clean, full build green, comparison renders end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * test(cli): narrow grade-compare baseline assertion off unknown-typed grading Assert the whole cell via toEqual instead of reaching into .grading.preset / .grading.lut on the unknown-typed field, keeping the test typecheck-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): agent-authored LUTs via --params + validate --from cube; never-read-.cube guardrail - resolve -t lut / -t grade --params '<json>': build a parametric .cube from explicit params (bypassing the intent cascade), validate, and freeze in one step. --intent becomes the optional description. Lets an agent commit a look it computed itself. - --from <file.cube> now validates the ingested LUT for lut/grade types and rejects an invalid/oversized cube (no partial write) — the escape hatch for a LUT the agent generated with its own code. - SKILL.md: hard rule to never read a .cube body into context (~size^3 lines, zero legible signal) — inspect via grade-compare (see it) or cube-validate (ok/size), read the manifest description for meaning; plus both authoring paths and the parametric-vs-film-stock ceiling note. Verified: media-use 116/116, lint + format clean; smokes — --params builds a valid frozen cube, grade --params returns a lut block, bad JSON and an oversized --from cube are both rejected with no stray file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): grade-compare validates referenced LUTs, warns on no-op cells, caps candidates Bug-bash follow-ups — grade-compare silently accepted bad input: - Validate LUT *content*, not just existence: each referenced .cube is parsed with core's parseCubeLut (now exported from @hyperframes/core) and rejected with a per-cell error ("LUT for \"<label>\" is not a valid .cube: ..."). A file that exists but isn't a valid cube no longer renders a silent no-op cell. - Warn on inactive cells: a grading that normalizes to inactive (e.g. a malformed {lut:12345}) emits a stderr warning naming the cell; the auto-prepended "original" baseline is intentionally inactive and stays silent. stdout remains valid JSON. - Cap candidates at 16 (excluding baseline): over-cap input renders the first N and reports {truncated:true, total:M} on stdout + a stderr note — no silent drop, no unbounded giant sheet. Verified: grade-compare 10/10; non-cube LUT → clear error; {lut:12345} → warning + ok; 20 cells → cells=17 truncated total=20; valid runs unchanged. Lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(cli): general `hyperframes compare` visual-variant primitive Generalize grade-compare's "render N variants → one labeled sheet → the agent looks and picks" loop into a standalone command that works on ANY variation (font, layout, motion, grade, whole compositions) — the tool never needs to know what differs. - `hyperframes compare <path...> [--at <sec>] [--labels a,b,c] [--out] [--cols] [--json]`: renders each agent-authored composition variant through the real runtime (captureCompositionFrame) and stitches one labeled comparison sheet + JSON ({ok, sheet, rendered, variants, truncated?/total?}). 2+ paths required; caps at 16 with loud truncation. It presents, it does not judge — choosing is the caller's job. - Factored the shared "render a labeled set → contact sheet" path so compare, grade-compare, and snapshot all sit on it (no duplication). grade-compare is now the first color-specific specialization of this primitive. - New pathArgs util + contactSheet test; hyperframes-cli SKILL documents compare as the agent's "see your own renders and choose" primitive. Verified: 26/26 across compare + grade-compare + snapshot + contactSheet (no regressions); compare renders 3 variants into one visibly-distinct labeled sheet; 2+-path error path clean; lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): green the skills CI — skip ffmpeg tests when absent, oxfmt markdown The "Test: skills" CI job runs bare `node --test` with no ffmpeg on PATH (by design — skills tests are meant to be node-builtin-only). The grade-analyzer + smart-grade tests shell to ffmpeg and were failing there with ENOENT. Guard them to skip when ffmpeg isn't on PATH; they still run locally / where it is. Also oxfmt README.md + hyperframes/media-use SKILL.md (the whole-repo `oxfmt --check .` Format job caught markdown left unformatted by the rebase conflict resolution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): skip core-conformance test when tsx is unavailable The "Test: skills" CI job installs no deps, so the normalizeHfColorGrading conformance test (which imports core's TS via `node --import tsx`) failed there. Guard it to skip when tsx can't resolve; runs locally / in the deps-installed Test job. Completes the skills-CI greening (the ffmpeg guards handled the rest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): escape grade-compare src double-quotes (CodeQL XSS) + Windows-safe compare test - grade-compare built `<img src="...">` (double-quoted) with the single-quote escaper, leaving `"` unescaped — a `"` in the frame path could break out (CodeQL: incomplete HTML attribute sanitization). Use escapeXml for src. - compare label test hard-coded POSIX paths that can't match on Windows; assert the derived labels (the subject); path resolution is covered elsewhere. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * refactor(media-use): generate LUT library from params (drop committed .cube files) The 3 bundled .cube files were 733 lines each (2,199 total) and were themselves buildCube output — pure repo bloat. Replace with compact per-look params in luts/index.json, generated on resolve; add an optional `url` for future scanned LUTs to be CDN-hosted + downloaded on demand (freezeUrl) instead of committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): serve library LUTs from CDN on-demand (static.heygen.ai/luts), params fallback Looks now carry a CDN `url` (hosted at s3://heygen-public/luts → static.heygen.ai/luts/<id>.cube); resolve downloads + validates + freezes on demand, like bgm/image. `params` stays as the deterministic offline fallback (--local-only, or if the download fails), so resolution is never blocked on the network. Provider prefers url, falls back to params. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(media-use): address #2041 review — atomic LUT writes, compare telemetry, follow-ups - Atomic .cube writes: library provider (url + params) and the parametric generator now write to a .tmp path, validate, then rename, so a crash can never orphan an invalid .cube at the final path (was validate-after-write). - track("media_use_resolve") now emits provenance.via (url/params-fallback/params). - grade-compare + compare: --timeout flag (was hardcoded 5000) and a media_use_compare event (cells, truncated, total, render_ready_timed_out); openSettledCompositionPage now surfaces the render-ready timeout. - compare staging skips node_modules/.git; --for gets an upfront existence check. - Rec.709 luma comment; HYPERFRAMES_ANALYZE_TIMEOUT_MS override; measured note uses basename; LUT s3 hosting moved from index.json into luts/README.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
8ba3c33915 |
fix(engine,producer,cli): close review gaps in sub-timeline fail-fast
Address PR #2045 review feedback: - Share a SubTimelineWaitOutcome type (engine) end-to-end instead of widening to string across CapturePerfSummary / RenderPerfSummary / telemetry, so the three layers can't drift. - Dedupe scriptLoadFailures on push — a 4xx response and its trailing requestfailed both recorded the same URL, doubling the failed-URL list in the fail-fast warning. - Thread the sub-timeline-wait outcome into render_error (not just render_complete): a render that fail-fasts and then fails downstream (pollVideosReady, extract, encode) previously dropped this signal on the floor. dedupPerfs is now function-scoped so the catch path can read it, same treatment as the existing captureAttempts array. |
||
|
Vance IngallsandClaude Fable 5
|
54359f3d6a |
fix(engine): fail-fast the sub-composition timeline wait when a script 404s
pollSubCompositionTimelines waits for every [data-composition-id] host to
register window.__timelines[id]. When the script carrying that registration
fails to load (404 / request failure), the registration can never arrive —
but the poll still burned the full playerReadyTimeout (45s), then warned and
shipped a silently animation-less render. Wild scale: the capture-setup
histogram over 30 days of local renders decays smoothly (402/503/364/282/191
per 5s bucket) then spikes to 705 at the 45s bucket — ~1,000 renders/month
across 402 distinct users, ~15 user-hours of pure waiting.
- Sessions now record failed SCRIPT resources (requestfailed + HTTP>=400
response, listeners that already existed for diagnostics) in
session.scriptLoadFailures.
- pollSubCompositionTimelines takes a failure getter and cuts the wait to a
2s grace once any script failed, with a loud warning naming the URL(s).
Late-registering fetch-async comps are unaffected: no script failure means
the full timeout still applies, and a registration landing inside the
grace window still wins (tested).
- Outcome telemetry: session.subTimelineWaitOutcome ("ready" | "timeout" |
"script_failure") -> CapturePerfSummary -> RenderPerfSummary.subTimelineWait
(worst across sessions) -> render_complete sub_timeline_wait, so the wild
rate becomes directly trackable instead of setup-histogram forensics.
Validation: the discovery comp (0768f038, its animations.js unreachable)
drops from ~72s to 23.1s total — poll cut at 2.1s with the script named;
healthy comp reports "ready". Canary suite 7/7 (PSNRs identical). 4 new
poll unit tests; engine suite 907 passed (14 failures are PRE-EXISTING on
main at v0.7.42 — 18 fail on a clean checkout, stash A/B verified).
tsc/oxlint/oxfmt clean.
Corpus note: 258/1,762 corpus comps (14%) reference local scripts missing
from the corpus fetch — their historical eval INIT timings measured this
timeout, not the engine. Capture-stage ratios remain valid (both paths paid
it equally).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Fable 5
|
924727a0b4 |
feat(producer,cli): drawElement priority inversion — single-worker streaming over auto-parallel (#2026)
* feat(producer,cli): drawElement priority inversion — single-worker streaming over auto-parallel clamp:parallel eats 50% of local renders (1,326/fortnight; DE engagement stuck at 3.8%) by routing multi-worker renders to unverified screenshot capture. Benchmarks (2026-07-08, 4 comps x W1/W2/W3/W5) show that above the ~900-frame amortization crossover, single-worker VERIFIED drawElement streaming beats screenshot-parallel at EVERY worker count (2,380f: 66s vs 109-127s; 3,600f: 33s vs 39-56s; parallel scaling flattens past W2), while below it DE's fixed init cost loses by <=2.2s. - shouldPreferSingleWorkerDrawElement (exported predicate + 7 unit tests): inverts an AUTO-resolved multi-worker render to workerCount=1 when the comp matches the benchmarked configuration — default-on DE (darwin hardware clamp upstream), no compile gate, no forced-screenshot hint, mp4 output, single-worker streaming eligible, and totalFrames >= HF_DE_SINGLE_MIN_FRAMES (default 900; 0 disables). Explicit --workers N is always honored. - Inverted renders keep the probe session and land on the worker-encode streaming drain — the ONLY path with runtime self-verification, so this moves ~40% of previously-clamped renders onto the verified fast path. Comps that later hit an init-time gate (~1.5% of local renders) render single-worker screenshot streaming; accepted trade. - Telemetry: de_worker_inversion on render_complete (orchestrator -> perfSummary.workerInversion -> CLI), plus the worker_resolution observability checkpoint now records deWorkerInversion. Validation: e2e matrix on 2,381f comp — auto->5 workers inverted to 1, DE verified 4x inf PSNR, RENDER_OK; short comp (360f) auto stays 5-worker; explicit WORKERS=3 honored; HF_DE_SINGLE_MIN_FRAMES=0 disables. Canary suite 7/7 (PSNRs identical). renderOrchestrator tests 86/86. tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(producer): review fixes — inversion routing guards, calibration skip, retry revert Max code-review round on the inversion (13 confirmed findings): - Streaming spawn-failure disk fallback now clamps default-on drawElement (deClampReason=disk_path, DE-mode probe closed) exactly like the pre-capture clamp — previously it carried useDrawElement=true onto the unverified disk path, the hole the verified-path confinement exists to close, newly reachable for every inverted render. - Predicate gained the routing knowledge it was blind to: layered/HDR and shader-transition comps (drawElement never runs there), supersampling (deviceScaleFactor>1 init gate), a probe session whose init gates already disengaged DE, and the PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true explicit parallel-DE opt-in (honored like --workers N). - Eligibility is evaluated BEFORE capture calibration and skips it when the inversion pins workers to 1 regardless of the estimate — the throwaway calibration browser + sample captures cost ~41s on the 2,381-frame benchmark comp (auto render: 111.6s -> 70.1s total). - Self-verify retry reverts the inversion: the re-render returns to the pre-inversion parallel screenshot path (disk) instead of single-worker screenshot streaming, the slowest shape for exactly the comps drawElement damages. - HF_DE_SINGLE_MIN_FRAMES="" (set-but-empty) now falls back to the 900 default instead of aliasing the 0 kill switch. - Timeout advisory uses the RESOLVED worker count — an inverted render that times out no longer prints "Retry with --workers 1" (the configuration that just failed). - Telemetry: deWorkerInversion recorded in capture observability (failed renders are attributable), emitted as literal false when not fired (queryable denominator), and the drawElement perf input shape is one exported DrawElementPerfInput type instead of three copies. - Tests: requestedWorkers undefined (the value production actually passes) + the four new predicate guards; 91/91. Validation: e2e auto render — calibration skipped (deInversionEligible), inversion fires, DE verified 4x inf, total 70.1s (was 111.6s); HF_DE_SINGLE_MIN_FRAMES=0 restores calibration + parallel; canary suite 7/7 (PSNRs identical); tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(producer,cli): review round 2 — loss-cohort telemetry, retry-plan helper, boundary tests - de_worker_inversion is now a tri-state string ("inverted" | "reverted" | "none") instead of a boolean: the self-verify retry marks the render "reverted" rather than resetting to false, so the dashboard can segment the lost-inversion cohort first-class instead of inferring it from deSelfVerifyFallback + frame-count joins (james-russo #1). - The retry rollback is extracted to resolveInversionRetryPlan (pure, exported) with unit coverage: pre-inversion worker-count restore, streaming re-resolution (multi-worker retry -> disk), "reverted" state, null when never inverted (james-russo #2). - WOULD_RESOLVE_MULTI_WORKER named constant replaces the bare sentinel 2 (james-russo #5); minFrames: -1 boundary case added (miga #3). 94/94 renderOrchestrator tests; tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(producer,cli): emit de_pre_inversion_workers for the parallel counterfactual The ramp-down decision needs "did DE beat the parallel render it displaced", not just "did DE beat single-worker screenshot". Emitting the worker count the auto-resolution chose BEFORE the inversion pinned it to 1 makes the parallel counterfactual computable per render (screenshot ms/frame from the verify samples / W x the measured parallel-efficiency curve). Set only when the inversion fired. Smoke: 2,381f auto render -> de_worker_inversion="inverted", de_pre_inversion_workers=5, mode=drawelement, verify armed 4. 99/99 tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Fable 5
|
4c8064d4d3 |
fix(cli): figma component import survives unrenderable nodes (#2022)
Live testing against a real community file (Ratings) found a nested instance node figma refuses to render as svg — which aborted the entire component import. The rasterize loop now retries the node as png, and only if both formats fail warns and skips THAT node (placeholder keeps its data-figma-rasterize marker, no src) instead of failing the import. On the file that surfaced this, the png retry recovers the node — 31/31 placeholders get assets. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
3d8372f880 |
feat(cli): associate signed-in HeyGen account with telemetry (#2020)
* feat(cli): associate signed-in HeyGen account with telemetry Sign-in telemetry currently attributes everything to the anonymous install id, so the sign-in funnel can be counted but a completed sign-in can't be tied to the account it produced. This associates the two. - On a completed sign-in, emit a PostHog `$identify` alias whose `$anon_distinct_id` is the install's anonymousId, so events recorded before sign-in stitch to the same person, and tag `auth_login_completed` with the account identity (the pre-plumbed `distinctId`). - `/v3/users/me` exposes no opaque user_id, so the identity key is the account email, falling back to username (single `identityKey` helper). - Both no-op under the `telemetry disable` opt-out and only fire after the user chooses to sign in. Privacy disclosure updated in lockstep, since this is the first PII the CLI attaches: the first-run telemetry notice and the telemetry section of docs/packages/cli.mdx now state that signing in links your account email to your usage. Tests: identifyUser payload + no-op, completion attribution incl. username fallback and no-identity-on-reject/empty. Verified end-to-end against the built CLI: pre-auth events anonymous, $identify carries $anon_distinct_id, completion carries the account email. * docs(cli): disclose the username identity fallback Review gating item: identityKey is `email ?? username`, but the first-run notice and cli.mdx said only "email", so an emailless account's username would reach PostHog undisclosed. `/v3/users/me` treats email as optional (pickString), so the fallback is live code, not dead — disclose it rather than assert an unverifiable email guarantee. Both surfaces now say "email, or username if the account has no email". Also soften the identityKey comment: it implied username is "less identifying", but HeyGen usernames are often email-shaped, so the note now states username is a fallback, not a privacy win. |
||
|
Vance IngallsandClaude Fable 5
|
b26c27576b |
feat(engine,producer,cli): verify video comps via deferred DE init + capture p50 (#2015)
* feat(engine,producer,cli): verify video comps via deferred DE init + capture p50 Closes the two biggest gaps in the first day of v0.7.38 wild data: 88% of drawElement renders (video comps initialized via probe sessions) ran with self-verification unarmed, and speedup was measurable on only 3 of 76 renders. - Deferred drawElement init: probe sessions initialize before video extraction, so they have no frame injector — ground-truth screenshots would capture black <video> boxes, and verification skipped the whole comp. DE init now stops after the gates for injector-less video comps (deInitDeferred; autoAlpha flag retracted in case no path completes it) and completeDeferredDrawElementInit finishes verification + canvas injection + worker-encode at capture time, once prepareCaptureSessionForReuse has attached the injector. Validated end-to-end: a probe-path video comp now arms 4 ground-truth frames with real video pixels (3x inf + 64.7dB) and renders drawElement verified. - capture_p50_ms: per-frame capture durations are sampled (capturePerf.frameMs; batch frames get the batch mean) and the median ships as CapturePerfSummary.p50TotalMs -> RenderPerfSummary.captureP50Ms -> render_complete capture_p50_ms. Unlike capture_avg_ms it is immune to first-frame warmup and stage-setup amortization — smoke: avg 15ms vs p50 8ms on the same render, p50 matching the measured steady-state floor. Dashboard speedup tiles can drop their frame-count floor once this ships. - video_count on render_complete: segments speedup by video-injection comps (whose per-frame gain is legitimately lower) vs pure-graphics. Canary suite 7/7; engine suite 905 passed (1 pre-existing upstream failure); tsc/oxlint/oxfmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(producer): complete deferred drawElement init on the disk capture path Review (miga): a probe-initialized video comp falling back to the disk path kept deInitDeferred and silently stayed in screenshot mode — a regression for PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true renders that previously ran drawElement there. Complete the deferred init on the sequential disk path under the same explicit-opt-in test the orchestrator clamp uses; default-on renders stay on the screenshot baseline (this path has no drain-time self-verification, per the #1998 confinement rule). Validated: video comp + PRODUCER_ENABLE_STREAMING_ENCODE=false + explicit opt-in logs "(deferred drawElement init)" completion on capture_disk and renders correct video pixels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Fable 5
|
1005703441 |
feat(engine,producer,cli): drawElement release telemetry on render_complete (#2002)
Default-on drawElement ships with a runtime self-verification net (#1998); this makes its in-the-wild behavior observable. Every render_complete event now answers: which capture mode actually ran, why drawElement disengaged when it did (compile gate / producer clamp / engine init gate), whether the self-verify net fired and why, and how much margin verification had. Follows the static-dedup telemetry pattern: engine session fields → CapturePerfSummary → RenderPerfSummary.drawElement → snake_case props on render_complete. New event props: de_capture_mode, de_compile_gate, de_clamp_reason, de_gate_reason, de_worker_encode, de_verify_armed, de_verify_checked, de_verify_min_db (margin above the 32dB threshold — drift here is the early-warning signal before fallbacks spike), de_verify_init_ms, de_self_verify_fallback, de_fallback_reason, de_blank_suspects, de_blank_deterministic_accepts, de_blank_recaptures, de_boundary_frames, de_ncpr_fallbacks. Validated end-to-end on live renders: drawelement path reports mode/verify counters/minDb/init cost; a blur-gated comp reports mode=screenshot + gate_reason=css_effect:filter; a forced verification failure reports self_verify_fallback=true + fallback_reason=psnr. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
769dc702c1 |
feat(cli): emit sign-in lifecycle telemetry (#2000)
* feat(cli): emit sign-in lifecycle telemetry The CLI tracks command and render lifecycles but emits nothing for `auth login`, so sign-in outcomes are invisible on the observability dashboards — a completed sign-in, an abandoned browser flow, and a rejected key all look identical (absent). This leaves a blind spot in the same funnel the render events already cover. Add three events mirroring the existing `trackX` pattern: - auth_login_started (method: oauth | api_key) - auth_login_completed (method) - auth_login_failed (method, reason) `reason` is a fixed low-cardinality enum (flow_error / no_credential / rejected / invalid_input). No token, key, identity, email, or free text is ever attached — consistent with the existing anonymous telemetry and the `telemetry disable` opt-out. Wired into both the OAuth and --api-key paths in `auth login`, with unit coverage for the new events. * fix(cli): close sign-in telemetry funnel dropout gaps Follow-up so `started` reconciles to `completed + failed` on the common abandonment paths, which the first cut missed: - Interactive prompt cancel (Ctrl-C) now surfaces as a throw that the single catch in the api-key path records as `aborted`, instead of a bare exit with no event. - A stdin read that times out in non-TTY `--api-key` mode now records `aborted` before the error propagates, rather than exiting silently. - OAuth split: a timed-out browser callback (user closed the tab) is tagged `flow_timeout`, separated from real `flow_error` (IdP/network), since the walk-away timeout is the dominant non-error dropout. Also pre-plumb an optional `distinctId` on the three trackers, mirroring trackRenderComplete/trackRenderError. Unused today; it lets a later identity-level attribution be a one-line callsite change rather than a signature sweep. Coverage added for the new reasons and forwarding. |
||
|
Vance IngallsandClaude Fable 5
|
e9076324e7 |
feat(cli): figma import telemetry — subcommand labels, typed error codes, figma_import event (#1979)
Closes the observability gaps on the figma integration: - withFigmaErrors takes a command label (figma:asset|tokens|component) and reports the failure inline before its process.exit — the top-level trackCommandFailures wrapper never sees self-exiting commands, so typed codes (NO_TOKEN, BAD_TOKEN, FORBIDDEN, RATE_LIMITED) were invisible. FigmaClientError codes surface as the error name for dashboarding the first-run funnel (NO_TOKEN -> later success = onboarding conversion). - new figma_import event per import: phase, duration, reused (dedup effectiveness), tokens variables-vs-styles mode + entry count (Enterprise gating rate), unresolved-binding + rasterized-node counts (fidelity degradation). No fileKeys, node ids, names, or descriptions. - /figma skill fires the events beacon (figma-motion / figma-shaders / figma-storyboard) for the MCP phases that never touch the CLI. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
James RussoandClaude Opus 4.8
|
232d591479 |
feat(cli/telemetry): surface unrecognized agents in the agent_runtime=null bucket (#1978)
* feat(cli/telemetry): surface unrecognized agents in the agent_runtime=null bucket agent_runtime is a closed allowlist: an agent we have no rule for collapses to null with no trace of what it was, so ~18% of CLI users are unattributable and new agents stay invisible until reverse-engineered by hand. Add detectAgentHints(), a self-populating residual signal computed only for the null bucket (gated off classified events): - agent_hint: value of AGENT / AI_AGENT (the emerging self-identification convention; Crush and Goose set AGENT=<name>) — names agents the allowlist misses. - term_program: raw TERM_PROGRAM (editor name) — catches the IDE-terminal class the same way the cursor/windsurf rules do. - agent_env_hints: sorted, comma-joined "agent-ish" env-var KEY names present but matched by no vendor rule — a fingerprint that clusters by agent. Privacy stays consistent with the existing "never read secret-shaped values" stance: agent_env_hints emits key names only; the three value-reads are vars whose sole purpose is non-secret identification, each passed through a strict short-slug allowlist so anything long/spaced/secret-shaped is dropped. Breaking down agent_hint / agent_env_hints filtered to agent_runtime IS NULL AND is_tty=false gives a ranked leaderboard of new agents to promote into VENDOR_RULES. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli/telemetry): guard agent_hint/term_program against short credential-shaped values Review feedback (Magi, #1978): the short-slug allowlist in sanitizeHint() still accepted short credential-shaped values (AGENT=sk-ant-api03, AGENT=AKIAIOSFODNN7EXAMPLE, AGENT=github_pat_abc), so the "never emit a secret" claim wasn't actually enforced — only overlong values were dropped. Add a credential-shape guard on top of the slug allowlist: - known token/credential prefixes (sk-, ghp_, github_pat_, akia, ya29, ...) - any unbroken alphanumeric run >= 16 chars (key bodies, hex, base64-ish), while agent names segment on _/-/. and keep each run short. Replace the single overlong-value test with the short credential shapes from the review (parametrized) plus a positive case (gemini_managed_agent) proving real multi-segment names still pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
James RussoandClaude Opus 4.8
|
9b41891f3a |
feat(cli): emit render_preflight_rejected telemetry for P1-3 pre-flight saves (#1856)
The P1-3 aspect/alpha/HDR pre-flight (#1843) aborts an incompatible render before any browser/ffmpeg work, but that "save" was invisible on dashboard 1783183 — indistinguishable from a deep failure or a user giving up. checkRenderResolutionPreflight now returns { message, kind } (kind = the existing low-cardinality OutputResolutionIssueKind), and the render command emits render_preflight_rejected { kind } before exiting. No parsers change — the helper already carried kind. trackRenderPreflightRejected is typed to the union so the metric can't carry free text. Tests: preflight tests assert kind for all five kinds; an events test locks the emit. Further follow-up (still log-only): encoder-frame-0-exit counter and a P1-4 doctor cli_env_check event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
James RussoandClaude Opus 4.8
|
733f88cb1f |
feat(producer,cli): render-reliability telemetry counters for capture hardening (#1850)
Follow-up to the render-reliability batch (#1841/#1842/#1843). Threads two capture-reliability counters through the existing observability → CLI-telemetry pipeline (no new PostHog wiring) so #1842's hardening is measurable on dashboard 1783183: - transient-retry burn (CaptureAttemptSummary.reason gains "transient-retry"; counted into RenderCaptureObservability.transientRetries on BOTH the recovered and the still-failed paths via a shared helper). - OOM classification (memoryExhaustionDetected set when describeMemoryExhaustion classifies the failure). Surfaced as capture_transient_retries + capture_memory_exhaustion_detected render-event props. Tests cover the attempt tagging and the payload mapping. Further follow-up (different subsystems): encoder-frame-0-exit signal, and P1-3 pre-flight-rejection / P1-4 cli_env_check counters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
23adfdc496 |
fix(cli): skip AI skills install when git is unavailable (#1803)
* fix(cli): skip AI skills install when git is unavailable init and `skills update` route through installAllSkills, which shells out to `npx skills add`. That CLI clones the repo with git, so on a machine without git the clone aborts mid-run and dumps a noisy multi-line `spawn git ENOENT` / "Installation failed" / "Canceled" block. init still exited 0 and scaffolded the project, but the output read like a hard failure (and surfaced as exit 1 on some platforms). Detect git up front alongside the existing npx check via a small table-driven preflight: best-effort callers (init) print one calm line and continue; strict callers (`skills update`) throw so the check-or-update recovery contract still fails loudly. The skills freshness check already degrades gracefully without git, so the happy path is unchanged. * feat(cli): record a diagnostic event when a skills install is skipped for a missing prerequisite When init's best-effort skills install bails because git (or npx) is absent from PATH, the skip was silent, so the rare boxes that hit it (fresh Windows without git) were invisible. Emit one low-cardinality event (reason: git_missing / npx_missing) on the best-effort skip path only, never on the happy path or the strict throw. Reuses the existing typed-event pattern, and trackEvent's opt-out gate already applies. |
||
|
Miguel Ángel
|
db61509ddc |
fix(cli): omit render duration when feedback command has none (#1797)
The standalone `feedback` command runs separately from `render`, so it has no access to the prior render's elapsed time, yet it always passed renderDurationMs: 0 to the feedback analytics event. Since that path is the one used in practice (the auto-prompt returns early for agent and non-interactive runtimes), nearly every feedback event recorded a render duration of exactly 0, which is misleading rather than absent. Make renderDurationMs optional and only include render_duration_ms in the event when a real value is supplied. The standalone command no longer passes a duration; the auto-prompt path still forwards the real elapsed time. |
||
|
WaterrrForeverandClaude Opus 4.8
|
d70ee134cc |
feat(cli): add skills version check, update, and freshness manifest (#1738)
* feat(cli): add skills version check, update, and freshness manifest
Give the HyperFrames skill bundle a content fingerprint so agents and
users can tell whether installed skills are the latest version, on any
platform that can run the CLI.
- skills-manifest.json (repo root): per-skill sha256 over the whole skill
directory; minimal {source, skills}, no version/timestamp so it is fully
deterministic. Generated by scripts/gen-skills-manifest.ts.
- `hyperframes skills check` [--json]: compares installed skills to the
manifest; exits non-zero when something is outdated (agent/CI gate).
- `hyperframes skills update`: thin wrapper over `npx skills update`.
- Passive nudge on render/lint/validate when skills are stale (24h cache,
same opt-out as the CLI self-update notice).
- "latest" resolved via `git ls-remote` + SHA-pinned raw URL to dodge
GitHub raw-CDN lag, falling back to the main branch URL.
- CI job + lefthook hook keep skills-manifest.json in sync with skills/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): add execFile to child_process mock in skills test
skills.test.ts mocks node:child_process but only declared execFileSync
and spawn. Loading skills.js transitively loads skillsManifest.ts, which
runs promisify(execFile) at module load, so vitest threw on the missing
execFile named export. Add a bare stub — these tests never invoke it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): init installs all skills; skills update pulls the full set
Make `hyperframes init` the single place skills are pulled in full, and
make "update" mean "get everything" rather than "refresh what's there".
- init now always installs/refreshes ALL skills (incl. ones not yet
present) instead of prompting "Install AI coding skills?" — opt out
with `init --skip-skills`. Both the interactive and non-interactive
paths pass `--all --yes` so the complete set is fetched.
- `hyperframes skills update` switches from `npx skills update` (which
only refreshes already-installed skills) to `skills add --all`, so it
installs missing skills too — the same install step init runs.
- SKILL.md documents init-installs-all and the new update semantics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): skills check treats missing skills as needing an update
The full skill set is now the goal (init and `skills update` both pull
all, including ones not installed), so a partial install is no longer
"a choice" — it's something to fix.
- diffSkills: updateAvailable is now true when anything is outdated OR
missing (local-only still doesn't count). So `skills check` exits
non-zero — and renders "Update:" instead of "up to date" — whenever a
skill is missing, not just when one is stale.
- The passive render/lint/validate nudge follows suit: it now counts
missing alongside outdated ("N skills out of date or missing"),
tracked via a new skillsMissingCount cache field.
- SKILL.md documents the stricter check.
Note: platforms that intentionally vendor only a subset of skills (e.g.
a Codex snapshot) will now see check report non-zero.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): install/update skills straight from the GitHub repo
`skills add owner/repo` can resolve through the skills.sh registry, which
lags behind the repo — so `update` could install a stale version while
`check` (which resolves latest directly from GitHub) keeps reporting
"outdated", an endless loop.
Switch the install source to the full GitHub URL
(https://github.com/heygen-com/hyperframes), which makes `skills add`
git-clone the repo directly at latest main, bypassing the registry. This
covers `hyperframes skills`, `hyperframes skills update`, and `init`'s
skill install — all of which go through SOURCES. Now install/update and
check agree on what "latest" means.
The init "install skills" hint now points at `npx hyperframes skills
update` so the manual path uses the same GitHub-direct fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): init checks skills against GitHub, installs only when stale
`hyperframes init` now runs the skills version check first and only
(re)installs when something is outdated or missing — instead of
unconditionally re-pulling every time. Re-running init on an
already-current project is now a no-op ("skills are already up to date").
- New ensureSkillsCurrent() helper, shared by both the interactive and
non-interactive init paths (no duplicated install logic).
- The check resolves "latest" straight from GitHub (same source the
install uses); best-effort — if it can't reach GitHub it installs anyway.
- SKILL.md updated to describe the check-then-install behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cli): address skills manifest review feedback
From the PR review (points 1, 2, 4, 5):
1. Remove the `local-only` skill status. checkSkills only ever hashes
manifest-listed skills, so a local-only status could never appear in
the end-to-end output — and making it appear would wrongly flag
unrelated skills (the `.../skills` dir is shared across sources).
diffSkills now reports only on manifest skills; skills on disk that
aren't in the manifest are ignored.
2. Drop the redundant per-directory sort in listFilesSorted — the single
final out.sort() is what guarantees a deterministic hash (verified:
manifest unchanged).
4. resolveLatestManifest local-path detection now uses path.isAbsolute,
so Windows absolute paths (C:\...) are treated as local instead of
falling through to a remote fetch.
5. fetchManifest validates the response shape (asSkillsManifest) instead
of a blind `as` cast, so a CDN error page served as 200 fails with a
clear error rather than a cryptic crash later in diffSkills.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): strict skills update + auto-discover any agent host
Address PR review (Magi blocker + James/Rames robustness):
- Blocker (Magi): `skills update` is the documented recovery path for
`skills check || skills update`, but it delegated to installAllSkills()
which swallowed missing-npx and failed `skills add` as "skipped",
exiting 0 even when nothing changed. Add a strict mode that throws on
failure; update sets a non-zero exit (init stays best-effort). New tests
simulate a non-zero `skills add` (exit 1) and the success path.
- Robustness (James/Rames #2): the upstream `skills` CLI installs into
~72 agent conventions; a hard-coded list (4, or even 11) can't track
that. Replace defaultSkillRoots with discoverSkillRoots — it scans cwd +
$HOME for any `<host>/skills/<manifest-skill>/SKILL.md` (plus the XDG
`.config/<host>/skills`), so detection is structural and future-proof,
no closed list. agentFromDir infers the host from the path.
- Tests (Rames #3): temp-fixture detection tests for every convention ×
{project, global}, scope priority, claude-code preference, the
no-install case, the --dir override, and an unknown/new host (proving
the no-closed-list property).
- Docs (Rames #4/#5): SKILL.md notes init's best-effort GitHub round-trip;
findRepoManifest climbs 16 levels (was 8) for deep monorepos.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): resolve CodeQL file-system race + de-flake Windows npx test
Two CI fixes:
- CodeQL (high, js/file-system-race) at gen-skills-manifest.ts: the
existsSync(outPath) precheck followed by writeFileSync(outPath) is a
check-then-write race. Read the committed manifest directly in a
try/catch instead (missing/unreadable ⇒ "no committed manifest"), so
there's no precheck to race against. Behavior is unchanged.
- Windows Tests: npxCommand.test.ts's real `npx --version` smoke test
cold-starts slower than vitest's 5s default on Windows runners and
timed out. Give the test 60s headroom (and a 30s exec timeout). Kept
as a real execution check — mocking would reduce it to a tautology.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): repair garbled npx smoke-test timeout comment
The explanatory comment for the 60s timeout was scrambled across the
callback/timeout arguments, failing oxfmt --check (and thus preflight,
which in turn skipped preview-parity and failed the regression gate).
Move it above the it() call so it no longer sits between call arguments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
kiritowooandkiritowoo
|
5242dde2dc |
feat(telemetry): attribute renders to the authoring workflow skill (#1695)
* feat(telemetry): attribute renders to the authoring workflow skill Add an optional `--skill` flag to `hyperframes render` and tag the `render_complete` / `render_error` events with `authoring_skill`, so render usage can be broken down per authoring workflow. The value is slug-gated (a malformed value is ignored) and the existing anonymous / opt-out telemetry pipeline is otherwise unchanged. Each end-user workflow that renders now passes `--skill=<name>` on its render command: embedded-captions, faceless-explainer, graphic-overlays, motion-graphics, music-to-video, pr-to-video, product-launch-video, remotion-to-hyperframes, website-to-video. Not instrumented, by design: general-video renders freeform with no canonical render command to attach to, and slideshow produces an interactive deck rather than a rendered video. Both can follow up if per-skill numbers are wanted. * fix(telemetry): address review — shared slug util, equals-form flag, invalid-value warning - Extract the SKILL_SLUG regex + a normalizeSkillSlug() helper into telemetry/skill.ts, shared by the `events` and `render` commands (the regex was duplicated). `render` adopts normalizeSkillSlug (so it now trims the value, matching `events`); `events` references the shared SKILL_SLUG. + unit test. - `render` warns on a non-empty but invalid --skill value (e.g. a camelCase typo) so attribution isn't silently lost — stderr only, never fails the render. - embedded-captions render script: `--skill embedded-captions` -> `--skill=embedded-captions`. On an older CLI that does not declare --skill, the space form leaks the value as a positional and clobbers the project dir (resolveProject fails); the equals form is parsed as a self-delimiting flag and safely ignored. Verified via Node parseArgs(strict:false). Addresses review feedback on the PR (shared util + .trim drift, version-skew safety, invalid-value visibility). --------- Co-authored-by: kiritowoo <295860553+kiritowoo@users.noreply.github.com> |
||
|
Miguel Ángel
|
4976b5e036 | fix(producer): split capture average timing (#1657) | ||
|
Miguel Ángel
|
f12754f5ad |
fix(cli): classify missing whisper-cpp as a setup gap, not a command error (#1628)
transcribe hard-failed with cli_error whenever whisper-cpp was absent. On Linux/Docker/CI (no Homebrew, no compiler toolchain) that is unavoidable, so it drove ~30k cli_error/day that are really "install the prerequisite" rather than bugs — and buried genuine transcription failures in the command-error budget. ensureWhisper now throws a typed WhisperUnavailableError when no binary exists and none can be built. The transcribe command reports that on a dedicated transcribe_unavailable metric instead of cli_error, and a new --optional flag lets pipelines skip captions and exit 0. Real transcription crashes still fail as cli_error. init and the skill pipelines already continue without captions. Also removes a stale doc reference to a `transcribe --provider groq` flag that does not exist. |
||
|
Vance IngallsandClaude Opus 4.8
|
7310223b66 |
feat(engine): static-frame dedup default-on + render telemetry (#1549)
* feat(engine): static-frame dedup for screenshot capture (opt-in) Skip re-seeking + re-screenshotting frames byte-identical to their predecessor. A frame is dedupable iff no GSAP tween or clip cut is active in it or its predecessor (predicted from window.__timelines + clip schedule) AND an empirical anchor-compare confirms it. Opt-in HF_STATIC_DEDUP=true, default off. Correctness (designed for the multi-worker / distributed render paths): - Reuse is keyed by the ABSOLUTE composition frame (derived from the frame's time), NOT the captureFrameCore frameIndex arg — chunked/parallel callers pass a chunk- relative index. Validated lossless (PSNR=inf) on both single- and multi-worker renders of a static-hold comp. - verifyStaticFramesSafe checks EVERY run (no longest-first budget truncation that left runs armed-but-unverified), and samples each run's FIRST reused frame, its END, and interior points at a stride; a hard cap disables dedup rather than trust an unverified set. - Conservative arming: skipped when capture mode != screenshot (BeginFrame tick semantics + the verifier's screenshot path wouldn't transfer), when a before-capture hook is set (per-frame video injection), when page-side compositing is active (shader / drawElement composite the plain verification screenshot can't reproduce), and when any data-start is a non-numeric reference expression the clip-boundary parser can't protect, or duration is unknown/zero. - Session reuse (prepareCaptureSessionForReuse) resets lastFrameBuffer + dedup counter so a probe/prior-render buffer can't bleed into the first static frame; the armed set is kept (same-composition reuse). Cost calibration bypasses dedup for its sparse, non-contiguous sample sweep, then restores the armed set. - HF_STATIC_DEDUP_SAMPLES is NaN-guarded. Disqualifies on signals the GSAP predictor can't see: video, canvas/webgl, zero tweens, running CSS/WAAPI animation. Pays on static-hold content (title cards, slideshow/kiosk loops, data-viz pauses); no-op on continuously-animated comps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(engine): static-frame dedup default-on + render telemetry Flip dedup from opt-in (HF_STATIC_DEDUP=true) to default-on (opt-out HF_STATIC_DEDUP=false). Verification (verifyStaticFramesSafe) is the safety net that keeps reuse sound at scale. Add end-to-end dedup observability. The capture session records enabled / armed / skipReason / predicted; these surface via CapturePerfSummary -> a dedupPerfs accumulator (disk sequential + parallel AND streaming sequential + parallel) -> aggregated into RenderPerfSummary.staticDedup (OR armed, SUM frames across workers) -> render_complete props static_dedup_{enabled,armed,skip_reason, predicted_frames,reused_frames}. skip_reason is a low-cardinality code: capture_mode | video_injection | page_composite | ineligible | verification_failed. Distributed chunks run on Linux/beginframe where dedup never arms, so they pass a throwaway dedupPerfs sink (no per-chunk reporting). Tests: aggregation logic (OR/SUM/skip-reason) + opt-out passthrough. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(engine): address review on dedup default-on + telemetry Review feedback (miga-heygen) + self-review fixes: - Retry double-count: executeDiskCaptureWithAdaptiveRetry pushed worker dedup perf inside the retry loop, so an adaptive retry counted frames twice (reused/predicted could exceed totalFrames). Reset dedupPerfs at the start of each attempt — retry now REPLACES rather than accumulates; common no-retry path is unchanged. - Opt-out parsing: HF_STATIC_DEDUP now disables on {false,0,off} case/space-insensitive (was strict !== "false", so `False`/`0` silently kept dedup on — the kill-switch could no-op). - Verification budget vs drift: verifyStaticFramesSafe returns {badFrame, budgetExhausted}; armStaticDedup reports a distinct `verification_budget` skip reason so a telemetry spike means "raise HF_STATIC_DEDUP_SAMPLES", not "compositions are non-static". - Index idiom: captureFrameCore now uses Math.floor(time*fps + 1e-9) (matches quantizeTimeToFrame) so the dedup lookup agrees with the frame the seek lands on even for non-exact times. - Stale "opt-in HF_STATIC_DEDUP=true" comments -> "opt-out HF_STATIC_DEDUP=false" across frameCapture.ts + types.ts. - Extract pushWorkerDedupPerfs helper (perfSummary.ts), used by the disk and streaming parallel paths — removes the duplicated push loop and drops captureStreamingStage back under the complexity threshold. - dedupPerfs is now required (not optional) on executeDiskCaptureWithAdaptiveRetry — a missing arg silently dropped telemetry. - Test: captureStreamingStage createInput() now provides the required dedupPerfs field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(engine): address deferred dedup-review items - Derivable state: drop session.staticDedupArmed/staticDedupPredicted; derive both from session.staticFrames in getCapturePerfSummary (armed ⟺ non-empty set, predicted === size) so they can't desync. - Config altitude: HF_STATIC_DEDUP now resolves into EngineConfig.staticFrameDedup (resolveConfig, opt-out on {false,0,off}), alongside forceScreenshot/browserGpuMode — armStaticDedup reads config instead of process.env. Default-on preserved (missing config → enabled). - Lossy aggregation: aggregateDedup now reports DISTINCT skip reasons (sorted, `|`-joined) across diverging unarmed workers instead of just the first. - discardWarmupCapture: also snapshot/restore staticDedupCount and lastFrameBuffer so a warmup capture can't leak a phantom reuse or a stale buffer anchor into the real summary. - Convention: perfSummary-dedup.test builds its job via createRenderJob instead of `as unknown as RenderJob`. - Docs: verification_budget added to skip-reason lists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
121cdd2d9f |
fix(telemetry): attribute studio renders to the browser user (joinable funnel) (#1492)
Studio-triggered renders emit render_complete / render_error from the CLI preview-server process, which stamps every event with the install's anonymousId (client.ts drainQueueToPayload). The browser, meanwhile, fires studio_session_start / studio_render_start under its own getAnonymousId(). So the render outcome and the render start never share a person_id — verified in data: of 15,125 users who started a studio render in 30d, ZERO have any render_complete under any source, and the 898 studio-tagged completers are disjoint server UUIDs. The studio render funnel — the product's core value moment and strongest retention signal — is therefore unmeasurable. Thread the browser's telemetry id through to the render-outcome events: - client.ts: trackEvent takes an optional distinctId; drainQueueToPayload uses `event.distinctId ?? config.anonymousId`. CLI renders unchanged. - events.ts: trackRenderComplete/trackRenderError forward an optional distinctId. - studioRenderTelemetry.ts: emitStudioRender* pass opts.distinctId through. - core studio-api (types.ts + routes/render.ts): the render route reads `telemetryDistinctId` from the request body (validated string) and passes it to the adapter's startRender, which already forwards opts to the emitters. - studio (useRenderQueue.ts): include getAnonymousId() as telemetryDistinctId in the render POST — the same id studio_* events already use. Result: studio render_complete/error now carry the browser user's id and join studio_session_start / studio_render_start. Older clients that don't send the field fall back to anonymousId (no regression). No new tracking surface — it's the existing anonymous studio id. Tests: per-event override forwarding (events), studio render distinctId threading + older-client fallback (studioRenderTelemetry), and route body → adapter forwarding incl. non-string rejection (core render route). |
||
|
Miguel Ángel
|
5f6ced116d |
feat(cli): report command failure reasons to telemetry (de-blind browser/info) (#1484)
Observability showed `browser` (~75% fail, ~1.3k users/day) and `info` (~60% fail) failing at high rates with no captured reason — only `cli_command_result success=false`. citty's `runMain` catches a command's thrown error and `process.exit(1)`s without re-throwing, so a thrown failure never reached the existing `cli_error` telemetry (which only fired from the uncaughtException / unhandledRejection handlers). Wrap every command's `run()` at the dispatch boundary (cli.ts) so a thrown failure reports its reason via `cli_error` (kind=command_error) before being re-thrown unchanged — citty's print + exit-1 behavior is preserved. This de-blinds every throw-style command at once: `browser ensure` (Chrome download), `tts`, `inspect`, `render`, etc. Paths that bypass the wrapper are handled inline: - `browser` self-exits (`path` download failure, unknown subcommand) — report inline; the ARM64 `ensure` branch previously swallowed a failed install and returned success, now reports and exits 1. - `resolveProject()` self-exits on InvalidProjectError (the dominant `info` failure — run outside a project) — report inline before exit. Hardening: - PII: `trackCliError` now redacts error_message + stack_trace via redactTelemetryString (matching render_* events) — CLI errors and stacks carry absolute install paths / cache dirs / user args. - Race: the wrapper awaits an on-demand telemetry import before re-throwing, so a command that fails before the lazy telemetry import settles still reports (a telemetry failure is swallowed and never masks the real error). Pure helpers in utils/command-failure-tracking.ts with unit tests for the throw / success / no-run / onFailure-rejection cases, the reporter wiring, and trackCliError redaction. CommandDef<any> mirrors citty's SubCommandsDef. Known scope: commands that print + `process.exit(1)` on their own validation paths (tts/validate/lint argument errors) remain wrapper-blind — follow-up. |
||
|
Miguel Ángel
|
40b49d4024 |
fix(cli): report transcribe failure reasons via cli_error (command_error) (#1479)
Transcribe failures are recorded as `cli_command_result success=false` but
without a reason: the command catches its own error, prints it, and
`process.exit(1)` — the message never reaches telemetry. `cli_error` was only
emitted from the uncaughtException / unhandledRejection handlers, so
self-handled command failures were invisible. That makes a high failure rate
countable but not debuggable.
Add `trackCommandFailure(command, err)` — a thin wrapper over the existing
`trackCliError({ kind: "command_error" })` that normalizes an unknown reason to
name/message/stack. It enqueues synchronously, so the process `exit` handler's
flushSync ships it alongside `cli_command_result`. Respects the telemetry
opt-out (gated in trackEvent) and reuses the existing PII redaction.
Wire it into all three of transcribe's failure exits (file-not-found,
empty-transcript import, and the transcribe() catch — ffmpeg / whisper-binary /
model-download errors). Now each failure carries its reason, so we can see how
much of the failure rate is environment vs user input.
The helper is generic — the same one-liner can be dropped into other commands'
failure paths, or centralized at the runMain boundary, as a follow-up.
|
||
|
Miguel Ángel
|
2ce5b421f1 |
fix(engine): respect cgroup memory limits in low-memory detection (#1373)
getSystemTotalMb returned os.totalmem() — the host's physical RAM — so a 4GB Docker container on a 32GB host never auto-flagged as low-memory and the low-memory render profile didn't activate exactly where it's needed most. Read the cgroup v2 limit (/sys/fs/cgroup/memory.max, with the v1 fallback and its no-limit sentinel handled) and use min(host, cgroup). The probe is best-effort and non-Linux platforms never touch /sys. Review follow-ups: worker sizing (calculateOptimalWorkers) and the getSystemResources diagnostics previously read os.totalmem() directly and now use getSystemTotalMb(), so container limits actually govern parallel spawn decisions; CLI telemetry reports the effective total as well. The cgroup probe result is cached for the process lifetime (the limit is immutable per process) with a test reset hook; a detected limit logs once so operators can see which source governs, and a present-but-unreadable cgroup file warns once instead of failing silently — absence stays silent. The root-path-vs-/proc/self/cgroup trade-off is documented at the path constants. cli/tsconfig.json gains the gcp-cloud-run/sdk source alias (matching the existing producer and aws-lambda entries) so the cli typecheck resolves from source in a fresh checkout. Refs #1193, #1194, #1195, #1236 |
||
|
James RussoandClaude Fable 5
|
30fcede44e |
refactor(cli): restore exact-match Cursor rule (revert unsourced loosening) (#1334)
Follow-up to #1328. That PR loosened the Cursor TERM_PROGRAM check from exact `=== "cursor"` to `?.toLowerCase() === "cursor"` "for parity with Windsurf" — but the parity is false. Windsurf is matched case-insensitively because its sources genuinely disagree on casing ("windsurf" vs "Windsurf"); Cursor consistently emits lowercase "cursor", so nothing justified loosening an existing, working, exact-match rule. Per review feedback on #1328 (Magi/Hermes), revert Cursor to exact match and drop the TERM_PROGRAM=Cursor test. Windsurf stays case-insensitive (sourced); its comment now documents the asymmetry as intentional. No functional change — Cursor always emitted lowercase, so detection is unchanged; this just removes an unsourced false-positive surface. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
James RussoandClaude Fable 5
|
e0ecd4d2d1 |
feat(cli): detect Windsurf, Cline, Gemini CLI, and Crush agents (#1328)
Rebased onto main after #1294 merged. Adds four coding-agent vendors to detectAgentRuntime() (existence-only checks, source/runtime-verified): - windsurf — TERM_PROGRAM=windsurf (case-insensitive) - cline — CLINE_ACTIVE (default vscode-terminal path) - gemini_cli — GEMINI_CLI (runtime-confirmed; distinct from the managed-agent /.agents/ detector, which runs ahead of VENDOR_RULES and wins when both match) - crush — CRUSH (runtime-confirmed) Also makes the cursor rule case-insensitive for parity with windsurf, and adds a code-resident "deliberately NOT added" section (OpenHands/Aider/Goose/ opencode/Roo/Amp/Devin/Jules/Factory) carrying the empirical rejection rationale. Test isolation: the Gemini managed-agent suite now clears its node:os/node:fs doMock registrations in afterEach so they don't leak into the env-var-only suites that follow it in the same file. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
James RussoandClaude Fable 5
|
0766eb8144 |
feat(cli): detect Gemini managed-agent sandbox in detectAgentRuntime (#1294)
* feat(cli): detect Gemini managed-agent sandbox in detectAgentRuntime
Add `gemini_managed_agent` to the AgentRuntime union and a dedicated
isGeminiManagedAgent() detector. Empirical signal pair (from live-sandbox
introspection by gemini-agent, env_id b9db4e56, 2026-06-09):
existsSync('/.agents/AGENTS.md') AND isGVisor()
The conjunction is what makes the rule safe:
- `/.agents/AGENTS.md` excludes generic gVisor surfaces (GKE Sandbox,
Cloud Run gen2) that don't mount the managed-agent layout.
- The gVisor kernel check excludes a dev box that happens to have a
stray `/.agents/` directory.
Implementation notes:
- Filesystem-based check runs ahead of the env-var-only VENDOR_RULES
loop. VENDOR_RULES is documented as "Only checks for the EXISTENCE
of well-known env vars — never reads their values"; the Gemini
signal is filesystem + kernel, not env, so it gets a dedicated
branch rather than shoehorning into the rule list.
- GEMINI_API_KEY is deliberately NOT keyed on — it's user-settable on
any host. The filesystem + kernel pair is the actually-distinctive
signal.
- Reuses the existing isGVisor() helper for the kernel half of the
conjunction; no duplication.
Tests (4 new, vitest):
- Positive: /.agents/AGENTS.md + 4.19.0-gvisor → gemini_managed_agent
- Negative: gVisor alone (no /.agents/) → null (generic gVisor surface)
- Negative: /.agents/AGENTS.md alone (no gVisor) → null (dev box false-positive guard)
- Precedence: Gemini signal wins over a coincident CLAUDECODE env var
Empirical caveat: signal was gathered from a single sandbox. Re-confirming
across additional sandbox spins is a follow-up; the rule is conservative
enough (conjunction of two independent signals) that a single-spin
false-positive is unlikely, but a single-spin variance bug (e.g. some
sandbox flavors omitting one of the two markers) would surface as
under-detection rather than over-detection.
Source for signals: introspection write-up at
/tmp/gemini-sandbox-detection-signals.md (gemini-agent, 2026-06-09).
* docs(cli): reframe Gemini-managed-agent detection rationale (load-bearing vs guard)
gemini-agent's uniqueness analysis (FS-root + cgroup + netns + DMI + PID-1
introspection of env d59d6361, 2026-06-09) revealed the two signals are
NOT co-equal:
- /.agents/AGENTS.md is the uniqueness anchor — definitionally a
managed-agent artifact, injected per-run by the platform, mtime
tracks the interaction. Nothing in the generic Google-Cloud-on-gVisor
universe (Cloud Run gen2, GKE Sandbox, Fly.io) mounts /.agents/.
- isGVisor() is a guard, not a second uniqueness signal. gVisor itself
is shared with GKE Sandbox + Cloud Run gen2 — its real job here is
ruling out a stray user-created /.agents/AGENTS.md on a non-sandbox
host.
The original 3-spin work proved *stability* (signals consistent across
sandbox spins). This pass adds *uniqueness* — confirming the signals
discriminate Antigravity from the broader gVisor universe, not just
that they're reliably present. Stability ≠ uniqueness; both are
required for a correct detection rule.
Code unchanged (the AND-gate is sound). Docstring reframed so a future
reader doesn't mistake the conjunction for two independent uniqueness
signals. Also enumerated the markers NOT keyed on (with reasons), so
future contributors don't reach for them by naming inference.
Source: gemini-agent uniqueness analysis write-up.
* fix(cli): key Gemini managed-agent detection on /.agents/ mount, not optional AGENTS.md
The detector keyed on existsSync('/.agents/AGENTS.md'), but Google's Managed
Agents docs are explicit that AGENTS.md is OPTIONAL: an agent may declare its
instructions inline via system_instruction in agent.yaml and ship no AGENTS.md
file ("system_instruction and AGENTS.md are additive; both apply when present").
The platform auto-discovers the agent under the /.agents/ directory; skills
mount at /.agents/skills/ and AGENTS.md at /.agents/AGENTS.md only when shipped.
Keying on the file generalized only to templates that happen to bundle an
AGENTS.md (like HeyGen's own gemini-agent and Thor's reference). A managed agent
defined with inline instructions or a skills-only definition was a silent
false-negative. All three prior verification spins used our own AGENTS.md-bearing
template, so the gap was never exercised.
Broaden to the /.agents/ directory mount (still gVisor-guarded — false-positive
surface is unchanged) so skills-only and inline-instruction agents are detected.
Adds a regression test for the skills-but-no-AGENTS.md case. Documents the one
residual gap (pure inline-only, no skills/no AGENTS.md) that needs an empirical
spin to confirm.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(cli): tighten /.agents/ to a directory check + sync agent_runtime docs
Self-review follow-ups (no behavior change for real managed agents):
- isGeminiManagedAgent now requires statSync("/.agents").isDirectory() rather
than existsSync("/.agents"), matching the documented "directory mount"
contract. existsSync matched any entry (a stray file/symlink named /.agents),
widening the gVisor-gated false-positive surface beyond what the comment
claimed. Tests now mock statSync accordingly (and drop a dead /.agents/skills
mock clause the code never read).
- system.ts: the agent_runtime doc comment hard-coded the vendor list and said
"detected by env-var existence only" — both stale once a filesystem/kernel
detector (gemini_managed_agent) exists. Point at the AgentRuntime union and
note the filesystem-marker case instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|