* fix(producer): render -c <scene> uses the scene's own duration, not the project's
When rendering a single sub-composition standalone (`hyperframes render -c
compositions/scene.html`), the producer extracts the scene's mount from
index.html and wraps it in a shallow clone of the master root. That clone
kept the master's `data-duration`, so the standalone composition advertised
the whole project's length instead of the scene's own: a 2s scene rendered
for the full 12s project, and a master that derives its length from sibling
mounts (now removed) produced "Composition has zero duration".
Re-point the extracted wrapper's `data-duration` at the scene's own, read
from the scene file's `<template>` root (the source of truth for that scene),
with a fallback to the mount's `data-duration`. Full-project renders are
unaffected — they never take the extraction branch.
Verified end-to-end via the pre-capture duration gate: a 2s scene now resolves
to 2s and a 10s scene to 10s (both were 12s), while the full index render stays
at 12s. Adds unit coverage for both the scene-file and mount-fallback paths.
* test(producer): distinguish scene and mount durations
Addresses non-blocking review feedback on #2256: the previous shape used
err.message.includes(...) inside a catch to distinguish a real headroom
violation from a statfsSync failure — a future message tweak would silently
fail open (statfs-unsupported and headroom-violation would take the same
code path). Now statfsSync failure returns early (skip the gate) and the
headroom check/throw happens in plain control flow outside any try/catch.
## What
Two fixes from adversarial testing of the DE parallel router (10 hostile comps, routed vs screenshot-baseline PSNR). The router itself held — both bugs are in general drawElement fast capture, and one slipped past self-verify.
### 1. Compile gate: ancestor background-image (`producer`)
`drawElementService`'s per-frame ancestor fill replicates what lies behind the captured subtree by walking up the DOM for the nearest non-transparent **`backgroundColor`**. A background-**image** (`linear-gradient`, `url()`) on `body`/`html`/a wrapper reads as transparent in that scan, so a deeper ancestor's solid color paints instead wherever the subtree leaves pixels uncovered.
Measured repro: body `linear-gradient` + html solid color + an element shrinking late in the comp → DE paints the html purple instead of the body gradient. 30.9 dB min frame vs baseline, visually unmistakable. Identical damage single-worker and parallel — general DE bug, in every wild DE render matching this (very common) authoring pattern.
Fix: `detectAncestorBackgroundImage()` in the compiler (DOM-aware — inline styles on the root's ancestor chain + `<style>` rules resolved via `querySelectorAll`, so class-selected wrappers are covered; backgrounds *inside* the root are deliberately not matched). New compile gate `ancestor_background_image`, same shape as the 3D/mix-blend gates, bypass `HF_FAST_CAPTURE_ANCESTOR_BG=true`.
### 2. Self-verify tail sample (`engine`)
The verify grid sampled at `(i+1)/(k+1)` → [20/40/60/80]% of the timeline. The damage above starts at ~79% and peaks after the last sample — verification **passed** on output that bottomed at 30.9 dB (threshold 32 dB would have caught it, it just never looked there).
Fix: `computeDeVerifySampleFractions()` — first k−1 samples evenly spaced, last pinned at 95%. Default k=4 grid becomes [25/50/75/95]%. Kills the whole late-onset damage class, not just this repro.
## Validation
- Repro comp (body gradient + shrink reveal): now gates → baseline route, 54.7 dB avg vs ground truth (was 41.6 avg / 30.9 min with the purple surround)
- Control comp (nested stacked fades, routed): still routes, verify grid `[90, 180, 270, 342] of 360`, passes, 60.6 dB avg — unchanged
- Full adversarial matrix context: 6/10 comps routed clean (49–68 dB min), blend/3D gated correctly, animated-canvas damage caught by verify at 16.5 dB with clean revert, video comps route legitimately (frames pre-extracted)
- Tests: 7 new detection cases (`htmlCompiler.test.ts`), 5 new grid cases (`frameCapture-verifySampleFractions.test.ts`); `compileStage.test.ts` + `frameCapture.test.ts` suites green
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* fix(hooks): scope pre-commit build check to the actual target repo
The PreToolUse hook matched any Bash command containing "git commit" and
always ran this repo's bun build/lint/typecheck from its own process cwd,
even when the command targeted a different repo (e.g. a sibling worktree
reached via a leading `cd`). Resolve the real target directory from the
command text first, and skip silently for any repo that isn't this one.
* fix(lefthook): force-add already-tracked files under gitignored paths
The format hook's auto-restage (`git add {staged_files}`) exits non-zero
for any staged file that lives under a gitignored directory (e.g.
.claude/settings.json, tracked despite .claude/ being ignored for worktree
noise), silently aborting the whole commit even though the file was
already correctly staged.
* fix(producer): repoint stale puppeteer symlinks to the pinned 25.x install
packages/producer's tracked node_modules symlinks still pointed at
puppeteer@24.43.1, which no longer exists after a fresh install resolves
package.json's ^25.2.1 range to 25.3.0 — breaking the producer TypeScript
build with a missing puppeteer-core module error.
* fix(producer): stop tracking node_modules symlinks
packages/producer/node_modules was accidentally swept into a prior commit
despite the repo-wide node_modules/ gitignore rule, and its ~30 tracked
symlinks silently drift from whatever bun install actually resolves —
the exact cause of the stale puppeteer symlinks fixed earlier in this
branch. CI always runs bun install --frozen-lockfile before building, so
nothing depends on these being pre-committed.
* fix(telemetry): expose stalled render stages
* fix(telemetry): preserve capture data on terminal stage events
* fix(telemetry): fix calibration TDZ crash, tag encode/assemble, extend heartbeat cadence
capture_calibration referenced captureStageObservationData before its
declaration (later in the same scope), which would throw a ReferenceError
for any render hitting the calibration path. Hoist the closure and split
workerCount's declaration from its resolution so calibration can safely
read it as undefined before capture strategy resolves worker count.
Also address the two non-blocking review items: wire encode/assemble
stages through captureStageObservationData for consistent tagging, and
extend the heartbeat schedule to repeat every 120s after the initial
30/60/120s ramp instead of going dark on stalls beyond two minutes.
Two non-blocking review notes from Rames, both addressed:
1. Trial polarity inverted to OPT-IN: disableDeParallelRouterTrial →
enableDeParallelRouterTrial. renderLocal is exported, so any programmatic
consumer (future studio-server path, test harness, distributed runner)
previously inherited the trial and its process-wide env-var/module-latch
state without knowing to disable it — and concurrent invocation races
that state. Now only the CLI's own sequential call sites opt in (the
single top-level render, and batch at concurrency 1); everyone else gets
no trial by default. The doc comment names the sequential-invocation
assumption explicitly.
2. deSelfVerifyFallback semantic narrowing documented at both declarations
(RenderCaptureObservability + RenderPerfSummary.drawElement): since the
pinned-fallback retry was widened, the flag means verify-triggered
SPECIFICALLY — OOM/capture_error fallbacks report false with
deFallbackReason carrying the reason. Dashboards keyed on
de_self_verify_fallback=true as "any fallback fired" must migrate to
de_fallback_reason IS NOT NULL (also called out in the PR body for the
observability rebuild to pick up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Layout findings now distinguish held defects from entrance/exit
transients: a dynamic issue seen at a single grid sample demotes to
info, while content_overlap held across two-plus samples (or 500ms+)
promotes to error, resolving the long-standing re-promotion TODO. Static
compositions keep their severity. check gains a sweep_static error when
a 3s+ composition shows zero geometry change across every sample (a
frozen timeline makes every green verdict unreliable); skipped when the
motion sidecar already reported motion_frozen. text_occluded findings
carry a coveredFraction; atomic labels (short, no whitespace) flag on
any cover while prose needs 15%, since partial cover changes what a
short label reads as.
Deprecation-test scaffolding consolidates into deprecationTestHarness;
tier logic and logger tests restructured under the complexity gate
without suppression markers.
Detection mechanics adapted from Adam Rosler's open-sourced
visual-linter design (github.com/Adam-Rosler/hyperframes-visual-linter-design);
the elementFromPoint paint model, opt-out attributes, and single-audit
architecture are unchanged.
One stderr notice per invocation and _meta.deprecated: true in JSON mode
(shared helper next to withMeta; layout owns both inspect and layout via
createInspectCommand). Help descriptions gain the pointer. No behavior
change; removal ships separately once migration telemetry says usage
has decayed.
fix(producer): route info/debug logs to stderr — the compiler's
'Localized remote media' line was landing on stdout ahead of validate's
--json payload, breaking every piped consumer. Diagnostics now share
stderr with warn/error; render progress uses its own channel.
Three defects found by max-effort code review of this branch:
1. The Bun OOM exact-match regex was defeated by this codebase's own
parallel-worker error wrapping. executeParallelCapture/formatWorkerFailure
(parallelCoordinator.ts) always wrap a worker's error as
"Worker N: <message>", optionally suffixed and joined with other workers'
segments, all prefixed "[Parallel] Capture failed: ". That wrapping
defeated the exact-message check for exactly the cohort (deParallelRouter
routed, N separate Chrome processes) the OOM-drops-to-1 fix targets — a
real OOM there would retry at the SAME worker count instead of dropping
to 1. Added a second pattern that recovers the signal by requiring
"out of memory" appear as the WHOLE content of a "Worker N: ..." segment
(bounded by end-of-string/"; "), preserving the same exact-match property
(no bare substring match) while surviving the wrapping. Verified against
the real wrapping logic, not a hand-typed guess at its shape.
2. shouldRetryViaPinnedFallback didn't exclude cancellation, so aborting a
render mid-capture on the pinned router/inversion cohort would detour
through spawning a fresh encoder/capture session before the outer catch's
RenderCancelledError branch ended the render — delaying "stop" with a
pointless resource spin-up/tear-down. Added an isCancellation param
(checked first, before isVerifyError) using the same
`err instanceof RenderCancelledError || abortSignal?.aborted` check the
outer catch already uses.
3. deFallbackReason (this PR's new "oom"/"capture_error" values) was set
locally but never mirrored into RenderCaptureObservability alongside
deSelfVerifyFallback, so a render that fails AFTER a fallback attempt
(perfSummary never built) was indistinguishable in render_error telemetry
from one that never attempted any fallback — undercutting the "how often
does the OOM retry fire on a render that still ultimately fails"
question this branch exists to answer. Threaded through
RenderCaptureObservability → RenderObservabilityTelemetryPayload →
renderObservabilityTelemetryPayload, mirroring the existing
deSelfVerifyFallback plumbing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
shouldRetryViaPinnedFallback retrying OOM was only half the fix: it reused
preInversionWorkerCount/preRouterWorkerCount unmodified, which is
calibration's own pick and can be >= the pinned count that just OOM'd
(calibration wanting 5 while the router pinned to 3). Retrying at equal or
higher parallelism than the failure isn't a remedy — it's the same bet
again, and worsens the odds for this render and anything sharing the host
(PRODUCER_MAX_CONCURRENT_RENDERS runs concurrent jobs in one process).
resolveInversionRetryPlan/resolveParallelRouterRetryPlan now drop to
workerCount=1 specifically when the retry is OOM-triggered — one Chrome
page, the leanest configuration available, not just a different capture
mode at the same worker count. Ordinary self-verify (blank/PSNR) retries
are unaffected — those aren't memory-related, so they keep the
pre-inversion/pre-router count as before.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The router/inversion pin a fixed worker count regardless of calibration —
exactly the scenario a host-contention timeout or worker crash is most
likely under. Previously only a drawElement self-verify failure (blank
frame / PSNR breach) triggered the existing fallback to the calibrated,
non-DE parallel-screenshot path; any other capture-stage failure on a
pinned render just hard-failed the whole job instead of reusing that same
tested safety net.
shouldRetryViaPinnedFallback widens the retry to any capture failure while
deWorkerInversion="inverted" or deParallelRouter="routed", excluding OOM
(the fallback's worker count can be >= the pinned count, so retrying would
likely just OOM again — fail fast instead).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
render_error previously carried zero DE-cohort context — a hard failure while
routed (worker crash, OOM, capture timeout from the fixed 3-worker pin
overriding calibration) was indistinguishable from any other failure. The
data existed (RenderCaptureObservability is mutated live and survives into
job.errorDetails on the failure path) but was never projected into the
render_error payload, which only ever drew de_* fields from perfSummary
(success-only).
- RenderCaptureObservability now also records dePreInversionWorkers /
dePreRouterWorkers — the worker count calibration would have picked absent
the experiment — so a resource-pressure failure can be correlated with the
router overriding a lower calibrated count.
- New capture-sourced de_* fields on RenderObservabilityTelemetryPayload,
shared by trackRenderComplete and trackRenderError. Explicit
perfSummary-sourced fields still win on render_complete (spread moved
first in the event object) — this is purely a failure-path fallback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
HF_DE_PARALLEL_STREAM was restored on every exit path, but the producer
server allows concurrent renders in one process — a router-eligible job's
mutation was still visible to an unrelated render already executing during
that window. Thread the router's decision as a per-render local instead of
a global env var; HF_DE_PARALLEL_STREAM stays as the manual opt-in.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Address unanimous review feedback on PR #2095 (Miga, Magi REQUEST_CHANGES,
Rames D Jusso): the router's process.env.HF_DE_PARALLEL_STREAM mutation
was only cleared on the DE self-verify-retry branch — every other exit
(happy path, any non-DE-verify error, abort) left it set, leaking the
parallel-streaming opt-in into the next render sharing the same process
(a regression/benchmark harness, or any batch host).
Capture the prior value before mutating and restore it in the outer
`finally` (executeRenderJob's own top-level try/finally, which runs on
every exit path by construction), not just the narrow retry branch. Note
`deParallelRouter` and the new `deParallelStreamEnvBefore` had to move
above the outer `try` — a `let` declared inside `try {}` is not visible
in the sibling `finally {}` block in JS, so the original placement
alongside the other DE state would not have compiled once referenced
from the finally.
Also: renamed the shared `preInversionWorkerCount` local to
`preRoutingWorkerCount` (Miga + Rames nit — it now serves both the
inversion and the router), and pinned worker count 3 explicitly overrides
calibration by design (documented per Miga/Rames's question, not a bug).
Verified end-to-end (not just unit tests): ran two executeRenderJob calls
back-to-back in one process, router-eligible then not — env is restored
to undefined after render 1 and stays clean through render 2, the exact
leak scenario the reviews described. 117 orchestrator tests pass (3 new,
covering the restoreEnv primitive directly).
Promotes the opt-in HF_DE_PARALLEL_STREAM mechanism (#2056) into the
auto-routing decision, gated behind its own default-off flag
(HF_DE_PARALLEL_ROUTER). This is the next step from the 2026-07-08
parallel-DE benchmark verdict: par3/single 1.16-1.36x on real-work
comps >=2,000 frames, no comp anywhere losing to single-worker.
shouldPreferParallelDrawElement mirrors shouldPreferSingleWorkerDrawElement
(#2026) but takes priority over it when both are eligible — its higher
default threshold (HF_DE_PARALLEL_MIN_FRAMES=2000 vs the inversion's 900)
means it only ever picks up the long tail the inversion's own benchmark
didn't cover. Fixed at 3 workers (benchmark-validated; not calibration-
derived), same shape as the inversion pinning to a fixed 1.
resolveParallelRouterRetryPlan mirrors resolveInversionRetryPlan for the
self-verify-failure rollback path: falls back to the ordinary (non-DE)
parallel-disk path at the pre-router worker count. The caller must clear
HF_DE_PARALLEL_STREAM before recomputing useStreamingEncode or the retry
would keep resolving to the parallel-streaming shape.
New telemetry (de_parallel_router, de_pre_router_workers) tags which
render used the router, separate from de_worker_inversion — needed
before the planned telemetry soak can segment revert-rate and
de_verify_min_db to the parallel cohort specifically; today there's no
way to tell those apart from ordinary single-worker DE renders.
Verified end-to-end: HF_DE_PARALLEL_ROUTER=true on a 2,381-frame comp
resolves to 3 workers with 3 separate drawElement sessions and renders
successfully; without the flag, behavior is unchanged (falls through to
the existing single-worker inversion, workerCount=1) — no regression to
current production routing. 114 orchestrator tests pass (15 new).
Address two max-effort code-review findings on PR #2056 not covered by
the earlier review-gap commit:
- captureFrameToBufferPipelined's static-dedup reuse branch never
advanced session.lastEncodeResultFrame, unlike its sibling real-capture
branches. The gap-check window is computed from that watermark, so
every consecutive reuse in a static run rescanned an ever-widening
window instead of just the newest frame — O(n^2) total work over a
long static stretch instead of O(n).
- The "single-threaded, no race" justification on the shared
parallelGuard closure was wrong: the guard has real internal await
points (recapture, PSNR) between reading and writing its
sizes/absFloor/acceptedSmall state, so concurrent workers' calls do
interleave there (confirmed). Replaced with the actual reason it's
safe: absFloor only ratchets down, sizes is append-only and
order-independent for the median, and acceptedSmall's fast path
re-validates by exact byte-equality regardless of which worker wrote
the reference buffer.
Address PR #2056 review feedback:
- Fix totalFrames progress inflation for interleaved tasks — divide
each task's span by its frameStride to match the actual per-worker
frame count (captureFrameRange steps by stride), instead of summing
raw endFrame-startFrame which double(N)-counts interleaved tasks.
- Attach a no-op .catch to each frame's pipelined encodeResult at kick
time so an abandoned promise (loop exits early on abort/error before
draining it) can't surface as an unhandled rejection during teardown.
- Document why the pipelined branch's stride=1 path is validation-only
in production (HF_DE_PARALLEL_STREAM always uses interleaved
distribution) so a future refactor doesn't unknowingly widen it.
- Comment the intentional single shared parallelGuard/parallelStats
across workers (safe single-threaded, better rolling-median signal).
Step 2 of the DE engagement plan: multi-worker drawElement capture through
the streaming encoder, with the full runtime self-verification net riding
along — the confinement rule that kept the parallel clamp in place is now
satisfied on this path. Opt-in via HF_DE_PARALLEL_STREAM=true; default
routing (including the #2026 single-worker inversion) is unchanged.
Mechanism:
- distributeFramesInterleaved + WorkerTask.frameStride: worker i captures
frames i, i+N, i+2N... — seek-based capture makes stride free and the
ordered writer's reorder window shrinks from totalFrames/N to N (contiguous
chunks serialize workers behind the writer).
- Depth-2 pipelined worker-encode produce in the parallel worker loop (the
same shape as the sequential loop; frame k's in-page encode overlaps
k+stride's produce). HF_DE_PAR_DEBUG=1 traces the first frames per worker.
- Drain guard extracted to createDrainFrameGuard (session-parameterized):
every parallel frame gets the SAME blank-guard + PSNR self-verify as the
sequential drain, against its owning worker's pre-injection ground truth
(all sessions arm identical sample indices from
CaptureOptions.compositionDurationSeconds).
- FrameReorderBuffer.abort(err): a failed worker (e.g. verification error)
rejects all parked and future waiters — without this, peers park forever
in waitForFrame and the pool (which awaits ALL workers before surfacing
errors) deadlocks. Found by the verify-trip test; unit-tested.
- The typed DrawElementVerificationError is preserved past the pool's
error-string flattening so the orchestrator's verify-retry recognizes it.
- Static-dedup stride hazard fixed: lastEncodeResult reuse now requires EVERY
frame in (lastEncodeResultFrame, i] to be predicted-static (sequential
capture reduces to the old has(i) check).
- Workers get separate browser PROCESSES under the flag: pages co-tenant in
one browser starve non-active pages of BeginFrames on the paint-wait path
(measured 86s vs 30s on a 3,245-frame rAF comp).
Validation:
- Happy path W3: verify samples pass across workers (4x inf on the 2,381f
comp), output vs single-worker DE = 59.3dB (encode noise floor) — the
interleave + dedup-stride produce identical pixels.
- Verify-trip (marginal comp + HF_DE_VERIFY_MIN_DB=45): fails at frame 649
(32.2dB < 45), peers abort instead of deadlocking, whole render retries
via parallel screenshot, RENDER_OK in 42.6s.
- Canary suite 7/7 with the flag off (default paths untouched); producer
orchestrator tests 99/99; engine suite 909 passed (14 pre-existing main
failures, stash-A/B verified); reorder-buffer abort unit tests.
Perf note: capture-only parallel speedup measured 1.38x (W2) / 1.52x (W3)
over single-worker DE in the spike; end-to-end numbers on this machine are
currently noisy (separate-browser init overhead + bench load) — clean
benchmarks before any default routing change. The flag stays explicit
opt-in; promoting it into the router replaces the #2026 W=1 pin for the
same cohort.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address PR #2045 review feedback:
- Share a SubTimelineWaitOutcome type (engine) end-to-end instead of
widening to string across CapturePerfSummary / RenderPerfSummary /
telemetry, so the three layers can't drift.
- Dedupe scriptLoadFailures on push — a 4xx response and its trailing
requestfailed both recorded the same URL, doubling the failed-URL
list in the fail-fast warning.
- Thread the sub-timeline-wait outcome into render_error (not just
render_complete): a render that fail-fasts and then fails downstream
(pollVideosReady, extract, encode) previously dropped this signal on
the floor. dedupPerfs is now function-scoped so the catch path can
read it, same treatment as the existing captureAttempts array.
Vance Ingalls