Commit Graph
3 Commits
Author SHA1 Message Date
Miguel Ángel 9825614d91 fix(studio): Studio's own text reads without em or en dashes (#5414)
* fix(studio): text in Studio reads without em or en dashes

* fix(studio): the dash check also catches HTML entities, and peak reads Unavailable

* test(studio): the dash check test covers every entity form

* fix(studio): messages Studio shows from lint, sdk, core and the renderer drop dashes too

* fix(producer): the gate check stops early on the engine's new fallback wording
2026-10-10 19:08:30 +00:00
Vance IngallsandClaude Fable 5.1 84e36fc16a feat(runtime): data-vfx-chain backdrop capture and two-source kernels (luma-matte, noise) (#4330)
* feat(runtime): data-vfx-chain backdrop capture and ref second sources

Interface v1.1's two remaining capture shapes, in the runtime.

`backdrop`: a host resolves its source from a sibling
`<canvas layoutsubtree class="hf-vfx-src" data-vfx-for="<host id>">`
before its own child wrapper, so the same kernel reads either its own
pixels or every layer below it — which element the wrapper holds is the
exporter's placement, not a def's declaration. `VfxEntry.capture` now
reports what was resolved rather than what the defs asked for.

The trap that shape creates: a `backdrop` wrapper lives OUTSIDE the host,
and children of a `layoutsubtree` canvas are never painted by the page
compositor. Skipping a hidden host the way a `self` host is skipped would
therefore delete every layer below the adjustment instead of passing it
through. A hidden backdrop host now still captures and KEEPS the bitmap,
painting no kernel.

`ref` params: a node's second source, captured into `u_src2` from the
wrapper the exporter emitted around the referenced element. Per node, not
per entry, and shared when two nodes name the same element. Every source
is drawn at the host's device size so `u_src`/`u_src2`/`.hf-vfx-out`
share one coordinate space, and each source's canvas gets its own
`paint` wait — paint records are per canvas. A source the clip runtime
has hidden captures EMPTY rather than waiting out a paint that cannot
come and then throwing "No cached paint record" every frame; an empty
matte is also what After Effects shows outside a matte layer's window.

Also: a `.hf-vfx-in` that measures 0×0 is now reported once instead of
capturing nothing in silence, and one that is not the canvas's immediate
child is refused at registration instead of throwing every frame. Both
are measured Chrome rules (vault `layoutsubtree-capture-rules`).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(runtime): luma-matte and noise two-source vfx kernels

`luma-matte` is the first def with a `ref` param: `out = src · coverage(u_src2)`
with Alpha / Alpha Inverted / Luma / Luma Inverted coverage. It is not one
After Effects effect but the runtime form of all of them — unsupported
track-matte sources, stencils, silhouettes, Set Matte — which the exporter
distinguishes by choosing what the host wraps and what `matte` names. `src`
is premultiplied, so one scalar multiply covers all four channels, and Luma
reads the matte over black, which is what a premultiplied texel already holds.

`noise` ports `ADBE Noise` from `AEFX_Noise.metallib`: Bob Jenkins' 1997
integer mix over `(x, y, seed32)`, the two glibc-style LCG rounds, and the
`0x7F80`/`0xFF` byte the IR XORs out of them — no table, no state, so it is
seek-safe by construction. Reading the metallib's struct metadata rather
than the deep dive's prose settles two of its open questions:
`ADBE Noise-0002` is `inUseColorNoise` (a checkbox), not a Uniform/Gaussian
popup, and the mystery seed floats are `inRandomSeed_High`/`_Low`, the two
halves of one 32-bit seed. What the CPU puts in those halves per frame is
still unprobed (plan Task 4.3), so the kernel seeds the low half with the
frame index; the tests assert distribution and seek determinism, not parity.
The shader declares `precision highp int` — the whole port rests on 32-bit
wraparound and a fragment shader's integers default to mediump, which ESSL
guarantees only 16 bits of.

Both defs declare `capture: "self"`, which says a texture is needed and
nothing about whose: `noise` on an adjustment layer is the same kernel, and
the runtime reads the `backdrop` placement off the DOM.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(vfx): browser checks for backdrop capture, luma-matte and noise

Task 2.6 Step 5 and Task 2.7's owed browser checks, folded into the
existing puppeteer harness: an identity backdrop kernel composites two
stacked blocks captured through a data-vfx-for sibling wrapper back
over themselves (PSNR >= 40dB vs a plain-DOM control panel); a backdrop
host outside its own clip window still keeps those blocks on the
wrapper canvas's own bitmap via capturePassThrough; luma-matte's four
modes are checked against alpha- and luma-varying matte strips at four
sample points each; noise is checked for amount=0 identity, same-time
determinism, and a changed pattern when its only seed (seek time)
changes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(runtime): external displacement maps and visible ref sources

Two gaps the exporter side found in the `ref` plumbing.

1. `displacement-map` had no `map` ref param, so the exporter's
   `params.map = "<element id>"` was dropped by `normalizeVfxParams` (it
   iterates the def) and the node silently self-displaced. `map` is now a
   ref param, bound to `u_src2` and selected by a new `u_hasSrc2` uniform
   the runtime sets from whether the ref resolved. Refs gain
   `optional: true` — empty `map` is the self-referential form every
   corpus instance uses, while an empty `matte` stays a loud failure — and
   a ref naming its own host resolves to self rather than capturing the
   same pixels twice (on a backdrop host it would otherwise have read the
   layers below as the map).

2. Interface v1.1 assumed a ref source is invisible in After Effects.
   retro-wave `Logo Anim` layer 5 is layer 4's displacement map AND an
   enabled, opaque layer, and the runtime cleared its wrapper's bitmap
   after the upload, deleting it from the frame. A wrapper marked
   `data-vfx-ref-visible` now keeps its bitmap and is captured at the
   REF's own box instead of the host's, so it paints at its own size and
   resolution. The three capture shapes are one `{ upload, keepBitmap }`
   mode: texture only, bitmap only (a backdrop pass-through), or both.

Sampling is unaffected by the box choice, and the browser test proves it
rather than assuming it: `drawElementImage` scales the element into the
whole canvas, so a 320px-wide matte's four strips still land as four
40px bands in a 160px host. The v1 rule — a ref is scaled into the host's
box, not placed in composition space — is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(runtime): capture a visible ref while its host is hidden; review nits

Addresses PR #4330 review feedback (jrusso1020, CHANGES_REQUESTED).

Blocking: both the engine-mode and preview-mode capture paths treated
"host not paintable" as binary — a backdrop entry got a pass-through
capture, everything else was skipped outright, including a ref source
that lives on a different, independently-visible element (e.g.
retro-wave's Logo Anim layer, which is simultaneously layer 4's
displacement map and its own fully opaque visible layer). Adds
visibleRefSources()/captureVisibleRefsOnly() and a unified
FrameCaptureMode discriminator so a visible ref still gets captured
even while its own host is hidden, on both paths.

Non-blocking, addressed:
- luma-matte's mandatory ref that names its own host now resolves to
  an explicit RefResolution {kind:"self"} and binds u_src2 to unit 0,
  instead of relying on an unbound sampler's implicit default.
  displacement-map's optional self-ref is unaffected (still folds into
  {kind:"none"}, using its existing u_hasSrc2 shader branch).
- noise shader now enforces rgb<=a (valid premultiplied alpha) after
  adding noise, independent of u_clipping — documented why u_clipping
  is currently a permanent no-op on this runtime's 8-bit render
  targets.
- added a hash-pinning test for hfJenkinsMix; documented, with a
  brute-force proof, which one of the reviewer's suggested mutations
  is unobservable within this fixture's reachable seed range.
- fixed two misplaced doc comments.

fallow flagged resolveVfxCapture/resolveRefSource as newly over the
complexity threshold after this diff; extracted resolveEmptyRefParam()
and captureHiddenEntry() to bring both back under.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(runtime): capture a visible ref while a hidden BACKDROP host passes through

Re-review of #4330 at b6d51bd9: the visible-ref-while-hidden fix only
covered non-backdrop hosts. captureHiddenEntry returned
capturePassThrough(entry) for a backdrop and never reached
captureVisibleRefsOnly; frameCaptureMode picked "passThrough" before
considering refs and sourcesForFrame("passThrough") was [entry.src]
alone. An adjustment layer whose displacement map is itself a visible
layer still lost that layer outside the adjustment's window.

- captureHiddenEntry runs the pass-through AND the visible-ref capture,
  OR-ing the results.
- sourcesForFrame("passThrough") is the backdrop wrapper plus any
  visible refs, and capturePaintedHost's pass-through branch also calls
  captureVisibleRefsOnly.
- Tests for the backdrop+visible-ref shape in both engine and preview
  modes (the reviewer's probe).

Also closes the non-blocking display:none item: isPaintableHost(src.inner)
missed a ref whose OWNING element is display:none, since display does not
inherit — the preview path waited out the 2 s ceiling, then failed with a
misleading 0×0 report. New isPaintableSource walks inner's ancestors for
display:none (visibility inherits, so one read suffices there); used at
both source-side sites. checkVisibility()/offsetParent are absent or
always-null in the unit harness's DOM. Test added; verified it fails on
the old inner-only check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 23:25:20 -07:00
Vance IngallsandClaude Sonnet 5 b32b9a7dab feat(runtime): data-vfx-chain — a seek-deterministic per-pixel effect element (#4306)
* feat(core): data-vfx-chain defs and chain parsing (mirrors data-fx-chain)

Mirrors audioFx.ts's shape for the per-pixel VFX chain: a versioned
JSON chain attribute (data-vfx-chain), a def registry with typed
params (number/enum/bool/ref), and parse/serialize/normalize helpers.
HF_VFX holds one def, fractal-noise, with a placeholder frag; the
kernel and its consumers (packages/core/src/runtime/vfx.ts) land in
Task 1.2/1.3 of the vfx-chain-runtime plan, so vfx.ts is added to
.fallowrc.jsonc's entry list for now.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(runtime): paint data-vfx-chain canvases on every seek

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(vfx): fractal-noise kernel (Basic/fBm approximation) with CPU reference

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(vfx): self capture through layoutsubtree with the page-composite readiness protocol

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(vfx): wave-warp kernel (Sine)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(vfx): displacement-map kernel (self-referential map)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(vfx): chain order is nesting order; preceding CSS effects are captured

...and following ones apply outside. The plan's one-line message for Task 2.4
is 105 characters; commitlint caps the header at 100, so the tail moved here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(vfx): seek determinism and CPU-reference PSNR for fractal-noise

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(vfx): size the output canvas from the host's layout box, not its transformed AABB

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(telemetry): vfx node count, capture class and types on render_complete

Threads vfxNodeCount, vfxCapture and vfxTypes from a static scan of the
compiled composition's data-vfx-chain hosts (mirrors the existing
compositionElementCount/colorGradingCount scan) through
RenderPerfSummary.drawElement into the render_complete event, next to
de_capture_mode. Exports chainCapture/parseVfxChain/HfVfxCapture from
@hyperframes/core's index for the producer to consume. The PostHog
dashboard tile is not created here — left for the coordinator.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(vfx): data-vfx-chain guide — element shape, capture table, Studio flag, known constraints

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(vfx): register chain hosts that arrive with a sub-composition mount

`initVfx(document.body, fps)` ran once, synchronously, three lines after
`void loadExternalCompositions(...)` — so every `data-vfx-chain` host that
enters the DOM only when a `data-composition-src` mount resolves was never
registered. An unregistered host paints nothing and reports nothing: no
`[HyperFrames] composition script error`, the `.hf-vfx-out` canvas keeps its
300x150 default, and `paintVfx` never even arms the page-composite flag.
Measured on a real multi-file export served raw: `#main-l5-text`'s output
canvas stayed 300x150 and `window.__timelines` never gained the sub-comp.

Re-scan in the loader's `.finally`, before `maybePublishRenderReady()`, so
registration precedes readiness. `initVfx` now releases the outgoing
registry's programs, capture textures and ping-pong targets first — the
output canvas hands back the same WebGL2 context on a re-scan, so the
previous pass's GL objects would otherwise leak.

Two browser cases in `vfxDeterminism.test.ts`, both against the real runtime
bundle: a host whose `.hf-vfx-in` IS the mount (already worked — the mount
replaces the wrapper's children, not the wrapper, so the registered source
node survives), and a host that arrives INSIDE a mounted sub-composition
(failed with `armed: false` and no error; passes now).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(vfx): runtime tsconfig typecheck passes

Under noUncheckedIndexedAccess, resolveCaptureSource's null return
widened VfxEntry.src past its VfxCaptureSource | undefined type; the
function is internal-only, so its return type moves to
VfxCaptureSource | undefined instead. Indexing PingPong.framebuffers
by a non-literal index produced | undefined against
WebGLFramebuffer | null, coalesced to null. No behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(vfx): structural DOM predicates instead of realm-bound instanceof

vfx.ts checked `instanceof HTMLCanvasElement`/`HTMLElement` directly, which
returns false for inner-realm nodes when the Studio previews across an
iframe realm. domRealm.ts had no canvas predicate yet, so this adds
isCanvasElement mirroring the existing isImageElement pattern, then swaps
all three vfx.ts checks to the structural predicates the lint guard requires.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(vfx): paint self-capture hosts under hyperframes snapshot, not only the engine path

`paintVfx(t, { engineMode: true })` armed the page-composite protocol and
returned, betting that every capturing chain runs under `frameCapture.ts`.
`hyperframes snapshot` (and check/compare/validate/layout, and Studio's
thumbnail capture) seek through the same `renderSeek`, never read
`__hf_page_composite_pending` and never call `__hf_page_composite_resolve` —
so a `self` chain painted nothing there, in silence, because its content
lives in a `layoutsubtree` canvas the browser never paints directly.

Engine mode now arms the preview-side capture as well and paints on whichever
completes first. A seek token stops a superseded async capture from repainting
a newer frame, and a registry-identity check stops it painting an entry
`initVfx` already released. `resolveVfxCapture` stays non-idempotent on
purpose: the engine resolves only after its own before-capture hooks have
mutated the DOM, so its capture is still the authoritative one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(vfx): an exporter-emitted .hf-vfx-out gets the box the spec fixes for it

findOrCreateOut adopted a canvas the exporter had already emitted without
applying the layout cssText, so it kept the default position:static;
display:inline and wrapped to the line AFTER the layer-wide .hf-vfx-src
canvas — painting a second, misplaced copy of the layer one full layer
height below the host. Measured on retro-wave #main-l6-text: .hf-vfx-src
rect top 315.1 h 93.0, .hf-vfx-out rect top 409.2, both style=null,
position static (ae-mcp findings Task 3.5b). Hosts with no emitted
out-canvas were never affected, which is why only this one host moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(telemetry): vfx_node_count counts hosts, not chain nodes — rename to vfx_host_count

The field increments once per `data-vfx-chain` attribute occurrence, never
per node inside a chain's JSON, so the name promised something the scan does
not measure. Renames the internal `vfxNodeCount` and the wire property
`vfx_node_count` to `vfxHostCount`/`vfx_host_count` across the orchestrator
scan, the perf summary, the CLI render path and the telemetry event. No
dashboard consumes the field yet, so the wire rename costs nothing now and
would cost a migration later.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(vfx): join the preview capture to the seek-completion barrier, not race it

`paintVfx` fired `capturePreviewThenPaint` and forgot it, so nothing the CLI
awaits waited for a `self` chain's paint. Every snapshot-family command
(`snapshot`, `check`, `compare`, `validate`, `layout`) seeks through
`seekCompositionTimeline`, which drains `window.__hfWaitForSeekCompletion`
and then allows one settle race before screenshotting — and only the
`"hf-seek"` event's `waitUntil()` could put anything in that barrier. The
screenshot therefore landed on however many hosts happened to have finished,
which varied with host count and machine speed.

Extracts the "track, untrack on settle, retain the first rejection" block
`dispatch()` built inline into an exported `registerSeekCompletion`, and has
`paintVfx` register the preview capture through it. Also awaits the per-host
paints with `Promise.all` instead of sequentially: N hosts cost about two
animation frames now, not 2N.

Measured on the pre-fix bundle with two `self` hosts, both output canvases
read `[0,0,0,0]` at the moment the barrier returned; they painted only
because the settle race that follows happened to be long enough. Both tests
added here fail on that bundle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(vfx): report WebGL context loss loudly instead of silently no-op painting

One WebGL2 context per `data-vfx-chain` host, against a browser cap that is
commonly 16 and shared with `colorGrading.ts`'s per-element contexts. A lost
context makes every later call on it do nothing, per spec — so the chain kept
"painting" a blank canvas with no error, which is exactly what the module's
loud-failure contract exists to prevent.

Adds the `webglcontextlost` listener pair's first half, following
`colorGrading.ts`'s precedent: `preventDefault()`, mark the entry, report
once through `reportVfxError`. The entry is then skipped by `paintVfx`,
`capturePreviewThenPaint` and `resolveVfxCapture` — silently, since the one
loud report already happened and repeating it per frame is spam.

Restore is left unhandled on purpose, with the reasoning in the doc comment:
`initVfx`'s existing re-scan already rebuilds every GL object, and a second
narrower rebuild path for `webglcontextrestored` would duplicate it. The
shared-context redesign the review floated is likewise out of scope here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* perf(vfx): cache uniform locations at pass-build time instead of querying every paint

`setPassUniforms` asked the driver for `u_size`, `u_t`, `u_fps` and every
param's location on every paint of every pass, and `bindPass` did the same
for `u_src` on every bind. `buildPasses` already constructs each `VfxPass`
once at init, right after the program links, and a linked program's
locations never change — so they are resolved there now and read from
`pass.locations` in the hot path.

Same values, same `null`-means-optimised-out semantics (`gl.uniform*(null,
…)` is a defined no-op, which is what the per-frame query produced too), so
the existing vfx.test.ts and the browser determinism suite cover it
unchanged — both still green, the latter against a rebuilt runtime bundle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(vfx): bound the paint wait per host so a stalled compositor can't hang the seek barrier

`paintVfx` registers `capturePreviewThenPaint` into the shared seek-completion
barrier (48aaa2bbb), and that promise settled only via `awaitCanvasPaint` — a
race between the canvas `paint` event and a double-rAF chain. Under BeginFrame
control (Linux headless-shell, `drawelement` capture) neither can fire before
the barrier is drained: `frameCapture.ts` awaits the barrier in
`prepareFrameForCapture` (~2692), while the frame's
`HeadlessExperimental.beginFrame` is issued later, in the per-frame capture
stage (~3797). Fire-and-forget that was harmless; registered it can stall every
`__hfWaitForSeekCompletion` caller, the engine's render path included.

rAF-under-BeginFrame: CONFIRMED gated, from this repo's own engine notes —
`frameCapture.ts` ~2445 ("waitForFunction uses rAF polling internally, which
won't fire in beginFrame mode") and ~4755 ("headless only fires rAF when a
frame is produced, and nothing produces one until a screenshot asks"). Not
reproduced locally: every environment here is macOS with a free-running
compositor (`beginFrameTimeTicks === 0`). `setTimeout` is not compositor-gated
there — `drawElementService.ts` ~341 describes its 250 ms paint fallback as
burning on every BeginFrame frame, which requires the timer to fire — so a
timer ceiling is a sound escape.

`awaitCanvasPaint` now resolves `"painted" | "timeout"` with a 2000 ms ceiling
(the repo's own `BEGINFRAME_PROBE_TIMEOUT_MS`; ~20x a real two-frame paint,
far short of the ~180 s CDP `protocolTimeout` a hang would burn). The race is
per host, so one stalled compositor no longer delays hosts that would have
painted; a timed-out host is skipped (no capture, no guessed pixels) and
reported loudly. The report is not suppressed in engine mode: `frameCapture.ts`
~2708 skips `__hf_page_composite_resolve` in `drawelement`/`beginframe` modes,
so there is no authoritative second attempt to defer to there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(ci): re-pin test-classification.mjs hash after registering vfxDeterminism.test.ts

An earlier commit added vfxDeterminism.test.ts to INTEGRATION_TEST_FILES;
check-test-reachability.mjs pins a sha256 of that file to force a conscious
review whenever it changes. Reviewed: runner[1]'s glob already covers the
file structurally, the classification move (unit -> integration) is correct
for a Chrome-launching test, and the pinned hash is now current.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(core): regenerate position-edits-render-inline.ts for Linux esbuild output

check:position-edits-render (bun run build:position-edits-render && git diff
--exit-code) failed on both Linux CI runners (Test (runtime), Tests on
windows-latest: studio-core) but passed on macOS: esbuild's minifier named
two identically-scoped WeakMap/function bindings differently on Linux than
on the mac that last committed this artifact, and the divergence appeared
only on this branch (main's own current tip passes CI cleanly), so it is
tied to a change on this branch rather than a random flake, even though it
is not attributable to any specific line of that change.

Confirmed stable across two independent Linux CI runs (byte-identical diff
both times) and reconstructed directly from that diff rather than guessed:
the committed content, byte length (3817), and 3/3 existing artifact tests
all match. A future contributor rebuilding on macOS will see this same
diff again in the other direction until the build script's cross-platform
determinism is fixed at the esbuild-invocation level, which is out of this
PR's scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 03:14:08 -07:00