mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
feat(sdk): expose a paint query for transparent-composition hit-testing (#3070)
* feat(sdk): expose a paint query for transparent-composition hit-testing A host layering a transparent composition over other content has to know whether a point carries ink before it decides to swallow a click. The adapter only answered "what element is here", so AI Studio wrote its own answer and could not reach the per-pixel alpha the adapter already samples for <img>. Add PreviewAdapter.paintsAt, plus the pieces it is built from on a new ./adapters/iframe subpath so a host with different hit-test policy can compose its own walk. The walk is geometric rather than elementsFromPoint-based: that stack omits pointer-events:none nodes, and a decorative overlay carrying it still paints, so a z-stack query would report no ink over visible artwork — the direction that makes a composition vanish from under the cursor. fullBleedFraction is an option rather than a constant because "a layer covering the whole frame is background, not artwork" is host policy, not a fact about the composition. * fix(sdk): scope the full-bleed frame to the root under the point compositionFrameArea took the smallest [data-composition-id] in the whole document, so an unrelated sub-composition sized the reference frame for points nowhere near it: a 300x300 badge in a corner made every mid-size painter in a 1920x1080 outer frame read as full-bleed, and the composition went click-through under artwork the user can plainly see. That is the direction the fail-safe exists to avoid, and the docs already described the intended behaviour — the innermost root CONTAINING the point. Also read the alpha channel instead of matching known transparent spellings. Only the `transparent` keyword computes to rgba(0, 0, 0, 0); a faded-out white stays rgba(255, 255, 255, 0), which the set counted as painted. That erred toward absorbing clicks rather than losing them, so it was a false positive rather than a hazard, but it is wrong. Both are pinned by tests that fail when the fix is reverted. * fix(sdk): stop the paint query answering "no ink" over visible artwork Four cases where the walk landed on the wrong side of its own fail-safe. The full-bleed veto tested the winner's border-box area even when the alpha sampler had just read an opaque pixel there, so a full-frame transparent PNG or SVG overlay — the case this feature exists for — reported background over visibly opaque artwork, with no fullBleedFraction that worked. Ink now carries how it was established, and a measured pixel is never vetoed. An image whose pixels could NOT be read stays inferred, so a tainted CDN overlay still yields to the veto rather than absorbing every click. Composition roots were excluded from candidacy outright, so a root carrying a background answered false even at fraction 0, where the docs promise every painting box counts. Roots are candidates now; the veto discounts them without a special case, since a root's box is the frame. An <img> with a clear pixel early-returned past its own background, padding plate and border, which any other element would have counted. A same-origin iframe mid-navigation exposes a readable but empty document, so the !doc guard never fired and a loading composition answered a confident "no ink" — the exact failure the null convention exists to prevent. Also: the sort comparator's epsilon tie was intransitive, leaving the smallest-first guarantee (and the lazy single-sample property that rides on it) engine-dependent; the guide's pass-through recipe called a function that does not exist and hand-waved the coordinate mapping that makes it correct; and the reference now states the under-counts alongside the over-counts, the walk's blindness to runtime-mounted content, and compositionPaintsAt's preconditions. Each fix is pinned by a test that fails when the fix is reverted. * refactor(sdk)!: invert the paint query to isProvablyEmptyAt paintsAt handed callers three falsy bottom values with opposite safe readings: false meant "no ink, pass the click through", null meant "not knowable, treat as painted", and undefined from an adapter without the method also meant painted. The idiomatic `if (!preview.paintsAt?.(x, y)) passThrough()` therefore did the dangerous thing for two of the three, and the convention needed defending in the interface docstring, the reference and the guide — plus a dedicated comment and a pinning test on the headless adapter to stop null regressing to false. Inverting the polarity collapses the tri-state to a plain boolean and makes the safe reading structural: true only when the composition was readable and nothing painted there, so ink, an unreadable or still-loading document, and a missing implementation all land on "keep the composition clickable". The prose stays as rationale, but nothing depends on a reader remembering it. PaintsAtOptions becomes PaintQueryOptions, since it now describes the walk that both the adapter method and the exported compositionPaintsAt share rather than one method's arguments. compositionPaintsAt keeps its ink-positive name: it answers the other question, and its docstring points callers who need the fail-safe contract at the adapter. Nothing is released yet, so no consumer is on the old name.
This commit is contained in:
Hblee
@@ -544,6 +544,12 @@
|
||||
// this PR only teaches the server scan to prefer its reported PID, but that
|
||||
// line shift makes fallow re-flag the inherited probe clones.
|
||||
"packages/cli/src/server/portUtils.ts",
|
||||
// iframe.test.ts: the remaining clone groups are pre-existing per-case arrange
|
||||
// blocks in the selection and draft-loop suites (build an adapter, wire a spy,
|
||||
// act). Appending the paint-query suite shifts their line numbers and re-flags
|
||||
// them; each block states its own setup on purpose, which a shared fixture
|
||||
// would hide.
|
||||
"packages/sdk/src/adapters/iframe.test.ts",
|
||||
// gsapParserAcorn.motionEval.test.ts: parallel arrange/act/assert cases for
|
||||
// the staggered-collection honesty pass (.from reveal vs .to landing on the
|
||||
// rest pose). Each asserts a distinct keyframe shape; collapsing the shared
|
||||
|
||||
@@ -99,6 +99,65 @@ iframeDoc.addEventListener("click", (e) => {
|
||||
|
||||
`resolveNearestHfElement` returns `null` when the walk exits the tree without finding a `[data-hf-id]` node, when the matching node carries `[data-hf-root]` (the root is transparent to selection), or when `isVisible` returns `false` for that node.
|
||||
|
||||
## Transparent compositions over other content
|
||||
|
||||
A composition authored as an overlay — a small graphic on an otherwise-empty 1080×1920 frame, layered over a video or an avatar — is still a rectangular DOM box covering every pixel of the frame. Without help it swallows every click, and whatever sits beneath it becomes unreachable.
|
||||
|
||||
`preview.isProvablyEmptyAt(x, y)` is the question you need answered: is this point provably free of ink, so a click may safely reach what sits beneath? Toggle `pointer-events` on your wrapper from the answer, and let the browser deliver the event to the right target:
|
||||
|
||||
```typescript
|
||||
const wrapper = document.querySelector<HTMLElement>("#composition-wrapper")!;
|
||||
|
||||
/**
|
||||
* Host-page pointer coordinates → the iframe document's own client space, which is what
|
||||
* the paint query samples against. The iframe renders at the composition's native size and is
|
||||
* CSS-scaled to fit, so the on-screen scale has to be divided out — skip this and you
|
||||
* sample the wrong pixel, and pass-through toggles over the wrong regions.
|
||||
*/
|
||||
function toCompositionPoint(clientX: number, clientY: number) {
|
||||
const rect = iframe.getBoundingClientRect();
|
||||
if (!rect.width || !rect.height) return null;
|
||||
const scaleX = rect.width / compositionNativeWidth;
|
||||
const scaleY = rect.height / compositionNativeHeight;
|
||||
if (!scaleX || !scaleY) return null;
|
||||
return { x: (clientX - rect.left) / scaleX, y: (clientY - rect.top) / scaleY };
|
||||
}
|
||||
|
||||
function updatePassThrough(clientX: number, clientY: number, altKey: boolean) {
|
||||
const point = toCompositionPoint(clientX, clientY);
|
||||
// Alt is the escape hatch for grabbing the composition itself in an empty region.
|
||||
// Every uncertain case — no point, still loading, adapter without the method — is
|
||||
// falsy here, so the composition stays clickable rather than vanishing.
|
||||
const passThrough =
|
||||
!!point && !altKey && !!preview.isProvablyEmptyAt?.(point.x, point.y);
|
||||
wrapper.style.pointerEvents = passThrough ? "none" : "";
|
||||
}
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Listen on the **host document**, not on the wrapper or the iframe. The first time this
|
||||
sets `pointer-events: none` the wrapper stops receiving events, so a listener attached
|
||||
there can never turn it back on — the pass-through state sticks.
|
||||
</Warning>
|
||||
|
||||
Three things are easy to get wrong here:
|
||||
|
||||
<Steps>
|
||||
<Step title="Decide before the press, not during it">
|
||||
The browser picks an event's target before any handler runs, so flipping `pointer-events` inside `mousedown` cannot retarget the click already in flight. Sample the pointer position on `mousemove` and keep the decision current.
|
||||
</Step>
|
||||
<Step title="Re-evaluate on every frame, not only on movement">
|
||||
Animated artwork moves under a stationary cursor. Anything that can change the answer — pointer movement, the Alt key, and the playhead — has to re-run the query from the last known position. Coalesce those triggers into one `requestAnimationFrame` query rather than answering each separately, and short-circuit before the query when the pointer is outside the composition's box: it is a walk over the document, so it does not belong on an ungated per-event path.
|
||||
</Step>
|
||||
<Step title="Let the polarity do the work">
|
||||
`isProvablyEmptyAt` is true only when it has established there is no ink. A document that hasn't loaded, an adapter without the method, and a point you couldn't map all come back falsy — which keeps the composition clickable. Don't invert it into a "does it paint" variable; that reintroduces the bug the polarity removes.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
Pass `{ fullBleedFraction: 0.9 }` if your editor treats a layer covering nearly the whole frame as background rather than artwork — a common choice, since a full-bleed wrapper is usually scaffolding rather than something the user is pointing at.
|
||||
|
||||
Do not reimplement this with `elementsFromPoint`. That stack omits `pointer-events: none` nodes, and a decorative overlay carrying `pointer-events: none` still paints — a z-stack query would report no ink over visible artwork and pass the click through anyway. The paint walk covers element boxes geometrically for that reason.
|
||||
|
||||
## Draft loop: 60fps drag without model mutations
|
||||
|
||||
The draft loop keeps the model clean during a drag. The SDK is **not** in the 60fps path — you call `preview.applyDraft` on every `pointermove` and `preview.commitPreview` once on `pointerup`. The model sees exactly one `moveElement` op per drag, rather than hundreds.
|
||||
|
||||
@@ -87,6 +87,7 @@ Injectable preview surface adapter. Decouples the SDK from the host's rendering
|
||||
```typescript
|
||||
interface PreviewAdapter {
|
||||
elementAtPoint(x: number, y: number, opts?: { atTime?: number }): ElementAtPointResult | null;
|
||||
isProvablyEmptyAt?(x: number, y: number, opts?: PaintQueryOptions): boolean;
|
||||
applyDraft(id: string, props: DraftProps): void;
|
||||
commitPreview(): void;
|
||||
cancelPreview(): void;
|
||||
@@ -100,6 +101,24 @@ interface PreviewAdapter {
|
||||
Synchronous hit-test at composition coordinates `(x, y)`. Returns the nearest `[data-hf-id]` element under the point, or `null` for a transparent hit (the composition root, an opacity-0 element, or nothing at all). Requires a same-origin iframe — cross-origin access throws a DOMException. The `atTime` option reflects GSAP state at the current playhead; seeking to a speculative time is not supported.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="isProvablyEmptyAt" type="(x, y, opts?) => boolean">
|
||||
Optional. Is `(x, y)` provably free of ink — is it safe to let a click pass through to whatever sits beneath? This is the question a host has to answer before a transparent composition layered over other content swallows a click: is the user pointing **at** artwork, or through an empty gap? Geometry alone cannot tell — a composition is mostly full-bleed wrapper `<div>`s that cover every pixel of the frame without painting anything.
|
||||
|
||||
**True only when the composition was readable and nothing painted there.** Ink present, a document still loading or unreadable, and an adapter that doesn't implement the method (`preview.isProvablyEmptyAt?.(x, y)` → `undefined` → falsy) all come back falsy. That polarity is deliberate: it puts the burden of proof on passing the click through, so every way of failing keeps the composition clickable rather than making it vanish from under the cursor. The obvious call site is safe by construction:
|
||||
|
||||
```typescript
|
||||
if (preview.isProvablyEmptyAt?.(x, y)) passThrough();
|
||||
```
|
||||
|
||||
Ink is a computed-style test — background colour, background image, visible border, the element's own text, or intrinsic media — with one exception: `<img>` (and the `<img>` inside a `<picture>`) routes through per-pixel alpha, so a transparent PNG paints only where its pixels do. A pixel-verified hit is never discounted by `fullBleedFraction`: box area is not ink area, so a full-frame transparent overlay stays clickable where it is actually opaque.
|
||||
|
||||
**Known over-counts** (report ink that isn't there, so a click selects the composition): a `background-image` that is itself mostly transparent reads as painting across its whole box; `<video>`, `<svg>` and `<canvas>` are unconditionally opaque; and an image whose pixels cannot be read — cross-origin without CORS, still loading, rotated, or above the sampler's size budget — falls back to opaque.
|
||||
|
||||
**Known under-counts** (miss ink that is there, so a click may pass through): `::before` / `::after` generated content, `box-shadow`, `outline` and `text-decoration` are not tested, and the first three paint outside the border box, so the element is not even a candidate. Content the SDK never stamped is invisible under the default `addressableOnly` — see below.
|
||||
|
||||
The walk is **geometric**, not `elementsFromPoint`-based, and is blind to `pointer-events` and `z-index` by design: a decorative overlay carrying `pointer-events: none` still paints, and a z-stack query would report no ink over visible artwork.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="applyDraft" type="(id: string, props: DraftProps) => void">
|
||||
Visually translates the preview element at 60fps during a drag: sets the element's CSS `translate` to its pre-drag value composed with the accumulated delta. Works on GSAP-animated elements (a `translate` set after GSAP's first parse composes with the animated transform). The **SDK is not called here** — this is a direct write to the preview surface by your pointer-move handler. Switching `id` mid-drag reverts the previous element's draft first.
|
||||
</ParamField>
|
||||
@@ -159,8 +178,27 @@ interface DraftProps {
|
||||
|
||||
`dx` and `dy` are the accumulated drag deltas in composition pixels. `width` and `height` are defined in the interface for forward compatibility but are not yet wired to any op.
|
||||
|
||||
### PaintQueryOptions
|
||||
|
||||
```typescript
|
||||
interface PaintQueryOptions {
|
||||
fullBleedFraction?: number;
|
||||
addressableOnly?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
<ParamField path="fullBleedFraction" type="number" default="0">
|
||||
A hit whose smallest painting box covers at least this fraction of the composition frame reads as background rather than ink. This is host policy, not a fact about the composition: an editor that treats "you clicked a layer covering the whole frame" as "you clicked the background" passes `0.9`, while a caller asking the literal ink question leaves it at `0`. Nested sub-compositions carry `data-composition-id` too, so the reference frame is the innermost composition root containing the point.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="addressableOnly" type="boolean" default="true">
|
||||
Consider only model-addressable elements (`[data-hf-id]`). Stamping happens once, on the document `openComposition` was given, so anything the runtime creates or fetches afterwards is invisible to the default walk: split-text word and character spans (splitting also empties the stamped parent's own text nodes, so the parent stops counting too), cloned nodes, and whole sub-composition scenes mounted from `data-composition-src`. Kinetic typography and registry-mounted lower-thirds — both canonical transparent-overlay content — therefore read as no-ink by default.
|
||||
|
||||
Set `false` to widen the walk to every element, which sees that content at the cost of a larger candidate set.
|
||||
</ParamField>
|
||||
|
||||
<Note>
|
||||
`ElementAtPointResult` and `DraftProps` are the structural shapes a `PreviewAdapter` produces and consumes. They are **not** re-exported from the `@hyperframes/sdk` barrel — you implement against these shapes rather than importing them.
|
||||
`ElementAtPointResult` and `DraftProps` are the structural shapes a `PreviewAdapter` produces and consumes. They are **not** re-exported from the `@hyperframes/sdk` barrel — you implement against these shapes rather than importing them. `PaintQueryOptions` **is** re-exported, since callers pass it rather than implement it.
|
||||
</Note>
|
||||
|
||||
---
|
||||
@@ -258,7 +296,9 @@ import { createHeadlessAdapter } from "@hyperframes/sdk";
|
||||
function createHeadlessAdapter(): PreviewAdapter;
|
||||
```
|
||||
|
||||
Returns a no-op `PreviewAdapter` for headless use: agents, CI pipelines, and server-side rendering. All methods are stubs — `elementAtPoint` always returns `null`, `applyDraft` and `commitPreview` are no-ops, and the `"selection"` event never fires.
|
||||
Returns a no-op `PreviewAdapter` for headless use: agents, CI pipelines, and server-side rendering. All methods are stubs — `elementAtPoint` always returns `null` and `isProvablyEmptyAt` always returns `false`, `applyDraft` and `commitPreview` are no-ops, and the `"selection"` event never fires.
|
||||
|
||||
`isProvablyEmptyAt` returns `false` on purpose: an adapter with no surface cannot establish that a point is free of ink, and answering `true` would tell a host it is safe to click through a composition nobody can see.
|
||||
|
||||
Pass this adapter when you open a composition for programmatic editing and do not need a live preview surface.
|
||||
|
||||
@@ -297,6 +337,18 @@ Returns a `PreviewAdapter` that bridges the SDK to a same-origin `<iframe>` cont
|
||||
|
||||
**Image-alpha hit-testing:** For `<img>` elements, the adapter samples the alpha channel of the pixel under the pointer using an `OffscreenCanvas`. Transparent pixels fall through to the element behind. Cross-origin images that taint the canvas are treated as opaque (safe fallback, logged once per src).
|
||||
|
||||
**Paint queries:** `isProvablyEmptyAt` answers whether a point is safe to click through — see the [`PreviewAdapter` interface](#previewadapter) above and the [transparent-overlay recipe](/sdk/guides/canvas-integration#transparent-compositions-over-other-content). The pieces it is built from are importable directly for hosts whose hit-test policy differs:
|
||||
|
||||
```typescript
|
||||
import {
|
||||
elementPaintsInk,
|
||||
compositionPaintsAt,
|
||||
imageAlphaOpaqueAt,
|
||||
alphaIsOpaque,
|
||||
mapPointToImagePixel,
|
||||
} from "@hyperframes/sdk/adapters/iframe";
|
||||
```
|
||||
|
||||
```typescript
|
||||
import { openComposition, createIframePreviewAdapter } from "@hyperframes/sdk";
|
||||
|
||||
@@ -329,6 +381,8 @@ if (hit) {
|
||||
| `createMemoryAdapter` | `@hyperframes/sdk` |
|
||||
| `createHeadlessAdapter` | `@hyperframes/sdk` |
|
||||
| `createIframePreviewAdapter`, `resolveNearestHfElement` | `@hyperframes/sdk` |
|
||||
| `PaintQueryOptions` | `@hyperframes/sdk` (type only) |
|
||||
| `elementPaintsInk`, `compositionPaintsAt`, `imageAlphaOpaqueAt`, `alphaIsOpaque`, `mapPointToImagePixel`, `INTRINSIC_PAINT_TAGS` | `@hyperframes/sdk/adapters/iframe` |
|
||||
| `createFsAdapter`, `FsAdapterOptions` | `@hyperframes/sdk/adapters/fs` |
|
||||
|
||||
<CardGroup cols={2}>
|
||||
|
||||
@@ -25,6 +25,12 @@
|
||||
"types": "./dist/adapters/headless.d.ts",
|
||||
"environments": ["browser", "bun", "node"]
|
||||
},
|
||||
"./adapters/iframe": {
|
||||
"source": "./src/adapters/iframe.ts",
|
||||
"runtime": "./dist/adapters/iframe.js",
|
||||
"types": "./dist/adapters/iframe.d.ts",
|
||||
"environments": ["browser", "bun", "node"]
|
||||
},
|
||||
"./editing": {
|
||||
"source": "./src/editing/affordances.ts",
|
||||
"runtime": "./dist/editing/affordances.js",
|
||||
|
||||
@@ -34,6 +34,11 @@
|
||||
"import": "./src/adapters/headless.ts",
|
||||
"types": "./src/adapters/headless.ts"
|
||||
},
|
||||
"./adapters/iframe": {
|
||||
"bun": "./src/adapters/iframe.ts",
|
||||
"import": "./src/adapters/iframe.ts",
|
||||
"types": "./src/adapters/iframe.ts"
|
||||
},
|
||||
"./editing": {
|
||||
"bun": "./src/editing/affordances.ts",
|
||||
"import": "./src/editing/affordances.ts",
|
||||
@@ -59,6 +64,10 @@
|
||||
"import": "./dist/adapters/headless.js",
|
||||
"types": "./dist/adapters/headless.d.ts"
|
||||
},
|
||||
"./adapters/iframe": {
|
||||
"import": "./dist/adapters/iframe.js",
|
||||
"types": "./dist/adapters/iframe.d.ts"
|
||||
},
|
||||
"./editing": {
|
||||
"import": "./dist/editing/affordances.js",
|
||||
"types": "./dist/editing/affordances.d.ts"
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
|
||||
import { createHeadlessAdapter } from "./headless.js";
|
||||
|
||||
describe("createHeadlessAdapter", () => {
|
||||
it("is never provably empty", () => {
|
||||
// An adapter with no surface cannot establish that a point is free of ink. Answering
|
||||
// true would tell a host it is safe to click through a composition nobody can see.
|
||||
expect(createHeadlessAdapter().isProvablyEmptyAt?.(10, 10)).toBe(false);
|
||||
expect(createHeadlessAdapter().elementAtPoint(10, 10)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,9 @@
|
||||
import type { PreviewAdapter, ElementAtPointResult, DraftProps } from "./types.js";
|
||||
import type {
|
||||
PreviewAdapter,
|
||||
ElementAtPointResult,
|
||||
DraftProps,
|
||||
PaintQueryOptions,
|
||||
} from "./types.js";
|
||||
import type { Composition } from "../types.js";
|
||||
|
||||
/** Null PreviewAdapter for headless use (agents, CI, server-side rendering). */
|
||||
@@ -7,6 +12,15 @@ class HeadlessPreviewAdapter implements PreviewAdapter {
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Never provably empty: an adapter with no surface cannot establish that a point is
|
||||
* free of ink, and claiming otherwise would tell a host it is safe to click through a
|
||||
* composition nobody can see.
|
||||
*/
|
||||
isProvablyEmptyAt(_x: number, _y: number, _opts?: PaintQueryOptions): boolean {
|
||||
return false;
|
||||
}
|
||||
|
||||
applyDraft(_id: string, _props: DraftProps): void {}
|
||||
|
||||
commitPreview(): void {}
|
||||
|
||||
@@ -15,6 +15,10 @@
|
||||
* - z-stack fallthrough via mock elementsFromPoint
|
||||
* - canvas taint → opaque fallback
|
||||
* - non-image regression (WS-A1 opacity behavior unchanged)
|
||||
*
|
||||
* Paint-query tests (elementPaintsInk / compositionPaintsAt / isProvablyEmptyAt) live at
|
||||
* bottom of this file and carry their own fake-DOM helpers: the walk is geometric, so
|
||||
* they need element boxes and computed style rather than a hit-test stack.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
@@ -24,6 +28,8 @@ import {
|
||||
createIframePreviewAdapter,
|
||||
alphaIsOpaque,
|
||||
mapPointToImagePixel,
|
||||
elementPaintsInk,
|
||||
compositionPaintsAt,
|
||||
_imgCanvasCache,
|
||||
} from "./iframe.js";
|
||||
import type { ElementAtPointResult } from "./types.js";
|
||||
@@ -707,6 +713,8 @@ class FakeHTMLImageElement {
|
||||
attrs: Record<string, string>;
|
||||
tagName: string;
|
||||
parentElement: FakeHTMLImageElement | null;
|
||||
/** A clear pixel falls through to the style/own-text checks, which read this. */
|
||||
childNodes: Array<{ nodeType: number; textContent: string }> = [];
|
||||
naturalWidth: number;
|
||||
naturalHeight: number;
|
||||
currentSrc: string;
|
||||
@@ -896,3 +904,596 @@ describe("WS-G: z-stack fallthrough via mock elementsFromPoint", () => {
|
||||
expect(createIframePreviewAdapter(noStack).elementAtPoint(50, 50)).toEqual(expected);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Paint query: elementPaintsInk / compositionPaintsAt / isProvablyEmptyAt ───
|
||||
|
||||
/**
|
||||
* The paint query decides whether a host layering a transparent composition over
|
||||
* other content swallows a click or lets it through, so both directions matter: a
|
||||
* false negative makes visible artwork unclickable, a false positive makes the
|
||||
* composition opaque to interaction again.
|
||||
*
|
||||
* These need richer fakes than the resolver tests above — boxes, computed style and
|
||||
* a document to walk — because the walk is geometric rather than a z-stack.
|
||||
*/
|
||||
|
||||
interface PaintRect {
|
||||
left: number;
|
||||
top: number;
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
const PAINT_DEFAULT_STYLE: Record<string, string> = {
|
||||
display: "block",
|
||||
visibility: "visible",
|
||||
opacity: "1",
|
||||
backgroundColor: "rgba(0, 0, 0, 0)",
|
||||
backgroundImage: "none",
|
||||
};
|
||||
|
||||
const paintStyles = new WeakMap<object, Record<string, string>>();
|
||||
|
||||
function domRect(r: PaintRect): DOMRect {
|
||||
return {
|
||||
...r,
|
||||
right: r.left + r.width,
|
||||
bottom: r.top + r.height,
|
||||
} as DOMRect;
|
||||
}
|
||||
|
||||
interface PaintNode {
|
||||
tagName: string;
|
||||
attrs: Record<string, string>;
|
||||
parentElement: PaintNode | null;
|
||||
childNodes: Array<{ nodeType: number; textContent: string }>;
|
||||
children: PaintNode[];
|
||||
getBoundingClientRect(): DOMRect;
|
||||
getAttribute(name: string): string | null;
|
||||
hasAttribute(name: string): boolean;
|
||||
querySelector(selector: string): PaintNode | null;
|
||||
}
|
||||
|
||||
function pnode(init: {
|
||||
tag?: string;
|
||||
attrs?: Record<string, string>;
|
||||
style?: Record<string, string>;
|
||||
rect?: PaintRect;
|
||||
text?: string;
|
||||
parent?: PaintNode | null;
|
||||
}): PaintNode {
|
||||
const rect = init.rect ?? { left: 0, top: 0, width: 100, height: 100 };
|
||||
const node: PaintNode = {
|
||||
tagName: (init.tag ?? "div").toUpperCase(),
|
||||
attrs: { "data-hf-id": `hf-${init.tag ?? "div"}`, ...init.attrs },
|
||||
parentElement: init.parent ?? null,
|
||||
childNodes: init.text === undefined ? [] : [{ nodeType: 3, textContent: init.text }],
|
||||
children: [],
|
||||
getBoundingClientRect: () => domRect(rect),
|
||||
getAttribute(name) {
|
||||
return Object.prototype.hasOwnProperty.call(this.attrs, name) ? this.attrs[name] : null;
|
||||
},
|
||||
hasAttribute(name) {
|
||||
return Object.prototype.hasOwnProperty.call(this.attrs, name);
|
||||
},
|
||||
querySelector(selector) {
|
||||
return this.children.find((c) => c.tagName.toLowerCase() === selector) ?? null;
|
||||
},
|
||||
};
|
||||
if (init.style) paintStyles.set(node, init.style);
|
||||
init.parent?.children.push(node);
|
||||
return node;
|
||||
}
|
||||
|
||||
/** An image node the walk can pick up: a FakeHTMLImageElement with a box and a parent. */
|
||||
function pimg(
|
||||
id: string,
|
||||
rect: PaintRect,
|
||||
opts?: { parent?: PaintNode | null; src?: string; naturalWidth?: number },
|
||||
): FakeHTMLImageElement {
|
||||
const img = new FakeHTMLImageElement(id);
|
||||
img.getBoundingClientRect = () => domRect(rect);
|
||||
img.parentElement = (opts?.parent ?? null) as unknown as FakeHTMLImageElement | null;
|
||||
if (opts?.src) {
|
||||
img.src = opts.src;
|
||||
img.currentSrc = opts.src;
|
||||
}
|
||||
if (opts?.naturalWidth !== undefined) img.naturalWidth = opts.naturalWidth;
|
||||
return img;
|
||||
}
|
||||
|
||||
function paintWin(): Window & typeof globalThis {
|
||||
return {
|
||||
HTMLImageElement: FakeHTMLImageElement,
|
||||
getComputedStyle(el: object) {
|
||||
const merged = { ...PAINT_DEFAULT_STYLE, ...(paintStyles.get(el) ?? {}) };
|
||||
return {
|
||||
...merged,
|
||||
getPropertyValue(name: string) {
|
||||
if (merged[name] !== undefined) return merged[name];
|
||||
return name.endsWith("-width") ? "0px" : "none";
|
||||
},
|
||||
} as unknown as CSSStyleDeclaration;
|
||||
},
|
||||
} as unknown as Window & typeof globalThis;
|
||||
}
|
||||
|
||||
function paintDoc(nodes: object[], readyState = "complete"): Document {
|
||||
const matching = (selector: string) => {
|
||||
if (selector === "*") return nodes;
|
||||
const attr = selector.slice(1, -1);
|
||||
return nodes.filter((n) => (n as PaintNode).hasAttribute(attr));
|
||||
};
|
||||
return {
|
||||
readyState,
|
||||
querySelectorAll: matching,
|
||||
querySelector: (selector: string) => matching(selector)[0] ?? null,
|
||||
} as unknown as Document;
|
||||
}
|
||||
|
||||
/** OffscreenCanvas stub that counts pixel reads, for the lazy-sampling guard. */
|
||||
function stubCountingCanvas(): { restore: () => void; reads: () => number } {
|
||||
const orig = globalThis.OffscreenCanvas as typeof OffscreenCanvas | undefined;
|
||||
let reads = 0;
|
||||
globalThis.OffscreenCanvas = class {
|
||||
constructor(
|
||||
public width: number,
|
||||
public height: number,
|
||||
) {}
|
||||
getContext(_type: string) {
|
||||
return {
|
||||
drawImage() {},
|
||||
getImageData() {
|
||||
reads++;
|
||||
return { data: new Uint8ClampedArray([255, 0, 0, 255]), width: 1, height: 1 };
|
||||
},
|
||||
};
|
||||
}
|
||||
} as unknown as typeof OffscreenCanvas;
|
||||
return {
|
||||
restore: () => {
|
||||
if (orig === undefined) {
|
||||
delete (globalThis as Record<string, unknown>).OffscreenCanvas;
|
||||
} else {
|
||||
globalThis.OffscreenCanvas = orig;
|
||||
}
|
||||
},
|
||||
reads: () => reads,
|
||||
};
|
||||
}
|
||||
|
||||
const el = (n: PaintNode) => n as unknown as Element;
|
||||
|
||||
describe("elementPaintsInk", () => {
|
||||
const win = paintWin();
|
||||
|
||||
it("treats a bare layout wrapper as NOT painting", () => {
|
||||
// The common case, and the reason this exists: a composition is mostly full-bleed
|
||||
// wrappers with no visual presence, and every one covers what sits beneath.
|
||||
expect(elementPaintsInk(el(pnode({})), win)).toBe(false);
|
||||
});
|
||||
|
||||
it("counts a background colour, but not a transparent one", () => {
|
||||
expect(elementPaintsInk(el(pnode({ style: { backgroundColor: "#ff0000" } })), win)).toBe(true);
|
||||
expect(elementPaintsInk(el(pnode({ style: { backgroundColor: "rgba(0,0,0,0)" } })), win)).toBe(
|
||||
false,
|
||||
);
|
||||
expect(elementPaintsInk(el(pnode({ style: { backgroundColor: "transparent" } })), win)).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("reads the alpha, so a NON-BLACK zero-alpha colour is still no ink", () => {
|
||||
// Only the `transparent` keyword computes to rgba(0,0,0,0); a faded-out white stays
|
||||
// rgba(255, 255, 255, 0), which a set of known spellings counts as painted.
|
||||
for (const color of [
|
||||
"rgba(255, 255, 255, 0)",
|
||||
"rgba(12, 34, 56, 0.0)",
|
||||
"rgb(255 255 255 / 0)",
|
||||
"hsla(0, 0%, 0%, 0)",
|
||||
"hsl(120 50% 50% / 0)",
|
||||
]) {
|
||||
expect(elementPaintsInk(el(pnode({ style: { backgroundColor: color } })), win), color).toBe(
|
||||
false,
|
||||
);
|
||||
}
|
||||
// A non-zero alpha still paints, however faint.
|
||||
expect(
|
||||
elementPaintsInk(el(pnode({ style: { backgroundColor: "rgba(255, 255, 255, 0.01)" } })), win),
|
||||
).toBe(true);
|
||||
// rgb() with no alpha component is opaque.
|
||||
expect(elementPaintsInk(el(pnode({ style: { backgroundColor: "rgb(1, 2, 3)" } })), win)).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("counts a background image", () => {
|
||||
expect(elementPaintsInk(el(pnode({ style: { backgroundImage: "url(a.png)" } })), win)).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("counts a visible border but not a zero-width or `none` one", () => {
|
||||
const bordered = pnode({
|
||||
style: { "border-top-style": "solid", "border-top-width": "2px" },
|
||||
});
|
||||
expect(elementPaintsInk(el(bordered), win)).toBe(true);
|
||||
|
||||
const zeroWidth = pnode({
|
||||
style: { "border-top-style": "solid", "border-top-width": "0px" },
|
||||
});
|
||||
expect(elementPaintsInk(el(zeroWidth), win)).toBe(false);
|
||||
|
||||
const styleNone = pnode({
|
||||
style: { "border-top-style": "none", "border-top-width": "2px" },
|
||||
});
|
||||
expect(elementPaintsInk(el(styleNone), win)).toBe(false);
|
||||
});
|
||||
|
||||
it("counts intrinsic media regardless of styling", () => {
|
||||
for (const tag of ["video", "canvas", "svg"]) {
|
||||
expect(elementPaintsInk(el(pnode({ tag })), win), tag).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("counts an element's OWN text, not text belonging to a descendant", () => {
|
||||
expect(elementPaintsInk(el(pnode({ tag: "span", text: "Clean" })), win)).toBe(true);
|
||||
|
||||
// Without the own-text rule every ancestor of a caption would count as painting,
|
||||
// and the whole frame would read as covered.
|
||||
const wrapper = pnode({});
|
||||
pnode({ tag: "span", text: "Clean", parent: wrapper });
|
||||
expect(elementPaintsInk(el(wrapper), win)).toBe(false);
|
||||
|
||||
expect(elementPaintsInk(el(pnode({ text: " \n " })), win)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("elementPaintsInk: <img> routes through the alpha sampler", () => {
|
||||
beforeEach(() => {
|
||||
_imgCanvasCache.clear();
|
||||
});
|
||||
|
||||
const rect = { left: 0, top: 0, width: 100, height: 100 };
|
||||
|
||||
/** Ink at the centre of a 100×100 image, under a controlled canvas behaviour. */
|
||||
function inkAtCentre(
|
||||
behavior: CanvasAlphaBehavior,
|
||||
build: () => Element = () => pimg("hf-img", rect) as unknown as Element,
|
||||
): boolean {
|
||||
const restore = stubOffscreenCanvas(behavior);
|
||||
try {
|
||||
return elementPaintsInk(build(), paintWin(), { x: 50, y: 50 });
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
}
|
||||
|
||||
it("an opaque pixel paints", () => {
|
||||
expect(inkAtCentre("opaque")).toBe(true);
|
||||
});
|
||||
|
||||
it("a transparent pixel does NOT paint — the whole point of the alpha path", () => {
|
||||
expect(inkAtCentre("transparent")).toBe(false);
|
||||
});
|
||||
|
||||
it("a tainted canvas fails safe to painted", () => {
|
||||
expect(inkAtCentre("tainted")).toBe(true);
|
||||
});
|
||||
|
||||
it("an unloaded image fails safe to painted", () => {
|
||||
expect(
|
||||
inkAtCentre(
|
||||
"transparent",
|
||||
() => pimg("hf-img", rect, { naturalWidth: 0 }) as unknown as Element,
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("falls back to the tag rule when no point is supplied", () => {
|
||||
// Point-free callers are asking "could this paint at all", which for an image is yes.
|
||||
expect(elementPaintsInk(pimg("hf-img", rect) as unknown as Element, paintWin())).toBe(true);
|
||||
});
|
||||
|
||||
it("<picture> defers to the <img> it wraps rather than painting unconditionally", () => {
|
||||
const withInnerImg = () => {
|
||||
const picture = pnode({ tag: "picture" });
|
||||
picture.children.push(pimg("hf-inner", rect) as unknown as PaintNode);
|
||||
return el(picture);
|
||||
};
|
||||
expect(inkAtCentre("transparent", withInnerImg)).toBe(false);
|
||||
});
|
||||
|
||||
it("a clear pixel still paints when the image itself has background or border ink", () => {
|
||||
// object-fit letterbox over a white plate: the bitmap misses, the chip is still visible.
|
||||
const restore = stubOffscreenCanvas("transparent");
|
||||
try {
|
||||
const img = pimg("hf-chip", rect);
|
||||
paintStyles.set(img, { backgroundColor: "#fff" });
|
||||
expect(elementPaintsInk(img as unknown as Element, paintWin(), { x: 50, y: 50 })).toBe(true);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
it("<picture> with no inner <img> still paints", () => {
|
||||
expect(elementPaintsInk(el(pnode({ tag: "picture" })), paintWin(), { x: 1, y: 1 })).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("compositionPaintsAt", () => {
|
||||
beforeEach(() => {
|
||||
_imgCanvasCache.clear();
|
||||
});
|
||||
|
||||
const FRAME: PaintRect = { left: 0, top: 0, width: 1000, height: 1000 };
|
||||
const root = () => pnode({ attrs: { "data-composition-id": "main" }, rect: FRAME });
|
||||
|
||||
it("an unpainted full-bleed wrapper does not mask a painted child", () => {
|
||||
const nodes = [
|
||||
root(),
|
||||
pnode({ rect: FRAME }),
|
||||
pnode({
|
||||
attrs: { "data-hf-id": "hf-sun" },
|
||||
style: { backgroundColor: "#ff0" },
|
||||
rect: { left: 100, top: 100, width: 180, height: 180 },
|
||||
}),
|
||||
];
|
||||
const paints = compositionPaintsAt(paintDoc(nodes), paintWin(), 150, 150, {
|
||||
fullBleedFraction: 0.9,
|
||||
});
|
||||
expect(paints).toBe(true);
|
||||
});
|
||||
|
||||
it("reports no ink where only an unpainted wrapper sits", () => {
|
||||
const nodes = [root(), pnode({ rect: FRAME })];
|
||||
expect(compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500)).toBe(false);
|
||||
});
|
||||
|
||||
it("reports no ink outside every box", () => {
|
||||
const nodes = [
|
||||
root(),
|
||||
pnode({
|
||||
style: { backgroundColor: "#f00" },
|
||||
rect: { left: 0, top: 0, width: 10, height: 10 },
|
||||
}),
|
||||
];
|
||||
expect(compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500)).toBe(false);
|
||||
});
|
||||
|
||||
it("fullBleedFraction rejects a lone full-bleed painter; the default accepts it", () => {
|
||||
const nodes = [root(), pnode({ style: { backgroundColor: "rgba(0,0,0,0.3)" }, rect: FRAME })];
|
||||
// Host policy: an editor reads "you clicked a layer covering the whole frame" as
|
||||
// "you clicked the background".
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500, { fullBleedFraction: 0.9 }),
|
||||
).toBe(false);
|
||||
// The literal ink question: a full-frame scrim does put ink there.
|
||||
expect(compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500)).toBe(true);
|
||||
});
|
||||
|
||||
it("measures full-bleed against the innermost root CONTAINING the point", () => {
|
||||
// A 300×300 painter really is full-bleed inside the 300×300 sub-composition it lives
|
||||
// in, even though it covers 9% of the outer frame.
|
||||
const nested: PaintRect = { left: 0, top: 0, width: 300, height: 300 };
|
||||
const nodes = [
|
||||
root(),
|
||||
pnode({ attrs: { "data-composition-id": "inner" }, rect: nested }),
|
||||
pnode({ style: { backgroundColor: "#f00" }, rect: nested }),
|
||||
];
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 150, 150, { fullBleedFraction: 0.9 }),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("never vetoes a PIXEL-VERIFIED hit, however large the box", () => {
|
||||
// The flagship shape: a full-frame transparent PNG overlay. Its box IS the frame, so an
|
||||
// area-based veto called every opaque pixel "background" and the visible artwork became
|
||||
// unclickable. Box area is not ink area once the pixel has actually been read.
|
||||
const restore = stubOffscreenCanvas("opaque");
|
||||
try {
|
||||
const frame: PaintRect = { left: 0, top: 0, width: 1000, height: 1000 };
|
||||
const nodes = [root(), pimg("hf-overlay", frame)];
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500, { fullBleedFraction: 0.9 }),
|
||||
).toBe(true);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
it("still vetoes a full-bleed image whose pixels could NOT be read", () => {
|
||||
// A tainted cross-origin image is a fail-safe "opaque", not a measurement. Exempting it
|
||||
// would make every full-frame CDN-backed overlay permanently absorb clicks.
|
||||
const restore = stubOffscreenCanvas("tainted");
|
||||
try {
|
||||
const frame: PaintRect = { left: 0, top: 0, width: 1000, height: 1000 };
|
||||
const nodes = [root(), pimg("hf-overlay", frame)];
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500, { fullBleedFraction: 0.9 }),
|
||||
).toBe(false);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
it("counts ink painted by the composition root itself", () => {
|
||||
// A root carrying a background is painting, and at fraction 0 the literal ink question
|
||||
// has to say so. A non-zero fraction still discounts it: a root's box IS the frame.
|
||||
const painted = () =>
|
||||
pnode({
|
||||
attrs: { "data-composition-id": "main", "data-hf-id": "hf-root" },
|
||||
style: { backgroundColor: "#111" },
|
||||
rect: { left: 0, top: 0, width: 1000, height: 1000 },
|
||||
});
|
||||
expect(compositionPaintsAt(paintDoc([painted()]), paintWin(), 500, 500)).toBe(true);
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc([painted()]), paintWin(), 500, 500, { fullBleedFraction: 0.9 }),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("ignores a sub-composition the point is NOT inside when sizing the frame", () => {
|
||||
// Regression: taking the smallest root in the DOCUMENT let a badge in one corner set the
|
||||
// reference frame for the whole canvas, so mid-size artwork out in the 1000×1000 outer
|
||||
// frame measured against 300×300, read as full-bleed, and went click-through under
|
||||
// something the user can plainly see — the unsafe direction.
|
||||
const badge: PaintRect = { left: 0, top: 0, width: 300, height: 300 };
|
||||
const nodes = [
|
||||
root(),
|
||||
pnode({ attrs: { "data-composition-id": "badge" }, rect: badge }),
|
||||
pnode({
|
||||
style: { backgroundColor: "#f00" },
|
||||
rect: { left: 500, top: 500, width: 300, height: 300 },
|
||||
}),
|
||||
];
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 600, 600, { fullBleedFraction: 0.9 }),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("disables the full-bleed rule when no root contains the point", () => {
|
||||
const nodes = [
|
||||
pnode({
|
||||
attrs: { "data-composition-id": "elsewhere" },
|
||||
rect: { left: 0, top: 0, width: 10, height: 10 },
|
||||
}),
|
||||
pnode({
|
||||
style: { backgroundColor: "#f00" },
|
||||
rect: { left: 400, top: 400, width: 200, height: 200 },
|
||||
}),
|
||||
];
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 500, 500, { fullBleedFraction: 0.9 }),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("skips display:none, visibility:hidden and zero-area boxes", () => {
|
||||
const painted = { backgroundColor: "#f00" };
|
||||
const box = { left: 0, top: 0, width: 100, height: 100 };
|
||||
for (const style of [
|
||||
{ ...painted, display: "none" },
|
||||
{ ...painted, visibility: "hidden" },
|
||||
]) {
|
||||
const nodes = [root(), pnode({ style, rect: box })];
|
||||
expect(compositionPaintsAt(paintDoc(nodes), paintWin(), 50, 50)).toBe(false);
|
||||
}
|
||||
const collapsed = [
|
||||
root(),
|
||||
pnode({ style: painted, rect: { left: 0, top: 0, width: 0, height: 0 } }),
|
||||
];
|
||||
expect(compositionPaintsAt(paintDoc(collapsed), paintWin(), 0, 0)).toBe(false);
|
||||
});
|
||||
|
||||
it("skips an element hidden by an ANCESTOR's opacity:0", () => {
|
||||
// getComputedStyle does not multiply the cascade, so a fully opaque child inside a
|
||||
// fade-in wrapper that has not started yet reads as painting unless the check walks up.
|
||||
const wrapper = pnode({
|
||||
style: { opacity: "0" },
|
||||
rect: { left: 0, top: 0, width: 200, height: 200 },
|
||||
});
|
||||
const child = pnode({
|
||||
attrs: { "data-hf-id": "hf-child" },
|
||||
style: { backgroundColor: "#f00" },
|
||||
rect: { left: 0, top: 0, width: 100, height: 100 },
|
||||
parent: wrapper,
|
||||
});
|
||||
expect(compositionPaintsAt(paintDoc([root(), wrapper, child]), paintWin(), 50, 50)).toBe(false);
|
||||
});
|
||||
|
||||
it("addressableOnly:false picks up a node the stamping pass never saw", () => {
|
||||
// Runtime-generated nodes (split-text word spans, clones) carry no data-hf-id.
|
||||
const generated = pnode({
|
||||
tag: "span",
|
||||
attrs: {},
|
||||
text: "word",
|
||||
rect: { left: 0, top: 0, width: 50, height: 20 },
|
||||
});
|
||||
delete generated.attrs["data-hf-id"];
|
||||
const nodes = [root(), generated];
|
||||
expect(compositionPaintsAt(paintDoc(nodes), paintWin(), 10, 10)).toBe(false);
|
||||
expect(
|
||||
compositionPaintsAt(paintDoc(nodes), paintWin(), 10, 10, { addressableOnly: false }),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("samples alpha lazily — the smallest painting box wins before the rest are read", () => {
|
||||
// Guards the per-frame cost: ink is evaluated down the ordered candidates, so a hit
|
||||
// on the most specific element must not have already paid for the ones behind it.
|
||||
const canvas = stubCountingCanvas();
|
||||
try {
|
||||
const nodes = [
|
||||
root(),
|
||||
pimg("hf-big", { left: 0, top: 0, width: 400, height: 400 }, { src: "http://x/big.png" }),
|
||||
pimg(
|
||||
"hf-small",
|
||||
{ left: 0, top: 0, width: 100, height: 100 },
|
||||
{ src: "http://x/small.png" },
|
||||
),
|
||||
];
|
||||
expect(compositionPaintsAt(paintDoc(nodes), paintWin(), 50, 50)).toBe(true);
|
||||
expect(canvas.reads()).toBe(1);
|
||||
} finally {
|
||||
canvas.restore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("IframePreviewAdapter.isProvablyEmptyAt", () => {
|
||||
beforeEach(() => {
|
||||
_imgCanvasCache.clear();
|
||||
});
|
||||
|
||||
const iframeWith = (doc: unknown, win: unknown) =>
|
||||
({ contentDocument: doc, contentWindow: win }) as unknown as HTMLIFrameElement;
|
||||
|
||||
it("is never provably empty when the document is not reachable", () => {
|
||||
// Unknowable is not empty: the composition stays clickable rather than vanishing.
|
||||
expect(createIframePreviewAdapter(iframeWith(null, paintWin())).isProvablyEmptyAt(1, 1)).toBe(
|
||||
false,
|
||||
);
|
||||
expect(createIframePreviewAdapter(iframeWith(paintDoc([]), null)).isProvablyEmptyAt(1, 1)).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("is never provably empty when contentDocument access throws (cross-origin)", () => {
|
||||
const hostile = {} as HTMLIFrameElement;
|
||||
Object.defineProperty(hostile, "contentDocument", {
|
||||
get() {
|
||||
throw new DOMException("Blocked a frame", "SecurityError");
|
||||
},
|
||||
});
|
||||
expect(createIframePreviewAdapter(hostile).isProvablyEmptyAt(1, 1)).toBe(false);
|
||||
});
|
||||
|
||||
it("is never provably empty while the document is still loading", () => {
|
||||
// A same-origin iframe mid-navigation is READABLE and empty, so the !doc guard never
|
||||
// fires — walking it would answer a confident "no ink" under an arriving composition.
|
||||
const stamped = [pnode({ attrs: { "data-hf-id": "hf-a" } })];
|
||||
expect(
|
||||
createIframePreviewAdapter(
|
||||
iframeWith(paintDoc(stamped, "loading"), paintWin()),
|
||||
).isProvablyEmptyAt(1, 1),
|
||||
).toBe(false);
|
||||
// Loaded but carrying nothing of the composition — equally unknowable.
|
||||
expect(
|
||||
createIframePreviewAdapter(iframeWith(paintDoc([]), paintWin())).isProvablyEmptyAt(1, 1),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("inverts the walk: empty where nothing paints, not empty over ink", () => {
|
||||
const nodes = [
|
||||
pnode({
|
||||
attrs: { "data-composition-id": "main" },
|
||||
rect: { left: 0, top: 0, width: 500, height: 500 },
|
||||
}),
|
||||
pnode({
|
||||
style: { backgroundColor: "#f00" },
|
||||
rect: { left: 0, top: 0, width: 100, height: 100 },
|
||||
}),
|
||||
];
|
||||
const adapter = createIframePreviewAdapter(iframeWith(paintDoc(nodes), paintWin()));
|
||||
expect(adapter.isProvablyEmptyAt(50, 50)).toBe(false); // over the painted box
|
||||
expect(adapter.isProvablyEmptyAt(400, 400)).toBe(true); // empty region
|
||||
});
|
||||
});
|
||||
|
||||
@@ -26,6 +26,16 @@
|
||||
* only when currentSrc/dimensions change. Phase 1 is optimized for static images.
|
||||
* - Phase 2 (full per-pixel alpha via drawElement rasterization) is NOT built
|
||||
* here — gated on a perf spike.
|
||||
*
|
||||
* Paint query (isProvablyEmptyAt):
|
||||
* - Answers "is this point provably free of ink", which is a different question from
|
||||
* "what element is here" and needs a different traversal. elementsFromPoint omits
|
||||
* `pointer-events: none` nodes; those still paint, so a paint query built on the
|
||||
* z-stack reports no ink over visible artwork. The walk therefore covers element
|
||||
* BOXES geometrically and is blind to pointer-events and z-index.
|
||||
* - Ink itself is a computed-style heuristic (background, border, own text, intrinsic
|
||||
* media) EXCEPT for <img>, which routes through the alpha sampler above — so a
|
||||
* transparent PNG only paints where its pixels do.
|
||||
*/
|
||||
|
||||
import {
|
||||
@@ -36,7 +46,12 @@ import {
|
||||
composeTranslate,
|
||||
readCurrentTranslate,
|
||||
} from "@hyperframes/core/runtime/position-edits";
|
||||
import type { PreviewAdapter, ElementAtPointResult, DraftProps } from "./types.js";
|
||||
import type {
|
||||
PreviewAdapter,
|
||||
ElementAtPointResult,
|
||||
DraftProps,
|
||||
PaintQueryOptions,
|
||||
} from "./types.js";
|
||||
import type { EditOp, Composition } from "../types.js";
|
||||
import { applyPatchesToDocument, applyOverrideSet } from "../engine/apply-patches.js";
|
||||
|
||||
@@ -374,11 +389,14 @@ function hasRotationOrSkew(el: Element | null, win: Window & typeof globalThis):
|
||||
* element which lives in the iframe's document.
|
||||
*/
|
||||
// fallow-ignore-next-line complexity
|
||||
function imageAlphaOpaqueAt(
|
||||
export function imageAlphaOpaqueAt(
|
||||
img: HTMLImageElement,
|
||||
clientX: number,
|
||||
clientY: number,
|
||||
win: Window & typeof globalThis,
|
||||
/** Set to true only when pixels were actually read (or the point provably missed the
|
||||
* rendered image). Lets callers tell a measured answer from a fail-safe assumption. */
|
||||
probe?: { sampled: boolean },
|
||||
): boolean {
|
||||
// Not loaded yet — treat as opaque (safe fallback)
|
||||
if (img.naturalWidth === 0 || img.naturalHeight === 0) return true;
|
||||
@@ -425,8 +443,12 @@ function imageAlphaOpaqueAt(
|
||||
);
|
||||
|
||||
// Point is outside the rendered image area — not this image's pixel.
|
||||
// Continue the z-stack (return false = miss on this element).
|
||||
if (mapped === null) return false;
|
||||
// Continue the z-stack (return false = miss on this element). Geometric certainty,
|
||||
// so it counts as sampled.
|
||||
if (mapped === null) {
|
||||
if (probe) probe.sampled = true;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Retrieve or build the offscreen canvas. Key on src + natural dimensions: a
|
||||
// srcset/responsive layout can serve the same URL at a different natural size,
|
||||
@@ -463,6 +485,7 @@ function imageAlphaOpaqueAt(
|
||||
// The mapped-pixel read also surfaces lazy canvas taint (SecurityError),
|
||||
// so no separate taint probe is needed.
|
||||
const data = ctx.getImageData(mapped.px, mapped.py, 1, 1);
|
||||
if (probe) probe.sampled = true;
|
||||
return alphaIsOpaque(data);
|
||||
} catch {
|
||||
// Taint discovered on getImageData — update cache and fall back opaque.
|
||||
@@ -472,6 +495,296 @@ function imageAlphaOpaqueAt(
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Paint query ──────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Elements that paint by virtue of what they are, whatever their CSS says.
|
||||
*
|
||||
* `img` and `picture` look redundant against the branches above them in `elementPaintsInk`,
|
||||
* and are not: those branches need a `point` to sample alpha against, so a point-free call
|
||||
* ("could this paint at all?") falls through to this set — as does any environment without
|
||||
* `window.HTMLImageElement`. The set is also exported, where it reads as a description of
|
||||
* intrinsic painters rather than a switch in one function.
|
||||
*/
|
||||
export const INTRINSIC_PAINT_TAGS: ReadonlySet<string> = new Set([
|
||||
"img",
|
||||
"picture",
|
||||
"video",
|
||||
"canvas",
|
||||
"svg",
|
||||
]);
|
||||
|
||||
/**
|
||||
* Does this computed colour put down no ink?
|
||||
*
|
||||
* The `transparent` keyword computes to `rgba(0, 0, 0, 0)`, but ANY colour can carry a zero
|
||||
* alpha — `rgba(255, 255, 255, 0)` is exactly as invisible and is what you get from fading a
|
||||
* white background out. Matching known spellings misses those, so the alpha is read instead.
|
||||
*/
|
||||
function isTransparentColor(value: string): boolean {
|
||||
if (!value || value === "transparent") return true;
|
||||
// rgb/hsl and their -a forms all carry alpha as the fourth component. Computed
|
||||
// background-color is serialized to rgb() by every engine we target, but matching both
|
||||
// costs one alternation and removes the dependency on that.
|
||||
const inner = /^(?:rgba?|hsla?)\(([^)]*)\)$/.exec(value)?.[1];
|
||||
if (inner === undefined) return false;
|
||||
// Handles both the legacy comma form and the `rgb(r g b / a)` slash form.
|
||||
const parts = inner.split(/[\s,/]+/).filter(Boolean);
|
||||
// An rgb() with no alpha component is fully opaque.
|
||||
const alpha = parts[3];
|
||||
return alpha !== undefined && Number.parseFloat(alpha) === 0;
|
||||
}
|
||||
|
||||
const BORDER_SIDES = ["top", "right", "bottom", "left"] as const;
|
||||
|
||||
/**
|
||||
* The element's OWN non-whitespace text — a direct child text node, not text that
|
||||
* lives in a descendant. A wrapper around a caption doesn't paint; the caption does.
|
||||
* Without this, every ancestor of a text node would count and the whole frame would
|
||||
* read as painted.
|
||||
*/
|
||||
function hasOwnText(el: Element): boolean {
|
||||
for (const node of Array.from(el.childNodes)) {
|
||||
if (node.nodeType === 3 /* TEXT_NODE */ && (node.textContent ?? "").trim() !== "") return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Does this element put ink on the frame, and — when `point` is supplied and the
|
||||
* element is an image — ink at that specific point?
|
||||
*
|
||||
* Mostly a heuristic over computed style, because a composition's ordinary elements
|
||||
* cannot be sampled without rasterizing them. `<img>` is the exception: the alpha
|
||||
* sampler above reads the actual pixel, so a transparent PNG paints only where its
|
||||
* pixels do. `<picture>` defers to the `<img>` it wraps, since the wrapper itself
|
||||
* paints nothing.
|
||||
*
|
||||
* KNOWN OVER-COUNT: a `background-image` that is itself mostly transparent reads as
|
||||
* painting across its whole box. Narrowing it needs a second sampling path (resolve
|
||||
* the `url()`, load it, map through background-size/position). Erring toward "paints"
|
||||
* is the safe direction — the cost is a click that selects the composition, not a
|
||||
* click that vanishes.
|
||||
*
|
||||
* Transparency by CSS (`display: none`, `visibility: hidden`, `opacity: 0`) is NOT
|
||||
* handled here; `compositionPaintsAt` filters those before asking.
|
||||
*/
|
||||
/**
|
||||
* How an element's ink was established.
|
||||
*
|
||||
* `"verified"` means pixels were read. `"inferred"` means computed style says something
|
||||
* paints across the box — true of a `background-image` that is mostly transparent, and of
|
||||
* an image whose pixels could not be sampled. The distinction is what lets the full-bleed
|
||||
* veto discount assumed box-filling paint without discarding a measured pixel.
|
||||
*/
|
||||
type InkKind = "none" | "verified" | "inferred";
|
||||
|
||||
/** Any side with a drawn border — a style that renders and a non-zero width. */
|
||||
function hasVisibleBorder(cs: CSSStyleDeclaration): boolean {
|
||||
for (const side of BORDER_SIDES) {
|
||||
const style = cs.getPropertyValue(`border-${side}-style`);
|
||||
const width = Number.parseFloat(cs.getPropertyValue(`border-${side}-width`) || "0");
|
||||
if (style && style !== "none" && style !== "hidden" && width > 0) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Computed-style ink: background, border, or the element's own text. */
|
||||
function styleInk(el: Element, win: Window & typeof globalThis): InkKind {
|
||||
const cs = win.getComputedStyle(el);
|
||||
|
||||
if (!isTransparentColor(cs.backgroundColor)) return "inferred";
|
||||
if (cs.backgroundImage && cs.backgroundImage !== "none") return "inferred";
|
||||
if (hasVisibleBorder(cs)) return "inferred";
|
||||
|
||||
return hasOwnText(el) ? "inferred" : "none";
|
||||
}
|
||||
|
||||
function inkAt(
|
||||
el: Element,
|
||||
win: Window & typeof globalThis,
|
||||
point?: { x: number; y: number },
|
||||
): InkKind {
|
||||
const tag = el.tagName.toLowerCase();
|
||||
|
||||
if (tag === "picture") {
|
||||
const img = el.querySelector("img");
|
||||
return img ? inkAt(img, win, point) : "inferred";
|
||||
}
|
||||
|
||||
if (point && win.HTMLImageElement && el instanceof win.HTMLImageElement) {
|
||||
const probe = { sampled: false };
|
||||
if (imageAlphaOpaqueAt(el, point.x, point.y, win, probe)) {
|
||||
// An unsampled "opaque" is the fail-safe, not a measurement, so it stays inferred.
|
||||
return probe.sampled ? "verified" : "inferred";
|
||||
}
|
||||
// A clear pixel does not settle the element: its own background plate, padding and
|
||||
// border still paint, and the point may have landed on them rather than the bitmap.
|
||||
return styleInk(el, win);
|
||||
}
|
||||
|
||||
if (INTRINSIC_PAINT_TAGS.has(tag)) return "inferred";
|
||||
|
||||
return styleInk(el, win);
|
||||
}
|
||||
|
||||
export function elementPaintsInk(
|
||||
el: Element,
|
||||
win: Window & typeof globalThis,
|
||||
point?: { x: number; y: number },
|
||||
): boolean {
|
||||
return inkAt(el, win, point) !== "none";
|
||||
}
|
||||
|
||||
interface PaintCandidate {
|
||||
el: Element;
|
||||
area: number;
|
||||
depth: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Area of the composition frame the point sits in — what a full-bleed layer is measured
|
||||
* against. Nested sub-compositions carry `data-composition-id` too, so the innermost
|
||||
* root CONTAINING the point wins.
|
||||
*
|
||||
* Containment is the load-bearing part. Taking the smallest root in the document
|
||||
* regardless of where the point falls lets an unrelated sub-composition elsewhere on the
|
||||
* frame shrink the reference: a 300x300 badge in a corner would make every mid-size
|
||||
* painter in the 1920x1080 outer frame read as full-bleed, and the composition would go
|
||||
* click-through under artwork the user can plainly see.
|
||||
*
|
||||
* Infinity when no root contains the point, which disables the full-bleed rule rather
|
||||
* than guessing at a frame.
|
||||
*/
|
||||
function compositionFrameArea(doc: Document, x: number, y: number): number {
|
||||
let smallest = Infinity;
|
||||
doc.querySelectorAll("[data-composition-id]").forEach((root) => {
|
||||
const rect = root.getBoundingClientRect();
|
||||
if (rect.width <= 0 || rect.height <= 0) return;
|
||||
if (!boxContains(rect, x, y)) return;
|
||||
smallest = Math.min(smallest, rect.width * rect.height);
|
||||
});
|
||||
return smallest;
|
||||
}
|
||||
|
||||
function boxContains(rect: DOMRect, x: number, y: number): boolean {
|
||||
return x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom;
|
||||
}
|
||||
|
||||
/**
|
||||
* Is the element rendered at all?
|
||||
*
|
||||
* `display: none` is checked belt-and-braces: a real engine collapses the box and the area
|
||||
* guard already rejects it, so the clause only earns its place against hosts that report a
|
||||
* box anyway. `visibility` and `opacity` genuinely need it, and opacity has to be read up
|
||||
* the ancestor chain — a fully opaque child inside a fade-in wrapper that has not started
|
||||
* yet is invisible, and getComputedStyle does not multiply the cascade for us.
|
||||
*/
|
||||
function isRenderedVisible(el: Element, win: Window & typeof globalThis): boolean {
|
||||
const cs = win.getComputedStyle(el);
|
||||
if (cs.display === "none" || cs.visibility === "hidden") return false;
|
||||
return isOpacityVisible(el, win);
|
||||
}
|
||||
|
||||
function depthOf(el: Element): number {
|
||||
let depth = 0;
|
||||
for (let p = el.parentElement; p; p = p.parentElement) depth++;
|
||||
return depth;
|
||||
}
|
||||
|
||||
/**
|
||||
* `el` as a paint candidate for the point, or null when it cannot contribute ink
|
||||
* there. Ink itself is NOT tested here — that is the expensive part, deferred until
|
||||
* the candidates are ordered.
|
||||
*/
|
||||
function paintCandidateAt(
|
||||
el: Element,
|
||||
win: Window & typeof globalThis,
|
||||
x: number,
|
||||
y: number,
|
||||
): PaintCandidate | null {
|
||||
// Composition roots are candidates like anything else: a root carrying a background is
|
||||
// painting, and at fullBleedFraction 0 the literal ink question has to say so. They are
|
||||
// not excluded here because the veto already handles them — a root's box IS the frame,
|
||||
// so any non-zero fraction discounts it as background without a special case.
|
||||
const rect = el.getBoundingClientRect();
|
||||
if (rect.width <= 0 || rect.height <= 0) return null;
|
||||
if (!boxContains(rect, x, y)) return null;
|
||||
if (!isRenderedVisible(el, win)) return null;
|
||||
|
||||
return { el, area: rect.width * rect.height, depth: depthOf(el) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Does the composition in `doc` put ink at (x, y)?
|
||||
*
|
||||
* A GEOMETRIC walk, not a z-stack one: every element whose border box contains the
|
||||
* point is a candidate, regardless of pointer-events, z-index, or what paints over
|
||||
* it. That is what makes the answer "is there artwork here" rather than "what would
|
||||
* receive this click" — an author's `pointer-events: none` overlay still paints.
|
||||
*
|
||||
* Candidates are ordered smallest-box-first (deeper node wins a near-tie, matching
|
||||
* the way a host picks the most specific element) and ink is evaluated lazily down
|
||||
* that order, so the common case costs at most one alpha sample rather than one per
|
||||
* candidate.
|
||||
*
|
||||
* Exported so a host whose hit-test policy differs can reuse the ink test without
|
||||
* taking the adapter with it. Two preconditions come with that:
|
||||
*
|
||||
* - `doc` must be hf-id-stamped (what `openComposition` produces). Against an unstamped
|
||||
* document the default walk matches nothing and every point answers a hard `false`,
|
||||
* indistinguishable from genuinely empty — pass `addressableOnly: false` there.
|
||||
* - `x`/`y` are the DOCUMENT's own client coordinates, not the host page's. A host with a
|
||||
* CSS-scaled iframe has to divide out that scale first, or it samples the wrong pixel.
|
||||
*
|
||||
* Note the polarity: this answers "does it paint", the positive question, and cannot say
|
||||
* "not knowable". A caller that wants the fail-safe contract — every uncertainty resolving
|
||||
* toward keeping the composition clickable — wants `PreviewAdapter.isProvablyEmptyAt`.
|
||||
*/
|
||||
export function compositionPaintsAt(
|
||||
doc: Document,
|
||||
win: Window & typeof globalThis,
|
||||
x: number,
|
||||
y: number,
|
||||
opts?: PaintQueryOptions,
|
||||
): boolean {
|
||||
const selector = (opts?.addressableOnly ?? true) ? "[data-hf-id]" : "*";
|
||||
const candidates: PaintCandidate[] = [];
|
||||
doc.querySelectorAll(selector).forEach((el) => {
|
||||
const candidate = paintCandidateAt(el, win, x, y);
|
||||
if (candidate) candidates.push(candidate);
|
||||
});
|
||||
|
||||
// Smallest area first, deeper element first on an exact tie. Compared exactly rather
|
||||
// than within an epsilon: a "close enough" tie relation is intransitive, which makes
|
||||
// Array.sort's output implementation-defined and the smallest-first guarantee (and the
|
||||
// lazy single-sample property that rides on it) engine-dependent.
|
||||
candidates.sort((a, b) => a.area - b.area || b.depth - a.depth);
|
||||
|
||||
let winner: PaintCandidate | undefined;
|
||||
let winnerInk: InkKind = "none";
|
||||
for (const candidate of candidates) {
|
||||
const ink = inkAt(candidate.el, win, { x, y });
|
||||
if (ink === "none") continue;
|
||||
winner = candidate;
|
||||
winnerInk = ink;
|
||||
break;
|
||||
}
|
||||
if (!winner) return false;
|
||||
|
||||
const fullBleed = opts?.fullBleedFraction ?? 0;
|
||||
if (fullBleed <= 0) return true;
|
||||
|
||||
// Never veto a measured pixel. The rule exists to discount paint ASSUMED to fill a box
|
||||
// — a full-frame wrapper with a background-image — and a full-bleed transparent PNG or
|
||||
// SVG overlay is precisely the case where that assumption is wrong: box area is not ink
|
||||
// area, and answering "background" there makes visible artwork unclickable.
|
||||
if (winnerInk === "verified") return true;
|
||||
|
||||
const frameArea = compositionFrameArea(doc, x, y);
|
||||
return frameArea === Infinity || winner.area < fullBleed * frameArea;
|
||||
}
|
||||
|
||||
// ─── IframePreviewAdapter ─────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
@@ -564,6 +877,33 @@ class IframePreviewAdapter implements PreviewAdapter {
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Is (x, y) provably free of ink? See PreviewAdapter.isProvablyEmptyAt.
|
||||
*
|
||||
* Every uncertain path answers false rather than claiming emptiness: an unreachable or
|
||||
* cross-origin document, and a same-origin iframe mid-navigation — srcdoc and blob:
|
||||
* assignment are async navigations, so `contentDocument` is a READABLE but empty
|
||||
* document while the composition is arriving. Walking that would find no ink and hand
|
||||
* back a confident "safe to click through" under a composition about to appear.
|
||||
*/
|
||||
isProvablyEmptyAt(x: number, y: number, opts?: PaintQueryOptions): boolean {
|
||||
let doc: Document | null;
|
||||
let win: (Window & typeof globalThis) | null;
|
||||
try {
|
||||
doc = this.iframe.contentDocument;
|
||||
win = this.iframe.contentWindow as (Window & typeof globalThis) | null;
|
||||
} catch {
|
||||
return false; // Cross-origin access throws. Unknowable is not empty.
|
||||
}
|
||||
if (!doc || !win) return false;
|
||||
if (doc.readyState === "loading") return false;
|
||||
if (!doc.querySelector("[data-hf-id]") && !doc.querySelector("[data-composition-id]")) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return !compositionPaintsAt(doc, win, x, y, opts);
|
||||
}
|
||||
|
||||
/**
|
||||
* Visually translate the target element inside the iframe at 60fps without
|
||||
* touching the model: sets the element's `translate` to its pre-drag value
|
||||
|
||||
@@ -46,6 +46,27 @@ export interface DraftProps {
|
||||
height?: number;
|
||||
}
|
||||
|
||||
export interface PaintQueryOptions {
|
||||
/**
|
||||
* A hit whose smallest painting box covers at least this fraction of the composition
|
||||
* frame reads as background rather than ink. 0 (the default) counts every painting box.
|
||||
*
|
||||
* Host policy, not a fact about the composition: an editor that treats "you clicked a
|
||||
* full-bleed layer" as "you clicked nothing" wants ~0.9, while a caller asking the
|
||||
* literal ink question wants 0.
|
||||
*/
|
||||
fullBleedFraction?: number;
|
||||
|
||||
/**
|
||||
* Consider only model-addressable elements (`[data-hf-id]`). Default true.
|
||||
*
|
||||
* False widens the walk to every element, which catches nodes created after the
|
||||
* document was stamped (split-text word spans, cloned nodes) at the cost of a larger
|
||||
* candidate set.
|
||||
*/
|
||||
addressableOnly?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Injectable preview adapter — decouples the SDK from the host preview surface.
|
||||
* The null/headless adapter stubs all methods (no browser needed).
|
||||
@@ -58,6 +79,30 @@ export interface PreviewAdapter {
|
||||
/** Sync hit-test at composition coordinates. Requires same-origin iframe. */
|
||||
elementAtPoint(x: number, y: number, opts?: { atTime?: number }): ElementAtPointResult | null;
|
||||
|
||||
/**
|
||||
* Is (x, y) provably free of ink — is it safe to let a click pass through to whatever
|
||||
* sits beneath this composition? Same coordinate space as `elementAtPoint`.
|
||||
*
|
||||
* The question a host layering a transparent composition over other content has to
|
||||
* answer before it swallows a click: is the user pointing AT something, or through an
|
||||
* empty gap at whatever sits beneath? Geometry alone cannot tell — a composition is
|
||||
* mostly full-bleed wrapper `<div>`s with no visual presence, and those boxes cover
|
||||
* every pixel of the frame.
|
||||
*
|
||||
* True ONLY when the composition was readable and nothing painted there. Everything
|
||||
* else is false: ink present, a document still loading or unreadable, or an adapter
|
||||
* that does not implement this at all (`preview.isProvablyEmptyAt?.(x, y)` → undefined
|
||||
* → falsy). The polarity is the point — it puts the burden of proof on passing the
|
||||
* click through, so every way of failing keeps the composition clickable instead of
|
||||
* making it vanish from under the cursor. A "does it paint" reading would leave two of
|
||||
* those three bottom values doing the dangerous thing.
|
||||
*
|
||||
* Deliberately not derived from `elementsFromPoint`, which omits `pointer-events: none`
|
||||
* nodes — those still paint, and reporting no ink over visible artwork is exactly the
|
||||
* failure this polarity exists to prevent.
|
||||
*/
|
||||
isProvablyEmptyAt?(x: number, y: number, opts?: PaintQueryOptions): boolean;
|
||||
|
||||
/** Apply draft CSS markers to the preview element (60fps, SDK not involved) */
|
||||
applyDraft(id: string, props: DraftProps): void;
|
||||
|
||||
|
||||
@@ -57,7 +57,12 @@ export type { HistoryModule, HistoryOptions, HistoryEntry } from "./history.js";
|
||||
export { createPersistQueue } from "./persist-queue.js";
|
||||
export type { PersistQueueModule, PersistQueueOptions } from "./persist-queue.js";
|
||||
|
||||
export type { PersistAdapter, PreviewAdapter, PersistVersionEntry } from "./adapters/types.js";
|
||||
export type {
|
||||
PersistAdapter,
|
||||
PreviewAdapter,
|
||||
PersistVersionEntry,
|
||||
PaintQueryOptions,
|
||||
} from "./adapters/types.js";
|
||||
|
||||
// Concrete adapter factories (browser-safe — Node-only fs adapter: @hyperframes/sdk/adapters/fs).
|
||||
export { createMemoryAdapter } from "./adapters/memory.js";
|
||||
|
||||
Reference in New Issue
Block a user