Files
hyperframes/docs/sdk/reference/adapters.mdx
T
Hblee 9769ba2c7b feat(sdk): expose a paint query for transparent-composition hit-testing (#3070)
* feat(sdk): expose a paint query for transparent-composition hit-testing

A host layering a transparent composition over other content has to know
whether a point carries ink before it decides to swallow a click. The adapter
only answered "what element is here", so AI Studio wrote its own answer and
could not reach the per-pixel alpha the adapter already samples for <img>.

Add PreviewAdapter.paintsAt, plus the pieces it is built from on a new
./adapters/iframe subpath so a host with different hit-test policy can compose
its own walk.

The walk is geometric rather than elementsFromPoint-based: that stack omits
pointer-events:none nodes, and a decorative overlay carrying it still paints,
so a z-stack query would report no ink over visible artwork — the direction
that makes a composition vanish from under the cursor.

fullBleedFraction is an option rather than a constant because "a layer covering
the whole frame is background, not artwork" is host policy, not a fact about
the composition.

* fix(sdk): scope the full-bleed frame to the root under the point

compositionFrameArea took the smallest [data-composition-id] in the whole
document, so an unrelated sub-composition sized the reference frame for points
nowhere near it: a 300x300 badge in a corner made every mid-size painter in a
1920x1080 outer frame read as full-bleed, and the composition went
click-through under artwork the user can plainly see. That is the direction the
fail-safe exists to avoid, and the docs already described the intended
behaviour — the innermost root CONTAINING the point.

Also read the alpha channel instead of matching known transparent spellings.
Only the `transparent` keyword computes to rgba(0, 0, 0, 0); a faded-out white
stays rgba(255, 255, 255, 0), which the set counted as painted. That erred
toward absorbing clicks rather than losing them, so it was a false positive
rather than a hazard, but it is wrong.

Both are pinned by tests that fail when the fix is reverted.

* fix(sdk): stop the paint query answering "no ink" over visible artwork

Four cases where the walk landed on the wrong side of its own fail-safe.

The full-bleed veto tested the winner's border-box area even when the alpha
sampler had just read an opaque pixel there, so a full-frame transparent PNG or
SVG overlay — the case this feature exists for — reported background over
visibly opaque artwork, with no fullBleedFraction that worked. Ink now carries
how it was established, and a measured pixel is never vetoed. An image whose
pixels could NOT be read stays inferred, so a tainted CDN overlay still yields
to the veto rather than absorbing every click.

Composition roots were excluded from candidacy outright, so a root carrying a
background answered false even at fraction 0, where the docs promise every
painting box counts. Roots are candidates now; the veto discounts them without
a special case, since a root's box is the frame.

An <img> with a clear pixel early-returned past its own background, padding
plate and border, which any other element would have counted.

A same-origin iframe mid-navigation exposes a readable but empty document, so
the !doc guard never fired and a loading composition answered a confident
"no ink" — the exact failure the null convention exists to prevent.

Also: the sort comparator's epsilon tie was intransitive, leaving the
smallest-first guarantee (and the lazy single-sample property that rides on it)
engine-dependent; the guide's pass-through recipe called a function that does
not exist and hand-waved the coordinate mapping that makes it correct; and the
reference now states the under-counts alongside the over-counts, the walk's
blindness to runtime-mounted content, and compositionPaintsAt's preconditions.

Each fix is pinned by a test that fails when the fix is reverted.

* refactor(sdk)!: invert the paint query to isProvablyEmptyAt

paintsAt handed callers three falsy bottom values with opposite safe readings:
false meant "no ink, pass the click through", null meant "not knowable, treat as
painted", and undefined from an adapter without the method also meant painted.
The idiomatic `if (!preview.paintsAt?.(x, y)) passThrough()` therefore did the
dangerous thing for two of the three, and the convention needed defending in the
interface docstring, the reference and the guide — plus a dedicated comment and
a pinning test on the headless adapter to stop null regressing to false.

Inverting the polarity collapses the tri-state to a plain boolean and makes the
safe reading structural: true only when the composition was readable and nothing
painted there, so ink, an unreadable or still-loading document, and a missing
implementation all land on "keep the composition clickable". The prose stays as
rationale, but nothing depends on a reader remembering it.

PaintsAtOptions becomes PaintQueryOptions, since it now describes the walk that
both the adapter method and the exported compositionPaintsAt share rather than
one method's arguments. compositionPaintsAt keeps its ink-positive name: it
answers the other question, and its docstring points callers who need the
fail-safe contract at the adapter.

Nothing is released yet, so no consumer is on the old name.
2026-08-06 16:27:38 -07:00

396 lines
21 KiB
Plaintext

---
title: "Adapters"
description: "Persistence and preview adapter interfaces, contracts, and the built-in factory functions."
---
The SDK decouples editing sessions from storage and preview surfaces through two injectable interfaces: `PersistAdapter` and `PreviewAdapter`. Both ship with concrete factory functions you pass to `openComposition()`. You can also implement either interface directly for custom storage backends (S3, IndexedDB, HTTP) or custom preview surfaces.
## PersistAdapter
```typescript
import type { PersistAdapter } from "@hyperframes/sdk";
```
Injectable storage adapter. Decouples the SDK from the underlying persistence mechanism so the same session code runs in tests (memory), local dev (filesystem), and production (cloud storage).
### Interface
```typescript
interface PersistAdapter {
read(path: string): Promise<string | undefined>;
write(path: string, content: string): Promise<void>;
flush(): Promise<void>;
listVersions(path: string): Promise<PersistVersionEntry[]>;
loadFrom(path: string, versionKey: string): Promise<string | undefined>;
on(event: "persist:error", handler: (event: PersistErrorEvent) => void): () => void;
}
```
<ParamField path="read" type="(path: string) => Promise<string | undefined>">
Returns the stored content for `path`, or `undefined` for a path that has never been written. Never throws for a missing path.
</ParamField>
<ParamField path="write" type="(path: string, content: string) => Promise<void>">
Persists `content` at `path`. Idempotent — a second call with the same path overwrites the prior value. Write failures must not propagate as thrown exceptions; fire `persist:error` instead.
</ParamField>
<ParamField path="flush" type="() => Promise<void>">
Forces any queued or in-flight writes to commit before resolving. Call before process exit or navigation to prevent data loss.
</ParamField>
<ParamField path="listVersions" type="(path: string) => Promise<PersistVersionEntry[]>">
Returns the version history for `path` ordered newest-first. Returns an empty array when no versions exist. See `PersistVersionEntry` below.
</ParamField>
<ParamField path="loadFrom" type="(path: string, versionKey: string) => Promise<string | undefined>">
Returns the HTML content for a specific version identified by `versionKey`. Returns `undefined` when the key does not exist.
</ParamField>
<ParamField path="on" type='(event: "persist:error", handler) => () => void'>
Subscribes to write failures. Returns an unsubscribe function. Adapters must emit this event — not throw — when a write fails, so the session continues running even when storage is temporarily unavailable.
</ParamField>
### Contract summary
- `read()` returns `undefined` for a path that has never been written — never throws ENOENT or a 404 equivalent.
- `write()` is idempotent; a second write to the same path replaces the stored content.
- `flush()` resolves when any pending writes are committed to durable storage.
- `listVersions()` returns entries newest-first; `loadFrom()` uses the keys from those entries.
- Write errors are emitted via `on('persist:error')`, never thrown — the session keeps running.
### PersistVersionEntry
```typescript
interface PersistVersionEntry {
/** Opaque key identifying this version (adapter-defined format). */
key: string;
/** Full HTML content — may be omitted by adapters that load content lazily via loadFrom(). */
content?: string;
timestamp?: number;
}
```
The `key` is adapter-defined and opaque to callers — pass it directly to `loadFrom()`. The filesystem adapter encodes milliseconds and a counter into the key; the memory adapter uses an incrementing `"v1"`, `"v2"` … scheme.
---
## PreviewAdapter
```typescript
import type { PreviewAdapter } from "@hyperframes/sdk";
```
Injectable preview surface adapter. Decouples the SDK from the host's rendering layer. The SDK is **not** in the 60fps draft loop: your pointer-move handler calls `applyDraft()` directly on the adapter at 60fps, and the SDK only gets involved once per gesture when `commitPreview()` fires to derive and dispatch the resulting op.
### Interface
```typescript
interface PreviewAdapter {
elementAtPoint(x: number, y: number, opts?: { atTime?: number }): ElementAtPointResult | null;
isProvablyEmptyAt?(x: number, y: number, opts?: PaintQueryOptions): boolean;
applyDraft(id: string, props: DraftProps): void;
commitPreview(): void;
cancelPreview(): void;
select(ids: string[], opts?: { additive?: boolean }): void;
on(event: "selection", handler: (ids: string[]) => void): () => void;
attachSync(comp: Composition): () => void;
}
```
<ParamField path="elementAtPoint" type="(x, y, opts?) => ElementAtPointResult | null">
Synchronous hit-test at composition coordinates `(x, y)`. Returns the nearest `[data-hf-id]` element under the point, or `null` for a transparent hit (the composition root, an opacity-0 element, or nothing at all). Requires a same-origin iframe — cross-origin access throws a DOMException. The `atTime` option reflects GSAP state at the current playhead; seeking to a speculative time is not supported.
</ParamField>
<ParamField path="isProvablyEmptyAt" type="(x, y, opts?) => boolean">
Optional. Is `(x, y)` provably free of ink — is it safe to let a click pass through to whatever sits beneath? This is the question a host has to answer before a transparent composition layered over other content swallows a click: is the user pointing **at** artwork, or through an empty gap? Geometry alone cannot tell — a composition is mostly full-bleed wrapper `<div>`s that cover every pixel of the frame without painting anything.
**True only when the composition was readable and nothing painted there.** Ink present, a document still loading or unreadable, and an adapter that doesn't implement the method (`preview.isProvablyEmptyAt?.(x, y)` → `undefined` → falsy) all come back falsy. That polarity is deliberate: it puts the burden of proof on passing the click through, so every way of failing keeps the composition clickable rather than making it vanish from under the cursor. The obvious call site is safe by construction:
```typescript
if (preview.isProvablyEmptyAt?.(x, y)) passThrough();
```
Ink is a computed-style test — background colour, background image, visible border, the element's own text, or intrinsic media — with one exception: `<img>` (and the `<img>` inside a `<picture>`) routes through per-pixel alpha, so a transparent PNG paints only where its pixels do. A pixel-verified hit is never discounted by `fullBleedFraction`: box area is not ink area, so a full-frame transparent overlay stays clickable where it is actually opaque.
**Known over-counts** (report ink that isn't there, so a click selects the composition): a `background-image` that is itself mostly transparent reads as painting across its whole box; `<video>`, `<svg>` and `<canvas>` are unconditionally opaque; and an image whose pixels cannot be read — cross-origin without CORS, still loading, rotated, or above the sampler's size budget — falls back to opaque.
**Known under-counts** (miss ink that is there, so a click may pass through): `::before` / `::after` generated content, `box-shadow`, `outline` and `text-decoration` are not tested, and the first three paint outside the border box, so the element is not even a candidate. Content the SDK never stamped is invisible under the default `addressableOnly` — see below.
The walk is **geometric**, not `elementsFromPoint`-based, and is blind to `pointer-events` and `z-index` by design: a decorative overlay carrying `pointer-events: none` still paints, and a z-stack query would report no ink over visible artwork.
</ParamField>
<ParamField path="applyDraft" type="(id: string, props: DraftProps) => void">
Visually translates the preview element at 60fps during a drag: sets the element's CSS `translate` to its pre-drag value composed with the accumulated delta. Works on GSAP-animated elements (a `translate` set after GSAP's first parse composes with the animated transform). The **SDK is not called here** — this is a direct write to the preview surface by your pointer-move handler. Switching `id` mid-drag reverts the previous element's draft first.
</ParamField>
<ParamField path="commitPreview" type="() => void">
Called once on pointer-up. Reads the accumulated draft delta, derives a `moveElement` op from it, dispatches it into the SDK, emits a patch event, and mirrors the committed position onto the live element (so it holds without a reload). This is the only moment the SDK becomes aware of a drag. If dispatch throws, the draft translate is reverted and the error propagates.
</ParamField>
<ParamField path="cancelPreview" type="() => void">
Restores the element's pre-drag `translate` without dispatching any op. The model is never changed. Call this on `Escape` keydown or when a drag is aborted.
</ParamField>
<ParamField path="select" type="(ids: string[], opts?: { additive?: boolean }) => void">
Sets the preview selection and fires `selectionchange` on the session. Pass `{ additive: true }` to merge `ids` into the current selection rather than replacing it.
</ParamField>
<ParamField path="on" type='(event: "selection", handler: (ids: string[]) => void) => () => void'>
Fired when the preview host changes the selection (for example, the user clicks an element). Returns an unsubscribe function. In the current release, callers listen to the session's own `selectionchange` event instead — this hook is wired in a future stage.
</ParamField>
<ParamField path="attachSync" type="(comp: Composition) => () => void">
Mirrors a composition's edits onto the adapter's own live document: an immediate full sync of the composition's current overrides, then a subscription that replays every future `patch` event — including undo/redo, since both fire through the same event with forward or inverse patches. Calling `attachSync` again while already attached detaches the previous subscription first. The full-override sync also re-runs on every iframe `load`, so a `srcdoc` navigation that races the attach (or drops patches committed during the load window) converges once the new document arrives. Returns an unsubscribe function.
Script-tag patches (`/script/gsap` and any future `/script/*` path) are never mirrored — rewriting a live `<script>` tag's content doesn't re-execute it, and re-running GSAP setup from scratch would conflict with running timeline state. Every other patch kind (style, text, attribute, timing, hold, element add/remove, stylesheet, variable value, variable declaration) mirrors as-is.
```typescript
const adapter = createIframePreviewAdapter(iframe, dispatch);
const comp = await openComposition(html, { preview: adapter });
const detach = adapter.attachSync(comp);
// later, if the host tears down the preview:
detach();
```
</ParamField>
### ElementAtPointResult
```typescript
interface ElementAtPointResult {
id: string;
tag: string;
}
```
The `id` is the element's `data-hf-id` value; `tag` is its lowercase tag name (e.g. `"div"`, `"img"`).
### DraftProps
```typescript
interface DraftProps {
dx?: number;
dy?: number;
width?: number;
height?: number;
}
```
`dx` and `dy` are the accumulated drag deltas in composition pixels. `width` and `height` are defined in the interface for forward compatibility but are not yet wired to any op.
### PaintQueryOptions
```typescript
interface PaintQueryOptions {
fullBleedFraction?: number;
addressableOnly?: boolean;
}
```
<ParamField path="fullBleedFraction" type="number" default="0">
A hit whose smallest painting box covers at least this fraction of the composition frame reads as background rather than ink. This is host policy, not a fact about the composition: an editor that treats "you clicked a layer covering the whole frame" as "you clicked the background" passes `0.9`, while a caller asking the literal ink question leaves it at `0`. Nested sub-compositions carry `data-composition-id` too, so the reference frame is the innermost composition root containing the point.
</ParamField>
<ParamField path="addressableOnly" type="boolean" default="true">
Consider only model-addressable elements (`[data-hf-id]`). Stamping happens once, on the document `openComposition` was given, so anything the runtime creates or fetches afterwards is invisible to the default walk: split-text word and character spans (splitting also empties the stamped parent's own text nodes, so the parent stops counting too), cloned nodes, and whole sub-composition scenes mounted from `data-composition-src`. Kinetic typography and registry-mounted lower-thirds — both canonical transparent-overlay content — therefore read as no-ink by default.
Set `false` to widen the walk to every element, which sees that content at the cost of a larger candidate set.
</ParamField>
<Note>
`ElementAtPointResult` and `DraftProps` are the structural shapes a `PreviewAdapter` produces and consumes. They are **not** re-exported from the `@hyperframes/sdk` barrel — you implement against these shapes rather than importing them. `PaintQueryOptions` **is** re-exported, since callers pass it rather than implement it.
</Note>
---
## Factory Functions
### createMemoryAdapter
```typescript
import { createMemoryAdapter } from "@hyperframes/sdk";
function createMemoryAdapter(): PersistAdapter & { injectFault(message: string): void };
```
Returns a `PersistAdapter` backed by an in-process `Map`. Writes are synchronous; `flush()` is a no-op. Versions are keyed `"v1"`, `"v2"` … and stored in memory with full content.
The returned value also exposes `injectFault(message)` — a test helper that causes the **next** `write()` call to fire a `persist:error` event with `message` instead of committing. Use this in unit tests to verify your error-handling code path.
```typescript
const persist = createMemoryAdapter();
const comp = await openComposition(html, { persist });
comp.setText("hf-title", "Hello");
await comp.flush();
const saved = await persist.read("composition.html");
```
<Note>
`createMemoryAdapter()` is best suited for tests, demos, and ephemeral in-process sessions. For local development, use `createFsAdapter()` so edits survive restarts.
</Note>
---
### createFsAdapter
```typescript
import { createFsAdapter } from "@hyperframes/sdk/adapters/fs";
function createFsAdapter(opts: FsAdapterOptions): PersistAdapter;
```
**Node.js only.** Returns a `PersistAdapter` that reads and writes files under a root directory. Import from the `@hyperframes/sdk/adapters/fs` subpath — this module uses Node `fs/promises` and is excluded from the browser-safe main bundle.
#### FsAdapterOptions
```typescript
interface FsAdapterOptions {
/** Root directory for composition files. */
root: string;
/** Max versions to keep per file. Default: 20. */
maxVersions?: number;
}
```
<ParamField path="root" type="string" required>
Absolute or relative path to the directory where composition files are written. Created with `mkdir -p` on first write.
</ParamField>
<ParamField path="maxVersions" type="number">
Maximum number of historical versions retained per file. Oldest versions are pruned automatically when the limit is exceeded. Defaults to `20`.
</ParamField>
The adapter writes the current composition at `{root}/{path}` and stores version snapshots in `{root}/.hf-versions/{path}/`. Version keys encode `Date.now()` and a monotonic counter (`"1750000000000-0001"`), so `listVersions()` returns them newest-first by lexicographic descending sort.
```typescript
import { openComposition } from "@hyperframes/sdk";
import { createFsAdapter } from "@hyperframes/sdk/adapters/fs";
const comp = await openComposition(html, {
persist: createFsAdapter({ root: "./project", maxVersions: 50 }),
persistPath: "index.html",
});
comp.setText("hf-title", "Saved");
await comp.flush();
// List saved versions
const adapter = createFsAdapter({ root: "./project" });
const versions = await adapter.listVersions("index.html");
const previous = await adapter.loadFrom("index.html", versions[1].key);
```
<Warning>
`createFsAdapter` uses Node.js `fs/promises`. Do not import it in browser or edge environments — import from `@hyperframes/sdk/adapters/fs` (the subpath) so bundlers can tree-shake it.
</Warning>
---
### createHeadlessAdapter
```typescript
import { createHeadlessAdapter } from "@hyperframes/sdk";
function createHeadlessAdapter(): PreviewAdapter;
```
Returns a no-op `PreviewAdapter` for headless use: agents, CI pipelines, and server-side rendering. All methods are stubs — `elementAtPoint` always returns `null` and `isProvablyEmptyAt` always returns `false`, `applyDraft` and `commitPreview` are no-ops, and the `"selection"` event never fires.
`isProvablyEmptyAt` returns `false` on purpose: an adapter with no surface cannot establish that a point is free of ink, and answering `true` would tell a host it is safe to click through a composition nobody can see.
Pass this adapter when you open a composition for programmatic editing and do not need a live preview surface.
```typescript
import { openComposition, createHeadlessAdapter } from "@hyperframes/sdk";
const comp = await openComposition(html, {
preview: createHeadlessAdapter(),
});
```
<Note>
`openComposition` does not create a preview adapter automatically. Omit `preview` when no preview
surface is needed, or pass `createHeadlessAdapter()` when an explicit no-op adapter makes shared
code clearer.
</Note>
---
### createIframePreviewAdapter
```typescript
import { createIframePreviewAdapter } from "@hyperframes/sdk";
function createIframePreviewAdapter(
iframe: HTMLIFrameElement,
dispatch?: (op: EditOp) => void,
): PreviewAdapter;
```
Returns a `PreviewAdapter` that bridges the SDK to a same-origin `<iframe>` containing the composition. Provides real hit-testing via `elementsFromPoint` (z-stack aware), draft drag support, and selection management.
**Requirements:**
- The iframe must be same-origin (e.g. a `srcdoc` or `blob:` URL). Cross-origin access to `contentDocument` throws a `DOMException`.
- Pass your session's `dispatch` callback to enable `commitPreview()` — without it, pointer-up is a no-op on the model.
**Image-alpha hit-testing:** For `<img>` elements, the adapter samples the alpha channel of the pixel under the pointer using an `OffscreenCanvas`. Transparent pixels fall through to the element behind. Cross-origin images that taint the canvas are treated as opaque (safe fallback, logged once per src).
**Paint queries:** `isProvablyEmptyAt` answers whether a point is safe to click through — see the [`PreviewAdapter` interface](#previewadapter) above and the [transparent-overlay recipe](/sdk/guides/canvas-integration#transparent-compositions-over-other-content). The pieces it is built from are importable directly for hosts whose hit-test policy differs:
```typescript
import {
elementPaintsInk,
compositionPaintsAt,
imageAlphaOpaqueAt,
alphaIsOpaque,
mapPointToImagePixel,
} from "@hyperframes/sdk/adapters/iframe";
```
```typescript
import { openComposition, createIframePreviewAdapter } from "@hyperframes/sdk";
const iframe = document.querySelector<HTMLIFrameElement>("#preview-frame")!;
// Optional, and dispatched with `?.` — the callback cannot fire before
// openComposition() resolves, but strict TypeScript cannot prove that and
// rejects a definite `let` captured before assignment (TS2454).
let comp: Awaited<ReturnType<typeof openComposition>> | undefined;
const preview = createIframePreviewAdapter(iframe, (op) => comp?.dispatch(op));
comp = await openComposition(html, { preview });
// Hit-test at pointer position
const hit = preview.elementAtPoint(pointerX, pointerY);
if (hit) {
preview.select([hit.id]);
// Drag: call applyDraft at 60fps, commitPreview on pointer-up
preview.applyDraft(hit.id, { dx: 12, dy: -5 });
preview.commitPreview();
}
```
---
## Export Map
| Symbol | Imported from |
|--------|---------------|
| `PersistAdapter`, `PreviewAdapter`, `PersistVersionEntry` | `@hyperframes/sdk` (types only) |
| `createMemoryAdapter` | `@hyperframes/sdk` |
| `createHeadlessAdapter` | `@hyperframes/sdk` |
| `createIframePreviewAdapter`, `resolveNearestHfElement` | `@hyperframes/sdk` |
| `PaintQueryOptions` | `@hyperframes/sdk` (type only) |
| `elementPaintsInk`, `compositionPaintsAt`, `imageAlphaOpaqueAt`, `alphaIsOpaque`, `mapPointToImagePixel`, `INTRINSIC_PAINT_TAGS` | `@hyperframes/sdk/adapters/iframe` |
| `createFsAdapter`, `FsAdapterOptions` | `@hyperframes/sdk/adapters/fs` |
<CardGroup cols={2}>
<Card title="Persistence Guide" icon="floppy-disk" href="/sdk/guides/persistence">
How to wire adapters into openComposition, handle errors, and restore versions.
</Card>
<Card title="Canvas Integration" icon="browser" href="/sdk/guides/canvas-integration">
Building a visual editor canvas with the iframe preview adapter and hit-testing.
</Card>
</CardGroup>