hyperframes/docs/sdk/guides/canvas-integration.mdx

---
title: "Canvas & Preview Integration"
description: "Connect a same-origin composition iframe to the SDK for hit-testing, draft preview, and selection."
---

The SDK's `PreviewAdapter` interface decouples the editing model from the visual surface. For browser-based editors, `createIframePreviewAdapter` bridges the SDK to a same-origin `<iframe>` containing the composition, giving you synchronous hit-testing, 60fps drag preview, and selection management — all without touching the model until the user commits.

<Note>
  The iframe must be same-origin (e.g. `srcdoc` or a `blob:` URL). Cross-origin iframe access throws a `DOMException`; the adapter does not guard this, so enforcing same-origin is the caller's responsibility.
</Note>

## Embedding the composition

Render the composition HTML into a same-origin `<iframe>` in your editor shell, then pass that element plus a dispatch callback to `createIframePreviewAdapter`:

```typescript
import { openComposition, createIframePreviewAdapter } from "@hyperframes/sdk";

// Assume `compositionHtml` is the composition's source HTML string.
const iframe = document.querySelector<HTMLIFrameElement>("#composition-frame")!;

// Build the adapter first so you can pass it to openComposition.
// The dispatch callback is called by commitPreview() after a drag completes.
const preview = createIframePreviewAdapter(iframe, (op) => {
  comp.dispatch(op);
});

const comp = await openComposition(compositionHtml, { preview });
```

The callback references `comp` before it is declared — that is intentional and safe: the arrow function captures `comp` by closure and is only ever invoked later (by `commitPreview()` on pointer-up), by which point `comp` is assigned. This is the standard way to break the adapter ⇄ session circular dependency.

The `dispatch` callback is optional. Omitting it means `commitPreview()` is a no-op, which is useful if you want to handle op derivation yourself.

## Keeping the preview in sync

Everything above wires up hit-testing and drag — but the iframe still won't reflect edits made any other way (an inspector panel calling `comp.setStyle()` directly, an undo, a collaborator's change replayed via `applyPatches()`). `attachSync` closes that gap: call it once you have both `preview` and `comp`, and every future edit — including undo/redo — mirrors onto the live iframe automatically.

```typescript
const detach = preview.attachSync(comp);

// later, when the editor unmounts or swaps compositions:
detach();
```

`attachSync` does an immediate full sync of `comp`'s current state first (so re-opening a composition with existing overrides isn't a blank iframe), then subscribes to the same `patch` event your other listeners use. You don't need to write your own mirroring code, and you don't need a separate mechanism for undo/redo — both flow through the same subscription. Script-tag edits (GSAP script rewrites) are the one thing it never mirrors, since replaying a live `<script>` tag doesn't re-execute it.

<Note>
  Calling `attachSync` again with a different `comp` detaches the previous subscription first — useful if your editor swaps which composition an iframe is bound to without remounting it.
</Note>

## Hit-testing: finding what the user clicked

`preview.elementAtPoint(x, y)` performs a synchronous hit-test at coordinates in the iframe's own coordinate space and returns the nearest `[data-hf-id]` element, or `null` for a transparent hit.

```typescript
iframe.addEventListener("load", () => {
  const frameDocument = iframe.contentDocument;
  if (!frameDocument) return;

  // Events inside an iframe do not bubble to the outer <iframe> element.
  frameDocument.addEventListener("click", (event) => {
    const hit = preview.elementAtPoint(event.clientX, event.clientY);
    if (hit) {
      // hit.id  — the data-hf-id value
      // hit.tag — the lowercased tag name (e.g. "div", "img", "video")
      preview.select([hit.id]);
    }
  });
});
```

The hit-test skips elements whose computed opacity is `0` (including ancestors with `opacity: 0`), and for `<img>` elements it samples the alpha at the clicked pixel using an offscreen canvas — a transparent pixel falls through to the element behind it. Cross-origin images that taint the canvas fall back to treating the pixel as opaque.

The `opts.atTime` parameter is accepted but does not seek the GSAP timeline. It reflects whatever frame the composition is currently paused at in the iframe. Accurate out-of-time-band opacity queries are a future capability.

### Walking a click target to the nearest HF element

If you are working with events on the iframe's `contentDocument` directly (e.g. via a `message` bridge), use the exported `resolveNearestHfElement` function. It walks up the DOM from any node until it finds a `[data-hf-id]` ancestor, skipping the root:

```typescript
import { resolveNearestHfElement } from "@hyperframes/sdk";

// Inside the iframe's own document context:
iframeDoc.addEventListener("click", (e) => {
  const result = resolveNearestHfElement(
    e.target as Element | null,
    (el) => {
      // Return false to treat this element as invisible and continue the walk.
      const style = el.ownerDocument.defaultView?.getComputedStyle(el);
      return style ? parseFloat(style.opacity) !== 0 : true;
    },
  );
  if (result) {
    // result.id, result.tag
  }
});
```

`resolveNearestHfElement` returns `null` when the walk exits the tree without finding a `[data-hf-id]` node, when the matching node carries `[data-hf-root]` (the root is transparent to selection), or when `isVisible` returns `false` for that node.

## Transparent compositions over other content

A composition authored as an overlay — a small graphic on an otherwise-empty 1080×1920 frame, layered over a video or an avatar — is still a rectangular DOM box covering every pixel of the frame. Without help it swallows every click, and whatever sits beneath it becomes unreachable.

`preview.isProvablyEmptyAt(x, y)` is the question you need answered: is this point provably free of ink, so a click may safely reach what sits beneath? Toggle `pointer-events` on your wrapper from the answer, and let the browser deliver the event to the right target:

```typescript
const wrapper = document.querySelector<HTMLElement>("#composition-wrapper")!;

/**
 * Host-page pointer coordinates → the iframe document's own client space, which is what
 * the paint query samples against. The iframe renders at the composition's native size and is
 * CSS-scaled to fit, so the on-screen scale has to be divided out — skip this and you
 * sample the wrong pixel, and pass-through toggles over the wrong regions.
 */
function toCompositionPoint(clientX: number, clientY: number) {
  const rect = iframe.getBoundingClientRect();
  if (!rect.width || !rect.height) return null;
  const scaleX = rect.width / compositionNativeWidth;
  const scaleY = rect.height / compositionNativeHeight;
  if (!scaleX || !scaleY) return null;
  return { x: (clientX - rect.left) / scaleX, y: (clientY - rect.top) / scaleY };
}

function updatePassThrough(clientX: number, clientY: number, altKey: boolean) {
  const point = toCompositionPoint(clientX, clientY);
  // Alt is the escape hatch for grabbing the composition itself in an empty region.
  // Every uncertain case — no point, still loading, adapter without the method — is
  // falsy here, so the composition stays clickable rather than vanishing.
  const passThrough =
    !!point && !altKey && !!preview.isProvablyEmptyAt?.(point.x, point.y);
  wrapper.style.pointerEvents = passThrough ? "none" : "";
}
```

<Warning>
  Listen on the **host document**, not on the wrapper or the iframe. The first time this
  sets `pointer-events: none` the wrapper stops receiving events, so a listener attached
  there can never turn it back on — the pass-through state sticks.
</Warning>

Three things are easy to get wrong here:

<Steps>
  <Step title="Decide before the press, not during it">
    The browser picks an event's target before any handler runs, so flipping `pointer-events` inside `mousedown` cannot retarget the click already in flight. Sample the pointer position on `mousemove` and keep the decision current.
  </Step>
  <Step title="Re-evaluate on every frame, not only on movement">
    Animated artwork moves under a stationary cursor. Anything that can change the answer — pointer movement, the Alt key, and the playhead — has to re-run the query from the last known position. Coalesce those triggers into one `requestAnimationFrame` query rather than answering each separately, and short-circuit before the query when the pointer is outside the composition's box: it is a walk over the document, so it does not belong on an ungated per-event path.
  </Step>
  <Step title="Let the polarity do the work">
    `isProvablyEmptyAt` is true only when it has established there is no ink. A document that hasn't loaded, an adapter without the method, and a point you couldn't map all come back falsy — which keeps the composition clickable. Don't invert it into a "does it paint" variable; that reintroduces the bug the polarity removes.
  </Step>
</Steps>

Pass `{ fullBleedFraction: 0.9 }` if your editor treats a layer covering nearly the whole frame as background rather than artwork — a common choice, since a full-bleed wrapper is usually scaffolding rather than something the user is pointing at.

Do not reimplement this with `elementsFromPoint`. That stack omits `pointer-events: none` nodes, and a decorative overlay carrying `pointer-events: none` still paints — a z-stack query would report no ink over visible artwork and pass the click through anyway. The paint walk covers element boxes geometrically for that reason.

## Draft loop: 60fps drag without model mutations

The draft loop keeps the model clean during a drag. The SDK is **not** in the 60fps path — you call `preview.applyDraft` on every `pointermove` and `preview.commitPreview` once on `pointerup`. The model sees exactly one `moveElement` op per drag, rather than hundreds.

`applyDraft` sets the element's CSS `translate` directly inside the iframe — the pre-drag value composed with the accumulated delta — so the drag is visible in any composition without composition-side CSS, and works on GSAP-animated elements (a `translate` set after GSAP's first parse composes with the animated transform instead of being overwritten). Nothing in the SDK model changes. `cancelPreview` restores the pre-drag translate; `commitPreview` derives one `moveElement` op and mirrors the committed position onto the live element.

```typescript
let dragging = false;
let startX = 0;
let startY = 0;
let targetId: string | null = null;

iframe.addEventListener("load", () => {
  const frameDocument = iframe.contentDocument;
  if (!frameDocument) return;

  frameDocument.addEventListener("pointerdown", (event) => {
    const hit = preview.elementAtPoint(event.clientX, event.clientY);
    if (!hit) return;

    dragging = true;
    targetId = hit.id;
    startX = event.clientX;
    startY = event.clientY;
    // The target comes from the iframe's realm, so `instanceof Element` against
    // this window's constructor is always false and capture would be skipped —
    // then a pointer leaving the frame loses `pointerup` and drag state sticks.
    if (event.target && "setPointerCapture" in event.target) {
      (event.target as Element).setPointerCapture(event.pointerId);
    }
  });

  frameDocument.addEventListener("pointermove", (event) => {
    if (!dragging || !targetId) return;
    preview.applyDraft(targetId, {
      dx: event.clientX - startX,
      dy: event.clientY - startY,
    });
  });

  frameDocument.addEventListener("pointerup", () => {
    if (!dragging || !targetId) return;
    preview.commitPreview();
    dragging = false;
    targetId = null;
  });

  frameDocument.addEventListener("pointercancel", () => {
    preview.cancelPreview();
    dragging = false;
    targetId = null;
  });
});
```

`DraftProps` accepts `dx`, `dy`, `width`, and `height`. Width and height are accepted by the interface but resize support (mapping to a `setStyle` op) is not yet wired — only `dx`/`dy` drive the draft CSS vars today.

Call `cancelPreview()` instead of `commitPreview()` to discard the drag without emitting any op. The model is never mutated and the CSS vars are cleared.

## Selection

`preview.select(ids, opts?)` sets the selection state and fires the session's `selectionchange` event on any listeners. Pass `{ additive: true }` to extend the current selection rather than replace it.

```typescript
// Replace selection
preview.select(["hf-title"]);

// Extend selection (e.g. shift-click)
preview.select(["hf-logo"], { additive: true });

// Clear selection
preview.select([]);
```

Listen to selection changes on the session via `comp.on("selectionchange", ...)` — the adapter fires that event, not a separate event on the iframe.

## Pairing with embedded override mode

For template-driven products you typically open the composition in embedded override mode and store only the sparse delta, not the full HTML. The preview adapter works identically in that mode — pass it the same way:

```typescript
const comp = await openComposition(templateHtml, {
  preview,
  overrides: existingOverrides,
  history: false,
});
```

See [Embedded Override Mode](/sdk/guides/embedded-override-mode) for the full pattern.

## What to build next

Once hit-testing and drag are working, you can use the affordance resolver to drive a context-aware inspector panel for whatever element is selected. See [Editing Affordances](/sdk/guides/editing-affordances) for how to translate a live element into capability flags and section applicability.

<CardGroup cols={2}>
  <Card title="Adapter reference" icon="plug" href="/sdk/reference/adapters">
    Full `PreviewAdapter`, `PersistAdapter`, and related type documentation.
  </Card>
  <Card title="Editing Affordances" icon="sliders" href="/sdk/guides/editing-affordances">
    Resolve which edit controls to show for the selected element.
  </Card>
</CardGroup>