Files
hyperframes/packages/core
Miguel Ángel ac671bdf5c feat(core): add TypeGPU/WebGPU runtime adapter (#755)
* feat(core): add TypeGPU/WebGPU runtime adapter

Adds a deterministic seek adapter for compositions that render with
TypeGPU or raw WebGPU. Follows the same push+poll pattern as the
Three.js adapter:

- Sets `window.__hfTypegpuTime` on every seek so render loops can
  poll it instead of `performance.now()`.
- Dispatches a `"hf-seek"` CustomEvent on `window` so compositions
  can imperatively re-render a single frame at the new seek position.

Compositions listen for the event and update their time uniform:

```js
window.addEventListener("hf-seek", (e) => render(e.detail.time));
```

Works with TypeGPU (docs.swmansion.com/TypeGPU) and raw WebGPU alike.
No assumptions are made about pipeline construction — multiple canvases
or renderers are supported by sharing the same event.

- 9 unit tests, all pass
- wired in init.ts adapter array
- `__hfTypegpuTime` declared in window.d.ts

* fix(core): deduplicate hf-seek dispatch across GPU adapters

Both three and typegpu adapters previously dispatched the same
"hf-seek" CustomEvent independently, causing any composition that
registered a listener to receive two events per seek tick — doubling
per-scrub GPU work even though the renders are idempotent.

Fix: extract a shared `dispatchSeekEvent` helper (seek-dispatch.ts)
that deduplicates by exact float equality within the same synchronous
call stack. Both adapters now call this helper instead of dispatching
directly.

Also adds:
- `resetSeekDispatchState()` export for test isolation
- `beforeEach` reset in three.test.ts and typegpu.test.ts
- New typegpu test: "duplicate seek to same time fires event only once"
- Docstring additions to typegpu.ts: render-mode determinism contract
  (await device.queue.onSubmittedWorkDone()) and navigator.gpu feature
  detection guidance for composition authors

* feat(core): video-texture render compat + TypeGPU skill

Adds the missing pieces for video-backed WebGPU effects in render mode:

- `video-texture-compat.ts`: monkey-patches `GPUQueue.copyExternalImageToTexture`
  to detect the engine's injected `<img class="__render_frame__">` siblings and
  transparently substitute them for `<video>` sources. Headless Chrome can't
  supply decoded video frames to WebGPU, but the engine's pre-extracted frame
  images work. Falls through to the original path in preview mode.

- `patchVideoTextureCompat()` wired in init.ts after adapter array creation.

- `skills/typegpu/SKILL.md`: full authoring guide for TypeGPU/WebGPU compositions
  covering contract, timeline registration, video-backed effects, frosted blur
  via downsample pass, WGSL patterns, and deterministic rendering.

* test(producer): add typegpu-adapter regression test

Self-contained WebGPU composition with:
- Procedural gradient background (no video dependency)
- Animated ring driven by hf-seek time uniform
- Pulsing center glow
- Two GSAP-driven captions testing adapter sync

Verifies the TypeGPU adapter's hf-seek → WebGPU render pipeline
produces deterministic frames. workers: 1 for consistency.

Note: output.mp4 baseline needs to be generated in CI — the local
Docker image can't launch Chrome (ARM/x86 mismatch on Mac).
2026-05-13 03:31:56 +02:00
..
2026-05-13 00:27:13 +00:00
2026-03-21 22:43:56 -07:00
2026-03-21 22:43:56 -07:00

@hyperframes/core

Types, parsers, generators, compiler, linter, runtime, and frame adapters for the Hyperframes video framework.

Install

npm install @hyperframes/core

Most users don't need to install core directly — the CLI, producer, and studio packages depend on it internally.

What's inside

Module Description
Types TimelineElement, CompositionSpec, Asset, canvas dimensions, defaults
Parsers parseHtml — extract timeline elements from HTML; parseGsapScript — parse GSAP animations
Generators generateHyperframesHtml — produce valid Hyperframes HTML from a composition spec
Compiler compileTimingAttrs — resolve data-start / data-duration into absolute times
Linter lintHyperframeHtml — validate Hyperframes HTML (missing attributes, overlapping tracks, etc.)
Runtime IIFE script injected into the browser — manages seek, media playback, and the window.__hf protocol
Frame Adapters Pluggable animation drivers (GSAP, Lottie, CSS, or custom)

Frame Adapters

A frame adapter tells the engine how to seek your animation to a specific frame:

import { createGSAPFrameAdapter } from "@hyperframes/core";

const adapter = createGSAPFrameAdapter({
  getTimeline: () => gsap.timeline(),
  compositionId: "my-video",
});

Implement FrameAdapter for custom animation runtimes:

import type { FrameAdapter } from "@hyperframes/core";

const myAdapter: FrameAdapter = {
  id: "my-adapter",
  getDurationFrames: () => 300,
  seekFrame: (frame) => {
    /* seek your animation */
  },
};

Parsing and generating HTML

import { parseHtml, generateHyperframesHtml } from "@hyperframes/core";

const { elements, metadata } = parseHtml(htmlString);
const html = generateHyperframesHtml(spec);

Linting

import { lintHyperframeHtml } from "@hyperframes/core/lint";

const result = lintHyperframeHtml(htmlString);
// result.findings: { severity, message, elementId }[]

Documentation

Full documentation: hyperframes.heygen.com/packages/core