mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-15 22:08:03 +00:00
Miguel Ángel
e87a389f42
Reported: compositions with many images stutter visibly during the
first ~second of preview playback, with the only workaround being to
render the composition to mp4 and watch the output file.
Root cause: every `<img>` that has never been painted before decodes
synchronously on the main thread the first time it enters the paint
tree. With a composition of several dozen images, that's several dozen
main-thread stalls stacked inside the first paint window — visible as
jank on every seek and the first few seconds of play.
Fix: a new `packages/core/src/runtime/imageDecode.ts` module that the
runtime binds alongside `bindMediaMetadataListeners`. For every `<img>`
in the document, it:
- Sets `decoding="async"` so the browser never falls back to sync-decode
on subsequent paints.
- Calls `img.decode()` off the main thread as soon as the image has a
`currentSrc` (either immediately if already loaded, or on the next
`load` event otherwise).
- Tracks bound images in a Set so repeated polling from the runtime's
observation loop is free after the first bind.
`img.decode()` is the canonical platform primitive for this — it's a
no-op for already-decoded images, runs on the decoder thread, and
resolves when the pixels are in the bitmap cache. Decode rejections
(CORS, corrupt file, 404) are intentionally swallowed because the
runtime's existing asset-error diagnostic pipeline already surfaces
those via the `error` event.
Lazy images (`loading="lazy"`) are intentionally NOT force-loaded — if
the author opted them out, we respect that; they'll decode when they're
needed.
The module is wired in three call sites in `init.ts`:
- Initial boot (after `runAdapters("discover")`)
- Post-load of external/inline compositions
- Every 10th poll tick (catches DOM changes from author scripts,
sub-composition mounts, and dynamically-inserted content)
Tests: `imageDecode.test.ts` covers nine cases — async-flag set,
immediate vs. load-deferred decode, idempotence, torn-down no-op,
swallowed rejections, missing currentSrc, multiple imgs in one pass,
and dynamically-added imgs across bind calls. 9/9 pass.
Docs: new "Preloading and decoding" subsection in `core.mdx` under
the Runtime section, documents the invariant for future contributors.
Closes #317.
@hyperframes/core
Types, parsers, generators, compiler, linter, runtime, and frame adapters for the Hyperframes video framework.
Install
npm install @hyperframes/core
Most users don't need to install core directly — the CLI, producer, and studio packages depend on it internally.
What's inside
| Module | Description |
|---|---|
| Types | TimelineElement, CompositionSpec, Asset, canvas dimensions, defaults |
| Parsers | parseHtml — extract timeline elements from HTML; parseGsapScript — parse GSAP animations |
| Generators | generateHyperframesHtml — produce valid Hyperframes HTML from a composition spec |
| Compiler | compileTimingAttrs — resolve data-start / data-duration into absolute times |
| Linter | lintHyperframeHtml — validate Hyperframes HTML (missing attributes, overlapping tracks, etc.) |
| Runtime | IIFE script injected into the browser — manages seek, media playback, and the window.__hf protocol |
| Frame Adapters | Pluggable animation drivers (GSAP, Lottie, CSS, or custom) |
Frame Adapters
A frame adapter tells the engine how to seek your animation to a specific frame:
import { createGSAPFrameAdapter } from "@hyperframes/core";
const adapter = createGSAPFrameAdapter({
getTimeline: () => gsap.timeline(),
compositionId: "my-video",
});
Implement FrameAdapter for custom animation runtimes:
import type { FrameAdapter } from "@hyperframes/core";
const myAdapter: FrameAdapter = {
id: "my-adapter",
getDurationFrames: () => 300,
seekFrame: (frame) => {
/* seek your animation */
},
};
Parsing and generating HTML
import { parseHtml, generateHyperframesHtml } from "@hyperframes/core";
const { elements, metadata } = parseHtml(htmlString);
const html = generateHyperframesHtml(spec);
Linting
import { lintHyperframeHtml } from "@hyperframes/core/lint";
const result = lintHyperframeHtml(htmlString);
// result.findings: { severity, message, elementId }[]
Documentation
Full documentation: hyperframes.heygen.com/packages/core
Related packages
@hyperframes/engine— rendering engine that drives the browser@hyperframes/producer— full render pipeline (capture + encode)hyperframes— CLI