Files
hyperframes/docs/packages/core.mdx
T
Miguel Ángel e87a389f42 fix(core): preload image decodes to eliminate jerky preview (GH #317)
Reported: compositions with many images stutter visibly during the
first ~second of preview playback, with the only workaround being to
render the composition to mp4 and watch the output file.

Root cause: every `<img>` that has never been painted before decodes
synchronously on the main thread the first time it enters the paint
tree. With a composition of several dozen images, that's several dozen
main-thread stalls stacked inside the first paint window — visible as
jank on every seek and the first few seconds of play.

Fix: a new `packages/core/src/runtime/imageDecode.ts` module that the
runtime binds alongside `bindMediaMetadataListeners`. For every `<img>`
in the document, it:

- Sets `decoding="async"` so the browser never falls back to sync-decode
  on subsequent paints.
- Calls `img.decode()` off the main thread as soon as the image has a
  `currentSrc` (either immediately if already loaded, or on the next
  `load` event otherwise).
- Tracks bound images in a Set so repeated polling from the runtime's
  observation loop is free after the first bind.

`img.decode()` is the canonical platform primitive for this — it's a
no-op for already-decoded images, runs on the decoder thread, and
resolves when the pixels are in the bitmap cache. Decode rejections
(CORS, corrupt file, 404) are intentionally swallowed because the
runtime's existing asset-error diagnostic pipeline already surfaces
those via the `error` event.

Lazy images (`loading="lazy"`) are intentionally NOT force-loaded — if
the author opted them out, we respect that; they'll decode when they're
needed.

The module is wired in three call sites in `init.ts`:
- Initial boot (after `runAdapters("discover")`)
- Post-load of external/inline compositions
- Every 10th poll tick (catches DOM changes from author scripts,
  sub-composition mounts, and dynamically-inserted content)

Tests: `imageDecode.test.ts` covers nine cases — async-flag set,
immediate vs. load-deferred decode, idempotence, torn-down no-op,
swallowed rejections, missing currentSrc, multiple imgs in one pass,
and dynamically-added imgs across bind calls. 9/9 pass.

Docs: new "Preloading and decoding" subsection in `core.mdx` under
the Runtime section, documents the invariant for future contributors.

Closes #317.
2026-04-18 18:30:16 +01:00

447 lines
13 KiB
Plaintext

---
title: "@hyperframes/core"
description: "Types, HTML generation, runtime, and linter — the foundation every other package depends on."
---
The core package provides the foundational types, HTML parsing/generation, runtime, and composition linter that all other Hyperframes packages build on. If you are building tooling, writing a custom integration, or extending Hyperframes itself, this is the package you need.
```bash
npm install @hyperframes/core
```
## When to Use
<Tip>
**Most users do not need to install `@hyperframes/core` directly.** The [CLI](/packages/cli), [producer](/packages/producer), and [studio](/packages/studio) packages all depend on core internally. You only need it if you are doing one of the things listed below.
</Tip>
**Use `@hyperframes/core` when you need to:**
- Lint compositions programmatically (CI pipelines, editor plugins)
- Parse HTML compositions into structured TypeScript objects
- Generate composition HTML from data (e.g., from an API or AI agent)
- Access the Hyperframes type system for your own tooling
- Embed the Hyperframes runtime in a custom player
**Use a different package if you want to:**
- Preview compositions in the browser — use the [CLI](/packages/cli) (`npx hyperframes preview`) or [studio](/packages/studio)
- Render compositions to MP4 — use the [CLI](/packages/cli) (`npx hyperframes render`) or [producer](/packages/producer)
- Capture frames from a headless browser — use the [engine](/packages/engine)
## Package Exports
The core package has four entry points:
| Import | Description |
|--------|-------------|
| `@hyperframes/core` | Types, parsers, generators, adapters, runtime utilities |
| `@hyperframes/core/lint` | Composition linter |
| `@hyperframes/core/compiler` | Timing compiler, HTML compiler, bundler, static guard |
| `@hyperframes/core/runtime` | Pre-built IIFE runtime for browser injection |
## Types
The core type system models compositions, timeline elements, and variables:
```typescript
import type {
TimelineElement,
TimelineMediaElement,
TimelineTextElement,
TimelineCompositionElement,
TimelineElementType, // "video" | "image" | "text" | "audio" | "composition"
CompositionSpec,
CompositionVariable,
CanvasResolution, // "landscape" | "portrait"
Orientation, // "16:9" | "9:16"
FrameAdapter,
FrameAdapterContext,
} from '@hyperframes/core';
// Type guards
import {
isTextElement,
isMediaElement,
isCompositionElement,
isStringVariable,
isNumberVariable,
isColorVariable,
isBooleanVariable,
isEnumVariable,
} from '@hyperframes/core';
// Constants
import {
CANVAS_DIMENSIONS, // { landscape: { width, height }, portrait: { width, height } }
TIMELINE_COLORS,
DEFAULT_DURATIONS,
} from '@hyperframes/core';
```
### Variable Types
Compositions can expose typed variables for dynamic content:
```typescript
import type {
CompositionVariableType, // "string" | "number" | "color" | "boolean" | "enum"
StringVariable,
NumberVariable,
ColorVariable,
BooleanVariable,
EnumVariable,
} from '@hyperframes/core';
```
### Keyframe Types
```typescript
import type {
Keyframe,
KeyframeProperties,
ElementKeyframes,
StageZoom,
StageZoomKeyframe,
} from '@hyperframes/core';
import { getDefaultStageZoom } from '@hyperframes/core';
```
## Parsing and Generating HTML
Round-trip between HTML and structured data:
```typescript
import { parseHtml, generateHyperframesHtml } from '@hyperframes/core';
import type { ParsedHtml, CompositionMetadata } from '@hyperframes/core';
// Parse HTML into structured data
const parsed: ParsedHtml = parseHtml(htmlString);
// parsed.elements, parsed.gsapScript, parsed.styles, parsed.resolution, parsed.keyframes
// Extract composition metadata
import { extractCompositionMetadata } from '@hyperframes/core';
const meta: CompositionMetadata = extractCompositionMetadata(htmlString);
// meta.id, meta.duration, meta.width, meta.height, meta.variables
// Generate HTML from structured data
const html = generateHyperframesHtml(elements, {
animations,
styles,
resolution: 'landscape',
compositionId: 'my-video',
});
```
### Modifying HTML
```typescript
import {
updateElementInHtml,
addElementToHtml,
removeElementFromHtml,
validateCompositionHtml,
} from '@hyperframes/core';
// Update an element's properties
const updatedHtml = updateElementInHtml(html, 'el-1', { start: 5 });
// Add a new element
const newHtml = addElementToHtml(html, newElement);
// Remove an element
const cleanHtml = removeElementFromHtml(html, 'el-1');
// Validate HTML structure
const result = validateCompositionHtml(html);
// result.valid, result.errors
```
### GSAP Script Parsing
```typescript
import {
parseGsapScript,
serializeGsapAnimations,
updateAnimationInScript,
addAnimationToScript,
removeAnimationFromScript,
getAnimationsForElement,
validateCompositionGsap,
keyframesToGsapAnimations,
gsapAnimationsToKeyframes,
SUPPORTED_PROPS, // animatable properties
SUPPORTED_EASES, // available easing functions
} from '@hyperframes/core';
import type { GsapAnimation, GsapMethod, ParsedGsap } from '@hyperframes/core';
// Parse GSAP script into structured animations
const parsed: ParsedGsap = parseGsapScript(scriptContent);
// parsed.animations, parsed.timelineVar, parsed.preamble, parsed.postamble
// Serialize back to script
const script = serializeGsapAnimations(parsed.animations);
```
### HTML Generation
```typescript
import {
generateHyperframesHtml,
generateGsapTimelineScript,
generateHyperframesStyles,
} from '@hyperframes/core';
// Generate a complete HTML composition
const html = generateHyperframesHtml(elements, options);
// Generate just the GSAP script
const script = generateGsapTimelineScript(animations, options);
// Generate CSS styles
const { coreCss, customCss, googleFontsLink } = generateHyperframesStyles(
elements, 'landscape', customStyles
);
```
### Template Utilities
```typescript
import {
generateBaseHtml,
getStageStyles,
GSAP_CDN,
BASE_STYLES,
ELEMENT_BASE_STYLES,
MEDIA_STYLES,
TEXT_STYLES,
ZOOM_CONTAINER_STYLES,
} from '@hyperframes/core';
// Generate base HTML structure for a resolution
const baseHtml = generateBaseHtml('landscape');
const styles = getStageStyles('portrait');
```
## Linter
The composition linter checks for structural issues that would cause rendering failures or unexpected behavior. You can run it from the CLI with `npx hyperframes lint`, or call it programmatically:
```typescript
import { lintHyperframeHtml, lintMediaUrls } from '@hyperframes/core/lint';
import type {
HyperframeLintResult,
HyperframeLintFinding,
HyperframeLintSeverity, // "error" | "warning"
HyperframeLinterOptions,
} from '@hyperframes/core/lint';
const result: HyperframeLintResult = lintHyperframeHtml(html, { filePath: 'index.html' });
// result.ok, result.errorCount, result.warningCount, result.findings
for (const finding of result.findings) {
console.log(finding.severity, finding.code, finding.message);
// finding.file, finding.selector, finding.elementId, finding.fixHint, finding.snippet
}
// Additional media URL validation
const mediaFindings = lintMediaUrls(result.findings);
```
Detected issues include:
- Missing timeline registration (`window.__timelines`)
- Unmuted video elements (causes autoplay failures)
- Missing `class="clip"` on timed visible elements
- Deprecated attribute names
- Missing composition dimensions (`data-width`, `data-height`)
- Invalid `data-start` references to nonexistent clip IDs
<Info>
For a full list of what the linter catches and how to fix each issue, see [Common Mistakes](/guides/common-mistakes) and [Troubleshooting](/guides/troubleshooting).
</Info>
## Compiler
The compiler sub-package handles timing resolution, HTML compilation, and bundling:
```typescript
// Timing compiler (browser-safe — no Node.js dependencies)
import {
compileTimingAttrs,
injectDurations,
extractResolvedMedia,
clampDurations,
} from '@hyperframes/core/compiler';
import type {
UnresolvedElement,
ResolvedDuration,
ResolvedMediaElement,
CompilationResult,
} from '@hyperframes/core/compiler';
// Compile timing attributes from HTML
const compiled: CompilationResult = compileTimingAttrs(html);
// Inject resolved durations back into HTML
const updatedHtml = injectDurations(html, compiled.durations);
// Extract resolved media elements
const media: ResolvedMediaElement[] = extractResolvedMedia(html);
```
```typescript
// HTML compiler (Node.js — requires media probing)
import { compileHtml } from '@hyperframes/core/compiler';
import type { MediaDurationProber } from '@hyperframes/core/compiler';
const prober: MediaDurationProber = async (src) => getDuration(src);
const compiledHtml = await compileHtml(html, prober);
```
```typescript
// HTML bundler (Node.js — bundles to single file)
import { bundleToSingleHtml } from '@hyperframes/core/compiler';
import type { BundleOptions } from '@hyperframes/core/compiler';
const bundled = await bundleToSingleHtml({ entryPath: './index.html', inline: true });
```
```typescript
// Static guard — validate HTML contract
import { validateHyperframeHtmlContract } from '@hyperframes/core/compiler';
import type {
HyperframeStaticGuardResult,
HyperframeStaticFailureReason,
} from '@hyperframes/core/compiler';
const guard: HyperframeStaticGuardResult = validateHyperframeHtmlContract(html);
// guard.ok, guard.failures[]
// Failure reasons: "missing_composition_id" | "missing_composition_dimensions"
// | "missing_timeline_registry" | "invalid_script_syntax"
// | "invalid_static_hyperframe_contract"
```
## Runtime
The Hyperframes runtime manages playback, seeking, and clip lifecycle in the browser. The core package provides utilities for building and loading the runtime:
```typescript
import {
loadHyperframeRuntimeSource,
buildHyperframesRuntimeScript,
HYPERFRAME_RUNTIME_ARTIFACTS,
HYPERFRAME_RUNTIME_CONTRACT,
HYPERFRAME_RUNTIME_GLOBALS,
HYPERFRAME_BRIDGE_SOURCES,
HYPERFRAME_CONTROL_ACTIONS,
} from '@hyperframes/core';
import type {
HyperframeControlAction,
HyperframesRuntimeBuildOptions,
} from '@hyperframes/core';
// Load the pre-built runtime IIFE
const runtimeSource = loadHyperframeRuntimeSource();
// Build a custom runtime script
const script = buildHyperframesRuntimeScript(options);
```
The pre-built runtime IIFE is available as a direct import:
```typescript
import runtime from '@hyperframes/core/runtime';
```
### Preloading and decoding
The runtime eagerly prepares every asset in the composition so the first
paint and the first play call never stall the compositor.
- **Media** — every `<video>` and `<audio>` has `preload="auto"` set and
`.load()` kicked on bind. The first `play()` lands on buffered data
instead of triggering a network fetch + decoder spin-up.
- **Images** — every `<img>` gets `decoding="async"` and
`img.decode()` invoked off the main thread. Compositions with many
images (several dozen through several hundred) otherwise stutter
during the first ~second of preview because each image decodes
synchronously the first time it enters the paint tree (see
[GH #317](https://github.com/heygen-com/hyperframes/issues/317)).
Both binders are **idempotent** — the runtime re-runs them periodically
as sub-compositions mount and DOM changes land, and they skip elements
they've already processed.
## Frame Adapters
The core package defines the [Frame Adapter](/concepts/frame-adapters) interface and provides the built-in GSAP adapter:
```typescript
import { createGSAPFrameAdapter } from '@hyperframes/core';
import type {
FrameAdapter,
FrameAdapterContext,
GSAPTimelineLike,
CreateGSAPFrameAdapterOptions,
} from '@hyperframes/core';
// Create a GSAP frame adapter
const adapter: FrameAdapter = createGSAPFrameAdapter({
id: 'my-composition',
fps: 30,
timeline: gsapTimeline,
});
// Adapter lifecycle
await adapter.init?.(context);
const durationFrames = adapter.getDurationFrames();
await adapter.seekFrame(42);
await adapter.destroy?.();
```
## Media Utilities
```typescript
import {
MEDIA_VISUAL_STYLE_PROPERTIES,
copyMediaVisualStyles,
quantizeTimeToFrame,
} from '@hyperframes/core';
import type { MediaVisualStyleProperty } from '@hyperframes/core';
// Quantize a time value to the nearest frame boundary
const frameTime = quantizeTimeToFrame(5.033, 30); // → 5.033... snapped to frame
// Copy visual styles between media elements
copyMediaVisualStyles(fromElement, toElement);
```
## Picker API
For element selection in editor UIs:
```typescript
import type {
HyperframePickerApi,
HyperframePickerBoundingBox,
HyperframePickerElementInfo,
} from '@hyperframes/core';
```
## Related Packages
<CardGroup cols={2}>
<Card title="CLI" icon="terminal" href="/packages/cli">
The easiest way to create, preview, lint, and render compositions.
</Card>
<Card title="Engine" icon="gear" href="/packages/engine">
Low-level frame capture pipeline that uses core types and runtime.
</Card>
<Card title="Producer" icon="film" href="/packages/producer">
Full rendering pipeline built on top of core and engine.
</Card>
<Card title="Studio" icon="palette" href="/packages/studio">
Visual composition editor that embeds the core runtime for preview.
</Card>
</CardGroup>