James RussoandClaude Opus 4.6 b23b0751da fix(player): parent-frame media playback for mobile (#266)
* fix(player): parent-frame media playback for mobile

Mobile browsers block media.play() inside iframes when the user
gesture happened in the parent frame — postMessage doesn't transfer
user activation (per the User Activation v2 spec).

## Problem

The player renders compositions in a sandboxed iframe. When a user
taps play in the parent frame, the player sends a postMessage to the
iframe's runtime, which calls audio.play(). On mobile, this fails
silently because the iframe has no user activation context.

## Solution

The player now extracts ALL timed media elements (audio/video with
data-start) from the iframe's DOM (same-origin access), creates
parent-frame copies, and disables the iframe originals. On play(),
parentMedia.play() runs synchronously in the gesture call stack,
satisfying mobile autoplay policy.

### Generic media handling

- Finds all `audio[data-start], video[data-start]` in the iframe
- Creates a parent-frame copy for each (Audio or Video element)
- Preserves data-start offsets for correct seek positioning
- Strips data-start from iframe elements so the runtime ignores them
- Falls back to iframe media for cross-origin iframes

### `audio-src` attribute

Convenience for the common single-narration case. When set, the
player starts preloading audio immediately — before the iframe loads.
This eliminates the loading delay that caused jittery playback.

### No active sync

Both parent media and the GSAP timeline are real-time systems. When
started simultaneously, they naturally stay within ~10ms — no drift
correction needed. Active sync with coarse granularity (50ms polling)
caused MORE jitter than it prevented via repeated audio seeks.

## CI

- Added unified `test` job replacing separate per-package test jobs
- Added root `test` script: `bun run --filter '*' test`
- New packages with test scripts are automatically included
- Added happy-dom for player DOM tests

## Tests

- 10 new tests for parent-frame media: preloading, play, pause,
  seek, muted/rate sync, cleanup, attribute changes
- All 21 player tests pass

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(shader-transitions): pass CI when no test files exist

Add --passWithNoTests to vitest run so the unified test job
doesn't fail on packages that have a test script but no test
files yet.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update tests for new id field and GSAP lint rule

- normalize.test.ts: loadTranscript now assigns id fields (w0, w1, etc.)
  to SRT/VTT results and empty string for words-json passthrough
- lintProject.test.ts: add GSAP CDN script to validHtml() fixture to
  satisfy the missing_gsap_script lint rule added in core

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add missing data-start/data-duration to validHtml fixture

The validHtml() test fixture was missing data-start and data-duration
attributes, triggering the root_composition_missing_data_start and
root_composition_missing_data_duration lint warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): fetch LFS objects for producer test job

Producer regression tests compare rendered output against reference MP4
files stored in git LFS. Without lfs: true, checkout fetches pointer
files instead of actual videos, causing "moov atom not found" errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: remove redundant test-producer job

The regression workflow already runs the same 28 producer fixtures
in a Docker container with prod-matching Chrome/fonts/ffmpeg, sharded
across 8 parallel matrix jobs with 40-min timeouts. The CI test-producer
job was a duplicate that ran on bare runners with worse determinism
and a 15-min timeout too short for all fixtures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:20:07 -07:00
2026-03-21 22:43:56 -07:00
2026-03-21 22:43:56 -07:00

Hyperframes

npm version License Node.js

Write HTML. Render video. Built for agents.

Hyperframes is an open-source video rendering framework that lets you create, preview, and render HTML-based video compositions — with first-class support for AI agents via MCP.

Why Hyperframes?

  • HTML-native — AI agents already speak HTML. No React required.
  • Frame Adapter pattern — bring your own animation runtime (GSAP, Lottie, CSS, Three.js).
  • Deterministic rendering — same input = identical output. Built for automated pipelines.
  • AI-first design — not a bolted-on afterthought.

Quick Start

npx hyperframes init my-video
cd my-video

Then open the project with your AI coding agent (Claude Code, Cursor, etc.) — it has HyperFrames skills installed and knows how to create and edit compositions.

npx hyperframes preview      # preview in browser (live reload)
npx hyperframes render   # render to MP4

Requirements: Node.js >= 22, FFmpeg

Documentation

Full documentation at hyperframes.heygen.com — start with the Quickstart, then explore guides, concepts, API reference, and package docs.

How It Works

Define your video as HTML with data attributes:

<div id="stage" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
  <video
    id="clip-1"
    data-start="0"
    data-duration="5"
    data-track="0"
    src="intro.mp4"
    muted
    playsinline
  ></video>
  <img id="overlay" data-start="2" data-duration="3" data-track="1" src="logo.png" />
  <audio
    id="bg-music"
    data-start="0"
    data-duration="9"
    data-track="2"
    data-volume="0.5"
    src="music.wav"
  ></audio>
</div>

Preview instantly in the browser. Render to MP4 locally. Let AI agents compose videos using tools they already understand.

Packages

Package Description
hyperframes CLI — create, preview, lint, and render compositions
@hyperframes/core Types, parsers, generators, linter, runtime, frame adapters
@hyperframes/engine Seekable page-to-video capture engine (Puppeteer + FFmpeg)
@hyperframes/producer Full rendering pipeline (capture + encode + audio mix)
@hyperframes/studio Browser-based composition editor UI

AI Agent Skills

HyperFrames ships skills that teach AI coding agents (Claude Code, Gemini CLI, Codex, Cursor) how to write correct compositions and GSAP animations. Use these instead of writing from scratch — they encode framework-specific patterns that generic docs don't cover.

# Install all skills (HyperFrames + GSAP) — runs automatically during `hyperframes init`
npx hyperframes skills

# Or install to a specific agent
npx hyperframes skills --claude
npx hyperframes skills --cursor

Or via npx skills add

# HyperFrames skills (hyperframes-compose, hyperframes-captions)
npx skills add heygen-com/hyperframes

# GSAP skills (gsap-core, gsap-timeline, gsap-scrolltrigger, gsap-plugins, gsap-performance, gsap-utils, gsap-react, gsap-frameworks)
npx skills add greensock/gsap-skills

Installed Skills

Source Skills What they teach
HyperFrames hyperframes-compose, hyperframes-captions HTML composition structure, class="clip" rules, data-* attributes, timeline registration, rendering constraints
GSAP gsap-core, gsap-timeline, gsap-performance, gsap-plugins, gsap-scrolltrigger, gsap-utils, gsap-react, gsap-frameworks Core API, timeline sequencing, ScrollTrigger, plugin usage, performance best practices

In Claude Code, invoke with /hyperframes-compose, /hyperframes-captions, /gsap-core, etc.

Contributing

See CONTRIBUTING.md for guidelines on how to contribute.

License

See LICENSE for details.

S
Description
Write HTML. Render video. Built for agents.
Readme
750 MiB
Languages
JavaScript 56%
TypeScript 43.8%