mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
Adds 11 progressively-disclosed reference files that the skill loads on
demand during translation. Total ~1500 LOC, every file under 200 lines
(skill-creator's progressive-disclosure budget).
api-map.md the comprehensive Remotion -> HF translation table
(the index; loaded at start of translation)
timing.md interpolate, spring (validated configs), easing,
count-up, stagger
sequencing.md Sequence, Series, Loop, Freeze, AbsoluteFill,
Composition root
media.md Audio, Video, Img, IFrame, OffthreadVideo,
staticFile, asset paths
transitions.md @remotion/transitions presentations -> manual GSAP
crossfades or HF shader-transitions
lottie.md @remotion/lottie -> HF lottie adapter (incl. AE
feature limitations note)
fonts.md Google Fonts loading, local @font-face, system
fallback noise floor
parameters.md Zod schemas, defaultProps, sync vs async
calculateMetadata
escape-hatch.md when to bow out + the runtime interop pattern
from PR #214
limitations.md known caveat patterns (volume ramps, Loop with
state, custom presentations, code-split components)
eval.md how to run the validation harness, threshold rule
of thumb, what the noise floor looks like
The references are evidence-driven rather than speculative: every spring
config, easing curve, and SSIM threshold is documented from the
validated T1/T2/T3 calibration runs (mean 0.974 / 0.985 / 0.953). The
escape-hatch boundaries match the lint blockers in PR 2 and the T4
fixtures in PR 5.
Replaces the placeholder .gitkeep from PR 1.
150 lines
3.8 KiB
Markdown
150 lines
3.8 KiB
Markdown
# Media translation: Audio, Video, Img, IFrame, staticFile
|
|
|
|
## Asset paths
|
|
|
|
Remotion's `staticFile("x.png")` resolves to the project's `public/` directory.
|
|
HF uses relative paths from the composition's `index.html`, conventionally
|
|
`assets/`:
|
|
|
|
```tsx
|
|
<Img src={staticFile("logo.png")} />
|
|
```
|
|
|
|
```html
|
|
<img src="assets/logo.png" />
|
|
```
|
|
|
|
When translating, copy the asset from `remotion-src/public/x` to
|
|
`hf-src/assets/x`. Multiple files can be batched with a setup script;
|
|
see T2's `setup.sh` for an example pattern.
|
|
|
|
## `<Audio>`
|
|
|
|
```tsx
|
|
<Audio src={staticFile("music.wav")} volume={0.5} />
|
|
```
|
|
|
|
```html
|
|
<audio
|
|
data-start="0"
|
|
data-duration="6"
|
|
data-track-index="2"
|
|
data-volume="0.5"
|
|
src="assets/music.wav"
|
|
></audio>
|
|
```
|
|
|
|
`data-start` and `data-duration` are required — the runtime needs them to
|
|
schedule the audio. Default to the composition's full duration if Remotion
|
|
didn't specify trim.
|
|
|
|
### Volume ramps
|
|
|
|
```tsx
|
|
<Audio src={staticFile("music.wav")} volume={(f) => interpolate(f, [0, 30], [0, 1])} />
|
|
```
|
|
|
|
HF supports static `data-volume` only for now. Volume ramps need to be
|
|
applied to the audio file at translation time (with ffmpeg `afade`) or the
|
|
ramp is dropped with a translation note.
|
|
|
|
### Trim / playbackRate
|
|
|
|
```tsx
|
|
<Audio src={staticFile("music.wav")} startFrom={60} endAt={180} playbackRate={1.5} />
|
|
```
|
|
|
|
```html
|
|
<audio
|
|
data-start="0"
|
|
data-duration="<resolved from trim>"
|
|
data-trim-start="2"
|
|
data-trim-end="6"
|
|
data-playback-rate="1.5"
|
|
src="assets/music.wav"
|
|
></audio>
|
|
```
|
|
|
|
`startFrom` / `endAt` are frame indexes; convert to seconds.
|
|
|
|
## `<Video>` and `<OffthreadVideo>`
|
|
|
|
```tsx
|
|
<Video src={staticFile("intro.mp4")} muted playsInline />
|
|
<OffthreadVideo src={staticFile("intro.mp4")} muted />
|
|
```
|
|
|
|
```html
|
|
<video
|
|
muted
|
|
playsinline
|
|
data-start="0"
|
|
data-duration="5"
|
|
data-track-index="0"
|
|
src="assets/intro.mp4"
|
|
></video>
|
|
```
|
|
|
|
`<OffthreadVideo>` is a Remotion-specific optimization for headless
|
|
rendering. HF runs in headless Chrome already, so the off-thread variant
|
|
collapses to a regular `<video>`.
|
|
|
|
`muted` and `playsinline` are required for the runtime to autoplay
|
|
(browser policy). Always emit them.
|
|
|
|
## `<Img>`
|
|
|
|
```tsx
|
|
<Img src={staticFile("logo.png")} style={{ width: 200, height: 200 }} />
|
|
```
|
|
|
|
```html
|
|
<img src="assets/logo.png" style="width: 200px; height: 200px;" />
|
|
```
|
|
|
|
Width/height get rounded to integer px. If the original style has
|
|
animated dimensions, the GSAP tween animates them — see [timing.md](timing.md).
|
|
|
|
## `<IFrame>`
|
|
|
|
```tsx
|
|
<IFrame src="https://example.com" />
|
|
```
|
|
|
|
```html
|
|
<iframe src="https://example.com"></iframe>
|
|
```
|
|
|
|
When HF detects a nested iframe in a composition, it auto-falls back to
|
|
**screenshot mode** rather than the deterministic BeginFrame mode. This
|
|
costs render performance but produces visibly-correct output. See
|
|
[hyperframes-vs-remotion.mdx](https://github.com/heygen-com/hyperframes/blob/main/docs/guides/hyperframes-vs-remotion.mdx)
|
|
for details.
|
|
|
|
## `delayRender()` / `continueRender()`
|
|
|
|
```tsx
|
|
const handle = delayRender();
|
|
useEffect(() => {
|
|
loadAsset().then(() => continueRender(handle));
|
|
}, []);
|
|
```
|
|
|
|
Drop. HF waits on asset readiness via the [Frame Adapter pattern](https://hyperframes.heygen.com/concepts/frame-adapters)
|
|
— images, videos, fonts, and Lottie animations all signal load
|
|
completion natively. There's nothing to do at the application level.
|
|
|
|
## When the asset isn't a file
|
|
|
|
If Remotion's media source is a Buffer, dataURL, or URL.createObjectURL,
|
|
the asset doesn't exist on disk and can't be copied via setup.sh. Two
|
|
options:
|
|
|
|
1. Materialize the asset at translation time — write the buffer to a file
|
|
in `hf-src/assets/`.
|
|
2. Embed as a data URL directly in the HTML (`src="data:image/png;base64,..."`)
|
|
for small assets (< 100 KB).
|
|
|
|
For audio/video Buffers, option 1 is preferred — base64-encoded media
|
|
bloats the HTML and slows the renderer.
|