mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
Xuanru Li
e786b78b33
Linear: VA-1859 ## Problem For a `fit_to_scene` B-roll where the composition's intrinsic timeline (e.g. `data-duration=1.0s` → 30 frames) is shorter than the scene it fills (e.g. 4.8s narration), the producer renders only the intrinsic 30 frames and the downstream compositor frame-holds/PTS-stretches that fixed clip to the scene length. Spreading 30 unique frames over 4.8s starves motion to ~6 effective fps → a visibly choppy result. Root cause: the producer welds one `composition.duration` to both the frame count and the 1:1 seek mapping, with no notion of a target output length. ## Fix Add optional `renderStretch: number` (default `1.0` = no-op), `renderStretch = intrinsic / target`: - **Frame count** comes from the target: `outputDuration = intrinsic / renderStretch`, `totalFrames = outputDuration × fps` (`probeStage.ts`). `composition.duration` stays intrinsic (drives video/audio windows). - **Per-frame seek** is scaled: `time = (frameIndex / fps) × renderStretch`, so the N output frames map across `[0, intrinsic]` — a fresh frame per output frame. All seek sites go through a single shared `outputFrameToTimelineSeconds(frameIndex, fps, renderStretch)` helper (`core.types.ts`), consumed by every capture path so none can silently diverge: - parallel (`parallelCoordinator.ts`), `sdr_streaming` (`captureStreamingStage.ts` ×3), `sdr_disk` (`captureStage.ts`), HDR loops. - DrawElement + static self-verify (`frameCapture.ts`) — ground-truth seek uses the same mapping, so PSNR compares like-for-like (no spurious verification failure on stretched comps). - Distributed path: `renderStretch` threaded through `DistributedRenderConfig` → chunk workers, and **folded into the plan hash only when `!= 1`** so a pre-stretch cached plan is never reused. With `renderStretch = 1` (or omitted → `?? 1`): every seek is `×1.0` (IEEE-754 identity), frame counts unchanged, and the plan hash is byte-identical — a provable no-op. `player.ts` absolute-seek is untouched. ## Verify - typecheck (core + engine + producer): pass. lint/format/fallow/commitlint: pass. `planHash` + `renderRequest` unit suites: pass. - Adversarial self-review found + fixed three capture-path gaps (streaming, self-verify, distributed) before this revision. - **Not yet runtime-verified** on a real render — needs a fit_to_scene render at `renderStretch < 1` confirming N distinct frames over the target length (draft until then). Paired with experiment-framework#42766, which computes and forwards `renderStretch = hf intrinsic / scene duration`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
@hyperframes/core
Types, parsers, generators, compiler, linter, runtime, and frame adapters for the Hyperframes video framework.
Install
npm install @hyperframes/core
Most users don't need to install core directly — the CLI, producer, and studio packages depend on it internally.
What's inside
| Module | Description |
|---|---|
| Types | TimelineElement, CompositionSpec, Asset, canvas dimensions, defaults |
| Parsers | parseHtml — extract timeline elements from HTML; parseGsapScript — parse GSAP animations |
| Generators | generateHyperframesHtml — produce valid Hyperframes HTML from a composition spec |
| Compiler | compileTimingAttrs — resolve data-start / data-duration into absolute times |
| Linter | lintHyperframeHtml — validate Hyperframes HTML (missing attributes, overlapping tracks, etc.) |
| Runtime | IIFE script injected into the browser — manages seek, media playback, and the window.__hf protocol |
| Frame Adapters | Pluggable animation drivers (GSAP, Lottie, CSS, or custom) |
Frame Adapters
A frame adapter tells the engine how to seek your animation to a specific frame:
import { createGSAPFrameAdapter } from "@hyperframes/core";
const adapter = createGSAPFrameAdapter({
getTimeline: () => gsap.timeline(),
compositionId: "my-video",
});
Implement FrameAdapter for custom animation runtimes:
import type { FrameAdapter } from "@hyperframes/core";
const myAdapter: FrameAdapter = {
id: "my-adapter",
getDurationFrames: () => 300,
seekFrame: (frame) => {
/* seek your animation */
},
};
Parsing and generating HTML
import { parseHtml, generateHyperframesHtml } from "@hyperframes/core";
const { elements, metadata } = parseHtml(htmlString);
const html = generateHyperframesHtml(spec);
Linting
import { lintHyperframeHtml } from "@hyperframes/core/lint";
const result = lintHyperframeHtml(htmlString);
// result.findings: { severity, message, elementId }[]
Documentation
Full documentation: hyperframes.heygen.com/packages/core
Related packages
@hyperframes/engine— rendering engine that drives the browser@hyperframes/producer— full render pipeline (capture + encode)hyperframes— CLI