fix(producer): give inlined media a document-unique render id (#3342)

* fix(producer): give inlined media a document-unique render id

Element ids are unique per composition file, but the render document is
the inlined union of every file. The producer merged the per-file media
lists and deduplicated by id, so clips that shared an id collapsed into a
single entry, and every id-keyed stage (extract, inject, visibility,
bounds) resolved to whichever element came first in the document. The
surviving clip's frames landed on the wrong element and the visible scene
rendered without footage.

Two shapes hit this, and neither is author error:

  - Two scenes that each declare `<video id="clip">`. Legal per file, and
    unavoidable when a scene is duplicated into a copy with inner ids
    kept, or when one file is mounted twice.
  - Two scenes that each declare a bare `<video>`. The timing compiler
    numbers auto-ids per file, so both arrive as `hf-video-0` with no
    authored id involved at all.

Stamp a document-unique `data-hf-render-id` while inlining, and read the
media list off the inlined document instead of merging per-file lists.
The render id equals the element id whenever that id is already unique,
so documents without a collision keep identical pipeline keys.

Author `id` attributes are left alone: 158 of the 161 registry blocks
reference their own ids from `#id` CSS or getElementById, so renaming
would trade broken footage for broken styling. The engine resolves media
elements through the render id instead, falling back to getElementById
for documents the producer never compiled.

Collecting from the inlined document also retires the per-file media
extraction in parseSubCompositions along with its offset bookkeeping;
host offsets are recovered from the composition hosts the clip sits in.

* fix(core): resolve render-frame siblings by render id in the runtime

The injector creates each `__render_frame_<id>__` sibling from the media
element's render id, but four runtime readers still built that id from the
plain `el.id`. On a document where two compositions share a media id, all
of them resolved the first collider's frame.

colorGrading is the one that changes pixels: findRenderFrameImage returns
the image the grading pass samples, with no class check to catch the
mismatch, so the second video was graded from the first one's frame.
media, mediaProxy and video-texture-compat use it as a render-mode or
substitute-source signal, where both colliders happen to agree during
render, but none of them should rest on that.

Add renderFrameSibling as the single owner of "which frame belongs to
this element" and route all four through it. It reads the stamped render
id and falls back to the author id, so a collision-free document resolves
exactly as before and an uncompiled one (preview, snapshot, check) is
unchanged.

The engine's in-page bridge keeps its own copy of the rule because code
serialized into page.evaluate cannot import; it now names core as the
definition, and a test pins the sibling-id format both sides build so
they cannot drift apart silently.

* refactor(engine): build render-frame sibling ids from core's definition

The drift guard named both sides but pinned one. renderFrameSibling.test
asserts core's format, while the engine rebuilt the same id from a literal
template at six independent sites. Changing the format on either side left
the test green and every runtime reader silently unable to find its frame —
this PR's own failure mode, one level up.

Export the affixes and renderFrameIdForRenderId from core, and take the id
from there at all six. Four sites resolve it on the Node side, where the
engine can import; the two that iterate the DOM in-page receive the affixes
as evaluate arguments, which avoids depending on bridge install order.

Also switch two `__hfMediaId?.(el) ?? el.id` reads to `||`. The bridge
returns "" for an element with neither id, so `??` kept the empty string
and built `__render_frame___`, which no reader looks for. Inert today
because the compiler assigns positional ids to id-less timed media, but it
made the two sides disagree in the one case they could.
This commit is contained in:
Miguel Ángel
2026-08-19 00:24:07 -04:00
committed by GitHub
parent ec0b23f3ce
commit 634df5a5af
20 changed files with 1086 additions and 453 deletions
+2
View File
@@ -89,3 +89,5 @@ export {
// Asset-path primitives (shared across core, producer, CLI) // Asset-path primitives (shared across core, producer, CLI)
export { CSS_URL_RE, PATH_ATTRS, isNonRelativeUrl, isPathInside } from "./assetPaths"; export { CSS_URL_RE, PATH_ATTRS, isNonRelativeUrl, isPathInside } from "./assetPaths";
export { MEDIA_RENDER_ID_ATTR, assignMediaRenderIds } from "./mediaRenderIds";
@@ -0,0 +1,79 @@
import { describe, it, expect } from "vitest";
import { parseHTML } from "linkedom";
import { MEDIA_RENDER_ID_ATTR, assignMediaRenderIds } from "./mediaRenderIds";
function stamp(html: string): string[] {
const { document } = parseHTML(html);
assignMediaRenderIds(document as unknown as Parameters<typeof assignMediaRenderIds>[0]);
return Array.from(document.querySelectorAll("video, audio, img")).map(
(el) => el.getAttribute(MEDIA_RENDER_ID_ATTR) ?? "",
);
}
describe("assignMediaRenderIds", () => {
it("keeps the element id when it is already unique", () => {
expect(stamp('<video id="hero" src="a.mp4">')).toEqual(["hero"]);
});
it("disambiguates a media id shared by two inlined compositions", () => {
// Two scenes each authored `<video id="clip">`: legal per file, duplicated
// once both are inlined into one render document.
expect(stamp('<video id="clip" src="a.mp4"><video id="clip" src="a.mp4">')).toEqual([
"clip",
"clip__hf2",
]);
});
it("disambiguates per-file auto-ids, which collide without any authored id", () => {
// The timing compiler numbers unnamed media per file, so two bare <video>s
// in two scenes both arrive as `hf-video-0`.
expect(stamp('<video id="hf-video-0" src="a.mp4"><video id="hf-video-0" src="b.mp4">')).toEqual(
["hf-video-0", "hf-video-0__hf2"],
);
});
it("keeps disambiguating past the first collision", () => {
const html = '<video id="c" src="a.mp4"><video id="c" src="a.mp4"><video id="c" src="a.mp4">';
expect(stamp(html)).toEqual(["c", "c__hf2", "c__hf3"]);
});
it("separates ids across tag types", () => {
expect(
stamp('<video id="m" src="a.mp4"><audio id="m" src="a.mp3"><img id="m" src="a.png">'),
).toEqual(["m", "m__hf2", "m__hf3"]);
});
it("does not renumber elements that already carry a render id", () => {
const { document } = parseHTML(
`<video id="clip" ${MEDIA_RENDER_ID_ATTR}="clip" src="a.mp4">` +
`<video id="clip" ${MEDIA_RENDER_ID_ATTR}="clip__hf2" src="a.mp4">`,
);
assignMediaRenderIds(document as unknown as Parameters<typeof assignMediaRenderIds>[0]);
expect(
Array.from(document.querySelectorAll("video")).map((el) =>
el.getAttribute(MEDIA_RENDER_ID_ATTR),
),
).toEqual(["clip", "clip__hf2"]);
});
it("does not claim an id that a later element already holds as its render id", () => {
// Re-running over a partially stamped document must not hand `clip__hf2`
// to the first element and collide with the element already holding it.
const { document } = parseHTML(
`<video id="clip__hf2" src="a.mp4">` +
`<video id="clip" ${MEDIA_RENDER_ID_ATTR}="clip__hf2" src="a.mp4">`,
);
assignMediaRenderIds(document as unknown as Parameters<typeof assignMediaRenderIds>[0]);
const ids = Array.from(document.querySelectorAll("video")).map((el) =>
el.getAttribute(MEDIA_RENDER_ID_ATTR),
);
expect(new Set(ids).size).toBe(2);
expect(ids[1]).toBe("clip__hf2");
});
it("leaves media without a src alone", () => {
const { document } = parseHTML('<video id="no-src"></video>');
assignMediaRenderIds(document as unknown as Parameters<typeof assignMediaRenderIds>[0]);
expect(document.querySelector("video")?.hasAttribute(MEDIA_RENDER_ID_ATTR)).toBe(false);
});
});
@@ -0,0 +1,87 @@
/**
* Document-unique identity for media elements in a compiled render document.
*
* Element `id`s are only unique within one composition FILE. The render
* document is the inlined union of every file, so ids collide there in two
* ways an author cannot avoid:
*
* 1. Two scenes each declare `<video id="clip">` — legal per file, duplicated
* once inlined.
* 2. Two scenes each declare a bare `<video>` — the timing compiler numbers
* auto-ids per file, so both become `hf-video-0`.
*
* The render pipeline keys media on that id (extract, inject, visibility,
* bounds), so a collision collapses N elements into one entry and every
* lookup resolves to whichever element happens to come first in the document.
* The surviving clip's frames land on the wrong element and the visible scene
* paints without footage.
*
* This module is the single owner of the fix: after inlining, every media
* element gets a document-unique `data-hf-render-id`. It equals the element's
* own id whenever that id is already unique, so uncolliding documents keep
* byte-identical pipeline keys and log output. Author-visible `id` attributes
* are never rewritten — 158 of the 161 registry blocks reference their own ids
* from `#id` CSS or `getElementById`, so renaming would break scene styling to
* fix scene footage.
*/
export const MEDIA_RENDER_ID_ATTR = "data-hf-render-id";
/** Elements the render pipeline addresses by id. */
const MEDIA_SELECTOR = "video[src], audio[src], img[src]";
interface MediaElementLike {
getAttribute(name: string): string | null;
setAttribute(name: string, value: string): void;
}
interface DocumentLike {
querySelectorAll(selector: string): Iterable<MediaElementLike>;
}
/**
* Derive a document-unique render id from an element's own id.
*
* `taken` accumulates every id handed out so far, including the plain ids of
* elements that have not been visited yet is NOT required: a later element
* whose plain id was already claimed simply gets a suffix. Document order
* therefore decides who keeps the plain id, which keeps the first (and, in the
* overwhelmingly common single-occurrence case, only) element stable.
*/
function uniqueRenderId(baseId: string, taken: Set<string>): string {
if (!taken.has(baseId)) return baseId;
let suffix = 2;
while (taken.has(`${baseId}__hf${suffix}`)) suffix += 1;
return `${baseId}__hf${suffix}`;
}
/**
* Stamp `data-hf-render-id` on every media element in a compiled document.
*
* Idempotent: an element that already carries the attribute keeps it, so
* re-compiling a document (the resolved-durations recompile path) does not
* renumber ids out from under an in-flight extraction.
*/
export function assignMediaRenderIds(document: DocumentLike): void {
const taken = new Set<string>();
const pending: MediaElementLike[] = [];
for (const el of document.querySelectorAll(MEDIA_SELECTOR)) {
const existing = el.getAttribute(MEDIA_RENDER_ID_ATTR);
if (existing) {
taken.add(existing);
continue;
}
pending.push(el);
}
for (const el of pending) {
const baseId = el.getAttribute("id");
// An element with no id yet is numbered by the timing compiler before this
// runs. If one slips through, fall back to a positional id rather than
// stamping an empty string that every other id-less element would share.
const renderId = uniqueRenderId(baseId || `hf-media-${taken.size}`, taken);
taken.add(renderId);
el.setAttribute(MEDIA_RENDER_ID_ATTR, renderId);
}
}
+8
View File
@@ -144,6 +144,14 @@ export {
MEDIA_DURATION_CLAMP_EPSILON_SECONDS, MEDIA_DURATION_CLAMP_EPSILON_SECONDS,
} from "./compiler/timingCompiler"; } from "./compiler/timingCompiler";
export { MEDIA_RENDER_ID_ATTR, assignMediaRenderIds } from "./compiler/mediaRenderIds";
export {
RENDER_FRAME_ID_PREFIX,
RENDER_FRAME_ID_SUFFIX,
renderFrameIdForRenderId,
} from "./runtime/renderFrameSibling";
// Lint moved to @hyperframes/lint. Import lint APIs from @hyperframes/lint // Lint moved to @hyperframes/lint. Import lint APIs from @hyperframes/lint
// directly, or via the back-compat stub at @hyperframes/core/lint. Not // directly, or via the back-compat stub at @hyperframes/core/lint. Not
// re-exported here — doing so would cycle core's main entry through the lint // re-exported here — doing so would cycle core's main entry through the lint
@@ -13,13 +13,15 @@
* sibling), the original `<video>` path is used unchanged. * sibling), the original `<video>` path is used unchanged.
*/ */
import { findInjectedRenderFrame } from "../renderFrameSibling.js";
/** /**
* Resolve the decoded render-frame `<img>` for a source `<video>`, if the * Resolve the decoded render-frame `<img>` for a source `<video>`, if the
* engine has injected one and it has decoded pixels. Returns null in preview * engine has injected one and it has decoded pixels. Returns null in preview
* mode or before the frame is decoded, so callers fall back to the video. * mode or before the frame is decoded, so callers fall back to the video.
* *
* The injector inserts the `<img>` as the video's immediate next sibling and * The injector inserts the `<img>` as the video's immediate next sibling and
* also gives it the id `__render_frame_<videoId>__`; we check the sibling * also gives it the id `__render_frame_<renderId>__`; we check the sibling
* first (cheap) and fall back to an id lookup in case a node was inserted * first (cheap) and fall back to an id lookup in case a node was inserted
* between them. * between them.
*/ */
@@ -33,11 +35,9 @@ function resolveRenderFrameImage(video: HTMLVideoElement): HTMLImageElement | nu
) { ) {
return sibling; return sibling;
} }
if (video.id) { const byId = findInjectedRenderFrame(video);
const byId = document.getElementById(`__render_frame_${video.id}__`); if (byId && byId.complete && byId.naturalWidth > 0) {
if (byId instanceof HTMLImageElement && byId.complete && byId.naturalWidth > 0) { return byId;
return byId;
}
} }
return null; return null;
} }
+3 -3
View File
@@ -41,6 +41,7 @@ import {
import { copyMediaVisualStyles } from "../inline-scripts/parityContract"; import { copyMediaVisualStyles } from "../inline-scripts/parityContract";
import { readVariablesForElement } from "./variableScope"; import { readVariablesForElement } from "./variableScope";
import { swallow } from "./diagnostics"; import { swallow } from "./diagnostics";
import { findInjectedRenderFrame } from "./renderFrameSibling";
type ColorGradingMediaElement = HTMLVideoElement | HTMLImageElement; type ColorGradingMediaElement = HTMLVideoElement | HTMLImageElement;
@@ -2470,9 +2471,8 @@ function isDrawableSource(source: TexImageSource): boolean {
} }
function findRenderFrameImage(video: HTMLVideoElement): HTMLImageElement | null { function findRenderFrameImage(video: HTMLVideoElement): HTMLImageElement | null {
if (!video.id) return null; const frame = findInjectedRenderFrame(video);
const frame = document.getElementById(`__render_frame_${video.id}__`); return frame && isDrawableSource(frame) ? frame : null;
return frame instanceof HTMLImageElement && isDrawableSource(frame) ? frame : null;
} }
function hasInjectedRenderFrame(element: ColorGradingMediaElement): boolean { function hasInjectedRenderFrame(element: ColorGradingMediaElement): boolean {
+2 -2
View File
@@ -3,6 +3,7 @@ import { interpolateVolumeGain, type VolumeKeyframe } from "./mediaVolumeEnvelop
import { elementVolumeLaneGain } from "./audioAutomationVolume.js"; import { elementVolumeLaneGain } from "./audioAutomationVolume.js";
import { readElementPlaybackRate, readMediaStart } from "./playbackRate.js"; import { readElementPlaybackRate, readMediaStart } from "./playbackRate.js";
import { clampAudioGain } from "../audioGain.js"; import { clampAudioGain } from "../audioGain.js";
import { findInjectedRenderFrame } from "./renderFrameSibling.js";
export { readElementPlaybackRate, resolveNaturalMediaTimelineDuration } from "./playbackRate.js"; export { readElementPlaybackRate, resolveNaturalMediaTimelineDuration } from "./playbackRate.js";
export function readElementPlaybackStart(el: Element): number { export function readElementPlaybackStart(el: Element): number {
@@ -410,8 +411,7 @@ export function syncRuntimeMedia(params: {
// effect during render, and the per-tick set just kicks Chrome's // effect during render, and the per-tick set just kicks Chrome's
// media pipeline for nothing. Preview is unaffected (the sibling // media pipeline for nothing. Preview is unaffected (the sibling
// only exists during render). // only exists during render).
const skipForInjectedVideo = const skipForInjectedVideo = el.tagName === "VIDEO" && !!findInjectedRenderFrame(el);
el.tagName === "VIDEO" && el.id && !!document.getElementById(`__render_frame_${el.id}__`);
if (!skipForInjectedVideo) { if (!skipForInjectedVideo) {
try { try {
el.currentTime = relTime; el.currentTime = relTime;
+2 -5
View File
@@ -1,6 +1,7 @@
import { postRuntimeMessage } from "./bridge"; import { postRuntimeMessage } from "./bridge";
import { swallow } from "./diagnostics"; import { swallow } from "./diagnostics";
import { evictMediaSyncState } from "./media"; import { evictMediaSyncState } from "./media";
import { findInjectedRenderFrame } from "./renderFrameSibling";
import type { RuntimeJson } from "./types"; import type { RuntimeJson } from "./types";
/** /**
@@ -62,11 +63,7 @@ function currentSrcValue(el: HTMLMediaElement): string {
*/ */
function isRenderMode(el: HTMLMediaElement): boolean { function isRenderMode(el: HTMLMediaElement): boolean {
if (window.__HF_EXPORT_RENDER_SEEK_CONFIG) return true; if (window.__HF_EXPORT_RENDER_SEEK_CONFIG) return true;
return ( return el instanceof HTMLVideoElement && !!findInjectedRenderFrame(el);
el instanceof HTMLVideoElement &&
!!el.id &&
!!document.getElementById(`__render_frame_${el.id}__`)
);
} }
/** /**
@@ -0,0 +1,74 @@
import { describe, it, expect, beforeEach } from "vitest";
import { MEDIA_RENDER_ID_ATTR } from "../compiler/mediaRenderIds";
import {
readMediaRenderId,
renderFrameElementId,
findInjectedRenderFrame,
} from "./renderFrameSibling";
function videoWith(attrs: Record<string, string>): HTMLVideoElement {
const el = document.createElement("video");
for (const [name, value] of Object.entries(attrs)) el.setAttribute(name, value);
return el;
}
describe("readMediaRenderId", () => {
it("prefers the stamped render id over the author id", () => {
expect(readMediaRenderId(videoWith({ id: "clip", [MEDIA_RENDER_ID_ATTR]: "clip__hf2" }))).toBe(
"clip__hf2",
);
});
it("falls back to the author id in an uncompiled document", () => {
expect(readMediaRenderId(videoWith({ id: "clip" }))).toBe("clip");
});
it("returns null when the element has neither", () => {
expect(readMediaRenderId(videoWith({}))).toBeNull();
});
});
describe("renderFrameElementId", () => {
// Pins the id format the engine's in-page bridge mirrors when it CREATES the
// sibling (screenshotService.ensureRenderFrameSiblings). If this format
// changes on one side only, the readers stop finding the frame.
it("wraps the render id in the injector's sibling id format", () => {
expect(renderFrameElementId(videoWith({ id: "hero" }))).toBe("__render_frame_hero__");
expect(renderFrameElementId(videoWith({ id: "c", [MEDIA_RENDER_ID_ATTR]: "c__hf2" }))).toBe(
"__render_frame_c__hf2__",
);
});
});
describe("findInjectedRenderFrame", () => {
beforeEach(() => {
document.body.innerHTML = "";
});
it("resolves each colliding video to its own frame, not the first one's", () => {
// Two scenes sharing `<video id="clip">`. Resolving by author id returned
// scene-a's frame for both, so scene-b read another clip's pixels.
document.body.innerHTML =
`<video id="clip" ${MEDIA_RENDER_ID_ATTR}="clip"></video>` +
`<img id="__render_frame_clip__" class="__render_frame__">` +
`<video id="clip" ${MEDIA_RENDER_ID_ATTR}="clip__hf2"></video>` +
`<img id="__render_frame_clip__hf2__" class="__render_frame__">`;
const [first, second] = Array.from(document.querySelectorAll("video"));
expect(findInjectedRenderFrame(first!)?.id).toBe("__render_frame_clip__");
expect(findInjectedRenderFrame(second!)?.id).toBe("__render_frame_clip__hf2__");
});
it("still resolves by author id when the document was never compiled", () => {
document.body.innerHTML =
'<video id="solo"></video><img id="__render_frame_solo__" class="__render_frame__">';
expect(findInjectedRenderFrame(document.querySelector("video")!)?.id).toBe(
"__render_frame_solo__",
);
});
it("returns null in preview, where no sibling exists", () => {
document.body.innerHTML = '<video id="solo"></video>';
expect(findInjectedRenderFrame(document.querySelector("video")!)).toBeNull();
});
});
@@ -0,0 +1,59 @@
/**
* Resolve the injected render-frame `<img>` that stands in for a `<video>`
* during render.
*
* The producer's frame-injection pipeline hides each `<video>` and paints from
* a sibling `<img id="__render_frame_<renderId>__">`. That id is derived from
* the element's *render* id, not its author id: author ids are only unique
* within one composition file, and the render document is the inlined union of
* many, so two scenes can carry the same `<video id="clip">`. Deriving from
* `el.id` there resolves every collider to the first one's frame the reader
* silently reads another clip's pixels.
*
* This module owns that derivation for the runtime. The engine mirrors the same
* rule in-page (`mediaRenderIdBridge`), because code shipped into
* `page.evaluate` cannot import; `renderFrameElementId` is the definition both
* sides follow, and its format is pinned by test.
*/
import { MEDIA_RENDER_ID_ATTR } from "../compiler/mediaRenderIds.js";
/**
* The render id an element is addressed by: its stamped, document-unique id
* when the producer compiled this document, else its author id. The two are
* the same string whenever no collision forced a suffix.
*/
export function readMediaRenderId(media: Element): string | null {
return media.getAttribute(MEDIA_RENDER_ID_ATTR) || media.id || null;
}
/**
* Affixes of the sibling id. Exported so the engine can build the same id
* inside `page.evaluate`, where it cannot import: it passes these through as
* evaluate arguments rather than repeating the literals. Keeping them here is
* what makes this module the one definition of the format.
*/
export const RENDER_FRAME_ID_PREFIX = "__render_frame_";
export const RENDER_FRAME_ID_SUFFIX = "__";
/** The id of the render-frame `<img>` paired with a given render id. */
export function renderFrameIdForRenderId(renderId: string): string {
return `${RENDER_FRAME_ID_PREFIX}${renderId}${RENDER_FRAME_ID_SUFFIX}`;
}
/** The id of the render-frame `<img>` paired with a media element. */
export function renderFrameElementId(media: Element): string | null {
const renderId = readMediaRenderId(media);
return renderId ? renderFrameIdForRenderId(renderId) : null;
}
/**
* The injected render-frame `<img>` for a media element, or null in preview
* (where the producer never creates one).
*/
export function findInjectedRenderFrame(media: Element): HTMLImageElement | null {
const frameId = renderFrameElementId(media);
if (!frameId) return null;
const frame = document.getElementById(frameId);
return frame instanceof HTMLImageElement ? frame : null;
}
+9 -2
View File
@@ -42,6 +42,7 @@ import {
} from "@hyperframes/core/audio-automation"; } from "@hyperframes/core/audio-automation";
import { chainTailSeconds } from "@hyperframes/core/audio-fx-tail"; import { chainTailSeconds } from "@hyperframes/core/audio-fx-tail";
import { import {
MEDIA_RENDER_ID_ATTR,
normalizePlaybackRate, normalizePlaybackRate,
parseStrictFiniteTimingNumber, parseStrictFiniteTimingNumber,
readMediaStart, readMediaStart,
@@ -486,15 +487,21 @@ export function parseAudioElements(html: string): AudioElement[] {
}; };
}; };
// A compiled render document stamps a document-unique render id; prefer it,
// because element ids are only unique within one composition file and the
// render document inlines many. See core's mediaRenderIds.ts.
const trackId = (el: RefResolverEl): string | null =>
el.getAttribute(MEDIA_RENDER_ID_ATTR) || el.getAttribute("id");
for (const el of document.querySelectorAll("audio[id][src]")) { for (const el of document.querySelectorAll("audio[id][src]")) {
const id = el.getAttribute("id"); const id = trackId(el);
if (!id || !el.getAttribute("src") || isHidden(el)) continue; if (!id || !el.getAttribute("src") || isHidden(el)) continue;
if (isKnownInactiveTimelineWindow(el, resolveStart(el))) continue; if (isKnownInactiveTimelineWindow(el, resolveStart(el))) continue;
elements.push(build(el, id, "audio")); elements.push(build(el, id, "audio"));
} }
for (const el of document.querySelectorAll('video[id][src][data-has-audio="true"]')) { for (const el of document.querySelectorAll('video[id][src][data-has-audio="true"]')) {
const id = el.getAttribute("id"); const id = trackId(el);
if (!id || !el.getAttribute("src") || isHidden(el)) continue; if (!id || !el.getAttribute("src") || isHidden(el)) continue;
if (isKnownInactiveTimelineWindow(el, resolveStart(el))) continue; if (isKnownInactiveTimelineWindow(el, resolveStart(el))) continue;
elements.push(build(el, `${id}-audio`, "video")); elements.push(build(el, `${id}-audio`, "video"));
+6 -1
View File
@@ -57,6 +57,7 @@ import type {
SubTimelineWaitOutcome, SubTimelineWaitOutcome,
} from "../types.js"; } from "../types.js";
import { cloneCaptureWarnings } from "./captureWarning.js"; import { cloneCaptureWarnings } from "./captureWarning.js";
import { installMediaRenderIdBridge } from "./mediaRenderIdBridge.js";
export { isMemoryExhaustionError, isTransientBrowserError } from "./captureFailure.js"; export { isMemoryExhaustionError, isTransientBrowserError } from "./captureFailure.js";
export type { CaptureOptions, CaptureResult, CaptureBufferResult, CapturePerfSummary }; export type { CaptureOptions, CaptureResult, CaptureBufferResult, CapturePerfSummary };
@@ -1296,6 +1297,9 @@ async function constructCaptureSession(
w.__name = <T>(fn: T, _name: string): T => fn; w.__name = <T>(fn: T, _name: string): T => fn;
} }
}); });
// Media elements are addressed by a document-unique render id, not by the
// per-file element id. Install the resolvers before any page script runs.
await installMediaRenderIdBridge(page);
// Fast capture: record accelerated canvases (webgl/webgl2/webgpu) and force // Fast capture: record accelerated canvases (webgl/webgl2/webgpu) and force
// preserveDrawingBuffer before any page script can create a context — their // preserveDrawingBuffer before any page script can create a context — their
// paint records freeze at the first frame, so captureDrawElementFrame // paint records freeze at the first frame, so captureDrawElementFrame
@@ -1964,7 +1968,8 @@ async function applyVideoMetadataHints(
continue; continue;
} }
const video = document.getElementById(hint.id) as HTMLVideoElement | null; const video = (window.__hfMediaEl?.(hint.id) ??
document.getElementById(hint.id)) as HTMLVideoElement | null;
if (!video) continue; if (!video) continue;
if (!video.hasAttribute("width")) video.setAttribute("width", String(hint.width)); if (!video.hasAttribute("width")) video.setAttribute("width", String(hint.width));
@@ -0,0 +1,48 @@
/**
* In-page resolution between a render media id and its DOM element.
*
* The pipeline addresses media by id, but element ids are only unique within
* one composition FILE and the render document inlines many, so
* `document.getElementById` silently resolves duplicates to whichever element
* comes first. The producer stamps a document-unique `data-hf-render-id`
* (core's mediaRenderIds.ts); these helpers are the single place that knows to
* prefer it, so every capture stage addresses the same element.
*
* Installed via `evaluateOnNewDocument` so it exists before any page script.
* Every call site keeps a `getElementById` fallback for documents that were
* never compiled by the producer (snapshot, check, and direct engine callers),
* where the authored id already is the identity.
*
* `__hfMediaId` mirrors core's `readMediaRenderId`, which is the definition of
* this rule; the copy exists only because code serialized into `page.evaluate`
* cannot import. The runtime readers in core call that function directly, and
* `renderFrameSibling.test.ts` pins the sibling-id format both sides build.
*/
import { MEDIA_RENDER_ID_ATTR } from "@hyperframes/core";
import type { Page } from "puppeteer-core";
declare global {
interface Window {
/** Element for a render media id, or null. */
__hfMediaEl?: (id: string) => Element | null;
/** Render media id for an element — its stamped id, else its plain id. */
__hfMediaId?: (el: Element) => string;
}
}
/**
* Install the resolvers. Runs on every document this page loads, so a
* mid-render navigation cannot leave a capture stage without them.
*/
export async function installMediaRenderIdBridge(page: Page): Promise<void> {
await page.evaluateOnNewDocument((attr: string) => {
window.__hfMediaEl = (id: string): Element | null => {
// CSS.escape covers ids with characters that are not valid in a selector
// literal; the attribute value still needs its own quote escaping.
const escaped = id.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
return document.querySelector(`[${attr}="${escaped}"]`) ?? document.getElementById(id);
};
window.__hfMediaId = (el: Element): string => el.getAttribute(attr) || el.id;
}, MEDIA_RENDER_ID_ATTR);
}
@@ -11,7 +11,11 @@ import { type CaptureOptions } from "../types.js";
import { COLOR_GRADING_SOURCE_HIDDEN_ATTR } from "@hyperframes/core/color-grading"; import { COLOR_GRADING_SOURCE_HIDDEN_ATTR } from "@hyperframes/core/color-grading";
import { import {
HF_COLOR_GRADING_CANVAS_ID_PREFIX, HF_COLOR_GRADING_CANVAS_ID_PREFIX,
MEDIA_RENDER_ID_ATTR,
MEDIA_VISUAL_STYLE_PROPERTIES, MEDIA_VISUAL_STYLE_PROPERTIES,
RENDER_FRAME_ID_PREFIX,
RENDER_FRAME_ID_SUFFIX,
renderFrameIdForRenderId,
} from "@hyperframes/core"; } from "@hyperframes/core";
export const cdpSessionCache = new WeakMap<Page, import("puppeteer-core").CDPSession>(); export const cdpSessionCache = new WeakMap<Page, import("puppeteer-core").CDPSession>();
@@ -404,6 +408,9 @@ export async function applyDomLayerMask(
(args: { (args: {
show: string[]; show: string[];
hide: string[]; hide: string[];
renderIdAttr: string;
renderFramePrefix: string;
renderFrameSuffix: string;
styleId: string; styleId: string;
hiddenAttr: string; hiddenAttr: string;
prevVisibilityAttr: string; prevVisibilityAttr: string;
@@ -413,6 +420,11 @@ export async function applyDomLayerMask(
const existing = document.getElementById(args.styleId); const existing = document.getElementById(args.styleId);
if (existing) existing.remove(); if (existing) existing.remove();
// Affixes come from core's renderFrameSibling, so the runtime readers and
// this lookup cannot drift apart on the id format.
const renderFrameId = (id: string) =>
`${args.renderFramePrefix}${id}${args.renderFrameSuffix}`;
const restoreMaskedElements = () => { const restoreMaskedElements = () => {
const masked = document.querySelectorAll(`[${args.hiddenAttr}="1"]`); const masked = document.querySelectorAll(`[${args.hiddenAttr}="1"]`);
for (const node of masked) { for (const node of masked) {
@@ -465,11 +477,21 @@ export async function applyDomLayerMask(
const showSelectors: string[] = []; const showSelectors: string[] = [];
for (const id of args.show) { for (const id of args.show) {
const el = document.getElementById(id); const el = window.__hfMediaEl?.(id) ?? document.getElementById(id);
if (el) rememberHiddenTimedDescendants(el); if (el) rememberHiddenTimedDescendants(el);
const escaped = CSS.escape(id); // Address the element by its render id when it has one. `#id` must not
showSelectors.push(`#${escaped}`, `#${escaped} *`); // be used as an extra fallback here: an id is duplicated exactly when
const renderEscaped = CSS.escape(`__render_frame_${id}__`); // two compositions share it, so `#id` would also unhide the other
// scene's element — the collision this render id exists to resolve.
if (el?.hasAttribute(args.renderIdAttr)) {
const attrEscaped = id.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
const byRenderId = `[${args.renderIdAttr}="${attrEscaped}"]`;
showSelectors.push(byRenderId, `${byRenderId} *`);
} else {
const escaped = CSS.escape(id);
showSelectors.push(`#${escaped}`, `#${escaped} *`);
}
const renderEscaped = CSS.escape(renderFrameId(id));
showSelectors.push(`#${renderEscaped}`, `#${renderEscaped} *`); showSelectors.push(`#${renderEscaped}`, `#${renderEscaped} *`);
const colorGradingEscaped = CSS.escape(`${args.canvasIdPrefix}${id}`); const colorGradingEscaped = CSS.escape(`${args.canvasIdPrefix}${id}`);
showSelectors.push(`#${colorGradingEscaped}`, `#${colorGradingEscaped} *`); showSelectors.push(`#${colorGradingEscaped}`, `#${colorGradingEscaped} *`);
@@ -491,11 +513,11 @@ export async function applyDomLayerMask(
} }
for (const id of args.hide) { for (const id of args.hide) {
const el = document.getElementById(id); const el = window.__hfMediaEl?.(id) ?? document.getElementById(id);
if (el) { if (el instanceof HTMLElement) {
rememberAndHideElement(el); rememberAndHideElement(el);
} }
const img = document.getElementById(`__render_frame_${id}__`); const img = document.getElementById(renderFrameId(id));
if (img) { if (img) {
rememberAndHideElement(img); rememberAndHideElement(img);
} }
@@ -508,6 +530,9 @@ export async function applyDomLayerMask(
{ {
show: showIds, show: showIds,
hide: extraHideIds, hide: extraHideIds,
renderIdAttr: MEDIA_RENDER_ID_ATTR,
renderFramePrefix: RENDER_FRAME_ID_PREFIX,
renderFrameSuffix: RENDER_FRAME_ID_SUFFIX,
styleId: DOM_LAYER_MASK_STYLE_ID, styleId: DOM_LAYER_MASK_STYLE_ID,
hiddenAttr: DOM_LAYER_MASK_HIDDEN_ATTR, hiddenAttr: DOM_LAYER_MASK_HIDDEN_ATTR,
prevVisibilityAttr: DOM_LAYER_MASK_PREV_VISIBILITY_ATTR, prevVisibilityAttr: DOM_LAYER_MASK_PREV_VISIBILITY_ATTR,
@@ -589,21 +614,29 @@ export async function removeDomLayerMask(page: Page, _extraHideIds: string[]): P
* callers that don't run through `initializeSession`. * callers that don't run through `initializeSession`.
*/ */
export async function ensureRenderFrameSiblings(page: Page): Promise<void> { export async function ensureRenderFrameSiblings(page: Page): Promise<void> {
await page.evaluate(() => { await page.evaluate(
for (const video of Array.from( (prefix: string, suffix: string) => {
document.querySelectorAll<HTMLVideoElement>("video[data-start]"), for (const video of Array.from(
)) { document.querySelectorAll<HTMLVideoElement>("video[data-start]"),
const next = video.nextElementSibling; )) {
if (next !== null && next.classList.contains("__render_frame__")) continue; const next = video.nextElementSibling;
const img = document.createElement("img"); if (next !== null && next.classList.contains("__render_frame__")) continue;
img.classList.add("__render_frame__"); const img = document.createElement("img");
img.id = `__render_frame_${video.id}__`; img.classList.add("__render_frame__");
img.style.pointerEvents = "none"; // Derive from the render id, not `video.id` — two scenes can share an
img.style.position = "absolute"; // element id, and two siblings sharing an id would collide in turn.
img.style.visibility = "hidden"; // `||`, not `??`: `__hfMediaId` returns "" for an element with neither
video.parentNode?.insertBefore(img, video.nextSibling); // id, and core's reader treats that as "no id" rather than a key.
} img.id = `${prefix}${window.__hfMediaId?.(video) || video.id}${suffix}`;
}); img.style.pointerEvents = "none";
img.style.position = "absolute";
img.style.visibility = "hidden";
video.parentNode?.insertBefore(img, video.nextSibling);
}
},
RENDER_FRAME_ID_PREFIX,
RENDER_FRAME_ID_SUFFIX,
);
} }
/** /**
@@ -622,7 +655,7 @@ export async function injectVideoFramesBatch(
return await page.evaluate( return await page.evaluate(
// fallow-ignore-next-line complexity // fallow-ignore-next-line complexity
async ( async (
items: Array<{ videoId: string; dataUri: string }>, items: Array<{ videoId: string; dataUri: string; frameId: string }>,
visualProperties: string[], visualProperties: string[],
colorGradingSourceHiddenAttr: string, colorGradingSourceHiddenAttr: string,
) => { ) => {
@@ -677,7 +710,8 @@ export async function injectVideoFramesBatch(
return false; return false;
}; };
for (const item of items) { for (const item of items) {
const video = document.getElementById(item.videoId) as HTMLVideoElement | null; const video = (window.__hfMediaEl?.(item.videoId) ??
document.getElementById(item.videoId)) as HTMLVideoElement | null;
if (!video) continue; if (!video) continue;
let img = video.nextElementSibling as HTMLImageElement | null; let img = video.nextElementSibling as HTMLImageElement | null;
@@ -713,7 +747,7 @@ export async function injectVideoFramesBatch(
if (isNewImage) { if (isNewImage) {
img = document.createElement("img"); img = document.createElement("img");
img.classList.add("__render_frame__"); img.classList.add("__render_frame__");
img.id = `__render_frame_${item.videoId}__`; img.id = item.frameId;
img.style.pointerEvents = "none"; img.style.pointerEvents = "none";
video.parentNode?.insertBefore(img, video.nextSibling); video.parentNode?.insertBefore(img, video.nextSibling);
} }
@@ -796,7 +830,9 @@ export async function injectVideoFramesBatch(
} }
return injectedIds; return injectedIds;
}, },
updates, // Build the sibling id with core's function rather than a template here,
// so the id the readers look up has exactly one definition.
updates.map((update) => ({ ...update, frameId: renderFrameIdForRenderId(update.videoId) })),
[...MEDIA_VISUAL_STYLE_PROPERTIES], [...MEDIA_VISUAL_STYLE_PROPERTIES],
COLOR_GRADING_SOURCE_HIDDEN_ATTR, COLOR_GRADING_SOURCE_HIDDEN_ATTR,
); );
@@ -10,6 +10,7 @@ import { copyFileSync, existsSync, linkSync, mkdirSync, readdirSync, rmSync } fr
import { isAbsolute, join, posix, resolve, sep } from "path"; import { isAbsolute, join, posix, resolve, sep } from "path";
import { parseHTML } from "linkedom"; import { parseHTML } from "linkedom";
import { import {
MEDIA_RENDER_ID_ATTR,
decodeUrlPathVariants, decodeUrlPathVariants,
fpsToFfmpegArg, fpsToFfmpegArg,
fpsToNumber, fpsToNumber,
@@ -539,7 +540,13 @@ export function parseVideoElements(html: string): VideoElement[] {
if (!src) continue; if (!src) continue;
// Generate a stable ID for videos without one — the producer needs IDs // Generate a stable ID for videos without one — the producer needs IDs
// to track extracted frames and composite them during encoding. // to track extracted frames and composite them during encoding.
const id = el.getAttribute("id") || `hf-video-${autoIdCounter++}`; // A compiled render document stamps a document-unique render id; prefer it,
// because element ids are only unique within one composition file and the
// render document inlines many.
const id =
el.getAttribute(MEDIA_RENDER_ID_ATTR) ||
el.getAttribute("id") ||
`hf-video-${autoIdCounter++}`;
if (!el.getAttribute("id")) { if (!el.getAttribute("id")) {
el.setAttribute("id", id); el.setAttribute("id", id);
} }
@@ -609,7 +616,9 @@ export function parseImageElements(html: string): ImageElement[] {
const src = el.getAttribute("src"); const src = el.getAttribute("src");
if (!src) continue; if (!src) continue;
const id = el.getAttribute("id") || `hf-img-${autoIdCounter++}`; // See parseVideoElements: the stamped render id wins over the authored id.
const id =
el.getAttribute(MEDIA_RENDER_ID_ATTR) || el.getAttribute("id") || `hf-img-${autoIdCounter++}`;
if (!el.getAttribute("id")) { if (!el.getAttribute("id")) {
el.setAttribute("id", id); el.setAttribute("id", id);
} }
+297 -277
View File
@@ -13,7 +13,12 @@ import { type FrameLookupTable } from "./videoFrameExtractor.js";
import { injectVideoFramesBatch, syncVideoFrameVisibility } from "./screenshotService.js"; import { injectVideoFramesBatch, syncVideoFrameVisibility } from "./screenshotService.js";
import { type BeforeCaptureHook } from "./frameCapture.js"; import { type BeforeCaptureHook } from "./frameCapture.js";
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js"; import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
import { HF_COLOR_GRADING_CANVAS_ID_PREFIX } from "@hyperframes/core"; import {
HF_COLOR_GRADING_CANVAS_ID_PREFIX,
RENDER_FRAME_ID_PREFIX,
RENDER_FRAME_ID_SUFFIX,
renderFrameIdForRenderId,
} from "@hyperframes/core";
export interface VideoFrameInjectorOptions extends Partial< export interface VideoFrameInjectorOptions extends Partial<
Pick<EngineConfig, "frameDataUriCacheLimit" | "frameDataUriCacheBytesLimitMb"> Pick<EngineConfig, "frameDataUriCacheLimit" | "frameDataUriCacheBytesLimitMb">
@@ -274,7 +279,11 @@ async function setVideoElementsVisibility(
): Promise<void> { ): Promise<void> {
if (videoIds.length === 0) return; if (videoIds.length === 0) return;
await page.evaluate( await page.evaluate(
(ids: string[], canvasIdPrefix: string, shouldShow: boolean) => { (
entries: Array<{ id: string; frameId: string }>,
canvasIdPrefix: string,
shouldShow: boolean,
) => {
const apply = (node: Element | null) => { const apply = (node: Element | null) => {
if (!(node instanceof HTMLElement)) return; if (!(node instanceof HTMLElement)) return;
if (shouldShow) { if (shouldShow) {
@@ -283,15 +292,17 @@ async function setVideoElementsVisibility(
node.style.setProperty("visibility", "hidden", "important"); node.style.setProperty("visibility", "hidden", "important");
} }
}; };
for (const id of ids) { for (const { id, frameId } of entries) {
const video = document.getElementById(id); const video = window.__hfMediaEl?.(id) ?? document.getElementById(id);
if (!video) continue; if (!video) continue;
apply(video); apply(video);
apply(document.getElementById(`__render_frame_${id}__`)); apply(document.getElementById(frameId));
apply(document.getElementById(`${canvasIdPrefix}${id}`)); apply(document.getElementById(`${canvasIdPrefix}${id}`));
} }
}, },
videoIds, // Sibling ids come from core's function, never a template here, so the
// format has one definition across the engine and the runtime readers.
videoIds.map((id) => ({ id, frameId: renderFrameIdForRenderId(id) })),
HF_COLOR_GRADING_CANVAS_ID_PREFIX, HF_COLOR_GRADING_CANVAS_ID_PREFIX,
visible, visible,
); );
@@ -307,8 +318,10 @@ export async function queryVideoElementBounds(
): Promise<VideoElementBounds[]> { ): Promise<VideoElementBounds[]> {
if (videoIds.length === 0) return []; if (videoIds.length === 0) return [];
return page.evaluate((ids: string[]): VideoElementBounds[] => { return page.evaluate((ids: string[]): VideoElementBounds[] => {
const resolveVideo = (id: string): HTMLVideoElement | null =>
(window.__hfMediaEl?.(id) ?? document.getElementById(id)) as HTMLVideoElement | null;
return ids.map((id) => { return ids.map((id) => {
const el = document.getElementById(id) as HTMLVideoElement | null; const el = resolveVideo(id);
if (!el) { if (!el) {
return { return {
videoId: id, videoId: id,
@@ -409,293 +422,300 @@ export async function queryElementStacking(
nativeHdrIds: Set<string>, nativeHdrIds: Set<string>,
): Promise<ElementStackingInfo[]> { ): Promise<ElementStackingInfo[]> {
const hdrIds = Array.from(nativeHdrIds); const hdrIds = Array.from(nativeHdrIds);
// fallow-ignore-next-line complexity return page.evaluate(
return page.evaluate((hdrIdList: string[]): ElementStackingInfo[] => {
const hdrSet = new Set(hdrIdList);
const elements = document.querySelectorAll("[data-start]");
const results: ElementStackingInfo[] = [];
// Walk up the DOM to find the effective z-index from the nearest
// positioned ancestor with a z-index. CSS z-index only applies to
// positioned elements; video elements inside positioned wrappers
// inherit the wrapper's stacking context.
//
// ## Supported subset
//
// This implementation looks for explicit `z-index` on positioned
// (non-static) ancestors. It does NOT detect the CSS stacking contexts
// created implicitly by other properties — including `opacity < 1`,
// `transform`, `filter`, `will-change`, `isolation: isolate`, and
// `mix-blend-mode`. GSAP routinely sets `transform` on wrappers, which
// creates an implicit stacking context with auto z-index; an HDR video
// inside such a wrapper with no explicit z-index will return the
// wrapper-of-the-wrapper's z-index here, potentially reordering layers
// incorrectly relative to sibling stacking contexts.
//
// The workaround is to set explicit `z-index` on the positioned wrapper
// when you want it treated as a compositing layer root. This matches
// what compositions need to do anyway for deterministic z-ordering.
function getEffectiveZIndex(node: Element): number {
let current: Element | null = node;
while (current) {
const cs = window.getComputedStyle(current);
const pos = cs.position;
const z = parseInt(cs.zIndex);
if (!Number.isNaN(z) && pos !== "static") return z;
current = current.parentElement;
}
return 0;
}
// Find border-radius that clips the element. Replaced elements like <video>
// clip to their own border-radius; ancestors need overflow !== visible.
// fallow-ignore-next-line complexity // fallow-ignore-next-line complexity
function getEffectiveBorderRadius(node: Element): [number, number, number, number] { (hdrIdList: string[], prefix: string, suffix: string): ElementStackingInfo[] => {
// Resolve a CSS border-radius value to pixels. Chrome's getComputedStyle const hdrSet = new Set(hdrIdList);
// returns percentages as-is (e.g. "50%"), not resolved to px. const elements = document.querySelectorAll("[data-start]");
// Uses offsetWidth/offsetHeight (layout dimensions before CSS transforms) const results: ElementStackingInfo[] = [];
// because CSS resolves percentages against the padding box, not the
// transformed bounding box. // Walk up the DOM to find the effective z-index from the nearest
function resolveRadius(value: string, el: Element): number { // positioned ancestor with a z-index. CSS z-index only applies to
if (value.includes("%")) { // positioned elements; video elements inside positioned wrappers
const pct = parseFloat(value) / 100; // inherit the wrapper's stacking context.
const w = el instanceof HTMLElement ? el.offsetWidth : 0; //
const h = el instanceof HTMLElement ? el.offsetHeight : 0; // ## Supported subset
return pct * Math.min(w, h); //
// This implementation looks for explicit `z-index` on positioned
// (non-static) ancestors. It does NOT detect the CSS stacking contexts
// created implicitly by other properties — including `opacity < 1`,
// `transform`, `filter`, `will-change`, `isolation: isolate`, and
// `mix-blend-mode`. GSAP routinely sets `transform` on wrappers, which
// creates an implicit stacking context with auto z-index; an HDR video
// inside such a wrapper with no explicit z-index will return the
// wrapper-of-the-wrapper's z-index here, potentially reordering layers
// incorrectly relative to sibling stacking contexts.
//
// The workaround is to set explicit `z-index` on the positioned wrapper
// when you want it treated as a compositing layer root. This matches
// what compositions need to do anyway for deterministic z-ordering.
function getEffectiveZIndex(node: Element): number {
let current: Element | null = node;
while (current) {
const cs = window.getComputedStyle(current);
const pos = cs.position;
const z = parseInt(cs.zIndex);
if (!Number.isNaN(z) && pos !== "static") return z;
current = current.parentElement;
} }
const parsed = parseFloat(value); return 0;
return Number.isNaN(parsed) ? 0 : parsed;
} }
// Check element itself (replaced elements clip to own border-radius) // Find border-radius that clips the element. Replaced elements like <video>
const selfCs = window.getComputedStyle(node); // clip to their own border-radius; ancestors need overflow !== visible.
const selfRadii: [number, number, number, number] = [ // fallow-ignore-next-line complexity
resolveRadius(selfCs.borderTopLeftRadius, node), function getEffectiveBorderRadius(node: Element): [number, number, number, number] {
resolveRadius(selfCs.borderTopRightRadius, node), // Resolve a CSS border-radius value to pixels. Chrome's getComputedStyle
resolveRadius(selfCs.borderBottomRightRadius, node), // returns percentages as-is (e.g. "50%"), not resolved to px.
resolveRadius(selfCs.borderBottomLeftRadius, node), // Uses offsetWidth/offsetHeight (layout dimensions before CSS transforms)
]; // because CSS resolves percentages against the padding box, not the
if (selfRadii[0] > 0 || selfRadii[1] > 0 || selfRadii[2] > 0 || selfRadii[3] > 0) { // transformed bounding box.
return selfRadii; function resolveRadius(value: string, el: Element): number {
} if (value.includes("%")) {
const pct = parseFloat(value) / 100;
// Walk ancestors looking for clipping container const w = el instanceof HTMLElement ? el.offsetWidth : 0;
let current: Element | null = node.parentElement; const h = el instanceof HTMLElement ? el.offsetHeight : 0;
while (current) { return pct * Math.min(w, h);
const cs = window.getComputedStyle(current);
if (cs.overflow !== "visible") {
const tl = resolveRadius(cs.borderTopLeftRadius, current);
const tr = resolveRadius(cs.borderTopRightRadius, current);
const brr = resolveRadius(cs.borderBottomRightRadius, current);
const bl = resolveRadius(cs.borderBottomLeftRadius, current);
if (tl > 0 || tr > 0 || brr > 0 || bl > 0) {
return [tl, tr, brr, bl];
} }
const parsed = parseFloat(value);
return Number.isNaN(parsed) ? 0 : parsed;
} }
current = current.parentElement;
}
return [0, 0, 0, 0];
}
// Walk ancestors to find the tightest overflow:hidden clip rect. // Check element itself (replaced elements clip to own border-radius)
// Returns null if no clipping ancestor exists. const selfCs = window.getComputedStyle(node);
function getClipRect( const selfRadii: [number, number, number, number] = [
node: Element, resolveRadius(selfCs.borderTopLeftRadius, node),
): { x: number; y: number; width: number; height: number } | null { resolveRadius(selfCs.borderTopRightRadius, node),
let current: Element | null = node.parentElement; resolveRadius(selfCs.borderBottomRightRadius, node),
let clip: { x: number; y: number; width: number; height: number } | null = null; resolveRadius(selfCs.borderBottomLeftRadius, node),
while (current) { ];
const cs = window.getComputedStyle(current); if (selfRadii[0] > 0 || selfRadii[1] > 0 || selfRadii[2] > 0 || selfRadii[3] > 0) {
if (cs.overflow === "hidden" || cs.overflow === "clip") { return selfRadii;
const r = current.getBoundingClientRect();
const ancestor = {
x: Math.round(r.x),
y: Math.round(r.y),
width: Math.round(r.width),
height: Math.round(r.height),
};
if (!clip) {
clip = ancestor;
} else {
// Intersect with existing clip
const x1 = Math.max(clip.x, ancestor.x);
const y1 = Math.max(clip.y, ancestor.y);
const x2 = Math.min(clip.x + clip.width, ancestor.x + ancestor.width);
const y2 = Math.min(clip.y + clip.height, ancestor.y + ancestor.height);
clip = { x: x1, y: y1, width: Math.max(0, x2 - x1), height: Math.max(0, y2 - y1) };
}
} }
current = current.parentElement;
}
return clip;
}
// Walk up the DOM multiplying each ancestor's opacity. GSAP animates // Walk ancestors looking for clipping container
// opacity on wrapper divs, not directly on the video element, so the let current: Element | null = node.parentElement;
// element's own opacity is often 1.0. Multiplying ancestors gives the while (current) {
// true effective opacity. const cs = window.getComputedStyle(current);
function getEffectiveOpacity(node: Element): number { if (cs.overflow !== "visible") {
let opacity = 1; const tl = resolveRadius(cs.borderTopLeftRadius, current);
let current: Element | null = node; const tr = resolveRadius(cs.borderTopRightRadius, current);
while (current) { const brr = resolveRadius(cs.borderBottomRightRadius, current);
const cs = window.getComputedStyle(current); const bl = resolveRadius(cs.borderBottomLeftRadius, current);
const val = parseFloat(cs.opacity); if (tl > 0 || tr > 0 || brr > 0 || bl > 0) {
// Note: `val || 1` would turn opacity:0 into 1 (0 is falsy) return [tl, tr, brr, bl];
opacity *= Number.isNaN(val) ? 1 : val;
current = current.parentElement;
}
return opacity;
}
// Compute the full CSS transform matrix from element-local coords to
// viewport coords by walking the offsetParent chain and accumulating
// position offsets + CSS transforms. This correctly handles GSAP
// animations on wrapper divs (rotation, scale) that getBoundingClientRect
// conflates into an axis-aligned bounding box.
// fallow-ignore-next-line complexity
function getViewportMatrix(node: Element): string {
const chain: HTMLElement[] = [];
let current: Element | null = node;
while (current instanceof HTMLElement) {
chain.push(current);
const next: Element | null =
(current.offsetParent as Element | null) ?? current.parentElement;
if (next === current) break;
current = next;
}
let mat = new DOMMatrix();
for (let i = chain.length - 1; i >= 0; i--) {
const htmlEl = chain[i];
if (!htmlEl) continue;
mat = mat.translate(htmlEl.offsetLeft, htmlEl.offsetTop);
const cs = window.getComputedStyle(htmlEl);
const origin = cs.transformOrigin.split(" ");
const ox = resolveLength(origin[0] ?? "0", htmlEl.offsetWidth);
const oy = resolveLength(origin[1] ?? "0", htmlEl.offsetHeight);
const individualTransform = composeIndividualTransforms(cs);
const hasIndividual = individualTransform !== null;
const hasTransform = cs.transform && cs.transform !== "none";
if (hasIndividual || hasTransform) {
mat = mat.translate(ox, oy);
if (hasIndividual) mat = mat.multiply(individualTransform);
if (hasTransform) {
try {
const t = new DOMMatrix(cs.transform);
if (
Number.isFinite(t.a) &&
Number.isFinite(t.b) &&
Number.isFinite(t.c) &&
Number.isFinite(t.d) &&
Number.isFinite(t.e) &&
Number.isFinite(t.f)
) {
mat = mat.multiply(t);
}
} catch {
// DOMMatrix constructor throws on malformed input — skip.
} }
} }
mat = mat.translate(-ox, -oy); current = current.parentElement;
} }
return [0, 0, 0, 0];
} }
return mat.toString();
}
// fallow-ignore-next-line complexity // Walk ancestors to find the tightest overflow:hidden clip rect.
function composeIndividualTransforms(cs: CSSStyleDeclaration): DOMMatrix | null { // Returns null if no clipping ancestor exists.
const translate = cs.getPropertyValue("translate").trim(); function getClipRect(
const rotate = cs.getPropertyValue("rotate").trim(); node: Element,
const scale = cs.getPropertyValue("scale").trim(); ): { x: number; y: number; width: number; height: number } | null {
const hasTranslate = translate && translate !== "none"; let current: Element | null = node.parentElement;
const hasRotate = rotate && rotate !== "none"; let clip: { x: number; y: number; width: number; height: number } | null = null;
const hasScale = scale && scale !== "none"; while (current) {
if (!hasTranslate && !hasRotate && !hasScale) return null; const cs = window.getComputedStyle(current);
let m = new DOMMatrix(); if (cs.overflow === "hidden" || cs.overflow === "clip") {
if (hasTranslate) { const r = current.getBoundingClientRect();
const parts = translate.split(/\s+/); const ancestor = {
const tx = parseFloat(parts[0] ?? "0") || 0; x: Math.round(r.x),
const ty = parseFloat(parts[1] ?? "0") || 0; y: Math.round(r.y),
if (tx !== 0 || ty !== 0) m = m.translate(tx, ty); width: Math.round(r.width),
height: Math.round(r.height),
};
if (!clip) {
clip = ancestor;
} else {
// Intersect with existing clip
const x1 = Math.max(clip.x, ancestor.x);
const y1 = Math.max(clip.y, ancestor.y);
const x2 = Math.min(clip.x + clip.width, ancestor.x + ancestor.width);
const y2 = Math.min(clip.y + clip.height, ancestor.y + ancestor.height);
clip = { x: x1, y: y1, width: Math.max(0, x2 - x1), height: Math.max(0, y2 - y1) };
}
}
current = current.parentElement;
}
return clip;
} }
if (hasRotate) {
const deg = parseFloat(rotate) || 0;
if (deg !== 0) m = m.rotate(deg);
}
if (hasScale) {
const parts = scale.split(/\s+/);
const sx = parseFloat(parts[0] ?? "1") || 1;
const sy = parseFloat(parts[1] ?? String(sx)) || sx;
if (sx !== 1 || sy !== 1) m = m.scale(sx, sy);
}
return m;
}
function resolveLength(value: string, basis: number): number { // Walk up the DOM multiplying each ancestor's opacity. GSAP animates
if (value.endsWith("%")) { // opacity on wrapper divs, not directly on the video element, so the
const pct = parseFloat(value) / 100; // element's own opacity is often 1.0. Multiplying ancestors gives the
return Number.isFinite(pct) ? pct * basis : 0; // true effective opacity.
function getEffectiveOpacity(node: Element): number {
let opacity = 1;
let current: Element | null = node;
while (current) {
const cs = window.getComputedStyle(current);
const val = parseFloat(cs.opacity);
// Note: `val || 1` would turn opacity:0 into 1 (0 is falsy)
opacity *= Number.isNaN(val) ? 1 : val;
current = current.parentElement;
}
return opacity;
} }
const n = parseFloat(value);
return Number.isFinite(n) ? n : 0;
}
function isElementPaintable(node: Element): boolean { // Compute the full CSS transform matrix from element-local coords to
const rect = node.getBoundingClientRect(); // viewport coords by walking the offsetParent chain and accumulating
const style = window.getComputedStyle(node); // position offsets + CSS transforms. This correctly handles GSAP
return ( // animations on wrapper divs (rotation, scale) that getBoundingClientRect
style.visibility !== "hidden" && // conflates into an axis-aligned bounding box.
style.display !== "none" && // fallow-ignore-next-line complexity
rect.width > 0 && function getViewportMatrix(node: Element): string {
rect.height > 0 const chain: HTMLElement[] = [];
); let current: Element | null = node;
} while (current instanceof HTMLElement) {
chain.push(current);
const next: Element | null =
(current.offsetParent as Element | null) ?? current.parentElement;
if (next === current) break;
current = next;
}
let mat = new DOMMatrix();
for (let i = chain.length - 1; i >= 0; i--) {
const htmlEl = chain[i];
if (!htmlEl) continue;
mat = mat.translate(htmlEl.offsetLeft, htmlEl.offsetTop);
const cs = window.getComputedStyle(htmlEl);
const origin = cs.transformOrigin.split(" ");
const ox = resolveLength(origin[0] ?? "0", htmlEl.offsetWidth);
const oy = resolveLength(origin[1] ?? "0", htmlEl.offsetHeight);
const individualTransform = composeIndividualTransforms(cs);
const hasIndividual = individualTransform !== null;
const hasTransform = cs.transform && cs.transform !== "none";
if (hasIndividual || hasTransform) {
mat = mat.translate(ox, oy);
if (hasIndividual) mat = mat.multiply(individualTransform);
if (hasTransform) {
try {
const t = new DOMMatrix(cs.transform);
if (
Number.isFinite(t.a) &&
Number.isFinite(t.b) &&
Number.isFinite(t.c) &&
Number.isFinite(t.d) &&
Number.isFinite(t.e) &&
Number.isFinite(t.f)
) {
mat = mat.multiply(t);
}
} catch {
// DOMMatrix constructor throws on malformed input — skip.
}
}
mat = mat.translate(-ox, -oy);
}
}
return mat.toString();
}
for (const el of elements) { // fallow-ignore-next-line complexity
const id = el.id; function composeIndividualTransforms(cs: CSSStyleDeclaration): DOMMatrix | null {
if (!id) continue; const translate = cs.getPropertyValue("translate").trim();
const rect = el.getBoundingClientRect(); const rotate = cs.getPropertyValue("rotate").trim();
const style = window.getComputedStyle(el); const scale = cs.getPropertyValue("scale").trim();
const zIndex = getEffectiveZIndex(el); const hasTranslate = translate && translate !== "none";
const isHdrEl = hdrSet.has(id); const hasRotate = rotate && rotate !== "none";
// The frame injector now uses `visibility: hidden` (without `opacity: 0`) const hasScale = scale && scale !== "none";
// to hide native <video> elements, so the element's own computed opacity if (!hasTranslate && !hasRotate && !hasScale) return null;
// remains the GSAP-controlled value. Walk from the element itself to let m = new DOMMatrix();
// multiply through any ancestor opacity stacks. if (hasTranslate) {
const opacity = getEffectiveOpacity(el); const parts = translate.split(/\s+/);
const visible = isElementPaintable(el); const tx = parseFloat(parts[0] ?? "0") || 0;
const renderFrame = document.getElementById(`__render_frame_${id}__`); const ty = parseFloat(parts[1] ?? "0") || 0;
const renderFrameVisible = renderFrame ? isElementPaintable(renderFrame) : false; if (tx !== 0 || ty !== 0) m = m.translate(tx, ty);
// offsetWidth/offsetHeight only exist on HTMLElement (not on }
// SVGElement, MathMLElement, etc.). Fall back to the bounding rect if (hasRotate) {
// dimensions for non-HTML elements so callers always get sensible const deg = parseFloat(rotate) || 0;
// layout numbers. if (deg !== 0) m = m.rotate(deg);
const htmlEl = el instanceof HTMLElement ? el : null; }
results.push({ if (hasScale) {
id, const parts = scale.split(/\s+/);
zIndex, const sx = parseFloat(parts[0] ?? "1") || 1;
x: Math.round(rect.x), const sy = parseFloat(parts[1] ?? String(sx)) || sx;
y: Math.round(rect.y), if (sx !== 1 || sy !== 1) m = m.scale(sx, sy);
width: Math.round(rect.width), }
height: Math.round(rect.height), return m;
layoutWidth: htmlEl?.offsetWidth || Math.round(rect.width), }
layoutHeight: htmlEl?.offsetHeight || Math.round(rect.height),
opacity, function resolveLength(value: string, basis: number): number {
visible, if (value.endsWith("%")) {
renderFrameVisible, const pct = parseFloat(value) / 100;
isHdr: hdrSet.has(id), return Number.isFinite(pct) ? pct * basis : 0;
// For HDR elements, use the full accumulated viewport matrix so the }
// affine blit can apply rotation/scale/translate properly. For DOM const n = parseFloat(value);
// elements, the element-level transform is sufficient for reference. return Number.isFinite(n) ? n : 0;
transform: isHdrEl ? getViewportMatrix(el) : style.transform || "none", }
borderRadius: isHdrEl ? getEffectiveBorderRadius(el) : [0, 0, 0, 0],
// `getComputedStyle` returns "" when the property doesn't apply (e.g. function isElementPaintable(node: Element): boolean {
// for non-replaced elements); normalize to the CSS defaults so callers const rect = node.getBoundingClientRect();
// can rely on a populated value. const style = window.getComputedStyle(node);
objectFit: style.objectFit || "fill", return (
objectPosition: style.objectPosition || "50% 50%", style.visibility !== "hidden" &&
clipRect: isHdrEl ? getClipRect(el) : null, style.display !== "none" &&
}); rect.width > 0 &&
} rect.height > 0
return results; );
}, hdrIds); }
for (const el of elements) {
// Report the render id so callers can match these back to the media list.
// `||`, not `??`: `__hfMediaId` yields "" for an element with neither id.
const id = window.__hfMediaId?.(el) || el.id;
if (!id) continue;
const rect = el.getBoundingClientRect();
const style = window.getComputedStyle(el);
const zIndex = getEffectiveZIndex(el);
const isHdrEl = hdrSet.has(id);
// The frame injector now uses `visibility: hidden` (without `opacity: 0`)
// to hide native <video> elements, so the element's own computed opacity
// remains the GSAP-controlled value. Walk from the element itself to
// multiply through any ancestor opacity stacks.
const opacity = getEffectiveOpacity(el);
const visible = isElementPaintable(el);
const renderFrame = document.getElementById(`${prefix}${id}${suffix}`);
const renderFrameVisible = renderFrame ? isElementPaintable(renderFrame) : false;
// offsetWidth/offsetHeight only exist on HTMLElement (not on
// SVGElement, MathMLElement, etc.). Fall back to the bounding rect
// dimensions for non-HTML elements so callers always get sensible
// layout numbers.
const htmlEl = el instanceof HTMLElement ? el : null;
results.push({
id,
zIndex,
x: Math.round(rect.x),
y: Math.round(rect.y),
width: Math.round(rect.width),
height: Math.round(rect.height),
layoutWidth: htmlEl?.offsetWidth || Math.round(rect.width),
layoutHeight: htmlEl?.offsetHeight || Math.round(rect.height),
opacity,
visible,
renderFrameVisible,
isHdr: hdrSet.has(id),
// For HDR elements, use the full accumulated viewport matrix so the
// affine blit can apply rotation/scale/translate properly. For DOM
// elements, the element-level transform is sufficient for reference.
transform: isHdrEl ? getViewportMatrix(el) : style.transform || "none",
borderRadius: isHdrEl ? getEffectiveBorderRadius(el) : [0, 0, 0, 0],
// `getComputedStyle` returns "" when the property doesn't apply (e.g.
// for non-replaced elements); normalize to the CSS defaults so callers
// can rely on a populated value.
objectFit: style.objectFit || "fill",
objectPosition: style.objectPosition || "50% 50%",
clipRect: isHdrEl ? getClipRect(el) : null,
});
}
return results;
},
hdrIds,
RENDER_FRAME_ID_PREFIX,
RENDER_FRAME_ID_SUFFIX,
);
} }
@@ -45,7 +45,10 @@ export function parseAudioElements(html: string): AudioElement[] {
const tagName = (match[1] ?? "").toLowerCase() as "audio" | "video"; const tagName = (match[1] ?? "").toLowerCase() as "audio" | "video";
const start = parseFloat(match[2] ?? ""); const start = parseFloat(match[2] ?? "");
const idMatch = fullTag.match(/id=["']([^"']+)["']/); // `(?<![\w-])` keeps the plain-id pattern off `data-hf-render-id="…"` (and
// `data-hf-id`), which would otherwise match first and report the wrong id.
const idMatch = fullTag.match(/(?<![\w-])id=["']([^"']+)["']/);
const renderIdMatch = fullTag.match(/data-hf-render-id=["']([^"']+)["']/);
const srcMatch = fullTag.match(/src=["']([^"']+)["']/); const srcMatch = fullTag.match(/src=["']([^"']+)["']/);
if (!srcMatch) continue; if (!srcMatch) continue;
@@ -65,7 +68,9 @@ export function parseAudioElements(html: string): AudioElement[] {
: 0; : 0;
elements.push({ elements.push({
id: idMatch?.[1] || `media-${elements.length}`, // The stamped render id is document-unique; the authored id is only
// unique within one composition file. See core's mediaRenderIds.ts.
id: renderIdMatch?.[1] || idMatch?.[1] || `media-${elements.length}`,
src: srcMatch[1] ?? "", src: srcMatch[1] ?? "",
start: isNaN(start) ? 0 : start, start: isNaN(start) ? 0 : start,
duration, duration,
@@ -2561,3 +2561,135 @@ describe("compileForRender non-media payload sniff (STUDIO-5433)", () => {
expect(warnings.join("\n")).toContain("a1"); expect(warnings.join("\n")).toContain("a1");
}); });
}); });
describe("duplicate media ids across nested compositions", () => {
// Element ids are unique per composition FILE; the render document is the
// inlined union of every file. The producer used to merge the per-file media
// lists and deduplicate by id, so colliding clips collapsed into one entry
// and the survivor's frames were injected onto whichever element came first
// in the document — leaving the visible scene without footage (#3340).
function sceneWithVideoId(label: string, mediaStart: number): string {
return `<div data-composition-id="${label}" data-start="0" data-duration="3"
data-width="640" data-height="360">
<video id="clip" src="../assets/long-take.mp4" data-start="0" data-duration="3"
data-media-start="${mediaStart}" data-track-index="0"></video>
</div>`;
}
function writeTwoSceneProject(
sceneAFile: string,
sceneBFile: string = sceneAFile,
sceneBody: (label: string, mediaStart: number) => string = sceneWithVideoId,
): { projectDir: string; indexPath: string } {
const projectDir = mkdtempSync(join(tmpdir(), "hf-dup-media-"));
const compositionsDir = join(projectDir, "compositions");
mkdirSync(compositionsDir, { recursive: true });
writeFileSync(
join(projectDir, "index.html"),
`<!DOCTYPE html>
<html><head></head><body>
<div id="root" data-composition-id="root" data-start="0" data-duration="6"
data-width="640" data-height="360">
<div id="scene-a-host" data-composition-id="scene-a" data-composition-src="compositions/${sceneAFile}"
data-start="0" data-duration="3" data-width="640" data-height="360"></div>
<div id="scene-b-host" data-composition-id="scene-b" data-composition-src="compositions/${sceneBFile}"
data-start="3" data-duration="3" data-width="640" data-height="360"></div>
</div>
</body></html>`,
);
const wrap = (label: string, mediaStart: number) =>
`<template id="${label}-template">\n${sceneBody(label, mediaStart)}\n</template>`;
writeFileSync(join(compositionsDir, sceneAFile), wrap("scene-a", 5));
if (sceneBFile !== sceneAFile) {
writeFileSync(join(compositionsDir, sceneBFile), wrap("scene-b", 50));
}
return { projectDir, indexPath: join(projectDir, "index.html") };
}
it("keeps both clips when two scenes declare the same video id", async () => {
const { projectDir, indexPath } = writeTwoSceneProject("scene-a.html", "scene-b.html");
const compiled = await compileForRender(projectDir, indexPath, projectDir);
expect(compiled.videos).toHaveLength(2);
expect(compiled.videos[0]).toMatchObject({ start: 0, end: 3, mediaStart: 5 });
expect(compiled.videos[1]).toMatchObject({ start: 3, end: 6, mediaStart: 50 });
});
it("gives the colliding clips ids that each address one element", async () => {
const { projectDir, indexPath } = writeTwoSceneProject("scene-a.html", "scene-b.html");
const compiled = await compileForRender(projectDir, indexPath, projectDir);
const ids = compiled.videos.map((v) => v.id);
expect(new Set(ids).size).toBe(2);
// One element per id is exactly what the frame injector relies on.
const { document } = parseHTML(compiled.html);
for (const id of ids) {
expect(document.querySelectorAll(`[data-hf-render-id="${id}"]`)).toHaveLength(1);
}
});
it("keeps author ids intact so scene CSS and scripts still resolve", async () => {
const { projectDir, indexPath } = writeTwoSceneProject("scene-a.html", "scene-b.html");
const compiled = await compileForRender(projectDir, indexPath, projectDir);
const { document } = parseHTML(compiled.html);
expect(document.querySelectorAll('video[id="clip"]')).toHaveLength(2);
});
it("keeps both clips when the same scene file is mounted twice", async () => {
// The author cannot make these unique: it is one file, mounted twice.
const { projectDir, indexPath } = writeTwoSceneProject("scene.html");
const compiled = await compileForRender(projectDir, indexPath, projectDir);
expect(compiled.videos).toHaveLength(2);
expect(compiled.videos[0]).toMatchObject({ start: 0, end: 3 });
expect(compiled.videos[1]).toMatchObject({ start: 3, end: 6 });
});
it("keeps both clips when neither scene names its video", async () => {
// No authored id at all: the timing compiler numbers auto-ids per file, so
// both scenes arrive as `hf-video-0`.
const { projectDir, indexPath } = writeTwoSceneProject(
"scene-a.html",
"scene-b.html",
(label, mediaStart) =>
`<div data-composition-id="${label}" data-start="0" data-duration="3"
data-width="640" data-height="360">
<video src="../assets/long-take.mp4" data-start="0" data-duration="3"
data-media-start="${mediaStart}" data-track-index="0"></video>
</div>`,
);
const compiled = await compileForRender(projectDir, indexPath, projectDir);
expect(compiled.videos).toHaveLength(2);
expect(compiled.videos.map((v) => v.mediaStart)).toEqual([5, 50]);
});
it("keeps both audio tracks when two scenes declare the same audio id", async () => {
const { projectDir, indexPath } = writeTwoSceneProject(
"scene-a.html",
"scene-b.html",
(label, mediaStart) =>
`<div data-composition-id="${label}" data-start="0" data-duration="3"
data-width="640" data-height="360">
<audio id="vo" src="../assets/narration.wav" data-start="0" data-duration="3"
data-media-start="${mediaStart}" data-track-index="1"></audio>
</div>`,
);
const compiled = await compileForRender(projectDir, indexPath, projectDir);
expect(compiled.audios).toHaveLength(2);
expect(compiled.audios[0]).toMatchObject({ start: 0, end: 3, mediaStart: 5 });
expect(compiled.audios[1]).toMatchObject({ start: 3, end: 6, mediaStart: 50 });
});
});
+34 -127
View File
@@ -30,6 +30,7 @@ import {
import { MAX_AUDIO_GAIN } from "@hyperframes/core/audio-gain"; import { MAX_AUDIO_GAIN } from "@hyperframes/core/audio-gain";
import { import {
assignBundledRuntimeCompositionIds, assignBundledRuntimeCompositionIds,
assignMediaRenderIds,
type BundledHostCompositionIdentity, type BundledHostCompositionIdentity,
buildVariablesByCompScript, buildVariablesByCompScript,
inlineSubCompositions as inlineSubCompositionsShared, inlineSubCompositions as inlineSubCompositionsShared,
@@ -45,12 +46,10 @@ import {
import { isUnresolvedAssetPlaceholder } from "@hyperframes/parsers/asset-resolution"; import { isUnresolvedAssetPlaceholder } from "@hyperframes/parsers/asset-resolution";
import { extractMediaMetadata, extractAudioMetadata } from "../utils/ffprobe.js"; import { extractMediaMetadata, extractAudioMetadata } from "../utils/ffprobe.js";
import { isPathInside, toExternalAssetKey } from "../utils/paths.js"; import { isPathInside, toExternalAssetKey } from "../utils/paths.js";
import { collectRenderMedia } from "./renderMediaCollector.js";
import { import {
parseVideoElements,
parseImageElements,
type VideoElement, type VideoElement,
type ImageElement, type ImageElement,
parseAudioElements,
type AudioElement, type AudioElement,
type AudioVolumeKeyframe, type AudioVolumeKeyframe,
type MediaProbeProfile, type MediaProbeProfile,
@@ -252,14 +251,6 @@ export interface RenderModeHints {
reasons: RenderModeHint[]; reasons: RenderModeHint[];
} }
function dedupeElementsById<T extends { id: string }>(elements: T[]): T[] {
const deduped = new Map<string, T>();
for (const element of elements) {
deduped.set(element.id, element);
}
return Array.from(deduped.values());
}
const INLINE_SCRIPT_PATTERN = /<script\b([^>]*)>([\s\S]*?)<\/script>/gi; const INLINE_SCRIPT_PATTERN = /<script\b([^>]*)>([\s\S]*?)<\/script>/gi;
const COMPILER_MOUNT_BLOCK_START = "/* __HF_COMPILER_MOUNT_START__ */"; const COMPILER_MOUNT_BLOCK_START = "/* __HF_COMPILER_MOUNT_START__ */";
const COMPILER_MOUNT_BLOCK_END = "/* __HF_COMPILER_MOUNT_END__ */"; const COMPILER_MOUNT_BLOCK_END = "/* __HF_COMPILER_MOUNT_END__ */";
@@ -618,26 +609,21 @@ async function compileHtmlFile(
} }
/** /**
* Parse sub-compositions referenced via data-composition-src. * Compile every sub-composition referenced via data-composition-src, keyed by
* Reads each file, compiles it, extracts video/audio, adjusts timing offsets. * its source path for the inliner to hoist into the render document.
* Recurses into nested sub-compositions with accumulated offsets. * Recurses so nested references are compiled too.
*
* Media used to be extracted here as well, with each file's clips offset onto
* the parent timeline. That is now read off the inlined document instead
* (collectRenderMedia): per-file extraction had to merge on element id, which
* is not unique across files, so colliding clips silently collapsed (#3340).
*/ */
async function parseSubCompositions( async function parseSubCompositions(
html: string, html: string,
projectDir: string, projectDir: string,
downloadDir: string, downloadDir: string,
parentOffset: number = 0,
parentEnd: number = Infinity,
visited: Set<string> = new Set(), visited: Set<string> = new Set(),
): Promise<{ ): Promise<{ subCompositions: Map<string, string> }> {
videos: VideoElement[];
audios: AudioElement[];
images: ImageElement[];
subCompositions: Map<string, string>;
}> {
const videos: VideoElement[] = [];
const audios: AudioElement[] = [];
const images: ImageElement[] = [];
const subCompositions = new Map<string, string>(); const subCompositions = new Map<string, string>();
const { document } = parseHTML(html); const { document } = parseHTML(html);
@@ -646,8 +632,6 @@ async function parseSubCompositions(
// Build work items, filtering out invalid/circular entries synchronously // Build work items, filtering out invalid/circular entries synchronously
const workItems: Array<{ const workItems: Array<{
srcPath: string; srcPath: string;
absoluteStart: number;
absoluteEnd: number;
filePath: string; filePath: string;
rawSubHtml: string; rawSubHtml: string;
nestedVisited: Set<string>; nestedVisited: Set<string>;
@@ -657,12 +641,6 @@ async function parseSubCompositions(
const srcPath = el.getAttribute("data-composition-src"); const srcPath = el.getAttribute("data-composition-src");
if (!srcPath) continue; if (!srcPath) continue;
const elStart = parseFloat(el.getAttribute("data-start") || "0");
const elEnd = parseStrictFiniteTimingNumber(el.getAttribute("data-end")) ?? Infinity;
const absoluteStart = parentOffset + elStart;
const absoluteEnd = Math.min(parentEnd, isFinite(elEnd) ? parentOffset + elEnd : Infinity);
const filePath = resolve(projectDir, srcPath); const filePath = resolve(projectDir, srcPath);
// Circular reference guard // Circular reference guard
@@ -678,7 +656,7 @@ async function parseSubCompositions(
const nestedVisited = new Set(visited); const nestedVisited = new Set(visited);
nestedVisited.add(filePath); nestedVisited.add(filePath);
workItems.push({ srcPath, absoluteStart, absoluteEnd, filePath, rawSubHtml, nestedVisited }); workItems.push({ srcPath, filePath, rawSubHtml, nestedVisited });
} }
// Parallelize file compilation + recursive parsing // Parallelize file compilation + recursive parsing
@@ -694,24 +672,13 @@ async function parseSubCompositions(
compiledSub, compiledSub,
projectDir, projectDir,
downloadDir, downloadDir,
item.absoluteStart,
item.absoluteEnd,
item.nestedVisited, item.nestedVisited,
); );
const subVideos = parseVideoElements(compiledSub);
const subAudios = parseAudioElements(compiledSub);
const subImages = parseImageElements(compiledSub);
return { return {
srcPath: item.srcPath, srcPath: item.srcPath,
compiledSub, compiledSub,
nested, nested,
subVideos,
subAudios,
subImages,
absoluteStart: item.absoluteStart,
absoluteEnd: item.absoluteEnd,
}; };
}), }),
); );
@@ -723,55 +690,9 @@ async function parseSubCompositions(
for (const [key, value] of r.nested.subCompositions) { for (const [key, value] of r.nested.subCompositions) {
subCompositions.set(key, value); subCompositions.set(key, value);
} }
videos.push(...r.nested.videos);
audios.push(...r.nested.audios);
images.push(...r.nested.images);
for (const v of r.subVideos) {
v.start += r.absoluteStart;
v.end += r.absoluteStart;
if (v.end > r.absoluteEnd) {
v.end = r.absoluteEnd;
}
if (v.start < r.absoluteEnd) {
videos.push(v);
}
}
for (const a of r.subAudios) {
a.start += r.absoluteStart;
a.end += r.absoluteStart;
if (a.end > r.absoluteEnd) {
a.end = r.absoluteEnd;
}
if (a.start < r.absoluteEnd) {
audios.push(a);
}
}
for (const img of r.subImages) {
img.start += r.absoluteStart;
img.end += r.absoluteStart;
if (img.end > r.absoluteEnd) {
img.end = r.absoluteEnd;
}
if (img.start < r.absoluteEnd) {
images.push(img);
}
}
if (
r.subVideos.length > 0 ||
r.subAudios.length > 0 ||
r.subImages.length > 0 ||
r.nested.videos.length > 0 ||
r.nested.audios.length > 0 ||
r.nested.images.length > 0
) {
}
} }
return { videos, audios, images, subCompositions }; return { subCompositions };
} }
/** /**
@@ -1130,6 +1051,12 @@ function inlineSubCompositions(
variableOverrides, variableOverrides,
); );
// Inlining is what makes element ids ambiguous: each composition file is
// internally consistent, the union of them is not. Hand every media element a
// document-unique key here, while the merged document is in hand and before
// anything downstream keys media on an id. See core's mediaRenderIds.ts.
assignMediaRenderIds(document as unknown as Document);
return document.toString(); return document.toString();
} }
@@ -1882,13 +1809,8 @@ export async function compileForRender(
options.log, options.log,
); );
// Parse sub-compositions first (extracts media + compiled HTML for each) // Compile each referenced sub-composition so the inliner can hoist it.
const { const { subCompositions } = await parseSubCompositions(compiledHtml, projectDir, downloadDir);
videos: subVideos,
audios: subAudios,
images: subImages,
subCompositions,
} = await parseSubCompositions(compiledHtml, projectDir, downloadDir);
// Ensure the HTML is a full document before inlining sub-compositions. // Ensure the HTML is a full document before inlining sub-compositions.
// When index.html is a fragment (no <html>/<head>/<body>), linkedom.parseHTML() // When index.html is a fragment (no <html>/<head>/<body>), linkedom.parseHTML()
@@ -2030,17 +1952,12 @@ export async function compileForRender(
externalAssets.set(relPath, absPath); externalAssets.set(relPath, absPath);
} }
// Parse main HTML elements // Read the media list off the inlined document rather than merging the
const mainVideos = parseVideoElements(html); // per-file lists. Merging deduplicated by element id, which is only unique
const mainAudios = parseAudioElements(html); // within one composition file: two scenes declaring `<video id="clip">` — or
const mainImages = parseImageElements(html); // two bare `<video>`s, both auto-numbered `hf-video-0` — collapsed into one
// entry and injected frames onto whichever element came first. See #3340.
// Keep inlined sub-composition media authoritative on ID collisions. const { videos, audios, images } = collectRenderMedia(html);
// inlineSubCompositions() hoists those nodes into the final HTML, so the
// producer should follow the same precedence the runtime sees in the merged DOM.
const videos = dedupeElementsById([...mainVideos, ...subVideos]);
const audios = dedupeElementsById([...mainAudios, ...subAudios]);
const images = dedupeElementsById([...mainImages, ...subImages]);
// Advisory video checks (sparse keyframes, VFR). Fire-and-forget — these spawn // Advisory video checks (sparse keyframes, VFR). Fire-and-forget — these spawn
// ffprobe subprocesses and should not block compilation since they only produce warnings. // ffprobe subprocesses and should not block compilation since they only produce warnings.
@@ -2570,23 +2487,13 @@ export async function recompileWithResolutions(
const html = injectDurations(compiled.html, resolutions); const html = injectDurations(compiled.html, resolutions);
// Re-parse sub-compositions with the updated parent bounds // Re-resolve the sub-composition map against the updated HTML, but keep the
const { // media list from the first pass. Resolving a composition's duration stamps a
videos: subVideos, // `data-end` on its host, and re-collecting would newly clamp clips to it —
audios: subAudios, // a retiming, not an identity fix. `compiled.videos` was already collected
images: subImages, // from this same inlined document, so it is complete and correctly keyed.
subCompositions, const { subCompositions } = await parseSubCompositions(html, projectDir, downloadDir);
} = await parseSubCompositions(html, projectDir, downloadDir); const { videos, audios, images } = compiled;
const mainVideos = parseVideoElements(html);
const mainAudios = parseAudioElements(html);
const mainImages = parseImageElements(html);
// Keep inlined sub-composition media authoritative on ID collisions.
const hasSubMedia = subVideos.length > 0 || subAudios.length > 0 || subImages.length > 0;
const videos = hasSubMedia ? dedupeElementsById([...mainVideos, ...subVideos]) : compiled.videos;
const audios = hasSubMedia ? dedupeElementsById([...mainAudios, ...subAudios]) : compiled.audios;
const images = hasSubMedia ? dedupeElementsById([...mainImages, ...subImages]) : compiled.images;
const remaining = compiled.unresolvedCompositions.filter( const remaining = compiled.unresolvedCompositions.filter(
(c) => !resolutions.some((r) => r.id === c.id), (c) => !resolutions.some((r) => r.id === c.id),
@@ -0,0 +1,158 @@
/**
* Collect the render pipeline's media list from the fully inlined document.
*
* Sub-composition media used to be gathered from each composition FILE before
* inlining, then merged with the main document's media and deduplicated by
* element id. That merge is unsound: ids are unique per file, not per render
* document, so two scenes that both declare `<video id="clip">` or that both
* declare a bare `<video>` and get the per-file auto-id `hf-video-0` collapse
* into a single entry. See mediaRenderIds.ts for the full failure.
*
* Reading the inlined document instead makes the render document the single
* source of truth for what media exists: every element is present exactly once,
* `assignMediaRenderIds` has already given it a document-unique key, and the
* timeline offsets are recoverable from the composition hosts it sits inside.
*/
import { parseHTML } from "linkedom";
import { MEDIA_RENDER_ID_ATTR } from "@hyperframes/core";
import {
parseVideoElements,
parseImageElements,
parseAudioElements,
type VideoElement,
type ImageElement,
type AudioElement,
} from "@hyperframes/engine";
/**
* Marks a host element that `inlineSubCompositions` hoisted a composition into.
* Set unconditionally on every inlined host, which makes it the reliable signal
* for "this ancestor shifts its children along the timeline".
*/
const COMPOSITION_HOST_ATTR = "data-composition-file";
interface HostWindow {
/** Seconds to add to a descendant's authored, scene-relative start. */
offset: number;
/** Absolute time past which a descendant is outside its host, or Infinity. */
limit: number;
}
const ROOT_WINDOW: HostWindow = { offset: 0, limit: Infinity };
function parseNumeric(value: string | null): number | null {
if (value == null || value === "") return null;
const parsed = Number.parseFloat(value);
return Number.isFinite(parsed) ? parsed : null;
}
/**
* Fold a media element's chain of composition hosts into one window.
*
* Mirrors the offset arithmetic `parseSubCompositions` applied while walking
* the composition file tree, so a document that has no id collisions produces
* exactly the timings it did before. Only `data-end` bounds a host: a host
* carrying just `data-duration` was unbounded there too, and widening that here
* would silently retime existing compositions rather than fix identity.
*/
function resolveHostWindow(element: Element): HostWindow {
const hosts: Element[] = [];
for (let ancestor = element.parentElement; ancestor; ancestor = ancestor.parentElement) {
if (ancestor.hasAttribute(COMPOSITION_HOST_ATTR)) hosts.push(ancestor);
}
if (hosts.length === 0) return ROOT_WINDOW;
let offset = 0;
let limit = Infinity;
// parentElement walks leaf → root; the offsets accumulate root → leaf.
for (const host of hosts.reverse()) {
const hostStart = parseNumeric(host.getAttribute("data-start")) ?? 0;
const hostEnd = parseNumeric(host.getAttribute("data-end"));
if (hostEnd != null) limit = Math.min(limit, offset + hostEnd);
offset += hostStart;
}
return { offset, limit };
}
/**
* Map each render id to the window of the composition hosts it is nested in.
* Keyed on the render id rather than document position so the caller never has
* to assume two separate parses walk the document in the same order.
*/
function collectHostWindows(html: string): Map<string, HostWindow> {
const { document } = parseHTML(html);
const windows = new Map<string, HostWindow>();
for (const element of document.querySelectorAll(`[${MEDIA_RENDER_ID_ATTR}]`)) {
const renderId = element.getAttribute(MEDIA_RENDER_ID_ATTR);
if (!renderId) continue;
windows.set(renderId, resolveHostWindow(element as unknown as Element));
}
return windows;
}
/**
* Shift a scene-relative window onto the root timeline.
* Returns null when the clip starts after its host has already ended, matching
* the `start < absoluteEnd` drop the file-tree walk applied.
*/
function toAbsoluteWindow(
start: number,
end: number,
window: HostWindow,
): { start: number; end: number } | null {
const absoluteStart = start + window.offset;
if (absoluteStart >= window.limit) return null;
const absoluteEnd = end + window.offset;
return { start: absoluteStart, end: Math.min(absoluteEnd, window.limit) };
}
export interface RenderMedia {
videos: VideoElement[];
audios: AudioElement[];
images: ImageElement[];
}
/**
* Parse every media element in the inlined render document, with each clip's
* window resolved onto the root timeline.
*
* Expects `assignMediaRenderIds` to have run: the parsers report the stamped
* render id as each element's `id`, which is what the rest of the pipeline
* keys on and what the engine resolves back to a DOM node.
*/
export function collectRenderMedia(html: string): RenderMedia {
const windows = collectHostWindows(html);
const windowFor = (id: string): HostWindow => windows.get(id) ?? ROOT_WINDOW;
const videos: VideoElement[] = [];
for (const video of parseVideoElements(html)) {
const absolute = toAbsoluteWindow(video.start, video.end, windowFor(video.id));
if (absolute) videos.push({ ...video, ...absolute });
}
const images: ImageElement[] = [];
for (const image of parseImageElements(html)) {
const absolute = toAbsoluteWindow(image.start, image.end, windowFor(image.id));
if (absolute) images.push({ ...image, ...absolute });
}
// A <video data-has-audio> track is reported as "<renderId>-audio"; strip the
// suffix to look the element's host window back up.
const audios: AudioElement[] = [];
for (const audio of parseAudioElements(html)) {
const elementId = audio.type === "video" ? audio.id.replace(/-audio$/, "") : audio.id;
// The mixer reads end === 0 as "run to the natural media length", so an
// unbounded track must stay unbounded rather than collapse onto its start.
const authoredEnd = audio.end > 0 ? audio.end : Infinity;
const absolute = toAbsoluteWindow(audio.start, authoredEnd, windowFor(elementId));
if (!absolute) continue;
audios.push({
...audio,
start: absolute.start,
end: Number.isFinite(absolute.end) ? absolute.end : 0,
});
}
return { videos, audios, images };
}