ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(producer,lint): id-less media renders blank wash instead of footage (#1790)

Browse Source 
* fix(producer,lint): id-less media renders blank wash instead of footage

A timed <video>/<audio> identified only by a Studio-stamped `data-hf-id`
(no real `id`) rendered as a flat white/grey wash with dropped audio, and
lint stayed silent so it surfaced only at render.

Root cause, two layers:

- lint `readAttr(tag, "id")` used a `\b` boundary, which treats the hyphen
  in `data-hf-id="…"` as a word break — so reading "id" matched the trailing
  `id="…"` inside `data-hf-id` and returned a phantom id. `media_missing_id`
  therefore never fired for media carrying only a data-hf-id. Switched to a
  `(?<![\w-])` lookbehind so a short name can't match the tail of a longer
  hyphenated attribute (also fixes "width" matching `data-width`, etc.).

- the render pipeline identifies media by the real `el.id`: frame extraction
  keys injected stills as `__render_frame_<id>__`, the runtime frame-swap
  matches on `el.id`, and the audio mixer selects `audio[id][src]`. An empty
  `el.id` meant injected frames/audio never matched. compileForRender now
  assigns a stable positional id to every id-less timed media element before
  any stage parses or serves the HTML.

Adds a producer regression fixture (video with data-hf-id, no id) and a lint
test covering the data-hf-id/id collision. Baseline mp4 generated separately.

* test(producer): baseline for video-hfid-no-id regression fixture

Golden compiled.html + output.mp4 (generated on linux/amd64 in the
Dockerfile.test image). Compare-mode passes: compilation, visual (0 failed
frames), and audio (correlation 1.000). A regression to the blank-wash
behaviour fails the visual check.
This commit is contained in:
Miguel Ángel
2026-06-29 18:28:24 -07:00
committed by GitHub
parent c9613cd826
commit 74f9c31b3f
8 changed files with 203 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/lint/src/rules/media.test.ts
+19
View File
@@ -48,6 +48,25 @@ describe("media rules", () => {
expect(finding?.message).toContain("FROZEN");
});
it("flags media that has data-hf-id but no real id", async () => {
// Regression: readAttr(tag, "id") used a \b boundary that matched the
// trailing `id="…"` inside `data-hf-id="…"`, so media carrying only a
// Studio-stamped data-hf-id passed the check and then rendered as a blank
// wash (video) / silent (audio). data-hf-id is NOT a render id.
const html = `
<html><body>
<div id="root" data-composition-id="c1" data-width="1920" data-height="1080">
<video data-hf-id="hf-v1a2b3" data-start="0" data-duration="10" src="clip.mp4" muted playsinline></video>
<audio data-hf-id="hf-a4c5d6" data-start="0" data-duration="10" src="narration.wav"></audio>
</div>
<script>window.__timelines = window.__timelines || {}; window.__timelines["c1"] = gsap.timeline({ paused: true });</script>
</body></html>`;
const result = await lintHyperframeHtml(html);
const findings = result.findings.filter((f) => f.code === "media_missing_id");
expect(findings).toHaveLength(2);
expect(findings.every((f) => f.severity === "error")).toBe(true);
});
it("does not flag media elements that have id", async () => {
const html = `
<html><body>
packages/lint/src/utils.ts
+10 -2
View File
@@ -112,7 +112,11 @@ export function findRootTag(source: string): OpenTag | null {
export function readAttr(tagSource: string, attr: string): string | null {
if (!tagSource) return null;
const escaped = attr.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const match = tagSource.match(new RegExp(`\\b${escaped}\\s*=\\s*["']([^"']+)["']`, "i"));
// `(?<![\w-])` not `\b`: a plain `\b` boundary treats the hyphen in a longer
// attribute as a word break, so reading "id" would wrongly match the trailing
// `id="…"` inside `data-hf-id="…"` (and "width" inside `data-width`, etc.).
// The lookbehind requires the match to start a fresh attribute name.
const match = tagSource.match(new RegExp(`(?<![\\w-])${escaped}\\s*=\\s*["']([^"']+)["']`, "i"));
return match?.[1] || null;
}
@@ -131,7 +135,11 @@ export function readAttr(tagSource: string, attr: string): string | null {
export function readJsonAttr(tagSource: string, attr: string): string | null {
if (!tagSource) return null;
const escaped = attr.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const match = tagSource.match(new RegExp(`\\b${escaped}\\s*=\\s*(?:"([^"]*)"|'([^']*)')`, "i"));
// See readAttr: `(?<![\w-])` prevents a short name from matching the tail of a
// longer hyphenated attribute (e.g. "id" inside `data-hf-id`).
const match = tagSource.match(
new RegExp(`(?<![\\w-])${escaped}\\s*=\\s*(?:"([^"]*)"|'([^']*)')`, "i"),
);
if (!match) return null;
return match[1] ?? match[2] ?? null;
}
packages/producer/src/services/htmlCompiler.ts
+35 -1
View File
@@ -1360,6 +1360,32 @@ function rewriteUnresolvableGsapToCdn(html: string, projectDir: string): string
* with all media metadata resolved.
*/
// fallow-ignore-next-line complexity
/**
* Every render stage identifies `<video>`/`<audio>` by their real `id`: frame
* extraction keys injected stills as `__render_frame_<id>__`, the runtime
* frame-swap matches on `el.id` (runtime/media.ts), and the audio mixer selects
* `audio[id][src]`. A timed media element with no `id` — e.g. one carrying only
* `data-hf-id`, which is what Studio stamps — has an empty `el.id`, so its
* injected frames never match: the video renders as a blank wash and any
* separate `<audio>` is silently dropped. Assign a stable positional `id` to
* every timed media element missing one, on the same HTML that is parsed for
* media and served to the renderer, so the whole pipeline shares one identity.
* `data-hf-id` is intentionally NOT reused as the id — it is a Studio edit
* handle, not the render identity.
*/
function assignMissingMediaIds(html: string): string {
const { document } = parseHTML(html);
const media = document.querySelectorAll("video[data-start], audio[data-start]");
let seq = 0;
let changed = false;
for (const el of Array.from(media)) {
if (el.getAttribute("id")) continue;
el.setAttribute("id", `hf-media-${seq++}`);
changed = true;
}
return changed ? document.toString() : html;
}
export async function compileForRender(
projectDir: string,
htmlPath: string,
@@ -1484,7 +1510,15 @@ export async function compileForRender(
// Collect assets that resolve outside projectDir (e.g. ../shared-assets/hero.png).
// These can't be served by the file server, so we map them to paths the
// orchestrator will copy into the compiled output directory.
const { html, externalAssets } = collectExternalAssets(embeddedHtml, projectDir);
const { html: htmlBeforeMediaIds, externalAssets } = collectExternalAssets(
embeddedHtml,
projectDir,
);
// Give every timed <video>/<audio> a real `id` before any stage parses or
// serves this HTML — id-less media (e.g. carrying only `data-hf-id`) would
// otherwise render as a blank wash with dropped audio. See assignMissingMediaIds.
const html = assignMissingMediaIds(htmlBeforeMediaIds);
for (const [relPath, absPath] of remoteMediaAssets) {
externalAssets.set(relPath, absPath);
packages/producer/tests/video-hfid-no-id/meta.json
+12
View File
@@ -0,0 +1,12 @@
{
"name": "Video identified only by data-hf-id (no real id)",
"description": "Regression for: a timed <video> carrying only a Studio-stamped data-hf-id and no real id rendered as a blank white/grey wash. The render pipeline identifies media by el.id (injected frames keyed __render_frame_<id>__, runtime frame-swap, audio mixer audio[id][src]); an empty el.id meant injected frames never matched. compileForRender now assigns a stable positional id to id-less timed media, so the footage composites correctly.",
"tags": ["video", "data-hf-id", "media-id", "regression"],
"minPsnr": 25,
"maxFrameFailures": 5,
"minAudioCorrelation": 0.0,
"maxAudioLagWindows": 120,
"renderConfig": {
"fps": 30
}
}
packages/producer/tests/video-hfid-no-id/output/compiled.html
+61
View File
File diff suppressed because one or more lines are too long
packages/producer/tests/video-hfid-no-id/output/output.mp4
BIN
View File
Binary file not shown.
packages/producer/tests/video-hfid-no-id/src/clip.mp4
BIN
View File
Binary file not shown.
packages/producer/tests/video-hfid-no-id/src/index.html
+66
View File
@@ -0,0 +1,66 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=1280, height=720" />
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
html, body { width: 1280px; height: 720px; overflow: hidden; background: transparent; }
#root { position: relative; width: 1280px; height: 720px; overflow: hidden; }
#root video { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: cover; z-index: 0; }
.label {
position: absolute;
inset: 0;
z-index: 10;
display: flex;
align-items: center;
justify-content: center;
font-family: system-ui, sans-serif;
font-size: 72px;
font-weight: 800;
color: #fff;
text-shadow: 0 4px 24px rgba(0, 0, 0, 0.6);
}
</style>
</head>
<body>
<div
id="root"
data-composition-id="main"
data-start="0"
data-duration="2"
data-width="1280"
data-height="720"
>
<!-- Regression: this full-bleed background video element carries only a
Studio-style data-hf-id and NO real id. Before the fix, the render
pipeline could not match its injected frames (keyed on the empty
el.id), so the footage rendered as a blank white/grey wash. The
baseline must show the testsrc2 footage, not a flat fill. -->
<video
data-hf-id="hf-bgvideo01"
class="clip"
src="clip.mp4"
muted
playsinline
data-start="0"
data-duration="2"
data-track-index="0"
></video>
<div id="caption" class="label clip" data-start="0" data-duration="2" data-track-index="10">
FOOTAGE
</div>
</div>
<script>
window.__timelines = window.__timelines || {};
var tl = gsap.timeline({ paused: true });
// static hold — the regression is about the video surface compositing,
// not motion; keep the timeline deterministic.
tl.fromTo("#caption", { opacity: 1 }, { opacity: 1, duration: 2 }, 0);
window.__timelines["main"] = tl;
</script>
</body>
</html>