fix(parsers,sdk,studio-server,studio): unify hf-id space across preview, disk, and SDK session (#1981)

* fix(parsers,sdk,studio-server,studio): unify hf-id space across preview, disk, and SDK session

Root-causes the setTiming element_not_found resolver-shadow divergence class:
timeline edits carry hf-ids read from the live preview DOM, but the preview
minted ids AFTER rewriting attributes (and never persisted them for sub-comps),
while the SDK session mints from the raw file — content-keyed minting then
yields different ids for the same element. Template-based comps were worse:
the SDK excluded the whole <template> subtree, so the session had zero
elements and every edit diverged.

- parsers: ensureHfIds now descends into <template> subtrees (linkedom's
  querySelectorAll does not), minting and pinning inner ids
- sdk: buildRoots/buildElement treat <template> as a transparent container,
  and resolution (resolveScoped, animation-id map) searches template subtrees
  via querySelectorAllDeep — template comps now model, resolve, and edit
- studio-server: the sub-comp preview route persists hf-ids to the raw file
  BEFORE the rewrite pipeline (mirrors the main route), pinning one id space
  across served DOM, disk, and SDK session
- studio: resolver-shadow skips structurally-empty sessions (no event, no
  attempt) and tags fail-open emissions with sourceReadFailed so read errors
  are distinguishable from unwired readers in telemetry

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(parsers,sdk,studio-server,studio): scope template descent, guard persist route

Addresses the 10 verified findings from the PR #1981 review:

- Restrict template transparency to COMPOSITION templates
  (<template data-composition-id>) everywhere — ensureHfIds, SDK
  buildChildren, querySelectorAllDeep. A plain <template> (runtime
  clone-source) keeps its old fully-excluded behavior: stamping its
  interior would duplicate one persisted id across every runtime clone,
  and modeling it would show phantom timeline clips.
- Guard the sub-comp persist: only .html files (the wildcard route can
  serve any project path — stamping an SVG corrupted it on disk),
  try/catch the read (file-removed race becomes 404, not 500), salt the
  etag (v2) so pre-fix cached clients don't 304 past the id pin, and
  thread the stamped content into buildSubCompositionHtml so served ids
  match the mint even when the disk write is skipped.
- Rewrite querySelectorAllDeep as a document-order DOM walk — appending
  template matches after top-level matches made duplicate-id tiebreaks
  disagree with the preview's unwrapped DOM (wrong-element edits).
- Recurse sourceMutation.querySelectorAllWithTemplates so server-side
  ops resolve ids at any template depth, matching SDK resolution.
- Replace the empty-session silent skip with ONE tagged session_empty
  event per session — silence would blind the tripwire to exactly the
  modeling-gap class that exposed the template bug. Attempts stay
  uncounted (an unmodelable comp can't cut over).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(studio-server): close TOCTOU in sub-comp hf-id persist (CodeQL js/file-system-race)

Replace the route-level stat/read/persist sequence with stampFileHfIds:
validation (fstat), read, mint, and write-back all go through ONE open
file descriptor (O_NOFOLLOW where supported), so the path cannot be
swapped between validation and write. Falls back to read-only stamping
when the file isn't writable — content-keyed minting means the SDK
derives the same ids from the same bytes even without the disk write.

Addresses miguel-heygen's blocking review on PR #1981.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(studio-server): linear-time template-attr match (CodeQL js/polynomial-redos)

promoteTemplateCompositionId's single-pattern regex backtracked
polynomially on crafted input. Two-step match: grab each <template>
open tag linearly, then find data-composition-id within that short
tag text. Same semantics (first template carrying the attr wins).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-07-06 00:16:07 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent e9076324e7
commit 3a717fa719
14 changed files with 628 additions and 43 deletions
+62
View File
@@ -103,6 +103,68 @@ describe("ensureHfIds", () => {
// once ids are persisted to source (pinning) AND the behavior for truly unpinned
// HTML (no data-hf-id in the input — unreachable in production after write-back
// landed in R7 Task 1-2, but still the correct contract for that path).
describe("ensureHfIds — template-inner minting", () => {
// linkedom's querySelectorAll does not descend into <template>, so extract
// ids by regex over the serialized output instead of the DOM-walk helper.
const rawIds = (html: string) => [...html.matchAll(/data-hf-id="([^"]+)"/g)].map((m) => m[1]);
it("mints ids on elements inside a <template> (template itself stays unstamped)", () => {
const html = doc(
`<template data-composition-id="t"><div class="clip" data-start="0" data-end="3">Hi</div><p>x</p></template>`,
);
const out = ensureHfIds(html);
expect(rawIds(out)).toHaveLength(2); // div + p, not the template
expect(out).not.toMatch(/<template[^>]*data-hf-id/);
});
it("template-inner ids equal the ids minted for the same content unwrapped (preview parity)", () => {
const inner = `<div class="clip" data-start="0" data-end="3">Hi</div><p>x</p>`;
const wrapped = ensureHfIds(doc(`<template data-composition-id="t">${inner}</template>`));
const unwrapped = ensureHfIds(doc(inner));
expect(rawIds(wrapped)).toEqual(rawIds(unwrapped));
});
it("pins existing template-inner ids and seeds them against fresh mints", () => {
const html = doc(
`<template data-composition-id="t"><div data-hf-id="hf-keep">a</div><p>b</p></template>`,
);
const out = ensureHfIds(html);
expect(out).toContain('data-hf-id="hf-keep"');
expect(new Set(rawIds(out)).size).toBe(2);
});
it("descends nested composition templates", () => {
const html = doc(
`<template data-composition-id="a"><div>a</div><template data-composition-id="b"><span>b</span></template></template>`,
);
const out = ensureHfIds(html);
expect(rawIds(out)).toHaveLength(2); // div + span
});
it("does NOT stamp inside a plain <template> (runtime clone-source)", () => {
// A plain template's content is cloned N times into the live DOM at
// runtime; a persisted inner id would be duplicated across every clone.
const html = doc(`<div class="stage">x</div><template><li class="row">item</li></template>`);
const out = ensureHfIds(html);
expect(rawIds(out)).toHaveLength(1); // only the stage div
expect(out).not.toMatch(/<li[^>]*data-hf-id/);
});
it("does NOT stamp inside a plain template nested in a composition template", () => {
const html = doc(
`<template data-composition-id="t"><div>a</div><template><li>clone-src</li></template></template>`,
);
const out = ensureHfIds(html);
expect(rawIds(out)).toHaveLength(1); // only the div
expect(out).not.toMatch(/<li[^>]*data-hf-id/);
});
it("is idempotent for template-inner ids", () => {
const once = ensureHfIds(doc(`<template data-composition-id="t"><div>a</div></template>`));
expect(ensureHfIds(once)).toBe(once);
});
});
describe("ensureHfIds — edit lifecycle (R1 stability)", () => {
it("pinned id survives a content edit (the §3 write-back guarantee)", () => {
// Element already carries data-hf-id in source (as it would after write-back).
+35 -6
View File
@@ -102,6 +102,35 @@ export function mintHfId(el: Element, assigned: Set<string>): string {
return id;
}
/**
* True for a `<template data-composition-id>` — the sub-composition authoring
* pattern whose content the studio preview unwraps into the served body. Only
* these templates are treated as transparent containers for hf-id purposes.
* A plain `<template>` (runtime clone-source: list item, particle, etc.) must
* NOT get inner ids: its content is cloned N times into the live DOM, so a
* persisted inner id would be duplicated across every clone.
*/
export function isCompositionTemplate(el: Element): boolean {
return el.tagName.toLowerCase() === "template" && el.getAttribute("data-composition-id") !== null;
}
/**
* Document-order walk of every element under `root`, descending into
* composition `<template>` subtrees — linkedom's querySelectorAll does not, so
* template-based sub-comps would otherwise never get inner ids (the preview
* unwraps the template and stamps the SAME content, so skipping here splits
* the id space between the served DOM and the raw file). Plain templates are
* skipped entirely (see isCompositionTemplate).
*/
function walkElements(root: Element, visit: (el: Element) => void): void {
for (const child of Array.from(root.children)) {
const isTemplate = child.tagName.toLowerCase() === "template";
if (isTemplate && !isCompositionTemplate(child)) continue;
visit(child);
walkElements(child, visit);
}
}
export function ensureHfIds(html: string): string {
// Mirror parseSourceDocument's fragment-wrapping so bare fragments don't land
// outside <body> in linkedom, which would cause body.querySelectorAll to return [].
@@ -117,16 +146,16 @@ export function ensureHfIds(html: string): string {
// Seed with already-present ids (pin) so fresh mints never collide with them.
// Scope to <body> to match the mint walk below — a stray data-hf-id in <head>
// must not pin an id into the set that a body element would then be bumped off.
for (const el of Array.from(body.querySelectorAll("[data-hf-id]"))) {
walkElements(body, (el) => {
const existing = el.getAttribute("data-hf-id");
if (existing) assigned.add(existing);
}
});
for (const el of Array.from(body.querySelectorAll("*"))) {
if (EXCLUDED_TAGS.has(el.tagName.toLowerCase())) continue;
if (el.getAttribute("data-hf-id")) continue; // pinned
walkElements(body, (el) => {
if (EXCLUDED_TAGS.has(el.tagName.toLowerCase())) return;
if (el.getAttribute("data-hf-id")) return; // pinned
el.setAttribute("data-hf-id", mintHfId(el, assigned));
}
});
return wrapped ? document.body.innerHTML || "" : document.toString();
}
+41 -17
View File
@@ -9,9 +9,15 @@
*/
import { parseHTML } from "linkedom";
import { ensureHfIds } from "@hyperframes/parsers/hf-ids";
import { ensureHfIds, isCompositionTemplate } from "@hyperframes/parsers/hf-ids";
import { parseGsapScriptAcornForWrite } from "@hyperframes/core/gsap-parser-acorn";
import { findRoot, getElementStyles, getOwnText, isNewHostBoundary } from "./engine/model.js";
import {
findRoot,
getElementStyles,
getOwnText,
isNewHostBoundary,
querySelectorAllDeep,
} from "./engine/model.js";
import type { HyperFramesElement, SdkDocument } from "./types.js";
// Tags that carry no editable content and must not enter the element tree.
@@ -67,7 +73,7 @@ function buildAnimationIdMap(document: Document): Map<string, string[]> {
if (!selector) continue;
let matches: Element[] = [];
try {
matches = Array.from(document.querySelectorAll(selector));
matches = querySelectorAllDeep(document, selector);
} catch {
continue; // selector not valid for querySelectorAll — skip
}
@@ -96,6 +102,35 @@ export function parsedAnimationIds(script: string): Set<string> {
return new Set(parseLocatedCached(script).map(({ id }) => id));
}
/**
* Build the element list for a parent's children, treating a COMPOSITION
* template (`<template data-composition-id>`) as a TRANSPARENT container: its
* inner elements are spliced in at the template's position, the template
* itself gets no node. This mirrors the studio preview, which unwraps exactly
* that pattern into the served body — so template-based sub-comps expose the
* same elements (and hf-ids) here as the timeline reads from the live preview
* DOM. A plain <template> (runtime clone-source) stays fully excluded: its
* inert interior is not editable and its content is duplicated at runtime.
*/
function buildChildren(
parent: Element,
scopePrefix: string,
animationIdsByHfId: Map<string, string[]>,
): HyperFramesElement[] {
const out: HyperFramesElement[] = [];
for (const child of Array.from(parent.children)) {
if (child.tagName.toLowerCase() === "template") {
if (isCompositionTemplate(child)) {
out.push(...buildChildren(child, scopePrefix, animationIdsByHfId));
}
continue;
}
const built = buildElement(child, scopePrefix, animationIdsByHfId);
if (built) out.push(built);
}
return out;
}
// fallow-ignore-next-line complexity
function buildElement(
el: Element,
@@ -143,11 +178,7 @@ function buildElement(
start !== null && endAttr !== null ? Math.max(0, parseFloat(endAttr) - start) : null;
const trackIndex = trackAttr !== null ? parseInt(trackAttr, 10) : null;
const children: HyperFramesElement[] = [];
for (const child of Array.from(el.children)) {
const built = buildElement(child, childPrefix, animationIdsByHfId);
if (built) children.push(built);
}
const children = buildChildren(el, childPrefix, animationIdsByHfId);
return {
id,
@@ -215,15 +246,8 @@ function extractDuration(doc: Document): number | null {
*/
export function buildRoots(document: Document): HyperFramesElement[] {
const body = document.body;
const roots: HyperFramesElement[] = [];
const animationIdsByHfId = buildAnimationIdMap(document);
if (body) {
for (const child of Array.from(body.children)) {
const built = buildElement(child, "", animationIdsByHfId);
if (built) roots.push(built);
}
}
return roots;
if (!body) return [];
return buildChildren(body, "", buildAnimationIdMap(document));
}
/**
+39 -5
View File
@@ -41,6 +41,42 @@ export function escapeHfId(id: string): string {
return id.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
}
/**
* querySelectorAll that also descends into COMPOSITION `<template>` subtrees
* (`data-composition-id` — the pattern the studio preview unwraps) — linkedom's
* querySelectorAll does not, so template-based sub-comp content would be
* unreachable for resolution/dispatch even though buildRoots models it.
*
* Implemented as a document-order DOM walk (not qsa + append) so duplicate-id
* tiebreaks resolve in TRUE document order — appending template matches after
* all top-level matches would make resolveScoped pick a different duplicate
* than the preview's unwrapped DOM does. Plain templates (runtime clone
* sources) are skipped, matching buildChildren and ensureHfIds.
*
* Throws like querySelectorAll on an invalid selector (Element.matches).
*/
export function querySelectorAllDeep(root: Document | Element, selector: string): Element[] {
const out: Element[] = [];
const start: Element | null =
"body" in root ? ((root as Document).body ?? null) : (root as Element);
const walk = (parent: Element): void => {
for (const child of Array.from(parent.children)) {
if (child.tagName.toLowerCase() === "template") {
if (child.getAttribute("data-composition-id") !== null) walk(child);
continue;
}
if (child.matches(selector)) out.push(child);
walk(child);
}
};
if (start) {
// When rooted at an Element (scoped-path step), the root itself is the
// context, not a candidate — only descendants match, like querySelectorAll.
walk(start);
}
return out;
}
/**
* True when an element lives at the top-level (canonical) scope — i.e. its
* scopedId equals its bare id because no ancestor opens a sub-composition
@@ -75,7 +111,7 @@ export function resolveScoped(document: Document, id: string): Element | null {
// resolution agrees with getElement (scopedId === id wins over document order).
if (parts.length === 1) {
const escaped = escapeHfId(id);
const matches = Array.from(document.querySelectorAll(`[data-hf-id="${escaped}"]`));
const matches = querySelectorAllDeep(document, `[data-hf-id="${escaped}"]`);
if (matches.length > 0) {
return matches.find((el) => isCanonicalScope(el)) ?? matches[0] ?? null;
}
@@ -84,16 +120,14 @@ export function resolveScoped(document: Document, id: string): Element | null {
// (the id studio passes when targeting the sub-comp root). data-hf-id takes
// precedence above; only when no hf-id matches do we treat the bare id as a
// composition id, making comp-ids first-class resolvable addresses.
return document.querySelector(`[data-composition-id="${escaped}"]`);
return querySelectorAllDeep(document, `[data-composition-id="${escaped}"]`)[0] ?? null;
}
let context: Element | Document = document;
for (const part of parts) {
const escaped = escapeHfId(part);
const found: Element | null =
context === document
? (context as Document).querySelector(`[data-hf-id="${escaped}"]`)
: (context as Element).querySelector(`[data-hf-id="${escaped}"]`);
querySelectorAllDeep(context, `[data-hf-id="${escaped}"]`)[0] ?? null;
if (!found) return null;
context = found;
}
+81
View File
@@ -0,0 +1,81 @@
/**
* Template-based sub-comp support — SDK must model elements inside <template>.
*
* The studio preview unwraps <template data-composition-id> content into the
* served body, so the timeline hands edits hf-ids that live inside the
* template in the raw file. Before this support, buildElement excluded the
* whole template subtree: getElements() returned [] for template comps and
* every edit produced a false element_not_found resolver divergence.
*/
import { describe, it, expect } from "vitest";
import { openComposition } from "./session.js";
const TEMPLATE_HTML = `
<template data-composition-id="test-minimal">
<div data-hf-id="hf-a" class="clip" data-start="0" data-end="3">Hello</div>
<div data-hf-id="hf-b" class="clip" data-start="3" data-end="6">World</div>
</template>
`.trim();
const TEMPLATE_UNSTAMPED_HTML = `
<template data-composition-id="test-minimal">
<div class="clip" data-start="0" data-end="3">Hello</div>
</template>
`.trim();
describe("template-based sub-comp compositions", () => {
it("getElements() models the template's inner elements (template itself absent)", async () => {
const comp = await openComposition(TEMPLATE_HTML);
const els = comp.getElements();
expect(els.map((e) => e.id)).toEqual(["hf-a", "hf-b"]);
expect(els.every((e) => e.tag !== "template")).toBe(true);
});
it("mints ids for unstamped template-inner elements on open", async () => {
const comp = await openComposition(TEMPLATE_UNSTAMPED_HTML);
expect(comp.getElements()).toHaveLength(1);
expect(comp.getElements()[0]?.id).toMatch(/^hf-/);
});
it("getElement resolves a template-inner id", async () => {
const comp = await openComposition(TEMPLATE_HTML);
expect(comp.getElement("hf-a")?.text).toBe("Hello");
});
it("setTiming on a template-inner element mutates and serializes inside the template", async () => {
const comp = await openComposition(TEMPLATE_HTML);
comp.setTiming("hf-a", { start: 1.5 });
const out = comp.serialize();
expect(out).toContain("<template");
// the mutated start must be INSIDE the template wrapper
const tpl = out.slice(out.indexOf("<template"), out.indexOf("</template>"));
expect(tpl).toContain('data-start="1.5"');
expect(comp.getElement("hf-a")?.start).toBe(1.5);
});
it("timed template-inner elements carry start/duration snapshots", async () => {
const comp = await openComposition(TEMPLATE_HTML);
const a = comp.getElement("hf-a");
expect(a?.start).toBe(0);
expect(a?.duration).toBe(3);
});
it("plain <template> (runtime clone-source) stays fully excluded", async () => {
const comp = await openComposition(
`<div data-hf-id="hf-stage" data-hf-root>x</div><template><li data-hf-id="hf-clone">item</li></template>`,
);
expect(comp.getElements().map((e) => e.id)).toEqual(["hf-stage"]);
expect(comp.getElement("hf-clone")).toBeNull();
});
it("duplicate hf-id resolves in true document order (template-inner first when it comes first)", async () => {
// A comp-template-inner element EARLIER in the document and a top-level
// element LATER share an id (copy-paste drift). The preview's unwrapped
// DOM resolves the first-in-document copy; the SDK must agree.
const comp = await openComposition(
`<template data-composition-id="t"><div data-hf-id="hf-dup" data-start="1" data-end="2">tpl</div></template><div data-hf-id="hf-dup" data-start="5" data-end="6">top</div>`,
);
expect(comp.getElement("hf-dup")?.text).toBe("tpl");
});
});
@@ -2,7 +2,7 @@ import { describe, it, expect, afterEach } from "vitest";
import { mkdtempSync, writeFileSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { persistHfIdsIfNeeded } from "./hfIdPersist.js";
import { persistHfIdsIfNeeded, stampFileHfIds } from "./hfIdPersist.js";
describe("persistHfIdsIfNeeded", () => {
const tmpDirs: string[] = [];
@@ -69,3 +69,47 @@ describe("persistHfIdsIfNeeded", () => {
expect(readFileSync(file, "utf-8")).toBe(newer);
});
});
describe("stampFileHfIds", () => {
const tmpDirs: string[] = [];
afterEach(() => {
for (const d of tmpDirs) rmSync(d, { recursive: true, force: true });
tmpDirs.length = 0;
});
function tmpFile(content: string): string {
const dir = mkdtempSync(join(tmpdir(), "hfid-stamp-test-"));
tmpDirs.push(dir);
const file = join(dir, "scene.html");
writeFileSync(file, content, "utf-8");
return file;
}
it("stamps ids and writes back through the same fd", () => {
const file = tmpFile(`<div class="clip" data-start="0" data-end="3">Hi</div>`);
const returned = stampFileHfIds(file);
expect(returned).toContain('data-hf-id="hf-');
expect(readFileSync(file, "utf-8")).toBe(returned);
});
it("does not rewrite an already-stamped file", () => {
const file = tmpFile(`<div data-hf-id="hf-keep">Hi</div>`);
const before = readFileSync(file, "utf-8");
const returned = stampFileHfIds(file);
expect(returned).toContain('data-hf-id="hf-keep"');
expect(readFileSync(file, "utf-8")).toBe(before); // byte-identical, no write
});
it("returns null for a missing file", () => {
const dir = mkdtempSync(join(tmpdir(), "hfid-stamp-test-"));
tmpDirs.push(dir);
expect(stampFileHfIds(join(dir, "nope.html"))).toBeNull();
});
it("returns null for a directory", () => {
const dir = mkdtempSync(join(tmpdir(), "hfid-stamp-test-"));
tmpDirs.push(dir);
expect(stampFileHfIds(dir)).toBeNull();
});
});
@@ -1,5 +1,14 @@
import { ensureHfIds } from "@hyperframes/parsers/hf-ids";
import { readFileSync, writeFileSync } from "node:fs";
import {
closeSync,
constants,
fstatSync,
ftruncateSync,
openSync,
readFileSync,
writeFileSync,
writeSync,
} from "node:fs";
/**
* Ensure `html` has `data-hf-id` attributes minted, and write the result back
@@ -36,3 +45,60 @@ export function persistHfIdsIfNeeded(filePath: string, html: string): string {
}
return normalized;
}
function openNoFollow(filePath: string, flags: number): number | null {
// O_NOFOLLOW is undefined on Windows; opening without it is the platform norm there.
const noFollow = constants.O_NOFOLLOW ?? 0;
try {
return openSync(filePath, flags | noFollow);
} catch {
return null;
}
}
/**
* Read `filePath`, mint any missing `data-hf-id`s, write the stamped content
* back if new ids were added, and return the stamped content — all through ONE
* file descriptor. Unlike the check-path / read-path / write-path sequence a
* route handler would otherwise do, the validation (fstat), read, and write
* all target the same open inode, so the path cannot be swapped (e.g. for a
* symlink) between validation and write (CodeQL js/file-system-race).
*
* Falls back to read-only stamping when the file isn't writable (read-only
* fs, sandbox) — serving stamped content without persisting is still correct;
* ids are content-keyed so the SDK mints the same ones from the same bytes.
*
* Returns null when the file is missing, unreadable, or not a regular file.
*
* Best-effort on concurrent saves: a user save landing between the read and
* the write below can still be overwritten (same microsecond window
* persistHfIdsIfNeeded documents) — the next save simply re-persists.
*/
export function stampFileHfIds(filePath: string): string | null {
let fd = openNoFollow(filePath, constants.O_RDWR);
let writable = true;
if (fd === null) {
fd = openNoFollow(filePath, constants.O_RDONLY);
writable = false;
}
if (fd === null) return null;
try {
if (!fstatSync(fd).isFile()) return null;
const html = readFileSync(fd, "utf-8");
const normalized = ensureHfIds(html);
// Attribute count, not string equality — linkedom serialization normalizes
// quote style/whitespace even when no ids were minted (see persistHfIdsIfNeeded).
const idsBefore = (html.match(/\bdata-hf-id=/g) ?? []).length;
const idsAfter = (normalized.match(/\bdata-hf-id=/g) ?? []).length;
if (writable && idsAfter > idsBefore) {
ftruncateSync(fd, 0);
writeSync(fd, normalized, 0, "utf-8");
}
return normalized;
} catch (err) {
console.warn("[hyperframes] stampFileHfIds: failed to stamp ids:", err);
return null;
} finally {
closeSync(fd);
}
}
@@ -468,6 +468,18 @@ describe("T7 — data-hf-id targeting (spec for R1)", () => {
expect(html).toContain('data-hf-id="hf-x7k2"');
});
it("resolves a data-hf-id inside a NESTED template (matches SDK deep resolution)", () => {
// ensureHfIds and the SDK descend nested composition templates, so ids
// exist at any template depth; the server-side patch path must resolve
// them too or those ops silently no-op while the SDK reports parity.
const source = `<template data-composition-id="a"><div>x</div><template data-composition-id="b"><p data-hf-id="hf-deep" data-start="0">deep</p></template></template>`;
const { html, matched } = patchElementInHtml(source, { hfId: "hf-deep" }, [
{ type: "attribute", property: "start", value: "2.5" },
]);
expect(matched).toBe(true);
expect(html).toContain('data-start="2.5"');
});
it("hfId lookup falls through to selector when hfId is not found in the document", () => {
const source = `<h1 class="headline" style="color: red">Hello</h1>`;
const { html, matched } = patchElementInHtml(
@@ -58,10 +58,14 @@ function querySelectorAllWithTemplates(root: Document | Element, selector: strin
// querySelectorAll doesn't traverse <template> content in linkedom.
// Search directly on each template element (NOT .content — removing from
// .content's DocumentFragment doesn't update the serialized output).
// Recurse so NESTED templates resolve too — ensureHfIds and the SDK's
// querySelectorAllDeep descend nested composition templates, so ids exist at
// any template depth; a one-level search here would silently no-op
// server-side ops on those ids while the SDK resolves them.
const templates = Array.from(root.querySelectorAll("template"));
for (const tmpl of templates) {
const inner = tmpl.querySelectorAll(selector);
if (inner.length > 0) return Array.from(inner);
const inner = querySelectorAllWithTemplates(tmpl, selector);
if (inner.length > 0) return inner;
}
return [];
}
@@ -191,9 +191,18 @@ const NON_RENDERED_TAGS = new Set(["SCRIPT", "STYLE", "LINK", "META", "TEMPLATE"
* already render correctly are untouched.
*/
function promoteTemplateCompositionId(rawComp: string, body: Element): void {
const templateCompositionId = rawComp.match(
/<template[^>]*\sdata-composition-id\s*=\s*["']([^"']+)["']/i,
)?.[1];
// Two-step match instead of one `[^>]*\s…` regex: the single-pattern form
// backtracks polynomially on crafted input (CodeQL js/polynomial-redos).
// Step 1 grabs each <template …> open tag (linear); step 2 finds the attr
// within that short tag text.
let templateCompositionId: string | undefined;
for (const tag of rawComp.matchAll(/<template\b[^>]*/gi)) {
const id = /\bdata-composition-id\s*=\s*["']([^"']+)["']/i.exec(tag[0] ?? "")?.[1];
if (id) {
templateCompositionId = id;
break;
}
}
if (!templateCompositionId) return;
if (body.querySelector("[data-composition-id]")) return;
@@ -241,11 +250,15 @@ export function buildSubCompositionHtml(
compPath: string,
runtimeUrl: string,
baseHref?: string,
rawOverride?: string,
): string | null {
const compFile = join(projectDir, compPath);
if (!existsSync(compFile)) return null;
const rawComp = readFileSync(compFile, "utf-8");
// rawOverride lets the preview route thread the hf-id-stamped content in
// directly, so the build uses pinned ids even when the persist-to-disk write
// was skipped (read-only fs, concurrent-save TOCTOU guard).
const rawComp = rawOverride ?? readFileSync(compFile, "utf-8");
let compHeadContent = "";
let rewrittenContent: string;
@@ -441,4 +441,95 @@ describe("hf-id surfacing in preview route", () => {
expect(servedIds.length).toBeGreaterThanOrEqual(2);
expect(servedIds).toEqual(diskIds);
});
it("sub-comp route writes data-hf-id back to disk on first serve", async () => {
const { readFileSync } = await import("node:fs");
const projectDir = createProjectDir();
const compPath = join(projectDir, "scene.html");
writeFileSync(compPath, `<div class="clip" data-start="0" data-end="3">Hi</div>`);
const app = new Hono();
registerPreviewRoutes(app, createAdapter(projectDir));
const res = await app.request("http://localhost/projects/demo/preview/comp/scene.html");
expect(res.status).toBe(200);
expect(readFileSync(compPath, "utf-8")).toContain('data-hf-id="hf-');
});
it("sub-comp served ids equal disk ids even when relative asset paths are rewritten", async () => {
// Regression guard for the setTiming element_not_found divergence class:
// the sub-comp route rewrites relative src/href BEFORE minting, so an
// element with a relative asset path got a preview-only id that existed
// nowhere in the raw file. Persisting ids from the RAW file first pins
// them; the rewrite then carries the pinned ids through unchanged.
const { readFileSync } = await import("node:fs");
const projectDir = createProjectDir();
const compPath = join(projectDir, "scene.html");
writeFileSync(
compPath,
`<div class="clip" data-start="0" data-end="3"><img src="assets/logo.png"></div>`,
);
const app = new Hono();
registerPreviewRoutes(app, createAdapter(projectDir));
const res = await app.request("http://localhost/projects/demo/preview/comp/scene.html");
expect(res.status).toBe(200);
const servedIds = [...(await res.text()).matchAll(/data-hf-id="(hf-[a-z0-9]+)"/g)]
.map((m) => m[1])
.sort();
const diskIds = [...readFileSync(compPath, "utf-8").matchAll(/data-hf-id="(hf-[a-z0-9]+)"/g)]
.map((m) => m[1])
.sort();
expect(servedIds.length).toBeGreaterThanOrEqual(2); // div + img
expect(servedIds).toEqual(diskIds);
});
it("template-based sub-comp: inner ids persist to disk and match the served (unwrapped) ids", async () => {
const { readFileSync } = await import("node:fs");
const projectDir = createProjectDir();
const compPath = join(projectDir, "test-minimal.html");
writeFileSync(
compPath,
`<template data-composition-id="test-minimal"><div class="clip" data-start="0" data-end="3">Hello</div><div class="clip" data-start="3" data-end="6">World</div></template>`,
);
const app = new Hono();
registerPreviewRoutes(app, createAdapter(projectDir));
const res = await app.request("http://localhost/projects/demo/preview/comp/test-minimal.html");
expect(res.status).toBe(200);
const servedIds = [...(await res.text()).matchAll(/data-hf-id="(hf-[a-z0-9]+)"/g)].map(
(m) => m[1],
);
const diskIds = [
...readFileSync(compPath, "utf-8").matchAll(/data-hf-id="(hf-[a-z0-9]+)"/g),
].map((m) => m[1]);
expect(diskIds.length).toBe(2);
for (const id of diskIds) expect(servedIds).toContain(id);
});
it("sub-comp route does NOT rewrite a non-HTML file on disk (GET must not corrupt assets)", async () => {
const { readFileSync } = await import("node:fs");
const projectDir = createProjectDir();
const svgPath = join(projectDir, "logo.svg");
const svgBytes = `<?xml version="1.0"?><svg xmlns="http://www.w3.org/2000/svg"><path d="M0 0"/></svg>`;
writeFileSync(svgPath, svgBytes);
const app = new Hono();
registerPreviewRoutes(app, createAdapter(projectDir));
await app.request("http://localhost/projects/demo/preview/comp/logo.svg");
// Whatever the route serves, a GET must leave the file byte-identical.
expect(readFileSync(svgPath, "utf-8")).toBe(svgBytes);
});
it("sub-comp route does NOT persist ids inside a plain <template> (runtime clone-source)", async () => {
const { readFileSync } = await import("node:fs");
const projectDir = createProjectDir();
const compPath = join(projectDir, "clones.html");
writeFileSync(
compPath,
`<div class="clip" data-start="0" data-end="3">stage</div><template><li class="row">item</li></template>`,
);
const app = new Hono();
registerPreviewRoutes(app, createAdapter(projectDir));
const res = await app.request("http://localhost/projects/demo/preview/comp/clones.html");
expect(res.status).toBe(200);
const disk = readFileSync(compPath, "utf-8");
expect(disk).toMatch(/<div[^>]*data-hf-id/); // stage div stamped
expect(disk).not.toMatch(/<li[^>]*data-hf-id/); // clone-source untouched
});
});
+38 -3
View File
@@ -12,7 +12,7 @@ import {
STUDIO_MOTION_PATH,
} from "../helpers/studioMotionRenderScript.js";
import { ensureHfIds } from "@hyperframes/parsers/hf-ids";
import { persistHfIdsIfNeeded } from "../helpers/hfIdPersist.js";
import { persistHfIdsIfNeeded, stampFileHfIds } from "../helpers/hfIdPersist.js";
const PROJECT_SIGNATURE_META = "hyperframes-project-signature";
const GSAP_CDN_VERSION = "3.15.0";
@@ -320,6 +320,29 @@ export function registerPreviewRoutes(api: Hono, adapter: StudioApiAdapter): voi
}
});
/**
* Pin hf-ids to the RAW sub-comp file before the build pipeline mutates
* attributes (rewriteRelativePaths etc.) — minting is content-keyed over
* attrs, so stamping only AFTER the rewrite mints preview-only ids that
* exist nowhere in the source. Pinned ids ride through the rewrite
* unchanged, keeping the served DOM, the disk file, and the studio SDK
* session in one id space. Mirrors the main-preview route's
* persistHfIdsIfNeeded call.
*
* Gated to composition files: the wildcard route serves any project path,
* and stamping a non-HTML file (SVG, etc.) would corrupt it on disk.
*
* Returns the stamped content to thread into the build (so served ids match
* the mint even when the disk write is skipped — read-only fs), undefined
* for non-HTML paths, or null when the file vanished after the caller's
* stat. stampFileHfIds does its validation, read, and write through one
* file descriptor, so there is no check/read/write path gap to race.
*/
function pinSubCompHfIds(compFile: string, compPath: string): string | undefined | null {
if (!/\.html?$/i.test(compPath)) return undefined;
return stampFileHfIds(compFile);
}
// Sub-composition preview
api.get("/projects/:id/preview/comp/*", async (c) => {
const project = await adapter.resolveProject(c.req.param("id"));
@@ -334,14 +357,26 @@ export function registerPreviewRoutes(api: Hono, adapter: StudioApiAdapter): voi
return c.text("not found", 404);
}
const etag = `"comp:${compPath}:${signature}"`;
// "v2" salts the etag for the hf-id-pinning change below: a client holding
// a pre-pin cached response (preview-only ids, unstamped disk file) must
// not revalidate to a 304 that skips the pin.
const etag = `"comp:v2:${compPath}:${signature}"`;
const ifNoneMatch = c.req.header("If-None-Match");
if (ifNoneMatch === etag) {
return new Response(null, { status: 304, headers: previewCacheHeaders(etag) });
}
const stamped = pinSubCompHfIds(compFile, compPath);
if (stamped === null) return c.text("not found", 404); // file removed between stat and read
const baseHref = `/api/projects/${project.id}/preview/`;
let html = buildSubCompositionHtml(project.dir, compPath, adapter.runtimeUrl, baseHref);
let html = buildSubCompositionHtml(
project.dir,
compPath,
adapter.runtimeUrl,
baseHref,
stamped,
);
if (!html) return c.text("not found", 404);
html = ensureHfIds(await transformPreviewHtml(html, adapter, project, compPath));
return c.html(
@@ -98,6 +98,23 @@ describe("A. Flag gating", () => {
expect(trackedEvents.filter((e) => e.event === "sdk_resolver_shadow")).toHaveLength(0);
});
it("A2c: empty session → ONE tagged session_empty event per session, no attempt", async () => {
mockFlags.STUDIO_SDK_RESOLVER_SHADOW_ENABLED = true;
flushAttemptCounts(); // drain counts left by earlier tests
const session = await openComposition("<!DOCTYPE html><html><body></body></html>");
const ops: PatchOperation[] = [{ type: "inline-style", property: "color", value: "blue" }];
runResolverShadow(session, "hf-anything", ops);
runResolverShadow(session, "hf-other", ops); // repeat edits do not re-emit
const events = trackedEvents.filter((e) => e.event === "sdk_resolver_shadow");
// The modeling gap stays VISIBLE (silence would blind the tripwire to the
// exact class that exposed the template-comp bug) but is distinguishable
// and rate-limited to once per session instance.
expect(events).toHaveLength(1);
expect(events[0]?.props.sessionEmpty).toBe(true);
expect(JSON.stringify(events[0]?.props.mismatches)).toContain("session_empty");
expect(flushAttemptCounts()).toBeNull(); // can't cut over → not in the denominator
});
it("A3: shadow depends ONLY on shadow flag, not on STUDIO_SDK_CUTOVER_ENABLED", async () => {
// The mock always returns STUDIO_SDK_CUTOVER_ENABLED=false. Use a divergence
// (poisoned session) so the flag-on case emits; flag-off must stay silent.
@@ -268,7 +285,12 @@ describe("C. Resolver-parity detection", () => {
it("C8 sourceHfIdCount: emitted element_not_found carries source occurrence count", async () => {
mockFlags.STUDIO_SDK_RESOLVER_SHADOW_ENABLED = true;
const session = { getElement: () => null, getElements: () => [] } as unknown as Composition;
// One unrelated element so the session isn't "empty" (empty sessions are
// skipped as structural modeling gaps) — it just can't resolve hf-dup.
const session = {
getElement: () => null,
getElements: () => [{ id: "hf-other" }],
} as unknown as Composition;
// id present twice in source (duplicate-id ambiguity) but absent from session
const source = `<div data-hf-id="hf-dup">a</div><div data-hf-id="hf-dup">b</div>`;
runResolverShadow(
@@ -282,7 +304,10 @@ describe("C. Resolver-parity detection", () => {
it("C8 sourceLooseMatchOnly: hfId matches source only as plain text, not a data-hf-id attribute", async () => {
mockFlags.STUDIO_SDK_RESOLVER_SHADOW_ENABLED = true;
const session = { getElement: () => null, getElements: () => [] } as unknown as Composition;
const session = {
getElement: () => null,
getElements: () => [{ id: "hf-other" }],
} as unknown as Composition;
// "hf-widget" appears only inside a class name, never as data-hf-id="hf-widget".
const source = `<div class="hf-widget-container">no attribute match here</div>`;
runResolverShadow(
@@ -436,7 +461,7 @@ describe("F. recordResolverParity", () => {
expect(ev?.sourceHfIdCount).toBeUndefined();
});
it("fails open: a readSource error still emits (no suppression)", async () => {
it("fails open: a readSource error still emits (no suppression), tagged sourceReadFailed", async () => {
mockFlags.STUDIO_SDK_RESOLVER_SHADOW_ENABLED = true;
const session = await openComposition(BASE_HTML);
await recordResolverParity(session, "hf-missing", "setTiming", () =>
@@ -445,6 +470,31 @@ describe("F. recordResolverParity", () => {
const ev = lastShadow();
expect(ev?.mismatchCount).toBe(1);
expect(ev?.sourceHfIdCount).toBeUndefined();
// Distinguishes "reader threw" from "no reader wired" — every wild emission
// of the setTiming class had an absent sourceHfIdCount and the two cases
// were indistinguishable in telemetry.
expect(ev?.sourceReadFailed).toBe(true);
});
it("does not tag sourceReadFailed when no reader is supplied", async () => {
mockFlags.STUDIO_SDK_RESOLVER_SHADOW_ENABLED = true;
const session = await openComposition(BASE_HTML);
await recordResolverParity(session, "hf-missing", "setTiming");
expect(lastShadow()?.sourceReadFailed).toBeUndefined();
});
it("empty session → ONE tagged session_empty event, no attempt, no element_not_found", async () => {
mockFlags.STUDIO_SDK_RESOLVER_SHADOW_ENABLED = true;
flushAttemptCounts(); // drain any counts left by earlier tests
const session = await openComposition("<!DOCTYPE html><html><body></body></html>");
await recordResolverParity(session, "hf-anything", "setTiming");
await recordResolverParity(session, "hf-other", "setTiming"); // no re-emit
const events = trackedEvents.filter((e) => e.event === "sdk_resolver_shadow");
expect(events).toHaveLength(1);
expect(events[0]?.props.sessionEmpty).toBe(true);
expect(JSON.stringify(events[0]?.props.mismatches)).toContain("session_empty");
expect(JSON.stringify(events[0]?.props.mismatches)).not.toContain("element_not_found");
expect(flushAttemptCounts()).toBeNull(); // can't cut over → not in the denominator
});
it("tags sourceLooseMatchOnly when hfId matches source only as plain text, not a data-hf-id attribute", async () => {
+41 -1
View File
@@ -24,7 +24,12 @@ import { trackStudioEvent, flushViaBeacon } from "./studioTelemetry";
// ─── Types ────────────────────────────────────────────────────────────────────
export interface SdkResolverMismatch {
kind: "element_not_found" | "value_mismatch" | "dispatch_error" | "animation_not_found";
kind:
| "element_not_found"
| "value_mismatch"
| "dispatch_error"
| "animation_not_found"
| "session_empty";
hfId?: string;
animationId?: string;
property?: string;
@@ -347,6 +352,34 @@ function redactMismatches(mismatches: SdkResolverMismatch[]): SdkResolverMismatc
* (see below). The session is shared with the cutover path, so it MUST end the
* call exactly as it started.
*/
// Sessions whose empty-session modeling gap has already been reported — one
// event per session instance, not one per edit (the per-edit storm is noise;
// the EXISTENCE of the gap is the signal).
const emptySessionReported = new WeakSet<Composition>();
/**
* An empty session structurally cannot resolve ANY id — a modeling gap (empty
* file, comp shape the SDK can't parse into elements), not a resolver
* divergence, and it can't cut over either, so it stays out of the attempt
* denominator. But silence would blind the tripwire to exactly the class that
* exposed the template-comp bug — so emit ONE distinguishable `session_empty`
* event per session, then skip. Returns true when the caller should skip.
*/
function reportEmptySession(session: Composition, opLabel: string): boolean {
if (session.getElements().length !== 0) return false;
if (!emptySessionReported.has(session)) {
emptySessionReported.add(session);
trackStudioEvent("sdk_resolver_shadow", {
opLabel,
sessionEmpty: true,
sessionElementCount: 0,
mismatchCount: 1,
mismatches: JSON.stringify([{ kind: "session_empty" } satisfies SdkResolverMismatch]),
});
}
return true;
}
export function runResolverShadow(
session: Composition,
hfId: string | null | undefined,
@@ -356,6 +389,7 @@ export function runResolverShadow(
if (!STUDIO_SDK_RESOLVER_SHADOW_ENABLED) return;
if (!hfId) return;
try {
if (reportEmptySession(session, "dom-edit")) return;
recordAttempt("dom-edit");
const mismatches = sdkResolverShadowCheck(session, hfId, ops, sourceContent);
// Emit only on divergence — parity is silent, matching recordResolverParity
@@ -409,6 +443,7 @@ export async function recordResolverParity(
if (!STUDIO_SDK_RESOLVER_SHADOW_ENABLED) return;
if (!session || !hfId) return;
try {
if (reportEmptySession(session, opLabel)) return;
recordAttempt(opLabel);
if (resolveSnapshot(session, hfId)) return; // resolves — parity, nothing to record
// Capture BEFORE any await: this call is fire-and-forget (`void recordResolverParity(...)`)
@@ -419,11 +454,13 @@ export async function recordResolverParity(
const sessionElementCount = session.getElements().length;
// Cheap check passed above, so the source read only runs on a real divergence.
let source: string | undefined;
let sourceReadFailed = false;
if (readSource) {
try {
source = await readSource();
} catch {
source = undefined; // fail-open: a read error must not drop a real divergence
sourceReadFailed = true;
}
}
// Runtime-generated node the static parse can't model — suppress (mirrors the dom-edit path).
@@ -444,6 +481,9 @@ export async function recordResolverParity(
// Lets telemetry consumers filter this cohort without parsing the
// sourceHfIdCount comment above.
...(strictCount === 0 ? { sourceLooseMatchOnly: true } : {}),
// The reader was wired but threw — distinguishes "read failed, emitted
// fail-open without the suppression/count checks" from "no reader wired".
...(sourceReadFailed ? { sourceReadFailed: true } : {}),
mismatchCount: 1,
mismatches: JSON.stringify([
{ kind: "element_not_found", hfId } satisfies SdkResolverMismatch,