fix(producer): localize remote <img> sources + await image readiness (#1197)

* fix(producer): localize remote <img> sources + await image readiness

Producer's frame-capture has `pollVideosReady` (waits readyState >= 2 for
every <video>) but no equivalent for <img>. Combined with htmlCompiler's
`collectExternalAssets` explicitly skipping http(s) URLs (line 805-806),
agent-pipeline-generated compositions (astral / daphne / hyperion
multi-v2 outputs with raw S3 <img src>) reach Chrome with a network
dependency that races the readiness gate AND can be evicted mid-render.
Either path produces blank-frame flicker.

Reproduction (02_kobe agent output, 42s render @ 30fps): scene_02's
remote S3 background-image painted from t=7.0s, vanished at t=10.5s
(frame size 139KB vs 700-940KB neighbors), back at t=11.0s. GSAP
timeline said opacity:1 throughout — Chrome simply didn't have the
pixels.

Two-layer fix:

1. **Producer** — `localizeRemoteImageSources` in `htmlCompiler.ts`
   mirrors the existing `localizeRemoteMediaSources` (video/audio) +
   `localizeRemoteFontFaces` pattern, reusing `downloadAndRewriteUrls`
   and the `_remote_media/` subdir. Wired into `compileForRender`
   between the media and font localize steps. Once the file is local,
   Chrome's image cache is bounded by disk reads, not S3 latency.

2. **Engine** — `pollImagesReady` + `decodeAllImages` helpers in
   `frameCapture.ts` parallel to `pollVideosReady`. Waits for every
   `<img>` (skipping data: URIs) to have `complete && naturalWidth > 0`,
   then forces GPU upload via `img.decode()`. Called from both the
   classic-xvfb path and the BeginFrame path after their respective
   video readiness checks. Defense-in-depth — Layer 1 closes the
   symptom for current+future agent-pipeline outputs; Layer 2 protects
   any future code path that leaves a remote URL in place.

Tests: 7 new cases in `htmlCompiler.test.ts` covering happy-path
rewrite, 404 fallback, dedup of duplicate URLs, non-HTTP and data:
URI passthrough, both quote styles, and the agent-pipeline shape where
`src` is not the first attribute. All pass alongside the existing 56
htmlCompiler tests.

* fix(producer): scope remote-img regex to real src; correct stale comments

Review follow-ups on the remote-<img> localization fix:

- Tighten REMOTE_IMG_TAG_RE with a (?<![\w-]) lookbehind so it matches a
  real `src` attribute only. The previous `\bsrc` also matched `data-src`
  (and `data-*-src`) lazy-loader placeholders, which would download/rewrite
  a URL the render never paints. Added a regression test; `srcset` stays
  excluded by the `\s*=` requirement.
- Fix comments that claimed frameCapture has "no pollImagesReady analog" —
  this PR adds exactly that, so the docstrings were self-contradictory.
  Reframed localization as the primary fix and pollImagesReady as the
  defense-in-depth layer, and documented the <img src>-only scope
  (srcset / <picture> / SVG <image> / CSS background-image are follow-ups).

Verified locally end-to-end on the 02_kobe repro: all 4 remote S3 <img>
URLs localize to _remote_media/, the render completes, and the frame at
t~10.5s that was a 139KB blank in the broken render now paints the trophy
background in every native-fps frame. htmlCompiler.test.ts 64 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(engine): pollImagesReady broken-image escape + skip decode on in-flight

Addresses two real bugs Magi caught in review on hf#1197:

1. pollImagesReady would spin the full pageReadyTimeout (45s default)
   for any <img> that settled with an error — Chrome marks 404 / decode
   failure / CORS rejection with (complete=true, naturalWidth=0), and
   the previous predicate `complete && naturalWidth > 0` returned false
   for those, so the poll ran to timeout. This is the HTMLImageElement
   equivalent of pollVideosReady's `ve.error` early-exit. Add a
   `complete && naturalWidth === 0` branch that treats settled-with-
   error as done — waiting won't make it load. Particularly relevant
   because localizeRemoteImageSources falls back to the original URL on
   download failure; that failed URL is now hit by a 45s stall instead
   of the broken-image marker rendering immediately.

2. decodeAllImages called img.decode() on every image, including those
   still in flight after pollImagesReady timed out. Per the WHATWG spec,
   decode() on a loading image awaits the fetch — never resolving
   until the network completes or puppeteer's evaluate timeout fires
   and throws an uncaught error that aborts the render. Pre-filter to
   only call decode() on images that successfully loaded.

Test coverage: new frameCapture-pollImagesReady.test.ts with 8 cases
covering empty docs, all-loaded, broken (complete + naturalWidth=0),
data: URI, empty src, in-flight → resolves, in-flight → timeout, and
the mixed batch. The broken-image test explicitly asserts elapsed <
500ms on a 1000ms timeout — guards against the regression Magi flagged.

* docs(engine): clarify decodeAllImages prevents init race, not eviction

Vai correctly noted that decode() forces initial GPU upload but does not
prevent Chrome from evicting decoded pixels mid-render. The producer-side
localizeRemoteImageSources is what bounds the eviction risk (local
file-server paging vs S3 re-fetch). Comment updated to reflect that split
of responsibilities.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
James Russo
2026-06-04 03:28:20 -04:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 2be41937a9
commit 72c461d86a
4 changed files with 481 additions and 1 deletions
@@ -0,0 +1,153 @@
/**
* Tests for `pollImagesReady` — the image-side analog of `pollVideosReady`.
*
* Critical contract:
* - Successfully loaded image (complete=true, naturalWidth>0) → settled.
* - Broken / 404 image (complete=true, naturalWidth=0) → settled.
* This mirrors `pollVideosReady`'s `ve.error` early-exit. Without it,
* the htmlCompiler 404-fallback path (where a remote <img> URL failed
* to download and the original URL is preserved) would silently spin
* the full `pageReadyTimeout` budget waiting for an image that will
* never load — a 45 s regression vs the pre-PR behavior.
* - In-flight image (complete=false) → still waiting.
* - data: URI src → settled (no network fetch).
* - Empty src → settled (nothing to load).
*/
import { describe, expect, it } from "vitest";
import type { Page } from "puppeteer-core";
import { pollImagesReady } from "./frameCapture.js";
interface ImageSpec {
src: string;
complete: boolean;
naturalWidth: number;
}
// Mock `page` whose `evaluate(fn)` invokes `fn` with a Node-side `document`
// mock that returns synthetic image objects matching the spec. Snapshots the
// image state at evaluate-time, so callers can mutate `imgs` between polls
// to simulate progressive load completion.
function makeMockPage(imgs: () => ImageSpec[]): Page {
return {
evaluate: async (fn: () => unknown) => {
const snapshot = imgs();
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const prevDoc = (globalThis as any).document;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).document = {
querySelectorAll: () =>
snapshot.map((spec) => ({
getAttribute: (attr: string) => (attr === "src" ? spec.src : null),
complete: spec.complete,
naturalWidth: spec.naturalWidth,
})),
};
try {
return await fn();
} finally {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).document = prevDoc;
}
},
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any as Page;
}
describe("pollImagesReady", () => {
it("resolves immediately when there are no <img> elements", async () => {
const page = makeMockPage(() => []);
const result = await pollImagesReady(page, 1000, 10);
expect(result).toBe(true);
});
it("resolves immediately when every image has loaded successfully", async () => {
const page = makeMockPage(() => [
{ src: "/a.png", complete: true, naturalWidth: 100 },
{ src: "/b.png", complete: true, naturalWidth: 200 },
]);
const result = await pollImagesReady(page, 1000, 10);
expect(result).toBe(true);
});
it("treats a broken image (complete=true, naturalWidth=0) as settled — does NOT wait for timeout", async () => {
// This is the bug Magi flagged. Without the broken-image escape, this
// test would block the full 1000ms timeout and return false.
const page = makeMockPage(() => [
{ src: "/a.png", complete: true, naturalWidth: 100 },
{ src: "https://broken.example.com/404.png", complete: true, naturalWidth: 0 },
]);
const t0 = Date.now();
const result = await pollImagesReady(page, 1000, 10);
const elapsed = Date.now() - t0;
expect(result).toBe(true);
// Must resolve fast — well under the 1000ms timeout.
expect(elapsed).toBeLessThan(500);
});
it("treats a data: URI src as settled regardless of complete/naturalWidth", async () => {
const page = makeMockPage(() => [
{ src: "data:image/svg+xml,%3Csvg/%3E", complete: false, naturalWidth: 0 },
]);
const result = await pollImagesReady(page, 1000, 10);
expect(result).toBe(true);
});
it("treats an empty src as settled (nothing to load)", async () => {
const page = makeMockPage(() => [{ src: "", complete: false, naturalWidth: 0 }]);
const result = await pollImagesReady(page, 1000, 10);
expect(result).toBe(true);
});
it("waits for an in-flight image and resolves once it completes", async () => {
// Image starts in-flight, then completes after ~50ms.
let started = false;
const startTime = { value: 0 };
const page = makeMockPage(() => {
if (!started) {
started = true;
startTime.value = Date.now();
}
const elapsed = Date.now() - startTime.value;
const loaded = elapsed >= 50;
return [{ src: "/slow.png", complete: loaded, naturalWidth: loaded ? 100 : 0 }];
});
const result = await pollImagesReady(page, 1000, 10);
expect(result).toBe(true);
});
it("times out and returns false when an in-flight image never resolves", async () => {
// Image stays in-flight (complete=false) for the full timeout.
const page = makeMockPage(() => [
{ src: "/never-loads.png", complete: false, naturalWidth: 0 },
]);
const result = await pollImagesReady(page, 100, 10);
expect(result).toBe(false);
});
it("mixed batch: loaded + broken + data: + in-flight → waits only on the in-flight image", async () => {
let resolved = false;
const start = Date.now();
const page = makeMockPage(() => {
const elapsed = Date.now() - start;
if (elapsed >= 30) resolved = true;
return [
{ src: "/loaded.png", complete: true, naturalWidth: 800 },
{ src: "https://broken.example.com/404.jpg", complete: true, naturalWidth: 0 },
{ src: "data:image/svg+xml,abc", complete: false, naturalWidth: 0 },
{
src: "/in-flight.png",
complete: resolved,
naturalWidth: resolved ? 200 : 0,
},
];
});
const t0 = Date.now();
const result = await pollImagesReady(page, 1000, 10);
const elapsed = Date.now() - t0;
expect(result).toBe(true);
// Should wait roughly for the in-flight image to settle (~30ms) — not the
// full timeout. Allow generous slack for CI scheduler jitter.
expect(elapsed).toBeLessThan(500);
});
});
@@ -550,6 +550,89 @@ async function pollVideosReady(
return check(); return check();
} }
// Wait for every `<img>` with a non-`data:` src to have settled — either
// successfully loaded (`complete && naturalWidth > 0`) or failed with a
// broken-image marker (`complete && naturalWidth === 0`, the HTMLImageElement
// equivalent of HTMLMediaElement.error). htmlCompiler localises remote `<img>`
// URLs to the local file server before this point, so in practice this polls
// for the local fetch to land — but the guard is a defensive net so that any
// future composition path that leaves a remote URL in place won't capture
// frames before the pixels arrive. Mirrors `pollVideosReady` for parity with
// the video-side readiness contract (videos exit-early on `ve.error`; images
// exit-early on `complete && naturalWidth === 0`).
/** @internal exported for unit testing only */
export async function pollImagesReady(
page: Page,
timeoutMs: number,
intervalMs: number = 100,
): Promise<boolean> {
const check = async (): Promise<boolean> => {
return Boolean(
await page.evaluate(() => {
const imgs = Array.from(document.querySelectorAll("img"));
return (
imgs.length === 0 ||
imgs.every((img) => {
const ie = img as HTMLImageElement;
const src = ie.getAttribute("src") || "";
if (!src || src.startsWith("data:")) return true;
// A `complete` image with zero naturalWidth has settled with an
// error (404 / decode failure / CORS rejection / blocked). Treat
// as done — waiting won't make it load — and let the render
// continue with the broken-image marker visible. Mirrors how
// pollVideosReady treats `ve.error`.
if (ie.complete && ie.naturalWidth === 0) return true;
if (ie.complete && ie.naturalWidth > 0) return true;
return false;
})
);
}),
);
};
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (await check()) return true;
await new Promise((resolve) => setTimeout(resolve, intervalMs));
}
return check();
}
// Force every successfully-loaded `<img>` to be GPU-uploaded before the first
// frame capture. `naturalWidth > 0` means the bitmap has been decoded into
// CPU memory, but compositor-side GPU upload can still happen lazily on first
// paint. Calling `img.decode()` returns a Promise that resolves once the image
// is ready for synchronous painting — eliminating the small first-frame race
// between "image is technically loaded" and "the rasterized texture is on the
// GPU and ready to composite".
//
// Note this is purely an init-time guard; it doesn't prevent Chrome from
// evicting decoded pixels mid-render. The producer-side `localizeRemoteImageSources`
// is what bounds the eviction risk (a re-fetch hits the local file server's
// disk-backed paging, not S3 over the network).
//
// Critical: `decode()` on an in-flight image waits for the fetch to resolve.
// If `pollImagesReady` timed out with some images still loading (`!complete`),
// calling `decode()` on them would block here until the network finally
// completes — or until puppeteer's evaluate timeout fires and throws an
// uncaught error that aborts the render. Skip in-flight and broken images;
// only force GPU upload for images that successfully loaded.
async function decodeAllImages(page: Page): Promise<void> {
await page.evaluate(async () => {
const imgs = Array.from(document.querySelectorAll("img"));
await Promise.all(
imgs.map((img) => {
const ie = img as HTMLImageElement;
if (typeof ie.decode !== "function") return Promise.resolve();
// Skip still-loading images (in-flight decode() would hang) and
// broken images (decode() rejects, but pre-filtering is clearer
// than relying on the .catch).
if (!ie.complete || ie.naturalWidth === 0) return Promise.resolve();
return ie.decode().catch(() => undefined);
}),
);
});
}
async function applyVideoMetadataHints( async function applyVideoMetadataHints(
page: Page, page: Page,
hints: readonly CaptureVideoMetadataHint[] | undefined, hints: readonly CaptureVideoMetadataHint[] | undefined,
@@ -707,6 +790,26 @@ export async function initializeSession(session: CaptureSession): Promise<void>
); );
} }
const imagesReady = await pollImagesReady(page, pageReadyTimeout);
if (!imagesReady) {
const failedImages = await page.evaluate(() => {
return Array.from(document.querySelectorAll("img"))
.filter((img) => {
const ie = img as HTMLImageElement;
const src = ie.getAttribute("src") || "";
if (!src || src.startsWith("data:")) return false;
return !(ie.complete && ie.naturalWidth > 0);
})
.map((img) => (img as HTMLImageElement).src || img.getAttribute("src") || "(no src)")
.join(", ");
});
console.warn(
`[FrameCapture] Some image elements did not load within ${pageReadyTimeout}ms: ${failedImages}. ` +
`Continuing render — affected images may appear blank/missing in early frames.`,
);
}
await decodeAllImages(page);
await page.evaluate(`document.fonts?.ready`); await page.evaluate(`document.fonts?.ready`);
await waitForOptionalTailwindReady(page, pageReadyTimeout); await waitForOptionalTailwindReady(page, pageReadyTimeout);
@@ -812,6 +915,28 @@ export async function initializeSession(session: CaptureSession): Promise<void>
); );
} }
// Image readiness — parity with pollVideosReady. Defense against remote
// <img> URLs that bypass the htmlCompiler localize step.
const bfImagesReady = await pollImagesReady(page, pageReadyTimeout);
if (!bfImagesReady) {
const failedImages = await page.evaluate(() => {
return Array.from(document.querySelectorAll("img"))
.filter((img) => {
const ie = img as HTMLImageElement;
const src = ie.getAttribute("src") || "";
if (!src || src.startsWith("data:")) return false;
return !(ie.complete && ie.naturalWidth > 0);
})
.map((img) => (img as HTMLImageElement).src || img.getAttribute("src") || "(no src)")
.join(", ");
});
console.warn(
`[FrameCapture] Some image elements did not load within ${pageReadyTimeout}ms: ${failedImages}. ` +
`Continuing render — affected images may appear blank/missing in early frames.`,
);
}
await decodeAllImages(page);
// Font check (no rAF dependency — uses fonts.ready API directly) // Font check (no rAF dependency — uses fonts.ready API directly)
await page.evaluate(`document.fonts?.ready`); await page.evaluate(`document.fonts?.ready`);
await waitForOptionalTailwindReady(page, pageReadyTimeout); await waitForOptionalTailwindReady(page, pageReadyTimeout);
@@ -11,6 +11,7 @@ import {
discoverAudioVolumeAutomationFromTimeline, discoverAudioVolumeAutomationFromTimeline,
inlineExternalScripts, inlineExternalScripts,
localizeRemoteMediaSources, localizeRemoteMediaSources,
localizeRemoteImageSources,
localizeRemoteFontFaces, localizeRemoteFontFaces,
recompileWithResolutions, recompileWithResolutions,
} from "./htmlCompiler.js"; } from "./htmlCompiler.js";
@@ -950,6 +951,146 @@ describe("localizeRemoteMediaSources", () => {
}); });
}); });
// ── localizeRemoteImageSources ───────────────────────────────────────────────
//
// Regression coverage for the agent-pipeline `<img>` flicker bug: producer's
// frame-capture has no `pollImagesReady` analog of `pollVideosReady`, so a
// composition with raw S3 `<img src="https://...">` URLs (astral / daphne /
// hyperion multi-v2 outputs) reaches Chrome with a network dependency that
// races the readiness gate AND can be evicted mid-render. Localising before
// render is the architectural fix; `pollImagesReady` in frameCapture is the
// defense-in-depth layer.
//
// Mirrors the localizeRemoteMediaSources test shape; fetch is patched in
// for success cases and a real 404 covers the fallback path.
describe("localizeRemoteImageSources", () => {
it("rewrites remote <img> src to _remote_media path when download succeeds", async () => {
const orig = globalThis.fetch;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = async () => new Response(new Uint8Array(100), { status: 200 });
try {
const dl = mkdtempSync(join(tmpdir(), "hf-img-ok-"));
const html = `<img class="hero" src="https://img-ok.example.com/photo.png" />`;
const { html: result, remoteMediaAssets } = await localizeRemoteImageSources(html, dl);
expect(result).not.toContain("https://img-ok.example.com/");
expect(result).toContain("_remote_media/");
expect(remoteMediaAssets.size).toBe(1);
} finally {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = orig;
}
});
it("preserves original URL on download failure without throwing", async () => {
const dl = mkdtempSync(join(tmpdir(), "hf-img-fail-"));
const url = "https://example.com/will-404-image-localize-test.png";
const html = `<img src="${url}" />`;
const { html: result, remoteMediaAssets } = await localizeRemoteImageSources(html, dl);
expect(result).toContain(url);
expect(remoteMediaAssets.size).toBe(0);
});
it("deduplicates: two <img> tags with the same src URL → one download", async () => {
const orig = globalThis.fetch;
let fetchCount = 0;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = async () => {
fetchCount++;
return new Response(new Uint8Array(100), { status: 200 });
};
try {
const dl = mkdtempSync(join(tmpdir(), "hf-img-dedup-"));
const html = `<img src="https://dedup-img.example.com/hero.jpg" />
<img src="https://dedup-img.example.com/hero.jpg" />`;
await localizeRemoteImageSources(html, dl);
expect(fetchCount).toBe(1);
} finally {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = orig;
}
});
it("does not rewrite local (non-HTTP) src paths", async () => {
const dl = mkdtempSync(join(tmpdir(), "hf-img-local-"));
const html = `<img src="assets/hero.png" />`;
const { html: result, remoteMediaAssets } = await localizeRemoteImageSources(html, dl);
expect(result).toContain("assets/hero.png");
expect(result).not.toContain("_remote_media/");
expect(remoteMediaAssets.size).toBe(0);
});
it("does not rewrite data: URI src", async () => {
const dl = mkdtempSync(join(tmpdir(), "hf-img-data-"));
const html = `<img src="data:image/svg+xml,%3Csvg/%3E" />`;
const { html: result, remoteMediaAssets } = await localizeRemoteImageSources(html, dl);
expect(result).toContain("data:image/svg+xml");
expect(remoteMediaAssets.size).toBe(0);
});
it("rewrites both double-quoted and single-quoted src attributes", async () => {
const orig = globalThis.fetch;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = async () => new Response(new Uint8Array(100), { status: 200 });
try {
const dl = mkdtempSync(join(tmpdir(), "hf-img-quotes-"));
const html = `<img src="https://q-img.example.com/dq.png" />
<img src='https://q-img.example.com/sq.jpg' />`;
const { html: result } = await localizeRemoteImageSources(html, dl);
expect(result).not.toContain("https://q-img.example.com/");
expect(result.match(/_remote_media\//g)?.length).toBe(2);
} finally {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = orig;
}
});
it("does not match data-src (lazy-loader placeholder), only the real src attribute", async () => {
// A lazy-loader emits the real asset in `data-src` and a placeholder in
// `src`. We must localise what Chrome actually paints (the real `src`),
// not the `data-src` URL — matching `data-src` would download an asset the
// render never shows and could break the loader's runtime swap.
const orig = globalThis.fetch;
let fetchCount = 0;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = async () => {
fetchCount++;
return new Response(new Uint8Array(100), { status: 200 });
};
try {
const dl = mkdtempSync(join(tmpdir(), "hf-img-datasrc-"));
const html = `<img data-src="https://lazy.example.com/real.png" src="https://cdn.example.com/placeholder.png" />`;
const { html: result } = await localizeRemoteImageSources(html, dl);
// The real src is localised; the data-src URL is left untouched.
expect(result).toContain("https://lazy.example.com/real.png");
expect(result).not.toContain("https://cdn.example.com/placeholder.png");
expect(fetchCount).toBe(1);
} finally {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = orig;
}
});
it("handles src attribute not as the first attribute (agent-pipeline shape)", async () => {
// The 02_kobe astral-pipeline composition that surfaced this bug emits
// <img> tags with `class` before `src`. Regex must not assume src position.
const orig = globalThis.fetch;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = async () => new Response(new Uint8Array(100), { status: 200 });
try {
const dl = mkdtempSync(join(tmpdir(), "hf-img-attr-order-"));
const html = `<img class="kobe-cutout" alt="kobe" src="https://astral.example.com/d828bca.png" />`;
const { html: result, remoteMediaAssets } = await localizeRemoteImageSources(html, dl);
expect(result).not.toContain("https://astral.example.com/");
expect(result).toContain("_remote_media/");
expect(remoteMediaAssets.size).toBe(1);
} finally {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).fetch = orig;
}
});
});
// ── localizeRemoteFontFaces ────────────────────────────────────────────────── // ── localizeRemoteFontFaces ──────────────────────────────────────────────────
describe("localizeRemoteFontFaces", () => { describe("localizeRemoteFontFaces", () => {
+62 -1
View File
@@ -877,6 +877,14 @@ const REMOTE_MEDIA_SUBDIR = "_remote_media";
// have `>` inside quoted attribute values (data-title etc.). // have `>` inside quoted attribute values (data-title etc.).
const REMOTE_MEDIA_TAG_RE = const REMOTE_MEDIA_TAG_RE =
/<(?:video|audio)\b[^>]*?\bsrc\s*=\s*["'](https?:\/\/[^"']+)["'][^>]*>/gi; /<(?:video|audio)\b[^>]*?\bsrc\s*=\s*["'](https?:\/\/[^"']+)["'][^>]*>/gi;
// Match <img> tags (including agent-pipeline-emitted variants where `src` is
// not the first attribute). Producer-side localisation is the primary fix for
// the remote-<img> flicker; frameCapture's `pollImagesReady`/`decodeAllImages`
// are the defense-in-depth layer for any remote URL that bypasses this step.
// The `(?<![\w-])` lookbehind pins the match to a real `src` attribute so we
// don't rewrite `data-src` / `data-*-src` (lazy-loader placeholders whose URL
// is not what Chrome actually paints). `srcset` is excluded by the `\s*=`.
const REMOTE_IMG_TAG_RE = /<img\b[^>]*?(?<![\w-])src\s*=\s*["'](https?:\/\/[^"']+)["'][^>]*>/gi;
/** /**
* Download a set of remote URLs in parallel into `remoteDir`, build the * Download a set of remote URLs in parallel into `remoteDir`, build the
@@ -971,6 +979,49 @@ export async function localizeRemoteMediaSources(
); );
} }
/**
* Download any remote `src` URLs on `<img>` elements into a local subdirectory
* of `downloadDir`, rewrite the HTML src attributes to relative paths, and
* return a `{ relativePath → absoluteLocalPath }` map for the orchestrator.
*
* Why: a composition with remote S3 `<img src>` URLs reaches Chrome unchanged;
* the readiness check can pass before the image is fully decoded, *and* Chrome
* may evict decoded pixels mid-render under memory pressure and re-fetch from
* the remote origin. Either path produces blank-frame flicker. Localising the
* sources before render eliminates both races — once the file is local,
* Chrome's image cache is bounded by fast disk reads, not S3 latency, so a
* mid-render re-fetch lands within a frame instead of flickering. This is the
* primary fix; frameCapture's `pollImagesReady` is the defense-in-depth layer.
*
* Scope: only `<img src>` is localised here. Remote `srcset`,
* `<picture><source>`, SVG `<image href>`, and CSS `background-image: url()`
* outside `@font-face` are NOT covered — agent-pipeline compositions emit
* plain `<img src>`, but those are open follow-ups if other shapes appear.
*
* This bites agent-pipeline-generated compositions (astral / daphne /
* hyperion `multi-v2` outputs) which render directly without going through
* `hyperframes publish`'s archive-time localize step.
*/
/** @internal exported for unit testing only */
export async function localizeRemoteImageSources(
html: string,
downloadDir: string,
): Promise<{ html: string; remoteMediaAssets: Map<string, string> }> {
const urlSet = new Set<string>();
const re = new RegExp(REMOTE_IMG_TAG_RE.source, REMOTE_IMG_TAG_RE.flags);
let m: RegExpExecArray | null;
while ((m = re.exec(html)) !== null) {
if (m[1]) urlSet.add(m[1]);
}
return downloadAndRewriteUrls(
urlSet,
html,
join(downloadDir, REMOTE_MEDIA_SUBDIR),
"Remote image download failed for",
"Localized remote image source(s)",
);
}
// Match url("https://...") or url('https://...') inside @font-face blocks. // Match url("https://...") or url('https://...') inside @font-face blocks.
// We scan the full HTML (which includes <style> blocks) — matching against // We scan the full HTML (which includes <style> blocks) — matching against
// @font-face context precisely would require a CSS parser; instead we match // @font-face context precisely would require a CSS parser; instead we match
@@ -1159,12 +1210,22 @@ export async function compileForRender(
externalAssets.set(relPath, absPath); externalAssets.set(relPath, absPath);
} }
// Download remote <img> sources. Same race shape as video/audio: the
// readiness gate can pass before Chrome decodes the pixels, and Chrome can
// evict decoded pixels mid-render and re-fetch, producing intermittent
// blank-frame flicker. Localising to disk removes both races.
const { html: htmlWithLocalImages, remoteMediaAssets: remoteImageAssets } =
await localizeRemoteImageSources(htmlWithLocalMedia, downloadDir);
for (const [relPath, absPath] of remoteImageAssets) {
externalAssets.set(relPath, absPath);
}
// Download remote @font-face src URLs and rewrite to local paths. // Download remote @font-face src URLs and rewrite to local paths.
// Remote font URLs fail with a CORS rejection at render time (S3 does not // Remote font URLs fail with a CORS rejection at render time (S3 does not
// allow http://localhost:PORT as origin), causing Chrome to silently fall // allow http://localhost:PORT as origin), causing Chrome to silently fall
// back to the next font in the stack. // back to the next font in the stack.
const { html, remoteMediaAssets: remoteFontAssets } = await localizeRemoteFontFaces( const { html, remoteMediaAssets: remoteFontAssets } = await localizeRemoteFontFaces(
htmlWithLocalMedia, htmlWithLocalImages,
downloadDir, downloadDir,
); );
for (const [relPath, absPath] of remoteFontAssets) { for (const [relPath, absPath] of remoteFontAssets) {