mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 09:16:55 +00:00
perf(studio): serve bounded JPEG timeline thumbnails (#4560)
This commit is contained in:
James Russo
@@ -378,6 +378,7 @@
|
||||
"linkedom": "^0.18.12",
|
||||
"postcss": "^8.5.8",
|
||||
"postcss-selector-parser": "^7.1.2",
|
||||
"sharp": "^0.35.0",
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.0.10",
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 23 KiB |
@@ -0,0 +1,49 @@
|
||||
# Bounded timeline JPEG thumbnails
|
||||
|
||||
Status: validating. Local only; PR publication authorized, merge/release excluded.
|
||||
Risk tier 3: image processing across the project file-serving boundary.
|
||||
|
||||
## Evidence and goal
|
||||
|
||||
Base: main 640e99043, isolated worktree. Timeline image strips request original 2560x1920 photos but budget them as 240x135. A frozen local project exposes 43 original JPEGs in its timeline. No private media will be committed or uploaded.
|
||||
|
||||
A client-side resizing experiment passed 35 tests and reduced retained pixel dimensions, but three-run cold-load readiness increased from a median 2.47s to about 3.8s. Offscreen encoding and bitmap resizing did not remove this regression. Reject that approach.
|
||||
|
||||
Deliver bounded JPEG thumbnail responses to the existing scheduler, preserving original composition assets and export behavior. Require before/after browser evidence, correct aspect, no missing thumbnails, no material startup regression, cache invalidation, cancellation, and focused negative-path tests before publishing.
|
||||
|
||||
## Reviewed plan
|
||||
|
||||
1. Add a fixed-size JPEG route using existing project resolution/path containment and Sharp (already a CLI dependency). No arbitrary remote URLs, requested dimensions or output paths. Limit input size/pixels and concurrent work. Keep a bounded in-memory cache, invalidate on file stat changes, deduplicate concurrent requests.
|
||||
2. Route only same-origin local preview JPEGs through it. Preserve other image types and external sources. Retain scheduler ownership and account for actual thumbnail pixels. Fall back to original sources on unsupported/unavailable thumbnail endpoints.
|
||||
3. Test unknown projects, traversal/symlink escapes, non-images, corrupt/oversized files, concurrent requests, changed assets and HTTP validation. Test client fallback and cancellation.
|
||||
4. Repeat frozen-project browser measurements and capture the actual component using synthetic public media. Run types, focused tests, lint/format and changed-file audit; review complete diff.
|
||||
|
||||
## Invariants and review
|
||||
|
||||
No user composition mutations, export substitution, private uploads or new remote processing. Reuse existing safe path resolution; reject traversal before opening a source. Read bounded input into memory so asynchronous processing does not reopen a retargetable path. Decoder metadata must confirm JPEG; reject other formats despite suffix. Cache is per API host, bounded by bytes and entries, includes file identity, and dies with host. Sharp failures return a bounded error and the client preserves original-image behavior. Cancellation must not leak object URLs. Avoid claiming decoded-pixel estimates equal process RSS. All work is reversible by reverting the PR; no data migration or persisted thumbnail artifacts.
|
||||
|
||||
## Non-goals
|
||||
|
||||
Photo lookahead, video preloading, PNG/WebP/GIF/SVG conversion, full-resolution rendering changes, server-side remote fetches, and general image transformation APIs are separate work.
|
||||
|
||||
## Validation log
|
||||
|
||||
Pending implementation and end-to-end comparison. Native dependency loading must be verified in supported CI hosts before merge.
|
||||
|
||||
### Measurements and implementation review
|
||||
|
||||
Three browser runs per variant, Chrome 147 headless at 1280x720, fixed composition/media, same Vite server and original/changed component alternated. Readiness is the harness threshold of at least 147 mounted timeline images decoded; it is not time-to-first-frame. Runtime clock was read from the actual composition player, after clicking Studio Play. Playback samples last ten seconds. Native-image dimensions are a resource-size estimate; summed Chrome process RSS is a separate noisy measurement.
|
||||
|
||||
| Median | Original | Server thumbnails |
|
||||
| --- | ---: | ---: |
|
||||
| Readiness | 2342 ms | 2418 ms |
|
||||
| RAF callbacks / ten seconds | 579 | 593 |
|
||||
| Largest RAF gap | 208 ms | 58 ms |
|
||||
| Gaps above 50 ms | 3 | 1 |
|
||||
| Chrome process RSS | 2205 MiB | 2239 MiB |
|
||||
|
||||
All three runs had 52 unique loaded timeline sources. Their decoded-pixel estimate was 956 MiB before and 4.4 MiB after. This is not a measured RAM reduction: RSS did not improve. Startup differs by 76 ms (3.2%); do not claim startup speedup. Frame pacing improved in these samples; machine load remains a limitation, and the measurements do not prove sustained whole-film FPS gains. The first cold server-cache sample in an earlier series took 2903 ms; cached samples took 2657 and 2519 ms. Composition photo downloads remain unchanged.
|
||||
|
||||
Public evidence uses a synthetic test chart in the real ImageThumbnail component, with original and thumbnail images at identical display size. User media stays local. Sharp sizing/input-limit semantics checked against https://sharp.pixelplumbing.com/api-resize/ and https://sharp.pixelplumbing.com/api-constructor/.
|
||||
|
||||
Review corrections: rejected client resize startup regression; bounded server work/cache and input sizes; preserved non-JPEG/external sources and old-server fallback; extracted path validation to lower route complexity; kept lockfile changes limited to the existing Sharp dependency mapping. Final local validation: 32 client/scheduler tests and 13 server/coordinator tests pass; Studio and studio-server type checks pass; lint/format and changed-file audit pass (duplication warnings remain). Server build passes. Implementation review additionally rejects non-JPEG magic bytes before invoking native decoding. PR CI and cross-platform native dependency checks remain pending.
|
||||
@@ -139,7 +139,8 @@
|
||||
"hono": "^4.0.0",
|
||||
"linkedom": "^0.18.12",
|
||||
"postcss": "^8.5.8",
|
||||
"postcss-selector-parser": "^7.1.2"
|
||||
"postcss-selector-parser": "^7.1.2",
|
||||
"sharp": "^0.35.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.0.10",
|
||||
|
||||
@@ -5,6 +5,7 @@ import { registerFileRoutes } from "./routes/files.js";
|
||||
import { registerPreviewRoutes } from "./routes/preview.js";
|
||||
import { registerLintRoutes } from "./routes/lint.js";
|
||||
import { registerRenderRoutes } from "./routes/render.js";
|
||||
import { registerImageThumbnailRoutes } from "./routes/imageThumbnail.js";
|
||||
import { registerThumbnailRoutes } from "./routes/thumbnail.js";
|
||||
import { registerWaveformRoutes } from "./routes/waveform.js";
|
||||
import { registerFontRoutes } from "./routes/fonts.js";
|
||||
@@ -29,6 +30,7 @@ export function createStudioApi(adapter: StudioApiAdapter): Hono {
|
||||
registerLintRoutes(api, adapter);
|
||||
registerRenderRoutes(api, adapter);
|
||||
registerThumbnailRoutes(api, adapter);
|
||||
registerImageThumbnailRoutes(api, adapter);
|
||||
registerSelectionRoutes(api, adapter);
|
||||
registerMediaRoutes(api, adapter);
|
||||
registerWaveformRoutes(api, adapter);
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { Hono } from "hono";
|
||||
import { mkdtemp, rm, writeFile, symlink, open } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import sharp from "sharp";
|
||||
import { registerImageThumbnailRoutes } from "./imageThumbnail";
|
||||
import type { StudioApiAdapter } from "../types";
|
||||
|
||||
let dir: string;
|
||||
let outside: string;
|
||||
let app: Hono;
|
||||
const url = "http://localhost/projects/p/image-thumbnail/photo.jpg";
|
||||
async function photo(path: string, width = 2560, height = 1920) {
|
||||
await sharp({ create: { width, height, channels: 3, background: "#6495ed" } })
|
||||
.jpeg()
|
||||
.toFile(path);
|
||||
}
|
||||
beforeEach(async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), "hf-image-thumbnail-"));
|
||||
outside = await mkdtemp(join(tmpdir(), "hf-image-outside-"));
|
||||
const adapter: StudioApiAdapter = {
|
||||
listProjects: () => [],
|
||||
resolveProject: (id) => (id === "p" ? { id, dir } : null),
|
||||
bundle: async () => null,
|
||||
lint: async () => ({ findings: [] }),
|
||||
runtimeUrl: "/runtime.js",
|
||||
rendersDir: () => dir,
|
||||
startRender: () => ({ id: "unused", status: "rendering", progress: 0, outputPath: "" }),
|
||||
};
|
||||
app = new Hono();
|
||||
registerImageThumbnailRoutes(app, adapter);
|
||||
await photo(join(dir, "photo.jpg"));
|
||||
});
|
||||
afterEach(async () => {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
await rm(outside, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("JPEG timeline thumbnail route", () => {
|
||||
it("returns a bounded JPEG, privately revalidated and unchanged on repeat requests", async () => {
|
||||
const response = await app.request(url);
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.headers.get("content-type")).toBe("image/jpeg");
|
||||
expect(response.headers.get("cache-control")).toBe("private, no-cache");
|
||||
const bytes = Buffer.from(await response.arrayBuffer());
|
||||
const metadata = await sharp(bytes).metadata();
|
||||
expect([metadata.width, metadata.height]).toEqual([180, 135]);
|
||||
const repeat = await app.request(url);
|
||||
expect(Buffer.from(await repeat.arrayBuffer())).toEqual(bytes);
|
||||
const etag = response.headers.get("etag");
|
||||
if (!etag) throw new Error("missing etag");
|
||||
expect((await app.request(url, { headers: { "If-None-Match": etag } })).status).toBe(304);
|
||||
});
|
||||
it("invalidates cached responses when the source changes", async () => {
|
||||
const first = await app.request(url);
|
||||
await photo(join(dir, "photo.jpg"), 1000, 2000);
|
||||
const next = await app.request(url);
|
||||
expect(next.headers.get("etag")).not.toBe(first.headers.get("etag"));
|
||||
const metadata = await sharp(Buffer.from(await next.arrayBuffer())).metadata();
|
||||
expect([metadata.width, metadata.height]).toEqual([68, 135]);
|
||||
});
|
||||
it("deduplicates concurrent requests and keeps their results readable", async () => {
|
||||
const responses = await Promise.all(Array.from({ length: 8 }, () => app.request(url)));
|
||||
const buffers = await Promise.all(
|
||||
responses.map(async (response) => {
|
||||
expect(response.status).toBe(200);
|
||||
return Buffer.from(await response.arrayBuffer());
|
||||
}),
|
||||
);
|
||||
expect(buffers.every((buffer) => buffer.equals(buffers[0]))).toBe(true);
|
||||
});
|
||||
it("respects EXIF orientation and rejects excessive pixel counts", async () => {
|
||||
const path = join(dir, "photo.jpg");
|
||||
const bytes = await sharp({
|
||||
create: { width: 2560, height: 1920, channels: 3, background: "#6495ed" },
|
||||
})
|
||||
.withMetadata({ orientation: 6 })
|
||||
.jpeg()
|
||||
.toBuffer();
|
||||
await writeFile(path, bytes);
|
||||
const response = await app.request(url);
|
||||
const metadata = await sharp(Buffer.from(await response.arrayBuffer())).metadata();
|
||||
expect([metadata.width, metadata.height]).toEqual([101, 135]);
|
||||
const header = bytes.indexOf(Buffer.from([0xff, 0xc0]));
|
||||
expect(header).toBeGreaterThan(0);
|
||||
bytes.writeUInt16BE(50000, header + 5);
|
||||
bytes.writeUInt16BE(50000, header + 7);
|
||||
await writeFile(path, bytes);
|
||||
expect((await app.request(url)).status).toBe(422);
|
||||
expect((await app.request(url.replace("photo.jpg", "missing.jpg"))).status).toBe(404);
|
||||
});
|
||||
|
||||
it("does not upscale small JPEGs", async () => {
|
||||
await photo(join(dir, "photo.jpg"), 40, 30);
|
||||
const response = await app.request(url);
|
||||
const metadata = await sharp(Buffer.from(await response.arrayBuffer())).metadata();
|
||||
expect([metadata.width, metadata.height]).toEqual([40, 30]);
|
||||
});
|
||||
it("rejects unknown projects, encoded traversal and symlink escapes", async () => {
|
||||
expect((await app.request(url.replace("/p/", "/unknown/"))).status).toBe(404);
|
||||
expect((await app.request(url.replace("photo.jpg", "..%2Foutside.jpg"))).status).toBe(404);
|
||||
await photo(join(outside, "private.jpg"));
|
||||
await symlink(join(outside, "private.jpg"), join(dir, "link.jpg"));
|
||||
expect((await app.request(url.replace("photo.jpg", "link.jpg"))).status).toBe(404);
|
||||
});
|
||||
it("rejects other formats even if renamed to JPEG", async () => {
|
||||
expect((await app.request(url.replace("photo.jpg", "photo.png"))).status).toBe(415);
|
||||
await writeFile(
|
||||
join(dir, "photo.jpg"),
|
||||
await sharp({ create: { width: 10, height: 10, channels: 4, background: "#fff" } })
|
||||
.png()
|
||||
.toBuffer(),
|
||||
);
|
||||
expect((await app.request(url)).status).toBe(422);
|
||||
});
|
||||
it("rejects malformed paths, corrupt images and oversized sources", async () => {
|
||||
expect((await app.request(url.replace("photo.jpg", "%FF.jpg"))).status).toBe(400);
|
||||
await writeFile(join(dir, "photo.jpg"), "not an image");
|
||||
expect((await app.request(url)).status).toBe(422);
|
||||
const file = await open(join(dir, "photo.jpg"), "w");
|
||||
await file.truncate(65 * 1024 * 1024);
|
||||
await file.close();
|
||||
expect((await app.request(url)).status).toBe(413);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,109 @@
|
||||
import type { Hono } from "hono";
|
||||
import { HTTPException } from "hono/http-exception";
|
||||
import { open, stat } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import sharp from "sharp";
|
||||
import type { StudioApiAdapter } from "../types.js";
|
||||
import { pinWithinProject } from "../helpers/safePath.js";
|
||||
import { requestSubPath } from "../helpers/requestSubPath.js";
|
||||
import { ThumbnailGenerationCoordinator } from "./thumbnailGenerationCoordinator.js";
|
||||
|
||||
const MAX_SOURCE_BYTES = 64 * 1024 * 1024;
|
||||
const MAX_CACHE_BYTES = 8 * 1024 * 1024;
|
||||
|
||||
async function generateThumbnail(path: string, signal: AbortSignal): Promise<Buffer> {
|
||||
const file = await open(path, "r");
|
||||
try {
|
||||
const info = await file.stat();
|
||||
if (!info.isFile() || info.size > MAX_SOURCE_BYTES) throw new Error("Unsupported image");
|
||||
const source = Buffer.alloc(info.size);
|
||||
let offset = 0;
|
||||
while (offset < source.length) {
|
||||
signal.throwIfAborted();
|
||||
const { bytesRead } = await file.read(source, offset, source.length - offset, offset);
|
||||
if (!bytesRead) break;
|
||||
offset += bytesRead;
|
||||
}
|
||||
if (source[0] !== 0xff || source[1] !== 0xd8) throw new Error("Unsupported image");
|
||||
const image = sharp(source.subarray(0, offset), { limitInputPixels: 40_000_000 });
|
||||
if ((await image.metadata()).format !== "jpeg") throw new Error("Unsupported image");
|
||||
signal.throwIfAborted();
|
||||
const result = await image
|
||||
.rotate()
|
||||
.resize(240, 135, { fit: "inside", withoutEnlargement: true })
|
||||
.jpeg({ quality: 80 })
|
||||
.toBuffer();
|
||||
signal.throwIfAborted();
|
||||
return result;
|
||||
} finally {
|
||||
await file.close();
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveSource(adapter: StudioApiAdapter, id: string, url: string) {
|
||||
const project = await adapter.resolveProject(id);
|
||||
if (!project) throw new HTTPException(404);
|
||||
let relative: string;
|
||||
try {
|
||||
relative = requestSubPath(url, "projects/:id/image-thumbnail");
|
||||
} catch {
|
||||
throw new HTTPException(400);
|
||||
}
|
||||
if (!/\.jpe?g$/i.test(relative)) throw new HTTPException(415);
|
||||
const path = pinWithinProject(project.dir, relative);
|
||||
if (!path) throw new HTTPException(404);
|
||||
const info = await stat(path).catch(() => {
|
||||
throw new HTTPException(404);
|
||||
});
|
||||
if (!info.isFile()) throw new HTTPException(404);
|
||||
if (info.size > MAX_SOURCE_BYTES) throw new HTTPException(413);
|
||||
return { path, info };
|
||||
}
|
||||
|
||||
export function registerImageThumbnailRoutes(api: Hono, adapter: StudioApiAdapter): void {
|
||||
const coordinator = new ThumbnailGenerationCoordinator(2);
|
||||
const cache = new Map<string, Buffer>();
|
||||
let cacheBytes = 0;
|
||||
const remember = (key: string, image: Buffer) => {
|
||||
cacheBytes -= cache.get(key)?.byteLength ?? 0;
|
||||
cache.delete(key);
|
||||
cache.set(key, image);
|
||||
cacheBytes += image.byteLength;
|
||||
while (cacheBytes > MAX_CACHE_BYTES || cache.size > 128) {
|
||||
const oldest = cache.keys().next().value;
|
||||
if (oldest === undefined) break;
|
||||
cacheBytes -= cache.get(oldest)?.byteLength ?? 0;
|
||||
cache.delete(oldest);
|
||||
}
|
||||
};
|
||||
api.get("/projects/:id/image-thumbnail/*", async (c) => {
|
||||
try {
|
||||
const { path, info } = await resolveSource(adapter, c.req.param("id"), c.req.url);
|
||||
const key = createHash("sha256")
|
||||
.update(JSON.stringify([path, info.dev, info.ino, info.size, info.mtimeMs, info.ctimeMs]))
|
||||
.digest("hex");
|
||||
const etag = `"${key}"`;
|
||||
c.header("ETag", etag);
|
||||
c.header("Cache-Control", "private, no-cache");
|
||||
c.header("X-Content-Type-Options", "nosniff");
|
||||
if (c.req.header("If-None-Match") === etag) return c.body(null, 304);
|
||||
let result = cache.get(key);
|
||||
if (result) remember(key, result);
|
||||
else {
|
||||
if (coordinator.protectedKeys().size >= 32) return c.text("thumbnail queue full", 429);
|
||||
result =
|
||||
(await coordinator.acquire(key, c.req.raw.signal, async (signal) => {
|
||||
const image = await generateThumbnail(path, signal);
|
||||
remember(key, image);
|
||||
return image;
|
||||
})) ?? undefined;
|
||||
}
|
||||
if (!result) return c.text("image unavailable", 422);
|
||||
c.header("Content-Type", "image/jpeg");
|
||||
return c.body(new Uint8Array(result));
|
||||
} catch (error) {
|
||||
if (error instanceof HTTPException) return error.getResponse();
|
||||
return c.text("image unavailable", 422);
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -2,8 +2,8 @@ import { memo, useCallback, useMemo, useRef, useState } from "react";
|
||||
import { useMountEffect } from "../../hooks/useMountEffect";
|
||||
import { useThumbnailLease } from "../../hooks/useThumbnailLease";
|
||||
import { createThumbnailKey, type ThumbnailPriority } from "../lib/thumbnailScheduler";
|
||||
import { TIMELINE_VIEWPORT_BUDGETS } from "../lib/timelineViewportBudgets";
|
||||
import { computeThumbnailStrip, probeImageAspect } from "./thumbnailUtils";
|
||||
import { decodeImageThumbnail } from "../lib/thumbnailImageDecoder";
|
||||
import { computeThumbnailStrip } from "./thumbnailUtils";
|
||||
|
||||
export interface ImageThumbnailProps {
|
||||
imageSrc: string;
|
||||
@@ -35,16 +35,7 @@ export const ImageThumbnail = memo(function ImageThumbnail({
|
||||
kind: "image" as const,
|
||||
priority,
|
||||
rich,
|
||||
load: async (signal: AbortSignal) => {
|
||||
const aspect = await probeImageAspect(imageSrc, signal, true);
|
||||
return {
|
||||
value: { kind: "image" as const, url: imageSrc, aspect },
|
||||
weight:
|
||||
TIMELINE_VIEWPORT_BUDGETS.posterMaxPhysicalWidth *
|
||||
TIMELINE_VIEWPORT_BUDGETS.posterMaxPhysicalHeight *
|
||||
4,
|
||||
};
|
||||
},
|
||||
load: (signal: AbortSignal) => decodeImageThumbnail(imageSrc, signal),
|
||||
}),
|
||||
[imageSrc, priority, projectId, rich, sessionEpoch],
|
||||
);
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
// @vitest-environment happy-dom
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { decodeImageThumbnail } from "./thumbnailImageDecoder";
|
||||
|
||||
class TestImage {
|
||||
static instances: TestImage[] = [];
|
||||
naturalWidth = 180;
|
||||
naturalHeight = 135;
|
||||
src = "";
|
||||
onload: (() => void) | null = null;
|
||||
onerror: (() => void) | null = null;
|
||||
constructor() {
|
||||
TestImage.instances.push(this);
|
||||
}
|
||||
}
|
||||
beforeEach(() => {
|
||||
TestImage.instances = [];
|
||||
vi.stubGlobal("Image", TestImage);
|
||||
});
|
||||
afterEach(() => vi.unstubAllGlobals());
|
||||
function latest() {
|
||||
const image = TestImage.instances.at(-1);
|
||||
if (!image) throw new Error("missing image");
|
||||
return image;
|
||||
}
|
||||
|
||||
describe("timeline image thumbnails", () => {
|
||||
it("requests a bounded local JPEG and accounts for its actual decoded dimensions", async () => {
|
||||
const result = decodeImageThumbnail(
|
||||
"/api/projects/p/preview/photo.jpg?v=2",
|
||||
new AbortController().signal,
|
||||
);
|
||||
expect(latest().src).toBe(
|
||||
new URL("/api/projects/p/image-thumbnail/photo.jpg?v=2", location.href).href,
|
||||
);
|
||||
latest().onload?.();
|
||||
expect(await result).toEqual({
|
||||
value: {
|
||||
kind: "image",
|
||||
url: new URL("/api/projects/p/image-thumbnail/photo.jpg?v=2", location.href).href,
|
||||
aspect: 4 / 3,
|
||||
},
|
||||
weight: 180 * 135 * 4,
|
||||
});
|
||||
expect(latest().src).toBe("");
|
||||
});
|
||||
it.each([
|
||||
"https://remote.test/photo.jpg",
|
||||
"/api/projects/p/preview/image.png",
|
||||
"/api/projects/p/preview/image.gif",
|
||||
"/api/projects/p/preview/image.webp",
|
||||
"/api/projects/p/preview/image.svg",
|
||||
"/other/photo.jpg",
|
||||
])("preserves %s", async (source) => {
|
||||
const result = decodeImageThumbnail(source, new AbortController().signal);
|
||||
expect(latest().src).toBe(source);
|
||||
latest().onload?.();
|
||||
expect((await result).value).toMatchObject({ url: source });
|
||||
});
|
||||
it("falls back for an older server and accounts for original dimensions", async () => {
|
||||
const result = decodeImageThumbnail(
|
||||
"/api/projects/p/preview/photo.jpg",
|
||||
new AbortController().signal,
|
||||
);
|
||||
latest().onerror?.();
|
||||
await Promise.resolve();
|
||||
expect(latest().src).toBe("/api/projects/p/preview/photo.jpg");
|
||||
latest().naturalWidth = 2560;
|
||||
latest().naturalHeight = 1920;
|
||||
latest().onload?.();
|
||||
expect((await result).weight).toBe(2560 * 1920 * 4);
|
||||
});
|
||||
it("does not retry a cancelled request", async () => {
|
||||
const controller = new AbortController();
|
||||
const result = decodeImageThumbnail("/api/projects/p/preview/photo.jpg", controller.signal);
|
||||
controller.abort();
|
||||
await expect(result).rejects.toMatchObject({ name: "AbortError" });
|
||||
expect(TestImage.instances).toHaveLength(1);
|
||||
expect(latest().src).toBe("");
|
||||
expect(latest().onload).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,73 @@
|
||||
import type { ThumbnailLoadedResult } from "./thumbnailScheduler";
|
||||
import { TIMELINE_VIEWPORT_BUDGETS } from "./timelineViewportBudgets";
|
||||
|
||||
function thumbnailSource(source: string): string {
|
||||
const url = new URL(source, window.location.href);
|
||||
if (
|
||||
url.origin !== window.location.origin ||
|
||||
!/^\/api\/projects\/[^/]+\/preview\/.+\.jpe?g$/i.test(url.pathname)
|
||||
)
|
||||
return source;
|
||||
url.pathname = url.pathname.replace(
|
||||
/^(\/api\/projects\/[^/]+)\/preview\//,
|
||||
"$1/image-thumbnail/",
|
||||
);
|
||||
return url.href;
|
||||
}
|
||||
|
||||
function loadImage(source: string, signal: AbortSignal): Promise<HTMLImageElement> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const image = new Image();
|
||||
const cleanup = () => {
|
||||
image.onload = null;
|
||||
image.onerror = null;
|
||||
signal.removeEventListener("abort", onAbort);
|
||||
};
|
||||
const onAbort = () => {
|
||||
cleanup();
|
||||
image.src = "";
|
||||
reject(new DOMException("Aborted", "AbortError"));
|
||||
};
|
||||
if (signal.aborted) {
|
||||
onAbort();
|
||||
return;
|
||||
}
|
||||
image.onload = () => {
|
||||
cleanup();
|
||||
resolve(image);
|
||||
};
|
||||
image.onerror = () => {
|
||||
cleanup();
|
||||
if (/\.svg($|\?)/i.test(source)) resolve(image);
|
||||
else {
|
||||
image.src = "";
|
||||
reject(new Error("Image thumbnail failed to load"));
|
||||
}
|
||||
};
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
image.src = source;
|
||||
});
|
||||
}
|
||||
|
||||
export async function decodeImageThumbnail(
|
||||
source: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<ThumbnailLoadedResult> {
|
||||
let url = thumbnailSource(source);
|
||||
let image: HTMLImageElement;
|
||||
try {
|
||||
image = await loadImage(url, signal);
|
||||
} catch (error) {
|
||||
if (signal.aborted || url === source) throw error;
|
||||
url = source;
|
||||
image = await loadImage(source, signal);
|
||||
}
|
||||
try {
|
||||
signal.throwIfAborted();
|
||||
const width = image.naturalWidth || TIMELINE_VIEWPORT_BUDGETS.posterMaxPhysicalWidth;
|
||||
const height = image.naturalHeight || TIMELINE_VIEWPORT_BUDGETS.posterMaxPhysicalHeight;
|
||||
return { value: { kind: "image", url, aspect: width / height }, weight: width * height * 4 };
|
||||
} finally {
|
||||
image.src = "";
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user