perf(producer): stream binary file responses, async-read HTML (#1735)

* perf(producer): stream binary file responses, async-read HTML

Replaces the per-request readFileSync in fileServer's static file handler
with a createReadStream pipe (binary) and an async readFile (HTML). Static
asset serving no longer blocks the Node event loop.

Why
---

The pre-fix handler called readFileSync(filePath) on every binary asset.
On video-heavy compositions Chrome requests several 32MB video files
back-to-back; each readFileSync(32MB) blocked the main event loop long
enough to wedge concurrent /health responses and other timers.

Scope clarification — this addresses the event-loop block documented at
renderOrchestrator.ts:1277-1306 (the video-heavy regression class). It is
NOT the fix for today's infinite-duration incident; Miguel is shipping
that upstream as a plan()-time duration guard. The two are complementary:

  - Miguel's guard kills the impossible-work input shape before chunk
    planning so the producer doesn't try to enumerate 300B frames.
  - This streaming fix removes the next-largest known main-thread block
    (large binary I/O during video-heavy renders), so future wedge
    classes don't kill otherwise-healthy probes either.

The companion worker_thread /health PR + the heygen-com/app probe-timeout
bump round out the defense-in-depth: even if some future code path
introduces another main-thread stall, the probe lives off-thread and the
budget is 30s anyway.

What changed
------------

fileServer.ts: switched both file branches off the sync I/O path.

  - Binary (the hot path for video-heavy renders): readFileSync(filePath)
    -> createReadStream + Readable.toWeb -> Response stream body.
    Content-Length is set via statSync so Chrome's range-aware media
    stack sees the size up front. The handler is now async because the
    HTML branch awaits.

  - HTML (small files; injected with pre/head/body scripts):
    readFileSync(filePath, "utf-8") -> readFile(filePath, "utf-8").
    The injection is still sync — pure string ops — only the disk read
    moved off-thread. Index HTMLs are tiny (~200KB max for AI-generated
    compositions) but a ms of stall per render-start adds up across a
    fleet.

Test
----

fileServer.test.ts: added a streaming regression that pins three
properties on a 5MB synthetic binary asset (chunk-boundary spanning):

  1. Correctness — served bytes match the file across multiple
     createReadStream chunks (default 64KB highWaterMark).
  2. Content-Length header is set from statSync.
  3. Four parallel fetches all return identical content; the streaming
     path doesn't serialize them.

All 31 fileServer tests pass locally (bun test).

TODO: link Miguel's upstream plan() duration guard PR once known.

— Jerrai

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(producer): implement Accept-Ranges + 206 Partial Content for fileServer

Delivers the range-request semantics the original PR body promised but
the diff did not implement. Without range support, Chrome's <video>
element issues full-file GETs on seek; with this commit it can issue
`Range: bytes=...` and get a sliced 206 back, so seek + partial-load
work without re-pulling the whole file.

- Add `parseRangeHeader` (exported for unit tests) covering the three
  RFC 7233 single-range forms: bytes=START-END (closed), bytes=START-
  (open-ended), bytes=-SUFFIX (last N bytes). Multi-range falls back to
  `absent` (full 200) so we never reassemble multipart/byteranges.
- Binary path now returns 206 Partial Content with Content-Range +
  sliced Content-Length on satisfiable ranges, 416 Range Not Satisfiable
  with `Content-Range: bytes (asterisk)/<size>` on unsatisfiable ranges,
  and 200 with `Accept-Ranges: bytes` on full-body GETs so clients know
  ranges are supported.
- Add unit tests for parseRangeHeader (10 cases: 3 forms, clamping,
  unsatisfiable edges, malformed inputs, multi-range fallback).
- Add integration test covering 200 + Accept-Ranges, all 3 range forms
  with byte-correct slices, 416 on out-of-bounds, and multi-range -> 200
  fallback.

Addresses Miga's review finding on #1735.

Co-Authored-By: Jerrai <noreply@anthropic.com>

— Jerrai

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
James Russo
2026-06-26 08:47:53 -07:00
committed by GitHub
co-authored by Jerrai
parent 01a10cdc53
commit ca9e1316be
2 changed files with 383 additions and 6 deletions
@@ -9,6 +9,7 @@ import {
HF_EARLY_STUB,
injectScriptsAtHeadStart,
isPathInside,
parseRangeHeader,
VIRTUAL_TIME_SHIM,
} from "./fileServer.js";
@@ -225,6 +226,94 @@ describe("isPathInside", () => {
});
});
describe("parseRangeHeader", () => {
const SIZE = 1000;
it("returns absent when there is no Range header", () => {
expect(parseRangeHeader(undefined, SIZE)).toEqual({ kind: "absent" });
expect(parseRangeHeader(null, SIZE)).toEqual({ kind: "absent" });
expect(parseRangeHeader("", SIZE)).toEqual({ kind: "absent" });
});
it("parses a closed range bytes=START-END", () => {
expect(parseRangeHeader("bytes=0-99", SIZE)).toEqual({
kind: "satisfiable",
start: 0,
end: 99,
});
expect(parseRangeHeader("bytes=100-199", SIZE)).toEqual({
kind: "satisfiable",
start: 100,
end: 199,
});
});
it("parses an open-ended range bytes=START- as start..EOF", () => {
expect(parseRangeHeader("bytes=100-", SIZE)).toEqual({
kind: "satisfiable",
start: 100,
end: SIZE - 1,
});
expect(parseRangeHeader("bytes=0-", SIZE)).toEqual({
kind: "satisfiable",
start: 0,
end: SIZE - 1,
});
});
it("parses a suffix range bytes=-N as the last N bytes", () => {
expect(parseRangeHeader("bytes=-50", SIZE)).toEqual({
kind: "satisfiable",
start: SIZE - 50,
end: SIZE - 1,
});
// Suffix larger than the file: clamp to the whole file.
expect(parseRangeHeader("bytes=-5000", SIZE)).toEqual({
kind: "satisfiable",
start: 0,
end: SIZE - 1,
});
});
it("clamps the end of a closed range to the last valid byte", () => {
// bytes=900-9999 on a 1000-byte file -> serve 900..999.
expect(parseRangeHeader("bytes=900-9999", SIZE)).toEqual({
kind: "satisfiable",
start: 900,
end: SIZE - 1,
});
});
it("returns unsatisfiable when start >= size", () => {
expect(parseRangeHeader("bytes=1000-2000", SIZE)).toEqual({ kind: "unsatisfiable" });
expect(parseRangeHeader("bytes=2000-", SIZE)).toEqual({ kind: "unsatisfiable" });
});
it("returns unsatisfiable when end < start in a closed range", () => {
expect(parseRangeHeader("bytes=200-100", SIZE)).toEqual({ kind: "unsatisfiable" });
});
it("returns unsatisfiable for a suffix request on a zero-byte file", () => {
expect(parseRangeHeader("bytes=-10", 0)).toEqual({ kind: "unsatisfiable" });
});
it("returns absent for non-bytes units, multi-range, and malformed inputs", () => {
expect(parseRangeHeader("items=0-1", SIZE)).toEqual({ kind: "absent" });
expect(parseRangeHeader("bytes=0-99,200-299", SIZE)).toEqual({ kind: "absent" });
expect(parseRangeHeader("bytes=abc-def", SIZE)).toEqual({ kind: "absent" });
expect(parseRangeHeader("bytes=", SIZE)).toEqual({ kind: "absent" });
expect(parseRangeHeader("bytes=-", SIZE)).toEqual({ kind: "absent" });
});
it("tolerates surrounding whitespace and case", () => {
expect(parseRangeHeader(" Bytes = 0-99 ", SIZE)).toEqual({
kind: "satisfiable",
start: 0,
end: 99,
});
});
});
describe("createFileServer", () => {
it("serves asset files through project-root symlinked directories", async () => {
const workspaceDir = mkdtempSync(join(tmpdir(), "hf-file-server-symlink-assets-"));
@@ -253,6 +342,150 @@ describe("createFileServer", () => {
}
});
it("streams binary file content without buffering through readFileSync", async () => {
// Regression test for the video-heavy event-loop block documented at
// renderOrchestrator.ts:1277-1306. Pre-fix the file route called
// readFileSync on every binary asset, which on 32MB+ videos stalled
// the Node event loop long enough to wedge concurrent /health probes.
// This test pins three properties of the streaming path:
//
// 1. Correctness: the served byte sequence matches the file exactly,
// across a chunk boundary (we use a 5 MB synthetic asset, well past
// Node's default 64KB createReadStream highWaterMark).
// 2. Content-Length is reported via statSync so range-aware HTTP
// consumers (Chrome's media stack) see the size up front.
// 3. Concurrent requests don't serialize behind each other — N
// parallel fetches all return identical content. With readFileSync
// they'd block the event loop in serial; with the stream they
// pipe interleaved chunks.
const projectDir = mkdtempSync(join(tmpdir(), "hf-file-server-stream-"));
try {
writeEmptyIndex(projectDir);
// 5 MB of deterministic bytes — large enough to span many 64KB read
// chunks, small enough to keep the test fast.
const size = 5 * 1024 * 1024;
const buf = Buffer.alloc(size);
for (let i = 0; i < size; i++) buf[i] = i & 0xff;
writeFileSync(join(projectDir, "big.bin"), buf);
await withFileServer(projectDir, async (server) => {
// Single-request correctness + content-length.
const r = await fetch(`${server.url}/big.bin`);
expect(r.status).toBe(200);
expect(r.headers.get("content-length")).toBe(String(size));
const out = Buffer.from(await r.arrayBuffer());
expect(out.length).toBe(size);
// Spot-check a few sentinel positions (full equality check is O(5MB)
// and unnecessary — if any chunk were misaligned we'd see it here).
expect(out[0]).toBe(0);
expect(out[255]).toBe(255);
expect(out[256]).toBe(0);
expect(out[size - 1]).toBe((size - 1) & 0xff);
// Concurrent requests don't corrupt each other.
const concurrent = await Promise.all(
Array.from({ length: 4 }, () => fetch(`${server.url}/big.bin`)),
);
for (const resp of concurrent) {
expect(resp.status).toBe(200);
const body = Buffer.from(await resp.arrayBuffer());
expect(body.length).toBe(size);
expect(body[0]).toBe(0);
expect(body[size - 1]).toBe((size - 1) & 0xff);
}
});
} finally {
rmSync(projectDir, { recursive: true, force: true });
}
});
it("serves Range requests with 206 Partial Content + Accept-Ranges", async () => {
// Pins the RFC 7233 implementation for the binary path: Chrome's <video>
// element issues `Range: bytes=...` when seeking, and the response must
// be 206 with `Content-Range` + a sliced body so the player can resume
// partial-load without re-pulling the whole file. Also pins that the
// server advertises `Accept-Ranges: bytes` on full-body GETs so clients
// know future Range requests are supported.
const projectDir = mkdtempSync(join(tmpdir(), "hf-file-server-range-"));
try {
writeEmptyIndex(projectDir);
// Use a 4 KB deterministic asset: small enough to keep the test
// fast, large enough that suffix / partial responses exercise the
// slicing math meaningfully.
const size = 4096;
const buf = Buffer.alloc(size);
for (let i = 0; i < size; i++) buf[i] = i & 0xff;
writeFileSync(join(projectDir, "asset.bin"), buf);
await withFileServer(projectDir, async (server) => {
// 1. Full GET advertises Accept-Ranges: bytes.
const full = await fetch(`${server.url}/asset.bin`);
expect(full.status).toBe(200);
expect(full.headers.get("accept-ranges")).toBe("bytes");
expect(full.headers.get("content-length")).toBe(String(size));
await full.body?.cancel();
// 2. Closed range: bytes=0-99 returns the first 100 bytes.
const head = await fetch(`${server.url}/asset.bin`, {
headers: { Range: "bytes=0-99" },
});
expect(head.status).toBe(206);
expect(head.headers.get("content-range")).toBe(`bytes 0-99/${size}`);
expect(head.headers.get("content-length")).toBe("100");
expect(head.headers.get("accept-ranges")).toBe("bytes");
const headBody = Buffer.from(await head.arrayBuffer());
expect(headBody.length).toBe(100);
expect(headBody[0]).toBe(0);
expect(headBody[99]).toBe(99);
// 3. Open-ended: bytes=4000- returns the tail.
const tail = await fetch(`${server.url}/asset.bin`, {
headers: { Range: "bytes=4000-" },
});
expect(tail.status).toBe(206);
expect(tail.headers.get("content-range")).toBe(`bytes 4000-${size - 1}/${size}`);
expect(tail.headers.get("content-length")).toBe(String(size - 4000));
const tailBody = Buffer.from(await tail.arrayBuffer());
expect(tailBody.length).toBe(size - 4000);
expect(tailBody[0]).toBe(4000 & 0xff);
expect(tailBody[tailBody.length - 1]).toBe((size - 1) & 0xff);
// 4. Suffix: bytes=-50 returns the last 50 bytes.
const suffix = await fetch(`${server.url}/asset.bin`, {
headers: { Range: "bytes=-50" },
});
expect(suffix.status).toBe(206);
expect(suffix.headers.get("content-range")).toBe(`bytes ${size - 50}-${size - 1}/${size}`);
expect(suffix.headers.get("content-length")).toBe("50");
const suffixBody = Buffer.from(await suffix.arrayBuffer());
expect(suffixBody.length).toBe(50);
expect(suffixBody[0]).toBe((size - 50) & 0xff);
expect(suffixBody[49]).toBe((size - 1) & 0xff);
// 5. Unsatisfiable: bytes=99999-99999 returns 416 with
// Content-Range: bytes */<size> per RFC 7233 §4.4.
const bad = await fetch(`${server.url}/asset.bin`, {
headers: { Range: "bytes=99999-99999" },
});
expect(bad.status).toBe(416);
expect(bad.headers.get("content-range")).toBe(`bytes */${size}`);
expect(bad.headers.get("accept-ranges")).toBe("bytes");
await bad.body?.cancel();
// 6. Multi-range falls back to 200 (we don't reassemble
// multipart/byteranges for the single-asset use case).
const multi = await fetch(`${server.url}/asset.bin`, {
headers: { Range: "bytes=0-9,20-29" },
});
expect(multi.status).toBe(200);
expect(multi.headers.get("accept-ranges")).toBe("bytes");
await multi.body?.cancel();
});
} finally {
rmSync(projectDir, { recursive: true, force: true });
}
});
it("decodes percent-encoded reserved characters in URL path segments", async () => {
const projectDir = mkdtempSync(join(tmpdir(), "hf-file-server-reserved-chars-"));
+150 -6
View File
@@ -11,7 +11,9 @@
import { Hono } from "hono";
import { serve } from "@hono/node-server";
import type { IncomingMessage } from "node:http";
import { readFileSync, existsSync, realpathSync, statSync } from "node:fs";
import { existsSync, realpathSync, statSync, createReadStream } from "node:fs";
import { readFile } from "node:fs/promises";
import { Readable } from "node:stream";
import { join, extname, resolve, sep } from "node:path";
import { injectScriptsAtHeadStart, injectScriptsIntoHtml } from "@hyperframes/core/compiler";
import { getVerifiedHyperframeRuntimeSource } from "./hyperframeRuntimeLoader.js";
@@ -96,6 +98,86 @@ const MIME_TYPES: Record<string, string> = {
".otf": "font/otf",
};
/**
* Result of parsing a `Range:` request header against a known total size.
*
* - `kind: "satisfiable"`: `start <= end < size`. The response should be 206
* with `Content-Range: bytes start-end/size` and the sliced body.
* - `kind: "unsatisfiable"`: the header was syntactically valid (`bytes=...`)
* but the resolved range falls outside `[0, size)` (e.g. `start >= size`,
* `end < start`, or a suffix request on a zero-byte file). Per RFC 7233
* the response should be 416 with `Content-Range: bytes (asterisk)/size`.
* - `kind: "absent"`: there is no `Range:` header on the request, or it is
* syntactically malformed, uses a non-`bytes` unit, or requests multiple
* ranges. RFC 7233 allows ignoring such headers and serving the full body
* with a 200, which is what callers should do.
*/
export type RangeRequest =
| { kind: "satisfiable"; start: number; end: number }
| { kind: "unsatisfiable" }
| { kind: "absent" };
/**
* Parse a single-range `Range:` request header per RFC 7233 §2.1.
*
* Supports the three forms of `bytes=...`:
* - `bytes=START-END`: closed range, both bounds inclusive.
* - `bytes=START-`: open-ended, serve from START to EOF.
* - `bytes=-SUFFIX`: last SUFFIX bytes.
*
* Multi-range requests (`bytes=0-99,200-299`) are treated as `absent`. The
* caller serves the full body with 200. The hyperframes producer's use case
* (Chrome `<video>` seeks, range-aware media stack) only ever issues single
* ranges, so we don't take on the multipart-byteranges complexity here.
*
* Exported for unit tests; not part of the public package surface.
*/
export function parseRangeHeader(header: string | null | undefined, size: number): RangeRequest {
if (!header) return { kind: "absent" };
const match = /^\s*bytes\s*=\s*(.*?)\s*$/i.exec(header);
if (!match) return { kind: "absent" };
const specList = match[1];
if (!specList || specList.includes(",")) {
// Multi-range: bail to full-body 200 rather than reassemble
// multipart/byteranges. Single-range is the only shape we serve.
return { kind: "absent" };
}
const dashIdx = specList.indexOf("-");
if (dashIdx < 0) return { kind: "absent" };
const rawStart = specList.slice(0, dashIdx).trim();
const rawEnd = specList.slice(dashIdx + 1).trim();
// Suffix form: `bytes=-N` returns the last N bytes.
if (rawStart === "" && rawEnd !== "") {
if (!/^\d+$/.test(rawEnd)) return { kind: "absent" };
const suffixLen = Number(rawEnd);
if (!Number.isFinite(suffixLen)) return { kind: "absent" };
if (size === 0 || suffixLen === 0) return { kind: "unsatisfiable" };
const start = Math.max(0, size - suffixLen);
return { kind: "satisfiable", start, end: size - 1 };
}
if (!/^\d+$/.test(rawStart)) return { kind: "absent" };
const start = Number(rawStart);
if (!Number.isFinite(start)) return { kind: "absent" };
// Open-ended form: `bytes=START-` returns from START to EOF.
if (rawEnd === "") {
if (start >= size) return { kind: "unsatisfiable" };
return { kind: "satisfiable", start, end: size - 1 };
}
// Closed form: `bytes=START-END`
if (!/^\d+$/.test(rawEnd)) return { kind: "absent" };
const requestedEnd = Number(rawEnd);
if (!Number.isFinite(requestedEnd)) return { kind: "absent" };
if (requestedEnd < start) return { kind: "unsatisfiable" };
if (start >= size) return { kind: "unsatisfiable" };
// Clamp the end to the last valid byte.
const end = Math.min(requestedEnd, size - 1);
return { kind: "satisfiable", start, end };
}
/**
* Options for {@link buildVirtualTimeShim}.
*/
@@ -609,7 +691,7 @@ export function createFileServer(options: FileServerOptions): Promise<FileServer
const app = new Hono();
app.get("/*", (c) => {
app.get("/*", async (c) => {
let requestPath = c.req.path;
if (requestPath === "/") requestPath = "/index.html";
@@ -665,7 +747,12 @@ export function createFileServer(options: FileServerOptions): Promise<FileServer
const contentType = MIME_TYPES[ext] || "application/octet-stream";
if (ext === ".html") {
const rawHtml = readFileSync(filePath, "utf-8");
// Use the async read here so we don't block the Node event loop while
// reading an HTML file (typically small, but a 200KB+ AI-generated
// composition during a concurrent render still costs a ms of stall).
// The injection step is sync — it's pure string ops on the buffered
// HTML — but the read itself is the only step that touches the disk.
const rawHtml = await readFile(filePath, "utf-8");
const isIndex = relativePath === "index.html";
let html = rawHtml;
if (preHeadScripts.length > 0) {
@@ -677,10 +764,67 @@ export function createFileServer(options: FileServerOptions): Promise<FileServer
return c.text(html, 200, { "Content-Type": contentType });
}
const content = readFileSync(filePath);
return new Response(content, {
// Stream binary file content rather than buffering it with readFileSync.
// On video-heavy compositions Chrome requests several 32MB video files
// back-to-back through this server; each readFileSync(32MB) blocked the
// Node event loop long enough to wedge concurrent /health responses (see
// renderOrchestrator.ts:1277-1306 documenting the same regression class).
// createReadStream() pipes bounded chunks asynchronously, so the event
// loop stays responsive even when several large assets are in flight
// simultaneously. Chrome reassembles the chunks transparently.
//
// We also honor `Range:` requests (RFC 7233) so Chrome's <video> element
// can seek into and partial-load large media without re-pulling the whole
// file. `Accept-Ranges: bytes` is advertised on every response (including
// full-body 200s) so the client knows ranges are supported.
const stat = statSync(filePath);
const totalSize = stat.size;
const rangeHeader = c.req.header("range");
const rangeRequest = parseRangeHeader(rangeHeader, totalSize);
if (rangeRequest.kind === "unsatisfiable") {
// 416 Range Not Satisfiable. RFC 7233 §4.4 mandates `Content-Range`
// carry the total length as `bytes */<size>` so clients know how to
// re-issue a valid range.
return new Response(null, {
status: 416,
headers: {
"Content-Type": contentType,
"Content-Range": `bytes */${totalSize}`,
"Accept-Ranges": "bytes",
},
});
}
if (rangeRequest.kind === "satisfiable") {
const { start, end } = rangeRequest;
const length = end - start + 1;
const stream = createReadStream(filePath, { start, end });
const webStream = Readable.toWeb(stream) as unknown as ReadableStream;
return new Response(webStream, {
status: 206,
headers: {
"Content-Type": contentType,
"Content-Length": String(length),
"Content-Range": `bytes ${start}-${end}/${totalSize}`,
"Accept-Ranges": "bytes",
},
});
}
// No Range header (or malformed/multi-range): full-body 200 with
// Accept-Ranges advertised so the client knows future Range requests
// are supported. Node Readable -> Web ReadableStream so Hono's
// Response can consume it. Node 18+ supports Readable.toWeb directly.
const stream = createReadStream(filePath);
const webStream = Readable.toWeb(stream) as unknown as ReadableStream;
return new Response(webStream, {
status: 200,
headers: { "Content-Type": contentType },
headers: {
"Content-Type": contentType,
"Content-Length": String(totalSize),
"Accept-Ranges": "bytes",
},
});
});